[NPU]:Replace 'cuda' in the project with abstract interfaces

[Feng0w0]

Replace the cuda field in the project with the abstract interface:
1.'cuda' -> get_device_type()
2.torch.cuda -> get_torch_device()

[gemini-code-assist]

Summary of Changes

Hello @Feng0w0, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly improves the project's hardware compatibility by abstracting device-specific code. It moves away from direct 'cuda' references to more general get_device_type() and get_torch_device() interfaces, making the codebase more flexible for different accelerators. A key part of this change is the implementation of a patch to ensure autocast functionality works correctly for NPU devices, thereby broadening the range of supported hardware without requiring extensive code modifications.

Highlights

• Device Abstraction: Replaced hardcoded 'cuda' device references with abstract get_device_type() calls across numerous models and pipelines, enhancing device-agnosticism.
• PyTorch CUDA Abstraction: Abstracted torch.cuda calls to get_torch_device() for retrieving the current device, further decoupling the codebase from CUDA-specific implementations.
• NPU Autocast Patching: Introduced a specific patching mechanism for torch.amp.autocast to correctly handle torch.float32 on NPU devices, addressing a known compatibility issue.
• Extended Compilation Support: Updated model compilation checks to explicitly include NPU devices, alongside CUDA, for improved hardware compatibility.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request effectively abstracts device-specific code by replacing hardcoded 'cuda' references to better support NPUs, primarily through helper functions like get_device_type() and get_torch_device(). The introduction of a monkey patch for torch.autocast is a clever solution for handling NPU-specific behavior. I've noted a couple of areas for improvement: there's some redundant code in diffsynth/models/longcat_video_dit.py where NPU-specific logic is manually added despite the monkey patch, and a potential bug in diffsynth/models/step1x_text_encoder.py with get_torch_device().current_device() that could fail on CPU-only systems. My suggestions aim to resolve these issues. Overall, this is a solid step towards making the codebase more device-agnostic.

[gemini-code-assist]

Using get_torch_device().current_device() will cause a crash on CPU-only systems. The get_torch_device function falls back to torch.cuda when the device is 'cpu', and torch.cuda.current_device() will then fail if no CUDA device is available. A simpler and more robust approach is to use get_device_type() directly, as it returns a device string ('cpu', 'cuda', 'npu') that is accepted by PyTorch tensor creation functions.

Suggested change

	device=get_torch_device().current_device(),
	device=get_device_type(),

[gemini-code-assist]

Using get_torch_device().current_device() will cause a crash on CPU-only systems. The get_torch_device function falls back to torch.cuda when the device is 'cpu', and torch.cuda.current_device() will then fail if no CUDA device is available. A simpler and more robust approach is to use get_device_type() directly, as it returns a device string ('cpu', 'cuda', 'npu') that is accepted by PyTorch tensor creation functions.

Suggested change

	device=get_torch_device().current_device(),
	device=get_device_type(),

[gemini-code-assist]

Using get_torch_device().current_device() will cause a crash on CPU-only systems. The get_torch_device function falls back to torch.cuda when the device is 'cpu', and torch.cuda.current_device() will then fail if no CUDA device is available. A simpler and more robust approach is to use get_device_type() directly, as it returns a device string ('cpu', 'cuda', 'npu') that is accepted by PyTorch tensor creation functions.

Suggested change

	device=get_torch_device().current_device(),
	device=get_device_type(),

[gemini-code-assist]

This conditional block is redundant. The monkey patch for torch.amp.autocast in diffsynth/core/npu_patch/npu_autocast_patch.py already handles this logic when get_device_type() returns 'npu'. Removing this will improve code clarity and avoid duplication. This comment applies to similar changes in this file (lines 590-591, 611-612, 626-627, and 810-811).