e172564eea
Update controlnet.py to fix the default controlnet weight as constant ( #4285 )
2024-08-09 13:40:05 -04:00
a3cc326748
Better fix for lowvram issue.
2024-08-09 12:16:25 -04:00
86a97e91fc
Fix controlnet regression.
2024-08-09 12:08:58 -04:00
5acdadc9f3
Fix issue with some lowvram weights.
2024-08-09 03:58:28 -04:00
55ad9d5f8c
Fix regression.
2024-08-09 03:36:40 -04:00
a9f04edc58
Implement text encoder part of HunyuanDiT loras.
2024-08-09 03:21:10 -04:00
a475ec2300
Cleanup HunyuanDit controlnets.
...
Use the: ControlNetApply SD3 and HunyuanDiT node.
2024-08-09 02:59:34 -04:00
06eb9fb426
feat: add support for HunYuanDit ControlNet ( #4245 )
...
* add support for HunYuanDit ControlNet
* fix hunyuandit controlnet
* fix typo in hunyuandit controlnet
* fix typo in hunyuandit controlnet
* fix code format style
* add control_weight support for HunyuanDit Controlnet
* use control_weights in HunyuanDit Controlnet
* fix typo
2024-08-09 02:59:24 -04:00
413322645e
Raw torch is faster than einops?
2024-08-08 22:09:29 -04:00
11200de970
Cleaner code.
2024-08-08 20:07:09 -04:00
037c38eb0f
Try to improve inference speed on some machines.
2024-08-08 17:29:27 -04:00
1e11d2d1f5
Better prints.
2024-08-08 17:29:27 -04:00
66d4233210
Fix.
2024-08-08 15:16:51 -04:00
591010b7ef
Support diffusers text attention flux loras.
2024-08-08 14:45:52 -04:00
08f92d55e9
Partial model shift support.
2024-08-08 14:45:06 -04:00
8115d8cce9
Add Flux fp16 support hack.
2024-08-07 15:08:39 -04:00
6969fc9ba4
Make supported_dtypes a priority list.
2024-08-07 15:00:06 -04:00
cb7c4b4be3
Workaround for lora OOM on lowvram mode.
2024-08-07 14:30:54 -04:00
1208863eca
Fix "Comfy" lora keys.
...
They are in this format now:
diffusion_model.full.model.key.name.lora_up.weight
2024-08-07 13:49:31 -04:00
e1c528196e
Fix bundled embed.
2024-08-07 13:30:45 -04:00
17030fd4c0
Support for "Comfy" lora format.
...
The keys are just: model.full.model.key.name.lora_up.weight
It is supported by all comfyui supported models.
Now people can just convert loras to this format instead of having to ask
for me to implement them.
2024-08-07 13:18:32 -04:00
c19dcd362f
Controlnet code refactor.
2024-08-07 12:59:28 -04:00
1c08bf35b4
Support format for embeddings bundled in loras.
2024-08-07 03:45:25 -04:00
b334605a66
Fix OOMs happening in some cases.
...
A cloned model patcher sometimes reported a model was loaded on a device
when it wasn't.
2024-08-06 13:36:04 -04:00
c14ac98fed
Unload models and load them back in lowvram mode no free vram.
2024-08-06 03:22:39 -04:00
2d75df45e6
Flux tweak memory usage.
2024-08-05 21:58:28 -04:00
8edbcf5209
Improve performance on some lowend GPUs.
2024-08-05 16:24:04 -04:00
a178e25912
Fix Flux FP64 math on XPU ( #4210 )
2024-08-05 01:26:20 -04:00
78e133d041
Support simple diffusers Flux loras.
2024-08-04 22:05:48 -04:00
7afa985fba
Correct spelling 'token_weight_pars_t5' to 'token_weight_pairs_t5' ( #4200 )
2024-08-04 17:10:02 -04:00
3b71f84b50
ONNX tracing fixes.
2024-08-04 15:45:43 -04:00
0a6b008117
Fix issue with some custom nodes.
2024-08-04 10:03:33 -04:00
f7a5107784
Fix crash.
2024-08-03 16:55:38 -04:00
91be9c2867
Tweak lowvram memory formula.
2024-08-03 16:44:50 -04:00
03c5018c98
Lower lowvram memory to 1/3 of free memory.
2024-08-03 15:14:07 -04:00
2ba5cc8b86
Fix some issues.
2024-08-03 15:06:40 -04:00
1e68002b87
Cap lowvram to half of free memory.
2024-08-03 14:50:20 -04:00
ba9095e5bd
Automatically use fp8 for diffusion model weights if:
...
Checkpoint contains weights in fp8.
There isn't enough memory to load the diffusion model in GPU vram.
2024-08-03 13:45:19 -04:00
f123328b82
Load T5 in fp8 if it's in fp8 in the Flux checkpoint.
2024-08-03 12:39:33 -04:00
63a7e8edba
More aggressive batch splitting.
2024-08-03 11:53:30 -04:00
ea03c9dcd2
Better per model memory usage estimations.
2024-08-02 18:09:24 -04:00
3a9ee995cf
Tweak regular SD memory formula.
2024-08-02 17:34:30 -04:00
47da42d928
Better Flux vram estimation.
2024-08-02 17:02:35 -04:00
ce9ac2fe05
Fix clip_g/clip_l mixup ( #4168 )
2024-08-01 21:40:56 -04:00
e638f2858a
Hack to make all resolutions work on Flux models.
2024-08-01 21:39:18 -04:00
d420bc792a
Tweak the memory usage formulas for Flux and SD.
2024-08-01 17:53:45 -04:00
d965474aaa
Make ComfyUI split batches a higher priority than weight offload.
2024-08-01 16:39:59 -04:00
1c61361fd2
Fast preview support for Flux.
2024-08-01 16:28:11 -04:00
a6decf1e62
Fix bfloat16 potentially not being enabled on mps.
2024-08-01 16:18:44 -04:00
48eb1399c0
Try to fix mac issue.
2024-08-01 13:41:27 -04:00
d7430a1651
Add a way to load the diffusion model in fp8 with UNETLoader node.
2024-08-01 13:30:51 -04:00
f2b80f95d2
Better Mac support on flux model.
2024-08-01 13:10:50 -04:00
1aa9cf3292
Make lowvram more aggressive on low memory machines.
2024-08-01 12:11:57 -04:00
eb96c3bd82
Fix .sft file loading (they are safetensors files).
2024-08-01 11:32:58 -04:00
5f98de7697
Load flux t5 in fp8 if weights are in fp8.
2024-08-01 11:05:56 -04:00
8d34211a7a
Fix old python versions no longer working.
2024-08-01 09:57:20 -04:00
1589b58d3e
Basic Flux Schnell and Flux Dev model implementation.
2024-08-01 09:49:29 -04:00
7ad574bffd
Mac supports bf16 just make sure you are using the latest pytorch.
2024-08-01 09:42:17 -04:00
e2382b6adb
Make lowvram less aggressive when there are large amounts of free memory.
2024-08-01 03:58:58 -04:00
c24f897352
Fix to get fp8 working on T5 base.
2024-07-31 02:00:19 -04:00
a5991a7aa6
Fix hunyuan dit text encoder weights always being in fp32.
2024-07-31 01:34:57 -04:00
2c038ccef0
Lower CLIP memory usage by a bit.
2024-07-31 01:32:35 -04:00
b85216a3c0
Lower T5 memory usage by a few hundred MB.
2024-07-31 00:52:34 -04:00
82cae45d44
Fix potential issue with non clip text embeddings.
2024-07-30 14:41:13 -04:00
25853d0be8
Use common function for casting weights to input.
2024-07-30 10:49:14 -04:00
79040635da
Remove unnecessary code.
2024-07-30 05:01:34 -04:00
66d35c07ce
Improve artifacts on hydit, auraflow and SD3 on specific resolutions.
...
This breaks seeds for resolutions that are not a multiple of 16 in pixel
resolution by using circular padding instead of reflection padding but
should lower the amount of artifacts when doing img2img at those
resolutions.
2024-07-29 20:48:50 -04:00
4ba7fa0244
Refactor: Move sd2_clip.py to text_encoders folder.
2024-07-28 01:19:20 -04:00
cf4418b806
Don't treat Bert model like CLIP.
...
Bert can accept up to 512 tokens so any prompt with more than 77 should
just be passed to it as is instead of splitting it up like CLIP.
2024-07-26 13:08:12 -04:00
8328a2d8cd
Let hunyuan dit work with all prompt lengths.
2024-07-26 12:11:32 -04:00
afe732bef9
Hunyuan dit can now accept longer prompts.
2024-07-26 11:52:58 -04:00
a9ac56fc0d
Own BertModel implementation that works with lowvram.
2024-07-26 04:47:17 -04:00
25b51b1a8b
Hunyuan DiT lora support.
2024-07-25 22:42:54 -04:00
a5f4292f9f
Basic hunyuan dit implementation. ( #4102 )
...
* Let tokenizers return weights to be stored in the saved checkpoint.
* Basic hunyuan dit implementation.
* Fix some resolutions not working.
* Support hydit checkpoint save.
* Init with right dtype.
* Switch to optimized attention in pooler.
* Fix black images on hunyuan dit.
2024-07-25 18:21:08 -04:00
f87810cd3e
Let tokenizers return weights to be stored in the saved checkpoint.
2024-07-25 10:52:09 -04:00
10c919f4c7
Make it possible to load tokenizer data from checkpoints.
2024-07-24 16:43:53 -04:00
10b43ceea5
Remove duplicate code.
2024-07-24 01:12:59 -04:00
0a4c49c57c
Support MT5.
2024-07-23 15:35:28 -04:00
88ed893034
Allow SPieceTokenizer to load model from a byte string.
2024-07-23 14:17:42 -04:00
334ba48cea
More generic unet prefix detection code.
2024-07-23 14:13:32 -04:00
14764aa2e2
Rename LLAMATokenizer to SPieceTokenizer.
2024-07-22 12:21:45 -04:00
b2c995f623
"auto" type is only relevant to the SetUnionControlNetType node.
2024-07-22 11:30:38 -04:00
4151fbfa8a
Add error message on union controlnet ( #4081 )
2024-07-22 11:27:32 -04:00
95fa9545f1
Only append zero to noise schedule if last sigma isn't zero.
2024-07-20 12:37:30 -04:00
6ab8cad22e
Implement beta sampling scheduler.
...
It is based on: https://arxiv.org/abs/2407.12173
Add "beta" to the list of schedulers and the BetaSamplingScheduler node.
2024-07-19 18:05:09 -04:00
855789403b
support clip-vit-large-patch14-336 ( #4042 )
...
* support clip-vit-large-patch14-336
* support clip-vit-large-patch14-336
2024-07-17 13:12:50 -04:00
6f7869f365
Get clip vision image size from config.
2024-07-17 13:05:38 -04:00
281ad42df4
Fix lowvram union controlnet bug.
2024-07-17 10:16:31 -04:00
c5a48b15bd
Make default hash lib configurable without code changes via CLI argument ( #3947 )
...
* cli_args: Add --duplicate-check-hash-function.
* server.py: compare_image_hash configurable hash function
Uses an argument added in cli_args to specify the type of hashing to default to for duplicate hash checking. Uses an `eval()` to identify the specific hashlib class to utilize, but ultimately safely operates because we have specific options and only those options/choices in the arg parser. So we don't have any unsafe input there.
* Add hasher() to node_helpers
* hashlib selection moved to node_helpers
* default-hashing-function instead of dupe checking hasher
This makes a default-hashing-function option instead of previous selected option.
* Use args.default_hashing_function
* Use safer handling for node_helpers.hasher()
Uses a safer handling method than `eval` to evaluate default hashing function.
* Stray parentheses are evil.
* Indentation fix.
Somehow when I hit save I didn't notice I missed a space to make indentation work proper. Oops!
2024-07-16 18:27:09 -04:00
8270c62530
Add SetUnionControlNetType to set the type of the union controlnet model.
2024-07-16 17:04:53 -04:00
821f93872e
Allow model sampling to set number of timesteps.
2024-07-16 15:18:40 -04:00
99458e8aca
Add `FrontendManager` to manage non-default front-end impl ( #3897 )
...
* Add frontend manager
* Add tests
* nit
* Add unit test to github CI
* Fix path
* nit
* ignore
* Add logging
* Install test deps
* Remove 'stable' keyword support
* Update test
* Add web-root arg
* Rename web-root to front-end-root
* Add test on non-exist version number
* Use repo owner/name to replace hard coded provider list
* Inline cmd args
* nit
* Fix unit test
2024-07-16 11:26:11 -04:00
1305fb294c
Refactor: Move some code to the comfy/text_encoders folder.
2024-07-15 17:36:24 -04:00
7914c47d5a
Quick fix for the promax controlnet.
2024-07-14 10:07:36 -04:00
a3dffc447a
Support AuraFlow Lora and loading model weights in diffusers format.
...
You can load model weights in diffusers format using the UNETLoader node.
2024-07-13 13:51:40 -04:00
29c2e26724
Better tokenizing code for AuraFlow.
2024-07-12 01:15:25 -04:00
8e012043a9
Add a ModelSamplingAuraFlow node to change the shift value.
...
Set the default AuraFlow shift value to 1.73 (sqrt(3)).
2024-07-11 17:57:36 -04:00
9f291d75b3
AuraFlow model implementation.
2024-07-11 16:52:26 -04:00
f45157e3ac
Fix error message never being shown.
2024-07-11 11:46:51 -04:00
5e1fced639
Cleaner support for loading different diffusion model types.
2024-07-11 11:37:31 -04:00
TTPlanetPig
comfyanonymous
来新璐
a-One-Fan
Silver
Alexander Brown
comfyanonymous
Chenlei Hu
喵哩个咪
Thomas Ward