Handle mixed-dtype mismatches in autocast linear and conv wrappers#9006
Open
JPPhoto wants to merge 3 commits intoinvoke-ai:mainfrom
Open
Handle mixed-dtype mismatches in autocast linear and conv wrappers#9006JPPhoto wants to merge 3 commits intoinvoke-ai:mainfrom
JPPhoto wants to merge 3 commits intoinvoke-ai:mainfrom
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes mixed-dtype mismatch failures in autocast-wrapped
LinearandConv2dlayers during invocation.This change:
CustomLinearso it handles dtype mismatches in the plain path, the bias-only mismatch case, and the sidecar aggregated-parameter patch pathtorch.Tensorparams instead of degrading them tometatensors - adds focused regression tests for plain mixed-dtype inference and sidecar parameter patching, with CPU dtype parametrization for portabilityPractical exposure:
float16orbfloat16but stored weights or bias remainfloat32mat1 and mat2 must have the same dtype,self and mat2 must have the same dtype)In practice, this prevents invocation-time failures that would otherwise appear only under certain precision / patching combinations, making them easy to miss and hard to reproduce.
Related Issues / Discussions
Solves the issue of partial-load execution that I experienced.
QA Instructions
Run the focused mixed-dtype regression tests via
pytest tests/backend/model_manager/load/model_cache/torch_module_autocast/custom_modules/test_all_custom_modules.py -k 'mixed_dtype_inference_without_patches or mixed_dtype_sidecar_parameter_patch or bias_only_mismatch'Merge Plan
Checklist
What's Newcopy (if doing a release after this PR)