feat: Add offline mode support to FallbackNetwork#93
Open
joshwhiton wants to merge 3 commits intohexgrad:mainfrom
Open
feat: Add offline mode support to FallbackNetwork#93joshwhiton wants to merge 3 commits intohexgrad:mainfrom
joshwhiton wants to merge 3 commits intohexgrad:mainfrom
Conversation
FallbackNetwork.from_pretrained() attempts network access even when TRANSFORMERS_OFFLINE=1 is set, because from_pretrained() does a HEAD request before falling back to cache. This PR: - Adds local_files_only parameter to FallbackNetwork and G2P - Respects TRANSFORMERS_OFFLINE and HF_HUB_OFFLINE env vars - Enables fully offline operation when models are pre-cached Use case: Air-gapped deployments, offline applications, reducing startup latency when network is slow/unavailable.
- PR hexgrad#90: Restrict spacy<4 to avoid pre-release/yanked versions Fixes Python 3.13 compatibility issues with thinc/blis dependencies - PR hexgrad#79: Strip whitespace from merged tokens Fixes lexicon lookup failures when multiple spaces appear between words
VBP (verb, present base form) should use present tense pronunciation, not past tense. For example, "I read books every day" should be pronounced "reed" not "red". Changes: - read VBP: "red" -> "reed" (US: ɹˈɛd -> ɹˈid, GB: ɹˈɛd -> ɹˈiːd) - reread VBP: match DEFAULT pronunciation - wound VBP: "wound" (injury) not "wound" (past of wind)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
local_files_onlyparameter toFallbackNetworkandG2PTRANSFORMERS_OFFLINEandHF_HUB_OFFLINEenv varsProblem
FallbackNetwork.from_pretrained()attempts network access even whenTRANSFORMERS_OFFLINE=1is set, becausefrom_pretrained()does a HEAD request to check for model updates before falling back to cache.Solution
Check the standard HuggingFace offline environment variables and pass
local_files_only=Truetofrom_pretrained()when they're set.Use case
Test
Notes
G2P(local_files_only=True)