Skip to content

Cache typed config with git config list --type=<X>#2268

Open
derrickstolee wants to merge 4 commits intogit-ecosystem:mainfrom
derrickstolee:config-list
Open

Cache typed config with git config list --type=<X>#2268
derrickstolee wants to merge 4 commits intogit-ecosystem:mainfrom
derrickstolee:config-list

Conversation

@derrickstolee
Copy link
Contributor

@derrickstolee derrickstolee commented Feb 13, 2026

In certain repos and on the Windows platform, git-credential-manager can take 8-15 seconds before looking up a credential or having any user-facing interaction. This is due to dozens of git config --get processes that take 150-250 milliseconds each. The config keys that cause this pain are http.<url>.sslCAinfo and http.<url>.cookieFile. When credential.useHttpPath is enabled, each key is checked as <url> is incrementally truncated by directory segment.

It would be best if we could call a single Git process to send multiple config queries instead of running multiple processes. gitgitgadget/git#2033 suggested this direction of a single process solution, but it's very complicated! During review of that effort, it was recommended to use git config list instead.

But then there's a different problem! In all released versions of Git, git config list silently ignores the --type argument. We need the --type argument to guarantee that the resulting output string matches the bool or path formats.

The core Git change in gitgitgadget/git#2044 is now merged to next and thus is queued for Git 2.54.0. (We should wait until it merges to master before merging this change, just in case.) We can only check compatibility using a version check since the original command silently misbehaves.

This pull request allows for caching the list of all config values that match the given types: bool, path, or none. These are loaded lazily so if a command doesn't need one of the types then the command doesn't run. We are also careful to clear this cache if GCM mutates the config.

Since we ask for Git config values using --type=bool, --type=path, and --no-type, we may launch three git config list commands to satisfy these needs.

There is a possibility that this feature is fast-tracked into microsoft/git, in which case the version check would need augmentation. I have that available in derrickstolee#1 as an example.

Disclaimer: I used Claude Code and GitHub Copilot CLI to assist with this change. I carefully reviewed the changes and made adjustments based on my own taste and testing.

I did an end-to-end performance test on a local monorepo and got these results for improvements to no-op git fetch calls:

Command Mean [s] Min [s] Max [s] Relative
Without Cache 14.986 ± 0.255 14.558 15.192 3.29 ± 0.17
With Cache 4.561 ± 0.223 4.390 4.935 1.00

derrickstolee and others added 2 commits February 5, 2026 09:37
Context:
Git configuration queries currently spawn a new git process for each
TryGet(), GetAll(), or Enumerate() call. In scenarios where multiple
config values are needed (e.g., credential helper initialization), this
results in dozens of git process spawns, each parsing the same config
files repeatedly. This impacts performance, especially on Windows where
process creation is more expensive.

Justification:
Rather than caching individual queries, we cache the entire config
output from a single 'git config list --show-origin -z' call. This
approach provides several benefits:
- Single process spawn loads all config values at once
- Origin information allows accurate level filtering (system/global/local)
- Cache invalidation on write operations keeps data consistent
- Thread-safe implementation supports concurrent access

We only cache Raw type queries since Bool and Path types require Git's
canonicalization logic. Cache is loaded lazily on first access and
invalidated on any write operation (Set, Add, Unset, etc.).

Implementation:
Added ConfigCacheEntry class to store origin, value, and level for each
config entry. The ConfigCache class parses the NUL-delimited output from
'git config list --show-origin -z' (format: origin\0key\nvalue\0) and
stores entries in a case-insensitive dictionary keyed by config name.

Level detection examines the file path in the origin to determine
System/Global/Local classification. Fallback to individual git config
commands occurs if cache load fails or for typed (Bool/Path) queries.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
Context:
The caching implementation needed comprehensive tests to verify correct
behavior across different scenarios: cache hits, cache invalidation,
level filtering, and multivar handling. Tests revealed a critical bug
where the cache returned the wrong value when multiple config levels
(system/global/local) defined the same key.

Justification:
Tests follow existing patterns in GitConfigurationTests.cs, creating
temporary repositories and verifying cache behavior through the public
IGitConfiguration interface rather than testing internal cache classes
directly. This ensures we test the actual behavior users will experience.

The precedence bug occurred because ConfigCache.TryGet() returned the
first matching entry when it should return the last one. Git outputs
config values in precedence order (system, global, local), with later
values overriding earlier ones. Returning the last match correctly
implements Git's precedence rules.

Implementation:
Added 8 new test methods covering:
- Cache loading and retrieval (TryGet, GetAll, Enumerate)
- Cache invalidation on write operations (Set, Add, Unset)
- Level filtering to isolate local/global/system values
- Typed queries that bypass cache for Git's canonicalization

Fixed ConfigCache.TryGet() to iterate through all matching entries and
return the last one instead of the first, ensuring local config wins
over global, which wins over system.

All 805 tests pass with the fix applied.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@derrickstolee derrickstolee changed the title [WIP] Cache typed config with git config list --type=<X> Cache typed config with git config list --type=<X> Mar 1, 2026
@derrickstolee derrickstolee marked this pull request as ready for review March 2, 2026 16:45
@derrickstolee derrickstolee requested a review from a team as a code owner March 2, 2026 16:45
@derrickstolee
Copy link
Contributor Author

The Git change is now merged into master so it's locked-in for Git 2.54.0.

@risndobn-dotcom

This comment was marked as spam.

dscho added a commit to microsoft/git that referenced this pull request Mar 6, 2026
See gitgitgadget#2044 for the original review of this change.

The main feature here in Git is that `git config list --type=<X>` works
correctly.

But the real benefit is that we can update GCM to use `git config list
--type=<X>` to get the full list of config and store those values in the
cache without dozens of `git config --get` calls.

See git-ecosystem/git-credential-manager#2268 for the change that adds
this cache with a version check for Git 2.54.0. If we additionally add
derrickstolee/Git-Credential-Manager-Core#1, then a v2.53.0.vfs.0.1
release would also allow this behavior change.

I tested all of these features together against my copy of the Office
monorepo running a basic `git fetch`. Note that these runs are not
actually downloading much data, if any.

| Command | Mean [s] | Min [s] | Max [s] | Relative |
|:---|---:|---:|---:|---:|
| Without Cache | 14.986 ± 0.255 | 14.558 | 15.192 | 3.29 ± 0.17 |
| With Cache | 4.561 ± 0.223 | 4.390 | 4.935 | 1.00 |

During this test, I used a single GCM version with these changes and
modified the environment variables to change the Git version executed by
GCM.

* [X] This is an early version of work already under review upstream.
(These exact commits are merged to `next`.)
Copy link
Contributor

@mjcheetham mjcheetham left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a rather involved change (caches!) and is Git version dependant. I'll need a few iterations to fully grok the implications.

At this point, I'm also strongly considering the effort compared to writing our own Git config file parser (including support for all the include and includeIf logic) rather than continuing to patch more layers on the git config handling.

One thing I've spotted so far is a blocker on the use of --show-scope that would also need to be version-gated sadly 😢

return;

using (ChildProcess git = _git.CreateProcess("config list --show-origin -z"))
using (ChildProcess git = _git.CreateProcess("config list --show-scope --show-origin -z"))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately --show-scope was only introduced in 2020 with Git 2.26.0 (commit 145d59f4823) so we cannot always use this option else Git exits with an error.

I originally used --show-scope but had to drop it to support older Git versions: #318

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I force-pushed to drop the commit that added --show-scope and the only impact was to remove --show-scope from the new code path. It reuses the existing parsing of --show-origin.

Range diff:

1:  cfd3298c = 1:  cfd3298c refactor: Add config caching to reduce git process calls
2:  625d2541 = 2:  625d2541 test: Add cache tests and fix precedence bug
3:  eb0d445a < -:  -------- refactor: Use git config --show-scope for reliable level detection
4:  c7687ad7 ! 3:  4f37e544 feat: Cache config entries by type for typed queries
    @@ src/shared/Core/GitConfiguration.cs: namespace GitCredentialManager
                      return;
     +            }

    --            using (ChildProcess git = _git.CreateProcess("config list --show-scope --show-origin -z"))
    -+            using (ChildProcess git = _git.CreateProcess($"config list --show-scope --show-origin -z {typeArg}"))
    +-            using (ChildProcess git = _git.CreateProcess("config list --show-origin -z"))
    ++            using (ChildProcess git = _git.CreateProcess($"config list --show-origin -z {typeArg}"))
                  {
                      git.Start(Trace2ProcessClass.Git);
                      // To avoid deadlocks, always read the output stream first and then wait
    @@ src/shared/Core/GitConfiguration.cs: namespace GitCredentialManager
     +                        cache.Load(data, _trace);
                              break;
                          default:
    -                         _trace.WriteLine($"Failed to load config cache (exit={git.ExitCode}), will use individual git config commands");
    +                         _trace.WriteLine($"Failed to load config cache (exit={git.ExitCode})");
     @@ src/shared/Core/GitConfiguration.cs: namespace GitCredentialManager
              {
                  if (_useCache)
5:  0fea6faf = 4:  3125246e feat: Gate config cache on Git 2.54.0+ version check

derrickstolee and others added 2 commits March 10, 2026 09:53
Context:
The config cache only stored raw (untyped) values, so Bool and Path
queries always fell back to spawning individual git processes. Since
Git's --type flag canonicalizes values (e.g., expanding ~/... for
paths, normalizing yes/on/1 to true for bools), serving these from
the raw cache would return incorrect values.

Justification:
Instead of bypassing the cache for typed queries, we maintain a
separate cache per GitConfigurationType. Each cache is loaded with
the appropriate --type flag passed to 'git config list', so Git
performs canonicalization during the bulk load. This preserves the
correctness guarantee while extending the performance benefit to
all query types.

The cache result is now authoritative when loaded: if a key is not
found in the cache, we return 'not found' directly rather than
falling back to an individual git process call. This avoids a
redundant process spawn when the key genuinely doesn't exist.

Implementation:
Changed _cache from a single ConfigCache to a Dictionary keyed by
GitConfigurationType. EnsureCacheLoaded() now accepts a type
parameter and passes --no-type, --type=bool, or --type=path to the
git config list command. InvalidateCache() clears all type-specific
caches on any write operation.

Renamed TypedQuery_DoesNotUseCache test to
TypedQuery_CanonicalizesValues since typed queries now use their
own type-specific cache rather than bypassing the cache entirely.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Context:
The config cache uses 'git config list --type=<X>' to load
type-specific caches (raw, bool, path). The --type flag for the
'list' subcommand requires a fix queued for the Git 2.54.0
release. On Git 2.53 and earlier, the command succeeds but
silently ignores the --type parameter, returning raw values
instead of canonicalized ones. This means bool caches would
contain 'yes'/'on' instead of 'true', and path caches would
contain unexpanded '~/...' instead of absolute paths.

Justification:
Because the command exits 0 on older Git, the cache appears to
load successfully and the fallback paths never trigger. This makes
the bug silent and data-dependent: lookups work for values that
happen to already be in canonical form but return wrong results
for others. A version gate is the only reliable way to avoid this.

The check is in the constructor body rather than the constructor
chain so we can log a trace message when caching is disabled. The
explicit useCache parameter is preserved for tests that need to
control caching behavior independently of version.

Implementation:
Added ConfigListTypeMinVersion constant (2.54.0) and a version
comparison in the GitProcessConfiguration constructor. When
useCache is requested but git.Version is below the minimum, the
constructor overrides useCache to false and emits a trace line.
All existing fallback paths continue to work unchanged for users
on older Git, who will benefit from the cache automatically once
they upgrade.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants