WIP: Add command to clean unreferencable things from binary caches

[autoantwort]

Motivation: From time to time the binary cache grows. For example mine is 70 GB large, but only a small subset of the cache entries are needed in the future. The rest can be deleted. Currently I only can delete the whole cache but then have to rebuild everything.

Intended and Implemented Features:

• List what is in the cache and show the difference if a port with the same version have multiple cache entries.
• Remove a port and everything that depends on this port
• Remove cache entries by an abi entry. For example powershell 7.2.1

Nice to have features:

• Show the root causes for duplicated cache entries (port, triplet, version are the same)
• Remove all old ports depending on the script hashes

[ras0219-msft]

Before investing too much into this PR; could you update the original post with a description of the intended behavior of the command and one or two example scenarios of how a user would use it in context?

[autoantwort]

I have updated the description. I normally used it as follows:

$ vcpkg binarycache list
$ vcpkg binarycache remove-with-key ports.cmake  30a2833518026e7bb744dfae1f9648a87a6564083ed1c5f57d50440edf5847d0
$ vcpkg binarycache remove-with-key post_build_checks 3 
$ vcpkg binarycache remove-recursive 30790a8b318ee41e5b1968e5c82668363f30a2d481a8911e6476abed4a2a4cb5  
$ vcpkg binarycache list

Example output of list:

lagerx64-osx -> 2021-06-17#1
  Version: 2a476956c4d7d05037f1d315fb1843f64e424c2974f94a472e9eac29152e3d28
    boost-hana: boost-hanax64-osx -> 1.78.0#0 3c1e1d70b3b9f6bb39100dd105d0cca00b98e811b3ab03193e8e624229c0b142
    post_build_checks: 2
    vcpkg-cmake: vcpkg-cmakex64-osx -> 2021-12-20#0 d758703cd6f959102e050c6532c856fb5003063fc36e85cf580997069e784c52
    vcpkg-cmake-config: vcpkg-cmake-configx64-osx -> 2021-12-28#0 082af33d3ab241480fb7531f3758587c04cf15c36a396207ab5855e17e9547ba
    zug: zugx64-osx -> 2021-04-23#0 1ebd70e9ab5ef2f03547354444a1cbccf515a6e62285ec354f4aa94b01c3e400

  Version: 34e1657253953b58882b3ca19fc3ba4688429b894305e457a1d5e658ff5cb727
    boost-hana: boost-hanax64-osx -> 1.78.0#0 7bdb03b705a3b655c91f9c073fb90935a7130724ede3e489e44fdd6b1f58d085
    post_build_checks: 3
    vcpkg-cmake: vcpkg-cmakex64-osx -> 2021-12-20#0 b46ccda4ba964fc2e0b3cd6c7cee0e84358fdb53e4f70793ed9333b7c3893a4e
    vcpkg-cmake-config: vcpkg-cmake-configx64-osx -> 2021-12-28#0 9fb5af6e30f4561dc54cb5e27d71eeb57098c844730c9249dc11448d07dbbdc2
    zug: zugx64-osx -> 2021-04-23#0 b2eeecd67e2b1d61ec0f1231d282301513338e92cc7e08a775dea9de9341772a

[reinago]

It might be worth it making vcpkg also remember when a package was last restored from the binary cache.

That way, when in a pinch, you could just remove the package from classic and re-install / delete install dir from projects that use manifest mode and which you use most often and configure. Then instruct vcpkg to delete everything that has not been restored for x days. That would especially help if you have some packages that are extremely costly to build (cgal, embree, ...) and you do not have the time to wait for that.

I prototyped this in ArchivesBinaryProvider::try_restore_n and it seems to do the job. The implementation is quite blunt though (it just places a marker file next to the archive).

[autoantwort]

@reinago I also came to the conclusion that this is a more feasible way to manage the binary cache, therefore I have created #945 that does exactly that (it relies on the "last access date" of your filesystem)