[fix] Fix memory leak in TopicName cache and reduce heap memory usage through deduplication

[lhotari]

Fixes #24445

Motivation

There is currently a memory leak in Pulsar Proxy and Pulsar client caused by the TopicName cache. The TopicName cache grows without bounds and eventually consumes all available heap memory when there is high cardinality of topic names resolved during the runtime of a single JVM.

Before PR #23052, Guava's LoadingCache was used to limit the cache size to 100,000 entries. That was replaced with ConcurrentHashMap due to CPU overhead concerns with Guava compared to plain ConcurrentHashMap.

However, #23052 only addressed cache expiration for the broker component. Since the TopicName cache is included in the Pulsar client, it should be self-contained without requiring additional maintenance tasks to keep the cache size bounded.

Modifications

• Revert [improve] [broker] Improve CPU resources usege of TopicName Cache #23052 changes: Remove the broker-specific cache management that was added in the original PR
• Add generic NameCache implementation: Create an abstract NameCache class that uses ConcurrentHashMap with SoftReference values to allow memory pressure-based garbage collection of unreferenced cache entries
• Implement memory-efficient caching: Use soft references to ensure cache entries are automatically cleared when the JVM runs low on memory
• Add string deduplication: Use Guava's Interner class to deduplicate String instances of TopicName and NamespaceName components (tenant, namespace, localName, etc.) to reduce heap memory usage
• Performance optimization: Implement custom cache that is ~3x more performant than Caffeine cache with similar functionality
• Add benchmarking: Include JMH benchmark to validate performance (achieves ~28M ops/second on Mac M3 Max)
• Unified implementation: Both TopicNameCache and NamespaceNameCache extend the same generic NameCache implementation

Key Benefits

1. Memory leak prevention: Cache automatically shrinks under memory pressure through soft references
2. Heap memory reduction: String interning significantly reduces memory footprint for applications with many topic names
3. Self-contained solution: No external maintenance tasks required - the cache manages itself
4. Performance maintained: Despite added safety features, performance remains excellent
5. Consistent behavior: Same caching strategy applied to both TopicName and NamespaceName

Performance Characteristics

• End-to-end performance: ~28M TopicName.get operations per second (Mac M3 Max)
• Custom implementation is ~3x faster than equivalent Caffeine cache
• Memory usage significantly reduced through string deduplication
• Automatic cache shrinking when memory pressure is detected

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

[Copilot]

Pull Request Overview

This PR fixes a memory leak in the Pulsar Proxy and client by reverting the previous caching strategy and introducing a new NameCache implementation that uses soft references and string interning to reduce heap memory usage.

• Introduces NameCache, TopicNameCache, and NamespaceNameCache classes to provide efficient, self-clearing caches.
• Updates TopicName and NamespaceName to use interning when parsing topic names.
• Removes obsolete configuration properties and scheduled tasks related to cache clearing.

Reviewed Changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
pulsar-common/src/main/java/org/apache/pulsar/common/naming/TopicNameCache.java	Added a new cache for TopicName using the NameCache abstraction.
pulsar-common/src/main/java/org/apache/pulsar/common/naming/TopicName.java	Updated to use TopicNameCache and string interning.
pulsar-common/src/main/java/org/apache/pulsar/common/naming/NamespaceNameCache.java	Added a new cache for NamespaceName using the NameCache abstraction.
pulsar-common/src/main/java/org/apache/pulsar/common/naming/NamespaceName.java	Updated to use NamespaceNameCache and string interning.
pulsar-common/src/main/java/org/apache/pulsar/common/naming/NameCache.java	Introduced the new NameCache base class with soft references and periodic maintenance.
pulsar-broker/src/test/resources/configurations/pulsar_broker_test_standalone.conf	Removed obsolete cache configuration properties.
pulsar-broker/src/test/resources/configurations/pulsar_broker_test.conf	Removed obsolete cache configuration properties.
pulsar-broker/src/test/java/org/apache/pulsar/common/naming/ServiceConfigurationTest.java	Removed tests for now-obsolete cache configuration.
pulsar-broker/src/test/java/org/apache/pulsar/broker/service/StandaloneTest.java	Removed assertions related to the obsolete cache configuration.
pulsar-broker/src/test/java/org/apache/pulsar/broker/PulsarServiceTest.java	Removed cache configuration tests and related lookup tests.
pulsar-broker/src/main/java/org/apache/pulsar/broker/service/BrokerService.java	Removed the scheduled cache clearance task.
pulsar-broker-common/src/main/java/org/apache/pulsar/broker/ServiceConfiguration.java	Removed obsolete configuration fields concerning the cache.
microbench/src/main/java/org/apache/pulsar/common/naming/package-info.java	New file addition for package-level documentation.
microbench/src/main/java/org/apache/pulsar/common/naming/TopicNameBenchmark.java	Added a benchmark to validate the performance of TopicName lookup.
conf/broker.conf	Removed obsolete cache configuration properties.

[BewareMyPower]

You can check benchmark.diff.txt for how I set up the tests.

BTW, @codelipenghui mentioned "over engineering" here. However, IMO, the cache itself is "over engineering" to me. I checked many places that TopicName is used. Mostly it's just used as a temporary variable to do some conversion.

Both the existing solution and this solution have issues when a lot of different topic names are used. The existing solution relies on manual call of clearIfReachedMaxCapacity. Introducing a scheduled task also brings unnecessary complexity. The solution in this PR relies on GC to clean up entries from cache. If any issue happens, it could be harder to diagnose.

Actually, I think we should remove the cache, which brings many issues that should never happen. From my perspective, introducing the cache is just to get around the poor implementation of TopicName's constructor and it's a "GC PTSD" coding style of Pulsar.

[lhotari]

We should not dismiss the requested change so quickly before reaching the consensus. Such change that affects a widely used common class needs a more careful review.

@BewareMyPower There was no review and there's rationale that this PR makes sense and improves Pulsar without causing performance regressions. I agree that in this case, it might take longer to reach consensus.

[lhotari]

You can check benchmark.diff.txt for how I set up the tests.

@BewareMyPower How did you setup JMH to run your benchmarks that you shared? JMH results could be skewed for multiple reasons in this case as explained in my previous comment, #24457 (comment)

[lhotari]

I changed TopicName's constructor to public and add a separated test:

    @Warmup(iterations = 10, time = 500, timeUnit = TimeUnit.MILLISECONDS)
    @Measurement(iterations = 5, time = 500, timeUnit = TimeUnit.MILLISECONDS)
    @Benchmark
    public void testNoCache() {
        for (int i = 0; i < 1000000; i++) {
            new TopicName("test-" + i);
        }
    }

@BewareMyPower This is an invalid JMH test case. In JMH, you need to use the "black hole" to consume computed values.
For example:

    @Warmup(iterations = 10, time = 500, timeUnit = TimeUnit.MILLISECONDS)
    @Measurement(iterations = 5, time = 500, timeUnit = TimeUnit.MILLISECONDS)
    @Benchmark
    public void testNoCache(Blackhole blackhole) {
        for (int i = 0; i < 1000000; i++) {
            blackhole.consume(new TopicName("test-" + i));
        }
    }

[lhotari]

    @Warmup(iterations = 10, time = 500, timeUnit = TimeUnit.MILLISECONDS)
    @Measurement(iterations = 5, time = 500, timeUnit = TimeUnit.MILLISECONDS)

Another detail is the short benchmark duration. A better choice could be to run a single longer iteration:

    @Warmup(iterations = 1, time = 10, timeUnit = TimeUnit.SECONDS)
    @Measurement(iterations = 1, time = 10, timeUnit = TimeUnit.SECONDS)

[lhotari]

I improved the JMH benchmark and fixed some flaws in it.

MacOS results (will soon run on Linux):

Benchmark                                (testMode)   Mode  Cnt         Score          Error  Units
TopicNameBenchmark.topicLookup001             basic  thrpt    3  26378883.001 ±  3852072.154  ops/s
TopicNameBenchmark.topicLookup001   invalidateCache  thrpt    3  24498489.317 ±  6733474.934  ops/s
TopicNameBenchmark.topicLookup001  strongReferences  thrpt    3  27146611.212 ± 20090217.310  ops/s
TopicNameBenchmark.topicLookup010             basic  thrpt    3  22199659.357 ±  5861674.879  ops/s
TopicNameBenchmark.topicLookup010   invalidateCache  thrpt    3  21913068.107 ±  3784974.042  ops/s
TopicNameBenchmark.topicLookup010  strongReferences  thrpt    3  22537707.826 ±  4154289.677  ops/s
TopicNameBenchmark.topicLookup100             basic  thrpt    3  16356441.900 ± 12729573.003  ops/s
TopicNameBenchmark.topicLookup100   invalidateCache  thrpt    3  15799393.963 ±  5250569.159  ops/s
TopicNameBenchmark.topicLookup100  strongReferences  thrpt    3  16439089.083 ± 11551885.443  ops/s

[lhotari]

Linux results

Benchmark                                      (testMode)   Mode  Cnt         Score           Error  Units
TopicNameBenchmark.topicLookup001                   basic  thrpt    3   8296768.710 ±   3847917.674  ops/s
TopicNameBenchmark.topicLookup001:async             basic  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup001         invalidateCache  thrpt    3   7719755.753 ±   1550321.165  ops/s
TopicNameBenchmark.topicLookup001:async   invalidateCache  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup001        strongReferences  thrpt    3   8340214.037 ±   1532080.072  ops/s
TopicNameBenchmark.topicLookup001:async  strongReferences  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup010                   basic  thrpt    3  33723499.610 ±  77685435.770  ops/s
TopicNameBenchmark.topicLookup010:async             basic  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup010         invalidateCache  thrpt    3  29339606.271 ±  43286582.744  ops/s
TopicNameBenchmark.topicLookup010:async   invalidateCache  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup010        strongReferences  thrpt    3  37706919.154 ±  33147758.020  ops/s
TopicNameBenchmark.topicLookup010:async  strongReferences  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup100                   basic  thrpt    3  37475011.818 ± 159568954.829  ops/s
TopicNameBenchmark.topicLookup100:async             basic  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup100         invalidateCache  thrpt    3  33964067.415 ±  28649039.174  ops/s
TopicNameBenchmark.topicLookup100:async   invalidateCache  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup100        strongReferences  thrpt    3  41307757.659 ±  48036888.867  ops/s
TopicNameBenchmark.topicLookup100:async  strongReferences  thrpt                NaN                    ---

flamegraph

[BewareMyPower]

Well, based on my improvement in #24463, I migrated the cache implementation of this PR. The benchmark ran on a ubuntu 22.04 runner in GitHub Actions, here is the result from See https://github.com/BewareMyPower/pulsar/actions/runs/15872123416/job/44750991782?pr=45:

TopicNameBenchmark.testConstruct      thrpt       254.510          ops/s
TopicNameBenchmark.testReadFromCache  thrpt       294.822          ops/s

The result is similar to my benchmark on macOS:

TopicNameBenchmark.testConstruct      thrpt       345.498          ops/s
TopicNameBenchmark.testReadFromCache  thrpt       375.209          ops/s

Both shows getting the topic name from cache is not significantly better than my latest optimization.

Please leave comments directly in BewareMyPower#45 if you think there is something wrong with my benchmark code since you said

I wouldn't trust JMH results from benchmarking on Mac OS.

[lhotari]

Both shows getting the topic name from cache is not significantly better than my latest optimization.

Great work @BewareMyPower . The main aspect why I wouldn't remove the cache is due to deduplication functionality of the cache. If TopicName instances are strongly referenced, the more there are, the worse the problem is.

Please leave comments directly in BewareMyPower#45 if you think there is something wrong with my benchmark code since you said

I wouldn't trust JMH results from benchmarking on Mac OS.

The benchmark code looks fine now. On Mac OS, the results just could be different than on Linux x86_64 in many cases. Especially if there's any code that accesses System.nanoTime(). It's many times slower on Mac OS than on Linux.

[lhotari]

Latest benchmark results on Linux after removing System.nanoTime and making cache maintenance handled during the single threaded maintenance task:

Benchmark                                      (testMode)   Mode  Cnt         Score           Error  Units
TopicNameBenchmark.topicLookup001                   basic  thrpt    3   8772255.043 ±   2202646.758  ops/s
TopicNameBenchmark.topicLookup001:async             basic  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup001         invalidateCache  thrpt    3   7787641.717 ±   4826933.972  ops/s
TopicNameBenchmark.topicLookup001:async   invalidateCache  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup001        strongReferences  thrpt    3   8697105.056 ±   2378331.526  ops/s
TopicNameBenchmark.topicLookup001:async  strongReferences  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup010                   basic  thrpt    3  59980650.077 ±  18606653.977  ops/s
TopicNameBenchmark.topicLookup010:async             basic  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup010         invalidateCache  thrpt    3  54319848.644 ± 104656951.523  ops/s
TopicNameBenchmark.topicLookup010:async   invalidateCache  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup010        strongReferences  thrpt    3  61755152.726 ±  28077697.228  ops/s
TopicNameBenchmark.topicLookup010:async  strongReferences  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup100                   basic  thrpt    3  93982448.999 ±  36720417.649  ops/s
TopicNameBenchmark.topicLookup100:async             basic  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup100         invalidateCache  thrpt    3  81061925.069 ±  55132917.420  ops/s
TopicNameBenchmark.topicLookup100:async   invalidateCache  thrpt                NaN                    ---
TopicNameBenchmark.topicLookup100        strongReferences  thrpt    3  73160471.462 ±  47659030.231  ops/s
TopicNameBenchmark.topicLookup100:async  strongReferences  thrpt                NaN                    ---

flamegraph

recent detail:

It would require a longer run to get more accurate results, but it looks like the performance is now very close to ConcurrentHashMap.get. Obviously running the benchmark with the current master branch would be a way to get an accurate comparison. I'll handle that later.

[poorbarcode]

#24457 (comment)
#24457 (comment)
#24457 (comment)

[lhotari]

#24457 (comment)
#24457 (comment)
#24457 (comment)

@poorbarcode we can proceed with your PR and come back to the usefulness of a soft reference cache and instance deduplication in optimizing memory usage of topic pattern listing / watching use cases.

[ben-manes]

A few observations if helpful,

• It looks like Pulsar is using an old version of Caffeine (2.9.1; May 2021) in order to run on Java 8. Is that still necessary or can you use the current version (3.2.1; June 2025)?
• You were using expireAfterAccess which incurs an expensive system time call on every read, while in the replacement it has no time threshold and an optional space limit. Can you use a size bounding for a fairer comparison?
• The maintenance is O(n) on the caller thread, which is likely fine but is a caller-facing penalty. The scan is of the underlying array, which at the previous setting of 100K entries would be up to 175-250K slots to traverse. That can have surprisingly bad performance when not handled properly. e.g. the clustering effect breaks ehcache's sampled lru) assumptions.
  • Caffeine uses strictly amortized O(1) algorithms that are inexpensive. However by default it will defer to ForkJoinPool.commonPool() since we don't know the usage costs of foreign code (e.g. CHM blocks on a hashbin collision so a long compute could stall eviction). You can use executor(Runnable::run) in most cases.
• If soft references are over used then they have traditionally been costly by increasing GC pauses for a slower system, making it look good in benchmarks and very problematic in real workloads. They can cause old gc thrashing, so newer GCs will often evict aggressively so there is less content captured on a real system. The eviction will likely discard useful content due to the global LRU where a polluter of inexpensive data forces the eviction of valuable expensive content. This spooky action at a distance due to the eviction policy not being localized can be harder to debug and causes overall slower system performance as projects evolve and the tradeoffs are forgotten.

[codelipenghui]

Hi @ben-manes

Thanks for your help.
I have created a PR #24527 to upgrade Caffeine Cache first.

[lhotari]

Closing since this is covered in other PRs.