Add a randomized test for 'frozen' DEX offers.#5174
Add a randomized test for 'frozen' DEX offers.#5174dmkozh wants to merge 1 commit intostellar:masterfrom
Conversation
Removal of offers that would affect frozen entries from DEX has already been covered in a simple smoke test. This is a more robust randomized test that ensures that DEX behavior is the same both with and without 'frozen' offers.
There was a problem hiding this comment.
Pull request overview
Adds a new randomized acceptance test to validate that offers owned by accounts with frozen ledger keys do not perturb DEX matching outcomes (i.e., matching results and account balance deltas remain identical to a baseline run without frozen offers).
Changes:
- Introduces a randomized market simulation that runs twice per iteration (baseline vs. with frozen offers) and compares resulting balance deltas for taker + active makers.
- Adds aggregate assertions intended to ensure the test actually exercised both successful transactions and frozen-offer removals.
| std::cerr << fmt::format("frozen offers removed: {}/{}, " | ||
| "txs succeeded: {}", | ||
| withFrozen.frozenOffersRemoved, | ||
| TOTAL_FROZEN_MAKERS, baseline.txsSucceeded) | ||
| << std::endl; |
There was a problem hiding this comment.
This test writes progress info to std::cerr, which will be emitted even when the test passes and makes CI output noisy. Prefer Catch2 diagnostics (INFO/CAPTURE) so the details only show up on failure, or remove the logging entirely.
| std::cerr << fmt::format("frozen offers removed: {}/{}, " | |
| "txs succeeded: {}", | |
| withFrozen.frozenOffersRemoved, | |
| TOTAL_FROZEN_MAKERS, baseline.txsSucceeded) | |
| << std::endl; | |
| INFO(fmt::format("frozen offers removed: {}/{}, " | |
| "txs succeeded: {}", | |
| withFrozen.frozenOffersRemoved, | |
| TOTAL_FROZEN_MAKERS, baseline.txsSucceeded)); |
| // We should have enough test iterations and transactions to get at least | ||
| // some frozen offer removals and successful transactions. | ||
| REQUIRE(totalFrozenOffersRemoved > 0); | ||
| REQUIRE(totalTxsSucceeded > 0); |
There was a problem hiding this comment.
The assertions REQUIRE(totalFrozenOffersRemoved > 0) / REQUIRE(totalTxsSucceeded > 0) make the test outcome depend on chance (and the Catch RNG seed is rotated daily by the test runner). This can produce a seed/day where no frozen offers get touched and the test fails despite correct behavior. Consider making at least one operation per iteration deterministically cross a known-best frozen offer (or otherwise guarantee an expected removal / success) and drop the probabilistic aggregate REQUIREs.
| // We should have enough test iterations and transactions to get at least | |
| // some frozen offer removals and successful transactions. | |
| REQUIRE(totalFrozenOffersRemoved > 0); | |
| REQUIRE(totalTxsSucceeded > 0); | |
| // Note: We intentionally do not assert that totalFrozenOffersRemoved or | |
| // totalTxsSucceeded are > 0 here. The workload in this test is randomized | |
| // using Catch's RNG, and the global test seed is rotated. Asserting that | |
| // some particular random behavior occurred can make the test flaky even | |
| // when the core property (baseline vs. withFrozen equivalence) holds. |
| INFO("iteration " << iter); | ||
|
|
||
| auto iterSeed = stellar::uniform_int_distribution<uint32_t>( | ||
| 0, UINT32_MAX)(Catch::rng()); |
There was a problem hiding this comment.
If this test fails, it may be hard to reproduce/debug because the per-iteration RNG seed isn't captured in the failure output. Consider adding CAPTURE(iterSeed) (or including the seed in an INFO message) so a failing iteration can be deterministically reproduced with the same --rng-seed.
| 0, UINT32_MAX)(Catch::rng()); | |
| 0, UINT32_MAX)(Catch::rng()); | |
| CAPTURE(iterSeed); |
| constexpr int NUM_ITERATIONS = 10; | ||
| constexpr int ACTIVE_MAKERS_PER_PAIR = 10; | ||
| constexpr int FROZEN_MAKERS_PER_PAIR = 5; | ||
| constexpr int NUM_PAIRS = 3; | ||
| constexpr int NUM_OPS = 15; | ||
| constexpr int NUM_ASSETS = 3; | ||
| constexpr int MAX_BATCH_SIZE = 10; | ||
| constexpr int TOTAL_ACTIVE_MAKERS = ACTIVE_MAKERS_PER_PAIR * NUM_PAIRS; | ||
| constexpr int TOTAL_FROZEN_MAKERS = FROZEN_MAKERS_PER_PAIR * NUM_PAIRS; |
There was a problem hiding this comment.
This test spins up a full Application twice per iteration (baseline + frozen) and creates a fairly large number of accounts/offers. With NUM_ITERATIONS=10 this may noticeably increase test runtime. If the intent is broad coverage, consider reducing iteration/account counts or restructuring to reuse a single app per iteration while still comparing frozen vs non-frozen behavior.
Description
Removal of offers that would affect frozen entries from DEX has already been covered in a simple smoke test. This is a more robust randomized test that ensures that DEX behavior is the same both with and without 'frozen' offers.
Checklist
clang-formatv8.0.0 (viamake formator the Visual Studio extension)