Skip to content

Add a randomized test for 'frozen' DEX offers.#5174

Open
dmkozh wants to merge 1 commit intostellar:masterfrom
dmkozh:cap77_test
Open

Add a randomized test for 'frozen' DEX offers.#5174
dmkozh wants to merge 1 commit intostellar:masterfrom
dmkozh:cap77_test

Conversation

@dmkozh
Copy link
Contributor

@dmkozh dmkozh commented Mar 12, 2026

Description

Removal of offers that would affect frozen entries from DEX has already been covered in a simple smoke test. This is a more robust randomized test that ensures that DEX behavior is the same both with and without 'frozen' offers.

Checklist

  • Reviewed the contributing document
  • Rebased on top of master (no merge commits)
  • Ran clang-format v8.0.0 (via make format or the Visual Studio extension)
  • Compiles
  • Ran all tests
  • If change impacts performance, include supporting evidence per the performance document

Removal of offers that would affect frozen entries from DEX has already been covered in a simple smoke test. This is a more robust randomized test that ensures that DEX behavior is the same both with and without 'frozen' offers.
Copilot AI review requested due to automatic review settings March 12, 2026 01:34
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new randomized acceptance test to validate that offers owned by accounts with frozen ledger keys do not perturb DEX matching outcomes (i.e., matching results and account balance deltas remain identical to a baseline run without frozen offers).

Changes:

  • Introduces a randomized market simulation that runs twice per iteration (baseline vs. with frozen offers) and compares resulting balance deltas for taker + active makers.
  • Adds aggregate assertions intended to ensure the test actually exercised both successful transactions and frozen-offer removals.

Comment on lines +2276 to +2280
std::cerr << fmt::format("frozen offers removed: {}/{}, "
"txs succeeded: {}",
withFrozen.frozenOffersRemoved,
TOTAL_FROZEN_MAKERS, baseline.txsSucceeded)
<< std::endl;
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test writes progress info to std::cerr, which will be emitted even when the test passes and makes CI output noisy. Prefer Catch2 diagnostics (INFO/CAPTURE) so the details only show up on failure, or remove the logging entirely.

Suggested change
std::cerr << fmt::format("frozen offers removed: {}/{}, "
"txs succeeded: {}",
withFrozen.frozenOffersRemoved,
TOTAL_FROZEN_MAKERS, baseline.txsSucceeded)
<< std::endl;
INFO(fmt::format("frozen offers removed: {}/{}, "
"txs succeeded: {}",
withFrozen.frozenOffersRemoved,
TOTAL_FROZEN_MAKERS, baseline.txsSucceeded));

Copilot uses AI. Check for mistakes.
Comment on lines +2289 to +2292
// We should have enough test iterations and transactions to get at least
// some frozen offer removals and successful transactions.
REQUIRE(totalFrozenOffersRemoved > 0);
REQUIRE(totalTxsSucceeded > 0);
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The assertions REQUIRE(totalFrozenOffersRemoved > 0) / REQUIRE(totalTxsSucceeded > 0) make the test outcome depend on chance (and the Catch RNG seed is rotated daily by the test runner). This can produce a seed/day where no frozen offers get touched and the test fails despite correct behavior. Consider making at least one operation per iteration deterministically cross a known-best frozen offer (or otherwise guarantee an expected removal / success) and drop the probabilistic aggregate REQUIREs.

Suggested change
// We should have enough test iterations and transactions to get at least
// some frozen offer removals and successful transactions.
REQUIRE(totalFrozenOffersRemoved > 0);
REQUIRE(totalTxsSucceeded > 0);
// Note: We intentionally do not assert that totalFrozenOffersRemoved or
// totalTxsSucceeded are > 0 here. The workload in this test is randomized
// using Catch's RNG, and the global test seed is rotated. Asserting that
// some particular random behavior occurred can make the test flaky even
// when the core property (baseline vs. withFrozen equivalence) holds.

Copilot uses AI. Check for mistakes.
INFO("iteration " << iter);

auto iterSeed = stellar::uniform_int_distribution<uint32_t>(
0, UINT32_MAX)(Catch::rng());
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this test fails, it may be hard to reproduce/debug because the per-iteration RNG seed isn't captured in the failure output. Consider adding CAPTURE(iterSeed) (or including the seed in an INFO message) so a failing iteration can be deterministically reproduced with the same --rng-seed.

Suggested change
0, UINT32_MAX)(Catch::rng());
0, UINT32_MAX)(Catch::rng());
CAPTURE(iterSeed);

Copilot uses AI. Check for mistakes.
Comment on lines +2006 to +2014
constexpr int NUM_ITERATIONS = 10;
constexpr int ACTIVE_MAKERS_PER_PAIR = 10;
constexpr int FROZEN_MAKERS_PER_PAIR = 5;
constexpr int NUM_PAIRS = 3;
constexpr int NUM_OPS = 15;
constexpr int NUM_ASSETS = 3;
constexpr int MAX_BATCH_SIZE = 10;
constexpr int TOTAL_ACTIVE_MAKERS = ACTIVE_MAKERS_PER_PAIR * NUM_PAIRS;
constexpr int TOTAL_FROZEN_MAKERS = FROZEN_MAKERS_PER_PAIR * NUM_PAIRS;
Copy link

Copilot AI Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test spins up a full Application twice per iteration (baseline + frozen) and creates a fairly large number of accounts/offers. With NUM_ITERATIONS=10 this may noticeably increase test runtime. If the intent is broad coverage, consider reducing iteration/account counts or restructuring to reuse a single app per iteration while still comparing frozen vs non-frozen behavior.

Copilot uses AI. Check for mistakes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants