Skip to content

Real-time progress output during long-running test execution #5

@suin

Description

@suin

Background

Kubernetes E2E tests can take a long time due to resource creation, waiting for Ready state, retries, etc. Currently, kest is designed as a "post-hoc diagnostic" tool — reports and events are only output after test completion.

Problem

During test execution, the only thing visible to the user is Bun's test name [pass/fail] Xms. There is no real-time feedback at all while a test is running.

What's invisible during execution:

Situation Current behavior
kubectl apply running stdout/stderr are piped — nothing is displayed
retryUntil waiting for a resource to become Ready (e.g. 60s) Process appears frozen
Errors during retry attempts Silently swallowed, moves to next retry
cleanup/revert phase Same silence

When a long-running test times out, there's no way to know where it was stuck in real-time. You only find out after completion by reading the report.

Expected behavior

Real-time visibility into what is happening during test execution. For example:

✓ Create Namespace kest-abc12
✓ Apply Deployment "my-app"
⟳ Assert Deployment "my-app" ... retrying (attempt 3/-, elapsed 4.2s)

Information that would be useful to see in real-time:

  • Currently executing action (apply, assert, delete, etc.)
  • Whether a retry is in progress, which attempt, and elapsed time
  • Success/failure of each action

Technical context

The current Recorder accumulates events purely in-memory with no streaming output mechanism. The following events are already being recorded, so a real-time output layer on top of these could make this possible:

  • ActionStart / ActionEnd
  • RetryStart / RetryAttempt / RetryEnd
  • RevertingsStart / RevertingsEnd

Considerations

  • Should this be enabled by default, or opt-in via an environment variable (e.g. KEST_VERBOSE=1) or option?
  • Output granularity: action level / command level / per-retry-attempt level
  • Output format depending on environment: plain text for CI, spinner/colors for interactive terminals
  • Coexistence with Bun's test runner output

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions