When should you use test.retry versus test.failFast in Playwright's test runner, and what are the performance and reliability trade‑offs?

Question

When should you use test.retry versus test.failFast in Playwright’s test runner, and what are the performance and reliability trade‑offs?

Answer

Playwright’s built‑in test runner offers two complementary mechanisms for handling failing tests:

1. test.retry – automatically re‑executes a failing test up to N times.
2. test.failFast – aborts the entire suite (or the current worker) after the first failure.

When to prefer test.retry

Best practice: Set a modest global retry (e.g., retries: 1) and enable per‑test overrides only for known flaky tests. Keep the retry count low (< 2) to avoid runaway suite times.

When to prefer test.failFast

Configuration tip: Use failFast: true in the project config for high‑impact test suites, and keep it false for exploratory or low‑risk suites.

Combining both

Playwright evaluates retries before fail‑fast. A test will be retried up to its configured limit; only after exhausting retries does a failure trigger the fail‑fast behavior. Example:

// playwright.config.ts
export default defineConfig({
  retries: 1,               // global retry for all tests
  projects: [
    {
      name: 'critical',
      testMatch: '**/critical/**/*.spec.ts',
      // Critical path: abort early after any failure (post‑retry)
      failFast: true,
    },
    {
      name: 'stable',
      testMatch: '**/stable/**/*.spec.ts',
      // No fail‑fast; let the suite run to completion
      failFast: false,
    },
  ],
});

Performance impact

• Runtime increase ≈ average_test_time × retries.
  If the average test takes 200 ms and you set retries: 2, expect ~ 40 % longer runs for flaky suites.
• CI cost grows linearly with added retries; monitor the flaky‑rate metric (failed / (failed + passed)) to keep it below ~ 5 % before raising the retry count.
• Fail‑fast reduces wasted time: a single failure can cut the remaining runtime by up to 80 % in large suites, but only if failures are truly blocking.

Recommended workflow for senior engineers

1. Instrument CI to collect flaky‑rate and failure‑type metrics.
2. Set a low global retry (0 or 1) to catch obvious flakiness.
3. Mark persistent flaky tests with test.fixme or test.slow rather than relying on retries.
4. Apply failFast only to projects where a failure indicates a systemic break (e.g., authentication, core API contracts).
5. Periodically review retry usage; aim to eliminate flaky tests rather than hide them.

By balancing test.retry for transient noise and test.failFast for critical breakages, you keep CI fast, reliable, and maintainable.

📲 Practice Offline on Mobile: Download the free QA Automation & SDET Prep app on Google Play & App Store.