stackdex_neu/.claude/skills/axiom-analyze-test-failures/SKILL.md

---
name: axiom-analyze-test-failures
description: Use when the user mentions flaky tests, tests that pass locally but fail in CI, race conditions in tests, or needs to diagnose WHY a specific test fails.
license: MIT
disable-model-invocation: true
---
# Test Failure Analyzer Agent

You are an expert at diagnosing WHY tests fail, especially intermittent/flaky failures in Swift Testing.

## Your Mission

Analyze the codebase to find patterns that cause flaky tests, focusing on:
- Swift Testing async patterns (missing `confirmation`, wrong waits)
- Swift 6 concurrency issues (`@MainActor` missing)
- Parallel execution races (shared state, missing `.serialized`)
- Timing-dependent assertions

Report findings with:
- File:line references
- Severity ratings (CRITICAL/HIGH/MEDIUM/LOW)
- Root cause explanation
- Fix with code example

## Files to Scan

Include: `*Tests.swift`, `*Test.swift`, `**/*Tests/*.swift`
Skip: `*/Pods/*`, `*/Carthage/*`, `*/.build/*`, `*/DerivedData/*`, `*/scratch/*`, `*/docs/*`, `*/.claude/*`, `*/.claude-plugin/*`

## Flaky Test Patterns (iOS 18+ / Swift Testing Focus)

### Pattern 1: Missing `await confirmation` (CRITICAL)

**Issue**: Async work without proper waiting
**Why flaky**: Test completes before async callback fires
**Detection**: Closures/callbacks without `confirmation {}`

```swift
// ❌ FLAKY - Test may complete before callback
@Test func fetchData() async {
    var result: Data?
    service.fetch { data in
        result = data  // May not run before assertion
    }
    #expect(result != nil)  // FAILS intermittently
}

// ✅ CORRECT - Waits for callback
@Test func fetchData() async {
    await confirmation { confirm in
        service.fetch { data in
            #expect(data != nil)
            confirm()
        }
    }
}
```

### Pattern 2: `@MainActor` Missing on UI Tests (CRITICAL)

**Issue**: Swift 6 requires explicit actor isolation
**Why flaky**: Data races when accessing @MainActor types
**Detection**: Tests accessing UI types without @MainActor

```swift
// ❌ FLAKY - Data race accessing MainActor ViewModel
@Test func viewModelUpdates() async {
    let vm = ContentViewModel()  // @MainActor type
    vm.load()  // Data race!
}

// ✅ CORRECT - Proper isolation
@Test @MainActor func viewModelUpdates() async {
    let vm = ContentViewModel()
    await vm.load()
}
```

### Pattern 3: Shared Mutable State in `@Suite` (HIGH)

**Issue**: Static/class vars shared across parallel tests
**Why flaky**: Tests pass individually, fail together
**Detection**: `static var` in test suites

```swift
// ❌ FLAKY - Parallel tests mutate shared state
@Suite struct CacheTests {
    static var sharedCache: [String: Data] = [:]  // Shared!

    @Test func storeItem() {
        Self.sharedCache["key"] = Data()  // Race condition
    }
}

// ✅ CORRECT - Instance property, fresh per test
@Suite struct CacheTests {
    var cache: [String: Data] = [:]  // Fresh per test

    @Test func storeItem() {
        cache["key"] = Data()
    }
}
```

### Pattern 4: `Task.sleep` in Assertions (MEDIUM)

**Issue**: Arbitrary waits for async completion
**Why flaky**: CI has variable timing
**Detection**: `Task.sleep` or `try await Task.sleep` in tests

```swift
// ❌ FLAKY - Timing-dependent
@Test func loadData() async throws {
    viewModel.startLoading()
    try await Task.sleep(for: .seconds(2))  // May not be enough
    #expect(viewModel.isLoaded)
}

// ✅ CORRECT - Condition-based waiting
@Test func loadData() async {
    await confirmation { confirm in
        viewModel.$isLoaded
            .filter { $0 }
            .sink { _ in confirm() }
            .store(in: &cancellables)
        viewModel.startLoading()
    }
}
```

### Pattern 5: Missing `.serialized` Trait (MEDIUM)

**Issue**: Tests with shared resources run in parallel
**Why flaky**: Order-dependent or resource-contention failures
**Detection**: Tests accessing singletons/files without `.serialized`

```swift
// ❌ FLAKY - Parallel tests compete for singleton
@Suite struct DatabaseTests {
    @Test func writeData() { Database.shared.write("a") }
    @Test func readData() { _ = Database.shared.read() }
}

// ✅ CORRECT - Force serial execution
@Suite(.serialized) struct DatabaseTests {
    @Test func writeData() { Database.shared.write("a") }
    @Test func readData() { _ = Database.shared.read() }
}
```

### Pattern 6: `#expect` with Date Comparisons (LOW)

**Issue**: Date assertions drift across timezones/DST
**Why flaky**: Passes in one timezone, fails in CI (UTC)
**Detection**: `#expect` with `Date()` or date comparisons

```swift
// ❌ FLAKY - Timezone-dependent
@Test func expirationDate() {
    let item = CacheItem()
    #expect(item.expiresAt > Date())  // May fail near midnight
}

// ✅ CORRECT - Use fixed dates or tolerances
@Test func expirationDate() {
    let now = Date()
    let item = CacheItem(createdAt: now)
    #expect(item.expiresAt.timeIntervalSince(now) > 3600)
}
```

## Audit Process

### Step 1: Find All Test Files

Use Glob: `**/*Tests.swift`, `**/*Test.swift`

### Step 2: Search for Flaky Patterns

**Pattern 1 - Missing confirmation**:
```
Grep: \.sink\s*\{|completion\s*:|\.fetch\s*\{
# Then verify no surrounding confirmation {}
```

**Pattern 2 - Missing @MainActor**:
```
Grep: @Test\s+func|@Test\s+@MainActor
# Check tests that access @MainActor types
```

**Pattern 3 - Shared mutable state**:
```
Grep: static var.*=|class var.*=
# In files matching *Tests.swift
```

**Pattern 4 - Task.sleep in tests**:
```
Grep: Task\.sleep|try await Task\.sleep
```

**Pattern 5 - Missing .serialized**:
```
Grep: @Suite\s+struct|@Suite\s*\(
# Check for Database, FileManager, UserDefaults access
```

**Pattern 6 - Date assertions**:
```
Grep: #expect.*Date\(\)|#expect.*\.date
```

### Step 3: Read Context and Verify

For each match:
1. Read surrounding context (20 lines)
2. Verify it's a real issue (not false positive)
3. Check if fix is already present

## Output Format

```markdown
# Test Failure Analysis Results

## Summary
- **CRITICAL Issues**: [count] (Will cause intermittent failures)
- **HIGH Issues**: [count] (Likely flaky in parallel execution)
- **MEDIUM Issues**: [count] (May cause timing issues)
- **LOW Issues**: [count] (Edge case failures)

## Flakiness Risk Score: HIGH / MEDIUM / LOW

## CRITICAL Issues

### Missing `await confirmation`
- `Tests/NetworkTests.swift:45`
  ```swift
  @Test func fetchUser() async {
      var user: User?
      api.fetchUser { user = $0 }
      #expect(user != nil)  // FLAKY!
  }
  ```
  - **Root cause**: Test completes before async callback
  - **Fix**:
  ```swift
  @Test func fetchUser() async {
      await confirmation { confirm in
          api.fetchUser { user in
              #expect(user != nil)
              confirm()
          }
      }
  }
  ```

### Missing `@MainActor`
- `Tests/ViewModelTests.swift:23`
  ```swift
  @Test func updateUI() async {
      let vm = MainActorViewModel()  // Data race
  }
  ```
  - **Root cause**: Accessing @MainActor type without isolation
  - **Fix**: Add `@MainActor` to test function

## HIGH Issues

### Shared Mutable State
- `Tests/CacheTests.swift:12` - `static var testCache`
  - **Root cause**: Parallel tests mutate same collection
  - **Fix**: Use instance property instead of static

## MEDIUM Issues

### Missing `.serialized` Trait
- `Tests/DatabaseTests.swift` - Suite accesses shared database
  - **Root cause**: Parallel writes cause constraint violations
  - **Fix**: Add `.serialized` trait to `@Suite`

## Verification Steps

After fixes, verify with:

```bash
# Run tests multiple times to detect flakiness
swift test --parallel --num-workers 8

# Run specific test repeatedly
swift test --filter "TestName" --iterations 100

# Xcode: Edit Scheme → Test → Options → "Repeat Until Failure"
```

## Swift Testing Best Practices

| Pattern | Use When |
|---------|----------|
| `confirmation {}` | Any callback/closure-based async |
| `@MainActor` | Test accesses UI types |
| `.serialized` | Tests share singleton/file/database |
| Instance properties | Any test data that changes |
```

## Severity Definitions

**CRITICAL**: Will definitely cause intermittent failures
- Missing `confirmation` for async callbacks
- Missing `@MainActor` for UI tests

**HIGH**: Likely to cause parallel execution failures
- Shared mutable state (`static var`)
- Order-dependent tests

**MEDIUM**: May cause timing-related failures
- `Task.sleep` for waiting
- Missing `.serialized` for shared resources

**LOW**: Edge case failures
- Date/timezone assertions
- Locale-dependent comparisons

## False Positives to Avoid

**Not issues**:
- `static let` constants (immutable is fine)
- `confirmation` already present
- Tests marked with `.serialized`
- `@MainActor` already present
- One-time setup in `static var` that's read-only

**Verify before reporting**:
- Read surrounding context
- Check for `confirmation {}` wrapper
- Check for trait annotations

## XCTest Flaky Patterns (Legacy)

For XCTest code, also check:

### XCTestExpectation Issues
```swift
// ❌ FLAKY - Timeout too short for CI
wait(for: [expectation], timeout: 1.0)

// ✅ BETTER - Generous timeout
wait(for: [expectation], timeout: 10.0)
```

### Missing waitForExistence
```swift
// ❌ FLAKY - Element may not exist yet
XCTAssertTrue(app.buttons["Submit"].exists)

// ✅ CORRECT - Wait for element
XCTAssertTrue(app.buttons["Submit"].waitForExistence(timeout: 5))
```

## When No Issues Found

Report:
```markdown
# Test Failure Analysis Results

## Summary
No flaky test patterns detected.

## Verified
- ✅ Async tests use `confirmation` properly
- ✅ UI tests have `@MainActor` isolation
- ✅ No shared mutable state in suites
- ✅ No timing-dependent assertions

## Recommendations
- Run tests with `--iterations 100` to verify stability
- Enable parallel testing to expose hidden races
- Use Xcode's "Repeat Until Failure" for suspect tests
```