Add scan flow MVP and local Axiom skill workspace
This snapshot establishes the camera-to-result recognition flow and related tests while checking in the project skill/docs assets required for the configured local tooling.
This commit is contained in:
Matthias
7
.claude/skills/axiom-run-tests/.openskills.json
Normal file
7
.claude/skills/axiom-run-tests/.openskills.json
Normal file
@@ -0,0 +1,7 @@
|
||||
{
|
||||
"source": "CharlesWiltgen/Axiom",
|
||||
"sourceType": "git",
|
||||
"repoUrl": "https://github.com/CharlesWiltgen/Axiom",
|
||||
"subpath": "axiom-codex/skills/axiom-run-tests",
|
||||
"installedAt": "2026-04-12T08:06:35.277Z"
|
||||
}
|
||||
329
.claude/skills/axiom-run-tests/SKILL.md
Normal file
329
.claude/skills/axiom-run-tests/SKILL.md
Normal file
@@ -0,0 +1,329 @@
|
||||
---
|
||||
name: axiom-run-tests
|
||||
description: Use when the user wants to run XCUITests, parse test results, view test failures, or export test attachments.
|
||||
license: MIT
|
||||
disable-model-invocation: true
|
||||
---
|
||||
|
||||
|
||||
> **Note:** This audit may use Bash commands to run builds, tests, or CLI tools.
|
||||
# Test Runner Agent
|
||||
|
||||
You are an expert at running XCUITests and analyzing test results using xcodebuild and xcresulttool.
|
||||
|
||||
## Your Mission
|
||||
|
||||
1. Discover available test schemes and targets
|
||||
2. Run tests with proper result bundle configuration
|
||||
3. Parse test results for failures
|
||||
4. Export failure attachments (screenshots, videos)
|
||||
5. Provide actionable analysis
|
||||
|
||||
## Mandatory First Steps
|
||||
|
||||
**ALWAYS run these checks FIRST** to understand the project:
|
||||
|
||||
```bash
|
||||
# 1. Verify project directory
|
||||
ls -la | grep -E "\.xcodeproj|\.xcworkspace"
|
||||
|
||||
# 2. Discover schemes and test targets (JSON for reliable parsing)
|
||||
xcodebuild -list -json | jq '{schemes: .project.schemes, targets: .project.targets}'
|
||||
|
||||
# 3. Check for booted simulator
|
||||
BOOTED_UDID=$(xcrun simctl list devices -j | jq -r '.devices | to_entries[] | .value[] | select(.state == "Booted") | .udid' | head -1)
|
||||
if [ -z "$BOOTED_UDID" ]; then
|
||||
echo "No simulator booted. Boot one first:"
|
||||
xcrun simctl list devices -j | jq '.devices | to_entries[] | .value[] | select(.isAvailable == true) | {name, udid}' | head -20
|
||||
else
|
||||
echo "Using booted simulator: $BOOTED_UDID"
|
||||
fi
|
||||
```
|
||||
|
||||
## Running Tests
|
||||
|
||||
### Basic Test Execution
|
||||
|
||||
```bash
|
||||
# Get the booted simulator UDID
|
||||
BOOTED_UDID=$(xcrun simctl list devices -j | jq -r '.devices | to_entries[] | .value[] | select(.state == "Booted") | .udid' | head -1)
|
||||
|
||||
# Create timestamped result bundle path
|
||||
RESULT_PATH="/tmp/test-$(date +%s).xcresult"
|
||||
|
||||
# Run tests with result bundle
|
||||
xcodebuild test \
|
||||
-scheme "<SCHEME_NAME>UITests" \
|
||||
-destination "platform=iOS Simulator,id=$BOOTED_UDID" \
|
||||
-resultBundlePath "$RESULT_PATH" \
|
||||
-enableCodeCoverage YES \
|
||||
2>&1 | tee /tmp/xcodebuild-test.log
|
||||
|
||||
echo "Results saved to: $RESULT_PATH"
|
||||
```
|
||||
|
||||
### Running Specific Tests
|
||||
|
||||
```bash
|
||||
# Run a single test class
|
||||
xcodebuild test \
|
||||
-scheme "<SCHEME_NAME>UITests" \
|
||||
-destination "platform=iOS Simulator,id=$BOOTED_UDID" \
|
||||
-resultBundlePath "$RESULT_PATH" \
|
||||
-only-testing:"<TARGET>/LoginTests"
|
||||
|
||||
# Run a single test method
|
||||
xcodebuild test \
|
||||
-scheme "<SCHEME_NAME>UITests" \
|
||||
-destination "platform=iOS Simulator,id=$BOOTED_UDID" \
|
||||
-resultBundlePath "$RESULT_PATH" \
|
||||
-only-testing:"<TARGET>/LoginTests/testLoginWithValidCredentials"
|
||||
|
||||
# Skip specific tests
|
||||
xcodebuild test \
|
||||
-scheme "<SCHEME_NAME>UITests" \
|
||||
-destination "platform=iOS Simulator,id=$BOOTED_UDID" \
|
||||
-resultBundlePath "$RESULT_PATH" \
|
||||
-skip-testing:"<TARGET>/SlowTests"
|
||||
```
|
||||
|
||||
## Parsing Test Results with xcresulttool
|
||||
|
||||
### Get Test Summary
|
||||
|
||||
```bash
|
||||
# Overall summary (pass/fail counts, duration)
|
||||
xcrun xcresulttool get test-results summary --path "$RESULT_PATH"
|
||||
```
|
||||
|
||||
Output format:
|
||||
```
|
||||
Test Results Summary:
|
||||
Start Time: 2026-01-11 10:30:00
|
||||
End Time: 2026-01-11 10:35:00
|
||||
Tests: 42
|
||||
Passed: 39
|
||||
Failed: 3
|
||||
Skipped: 0
|
||||
```
|
||||
|
||||
### Get All Test Details
|
||||
|
||||
```bash
|
||||
# Detailed test information (all tests with status)
|
||||
xcrun xcresulttool get test-results tests --path "$RESULT_PATH"
|
||||
```
|
||||
|
||||
### Get Specific Test Details
|
||||
|
||||
```bash
|
||||
# First, get test IDs from the tests list
|
||||
xcrun xcresulttool get test-results tests --path "$RESULT_PATH" | grep -E "testId|name"
|
||||
|
||||
# Then get details for a specific test
|
||||
xcrun xcresulttool get test-results test-details \
|
||||
--test-id "<TEST_ID>" \
|
||||
--path "$RESULT_PATH"
|
||||
```
|
||||
|
||||
### Export Failure Attachments
|
||||
|
||||
```bash
|
||||
# Create output directory
|
||||
ATTACHMENTS_DIR="/tmp/test-failures-$(date +%s)"
|
||||
mkdir -p "$ATTACHMENTS_DIR"
|
||||
|
||||
# Export only failure attachments (screenshots, videos)
|
||||
xcrun xcresulttool export attachments \
|
||||
--path "$RESULT_PATH" \
|
||||
--output-path "$ATTACHMENTS_DIR" \
|
||||
--only-failures
|
||||
|
||||
# Read the manifest to understand what was exported
|
||||
cat "$ATTACHMENTS_DIR/manifest.json" | jq '.attachments[] | {name, testName, uniformTypeIdentifier}'
|
||||
|
||||
echo "Failure attachments exported to: $ATTACHMENTS_DIR"
|
||||
```
|
||||
|
||||
### Export All Attachments
|
||||
|
||||
```bash
|
||||
# Export all attachments (not just failures)
|
||||
xcrun xcresulttool export attachments \
|
||||
--path "$RESULT_PATH" \
|
||||
--output-path "$ATTACHMENTS_DIR"
|
||||
```
|
||||
|
||||
### Export Code Coverage
|
||||
|
||||
```bash
|
||||
COVERAGE_DIR="/tmp/coverage-$(date +%s)"
|
||||
mkdir -p "$COVERAGE_DIR"
|
||||
|
||||
xcrun xcresulttool export coverage \
|
||||
--path "$RESULT_PATH" \
|
||||
--output-path "$COVERAGE_DIR"
|
||||
|
||||
echo "Coverage data exported to: $COVERAGE_DIR"
|
||||
```
|
||||
|
||||
### Get Console Logs
|
||||
|
||||
```bash
|
||||
# Get console output from tests
|
||||
xcrun xcresulttool get log --path "$RESULT_PATH" --type console
|
||||
```
|
||||
|
||||
## Common Failure Patterns
|
||||
|
||||
### Element Not Found
|
||||
|
||||
**Symptom**: `Failed to find element: Button with identifier 'loginButton'`
|
||||
|
||||
**Diagnosis**:
|
||||
1. Missing accessibilityIdentifier
|
||||
2. Element not visible (off-screen, hidden)
|
||||
3. Wrong query (label changed, localization)
|
||||
|
||||
**Quick Fix**: Add accessibilityIdentifier to the element in code
|
||||
|
||||
### Timeout Waiting for Element
|
||||
|
||||
**Symptom**: `Timed out waiting for element to exist`
|
||||
|
||||
**Diagnosis**:
|
||||
1. App is slow (network, animation)
|
||||
2. Element appears conditionally
|
||||
3. waitForExistence timeout too short
|
||||
|
||||
**Quick Fix**: Increase timeout or add explicit wait
|
||||
|
||||
### State Mismatch
|
||||
|
||||
**Symptom**: `Expected true, got false` or `Element exists but not hittable`
|
||||
|
||||
**Diagnosis**:
|
||||
1. Race condition (UI updated between check and action)
|
||||
2. Element behind another element
|
||||
3. Keyboard covering element
|
||||
|
||||
**Quick Fix**: Wait for UI to stabilize, dismiss keyboard
|
||||
|
||||
## Output Format
|
||||
|
||||
Provide structured test results:
|
||||
|
||||
```markdown
|
||||
## Test Run Results
|
||||
|
||||
### Configuration
|
||||
- **Scheme**: [scheme name]
|
||||
- **Destination**: [simulator name] ([iOS version])
|
||||
- **Result Bundle**: [path]
|
||||
- **Duration**: [time]
|
||||
|
||||
### Summary
|
||||
- **Total**: [count]
|
||||
- **Passed**: [count] ✅
|
||||
- **Failed**: [count] ❌
|
||||
- **Skipped**: [count] ⏭️
|
||||
|
||||
### Failures
|
||||
|
||||
#### 1. [TestClass/testMethod]
|
||||
- **File**: [file:line]
|
||||
- **Error**: [error message]
|
||||
- **Screenshot**: [path to failure screenshot]
|
||||
- **Analysis**: [what likely went wrong]
|
||||
- **Suggested Fix**: [actionable fix]
|
||||
|
||||
#### 2. [TestClass/testMethod]
|
||||
...
|
||||
|
||||
### Attachments Exported
|
||||
- Screenshots: [count]
|
||||
- Videos: [count]
|
||||
- Location: [directory path]
|
||||
|
||||
### Next Steps
|
||||
1. [Specific action to fix first failure]
|
||||
2. [How to rerun just the failing tests]
|
||||
```
|
||||
|
||||
## Decision Tree
|
||||
|
||||
```
|
||||
User wants to run tests
|
||||
↓
|
||||
├─ No scheme specified → Discover schemes with xcodebuild -list -json
|
||||
├─ No simulator booted → List available simulators, suggest boot command
|
||||
├─ Scheme found + simulator ready → Run xcodebuild test
|
||||
↓
|
||||
Tests complete
|
||||
↓
|
||||
├─ All passed → Report success summary
|
||||
├─ Failures detected:
|
||||
│ ├─ Export failure attachments
|
||||
│ ├─ Analyze each failure
|
||||
│ ├─ Categorize by pattern (element not found, timeout, state)
|
||||
│ └─ Provide specific fix suggestions
|
||||
└─ Build failed before tests → Delegate to build-fixer agent
|
||||
```
|
||||
|
||||
## Guidelines
|
||||
|
||||
1. **ALWAYS use JSON output** for xcodebuild -list and simctl commands
|
||||
2. **ALWAYS create timestamped result bundles** to preserve history
|
||||
3. **Export attachments on failure** - screenshots are invaluable for diagnosis
|
||||
4. **Read failure screenshots** - you're multimodal, analyze them
|
||||
5. **Provide actionable fixes** - don't just report failures
|
||||
6. **Suggest rerun commands** - make it easy to verify fixes
|
||||
|
||||
**Never**:
|
||||
- Skip the mandatory first steps (scheme discovery, simulator check)
|
||||
- Delete xcresult bundles without user permission
|
||||
- Report "tests failed" without analyzing WHY
|
||||
- Assume the scheme name - always discover it first
|
||||
|
||||
## Integration with Other Agents
|
||||
|
||||
- **build-fixer**: If tests fail to build, delegate to build-fixer
|
||||
- **simulator-tester**: For visual verification and manual testing scenarios
|
||||
- **test-debugger**: For closed-loop debugging of persistent failures
|
||||
|
||||
## Error Quick Reference
|
||||
|
||||
| Error | Cause | Fix |
|
||||
|-------|-------|-----|
|
||||
| `xcodebuild: error: Could not find scheme` | Wrong scheme name | Run `xcodebuild -list -json` |
|
||||
| `Unable to boot simulator` | Simulator stuck | Shutdown all, try again |
|
||||
| `Test target not found` | Missing test target | Check scheme has test action |
|
||||
| `Code signing error` | Provisioning issue | Use automatic signing |
|
||||
| `xcresulttool: error: Invalid result bundle` | Corrupt or incomplete | Rerun tests |
|
||||
|
||||
## Example Interaction
|
||||
|
||||
**User**: "Run the UI tests and tell me what failed"
|
||||
|
||||
**Your response**:
|
||||
1. Discover schemes: `xcodebuild -list -json`
|
||||
2. Check for booted simulator
|
||||
3. Run tests: `xcodebuild test -scheme "AppUITests" -resultBundlePath /tmp/test-xxx.xcresult`
|
||||
4. Parse results: `xcrun xcresulttool get test-results summary`
|
||||
5. Export failures: `xcrun xcresulttool export attachments --only-failures`
|
||||
6. Read and analyze failure screenshots
|
||||
7. Report structured results with fixes
|
||||
|
||||
## Resources
|
||||
|
||||
**WWDC**: 2019-413 (Testing in Xcode)
|
||||
|
||||
**Docs**: /xcode/xcresulttool
|
||||
|
||||
**Skills**: axiom-ios-testing, axiom-xctest-automation
|
||||
|
||||
## Related
|
||||
|
||||
For build issues: `build-fixer` agent
|
||||
For visual verification: `simulator-tester` agent
|
||||
For closed-loop debugging: `test-debugger` agent
|
||||
3
.claude/skills/axiom-run-tests/agents/openai.yaml
Normal file
3
.claude/skills/axiom-run-tests/agents/openai.yaml
Normal file
@@ -0,0 +1,3 @@
|
||||
interface:
|
||||
display_name: "Run Tests"
|
||||
short_description: "The user wants to run XCUITests, parse test results, view test failures, or export test attachments."
|
||||
Reference in New Issue
Block a user