matthias/webdev-pipeline
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: add OpenRouter audit generation pipeline

Browse Source
This commit is contained in:
Matthias Meister
2026-06-05 11:06:01 +02:00
parent 370aeec2a0
commit 03cb65fde4
29 changed files with 5462 additions and 74 deletions
Download Patch File Download Diff File Expand all files Collapse all files

48
backlog/tasks/task-11 - Create-the-OpenRouter-AI-audit-pipeline.md
View File

@@ -1,9 +1,10 @@
---
id: TASK-11
title: Create the OpenRouter AI audit pipeline
status: To Do
status: Done
assignee: []
created_date: '2026-06-03 19:13'
updated_date: '2026-06-05 09:04'
labels:
- mvp
- agent
@@ -26,19 +27,44 @@ Implement the LLM-powered audit generation pipeline using Vercel AI SDK and Open
## Acceptance Criteria
<!-- AC:BEGIN -->
- [ ] #1 Vercel AI SDK is configured with OpenRouter and environment/Convex secrets
- [ ] #2 Model profiles exist for classification, multimodal audit analysis, German text generation, and final quality review
- [ ] #3 Structured audit outputs use Zod schemas and are stored in Convex with raw prompts/responses and model metadata
- [ ] #4 Screenshots can be passed to multimodal-capable models where supported
- [ ] #5 Generated customer-facing text follows Ich-Form, German language, no scores, no prices, no generic KI-Slop, and factual observation plus suggestion style
- [x] #1 Vercel AI SDK is configured with OpenRouter and environment/Convex secrets
- [x] #2 Model profiles exist for classification, multimodal audit analysis, German text generation, and final quality review
- [x] #3 Structured audit outputs use Zod schemas and are stored in Convex with raw prompts/responses and model metadata
- [x] #4 Screenshots can be passed to multimodal-capable models where supported
- [x] #5 Generated customer-facing text follows Ich-Form, German language, no scores, no prices, no generic KI-Slop, and factual observation plus suggestion style
<!-- AC:END -->
## Implementation Plan
<!-- SECTION:PLAN:BEGIN -->
1. Add OpenRouter provider setup through Vercel AI SDK.
2. Define Zod schemas for internal findings, audit summary, email draft, subject, call script, follow-up, and quality review.
3. Build model-profile configuration for fast classification, multimodal analysis, and German copy generation.
4. Combine lead, crawl, screenshot, PageSpeed, and selected skills into prompt inputs.
5. Persist all prompts, model responses, normalized findings, final texts, and generation errors in Convex.
1. Worker A: add OpenRouter/Vercel AI SDK dependencies, provider config, model profiles, and schema helpers with RED/GREEN tests.
2. Worker B: add Convex schema and persistence contracts for structured LLM generations with RED/GREEN source/type tests.
3. Worker C: add evidence/prompt input builder combining lead, crawl, screenshots, PageSpeed, and local skills with RED/GREEN tests.
4. Worker D: add Node audit-generation action queue/process flow with screenshots, AI SDK structured outputs, audit/outreach persistence, and failure recording with RED/GREEN tests.
5. Worker E: add German copy quality guard tests/helpers for Ich-Form, no scores, no prices, no generic KI-Slop, and observation-plus-suggestion style.
6. Orchestrator: review worker patches, resolve integration gaps through Spark follow-up workers, run full verification, and check acceptance criteria without marking Done.
<!-- SECTION:PLAN:END -->
## Implementation Notes
<!-- SECTION:NOTES:BEGIN -->
2026-06-05: Started TASK-11 implementation on branch codex-task-11-openrouter-audit-pipeline using subagent-driven and test-driven workflow. Existing TASK-25 worktree changes were present and will not be reverted or touched unless required.
Wave 1 dispatched with gpt-5.3-codex-spark: Worker A owns AI SDK/OpenRouter dependencies, model profiles, and Zod schemas; Worker B owns Convex auditGenerations schema/persistence; Worker C owns pure audit evidence builder; Worker E owns German customer-copy guard. Orchestrator remains integration/review only and is not hand-coding feature patches.
Implemented Worker-E German copy guard slice in pure deterministic helpers (lib/ai/german-copy-guard.ts) plus TDD tests (tests/german-copy-guard.test.ts). Added issue coverage for language quality, Ich-Form, score/page-speed artifacts, Preise, KI-Slop, anklagende Sprache, technische Artefakte, Beobachtung+Vorschlag. Keinen Fremdscope verändert.
Wave 1 review complete. Spec/code-quality reviewers found expected blocker: auditGenerationAction is not implemented yet and queue currently uses a temporary any reference. Follow-up scope: Worker D will add Node action, typed scheduler reference, screenshot multimodal handoff, AI SDK calls, audit/outreach persistence, and prompt/response size/sanitization guards. Worker F will harden German short-text detection, document model override env vars, and remove generated JS artifacts.
Wave 2 dispatched with gpt-5.3-codex-spark: Worker D owns auditGenerationAction, typed scheduler reference, multimodal screenshot handoff, AI SDK structured stages, audit/outreach persistence, and prompt/response persistence hardening. Worker F owns German short-text guard hardening, OpenRouter override env docs, and removal of generated JS artifacts. Orchestrator remains review/verification only.
Final review before closure: spec reviewer passed all five TASK-11 acceptance criteria, but code-quality reviewer found P1 risks in auditGenerationAction error handling and lead status patching, plus P2 hardening around UTF-8 byte capping/secret redaction. Worker H dispatched with gpt-5.3-codex-spark to address those findings before acceptance criteria are checked.
Implementation complete pending user confirmation. Built OpenRouter/Vercel AI SDK audit-generation pipeline with model profiles, Zod structured outputs, evidence builder, multimodal screenshot handoff, Convex auditGenerations persistence with prompt/response/model metadata, German copy guard, audit/outreach upserts, guarded lead status transition, action-level failure handling, UTF-8 byte-safe truncation, env-secret redaction, and model-profile driven generation parameters. Verification passed: pnpm test (235/235); pnpm exec tsc -p tsconfig.json --pretty false; pnpm lint (0 errors, existing BetterAuth generated warnings only); pnpm exec convex codegen --dry-run --typecheck enable; pnpm build. Final Spark review found no blocking/important issues; residual P3: PageSpeed evidence freshness on re-runs may need future runtime coverage.
<!-- SECTION:NOTES:END -->
## Final Summary
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
Implemented the OpenRouter/Vercel AI SDK audit-generation pipeline end to end: model profiles, Zod structured outputs, Convex audit generation persistence, evidence builder, multimodal screenshots, German copy guard, audit/outreach draft persistence, guarded lead transition, and hardening for failure handling/secret redaction. Verified with pnpm test, TypeScript, lint, Convex codegen/typecheck, build, and final Spark review.
<!-- SECTION:FINAL_SUMMARY:END -->