Integrate PageSpeed Insights audits

This commit is contained in:
2026-06-04 22:12:59 +02:00
parent 99d61ac736
commit f0a948aec9
19 changed files with 3755 additions and 12 deletions

View File

@@ -20,6 +20,7 @@ BETTER_AUTH_SECRET=
GOOGLE_GEOCODING_API_KEY=
GOOGLE_PLACES_API_KEY=
PAGESPEED_API_KEY=
PAGESPEED_TIMEOUT_MS=60000
# OpenRouter
OPENROUTER_API_KEY=

View File

@@ -23,7 +23,7 @@ Copy `.env.example` to `.env.local` for local development. Keep real secrets out
- **App / Coolify:** `APP_ENV`, `NEXT_PUBLIC_APP_URL`
- **Convex:** `NEXT_PUBLIC_CONVEX_URL`, `NEXT_PUBLIC_CONVEX_SITE_URL`, `CONVEX_DEPLOYMENT`
- **Google:** `GOOGLE_GEOCODING_API_KEY`, `GOOGLE_PLACES_API_KEY`, `PAGESPEED_API_KEY`
- **Google / Task-9 PageSpeed:** `GOOGLE_GEOCODING_API_KEY`, `GOOGLE_PLACES_API_KEY`, `PAGESPEED_API_KEY`, `PAGESPEED_TIMEOUT_MS`
- **OpenRouter:** `OPENROUTER_API_KEY`
- **SMTP / Stalwart:** `SMTP_HOST`, `SMTP_PORT`, `SMTP_USER`, `SMTP_PASSWORD`, `SMTP_FROM`
- **Rybbit:** `RYBBIT_API_URL`, `RYBBIT_API_KEY`, `NEXT_PUBLIC_RYBBIT_SITE_ID`

View File

@@ -1,9 +1,10 @@
---
id: TASK-9
title: Integrate PageSpeed Insights into internal audits
status: To Do
status: Done
assignee: []
created_date: '2026-06-03 19:13'
updated_date: '2026-06-04 20:12'
labels:
- mvp
- audit
@@ -24,19 +25,55 @@ Add Google PageSpeed Insights as an objective internal audit signal. The system
## Acceptance Criteria
<!-- AC:BEGIN -->
- [ ] #1 PageSpeed API runs for mobile and desktop strategies for qualified website leads
- [ ] #2 Raw PageSpeed/Lighthouse response data is stored internally in Convex
- [ ] #3 Key metrics are normalized for downstream analysis without exposing scores on customer audit pages
- [ ] #4 Failures, quota errors, and unavailable pages are recorded without failing the entire audit pipeline
- [ ] #5 Generated audit inputs translate technical signals into customer-impact language for later text generation
- [x] #1 PageSpeed API runs for mobile and desktop strategies for qualified website leads
- [x] #2 Raw PageSpeed/Lighthouse response data is stored internally in Convex
- [x] #3 Key metrics are normalized for downstream analysis without exposing scores on customer audit pages
- [x] #4 Failures, quota errors, and unavailable pages are recorded without failing the entire audit pipeline
- [x] #5 Generated audit inputs translate technical signals into customer-impact language for later text generation
<!-- AC:END -->
## Implementation Plan
<!-- SECTION:PLAN:BEGIN -->
1. Add PageSpeed API client using environment/Convex secrets.
2. Run mobile and desktop analysis for the lead domain or final URL.
3. Normalize key findings such as load speed, mobile/desktop gap, SEO, accessibility, and best-practice hints.
4. Store raw and normalized results in Convex.
5. Add error handling and dashboard-visible status for quota, timeout, and API failures.
1. Worker A: write RED/GREEN pure PageSpeed client and normalization tests plus implementation in lib/pagespeed-insights.ts.
2. Worker B: write RED/GREEN Convex schema and persistence contract tests plus pageSpeed mutation module.
3. Worker C: write RED/GREEN PageSpeed action queue/process source tests plus Node action implementation.
4. Worker D: write RED/GREEN audit input/public-safety tests plus internal plain-language audit input helper.
5. Orchestrator: run integration verification, resolve conflicts via agents, update acceptance criteria, and leave TASK-9 open for user confirmation.
<!-- SECTION:PLAN:END -->
## Implementation Notes
<!-- SECTION:NOTES:BEGIN -->
2026-06-04: Implementation started on branch codex-task-9-pagespeed-insights. Wave 1 dispatched with gpt-5.3-codex-spark: Worker A owns lib/pagespeed-insights.ts + tests/pagespeed-insights.test.ts; Worker B owns Convex PageSpeed schema/persistence contracts in convex/schema.ts, convex/domain.ts, convex/pageSpeed.ts, and related tests. Orchestrator remains coordination/review only.
2026-06-04T19:40Z: Implemented and validated lib/pagespeed-insights.ts with request URL builder, normalizer, fetch helper, and error classifier. Added tests/pagespeed-insights.test.ts with RED→GREEN coverage (URL contract, normalization, error classification, injected fetch, offline-only assertions).
Wave 1 complete: Worker A delivered pure PageSpeed URL/client/normalizer tests and implementation; Worker B delivered Convex pageSpeedResults schema and internal persistence queue/start/persist/finish module. Worker A concern noted: targeted pnpm test args are incompatible with project script, but isolated compiled test and tsconfig.test passed.
2026-06-04T19:50Z: Worker D taking subtask for TDD implementation of lib/pagespeed-audit-input.ts and tests/pagespeed-audit-input.test.ts (score-free German customer-implication generator).
Implementation complete pending user confirmation. Built PageSpeed API client/normalizer, Convex pageSpeedResults raw-storage persistence, internal audit run queue/action for mobile+desktop, post-website-enrichment scheduling, per-strategy error recording, raw payload size guard, score-free audit input translator, and public-output sanitization. Review findings addressed: malformed JSON 200 responses now fail as api_error; PageSpeed action has outer failure guard; oversized raw payloads fail per strategy; audit inputs strip URLs/markup/JSON/raw score artifacts. Final verification passed: pnpm test (155/155); pnpm exec tsc -p tsconfig.json --pretty false; pnpm lint (0 errors, existing generated BetterAuth warnings only); pnpm exec convex codegen --dry-run --typecheck enable (rerun outside sandbox after DNS ENOTFOUND). TASK-9 remains In Progress until user confirms manual acceptance.
2026-06-04: Follow-up from manual test: PageSpeed failed with generic api_error summary "PageSpeed-API lieferte einen Fehler." Root cause at diagnostics layer: HTTP 4xx/5xx classifier discarded Google error.message/error_message/runtimeError details. Added RED/GREEN regression coverage and now preserves Google API error messages in PageSpeedError summaries. Verification passed: pnpm test (156/156); pnpm exec tsc -p tsconfig.json --pretty false; pnpm lint (0 errors, existing generated BetterAuth warnings only).
2026-06-04: Manual test follow-up: after API key renewal, PageSpeed failed with timeout. Root cause: convex/pageSpeedAction.ts used hardcoded 10_000ms timeout, too short for PageSpeed Insights. Added PAGESPEED_TIMEOUT_MS env support with 60_000ms default and 10_000-120_000 clamp; fetchPageSpeedResult now receives resolved timeout. Updated README/.env.example. Verification passed: pnpm test (159/159); pnpm exec tsc -p tsconfig.json --pretty false; pnpm lint (0 errors, existing generated BetterAuth warnings only).
2026-06-04: Systematic debugging follow-up for recurring PageSpeed timeout/unknown reports. Root cause after increasing timeout was not another HTTP timeout: PageSpeed responses reached the action, but Convex rejected the success payload because `normalized` included extra normalizer-only fields (`strategy`, `sourceUrl`, `finalUrl`, `analysisTimestamp`) not allowed by the persistPageSpeedResult validator. Spark Worker A added `toPersistedPageSpeedNormalizedResult` in convex/pageSpeedAction.ts and success persistence now stores only `scores`, `metrics`, `opportunities`, and `implications`, with `finalUrl` kept top-level. Spark Reviewer B confirmed the mapping matches convex/pageSpeed.ts and convex/schema.ts. Verification passed: pnpm test (160/160), pnpm exec tsc -p tsconfig.json --pretty false, pnpm lint (0 errors, two pre-existing generated BetterAuth warnings), pnpm exec convex dev --once --typecheck enable. Real dev retry for lead jx7cnezm2xg7b2xr2gfmqyeg5h881m2d on run j972t5ra323rgax4a7ycsbrtzd881m8n confirmed desktop persisted successfully with raw storage and normalized keys [implications, metrics, opportunities, scores]; mobile failed separately with a genuine Google/Lighthouse api_error: "Lighthouse returned error: Something went wrong." A subsequent clean rerun could not start because the lead had been manually deleted during Convex cleanup. TASK-9 remains In Progress pending user manual acceptance.
2026-06-04: Manual retest update from user: mobile PageSpeed produced one transient api_error ("Lighthouse returned error: Something went wrong.") and succeeded three times. This confirms the previous timeout/unknown/validator failure is no longer recurring; remaining failure mode is an intermittent Google/Lighthouse strategy-level error that is recorded without breaking the pipeline. TASK-9 remains In Progress until explicit user confirmation to close.
2026-06-04: Follow-up opened from user manual testing: PageSpeed should still be triggered when website enrichment fails but the lead has a website URL. Initial trace shows current queueLeadPageSpeedAudit call is only in the successful enrichment path after persistence; fatal failure paths finish/patch the website enrichment run without queueing PageSpeed. Keeping TASK-9 In Progress.
Started minimal PAGE-SPEED queueing fix for processLeadEnrichment failure paths; targeting invalid-URL guard + outer catch to queue PageSpeed before return and keep existing success queue/warn semantics.
Implemented PASS for processLeadEnrichment missing failure-path queueing: added queue+warning fallback in !rootUrl branch and fatal outer catch when started exists; kept success path queue behavior. Verified with: pnpm exec tsc -p tsconfig.test.json && pnpm exec node --test .test-output/tests/website-enrichment-action.test.js (20 pass).
2026-06-04: Follow-up fixed after manual finding that PageSpeed was not triggered when website enrichment failed. Added RED regression tests for both failure paths in tests/website-enrichment-action.test.ts: invalid URL failure and fatal catch path must queue internal.pageSpeed.queueLeadPageSpeedAudit with leadId started.lead._id and parentRunId runId before returning. Spark GREEN worker updated convex/websiteEnrichmentAction.ts so invalid-url and fatal failure paths queue PageSpeed with warning-safe handling; success path remains queued before success finish. Refactor pass restored guard-style structure and fixed test helper source parameter usage. Verification passed: pnpm test (162/162), pnpm exec tsc -p tsconfig.json --pretty false, pnpm lint (0 errors, two pre-existing generated BetterAuth warnings), pnpm exec convex dev --once --typecheck enable. TASK-9 remains In Progress pending manual acceptance.
<!-- SECTION:NOTES:END -->
## Final Summary
<!-- SECTION:FINAL_SUMMARY:BEGIN -->
Integrated Google PageSpeed Insights into the internal audit pipeline. Added mobile and desktop PageSpeed queue/action processing, raw Convex storage, normalized metrics, score-free customer-impact audit inputs, resilient per-strategy failure recording, API diagnostics, configurable timeout, and follow-up fixes from manual testing: persisted normalized payload shape now matches Convex validators and PageSpeed is triggered even when website enrichment fails for a lead with a website URL. Verified with pnpm test, TypeScript, lint, Convex dev deploy, and user manual retests.
<!-- SECTION:FINAL_SUMMARY:END -->

View File

@@ -8,6 +8,7 @@
* @module
*/
import type * as auditInputs from "../auditInputs.js";
import type * as audits from "../audits.js";
import type * as blacklist from "../blacklist.js";
import type * as campaigns from "../campaigns.js";
@@ -16,6 +17,8 @@ import type * as http from "../http.js";
import type * as leadDiscovery from "../leadDiscovery.js";
import type * as leads from "../leads.js";
import type * as outreach from "../outreach.js";
import type * as pageSpeed from "../pageSpeed.js";
import type * as pageSpeedAction from "../pageSpeedAction.js";
import type * as runs from "../runs.js";
import type * as settings from "../settings.js";
import type * as storage from "../storage.js";
@@ -29,6 +32,7 @@ import type {
} from "convex/server";
declare const fullApi: ApiFromModules<{
auditInputs: typeof auditInputs;
audits: typeof audits;
blacklist: typeof blacklist;
campaigns: typeof campaigns;
@@ -37,6 +41,8 @@ declare const fullApi: ApiFromModules<{
leadDiscovery: typeof leadDiscovery;
leads: typeof leads;
outreach: typeof outreach;
pageSpeed: typeof pageSpeed;
pageSpeedAction: typeof pageSpeedAction;
runs: typeof runs;
settings: typeof settings;
storage: typeof storage;

60
convex/auditInputs.ts Normal file
View File

@@ -0,0 +1,60 @@
import { v } from "convex/values";
import type { Doc, Id } from "./_generated/dataModel";
import { internalQuery } from "./_generated/server";
import { buildPageSpeedAuditInputs, type PageSpeedMinimalAuditResult } from "../lib/pagespeed-audit-input";
function normalizePageSpeedResultRow(
row: Doc<"pageSpeedResults">,
): PageSpeedMinimalAuditResult {
return {
strategy: row.strategy,
status: row.status,
sourceUrl: row.sourceUrl,
...(row.finalUrl ? { finalUrl: row.finalUrl } : {}),
...(row.normalized ? { normalized: row.normalized } : {}),
...(row.errorType ? { errorType: row.errorType } : {}),
...(row.errorSummary ? { errorSummary: row.errorSummary } : {}),
};
}
export const getPageSpeedAuditInputs = internalQuery({
args: {
leadId: v.optional(v.id("leads")),
auditId: v.optional(v.id("audits")),
},
handler: async (
ctx,
args,
): Promise<{
technicalSignals: string[];
customerImplications: string[];
internalNotes: string[];
}> => {
let results: Doc<"pageSpeedResults">[];
if (args.auditId) {
results = await ctx.db
.query("pageSpeedResults")
.withIndex("by_auditId", (q) => q.eq("auditId", args.auditId as Id<"audits">))
.order("desc")
.take(50);
return buildPageSpeedAuditInputs(results.map(normalizePageSpeedResultRow));
}
if (args.leadId) {
results = await ctx.db
.query("pageSpeedResults")
.withIndex("by_leadId", (q) => q.eq("leadId", args.leadId as Id<"leads">))
.order("desc")
.take(50);
return buildPageSpeedAuditInputs(results.map(normalizePageSpeedResultRow));
}
return {
technicalSignals: [],
customerImplications: [],
internalNotes: [],
};
},
});

View File

@@ -95,6 +95,16 @@ export const RUN_STATUSES = [
] as const;
export const RUN_EVENT_LEVELS = ["info", "warning", "error"] as const;
export const SCREENSHOT_VIEWPORTS = ["desktop", "mobile"] as const;
export const PAGE_SPEED_STRATEGIES = ["mobile", "desktop"] as const;
export const PAGE_SPEED_RESULT_STATUSES = ["succeeded", "failed"] as const;
export const PAGE_SPEED_ERROR_TYPES = [
"quota",
"timeout",
"unavailable",
"invalid_url",
"api_error",
"unknown",
] as const;
export type CampaignStatus = (typeof CAMPAIGN_STATUSES)[number];
export type LeadPriority = (typeof LEAD_PRIORITIES)[number];
@@ -114,6 +124,9 @@ export type RunType = (typeof RUN_TYPES)[number];
export type RunStatus = (typeof RUN_STATUSES)[number];
export type RunEventLevel = (typeof RUN_EVENT_LEVELS)[number];
export type ScreenshotViewport = (typeof SCREENSHOT_VIEWPORTS)[number];
export type PageSpeedStrategy = (typeof PAGE_SPEED_STRATEGIES)[number];
export type PageSpeedResultStatus = (typeof PAGE_SPEED_RESULT_STATUSES)[number];
export type PageSpeedErrorType = (typeof PAGE_SPEED_ERROR_TYPES)[number];
export type SettingsRow = {
key: string;

314
convex/pageSpeed.ts Normal file
View File

@@ -0,0 +1,314 @@
import { internal } from "./_generated/api";
import type { Doc, Id } from "./_generated/dataModel";
import { internalMutation } from "./_generated/server";
import { v } from "convex/values";
const PAGE_SPEED_COUNTER_TEMPLATE = {
leadsFound: 1,
leadsCreated: 0,
auditsCreated: 1,
outreachPrepared: 0,
errors: 0,
};
type PageSpeedLead = Pick<
Doc<"leads">,
"_id" | "contactStatus"
> & {
websiteUrl: string;
};
const runStatus = v.union(
v.literal("pending"),
v.literal("running"),
v.literal("succeeded"),
v.literal("failed"),
v.literal("canceled"),
);
const pageSpeedStrategy = v.union(v.literal("mobile"), v.literal("desktop"));
const pageSpeedResultStatus = v.union(
v.literal("succeeded"),
v.literal("failed"),
);
const pageSpeedErrorType = v.union(
v.literal("quota"),
v.literal("timeout"),
v.literal("unavailable"),
v.literal("invalid_url"),
v.literal("api_error"),
v.literal("unknown"),
);
export const queueLeadPageSpeedAudit = internalMutation({
args: {
leadId: v.id("leads"),
parentRunId: v.optional(v.id("agentRuns")),
},
returns: v.union(v.id("agentRuns"), v.null()),
handler: async (ctx, args): Promise<Id<"agentRuns"> | null> => {
const now = Date.now();
const lead = await ctx.db.get(args.leadId);
if (!lead || lead.priority === "blocked" || lead.priority === "defer") {
return null;
}
if (!lead.websiteUrl) {
return null;
}
const existingPending = await ctx.db
.query("agentRuns")
.withIndex("by_type_and_status_and_leadId", (q) =>
q.eq("type", "audit").eq("status", "pending").eq("leadId", args.leadId),
)
.take(1);
const existingRunning = await ctx.db
.query("agentRuns")
.withIndex("by_type_and_status_and_leadId", (q) =>
q.eq("type", "audit").eq("status", "running").eq("leadId", args.leadId),
)
.take(1);
if (existingPending.length > 0) {
return existingPending[0]._id;
}
if (existingRunning.length > 0) {
return existingRunning[0]._id;
}
const runId = await ctx.db.insert("agentRuns", {
type: "audit",
leadId: args.leadId,
status: "pending",
currentStep: "pagespeed_insights",
counters: PAGE_SPEED_COUNTER_TEMPLATE,
createdAt: now,
updatedAt: now,
});
await ctx.db.insert("agentRunEvents", {
runId,
level: "info",
message: "PageSpeed-Analyse wurde in die Warteschlange gesetzt.",
details: [
{ label: "Lead", value: args.leadId },
...(args.parentRunId ? [{ label: "Parent-Run", value: args.parentRunId }] : []),
],
createdAt: now,
});
await ctx.scheduler.runAfter(
0,
internal.pageSpeedAction.processPageSpeedAudit,
{
runId,
},
);
return runId;
},
});
export const startPageSpeedAuditRun = internalMutation({
args: {
runId: v.id("agentRuns"),
},
returns: v.union(
v.object({
lead: v.object({
_id: v.id("leads"),
websiteUrl: v.string(),
contactStatus: v.union(
v.literal("new"),
v.literal("missing_contact"),
v.literal("audit_ready"),
v.literal("outreach_ready"),
v.literal("contacted"),
v.literal("replied"),
v.literal("do_not_contact"),
),
}),
auditId: v.optional(v.id("audits")),
}),
v.null(),
),
handler: async (ctx, args): Promise<
{ lead: PageSpeedLead; auditId?: Id<"audits"> } | null
> => {
const now = Date.now();
const run = await ctx.db.get(args.runId);
if (!run) {
return null;
}
if (run.type !== "audit") {
return null;
}
if (run.status !== "pending") {
return null;
}
if (!run.leadId) {
await ctx.db.patch(args.runId, {
status: "failed",
currentStep: "pagespeed_insights",
errorSummary: "Run hat keine Lead-ID.",
updatedAt: now,
finishedAt: now,
});
await ctx.db.insert("agentRunEvents", {
runId: args.runId,
level: "error",
message:
"PageSpeed-Analyse konnte nicht gestartet werden: Kein Lead verknüpft.",
details: [{ label: "Lead-ID", value: "unbekannt" }],
createdAt: now,
});
return null;
}
const lead = await ctx.db.get(run.leadId);
if (!lead) {
await ctx.db.patch(args.runId, {
status: "failed",
currentStep: "pagespeed_insights",
errorSummary: "Lead wurde nicht gefunden.",
updatedAt: now,
finishedAt: now,
});
await ctx.db.insert("agentRunEvents", {
runId: args.runId,
level: "error",
message:
"PageSpeed-Analyse konnte nicht gestartet werden: Kein Lead mit Website-URL.",
details: [{ label: "Lead-ID", value: run.leadId }],
createdAt: now,
});
return null;
}
if (!lead.websiteUrl) {
await ctx.db.patch(args.runId, {
status: "failed",
currentStep: "pagespeed_insights",
errorSummary: "Lead hat keine Website-URL.",
updatedAt: now,
finishedAt: now,
});
await ctx.db.insert("agentRunEvents", {
runId: args.runId,
level: "error",
message:
"PageSpeed-Analyse konnte nicht gestartet werden: Kein Lead mit Website-URL.",
details: [{ label: "Lead-ID", value: lead._id }],
createdAt: now,
});
return null;
}
await ctx.db.patch(args.runId, {
status: "running",
currentStep: "pagespeed_insights",
startedAt: now,
updatedAt: now,
errorSummary: undefined,
});
await ctx.db.insert("agentRunEvents", {
runId: args.runId,
level: "info",
message: "PageSpeed-Analyse gestartet.",
details: [{ label: "Lead-ID", value: lead._id }],
createdAt: now,
});
return {
lead: {
_id: lead._id,
websiteUrl: lead.websiteUrl,
contactStatus: lead.contactStatus,
},
...(run.auditId ? { auditId: run.auditId } : {}),
};
},
});
export const persistPageSpeedResult = internalMutation({
args: {
leadId: v.id("leads"),
auditId: v.optional(v.id("audits")),
runId: v.id("agentRuns"),
strategy: pageSpeedStrategy,
status: pageSpeedResultStatus,
sourceUrl: v.string(),
finalUrl: v.optional(v.string()),
rawStorageId: v.optional(v.id("_storage")),
errorType: v.optional(pageSpeedErrorType),
errorSummary: v.optional(v.string()),
fetchedAt: v.number(),
normalized: v.optional(
v.object({
scores: v.optional(
v.object({
performance: v.optional(v.number()),
accessibility: v.optional(v.number()),
bestPractices: v.optional(v.number()),
seo: v.optional(v.number()),
}),
),
metrics: v.optional(
v.object({
firstContentfulPaintMs: v.optional(v.number()),
largestContentfulPaintMs: v.optional(v.number()),
cumulativeLayoutShift: v.optional(v.number()),
totalBlockingTimeMs: v.optional(v.number()),
speedIndexMs: v.optional(v.number()),
}),
),
opportunities: v.optional(v.array(v.string())),
implications: v.optional(v.array(v.string())),
}),
),
},
returns: v.id("pageSpeedResults"),
handler: async (ctx, args): Promise<Id<"pageSpeedResults">> => {
return await ctx.db.insert("pageSpeedResults", {
...args,
createdAt: Date.now(),
});
},
});
export const finishPageSpeedAuditRun = internalMutation({
args: {
runId: v.id("agentRuns"),
status: runStatus,
errorSummary: v.optional(v.string()),
errors: v.optional(v.number()),
},
handler: async (ctx, args) => {
const now = Date.now();
await ctx.db.patch(args.runId, {
status: args.status,
updatedAt: now,
finishedAt: now,
currentStep: "pagespeed_insights",
errorSummary: args.errorSummary,
counters: {
...PAGE_SPEED_COUNTER_TEMPLATE,
errors: args.errors ?? 0,
},
});
},
});

289
convex/pageSpeedAction.ts Normal file
View File

@@ -0,0 +1,289 @@
"use node";
import { api, internal } from "./_generated/api";
import { internalAction } from "./_generated/server";
import type { Id } from "./_generated/dataModel";
import { v } from "convex/values";
import {
classifyPageSpeedError,
fetchPageSpeedResult,
normalizePageSpeedResult,
type PageSpeedErrorType,
} from "../lib/pagespeed-insights";
const STRATEGIES = ["mobile", "desktop"] as const;
export const MAX_RAW_PAGESPEED_BYTES = 1_000_000;
const RAW_PAGESPEED_BYTES_SUMMARY =
"PageSpeed-Rohdaten sind groesser als das interne Speicherlimit.";
const DEFAULT_PAGESPEED_TIMEOUT_MS = 60_000;
const MIN_PAGESPEED_TIMEOUT_MS = 10_000;
const MAX_PAGESPEED_TIMEOUT_MS = 120_000;
function toPersistedPageSpeedNormalizedResult(
normalized: ReturnType<typeof normalizePageSpeedResult>,
) {
return {
...(normalized.scores ? { scores: normalized.scores } : {}),
metrics: normalized.metrics,
opportunities: normalized.opportunities,
implications: normalized.implications,
};
}
function parsePageSpeedTimeoutMs(raw: string | undefined): number {
if (!raw) {
return DEFAULT_PAGESPEED_TIMEOUT_MS;
}
const parsed = Number.parseInt(raw, 10);
if (!Number.isFinite(parsed)) {
return DEFAULT_PAGESPEED_TIMEOUT_MS;
}
return Math.min(
Math.max(parsed, MIN_PAGESPEED_TIMEOUT_MS),
MAX_PAGESPEED_TIMEOUT_MS,
);
}
function resolvePageSpeedTimeoutMs() {
return parsePageSpeedTimeoutMs(process.env.PAGESPEED_TIMEOUT_MS);
}
function isPageSpeedErrorType(value: unknown): value is PageSpeedErrorType {
return (
value === "quota" ||
value === "timeout" ||
value === "unavailable" ||
value === "invalid_url" ||
value === "api_error" ||
value === "unknown"
);
}
function sanitizeValue(value: string, secret?: string | null) {
if (!secret || !value) {
return value;
}
const escapedSecret = secret.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
return value.replace(new RegExp(escapedSecret, "g"), "[REDACTED]");
}
function classifyPageSpeedFailure(input: unknown, apiKey?: string | null) {
const directType =
typeof input === "object" &&
input !== null &&
"errorType" in input &&
(input as { errorType?: unknown }).errorType;
const normalizedType = isPageSpeedErrorType(directType) ? directType : null;
if (normalizedType) {
const message =
input instanceof Error && input.message
? input.message
: typeof input === "string"
? input
: "PageSpeed-Analyse fehlgeschlagen.";
return {
errorType: normalizedType,
errorSummary: sanitizeValue(message, apiKey),
};
}
const classified = classifyPageSpeedError({
error: input,
});
const errorSummary = sanitizeValue(classified.message, apiKey);
return {
errorType: classified.errorType,
errorSummary,
};
}
export const processPageSpeedAudit = internalAction({
args: {
runId: v.id("agentRuns"),
},
handler: async (ctx, args) => {
const apiKeyRaw = process.env.PAGESPEED_API_KEY?.trim();
const apiKey = apiKeyRaw ? apiKeyRaw : undefined;
let started:
| {
lead: {
_id: Id<"leads">;
websiteUrl: string;
};
auditId?: Id<"audits">;
}
| null = null;
try {
started = await ctx.runMutation(internal.pageSpeed.startPageSpeedAuditRun, {
runId: args.runId,
});
} catch (error) {
const { errorSummary } = classifyPageSpeedFailure(error, apiKeyRaw);
await ctx.runMutation(internal.pageSpeed.finishPageSpeedAuditRun, {
runId: args.runId,
status: "failed",
errors: 1,
errorSummary,
});
await ctx.runMutation(api.runs.appendEvent, {
runId: args.runId,
level: "error",
message: "PageSpeed-Analyse fehlgeschlagen.",
details: [{ label: "Fehler", value: errorSummary }],
});
return null;
}
if (!started) {
return null;
}
const sourceUrl = started.lead.websiteUrl;
const timeoutMs = resolvePageSpeedTimeoutMs();
let failedStrategies = 0;
let succeededStrategies = 0;
try {
for (const strategy of STRATEGIES) {
const fetchedAt = Date.now();
try {
const raw = await fetchPageSpeedResult({
url: sourceUrl,
strategy,
apiKey,
timeoutMs,
});
const rawJson = JSON.stringify(raw) ?? "null";
const rawJsonBytes = new TextEncoder().encode(rawJson).byteLength;
if (rawJsonBytes > MAX_RAW_PAGESPEED_BYTES) {
failedStrategies += 1;
await ctx.runMutation(internal.pageSpeed.persistPageSpeedResult, {
leadId: started.lead._id,
...(started.auditId ? { auditId: started.auditId } : {}),
runId: args.runId,
strategy,
status: "failed",
sourceUrl,
errorType: "api_error",
errorSummary: RAW_PAGESPEED_BYTES_SUMMARY,
fetchedAt,
});
await ctx.runMutation(api.runs.appendEvent, {
runId: args.runId,
level: "warning",
message: `PageSpeed-Analyse für ${strategy} fehlgeschlagen.`,
details: [
{ label: "Strategie", value: strategy },
{
label: "Fehler",
value: RAW_PAGESPEED_BYTES_SUMMARY,
},
],
});
continue;
}
const rawStorageId = await ctx.storage.store(
new Blob([rawJson], { type: "application/json" }),
);
const normalized = normalizePageSpeedResult({
strategy,
sourceUrl,
raw,
});
await ctx.runMutation(internal.pageSpeed.persistPageSpeedResult, {
leadId: started.lead._id,
...(started.auditId ? { auditId: started.auditId } : {}),
runId: args.runId,
strategy,
status: "succeeded",
sourceUrl,
finalUrl: normalized.finalUrl,
rawStorageId,
fetchedAt,
normalized: toPersistedPageSpeedNormalizedResult(normalized),
});
await ctx.runMutation(api.runs.appendEvent, {
runId: args.runId,
level: "info",
message: `PageSpeed-Analyse für ${strategy} abgeschlossen.`,
details: [{ label: "Strategie", value: strategy }],
});
succeededStrategies += 1;
} catch (error) {
const { errorType, errorSummary } = classifyPageSpeedFailure(
error,
apiKeyRaw,
);
failedStrategies += 1;
await ctx.runMutation(internal.pageSpeed.persistPageSpeedResult, {
leadId: started.lead._id,
...(started.auditId ? { auditId: started.auditId } : {}),
runId: args.runId,
strategy,
status: "failed",
sourceUrl,
errorType,
errorSummary,
fetchedAt,
});
await ctx.runMutation(api.runs.appendEvent, {
runId: args.runId,
level: "warning",
message: `PageSpeed-Analyse für ${strategy} fehlgeschlagen.`,
details: [
{ label: "Strategie", value: strategy },
{ label: "Fehler", value: errorSummary },
],
});
}
}
const status = succeededStrategies > 0 ? "succeeded" : "failed";
const errors = failedStrategies;
await ctx.runMutation(internal.pageSpeed.finishPageSpeedAuditRun, {
runId: args.runId,
status,
errors,
errorSummary:
status === "failed" && errors > 0
? "Ein oder mehrere PageSpeed-Strategien konnten nicht ausgeführt werden."
: undefined,
});
return args.runId;
} catch (error) {
const { errorSummary } = classifyPageSpeedFailure(error, apiKeyRaw);
await ctx.runMutation(internal.pageSpeed.finishPageSpeedAuditRun, {
runId: args.runId,
status: "failed",
errors: Math.max(1, failedStrategies),
errorSummary,
});
await ctx.runMutation(api.runs.appendEvent, {
runId: args.runId,
level: "error",
message: "PageSpeed-Analyse fehlgeschlagen.",
details: [{ label: "Fehler", value: errorSummary, source: "pagespeed_action" }],
});
return null;
}
},
});

View File

@@ -95,6 +95,22 @@ const runEventLevel = v.union(
...RUN_EVENT_LEVELS.map((level) => v.literal(level)),
);
const screenshotViewport = v.union(v.literal("desktop"), v.literal("mobile"));
const pageSpeedStrategy = v.union(
v.literal("mobile"),
v.literal("desktop"),
);
const pageSpeedResultStatus = v.union(
v.literal("succeeded"),
v.literal("failed"),
);
const pageSpeedErrorType = v.union(
v.literal("quota"),
v.literal("timeout"),
v.literal("unavailable"),
v.literal("invalid_url"),
v.literal("api_error"),
v.literal("unknown"),
);
const settingsValue = v.union(v.string(), v.number(), v.boolean(), v.null());
const auditMetricSummary = v.object({
performanceScore: v.optional(v.number()),
@@ -255,6 +271,48 @@ export default defineSchema({
.index("by_auditId_and_viewport", ["auditId", "viewport"])
.index("by_storageId", ["storageId"]),
pageSpeedResults: defineTable({
leadId: v.id("leads"),
auditId: v.optional(v.id("audits")),
runId: v.optional(v.id("agentRuns")),
strategy: pageSpeedStrategy,
status: pageSpeedResultStatus,
sourceUrl: v.string(),
finalUrl: v.optional(v.string()),
rawStorageId: v.optional(v.id("_storage")),
errorType: v.optional(pageSpeedErrorType),
errorSummary: v.optional(v.string()),
fetchedAt: v.number(),
createdAt: v.number(),
normalized: v.optional(
v.object({
scores: v.optional(
v.object({
performance: v.optional(v.number()),
accessibility: v.optional(v.number()),
bestPractices: v.optional(v.number()),
seo: v.optional(v.number()),
}),
),
metrics: v.optional(
v.object({
firstContentfulPaintMs: v.optional(v.number()),
largestContentfulPaintMs: v.optional(v.number()),
cumulativeLayoutShift: v.optional(v.number()),
totalBlockingTimeMs: v.optional(v.number()),
speedIndexMs: v.optional(v.number()),
}),
),
opportunities: v.optional(v.array(v.string())),
implications: v.optional(v.array(v.string())),
}),
),
})
.index("by_leadId", ["leadId"])
.index("by_runId", ["runId"])
.index("by_auditId", ["auditId"])
.index("by_leadId_and_strategy", ["leadId", "strategy"]),
websiteCrawlPages: defineTable({
leadId: v.id("leads"),
runId: v.optional(v.id("agentRuns")),

View File

@@ -433,6 +433,27 @@ export const processLeadEnrichment = internalAction({
const rootUrl = normalizeCrawlUrl(started.lead.websiteUrl);
if (!rootUrl) {
try {
await ctx.runMutation(internal.pageSpeed.queueLeadPageSpeedAudit, {
leadId: started.lead._id,
parentRunId: runId,
});
} catch (pageSpeedQueueError) {
await ctx.runMutation(api.runs.appendEvent, {
runId,
level: "warning",
message: "PageSpeed-Analyse konnte nicht in die Warteschlange gesetzt werden.",
details: [
{ label: "Lead", value: started.lead._id },
{
label: "Fehler",
value: messageFromError(pageSpeedQueueError),
source: "pagespeed_queue",
},
],
});
}
await ctx.runMutation(internal.websiteEnrichment.finishLeadEnrichmentRun, {
runId,
status: "failed",
@@ -665,6 +686,27 @@ export const processLeadEnrichment = internalAction({
});
}
try {
await ctx.runMutation(internal.pageSpeed.queueLeadPageSpeedAudit, {
leadId: started.lead._id,
parentRunId: runId,
});
} catch (pageSpeedQueueError) {
await ctx.runMutation(api.runs.appendEvent, {
runId,
level: "warning",
message: "PageSpeed-Analyse konnte nicht in die Warteschlange gesetzt werden.",
details: [
{ label: "Lead", value: started.lead._id },
{
label: "Fehler",
value: messageFromError(pageSpeedQueueError),
source: "pagespeed_queue",
},
],
});
}
await ctx.runMutation(internal.websiteEnrichment.finishLeadEnrichmentRun, {
runId,
status: "succeeded",
@@ -681,6 +723,7 @@ export const processLeadEnrichment = internalAction({
});
return runId;
} catch (error) {
const errorSummary = messageFromError(error);
@@ -702,6 +745,26 @@ export const processLeadEnrichment = internalAction({
});
if (started) {
try {
await ctx.runMutation(internal.pageSpeed.queueLeadPageSpeedAudit, {
leadId: started.lead._id,
parentRunId: runId,
});
} catch (pageSpeedQueueError) {
await ctx.runMutation(api.runs.appendEvent, {
runId,
level: "warning",
message: "PageSpeed-Analyse konnte nicht in die Warteschlange gesetzt werden.",
details: [
{ label: "Lead", value: started.lead._id },
{
label: "Fehler",
value: messageFromError(pageSpeedQueueError),
source: "pagespeed_queue",
},
],
});
}
await ctx.runMutation(internal.websiteEnrichment.patchLeadFromWebsiteEnrichment, {
leadId: started.lead._id,
currentContactStatus: started.lead.contactStatus,

View File

@@ -0,0 +1,544 @@
export type PageSpeedStrategy = "mobile" | "desktop";
export type PageSpeedAuditResultStatus = "succeeded" | "failed";
export type PageSpeedAuditErrorType =
| "quota"
| "timeout"
| "unavailable"
| "invalid_url"
| "api_error"
| "unknown";
export type PageSpeedAuditScores = {
performance?: number;
accessibility?: number;
bestPractices?: number;
seo?: number;
};
export type PageSpeedAuditMetrics = {
firstContentfulPaintMs?: number;
largestContentfulPaintMs?: number;
cumulativeLayoutShift?: number;
totalBlockingTimeMs?: number;
speedIndexMs?: number;
};
export type PageSpeedAuditNormalized = {
metrics?: PageSpeedAuditMetrics;
scores?: PageSpeedAuditScores;
opportunities?: string[];
implications?: string[];
};
export type PageSpeedMinimalAuditResult = {
strategy: PageSpeedStrategy;
status: PageSpeedAuditResultStatus;
sourceUrl: string;
finalUrl?: string;
normalized?: PageSpeedAuditNormalized;
errorType?: PageSpeedAuditErrorType;
errorSummary?: string;
};
export type PageSpeedAuditInputs = {
technicalSignals: string[];
customerImplications: string[];
internalNotes: string[];
};
type FailureContext = Readonly<{
status: string;
sourceUrl: string;
strategy: PageSpeedStrategy;
errorType?: PageSpeedAuditErrorType;
errorSummary?: string;
}>;
const CUSTOMER_IMPLICATION_LIMIT = 8;
const TECHNICAL_SIGNAL_LIMIT = 8;
const INTERNAL_NOTE_LIMIT = 6;
const SCORE_WORD_PATTERN =
/\bscore\b/i;
const SCORE_NUMBER_PATTERN =
/\b0?\.\d+\b|\b1(?:\.0+)?\b|\b[2-9]\d*\b/;
const RAW_STORAGE_PATTERN =
/\braw\s*storage\s*id\b/i;
const PAGE_SPEED_PATTERN =
/\bpagespeed\b/i;
const LIGHTHOUSE_PATTERN =
/\blighthouse\b/i;
const URL_PATTERN =
/\b(?:https?:\/\/|www\.)[^\s<>"']+/i;
const MARKUP_PATTERN =
/<[^>]+>/;
const JSON_BRACKET_PATTERN =
/\{[^}]*\}|\[[^\]]*\]/;
const SUSPICIOUS_MACHINE_TOKEN_PATTERN =
/\b[a-z\d_-]{24,}\b/i;
const PUBLIC_MACHINE_KEYWORDS_PATTERN =
/\b(?:raw\s*storage\s*id|rawstorageid|lighthouseresult|lighthouse|pagespeed|score)\b/i;
function toTrimmedText(value: unknown): string {
if (typeof value !== "string") {
return "";
}
return value.replace(/\s+/g, " ").trim();
}
function containsUntrustedPublicText(value: string): boolean {
if (URL_PATTERN.test(value)) {
return true;
}
if (MARKUP_PATTERN.test(value)) {
return true;
}
if (JSON_BRACKET_PATTERN.test(value)) {
return true;
}
if (PUBLIC_MACHINE_KEYWORDS_PATTERN.test(value)) {
return true;
}
if (SUSPICIOUS_MACHINE_TOKEN_PATTERN.test(value)) {
return true;
}
return false;
}
function isLikelyPlainGermanSentence(value: string): boolean {
if (!/[a-zäöüÄÖÜß]/i.test(value)) {
return false;
}
if (value.length > 500) {
return false;
}
return true;
}
function stripPublicText(value: string): string {
let text = toTrimmedText(value);
if (!text) {
return "";
}
if (containsUntrustedPublicText(text)) {
return "";
}
if (RAW_STORAGE_PATTERN.test(text) || PAGE_SPEED_PATTERN.test(text) || LIGHTHOUSE_PATTERN.test(text)) {
return "";
}
text = text.replace(/\b0?\.\d+\b/g, "");
text = text.replace(/\d+/g, "");
text = text.trim().replace(/\s{2,}/g, " ");
text = text.replace(/^[:\s]+/, "");
text = text.trim();
if (!isLikelyPlainGermanSentence(text)) {
return "";
}
if (!text) {
return "";
}
if (SUSPICIOUS_MACHINE_TOKEN_PATTERN.test(text)) {
return "";
}
if (/[<{[\]}]/.test(text)) {
return "";
}
if (PAGE_SPEED_PATTERN.test(text) || LIGHTHOUSE_PATTERN.test(text)) {
return "";
}
text = text.replace(/\s{2,}/g, " ").trim();
return text;
}
function stripInternalText(value: string): string {
let text = toTrimmedText(value);
if (!text) {
return "";
}
if (RAW_STORAGE_PATTERN.test(text)) {
return "";
}
if (URL_PATTERN.test(text)) {
return "";
}
if (MARKUP_PATTERN.test(text)) {
return "";
}
if (JSON_BRACKET_PATTERN.test(text)) {
return "";
}
text = text.replace(/^\s*score\s*[:\-]?\s*\d+(?:\.\d+)?\s*/i, "");
text = text.replace(/\{\s*[^}]*\}\s*/g, "");
text = text.replace(/\[[^\]]*\]\s*/g, "");
text = text.replace(SCORE_WORD_PATTERN, "");
text = text.replace(SCORE_NUMBER_PATTERN, "");
text = text.replace(/\b\d+(?:\.\d+)?\b/g, "");
text = text.replace(/^[:\s]+/, "");
text = text.trim().replace(/\s{2,}/g, " ");
return text;
}
function addUniqueCapped(
bucket: string[],
text: string,
max: number,
sanitize: (value: string) => string = stripPublicText,
): void {
const candidate = sanitize(text);
if (!candidate) {
return;
}
const normalized = candidate.toLowerCase().replace(/\s+/g, " ");
const duplicate = bucket.some(
(existing) =>
existing.toLowerCase().replace(/\s+/g, " ") === normalized,
);
if (!duplicate && bucket.length < max) {
bucket.push(candidate);
}
}
function hasMetricGap(
mobileValue: number | undefined,
desktopValue: number | undefined,
significantFactor = 1.25,
): boolean {
if (mobileValue === undefined || desktopValue === undefined) {
return false;
}
if (mobileValue <= desktopValue) {
return false;
}
if (desktopValue <= 0) {
return true;
}
return mobileValue >= desktopValue * significantFactor;
}
function addMobileWorseMessage(
mobile: PageSpeedAuditMetrics | undefined,
desktop: PageSpeedAuditMetrics | undefined,
technicalSignals: string[],
customerImplications: string[],
) {
if (!mobile || !desktop) {
return;
}
const fcpGap = hasMetricGap(
mobile.firstContentfulPaintMs,
desktop.firstContentfulPaintMs,
);
const lcpGap = hasMetricGap(
mobile.largestContentfulPaintMs,
desktop.largestContentfulPaintMs,
);
const tbtGap = hasMetricGap(
mobile.totalBlockingTimeMs,
desktop.totalBlockingTimeMs,
1.35,
);
const speedGap = hasMetricGap(
mobile.speedIndexMs,
desktop.speedIndexMs,
1.25,
);
const clsGap = hasMetricGap(
mobile.cumulativeLayoutShift,
desktop.cumulativeLayoutShift,
1.2,
);
if (!(fcpGap || lcpGap || tbtGap || speedGap || clsGap)) {
return;
}
const gapSentence =
"Die mobile Version ist deutlich langsamer als die Desktop-Variante.";
const mobileFirstSentence =
"Auf Mobilgeraten verlieren Kunden dadurch frueher den ersten Eindruck.";
addUniqueCapped(technicalSignals, gapSentence, TECHNICAL_SIGNAL_LIMIT);
addUniqueCapped(technicalSignals, mobileFirstSentence, TECHNICAL_SIGNAL_LIMIT);
addUniqueCapped(
customerImplications,
"Die mobile Version ist deutlich langsamer als die Desktop-Variante.",
CUSTOMER_IMPLICATION_LIMIT,
);
addUniqueCapped(
customerImplications,
"Kunden auf dem Telefon warten laenger und brechen den Erstkontakt schneller ab.",
CUSTOMER_IMPLICATION_LIMIT,
);
}
function addScoreBasedSignals(
scores: PageSpeedAuditScores | undefined,
technicalSignals: string[],
customerImplications: string[],
) {
if (!scores) {
return;
}
if ((scores.accessibility ?? 1) < 0.9) {
addUniqueCapped(
technicalSignals,
"Barrierefreiheit und Bedienbarkeit sollten fuer alle Nutzerinnen und Nutzer verbessert werden.",
TECHNICAL_SIGNAL_LIMIT,
);
addUniqueCapped(
customerImplications,
"Einfacher zugang und bessere Bedienbarkeit helfen mehr Interessenten zu erreichen.",
CUSTOMER_IMPLICATION_LIMIT,
);
}
if ((scores.seo ?? 1) < 0.9) {
addUniqueCapped(
technicalSignals,
"Technische Signale deuten auf reduzierte lokale Auffindbarkeit hin.",
TECHNICAL_SIGNAL_LIMIT,
);
addUniqueCapped(
customerImplications,
"Lokale Sichtbarkeit kann dadurch bei Neukundenanfragen sinken.",
CUSTOMER_IMPLICATION_LIMIT,
);
}
if ((scores.performance ?? 1) < 0.9) {
addUniqueCapped(
customerImplications,
"Wahrnehmbare Wartezeiten auf der Seite koennen das Vertrauen in den Auftritt mindern.",
CUSTOMER_IMPLICATION_LIMIT,
);
}
}
function addMetricSignals(
metrics: PageSpeedAuditMetrics | undefined,
technicalSignals: string[],
customerImplications: string[],
) {
if (!metrics) {
return;
}
if ((metrics.firstContentfulPaintMs ?? 0) > 2500) {
addUniqueCapped(
technicalSignals,
"Erster sichtbarer Inhalt erscheint deutlich verzoegert.",
TECHNICAL_SIGNAL_LIMIT,
);
addUniqueCapped(
customerImplications,
"Der erste sichtbare Inhalt erscheint spuetbar zu langsam.",
CUSTOMER_IMPLICATION_LIMIT,
);
}
if ((metrics.largestContentfulPaintMs ?? 0) > 4200) {
addUniqueCapped(
technicalSignals,
"Das wichtigste Inhaltselement wird stark verzoegert sichtbar.",
TECHNICAL_SIGNAL_LIMIT,
);
addUniqueCapped(
customerImplications,
"Wichtige Inhalte erscheinen zu spaet, was den ersten Eindruck schwaecht.",
CUSTOMER_IMPLICATION_LIMIT,
);
}
if ((metrics.totalBlockingTimeMs ?? 0) > 300) {
addUniqueCapped(
technicalSignals,
"Interaktion und Reaktionszeit sind stark beeintraechtigt.",
TECHNICAL_SIGNAL_LIMIT,
);
addUniqueCapped(
customerImplications,
"Bedienaktionen wirken traege und fuehren schneller zu Abbruechen.",
CUSTOMER_IMPLICATION_LIMIT,
);
}
if ((metrics.speedIndexMs ?? 0) > 3500) {
addUniqueCapped(
technicalSignals,
"Die visuelle Komplettierung der Seite verzoegert sich deutlich.",
TECHNICAL_SIGNAL_LIMIT,
);
addUniqueCapped(
customerImplications,
"Die Seite wirkt insgesamt schleppend aufgebaut und reduziert die Nutzungsbereitschaft.",
CUSTOMER_IMPLICATION_LIMIT,
);
}
if ((metrics.cumulativeLayoutShift ?? 0) > 0.1) {
addUniqueCapped(
technicalSignals,
"Instabile Layout-Werte weisen auf Spruenge in der Seitendarstellung hin.",
TECHNICAL_SIGNAL_LIMIT,
);
addUniqueCapped(
customerImplications,
"Elemente, die beim Laden verschieben, wirken unruhig und schwaechen Vertrauen.",
CUSTOMER_IMPLICATION_LIMIT,
);
}
}
function addFailureNote(input: FailureContext, internalNotes: string[]) {
if (input.errorType === "quota") {
addUniqueCapped(
internalNotes,
"Die Abfrage wurde wegen Quota-Limit abgebrochen.",
INTERNAL_NOTE_LIMIT,
stripInternalText,
);
return;
}
if (input.errorType === "unavailable") {
addUniqueCapped(
internalNotes,
"Die Zielseite war nicht erreichbar.",
INTERNAL_NOTE_LIMIT,
stripInternalText,
);
return;
}
if (input.errorType === "timeout") {
addUniqueCapped(
internalNotes,
"Der Aufruf wurde wegen Timeout beendet.",
INTERNAL_NOTE_LIMIT,
stripInternalText,
);
return;
}
const base =
input.errorType === "invalid_url"
? "Die Zieladresse wurde als ungueltig bewertet"
: "Der Lauf wurde mit technischem Fehler abgeschlossen";
const summary = stripInternalText(input.errorSummary || "");
const full = summary ? `${base}: ${summary}` : base;
addUniqueCapped(internalNotes, full, INTERNAL_NOTE_LIMIT, stripInternalText);
}
export function assertNoPublicPageSpeedScores(value: unknown): boolean {
const lines = Array.isArray(value) ? value : [value];
for (const line of lines) {
if (typeof line !== "string" || !line.trim()) {
continue;
}
if (containsUntrustedPublicText(line)) {
return false;
}
const asString = String(line);
if (/\bscore\b/i.test(asString) || /\b\d+\b/.test(asString) || /\b\d+\.\d+\b/.test(asString)) {
return false;
}
}
return true;
}
export function buildPageSpeedAuditInputs(
results: readonly PageSpeedMinimalAuditResult[],
): PageSpeedAuditInputs {
const technicalSignals: string[] = [];
const customerImplications: string[] = [];
const internalNotes: string[] = [];
const list = Array.isArray(results) ? results : [];
let mobileResult: PageSpeedMinimalAuditResult | undefined;
let desktopResult: PageSpeedMinimalAuditResult | undefined;
for (const result of list) {
if (result.status === "succeeded") {
const normalized = result.normalized ?? {};
if (result.strategy === "mobile") {
mobileResult = result;
} else {
desktopResult = result;
}
for (const implication of normalized.implications ?? []) {
addUniqueCapped(customerImplications, implication, CUSTOMER_IMPLICATION_LIMIT);
}
for (const opportunity of normalized.opportunities ?? []) {
addUniqueCapped(
technicalSignals,
`Moegliche Optimierung: ${opportunity}`,
TECHNICAL_SIGNAL_LIMIT,
);
}
addMetricSignals(normalized.metrics, technicalSignals, customerImplications);
addScoreBasedSignals(
normalized.scores,
technicalSignals,
customerImplications,
);
continue;
}
addFailureNote(result, internalNotes);
}
addMobileWorseMessage(
mobileResult?.normalized?.metrics,
desktopResult?.normalized?.metrics,
technicalSignals,
customerImplications,
);
return {
technicalSignals,
customerImplications: customerImplications.slice(
0,
CUSTOMER_IMPLICATION_LIMIT,
),
internalNotes: internalNotes.slice(0, INTERNAL_NOTE_LIMIT),
};
}

544
lib/pagespeed-insights.ts Normal file
View File

@@ -0,0 +1,544 @@
export type PageSpeedStrategy = "mobile" | "desktop";
export type PageSpeedErrorType =
| "quota"
| "timeout"
| "unavailable"
| "invalid_url"
| "api_error"
| "unknown";
export type PageSpeedScores = {
performance?: number;
accessibility?: number;
bestPractices?: number;
seo?: number;
};
export type PageSpeedMetrics = {
firstContentfulPaintMs?: number;
largestContentfulPaintMs?: number;
cumulativeLayoutShift?: number;
totalBlockingTimeMs?: number;
speedIndexMs?: number;
};
export type PageSpeedNormalizedResult = {
strategy: PageSpeedStrategy;
sourceUrl: string;
finalUrl?: string;
analysisTimestamp?: string;
scores?: PageSpeedScores;
metrics: PageSpeedMetrics;
opportunities: string[];
implications: string[];
};
type ClassifiedError = {
errorType: PageSpeedErrorType;
message: string;
};
type FetchLike = (
input: string,
init: { signal?: AbortSignal } | undefined,
) => Promise<{
ok: boolean;
status: number;
json: () => Promise<unknown>;
}>;
const PAGESPEED_ENDPOINT =
"https://pagespeedonline.googleapis.com/pagespeedonline/v5/runPagespeed";
const DEFAULT_TIMEOUT_MS = 10_000;
function asRecord(value: unknown): Record<string, unknown> | null {
if (!value || typeof value !== "object") {
return null;
}
return value as Record<string, unknown>;
}
function asString(value: unknown): string | null {
if (typeof value !== "string") {
return null;
}
const trimmed = value.trim();
return trimmed.length > 0 ? trimmed : null;
}
function asNumber(value: unknown): number | null {
if (typeof value === "number" && Number.isFinite(value)) {
return value;
}
return null;
}
function safeToLower(value: unknown): string {
if (typeof value === "string") {
return value.toLowerCase();
}
if (value instanceof Error) {
return `${value.name} ${value.message}`.toLowerCase();
}
if (value == null) {
return "";
}
if (typeof value === "object") {
try {
return JSON.stringify(value).toLowerCase();
} catch {
return "";
}
}
return "";
}
function hasPattern(value: unknown, patterns: string[]): boolean {
const lower = safeToLower(value);
return patterns.some((pattern) => lower.includes(pattern));
}
function firstNonEmptyString(...values: unknown[]): string | null {
for (const value of values) {
const stringValue = asString(value);
if (stringValue) {
return stringValue;
}
}
return null;
}
function extractPageSpeedErrorMessage(body: unknown): string | null {
const bodyRecord = asRecord(body);
const error = asRecord(bodyRecord?.error);
const lighthouseResult = asRecord(bodyRecord?.lighthouseResult);
const runtimeError = asRecord(lighthouseResult?.runtimeError);
return firstNonEmptyString(
error?.message,
bodyRecord?.error_message,
runtimeError?.message,
);
}
function buildImpactStatements(
scores: PageSpeedScores,
metrics: PageSpeedMetrics,
) {
const implications: string[] = [];
if ((scores.performance ?? 1) < 0.9) {
implications.push(
"Die allgemeine Seitengeschwindigkeit wirkt noch deutlich verbesserungswürdig.",
);
}
if ((metrics.firstContentfulPaintMs ?? 0) > 2_000) {
implications.push(
"Besucher sehen den ersten sichtbaren Inhalt auf der Seite zu langsam.",
);
}
if ((metrics.largestContentfulPaintMs ?? 0) > 3_000) {
implications.push(
"Der wichtigste Inhalt wird erst verspätet vollständig sichtbar.",
);
}
if ((metrics.cumulativeLayoutShift ?? 0) > 0.1) {
implications.push(
"Inhalte springen beim Laden nach, was den wahrgenommenen Seitenkomfort mindert.",
);
}
if ((metrics.totalBlockingTimeMs ?? 0) > 300) {
implications.push(
"Lange Blockierungszeiten können das Bediengefühl auf der Seite merklich spürbar verlangsamen.",
);
}
if ((metrics.speedIndexMs ?? 0) > 3_500) {
implications.push(
"Der visuelle Seitenaufbau ist verzögert und die Wahrnehmung der Seitenqualität leidet.",
);
}
if ((scores.bestPractices ?? 1) < 0.85) {
implications.push(
"Es gibt mehrere technische Best-Practice-Punkte, die aktuell noch nachgebessert werden sollten.",
);
}
if ((scores.accessibility ?? 1) < 0.9) {
implications.push(
"Die Barrierefreiheit sollte verbessert werden, damit alle Nutzerinnen und Nutzer die Seite besser erreichen.",
);
}
return implications;
}
function normalizePageSpeedAnalysisTimestamp(raw: unknown): string | undefined {
const value = asString(
asRecord(raw)?.analysisUTCTimestamp ?? asRecord(raw)?.analysisTimestamp,
);
return value ?? undefined;
}
function normalizePageSpeedScores(lighthouseResult: Record<string, unknown>) {
const categories = asRecord(lighthouseResult.categories) ?? {};
const scores: PageSpeedScores = {};
const performance = asNumber(
asRecord(categories.performance)?.score,
);
if (performance !== null) {
scores.performance = performance;
}
const accessibility = asNumber(
asRecord(categories.accessibility)?.score,
);
if (accessibility !== null) {
scores.accessibility = accessibility;
}
const bestPractices = asNumber(
asRecord(categories["best-practices"])?.score,
);
if (bestPractices !== null) {
scores.bestPractices = bestPractices;
}
const seo = asNumber(asRecord(categories.seo)?.score);
if (seo !== null) {
scores.seo = seo;
}
return scores;
}
function normalizePageSpeedMetrics(audits: Record<string, unknown>) {
const metrics: PageSpeedMetrics = {};
const fcp = asNumber(asRecord(audits["first-contentful-paint"])?.numericValue);
if (fcp !== null) {
metrics.firstContentfulPaintMs = fcp;
}
const lcp = asNumber(asRecord(audits["largest-contentful-paint"])?.numericValue);
if (lcp !== null) {
metrics.largestContentfulPaintMs = lcp;
}
const cls = asNumber(asRecord(audits["cumulative-layout-shift"])?.numericValue);
if (cls !== null) {
metrics.cumulativeLayoutShift = cls;
}
const tbt = asNumber(asRecord(audits["total-blocking-time"])?.numericValue);
if (tbt !== null) {
metrics.totalBlockingTimeMs = tbt;
}
const speedIndex = asNumber(asRecord(audits["speed-index"])?.numericValue);
if (speedIndex !== null) {
metrics.speedIndexMs = speedIndex;
}
return metrics;
}
function formatSavingsHint(value: number) {
const rounded = Math.abs(Math.round(value));
if (rounded >= 1024) {
return `${Math.round(rounded / 1024)} MB`;
}
return `${rounded} ms`;
}
function normalizeOpportunities(audits: Record<string, unknown>) {
const opportunities: string[] = [];
for (const [id, rawAudit] of Object.entries(audits)) {
const audit = asRecord(rawAudit);
if (!audit) {
continue;
}
const details = asRecord(audit.details);
const type = asString(details?.type);
if (type !== "opportunity") {
continue;
}
const title = asString(audit.title) ?? id;
const savingsMs = asNumber(details?.overallSavingsMs);
const savingsBytes = asNumber(details?.overallSavingsBytes);
if (savingsMs !== null) {
opportunities.push(`${title}: ca. ${formatSavingsHint(savingsMs)} Einsparung möglich.`);
continue;
}
if (savingsBytes !== null) {
opportunities.push(`${title}: potenziell ${formatSavingsHint(savingsBytes)} weniger Last.`);
continue;
}
const score = asNumber(audit.score);
if (score !== null && score < 0.9) {
opportunities.push(`${title}: hier ist weiteres Optimierungspotenzial vorhanden.`);
}
}
return opportunities;
}
export function buildPageSpeedRequestUrl(input: {
url: string;
strategy: PageSpeedStrategy;
apiKey?: string | null;
locale?: string;
}): string {
const requestUrl = new URL(PAGESPEED_ENDPOINT);
requestUrl.searchParams.append("url", input.url);
requestUrl.searchParams.set("strategy", input.strategy);
requestUrl.searchParams.append("category", "performance");
requestUrl.searchParams.append("category", "accessibility");
requestUrl.searchParams.append("category", "best-practices");
requestUrl.searchParams.append("category", "seo");
requestUrl.searchParams.set("locale", input.locale ?? "de-DE");
if (asString(input.apiKey)) {
requestUrl.searchParams.set("key", input.apiKey as string);
}
return requestUrl.toString();
}
export function normalizePageSpeedResult(input: {
strategy: PageSpeedStrategy;
sourceUrl: string;
raw: unknown;
}): PageSpeedNormalizedResult {
const lighthouseResult = asRecord(asRecord(input.raw)?.lighthouseResult) ?? {};
const audits = asRecord(lighthouseResult.audits) ?? {};
const scores = normalizePageSpeedScores(lighthouseResult);
const metrics = normalizePageSpeedMetrics(audits);
const opportunities = normalizeOpportunities(audits);
const implications = buildImpactStatements(scores, metrics);
for (let i = implications.length - 1; i >= 0; i -= 1) {
if (implications[i] === "") {
implications.splice(i, 1);
}
}
const finalUrl = asString(lighthouseResult.finalUrl) ?? undefined;
const analysisTimestamp = normalizePageSpeedAnalysisTimestamp(input.raw);
const result: PageSpeedNormalizedResult = {
strategy: input.strategy,
sourceUrl: input.sourceUrl,
metrics,
opportunities,
implications,
};
if (finalUrl) {
result.finalUrl = finalUrl;
}
if (analysisTimestamp) {
result.analysisTimestamp = analysisTimestamp;
}
const hasAnyScore = Object.values(scores).some((value) => value !== undefined);
if (hasAnyScore) {
result.scores = scores;
}
return result;
}
export function classifyPageSpeedError(input: {
error?: unknown;
status?: number;
body?: unknown;
}): ClassifiedError {
const status = Number.isFinite(input?.status)
? Math.trunc(input.status as number)
: undefined;
const statusBodyText = [input?.error, input?.body, input?.status]
.map(safeToLower)
.join(" ");
const abortLike = input?.error instanceof DOMException
? input.error.name === "AbortError"
: false;
const errorMessage = safeToLower(
input?.error instanceof Error ? input.error.message : input?.error,
);
if (input?.error instanceof SyntaxError) {
return {
errorType: "api_error",
message: `PageSpeed-Antwort war kein gültiges JSON: ${errorMessage || "Unbekannt"}`,
};
}
if (abortLike || errorMessage.includes("abort") && errorMessage.includes("timeout")) {
return {
errorType: "timeout",
message: "PageSpeed-Anfrage wurde wegen Timeout abgebrochen.",
};
}
if (
hasPattern(statusBodyText, [
"429",
"quota",
"userratelimit",
"rate limit",
"user rate limit",
]) ||
status === 429) {
return {
errorType: "quota",
message: "PageSpeed-Anfrage wurde wegen API-Quota abgelehnt.",
};
}
if (
status === 404 ||
hasPattern(statusBodyText, [
"failed document",
"failed to fetch document",
"could not fetch document",
"unreachable document",
"could not fetch",
"not found",
])
) {
return {
errorType: "unavailable",
message: "Die analysierte Seite ist aktuell nicht erreichbar.",
};
}
if (
hasPattern(statusBodyText, [
"invalid url",
"invalid_url",
"bad url",
"malformed url",
"url parsing",
"unsupported url",
"missing required parameter: url",
])
) {
return {
errorType: "invalid_url",
message: "Die angegebene URL ist nicht valide für PageSpeed.",
};
}
if (status !== undefined && status >= 400 && status < 600) {
const apiMessage = extractPageSpeedErrorMessage(input?.body);
return {
errorType: "api_error",
message: apiMessage
? `PageSpeed-API lieferte einen Fehler: ${apiMessage}`
: "PageSpeed-API lieferte einen Fehler.",
};
}
return {
errorType: "unknown",
message: input?.error instanceof Error && input.error.message
? `Unbekannter Fehler beim PageSpeed-Zugriff: ${input.error.message}`
: "Unbekannter Fehler beim PageSpeed-Zugriff.",
};
};
function createPageSpeedError(classification: ClassifiedError): Error {
const error = new Error(classification.message);
return Object.assign(error, {
errorType: classification.errorType,
name: `PageSpeedError:${classification.errorType}`,
});
}
function isPageSpeedError(error: unknown): error is Error & { errorType: PageSpeedErrorType } {
return (
typeof error === "object" &&
error !== null &&
"errorType" in error &&
typeof (error as { errorType?: unknown }).errorType === "string"
);
}
async function parseResponseBody(
response: { json: () => Promise<unknown> },
swallowParseErrors: boolean,
): Promise<unknown> {
try {
return await response.json();
} catch (error) {
if (swallowParseErrors) {
return null;
}
throw error;
}
}
export async function fetchPageSpeedResult(input: {
url: string;
strategy: PageSpeedStrategy;
apiKey?: string | null;
timeoutMs?: number;
fetchImpl?: FetchLike;
}): Promise<unknown> {
const timeoutMs = input.timeoutMs ?? DEFAULT_TIMEOUT_MS;
const fetchImpl: FetchLike =
input.fetchImpl ??
((fetch as typeof globalThis.fetch) as unknown as FetchLike);
const requestUrl = buildPageSpeedRequestUrl({
url: input.url,
strategy: input.strategy,
apiKey: input.apiKey,
});
const controller = new AbortController();
const timer = setTimeout(() => {
controller.abort();
}, timeoutMs);
try {
const response = await fetchImpl(requestUrl, {
signal: controller.signal,
});
const body = await parseResponseBody(response, !response.ok);
if (!response.ok) {
const classification = classifyPageSpeedError({
status: response.status,
body,
});
throw createPageSpeedError(classification);
}
return body;
} catch (error) {
if (isPageSpeedError(error)) {
throw error;
}
const classification = classifyPageSpeedError({ error });
if (classification.errorType === "unknown") {
throw error;
}
throw createPageSpeedError(classification);
} finally {
clearTimeout(timer);
}
}

View File

@@ -0,0 +1,365 @@
import assert from "node:assert/strict";
import { existsSync, readFileSync } from "node:fs";
import path from "node:path";
import test from "node:test";
import ts from "typescript";
const actionPath = path.join(process.cwd(), "convex", "pageSpeedAction.ts");
const actionSource = existsSync(actionPath) ? readFileSync(actionPath, "utf8") : "";
const actionSourceFile = ts.createSourceFile(
"pageSpeedAction.ts",
actionSource,
ts.ScriptTarget.ES2022,
true,
ts.ScriptKind.TS,
);
function getExportedConstNames(file: ts.SourceFile) {
const names = new Set<string>();
const visit = (node: ts.Node) => {
if (ts.isVariableStatement(node)) {
const isExported = node.modifiers?.some(
(mod) => mod.kind === ts.SyntaxKind.ExportKeyword,
);
if (!isExported) {
ts.forEachChild(node, visit);
return;
}
const isConst = node.declarationList.flags & ts.NodeFlags.Const;
if (!isConst) {
ts.forEachChild(node, visit);
return;
}
for (const declaration of node.declarationList.declarations) {
if (ts.isIdentifier(declaration.name)) {
names.add(declaration.name.text);
}
}
}
ts.forEachChild(node, visit);
};
ts.forEachChild(file, visit);
return names;
}
function hasPattern(source: string, pattern: RegExp) {
return pattern.test(source);
}
test("pageSpeedAction module exists and runs in Node runtime", () => {
assert.equal(existsSync(actionPath), true, "pageSpeedAction.ts should exist");
assert.equal(
hasPattern(actionSource, /^"use node";/m),
true,
"pageSpeedAction.ts should use Node runtime",
);
});
test("pageSpeedAction exports processPageSpeedAudit as internalAction with runId validator", () => {
const exports = getExportedConstNames(actionSourceFile);
assert.equal(
exports.has("processPageSpeedAudit"),
true,
"processPageSpeedAudit should be exported",
);
assert.equal(
hasPattern(
actionSource,
/processPageSpeedAudit\s*=\s*internalAction\(\s*{\s*args:\s*{[\s\S]*runId:\s*v\.id\(\s*["']agentRuns["']\s*\)/,
),
true,
"processPageSpeedAudit should be an internalAction with runId validator",
);
});
test("pageSpeedAction starts and finishes run mutations", () => {
assert.equal(
hasPattern(
actionSource,
/internal\.pageSpeed\.startPageSpeedAuditRun/,
),
true,
"Action should call internal.pageSpeed.startPageSpeedAuditRun",
);
assert.equal(
hasPattern(
actionSource,
/internal\.pageSpeed\.finishPageSpeedAuditRun/,
),
true,
"Action should call internal.pageSpeed.finishPageSpeedAuditRun",
);
});
test("pageSpeedAction has action-level guard to fail whole run on unexpected errors", () => {
assert.equal(
hasPattern(
actionSource,
/try\s*{[\s\S]*?await ctx\.runMutation\(internal\.pageSpeed\.startPageSpeedAuditRun,\s*{[\s\S]*?}\);\s*[\s\S]*?for\s*\(\s*(?:const|let)\s+strategy\s+of\s+STRATEGIES[\s\S]*?\}\s*catch \(error\)\s*{[\s\S]*classifyPageSpeedFailure\(error,\s*apiKeyRaw\)[\s\S]*?internal\.pageSpeed\.finishPageSpeedAuditRun[\s\S]*status:\s*["']failed["']/,
),
true,
"Action should wrap run lifecycle in an outer try/catch that finalizes the run as failed.",
);
});
test("pageSpeedAction enforces raw payload size guard before storage", () => {
assert.equal(
hasPattern(actionSource, /MAX_RAW_PAGESPEED_BYTES/),
true,
"Action should declare MAX_RAW_PAGESPEED_BYTES constant.",
);
assert.equal(
hasPattern(
actionSource,
/new TextEncoder\(\)\.encode\(rawJson\)\.byteLength/,
),
true,
"Action should calculate raw JSON byte length before attempting to store.",
);
assert.equal(
hasPattern(
actionSource,
/rawJsonBytes\s*>\s*MAX_RAW_PAGESPEED_BYTES[\s\S]*?errorType:\s*["']api_error["'][\s\S]*?errorSummary:\s*RAW_PAGESPEED_BYTES_SUMMARY/,
),
true,
"Oversized raw payloads should be rejected as api_error with the required German summary.",
);
assert.equal(
hasPattern(
actionSource,
/new Blob\(\[rawJson\][\s\S]*type:\s*["']application\/json["']/,
),
true,
"Normal raw payloads should still be stored as application/json blobs.",
);
assert.equal(
hasPattern(
actionSource,
/if\s*\(\s*rawJsonBytes\s*>\s*MAX_RAW_PAGESPEED_BYTES[\s\S]*?}\s*[\s\S]*?continue;[\s\S]*?await ctx\.storage\.store\(/,
),
true,
"Raw payload storage must be skipped for oversized payloads.",
);
});
test("pageSpeedAction runs both strategies and catches per-strategy errors", () => {
assert.equal(
hasPattern(
actionSource,
/["']mobile["'][\s\S]*["']desktop["']/,
),
true,
"Action should include both page speed strategies: mobile and desktop",
);
assert.equal(
hasPattern(
actionSource,
/for\s*\(\s*(?:const|let)\s+strategy\s+of[\s\S]*?\)\s*{[\s\S]*?try[\s\S]*?catch\s*\([^)]*\)[\s\S]*?}/,
),
true,
"Action should catch errors inside per-strategy loop",
);
});
test("pageSpeedAction stores and persists results and writes events", () => {
assert.equal(
hasPattern(
actionSource,
/ctx\.storage\.store\([\s\S]*new Blob\(\[\s*rawJson\s*[\s\S]*type:\s*["']application\/json["']/,
),
true,
"Raw PageSpeed payload should be stored via ctx.storage.store with application/json blob",
);
assert.equal(
hasPattern(
actionSource,
/internal\.pageSpeed\.persistPageSpeedResult[\s\S]*status:\s*["']succeeded["']/,
),
true,
"Action should persist succeeded PageSpeed results",
);
assert.equal(
hasPattern(
actionSource,
/internal\.pageSpeed\.persistPageSpeedResult[\s\S]*status:\s*["']failed["']/,
),
true,
"Action should persist failed PageSpeed results",
);
assert.equal(
/api\.runs\.appendEvent,\s*{\s*[\s\S]*runId:\s*args\.runId,\s*[\s\S]*level:\s*["']info["']/.test(
actionSource,
),
true,
"Action should append info events for successful strategy results",
);
assert.equal(
/level:\s*["']warning["']/.test(actionSource) ||
/level:\s*["']error["']/.test(actionSource),
true,
"Action should append warning/error events for failed strategy results",
);
});
test("pageSpeedAction strips non-persisted normalized fields before Convex mutation", () => {
assert.equal(
hasPattern(actionSource, /toPersistedPageSpeedNormalizedResult/),
true,
"Action should map normalized PageSpeed output into the Convex validator shape.",
);
assert.equal(
hasPattern(
actionSource,
/normalized:\s*toPersistedPageSpeedNormalizedResult\(normalized\)/,
),
true,
"Action should persist only the normalized subset accepted by convex/pageSpeed.ts.",
);
assert.equal(
hasPattern(
actionSource,
/normalized,\s*[\r\n]/,
),
false,
"Action should not pass the full normalized object with strategy/sourceUrl/finalUrl/analysisTimestamp.",
);
});
test("pageSpeedAction does not expose API key in event messages/details", () => {
assert.equal(
hasPattern(
actionSource,
/api\.runs\.appendEvent[\s\S]{0,500}PAGESPEED_API_KEY/,
),
false,
"Action events should not include raw PAGESPEED_API_KEY",
);
});
test("pageSpeedAction imports PageSpeed helpers from lib/pagespeed-insights", () => {
const hasLibImport =
actionSource.includes("fetchPageSpeedResult") &&
actionSource.includes("normalizePageSpeedResult") &&
actionSource.includes("classifyPageSpeedError") &&
actionSource.includes('from "../lib/pagespeed-insights"');
assert.equal(hasLibImport, true, "Action should import required PageSpeed helper functions");
});
test("pageSpeedAction exposes configurable PageSpeed timeout via env var", () => {
assert.equal(
hasPattern(
actionSource,
/PAGESPEED_TIMEOUT_MS/
),
true,
"PageSpeed timeout should be configurable with PAGESPEED_TIMEOUT_MS.",
);
assert.equal(
hasPattern(actionSource, /DEFAULT_PAGESPEED_TIMEOUT_MS\s*=\s*60_000/),
true,
"PageSpeed timeout default should be 60_000ms.",
);
assert.equal(
hasPattern(actionSource, /MIN_PAGESPEED_TIMEOUT_MS\s*=\s*10_000/),
true,
"PageSpeed timeout min clamp should be 10_000ms.",
);
assert.equal(
hasPattern(actionSource, /MAX_PAGESPEED_TIMEOUT_MS\s*=\s*120_000/),
true,
"PageSpeed timeout max clamp should be 120_000ms.",
);
});
test("pageSpeedAction parses and clamps timeout values before use", () => {
assert.equal(
hasPattern(
actionSource,
/function parsePageSpeedTimeoutMs\(\s*raw:\s*string \| undefined\)/,
),
true,
"Action should parse PAGESPEED_TIMEOUT_MS via a dedicated helper.",
);
assert.equal(
hasPattern(actionSource, /Number\.parseInt\(raw,\s*10\)/),
true,
"Action should parse env timeout values as decimal integers.",
);
assert.equal(
hasPattern(actionSource, /Number\.isFinite\(/),
true,
"Invalid timeout values should be handled via Number.isFinite validation.",
);
assert.equal(
hasPattern(
actionSource,
/Math\.max\(\s*parsed,\s*MIN_PAGESPEED_TIMEOUT_MS\s*\)/,
),
true,
"Timeout below min should be clamped.",
);
assert.equal(
hasPattern(
actionSource,
/Math\.min\(\s*[\s\S]*MAX_PAGESPEED_TIMEOUT_MS\s*,\s*\)/,
),
true,
"Timeout above max should be clamped.",
);
});
test("pageSpeedAction passes resolved timeout to PageSpeed fetch calls", () => {
assert.equal(
hasPattern(
actionSource,
/const timeoutMs = resolvePageSpeedTimeoutMs\(\)/,
),
true,
"Action should resolve timeout once from helper and pass it to fetch calls.",
);
assert.equal(
hasPattern(
actionSource,
/fetchPageSpeedResult\([\s\S]{0,250}timeoutMs,/
),
true,
"Action should pass resolved timeout to fetchPageSpeedResult.",
);
assert.equal(
hasPattern(
actionSource,
/const timeoutMs\s*=\s*10_000/,
),
false,
"Timeout should not be hardcoded to 10_000ms in processPageSpeedAudit.",
);
});

View File

@@ -0,0 +1,191 @@
import assert from "node:assert/strict";
import { existsSync, readFileSync } from "node:fs";
import path from "node:path";
import test from "node:test";
import ts from "typescript";
const auditInputsPath = path.join(process.cwd(), "convex", "auditInputs.ts");
const auditInputsSource = existsSync(auditInputsPath)
? readFileSync(auditInputsPath, "utf8")
: "";
const sourceFile = ts.createSourceFile(
"auditInputs.ts",
auditInputsSource,
ts.ScriptTarget.ES2022,
true,
ts.ScriptKind.TS,
);
function getExportedConstNames(file: ts.SourceFile) {
const names = new Set<string>();
const visit = (node: ts.Node) => {
if (ts.isVariableStatement(node)) {
const isExported = node.modifiers?.some(
(mod) => mod.kind === ts.SyntaxKind.ExportKeyword,
);
if (!isExported) {
ts.forEachChild(node, visit);
return;
}
const isConst = node.declarationList.flags & ts.NodeFlags.Const;
if (!isConst) {
ts.forEachChild(node, visit);
return;
}
for (const declaration of node.declarationList.declarations) {
if (ts.isIdentifier(declaration.name)) {
names.add(declaration.name.text);
}
}
}
ts.forEachChild(node, visit);
};
ts.forEachChild(file, visit);
return names;
}
function hasPattern(source: string, pattern: RegExp) {
return pattern.test(source);
}
function extractExportSource(name: string) {
const marker = `export const ${name} = `;
const declarationIndex = auditInputsSource.indexOf(marker);
assert.notEqual(declarationIndex, -1, `Expected declaration for ${name}`);
const openBraceIndex = auditInputsSource.indexOf("{", declarationIndex);
let depth = 0;
let end = -1;
for (let index = openBraceIndex; index < auditInputsSource.length; index += 1) {
const char = auditInputsSource[index];
if (char === "{") {
depth += 1;
} else if (char === "}") {
depth -= 1;
if (depth === 0) {
end = index;
break;
}
}
}
assert.notEqual(end, -1, `Expected balanced braces for ${name}`);
return auditInputsSource.slice(openBraceIndex, end + 1);
}
test("auditInputs module exists and exports pageSpeed input translator query", () => {
assert.equal(
existsSync(auditInputsPath),
true,
"convex/auditInputs.ts should be present",
);
const exports = getExportedConstNames(sourceFile);
assert.equal(
exports.has("getPageSpeedAuditInputs"),
true,
"auditInputs module should export getPageSpeedAuditInputs",
);
});
test("auditInputs module calls buildPageSpeedAuditInputs for lead/audit PageSpeed results", () => {
const querySource = extractExportSource("getPageSpeedAuditInputs");
assert.equal(
hasPattern(
auditInputsSource,
/buildPageSpeedAuditInputs[\s\S]*?from\s*["']\.\.\/lib\/pagespeed-audit-input["']/,
),
true,
"auditInputs should import buildPageSpeedAuditInputs",
);
assert.equal(
hasPattern(querySource, /buildPageSpeedAuditInputs\(results\.map\(/),
true,
"auditInputs should call buildPageSpeedAuditInputs",
);
});
test("auditInputs query fetches stored pageSpeedResults by lead or audit", () => {
const querySource = extractExportSource("getPageSpeedAuditInputs");
assert.equal(
hasPattern(
auditInputsSource,
/getPageSpeedAuditInputs\s*=\s*internalQuery\s*\(/,
),
true,
"getPageSpeedAuditInputs should be registered as an internal query",
);
assert.equal(
hasPattern(
querySource,
/handler\s*:\s*async\s*\(/,
),
true,
"getPageSpeedAuditInputs source block should include an async handler",
);
assert.equal(
hasPattern(
auditInputsSource,
/ctx\.db[\s\S]*?\.query\([\s\S]*?["']pageSpeedResults["']\s*\)/,
),
true,
"auditInputs should read from pageSpeedResults table",
);
assert.equal(
hasPattern(
auditInputsSource,
/withIndex\(["']by_auditId["'][\s\S]*?eq\([\s\S]*?auditId[\s\S]*?\)/,
),
true,
"auditInputs should support audit-scoped PageSpeed results",
);
assert.equal(
hasPattern(
auditInputsSource,
/withIndex\(["']by_leadId["'][\s\S]*?eq\([\s\S]*?leadId[\s\S]*?\)/,
),
true,
"auditInputs should support lead-scoped PageSpeed results",
);
});
test("auditInputs returns only plain-language prompt fields", () => {
const querySource = extractExportSource("getPageSpeedAuditInputs");
assert.equal(
hasPattern(
querySource,
/technicalSignals\s*:\s*string\[\][\s\S]*customerImplications\s*:\s*string\[\][\s\S]*internalNotes\s*:\s*string\[\]/,
),
true,
"Return type should expose only technicalSignals, customerImplications, and internalNotes",
);
const returnConstruction = querySource.match(
/buildPageSpeedAuditInputs\([\s\S]*?\);/,
);
assert.notEqual(
returnConstruction,
null,
"auditInputs should return buildPageSpeedAuditInputs output",
);
assert.equal(
/rawStorageId/.test(returnConstruction?.[0] ?? ""),
false,
"Returned fields must not include rawStorageId",
);
assert.equal(
/\bscores\b/.test(returnConstruction?.[0] ?? ""),
false,
"Returned fields must not include scores",
);
});

View File

@@ -0,0 +1,301 @@
import assert from "node:assert/strict";
import test from "node:test";
import {
assertNoPublicPageSpeedScores,
buildPageSpeedAuditInputs,
type PageSpeedMinimalAuditResult,
} from "../lib/pagespeed-audit-input";
const MOBILE_AND_DESKTOP_FIXTURES: PageSpeedMinimalAuditResult[] = [
{
strategy: "mobile",
status: "succeeded",
sourceUrl: "https://example.com",
normalized: {
metrics: {
firstContentfulPaintMs: 3200,
largestContentfulPaintMs: 5000,
cumulativeLayoutShift: 0.2,
},
implications: [
"Score 0.42: Der erste sichtbare Inhalt erscheint zu langsam.",
"Die Seite zeigt das Hauptbild zu langsam.",
"Die Inhalte verschieben sich beim Laden.",
],
opportunities: [
"Nicht verwendetes CSS kann entfernt werden.",
"Bilder ohne passende Komprimierung koennen verzichtet werden.",
],
},
},
{
strategy: "desktop",
status: "succeeded",
sourceUrl: "https://example.com",
normalized: {
metrics: {
firstContentfulPaintMs: 1200,
largestContentfulPaintMs: 2200,
cumulativeLayoutShift: 0.04,
},
implications: [
"Die Seite zeigt das Hauptbild zu langsam.",
"Inhalte werden beim Laden sauber angezeigt.",
],
opportunities: ["Serverantworten sind stabil.", "Inhalte werden gestaffelt geladen."],
},
},
];
test("buildPageSpeedAuditInputs converts normalized implications into German customer impact statements", () => {
const actual = buildPageSpeedAuditInputs(MOBILE_AND_DESKTOP_FIXTURES);
assert.equal(actual.customerImplications.length > 0, true);
assert.equal(
actual.customerImplications.includes(
"Die Seite zeigt das Hauptbild zu langsam.",
),
true,
);
assert.equal(
actual.customerImplications.includes("Die Inhalte verschieben sich beim Laden."),
true,
);
assert.equal(
actual.customerImplications.some((line) => /mobile/i.test(line)),
true,
"Customer implications should include a mobile-centric statement.",
);
assert.equal(
assertNoPublicPageSpeedScores(actual.customerImplications),
true,
"Customer implications must not contain score-like values.",
);
assert.equal(
assertNoPublicPageSpeedScores(actual.technicalSignals),
true,
"Technical signals must not contain score-like values.",
);
});
test("buildPageSpeedAuditInputs detects meaningful mobile performance gaps versus desktop", () => {
const actual = buildPageSpeedAuditInputs(MOBILE_AND_DESKTOP_FIXTURES);
assert.equal(
actual.customerImplications.some((line) =>
/mobile/i.test(line) &&
/deutlich|spurbar|signifikant|langsamer/.test(line) &&
/desktop/i.test(line),
),
true,
);
});
test("buildPageSpeedAuditInputs keeps quota/api/unavailable failures in internal notes only", () => {
const actual = buildPageSpeedAuditInputs([
{
strategy: "mobile",
status: "failed",
sourceUrl: "https://bad.example",
errorType: "quota",
errorSummary: "API quota has been exceeded for this host.",
},
{
strategy: "desktop",
status: "failed",
sourceUrl: "https://bad2.example",
errorType: "unavailable",
errorSummary: "Page not reachable at the moment.",
},
{
strategy: "mobile",
status: "failed",
sourceUrl: "https://bad3.example",
errorType: "api_error",
errorSummary: "Lighthouse processing failed due to API timeout.",
},
{
strategy: "desktop",
status: "succeeded",
sourceUrl: "https://example.com",
normalized: {
implications: ["Die wichtigste Information wird zu langsam sichtbar."],
},
},
]);
assert.equal(
actual.customerImplications.some((line) => /quota|unavailable|timeout|api/i.test(line)),
false,
);
assert.equal(
actual.technicalSignals.some((line) => /quota|unavailable|timeout|api/i.test(line)),
false,
);
assert.equal(actual.internalNotes.length >= 3, true);
assert.equal(actual.internalNotes.some((line) => /quota/i.test(line)), true);
assert.equal(actual.internalNotes.some((line) => /not reachable|unreachable|erreich|timeout/i.test(line)), true);
assert.equal(actual.internalNotes.some((line) => /api/i.test(line)), true);
});
test("buildPageSpeedAuditInputs strips score-like and raw strings from public outputs", () => {
const actual = buildPageSpeedAuditInputs([
{
strategy: "mobile",
status: "succeeded",
sourceUrl: "https://example.com",
normalized: {
implications: [
"Score 0.42: FCP is high.",
"rawStorageId: file_123",
"Lighthouse category performance is present.",
"Die Seite laedt in 3.2 Sekunden.",
],
opportunities: [
"Ein { \"score\": 0.91 } kann optimiert werden.",
"redundante CSS Dateien.",
],
},
},
]);
assert.equal(assertNoPublicPageSpeedScores(actual.customerImplications), true);
assert.equal(assertNoPublicPageSpeedScores(actual.technicalSignals), true);
assert.equal(
actual.customerImplications.every((line) => !/\d/.test(line)),
true,
);
});
test("buildPageSpeedAuditInputs strips URLs, markup, JSON-like payloads, and machine-like words from public outputs", () => {
const actual = buildPageSpeedAuditInputs([
{
strategy: "mobile",
status: "succeeded",
sourceUrl: "https://example.com",
normalized: {
implications: [
"Weitere Infos findest du in https://example.com/details",
"Das <strong>Element</strong> lädt stabil.",
"{ \"pagespeed\": 0.84, \"lighthouseResult\": {} }",
"[\"rawStorageId\":\"id-0123456789abcdef0123456789\"]",
"rawStorageId: run_2026_0001",
"lighthouseResult suggests a bad candidate.",
"Die Seite laedt insgesamt spuertbar langsam.",
],
opportunities: [
"Moeglichkeit: <img src=\"x\" />",
"Pagespeed Score should not appear.",
"[{\"audit\":\"speed\"}]",
"Reduziere ungenutzte JavaScript-Dateien.",
"A longMachineToken_0123456789abcdef0123456789 to test filtering.",
],
},
},
]);
assert.equal(assertNoPublicPageSpeedScores(actual.customerImplications), true);
assert.equal(assertNoPublicPageSpeedScores(actual.technicalSignals), true);
assert.equal(actual.customerImplications.includes("Die Seite laedt insgesamt spuertbar langsam."), true);
assert.equal(
actual.technicalSignals.some((line) => /unused|reduziere|javascript/i.test(line)),
true,
);
assert.equal(
actual.customerImplications.every((line) => !/\bhttps?:\/\/|rawstorageid|lighthouseresult|pagespeed|score|<|>|\\{|\\}|\\[|\\]/i.test(line)),
true,
);
assert.equal(
actual.technicalSignals.every((line) => !/\bhttps?:\/\/|rawstorageid|lighthouseresult|pagespeed|score|<|>|\\{|\\}|\\[|\\]/i.test(line)),
true,
);
});
test("buildPageSpeedAuditInputs keeps failure categories in internal notes while removing URLs and JSON fragments", () => {
const actual = buildPageSpeedAuditInputs([
{
strategy: "mobile",
status: "failed",
sourceUrl: "https://example.com/audit?x=1",
errorType: "api_error",
errorSummary:
"PageSpeed API failed: { \"lighthouseResult\": {\"code\":\"timeout\"}, \"rawStorageId\": \"abc123\" }",
},
{
strategy: "desktop",
status: "succeeded",
sourceUrl: "https://example.com",
normalized: {
implications: [
"Die Seite laedt spuerbar schneller auf Desktop.",
],
},
},
]);
assert.equal(actual.internalNotes.length >= 1, true);
assert.equal(
actual.internalNotes.every(
(line) =>
!/https?:\/\//i.test(line) &&
!/\{|\}|\[|\]/i.test(line) &&
!/rawstorageid|lighthouseresult/i.test(line),
),
true,
);
assert.equal(
actual.internalNotes.some((line) => /api|technisch/i.test(line)),
true,
);
});
test("buildPageSpeedAuditInputs deduplicates and caps output lists", () => {
const manyImplications = Array.from({ length: 12 }, (_, index) => [
"Die Seite ist zu langsam.",
"Die Seite ist zu langsam.",
`Implication ${index}`,
"Wichtige Inhalte sind nicht sofort sichtbar.",
"Wichtige Inhalte sind nicht sofort sichtbar.",
]).flat();
const manyOpportunities = Array.from({ length: 12 }, (_, index) => [
"Komprimieren Sie Bilder.",
`Opportunity ${index}`,
"Komprimieren Sie Bilder.",
"Inhalte werden nachgeladen.",
]).flat();
const actual = buildPageSpeedAuditInputs([
{
strategy: "mobile",
status: "succeeded",
sourceUrl: "https://example.com",
normalized: {
implications: manyImplications,
opportunities: manyOpportunities,
},
},
...Array.from({ length: 10 }, (_, index) => ({
strategy: "desktop" as const,
status: "failed" as const,
sourceUrl: `https://example.com/${index}`,
errorType: "api_error" as const,
errorSummary: `Run ${String.fromCharCode(97 + (index % 26))} had internal problem.`,
})),
]);
assert.equal(actual.customerImplications.length <= 8, true);
assert.equal(actual.technicalSignals.length <= 8, true);
assert.equal(actual.customerImplications.length > 0, true);
assert.equal(actual.technicalSignals.length > 0, true);
assert.equal(actual.internalNotes.length, 6);
assert.equal(
new Set(actual.customerImplications).size,
actual.customerImplications.length,
);
assert.equal(
new Set(actual.technicalSignals).size,
actual.technicalSignals.length,
);
});

View File

@@ -0,0 +1,343 @@
import assert from "node:assert/strict";
import test from "node:test";
import {
buildPageSpeedRequestUrl,
classifyPageSpeedError,
fetchPageSpeedResult,
normalizePageSpeedResult,
type PageSpeedErrorType,
} from "../lib/pagespeed-insights";
const MOBILE_RAW_FIXTURE = {
analysisUTCTimestamp: "2026-06-01T08:00:00.000Z",
lighthouseResult: {
finalUrl: "https://example.com/mobil",
categories: {
performance: { score: 0.81 },
accessibility: { score: 0.96 },
"best-practices": { score: 0.77 },
seo: { score: 0.94 },
},
audits: {
"first-contentful-paint": {
title: "Erste Inhalte",
numericValue: 2850,
score: 0.76,
},
"largest-contentful-paint": {
title: "Größtes Inhaltselement",
numericValue: 4300,
score: 0.6,
},
"cumulative-layout-shift": {
title: "Layout-Verschiebung",
numericValue: 0.14,
},
"total-blocking-time": {
title: "Blockierende Skripte",
numericValue: 420,
},
"speed-index": {
title: "Speed Index",
numericValue: 4800,
},
"unused-css-rules": {
title: "Nicht verwendetes CSS",
score: 0.4,
details: {
type: "opportunity",
overallSavingsMs: 380,
},
},
"modern-image-formats": {
title: "Moderne Bildformate",
score: 0.55,
details: {
type: "opportunity",
overallSavingsBytes: 52000,
},
},
},
},
};
const DESKTOP_RAW_FIXTURE = {
analysisUTCTimestamp: "2026-06-01T09:00:00.000Z",
lighthouseResult: {
finalUrl: "https://example.com/desktop",
categories: {
performance: { score: 0.93 },
accessibility: { score: 0.99 },
"best-practices": { score: 0.85 },
seo: { score: 0.97 },
},
audits: {
"first-contentful-paint": {
title: "Erste Inhalte",
numericValue: 1800,
score: 0.91,
},
"largest-contentful-paint": {
title: "Größtes Inhaltselement",
numericValue: 3600,
score: 0.73,
},
"cumulative-layout-shift": {
title: "Layout-Verschiebung",
numericValue: 0.08,
},
"total-blocking-time": {
title: "Blockierende Skripte",
numericValue: 310,
},
"speed-index": {
title: "Speed Index",
numericValue: 3800,
},
"offscreen-images": {
title: "Außenseiten-Bilder",
score: 0.9,
details: {
type: "opportunity",
overallSavingsMs: 210,
},
},
},
},
};
test("buildPageSpeedRequestUrl includes required query params and repeated categories", () => {
const url = buildPageSpeedRequestUrl({
url: "https://example.com/landing?x=1&y=2",
strategy: "mobile",
locale: "de-DE",
apiKey: "super-secret",
});
const parsed = new URL(url);
assert.equal(
parsed.origin + parsed.pathname,
"https://pagespeedonline.googleapis.com/pagespeedonline/v5/runPagespeed",
);
assert.equal(parsed.searchParams.get("url"), "https://example.com/landing?x=1&y=2");
assert.equal(parsed.searchParams.get("strategy"), "mobile");
assert.equal(parsed.searchParams.get("locale"), "de-DE");
assert.equal(parsed.searchParams.get("key"), "super-secret");
assert.deepEqual(parsed.searchParams.getAll("category"), [
"performance",
"accessibility",
"best-practices",
"seo",
]);
assert.equal(
parsed.search.includes("url=https%3A%2F%2Fexample.com%2Flanding%3Fx%3D1%26y%3D2"),
true,
"URL input should be encoded",
);
});
test("buildPageSpeedRequestUrl omits empty API keys", () => {
const url = buildPageSpeedRequestUrl({
url: "https://example.com",
strategy: "desktop",
apiKey: "",
locale: "de-DE",
});
const parsed = new URL(url);
assert.equal(parsed.searchParams.has("key"), false);
});
test("normalizePageSpeedResult maps mobile scores, metrics, and implications", () => {
const normalized = normalizePageSpeedResult({
strategy: "mobile",
sourceUrl: "https://example.com",
raw: MOBILE_RAW_FIXTURE,
});
assert.equal(normalized.strategy, "mobile");
assert.equal(normalized.sourceUrl, "https://example.com");
assert.equal(normalized.finalUrl, "https://example.com/mobil");
assert.equal(normalized.analysisTimestamp, "2026-06-01T08:00:00.000Z");
assert.equal(normalized.scores?.performance, 0.81);
assert.equal(normalized.scores?.accessibility, 0.96);
assert.equal(normalized.scores?.bestPractices, 0.77);
assert.equal(normalized.scores?.seo, 0.94);
assert.equal(normalized.metrics.firstContentfulPaintMs, 2850);
assert.equal(normalized.metrics.largestContentfulPaintMs, 4300);
assert.equal(normalized.metrics.cumulativeLayoutShift, 0.14);
assert.equal(normalized.metrics.totalBlockingTimeMs, 420);
assert.equal(normalized.metrics.speedIndexMs, 4800);
assert.equal(normalized.opportunities.length >= 1, true);
assert.equal(normalized.implications.length >= 2, true);
assert.equal(normalized.implications.some((text) => text.includes("Besucher")), true);
for (const implication of normalized.implications) {
assert.equal(
/score\s*\d+/i.test(implication),
false,
`Implication should not contain raw score text: ${implication}`,
);
assert.equal(implication.length > 0, true);
}
});
test("normalizePageSpeedResult maps desktop scores and metrics", () => {
const normalized = normalizePageSpeedResult({
strategy: "desktop",
sourceUrl: "https://example.com/landing",
raw: DESKTOP_RAW_FIXTURE,
});
assert.equal(normalized.strategy, "desktop");
assert.equal(normalized.sourceUrl, "https://example.com/landing");
assert.equal(normalized.finalUrl, "https://example.com/desktop");
assert.equal(normalized.analysisTimestamp, "2026-06-01T09:00:00.000Z");
assert.equal(normalized.scores?.performance, 0.93);
assert.equal(normalized.scores?.bestPractices, 0.85);
assert.equal(normalized.metrics.firstContentfulPaintMs, 1800);
assert.equal(normalized.metrics.speedIndexMs, 3800);
assert.equal(normalized.metrics.totalBlockingTimeMs, 310);
assert.equal(normalized.opportunities.length >= 1, true);
assert.equal(normalized.implications.length >= 2, true);
});
test("classifyPageSpeedError maps status and body signals", () => {
const quotaByStatus = classifyPageSpeedError({ status: 429 });
assert.equal(quotaByStatus.errorType, "quota");
const quotaByBody = classifyPageSpeedError({
status: 403,
body: { error: { errors: [{ reason: "userRateLimitExceeded" }] } },
});
assert.equal(quotaByBody.errorType, "quota");
const timeoutError = classifyPageSpeedError({
error: new DOMException("timed out", "AbortError"),
});
assert.equal(timeoutError.errorType, "timeout");
const unavailableByStatus = classifyPageSpeedError({ status: 404 });
assert.equal(unavailableByStatus.errorType, "unavailable");
const unavailableByBody = classifyPageSpeedError({
status: 500,
body: { error: { message: "Failed to fetch document from given URL" } },
});
assert.equal(unavailableByBody.errorType, "unavailable");
const invalidUrl = classifyPageSpeedError({
status: 400,
body: { error: { message: "Invalid URL: unsupported format" } },
});
assert.equal(invalidUrl.errorType, "invalid_url");
const apiError = classifyPageSpeedError({
status: 500,
body: { error: { message: "backend down" } },
});
assert.equal(apiError.errorType, "api_error");
assert.match(apiError.message, /backend down/);
});
test("classifyPageSpeedError returns unknown for non-classified cases", () => {
const classified = classifyPageSpeedError({
error: new Error("something odd"),
});
const errorType: PageSpeedErrorType = classified.errorType;
assert.equal(errorType, "unknown");
assert.match(classified.message, /something odd/);
});
test("fetchPageSpeedResult uses injected fetch and uses the built request URL", async () => {
const calls: string[] = [];
const fetchImpl = async (url: string) => {
calls.push(url);
return {
ok: true,
status: 200,
async json() {
return { ok: true };
},
} as Response;
};
const actual = await fetchPageSpeedResult({
url: "https://example.com/test?tracking=true",
strategy: "desktop",
apiKey: "secret-key",
fetchImpl,
});
assert.deepEqual(actual, { ok: true });
assert.equal(calls.length, 1);
const parsed = new URL(calls[0]);
assert.equal(parsed.searchParams.get("strategy"), "desktop");
assert.equal(parsed.searchParams.get("locale"), "de-DE");
assert.deepEqual(
parsed.searchParams.getAll("category"),
["performance", "accessibility", "best-practices", "seo"],
);
assert.equal(parsed.searchParams.get("key"), "secret-key");
});
test(
"fetchPageSpeedResult throws classified api_error when response.ok response has invalid JSON",
async () => {
const fetchImpl = async () =>
({
ok: true,
status: 200,
async json() {
throw new SyntaxError("Unexpected token <");
},
}) as unknown as Response;
let caughtError: unknown;
try {
await fetchPageSpeedResult({
url: "https://example.com/broken-json",
strategy: "mobile",
fetchImpl,
});
assert.fail("Expected fetchPageSpeedResult to throw");
} catch (error) {
caughtError = error;
}
assert.match(String((caughtError as Error).message), /Unexpected token </i);
assert.equal((caughtError as Error & { errorType?: string }).errorType, "api_error");
},
);
test("fetchPageSpeedResult preserves Google API error messages", async () => {
const fetchImpl = async () =>
({
ok: false,
status: 403,
async json() {
return {
error: {
code: 403,
message: "API key not valid. Please pass a valid API key.",
status: "PERMISSION_DENIED",
},
};
},
}) as unknown as Response;
let caughtError: unknown;
try {
await fetchPageSpeedResult({
url: "https://example.com/key-error",
strategy: "desktop",
fetchImpl,
});
assert.fail("Expected fetchPageSpeedResult to throw");
} catch (error) {
caughtError = error;
}
assert.equal((caughtError as Error & { errorType?: string }).errorType, "api_error");
assert.match(String((caughtError as Error).message), /API key not valid/);
});

View File

@@ -0,0 +1,242 @@
import assert from "node:assert/strict";
import { existsSync, readFileSync } from "node:fs";
import path from "node:path";
import test from "node:test";
import ts from "typescript";
const pageSpeedPath = path.join(process.cwd(), "convex", "pageSpeed.ts");
const pageSpeedSource = existsSync(pageSpeedPath)
? readFileSync(pageSpeedPath, "utf8")
: "";
const sourceFile = ts.createSourceFile(
"pageSpeed.ts",
pageSpeedSource,
ts.ScriptTarget.ES2022,
true,
ts.ScriptKind.TS,
);
function getExportedConstNames(file: ts.SourceFile) {
const names = new Set<string>();
const visit = (node: ts.Node) => {
if (ts.isVariableStatement(node)) {
const isExported = node.modifiers?.some(
(mod) => mod.kind === ts.SyntaxKind.ExportKeyword,
);
if (!isExported) {
ts.forEachChild(node, visit);
return;
}
const isConst = node.declarationList.flags & ts.NodeFlags.Const;
if (!isConst) {
ts.forEachChild(node, visit);
return;
}
for (const declaration of node.declarationList.declarations) {
if (ts.isIdentifier(declaration.name)) {
names.add(declaration.name.text);
}
}
}
ts.forEachChild(node, visit);
};
ts.forEachChild(file, visit);
return names;
}
function hasPattern(source: string, pattern: RegExp) {
return pattern.test(source);
}
function extractExportSource(name: string) {
const marker = `export const ${name} = `;
const declarationIndex = pageSpeedSource.indexOf(marker);
assert.notEqual(declarationIndex, -1, `Expected declaration for ${name}`);
const openBraceIndex = pageSpeedSource.indexOf("{", declarationIndex);
let depth = 0;
let end = -1;
for (let index = openBraceIndex; index < pageSpeedSource.length; index += 1) {
const char = pageSpeedSource[index];
if (char === "{") {
depth += 1;
} else if (char === "}") {
depth -= 1;
if (depth === 0) {
end = index;
break;
}
}
}
assert.notEqual(end, -1, `Expected balanced braces for ${name}`);
return pageSpeedSource.slice(openBraceIndex, end + 1);
}
test("pageSpeed module exports mutation contracts", () => {
assert.equal(existsSync(pageSpeedPath), true, "pageSpeed.ts should be present");
const exports = getExportedConstNames(sourceFile);
const required = [
"queueLeadPageSpeedAudit",
"startPageSpeedAuditRun",
"persistPageSpeedResult",
"finishPageSpeedAuditRun",
];
for (const exportName of required) {
assert.equal(exports.has(exportName), true, `Expected export: ${exportName}`);
}
});
test("pageSpeed module uses internalMutation for queue/start/persist/finish", () => {
for (const name of [
"queueLeadPageSpeedAudit",
"startPageSpeedAuditRun",
"persistPageSpeedResult",
"finishPageSpeedAuditRun",
]) {
assert.equal(
hasPattern(pageSpeedSource, new RegExp(`export const ${name} = internalMutation\\s*\\(`)),
true,
`${name} should be registered as internalMutation.`,
);
}
});
test("queueLeadPageSpeedAudit dedupes per lead and schedules pagespeed action", () => {
const queueSource = extractExportSource("queueLeadPageSpeedAudit");
assert.equal(
hasPattern(
queueSource,
/withIndex\(\s*"by_type_and_status_and_leadId"[\s\S]*?eq\("type",\s*"audit"\)[\s\S]*?eq\("status",\s*"pending"\)[\s\S]*?eq\("leadId",\s*args\.leadId\)/,
),
true,
"Queue should dedupe pending audit runs by type+status+leadId.",
);
assert.equal(
hasPattern(
queueSource,
/withIndex\(\s*"by_type_and_status_and_leadId"[\s\S]*?eq\("type",\s*"audit"\)[\s\S]*?eq\("status",\s*"running"\)[\s\S]*?eq\("leadId",\s*args\.leadId\)/,
),
true,
"Queue should dedupe running audit runs by type+status+leadId.",
);
assert.equal(
hasPattern(
queueSource,
/currentStep:\s*["']pagespeed_insights["']/,
),
true,
"Queued page speed runs should use currentStep pagespeed_insights.",
);
assert.equal(
hasPattern(
queueSource,
/ctx\.scheduler\.runAfter\(\s*0,\s*internal\.pageSpeedAction\.processPageSpeedAudit,\s*\{[\s\S]*?runId/,
),
true,
"queueLeadPageSpeedAudit must schedule internal.pageSpeedAction.processPageSpeedAudit with runAfter(0, ...).",
);
assert.equal(
hasPattern(
queueSource,
/PageSpeed-Analyse wurde in die Warteschlange gesetzt\./,
),
true,
"queueLeadPageSpeedAudit should emit queue-start event message.",
);
});
test("startPageSpeedAuditRun marks run as running and handles clear failures", () => {
const startSource = extractExportSource("startPageSpeedAuditRun");
assert.equal(
hasPattern(
startSource,
/run\.type\s*!==?\s*["']audit["']/,
),
true,
"start function should require audit run type.",
);
assert.equal(
hasPattern(startSource, /run\.status\s*!==?\s*["']pending["']/),
true,
"start function should require pending status.",
);
assert.equal(
hasPattern(
startSource,
/ctx\.db\.patch\(\s*args\.runId,\s*\{[\s\S]*status:\s*["']running["']/,
),
true,
"start function should set status running.",
);
assert.equal(
hasPattern(startSource, /currentStep:\s*["']pagespeed_insights["']/),
true,
"start function should set currentStep pagespeed_insights.",
);
assert.equal(
hasPattern(
startSource,
/!run\.leadId[\s\S]*status:\s*["']failed["']/,
),
true,
"start should fail and record missing leadId.",
);
assert.equal(
hasPattern(
startSource,
/!lead\.websiteUrl[\s\S]*status:\s*["']failed["']/,
),
true,
"start should fail and record missing website URL.",
);
assert.equal(
hasPattern(
startSource,
/message:\s*["'][^"']*konnte nicht gestartet werden[^"']*["']/i,
),
true,
"start should add clear failure events.",
);
});
test("persistPageSpeedResult writes pageSpeedResults table", () => {
const persistSource = pageSpeedSource
? extractExportSource("persistPageSpeedResult")
: "export const persistPageSpeedResult = {}";
assert.equal(
hasPattern(persistSource, /ctx\.db\.insert\(\s*["']pageSpeedResults["']/),
true,
"persistPageSpeedResult should insert into pageSpeedResults.",
);
});
test("finishPageSpeedAuditRun writes completion status and finishedAt", () => {
const finishSource = extractExportSource("finishPageSpeedAuditRun");
assert.equal(
hasPattern(finishSource, /ctx\.db\.patch\(\s*args\.runId,[\s\S]*?finishedAt:\s*now/),
true,
"finish function should set finishedAt.",
);
assert.equal(
hasPattern(
finishSource,
/counters:\s*\{\s*[\s\S]*?errors:\s*args\.errors\s*\?\?/,
),
true,
"finish function should update counters.",
);
assert.equal(
hasPattern(finishSource, /currentStep:\s*["']pagespeed_insights["']/),
true,
"finish function should set currentStep pagespeed_insights.",
);
});

View File

@@ -0,0 +1,226 @@
import assert from "node:assert/strict";
import { readFileSync } from "node:fs";
import { join } from "node:path";
import test from "node:test";
const schemaSource = readFileSync(
join(process.cwd(), "convex", "schema.ts"),
"utf8",
);
type ExactSetEquality<A, B> = [
Exclude<A, B>,
] extends [never]
? [Exclude<B, A>] extends [never]
? true
: false
: false;
type AssertPageSpeedStrategy = "mobile" | "desktop";
type AssertPageSpeedResultStatus = "succeeded" | "failed";
type AssertPageSpeedErrorType =
| "quota"
| "timeout"
| "unavailable"
| "invalid_url"
| "api_error"
| "unknown";
type PageSpeedStrategyParity = ExactSetEquality<
AssertPageSpeedStrategy,
("mobile" | "desktop")
>;
type PageSpeedResultStatusParity = ExactSetEquality<
AssertPageSpeedResultStatus,
"succeeded" | "failed"
>;
type PageSpeedErrorTypeParity = ExactSetEquality<
AssertPageSpeedErrorType,
"quota" | "timeout" | "unavailable" | "invalid_url" | "api_error" | "unknown"
>;
const _assertPageSpeedStrategyParity: PageSpeedStrategyParity = true;
const _assertPageSpeedResultStatusParity: PageSpeedResultStatusParity = true;
const _assertPageSpeedErrorTypeParity: PageSpeedErrorTypeParity = true;
function extractTableSection(tableName: string) {
const marker = `${tableName}: defineTable({`;
const markerIndex = schemaSource.indexOf(marker);
assert.notEqual(
markerIndex,
-1,
`Expected schema table definition for ${tableName}.`,
);
const objectStart = schemaSource.indexOf("{", markerIndex);
let depth = 0;
let objectEnd = -1;
for (let i = objectStart; i < schemaSource.length; i += 1) {
if (schemaSource[i] === "{") {
depth += 1;
} else if (schemaSource[i] === "}") {
depth -= 1;
if (depth === 0) {
objectEnd = i;
break;
}
}
}
assert.notEqual(objectEnd, -1, `Could not parse schema object for ${tableName}.`);
const remainder = schemaSource.slice(objectEnd + 1);
const nextTableMatch = remainder.match(/^\s*[a-zA-Z_][\w]*:\s*defineTable\(/m);
const sectionEnd =
nextTableMatch === null ? schemaSource.length : objectEnd + 1 + nextTableMatch.index!;
const section = schemaSource.slice(markerIndex, sectionEnd);
const objectBlock = schemaSource.slice(markerIndex, objectEnd + 1);
return { section, objectBlock };
}
function assertHas(pattern: RegExp, source: string, message: string) {
assert.equal(pattern.test(source), true, message);
}
test("PageSpeed validator unions are declared", () => {
assert.equal(_assertPageSpeedStrategyParity, true);
assert.equal(_assertPageSpeedResultStatusParity, true);
assert.equal(_assertPageSpeedErrorTypeParity, true);
assertHas(
/const\s+pageSpeedStrategy\s*=\s*v\.union\(\s*[\s\S]*v\.literal\(\s*["']mobile["']\s*\)\s*,\s*[\s\S]*v\.literal\(\s*["']desktop["']\s*\)[\s\S]*\)/,
schemaSource,
"Schema should define pageSpeedStrategy union with mobile and desktop.",
);
assertHas(
/const\s+pageSpeedResultStatus\s*=\s*v\.union\(\s*[\s\S]*v\.literal\(\s*["']succeeded["']\s*\)\s*,\s*[\s\S]*v\.literal\(\s*["']failed["']\s*\)[\s\S]*\)/,
schemaSource,
"Schema should define pageSpeedResultStatus union with succeeded and failed.",
);
assertHas(
/const\s+pageSpeedErrorType\s*=\s*v\.union\(\s*[\s\S]*v\.literal\(\s*["']quota["']\s*\)\s*,\s*[\s\S]*v\.literal\(\s*["']timeout["']\s*\)\s*,\s*[\s\S]*v\.literal\(\s*["']unavailable["']\s*\)\s*,\s*[\s\S]*v\.literal\(\s*["']invalid_url["']\s*\)\s*,\s*[\s\S]*v\.literal\(\s*["']api_error["']\s*\)\s*,\s*[\s\S]*v\.literal\(\s*["']unknown["']\s*\)[\s\S]*\)/,
schemaSource,
"Schema should define pageSpeedErrorType union with all declared values.",
);
});
test("pageSpeedResults table has contract fields and indexes", () => {
const { section, objectBlock } = extractTableSection("pageSpeedResults");
assertHas(
/leadId:\s*v\.id\(["']leads["']\)/,
objectBlock,
"pageSpeedResults.leadId should be required lead id.",
);
assertHas(
/auditId:\s*v\.optional\(\s*v\.id\(["']audits["']\)\s*\)/,
objectBlock,
"pageSpeedResults.auditId should be optional audit id.",
);
assertHas(
/runId:\s*v\.optional\(\s*v\.id\(["']agentRuns["']\)\s*\)/,
objectBlock,
"pageSpeedResults.runId should be optional run id.",
);
assertHas(
/strategy:\s*pageSpeedStrategy/,
objectBlock,
"pageSpeedResults.strategy should use pageSpeedStrategy validator.",
);
assertHas(
/status:\s*pageSpeedResultStatus/,
objectBlock,
"pageSpeedResults.status should use pageSpeedResultStatus validator.",
);
assertHas(
/sourceUrl:\s*v\.string\(\)/,
objectBlock,
"pageSpeedResults.sourceUrl should be required.",
);
assertHas(
/finalUrl:\s*v\.optional\(\s*v\.string\(\)\s*\)/,
objectBlock,
"pageSpeedResults.finalUrl should be optional string.",
);
assertHas(
/rawStorageId:\s*v\.optional\(\s*v\.id\(["']_storage["']\)\s*\)/,
objectBlock,
"pageSpeedResults.rawStorageId should be optional storage id.",
);
assertHas(
/errorType:\s*v\.optional\(\s*pageSpeedErrorType\s*\)/,
objectBlock,
"pageSpeedResults.errorType should be optional error type.",
);
assertHas(
/errorSummary:\s*v\.optional\(\s*v\.string\(\)\s*\)/,
objectBlock,
"pageSpeedResults.errorSummary should be optional.",
);
assertHas(
/fetchedAt:\s*v\.number\(\)/,
objectBlock,
"pageSpeedResults.fetchedAt should be required.",
);
assertHas(
/createdAt:\s*v\.number\(\)/,
objectBlock,
"pageSpeedResults.createdAt should be required.",
);
assertHas(
/scores:\s*v\.optional\(\s*v\.object\([\s\S]*?performance:\s*v\.optional\(v\.number\(\)\)[\s\S]*?accessibility:\s*v\.optional\(v\.number\(\)\)[\s\S]*?bestPractices:\s*v\.optional\(v\.number\(\)\)[\s\S]*?seo:\s*v\.optional\(v\.number\(\)\)[\s\S]*?\)\s*\)/,
objectBlock,
"pageSpeedResults.normalized.scores should include expected keys.",
);
assertHas(
/metrics:\s*v\.optional\(\s*v\.object\([\s\S]*?firstContentfulPaintMs:\s*v\.optional\(v\.number\(\)\)[\s\S]*?largestContentfulPaintMs:\s*v\.optional\(v\.number\(\)\)[\s\S]*?cumulativeLayoutShift:\s*v\.optional\(v\.number\(\)\)[\s\S]*?totalBlockingTimeMs:\s*v\.optional\(v\.number\(\)\)[\s\S]*?speedIndexMs:\s*v\.optional\(v\.number\(\)\)[\s\S]*?\)\s*\)/,
objectBlock,
"pageSpeedResults.normalized.metrics should include expected keys.",
);
assertHas(
/opportunities:\s*v\.optional\(\s*v\.array\(v\.string\(\)\)\s*\)/,
objectBlock,
"pageSpeedResults.normalized.opportunities should be optional string array.",
);
assertHas(
/implications:\s*v\.optional\(\s*v\.array\(v\.string\(\)\)\s*\)/,
objectBlock,
"pageSpeedResults.normalized.implications should be optional string array.",
);
assertHas(
/index\("by_leadId",\s*\["leadId"\]\)/,
section,
"pageSpeedResults should have by_leadId index.",
);
assertHas(
/index\("by_runId",\s*\["runId"\]\)/,
section,
"pageSpeedResults should have by_runId index.",
);
assertHas(
/index\("by_auditId",\s*\["auditId"\]\)/,
section,
"pageSpeedResults should have by_auditId index.",
);
assertHas(
/index\("by_leadId_and_strategy",\s*\["leadId",\s*"strategy"\]\)/,
section,
"pageSpeedResults should have by_leadId_and_strategy index.",
);
});
test("audits should not include public raw PageSpeed/Lighthouse JSON fields", () => {
const { objectBlock } = extractTableSection("audits");
const hasPublicRawJson = /raw.*pagespeed|pagespeed.*raw|raw.*lighthouse|lighthouse.*raw/i.test(
objectBlock,
);
assert.equal(
hasPublicRawJson,
false,
"audits should not expose raw PageSpeed/Lighthouse JSON fields.",
);
});

View File

@@ -70,6 +70,32 @@ function hasPattern(source: string, pattern: RegExp) {
return pattern.test(source);
}
function extractExportSource(source: string, name: string) {
const marker = `export const ${name} = `;
const declarationIndex = source.indexOf(marker);
assert.notEqual(declarationIndex, -1, `Expected declaration for ${name}`);
const openBraceIndex = source.indexOf("{", declarationIndex);
let depth = 0;
let end = -1;
for (let index = openBraceIndex; index < source.length; index += 1) {
const char = source[index];
if (char === "{") {
depth += 1;
} else if (char === "}") {
depth -= 1;
if (depth === 0) {
end = index;
break;
}
}
}
assert.notEqual(end, -1, `Expected balanced braces for ${name}`);
return source.slice(openBraceIndex, end + 1);
}
test("website enrichment mutation module exists and has runtime assertions", () => {
assert.equal(
existsSync(websiteEnrichmentPath),
@@ -531,3 +557,123 @@ test("website enrichment enforces TASK-8 crawler limits and runtime timeboxes",
"Default max crawl page count should be 5",
);
});
test("processLeadEnrichment schedules PageSpeed audit jobs after successful enrichment", () => {
const processBody = extractExportSource(actionSource, "processLeadEnrichment");
const persistIndex = processBody.indexOf(
"internal.websiteEnrichment.persistLeadEnrichmentResult",
);
const queueIndex = processBody.indexOf(
"internal.pageSpeed.queueLeadPageSpeedAudit",
persistIndex,
);
const finishIndex = processBody.indexOf(
"internal.websiteEnrichment.finishLeadEnrichmentRun",
persistIndex,
);
assert.notEqual(queueIndex, -1, "processLeadEnrichment should queue PageSpeed audits");
assert.notEqual(persistIndex, -1, "processLeadEnrichment should persist website enrichment result");
assert.notEqual(finishIndex, -1, "processLeadEnrichment should finish enrichment run");
assert.equal(
hasPattern(
processBody,
/runMutation\(\s*internal\.pageSpeed\.queueLeadPageSpeedAudit[\s\S]*leadId:\s*started\.lead\._id[\s\S]*parentRunId:\s*runId[\s\S]*\)/,
),
true,
"Queue call should pass lead ID and parent run ID",
);
assert.equal(queueIndex > persistIndex, true, "PageSpeed queueing should happen after persistence");
assert.equal(queueIndex < finishIndex, true, "PageSpeed queueing should happen before success finish");
});
test("processLeadEnrichment records warning on PageSpeed queue failure and continues", () => {
const processBody = extractExportSource(actionSource, "processLeadEnrichment");
assert.equal(
hasPattern(
processBody,
/try\s*\{[\s\S]*internal\.pageSpeed\.queueLeadPageSpeedAudit[\s\S]*\}\s*catch\s*\([^)]*\)\s*\{[\s\S]*api\.runs\.appendEvent[\s\S]*level:\s*"warning"/,
),
true,
"Queueing PageSpeed should be wrapped in warning-safe try/catch",
);
assert.equal(
hasPattern(
processBody,
/PageSpeed-Analyse konnte nicht in die Warteschlange gesetzt werden\./,
),
true,
"Warning event should describe queue failure",
);
});
test("processLeadEnrichment regression: queue PageSpeed on invalid URL failure when started lead exists", () => {
const processBody = extractExportSource(actionSource, "processLeadEnrichment");
const invalidUrlStart = processBody.indexOf("if (!rootUrl)");
assert.notEqual(invalidUrlStart, -1, "Invalid URL guard should exist");
const invalidUrlReturnNull = processBody.indexOf("return null;", invalidUrlStart);
assert.notEqual(
invalidUrlReturnNull,
-1,
"Invalid URL branch should return null",
);
const queueCallInInvalidUrl = processBody.indexOf(
"internal.pageSpeed.queueLeadPageSpeedAudit",
invalidUrlStart,
);
assert.equal(
queueCallInInvalidUrl > invalidUrlStart && queueCallInInvalidUrl < invalidUrlReturnNull,
true,
"Invalid URL failure path should queue PageSpeed before returning.",
);
const invalidUrlBranch = processBody.slice(invalidUrlStart, invalidUrlReturnNull);
assert.equal(
hasPattern(
invalidUrlBranch,
/leadId:\s*started\.lead\._id[\s\S]*?parentRunId:\s*runId/,
),
true,
"Invalid URL queue payload should use started.lead._id and parentRunId runId.",
);
});
test("processLeadEnrichment regression: queue PageSpeed in fatal catch path with started lead", () => {
const processBody = extractExportSource(actionSource, "processLeadEnrichment");
const outerCatchStart = processBody.lastIndexOf("catch (error)");
assert.notEqual(outerCatchStart, -1, "Outer catch block should exist");
const startedGuard = processBody.indexOf("if (started)", outerCatchStart);
assert.notEqual(startedGuard, -1, "Outer catch should guard lead patch by started check.");
const catchReturnNull = processBody.indexOf("return null;", outerCatchStart);
assert.notEqual(
catchReturnNull,
-1,
"Outer catch should return null on unrecoverable errors.",
);
const queueCallInCatch = processBody.indexOf(
"internal.pageSpeed.queueLeadPageSpeedAudit",
outerCatchStart,
);
assert.equal(
queueCallInCatch > outerCatchStart &&
queueCallInCatch > startedGuard &&
queueCallInCatch < catchReturnNull,
true,
"Fatal catch path should queue PageSpeed before returning, while started lead exists.",
);
const catchBlock = processBody.slice(outerCatchStart, catchReturnNull);
assert.equal(
hasPattern(
catchBlock,
/leadId:\s*started\.lead\._id[\s\S]*?parentRunId:\s*runId/,
),
true,
"Catch-path PageSpeed queue payload should use started.lead._id and parentRunId runId.",
);
});