1.9 KiB
id, title, status, assignee, created_date, updated_date, labels, dependencies, priority, ordinal
| id | title | status | assignee | created_date | updated_date | labels | dependencies | priority | ordinal |
|---|---|---|---|---|---|---|---|---|---|
| TASK-23 | Improve website email extraction | In Progress | 2026-06-04 17:28 | 2026-06-04 17:34 | high | 25000 |
Description
Fix TASK-8 website enrichment so Playwright crawls contact/imprint/footer email patterns that are visible on crawled pages but currently missed by the extractor.
Acceptance Criteria
- #1 Extract mailto href emails even with query parameters and labels
- #2 Extract common obfuscated German website email patterns such as [at], (at), at, and spaced @/dot forms
- #3 Treat emails found on Kontakt/Impressum pages or footer contact context as business contact candidates without guessing addresses
- #4 Keep TASK-7 rules intact: no generated emails, named emails require explicit business context
- #5 Verify with focused RED/GREEN tests and full suite
Implementation Notes
Updated website-crawler extractor to support mailto query stripping/decoding, HTML entity decoding for email separators, obfuscated [at]/(at)/dot/punkt and spaced @/dot forms, and expanded business-context detection for footer/impressum/contact regions. Limited to lib/website-crawler.ts only.
Implemented via subagents/TDD: added RED tests for mailto query params, obfuscated email forms, footer/impressum usability, no-guessing false-positive guard, and mailto dedupe. Extractor now decodes common HTML entities, strips/decodes mailto query strings, parses [at]/(at)/punkt/dot/spaced forms with guardrails, expands footer/impressum/contact business context, and leaves TASK-7 selection unchanged. Verification passed: pnpm exec tsc -p tsconfig.json; pnpm test (114/114); pnpm lint (existing generated BetterAuth warnings only); pnpm exec convex codegen --dry-run --typecheck enable.