webdev-pipeline/backlog/tasks/task-7 - Add-lead-qualification-deduplication-and-blacklist-handling.md

id, title, status, assignee, created_date, updated_date, labels, dependencies, references, priority, ordinal

id	title	status	assignee	created_date	updated_date	labels	dependencies	references	priority	ordinal
TASK-7	Add lead qualification, deduplication, and blacklist handling	Done		2026-06-03 19:13	2026-06-04 14:09	mvp leads quality	TASK-6	PRD.md	high	7000

Description

Implement the rules that turn raw business discoveries into usable lead states. Leads should be classified by contact availability, website potential, duplicate risk, existing contact history, and manual blacklist matches before any audit or outreach work starts.

Acceptance Criteria

• #1 Leads with no usable email are placed in Kontakt fehlt while preserving phone and source data
• #2 Generic business emails are preferred and named emails are accepted only when explicitly found as business contact addresses
• #3 Hard duplicates are detected by domain, Google Place ID, or email; probable duplicates are flagged by name plus address or phone
• #4 Manual blacklist entries for domain, email, phone, company name, and Place ID are enforced during discovery and review
• #5 Priority values Hoch, Mittel, Niedrig, Zurückstellen, and Gesperrt are assigned or editable with clear reasons

Implementation Plan

Subagent-driven TDD execution plan

Orchestrator responsibilities:

1. Coordinate TASK-7 implementation end to end.
2. Use gpt-5.3-codex-spark subagents for implementation and review slices.
3. Enforce TDD: write failing tests first, verify red, implement minimal production code, verify green, then refactor.
4. Keep Backlog notes current and do not mark Done until user confirms manual testing.

Implementation slices:

1. Rules/backend qualification: add tests and implementation for email usability, generic vs named email handling, hard duplicates by domain/place/email, probable duplicates by company+address or company+phone, blacklist normalization, and priority/status reason derivation.
2. Convex integration: extend schema/types/indexes and lead/blacklist APIs for qualification, editable priority/status/reasons, blacklist CRUD, and discovery/review enforcement.
3. Dashboard UI: replace Leads and Sperrliste placeholders with scan-friendly review tools that expose source data, duplicate/blacklist reasons, and editable priority/status controls.
4. Funnel/model polish: map blocked priority to Gesperrt and keep deferred/review funnel behavior coherent.
5. Verification: run targeted tests during each TDD slice, then pnpm test and pnpm lint at the end.

Acceptance criteria mapping:

• AC1: contact qualification stores leads without usable email as Kontakt fehlt while preserving phone/source metadata.
• AC2: email rules prefer generic business addresses and only allow named emails when explicitly sourced as business contact addresses.
• AC3: duplicate rules distinguish hard duplicates and probable duplicates.
• AC4: blacklist entries for domain/email/phone/company/place ID apply during discovery and review.
• AC5: Hoch, Mittel, Niedrig, Zurückstellen, and Gesperrt are assignable/editable with clear reasons.

Implementation Notes

Execution started with subagent-driven TDD orchestration using gpt-5.3-codex-spark as requested.

Aufgabe 7: implementiere Google-Places-Email-Review-Regeln, Sperrlisten-Enforcement für bestehende Leads, und korrigiere Firmen-Normalisierung in Blacklist-Matching. Beginne mit neuen TDD-Tests in lib/lead-discovery-google + Convex-Review-Pfad.

TASK-7 implemented: added review-based email contact patch in convex/leads.ts, bounded blacklist enforcement on create/update in convex/blacklist.ts, company normalization fix in getBlacklistLookupValues/getBlacklistMatches, and schema support for new lead matching fields/reasons/blocked priority. Tests: pnpm -s test ✅ and pnpm -s tsc ✅.

Progress: implementing code-quality fixes in convex/blacklist.ts, convex/leads.ts, convex/leadDiscovery.ts; running requested test/type/lint commands after changes. Plan: tighten mutation patch typing, bound blacklist propagation, split website signal, and avoid empty normalized writes.

Executed requested TASK-7 backend quality fixes in scoped files and validated with pnpm -s test, pnpm -s tsc, and targeted eslint. Outstanding follow-up: keep an eye on very large blacklist match sets; enforcement currently remains batch-at-a-time by design.

TASK-7 implementation verified by orchestrator. Added lead qualification helpers and Convex integration for usable email handling, hard/probable duplicate detection, blacklist enforcement with scheduled backfill/apply batches, blocked priority/reason fields, and dashboard Leads/Sperrliste review UI. Verified: pnpm -s test (67 pass), pnpm -s tsc (exit 0), pnpm -s lint (0 errors, 2 generated Better Auth warnings). Browser plugin could not open localhost due ERR_BLOCKED_BY_CLIENT; route HEAD checks redirect to /login as expected for protected dashboard pages.

Final Summary

Implemented lead qualification, duplicate handling, blacklist enforcement, blocked priority/reason support, and dashboard review surfaces. Verified acceptance criteria #1-#5 with tests/typecheck/lint; user confirmed TASK-7 is done.