Fix FinTS transaction deduplication

backlog/tasks/task-15 - Fix-FinTS-transaction-deduplication.md

@@ -0,0 +1,52 @@
+---
+id: TASK-15
+title: Fix FinTS transaction deduplication
+status: Done
+assignee: []
+created_date: '2026-06-23 10:22'
+updated_date: '2026-06-23 10:36'
+labels: []
+dependencies: []
+priority: high
+ordinal: 15000
+---
+
+## Description
+
+<!-- SECTION:DESCRIPTION:BEGIN -->
+Repair FinTS import deduplication so placeholder MT940/CAMT references do not cause valid comdirect Girokonto transactions to be skipped.
+<!-- SECTION:DESCRIPTION:END -->
+
+## Acceptance Criteria
+<!-- AC:BEGIN -->
+- [x] #1 FinTS placeholder references such as NONREF and NOTPROVIDED are not stored as externalRef
+- [x] #2 Multiple FinTS rows with placeholder references but distinct transaction content import successfully
+- [x] #3 Real duplicate externalRef values are still skipped
+- [x] #4 Repeated import of the same rows skips via dedupHash
+- [x] #5 Focused tests cover FinTS mapping and import commit behavior
+<!-- AC:END -->
+
+## Implementation Plan
+
+<!-- SECTION:PLAN:BEGIN -->
+1. Add failing regression tests for FinTS placeholder references and import skip behavior
+2. Normalize FinTS external references so placeholders are discarded
+3. Add import skip diagnostics for externalRef vs dedupHash
+4. Run focused tests and project verification
+<!-- SECTION:PLAN:END -->
+
+## Implementation Notes
+
+<!-- SECTION:NOTES:BEGIN -->
+Implemented FinTS externalRef normalization for MT940/CAMT placeholder values and added import commit diagnostics split by externalRef and dedupHash skips. Verified with focused Vitest, broader Convex tests, focused ESLint, diff whitespace check, and production build.
+
+Follow-up from live Convex logs: all 42 rows were skipped by externalRef. Read-only Convex data inspection showed FinTS externalRef values like POS 13, which are MT940 positional markers and collide across statement periods. Extended normalization to discard POS-style references and added regression coverage for legacy POS externalRef records not blocking new rows.
+
+Follow-up from visible UI gaps: imported rows existed, but transactions.list used by_user_effmonth ordering for assignment-month mode, which sorts by effectiveMonth and creation time rather than bookingDate. Added by_user_effmonth_booking index and switched effective-basis listing to it so current month rows are ordered by booking date descending. Added regression coverage.
+<!-- SECTION:NOTES:END -->
+
+## Final Summary
+
+<!-- SECTION:FINAL_SUMMARY:BEGIN -->
+Fixed FinTS transaction deduplication by discarding placeholder and POS-style external references, added import skip diagnostics, and corrected effective-month transaction listing to order by booking date. Verified with focused and related Vitest suites, focused ESLint, diff check, and production build.
+<!-- SECTION:FINAL_SUMMARY:END -->