feat(knowledge-graph): True entity extraction and interactive knowledge graph from documents

[nymulinfoinlet]

README claims 'Auto-extracts entities and relationships' but actual implementation is a RAG chunking pipeline. The 'Concept Map' page is tied to problem-solver output, not document analysis.

Implement real entity extraction from uploaded documents, relationship mapping, and interactive force-directed graph visualization (D3.js). Cross-document linking.

Closes gap against: RemNote (knowledge graph — their core differentiator). Delivers on existing README claim.

Effort: XL

[m13v]

entity extraction from uploaded documents is one of those features that sounds straightforward but has a nasty accuracy cliff. the gap between 'works on clean academic papers' and 'works on messy real-world PDFs' is enormous.

a few things we learned building extraction pipelines for unstructured data:

1. extraction quality depends entirely on your chunking strategy. if you chunk by fixed token count (which most RAG pipelines do), entities that span chunk boundaries get split and you miss relationships between them. chunk by semantic paragraphs instead, and run entity extraction per-chunk with a merging pass afterward to resolve duplicate entities across chunks.

2. for the knowledge graph visualization, D3 force-directed graphs look great in demos but become unusable past ~200 nodes. the better UX is a hierarchical tree with expandable clusters, where the top level shows document-level entities and you drill into cross-document relationships. keeps it navigable even with thousands of entities.

3. cross-document linking is where the real value is, but it requires entity resolution (matching 'React' in doc A with 'ReactJS' in doc B with 'React.js' in doc C). embedding-based matching with a similarity threshold works better than string matching here. we store entity embeddings alongside the entity text and do a kNN lookup on every new entity to find potential matches.

[m13v]

our entity extraction and embedding-based matching pipeline for knowledge entries: https://github.com/m13v/ai-browser-profile/blob/main/ai_browser_profile/embeddings.py - includes the kNN lookup for entity resolution across different sources. and the database schema for storing extracted entities with relationships: https://github.com/m13v/ai-browser-profile/blob/main/ai_browser_profile/db.py