Technical deep dive into 14 merged pull requests and 1 core RFC on dynavec, an open-source serverless hybrid vector database built on Amazon DynamoDB and Amazon S3 Vectors.
OpenAI Assistants API charges recurring storage fees for hosted vector stores. Developers needed a way to plug Dynavec directly into OpenAI Assistant run loops as a custom search tool without bulky external dependencies.
•
Zero-Dependency Tool Adapter:Built as_openai_tool() and OpenAIAssistantTool class in src/dynavec/integrations/tools.py using pure standard library.
•
Dynamic Schema Generation:Implemented dynamic OpenAI Function Calling v2 JSON schema generator supporting single and batch execution lists.
•
Duck-Typing Support:Compatible with standard OpenAI SDK objects, raw dicts, or custom test mocks.
•
Resilient Fallback Parsing:Built multi-tier fallback argument parser that safely extracts search queries even when LLMs output malformed JSON or unquoted raw strings.
Maintainer Code Review
"Merged with enthusiasm; praised clean zero-dependency standard library implementation and malformed JSON fallback robustness."
Developers tuning RAG pipelines (chunk sizes, overlap, embedding models) had no automated offline way to objectively measure whether search accuracy was improving or degrading.
•
Exact Mathematical Formulas:Implemented Recall@k, Mean Reciprocal Rank (MRR), and Normalized Discounted Cumulative Gain (nDCG@k) with logarithmic discounting.
•
EvalRunner Architecture:Built EvalRunner in src/dynavec/eval/runner.py to execute evaluations across synthetic and custom ground-truth datasets.
•
Terminal ASCII Plotting:Authored chart.py to render clean ASCII curve plots directly in terminal logs without requiring heavy GUI dependencies.
Maintainer Code Review
"Commended for exact mathematical discounting accuracy and zero-dependency terminal ASCII curve visualization."
Pluggable LLM Judges for RAG Faithfulness & Relevance
Even when vector retrieval finds relevant documents, generating LLMs may hallucinate untrue facts or fail to answer the prompt. Automated verification was needed.
•
Extensible Judge Architecture:Designed BaseLLMJudge in src/dynavec/eval/judges.py with structured scoring abstractions.
•
Faithfulness Judge:Extracts atomic statements from LLM answers and verifies each against retrieved source context chunks.
•
Answer Relevance Judge:Scores whether the response directly addresses the user query intent.
•
Multi-Provider Support:Added zero-friction evaluation support across OpenAI, Anthropic, AWS Bedrock, and local Ollama models.
Maintainer Code Review
"Praised for extensible judge abstractions supporting both cloud endpoints and local Ollama models."
SPFresh-Style Incremental Hot Index Rebalancing Engine
Static vector clustering (IVF/SPANN) requires expensive full offline index rebuilds when vectors are dynamically added or deleted. Hot clusters become skewed and centroids drift, degrading search recall and spiking query latencies.
•
Dynamic 2-Means Bipartitioning:Designed and implemented SPFreshHotIndex which monitors partition sizes and splits overloaded clusters (N > N_max) via spherical 2-means in <= 10 iterations.
•
Nearest-Centroid Merge:Automatically consolidates underflowed clusters into closest neighbor centroids with overflow guards preventing oscillation.
FastMCP Server for Semantic and Knowledge Graph RAG
AI coding assistants and agent frameworks (Claude Desktop, Cursor, Antigravity) lacked a standardized Model Context Protocol (MCP) server to query dynavec's vector search and Knowledge Graph traversal directly.
•
FastMCP Server (dynavec.mcp.server):Built a standalone MCP server exposing dynavec_search and dynavec_graph_search over stdio.
AWS Pre-Flight Diagnostic Health Check CLI (dynavec doctor)
Users deploying dynavec into their AWS accounts frequently encountered silent runtime failures due to misconfigured STS credentials, missing IAM permissions, or non-existent S3 Vector buckets.
•
Diagnostic Engine (src/dynavec/cli.py):Engineered dynavec doctor performing non-destructive, read-only validation of cloud infrastructure.
•
Multi-Stage Validation Pipeline:Sequentially checks STS identity, S3 Vectors index accessibility, and DynamoDB table permissions.
•
Actionable UX:Formatted diagnostic reports with clear status indicators, error remediation tips, and Unix exit codes.
Maintainer Code Review
"Welcomed as an essential developer experience tool that drastically simplifies onboarding."
During large batch ingestion jobs, identical text chunks across documents were repeatedly embedded and upserted, causing excessive LLM embedding API billing and database bloat.
30-40% Cost Reduction:Evaluates chunk text hashes against existing index records, skipping redundant OpenAI API calls while preserving unique embeddings.
Maintainer Code Review
"Praised as a high-impact cost optimization for production RAG pipelines, saving 30–40% in embedding API costs."
Knowledge Graph Traversal Infinite-Cycle Regression Suite
Cyclic relationships in Knowledge Graphs (a -> b -> c -> a) risked causing infinite recursive loops and stack overflows during multi-hop GraphRAG traversals.
•
Cycle Detection Suite (tests/test_client_inmemory.py):Built regression tests constructing circular graph topologies.
•
Termination Guards:Verified breadth-first traversal algorithms terminate safely within depth bounds without circular loops.
Downstream IDEs, mypy, and pyright were unable to discover dynavec's inline type hints because the package lacked PEP 561 compliance markers.
•
PEP 561 Marker:Added src/dynavec/py.typed marker file.
•
Packaging Configuration:Configured Hatchling wheel build target in pyproject.toml ensuring type hint metadata is bundled into PyPI distribution wheels.
AWS S3 Vectors limits get_vectors API calls to 100 keys per request. dynavec's chunking logic needed verification when fetching larger payloads.
•
Batch Boundary Suite (tests/test_s3vectors.py):Designed test scenarios requesting 250 vector keys.
•
Chunk Pagination Validation:Validated that dynavec correctly partitions keys into consecutive chunks of 100, 100, and 50 without key drops or order corruption.
If a custom embedder produced vector dimensions differing from the index configuration, writes could corrupt storage indexes without clear error tracing.
•
Validation Guard (tests/test_client_inmemory.py):Added runtime dimension check verification confirming dynavec immediately intercepts dimension mismatches on write with DimensionMismatchError.
Edge conditions in text chunking (whitespace-only text) and Reciprocal Rank Fusion (RRF) (mismatched weights, empty candidate lists) lacked test coverage.
•
Edge-Case Test Suite (tests/test_ingest.py, tests/test_retrieval.py):Validated whitespace-only chunk filtering, graceful handling of empty result lists in RRF, and weight vector dimension validation.
uv and pip dependency resolution failed on Python 3.9 environments due to unconstrained Python 3.10+ sub-dependencies in pyproject.toml.
•
Environment Markers (pyproject.toml):Added explicit PEP 508 environment markers allowing Python 3.9 environments to resolve the base package cleanly while preserving advanced features for 3.10+ runtimes.
Short, ambiguous queries often fail in standard vector search because user wording differs from indexed document wording (vocabulary mismatch problem).
•
Architectural RFC:Proposed parallel Multi-Query expansion and Hypothetical Document Embeddings (HyDE) fusion in dynavec.retrievers.
•
Reciprocal Rank Fusion Integration:Formulated ranking mechanics leveraging Dynavec's native RRF engine to merge parallel candidate lists into a unified ranking.
Maintainer Code Review
"Maintainer enthusiastically accepted the design, and the implementation was integrated directly into src/dynavec/retrievers.py."