What Is DiskANN? Billion-Scale Vector Search Explained from @couchbase, by @hannahblaurel couchbase.com/blog/diskann/?…

🎉 Zvec 0.5.0 Release is Live! This release focuses on retrieval, indexing, ecosystem, and platform & hardware. Give it a try and let us know what you think! ✨ ✨ What's new: • 🔍 Retrieval: Native full-text search works out of the box; a single query fuses full-text search, semantic vectors, and conditional filters, with multi-way result merging handled automatically; the MultiQuery interface is implemented natively in C/C , so every language SDK can integrate hybrid retrieval with ease • 💾 On-Disk Indexing: New DiskANN on-disk index significantly reduces memory overhead in large-scale scenarios; it complements HNSW/IVF/Flat to cover the full range of retrieval needs, from pure in-memory to on-disk • 🌐 Ecosystem: Brand-new official Go SDK and Rust SDK for smoother multi-language access; plus the new visual tool Zvec Studio for zero-code data browsing and query debugging • 🖥️ Platform & Hardware: Added RISC-V support, further expanding hardware compatibility 📚 Learn more: • 📄 Release Notes: zvec.org/en/blog/2026-06-12-… • 🧭 Roadmap: github.com/alibaba/zvec/issu… • 🐛 Report bugs / requests: github.com/alibaba/zvec/issu… • 💬 Discuss & share: github.com/alibaba/zvec/disc…

📣 New Podcast! "The Billion-Vector Problem: HNSW vs. DiskANN in Azure AI Search" on @Spreaker #ann #azure #copilot #cosmosdb #diskann spreaker.com/episode/the-bil…

Run around 3x more transactions per second on Azure HorizonDB over self-managed PostgreSQL. Commit data across zones before it even hits disk, showing true durability by design. See it in action. youtu.be/EzEPFMJuvrk Run enterprise Postgres workloads on Azure HorizonDB with 3x the throughput of self-managed deployments — zone-resilient by default, no architectural trade-offs. Call AI models directly from SQL, build durable vector pipelines inside the database, and deliver high-accuracy similarity search at massive scale with DiskANN and AI re-ranking, all without leaving Postgres. #AzureHorizonDB #microsoftazure, #ai, #azure, #cloudcomputing, #generativeai

@SurrealDB diskANN is cool, but this seems cooler.

Call AI models directly using AI functions in Azure HorizonDB. AI Model Management auto-registers GPT, embedding, and re-ranking models. Try it now. youtu.be/EzEPFMJuvrk Run enterprise PostgreSQL workloads on Azure HorizonDB with 3x the throughput of self-managed deployments — zone-resilient by default, no architectural trade-offs. Call AI models directly from SQL, build durable vector pipelines inside the database, and deliver high-accuracy similarity search at massive scale with DiskANN and AI re-ranking, all without leaving PostgreSQL. #AzureHorizonDB #microsoftazure, #ai, #azure, #cloudcomputing, #generativeai

Build durable AI pipelines inside Azure HorizonDB. The create_pipeline function chunks, embeds, and stores vectors asynchronously as data lands. See it in action. youtu.be/EzEPFMJuvrk Run enterprise PostgreSQL workloads on Azure HorizonDB with 3x the throughput of self-managed deployments — zone-resilient by default, no architectural trade-offs. Call AI models directly from SQL, build durable vector pipelines inside the database, and deliver high-accuracy similarity search at massive scale with DiskANN and AI re-ranking, all without leaving PostgreSQL. #AzureHorizonDB #microsoftazure, #ai, #azure, #cloudcomputing, #generativeai

Scale vector search across massive datasets with DiskANN in Azure HorizonDB — quantized vectors in memory, full-precision graphs on disk. Wrap results in rank() to apply a Cohere model and surface the most relevant hits. See how it works.  youtu.be/EzEPFMJuvrk Run enterprise PostgreSQL workloads on Azure HorizonDB with 3x the throughput of self-managed deployments — zone-resilient by default, no architectural trade-offs. Call AI models directly from SQL, build durable vector pipelines inside the database, and deliver high-accuracy similarity search at massive scale with DiskANN and AI re-ranking, all without leaving PostgreSQL. #AzureHorizonDB #microsoftazure, #ai, #azure, #cloudcomputing, #generativeai

Debug slow PostgreSQL queries in VS Code with a visual execution plan, then let Copilot generate the fix. Get started. youtu.be/EzEPFMJuvrk Run enterprise PostgreSQL workloads on Azure HorizonDB with 3x the throughput of self-managed deployments — zone-resilient by default, no architectural trade-offs. Call AI models directly from SQL, build durable vector pipelines inside the database, and deliver high-accuracy similarity search at massive scale with DiskANN and AI re-ranking, all without leaving PostgreSQL. #AzureHorizonDB #microsoftazure, #ai, #azure, #cloudcomputing, #generativeai

データ関連だとMicrosoftがDiskANNを本格的に使い始める方向性が見えてよかった #MicrosoftBuild

diskANNが検索に使われるようです ベクトルの量子化 (1%に圧縮)が確かできたはずなので、高速化だったり大きなデータ扱いやすくなるとかなんでしょうか？

the index tells you what the database thinks it is. DiskANN says memory, not archive

10/ SurrealDB 3.1 is another step toward that future: DiskANN support ANN lookup improvements stability work release process overhaul production hardening The database world is getting interesting again. And honestly? We’re still early. surrealdb.com/blog/surrealdb…

4/ DiskANN changes the equation. Instead of assuming your index lives entirely in memory, it’s designed around SSD-aware graph traversal. Which means: larger indexes lower memory overhead cheaper scaling faster recovery more predictable infra economics Without giving up high-quality recall.

1/ Databases are entering a weird new era. Not because of AI hype. Because for the first time in decades, databases are being asked to behave less like storage engines… and more like memory systems. Today we shipped SurrealDB 3.1. And buried inside the release is something important: DiskANN.

SurrealDB 3.1 is here and builds on the foundations of 3.0, with a focus on stability, a second approximate-nearest-neighbour vector index in DiskANN, GraphQL upgrades, and significant security hardening. Explore the highlights.👉 sdb.li/3Q7X4Ey

Concepts to Learn to Take Your RAG System from Notebook to Prod I have been building RAG pipelines and chatbots for quite some time now, and even now I still learn new concepts or connect dots from the systems principles I am studying currently. RAG pipelines or QA chatbots are among the most common resume projects nowadays. They are decent but definitely not enough. Most look perfect in notebooks and demos but rarely account for real-world problems. Still, they can genuinely teach the intricacies of a production pipeline. At the prototype level, basic chunking vector search LLM bit prompt FAFO is enough. At production level, you must design a system that stays fresh, cheap, fast, and reliable as data volume, user traffic, and business complexity grow—especially when things collapse at scale with millions of documents, 10k QPS, or data changing every hour. These production concepts will strengthen your projects and signal robust system design thinking: **Core RAG Pipeline Stages** ↬ Ingestion Pipeline ↬ Semantic Chunking ↬ Recursive / Hierarchical Chunking ↬ Parent-Child Relationships ↬ Metadata Enrichment (timestamps, ACLs, doc_type) ↬ Embedding Model Routing **Vector & Indexing Layer** ↬ Dense Sparse Hybrid Indexing ↬ Multi-Vector / Late Interaction (ColBERT) ↬ Quantization (INT8 / Binary / Product Quantization) ↬ Incremental Indexing CDC (Change Data Capture) via Kafka ↬ HNSW / IVF / DiskANN Index Types (tradeoffs in recall vs memory vs speed) **Vector Database Internals & Concepts** ↬ ANN Algorithms (HNSW graph navigation, IVF clustering) ↬ Payload / Metadata Filtering (pre vs post-filtering) ↬ Sharding Replication Strategies ↬ Quantization & Compression for cost/memory efficiency ↬ Index Tuning Parameters (M, efConstruction in HNSW) **Serving Layer** ↬ Dual Pipelines (Batch Indexing vs Real-time Query Serving) ↬ Inference Optimization (batching, quantization, GPU/CPU routing) ↬ Latency Budget Enforcement (P95 < 1.5s) ↬ Autoscaling & Load Balancing for Query Serving ↬ Semantic KV Caching Layers **Pre-Retrieval Intelligence** ↬ Query Rewriting (HyDE, Multi-Query, Step-Back) ↬ Query Classification & Adaptive Routing ↬ Semantic Cache Layer ↬ Intent Detection **Retrieval & Post-Retrieval** ↬ Hybrid Retrieval ↬ Metadata Filtering ↬ Cross-Encoder Reranking ↬ MMR Diversity ↬ Context Compression (LLMLingua) ↬ CRAG / Self-RAG Reflection **Advanced Architectures** ↬ GraphRAG (Entity Community Summaries) ↬ Agentic RAG (ReAct Tool Use) ↬ Corrective / Adaptive RAG **Production Scaling Realities** ↬ Hot / Warm / Cold Indexing Tiers ↬ Vector DB Sharding Cross-Region Replication ↬ Embedding Drift Detection ↬ Periodic Fine-Tuning ↬ Freshness SLAs & Priority Queues ↬ Cost-per-Query Monitoring **Multi-Tenancy (Highly Relevant for Enterprise/SaaS RAG)** ↬ Namespace / Partition / Collection per Tenant ↬ Metadata-based Isolation Row-Level Security ↬ Silo vs Shared Index Patterns (tradeoffs in isolation vs cost) ↬ ACL Enforcement at Query Time **Observability & Reliability** ↬ RAGAS / DeepEval Metrics ↬ Golden Dataset Regression Testing ↬ End-to-End Distributed Tracing ↬ User Feedback Auto-Retraining Loop ↬ SLO / Error Budget Tracking **Security & Guardrails** ↬ Row-Level Security & ACLs at DB layer ↬ PII Redaction Pipeline ↬ Input/Output Moderation ↬ Prompt Injection Defense - Directed and ideated by yours truly, enhanced and formatted by Grok

次世代のAI記憶基盤！超高速ベクターDB『Caliby』 MITチームが開発した、AIエージェントやRAGを劇的に進化させる埋め込み型データベースです！✨ ・pgvectorより4倍速く、FAISSを超える検索性能 ・pip installだけで導入完了！Dockerやサーバー構築は一切不要 ・HNSW/DiskANN/IVF PQなど、主要な検索アルゴリズムをフルサポート ・テキストとベクトルデータを一つのシステムで効率的に一元管理 ・SIMD（AVX-512）活用によりCPUでの処理が驚異的なスピードに 個人開発やエッジAIの長期記憶に最適な、まさに「AIデータ界のDuckDB」と呼べる強力なツールです。🚀 #AI #エンジニア

This is great read from the @RLanceMartin about the memory in Claude Managed Agents, combination of the RLM style and Letta's memGPT approach. However, still file system based. Some interesting approaches from vectorDB companies also promising @milvusio @SurrealDB @lancedb Personally, I built SpecMem and AgentVectorDB using LanceDB's DiskANN based approach but Agent Memory could be the topic of the 2026 to explore. Lance, you should try Lance too .. 😂