Milvus Architecture & Performance

[rganju1]

What are the current best practices for index selection in Milvus 2.6 for high-dimensional vectors?”

“Are there recommended approaches to optimize query latency when dealing with billions of vectors?”

“How does Milvus handle node failures in a distributed cluster, and what are the recovery guarantees?”

[yhmo]

Major points in index selection:

1. recall
2. search performance
3. memory usage

Minor points in index selection:

1. time cost to build index
2. disk usage

Some index types are good on recall, some others are good on performance or memory usage. There is no "best index" that can get "best recall, best performance, and low memory usage".

This doc is about index selection based on your recall/performance/memory_usage requirement:
https://milvus.io/docs/index-explained.md

Some suggestiuons on "approaches to optimize query latency when dealing with billions of vectors":

• Use a distributed milvus service instead of standalone, more query nodes, more CPU cores per query node
• If the memory capacity of each query node is sufficient, HNSW index is recommended. If you can accept low recall rate, you can choose IVF_PQ, or IVF_FLAT with a smaller value of "nprobe".
• Split data into partitions, searching in a specific partition is faster than searching in the entire collection
• Create an index on scalar field if the field is involved in a filtering expression. For example, search with the expression 'title LIKE city%', you can create NGRAM index on the "title" field.
• If not necessary, don't return long-length fields in the "output_fields" parameter of search()

In each cluster, there are some "coordinators" to manage nodes. "query_coord" for query nodes, "data_coord" for data nodes and index nodes. If one of the coordinators is down, the service will be blocked/unhealthy.
If a query node is down, the k8s will automatically pull up a new node. The new query node will read index into its memory, once it is ready, the search service is healthy. If the k8s has no enough resources to pull up a new node, the query_coord will try balance the index data in existing query nodes.