There are two huge things your 5 minute setup is missing which are very hard techinically to tackle
1. Incrementally updating the search space. Not that easy to do, and becomes more important to not just do the dumb thing of retraining the entire index on every update for larger datasets.
2. Combining vector search and some database-like search in an efficient manner. I don't know if this Google post really solves that problem or if they just do the vector lookup followed by a parallelized linear scan, but this is still an open research/unsolved problem.
Spot on! Both of those were motivating factors when building Weaviate (Open Source Vector Search Engine). We really wanted it to feel like a full database or search engine. You should be able to do anything you would do with Elasticsearch, etc. There should be no waiting time between creating an object and searching. Incremental Updates and Deletes supported, etc.
Correct, that would take more than 5 minutes, although still possible to do with Faiss (and not that hard relatively speaking - in the Teclis demo, I indeed did your second point - combine results with a keyword search engine and there are many simple solutions you can use out there like Meilisearch, Sonic etc.e). If you were to try using an external API for vector search, you would still need to build keyword based search separately (and then combining/ranking logic) so then you may be better off just building the entire stack anyway.
Anyway, for me, the number one priority was latency and it is hard to beat on-premise search for that.
Even then, a vector search API is just one component you will need in your stack. You need to pick the right model, create vectors (GPU intensive), then possibly combine search results with keyword based search (say BM25) to improve accuracy etc. I am still waiting to see an end-to-end API doing all this.
>then possibly combine search results with keyword based search (say BM25) to >improve accuracy etc. I am still waiting to see an end-to-end API doing all this
>
> I am still waiting to see an end to end API doing all this
That’s kinda the idea of Weaviate. You might like the Wikipedia demo dataset that contains all this. You indeed need to run this demo on your own infra but the whole setup (from vector DB to ML models) is containerized https://github.com/semi-technologies/semantic-search-through...
Exactly right. Things like data freshness (live index updates), CRUD operations, metadata filtering, and horizontal scaling are all “extras” that don’t come with Faiss. Hence the need for solutions like Google Matching Engine and Pinecone.io.
And even if you do just want ANN and nothing else, some people just want to make API calls to a live service and not worry about anything else.
Can you expand more or provide a concrete example for the second point? What kind of database-like searches are you thinking about for spatial data? Things like range-queries can already be (approximately) done. Or are you thinking about relational style queries on data associated with each point?
Yes exactly, relational style queries with each data point. Maybe you have some metadata about your images and maybe you need to join against another table to properly query them. But at the same time you want to only grab the first k nearest neighbors according to vector similarity.
1. Incrementally updating the search space. Not that easy to do, and becomes more important to not just do the dumb thing of retraining the entire index on every update for larger datasets.
2. Combining vector search and some database-like search in an efficient manner. I don't know if this Google post really solves that problem or if they just do the vector lookup followed by a parallelized linear scan, but this is still an open research/unsolved problem.