Choosing Indexes for Similarity Search (Faiss in Python)
19.4 هزار بار بازدید -
3 سال پیش
-
Facebook AI Similarity Search (Faiss)
Facebook AI Similarity Search (Faiss) is a game-changer in the world of search. It allows us to efficiently search a huge range of media, from GIFs to articles - with incredible accuracy in sub-second timescales for billion+ size datasets.
The success in Faiss is due to many reasons. One of those, in particular, is its flexibility. Faiss recognizes that there is no 'one-size-fits-all' in similarity search.
Instead, Faiss comes with a wide range of search indexes - which we can mix and match to our choosing.
However, this great flexibility produces a question - how do we know which size fits our use case?
Which index do we choose? Should we use multiple indexes, or is one enough?
This video will explore the pros and cons of some of the most important indexes - Flat, LSH, HNSW, and IVF. We will learn how we decide which to use and the impact of parameters in each index to build some of the best indexes for semantic search.
🌲 Pinecone Article:
https://www.pinecone.io/learn/vector-...
🎉 Sign-up For New Articles Every Week on Medium!
Medium: membership
Download script for Sift1M dataset:
https://gist.github.com/jamescalam/a0...
Similarity Search Series:
Vector Similarity Search and Faiss Co...
🤖 70% Discount on the NLP With Transformers in Python course:
https://bit.ly/3DFvvY5
👾 Discord
Discord: discord
Mining Massive Datasets Book (Similarity Search):
📚 https://amzn.to/3CC0zrc (3rd ed)
📚 https://amzn.to/3AtHSnV (1st ed, cheaper)
🕹️ Free AI-Powered Code Refactoring with Sourcery:
https://sourcery.ai/?utm_source=YouTu...
The success in Faiss is due to many reasons. One of those, in particular, is its flexibility. Faiss recognizes that there is no 'one-size-fits-all' in similarity search.
Instead, Faiss comes with a wide range of search indexes - which we can mix and match to our choosing.
However, this great flexibility produces a question - how do we know which size fits our use case?
Which index do we choose? Should we use multiple indexes, or is one enough?
This video will explore the pros and cons of some of the most important indexes - Flat, LSH, HNSW, and IVF. We will learn how we decide which to use and the impact of parameters in each index to build some of the best indexes for semantic search.
🌲 Pinecone Article:
https://www.pinecone.io/learn/vector-...
🎉 Sign-up For New Articles Every Week on Medium!
Medium: membership
Download script for Sift1M dataset:
https://gist.github.com/jamescalam/a0...
Similarity Search Series:
Vector Similarity Search and Faiss Co...
🤖 70% Discount on the NLP With Transformers in Python course:
https://bit.ly/3DFvvY5
👾 Discord
Discord: discord
Mining Massive Datasets Book (Similarity Search):
📚 https://amzn.to/3CC0zrc (3rd ed)
📚 https://amzn.to/3AtHSnV (1st ed, cheaper)
🕹️ Free AI-Powered Code Refactoring with Sourcery:
https://sourcery.ai/?utm_source=YouTu...
3 سال پیش
در تاریخ 1400/05/18 منتشر شده
است.
19,434
بـار بازدید شده