Scaling Vector Search with Approximate Nearest Neighbors
Searching through millions of vectors one by one is too slow. In this post, we explore Approximate Nearest Neighbor (ANN) algorithms that allow us to find 'good enough' results instantly.
Some of the things I've built.
Small Business Accounting Software. Features include invoicing and basic expense management
An ongoing project for managing personal financial planner.
An interview preparation kit for aspiring software engineers.
Thoughts on software engineering and more.
Searching through millions of vectors one by one is too slow. In this post, we explore Approximate Nearest Neighbor (ANN) algorithms that allow us to find 'good enough' results instantly.
Keyword search has its limits. This post dives into vector embeddings, the core of modern semantic search. I'll explore how lists of numbers can capture complex meanings and relationships, allowing search engines to understand intent and context, not just keywords.
To truly understand how search works, you have to build it. This post demystifies search engine architecture by rebuilding it from the ground up in Python, exploring the fundamentals of web crawlers, inverted indexes, and simple ranking algorithms.
Lucene is a tool for text analysis. It uses an analyzer to split the text into tokens and process them. An analyzer has three parts: char filters, tokenizer, and token filters. They can do different things with the text, such as remove HTML tags, find synonyms, or make ngrams.
Cache stampede occurs when multiple requests regenerate the same cache item simultaneously, overloading the database. Learn its causes and solutions like query optimization, locking, pre-populating the cache, and probabilistic early expiration.