Contextual embedding in RavenDB (or how to save 220,000,000 USD) - Ayende @ Rahien
Oren Eini
CEO of RavenDB
a NoSQL Open Source Document Database
Get in touch with me:
oren@ravendb.net<br>+972 52-548-6969
Posts: 7,646
Comments: 51,329
Copyright ©️ Ayende Rahien 2004 — 2026
Privacy Policy<br>Terms
time to read 12 min | 2307 words
A while ago, MongoDB purchased VoyageAI for 220 million dollars. Since then, they have released a couple of dedicated embedding models. For example, you can read their blog post on voyage-context-4.<br>The key premise in those sorts of models is that you can feed the model text of any size, and it will automatically handle generating embedding vectors, smart chunking, providing context, etc.<br>I ran into this recently and was curious to see how this can work. In particular, since RavenDB handles both embedding generation and vector search, I decided to do a full evaluation of MongoDB’s way of chunking. MongoDB built their own model to achieve this, but RavenDB’s approach to embedding generation is to rely on any embedding model you prefer to use.<br>Before we get into the full details, let’s talk for a second about what the point of contextual embedding is, so we are all on the same page.<br>Embedding models take your data and translate it into a multidimensional mathematical space based on its meaning. Similar items will be located near one another in this multidimensional space, and we can take advantage of that using vector search. That is why you can find Mozzarella & Ravioli if you want Italian food today, as in this example:
The problem is that all embedding models have a context limit. There is only so much text that you can push into the model before it will give up on you. If you want to search through a much bigger piece of text using semantic search, you need a different approach.<br>The industry standard approach to handling this is via chunking. In other words, you take a long piece of text, split it into separate parts called chunks, and generate an embedding for each one separately.<br>The easiest way to think about this is that you have a long document, and you generate a separate embedding vector for each page of text independently. Instead of having to digest a whole article, you feed a bounded chunk (page) to the model to generate an embedding vector.<br>Chunking is a neat trick, but it leads to its own set of problems. Assuming we have a large document that talks about new features in RavenDB, with a particular page that expounds on the details of “the database’s ACID guarantees". What would the embedding vector for that page look like?<br>If we just chunk the data naively, we’ll get a vector that is related to the generic concept of ACID in databases. The chunking approach loses the context of the data; it doesn’t understand that the database in question is RavenDB.<br>Contextual embedding allows you to bake a global perspective directly into every chunk’s embedding. In other words, the embedding for that page would know that the database that is being talked about is RavenDB.<br>If you are dealing with large texts and want to have high-quality search, contextual embedding is a feature you want. I guess that explains why MongoDB paid 220 million dollars for Voyage AI.<br>Sadly, I left that sum of money in my other pants, so RavenDB’s strategy for dealing with this scenario is quite different. We planfor models to become a commodity, so there is little benefit in trying to produce your own models at this point in time.<br>Instead, RavenDB takes the approach of working with all off-the-shelf models. That means that we are far more flexible, using the latest state-of-the-art models, instead of having to keep chasing them. But only some models support contextual embedding…<br>Luckily, we figured out that we can add this feature from RavenDB’s side, without needing to develop a custom embedding model for this. The technical announcement about it is here, with all the details. But the gist of it is that RavenDB allows you to attach context to the value you send for embedding.<br>The scenario below shows an example of storing litigation files using RavenDB and enabling proper semantic search over large amounts of data:<br>const tokenCount = 2048;<br>const overlap = 128;
const chunk = (field) => text.splitParagraphs(field, tokenCount, overlap);
embeddings.generate({<br>FullDetails: chunk(this.FullDetails),<br>CaseSummary: chunk(this.ExtractedSummary),<br>PartiesInvolved: chunk(this.MetadataParties),<br>Precedents: chunk(this.CitedAuthorities),<br>RatioDecidendi: chunk(this.CoreLegalRules),<br>ObiterDicta: chunk(this.DissentingArguments)<br>})<br>.withContextPrefix(this.Headline);<br>You can see that we generate embeddings for quite a few fields. For all of them, we use a chunking strategy of 2K tokens with an overlap of 128 tokens. Note the last line that adds a withContextPrefix call, where we add the Headline as part of the context for the data we’ll be...