That assumption breaks down in real traffic, especially when the traffic is multilingual or domain-specific. A general embedding model, trained on broad web text, does not naturally separate queries that look alike but mean different things. Consider a customer who asks "how do I cancel my subscription" versus "how do I pause my subscription." The word sets overlap heavily, and the cosine similarity between those embeddings may be high. But the answers are different: one involves a permanent action, the other a temporary hold. If the threshold is set too low, the cache returns the wrong response. These are called false cache hits, and they are more common than many engineers expect.
A 2026 preprint by Guerin Green of Novel Cognition research Cognition examined this problem directly. The author built a labelled set of 243 English customer-support question pairs across eight domains. The set included 101 heavily reworded paraphrases, 102 near-miss pairs that differ by a single word or by word order but need different answers, and 40 lightly reworded paraphrases. Using the open embedding model Qwen3-Embedding-0.6B, the results were striking. The near-miss pairs scored higher by cosine similarity, with a median of 0.801, than the genuine paraphrases, which had a median of 0.691. The area under the ROC curve was 0.144, with a 95% bootstrap interval from 0.097 to 0.195. That means the distributions were inverted: the model considered near-misses more similar than true paraphrases.
At a similarity threshold of 0.85, which is the default of one commercial service, the model matched none of the 101 paraphrases and 29 of the 102 near-misses, or 28.4 percent. Raising the threshold did not cleanly fix the problem. False matches only reached zero above 0.985, but at that level, 2 of the 40 near-verbatim repeats were still missed. So a single threshold cannot both catch paraphrases and avoid near-miss errors on this traffic.
The preprint also found that similarity tracked word-set Jaccard overlap, with a Pearson correlation of 0.695, even within each label. That suggests the embedding model was relying on surface lexical overlap rather than deeper meaning. Adding a query-instruction prefix halved the false matches at 0.85, bringing them to 12.7 percent, but it did not change the ordering of the pairs. In other words, the prefix helped but did not rescue the threshold.
The key argument from the preprint is that a similarity threshold is not a transferable constant. It is a property of one embedding model on one traffic distribution. What works for general English queries will not necessarily work for domain-specific terms or for multilingual queries, where embeddings often behave differently. A threshold that is safe for one language may be too loose for another, or too strict for a third. The author released the pairs, the scores, and an analysis script under CC BY 4.0, so practitioners can recompute every statistic and apply the same labelled near-miss protocol to their own traffic.
For engineers building semantic caching, the practical lesson is to measure, not assume. The default threshold in a library is a starting point, not a guarantee. You need a labelled set of near-miss queries from your own domain, scored with your own embedding model, to find a threshold that balances cache hits against false cache hits. Other embedding models will need their own measurement. A single number cannot carry the weight of multilingual, domain-specific traffic. The threshold is a decision boundary, and like any decision boundary, it only works if it is calibrated to the data it actually sees.