Is 99.9x% cache hit always better?

ozer_231 pts0 comments

Support a problem needs at least 10k tokens new information.If cache hit is 99% ... input 0 cache created 10k (prefect) cache read 1M!Extremely, if a clever agent get all 10k information, and give the answer directly, the cache hit is 0%!Cache hit rate should be healthy, but if it gets 99.9x%, I think it is that the information incr is too small per step!

cache information always better support problem

Related Articles