Ask HN: Are current on-device LLMs useless?

jdthedisciple1 pts0 comments

I found this model lfm2.5-thinking which is designed for on-device deployment and is really tiny: just 1.2b params (~700 MB)My first and very short interaction was so terrible that I felt no need to continue, but not in the way you think:Here is the extremely brief transcript: hey Thinking... Okay, let s see what the user wrote here. They just said heat . Hmm, maybe they want a response to that? ... First heat could be a standalone message... You saw that? It misread my hey as heat , and kept going like this throughout the entire thinking process.I thought I know what to expect from tiny local models like this.But I did not expect it to be this bad. How would this be useful for anything local if it cannot even read a three letter-word properly?Is this just lfm2.5 being terrible, or is this level of error a common trait in this model size range. Curious to hear from other folks experiences.

quot thinking heat device model lfm2

Related Articles