Show HN: WiFi-LLM

bertaye2 pts0 comments

Hello, while I was learning LLM Inference basics I realized with few tricks I could use my ESP32 for inference if I stream the weights only when they are needed from a host over WiFi.Speed? Really bad. Lol. But this is just a technical demo of how it can be done. I really enjoyed while coding it. It has a sliding window which allows infinite generation ( of course meaningless :) )Hope you will enjoy!

wifi while inference really show hello

Related Articles