/thoughts - AI | reality | brain

Recently, our team bought a B300 GPU system.<br>Just look at this modern technological marvel.<br>Supermicro Gold Series AS -8126GS-NB3RT | 8U GPU Server<br>Ready to deploy AS -8126GS-NB3RT ships within 24 hrs. DP AMD 8U System with NVIDIA HGX B300 8-GPU<br>SupermicroSupermicro

I grew up with the beloved family PC: dial-up Internet, CRT monitor, 20GB of hard drives, entire drawers of CDs filled with movies and music and games, barely a gig of RAM, I don't even remember what the CPU was, just the peeling blue Intel sticker. Dad used to take files to work on 3.5" floppy disks, one of which I broke by playing around with the write-protect tab.<br>Now, I'm staring at a machine with 2.5 terabytes of cutting-edge GPU VRAM (not to mention the massive multi-GPU system itself), 70 terabytes of NVMe SSDs, 3 terabytes of RAM, powerful AMD Epyc workhorses. The NICs have port capacities that literally require the high-speed network PHY systems of the kind that we at Marvell make.<br>I know not what to make of the callousness with which we treat modern technological systems in everyday life, when in retrospect, we've essentially made rocks think for us, to borrow a quote from someone on the Internet. Instead, now you get to enjoy my unprovoked ramblings from a restless freight train of thought at 2AM.<br>Like most people in this industry, I too have been obsessed with LLMs, but not in the megalomaniacal, happy-to-take-a-blowtorch-to-the-team's-AI-budget kind of way. My mind has simply been churning constantly on what this means for tech, for me and mine, for society.<br>Half my team has embraced the AI-first method of working like people possessed, spearheaded by my manager, and believe we will all be out of a job in a few years. Meanwhile, to me it feels like the current industry state is propped up on hype and falsehoods and a temporary race for infrastructure buildout. Ethics and morality have been thrown away with no regard for humanity and critical thinking and sustainable development.<br>And all the usual stuff continue blissfully: political climates worsen everyday, the actual climate isn't far behind, we're plagued by resource shortages and geopolitical shifts and wars and the ever increasing class imbalance and wealth disparity, etc. etc.<br>What I dream of is nuclear fusion, autonomous robotics, better human rights, maybe UBI, democratized resources and technology, free and advanced healthcare... the list goes on.<br>What we get is advances in predatory, AI-driven, psychology-exploiting advertising, relentless slop generation as if there wasn't enough human-made slop on the Internet already, rising costs, resource shortages...<br>Where is the future I dreamt of? The one I envisioned as a hopeful teen, the one that everyone spoke of in wistful and excited tones?<br>At least my work isn't actively harmful and is arguably ethically neutral, as it's just engineering. But what is it in service of? What do I have control over? What must I strive for that I can reasonably achieve in this lifetime that leaves the world in a state better than the one I experience it in?<br>What is intelligence? Sentience? Do current LLMs have it? What would it take, to replicate the human brain and its neural pathways? Is that even the path we want to take, or is there a fundamentally different way of reaching AI? This is what the whole field of symbolic AI, ontologies, knowledge representation and reasoning, first-order logic, and other concepts deal with. But unlike the original perceptron, i.e. the first artificial neuron, which was modeled after the human neuron and designed to recreate the physical building block of the human brain, the core unit of symbolic AI is first-order logic and conceptual connectivity, which is something that humans gleaned from our own working model of our reasoning process. And if human intelligence is fallible, then so is any construct we come up with which cannot be validated by the cold confines of reality. Yet we as humans stubbornly believe we are fundamentally different and 'better' than artificial neural networks just because we can do such 'reasoning' in a deterministic and logical manner. But who is to say that determinism is desired in an intelligence system? What if determinism is but a way of facilitating true intelligence's capabilities? Why must it be a requisite part of intelligence itself?<br>A counter argument: if we were to consider symbolic approaches to be superior to statistical ones, what then? What if we were able to design an ontology engine with the sum of all human knowledge? Would it scale? Decades of research point otherwise... And yet, statistical AI was also ridiculed the same way before symbolic AI for the very same reason. And the game changed as hardware advanced magnitudes of power over the years, and GPUs made statistical AI a reality. What if the breakthroughs in computing technologies unlock architectures of scale where reasoning engines that could hold the world's knowledge...