Nuance in all things. A dive into (Anti-) “AI” Myths | K4tana - Cutting Edge Research Blogs

“AI” is weird, I get it. Somehow, it’s everywhere, and chances are you either hate it with a passion or love it and wish your entire life could make use of it. Sure, that’s your right. And I’m not saying you’re wrong. Indeed, I have used “AI” before and still think it’s an interesting tool when you want to get into a subject that is fairly technical and invites to hands-on learning. But this text is not about that. This text initially started as a rant about people complaining about “AI”, but has transformed to a piece about the basic properties of “AI”, why I think most people talking about “AI” are wrong in one aspect or another, and why I think this will ultimately crash. But first, let’s get some tech facts straight.

“AI” is a bad term.

I prefer using Machine Learning (ML) for the “old” algorithms like random forest, neural nets or SVMs, GANs etc., and “transformers” or “LLMs” for the newer model-combinations underpinning chatgpt, claude, llama et al. While the former models are all relatively old – neural nets were invented as a “perceptron” by a neuroscientist in the 80s – their explosion happened with the computation revolution from 2000 onwards. Cluster computing, the mass adoption of the internet and easy cash due to the bull market before 2008 made it easier to gather training data, build infrastructure and develop models. Since then, models like decision trees (CARTs), SVMs and such are considered “off the shelf” models, with a rich knowledge base behind them. You can train such models on an average computer, with decent results. And they are everywhere. When you swipe your credit card somewhere, one of these models will predict the risk of you defaulting on the credit in milliseconds and either block or greenlight your payment. When you click on something on Youtube, a system will integrate the data and recommend you new videos based on some optimization algorithm based on an off-the-shelf system. These things are computationally efficient and easy to implement. “AI”, if you will, is everywhere. Has been for close to a decade now.

Now, transformers are a different beast altogether. Their main hallmark was initially proposed by Google. The idea behind it is basically simulating a large net of neurons and making it computationally efficient, giving it the possibility to weigh certain parts of the input more or less than others, and cross-referencing data better through a “knowledge map”. This leads to a similar mechanism our brain implements: We heavily filter our sensory input and only retain the important stuff. By doing this, the machine can be more efficient in dealing with information and give you a better result. The output of a transformer is purely statistical: Given input A, it will generate an output B that has, according to their inner wiring and training data, the highest probability of fitting the input. If you ask it, “How are you?”, it will answer “Fine, how are you?” back. It fits statistically, because its training data showed this pattern of communication very often, making it likely that “Fine, how are you” is the best response to your question. That’s basically how chatgpt works. For context, this is not how all transformers work, there are also other ones for visual generation, for translation and other things. I will focus on chat and image transformers in this piece, since those are the ones currently talked about the most. So, now that you have a bit of an understanding, let’s get to the most uncomfortable stuff.

Most people talking about “AI” are probably wrong

The attentive reader will have noticed that I am, right now, writing about “AI”. And I will not exclude myself from belonging to “most people”. Thus, I will preface the following text with a disclaimer: I’m not a “trained” data scientist (although data science is what I’m hoping to do after my PhD), I have no in-depth knowledge of how the large transformers and cutting-edge models work. That being said, I did have above average training in “AI” and statistics during my studies and continue to learn more about the subject due to my job and just because the topic is interesting. This means: Take everything here with a grain of salt. I think I know what I’m talking about most of the time, but the burden of falsification is on you, and everyone can make mistakes (especially scientists, and especially me). Alas, let’s start with the most obvious fact about “AI”.

“AI” is hype tech

There seems to be no middle ground when it comes to opinions about “AI”. I find this funny, because this is basically what any hype technology has ever evoked in people. Extreme polarization is a hallmark of new tech. Here’s a few examples: Train rides were once suspected to “injure the brain” when they became popular. Hell, even bicycles caused adverse reactions when they became a serious means of transport for women. In Germany, there is a...