Is symbolic AI more relevant than ever? | heise online

heise+ entdecken

SuchenAbo

Suchen

Alle Magazine im Browser lesen<br>AnzeigeSpecial: Collaboration im KI-Zeitalter

Newsletter<br>heise-Bot<br>Push-Nachrichten

${lead}

${lead}

${content}

${content}

${content}

${content}

Anzeige<br>Special: Collaboration im KI-Zeitalter

Advertisement

Advertisement

When artificial intelligence (KI) is discussed today, it's almost exclusively about large language models. And thus, without it always being explicitly stated, about a very specific type of AI: about neural networks, about statistical learning from vast amounts of data. The implicit promise is that the way forward is primarily a question of quantity. More parameters, more data, more computing power, more energy, and a little patience. Then the rest will follow automatically.

Continue after ad

I want to question this assumption. Not because I want to downplay the successes of recent years; they are real and impressive. But because a suspicion won't let me go: Perhaps we are sitting in a local maximum and mistake it for the summit. And perhaps a look back can help to see that this summit is not the only one. Because the pendulum of AI research has swung elsewhere before.

Golo Roden is the founder and CTO of the native web GmbH. He works on the design and development of web and cloud applications and APIs, with a focus on event-driven and service-based distributed architectures. His guiding principle is that software development is not an end in itself, but must always follow an underlying technical expertise.

A pendulum that has been swinging for decades

It's worth briefly recalling that today's dominant, data-driven AI is by no means the only option. For decades, the prevailing paradigm was completely different: symbolic AI. It assumed that intelligence fundamentally consists of manipulating symbols according to explicit rules, meaning that thinking is something that can be written down and understood.

This idea was not a footnote of a few years. It ranges from the famous Dartmouth Conference in 1956 to early systems like the Logic Theorist and the General Problem Solver to the expert systems that were celebrated as a commercial breakthrough in the eighties. For about three decades, symbolic AI was not just one trend among others but simply what was understood as AI.

This approach did not fail due to naivety, as is often told in retrospect. It failed due to two very concrete problems: scaling and brittleness. Anyone who has to manually input knowledge rule by rule cannot keep up with the complexity of the real world at some point. And anyone who relies on rigid rules will have their system break as soon as reality does not conform to the intended cases.

The decline of symbolic AI was accompanied by two so-called AI winters, phases in which expectations were disappointed and funding was cut. That the learning approach triumphed afterward had less to do with the theoretical superiority of an idea than with two sober prerequisites that were suddenly met: sufficient computing power and sufficient data. Only when both were available in abundance could neural networks show what they are capable of.

Continue after ad

So, the connectionist learning approach, which does not rely on predefined rules but on statistical patterns in data, stepped into this vacuum. The pendulum swung from one side to the other. And it has been swinging ever further in the same direction ever since, to the point where hardly anyone seriously considers alternatives today. That's precisely what I consider a mistake.

Is more computing power really more understanding?

The bet of the present is that the remaining weaknesses of neural models can be scaled away. Larger models, more training data, and the gaps will close. This expectation is not unfounded, as many capabilities have indeed only emerged with size. The question is only whether this applies to all weaknesses or whether some of them are structural.

Cognitive scientist Gary Marcus has formulated this criticism early and concisely. In his much-discussed essay “Deep Learning: A Critical Appraisal” from 2018, he lists ten problems that, in his view, cannot be solved by scaling alone. These include the enormous hunger for data, the difficulty of generalizing beyond the training distribution, and above all, the lack of compositionality and systematic reasoning.

Compositionality refers to the ability to combine known building blocks into new, never-before-seen combinations while remaining reliable. A human who knows the meaning of words and some rules can form and understand sentences they have never heard before. Purely neural systems are surprisingly unreliable in this regard. They shine on the surface and falter in depth, producing brilliant surfaces and stumbling over simple but systematic conclusions.

In addition, there is an economic observation. The gains from pure enlargement do not follow a linear curve; they flatten...