W. J. Zeng
"I feel like I just have a few years yet to prove some interesting results before AI takes over." - My<br>friend in a Princeton bar after giving a talk at the IAS on his work building towards classifying topological<br>phases.
An imagined conversation:
Arnheim stood looking out the window with his hand in a fist. He'd said off hand that when there is a<br>super-intelligent AI, then that AI will give us all the answers. There'll be no reason left to do science, or at<br>least there won't be any joy left in it. He clenched his fist.
Ulrich watched that tension express in Arnheim and adopted a counter-pose. He relaxed back in the soft<br>upholstery, crossing one leg over the other and draping his arm over the back of the chair.
Arnheim couldn't see this of course; perhaps Ulrich was performing for himself. Ulrich said that science is<br>fundamentally a human activity and that this won't change with the advent of AI. Science has always been about<br>what humans know, and more than that, at its best and since its founding it's been a personal activity. The<br>motto of the Royal Society is, after all, nullus et verba: take nothing on someone's word. He said science will<br>continue on because an AI knowing something doesn't cause any personal or human knowledge to appear. Ulrich then<br>felt this wasn't sufficiently concrete, so he appealed to an example. Imagine if you knew for sure that there<br>was a race of super-intelligent aliens somewhere in the galaxy that are immensely more advanced in their<br>sciences than we are. Does their existence dissuade you from doing science here on Earth?
Arnheim turned now and explained that the super-intelligent future is one where those aliens are not somewhere<br>vaguely out there, but in direct and close contact with us. Given this, wouldn't we spend a lot of time asking<br>them questions instead of doing science ourselves? Or at least the activity of science would change from asking<br>questions of nature directly to asking questions of the aliens and verifying their answers? Whatever this<br>version of science is, he continued, it seems to be entirely different; true discovery is replaced with its<br>shadow: validation activities. Arnheim gestured around the room as he continued. Further, the collaboration<br>between humans would change. Instead of querying each other, we'll each of us keep going back to the aliens/AI<br>super intelligence. Where has the energy and the community of science gone when we are each already asking each<br>other what our AI chatbots have to say on the topic?
Arnheim started to pace over to the fireplace and continued. A super intelligent AI upends science even more than<br>the aliens because (if suitably aligned) it will be extremely good at explaining things to us. In this world,<br>even if a human discovers something truly novel then the first thing to do will be to tell the AI about it.<br>Other humans will have the AI explain it to them, probably better than the discovering human would. Perhaps the<br>discovering human will have the AI explain it back to them in ways better than they could have done on their<br>own. Arnheim ended by questioning the room, "Is this what the warmth of scientific community is to be replaced<br>with? Science into solitary."
Ulrich was less sure that solitary science is such a bad thing, but he couldn't resist commenting in a way that,<br>more accidentally than not, might have been seen as trying to assure Arnheim. Ulrich said that we are already in<br>a world where direct human activity for discovery has been decreasing. Neither scientific journals nor the arXiv<br>have degraded the character of science, although we now interact with such artifacts rather than reproducing<br>experiments. Is a super-intelligent AI not just an extension of this? Then Ulrich went off script a bit. Ulrich<br>said that in a way this trajectory of impersonalization in learning about reality goes all the way back to at<br>least the invention of writing. He called Arnheim a Socrates who is chastising his students about how writing<br>would dull the mind's experience of personal knowledge by projecting it out into words and letters. Or perhaps<br>Arnheim would like to go back even further and take issue with the invention of language itself? Wasn't it<br>language that interrupted the truly personal experience of reality? [c.f. The Truth of Fact, the Truth of<br>Feeling by Ted Chiang]. With language you can ask someone for knowledge. The average person you meet on the<br>street might not be a super-intelligence but probably they know something you do not, and that dissuades you<br>from finding out on your own. Or would Arnheim go even further to the moment of sentience as the break where we<br>lost control of our own real joy and discovery in pure experience?
At this last point Ulrich had to stop somewhat abruptly because he realized he might agree with it. Arnheim<br>failed to notice the weakness though, and so after a short pause Ulrich continued. Isn't the super intelligent<br>AI future one where we just extend along this curve from...