To Understand Language is to Understand Generalization | Eric Jang

In my essay "Just ask for Generalization", I argued that some optimization capabilities, such as reinforcement learning from sub-optimal trajectories, might be better implemented by generalization than by construction. We have to generalize to unseen situations at deployment time anyway, so why not focus on generalization capability as the first class citizen, and then "just ask for optimality" as an unseen case? A corollary to this design philosophy is that we should discard inductive biases that introduce optimization bottlenecks for "the data sponge": if an inductive bias turns out to be merely "data in disguise", it may not only cease to provide a benefit in the high data regime, but actually hinder the model on examples where the inductive bias no longer applies.

Pushing as much human-written code to "language + deep learning magic" sounds like a lot of fun, but how does a practitioner execute this "just-ask-for-generalization" recipe? If we want to infer optimal behavior via a deep neural net without training explicitly for it, we need to answer hard questions: given a model family and some training data, what kinds of test examples can we expect the model to generalize to? How much is "too much to ask for"?

And how do you define generalization, anyway? ML theory gives us some basic definitions like generalization gap and excess risk (i.e. the difference between training and testing losses), but such definitions are not useful for estimating what it takes to achieve a qualitative degree of capability not seen in the training data. For example, if I am training a household robot to be able to wash dishes in any home, how many homes do I need to collect training data in before the learned policy starts to work in any kitchen? This practical question comes in many formal disguises:

"What data is out-of-distribution?"

"Is my model + data robust to adversarial examples?"

"How can we train models to know what they don't know?"

"What is extrapolation?"

Like the parable of the blind men and the elephant, computer scientists have come up with different abstract frameworks to describe what it would take to make our machines smarter: equivariance algebra, causal inference, disentangled representations, Bayesian uncertainty, hybrid symbolic-learning systems, explainable predictions, to name a few.

I'd like to throw in another take on the elephant: the aforementioned properties of generalization we seek can be understood as nothing more than the structure of human language. Before you think "ew, linguistics" and close this webpage, I promise that I'm not advocating for hard-coding formal grammars as inductive biases into our neural networks (see paragraph 1). To the contrary, I argue that considering generalization as being equivalent to language opens up exciting opportunities to scale up non-NLP models the way we have done for language.

Compositionality in Language

Hupkes et al. 2020 discusses a few different aspects of "compositionality" in language models. Language is nothing more than the composition of a discrete set of tokens, so what the authors are really doing is specifying a grammar on how the smallest units of discrete meaning (words) fit together to form new meanings, i.e. the structure of language itself. Here is a table in which I've paraphrased the definitions and provided some training examples and test-time capabilities.

Generalization Type<br>Definition<br>Training Examples<br>Testing Examples

Systematicity<br>Recombine constituents that have not been seen together during training<br>{"Bob ate pizza", "Alice ran home"}<br>"Bob ran home"

Productivity<br>Test sequences longer than ones seen during training<br>books with 100k-200k words.<br>books with 200k+ words.

Substitutivity<br>Meaning of an expression is unchanged if a constituent is replaced with something of the same meaning<br>"bob ate pizza for lunch"<br>"bob had Dominos at noon" taken to mean (almost) the same thing

Localism<br>The meaning of local parts are unchanged by the global context.<br>Arithmetic tasks like {(5)-4, (2+3)}<br>(2+3)-4 : (2+3) locally evaluates to 5, then 5-4 locally evaluates to 1. (2+3) representation not influenced by the presence of -4.

Overgeneralization<br>Correctly handle exceptions to rules and patterns<br>{live->lived, laugh->laughed, love->loved}<br>{kill->killed,break->broke (not breaked)}

Compositionality in Everything Else

Hupkes' categorizations of compositionality can be applied to non-NLP domains as well. In "A Survey of Generalization in Deep Reinforcement Learning", the authors provide an intuition of how these categorizations can be applied to a robotic block-stacking task:

Systematicity - Stack blocks in new configurations not seen in training

Productivity - Stacking more blocks than was done in training

Substitutivity - Stacking blocks it hasn't seen before (e.g. understanding that block color does not affect physical properties)

Localism - Position of...