Thread: Differentiable Self-organizing Systems
Distill
Thread: Differentiable Self-organizing Systems
How can we construct robust, general-purpose self-organising systems?
Authors
Affiliations
Alexander Mordvintsev
Ettore Randazzo
Eyvind Niklasson
Michael Levin
Allen Discovery Center at Tufts University
Sam Greydanus
Oregon State University and the ML Collective
Published
Aug. 27, 2020
DOI
10.23915/distill.00027
Self-organisation is omnipresent on all scales of biological life. From complex interactions between molecules<br>forming structures such as proteins, to cell colonies achieving global goals like exploration by means of the<br>individual cells collaborating and communicating, to humans forming collectives in society such as tribes,<br>governments or countries. The old adage “the whole is greater than the sum of its parts”, often ascribed to<br>Aristotle, rings true everywhere we look.
The articles in this thread focus on practical ways of designing self-organizing systems. In particular we use<br>Differentiable Programming (optimization) to learn agent-level policies that satisfy system-level objectives. The<br>cross-disciplinary nature of this thread aims to facilitate ideas exchange between ML and developmental biology<br>communities.
Articles & Comments
Distill has invited several researchers to publish a “thread” of short articles exploring differentiable<br>self-organizing systems,<br>interspersed with critical commentary from several experts in adjacent fields.<br>The thread will be a living document, with new articles added over time.<br>Articles and comments are presented below in chronological order:
Building their own bodies is the very first skill all living creatures possess. How can we design systems that<br>grow, maintain and repair themselves by regenerating damages? This work investigates morphogenesis, the<br>process by which living creatures self-assemble their bodies. It proposes a differentiable, Cellular Automata<br>model of morphogenesis and shows how such a model learns a robust and persistent set of dynamics to grow any<br>arbitrary structure starting from a single cell.
Read Full Article
This work presents a follow up to Growing Neural CAs, using a similar computational model for the goal of<br>digit “self-classification”. The authors show how neural CAs can self-classify the MNIST digit they form. The<br>resulting CAs can be interacted with by dynamically changing the underlying digit. The CAs respond to<br>perturbations with a learned self-correcting classification behaviour.
Read Full Article
Here the authors apply Neural Cellular Automata to a new domain: texture synthesis. They begin by training NCA<br>to mimic a series of textures taken from template images. Then, taking inspiration from adversarial<br>camouflages which appear in nature, they use NCA to create textures which maximally excite neurons in a<br>pretrained vision model. These results reveal that a simple model combined with well-known objectives can lead<br>to robust and unexpected behaviors.
Read Full Article
This work takes existing Neural CA models and shows how they can be adversarially reprogrammed to perform novel tasks.<br>MNIST CA can be deceived into outputting incorrect classifications and the patterns in Growing CA can be made to have their shape and colour altered.
Read Full Article
This is a living document
Expect more articles on this topic, along with critical comments from<br>experts.
Get Involved
The Self-Organizing systems thread is open to articles exploring differentiable self-organizing sytems.<br>Critical<br>commentary and discussion of existing articles is also welcome. The thread<br>is organized through the open #selforg channel on the<br>Distill slack. Articles can be<br>suggested there, and will be included at the discretion of previous<br>authors in the thread, or in the case of disagreement by an uninvolved<br>editor.
If you would like get involved but don’t know where to start, small<br>projects may be available if you ask in the channel.
About the Thread Format
Part of Distill’s mandate is to experiment with new forms of scientific<br>publishing. We believe that that reconciling faster and more continuous<br>approaches to publication with review and discussion is an important open<br>problem in scientific publishing.
Threads are collections of short articles, experiments, and critical<br>commentary around a narrow or unusual research topic, along with a slack<br>channel for real time discussion and collaboration. They are intended to<br>be earlier stage than a full Distill paper, and allow for more fluid<br>publishing, feedback and discussion. We also hope they’ll allow for wider<br>participation. Think of a cross between a Twitter thread, an academic<br>workshop, and a book of collected essays.
Threads are very much an experiment. We think it’s possible they’re a<br>great format, and also possible they’re terrible. We plan to trial two<br>such threads and then re-evaluate our thought on the format.
Editorial Note
Part of Distill’s mandate...