Eukaryotic Virtual Reality

Eukaryotic Virtual Reality
The Emergent Art of Artificial Life

Jeffrey Ventrella, September, 1995

(published in ISEA (Electronic Arts) Conference Proceedings
and presented at ISEA 95, Montreal Canada, Sept. 1995)

1. Introduction

In this paper I discuss an approach to the science of artificial life, from the viewpoint of an artist using techniques from this field to create images, animations, and interactive microworlds, via computer programming. As a point of departure, I consider current popular forms of virtual reality-in which the human subject is immersed in a simulated three-dimensional space, usually represented visually via established computergraphic rendering techniques. These spaces have been described as "lonely", because they often lack the element of life we encounter continuously in real reality.

I would like take a more ecosystems approach to inventing a virtual reality, where experience is characterized by interactions with adapting artificial life forms, rather than an approach where experience is characterized by moving through perspective space as a virtual eye-self. The information-dynamism of artificial life does not always depend on rendering of visual surfaces in 3D spaces, and instead emphasizes the inner process, growth, adaptations, and interactions of various autonomous agents, possibly including a human participant. It emphasizes behavior.

Artificial life has been primarily a scientific discipline, aimed at complementing traditional biology (largely an analytic science - the study of carbon-based earth-life), with synthesis - a form of theoretical biology. Artificial life research abstracts the functions of life away from one particular physical manifestation and attempts to understand it in terms of information dynamics. For this reason, many artificial life artifacts take the form of computer programs which exhibit emergent properties reminiscent of life.

Some artists and computer graphics researchers have begun to adopt artificial life principles and techniques in developing visual works and mechanical automata. When artificial life is viewed as a new experimental artform, a different set of issues may arise, issues concerned with representation, cultural implications, questions of authorship, and the creative process. In this paper, I trace the discoveries in my personal journey as an artist inspired by biology, who became a self-taught computer programmer. I also cite a few key thinkers and makers who are starting to bring artificial life into the realm of a visual and cultural study.
Euclidean Reality
A kind of giddiness, perhaps experienced by Renaissance folks, when they figured out the perspective trick, is back again - but it is cast in a new light: immersive 3D computer graphics. These new representations of reality take the method of perspective into a new realm - projective geometry is encoded as a real-time animation algorithm. And now people can go into these virtual places and look at what lies behind the picture plane that isn't there. It is the newer virtual reality of illusionistic space - and it is tied ultimately to the single viewpoint of the self.

But for me, a more intriguing virtual reality that is emerging in our culture is not about 3-space, computing vanishing points, or being Euclidean. Artificial life renders the complexity of dynamical systems - chaotic realities, where boundaries are blurred, and viewpoint may be fragmented or many, or moot.

Here, metaphors come from biology and the study of nature, in which myriad living agents are intertwined in an ecological whole which is larger than the sum of its parts.

Virtual reality doesn't have to be a lonely place. Many of us who are building computer-based media have an agenda: to invent and then populate virtual realities with interacting, adaptive, quasi-intelligent entities. The human participant can become one member of an ecological system, not merely a lone wandering self in a space of Euclidean objects. Real reality is, in fact, very much about living among - communicating with, eating, and (for some), being eaten by - other living things.
Eukaryotic Reality
Lynn Margulis theorizes that certain bacteria of the early earth evolved such a tight symbiosis that they merged as meta-organisms, composite individuals - the eukaryotes (1) . This is one of many critical shifts in biological evolution, where "individualness" blurs. And the cells which physically comprise you the individual happen to be eukaryotic cells. What of meta-human systems? Cities, nations, languages, and the robotic civilizations expounded by Hans Moravec (2). Chris Langton suggests that human culture should be studied with the same lens as that which we use to study nature (3). They're both parts of one emergent phenomenon - life.

Then there's the internet: this massive, decentralized organism evolves around us, like a cortex - we are immersed within it.

This net is not a visual space that we occupy, as Bill Mitchell reminds us in City of Bits (4). It is cyberspace. And it's an experience about many interacting entities - a plurality of contributors to the swarm. As a network, it has its own emergent properties, like living phenomena. Kevin Kelley calls this a "vivisystem" (5). And this vivisystem includes both the human and the post-human. In my lifetime I may engage in conversational email with an AI program or have screensaver fish as pets that do my spellchecking. This multifaceted world - where one can practice gender-bending for a new experience in virtual sex, or where one can play multi-player videogames with a combination of real friends and artificially intelligent cartoon characters - is the most interesting virtual reality to me.
Here I wish to recognize a virtual reality which is not characterized by sleek, surreal polygonal spaces, such as we've seen popularized in the media. It is a virtual reality characterized by livingness, which can inhabit many kinds of visual forms, sometimes not visually recognized as familiar objects we've seen in the world - perhaps not having any attributes which would allow it to be rendered in any classical computergraphical sense. But it may exhibit behaviors to an oddly familiar tune, like something we've experienced at some point in our lives as earthlings among other earthlings. Take the cellular automaton known as "Life" (6).

It intrigues, mesmerizes, and inspires the creation of entire periodicals and books, though it consists merely of a rigid square grid of black and white squares blinking on and off, according to a few basic rules. Why such mass interest in such a simple visual world? It has to do with behavior. More recently, at a workshop on artificial life at MIT, clunky assemblages of blocks and simple stick figures moving about in sparsely-rendered 3D spaces have gained attention - why? They demonstrate evolution as a way for a population of these sparse objects to learn to walk or swim (7). It's in the behavior.

2. Designing Emergence
Another Kind of Paintbrush
I am interested in painting expressions of reality with another brush than the kind offered in commercial 3D modeling, animation, and rendering packages. I am interested in composing computer programs which in turn generate dynamical systems with many interacting parts, buzzing like life, and not being constrained to any particular dimensionality. Ten years ago I made a strong separation from the arts of rendering optical reality. I discovered an entirely new visible language tool - available in it's most basic form only to computer programmers: recursion: self-consuming, self-feeding loops which can generate non-Euclidean spaces, chaotic attractors, growing biomorphs, and self-replicating cellular configurations.

These ever-changing visual forms spoke to me of something in the world that is deeper than what lies on the optical surface (where classical computer graphics systems texturemap polygons and raytrace shadows).

In the view of some artists, the computer is best exploited not a glorified crayon, nor even as an enhanced, magical canvas to be painted over with intelligent images. It is a process machine. It is about process - more like a musical instrument than a paint brush. And behaving more like a mind, a living system, than like a camera or an airbrush.
Genetic Tweak Animations
Years ago, as I was developing hand-drawn animations, a friend suggested that I should get to know his Amiga computer and check out a program that lets you draw simple fractals in BASIC. Being computer phobic, I was defiant at first, and preferred to keep drawing animation cels by hand. But in the span of a few hours I became fascinated with this new way of describing an image, and a transformation of that image. With this program (which looked like alphabet soup to me), I could (very clumsily) edit the first in a string of four numbers in the midst of the soup to be slightly different than before, run the program, take a frame of the computer screen with my movie camera, and repeat the process over and over again.

The changes in the images implied that the developed film would reveal a complex organism which gracefully writhed in harmonic motion, like some underwater sea creature. As I began to contract the flu late into the night, and as my thumb began to form a blister from repeatedly clicking the camera, I began to recognize the significance of the forms and motions that were unfolding in the as yet undeveloped film: simple, linear changes in one small number in the series begat a cascade of complex hierarchical movements in the form, due to the recursive nature of the fractal. The code was terse and the transformation was simpleminded, but the generated form was chaotic, complex and surprisingly organic.
Tree Epiphany
Soon, I figured out how to create tree shapes using recursive algorithms. My excitement was like that reported by Richard Dawkins when he first brought his biomorphs to life for his Blind Watchmaker software (8). I began to appreciate the vastness of the family tree of trees (a "genetic space"), and had to take a walk in the night air. I began to watch the bare tree branches (it was winter) as they glided past my vision while I walked down the street. I saw the trees in a new way. I saw them growing - I saw them as grown things. This led to a trend of looking at all things as grown from generative forces - interacting with other things as they take shape. It was a new eye on things as procedural things, not designed or created, but evolved through a dynamism.

"Procedural modeling" was a refreshing new approach to many in the computergraphics world who were growing tired of chrome spheres and texturemapped teapots, and worlds constructed with building blocks. Some computer art purists have suggested that the computer, being a dynamic information processing machine, is best at modeling dynamic phenomena: growth, thought, evolution, language, rather than building upon past imaging technologies and artforms. Artist Ed Zajec sees processing techniques such as recursion as a way of expressing with as much potential as the technique of perspective had in the Renaissance (9). Recursion is a motor which aids in the emergence of form, its generation, its development through time. It also appeals to the musically-inclined.
Composing Seeds
Recursion means you can artfully compose a seed (a few numbers, an equation, a geometrical transformation, a musical motif, etc.) and an environment within which that seed will grow, and then let it grow. The grown result can then be evaluated aesthetically and then the seed can be re-designed, in an attempt to create more interesting or expressive grown forms and motions. It's usually difficult to relate the change in the seed with the change in the grown thing, because the "embryology" - the expression from genotype to phenotype, may be complex and unpredictable (10). But after a while of doing this, one can acquire a skill at predicting the results of making such changes to the seed. My explorations in this activity led to an obvious approach to crafting families of images and animations: to use genetic algorithms (the technique of modeling the dynamics of Darwinian evolution in software).

Genetic algorithms are optimizing schemes which use the genetic operators of reproduction, mating, mutating, and dying, to evolve a population of genotypes (the parameters which determine the characteristics of a family of images, for instance), to improve what the genotypes represent: the phenotypes (the images themselves).

The genetic algorithm became a tool to help me breed these images and motions, where my aesthetic evaluations became a "survival of the fittest". This kind of technique is similar to what Karl Sims had developed, using genetic programming (expressions in the LISP language which can mate and mutate) for image breeding and animation scripting. It suggests a possibly whole new kind of artmaking process, which Sims called, "interactive evolution" (11).
Evolving Animals
I had also used genetic algorithms in an experiment for evolving populations of simple, articulated stick figures rendered in a virtual 3D world with gravity, friction, and momentum (12). These were presented at the latest Artificial Life gathering at MIT. Variety in these populations was determined by a genotype (a set of genes controlling parameters for anatomies and motions among the body parts). These very simple shapes evolved seemingly purposeful, and often amusing motions, as the population evolved, driven by fitness pressures for locomotion. They grew better proportioned extremities and rhythmic gaits—and many unexpected solutions to the general task of locomotion.

Another set of creatures I had developed existed in a virtual underwater world. Their evolutionary fitness pressures were for covering more distances in water, in any way they happened to chance upon. In these populations, starting from random primordial beginnings, later generations produced longer limbs and wider, more coordinated strokes.

At the MIT conference, Karl Sims presented a family of 3D land and undersea creatures composed of blocks, whose behaviors and anatomies had evolved purposeful behavior through competitive situations, using the genetic programming technique (13). These objects incorporated a physically-based model and embryological process determining morphology and motion in individuals of the population.

Snapshots of these block assemblages reveal little information, but as soon as they are animated, one can see an obvious lifelike sense of purpose in their motions. This is the outcome of a long line of generations in which the population had adapted to the Darwinian pressures set up by Sims in his simulations.

These kinds of computer animated characters are not designed in the strict sense. They are evolved. Which is not to say there is no Art in their creations. The programmer has unlimited choice in how to represent the scheme for which genes are expressed as body and motion. The programmer also has unlimited choice in how to set up the environment, which kind of physics within which to imbed the creatures, and what the evolutionary pressures will be for survival. In terms of expressivity and narrative potential, this technique is a far cry from classical character animation, in which the artist has very much control of the subtle nuances of movement in a character. But it suggests some new techniques for making virtual characters do some remarkable things without telling them exactly how, and letting their personalities and "body language" evolve, through Darwinian encouragement. It is an art of crafting evolutionary dynamics, of setting up the degrees of freedom for motion and parameters for anatomical variation, and of crafting genotypes and their expressions into phenotypes. My efforts in bringing some of the expressivity and humor of classical animation to this new techniques resulted in an interactive artifact which I call "Disney Meets Darwin" (14).
Grown Art
The American school of Action Painting signifies an art of act, process and emergence. The painter would engage in an energized gestural dialogue with the canvas. Each stroke (or dribble) the painter made spoke back - dictated what the next stroke should be. The continual iteration of the painter's evolving vision, as it interacted with paint and canvas, brought forth a grown form - an expression of a process, something emergent.

This may be said also of painters who use an "automatist" approach, and some abstract surrealist styles, in which, as the artist assumes an altered state of mind, the forms semi-automatically emerge from the canvas. These styles of painting emphasize the organic, and they often bring forth biomorphic forms. Why?
Harold Cohen's lifelong, continually evolving computer program "Aaron", was conceived originally as a means for Cohen to encode his drawing methodology, to represent it algorithmically in the form of an artificial intelligence work or art. Aaron tirelessly generates unique, one-of-a-kind artworks, all of which bear the unmistakable signature of Aaron's master (Cohen himself) (15). Cohen has made a whole years-long art project of representing the process of his artmaking. Which is the real art of Harold Cohen, his brainchild Aaron, or the thousands of works that Aaron has generated, and will continue to generate when Cohen is no longer alive?

The use of evolutionary computation techniques, such as genetic algorithms, suggests new approaches to a familiar notion: art is process. Art and design creations evolve in the maker's mind as well as in the work itself (and they also build upon the evolved visual language which is its cultural context).

In the evolution of a creative work, visions, ideas and methods emerge in the maker, prior to, and simultaneous with the act of crafting them. They change, mutate, some die off, many live on, reproduce, and merge with others. The act of creating is partly an evolutionary act - a bit of bottom-up emergence and serendipity with a bit of top-down design and planning. The evolutionary mind is like a microcosm of nature. Richard Dawkins' term, "memes" denotes the ideological equivalent to biological genes in nature (16). A powerful meme (highly fit in the Darwinian sense) can spread like wildfire from mind to mind, reproducing and combining with other powerful memes. Even within one individual, a mind full of memes, and the context within which the individual is situated, can serve as the ecosystem in which a creative work is born. Thus, I believe that specialized genetic algorithms designed to aid in the creative act ("memetic algorithms") could be designed and would become useful tools as apprentices to some artists and designers.

3. Artificial Life Media
The science of artificial life has begun to trickle down from the ivory towers in Santa Fe and elsewhere, and into popular print media, animation studios, computer game factories, and art galleries. And the notion of using computational evolution and autonomous agents in our technologies is becoming more commonplace. Top-down Design and bottom-up Evolution may merge in our technological future. The Made and the Born, as Kevin Kelley suggests, will merge into hybrid forms.
Computer Games That Play You
The experimental interface agents designed by Pattie Maes and other researchers at the Media Lab can evolve to learn about a user's interests and adapt to his/her style of doing work, like sorting out email messages, for instance. Likewise, the characters and other behavioral objects in a computer game can adapt to a player's skill level and style of playing, as the game is being played, or over the span of many games.

In computer games, behavioral entities - moving sprites, warships, race cars, pong balls, cartoon characters - which were already designed to live in a microworld - can be made to adapt to their worlds, through some automatic optimization schemes, like quick genetic algorithms or fast-adapting neural nets.

They can become successful in their niche, able to deal with gravitation or any other particular attributes of the world within the game, and to respond to the actions of humans and other entities in predictable, interesting, or entertaining ways. Here is an arena in which artificial life techniques can, and already have begun, to be used. For instance, Craig Reynolds (inventor of the flocking and schooling 'boids', and other autonomous entities), is currently applying his techniques towards an entertainment media authoring system, which will aid in the design of computer games (17).
Post-human Ecosystems
Perhaps the newest and most fascinating arena for the propagation of artificial life is the internet. In a virtual chat, I may be engaged in a text-conversation with another person, whose physical coordinates are unknown, and unimportant ("the net negates geometry" - Mitchell (18)), and even whose gender (in real life) is unknown, and whose "posed" identity may be artificial. "Robots", programmed to pose as humans, populate some text-based virtual realities, too.

Primordial soups, like Tom Ray's "Tierra" (19), may begin to penetrate fertile pockets in the net, and work overnight in computers on the sleeping side of Earth, while the other side works in the daylight.

These soups will be designed to evolve software functions which could never be designed by humans, to do complex tasks. Perhaps anti-viral systems will have to be bred in this way. Perhaps the neuromuscular systems for the dinosaurs in Jurassic Park sequels will be evolved in this way.

Imagine breeding images or animations or populations of virtual characters in the dark corners of computer memory at night, letting them interact and grow more complex as you sleep. As far as the future of evolutionary art and artificial life over the net is concerned, the possibilities seem quite large, and the "medium" is very new and formless.

4. Conclusions
These are my thoughts and musings on artificial life, and what it signifies in terms of a new visual language and cultural phenomenon. The art of artificial life, it seems to me, is inherently computational, dynamic, and emergent. And it is also resonant with such trends as: the institutionalized obsession with genomes; the mutability of life; cyborgs; Michael Jacksonian morphing; and the view of humanity as intimately entwined in the web of Earth's ecosystem. The tools of artificial life are not so good for rendering the domain of "Man", nor the projected spaces of a human view on a human world, as Renaissance Perspective was. The art of artificial life renders the dynamics and emergence of earth-life, from which humanity emerged - as well as many other possible (and impossible) forms of life. And in addition to this emergent method of rendering reality, artificial life is also becoming a reality in itself, as the post-human cortex envelops the earth.

1. Lynn Margulus' idea is mentioned in Dawkins' The Blind Watchmaker, Norton 1987, p 176

2. Hans Moravec's paper - Human Culture: A Genetic Takeover Underway, in the first Artificial Life proceedings - editor Chris Langton. Addison-Wesley 1991 p. 167

3. Chris Langton suggested this in a speech at the Fourth Workshop on Artificial Life at MIT.

4. from Bill Mitchell's book, City of Bits.

5. from the beginning of Kevin Kelley's book, Out of Control. Addison-Wesley 1994

6. The "Game of Life" was invented by John Conway, and is explained nicely in William Poundstone's book, The Recursive Universe. 1985, William Poundstone

7. Animated behavior of physically-based figures in virtual worlds were demonstrated at Artificial Life IV by Sims, Terzopoulos, and Ventrella.

8. Richard Dawkins' excitement upon creating his biomorphs is described in his book, The Blind Watchmaker, and also mentioned by Kevin Kelley in Out of Control, andWired, July 1995, p. 122.

9 Ed Zajec, professor of Computer Graphics at Syracuse University's Art Media Studies Dept. (personal communication)

10. Genotype and phenotype are terms derived from biology, used in the computer science optimizing technique of genetic algorithms.

11. Karl Sims' paper, Interactive Evolution for Computer Graphics, in Computer Graphics, vol 25, number 4, July, 1991

12. Ventrella - Explorations in the Emergence of Morphology and Locomotion Behavior in Animated Figures, Artificial Life IV proceedings, MIT Press 1994.

13. Sims' technique is explained in his Artificial Life IV proceedings paper, Evolving 3D Morphology and Behavior by Competition., MIT Press, 1994

14. Ventrella - Disney Meets Darwin, MIT Media Lab thesis document, 1994.

15. Cohen's work is described in McCorduck's book, Aaron's Code, W. H. Freeman and Co. 1990

16. This idea of memes appears in Dawkins' book, The Selfish Gene. Oxford Univ. Press 1976

17. A project underway at Silicon Studio, SGI, Mountain View, CA, to develop an authoring system for entertainment media and game design.

18. From the book, City of Bits, by William Mitchell.

19. Tom Ray's Tierra program is explained in Artificial Life II, edited by Chris Langton, Addison-Wesley, 1992, page 371.