If we want to describe what new media does to old media with a single term, ‘mapping’ is a good candidate. Software allows us to re-map old media objects into new structures – thus turning media into what I call ‘meta-media.’ 1
In contrast to media, meta-media acquires three new properties. With software, the data can be translated into another domain – time into 2D space, 2D image into 3D space, sound into 2D image, and so on. (More complex and unusual mappings are also possible – and the search for such new mappings that allow us to access old media objects in new ways congruent with information interfaces we use in our everyday life represents one of the most fruitful research directions in new media art.) In addition, the media object can be manipulated using all GUI techniques already developed: move, transform, zoom, multiple views, filter, summarize.2 And finally, the media objects can now be ‘processed’ using the standard techniques of computerized data processing: search, sort, replace, etc.
Let me provide a few examples of meta-media which all involve new interfaces to cinema. For instance, software developed by Steve Mamber (UCLA) allows the user to “map” the feature film into a matrix of still images, each image representing a shot from a film. Here time is mapped into space. Another software tool written by Mamber takes shots from the film and reconstructs their architecture as 3D navigable spaces (thus reversing the normal procedure of computer animation). Here is the mapping goes from 2D to 3D - from a flat surface of a movie screen into a virtual computer space.
The project Invisible Shape of Things Past by Art+Com (Berlin) maps historical films of Berlin into new spatial structures that are integrated into a 3D navigable reconstruction of the city. See www.artcom.de/. Another groundbreaking mapping project by Art+Com is an Interactive Generative Stage set whose parameters are interactively controlled by actors during the opera.3 During the opera performance, computer reads in the body movements and gestures of an actor and uses this information to control the generation of a virtual set projected on a screen behind the stage. In this case positions of a human body are mapped into various parameters of a virtual architecture such as the layout, texture, color, and light.
For the designer of this project Joachim Sauter it was important to preserve the constraints of the traditional opera format – actors fore grounded by lighting with the set behind them – while carefully adding new dimensions to it. 4 Therefore following the conventions of traditional opera the virtual set appears as a backdrop behind the actors – except now it not a static picture but a dynamic construction that changes throughout the opera.
Note that the mappings in these examples preserve the granularity and the syntactical structure of the old media object, while giving us new ways to navigate it, to experience its structure, to compress and expand our views of the object, and to interactively control it. In the case of Mamber’s project, the film still consists from shots that can be played from beginning to the end – or we can use the new representation of all the shots in a film as a single interactive 2D image matrix. In the case of Invisible Shape we can similarly play the historical film segment from the beginning to end – or we can navigate the 3D model of Berlin to see where these films were taken.
This is why I refer to this type of new media as “meta-media.” A meta-media object contains both language and meta-language – both the original media structure (a film, an architectural space, a sound track) and the software tools that allow the user to generate descriptions of this structure and to change this structure.
If you think that meta-media is a conservative phenomenon which ‘betrays’ the movement of computer culture to develop its own unique cultural techniques – Artificial Intelligence, Artificial Life, simulation, database navigation, virtual worlds, etc. – you are wrong. Since the 1970s modern computing has been grounded in Alan Kay’s concept (influenced by previous groundbreaking work in human computer interface, most importantly Sutherland’s 1962 Sketchpad software) of a computer as a “personal expressive media.” After he arrived to Xerox PARC, Kay directed the development of a word processor program, a music composition program, a paint program, and other tools that redefined the computer as a simulation machine for old media. So while the routine use of computers as media simulators did not become possible until the 1980s, the paradigm itself was already set around 1970. Gradually, other roles of a modern computer - a machine for computation, real-time control, and network communication – became less visible than its role as “simulation engine.” (Of course, the development of the World Wide Web in the 1990s made network communication role also quite visible to the public.). In summary, the computer’s ability to simulate other media (which means simulating their interfaces and “data formats” such as written text, image, and sound) is not an after-thought – it is the essence of a modern post-1970 computer.
(It is possible to state this idea even more radically - by moving the date even earlier, to 1930s. We can say that when Alan Turing defined computer as a general-purpose simulation machine that can simulate most other machines have already invented, the idea of media simulation was implicitly introduced. But it was only in the 1950s-1970s that the work of Sutherland, Engelhard, Kay, and others made this idea into reality by allowing the computer to systematically simulate the operations of drawing, drafting, painting, photo manipulation, sound generation and editing, and so on.)
What is crucial to realize is that computer’s simulation role is as revolutionary as its other roles. Most software tools for media creation and manipulation do not simply simulate old media interfaces – a book page and a table of contents in Acrobat, a pan and a zoom of a virtual camera in Maya, time code count and a razor blade in Final Cut Pro – but also allow for new type of operations on the media content.5 In other words, these tools carry the potential to transform media into meta-media.
Today it is meta-media paradigm – rather than other seemingly ‘truly’ original computer techniques or computer programming itself – that is in the center of computer culture. Outside of the professional and scientific domains, consumers think of computers as machines for downloading, storing, transmitting and editing media. Software such as media players and editors, CD and DVD burners, ports to connect digital still and video camera or MP3 players is what consumer computing is about today.
This is not accidental. The logic of meta-media fits well with other key aesthetic paradigms of today – the remixing of previous cultural contents and forms of a given media (most visible in music, architecture, design, and fashion), and the second type of remixing – that of national cultural traditions now submerged into the medium of globalization. (In the first approximation, the terms “postmodernism” and “globalization” can be used as aliases for these two remix paradigms.) Meta-media then can be thought alongside these two types of remixing as the third type: the remixing between the interfaces of various cultural forms and the new software techniques – in short, the remix between culture and computers.
Not every re-mix by itself is great – it all depends on who is doing the mix. If we look at interfaces of popular commercial software for media access and manipulation from this perspective, they begin to look like the work of a strange DJ who mixes operations from the old interfaces of various media with new operations of GUI in somewhat erratic and unpredictable ways. My favorite example of such remix – and least favorite of all popular commercial software - is the interface of Adobe Acrobat Reader. It combines (1) the interface from time media software (VCR style arrow buttons); (2) the interface from image manipulation software (a zoom tool); (3) the interface elements which have strong association with print tradition - although they never existed in print (page icons also controlling the zoom factor); (4) the interfaces which have existed in books (the bookmarks window); (5) the standard elements of GUI such as search, filter, multiple windows).
But in other cases media/software remix can provide us with fundamentally new and stylistically /metaphorically coherent paradigms for representing the visible world, our knowledge, human history, and fellow human beings. The already mentioned projects The Invisible Shape of Things Past and Interactive Generative Stage are among my favorite examples of successful meta-media. However, the artists and designers do not have the exclusive license to create good meta-media. For instance, one of the most culturally important examples of meta-media with which everybody is familiar today is hypertext. Developed by Ted Nelson in the middle of the 1960s, hypertext gives us a new paradigm for organizing any information and knowledge. Since Tim Berners-Lee’s World Wide Web popularized hypertext in the 1990s, hypertext became as central to contemporary culture as other media techniques on which we have been dependent for many centuries – for instance, one-point linear perspective used for image organization or codex system used for text organization.
Inventing meta-media is not easy. It requires in-depth understanding of not only computer science but also the history and conventions of various media and cultural forms. The pay-off, however, is enormous. Until digital computers, each introduction of new media, from Guttenberg to SONY, required manufacturing and adaptation of new hardware: printing presses, film cameras and projectors, video recorders, lenses, and so on. But meta-media requires only new software, and therefore it can be developed by a single person or a small group and easily disseminated. Never before inventing new media was so easy – at least from a technical point of view.
1 - I introduced the term ‘meta-media’ in my Avant-garde as Software (1999) available from www.manovich.net/texts_04.htm.
2 - In my book The Language of New Media (The MIT Press, 2001) I discuss in more detail how the operations specific to particular media are transformed to become general-purpose media interfaces.
3 - The full name of the project is Interactive generative stage and dynamic costume for André Werners `Marlowe, the Jew of Malta.’ See for more information and project visuals.
4 - Joachim Sauter, personal communication, Berlin,
July 2002.
5 - Re-mapping media data into a new domain is one of the most important among these operations.
Lev Manovich
He is an Associate Professor in the Visual Arts Department, University of California, San Diego where he teaches courses in new media art and theory. He is the author of The Language of New Media, Tekstura: Russian Essays on Visual Culture, as well as many articles. Manovich has been working with computer media as an artist, computer animator, designer, and programmer since 1984. His art projects include little movies, the first digital film project designed for the Web (1994), Freud-Lissitzky Navigator, a conceptual software for navigating twentieth century history, and Anna and Andy, a streaming novel.
