Manovich, Lev. The Language of New Media. 1st MIT Press pbk. ed. Cambridge, Mass.: MIT Press, 2002. Print.
- M. claims that his goal in this work is to record the “research paradigm” or the history of computer media in its first decade of existence, before it “slips into invisibility” (8). To do so he places the language of new media in a history of “modern and visual media cultures.” In so doing he asks a similar question to Bolter & Grusin: namely, how does new media rely on older cultural media forms and how does new media break with those forms? He also attempts to answer the question: how does new media effect the visual culture at large – outside the conscripts of computers, games, cinema, and animation? Specifically, M. claims to consider the following in this work:
- the parallels between cinema history and the history of new media;
- the identity of digital cinema;
- the relations between the language of multimedia and 19th century pro-cinematic cultural forms;
- the functions of screen, mobile camera, and montage in new media as compared to cinema;
- the historical ties between new media and avant-garde film (9).
- M. calls his method “digital materialism” or a new media theory that is built from the ground up by considering the principles of computer hardware and software and the operations involved in creating “cultural objects on a computer to uncover a new cultural logic at work” (10).
- Here’s the organization of the book built on a bottom up approach (or a movement from binary code to executable code to a high-level computer language):
- M. wants to align the work of this book within other areas of culture including: other arts and media traditions, computer technology, contemporary visual culture, and contemporary information culture (M. claims ownership of this term – the ways in which information is presented in cultural sites and objects as well as the historical methods for organizing and retrieving information) (13-4).
- M. uses the term “object” to describe new media artifacts consciously; however, he does so to keep consistent with other paradigms of computer science when offering a theory of computerized culture (not because he’s pointing toward some sort of OOO/speculative realist ontology). M. uses the term “representation” to refer to how new media artifacts in some way (as cultural objects) construct an outside referent (the real). He realizes that all representations are incomplete and the organization of any data provides particular representations that portray/represent the human subject in a privileged manner. Here are some nuances of his use of the term:
- representation-simulation: this is representation through simulation of the real that occurs across the screen.
- representation-control: this is representation that plays itself out through user-control. . . as in a GUI interface. The interface (or image-interface) is an illusion that controls the opposition between the depth of the immersive digital environment and the flat surface of the screen as glass rectangle.
- representation-action: This understanding of representation considers the ways that immersive image-interfaces and GUIs provide the opportunity for new media interfacers to “enable action” inside of the screen (to manipulate reality).
- representation-communication: This is the opposition between representational technologies of film, sound, video and real-time telecommunication technologies like the telephone, television, and telegraph.
- Visual illusionism – simulation: Illusionism refers to traditional techniques of art like perspectival painting, cinema, panorama, etc. that attempt to create a visual semblance of reality while simulation refers to the computer methods used to model those realities interactively (through manipulation of objects, shapes, etc. in new media objects).
- representation – information: This opposition refers to what M. considers opposing goals of new media design: the desire for immersion into an imaginary, fictional universe and the provision of information that makes a real-life difference in folks lives. (15-7).
Chapter One: What is New Media?
- M. claims that we shouldn’t be surprised that new media and the computer begin to develop around the same time (19th century) as both were necessary for the dissemination of ideology and objects to a large number of people – so large a number that new media plays an important role in assuring ideological consensus and beuracratic function (22).
- Principle #1: Numerical Representation – All new media objects are composed of digital code. This means a new media object can be described formally (mathematically). It also means that any new media object is subject to algorithmic manipulation . . . or, said differently, media becomes programmable (20).
- Principle #2: Modularity: All new media objects are composed of a “modular structure” or “fractal structure.” What this means is that collections of discrete units (pixels, polygons, characters, etc.) compose any new media object. . . and those units are composed of numbers (principle 1).
- Principle #3: Automation: Because of the modularity of numerical representations that compose units of new media objects the process of creation, manipulation, and access can be made automated by constructing commands and templates or actions that automate. This is particuarly true in .xml tagged language units used by computers like Watson.
- Principle #4: Variability: Because of the numerical basis of module units, new media objects aren’t fixed but can forever be reconfigured by renegotiating the modular structure (and sometime renegotiating it automatically!). As such, new media can respond to an “on demand” function of completely customizable, configurable user desire. What are some repurcussions of this?
- Media elements are stored in databases because databases construct patterns out of randomness through the ordering of variable modular units.
- It is possible to separate content (data) from interface. As such, a single database can yield multiple interfaces and a new media object is merely one interface into a multimedia database.
- User demographic information can be used by computers to automatically create new media objects to fit user experience/desire (think the way web hardware reads browser information to create a particular user expereince). This also means that user interactivity directs manipulation of a new media object (branching or “menu-based” interactivity).
- Hypermedia are new media structures that incoporate hyperlinks to unite different media. The web is a great example of this.
- Periodic updates to new media objects is made possible through .xml data characterization. Think about the constantly updating nature of new media objects in mobile devices.
- New media objects are scalable (think browser window sizes, etc.) in order to respond to user demands/limitations because data is represented mathematically and can be reordered algorithmically automatically.
- Variability and its subfunctions result in a logic of individuality for experiences with new media objects. Old media catered to the masses by providing the same goods; in new media individual desire is met by the ability of media to transform to meet user identity/desire (41).
- Principle #5: Transcoding: M. claims this is the most important and substantial aspect of new media in computerized environments. New media can be considered in two “layers”: the cultural layer that we interact with on a day-to-day basis (pictures, stories, points of view, movies, etc.) and the computer layer or the layer of process and packet, data sorting and matching, functions, variables, computer languages, and data structures. M. thinks the principle of transcoding is really important because, he argues, the computer layer’s ontology, epistemology, and pragmatics come to influence the cultural layer of the computer output. Likewise, (we hope!) the cultural layer comes to influence the computer layer. Eventually this results in a “composite” or amalgam of computer-culture that reconceptualizes both realms. (30-50). This could even move “media studies” toward what M. calls “software studies.”
- What are some of the principles and ideas that Manovich pushes against in this work (because he finds elements of them in old media as well)?
- M. notes that he doesn’t use digital in this book because “digital” doesn’t necessarily correlate with “perfect replication” due to changes in compression when converting analog to digital (and digital to digital). Likewise, M. doesn’t use “interactivity” because, by definition, HCI is interactive . . . it is an interaction (55). Further, interactivity isn’t really as liberatory as you might think. . . in fact, it is the instantiation of a system of predetermined possibilities. As M. notes, when you click through images, links, videos, etc., you are actually “asked to follow pre-programmed, objectively existing associations” in such a way that you are, following Althusser, interpellated or “asked to mistake the structure of somebody else’s mind for our own” (61).
Chapter Two: The Interface
- M. identifies the GUI as a code that carries the cultural messages of a variety of media to the end user. Because the GUI is a code it provides its own “model of the world, its own logical system, or ideology” and as such “subsequent cultural messages or whole languages created with this code will be limited by its accompanying model, system, or ideology” (64). So, in other words, the interface becomes the site of transformation wherein new media objects are subjected to the logic of the interface in ways that further alters their capabilities and functions.
- M. argues that the content-interface binary that has long framed conventional media doesn’t necessarily work in new media objects: they are merged into one entity because the content is motivated, changed, and manipulated by the interface (67).
- Cultural interfaces “describe a human-computer-culture interface” that we use more and more often to interact with our cultural data (news, pictures, sound, movies, texts, etc.). M. coniders three cultural interfaces in new media objects during the 1990s (interfaces with histories that extend back into other forms of media that have in some way “organized information, correlated space and time, and strcutured the human experience of accessing infromation:
- Printed word: this cultural tradition manifests in numerous ways on the screen – pages (rectangles with objects embedded inside them)[1. Wow, what of Manovich’s claim on 77 that “the sheer existence and popularity of hyperlinking expemplifes the continuing decline of the field of rhetoric in the modern era”? M. makes this claim on the basis that the link operates as a metonymy for its whole; however, surely that doesn’t diminish the rhetorical effect of new media objects on the web?] are the predominant form; however, they are being reconfigured through the computer logic of hyperlinking.
- Cinema: cinema isn’t simply film but the entire ensemble of cultural forms, mechanisms, languages, and media that constitute what we think of when we think of cinema. M. claims that this cultural tradition is becoming progressively more important in new media environments. M. traces the explosion of cinematic cultural interfaces to the explosion of personal, mobile cameras in the 1980s and 1990s but traces cinemas interface and representation all the way back to different forms of (rectangular) Western painting. M. notes that the most intense and aggressive transformation of the cinematic interface into new media objects occurs in the computer game (83).
- HCI: Representation versus control: Relying on Bolter and Grusin, M. notes that the HCI is a representative example of remediation or the way that the medium remediates or reworks the media on the level of content as well as on the level of form (89) and the HCI represents the status of cinema 100 years ago. . . it is still in its infant stage.
- The screen is a historic technology made even more important with the computer. As M. claims, it is through the screen that “the user experiences the illusion of navigating through virtual spaces, of being physically present somewhere else or of being hailed by the computer itself” (94). M. sees the screen as the hegemon in the visual field because almost all experiences with media in the contemporary era take place through the interface and screen.
- M. notes that the body is rendered immobile and non-existent with most cinema, film, and painting technologies that are mediated in screens; however, with the advent of VR (and I would add games like Xbox’s Kinect and the Wii) the screen establishes a new relationship with the body – a relationship that brings the representational apparatus of the screen more securely to the body (111). In other words, as M. goes on to speculate, the representational apparatus of the screen is becoming more integrated into our own biocircuitry (laptops, mobile phones) until – possibly – we simply have a chip implanted into our retina that allows us to “see” through the screen of technobiological vision (114).
Chapter Three: The Operations
- This chapter is about the level of technology that runs on top of the interface – application software (117). The processes of copy, cut, paste, search, composit, transform, and filter are all essential operations or techiniques of working with new media to produce objects in digital environments. Specifically, in this chapter M. will consider selection (used by experts and amateurs), composition (used by experts), and teleaction (used primarily by users). As he notes, these actions are more than just descriptors for how we work with digital media; rather, they are logics that become part of “how we understand ourselves, others, and the world” (118). At their heart, operations are another iteration of the principle of transcoding – they are computer-side manipulations that then manifest in cultural new media objects.
- M notes that while these sorts of operations have been made available to the lay consumer/producer and the gap between the professional and the amateur has shrunk, it is by no means erased. In fact, M. claims that the development of even more complex languages is a strategy employed by professionals to keep themselves differentiated from the amateur user (120).
- The Logic of Selection: New media objects aren’t created from scratch. Because of their modularity, they are assembled from pre-made parts. As such, authentic creation of new media is a fiction . . . the production of new media objects is an excercise in selection or assemblage (124). THis applies not only to the production of new media objects but also to their use (think selections in games. . . but what about delimited games or auto-generated, non-repeating dungeon crawlers?). This results in the production of “unique identities” along pre-formatted lines. As M. notes rather ascerbically, “I would prefer using Microsoft Windows exactly the way it was installed at the factory instead of customizing it in the hope of expressing my ‘unique identity'” (129)[3. Surely this isn’t still the case. . . a key feature of mobile media is just this customization toward unique pre-selected identities.].To summarize: “The practice of putting together a media object from already existing commercially distributed media elements existed with old media, but new media technology further standardized it and made it much easier to perform” (130). In closing, M. highlights the DJ as a modern cultural figure who embodies the logic of selection.
- Compositing: the process of fitting together and adjusting a host of different primary and secondary elements and dissimilar identities into a complete whole that renders the boundaries between the original objects invisible. Compositing in digital environments is a three step process: 1) construction of a seamless 3-D virtual space from different elements; 2) Simulation of a camera move through this space; 3) Simulation of the artifacts of a particular media (137). Compositing is usually the end result of selection. It is rendering and making a seamless, complete new media object from its disparate parts. On 3-D: if we’re moving toward 3-D then digital compositing in an historical intermediary step between 2 & 3. In his discussion of compositing and montage M. draws out an important distinction: “if film technology, film practice, and film theory privilege the temporal development of a moving image, computer technology privileges spatial dimensions. Spatial dimensions can be defined as 1) spatial order of layers in a composite (2.5-D); 2) virtual space constructed through compositing (3-D space); 3) 2-D movement of layers in relation to the image frame (2-D space); and 4) relationship between the moving image and linked information in the adjustment windows (2-D space) (157).
- Teleaction: this operation isn’t applied to create new media but to access it. The prefix “tele” implies a particular distance to new media objects (and anything) that requires us to reconsider the traditional cultural domain of representation. M. notes that real-time telecommunication technologies have long been subordinated to the representational technologies that preceded them (think the performance of a radio broadcast for example); however, the acts of telepresence made possible through new media (webcams in Tahiti) troubles this subjection. In fact, telepresence affords the opportunity to act or manipualte reality remotely through representational images viewed through new media objects (think Drone bombers).
- M. relies on Latour to argue that some images are actually instruments in that they can be used as instruments of control and power.
- What’s the real import of telepresence? It doesn’t allow for the kind of critical reflection necessary to really consider the repercussions of acting in a remote physical location (again, consider the drone strikes in Pakistan/Afghanistan as a terrible instantiation of this idea). The distance is collapsed, and with that collapse goes the temporal period that might have mediated different aspects of the decision making process.
Chapter Five: The Forms
- M. notes early on in this chapter that “all new media design can be reduced to these two approaches: creating works in new media can be undesrtood as either constructing the right interface to a multimedia database (the Web) or as defining navigation methods through spatialized representations (games and VR) (215). The first is interested in getting folks effeciently to theinformation they desire. The second is an infinite delay of desire through participation in immersive virtual worlds. So, in this chapter M. considers the database and navigable space as the perfect vehicles for new media objects (or the forms they take). At this point we’ve moved to the top of the digital materialist paradigm (binary–>assembler language–>complex language–>application).
- M. sees the tension between surface-depth and information-immersion in new media as a more general opposition: action-representation (216).
- M. notes that while database and narrative are natural enemies, Brooke brings the two together by connecting them to Hayles (Database=randonmness, narrative=pattern. From the database a story is told by the user. . . by ordering the modular data is a particular way in Web 2.0 technologies).
- Conventional Semiotics: Syntagmatic dimension: signs are sturng together in a linear sequence to produce meaning. These signs are related physically, presently, and really. Paradigmatic dimension: signs are grouped together in sets that share a commonality. These signs are related imaginatively, conceptually, and immaterially (230).
- According to M., the database works on a paradigmatic logic that assembled possible like-kind components and through the process of selection a new media object is rendered real (narrative is selection?). So, “On the material level, a narrative is just a set of links [coordinating the database]; the elements themselves remain stored in the database. The narrative is virtual while the database exists materially (231). Why is the database arranged in a narrative? According to M., this occurs because the syntagmatic logic is the logic of cinema. . . and cinema is the “dominant semiological order of the 20th century” (232).
- It would seem that navigable space is just another kind of interface to the database; however, M. argues that this isn’t the case because navigation is a cultural form in its own right . . a form that might be unique to new media in that space becomes a media type in these environments (251).