From Yijing to Hypermedia: Some Notes on Computer-mediated Literary Theory and Criticism

Let me start by first going into a little more detail with regard to the multimedial and non-linear character of hypermedia. The multimedial character lies in the fact that they combine several (at least two) conventional media. In the case of hypermedial literature this means that besides words it is composed of images and sounds or combinations of these. In theory, the other senses, such as smell and touch, could also play their part. However, the digitalization of these senses faces serious obstacles and therefor is still in its infancy, so that – apart from devices such as joysticks with force feedback – in practice the emphasis still is on audiovisuals.

The non-linear aspect means, briefly, that the order in which the various components constituting the hypermedial work of art are presented is not a fixed one but depends on the individual choices made by the reader, viewer or listener. In the case of literature therefore this means that readers decide, either partly or completely, in what order the constituent elements (sentences, or larger textual parts, that is: textones) can be read. In other words, readers of hypermedia relate to the text in an interactive way. Readers attain (a certain measure of) control over the text’s composition and the text in its turn responds to the readers” choices. Hypermedial readers share with the readers of conventional linear texts the freedom of textual interpretation, but differ from the latter in being able to exert influence on the creation of the actual text during the reading process. A limit case is the fully automatic text where the sequence of the elements is determined by for example a reader-operated random generator. Such a text is non-linear but interactive in a formal sense only. The opposite limit case is the empty page, where it is up to the reader not just to settle the sequence of the elements but to select the elements themselves. Here the reader has become the writer–and as such hyperactive.

Works of art may be non-linear without being multimedial (for example a hypertext constructed from words only) or they may be multimedial without being non-linear (for example the classic film). If the literary hypertext already constitutes an important transformation of classic, linear literature, when it becomes multimedial the outcome is a cultural form that is vastly different from what ever since the days of Homer has been considered “literature”.

My intention in this paper is to discuss some basic characteristics of hypermedia and then examine their implications for the criticism of art. First, however, I will briefly explain three assumptions that guide my thinking on this topic. They need to be stated fully here at the outset, since otherwise my line of reasoning could be easily misunderstood.

Firstly, I do not presume hypermedial literature to be an entirely new phenomenon. Many examples of non-linear and multimedial literature  are known from the past. One of the oldest Chinese works of literature, the Yijing or The Book of Changes, is an unambiguous example of a non-linear text (see Figure 1).[2]

After all, the book’s fundamental constituents are 64 hexagrams, each consisting of six parallel horizontal lines that can be either continuous or broken, thus allowing 64 different combinations to be formed which each have a specific meaning (the creative, conflict, nourishment, progress, limitation etc.). Those wishing to consult the Yijing for practical guidance in their lives must use a certain random procedure (for example, flipping three coins simultaneously a number of times) to arrive at a certain hexagram. Depending on the coin combinations lines can also be unstable, which means they are about to turn into their opposites. In that case the hexagram is transformed into one or more other hexagrams that also become applicable (in principle) to the life problem that is being posed. A more recent example of a paper non-linear text is is Raymond Queneau’s 1961 volume of poetry Cent mille milliards de poèmes [Hundred Thousand Billion Poems]. This contains ten sonnets, but since each page is horizontally cut into fourteen strips each containing one of the sonnet’s lines these strips can be combined according to readers’ choices to form no less than 10¹⁴ different possible sonnets (see Figure 2).[3]

Even though both the I Ching and Queneau’s volume seem to be traditional books – and as such they can be read, from the first line on the first page to the last line on the last page – their reading instructions in effect make them into a textual space that is traversed by a huge (though finite) number of different paths that readers can travel. We could also look on each of these texts as a database with a finite number of elements and a finite set of rules that state how these elements may be combined. Therefore it is not surprising that these “analog” examples of non-linear texts are excellently suited to being digitized. An internet search engine will come up with several digitized versions of both the Yijing and Queneau’s volume of poetry (see Figures 3 and 4). One could even contend that these texts were designed to assume a form that did not yet exist at the time and that they have only now found their “natural habitat” in a digitized state. But many classic, linear texts too are being re-mediated through the digital medium. On the World Wide Web many linear texts can be found that have been cut into pieces but nevertheless have become part of the Web’s gigantic database through internal and external hyperlinks.

The multimedial nature of the hypermedia is not something new, either. On the contrary: in early forms of writing, such as the Egyptian hieroglyphs and the Chinese ideogram, visual representation and linguistic meaning still form an intimate relationship and can’t even be properly distinguished. But also after their differentiation and separation as symbolic forms, the different media often still remained closely connected. Thinking of the early origins of the theatre, we may even presume that the link of texts with sounds, music and images precedes monomedial writing. We encounter this multimediality in many combinations, on the stage, in opera and in film. In the field of literature multimediality – from the illuminated medieval manuscript up to the present-day comic books – has a long history. What distinguishes the hypermedia from the old multimedia is their non-linear nature. It is the combination of non-linearity and multimediality that turns the hypermedia into “new media”. They are not new in the sense that no single element has a precedent, but rather that “old” elements are recombined so as to form a new medium. In this re-mediation those elements continue to exist, but they often get a new function and a new meaning.

A second assumption that I would like to go into briefly is that I do not expect that the hypermedia will be the end of classic literature, nor that they will render classic literature superfluous because of some kind of superiority. The history of the media shows that new media rarely really replace the old ones. The development of writing did not end oral communication, and in the same way the computer will not put an end to writing or the printed book. Neither did movies end theater performances, or television replace the radio. However, this does not mean that the rise of new media does not affect the old ones. Sometimes an old medium continues to exist as a cultural expression, but the material carrier changes because of the arrival of the new medium. The cultural form of the “novel” does not change, whether it is printed on paper or offered as an e-book that can be read on a  smartphone or an iPad. Or, in any case, it does not necessarily have to change. The rise of a new medium does, it is true, often affect (the use of) the cultural form. Anyone who reads the complete works by Nietzsche on a CD-ROM will more easily be tempted to use the search engine to start following certain “paths” through the oeuvre. For example, when I’m interested in Nietzsches ideas about the role of contingency in human life, using the search function I can easily get a list of all passages in the oeuvre of Nietzsche where the world Zufall (contingency) and Leben (life) appear together. In that case my traditional “horizontal” (syntagmatic) reading of a text is replaced by a more “vertical” (paradigmatic) approach (cf. Barthes 1967).[4] New media afford authors and readers new ways of reading and writing.[5] At the same time new media may obscure or discourage certain uses of the older media. The rise of film, for example, has contributed to the disappearance of the lengthy descriptions of nature in literature, as we found them in nineteenth-century naturalistic novels, and motivated the shift to the characters’ inner world (which obviously cannot be directly captured in images).

Conversely, new media are often judged in accordance with the standards of the old media, for example when a movie or a computer game is measured with the standards applying to literature. I assume that each of the various media possess their own, medium-specific affordances and qualities (cf. Hayles 2002). As we often “drive into the future using our rear view mirror” (McLuhan and Fiore 1967), it often takes some time before it fully develops its potentialities. For example, it took some time before film had developed its own medium-specific “grammar” (part of which is for example the principle of montage) and could develop from a fairground attraction into a fully-fledged art form (De Mul 2010a, 67ff., De Mul 2007, 64ff.). The use of hypermedia within the realm of literature, too, still seem to be in search for a new grammar and it may be for this reason that it is still often the case that they are judged in a somewhat derogatory way by those representing the “old literature”.

A third assumption finally that I would like to discuss briefly is linked to the above-mentioned remark about the digital media’s “own grammar”. In fact, this is a misleading term, because, maybe more than in the case of any other medium, the hypermedia “are not one”. Partly thanks to the possibility of computer programming the hypermedia do not so much have a uniform cultural manifestation; the term is rather a catch-all name for a whole family of forms of which the various characteristics do not (to the same degree) belong to each individual member. It is a cultural form which for good reasons has a name in the plural. This is not the place to elaborate a complete taxonomy, but I would like to give an idea of the scope of this family by mentioning some polar characteristics.

One possible principle of organization depends on the antithesis of static and dynamic hypermedia. Whereas a CD-ROM with the complete works of an author is static in the sense that no elements will ever be added to it or deleted from it (apart from possibly adding “notes” or “bookmarks”), the World Wide Web is an example of a medium that is continuously subject to change.

Another principle of arrangement could be the degree to which the “user” is enabled to contribute to the dynamics. At one end is the situation in which users can only explore the hypermedial space, at the other extreme is the situation in which users themselves construct this space to a lesser or greater degree. In most computer games all possibilities are determined in advance and the player has to follow a course given prior to the game in order to reach the finish. In Second Life and World of Warcraft, to the contrary, the “inhabitants” of these online worlds can construct much of their own living surroundings, houses, tools etc.

Yet another arrangement is based on the degree or nature of the manoeuvrability among the different elements. In some cases each element can be linked to any other element (as we saw in the case of the Yijing), but the maneuverability is usually channeled through compulsory links, which may or may not be conditional. For example, in a computer game you can only leave the room by means of a door, but not before you have found the key. Literary hypertexts such as Victory Garden, too, often make use of such conditional links.[6] They enable the author to control the number and the nature of the tracks that the reader has to follow through the hypertext. The function of the “user” – which may vary from distant beholder to player of a part to (fellow) author – may also be a suitable organizing principle.

When we combine the above-mentioned standards and other standards, we can draft a taxonomy which offers room to numerous variations of the hypermedia, varying from computer games to animated poems and from online communities to multimedial encyclopedias. This sometimes confusing multitude and heterogeneity make it difficult to speak about hypermedia and hypermedial art criticism in general terms. In the following I will, as said above, argue from the literary perspective in particular. The three characteristics of hypermedial literature and their implications for the criticism that I want to discuss now, will however also apply to several other subcategories of the hypermedia.

The first characteristic of the hypermedia is that they have a synesthetic effect on traditional cultural forms. From a material point of view this has to do with the fact that the digital field is characterized by a common language. In the hypermedia all classic media are re-mediated by the same binary code. Thanks to this universal coding we can not only easily link words to sounds and colors, but they may in the literal sense of the word be mutually translated. The letters of the alphabet are literally graphic pictures which may be turned into sounds. Although this “digital synesthesia” initially only concerns the material “basis” of the cultural forms, it also affects the cultural forms themselves. The individual art forms are not only incorporated in the hypermedia, they also get involved in all sorts of intermedial relations. For example, we see that computer games show the style features of film, and at the same time film assumes elements derived from the computer game. In this way the camera work in the fight scenes in a movie like The Matrix was inspired by the visualization of fights in computer games.[7] The universal coding also enables people to reuse in a game settings and characters which were in the first place designed for a movie (or the other way around). More and more often settings and characters are already in the design stage equipped for use in several media.

Not only within the arts do the various art forms engage in a diversity of mutual intermedial relations; art itself also to an increasing degree engages in relations with other fields, such as education, economy and warfare – think of such phenomena as “serious gaming” and “edutainment”. A game such as America’s Army (americasarmy.com) is not only a computer game which is made available for free by the American army, but as soon as the gamer has been seduced by the game into being recruited, the same computer program also turns into a training simulator and a laboratory to test new weapon systems.

Whereas modernity may be regarded as an era that was characterized by a differentiation among the various cultural fields (economy, politics, art, religion etc.) as well as within those fields (the art forms and genres within the field of art) the hypermedia are first and foremost postmodern media thanks to their integrative and intermedial dimension. The “pathos of purity” that characterized modernity gives way to a postmodern “pathos of inpurity.”[8] This gives rise to special problems for literary criticism. What standards should be used when hypermedial works of literature are to be judged? Literary critics not seldom have a monomedial background and therefore tend to judge such works from a sheer literary perspective. But even with a multimedial background, it remains hard to decide on what basis hypermedial works of art should be assessed. Should Mark Strand’s poem In Celebration still be judged as a literary work when it gets animated and vocalized via a synthesizer? Or does it rather becomes an animated cartoon? Or should one rather look for intermedial quality standards, for example the degree and nature of synesthesia?

The second constituting characteristic of the hypermedia that I want to mention here relates to fundamental instability. Two aspects of this may be distinguished. In the first place the hypermedia are unstable because they are rather generators for “works” in the classic sense of the word than such works themselves. For exampe, Queneau’s Cent mille milliards de poèmes, mentioned above, is more of a generator of poems (by means of which 100,000 billion different poems may be drafted) than a compilation of poems. Moreover, many hypermedia are always “under construction”, not so much because they are not finished yet as because being in progress is one of their characteristics.

This, too, causes art criticism to be faced with major problems. What exactly does the critic of hypermedia judge? Should the literary critic first read all of the 100,000 billion different poems before she can assess Queneau’s “work”, or should she rather assess the engine itself? To the extent that the reader and the critic in the hypermedial sphere become themselves the creators of a work of art (they will probably discover poems in Queneau’s “compilation” that the poet himself was never aware of) they also become their own judges. But just as the hypermedial author becomes the creator of a narrative (or poetic) space rather than a creator of linear narratives or poems, so the critic will increasingly become a judge of narrative and poetic machines. He or she will rather be a meta-critic than a critic.

In this case, too, we may ask the question what standards must be adopted. Considering Walter Benjamin’s famous distinction between the cult value of the unique piece of art (the Mona Lisa) and the exhibition value of the reproductive arts (such as photography and film), such a standard in the case of the hypermedia might well be the manipulation value of the “work”, that is, how far and in what ways the elements of the work may be recombined to produce esthetically valuable constellations (De Mul 2009, 2008b). [9]

The third and last characteristic of hypermedia is closely linked to the previous ones. When the work is an experience machine rather than a linear reading experience, the question arises whether this must have implications for the criticism of hypermedial literary works. In other words, the question is whether the criticism of hypermedial literature should not itself be transformed into hypermedial criticism. If the hypermedia liberate “users” in the sense that they become fellow creators of the hypermedial work, one could wonder whether the readers of literary criticism should not also become its fellow creators. This would imply for the critic that she, too, no longer writes linear essays but applies herself to the creation of hypermedial criticisms. These are not only characterized by their multimedial nature (they may, for example, completely or partly be shaped as a picture or sound essay), but also by having the nature of argument spaces within which readers may conceive their own judgments – not in the sense that as readers they subject the critic’s criticism to their own judgements but in the sense that they become the fellow authors of the criticism.

Against this background we could now image what a literary theory or criticism journal in the age of “Big Data” might look like. Its basis will be a website, powered by a cloud of relational databases containing all works, letters, interviews etc. by a particular author, as well of all other authors whose work is datafied. Of course it will still be possible to display different versions of the texts (for example handwritten drafts, computer files with the subsequent versions, later revisions etc.), the legacy, translations etc

However, thanks to user participation it will also contain a lot of additional useful information, such as annotations, articles and commentaries. In addition it will contain links to relevant audiovisual material, such as recordings of lectures and debates. In may contain graphics and simulations in order to visualize complex relationships between works, authors, and ideas. And of course human users will also be able to communicate, via connected blogs, live chat, tweets etc.

Moreover, due to ranking of other users and their behaviors, thanks to smart datamining and profiling algorithms it directs the attention of the users to specific meaningful re-arrangements of the material and related clusters.[10] “Researchers that referred to this commentary were also interested in the following comments.” Or: “If you liked this conclusion, don’t forget to read the following counter-arguments”. Even more interesting is the possibility to add our own experiences, both as reader and author, to the corpus. In a way, the distinction between reader and writer will get blurred, just like the distinction between secondary and primary literature, and finally between the human user and the global web of information. Welcome in the world of “Big Literature.”

In this case, too, it is clear that the hypermedial criticism that is envisaged here does not make classic criticism fully obsolete. Obsolete become only they who see classic criticism as the only one possible.

Figures

Figure 1: Yijing or The Book of Changes, ca.1,000 BCE

Figure 2: Raymond Queneau,Cent mille milliards de poems, 1961

Figure 3: Click on the link to see one of the many website versions of Cent mille milliards de poems

Figure 4. One of the many Yijing websites

(http://www.psychicscience.org/ching3.aspx)