First of all, what is gained through the abstraction of the words is almost entirely lost through the concreteness of the image. The very dense medium of writing stands in sharp contrast to the diluted medium of images. To download text on the network is quite different from displaying images. If this were the only reason, we would be alert to the differences between images and texts. When the complexity of the image reaches high levels, decoding the image becomes as tedious as decoding texts, and the result less precise. All this explains why people try to use a combination of images and words. It also helps in understanding strategies for their combination. As a strategy of relating text and image, redundancy helps in focusing interpretation. The strategy of complementing helps in broadening the interpretation. Other strategies, ranging from contrasting texts and images to paraphrasing texts through images, or substituting texts for images, or images for text, result in forceful ways of influencing interpretation by introducing explanatory contexts. A very large portion of today's culture-from the comic strip to picture novels and advertisements, to soap operas on the Internet-is embodied in works using such and similar strategies.
What interests us here is whether images can replace the experience required to understand a text. If the answer is affirmative, such images would be almost like the partner in conversation. As products of human experience, images, just like language, embody that particular experience. This automatically makes the problem of understanding images more involved than just seeing them. But we knew this from written language. Seeing words or sentences or texts on paper (in script or in print) is only preliminary to understanding. The naturalness of images (especially those resembling the physical universe of our existence) makes access to them sometimes easier than access to written language. But this access is never automatic, and should never be taken for granted. In addition, while the written word does not invite to imitation, images play a more active role, triggering reactions different from those triggered by words. The code of language and visual codes are not reducible to each other; neither is their pragmatic function the same.
Research reports are quasi-unanimous in emphasizing that the usefulness of pictures in increasing text comprehension seems not to depend on the mere presence of the image, but on the specific characteristics of the reader. These make clear the role played by what was defined as background knowledge, without which texts, images, and other forms of expression stabilized as languages make little sense, if any, to their readers, viewers, or listeners. In order to arrive at such conclusions, researchers went through real-time measurements of the so-called processing of texts, in comparison to picture-text processing. The paradigm employed uses eye movement recordings and comprehension measures to study picture-text interactions. Pictures helped what the researchers defined as poor readers. For skilled readers, pictures were neutral when the information was important. The presence of pictures interfered with reading when the information in the text was less important. Researchers also established that the type of text-expository or narrative-is not a factor and that pictures can help in recall of text details. This has been known for at least 300 years, if not longer. Actors in Shakespeare's time were prompted to recall their lines through visual cues embodied in the architecture of the theater. After all was measured and analyzed, the only dependable conclusion was that the effects of images on comprehension of written language are not easy to explain. Again, this should not come as a surprise as long as we use literacy-based quantifiers to understand the limits of literacy. Whether images are accidental or forced upon the reader, whether the text is quasi-linear or very sophisticated (i.e., results from practical experiences of high complexity), the relation does not seem to follow any pattern. Such experiments, along with many others based on a literacy premise, proved unsuitable for discovering the sources and nature of reading difficulties.
Eye movement and comprehension measures used to study picture-text interactions only confirmed that today there are fewer commonalties, even among young students (not to mention among adults already absorbed in life and work) than at the time of the emergence of writing and reading. The diversification of forms of human experience, seen against the background of a relatively stable language adopted as a standard of culture, hints at the need to look at this relation as one of the possible explanations for the data, even for the questions that prompted the experiments in the first place. These questions have bearing on the general issue of literacy. Why reading, comprehension, and recall of written language have become more uncertain in recent years, despite efforts made by schools, parents, employers, and governments to improve instruction, remains unanswered. Regardless of how much we are willing to help the understanding of a text through the use of images, the necessity of the text, as an expression of a literate practical experience, is not enhanced. Conclusions like these are not easy to draw because we are still conditioned by literacy. Experiences outside the frame of literacy come much more naturally together because their necessity is beyond the conditioning of our rational discourse. This is how I can explain why on the Internet, the tenor of social and political dialogue is infinitely more free of prejudice than the information provided through books, newspapers, or TV. These observations should not be misconstrued as yet another form of technological determinism. The emphasis here, as elsewhere in the book, is on new pragmatic circumstances themselves, not on the means involved.
The research reported above, as any research we hear about in our days, was carried out on a sample. A sample, as representative as it can be, is after all a scaled- down model of society. The issue critical to literacy being the scale of human practical existence, scaled-down models are simply not suited for our attempt to understand language changes when the complexity of our pragmatic self-constitution increases. We need to consider language, images, sounds, textures, odors, taste, motion, not to mention sub-verbal levels, where survival strategies are encoded, and beliefs and emotions are internalized, as they pertain to the pragmatic context of our existence. Literacy is not adequate for satisfactorily encoding the complexity and dynamics of practical experiences corresponding to the new scale that humankind has reached. The corresponding expectations of efficiency are also beyond the potential of literacy-based productivity. Ill-suited to address the mediated nature of human experience at this scale, literacy has to be integrated with other literacies. Its privileged status in our civilization can no longer be maintained.
Korzybski was probably right in stating that language is a "map for charting what is happening both inside and outside of our skins." At the new stage that civilization has reached, it turns out that none of the maps previously drawn is accurate. If we really want details essential to the current and future development of our species, we have to recognize the change in metrics, i.e., in the scale of the charted entity, as well as in dynamics. The world is changing because we change, and as a result we introduce new dimensions in this world.
Even when we notice similarities to some past moment-let us take orality as an example-they are only apparent and meaningless if not put in proper context. Technology made talking to each other at long distances (tele-communication) quite easy, because we found ways to overcome the constraints resulting from the limited speed of sound. The most people could do when living on two close hills was to visit, or to yell, or to signal with fire or lights. Now we can talk to somebody flying on an airplane, to people driving or walking, or climbing Mount Everest. Cellular telephony places us on the map of the world as precisely as the global positioning system (GPS) deployed on satellites. The telephone, in its generalized reality as a medium for orality, defies co- presence and can be accessed virtually from anywhere. Telephony as a practical experience in modern communication revived orality under circumstances of highly integrated, parallel, and distributed forms of human activity on a global scale. On the digital networks that increasingly represent the medium of self-constitution, we are goal and destination at the same time. In one click we are wherever we want to be, and to a great extent what we want to be or are able to do. With another click, we are only the instantiation of someone else's interest, acts, knowledge, or questioning. The use of images belongs to the same broad framework. So does television, omnipresent and, at times, seemingly omnipotent. We became connected to the world, but disconnected from ourselves. As bandwidth available for interacting through a variety of backchannels expands from copper wire to new fiberglass data highways, a structure is put in place that effectively resets our coordinates in the world of global activity. Defying the laws of physics, we can be in more than one place at the same time. And we can be more than one person at the same time. Understanding language under such circumstances becomes a totally new experience of self-constitution.
Still, understanding language is understanding those who express themselves through language, regardless of the medium or the carrier. Literacy brought to culture the means for effectively understanding language in a civilization whose scale was well adapted to the linear nature of writing and reading, and to the logic of truth embodied in language. However, literacy lacks heuristic dimensions, is slow, and of limited interactivity. It rationalizes even the irrational, taking into bureaucratic custody all there is to our life. Common experience, in a limited framework characteristic of the beginning of language notation, is bound to facilitate interpretation and support conflicting choices. Divergent experiences, many driven by the search for the useful, the efficient, the mediating, experiences having less in common among themselves, make language less adapted to our self-constitution, and thus less easy to understand. In such a context, literacy can be perceived only as a phenomenon that makes all things it encomapsses uniform; therefore literacy is resisted. Far from being only a matter of skill, literacy is an issue of shared knowledge formed in work and social life. Changes in the pragmatic framework brought about the realization that literacy today might be better suited to bridging various fragmented bodies of knowledge or experiences, than to actually embodying them. Literacy might still affect the manner in which we use specialized languages as tools adapted to the various ways we see the world, the manner in which we try to change it and report on what happens as a result. But even under these charitable assumptions, it does not follow that literacy will, or should, continue to remain the panacea for all human expression, communication, and signification.
The Functioning of Language
To function is a verb derived from experiences involving machines. We expect from machines uniform performance within a defined domain. In adopting the metaphor of functioning to refer to language, we should be aware that it entails understandings originating from human interaction involving sign systems, in particular those eventually embodied in literacy. The argument we want to pursue is straightforward: identify language functions as they are defined through various pragmatic contexts; compare processes through which these functions are accomplished; and describe pragmatic circumstances in which a certain functioning mechanism no longer supports practical experiences at the efficiency level required by the scale of the pragmatic framework.