Skip to Main content Skip to Navigation

Visual, audible and verbal: synthetic intertextuality

Abstract : Basically all communication is inter-semiotic. even those texts which traditionally have been understood as based on one code now seem to be the realization of at least two code systems. the verbal communication in its two basic variants is also poly-code: the oral presentation of a text cannot help mobilizing the so called para-verbal aspects of speech (tempo, timbre, intonation, etc.) which account for a large portion of information and, at the same time, modulate the message conveyed by the lexemes and syntax, and sometimes, as in the case with ironic intonation, even completely invert the content of the speech. no less important, although more obscure, is the manifestation of the visual aspect in the written (typed) word. We are not speaking of such universally known facts as William Blake's hand-made books, or contemporary visual poetry where the word may be completely ousted by an image. the very type of a font, the structure of a page (the layout of the text and the way it is positioned in between the margins), the cover, as well as the texture of paper - it all can produce additional information which may drastically affect the process of reading and evaluating texts. the very fact that the verbal and the para-verbal correspond and correlate calls for the theory of inter-semiotic intertextuality, where the visual, being just one element of an intersemiotic continuum, may be viewed upon as influenced by other constituents of the whole - verbal, audible, plastique, etc., and, the other way round, as exerting influence on those. The scholars who would attempt to create the basics of this theory should outline the set of priorities under which such poly-code synthetic texts might function, and the very mechanics of their functioning. there is a certain historic background to this theory, rooted way back in the romanticists' endevour to build up the theory of the universal art, and in richard Wagner's speculations on the synthetic character of the opera. Some attempts have already been made in the field at present, and those proved to be successful. in russia, namely in Moscow and Kaliningrad, a group of scholars spent a couple of decades in the late 1900s trying to revise the efforts made earlier and to give those a new life. to follow these, though rather independently, a russian scholar, psycholinguist alexander Sonin, has undertaken recently the analysis of the verbal-visual texts (commercial advertising and comic strips) showing quite clearly how the two constituents of the multicode text collaborate in creation of the aggregate meaning of the whole. So, the author shows that within the procedures of the cognitive processing of the poly-code text consisting of verbal and visual structures, the latter presuppose the basic representations of the content, while the verbal component tends to modulate the emerging meanings. in other words, in the process of the poly-code text's comprehension its visual component forms a context within which the verbal information in decoded, helping in its turn to decode the information which the visual component contains. Cognitive science provides the theoretical background to the studies of the poly-code text and the role which the visual component of this text plays in the formation of the whole structure. the semantic representation of the aggregate content is formed, presumably, within what the theory of the conceptual integration calls "the Blended Space", "the inputs" being formed by the contents of the verbal and / or visual components of the whole. although with a different purpose, the potential of G. Fauconnier and M. turner's theory of conceptual integration in dealing with the poly-code texts (visual and verbal) has been vividly illustrated by Kenneth a. Mcelhanon in his studies of commercial advertising. the question, though, remains: although we can easily model the processes which are under way on the mental level, to show how those are imbedded in texts still makes a problem, due to the fact that the gap between the verbal and the pre-verbal is still beyond the scholars' understanding. in the meantime still there are perspectives to deal with. in addition to alexander Sonin's observations on how the different structures of the poly-code text correlate on the cognitive level, some other theories may also be taken into consideration. though not dealing with the visual text (her interests lie within the sphere of a different type of the synthetic text, namely - verbal and musical), M. langleben in her studies of the classical song shows how the musical component of the wholesome structure forms a sort of a superstructure which, in the process of the song's movement, restructures the meaning of the verbal text - in much the same manner as it happens with static visual-verbal structures in a. Sonin's examples. to combine all the above theories in one, and to show how the aggregate meaning of the whole is being formed within the frame of a poly-code text is possible in the perspective of yet another theory which was formed to satisfy the needs of the contemporary aesthetics, the theory of intertextuality. although dealing with verbal texts, this theory in its classical form, elaborated by J. Kristeva, a Bulgarian-French philosopher and literary critic, may be quite helpful in solving the problem of how the components of different semiotic origin correlate in the poly-code text. the basic idea of the theory of the intertext (corresponding to the basic views of richard Wagner) is that of the counter-annihilation of the discourses that collide in the process if the intertext-making - with a perspective of creation of new meaning which cannot be traced back to the sum of the initial constituents of the emerging semantic structure. Having elaborated a theory of this kind the scholars will be able to deal with a host of texts where the aggregate semantics is being formed on the basis of donations from different semiotic sources, actually, with any text of Culture (literary, visual, plastic, musical, etc.), which, by its very nature, is the manifestation of the principles of synthetic, and intersemiotic, intertextuality.
Document type :
Conference papers
Complete list of metadatas
Contributor : Compte Laboratoire Geriico <>
Submitted on : Tuesday, July 2, 2013 - 3:06:37 PM
Last modification on : Tuesday, July 2, 2013 - 3:06:37 PM


  • HAL Id : hal-00840451, version 1



Lyudmila Isaeva, Natalia Komina, Stanislav Krestinski. Visual, audible and verbal: synthetic intertextuality. Communiquer dans un monde de normes. L'information et la communication dans les enjeux contemporains de la " mondialisation "., Mar 2012, France. ⟨hal-00840451⟩



Record views