The Legacy and Future of Saussurean Semiology in the Study of Art and Visual Communication

Maja Rudloff & Rasmus Kjærboe

Roskilde University, Denmark

The Hirschsprung Collection, Denmark

Abstract

Keywords: visual communication, semiology, images, art, art history

If we are to discover the true nature of language we must learn what it has in common with all other semiological systems; linguistic forces that seem very important at first glance (e.g., the role of the vocal apparatus) will receive only secondary consideration if they serve only to set language apart from the other systems. This procedure will do more than to clarify the linguistic problem. By studying rites, customs, etc. as signs, I believe that we shall throw new light on the facts and point up the need for including them in a science of semiology and explaining them by its laws.” (Course, 1966, p. 17)

Although Ferdinand de Saussure, the founding father of semiology, based and developed his seminal work on linguistic theory, he also introduced the idea that human communication in a much wider sense functions as a “system of signs.” In his most famous and cited work,Cours de linguistique générale(1916)1, Saussure was, as Christensen (2016, p. 489) has recently argued, not explicit about the relation between word and image. However, in the quote chosen to begin our essay, he suggests that linguistics should one day be part of a more comprehensive study of signs, which would study the production of meaning in culture and society. Throughout his work,we find indications and suggestions that Saussure himself considered “semiology”,defined as a “…science that studies the life of signs within society” (Course, 1966,p. 16), to be universal in understanding human culture and communication. By emphasizing that language is not the only set of symbols to carry meaning, Saussure laid the groundwork for analyzing any given form of human communication as a structure once one understands how the general structure of language works.Consequently, his work has had tremendous impact not only on studies of language and literature but also on other fields such as visual studies, albeit through its theoretical application and development by others.

While Saussure’s influence on linguistic and literary studies has been widely discussed and well-documented, studies about his effect on theoretical approaches to visual culture, images and art have been scarcer. Saussure expert Jonathan Culler has argued that the articulation and establishment of a new academic discipline, like semiology, is not just a rare event: it will also almost certainly affect other disciplines,opening up the possibility for the founding of new disciplines and causing selfreflection and scrutiny within the ranks of those already established (1981, pp. 22-23).Culler specifically states that “[t]he emergence of a new discipline […] creates a past,articulates a present, and projects a future…” (1981, p. 24). Following Culler’s line of thought, it seems pertinent to ask how the emergence of semiology has outlined a new trajectory for visual and pictorial studies, and how it has affected art history, an already established discipline devoted to the study of visuality as communication at the time of Saussure’s lectures and the later publication ofCours. In this essay, we interrogate precisely these questions: First, we outline how Saussure’s thoughts on the sign and its relation to visual communication, and “still” images in particular,has developed. Second, we look at its impact on art history, an already established science of visuality, throughout the 20th and 21st centuries. Third, we briefly discuss whether art history’s answer to reading Saussure is a denial or a positive affirmation of political responsibility shared by visual studies in general. This is by no means an exhaustive analysis; rather, we want to highlight what, to us, seem a few of the most pertinent parts of the Saussurean legacy for the study of visual communicationas a system, and some of the challenges and opportunities this legacy has created.

1. Developing a Theory of Pictorial Language

Saussure’sCoursdid not just turn into a foundation for the advancement of present-day linguistics and semiotics; it laid the preparation for the achievement of structuralism in the late 1950s and mid-1960s and inspired later poststructuralist thinking (Christensen, 2016, p. 489; Culler, 1976, p. 93). It was the French structuralists in particular who followed Saussure’s request to study “rites, customs,etc. as signs” (Course, 1966, p. 17), that is, to study human communication in broader terms, thereby exploring the structural systematics of human communication as a set of relations in which meaning is created by differences between signifying elements.One main strand of exploration concerning visual sign language and its relationship to written and verbal language. Discussions on iconic versus linguistic language have always centered on the question of articulation, i.e., whether iconic language possesses an equivalent to letters and phonemes or not, and what constitutes the pictorial signs’ relation to reality. The French scholar Roland Barthes was a pioneer in exploring the language of images and other “imitative arts”, which he insisted had a different, discontinuous structure than that of written text and, consequently, must be studied as such (Barthes, 1977, p. 16). InMythologies(1957), Barthes rooted visual semiology in his interrogation of cultural objects in everyday life, but it was in a number of essays written between 1963 and 1971 (collected and translated by Stephen Heath in the still widely readImage, Music, Text) that he explored and developed his thoughts on a language of images and thus tried to translate the meaning-making processes of pictorial signs into structural terms.

Building on Saussurean structuralism Barthes explored images as “paroles”that must be understood in relation to “langue” and to “language” (the underlying signifying system). One of his main arguments was that the pictorial sign system differs from that of the language system in that it is not, to the same extent and in the same manner at least, characterized by a double articulation (1957). In the language system, there is significant difference between the signifier and the signified.Consequently, Saussure was adamant that the code between signifier and signified was arbitrary, or unmotivated, though he admitted there are some cases of “relative arbitrariness” exclusively, however, in a sign’s relationship to other existing signs in the system and not to reality (Course, 1966, pp. 131-132). In opposition, the analogue,or motivated, relationship between the signifier and signified of pictorial signs is exactly what distinguishes the iconographic system, like other visual reproductions of reality, from its written counterpart according to Barthes (1977). This does not indicate that pictorial language in all its variety functions by identical codes or rules of transformation. Rather, Barthes distinguished between the varying analogical natures of different types of images, such as photography, drawing and painting, by arguing that their respective signifiers provide different treatments of reality. Barthes gave special status to the photograph due to its “copy”-like quality and described it as a “perfect analogon” (1977, p. 17) to reality. Importantly though, he clearly distinguished it from reality itself, insisting that there is a qualitative difference between objects and their photographic renditions. Thus, while Barthes considered the denoted message of a photograph to be codeless, and thus different than other pictorial and iconographic messages such as drawings and paintings, which even in their denoted state are coded, motivated by cultural and historical factors such as aesthetics, style and ideology, he did not consider the photograph to be without invested meaning. The link between signifier and signified (the code of connotation)remains entirely historical and cultural, and thus conventional, even in photography(Barthes, 1977, pp. 27-28). Despite the immediate similarity, then, between a visual signifier and its signified (between concept and image acoustique) their relationship is not as straightforward as one would initially assume.

The analytical insight that it is exactly the analogue, or imitative, quality of an image, and photography in particular, which blurs its representational character and makes it appear purely denotative was a highly important contribution to subsequent structural inquiries of pictorial language; not least to the development of poststructuralism. The discovery of the image’s lack of inherent truth, or its “antirealism”, was one, which also severely challenged existing aesthetic and philosophical thinking about visual art and art history, as we shall later discuss.

2. Towards a Multidisciplinary and Multimodal Semiology

Barthes’ development of the Saussurean idea of the unmotivated sign, i.e., that there exists no “natural” meaning, only “historical”, “conventional” or “cultural”, inevitably involved philosophical theorizing on the role of signs and sign systems other than the linguistic in the construction of knowledge about our social life and society, as Saussure had predicted. The revolutionary assertion that language, far from simply naming an objective reality, actually plays an important role in constructing it (ultimately impacting the way we think and feel about it) was what made German philosopher Ernst Cassirer compare the importance of Saussure’s new science of linguistics to that of the seventeenth century science of Galileo (Culler, 1976, p. 114). It provided Barthes(and others) with the tools for critically engaging with whole regimes of signifying systems that are not exclusively verbal. Barthes’ extremely influential work on popular culture and mass-produced images in the texts “The Photographic Image” (1961)and “Rhetoric of the Image” (1964) paved the way for cultural and critical studies to engage structuralism and semiotics in the analysis of other and related areas of visual communication—often in combination with disciplines such as psychoanalysis and theory of the gaze. Art, film and advertising are all examples of forms of primarily visual sign language that benefitted from these insights. It is hard, if not impossible, to imagine Thomas Mitchell’s reflections on images and art (1974), Judith Williamson’s analysis of gender advertisement (1978), Christian Metz’s theory of film (1974) or Stuart Hall’s work on cultural representation (1997) without Saussure’s insights onlangueand Barthes’ later reworking and application of his theories.

In recent decades, semiological research on visual and other forms of human communication has also inspired and expanded into studies of the convergence and interaction ofdifferentsign systems. From the latter part of the 20th century onwards,Gunther Kress (2001) and Theo van Leeuwen (2010) in particular have been driving forces in developing an independent theory on multimodal semiotics. They attempt to understand and interpret the complex interactions between different types of modes and semiotic resources in various kinds of “texts”. In doing so, they expand Saussure’s mono-modal approach to include an understanding that contemporary communication most often includes several different modes of expression that are all equal in status and importance to written or spoken language.

3. Structuralist Hierarchy between Words and Images

Considering Saussure’s influence throughout the 20th and 21st centuries, there is no doubt, as Newton (1988, p. 83) has argued, that the utilization of language as a model for understanding parts of reality that are predominantly non-linguistic in character sets up structuralism as a ground-breaking option compared to other strategies such as positivism or empiricism. While a linguistic model of analysis has provided an important new set of tools to interpret communication and meaning making, the transfer of this model to other areas of human interaction and culture has also had a significant impact on these other research areas and their subjects, including visual and pictorial studies.

The Saussurean root of semiology in linguistics is mirrored by a terminology and methodology that favors textual analysis and thus treats other semiological systems,including the visual, as systems that can be “read”. Within the field of art and iconography this approach is exposed in a large amount of research occupied with “reading” art and images, such asThe Language of Images(Mitchell, 1974),Reading Images(Kress &van Leeuwen, 1990) andThe Language of Displayed Art(O’Toole, 1994). Throughout his structuralist phase even Barthes, like Saussure before him, considered language to be the superior system of human communication, and so he insisted on a hierarchy between words and images that favored linguistic supremacy (Christensen, 2016, p. 495). From an even more skeptical point of view, the heavy influence of structuralism on the elevation of speech and writing as a prime model of communication has affected academic approaches to visual modes of human communication negatively. According to visual arts scholar Michael Jay (1994, pp. 14-15), the “essentially ocularphobic” and “antivisual” discourse in namely French 20th century thought led to an outright “denigration of vision” and “hostility to visual primacy”.

Yet, it is hardly an overstatement that the history of modernity is also the history of images and their consumption. By the second half of the 20th century, the immensity of cheap and effective visual production, dissemination and reception certainly necessitated new approaches; thus, new types of cultural and specifically visual studies were created.The increasing ubiquity of images and not least the increasing willingness of scholars in different disciplines to consider pictures and other visual materials as subjects of inquiry on the same level as texts resulted in new research areas such as visual anthropology,visual culture studies and visual methodologies to name but a few. An obvious and important strand of this research, albeit not given any further attention here, comes from the technological development of digital media, the overflow of visual resources, and the new forms of use they have generated. Through a continued process of adaptation and permutation, the fundamental idea of analysing the visual as a language of differences has been an ongoing project. However, when the new visual communication studies were beginning to come into being, the study of visuality had long been the domain of a well-established and specialized university discipline: art history.

4. Art History and the Challenge from Semiology

What happens when thisother, much older discipline devoted to the study of visuality—art history—is confronted with a not quite defined discipline, which poses several fundamental challenges to the ideas of what the visual is, and how it signifies? What happens to the privileged discipline of art history in light of the fresh and vital ideas that visuality is now a semiologicalsystem, and that no meaning or value originates outside of that same system? Given the immense prestige of art history for most of the 20th century and its inertness in the post-war period, the Saussurean “challenge” is important. This would become a battle over methods and the justification of what to study … and what not to.

Art history—which to some degree is based on a range of “common sense”ideas about images ranging far beyond academia—is especially challenged by two consequences of Saussure’s semiology and its later elaboration in visual and cultural studies: anti-realism and relativism. Of these, anti-realism would prove the easiest for art history to overcome or simply bypass, but not without problems. If concepts and their expression are not motivated—i.e., if everything is conventional and a naturalist painting of a tree is not much closer to the actual tree than any other sign, such as the word “tree” or a few superimposed triangles denoting a Christmas fir—then classical art history is in trouble. For much of the existence of art commentary and criticism in the Western tradition, an essential relationship between a picture and its referent has been presupposed. Naturalism and “truth to life” have similarly been held as important standards by which to judge order and understand art production. This was the premise in writings from antiquity in, for example, Pliny’sNatural History,and to those of the “first art historian”, Giorgio Vasari, writing in late Renaissance Florence in the 1560s. It was also the undisputed premise in the writings of J.J.Winckelmann, the idealist founder of modern art history in the late 18th century,and it can be argued that this premise underpinned most of art historical writing as it developed into a university discipline in the latter half of the 19th century (Podro,1984). Very broadly said, to classical art history, as it developed in dialogue with centuries of commentaries and judgements on art, the highest level of civilization and knowledge—and even pleasure—would result from those depictions considered most realistic or rooted in Nature. Judgments on pictorial realism and truthfulness—to whatever prevailing standard—would be an important methodological concern and aim for the discipline of art history as it emerged.

Saussure’s semiology seemingly throws a wrench into the workings of motivated signs. There is no outside and true standard with which to judge a signifier’s essential relation to a signified, which also means there is no way to judge an artwork’s fidelity to nature; Saussure’s swift and decisive dismissal of onomatopoeia serves as a pertinent example (Course, 1966, p. 176). This critique of representation as “truthful”holds even though Saussure says little about imagesper se, and Barthes at some point argues for a degree of motivation. Pragmatically, most visual representations appeal to some referent through likeness, but likeness seems to be mostly a conventional,and probably a culturally defined, concept, with no way to distinguish quality—i.e.,“more” or “less” likeness. There is no way to conclude that, for example, ancient Greek art with its foreshortenings and attention to anatomy is “truer” and therefore more essential and “advanced” than ancient Egyptian art which had comparatively different means of expression. Brilliant portraitists such as Hans Holbein the Younger could therefore no longer be regarded as representing “the final truth” about the person portrayed or any other truth verifiableoutsideof the semiology of art as a system or the semiology of visuality overall. Brilliant artists go from visionary seers and truthtellers to merely skilled manipulators of a given visuallangue.

Yet, anti-realism and pragmatism have always been obvious elements in art production, and therefore unsurprisingly also of commentary on the arts. Witness Jacob Cats’ emblem books of the 17th century, which made a sport out of assigning complicated moral lessons to complex and often outlandish pictures (Luijten, 1996).Consider when, in the 20th century, German-American art historian Erwin Panofsky,who was in turn indebted to philosopher Ernst Cassirer’s work on symbolic forms,practically refounded iconography as the encyclopaedic knowledge of pictorial conventions of different ages and environments (Hasenmueller, 1978). Austrian-British art historian Ernst Gombrich published his widely readArt and Illusion: A Study in the Psychology of Pictorial Representationin 1960 to bring to light a century’s worth of quiet art historical scepsis toward the idea of natural representation, and would almost become an early advocate for visual culture studies by seemingly giving up on art(Wood, 2009). Certainly, in art criticism, and especially in the commentary trying to make sense of artistic modernism and avant-gardism, the move away from naturalism and the exponential increase in experimentation saw various pragmatic, anti-realist defences against the proponents of “truth to Nature”, until eventually elements of semiology, semiotics and structuralism would be directly referenced, albeit sparingly, in art history and commentary from the 1970s onwards (e.g., Krauss, 1977; Bryson, 1981).Finally, in 1991, the by-then substantial Saussurean legacy developed elsewhere would symbolically arrive in an embattled article, “Semiotics and Art History”, by semioticians Mieke Bal and Norman Bryson, beginning with the line: “The basic tenet of semiotics,the theory of sign and sign-use, is antirealist” (1991). But by then, somewhat ironically,the swift entry of Saussure into art history curricula would come wrapped in his poststructuralist interpreters: Jacques Derrida, Jean-François Lyotard, Jean Baudrillard and the late authorship of Barthes. This meant a considerable disconnect between its critics—who already presupposed a passing knowledge of structuralist thought—and an art history which in many respects had given up on “theory”, and was therefore still mired in a very implicit Hegelian-type idealism (Elkins, 1988; Summers, 1989).Institutionally, this disconnect would lead more than a few art historians to simply dismiss serious concerns over the discipline as indeed mere “theory”, at least in the experience of the authors of the present article.

5. The Challenge to Art History from Saussurean Relativism

If art history had less trouble overcoming Saussurean anti-realism as it directly relates to a critique of representation and truth in visual representations, then the other, and very much related, challenge remains unresolved: relativism. How do we justify the intense study of pictorial artworks—mostly produced in the West during the last 500 years—out of so many different “semiological systems”? How do we even separate art from non-art, let alone good art worthy of many hours of study from “bad” or simply unimportant art? Traditional art history was, and is, in thrall to an idea of genius artists rising above the crowd and leading the way to vaguely defined “insights” via their art (Soussloff, 1997). Even art history indebted to critical theory and Marxism could be criticized of carrying around the heavy baggage of a “notion of immanent aesthetic value” (Moxey, 1991). The radical consequence of Saussure’s legacy—that there is no outside to the semiological system—is that there are no permanent,transcendent or immanent values with which to justify the obsessive study of pictorial art in the Western tradition at the expense of other signifying systems. If we want to know about pictures, and if we deem visuality an important semiological system, the elitist, exclusionary history of Western art, which privileges easel painting, seems a very narrow slice of the whole of the field of visual production and consumption of images and pictures in both the past and future. And yet, art history has indeed held a place as the most prestigious discipline directly devoted to visual study: it is the “corpse that cannot be buried” (Soussloff, 2005, p. 204). Consequently, in the wake of poststructuralism, academic art history saw prolonged critiques of its field of study and its delimitation and hierarchies (e.g., Pollock, 1988; Preziosi, 1991), but also of the difficulty art history had with addressing visuality beyond the narrow confines of “high art,” sculpture and easel painting (Walker & Chaplin, 1997, pp. 31-48).

Art history obviously still exists as a discipline in the second decade of the 21st century, and mainstream art history is still kept separate from other types of visual studies, such as visual culture, even in the face of the challenge from post-structuralist semiotics. In practice—even when the justification for it no longer makes sense—some areas of visual production receive a lot of positive attention, while others do not. The continued status of art history as a distinct approach to visuality skews the whole study of visuality, which is a problem both from the standpoint of politics and of common sense. One might even say that art historians have seized upon Saussure’s insight that semiological systems are self-referential and conventional while ignoring the relativism.With a slight shift in focus, the interpretive power of art history remains unscathed by now becoming a mastery of “codes”. The discipline has done so while paying lipservice to its many feminist and post-colonialist critics that have questioned the biases of art history and the discipline’s complicity in racism, sexism and classism. And it has done so while ignoring other types of radical challenge to its hierarchies of value,immanent aesthetics, and its separation from the study of the larger system of visuality going back to Saussure’s original semiology. The prestige and money invested in art and art history is considerable, and as Western capitalism has been able to contain, embrace and even capitalize on all of its critics, so has art history.

Some prominent art historians have indeed engaged vividly with the breadth of the Saussurean legacy, as much of the referenced literature shows, but this engagement does not seem to have fundamentally changed the discipline. As argued, the larger institution of art history has been markedly schizophrenic in meeting semiology’s wider consequences by gradually acknowledging the “lesser evil” of anti-realism,all the while ignoring relativism. This state of affairs leads the authors to two not necessarily mutually exclusive interpretations of the future—one pessimistic, and one optimistic—that concern art history in particular, but also, by implication, a wider field of visual studies.

6. The Future

From a pessimistic viewpoint, tradition and the prestige of art history serve as effective brakes on any challenge to the narrowness and exclusivity of the field of study. If the divisions between “art” and “the rest of visuality” stand, the whole field of visual studies is potentially skewed. A case in point is the emergence of “visual culture studies” as an answer to the shortcomings of art history and its subsequent failure of overtaking or reforming the scope of its “parent” or “sibling”. First institutionalised at the University of Rochester in 1989, later emerging in connection to art history departments across the(Western) world (Smith, 2008), for a moment, the discipline of visual culture studies did seem to be a way forward to those disillusioned with the failures of art history to meet the challenges of its relativist critics. As a sub-department or a bachelor’s or graduate degree, visual culture might have been imagined as a kind of Trojan Horse by those who hoped to radically reorient art history from within.

But as it stands today, visual culture seems to stand at a remove while the priorities and topic of study of institutionalised art history have not fundamentally changed. In some instances, visual culture rather seems to have become a kind of supplementary art history (without history) dedicated to examining contemporary fine art. In this pessimistic version of events, the Saussurean legacy has contributed tremendously to the emergence of new ways of studying visual communication, but the legacy’s various interpreters have only had an impact where there was not already a strong tradition such as art history in place.

In an optimistic version of the future, the discipline of art history is indeed changing,but not by dissolving itself into a mist of relativism. Saussure held his lectures that would in time become theCoursat a time when academia prided itself on objectivity and stringent logic, as did art history, but the afterlife of his thoughts would do more than perhaps any other single influence to undermine belief in a value-free and objective science of the cultural. A solution to the Saussurean “challenge” for a number of art historians has been to take a stand probably alien to the theorist himself and explicitly acknowledge the politics of their discipline. This no longer means to abandon art history, but to be pragmatic and mobilize its prestige as a potential power for change and to commence a gradual, revisionist course of re-distribution of attention and scope.

Parts of art history have since the 1990s sought to allocate real prestige to new and often deprived groups of people—producers and audiences alike—or to marginalized areas of study, chiefly including more focus on women artists and artists of color.Critical self-examination of the in- and exclusionary structures that govern the making of art and its history/ies has also led to the admission of new forms of visual (and increasingly aural) products from around the world as worthy of study under the prestigious label of “art”. Some of this change of focus has happened in alliance with other related and developing areas of study, such as critical museum studies and activist ideas of social change (e.g., Simon, 2010). The “new museology” (e.g., Vergo, 1989) has broadened the critical attention in art history to include its institutions, their curatorial practices and their audiences, identifying for example the mechanisms of visiting art museums as social structures that are culturally and historically contingent rather than stable and pre-defined categories (see also Bourdieu, Darbel & Schnapper, 1991).

In this optimistic view, then, the legacy of Saussure is contributing to a long and slow expansion of art history. This no longer means to abandon art history, but to be pragmatic, “working with what you have got”, and mobilize its prestige as a potential power for change. Instead of dismantling the discipline—which has proved impossible in practice—it is being opened up and harnessed for change.

Glimmers of a political project are perhaps most visible when seen in an old and entrenched discipline such as art history, but all over the various fields of visual studies, it has been a central concern how to address ethics and politics for decades.This development is also—at least in part—related to questions raised by relativism.In the end, the answer to the Saussurean challenge might not be to hope for more relativism in the future of what to study—in art, in pictures or in visuality—but rather to admit to and then live with the inherent absurdity of ever privileging one topic.From that admission, the legacy of Saussure might be a necessary “reality check”,a prerequisite for continuing to do studies that from a contemporary perspective are both considered and responsible.

Note

1 Published three years after his death by Charles Bally and Albert Séchehaye, the book was based on edited student notes from three series of lectures that Saussure gave at the Université de Genève 1907-1911 (Christensen, 2016, p. 490).