sounding garments & sound-gardens: emotional interface zones as auditory spaces

Hana Zeqa, Laura Thaçi, Elisabeth Mirnig & Werner Jauk

the transparency of envelopes of the body as social interfaces amplifying the hedonic body

I. An Every Body Culture
Human Virtualities – Extending the Needs of the Body
Where, through “robotics,” digital culture extended the mechanical body – overcoming the body –, post-digital culture extends the hedonic body – where visual culture turns auditory by respecting the “needs of the body”.
Media theory is the theory of how any extension of (wo)men (McLUHAN) extends “perception” as body-environment-interaction / B-E-I (GIBSON 1982) for better (optimized) “survival.” While “idealistic” cultures exerted discipline over the hedonic body, pop-cultures “amplify” the hedonic body by technologies extending it “close to body.” As technological amplification mediatizing hearing, popular music becomes paradigmatic of close to the body garments & gardens structured by “pleasure”―that is, extending bodily tension to amplify “pleasure”. Hedonic hearing generally is the benchmark for the interaction of bodies with environments creating human virtualities: an auditory culture.

Any kind of “aesthetic” behaviour is explorative. It is body-environment-interaction because of the “affordance” of the sensorial perceived intensity of stimuli in the environment. This also makes it “intentional” in the pre-enlightenment sense, before bodily hedonic processes of tension-relaxation shifted to (relational) thinking through the dominance of seeing, which just meant being “in tension”. Today, biosemiotics explains the emergence of signs from those “primary meanings,” and refers to tension triggered by environmental stimuli as their meanings to the body. Even early semiotics called it “signal,” a “phenomenon which provokes reactions in […] organisms, without being the cause of these reactions”(Pazukhin 1972, p 41).

While symbols are (political) distinctive media, because of the need to have knowledge about their referential meaning, signals are “human,” that is, their meaning is common to every “body”.

In summary, it can be stated: tension is the “drive” of any exploratory behaviour. It is the “affordance” of the environmental stimuli; it is intensity, which brings the body “in tension” to interact with the environment in order to create a homoeostatic level of arousal and, through it, culture.

This is how we interact with physical and social environments. Intensity indicates closeness of occurrences to the body. In physical situations, motion – heard all around the body – induces “tension”. In social interaction, crossing zones of privacy / intimacy induces “tension”. Experiments show that there are interaction-spaces around the body: About twice the height of the body marks the social interaction space, the range of the arms covers the intimacy-space. Finally, the feeling of “tension” within these social distance-zones is modified by the duration of eye contact; reinforced over 2 sec.

Although enormous cultural differences exist, there is a natural bodily basis for the borders of the body and its interface-zones. Garments, and gardens, shield intimacy from – and play with – entry into natural and social environments.

The Sound of Music & Auditory Culture
Hearing is like an envelope covering the body, “functioning” as an interface: sound is an artifact of motion, the ear as an extension of the skin experiencing this “vibrating” motion.

Sound is an artifact of motion. Auditory controlled body-environment-interaction “perceives” motion as “vibration” all around the body setting /putting the body in tension, in e-motion. At the same time, the body expresses its tension as the meaning of this perception to the body producing expressive voice and expressive behaviour. Instrumentarisation and mediatisation of this auditory controlled B-E-I leads to music.

First it was the instrumentarisation of the bodily behaviour, as bodily imitation of the impression of motion of sound around the body and the expression of its meaning to the body; its tension, the sound-gesture, which showed the melody-contour to singers of a clerical choir in the Middle Ages. Its projection to the plain led to the first graphic-notation, the “neuma”. Its gridding in time- and frequency-domain led to the notation, making possible the com-position of codes for sounds, the prescribed Werk. This process of mediatisation is a process of extending the body, and a modal turn from hearing to seeing, by this from experiencing “tension – relaxation,” from hearing sounds to understanding codes of sounds by “relational thinking”.

While notation is converted to sound through instruments, digital codes allow us to mathematically generate sound by technical conversion, finally shaping the sound by bodily tension directly. This step forward is a step “backwards” to a music close to the body, structured immediately by bodily “tension-relaxation”. Despite producing music, this behaviour is a hedonic interface perceiving motions even as it expresses and amplifies their e-motions.

The Sound of Garments & Gardens
Sound-space is an interface-zone to the skin experiencing motion. Garments are extensions of the skin, feeling the environment as well as expressing the bodily tension, the e-motion of the intimacy of the body by drapery and variable morphs echoing the tension of the body. Gardens are further extensions of the body to the social and natural environment ―enlarging the private space, creating a playing ground, extending intimacy to social space― in contrast to the structure by force of law; the organized, symbolic social structures of a public space. Garments, like gardens, are interface-zones of the body into natural and social environments that create them. Their symbolic communication is “fashion,” which creates specific forms by referential processes of culture. Despite this symbolic use, there is a basic natural benefit of these “sounding” envelopes for human life.
This “hearing” kind of bodily interaction is common to every body; it is the natural basis democratizing the culture by / for every body; while culture, as symbolic form, is a sociopolitical distinctive medium.

II. Cultural Studies between Kulturwissenschaft & Epistemological Media-Art. Culture as Symbolic Form & Culture out of Nature.
Enriching a modern culture of seeing and the extension of the mechanical body through robotics in the digital age is overcoming the (mechanical) body by the hedonic interaction of hearing; a necessary step, following the transgression of the mechanistic paradigm and the overcoming of the mechanic body, where post-digital culture converges the hedonic body with virtualities. As pertains to the needs of the body, this culture will be a human culture.

Through this, the concept of knowledge and methods to reach it changed: although the externalization of perception is aspired to, its formalization is turned from symbolic representation to immediate presence. Methods turned from seeing and standing in front of an object to understand it, to formalizing it symbolically, to hearing being inside an occurrence-space experiencing it bodily. To get knowledge goes from Kulturwissenschaft to epistemological media-art.

While Kulturwissenschaft focuses on the culture of symbolic representations of interactions, biosemiotics focuses on the upcoming of meanings because of bodily meanings, “the branch of semiotics dealing with the organic part of the semiosphere” (MAGNUS 2020, 649-650).

In Kulturwissenschaften, (popular) music, as well as the culture of garments and gardens, are primarily discussed as self-referential systems; as “fashions” in time and space. Both have a common basis, the nature of the body; both are mediated “products” of needs of the body. Even the need to be protected and nourished more and more became shaping envelopes of the hedonic body. Protection of the physical body became its decoration: garment became fashion, nutrition became pleasure, guarding the home became gardening the pleasure-garden.

Nevertheless, its theoretical exploration didn’t cease to be a game with signs referring to specific cultural contexts (re-)creating “new” realities (varying in closeness to, or domination, of nature.)

The project sounding garments & sound-gardens is about the basis of this semiotic “point of view.” Where do signs come from? Is there a natural basis to culture, a pre-semiotic communication “beyond semiotics” (JAUK 2013) based on the nature of the body? This explores the hedonic basis of rational “symbols” signifying point of view. The specific, phylogenetically older “hearing” that focuses on the tension of any behaviour referring to the “needs” of the body is the basis for any explorative mechanical interactions. This is the basis of an Erlebnisgesellschaft, as well as for the turn from the mechanical-digital overcoming the mechanical body to the post-digital integration of the hedonic body and, through it, to an “auditory culture” as new humanism.

Hearing, mediatized and, through this, formalized as music, becomes the paradigm for interaction in creating a digital humanism.

Science & Arts – Methods of Re-Constructing Realities
Epistemological media art is interested in knowledge. While science represents the presence, media-art experiences the presence. While sciences tries to get knowledge by externalising experiences through codes that represent presence, media art externalizes bodily experience through bodily experience (communicating bodily experiences directly by experiencing the presence) to reach body-knowledge. This, however, happens within the same experimental design: varying a “variable” to observe the effects on another, which is to be “explained”; observation in controlled labour-situations replaced by real-life bodily experience in social public spaces. Research is a part of reality-construction.

III. A Piece of Art & Science. Experiencing Body-Knowledge and its Extension
(Pop-)Music as Paradigm.
To “explain” and act culturally with natural bodily tension as meaning (of sound) to the body and its extension is to formalize and communicate body-knowledge.

Auditory behaviour as hedonic B-E-I is the paradigm for any explorative behaviour and its extension. The interaction of the body with the environment to create garments and gardens is a hedonic extension of the hedonic body. These extensions amplify the tension. Auditory interaction is not only the basis of hedonic interaction; sound also externalizes this tension to make it not understood, but experienced.

Hypotheses: in extending the body close to the body, music – as well as fashion – are primarily extending the hedonic body by mechanical instruments. How the mechanics of the body “move” is given by the tension of the body expressing this meaning to it im-mediately and communicating it through “emotional contagion” (HATFIELD 1994), copying the behaviour and internalizing the tension. The same can be said for “sound-gesture”. Garments and gardens are hedonic interfaces to the social spaces following the paradigm of auditory space.

Experimental Art Setting: in some kind of experimental setting, we vary the kind of extension of the tension of the body (to indicate emotional qualities in a social interaction). This kind of mediatisation of bodily tension is varied, to experience the effect on communicating bodily expression. So we have a more “physical” extension of the expressive body vs. a more “artificial” extension of the expressive body, for the (possible) differences of the “effect” on bodily communication to be experienced as body-knowledge.

Process of Operationalization: based on the theory of the development of music as the mediatisation of bodily tension in interaction with the (social) environment, and according to the aims of an epistemological media-art, the project has to make this process of mediatisation through garments felt.

Despite a labour-experiment, tension is induced in a real-life-situation by bodily interactions of critical closeness. Social interaction takes placed within a range, where attention – or rejection – inducing behaviour is initiated to “produce” tension in the participants. In addition to its expression through garments, this tension is externalized and amplified as sound (following the basic theory of “sound-gesture”) to create an emotional-social sound-garden of pleasure, communicating body-knowledge.

Finally, any kind of media “creates” specific mediated realities. How are these different realities experienced?

The extension of the hedonic body is varied through two qualities of garments expressing bodily tension in social interaction in their specific “medium”: a) the dynamics of drapery by handcrafted tailoring-techniques and b) the dynamics of morphing by digital interface-techniques.

a) the dynamics of drapery (Elisabeth Mirnig) operationalizes the dynamics of tension by focusing on the drapery of a loose body-suit with openings for the natural interfaces, externalizing the dynamics of (e-)motion through the dynamics of clear “lines” (as it is simplified in motion tracking systems), making tension visible. Winckelmann calls garments “the echo of the body”, their tension. By means of tailoring techniques and the use of the imagination in the fine arts and sculpture, garments amplify the body echoing its tension. In addition, they “modification” of the body-shape through mechanical tools (e.g. belts, etc.)

b) the dynamics of morphing (Hana Zeqa & Laura Thaçi) operationalizes the dynamics of tension by “extending” specific body parts of “garments,” based on natural ex-tension of the body associated with “emotion” in specific (social) interactions. Based on studies of the “acoustic driving effect,” which shows respiration frequency and amplitude as synchronous to temporal dynamic aspects of sound and music, these parameters are detected by strain gauges measuring the movement of a person’s chest. If sufficient movement is detected, the garments feature is triggered, and the garment of multiple woven fabrics reshapes the natural extension. At the same time, a signal is sent to the central workstation, where sound and light externalizes and amplifies the body-tension.

This amplifying externalization of the different kinds of mediatisation of bodily tension in garments through sound and light makes the garden a sound-garden and brings knowledge as a bodily experience, as body-knowledge; being part of a real-life-situation. AB studies the hedonic social interaction even as it creates a cultural situation. This follows post-modern and post-digital concepts of “science & art” experiencing and creating realities as cultural process, as human life.

References:
Gibson, J, J. (1982). Wahrnehmung und Umwelt München.

Hatfield, E., et al. (1994). Emotional Contagion, Cambridge

Jauk, W. (2013). “Beyond Semiotics? Music, A Phenomenon of Mediatization. The Extension of the Hedonistic Body and Its Communicative Aspects,” in: New Unknown Music. Essays in Honour of Nikša Gligo, Eds. Dalibor Davidović & Nada Bezić, Zagreb, pp. 407–421.

Magnus, R. u. Kull, K. (2020), “Roots of Culture in the Umwelt,” in: The Oxford Handbook of Culture and Psychology, Ed. Jaan Valsiner, Oxford, pp. 649-661.

Pazukhin, R. (1972). “The concept of signal,” in: Lingua Posnaniensis 16, pp. 25-43.