Impermanent Zen Garden · 无常禅园

Read More

本文中文版链接 (Chinese version):

Impermanent Zen Garden contrasts the Buddhist idea of impermanence with the principle of mindfulness meditation. Zen practitioners believe that through mindful observation one can gain wisdom. Therefore, Zen gardens were built around the world to provide serene environments for meditators to focus their minds.

However, in this Impermanent Zen Garden, almost every aspect of the environment is constantly changing. Once the meditator focuses their mind on any object in the meditation room or in the garden, they will notice the moving mountains on the doors, the floating clouds on the walls, the smoke being blown on the ceiling, the water permeating through the tatami, the water stains coming and going on the garden walls, the touring glosses on the rocks, the traveling ripples on the sand, and the blinking stars in the night-sky. Not any two moments are identical, so that it seems impossible to fully observe any moment.

Therefore, the dilemma is thrown to the audience, and the decision is up to them to make. Are you going to concede that the world is ultimately agnostic, or are you determined to embrace each and every present moment in this impermanent world?

Impermanent Zen Garden is a dynamic environment made with Unity. The ever-changing contents are generated in real time using custom shader programs.

Shan Shui in the World · 世间山水

Read More

本文中文版链接 (Chinese version):

Shan Shui in the World presents shanshui (山水, landscape) paintings of selected places in the world generated by a computational process based on geography-related information.

This project revisits the ideas implicit in Chinese literati paintings of shan shui: the relationship between urban life and people’s yearning for the nature, and between social responsibility and spiritual purity. For an audience living in an urban area, a traditional shanshui painting provides them with spiritual support through the depiction of the natural scene of elsewhere. With generative technology, however, Shan Shui in the World has the ability to represent any place in the world—including the city where the audience is—in the form of a shanshui painting based on geography-related information of the place.

The notion that shan shui can exist right here (though in a generative parallel world) not only underscores the contrast between the artificial world and nature, but also reminds the audience of an alternative approach to spiritual strength: instead of resorting to the shan shui of elsewhere, we may be able to obtain inner peace from the “shan shui” of our present location by looking inward.

The Generative Process

In this first production of Shan Shui in the World, the shan shui of Manhattan, New York is generated based on its building information. The generative engine was written in C++ with use of creative coding toolkit openFrameworks. The code that renders the shanshui painting was written in OpenGL Shading Language as fragment shaders.

Height and area of the buildings in Manhattan, New York plotted according to their location.

Adjacent buildings merged into mountains, indicated by colors.

Outline of the mountains generated based on building information.

Mountains rendered in the style of ink-wash painting.

Mountains rendered in the style of blue-green shan shui.


The generative shanshui paintings were printed and framed into traditional Chinese scroll paintings, and inscribed and sealed by hand.

A partially unfurled handscroll, together with a furled one in a samite box.

Details of a scroll painting.

Two seals and their imprints, together with red ink and a carving knife.

Generative Shanshui Paintings

Scroll of Shan Shui in Manhattan, New York. 2016. Handscroll. Ink on paper. (192 × 12 inch) Scroll left to see the whole painting.

Downtown Manhattan, New York, High Distance. 2016. Hanging scroll. Ink on paper. (24 × 55 inch)

Uptown Manhattan, New York, Level Distance. 2016. Hanging scroll. Ink on paper. (24 × 55 inch)

Scroll of Blue-green Shan Shui in Manhattan, New York. 2016. Handscroll. Ink and colors on silk. (178 × 12 inch) Scroll left to see the whole painting.

Blue-green Downtown Manhattan, New York, High Distance. 2016. Hanging scroll. Ink and colors on silk. (24 × 55 inch)

Blue-green Uptown Manhattan, New York, Level Distance. 2016. Hanging scroll. Ink and colors on silk. (24 × 55 inch)

Scroll of Blue-green Shan Shui in Baltimore. 2016. Hanging scroll. Ink and colors on silk. (20 × 55 inch)



(Credits: The geographical data used by Shan Shui in the World is from © OpenStreetMap contributors, Who’s On First, Natural Earth, and through Mapzen.)

Observe the Heart · 观心

Read More

The video above shows a technical prototype for Observe the Heart.

If you ask a Zen master how to meditate, he might answer you, "Observe the heart." But the heart is so abstract to imagine, not even to mention observation. Observe the Heart is an artistic attempt to represent the meditator's mental state, generating visuals and sounds based on realtime brainwave input. The generative visuals are projected back onto the meditator, transforming the introspective meditation into an observable performance, in a sense.

There is more to tell about the concept. While third-party audiences can watch and hear one's meditation, the meditator themselves couldn't experience the generative contents in real time (given that they close their eyes during the meditation, and may even wear earplugs to block the sound). It is then questionable who is this meditation for. Moreover, the meditator will nonetheless be curious about how their meditation looks and sounds like, and this mental activity will be captured by the brainwave sensor and be reflected by the generative output. Therefore, it could make it even harder for the meditator to really "observe the heart".

The experience is designed to be installed in a dark room. The meditator sits in the center of the ground, with a projector projecting the generative visuals onto them. The audiences watch the meditation from above in order to get a better view. In this demonstrative production, a NeuroSky MindWave Mobile EEG headset is used to sense the meditator's brainwave. An openFrameworks application analyses the brainwave signal, and drives a GLSL fragment shader to render the generative visuals, and a Max patch to generate the sound. The generative approaches could be enriched for better output in future productions.

Shader Odyssey

Read More

A tribute to Stanley Kubrick's 1968 masterpiece, Shader Odyssey is a two-minute movie entirely done in one fragment shader, which is a kind of program that runs on the GPU (graphics processing unit) of the computer. All the graphics are computer-generated in real time — no 3D modeling or pre-rendered texture is used. The blending and timing are done in the shader as well.

One of the characteristics of shader programs is that in each frame, the color of all the pixels are computed in parallel (simultaneously), which makes it feasible (not computation-heavy) to scale the canvas and duplicate the content to a great degree. That's one of the reason why this movie includes so many repeating patterns—by duplicating things, I also want to evoke a feeling from the audience about how tiny and insignificant we human-beings are in front of the cosmos.

The original shader code can be accessed via GitHub.

(Music credit: The City of Prague Philharmonic Orchestra, 2001: A Space Odyssey, 2002.)

Oculi: A Show of Alternative Spaces

Read More

腾讯视频链接 (video link for visitors in China):

Long after our farewells to the forest, the savannah and the cave, the homo sapiens are so accustomed to living in human-scaled cube-shaped white boxes. Having been interacting with this kind of space since birth, we take our boxed life for granted. We tend to not take much consideration of other forms of space, and even neglect our consciousness of space most of the time.

Oculi aims to wake up the audience’s sense of space. Being presented within an apartment, it consists of six installations under the name of oculus. Through the use of projection mapping, each oculus brings the image of a poetic alternative space into the exhibition venue, superimposing it onto the physical space of the apartment. These shape, scale or location alternations, in contrast to the otherwise ordinary living environment, evoke the audience’s nostalgia for their forgotten sensation and imagination of space.

Little Universe · 小宇宙

Read More

腾讯视频链接 (video link for visitors in China):

There they are, poetic objects in the Little Universe, revolving around themselves, whispering to each other, regardless of whether or not they are visited by you—their exotic guest.

Colorful the objects are, and they are not mindless clouds of stardust. Each object has its own temperament— a tendency to get close to someone, and to run away from another. In this way, the Little Universe reveals its rich dynamics.


This interactive installation is designed to bring about a serene and meditative atmosphere. It has an intimate physical form, inviting the audience to interact with it, to touch and move it with their hands. Without interfering with the dialogue between the objects, the audience can still enjoy the poetic scene by watching, listening and contemplating their behaviors and relationships.

While the audience is not explicitly informed of the logic behind the behaviors, the five-element theory (五行) of Chinese philosophy inspires the distinct characteristics and relationships between the objects. The colors of the five objects represent the five elements—metal (金), wood (木), water (水), fire (火), and earth (土). There are two orders of presentation within the five-element system: in the order of mutual generation (相生), one element can give birth to another, while in the order of mutual overcoming (相克), the element can suppress yet another. Represented by the attracting and repelling forces between the particle systems surrounding the objects, the relationships between the poetic objects enrich the contemplative intricacy of the Little Universe.


The Little Universe installation is built to integrate four components:

  1. poetic objects with which the users interact,
  2. projection system,
  3. sound generation system, and
  4. position detection/tracking system.

The poetic objects are made with foam balls implanted with microcontrollers and infrared LEDs facing upwards, atop a flat surface as the cosmic background. The infrared light from the objects is captured by a Microsoft Kinect motion sensor, and analyzed by an openFrameworks application in order to locate the objects. A particle system is projected onto the objects, and based on the position of the objects and their relationships, the behaviors of the particle systems are updated and rendered by the openFrameworks application. The image is then processed by MadMapper and sent to the projector, which has been calibrated so that the projection will be aligned with the physical objects. Control messages are sent from the openFrameworks application to Ableton Live via the Open Sound Control (OSC) protocol, generating the ambient music and sound effects.

The position detection system precisely locates the objects on the table and tracks them throughout the installation. The projection on each object behaves with distinct characteristics depending on the viewer’s interaction, which requires the system to be able to distinguish the objects from each other. Thus, the application has to be always aware of the exact location of the individual objects.

The tracking system comprises of camera vision alongside infrared LEDs. Utilizing blob detection on Kinect’s infrared stream, the application will precisely locate the objects according to their implanted infrared LEDs. A signal processing procedure enables the system to distinguish the objects within 300 ms post detection.

Kinect’s infrared stream can afford up to 30 FPS, which translates to a 30 Hz sampling rate. Hence, theoretically, we can reconstruct the any input signals with a bandwidth less than 15 Hz. The infrared LEDs on the objects are constantly blinking, which generate patterns that are perceived as square wave signals by the camera vision system. All the LEDs in the objects generate periodic square waves with a period equal to one third of a second. However, each LED generates a signal with a distinct duty cycle. The application tells the objects apart by measuring the duty cycles of the LEDs through the Kinect’s data frames. After detection, the application starts tracking the objects, and in the event of tracking loss or confusion between objects, a precise identification will be performed again through frequency analysis.

(Little Universe is a collaboration among SHI Weili, Miri PARK, and Saman TEHRANI, with the guidance of Marko TANDEFELT.)

The Humanistic Movement

Read More

腾讯视频链接 (video link for visitors in China):

The Humanistic Movement (THM) is a generative music system. It collaborates with a human dancer, and improvises music using the dancer’s body movements as its inspiration source.

THM is not a body-instrument. It hates one-to-one mapping from gesture to sound, which awkwardly limits the dancer’s movements, making her, rather than the system, responsible for the composition. THM wants the dancer to dance, with confidence that the system will take good care of the music.

Master’s Spirit in Markov Chains

And the dancer need not worry that without her direct control, the system would generate ugly sounds. In fact, THM’s musical style comes from Wolfgang Amadeus Mozart. It has calculated second-order Markov Chains1 of note progression of the first movement of Mozart’s Piano Sonata No. 11 in A major, so it really has Mozart’s style in mind. For every two-note sequence in this work, THM knows the frequencies of all possible following notes. For example, it knows that after the note sequence E4 B3, the frequencies of the following notes are:

With this knowledge, when generating a new note, THM looks back for the last two notes it has generated, and looks them up in the Markov chains. It can then follow the frequency table of the following notes, so that it plays in the style of the great master. Because of the randomness built in this process, the music is new in every performance, yet the system has a consistent style all the time, just like a real musician with her own personality.

Movement-Influenced Melody and Tempo

While THM has its own music ideas, the dancer still has influence on the composition, with the direction and acceleration of her arm captured by the Myo armband in real time. THM always bases its work on the current state of the dancer, making the music in tune with the dance.

Whenever a new note comes, the system first examines whether the dancer’s arm is pointing higher or lower than the its direction at the last note, and will accordingly look for relatively higher or lower notes in its style reference. In this way, the dancer has influence on the melody with her dance movement. Meanwhile, she has not to be overstressed, since the responsibility of deciding the exact notes are on the shoulder of THM. So the dancer can move freely, and feel that the melody flows in accordance with her movements.

The relation between arm direction and note progression is most perceivable when the music has a slow tempo. When the music goes faster, the link becomes harder to perceive. Furthermore, this is still a low level mapping which cannot represent higher level states, such as emotion, of the dancer. In order to improve its intelligence, THM introduces tempo alteration in accordance with the intensity of the dancer’s movements. In some parts of the music, the system examines the acceleration the dancer’s arm, and generates fast (e.g. eighth notes) or slow (e.g. quarter notes) notes according to the reading. The acceleration indicates the speed and complexity of the dancer’s movements, and therefore is a good representation of her emotion. By mapping it to the intensity of the music, THM receives a higher level of influence from the dancer.

In Pursuit of Rhythm

The rhythm of time-based artwork is a complex notion, and THM seeks to achieve some extent of it through the organization of musical structure. Besides the above-mentioned tempo alteration, several other strategies are employed in this work.

Repetition and variation of previously generated music occurs on a bar scale. Although the work of THM is not melodic enough for the audience to memorize a long segment of it, the reoccurrence of a bar seconds after its first appearance is readily recognizable. When the audience realize that the music is repeating it self (and that the system has memory of its previous work), they are more willing to enjoy the piece as a planned work rather than totally random hits on the keyboard.

The musical form of THM was also carefully planned. In order to build up the mood of the work effectively, the music has a predefined gradually developing structure:

The music starts slowly with whole notes whose pitch proceed in tune with the dancer’s movements. It speeds up across time, gradually switching through half, quarter, eighth and sixteenth notes. When different note-lengths coexist in one bar, there will be tempo alteration according to the dancer’s movements. When a bar has fixed-length notes which are fast enough to form a sub-melody, repetition and variation will be employed to enhance the rhythm. Chords on a lower octave are gradually introduced to further enrich the sound. After eight bars of sixteenth notes, which is the fastest part of the piece, the music slows down and finally ends with a whole note on A4, which is its key note.

System Architecture

This article mainly covers the composition logic of the THM system. The whole architecture of the system is shown in the graph above. Before the composer can make use of the dancer’s movement data, the data has to be captured by the Myo armband and pre-processed by the corresponding infrastructure, which was discussed in The Humanistic Movement: Bodily Data Gathering and Cross-application Interoperability. After the notes get composed, they are sent to the sound engine, which was implemented in Ableton Live with Max for Live, and to the visual engine, which was implement in C++ based on openFrameworks, to produce sound and corresponding visuals.

The concept behind THM was further discussed in The Humanistic Movement: Proposal.


  1. The order of Markov chains decides the extent of simulation of the master’s note choices. With the order of zero, the system chooses the notes based on the master’s overall distribution of note choice frequencies, with no knowledge of its previous composition. The higher the order is, the more previously composed notes the system will look back for. Second order Markov chains can already support significantly accurate simulation of the master’s style, and is reasonably simple to implement.

New York City Panorama Symphony

Read More

This project enables the audience to listen to New York City's skyline as a piece of polyphonic music. Panning across the panorama, the audience can not only enjoy a spectacular view of the city's skyscrapers, but also feel the rhythm and texture of these buildings by ear—a somehow exotic, but truly panoramic experience.

In preparation for music generation, a panorama photo of New York City was cleaned up and downgraded into 8 levels of grayscale. The processed image was scanned by a Processing sketch. For every vertical line of pixels, the height of the highest non-white pixel defines its base frequency; the overall level of the line's darkness defines the amplitude of the base frequency. To enrich the sound, the 6 lowest overtones of the base frequency have their respective amplitude defined by the amount of each grayscale level in the line, from the darkest to the lightest one—this is how the texture of the buildings is represented in the music. Via Open Sound Control protocol, all these calculated data are sent from the Processing sketch to a Max patch, where the music is generated accordingly.

(Photo credits: photographed by Jnn13, stitched by LiveChocolate)

The Critique of Poetic Interaction

Read More

This article seeks to corroborate Jon Kolko's argument on poetic interaction, as a desirable higher-level goal of interaction design.

Chapter 1: The Critique of Poetic Interaction

Interaction Design Ought to Be More than Usable

In his insightful book Thoughts on Interaction Design, Jon Kolko sets the tone for this discipline. He leaps forward "beyond style" (as Don Norman praises the book), grasps the substance of interaction design, and through a high-level overview of the way interaction designers work, distinguishes them from artists, graphical designers, industrial designers, interactive designers, GUI designers, information architects, UX managers and marketers. With better self-awareness, interaction designers should find their rightful place in the industry, and their voice should be properly heard.

Kolko states more than that. Good interaction designers should be more than dutiful, and good interaction design is more than making things usable. In the second half of the book, higher responsibility is put on designers' shoulders. On the social side, designers should imbue their works with judgment, and use their influential power to shape people's behavior; and beyond that, designers ought to aim at those large-scale social or cultural wicked problems, employ a whole new set of design methods with the participation of citizens themselves, in order to wake up their sense of social responsibility.

On the aesthetic side, Kolko insists that designers are far beyond stylists of transient and cheap feelings. Instead, they are responsible for the authenticity of overall experiences; they are poets of interactions. In the fifth chapter, Poetry, Spirit, and Soul, the concept of poetic interaction is proposed as designers' goal.

Ends and Means of the Poetic Pursuit

Poetic interaction is defined by its poetic effect1:

  • resonates immediately
  • continues to inform later

In order to achieve this effect, the poetic elements of interaction are claimed to be:

  • honesty
  • mindfulness
  • vivid and refined attention to sensory detail

The poetic pursuit has rich connotations. It stresses on immediate engagement, yet is at the same time foresighted. One the surface it quests for feeling, yet it is endowed with an analytical body. The formulation of poetic effect grasps the sequential manifestation of emotion and reason. A design with this poetic manner is calm, yet without loss of humanity.

However, the ends and means of poetic interaction are not in perfect coherence.

First, the three poetic elements are not subordinate to the same agent. As the means to attain the poetic effect on the user's side, it behooves these elements to be applicable on the designer's. While honesty and attention to sensory detail can be seen as qualities of a design, the second element, mindfulness, is an effect on the user–itself being an end to pursuit. I actually like this symmetry of agents, and it will be better revealed in a refined definition of poetic interaction later on.

Second, the poetic elements don't share an equal position.

In his diction, Kolko seems to treat honesty (and its synonyms) as an all-encompassing concept. To him, honesty implies integrity, and virtually consists of integrity to the business vision, integrity to the consumer, and integrity to materials. This resembles the concept of authenticity he has introduced in the book's previous chapter, Experience and Authenticity. Authenticity is said to depend entirely on designers' "craftsmanship and their intent and ability to evoke emotion," and in my opinion, is to a large extent interchangeable with honesty.

Vivid and refined attention to sensory detail, however, is dwarfed by the concepts above, and well falls into their region. Isn't this attention a kind of integrity–integrity to sensory detail? There is a need to sort out all these concepts. They are all aesthetic characteristics of a design, and can only be achieved by a wholehearted designer.

Third, probably due to the brevity of the book chapter, how the means support the ends are not fully reasoned out. Via reading one vaguely feels that aesthetic meticulousness may contribute to user's immediate resonance in usage, and a state of mindfulness may help better reflection afterwards. But these are not explicitly discussed. The chapter did discuss the approach to mindfulness. Influenced by the concept of mental state known as flow, Kolko claims that in order to encourage a mindful state, the interaction need to be "appropriately complicated." This is as insincere as the term planned obsolescence which he had attacked. And this is evil–the complexity of an activity is determined by the activity itself, and should not be artificially convoluted by its form.

In spite of the imperfections mentioned above, the book chapter is still marvelous and thought-provoking. The definition of poetic effect shows an excellent taste of interaction design, and precisely points out a worthwhile goal for designers to achieve. The discussion on poetic elements is a nice start point to form a framework of methodology. The idea of poetic interaction deserves study, corroboration, being fully understood, and better spread.

Chapter 2: Use Drama Theory to Guide Interaction Design

Why Theater is More (Or No More) Suitable Than Other Humanities

It is spontaneous, yet it is rehearsed. It is participatory, yet it is presented.
It is real, yet it is simulated.
It is understandable, yet it is obscure.
It is unique to the moment, yet it is repeatable. The actors are themselves, yet they are characters. The audience believes, yet it does not believe.
The audience is involved, yet it remains apart.

Robert Cohen, Theatre2

Tracing the source of the word "poetic", we find Aristotle's Poetics. Surprisingly or not, under this title is actually the world's earliest treatise on drama theory. Indeed, in the majority of times in history, most of the plays were written in verse, and their writers were called poets. Theater originated from festive ceremonies and performances of the bards. The theoretical framework of this discipline has been beautifully set up by one of the greatest philosophers of humankind more than two thousand years ago, and has been practiced and enriched generation after generation. Confident enough about using drama theory to guide our reflection on poetic interaction?

Wait, why not other humanities?

One reason is drama's resemblance to human-computer interaction. As stressed by Kolko, interaction designers differ from other designers in that they work with the fourth dimension (time) in mind. Humanities like painting and sculpture lack this property. Among other disciplines, literature, music and movie are basically non-interactive, leaving theater as the only one that has direct interaction between performers and audience. Though theater audience cannot direct a play like a user manipulates his computer, it is indeed one salient feature of theater that during the play, the performers and the audience both have influence on each other. Given these similarities, we may be hopeful that interaction designers can learn something from dramatists.

Yet the real surprise comes from Poetics itself–it talks about poetic effect (Chapter 26)! The poetic effect, as Aristotle writes it, consists of "not any or every pleasure, but the very special kind." This proper pleasure is the goal of tragedy, and its named catharsis.

Catharsis is achieved by arousing and purgation of certain emotions:

  • pity, occasioned by undeserved misfortune
  • fear, occasioned by undeserved misfortune of one like ourselves

Can we say that there exists perfect match between Aristotle's catharsis and Kolko's poetic effect? Pity resonates immediately, and fear is aroused by reflection. Theater and interaction design, we can say, seek the same ends! The pursuit of poetic interaction is now well-grounded and worthwhile. Theater, especially that of ancient Greek, as claimed by Nietzsche, is the greatest production by the combination of Dionysian emotion and Apollonian reason. So why not learn the methodology from Aristotle and other masters after him?

Very promising. Just do it with caution. Theater and interaction design are different disciplines, anyway. Like any other literary and artistic works, a drama is always a finished, self-contained one with content in itself. In other words, drama is representation of its content. But interaction design is largely about form. The content of an interaction is essentially produced by the user. An interaction design is never finished before the interaction. In this sense, as Kolko states, interaction designers have much less control of their works. Interaction designers and dramatist work in different ways, and have different emphases.

So much roundabout. Let's get on with the meat of Poetics.

Tragic Elements: A Comparison with Poetic Interaction

In the early 1990s, Brenda Laurel wrote a seminal book, Computers as Theatre, to apply her dramatic expertise to the field of human-computer interaction. Her stress is on the representative essence of theater, and the book has a focus on the field of interactive fantasy, the automated production of plots. Nonetheless, when she introduces dramatic theories, the juxtaposition between theater and human-computer interaction is illuminating. I suggest a reading of this book for interaction designers with even a minor interest on theater. It will be informative on both topics.

The Aristotelian drama theory is built upon a structure of the six elements of drama. There is a precedence among these elements. Laurel believes that the same elements are applicable to the analysis of human-computer interaction, which I consider is a little far- fetched. But her explanation of their hierarchical structure is instructive for us to examine the poetic elements of interaction.

The tragic elements are listed below, in a descending order of significance:

  • plot
  • character
  • thought
  • diction
  • melody
  • spectacle

Laurel explains that, each element is the formal cause of all those below it, and the material cause of all those above it. Or we can says that these elements are ends and means of each other. The lower elements serve to enhance the upper ones, function as materials in the building of the latter. And ultimately, all these tragic elements serve one goal: to achieve catharsis in the audience.

Of the six tragic elements, the lower four are less significant. They are more of technical means than substance. Plot and character form the core of tragedy. Plot is the whole action represented in the performance. Each character is a set of traits, forming an agent of the action. There exist different voices on priorities of the two. Many dramaturges maintain that character is what a play seeks to represent and depict. They insist that character is the carrier of emotions, and character depth is what gives a play its vitality. This argument is chiefly against the so-called well-made play, in which the mechanical perfectionism of plot stifles the naturalness and freshness of a play. Aristotle insists, on the other side, that performers do not act in order to portray the characters, instead, they impersonate characters for the sake of plot.

The debate can go on forever. From it we can have a feel for the complementary relationship between tragic plot and character. The plot is human plot; and the characters are created in order to present the story. And it is natural for us to make an analogy between these two tragic elements with our poetic elements. Mindfulness is reached when the user is totally immersed in the current action. And if we merge attention to sensory detail into honesty, it will be clear that the combined concept represents the characteristics of a design. Since the subject of mindfulness is the user rather than the designer, it is fair to say that mindfulness is one step closer than honesty to the poetic effect, whose subject is also the user. That is to say, mindfulness is the formal cause of honesty, and honesty is the material cause of mindfulness. Like plot and character, only when mindfulness and honesty are perfectly achieved on each side of a design, poetic effect, the catharsis of interaction, can be accomplished.

We are ready to give a refined definition of poetic interaction with the support of drama theory.

A Refined Definition of Poetic Interaction

A poetic interaction is one that has poetic effect:

  • resonates immediately
  • continues to inform later

In order to achieve this effect, the poetic elements of interaction are:

  • mindfulness, of the user
  • honesty, of the design(er)

The poetic element honesty, then, can be analyzed in detail. It includes, but should not be limited to:

  • integrity to the business vision
  • integrity to the consumer
  • integrity to materials
  • integrity to sensory detail

Here we have finished the definition of poetic interaction. It has only been slightly modified. But we are now more confident of it. We know that the poetic effect of interaction design is well-reasoned, worthwhile, and shares the same spiritual pursuit of other forms of human culture. The poetic elements are well-structured, and although the description is concise, we have a sea of theories and methodologies to explore from this start point.

Don't stop here. Don't let this discussion on definition misguide you, to consider that we have already know enough about poetic interaction. What interaction design can learn from theater is way beyond definition of terms. Just as Laurel asked, who better understands human interaction than the dramatists? They are the experts on how to keep people's attention engaged, how to reveal information on the perfect spot, what is the best scope of one interaction in order to form a holistic experience, how to exploit multisensory representation to make an authentic feel, and many, many more. The poetic effect is a goal of both interaction designers and dramatists. We feel lucky that we have them as companies.


  1. The terms poetic effect and poetic elements are actually named by myself, but their descriptions are given by Jon Kolko.

  2. Note that in this article, the words theater (theatre), drama, tragdy, etc. are used interchangeably.


  • Aristotle. Poetics. Athens. c.335 BCE.
  • Csikszentmihalyi, Mihaly. Flow: The Psychology of Optimal Experience. New York: Harper & Row, 1990.
  • Cohen, Robert. Theatre. 6th ed. Boston: McGraw-Hill, 2003.
  • Kolko, Jon. Thoughts on Interaction Design. Burlington, Massachusetts: Morgan Kaufmann, 2010.
  • Laurel, Brenda. Computers as Theatre. Reading, Massachusetts: Addison-Wesley, 1991.

(Title image credit: Jerzy Strzelecki, The Greek theatre in Syracuse, Sicily, Italy)