Much of how we construct meaning in the real world is qualitative rather than quantitative. We think and act in response to, and in dialogue with, qualities of phenomena, and relationships between them. Yet, quantification has become a default mode for information display, and for interfaces supporting decision-making and behaviour change.
There are more opportunities within design and human-computer interaction for qualitative displays and interfaces, for information presentation, and an aid to help people explore their own thinking and relationships with ideas. Here we attempt one dimension of a tentative classification to support projects exploring opportunities for qualitative displays within design.
This blog post is a slightly edited version of a late-breaking work submission presented at CHI’17, May 06—11, 2017, Denver, CO, USA, and published in the CHI Extended Abstracts at http://dx.doi.org/10.1145/3027063.3053165
Outside of the digital, we largely live and think and act and feel in response to, and in dialogue with, the perceived qualities of people, things and phenomena, and the relationships between them, rather than their number.
Much of our experience of—and meaning-making in—the real world is qualitative rather than quantitative. How friendly was she? How tired do I feel right now? Who’s the tallest in the group? How windy is it out there? Which route shall we take to work? How was your meal? Which apple looks tastier? Which piece of music best suits the mood? Do I need to use the bathroom? Particularly rarely do we deal with quantities in relation to abstract concepts—two coffees, half a biscuit, three children, but rarely 0.5 loves or 6.8 sadnesses.
And yet, quantification has become the default mode of interaction with technology, of display of information, and of interfaces which aim to support decision-making and behaviour change in everyday life . We need not elaborate here the phenomena of the quantified self [36, 42] and personal informatics more widely [24, 12], except to note the prevalence of numerical approaches (Figure 1) and the relative unusualness of non-numerical, pattern-based forms (Figure 2).
Figure 1: A typical form of quantitative interface: a Fitbit’s display of number of stepsÂ taken.
But what might we be missing through this focus on quantification? It seems as though there might be opportunities for human-computer interaction (HCI) to explore forms of qualitative display and interface, as an approach to information presentation and interaction, as an aid to help people explore their own and each other’s thinking, and specifically to help people understand their relationships and agency with systems.
In this article, we discuss qualitative displays and interfaces, and attempt one dimension of a tentative classification supporting design projects exploring this space.
What could qualitative displays and interfaces be?
Here we define a qualitative display as being a way in which information is presented primarily through representing qualities of phenomena; a qualitative interface enables people to interact with a system through responding to or creating these qualities. ‘Displays’ are not necessarily solely visual—obvious to say, perhaps, but not always made explicit.
Before exploring some examples, we will look at some theoretical issues. The terms ‘qualitative interface’ or ‘qualitative display’ are not commonly used outside of some introductory human factors textbooks, but forms of interface along these lines are found in lots of projects at CHI, TEI, DIS, Ubicomp (all academic human-computer interaction conferences) and other venues, without authors explicitly drawing our attention to the concept—it is perhaps just too obvious and too broad to merit specific comment in HCI and interaction design research. But, assuming the idea does have value, what are some characteristics?
A human face is a qualitative interface, perhaps the earliest we encounter [e.g. 40] along with the voice. We learn to read and interpret emotions in others’ expressions, to recognize commonalities and differences across people, to make inferences about internal and external factors affecting the person, and monitor the effects we or others are having on that person. We understand that the face and voice and our ability to read them are abstractions, interpretations, not perfect knowledge, but a model which enables us to make decisions in conjunction with our reading of our own emotions.
In a sense, the whole world, as we perceive it, is a very complex qualitative interface. The most accurate model of a phenomenon is the phenomenon itself, but it is only useful to us to the extent we can understand what we are observing, detect the patterns we need to, and recognize that we are constructing the ‘reality’ we perceive. We are always creating a model  and that model is necessarily not reality itself; all displays of information are representations of a simplified model of phenomena in the world. Levels of indexicality , drawing on Charles Peirce’s semiology, are relevant here, addressing the “causal distance” between the phenomenon and how it is displayed.
One advantage of interfaces seeking to provide a qualitative display is that they have the potential to enable the preservation of at least some of the complexity of real phenomena—representing complexity without attenuating variety —even if we do not pay attention to it until we actually need to, in much the same way as certain phenomena in the real world become salient only when we need to deal with them. Looking out of the window or opening the door to see and feel and hear what the weather is like outside presents us with complex phenomena, but we are able to interpret what actions we need to take, in a more experientially salient way than looking at some numbers on a weather app.
Figure 4: It’s easy to imagine the feel of the wind on ourselves when we watch this scarf tied around a lamp post flapping in the breeze. Figure 5: A windsock gives us more sense of the wind’s qualities than a numerical display.
The feel of the wind on our skin, or watching the wind affect the environment, gives us a better sense of whether we need a scarf or coat than knowing the quantitative value of the wind speed and direction (Figures 3, 4 and 5). We can see, hear and feel not just wind speed and direction, but other qualities of it—is it continuous? in short gusts? damp, dry?
Qualitative displays could enable us to learn to recognize patterns in the world (and in data sets), and the characteristics of state changes, similarly to benefits identified in sonification research . We should consider that ‘qualitative’ does not simply imply the absence of numbers. The examples we use in this paper might involve elements that could easily be quantified (rain drops, ink in a pen) but are given meaning through their display in a way that emphasises a quality or characteristic of the phenomenon. We recognise that this is potentially an ambiguous area, and are open to evolving the concept.
A possible spectrum of one dimension of qualitative displays: directness of connection
Here’s a tentative spectrum of one dimension of qualitative displays, relating phenomena to the display in terms of how directly they are connected.
(Levels 0—1 involve direct use of a real-world phenomenon in the display; from about Level 2 up to Level 5, they involve increasing degrees of translation or transduction of the phenomena. This parallels ideas in indexical visualisation  and embedded data representation  in terms of ‘situatedness’ or causal distance to phenomena.)
Level 0: The phenomenon itself ‘creates’ the display directly
Level 1: The display is an ‘accidental’ side-effect of the phenomenon
Level 2: The side-effect is ‘incorporated’ into a display that gives it meaning
Level 3: The display is a designed side-effect of the phenomenon
Level 4: Some minor processing of the phenomenon creates the display
Level 5: Major processing of the phenomenon creates the display
Figure 6: Some examples of displays from Levels 0, 1 and 2. Level 0: The pattern of raindrops hitting a translucent umbrella—frequency, coverage, and sound—directly creates a ‘rain display’ for the user, providing insight into the current state and enabling decisions about whether the umbrella is still needed; City lights create a display showing the shape of the city’s districts and indicator of population density; Water trapped in a train carriage window moves as the train ac-/de-celerates, creating a dynamic display of the train’s motion; A transparent pen is a physical progress bar for the amount of ink remaining—it could be quantified, but it is perhaps the quality of being not-yet-run-out which matters to the user. Level 1: A worn patch on a map accidentally provides a display of ‘you are here’; Use marks  from previous users demonstrate how to use a swipe-card for entry to a building; A spoon worn through decades of use is an accidental display of the way in which it has been used ; Footprints in the snow ‘accidentally’ provide a display of previous walkers’ paths. Level 2: ‘This Color For Best Taste’ label gives ‘meaning’ to the colour of a mango’s skin for the consumer (Photo used with permission of Reddit user /u/cwm2355); Writing ‘Clean Me’ or other messages in dust on a car gives meaning to the dusty property; Admiral Robert Fitzroy’s Storm Glass, as used on the voyage of the Beagle (1831—6), incorporates crystals whose changing appearance was believed to enable weather forecasting (Photo: ReneBNRW, Wikimedia Commons, public domain dedication); George Merryweather’s Tempest Prognosticator (1851) incorporates “a jury of philosophical councillors”, 12 leeches whose movement on detecting an approaching storm causes a bell to ring (Photo: Badobadop, Wikimedia Commons, CC-BY-SA).Figure 7: Some examples of displays from Levels 3, 4 and 5. Level 3: IceAlert is designed so that freezing temperatures cause the blue reflectors to rotate to become visible; A ‘participatory bar chart’ by Dan Lockton along the lines of [22, 33, 16], designed so that ‘voting’ increases the visible height of the bar, though the votes are not numbered; A non-numerical weighing scale by Chang Hee Lee designed so liquid trapped under glass changes shape; Toilet stall door lock designed so display rotates from ‘Vacant’ to ‘Engaged’—the position of the lock itself gives us a display of actionable information. Level 4:Chronocyclegraphs (1917) by Frank and Lillian Gilbreth, tracing manual workers’ movements  (Photo from , Archive.org, out of copyright]; Live Wire (Dangling String) by Natalie Jeremijenko (1995) moved a wire in proportion to local network traffic; Melbourne Mussel Choir, also by Natalie Jeremijenko with Carbon Arts  uses mussels with Hall effect sensors to translate the opening and closing of their shells into music; Availabot (2006), by Schulze & Webb, later BERG , is a USB puppet which “stands to attention when your chat buddy comes online”. Level 5:Powerchord by Dan Lockton  provides real-time sonification of electricity use, translating it into birdsong or other ambient sound; Immaterials: Ghost in the Field by Timo Arnall  visualizes “the three-dimensional physical space in which an RFID tag and a reader can interact with each other”; Ritual Machine 2 by the Family Rituals 2.0 project  uses patterns on a flip-dot display to visualize the countdown to a shared event for two people; Tempescope by Ken Kawamoto  visualizes weather conditions elsewhere in the world through re-creating them in a tabletop display (Photo used from Tempescope PressÂ Kit).
The boundaries between levels here are dependent on observers’ interpretations of what is signified (whether an effect is accidental or deliberate is a common question in design (teleonomy )). Nevertheless, this spectrum permits a classification of some examples and is being applied by the authors in undergraduate design studio projects. We note the absence of screen-based examples: this is not intentional, and we welcome adding relevant examples. There are many intersecting research areas we aim to explore; in current HCI research, the most relevant are data physicalisation, embedded data representation, tangible interaction, sonification, and glanceable displays.
The work of Yvonne Jansen, Pierre Dragicevic and others  in data physicalisation, including compilation of examples, and embedded data representation , provides us with many instances of qualitative display, mostly at what we are calling Levels 2—5; likewise, development of ubiquitous computing, tangible interaction and tangible user interfaces [39, 18, 17] and Hiroshi Ishii’s subsequent vision of tangible bits  offers a huge set of projects, many of which provide qualitative interfaces for data or system interaction (usually at Levels 4—5).
Sonification  and glanceable displays [e.g. 9, 34] also offer us diverse sets of examples often using non-numerical representation, also largely at levels 4—5. As noted earlier, qualitative does not just mean non-quantitative, and the boundaries may be blurred: if a sonification directly maps numerical values to tones, is it much different to an unlabelled line chart? Or are sparklines , for example, a way of turning quantitative data into a form of qualitative presentation?
Even with a quantitative display, how a person interprets it may have a qualitative dimension: Figure 8 shows an electricity monitor used by a study participant  who accidentally set it to display kg CO2/day equivalent; this “meant nothing” to her but she interpreted the display such that “>1” meant “expensive”. ‘Annotations’ of values as users construct their own meaning  may fit here; the aim must, however, be to avoid the kind of reductive ‘qualitative’ nature of a limited set of labels .
Figure 8: A quantitative electricity display that was used ‘qualitatively’ by a householder (see text). Figure 9: An example of MONIAC, the Phillips Machine, at the Reserve Bank of New Zealand (Photo by Kaihsu Tai, Wikimedia Commons, public domain dedication).
Analogy and metaphor are important here, and the almost-forgotten field of Analogue Computing offers us an intriguing perspective. By “build[ing] models that created a mapping between two physical phenomena” , some analogue computers effectively operated as ‘direct’ displays of an analogue of the ‘original’ phenomenon—a kind of meta-level 2 type qualitative display, with devices such as the 1949 Phillips Machine  (Figure 9), which performed operations on flows of coloured water to model the economy of a country, enabling an interactive visualization of a system in operation as it operates (there are parallels with Bret Victor and Nicky Case’s work on explorable explanations [38, 8], and the development of visual programming languages).
Other areas of pertinent research and inspiration, are synaesthesia and mental imagery: sensory overlaps, fusions and mappings offer a fertile field for exploring qualitative displays of phenomena.
Conclusion: What use is all ofÂ this?
We’re interested in using qualitative displays and interfaces for supporting decision-making, behaviour change and new practices through enabling new forms of understanding—as an aid to help people explore their own and each other’s thinking, and specifically to help people understand their relationships and agency with the systems around them . Projects using qualitative displays are unlikely simply to be de-quantified ‘conversion’ of existing numerical displays; instead, the aim will be to make use of the approach to represent and translate phenomena appropriately, in ways which enable users to construct meaning and afford new ways of understanding, enabling nuance and avoiding reductiveness.
The spectrum of the ‘directness’ dimension introduced here provides a possible starting point for this work, by giving a framework for analysing examples and suggesting ways of handling phenomena to be displayed, and is currently being used by the authors to brief an undergraduate design studio project on materialising environmental phenomena to reveal hidden relationships. We welcome the opportunity to learn from others who have thought about these kinds of ideas to inform our future explorations of this area.
Thanks to Dr Delfina Fantini van Ditmar, Dr Laura Ferrarello, Flora Bowden, Gyorgyi Galik, Stacie Rohrbach, Ross Atkin, Shruti Grover, Veronica Ranner and Dixon Lo for discussions in which some of these ideas were formulated and explored, and to the CHI reviewers. Unless otherwise noted, photos are by the authors.
9. Sunny Consolvo, Predrag Klasnja, David W. McDonald, Daniel Avrahami, Jon Froehlich, Louis LeGrand, Ryan Libby, Keith Mosher, and James A. Landay. 2008. Flowers or a Robot Army? Encouraging Awareness & Activity with Personal, Mobile Displays. In Proceedings of 10th International Conference on Ubiquitous Computing (UbiComp’08): 54—63. https://doi.org/10.1145/1409635.1409644
12. Chris Elsden, David Kirk, Mark Selby, and Chris Speed. 2015. Beyond Personal Informatics: Designing for Experiences with Data. In Proceedings of the SIGCHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’15): 2341—2344. https://dx.doi.org/10.1145/2702613.2702632
14. Heinz von Foerster. 1973. On constructing a reality. In F.E. Preiser (Ed.). Environmental Design Research Vol. 2. Dowden, Hutchinson & Ross, Stroudberg: 35—46. Reprinted in Heinz von Foerster. 2003. Understanding Understanding—Essays on Cybernetics and Cognition. Springer-Verlag, New York: 211—228. https://dx.doi.org/10.1007/0-387-21722-3_8
17. Eva Hornecker and Jacob Buur. 2006. Getting a grip on tangible interaction: a framework on physical space and social interaction. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’06): 437—446. https://dx.doi.org/10.1145/1124772.1124838
18. Hiroshi Ishii and Brygg Ullmer. 1997. Tangible bits: towards seamless interfaces between people, bits and atoms. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’97): 234—241. https://dx.doi.org/10.1145/258549.258715
20. Yvonne Jansen, Pierre Dragicevic, Petra Isenberg, Jason Alexander, Abhijit Karnik, Johan Kildal, Sriram Subramanian, and Kasper HornbÃ¦k. 2015. Opportunities and Challenges for Data Physicalization. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’15): 3227—3236. https://dx.doi.org/10.1145/2702123.2702180
23. David Kirk, David Chatting, Paulina Yurman, and Jo-Anne Bichard. 2016. Ritual Machines I & II: Making Technology at Home. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’16): 2474—2486. http://dx.doi.org/10.1145/2858036.2858424
24. Ian Li, Anind Dey, and Jodi Forlizzi. 2010. A stage-based model of personal informatics systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’10): 557—566. https://dx.doi.org/10.1145/1753326.1753409
26. Dan Lockton. 2016. Designing Agency in the City. In Lacey Pipkin (Ed.), The Pursuit of Legible Policy: Agency and Participation in the Complex Systems of the Contemporary Megalopolis. BurÃ³-BurÃ³, Mexico City: 53—61. http://legiblepolicy.info/book/ Legible-Policies_BB.pdf
28. Dan Lockton, Flora Bowden, Catherine Greene, Clare Brass, and Rama Gheerawo. 2013. People and energy: A design-led approach to understanding everyday energy use behaviour. In Proceedings of EPIC 2013: Ethnographic Praxis in Industry Conference: 348—362. https://dx.doi.org/ Â 10.1111/j.1559—8918.2013.00029.x
29. Dan Lockton, Flora Bowden, Clare Brass, and Rama Gheerawo. 2014. Powerchord: Towards ambient appliance-level electricity use feedback through real-time sonification. In Proceedings of UCAmI 2014: 8th International Conference on Ubiquitous Computing & Ambient Intelligence: 48—51. https://dx.doi.org/10.1007/978-3-319-13102-3_10
30. George Merryweather. 1851. An essay explanatory of the Tempest Prognosticator in the building of the Great Exhibition for the Works of Industry of All Nations. John Churchill, London. Retrieved Jan 10, 2017 from https://archive.org/details/b2804163x
31. Bruno Munari. 1971. Design as Art (trans. Patrick Creagh). Pelican Books, London.
32. Dietmar Offenhuber and Orkan Telhan. 2015. Indexical Visualization—the Data-Less Information Display. In Ulrik Ekman, Jay David Bolter, Lily Diaz, Morten SÃ¸ndergaard, and Maria Engberg (eds.). Ubiquitous Computing, Complexity and Culture: 288—303. Routledge, New York.
33. Jennifer Payne, Jason Johnson, and Tony Tang. 2015. Exploring Physical Visualization. In Jason Alexander, Yvonne Jansen, Kasper HornbÃ¦k, Johan Kildal and Abhijit Karnik. Exploring the Challenges of Making Data Physical. Proceedings of the SIGCHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ’15): http://architectures.danlockton.co.uk/wp-content/2015-chi2015workshop-physvis.pdf
34. Tim Regan, David Sweeney, John Helmes, Vasillis Vlachokyriakos, SiÃ¢n Lindley, and Alex Taylor. 2015. Designing Engaging Data in Communities. In Proceedings of the SIGCHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA ‘15): 271—274. http://dx.doi.org/ Â 10.1145/2702613.2725432
35. Stefania Serafin, Karmen Franinovic, Thomas Hermann, Guillaume Lemaitre, Michal Rinott, and Davide Rocchesso. 2011. Sonic Interaction Design. In Thomas Hermann, Andy Hunt, and John Neuhoff (Eds.), The Sonification Handbook. Logos, Berlin: 87—110. http://sonification.de/handbook/ index.php/chapters/chapter5/
We often use landscapes as metaphors in everyday speech, particularly to talk about complex systems–understanding a complex information system as an “information landscape”, for example, helps convey the idea that such a system, like a landscape, is vast and encompasses many interacting variables. However, while landscape metaphors are common in speech–terms like “stakeholder landscape”, “lie of the land”, “ocean of possibilities”, “food desert”, even the word “field”–landscape metaphors have been used more rarely in visual applications.
On March 30th, 45 Juniors from Carnegie Mellon University’s School of Design’s “Persuasion” class, taught by Michael Arnold Mages, Dan Lockton, and Stephen Neely, took part in a workshop to explore practically how physical and visual landscape metaphors could help elicit new insights about complex experiences–in this case, modeling and reflecting on group design projects. Facilitated by MA Design student and Research Assistant Delanie Ricketts and Assistant Professor Dan Lockton, as part of the School of Design’s new Imaginaries Lab, the workshop involved students collaboratively creating ‘landscape’ models representing projects they have worked on, using simple paper cut-outs of features such as hills, trees, weather, and people. Each group used the elements in different ways to represent different aspects of their projects, through creating ‘timeline’ landscapes in both two and three-dimensional formats.
Some projects started with rocky beginnings, represented by different cones or hills, in order to show how difficult that part of the project was. Other projects started with trees, rivers, and stars, representing periods of calm ideation, research, or general feelings of optimism. When projects encountered new difficulties later on, many groups represented these periods with lightning, rain, hills, and cones. Several groups used (and came up with names for) metaphors within the general landscape metaphor to represent specific parts of their project experiences, such as a “plateau of exhaustion” before the project came to an end.
Delanie’s previous prototypes of the landscape metaphor visuals, as part of her research assistantship project, have focused on how they could facilitate individual reflection on one’s own career path. However, while people found the metaphor and elements to be a useful and creative reflection tool, several expressed that it was difficult to show how their perspective changed over time within a two-dimensional format. In this second iteration of elements, we aimed to provide greater variation as well as enable three-dimensional expression. In addition, we wanted to explore how the metaphor could be used to think through a different topic, project planning or reflecting rather than career, and in a group rather than individual context.
Students’ responses to trying out this second iteration of landscape elements, applied to group projects rather than individual career paths, suggested that they found the process fun and creative, while also abstract. Many participants commented that the tool helped them understand their project and teammates’ perspectives better, especially in terms of stress, productivity, and overall emotional satisfaction at different points throughout a project’s lifetime. The format is more useful for surfacing –Â and reconciling –Â overarching understandings than probing deeper insights about the specifics of complex experiences, but, in triggering discussion, it has value in enabling members of a team to understand and interrogate each other’s perspectives and mental models of a situation (echoing ideas from organizational systems thinking experts such as Peter Senge).
We aim to develop the landscapes kit further, through iterations with application in individual reflection, project planning, and research settings.
Many thanks to Chris Stygar, Josiah Stadelmeier, and the whole School of Design 3D Lab for their help in developing the materials for the project, the Design graduate students and juniors for taking part in the different stages of the project, and Manya Krishnaswamy for helping facilitate. Thanks to Joe Lyons for putting the article on the School website.
In Materializing the Invisible, we considered invisible and intangible phenomena—the systems, constructs, relationships, infrastructures, backends and other entities, physical and conceptual, which comprise or influence much of our experience of, and interaction with, environments both physical and digital. ‘The invisible’ here is potentially everything from how the building’s heating system works, to the algorithms behind targeted ads, to who’s friends with whom, to where corruption is occurring in government, to where your IoT fridge sends the data it collects, to people’s mental imagery of time, to the electricity use of devices, to networks of cameras and sensors, to how political decisions are made. It also potentially includes things that happen at scales or in dimensions we can’t directly comprehend, from planetary processes such as climate, to the interaction of electromagnetic fields, to the microscopic. And things that happen, that enable day-to-day functioning of our lives, but we don’t know much about. Where does our food come from? Where does our waste water go? What route did that package take to get to us?
The process of revealing the invisible can improve understanding, help people explore their own thinking and relationships with these complex concepts, highlight problems, power structures and inequalities, reveal hidden truths, connect people better to the world around them, and enable people to act. It is not necessarily about visualizing the invisible—it can be about making it audible, tangible, smellable, or otherwise experienceable: we explored techniques from fields including data visualization, sonification, data physicalization, ubiquitous computing, tangible interaction, analog computing, qualitative displays, and the study of synaesthesia to create ways to materialize these invisible phenomena.
As a starting exercise we examined some ‘invisible’ and unknown things within the building itself (Margaret Morrison Carnegie Hall), noting questions and ideas with Post-It notes in situ. These ranged from questions about who has access to certain rooms or controls, to what some of the controls are in the first place. There were also traces of action and use—patterns which might be invisible in the sense of not being paid attention to, but nevertheless present in the use of the building.
The class project was to choose a phenomenon which is ‘invisible’ within a physical, digital or hybrid environment, find a way of getting access to it, and design and build / make / create a way of materializing the phenomenon, making it accessible to people more widely. As a group we brainstormed different phenomena which might be investigable, and possible forms of representation.
Ji Tae Kim’s project Whitespace looked at the invisible aspects of communication in text messaging, following on from his previous project Fear of Missing Out. Whitespace explores ways to materialize and express “rich contextual and verbal cues” through “an intuitive extension to instant messaging”. Working prototypes used copper tracks, Bare Conductive ink and Touch Board, and Arduino.
Jasper Tom and Chris Perry‘s project Kairos examined “an invisible phenomenon ingrained in everyday life”: the passage of time in a space, specifically around working at a desk. The question “Where did the time go?” and the idea of desk legacy, the patterns of use left by a previous user of the desk in a shared workspace, informed by analysis of timelapse video of the studio, came together with inspirations such as Daniel Rozin’s Wooden Mirror, MIT Tangible Media Group projects such as Daniel Leithinger’s work, and Tempurpedic foam, to create a desk surface which could ‘play back’ the patterns of how it had been used, via an interface using wooden blocks. A working prototype of part of the surface used Arduino and servo motors to demonstrate the effect.
One interesting aspect discussed during Jasper and Chris’s presentation was how while evidence of physical work is often obvious in space, such as a painter’s palette, the evidence of digital work is often invisible—a slightly worn keyboard, perhaps, but little else.
Gilly Johnson and Ty Van de Zande worked together to explore aspects of human movement (dance and exercise), and the related issues of hydration and focus. Focus + Movement proposed a color-changing bodysuit which could work together as part of a system with a water bottle, both to make the invisible patterns visible, and to enable reflection. Gilly and Ty captured movement by dancers using a Kinect, connected to Max MSP, and then simulated the body suit via After Effects.
In this short introductory unit, we looked at ways in which the design of environments, and features within them, affects people’s behavior and interaction with each other. Design influences what people do, but often the ‘links’ are invisible or only apparent by their effects. Or, we notice them in passing, but do not take time to reflect on them or draw parallels across situations.
As designers pioneering new approaches to creating environments for human experience, cultivating a kind of ‘hypersensitivity’ to noticing—and learning from—the ways in which design and behavior interact can be part of developing the attention to detail which will serve you well professionally. Details of the unit in the syllabus.
We started with quick observation exercises aimed at developing (or refreshing) a capacity for noticing, for paying attention to the ways in which people and environments affect each other. We looked around campus for instances of points of confusion, unintended uses, constraints, and disobedience in physical environment settings, and discussed how these effects manifest in different ways—what could we find? (Photos here by Chris Perry, Gilly Johnson, Jasper Tom, Ty Van de Zande and Dan Lockton.)
We examined ideas around how environments influence people, and are in turn influenced, both physically and digitally, from thigmotaxis to stigmergy, shearing layers and pace layers, fundamental attribution error and design for behavior change. We also thought about the practice of observation, noticing and deconstruction of people’s actions in different ways, and in different levels of detail. The project brief was around designing a way to do research in this field—designing a ‘probe’ rather than a solution to a problem:
Choose a situation where ‘design’ seems to be affecting people’s behavior in an environment (physical or digital)
Find a way of studying what’s going on—what patterns exist? In what different ways are people’s behavior affected?
Visualize (or otherwise communicate) what you find
(optional: suggest ways things could be different, if you feel they need to be)
Keep a blog of your process (photos, sketches, notes)
Here are the projects:
Comparing a coffee shop and a tea shop: Gilly Johnson
Gilly Johnson compared structural and systemic aspects of the atmosphere and experience in Coffee Tree Roasters in Shadyside, and Dobra Tea in Squirrel Hill, including the layout and spatial division, and emerging themes such as service and trust: full details of the project.
Fear of Missing Out: Ji Tae Kim
Ji Tae Kim examined how the design of messaging and social media leads to ‘fear of missing out’ through unplugging himself for a week: full details of the project.
Greyhound Station: Jasper Tom
Jasper Tom investigated how the design of Pittsburgh’s Greyhound Bus Station influences patterns of people’s behavior: full details of the project.
Managing information across environments: Ty Van de Zande
Ty Van de Zande looked at how people manage information such as to-do lists across physical and digital environments, and developed a framework for investigating this in a structured way: more details of the project.
How to Cross the Road: Chris Perry
Chris Perry observed the different ways in which people use a pedestrian crossing at Morewood Avenue and Forbes Avenue, at the entrance to CMU, and how the design affects those actions: more details of the project.
2016 has been a complicated year, and circumstances have meant that I have, rather unprofessionally, neglected this blog, my website, and the Design with Intent newsletter (which now has nearly 400 very patient subscribers).
In summer 2016 I moved on from the RCA to Carnegie Mellon University, in Pittsburgh, Pennsylvania, to become a tenure-track Assistant Professor in the School of Design. This has meant a huge life upheaval for Harriet and me, moving from a boat on the Thames at Richmond to, eventually, what I must learn to call a duplex in Friendship, a pleasant neighbo(u)rhood in Pittsburgh built for the middle managers of Carnegie Steel. At CMU, in my first semester, I have taught Play Lab, a senior design lab discussed at length here and contributed to a range of other graduate and undergraduate programmes. In the Spring, I will be teaching the junior Environments Studio – an exploratory new studio class – and, with Michael Arnold Mages and Stephen Neely, a required class called Persuasion. I will also be running PhD seminars in Research(ing) by Design(ing), continuing some of Cameron Tonkinwise’s work. (I remain a visiting tutor at the RCA, temporarily at least, to continue to supervise the PhDs of NazlÄ± TerzioÄŸlu, Chang Hee Lee and Dave Pao; Delfina Fantini van Ditmar passed her PhD in the summer.)
Carnegie Mellon was attractive for many reasons. The School of Design has a vision for design education and research, Transition Design, which is more exciting (and reflectively critical) than simply repeating “we’re number 1”. Transition Design is very relevant to my previous research in design for behaviour change, sustainable design, energy use, designing agency, understanding the systems around us, community-led design and many other areas, and in acknowledging “that we are living in ‘transitional times’” it enables design to be seen as a tool for engaging with complexity, with “an understanding of the interconnectedness of social, economic, political and natural systems”. I wanted to work at a university that had a professional approach to employment and career development for younger faculty, and CMU offers this to an impressive extent, along with proper employment contracts and administrative support to faculty. I wanted to work somewhere that values people’s contributions, has an informed and mature approach to research, teaching and service, that is welcoming, friendly and interested, and where the obsession is with doing things well rather than simply bringing in more and more money, and while CMU brings in plenty of money, there is a genuine commitment to excellence along with it. Most of all, I wanted to work somewhere where I would have a chance to develop and follow a vision for a programme of research and teaching, and where I would be supported and trusted to do so, where a career was possible and encouraged and with a much much flatter hierarchy of management, and I think CMU can deliver this. As an Assistant Professor, I have a lot of freedom and autonomy, much more than I would have (at this level) at any of the UK institutions I have ever worked at. It gave me lots of pride to see my Play Lab students’ work on show at Focus, the senior design exhibition, earlier this month. CMU has been welcoming, exciting, enjoyable and an enthusiastic employer so far, and I want to thank everyone who has made Harriet’s and my first few months so good.
The Britain that Harriet and I left, or rather the pre-23-June Britain, feels like the past, another country now, where, indeed, they do things differently (and with much more decorum and less pride in ignorance). It has been a very strange, and sad thing to watch from abroad as the UK determinedly keeps shooting itself in the same foot so comprehensively and with so much nastiness, and I miss, basically, the idea that the future can be better than the past. I like 1950s cars and design and writing and lots of other things, but I don’t want to live there. But of course, it’s been a very weird time to live in America too, and to see what a hugely divided country it is, and yet also not to have to experience that in everyday life: the milieu of a university professor in the US is not one where I see much opinion other than utter disdain for, and horror at, what has happened. The situation in the US, the UK, and in many other countries where danger is on the rise, leads me to articulate a possible new role / challenge for the field of design for behaviour change, which I will outline in a forthcoming blog post introducing my new lab at CMU, the Imaginaries Lab.
Aside from academic papers, a couple of books have been published:
Seniors (4th-year undergraduates) in Industrial and Product Design at Carnegie Mellon take three ‘Senior Design Labs’, Wonder Lab, Speak Lab, and Play Lab, each of which aims to help students develop some ‘design agility’. They set out to enable students to integrate and revisit skills they’ve developed through their time at CMU, but applying them in new and different situations. The idea is that this helps graduating students develop a shift in perspective on their own abilities and identities as designers, and gives them confidence to tackle new kinds of problems and challenges in a reflective way, through knowing themselves better.
Play Lab2016 was specifically about exploring ambiguity. Over five weeks, the 33 students worked on the idea of future(s), and designers’ role in both creating (in a sense), and responding to, ideas of possible futures that by definition, don’t exist yet. It’s about developing and being able to show a thought process which is not just about problem-solving, but comfortable with ambiguity, problem-finding, and problem-worrying.
Our starting points for exploration were three quotes: most famously, as phrased by the novelist William Gibson, that “the future is already here—it’s just not very evenly distributed” (there are different versions of this quote going back to the early 1990s); Jenny Holzer’s “You live the surprise results of old plans” and “A good science fiction story should be able to predict not the automobile but the traffic jam”, attributed to Frederik Pohl. The basic idea is powerful from a design point of view: what ‘futures’ do we see emerging right now, what might they grow into, what consequences could they have, and how would designers be involved? What power, or agency, do designers even have here? Through a series of small exercises, students build toward designing and creating ‘pockets of the future’ that can be glimpsed or experienced byÂ others.
By the end of theÂ course:
Comfortable discomfort: you should be comfortable—well, as comfortable as you can be—with applying the design skills you have developed over your time at CMU, to inherently ambiguous questions about theÂ future.
2. Problem-worrying: you should have carried out a project through which you don’t ‘solve’ a problem, but rather, ‘worry’ at the questions involved, creatively exploring possibilities and opportunities through synthesizing your design and research skills, thinking through ‘making’.
3. Reflective designing: you should be confident in using the project to demonstrate your ability to engage creatively to explore ambiguous futures, with a critical eye on your own role as a designer in shaping and responding toÂ trends.
The course overall comprised five exercises, starting with small challenges around observation and speculation, and building to a bigger project in the final two weeks of the five-week session. The classes were a mixture of group discussion, presentations, demonstrations and one-to-one advising in the studio. Students kept a blog as part of the course, which was part of the assessment; these could be made public or kept private depending on students’ preferences. As students come from different design specialisms, it was expected that they will use whatever physical or digital media suit their skills and confidence. The fuzzy edges of ‘the future’ permit a deliberate fuzziness in the resolution of the work: ambiguity was encouraged.
It’s worth noting that Play Lab took place in three groups over the semester, and the details of what we focused on with each group differed slightly as questions and ideas and issues arose. With the final group, the presidential election results occurred on the morning of one of the classes, and provided an emotional and quite difficult atmosphere to our discussions of the future. Questions around filter bubbles, fake news (a form of design fiction, surely), blurred boundaries between truth and falsehood, and changing geopolitics ended up influencing a number of the projects. Oddly, many of the themes from J.G. Ballard’s ‘The Future of the Future’, written for Vogue in 1977, also seemed to recur, unprompted, throughout theÂ class.
While Play Lab 2016 was not explicitly focused on speculative and critical design (Deepa Butoliya’s class gave her students a proper grounding in that), much of what students produced was nevertheless approaching this area. It was, at least, using design to provoke discussion, particularly in relation to the future mundane, as Nick Foster has called it. If the ideas and scenarios the students created ended up telling us more about today’s preoccupations and concerns, from the perspective of North American undergraduate design students, than being politically motivated calls to action, that is little different to the majority of work on design fictions, and no worse for it. As we will see with the projects themselves, the broad themes give us an insight into what worries, inspires, and preoccupies a group of talented designers about to go out into the uncertain world ofÂ 2017.
“the future is already here—it’s just not very evenly distributed”
and thinking about how this idea is manifested in practice. What do we see around us—things, actions, habits—that are potentially ‘from’ a future? What do we perceive to be anachronisms, in ‘both’ directions (past-in-the-present and ‘future’-in-the-present)? What is the ‘texture’ of the present? What things come and go, and what stay? Can we tell which ways of doing things are going to become ‘normal’ or popular, the seeds of possible futures, and which won’t? How does this differ across cultures, across countries, but also within one place? As designers, are our observations and thinking here different compared with what people with different agendas and skills might pay attention to?
We considered how we ‘read’ modern behaviors into images from the past, and how that relates to what we assume, as designers, about the way people ‘will’ use things that we design. Looking around the room for examples, we saw things such as the projector propped up on a diary, a trash can being used to hold the door open and the first aid kit box being used as a noticeboard, all of which are not necessarily what the designers intended, but nevertheless are almost second nature to us. (See Jane Fulton Suri’s Thoughtless Acts, Uta Brandes and Michael Erlhoff’s Non-Intentional Design and Richard Wentworth’s beautiful Making Do and Getting By for much more of this kind ofÂ thing).
“Cars lumbered past like ponderous elephants of rusty steel, not so different from the cars of 30 years ago, and seemed not to belong in the same world as the tattooed kid punching code into his laptopÂ nearby.
Under the spell of this book, I suddenly understood my surroundings not as a discrete contemporary tableau but as a hodgepodge of 1910, 1980, 2011 andÂ 2020.”
We also considered how this kind of ‘hodgepodge’ could mean that, in a sense, we are all living in different worlds, a flaky, uneven present– what is ‘new behavior’ or ‘a new way of doing things’ for one person may be something quite mundane for someone else. (This theme came up later in a number of the projects showing the point of transition from one way of doing things to another—when some people have adopted a new way, and others haven’t.)
We also talked through Voros’s futures cone, as explored by people such as Stuart Candy (see pages 42/43 of the PDF) and Jessica Bland and Stian Westlake (see image)—and featured in Chapter 1 of Dunne & Raby’s Speculative Everything, one of the readings (see below)—and what it might mean for designers plotting a career trajectory. Futures that might once have been merely ‘possible’, such as driverless cars, are rapidly moving right to the center of the ‘probable’ cone, while other futures recede in probability. What happens if you specialize in something that goes nowhere? And yet, as a designer, you are potentially partly responsible for the way in which the future develops. (Sjef van Gaalen has some thoughts about how the cone interacts with Rumsfeld’s unknown unknowns.) We also looked briefly at the idea of ‘backcasting’ from the right-hand side of the cone to the left (the present), discussed some of the deficiencies of treating the present as a ‘singular’ point, and asked what a cone extending backwards into multiple ‘pasts’ might look like. (Some of my own thoughts about how this potentially relates to Transition Design areÂ here.)
Introducing the first exercise, we thought about how observation, and noticing things around you—objects, behaviors, trends, outliers, social ‘rules’ and norms, and ways of doing things (practices)—can be a valuable habit to get into as a designer, and how using this form of observation to try to see some of the ‘unevenness’ of the present could be a way of uncovering possible ‘micro-futures’ that might be the start of something. This is the idea of ‘weak signals’ as explained by Near Future Laboratory.
Was this man I saw at Helsinki airport in 2008, using a (for the time) startling, apparently enormous 20" HP laptop, with an external hinge, an early adopter of a trend that would soon be ubiquitous? Were we soon going to be seeing people using giant laptops in coffee shops everywhere? Was this a ‘pocket of the future’ in the present? Or was this an aberration? What about devices such as PDAs and personal organizers, which in some way seemed to presage today’s total cellphone saturation, and yet never reached those levels of popularity?
Jan Chipchase’s TED talk from 2007—nine years ago—on The Anthropology of Mobile Phones, describing his research fieldwork for Nokia, provides a fascinating ‘historical’ insight into how possible future(s) for cellphone use and communication practices were explored and envisaged through in-context research in a huge number of countries and cultures, and makes an interesting comparison with what actually happened in the subsequent decade. (Jan’s Future Perfect blog is an incredible chronicle / travelogue, and diving into the archives at any point is well worth it.) Exercises like asking people to empty their bags, to get insights into the minutiae of everyday life, can also be used to explore possible futures—what do you carry around that you expect to be doing so for the foreseeable future? What is temporary? What willÂ change?
We finished the class by looking at a few examples, mostly collected online, of what might seem like ‘odd’ behaviors or outright misunderstandingsÂ , but which nevertheless maybe offer something interesting, the tantalizing seed of a different kind of system, where things work differently. Some involve adaptation, some ‘fighting back’ against a system (like make-up and fashion to prevent facial recognition), some involve people believing that something works in a particular way that it doesn’t really, but maybe itÂ could.
Look around you—on campus, online, anywhere—at the ways people are using things (not just technology, but clothes, fashions, mannerisms even). Notice unusual or intriguing sets of behaviors, and consider whether they could be possible ‘micro-futures’—small trends, kernels of the future in today, or maybe even today in theÂ future.
They might be ways of doing things that seem culturally different, or age-specific. They might be things that just seem strange, or wrong. Or they might take a bit of noticing. Look at other people, but also perhaps yourself—what micro-futures might you be engaged in creating? Are there new habits you’ve developed, or you see yourself developing?
Document them somehow, using photos, video, screenshots, or even just your own notes (it’s not always easy to photograph people doing things!). Ideally, find at least 3 possible micro-futures.
Readings for Exercise 1 (on Box, needs CMUÂ login)
Jillian Nelson: Railings as bike parking, Pocketbra, Phone as mirror, Holding papers inside of laptop, Backs of chairs as coat or bag storage, Roads / sidewalks as a billboard, Using a trashcan as a ladder to climb into window, I used a hammer to open up a beer bottle once, Kenny and the cerealÂ bar
Leah Anton: People who brush their teeth in the shower, People who share their food over the internet, People who ask “How are you?” and meanÂ it
Vicky Hwang: (1, 2, 3, 4, 5) Bidets and toilets, Hacking cables to strengthen them, Ways to open beer bottles, Skipping songs, Digital and seniority
Jackie Kang: How-to videos, Social media/reality stars,Â Google
Brian Yang: Public transport in Asia, Toothpaste maximizer, Velcro spice rack, FoodÂ caddy
Catherine Zheng: Prescriptive lifestyles, Distracted walking and waiting, Cosmetic augmentation
Lea Cody: Status update, Health tracking, The AmericanÂ Death
Linna Griffin: Augmenting nature through tech, Citizen journalism, Temporary beauty technology
Gabriel Mitchell: What will augmented reality actually be like?, What will bots actually be like?, How much information is tooÂ much?
Temple Rea: VR Experiences, Transcranial direct stimulation, Facial Action Coding System, Inherent vs explicit form, Tools for creators, The relationship between technology and ourÂ feelings
Zai Aliyu: Emotions and intelligence in technology, Civil inattention, Ephemerality, Impression management
Kate Apostolou: The political reality show, Emotional expression through technology, Technology’s effect on spatialÂ thinking
Kaleb Crawford: Live-streaming, live-chatting, and and the meta-memetic zeitgeist, A/B Testing, Rhetoric Analysis, Context Collapse, Self-Identity through Non-Identity: Social media presence for a post-ironic generation
Ruby He: Banter with robots, Packaged empathy, MomentsÂ relived
Alisa Le: Gender identity and gender expression, Sharing economy, Automating decisions and simplifying dailyÂ routine
Julia Wong: Flexible and mobile environments, Grunge, “nostalgia” and celebration of the “retro” / less perfect and “genuine”, ExtendedÂ self
Jeff Houng: Tattoos and Body Self Expression, Biometric Data or Authentication, The Pursuit of Hyper-Efficient Wellness
We looked at these not so much as theories that need to be understood, but as examples of formal corporate / business attempts to ‘deal with’ ambiguity in futures and understand how ideas and practices spread.
‘A good science fiction story should be able to predict not [just] the automobile but the traffic jam’: Attributed to FrederikÂ Pohl
The distinction between side-effects and side-shows is somewhat blurred, but we might consider side-effects (in relation to cars) to include, as in the Frederik Pohl quote, immediate effects such as traffic jams and accidentsÂ , but also, potentially, longer-term effects such as obesity, changes in city planning, and, potentially—via lead—violent crime. And of course wars driven, in some part, by the quest forÂ oil.
It’s also important to remember what might be seen as positive side-effects: widening access to travel has enabled huge changes in mobility, working possibilities (including jobs for designers!), economic development, meeting new people, cross-fertilization of cultures, and so on. And, as Robert K. Merton pointed out, even if certain effects were not specifically desired by people who planned the system, they may well be desirable for someÂ people:
Side-shows are the parallel-but-related developments. In relation to the car, that might mean the development of gas stations, roadside diners, car culture and everything that goes with it—which all involve design to a greater or lesser extent. They might also include frictions and counter-movements: people opting out, or being forced out, of the new norm, or finding ways to subvert it (as in Superflux’s ‘Uninvited Guests’).
The line is blurred between side-shows and side effects, hence considering them together. There are many levels of abstraction at which systems can change, and the boundaries of how we frame our analysis are necessarily somewhat arbitrary. A version of Jerome Glenn’s Futures Wheel featured in Extrapolation Factory’s Operator’s Manual was a useful tool for thinking through side-effects and side-shows, or secondary and tertiary consequences—and some students managed to apply this very effectively to the micro-futures they had identified in ExerciseÂ 1.
Pick one of the possible micro-futures you explored in Exercise 1 (or swap with someone else’s) and explore/study it further as a concept or phenomenon. (If you don’t like it, find another.)
How does it fit with the ‘ecosystem’ of people’s lives now? How could it fit in the future? How would it change dailyÂ life?
Using what you know as a designer, about both design trends and human behavior, can you extrapolate a few years into the future to create a short scenario for how things might be? (Remember, there are, of course, no ‘wrong’ predictions here, since we don’t know what we don’t knowÂ yet.)
Think through the side-effects and side-shows that might exist in this world, and the role of the designer:
Side-effects: What consequences (potentially unanticipated—except by you!) might there be, from the new thing or way of acting? What effects could it have on society more widely? Will some people ‘opt out’ of it? Will some people rebel against it? Are there technological or environmental ‘limits’ we mightÂ reach?
Side-shows: What other things are going on in this world? What parallel developments, related or unrelated, go along with the phenomenon you’re examining? What are the big issues in this world, the areas of publicÂ debate?
Role of the designer: What might the briefs be for designers, in this world? What skills are in demand? How does society treat designers? How do designers treat society? Are designers mainly dealing with side-effects?
Explore these ideas visually and in text, and document them on your blog. Visually, you could sketch, take photos, collect images from elsewhere and annotate them or modify them—anything you need to do to tell your story and explain your thinking to others. In text, there’s no need for a long essay, but aim for a few hundred words. Be prepared to present what you’veÂ done.
Some questions to helpÂ you
These are a few (optional!) questions and things to look up which might be relevant, depending on how ‘big’ the idea is that you’re workingÂ on.
Questions about ethics —Look up Kant’s ‘categorical imperative’. What would happen if this new behavior, object, etc, became universal, i.e. if everyone acted or were influenced in this way? Would you envisage that that this ‘should become a universal law’? If not, why not? – Look up Rawls’s ‘Veil of Ignorance’. What are you, as a designer, assuming your status would be in the world you envision? Would you support the idea if you didn’t know what place you would have in the scenario, what background you would have, and whether you were on the ‘receiving end’?
Questions about sustainability If you are thinking about ‘sustainability’, what definition are you using, tacitly or explicitly? Where is the boundary drawn between humans and nature? Is your treatment of sustainability primarily ecological, or does it include a social component?
Questions about power Would this world involve one group of people having power or advantage over others? Would it create new power structures in society, or reinforce existing power structures? Or could it break down existing structures, and give different people agency to change the system?
These questions are based on some in a forthcoming book chapter by myself and Veronica Ranner: Lockton, D. & Ranner, V. (2017). ‘Plans and speculated actions: Design, behaviour and complexity in sustainable futures’. In: J. Chapman (ed.), The Routledge Handbook of Sustainable Product Design, Routledge, London.
Zai Aliyu: How might emotional intelligence in technology help us better express ourselves to others, empathize with others, or improve day-to-day interactions? In this faster moving world, how might the integration of ephemerality into our day-to-day lives change the way we interact, share, and connect with others, as well as our perception of identity?
Kate Apostolou: What is the future of spatial thinking in a world where people depend entirely on technology (smartphones and self-driving cars) to getÂ around?
Kaleb Crawford: What does a future look like where online content and copywriting is algorithm driven, not personally authored?
Ruby He: A New Age of Communication: Conversations withÂ Machines
Throughout the class, we looked at a variety of examples of design fiction, speculative design and other forms of making futures (and alternative pasts and presents) experiential—or at least tangible—through design. We explored the notion of diegetic prototypes and design fictions as detailed by David Kirby, Bruce Sterling and Julian Bleecker, and the influence of Hollywood, and popular visions of ‘the future’, including the very current Black Mirror, on how we as designers imagine what might exist, and what might be designed to bring that ‘alive’ to an audience. A major part of bringing the futures alive was finding ways to incorporate some of the side effects and side-shows uncovered in Exercise 2, either directly as part of a story, or through developing a scenario in which some designed product or service responded to those side-effects.
The aim of looking at existing examples was partly to consider the speculated futures themselves, but also to evaluate and critique the designed forms in which those ideas were presented, and relate them to the skills and ambitions the students have, and what might be achievable to create within a short period. So, from newspapers to photoshoots, websites to day-in-the-life videos, comics to service blueprints, renderings to prank product packaging, we examined these forms as potential models and inspirations, for their effectiveness and limitations.
It is important to remember: these were going to be undergraduate student projects, to end up in portfolios, be seen by potential employers and be shown in the end-of-semester exhibition—they needed to be comprehensible to an audience without extensive introduction or background-setting, and, while they could be amusing or serious, they needed to demonstrate the skills the students wanted to exhibit. Telling a story somehow, through either acting a scenario or showing us artefacts from that scenario (or both), seemed to be a good way of doing this, and we looked at some basic story forms via TV Tropes, Kurt Vonnegut andÂ Plotto.
One novel, contemporary form which emerged was the creation of digital ‘meta-artefacts’: not just webpages, but search engine results pages, reviews, forums, subreddits and even Wikpedia articles as design fictions, particularly to show the side-effects or side-shows, or other consequences of the main idea. Starting with Lauren Zemering, who saw the value of technology blogs and reviews as a potential way to highlight side-effects, students explored a range of media and platforms. I have no doubt that some of these have been done before in design fiction contexts, but the Play Lab students certainly took them in some interesting directions:
But this is getting ahead of ourselves somewhat. Exercise 3 was intended to help students start to world-build, to create a scenario in which the trend they identified in Exercise 1, and the side-effects and side-shows explored in Exercise 2, started to come together, as a way of providing a structure for Exercise 4 where the ‘future’ would be made. In some cases students pivoted from, or evolved significantly, the ideas they had initially been exploring, to produce a story which better fitted what they were interested in.
Create a scenario around the future(s) world(s) that you’re gradually exploring. What is a ‘realistic’ (maybe) everyday (maybe) situation that might exist in this world? What are the behaviors? What are the ambiguities? What is expected vs uncertain? Try to preserve some ambiguity: do you understand everything about the world around you now? Would you expect to do so in theÂ future?
Build a story around your scenario—bearing in mind that in Exercise 4, you will be ‘making’ parts of it, so if particular objects or settings are going to be important in the story, think about how you might do this. If you need to do so, work in pairs or groups of 3 to plan how to act each person’s story out (you’ll need one story one per person in the end—so split your time accordingly). Don’t script everything in detail, but work out some possible roles for different people and the general outline of aÂ story.
The ‘deliverable’ is: – notes on the scenario / story and people’s behaviors (on your blog) – notes on what objects or situations you might need to ‘make’ later (on your blog) – a brief run-through in class of the outline of your story (not a full-on dramatic production)
Ruby He: Scenario development around Conversations withÂ Machines
Alisa Le: Scenario development around Raising the Gender-Neutral Generation
Julia Wong: (1, 2) Scenario development around traveling in theÂ future
Jeff Houng: Scenario development around Hyper-personalization leading to experiential echo-chambers
Exercise 4 and 5: Make an experiential ‘pocket’ of a future; reflect, critique andÂ document
The final segment of the class, approximately the last 2Â½ weeks, was spent as studio sessions where students developed and implemented their scenarios from Exercise 3 in an experiential, or at least presentable, form, with regular desk crits and discussion with each other. The third group of students did this alongside preparing Focus, their senior design exhibition, but some students from this group nevertheless managed to get their Play Lab projects done in time to have onÂ show.
As well as continuing to look at examples of design fictions of different forms, we also did a group exercise to generate taglines and project titles for each other’s projects (based around the notion that if someone else can explain your project back to you, they probably understand it). After looking at some famous movie taglines and the ways in which they perhaps hint at the story but did not give it away, we also examined the phenomenon of the one-line elevator pitch, which often references other, existing products or services.
It’s clear that while these will often relate to very specific, current contexts, there are some structures which could almost be used generatively for creating (and explaining) speculative concepts, e.g. “Facebook for the dead” or “LinkedIn on steroids” in Chris Eleftheriadis’s list. In the event, the Play Lab students variously found these approaches useful or not, with a few usable taglines and project names being generated: Courtney Pozzi’s “First we cured cancer, now let’s cure ugly”, Hannah Salinas’s “Watch the world go by” and and Lea Cody’s notion of a “hangover for your whole life” being particularly memorable.
Create a way for an audience to ‘experience’ the future you are exploring, through bringing your story /scenario to life somehow in a form which can be presented. This could involve objects / artifacts, a website, illustrations, a video or a performance which you or others ‘act’ out, or maybe something that the audience can interact with. The aim is that an audience is able to get slightly more than a glimpse of this future, but also some of the frictions, ambiguities, tensions and uncertainties in it—what are the side-effects, the side-shows, theÂ issues?
The deliverable for this is a 5 minute maximum presentation (+3 mins of questions) of your ‘pocket of a future’ (this might be the wrongÂ term!).
Reflect, critique andÂ document
On your blog, write a short (500—800 word) reflection on the role of designers in this possible future. Has it come about through design, or through external factors? Are designers in this world acting within the story or scenario, or acting to oppose it? What could a common brief be for a designer (or design student) in thisÂ world?
Also, reflect on your speculation itself: is this a future you want to be part of creating? If so, how do you get there? If not, what do designers need to do now to prevent it? Make sure that your blog overall documents your process throughout Play Lab, including the development of ideas through Exercises 1, 2, 3 andÂ 4.
All the projects are featured at cmuplaylab.com, but it’s worth some discussion here of the trends that were apparent, and the variety of ways in which students tackled the issues they worked on. Over the course of the three groups, some larger themes emerged, which I attempt to summarize here. The preoccupations of imagined futures inevitably represent reflections on many of the issues of concern in the present, and perhaps tell us more about ourselves right now than we might first appreciate.
Many students were interested in ideas of the evolution of artificial intelligence, the Internet of Things, the usage of our data, and ‘intelligent’ devices more generally (see below). But two projects in particular explored the everyday environment of the ‘smart’ home and what it might be like to live in a context where our devices talk to us, day andÂ night.
Outside the home, four projects looked at what we might think of as futures of place, at different scales. Via a passport, travel guide and a Wikipedia page, Julia Wong’s Ab Ovo (1, 2, 3, 4) examines this globally, “extreme globalization leading up to the designation of permanent and mobile zones in cities and eventually the formation of a new culture and nation… due to the transition of economic needs and ultimately a progression of human values on a globalÂ scale.”
Brian Yang’s Big City X Tiny Home imagines a future where tiny homes become increasingly the default for city living as population rises. Through a property website and a clever fold-out IKEA advertising flyer, Brian explores how renters might find the homes, and what might be important to them (how to choose furniture, which amenities are shared, and soÂ on).
Albert Topdjian’s Green Space Initiative (e.g. 1, 2, 3) is an intriguing experiment in actually making the imagined future experiential in the present. The project envisioned a situation “where the density of city populations increases, and the available green spaces diminish, citizens can retreat into their personal/public indoor green rooms. There, one can relax, reflect, and socialize like it was in the good ol’ days of public parks, and green lawns.” Albert brought this to life by installing a patch of artificial grass in Carnegie Mellon’s University Center, branded #gspitt, and seeing how students used it. In documenting this via a Tumblr, he also fused this ‘feed from the future’ with contemporary tweets about green space fromÂ others.
The mainstreaming of forms of augmented reality in 2016 was reflected in two projects looking at how its evolution might affect our everyday interactions with the world and each other, particularly the notion of its enabling of / convergence with filter bubble-type phenomena.
Rae Headrick imagines the 2063 Channel Expo, an event unveiling hundreds of channels of augmented reality which people can choose to tune into for the next three years—“your chance to discover new realities and transform your life in one flip (in the Tuning Ceremony).”
Perhaps representing a closer future, Linna Griffin’s Update is a comic exploring the issues around a teenager making the decision to have the surgery needed for an AR implant—the peer pressure, the societal norms, differences in access between wealthier and poorer families, and the history of AR evolution as taught in schools. Linna reflects on these ideas here. With other students playing different roles, Linna’s live-reading of the comic in the final Play Lab presentations was a dramatic and effective way to bring the ideas and issues toÂ life.
The future of theÂ self
How will we present ourselves to others in the future? How will we curate and re-create the ways in which others see us? Three projects examined this from different perspectives.
Zai Aliyu’s Identity Curation explores two alternatives with opposite extremes of people’s investment in curation of their identity online at levels from the individual to the communal: “In one future, identity curation is made harder through ephemerality. What if all forms of media or posted content are temporal and they have an expiry date? In the other future, identity curation is made easier through impression management. What if we can cure this sense of disconnection that results from identity curation?”. In contrast, Leah Anton’s Transpersona imagined a service which enables people to adopt ‘personas’, to “break down cultural barriers by giving users the ability to behave as if they are from another background”, particularly “to perform appropriately in professional situations such as interviews and presentations”.
Alisa Le’s Beyond the Binary looked at the future of gender expression—“how different the world would be if from the beginning humans were not raised based on these stereotypes and “norms””, via “an imaginary gift set a new parent might receive during a baby shower or a similar celebration”. Alisa also explored more questions around design’s role in perceptions and reinforcement of gender norms inÂ society.
Dealing withÂ death
Evolution in the way we deal with death is the focus of projects by Vivian Qiu and RachelÂ Chang.
Vivian’s ASHES. is a “high brow magazine for the future elite to plan their own funeral”, imagining a world where “dying becomes a claim to individuality or a exposition of values” through articles such as ‘Curate Your Digital Legacy’ with North West, and ‘5 Mournists You Should Know’ covering ‘notable designers in the field’. Rachel’s Orpheus explores how “as a way of memorializing dead loved ones, bots built from the immense digital trail individuals leave behind become common. Preparing the bot is just another step of estate planning.” The Orpheus prototype concentrates on the touchpoints of the service, while Rachel’s reflection examines some deeper questions around artificial intelligence, design, andÂ death.
The body asÂ site
The relationship between our bodies and technological advances was a theme running through a number of projects, but three in particular explored this in moreÂ depth.
Courtney Pozzi’s Cosmetic Genetics examines the development and mainstreaming of gene therapy and genetic engineering via a projected timeline and scenes from that timeline, with newspaper and magazine headlines and covers. Building on the pattern seen in reconstructive surgery and popularized drugs, the project explores the “idea of a tipping point, or the when, what, and whys behind how an idea detrimentally splits between its utopia and dystopia, and how both of these worlds may coexist within a singular future.” Courtney explores these questions further in her reflection in relation to speculative design and ambiguity.
Brandon Kirkley created a story taking place in a future where “FashionTech—fashion with wearable technology that showcases one’s emotions—becomes popular”. Dealing with social interactions emerging from the visible display of emotions, and the consequences, the video “explores an everyday situation in which someone denies how they are feeling—despite what their clothes say.” Brandon also reflects on the role of the designer in relation to emotion, and the idea of such an approach making communications between humansÂ easier.
Voices in theÂ head
The idea of voice interfaces, and more specifically, invisible voice technology audible only to the user, is explored, in different ways, in two projects. Ruby He’s Visual Aids in an Audio World imagines the (AR) iconography needed to accompany conversations with each other in a world “when information and interactions are almost entirely in our ears, it will be much easier for people engage in audio realms different from their immediate surroundings by playing music, listening to podcasts, etc without others even knowing.”
Going perhaps one step further, Zac Mau’s Future of iOS brings to life the idea that “within 50 years we will stop using phones to natively access the Cloud and digital service platforms including Facebook, Maps and Uber, and instead stream all of our digital content directly through our brains. This would result in a thought-based interface that we mentally interacted with, rather than physically”. Zac illustrates this very effectively through “a day-in-the-life kind of video, that briefly takes the audience through this future where speaking to an OS in your head is already theÂ norm”.
Changing relationships withÂ language
How will changes in technology affect how we talk to each other? Two projects took this question in different directions. Kaleb Crawford’s Marko V examines ‘contextual computation co-authorship’ via “a speculative fiction that explores future social media interactions at the intersection of conversational interfaces, automated text generation, and post-context-collapse. Told through four audio-vignettes, the story explores how Adrian’s use of a conversational writing assistant, Marko, amplify tensions in his social relationships with a loved one, his ego, and his friends.”
Jiyoung Ahn’s Trilingo, in contrast, imagines the evolution of a new language, CCE, which fuses vocabulary elements of Chinese and English with the formal structures of computer programming languages, to enable easier human-machine communication. The idea is that children are learning this at school, and the need exists for older generations, particularly grandparents, to learn CCE as a second language. Trilingo is an educational language learning platform aiming to bridge thisÂ gap.
Memory andÂ privacy
Questions around the use of data, and questions of memory and privacy, ran through three quite different projects. Kaitlin Wilkinson’s The Private Life considers a future world where privacy has been effectively abandoned—“People have become accustomed to having personal data tracked and shared at all times. This data ranges from things about health, fitness, eating habits, social habits, experiences, etc. The idea of personal information is not understood.” Kaitlin envisions a museum exhibit on personal documentation such as letters and private photos, things which “no longer play a role in the life of the average person”, complete with almost incredulous copy explaining to the public the “primitive act of personal documentation” and the “unfathomable” ways in which people “tried to hide aspects of their lives from publicÂ view”.
Jillian Nelson’s Lifeed extrapolates from ideas of lifelogging and sousveillance to imagine a service which “creates a catalogue of everything you see and experience in order to use it as a way to document your life story for yourself, your descendants, and for those seeking to learn about the past in the far future.” Lifeed would enable reflection and change our relationship with memory, and primarily be for people themselves rather than for sharing on social media; Jillian discusses the ideas here, including the possibility of creating ‘life documentaries’ in theÂ present.
We saw projects earlier dealing with wider contexts of ‘smart’ homes, but two projects focused on specific product-service systems within the field. Lauren Zemering’s Hygienie is a “line of bathroom devices [that] allow users to monitor and track their shower, teeth brushing, and toilet habits. The devices not only monitor and track them, but are also able to provide feedback that can help improve a person’s health and hygiene. AND they can be awarded points and compete with their peers!” The site presents the system from the manufacturer’s point of view (including a revealing FAQ), but also shows some of the wider side-effects of the data’s use via a Google search page. Lauren reflects on these ideasÂ here.
Justin Finkenaur’s Intelligent Furnishings explored the development of ‘smart’ furniture which, in response to sedentary lifestyles, involves ‘automating’ furniture so it adjusts itself to the ergonomic needs of individual people. The proposed service was explored through service blueprints, experience maps and a storyboard.
When the millennials age
Two projects, from Vicky Hwang and Daniel Kison, deal with questions of what happens as the millennial generation ages. Vicky (forum text and a discussion of the video) imagines a disorienting experience via a forum where the ageing millennials ask for help with the ‘New Web’. Daniel takes as his starting point the current notion that “Millennials are killing everything” and imagines what vanished activities the post-millennial generations might want to ‘live’ for the first time,through a store display of virtual experiences.
Our relationship with celebrity, and how it might change, was the subject of projects by Praewa Suntiasvaraporn and Hannah Salinas. Hannah’s Watch The World Go By tells the story of Alexa, a vlogger, and Vicky, a ‘watcher’, who spends her day watching Alexa via screens everywhere at home and at work. The video explores the effects this has on the lives of Vicky and her friends. Hannah’s blog post shows the significant amount of work required to produce theÂ video.
Consumer Culture, by Praewa Suntiasvaraporn, explores food celebrity, an evolution of current trends such as mok-bang and the food porn of Instagram. Praewa created a world around this, including personal artefacts from the life of food celebrity Mikah Wong, exposed through hidden cameras placed by an infatuated fan. The project examines how lifestyle eating, food presentation and photography become so significant that they dominate popularÂ culture.
The medicalization of everything
Three projects examined, in very different ways, the potential evolution of the current trend towards the ‘medicalization’ of human conditions and problems, perhaps a form of solutionism. Catherine Zheng’s Prescriptive Lifestyles is a range of medications, branded stasis: perfect state, which each target common emotions and situations such as ‘timidity’, ‘focal deficiency’ and ‘sluggishness’. In Catherine’s reflection, she notes how “No longer are people able to simply be themselves, as there is always a way to “become better”, whether that means acting, looking, thinking, or feeling the “acceptable” way.”
Jonathan Kim’s NowFood.Inc envisioned a range of ‘medicalized’ food products and consumption-related drugs such as Muse, a treatment which reduces the immediate side-effects ofÂ smoking.
Finally, Lea Cody’s Physio is an exploration of the evolution of the fitness tracker, going beyond quantified self into the realm of actually changing behavior through operant conditioning—a “health improvement system”: “After receiving a Hormonal Adjustment Implant, the user will obtain a prescription for Physio, the external tracker. Physio will reward ‘good’ behavior like exercise with accelerated effects of increased agility, decreased anxiety, and euphoria. It will punish ‘bad’ behavior like poor diet, smoking, and lethargy with decreased agility, irritability, and depression.” Lea’s project provides an interesting example for the possibility space in design for behavior change around health, and I think could serve as a useful touchstone for this particular approach.
In her reflection, Lea suggests something which perhaps offers a succinct summary of the feelings many (not all) of the Play Lab students had about the ideas they explored:
“I believe that the role of the designer in this scenario, as in it seems all current scenarios, is to rein in the possibilities of technology to ensure a human-centered, benevolent focus.”
This is the first class I have taught at Carnegie Mellon, and I was not sure what to expect. But the students have, very pleasantly, allayed any trepidation I had: they have been enthusiastic, insightful, critical and creative, and have worked very hard in short periods of time to produce the work you’ve seen on this page. I am proud of them for what they have achieved, given some very vagueÂ briefs.
What is the value of this class? I feel that being able to explore, and make ‘real’, different visions of possible futures is an important part of ‘designing agency’ and indeed Transition Design, but most importantly, to be able to think through, manage and cope with uncertainties, ambiguities and potential side-effects in a design process. If, as outlined in the introduction, Play Lab can help undergraduate design students “develop a shift in perspective on their own abilities and identities as designers”, “give them confidence to tackle new kinds of problems and challenges in a reflective way”, and help them “developing… a thought process which is not just about problem-solving, but comfortable with ambiguity, problem-finding, and problem-worrying”, then it will have succeeded. Maybe I am just teaching the class I wish I had had as an undergraduate design student, and am lucky enough to be working somewhere that gives me the freedom to doÂ that.
The ‘critical’ in the Play Lab projects is, largely, not too critical, but it is certainly demonstrative of a curious and enquiring mindset, and, with at least a few of these projects, I think there are hints of comfort with ambiguity and contrast and the good and bad all at once, the “complicated pleasures” that Dunne and Raby speak of. Dave Wolfenden, a DDes candidate at Carnegie Mellon, is researching the notion of negative capability, a term coined by the poet John Keats to describe the state of someone “capable of being in uncertainties, mysteries, doubts, without any irritable reaching after fact & reason”, and after hearing Dave speak about his research, I realize that perhaps this is what Play Lab aimedÂ for.
The future will be at least as complex as the present, and the projects resulting from Play Lab show a recognition of that. The design fictions the students have created are, no doubt, very much rooted in the concerns they have about the world they are growing up in, and the world into which they are about to head out once they graduate (yes, in the eyes of many of the critics of speculative and critical design, most of these projects would probably, rightly, be categorized as arising from a privileged group of students with a particular mindset, and dealing primarily with people like them, but going beyond that was not the point of theÂ class).
As Christopher Beanland phrases it (in an article on an abandoned British maglev train project, a kind of Brummie Aramis): “When you get to a certain age you realise how much more visions of the future say about the present they’re concocted in than the actual future they purport to show us hurtling towards”. Maybe what the Play Lab students have created, to draw on StanisÅ‚aw Lem, is not a collection of futures, but a set of mirrors for theÂ present.
Thank you to the students, first of all, for their enthusiasm and for giving me a wonderful welcome to CMU: Jiyoung Ahn, Rachel Chang, Rae Headrick, Jonathan Kim, Brandon Kirkley, Courtney Pozzi, Vivian Qiu, Diana Sun, Albert Topdjian, Kaitlin Wilkinson, Lauren Zemering, Leah Anton, Justin Finkenaur, Vicky Hwang, Jackie Kang, Daniel Kison, Zac Mau, Jillian Nelson, Hannah Salinas, Praewa Suntiasvaraporn, Brian Yang, Catherine Zheng, Zai Aliyu, Kate Apostolou, Lea Cody, Kaleb Crawford, Linna Griffin, Ruby He, Jeffrey Houng, Alisa Le, Gabriel Mitchell, Temple Rea, and Julia Wong; thanks too to my fellow senior design labs leaders Mark Baskinger and Michael Arnold Mages for co-ordinating and helping me get started; to Peter Scupelli for the very useful insights into previous senior design labs; to Bruce Hanington for the guest lecture for group 2; to Harriet Riley for making shah biscuits and bringing Marks & Spencer tea back from Britain for the class; to Veronica Ranner and Ahmed Ansari for lots of chats about ideas; and to Molly Wright Steenson, Stella Boess, Deepa Butoliya, Dimeji Onafuwa, Theora Kvitka, Terry Irwin, Dan Boyarski, Shruti Aditya Chowdhury, Tammar Zea-Wolfson, Robert Managad and Matthew McGehee and anyone else I have missed, who came to visit the groups’ presentations and asked some very good questions.