Challenge
The exisiting interactive placemaking designs excel at the scale of the display and the creative content. Yet few of them pay attention to create a sense of immersion to participants in the meantime.
The challenge for this project is to design and implement a novel interactive audio-visual system for urban environments that enables users to gain a sense of immersion and feelings of awe.
My role
Solely responsible for the complete process from research, ideation, product design, user test to prototype development.
Tools
Processing | Max | Photoshop
Timeframe
3 months (2020)
Concept video
Background research
Design inspiration
"What is behind the clouds?"
Starting from a young age, I have always wondered what is behind the amazing clouds. What is out there to be comprehended? Movies and books that explore the universe alway fascinate me.
I was inspired to create an interactive audio-visual system that enables users feel 'immersed in the universe' to experience the wonder of it and leave behind humanly problems.
Design precedent
When it comes seamless immersive digital experiences, teamLab has to be the one to mention. Their creations of immersive and interactive environments blur the boundary between virtual and physical worlds.
Ideation
I developed on the idea of creating an immersive experience that gives users the illusion that they are exploring the universe and interact with the objects in the simulated system.
Design location
The location of design will be indoor which is more sound-proof and immersive. It will be located in a large room with high celling. Design can be projected on four walls to increase a sense of immersion.
Develop prototypes
Processing and Max were the main tools I used in developing prototypes in this project.
Server-client framework - A server and client framework was implemented. On the server side, users can interact with the design by pressing keys on keyboard and using their computer camera for face detection. The client side is where the interaction display mapped at a real location.
3D planet images - I imported 3D images of planets and coded to let them ‘float’ from centre of the screen to the edge by increased sizes so that users can feel they are traveling in space in the opposite direction.
Vortex - The vortex is spinning from the centre in a steady and constant way to empathise on the eternal balance of the universe. The background music is ambient.
User interaction - When user types ‘a’, ‘w’ or ‘d’ on keyboard, a planet appears on screen. When user types ‘j’, ‘i’ or ‘l’, a planet disappears from screen. And each key triggers a particular sound effect.
User test
User test was conducted virtually via Zoom due to COVID19. User played on the client and server framework codes. Post user test, I asked questions such as ‘how did you feel when you interacted with the design?’
"I felt like I was immersed in the universe. But I think more planets should appear each time and more types of interaction should be added.
Iteration
Based on the user test results, I made following changes:
More planets - I changed the display of planets from one at a time to three each time button is triggered.
Blackhole interaction - I added a blackhole picture. When user interacts with the design, his/her computer camera becomes active. Once user’s frontal face is detected, the blackhole will alter size based on the distance the user is to the camera. More users interact, more blackholes will appear.
Sound visualisation - I used Max to analyse sounds in the environment. Two purple circles are added to visualise the sound analysis of pitch and timbre of surrounding noises. and changing sizes based on the amplitude and pitch of sounds in the room.
Physical & virtual prototypes
To be able to demonstrate the system design, I applied both physical and virtual prototypes. I covered the laptop keyboard with a black cardboard, and used paper to draw the camera and play zone to demonstrate the environment of the design. The computer screen acted as the front wall where the interaction happens.
Implementation
A play zone will be allocated near the display wall. When user steps on up arrow, three planets will appear and when stepping on down arrow, 3 planets will disappear. Each arrow has an individual sound effect.
Cameras in front detect user’s face, and the blackhole alters sizes based on the distance user is to the wall. The purple circles alter sizes based on the timbre and pitch of sounds in the room.
Demo video
What I’ve learned
Designing and developing ‘Multiverse’ was such a fun experience. During a quarantine situation, it was impossible to do real life user test and build real products. So I had to think outside the box. The process of making physical prototypes with limited resources was a great experiment. I learned that during special occasions, even thought it could become difficult to develop on the project, it was also an opportunity to exercise our creative muscles.
