What Is Kioku?
What Can Kioku Do?
CAPTURE A MOMENT FOR LOVED ONES
User can capture a 10-second moment at any time with a simple tap. The moment will be automatically stored in Kioku's cloud system, and a series of tags will be generated to define the context of that moment (i.e., "flower", "daytime", "outdoor").
EXPERIENCE A MOMENT IN RELEVANT CONTEXT
When another family member passes through a environment that Kioku identifies as contextually relevant with a stored moment based on tag matching result, that moment will appear in the environment as a virtual bubble. By gazing at the bubble and double tap the AR device, users can experience the moment in first-person point of view.
REMINISCE THROUGH SAVED MOMENTS
After viewing the moment on site, user can choose to save that moment into his/her personal library, which can be accessed later by swiping outward on the AR device. When user wanders in the library, unfrequented moments will move closer to the user and start fading if continue to be neglected.
Why We Created Kioku?
CONNECT REMOTE FAMILY
How can we leverage AR technologies to enhance remote family bonding?
How might we synchronize context in a passive an experience-centric way to improve remote communication experience?
Kioku embraces the physical environment as the new communication interface, allowing remote families experience the mundane daily moments together across time and space.
Since Kioku augments the environment only when it is contextually relevant, it creates a passive and experience-centric communication method that does not require active and focused participation.
UNDERSTANDING THE CONTEXT
This project is sponsored by Microsoft, who invites us to design a product or service that demonstrates the best qualities of Mixed Reality. The solution needs to bring a positive cultural or social impact.
The team first defined mixed reality as "combing the virtual and the real worlds". Then we identified the differentiated values of mixed reality as creating "contextual information" and "multi-sensory stimulation". The team brainstormed design territories that have potentials in using mixed reality, and we eventually choose "communication in a family setting" as our research focus.
We interviewed a total of 21 people including 10 remote family members. The team became interested in learning more about communication among remote family members as a way of bonding.
We recruited 2 pairs of co-living and remote family members, and observed their ways of communication under a scenario that is common for family living: cooking.
We analyzed 15 current communication tools used by remote families, and mapped them on a 2x2 matrix.
After summarizing our discoveries with sticky notes, we identified 3 main insights:
01 - Remote families are faced with a loss of shared context.
When families are physically living together, they are able to share each other's context instantaneously; when they are apart, they lost such privilege and thus experience communication difficulties with nonaligned or unfamiliar contexts.
02 - Mundane everyday moments are often neglected in remote communication.
Even if these moments are shared, most of the time they are verbally described in a fragmented way: information about environments, objects, facial expressions, and other contextual details that that constitute an experience are often lost in the sharing process.
03 - Current communication tools cause anxiety and lower the quality of remote communication.
As communication tools constantly notify people to take actions to messages, remote communication is no longer about sharing each other's context, but about ignoring each other's context: the sender injects his/her context into the message, and expect quick responses without thinking about the situation of the receiver; the receiver is forced to pay crazy amount of attention to devices, and may fail to fully engage in a conversation because he/she might be in a completely irrelevant context.
Based on these insights, the team determined that our design objective is:
IDEATION & SPEED DATING
Based on the design objective, the team generated 3 possible design directions to help us ideating design concepts - synchronize context by:
- only initiating communication experience when family members are in similar contexts;
- creating a new communication environment that invites family members into the same context;
- providing transparency to each family member's local context.
The team brainstormed ideas and conducted speed dating sessions with 10 people. We gathered feedbacks that lead to 3 design implications to guide us moving forward.
The team therefore proposed to design a passive and experience-oriented communication tool that challenges people to rethink what it means to communicate:
How does it feel to know your dear one's life without the fatigue of waiting for responses and replying instantly?
How does it feel to encounter serendipitous moments and experience your dear one's life in similar scenarios and settings?
Our final concept, which is named Kioku (Japanese word for memory), is an augmented reality communication service that allows remote families naturally share everyday little moments in similar contexts despite time and location differences.
USER JOURNEY & INTERACTIONS
After defining the basic user flow for the system, the team went on to detail out specific interactions within each component based on different scenarios.
Kioku embraces the physical environment as the communication platform and present augmented information in a context that relates to both the sender and the receiver.
Kioku leverages current machine learning technologies for context recognition. After a family member captures a moment, Kioku will analyze and generate a set of categorical tags that define the context of that moment. When another family member uses Kioku elsewhere, the system will constantly scan his/her physical surroundings and detect if any stored tag combination matches with the current context. If there is a match, the related moment will be retrieved from the cloud and ready to be experienced.
We made some quick image collages to test which form works the best as a moment notifier, and we chose the crystal ball form in the end. Here are some of our key findings:
- Organic forms are more approachable but less obvious compared to perfect geometric forms;
- Solid shapes with no transparency would block user's views;
- It's hard to define the boundary for contextual notifiers (highlight a random object in the environment).
2D GRAPHIC NOTIFIER
3D CONTAINER NOTIFIER
NOTIFIER PLACEMENT & INTERACTIONS
The team quickly made some low fidelity prototypes to test the user experience with the notifier and the moment itself, and we found the following insights:
- The most comfortable setting for moment viewing screen is a curve plane that covers 110° horizontally, and 50° up/70° down vertically of the visual field;
- The sphere notifier has the most natural yet noticeable presence if its diameter is between 20-25cm within field of view;
- Using sound as the moment notifier instead of visuals will not work if the user is in a noisy environment;
- Using long-gaze gesture to open the moment will not work for crowded environment. Instant gaze to select and double tap to open is more feasible.
To continue with this project, we'd like to:
- including on-boarding experience for the service;
- Design interfaces that allow user to manage moments in the personal library;
- Prototype the experience with Unity + AR SDK.