Designer Role
Product, UX/UI, Research
Duration
June - November 2024
SERVICE DEDSIGN
Immersive Music Experience and Lyric Translation
With Vision Pro
overview
Step into the music, feel the meaning
Immersive and in real-time with Vision Pro.
Eager to be surrounded by a space that matches the vibe of your favorite song?
Curious about the deeper meaning of foreign lyrics?
With Vision Pro, dive into an immersive music experience with real-time lyric translations, allowing you to feel every beat and word visually.
Design process
Discover
User INTERVIEW
Define
Develop
Deliver
Competitor Analysis
Affinity Diagram
Service Goal
User Scenario
Concept model
Task flow
Wireframing
USABILITY TESTING
Final design
Background
Rising Demand for Seamless Foreign Music Experiences
When listening to foreign music, constantly looking up lyric translations can break the flow and diminish immersion. Listeners need a seamless way to understand lyrics in real-time, staying fully engaged in the music.
45.1%
of 2023's top global streaming tracks were non-English music.
Luminate’s report 2023
That was up from 37.9% in 2022
and 33% in 2021.
Platforms like Genius and Musixmatch highlight the growing demand for lyric access, allowing millions of users to better connect with foreign music.
problem Statement
Foreign music listeners prioritize the atmosphere of the music but struggle to maintain immersion, as searching for lyric translations disrupts the experience.
Competitor Analysis
Uncovering Key Insights and Opportunities
We conducted a detailed analysis of key players in the lyric translation space, focusing on Musixmatch, Genius, and YouTube personal lyric videos. The goal was to identify strengths, weaknesses, and opportunities for innovation to create a more immersive and seamless experience for foreign music listeners.
User-created videos featuring translated lyrics alongside the music, often with engaging visual elements.
Combines visuals and lyrics for an enhanced viewing experience.
Quality and accuracy depend on the creator, and there are potential copyright issues.
YouTube Lyric Videos
The world's largest lyrics platform, integrated with Spotify and Apple Music for real-time synced lyrics. Users can contribute by translating or editing lyrics, supporting over 62 languages.
Extensive language support and crowd-sourced accuracy.
Requires a separate app and disrupts the listening experience by switching between platforms.
Musixmatch
Provides lyrics with in-depth annotations and explanations. Integrated with Spotify to show lyrics and background insights simultaneously.
Rich explanations and background information for a deeper understanding of music.
Limited support for non-English languages and lacks consistent lyric translations.
Genius
💡 Opportunities
Create a visually immersive experience that connects the music's atmosphere with real-time lyrics.
Provide translations that convey emotional and cultural depth, enriching the music experience beyond simple text.
Context-Rich Lyric Translation
Ensure all features are integrated within a single platform to maintain uninterrupted music flow.
Seamless User Experience
Immersive Music Experience
HMW create a seamless and immersive music experience that visually connects lyrics to the song’s atmosphere while enhancing emotional and cultural understanding?
User INTERVIEW
Finding What Matters,
A Truly Immersive Music Experience
We conducted 5 interviews and clearly identified their needs.
User interviews revealed that it’s not just about translating lyrics. Listeners are looking for a deeper, more immersive music experience where lyrics add to the mood without breaking the flow.
This shifted our focus to creating a seamless, visually engaging environment that lets users fully dive into the music.
Listening Disruption
Mood Alignment
Immersive Experience
“When I'm curious about the meaning of lyrics while listening to foreign music, I have to leave the streaming site and search for a translation.”
“I prefer translated lyrics that match the mood of the music, ideally presented visually, rather than simple translations from a translator.”
“Lyrics are secondary; feeling the atmosphere of the song is more important for fully enjoying the music.”
Affinity Diagram
Structuring Our Vision
We used an Affinity Diagram to organize and analyze user insights, which allowed us to identify key themes and brainstorm feature ideas. From this process, we derived several essential components for our service
Based on the insights, we determined that Apple Music would be the ideal streaming platform for integration, given its widespread use and compatibility.
Seamless Music Experience
Features that ensure uninterrupted music flow, like integrated real-time lyric translations.
To deliver the immersive experience we envisioned, we chose Apple Vision Pro as our primary device, leveraging its advanced visual and spatial capabilities.
Immersive Visuals
Elements that visually connect lyrics to the music’s atmosphere, enhancing user engagement.
Platform and Device Decisions
Key Features
Service Goal
A Richer Way to Enjoy Foreign Music
Our service aims to deliver a fully immersive music experience by leveraging Apple Vision Pro to create dynamic environments that match the music's atmosphere. We also provide real-time lyric translation with an interactive chatbot and a unique humming-based song search feature for seamless music discovery.
Interrupted
Flow
Seamless
Experience
Language
Barriers
Contextual
Understanding
Limited
Foreign Song Search
Effortless
Discovery
Painpoints
Solution
Needs
Immersive
Environment
Translation
& Chatbot
Humming
Search
😞
🧐
✨
Narrative Design
Escaping to Immersive Spaces
Sarah, a marketing manager in New York, often takes short breaks between her demanding tasks. She loves listening to calming music while working but struggles to disconnect during her breaks.
Using the immersive video playback feature, Sarah transforms her break time into a virtual escape to serene landscapes that match her playlist's vibe. For Sarah, the immersive environment helps her recharge and return to work with renewed focus.
Learning Korean Through K-pop
Tom, a university student in the UK, is an avid K-pop fan and is also learning Korean. He loves discovering the deeper meanings behind his favorite songs but often finds lyric translations inconsistent or difficult to access.
With real-time lyric translation and the AI chatbot, Tom can instantly grasp the meaning of the lyrics and ask follow-up questions about cultural references or idioms. For Tom, this feature bridges the gap between his passion for music and his language-learning journey.
Sarah’s Story
Tom’s Story
Meet Sarah, a busy marketing professional, and Tom, a K-pop enthusiast studying abroad. These stories illustrate how our features enrich their music experiences and solve everyday frustrations.
Feature in Action: Sarah’s Breaks and Tom’s Discoveries
User Scenario
From Humming to Immersion,
the User’s Path Unfolds Seamlessly
In crafting this user scenario, we outlined how each feature contributes to a seamless and immersive music experience. We detailed the user’s journey from discovering songs through humming to engaging with real-time lyric translations and a visually rich environment.
#1 Melody Recall
#2 Humming Search
#3 Song Match
The user remembers a catchy foreign song but can't recall the title.
The user hums the melody, and the system quickly identifies the song.
The matched song is displayed, ready to be played.
#7 Chatbot Insight
#8 Deeper Context
#9 Mood Visuals
The user remembers a catchy foreign song but can't recall the title.
The chatbot provides cultural references and deeper meanings behind the lyrics.
The immersive environment enhances the song's mood with matching visuals.
#4 Playback
#5 Lyric Sync
#6 Gesture Query
The user starts listening to the song.
The lyrics appear in real-time, perfectly synchronized with the music.
Curious about a specific lyric, the user uses a hand gesture to explore further.
#10 Full Immersion
#11 Seamless Switch
#12 Experience Reflection
The user feels fully engaged, enjoying both the audio and visual experience.
The user easily switches to another song, repeating the immersive experience.
The user leaves feeling enriched, fully enjoying music from any language.
Concept model
Building the Experience Architecture
Our concept model encompasses both the core features we have designed—humming-based song search, real-time lyric translation, an AI-powered chatbot, and immersive video playback—and additional elements for potential future development. This approach demonstrates our comprehensive vision for a seamless, immersive music experience using Apple Vision Pro, while prioritizing the features that are currently realized in the design.
Task flow
Seamless Music Interaction Journey
This flow outlines the user’s journey from launching Apple Music to engaging with immersive environments and accessing song-related features. It illustrates key decision points and interactions, such as voice-based and gesture-based searches, real-time lyric translations, and customizable visual experiences. By mapping each step, we highlight the intuitive and efficient navigation designed to enhance music discovery and enjoyment.
Final Design
Harmonizing Design
with Existing Platform and Emerging Device
With a clear task flow guiding the user journey, we moved directly to final designs. This approach ensured our efforts were focused on refining the visual and interaction details while adhering to Apple Music's established ecosystem.
The design shown here represents the final version at this stage, incorporating extensive research and internal feedback. Future iterations will integrate findings from planned user testing to further enhance the experience.
Humming Search
Can’t remember the title or lyrics? Cover one ear and hum the melody – we’ll find the perfect match for you.
We provide clear and friendly guidance for using gestures to make your experience seamless.
As you hum, a sound wave visualization appears at the bottom, giving you real-time feedback on your input.
Immersive Video Playback
Step into a world that matches the vibe of your music.
Experience your playlist in stunning, immersive environments.
Preview and choose your desired virtual space to match the mood of your music.
While listening to music, clench and release your fist facing forward to enter the immersive music experience space.
Real-Time Lyric Translation
AI-Powered Chatbot
Pinch and Drag real-time translated lyrics to explore deeper meanings with our chatbot.
Auto-complete suggested questions are provided for a smoother experience.
Enjoy foreign music without language barriers. See lyrics translated in real time, synced perfectly with the song.
Curious about a lyric’s meaning or context? Highlight the text and let our chatbot provide the answers you need.
USABILITY TESTING
Heuristic Evaluation for Enhanced Usability
To be updated
Reflection
What I learned
Next Steps
This project underscored the critical role of user research. Initially, I assumed the main problem was the inconvenience of lyric translation, but research revealed that users were more interested in a fully immersive music experience. Additionally, designing for an emerging device like Vision Pro challenged me to understand new mental models and consider unique aspects of VR interface design. It was also a valuable opportunity to deepen my understanding of Apple’s design system and how to adapt existing guidelines for a cutting-edge platform.
I’d like to further explore the potential for user-contributed content, such as allowing users to upload translations or immersive videos to enrich the platform. Additionally, I regret not being able to conduct user testing with frequent Vision Pro users. Moving forward, I aim to test the design with real users of the device to gain deeper insights and refine the experience.