Immersive Music Experience
and Lyric Translation With Vision Pro
Immersive Music Experience
and Lyric Translation With Vision Pro
Competitor Analysis
Scenario Mapping
High-Fidelity Design
Competitor Analysis
Scenario Mapping
High-Fidelity Design
What I Faced
When music stays passive
When music stays passive
Existing music platforms focus on passive listening, offering limited ways to explore or interact with music. Discovery feels flat, lyric comprehension is restricted by language barriers, and video experiences remain detached from the music itself.
Existing music platforms focus on passive listening, offering limited ways to explore or interact with music. Discovery feels flat, lyric comprehension is restricted by language barriers, and video experiences remain detached from the music itself.
Platforms like Genius and Musixmatch highlight the growing demand for lyric access, allowing millions of users to better connect with foreign music.
What I Did
Spatial journeys for music
Spatial journeys for music
Defined core music scenarios and user journeys for a spatial interface. Prototyped flows for humming search, lyric translation, and immersive playback, built a concept model to unify features, and created high-fidelity screens to showcase Vision Pro’s immersive potential.
Defined core music scenarios and user journeys for a spatial interface. Prototyped flows for humming search, lyric translation, and immersive playback, built a concept model to unify features, and created high-fidelity screens to showcase Vision Pro’s immersive potential.
What it became
Step into the music, feel the meaning
Immersive and in real-time with Vision Pro
Step into the music, feel the meaning immersive and in real-time with Vision Pro
I designed an immersive music experience for Apple Vision Pro, prioritizing user needs and seamless interaction. This project revealed that users value immersive music enjoyment over standalone lyric translation. It allowed me to explore the mental models of new devices and adapt designs to Vision Pro's unique capabilities.
I designed an immersive music experience for Apple Vision Pro, prioritizing user needs and seamless interaction. This project revealed that users value immersive music enjoyment over standalone lyric translation. It allowed me to explore the mental models of new devices and adapt designs to Vision Pro's unique capabilities.
Learning from the landscape
Why existing solutions fall short of immersion
Why existing solutions fall short of immersion
I conducted a detailed analysis of key players in the lyric translation space, focusing on Musixmatch, Genius, and YouTube personal lyric videos. The goal was to identify strengths, weaknesses, and opportunities for innovation to create a more immersive and seamless experience for foreign music listeners.
I conducted a detailed analysis of key players in the lyric translation space, focusing on Musixmatch, Genius, and YouTube personal lyric videos. The goal was to identify strengths, weaknesses, and opportunities for innovation to create a more immersive and seamless experience for foreign music listeners.
YouTube Lyric Videos
Quality and accuracy depend on the creator, and there are potential copyright issues.
Combines visuals and lyrics for an enhanced viewing experience.
User-created videos featuring translated lyrics alongside the music, often with engaging visual elements.
Limited support for non-English languages and lacks consistent lyric translations.
Provides lyrics with in-depth annotations and explanations. Integrated with Spotify to show lyrics and background insights simultaneously.
Rich explanations and background information for a deeper understanding of music.
Genius
Extensive language support and crowd-sourced accuracy.
Requires a separate app and disrupts the listening experience by switching between platforms.
The world's largest lyrics platform, integrated with Spotify and Apple Music for real-time synced lyrics. Users can contribute by translating or editing lyrics, supporting over 62 languages.
Musixmatch
YouTube Lyric Videos
Quality and accuracy depend on the creator, and there are potential copyright issues.
Combines visuals and lyrics for an enhanced viewing experience.
User-created videos featuring translated lyrics alongside the music, often with engaging visual elements.
Limited support for non-English languages and lacks consistent lyric translations.
Provides lyrics with in-depth annotations and explanations. Integrated with Spotify to show lyrics and background insights simultaneously.
Rich explanations and background information for a deeper understanding of music.
Genius
Extensive language support and crowd-sourced accuracy.
Requires a separate app and disrupts the listening experience by switching between platforms.
The world's largest lyrics platform, integrated with Spotify and Apple Music for real-time synced lyrics. Users can contribute by translating or editing lyrics, supporting over 62 languages.
Musixmatch
YouTube Lyric Videos
Quality and accuracy depend on the creator, and there are potential copyright issues.
Combines visuals and lyrics for an enhanced viewing experience.
User-created videos featuring translated lyrics alongside the music, often with engaging visual elements.
Limited support for non-English languages and lacks consistent lyric translations.
Provides lyrics with in-depth annotations and explanations. Integrated with Spotify to show lyrics and background insights simultaneously.
Rich explanations and background information for a deeper understanding of music.
Genius
Extensive language support and crowd-sourced accuracy.
Requires a separate app and disrupts the listening experience by switching between platforms.
The world's largest lyrics platform, integrated with Spotify and Apple Music for real-time synced lyrics. Users can contribute by translating or editing lyrics, supporting over 62 languages.
Musixmatch
YouTube Lyric Videos
Quality and accuracy depend on the creator, and there are potential copyright issues.
Combines visuals and lyrics for an enhanced viewing experience.
User-created videos featuring translated lyrics alongside the music, often with engaging visual elements.
Limited support for non-English languages and lacks consistent lyric translations.
Provides lyrics with in-depth annotations and explanations. Integrated with Spotify to show lyrics and background insights simultaneously.
Rich explanations and background information for a deeper understanding of music.
Genius
Extensive language support and crowd-sourced accuracy.
Requires a separate app and disrupts the listening experience by switching between platforms.
The world's largest lyrics platform, integrated with Spotify and Apple Music for real-time synced lyrics. Users can contribute by translating or editing lyrics, supporting over 62 languages.
Musixmatch
YouTube Lyric Videos
Quality and accuracy depend on the creator, and there are potential copyright issues.
Combines visuals and lyrics for an enhanced viewing experience.
User-created videos featuring translated lyrics alongside the music, often with engaging visual elements.
Limited support for non-English languages and lacks consistent lyric translations.
Provides lyrics with in-depth annotations and explanations. Integrated with Spotify to show lyrics and background insights simultaneously.
Rich explanations and background information for a deeper understanding of music.
Genius
Extensive language support and crowd-sourced accuracy.
Requires a separate app and disrupts the listening experience by switching between platforms.
The world's largest lyrics platform, integrated with Spotify and Apple Music for real-time synced lyrics. Users can contribute by translating or editing lyrics, supporting over 62 languages.
Musixmatch
Insights from interviews
Lyrics should enhance, not interrupt
Lyrics should enhance,
not interrupt
User interviews revealed that it’s not just about translating lyrics. Listeners are looking for a deeper, more immersive music experience where lyrics add to the mood without breaking the flow. This shifted the focus to creating a seamless, visually engaging environment that lets users fully dive into the music.
User interviews revealed that it’s not just about translating lyrics. Listeners are looking for a deeper, more immersive music experience where lyrics add to the mood without breaking the flow. This shifted the focus to creating a seamless, visually engaging environment that lets users fully dive into the music.
🗣️
“When I'm curious about the meaning of lyrics while listening to foreign music, I have to leave the streaming site and search for a translation.”
“When I'm curious about the meaning of lyrics while listening to foreign music, I have to leave the streaming site and search for a translation.”
Listening Disruption
“I prefer translated lyrics that match the mood of the music, ideally presented visually, rather than simple translations from a translator.”
“I prefer translated lyrics that match the mood of the music, ideally presented visually, rather than simple translations from a translator.”
Mood Alignment
“Lyrics are secondary; feeling the atmosphere of the song is more important for fully enjoying the music.”
“Lyrics are secondary; feeling the atmosphere of the song is more important for fully enjoying the music.”
Immersive Experience
Synthesizing insights
From patterns to core ideas
From patterns to core ideas
I used an Affinity Diagram to organize and analyze user insights, which allowed me to identify key themes and brainstorm feature ideas. From this process, I derived several essential components for this service
I used an Affinity Diagram to organize and analyze user insights, which allowed me to identify key themes and brainstorm feature ideas. From this process, I derived several essential components for this service
Core Ideas
Seamless Music Experience
Features that ensure uninterrupted music flow, like integrated real-time lyric translations.
Seamless Music Experience
Features that ensure uninterrupted music flow, like integrated real-time lyric translations.
Immersive Visuals
Elements that visually connect lyrics to the music’s atmosphere, enhancing user engagement.
Immersive Visuals
Elements that visually connect lyrics to the music’s atmosphere, enhancing user engagement.
Why Apple Music on Vision Pro
Based on the insights, we determined that Apple Music would be the ideal streaming platform for integration, given its widespread use and compatibility. To deliver the immersive experience we envisioned, we chose Apple Vision Pro as our primary device, leveraging its advanced visual and spatial capabilities.
Based on the insights, we determined that Apple Music would be the ideal streaming platform for integration, given its widespread use and compatibility. To deliver the immersive experience we envisioned, we chose Apple Vision Pro as our primary device, leveraging its advanced visual and spatial capabilities.
User Scenario
In crafting this scenario, I outlined how each feature creates a seamless and immersive music experience. I detailed the journey from discovering songs through humming to engaging with real-time lyric translations and a visually rich environment.
#1 Melody Recall
#2 Humming Search
#3 Song Match
The user remembers a catchy foreign song but can't recall the title.
The user hums the melody, and the system quickly identifies the song.
The matched song is displayed, ready to be played.
#7 Chatbot Insight
#8 Deeper Context
#9 Mood Visuals
The user remembers a catchy foreign song but can't recall the title.
The chatbot provides cultural references and deeper meanings behind the lyrics.
The immersive environment enhances the song's mood with matching visuals.
#4 Playback
#5 Lyric Sync
#6 Gesture Query
The user starts listening to the song.
The lyrics appear in real-time, perfectly synchronized with the music.
Curious about a specific lyric, the user uses a hand gesture to explore further.
#10 Full Immersion
#11 Seamless Switch
#12 Experience Reflection
The user feels fully engaged, enjoying both the audio and visual experience.
The user easily switches to another song, repeating the immersive experience.
The user leaves feeling enriched, fully enjoying music from any language.
In crafting this scenario, I outlined how each feature creates a seamless and immersive music experience. I detailed the journey from discovering songs through humming to engaging with real-time lyric translations and a visually rich environment.
Mapping the ecosystem
Mapping the future of immersive music
Mapping the future of immersive music
I created this concept model to capture not only the key features I designed, such as humming-based song search, real-time lyric translation, an AI-powered chatbot, and immersive video playback, but also future possibilities, showcasing my vision for a seamless, immersive music experience with Apple Vision Pro.
I created this concept model to capture not only the key features I designed, such as humming-based song search, real-time lyric translation, an AI-powered chatbot, and immersive video playback, but also future possibilities, showcasing my vision for a seamless, immersive music experience with Apple Vision Pro.
Outlining user paths
Seamless Music Interaction Journey
Seamless Music Interaction Journey
Users can hum a melody to discover songs effortlessly, view real-time lyric translations, and dive into immersive environments that match the music’s mood. The journey connects music discovery with deeper enjoyment, making listening more engaging and seamless.
Users can hum a melody to discover songs effortlessly, view real-time lyric translations, and dive into immersive environments that match the music’s mood. The journey connects music discovery with deeper enjoyment, making listening more engaging and seamless.
usability testing & Iteration
Fixing gaps in visibility and flow
Fixing gaps in visibility
and flow
Heuristic evaluation revealed low scores in system visibility, logical flow of tasks, and visual feedback during progress. To address this, we made improvements to the chatbot and lyric selection UI to enhance the overall user experience.
Heuristic evaluation revealed low scores in system visibility, logical flow of tasks, and visual feedback during progress. To address this, we made improvements to the chatbot and lyric selection UI to enhance the overall user experience.
To-be
I added a “Edit” button that allows users to change their selected lyrics even after entering the chatbot.
To-be
As-is
Once users entered the chatbot after selecting a lyric, they could not change their selection, leading to frustration if they wanted to explore different lyrics.
As-is
Users were uncertain about how to enter the chatbot during the lyrics selection process, leading to confusion. They were unsure where to select the lyrics, which resulted in unclear navigation.
I introduced a clear instruction to guide users through the lyrics selection process. Additionally, I used a box format to highlight the selectable lyrics area and incorporated interactive elements to encourage users to make a selection.
Final Outcomes
Humming Search
Humming Search
Users can find a song just by humming its melody, even when they don’t know the title or lyrics.
Users can find a song just by humming its melody, even when they don’t know the title or lyrics.
Can’t remember the title or lyrics?
Cover one ear and hum the melody
then we’ll find the perfect match for you.
Can’t remember the title or lyrics?
Cover one ear and hum the melody then we’ll find the perfect match for you.
We provide clear and friendly guidance for using gestures to make your experience seamless.
As you hum, a sound wave visualization appears at the bottom, giving you real-time feedback on your input.
Immersive Video Playback
Immersive Video Playback
Users can listen to music in visually immersive environments that match the song’s mood or setting, enhancing the overall experience.
Users can listen to music in visually immersive environments that match the song’s mood or setting, enhancing the overall experience.
Step into a world that matches
the vibe of your music.
Experience your playlist in stunning,
immersive environments.
Step into a world that matches the vibe of your music.
Experience your playlist in stunning, immersive environments.
Preview and choose your desired virtual space to match the mood of your music.
While listening to music, clench and release your fist facing forward to enter the immersive music experience space.
Real-Time Lyric Translation
Real-Time Lyric Translation
Users can see lyrics instantly translated in real-time, allowing them to understand and enjoy foreign songs without leaving the platform.
Users can see lyrics instantly translated in real-time, allowing them to understand and enjoy foreign songs without leaving the platform.
Enjoy foreign music without language barriers.
See lyrics translated in real time,
synced perfectly with the song.
Pinch and Drag real-time translated lyrics to explore deeper meanings with our chatbot.
Enjoy foreign music without language barriers.
See lyrics translated in real time, synced perfectly with the song.
AI-Powered Chatbot
AI-Powered Chatbot
Users can ask questions about lyrics and receive cultural, contextual, or detailed explanations through an interactive chatbot.
Users can ask questions about lyrics and receive cultural, contextual, or detailed explanations through an interactive chatbot.
Curious about a lyric’s meaning or context?
Highlight the text and let our chatbot provide the answers you need.
Auto-complete suggested questions are provided for a smoother experience.
Curious about a lyric’s meaning or context?
Highlight the text and let our chatbot provide the answers you need.
Reflection
This project underscored the critical role of user research. Initially, I assumed the main problem was the inconvenience of lyric translation, but research revealed that users were more interested in a fully immersive music experience. Additionally, designing for an emerging device like Vision Pro challenged me to understand new mental models and consider unique aspects of VR interface design. It was also a valuable opportunity to deepen my understanding of Apple’s design system and how to adapt existing guidelines for a cutting-edge platform.
This project underscored the critical role of user research. Initially, I assumed the main problem was the inconvenience of lyric translation, but research revealed that users were more interested in a fully immersive music experience. Additionally, designing for an emerging device like Vision Pro challenged me to understand new mental models and consider unique aspects of VR interface design. It was also a valuable opportunity to deepen my understanding of Apple’s design system and how to adapt existing guidelines for a cutting-edge platform.