MSX: Designing Future In-Car Experiences for the Autonomous Era

The MSX project, developed by students at Carnegie Mellon's Entertainment Technology Center in partnership with Honda's Auto Development Center, HDMA and 99P Labs, reimagines the future of autonomous vehicles as immersive “third spaces” for families. With a target year of 2040, the team designed experiences for fully self-driving cars that enable social interaction, learning, and entertainment. By integrating mixed reality (MR) technologies, the goal was to transform car travel from idle time into shared, meaningful moments.

‍

The team began by analyzing how vehicles have evolved into important personal spaces post-COVID. Drawing from this trend, they explored how extended reality (XR) could reshape mobility. Their research spanned self-driving technologies, flexible seating layouts, MR headsets, spatial computing, and AI. They evaluated how future car interiors could become adaptive environments—complete with gesture-based interfaces, synchronized MR content, and intelligent, personalized assistants.

‍

Grounded in user research, the team interviewed parents and children to understand current habits and pain points of in-car family travel. They found that many parents already view car rides as opportunities to bond, play, or learn together. Insights from these studies directly shaped their design direction—centering experiences around face-to-face interaction, shared storytelling, and passive engagement that complements the driving environment.

‍

The team’s prototype centers on a 2-hour drive to Yellowstone National Park, focusing on the arrival phase. Through Apple Vision Pro headsets, passengers interact with a geyser simulation and a virtual bison, blending narrative, sound design, and visual effects. These interactions support both playful discovery and educational engagement, demonstrating how MR can enrich travel for all ages.

‍

MSX was built using Apple’s Reality Composer Pro and Swift, leveraging spatial awareness and hand tracking for intuitive interaction. Multiplayer features were enabled via Firebase Realtime Database, allowing shared experiences across multiple headsets. Custom art assets, shaders, and particle systems were carefully optimized for headset performance. Audio design, narration, and AI voice tools like Eleven Labs further enhanced immersion.

‍

After multiple rounds of playtesting with faculty and families, the team refined their concept based on real feedback. Participants showed enthusiasm for the future of self-driving cars and XR entertainment, especially among younger users. The final prototype was presented to Honda 99P Labs, with discussions around real-world feasibility, user adoption, and market fit. The project showcased how entertainment design can anticipate and shape tomorrow’s mobility experiences—by focusing on people, not just technology.