Back to selected work

Multimodal XR.

A multimodal XR prototype bridging physical controls, voice, and gesture tracking with a real-time digital twin to create natural ambient interfaces.

The experience.

What if your next interface wasn’t confined to a screen, but responded to natural gestures, voice, and physical touch? Lights That Listen links physical controls, audio, and gesture tracking with a live digital twin in Unity.

Developed for User Experience Design of XR (DES527), the prototype explores ambient multimodal interaction - turning hand movements, voice commands, and tactile inputs into immediate, synchronized responses.

PROJECT DEMO

In motion.