The goal of this project was to create a showcase of AI technologies in VR. This project marks the evolution of a previous prototype developed more than a year ago, during the early rise of Generative AI. You can view our initial prototype here. Unfortunately, the platform we utilized back then, Mozilla Hubs, is now shutting down.
With the release of the Meta Quest 3 boasting enhanced passthrough capabilities, we saw the opportunity to explore how we could integrate AI technology directly into VR, specifically in Mixed Reality environments.
For this we utilized ShapesXR, a collaborative VR modeling application designed for users to seamlessly share a virtual space, interact with each other, and access the same models in real-time. Primarily utilized for collaborative storyboarding, prototyping, and industrial design, ShapesXR offers an immersive experience.
Within the app, users have the flexibility to create their own models or import them in obj, GLB, and glTF formats. Additionally, the app supports the import of png or jpg images, with a file size limit of 50 MB per file. In our project, we also delved into the Figma integration, which proved to be very useful for designing UI elements that can be easily modified and updated with just one click.
We designed a series of interactive scenes in ShapesXR, seamlessly blending virtual objects with real furniture. When experienced through the Quest 3 headset, the Mixed Reality (MR) experience offers a sense of magic, as physical and virtual assets intertwine. We focused on creating assets using various AI 3D tools, including text to image textures, text to voice, text to 3D models, and text to 360 panoramas.
You can watch a video showcasing our project (video link). And you can also visit the space using this link: https://space.shapes.app/space/share/t8g5x9qsfmrll3rbws98xipxylklvsc5snvfwjv-ira-/0/0
If you have a compatible VR headset, we invite you to immerse yourself in the experience by downloading ShapesXR and joining the room with the 74eeeer5 code.
While we’ve provided a glimpse into our project through this blog post, we encourage you to explore the full experience yourself. Dive into our collaborative VR world and discover the intersection of AI and immersive technologies.
This video was created for the WACC Canvas Conference held in Tacoma, WA. 2024.
Last Updated April 29, 2024