Hands-on with Gemini-powered Google smart glasses, Android XR, and Project Moohan headset

Naturally, you can work in a mixed reality environment with a Bluetooth-connected keyboard and mouse, and you can put yourself in an immersive environment if you want to focus, or leave vision mode on to make sure your co-workers aren’t. Taking pictures and laughing while wearing a silly headset to get things done. It was not clear whether you can connect the headset to a laptop to bring your work into mixed reality, a feature available in the Apple Vision Pro.

Gemini in XR

Tapping the side of the headset brings up the app launcher, and this is where you can switch to the Gemini if ​​you want it to stay constantly ‘on’. Once turned on, there is an icon at the top of the virtual space so you are aware that everything you say and look at is being recorded by Gemini.

In Transparent mode, you can walk up to something and ask Gemini about it — as one Googler demoed the headset (before I tried it) walked up to another person wearing a FC Barcelona shirt and asked Gemini to find the “ranking of this team.” Gemini quickly registers the team name and displays search results with league standings and results of recent matches.

You can ask Gemini anything like this and he will answer with visual results displayed in the headset. I asked him to “take me to Peru,” and he opened up a 3D version of Google Maps for me. I was able to navigate and position myself in Lima, and in cities where the Maps app already has a lot of 3D models, you can explore the areas in more detail. You can keep talking to Gemini on these experiences, so I asked questions like when would be the best time to visit and got a quick answer.

In another example, I peeked inside a restaurant in New York City for a virtual tour of the place. Google says it can use artificial intelligence to collect interior images of a place and display them so you feel like you’re there. He did a very good job, and I asked Jiminy if the place took reservations, without having to say the name specifically, because I was staring at the name of the restaurant. He – she He does Make a reservation, but Gemini couldn’t make a reservation for me. (This integration may come later.)

Then, I watched some videos on YouTube, where the 2D content looks sharp and colorful. The stereoscopic content was better. I felt my senses surrounded. I saw some hikers walking along the path and asked Jiminy where it was all, and he said, “New Zealand.” I haven’t been able to verify this, but it seems to be the correct answer. I’ve seen more spatial playbacks of 2D videos where the default player added depth and layers to make them look 3D. I went into the Google TV app and enabled Cinema Mode to launch a virtual theater for watching movies and shows, just like on other VR headsets.

Research circlea feature that Google debuted earlier this year On Android phoneswhich is also available in Android XR. All you have to do is walk up to a physical object near you, press the top button on the headset, then pinch your fingers and draw a circle around the thing you want to learn more about. You will get a Google search page with results.

Smart glasses

Moohan’s project looks very much like Google and Samsung are catching up with the rest of the VR market, though the Gemini integration gives their efforts a unique layer. However, I will admit that I was more excited to try out the smart glasses, as Gemini feels they could be more useful. You were not disappointed. I walked into another room and had several pairs of glasses in front of me. Some were sunglasses, others had clear lenses. Like a headset, you can load it with your prescription. Google did not provide a name for the prototype glasses.

The glasses, whose name has not yet been revealed, will come in clear and colored lens options.

Courtesy of Google