Apple’s upcoming Vision Pro headset will feature a new augmented reality (AR) version of the Visual Lookup feature found in iPhones and iPads. This feature, dubbed “Visual Search,” will allow wearers to get information about various items by simply looking at them.
The feature was discovered by MacRumors contributor Steve Moser, who inspected code within the Xcode beta of visionOS, Apple’s AR and reality operating system that will power the Vision Pro headset. Moser found references to Visual Search in the code, which suggests that the feature is still under development.
Visual Search is expected to work by using the Vision Pro headset’s cameras to scan the environment and identify objects. Once an object is identified, the headset will then display information about the object, such as its name, description, and price.
Visual Search could be a powerful tool for users who want to learn more about the world around them. For example, users could use Visual Search to identify plants and animals, learn about historical landmarks, or even find product information while shopping.
It looks like ‘Visual Search’ is part of Spotlight instead of the camera. I wish Apple had shown off this feature and ARKit during the keynote. Related accessibility feature: “”Describe Passthrough” requires Image Captions to be enabled in Settings Accessibility > VoiceOver.”
With Visual Search, users can use the Vision Pro headset to get information about an item, detect and interact with text in the world around them, copy and paste printed text from the real world into apps, translate text between 17 different languages, and more.
Real world text that includes contact information, webpages, and unit conversions and similar information can be acted upon in visionOS. So, for example, if a printed handout has a website link in it, you can scan the link with the Vision Pro, opening up a Safari window to view the website. Or, if a recipe calls for grams and you need ounces, you can convert using the headset.
MacDailyNews Take: Ever been to a baseball game and seen some guys listening to the radio broadcast wearing headphones during it?
Please help support MacDailyNews. Click or tap here to support our independent tech blog. Thank you!
Support MacDailyNews at no extra cost to you by using this link to shop at Amazon.
The post Apple Vision Pro will come with a super-powered Visual Search feature appeared first on MacDailyNews.
