Company Updates
Preview: Envisioning the Future of Grocery Shopping with GPT-4V
AI has long been at the heart of everything we do, and we’re always looking to partner with AI industry leaders and leverage the latest AI advancements to deepen the relationship people have with food and grocery shopping. Our early access and adoption of the new vision-enabled GPT-4 Turbo with Vision (GPT-4V) is the latest such example. Today, we’re sharing a preview of what we’re cooking up next: upgrading our existing AI-powered Ask Instacart search feature with this new vision capability. Soon, in addition to being able to ask food-related questions in natural language, Instacart users will be able to use Ask Instacart to convert handwritten recipes and shopping lists directly into digital, shoppable item lists in the Instacart app. The vision-enabled Ask Instacart will decipher ingredients and quantities in a single step so that our users don’t have to do the mental translation and manually look for one item at a time.
Here’s how it will work:
- 📸 Snap: Simply open the photo feature in your Instacart app and snap a photo of your list– perhaps it’s a family recipe for Thanksgiving apple pie lovingly handwritten by Grandma, or a scribbled shopping list of groceries you’ve jotted down throughout the week. Don’t worry if the handwriting isn’t the cleanest; GPT-4V’s language capability takes into account context from other legible words to help clarify the meaning of any item that isn’t clear.
- ☑️ Select: Thanks to GPT-4V, the Instacart app will analyze the image, recognize the text, and convert it into a comprehensive, shoppable list of items. From there, you can select specific ingredients you need, or de-select any ingredients you might already have at home– like cinnamon, in the case of Grandma’s apple pie.
- 🥧 Savor: With your new digital shopping list, you can instantly place an order and get all of the ingredients or grocery supplies you need delivered straight to your door in as fast as an hour– leaving you more time to savor the fruits of your efforts.
We hope to launch this feature in the Instacart app to a small number of customers in the next few weeks and more widely in the coming months. This feature is the latest in a set of generative AI-enabled shopping experiences we’re building to help people more easily answer the eternal, “What’s for dinner?” question. From the Instacart plugin for ChatGPT to Ask Instacart, which lets people ask open-ended food questions in natural language– like, “What are dairy-free snacks for kids?” or even, “What should I make for dinner if I have kale and cheddar in my fridge?”–we believe that AI can help us create personalized, inspirational, value-driven shopping experiences that enrich peoples’ relationship with food and the retailers and brands they love. Now with our early adoption of GPT-4V, this new vision-enabled feature promises to be a big help to anyone following a recipe or looking to save some time tackling their grocery shopping list.
Most Recent in Company Updates
Company Updates
Instacart’s Cornucopia of Thanksgiving and Winter Holiday Insights
As the air turns crisp and the anticipation of holiday gatherings begin, it's time to rediscover the foods and festivities that define the season! According to a recent Instacart survey of over 2,000 U.S. adults…...
Oct 24, 2024Company Updates
Instacart Expands Presence in Canada with Addition of Sobeys, FreshCo, Farm Boy, and Longo’s on the Instacart App
Customers in Ontario can now find their favorite Empire Company Limited banner stores, including Sobeys, FreshCo, Farm Boy, and Longo’s, on the Instacart App for same-day delivery in as fast as an hour. Rolling out…...
Oct 24, 2024Company Updates
Joining Our Table: Meet Chief Technology Officer Anirban Kundu
We are thrilled to announce Anirban Kundu will be joining us as our new Chief Technology Officer! Anirban joins us from Uber, where he was VP and Head of Engineering for Uber Delivery. At Uber,…...
Oct 16, 2024