The story behind an Instacart order, part 3: predicting the shop

This is the third installment of our “Story Behind an Order” series, where we’re walking you through the technology that surfaces grocery items in our apps, enables storefront browsing, facilitates communication before and during the shop, and powers our last-mile logistics to get dinner to your door in as fast as an hour. If you haven’t already, make sure to check out Part One to see how we gather and normalize our grocery catalog data, and Part Two, which covers the tech behind storefront browsing and buying.

In our last installment, we learned about the tech that makes it easy to find and buy the ingredients for your favorite weeknight lemony herbed salmon. In this third installment, we’ll concentrate on the predictive models that help us determine if your Meyer Lemons, Fresh Dill, and Salmon Fillets are in stock. We’re also taking a look at the machine learning model that makes it easier for customers to choose and communicate their preferred item replacements.

Training Data

Our catalog data is the technical foundation of our four-sided marketplace.

We have about five million unique products in our catalog and about 950,000,000 product listings in total. Each of these listings has a name, a product ID, and several sub-attributes like departments, aisle numbers, dietary/cuisine tags, and nutritional information. We also rely on historical data, detailing how products behave in the marketplace. This data includes item availability history, the number of times an item has been chosen as a replacement, and more.

Together, this data powers many of our machine learning models. Two models in particular work-hand-in-hand to ensure a smooth customer and shopper experience: our Item Availability Model and our Replacement Recommendation Model.

Pinpointing item availability

We predict the availability of over 500 million listings every 60 minutes.

Our Item Availability Model relies on historical retailer availability data, store location, an item’s purchase history, and shopper inputs to predict the likelihood that a particular item in our catalog is or isn’t in stock at any one of nearly 25,000 physical stores.

This is really hard to do. We get a sense of an item’s availability about once a day from our retail partners, but as we all know availability can be extremely variable throughout the day. One data drop a day doesn’t give us the hour-by-hour predictions we need to set expectations appropriately…especially for a harder-to-come-by fruit variety like the Meyer Lemon! Some locations may get new shipments from their growers seasonally. And when the fruit is in season, some store locations may only restock lemons in the mornings, while others may be a bit busier and stock produce section multiple times per day.

To understand variability throughout the day, we’ve built a model that looks at time-centric data features — notably the time of day and the day of the week that the item has been picked by a shopper in-store — to give each listing an availability score.

Here’s a look at how the availability score for Meyer Lemons may change throughout the day:

Surfacing the right replacements

Typically a customer shopping for themselves will replace 7% of the items they’re looking for.

Sadly, the Meyer Lemons from our favorite mom and pop retailer received a low availability score. When this happens, our Replacement Recommendation model takes over, firing up a flow in our customer-facing app asking customers to pre-select a replacement as they browse.

What makes replacements particularly hard to predict is the vast array of contexts in which our customers purchase items. For example, if we’re hoping to squeeze lemon juice over our salmon to flavor it we can opt for lemon juice as a replacement, but if we’re hoping to garnish the salmon with a lemon slice, we may want an organic lemon.

Our current model relies on a combination of categorical data like product names and an item’s replacement history to draw invisible lines between related item listings and stack rank potential replacements.

We give historical data like replacement history a lot more weight than categorical data when scoring for a few reasons. Historical data helps us find items that seem like atypical replacements when viewed through the lens of categorical data (aisle, department, name) but are perfectly normal in the human brain. In the case of Meyer Lemons, customers usually pick Meyer Lemon Juice as a replacement which usually doesn’t live in the produce section. By giving more weight in our scoring to historical shopper and customer preferences, we can see that the juice is much more widely preferred, despite products like Ms. Meyer’s Lemon Soap having a strong name match.

Should a customer opt not to select a replacement in-app and leave it up to the shopper (which happens approximately half the time) the replacement model will surface a few of the top historical replacements into the app flow for shoppers to pick from.

Smoothing out potential roadblocks like out of stock items is critical for the success of our fulfillment chain. These two predictive models have worked hand-in-hand to set availability expectations for one listing’s availability and give us the opportunity to quickly and easily communicate the context of an order to a shopper. Stay tuned for the last installment of this series, where we’ll dig into the tech that guides a shopper through the store and right to your door.

Can’t get enough of data science, mobile development, and machine learning? Our Engineering and Product teams are hiring! Check out our current openings.

*Learn more about Engineering at Instacart on our Tech blog.

Instacart

Author

Instacart is the leading grocery technology company in North America, partnering with more than 1,400 national, regional, and local retail banners to deliver from more than 80,000 stores across more than 14,000 cities in North America. To read more Instacart posts, you can browse the company blog or search by keyword using the search bar at the top of the page.