Instacart Market Basket Analysis
My solution for the Instacart Market Basket Analysis competition hosted on Kaggle.
The dataset is an open-source dataset provided by Instacart (source)
This anonymized dataset contains a sample of over 3 million grocery orders from more than 200,000 Instacart users. For each user, we provide between 4 and 100 of their orders, with the sequence of products purchased in each order. We also provide the week and hour of day the order was placed, and a relative measure of time between orders.
Below is the full data schema (source)
orders(3.4m rows, 206k users):
order_id: order identifier
user_id: customer identifier
eval_set: which evaluation set this order belongs in (see
order_number: the order sequence number for this user (1 = first, n = nth)
order_dow: the day of the week the order was placed on
order_hour_of_day: the hour of the day the order was placed on
days_since_prior: days since the last order, capped at 30 (with NAs for
product_id: product identifier
product_name: name of the product
aisle_id: foreign key
department_id: foreign key
aisle_id: aisle identifier
aisle: the name of the aisle
department_id: department identifier
department: the name of the department
order_id: foreign key
product_id: foreign key
add_to_cart_order: order in which each product was added to cart
reordered: 1 if this product has been ordered by this user in the past, 0 otherwise
SETis one of the four following evaluation sets (
"prior": orders prior to that users most recent order (~3.2m orders)
"train": training data supplied to participants (~131k orders)
"test": test data reserved for machine learning competitions (~75k orders)
The task is to predict which products a user will reorder in their next order. The evaluation metric is the F1-score between the set of predicted products and the set of true products.
The task was reformulated as a binary prediction task: Given a user, a product, and the user's prior purchase history, predict whether or not the given product will be reordered in the user's next order. In short, the approach was to fit a variety of generative models to the prior data and use the internal representations from these models as features to second-level models.
The first-level models vary in their inputs, architectures, and objectives, resulting in a diverse set of representations.
- Product RNN/CNN (code): a combined RNN and CNN trained to predict the probability that a user will order a product at each timestep. The RNN is a single-layer LSTM and the CNN is a 6-layer causal CNN with dilated convolutions.
- Aisle RNN (code): an RNN similar to the first model, but trained at the aisle level (predict whether a user purchases any products from a given aisle at each timestep).
- Department RNN (code): an RNN trained at the department level.
- Product RNN mixture model (code): an RNN similar to the first model, but instead trained to maximize the likelihood of a bernoulli mixture model.
- Order size RNN (code): an RNN trained to predict the next order size, minimizing RMSE.
- Order size RNN mixture model (code): an RNN trained to predict the next order size, maximizing the likelihood of a gaussian mixture model.
- Skip-Gram with Negative Sampling (SGNS) (code): SGNS trained on sequences of ordered products.
- Non-Negative Matrix Factorization (NNMF) (code): NNMF trained on a matrix of user-product order counts.
The second-level models use the internal representations from the first-level models as features.
The final reorder probabilities are a weighted average of the outputs from the second-level models. The final basket is chosen by using these probabilities and choosing the product subset with maximum expected F1-score.
64 GB RAM and 12 GB GPU (recommended), Python 2.7