A walkthrough on public data of the same basket-analysis methodology I run with enterprise retail clients. Methodology is real. Numbers are real model output from this dataset; in production deployments the absolute lift is larger because the assortment is bigger and the floor moves are tighter.
The Challenge
Modern grocery stores carry tens of thousands of SKUs across a footprint laid out for receiving efficiency, not for the customer. The default shelf plan is by supplier or by category code. Both ignore the question that actually drives baskets: what do shoppers buy together, and how do we put those things within sight of each other?
When complementary products sit on opposite ends of the store, the pickup rate on the second item collapses. A tired, confused customer often buys less than they intended. Promotions discount the wrong items — single-product deals when the real lever is a pair. The merchandising team has the data to fix this. They usually don't have the model.
The job: take a transaction log of 9,835 shopping baskets across 169 product categories — no purchase amounts, no customer IDs, just baskets — and surface a small, defensible set of rules the floor team can actually run on.
The Approach
I ran the data through four steps. Each one is about earning the right to act on the next.
1. Profile the assortment. Before mining rules, map what's actually selling. Whole milk shows up in 25.6% of baskets — it's the anchor item. Other vegetables (19.3%), rolls/buns (18.4%), soda (17.4%), and yogurt (14.0%) round out the top five. These are the items every adjacency analysis will eventually point back to. Knowing their support upfront tells you which rules are going to be load-bearing.
2. Pivot transactions into the basket matrix. Raw log is one row per line item. Apriori needs one row per basket, with a binary column per item. mlxtend.preprocessing.TransactionEncoder does this in one call — 9,835 baskets × 169 categories.
3. Mine frequent itemsets with the right thresholds. I ran Apriori at minimum support 0.02 (the itemset appears in at least 2% of baskets) and derived association rules at minimum confidence 0.35 (the consequent appears in at least 35% of baskets containing the antecedent). The result: 122 frequent itemsets and 26 rules above the action threshold. Lower the thresholds and the rule list explodes into noise; raise them and you miss the long-tail co-purchases that matter for endcap planning. The right thresholds are the conversation between the model and the merchandising team.
4. Sort by lift, not by confidence. Confidence alone rewards rules whose consequent is already popular. Lift normalises against base rate — it's the multiplier on chance. A rule with confidence 50% and lift 2.0 means "customers who buy A buy B at twice the rate of the average shopper." That's the number the floor team will defend in a meeting.
The Outcome
Six high-confidence, high-lift rules anchor the action set, and they cluster around three themes:
- The whole-milk gravity well. Whole milk is in a quarter of all baskets. Rules ending in whole milk (butter → whole milk, curd → whole milk, yogurt + other vegetables → whole milk) hit confidence 49–51% and lift ~1.9–2.0. Whatever is co-located with whole milk gets dragged along.
- The vegetable cluster. Root vegetables + whole milk → other vegetables is the strongest rule in the dataset (confidence 47%, lift 2.45). Vegetable items cross-recommend each other at 2× the random rate. This is the merchandising opportunity hidden by the legacy "produce in one corner" layout.
- The dairy-bakery anchor. Yogurt, curd, and butter form a tight cluster against milk and other vegetables. The pattern argues for a continuous shopper path: produce → dairy → bakery, not three disjoint zones.
Those rules map to three operational decisions a merchandising team would actually run:
- Shelf adjacency. Co-locate whole milk, bread rolls, yogurt, and the root-vegetable / other-vegetable cluster with deliberate sight-lines to one another. Replace the legacy "by supplier" layout where it conflicts.
- Endcap inventory. Categories that never appear in high-confidence rules — the dataset has dozens — demoted off prime endcap real estate. Reclaim the space for items that actually drive basket size.
- Cross-promo bundles. The top five rules seed promotional bundles, so discounts pull both items rather than cannibalising margin on one. A "milk + cereal" promo isn't a discount — it's an attachment play.
Business Impact
In production, this analysis gets rerun quarterly, because the assortment, promo calendar, and seasonality shift the rule list. The model earns its keep when the team looks at which rules disappeared and which new ones surfaced — those transitions tell you the floor is changing under you.
The pattern the project crystallises: Apriori is a 30-year-old algorithm. On the right business question, it still beats intuition. Cheap unsupervised models on the right data move the P&L further than expensive supervised models on the wrong data.
Key Takeaways
- Lift is the metric, not confidence. Confidence rewards popularity; lift rewards genuine association above base rate.
- The threshold conversation is the project. Min-support and min-confidence aren't hyperparameters to tune — they're the boundary between "noise" and "actionable" that the merchandising team has to own.
- Read the rules into a floor walk. The best output isn't a rule list; it's a redrawn aisle map that visibly explains why it changed.
- Apriori is the cheap MVP. If it surfaces six rules with lift > 1.8, you've earned the right to graduate to FP-Growth or sequence mining. If it doesn't, no fancier model will help.