Customer Intelligence & Segmentation Platform
A full-stack retail data workflow turning 8.5M+ transactions into reusable inputs for segmentation and marketing intelligence.
The Data Problem
The Operating Context
"The business struggled to operationalize segmentation. Static dashboarding wasn't enough; marketers needed a consistent data workflow that could prepare cohorts and support downstream clustering without waiting for Data Science."
Marketing teams were dependent on data requests for every campaign. We needed a repeatable processing workflow that could aggregate large transaction volumes in ClickHouse and hand prepared features to downstream segmentation without exposing code.
- The Reliability Trade-offs: Each workflow balanced data preparation effort, downstream usefulness, and measurable reliability under practical constraints.
- Constraint 01: Non-Technical End Users: The workflow had to abstract algorithm tuning while keeping upstream transformations and downstream Silhouette Scores traceable.
- Constraint 02: CPU-Aware Processing: Pushed heavy aggregations down to ClickHouse and kept only prepared ML arrays in memory for the interactive Streamlit surface.
The Data Flow
Pipeline Rationale
Data Flow
Built a stateful Streamlit workflow that reads prepared retail data from ClickHouse, applies RFM and market-basket transformations, and passes the resulting features to Scikit-learn and Apriori components.
Downstream Outcome
Provided an interactive A/B testing simulator and ABC-XYZ product matrix from prepared retail data, extending reporting into proactive planning.
Data engine
ClickHouse columnar database for aggregating 8.5M+ retail interactions before feature delivery.
Downstream analysis
Scikit-learn K-Means (RFM) and Apriori (Market Basket) rule mining.
Validation Results
Cohort exploration: Reduces manual data requests by making prepared segments available for interactive exploration.
Bundle output: Identifies specific cross-selling pairs (Support & Lift) across 70K+ SKUs dynamically.
Planning output: Extends prepared customer and product data into a real-time ABC-XYZ matrix.
Pipeline Delivery & Reflections
Feature preparation: RFM and transaction transformations establish the inputs that K-Means/MiniBatch can evaluate against Silhouette Scores before dashboard delivery.
Workflow state: Engineered Streamlit caching and session keys to persist prepared cohort filters across multiple application pages.
Business delivery: Used prepared segments to power a dedicated A/B testing simulator that projects actual revenue impact.