A/B Testing with Python: Analyzing Marketing Campaigns to Choose the Best Strategy

A/B testing is a fundamental technique for comparing two marketing strategies to determine which one yields higher conversion rates, sales, or other business goals. This guide walks through a complete workflow using an open‑source dataset from Kaggle.

Dataset Overview : The data contains ten features for each campaign, including campaign name, date, amount spent, number of impressions, reach, website clicks, searches received, content viewed, items added to cart, and purchases.

Setup : Install the required Python libraries.

pip install -i https://pypi.tuna.tsinghua.edu.cn/simple statsmodels
pip install -i https://pypi.tuna.tsinghua.edu.cn/simple plotly

Import and Load Data :

import pandas as pd
import datetime
from datetime import date, timedelta
import plotly.graph_objects as go
import plotly.express as px
import plotly.io as pio
pio.templates.default = "plotly_white"

pd.set_option('max_colwidth', 300)
pd.set_option('display.max_columns', None)
pd.set_option('display.max_rows', None)

control_data = pd.read_csv("control_group.csv", sep=";")
test_data = pd.read_csv("test_group.csv", sep=";")

Data Cleaning : Rename columns for consistency and fill missing values with column means.

control_data.columns = ["Campaign Name", "Date", "Amount Spent", "Number of Impressions", "Reach", "Website Clicks", "Searches Received", "Content Viewed", "Added to Cart", "Purchases"]
test_data.columns = ["Campaign Name", "Date", "Amount Spent", "Number of Impressions", "Reach", "Website Clicks", "Searches Received", "Content Viewed", "Added to Cart", "Purchases"]

for col in ["Number of Impressions", "Reach", "Website Clicks", "Searches Received", "Content Viewed", "Added to Cart", "Purchases"]:
    control_data[col].fillna(value=control_data[col].mean(), inplace=True)
    test_data[col].fillna(value=test_data[col].mean(), inplace=True)

Merge Datasets and verify balanced sample sizes.

ab_data = control_data.merge(test_data, how="outer").sort_values(["Date"]).reset_index(drop=True)
print(ab_data["Campaign Name"].value_counts())

Exploratory Analysis : Visualize relationships and compare metrics between the control and test campaigns using Plotly.

Impressions vs. Amount Spent

Total Searches

Website Clicks

Content Viewed

Added to Cart

Amount Spent

Purchases

Example scatter plot for impressions and spend:

figure = px.scatter(data_frame=ab_data,
                    x="Number of Impressions",
                    y="Amount Spent",
                    size="Amount Spent",
                    color="Campaign Name",
                    trendline="ols")
figure.show()

Pie charts are used to compare totals for searches, clicks, content views, add‑to‑cart actions, spend, and purchases between the two campaigns.

Key Findings :

The control campaign generates more impressions for the same spend.

Test campaign yields slightly higher search volume and website clicks.

Control campaign has higher content‑view counts and more items added to cart.

Spending is higher in the test campaign, but purchase counts differ by only about 1%.

Conversion rate (purchases per add‑to‑cart) is higher in the test campaign.

Overall, the control campaign delivers more overall engagement and sales with lower cost, while the test campaign shows a higher conversion efficiency. Depending on business goals—broad reach versus higher conversion—different strategies may be preferred.

Conclusion : A/B testing reveals that the control campaign is more effective for driving overall sales and user engagement, whereas the test campaign can be leveraged for targeted audiences where conversion efficiency is critical.