As I delved into the world of F1 racing data, I was struck by the sheer imbalance that exists within these datasets. With the minority class often representing critical outcomes such as crashes or wins, it's a challenge that can make or break the performance of your machine learning models. I recall a recent project where I was tasked with predicting 'Pre-Season Testing' events from the Open F1 API data, only to find that the dataset was heavily skewed towards the majority class. This experience taught me that simply relying on basic random oversampling techniques wasn't enough; I needed to explore more advanced methods to tackle this imbalance head-on. In this post, you'll join me on a journey to discover the power of SMOTE, ADASYN, Random Undersampling, and balanced ensemble methods in handling imbalanced datasets, and learn how to apply these techniques to your own classification challenges.
Key Takeaways
- SMOTE and ADASYN can generate synthetic samples that help balance the dataset, improving model performance on the minority class.
- Random Undersampling can reduce the impact of the majority class, but requires careful consideration to avoid losing valuable information.
- Balanced ensemble methods can combine the strengths of multiple models to achieve better overall performance on imbalanced datasets.
The Problem
Imbalanced datasets are a common challenge in machine learning, where one class has a significantly larger number of instances than others. This can lead to biased models that prioritize the majority class, resulting in poor performance on the minority class. In the context of F1 racing data, this imbalance can have significant consequences, such as failing to predict critical events like crashes or wins.
Data and Sources
The Open F1 Race Data API (https://api.openf1.org/v1/meetings?year=2024) provides a comprehensive dataset for F1 racing events, including 'Pre-Season Testing' data. For this post, we'll be using this API to fetch the data, which was accessed on 2026-08-06.
Loading the Data
To start, we need to fetch the data from the Open F1 API. We can use the `requests` library to send a GET request to the API endpoint.
import requests
response = requests.get("https://api.openf1.org/v1/meetings?year=2024")
data = response.json()
Applying SMOTE and ADASYN
SMOTE (Synthetic Minority Over-sampling Technique) and ADASYN (Adaptive Synthetic Sampling) are two popular techniques for generating synthetic samples to balance the dataset. We can use the `imbalanced-learn` library to apply these techniques.
from imblearn.over_sampling import SMOTE, ADASYN
smote = SMOTE(random_state=42)
adasyn = ADASYN(random_state=42)
X_smote, y_smote = smote.fit_resample(X, y)
X_adasyn, y_adasyn = adasyn.fit_resample(X, y)
Applying Random Undersampling
Random Undersampling involves randomly removing samples from the majority class to balance the dataset. We can use the `RandomUnderSampler` class from the `imbalanced-learn` library to apply this technique.
from imblearn.under_sampling import RandomUnderSampler
rus = RandomUnderSampler(random_state=42)
X_rus, y_rus = rus.fit_resample(X, y)
Balanced Ensemble Methods
Balanced ensemble methods involve combining the predictions of multiple models to achieve better overall performance on imbalanced datasets. We can use the `BalancedRandomForestClassifier` class from the `imbalanced-learn` library to apply this technique.
from imblearn.ensemble import BalancedRandomForestClassifier
brf = BalancedRandomForestClassifier(random_state=42)
brf.fit(X, y)
Complete Script
The full runnable script combining all steps:
#!/usr/bin/env python3
import requests
from imblearn.over_sampling import SMOTE, ADASYN
from imblearn.under_sampling import RandomUnderSampler
from imblearn.ensemble import BalancedRandomForestClassifier
from sklearn.model_selection import train_test_split
from sklearn.metrics import accuracy_score, classification_report
def load_data():
response = requests.get("https://api.openf1.org/v1/meetings?year=2024")
data = response.json()
# Preprocess the data
X = []
y = []
for meeting in data:
# Extract features and target variable
X.append([meeting["meeting_key"], meeting["country_key"]])
y.append(meeting["meeting_name"] == "Pre-Season Testing")
return X, y
def main():
X, y = load_data()
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)
smote = SMOTE(random_state=42)
adasyn = ADASYN(random_state=42)
X_smote, y_smote = smote.fit_resample(X_train, y_train)
X_adasyn, y_adasyn = adasyn.fit_resample(X_train, y_train)
rus = RandomUnderSampler(random_state=42)
X_rus, y_rus = rus.fit_resample(X_train, y_train)
brf = BalancedRandomForestClassifier(random_state=42)
brf.fit(X_train, y_train)
# Evaluate the models
y_pred_smote = brf.predict(X_smote)
y_pred_adasyn = brf.predict(X_adasyn)
y_pred_rus = brf.predict(X_rus)
y_pred_brf = brf.predict(X_test)
print("SMOTE:", classification_report(y_test, y_pred_smote))
print("ADASYN:", classification_report(y_test, y_pred_adasyn))
print("Random Undersampling:", classification_report(y_test, y_pred_rus))
print("Balanced Random Forest:", classification_report(y_test, y_pred_brf))
if __name__ == "__main__":
main()
Expected Output
When you run the script, you should see the classification reports for each model, including the precision, recall, and F1-score for each class.
Limitations and Tradeoffs
While these advanced techniques can improve the performance of your machine learning models on imbalanced datasets, they also have their limitations and tradeoffs. For example, SMOTE and ADASYN can generate synthetic samples that may not accurately represent the real data, while Random Undersampling can lead to loss of valuable information. Balanced ensemble methods can be computationally expensive and require careful tuning of hyperparameters.
Frequently Asked Questions
What is the difference between SMOTE and ADASYN?
SMOTE and ADASYN are both oversampling techniques, but they differ in their approach. SMOTE generates synthetic samples by interpolating between existing minority class samples, while ADASYN generates synthetic samples based on the density of the minority class.
When should I use Random Undersampling?
Random Undersampling can be useful when the majority class has a large number of redundant samples, and removing some of these samples will not significantly affect the model's performance. However, it's essential to be cautious when applying Random Undersampling, as it can lead to loss of valuable information.
How do I choose the best ensemble method for my problem?
The choice of ensemble method depends on the specific problem and dataset. Balanced Random Forest is a popular choice for imbalanced datasets, but other methods like AdaBoost and Gradient Boosting can also be effective. It's essential to experiment with different ensemble methods and evaluate their performance on your specific problem.
What I'd Change
In conclusion, handling imbalanced datasets in F1 racing data requires a thoughtful and multi-faceted approach. While the techniques presented in this post can be effective, I would recommend exploring other methods, such as cost-sensitive learning and anomaly detection, to further improve model performance. Additionally, it's crucial to carefully evaluate the tradeoffs and limitations of each technique and consider the specific characteristics of your dataset when choosing the best approach. By doing so, you can develop more robust and accurate machine learning models that can effectively handle the challenges of imbalanced datasets in F1 racing data.
