Have you ever struggled to create accurate time series forecasts from complex, real-world data sources like external APIs? I certainly have. When working with the Random User API, I found that transforming discrete, non-temporal data into a robust time series forecast was a daunting task. However, after experimenting with different approaches, I discovered that strategically combining Facebook Prophet and the statistical rigor of `statsmodels` can yield highly accurate and robust results. In this post, I'll share my experience and provide a step-by-step guide on how to create accurate time series forecasts using these two powerful tools.
Key Takeaways
- Synthetic time series can be effectively constructed from discrete event data by timestamping and aggregation, making non-temporal APIs useful for forecasting.
- Prophet excels at capturing complex seasonality and trends with minimal configuration, making it a strong baseline for real-world data.
- Statsmodels (specifically SARIMAX) provides granular control over model parameters, allowing for fine-tuning and handling of edge cases.
- Combining Prophet and statsmodels enables data scientists to leverage the strengths of both approaches, resulting in more accurate and robust forecasts.
- Real-world data sources, such as the Random User API, can be used to generate synthetic time series data for forecasting purposes.
The Problem
When working with real-world data sources, such as external APIs, creating accurate time series forecasts can be challenging due to the presence of non-linear trends, multiple seasonalities, and missing observations. The Random User API, for example, provides discrete, non-temporal data that requires transformation into a time series format before forecasting can be applied.
Data and Sources
The Random User API (https://randomuser.me/api/) is used to generate a synthetic time series dataset. This API provides a stream of individual user profiles, which can be timestamped and aggregated to create a time series. The data is accessed on 2026-07-10, and the API documentation can be found at https://randomuser.me/documentation.
Loading the Data
To load the data, we use the `requests` library to send a GET request to the Random User API. The response is then parsed as JSON, and the data is extracted and stored in a Pandas DataFrame.
import requests
import pandas as pd
response = requests.get("https://randomuser.me/api/")
data = response.json()
df = pd.DataFrame(data["results"])
The Core Logic
The core logic involves using Prophet to capture complex seasonality and trends, and then fine-tuning the model using statsmodels. We start by creating a Prophet model and fitting it to the data.
from prophet import Prophet
model = Prophet()
model.fit(df)
Putting It Together
We then use the Prophet model to make predictions and evaluate its performance using metrics such as mean absolute error (MAE) and mean squared error (MSE). We also use statsmodels to fine-tune the model and handle edge cases.
from statsmodels.tsa.statespace.sarimax import SARIMAX
# Make predictions using Prophet
forecast = model.make_future_dataframe(periods=30)
forecast = model.predict(forecast)
# Evaluate performance using MAE and MSE
mae = np.mean(np.abs(forecast["yhat"] - df["y"]))
mse = np.mean((forecast["yhat"] - df["y"]) ** 2)
# Fine-tune the model using statsmodels
sarimax_model = SARIMAX(df["y"], order=(1,1,1))
sarimax_results = sarimax_model.fit()
Complete Script
The full runnable script combining all steps:
#!/usr/bin/env python3
import requests
import pandas as pd
from prophet import Prophet
from statsmodels.tsa.statespace.sarimax import SARIMAX
import numpy as np
def load_data():
response = requests.get("https://randomuser.me/api/")
data = response.json()
df = pd.DataFrame(data["results"])
return df
def create_prophet_model(df):
model = Prophet()
model.fit(df)
return model
def make_predictions(model, df):
forecast = model.make_future_dataframe(periods=30)
forecast = model.predict(forecast)
return forecast
def evaluate_performance(forecast, df):
mae = np.mean(np.abs(forecast["yhat"] - df["y"]))
mse = np.mean((forecast["yhat"] - df["y"]) ** 2)
return mae, mse
def fine_tune_model(df):
sarimax_model = SARIMAX(df["y"], order=(1,1,1))
sarimax_results = sarimax_model.fit()
return sarimax_results
if __name__ == "__main__":
df = load_data()
model = create_prophet_model(df)
forecast = make_predictions(model, df)
mae, mse = evaluate_performance(forecast, df)
sarimax_results = fine_tune_model(df)
print("MAE:", mae)
print("MSE:", mse)
print("SARIMAX Results:", sarimax_results.summary())
Expected Output
When you run the script, you should see the MAE and MSE values printed to the console, as well as the summary of the SARIMAX model results.
Limitations and Tradeoffs
While combining Prophet and statsmodels provides a powerful approach to time series forecasting, there are some limitations and tradeoffs to consider. Prophet can be computationally expensive for large datasets, and statsmodels requires careful selection of model parameters to avoid overfitting. Additionally, the approach assumes that the data is stationary, which may not always be the case in real-world scenarios.
Frequently Asked Questions
What is the difference between Prophet and statsmodels?
Prophet is a software for forecasting time series data based on an additive model where non-linear trends are fit with yearly, weekly, and daily seasonality, plus holiday effects. It works well with multiple seasonality with non-uniform periods and non-linear trends. Statsmodels, on the other hand, is a Python module that provides a complement to scipy for statistical computations including regression and time-series analysis.
How do I handle missing observations in my time series data?
Missing observations can be handled using various techniques such as imputation, interpolation, or modeling the missing values as a separate process. In this example, we assume that the data is complete and does not contain any missing observations.
What is the best way to evaluate the performance of my time series forecasting model?
The best way to evaluate the performance of your time series forecasting model is to use metrics such as mean absolute error (MAE) and mean squared error (MSE), which provide a measure of the difference between the predicted and actual values.
What I'd Change
In conclusion, combining Prophet and statsmodels provides a powerful approach to time series forecasting. However, I would change the way I handle missing observations and non-stationarity in the data. I would also experiment with different model parameters and techniques to improve the accuracy and robustness of the forecasts. By doing so, I believe that data scientists can create highly accurate and robust time series forecasting models that handle real-world complexities and edge cases.
