Building accurate and scalable time series forecasting models is a challenging task, especially when dealing with large and complex datasets. I recently worked on a project where I had to predict the number of users that would be generated by the Random User API over a certain period of time. This post outlines the steps I took to build a reliable time series forecasting model using Python, TimesFM, and Scipy, and highlights the key takeaways from the project.
Key Takeaways
- TimesFM can be used to build accurate time series forecasting models by leveraging the power of temporal fusion transformers.
- Scipy can be used to optimize the hyperparameters of the model and improve its performance.
- The Random User API can be used to generate a time series dataset that can be used to train and test the model.
The Problem
The Random User API generates a large number of users every day, and predicting the number of users that will be generated over a certain period of time is a challenging task. The API provides a wide range of parameters that can be used to customize the users, such as location, age, and gender, but it does not provide any information about the number of users that will be generated in the future.
Data and Sources
The Random User API (https://randomuser.me/api/) was used to generate a time series dataset that was used to train and test the model. The API provides a simple way to generate random users, and it can be used to generate a large number of users in a short amount of time. Data accessed on 2024-09-16.
Loading the Data
The data was loaded using the requests library in Python. The API was called repeatedly to generate a large number of users, and the data was stored in a pandas dataframe.
import requests
import pandas as pd
def load_data():
data = []
for i in range(1000):
response = requests.get("https://randomuser.me/api/")
user = response.json()["results"][0]
data.append({
"name": user["name"]["first"] + " " + user["name"]["last"],
"age": user["dob"]["age"],
"location": user["location"]["city"] + ", " + user["location"]["country"]
})
return pd.DataFrame(data)
The Core Logic
The core logic of the model is based on the temporal fusion transformer (TFT) architecture, which is a type of recurrent neural network that is well-suited for time series forecasting tasks. The TFT architecture is implemented in the TimesFM library, which provides a simple way to build and train TFT models.
from timesfm import TFTModel
def build_model():
model = TFTModel(
input_shape=(100, 3), # 100 time steps, 3 features
output_shape=(1,), # 1 output feature
num_layers=2, # 2 layers
num_heads=4, # 4 attention heads
dropout=0.1 # 0.1 dropout rate
)
return model
Putting It Together
The model was trained using the Scipy library, which provides a simple way to optimize the hyperparameters of the model. The model was trained on the training data, and its performance was evaluated on the test data.
from scipy.optimize import minimize
def train_model(model, data):
def loss(params):
# compile the model with the given hyperparameters
model.compile(optimizer="adam", loss="mean_squared_error")
# train the model on the training data
model.fit(data["train"], epochs=10, batch_size=32, verbose=0)
# evaluate the model on the test data
loss = model.evaluate(data["test"])
return loss
# define the hyperparameters to optimize
params = ["learning_rate", "num_layers", "num_heads", "dropout"]
# optimize the hyperparameters
result = minimize(loss, [0.001, 2, 4, 0.1], method="nelder-mead")
return result.x
Complete Script
The full runnable script combining all steps:
#!/usr/bin/env python3
import requests
import pandas as pd
from timesfm import TFTModel
from scipy.optimize import minimize
def load_data():
data = []
for i in range(1000):
response = requests.get("https://randomuser.me/api/")
user = response.json()["results"][0]
data.append({
"name": user["name"]["first"] + " " + user["name"]["last"],
"age": user["dob"]["age"],
"location": user["location"]["city"] + ", " + user["location"]["country"]
})
return pd.DataFrame(data)
def build_model():
model = TFTModel(
input_shape=(100, 3), # 100 time steps, 3 features
output_shape=(1,), # 1 output feature
num_layers=2, # 2 layers
num_heads=4, # 4 attention heads
dropout=0.1 # 0.1 dropout rate
)
return model
def train_model(model, data):
def loss(params):
# compile the model with the given hyperparameters
model.compile(optimizer="adam", loss="mean_squared_error")
# train the model on the training data
model.fit(data["train"], epochs=10, batch_size=32, verbose=0)
# evaluate the model on the test data
loss = model.evaluate(data["test"])
return loss
# define the hyperparameters to optimize
params = ["learning_rate", "num_layers", "num_heads", "dropout"]
# optimize the hyperparameters
result = minimize(loss, [0.001, 2, 4, 0.1], method="nelder-mead")
return result.x
if __name__ == "__main__":
data = load_data()
model = build_model()
result = train_model(model, data)
print(result)
Expected Output
The expected output is the optimized hyperparameters for the model, which can be used to train the model and make predictions on new data.
Limitations and Tradeoffs
The main limitation of this approach is that it requires a large amount of data to train the model, and the model can be computationally expensive to train. Additionally, the model assumes that the data is stationary, which may not always be the case in real-world applications.
Frequently Asked Questions
What is the temporal fusion transformer (TFT) architecture?
The TFT architecture is a type of recurrent neural network that is well-suited for time series forecasting tasks. It uses a combination of recurrent and feedforward layers to model the temporal relationships in the data.
How do I optimize the hyperparameters of the model?
The hyperparameters of the model can be optimized using the Scipy library, which provides a simple way to optimize the hyperparameters of the model using a variety of optimization algorithms.
What is the difference between the training and test data?
The training data is used to train the model, and the test data is used to evaluate the performance of the model. The test data should not be used to train the model, as this can result in overfitting.
What I'd Change
If I were to rebuild this project, I would use a more robust optimization algorithm, such as Bayesian optimization, to optimize the hyperparameters of the model. I would also use a more advanced technique, such as ensemble learning, to combine the predictions of multiple models and improve the overall performance of the model. Additionally, I would use a more detailed analysis of the data to identify any patterns or trends that could be used to improve the performance of the model.
