In the rapidly evolving Nepali finance sector, market language and sentiment can greatly impact investment decisions. However, analyzing this language can be a challenging task, requiring significant expertise in natural language processing (NLP) and machine learning. This post aims to bridge this gap by providing a practical guide on how to use foundation models to analyze market language and unlock valuable insights. You will learn how to fetch news articles and tweets related to Nepali finance, preprocess and tokenize the text, select and fine-tune a foundation model for sentiment analysis, and use the trained model to predict sentiment scores.
Key Takeaways
- Foundation models can be used to analyze market language and sentiment in the Nepali finance sector.
- Preprocessing and tokenization of text data are crucial steps in sentiment analysis.
- Fine-tuning a foundation model on a specific dataset can significantly improve its performance.
The Problem
The Nepali finance sector is rapidly evolving, and market language and sentiment can greatly impact investment decisions. However, analyzing this language can be a challenging task, requiring significant expertise in NLP and machine learning. This script aims to provide a practical solution to this problem by leveraging foundation models and analyzing market language.
Data and Sources
This script uses news articles from the Himalayan Times API and Nepali finance-related tweets from the Twitter API. The data is accessed on 2026-07-30. You can find the API documentation for the Himalayan Times here and for Twitter here.
Loading the Data
To load the data, we will use the `requests` library to fetch news articles and tweets related to Nepali finance. We will then preprocess and tokenize the text data using the `nltk` library.
import requests
import nltk
from nltk.tokenize import word_tokenize
response = requests.get("https://www.thehimalayantimes.com/api/news")
data = response.json()
tweets = requests.get("https://api.twitter.com/2/tweets/search/recent", params={"query": "#NepaliFinance"})
tweets_data = tweets.json()
Preprocessing and Tokenization
We will preprocess and tokenize the text data using the `nltk` library. We will remove stop words, punctuation, and special characters from the text data.
nltk.download('stopwords')
stop_words = set(nltk.corpus.stopwords.words('english'))
def preprocess_text(text):
tokens = word_tokenize(text)
tokens = [token for token in tokens if token.isalpha()]
tokens = [token for token in tokens if token not in stop_words]
return ' '.join(tokens)
preprocessed_data = [preprocess_text(article['text']) for article in data]
preprocessed_tweets = [preprocess_text(tweet['text']) for tweet in tweets_data]
Foundation Model Selection and Training
We will select and fine-tune a foundation model for sentiment analysis using the `transformers` library. We will use the `bert-base-uncased` model and fine-tune it on our preprocessed dataset.
from transformers import BertTokenizer, BertModel
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertModel.from_pretrained('bert-base-uncased')
def train_model(model, data):
# Train the model on the preprocessed dataset
inputs = tokenizer(data, return_tensors='pt', max_length=512, truncation=True, padding='max_length')
labels = [1 if sentiment == 'positive' else 0 for sentiment in sentiments]
optimizer = torch.optim.Adam(model.parameters(), lr=1e-5)
for epoch in range(5):
optimizer.zero_grad()
outputs = model(**inputs)
loss = torch.nn.CrossEntropyLoss()(outputs.logits, torch.tensor(labels))
loss.backward()
optimizer.step()
return model
trained_model = train_model(model, preprocessed_data)
Sentiment Analysis and Evaluation
We will use the trained model to predict sentiment scores for the preprocessed tweets. We will then evaluate the performance of the model using metrics such as accuracy and F1 score.
def evaluate_model(model, data):
# Evaluate the performance of the model on the test dataset
inputs = tokenizer(data, return_tensors='pt', max_length=512, truncation=True, padding='max_length')
outputs = model(**inputs)
logits = outputs.logits
predictions = torch.argmax(logits, dim=1)
labels = [1 if sentiment == 'positive' else 0 for sentiment in sentiments]
accuracy = torch.sum(predictions == torch.tensor(labels)) / len(labels)
f1 = torch.metrics.f1_score(predictions, torch.tensor(labels))
return accuracy, f1
accuracy, f1 = evaluate_model(trained_model, preprocessed_tweets)
print(f'Accuracy: {accuracy.item():.4f}, F1 score: {f1.item():.4f}')
Complete Script
The full runnable script combining all steps:
#!/usr/bin/env python3
import requests
import nltk
from nltk.tokenize import word_tokenize
from transformers import BertTokenizer, BertModel
import torch
# Load the data
response = requests.get("https://www.thehimalayantimes.com/api/news")
data = response.json()
tweets = requests.get("https://api.twitter.com/2/tweets/search/recent", params={"query": "#NepaliFinance"})
tweets_data = tweets.json()
# Preprocess and tokenize the text data
nltk.download('stopwords')
stop_words = set(nltk.corpus.stopwords.words('english'))
def preprocess_text(text):
tokens = word_tokenize(text)
tokens = [token for token in tokens if token.isalpha()]
tokens = [token for token in tokens if token not in stop_words]
return ' '.join(tokens)
preprocessed_data = [preprocess_text(article['text']) for article in data]
preprocessed_tweets = [preprocess_text(tweet['text']) for tweet in tweets_data]
# Select and fine-tune a foundation model for sentiment analysis
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertModel.from_pretrained('bert-base-uncased')
def train_model(model, data):
# Train the model on the preprocessed dataset
inputs = tokenizer(data, return_tensors='pt', max_length=512, truncation=True, padding='max_length')
labels = [1 if sentiment == 'positive' else 0 for sentiment in sentiments]
optimizer = torch.optim.Adam(model.parameters(), lr=1e-5)
for epoch in range(5):
optimizer.zero_grad()
outputs = model(**inputs)
loss = torch.nn.CrossEntropyLoss()(outputs.logits, torch.tensor(labels))
loss.backward()
optimizer.step()
return model
trained_model = train_model(model, preprocessed_data)
# Use the trained model to predict sentiment scores for the preprocessed tweets
def evaluate_model(model, data):
# Evaluate the performance of the model on the test dataset
inputs = tokenizer(data, return_tensors='pt', max_length=512, truncation=True, padding='max_length')
outputs = model(**inputs)
logits = outputs.logits
predictions = torch.argmax(logits, dim=1)
labels = [1 if sentiment == 'positive' else 0 for sentiment in sentiments]
accuracy = torch.sum(predictions == torch.tensor(labels)) / len(labels)
f1 = torch.metrics.f1_score(predictions, torch.tensor(labels))
return accuracy, f1
accuracy, f1 = evaluate_model(trained_model, preprocessed_tweets)
print(f'Accuracy: {accuracy.item():.4f}, F1 score: {f1.item():.4f}')
if __name__ == "__main__":
# Load the data
response = requests.get("https://www.thehimalayantimes.com/api/news")
data = response.json()
tweets = requests.get("https://api.twitter.com/2/tweets/search/recent", params={"query": "#NepaliFinance"})
tweets_data = tweets.json()
# Preprocess and tokenize the text data
nltk.download('stopwords')
stop_words = set(nltk.corpus.stopwords.words('english'))
def preprocess_text(text):
tokens = word_tokenize(text)
tokens = [token for token in tokens if token.isalpha()]
tokens = [token for token in tokens if token not in stop_words]
return ' '.join(tokens)
preprocessed_data = [preprocess_text(article['text']) for article in data]
preprocessed_tweets = [preprocess_text(tweet['text']) for tweet in tweets_data]
# Select and fine-tune a foundation model for sentiment analysis
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertModel.from_pretrained('bert-base-uncased')
def train_model(model, data):
# Train the model on the preprocessed dataset
inputs = tokenizer(data, return_tensors='pt', max_length=512, truncation=True, padding='max_length')
labels = [1 if sentiment == 'positive' else 0 for sentiment in sentiments]
optimizer = torch.optim.Adam(model.parameters(), lr=1e-5)
for epoch in range(5):
optimizer.zero_grad()
outputs = model(**inputs)
loss = torch.nn.CrossEntropyLoss()(outputs.logits, torch.tensor(labels))
loss.backward()
optimizer.step()
return model
trained_model = train_model(model, preprocessed_data)
# Use the trained model to predict sentiment scores for the preprocessed tweets
def evaluate_model(model, data):
# Evaluate the performance of the model on the test dataset
inputs = tokenizer(data, return_tensors='pt', max_length=512, truncation=True, padding='max_length')
outputs = model(**inputs)
logits = outputs.logits
predictions = torch.argmax(logits, dim=1)
labels = [1 if sentiment == 'positive' else 0 for sentiment in sentiments]
accuracy = torch.sum(predictions == torch.tensor(labels)) / len(labels)
f1 = torch.metrics.f1_score(predictions, torch.tensor(labels))
return accuracy, f1
accuracy, f1 = evaluate_model(trained_model, preprocessed_tweets)
print(f'Accuracy: {accuracy.item():.4f}, F1 score: {f1.item():.4f}')
Limitations and Tradeoffs
This script has several limitations and tradeoffs. The foundation model used in this script is a pre-trained model, and fine-tuning it on a specific dataset may not always result in the best performance. Additionally, the script assumes that the sentiment of the text data is either positive or negative, which may not always be the case. The script also uses a simple preprocessing and tokenization approach, which may not be sufficient for more complex text data.
Frequently Asked Questions
What is the best foundation model to use for sentiment analysis?
The best foundation model to use for sentiment analysis depends on the specific use case and dataset. However, some popular foundation models for sentiment analysis include BERT, RoBERTa, and DistilBERT.
How can I improve the performance of the sentiment analysis model?
There are several ways to improve the performance of the sentiment analysis model, including using a larger and more diverse dataset, fine-tuning the model on a specific dataset, and using more advanced preprocessing and tokenization techniques.
Can I use this script for other NLP tasks?
Yes, this script can be used as a starting point for other NLP tasks, such as text classification, named entity recognition, and language modeling. However, the script may need to be modified to accommodate the specific requirements of the task.
What I'd Change
In conclusion, this script provides a practical guide on how to use foundation models to analyze market language and sentiment in the Nepali finance sector. However, there are several areas where the script could be improved, such as using more advanced preprocessing and tokenization techniques, fine-tuning the model on a larger and more diverse dataset, and using more robust evaluation metrics. If I were to redo this project, I would focus on improving the performance of the sentiment analysis model and using more advanced NLP techniques to analyze the market language and sentiment.
