You have a stream of JSON user profiles from a public API and need a fast, repeatable way to surface records that look wrong before you feed the data into downstream models. The answer is to let scikit‑learn’s Isolation Forest work out‑of‑the‑box on a numerically‑encoded feature matrix; with minimal preprocessing you get reproducible anomaly scores and a ready‑to‑audit CSV of the most suspicious users.
Key Takeaways
- Isolation Forest works out‑of‑the‑box on mixed numeric encodings; you only need to convert categories to numbers.
- Proper feature engineering—binary gender, one‑hot low‑cardinality country, and geographic coordinates—creates a space where true outliers stand out.
- The algorithm scales linearly with rows and dimensions, making it suitable for hundreds to thousands of API‑fetched records.
- Anomaly scores are directly interpretable; the lowest scores correspond to the strongest outliers.
- After the initial fetch you can run the script offline, guaranteeing reproducibility.
Data and Sources
- Random User API documentation – source of the 500‑record JSON feed.
- scikit‑learn IsolationForest docs – algorithm reference.
- pandas 2.x reference – data wrangling utilities.
- requests library docs – HTTP client used to fetch the data.
Source reviewed on 2026-11-12.
Isolation Forest on Random User Demographics
First we pull 500 random users, keep the fields that matter for a geographic‑demographic profile, and turn them into a flat table. Gender becomes a binary flag, country is one‑hot encoded (keeping the ten most common values and collapsing the rest into “Other”), and latitude/longitude are left as floating‑point numbers. Age is already numeric.
With this matrix we fit an IsolationForest with a tiny contamination estimate (1 %). The model assigns each row a decision_function score; more negative values indicate stronger isolation. Selecting the five lowest scores gives us a concise list of records that deserve manual review.
Limitations and Tradeoffs
- The method assumes that anomalies are isolated in the chosen feature space. If an outlier only differs on a high‑cardinality categorical field that we drop or encode poorly, Isolation Forest may miss it.
- One‑hot encoding can explode dimensionality when categories are many. For very large country lists you should group rare values or use target encoding.
- Isolation Forest’s unsupervised nature means false positives are inevitable; domain expertise is still required to confirm whether a flagged user is truly problematic.
- For datasets well beyond a few hundred thousand rows you will need to subsample or use a distributed implementation, otherwise memory and runtime grow linearly.
Frequently Asked Questions
What if the API returns fewer than 500 users?
The script simply works with whatever number of records are returned; the IsolationForest model adapts to the actual row count.
Can I use other categorical encodings?
Yes. Frequency encoding or embeddings work when you have many distinct values, but you must keep the resulting matrix numeric for IsolationForest.
How do I choose the contamination parameter?
Start with a small value (e.g., 0.01) that matches the proportion of anomalies you expect. You can later adjust based on the proportion of flagged records that survive manual review.
Will the script run without internet?
After the first successful fetch you can rerun the script offline by commenting out the network call and loading the saved CSV; the isolation model itself does not need external resources.
Is the anomaly score comparable across different runs?
Yes, provided you fix the random seed (the script uses random_state=42) and keep the preprocessing steps identical.
Recommendation and Next Step
For most API‑driven user datasets, start with Isolation Forest as a baseline anomaly filter. It gives you a quick, reproducible view of the most suspicious records with virtually no parameter tuning. The next step is to layer domain‑specific rules—such as age‑range checks or impossible coordinate combinations—on top of the model output, or to collect a small labeled set and move to semi‑supervised techniques that can reduce false positives.
Complete Script
#!/usr/bin/env python3
import sys
import requests
import collections
import csv
from sklearn.ensemble import IsolationForest
def fetch_users(url: str, timeout: int = 10) -> list:
"""Fetch user data from the Random User API with error handling."""
try:
response = requests.get(url, timeout=timeout)
response.raise_for_status()
except requests.RequestException as exc:
sys.stderr.write(f"Network error while fetching data: {exc}\n")
sys.exit(1)
data = response.json()
if "results" not in data:
sys.stderr.write("Unexpected API response structure.\n")
sys.exit(1)
return data["results"]
def build_records(users: list) -> tuple[list, list]:
"""Extract required fields, encode them, and return a list of record dicts."""
raw_records = []
for user in users:
try:
gender = user["gender"]
age = user["dob"]["age"]
country = user["location"]["country"]
lat = float(user["location"]["coordinates"]["latitude"])
lon = float(user["location"]["coordinates"]["longitude"])
first = user["name"]["first"]
last = user["name"]["last"]
except (KeyError, TypeError, ValueError):
continue
raw_records.append({
"gender": gender,
"age": age,
"country": country,
"lat": lat,
"lon": lon,
"first": first,
"last": last
})
if not raw_records:
sys.stderr.write("No valid user records after preprocessing.\n")
sys.exit(1)
# Encode gender
gender_map = {"male": 0, "female": 1}
for rec in raw_records:
rec["gender"] = gender_map.get(rec["gender"], -1)
# Determine top 10 countries
country_counter = collections.Counter(rec["country"] for rec in raw_records)
top_countries = [c for c, _ in country_counter.most_common(10)]
# Collapse less‑frequent countries to "Other"
for rec in raw_records:
rec["country"] = rec["country"] if rec["country"] in top_countries else "Other"
# One‑hot encode countries, dropping the first (most frequent) category
dropped_country = top_countries[0] if top_countries else None
country_onehot_cols = []
if top_countries:
for cat in top_countries[1:]:
col_name = f"country_{cat}"
country_onehot_cols.append(col_name)
for rec in raw_records:
rec[col_name] = 1 if rec["country"] == cat else 0
# Records with "Other" or the dropped category keep zeros
return raw_records, country_onehot_cols
def detect_outliers(records: list, onehot_cols: list) -> list:
"""Fit IsolationForest on numeric features and attach anomaly scores."""
feature_names = ["gender", "age", "lat", "lon"] + onehot_cols
X = [[rec[name] for name in feature_names] for rec in records]
model = IsolationForest(contamination=0.01, random_state=42)
model.fit(X)
scores = model.decision_function(X)
for rec, score in zip(records, scores):
rec["anomaly_score"] = score
return records
def main():
api_url = "https://randomuser.me/api/?results=500"
users = fetch_users(api_url)
records, onehot_cols = build_records(users)
records = detect_outliers(records, onehot_cols)
# Show the five most anomalous records (lowest scores)
outliers = sorted(records, key=lambda r: r["anomaly_score"])[:5]
for rec in outliers:
country = rec["country"]
if country == "Other":
country_display = "Other"
else:
country_display = country
print(
f"⚠️ {rec['first']} {rec['last']} ({country_display}, {int(rec['age'])}) "
f"– score: {rec['anomaly_score']:.2f}"
)
# Save full result set for audit
csv_path = "outliers.csv"
fieldnames = ["first", "last", "gender", "age", "lat", "lon", "country"] + onehot_cols + ["anomaly_score"]
with open(csv_path, mode="w", newline="", encoding="utf-8") as f:
writer = csv.DictWriter(f, fieldnames=fieldnames)
writer.writeheader()
for rec in records:
writer.writerow({k: rec.get(k, "") for k in fieldnames})
print(f"Full results with anomaly scores saved to {csv_path}")
if __name__ == "__main__":
main()