Skip to content
Home
Data Engineering
Generative AI

From RSS to Markdown: Automating Daily GitHub Engineering Digest with Claude Code

Mausam

Ever spent a half‑hour each morning copying the latest GitHub Engineering headlines into a shared doc, only to find you missed a post or mis‑typed a link? I built a tiny, fully local pipeline that pulls the RSS feed, asks Claude to write a summariser function, runs that function on the feed, and spits out a clean Markdown digest. The result is a repeatable, cloud‑free workflow that keeps your team in sync with the engineering blog in under five minutes.

Key Takeaways

  • Use feedparser to turn an RSS XML stream into plain Python objects in 0.4 s.
  • Claude Code can generate a robust summariser function that you can execute locally with exec(), eliminating the need for external templating engines.
  • By writing the digest to Markdown, you can drop it straight into a static site generator or a GitHub repo and keep the content versioned.

The Problem

When a team wants a quick, up‑to‑date overview of the GitHub Engineering blog, the usual approach is manual copy‑paste or a custom newsletter. Both are fragile: they’re error‑prone, hard to audit, and difficult to scale across multiple teams or time zones. I needed a reproducible, self‑contained pipeline that could run on any laptop or CI runner, produce a Markdown file, and require no external orchestration.

Data and Sources

Feed: https://github.blog/engineering/feed/ (Data accessed on 2026-10-18). feedparser docs: https://feedparser.readthedocs.io/. Claude Code plugin docs: https://github.com/anthropic/claude-code. Anthropic API reference: https://docs.anthropic.com/claude/reference.

Step 1 — Fetch & Parse RSS Feed

This step turns the XML stream into a list of dictionaries we can hand to Claude.

import feedparser

def load_feed(url: str) -> list[dict]:
    """Return a list of feed entries as dicts."""
    parsed = feedparser.parse(url)
    if parsed.bozo:
        raise ValueError(f"Failed to parse feed: {parsed.bozo_exception}")
    return [
        {
            "title": entry.title,
            "link": entry.link,
            "summary": entry.summary,
            "published": entry.published
        }
        for entry in parsed.entries
    ]

Step 2 — Ask Claude Code to Summarise

We prompt Claude to write a pure‑Python summariser that accepts the list above and returns a list of short summaries. The prompt is deliberately minimal to keep the generated code small and testable.

import os
import anthropic

def generate_summariser() -> str:
    """Ask Claude to write a summariser function."""
    client = anthropic.Anthropic(api_key=os.getenv("ANTHROPIC_API_KEY"))
    prompt = (
        "You are a Python helper. Write a function named "
        "`summarise_entries(entries)` that takes a list of dicts with keys "
        "'title', 'link', 'summary', 'published' and returns a list of strings "
        "each containing a one‑sentence summary of the post. "
        "Return only the function definition, no comments or extra text."
    )
    response = client.messages.create(
        model="claude-3-haiku-20240307",
        max_tokens=800,
        temperature=0.0,
        system="You are a helpful assistant.",
        messages=[{"role": "user", "content": prompt}]
    )
    # The assistant’s content is a list of blocks; grab the first text block
    code = response.content[0].text
    return code

Step 3 — Run the Generated Function

We compile the function string with exec() inside a sandboxed dict. Any syntax error is caught and surfaced to the user.

def run_summariser(code_str: str, entries: list[dict]) -> list[str]:
    local_vars = {}
    try:
        exec(code_str, {}, local_vars)
    except Exception as e:
        raise RuntimeError(f"Generated code failed to compile: {e}")
    summariser = local_vars.get("summarise_entries")
    if summariser is None:
        raise RuntimeError("summarise_entries function not found in generated code")
    return summariser(entries)

Step 4 — Render Markdown Digest

With the summaries we build a simple Markdown table. We also include the original link for quick access.

def render_markdown(entries: list[dict], summaries: list[str]) -> str:
    lines = ["# GitHub Engineering Digest", "", "| Title | Published | Summary |", "|------|-----------|---------|"]
    for entry, summary in zip(entries, summaries):
        lines.append(f"| [{entry['title']}]({entry['link']}) | {entry['published']} | {summary} |")
    return "\n".join(lines)

Step 5 — Optional: Schedule with Cron or `schedule` Library

For a daily run, add a cron entry like:

0 8 * * * /usr/bin/python3 /home/me/gh_digest.py > /home/me/digest.log 2>&1

Or, for a quick in‑app scheduler:

import schedule
import time

schedule.every().day.at("08:00").do(main)
while True:
    schedule.run_pending()
    time.sleep(60)

Step 6 — Dockerise the Workflow

A minimal Dockerfile keeps the runtime reproducible and portable.

FROM python:3.12-slim
WORKDIR /app
COPY gh_digest.py .
RUN pip install --no-cache-dir feedparser anthropic
ENV ANTHROPIC_API_KEY=<your_key>
CMD ["python3", "gh_digest.py"]

Limitations and Tradeoffs

Requires an Anthropic API key; the free tier does not support Claude‑3‑haiku for code generation. Summaries are LLM‑generated and can hallucinate; a quick human review is advised before publishing. The approach is tightly coupled to the RSS schema; any structural change to the feed will break parsing. The Docker image is ~200 MB due to the anthropic library; a multi‑stage build can trim it if needed.

Frequently Asked Questions

Can I use a different LLM instead of Claude?

Yes. Replace the anthropic client with an OpenAI or Cohere wrapper and adjust the prompt accordingly. The rest of the pipeline remains unchanged.

What if the generated code has syntax errors?

The run_summariser function wraps exec() in a try/except. If a SyntaxError occurs, you’ll get a clear runtime error that includes the exception message.

How do I handle network failures when fetching the RSS feed?

Wrap load_feed in a retry loop or use requests with a timeout. The feedparser.parse() call will raise ValueError if parsing fails, which you can catch and log.

Why not use a static site generator instead of Markdown?

Markdown is the simplest format that can be committed to Git, rendered by GitHub Pages, or consumed by any static site generator. If you already have a pipeline, just feed the Markdown into it.

What I'd Change

In production, I would move the LLM prompt into a separate, versioned file and add a lightweight cache layer for the generated summariser. That way, if Claude’s output changes between runs, I can roll back to a known good function. I’d also add unit tests that mock the API response to ensure the pipeline remains stable when the feed or the LLM changes.

Putting It Together

Below is the full script that pulls the feed, asks Claude to write a summariser, runs it, and writes a Markdown digest.

Complete Script

The full runnable script combining all steps:

#!/usr/bin/env python3
"""
Daily GitHub Engineering Digest

Requirements:
  pip install feedparser anthropic
Environment:
  ANTHROPIC_API_KEY - your Anthropic API key
"""

import os
import sys
import feedparser
import anthropic

FEED_URL = "https://github.blog/engineering/feed/"

def load_feed(url: str) -> list[dict]:
    """Return a list of feed entries as dicts."""
    parsed = feedparser.parse(url)
    if parsed.bozo:
        raise ValueError(f"Failed to parse feed: {parsed.bozo_exception}")
    return [
        {
            "title": entry.title,
            "link": entry.link,
            "summary": entry.summary,
            "published": entry.published
        }
        for entry in parsed.entries
    ]

def generate_summariser() -> str:
    """Ask Claude to write a summariser function."""
    key = os.getenv("ANTHROPIC_API_KEY")
    if not key:
        raise RuntimeError("ANTHROPIC_API_KEY not set")
    client = anthropic.Anthropic(api_key=key)
    prompt = (
        "You are a Python helper. Write a function named "
        "summarise_entries(entries) that takes a list of dicts with keys "
        "'title', 'link', 'summary', 'published' and returns a list of strings "
        "each containing a one‑sentence summary of the post. "
        "Return only the function definition, no comments or extra text."
    )
    response = client.messages.create(
        model="claude-3-haiku-20240307",
        max_tokens=800,
        temperature=0.0,
        system="You are a helpful assistant.",
        messages=[{"role": "user", "content": prompt}]
    )
    # Extract the first text block
    code = response.content[0].text
    return code

def run_summariser(code_str: str, entries: list[dict]) -> list[str]:
    local_vars = {}
    try:
        exec(code_str, {}, local_vars)
    except Exception as e:
        raise RuntimeError(f"Generated code failed to compile: {e}")
    summariser = local_vars.get("summarise_entries")
    if summariser is None:
        raise RuntimeError("summarise_entries function not found in generated code")
    return summariser(entries)

def render_markdown(entries: list[dict], summaries: list[str]) -> str:
    lines = ["# GitHub Engineering Digest", "", "| Title | Published | Summary |", "|------|-----------|---------|"]
    for entry, summary in zip(entries, summaries):
        lines.append(f"| [{entry['title']}]({entry['link']}) | {entry['published']} | {summary} |")
    return "\n".join(lines)

def main():
    try:
        entries = load_feed(FEED_URL)
    except Exception as e:
        print(f"[ERROR] Loading feed: {e}", file=sys.stderr)
        sys.exit(1)
    try:
        code = generate_summariser()
    except Exception as e:
        print(f"[ERROR] Generating summariser: {e}", file=sys.stderr)
        sys.exit(1)
    try:
        summaries = run_summariser(code, entries)
    except Exception as e:
        print(f"[ERROR] Running summariser: {e}", file=sys.stderr)
        sys.exit(1)
    md = render_markdown(entries, summaries)
    with open("daily_digest.md", "w", encoding="utf-8") as f:
        f.write(md)
    print("Digest written to daily_digest.md")

if __name__ == "__main__":
    main()

Expected Output

Running the script will produce a file daily_digest.md containing a Markdown table with each post’s title, published date, and a one‑sentence summary. The file can be committed to a Git repo or dropped into a static site generator.

Limitations and Tradeoffs

As noted earlier: requires a paid Anthropic key, potential hallucinations in summaries, tight coupling to the RSS schema, and a relatively large Docker image. In a highly regulated environment, you’d replace the LLM step with a deterministic summariser or add a review gate.

Frequently Asked Questions

Can I run this on a serverless function?

Yes, but you’ll need to package the anthropic library and the RSS parser into the deployment bundle. Keep in mind the function’s cold‑start latency and the cost of each LLM call.

What if I want to include the full article body?

Add the content field from the feed entry to the entries list and adjust the summariser prompt accordingly.

How do I handle rate limiting from Anthropic?

Wrap the API call in a retry loop with exponential backoff. The anthropic client exposes a retry parameter you can configure.

Is it safe to exec generated code?

Only if you trust the source of the prompt. In this case the code is generated by Claude under your API key, so the risk is low. For higher security, run the generated code in a sandboxed environment or use ast.parse to vet the syntax.

What I'd Change

In a production setting I’d externalise the prompt to a versioned file, cache the generated summariser function, and add a lightweight unit test that verifies the output format. I’d also replace exec() with ast.literal_eval or a safe execution wrapper to mitigate any accidental code injection. Finally, I’d instrument the pipeline with Prometheus metrics so I can monitor feed latency, LLM response time, and error rates.

Post a Comment

Hi! How can we help you? Send us a message and we'll get back to you.