Getting Journal Club onto my phone · sk heavy industries

Journal Club is a site that provides bite-sized video overviews of complex technical white papers. I started using it over 6 months ago and I've found it very useful for catching up on white papers, not just in technology but across various scientific disciplines. The downside for me is that there isn't much of an offline mode: no download and no podcast feed. I often listen on the go, so I wanted a way to get these on my local device.

I hacked up some code to get the latest episodes on my phone, including the audio, the site's paper overview and the original white paper. I download the content and push it into my iCloud Drive, to a folder that's on both my Mac and my phone. That's the entire idea behind this tool. It signs in as me, pulls each episode into an iCloud folder, and lets Apple carry it to the phone. I will note that journalclub.io is a paid subscription and I am in no way affiliated with them or receiving any kickback, etc for this code or post.

You can find the code on GitHub: github.com/mr-sk/journalclub-on-the-go.

Signing in

The site has no API, so the tool drives a real browser with Playwright. Playwright seems to be getting more and more popular as a much needed modern upgrade to Selenium.

The first run opens a window and waits while I log in to get an active session. Every run after that loads that session and works headless.

print("Sign in to Journal Club in the browser window.")
input("Press Return after signing in: ")
save_auth(context)          # stores the browser session, not the password

The script never asks for my password and never stores it. What it keeps is the session the browser already holds, written to a file only I can read and kept out of the repo.

Pulling an episode

Each episode page has three things: the audio, the write-up, and a link to the paper. The audio is a direct download once you are signed in. The write-up I turn into a PDF by printing the page with Chromium which gives a clean copy to read next to the audio.

A download can return a 200 status code and still not be the file I requested. A login wall or a publisher block will hand back an HTML page with a success code, and if you trust the status you save that HTML as episode.mp3. So nothing here is trusted on the status alone. Every file is checked by its first few bytes.

def is_audio(body, content_type):
    head = body[:16].lstrip().lower()
    if head.startswith((b"<!doctype html", b"<html", b"<?xml")):
        return False                                   # a web page, not a file
    return (
        body.startswith(b"ID3")                                   # mp3 with tags
        or (len(body) > 1 and body[0] == 0xFF and body[1] & 0xE0 == 0xE0)  # mp3 frame
        or body[4:8] == b"ftyp"                                   # m4a / mp4
        or body.startswith((b"OggS", b"fLaC"))
    )

If the bytes do not look like audio the download is thrown away instead of saved. The PDF check is the same idea: it looks for a %PDF- header.

Getting the paper

The episode links to the paper by DOI, and a DOI is a permanent redirect to wherever the publisher keeps the article. That landing page is almost never the PDF. Sometimes the file is behind a paywall, or it exists but sits behind a bot check. There is no single way to get it so the tool tries a series of routes and takes the first one that returns real PDF bytes.

In order:

Ask the DOI itself for a PDF. A few publishers honor a direct request and return the file.
Ask Crossref. Its metadata sometimes lists a PDF link for the article.
Guess the publisher's PDF URL from the shape of the landing page. Each large publisher has a pattern.
Read the page metadata where many articles advertise the file in a citation_pdf_url tag.
Scan the page for anything that looks like a PDF link by URL or by button text.

The third step is mostly pattern matching learned the tedious way one publisher at a time.

# a DOI landing page is not the PDF; each publisher hides the file differently
if "sciencedirect" in host:
    candidates.append(f"{article_url}/pdfft?download=true")
elif host.endswith("mdpi.com"):
    candidates.append(f"{article_url}/pdf")
elif "onlinelibrary.wiley.com" in host:
    candidates.append(f"{origin}/doi/pdfdirect/{doi}")

Every candidate those steps produce goes through the same byte check as the audio. If it is a PDF it is saved and the search stops.

Some publishers reject a plain request but serve the exact same URL to a real browser that has already cleared their cookie or challenge flow. So before giving up on a candidate the tool loads it in the signed in tab and examines what comes back.

for url in candidates:
    body = try_pdf_url(context, url)          # a plain request first
    if body:
        save(body); break
    # some publishers only hand the file to the real, signed in tab
    response = page.goto(url, wait_until="commit")
    if response and is_pdf(response.body(), response.headers.get("content-type", "")):
        save(response.body()); break

If that fails, the tool writes an Original Paper.webloc shortcut into the episode folder, so I can open the DOI and grab the PDF by hand later. The audio and the write-up are already there, so a missing paper never holds up the rest of the run.

Leveraging OpenClaw

The tool keeps a small state file of episodes it has finished so a run only fetches what is new and it can be interrupted and picked up again without redoing work. A file lock keeps two runs from overlapping and by default it takes at most seven new episodes at a time.

For the unattended run there is one fixed wrapper, scheduled_sync.sh, that takes no arguments. The agent that runs it on a schedule gets exactly one command to call. On my machine that agent is lil-sis, the local model on a Mac Studio that already handles my other nightly jobs through OpenClaw. When the sync finishes it posts a short report to Discord.

What it doesn't do

It cannot beat a publisher that truly blocks automated downloads. When that happens I get the shortcut and a manual step, and that is fine. It leans on iCloud to move the files, so the phone gets them on Apple's schedule rather than the instant they are written. And it works only with my own membership. It downloads what I already have access to, for my own offline listening. It does not share or redistribute anything, and the login session is treated like a password.

The takeaway

I can further enjoy the content provided by journalclub.io and I suggest you take a look if you enjoy reading technical white papers. Episodes now show up on my phone overnight, and the paper is usually there as well when I open the audio on the go.

If you build something like this or point it at your own account, I would like to hear how it goes. You can reach me at sk@skheavyindustries.com.