Lately I spend an hour every morning listening to AI podcasts on Voxbatim, a podcast app I built for myself to keep up with AI news while practicing listening and shadowing. The episode I listen to is prepared the night before by a scheduled task that downloads the audio, transcribes it, removes ads, splits it into sentences, generates word timings, corrects the captions, translates them into Chinese, and publishes the result.

It all started with this request to Claude.

Since then, I've kept redrawing the line between what the tools do, what the model does, and what I do.

First, get one episode working

At first I gave Claude only that goal, no steps.

The audio has to be self-hosted. The first version used the publisher's audio URL and the transcript from the feed, and they didn't line up. The publisher inserts ads into the audio each time it's downloaded, so the actual audio had an extra 62 seconds at the start and another 116 seconds at the 16-minute mark, and clicking a sentence jumped to the wrong place. The inserted ads can also differ on every request. So now it downloads one copy of the audio, transcribes that copy with a local whisper, and the site plays that same copy.

Word-by-word highlighting takes a second transcription pass. The word-level times whisper gives by default aren't accurate enough. Accurate ones need its word alignment (DTW). But DTW only works with one speed optimization (flash attention) turned off, and turning it off changes the transcribed text too. In my runs it dropped fillers like "you know" and "like", and shadowing needs every word. So the captions are transcribed as usual, and the word timings come from a second pass that keeps only the times. Once it was built, I heard two problems.

• Every word seemed to light up only after it was spoken, because each word's end time had been used as its start time.
• "repercussions of the crisis at hand" never lit up. The caption line ended before those words were spoken. Of the 263 lines with word timings, 21 had this problem, and whole-sentence highlighting had hidden it. The fix was retime.mjs, which resets each line's start and end from its word timings.

These steps ended up as a skill called add-episode. Scripts like ingest.mjs do the actual processing, and the skill decides the order they run in and makes the judgment calls when reading their output.

While shadowing, I noticed that Apple Podcasts' own transcript was more accurate than whisper's in places, so I wanted to use it as a cross-check. The first time I asked Claude to fetch it, it found that Apple has no public API, and the only known method was reading the Podcasts app's local cache on the Mac, which macOS permissions blocked. So I stuck with whisper.

Later I used Claude Code's /goal command to make "retrieve the Apple Podcasts transcript of How Meta designed Muse" a goal it had to complete, without stopping to ask me. This time the Mac's cache opened but held no transcript, so it looked elsewhere and found the open-source tool getscript. It read the code but didn't install it (by default the tool uploads every transcript it fetches to a third-party site), and reimplemented only the part it needed, which signs with the macOS Podcasts app's system framework, exchanges that for an access token, and then requests the transcript from Apple's API. In about 5 minutes it had 399 sentences and 7,300 words, each word with start and end times. The method became apple-transcript.mjs and a skill.

From then on, every episode was checked against three versions (whisper, Apple, and YouTube's auto-captions), and Claude made the call after seeing the English line, the Chinese translation, the context, and both outside transcripts together. Where at least one outside transcript matched its judgment, it corrected the line directly. It asked me about the rest one by one, whether to change each line and, if so, to what. Corrections never go by majority vote, because two recognizers can agree and both be wrong. In How Meta designed Muse, Apple and YouTube together disagreed with whisper in 19 places. Of those, 5 were corrected, 9 differed only in unimportant function words, 1 couldn't be decided, and 4 kept whisper's version. A majority vote would have overwritten all 4. The first 4 rows of the table are where whisper was kept, and the last 3 were corrected.

The pitfalls of running it on a schedule

At first I wanted to trigger the add-episode skill on a schedule, have it pick an episode by my rules, and take it all the way from download to commit. Producing one episode already worked. The hard part was getting it to start itself, ask me things, and wrap up when I'm not at the computer.

In the third attempt, the prompt even gave the correct script path. A prompt can constrain an agent's decisions when something unexpected happens (see the second item below), but not habits like this one. Conditions that must hold belong in the script.

The items below are all default behaviors of the desktop app or Claude Code (as of desktop app 4.23118.0 and Claude Code 2.1.319).

• Sessions started by a desktop scheduled task can't turn on Remote Control.

  The flow needs to ask me questions on my phone, so it's split into three layers.

  

  1Desktop scheduled task (only keeps time) 2 → launcher session (only runs nightly-pick.sh) 3 → claude --remote-control in its own tmux (does the work, and I can join from my phone)

  The official docs (Remote Control) say, "Remote Control only activates when you explicitly run claude remote-control, claude --remote-control, or /remote-control, unless auto-connect is turned on."

  Nobody is there to turn it on by hand in a scheduled-task session, so it can only rely on auto-connect. In the desktop app's bundle, the auto-connect condition is remoteControlAutoEligible && !scheduledTaskId, which deliberately excludes scheduled tasks.

• Sessions started by a scheduled task get a built-in preamble prompt that says "The user is not present" and "make reasonable choices".

  The launcher session readily takes that as license to find another way, picking an episode and running the flow itself. But it has no Remote Control, so I can't reach it from my phone.

  The prompt needs a line that forbids this and says to report the error directly, with no retries and no workarounds. When the script failed under bash, it only reported the error, saying "I didn't retry it and didn't run the pick another way."

• Claude Code uses the global permission mode by default.

  Mine is plan mode, so the session sat at "Ready to code?" doing nothing. The fix is to pass --permission-mode acceptEdits explicitly at startup.

• When Claude Code thinks I'm sitting at the terminal, it doesn't push to my phone.

  In a test run of the nightly session, the push came back Not sent — this terminal is active.

  The nightly session now starts with the environment variable CLAUDE_CODE_DISABLE_NOTIFICATION_PRESENCE_CHECK=1, which skips that check.

  The official docs (Environment variables) say, "By default the tool skips both the desktop notification and the mobile push when it detects recent keyboard activity or terminal focus."

  Focus means the window currently receiving keyboard input. Claude Code first checks whether the terminal has focus. If it can't tell, it checks whether I've typed anything in the last 60 seconds. The nightly session runs in a background tmux with no terminal window attached, and tmux's focus-events option is off by default, so it doesn't pass focus changes to the programs inside anyway. With focus unknown, the prompt passed in at startup counts as input, so for the first minute after startup it decides I've just typed something and doesn't send the push.

• The desktop app adds a fixed staggering delay (jitter) to every scheduled task.

  I wanted the task to start around 9 p.m., so Claude moved the cron 511 seconds earlier to account for the delay, setting it to 20:52.

  The official docs (Desktop scheduled tasks) say, "Each task gets a small delay of a few minutes after the scheduled time to stagger API traffic. The delay is deterministic: the same task always starts at the same offset."

  Scheduling happens on your machine. While the desktop app is open, it checks the schedule every minute and starts a session locally when a task is due. But each session immediately calls Anthropic's API, so starts are staggered to keep everyone's tasks set for the same time from hitting the API in the same second. In the desktop app's bundle, the offset is first 4 bytes of sha256(task ID) % (cap in minutes × 60) seconds, so it depends only on the task ID, not on the scheduled time. Anthropic sets the cap remotely. Working back from the logs, it's currently 14 minutes, which puts my task at exactly 511 seconds. On the five nights in the logs, the minute check noticed the task between 20:52:30 and 20:52:58 and started it exactly 511 seconds later, between 21:01:01 and 21:01:29.

Running every night means saving tokens

Once the scheduled task was stable, I started saving tokens. I reviewed the most recent episode at the time, Dive Club's James McDonald - Building Beautiful Websites with Claude. The main session ran 87 turns, about 1.69M weighted tokens.

Of this episode's weighted tokens, 45% were cache reads, 31% cache writes, and only 24% output. Cache reads are the cheapest, yet the biggest share, because every turn rereads the entire context. The cost is roughly context size × number of turns. In an earlier run, for example, the 59K-character log ingest.mjs wrote was read into the context whole and then reread for 43 turns. In this episode, Claude paged through the caption comparison results with ad-hoc node -e snippets, one more turn per page.

So I asked Claude to cut tokens and to ask me as little as possible. It changed three things.

• Shrink what every turn rereads.

  The nightly session gets only the tools it uses, loads no MCP servers, and runs with three plugins turned off. 72KB of project conventions in the repo's CLAUDE.md concerned only the front end, so they were moved into a separate file under .claude/rules/, scoped to src/ with paths at the top (Path-specific rules). It loads only when a file under src/ is read or written, and the nightly flow never touches src/. The first turn's context was cut in half.

• Run fewer turns.

  Candidates are now computed in one go by candidates.mjs, and Claude only judges whether the topics fit, so picking (counted up to the first question to me) dropped from 12 turns to 4. The report is now written once instead of twice (the two copies used to be edited and then diffed against each other), so wrapping up dropped from 12 turns to 8.

• Ask less.

  Caption correction no longer asks me line by line. Across the 15 episodes produced up to then, caption correction had asked me 129 questions. I picked Claude's proposal 125 times and went with its recommendation the other 4 times. Earlier still, it once sent six audio clips in a row, asking whether the speaker said "learned to trade" or "learned the trade". I use Voxbatim to practice listening. I'm exactly the wrong person to judge that.

After these changes, I ran one test, which added Syntax's Mourning the Death of Code. In the table, claude -p means the calls the flow makes internally for translation and caption judgments, each one a separate short session.

Each column is a single run on a different episode. A normal run should ask me only twice (picking the episode and the push). The second run hit edge cases the first didn't (whether to trim the silence at the end, a caption boundary that needed to move more than 3 seconds, and so on), which added 4 questions and more tokens than an ideal run.

The main session alone dropped 56% in weighted tokens. Counting the claude -p calls, the total dropped by about a third, and per minute of audio by about half.

Draw the flow so you can question it

By this point the whole add-episode flow had been vibe coded. I stated requirements and Claude implemented them, but I couldn't actually say when each script ran or what it did, so I couldn't tell which parts didn't make sense. To fix that, I asked Claude to draw the current flow as a diagram.

The diagram has four columns, one each for me, the main session, the scripts, and claude -p.

After looking at it, I first asked whether the flows before and after the changes were basically the same.

The answer was that only three steps had changed. A script computes the candidates, correction no longer asks me, and the report is written once. Much of the savings doesn't show on the diagram, because the startup flags, splitting CLAUDE.md, and the rules for saving turns add or remove no steps. They only make each turn read less.

Then I asked, "Can this flow be improved even further?" Claude ran the numbers on the claude -p calls too, and found that the biggest savings were now somewhere else.

• The claude -p calls now cost more than the main session (0.83M vs. 0.74M), and 62% of that was content written to the 1-hour cache at twice the input price and never read again.
• The main session was still chaining commands one at a time. Most of its 57 turns went to starting a script, waiting for the completion notice, grepping the output, and deciding on the next command. All of that is deterministic, yet every turn rereads the entire context.

Of the suggestions it listed, the one with the biggest payoff was an orchestrator script that would chain the deterministic steps and leave the main session only what needs judgment. I chose that one and asked Claude to measure tokens, turns, and time per episode (excluding waits for me) once it was done. Before it started, I questioned a few nodes on the diagram, on the condition that accuracy must not drop, and each one led to a change.

• Why is the audio uploaded twice?

  Trimming the edge silence after the upload left the CDN serving the old file, which needed a manual purge. Now it trims first and uploads once.

• Why does translation come before correction?

  When correction changes words, the translation has to be patched. Now it translates once, after correction. Caption judgments had been using the Chinese as evidence, so we measured first, with two runs using the Chinese and two without. Without it, the judgments were no worse, so translation moved.

• Why use flags to skip what ingest.mjs no longer does?

  That code is now deleted, and the standalone scripts that already existed handle those steps.

• Why do scheduled and manual runs take different paths?

  Both now start from nightly-pick.sh.

Separately, the claude -p calls switched to the 5-minute cache, cutting the write price from 2× to 1.25×.

The result is add-episode.mjs, an orchestrator that runs every deterministic step from ingest.mjs through the commit. The redesign followed one principle:

add-episode.mjs records each step's result in state.json. When something needs judgment, such as the caption lines a spot check picked out, it pauses before committing and writes those items to summary.json. The main session reads only that summary, handles those items, and resumes the script.

Keeping me out became three rules.

• Stop asking about caption moves I can't evaluate.

  The old rule asked me about any caption boundary that moved more than 3 seconds. I had nothing to check it against and always accepted Claude's recommendation. Looking back, all of the roughly 11 moves were correct. Now they're logged for review before going live.

• Only ask questions that come with material I can check.

  This removed every request to listen to an audio clip.

• Check before going live (the push eval).

  After the commit, the main session checks for anything that could affect my listening or reading the next day. If nothing does, it runs push-night.mjs to go live. Otherwise, it asks me, with evidence. The check that only this episode gets pushed lives in the script rather than the prompt, because git push would also push every other unpushed commit.

Here's the flow now.

The main session went from 12 steps to 5, and the places that ask me from 5 to 2. The go-live question comes up only when the push eval doesn't pass.

Finally, I compared a run made entirely with the current flow against the pre-orchestrator one. The latest run, which added Lenny's Podcast's Inside the black box of AI: How AI models actually think | Emmanuel Ameisen (Anthropic), was fully automatic from pick to going live and asked me only which episode to add.

"First round" is the same run in both tables. Total weighted tokens is the sum of the main session's and claude -p's, added before rounding, so in some columns it can differ by 0.01M from the sum of the two displayed values.

The episode was a quarter longer, yet it took about the same time, total weighted tokens fell 35%, and per audio minute they roughly halved.

There's no comparable number for accuracy. It rests on a vibe check while I listen, and I haven't noticed a change. Moving translation later was the only change whose effect on accuracy went through a controlled comparison. I only find what the flow misses when I listen, like the sponsor reads in the next section.

What shouldn't be code yet

Once, partway through Lenny's Podcast's OpenAI’s Head of ChatGPT: We’re entering a new era of AI (again) | Tibo Sottiaux, I heard a sponsor read.

DAI was found while building the first version, and the next day Claude wrote a script to remove it. Twice after that, a sponsor read in a new episode slipped past the script's rules and stayed in. Both times Claude and I cut it right there in the conversation, but Claude never suggested building that into the script. So once the nightly run took over, sponsor reads stayed in the episodes it added. That's how the sponsor reads in this episode slipped through.

Of the episodes added after the scheduled task started, 6 went live with sponsor reads. The nightly Claude read them, but its rules only covered DAI, so four times it wrote in its reports that they were "part of the show".

The next change is a new step after ingest.mjs, in which Claude reads the caption file index.csv and writes each sponsor read's start and end lines, times, and brand into the report. I haven't decided yet whether to cut them from the audio or skip them during playback.

What the tools, the model, and I each do

Looking back, the division of labor shifted along the way. At first Claude did everything while I answered its questions. Once things worked, the stable steps went into scripts one by one. After the orchestrator, it looks like this.

• Scripts do the deterministic work, such as starting the session, chaining the steps, and checking that only this episode gets pushed.
• Claude does what needs judgment, which means correcting captions, translating, handling what the script hands back, and judging before going live whether anything would affect listening and reading.
• I only pick the episode and, when there's a risk, decide whether it goes live. When I notice something off while listening, I go back and change the flow.

This approach doesn't fit every project. If the inputs, outputs, and failure conditions are clear from the start, just write the script. Letting the model explore first suited this project because at the start I didn't even know where the captions would come from or what it would take to make them shadowable.

In the end, what matters is whether there's something I can practice with right away when I open Voxbatim in the morning, and how much of my time it took the night before and the day after.