Eric J Ma's Website

I took part in Round 2 of the alphaXiv x marimo competition. The task was to pick a recent ML paper, build a marimo notebook that teaches it, and submit a five-minute walkthrough. I had two papers I wanted to tackle, BitNet's 1.58-bit weights and JiT's "predict the clean image, not the noise," and a tool I've been enjoying: pair-programming marimo notebooks with a coding agent.

The real surprise was how much of the paper I learned by the end of the sprint. I came out the other side understanding both papers more viscerally than if I'd read them quietly for a week. The mechanism was the artifact we were building together. The notebook gave me something to poke at, and poking taught me what reading alone couldn't.

Judgment is the slow part

Judgment takes years. It's the feel for when a result is wrong, the instinct for which experiment to run next, and the sense that a curve looks fishy before you can articulate why. Those instincts compound slowly, and no agent hands them to you. I wrote about this recently, and I still believe it.

But there's a prerequisite elementary step: onboarding onto the basic vocabulary and mental models of a new paper or field. Things like wrestling with what "straight-through estimator" actually means in your hands, or making sense of why predicting the clean image might be easier than predicting the noise, when the established literature screams the opposite. That part has always been slow for me, because it's a wrestling match with jargon. I read a sentence, hit a term, go look it up, lose the thread, reread, repeat.

The onboarding can go faster, even if the judgment can't.

Aesthetics do pedagogical work

Before I started building, I had the agent study the three winning notebooks from round 1. Partly this was competitive intelligence, to obtain a clear bar for what good looked like. But there was a separate surprise: I genuinely enjoyed reading them! The best ones opened with an HTML hero banner carrying the paper title and a one-line thesis. They paired that with interactive widgets where you dragged a slider and watched a distribution react, which made the concepts tactile. Some of them were also game-like. You'd try to find a hidden structure, and being wrong was part of the play rather than a verdict. And they closed with a summary callout that re-stated the conclusions cleanly.

What makes these notebooks teach is the loop they put you in: predict, then see whether you were right. The interactive widgets drive that loop. Everything else exists to support it: a hero banner gives you a mental anchor before the math lands, and game-like elements turn being wrong into play, which is where the learning actually lives. When being wrong costs nothing, you try more things, and trying more things is what teaches you.

Pair-programming the paper, not just the code

I fired up a marimo notebook on a molab sandbox with an NVIDIA RTX PRO 6000 Blackwell, and pair-programmed with GLM-5.2 through opencode. I had the paper and the questions, but I didn't have the patience or tenacity to push through the initial energy barrier on my own. The agent had both, in infinite measure.

The shape of the work looked like this. I'd ask a question, sometimes framed as "explain this to me like I'm five." The agent would answer in plain words, and then build something that let me feel the answer. For BitNet, that meant a weight heatmap crushing 32-bit floats into three colors, a slider that let me drag the precision and watch accuracy react, and a live training race on the GPU between binary, ternary, and full-precision models. For JiT, it meant a manifold game where I dragged noisy points and watched what "predict clean" versus "predict noise" actually did to the targets.

Here's the thing about ELI5 as an operating mode. When I asked for it, I stopped wrestling with jargon and started wrestling with concepts, analogies, and ideas. That's the match I actually wanted. The jargon-wrestling was always fake difficulty, syntax I had to look up before I could even formulate the real question. ELI5 crushed the syntax barrier and put me in front of the idea immediately.

The artifacts taught me what the abstract couldn't

The clearest example came halfway through the BitNet notebook. My mental model of the paper went something like: take a pretrained language model, crush its weights to {-1, 0, +1}, and it still works. That's the one-line version, and it's what you'd believe if you stopped at the abstract.

I told the agent to demonstrate it on a real pretrained GPT-2. The perplexity went from 51 to 33,000. The output collapsed to "present present present present."

But that's not what the paper claims! BitNet models are born ternary. They're trained from scratch with the straight-through estimator, which lets gradients flow through the rounding step during training. Retrofitting ternary onto a model that spent its whole pretraining life tuning exact magnitudes breaks the calibration of every layer at once. The agent showed me, by running the experiment I asked for and getting a result that contradicted my hunch.

So I asked the obvious follow-up: "If we can't retrofit, what's the fix?" That curiosity became the notebook's novel extension. Fine-tune the retrofitted model with the same straight-through estimator, on a small AI-domain corpus, and see what happens. Across three model families (distilgpt2, Qwen2.5-0.5B, Pythia-410m), perplexity recovered by about two orders of magnitude. The retrofitted text went from repetition to real on-topic English about artificial intelligence. The mechanism that makes BitNet trainable from scratch is also the mechanism that partially rescues a retrofitted model. That extension exists because I was wrong about the abstract first. The agent's experiment surfaced the contradiction; reading alone, I could have stayed wrong indefinitely.

The second paper, JiT, had its own version of this. The paper's title is "Back to Basics: Let Denoising Generative Models Denoise," and I couldn't find a quantitative denoising benchmark inside it. So I got curious. We trained a small x₀-predictor on CIFAR-10, then used it as a one-pass denoiser on held-out test images. It improved the peak signal-to-noise ratio by up to 22 dB, beating a Gaussian-blur baseline at every noise level. The conceptual payoff was even cleaner: denoising and generation turn out to be the same operation, both projection onto the data manifold. That extension lives in the notebook now, and it exists because I asked "what does the title actually mean in practice?" and the agent helped me build the answer. (This JiT notebook is the one that ended up winning Round 2.)

Five minutes forced clarity

The competition required a five-minute walkthrough video, and that constraint turned out to be its own learning tool. Five minutes forces you to decide what actually matters. Combine that with ELI5 and you get a forcing function: explain the paper simply, in spoken English, in five minutes, to a viewer who hasn't read it. If you can't, you don't understand it yet.

The agent drafted a script structured as a guided tour through the cells. I sent it to a panel of other subagent reviewers and they tore it apart. Repetitive, shallow, vague.

That review was a gift. Rewriting the script with the agent's help forced me to decide, in plain spoken English, what each section actually taught and why it mattered. Every time I hit a sentence I couldn't say cleanly, that was a gap in my own understanding. Vague claims like "it works well" had to survive a real question like "compared to what, at what scale?" Oversells had to be walked back to what the data actually supported. Limitations I'd been quietly hoping no one would ask about had to be named out loud, because a judge (or at least, my thesis committee) absolutely would.

Using the agent to help me communicate clearly made the script much better, and making the script better made my understanding better. Writing in my own voice, with the cells in front of me, is where the last misunderstandings surfaced. Rehearsing for the presentation and trying to anticipate questions about each cell forced me to be intellectually honest about my own gaps. Reading alone makes it easy to hide those gaps from yourself.

I stayed in the driver's seat

This is the part I want to be careful about, because it's easy to misread. The agent did a lot of the typing. It wrote the training loops, built the custom anywidgets, ran the parameter sweeps, refactored the underscore-prefixed names I hate. If you'd watched the session from the outside, you'd have seen the agent producing most of the code.

But I was driving. Every prompt I sent encoded a decision. The agent could type, but it couldn't make the judgment calls. Recognizing that the collapsed retrofit was a problem worth solving was mine, and so was seeing that JiT's blurry generations were a real flaw to fix, not just how diffusion looks (hence my prompt: "figure out how to solve the image blur problem on a loop until you are sure you have solved it"). And catching that the script oversold the recovery took a feel for what honest claim the data would support, which is the kind of judgment that takes years.

The agent's speed amplified my judgment, exactly the way I wrote about before. It compressed the onboarding loop, from a week of reading and rereading into an evening of asking and building and poking. The years I'd spent building judgment finally had leverage.

Your curiosity sets the direction

Your curiosity has to set the direction. The agent is patient, the agent is fast, the agent will happily build whatever you ask for and a dozen things you didn't. The discipline is yours. Decide what you actually want to understand. Ask for it in plain words. Demand artifacts you can poke, not paragraphs you can read. When the artifact contradicts your hunch, follow the contradiction; that's where the learning lives. And when you think you understand the whole thing, write the five-minute spoken script. The gaps will announce themselves.

Some things genuinely take time; onboarding doesn't have to be one of them. Keep your hands on the wheel.