AP Statistics & Warhammer

I have one kid who is insanely interested in Warhammer. This is a Dungeons & Dragons-like game where armies battle each other. Things have evolved since my DnD days. There are units that kids buy for hundreds of dollars, and have long-running fights (and arguments) over which army is better. The same kid is starting AP statistics next year.

So, with the power of AI, I present – Warhammer Statistics – a guide to understanding AP Statistics content with the theme of Warhammer. It’s a taste of what his next class will be (while we are in doldrums of summer).

Claude Sonnet 4 via Github co-pilot was used for this and it took some tricks. Of course, the first prompt was a bit vague:

I need to help my son pass AP Statistics. He's really into WarHammer, and so I'd like to build a web site that guides him through statistics terms, examples, and questions. The progress should be slow, ensuring mastery, but also engaging - making sure he pays attention. Please create a web app that teaches statistics bit by bit by referencing war-hammer.

But it came back with a solid UI, lots of Warhammer references, some gamification, and some initial lessons. The lessons though didn’t match AP Statistics concepts though. And the initial UI let the student skip right through (which of course the kid would do). And the “XP Earned” that it created wasn’t motivating. I’d rather have rank advancement then a high-score goal.

After fixing those, I noticed that AI likes to build monolithically. It was putting all “lesson” data in a giant javascript file. If you’re familiar with AI context windows, you can see why this is a problem. Claude Sonnet 4 has a 200k context window limit, which translates to about 150k words. With all the code and the prompt history, I was surpassing that. So time to refactor into meaningful chunks that can be passed to the AI without it losing it’s place. Why not ask AI to do it?

I'd like to make the code easier for you to update, especially as I want to add more activities and make the quizes longer. Can you re-factor this code base to make it more extensible and scalable?

Ooops. This took some time and quite a few iterations for it to finish – and resulted in missing function references. It looks like the AI took a boil the ocean approach to it – which ended up with modularization eventually, but also broke things along the way.

A better prompt would’ve been to focus on certain aspects bit by bit and keep it focused on the user’s perspective. “I’d like to add lessons easily – each containing an english explanation, warhammer application, and activity to verify understanding. Starting with Chapter 1, please refactor the lesson javascript into more more modular components”.

Another learning was around my ability to test various lessons. To do so, I had to be the student and go through the various lessons before it- scoring well as I go along. Which was kind of fun to do the first few times, but became a drag. One small prompt worked great on this:

as I'm testing this, especially in adding lessons and activities, I'd like to skip ahead to chapters that haven't been unblocked, or quizzes that haven't been unlocked. This should not be a "student" mode. The student still needs to do all the necessary lessons and activities. Can you create an "admin" mode with a password that allows me to skip around?

The only learning on this prompt was that it relied on the gold-old plain text password in a config file approach — and made the password “password”. Yes, the kid would’ve figured that out easily!! A regular has and harder password though would give him some pause. Easy fix.

Just a few more learnings as I iterated on this:

1. The AI likes to hallucinate on what’s required for AP statistics – even when given a reference to what’s required. So a careful, thorough review was needed to verify I wasn’t leading the kid astray.
2. It was always better to pass an error from the browser’s developer tools than it was describing the user experience issue – especially as the code base grew.
3. One issue I haven’t figured out yet is how to enable in-IDE automated web-app testing for the AI. This would’ve helped immensely find and resolve errors for me. Any ideas?