Methods and code

This is a reference implementation and analysis companion for Teaching a small language model to play. The code exposes the calculations behind the experiment. Model weights, training trajectories, checkpoints and native emulator integration are outside its scope.

From observations to training updates

The training sequence is pretrained Qwen3.5-0.8B → behavior cloning → PPO. The supervised-stage details specify the Courtyard initialization and subsequent multi-level training. Each recorded demonstration pairs the request-time observation with the teacher's action. Whole trajectories stay in their assigned training, validation or held-out split.

• Observation construction and physical button map: normalize the causal state, consolidate shot feedback and encode movement and throw questions. Compare the example request with its Qwen system and user messages. The native adapter supplies positions, events and physics forecasts before this function; the example does not disclose future observations or outcomes.
• Action probabilities and model readout: six movement/A choices and two facing-conditioned throw distributions give 18 joint actions. Dedicated evaluation chooses their maximum, resolving ties by the earliest action. All combinations are listed in the supplement.
• BC objective and update with trajectory weighting: weighted negative log likelihood, accumulated gradients and the historical final-batch denominator.
• Frame-discounted advantages, PPO update and KL checks, and critic architecture: recorded rollout targets stay fixed during each actor update; critic features are detached.
• Head Goon hit replacement, event rewards, reward coefficients and surface rebound attribution: the native hook attributes each damaging hit to its throw, then awards 20 × 0.5^b for its preceding surface rebounds. Episode metrics retain the other event terms. These functions accept observed events; they do not implement a game runner.
• BC settings and PPO settings preserve the model revision, optimizer settings and collection/update schedule.

The implementation lives in src/head_goon_study/ in the research repository. The linked browsing copies are generated from those files. The article describes the measured control loop; timing.py is its small scheduling reference.

Run the included analysis

From the repository root, with Python 3.12 or later:

python paper/code/analysis.py
python -m unittest discover -s paper/code -p test_study.py

The numerical analysis uses only the Python standard library. It reads the 560 exported game records, preserving separate fixed-entry cohorts and matched randomized scenarios. Computed results include every reported outcome. Latency plots use the included applied-response observations.

For the synthetic CPU training checks, install the optional training dependencies:

python -m pip install -e '.[training-core]'
python -m unittest discover -s tests

To rebuild figures and article pages:

python -m pip install -e '.[figures]'
python scripts/build_paper.py
python -m http.server 8000

Open /paper/index.html. The selected gameplay clip is hosted by the game site, separately from the code repository. Tests exercise arithmetic and recorded inputs without loading a policy.

Evidence and provenance

The supplement links outcome records, timings, scenario schedules and source hashes. The extraction map records original and extracted code hashes and each adaptation. The original-archive export utility is retained under provenance/historical/; it is unnecessary for analysis of the included data.