# Parameter Golf

## Docs

- [Model Architecture](https://mintlify.wiki/openai/parameter-golf/concepts/model-architecture.md): The baseline GPT architecture in train_gpt.py: U-Net skip connections, GQA, RoPE, relu² MLP, and tied embeddings.
- [Model Quantization & Compression](https://mintlify.wiki/openai/parameter-golf/concepts/quantization.md): How the baseline compresses a bf16 model into a 16 MB int8+zlib artifact using per-row quantization and zlib level-9 compression.
- [Scoring & Evaluation](https://mintlify.wiki/openai/parameter-golf/concepts/scoring.md): How bits-per-byte is calculated, what counts toward the 16MB artifact, and the submission verification process.
- [Tokenizer & Vocabulary](https://mintlify.wiki/openai/parameter-golf/concepts/tokenizer.md): How the challenge supports custom tokenizers, how bytes-per-token is counted for BPB, and the tradeoffs of different vocabulary sizes.
- [Data setup](https://mintlify.wiki/openai/parameter-golf/getting-started/data-setup.md): Download and configure the FineWeb dataset and tokenizer variants for training.
- [Local training (Apple Silicon)](https://mintlify.wiki/openai/parameter-golf/getting-started/local-training.md): Run your first parameter golf training job on a Mac with Apple Silicon using MLX.
- [Remote GPU training](https://mintlify.wiki/openai/parameter-golf/getting-started/remote-gpu.md): Scale from local experiments to cloud H100s on Runpod for full leaderboard-eligible runs.
- [Parameter Golf](https://mintlify.wiki/openai/parameter-golf/introduction.md): Train the smallest language model you can that fits in 16MB. Best model wins.
- [Leaderboard](https://mintlify.wiki/openai/parameter-golf/leaderboard.md): Current standings for the Parameter Golf challenge, scored by bits-per-byte on the FineWeb validation set.
- [Data Pipeline](https://mintlify.wiki/openai/parameter-golf/reference/data-pipeline.md): How FineWeb training and validation data is downloaded, tokenized, stored, and streamed during training.
- [Environment Variables](https://mintlify.wiki/openai/parameter-golf/reference/environment-variables.md): Complete reference for all environment variables used to configure training, quantization, distributed setup, and data pipeline.
- [FAQ](https://mintlify.wiki/openai/parameter-golf/reference/faq.md): Frequently asked questions about the Parameter Golf challenge rules, scoring, submissions, and evaluation.
- [Hyperparameters](https://mintlify.wiki/openai/parameter-golf/reference/hyperparameters.md): Complete reference for all configurable hyperparameters in train_gpt.py, organized by category.
- [Non-Record Submissions](https://mintlify.wiki/openai/parameter-golf/submission/non-record-submissions.md): How to submit creative, experimental, or unlimited-compute approaches that don't claim a new SOTA record.
- [Record Submissions](https://mintlify.wiki/openai/parameter-golf/submission/record-submissions.md): Requirements and checklist for submitting a new SOTA record to the parameter-golf leaderboard.
- [Submission Requirements](https://mintlify.wiki/openai/parameter-golf/submission/requirements.md): What every valid parameter-golf submission must include, from file structure to the submission.json format.