You're in a Research Scientist interview at Google. Interviewer: We have a base LLM that's terrible at maths. How would you turn it into a maths & reasoning powerhouse? You: I'll get some problems labeled and fine-tune the model. Interview over. Here's what you missed:

When outputs are verifiable, labels become optional. Maths, code, and logic can be automatically checked and validated. Let's use this fact to build a reasoning model without manual labelling. We'll use: - @UnslothAI for parameter-efficient finetuning. - @HuggingFace TRL to apply GRPO. Let's go! 🚀

What is GRPO? Group Relative Policy Optimization is a reinforcement learning method that fine-tunes LLMs for math and reasoning tasks using deterministic reward functions, eliminating the need for labeled data. Here's a brief overview of GRPO before we jump into code:

1️⃣ Load the model We start by loading Qwen3-4B-Base and its tokenizer using Unsloth. You can use any other open-weight LLM here. Check this 👇

2️⃣ Define LoRA config We'll use LoRA to avoid fine-tuning the entire model weights. In this code, we use Unsloth's PEFT by specifying: - The model - LoRA low-rank (r) - Modules for fine-tuning, etc. Check this 👇

3️⃣ Create the dataset We load Open R1 Math dataset (a math problem dataset) and format it for reasoning. Each sample includes: - A system prompt enforcing structured reasoning - A question from the dataset - The answer in the required format Check this code 👇

4️⃣ Define reward functions In GRPO we use deterministic functions to validate the response and assign a reward. No manual labelling required! The reward functions: - Match format exactly - Match format approximately - Check the answer - Check numbers Check this out 👇

5️⃣ Use GRPO and start training Now that we have the dataset and reward functions ready, it's time to apply GRPO. HuggingFace TRL provides everything we described in the GRPO diagram, out of the box, in the form of the GRPOConfig and GRPOTrainer. Check this out👇

6️⃣ Comparison Again, we can see how GRPO turned a base model into a reasoning powerhouse. Check this out👇

Before we conclude, let me address an important question: When should you use reinforcement fine-tuning (RFT) versus supervised fine-tuning (SFT)? I created this diagram to provide an answer: