AI Builders and Learners SF

Supervised fine-tuning teaches a model to imitate examples. Reinforcement fine-tuning teaches it to optimize an objective you define, which is how recent open models picked up reasoning, reliable tool use, and consistent output formats. GRPO (Group Relative Policy Optimization) is the method behind a lot of that work, and it is simpler and cheaper to run than the PPO setups that came before it.

​GRPO drops the separate value model that makes PPO expensive. For each prompt, it samples a group of completions, scores them with a reward function, and uses the group's own spread to estimate which responses were better than average. You train on prompts plus a reward function instead of a large hand-labeled dataset, and verifiable rewards (did the answer match, did the output parse, did it hit the format) get you a long way without preference annotation.

​In this hands-on workshop, we'll fine-tune an open-weight LLM with GRPO using Hugging Face TRL, write reward functions that actually shape behavior, and deploy the result behind a simple UI. The whole pipeline runs on Flyte 2/Union, so data prep is cached, runs are reproducible and recoverable, and the same code scales from a laptop to a multi-node cluster without rewrites.
​By the end, you'll have a working GRPO-trained model and a reusable RL pipeline you can point at your next task.

​What we'll cover

• ​A practical intro to GRPO
• ​Writing reward functions
• ​Sandboxes for safe code execution during training
• ​Fine-tuning an open-weight LLM with Hugging Face TRL's GRPOTrainer
• ​Orchestrating with Flyte 2: cached data prep, GPU-aware training, and durable, reproducible runs at any scale
• ​Deploying the model with a UI, with a path to scaled inference

​What you'll leave with

• ​An LLM fine-tuned with GRPO against a reward function you wrote
• ​A reusable RL training and deployment pipeline you can adapt to your own task
• ​The knowledge to design reward functions and prompt sets for future GRPO projects

​Most agents use tools by emitting one structured tool call at a time. The model picks a tool, waits for the result, picks the next, and every intermediate result flows back through its context. As the number of tools and steps grows, that gets expensive, slow, and brittle.

​Code mode flips this. Instead of calling tools one at a time, the agent writes code that calls them as functions, runs that code in a sandbox, and returns only what matters back to the model. Why this works well in practice: models are trained on far more code than tool-call traces, so they compose operations more reliably in a language they already know. Loops, branching, filtering, and data transforms happen in the sandbox instead of as separate round-trips. Large intermediate results stay out of the context window, and tool definitions load on demand instead of all up front. The result is an agent that does more per step and spends fewer tokens doing it.

​The obvious worry is letting a model write and run code. We'll handle that with Flyte's sandboxed orchestrator: the generated code can do pure control flow like loops, branching, and wiring tool calls together, but has no access to the filesystem, network, or OS. The actual tool or MCP calls get dispatched to isolated container tasks. You get the flexibility of letting a model write code without handing it the keys to the machine.
​In this hands-on workshop, we'll build a code mode agent that writes and executes code to call tools, then deploy it behind a simple UI. The whole pipeline runs on Flyte 2/Union, so runs are durable and reproducible, steps are cached, and the same code scales from a laptop to a multi-node cluster without rewrites.
​By the end, you'll have a working code mode agent and a reusable pattern you can point at your own tools.

​What we'll cover

• ​Why direct tool-calling breaks down at scale, and what code mode changes
• ​Running model-generated code safely in Flyte's sandboxed orchestrator, with heavy work dispatched to isolated tasks
• ​Orchestrating with Flyte 2: cached steps, durable runs, and execution that scales
• ​Deploying the agent with a UI, with a path to production

​What you'll leave with

• ​A working code mode agent that writes and runs code to use tools
• ​A reusable pattern you can adapt to your own tools and MCP servers
• ​A clear sense of when code mode beats direct tool-calling, and when it does not

July's book is "RAG with Python Cookbook"!

This is a casual-style event. Not a structured presentation on topics. Sometimes, the discussion even drifts away from the chapters, but feel free to grab the mic to help steer it back.

Feel free to join the discussion even if you have not read the book chapters! :)