The control plane could not be reached.
Retry
My projects — Sign in to load
My trainings — Sign in to load
Running now —
Failed · need attention —
Projects will appear after connecting.
Failed runs and their cause will appear here.
A project is one model line: a single Hugging Face repository whose trainings publish as numbered versions, plus the jobs, RL objectives and datasets behind them. Promote a version to make it the model other work can start from.
Judge and version Holdout configuration
Choose a configuration to evaluate and compare models.
Select models to evaluate or compare their results.
Run history
Comparison Shared configuration
Model A Model B
Differences use examples scored for both models. Missing scores are shown as —.
Criterion · shared scored examples Model A Model B Difference B − A
Shared evaluation configuration All runs use this snapshot. Create a new evaluation to change the dataset or judge.
Back to models
Model / revision Inference / Judge Scored / failures Mean score Approval Actions
Close
For Judge access or connection errors, check the VPN on the Mission Control server and LiteLLM credentials. Once access is restored, use Judge saved responses to retry without running inference again.
Saved run
Outcome All responses Inference failures Judge execution failures Errors reported by Judge Rejected by Judge Error type All error types
Copy all
Example Input Response / inference Judge Gold
Close
Close
Overview & history
Runs
Metrics
Config & parameters
Loading run configuration…
Favorite models, shared with every project for demos and as training sources. Every other model stays inside its project — open the project and use “Favorite” to list it here.
Queued jobs — Requested GPUs — Ready definitions — Assigned users —
Each objective is one reward program for small-model RL: name it, prepare the required thinking bootstrap and independent GRPO data, write and test the Python reward, certify it on the pinned data, then launch GRPO.
1 Objective Name, project and status 2 Bootstrap Teach thinking with supervised data 3 GRPO data Prompts and gold answers 4 Reward code Program and test the reward 5 Certify Prove it on the pinned dataset 6 Launch Training recipe and GRPO job
Step 2 · Bootstrap Teach the model to think before RL Choose supervised examples with a non-empty thinking column and a final answer. This dataset has its own split, revision and column mapping, independent of the RL data in step 3.
Thinking bootstrap dataset · required Supervised rows with a literal thinking column and a final answer. Cold Start finetunes the source model on them before RL so it learns the reasoning envelope. Every RL objective includes this supervised stage before RL.
Not inspected
Open Settings › Datasets
Inspect the thinking bootstrap dataset to see its columns, sample rows and task values.
2 Thinking columnsMap dataset columns to the reasoning contract. The thinking role is locked to the literal thinking column and every selected row must fill it.
Instruction source Fixed instruction Dataset column Fixed instruction Required Preserves line breaks. Use the teacher’s task policy, including what to preserve or remove. Choose a dataset column when instructions vary by row. The token profile includes this instruction. Instruction Input Thinking Locked Final answer Task Optional When unmapped, every row uses the objective task.
Back to objective Continue to RL data
Step 3 · RL data
Choose the prompts and gold answers for RL
RL generates answers for each prompt and scores them with your reward. Choose this dataset independently of the supervised bootstrap. Gold answers are available to the reward only; the model sees the instruction and input.
Not inspected
Training dataset · RL prompts Usually the larger task dataset. Instruction and input become the prompt; an optional task column keeps only rows of this objective; a reference answer supplies the gold text for rewards that compare predictions.
Not inspected
Open Settings › Datasets
Inspect the training dataset to see its columns, sample rows and task values.
1 Prompt columnsMap the training dataset onto the prompt contract. Roles are guessed from column names after an inspection; adjust them here.
Instruction source Fixed instruction Dataset column Fixed instruction Required Preserves line breaks. Use the teacher’s task policy, including what to preserve or remove. Choose a dataset column when instructions vary by row. The token profile includes this instruction. Instruction Required Input Optional Task Optional When unmapped, every row uses the objective task. Reference answer Optional Available to reward code as source.reference_text during RL. Never included in the model prompt.
Back to bootstrap
Continue to reward code
Step 4 · Reward code
Python reward studio
Write reward(sample) with the RL Lab helpers, run it instantly on hand-written probes or on rows sampled from the dataset, and inspect every score, component and trace. Prefer an IDE or an LLM? Download the bundle and run the same samples locally. The exact certified code is embedded into RL.
Unsaved draft
LLM judge
Compare gold and predicted answers with Qwen3.8-27B
Judge reasoning effort Low Medium Extra high Choose the judge's reasoning depth independently of its token limits. Low remains the default; changing effort leaves the output budget unchanged. Judge context tokens Prompt plus judge output. Leave room for gold, prediction and rubric as well as the output budget. GPU preflight checks whether the allocation fits. Judge output tokens · thinking + final score Total budget for reasoning and the final scalar. Must be smaller than its context. Truncated responses are rejected and retained in the audit. The accepted answer remains one number in [0, 1]. Judge thinking token cap (optional) Leave blank to let thinking finish naturally within the total output limit. An explicit cap can interrupt thinking; responses that hit the cap are rejected and retained in the audit. If set, reserve at least 32 output tokens for the scalar score. Judge timeout (seconds) Allow time for reasoning before the scalar score, especially with a large output budget. Concurrent judge requests Default 8; configurable from 1 to 64. Using 32 needs at least 32 eligible B×G candidates (prompts per round × completions per prompt) and enough judge GPU capacity for their context and output. This is a request limit, not a throughput guarantee.
vLLM serves Qwen/Qwen3.8-27B with FP8 weights on one dedicated GPU. Select at least 2 GPUs at launch: one for the judge and the rest for training.
Use Certify to validate the dataset and code here. Judge probes require the allocated GPU and run before bootstrap; a failed preflight stops training.
Judge prompt template (Jinja2) Required Include gold and predicted below. The runtime substitutes the full texts. The rl_judge_prompt() helper retains this exact template.
{{ gold }} · required
{{ predicted }} · required
{{ input }} · optional
{{ thinking }} · optional
Templates support text and the four field substitutions only; loops, conditions, filters and attribute access are not supported.
Return a single number in [0, 1]: 0, 0.75 or 1.0. Decimals use a point. JSON objects such as {"score": 0.75}, quoted numbers, extra text, nonfinite values and out-of-range scores are rejected.
Preview substituted prompt
Uses a sampled row if available, with gold copied as an illustrative prediction; otherwise uses a small OCR example. This renders the template only. It does not call the judge or measure a score.
Render preview
Preview has not been rendered.
Replace reward with judge example
The example rejects incomplete thinking, then calls the judge with gold and predicted final text. Keep the helper call when this option is enabled. Disabling requires removing the helper call from your code.
Starter example
Catch up example
Movement explanation example
Load a saved reward function
Load from library…
Choose source model
Ready · Ctrl/⌘+Enter runs the probes
Ln 1, Col 1
Unsaved code
Ctrl/⌘ +Enter run · Ctrl/⌘ +S save · Ctrl +Space complete
Runtime API
Define reward(sample)
The sample is read-only and identical in simulation and RL. Split logic into top-level helper functions and import re, math, string or unicodedata when you need them. Autocomplete offers only the RL Lab helpers and snippets; click a helper or snippet to insert it.
Sample
Helpers
Snippets
Return
Rules
{
"task": str,
"prompt": str,
"completion": str,
"thinking": str,
"thinking_tokens": int | None,
"thinking_soft_limit_tokens": int,
"final": str,
"reasoning_valid": bool,
"reasoning_error": str | None,
"source": {"source_text": str, "reference_text"?: str},
"stop_reason": str,
"words": int,
}
taskstr Objective slug the program is certified for (for example catchup_digest).
promptstr Raw task input without chat-template markup: the instruction plus the input field of the dataset row.
completionstr Complete model output including the <think>…</think><answer>…</answer> envelope.
thinkingstr Parsed reasoning text; empty when the envelope is missing.
thinking_tokensint | None Exact parsed thinking token count using the pinned student tokenizer, without special tokens or envelope. Unavailable is None, never a word/character estimate. Programs referencing this field require the tokenizer before any judge call.
thinking_soft_limit_tokensint Trusted root GRPO configuration value (default 8192, integer >= 1000), not dataset-supplied metadata. The visible OCR reward uses 0.01/500 * (max(0, thinking_tokens - 1000) + max(0, thinking_tokens - thinking_soft_limit_tokens)). This soft threshold increases marginal penalty; it does not stop generation. Custom programs must use it explicitly.
finalstr Parsed final answer; empty when the envelope is invalid.
reasoning_validbool True when the envelope parsed cleanly. Gate on this first.
reasoning_errorstr | None Parser error message when reasoning_valid is False.
source{"source_text": str, "reference_text"?: str} Grounding text equals prompt. Optional reference_text is the mapped gold answer, available only to the reward and never to the model prompt.
stop_reasonstr eos, length or stop_sequence. length means the generation hit max_new_tokens and is probably truncated.
wordsint Deterministic word count of final.
sample["thinking_tokens"] counts the parsed thinking with the pinned student tokenizer, without special tokens. It is not a word count, character estimate or judge token count. Select the registered source model in Launch for probe evaluation and bundle export. When unavailable, the value is None; rewards requiring it cannot be scored.
Filter helpers
llm_judge(gold, predicted, prompt) -> float
Compare gold and predicted text with the configured Qwen judge on its exclusive runtime GPU. The prompt must exactly match the saved Jinja2 template, including gold and predicted substitutions. Use {{ gold | tojson }} and {{ predicted | tojson }} for lossless JSON string framing; only this filter on a bare field is allowed, without arguments or filter chains. Plain {{ gold }} and {{ predicted }} remain available for custom templates; {{ input }} and {{ thinking }} are optional for custom rubrics. The OCR outcome rubric uses only gold and predicted to judge final-answer semantic transcription fidelity. Source input and student thinking remain in separate audit fields; retaining them does not include them in the rendered judge prompt. The judge's own reasoning is a separate audit trace, not the student's thinking. Envelope checks and the configured student-token penalty remain separate reward terms. Returns any single finite number in [0, 1], without category rounding (for example 0.75); JSON objects, quoted numbers and extra text are rejected. There are at most four calls per sample. Infrastructure errors fail the run rather than becoming scores. Controller validation checks code and dataset only; judge execution requires GPU allocation and real preflight before bootstrap.
score = llm_judge(sample["source"]["reference_text"], sample["final"], rl_judge_prompt())
clamp(value, minimum=0.0, maximum=1.0) -> float
Bound a number inside [minimum, maximum]. Bounds must be finite and ordered.
quality = clamp(words / 120.0)
safe_divide(numerator, denominator, default=0.0) -> float
Divide two finite numbers and return default when the denominator is zero.
precision = safe_divide(matched, len(answer_links))
mean(values) -> float
Arithmetic mean of finite numbers; 0.0 for an empty input. Bounded to 10,000 items.
score = mean(components.values())
words(text) -> tuple[str, ...]
Lower-cased alphanumeric tokens (Unicode letters and digits).
first_words = words(final)[:20]
word_count(text) -> int
Number of tokens returned by words(text).
final_words = word_count(final)
contains(text, needle, case_sensitive=False) -> bool
Case-insensitive substring test by default.
has_date = contains(final, "13/07/2023")
count_occurrences(text, needle, case_sensitive=False) -> int
Number of non-overlapping needle occurrences; 0 for an empty needle.
paragraph_breaks = count_occurrences(final, "\n\n")
links(text) -> tuple[str, ...]
Every http(s) URL in text with trailing punctuation removed.
source_links = links(source)
fraction_present(values, text) -> float
Share of non-empty values found in text (case-insensitive). 1.0 when there is nothing to find.
link_recall = fraction_present(source_links, final)
ordered_fraction(values, text) -> float
Share of values that appear in text in the same order as given; rewards chronology.
chronology = ordered_fraction(source_links, final)
token_overlap(left, right) -> float
Jaccard overlap of the word sets of two texts, from 0.0 to 1.0.
grounding = clamp(token_overlap(final, source) * 3.0)
Plain Python the sandbox also accepts
Builtins: abs() all() any() bool() enumerate() float() int() len() max() min() range() round() sorted() str() sum() zip()
Methods: .capitalize() .casefold() .count() .endswith() .find() .isalnum() .isalpha() .isdigit() .isspace() .join() .lower() .lstrip() .partition() .removeprefix() .removesuffix() .replace() .rsplit() .rstrip() .split() .splitlines() .startswith() .strip() .title() .upper() .get() .items() .keys() .values() .search() .match() .fullmatch() .findall() .finditer() .sub() .subn() .group() .groups() .groupdict() .start() .end() .span()
Imports (top of the program): import re import math import string import unicodedata
Return a floor reward when the thinking envelope is broken.
if not sample.get("reasoning_valid", False):
return {
"score": -1.0,
"components": {"reasoning_format": -1.0},
"trace": {"reasoning_error": sample.get("reasoning_error")},
}
Penalize generations that exhausted the token budget.
if sample.get("stop_reason") == "length":
return {
"score": -1.0,
"components": {"incomplete_generation": -1.0},
"trace": {"stop_reason": "length"},
}
Reward answers inside a word-count window.
final_words = word_count(final)
length_quality = clamp(min(final_words / 30.0, 160.0 / max(final_words, 1)))
Precision, recall and chronology of source links.
source_links = links(source)
answer_links = links(final)
link_precision = fraction_present(answer_links, source) if answer_links else 0.0
link_recall = fraction_present(source_links, final)
chronology = ordered_fraction(source_links, final)
Weighted components Combine named components whose sum is the score.
components = {
"grounding": 0.5 * clamp(token_overlap(final, source) * 3.0),
"length_quality": 0.3 * length_quality,
"clean_ending": 0.2 * (1.0 if final.strip().endswith((".", "!", "?")) else 0.0),
}
return {"score": sum(components.values()), "components": components, "trace": {}}
Test the answer against a regular expression. Add "import re" at the top of the program.
cites_article = re.search(r"art\.\s*\d+", final, re.IGNORECASE) is not None
date_count = len(re.findall(r"\b\d{2}/\d{2}/\d{4}\b", final))
# Simple
return 0.75
# Auditable
return {
"score": 0.75,
"components": {"grounding": 0.5, "format": 0.25},
"trace": {"matched": 3},
}
Component names are lowercase slugs (a-z, 0-9, _). Component values must sum exactly to score. Trace values must be JSON-safe and stay under 16 KiB per example. GRPO only uses the ranking inside a group of completions, so the order your score induces matters more than absolute totals.
Allowed
Exactly one top-level def reward(sample), plus any number of top-level helper functions with plain positional parameters (literal defaults allowed). Helpers may call each other but never recursively. import re, math, string and unicodedata at the top of the program (also "import re as regex" or "from math import sqrt"); only the documented members of each module are available. Assignments, if/elif/else, for loops (nested at most twice), break, continue and return. Numbers, strings, f-strings, lists, tuples, sets and dicts, comprehensions and generator expressions. Arithmetic, comparisons, boolean logic, | for regex flags, slicing and mapping lookups. The documented helpers, builtins, string/dict/regex methods and module members. Not available
Any other module, imports inside functions, module-level assignments, nested functions, class, while, try/except, lambda, global, async, yield, assert and del. Attribute access other than the documented methods and module members; use sample["key"] or sample.get("key"). Names that start with an underscore, exec/eval/open and every other ambient builtin; helpers and imports may not shadow the documented helper names. Power exponents that are not numeric literals of magnitude 8 or less. Limits
Code up to 32768 bytes and 2500 syntax nodes.
At most 64 components; trace up to 16384 bytes per example.
Scores between -10.0 and 10.0; loops nested at most 2 deep; iterables bounded to 10000 items.
Simulations run in a separate interpreter with no ambient builtins, an empty working directory, no network, and CPU, memory, output and wall-time limits.
Saved functions Reusable reward(sample) programs · 0 in the library
Search reward functions
Load one into this objective, clone it, or delete it. Certification always binds to the objective, never to the library.
No saved functions yet. Save one with “Save to library”.
Probes Hand-written thinking / final pairs for fast good-vs-broken checks
Probe examples Edit thinking and final output together; both fields are available to your reward function. Dataset rows can be copied in as probes.
Scratch runs are instant and never persisted. Record a run to keep an exploratory snapshot in the objective history.
Back to RL data
Continue to certification
Step 6 · Launch
Training recipe and RL handoff
Set the input and output limits for your model. The advisor suggests budgets from a separate dataset profile; it never changes your token limits. Review the remaining training settings before certifying this recipe.
Draft
Datasets Choose the training dataset
Thinking contract Literal thinking → canonical envelope
Reward evidence Run a successful simulation
Source model Choose a registered model
Checking gateway connection…
Endpoints Deploy, test and monitor your model APIs.
+ New endpoint
Model Provider Capacity Review
Advanced
— —
Advanced: registered fixed servers and the temporary cloud machines the controller provisions for jobs. Most operators only need to pick compute inside the job builder.
Fixed servers — Temporary servers — GPUs available — GPUs allocated —
Search servers Filter servers by lifecycle Fixed and temporary Fixed servers Temporary servers Filter servers by status All statuses Available capacity Fully allocated Provisioning Cleanup required Offline 0 resources
Advanced: cloud provider accounts (Runpod, Vast.ai) whose balance, GPU catalog and prices feed the temporary-compute picker.
Register Hugging Face artifacts available as finetuning sources.
Compose GRPO reward recipes from repository-owned gates, weighted components and counterweights, score pasted completions to see the ranking the trainer would see, then save the recipe for training jobs.
Start a recipe from a template Start from a template…
Synchronized datasets become available in job creation.
Manage the private tokenizer artifacts offered to pretraining jobs.
Review Google access requests, approve operators, and delegate administrator access. Only administrators can open this section.