Skip to content
Experts · Local Authoring & Calibration

Build your task locally. Submit evidence through your project.

Prepare, calibrate, and rehearse your assigned Terminal-Bench 4.0 task locally with Linux Docker, Harbor 0.18.0, and pytest before opening or syncing your cohort pull request.

Python 3.12+ · Harbor 0.18.0Linux Docker EngineUnix LF · Non-Root rwe
WORKSTATION CALIBRATION CHECKLIST
  1. Pinned local Harbor 0.18.0

    Run offline container rehearsals before pushing to your PR branch.

    0.18.0
  2. Positive & negative controls

    Verify Oracle = 1.0 and NOP = 0.0 inside clean Linux containers.

    1.0 / 0.0
  3. Automated PR stage tags

    Pushed commits sync CI: 14/14 Passed and reviewer status automatically.

    SHA LOCK
Local rehearsals never consume your project model-trial allowance

Four-step local workflow

Rehearse locally before opening your pull request.

Follow the four steps below so every pushed revision passes static hygiene and container calibration on the first CI run.

STEP 01

01 / Get the assigned workspace

Claim an available category opportunity and submit your proposal. After approval, accept the project’s GitHub invitation, fork the assigned repository where permitted, and use the task ID and submission branch shown on your active task page. The approved project invitation establishes your repository access.

Step 01 · Provisioned Branch Access
STEP 02

02 / Install and inspect the tools

Install Git, Python 3.12 and Docker with a Linux engine. Create a clean Python virtual environment and install the pinned local Harbor harness and pytest runner so you can rehearse both task containers offline on your workstation.

Step 02 · Pinned Local Harness
STEP 03

03 / Validate Oracle and Nop controls

Oracle must complete with reward 1 and Nop with reward 0 (with zero skipped CTRF tests and deterministic frozen-artifact regrades). Use local rehearsals to validate your task before independent project review. Official certification requires the authorized audit process. Keep Tbench model-provider credentials and GitHub App keys off contributor devices.

Step 03 · Zero Credential Leakage
STEP 04

04 / Submit an immutable revision

Push to your assigned fork/submission branch and open the upstream PR. Independent 58-rule static checks, active hostile-input probes, and Oracle/Nop calibration verify the submission. First-time private-fork workflows may need maintainer approval. Counted model trials remain commit-bound: pass@2 uses two of the ten allowed attempts per task cycle on the canonical profile, while TB 4.0 cohorts execute Pass@3 panels (Opus 5.5 high in Stage 06 and GPT-6.2 Astra max in Stage 12) with uncached token telemetry. Local rehearsals are separate from that allowance.

Step 04 · Commit-Bound CI & Trials

Workstation commands

Linux, macOS, and Windows PowerShell setup.

Keep Docker Desktop in Linux container mode and configure Git to commit pure Unix LF line endings across all shell scripts, Dockerfiles, and Python modules.

POSIX · BASH / ZSH

Create virtual environment & run controls

git config core.autocrlf input
python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install "harbor==0.18.0" "pytest>=8.0"
docker info
harbor run --agent oracle tasks/<slug>
harbor run --agent nop tasks/<slug>
WINDOWS · POWERSHELL

Activate environment & verify LF hygiene

git config core.autocrlf input
py -3.12 -m venv .venv
.\.venv\Scripts\Activate.ps1
python -m pip install "harbor==0.18.0" "pytest>=8.0"
docker info
harbor run --agent oracle tasks/<slug>
harbor run --agent nop tasks/<slug>

Ready to push your branch

Use your active task page for live PR status and stage evidence.

Use your active task page and project playbook for proposal status, repository links, automated PR stage tags, and remaining trial allowance.