VividKit Crystal clear AI coding
VI
Choose Your Setup Get Started with AgentKit Continue with ClaudeKit Migrate from ClaudeKit to AgentKit Clean cutover from ClaudeKit What Is AgentKit? How AgentKit Works AgentKit Kits, Skills, Agents & Hooks AgentKit Runtime Support AgentKit Desktop App AgentKit Configuration AgentKit CLI Quick Reference AgentKit Skills Cheatsheet AgentKit Workflows AgentKit Statusline Update AgentKit CLI & Kits AgentKit Helper Clean reinstall of AgentKit AgentKit Troubleshooting What is ClaudeKit? ClaudeKit CLI Guide Command Finder Commands Cheat Sheet Workflow Recipes CLI Quick Reference Claude Mechanics Permission Modes ClaudeKit Hooks Coexistence ClaudeKit x Codex CK → Other coding tools How ClaudeKit Works Inside ClaudeKit Finding Your Unknowns Session Recovery Fix from Logs UI/UX Guide Accounts Switcher IDE Config Remote Control Codex App System Diagramming UI Review Gate Promotions Support VividKit

Quick Ref / Command

Engineer Kit

/ak:deep-swe

DeepSWE coding-agent benchmark

Run a costed external coding-agent benchmark on DeepSWE through Pier and OpenRouter, with safe setup, single-task smoke, subset/full run controls, and exact reporting.

01

Confirm scope

02

Check prerequisites

03

Install runner

04

Verify model slug

Rule 01

This is an external benchmark, not local repo tuning.

Rule 02

Never echo, persist, commit, or place OPENROUTER_API_KEY in artifacts.

Rule 03

Inspect current Pier help before relying on flags.

Rule 04

One task comes before subset; explicit approval comes before full corpus.

Rule 05

Do not submit leaderboard results without a user request.

Execution Map

Core lanes from input to output, aligned with the command's real execution path.

01

Start

Input, scope, route

  1. 1 Confirm scope Use this only for a costed external DeepSWE model benchmark, not repository-local optimization.
  2. 2 Check prerequisites Verify uv, git, Docker, Docker daemon, and OpenRouter key presence without exposing secrets.
02

Work

Agent / skill execution

  1. 3 Install runner Clone deep-swe, install datacurve-pier with uv, and inspect pier --help and pier run --help.
  2. 4 Verify model slug Check the exact OpenRouter model slug and use the openrouter/<vendor/model> form documented by the current toolchain.
03

Verify

Gate, review, validation

  1. 5 Run one task Start with a single deep-swe/tasks/<task-id> Pier run before any sampled subset.
  2. 6 Sample subset For deterministic samples, confirm installed flags for task count and sample seed, then run a small fixed sample.
04

Close

Report, handoff, artifact

  1. 7 Full corpus gate Before all 113 tasks, present exact command, cost exposure, and stop condition, then wait for explicit user confirmation.
  2. 8 Inspect report Use current Pier commands such as view, analyze, or critique when available, and report command, version, task count, model, score/reward, cost, and blockers.

Syntax, arguments, subcommands

How to invoke the skill: syntax, positional arguments, shared options, then each subcommand with its own syntax and outcome.

Syntax

/ak:deep-swe <OpenRouter model slug> [benchmark request]

Arguments

<OpenRouter model slug> Required

Model slug

Exact OpenRouter vendor/model identifier to evaluate on DeepSWE. The Skill verifies it before spending tokens; it is not an AgentKit-owned catalog entry.

/ak:deep-swe openai/gpt-4o-mini
[benchmark request]

Benchmark request

Optional natural-language boundary such as one named smoke task, a deterministic subset, or verification of an existing Pier job. It does not authorize a full corpus run or result submission by default.

/ak:deep-swe openai/gpt-4o-mini "Run one named DeepSWE task as a smoke test. Stop after the task and do not submit results."

Sample Prompt

Concrete invocations for each flag, subcommand, mode, or route available in the live workflow.

Smoke one task Recommended 
/ak:deep-swe openai/gpt-4o-mini "Run one named DeepSWE task as a smoke test. Show the exact Pier command and estimated spend exposure before execution. Stop after the task and do not submit results."
Use when:
You want to score a specific OpenRouter model slug on DeepSWE without approving a larger run yet.
Expected:
Checks uv, git, Docker, the daemon, and key presence; validates Pier help and runs only one named task before reporting local evidence.
Deterministic subset 
/ak:deep-swe openai/gpt-4o-mini "After a successful smoke task, prepare a 10-task deterministic subset with --n-tasks 10 and --sample-seed 0 only if current pier run --help confirms those flags."
Use when:
You approved expansion from smoke evidence to a small fixed sample.
Expected:
Rechecks the installed Pier flag surface, presents the bounded subset command and spend exposure, then reports task count, score or reward, cost if available, and blockers.
Score verification 
/ak:deep-swe verify the Pier job result for openai/gpt-4o-mini
Use when:
You need to verify or summarize an existing DeepSWE Pier benchmark result.
Expected:
Inspects the generated job directory with available Pier view, analyze, or critique commands and reports command, version, task count, model slug, score or reward, cost, blockers, and submission status.

Handled Scope

  • DeepSWE benchmark setup
  • Pier runner commands
  • OpenRouter model routing
  • Cost-aware evaluation
  • Benchmark result inspection

Next