CI Integration

Use CI when a Wendell suite should guard agent, prompt, policy, or tool changes before they reach production.

GitHub Actions for local suites

Use this path when your repo commits wendell.toml and local JSON suites, then runs wendell test. This does not require a Wendell API key unless your own agent adapter needs hosted credentials.

The workflow runs from the repository root, so wendell test loads ./wendell.toml by default. Use --config only for an alternate file or path.

Create .github/workflows/wendell.yml:

name: Wendell regression tests

on:
  pull_request:
  push:
    branches: [main]

jobs:
  wendell:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm

      - uses: actions/setup-python@v5
        with:
          python-version: "3.13"

      - name: Install agent dependencies
        run: npm ci

      - name: Install uv
        run: |
          python -m pip install --user uv
          echo "$HOME/.local/bin" >> "$GITHUB_PATH"

      - name: Install Wendell
        run: uv tool install --force wendell

      - name: Run Wendell suite
        run: wendell test

Replace the Node setup and npm ci step with the dependency installation your agent adapter actually needs. CI should run the same agent_command configured in wendell.toml, not a special passing adapter.

GitHub Actions for hosted suites

Add WENDELL_INKPASS_API_KEY to your repository or organization secrets, then create .github/workflows/wendell.yml.

After wendell register works locally, export the stored runner credential directly into a GitHub Actions secret:

wendell auth export --format raw | gh secret set WENDELL_INKPASS_API_KEY

For organization-level or non-GitHub secret managers, pipe the same raw output into that platform's secret command. Use JSON when your automation needs metadata:

wendell auth export --format json

Then add the workflow:

name: Wendell Agent Tests

on:
  pull_request:
  push:
    branches: [main]

jobs:
  wendell:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-python@v5
        with:
          python-version: "3.13"

      - name: Install Wendell
        run: |
          curl -fsSL https://www.wendellai.com/install | bash
          echo "$HOME/.local/bin" >> "$GITHUB_PATH"

      - name: Run Wendell
        env:
          WENDELL_INKPASS_API_KEY: ${{ secrets.WENDELL_INKPASS_API_KEY }}
          WENDELL_APP_AGENT_COMMAND: python scripts/run_my_agent.py
        run: wendell run --suite refund-agent-regression --github-summary

Production is the default API target. Do not set api_url unless you are targeting staging, preview, or a local API.

--github-summary appends the gate result, run id, score, and private report link to the GitHub Actions step summary. JSON output includes remote.private_report_url when Wendell can create a private dashboard handoff link for the run, so other CI systems can surface the same URL in summaries or deployment comments.

In GitHub Actions, wendell run automatically attaches workflow context to the uploaded run: repository, workflow, job, run id, run attempt, run URL, event name, actor, ref, pull request head/base refs, and commit SHA. You can still add or override fields with [external_ci_ref] in wendell.toml.

Blocking Gates

Use blocking mode when Wendell should fail CI on regressions:

project = "refund-agent"
mode = "blocking"
agent = "refund_agent"
agent_command = "python scripts/wendell_agent_adapter.py"
agent_timeout_seconds = 120
upload_traces = true

[gates]
suite_min_score = 0.90
scenario_min_score = 0.85
critical_failures_allowed = 0

Use advisory mode while tuning a new suite. Advisory runs still upload reports and print failures, but exit successfully.

Set agent_timeout_seconds higher when your production adapter legitimately needs more than the 120-second default for one scenario turn.

Adapter Command

WENDELL_APP_AGENT_COMMAND should run the same adapter your wendell.toml points at, or the adapter should delegate to that environment variable.

The adapter receives public scenario input on stdin and must print JSON with message, tool_calls, and optional metrics. The generated scripts/wendell_agent_adapter.py template enforces this JSON contract when it delegates to WENDELL_APP_AGENT_COMMAND.

Wendell executes adapter commands directly, without a shell. Keep agent_command and WENDELL_APP_AGENT_COMMAND to an executable plus arguments. Set provider keys and other secrets through CI env, not inline command assignments.

Run the local contract check in Agent Adapter Contract before adding the workflow to CI. CI should call the same production agent path, not a special passing adapter.

CI Secrets

Keep these values in CI secret storage:

Secret	Purpose
WENDELL_INKPASS_API_KEY	Wendell runner credential for hosted suites and trace uploads.
Agent provider keys	Only needed if your own agent adapter calls an LLM or hosted agent service.

CI runner keys that only execute published suites need wendell:test-suites:read, wendell:runs:create, and wendell:runs:read. Do not give CI authoring or publishing scopes unless that workflow intentionally creates or publishes suites.

Wendell itself does not require an OpenAI, OpenRouter, Anthropic, or other LLM provider key to install, create Playbook drafts, generate hosted suites, or run tests. Wendell's hosted service generates suites from reviewed Playbooks. Your own adapter needs provider credentials only when your production agent uses them.