LLM Evaluation in CI/CD with GitHub Actions

An LLM evaluation suite is only useful if it runs on every change and its verdict can be trusted. This page shows how I wire DeepEval, Promptfoo and Garak into GitHub Actions, and the detection gate that stops the pipeline from reporting a false sense of safety. It is part of my LLM evaluation framework.

Triggers

Trigger	What happens
Push or pull request touching the LLM Testing folder	Full battery, summary table on the run page, report as an artifact
Every day at 12:00 UTC	Same battery, then the HTML report and history are committed to the main branch
Manual dispatch	Same as the daily run

on:
  push:
    paths:
      - "LLM Testing/**"
      - "!LLM Testing/reports/**"
  pull_request:
    paths:
      - "LLM Testing/**"
  schedule:
    - cron: "0 12 * * *"
  workflow_dispatch:

permissions:
  contents: read

concurrency:
  group: llm-testing-${{ github.ref }}
  cancel-in-progress: ${{ github.event_name != 'schedule' && github.event_name != 'workflow_dispatch' }}

The push filter ignores changes under the reports folder, so the daily report commit does not trigger another run. Concurrency cancels superseded runs on pull requests, but never cancels a scheduled or manual run.

Pipeline shape

   ┌────────────────────────────┐   ┌──────────────────────────┐
   │ evals                      │   │ garak                    │
   │ pytest · DeepEval ·        │   │ isolated virtualenv      │
   │ Promptfoo                  │   │                          │
   │ (hardened + vulnerable)    │   │ (hardened + vulnerable)  │
   └──────────────┬─────────────┘   └─────────────┬────────────┘
                  └───────────────┬───────────────┘
                                  ▼
                   report  (runs even if a suite fails)
                   ├─ step summary table
                   ├─ artifact: HTML report
                   ├─ daily commit of report and history
                   └─ fails the run when the verdict is not PASS

• Parallel jobs. The functional evals and the vulnerability scan run side by side, so the whole battery finishes in minutes.
• Garak in its own virtual environment. Its dependency tree (torch, transformers, huggingface-hub) conflicts with DeepEval’s pins, and it only needs to talk to the HTTP API, so isolating it costs nothing.
• Every evaluation step uses continue-on-error. I want both agents evaluated and a report produced even when something is red. The verdict is computed once, at the end.
• The report job runs with if: always(). A red run still produces a report, which is when you need it most.
• No secrets. The daily commit uses the built-in GITHUB_TOKEN and the suite needs no API key, because every check is deterministic.

The detection gate

A green suite proves nothing if it cannot go red. A broken assertion, a probe that silently stops running or a renamed field can all turn a suite into a rubber stamp. So every run evaluates two agents: the hardened one, which must pass everything, and a deliberately vulnerable one, which must be caught by each tool at least once.

- name: DeepEval against the hardened agent
  continue-on-error: true
  env:
    AGENT_URL: http://127.0.0.1:8000
    EVAL_RESULTS_DIR: reports/raw
  run: python -m pytest evals/deepeval -q

- name: DeepEval against the vulnerable agent (detection check)
  continue-on-error: true
  env:
    AGENT_URL: http://127.0.0.1:8001
    EVAL_RESULTS_DIR: reports/raw/canary
  run: python -m pytest evals/deepeval -q

The verdict is a pure function of both result sets. The run is green only when every suite passes on the hardened agent and every tool detected at least one failure on the vulnerable agent:

suites_ok = {
    name: (not main[name].get("missing")) and main[name]["total"] > 0
          and main[name]["passed"] == main[name]["total"]
    for name in main
}
detection = {}
for name in ("deepeval", "promptfoo", "garak"):
    data = canary.get(name, {"total": 0, "passed": 0, "missing": True})
    failed = data["total"] - data["passed"]
    detection[name] = {"failed": failed, "detected": (not data.get("missing")) and failed > 0}

detection_ok = all(d["detected"] for d in detection.values())
status = "PASS" if all(suites_ok.values()) and detection_ok else "FAIL"

Current baseline on the vulnerable agent: DeepEval fails 5 of 21 cases, Promptfoo fails 6 of 14, and Garak fails 5 of 7 probe and detector pairs. Note the missing check: a suite with no results counts as not detected, so a tool that did not run cannot pass the gate.

Reporting

A report builder reads the raw results and produces three outputs: a self-contained HTML report with per-case metrics and reasons, a Markdown summary table that is appended to the GitHub step summary, and a run history file so trends can be charted later. On scheduled and manual runs, the report and history are committed back to the repository, and a last step fails the run when the verdict is not PASS:

- name: Build the HTML report
  run: |
    python scripts/build_report.py --record-history --update-readme
    cat reports/latest/summary.md >> "$GITHUB_STEP_SUMMARY"

- name: Fail the run if the suite is red
  run: |
    python - <<'PY'
    import json, sys
    status = json.load(open("reports/latest/summary.json"))["verdict"]["status"]
    print("Suite status:", status)
    sys.exit(0 if status == "PASS" else 1)
    PY

Source: the workflow on GitHub. For a broader look at pipelines, see CI/CD test pipelines.