Automating Mozilla Observatory API Scans

Q: Do I still need to poll until the scan state is FINISHED?

No. That was the v1 API model, which shut down on 31 October 2024. The v2 POST /api/v2/scan responds synchronously with the finished grade and score in a single request, so there is no state field to poll. Retry only on transport errors like 429 or 5xx.

Q: Where did the full list of response headers go?

The public v2 /api/v2/scan response was trimmed to summary fields and no longer includes the site's response headers or per-test breakdown. Run the npx @mdn/mdn-http-observatory host CLI, or a self-hosted Observatory server, to get the tests map with each check's pass flag and scoreModifier.

Q: How can score be 105 when the maximum is 100?

The base score runs 0 to 100, but tests that exceed a strong baseline add positive scoreModifier bonus points on top. The grader caps the letter-grade lookup at 100, so both 100 and 105 map to A+; the extra points signal headroom, not a higher grade.

Q: Should my CI compare grades as strings?

No. Lexical comparison is wrong because A+ < B is true as text but backwards as a gate. Map each grade to an index in an ordered list from F to A+ and compare the ranks.

Q: What grade should I set as the CI threshold?

Pin MIN_GRADE to the grade the site currently earns so the gate catches regressions immediately, then raise it as you add headers. B+ (score 80) is a practical floor for a site running HSTS and a basic CSP; A or A+ requires a strict, inline-free CSP.

This guide is part of the Header Grading with Mozilla Observatory and securityheaders.com reference and shows the exact API v2 flow to run Observatory from a script: send one POST to trigger an analyze, read the grade and score fields from the returned JSON, extract per-test results, and exit non-zero when the grade drops below a threshold so a pipeline can block the deploy. The old poll-until-FINISHED state machine from the v1 API is gone — the public v2 endpoint responds synchronously with the graded result, which makes the loop below shorter, not longer.

Configuration Syntax & Exact Values

The single request that runs a scan and returns the grade:

POST /api/v2/scan?host=example.com HTTP/1.1
Host: observatory-api.mdn.mozilla.net

The v1 host http-observatory.security.mozilla.org/api/v1/analyze was shut down on 31 October 2024. Use the v2 host observatory-api.mdn.mozilla.net/api/v2/scan — nothing else resolves. On success the endpoint returns this JSON body:

{
  "id": 77666718,
  "details_url": "https://developer.mozilla.org/en-US/observatory/analyze?host=example.com",
  "algorithm_version": 4,
  "scanned_at": "2026-07-24T08:20:18.926Z",
  "error": null,
  "grade": "A+",
  "score": 105,
  "status_code": 200,
  "tests_failed": 0,
  "tests_passed": 10,
  "tests_quantity": 10
}

Field-by-field, the values automation reads:

grade — the letter grade A+ … F. This is the field a CI gate compares against a threshold. It is a normalisation of score, not an independent value.
score — the raw score. Base range is 0–100; passing tests carry positive scoreModifier bonuses that can push it above 100 (the 105 above is an A+ with bonus credit). The grader floors the score at 0, rounds down to the nearest multiple of 5, caps the lookup at 100, then maps to a letter.
error — null on a successful scan. If scanning failed (unresolvable host, unreachable site) the top-level object is replaced by { "error": "...", "message": "..." } with no grade field, so a script must test for grade before trusting it.
status_code — the HTTP status Observatory received from the scanned site (200 here), not the status of the API call itself.
tests_passed / tests_failed / tests_quantity — the summary counts across the ten checks. algorithm_version: 4 runs exactly ten tests, so tests_quantity is 10.
details_url — a human URL. The public v2 endpoint deliberately omits the full per-test breakdown and response headers that v1 returned; details_url is where a person reads them, and the npx CLI is where a script reads them (shown below).

One POST returns the graded result; the cooldown may hand back a cached scan, and the grade rank drives the pipeline exit code.

Server-Side Configuration

The “platform” here is the automation runtime that calls the API. Each block is runnable as-is; substitute your host and minimum grade.

curl + jq

#!/usr/bin/env bash
set -euo pipefail
HOST="example.com"
MIN_GRADE="B+"
API="https://observatory-api.mdn.mozilla.net/api/v2/scan"

# Ordered grades, worst -> best; array index is the rank.
GRADES=(F D- D D+ C- C C+ B- B B+ A- A "A+")
rank() { for i in "${!GRADES[@]}"; do [ "${GRADES[$i]}" = "$1" ] && { echo "$i"; return; }; done; echo -1; }

resp="$(curl -fsS -X POST "${API}?host=${HOST}")"
grade="$(jq -r '.grade // empty' <<<"$resp")"
[ -z "$grade" ] && { echo "scan error: $(jq -r '.error // "unknown"' <<<"$resp")"; exit 2; }

score="$(jq -r '.score' <<<"$resp")"
echo "grade=$grade score=$score (min=$MIN_GRADE)"
[ "$(rank "$grade")" -ge "$(rank "$MIN_GRADE")" ] || { echo "FAIL: below threshold"; exit 1; }

curl -fsS matters: -f turns an HTTP 4xx/5xx into a non-zero exit so a broken API call cannot masquerade as a passing scan, -s silences the progress meter, and -S still prints errors. A missing grade field means the JSON was an error object, so the script exits 2 (distinct from a real grade failure).

Python (requests)

#!/usr/bin/env python3
import sys, requests

HOST, MIN_GRADE = "example.com", "B+"
GRADES = ["F","D-","D","D+","C-","C","C+","B-","B","B+","A-","A","A+"]
API = "https://observatory-api.mdn.mozilla.net/api/v2/scan"

r = requests.post(API, params={"host": HOST}, timeout=120)
r.raise_for_status()
data = r.json()

grade = data.get("grade")
if grade is None:
    sys.exit(f"scan error: {data.get('error', 'unknown')}")

print(f"grade={grade} score={data['score']} "
      f"passed={data['tests_passed']}/{data['tests_quantity']}")
if GRADES.index(grade) < GRADES.index(MIN_GRADE):
    sys.exit(f"FAIL: {grade} is below {MIN_GRADE}")

raise_for_status() is the Python equivalent of curl -f; without it a 429 cooldown-limit body would parse into a KeyError, not a clear failure. The same ranked-list comparison avoids ever doing string comparison on grades — "B+" < "A" is true lexically but meaningless as a gate.

GitHub Actions

name: observatory
on: [deployment_status]
jobs:
  scan:
    runs-on: ubuntu-latest
    steps:
      - name: HTTP Observatory grade gate
        env:
          HOST: example.com
          MIN_GRADE: "B+"
        run: |
          resp=$(curl -fsS -X POST \
            "https://observatory-api.mdn.mozilla.net/api/v2/scan?host=${HOST}")
          grade=$(echo "$resp" | jq -r '.grade // empty')
          [ -z "$grade" ] && { echo "::error::scan failed"; exit 2; }
          echo "grade=$grade" | tee -a "$GITHUB_STEP_SUMMARY"
          grades="F D- D D+ C- C C+ B- B B+ A- A A+"
          r() { i=0; for g in $grades; do [ "$g" = "$1" ] && echo $i; i=$((i+1)); done; }
          [ "$(r "$grade")" -ge "$(r "$MIN_GRADE")" ] || \
            { echo "::error::$grade below $MIN_GRADE"; exit 1; }

Triggering on deployment_status scans the live host only after a deploy publishes, so the grade reflects what visitors actually receive. jq is preinstalled on ubuntu-latest.

GitLab CI

observatory:
  image: alpine:3
  variables:
    HOST: example.com
    MIN_GRADE: "B+"
  before_script:
    - apk add --no-cache curl jq bash
  script:
    - |
      resp=$(curl -fsS -X POST \
        "https://observatory-api.mdn.mozilla.net/api/v2/scan?host=$HOST")
      grade=$(echo "$resp" | jq -r '.grade // empty')
      [ -n "$grade" ] || { echo "scan failed"; exit 2; }
      grades="F D- D D+ C- C C+ B- B B+ A- A A+"
      rank() { i=0; for g in $grades; do [ "$g" = "$1" ] && echo $i; i=$((i+1)); done; }
      test "$(rank "$grade")" -ge "$(rank "$MIN_GRADE")" || exit 1

The apk add line installs the three tools Alpine lacks by default; on a fuller base image drop before_script.

Distinguish three failure modes: retriable transport errors, a scan-config error with no grade, and a real grade below the gate.

Diagnostic & Verification Steps

Run the scan by hand once to confirm the endpoint, the shape of the JSON, and the grade before wiring it into a pipeline.

Trigger a scan and read the summary:

curl -fsS -X POST \
  "https://observatory-api.mdn.mozilla.net/api/v2/scan?host=developer.mozilla.org" \
  | jq '{grade, score, tests_passed, tests_failed}'

Expected output:

{
  "grade": "A+",
  "score": 105,
  "tests_passed": 10,
  "tests_failed": 0
}

List the per-test results. The public /api/v2/scan response carries only the summary counts. To read which of the ten checks failed — and each check’s scoreModifier — use the official CLI, which returns a scan object plus a tests map:

npx @mdn/mdn-http-observatory developer.mozilla.org \
  | jq '.tests | to_entries[] | {test: .key, pass: .value.pass, modifier: .value.scoreModifier}'

Expected output (abridged — ten entries):

{ "test": "content-security-policy", "pass": true, "modifier": 5 }
{ "test": "cookies", "pass": true, "modifier": 0 }
{ "test": "cross-origin-resource-sharing", "pass": true, "modifier": 0 }
{ "test": "redirection", "pass": true, "modifier": 0 }
{ "test": "referrer-policy", "pass": true, "modifier": 5 }
{ "test": "strict-transport-security", "pass": true, "modifier": 0 }
{ "test": "subresource-integrity", "pass": true, "modifier": 0 }
{ "test": "x-content-type-options", "pass": true, "modifier": 0 }
{ "test": "x-frame-options", "pass": true, "modifier": 0 }

Print only the failing tests — the one line you actually want in a build log:

npx @mdn/mdn-http-observatory example.com \
  | jq -r '.tests | to_entries[] | select(.value.pass==false) | .key'

On a site missing its policy this returns the offending check names, e.g. content-security-policy and strict-transport-security, mapping directly to the Content-Security-Policy and HSTS references for the fix.

The grade is a bucketed score; a B+ gate corresponds to score 80, so any regression that drops the raw score below 80 fails the build.

Edge Cases, Security Implications & Safe Rollback

Three traps are specific to automating this API:

The cooldown returns a cached scan. The endpoint runs at most one fresh scan per host per minute (api.cooldown, default 60s); inside that window it hands back the previous result. A pipeline that deploys and immediately scans within a minute of a prior scan can grade the old deploy and pass on stale headers. Space the scan after the deploy, or trigger it on deployment_status/an environment-ready event rather than on push, so the minute has elapsed and the live host is current.
An error object has no grade. When the host is unresolvable, unreachable, or redirects off-scope, the body is { "error": "...", "message": "..." } with no grade/score. Scripts that read .grade unconditionally see null and, with a naive string compare, may treat it as the lowest grade and fail — or worse, skip the check. Always branch on the presence of grade and exit with a distinct code (2 above) so an infrastructure fault is never confused with a policy regression.
Shared CI egress IPs hit the rate limit. Many runners share outbound IPs, and hammering the public endpoint across parallel jobs can draw 429 responses. Scan once per pipeline, not once per job, and treat 429/5xx as retriable with backoff — never as a grade failure. For heavy or private use, run the open-source @mdn/mdn-http-observatory server against your own Postgres and point the scripts at your instance; the JSON contract is identical.

Rollback is trivial because the scanner is read-only — it changes nothing on the target. The reversible risk lives in the gate, not the scan: introduce the job as non-blocking first (log the grade, exit 0 regardless, or continue-on-error: true), confirm the grade is stable across a few deploys, then set MIN_GRADE to the current grade and make the step blocking. If a legitimate deploy is blocked by a scanner outage, relax to warn-only for that run rather than lowering MIN_GRADE, which would permanently weaken the gate.

Conclusion

Roll the gate out in stages: run the scan against staging in warn-only mode to learn the baseline grade and the shape of the JSON, then enable it in production as a blocking step pinned to the current grade with a short backoff on transient errors, and only raise MIN_GRADE after the corresponding headers ship and hold. Trigger on a post-deploy event so the minute cooldown has passed and the grade reflects live traffic, and the pipeline will catch a dropped header before users do.

Frequently Asked Questions

Do I still need to poll until the scan state is FINISHED? No. That was the v1 API model, which shut down on 31 October 2024. The v2 POST /api/v2/scan responds synchronously with the finished grade and score in a single request, so there is no state field to poll. Retry only on transport errors like 429 or 5xx.

Where did the full list of response headers go? The public v2 /api/v2/scan response was trimmed to summary fields and no longer includes the site’s response headers or per-test breakdown. Run the npx @mdn/mdn-http-observatory <host> CLI, or a self-hosted Observatory server, to get the tests map with each check’s pass flag and scoreModifier.

How can score be 105 when the maximum is 100? The base score runs 0–100, but tests that exceed a strong baseline add positive scoreModifier bonus points on top. The grader caps the letter-grade lookup at 100, so both 100 and 105 map to A+; the extra points signal headroom, not a higher grade.

Should my CI compare grades as strings? No. Lexical comparison is wrong — "A+" < "B" is true as text but backwards as a gate. Map each grade to an index in an ordered list from F to A+ and compare the ranks, exactly as the scripts above do.

What grade should I set as the CI threshold? Pin MIN_GRADE to the grade the site currently earns so the gate catches regressions immediately, then raise it as you add headers. B+ (score 80) is a practical floor for a site running HSTS and a basic CSP; A/A+ requires a strict, inline-free CSP.