Header Grading with Mozilla Observatory and securityheaders.com

This guide is part of the Security Header Auditing & Compliance reference and treats the two grading scanners — Mozilla Observatory and securityheaders.com — as measurement instruments with known scoring rules, not as oracles of security. Both tools parse the response headers of a single URL and emit a letter grade. They are useful as a regression signal and a fast external check, but their grades are derived from header presence and shape, not from whether your policy actually stops the attack it claims to. A site can score A+ with a Content Security Policy that permits unsafe-inline, and a hardened site can score poorly because it omits a header the grader weights heavily. The practical question — what your own grade means and how to move it — is dissected in interpreting your securityheaders.com grade.

Threat Model & Protocol Mechanics

Neither scanner mitigates a threat; both measure whether the headers that mitigate threats are present and well-formed. Understanding what each tool inspects, and how it converts that inspection into a number or letter, is the difference between chasing a grade and hardening a site.

What Mozilla Observatory checks

Observatory (the HTTP Observatory, now hosted at observatory.mozilla.org and exposed through the MDN scanner) runs a fixed battery of tests against the response headers and a few derived signals. Each test contributes a positive or negative point adjustment to a baseline score of 100. The final numeric score maps to a letter grade. The tests it weights are:

• Content-Security-Policy — the single highest-impact test. A policy that forbids unsafe-inline in script-src and avoids broad * sources earns the full bonus; a missing CSP is the largest single penalty. This is the CSP reference.
• Strict-Transport-Security — presence with a sane max-age; this is the HSTS reference.
• X-Frame-Options or CSP frame-ancestors — clickjacking framing control, the X-Frame-Options reference.
• X-Content-Type-Options: nosniff — MIME sniffing suppression.
• Referrer-Policy and Permissions-Policy — privacy and feature-surface controls, the Referrer-Policy and Permissions-Policy reference.
• Subresource Integrity, Cross-Origin Resource Sharing posture, cookie flags (Secure, HttpOnly, SameSite), and redirection (HTTP must redirect to HTTPS before any header is honored).

The Observatory scoring algorithm and its modifiers

Observatory starts every host at 100 points and applies each test’s modifier. Modifiers are not symmetric: the CSP and HSTS tests can each subtract 25 points when absent, while a strong CSP that uses nonces or hashes and forbids unsafe-inline can add points above the baseline, allowing scores over 100 that still cap the letter at A+. A representative set of modifiers:

• Missing CSP: -25. CSP present but permitting unsafe-inline in script-src: -10. CSP that forbids unsafe-inline and broad sources: 0 to +5. CSP delivered only as Content-Security-Policy-Report-Only: counts as no enforcing policy for scoring.
• Missing HSTS: -20. HSTS present: 0. HSTS present and preloaded: +5.
• Missing framing control (neither X-Frame-Options nor frame-ancestors): -20.
• Missing X-Content-Type-Options: nosniff: -5.
• Missing Referrer-Policy: -5 (a small penalty — its absence will not by itself drop you a letter).
• Cookies without Secure/HttpOnly: -20 to -40 depending on whether session cookies are involved.
• HTTP not redirecting to HTTPS: -20 and the scan effectively caps low.

Because the baseline is 100 and the worst penalties stack, a site with no security headers at all lands well below zero and is clamped to F (0). The letter bands are: A+ ≥ 100, A 90–99, B 70–89, C 50–69, D 30–49, F < 30.

What securityheaders.com checks

securityheaders.com (Scott Helme’s scanner) does not use a numeric points model. It inspects a smaller set of headers for presence and applies a few content rules, then assigns a letter from A+ down to F. The headers it grades are: Content-Security-Policy, Strict-Transport-Security, X-Frame-Options, X-Content-Type-Options, Referrer-Policy, and Permissions-Policy. It also surfaces, but does not grade, “missing headers” warnings and flags information-disclosure headers like Server and X-Powered-By.

securityheaders.com A+ to F criteria

The grade is essentially a tiered presence check with two content gates:

• F — almost no security headers present.
• D / C / B — a progressively larger subset present. Roughly, framing + nosniff reaches the C/B range; adding HSTS and Referrer-Policy climbs toward B/A.
• A — all six graded headers present in valid form.
• A+ — all six present and the CSP must not contain unsafe-inline or unsafe-eval in script-src (the content gate), with HSTS present. A policy that includes unsafe-inline caps the grade at A even when every header is present.

The second gate matters: the only difference between A and A+ for a complete header set is whether the CSP would actually constrain inline script. That is the one place where the grader’s letter tracks real security — everywhere else it tracks presence.

CSP scoring nuances

CSP is where both tools diverge most from “secure”. Observatory rewards nonce/hash-based policies and penalizes unsafe-inline, but it cannot tell whether your nonces are actually unique per request, whether a script-src allowlist hosts a JSONP endpoint that defeats the policy, or whether object-src is left open. securityheaders.com checks only for the literal unsafe-inline/unsafe-eval tokens in script-src. Neither tool evaluates strict-dynamic, base-uri, or frame-ancestors for soundness. The result: a policy can pass both graders’ CSP checks and still be trivially bypassable through an allowlisted CDN that serves arbitrary script.

A high grade is not “secure”

Treat the grade as a necessary-but-not-sufficient signal. A+ confirms the headers are present and the CSP forbids inline script. It does not confirm the CSP is bypass-resistant, that cookies are scoped correctly, that TLS configuration is sound (graders read headers, not cipher suites), or that report-only policies are catching real violations. Conversely, a B or C can hide a perfectly defensible posture — a same-origin API with no HTML surface legitimately omits framing and CSP headers and will be marked down for headers it does not need.

Directive Syntax & Spec

The “syntax” of a grading run is the set of tests each scanner applies and the impact each carries. The table below maps the graded test to its effect on the result so you can predict a grade before you scan.

The most common grading surprise: a Content-Security-Policy-Report-Only header earns zero grading credit on both tools. Report-only is for tuning a policy before enforcement; the graders only read the enforcing Content-Security-Policy header.

Platform-Specific Implementation

“Implementation” here means running each scanner reproducibly — by hand, in a terminal, and in a pipeline. Configuring the headers themselves lives in the Nginx, Apache, and Cloudflare references.

Running Observatory via web

Open observatory.mozilla.org, enter the hostname, and start the scan. The scan is rate-limited per host (roughly one fresh scan per minute) and results are cached; use the “rescan” control to force a fresh run after a deploy. The web UI shows the per-test breakdown, the point modifier for each, and the final score and grade. Use it for the human-readable test list, not for automation.

Running Observatory via CLI

The maintained client is the Node package observatory-cli (the legacy Python mozilla-observatory client still works against the v1 API). Install and run:

npm install -g observatory-cli
observatory example.com --format=report

The CLI prints each test name, its score modifier, and the pass/fail state, then the overall grade. Pipe --format=json for machine parsing. The CLI hits the same hosted API, so the same rate limits apply.

Running Observatory via API

The HTTP Observatory exposes a REST API. Kick off a scan and poll for the result:

# Trigger a scan (POST), then retrieve the analysis
curl -s -X POST "https://http-observatory.security.mozilla.org/api/v1/analyze?host=example.com" \
  -d "hidden=true&rescan=true"
curl -s "https://http-observatory.security.mozilla.org/api/v1/analyze?host=example.com"

The first call enqueues the scan; the second returns the finished record once state is FINISHED. The hidden=true parameter keeps the result off the public results board.

securityheaders.com via web

Open securityheaders.com, enter the URL (full URL, not just hostname — it follows the exact path you give it), and tick “Hide results” for a private one-off scan. The report shows the letter grade, the raw response headers, the graded headers as present/missing, and a “missing headers” advisory list.

securityheaders.com via API

securityheaders.com returns the grade in a response header (X-Grade) when you request the analysis endpoint with followRedirects=on and hide=on. Ask for the headers only:

curl -sI "https://securityheaders.com/?q=https%3A%2F%2Fexample.com&hide=on&followRedirects=on" \
  | grep -i '^x-grade:'
# x-grade: A+

The q parameter is the URL-encoded target. hide=on keeps the scan off the public list; followRedirects=on makes the grader follow your HTTP→HTTPS redirect to the final URL it actually grades.

Automating both in CI

Wire both scanners into the pipeline so a deploy that drops a header fails the build before it reaches users. The mechanics of a full pipeline gate — caching, threshold enforcement, and failing on regression — are covered in automated header scanning in CI/CD. A minimal dual-scanner gate:

#!/usr/bin/env bash
set -euo pipefail
TARGET="https://example.com"
HOST="example.com"

# securityheaders.com grade gate
grade=$(curl -sI "https://securityheaders.com/?q=https%3A%2F%2F${HOST}&hide=on&followRedirects=on" \
  | grep -i '^x-grade:' | tr -d '\r' | awk '{print $2}')
case "$grade" in
  A+|A) echo "securityheaders.com: $grade" ;;
  *) echo "securityheaders.com regression: $grade"; exit 1 ;;
esac

# Observatory score gate
curl -s -X POST "https://http-observatory.security.mozilla.org/api/v1/analyze?host=${HOST}" \
  -d "hidden=true&rescan=true" >/dev/null
sleep 5
score=$(curl -s "https://http-observatory.security.mozilla.org/api/v1/analyze?host=${HOST}" \
  | python3 -c 'import sys,json;print(json.load(sys.stdin).get("score",-1))')
[ "$score" -ge 90 ] || { echo "Observatory score $score below 90"; exit 1; }
echo "Observatory score: $score"

Verification & Diagnostic Workflows

Treat the grade as the headline and the raw test data as the truth. Always pull the structured output so you can see which test moved the grade, not just the letter.

Observatory JSON result (after the scan reaches FINISHED):

curl -s "https://http-observatory.security.mozilla.org/api/v1/analyze?host=example.com"

{
  "scan_id": 48210912,
  "grade": "A+",
  "score": 105,
  "likelihood_indicator": "LOW",
  "state": "FINISHED",
  "tests_passed": 11,
  "tests_quantity": 12
}

Per-test detail (the endpoint that tells you which modifier fired):

curl -s "https://http-observatory.security.mozilla.org/api/v1/getScanResults?scan=48210912"

{
  "content-security-policy": { "pass": true, "score_modifier": 5,
    "result": "csp-implemented-with-no-unsafe" },
  "strict-transport-security": { "pass": true, "score_modifier": 0,
    "result": "hsts-implemented-max-age-at-least-six-months" },
  "x-frame-options": { "pass": true, "score_modifier": 0,
    "result": "x-frame-options-sameorigin-or-deny" }
}

securityheaders.com expected output — the X-Grade header is the single source of truth for automation:

curl -sI "https://securityheaders.com/?q=https%3A%2F%2Fexample.com&hide=on&followRedirects=on" \
  | grep -i '^x-grade:'
# x-grade: A+

Browser DevTools cross-check — open the Network tab, click the document request, and read the Response Headers. The grader sees exactly what DevTools shows for that final URL; if they disagree, you are scanning a different URL (often a redirect target or a CDN-cached variant) than the one in your browser.

Troubleshooting, Misconfigurations & Safe Rollback

The recurring confusion with graders is a low grade on a genuinely well-defended site. The map below pairs each symptom with its cause.

• Strong site, low grade, “no CSP” in the report → you ship a Content-Security-Policy-Report-Only header but no enforcing Content-Security-Policy. Graders ignore report-only entirely. Promote a tested policy to the enforcing header to earn the score.
• All headers present, capped at A not A+ → the CSP contains unsafe-inline or unsafe-eval in script-src. Move to nonces or hashes to clear the A+ content gate. See the CSP reference.
• Grade dropped after a deploy with no header change → the scanner followed a new redirect or hit a different edge node/CDN cache that lacks the headers. Confirm the exact URL graded with curl -sI and check that headers are emitted on every node.
• Observatory penalizes a header-less JSON API → the tools assume an HTML browsing context. An API with no HTML surface legitimately omits CSP and framing headers; document the deviation rather than adding headers it does not need, and exclude such hosts from the grade gate.
• Grade higher than reality → A+ on a CSP whose script-src allowlists a host serving arbitrary JS. The grader checks tokens, not allowlist soundness. Audit the allowlist manually; the grade will not catch it.
• Header present in browser, “missing” in scanner → the header is set on 200 responses only, and the grader followed a redirect whose response lacks it. Emit headers with the always/equivalent flag on all responses.
• Rollback — grading is read-only; there is nothing to roll back in the scanner. If a CI gate blocks a deploy on a grade regression, the safe action is to restore the missing header at the edge, not to lower the threshold. Lowering the threshold to pass a regression hides the very drift the gate exists to catch.

Frequently Asked Questions

Does a Content-Security-Policy-Report-Only header improve my grade? No. Both Observatory and securityheaders.com count only the enforcing Content-Security-Policy header. Report-only is for tuning a policy before enforcement and earns zero grading credit. Promote the policy to the enforcing header once it is clean.

Why does Observatory give a score above 100 but still grade A+? Observatory starts at 100 and awards bonus points for a nonce/hash-based CSP that forbids unsafe-inline and for HSTS preload, so scores can exceed 100. The letter grade is capped at A+, so any score of 100 or more shows as A+.

My site is hardened but only scores B — is the scanner wrong? Usually it is measuring presence, not your actual risk. A same-origin API with no HTML surface legitimately omits CSP and framing headers and is marked down for them. The scanner is not wrong, it is scoring for a browsing context that may not apply; exclude such hosts from the gate.

Can I get A+ with an insecure CSP? Yes. The A+ content gate only rejects literal unsafe-inline/unsafe-eval in script-src. A policy that allowlists a CDN serving arbitrary JavaScript passes the gate and scores A+ while remaining bypassable. The grade is a presence check, not a bypass audit.

Do the scanners check my TLS configuration? No. Both read HTTP response headers. They confirm HSTS is present and that HTTP redirects to HTTPS, but they do not evaluate cipher suites, protocol versions, or certificate chains. Use a TLS-specific tool for that, covered in auditing headers with curl, openssl and testssl.

Related

• Interpreting your securityheaders.com grade
• Automated header scanning in CI/CD
• Auditing headers with curl, openssl and testssl
• Content Security Policy (CSP) Essentials
• Security Header Auditing & Compliance