Learn · Cloud policy
Invariant: prove a guardrail change
A reviewer reads the diff. A solver reads every request in its model. Five lessons cover what that gives you, how to read a counterexample, how to put it in CI, and how far a pass really goes.
See it in action · On the command line
Five policy changes. One question for the solver.
Pick a change to a service control policy and watch what Invariant says about it. Two of them show where the tool's model stops.
Real runs of invariant-sast 0.3.5, recorded by tools/record_invariant_demo.py and replayed here. Each change is compared with the reference policy shown in the diff. The results table is omitted; the warnings, summary and findings are verbatim.
Drop a denied action
Pull request: service-control-policies/protect-org.json
{ "Version": "2012-10-17", "Statement": [ { "Sid": "ProtectOrgAndTrail", "Effect": "Deny", "Action": [- "organizations:LeaveOrganization", "cloudtrail:StopLogging" ], "Resource": "*" } ] }CI job
$ invariant regression --manifest manifest.json --output cliSummary: 1 passed, 1 failed, 0 skipped 1 check(s) FAILED (most severe first): [CRITICAL] INV_AWS_001 service-control-policies/protect-org.json Candidate no longer denies a request the reference denied: {'resource': '""', 'action': '"organizations:LeaveOrganization"'}exit 1Pipeline blocked
The deny list lost an action. Z3 finds a request the old policy denied and the new one doesn't, and names the action.
Rewrite with NotAction
Pull request: service-control-policies/protect-org.json
{ "Version": "2012-10-17", "Statement": [ { "Sid": "ProtectOrgAndTrail", "Effect": "Deny",- "Action": [- "organizations:LeaveOrganization",- "cloudtrail:StopLogging"- ],- "Resource": "*"+ "Resource": "*",+ "NotAction": [+ "organizations:*",+ "iam:*",+ "sts:*"+ ] } ] }CI job
$ invariant regression --manifest manifest.json --output cliSummary: 1 passed, 1 failed, 0 skipped 1 check(s) FAILED (most severe first): [CRITICAL] INV_AWS_001 service-control-policies/protect-org.json Candidate no longer denies a request the reference denied: {'action': '"organizations:LeaveOrganization"', 'resource': '""'}exit 1Pipeline blocked
Denying everything except a few services looks stricter, but it exempts organizations:*. The solver shows LeaveOrganization is no longer denied.
Add a break-glass exemption
Pull request: service-control-policies/protect-org.json
{ "Version": "2012-10-17", "Statement": [ { "Sid": "ProtectOrgAndTrail", "Effect": "Deny", "Action": [ "organizations:LeaveOrganization", "cloudtrail:StopLogging" ],- "Resource": "*"+ "Resource": "*",+ "Condition": {+ "ArnNotLike": {+ "aws:PrincipalARN": "arn:aws:iam::*:role/BreakGlass"+ }+ } } ] }CI job
$ invariant regression --manifest manifest.json --output cliSummary: 0 passed, 2 failed, 0 skipped 2 check(s) FAILED (most severe first): [CRITICAL] INV_AWS_001 service-control-policies/protect-org.json Candidate no longer denies a request the reference denied: {'principal': '"arn:aws:iam:::role/BreakGlass"', 'action': '"organizations:LeaveOrganization"', 'resource': '""'} [MEDIUM] INV_AWS_004 service-control-policies/protect-org.json 1 statement(s) using operators the solver can't model changed and need manual review: ProtectOrgAndTrailexit 1Pipeline blocked
The counterexample names the exempted principal. INV_AWS_004 also asks for manual review, because this statement uses an operator outside the solver's model.
Refactor to wildcards
Pull request: service-control-policies/protect-org.json
{ "Version": "2012-10-17", "Statement": [ { "Sid": "ProtectOrgAndTrail", "Effect": "Deny", "Action": [- "organizations:LeaveOrganization",- "cloudtrail:StopLogging"+ "organizations:Leave*",+ "cloudtrail:Stop*" ], "Resource": "*" } ] }CI job
$ invariant regression --manifest manifest.json --output cliSummary: 2 passed, 0 failed, 0 skippedexit 0Pipeline passes
The wildcards still cover both actions, so nothing the reference denied is now allowed. This pass covers this check against this reference, not account-wide security.
Exempt one account (a model gap)
Pull request: service-control-policies/protect-org.json
{ "Version": "2012-10-17", "Statement": [ { "Sid": "ProtectOrgAndTrail", "Effect": "Deny", "Action": [ "organizations:LeaveOrganization", "cloudtrail:StopLogging" ],- "Resource": "*"+ "Resource": "*",+ "Condition": {+ "StringNotEquals": {+ "aws:PrincipalAccount": "111122223333"+ }+ } } ] }CI job
$ invariant regression --manifest manifest.json --output cliWARNING: unhandled condition operator 'StringNotEquals' -- treated as non-constraining wherever it appears in this run. Verify affected statements manually.Summary: 1 passed, 1 failed, 0 skipped 1 check(s) FAILED (most severe first): [MEDIUM] INV_AWS_004 service-control-policies/protect-org.json 1 statement(s) using operators the solver can't model changed and need manual review: ProtectOrgAndTrailexit 1Pipeline blocked
This change does widen access: account 111122223333 is no longer denied. In 0.3.5 the solver doesn't model StringNotEquals, so INV_AWS_001 passes. The warning and INV_AWS_004 still fail the run, so a person has to review it.
See it in action · In a pull request pipeline
A real landing zone, three real pull requests.
The same checks running in GitHub Actions on AWS's LZA Universal Configuration. One pull request weakens a guardrail, one adds a risky IAM policy, and one is a harmless cleanup.
Real pull requests in heimdallsystems/invariant-examples, against AWS's LZA Universal Configuration v1.4.0, checked by invariant-sast 0.3.5 in GitHub Actions. Recorded by tools/record_invariant_ci_demo.py and replayed here. Paths are shown relative to lza-universal-configuration/modules/base/default, the results table is omitted, and the rest of each log is verbatim.
Allow deleting AWS Config rules
Pull request #1: service-control-policies/lza-core-guardrails-1.json
@@ -6,7 +6,6 @@ "Effect": "Deny", "Action": [ "config:PutConfigRule",- "config:DeleteConfigRule", "config:DeleteEvaluationResults", "config:UntagResource" ],GitHub Actions
guardrail-regression › Build manifest.json from the diffWrote 1 entry to manifest.jsonguardrail-regression › Prove changed policies against reference/Summary: 0 passed, 2 failed, 0 skipped 2 check(s) FAILED (most severe first): [CRITICAL] INV_AWS_001 service-control-policies/lza-core-guardrails-1.json Candidate no longer denies a request the reference denied: {'principal': '""', 'resource': '""', 'action': '"config:DeleteConfigRule"'} [MEDIUM] INV_AWS_004 service-control-policies/lza-core-guardrails-1.json 1 statement(s) using operators the solver can't model changed and need manual review: GRCFGRError: Process completed with exit code 1.guardrail-baseline › Fail on findings this pull request addsNEW INV_AWS_001 service-control-policies/lza-core-guardrails-1.jsonNEW INV_AWS_004 service-control-policies/lza-core-guardrails-1.json2 new failed check(s), 6 already on the base branchError: Process completed with exit code 1.- guardrail-baseline failed
- guardrail-regression failed
- Code scanning (invariant) passed
Merge blocked
Removing one action from a real LZA guardrail. The regression proof names the request that now gets through, and the new-findings gate blocks the pull request. INV_AWS_004 also asks for a manual review, because GRCFGR uses a condition the solver doesn't model. See pull request #1 · See the run
Add a CI deploy role policy
Pull request #2: iam-policies/ci-deploy-role-policy.json
@@ -0,0 +1,11 @@+{+ "Version": "2012-10-17",+ "Statement": [+ {+ "Sid": "DeployArtifacts",+ "Effect": "Allow",+ "Action": ["s3:PutObject", "iam:PassRole"],+ "Resource": "*"+ }+ ]+}GitHub Actions
guardrail-regression › Build manifest.json from the diffThe following changed files have no reference/ counterpart and were excluded from this run: - iam-policies/ci-deploy-role-policy.json (expected reference/iam-policies/ci-deploy-role-policy.json)Wrote 0 entries to manifest.jsonguardrail-regression › Prove changed policies against reference/Summary: 0 passed, 0 failed, 0 skippedguardrail-baseline › Fail on findings this pull request addsNEW INV_AWS_003 iam-policies/ci-deploy-role-policy.json1 new failed check(s), 6 already on the base branchError: Process completed with exit code 1.- guardrail-baseline failed
- guardrail-regression passed
- Code scanning (invariant) passed
Merge blocked
A new file has no reviewed copy to prove against, so the regression job excludes it and says so. The baseline scan still checks every IAM policy, finds iam:PassRole, and the gate fails because this finding is new. The six findings already on main don't block. See pull request #2 · See the run
Remove a duplicate S3 resource
Pull request #3: iam-policies/ssm-s3-policy.json
@@ -8,7 +8,6 @@ "arn:${PARTITION}:s3:::aws-ssm-${REGION}/*", "arn:${PARTITION}:s3:::aws-windows-downloads-${REGION}/*", "arn:${PARTITION}:s3:::amazon-ssm-${REGION}/*",- "arn:${PARTITION}:s3:::aws-ssm-${REGION}/*", "arn:${PARTITION}:s3:::amazon-ssm-packages-${REGION}/*", "arn:${PARTITION}:s3:::${REGION}-birdwatcher-prod/*", "arn:${PARTITION}:s3:::aws-ssm-distributor-file-${REGION}/*",GitHub Actions
guardrail-regression › Build manifest.json from the diffWrote 1 entry to manifest.jsonguardrail-regression › Prove changed policies against reference/Summary: 3 passed, 0 failed, 0 skippedguardrail-baseline › Fail on findings this pull request adds0 new failed check(s), 6 already on the base branch- guardrail-baseline passed
- guardrail-regression passed
- Code scanning (invariant) passed
Checks pass
A cleanup that grants nothing new. The regression proof passes all three checks and the pull request adds no findings. See pull request #3 · See the run
About this guide
Teams change AWS guardrails through ordinary pull requests, and the reviewer sees JSON. Whether a deleted condition or a new NotAction widened access depends on requests nobody wrote down, so reading the diff can't settle it. Pattern-based tools can flag a bare iam:*, but they don't reason about the set of requests a policy allows.
Invariant hands the policies to the Z3 solver and asks one precise question at a time. When it finds a request that got through, you get that request. This guide is for engineers who own Service Control Policies, Resource Control Policies or IAM policies, including Landing Zone Accelerator estates.
You'll be able to
- Explain what Invariant checks and how that differs from pattern matching
- Scan a policy directory against the bundled Landing Zone Accelerator baseline
- Reproduce a regression and read its counterexample
- Add JSON and SARIF output and a failing exit code to CI
- Say exactly what a clean run does and doesn't show
1. The question a reviewer can't answer
Invariant has three main checks, each a question for the solver:
| Check | Question | Needs a reference? |
|---|---|---|
INV_AWS_001 | Does the candidate SCP or RCP still deny everything its reference denied? | Yes |
INV_AWS_002 | Does the candidate IAM policy grant anything its reference didn't? | Yes |
INV_AWS_003 | Can this IAM policy grant a known privilege-escalation action on some resource? | No |
A further check, INV_AWS_004, looks at changes to statements using operators the solver doesn't model, so you know where to look by hand.
2. Your first scan
The package is invariant-sast. The command it installs is invariant.
python -m pip install invariant-sast
cd your-lza-config-repo
invariant run
This walks the directory for service-control-policies/, rcp-policies/ and iam-policies/ at any depth. Files with a same-named counterpart in the bundled Landing Zone Accelerator guardrails get the relative checks, and every IAM policy gets INV_AWS_003. Custom policies with no counterpart are reported as skipped for the relative checks, not silently passed.
To also compare everything changed since a Git ref:
invariant run --diff-base origin/main
3. Read a counterexample
Create these four files in an empty folder. The change removes one action from an SCP's deny list and adds a CI role with iam:PassRole.
reference/service-control-policies/protect-org.json
{"Version": "2012-10-17", "Statement": [{"Sid": "ProtectOrgAndTrail", "Effect": "Deny",
"Action": ["organizations:LeaveOrganization", "cloudtrail:StopLogging"], "Resource": "*"}]}
service-control-policies/protect-org.json
{"Version": "2012-10-17", "Statement": [{"Sid": "ProtectOrgAndTrail", "Effect": "Deny",
"Action": ["cloudtrail:StopLogging"], "Resource": "*"}]}
iam-policies/ci-deployer.json
{"Version": "2012-10-17", "Statement": [{"Sid": "DeployArtifacts", "Effect": "Allow",
"Action": ["s3:PutObject", "iam:PassRole"], "Resource": "*"}]}
manifest.json
[{"reference": "reference/service-control-policies/protect-org.json",
"candidate": "service-control-policies/protect-org.json"},
{"reference": null, "candidate": "iam-policies/ci-deployer.json"}]
Now run it:
invariant regression --manifest manifest.json --output cli
Summary: 1 passed, 2 failed, 2 skipped
2 check(s) FAILED (most severe first):
[CRITICAL] INV_AWS_001 service-control-policies/protect-org.json
Candidate no longer denies a request the reference denied:
{'resource': '""', 'action': '"organizations:LeaveOrganization"'}
[HIGH] INV_AWS_003 iam-policies/ci-deployer.json
Grants 1 privilege-escalation primitive(s) unconditionally on at least one resource: iam:PassRole
Read the first finding as a witness. The action is the one you removed, and the empty resource is simply a value the solver picked, since the statement applies to all resources. If the widening is intended, record that with a suppression and explain why in the pull request. If it isn't, revert the change.
4. Put it in CI
The exit code is 0 for no findings and 1 when a check fails, so a plain run gates a pipeline. For machine-readable results:
invariant regression --manifest manifest.json --output json --output sarif --out-dir solver-results
This writes results.json and results.sarif. Add --soft-fail to exit 0 and report without blocking, which helps when a separate job does the gating.
On a real estate, main usually has findings already, so a gate that fails on every finding is always red. The invariant-examples repository shows one way through, used in the pipeline demo above:
- A regression job proves each changed policy against a reviewed copy in
reference/, and fails on any weakening. - A baseline job scans everything and reports to code scanning, so existing findings stay visible. On a pull request it also scans the base branch and fails only on findings the pull request adds.
Two things to know about SARIF. Invariant's results name the policy file but not a line, so GitHub's own code scanning check can't tie an alert to the changed lines and won't fail a pull request on it. That's why the example compares findings itself. GitLab ingests findings only from a job that succeeds, which is why its jobs run with --soft-fail and a separate job gates.
Copy the example workflow, or start from the templates in the Invariant repository.
5. What a pass means
A failure comes with a concrete request. A pass needs more care:
- It covers individual policy documents, not an organisation's effective permissions, so it isn't proof of account-wide security.
- The solver works on a model of selected request keys and conditions. Unmodelled operators are treated as non-constraining and print a warning, so a pass on those statements is weaker.
- As of 0.3.5 the checks set no solver timeout, and a result other than satisfiable isn't reported as a finding. No finding isn't proof.
- Relative checks only run where there is a reference. Read the skipped lines.
The docs explain how the solver works and where it's conservative. For the reasoning behind this approach, read A solver checks every input in its model.