CI/CD: the gate in your pipeline

Release. ci, report and sarif arrive with ArcLang 4.6.0. A pipeline generated for an older --arclang-version fails on arclang ci run. The Configure CI command of the VS Code extension needs the 4.6 compiler.

ArcLang checks a model the way a pipeline checks code: every model compiled, the production gate, the simulations that simulate, a SARIF file for code scanning, and on a pull request one comment that says what the change does to the model. The pipeline files only call the compiler; everything is computed by arclang.

What was run, and what was not. The generated pipelines are tested as files: valid YAML, no secret, pinned versions, and the commands they call are run in the tests. They have not been run against GitHub, GitLab, Azure DevOps or Jenkins themselves. The first run of your pipeline is their test, and each section below says what is likely to need a look.

Quick start

arclang ci init                       # the provider is read from the git remote
arclang ci init --provider all        # GitHub, GitLab, Azure DevOps and Jenkins
arclang ci init --dry-run             # print the files, write nothing

ci init never overwrites a file: if one exists, nothing is written and the command says which (--force replaces them). Options: --provider github|gitlab|azure|jenkins|all, --standard iso26262|do178c|iec61508, --model <file> (repeatable), --arclang-version, --release-repo, --branch. Without --model the pipeline checks every entry-point model, that is every .arc file that no other .arc file imports.

In VS Code, ArcLang: Configure CI offers the same, with a preview of each file before anything is written.

The rest of this page runs in a git repository built when the page is built. It holds one model, the braking chain of Semantic git, with a 100 ms budget that the chain meets.

model Braking {
  requirements safety {
    req "REQ-001" "Braking reaction" { id: "REQ-001" description: "Brake within 100 ms of detection" }
  }
  system_analysis "Braking" {
    function "Detect obstacle" { id: "SF-DET" latency: 40 ms }
    function "Command braking" { id: "SF-BRK" latency: 50 ms }
    functional_chain "Detection to braking" {
      id: "FC-BRAKE"
      involves: ["SF-DET", "SF-BRK"]
      latency_budget: 100 ms
    }
  }
  architecture logical {
    component "Brake controller" { id: "LC-BRAKE" function "Command braking" }
  }
}
trace "LC-BRAKE" satisfies "REQ-001"
test_case "TC-001" { verifies: ["REQ-001"] method: "test" }

$ arclang ci models
brake.arc

$ arclang ci init --provider github --dry-run | grep -E '^--- |^    name:|run: arclang|uses: '
--- .github/workflows/arclang.yml
    name: ArcLang gate
        uses: actions/checkout@v4
        uses: actions/cache@v4
        run: arclang ci run --standard iso26262 --sarif arclang-results/arclang.sarif --summary arclang-results/summary.md
        uses: github/codeql-action/upload-sarif@v3
        uses: actions/upload-artifact@v4

arclang ci run: what a pipeline runs

For each model: compile it, run the simulations of its test cases that simulate: (see Simulation), run the production gate. It prints one line per model, writes the SARIF file and a Markdown summary, and exits non-zero unless every model compiles, passes its gate, and has no failing simulation.

$ arclang ci run --sarif arclang.sarif --summary summary.md
PASS  brake.arc  (gate ISO26262: 0 blocker(s), 1 warning(s); requirements 1/1 satisfied, 1/1 verified)
SARIF: arclang.sarif
ArcLang gate: PASS (1 model(s))

• No model found is a failure, not a pass: nothing was checked. --allow-empty accepts it.
• A simulation that could not run (no OpenModelica, a stale cache) is inconclusive: a verification.inconclusive warning in the SARIF file and the summary, never a pass. The gate counts a requirement whose verification is inconclusive as not verified.
• --verify never skips the simulations.

Now a feature branch slows the detection function from 40 ms to 60 ms. The two functions then sum to 110 ms against a 100 ms budget, and the check goes red:

$ arclang ci run
FAIL  brake.arc  (gate ISO26262: 1 blocker(s), 1 warning(s); requirements 1/1 satisfied, 1/1 verified)
      brake.arc:8  [gate.timing.budget] functional_chain 'Detection to braking': sum of function latencies 110.0 ms exceeds the 100.0 ms budget
Error: ArcLang gate: FAIL (1 of 1 model(s): brake.arc)

The pull-request report

arclang report --pr --from origin/main [--to <head>] [models...]
                    [-f md|json|html] [-o file] [--out-dir dir]
                    [--fail-on blocker|major|never] [--standard ...] [--note "..."]

For each model, the base against the head: the gate before and after with the new blockers named, the coverage change, the latency budgets, the constraints that flip, the simulation verdicts (pass -> fail, stale), how many elements the changes reach, the semantic diff by layer, and the before and after pictures.

$ arclang report --pr --from main --out-dir results -o report.md
Error: the pull request makes the gate worse (--fail-on blocker): brake.arc

$ grep -v '^Base ' report.md | sed -n 1,26p
<!-- arclang-pr-report -->
## ArcLang gate: **FAIL**

1 of 1 model(s) fail the ISO26262 gate; 1 new blocker(s) in this change.


| Model | Gate before | Gate after | New blockers | Requirements | Changes |
|---|---|---|---|---|---|
| `brake.arc` | PASS | FAIL (1 blocker) | 1 | 1/1 -> 1/1 satisfied, 1/1 -> 1/1 verified | +0 -0 ~1 |

### `brake.arc`

Production gate: **PASS** -> **FAIL (1 blocker)**

New blockers:

- [timing.budget] functional_chain 'Detection to braking': sum of function latencies 110.0 ms exceeds the 100.0 ms budget

Regressions:

- budget Detection to braking is exceeded (110 ms for 100 ms)

Requirements: 1/1 -> 1/1 satisfied, 1/1 -> 1/1 verified

| Latency budget | Before | After | Limit |
|---|---|---|---|

• The verdict is the gate's. A model that does not compile at the head, or whose gate fails, is FAIL; the report never says PASS in that case, whatever --fail-on says. No model at all is "no model found", not a pass.
• Exit status. --fail-on blocker (the default) exits 1 when a blocker appears that the base did not have, or a model no longer compiles. major also exits 1 on a regression: a requirement no longer satisfied or verified, a constraint that stops holding, a budget newly exceeded, a simulation that stops passing. never always exits 0. A gate that already failed at the base and fails the same way is reported FAIL but is not "new": it is the ArcLang gate job (ci run) that stays red.
• The first line of the report is a hidden marker, <!-- arclang-pr-report -->. The pipelines look for it to update the same comment instead of adding one per push. A comment is cut under 60 000 characters (GitHub's limit is 65 536), with a line that says so.
• Pictures. With --out-dir the report links diagrams/<model>-before.svg and -after.svg: an architecture map of the elements by Arcadia layer (added green, modified amber, removed dashed red). It is not a drawing of the architecture; arclang diagram is. A comment cannot display a workspace file, so the pipelines upload arclang-results/ as an artifact and --note adds the line that says where. The images are SVG only: the compiler has no rasterizer.
• The simulation verdicts of the base are read from the verdict cache of that revision, which is not in git. A base verdict is "inconclusive" unless the cache is there.

SARIF and code scanning

arclang sarif [models...] (also arclang check --format sarif and arclang gate --format sarif) writes a SARIF 2.1.0 log, validated against the OASIS schema in the compiler's tests: one run, one rule per rule id with its meaning and how to answer it, one result per diagnostic with the file and line, the element as a logical location, a stable fingerprint, and the fix in the message and in properties.fixHint.

The line is exact for a compile error. For the other results it is the line where the element named in the message is declared (properties.locationPrecision: element), or the first line of the file (file) when the message names none. arclang sarif exits 1 when a model does not compile or its gate fails (--fail-on never for 0), so that if: always() steps can upload the file.

$ arclang sarif brake.arc --fail-on never | grep -E '"(ruleId|level)"'
          "level": "note",
          "ruleId": "method.layer-order",
          "level": "error",
          "ruleId": "gate.timing.budget",
                "level": "error"
                "level": "note"

On GitHub the results show as alerts on the pull request and in the Security tab. Code scanning on a private repository needs GitHub Advanced Security.

Make the gate a required check

The check name is the job name, ArcLang gate: do not rename the job without updating the protection.

• GitHub: Settings, Branches, the rule of the default branch, Require status checks to pass, add ArcLang gate (it appears in the list after one run). The compiler repository has a script, tools/ci/require-arclang-gate.sh OWNER/REPO [BRANCH], that prints the gh api command and runs nothing unless you pass --apply. The call is a PUT that replaces the whole branch protection, so read the current one first.
• Azure DevOps: a Build validation branch policy on the pipeline.
• GitLab: Pipelines must succeed in the merge request settings.
• Jenkins: the status that your SCM plugin publishes.

Provider notes

GitHub Actions. One job, ArcLang gate: checkout with full history, install, restore the simulation cache, ci run, job summary, SARIF upload (github/codeql-action/upload-sarif, category arclang), the report and its comment on pull requests, upload of arclang-results/. Permissions: contents: read, pull-requests: write, security-events: write, actions: read; the token is github.token and nothing else is read. For a pull request from a fork GitHub gives the token read access only: the SARIF upload and the comment fail and are allowed to (continue-on-error), and the gate still runs and decides.

GitLab CI. Job arclang-gate (image ubuntu:24.04, GIT_DEPTH: 0). GitLab does not read SARIF, so the file is a job artifact. The merge-request comment needs a project access token with the api scope, stored in the masked CI/CD variable ARCLANG_GITLAB_TOKEN, because CI_JOB_TOKEN cannot write merge-request notes. Without the variable the job says so and the report stays in the artifacts. The simulations need Docker on the runner; with none they are inconclusive.

Azure DevOps. Job ArcLangGate, displayed as ArcLang gate. The comment uses System.AccessToken: give the project's build service identity Contribute to pull requests on the repository. The SARIF file is published as the CodeAnalysisLogs artifact, which the SARIF SAST Scans Tab extension displays. The simulation cache is restored from the repository root only.

Jenkins. A declarative Jenkinsfile (an existing one is never overwritten). A multibranch pipeline with a pull-request source sets CHANGE_ID and CHANGE_TARGET: the report is built, printed in the console and archived. Posting it as a comment is not done, because it depends on the SCM plugin. The SARIF file is read with recordIssues(tool: sarif(...)) when the Warnings Next Generation plugin has it, and the step is skipped otherwise.

Caching the simulation verdicts

arclang verify keeps one verdict per simulated test case in .arclang/sim-cache/ beside the model. The key covers the generated Modelica package, every Modelica source, and the case's parameters and expectations: a verdict is reused while none of them changes, and is "stale" (inconclusive) when one does. The generated pipelines cache that directory (GitHub: actions/cache on **/.arclang/sim-cache; GitLab: cache:paths; Azure DevOps: Cache@2). ci run still runs verify; a cached verdict whose key matches is not simulated again.

OpenModelica comes from ARCLANG_OMC (a local omc), ARCLANG_OMC_DOCKER (an image), or Docker with the default pinned image. GitHub's ubuntu-latest has Docker. The first run pulls the image and installs the Modelica Standard Library into a Docker volume, and the generated workflow caches neither. See Simulation.

Security

• No secret in a generated file. The compiler's tests fail on a literal token or key. The tokens are the provider's own, read at run time (github.token, System.AccessToken); ARCLANG_GITLAB_TOKEN is a variable you create.
• Pinned. The compiler version is the one that wrote the file. The binary is checked against the release's SHA256SUMS, and the release carries a build provenance attestation (gh attestation verify). Actions are pinned to a major version (@v4); pin them to a commit SHA if your policy asks for it.
• Values that end up in a shell command or a YAML scalar (--model, --branch, the version, the release repository) must be plain (A-Za-z0-9._-/+) or ci init refuses them. Branch names from the event go through environment variables, never interpolated into scripts.
• The report is computed from the head's files. On pull_request (not pull_request_target) from a fork the token is read-only, which is what you want for code you did not write.

In VS Code

ArcLang: Configure CI shows a quick pick of the four systems, with the one your git remote points at ticked, a read-only preview of each file, and writes nothing until you choose Write. Replacing a file that exists needs a second confirmation, and the files open afterwards.

A status bar item shows the latest GitHub Actions run of the current branch for a repository that has the generated workflow. It is read through the GitHub CLI (gh) and its own login: ArcLang stores no token, says in words when gh is missing or not logged in, and reads nothing in an untrusted workspace. A run that was canceled or skipped shows ?; only a success shows as passing. Settings: arclang.ci.statusBar, arclang.ci.pollSeconds, arclang.ci.gh (the last one is a user setting only).

Limits

• The pipelines are not run against the four providers by us (see the top of the page).
• GitLab does not ingest SARIF, and its comment needs a token you provide. Jenkins posts no comment.
• The pictures are SVG, and they are an element map, not the architecture diagrams.
• A repository with no .arc entry point has "no model found": name the models with --model, or on the command line.
• The release binary the pipelines download is Linux x86-64; other runners need the compiler built or installed another way.
• The base's simulation verdicts are only known when the cache is there.
• Not available in the browser playground: there is no git or CI there.