Skip to content

GitHub App ​

The App is the server's only identity towards GitHub and the only way GitHub reaches the server. The server creates it from a manifest at /setup, so the permissions and events below are set for you.

Permissions ​

The manifest the server generates, in internal/gh/manifest.go, requests these permissions:

PermissionLevelWhyNeeded when
MetadataReadRequired for any AppAlways
ContentsReadRead stackorder.yaml, .stackorder.yaml files and CODEOWNERS at the default branch, and branch heads. GitHub grants it on every file of an installed repository, not only theseAlways
Pull requestsWriteThe sticky comment, the comment of each apply, refusal comments, reactions to commands, reviews, mergeability and head commitsAlways
ChecksWriteOne check run per stack per phase, the roll-ups, named checksAlways
ActionsWriteworkflow_dispatch of stackorder-run.yml; reading workflow runs and jobs to reconcileAlways
IssuesWriteDrift issuesdrift.open_issue: true
MembersReadTeam membershipapply.allowed_teams, or CODEOWNERS teams with apply.require_codeowner_review
DeploymentsRead and writeAnswering deployment_protection_rule requestsThe App is a custom deployment protection rule

The last three are needed only for the features in the right column, but the manifest requests them so that turning a feature on needs no change to the App. Remove them in the App's settings if you will never use them. On GitHub Enterprise Server the manifest leaves out Deployments, since not every release offers custom deployment protection rules. The server treats itself as talking to Enterprise Server whenever GITHUB_WEB_URL is not https://github.com; set GITHUB_ENTERPRISE to true or false to decide it explicitly, for instance to keep the full manifest when reaching github.com through a proxy whose web URL differs.

What the App cannot do ​

The App has no Secrets, Administration, Environments or Workflows permission. It cannot change workflow files, secrets, environments or protection rules, so a compromised server cannot widen its own access.

An App also cannot be a required reviewer on an environment. Even the Deployments permission never lets the server approve a human gate; it only lets the server answer for its own protection rule.

Events ​

The manifest subscribes to these events. GitHub delivers installation and installation_repositories to every App without a subscription.

EventUsed for
installation, installation_repositoriesTracking installations, suspensions and the repositories they cover
pull_requestSuperseding plans on a new head; on merge, recording the default-branch graph, releasing locks (before_merge) or starting the apply (on_merge); on close without merge, warning about held locks
pull_request_reviewSubscribed so approvals reach the server; the gate reads reviews from GitHub when it runs
issue_commentComment commands
pushRefreshing stackorder.yaml and the default branch name on default-branch pushes; semver tags on module repositories
check_runRe-runs requested from the checks UI (rerequested)
check_suiteCheck suite re-requests
workflow_runBinding dispatches to workflow runs and closing each wave
workflow_jobRecording a stack's job and moving the stack to planning or applying when the job starts. The server finds the stack in the job's name: plan <key> and <mode> wave <n> <key>, as the reusable workflows name their jobs, or GitHub's default matrix form, <job> (<stack>, …)
deployment_protection_ruleApproving or rejecting deployments as a custom protection rule; not on Enterprise Server

Every minute the server also reconciles dispatches whose webhooks were lost: it resends dispatches GitHub never accepted, binds dispatches to workflow runs by their title, marks the stacks of a dispatch unbound for 30 minutes unknown, and closes dispatches whose workflow run completed without every stack reporting. The CLI's result call stays the source of truth.

Install flow ​

  1. Start the server with STACKORDER_BASE_URL set and no App variables. It comes up in setup mode and logs a setup_url line: https://<server>/setup?token=<token>, with a one-time setup token.
  2. Open that URL. The server trades the token for a cookie and redirects to /setup. Add &org=<organisation> to the URL to create the App in an organisation. &name= chooses its name, up to 34 characters; the default is stackorder-<host>.
  3. The page posts the manifest to GitHub. Confirm the App there.
  4. GitHub redirects to /setup/callback, which exchanges the one-time code for the App id, private key, webhook secret and OAuth client id and secret, and prints them once, as environment variables, with the next steps, a link to install the App and a link to its settings to upload its logo.
  5. Store the values in your secret store, restart the server with them, and install the App on your repositories. GitHub then sends the browser to /setup/installed.
  6. Upload the App's logo.

Installations made while the server was in setup mode are learned at start-up: the server syncs the App's installations and repositories when it starts and daily at 04:00 UTC. The sync also forgets installations and repositories GitHub no longer lists, as a lost uninstall or removal webhook would have, but only after every listing succeeded. An installation the listing leaves out is forgotten, with its repositories and their history, only once GET /app/installations/{id} confirms it with a 404; a 200 keeps it, and any other answer keeps it and retries at the next sync, so one short or empty listing from a proxy in front of the GitHub API or during an incident cannot delete it. A listing that leaves out a majority of the stored installations is treated as degraded and keeps every one of them, logging a warning; their repositories are still reconciled from each listed installation's own repository listing, so the guard keeps installations, not the repositories a listed installation's own complete listing no longer names. A listing that leaves out a majority of a listed installation's repositories keeps that installation's repositories, and because a repository has no per-repository confirmation, a listing that leaves out every one of them keeps them as well, so an empty or short 200 cannot delete a single-repository installation's only repository. Omission is measured against the installations and repositories stored before the sync, so repositories the same sync newly discovers do not dilute it. A repository listing whose pages hold fewer repositories than the total_count GitHub reports — a dropped page, or a repository removed while the pages are read — counts as a failed listing, and because any failed listing makes the whole sync forget nothing, one short page keeps every stored installation and repository, not only the installation whose pages were short; the sync tries again at the next run. The deleted and installation_repositories removed webhooks still forget installations and repositories at once, as the authoritative signal that GitHub removed them.

The manifest sets the webhook URL to <base URL>/webhooks/github, the setup redirect to /setup/callback, the sign-in callback to /auth/callback and the post-installation page to /setup/installed. The App is private. On GitHub Enterprise Server set GITHUB_API_URL and GITHUB_WEB_URL before opening /setup; the flow is the same.

Without the token, /setup answers 403, so nobody else who reaches the server between the deploy and the setup can create the App under their own account. The token stops working on the instance that created an App, and a generated one changes at every start.

Opening /setup on a server that already has App credentials shows a page saying so. Only with STACKORDER_ALLOW_RESETUP=true does it link to creating another App anyway (/setup?force=1), for instance after moving the server to a new URL, which needs the setup token too; without it /setup?force=1 and /setup/callback answer 404.

An App without a logo shows the avatar of the account that owns it, so its PR comments and checks look as if the person who ran /setup posted them. Neither the manifest nor GitHub's API can set the logo, so upload it once by hand:

  1. Save the Stackorder logo from <base URL>/setup/logo.png, a 512×512 PNG.
  2. Open the App's settings: https://github.com/settings/apps/<slug> for an App on a personal account, https://github.com/organizations/<organisation>/settings/apps/<slug> for one in an organisation. On GitHub Enterprise Server the host is GITHUB_WEB_URL. The page /setup/callback prints links to both the logo and the settings.
  3. On the General tab, under Display information, choose Upload a logo, select the file, and save.

GitHub shows the new logo on the comments and checks the App already posted too.

Token handling ​

  • The App's private key signs a 10-minute JWT.
  • The server exchanges the JWT for installation tokens, valid for one hour, cached per installation until 5 minutes before expiry.
  • Every call to GitHub uses the token of the installation it concerns, so a repository in one organisation can never be touched with another organisation's token.
  • Failed calls are retried with backoff, and Retry-After and rate-limit resets are respected. A call that creates something new — a check run, a comment, an issue or a workflow dispatch — is retried only while the request provably never reached GitHub, or when GitHub reports a rate limit and did no work, so a lost response or a server error does not post or dispatch it twice. Reads and updates, which GitHub applies the same way however often they are repeated, are always retried.

Human sign-in uses the App's OAuth client with the read:org scope. A session is issued only to a user whose own account or one of whose organisations has the App installed, and the UI shows only those accounts' repositories.

Rotating anything means rotating the App's private key, which runners never see. Generate a new key in the App settings, update GITHUB_APP_PRIVATE_KEY, redeploy, then delete the old key.

Comment commands ​

Commands are pull request comments. The first line whose first two words are stackorder and a known verb, both case insensitive, is the command; lines in code blocks, indented by four spaces or quoted with > are ignored. Stack keys follow the verb, separated by spaces or commas, with or without backticks. stackorder followed by an unknown word gets the help text.

CommandEffect
stackorder plan [key…]Re-plans the affected stacks of the head commit, or only the named ones, in a new run dispatched to stackorder-run.yml with mode: plan.
stackorder apply [key…]Applies the planned stacks of the head commit through the apply gate, or only the named subset; dependency waves are still honoured within the subset. Refused in on_merge repositories.
stackorder unlock [key…]Releases the orchestration locks this pull request holds, all of them or the named ones.
stackorder helpReplies with the list of commands.
text
stackorder apply stacks/prod/vpc stacks/prod/apps
  • Commands are accepted only from users with push permission on the repository; others are ignored, and audited as command_ignored.
  • Each pull request may send 10 commands a minute, counted from the audit log. The first command over the limit gets a comment asking to wait.
  • The App adds an eyes reaction when it accepts a command and a rocket when it dispatches, so a dropped command is visible. For an apply it then posts the apply comment under the command, edits it as the apply runs, and replies when the apply finishes, whether it succeeded or failed. Every refusal is a comment with the reason.
  • A comment runs at most once, however often GitHub delivers it. A command that fails part way is answered with a comment asking for it to be posted again.

Checks and comments ​

Check runMeaning
stackorder/resolveThe graph was resolved; fails on a dependency cycle, with the cycle spelled out
stackorder/plan: <key>The plan for one stack
stackorder/planRoll-up of every stack's plan for the head commit
stackorder/apply: <key>The apply for one stack
stackorder/applyRoll-up of the apply; green when the last wave is green, and green for a before_merge pull request that affects nothing
stackorder/<check-name>: <key>A named policy or cost check reported with stackorder check: pass succeeds, warn is neutral, fail fails

Require stackorder/plan in branch protection, and stackorder/apply too in before_merge mode. In on_merge mode the apply runs after the merge, so a required stackorder/apply check would block every merge. Stackorder reports; GitHub enforces the merge.

Each pull request has one sticky comment, found by the hidden marker <!-- stackorder:sticky --> on its first line and by the App's own login. It shows the plans of the head commit, with a collapsible section per stack, and lists the pull request's applies under Applies, each linking its apply comment, or the run details when it has none.

Each apply run of a pull request has at most one comment, whose first line is the hidden marker <!-- stackorder:run:<run id> -->. It shows the waves, the pending environment approvals and a row per stack, and the server edits it as the apply runs. An apply that finishes before its first update gets none, and its reply is its only comment: every stack was a no-op, GitHub refused the dispatch, or the server abandoned the apply before dispatching it. The server records the comment's id, so a run never gets a second comment while it can edit the one it has. If editing is refused because the comment is gone or no longer the App's — for a running apply whose comment was deleted, or whose comment and check runs the App can no longer edit after the App was replaced — the server posts a fresh comment and creates fresh check runs. It finds its own comment by its marker, so it posts at most one, and records each new check run's id so later updates reuse it. Only a finished run keeps no comment.

After the App of a deployment is replaced, so that the installation runs under a new App id — a new App registration, or a database restored onto a server configured with a different App — GitHub refuses edits of the old App's check runs and comments with 403. Open runs recover on their next update, each getting new check runs and a new comment under the new App; the old objects, which the new App cannot edit or delete, are left in place.

Fork pull requests ​

The pull_request trigger gives forks a read-only GITHUB_TOKEN and no id-token permission, so neither the AWS role nor the Stackorder API can be reached from a fork. The reusable plan.yml runs nothing for a fork but a job that explains this in its summary, and the server posts a neutral check. See Workflows.