Skip to main content

How to Run Your Own Building Block Runner

What is this guide about?

This guide explains what building block runners are, why you might want to self-host one, and the high-level steps to get one running.

What is a Building Block Runner?​

A building block runner is a small service that connects meshStack to your CI/CD platform. When an application team adds a building block to their workspace or project, meshStack queues a building block run and routes it to the runner assigned to that building block definition. The runner picks up the queued run, triggers the corresponding automation (e.g. a Terraform script, a GitHub Actions workflow, a GitLab pipeline, or an Azure DevOps pipeline), and reports progress and status back to meshStack.

meshStack ships with hosted runners for Terraform, Manual, GitHub Actions, GitLab CI/CD, and Azure DevOps Pipelines. For most use cases, you don't need to run your own.

Should You Run Your Own Runner?​

You need a self-hosted runner when:

  • You need to reach a private network (private cloud or on-premise Git servers) that isn't reachable from meshStack's infrastructure
  • You want dedicated isolation — a private runner only picks up runs assigned to it, with no cross-workspace interference
  • Your organization's security policies require that pipeline credentials never leave your network

How Runners Work​

The runner polls meshStack every few seconds for pending building block runs. When a run is available, it triggers your pipeline and reports status back. Depending on the execution mode on the building block definition, the runner either polls the automation pipeline for completion (synchronous) or hands off and waits for the automation pipeline to report back (asynchronous).

Secret inputs are encrypted with the runner's RSA public key — only the runner, using the matching private key, can decrypt them at runtime.

meshStack building block runners are deployed as Docker containers. They are open sourced on GitHub, so you can run them on your own infrastructure, customize them for your needs, or simply use meshStack's hosted runners.

Changing the Runner's Key Pair

If you replace the key pair after building blocks have stored encrypted inputs, those inputs can no longer be decrypted. Static inputs need a new definition version; user inputs need to be re-submitted by workspace users. Plan key rotation carefully.

Setting Up a Self-Hosted Runner​

The easiest way to set up a runner is through the Platform Builder or Admin Area UI in meshStack — the interface walks you through each field and generates the configuration for you. The high-level steps are:

  1. Generate an RSA key pair. meshStack encrypts sensitive building block inputs with the runner's public key. Only your runner, with the private key, can decrypt them.
  2. Register the runner in meshStack. Create a Building Block Runner in the Platform Builder/Admin Area (or via the meshObject API), providing your public key and the implementation type. meshStack assigns the runner a UUID.
  3. Create an API key for the runner. The runner needs an API key to poll meshStack for work. Once the runner is registered, provision one from the runner's detail page in meshStack — see Runner Visibility and API Key Permissions below for the exact permissions required.
  4. Deploy the runner container. Run the Docker image from the meshcloud container registry, passing the runner UUID, meshStack URL, API credentials, and private key as configuration. The runner starts polling immediately.
  5. Verify and assign. Confirm the runner shows as Active in meshStack, then select it on the Implementation tab of your building block definition.
Runner Status

meshStack marks a runner Active if it has polled within the last 5 minutes, Pending right after registration, and Error if it goes silent. You can see the last known runner version in the runner details view.

Runner Visibility and API Key Permissions​

Every runner is owned by a workspace and has a restriction that determines where it can be used:

  • Private runners (default) are restricted to the workspace that owns them. A private runner only executes runs for building block definitions owned by that same workspace. This is the only option available when registering a runner from a workspace's Platform Builder.
  • Public (shared) runners are owned by the admin workspace and available to every workspace in meshStack — they can execute runs for building block definitions owned by any workspace. Only admins can create public runners, from the Admin Area. meshStack's built-in hosted runners are public runners.

The API key that authenticates your runner needs different permissions depending on which type of runner it belongs to:

For a private runner owned by a platform team's workspace:

  • Read Runs — fetch run specifications and inputs
  • Write Runs — trigger and update runs
  • Write Run Sources — report run status, logs, and outputs

For a public runner owned by the admin workspace:

  • Read Runs (Admin) — fetch run specifications and inputs across all workspaces
  • Write Runs (Admin) — trigger and update runs across all workspaces
  • Write Run Sources (Admin) — report run status, logs, and outputs across all workspaces
Provisioning the Key

Use the Provision API Key action on the runner's detail page to have meshStack create the key for you with the correct permission set already applied, based on the runner's visibility. You don't need to pick permissions manually.

API Key and Runner Workspace

The API key must belong to the same workspace that owns the runner. For a private runner, that workspace must also own the building block definitions the runner should execute — for example, if workspace A owns a building block definition and a private runner assigned to it, a building block instantiated from that definition in workspace B is still executed using the runner (and API key) in workspace A. Public runners don't have this restriction, since they can execute definitions from any workspace.

What's Coming: Run Controller​

We're working on open-sourcing our run controller — a Kubernetes-native variant that spawns a short-lived Kubernetes Job for each building block run instead of a single continuously polling container. This is the same architecture we use for our built-in runners already.

Benefits over the Docker runner:

  • Parallelization — runs execute concurrently because each gets its own isolated job
  • All implementation types in one — a single run controller handles Terraform, GitHub Actions, GitLab CI/CD, Azure DevOps Pipelines, and Manual building blocks without deploying a separate container per type

If you set up a Docker-based runner today, you'll be able to migrate it to a run controller in the future without re-registering or rotating keys.

Concepts​

Guides​