Provider plugins

A provider is a deployment target: somewhere Loomplane can run a service or provision a resource. Loomplane ships with many (see the Clouds overview), and you can add your own without changing Loomplane.

A plugin is a small HTTP service. Loomplane reads its description at startup and puts it in the catalog next to the built-in targets, so manifests can name it, plans check against what it says it runs, and deployments, retries and drift checks drive it like any other. It runs on your infrastructure, so the credential it deploys with never leaves it, apart from the copy Loomplane keeps in Vault.

A provider in GoCopy Anchor Link

pkg/provider/sdk is the Go side of the protocol. You implement five methods and hand the result to sdk.Handler:

type Provider interface {
	Describe() provider.Provider
	Check(ctx context.Context, c sdk.Credential) (*sdk.Account, error)
	Apply(ctx context.Context, job *sdk.Job) (*sdk.Outcome, error)
	Remove(ctx context.Context, job *sdk.Job) error
	Observe(ctx context.Context, t *sdk.Target) (*sdk.Observation, error)
}

http.ListenAndServe(":9100", sdk.Handler(myProvider, os.Getenv("PLUGIN_TOKEN")))

examples/provider-plugin is a working one, a static site host in about 270 lines. Read it first, since it's the shortest way to a provider that works.

Describe returns the catalog entry: a name manifests use (provider: static-host), what it runs (Runs, by workload: http-service, web-app, static-site, worker), what it provisions (Provisions, by resource kind such as postgres or redis), its regions, and the Cloud its connections are for. Implement sdk.Connector too and the dashboard asks for a credential in your words, with the settings you list.

Check decides whether a credential works. You get what someone pasted under Integrations and the settings they chose. Return the account it acts as, details safe to show, and anything it's missing. A credential the cloud refused is an error wrapping sdk.ErrRejected, and Loomplane tells the person and never stores it. Any other error means the cloud couldn't be asked, and they can try again.

Apply makes a service or resource exist as the job describes. A service's job has its source (job.Source(), a tarball of the repository at its commit with one top-level directory, and that commit as job.Commit), the path of its entrypoint, its Dockerfile and build directory if it has one (Context, from the repository root), its port, whether it's public, its size and scale, and its variables, with the secret ones listed so you can put them in your cloud's secret store. A resource's job has its kind and version and no commit. A gateway's has no commit either, and its source is the program Loomplane generates for its routes.

Return an ExternalID you'll recognize later, a URL for a public service, a Revision that drift checks compare with, and for a resource an Address (a database URL, say). Loomplane keeps the address in Vault and hands it, as a secret, to the services that need the resource.

Remove deletes what a binding names, with the credential of the connection that deployed it, even when the environment deploys with another one now. Something already gone isn't an error. You also get a Remove for what a service or resource ran as before it moved to another provider, region or connection, once the new one is up. If that fails, later deployments of the action send it again, so it can arrive long after the move.

Observe reads what's live without changing anything. Return Exists: false when it's gone, and the live Revision. The drift check calls it on a schedule.

Run itCopy Anchor Link

Add the plugin to Loomplane's configuration:

providers:
  plugins:
    - url: http://static-host:9100
      token: "${STATIC_HOST_PLUGIN_TOKEN}"

The token is a shared secret between Loomplane and the plugin, and sdk.Handler refuses any request without it. Loomplane describes every plugin when it starts and won't start if one doesn't answer or speaks another protocol version, because it would otherwise accept plans it can't carry out. A plugin that stops answering later makes /_/health unhealthy within one check interval.

Then connect the plugin's cloud under Integrations and use it in loomplane.yaml:

services:
  docs:
    source: apps/docs
    provider: static-host

Rules a provider has to followCopy Anchor Link

Apply, Remove and Observe must be idempotent. A failed deployment can be retried from the step that failed, Loomplane can restart in the middle of one, and a plugin that restarts loses its operations, so the same job can arrive more than once. job.Key is the same for every attempt at the same action. Use it, or names derived from the job, so a second attempt finds what the first made and converges on it.

Long work should checkpoint. job.Checkpoint(data) saves something a retry gets back as job.Resume, such as a build's ID or an operation name. job.Progress(message) shows on the deployment's page what the step is doing.

Never run the repository's code on the plugin's machine unless that machine is meant for it. Build on your cloud's builder, the way the built-in targets use Cloud Build and CodeBuild.

Provider plugins are trusted with the credentials of the cloud they deploy to, so only configure ones you run or trust.

Build logsCopy Anchor Link

To opt a Go plugin into build logs, implement sdk.OperationLogger with OperationLogs() bool returning true and call job.Log(stream, message). The handler serves bounded pages at /v1/operations/{id}/logs. Older plugins can keep using progress callbacks. A plugin restart or retention loss produces a gap, and Loomplane doesn't pretend the older lines are complete.

The protocolCopy Anchor Link

You don't need Go. A plugin in any language that answers these requests as JSON, with Authorization: Bearer <token> on every request, is a provider.

Apply and Remove run as operations because builds and databases take minutes. Loomplane starts one, then reads it until it's done. Starting the same job while it's still running should return the running operation. An operation you no longer have is a 404, and Loomplane starts it again from the last checkpoint. pkg/provider/sdk/protocol.go has every type.

Test itCopy Anchor Link

internal/plugins has the client Loomplane uses, and its tests run it against sdk.Handler with a fake provider, which is the quickest check that a provider speaks the protocol. tests/integration/plugin_test.go goes further: it connects the example's cloud, then plans, deploys, re-plans and removes a site through a running Loomplane.