Add --endpoint-url flag to target externally-managed emulators

CLAUDE.md

@@ -133,6 +133,14 @@ Each `[[containers]]` block may set an optional `image` (override the default Do
 
 There is no `--offline` flag. Instead `container.Start` degrades gracefully when internet requests fail (Docker Hub unreachable, proxy/TLS interception, license server unreachable): local images are used when pulls fail, and the license pre-flight is skipped on transport-level failures, non-definitive server responses (5xx/407), or unsupported-tag rejections so the container validates its own bundled license. Definitive license rejections (HTTP 400/401/403) drop the cached license and offer an in-place re-login instead of requiring a manual `lstk logout` (DEVX-658). The exact fallback and retry rules live in `internal/container/CLAUDE.md`; pair them with a custom `image` in the config to point at a locally loaded image or an internal-registry mirror.
 
+# Targeting an External Emulator (`--endpoint-url`)
+
+`--endpoint-url <url>` (a root persistent flag) lets `aws`, `az`, `terraform`/`tf`, `cdk`, `sam`, `snapshot save`/`load` (incl. the `lstk save`/`lstk load` aliases), `snapshot remove`, `snapshot list s3://...`, `reset`, and `status` target an emulator lstk did not start — docker compose, host-network mode, CI, a different machine, or a LocalStack cloud-hosted **ephemeral instance** — instead of discovering one via local Docker. `LSTK_ENDPOINT_URL` is the equivalent, emulator-neutral environment variable; `AWS_ENDPOINT_URL` is a full synonym one precedence tier lower (so `LSTK_ENDPOINT_URL` wins if both are set), kept for users already accustomed to that variable from AWS tooling — it applies to every command above, not just the AWS-specific ones. Both `http://` and `https://` are accepted (any other scheme is rejected); the resolved scheme is preserved end-to-end all the way to whatever ultimately makes the request — the wrapped `aws`/`terraform`/`cdk`/`sam` tools, and the emulator API calls behind `snapshot`/`reset`/`status` — never normalized to `http`, which is what makes `https://` ephemeral instances work. Certificate trust follows standard TLS verification with no skip-verify escape hatch, so a self-signed/untrusted certificate fails with the underlying TLS error rather than a dedicated message. Resolution, validation, and the HTTP health/type probe live in `internal/endpoint/target.go` (`endpoint.Resolve`); an externally-managed target is represented downstream as a `runtime.NewExternalRuntime` stub (`internal/runtime/external.go`) so callers built around Docker-based discovery (`container.RunningEmulators`, `container.ResolveRunningContainerName`) work unchanged against it with zero real Docker involvement.
+
+`lstk logs`, `stop`, `restart`, `volume` (`path`/`clear`), `start`, and the bare `lstk` root command (which starts the emulator the same way `start` does) have no remote equivalent and reject *any* resolved endpoint source — the flag, `LSTK_ENDPOINT_URL`, or `AWS_ENDPOINT_URL` — with an actionable error naming which one fired. This rejection is unconditional: it fires even when a local Docker-managed emulator is genuinely running and reachable, since proceeding against it while an endpoint source says "the real target is elsewhere" is exactly the wrong-target risk (stopping/restarting/streaming logs from/clearing the volume of/starting a redundant instance of the wrong emulator) this guards against — an ambient env var left over from other work in the same shell session is not treated as harmless. `cmd/endpoint.go`'s `rejectEndpointURL` (built on `endpoint.ResolvedSource`, a presence-only check with no network probe) implements this and must be called before any local "is it running" lookup in each of these commands' `RunE`.
+
+Emulator type (aws/azure/snowflake) is always auto-detected by probing `/_localstack/health` (falling back to `/_localstack/info` for Azure, whose health response omits `version`) — there is no manual override flag or config setting; an inconclusive result is a hard failure. `terraform`/`cdk`/`sam` (AWS-only) reject a detected non-AWS type with the same error shape used for a wrong locally-running emulator.
+
 # Emulator Setup Commands
 
 Use `lstk setup <emulator>` to set up CLI integration for an emulator type:
@@ -164,9 +172,11 @@ Wrapped external tools (`aws`, `terraform`, `cdk`, `sam`, `az`, and extensions)
 
 `lstk snapshot` captures and restores the running emulator's state (for Snowflake and Azure a heads-up is shown that results may be incomplete). Domain logic lives in `internal/snapshot/`; `cmd/snapshot.go` is wiring + output-mode selection. Top-level `lstk save` / `lstk load` are aliases for the save/load subcommands.
 
-- `lstk snapshot save [destination]` (`-s`/`--services`: comma-separated list to limit a save to a subset of services; applies uniformly to local files, `pod:` cloud snapshots, and S3-remote saves) / `lstk snapshot load REF` (`--merge`: `account-region-merge` default, `overwrite`, `service-merge`) / `list` (cloud; `--all` for org-wide) / `remove REF` / `show REF` (remove/show are cloud-only).
+- `lstk snapshot save [destination]` (`-s`/`--services`: comma-separated list to limit a save to a subset of services; applies uniformly to local files, `pod:` cloud snapshots, and S3-remote saves) / `lstk snapshot load REF` (`--merge`: `account-region-merge` default, `overwrite`, `service-merge`) / `list` (cloud; `--all` for org-wide) / `remove REF` / `show REF`.
+- `show` and bare `list` (no `s3://` arg) only ever call the LocalStack platform API — never the emulator. `remove`, by contrast, proxies its delete through the running emulator despite being a "cloud" (`pod:`) concept, so it still requires one to be reachable; don't conflate "operates on cloud-hosted storage" with "never touches the emulator" — they're different axes.
 - A REF is a local `.snapshot` file, a `pod:` cloud snapshot on the LocalStack platform (requires auth), or an `s3://bucket/prefix` remote in the user's own bucket (the emulator performs the transfer; S3 supports save/load/list only).
 - A `[[containers]]` block (AWS only) can set `snapshot = "pod:..."` to auto-load after a fresh start; `lstk start --snapshot REF` overrides it for one run, `--no-snapshot` skips it.
+- `save`/`load`/`remove` and `list s3://...` support the global `--endpoint-url` targeting described under "Targeting an External Emulator"; `show` and bare `list` silently ignore it (they never touch the emulator regardless).
 
 REF parsing helpers, S3 credential precedence and remote-upsert mechanics, and the auto-load wiring are documented in `internal/snapshot/CLAUDE.md`.
 

README.md

@@ -46,6 +46,7 @@ Running `lstk` will automatically handle authentication, configuration, and cont
 - **Browser-based login** — authenticate via browser and store credentials securely in the system keyring, or use `LOCALSTACK_AUTH_TOKEN` for CI
 - **Snapshots** — save, load, and manage emulator state as local files, cloud snapshots, or in your own S3 bucket
 - **Cloud CLI proxies** — run `aws`, `az`, `terraform`, `cdk`, and `sam` commands against LocalStack with the endpoint, credentials, and region pre-configured
+- **Target an external emulator** — pass `--endpoint-url <url>` (or set `LSTK_ENDPOINT_URL`) to point most commands at an already-running LocalStack instance — docker compose, host-network mode, CI, a different machine, or a cloud-hosted ephemeral instance (`https://` is supported) — instead of one lstk manages locally
 - **Extensions** — Git-style `lstk-<name>` executables extend the CLI with new commands; see [docs/extensions-authoring.md](docs/extensions-authoring.md)
 - **Self-update** — `lstk update` checks for and installs the latest release
 - **Structured JSON output** — pass `--json` to a supported command for a machine-readable envelope instead of formatted text; see [docs/structured-output.md](docs/structured-output.md)

cmd/aws.go

@@ -39,6 +39,16 @@ Examples:
   lstk aws s3 mb s3://my-bucket`,
 		DisableFlagParsing: true,
 		PreRunE: func(cmd *cobra.Command, args []string) error {
+			// --endpoint-url is recognized only when it precedes "aws" (like
+			// --json below): the aws CLI has its own native --endpoint-url flag,
+			// so a post-command one must reach it untouched.
+			if strippedArgs, v, ok := stripPreCommandEndpointURL(cmd.CalledAs()); ok {
+				if err := cmd.Flags().Set(endpoint.FlagName, v); err != nil {
+					return err
+				}
+				args = strippedArgs
+			}
+
 			var gf globalFlags
 			passthrough, gf = stripGlobalFlags(args)
 			if gf.nonInteractive {
@@ -72,38 +82,52 @@ Examples:
 				return awscli.Exec(cmd.Context(), "", false, os.Stdout, os.Stderr, passthrough)
 			}
 
-			rt, err := runtime.NewDockerRuntime(cfg.DockerHost)
+			target, err := endpoint.Resolve(cmd.Context(), cmd)
 			if err != nil {
-				return err
+				return emitValidationError(sink, err)
 			}
 
-			appCfg, err := config.Get()
-			if err != nil {
-				return fmt.Errorf("failed to get config: %w", err)
-			}
+			var endpointURL string
+			if target != nil {
+				if target.Type != config.EmulatorAWS {
+					return emitValidationError(sink, fmt.Errorf("lstk aws requires the AWS emulator, but the endpoint at %s is a %s emulator", target.URL, target.Type.DisplayName()))
+				}
+				endpointURL = target.URL
+			} else {
+				rt, err := runtime.NewDockerRuntime(cfg.DockerHost)
+				if err != nil {
+					return err
+				}
 
-			awsContainer := config.ContainerConfig{Type: config.EmulatorAWS, Port: config.DefaultPort}
-			for _, c := range appCfg.Containers {
-				if c.Type == config.EmulatorAWS {
-					awsContainer = c
-					break
+				appCfg, err := config.Get()
+				if err != nil {
+					return fmt.Errorf("failed to get config: %w", err)
 				}
-			}
 
-			if err := rt.IsHealthy(cmd.Context()); err != nil {
-				rt.EmitUnhealthyError(sink, err)
-				return output.NewSilentError(fmt.Errorf("runtime not healthy: %w", err))
-			}
+				awsContainer := config.ContainerConfig{Type: config.EmulatorAWS, Port: config.DefaultPort}
+				for _, c := range appCfg.Containers {
+					if c.Type == config.EmulatorAWS {
+						awsContainer = c
+						break
+					}
+				}
 
-			runningName, err := container.ResolveRunningContainerName(cmd.Context(), rt, awsContainer)
-			if err != nil {
-				return fmt.Errorf("checking emulator status: %w", err)
-			}
-			if runningName == "" {
-				return container.HandleNoRunningContainer(sink, awsContainer)
-			}
+				if err := rt.IsHealthy(cmd.Context()); err != nil {
+					rt.EmitUnhealthyError(sink, err)
+					return output.NewSilentError(fmt.Errorf("runtime not healthy: %w", err))
+				}
 
-			host, _ := endpoint.ResolveHost(cmd.Context(), awsContainer.Port, cfg.LocalStackHost)
+				runningName, err := container.ResolveRunningContainerName(cmd.Context(), rt, awsContainer)
+				if err != nil {
+					return fmt.Errorf("checking emulator status: %w", err)
+				}
+				if runningName == "" {
+					return container.HandleNoRunningContainer(sink, awsContainer)
+				}
+
+				host, _ := endpoint.ResolveHost(cmd.Context(), awsContainer.Port, cfg.LocalStackHost)
+				endpointURL = "http://" + host
+			}
 
 			profileExists, _ := awsconfig.ProfileExists(cmd.Context())
 			if !profileExists {
@@ -119,7 +143,7 @@ Examples:
 				stderr = &terminal.StopOnWriteWriter{W: os.Stderr, Spinner: s}
 			}
 
-			return awscli.Exec(cmd.Context(), "http://"+host, profileExists, stdout, stderr, passthrough)
+			return awscli.Exec(cmd.Context(), endpointURL, profileExists, stdout, stderr, passthrough)
 		},
 	}
 }

cmd/az.go

@@ -21,6 +21,12 @@ import (
 )
 
 func newAzCmd(cfg *env.Env) *cobra.Command {
+	// DisableFlagParsing means Cobra won't strip a pre-command --endpoint-url;
+	// PreRunE does that and stashes the corrected args here for RunE to
+	// forward, mirroring aws.go/terraform.go/cdk.go/sam.go's passthrough
+	// pattern (RunE's own args parameter is NOT updated by PreRunE's local
+	// changes — Cobra calls each independently with its own resolved args).
+	var passthrough []string
 	cmd := &cobra.Command{
 		Use:   "az [args...]",
 		Short: "Run Azure CLI commands against LocalStack",
@@ -40,9 +46,19 @@ Examples:
 			if jsonPrecedesCommandName(cmd.CalledAs()) {
 				cfg.JSON = true
 			}
+			// --endpoint-url is recognized only when it precedes "az", the same
+			// pre-command-only placement --json already gets here.
+			if strippedArgs, v, ok := stripPreCommandEndpointURL(cmd.CalledAs()); ok {
+				if err := cmd.Flags().Set(endpoint.FlagName, v); err != nil {
+					return err
+				}
+				args = strippedArgs
+			}
+			passthrough = args
 			return initConfigDeferCreate(nil)(cmd, args)
 		},
-		RunE: func(cmd *cobra.Command, args []string) error {
+		RunE: func(cmd *cobra.Command, _ []string) error {
+			args := passthrough
 			sink := output.NewPlainSink(os.Stdout)
 
 			configDir, err := config.ConfigDir()
@@ -60,7 +76,12 @@ Examples:
 				return output.NewSilentError(fmt.Errorf("azure CLI integration not set up"))
 			}
 
-			if _, err := azPreflight(cmd.Context(), cfg, sink); err != nil {
+			target, err := endpoint.Resolve(cmd.Context(), cmd)
+			if err != nil {
+				return emitValidationError(sink, err)
+			}
+
+			if _, err := azPreflight(cmd.Context(), cfg, sink, target); err != nil {
 				return err
 			}
 
@@ -93,7 +114,9 @@ func newAzStartInterceptionCmd(cfg *env.Env) *cobra.Command {
 		PreRunE: initConfigDeferCreate(nil),
 		RunE: func(cmd *cobra.Command, args []string) error {
 			preflight := func(ctx context.Context, sink output.Sink) (string, error) {
-				return azPreflight(ctx, cfg, sink)
+				// --endpoint-url is out of scope for start-interception (see
+				// design.md's Non-Goals) — always resolve via Docker.
+				return azPreflight(ctx, cfg, sink, nil)
 			}
 
 			// Run preflight under the same sink as the operation so its errors render
@@ -136,7 +159,13 @@ func newAzStopInterceptionCmd(cfg *env.Env) *cobra.Command {
 // running, and *.localhost.localstack.cloud resolves. On failure it emits the matching
 // ErrorEvent and returns a silent error. On success it returns the resolved LocalStack
 // Azure endpoint URL.
-func azPreflight(ctx context.Context, cfg *env.Env, sink output.Sink) (string, error) {
+//
+// target, when non-nil, is an already-resolved externally-managed endpoint
+// (--endpoint-url/LSTK_ENDPOINT_URL/AWS_ENDPOINT_URL): all of the Docker/DNS
+// checks below are skipped in favor of it. 'start-interception'/
+// 'stop-interception' always pass nil — they are out of scope for
+// --endpoint-url (see design.md's Non-Goals).
+func azPreflight(ctx context.Context, cfg *env.Env, sink output.Sink, target *endpoint.Target) (string, error) {
 	if err := azurecli.CheckInstalled(); err != nil {
 		sink.Emit(output.ErrorEvent{
 			Title:   "az CLI not found in PATH",
@@ -145,6 +174,15 @@ func azPreflight(ctx context.Context, cfg *env.Env, sink output.Sink) (string, e
 		return "", output.NewSilentError(err)
 	}
 
+	if target != nil {
+		if target.Type != config.EmulatorAzure {
+			err := fmt.Errorf("lstk az requires the Azure emulator, but the endpoint at %s is a %s emulator", target.URL, target.Type.DisplayName())
+			sink.Emit(output.ErrorEvent{Title: err.Error()})
+			return "", output.NewSilentError(err)
+		}
+		return azureconfig.BuildEndpoint(target.HostPort()), nil
+	}
+
 	appCfg, err := config.Get()
 	if err != nil {
 		return "", fmt.Errorf("failed to get config: %w", err)

cmd/cdk.go

@@ -4,6 +4,7 @@ import (
 	"fmt"
 	"os"
 
+	"github.com/localstack/lstk/internal/config"
 	"github.com/localstack/lstk/internal/endpoint"
 	"github.com/localstack/lstk/internal/env"
 	cdkcli "github.com/localstack/lstk/internal/iac/cdk/cli"
@@ -30,7 +31,8 @@ lstk-specific flags (must appear before the cdk action):
 CDK always targets the default LocalStack account 000000000000; there is no --account flag.
 
 Supported environment variables:
-  AWS_ENDPOINT_URL      Override the auto-resolved LocalStack endpoint
+  LSTK_ENDPOINT_URL     Target an externally-managed emulator
+  AWS_ENDPOINT_URL      Same as LSTK_ENDPOINT_URL (lower precedence if both are set)
   AWS_ENDPOINT_URL_S3   Override the auto-derived S3 endpoint
   LSTK_CDK_CMD          CDK binary to invoke (default cdk)
   AWS_REGION            Fallback for --region
@@ -41,6 +43,15 @@ Examples:
   lstk cdk synth`,
 		DisableFlagParsing: true,
 		PreRunE: func(cmd *cobra.Command, args []string) error {
+			// --endpoint-url is recognized only when it precedes "cdk", the
+			// same pre-command-only placement --json already gets here.
+			if strippedArgs, v, ok := stripPreCommandEndpointURL(cmd.CalledAs()); ok {
+				if err := cmd.Flags().Set(endpoint.FlagName, v); err != nil {
+					return err
+				}
+				args = strippedArgs
+			}
+
 			var gf globalFlags
 			passthrough, gf = stripGlobalFlags(args)
 			if gf.nonInteractive {
@@ -79,14 +90,33 @@ Examples:
 
 			region := resolveRegion(regionFlag)
 
+			target, err := endpoint.Resolve(cmd.Context(), cmd)
+			if err != nil {
+				return emitValidationError(sink, err)
+			}
+			if target != nil && target.Type != config.EmulatorAWS {
+				return emitValidationError(sink, fmt.Errorf("lstk cdk requires the AWS emulator, but the endpoint at %s is a %s emulator", target.URL, target.Type.DisplayName()))
+			}
+
 			awsContainer := resolveAWSContainer()
 
 			// Offline subcommands never contact AWS, so they run without a
-			// running emulator. We still resolve the endpoint (DNS only) and
-			// inject it, so a synth-time context lookup routes to LocalStack.
+			// running emulator. We still resolve the endpoint (DNS only, or the
+			// externally-managed target) and inject it, so a synth-time context
+			// lookup routes to LocalStack.
 			if cdkcli.IsOffline(cdkArgs) {
-				host, _ := endpoint.ResolveHost(cmd.Context(), awsContainer.Port, cfg.LocalStackHost)
-				return cdkcli.Run(cmd.Context(), "http://"+host, region, sink, logger, cdkArgs)
+				endpointURL := ""
+				if target != nil {
+					endpointURL = target.URL
+				} else {
+					host, _ := endpoint.ResolveHost(cmd.Context(), awsContainer.Port, cfg.LocalStackHost)
+					endpointURL = "http://" + host
+				}
+				return cdkcli.Run(cmd.Context(), endpointURL, region, sink, logger, cdkArgs)
+			}
+
+			if target != nil {
+				return cdkcli.Run(cmd.Context(), target.URL, region, sink, logger, cdkArgs)
 			}
 
 			rt, err := runtime.NewDockerRuntime(cfg.DockerHost)

cmd/endpoint.go

@@ -0,0 +1,29 @@
+package cmd
+
+import (
+	"fmt"
+	"strings"
+
+	"github.com/localstack/lstk/internal/endpoint"
+	"github.com/localstack/lstk/internal/output"
+	"github.com/spf13/cobra"
+)
+
+// rejectEndpointURL returns an actionable error if any endpoint URL source —
+// an explicit --endpoint-url flag, LSTK_ENDPOINT_URL, or AWS_ENDPOINT_URL — is
+// present for this invocation, for a command that has no remote equivalent
+// (the Docker-lifecycle/filesystem operations: logs, stop, restart, volume,
+// start).
+func rejectEndpointURL(cmd *cobra.Command, sink output.Sink, label string) error {
+	source, _, ok := endpoint.ResolvedSource(cmd)
+	if !ok {
+		return nil
+	}
+	how := source + " is set"
+	if strings.HasPrefix(source, "--") {
+		how = source + " was passed"
+	}
+	err := fmt.Errorf("%s does not support %s: it operates on a local Docker container or local filesystem state with no remote equivalent (%s)", label, source, how)
+	sink.Emit(output.ErrorEvent{Title: err.Error()})
+	return output.NewSilentError(err)
+}