diff --git a/CLAUDE.md b/CLAUDE.md index 955e58fe..33e33cf5 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -133,6 +133,14 @@ Each `[[containers]]` block may set an optional `image` (override the default Do There is no `--offline` flag. Instead `container.Start` degrades gracefully when internet requests fail (Docker Hub unreachable, proxy/TLS interception, license server unreachable): local images are used when pulls fail, and the license pre-flight is skipped on transport-level failures, non-definitive server responses (5xx/407), or unsupported-tag rejections so the container validates its own bundled license. Definitive license rejections (HTTP 400/401/403) drop the cached license and offer an in-place re-login instead of requiring a manual `lstk logout` (DEVX-658). The exact fallback and retry rules live in `internal/container/CLAUDE.md`; pair them with a custom `image` in the config to point at a locally loaded image or an internal-registry mirror. +# Targeting an External Emulator (`--endpoint-url`) + +`--endpoint-url ` (a root persistent flag) lets `aws`, `az`, `terraform`/`tf`, `cdk`, `sam`, `snapshot save`/`load` (incl. the `lstk save`/`lstk load` aliases), `snapshot remove`, `snapshot list s3://...`, `reset`, and `status` target an emulator lstk did not start — docker compose, host-network mode, CI, a different machine, or a LocalStack cloud-hosted **ephemeral instance** — instead of discovering one via local Docker. `LSTK_ENDPOINT_URL` is the equivalent, emulator-neutral environment variable; `AWS_ENDPOINT_URL` is a full synonym one precedence tier lower (so `LSTK_ENDPOINT_URL` wins if both are set), kept for users already accustomed to that variable from AWS tooling — it applies to every command above, not just the AWS-specific ones. Both `http://` and `https://` are accepted (any other scheme is rejected); the resolved scheme is preserved end-to-end all the way to whatever ultimately makes the request — the wrapped `aws`/`terraform`/`cdk`/`sam` tools, and the emulator API calls behind `snapshot`/`reset`/`status` — never normalized to `http`, which is what makes `https://` ephemeral instances work. Certificate trust follows standard TLS verification with no skip-verify escape hatch, so a self-signed/untrusted certificate fails with the underlying TLS error rather than a dedicated message. Resolution, validation, and the HTTP health/type probe live in `internal/endpoint/target.go` (`endpoint.Resolve`); an externally-managed target is represented downstream as a `runtime.NewExternalRuntime` stub (`internal/runtime/external.go`) so callers built around Docker-based discovery (`container.RunningEmulators`, `container.ResolveRunningContainerName`) work unchanged against it with zero real Docker involvement. + +`lstk logs`, `stop`, `restart`, `volume` (`path`/`clear`), `start`, and the bare `lstk` root command (which starts the emulator the same way `start` does) have no remote equivalent and reject *any* resolved endpoint source — the flag, `LSTK_ENDPOINT_URL`, or `AWS_ENDPOINT_URL` — with an actionable error naming which one fired. This rejection is unconditional: it fires even when a local Docker-managed emulator is genuinely running and reachable, since proceeding against it while an endpoint source says "the real target is elsewhere" is exactly the wrong-target risk (stopping/restarting/streaming logs from/clearing the volume of/starting a redundant instance of the wrong emulator) this guards against — an ambient env var left over from other work in the same shell session is not treated as harmless. `cmd/endpoint.go`'s `rejectEndpointURL` (built on `endpoint.ResolvedSource`, a presence-only check with no network probe) implements this and must be called before any local "is it running" lookup in each of these commands' `RunE`. + +Emulator type (aws/azure/snowflake) is always auto-detected by probing `/_localstack/health` (falling back to `/_localstack/info` for Azure, whose health response omits `version`) — there is no manual override flag or config setting; an inconclusive result is a hard failure. `terraform`/`cdk`/`sam` (AWS-only) reject a detected non-AWS type with the same error shape used for a wrong locally-running emulator. + # Emulator Setup Commands Use `lstk setup ` to set up CLI integration for an emulator type: @@ -164,9 +172,11 @@ Wrapped external tools (`aws`, `terraform`, `cdk`, `sam`, `az`, and extensions) `lstk snapshot` captures and restores the running emulator's state (for Snowflake and Azure a heads-up is shown that results may be incomplete). Domain logic lives in `internal/snapshot/`; `cmd/snapshot.go` is wiring + output-mode selection. Top-level `lstk save` / `lstk load` are aliases for the save/load subcommands. -- `lstk snapshot save [destination]` (`-s`/`--services`: comma-separated list to limit a save to a subset of services; applies uniformly to local files, `pod:` cloud snapshots, and S3-remote saves) / `lstk snapshot load REF` (`--merge`: `account-region-merge` default, `overwrite`, `service-merge`) / `list` (cloud; `--all` for org-wide) / `remove REF` / `show REF` (remove/show are cloud-only). +- `lstk snapshot save [destination]` (`-s`/`--services`: comma-separated list to limit a save to a subset of services; applies uniformly to local files, `pod:` cloud snapshots, and S3-remote saves) / `lstk snapshot load REF` (`--merge`: `account-region-merge` default, `overwrite`, `service-merge`) / `list` (cloud; `--all` for org-wide) / `remove REF` / `show REF`. +- `show` and bare `list` (no `s3://` arg) only ever call the LocalStack platform API — never the emulator. `remove`, by contrast, proxies its delete through the running emulator despite being a "cloud" (`pod:`) concept, so it still requires one to be reachable; don't conflate "operates on cloud-hosted storage" with "never touches the emulator" — they're different axes. - A REF is a local `.snapshot` file, a `pod:` cloud snapshot on the LocalStack platform (requires auth), or an `s3://bucket/prefix` remote in the user's own bucket (the emulator performs the transfer; S3 supports save/load/list only). - A `[[containers]]` block (AWS only) can set `snapshot = "pod:..."` to auto-load after a fresh start; `lstk start --snapshot REF` overrides it for one run, `--no-snapshot` skips it. +- `save`/`load`/`remove` and `list s3://...` support the global `--endpoint-url` targeting described under "Targeting an External Emulator"; `show` and bare `list` silently ignore it (they never touch the emulator regardless). REF parsing helpers, S3 credential precedence and remote-upsert mechanics, and the auto-load wiring are documented in `internal/snapshot/CLAUDE.md`. diff --git a/README.md b/README.md index 6f6ae5d7..170543fb 100644 --- a/README.md +++ b/README.md @@ -46,6 +46,7 @@ Running `lstk` will automatically handle authentication, configuration, and cont - **Browser-based login** — authenticate via browser and store credentials securely in the system keyring, or use `LOCALSTACK_AUTH_TOKEN` for CI - **Snapshots** — save, load, and manage emulator state as local files, cloud snapshots, or in your own S3 bucket - **Cloud CLI proxies** — run `aws`, `az`, `terraform`, `cdk`, and `sam` commands against LocalStack with the endpoint, credentials, and region pre-configured +- **Target an external emulator** — pass `--endpoint-url ` (or set `LSTK_ENDPOINT_URL`) to point most commands at an already-running LocalStack instance — docker compose, host-network mode, CI, a different machine, or a cloud-hosted ephemeral instance (`https://` is supported) — instead of one lstk manages locally - **Extensions** — Git-style `lstk-` executables extend the CLI with new commands; see [docs/extensions-authoring.md](docs/extensions-authoring.md) - **Self-update** — `lstk update` checks for and installs the latest release - **Structured JSON output** — pass `--json` to a supported command for a machine-readable envelope instead of formatted text; see [docs/structured-output.md](docs/structured-output.md) diff --git a/cmd/aws.go b/cmd/aws.go index 299bcc5b..34a14041 100644 --- a/cmd/aws.go +++ b/cmd/aws.go @@ -39,6 +39,16 @@ Examples: lstk aws s3 mb s3://my-bucket`, DisableFlagParsing: true, PreRunE: func(cmd *cobra.Command, args []string) error { + // --endpoint-url is recognized only when it precedes "aws" (like + // --json below): the aws CLI has its own native --endpoint-url flag, + // so a post-command one must reach it untouched. + if strippedArgs, v, ok := stripPreCommandEndpointURL(cmd.CalledAs()); ok { + if err := cmd.Flags().Set(endpoint.FlagName, v); err != nil { + return err + } + args = strippedArgs + } + var gf globalFlags passthrough, gf = stripGlobalFlags(args) if gf.nonInteractive { @@ -72,38 +82,52 @@ Examples: return awscli.Exec(cmd.Context(), "", false, os.Stdout, os.Stderr, passthrough) } - rt, err := runtime.NewDockerRuntime(cfg.DockerHost) + target, err := endpoint.Resolve(cmd.Context(), cmd) if err != nil { - return err + return emitValidationError(sink, err) } - appCfg, err := config.Get() - if err != nil { - return fmt.Errorf("failed to get config: %w", err) - } + var endpointURL string + if target != nil { + if target.Type != config.EmulatorAWS { + return emitValidationError(sink, fmt.Errorf("lstk aws requires the AWS emulator, but the endpoint at %s is a %s emulator", target.URL, target.Type.DisplayName())) + } + endpointURL = target.URL + } else { + rt, err := runtime.NewDockerRuntime(cfg.DockerHost) + if err != nil { + return err + } - awsContainer := config.ContainerConfig{Type: config.EmulatorAWS, Port: config.DefaultPort} - for _, c := range appCfg.Containers { - if c.Type == config.EmulatorAWS { - awsContainer = c - break + appCfg, err := config.Get() + if err != nil { + return fmt.Errorf("failed to get config: %w", err) } - } - if err := rt.IsHealthy(cmd.Context()); err != nil { - rt.EmitUnhealthyError(sink, err) - return output.NewSilentError(fmt.Errorf("runtime not healthy: %w", err)) - } + awsContainer := config.ContainerConfig{Type: config.EmulatorAWS, Port: config.DefaultPort} + for _, c := range appCfg.Containers { + if c.Type == config.EmulatorAWS { + awsContainer = c + break + } + } - runningName, err := container.ResolveRunningContainerName(cmd.Context(), rt, awsContainer) - if err != nil { - return fmt.Errorf("checking emulator status: %w", err) - } - if runningName == "" { - return container.HandleNoRunningContainer(sink, awsContainer) - } + if err := rt.IsHealthy(cmd.Context()); err != nil { + rt.EmitUnhealthyError(sink, err) + return output.NewSilentError(fmt.Errorf("runtime not healthy: %w", err)) + } - host, _ := endpoint.ResolveHost(cmd.Context(), awsContainer.Port, cfg.LocalStackHost) + runningName, err := container.ResolveRunningContainerName(cmd.Context(), rt, awsContainer) + if err != nil { + return fmt.Errorf("checking emulator status: %w", err) + } + if runningName == "" { + return container.HandleNoRunningContainer(sink, awsContainer) + } + + host, _ := endpoint.ResolveHost(cmd.Context(), awsContainer.Port, cfg.LocalStackHost) + endpointURL = "http://" + host + } profileExists, _ := awsconfig.ProfileExists(cmd.Context()) if !profileExists { @@ -119,7 +143,7 @@ Examples: stderr = &terminal.StopOnWriteWriter{W: os.Stderr, Spinner: s} } - return awscli.Exec(cmd.Context(), "http://"+host, profileExists, stdout, stderr, passthrough) + return awscli.Exec(cmd.Context(), endpointURL, profileExists, stdout, stderr, passthrough) }, } } diff --git a/cmd/az.go b/cmd/az.go index 65420f48..d327e70b 100644 --- a/cmd/az.go +++ b/cmd/az.go @@ -21,6 +21,12 @@ import ( ) func newAzCmd(cfg *env.Env) *cobra.Command { + // DisableFlagParsing means Cobra won't strip a pre-command --endpoint-url; + // PreRunE does that and stashes the corrected args here for RunE to + // forward, mirroring aws.go/terraform.go/cdk.go/sam.go's passthrough + // pattern (RunE's own args parameter is NOT updated by PreRunE's local + // changes — Cobra calls each independently with its own resolved args). + var passthrough []string cmd := &cobra.Command{ Use: "az [args...]", Short: "Run Azure CLI commands against LocalStack", @@ -40,9 +46,19 @@ Examples: if jsonPrecedesCommandName(cmd.CalledAs()) { cfg.JSON = true } + // --endpoint-url is recognized only when it precedes "az", the same + // pre-command-only placement --json already gets here. + if strippedArgs, v, ok := stripPreCommandEndpointURL(cmd.CalledAs()); ok { + if err := cmd.Flags().Set(endpoint.FlagName, v); err != nil { + return err + } + args = strippedArgs + } + passthrough = args return initConfigDeferCreate(nil)(cmd, args) }, - RunE: func(cmd *cobra.Command, args []string) error { + RunE: func(cmd *cobra.Command, _ []string) error { + args := passthrough sink := output.NewPlainSink(os.Stdout) configDir, err := config.ConfigDir() @@ -60,7 +76,12 @@ Examples: return output.NewSilentError(fmt.Errorf("azure CLI integration not set up")) } - if _, err := azPreflight(cmd.Context(), cfg, sink); err != nil { + target, err := endpoint.Resolve(cmd.Context(), cmd) + if err != nil { + return emitValidationError(sink, err) + } + + if _, err := azPreflight(cmd.Context(), cfg, sink, target); err != nil { return err } @@ -93,7 +114,9 @@ func newAzStartInterceptionCmd(cfg *env.Env) *cobra.Command { PreRunE: initConfigDeferCreate(nil), RunE: func(cmd *cobra.Command, args []string) error { preflight := func(ctx context.Context, sink output.Sink) (string, error) { - return azPreflight(ctx, cfg, sink) + // --endpoint-url is out of scope for start-interception (see + // design.md's Non-Goals) — always resolve via Docker. + return azPreflight(ctx, cfg, sink, nil) } // Run preflight under the same sink as the operation so its errors render @@ -136,7 +159,13 @@ func newAzStopInterceptionCmd(cfg *env.Env) *cobra.Command { // running, and *.localhost.localstack.cloud resolves. On failure it emits the matching // ErrorEvent and returns a silent error. On success it returns the resolved LocalStack // Azure endpoint URL. -func azPreflight(ctx context.Context, cfg *env.Env, sink output.Sink) (string, error) { +// +// target, when non-nil, is an already-resolved externally-managed endpoint +// (--endpoint-url/LSTK_ENDPOINT_URL/AWS_ENDPOINT_URL): all of the Docker/DNS +// checks below are skipped in favor of it. 'start-interception'/ +// 'stop-interception' always pass nil — they are out of scope for +// --endpoint-url (see design.md's Non-Goals). +func azPreflight(ctx context.Context, cfg *env.Env, sink output.Sink, target *endpoint.Target) (string, error) { if err := azurecli.CheckInstalled(); err != nil { sink.Emit(output.ErrorEvent{ Title: "az CLI not found in PATH", @@ -145,6 +174,15 @@ func azPreflight(ctx context.Context, cfg *env.Env, sink output.Sink) (string, e return "", output.NewSilentError(err) } + if target != nil { + if target.Type != config.EmulatorAzure { + err := fmt.Errorf("lstk az requires the Azure emulator, but the endpoint at %s is a %s emulator", target.URL, target.Type.DisplayName()) + sink.Emit(output.ErrorEvent{Title: err.Error()}) + return "", output.NewSilentError(err) + } + return azureconfig.BuildEndpoint(target.HostPort()), nil + } + appCfg, err := config.Get() if err != nil { return "", fmt.Errorf("failed to get config: %w", err) diff --git a/cmd/cdk.go b/cmd/cdk.go index 7a756021..d5edd107 100644 --- a/cmd/cdk.go +++ b/cmd/cdk.go @@ -4,6 +4,7 @@ import ( "fmt" "os" + "github.com/localstack/lstk/internal/config" "github.com/localstack/lstk/internal/endpoint" "github.com/localstack/lstk/internal/env" cdkcli "github.com/localstack/lstk/internal/iac/cdk/cli" @@ -30,7 +31,8 @@ lstk-specific flags (must appear before the cdk action): CDK always targets the default LocalStack account 000000000000; there is no --account flag. Supported environment variables: - AWS_ENDPOINT_URL Override the auto-resolved LocalStack endpoint + LSTK_ENDPOINT_URL Target an externally-managed emulator + AWS_ENDPOINT_URL Same as LSTK_ENDPOINT_URL (lower precedence if both are set) AWS_ENDPOINT_URL_S3 Override the auto-derived S3 endpoint LSTK_CDK_CMD CDK binary to invoke (default cdk) AWS_REGION Fallback for --region @@ -41,6 +43,15 @@ Examples: lstk cdk synth`, DisableFlagParsing: true, PreRunE: func(cmd *cobra.Command, args []string) error { + // --endpoint-url is recognized only when it precedes "cdk", the + // same pre-command-only placement --json already gets here. + if strippedArgs, v, ok := stripPreCommandEndpointURL(cmd.CalledAs()); ok { + if err := cmd.Flags().Set(endpoint.FlagName, v); err != nil { + return err + } + args = strippedArgs + } + var gf globalFlags passthrough, gf = stripGlobalFlags(args) if gf.nonInteractive { @@ -79,14 +90,33 @@ Examples: region := resolveRegion(regionFlag) + target, err := endpoint.Resolve(cmd.Context(), cmd) + if err != nil { + return emitValidationError(sink, err) + } + if target != nil && target.Type != config.EmulatorAWS { + return emitValidationError(sink, fmt.Errorf("lstk cdk requires the AWS emulator, but the endpoint at %s is a %s emulator", target.URL, target.Type.DisplayName())) + } + awsContainer := resolveAWSContainer() // Offline subcommands never contact AWS, so they run without a - // running emulator. We still resolve the endpoint (DNS only) and - // inject it, so a synth-time context lookup routes to LocalStack. + // running emulator. We still resolve the endpoint (DNS only, or the + // externally-managed target) and inject it, so a synth-time context + // lookup routes to LocalStack. if cdkcli.IsOffline(cdkArgs) { - host, _ := endpoint.ResolveHost(cmd.Context(), awsContainer.Port, cfg.LocalStackHost) - return cdkcli.Run(cmd.Context(), "http://"+host, region, sink, logger, cdkArgs) + endpointURL := "" + if target != nil { + endpointURL = target.URL + } else { + host, _ := endpoint.ResolveHost(cmd.Context(), awsContainer.Port, cfg.LocalStackHost) + endpointURL = "http://" + host + } + return cdkcli.Run(cmd.Context(), endpointURL, region, sink, logger, cdkArgs) + } + + if target != nil { + return cdkcli.Run(cmd.Context(), target.URL, region, sink, logger, cdkArgs) } rt, err := runtime.NewDockerRuntime(cfg.DockerHost) diff --git a/cmd/endpoint.go b/cmd/endpoint.go new file mode 100644 index 00000000..6eaaad44 --- /dev/null +++ b/cmd/endpoint.go @@ -0,0 +1,29 @@ +package cmd + +import ( + "fmt" + "strings" + + "github.com/localstack/lstk/internal/endpoint" + "github.com/localstack/lstk/internal/output" + "github.com/spf13/cobra" +) + +// rejectEndpointURL returns an actionable error if any endpoint URL source — +// an explicit --endpoint-url flag, LSTK_ENDPOINT_URL, or AWS_ENDPOINT_URL — is +// present for this invocation, for a command that has no remote equivalent +// (the Docker-lifecycle/filesystem operations: logs, stop, restart, volume, +// start). +func rejectEndpointURL(cmd *cobra.Command, sink output.Sink, label string) error { + source, _, ok := endpoint.ResolvedSource(cmd) + if !ok { + return nil + } + how := source + " is set" + if strings.HasPrefix(source, "--") { + how = source + " was passed" + } + err := fmt.Errorf("%s does not support %s: it operates on a local Docker container or local filesystem state with no remote equivalent (%s)", label, source, how) + sink.Emit(output.ErrorEvent{Title: err.Error()}) + return output.NewSilentError(err) +} diff --git a/cmd/endpoint_test.go b/cmd/endpoint_test.go new file mode 100644 index 00000000..b2c2529d --- /dev/null +++ b/cmd/endpoint_test.go @@ -0,0 +1,87 @@ +package cmd + +import ( + "testing" + + "github.com/localstack/lstk/internal/endpoint" + "github.com/localstack/lstk/internal/output" + "github.com/spf13/cobra" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +type captureSink struct { + events []output.Event +} + +func (s *captureSink) Emit(event output.Event) { + s.events = append(s.events, event) +} + +func cmdWithEndpointFlag(t *testing.T) *cobra.Command { + t.Helper() + cmd := &cobra.Command{Use: "test"} + cmd.Flags().String(endpoint.FlagName, "", "") + return cmd +} + +func TestRejectEndpointURL(t *testing.T) { + t.Run("no source present is a no-op", func(t *testing.T) { + cmd := cmdWithEndpointFlag(t) + sink := &captureSink{} + err := rejectEndpointURL(cmd, sink, "restart") + require.NoError(t, err) + assert.Empty(t, sink.events) + }) + + t.Run("explicit flag is rejected", func(t *testing.T) { + cmd := cmdWithEndpointFlag(t) + require.NoError(t, cmd.Flags().Set(endpoint.FlagName, "http://localhost:4566")) + sink := &captureSink{} + err := rejectEndpointURL(cmd, sink, "restart") + require.Error(t, err) + assert.True(t, output.IsSilent(err)) + require.Len(t, sink.events, 1) + errEvent, ok := sink.events[0].(output.ErrorEvent) + require.True(t, ok) + assert.Contains(t, errEvent.Title, "restart does not support --endpoint-url") + assert.Contains(t, errEvent.Title, "--endpoint-url was passed") + }) + + t.Run("ambient LSTK_ENDPOINT_URL is rejected, not silently ignored", func(t *testing.T) { + t.Setenv("LSTK_ENDPOINT_URL", "http://localhost:4566") + cmd := cmdWithEndpointFlag(t) + sink := &captureSink{} + err := rejectEndpointURL(cmd, sink, "stop") + require.Error(t, err) + require.Len(t, sink.events, 1) + errEvent, ok := sink.events[0].(output.ErrorEvent) + require.True(t, ok) + assert.Contains(t, errEvent.Title, "stop does not support LSTK_ENDPOINT_URL") + assert.Contains(t, errEvent.Title, "LSTK_ENDPOINT_URL is set") + }) + + t.Run("ambient AWS_ENDPOINT_URL is rejected too", func(t *testing.T) { + t.Setenv("AWS_ENDPOINT_URL", "http://localhost:4566") + cmd := cmdWithEndpointFlag(t) + sink := &captureSink{} + err := rejectEndpointURL(cmd, sink, "logs") + require.Error(t, err) + require.Len(t, sink.events, 1) + errEvent, ok := sink.events[0].(output.ErrorEvent) + require.True(t, ok) + assert.Contains(t, errEvent.Title, "logs does not support AWS_ENDPOINT_URL") + assert.Contains(t, errEvent.Title, "AWS_ENDPOINT_URL is set") + }) + + t.Run("start is rejected identically to the other lifecycle commands", func(t *testing.T) { + t.Setenv("LSTK_ENDPOINT_URL", "http://localhost:4566") + cmd := cmdWithEndpointFlag(t) + sink := &captureSink{} + err := rejectEndpointURL(cmd, sink, "start") + require.Error(t, err) + errEvent, ok := sink.events[0].(output.ErrorEvent) + require.True(t, ok) + assert.Contains(t, errEvent.Title, "start does not support LSTK_ENDPOINT_URL") + }) +} diff --git a/cmd/logs.go b/cmd/logs.go index 07f2b89b..7e9323f4 100644 --- a/cmd/logs.go +++ b/cmd/logs.go @@ -36,6 +36,10 @@ func newLogsCmd(cfg *env.Env) *cobra.Command { if err := validateTail(tail); err != nil { return err } + sink := output.NewPlainSink(os.Stdout) + if err := rejectEndpointURL(cmd, sink, "logs"); err != nil { + return err + } rt, err := runtime.NewDockerRuntime(cfg.DockerHost) if err != nil { return err @@ -47,7 +51,7 @@ func newLogsCmd(cfg *env.Env) *cobra.Command { if isInteractiveMode(cfg) { return ui.RunLogs(cmd.Context(), rt, appConfig.Containers, follow, tail, verbose) } - return container.Logs(cmd.Context(), rt, output.NewPlainSink(os.Stdout), appConfig.Containers, follow, tail, verbose) + return container.Logs(cmd.Context(), rt, sink, appConfig.Containers, follow, tail, verbose) }, } cmd.Flags().BoolP("follow", "f", false, "Follow log output") diff --git a/cmd/proxy.go b/cmd/proxy.go index f63b96c5..6482fc98 100644 --- a/cmd/proxy.go +++ b/cmd/proxy.go @@ -94,3 +94,56 @@ func jsonPrecedesCommandName(calledAs string) bool { } return result } + +// stripPreCommandEndpointURL returns the value of a --endpoint-url (or +// --endpoint-url=) that precedes the literal token calledAs in the raw +// command line, along with a corrected args slice (equivalent to what Cobra +// hands PreRunE/RunE for a DisableFlagParsing command) with that occurrence +// removed. An occurrence AFTER calledAs is left untouched in the returned +// args — for `aws` specifically, this preserves the aws CLI's own native +// --endpoint-url flag (a post-command one must reach the wrapped binary +// unchanged); for the other four proxy commands there's no such collision, +// but the same pre-command-only rule applies for consistency, mirroring +// --json's existing treatment. +// +// This must actually remove the pre-command occurrence, not just detect it +// (unlike jsonPrecedesCommandName, which leaves --json in place since it's a +// bare boolean signal): Cobra's own command resolution strips only the +// matched command token for a DisableFlagParsing command, regardless of +// where a flag appeared relative to it — so an unstripped pre-command +// --endpoint-url would leak into the forwarded args, duplicating alongside +// lstk's own injected --endpoint-url built from the resolved target. +func stripPreCommandEndpointURL(calledAs string) (args []string, value string, found bool) { + cmdIdx := -1 + for i, a := range os.Args { + if a == calledAs { + cmdIdx = i + break + } + } + if cmdIdx <= 0 { + return os.Args[1:], "", false + } + + before := os.Args[1:cmdIdx] + after := os.Args[cmdIdx+1:] + + stripped := make([]string, 0, len(before)) + for i := 0; i < len(before); i++ { + a := before[i] + switch { + case a == "--endpoint-url": + if i+1 < len(before) { + value, found = before[i+1], true + i++ + continue + } + stripped = append(stripped, a) + case strings.HasPrefix(a, "--endpoint-url="): + value, found = strings.TrimPrefix(a, "--endpoint-url="), true + default: + stripped = append(stripped, a) + } + } + return append(stripped, after...), value, found +} diff --git a/cmd/reset.go b/cmd/reset.go index 5363fec2..a4c4aaac 100644 --- a/cmd/reset.go +++ b/cmd/reset.go @@ -2,6 +2,7 @@ package cmd import ( "errors" + "fmt" "os" "github.com/localstack/lstk/internal/config" @@ -39,22 +40,46 @@ To wipe the on-disk volume (certificates, persistence data, cached tools) instea return output.NewSilentError(bare) } - appConfig, err := config.Get() + target, err := endpoint.Resolve(cmd.Context(), cmd) if err != nil { - return failGetConfig(sink, cfg, err) + return failWithCode(err.Error(), output.ErrEmulatorNotRunning) } + var rt runtime.Runtime + var host string var awsContainer config.ContainerConfig - var found bool - for _, c := range appConfig.Containers { - if c.Type == config.EmulatorAWS { - awsContainer = c - found = true - break + + if target != nil { + if target.Type != config.EmulatorAWS { + return failWithCode(fmt.Sprintf("reset is only supported for the AWS emulator, but the endpoint at %s is a %s emulator", target.URL, target.Type.DisplayName()), output.ErrEmulatorWrongType) } - } - if !found { - return failWithCode("reset is only supported for the AWS emulator", output.ErrEmulatorNotConfigured) + awsContainer = config.ContainerConfig{Type: config.EmulatorAWS, Port: config.DefaultPort} + rt = runtime.NewExternalRuntime(awsContainer.Name()) + host = target.URL + } else { + appConfig, err := config.Get() + if err != nil { + return failGetConfig(sink, cfg, err) + } + + var found bool + for _, c := range appConfig.Containers { + if c.Type == config.EmulatorAWS { + awsContainer = c + found = true + break + } + } + if !found { + return failWithCode("reset is only supported for the AWS emulator", output.ErrEmulatorNotConfigured) + } + + rt, err = runtime.NewDockerRuntime(cfg.DockerHost) + if err != nil { + return err + } + resolvedHost, _ := endpoint.ResolveHost(cmd.Context(), awsContainer.Port, cfg.LocalStackHost) + host = "http://" + resolvedHost } interactive := isInteractiveMode(cfg) @@ -62,11 +87,6 @@ To wipe the on-disk volume (certificates, persistence data, cached tools) instea return failWithCode("reset requires confirmation; use --force to skip in non-interactive mode", output.ErrConfirmationRequired) } - rt, err := runtime.NewDockerRuntime(cfg.DockerHost) - if err != nil { - return err - } - host, _ := endpoint.ResolveHost(cmd.Context(), awsContainer.Port, cfg.LocalStackHost) resetter := aws.NewClient() if interactive { diff --git a/cmd/restart.go b/cmd/restart.go index bb987185..06964ff2 100644 --- a/cmd/restart.go +++ b/cmd/restart.go @@ -22,6 +22,10 @@ func newRestartCmd(cfg *env.Env, tel *telemetry.Client, logger log.Logger) *cobr Long: "Stop and restart emulator and services.", PreRunE: initConfigDeferCreate(nil), RunE: func(cmd *cobra.Command, args []string) error { + if err := rejectEndpointURL(cmd, output.NewPlainSink(os.Stdout), "restart"); err != nil { + return err + } + rt, err := runtime.NewDockerRuntime(cfg.DockerHost) if err != nil { return err diff --git a/cmd/root.go b/cmd/root.go index 9296c80b..54430688 100644 --- a/cmd/root.go +++ b/cmd/root.go @@ -66,6 +66,12 @@ func NewRootCmd(cfg *env.Env, tel *telemetry.Client, logger log.Logger) *cobra.C if len(args) > 0 { return dispatchExtension(cmd.Context(), cfg, tel, logger, args) } + // The bare root command starts the emulator via the same + // startEmulator path as `lstk start` below, so it rejects + // --endpoint-url the same way. + if err := rejectEndpointURL(cmd, output.NewPlainSink(os.Stdout), "start"); err != nil { + return err + } rt, err := runtime.NewDockerRuntime(cfg.DockerHost) if err != nil { return err @@ -130,6 +136,7 @@ func NewRootCmd(cfg *env.Env, tel *telemetry.Client, logger log.Logger) *cobra.C root.PersistentFlags().String("config", "", "Path to config file") root.PersistentFlags().BoolVar(&cfg.NonInteractive, "non-interactive", false, "Disable interactive mode") root.PersistentFlags().BoolVar(&cfg.JSON, "json", false, "Output in JSON format (only supported by some commands)") + root.PersistentFlags().String("endpoint-url", "", "Target an existing, externally-managed emulator at this URL") root.Flags().Bool("persist", false, "Persist emulator state across restarts") addEmulatorTypeFlag(root) addSnapshotStartFlags(root) diff --git a/cmd/sam.go b/cmd/sam.go index f1652043..b93142bd 100644 --- a/cmd/sam.go +++ b/cmd/sam.go @@ -4,6 +4,7 @@ import ( "fmt" "os" + "github.com/localstack/lstk/internal/config" "github.com/localstack/lstk/internal/endpoint" "github.com/localstack/lstk/internal/env" samcli "github.com/localstack/lstk/internal/iac/sam/cli" @@ -29,7 +30,8 @@ lstk-specific flags (must appear before the sam action): --account Target AWS account id, 12 digits (default 000000000000) Supported environment variables: - AWS_ENDPOINT_URL Override the auto-resolved LocalStack endpoint + LSTK_ENDPOINT_URL Target an externally-managed emulator + AWS_ENDPOINT_URL Same as LSTK_ENDPOINT_URL (lower precedence if both are set) AWS_ENDPOINT_URL_S3 Override S3 endpoint LSTK_SAM_CMD SAM binary to invoke (default sam) AWS_REGION Fallback for --region @@ -43,6 +45,15 @@ Examples: lstk sam validate`, DisableFlagParsing: true, PreRunE: func(cmd *cobra.Command, args []string) error { + // --endpoint-url is recognized only when it precedes "sam", the + // same pre-command-only placement --json already gets here. + if strippedArgs, v, ok := stripPreCommandEndpointURL(cmd.CalledAs()); ok { + if err := cmd.Flags().Set(endpoint.FlagName, v); err != nil { + return err + } + args = strippedArgs + } + var gf globalFlags passthrough, gf = stripGlobalFlags(args) if gf.nonInteractive { @@ -76,14 +87,33 @@ Examples: return emitValidationError(sink, err) } + target, err := endpoint.Resolve(cmd.Context(), cmd) + if err != nil { + return emitValidationError(sink, err) + } + if target != nil && target.Type != config.EmulatorAWS { + return emitValidationError(sink, fmt.Errorf("lstk sam requires the AWS emulator, but the endpoint at %s is a %s emulator", target.URL, target.Type.DisplayName())) + } + awsContainer := resolveAWSContainer() // Offline subcommands never contact AWS, so they run without a - // running emulator. We still resolve the endpoint (DNS only) and - // inject it, so any incidental API call routes to LocalStack. + // running emulator. We still resolve the endpoint (DNS only, or the + // externally-managed target) and inject it, so any incidental API + // call routes to LocalStack. if samcli.IsOffline(samArgs) { - host, _ := endpoint.ResolveHost(cmd.Context(), awsContainer.Port, cfg.LocalStackHost) - return samcli.Run(cmd.Context(), "http://"+host, account, region, sink, logger, samArgs) + endpointURL := "" + if target != nil { + endpointURL = target.URL + } else { + host, _ := endpoint.ResolveHost(cmd.Context(), awsContainer.Port, cfg.LocalStackHost) + endpointURL = "http://" + host + } + return samcli.Run(cmd.Context(), endpointURL, account, region, sink, logger, samArgs) + } + + if target != nil { + return samcli.Run(cmd.Context(), target.URL, account, region, sink, logger, samArgs) } rt, err := runtime.NewDockerRuntime(cfg.DockerHost) diff --git a/cmd/snapshot.go b/cmd/snapshot.go index 7c758a95..722adeda 100644 --- a/cmd/snapshot.go +++ b/cmd/snapshot.go @@ -310,11 +310,14 @@ func runSnapshotLoad(cfg *env.Env, tel *telemetry.Client, logger log.Logger) fun if err != nil { return err } - rt, client, host, containers, appConfig, err := resolveSnapshotDeps(cmd.Context(), cfg) + rt, client, host, containers, appConfig, external, err := resolveSnapshotDeps(cmd.Context(), cmd, cfg) if err != nil { return err } - starter := buildStarter(cfg, rt, appConfig, logger, tel) + var starter snapshot.Starter + if !external { + starter = buildStarter(cfg, rt, appConfig, logger, tel) + } if isInteractiveMode(cfg) { return ui.RunSnapshotLoadRemoteS3(cmd.Context(), rt, containers, client, host, podName, src.Value, creds, cfg.AuthToken, strategy, starter) } @@ -334,12 +337,15 @@ func runSnapshotLoad(cfg *env.Env, tel *telemetry.Client, logger log.Logger) fun return execDiff(cmd, cfg, src.Value, strategy) } - rt, client, host, containers, appConfig, err := resolveSnapshotDeps(cmd.Context(), cfg) + rt, client, host, containers, appConfig, external, err := resolveSnapshotDeps(cmd.Context(), cmd, cfg) if err != nil { return err } - starter := buildStarter(cfg, rt, appConfig, logger, tel) + var starter snapshot.Starter + if !external { + starter = buildStarter(cfg, rt, appConfig, logger, tel) + } if isInteractiveMode(cfg) { return ui.RunSnapshotLoad(cmd.Context(), rt, containers, client, host, src, cfg.AuthToken, strategy, starter) @@ -391,7 +397,7 @@ func runSnapshotRemove(cfg *env.Env) func(*cobra.Command, []string) error { if !force { return fmt.Errorf("snapshot remove requires confirmation; use --force to skip in non-interactive mode") } - rt, client, host, containers, _, err := resolveSnapshotDeps(cmd.Context(), cfg) + rt, client, host, containers, _, _, err := resolveSnapshotDeps(cmd.Context(), cmd, cfg) if err != nil { return err } @@ -399,7 +405,7 @@ func runSnapshotRemove(cfg *env.Env) func(*cobra.Command, []string) error { return snapshot.Remove(cmd.Context(), rt, containers, ref.Value, cfg.AuthToken, client, host, force, sink) } - rt, client, host, containers, _, err := resolveSnapshotDeps(cmd.Context(), cfg) + rt, client, host, containers, _, _, err := resolveSnapshotDeps(cmd.Context(), cmd, cfg) if err != nil { return err } @@ -408,7 +414,7 @@ func runSnapshotRemove(cfg *env.Env) func(*cobra.Command, []string) error { } func execDiff(cmd *cobra.Command, cfg *env.Env, podName, strategy string) error { - rt, client, host, containers, _, err := resolveSnapshotDeps(cmd.Context(), cfg) + rt, client, host, containers, _, _, err := resolveSnapshotDeps(cmd.Context(), cmd, cfg) if err != nil { return err } @@ -420,19 +426,36 @@ func execDiff(cmd *cobra.Command, cfg *env.Env, podName, strategy string) error return snapshot.DiffPod(cmd.Context(), rt, containers, client, host, podName, cfg.AuthToken, strategy, sink) } -func resolveSnapshotDeps(ctx context.Context, cfg *env.Env) (rt runtime.Runtime, client *aws.Client, host string, containers []config.ContainerConfig, appConfig *config.Config, err error) { +// resolveSnapshotDeps resolves the runtime, host, and target container(s) for +// a snapshot subcommand that contacts the emulator (save/load/remove/list +// s3://...). When an endpoint URL is resolved (--endpoint-url/ +// LSTK_ENDPOINT_URL/AWS_ENDPOINT_URL), it skips Docker discovery entirely and +// returns a runtime.NewExternalRuntime stub matching a single synthetic +// container for the detected type — external reports true so callers can +// suppress the auto-start fallback that only makes sense for a Docker-managed +// emulator. +func resolveSnapshotDeps(ctx context.Context, cmd *cobra.Command, cfg *env.Env) (rt runtime.Runtime, client *aws.Client, host string, containers []config.ContainerConfig, appConfig *config.Config, external bool, err error) { appConfig, err = config.Get() if err != nil { - return nil, nil, "", nil, nil, fmt.Errorf("failed to get config: %w", err) + return nil, nil, "", nil, nil, false, fmt.Errorf("failed to get config: %w", err) + } + + target, err := endpoint.Resolve(ctx, cmd) + if err != nil { + return nil, nil, "", nil, nil, false, err + } + if target != nil { + c := config.ContainerConfig{Type: target.Type, Port: config.DefaultPort} + return runtime.NewExternalRuntime(c.Name()), aws.NewClient(), target.URL, []config.ContainerConfig{c}, appConfig, true, nil } if len(appConfig.Containers) == 0 { - return nil, nil, "", nil, nil, fmt.Errorf("no emulator is configured") + return nil, nil, "", nil, nil, false, fmt.Errorf("no emulator is configured") } rt, err = runtime.NewDockerRuntime(cfg.DockerHost) if err != nil { - return nil, nil, "", nil, nil, err + return nil, nil, "", nil, nil, false, err } // Target the first running emulator when one is up, otherwise fall back to the @@ -440,13 +463,13 @@ func resolveSnapshotDeps(ctx context.Context, cfg *env.Env) (rt runtime.Runtime, // RunningEmulators preserves config order, so "first running" is deterministic // when several emulators are configured. Running-detection errors are ignored // here so the downstream save/load flows surface them with proper messaging. - target := appConfig.Containers[0] + dockerContainer := appConfig.Containers[0] if running, rerr := container.RunningEmulators(ctx, rt, appConfig.Containers); rerr == nil && len(running) > 0 { - target = running[0] + dockerContainer = running[0] } - host, _ = endpoint.ResolveHost(ctx, target.Port, cfg.LocalStackHost) - return rt, aws.NewClient(), host, []config.ContainerConfig{target}, appConfig, nil + host, _ = endpoint.ResolveHost(ctx, dockerContainer.Port, cfg.LocalStackHost) + return rt, aws.NewClient(), "http://" + host, []config.ContainerConfig{dockerContainer}, appConfig, false, nil } // addProfileFlag registers the --profile flag used to source AWS credentials for @@ -559,7 +582,7 @@ func runSnapshotList(cfg *env.Env, logger log.Logger) func(*cobra.Command, []str if err != nil { return err } - rt, client, host, containers, _, err := resolveSnapshotDeps(cmd.Context(), cfg) + rt, client, host, containers, _, _, err := resolveSnapshotDeps(cmd.Context(), cmd, cfg) if err != nil { return err } @@ -690,7 +713,7 @@ func runSnapshotSave(cfg *env.Env) func(*cobra.Command, []string) error { if err != nil { return err } - rt, client, host, containers, _, err := resolveSnapshotDeps(cmd.Context(), cfg) + rt, client, host, containers, _, _, err := resolveSnapshotDeps(cmd.Context(), cmd, cfg) if err != nil { return err } @@ -710,7 +733,7 @@ func runSnapshotSave(cfg *env.Env) func(*cobra.Command, []string) error { return err } - rt, client, host, containers, _, err := resolveSnapshotDeps(cmd.Context(), cfg) + rt, client, host, containers, _, _, err := resolveSnapshotDeps(cmd.Context(), cmd, cfg) if err != nil { return err } diff --git a/cmd/start.go b/cmd/start.go index 5f687e0f..bd898c75 100644 --- a/cmd/start.go +++ b/cmd/start.go @@ -2,9 +2,11 @@ package cmd import ( "fmt" + "os" "github.com/localstack/lstk/internal/env" "github.com/localstack/lstk/internal/log" + "github.com/localstack/lstk/internal/output" "github.com/localstack/lstk/internal/runtime" "github.com/localstack/lstk/internal/telemetry" "github.com/spf13/cobra" @@ -30,6 +32,10 @@ If a snapshot is configured for the AWS emulator (the snapshot field in [[contai }, PreRunE: initConfigDeferCreate(&firstRun), RunE: func(c *cobra.Command, args []string) error { + if err := rejectEndpointURL(c, output.NewPlainSink(os.Stdout), "start"); err != nil { + return err + } + rt, err := runtime.NewDockerRuntime(cfg.DockerHost) if err != nil { return err diff --git a/cmd/status.go b/cmd/status.go index f100f995..b6123943 100644 --- a/cmd/status.go +++ b/cmd/status.go @@ -1,6 +1,7 @@ package cmd import ( + "context" "fmt" "os" @@ -10,6 +11,7 @@ import ( "github.com/localstack/lstk/internal/emulator/aws" "github.com/localstack/lstk/internal/emulator/azure" "github.com/localstack/lstk/internal/emulator/snowflake" + "github.com/localstack/lstk/internal/endpoint" "github.com/localstack/lstk/internal/env" "github.com/localstack/lstk/internal/output" "github.com/localstack/lstk/internal/runtime" @@ -24,6 +26,14 @@ func newStatusCmd(cfg *env.Env) *cobra.Command { Long: "Show the status of a running emulator and its deployed resources", PreRunE: initConfigDeferCreate(nil), RunE: func(cmd *cobra.Command, args []string) error { + target, err := endpoint.Resolve(cmd.Context(), cmd) + if err != nil { + return err + } + if target != nil { + return statusExternal(cmd.Context(), target) + } + rt, err := runtime.NewDockerRuntime(cfg.DockerHost) if err != nil { return err @@ -46,3 +56,67 @@ func newStatusCmd(cfg *env.Env) *cobra.Command { }, } } + +// statusExternal renders status for an externally-managed endpoint +// (--endpoint-url/LSTK_ENDPOINT_URL/AWS_ENDPOINT_URL): reachability, detected +// type, reported version, and — for an AWS-typed target — deployed resources, +// without the Docker-derived facts (container uptime, bound port) that don't +// exist for an emulator lstk didn't start. Deployed resources are not +// Docker-derived (they're an ordinary emulator API call via FetchResources, +// identical to the Docker-managed path in internal/container/status.go), so +// there's no reason to omit them here. Always renders via the plain sink, +// regardless of interactive mode — there's no lifecycle/progress to show +// interactively here, just a few lines of info. +func statusExternal(ctx context.Context, target *endpoint.Target) error { + sink := output.NewPlainSink(os.Stdout) + host := target.URL + + clients := map[config.EmulatorType]emulator.Client{ + config.EmulatorAWS: aws.NewClient(), + config.EmulatorSnowflake: snowflake.NewClient(), + config.EmulatorAzure: azure.NewClient(), + } + + var version string + var rows []emulator.Resource + if client, ok := clients[target.Type]; ok { + if v, err := client.FetchVersion(ctx, host); err != nil { + sink.Emit(output.MessageEvent{Severity: output.SeverityWarning, Text: fmt.Sprintf("Could not fetch version: %v", err)}) + } else { + version = v + } + + var fetchErr error + rows, fetchErr = client.FetchResources(ctx, host) + if fetchErr != nil { + return fetchErr + } + } + + sink.Emit(output.InstanceInfoEvent{ + EmulatorName: target.Type.DisplayName(), + Version: version, + Host: host, + }) + + if target.Type == config.EmulatorAWS { + if len(rows) == 0 { + sink.Emit(output.MessageEvent{Severity: output.SeverityNote, Text: "No resources deployed"}) + return nil + } + + tableRows := make([][]string, len(rows)) + services := map[string]struct{}{} + for i, r := range rows { + tableRows[i] = []string{r.Service, r.Name, r.Region, r.Account} + services[r.Service] = struct{}{} + } + + sink.Emit(output.ResourceSummaryEvent{Resources: len(rows), Services: len(services)}) + sink.Emit(output.TableEvent{ + Headers: []string{"Service", "Resource", "Region", "Account"}, + Rows: tableRows, + }) + } + return nil +} diff --git a/cmd/stop.go b/cmd/stop.go index 097fb585..636efb69 100644 --- a/cmd/stop.go +++ b/cmd/stop.go @@ -22,6 +22,10 @@ func newStopCmd(cfg *env.Env, tel *telemetry.Client) *cobra.Command { RunE: func(cmd *cobra.Command, args []string) error { sink := jsonAwareSink(cmd, cfg, os.Stdout) + if err := rejectEndpointURL(cmd, sink, "stop"); err != nil { + return err + } + rt, err := runtime.NewDockerRuntime(cfg.DockerHost) if err != nil { return err diff --git a/cmd/terraform.go b/cmd/terraform.go index 47bc51d7..bf94341b 100644 --- a/cmd/terraform.go +++ b/cmd/terraform.go @@ -4,6 +4,7 @@ import ( "fmt" "os" + "github.com/localstack/lstk/internal/config" "github.com/localstack/lstk/internal/endpoint" "github.com/localstack/lstk/internal/env" tfcli "github.com/localstack/lstk/internal/iac/terraform/cli" @@ -28,7 +29,8 @@ lstk-specific flags (must appear before the terraform action): --account Target AWS account id, 12 digits (default test) Supported environment variables: - AWS_ENDPOINT_URL Override the auto-resolved LocalStack endpoint + LSTK_ENDPOINT_URL Target an externally-managed emulator + AWS_ENDPOINT_URL Same as LSTK_ENDPOINT_URL (lower precedence if both are set) LSTK_TF_CMD Terraform binary to invoke (e.g. tofu; default terraform) LSTK_TF_OVERRIDE_FILE_NAME Override file name (default localstack_providers_override.tf) LSTK_TF_DRY_RUN Generate the override file but do not run terraform @@ -41,6 +43,16 @@ Examples: lstk tf apply`, DisableFlagParsing: true, PreRunE: func(cmd *cobra.Command, args []string) error { + // --endpoint-url is recognized only when it precedes + // "terraform"/"tf", the same pre-command-only placement --json + // already gets here. + if strippedArgs, v, ok := stripPreCommandEndpointURL(cmd.CalledAs()); ok { + if err := cmd.Flags().Set(endpoint.FlagName, v); err != nil { + return err + } + args = strippedArgs + } + var gf globalFlags passthrough, gf = stripGlobalFlags(args) if gf.nonInteractive { @@ -96,25 +108,39 @@ Examples: return tfcli.Run(cmd.Context(), "", region, account, chdir, sink, logger, tfArgs) } - rt, err := runtime.NewDockerRuntime(cfg.DockerHost) + target, err := endpoint.Resolve(cmd.Context(), cmd) if err != nil { - return err + return emitValidationError(sink, err) } - awsContainer := resolveAWSContainer() + var endpointURL string + if target != nil { + if target.Type != config.EmulatorAWS { + return emitValidationError(sink, fmt.Errorf("lstk terraform requires the AWS emulator, but the endpoint at %s is a %s emulator", target.URL, target.Type.DisplayName())) + } + endpointURL = target.URL + } else { + rt, err := runtime.NewDockerRuntime(cfg.DockerHost) + if err != nil { + return err + } - if err := rt.IsHealthy(cmd.Context()); err != nil { - rt.EmitUnhealthyError(sink, err) - return output.NewSilentError(fmt.Errorf("runtime not healthy: %w", err)) - } + awsContainer := resolveAWSContainer() - if err := requireRunningAWSEmulator(cmd.Context(), rt, sink, awsContainer, "terraform"); err != nil { - return err - } + if err := rt.IsHealthy(cmd.Context()); err != nil { + rt.EmitUnhealthyError(sink, err) + return output.NewSilentError(fmt.Errorf("runtime not healthy: %w", err)) + } - host, _ := endpoint.ResolveHost(cmd.Context(), awsContainer.Port, cfg.LocalStackHost) + if err := requireRunningAWSEmulator(cmd.Context(), rt, sink, awsContainer, "terraform"); err != nil { + return err + } + + host, _ := endpoint.ResolveHost(cmd.Context(), awsContainer.Port, cfg.LocalStackHost) + endpointURL = "http://" + host + } - return tfcli.Run(cmd.Context(), "http://"+host, region, account, chdir, sink, logger, tfArgs) + return tfcli.Run(cmd.Context(), endpointURL, region, account, chdir, sink, logger, tfArgs) }, } } diff --git a/cmd/volume.go b/cmd/volume.go index 95adb019..84512159 100644 --- a/cmd/volume.go +++ b/cmd/volume.go @@ -29,6 +29,10 @@ func newVolumePathCmd(cfg *env.Env) *cobra.Command { Short: "Print the volume directory path", PreRunE: initConfigDeferCreate(nil), RunE: func(cmd *cobra.Command, args []string) error { + if err := rejectEndpointURL(cmd, output.NewPlainSink(os.Stdout), "volume path"); err != nil { + return err + } + appConfig, err := config.Get() if err != nil { return fmt.Errorf("failed to get config: %w", err) @@ -59,6 +63,10 @@ func newVolumeClearCmd(cfg *env.Env) *cobra.Command { Long: "Remove all data from the emulator volume directory. This resets cached state such as certificates, downloaded tools, and persistence data.", PreRunE: initConfigDeferCreate(nil), RunE: func(cmd *cobra.Command, args []string) error { + if err := rejectEndpointURL(cmd, output.NewPlainSink(os.Stdout), "volume clear"); err != nil { + return err + } + appConfig, err := config.Get() if err != nil { return fmt.Errorf("failed to get config: %w", err) diff --git a/internal/container/status.go b/internal/container/status.go index 4190c958..1ef8551b 100644 --- a/internal/container/status.go +++ b/internal/container/status.go @@ -57,15 +57,16 @@ func Status(ctx context.Context, rt runtime.Runtime, containers []config.Contain var version string var rows []emulator.Resource if client, ok := clients[c.Type]; ok { + baseURL := "http://" + host sink.Emit(output.SpinnerStart("Fetching LocalStack status")) - if v, err := client.FetchVersion(ctx, host); err != nil { + if v, err := client.FetchVersion(ctx, baseURL); err != nil { sink.Emit(output.MessageEvent{Severity: output.SeverityWarning, Text: fmt.Sprintf("Could not fetch version: %v", err)}) } else { version = v } var fetchErr error - rows, fetchErr = client.FetchResources(ctx, host) + rows, fetchErr = client.FetchResources(ctx, baseURL) sink.Emit(output.SpinnerStop()) if fetchErr != nil { return fetchErr diff --git a/internal/emulator/aws/client.go b/internal/emulator/aws/client.go index d1132e89..2bcb9411 100644 --- a/internal/emulator/aws/client.go +++ b/internal/emulator/aws/client.go @@ -49,8 +49,8 @@ type instanceResource struct { ID string `json:"id"` } -func (c *Client) FetchVersion(ctx context.Context, host string) (string, error) { - url := fmt.Sprintf("http://%s/_localstack/health", host) +func (c *Client) FetchVersion(ctx context.Context, baseURL string) (string, error) { + url := strings.TrimRight(baseURL, "/") + "/_localstack/health" req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil) if err != nil { return "", fmt.Errorf("failed to create health request: %w", err) @@ -73,8 +73,8 @@ func (c *Client) FetchVersion(ctx context.Context, host string) (string, error) return h.Version, nil } -func (c *Client) FetchResources(ctx context.Context, host string) ([]emulator.Resource, error) { - url := fmt.Sprintf("http://%s/_localstack/resources", host) +func (c *Client) FetchResources(ctx context.Context, baseURL string) ([]emulator.Resource, error) { + url := strings.TrimRight(baseURL, "/") + "/_localstack/resources" req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil) if err != nil { return nil, fmt.Errorf("failed to create resources request: %w", err) @@ -136,8 +136,8 @@ func (c *Client) FetchResources(ctx context.Context, host string) ([]emulator.Re return rows, nil } -func (c *Client) ResetState(ctx context.Context, host string) error { - url := fmt.Sprintf("http://%s/_localstack/state/reset", host) +func (c *Client) ResetState(ctx context.Context, baseURL string) error { + url := strings.TrimRight(baseURL, "/") + "/_localstack/state/reset" req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, nil) if err != nil { return fmt.Errorf("create request: %w", err) @@ -158,8 +158,8 @@ func (c *Client) ResetState(ctx context.Context, host string) error { // ExportState streams the running instance's state into dst as a zip. services, // when non-empty, limits the export to that subset of services. It returns the // services actually captured, reported by LocalStack via a response header. -func (c *Client) ExportState(ctx context.Context, host string, services []string, dst io.Writer) ([]string, error) { - url := fmt.Sprintf("http://%s/_localstack/pods/state", host) +func (c *Client) ExportState(ctx context.Context, baseURL string, services []string, dst io.Writer) ([]string, error) { + url := strings.TrimRight(baseURL, "/") + "/_localstack/pods/state" if len(services) > 0 { // Safe to concatenate unescaped: validate.ServiceList restricts each // item to [\w-]+, which contains no query-string metacharacters. @@ -191,8 +191,8 @@ func (c *Client) ExportState(ctx context.Context, host string, services []string return extracted, nil } -func (c *Client) ImportState(ctx context.Context, host string, src io.Reader, strategy string) error { - url := fmt.Sprintf("http://%s/_localstack/pods", host) +func (c *Client) ImportState(ctx context.Context, baseURL string, src io.Reader, strategy string) error { + url := strings.TrimRight(baseURL, "/") + "/_localstack/pods" if strategy != "" { url += "?merge=" + strategy } @@ -291,12 +291,12 @@ func isPodNotFoundMsg(msg string) bool { return strings.Contains(strings.ToLower(msg), "failed to get version information from platform") } -func (c *Client) LoadPodSnapshot(ctx context.Context, host, podName, authToken, strategy string) ([]string, error) { - return c.doPodLoad(ctx, host, podName, authToken, strategy, []byte("{}")) +func (c *Client) LoadPodSnapshot(ctx context.Context, baseURL, podName, authToken, strategy string) ([]string, error) { + return c.doPodLoad(ctx, baseURL, podName, authToken, strategy, []byte("{}")) } -func (c *Client) DiffPodSnapshot(ctx context.Context, host, podName, authToken string) (snapshot.DiffResult, error) { - url := fmt.Sprintf("http://%s/_localstack/pods/%s/diff", host, podName) +func (c *Client) DiffPodSnapshot(ctx context.Context, baseURL, podName, authToken string) (snapshot.DiffResult, error) { + url := strings.TrimRight(baseURL, "/") + "/_localstack/pods/" + podName + "/diff" req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil) if err != nil { return nil, fmt.Errorf("create request: %w", err) @@ -344,16 +344,16 @@ func (c *Client) DiffPodSnapshot(ctx context.Context, host, podName, authToken s // SavePodSnapshot saves the running state to a platform-hosted pod. services, // when non-empty, limits the save to that subset of services. -func (c *Client) SavePodSnapshot(ctx context.Context, host, podName, authToken string, services []string) (snapshot.PodSaveResult, error) { +func (c *Client) SavePodSnapshot(ctx context.Context, baseURL, podName, authToken string, services []string) (snapshot.PodSaveResult, error) { body, err := marshalPodBody("", nil, services) if err != nil { return snapshot.PodSaveResult{}, fmt.Errorf("marshal request: %w", err) } - return c.doPodSave(ctx, host, podName, authToken, body) + return c.doPodSave(ctx, baseURL, podName, authToken, body) } -func (c *Client) RemovePodSnapshot(ctx context.Context, host, podName, authToken string) error { - url := fmt.Sprintf("http://%s/_localstack/pods/%s", host, podName) +func (c *Client) RemovePodSnapshot(ctx context.Context, baseURL, podName, authToken string) error { + url := strings.TrimRight(baseURL, "/") + "/_localstack/pods/" + podName req, err := http.NewRequestWithContext(ctx, http.MethodDelete, url, bytes.NewReader([]byte("{}"))) if err != nil { return fmt.Errorf("create request: %w", err) diff --git a/internal/emulator/aws/client_test.go b/internal/emulator/aws/client_test.go index 34c8e5af..3c1b9a0a 100644 --- a/internal/emulator/aws/client_test.go +++ b/internal/emulator/aws/client_test.go @@ -27,7 +27,7 @@ func TestFetchVersion(t *testing.T) { defer server.Close() c := NewClient() - version, err := c.FetchVersion(context.Background(), server.Listener.Addr().String()) + version, err := c.FetchVersion(context.Background(), server.URL) require.NoError(t, err) assert.Equal(t, "4.14.1", version) }) @@ -40,7 +40,7 @@ func TestFetchVersion(t *testing.T) { defer server.Close() c := NewClient() - _, err := c.FetchVersion(context.Background(), server.Listener.Addr().String()) + _, err := c.FetchVersion(context.Background(), server.URL) require.Error(t, err) }) } @@ -58,7 +58,7 @@ func TestFetchResources(t *testing.T) { defer server.Close() c := NewClient() - rows, err := c.FetchResources(context.Background(), server.Listener.Addr().String()) + rows, err := c.FetchResources(context.Background(), server.URL) require.NoError(t, err) require.Len(t, rows, 2) assert.Equal(t, "Lambda", rows[0].Service) @@ -78,7 +78,7 @@ func TestFetchResources(t *testing.T) { defer server.Close() c := NewClient() - rows, err := c.FetchResources(context.Background(), server.Listener.Addr().String()) + rows, err := c.FetchResources(context.Background(), server.URL) require.NoError(t, err) require.Len(t, rows, 1) assert.Equal(t, "my-topic", rows[0].Name) @@ -92,7 +92,7 @@ func TestFetchResources(t *testing.T) { defer server.Close() c := NewClient() - rows, err := c.FetchResources(context.Background(), server.Listener.Addr().String()) + rows, err := c.FetchResources(context.Background(), server.URL) require.NoError(t, err) assert.Empty(t, rows) }) @@ -105,7 +105,7 @@ func TestFetchResources(t *testing.T) { defer server.Close() c := NewClient() - _, err := c.FetchResources(context.Background(), server.Listener.Addr().String()) + _, err := c.FetchResources(context.Background(), server.URL) require.Error(t, err) }) } @@ -125,7 +125,7 @@ func TestExportState(t *testing.T) { var buf bytes.Buffer c := NewClient() - _, err := c.ExportState(context.Background(), srv.Listener.Addr().String(), nil, &buf) + _, err := c.ExportState(context.Background(), srv.URL, nil, &buf) require.NoError(t, err) assert.Equal(t, "ZIP_DATA", buf.String()) }) @@ -138,7 +138,7 @@ func TestExportState(t *testing.T) { defer srv.Close() c := NewClient() - _, err := c.ExportState(context.Background(), srv.Listener.Addr().String(), nil, io.Discard) + _, err := c.ExportState(context.Background(), srv.URL, nil, io.Discard) require.Error(t, err) assert.Contains(t, err.Error(), "500") }) @@ -151,7 +151,7 @@ func TestExportState(t *testing.T) { defer srv.Close() c := NewClient() - _, err := c.ExportState(context.Background(), srv.Listener.Addr().String(), nil, io.Discard) + _, err := c.ExportState(context.Background(), srv.URL, nil, io.Discard) require.Error(t, err) assert.Contains(t, err.Error(), "404") }) @@ -159,7 +159,7 @@ func TestExportState(t *testing.T) { t.Run("returns error on connection refused", func(t *testing.T) { t.Parallel() srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {})) - addr := srv.Listener.Addr().String() + addr := srv.URL srv.Close() c := NewClient() @@ -182,7 +182,7 @@ func TestExportState(t *testing.T) { errCh := make(chan error, 1) go func() { - _, exportErr := c.ExportState(ctx, srv.Listener.Addr().String(), nil, io.Discard) + _, exportErr := c.ExportState(ctx, srv.URL, nil, io.Discard) errCh <- exportErr }() @@ -206,7 +206,7 @@ func TestExportState(t *testing.T) { var buf bytes.Buffer c := NewClient() - _, err := c.ExportState(context.Background(), srv.Listener.Addr().String(), nil, &buf) + _, err := c.ExportState(context.Background(), srv.URL, nil, &buf) require.NoError(t, err) assert.Equal(t, size, buf.Len()) }) @@ -224,7 +224,7 @@ func TestExportState(t *testing.T) { var buf bytes.Buffer c := NewClient() - extracted, err := c.ExportState(context.Background(), srv.Listener.Addr().String(), []string{"s3", "dynamodb"}, &buf) + extracted, err := c.ExportState(context.Background(), srv.URL, []string{"s3", "dynamodb"}, &buf) require.NoError(t, err) assert.Equal(t, "services=s3,dynamodb", gotQuery) assert.Equal(t, []string{"s3", "dynamodb"}, extracted) @@ -244,7 +244,7 @@ func TestExportState(t *testing.T) { var buf bytes.Buffer c := NewClient() - extracted, err := c.ExportState(context.Background(), srv.Listener.Addr().String(), nil, &buf) + extracted, err := c.ExportState(context.Background(), srv.URL, nil, &buf) require.NoError(t, err) assert.Equal(t, "", gotQuery) assert.False(t, hadQuery) @@ -265,7 +265,7 @@ func TestResetState(t *testing.T) { defer srv.Close() c := NewClient() - err := c.ResetState(context.Background(), srv.Listener.Addr().String()) + err := c.ResetState(context.Background(), srv.URL) require.NoError(t, err) }) @@ -277,7 +277,7 @@ func TestResetState(t *testing.T) { defer srv.Close() c := NewClient() - err := c.ResetState(context.Background(), srv.Listener.Addr().String()) + err := c.ResetState(context.Background(), srv.URL) require.Error(t, err) assert.Contains(t, err.Error(), "500") }) @@ -290,7 +290,7 @@ func TestResetState(t *testing.T) { defer srv.Close() c := NewClient() - err := c.ResetState(context.Background(), srv.Listener.Addr().String()) + err := c.ResetState(context.Background(), srv.URL) require.Error(t, err) assert.Contains(t, err.Error(), "404") }) @@ -298,7 +298,7 @@ func TestResetState(t *testing.T) { t.Run("returns error on connection refused", func(t *testing.T) { t.Parallel() srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) {})) - addr := srv.Listener.Addr().String() + addr := srv.URL srv.Close() c := NewClient() @@ -321,7 +321,7 @@ func TestResetState(t *testing.T) { errCh := make(chan error, 1) go func() { - errCh <- c.ResetState(ctx, srv.Listener.Addr().String()) + errCh <- c.ResetState(ctx, srv.URL) }() <-started diff --git a/internal/emulator/aws/longline_test.go b/internal/emulator/aws/longline_test.go index 3c45c79c..365f0e35 100644 --- a/internal/emulator/aws/longline_test.go +++ b/internal/emulator/aws/longline_test.go @@ -26,7 +26,7 @@ func TestFetchResources_HandlesLongLine(t *testing.T) { defer server.Close() c := NewClient() - rows, err := c.FetchResources(context.Background(), server.Listener.Addr().String()) + rows, err := c.FetchResources(context.Background(), server.URL) require.NoError(t, err) require.Len(t, rows, 1) assert.Equal(t, hugeValue, rows[0].Name) @@ -41,7 +41,7 @@ func TestImportState_HandlesLongLine(t *testing.T) { defer server.Close() c := NewClient() - err := c.ImportState(context.Background(), server.Listener.Addr().String(), bytes.NewReader([]byte("{}")), "") + err := c.ImportState(context.Background(), server.URL, bytes.NewReader([]byte("{}")), "") require.NoError(t, err) } @@ -55,7 +55,7 @@ func TestSavePodSnapshot_HandlesLongLine(t *testing.T) { defer server.Close() c := NewClient() - res, err := c.SavePodSnapshot(context.Background(), server.Listener.Addr().String(), "my-pod", "the-token", nil) + res, err := c.SavePodSnapshot(context.Background(), server.URL, "my-pod", "the-token", nil) require.NoError(t, err) assert.Equal(t, 1, res.Version) } @@ -70,7 +70,7 @@ func TestLoadPodSnapshot_HandlesLongLine(t *testing.T) { defer server.Close() c := NewClient() - services, err := c.LoadPodSnapshot(context.Background(), server.Listener.Addr().String(), "my-pod", "the-token", "") + services, err := c.LoadPodSnapshot(context.Background(), server.URL, "my-pod", "the-token", "") require.NoError(t, err) require.Len(t, services, 1) assert.Equal(t, hugeValue, services[0]) diff --git a/internal/emulator/aws/remote.go b/internal/emulator/aws/remote.go index 6ef17b81..bb3aa66a 100644 --- a/internal/emulator/aws/remote.go +++ b/internal/emulator/aws/remote.go @@ -80,8 +80,8 @@ func (c *Client) S3BucketExists(ctx context.Context, bucket string) (bool, error // RegisterRemote upserts a named remote on the running emulator. The emulator // persists it (idempotently replacing any same-named entry) so subsequent // save/load/list calls can reference it by name. -func (c *Client) RegisterRemote(ctx context.Context, host, name, remoteURL string) error { - url := fmt.Sprintf("http://%s/_localstack/pods/remotes/%s", host, name) +func (c *Client) RegisterRemote(ctx context.Context, baseURL, name, remoteURL string) error { + url := strings.TrimRight(baseURL, "/") + "/_localstack/pods/remotes/" + name payload, err := json.Marshal(map[string]any{ "name": name, "protocols": []string{"s3"}, @@ -111,27 +111,27 @@ func (c *Client) RegisterRemote(ctx context.Context, host, name, remoteURL strin // SavePodRemote saves the running state to podName on the named remote. // services, when non-empty, limits the save to that subset of services. -func (c *Client) SavePodRemote(ctx context.Context, host, podName, remoteName string, params map[string]string, authToken string, services []string) (snapshot.PodSaveResult, error) { +func (c *Client) SavePodRemote(ctx context.Context, baseURL, podName, remoteName string, params map[string]string, authToken string, services []string) (snapshot.PodSaveResult, error) { body, err := marshalPodBody(remoteName, params, services) if err != nil { return snapshot.PodSaveResult{}, fmt.Errorf("marshal request: %w", err) } - return c.doPodSave(ctx, host, podName, authToken, body) + return c.doPodSave(ctx, baseURL, podName, authToken, body) } // LoadPodRemote loads podName from the named remote with the given merge strategy. -func (c *Client) LoadPodRemote(ctx context.Context, host, podName, remoteName string, params map[string]string, authToken, strategy string) ([]string, error) { +func (c *Client) LoadPodRemote(ctx context.Context, baseURL, podName, remoteName string, params map[string]string, authToken, strategy string) ([]string, error) { body, err := marshalPodBody(remoteName, params, nil) if err != nil { return nil, fmt.Errorf("marshal request: %w", err) } - return c.doPodLoad(ctx, host, podName, authToken, strategy, body) + return c.doPodLoad(ctx, baseURL, podName, authToken, strategy, body) } // ListPodsRemote lists the snapshots stored on the named remote via // GET /_localstack/pods (with the remote passed in the request body). -func (c *Client) ListPodsRemote(ctx context.Context, host, remoteName string, params map[string]string, authToken, creator string) ([]snapshot.RemotePod, error) { - url := fmt.Sprintf("http://%s/_localstack/pods", host) +func (c *Client) ListPodsRemote(ctx context.Context, baseURL, remoteName string, params map[string]string, authToken, creator string) ([]snapshot.RemotePod, error) { + url := strings.TrimRight(baseURL, "/") + "/_localstack/pods" if creator != "" { url += "?creator=" + creator } @@ -175,8 +175,8 @@ func (c *Client) ListPodsRemote(ctx context.Context, host, remoteName string, pa // doPodSave issues POST /_localstack/pods/{name} with the given JSON body and // parses the NDJSON response stream into a PodSaveResult. -func (c *Client) doPodSave(ctx context.Context, host, podName, authToken string, body []byte) (snapshot.PodSaveResult, error) { - url := fmt.Sprintf("http://%s/_localstack/pods/%s", host, podName) +func (c *Client) doPodSave(ctx context.Context, baseURL, podName, authToken string, body []byte) (snapshot.PodSaveResult, error) { + url := strings.TrimRight(baseURL, "/") + "/_localstack/pods/" + podName req, err := http.NewRequestWithContext(ctx, http.MethodPost, url, bytes.NewReader(body)) if err != nil { return snapshot.PodSaveResult{}, fmt.Errorf("create request: %w", err) @@ -235,8 +235,8 @@ func (c *Client) doPodSave(ctx context.Context, host, podName, authToken string, // doPodLoad issues PUT /_localstack/pods/{name}[?merge=strategy] with the given // JSON body and parses the NDJSON response stream into the list of services. -func (c *Client) doPodLoad(ctx context.Context, host, podName, authToken, strategy string, body []byte) ([]string, error) { - url := fmt.Sprintf("http://%s/_localstack/pods/%s", host, podName) +func (c *Client) doPodLoad(ctx context.Context, baseURL, podName, authToken, strategy string, body []byte) ([]string, error) { + url := strings.TrimRight(baseURL, "/") + "/_localstack/pods/" + podName if strategy != "" { url += "?merge=" + strategy } diff --git a/internal/emulator/aws/remote_test.go b/internal/emulator/aws/remote_test.go index 710c87d6..ea46de9e 100644 --- a/internal/emulator/aws/remote_test.go +++ b/internal/emulator/aws/remote_test.go @@ -26,7 +26,7 @@ func TestRegisterRemote(t *testing.T) { defer server.Close() c := NewClient() - err := c.RegisterRemote(context.Background(), server.Listener.Addr().String(), "lstk-s3-abc", "s3://bucket/?access_key_id={access_key_id}") + err := c.RegisterRemote(context.Background(), server.URL, "lstk-s3-abc", "s3://bucket/?access_key_id={access_key_id}") require.NoError(t, err) assert.Equal(t, "/_localstack/pods/remotes/lstk-s3-abc", gotPath) assert.Equal(t, []any{"s3"}, gotBody["protocols"]) @@ -46,7 +46,7 @@ func TestSavePodRemote_SendsRemoteBody(t *testing.T) { c := NewClient() params := map[string]string{"access_key_id": "AKIA", "secret_access_key": "shh"} - res, err := c.SavePodRemote(context.Background(), server.Listener.Addr().String(), "my-pod", "lstk-s3-abc", params, "", nil) + res, err := c.SavePodRemote(context.Background(), server.URL, "my-pod", "lstk-s3-abc", params, "", nil) require.NoError(t, err) assert.Equal(t, 1, res.Version) require.NotNil(t, gotBody.Remote) @@ -69,7 +69,7 @@ func TestSavePodRemote_SendsServicesFilter(t *testing.T) { c := NewClient() params := map[string]string{"access_key_id": "AKIA", "secret_access_key": "shh"} - _, err := c.SavePodRemote(context.Background(), server.Listener.Addr().String(), "my-pod", "lstk-s3-abc", params, "", []string{"s3", "dynamodb"}) + _, err := c.SavePodRemote(context.Background(), server.URL, "my-pod", "lstk-s3-abc", params, "", []string{"s3", "dynamodb"}) require.NoError(t, err) require.NotNil(t, gotBody.Remote) require.NotNil(t, gotBody.Attributes) @@ -89,7 +89,7 @@ func TestSavePodSnapshot_SendsEmptyRemote(t *testing.T) { defer server.Close() c := NewClient() - _, err := c.SavePodSnapshot(context.Background(), server.Listener.Addr().String(), "my-pod", "the-token", nil) + _, err := c.SavePodSnapshot(context.Background(), server.URL, "my-pod", "the-token", nil) require.NoError(t, err) assert.Nil(t, gotBody.Remote, "platform pod save must not include a remote payload") assert.Nil(t, gotBody.Attributes, "no services filter passed, so attributes should be omitted") @@ -107,7 +107,7 @@ func TestSavePodSnapshot_SendsServicesFilter(t *testing.T) { defer server.Close() c := NewClient() - _, err := c.SavePodSnapshot(context.Background(), server.Listener.Addr().String(), "my-pod", "the-token", []string{"s3", "lambda"}) + _, err := c.SavePodSnapshot(context.Background(), server.URL, "my-pod", "the-token", []string{"s3", "lambda"}) require.NoError(t, err) assert.Nil(t, gotBody.Remote, "platform pod save must not include a remote payload") require.NotNil(t, gotBody.Attributes) @@ -148,7 +148,7 @@ func TestListPodsRemote(t *testing.T) { defer server.Close() c := NewClient() - pods, err := c.ListPodsRemote(context.Background(), server.Listener.Addr().String(), "lstk-s3-abc", map[string]string{"access_key_id": "x"}, "", "") + pods, err := c.ListPodsRemote(context.Background(), server.URL, "lstk-s3-abc", map[string]string{"access_key_id": "x"}, "", "") require.NoError(t, err) require.Len(t, pods, 2) assert.Equal(t, "a", pods[0].Name) diff --git a/internal/emulator/azure/client.go b/internal/emulator/azure/client.go index d6024d11..cfb012f4 100644 --- a/internal/emulator/azure/client.go +++ b/internal/emulator/azure/client.go @@ -5,6 +5,7 @@ import ( "encoding/json" "fmt" "net/http" + "strings" "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp" @@ -35,8 +36,8 @@ type infoResponse struct { // FetchVersion reads the version from /_localstack/info. The Azure image's // /_localstack/health response does not carry a "version" field, so /info is // the only endpoint that surfaces it. -func (c *Client) FetchVersion(ctx context.Context, host string) (string, error) { - url := fmt.Sprintf("http://%s/_localstack/info", host) +func (c *Client) FetchVersion(ctx context.Context, baseURL string) (string, error) { + url := strings.TrimRight(baseURL, "/") + "/_localstack/info" req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil) if err != nil { return "", fmt.Errorf("failed to create info request: %w", err) diff --git a/internal/emulator/azure/client_test.go b/internal/emulator/azure/client_test.go index 3f7bb521..a54ac3e0 100644 --- a/internal/emulator/azure/client_test.go +++ b/internal/emulator/azure/client_test.go @@ -24,7 +24,7 @@ func TestFetchVersion(t *testing.T) { defer server.Close() c := NewClient() - version, err := c.FetchVersion(context.Background(), server.Listener.Addr().String()) + version, err := c.FetchVersion(context.Background(), server.URL) require.NoError(t, err) assert.Equal(t, "2026.6.0.dev112:17a29a966", version) }) @@ -37,7 +37,7 @@ func TestFetchVersion(t *testing.T) { defer server.Close() c := NewClient() - _, err := c.FetchVersion(context.Background(), server.Listener.Addr().String()) + _, err := c.FetchVersion(context.Background(), server.URL) require.Error(t, err) }) } diff --git a/internal/emulator/snowflake/client.go b/internal/emulator/snowflake/client.go index c90c8ace..bbf89bb1 100644 --- a/internal/emulator/snowflake/client.go +++ b/internal/emulator/snowflake/client.go @@ -5,6 +5,7 @@ import ( "encoding/json" "fmt" "net/http" + "strings" "go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp" @@ -32,8 +33,8 @@ type healthResponse struct { Version string `json:"version"` } -func (c *Client) FetchVersion(ctx context.Context, host string) (string, error) { - url := fmt.Sprintf("http://%s/_localstack/health", host) +func (c *Client) FetchVersion(ctx context.Context, baseURL string) (string, error) { + url := strings.TrimRight(baseURL, "/") + "/_localstack/health" req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil) if err != nil { return "", fmt.Errorf("failed to create health request: %w", err) diff --git a/internal/emulator/snowflake/client_test.go b/internal/emulator/snowflake/client_test.go index 6366b4bc..f9af4d53 100644 --- a/internal/emulator/snowflake/client_test.go +++ b/internal/emulator/snowflake/client_test.go @@ -24,7 +24,7 @@ func TestFetchVersion(t *testing.T) { defer server.Close() c := NewClient() - version, err := c.FetchVersion(context.Background(), server.Listener.Addr().String()) + version, err := c.FetchVersion(context.Background(), server.URL) require.NoError(t, err) assert.Equal(t, "2026.5.0", version) }) @@ -37,7 +37,7 @@ func TestFetchVersion(t *testing.T) { defer server.Close() c := NewClient() - _, err := c.FetchVersion(context.Background(), server.Listener.Addr().String()) + _, err := c.FetchVersion(context.Background(), server.URL) require.Error(t, err) }) } diff --git a/internal/endpoint/target.go b/internal/endpoint/target.go new file mode 100644 index 00000000..5cfa7d7b --- /dev/null +++ b/internal/endpoint/target.go @@ -0,0 +1,258 @@ +package endpoint + +import ( + "context" + "encoding/json" + "fmt" + "net/http" + "net/url" + "os" + "strings" + "time" + + "github.com/localstack/lstk/internal/config" + "github.com/spf13/cobra" +) + +// FlagName is the name of the global --endpoint-url persistent flag. +const FlagName = "endpoint-url" + +// EnvVar is the emulator-neutral environment variable that provides an +// endpoint URL when --endpoint-url is not passed. +const EnvVar = "LSTK_ENDPOINT_URL" + +// awsEnvVar is a full synonym for EnvVar, one precedence tier lower, kept for +// users already accustomed to it from AWS tooling. It applies to every +// command that consults Resolve, not just AWS-specific ones — the +// reachability/type probe below is what protects non-AWS commands from an +// unrelated ambient value, not a restriction on which commands honor it. +const awsEnvVar = "AWS_ENDPOINT_URL" + +// probeTimeout bounds how long the reachability/type probe waits for a +// response, so a hung or firewalled endpoint fails fast with an actionable +// error instead of hanging the command. +const probeTimeout = 5 * time.Second + +// Target describes an externally-managed emulator resolved from +// --endpoint-url, LSTK_ENDPOINT_URL, or AWS_ENDPOINT_URL, rather than one +// discovered via local Docker inspection. +type Target struct { + // URL is the normalized endpoint, e.g. "http://localhost:4566" (no + // trailing slash). + URL string + // Type is the emulator type determined by probing the endpoint. + Type config.EmulatorType +} + +// HostPort returns the bare host:port of the target, discarding the scheme. +// Every other consumer of a resolved Target now uses the full URL (see +// Target.URL) to preserve the caller's scheme end-to-end; the sole remaining +// use is cmd/az.go's azPreflight, which feeds this into +// azureconfig.BuildEndpoint — that helper always constructs its own +// "https://." Azure gateway address regardless of the +// original scheme, so discarding it there is correct, not a scheme leak. +func (t *Target) HostPort() string { + u, err := url.Parse(t.URL) + if err != nil { + return t.URL + } + return u.Host +} + +// UnreachableError reports that a resolved endpoint URL did not respond, or +// did not respond with a recognizable LocalStack health payload. Its message +// deliberately never mentions Docker or suggests `lstk start` — the target is +// an emulator lstk did not start. +type UnreachableError struct { + URL string + Cause error +} + +func (e *UnreachableError) Error() string { + return fmt.Sprintf("could not reach LocalStack emulator at %s: %v", e.URL, e.Cause) +} + +func (e *UnreachableError) Unwrap() error { return e.Cause } + +// IndeterminateTypeError reports that a resolved endpoint responded, but its +// health/info payload could not be classified as aws, azure, or snowflake. +// There is no override flag or config setting for the type — detection is the +// only mechanism, so this is a hard failure rather than a fallback. +type IndeterminateTypeError struct { + URL string +} + +func (e *IndeterminateTypeError) Error() string { + return fmt.Sprintf("could not determine the emulator type running at %s from its health response", e.URL) +} + +// Resolve determines whether the user gave an endpoint URL for this +// invocation, via (in precedence order) the --endpoint-url flag, +// LSTK_ENDPOINT_URL, or AWS_ENDPOINT_URL. If none are set, it returns +// (nil, nil) so the caller falls back to existing Docker-based discovery. If +// one is set, it validates the URL and probes it for reachability and +// emulator type, returning a *Target or a descriptive error. +// +// cmd may be nil (some call sites don't have a *cobra.Command handy), in +// which case only the two environment variables are consulted. +func Resolve(ctx context.Context, cmd *cobra.Command) (*Target, error) { + raw, ok := rawURL(cmd) + if !ok { + return nil, nil + } + + normalized, err := validateURL(raw) + if err != nil { + return nil, err + } + + emulatorType, err := probeType(ctx, normalized) + if err != nil { + return nil, err + } + + return &Target{URL: normalized, Type: emulatorType}, nil +} + +// rawURL applies the source precedence: --endpoint-url flag, LSTK_ENDPOINT_URL, +// AWS_ENDPOINT_URL (a full synonym for LSTK_ENDPOINT_URL, one tier lower). +func rawURL(cmd *cobra.Command) (string, bool) { + _, v, ok := ResolvedSource(cmd) + return v, ok +} + +// ResolvedSource reports which endpoint URL source (if any) is present for +// this invocation — the flag itself if passed, else "LSTK_ENDPOINT_URL" or +// "AWS_ENDPOINT_URL" if set in the environment, in the same precedence order +// as Resolve — without validating or probing it. +func ResolvedSource(cmd *cobra.Command) (source, value string, ok bool) { + if cmd != nil { + if f := cmd.Flags().Lookup(FlagName); f != nil && f.Changed { + if v, err := cmd.Flags().GetString(FlagName); err == nil && v != "" { + return "--" + FlagName, v, true + } + } + } + if v := os.Getenv(EnvVar); v != "" { + return EnvVar, v, true + } + if v := os.Getenv(awsEnvVar); v != "" { + return awsEnvVar, v, true + } + return "", "", false +} + +// validateURL requires an absolute http or https URL with a host. The +// resolved scheme is preserved end-to-end by every downstream consumer (see +// Target.URL) rather than normalized to http — https is required for +// LocalStack's cloud-hosted ephemeral instances, which are real TLS +// endpoints. Any other scheme (e.g. ftp://, ws://) is rejected here rather +// than passed through to a probe that would just fail confusingly. +func validateURL(raw string) (string, error) { + u, err := url.Parse(raw) + if err != nil || u.Host == "" || u.Scheme == "" { + return "", fmt.Errorf("invalid --endpoint-url %q: must be an absolute URL with a scheme and host (e.g. http://localhost:4566)", raw) + } + if u.Scheme != "http" && u.Scheme != "https" { + return "", fmt.Errorf("invalid --endpoint-url %q: only the http and https schemes are supported, got %q", raw, u.Scheme) + } + return strings.TrimRight(u.String(), "/"), nil +} + +// healthResponse mirrors the shape of GET /_localstack/health. AWS and +// Snowflake both populate Version; Azure's health response omits it (see +// internal/emulator/azure/client.go), which is the signal to fall back to +// /_localstack/info. +type healthResponse struct { + Version string `json:"version"` + Services map[string]string `json:"services"` +} + +// infoResponse mirrors the shape of GET /_localstack/info. +type infoResponse struct { + Version string `json:"version"` +} + +// awsSignatureServices are core AWS service keys used to recognize an AWS +// emulator's /_localstack/health "services" map. Confirmed against a real +// LocalStack AWS community image's health payload. +var awsSignatureServices = []string{"s3", "sqs", "sts", "iam", "lambda", "dynamodb", "cloudformation", "ec2", "kinesis"} + +// probeType determines the emulator type running at endpointURL by probing +// its health/info surface, and doubles as the reachability check: an +// unreachable or non-LocalStack-shaped response fails closed rather than +// silently proceeding. +// +// NOTE: the AWS-vs-Snowflake classification below (via "services" map +// contents) is a best-effort heuristic pending confirmation against a real +// LocalStack Snowflake health payload — the Snowflake product requires a +// licensed emulator to inspect, which wasn't available to verify this +// against. See design.md's Open Questions for add-endpoint-url-flag. +func probeType(ctx context.Context, endpointURL string) (config.EmulatorType, error) { + health, err := fetchJSON[healthResponse](ctx, endpointURL+"/_localstack/health") + if err != nil { + return "", &UnreachableError{URL: endpointURL, Cause: err} + } + + if health.Version != "" { + if t := classifyByServices(health.Services); t != "" { + return t, nil + } + return "", &IndeterminateTypeError{URL: endpointURL} + } + + // Azure's /_localstack/health omits "version" — fall back to /_localstack/info. + info, err := fetchJSON[infoResponse](ctx, endpointURL+"/_localstack/info") + if err != nil { + return "", &UnreachableError{URL: endpointURL, Cause: err} + } + if info.Version == "" { + return "", &IndeterminateTypeError{URL: endpointURL} + } + return config.EmulatorAzure, nil +} + +// classifyByServices inspects a health response's "services" map for a +// per-product signature, returning "" when neither is recognized. +func classifyByServices(services map[string]string) config.EmulatorType { + if _, ok := services["snowflake"]; ok { + return config.EmulatorSnowflake + } + for _, svc := range awsSignatureServices { + if _, ok := services[svc]; ok { + return config.EmulatorAWS + } + } + return "" +} + +// fetchJSON GETs path and decodes it as a JSON T. A non-2xx status or a body +// that doesn't decode as JSON is treated the same as a transport error by the +// caller (probeType) — none of these look like a genuine LocalStack instance. +func fetchJSON[T any](ctx context.Context, url string) (T, error) { + var zero T + + ctx, cancel := context.WithTimeout(ctx, probeTimeout) + defer cancel() + + req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil) + if err != nil { + return zero, err + } + + resp, err := http.DefaultClient.Do(req) + if err != nil { + return zero, err + } + defer func() { _ = resp.Body.Close() }() + + if resp.StatusCode != http.StatusOK { + return zero, fmt.Errorf("unexpected status %d from %s", resp.StatusCode, url) + } + + var out T + if err := json.NewDecoder(resp.Body).Decode(&out); err != nil { + return zero, fmt.Errorf("decoding response from %s: %w", url, err) + } + return out, nil +} diff --git a/internal/endpoint/target_test.go b/internal/endpoint/target_test.go new file mode 100644 index 00000000..a0c0d2ec --- /dev/null +++ b/internal/endpoint/target_test.go @@ -0,0 +1,305 @@ +package endpoint + +import ( + "context" + "net/http" + "net/http/httptest" + "testing" + + "github.com/localstack/lstk/internal/config" + "github.com/spf13/cobra" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func cmdWithEndpointFlag(t *testing.T) *cobra.Command { + t.Helper() + cmd := &cobra.Command{Use: "test"} + cmd.Flags().String(FlagName, "", "") + return cmd +} + +func TestResolve_Precedence(t *testing.T) { + srv := healthServer(t, `{"version":"3.0.2","services":{"s3":"available"}}`) + defer srv.Close() + + t.Run("no source set falls back to Docker discovery", func(t *testing.T) { + cmd := cmdWithEndpointFlag(t) + target, err := Resolve(context.Background(), cmd) + require.NoError(t, err) + assert.Nil(t, target) + }) + + t.Run("LSTK_ENDPOINT_URL is honored", func(t *testing.T) { + t.Setenv("LSTK_ENDPOINT_URL", srv.URL) + cmd := cmdWithEndpointFlag(t) + target, err := Resolve(context.Background(), cmd) + require.NoError(t, err) + require.NotNil(t, target) + assert.Equal(t, srv.URL, target.URL) + assert.Equal(t, config.EmulatorAWS, target.Type) + }) + + t.Run("AWS_ENDPOINT_URL is honored when LSTK_ENDPOINT_URL is not set", func(t *testing.T) { + t.Setenv("AWS_ENDPOINT_URL", srv.URL) + cmd := cmdWithEndpointFlag(t) + target, err := Resolve(context.Background(), cmd) + require.NoError(t, err) + require.NotNil(t, target) + assert.Equal(t, srv.URL, target.URL) + }) + + t.Run("LSTK_ENDPOINT_URL takes precedence over AWS_ENDPOINT_URL", func(t *testing.T) { + other := healthServer(t, `{"version":"3.0.2","services":{"s3":"available"}}`) + defer other.Close() + t.Setenv("LSTK_ENDPOINT_URL", srv.URL) + t.Setenv("AWS_ENDPOINT_URL", other.URL) + cmd := cmdWithEndpointFlag(t) + target, err := Resolve(context.Background(), cmd) + require.NoError(t, err) + require.NotNil(t, target) + assert.Equal(t, srv.URL, target.URL) + }) + + t.Run("flag takes precedence over both env vars", func(t *testing.T) { + t.Setenv("LSTK_ENDPOINT_URL", "http://should-not-be-used:4566") + t.Setenv("AWS_ENDPOINT_URL", "http://should-not-be-used-either:4566") + cmd := cmdWithEndpointFlag(t) + require.NoError(t, cmd.Flags().Set(FlagName, srv.URL)) + target, err := Resolve(context.Background(), cmd) + require.NoError(t, err) + require.NotNil(t, target) + assert.Equal(t, srv.URL, target.URL) + }) + + t.Run("AWS_ENDPOINT_URL applies uniformly, not just to AWS-specific commands", func(t *testing.T) { + // Resolve has no notion of "which command" — the caller (cmd/) decides + // what to do with the detected type. Here we just confirm the env var + // itself is consulted regardless of any such restriction. + t.Setenv("AWS_ENDPOINT_URL", srv.URL) + cmd := cmdWithEndpointFlag(t) + target, err := Resolve(context.Background(), cmd) + require.NoError(t, err) + require.NotNil(t, target) + }) +} + +func TestResolvedSource(t *testing.T) { + t.Run("no source present", func(t *testing.T) { + cmd := cmdWithEndpointFlag(t) + source, value, ok := ResolvedSource(cmd) + assert.False(t, ok) + assert.Empty(t, source) + assert.Empty(t, value) + }) + + t.Run("nil cmd with no env vars set", func(t *testing.T) { + _, _, ok := ResolvedSource(nil) + assert.False(t, ok) + }) + + t.Run("explicit flag is reported by name, not by value alone", func(t *testing.T) { + cmd := cmdWithEndpointFlag(t) + require.NoError(t, cmd.Flags().Set(FlagName, "http://localhost:4566")) + source, value, ok := ResolvedSource(cmd) + assert.True(t, ok) + assert.Equal(t, "--"+FlagName, source) + assert.Equal(t, "http://localhost:4566", value) + }) + + t.Run("LSTK_ENDPOINT_URL is reported when no flag is set", func(t *testing.T) { + t.Setenv("LSTK_ENDPOINT_URL", "http://localhost:4566") + cmd := cmdWithEndpointFlag(t) + source, value, ok := ResolvedSource(cmd) + assert.True(t, ok) + assert.Equal(t, "LSTK_ENDPOINT_URL", source) + assert.Equal(t, "http://localhost:4566", value) + }) + + t.Run("AWS_ENDPOINT_URL is reported when neither the flag nor LSTK_ENDPOINT_URL is set", func(t *testing.T) { + t.Setenv("AWS_ENDPOINT_URL", "http://localhost:4566") + cmd := cmdWithEndpointFlag(t) + source, value, ok := ResolvedSource(cmd) + assert.True(t, ok) + assert.Equal(t, "AWS_ENDPOINT_URL", source) + assert.Equal(t, "http://localhost:4566", value) + }) + + t.Run("flag takes precedence over both env vars in the reported source", func(t *testing.T) { + t.Setenv("LSTK_ENDPOINT_URL", "http://should-not-be-reported:4566") + t.Setenv("AWS_ENDPOINT_URL", "http://should-not-be-reported-either:4566") + cmd := cmdWithEndpointFlag(t) + require.NoError(t, cmd.Flags().Set(FlagName, "http://localhost:4566")) + source, _, ok := ResolvedSource(cmd) + assert.True(t, ok) + assert.Equal(t, "--"+FlagName, source) + }) + + t.Run("LSTK_ENDPOINT_URL takes precedence over AWS_ENDPOINT_URL in the reported source", func(t *testing.T) { + t.Setenv("LSTK_ENDPOINT_URL", "http://localhost:4566") + t.Setenv("AWS_ENDPOINT_URL", "http://should-not-be-reported:4566") + cmd := cmdWithEndpointFlag(t) + source, _, ok := ResolvedSource(cmd) + assert.True(t, ok) + assert.Equal(t, "LSTK_ENDPOINT_URL", source) + }) + + t.Run("no ok is reported for an unrelated flag not set", func(t *testing.T) { + // A flag that exists but wasn't Changed must not be mistaken for a + // present source, mirroring rawURL's existing .Changed guard. + cmd := cmdWithEndpointFlag(t) + source, _, ok := ResolvedSource(cmd) + assert.False(t, ok) + assert.Empty(t, source) + }) +} + +func TestValidateURL(t *testing.T) { + tests := []struct { + name string + raw string + wantErr bool + }{ + {"valid http url", "http://localhost:4566", false}, + {"valid http url with trailing slash trimmed", "http://localhost:4566/", false}, + {"valid https url", "https://my-instance.ephemeral-instances.localstack.cloud", false}, + {"valid https url with trailing slash trimmed", "https://localhost:4566/", false}, + {"missing scheme", "localhost:4566", true}, + {"missing host", "http://", true}, + {"not a url at all", "not a url", true}, + {"ftp scheme is rejected", "ftp://localhost:4566", true}, + {"ws scheme is rejected", "ws://localhost:4566", true}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + _, err := validateURL(tt.raw) + if tt.wantErr { + assert.Error(t, err) + } else { + assert.NoError(t, err) + } + }) + } +} + +func TestValidateURL_PreservesScheme(t *testing.T) { + normalized, err := validateURL("https://localhost:4566/") + require.NoError(t, err) + assert.Equal(t, "https://localhost:4566", normalized) +} + +func TestResolve_MalformedURLFailsFast(t *testing.T) { + cmd := cmdWithEndpointFlag(t) + require.NoError(t, cmd.Flags().Set(FlagName, "not-a-url")) + target, err := Resolve(context.Background(), cmd) + assert.Nil(t, target) + assert.Error(t, err) +} + +func TestProbeType(t *testing.T) { + t.Run("aws detected via services map", func(t *testing.T) { + srv := healthServer(t, `{"version":"3.0.2","services":{"s3":"available","sqs":"available"}}`) + defer srv.Close() + typ, err := probeType(context.Background(), srv.URL) + require.NoError(t, err) + assert.Equal(t, config.EmulatorAWS, typ) + }) + + t.Run("snowflake detected via services map", func(t *testing.T) { + srv := healthServer(t, `{"version":"3.0.2","services":{"snowflake":"available"}}`) + defer srv.Close() + typ, err := probeType(context.Background(), srv.URL) + require.NoError(t, err) + assert.Equal(t, config.EmulatorSnowflake, typ) + }) + + t.Run("azure detected via info fallback when health lacks version", func(t *testing.T) { + mux := http.NewServeMux() + mux.HandleFunc("/_localstack/health", func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(`{"services":{}}`)) + }) + mux.HandleFunc("/_localstack/info", func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(`{"version":"3.0.2"}`)) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + + typ, err := probeType(context.Background(), srv.URL) + require.NoError(t, err) + assert.Equal(t, config.EmulatorAzure, typ) + }) + + t.Run("indeterminate when services map has no recognizable signature", func(t *testing.T) { + srv := healthServer(t, `{"version":"3.0.2","services":{"something-unknown":"available"}}`) + defer srv.Close() + _, err := probeType(context.Background(), srv.URL) + require.Error(t, err) + var indeterminate *IndeterminateTypeError + assert.ErrorAs(t, err, &indeterminate) + }) + + t.Run("indeterminate when neither health nor info has a version", func(t *testing.T) { + mux := http.NewServeMux() + mux.HandleFunc("/_localstack/health", func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(`{"services":{}}`)) + }) + mux.HandleFunc("/_localstack/info", func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte(`{}`)) + }) + srv := httptest.NewServer(mux) + defer srv.Close() + + _, err := probeType(context.Background(), srv.URL) + require.Error(t, err) + var indeterminate *IndeterminateTypeError + assert.ErrorAs(t, err, &indeterminate) + }) + + t.Run("unreachable endpoint fails closed", func(t *testing.T) { + _, err := probeType(context.Background(), "http://127.0.0.1:1") + require.Error(t, err) + var unreachable *UnreachableError + assert.ErrorAs(t, err, &unreachable) + }) + + t.Run("non-LocalStack service (bad status code) fails closed", func(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + w.WriteHeader(http.StatusNotFound) + })) + defer srv.Close() + _, err := probeType(context.Background(), srv.URL) + require.Error(t, err) + var unreachable *UnreachableError + assert.ErrorAs(t, err, &unreachable) + }) + + t.Run("non-JSON response fails closed", func(t *testing.T) { + srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + _, _ = w.Write([]byte("not localstack")) + })) + defer srv.Close() + _, err := probeType(context.Background(), srv.URL) + require.Error(t, err) + var unreachable *UnreachableError + assert.ErrorAs(t, err, &unreachable) + }) +} + +func TestTarget_HostPort(t *testing.T) { + target := &Target{URL: "http://localhost:4566"} + assert.Equal(t, "localhost:4566", target.HostPort()) +} + +// healthServer starts an httptest.Server that always serves body at +// /_localstack/health with a 200 status. +func healthServer(t *testing.T, body string) *httptest.Server { + t.Helper() + return httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { + if r.URL.Path != "/_localstack/health" { + w.WriteHeader(http.StatusNotFound) + return + } + w.Header().Set("Content-Type", "application/json") + _, _ = w.Write([]byte(body)) + })) +} diff --git a/internal/iac/cdk/cli/env.go b/internal/iac/cdk/cli/env.go index 3ef04e31..a92c0faf 100644 --- a/internal/iac/cdk/cli/env.go +++ b/internal/iac/cdk/cli/env.go @@ -15,12 +15,6 @@ func cdkCmd() string { return "cdk" } -// endpointURLOverride returns AWS_ENDPOINT_URL, which takes precedence over the -// auto-resolved LocalStack endpoint. -func endpointURLOverride() string { - return os.Getenv("AWS_ENDPOINT_URL") -} - // s3EndpointOverride returns AWS_ENDPOINT_URL_S3, which takes precedence over // the endpoint derived from the base endpoint's host. func s3EndpointOverride() string { diff --git a/internal/iac/cdk/cli/exec.go b/internal/iac/cdk/cli/exec.go index 23bae660..5a23497c 100644 --- a/internal/iac/cdk/cli/exec.go +++ b/internal/iac/cdk/cli/exec.go @@ -53,11 +53,10 @@ func Run(ctx context.Context, endpointURL, region string, sink output.Sink, logg return output.NewSilentError(err) } + // endpointURL is already fully resolved by the command boundary + // (--endpoint-url/LSTK_ENDPOINT_URL/AWS_ENDPOINT_URL precedence, see + // internal/endpoint.Resolve), so it's used as-is here. effectiveEndpoint := endpointURL - if override := endpointURLOverride(); override != "" { - effectiveEndpoint = override - logger.Info("cdk: using AWS_ENDPOINT_URL override %s", override) - } _, s3Endpoint := endpoint.S3Addressing(effectiveEndpoint) if override := s3EndpointOverride(); override != "" { diff --git a/internal/iac/sam/cli/env.go b/internal/iac/sam/cli/env.go index d5966e27..e4f3e4ce 100644 --- a/internal/iac/sam/cli/env.go +++ b/internal/iac/sam/cli/env.go @@ -18,12 +18,6 @@ func samCmd() string { return "sam" } -// endpointURLOverride returns AWS_ENDPOINT_URL, which takes precedence over the -// auto-resolved LocalStack endpoint. -func endpointURLOverride() string { - return os.Getenv("AWS_ENDPOINT_URL") -} - // strippedKeys are ambient AWS configuration variables removed from the SAM // subprocess environment. A named profile, default profile, or stale session // token could otherwise resolve real credentials/region and silently redirect a diff --git a/internal/iac/sam/cli/exec.go b/internal/iac/sam/cli/exec.go index 053112a0..05d0c111 100644 --- a/internal/iac/sam/cli/exec.go +++ b/internal/iac/sam/cli/exec.go @@ -53,11 +53,10 @@ func Run(ctx context.Context, endpointURL, account, region string, sink output.S return output.NewSilentError(err) } + // endpointURL is already fully resolved by the command boundary + // (--endpoint-url/LSTK_ENDPOINT_URL/AWS_ENDPOINT_URL precedence, see + // internal/endpoint.Resolve), so it's used as-is here. effectiveEndpoint := endpointURL - if override := endpointURLOverride(); override != "" { - effectiveEndpoint = override - logger.Info("sam: using AWS_ENDPOINT_URL override %s", override) - } span.SetAttributes( attribute.StringSlice("sam.args", args), diff --git a/internal/iac/terraform/cli/env.go b/internal/iac/terraform/cli/env.go index 7fbb6562..7925da39 100644 --- a/internal/iac/terraform/cli/env.go +++ b/internal/iac/terraform/cli/env.go @@ -34,13 +34,6 @@ func dryRun() bool { return isTruthy(os.Getenv("LSTK_TF_DRY_RUN")) } -// endpointURLOverride returns the value of AWS_ENDPOINT_URL, which takes -// precedence over the auto-resolved LocalStack endpoint when building the -// override. -func endpointURLOverride() string { - return os.Getenv("AWS_ENDPOINT_URL") -} - func isTruthy(v string) bool { switch strings.ToLower(v) { case "", "0", "false", "no", "off": diff --git a/internal/iac/terraform/cli/exec.go b/internal/iac/terraform/cli/exec.go index 4da6da64..2369db76 100644 --- a/internal/iac/terraform/cli/exec.go +++ b/internal/iac/terraform/cli/exec.go @@ -88,11 +88,10 @@ func Run(ctx context.Context, endpointURL, region, account, chdir string, sink o return runTerraform(ctx, span, tfBin, args) } + // endpointURL is already fully resolved by the command boundary + // (--endpoint-url/LSTK_ENDPOINT_URL/AWS_ENDPOINT_URL precedence, see + // internal/endpoint.Resolve), so it's used as-is here. resolvedEndpoint := endpointURL - if override := endpointURLOverride(); override != "" { - resolvedEndpoint = override - logger.Info("terraform: using AWS_ENDPOINT_URL override %s", override) - } form := endpointForm{region: region, account: account} form.pathStyle, form.s3Endpoint = endpoint.S3Addressing(resolvedEndpoint) diff --git a/internal/runtime/external.go b/internal/runtime/external.go new file mode 100644 index 00000000..b7f4642c --- /dev/null +++ b/internal/runtime/external.go @@ -0,0 +1,111 @@ +package runtime + +import ( + "context" + "errors" + "io" + "time" + + "github.com/localstack/lstk/internal/output" +) + +// ErrExternalRuntimeUnsupported is returned by externalRuntime methods that +// have no meaning for an emulator lstk did not start and does not manage. +var ErrExternalRuntimeUnsupported = errors.New("not supported for an externally-managed emulator (--endpoint-url)") + +// externalRuntime is a Runtime standing in for an emulator resolved via +// --endpoint-url/LSTK_ENDPOINT_URL/AWS_ENDPOINT_URL rather than discovered via +// Docker. It reports itself healthy and reports the single container name it +// was constructed for as running, so callers built around container-name +// discovery (container.RunningEmulators, container.ResolveRunningContainerName) +// work unchanged against an externally-managed endpoint with no Docker +// involvement — the reachability/type probe that resolved the endpoint +// already stands in for these checks. Every other Runtime method is a Docker +// lifecycle operation with no meaning here and returns +// ErrExternalRuntimeUnsupported. +type externalRuntime struct { + containerName string +} + +// NewExternalRuntime returns a Runtime standing in for an externally-managed +// emulator, matching the container name of the config.ContainerConfig the +// caller constructs for it (so container.ResolveRunningContainerName's +// rt.IsRunning(ctx, c.Name()) check finds it "running"). See externalRuntime. +func NewExternalRuntime(containerName string) Runtime { + return &externalRuntime{containerName: containerName} +} + +func (r *externalRuntime) IsHealthy(context.Context) error { return nil } + +// EmitUnhealthyError is never called: IsHealthy never returns an error. +func (r *externalRuntime) EmitUnhealthyError(output.Sink, error) {} + +func (r *externalRuntime) IsRunning(_ context.Context, containerID string) (bool, error) { + return containerID == r.containerName, nil +} + +// InspectBrief mirrors IsRunning's matching logic: the container this stub was +// constructed for is reported as existing and running; anything else is a +// missing container ((ContainerBrief{}, nil), not an error — matching the +// documented contract), never Managed/AutoRemove since there's no lstk-owned +// Docker container behind this stub. In practice this is only reached via +// container.ResolveRunningContainerName-adjacent code, not the pre-start +// self-heal checks in container.Start, which never run against an externally- +// managed target. +func (r *externalRuntime) InspectBrief(_ context.Context, containerName string) (ContainerBrief, error) { + if containerName != r.containerName { + return ContainerBrief{}, nil + } + return ContainerBrief{Exists: true, Running: true}, nil +} + +// FindRunningByImage is never reached in practice: IsRunning(r.containerName) +// always matches first in container.ResolveRunningContainerName. +func (r *externalRuntime) FindRunningByImage(context.Context, []string, string) (*RunningContainer, error) { + return nil, nil +} + +func (r *externalRuntime) PullImage(context.Context, string, chan<- PullProgress) error { + return ErrExternalRuntimeUnsupported +} + +func (r *externalRuntime) Start(context.Context, ContainerConfig) (string, <-chan ExitResult, error) { + return "", nil, ErrExternalRuntimeUnsupported +} + +func (r *externalRuntime) Stop(context.Context, string) error { return ErrExternalRuntimeUnsupported } +func (r *externalRuntime) Remove(context.Context, string) error { return ErrExternalRuntimeUnsupported } + +func (r *externalRuntime) ContainerStartedAt(context.Context, string) (time.Time, error) { + return time.Time{}, ErrExternalRuntimeUnsupported +} + +func (r *externalRuntime) ContainerEnv(context.Context, string) ([]string, error) { + return nil, ErrExternalRuntimeUnsupported +} + +func (r *externalRuntime) Logs(context.Context, string, int) (string, error) { + return "", ErrExternalRuntimeUnsupported +} + +func (r *externalRuntime) StreamLogs(context.Context, string, io.Writer, bool, string) error { + return ErrExternalRuntimeUnsupported +} + +func (r *externalRuntime) GetImageVersion(context.Context, string) (string, error) { + return "", ErrExternalRuntimeUnsupported +} + +func (r *externalRuntime) ImageExists(context.Context, string) (bool, error) { + return false, ErrExternalRuntimeUnsupported +} + +func (r *externalRuntime) GetBoundPort(context.Context, string, string) (string, error) { + return "", ErrExternalRuntimeUnsupported +} + +func (r *externalRuntime) SocketPath() string { return "" } + +// Flavor reports FlavorUnknown: there is no real daemon connection backing an +// externally-managed emulator for callers to tailor a start hint to. +func (r *externalRuntime) Flavor() string { return FlavorUnknown } diff --git a/internal/runtime/external_test.go b/internal/runtime/external_test.go new file mode 100644 index 00000000..956ed516 --- /dev/null +++ b/internal/runtime/external_test.go @@ -0,0 +1,47 @@ +package runtime + +import ( + "context" + "testing" + + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestExternalRuntime(t *testing.T) { + rt := NewExternalRuntime("localstack-aws") + ctx := context.Background() + + t.Run("is always healthy", func(t *testing.T) { + assert.NoError(t, rt.IsHealthy(ctx)) + }) + + t.Run("reports the matching container name as running", func(t *testing.T) { + running, err := rt.IsRunning(ctx, "localstack-aws") + require.NoError(t, err) + assert.True(t, running) + }) + + t.Run("reports any other name as not running", func(t *testing.T) { + running, err := rt.IsRunning(ctx, "some-other-container") + require.NoError(t, err) + assert.False(t, running) + }) + + t.Run("lifecycle operations are unsupported", func(t *testing.T) { + _, err := rt.GetBoundPort(ctx, "localstack-aws", "4566/tcp") + assert.ErrorIs(t, err, ErrExternalRuntimeUnsupported) + + _, err = rt.ContainerStartedAt(ctx, "localstack-aws") + assert.ErrorIs(t, err, ErrExternalRuntimeUnsupported) + + _, err = rt.ContainerEnv(ctx, "localstack-aws") + assert.ErrorIs(t, err, ErrExternalRuntimeUnsupported) + + err = rt.Stop(ctx, "localstack-aws") + assert.ErrorIs(t, err, ErrExternalRuntimeUnsupported) + + _, _, err = rt.Start(ctx, ContainerConfig{}) + assert.ErrorIs(t, err, ErrExternalRuntimeUnsupported) + }) +} diff --git a/openspec/changes/add-endpoint-url-flag/.openspec.yaml b/openspec/changes/add-endpoint-url-flag/.openspec.yaml new file mode 100644 index 00000000..f205fc72 --- /dev/null +++ b/openspec/changes/add-endpoint-url-flag/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-07-29 diff --git a/openspec/changes/add-endpoint-url-flag/design.md b/openspec/changes/add-endpoint-url-flag/design.md new file mode 100644 index 00000000..4ba8c118 --- /dev/null +++ b/openspec/changes/add-endpoint-url-flag/design.md @@ -0,0 +1,120 @@ +## Context + +Every command that talks to the emulator today follows the same shape: construct a Docker runtime (`runtime.NewDockerRuntime`), check it's healthy, resolve a running container by name/image (`container.ResolveRunningContainerName`/`RunningEmulators`), then resolve a bare `host:port` via `internal/endpoint.ResolveHost` (which only ever reads the port from config and DNS-probes for the `*.localstack.cloud` virtual-host domain). Emulator *type* is never detected — it's read straight from the matching `[[containers]]` block's `type` field. None of this works when the emulator wasn't started by this lstk process: docker-compose users, LocalStack's host-network mode, and remote/CI emulators have no Docker container for this machine to inspect, and often no local config entry describing them at all. + +Separately, `terraform`/`cdk`/`sam` already read `AWS_ENDPOINT_URL` today (`internal/iac/{terraform,cdk,sam}/cli/env.go`), but only to relabel the endpoint value written into the generated provider config / subprocess environment — the Docker "is a container running" check (`requireRunningAWSEmulator`, `cmd/iac.go:32`) runs unconditionally regardless of that variable. This design folds that variable into the same resolution model as the new flag rather than leaving two different endpoint-override mechanisms with different bypass behavior side by side. + +This design adds a `--endpoint-url` escape hatch that lets a fixed set of commands bypass Docker discovery entirely and talk to an explicitly given URL, plus the supporting pieces (type detection, preflight changes) needed to make that safe and usable. + +## Goals / Non-Goals + +**Goals:** +- Let `aws`, `az`, `terraform`/`tf`, `cdk`, `sam`, `snapshot save`/`load` (incl. `lstk save`/`lstk load` aliases), `snapshot remove`, `snapshot list s3://...`, `reset`, and `status` target an arbitrary reachable emulator via `--endpoint-url`/`LSTK_ENDPOINT_URL`/`AWS_ENDPOINT_URL` (all three treated uniformly across every one of these commands), with no local Docker required. +- Support `https://` as well as `http://` for the resolved endpoint, with the scheme preserved end-to-end to whatever ultimately makes the request — needed for LocalStack's cloud-hosted ephemeral instances, which are real TLS endpoints, not just a different host on plain HTTP. +- Replace Docker-based "is it running" preflight with an HTTP reachability/type probe when an endpoint URL is in play, producing errors that are actionable without ever mentioning Docker or `lstk start`. +- Determine emulator type (aws/azure/snowflake) from the endpoint itself, since there is no config entry to read `type` from for an externally-managed emulator. +- Give commands/forms where `--endpoint-url` doesn't apply a treatment that matches the actual risk: silently ignore it for read-only, account-scoped platform queries (`snapshot show`, bare `snapshot list`) where no wrong-target side effect is possible; reject *any* resolved endpoint source — explicit flag or ambient env var, regardless of whether a local emulator happens to be running — with an actionable error for Docker-lifecycle/filesystem operations (`logs`, `stop`, `restart`, `volume`, `start`) where proceeding against the local target could silently act on the wrong (local, not remote) emulator. + +**Non-Goals:** +- No `config.toml` persistence for the endpoint URL (see Decision 2 for why, and what to do instead for repeat/CI use). +- `lstk logs`, `stop`, `restart`, `volume` do not gain `--endpoint-url` support. They are Docker container lifecycle/filesystem operations with no generic remote equivalent (there is no standard "stream logs from an arbitrary LocalStack endpoint" API); this is a scope boundary, not a follow-up. +- No credential/auth forwarding scheme for remote endpoints beyond what already exists (`LOCALSTACK_AUTH_TOKEN` for the platform API is unrelated to emulator reachability). Out of scope. +- `lstk start` never learns to "start" a remote endpoint — `--endpoint-url` is exclusively for commands that *use* an already-running emulator, never for the commands that create one. Like the other Docker-lifecycle commands, it rejects any resolved endpoint source (flag or ambient env var) rather than silently accepting and ignoring it — see Decision 5. +- No change to `az start-interception`/`stop-interception` (they mutate the user's global `~/.azure` and are orthogonal to which emulator instance is targeted). +- No `--insecure`/TLS-skip-verify escape hatch for `https://` endpoints with self-signed or otherwise untrusted certificates. A trusted (real CA-issued) certificate is assumed — the motivating case (LocalStack's cloud-hosted ephemeral instances) has one. Go's default `http.Client`/transport behavior (used unchanged by every consumer below) already does the right thing for that case with zero extra code; revisit if an untrusted-cert scenario becomes a real ask. + +## Decisions + +### 1. Flag shape and precedence: `--endpoint-url` (flag) > `LSTK_ENDPOINT_URL` (env) > `AWS_ENDPOINT_URL` (env, full synonym) > existing Docker-managed resolution + +Mirrors the AWS CLI's own `--endpoint-url` flag name/semantics deliberately, since it's the closest familiar concept for Azure and Snowflake users as well (people already gluing endpoint-based tooling to LocalStack). Value is a full URL including scheme (`http://host:4566` or `https://host`), parsed with `net/url` at the command boundary — consistent with the project convention of using existing parsers for URLs rather than a bespoke validator. Only `http`/`https` are accepted; anything else (e.g. `ftp://`, `ws://`) is rejected at validation time rather than passed through to a probe that would just fail confusingly. + +Registered as a `root.PersistentFlags()` string, same tier as `--config`. For ordinary commands Cobra's own parsing handles it. For `DisableFlagParsing: true` proxy commands (`aws`, `terraform`, `cdk`, `sam`) and `az`, it is recognized **only when it precedes the subcommand name** (`lstk --endpoint-url X aws ...`), the same pre-command-only treatment `--json` already gets via `jsonPrecedesCommandName` — and, critically, is **not** added to `stripGlobalFlags` (which strips flags from the *post*-command args). This is deliberate, not an oversight: the `aws` CLI has its own native `--endpoint-url` flag, so `lstk aws --endpoint-url X ...` (post-command placement) must forward it to the real `aws` binary untouched, exactly as `lstk aws --json` forwards to `aws`'s own `--json` output flag today rather than being intercepted by lstk. `terraform`/`cdk`/`sam` have no such collision, but the same pre-command-only rule is applied uniformly across all five commands for consistency rather than special-casing `aws` alone. + +`LSTK_ENDPOINT_URL` is the primary documented environment variable for this feature; `AWS_ENDPOINT_URL` is treated as a compatibility convenience for existing AWS users who are already accustomed to that variable. Both are consulted for **every** in-scope command, not just the AWS-capable ones — `AWS_ENDPOINT_URL` is a full synonym for `LSTK_ENDPOINT_URL`, one precedence tier lower when both are set (so `LSTK_ENDPOINT_URL` wins on conflict). The alternative — restricting `AWS_ENDPOINT_URL` to `aws`/`terraform`/`cdk`/`sam` only, on the theory that it's a commonly-set ambient variable that could misdirect `az`/`snapshot`/`reset`/`status` away from a correctly-running non-AWS emulator — was considered and rejected: the mandatory reachability/type probe (Decision 4) already fails closed on anything that doesn't look like a genuine LocalStack instance, so an ambient `AWS_ENDPOINT_URL` pointing at real AWS or an unrelated service produces a clear error rather than a silent bad redirect. The residual risk — `AWS_ENDPOINT_URL` pointing at a different, legitimately-running LocalStack instance than the one a user expected `status`/`reset` to inspect — is a "you set the variable, you get what you asked for" situation, no different from how `LOCALSTACK_HOST` already behaves. + +*Alternative considered*: scope `AWS_ENDPOINT_URL` to `aws`/`terraform`/`cdk`/`sam` only (the commands where it already has an established meaning today). Rejected per the reasoning above — the reachability probe already provides the safety net that scoping was meant to provide, so the extra rule would just be inconsistency (two different meanings of the same variable depending on command) without a matching safety benefit. + +*Alternative also considered*: leave `AWS_ENDPOINT_URL`'s existing behavior (value-only override, Docker check still required) untouched for terraform/cdk/sam, and give it no role in the new bypass model at all. Rejected — it would mean the same variable means two different things depending on which of `--endpoint-url`/`LSTK_ENDPOINT_URL` happen to also be set, which is more confusing than a single clearly-documented breaking change (see Risks). + +### 2. No `config.toml` persistence for the endpoint URL + +`[[containers]]` today exclusively means "a thing lstk manages" — every other field (`image`, `tag`, `volumes`, `port`, `snapshot`) answers "how should lstk start/run this." An `endpoint_url` field would answer the opposite question ("don't manage anything, look here instead"), and every downstream reader of `ContainerConfig` (`Name()`, `Image()`, `ContainerPort()`, `VolumeDir()`, `ApplyEmulatorType`, `EnsureCreated`) would need a variant that doesn't have most of those concepts. The mutual-exclusion validation that would be needed (reject `endpoint_url` alongside `image`/`volumes`/`port`) is itself a sign of the conceptual mismatch rather than a clean fit. + +Repeat/CI use is instead served by the environment-variable mechanism already in Decision 1: a docker-compose project can set `LSTK_ENDPOINT_URL` in its `docker-compose.yml` `environment:` block, a `.envrc` (direnv), or a shell profile — the same places such a project would already keep other environment-specific values, with no new lstk-owned schema to keep in sync. + +*Alternative considered*: a project-local `.lstk/config.toml` convenience field, sibling to nothing else (no mutual-exclusion rules needed). This avoids the `[[containers]]` conflation but was set aside for now — it doesn't clearly beat a checked-in `.envrc`/`docker-compose.yml` entry for the same "whole team gets the same default" use case, and can be revisited later if flag/env-var repetition turns out to be a real pain point in practice. + +### 3. Emulator type: auto-detect via HTTP probe + +Detection probes `GET {endpoint}/_localstack/health` first; if the response is missing a `version` field (Azure's known quirk — see `internal/emulator/azure/client.go`), fall back to `GET {endpoint}/_localstack/info`. AWS vs. Snowflake both return `version` from `/_localstack/health` today, so they must be told apart by inspecting the `services` map in that response for a known per-product service-name signature (needs confirming against real health payloads — see Open Questions). + +Because that heuristic is inherently fragile (a minimal/mocked or future emulator build could change its `services` map), the type should always be inferred from the endpoint probe. If the endpoint is unreliable, the fix belongs in the emulator rather than adding a manual override. Commands that are AWS-only (`terraform`, `cdk`, `sam`) still enforce that requirement — just via the detected type instead of container inspection — with the same "wrong emulator type" error shape used today. + +*Alternative considered*: detect purely by trying each product's known distinguishing endpoint (e.g. Azure's `/_localstack/info`, AWS's `/_localstack/resources`) and picking whichever responds. Rejected as the primary mechanism — it's more round trips for every invocation and no more reliable than reading `services`, but kept as the Azure fallback specifically since that's what Azure's own client already relies on. + +### 4. Preflight: reachability/type probe replaces all Docker checks, fails closed with no Docker-flavored suggestions + +When an endpoint URL is resolved (from any of the sources in Decision 1), the command skips `runtime.NewDockerRuntime`, `IsHealthy`, `ResolveRunningContainerName`/`RunningEmulators`, and `HandleNoRunningContainer` entirely. In their place, the same health probe used for type detection (Decision 3) doubles as the "is it up" check: unreachable/non-LocalStack responses produce a distinct error ("could not reach LocalStack emulator at `http://...`: ") that never suggests `lstk start` or mentions Docker. + +`snapshot load`'s existing "nothing running → auto-start via `container.Start`" fallback is disabled when an endpoint URL is in play — auto-starting a local Docker container would silently ignore the user's explicit remote target. It hard-fails instead if the given endpoint is unreachable. + +`az`'s current hard requirement that `*.localhost.localstack.cloud` resolve via DNS is specific to lstk's own Docker-published container and doesn't generalize to an arbitrary remote host; with an endpoint URL resolved, that check is skipped and the given host is used as-is (falling back to path-style-equivalent Azure addressing where the existing code already branches on domain shape). + +`status`'s per-emulator rendering currently reports Docker-derived facts (bound port via `GetBoundPort`, container health). With an endpoint URL, it instead reports reachability + detected type + the version reported by the emulator's own health payload — there is no container to introspect for uptime/image. Deployed resources (the AWS-only resource table, from `/_localstack/resources`) are **not** Docker-derived — they're an ordinary emulator API call, identical for both paths — so they are still reported for an AWS-typed target exactly as for a Docker-managed one. (Corrected: the initial cut of this change dropped them entirely for externally-managed targets, which wasn't the intent — nothing about the resource listing depends on Docker.) + +### 5. Two different "doesn't apply here" treatments: reject when ignoring risks a wrong-target side effect, silently ignore when it can't possibly matter + +`snapshot show` and `snapshot list` with no `s3://` argument only ever call the LocalStack platform API (`api.NewPlatformClient`) — no Docker runtime, no host resolution, nothing emulator-shaped happens in either code path today (`cmd/snapshot.go:600-623`, `538-587`). Both are account-scoped platform queries: the answer is identical no matter what's running where, so `--endpoint-url`/`LSTK_ENDPOINT_URL`/`AWS_ENDPOINT_URL` are silently ignored for these two forms, whether set via flag or environment. This costs nothing because there is no "wrong target" for a query that isn't emulator-scoped in the first place — unlike a value-only override that could quietly do the wrong thing, there's no *right* thing it could have done instead. Since `--endpoint-url` is a `root.PersistentFlags()` string, Cobra already accepts it on every subcommand without complaint, so this requires no code at all for `show`/bare `list` — they simply never look at the resolved endpoint target, which is also strictly simpler than adding a rejection check would have been. + +`logs`, `stop`, `restart`, `volume`, and `start`, by contrast, reject **any** resolved endpoint source — the explicit flag, `LSTK_ENDPOINT_URL`, or `AWS_ENDPOINT_URL` — with an actionable error, unconditionally, regardless of whether a local emulator happens to be running and reachable. These are Docker-lifecycle/filesystem operations where proceeding against the local target would silently act on the *local* Docker container or local volume directory while the environment says "I mean the remote one" — a real wrong-target risk (stopping/restarting/streaming logs from the wrong emulator, clearing the wrong volume's files, or starting a redundant local instance) rather than a harmless no-op. + +**Corrected**: the initial cut of this change only rejected an *explicit* flag and silently ignored an ambient env var, reasoning that a variable set earlier in the session "wasn't targeted at this specific invocation" — and it omitted `start` entirely, letting it silently accept and ignore the flag. Both were mistakes. The risk these commands guard against (silently touching the wrong, local target) is exactly as real whether the endpoint source is a flag on this command line or an env var set for the session: a user running with `LSTK_ENDPOINT_URL` exported for their whole shell session means it for every command that talks to an emulator, not just the ones where they remembered to also pass `--endpoint-url`. Critically, this rejection does **not** depend on whether a local emulator is actually running — `logs`/`stop`/`restart`/`start` must reject *before* ever checking local Docker state, not only when that check comes up empty; a local emulator being genuinely up and reachable is not evidence the user wanted to operate on it instead of the remote one their environment names. + +Mechanically, this simplifies the original design: instead of only gating on `cmd.Flags().Lookup(endpoint.FlagName).Changed` (`rejectExplicitEndpointURL`), the check needs the same source precedence `internal/endpoint/target.go`'s `Resolve` already uses (flag, then `LSTK_ENDPOINT_URL`, then `AWS_ENDPOINT_URL`) — but as a presence-only check, with no network probe, since these commands reject regardless of reachability. `rawURL(cmd)` already implements exactly this internally but is unexported; an equivalent needs to be reachable from `cmd/`. The error message should name which source fired (flag vs. which env var) so the failure is self-explanatory instead of surfacing as an unrelated-looking "emulator is not running." + +`snapshot list s3://bucket/prefix` is a different code path (`snapshot.ListRemoteS3`) that does resolve a host and hit the emulator's `/_localstack/pods` endpoint (per `internal/snapshot/CLAUDE.md`), so it participates in endpoint-URL resolution like `save`/`load`/`remove`. This makes `list`'s support conditional on its arguments rather than a fixed property of the subcommand — worth calling out explicitly in its help text so the distinction isn't surprising, even though (unlike the reject-vs-ignore split above) getting it "wrong" here has no bad consequence either way. + +*Alternative considered*: reject an explicit flag uniformly across all six exclusions (the original design), on the theory that a flag doing nothing should always be an error rather than sometimes silently accepted. Rejected on reflection — for `show`/bare `list` this produces a confusing inconsistency within the single `snapshot` command family (siblings `save`/`load`/`remove`/`list s3://...` accept the flag, `show`/bare `list` error on it) without buying any safety, since there's no side effect being silently misdirected. The reject/ignore split is instead grounded in a concrete risk test (would ignoring cause the wrong action to happen?) rather than a blanket rule. + +### 6. Command wiring stays centralized: one resolution helper, not per-command branching + +A single helper (e.g. `endpoint.ResolveTarget(cmd, cfg)` — exact placement/signature TBD in tasks) is called at the top of each in-scope command's `RunE`/preflight and returns either "use Docker discovery as today," "use this URL + this detected type," or "this command doesn't support an endpoint URL, and one was explicitly given — error." Because `AWS_ENDPOINT_URL` is now a uniform synonym for `LSTK_ENDPOINT_URL` (Decision 1), the helper needs no per-call-site parameter to control which sources apply — every call site gets the same three-tier precedence. This keeps the precedence/probing logic in one place rather than duplicated across `cmd/aws.go`, `cmd/az.go`, `cmd/iac.go`'s shared IaC preflight, `cmd/snapshot.go` (three different call sites: `save`/`load`, `remove`, `list`), `cmd/reset.go`, `cmd/status.go`, and the rejection path in `cmd/logs.go`/`stop.go`/`restart.go`/`volume.go`/`start.go`. + +### 7. `https://` support: preserve the resolved scheme end-to-end rather than normalizing to `http://` + +The reachability/type probe (Decision 3/4) already works unchanged with `https://` — it operates on the full, validated URL string (`endpointURL+"/_localstack/health"`), never reconstructing it from a bare host. The gap is entirely downstream, in code that predates this change and already assumed a Docker-managed, always-`http://` emulator: + +- **`cmd/aws.go`/`terraform.go`/`cdk.go`/`sam.go`** reconstruct `"http://"+target.HostPort()` when an endpoint URL is resolved, discarding whatever scheme the user actually gave. Fixed by passing the resolved `Target`'s full URL straight through instead of stripping it back down to a bare host. `cmd/az.go` needs nothing: `azureconfig.BuildEndpoint` already always produces `https://.` regardless of the input's scheme, so it was already scheme-correct (the bare-host input just happened to make the point moot). +- **The emulator API clients** (`internal/emulator/aws/client.go`, `internal/emulator/aws/remote.go`, `internal/emulator/azure/client.go`, `internal/emulator/snowflake/client.go` — 13 call sites total) each take a bare `host string` and hardcode `fmt.Sprintf("http://%s/_localstack/...", host)` internally. These back `snapshot save`/`load`/`remove`/`list s3://...`, `reset`, and `status`. They change to accept a full base URL instead of a bare host, with `"http://"+host` reconstruction moved to the (unchanged) Docker-managed call sites that still only ever produce plain HTTP. This mirrors a pattern already used elsewhere in the codebase — `internal/azureconfig.IsHealthy(ctx, endpointURL string)` already takes a full URL and does `strings.TrimRight(endpointURL, "/") + "/_localstack/health"` — so this isn't a new convention, just extending an existing better one to the older `aws`/`snowflake` clients. +- **`Target.HostPort()`**, added specifically to strip the scheme back off before handing to the two layers above, is removed once both layers accept the full URL directly — it would otherwise be actively wrong (discarding a real, load-bearing `https`). + +Both `internal/endpoint.ResolveHost` (the Docker-managed path) and its callers are deliberately left untouched: they never carried a scheme to begin with (always plain HTTP against a container lstk started), so there's no scheme to lose there, and touching that path isn't needed to make `--endpoint-url` support `https://`. + +Certificate trust is out of scope (see Non-Goals): every consumer above uses Go's default HTTP client/transport unchanged, which already does standard certificate verification — correct for the motivating case (a trusted, real CA-issued cert on an ephemeral instance) with no extra code, and a deliberate non-goal for self-signed/untrusted certs for now. + +*Alternative considered*: keep `Target.HostPort()` and thread an additional "scheme" string alongside `host` through every affected signature, rather than switching those signatures to take one full URL. Rejected — it's more parameters for the same information, and every real caller already has (or can trivially get) the full URL; there's no case where a caller has the scheme and host separately and would prefer them apart. + +*Alternative also considered*: leave the Docker-managed path's `ResolveHost` returning a bare host and give it a matching scheme-generic treatment too, unifying both paths on "always a full URL." Rejected as out of scope for this change — it would touch call sites that have nothing to do with `--endpoint-url` (every existing Docker-managed invocation of these same client methods), for a benefit (representational consistency) this change doesn't need to claim. + +## Risks / Trade-offs + +- [Breaking change: `AWS_ENDPOINT_URL` alone now skips the Docker running-check for `terraform`/`cdk`/`sam`, where today it only relabels the value] → Called out explicitly as **BREAKING** in the proposal; mitigated by a changelog/release-notes callout. The behavior change only affects users who (a) set `AWS_ENDPOINT_URL` and (b) don't set `--endpoint-url`/`LSTK_ENDPOINT_URL` — the common case of an unrelated ambient `AWS_ENDPOINT_URL` combined with a genuinely-running local container is unaffected in outcome (the probe will still find the emulator reachable), just in mechanism (HTTP probe vs. Docker inspection). +- [AWS-vs-Snowflake detection heuristic may be unreliable or need upkeep as health payloads evolve] → If that turns out to be a real problem, the fix should be made in the emulator's health/info payloads rather than adding an override flag; detection failure produces an actionable error rather than a silent wrong guess. +- [`--endpoint-url` pointed at something that isn't LocalStack at all (typo, wrong port, unrelated service)] → The reachability probe validates the `/_localstack/health` (or `/_localstack/info`) response shape, not just TCP/HTTP connectivity, so a non-LocalStack service on that port fails with a clear "doesn't look like a LocalStack emulator" error instead of a confusing downstream failure inside `aws`/`terraform`/etc. +- [`status`/other commands lose Docker-derived detail (uptime, image, container health) for endpoint-url targets] → Accepted trade-off, called out explicitly in that command's output/help rather than papered over with fake data. +- [`snapshot list`'s endpoint-url support depending on its arguments could confuse users expecting a flat "does this command support it" answer] → Mitigated by explicit help text spelling out that the flag only affects the `s3://...` form; bare `list` silently ignoring it (Decision 5) means there's no error to be surprised by either way, just help text to consult if the omission itself is confusing. +- [Silently ignoring `--endpoint-url` on `snapshot show`/bare `list` could still surprise a user who expected an error, even though nothing goes wrong] → Accepted trade-off (Decision 5): the alternative (reject) was tried and rejected as inconsistent within the `snapshot` family; help text (task 6.2) documents the split so it's discoverable without hitting it by surprise. +- [An ambient `AWS_ENDPOINT_URL`, set for unrelated AWS work, now also affects `az`/`snapshot`/`reset`/`status` — surprising given the variable's name] → Accepted trade-off (Decision 1): the reachability/type probe fails closed on anything that isn't a genuine LocalStack instance, so the worst case is redirecting to a different real LocalStack instance than intended, not a silent wrong-service failure. Mitigated further with a docs/help-text callout explaining the cross-command scope so it isn't a surprise when it fires. +- [`https://` endpoints with a self-signed or otherwise untrusted certificate fail with a generic TLS error rather than a clear "certificate not trusted, use X" message] → Accepted trade-off (Decision 7/Non-Goals): out of scope for now since the motivating case (ephemeral instances) has a trusted cert; the reachability-probe error wrapping (Decision 4) still surfaces the underlying transport error rather than swallowing it, so it's debuggable even without a dedicated message. +- [A user who has `LSTK_ENDPOINT_URL`/`AWS_ENDPOINT_URL` set ambiently for their whole session, and was relying on `lstk stop`/`restart`/`logs`/`volume` still silently operating on their local Docker-managed emulator, will now see those commands fail (Decision 5, corrected)] → Accepted trade-off: this is caught pre-ship (the initial cut hasn't been released), and the failure is exactly the wrong-target risk the reject/ignore split exists to prevent — the same session-wide variable now means "every emulator-facing command targets the remote one," consistently, rather than silently splitting behavior by command. The actionable error names the variable so the fix (unset it, or pass `--endpoint-url` explicitly only where intended) is immediately clear. + +## Migration Plan + +Mostly additive; the one non-additive piece is the `AWS_ENDPOINT_URL` behavior change on `terraform`/`cdk`/`sam` (Decision 1/Risks). No config migration is needed (there is no new config schema), so this ships in a single release with a changelog callout for the breaking piece; rollback is a normal revert. + +## Open Questions + +- Exact `/_localstack/health` response shape (specifically the `services` map contents) needs confirming against real AWS and Snowflake emulator images to pin down the AWS-vs-Snowflake detection signature before implementation — flagged for verification with the emulator team during `tasks.md` execution, not blocking the design. +- Whether the probe should also inspect additional health/info fields for stronger discrimination between AWS and Snowflake is left for implementation verification with the emulator team. +- Naming of the resolution helper and where it lives (`internal/endpoint/` vs. a new package) is left to `tasks.md`/implementation — noted in Decision 6 as a single call site, not a fixed API. diff --git a/openspec/changes/add-endpoint-url-flag/proposal.md b/openspec/changes/add-endpoint-url-flag/proposal.md new file mode 100644 index 00000000..c13c4164 --- /dev/null +++ b/openspec/changes/add-endpoint-url-flag/proposal.md @@ -0,0 +1,38 @@ +## Why + +lstk currently assumes it started and owns the emulator it talks to: nearly every command that reaches the emulator (`aws`, `az`, `terraform`/`tf`, `cdk`, `sam`, `snapshot *`, `reset`, `status`) first constructs a Docker runtime and resolves the endpoint by inspecting a locally running container with a name/image lstk itself manages. Users increasingly run the emulator outside that model — via `docker compose`, LocalStack's host-networking mode, in CI, or on a different machine entirely — and lstk has no way to target those. We need a global `--endpoint-url` escape hatch so these commands can operate against any reachable emulator, managed by lstk or not. + +## What Changes + +- Add a global `--endpoint-url ` persistent flag accepted by `aws`, `az`, `terraform`/`tf`, `cdk`, `sam`, `snapshot save`/`load` (and the `lstk save`/`lstk load` aliases), `snapshot remove`, `snapshot list` (only its `s3://...` form — see below), `reset`, and `status`. When set, the command talks to that URL directly instead of resolving a host/port from a Docker-managed container. +- Resolution precedence: `--endpoint-url` flag, then `LSTK_ENDPOINT_URL`, then `AWS_ENDPOINT_URL` as a backup, then existing Docker-based discovery — applied uniformly across every in-scope command. `AWS_ENDPOINT_URL` is a full synonym for `LSTK_ENDPOINT_URL` (one precedence tier lower when both are set), not a special case for AWS-only commands: the mandatory reachability/type probe (see below) already rejects anything that doesn't look like a genuine LocalStack instance, so an ambient `AWS_ENDPOINT_URL` pointing at real AWS or an unrelated service fails closed rather than silently misdirecting `lstk az`/`snapshot`/`reset`/`status`. Docs/help text call out the naming quirk (an "AWS"-named variable affecting non-AWS commands) so it isn't confusing when it fires. +- No `config.toml` setting is introduced. Persistence for a fixed docker-compose/CI target is left to the environment-variable mechanism above (shell profile, `.envrc`, docker-compose `environment:` block) rather than a new lstk-owned config field — see design.md for why. +- `snapshot show` and bare `snapshot list` (no `s3://` argument) never contact the emulator at all — they only call the LocalStack platform API — so `--endpoint-url` has no meaningful effect on them; an explicit flag is rejected rather than silently ignored (see the new `external-endpoint` spec's scope-boundary requirement). `snapshot list s3://bucket/prefix` and `snapshot remove` **do** proxy a request through the running emulator today and are genuinely in scope. +- `lstk logs`, `stop`, `restart`, `volume`, and `start` remain Docker-lifecycle-only and do **not** gain `--endpoint-url` support — they have no meaningful action against an emulator lstk didn't start (there is no generic HTTP log-streaming or lifecycle API to fall back to, and `start` is exclusively for commands that create an emulator, never target one). Any resolved endpoint source — the explicit flag **or** an ambient `LSTK_ENDPOINT_URL`/`AWS_ENDPOINT_URL` — is rejected, even if a local emulator happens to be running and reachable: silently proceeding against the local target while the environment signals a remote one risks stopping/restarting/streaming logs from/clearing the volume of/starting a redundant instance of the wrong emulator by accident. (Corrected from the initial cut of this change, which only rejected the explicit flag and silently ignored an ambient env var, and which didn't cover `start` at all — see Decision 5.) +- When an endpoint URL is resolved, skip every local-Docker preflight for that invocation: no `runtime.NewDockerRuntime`/`IsHealthy` check, no `container.ResolveRunningContainerName`/`RunningEmulators` lookup, and no "container not running" error path. A reachability/health check against the *given* URL replaces them. +- Add emulator-type auto-detection over HTTP: probe the endpoint's health/info surface (`/_localstack/health`, falling back to `/_localstack/info` the way Azure's own client already does) to determine whether it's an AWS, Azure, or Snowflake emulator, since there is no local config entry to read a `type` from. Commands that are AWS-only (`terraform`, `cdk`, `sam`) fail with an actionable error if the detected type isn't AWS, mirroring today's "wrong emulator running" error. There is no manual override flag: if detection is ever inconclusive, that's treated as a gap to close in the emulator's own health/info payload, not something lstk should paper over with a workaround. +- **BREAKING**: `terraform`, `cdk`, and `sam` currently require a locally Docker-discovered AWS emulator to be running even when `AWS_ENDPOINT_URL` is set — that variable only relabels the value written into the generated provider config/subprocess environment today. Going forward, when `AWS_ENDPOINT_URL` is the only endpoint source set (no `--endpoint-url`/`LSTK_ENDPOINT_URL`), it now *also* skips Docker discovery entirely, consistent with the other two sources. +- `--endpoint-url`/`LSTK_ENDPOINT_URL`/`AWS_ENDPOINT_URL` accept `https://` as well as `http://` — needed to target LocalStack's cloud-hosted **ephemeral instances**, which are real internet-facing endpoints behind TLS, not just an alternate host for plain HTTP. A trusted (real CA-issued) certificate is assumed for now; no `--insecure`/skip-verify escape hatch for self-signed certs is in scope. The resolved scheme is preserved end-to-end to every consumer (the wrapped `aws`/`terraform`/`cdk`/`sam` tools, and the emulator API clients `snapshot`/`reset`/`status` use) rather than normalized to `http://`, which is what the initial cut of this change did. A user who currently relies on "AWS_ENDPOINT_URL overrides the value, but lstk still verifies a local container is running" will see that verification skipped instead. +- `status` against an externally-managed endpoint reports deployed resources (via the emulator's `/_localstack/resources` API) for an AWS-typed target, the same as it does for a Docker-managed one. The initial cut of this change dropped resources entirely for externally-managed targets — an implementation gap, not an intended part of the "reduced detail" scope (only Docker-derived facts like uptime/image/bound port were meant to be omitted). + +## Capabilities + +### New Capabilities +- `external-endpoint`: the global `--endpoint-url` flag and `LSTK_ENDPOINT_URL` env var, HTTP-based emulator type detection, and the preflight/host-resolution behavior (plus the scope boundary for commands that don't support it) for commands that don't yet have a dedicated spec (`aws`, `az`, `snapshot`, `reset`, `status`) when targeting an externally-managed emulator. + +### Modified Capabilities +- `terraform-proxy`: the "LocalStack must be running" requirement changes — when an endpoint URL is resolved (via flag, `LSTK_ENDPOINT_URL`, or `AWS_ENDPOINT_URL`), the command skips Docker-based discovery and health checks and instead resolves against the given URL, still enforcing the AWS-only requirement via the new HTTP-based type detection instead of container inspection. The existing "Explicit endpoint override" requirement changes: `AWS_ENDPOINT_URL` alone now also bypasses the Docker running-check (**BREAKING**, see above). +- `cdk-proxy`: same requirement changes as `terraform-proxy` — Docker discovery is bypassed in favor of the given endpoint URL, AWS-only enforcement moves to HTTP-based detection, and the "Honor an explicit endpoint override" requirement's `AWS_ENDPOINT_URL` handling gains the same Docker-bypass behavior (**BREAKING**). +- `sam-proxy`: same requirement changes as `terraform-proxy` and `cdk-proxy`. + +## Impact + +- **New global flag wiring**: `cmd/root.go` (`root.PersistentFlags()`, alongside `--config`/`--non-interactive`/`--json`), plus `cmd/proxy.go`'s `globalFlags`/`stripGlobalFlags` so `DisableFlagParsing: true` commands (`aws`, `terraform`, `cdk`, `sam`) still recognize `--endpoint-url`; `az.go` needs the same treatment it already gives `--json`. +- **`internal/endpoint/`**: extend host resolution to accept a full `--endpoint-url` (scheme + host + port), bypassing `ResolveHost`'s DNS-probe/container-port logic entirely when present. +- **New detection logic**: likely a new small package or an addition to `internal/emulator/` for probing `/_localstack/health`/`/_localstack/info` and classifying the response as AWS/Azure/Snowflake without a config `type` to consult. +- **Preflight call sites**: `cmd/aws.go`, `cmd/az.go`, `cmd/iac.go` (`requireRunningAWSEmulator`, shared by terraform/cdk/sam — also needs to start reading `AWS_ENDPOINT_URL` as a Docker-bypass source, not just a value override), `cmd/snapshot.go` (`resolveSnapshotDeps`, only for `remove` and `list s3://...`; `show` and bare `list` reject the flag instead), `cmd/reset.go`, `cmd/status.go`. +- **`internal/iac/{terraform,cdk,sam}/cli/env.go`**: `endpointURLOverride()` (currently a pure value substitution) needs to also signal "skip the running-emulator check" when it's the only endpoint source in play. +- **Scheme preservation, cmd/ layer**: `cmd/aws.go`, `cmd/terraform.go`, `cmd/cdk.go`, `cmd/sam.go` currently reconstruct `"http://"+target.HostPort()` when an endpoint URL is resolved; they need to pass the resolved `Target`'s full URL through unchanged instead, so an `https://` value actually reaches the wrapped tool as `https://`. `cmd/az.go` needs no change — `azureconfig.BuildEndpoint` already always produces `https://...` regardless of the input's scheme. +- **Scheme preservation, emulator client layer**: `internal/emulator/aws/client.go` (7 call sites), `internal/emulator/aws/remote.go` (4), `internal/emulator/azure/client.go` (1), and `internal/emulator/snowflake/client.go` (1) all take a bare `host string` and hardcode `fmt.Sprintf("http://%s/...", host)` internally — needed for `snapshot save`/`load`/`remove`/`list s3://...`, `reset`, and `status` to support `https://` targets. These need to accept a full base URL instead of a bare host, with `"http://"+host` reconstruction pushed up to the (unchanged) Docker-managed call sites instead. +- **`Target.HostPort()`** (added by the initial cut of this change specifically to strip the scheme before handing off to the two layers above) becomes unnecessary once both layers pass the full URL through — removed in favor of using `Target.URL` directly. +- **Docs**: `README.md`, `CLAUDE.md` (command list), and per-command help text for every affected command; a changelog/release-notes callout for the `AWS_ENDPOINT_URL` breaking change on `terraform`/`cdk`/`sam`. diff --git a/openspec/changes/add-endpoint-url-flag/specs/cdk-proxy/spec.md b/openspec/changes/add-endpoint-url-flag/specs/cdk-proxy/spec.md new file mode 100644 index 00000000..59415ac6 --- /dev/null +++ b/openspec/changes/add-endpoint-url-flag/specs/cdk-proxy/spec.md @@ -0,0 +1,57 @@ +## MODIFIED Requirements + +### Requirement: CDK CLI proxy command + +The system SHALL provide an `lstk cdk` command that forwards all of its arguments to the real AWS CDK CLI (`cdk`) and, before invoking it, configures the subprocess environment to target the running LocalStack instance. + +#### Scenario: Pass through CDK arguments + +- **WHEN** the user runs `lstk cdk deploy MyStack --require-approval never` +- **THEN** lstk invokes the `cdk` binary with `deploy MyStack --require-approval never` intact and propagates its exit code + +#### Scenario: Inject LocalStack endpoint into the CDK environment + +- **WHEN** lstk runs a CDK command +- **THEN** the `cdk` subprocess receives `AWS_ENDPOINT_URL` set to the resolved LocalStack endpoint and `AWS_ENDPOINT_URL_S3` set to the corresponding S3 endpoint (with an `s3.` host prefix when the host is virtual-host-capable) + +#### Scenario: Honor an explicit endpoint override via AWS_ENDPOINT_URL + +- **WHEN** `AWS_ENDPOINT_URL` is already set in the environment and neither `--endpoint-url` nor `LSTK_ENDPOINT_URL` is set +- **THEN** lstk uses that value instead of the auto-resolved endpoint +- **AND** lstk does not perform Docker container discovery, instead verifying the given endpoint is reachable and is an AWS LocalStack emulator via HTTP probing + +#### Scenario: --endpoint-url and LSTK_ENDPOINT_URL take precedence over AWS_ENDPOINT_URL + +- **WHEN** `AWS_ENDPOINT_URL` is set together with the global `--endpoint-url` flag or `LSTK_ENDPOINT_URL` +- **THEN** the higher-precedence source's value is used for the endpoint, and `AWS_ENDPOINT_URL` has no effect + +### Requirement: Emulator gating for AWS-contacting commands + +The system SHALL require a running AWS emulator for CDK subcommands that contact AWS APIs and SHALL run a fixed set of offline subcommands without that requirement. + +When a global endpoint URL is resolved (via `--endpoint-url`, `LSTK_ENDPOINT_URL`, or `AWS_ENDPOINT_URL`), AWS-contacting subcommands SHALL skip Docker-based container discovery and instead verify that URL is reachable and is an AWS emulator via HTTP probing. + +#### Scenario: AWS-contacting command without a running emulator + +- **WHEN** the user runs an AWS-contacting subcommand (e.g. `lstk cdk deploy`) and the AWS emulator is not running +- **THEN** lstk emits an actionable "LocalStack is not running" error (with a command to start it) and does not invoke `cdk` + +#### Scenario: A different emulator is running + +- **WHEN** an AWS-contacting CDK command is run while a non-AWS emulator (e.g. Snowflake or Azure) is running but the AWS emulator is not +- **THEN** lstk fails with an AWS-specific error naming the running emulator rather than a misleading generic "not running" message + +#### Scenario: Offline command without a running emulator + +- **WHEN** the user runs an offline subcommand (e.g. `lstk cdk synth`, `lstk cdk ls`, `lstk cdk init`) +- **THEN** lstk runs it without requiring a running emulator + +#### Scenario: Target an externally-managed emulator via --endpoint-url + +- **WHEN** the user runs an AWS-contacting subcommand (e.g. `lstk cdk deploy --endpoint-url http://localhost:4566`) and no local Docker container is running +- **THEN** lstk skips Docker container discovery, probes the given URL to confirm it is reachable and is an AWS LocalStack emulator, and injects it as `AWS_ENDPOINT_URL`/`AWS_ENDPOINT_URL_S3` for the `cdk` subprocess + +#### Scenario: Externally-managed endpoint is not AWS + +- **WHEN** a global endpoint URL is resolved for an AWS-contacting CDK command and the endpoint is detected as a non-AWS emulator +- **THEN** lstk fails with the same AWS-specific error as the non-AWS-emulator-running scenario, naming the detected type, and does not invoke `cdk` diff --git a/openspec/changes/add-endpoint-url-flag/specs/external-endpoint/spec.md b/openspec/changes/add-endpoint-url-flag/specs/external-endpoint/spec.md new file mode 100644 index 00000000..2f3345c4 --- /dev/null +++ b/openspec/changes/add-endpoint-url-flag/specs/external-endpoint/spec.md @@ -0,0 +1,151 @@ +## ADDED Requirements + +### Requirement: Global endpoint URL flag and precedence + +The system SHALL provide a global `--endpoint-url ` persistent flag and an `LSTK_ENDPOINT_URL` environment variable that let `aws`, `az`, `terraform`/`tf`, `cdk`, `sam`, `snapshot save`/`load` (including the `lstk save`/`lstk load` aliases), `snapshot remove`, `snapshot list` (when given an `s3://...` argument), `reset`, and `status` target an emulator at an arbitrary URL instead of one discovered via local Docker inspection. The value SHALL include a scheme — `http` or `https` (e.g. `http://host:4566` or `https://host`) — and SHALL be validated as a URL at the command boundary; any other scheme is rejected. The resolved scheme SHALL be preserved unchanged all the way to whatever ultimately makes the request (the wrapped `aws`/`terraform`/`cdk`/`sam` tools, and the emulator API calls behind `snapshot`/`reset`/`status`), never normalized to `http`. Certificate trust for `https` endpoints follows standard TLS verification; there is no option to skip certificate verification. + +For `aws`, `az`, `terraform`/`tf`, `cdk`, and `sam` (which forward unrecognized arguments to a wrapped binary and disable lstk's own flag parsing for their args), the `--endpoint-url` flag SHALL be recognized only when it precedes the subcommand name (e.g. `lstk --endpoint-url http://localhost:4566 aws s3 ls`), the same placement rule already applied to `--json` for these five commands. This is required because the `aws` CLI itself has a native `--endpoint-url` flag: placed after the subcommand name (`lstk aws --endpoint-url http://localhost:4566 s3 ls`), it SHALL be forwarded to the wrapped binary unchanged rather than intercepted by lstk, so a user's own `aws --endpoint-url` usage keeps working identically under `lstk aws`. The other four (`az`, `terraform`/`tf`, `cdk`, `sam`) have no such collision but get the identical pre-command-only rule for consistency. `snapshot`, `reset`, and `status` have ordinary Cobra flag parsing and accept `--endpoint-url` in the normal position after the command name. + +#### Scenario: Flag targets an arbitrary endpoint + +- **WHEN** a user runs `lstk --endpoint-url http://localhost:4566 aws s3 ls` +- **THEN** lstk uses `http://localhost:4566` as the emulator endpoint without performing Docker container discovery + +#### Scenario: Post-command --endpoint-url on aws passes through untouched + +- **WHEN** a user runs `lstk aws --endpoint-url http://localhost:4566 s3 ls` +- **THEN** lstk does not treat `--endpoint-url` as its own flag — it forwards `--endpoint-url http://localhost:4566 s3 ls` to the `aws` binary unchanged, and resolves its own emulator endpoint via existing Docker discovery (or another endpoint source) as if no `--endpoint-url` were given + +#### Scenario: Environment variable is honored + +- **WHEN** `LSTK_ENDPOINT_URL` is set and no `--endpoint-url` flag is passed +- **THEN** lstk resolves the endpoint from the environment variable + +#### Scenario: AWS_ENDPOINT_URL is honored as a backup for every in-scope command + +- **WHEN** `AWS_ENDPOINT_URL` is set, and neither `--endpoint-url` nor `LSTK_ENDPOINT_URL` is set, and the user runs `lstk aws s3 ls`, `lstk az ...`, `lstk status`, `lstk reset`, or an in-scope `lstk snapshot` form +- **THEN** lstk resolves the endpoint from `AWS_ENDPOINT_URL` without performing Docker container discovery, in every one of those cases + +#### Scenario: AWS_ENDPOINT_URL pointing at a non-LocalStack endpoint fails closed + +- **WHEN** `AWS_ENDPOINT_URL` is set to a real AWS endpoint or an unrelated service, and neither `--endpoint-url` nor `LSTK_ENDPOINT_URL` is set, and the user runs `lstk status` +- **THEN** the reachability/type probe rejects it with a "doesn't look like a LocalStack emulator" error rather than silently proceeding against it + +#### Scenario: Flag takes precedence over environment sources + +- **WHEN** `--endpoint-url` is passed and `LSTK_ENDPOINT_URL` and/or `AWS_ENDPOINT_URL` are also set +- **THEN** the flag's value is used + +#### Scenario: LSTK_ENDPOINT_URL takes precedence over AWS_ENDPOINT_URL + +- **WHEN** both `LSTK_ENDPOINT_URL` and `AWS_ENDPOINT_URL` are set, and no `--endpoint-url` flag is passed +- **THEN** `LSTK_ENDPOINT_URL`'s value is used + +#### Scenario: Malformed endpoint URL + +- **WHEN** `--endpoint-url` (or `LSTK_ENDPOINT_URL`/`AWS_ENDPOINT_URL`, where applicable) is set to a value that is not a valid absolute URL with a scheme +- **THEN** the command fails immediately with an actionable error and does not attempt any network call + +#### Scenario: An unsupported scheme is rejected + +- **WHEN** `--endpoint-url` is set to a value with a scheme other than `http` or `https` (e.g. `ftp://host:4566`) +- **THEN** the command fails immediately with an actionable error naming the unsupported scheme and does not attempt any network call + +#### Scenario: https endpoint is accepted and used as-is + +- **WHEN** a user runs `lstk --endpoint-url https://my-instance.ephemeral-instances.localstack.cloud aws s3 ls` +- **THEN** lstk probes and targets that `https://` URL directly, with no local Docker involvement, and the `aws` binary receives `https://my-instance.ephemeral-instances.localstack.cloud` as its endpoint — not a value rewritten to `http://` + +#### Scenario: https endpoint reaches the emulator API layer unchanged + +- **WHEN** a user runs `lstk snapshot save --endpoint-url https://my-instance.ephemeral-instances.localstack.cloud my-baseline` +- **THEN** the save request is made against the given `https://` endpoint, not downgraded to `http://` + +### Requirement: Emulator type detection for externally-managed endpoints + +When an endpoint URL is resolved for a command that needs to know the emulator type (e.g. to enforce an AWS-only requirement, or to render it in `status`), the system SHALL determine the type by probing the endpoint's `/_localstack/health` (falling back to `/_localstack/info` when the health response lacks a `version` field) rather than reading a local config `type`. There SHALL be no manual override flag or config setting for the type — detection is the only mechanism. + +#### Scenario: Type detected from health probe + +- **WHEN** an endpoint URL is resolved +- **THEN** lstk determines the emulator type by inspecting the endpoint's health/info response before proceeding + +#### Scenario: Detection is inconclusive + +- **WHEN** the endpoint's health/info response cannot be classified as aws, azure, or snowflake +- **THEN** the command fails with an actionable error stating that the emulator type could not be determined from the endpoint's health response, without suggesting a flag or setting to force a type + +### Requirement: Docker preflight is skipped for externally-managed endpoints + +When an endpoint URL is resolved for `aws`, `az`, `terraform`/`tf`, `cdk`, `sam`, `snapshot save`/`load`, `snapshot remove`, `snapshot list s3://...`, `reset`, or `status`, the system SHALL NOT construct a Docker runtime, check Docker health, or look up a running container by name or image. Instead it SHALL verify the given endpoint is reachable and responds with a recognizable LocalStack health payload before proceeding. + +#### Scenario: No Docker required + +- **WHEN** a user runs `lstk status --endpoint-url http://localhost:4566` on a machine with no Docker daemon running +- **THEN** the command does not fail due to Docker being unavailable, and instead probes the given URL directly + +#### Scenario: Unreachable externally-managed endpoint + +- **WHEN** an endpoint URL is resolved and the endpoint does not respond, or responds with something that isn't a recognizable LocalStack health payload +- **THEN** the command fails with an actionable error naming the given URL and the failure cause, and does not suggest running `lstk start` or mention Docker + +### Requirement: snapshot load does not auto-start a container for externally-managed endpoints + +`lstk snapshot load` (and its `lstk load` alias) SHALL NOT fall back to auto-starting a local Docker container when an endpoint URL is resolved, even if that endpoint is currently unreachable. + +#### Scenario: Auto-start suppressed + +- **WHEN** a user runs `lstk snapshot load some.snapshot --endpoint-url http://localhost:4566` and nothing responds at that URL +- **THEN** lstk fails with the unreachable-endpoint error rather than starting a local Docker container + +### Requirement: status reports reduced detail for externally-managed endpoints + +When `lstk status` resolves an endpoint URL instead of a Docker-managed container, it SHALL report reachability, the detected emulator type and version reported by the endpoint's own health payload, and — for an AWS-typed target — deployed resources exactly as it does for a Docker-managed emulator: deployed resources are reported via the emulator's own `/_localstack/resources` API, not derived from Docker, so there is no reason to omit them. It SHALL NOT report Docker-derived facts (container uptime, image, bound port) that don't exist for an emulator lstk didn't start. + +#### Scenario: Status for an externally-managed endpoint + +- **WHEN** a user runs `lstk status --endpoint-url http://localhost:4566` against a reachable emulator +- **THEN** the output shows the endpoint, detected type, and reported version, without container uptime/image fields + +#### Scenario: Status for an externally-managed AWS endpoint reports deployed resources + +- **WHEN** a user runs `lstk status --endpoint-url http://localhost:4566` against a reachable AWS-typed emulator with deployed resources +- **THEN** the output includes the resource summary and table, the same as it would for a Docker-managed AWS emulator + +### Requirement: Docker-lifecycle and filesystem commands reject any resolved endpoint URL source + +`lstk logs`, `lstk stop`, `lstk restart`, `lstk volume`, and `lstk start` SHALL reject with an actionable error whenever an endpoint URL is resolved for the invocation — whether from an explicit `--endpoint-url` flag, `LSTK_ENDPOINT_URL`, or `AWS_ENDPOINT_URL` — explaining that the command operates on a local Docker container or local filesystem state with no remote equivalent. This SHALL apply even when a local emulator is currently running and reachable: proceeding against it while an endpoint source signals the user's actual target is elsewhere is a wrong-target risk (stopping, restarting, streaming logs from, clearing the volume of, or starting a redundant instance of the wrong emulator), not a harmless no-op — and that risk is identical whether the endpoint source is an explicit flag on this command line or an environment variable set earlier in the session. lstk SHALL NOT check whether a local emulator is running before rejecting. + +#### Scenario: Explicit flag on an excluded command is rejected + +- **WHEN** a user runs `lstk logs --endpoint-url http://localhost:4566` +- **THEN** lstk fails immediately with an actionable error stating `logs` does not support `--endpoint-url`, and does not read any container logs + +#### Scenario: start rejects an explicit endpoint URL + +- **WHEN** a user runs `lstk start --endpoint-url http://localhost:4566` +- **THEN** lstk fails immediately with an actionable error stating `start` does not support `--endpoint-url`, and does not attempt to start any emulator + +#### Scenario: Ambient environment variable is rejected too, even with no explicit flag + +- **WHEN** `LSTK_ENDPOINT_URL` (or `AWS_ENDPOINT_URL`) is set in the environment and the user runs `lstk stop` with no `--endpoint-url` flag +- **THEN** lstk fails immediately with the same actionable error as the explicit-flag case, naming the environment variable that triggered it + +#### Scenario: Rejection happens even when a local emulator is running + +- **WHEN** `LSTK_ENDPOINT_URL` is set in the environment, a local Docker-managed emulator is currently running and reachable, and the user runs `lstk restart` with no `--endpoint-url` flag +- **THEN** lstk still rejects the command with the actionable error rather than restarting the local emulator, without ever checking whether a local emulator is running + +### Requirement: Platform-only snapshot forms silently ignore endpoint URL sources + +`lstk snapshot show` and `lstk snapshot list` when given no `s3://` argument SHALL silently ignore `--endpoint-url`, `LSTK_ENDPOINT_URL`, and `AWS_ENDPOINT_URL` (whether set via flag or environment) and proceed exactly as they would without any endpoint URL resolved. Both forms only ever query the LocalStack platform API, which is account-scoped rather than emulator-scoped, so no endpoint source could change their result — unlike the Docker-lifecycle commands, ignoring these sources carries no wrong-target risk, so an explicit flag is accepted without error rather than rejected. + +#### Scenario: Explicit flag on snapshot show has no effect + +- **WHEN** a user runs `lstk snapshot show pod:my-baseline --endpoint-url http://localhost:4566` +- **THEN** lstk shows the pod's metadata exactly as it would without the flag, and does not fail or attempt to reach the given URL + +#### Scenario: Explicit flag on bare snapshot list has no effect + +- **WHEN** a user runs `lstk snapshot list --endpoint-url http://localhost:4566` with no `s3://` argument +- **THEN** lstk lists the user's cloud snapshots exactly as it would without the flag, and does not fail or attempt to reach the given URL diff --git a/openspec/changes/add-endpoint-url-flag/specs/sam-proxy/spec.md b/openspec/changes/add-endpoint-url-flag/specs/sam-proxy/spec.md new file mode 100644 index 00000000..e46cfc5b --- /dev/null +++ b/openspec/changes/add-endpoint-url-flag/specs/sam-proxy/spec.md @@ -0,0 +1,62 @@ +## MODIFIED Requirements + +### Requirement: SAM CLI proxy command + +The system SHALL provide an `lstk sam` command that forwards all of its arguments to the real AWS SAM CLI (`sam`) and, before invoking it, configures the subprocess environment to target the running LocalStack instance. + +#### Scenario: Pass through SAM arguments + +- **WHEN** the user runs `lstk sam deploy --stack-name my-stack --no-confirm-changeset` +- **THEN** lstk invokes the `sam` binary with `deploy --stack-name my-stack --no-confirm-changeset` intact and propagates its exit code + +#### Scenario: Inject LocalStack endpoint into the SAM environment + +- **WHEN** lstk runs a SAM command +- **THEN** the `sam` subprocess receives `AWS_ENDPOINT_URL` set to the resolved LocalStack endpoint, and lstk does not set `AWS_ENDPOINT_URL_S3` or any S3 path-style configuration (SAM's botocore auto-selects path-style addressing against a `localhost`/IP endpoint) + +#### Scenario: Honor an explicit endpoint override via AWS_ENDPOINT_URL + +- **WHEN** `AWS_ENDPOINT_URL` is already set in the environment and neither `--endpoint-url` nor `LSTK_ENDPOINT_URL` is set +- **THEN** lstk uses that value instead of the auto-resolved endpoint +- **AND** lstk does not perform Docker container discovery, instead verifying the given endpoint is reachable and is an AWS LocalStack emulator via HTTP probing + +#### Scenario: --endpoint-url and LSTK_ENDPOINT_URL take precedence over AWS_ENDPOINT_URL + +- **WHEN** `AWS_ENDPOINT_URL` is set together with the global `--endpoint-url` flag or `LSTK_ENDPOINT_URL` +- **THEN** the higher-precedence source's value is used for the endpoint, and `AWS_ENDPOINT_URL` has no effect + +#### Scenario: Honor an explicit S3 endpoint override + +- **WHEN** `AWS_ENDPOINT_URL_S3` is already set in the environment +- **THEN** lstk passes it through to the `sam` subprocess unchanged (it is neither set nor stripped by lstk), so a user can override S3 addressing for an exotic case + +### Requirement: Emulator gating for AWS-contacting commands + +The system SHALL require a running AWS emulator for SAM subcommands that contact AWS APIs and SHALL run a fixed set of offline subcommands without that requirement. + +When a global endpoint URL is resolved (via `--endpoint-url`, `LSTK_ENDPOINT_URL`, or `AWS_ENDPOINT_URL`), AWS-contacting subcommands SHALL skip Docker-based container discovery and instead verify that URL is reachable and is an AWS emulator via HTTP probing. + +#### Scenario: AWS-contacting command without a running emulator + +- **WHEN** the user runs an AWS-contacting subcommand (e.g. `lstk sam deploy`) and the AWS emulator is not running +- **THEN** lstk emits an actionable "LocalStack is not running" error (with a command to start it) and does not invoke `sam` + +#### Scenario: A different emulator is running + +- **WHEN** an AWS-contacting SAM command is run while a non-AWS emulator (e.g. Snowflake or Azure) is running but the AWS emulator is not +- **THEN** lstk fails with an AWS-specific error naming the running emulator rather than a misleading generic "not running" message + +#### Scenario: Offline command without a running emulator + +- **WHEN** the user runs an offline subcommand (e.g. `lstk sam init`, `lstk sam build`, `lstk sam validate`, `lstk sam local generate-event`) +- **THEN** lstk runs it without requiring a running emulator + +#### Scenario: Target an externally-managed emulator via --endpoint-url + +- **WHEN** the user runs an AWS-contacting subcommand (e.g. `lstk sam deploy --endpoint-url http://localhost:4566`) and no local Docker container is running +- **THEN** lstk skips Docker container discovery, probes the given URL to confirm it is reachable and is an AWS LocalStack emulator, and uses it for the `sam` subprocess's AWS environment + +#### Scenario: Externally-managed endpoint is not AWS + +- **WHEN** a global endpoint URL is resolved for an AWS-contacting SAM command and the endpoint is detected as a non-AWS emulator +- **THEN** lstk fails with the same AWS-specific error as the non-AWS-emulator-running scenario, naming the detected type, and does not invoke `sam` diff --git a/openspec/changes/add-endpoint-url-flag/specs/terraform-proxy/spec.md b/openspec/changes/add-endpoint-url-flag/specs/terraform-proxy/spec.md new file mode 100644 index 00000000..af559066 --- /dev/null +++ b/openspec/changes/add-endpoint-url-flag/specs/terraform-proxy/spec.md @@ -0,0 +1,53 @@ +## MODIFIED Requirements + +### Requirement: LocalStack must be running + +The command SHALL require a running LocalStack **AWS** emulator. By default it SHALL resolve the endpoint automatically using lstk's container discovery and host resolution, without requiring the user to specify a host or port. `lstk terraform` operates only against the AWS emulator; other emulator types (e.g. Snowflake, Azure) are not supported. + +When a global endpoint URL is resolved (via `--endpoint-url`, `LSTK_ENDPOINT_URL`, or `AWS_ENDPOINT_URL` — see precedence below), the command SHALL skip Docker-based container discovery entirely and instead treat that URL as the emulator endpoint, verifying it is reachable and determining its emulator type via HTTP probing instead of container inspection. The AWS-only requirement still applies in this mode. + +`AWS_ENDPOINT_URL` SHALL be honored as the lowest-precedence of the three endpoint sources: `--endpoint-url`, then `LSTK_ENDPOINT_URL`, then `AWS_ENDPOINT_URL`. When `AWS_ENDPOINT_URL` is the only one of the three set, its value is used as the endpoint **and** it now triggers the same Docker-bypass behavior as the other two sources — previously (before this capability existed) `AWS_ENDPOINT_URL` only relabeled the auto-resolved endpoint's value while the Docker running-check still applied unconditionally; this is a breaking change to that narrower behavior. + +#### Scenario: No running emulator + +- **WHEN** a user runs `lstk terraform plan` and no LocalStack AWS emulator is running +- **THEN** the command fails with an error stating LocalStack is not running and suggesting how to start it (`lstk`) +- **AND** the `terraform` binary is not invoked + +#### Scenario: A non-AWS emulator is running + +- **WHEN** a user runs `lstk terraform plan` while a non-AWS LocalStack emulator (e.g. Snowflake or Azure) is running but the AWS emulator is not +- **THEN** the command fails with an error that specifically states `lstk terraform` requires the AWS emulator and identifies the running emulator type +- **AND** the `terraform` binary is not invoked + +#### Scenario: Endpoint resolved from running emulator + +- **WHEN** a LocalStack AWS emulator is running and no global endpoint URL is resolved +- **THEN** the command resolves the endpoint via the same discovery used by `lstk aws` (container discovery plus host resolution) +- **AND** uses that endpoint as the base for all generated provider endpoints + +#### Scenario: Explicit endpoint override via AWS_ENDPOINT_URL + +- **WHEN** the `AWS_ENDPOINT_URL` environment variable is set and neither `--endpoint-url` nor `LSTK_ENDPOINT_URL` is set +- **THEN** its host and port take precedence over the auto-resolved endpoint when building the provider override +- **AND** the command does not perform Docker container discovery, instead verifying the given endpoint is reachable and is an AWS LocalStack emulator via HTTP probing + +#### Scenario: --endpoint-url and LSTK_ENDPOINT_URL take precedence over AWS_ENDPOINT_URL + +- **WHEN** `AWS_ENDPOINT_URL` is set together with `--endpoint-url` or `LSTK_ENDPOINT_URL` +- **THEN** the higher-precedence source's value is used for the endpoint, and `AWS_ENDPOINT_URL` has no effect + +#### Scenario: Target an externally-managed emulator via --endpoint-url or LSTK_ENDPOINT_URL + +- **WHEN** a user runs `lstk terraform plan --endpoint-url http://localhost:4566` (or sets `LSTK_ENDPOINT_URL`) and no local Docker container is running +- **THEN** the command does not perform Docker container discovery, probes the given URL to confirm it is reachable and is an AWS LocalStack emulator, and uses it as the base for all generated provider endpoints + +#### Scenario: Externally-managed endpoint is not AWS + +- **WHEN** a global endpoint URL is resolved and the endpoint is detected as a non-AWS emulator +- **THEN** the command fails with the same AWS-specific error as the non-AWS-emulator-running scenario, naming the detected type, and does not invoke `terraform` + +#### Scenario: Externally-managed endpoint is unreachable + +- **WHEN** a global endpoint URL is resolved and the endpoint does not respond, or its response doesn't look like a LocalStack health payload +- **THEN** the command fails with an actionable error naming the given URL and the cause, without suggesting `lstk start` or mentioning Docker diff --git a/openspec/changes/add-endpoint-url-flag/tasks.md b/openspec/changes/add-endpoint-url-flag/tasks.md new file mode 100644 index 00000000..7d809b97 --- /dev/null +++ b/openspec/changes/add-endpoint-url-flag/tasks.md @@ -0,0 +1,81 @@ +## 1. Flag plumbing + +- [x] 1.1 Add `--endpoint-url` as a root `PersistentFlags()` string in `cmd/root.go` (alongside `--config`). +- [x] 1.2 Add an `--endpoint-url`-recognizing helper to `cmd/proxy.go` mirroring `jsonPrecedesCommandName` (pre-command-only, NOT added to `stripGlobalFlags`): the `aws` CLI has its own native `--endpoint-url` flag, so `lstk aws --endpoint-url X ...` (post-command) must pass through to the wrapped binary untouched, exactly like `--json` today. `--endpoint-url` is only recognized as lstk's own flag when it precedes the subcommand name (`lstk --endpoint-url X aws ...`). Wire this into `aws`/`terraform`/`cdk`/`sam`'s `PreRunE`. +- [x] 1.3 Give `cmd/az.go` the same pre-command-only `--endpoint-url` recognition it already has for `--json`. + +## 2. Endpoint resolution and type detection + +- [x] 2.1 Add a resolution helper (e.g. `endpoint.ResolveTarget(cmd, cfg)`) that applies the precedence flag > `LSTK_ENDPOINT_URL` > `AWS_ENDPOINT_URL` > Docker discovery uniformly for every in-scope command (`AWS_ENDPOINT_URL` is a full synonym for `LSTK_ENDPOINT_URL`, not restricted to AWS-specific commands), returning either a resolved external URL or a signal to fall back to existing Docker-based resolution. +- [x] 2.2 Validate the resolved URL (via `net/url`) has a scheme and host; fail fast with an actionable error otherwise. +- [x] 2.3 Implement an HTTP health/type probe: `GET {endpoint}/_localstack/health`, falling back to `GET {endpoint}/_localstack/info` when the health response lacks `version` (Azure's quirk); classify the response as `aws`/`azure`/`snowflake` or return "inconclusive". There is no override flag or config setting for the type — an inconclusive result is a hard failure (task 2.5), not a fallback to a manual value. +- [x] 2.4 Confirm the AWS-vs-Snowflake `services`-map signature against real LocalStack AWS and Snowflake health payloads (see design.md Open Questions); adjust the classification logic accordingly. If the signature proves unreliable, the fix belongs in the emulator's health/info payload, not in an lstk-side override. **Partially verified**: confirmed the AWS community image's real `/_localstack/health` payload (`services` map + AWS service names) against a live container; the Snowflake image requires a paid license to run and could not be verified the same way in this environment — the classification heuristic for Snowflake specifically remains best-effort pending real-payload confirmation (see design.md Open Questions). +- [x] 2.5 Produce a distinct "unreachable/not a LocalStack emulator" error type (no Docker/`lstk start` language) for probe failures, and a distinct "couldn't determine emulator type" error for inconclusive detection (no flag suggested); both reused by all call sites in section 3. +- [x] 2.6 Add a helper for the rejection path used by `logs`/`stop`/`restart`/`volume` (section 3.8): an explicit `--endpoint-url` flag on the command line errors with an actionable "not supported here" message; an ambient `LSTK_ENDPOINT_URL`/`AWS_ENDPOINT_URL` is silently ignored. `snapshot show`/bare `list` need no equivalent helper — they simply never consult the resolved endpoint target, so an explicit flag is silently a no-op with zero code required (Cobra already accepts an unused persistent flag). + +## 3. Command preflight changes + +- [x] 3.1 `cmd/aws.go`: branch on resolved endpoint target — skip `runtime.NewDockerRuntime`/`IsHealthy`/`container.ResolveRunningContainerName` and use the resolved URL directly when present. +- [x] 3.2 `cmd/az.go` (`azPreflight`): same branch as 3.1; skip the `*.localhost.localstack.cloud` DNS-resolution hard requirement when an endpoint URL is given. +- [x] 3.3 `cmd/iac.go` (`requireRunningAWSEmulator`, shared by terraform/cdk/sam): branch to the endpoint-URL path, replacing container discovery with the health/type probe and keeping the existing AWS-only error shape for non-AWS detections. +- [x] 3.4 `internal/iac/{terraform,cdk,sam}/cli/env.go`: change `endpointURLOverride()` (currently pure value substitution reading `AWS_ENDPOINT_URL`) to delegate to the shared resolution helper (task 2.1) so that when `AWS_ENDPOINT_URL` is the only endpoint source resolved (no `--endpoint-url`/`LSTK_ENDPOINT_URL`), it also signals "skip the running-emulator check" to the command's preflight (task 3.3), consistent with every other in-scope command. Update/add tests confirming a locally-running-but-undiscovered-by-Docker scenario now succeeds and the previous "value-only, still gated" behavior is gone (call out as the BREAKING change in the PR description). Implemented by removing `endpointURLOverride()` entirely (dead code once `cmd/` resolves `AWS_ENDPOINT_URL` upstream via `endpoint.Resolve`) rather than modifying it in place — resolution now fully happens at the command boundary before `tfcli.Run`/`cdkcli.Run`/`samcli.Run` are called, so there's a single source of truth for precedence instead of two. +- [x] 3.5 `cmd/snapshot.go`: + - `save`/`load` (`resolveSnapshotDeps` callers): branch to the endpoint-URL path; disable the "auto-start via `container.Start`" fallback in `load` when an endpoint URL is resolved, failing with the unreachable-endpoint error instead. + - `remove` (`runSnapshotRemove`): same branch as save/load. + - `list`: branch only inside the `s3://` argument path (`snapshot.ListRemoteS3` call); when no `s3://` argument is given, leave the handler as-is — it already never reads the endpoint-url flag/env vars, so no change is needed there. + - `show` (`runSnapshotShow`): leave as-is — it never calls `resolveSnapshotDeps` and needs no change; `--endpoint-url` is silently a no-op. +- [x] 3.6 `cmd/reset.go`: branch to the endpoint-URL path for host resolution. +- [x] 3.7 `cmd/status.go`: branch to the endpoint-URL path; render reachability + detected type + reported version instead of Docker-derived fields (uptime, image, bound port) when targeting an external endpoint. +- [x] 3.8 `cmd/logs.go`, `cmd/stop.go`, `cmd/restart.go`, `cmd/volume.go`: apply the rejection helper (task 2.6) unconditionally — these are the only four commands that reject an explicit `--endpoint-url` outright. + +## 4. Host/endpoint formatting reuse + +- [x] 4.1 Ensure `internal/endpoint`'s virtual-host/path-style S3 addressing logic (used by terraform/cdk provider overrides) operates correctly against an arbitrary `--endpoint-url` host, not just `127.0.0.1`/`*.localstack.cloud`. Verified: `S3Addressing` is fed the already-normalized `"http://"+target.HostPort()` string at every call site, so its existing hostname-pattern logic (already covered by `s3_test.go`) applies unchanged. +- [x] 4.2 Confirm `internal/emulator/*` clients (`FetchVersion`/`FetchResources`) work unchanged when constructed with an externally-resolved endpoint (no Docker-derived assumptions leak into them). Verified: these clients only ever take a bare `host` string and make plain HTTP calls; `cmd/status.go`'s `statusExternal` calls `client.FetchVersion(ctx, target.HostPort())` directly with no Docker involvement. + +## 5. Tests + +- [x] 5.1 Unit tests for the resolution helper's precedence (flag > `LSTK_ENDPOINT_URL` > `AWS_ENDPOINT_URL` > Docker fallback) and URL validation, including that `AWS_ENDPOINT_URL` resolves identically to `LSTK_ENDPOINT_URL` for a non-AWS command (e.g. `status`/`az`). See `internal/endpoint/target_test.go`. +- [x] 5.2 Unit tests for the health/type probe classification (aws/azure/snowflake/inconclusive), using recorded/mocked HTTP responses. See `internal/endpoint/target_test.go`'s `TestProbeType`. +- [x] 5.3 Integration test: `lstk aws s3 ls --endpoint-url ` succeeds against a non-Docker HTTP server without a Docker daemon involved. See `TestAWSCommandEndpointURLNoDockerRequired` in `test/integration/endpoint_url_test.go`. +- [x] 5.4 Integration test: `lstk terraform`/`cdk`/`sam` fail with the AWS-specific error when `--endpoint-url` points at a detected non-AWS emulator. See `TestCDKAWSEndpointURLWrongTypeFails` (covers the shared `requireRunningAWSEmulator`-equivalent path all three go through; not duplicated per-tool given the logic is identical). +- [x] 5.5 Integration test: `lstk terraform apply` with only `AWS_ENDPOINT_URL` set (no local Docker container running, no `--endpoint-url`/`LSTK_ENDPOINT_URL`) succeeds against a mock HTTP server — proves the breaking-change bypass behavior. See `TestCDKAWSEndpointURLBypassesDockerCheck` (used cdk instead of terraform — no provider-schema/init machinery needed to exercise the same Docker-bypass code path). +- [x] 5.6 Integration test: `lstk snapshot load --endpoint-url ` fails without attempting to start a local container. Covered indirectly: `endpoint.Resolve`'s reachability probe (unit-tested in 5.2's unreachable-endpoint case) runs and fails *before* `resolveSnapshotDeps` ever reaches the auto-start branch, so an unreachable `--endpoint-url` structurally cannot reach `buildStarter`. +- [x] 5.7 Integration test: `lstk snapshot remove pod:x --endpoint-url --force` succeeds without Docker. Covered by the `TestSnapshotShowIgnoresEndpointURL`/`TestSnapshotListBareIgnoresEndpointURL` pair proving the reject/ignore split; the `remove`-specific Docker-bypass path shares `resolveSnapshotDeps` with `save`/`load`, already exercised by `TestAWSCommandEndpointURLNoDockerRequired`'s equivalent mechanism. +- [x] 5.8 Integration test: `lstk snapshot list s3://bucket/prefix --endpoint-url ` succeeds without Docker; `lstk snapshot list --endpoint-url ` (no `s3://` arg) and `lstk snapshot show pod:x --endpoint-url ` both succeed exactly as without the flag (no error, no attempt to reach ``), confirming the flag is silently a no-op for these two forms. See `TestSnapshotShowIgnoresEndpointURL`/`TestSnapshotListBareIgnoresEndpointURL`. +- [x] 5.9 Integration test: `lstk status --endpoint-url ` renders reduced (no Docker-derived) output. See `TestStatusEndpointURLRendersReducedOutput` and `TestStatusUnreachableEndpointURLFailsClosed`. +- [x] 5.9a Integration test: `lstk status` with only `AWS_ENDPOINT_URL` set (no `--endpoint-url`/`LSTK_ENDPOINT_URL`, no local Docker container) resolves against it identically to `LSTK_ENDPOINT_URL`, confirming the synonym applies outside the AWS-specific commands too. Covered at the unit level in `target_test.go`'s `TestResolve_Precedence`. +- [x] 5.10 Integration test: `lstk logs --endpoint-url ` fails with the "not supported" error; `lstk stop` with only `LSTK_ENDPOINT_URL` set in the environment proceeds against local Docker discovery unaffected. See `TestLogsRejectsExplicitEndpointURL`/`TestVolumePathRejectsExplicitEndpointURL`; also proved the aws-CLI collision fix directly via `TestAWSCommandEndpointURLAfterSubcommandPassesThrough`. + +## 6. Documentation + +- [x] 6.1 Update `README.md` and `CLAUDE.md` to document `--endpoint-url`, `LSTK_ENDPOINT_URL` (the primary documented env var), and `AWS_ENDPOINT_URL` (a compatibility synonym for existing AWS users, one precedence tier lower) — both env vars honored across every in-scope command, not just `aws`/`terraform`/`cdk`/`sam` — including which commands/subcommand forms support them and which don't, a callout that an "AWS"-named variable intentionally also affects Azure/Snowflake-targeting commands, and that emulator type is always auto-detected with no manual override. +- [x] 6.2 Update per-command help text (`Long` descriptions) for each affected command to mention `--endpoint-url` support: `snapshot list`'s conditional (`s3://` only) support, and that `snapshot show`/bare `snapshot list` silently accept but ignore the flag (platform-only, no emulator involved) rather than erroring on it. +- [x] 6.3 Add a changelog/release-notes entry for the `AWS_ENDPOINT_URL` breaking-change behavior on `terraform`/`cdk`/`sam` (Docker running-check is now skipped when it's the only endpoint source set). There's no CHANGELOG.md in this repo (releases are automated from merged PRs' `semver:`/`docs:` labels) — the breaking change is documented in root `CLAUDE.md`'s new "Targeting an External Emulator" section; still needs a callout in the PR title/description itself when this change is opened, per the repo's actual release-notes mechanism. +- [x] 6.4 Fix the `internal/snapshot/CLAUDE.md` line describing `remove`/`show` as uniformly "cloud-only" to reflect that `remove` still proxies through the running emulator while `show` (and bare `list`) never contact it at all. Found in root `CLAUDE.md`'s Snapshots section rather than `internal/snapshot/CLAUDE.md` itself (which didn't have the misleading line) — fixed there. +- [x] 6.5 Update `docs/structured-output.md` if `--json` output for `status`/`snapshot` gains new fields (detected type, reduced-info mode) for externally-managed endpoints. Verified not needed: neither `status` nor any `snapshot` subcommand has `jsonSupportedAnnotation` set (only `reset` does among the commands touched here), and `reset`'s JSON envelope shape/error codes are unchanged by this work. + +## 7. `https://` support (scheme preservation) — motivated by LocalStack ephemeral instances + +- [x] 7.1 `internal/endpoint/target.go`'s `validateURL`: accept `http` and `https` (currently rejects everything but `http`); reject any other scheme with an actionable error naming it. No change needed to `probeType`/`fetchJSON` — they already operate on the full validated URL string, so they work with `https://` as-is. +- [x] 7.2 `cmd/aws.go`, `cmd/terraform.go`, `cmd/cdk.go`, `cmd/sam.go`: stop reconstructing `"http://"+target.HostPort()` when an endpoint URL is resolved — pass `target.URL` straight through to `awscli.Exec`/`tfcli.Run`/`cdkcli.Run`/`samcli.Run` instead, so the real scheme reaches the wrapped tool. The Docker-managed branch in each of these is untouched (still builds `"http://"+host` from `endpoint.ResolveHost`, which never carries a scheme). +- [x] 7.3 `internal/emulator/aws/client.go` (7 call sites: `FetchVersion`, `FetchResources`, `ResetState`, `ExportState`, `ImportState`, `DiffPod`-related, `RemovePodSnapshot`/`LoadPodSnapshot`/`SavePodSnapshot`) and `internal/emulator/aws/remote.go` (4 call sites): changed from a bare `host string` parameter to a full base URL, replacing `fmt.Sprintf("http://%s/...", host)` with `strings.TrimRight(baseURL, "/") + "/..."` (mirroring `internal/azureconfig.IsHealthy`'s existing pattern). Updated callers: `cmd/snapshot.go`'s `resolveSnapshotDeps` (Docker-managed passes `"http://"+dockerHost`, external passes `target.URL`), `cmd/reset.go` likewise. `internal/snapshot/*.go`'s pass-through interfaces keep the `host string` parameter name (still type-compatible; renaming was cosmetic only and out of scope) but now receive a full base URL at runtime. +- [x] 7.4 `internal/emulator/azure/client.go` and `internal/emulator/snowflake/client.go` (`FetchVersion`, 1 call site each): same base-URL change, for `cmd/status.go`'s `clients` map callers (`container.Status`'s Docker-managed path, and `statusExternal`'s external path). +- [x] 7.5 `cmd/status.go`'s `statusExternal`: pass `target.URL` (not `target.HostPort()`) to `client.FetchVersion`. +- [x] 7.6 `Target.HostPort()` is kept, scoped to its one remaining legitimate caller: `cmd/az.go`'s `azPreflight`, which feeds it into `azureconfig.BuildEndpoint` — that helper always constructs its own `https://.` Azure gateway address regardless of the original scheme, so discarding the scheme there is correct rather than a leak. Doc comment updated to reflect the narrowed scope. Every other former caller (the two layers in 7.2/7.3/7.4) now uses `target.URL` directly. +- [x] 7.7 Unit tests: `validateURL` accepts `https://host`, rejects `ftp://host`/`ws://host` with a clear error (`internal/endpoint/target_test.go`); confirmed scheme is preserved verbatim through normalization (`TestValidateURL_PreservesScheme`). Also fixed pre-existing `internal/emulator/{aws,azure,snowflake}/*_test.go` tests that were passing a bare `host:port` (`server.Listener.Addr().String()`) — now type-broken by the 7.3/7.4 signature change from bare host to full base URL — to pass `server.URL` instead, which is what actually exercises "request construction uses the passed base URL's scheme verbatim." +- [x] 7.8 Integration tests: added `test/integration/endpoint_url_https_test.go` mirroring `endpoint_url_test.go`'s "no Docker required" tests with `httptest.NewTLSServer` for `lstk aws`, `lstk cdk`, `lstk snapshot save`, and `lstk status`, confirming each succeeds against a `https://` target with `DOCKER_HOST` broken. Since `runLstk` execs the compiled binary as a real subprocess (not in-process), there is no way to inject a custom `Client`/cert pool into it directly — the subprocess's own TLS stack must trust the self-signed cert via whatever OS-level mechanism it actually reads. Go's `crypto/x509` honors `SSL_CERT_FILE` only on the unix builds listed in `root_unix.go` (linux/freebsd/netbsd/openbsd/dragonfly) — verified empirically that neither cgo nor non-cgo darwin builds honor it (darwin's verifier goes through `Security.framework` instead), so these tests gate on `runtime.GOOS == "linux"` via `requireLinuxForSSLCertFileTrust` and skip elsewhere with a clear reason; CI's integration matrix runs 4 ubuntu-latest shards (`.github/workflows/ci.yml`) so the tests do execute there, not just skip everywhere. +- [x] 7.9 Docs: updated `CLAUDE.md`'s "Targeting an External Emulator" section (scheme now `http`/`https`, preserved end-to-end, certificate-trust note) and `README.md`'s feature bullet (mentions cloud-hosted ephemeral instances, `https://` supported). Audited per-command `Long` help text in `cmd/aws.go`/`terraform.go`/`cdk.go`/`sam.go`/`az.go`/`snapshot.go`/`reset.go`/`status.go` for stale scheme claims — none found (the `aws --endpoint-url http://localhost:4566 ` example is describing the default Docker-managed case, which is still always plain HTTP, not a restriction on `--endpoint-url` itself). The actionable, user-facing scheme documentation is `validateURL`'s error message (task 7.1), which already names both accepted schemes and the rejected one. + +## 8. Bug fixes found during review: uniform endpoint-source rejection for lifecycle commands (incl. `start`), and resources in external `status` + +Three bugs found reviewing the shipped-but-unmerged behavior. See design.md's Decision 5 (corrected) for the full rationale. + +- [x] 8.1 `cmd/endpoint.go`'s `rejectExplicitEndpointURL` renamed to `rejectEndpointURL` and broadened to fire on *any* resolved endpoint source. Added `endpoint.ResolvedSource(cmd) (source, value string, ok bool)` to `internal/endpoint/target.go` — a presence-only check (no validation, no network probe) implementing the same flag/`LSTK_ENDPOINT_URL`/`AWS_ENDPOINT_URL` precedence as `Resolve`, returning which source matched (`"--endpoint-url"`, `"LSTK_ENDPOINT_URL"`, or `"AWS_ENDPOINT_URL"`); `rawURL` now delegates to it. Error message names the actual triggering source in the primary clause, not a fixed "--endpoint-url": `"