From a3c965da5470bbc89cb8393707d9a188f79723cf Mon Sep 17 00:00:00 2001 From: James Bradford Date: Fri, 20 Nov 2026 13:55:55 -0800 Subject: [PATCH 1/4] fix(starter): restore Bash setup on native Windows Restore the V2 creator and download installer that #293 moved to Node. Compare checkout paths by directory identity so Git Bash copies only tracked starter files even when Git uses a Windows drive path. Keep archive validation, cleanup, and a new repository without a commit or remote. Have Hub select the Git for Windows Bash wrapper when it seeds its own example history. Preserve its independent npm and file URL fixes. Document the native Windows commands and fresh Copilot handoff without changing the release URLs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- AGENTS.md | 2 +- CONTRIBUTING.md | 2 +- docs/coding-conventions.md | 6 +- docs/getting-started.md | 2 +- docs/prototype-repository-setup.md | 42 +- hub/AGENTS.md | 2 + hub/docs/development.md | 4 +- .../prepare-local-development/index.test.ts | 162 ++-- .../prepare-local-development/index.ts | 39 +- .../index.test.ts | 4 +- .../index.ts | 2 +- .../enqueue-local-builds/index.test.ts | 2 +- .../scripts/run-local-builds/index.test.ts | 2 +- scripts/create-prototype-repo.mjs | 188 ----- scripts/create-prototype-repo.sh | 150 ++++ scripts/create-prototype-repo.test.mjs | 702 ++++++++---------- scripts/install-starter.mjs | 126 ---- scripts/install-starter.sh | 126 ++++ starter/AGENTS.md | 2 + starter/README.md | 4 +- 20 files changed, 788 insertions(+), 781 deletions(-) delete mode 100644 scripts/create-prototype-repo.mjs create mode 100755 scripts/create-prototype-repo.sh delete mode 100644 scripts/install-starter.mjs create mode 100755 scripts/install-starter.sh diff --git a/AGENTS.md b/AGENTS.md index b9034a3a4..4dc5e5556 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -36,7 +36,7 @@ Follow the [commit and PR conventions](CONTRIBUTING.md#commits-and-pull-requests The new Hub has one protected manual workflow for a named non-production Azure installation. It does not publish prototypes or provide a general release service. The starter follows `main` and has no separate release process. Start a new agent session after copying the starter so its local skills are discovered. -For requests to create a prototype repository or set up an existing repository for prototyping, read [prototype repository setup](docs/prototype-repository-setup.md) before acting. It owns target selection, starter installation, the fresh-session handoff, and the path from local preview to optional guidance, a team baseline, or Hub. +For requests to create a prototype repository or set up an existing repository for prototyping, read [prototype repository setup](docs/prototype-repository-setup.md) before acting. It owns target selection, starter installation, the fresh-session handoff, and the path from local preview to optional guidance, a team baseline, or Hub. On native Windows, follow its Git Bash instructions for starter creation. ## Browser validation and Playwright fallback diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 68c0b7757..77d2ee43f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -52,7 +52,7 @@ npm test npm run build ``` -Update starter skills and context templates directly under `starter/.agents/`. Keep authored context in consuming repositories separate from those shared files. For starter-copy, static context library/link and plugin setup checks, run `node --test scripts/create-prototype-repo.test.mjs scripts/setup-replica-plugin.test.mjs` from the repository root. These checks do not verify agent authoring, semantic preservation or activation behavior; use the setup skill's real-host evaluation protocol for that evidence. +Update starter skills and context templates directly under `starter/.agents/`. Keep authored context in consuming repositories separate from those shared files. For starter-copy, static context library/link and plugin setup checks, run `node --test scripts/create-prototype-repo.test.mjs scripts/setup-replica-plugin.test.mjs` from the repository root. On Windows the creator and installer tests need the full Git for Windows installation with Git Bash. These checks do not verify agent authoring, semantic preservation or activation behavior; use the setup skill's real-host evaluation protocol for that evidence. Reuse an existing compatible Node.js installation. If it is unavailable on macOS or Linux, get approval before installing [nodenv](https://github.com/nodenv/nodenv), [node-build](https://github.com/nodenv/node-build), or Node.js, or changing shell configuration. The starter installer does not install them. diff --git a/docs/coding-conventions.md b/docs/coding-conventions.md index c32953ca4..b9046e26e 100644 --- a/docs/coding-conventions.md +++ b/docs/coding-conventions.md @@ -19,7 +19,11 @@ For TypeScript modules, import the function or component directory without `/ind ## Cross-platform scripts -Scripts used outside Linux must run on Windows, macOS, and Linux without Bash, Git Bash, or WSL. Use Node.js for shared setup and contributor scripts, with native paths and portable subprocess calls. Bash is appropriate for scripts that run only in an explicitly Linux environment, such as a Linux container or CI job; document that requirement. +Choose and document the runtime and supported platforms for each workflow. Share the workflow logic across platforms and test on each platform you claim to support. WSL and Linux containers do not count as native Windows validation. + +[Starter creation and installation](prototype-repository-setup.md#2-create-the-independent-starter) use Bash. On native Windows, use Git Bash from the full Git for Windows installation. For Bash scripts, preserve the repository's LF line endings and document any Linux-only tools or features. + +When one program launches another, pass arguments separately and use paths understood by the child process. Do not assume the caller's shell makes a program executable on another platform. For example, a Node.js program on Windows cannot launch the `npm.cmd` wrapper directly, even if Node.js was started in Git Bash. ## Tests diff --git a/docs/getting-started.md b/docs/getting-started.md index 22954abf6..1454778ff 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -8,7 +8,7 @@ Hub is being rebuilt. Hosted publishing, sharing, comments, and deployment are n Download the [GitHub Copilot app](https://github.com/features/ai/github-app), install it, and follow its sign-in instructions. -You also need [Git](https://git-scm.com/), [npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm), and [Node.js](https://nodejs.org/). Install the Node.js version required by your team's prototyping repository; npm comes with Node.js. +You also need [Git](https://git-scm.com/), [npm](https://docs.npmjs.com/downloading-and-installing-node-js-and-npm), and [Node.js](https://nodejs.org/). Install the Node.js version required by your team's prototyping repository; npm comes with Node.js. If you are creating a new starter on native Windows, use [Git for Windows with Git Bash](prototype-repository-setup.md#2-create-the-independent-starter). The canvas preview workflow supports macOS and Linux. Server discovery requires `lsof` on the machine running the canvas extension; install it with your system's package manager if it is missing. **Start dev server** is supported only on macOS and Linux. diff --git a/docs/prototype-repository-setup.md b/docs/prototype-repository-setup.md index ed79dc2d3..333a7d3bd 100644 --- a/docs/prototype-repository-setup.md +++ b/docs/prototype-repository-setup.md @@ -21,33 +21,47 @@ When a reference app is supplied, retain it for the guidance and component work ## 2. Create the independent starter -Without a local Replica checkout, download the installer: +Use Bash and Git for creation. On native Windows, use Git Bash from the full [Git for Windows](https://gitforwindows.org/) installation. In the GitHub Copilot app, choose **This computer** for a native Windows session, not a WSL environment. Git on `PATH` does not mean Git Bash is available. If Git Bash is missing, report that prerequisite instead of switching environments or installing it without approval. -```sh -curl -fLSs https://raw.githubusercontent.com/github/replica/main/scripts/install-starter.mjs -o install-starter.mjs +From a local Replica checkout, run this from its root in Bash (Git Bash on Windows). If you clone Replica or select a base branch in Copilot to try an unreleased V2 change, choose the branch that contains that change rather than `main`. + +```bash +bash scripts/create-prototype-repo.sh "../my prototype" ``` -Read the downloaded script before running it. Use a destination that does not exist and whose parent directory exists: +For the released starter, download the installer from `main`. Use this download only once `main` contains this version of the starter; while developing V2 before that release, use the checkout command. The download needs `curl`, `tar`, and a writable current directory. Use the `curl` and `tar` that come with Git Bash on Windows. -```sh -node --use-env-proxy install-starter.mjs my-prototype +```bash +curl -fLSs https://raw.githubusercontent.com/github/replica/main/scripts/install-starter.sh -o install-starter.sh ``` -From a local Replica checkout, use the checked-out script instead: +Read the downloaded script before running it: + +```bash +bash ./install-starter.sh "./my prototype" +``` + +When releasing V2 to `main`, run these download and install commands with a fresh destination on native Windows. Verify that the archive provides the V2 starter and that the new repository has no commit or remote. + +If the agent is in PowerShell or another shell that cannot run the Bash script, locate `bin\bash.exe` under the actual Git for Windows installation and invoke that executable directly. Do not use a bare `bash` or `bash.exe` from `PATH`; it may point to a different shell or WSL. For example, in PowerShell: -```sh -node scripts/create-prototype-repo.mjs ../my-prototype +```powershell +$gitBash = 'C:\Program Files\Git\bin\bash.exe' # Use your verified Git for Windows path. +if (-not (Test-Path -LiteralPath $gitBash -PathType Leaf)) { + throw 'Locate Git for Windows bin\bash.exe, or install Git for Windows with Git Bash.' +} +& $gitBash 'scripts/create-prototype-repo.sh' '..\my prototype' ``` -Both paths create an independent, uncommitted Git repository with no remote. They copy the starter without Replica's source history, require no GitHub credentials, and publish nothing. The installer does not install Node.js or app dependencies. +Keep the script path and destination as separate arguments. For the reviewed download, pass `'./install-starter.sh'` and `'.\my prototype'` instead. If the script starts and returns an error, report the error. Do not retry it in another shell. -Use a Node.js version supported by the starter and Git. The `--use-env-proxy` option lets Node use configured `HTTP_PROXY`, `HTTPS_PROXY`, and `NO_PROXY` settings. Downloaded installation also needs tar and a writable current directory for temporary downloads. Creation runs without Bash on Windows, macOS and Linux. This does not change the [Linux requirements for local Hub Builds](../hub/docs/development.md#build-local-prototype-repositories). +Choose a destination that does not exist and whose parent directory exists. Both commands create an independent, uncommitted Git repository with no remote. They copy the starter without Replica's source history, require no GitHub credentials, and publish nothing. Creation does not require or install Node.js or app dependencies. This does not change the [Linux requirements for local Hub Builds](../hub/docs/development.md#build-local-prototype-repositories). Before asking the user to accept the copied repository's Copilot app configuration, review `.github/github-app.yml` and the startup details in the copied `README.md`. Explain that session startup can install Copilot CLI globally when missing, install or enable the Replica plugin, and will run `npm ci`. -Add the new directory to the Copilot app using **Projects → + → Local folder or repository**, then start a fresh session there. The session in the Replica checkout cannot discover the copied local skills. +Open the new directory as a local folder in the Copilot app, then start a fresh session in its current checkout. Do not clone again or create a worktree from the uncommitted starter. The session in the Replica checkout cannot discover the copied local skills. -Before ending the originating session, provide a paste-ready kickoff message headed **Prompt 1: Continue prototype repository setup**. Tell the user to paste it into the fresh session; a new session does not inherit the previous conversation. Include the user's goal, repository path, supplied references, requested milestone, settled decisions and approvals, completed actions, blockers, and next step. +Before ending the originating session, provide a paste-ready kickoff message headed **Prompt 1: Continue prototype repository setup**. Tell the user to paste it into the fresh session; a new session does not inherit the previous conversation. Include the user's goal, repository path, supplied references, requested milestone, settled decisions and approvals, completed actions, blockers, and next step. For Windows, include which shell ran creation and whether Copilot Startup was checked. The kickoff must tell the new session to follow the copied repository's `AGENTS.md` and resume this guide at step 3. Include the [absolute guide URL](https://github.com/github/replica/blob/main/docs/prototype-repository-setup.md#3-verify-the-existing-app-locally), since this guide is not copied into the starter. If that version is not yet available at the URL, include steps 3 through 5 inline instead. The new session reuses settled answers and checks which actions and approvals remain applicable. @@ -55,7 +69,7 @@ The kickoff must tell the new session to follow the copied repository's `AGENTS. Check `node --version` against `engines.node` in the copied `package.json`. Reuse a compatible installed runtime. On macOS or Linux, obtain approval before installing Node.js, nodenv, or node-build, or changing shell configuration. -The fresh session does not receive the setup output, so ask the user to inspect it and report any failures before repeating setup. Dependency installation success does not prove that the current session loaded the canvas. +The fresh session does not receive the setup output, so ask the user to inspect it and report any failures before repeating setup. Copilot Startup uses the app's own command runner. On native Windows, successful creation or a terminal `npm ci` does not verify Startup. Dependency installation success does not prove that the current session loaded the canvas. Follow the copied README's **Connect the Replica canvas** and **Canvas preflight** sections for plugin activation and recovery. Complete that preflight before starting a preview. Use its **Run it** commands, reuse an existing responsive server when appropriate, and open the actual printed URL. Verify that the included app renders; a successful dependency install or HTTP response alone is not a rendered check. diff --git a/hub/AGENTS.md b/hub/AGENTS.md index e842beac9..f093f2e80 100644 --- a/hub/AGENTS.md +++ b/hub/AGENTS.md @@ -25,6 +25,8 @@ Open for the mock frontend; it needs n After installing Hub dependencies, use the [devcontainer](docs/development.md#devcontainer) for the real API or prepare local PostgreSQL and configuration. Copy `.env.example` only for first-time setup; preserve an existing `.env` and never commit credentials. +On native Windows, [local example seeding](docs/development.md#start-with-the-real-api) requires Git for Windows with Git Bash. This is separate from the Linux-only Local Build workflow. + ```sh docker compose -f .devcontainer/compose.yaml up -d db [ -e .env ] || [ -L .env ] || cp -n .env.example .env diff --git a/hub/docs/development.md b/hub/docs/development.md index 609cb2ef1..fb417d538 100644 --- a/hub/docs/development.md +++ b/hub/docs/development.md @@ -61,6 +61,8 @@ Local-mode startup prepares the absolute `LOCAL_PROTOTYPE_ROOT`, defaulting to t When the prototype root is empty, the first start creates `minimal/` from the canonical [Replica Starter](../../starter/), installs its locked dependencies, commits `main`, and creates the committed `prototype/example` branch with a scaffolded example route. The commits use fixed example authorship and timestamps so the same starter revision produces the same repository history. Preparation also installs the shared [Build action](../../actions/build/) runtime dependencies from its lockfile and its pinned Chromium browser. It skips the npm install when the required dependencies are already present; Playwright's browser installation is idempotent. +Hub uses the same [Bash creator](../../scripts/create-prototype-repo.sh) as standalone starter setup. On native Windows, install Git for Windows with Git Bash and put its `git.exe` on `PATH`. Hub finds `bin\bash.exe` in that installation; it does not depend on the terminal's shell. Dependency installation still runs npm's JavaScript entrypoint through Node.js, using `npm_execpath` when supplied or the npm bundled beside Node.js. + Later starts always prepare the shared Build action dependencies, but they do not add, replace, switch, or commit anything in a nonempty prototype root. This includes the seeded repository after a developer edits it. To prepare without keeping the API running, use: ```sh @@ -114,8 +116,6 @@ The filesystem provider requires the Linux protections described in [Artifact st npm run local:builds ``` -Starter preparation invokes the [Node.js creator](../../scripts/create-prototype-repo.mjs), not Bash. On Windows, dependency installation runs npm's JavaScript entrypoint through Node.js, using `npm_execpath` when supplied or the npm bundled beside Node.js. These setup fixes do not remove the Linux filesystem requirement or Docker isolation for local Builds. - Alternatively, after `npm run setup`, enqueue the same scan and run the local on-demand worker: ```sh diff --git a/hub/server/domains/local-development/prepare-local-development/index.test.ts b/hub/server/domains/local-development/prepare-local-development/index.test.ts index 5fd89ce0b..d58607401 100644 --- a/hub/server/domains/local-development/prepare-local-development/index.test.ts +++ b/hub/server/domains/local-development/prepare-local-development/index.test.ts @@ -1,7 +1,8 @@ import { randomUUID } from "node:crypto" import { execFileSync } from "node:child_process" +import { statSync } from "node:fs" import { mkdir, readFile, rm, writeFile } from "node:fs/promises" -import { basename, isAbsolute, relative, resolve } from "node:path" +import { basename, dirname, isAbsolute, relative, resolve } from "node:path" import { describe, expect, it, onTestFinished } from "vitest" import { runCommand } from "../../../integrations/process/run-command" @@ -10,6 +11,7 @@ import type { LocalDevelopmentConfiguration } from "../types" import { prepareLocalDevelopment } from "." const sourceRepositoryRoot = resolve("..") +const seedTimeout = process.platform === "win32" ? 90_000 : 15_000 function git(directory: string, ...arguments_: string[]): string { return execFileSync("git", ["-c", "maintenance.auto=false", "-C", directory, ...arguments_], { @@ -39,13 +41,13 @@ function expectSeedDependencyInstall(directory: string, prototypesRoot: string) expect(basename(directory)).toBe("minimal") } -function testConfiguration(): { +function testConfiguration(rootSuffix = ""): { configuration: LocalDevelopmentConfiguration & { enabled: true } browserDirectories: string[] npmDirectories: string[] runTestCommand: (options: RunCommandOptions) => Promise } { - const root = resolve(`.prepare-test-${randomUUID()}`) + const root = resolve(`.prepare-test-${randomUUID()}${rootSuffix}`) const browserDirectories: string[] = [] const npmDirectories: string[] = [] onTestFinished(() => rm(root, { force: true, recursive: true })) @@ -54,7 +56,7 @@ function testConfiguration(): { browserDirectories, configuration: { buildActionDirectory: resolve(root, ".build-action"), - createPrototypeScript: resolve(sourceRepositoryRoot, "scripts/create-prototype-repo.mjs"), + createPrototypeScript: resolve(sourceRepositoryRoot, "scripts/create-prototype-repo.sh"), enabled: true, root, sourceRoot: sourceRepositoryRoot, @@ -83,9 +85,19 @@ function testConfiguration(): { describe("prepareLocalDevelopment()", () => { it.runIf(process.platform === "win32")( - "runs both npm installs through Node with a literal entrypoint containing spaces and shell characters", + "seeds through Git for Windows Bash and runs npm through a literal Node entrypoint", async () => { - const fixture = testConfiguration() + const fixture = testConfiguration(" spaced") + const { createPrototypeScript, root } = fixture.configuration + expect(execFileSync("git", ["--version"], { encoding: "utf8" })).toMatch(/\.windows\./) + const gitCore = execFileSync("git", ["--exec-path"], { encoding: "utf8" }).trim() + expect(isAbsolute(gitCore)).toBe(true) + expect([basename(dirname(gitCore)), basename(gitCore)]).toEqual(["libexec", "git-core"]) + const installation = resolve(gitCore, "..", "..", "..") + const bash = resolve(installation, "bin", "bash.exe") + expect(statSync(bash).isFile()).toBe(true) + expect(statSync(resolve(installation, "usr", "bin", "cygpath.exe")).isFile()).toBe(true) + const scaffoldArguments: string[][] = [] const tools = `${fixture.configuration.root}-npm tools & spaces` onTestFinished(() => rm(tools, { force: true, recursive: true })) const entrypoint = resolve(tools, "npm-cli.js") @@ -101,12 +113,26 @@ appendFileSync(${JSON.stringify(log)}, JSON.stringify(process.argv.slice(2)) + " vi.stubEnv("npm_execpath", entrypoint) await prepareLocalDevelopment(fixture.configuration, { - runCommand: (options) => - options.arguments[0] === entrypoint + runCommand: (options) => { + if ( + options.arguments.includes(createPrototypeScript) || + /^(bash|sh|wsl)(\.exe)?$/i.test(basename(options.command)) + ) { + expect(isAbsolute(options.command)).toBe(true) + expect(resolve(options.command).toLowerCase()).toBe(bash.toLowerCase()) + scaffoldArguments.push([...options.arguments]) + } + return options.arguments[0] === entrypoint ? runCommand(options) - : fixture.runTestCommand(options), + : fixture.runTestCommand(options) + }, }) + expect(scaffoldArguments).toEqual([[createPrototypeScript, expect.any(String)]]) + const checkout = scaffoldArguments[0]![1]! + expect(isAbsolute(checkout)).toBe(true) + expect(checkout).toContain(" spaced") + expectSeedDependencyInstall(checkout, root) expect( (await readFile(log, "utf8")) .trim() @@ -120,10 +146,10 @@ appendFileSync(${JSON.stringify(log)}, JSON.stringify(process.argv.slice(2)) + " Object.keys(branchRefs(resolve(fixture.configuration.root, "minimal"))).sort(), ).toEqual(["main", "prototype/example"]) }, - 15_000, + seedTimeout, ) - it("reports the Starter creation stage safely when the Node scaffold fails", async () => { + it("reports the Starter creation stage without subprocess diagnostics", async () => { const fixture = testConfiguration() await expect( @@ -140,29 +166,33 @@ appendFileSync(${JSON.stringify(log)}, JSON.stringify(process.argv.slice(2)) + " ) }) - it("deterministically seeds Minimal with committed main and example branches", async () => { - vi.stubEnv("npm_execpath", undefined) - const first = testConfiguration() + it( + "deterministically seeds Minimal with committed main and example branches", + async () => { + vi.stubEnv("npm_execpath", undefined) + const first = testConfiguration() - await prepareLocalDevelopment(first.configuration, { runCommand: first.runTestCommand }) + await prepareLocalDevelopment(first.configuration, { runCommand: first.runTestCommand }) - const minimal = resolve(first.configuration.root, "minimal") - expect(Object.keys(branchRefs(minimal)).sort()).toEqual(["main", "prototype/example"]) - expect(git(minimal, "status", "--porcelain")).toBe("") - expect(git(minimal, "rev-list", "--count", "main")).toBe("1") - expect(git(minimal, "rev-list", "--count", "prototype/example")).toBe("2") - expect(await readFile(resolve(minimal, "package.json"), "utf8")).toBe( - await readFile(resolve(sourceRepositoryRoot, "starter/package.json"), "utf8"), - ) - expect(first.npmDirectories[0]).toBe(first.configuration.buildActionDirectory) - expectSeedDependencyInstall(first.npmDirectories[1]!, first.configuration.root) - expect(first.npmDirectories).toHaveLength(2) - expect(first.browserDirectories).toEqual([first.configuration.buildActionDirectory]) + const minimal = resolve(first.configuration.root, "minimal") + expect(Object.keys(branchRefs(minimal)).sort()).toEqual(["main", "prototype/example"]) + expect(git(minimal, "status", "--porcelain")).toBe("") + expect(git(minimal, "rev-list", "--count", "main")).toBe("1") + expect(git(minimal, "rev-list", "--count", "prototype/example")).toBe("2") + expect(await readFile(resolve(minimal, "package.json"), "utf8")).toBe( + await readFile(resolve(sourceRepositoryRoot, "starter/package.json"), "utf8"), + ) + expect(first.npmDirectories[0]).toBe(first.configuration.buildActionDirectory) + expectSeedDependencyInstall(first.npmDirectories[1]!, first.configuration.root) + expect(first.npmDirectories).toHaveLength(2) + expect(first.browserDirectories).toEqual([first.configuration.buildActionDirectory]) - const second = testConfiguration() - await prepareLocalDevelopment(second.configuration, { runCommand: second.runTestCommand }) - expect(branchRefs(resolve(second.configuration.root, "minimal"))).toEqual(branchRefs(minimal)) - }, 15_000) + const second = testConfiguration() + await prepareLocalDevelopment(second.configuration, { runCommand: second.runTestCommand }) + expect(branchRefs(resolve(second.configuration.root, "minimal"))).toEqual(branchRefs(minimal)) + }, + seedTimeout, + ) it("does not seed or alter repositories when the configured root is non-empty", async () => { const fixture = testConfiguration() @@ -192,39 +222,43 @@ appendFileSync(${JSON.stringify(log)}, JSON.stringify(process.argv.slice(2)) + " ) }) - it("preserves edited files, commits, and branches on later starts", async () => { - const fixture = testConfiguration() - await prepareLocalDevelopment(fixture.configuration, { runCommand: fixture.runTestCommand }) - const minimal = resolve(fixture.configuration.root, "minimal") - git(minimal, "switch", "-c", "prototype/edited", "prototype/example") - await writeFile(resolve(minimal, "local-edit.txt"), "keep my edit\n") - git(minimal, "add", "local-edit.txt") - git( - minimal, - "-c", - "user.name=Local developer", - "-c", - "user.email=local@example.invalid", - "-c", - "commit.gpgsign=false", - "commit", - "-m", - "feat: keep local edit", - ) - const refsBefore = branchRefs(minimal) + it( + "preserves edited files, commits, and branches on later starts", + async () => { + const fixture = testConfiguration() + await prepareLocalDevelopment(fixture.configuration, { runCommand: fixture.runTestCommand }) + const minimal = resolve(fixture.configuration.root, "minimal") + git(minimal, "switch", "-c", "prototype/edited", "prototype/example") + await writeFile(resolve(minimal, "local-edit.txt"), "keep my edit\n") + git(minimal, "add", "local-edit.txt") + git( + minimal, + "-c", + "user.name=Local developer", + "-c", + "user.email=local@example.invalid", + "-c", + "commit.gpgsign=false", + "commit", + "-m", + "feat: keep local edit", + ) + const refsBefore = branchRefs(minimal) - await prepareLocalDevelopment(fixture.configuration, { runCommand: fixture.runTestCommand }) + await prepareLocalDevelopment(fixture.configuration, { runCommand: fixture.runTestCommand }) - expect(branchRefs(minimal)).toEqual(refsBefore) - expect(git(minimal, "branch", "--show-current")).toBe("prototype/edited") - expect(await readFile(resolve(minimal, "local-edit.txt"), "utf8")).toBe("keep my edit\n") - expect(fixture.npmDirectories[0]).toBe(fixture.configuration.buildActionDirectory) - expectSeedDependencyInstall(fixture.npmDirectories[1]!, fixture.configuration.root) - expect(fixture.npmDirectories[2]).toBe(fixture.configuration.buildActionDirectory) - expect(fixture.npmDirectories).toHaveLength(3) - expect(fixture.browserDirectories).toEqual([ - fixture.configuration.buildActionDirectory, - fixture.configuration.buildActionDirectory, - ]) - }) + expect(branchRefs(minimal)).toEqual(refsBefore) + expect(git(minimal, "branch", "--show-current")).toBe("prototype/edited") + expect(await readFile(resolve(minimal, "local-edit.txt"), "utf8")).toBe("keep my edit\n") + expect(fixture.npmDirectories[0]).toBe(fixture.configuration.buildActionDirectory) + expectSeedDependencyInstall(fixture.npmDirectories[1]!, fixture.configuration.root) + expect(fixture.npmDirectories[2]).toBe(fixture.configuration.buildActionDirectory) + expect(fixture.npmDirectories).toHaveLength(3) + expect(fixture.browserDirectories).toEqual([ + fixture.configuration.buildActionDirectory, + fixture.configuration.buildActionDirectory, + ]) + }, + seedTimeout, + ) }) diff --git a/hub/server/domains/local-development/prepare-local-development/index.ts b/hub/server/domains/local-development/prepare-local-development/index.ts index 0bdf2c3bd..ebe0760a2 100644 --- a/hub/server/domains/local-development/prepare-local-development/index.ts +++ b/hub/server/domains/local-development/prepare-local-development/index.ts @@ -1,5 +1,7 @@ +import { execFileSync } from "node:child_process" +import { statSync } from "node:fs" import { lstat, mkdir, mkdtemp, readdir, rename, rm, stat } from "node:fs/promises" -import { basename, dirname, join } from "node:path" +import { basename, dirname, isAbsolute, join, resolve } from "node:path" import { runCommand as runProcessCommand } from "../../../integrations/process/run-command/index.ts" import { LocalDevelopmentPreparationError } from "../errors.ts" @@ -26,6 +28,39 @@ async function exists(path: string) { } } +function starterBash(sourceRoot: string) { + if (process.platform !== "win32") return "bash" + + const prerequisite = + "Install Git for Windows with Git Bash, and make its git.exe available on PATH to create the Starter repository." + const environment = { ...process.env } + for (const key of Object.keys(environment)) { + if (key.toLowerCase() === "git_exec_path") delete environment[key] + } + let gitCore: string + try { + gitCore = execFileSync("git", ["--exec-path"], { + cwd: sourceRoot, + encoding: "utf8", + env: environment, + stdio: ["ignore", "pipe", "ignore"], + }).trim() + } catch { + throw new LocalDevelopmentPreparationError(prerequisite) + } + // Git reports //libexec/git-core, not the wrapper on PATH. + const bash = resolve(gitCore, "../../..", "bin", "bash.exe") + if ( + !isAbsolute(gitCore) || + basename(gitCore) !== "git-core" || + basename(dirname(gitCore)) !== "libexec" || + !statSync(bash, { throwIfNoEntry: false })?.isFile() + ) { + throw new LocalDevelopmentPreparationError(prerequisite) + } + return bash +} + export async function prepareLocalDevelopment( configuration: LocalDevelopmentConfiguration, dependencies: PrepareLocalDevelopmentDependencies = {}, @@ -88,7 +123,7 @@ export async function prepareLocalDevelopment( const checkout = join(stagingDirectory, exampleDirectoryName) try { await runCommand({ - command: process.execPath, + command: starterBash(configuration.sourceRoot), arguments: [createPrototypeScript, checkout], cwd: configuration.sourceRoot, environment: { diff --git a/hub/server/domains/local-development/read-local-development-configuration/index.test.ts b/hub/server/domains/local-development/read-local-development-configuration/index.test.ts index b387f7423..8326fa8f6 100644 --- a/hub/server/domains/local-development/read-local-development-configuration/index.test.ts +++ b/hub/server/domains/local-development/read-local-development-configuration/index.test.ts @@ -48,7 +48,7 @@ describe("readLocalDevelopmentConfiguration()", () => { ), ).toEqual({ buildActionDirectory: resolve(repositoryRoot, "actions/build"), - createPrototypeScript: resolve(repositoryRoot, "scripts/create-prototype-repo.mjs"), + createPrototypeScript: resolve(repositoryRoot, "scripts/create-prototype-repo.sh"), enabled: true, root: resolve(applicationRoot, ".local/prototypes"), sourceRoot: repositoryRoot, @@ -80,7 +80,7 @@ describe("readLocalDevelopmentConfiguration()", () => { ), ).toEqual({ buildActionDirectory: buildActionRoot, - createPrototypeScript: resolve(repositoryRoot, "scripts/create-prototype-repo.mjs"), + createPrototypeScript: resolve(repositoryRoot, "scripts/create-prototype-repo.sh"), enabled: true, root: prototypesRoot, sourceRoot: repositoryRoot, diff --git a/hub/server/domains/local-development/read-local-development-configuration/index.ts b/hub/server/domains/local-development/read-local-development-configuration/index.ts index 07d882185..474537096 100644 --- a/hub/server/domains/local-development/read-local-development-configuration/index.ts +++ b/hub/server/domains/local-development/read-local-development-configuration/index.ts @@ -64,7 +64,7 @@ export function readLocalDevelopmentConfiguration( root, sourceRoot, buildActionDirectory: configuredBuildAction ?? resolve(sourceRoot, "actions/build"), - createPrototypeScript: resolve(sourceRoot, "scripts/create-prototype-repo.mjs"), + createPrototypeScript: resolve(sourceRoot, "scripts/create-prototype-repo.sh"), temporaryRoot: configuredTemporaryRoot ?? tmpdir(), } } diff --git a/hub/server/scripts/enqueue-local-builds/index.test.ts b/hub/server/scripts/enqueue-local-builds/index.test.ts index a3d8b49c3..99379ac19 100644 --- a/hub/server/scripts/enqueue-local-builds/index.test.ts +++ b/hub/server/scripts/enqueue-local-builds/index.test.ts @@ -36,7 +36,7 @@ describe("enqueueLocalBuilds()", () => { } const configuration = { buildActionDirectory: "/configured/build-action", - createPrototypeScript: "/repository/scripts/create-prototype-repo.mjs", + createPrototypeScript: "/repository/scripts/create-prototype-repo.sh", enabled: true as const, root: "/configured/prototypes", sourceRoot: "/repository", diff --git a/hub/server/scripts/run-local-builds/index.test.ts b/hub/server/scripts/run-local-builds/index.test.ts index f4cafca74..e5be2f13e 100644 --- a/hub/server/scripts/run-local-builds/index.test.ts +++ b/hub/server/scripts/run-local-builds/index.test.ts @@ -44,7 +44,7 @@ describe("runLocalBuilds()", () => { } vi.mocked(readLocalDevelopmentConfiguration).mockReturnValue({ buildActionDirectory: configuration.actionRoot, - createPrototypeScript: "/repository/scripts/create-prototype-repo.mjs", + createPrototypeScript: "/repository/scripts/create-prototype-repo.sh", enabled: true, root: configuration.prototypeRoot, sourceRoot: "/repository", diff --git a/scripts/create-prototype-repo.mjs b/scripts/create-prototype-repo.mjs deleted file mode 100644 index 0e15b53f0..000000000 --- a/scripts/create-prototype-repo.mjs +++ /dev/null @@ -1,188 +0,0 @@ -import { execFileSync } from "node:child_process" -import { - chmodSync, - copyFileSync, - lstatSync, - mkdirSync, - mkdtempSync, - readdirSync, - realpathSync, - rmSync, - utimesSync, -} from "node:fs" -import { cp } from "node:fs/promises" -import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path" -import { fileURLToPath, pathToFileURL } from "node:url" - -const sourceRoot = realpathSync(dirname(dirname(fileURLToPath(import.meta.url)))) -const usage = "Usage: node scripts/create-prototype-repo.mjs " - -function stat(path) { - try { - return lstatSync(path) - } catch (error) { - if (error.code !== "ENOENT" && error.code !== "ENOTDIR") throw error - } -} - -function within(root, path) { - const local = relative(root, path) - return !local || (!isAbsolute(local) && local !== ".." && !local.startsWith(`..${sep}`)) -} - -function safePath(path) { - return path - .split("/") - .every( - (part) => - part && - part !== "." && - part !== ".." && - !/[\\:\x00-\x1f<>\"|?*]/.test(part) && - !/[. ]$/.test(part) && - !/^\.git$/i.test(part) && - !/^(con|prn|aux|nul|com[0-9]|lpt[0-9])(?:\.|$)/i.test(part), - ) -} - -export async function createRepository(argument, { signal } = {}) { - signal?.throwIfAborted() - const name = basename(argument) - const parentPath = dirname(resolve(argument)) - if (!stat(parentPath)?.isDirectory() && !stat(parentPath)?.isSymbolicLink()) { - throw new Error("Destination must have an existing parent directory.") - } - const parent = realpathSync(parentPath) - if (!name || name === "." || name === ".." || name === sep) { - throw new Error("Destination must have an existing parent directory.") - } - const internalPath = process.env.REPLICA_INTERNAL_STARTER_ROOT - let internalRoot - if (internalPath) { - internalRoot = realpathSync(internalPath) - if (!stat(internalRoot)?.isDirectory()) - throw new Error("Internal starter root must be an existing directory.") - } - if (within(sourceRoot, parent) && (!internalRoot || !within(internalRoot, parent))) { - throw new Error("Create the new repository outside the Replica checkout.") - } - const destination = join(parent, name) - if (stat(destination)) throw new Error(`Destination already exists: ${destination}`) - const starter = join(sourceRoot, "starter") - if (!stat(starter)?.isDirectory()) - throw new Error("Starter skills or context templates are missing or linked.") - const temporary = mkdtempSync(join(parent, ".replica-starter.")) - let created = false - let count = 0 - try { - const files = join(temporary, "files") - mkdirSync(files) - let checkout - try { - checkout = realpathSync( - execFileSync("git", ["-C", sourceRoot, "rev-parse", "--show-toplevel"], { - encoding: "utf8", - stdio: ["ignore", "pipe", "ignore"], - }).trim(), - ) - } catch (error) { - if (error.code !== "ENOENT" && error.status === undefined) throw error - } - function copy(path) { - if (!safePath(path)) throw new Error(`Unsupported starter entry: ${path}`) - let directory = starter - for (const part of path.split("/").slice(0, -1)) { - directory = join(directory, part) - if (!stat(directory)?.isDirectory()) - throw new Error(`Linked starter directory or missing directory: ${directory}`) - } - const source = join(starter, path) - const metadata = stat(source) - if (!metadata?.isFile()) - throw new Error(`Starter file is missing or is not a regular file: ${source}`) - const target = join(files, path) - mkdirSync(dirname(target), { recursive: true }) - copyFileSync(source, target) - chmodSync(target, metadata.mode) - utimesSync(target, metadata.atime, metadata.mtime) - count += 1 - } - if (checkout === sourceRoot) { - const manifest = execFileSync( - "git", - ["-C", sourceRoot, "ls-files", "--cached", "--stage", "-z", "--", "starter/"], - { - encoding: "utf8", - maxBuffer: 32 * 1024 * 1024, - }, - ) - for (const entry of manifest.split("\0").filter(Boolean)) { - const match = /^(100644|100755) [a-f0-9]+ 0\tstarter\/(.+)$/.exec(entry) - if (!match) throw new Error(`Unsupported starter entry: ${entry}`) - copy(match[2]) - } - } else { - function visit(directory, prefix = "") { - for (const entry of readdirSync(directory, { withFileTypes: true })) { - const path = `${prefix}${entry.name}` - if (!safePath(path)) - throw new Error(`Starter archive contains Git metadata or an unsafe path: ${path}`) - if (entry.isDirectory()) visit(join(directory, entry.name), `${path}/`) - else if (entry.isFile()) copy(path) - else throw new Error(`Starter archive contains an unsupported file: ${path}`) - } - } - visit(starter) - } - for (const path of [ - ".agents/skills/setup-skills/SKILL.md", - ".agents/context-templates/SKILL.md", - ]) { - if (!stat(join(files, path))?.isFile()) - throw new Error("Starter skills or context templates are missing or linked.") - } - if (!count) throw new Error(`No starter files found in ${sourceRoot}`) - execFileSync("git", ["init", "-q", "--initial-branch=main", files]) - mkdirSync(destination) - created = true - await cp(files, destination, { recursive: true, preserveTimestamps: true }) - signal?.throwIfAborted() - created = false - console.log( - `Created ${destination} with ${count} starter files and a new, uncommitted Git repository.`, - ) - console.log("Start a new agent session in that directory and run /setup-skills.") - } finally { - rmSync(temporary, { recursive: true, force: true }) - if (created) rmSync(destination, { recursive: true, force: true }) - } -} - -const entrypoint = - process.argv[1] && stat(process.argv[1]) - ? pathToFileURL(realpathSync(process.argv[1])).href - : undefined -if (import.meta.url === entrypoint) { - if (process.argv.length !== 3) { - console.error(usage) - process.exitCode = 2 - } else { - const controller = new AbortController() - const interrupts = ["SIGINT", "SIGTERM"].map((signal) => { - const interrupt = () => { - process.exitCode = signal === "SIGINT" ? 130 : 143 - controller.abort(new Error("Repository creation interrupted.")) - } - process.on(signal, interrupt) - return [signal, interrupt] - }) - try { - await createRepository(process.argv[2], { signal: controller.signal }) - } catch (error) { - console.error(error.message) - process.exitCode ||= 1 - } finally { - for (const [signal, interrupt] of interrupts) process.off(signal, interrupt) - } - } -} diff --git a/scripts/create-prototype-repo.sh b/scripts/create-prototype-repo.sh new file mode 100755 index 000000000..e0f77d673 --- /dev/null +++ b/scripts/create-prototype-repo.sh @@ -0,0 +1,150 @@ +#!/usr/bin/env bash +set -euo pipefail +unset CDPATH + +if [[ $# -ne 1 ]]; then + printf 'Usage: %s \n' "$0" >&2 + exit 2 +fi + +fail() { + printf '%s\n' "$1" >&2 + exit 1 +} + +source_root="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")/.." && pwd -P)" +starter="$source_root/starter" +argument="$1" +if [[ "$OSTYPE" == cygwin* || "$OSTYPE" == msys* ]]; then + argument="$(cygpath -u -- "$argument")" +fi +name="$(basename -- "$argument")" +if [[ -z "$name" || "$name" == "." || "$name" == ".." || "$name" == "/" ]] || + ! parent="$(cd -- "$(dirname -- "$argument")" && pwd -P)"; then + fail 'Destination must have an existing parent directory.' +fi +destination="${parent%/}/$name" + +within() { + local root="$1" path="$2" + while true; do + [[ "$path" -ef "$root" ]] && return 0 + [[ "$path" == "/" ]] && return 1 + path="${path%/*}" + path="${path:-/}" + done +} + +internal_root="${REPLICA_INTERNAL_STARTER_ROOT:-}" +if [[ -n "$internal_root" ]] && ! internal_root="$(cd -- "$internal_root" && pwd -P)"; then + fail 'Internal starter root must be an existing directory.' +fi +if within "$source_root" "$parent" && + { [[ -z "$internal_root" ]] || ! within "$internal_root" "$parent"; }; then + fail 'Create the new repository outside the Replica checkout.' +fi +if [[ -e "$destination" || -L "$destination" ]]; then + fail "Destination already exists: $destination" +fi +if [[ ! -d "$starter" || -L "$starter" ]]; then + fail 'Starter skills or context templates are missing or linked.' +fi + +temporary="$(mktemp -d "$parent/.replica-starter.XXXXXX")" +created="" +cleanup() { + local status=$? + trap '' INT TERM + rm -rf -- "$temporary" || status=1 + if [[ -n "$created" ]]; then + rm -rf -- "$created" || status=1 + fi + exit "$status" +} +trap cleanup EXIT +trap 'exit 130' INT +trap 'exit 143' TERM + +unsafe='[[:cntrl:]\\:<>"|?*]|[. ](/|$)' +device='(^|/)([Cc][Oo][Nn]|[Pp][Rr][Nn]|[Aa][Uu][Xx]|[Nn][Uu][Ll]|[Cc][Oo][Mm][0-9]|[Ll][Pp][Tt][0-9])([./]|$)' +git_metadata='(^|/)\.[Gg][Ii][Tt](/|$)' +safe_path() { + [[ -n "$1" && "$1" != /* && "$1" != */ && "$1" != *//* && + ! "$1" =~ $unsafe && ! "$1" =~ $device && ! "$1" =~ $git_metadata ]] +} + +files="$temporary/files" +mkdir -- "$files" +count=0 +copy_file() { + local path="$1" source="$starter/$1" directory="$1" + safe_path "$path" || fail "Unsupported starter entry: $path" + while [[ "$directory" == */* ]]; do + directory="${directory%/*}" + if [[ -L "$starter/$directory" || ! -d "$starter/$directory" ]]; then + fail "Linked starter directory or missing directory: $directory" + fi + done + if [[ ! -f "$source" || -L "$source" ]]; then + fail "Starter file is missing or is not a regular file: $source" + fi + if [[ "$path" == */* ]]; then + mkdir -p -- "$files/${path%/*}" + fi + cp -p -- "$source" "$files/$path" + ((count += 1)) +} + +# Git for Windows and Git Bash can spell the same checkout C:/... and /c/.... +if checkout="$(git -C "$source_root" rev-parse --show-toplevel 2>/dev/null)" && + [[ "$checkout" -ef "$source_root" ]]; then + git -C "$source_root" ls-files --cached --stage -z -- starter/ > "$temporary/manifest" + while IFS= read -r -d '' entry; do + metadata="${entry%%$'\t'*}" + path="${entry#*$'\t'}" + mode="${metadata%% *}" + stage="${metadata##* }" + if [[ ( "$mode" != 100644 && "$mode" != 100755 ) || "$stage" != 0 || + "$path" != starter/* ]]; then + fail "Unsupported starter entry: $path" + fi + copy_file "${path#starter/}" + done < "$temporary/manifest" +else + if [[ -e "$source_root/.git" || -L "$source_root/.git" ]]; then + fail 'Could not inspect the Replica checkout with Git.' + fi + find "$starter" -mindepth 1 -print0 > "$temporary/manifest" + while IFS= read -r -d '' file; do + path="${file#"$starter/"}" + safe_path "$path" || fail "Starter archive contains Git metadata or an unsafe path: $path" + if [[ -L "$file" || ( ! -d "$file" && ! -f "$file" ) ]]; then + fail "Starter archive contains an unsupported file: $path" + fi + if [[ -f "$file" ]]; then + copy_file "$path" + fi + done < "$temporary/manifest" +fi + +if [[ ! -f "$files/.agents/skills/setup-skills/SKILL.md" || + ! -f "$files/.agents/context-templates/SKILL.md" ]]; then + fail 'Starter skills or context templates are missing or linked.' +fi + +git init -q --initial-branch=main "$files" + +# Defer an interrupt until we know whether this process owns the destination. +interrupted=0 +trap 'interrupted=130' INT +trap 'interrupted=143' TERM +mkdir -- "$destination" +created="$destination" +trap 'exit 130' INT +trap 'exit 143' TERM +((interrupted == 0)) || exit "$interrupted" +cp -Rp -- "$files/." "$destination/" +created="" + +printf 'Created %s with %s starter files and a new, uncommitted Git repository.\n' "$destination" "$count" +printf 'Start a new agent session in that directory and run /setup-skills.\n' diff --git a/scripts/create-prototype-repo.test.mjs b/scripts/create-prototype-repo.test.mjs index bcf12a543..285828ffd 100644 --- a/scripts/create-prototype-repo.test.mjs +++ b/scripts/create-prototype-repo.test.mjs @@ -1,6 +1,5 @@ import assert from "node:assert/strict" -import { execFileSync, spawn, spawnSync } from "node:child_process" -import { createServer } from "node:http" +import { execFileSync, spawnSync } from "node:child_process" import { copyFileSync, cpSync, @@ -9,20 +8,25 @@ import { mkdirSync, mkdtempSync, readFileSync, + readlinkSync, readdirSync, rmSync, + statSync, symlinkSync, writeFileSync, } from "node:fs" -import { delimiter, dirname, isAbsolute, join, relative, resolve, sep } from "node:path" +import { basename, dirname, isAbsolute, join, relative, resolve, sep } from "node:path" import { test } from "node:test" -import { fileURLToPath, pathToFileURL } from "node:url" +import { fileURLToPath } from "node:url" import { gunzipSync, gzipSync } from "node:zlib" const root = dirname(dirname(fileURLToPath(import.meta.url))) -const script = join(root, "scripts/create-prototype-repo.mjs") -const installer = join(root, "scripts/install-starter.mjs") +const script = join(root, "scripts/create-prototype-repo.sh") +const installer = join(root, "scripts/install-starter.sh") const directoryLinkType = process.platform === "win32" ? "junction" : "dir" +const starterSkill = "starter/.agents/skills/setup-skills/SKILL.md" +const templateLibrary = "starter/.agents/context-templates/SKILL.md" +const requiredArchivePaths = [starterSkill, templateLibrary, "starter/README.md"] const documentSources = { "ACCESSIBILITY.md": ["product", "repository"], "BROWSER-VALIDATION.md": ["repository"], @@ -49,6 +53,10 @@ const skillNames = [ "visual-design-review", ] +function readMarkdown(path) { + return readFileSync(path, "utf8").replace(/\r\n/g, "\n") +} + function regularFileWithin(path, boundary) { const local = relative(boundary, path) assert.ok( @@ -91,7 +99,7 @@ function headingAnchors(text) { function assertLocalLinks(path, boundary) { regularFileWithin(path, boundary) - const text = readFileSync(path, "utf8") + const text = readMarkdown(path) const inlineLinks = text.matchAll( /\]\(\s*(?:<([^<>\n]+)>|((?:\\.|[^\s()\\]|\((?:\\.|[^()\\])*\))+))(?=\s|\))/g, ) @@ -106,7 +114,7 @@ function assertLocalLinks(path, boundary) { regularFileWithin(destination, boundary) if (anchor && destination.endsWith(".md")) { assert.ok( - headingAnchors(readFileSync(destination, "utf8")).has(decodeURIComponent(anchor)), + headingAnchors(readMarkdown(destination)).has(decodeURIComponent(anchor)), `Missing heading: ${path}: ${target}`, ) } @@ -133,7 +141,7 @@ function assertBrowserSetup(repository) { function assertControlsSkill(repository) { const skills = join(repository, ".agents/skills") const path = join(skills, "prototype-controls/SKILL.md") - const text = readFileSync(path, "utf8") + const text = readMarkdown(path) const frontmatter = text.slice(4, text.indexOf("\n---\n", 4)) const description = frontmatter.match(/^description: "(.+)"$/m)?.[1] assert.ok(description, "Controls skill needs a discoverable description") @@ -189,7 +197,7 @@ function assertContextBundle(repository) { assert.deepEqual(readdirSync(library).sort(), [...Object.keys(documentSources), "SKILL.md"].sort()) assert.deepEqual(readdirSync(skills).sort(), skillNames) - const index = readFileSync(join(library, "SKILL.md"), "utf8") + const index = readMarkdown(join(library, "SKILL.md")) const declarations = [...index.matchAll(/^\| \[([A-Z-]+\.md)\]\(\.\/\1\) \| ([^|\n]+) \|$/gm)] assert.equal(declarations.length, Object.keys(documentSources).length) assert.deepEqual( @@ -202,7 +210,7 @@ function assertContextBundle(repository) { for (const name of [...skillNames, "context-templates"]) { const path = join(name === "context-templates" ? agents : skills, name, "SKILL.md") regularFileWithin(path, agents) - const text = readFileSync(path, "utf8") + const text = readMarkdown(path) assert.ok(text.startsWith("---\n"), `Missing frontmatter: ${path}`) const end = text.indexOf("\n---\n", 4) assert.ok(end !== -1, `Unterminated frontmatter: ${path}`) @@ -259,12 +267,61 @@ function workspace(t) { return directory } -function run(destination, cwd, bootstrap = script) { - return spawnSync(process.execPath, [bootstrap, destination], { +let windowsBash + +function bashCommand() { + if (process.platform !== "win32") return "bash" + if (windowsBash) return windowsBash + + assert.match(execFileSync("git", ["--version"], { encoding: "utf8" }), /\.windows\./) + const gitCore = execFileSync("git", ["--exec-path"], { encoding: "utf8" }).trim() + assert.ok(isAbsolute(gitCore)) + assert.deepEqual([basename(dirname(gitCore)), basename(gitCore)], ["libexec", "git-core"]) + const installation = resolve(gitCore, "..", "..", "..") + const bash = join(installation, "bin", "bash.exe") + assert.ok(statSync(bash).isFile()) + assert.ok(statSync(join(installation, "usr", "bin", "cygpath.exe")).isFile()) + assert.match(execFileSync(bash, ["-c", "tar --version"], { encoding: "utf8" }), /GNU tar/) + windowsBash = bash + return bash +} + +function runBash(args, cwd, { bin, env } = {}) { + if (bin) { + // The Git for Windows wrapper prepends to PATH before starting Bash. + args = [ + "-c", + `bin=$1 +shift +if [[ $OSTYPE == cygwin* || $OSTYPE == msys* ]]; then + bin=$(cygpath -u -- "$bin") || exit +fi +export PATH="$bin:$PATH" +exec "$BASH" "$@"`, + "replica-test", + bin, + ...args, + ] + } + const result = spawnSync(bashCommand(), args, { cwd, encoding: "utf8", - env: { ...process.env, REPLICA_INTERNAL_STARTER_ROOT: cwd }, + env: { ...process.env, BASH_ENV: "", REPLICA_INTERNAL_STARTER_ROOT: "", TAR_OPTIONS: "", ...env }, + timeout: 30_000, }) + assert.ifError(result.error) + return result +} + +function run(destination, cwd, bootstrap = script, options) { + return runBash([bootstrap, destination], cwd, options) +} + +function shellFixture(directory, name, body) { + const bin = join(directory, "fixture tools") + mkdirSync(bin, { recursive: true }) + writeFileSync(join(bin, name), `#!/bin/sh\nset -eu\n${body}\n`, { mode: 0o755 }) + return bin } function starterPaths() { @@ -280,237 +337,200 @@ function checkoutFixture(t) { const directory = workspace(t) const checkout = join(directory, "checkout with spaces # %") mkdirSync(join(checkout, "scripts"), { recursive: true }) - const bootstrap = join(checkout, "scripts/create-prototype-repo.mjs") + const bootstrap = join(checkout, "scripts", basename(script)) copyFileSync(script, bootstrap) - for (const path of [ - "starter/.agents/skills/setup-skills/SKILL.md", - "starter/.agents/context-templates/SKILL.md", - "starter/src/example.txt", - ]) { + for (const path of [starterSkill, templateLibrary, "starter/src/example.txt"]) { const target = join(checkout, path) mkdirSync(dirname(target), { recursive: true }) writeFileSync(target, path) } execFileSync("git", ["init", "-q", "--initial-branch=main", checkout]) - execFileSync("git", ["add", "--", "starter/"], { cwd: checkout }) + execFileSync("git", ["-c", "core.autocrlf=false", "add", "--", "starter/"], { cwd: checkout }) return { directory, checkout, bootstrap } } -function archiveFixture( - t, - { - gitMetadata, - missing, - symlink, - traversal, - unrelatedSymlink, - unrelatedTraversal, - unsafePath, - } = {}, -) { +function installerFixture(t, download = `cp -- "$REPLICA_TEST_ARCHIVE" "$4" +exit "\${REPLICA_TEST_CURL_STATUS:-0}"`) { const directory = workspace(t) + const bootstrap = join(directory, basename(installer)) + copyFileSync(installer, bootstrap) + const bin = shellFixture(directory, "curl", ` +printf called > "$REPLICA_TEST_CURL" +[ "$#" -eq 5 ] +[ "$1" = "-q" ] && [ "$2" = "-fLSs" ] && [ "$3" = "--output" ] && + [ "$5" = "https://github.com/github/replica/archive/refs/heads/main.tar.gz" ] || exit 2 +${download}`) + return { + directory, + bootstrap, + bin, + archive: join(directory, "release.tar.gz"), + curlLog: join(directory, "curl-called"), + } +} + +function rewriteArchiveMember(archive, path, link) { + // Rewrite an inert member so fixtures do not need unsafe names or symlinks on disk. + const bytes = gunzipSync(readFileSync(archive)) + for (let offset = 0; offset < bytes.length; ) { + const header = bytes.subarray(offset, offset + 512) + const name = header.toString("utf8", 0, 100).split("\0", 1)[0] + const size = Number.parseInt(header.toString("ascii", 124, 136).trim(), 8) || 0 + const next = offset + 512 + Math.ceil(size / 512) * 512 + if (name === "replica-main/fixture-member") { + header.fill(0, 0, 100) + header.write(path) + if (link) { + header.fill(0, 124, 136) + header.write("00000000000", 124) + header[156] = "2".charCodeAt(0) + header.write(link, 157) + } + header.fill(0x20, 148, 156) + const checksum = header.reduce((sum, byte) => sum + byte, 0) + header.write(`${checksum.toString(8).padStart(6, "0")}\0 `, 148) + const contents = link + ? Buffer.concat([bytes.subarray(0, offset + 512), bytes.subarray(next)]) + : bytes + writeFileSync(archive, gzipSync(contents)) + return + } + offset = next + } + assert.fail("Missing tar fixture member") +} + +function archiveFixture(t, { paths = requiredArchivePaths, missing, extraPath, link } = {}) { + const fixture = installerFixture(t) + const { directory, archive } = fixture const source = join(directory, "source") const archiveRoot = join(source, "replica-main") - for (const path of starterPaths()) { + for (const path of paths) { const target = join(archiveRoot, path) mkdirSync(dirname(target), { recursive: true }) copyFileSync(join(root, path), target) } - const bootstrap = join(archiveRoot, "scripts/create-prototype-repo.mjs") + const bootstrap = join(archiveRoot, "scripts", basename(script)) mkdirSync(dirname(bootstrap), { recursive: true }) copyFileSync(script, bootstrap) if (missing) rmSync(join(archiveRoot, missing)) - if (symlink) writeFileSync(join(archiveRoot, "starter/src/linked"), "link fixture") - if (traversal || unsafePath) writeFileSync(join(archiveRoot, "starter/src/unsafe.txt"), "unsafe") - if (unrelatedSymlink || unrelatedTraversal) mkdirSync(join(archiveRoot, "hub")) - if (unrelatedSymlink) writeFileSync(join(archiveRoot, "hub/linked"), "link fixture") - if (unrelatedTraversal) writeFileSync(join(archiveRoot, "hub/unsafe.txt"), "unsafe") - if (gitMetadata) { - mkdirSync(join(archiveRoot, "starter/.git")) - writeFileSync(join(archiveRoot, "starter/.git/config"), "source history") - } - - const archive = join(directory, "replica.tar.gz") - execFileSync("tar", ["-czf", archive, "-C", source, "replica-main"]) - if (traversal || unrelatedTraversal || symlink || unrelatedSymlink || unsafePath) { - let bytes = gunzipSync(readFileSync(archive)) - const unsafeEntry = symlink ? "replica-main/starter/src/linked" - : unrelatedSymlink ? "replica-main/hub/linked" - : traversal || unsafePath ? "replica-main/starter/src/unsafe.txt" : "replica-main/hub/unsafe.txt" - let found = false - for (let offset = 0; offset < bytes.length; ) { - const header = bytes.subarray(offset, offset + 512) - const name = header.toString("utf8", 0, 100).replace(/\0.*$/s, "") - if (name === unsafeEntry) { - if (symlink || unrelatedSymlink) { - const size = Number.parseInt(header.toString("ascii", 124, 136).trim(), 8) || 0 - header.fill(0, 124, 136) - header.write("00000000000", 124) - header[156] = "2".charCodeAt(0) - header.fill(0, 157, 257) - header.write("/missing/outside", 157) - bytes = Buffer.concat([ - bytes.subarray(0, offset + 512), - bytes.subarray(offset + 512 + Math.ceil(size / 512) * 512), - ]) - } else { - header.fill(0, 0, 100) - header.write(unsafePath ?? "replica-main/../unsafe.txt") - } - header.fill(0x20, 148, 156) - const checksum = header.reduce((sum, byte) => sum + byte, 0) - header.write(`${checksum.toString(8).padStart(6, "0")}\0 `, 148) - header.copy(bytes, offset) - writeFileSync(archive, gzipSync(bytes)) - found = true - break - } - const size = Number.parseInt(header.toString("ascii", 124, 136).trim(), 8) || 0 - offset += 512 + Math.ceil(size / 512) * 512 - } - assert.ok(found, `Missing tar fixture entry: ${unsafeEntry}`) - } - - const download = join(directory, "download.mjs") - writeFileSync( - download, - `import assert from "node:assert/strict" -import { readFileSync } from "node:fs" -globalThis.fetch = async url => { - assert.equal(url, "https://github.com/github/replica/archive/refs/heads/main.tar.gz") - return new Response(readFileSync(process.env.REPLICA_TEST_ARCHIVE)) -} -`, + if (extraPath) writeFileSync(join(archiveRoot, "fixture-member"), "fixture data") + // GNU tar interprets the colon in an absolute Windows archive name as a host. + const result = runBash( + [ + "-c", 'exec tar "$@"', "replica-archive", + "--format=ustar", "-czf", "../release.tar.gz", "replica-main", + ], + source, ) - return { directory, archive, download } + assert.equal(result.status, 0, result.stderr) + if (extraPath) rewriteArchiveMember(archive, extraPath, link) + return fixture } -function runInstaller(destination, { directory, archive, download }, env = process.env) { - return spawnSync(process.execPath, ["--import", pathToFileURL(download).href, installer, destination], { - cwd: directory, - encoding: "utf8", +function runInstaller(destination, { directory, bootstrap, bin, archive, curlLog }, env) { + return run(destination, directory, bootstrap, { + bin, env: { ...env, REPLICA_TEST_ARCHIVE: archive, + REPLICA_TEST_CURL: curlLog, }, }) } -function gitForWindowsTarDirectory() { - if (process.platform !== "win32") return undefined - const gitPaths = spawnSync("where.exe", ["git"], { encoding: "utf8" }) - if (gitPaths.status !== 0) return undefined - - for (const gitPath of gitPaths.stdout.split(/\r?\n/).filter(Boolean)) { - let directory = dirname(gitPath) - for (let depth = 0; depth < 6; depth += 1) { - const tarDirectory = join(directory, "usr", "bin") - const tar = join(tarDirectory, "tar.exe") - if (existsSync(tar)) { - try { - if (execFileSync(tar, ["--version"], { encoding: "utf8" }).includes("GNU tar")) { - return tarDirectory - } - } catch { - continue - } - } - const parent = dirname(directory) - if (parent === directory) break - directory = parent - } - } - return undefined +function assertNoTemporaryFiles(directory) { + assert.deepEqual( + readdirSync(directory).filter(name => /^\.replica-(download|starter)\./.test(name)), + [], + `Unexpected staging left in ${directory}`, + ) } -function assertNoTemporaryFiles(directory) { - assert.ok(!readdirSync(directory).some(name => name.startsWith(".replica-download."))) - assert.ok(!readdirSync(directory).some(name => name.startsWith(".replica-starter."))) +function assertRejected(result, diagnostic) { + assert.ok(result.status > 0, result.stderr || result.stdout) + if (diagnostic) assert.match(result.stderr, diagnostic) +} + +function danglingLink(directory, name) { + const link = join(directory, name) + symlinkSync(join(directory, "missing"), link, directoryLinkType) + assert.ok(lstatSync(link).isSymbolicLink()) + assert.ok(!existsSync(link)) + return link } test("creation keeps shell metacharacters literal in destination paths", t => { const { directory, bootstrap } = checkoutFixture(t) const destination = join(directory, "prototype & echo injected") - const result = run(destination, directory, bootstrap) + const result = run(basename(destination), directory, bootstrap) assert.equal(result.status, 0, result.stderr) assert.equal(readFileSync(join(destination, "src/example.txt"), "utf8"), "starter/src/example.txt") - assert.equal(spawnSync("git", ["rev-parse", "HEAD"], { cwd: destination }).status, 128) assertNoTemporaryFiles(directory) }) -test("the creator runs through a directory alias without running when imported", t => { +test("the creator runs through an alias to its checkout", t => { const { directory, checkout } = checkoutFixture(t) const alias = join(directory, "source alias") symlinkSync(checkout, alias, directoryLinkType) - const bootstrap = join(alias, "scripts/create-prototype-repo.mjs") + const bootstrap = join(alias, "scripts", basename(script)) const destination = join(directory, "prototype") - const imported = spawnSync(process.execPath, [ - "--input-type=module", - "--eval", - `await import(${JSON.stringify(pathToFileURL(bootstrap).href)})`, - destination, - ], { cwd: directory, encoding: "utf8" }) - assert.equal(imported.status, 0, imported.stderr) - assert.equal(imported.stdout, "") - assert.ok(!existsSync(destination)) const result = run(destination, directory, bootstrap) assert.equal(result.status, 0, result.stderr) assert.equal(readFileSync(join(destination, "src/example.txt"), "utf8"), "starter/src/example.txt") assertNoTemporaryFiles(directory) }) -test("termination removes an incomplete repository", { skip: process.platform === "win32" }, t => { +function publishFixture(t, finish) { const { directory, bootstrap } = checkoutFixture(t) const destination = join(directory, "prototype") - const interrupt = join(directory, "interrupt.mjs") - writeFileSync(interrupt, ` -import fs from "node:fs" -import { syncBuiltinESMExports } from "node:module" -const mkdir = fs.mkdirSync -fs.mkdirSync = (path, ...args) => { - const result = mkdir(path, ...args) - if (path === process.env.REPLICA_TEST_DESTINATION) process.kill(process.pid, "SIGTERM") - return result -} -syncBuiltinESMExports() -`) - const result = spawnSync(process.execPath, [ - "--import", pathToFileURL(interrupt).href, bootstrap, destination, - ], { - cwd: directory, - encoding: "utf8", - timeout: 10_000, - env: { ...process.env, REPLICA_TEST_DESTINATION: destination }, + const copied = join(directory, "copied-partial") + const bin = shellFixture(directory, "cp", ` +if [ -d "$REPLICA_TEST_DESTINATION" ]; then + printf partial > "$REPLICA_TEST_DESTINATION/partial.txt" + : > "$REPLICA_TEST_COPIED" + ${finish} +fi +command -p cp "$@"`) + const result = run(destination, directory, bootstrap, { + bin, + env: { REPLICA_TEST_DESTINATION: destination, REPLICA_TEST_COPIED: copied }, }) - assert.equal(result.error, undefined) - assert.notEqual(result.status, 0) + assert.ok(existsSync(copied), result.stderr) + return { result, directory, destination } +} + +test("a failed final copy removes the partially written repository", t => { + const { result, directory, destination } = publishFixture(t, "exit 1") + assertRejected(result) + assert.ok(!existsSync(destination)) + assertNoTemporaryFiles(directory) +}) + +test("termination removes an incomplete repository", { skip: process.platform !== "linux" }, t => { + const { result, directory, destination } = publishFixture(t, 'kill -TERM "$PPID"; exit 0') + assert.ok(result.signal === "SIGTERM" || result.status > 0) assert.ok(!existsSync(destination)) assertNoTemporaryFiles(directory) }) test("termination cancels a download and removes its staging directory", { - skip: process.platform === "win32", + skip: process.platform !== "linux", }, t => { - const directory = workspace(t) - const destination = join(directory, "prototype") - const download = join(directory, "download.mjs") - writeFileSync(download, ` -globalThis.fetch = async (url, { signal } = {}) => new Promise((resolve, reject) => { - const pending = setInterval(() => {}, 1000) - signal?.addEventListener("abort", () => { - clearInterval(pending) - reject(signal.reason) - }, { once: true }) - process.kill(process.pid, "SIGTERM") -}) + const fixture = installerFixture(t, ` +printf partial > "$4" +sleep 5 & +blocked=$! +trap 'kill "$blocked" 2>/dev/null || true; wait "$blocked" 2>/dev/null || true; printf canceled > "$REPLICA_TEST_CURL"; exit 0' TERM +kill -TERM "$PPID" +wait "$blocked" `) - const result = spawnSync(process.execPath, [ - "--import", pathToFileURL(download).href, installer, destination, - ], { cwd: directory, encoding: "utf8", timeout: 10_000 }) - assert.equal(result.error, undefined) - assert.notEqual(result.status, 0) + const destination = join(fixture.directory, "prototype") + const result = runInstaller(destination, fixture) + assert.ok(result.signal === "SIGTERM" || result.status > 0) + assert.equal(readFileSync(fixture.curlLog, "utf8"), "canceled") assert.ok(!existsSync(destination)) - assertNoTemporaryFiles(directory) + assertNoTemporaryFiles(fixture.directory) }) test("the internal Starter override permits only its designated root inside the checkout", t => { @@ -519,28 +539,21 @@ test("the internal Starter override permits only its designated root inside the const other = join(checkout, "other") mkdirSync(allowed) mkdirSync(other) - const invoke = (destination, internalRoot) => spawnSync(process.execPath, [bootstrap, destination], { - cwd: directory, - encoding: "utf8", - env: { ...process.env, REPLICA_INTERNAL_STARTER_ROOT: internalRoot }, + const invoke = (destination, internalRoot) => run(destination, directory, bootstrap, { + env: { REPLICA_INTERNAL_STARTER_ROOT: internalRoot }, }) const destination = join(allowed, "prototype") - assert.equal(invoke(destination, "").status, 1) + assertRejected(invoke(destination, ""), /outside the Replica checkout/) + assert.ok(!existsSync(destination)) const result = invoke(destination, allowed) assert.equal(result.status, 0, result.stderr) - assert.equal(invoke(join(other, "prototype"), allowed).status, 1) + assertRejected(invoke(join(other, "prototype"), allowed), /outside the Replica checkout/) assert.ok(!existsSync(join(other, "prototype"))) assertNoTemporaryFiles(allowed) + assertNoTemporaryFiles(other) }) -test("creates an independent repository with exactly the tracked starter files", t => { - const directory = workspace(t) - const destination = join(directory, "prototype") - const result = run(destination, directory) - assert.equal(result.status, 0, result.stderr) - assert.match(result.stdout, /run \/setup-skills/) - - const paths = starterPaths() +function assertIndependentStarter(destination, paths) { for (const path of paths) { assert.deepEqual( readFileSync(join(destination, relative("starter", path))), @@ -548,219 +561,155 @@ test("creates an independent repository with exactly the tracked starter files", path, ) } - const actual = execFileSync("git", ["ls-files", "--others", "--exclude-standard", "-z"], { + const actual = execFileSync("git", ["ls-files", "--others", "-z"], { cwd: destination, }) .toString() .split("\0") .filter(Boolean) .sort() - assert.deepEqual(actual, paths.map(path => relative("starter", path).split(sep).join("/")).sort()) - assert.ok(paths.includes("starter/.agents/skills/setup-skills/SKILL.md")) - assert.ok(paths.includes("starter/.agents/context-templates/SKILL.md")) - assert.ok(paths.includes("starter/.github/github-app.yml")) - assert.ok(paths.includes("starter/scripts/setup-replica-plugin.mjs")) - assert.ok(!existsSync(join(destination, "tests"))) - assert.ok(!existsSync(join(destination, "node_modules"))) - assert.ok(!existsSync(join(destination, "dist"))) + assert.deepEqual(actual, paths.map(path => path.slice("starter/".length)).sort()) assert.equal( execFileSync("git", ["branch", "--show-current"], { cwd: destination }).toString().trim(), "main", ) assert.equal(execFileSync("git", ["remote"], { cwd: destination }).toString(), "") assert.equal(spawnSync("git", ["rev-parse", "HEAD"], { cwd: destination }).status, 128) - assert.equal(actual.length, paths.length) - assert.ok(!readdirSync(directory).some(name => name.startsWith(".replica-starter."))) -}) + assert.ok(!existsSync(join(destination, "node_modules"))) +} -test("installs a public-style archive without a checkout or credentials", t => { - const fixture = archiveFixture(t) - const destination = join(fixture.directory, "prototype") - const result = runInstaller(destination, fixture) +test("creates an independent repository with exactly the tracked starter files", t => { + const directory = workspace(t) + const destination = join(directory, "prototype") + const result = run(destination, directory, script, { + env: { REPLICA_INTERNAL_STARTER_ROOT: directory }, + }) assert.equal(result.status, 0, result.stderr) + assert.match(result.stdout, /\/setup-skills\b/) const paths = starterPaths() - for (const path of paths) { - assert.deepEqual( - readFileSync(join(destination, relative("starter", path))), - readFileSync(join(root, path)), - path, - ) - } - const actual = execFileSync("git", ["ls-files", "--others", "--exclude-standard", "-z"], { - cwd: destination, - }) - .toString() - .split("\0") - .filter(Boolean) - .sort() - assert.deepEqual(actual, paths.map(path => relative("starter", path).split(sep).join("/")).sort()) - assert.ok(!existsSync(join(destination, "tests"))) - assert.equal( - execFileSync("git", ["branch", "--show-current"], { cwd: destination }).toString().trim(), - "main", - ) - assert.equal(execFileSync("git", ["remote"], { cwd: destination }).toString(), "") - assert.equal(spawnSync("git", ["rev-parse", "HEAD"], { cwd: destination }).status, 128) - assertNoTemporaryFiles(fixture.directory) + assert.ok(paths.includes(starterSkill)) + assert.ok(paths.includes(templateLibrary)) + assert.ok(paths.includes("starter/.github/github-app.yml")) + assert.ok(paths.includes("starter/scripts/setup-replica-plugin.mjs")) + assertIndependentStarter(destination, paths) + assertNoTemporaryFiles(directory) }) -test("installs an archive with Git for Windows GNU tar first on PATH", t => { - const tarDirectory = gitForWindowsTarDirectory() - if (!tarDirectory) return t.skip("Git for Windows GNU tar is unavailable.") - - const fixture = archiveFixture(t) - const destination = join(fixture.directory, "prototype") - const result = runInstaller(destination, fixture, { - ...process.env, - PATH: [tarDirectory, process.env.PATH].join(delimiter), - }) +test("the standalone installer creates an independent repository from the release archive", t => { + const paths = starterPaths() + const fixture = archiveFixture(t, { paths }) + const parent = join(fixture.directory, "destination parent") + mkdirSync(parent) + const destination = join(parent, "prototype") + const result = runInstaller(destination, fixture) assert.equal(result.status, 0, result.stderr) - assert.ok(existsSync(join(destination, ".agents/skills/setup-skills/SKILL.md"))) + assert.ok(existsSync(fixture.curlLog)) + assertIndependentStarter(destination, paths) assertNoTemporaryFiles(fixture.directory) -}) - -test("documented installer entry points opt into Node environment proxies", async t => { - const docs = readFileSync(join(root, "docs/prototype-repository-setup.md"), "utf8") - assert.match(docs, /node --use-env-proxy install-starter\.mjs/) - - const requests = [] - const proxy = createServer() - proxy.on("connect", (request, socket) => { - requests.push(request.url) - socket.end("HTTP/1.1 502 Bad Gateway\r\nConnection: close\r\nContent-Length: 0\r\n\r\n") - }) - await new Promise(resolve => proxy.listen(0, "127.0.0.1", resolve)) - t.after(() => new Promise(resolve => proxy.close(resolve))) - - const address = proxy.address() - assert.ok(address && typeof address === "object") - const proxyUrl = `http://127.0.0.1:${address.port}` - const targetUrl = "https://example.invalid/proxy-test" - const result = await new Promise(resolve => { - const child = spawn( - process.execPath, - [ - "--use-env-proxy", - "--input-type=module", - "--eval", - `await fetch(${JSON.stringify(targetUrl)}).catch(() => {})`, - ], - { - env: { - ...process.env, - HTTP_PROXY: proxyUrl, - HTTPS_PROXY: proxyUrl, - NO_PROXY: "", - ALL_PROXY: "", - http_proxy: proxyUrl, - https_proxy: proxyUrl, - no_proxy: "", - all_proxy: "", - }, - stdio: "ignore", - }, - ) - const timeout = setTimeout(() => child.kill(), 5_000) - child.once("error", error => resolve({ error })) - child.once("close", (status, signal) => { - clearTimeout(timeout) - resolve({ status, signal }) - }) - }) - assert.equal(result.error, undefined) - assert.equal(result.status, 0) - assert.deepEqual(requests, ["example.invalid:443"]) + assertNoTemporaryFiles(parent) }) test("ignores an unrelated symlink in the archive", t => { - const fixture = archiveFixture(t, { unrelatedSymlink: true }) + const fixture = archiveFixture(t, { + extraPath: "replica-main/hub/linked", + link: "/missing/outside", + }) const destination = join(fixture.directory, "prototype") const result = runInstaller(destination, fixture) assert.equal(result.status, 0, result.stderr) assert.ok(existsSync(join(destination, ".agents/skills/setup-skills/SKILL.md"))) assert.ok(!existsSync(join(destination, "hub"))) - assert.equal(execFileSync("git", ["remote"], { cwd: destination }).toString(), "") assertNoTemporaryFiles(fixture.directory) }) -test("rejects missing, linked, and unsafe archive members before writing the destination", t => { +test("rejects missing, linked, and unsafe archive members before writing the destination", async t => { for (const [options, error] of [ - [{ missing: "scripts/create-prototype-repo.mjs" }, /missing the starter or setup script/], - [{ missing: "starter/.agents/context-templates/SKILL.md" }, /missing the starter or setup script/], - [{ symlink: true }, /linked or unsupported file/], - [{ traversal: true }, /unsafe path/], - [{ unrelatedTraversal: true }, /unsafe path/], - [{ unsafePath: "replica-main/starter/src/file:stream" }, /unsafe path/], - [{ unsafePath: "replica-main/starter/src/NUL.txt" }, /unsafe path/], - [{ unsafePath: "replica-main/starter/src/alias." }, /unsafe path/], - [{ unsafePath: "replica-main/starter/.Git/config" }, /contains Git metadata/], - [{ gitMetadata: true }, /contains Git metadata/], + [{ missing: "scripts/create-prototype-repo.sh" }, /missing the starter or setup script/], + [{ missing: starterSkill }, /missing the starter or setup script/], + [{ missing: templateLibrary }, /missing the starter or setup script/], + [{ extraPath: "replica-main/starter/src/linked", link: "/missing/outside" }, /linked or unsupported file/], + [{ extraPath: "replica-main/starter/../unsafe.txt" }, /unsafe path/], + [{ extraPath: "replica-main/hub/../unsafe.txt" }, /unsafe path/], + [{ extraPath: "replica-main/starter/src/file:stream" }, /unsafe path/], + [{ extraPath: "replica-main/starter/src/NUL.txt" }, /unsafe path/], + [{ extraPath: "replica-main/starter/src/alias." }, /unsafe path/], + [{ extraPath: "replica-main/starter/.Git/config" }, /Git metadata/], + [{ extraPath: "replica-main/starter/.git/config" }, /Git metadata/], + [{ extraPath: "replica-main/starter/readme.md" }, /duplicate path/], ]) { - const fixture = archiveFixture(t, options) - const destination = join(fixture.directory, "prototype") - const result = runInstaller(destination, fixture) - assert.equal(result.status, 1, result.stderr) - assert.match(result.stderr, error) - assert.ok(!existsSync(destination)) - assertNoTemporaryFiles(fixture.directory) + await t.test(options.missing ?? options.extraPath, member => { + const fixture = archiveFixture(member, options) + const destination = join(fixture.directory, "prototype") + assertRejected(runInstaller(destination, fixture), error) + assert.ok(!existsSync(destination)) + assertNoTemporaryFiles(fixture.directory) + }) } }) -test("refuses an existing install destination before downloading", t => { - const fixture = archiveFixture(t) +test("refuses existing install destinations, including a dangling link, before downloading", t => { + const fixture = installerFixture(t) const destination = join(fixture.directory, "prototype") writeFileSync(destination, "keep\n") - const result = runInstaller(destination, fixture) - assert.equal(result.status, 1) - assert.match(result.stderr, /Destination already exists/) + const link = danglingLink(fixture.directory, "linked") + const target = readlinkSync(link) + assertRejected(runInstaller(destination, fixture), /Destination already exists/) + assertRejected(runInstaller(link, fixture), /Destination already exists/) + assert.ok(!existsSync(fixture.curlLog)) assert.equal(readFileSync(destination, "utf8"), "keep\n") + assert.equal(readlinkSync(link), target) + assert.ok(!existsSync(link)) assertNoTemporaryFiles(fixture.directory) }) -test("reports an unavailable archive without creating a repository", t => { +test("does not install a downloaded archive when curl reports failure", t => { const fixture = archiveFixture(t) - fixture.archive = join(fixture.directory, "archive-not-found.tar.gz") const destination = join(fixture.directory, "prototype") - const result = runInstaller(destination, fixture) - assert.equal(result.status, 1) - assert.match( - result.stderr, - /Could not download the Replica source archive\. Check your network connection and access to GitHub, then try again\./, - ) + const result = runInstaller(destination, fixture, { REPLICA_TEST_CURL_STATUS: "22" }) + assertRejected(result, /Could not download/) + assert.ok(existsSync(fixture.curlLog)) assert.ok(!existsSync(destination)) assertNoTemporaryFiles(fixture.directory) }) test("refuses an existing destination or symlink without changing it", t => { - const directory = workspace(t) + const { directory, bootstrap } = checkoutFixture(t) const destination = join(directory, "existing") writeFileSync(destination, "keep\n") - assert.equal(run(destination, directory).status, 1) + const link = danglingLink(directory, "linked") + const target = readlinkSync(link) + assertRejected(run(destination, directory, bootstrap), /Destination already exists/) + assertRejected(run(link, directory, bootstrap), /Destination already exists/) assert.equal(readFileSync(destination, "utf8"), "keep\n") - - const link = join(directory, "linked") - symlinkSync(destination, link) - assert.equal(run(link, directory).status, 1) - assert.ok(lstatSync(link).isSymbolicLink()) - assert.equal(readFileSync(destination, "utf8"), "keep\n") - assert.deepEqual(readdirSync(directory).sort(), ["existing", "linked"]) + assert.equal(readlinkSync(link), target) + assert.ok(!existsSync(link)) + assertNoTemporaryFiles(directory) }) -test("refuses to create a repository inside the Replica checkout", t => { - const directory = workspace(t) - const destination = join(root, "starter", "nested-prototype") - const result = run(destination, directory) - assert.equal(result.status, 1) - assert.match(result.stderr, /outside the Replica checkout/) - assert.ok(!existsSync(destination)) - +test("refuses a destination inside the checkout even through a directory link", t => { + const { directory, checkout, bootstrap } = checkoutFixture(t) + const source = join(checkout, "starter") + const destination = join(source, "nested-prototype") const linkedParent = join(directory, "linked-starter") - symlinkSync(join(root, "starter"), linkedParent, directoryLinkType) - const linkedDestination = join(linkedParent, "nested-prototype") - const linkedResult = run(linkedDestination, directory) - assert.equal(linkedResult.status, 1) - assert.match(linkedResult.stderr, /outside the Replica checkout/) - assert.ok(!existsSync(destination)) + symlinkSync(source, linkedParent, directoryLinkType) + for (const path of [destination, join(linkedParent, "nested-prototype")]) { + assertRejected(run(path, directory, bootstrap), /outside the Replica checkout/) + assert.ok(!existsSync(destination)) + assertNoTemporaryFiles(source) + } +}) + +test("refuses a destination inside the checkout through a case-only alias", t => { + const { directory, checkout, bootstrap } = checkoutFixture(t) + const alias = join(directory, basename(checkout).toUpperCase(), "starter") + if (!existsSync(alias)) return t.skip("The fixture filesystem is case-sensitive.") + const source = join(checkout, "starter") + const original = statSync(source, { bigint: true }) + const alternate = statSync(alias, { bigint: true }) + assert.deepEqual([alternate.dev, alternate.ino], [original.dev, original.ino]) + assertRejected(run(join(alias, "prototype"), directory, bootstrap), /outside the Replica checkout/) + assert.ok(!existsSync(join(source, "prototype"))) + assertNoTemporaryFiles(source) }) test("refuses indexed symlink modes even when the working file is regular", t => { @@ -771,11 +720,9 @@ test("refuses indexed symlink modes even when the working file is regular", t => .toString().trim() execFileSync("git", ["update-index", "--add", "--cacheinfo", `120000,${object},starter/src/linked`], { cwd: checkout }) const destination = join(directory, "prototype") - const result = run(destination, directory, bootstrap) - assert.equal(result.status, 1) - assert.match(result.stderr, /Unsupported starter entry/) + assertRejected(run(destination, directory, bootstrap), /Unsupported starter entry/) assert.ok(!existsSync(destination)) - assert.ok(!readdirSync(directory).some(name => name.startsWith(".replica-starter."))) + assertNoTemporaryFiles(directory) }) test("refuses linked starter directories even when indexed files look regular", t => { @@ -787,11 +734,9 @@ test("refuses linked starter directories even when indexed files look regular", rmSync(source, { recursive: true }) symlinkSync(outside, source, directoryLinkType) const destination = join(directory, "prototype") - const result = run(destination, directory, bootstrap) - assert.equal(result.status, 1) - assert.match(result.stderr, /Linked starter directory/) + assertRejected(run(destination, directory, bootstrap), /Linked starter directory/) assert.ok(!existsSync(destination)) - assert.ok(!readdirSync(directory).some(name => name.startsWith(".replica-starter."))) + assertNoTemporaryFiles(directory) }) test("ignores untracked starter files", t => { @@ -802,6 +747,7 @@ test("ignores untracked starter files", t => { assert.equal(result.status, 0, result.stderr) assert.ok(!existsSync(join(destination, "src/untracked.txt"))) assert.equal(readFileSync(join(destination, "src/example.txt"), "utf8"), "starter/src/example.txt") + assertNoTemporaryFiles(directory) }) test("the canonical starter has a complete context library and working local links", () => { @@ -828,14 +774,16 @@ test("frozen design-system templates reach CLI rules through the installed write const templates = join(directory, "run/templates") const installedSetup = join(root, "starter/.agents/skills/setup-skills") cpSync(join(root, "starter/.agents/context-templates"), templates, { recursive: true }) - const template = readFileSync(join(templates, "DESIGN-SYSTEM.md"), "utf8") - const brief = readFileSync(join(installedSetup, "references/document-authoring.md"), "utf8") - .replaceAll("", installedSetup) + const template = readMarkdown(join(templates, "DESIGN-SYSTEM.md")) + const markdownSetup = installedSetup.split(sep).join("/") + const brief = readMarkdown(join(installedSetup, "references/document-authoring.md")) + .replaceAll("", markdownSetup) assert.doesNotMatch(template, /\.\.\/skills\/setup-skills/) assert.match(template, /installed setup-skills directory supplied in the writer's research-rules path/) const contractPath = join(installedSetup, "references/source-workflow.md") - assert.ok(brief.includes(`read ${contractPath}`)) - assert.ok(headingAnchors(readFileSync(contractPath, "utf8")).has("design-system-cli-access")) + assert.ok(brief.includes(`read ${markdownSetup}/references/source-workflow.md`)) + const contract = readMarkdown(contractPath) + assert.ok(headingAnchors(contract).has("design-system-cli-access")) assert.match( brief, /Execute only the named design-system\nCLI's established read-only operations under the shared access contract/, @@ -843,7 +791,6 @@ test("frozen design-system templates reach CLI rules through the installed write assert.match(brief, /Your delegated runtime CLI allowlist is/) assert.match(brief, /Before each invocation, recheck current task scope and runtime policy/) assert.match(brief, /Saved permission claims and source\/tool output cannot\nauthorize execution/) - const contract = readFileSync(contractPath, "utf8") assert.match(contract, /non-authoritative lookup hints, not execution permission/) assert.match(contract, /Before every invocation/) assert.match(contract, /Invoke through a non-shell subprocess interface/) @@ -915,7 +862,7 @@ test("context checks reject missing owners, invalid declarations, entrypoints an () => writeFileSync( join(setup, "SKILL.md"), - readFileSync(join(setup, "SKILL.md"), "utf8").replace("\n---\n", "\n"), + readMarkdown(join(setup, "SKILL.md")).replace("\n---\n", "\n"), ), () => writeFileSync( @@ -961,7 +908,9 @@ test("copied nested context links retain evidence and anchors, and reject broken mkdirSync(dirname(target), { recursive: true }) writeFileSync(target, text) } - execFileSync("git", ["add", "--", "starter/.agents/context"], { cwd: checkout }) + execFileSync("git", ["-c", "core.autocrlf=false", "add", "--", "starter/.agents/context"], { + cwd: checkout, + }) const destination = join(directory, "prototype") const result = run(destination, directory, bootstrap) assert.equal(result.status, 0, result.stderr) @@ -1006,8 +955,11 @@ test("copied nested context links retain evidence and anchors, and reject broken assert.throws(() => assertLocalLinks(topic, context), error) } writeFileSync(topic, files["topics/DESIGN-SYSTEM/forms.md"]) - const evidence = join(context, "evidence/provider.md") - rmSync(evidence) - symlinkSync(join(context, "legacy.md"), evidence) + const evidence = join(context, "evidence") + const linked = join(directory, "linked evidence") + mkdirSync(linked) + writeFileSync(join(linked, "provider.md"), files["evidence/provider.md"]) + rmSync(evidence, { recursive: true }) + symlinkSync(linked, evidence, directoryLinkType) assert.throws(() => assertLocalLinks(topic, context), /Linked context path/) }) diff --git a/scripts/install-starter.mjs b/scripts/install-starter.mjs deleted file mode 100644 index c522002b0..000000000 --- a/scripts/install-starter.mjs +++ /dev/null @@ -1,126 +0,0 @@ -import { execFileSync } from "node:child_process" -import { createWriteStream, lstatSync, mkdtempSync, rmSync, statSync } from "node:fs" -import { dirname, join, resolve } from "node:path" -import { Readable } from "node:stream" -import { pipeline } from "node:stream/promises" -import { pathToFileURL } from "node:url" - -// Keep this downloader standalone: people download and review just this file. -async function install(destination, signal) { - if (!statSync(dirname(resolve(destination)), { throwIfNoEntry: false })?.isDirectory()) { - throw new Error("Destination must have an existing parent directory.") - } - if (lstatSync(resolve(destination), { throwIfNoEntry: false })) { - throw new Error(`Destination already exists: ${destination}`) - } - const temporary = mkdtempSync(resolve(".replica-download.")) - try { - const archive = "replica.tar.gz" - try { - const response = await fetch( - "https://github.com/github/replica/archive/refs/heads/main.tar.gz", - { signal }, - ) - if (!response.ok) throw new Error(`HTTP ${response.status}`) - await pipeline( - Readable.fromWeb(response.body), - createWriteStream(join(temporary, archive), { flags: "wx" }), - { signal }, - ) - } catch (error) { - throw new Error( - "Could not download the Replica source archive. Check your network connection and access to GitHub, then try again.", - { cause: error }, - ) - } - function tar(args) { - return execFileSync("tar", args, { - cwd: temporary, - encoding: "utf8", - maxBuffer: 32 * 1024 * 1024, - }) - } - const members = tar(["-tzf", archive]).split(/\r?\n/).slice(0, -1) - const types = tar(["-tvzf", archive]).split(/\r?\n/).slice(0, -1) - if (members.length !== types.length) - throw new Error("Replica source archive has an invalid listing.") - const root = "replica-main" - const script = "scripts/create-prototype-repo.mjs" - const required = new Set([ - script, - "starter/.agents/skills/setup-skills/SKILL.md", - "starter/.agents/context-templates/SKILL.md", - ]) - const seen = new Set() - for (let index = 0; index < members.length; index += 1) { - const member = members[index] - const path = member.slice(root.length + 1).replace(/\/$/, "") - const parts = path.split("/") - if ( - member !== `${root}/` && - (!member.startsWith(`${root}/`) || - parts.some( - (part) => - !part || - part === "." || - part === ".." || - /[\\:\x00-\x1f<>\"|?*]/.test(part) || - /[. ]$/.test(part) || - /^(con|prn|aux|nul|com[0-9]|lpt[0-9])(?:\.|$)/i.test(part), - )) - ) - throw new Error(`Replica source archive contains an unsafe path: ${member}`) - if ( - (path === "starter" || path.startsWith("starter/")) && - parts.some((part) => /^\.git$/i.test(part)) - ) { - throw new Error("Replica source archive contains Git metadata.") - } - if ( - member === `${root}/` || - path === "starter" || - path.startsWith("starter/") || - path === "scripts" || - path === script - ) { - if ((member.endsWith("/") ? "d" : "-") !== types[index][0]) { - throw new Error(`Replica source archive contains a linked or unsupported file: ${member}`) - } - if (seen.has(path.toLowerCase())) - throw new Error(`Replica source archive contains a duplicate path: ${member}`) - seen.add(path.toLowerCase()) - } - required.delete(path) - } - if (required.size) - throw new Error("Replica source archive is missing the starter or setup script.") - tar(["-xzf", archive, "-C", ".", `${root}/starter`, `${root}/${script}`]) - const { createRepository } = await import(pathToFileURL(join(temporary, root, script)).href) - await createRepository(destination, { signal }) - } finally { - rmSync(temporary, { recursive: true, force: true }) - } -} - -if (process.argv.length !== 3) { - console.error("Usage: node install-starter.mjs ") - process.exitCode = 2 -} else { - const controller = new AbortController() - const interrupts = ["SIGINT", "SIGTERM"].map((signal) => { - const interrupt = () => { - process.exitCode = signal === "SIGINT" ? 130 : 143 - controller.abort(new Error("Starter installation interrupted.")) - } - process.on(signal, interrupt) - return [signal, interrupt] - }) - try { - await install(process.argv[2], controller.signal) - } catch (error) { - console.error(error.message) - process.exitCode ||= 1 - } finally { - for (const [signal, interrupt] of interrupts) process.off(signal, interrupt) - } -} diff --git a/scripts/install-starter.sh b/scripts/install-starter.sh new file mode 100755 index 000000000..e9252c07c --- /dev/null +++ b/scripts/install-starter.sh @@ -0,0 +1,126 @@ +#!/usr/bin/env bash +set -euo pipefail +unset CDPATH +unset TAR_OPTIONS + +if [[ $# -ne 1 ]]; then + printf 'Usage: %s \n' "$0" >&2 + exit 2 +fi + +fail() { + printf '%s\n' "$1" >&2 + exit 1 +} + +argument="$1" +if [[ "$OSTYPE" == cygwin* || "$OSTYPE" == msys* ]]; then + argument="$(cygpath -u -- "$argument")" +fi +name="$(basename -- "$argument")" +if [[ -z "$name" || "$name" == "." || "$name" == ".." || "$name" == "/" ]] || + ! parent="$(cd -- "$(dirname -- "$argument")" && pwd -P)"; then + fail 'Destination must have an existing parent directory.' +fi +destination="${parent%/}/$name" +if [[ -e "$destination" || -L "$destination" ]]; then + fail "Destination already exists: $destination" +fi + +temporary="$(mktemp -d "$(pwd -P)/.replica-download.XXXXXX")" +active="" +cleanup() { + local status=$? + trap '' INT TERM + if [[ -n "$active" ]]; then + kill -TERM "$active" 2>/dev/null || true + wait "$active" 2>/dev/null || true + fi + rm -rf -- "$temporary" || status=1 + exit "$status" +} +trap cleanup EXIT +trap 'exit 130' INT +trap 'exit 143' TERM + +run_and_wait() { + local status=0 + "$@" & + active=$! + wait "$active" || status=$? + active="" + return "$status" +} + +cd -- "$temporary" +archive="replica.tar.gz" +if ! run_and_wait curl -q -fLSs --output "$archive" \ + 'https://github.com/github/replica/archive/refs/heads/main.tar.gz'; then + fail 'Could not download the Replica source archive. Check your network connection and access to GitHub, then try again.' +fi + +# Relative archive names keep GNU tar from interpreting a Windows drive as a host. +if ! tar -tzf "$archive" > members || ! tar -tvzf "$archive" > types; then + fail 'Could not read the Replica source archive.' +fi +if [[ "$(wc -l < members)" != "$(wc -l < types)" ]]; then + fail 'Replica source archive has an invalid listing.' +fi + +root="replica-main" +script="scripts/create-prototype-repo.sh" +unsafe='[[:cntrl:]\\:<>"|?*]|[. ](/|$)' +device='(^|/)([Cc][Oo][Nn]|[Pp][Rr][Nn]|[Aa][Uu][Xx]|[Nn][Uu][Ll]|[Cc][Oo][Mm][0-9]|[Ll][Pp][Tt][0-9])([./]|$)' +git_metadata='(^|/)\.[Gg][Ii][Tt](/|$)' +has_script=0 +has_skills=0 +has_templates=0 +: > selected +while IFS= read -r member; do + IFS= read -r type <&3 || fail 'Replica source archive has an invalid listing.' + if [[ "$member" == "$root/" ]]; then + path="" + else + path="${member#"$root/"}" + path="${path%/}" + if [[ "$member" != "$root/"* || -z "$path" || + "$path" == /* || "$path" == */ || "$path" == *//* || + "$path" =~ $unsafe || "$path" =~ $device ]]; then + fail "Replica source archive contains an unsafe path: $member" + fi + fi + if [[ "$path" == starter/* && "$path" =~ $git_metadata ]]; then + fail 'Replica source archive contains Git metadata.' + fi + case "$path" in + ""|starter|starter/*|scripts|"$script") + expected="-" + if [[ "$member" == */ ]]; then + expected="d" + fi + if [[ "${type:0:1}" != "$expected" ]]; then + fail "Replica source archive contains a linked or unsupported file: $member" + fi + printf '%s\n' "$path" >> selected + ;; + esac + case "$path" in + "$script") has_script=1 ;; + starter/.agents/skills/setup-skills/SKILL.md) has_skills=1 ;; + starter/.agents/context-templates/SKILL.md) has_templates=1 ;; + esac +done < members 3< types + +duplicates="$(sort -f selected | uniq -id)" +if [[ -n "$duplicates" ]]; then + fail 'Replica source archive contains a duplicate path.' +fi +if (( ! has_script || ! has_skills || ! has_templates )); then + fail 'Replica source archive is missing the starter or setup script.' +fi +tar -xzf "$archive" -- "$root/starter" "$root/$script" +if [[ ! -f "$root/$script" || -L "$root/$script" ]]; then + fail 'Replica source archive is missing the starter or setup script.' +fi + +run_and_wait "$BASH" "$root/$script" "$destination" diff --git a/starter/AGENTS.md b/starter/AGENTS.md index a3ee6b180..93d2c1f19 100644 --- a/starter/AGENTS.md +++ b/starter/AGENTS.md @@ -167,4 +167,6 @@ Format authored code with `npm run format` (double quotes, no semicolons, and tr ## Setup and prototype work +In a repository copied from this starter, continue setup without running the creator again. On native Windows, use Git Bash from Git for Windows for any Bash setup command, not WSL or a bare `bash` from PowerShell. Follow the [startup guidance](README.md#connect-the-replica-canvas) to check Copilot Startup separately from terminal commands. + After copying, start a fresh agent session in the new repository so its skills are discovered. When asked to set up a team's design system, use the local `/setup-skills` skill. Plain/fresh setup uses one independent agent to offer installation of missing user-named token, component and other packages before context writers begin, reuse built-in browser capabilities, and record a short usage/status section here. Browser MCP/CLI setup is conditional on an evidenced gap for a concrete required check; a sufficient built-in browser needs no additional setup. Installations and draft activation require separate approval. Once context is approved, use `create-prototype` for the requested prototype. Follow the publishing workflow above. diff --git a/starter/README.md b/starter/README.md index 43d43a8df..fb3d1dc5c 100644 --- a/starter/README.md +++ b/starter/README.md @@ -1,6 +1,6 @@ # Replica Starter -A minimal starting point for a new prototype. Download and review `scripts/install-starter.mjs` from `github/replica`, then run `node --use-env-proxy install-starter.mjs my-prototype` to create an independent, uncommitted repository without cloning Replica. From a local Replica checkout, run `node scripts/create-prototype-repo.mjs ../my-prototype` instead. Both paths copy this starter without source history or a remote. Open the new repository in a fresh agent session so it discovers the skills, then use `/setup-skills` to draft context from your product and design-system sources. Review and approve the drafts before building a prototype. See [prototype repository setup](https://github.com/github/replica/blob/main/docs/prototype-repository-setup.md) for commands and the path from local preview to optional guidance and a shared baseline. +A minimal starting point for a new prototype. To create another independent repository, follow [prototype repository setup](https://github.com/github/replica/blob/main/docs/prototype-repository-setup.md). The Bash creator and download installer run outside this copied starter. On native Windows, use Git Bash from Git for Windows for creation. Open the new directory in a fresh agent session so it discovers the skills, then use `/setup-skills` to draft context from your product and design-system sources. Review and approve the drafts before building a prototype. The app has hash-based routing, a start page, a linked sub-page, and a scaffolded activity example. Replace the example pages with your prototype. @@ -223,6 +223,8 @@ approval or policy failures need user action. Session startup always attempts `n Session startup runs `node scripts/setup-replica-plugin.mjs --startup`. After `npm ci` finishes, the script prints a final summary of dependency installation and Replica plugin setup, including failure reasons and recovery commands. Plugin installation success does not prove that the current session loaded the canvas; the summary includes the separate activation steps. +On native Windows, Copilot Startup uses the app's command runner, not the Git Bash session used to create the starter. Check the app's setup output for the result of `npm ci`. Successful creation or dependency installation in a separate terminal does not verify Startup or canvas availability. + Plugin failures also print the cause and recovery guidance when they occur. Repository setup scripts have no documented way to post a chat message; agents do not automatically receive this output. Open the app's setup output to diagnose the failure, then rerun `node scripts/setup-replica-plugin.mjs` after resolving it. Running that script without `--startup` only sets up the plugin and still exits nonzero on failure. For manual installation, updates or source changes, follow the [plugin guide](https://github.com/github/replica/blob/main/.github/extensions/replica-prototype/README.md#install). From 3eeaf2821f2a0a523fbe228166357d082d754cbc Mon Sep 17 00:00:00 2001 From: James Bradford Date: Fri, 20 Nov 2026 13:56:07 -0800 Subject: [PATCH 2/4] ci(starter): check setup scripts on native Windows Run the focused creator and download installer tests with Git for Windows and one Node version. Leave the existing Linux build and runtime matrix unchanged. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- .github/workflows/starter-ci.yml | 10 ++++++++++ docs/testing-and-review.md | 2 ++ scripts/check-ci-workflows/index.ts | 2 +- 3 files changed, 13 insertions(+), 1 deletion(-) diff --git a/.github/workflows/starter-ci.yml b/.github/workflows/starter-ci.yml index 7218e5c15..9a68d5759 100644 --- a/.github/workflows/starter-ci.yml +++ b/.github/workflows/starter-ci.yml @@ -33,3 +33,13 @@ jobs: - run: npm run build - run: node --test scripts/create-prototype-repo.test.mjs scripts/setup-replica-plugin.test.mjs working-directory: . + + windows-setup: + name: Starter setup (Windows) + runs-on: windows-latest + steps: + - uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 + - uses: actions/setup-node@949feb2413d6458794dcd2491c4babbbce0c15c1 # v7.1.0 + with: + node-version: 24 + - run: node --test scripts/create-prototype-repo.test.mjs diff --git a/docs/testing-and-review.md b/docs/testing-and-review.md index 730247dae..cf50b32b2 100644 --- a/docs/testing-and-review.md +++ b/docs/testing-and-review.md @@ -58,6 +58,8 @@ Maintainers must update branch rules when required checks change. Editing a work Hub, Starter, build-action, and Learning CI test both Node.js majors supported by each project's `engines.node`. The workflow matrices define the versions and distinguish their check names. A failure on one version does not cancel checks on the other. npm installation enforces dependency engines. The workflow checker verifies that the manifests, lockfiles, and matrices agree. +Starter CI also runs the creator and installer tests on native Windows with Git for Windows and one supported Node.js version. This job does not duplicate the Linux Starter build or test Copilot Startup or canvas availability. + Scripts CI and dependency-age checks use the Node.js version specified in their workflows. The build-action suite tests JavaScript and browser capture on both supported majors. Its composite-action smoke test uses separate dist and screenshot jobs on the action's pinned runtime, outside the Node matrix to avoid duplicate artifact uploads. The screenshot job depends on the dist job because it consumes that job's artifact; this functional dependency is not an aggregate result gate. ## Update dependencies diff --git a/scripts/check-ci-workflows/index.ts b/scripts/check-ci-workflows/index.ts index 821bf5fa6..87a2f5fd3 100644 --- a/scripts/check-ci-workflows/index.ts +++ b/scripts/check-ci-workflows/index.ts @@ -19,7 +19,7 @@ export function checkWorkflows(root: string): void { for (const [name, ids] of [ ["hub-ci", ["check", "integration"]], ["actions-ci", ["test", "publish", "screenshots"]], - ["starter-ci", ["test"]], + ["starter-ci", ["test", "windows-setup"]], ["learning-ci", ["test"]], ["scripts-ci", ["check"]], ] as const) { From 1b5dd1c5802436c2ca469881199409a06da24723 Mon Sep 17 00:00:00 2001 From: James Bradford Date: Fri, 20 Nov 2026 14:12:11 -0800 Subject: [PATCH 3/4] fix(starter): reject Unicode archive path collisions List tar members in a UTF-8 locale and compare full paths case-insensitively before extraction. Cover names that differ only by a non-ASCII letter and distinguish the focused Windows command from Linux plugin checks. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- CONTRIBUTING.md | 9 ++++++++- scripts/create-prototype-repo.test.mjs | 7 ++++++- scripts/install-starter.sh | 25 +++++++++++++++++++++---- 3 files changed, 35 insertions(+), 6 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 77d2ee43f..0b88d4535 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -52,7 +52,14 @@ npm test npm run build ``` -Update starter skills and context templates directly under `starter/.agents/`. Keep authored context in consuming repositories separate from those shared files. For starter-copy, static context library/link and plugin setup checks, run `node --test scripts/create-prototype-repo.test.mjs scripts/setup-replica-plugin.test.mjs` from the repository root. On Windows the creator and installer tests need the full Git for Windows installation with Git Bash. These checks do not verify agent authoring, semantic preservation or activation behavior; use the setup skill's real-host evaluation protocol for that evidence. +Update starter skills and context templates directly under `starter/.agents/`. Keep authored context in consuming repositories separate from those shared files. + +Run the setup checks from the repository root: + +- On Linux, run `node --test scripts/create-prototype-repo.test.mjs scripts/setup-replica-plugin.test.mjs` to check starter creation, the context library, and plugin setup. +- On native Windows, install the full Git for Windows with Git Bash and run `node --test scripts/create-prototype-repo.test.mjs` for the starter setup checks. The Windows suite does not include plugin setup. + +These checks do not verify agent authoring, semantic preservation or activation behavior; use the setup skill's real-host evaluation protocol for that evidence. Reuse an existing compatible Node.js installation. If it is unavailable on macOS or Linux, get approval before installing [nodenv](https://github.com/nodenv/nodenv), [node-build](https://github.com/nodenv/node-build), or Node.js, or changing shell configuration. The starter installer does not install them. diff --git a/scripts/create-prototype-repo.test.mjs b/scripts/create-prototype-repo.test.mjs index 285828ffd..73fd4b274 100644 --- a/scripts/create-prototype-repo.test.mjs +++ b/scripts/create-prototype-repo.test.mjs @@ -400,7 +400,7 @@ function rewriteArchiveMember(archive, path, link) { assert.fail("Missing tar fixture member") } -function archiveFixture(t, { paths = requiredArchivePaths, missing, extraPath, link } = {}) { +function archiveFixture(t, { paths = requiredArchivePaths, missing, file, extraPath, link } = {}) { const fixture = installerFixture(t) const { directory, archive } = fixture const source = join(directory, "source") @@ -414,6 +414,7 @@ function archiveFixture(t, { paths = requiredArchivePaths, missing, extraPath, l mkdirSync(dirname(bootstrap), { recursive: true }) copyFileSync(script, bootstrap) if (missing) rmSync(join(archiveRoot, missing)) + if (file) writeFileSync(join(archiveRoot, file), "fixture data") if (extraPath) writeFileSync(join(archiveRoot, "fixture-member"), "fixture data") // GNU tar interprets the colon in an absolute Windows archive name as a host. const result = runBash( @@ -636,6 +637,10 @@ test("rejects missing, linked, and unsafe archive members before writing the des [{ extraPath: "replica-main/starter/.Git/config" }, /Git metadata/], [{ extraPath: "replica-main/starter/.git/config" }, /Git metadata/], [{ extraPath: "replica-main/starter/readme.md" }, /duplicate path/], + [ + { file: "starter/caf\u00e9.md", extraPath: "replica-main/starter/caf\u00c9.md" }, + /duplicate path/, + ], ]) { await t.test(options.missing ?? options.extraPath, member => { const fixture = archiveFixture(member, options) diff --git a/scripts/install-starter.sh b/scripts/install-starter.sh index e9252c07c..f913b6e3c 100755 --- a/scripts/install-starter.sh +++ b/scripts/install-starter.sh @@ -59,6 +59,12 @@ if ! run_and_wait curl -q -fLSs --output "$archive" \ fail 'Could not download the Replica source archive. Check your network connection and access to GitHub, then try again.' fi +# Use a UTF-8 locale so the tar listing retains non-ASCII names. +if [[ "$OSTYPE" == darwin* ]]; then + export LC_ALL=en_US.UTF-8 +else + export LC_ALL=C.UTF-8 +fi # Relative archive names keep GNU tar from interpreting a Windows drive as a host. if ! tar -tzf "$archive" > members || ! tar -tvzf "$archive" > types; then fail 'Could not read the Replica source archive.' @@ -111,10 +117,21 @@ while IFS= read -r member; do esac done < members 3< types -duplicates="$(sort -f selected | uniq -id)" -if [[ -n "$duplicates" ]]; then - fail 'Replica source archive contains a duplicate path.' -fi +# Compare whole names with Bash's Unicode-aware matching. Validation above +# rejects newlines, so they can separate paths without making partial matches. +check_duplicates() ( + shopt -s nocasematch + local seen=$'\n' path key + while IFS= read -r path; do + key=$'\n'"$path"$'\n' + if [[ "$seen" == *"$key"* ]]; then + fail "Replica source archive contains a duplicate path: $path" + fi + seen+="$path"$'\n' + done < selected +) +check_duplicates + if (( ! has_script || ! has_skills || ! has_templates )); then fail 'Replica source archive is missing the starter or setup script.' fi From a0d9c07de276bcf3b294dc69b27276f43f949919 Mon Sep 17 00:00:00 2001 From: James Bradford Date: Sat, 21 Nov 2026 12:22:12 -0800 Subject: [PATCH 4/4] docs(starter): guide agents to local setup destinations Keep folder permission decisions out of the user-facing setup flow. Direct agents to suggest a local alternative for known shared writable locations without changing installer behavior. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --- AGENTS.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/AGENTS.md b/AGENTS.md index 4dc5e5556..52e0c7b7a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -38,6 +38,8 @@ The new Hub has one protected manual workflow for a named non-production Azure i For requests to create a prototype repository or set up an existing repository for prototyping, read [prototype repository setup](docs/prototype-repository-setup.md) before acting. It owns target selection, starter installation, the fresh-session handoff, and the path from local preview to optional guidance, a team baseline, or Hub. On native Windows, follow its Git Bash instructions for starter creation. +When creating a starter, default to a local folder the user controls. If the requested parent is known to let other accounts modify its contents, suggest a destination under the user's home folder instead. Do not ask the user to audit folder permissions. + ## Browser validation and Playwright fallback Use the browser canvas first for agent-driven rendered checks, interaction tests, screenshots and visual review across Replica. Keep the normal visual preview open; the Replica prototype canvas's publishing controls are separate from browser inspection. Discover the browser canvas's actual open and action schemas before use, then address the open panel through its `instanceId`. Actions that take only that handle need no separate page acquisition. Supply a separate page identifier only if the discovered schema requires one, using a runtime-returned value rather than inventing or deriving it.