Skip to content

Improve Playwright test coverage and stability for EmbeddedChat (#1227)#1233

Open
7vignesh wants to merge 1 commit intoRocketChat:developfrom
7vignesh:issue-1227-playwright-tests
Open

Improve Playwright test coverage and stability for EmbeddedChat (#1227)#1233
7vignesh wants to merge 1 commit intoRocketChat:developfrom
7vignesh:issue-1227-playwright-tests

Conversation

@7vignesh
Copy link

@7vignesh 7vignesh commented Mar 21, 2026

Overview(closes #1227)

Improves Playwright test coverage for the EmbeddedChat React package to validate the unauthenticated login flow and make the e2e suite more stable and CI-friendly.

Changes

🧪 Test coverage

  • Replaced the previous basic rendering/title checks with 3 focused tests:
    • renders unauthenticated chat state — verifies EmbeddedChat loads with login prompt, JOIN button, and disabled input.
    • opens login modal from join button — asserts that clicking JOIN opens the password login modal with expected fields.
    • shows required field validation for empty login submit — ensures submitting an empty login form shows required-field validation messages.
  • Added test.beforeEach to centralize page.goto("/") and keep tests deterministic.
  • Uses role/text/placeholder-based selectors instead of brittle CSS selectors where possible.

Note: These tests intentionally cover only the unauthenticated login path (UI and validation), not a full successful login.

🔧 Configuration

  • App.tsx

    • Makes Rocket.Chat host configurable via VITE_RC_HOST, defaulting to http://127.0.0.1:3000.
    • Keeps the example focused on an unauthenticated room state.
  • playwright.config.ts

    • Reporter:
      • CI: github + html (with open: 'never')
      • Local: list + html (with open: 'never')
    • Diagnostics: trace: 'retain-on-failure' and screenshot: 'only-on-failure' for better debugging of flaky failures.
    • Base URL: http://127.0.0.1:5173
  • README.md

    • Adds comprehensive documentation for setup, browser installation, commands, coverage, and configuration.

Acceptance Criteria Met ✅

  • Stable, deterministic tests — Uses semantic selectors (role, text, placeholder); no external service dependencies
  • Mocking/fixtures — Tests verify unauthenticated UI only; no external API calls
  • CI-ready — GitHub reporter integration, forbidOnly guard, configurable retries (2 on CI)
  • Clear failure diagnostics — Screenshots + traces auto-collected on failure
  • Optimized execution — 3 lightweight tests; expected <10s runtime

How to test locally

# Install system dependencies (Linux)
sudo yarn playwright install-deps

# Install Playwright browser
cd packages/e2e-react
npx playwright install chromium

# Run tests
yarn workspace e2e-react test

# Debug mode (interactive)
yarn workspace e2e-react test --debug

# View HTML report
yarn workspace e2e-react show-report

Copilot AI review requested due to automatic review settings March 21, 2026 14:24
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Improves the packages/e2e-react Playwright E2E suite for the EmbeddedChat React example by adding more focused unauthenticated-flow tests and making CI diagnostics/reporting more helpful and deterministic.

Changes:

  • Replaces the previous basic checks with 3 unauthenticated login-flow UI/validation tests and centralizes navigation via beforeEach.
  • Makes the Rocket.Chat host configurable via VITE_RC_HOST (with a local default).
  • Updates Playwright reporting and failure diagnostics; expands README with setup/run instructions.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
packages/e2e-react/tests/example.spec.ts Reworks E2E tests to validate unauthenticated state, login modal opening, and required-field validation.
packages/e2e-react/src/App.tsx Switches EmbeddedChat host to an env-configurable value with a local default.
packages/e2e-react/playwright.config.ts Adjusts reporters for CI/local and improves failure artifacts (trace/screenshot).
packages/e2e-react/README.md Adds detailed setup/running/configuration documentation for the E2E package.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

/* Base URL to use in actions like `await page.goto('/')`. */
baseURL: 'http://127.0.0.1:5173',

/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comment above trace is now inaccurate: the config uses trace: 'retain-on-failure' (not “when retrying the failed test”). Update the comment to reflect the current behavior so future changes don’t misinterpret what artifacts are collected.

Suggested change
/* Collect trace when retrying the failed test. See https://playwright.dev/docs/trace-viewer */
/* Collect and retain trace on test failure. See https://playwright.dev/docs/trace-viewer */

Copilot uses AI. Check for mistakes.

### View HTML report
```bash
yarn workspace e2e-react show-report
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

README instructs running yarn workspace e2e-react show-report, but this workspace’s package.json doesn’t define a show-report script. Either add a show-report script (e.g., invoking playwright show-report) or update the docs to use npx playwright show-report / yarn playwright show-report so the command works.

Suggested change
yarn workspace e2e-react show-report
npx playwright show-report

Copilot uses AI. Check for mistakes.
import { EmbeddedChat } from "@embeddedchat/react";

function App() {
const host = import.meta.env.VITE_RC_HOST || "http://127.0.0.1:3000";
Copy link

Copilot AI Mar 21, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

import.meta.env.VITE_RC_HOST will fail TypeScript type-checking unless ImportMetaEnv is augmented (Vite’s default ImportMetaEnv doesn’t declare this key). Add a declaration in src/vite-env.d.ts for readonly VITE_RC_HOST?: string (or access via bracket notation with an explicit cast) so yarn build (which runs tsc) stays green.

Suggested change
const host = import.meta.env.VITE_RC_HOST || "http://127.0.0.1:3000";
const host =
(import.meta.env as { VITE_RC_HOST?: string }).VITE_RC_HOST ||
"http://127.0.0.1:3000";

Copilot uses AI. Check for mistakes.
@7vignesh 7vignesh changed the title improve Playwright test coverage and stability for #1227 Improve Playwright test coverage and stability for EmbeddedChat (#1227) Mar 21, 2026
@7vignesh 7vignesh changed the title Improve Playwright test coverage and stability for EmbeddedChat (#1227) Improve Playwright test coverage and stability for EmbeddedChat (Closes #1227) Mar 21, 2026
@7vignesh 7vignesh changed the title Improve Playwright test coverage and stability for EmbeddedChat (Closes #1227) Improve Playwright test coverage and stability for EmbeddedChat (#1227) Mar 21, 2026
@sudorishabh
Copy link

Hey @7vignesh @Spiral-Memory, I noticed a few potential issues in this code, happy to help fix them if you’d like.

@Spiral-Memory
Copy link
Collaborator

I'm good with it. Happy to see people collaborating. It's all community efforts.

Feel free to discuss with Vignesh first and start collaborating.

@sudorishabh
Copy link

sudorishabh commented Mar 21, 2026

Yes, this is really exciting for me as well, since it's my first time contributing to a project. It truly feels like being part of a team working together to make something successful.

@Spiral-Memory @7vignesh, let me mention what I figured out we can improve here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve Playwright Test Coverage and Stability to Eliminate Manual UI Testing

4 participants