semantic-playwright

Click by description, not selectors.

A Playwright extension for semantic element targeting. Write resilient tests that survive website redesigns.

Installation

npm install semantic-playwright playwright

Quick Start

import { test } from '@playwright/test';
import { semantic } from 'semantic-playwright';

test('login flow', async ({ page }) => {
  await page.goto('https://example.com/login');

  const s = semantic(page);

  // Click and fill by description - no CSS selectors needed
  await s.fill('email input', 'user@example.com');
  await s.fill('password field', 'secret123');
  await s.click('sign in button');

  // Wait for success
  await s.waitFor('welcome message');
});

Why?

Modern web UIs change constantly. CSS selectors like #main-nav > ul > li:nth-child(3) > a.btn-primary break the moment a designer moves a button. This library provides self-healing locators that target elements by their semantic purpose.

// Fragile - breaks on redesign
await page.click('#nav-signin-btn');

// Resilient - survives redesign
await s.click('sign in button');

How It Works

The semantic engine classifies DOM elements using multiple signals:

• ARIA roles: role="button", role="navigation"
• Semantic HTML: <nav>, <main>, <button>, <form>
• Text content: "Sign In", "Submit", "Search"
• Class patterns: btn, nav, form-input
• Context: Element inside <nav> likely navigation-related

No LLM required. Pure heuristic matching. Fast, local, privacy-preserving.

API

semantic(page)

Wrap a Playwright page with semantic methods:

const s = semantic(page);

s.click(description, options?)

Click an element by description:

await s.click('submit button');
await s.click('login', { purposeHint: 'action' });

s.fill(description, value, options?)

Fill a form field by description:

await s.fill('email input', 'test@example.com');
await s.fill('search box', 'playwright');

s.locator(description, options?)

Get a Playwright Locator for chaining:

const btn = await s.locator('checkout button');
await btn.hover();
await btn.click();

s.analyze(includeUnknown?)

Analyze page structure:

const { elements, summary } = await s.analyze();
console.log(summary);
// { navigation: 12, action: 8, form: 5, content: 20, data: 3, unknown: 0 }

s.query(category, textQuery?)

Find elements by category:

const buttons = await s.query('action');
const searchInputs = await s.query('form', 'search');

s.findAll(description, options?)

Find all matching elements with scores:

const matches = await s.findAll('submit');
for (const { locator, score, text } of matches) {
  console.log(`${text}: ${score}`);
}

s.waitFor(description, options?)

Wait for element to appear:

await s.waitFor('success notification', { timeout: 10000 });

Categories

Category	Description	Examples
navigation	Navigation elements	Nav bars, menus, breadcrumbs
action	Interactive actions	Buttons, CTAs, links styled as buttons
form	Form inputs	Text fields, selects, checkboxes
content	Main content	Articles, headings, paragraphs
data	Data display	Tables, lists, grids

Options

interface SemanticOptions {
  purposeHint?: 'navigation' | 'action' | 'form' | 'content' | 'data';
  timeout?: number;    // Default: 5000ms
  minScore?: number;   // Default: 20
}

Architecture

src/
├── index.ts           # Main API - SemanticPage wrapper
├── semantic-mapper.ts # Classification engine & JS generators

Key design decisions:

• Scripts execute in browser context via page.evaluate()
• Classification is entirely heuristic-based (no external calls)
• Selectors generated are full CSS paths for reliability
• Scoring system prefers exact matches, falls back gracefully

Development

npm install
npm run build
npm test

Testing Philosophy

Tests should use real websites to validate resilience:

1. Test against sites that redesign frequently
2. Snapshot selectors, verify semantic descriptions still work after updates
3. Compare failure rates: CSS selectors vs semantic descriptions

Roadmap

v0.1.0 (Current)

• Core semantic classification engine
• semantic() page wrapper
• click(), fill(), locator() methods
• analyze() and query() methods
• findAll() and waitFor() methods

v0.2.0

• Playwright Test fixtures (useSemanticPage)
• Custom matchers (expect(s).toHaveElement('login button'))
• Confidence thresholds configuration
• Debug mode with match explanations

v0.3.0

• Learning/correction persistence
• Domain-specific overrides
• Screenshot-on-failure with element highlighting
• Playwright Reporter integration

v1.0.0

• Comprehensive test suite
• Performance benchmarks
• Documentation site
• VS Code extension for selector conversion

Prior Art

This library is inspired by and aims to improve upon:

• Browser-Use: AI agent browser automation (requires LLM)
• Healenium: Self-healing Selenium (Java-focused)
• Test.ai: AI-powered testing (SaaS, expensive)

Our differentiator: No LLM required. Fast, local, free.

License

MIT