Self‑Healing Playwright Tests with LLM‑Driven Locator Recovery

Playwright is an open‑source framework for web automation and end‑to‑end testing. By coupling it with a large language model (LLM), the test suite can automatically recover from UI changes: when a locator fails, the framework extracts the current DOM, asks the LLM for a new locator, and retries the action.

Three‑stage self‑healing pipeline

Detection : a Playwright action throws because the target element is not found within a fast 3‑second timeout.

Diagnosis : the framework captures a lightweight DOM snapshot (only interactive elements) and sends it, together with the original locator and error message, to the LLM (or a rule‑based matcher).

Remediation : the LLM returns a single Playwright locator and a confidence score; if confidence ≥ 0.75 the new locator is cached and the original action is retried, otherwise the test fails explicitly.

Test action fails  
        │  
        ▼  
waitFor(selector, 3s timeout)  ← fast fail, don't block 90s  
        │ timeout  
        ▼  
extractDomSnapshot(page)       ← trim DOM to 150 interactive elements  
        │  
        ▼  
askGroqForLocator(prompt)      ← Llama 3.1‑8b‑instant via Groq API  
        │  
        ▼  
confidence >= 0.75?  
   YES → saveCache() → retry action with healed locator  
   NO  → throw error (explicit fail, no silent pass)

The confidence gate is crucial: low‑confidence suggestions are rejected so that a test does not silently pass with an incorrect element.

Project setup and dependencies

mkdir playwright-self-healing-js
cd playwright-self-healing-js
npm init -y
npm install --save-dev @playwright/test
npm install groq-sdk dotenv
npx playwright install

Create a .env file containing the Groq API key: GROQ_API_KEY=gsk_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx The free Groq tier provides 14,400 requests per day and 30 requests per minute, which is ample for a typical test suite.

File structure

playwright-self-healing-js/
├── playwright.config.js
├── package.json
├── .env
├── src/
│   ├── self-healer.js   ← core: DOM snapshot + Groq + cache
│   └── fixtures.js      ← Playwright fixture wrapping all actions
└── tests/
    └── login.spec.js    ← four test cases

DOM snapshot extraction (src/self-healer.js)

async function extractDomSnapshot(page) {
  if (page.isClosed()) {
    throw new Error('[self‑heal] Page already closed — cannot extract snapshot');
  }
  return page.evaluate(() => {
    const selectors = [
      'button', 'a', 'input', 'select',
      'textarea', '[role]', '[data-testid]', 'label',
    ];
    const nodes = document.querySelectorAll(selectors.join(','));
    return Array.from(nodes)
      .slice(0, 150)
      .map((el) => {
        const attrs = [];
        ['id', 'class', 'name', 'type', 'role', 'aria-label',
         'data-testid', 'placeholder', 'for'].forEach((a) => {
          const v = el.getAttribute(a);
          if (v) attrs.push(`${a}="${v.slice(0, 60)}"`);
        });
        const text = (el.textContent ?? '')
          .trim().replace(/\s+/g, ' ').slice(0, 80);
        return `<${el.tagName.toLowerCase()} ${attrs.join(' ')}>${text}</${el.tagName.toLowerCase()}>`;
      })
      .join('
');
  });
}

The guard page.isClosed() prevents a misleading "Target page, context or browser has been closed" error when the test times out before the healing logic runs.

Groq LLM call

async function askGroqForLocator(originalLocator, domSnapshot, errorMessage) {
  const prompt = `You are a Playwright automation expert. A UI locator has broken.

BROKEN LOCATOR: ${originalLocator}
ERROR: ${errorMessage}

DOM SNAPSHOT:
${domSnapshot}

Return ONE Playwright locator using this priority:
 1. page.getByRole('...', { name: '...' })
 2. page.getByTestId('...')
 3. page.getByLabel('...')
 4. page.getByText('...')
 5. page.locator('css') — last resort

Return ONLY valid JSON:
{
  "locator": "page.getByRole('button', { name: 'Login' })",
  "confidence": 0.92,
  "strategy": "role"
}`;
  const completion = await groq.chat.completions.create({
    model: 'llama-3.1-8b-instant',
    messages: [{ role: 'user', content: prompt }],
    temperature: 0.1,
    max_tokens: 200,
    response_format: { type: 'json_object' },
  });
  const parsed = JSON.parse(completion.choices[0]?.message?.content ?? '{}');
  return {
    locator: parsed.locator ?? '',
    confidence: parsed.confidence ?? 0,
    strategy: parsed.strategy ?? 'unknown',
  };
}

Healing logic with cache

async function healLocator(page, originalLocator, error) {
  const cache = loadCache();
  const cached = cache[originalLocator];
  // Return cached result if still valid (1 hour TTL)
  if (cached && (Date.now() - cached.timestamp) < CACHE_TTL_MS) {
    console.log(`[self‑heal] [v] Cache hit: "${originalLocator}" → "${cached.newLocator}"`);
    return { success: true, newLocator: cached.newLocator, confidence: cached.confidence, strategy: 'cache' };
  }
  const domSnapshot = await extractDomSnapshot(page);
  const suggestion = await askGroqForLocator(originalLocator, domSnapshot, error.message);
  // Confidence gate
  if (!suggestion.locator || suggestion.confidence < 0.75) {
    console.warn(`[self‑heal] [!] Low confidence (${suggestion.confidence}). Skipping auto‑heal.`);
    return { success: false, newLocator: null, confidence: suggestion.confidence, strategy: suggestion.strategy };
  }
  // Persist to cache and audit log
  cache[originalLocator] = {
    newLocator: suggestion.locator,
    confidence: suggestion.confidence,
    timestamp: Date.now(),
  };
  saveCache(cache);
  const logLine = `[${new Date().toISOString()}] HEALED: "${originalLocator}" → "${suggestion.locator}" (confidence: ${suggestion.confidence})`;
  fs.appendFileSync('./healing-report.log', logLine + '
');
  return { success: true, newLocator: suggestion.locator, confidence: suggestion.confidence, strategy: suggestion.strategy };
}

Playwright fixture (src/fixtures.js)

const FAST_TIMEOUT = 3_000;

async function withHeal(page, originalSelector, action) {
  try {
    // Fail fast: trigger healing after 3 s
    await page.locator(originalSelector).waitFor({ state: 'attached', timeout: FAST_TIMEOUT });
    await action(page.locator(originalSelector));
  } catch (err) {
    const result = await healLocator(page, originalSelector, err);
    if (!result.success || !result.newLocator) throw err;
    const healedLocator = new Function('page', `return ${result.newLocator}`)(page);
    await action(healedLocator);
  }
}

const test = base.extend({
  healPage: async ({ page }, use) => {
    await use({
      click: (selector) => withHeal(page, selector, (loc) => loc.click()),
      fill: (selector, value) => withHeal(page, selector, (loc) => loc.fill(value)),
      selectOption: (selector, value) => withHeal(page, selector, async (loc) => { await loc.selectOption(value); }),
      check: (selector) => withHeal(page, selector, (loc) => loc.check()),
      getText: async (selector) => { /* with heal fallback */ },
      isVisible: async (selector) => { /* boolean, never throws */ },
    });
  },
});

Note the explicit async wrapper for selectOption to avoid a TypeScript type mismatch (the method returns Promise<string[]>).

Test cases (tests/login.spec.js)

const { test, expect } = require('../src/fixtures');
const BASE_URL = 'https://the-internet.herokuapp.com/login';

// TC-01: baseline with correct locators
test('TC-01 | Login with correct locators (baseline)', async ({ page, healPage }) => {
  await page.goto(BASE_URL);
  await healPage.fill('#username', 'tomsmith');
  await healPage.fill('#password', 'SuperSecretPassword!');
  await healPage.click('button[type="submit"]');
  await expect(page.getByText('You logged into a secure area!')).toBeVisible();
});

// TC-02: broken locators trigger self‑heal
test('TC-02 | Login with BROKEN locators (self‑heal triggered)', async ({ page, healPage }) => {
  await page.goto(BASE_URL);
  await healPage.fill('#user-name-input', 'tomsmith');   // broken
  await healPage.fill('#pass-word-field', 'SuperSecretPassword!'); // broken
  await healPage.click('#login-submit-btn'); // broken
  await expect(page.getByText('You logged into a secure area!')).toBeVisible();
});

// TC-03: cache hit on second run
test('TC-03 | Second run — healer reads from cache', async ({ page, healPage }) => {
  await page.goto(BASE_URL);
  await healPage.fill('#user-name-input', 'tomsmith');
  await healPage.fill('#pass-word-field', 'SuperSecretPassword!');
  await healPage.click('#login-submit-btn');
  await expect(page.getByText('You logged into a secure area!')).toBeVisible();
});

// TC-04: negative path – wrong password
test('TC-04 | Login fails with wrong password', async ({ page, healPage }) => {
  await page.goto(BASE_URL);
  await healPage.fill('#username', 'tomsmith');
  await healPage.fill('#password', 'vagrantwashere');
  await healPage.click('button[type="submit"]');
  const flash = page.locator('#flash');
  await expect(flash).toBeVisible();
  await expect(flash).toContainText('Your password is invalid!');
});

Playwright configuration (playwright.config.js)

module.exports = defineConfig({
  testDir: './tests',
  timeout: 90_000, // 30 s is insufficient for three Groq calls + assertions
  retries: 0,      // retries are handled by the healer
  workers: 1,      // single worker avoids concurrent cache writes
  reporter: [
    ['list'],
    ['html', { outputFolder: 'playwright-report', open: 'never', port: 9324 }],
  ],
  use: {
    headless: true,
    screenshot: 'only-on-failure',
    video: 'retain-on-failure',
  },
});

The 90 s timeout accommodates the three sequential Groq calls (≈ 300 ms each plus network overhead) that occur in TC‑02.

Actual bugs and fixes

TypeScript: 'el' is of type 'unknown'

The error occurs because the tsconfig.json does not include the DOM library. Adding "DOM" to the lib array resolves the issue.

{
  "compilerOptions": {
    "lib": ["ES2020", "DOM"]
  }
}

Another TypeScript warning about .map() is fixed by annotating the callback parameter:

.map((el: Element) => { ... })

Practical recommendations

Never silently accept a low‑confidence heal. The 0.75 threshold is chosen because below it the LLM is essentially guessing; let the test fail so a human can review.

When using a file‑based cache keep workers: 1. Parallel workers corrupt the JSON file; for true parallelism switch to SQLite or Redis.

Add healing-cache.json to .gitignore. The timestamps and locator strings are only meaningful on the local machine; committing the cache provides no value.

Running the suite

export GROQ_API_KEY=gsk_xxxxxxxxxxxxxxxxxx
npm test               # run all tests headlessly
npm run test:headed    # run with visible browser
npm run test:report    # open HTML report (port 9324)

Expected output for the first run demonstrates detection, LLM calls, successful heals, cache hits on the third test, and a final pass:

[chromium] › TC-01 | Login with correct locators            ✓ 1.2s
[chromium] › TC-02 | Login with BROKEN locators
  [self‑heal] 🔍 Locator failed: "#user-name-input". Calling Groq...
  [self‑heal] ✅ Healed → page.getByLabel('Username') (confidence: 0.94)
  [self‑heal] 🔍 Locator failed: "#pass-word-field". Calling Groq...
  [self‑heal] ✅ Healed → page.getByLabel('Password') (confidence: 0.96)
  [self‑heal] 🔍 Locator failed: "#login-submit-btn". Calling Groq...
  [self‑heal] ✅ Healed → page.getByRole('button', { name: 'Login' }) (confidence: 0.91)   ✓ 7.4s
[chromium] › TC-03 | Second run — cache hit            ✓ 1.8s
[chromium] › TC-04 | Login fails with wrong password   ✓ 1.1s

4 passed (11.5s)

Conclusion

Self‑healing test automation does not replace well‑crafted locators, but it keeps a test suite green when UI changes propagate slowly across the system. The audit log records each broken selector, and the architecture can be swapped to other LLM providers such as Ollama or Gemini for potentially better results.