Skip to content
Integration-Automation
WebRunner
Public

About

A framework for Web automation

integration-automation.github.io/WebRunner/

Topics

automationkeyword-driven-testingpythonreport-generatorseleniumselenium-pythonselenium-webdriverselenium4testingweb-testing

Resources

Readme
MIT license

Contributing

Contributing

Security policy

Security policy
Activity
Custom properties

Stars

7 stars

Watchers

1 watching

Forks

2 forks
Report repository
BranchesTags

Repository files navigation

WebRunner

Cross-platform web automation: Selenium + Playwright, plus a JSON-driven action executor with batteries included.

PyPI Version Python Version License Documentation Status

繁體中文 | 简体中文

WebRunner (je_web_runner) started as a Selenium wrapper and grew into a full automation platform: a Selenium and a Playwright backend behind one JSON-driven action executor, plus modules for reporting, observability, orchestration, security, and AI assistance. Every executor command has a deterministic name (WR_*) and a single dispatch point, so an action JSON can mix browser, HTTP, database, and webhook calls in the same script.

Auto-generated reference — every registered WR_* command (signature + summary) is exported under docs/reference/command_reference.md, and a JSON Schema for action JSON files lives at docs/reference/webrunner-action-schema.json.

Table of Contents

Highlights

  • Two backends, one executor. Selenium is the default; the Playwright backend covers most of the same operations under WR_pw_* and is fully opt-in. A comparison table lists the equivalent commands and what exists on one backend only.
  • Action JSON as a contract. Every command resolves through Executor.event_dict; legacy aliases stay alongside snake_case names for back-compat, and a JSON Schema is exported for IDE autocomplete.
  • Reports in five formats. HTML, JSON, XML, JUnit XML (CI-native), and Allure result files; a single manifest binds every output for downstream globs.
  • Orchestration built in. Tag filters, dependency declarations with topological order, ledger-backed re-run-only-failed, flaky detection, A/B run mode, multi-user matrix, deterministic sharding, watch mode, and a stdlib scheduler.
  • Observability without extra plumbing. Auto-screenshot on failure, retry policy, OpenTelemetry hook, live HTTP dashboard, replay studio (HTML timeline), HAR capture + diff.
  • Quality & security guards. Action linter, migration helper, hard-coded secrets scanner, HTTP security headers audit, axe-core accessibility audit, Lighthouse runner, perf metrics (FCP/LCP/CLS), visual regression, snapshot testing, network throttling, arbitrary-script gate.
  • Browser internals. Raw CDP, console + network event capture, localStorage / sessionStorage / IndexedDB, service worker / cache control, Shadow DOM piercing, multi-iframe, file upload / download, browser extension loaders.
  • Advanced WebDriverWrapper surface. set_driver(experimental_options=, extension_paths=, enable_bidi=, install_driver=), attach_to_existing_browser, native CDP shortcuts (set_timezone / set_locale / set_device_metrics / set_user_agent / set_extra_http_headers / set_geolocation / set_network_conditions / block_urls / set_cache_disabled / set_download_directory), Fetch interception primitives (enable_fetch_interception / continue_request / fulfill_request / fail_request), W3C BiDi listeners (add_console_listener / add_js_error_listener), save_cookies / load_cookies for session reuse, save_full_page_screenshot, print_page (PDF), reload(ignore_cache), bring_to_front, switch_to_window_by_url|title, page metadata getters (get_current_url / get_title / get_page_source / get_window_handles / new_window / close_window). All exposed via WR_* aliases too.
  • Standalone CDP / BiDi modules. Background CDPEventListener (WebSocket loop + sync send / on / context manager), record_trace(driver, path) for Chrome DevTools-loadable performance traces, and a bidi_network module wrapping driver.network.add_request_handler / add_response_handler / add_auth_handler for cross-browser request interception.
  • Test data & fixtures. Faker integration, factory pattern, testcontainers (Postgres / Redis / generic), per-environment .env loader with ${ENV.X} placeholder expansion, CSV/JSON data-driven runner with ${ROW.x}.
  • Auth, API, DB. OAuth2 / OIDC client-credentials / password / refresh-token flows with token cache, HTTP API testing commands with JSON assertions, SQLAlchemy-backed database validation.
  • Integrations. TCP socket server with token + TLS, BrowserStack / Sauce Labs / LambdaTest cloud Grid, Appium mobile, JIRA + TestRail, Slack / generic webhook notifier, GitHub Actions inline annotations, Locust load testing.
  • AI hooks. Pluggable LLM callable powers self-healing locators and natural-language → action JSON drafts.
  • Cross-platform & multi-browser. Windows, macOS, Linux, Raspberry Pi · Chrome, Chromium, Firefox, Edge, IE, Safari · Chromium, Firefox, WebKit (Playwright).

Installation

Stable:

pip install je_web_runner

Development:

pip install je_web_runner_dev

je_web_runner_dev follows the dev branch: CI publishes a new version each time a push to dev passes the tests and changes what the package ships.

Optional dependencies (each enables a slice of features; install only what you use):

pip install playwright           # Playwright backend
python -m playwright install     # Browser binaries for Playwright
pip install Pillow               # Visual regression
pip install faker                # Random test data (WR_faker_*)
pip install sqlalchemy           # Database validation (WR_db_*)
pip install opentelemetry-sdk    # Distributed traces (WR_set_action_span_factory)
pip install Appium-Python-Client # Mobile native (WR_appium_*)
pip install testcontainers       # Spin up Postgres / Redis (WR_tc_*)
pip install locust               # Load testing (WR_locust_*)

Hard requirements: Python 3.10+, selenium>=4.0.0, requests, python-dotenv, webdriver-manager, defusedxml, Pillow, je_action_core (the action executor's shared core).

Architecture

System overview

flowchart LR
  subgraph Authoring
    A1["Action JSON files"]
    A2["Programmatic Python API"]
    A3["Browser recorder<br/>(JS injection)"]
    A4["LLM NL → action draft"]
  end

  subgraph Core
    EXE["Executor<br/>event_dict"]
    REC["Test record<br/>singleton"]
    LDG["Run ledger /<br/>flaky detection"]
  end

  subgraph Backends
    SEL["Selenium<br/>WebDriverWrapper"]
    PW["Playwright<br/>PlaywrightWrapper"]
    APM["Appium<br/>Mobile"]
    HTTP["HTTP API<br/>requests"]
    DB["Database<br/>SQLAlchemy"]
  end

  subgraph Outputs
    REP["Reports<br/>HTML/JSON/XML/JUnit/Allure"]
    OBS["Observability<br/>OTel · dashboard · replay"]
    NOT["Notifiers<br/>Slack · webhook · GH · JIRA · TestRail"]
  end

  A1 --> EXE
  A2 --> EXE
  A3 --> A1
  A4 --> A1
  EXE --> SEL
  EXE --> PW
  EXE --> APM
  EXE --> HTTP
  EXE --> DB
  SEL --> REC
  PW --> REC
  APM --> REC
  HTTP --> REC
  DB --> REC
  REC --> LDG
  REC --> REP
  REC --> OBS
  REC --> NOT
Loading

Action lifecycle

flowchart LR
  IN["Action<br/>[cmd, args, kwargs]"] --> VAL["JSON validator<br/>(WR_validate_*)"]
  VAL --> ENV["${ENV.X} / ${ROW.x}<br/>placeholder expansion"]
  ENV --> SPAN["OTel span factory<br/>(optional)"]
  SPAN --> RETRY["Retry policy<br/>retries × backoff"]
  RETRY --> GATE["Arbitrary-script<br/>gate"]
  GATE --> DISP["event_dict[cmd](*args, **kwargs)"]
  DISP --> RECORD["test_record_instance<br/>append()"]
  DISP -- failure --> SHOT["Auto-screenshot<br/>(failure dir)"]
  RECORD --> DONE["Result dict"]
  SHOT --> DONE
Loading

Backend dispatch

flowchart TB
  CMD["Action command name"] --> ROUTE{"prefix?"}
  ROUTE -- "WR_pw_*" --> PW["Playwright backend<br/>(PlaywrightWrapper)"]
  ROUTE -- "WR_pw_element_*" --> PWE["Playwright element<br/>(PlaywrightElementWrapper)"]
  ROUTE -- "WR_appium_*" --> APM["Appium driver"]
  ROUTE -- "WR_http_*" --> HTTP["requests wrapper"]
  ROUTE -- "WR_db_*" --> DB["SQLAlchemy validator"]
  ROUTE -- "WR_pw_a11y_* / WR_a11y_*" --> AXE["axe-core audit"]
  ROUTE -- "WR_pw_throttle / WR_throttle" --> THR["Network throttling<br/>(CDP)"]
  ROUTE -- "WR_pw_route_*" --> ROUTE_MOCK["Playwright route mock"]
  ROUTE -- "WR_*<br/>(default)" --> SEL["Selenium backend<br/>(WebDriverWrapper)"]
  ROUTE -- "WR_element_*<br/>(default)" --> SE["Selenium element<br/>(WebElementWrapper)"]
Loading

Module map

Every utils/ subpackage with a one-line summary, grouped into the core engine, the je_web_runner.api facade themes and the rest: docs/reference/utils_index.md (generated by scripts/gen_utils_index.py).

je_web_runner/
├── __init__.py
├── __main__.py                    # CLI: --execute_dir / --watch / --tag / --shard / --migrate ...
├── element/web_element_wrapper.py
├── manager/webrunner_manager.py
├── webdriver/
│   ├── webdriver_wrapper.py             # Selenium core
│   ├── webdriver_with_options.py
│   ├── playwright_wrapper.py            # Playwright sync backend (full)
│   ├── playwright_element_wrapper.py
│   └── playwright_locator.py            # TestObject ↔ Playwright selector
└── utils/
    ├── ab_run/                  # A/B run mode (run_ab + diff_records)
    ├── accessibility/           # axe-core audit
    ├── ai_assist/               # Pluggable LLM scaffold
    ├── api/                     # HTTP API testing commands
    ├── appium_integration/      # Mobile native via Appium
    ├── auth/                    # OAuth2 / OIDC token helpers
    ├── callback/                # Callback executor
    ├── cdp/                     # Raw CDP passthrough
    ├── ci_annotations/          # GitHub Actions ::error::
    ├── cli/                     # CLI parser, watch mode, dispatch
    ├── cloud_grid/              # BrowserStack / Sauce Labs / LambdaTest
    ├── dashboard/               # Live progress HTTP server
    ├── database/                # SQL validation (SQLAlchemy)
    ├── data_driven/             # CSV/JSON dataset + ${ROW.x}
    ├── docs/                    # Auto-generated command reference
    ├── dom_traversal/           # Shadow DOM / iframe helpers
    ├── env_config/              # .env loader + ${ENV.X}
    ├── exception/               # Exception hierarchy
    ├── executor/                # Action executor + retry/screenshot/gate
    ├── extensions/              # Browser extension loaders
    ├── factories/               # Factory pattern helpers
    ├── file_process/            # File utilities
    ├── file_transfer/           # Upload / download helpers
    ├── generate_report/         # HTML/JSON/XML/JUnit/Allure + manifest
    ├── har_diff/                # HAR file diff
    ├── json/                    # JSON I/O + validator (length 1/2/3)
    ├── lighthouse/              # Lighthouse CLI runner
    ├── linter/                  # action_linter + migration
    ├── load_test/               # Locust wrapper
    ├── logging/                 # Rotating file handler
    ├── multi_user/              # Multi-user matrix runner
    ├── network_emulation/       # Throttling presets via CDP
    ├── notifier/                # Slack / generic webhooks
    ├── observability/           # Console+network capture · OTel
    ├── package_manager/         # Dynamic plugin loader
    ├── perf_metrics/            # FCP / LCP / CLS / TTFB
    ├── pom_generator/           # POM skeleton from URL/HTML
    ├── project/                 # Project template generator
    ├── recorder/                # JS-injection recorder + PII mask
    ├── replay_studio/           # HTML timeline studio
    ├── run_ledger/              # ledger · flaky · classifier
    ├── schema/                  # Action JSON Schema export
    ├── scheduler/               # stdlib-sched scheduled runner
    ├── secrets_scanner/         # Hard-coded credential scanner
    ├── security_headers/        # HTTP headers audit
    ├── selenium_utils_wrapper/  # Keys / Capabilities
    ├── self_healing/            # Fallback locator registry
    ├── service_worker/          # SW unregister + cache clear
    ├── sharding/                # Deterministic test sharding
    ├── snapshot/                # Text/DOM snapshot testing
    ├── socket_server/           # TCP server with token + TLS
    ├── storage/                 # localStorage / session / IDB
    ├── test_data/               # Faker integration
    ├── test_filter/             # Tag filter + dependency graph
    ├── test_management/         # JIRA + TestRail
    ├── test_object/             # TestObject + record
    ├── test_record/             # Action recording
    ├── testcontainers_integration/   # Postgres / Redis / generic
    ├── visual_regression/       # Pillow-based image diff
    └── xml/                     # XML utilities

Cookbook

The examples/ directory ships runnable recipes that exercise the new helpers against real Chrome / network. Each is invoked from the repo root:

Example Demonstrates
counting_stars.{py,json} WR_sleep, WR_set_driver with Chrome flags, autoplay-policy override, JS-driven video.play(), skip-ad polling.
google_search.py Consent dismissal, search-box typing, ENTER submit, result heading scrape.
form_submit.py form_autofill.plan_fill_actions + state_diff.capture_state round trip against httpbin/forms/post.
smart_wait_demo.py wait_for_fetch_idle + wait_for_spa_route_stable + memory_leak.detect_growth against a real page.
fanout_demo.py fanout.run_fan_out parallel HTTP preflights.
pii_redact_demo.py pii_scanner.scan_text + redact_text + assert_no_pii (pure logic).
quick_smoke.json Minimal WR_set_driver → WR_sleep → WR_execute_script → WR_quit_all smoke via the executor CLI.

Run a Python example directly:

python examples/google_search.py

Run an action JSON example through the executor:

python -m je_web_runner -e examples/quick_smoke.json

Test Tiers

test/
├── unit_test/         # mock-based unit tests
├── integration_test/  # wired-modules tests with real I/O
└── e2e_test/          # real-browser tests; skips without Selenium Grid
  • Unit (test/unit_test/test_*.py) — runs everywhere; pulled in by both test_dev.yml and test_stable.yml.
  • Integration (test/integration_test/) — wires 2+ modules together with real SQLite, in-process HTTP servers, and real subprocesses for the MCP / LSP. Same workflows as unit, second step.
  • E2E (test/e2e_test/) — talks to a Selenium Grid via WEBRUNNER_E2E_HUB. Locally: cd docker && docker compose up -d. CI: .github/workflows/e2e_browser.yml boots selenium/hub:4.20.0 + selenium/node-chrome daily / on demand.

Thematic API Façade

The utility helpers live under je_web_runner.utils.<area>; every subpackage outside the core engine is also re-exported by exactly one theme under je_web_runner.api (a name two packages of one theme share is prefixed with its package, e.g. cors_matrix_classify):

from je_web_runner.api import (
    authoring,      # action_formatter, md_authoring, templates, sel_to_pw, bootstrap
    debugging,      # cross_browser, pr_comment, extension_harness
    frontend,       # device emulation, geo/locale, multi-tab, shadow pierce, …
    infra,          # driver pin, k8s runner, pipeline, lock, watch_mode, …
    mobile,         # Appium gestures
    networking,     # api_mock, contract_testing, GraphQL, mock services, har_replay
    observability,  # timeline, failure bundle, trace recorder, OTLP, BiDi, cdp_tap
    quality,        # a11y_diff, a11y_trend, perf budgets/drift, trend, failure cluster
    reliability,    # adaptive retry, browser pool, smart wait, throttler, supervisor
    security,       # PII, license, CSP, cookie consent, header tampering
    test_data,      # DB fixtures, fixture record/replay, form auto-fill
    accessibility_i18n,  # screen reader, forced colours, pseudo-localisation, RTL, OCR, visual AI
    ai,             # LLM assist, auto-repair, edge cases, triage, RAG / tool-call / hallucination checks
    api_testing,    # HTTP, auth, GraphQL N+1, gRPC, OpenAPI drift, rate limits, HAR tools
    audit,          # clickjacking, CORS, cookies, CSP, XSS taint, open redirect, TLS, SRI, secrets, SBOM
    diagnostics,    # OpenTelemetry bridge, backend log correlation, console error budget
    governance,     # flakes, quarantine, owners, test debt, ROI, cost, PR risk, merge gates
    messaging,      # email render / deliverability, push, OTP, webhooks, message queues, Slack digest
    mobile_pwa,     # touch gestures, viewport, virtual keyboard, pull-to-refresh, real-device clouds
    orchestration,  # A/B runs, personas, multi-user, flag matrix, chaos, data-driven, DB snapshots
    performance,    # web vitals, INP, Lighthouse, bundle / third-party budgets, load tests
    platform,       # CDP, BiDi, Chrome profiles, extensions, cloud grid, AutoControl bridge, CLI
    tooling,        # recorder, action schema, command docs, POM generator, replay studio, dashboards
    web_platform,   # WebSocket / SSE / WebRTC, storage, service workers, WebAuthn, WebUSB, payments…
)

The original Selenium-flavoured top-level surface (webdriver_wrapper_instance, execute_action, TestObject, …) is unchanged.

Quick Start

Direct API

from je_web_runner import TestObject, get_webdriver_manager, web_element_wrapper

manager = get_webdriver_manager("chrome")
manager.webdriver_wrapper.to_url("https://www.google.com")
manager.webdriver_wrapper.implicitly_wait(2)

search_box = TestObject("q", "name")
manager.webdriver_wrapper.find_element(search_box)
web_element_wrapper.click_element()
web_element_wrapper.input_to_element("WebRunner automation")

manager.quit()

JSON action list (modern aliases)

from je_web_runner import execute_action

actions = [
    ["WR_new_driver", {"webdriver_name": "chrome"}],
    ["WR_to_url", {"url": "https://www.google.com"}],
    ["WR_implicitly_wait", {"time_to_wait": 2}],
    ["WR_save_test_object", {"test_object_name": "q", "object_type": "NAME"}],
    ["WR_find_recorded_element", {"element_name": "q"}],
    ["WR_element_click"],
    ["WR_element_input", {"input_value": "WebRunner automation"}],
    ["WR_quit_all"],
]
execute_action(actions)

execute_action returns a dict keyed execute: <action>, one entry per step (a repeated identical action gets #2, #3 … so every outcome is kept); it records a failed step and moves on. To run one action and get its value back, use execute_one(["WR_get_current_url"]): it goes through the same command gates, retry policy and failure screenshots, prints nothing, and raises WebRunnerExecuteException (with the original error as its cause) when the action fails.

The legacy names (WR_get_webdriver_manager, WR_SaveTestObject, WR_quit, WR_input_to_element, …) still work — see Quality & Security for the one-shot migration helper.

Mixed positional + keyword arguments

[
    ["WR_to_url", ["https://example.com"], {"timeout": 30}],
]

The validator accepts length-1, length-2 ([cmd, dict_or_list]), and length-3 ([cmd, [positional], {kwargs}]) actions.

Public API & Deprecation Policy

What is public — these are the only things other code should depend on, and the only things this policy protects:

Surface What it covers
Top-level names Everything listed in je_web_runner.__all__ (from je_web_runner import …)
CLI python -m je_web_runner flags, including the original -e/--execute_file, -d/--execute_dir and --execute_str (Windows double-encoded JSON included)
Action JSON The WR_* command names registered in executor.event_dict, and the webdriver_wrapper / meta top-level keys of an action file
Socket server start_web_runner_socket_server, send_command, read_frame, encode_frame and the length-prefixed framing
Supported module paths je_web_runner.utils.executor.action_executor (executor, execute_one), je_web_runner.utils.logging.loggin_instance (web_runner_logger, WebRunnerLoggingHandler), je_web_runner.webdriver.webdriver_wrapper (WebDriverWrapper, webdriver_wrapper_instance, the _options_dict / _webdriver_dict / _webdriver_manager_dict patch targets)

The three module paths are supported because other repositories already import them: AutoControlGUI's WebRunner bridge takes executor and execute_one, Jeffrey_RPA hooks loggin_instance by that exact (misspelled) name and patches the wrapper dictionaries. They are treated as public rather than asked to move. test/unit_test/test_public_api.py fails if any of them disappears.

What is not public: every other module under je_web_runner.utils., anything named with a leading underscore other than the patch targets above, the shape of report files, and the log format. They can move in any release.

Retiring something public

  1. The replacement ships first, and the old name keeps working as an alias.
  2. The alias raises DeprecationWarning naming the replacement, and the release notes say so.
  3. The alias stays for at least two further releases before it can be removed, and only a release that names the removal may drop it.
  4. A name listed in architecture.md §6 as a cross-project contract is different: it never changes without the consuming repository changing in the same round, and §6 is updated on both sides.

Core API

The original Selenium-flavoured API remains the canonical entry point for programmatic use. Sections preserved from the original README:

  • WebDriver Manager — get_webdriver_manager, new_driver, change_webdriver, close_choose_webdriver, quit.
  • WebDriver Wrapper — to_url, forward, back, refresh, find_element, find_elements, implicitly_wait, explict_wait (alias WR_explicit_wait), set_script_timeout, set_page_load_timeout, the full ActionChains-backed mouse/keyboard surface, cookies, execute_script, window management, screenshots, frame/window/alert switching, get_log.
  • Web Element Wrapper — click_element, input_to_element, clear, submit, get_attribute, get_property, get_dom_attribute, is_displayed, is_enabled, is_selected, value_of_css_property, screenshot, change_web_element, check_current_web_element, plus the new select_by_value / select_by_index / select_by_visible_text.
  • TestObject — TestObject(name, type), create_test_object, get_test_object_type_list (returns ['ID', 'NAME', 'XPATH', 'CSS_SELECTOR', 'CLASS_NAME', 'TAG_NAME', 'LINK_TEXT', 'PARTIAL_LINK_TEXT']).

Programmatic examples for each surface are kept identical to the previous edition; see the relevant Sphinx pages under docs/source/Eng/doc/ for full code snippets.

Action Executor

The executor maps a string command name to a Python callable. Every backend, integration, and helper registers under event_dict.

Action shapes

["command"]                                    # no args
["command", {"key": "value"}]                  # kwargs
["command", [arg1, arg2]]                      # positional
["command", [arg1], {"key": "value"}]          # positional + kwargs (length 3)

Length-3 example

[
    ["WR_pw_evaluate", ["() => document.title"], {"arg": None}],
]

Pacing actions

WR_sleep blocks the executor thread for a given number of seconds — useful when the page needs settle time, when a JS animation needs to finish, or when an example wants to hold the browser open for the user to watch:

[
    ["WR_to_url", {"url": "https://example.com"}],
    ["WR_sleep", {"seconds": 2.5}],
    ["WR_get_screenshot_as_png"],
]

Negative or non-numeric seconds raise ValueError. For pacing inside JavaScript (e.g. waiting on a custom event from the page) use WR_execute_async_script with a setTimeout-driven callback.

Top-level shapes

[ ...actions... ]                                                  # bare list

{
  "webdriver_wrapper": [ ...actions... ],
  "meta": {"tags": ["smoke", "fast"], "depends_on": ["login"]}     # optional
}

meta.tags and meta.depends_on are picked up by the CLI for filtering and topological execution.

Adding custom commands

from je_web_runner import add_command_to_executor

def my_step(name: str) -> None:
    print(f"hello {name}")

add_command_to_executor({"my_command": my_step})

Retry, screenshots, scripts

from je_web_runner.utils.executor.action_executor import executor

executor.set_retry_policy(retries=2, backoff=0.5)             # global retry
executor.set_failure_screenshot_dir("./failures")              # auto PNG on raise
executor.set_allow_arbitrary_script(False)                     # gate WR_execute_script / WR_pw_evaluate / WR_cdp
executor.allow_packages("my_helpers")                         # WR_add_package_to_executor may load these
executor.set_allow_arbitrary_packages(False)                   # and refuses every other package

Backends

Selenium (default)

Selenium is the original backend. Every legacy command (and its modern alias) routes here unless an explicit WR_pw_* / WR_appium_* prefix is used.

The Selenium backend has the twins of the Playwright element and page helpers: JSON-usable waits (WR_wait_for_element with present / visible / clickable / hidden, WR_wait_for_url, WR_wait_for_title, WR_wait_for_ready_state), WR_find_element_by / WR_find_elements_by for a raw selector, WR_element_check / _uncheck / _hover / _inner_text / _inner_html, WR_alert_accept (with prompt_text) / WR_alert_dismiss / WR_alert_text, and on Chromium WR_emulate_device / WR_list_devices, WR_grant_permissions / WR_clear_permissions and WR_clock_freeze. Unlike the older Selenium commands these fail their action when they fail.

The older Selenium commands fail their action too. Every Selenium wrapper call (WR_to_url, WR_element_click, WR_element_assert, WR_check_current_webdriver …) records its error in the test record and then raises it, so a wrong WR_element_assert fails. They used to let the action pass; executor.set_raise_wrapper_errors(False), or ["WR_set_raise_wrapper_errors", [false]] first in an action file, brings that back.

With a driver started with enable_bidi=True (Chrome or Firefox) the Selenium backend also has the Playwright event capture, HAR and response mocks over W3C BiDi: WR_event_capture_start (dom_mutations=True also records DOM changes) / _stop / _clear, WR_console_messages, WR_network_responses, WR_dom_mutations, WR_assert_no_console_errors / WR_assert_no_5xx / WR_assert_no_4xx_or_5xx, WR_start_har_recording / WR_stop_har_recording(har_path) (headers, status and timings, no bodies), and WR_route_mock / WR_route_mock_json / WR_route_unmock / WR_route_clear. A mock pattern is an exact URL, */path (that path on any host) or a dict of URL parts; mock what the page fetches, not the page itself, whose navigation a mock would block.

Playwright

The Playwright backend covers most everyday operations of the Selenium wrapper under WR_pw_*, but it is not a one-to-one copy. The comparison table in docs/source/Eng/doc/backends/backends_doc.rst lists the equivalent commands and how they differ (for example, timeouts in milliseconds instead of seconds, tabs switched by index only), what cannot exist on the other backend (Internet Explorer, Selenium Grid, Appium and OS window geometry on Selenium only; WebKit, the trace viewer and automatic actionability waits on Playwright only), and what is not on the other backend yet.

The page-level shortcuts pass extra Playwright options through (button, modifiers, timeout, force, wait_until …), page-level actions are added to the test record like Selenium steps, and pages the site opens itself (popups, window.open) are tracked so WR_pw_switch_to_page can reach them. The commands:

  • Lifecycle / pages / navigation — WR_pw_launch, WR_pw_quit, WR_pw_new_page, WR_pw_switch_to_page, WR_pw_close_page, WR_pw_to_url, WR_pw_forward, WR_pw_back, WR_pw_refresh, WR_pw_url, WR_pw_title, WR_pw_content.
  • Find — WR_pw_find_element, WR_pw_find_elements, WR_pw_find_element_with_test_object_record, WR_pw_find_with_healing.
  • Page-level shortcuts — WR_pw_click, WR_pw_dblclick, WR_pw_hover, WR_pw_fill, WR_pw_type_text, WR_pw_press, WR_pw_check, WR_pw_uncheck, WR_pw_select_option, WR_pw_drag_and_drop.
  • Element-level (after WR_pw_find_element_with_test_object_record) — WR_pw_element_click, WR_pw_element_dblclick, WR_pw_element_fill, WR_pw_element_type_text, WR_pw_element_press, WR_pw_element_check, WR_pw_element_uncheck, WR_pw_element_select_option, WR_pw_element_get_attribute, WR_pw_element_inner_text, WR_pw_element_inner_html, WR_pw_element_is_visible, WR_pw_element_is_enabled, WR_pw_element_is_checked, WR_pw_element_scroll_into_view, WR_pw_element_screenshot, WR_pw_element_change.
  • Script / cookies / waits / viewport / mouse / keyboard / frames — WR_pw_evaluate, WR_pw_get_cookies, WR_pw_add_cookies, WR_pw_clear_cookies, WR_pw_screenshot, WR_pw_wait_for_selector, WR_pw_wait_for_load_state, WR_pw_wait_for_timeout, WR_pw_wait_for_url, WR_pw_set_viewport_size, WR_pw_mouse_*, WR_pw_keyboard_*.
  • Mobile emulation / locale / clock — WR_pw_emulate("iPhone 13"), WR_pw_set_locale, WR_pw_set_timezone, WR_pw_clock_install / _set_time / _run_for, WR_pw_set_geolocation, WR_pw_grant_permissions.
  • Context options — WR_pw_launch(context_options={...}), WR_pw_set_context_options (any browser.new_context option), WR_pw_set_user_agent, WR_pw_set_extra_http_headers, WR_pw_save_storage_state. The wrapper keeps one merged set of context options, so emulation, timezone, locale and HAR recording no longer replace each other; a setting that needs a new context carries cookies, localStorage and the current page's URL over.
  • HAR + route mock — WR_pw_start_har_recording, WR_pw_stop_har_recording, WR_pw_route_mock, WR_pw_route_mock_json, WR_pw_route_unmock, WR_pw_route_clear.
  • Trace viewer / video — WR_pw_tracing_start, WR_pw_tracing_save_chunk(path), WR_pw_tracing_stop(path) (open the zip with playwright show-trace); WR_pw_video_start(video_dir, width, height) and WR_pw_video_stop (returns the video files). While a trace is recording and executor.set_failure_screenshot_dir(...) is set, each failed action also saves <timestamp>_<command>.trace.zip beside its screenshot.
  • Selenium twins — WR_pw_add_init_script, WR_pw_block_urls / WR_pw_unblock_urls (* wildcards; both survive context rebuilds), WR_pw_clear_geolocation, WR_pw_bring_to_front, WR_pw_switch_to_page_by_url / _by_title, WR_pw_screenshot_base64, WR_pw_print_page (headless Chromium), WR_pw_scroll / _to_top / _to_bottom, WR_pw_drag_and_drop_offset. Also WR_pw_get_cookie / WR_pw_delete_cookie, WR_pw_element_submit, WR_pw_element_value_of_css_property, and the assertions WR_pw_check_current_page / WR_pw_element_assert, which fail their action on a mismatch.
  • Frames, user-facing lookup, dialogs — WR_pw_switch_to_frame (nested with a list) / _to_parent_frame / _to_main_frame: selector commands act in the selected iframe until you switch back or navigate. WR_pw_find_by(by, value) finds by role (with name), text, label, placeholder, alt_text, title or test_id and makes the first match the current element. WR_pw_set_dialog_policy("accept" | "dismiss", prompt_text) answers alerts, confirms and prompts on every page; WR_pw_last_dialog returns the last one.
  • Attach, profile, downloads, cache — WR_pw_connect_over_cdp(endpoint_url) attaches to a Chromium started with a debugging port (quitting only disconnects); WR_pw_connect(ws_endpoint) reaches a Playwright browser server; WR_pw_launch_persistent(user_data_dir, extension_paths=[...]) opens a profile and loads unpacked extensions. WR_pw_download(selector, save_to) clicks and saves the download (to a file, or into a directory under the suggested name). WR_pw_set_cache_disabled switches the HTTP cache off and on in every browser.
  • Visual regression, recorder, callbacks — WR_pw_visual_capture_baseline / WR_pw_visual_compare and WR_pw_recorder_start / _stop / _pull_events / _save do on the Playwright page what their WR_* twins do on Selenium, and every WR_pw_* command can trigger a callback.
  • Several browsers — WR_pw_new_browser(browser, ...) opens another browser and switches to it (returns its index), WR_pw_switch_browser(index), WR_pw_close_browser, WR_pw_browser_count; each keeps its own pages and settings, and WR_pw_quit closes them all.

Existing scripts can move to Playwright incrementally; TestObject records are translated to Playwright selectors automatically (CSS_SELECTOR → as-is, XPATH → xpath=…, ID → #…, NAME → [name="…"], LINK_TEXT → text=…, PARTIAL_LINK_TEXT → :has-text("…")).

Cloud Grid

from je_web_runner.utils.cloud_grid.cloud_drivers import (
    connect_browserstack,
    build_browserstack_capabilities,
)

connect_browserstack(
    username="...",
    access_key="...",
    capabilities=build_browserstack_capabilities(
        browser_name="chrome",
        browser_version="latest",
        os_name="Windows",
        os_version="11",
        project="WebRunner",
        build="ci-2026-04-26",
    ),
)
# All existing WR_* commands now run against the cloud session.

connect_saucelabs and connect_lambdatest follow the same shape.

Appium (mobile)

from je_web_runner.utils.appium_integration.appium_driver import (
    start_appium_session,
    build_android_caps,
    build_ios_caps,
)

start_appium_session(
    "https://appium.example/wd/hub",
    capabilities=build_android_caps(app="/path/to/app.apk"),
)
# WR_* commands now drive the mobile session.

Reports

from je_web_runner import (
    generate_html_report,
    generate_json_report,
    generate_xml_report,
    generate_junit_xml_report,
    generate_allure_report,
)
from je_web_runner.utils.generate_report.report_manifest import generate_all_reports

# Run every generator + write a manifest binding all outputs:
result = generate_all_reports("run_2026_04_26", allure_dir="allure-results")
print(result["manifest_path"])  # → run_2026_04_26.manifest.json
Format Output shape Spec-driven?
JSON <base>_success.json + <base>_failure.json split
HTML <base>.html single
XML <base>_success.xml + <base>_failure.xml split
JUnit XML <base>_junit.xml single
Allure <allure_dir>/<uuid>-result.json (× N) directory

The manifest captures the actual paths produced — CI globs no longer need to know the per-format conventions.

Interactive HTML report. generate_interactive_html_report(html_name, screenshot_dir=None, har_path=None, a11y_results=None) (WR_generate_interactive_html_report) writes one self-contained <html_name>.html, with no external resources and no script; each section expands with <details>:

  • a timeline of every recorded step, failures open, with the screenshots from screenshot_dir placed where they were taken and embedded (the executor's failure screenshots are named by time, other images go by their modification time);
  • each failure's full error, plus a unified diff when an assertion compares two values (a != b, expected X but got Y);
  • a network waterfall from a HAR 1.2 file (WR_pw_start_har_recording / WR_start_har_recording write one), with method, status, duration and URL per request;
  • axe-core violation badges by impact and the violations themselves, from the result dict or its JSON file.

Every recorded value is escaped.

Observability

from je_web_runner import (
    test_record_instance,
    summarise_run,
    notify_run_summary,
)
from je_web_runner.utils.executor.action_executor import executor
from je_web_runner.utils.observability.otel_tracing import install_executor_tracing
from je_web_runner.utils.dashboard.live_dashboard import start_dashboard
from je_web_runner.utils.replay_studio.replay_studio import export_replay_studio

executor.set_failure_screenshot_dir("./failures")
install_executor_tracing("webrunner")                 # one OTel span per action
start_dashboard("127.0.0.1", 8080)                    # browser-friendly progress UI
test_record_instance.set_record_enable(True)

# … run actions …

export_replay_studio("./run.html", screenshot_dir="./failures")
notify_run_summary("https://hooks.slack.com/services/...")

Failure screenshot, OpenTelemetry tracing, retry policy, and the live dashboard all hook into the same Executor.event_dict so they compose without coupling.

There are two dashboards. start_dashboard above follows the current run live. DashboardServer (utils/live_dashboard) aggregates history from files other modules write:

from je_web_runner import DashboardConfig, DashboardServer

server = DashboardServer(DashboardConfig(
    ledger_path="ledger.json",                 # runs, pass-rate chart, flake scores
    quarantine_path="quarantine.json",
    locator_findings_path="locator_health.json",
    schedule_path="schedule.json",             # test_scheduler output
    triage_report_path="triage.json",          # failure_triage output
    bind_port=8765,                            # 127.0.0.1 unless bind_host says otherwise
))
print(server.start())                          # http://127.0.0.1:8765, served from a daemon thread
  • Pages: Overview (summary cards coloured by status, a daily pass-rate chart with failure bars, the latest runs), Runs (?limit=, "Show more"), Flake, Quarantine (with links to triage tickets), Locators (weakest locators and self-healing fallback offenders), Schedule and Triage. Any source left as None shows an empty state saying which DashboardConfig field to set.
  • Tables sort by clicking a header and filter as you type (Runs also by result); times show in the browser's time zone. The page refreshes its data every 15 seconds, keeping your filter, and the Pause button stops it. Light and dark follow the system setting.
  • The same data is JSON under /api/summary, /api/runs, /api/flake, /api/quarantine, /api/locators, /api/schedule, /api/triage and /api/trend.
  • Standard library only. Every page works without JavaScript; the script and stylesheet are served from the same origin under a strict Content-Security-Policy (default-src 'self'), and every value is HTML-escaped.

Test Orchestration

# Filter by tag, run in parallel processes, persist a ledger, fail fast on dep breaks.
python -m je_web_runner \
    --execute_dir ./actions \
    --tag smoke,fast \
    --exclude-tag slow \
    --parallel 4 \
    --parallel-mode process \
    --ledger ./.run_ledger.json

# Re-run only the files that failed last time:
python -m je_web_runner --execute_dir ./actions --rerun-failed ./.run_ledger.json

# Watch a directory and re-run on file change:
python -m je_web_runner --execute_dir ./actions --watch ./actions

# Distribute across 4 runners deterministically (per machine):
python -m je_web_runner --execute_dir ./actions --shard 1/4
python -m je_web_runner --execute_dir ./actions --shard 2/4
python -m je_web_runner --execute_dir ./actions --shard 3/4
python -m je_web_runner --execute_dir ./actions --shard 4/4

Companion APIs — WR_run_for_users (multi-user matrix), WR_run_ab (A/B mode), WR_flakiness_stats, WR_classify_failure, WR_schedule + WR_run_scheduler_for.

Async execution

--parallel-mode async (or AsyncExecutor from Python) runs action files on asyncio: one headless Chromium, and for each file a fresh browser context, so concurrent files act as separate users with their own cookies, storage and tabs. --parallel N caps how many run at once. It needs Playwright.

python -m je_web_runner -d ./user_journeys --parallel 8 --parallel-mode async --ledger ledger.json
import asyncio
from je_web_runner.utils.async_executor import AsyncExecutor

async def main():
    async with AsyncExecutor(browser="chromium", headless=True) as runner:
        results = await runner.run_many([alice_actions, bob_actions], concurrency=2)  # [(record, failed), ...]

asyncio.run(main())

The WR_apw_* commands act in the file's own context: WR_apw_goto, WR_apw_click, WR_apw_fill, WR_apw_press, WR_apw_text, WR_apw_title, WR_apw_url, WR_apw_wait_for_selector, WR_apw_assert_text, WR_apw_assert_title, WR_apw_new_page / WR_apw_switch_to_page / WR_apw_close_page (tabs of the same user), WR_apw_screenshot, WR_apw_evaluate (behind the arbitrary-script gate), WR_apw_sleep (pauses that file only), and WR_apw_websocket_messages / WR_apw_wait_for_websocket_message for streaming pages. Any other WR_* command runs in a worker thread with the usual gates, retries and failure screenshots, but it shares the process-wide Selenium and Playwright singletons, as in --parallel-mode thread. Outside the async executor a WR_apw_* command fails with a message saying where it runs.

Quality & Security

  • Action linter — WR_lint_action / WR_lint_action_file flag legacy command names, hard-coded URLs, dangerous scripts, missing tags, duplicate consecutive actions.
  • Migration helper — python -m je_web_runner --migrate ./actions rewrites the eleven legacy aliases to their preferred names (--migrate-dry-run reports without writing).
  • Hard-coded secrets scanner — WR_scan_secrets_file / WR_assert_no_secrets catch AWS / GitHub / Slack / JWT / Google / private-key strings before they land in commits.
  • Security headers audit — WR_audit_security_headers_url checks HSTS / CSP / X-Frame-Options / X-Content-Type-Options / Referrer-Policy / Permissions-Policy.
  • Accessibility audit — WR_a11y_run_audit injects user-supplied axe-core (load_axe_source) and runs against the active session; Playwright variant WR_pw_a11y_run_audit.
  • Lighthouse — WR_lighthouse_run shells out to the official lighthouse Node CLI; WR_lighthouse_assert_scores enforces budgets.
  • Page perf metrics — WR_perf_collect / WR_pw_perf_collect snapshot FCP / LCP / CLS / TTFB / domContentLoaded / load via PerformanceObserver, with the LCP element (lcp_element, lcp_url) and the top layout-shift sources (cls_sources); WR_perf_assert_within checks thresholds.
  • Core Web Vitals budget — WR_assert_web_vitals(budgets=None, observe_ms=1000, source="page", url=None, json_path=None, html_path=None) asserts LCP, CLS, INP and FCP in one step. source="page" measures the running backend's current page (Selenium, else Playwright), with INP from the event-timing log (interactions of 104 ms or more before the call still count; INP is not measured when there were none). source="lighthouse" runs Lighthouse on url and reads its lab audits, with Total Blocking Time (tbt) instead of INP. Budgets default to Google's "good" limits (LCP 2500 ms, CLS 0.1, INP 200 ms, FCP 1800 ms); each metric is rated good / needs-improvement / poor. A breach fails the action naming the metric, its rating and the diagnostics (LCP element, shifting elements, slowest interaction); json_path / html_path write the breakdown, pass or fail.
  • Visual regression — WR_visual_capture_baseline + WR_visual_compare (Pillow soft-dep).
  • Snapshot testing — WR_match_snapshot / WR_update_snapshot (text/DOM, unified diff on mismatch).
  • Network throttling — WR_throttle("slow_3g") / WR_pw_throttle("offline"); presets cover Slow 3G, Fast 3G, Regular 4G, Wi-Fi, Offline, no-throttling.
  • HAR diff — WR_diff_har / WR_diff_har_files show added / removed / status-changed requests between two runs.
  • Arbitrary-script gate — executor.set_allow_arbitrary_script(False) blocks WR_execute_script / WR_execute_async_script / WR_pw_evaluate / WR_cdp / WR_pw_cdp for untrusted action JSON.
  • Package gate — WR_add_package_to_executor imports a Python package and registers its functions as commands, so an action file that can name os or subprocess can run anything. executor.allow_packages("name", …) lists the packages it may load (submodules included) and every other package is refused before it is imported, unless the host calls executor.set_allow_arbitrary_packages(True); neither is an action command, so an action file cannot open its own gate.

Extended Capabilities

Reliability & flake reduction:

  • Adaptive retry — je_web_runner.utils.adaptive_retry.run_with_retry(fn, policy=...) replays only failures the classifier marks transient / flaky / environment; real bugs short-circuit.
  • Locator strength scorer — linter.locator_strength.score_locator(strategy, value) ranks locators 0–100; assert_strength fails CI on fragile XPath / TAG_NAME picks.
  • Smart wait — smart_wait.wait_for_fetch_idle and wait_for_spa_route_stable patch window.fetch and history.pushState to detect SPA quiescence — no more time.sleep.
  • Service throttler — throttler.throttle("payments-api") is a file-semaphore that caps cross-shard concurrency on a shared service.

Debugging & observability:

  • Timeline merger — observability.timeline.build(spans=, console=, responses=) merges OTel spans, console messages, and network responses into one chronologically-sorted event list.
  • Failure bundle — failure_bundle.FailureBundle("login_test", error_repr).add_screenshot(...).write("bundle.zip") packages screenshots / DOM / network / console / trace into a single replayable zip with manifest.
  • Memory leak detector — memory_leak.detect_growth(driver, action, iterations=10, growth_bytes_per_iter_budget=...) polls performance.memory.usedJSHeapSize and fails on linear-fit growth above budget.
  • Playwright trace recorder — trace_recorder.TraceRecorder(output_dir="trace-out").start(context, name); …; .stop(context) always writes a .zip viewable with playwright show-trace.
  • CSP reporter — csp_reporter.CspViolationCollector injects a securitypolicyviolation listener and exposes assert_none() / assert_no_directive("script-src").

Test data & determinism:

  • Record/replay fixture — snapshot.fixture_record.FixtureRecorder("fx.json", mode="auto") saves the producer's output the first time, replays it forever after.
  • DB fixture loader — database.fixtures.load_fixture_file("seed.json") + load_into_connection(conn, fixture) seeds testcontainers Postgres / MySQL / SQLite from a {table: [rows]} JSON.

API & contract testing:

  • API mocking — api_mock.MockRouter().add("GET", "/api/users/*", body={"id": 1}).attach_to_page(page) intercepts Playwright routes; URL globs and re: regex patterns supported.
  • Contract testing — contract_testing.validate_response(body, schema) runs a JSON-Schema subset; validate_against_openapi(body, doc, "/users/{id}", "GET", 200) resolves $ref and checks the right schema for the response status.
  • GraphQL helper — graphql.GraphQLClient("https://api/graphql").execute("{ me { id } }"); extract_field(payload, "me.id") plucks values via dotted path.
  • In-process mock services — mock_services.MockOAuthServer().start() issues fake bearer tokens, MockSmtpServer captures sent mails, MockS3Storage is a memory KV.

Security probes:

  • Header tampering — header_tampering.HeaderTampering().set_header("X-Forwarded-For", "192.0.2.1").attach_to_page(page) mutates outbound requests so testers can probe missing-CSRF / wrong-origin / stripped-auth handling.
  • License scanner — license_scanner.scan_text(bundle_text) finds SPDX identifiers and known license phrases (AGPL/GPL/MIT/Apache-2.0/MPL/ISC/BSD) so SBOM gates can assert_allowed_licenses.

Browser & locale:

  • Device emulation presets — device_emulation.playwright_kwargs("iPhone 15 Pro") and apply_to_chrome_options(opts, "Desktop 1080p"); viewport + DPR + UA + touch in one call.
  • Geo / TZ / locale — geo_locale.GeoOverride(latitude=51.5, longitude=-0.13, timezone="Europe/London", locale="en-GB") produces both CDP commands and Playwright new_context kwargs.
  • Multi-tab choreographer — multi_tab.TabChoreographer().open_new(driver, "side", url=...) registers tabs by alias so action JSON can WR_switch_tab("side").
  • WebAuthn virtual authenticator — webauthn.enable_virtual_authenticator(driver) uses CDP WebAuthn.* to simulate passkey / FIDO2 sign-in flows.
  • Cookie consent dismisser — cookie_consent.ConsentDismisser().dismiss(driver) clicks the first matching OneTrust / TrustArc / Cookiebot / Didomi / Quantcast button; selector list extensible via register_selector.

Reporting & CI:

  • PR comment poster — pr_comment.post_or_update_comment("owner/repo", 42, body, token=...) is idempotent via a hidden HTML marker so retried CI runs don't pile up.
  • Trend dashboard — trend_dashboard.compute_trend("ledger.json") buckets the ledger by day; render_html(trend) produces a self-contained SVG line chart + table.

Orchestration & developer experience:

  • Action template library — action_templates.render_template("login_basic", {...}) substitutes {{placeholders}} in built-in flows (login, accept-cookies, switch-locale, close-modal).
  • Diff-aware shard — sharding.diff_shard.select_for_changed(candidates, base_ref="main") filters candidates to those touched by the current branch's git diff.
  • Watch mode — watch_mode.watch_loop(directory, on_change=callback, interval=0.5) re-runs a callback whenever JSON files change.
  • Kubernetes runner — k8s_runner.render_job_manifests(ShardJobConfig(name_prefix="run", image=..., total_shards=8, actions_dir="/actions")) produces one batch/v1 Job per shard.
  • Per-route perf budgets — perf_metrics.budgets.evaluate_metrics("/checkout", {"lcp_ms": 2300}, budgets) plus assert_within_budget(result) enforce route-specific thresholds.

AI assistance:

  • Failure RCA — ai_assist.llm_assist.explain_failure(test_name, error_repr, console=, network=, steps=) asks the registered LLM for {likely_cause, evidence, next_steps, confidence}.

MCP Server

WebRunner ships a Model Context Protocol server so any MCP-aware client (Claude, IDE plugins, etc.) can drive WebRunner over JSON-RPC stdio.

python -m je_web_runner.mcp_server

The default tool list (22 tools) exposes:

Live browser execution:

  • webrunner_run_actions — execute any WR_* action list. Covers all 556 WR_* commands, including the advanced WebDriverWrapper additions: WR_attach_to_existing_browser, WR_execute_cdp_cmd, WR_set_timezone / _locale / _device_metrics / _user_agent / _extra_http_headers / _geolocation / _network_conditions, WR_block_urls / _set_cache_disabled / _set_download_directory, WR_save_cookies / _load_cookies / _clear_origin_storage, WR_save_full_page_screenshot / _print_page, WR_reload(ignore_cache=True), WR_bring_to_front, WR_switch_to_window_by_url|title, WR_new_window / _close_window, page metadata getters, Fetch interception primitives, WR_add_script_to_evaluate_on_new_document, …
  • webrunner_run_action_files — batch-run JSON files on disk
  • webrunner_list_commands — discover the full WR_* surface

Plus the utility tools below (no live browser):

Action JSON authoring & linting:

  • webrunner_lint_action, webrunner_score_action_locators, webrunner_locator_strength
  • webrunner_format_actions, webrunner_parse_markdown, webrunner_render_template
  • webrunner_translate_actions_to_playwright, webrunner_translate_python_to_playwright

Code generation:

  • webrunner_pom_from_html

Quality / triage:

  • webrunner_a11y_diff, webrunner_cluster_failures, webrunner_compute_trend

Security & privacy:

  • webrunner_scan_pii, webrunner_redact_pii

Reporting & contract:

  • webrunner_summary_markdown, webrunner_validate_response

Sharding & infra:

  • webrunner_diff_shard, webrunner_render_k8s, webrunner_partition_shard
from je_web_runner.mcp_server import McpServer, Tool, build_default_tools, serve_stdio

# Or build a custom server
server = McpServer()
for tool in build_default_tools():
    server.register(tool)
server.register(Tool(
    name="my_custom_tool",
    description="…",
    input_schema={"type": "object", "properties": {"x": {"type": "string"}}},
    handler=lambda args: f"hello {args['x']}",
))
serve_stdio(server=server)

The server speaks both MCP eras, decided per request:

  • 2026-07-28 (stateless): a request whose params._meta names io.modelcontextprotocol/protocolVersion (and carries io.modelcontextprotocol/clientCapabilities) is served on its own, with no initialize. Methods: server/discover (supported versions, capabilities, instructions), tools/list, tools/call. Results carry resultType, the server's identity in _meta, and ttlMs / cacheScope where cacheable; another version in _meta gets error -32022 listing the supported ones.

  • 2025-11-25, 2025-06-18, 2025-03-26 and 2024-11-05 (handshake): initialize answers with the client's version when it is one of these, otherwise with 2025-11-25, and also returns the package version and short instructions for the model. Methods: initialize, notifications/initialized, tools/list, tools/call, ping.

  • stdio is UTF-8 with \n line ends whatever the console code page. Anything else the process writes to stdout (print, child processes) goes to stderr, so only protocol messages reach the client.

  • A tool that fails returns a result with isError: true and the error text, so the client can correct its call. webrunner_run_actions and webrunner_run_action_files also set isError when an action failed, and list the failed record keys under failed.

  • An unknown tool or non-object arguments is a JSON-RPC -32602 error. Notifications are never answered, and batch requests are rejected with -32600.

  • The run tools refuse WR_add_package_to_executor, WR_add_package_to_callback_executor, WR_set_allow_arbitrary_script and the WR_ac_* commands that drive or read the desktop (all but WR_ac_available and WR_ac_list_commands), including inside nested action lists: a model must not load os, re-open a script gate the operator closed, or reach the desktop through AutoControl. WEBRUNNER_MCP_ALLOW_UNSAFE_COMMANDS=1 lifts this. When WEBRUNNER_MCP_ROOT is set, webrunner_run_action_files and webrunner_compute_trend only read files inside that directory.

  • Every tool has a title and annotations: the offline tools and webrunner_list_commands are readOnlyHint: true, the two run tools destructiveHint: true and openWorldHint: true. Input schemas describe each argument and reject unknown ones; arguments that do not match come back as an isError result naming the problem. A result that is a JSON object is also sent as structuredContent, and the two run tools declare an outputSchema.

Action JSON LSP

A standard Language Server Protocol implementation for action JSON files:

python -m je_web_runner.action_lsp

textDocument/completion returns every registered WR_* command; textDocument/publishDiagnostics runs the action linter on didOpen / didChange. Pair with VS Code's Configure JSON Language Servers or the JetBrains LSP plugin.

More Capabilities

Smaller modules, grouped by what they help with. The larger areas have their own sections above and in Specialized Modules.

CLI & orchestration polish

  • Regex test selector — test_filter.name_filter.filter_paths(paths, include=["smoke.*"], exclude=["slow"]) keeps only matching candidate paths; orthogonal to the existing tag filter.
  • Process supervisor — process_supervisor.ProcessSupervisor().kill_orphans() walks the OS process table for chromedriver / geckodriver / msedgedriver and kills stragglers (skips os.getpid() automatically). with_watchdog(callable, timeout_seconds=300) wraps a long callable with a hard wall-clock raise.
  • Pipeline DSL — pipeline.load_pipeline({"stages": [...]}) + run_pipeline(pipeline, runner) execute multi-stage gates: continue_on_failure=True makes a stage non-blocking (linters / scanners), otherwise downstream stages skip.

Frontend / mobile / coverage

  • Storybook visual snapshots — storybook.visual_snapshots.capture_story_snapshots(stories, base_url, take_screenshot, navigate, baseline_dir=...) walks every story, persists deterministic filenames (components-button--primary.png), and diffs against an optional baseline. assert_no_visual_regressions(report) is the gate.
  • Appium gestures — appium_integration.gestures ships swipe, scroll, long_press, pinch, double_tap that prefer Appium's mobile: named-gesture extension and fall back to W3C Actions on older drivers.
  • Coverage map — coverage_map.build_coverage_map("./actions") walks every action JSON file, normalises WR_to_url paths (/users/42 → /users/:id) and produces a route → files reverse index. coverage.uncovered(declared_routes) answers "which routes have no test?".

Debugging & reproducibility

  • CDP message tap — cdp_tap.CdpRecorder("cdp.ndjson").attach(driver) wraps execute_cdp_cmd so every command + return value is appended to an ndjson log; CdpReplayer(load_recording(...)) plays it back against a stub for offline debugging.
  • Cross-browser parity — cross_browser.diff_runs([chromium_run, firefox_run, webkit_run]) diffs title / DOM hash / console / network status / screenshot hash, classifying each finding as major (5xx, title, DOM mismatch) or minor. assert_parity(report, only_major=True) is the gate.
  • Browser state diff — state_diff.capture_state(driver) snapshots cookies + localStorage + sessionStorage; diff_states(before, after) lists added / removed / changed keys per section so cart / auth flows stay traceable.

Authoring / scaffolding

  • Page Object codegen — pom_codegen.discover_elements_from_html(html) walks every element with data-testid / id / form name; render_pom_module(elements, class_name="LoginPage") returns a Python module with one TestObject property per element.

CI reproducibility

  • Workspace lock file — workspace_lock.build_lock(drivers=..., playwright_versions={"chromium": "127.0.0.0"}) snapshots every Python distribution + driver version + Playwright browser version; write_lock(lock, ".webrunner/lock.json") and diff_locks(before, after) complete the pipeline.

Long-running observability

  • A11y trend dashboard — a11y_trend.aggregate_history(history) buckets axe runs by day and impact; render_html(points) produces a self-contained SVG line chart so regressions are visible at a glance.
  • Perf drift detector — perf_drift.detect_drift({"lcp_ms": samples}, baseline_window=20, recent_window=5) compares the recent P95 against a rolling baseline P95 and flags drift outside tolerance. assert_no_regression(report) is the strict path; higher_is_better={"frame_rate"} for inverted metrics.

Authoring / formatting

  • Action JSON formatter — action_formatter.format_actions(actions) writes a canonical multi-line array with kwargs in a stable preferred-then-alphabetical order; format_file(path) reformats in place and reports (text, changed).
  • Markdown → action JSON — md_authoring.parse_markdown(text) understands - open <url>, - click #id, - type "x" into <selector>, - wait 3s, - assert title "...", - press Enter, - screenshot, - run template <name>, - quit. Lines that don't match are preserved as WR__note so the round-trip is loss-less.

Triage / production observability

  • Failure clustering — failure_cluster.cluster_failures(failures, top_n=5) reduces each error message to a stable signature (strips timestamps, hex addresses, line numbers, paths, large numerics, quoted substrings) so the same root cause across runs lands in one bucket.
  • Synthetic monitoring — synthetic_monitoring.SyntheticMonitor(alert_sink).register("homepage", check) reruns checks; the sink only fires on edge transitions (green → red / red → green) with failure_threshold / recovery_threshold to silence flapping.
  • OTLP exporter — observability.otlp_exporter.configure_otlp_export(provider, OtlpExportConfig(endpoint="https://otlp:4317")) ships the existing OTel spans to Jaeger / Tempo / any OTLP backend (gRPC by default, HTTP fallback).

Frontend / component

  • Storybook integration — storybook.discover_stories(index_path) reads Storybook 7+ index.json (or legacy stories.json); plan_actions_for_stories(stories, base_url, run_a11y=True) builds a flat action list visiting each story in iframe mode and running axe + screenshot.
  • Shadow DOM auto-pierce — dom_traversal.shadow_pierce.find_first(driver, "button.primary") recursively walks open shadow roots (Selenium execute_script or Playwright evaluate) so a single CSS selector can match across shadow boundaries.

Onboarding / migration

  • Workspace bootstrapper — bootstrapper.init_workspace("my-tests") drops actions/sample.json, .webrunner/ledger.json, pinned-driver template, JSON schema, pre-commit hook, and a starter GitHub Actions workflow.
  • Driver pinner — driver_pin.install_for_browser(".webrunner/drivers.json", "firefox") reads a JSON pin file (name / version / url / archive_format / binary_inside), downloads + extracts once, then serves from cache. Bypasses the GitHub API rate limit that webdriver-manager hits in CI.
  • Selenium → Playwright translator — sel_to_pw.translate_python_source(text) rewrites driver.find_element(By.ID, "x") → page.locator("#x") and similar; translate_action_list(actions) rewrites WR_* action JSON to its WR_pw_* equivalent (drops WR_implicitly_wait since Playwright auto-waits).

Test authoring

  • Form auto-fill — form_autofill.plan_fill_actions(fields, fixture, submit_locator=...) infers each field from data-testid / id / name / placeholder / label / type and emits a ready-to-run WR_save_test_object + WR_element_input sequence.

Quality

  • A11y diff — accessibility.a11y_diff.diff_violations(baseline, current) buckets axe-core findings into added / resolved / persisting keyed on (rule_id, target); assert_no_regressions(diff, allow_rules=...) is the CI gate.

Performance / orchestration

  • Fan-out — fanout.run_fan_out([("preflight-a", task_a), task_b, ...], max_workers=4) runs read-only callables concurrently inside one test, returning per-task duration + outcome with raise_for_failures() for the strict path.
  • Event bus — event_bus.EventBus(".webrunner/events.log").publish("setup-done", {"shard": 1}); subscribers poll() from a remembered offset or wait_for(topic, predicate=..., timeout=30). File-backed ndjson — no Redis dependency.

Browser internals

  • Extension test harness — extension_harness.parse_manifest("./ext") reads MV2 / MV3 manifests; apply_to_chrome_options(options, [ext_dir]) adds --load-extension flags; playwright_persistent_context_args(...) returns the kwargs needed for launch_persistent_context.

Reliability & dev-loop

  • Browser pool — browser_pool.BrowserPool(factory, size=4, max_uses=50).warm(); with pool.session() as ses: … removes browser cold-start from local dev. Health check + recycle policy built in.
  • WebDriver BiDi bridge — bidi_backend.BidiBridge().subscribe(target, "console", callback) works against either Selenium 4 BiDi (driver.script.add_console_message_handler) or Playwright page.on(...). register_translator lets you wire custom event names.

Determinism & offline runs

  • HAR replay server — har_replay.HarReplayServer(load_har("recorded.har")).start() boots a local HTTP server that serves recorded responses; supports literal / glob / re: URL matching with rotation across duplicates. Drop-in for staging-API outages.

Quality / privacy

  • PII scanner — pii_scanner.scan_text(text) finds emails, E.164 phones, Luhn-validated credit cards, US SSN, ROC ID, and IPv4. assert_no_pii(text, allow_categories=...) for CI gates; redact_text(text) returns a sanitised copy.
  • Visual diff review UI — visual_review.VisualReviewServer(baseline_dir, current_dir).start() opens a local web UI showing each baseline / current pair side-by-side with an Accept current as baseline button (idempotent file copy with path-traversal guard).

Test orchestration

  • Test impact analysis — impact_analysis.build_index("./actions") walks every action JSON file and projects locator names, URLs, template names, and WR_* commands into a reverse index; affected_action_files(index, locators=["primary_cta"]) answers "which tests touch this?" so diff-aware shards can go beyond filename matching. build_index("./actions", cache_path=".webrunner_impact_cache.json") keeps each file's names in a cache and parses only what changed: a file with the same modification time and size is reused, and one whose time changed (a fresh checkout) is reused when its SHA-256 matches. Measured on 3000 files of 30 actions: 510 ms uncached, 138 ms cached.

Specialized Modules

A second wave of utility modules, each in its own subpackage under je_web_runner/utils/, organised by capability area. Each module is fully unit-tested and ships independent of the core executor (import only what you use).

Web Platform APIs

  • webtransport_assert — HTTP/3 WebTransport datagram + stream frame recorder with count / payload / JSON-shape / stream-complete assertions (mirror of websocket_assert and sse_assert).
  • indexed_db_explorer — Browser-side harvest JS + typed IdbSnapshot; assertions cover store existence, record count, key presence, index presence, plus per-store diff.
  • file_system_access — JS shim mocking showOpenFilePicker / showSaveFilePicker / showDirectoryPicker; records every write performed against the fake handle for later assertion.
  • notifications_audit — Tracks Notification.requestPermission call timing (user-gesture check, min page age) and policy violations (re-prompt after deny, notification spam after deny, tag reuse).
  • sse_assert — Server-Sent Events stream recorder + chunk-buffer feed + count / data-contains / JSON-shape / strictly-increasing-id assertions.
  • websocket_assert — WebSocket frame recorder + count / payload / pubsub-pattern / JSON-shape assertions.
  • webrtc_assert — PeerSnapshot.from_dict, aggregate_stats (getStats), and connected / track-present / SDP-codec / packet-loss / min-bytes assertions.
  • view_transitions — Instrumentation snippet for the View Transitions API + duration budget / CLS budget / group-name asserts.

Security & Headers

  • mixed_content_audit — HAR + console-message scan for HTTP resources on HTTPS pages (active vs passive vs HSTS-upgrade).
  • clickjacking_audit — X-Frame-Options + frame-ancestors parser + iframe-probe page generator; STRICT / SAMEORIGIN / ALLOWED / MISSING verdict.
  • open_redirect_detector — Eight-payload probe set (//evil, @userinfo, javascript:, data:, mixed-case bypass…) + classifier (BLOCKED / ALLOWED / AMBIGUOUS).
  • sri_verify — Parse <script> / <link rel=stylesheet> tags → validate integrity= strength + crossorigin requirement + recompute hash from a caller-supplied payload provider.
  • coop_coep_audit — crossOriginIsolated page-header check (COOP same-origin + COEP require-corp / credentialless) + per-resource CORP / CORS validator.
  • token_leak_detector — Scan response bodies / HAR / log lines for leaked JWTs (with header validation), AWS / GitHub / Slack / Stripe / Google / generic-bearer tokens. Deduped by token suffix.
  • consent_audit — Cookie catalogue (GA / FB pixel / Hotjar / LinkedIn / Mixpanel / Stripe / Intercom / CSRF / session) + pre-consent + post-reject reintroduction detector.
  • pii_in_screenshot — OCR + PII regex (Luhn-validated card, SSN, ROC ID, IBAN, IPv4, phone, email) over screenshots; reuses ocr_assert for the OCR layer.

Performance Budgets

  • inp_tracker — Interaction-to-Next-Paint instrumentation + p98-INP + good/needs-work/poor rating per Google's thresholds.
  • hydration_check — SSR hydration mismatch detection (DOM diff with framework-attr/comment stripping + console-marker scan covering React / Vue / Svelte / Astro / Nuxt).
  • bundle_budget — HAR → per-AssetKind transfer totals (script / stylesheet / image / font / media) + breach detail + biggest-asset ranking.
  • third_party_budget — Vendor catalogue (GA / FB Pixel / Hotjar / Intercom / Stripe / Segment / Mixpanel / Amplitude / Sentry / etc.) + req / byte / blocking-ms / vendor-count budgets.
  • long_animation_frame — long-animation-frame PerformanceObserver listener + per-script attribution (forced reflow time, pause time).
  • console_error_budget — JS console / unhandled-rejection budget with regex ignore patterns; Selenium and CDP adapters.

Backend Integration

  • grpc_tester — gRPC stub method wrapper + gRPC-Web framing (length-prefix encode/decode + trailer parser) + status assertions.
  • webhook_receiver — Stdlib threaded HTTP server (random port) + wait_for(predicate) polling + path / header / JSON-predicate assertion helpers. Drop-in for "did the app POST a webhook?" tests.
  • idempotency_check — Run a request twice + compare status / body / state / side-effect count. ignore_body_keys, allow_status_change_to for legitimate 409-on-second.
  • pagination_audit — Walk all pages via caller-supplied fetcher; detects duplicates across pages, cursor-loop, off-by-one totals, and sort-order violations.
  • backend_log_correlator — W3C traceparent → fetch matching log lines from Loki / Elasticsearch / JSON-lines file → attach to a failure bundle.
  • email_render — MailHog / Mailpit / .eml capture → cross-viewport screenshot via pluggable render driver.

AI / Workflow

  • failure_narrator — Load failure-bundle directory → LLM-driven natural-language "why this failed" summary → strict JSON envelope → markdown report. LLM client pluggable.
  • repro_minimizer — Classic delta-debugging (ddmin) shrinking a failing action list to its smallest still-failing subsequence.
  • locator_hardener — Heuristic fragility score (nth-of-type / text-xpath / hashed-class / deep-descendant) → LLM-suggested stable selectors with safety filter on the response.
  • test_categorizer — Regex rules over action-name patterns → auto tag: smoke / regression / perf / a11y / security / payment / data_driven / visual / api.
  • exploratory_ai — Agentic exploratory tester with PageObserver + ActionPlanner protocols; ships a deterministic RandomPlanner as fuzz fallback, collects BugSignals from observed errors.
  • story_to_actions — LLM-driven translation of a user story + optional Figma frame hints into validated WR action JSON; validator rejects unsafe action names and bad locator strategies.
  • session_to_test — rrweb / generic-event-stream → WR action JSON; auto-detects input format.
  • test_auto_repair — LLM-driven test rewrite from a failure bundle + git diff context.
  • edge_case_generator — LLM edge-case variant generator (complement to mutation_testing).
  • multimodal_qa — Send screenshot + question to a vision LLM, parse pass/fail/uncertain verdict with confidence floor; useful for UI "is this correct?" checks beyond pixel diff.
  • prompt_drift_monitor — Track an app-internal LLM feature's output drift via baseline embeddings + must_include / must_exclude lexical anchors.
  • test_dedup_ai — Structural (canonical fingerprint) + semantic (cosine clustering with pluggable embedder) dedupe of action JSON files.
  • walkthrough_docs — Generate step-by-step SOP / Confluence-style docs from recorded runs.

a11y / i18n / Visual

  • ocr_assert — OCR-based text assertion (contains / fuzzy / any) for canvas / WebGL / image content; whitespace + accent normalisation built in.
  • screen_reader_runner — Walk an accessibility tree to simulate NVDA / VoiceOver reading order + flag unnamed interactive elements, heading-level skips, missing alt, generic link text.
  • pseudo_localization — Pseudo-localise strings (__éxámplé strîng__) + scan rendered page for hard-coded text leaks. Preserves {name} / %d / <tag> placeholders.
  • forced_colors_mode — CDP-features builder for the four CSS media queries (color-scheme / reduced-motion / forced-colors / contrast) + computed-style diff with "became invisible" detection.
  • visual_ai — aHash / dHash / pHash + SSIM-proxy for canvas / chart visual diff.

Governance & Reporting

  • pr_risk_score — Fuses flake / impact-analysis / locator-health / coverage signals into a 0-100 PR risk score with markdown report and is_blocking gate.
  • flag_matrix — Feature-flag combination matrix with forbid / require constraints, pinned baselines, deterministic sampling, and greedy smallest-failing-subset cover.
  • chaos_hooks — Seeded chaos injection (offline / throttle / mid-flow reload / tab-background) with deterministic schedule per action list.
  • db_snapshot — Per-test DB savepoint/rollback isolation with pluggable backend protocol; ships an InMemoryBackend for unit-testing the workflow itself.
  • time_freezer — CDP injection script that overrides Date / Date.now / performance.now; freeze or slow-motion modes for deterministic time-bound tests.
  • persona_runner — Same suite × N personas (admin / free / enterprise / guest) matrix; summary flags persona-specific vs file-specific regressions.
  • git_bisect_flake — Ledger-only or probe-driven bisect for the regression commit that caused a test to start failing.
  • test_cost_estimator — Per-runner rate-card (Sauce / BrowserStack / LambdaTest / GitHub Actions) × ledger minutes → USD + CO₂ estimate per suite / runner / test.
  • slack_digest — Render Slack Block-Kit + Teams Adaptive Card + plain-text test digest with quarantine activity, top-risk PRs, cost trend, and pass-rate delta.
  • quarantine_age_report — Add fresh / lingering / stale / abandoned tier per quarantined test + escalation alerts.
  • test_debt_dashboard — Scan pytest skip / xfail / TODO + JSON _skip markers + age + CODEOWNERS-derived owner mapping.
  • sla_tracker — Weekly / daily bucketed % of suites finishing under an SLA duration threshold + trend.
  • bug_repro_stability — Repeat a failing probe N times → classify deterministic / flaky / non-reproducible + error-signature grouping + longest pass / fail streak.
  • test_owners_map — CODEOWNERS parser (last-match-wins glob semantics) + per-test override layer + unowned-test audit.
  • failure_triage — AI failure root-cause analysis on failure bundles.
  • flake_detector — Time-decayed flake scoring + quarantine registry.
  • locator_health — Project-wide locator audit + upgrade suggestions.
  • mutation_testing — Action JSON mutation testing (kill-rate / score).
  • live_dashboard — Aggregated web UI: runs + flake + quarantine + locators + schedule + triage (see Observability).
  • test_scheduler — Value-density scheduler under time + cloud budget.

Other Specialised Modules

  • chrome_profile — Persistent Chrome profile + stealth + snapshot / sync-back.
  • device_cloud — Real-device cloud (BrowserStack / Sauce / LambdaTest) connector.
  • otel_bridge — W3C traceparent injection for distributed tracing.
  • otp_interceptor — MailHog / Mailpit / IMAP / SMS OTP polling for 2FA flows.
  • download_verify — PDF / CSV / Excel / JSON / SHA256 download assertions.
  • openapi_to_e2e — OpenAPI / Swagger spec → WR_http_* action JSON generator.
  • cross_tab_sync — Multi-page BroadcastChannel / storage propagation asserts.

Modern web platform & runtime APIs

Modules covering newer browser surfaces that are awkward to drive through plain WebDriver:

  • popover_assert — <dialog> / popover open / close / invoker / "only one modal" assertions.
  • cookie_store_api — Async cookieStore API harvest + change-event assertions + secure-only enforcement.
  • speculation_rules — Speculation Rules (prerender / prefetch) verification, prerendering activation, no-double-fire.
  • web_locks — Multi-tab Web Locks contention harness with deadlock + serialisation + acquired-count assertions.
  • storage_buckets — Storage Buckets API isolation, durability hint, and IDB-per-bucket isolation checks.
  • hydration_streaming — Streaming SSR per-boundary timing (arrival, interactive) + order assertions.
  • web_push_assert — Push subscription VAPID key match, endpoint allowlist, userVisibleOnly, showNotification payload.
  • background_sync_assert — Background Sync register / fire / retry / lastChance (quota-exhaustion) assertions.
  • wake_lock_assert — Screen wake lock acquire / release / leak / re-acquire-on-visibility detection.
  • pip_assert — Picture-in-Picture (video + Document PiP) enter / exit / size assertions.
  • web_share_assert — navigator.share payload recording + fallback-UI assertions.
  • compression_streams — CompressionStream gzip / deflate / brotli round-trip + compression ratio budget.
  • compute_pressure — Compute Pressure API fake observer + app throttle-reaction assertions.

Modern auth, payments, identity

  • webauthn_mock — Deterministic navigator.credentials shim for Passkey / FIDO2 / WebAuthn flows; build canned credentials by user.
  • credential_management — Password / Federated Credential Management API mock + autofill / preventSilentAccess assertions.
  • payment_request_assert — Payment Request API shim + Apple Pay / Google Pay sheet validation (currency, shipping, complete()).
  • three_d_secure_flow — 3-D Secure 2.x branch model (frictionless / challenge / fallback / reject) + silent-finalize detection.

Mobile-web specific

  • touch_gesture — tap / swipe / pinch / long_press CDP-frame builder + event assertions.
  • viewport_audit — Viewport meta + safe-area-inset audit + WCAG 1.4.4 user-scalable audit.
  • virtual_keyboard — visualViewport before / after + keyboard inset CSS variable + focused-element visibility.
  • pull_to_refresh — overscroll-behavior + threshold + refresh handler + network-refetch assertions for PWAs.

LLM / AI feature testing

  • rag_grounding_assert — RAG citation in retrieved set, lexical overlap, unsupported-claim phrase scan.
  • llm_token_cost_tracker — Per-test token / $ ledger with per-model rate card + budget assertion.
  • streaming_chat_assert — TTFT / inter-token gap / UTF-8 cleanliness / duplicate-or-OOS chunk assertions for streaming chat.
  • tool_call_assert — LLM tool / function-call name + ordering + JSON Schema argument validation.
  • hallucination_probe — Ground-truth probe runner + refusal detection + hallucination-rate budget.

Email & notification delivery

  • email_deliverability — SPF / DKIM / DMARC headers + List-Unsubscribe (Gmail/Yahoo bulk rules) + BCC-leak audit.
  • inbox_render_outlook — Outlook (Word renderer) / Gmail / Apple Mail render-compatibility pre-flight findings.
  • push_delivery — FCM / APNs payload size + required fields + PII scan + collapse key + TTL validation.

Performance budgets (cont.)

  • memory_pressure_emulate — CDP memory / CPU pressure emulation profiles + run-under-profile assertions.
  • third_party_block_test — Vendor-by-vendor block-resilience matrix (no-vendor / blocked / passed).
  • bundle_diff_pr — PR bundle delta (added / removed / grew) + growth-gate + markdown report.
  • lcp_image_audit — LCP image preloaded + no loading="lazy" + fetchpriority="high" assertions.
  • font_loading_strategy — @font-face font-display strategy + size-adjust fallback for FOUT / FOIT / FOFT verification.
  • resource_hints_audit — preload / prefetch / preconnect used vs declared + preload as= validation.
  • critical_css_audit — Inline-CSS-in-<head> budget + render- blocking external stylesheet preload audit.
  • lighthouse_regression — Lighthouse score regression vs baseline + Core Web Vitals metric budgets.

Security & headers (cont.)

  • prompt_injection_scanner — LLM jailbreak payload library + canary-leak detection.
  • cors_matrix — CORS preflight matrix probe + credentials / origin policy assertions.
  • oauth_pkce_replay — Confirm authorization server rejects replayed OAuth state / PKCE verifier.
  • cookie_chips_audit — CHIPS Partitioned cookie compliance (third-party requires Partitioned + Secure + SameSite=None).
  • sbom_diff — CycloneDX SBOM diff (added / removed / upgrade / license / vulnerability gates).
  • webhook_signature_verify — GitHub / Stripe / Slack / generic HMAC webhook signature verifier.
  • dom_xss_taint — Lightweight DOM-XSS taint tracking via JS instrumentation + canary detection.
  • csp_violation_parser — CSP report-uri / report-to payload parser + recon-attempt heuristic.
  • hsts_preload_audit — HSTS preload-list compliance (max-age ≥ 1y + includeSubDomains + preload).
  • tls_cipher_audit — Live TLS handshake + version + cipher allowlist + certificate subject check.
  • cookie_scope_abuse — Session-like cookie scope (apex domain / Path=/) + HttpOnly / Secure / SameSite audit.

Backend integration (cont.)

  • graphql_n_plus_1 — N+1 query detector with per-field SQL template repetition + cartesian-fanout heuristic.
  • mq_assert — Kafka / RabbitMQ / SQS-style message-queue publish assertions (drain + matcher + idempotency + ordering).
  • grpc_streaming_assert — gRPC streaming (unary / server / client / bidi) frame count + size + order + half-close assertions.
  • openapi_drift — Live API vs OpenAPI spec drift (undocumented endpoint / method / status, zombie endpoints).
  • api_version_compat — Old-client vs new-server backward-compat matrix on response shape + required request fields.
  • rate_limit_assert — 429 + Retry-After + X-RateLimit-* monotonic + recovery-after-wait assertions.
  • har_to_openapi — HAR → OpenAPI 3.1 reverse engineering (path templates, query params, response schemas).

QA governance & DevX (cont.)

  • failure_auto_tag — Heuristic + LLM failure auto-tagger (flaky-locator / timeout / js-error / network-5xx …).
  • test_self_describe — Reverse-engineer Gherkin Given / When / Then paragraph from action JSON.
  • pr_title_generator — Conventional-Commits PR title from diff + commit history.
  • action_refactor_suggester — Action-JSON refactor smells (hard sleep, positional XPath, duplicated locator, click-wait-click).
  • test_roi_scorer — Find-rate × cost × coverage × recency weighted ROI score per test.
  • pre_merge_gate_dsl — Declarative when / require pre-merge gate rules over a PrFacts snapshot.
  • commit_msg_trigger — Parse [skip ci] / [ci e2e] / [ci shard=3/8] / Closes #123 from commit message.
  • flakiness_graveyard — Quarantine / revive / bury ledger with TTL for stale flaky tests.
  • test_blame_owner — CODEOWNERS + git-blame + HEAD + default → test-owner resolution chain.
  • test_dup_dry — Structural action-JSON duplicate + prefix- overlap detection (extract-helper opportunity).
  • snapshot_diff_approval — Baseline / pending / rejected snapshot register + approval workflow.
  • failure_cluster_dbscan — Failure-message tokeniser + DBSCAN root-cause clustering (pure-Python, no sklearn).
  • test_naming_lint — should_when / given_when_then / camel_subject naming convention linter.

i18n / a11y (cont.)

  • rtl_layout_verify — RTL direction + logical-property (margin-inline-start) + bidi-isolation audit.
  • dst_boundary_test — DST spring-forward / fall-back gap & overlap detection + scheduled-fire model.
  • number_currency_locale — Number / currency / date locale- format assertion helpers (incl. Indian lakh grouping).
  • wcag22_touch_target — WCAG 2.2 SC 2.5.8 target-size auditor with spacing-circle exception.

Emerging-tech device APIs

  • webgpu_pixel_verify — WebGPU canvas pixel readback + mean / solid-colour / tile-diff assertions.
  • webhid_mock — WebHID device shim with input / output report capture harness.
  • webusb_mock — WebUSB device shim with control / bulk transfer capture.
  • webserial_mock — Web Serial UART shim + line-write capture.
  • webcodecs_assert — WebCodecs chunk codec / resolution / keyframe-interval / framerate assertions.
  • speech_api_assert — SpeechSynthesis / SpeechRecognition mock + utterance / language / volume assertions.

For per-module reference also see CLAUDE.md, the auto-generated docs/reference/command_reference.md, and the Sphinx chapter under docs/source/Eng/doc/specialized_modules/.

Advanced WebDriverWrapper

The Selenium wrapper is now composed via mixins under je_web_runner/webdriver/_wrapper_mixins/ (lifecycle / element / wait stay in webdriver_wrapper.py; cookies / actions / media / navigation / scripting are the mixin themes). External imports — webdriver_wrapper_instance, WebDriverWrapper, the _options_dict / _webdriver_dict / _webdriver_manager_dict patch targets — are unchanged.

Stealth / anti-bot launch

from je_web_runner import webdriver_wrapper_instance

webdriver_wrapper_instance.set_driver(
    "chrome",
    options=[
        "--disable-blink-features=AutomationControlled",
        f"--user-data-dir={profile_dir}",
    ],
    experimental_options={
        "excludeSwitches": ["enable-automation"],
        "useAutomationExtension": False,
    },
    extension_paths=["/path/to/extension.crx"],  # optional
    enable_bidi=True,                            # for add_console_listener etc.
    install_driver=False,                        # skip webdriver-manager; Selenium Manager finds the driver
)
webdriver_wrapper_instance.add_script_to_evaluate_on_new_document(
    "Object.defineProperty(navigator, 'webdriver', {get: () => undefined});"
)

Attach to a manually started Chrome (session reuse)

# Step 1 — user starts Chrome themselves:
#   chrome.exe --remote-debugging-port=9222 --user-data-dir="C:/temp/profile"
webdriver_wrapper_instance.attach_to_existing_browser("127.0.0.1:9222")

CDP shortcuts (emulation, network, downloads)

w = webdriver_wrapper_instance
w.set_timezone("Asia/Tokyo")
w.set_locale("ja-JP")
w.set_device_metrics(390, 844, device_scale_factor=3, mobile=True)
w.set_user_agent("Mozilla/5.0 (custom)")
w.set_extra_http_headers({"X-Test-Run": "ci-123"})
w.set_geolocation(35.68, 139.69, accuracy=50)
w.set_network_conditions(offline=False, latency=200,
                          download_throughput=50_000, upload_throughput=10_000)
w.block_urls(["*.doubleclick.net/*", "*.googletagmanager.com/*"])
w.set_cache_disabled(True)
w.set_download_directory("./downloads")
w.clear_origin_storage("https://example.com")    # cookies + localStorage + IDB + cache

Session persistence

w.to_url("https://example.com/")
# … log in, etc. …
w.save_cookies("./cookies.json")

# Later (after browser restart):
w.to_url("https://example.com/")
added = w.load_cookies("./cookies.json")          # → number of cookies applied

Fetch interception primitives

w.enable_fetch_interception(patterns=["*/api/*"])
# In a Fetch.requestPaused event callback (subscribe via CDPEventListener):
w.fulfill_request(req_id, response_code=200,
                  body=b'{"ok": true}',
                  response_headers={"Content-Type": "application/json"})
# Or:  w.continue_request(req_id, url=rewritten_url)
# Or:  w.fail_request(req_id, error_reason="AccessDenied")

Page metadata + multi-tab navigation

w.get_current_url(); w.get_title(); w.get_page_source()
w.get_window_handles(); w.get_current_window_handle()
w.new_window("tab")
w.switch_to_window_by_url("checkout")    # restores original if no match
w.close_window()                         # vs quit() which terminates the driver
w.reload(ignore_cache=True)              # CDP Page.reload — Ctrl+Shift+R equivalent
w.bring_to_front()
w.save_full_page_screenshot("./shot.png")   # full page, beyond viewport
w.print_page("./page.pdf")

W3C BiDi listeners (Selenium 4.16+)

webdriver_wrapper_instance.set_driver("chrome", enable_bidi=True)
sub_id = webdriver_wrapper_instance.add_console_listener(
    lambda entry: print(entry.text)
)
err_id = webdriver_wrapper_instance.add_js_error_listener(
    lambda err: print("page exception:", err)
)
# …later…
webdriver_wrapper_instance.remove_console_listener(sub_id)
webdriver_wrapper_instance.remove_js_error_listener(err_id)

Background CDP event loop (independent module)

CDPEventListener opens its own CDP WebSocket on a worker thread so commands and events share the same target session — required because Selenium's execute_cdp_cmd cannot subscribe to events.

from je_web_runner import CDPEventListener

with CDPEventListener.from_driver(driver) as listener:
    listener.on("Fetch.requestPaused", handle_paused)
    listener.send("Fetch.enable", {"patterns": [{"urlPattern": "*"}]})
    # … drive the browser …

Requires pip install websocket-client (lazy-loaded; a clear CDPEventLoopError is raised if missing).

Performance tracing

from je_web_runner import record_trace

record_trace(
    driver, "perf.json",
    categories=["devtools.timeline", "loading"],
    duration=10.0,
)
# Open perf.json in chrome://tracing or DevTools "Performance".

Cross-browser BiDi network (Selenium 4.16+, Firefox-compatible)

from je_web_runner import (
    bidi_add_request_handler,
    bidi_add_response_handler,
    bidi_clear_network_handlers,
)

sub = bidi_add_request_handler(driver, lambda req: print(req.url))
bidi_clear_network_handlers(driver)

Every method above is also reachable from action JSON via WR_* aliases (WR_set_timezone, WR_save_cookies, WR_enable_fetch_interception, …) so the same surface drives the MCP server too.

Browser Internals

from je_web_runner import (
    selenium_cdp,                 # raw CDP
    pw_emulate, pw_set_locale,    # mobile / locale
)
from je_web_runner.utils.storage.browser_storage import (
    selenium_local_storage_set,
    selenium_indexed_db_drop,
)
from je_web_runner.utils.observability.event_capture import (
    start_event_capture,
    assert_no_console_errors,
    assert_no_5xx,
)
from je_web_runner.utils.dom_traversal.shadow_iframe import (
    selenium_query_in_shadow,
    playwright_shadow_selector,
    selenium_switch_iframe_chain,
)
from je_web_runner.utils.file_transfer.file_helpers import (
    selenium_upload_file,
    wait_for_download,
)
from je_web_runner.utils.extensions.extension_loader import (
    selenium_chrome_options_with_extension,
    playwright_extension_launch_args,
)

Service worker / cache control, console + network event capture and assertions, file upload via element + download dir watcher, browser extension loaders for Chromium-family.

Test Data

from je_web_runner import (
    load_env, get_env, expand_in_action,                   # .env + ${ENV.X}
    load_dataset_csv, load_dataset_json, run_with_dataset, # data-driven + ${ROW.x}
)
from je_web_runner.utils.test_data.faker_integration import (
    fake_email, fake_name, fake_credit_card, fake_value,   # faker
)
from je_web_runner.utils.factories.factory import user_factory, order_factory
from je_web_runner.utils.testcontainers_integration.containers import (
    start_postgres,
    start_redis,
    cleanup_all,
)

Every helper is JSON-callable too (WR_load_env, WR_load_dataset_csv, WR_run_with_dataset, WR_faker_email, WR_user_factory, WR_tc_postgres, …).

Auth & APIs

from je_web_runner import (
    http_get, http_post, http_assert_status, http_assert_json_contains,
)
from je_web_runner.utils.auth.oauth import (
    client_credentials_token,
    bearer_header,
)
from je_web_runner.utils.database.db_validate import (
    db_query,
    db_assert_count,
    db_assert_value,
)

token = client_credentials_token(
    "https://idp.example/oauth2/token",
    "client-id", "client-secret",
    cache_key="default",
)
http_get("https://api.example/users/me", headers=bearer_header(token["access_token"]))
http_assert_status(200)
http_assert_json_contains("role", "admin")

db_assert_count(
    "postgresql+psycopg://user:pw@host/db",
    "SELECT 1 FROM orders WHERE user_id = :uid",
    expected=1,
    params={"uid": 42},
)

OAuth2 helpers cache tokens in-process and refresh 30 seconds before expiry.

Recorder

from je_web_runner import (
    recorder_start,
    recorder_stop,
    recorder_save_recording,
)

recorder_start(webdriver_wrapper_instance)
# … user clicks / inputs in the browser …
recorder_save_recording(
    webdriver_wrapper_instance,
    output_path="./recorded.json",
    raw_events_path="./raw.json",  # optional — debugging
)

The recorder injects a static JS listener (no CDP, no eval), so it works on Chrome / Firefox / Edge alike. Sensitive fields are masked by default — type=password, names matching password / card_number / cvv / ssn / secret / token / api_key / otp / passcode, and 13–19-digit numeric values are replaced with ***MASKED***.

CI / Integrations

from je_web_runner.utils.notifier.webhook_notifier import notify_run_summary
from je_web_runner.utils.test_management.jira_client import jira_create_failure_issues
from je_web_runner.utils.test_management.testrail_client import (
    testrail_send_results,
    testrail_results_from_pairs,
)
from je_web_runner.utils.ci_annotations.github_annotations import (
    emit_failure_annotations,
    emit_from_junit_xml,
)

For GitHub Actions inline annotations, run emit_from_junit_xml("run_junit.xml") after generate_junit_xml_report — failed test cases surface as ::error file=…:: lines on the PR diff.

docker/docker-compose.yml ships a Selenium Grid 4 stack (hub + Chrome + Firefox nodes); docker/.env.example exposes the version pin and concurrency settings.

The IDE config examples under docs/ide/ wire VS Code and JetBrains to the action JSON schema produced by WR_export_action_schema.

Desktop control (AutoControl)

pip install je_web_runner[autocontrol] adds AutoControl (je_auto_control), which drives the real mouse, keyboard and screen. The WR_ac_* commands reach it from an action file, and WebRunner imports it only when one of them runs.

Command What it does
WR_ac_available Whether je_auto_control is installed (looked up, not imported)
WR_ac_list_commands The AC_* commands the bridge will run
WR_ac_run Run one AutoControl action and return its value: ["WR_ac_run", [["AC_write", {"write_string": "hello"}]]]
WR_ac_run_actions Run a list of AutoControl actions in order and return their values; the first failure stops it
WR_ac_fill_native_file_dialog Type a path (made absolute) into the operating system's open or save dialog, then press Enter (submit=false skips it); waits wait_seconds (1) for the dialog first
WR_ac_assert_image_on_screen Fail unless a template image is on the screen; returns its centre [x, y] (detect_threshold is AutoControl's match threshold)
WR_ac_click_element_native Click an element with the real mouse (a trusted click, not WebDriver's): the element selector / by finds, else the current element; mouse_button, and scale for the monitor's display scale; returns the clicked [x, y]
WR_ac_basic_auth Answer the browser's HTTP basic-auth dialog with the credentials in two environment variables (username_env, password_env: names, never values); with url it first opens that page without waiting; then types username, Tab, password and Enter (submit=false skips it) after wait_seconds (1)

A failed AutoControl action fails its WR_ac_* action with AutoControlBridgeError. The bridge refuses the AutoControl commands that reach beyond the desktop: shell commands and programs (AC_shell_command, AC_execute_process), package loading (AC_add_package_*), action lists and files (AC_execute_action, AC_execute_files), AC_run_agent, and calls back into WebRunner (AC_web_*). They are refused anywhere in the action, including loop bodies and bodies passed as JSON strings. The MCP server refuses every WR_ac_* command but WR_ac_available and WR_ac_list_commands unless WEBRUNNER_MCP_ALLOW_UNSAFE_COMMANDS=1.

WR_ac_click_element_native scrolls the element to the middle of the viewport and maps its centre from the page to the screen with the window's screenX/Y, its outer and inner size and devicePixelRatio. Without scale it takes devicePixelRatio as the display scale, which is right at 100 % page zoom; on a zoomed page pass the monitor's scale (1.25 for 125 %). The window must not be covered.

WR_ac_basic_auth exists because a classic get of a page behind basic auth waits, while the browser's dialog is open, until it times out, and chromedriver does not answer the dialog through BiDi. So pass the page as url: the command checks that the page has the keyboard focus (document.hasFocus(); AutoControl types into whichever window is in front, so it refuses otherwise), starts the navigation without waiting, then types into the dialog. The credentials go through AutoControl's AC_write_secret, which types them exactly and keeps them out of its log, its record and its return value; the action file, WebRunner's log and both run records hold only the variable names. It needs a je_auto_control that has AC_write_secret. For Playwright, set http_credentials with WR_pw_set_context_options instead.

Native basic-auth navigation requires HTTPS. Plain HTTP is allowed only for localhost or a literal loopback address used by local test servers. URLs with embedded credentials are refused before navigation or typing.

The native commands refuse a Selenium driver whose window is not on this machine's screen: a headless browser, or a remote one (a grid or a device cloud). A Playwright browser is not checked, because Playwright does not report whether it was launched headless: launch it with headless=False.

The other direction, AutoControl's AC_web_* commands running WR_* commands, is part of AutoControl.

AI Assistance

from je_web_runner.utils.ai_assist.llm_assist import (
    set_llm_callable,
    suggest_locator,
    generate_actions_from_prompt,
)

# Plug in any callable that returns a string:
def my_llm(prompt: str) -> str:
    # call OpenAI / Anthropic / local Ollama / mock
    ...

set_llm_callable(my_llm)

locator = suggest_locator(html_blob, description="primary submit button")
draft = generate_actions_from_prompt("log in as alice and place an order")

WebRunner intentionally ships no built-in LLM client — the boundary is a single Callable[[str], str] so swapping provider is one line.

CLI Usage

# Original entry points (unchanged):
python -m je_web_runner -e actions.json
python -m je_web_runner -d ./actions/
python -m je_web_runner --execute_str '[["WR_quit_all"]]'

# Newer flags:
python -m je_web_runner -d ./actions --tag smoke --exclude-tag slow
python -m je_web_runner -d ./actions --parallel 4 --parallel-mode process
python -m je_web_runner -d ./actions --ledger ledger.json
python -m je_web_runner -d ./actions --rerun-failed ledger.json
python -m je_web_runner -d ./actions --shard 1/4
python -m je_web_runner -d ./actions --watch ./actions
python -m je_web_runner --report run                          # JSON + HTML + XML + JUnit
python -m je_web_runner --validate ./action_smoke.json
python -m je_web_runner --migrate ./actions --migrate-dry-run
python -m je_web_runner -d ./actions --affected-by locator:login_button --affected-by url:/checkout
python -m je_web_runner -d ./actions --changed-since origin/main --impact-cache .webrunner_impact_cache.json

Installing the package also installs two console scripts, webrunner and web_runner, with the same flags, output and exit codes as python -m je_web_runner (webrunner -e actions.json, pipx run --spec je_web_runner webrunner -d ./actions).

Compose any of the flags above; the dispatcher applies impact selection → tag filters → ledger / re-run-failed → sharding → dependency-aware ordering before handing files to the runner.

Impact-based selection. --affected-by KIND:VALUE (repeatable; KIND is locator, url, template or command; @file reads one per line) runs only the action files that use one of those names. --changed-since REF runs the action files git diff REF...HEAD changed, plus every file sharing a locator or template with them, so changing a setup file that saves login_button also runs the tests that use it (URLs and commands are not followed: almost every file shares them). Both can be combined; tags, --rerun-failed and --shard then apply to what they selected. --impact-cache PATH keeps the impact index between runs.

python -m je_web_runner exits with status 1 when an action or a file of the run failed, after the rest has run, and prints error: N action(s) or file(s) failed on stderr. Under TestPioneer, which sets TEST_PIONEER_ARTIFACT_DIR, a report asked for under a relative name (JSON, XML, HTML, JUnit, Allure, interactive HTML, and generate_all_reports) is written below that directory.

Test Record

from je_web_runner import test_record_instance

test_record_instance.set_record_enable(True)
# … perform automation …
records = test_record_instance.test_record_list
# Each record: {"function_name", "local_param", "time", "program_exception"}
test_record_instance.clean_record()

Exception Handling

WebRunner provides a hierarchy of custom exceptions — every helper raises a domain-specific subclass of WebRunnerException:

Exception Description
WebRunnerException Base
WebRunnerWebDriverNotFoundException WebDriver not found
WebRunnerOptionsWrongTypeException Invalid options type
WebRunnerArgumentWrongTypeException Invalid argument type
WebRunnerWebDriverIsNoneException WebDriver is None
WebRunnerExecuteException Action execution error
WebRunnerJsonException JSON processing error
WebRunnerGenerateJsonReportException JSON / XML / JUnit / Allure report error
WebRunnerHTMLException HTML report error
WebRunnerAddCommandException Custom command registration error
WebRunnerAssertException Assertion failure
XMLException / XMLTypeException XML processing error
CallbackExecutorException Callback execution error
PlaywrightBackendError Playwright backend / element failure
PlaywrightLocatorError TestObject → Playwright selector mapping
RecorderError / VisualRegressionError Recorder / visual regression
HealingError / EnvConfigError / DataDrivenError Self-healing / env / dataset
HttpAssertionError / HttpError HTTP API assertions
AccessibilityError / LighthouseError a11y / Lighthouse
NotifierError / JiraError / TestRailError Notifications / test management
CDPError / StorageError / ServiceWorkerError Browser internals
OAuthError / DatabaseValidationError Auth / DB
NetworkEmulationError / LoadTestError Throttling / Locust
ShardingError / MigrationError / ActionLinterError Orchestration / linting
LLMAssistError / OTelTracingError AI / observability

Logging

  • Log file: $WEBRUNNER_LOG_PATH (a full file path), else $WEBRUNNER_LOG_DIR/WEBRunner.log, else ~/.je_web_runner/logs/WEBRunner.log. A relative path resolves against the cwd at import; os.devnull turns the file off.
  • When: the file (and its directory) is created on the first record; importing je_web_runner writes nothing.
  • Mode: append; each line carries the process id; a file over 10 MB is moved to <name>.1 when a process opens it.
  • Level: WARNING+
  • Format: %(asctime)s | %(process)d | %(name)s | %(levelname)s | %(message)s

Supported Browsers

Browser Selenium key Playwright
Google Chrome chrome chromium
Chromium chromium chromium
Mozilla Firefox firefox firefox
Microsoft Edge edge chromium
Internet Explorer ie n/a
Apple Safari safari webkit

Supported Platforms

  • Windows
  • macOS
  • Ubuntu / Linux
  • Raspberry Pi

License

This project is licensed under the MIT License.

About

A framework for Web automation

integration-automation.github.io/WebRunner/

Topics

automationkeyword-driven-testingpythonreport-generatorseleniumselenium-pythonselenium-webdriverselenium4testingweb-testing

Resources

Readme
MIT license

Contributing

Contributing

Security policy

Security policy
Activity
Custom properties

Stars

7 stars

Watchers

1 watching

Forks

2 forks
Report repository

Releases

Packages

Used by

Contributors

Languages