Dummy Run: letting fictional users try your app first
Dummy Run is an open-source tool in which AI personas use your app in a real browser and report what broke, with the evidence to back it up.
Dummy Run is an open-source tool that lets AI act as your users and try your application before your customers do. It is available on GitHub under the Apache 2.0 licence. The name is British English for a trial run; here, the dummies do the running.
The problem #
Most teams now build a preview environment for every pull request. In principle, someone checks each preview before it is merged. In practice, that check is often brief, it follows the expected path, and it is done from a single account. Few people test as a user with restricted permissions, a different device or an unusual way of working, which is where many real problems appear.
How it works #
Dummy Run assigns that work to a cast of personas. Each persona is a fictional user with a role, a set of permissions, goals, habits and a device. A run signs one persona in to the preview in a real Chrome or Firefox browser. The persona works towards its goals and reports:
- what broke
- what it found confusing
- anything its role should not have been able to see or do
Findings backed by evidence #
AI-generated test results are only useful if they can be trusted, so Dummy Run is built around evidence. Each run records its steps, screenshots and logs. A reviewer checks every claim against that evidence and the code, and only claims that hold up reach the results board.
Nothing is raised automatically. Findings are published to the board, and a person decides what becomes a ticket.
Data residency #
Teams choose their model provider: Azure OpenAI, Anthropic, Gemini, Amazon Bedrock, or any OpenAI-compatible service, including a model hosted locally. A provider can be set per market, so runs for a UK client remain in the UK and runs for an EU client remain in the EU.
Observability #
Every run is recorded as an OpenTelemetry trace, with a span for each step and an event for each finding. Synthetic users can therefore be viewed alongside real traffic in an existing observability stack. A Grafana integration provides dashboards, and a local stack can be started with a single docker compose up.
Who it is for #
- QA leads who want role-based exploratory testing on every pull request, without maintaining scripts.
- Developers who want to see their product used as a customer would use it.
- Agencies that run one cast against the previews of several clients.
- Product and UX teams who want points of friction tied to a specific step and screenshot.
- Compliance and security teams who need to know what each role can see or do beyond its permissions.
Getting involved #
Documentation is at dummyrun.dev and the source code is at github.com/dummy-run/dummy-run. Issues and discussions are open, and feedback from teams trying it on their own applications is very welcome.