PRODUCT · RUN
RunningOps
Write the test as a prompt. Run it on any model, at any scale.
LLM-based test automation that runs web, Android, iOS and API tests from plain-language prompts on a fleet of real and virtual devices, in the cloud, a private cloud or on-prem.
- Who it is for
- Test automation teams that want prompt-driven runs at the scale of a regression suite, with control over which LLM runs them and where the data lives.
- Last updated
What it is
RunningOps is an LLM-based test execution platform. A test is written as a prompt and run on web, Android, iOS or API targets; runs are queued, paced and self-healed at suite scale, every result carries screenshots, video and the agent's reasoning, and the device pool mixes virtual device twins with physical phones and real SIMs.
What it does
Batches and suites
Submit up to 200 runs in one call, group them into suites and read every case's result, evidence and cost.
Self-healing, honest results
Engine faults and lost sessions are retried automatically; every run ends as passed, test failure, environment or platform fault, never a fake green.
Bring your own model
Anthropic Claude, OpenAI, Azure OpenAI, Google Gemini, DeepSeek, Qwen, Kimi, Ollama or any OpenAI-compatible endpoint, with outage-aware failover.
LLM cost control
Tokens and dollars on every step, model priority by cost, success or speed, zero tokens for unchanged passing scenarios, and a monthly budget cap.
Real devices and SIMs
470+ device VMs across 29 makers plus your own phones on your hosts; a run leases a number, reads the OTP from the SIM and continues.
From prompt to script
A passing scenario replays as a Playwright script or exports to Selenium, Testinium (Gauge) and Maestro for your own pipeline.
How it works
- 01
Describe the test
Write the scenario in your own words; the engine is selected automatically for web, mobile or API.
- 02
Run at scale
Runs are queued and paced per organisation; you watch the browser or phone screen live as each step completes.
- 03
Keep the evidence
Screenshots, video, logs and the agent's reasoning stay with every run; failed steps can be fixed by AI and resumed.
Where it sits in the lifecycle
RunningOps executes the suites at the Build stage; TestingOps hands it the cases, DeviceOps supplies the virtual devices it runs on.
Where it runs
Cloud with nothing to install; private browser engine on AWS, GCP, Azure or any cloud with artifacts in your own storage; on-prem engines and device hosts inside your network with self-hosted LLMs (Ollama, vLLM).
Works with
Browser Use Cloud, Browserbase, BrowserStack, Sauce Labs, LambdaTest, Testinium DevicePark, your own Appium server; ten LLM providers; Playwright, Selenium, Gauge and Maestro export.
Frequently asked questions
- What does 'run a test from a prompt' mean?
- You describe the scenario in plain language; an LLM agent drives the browser or device step by step, checks the outcome and records evidence. API steps run without a browser or a model.
- Which LLMs can run my tests?
- The platform's Claude models or your own keys for Anthropic, OpenAI, Azure OpenAI, Google Gemini, Browser Use, DeepSeek, Qwen, Kimi, Ollama and OpenAI-compatible endpoints.
- How does RunningOps keep LLM costs down?
- Every step reports tokens and cost, runs can pick the best model from your own history, scenarios that already passed replay their script with zero tokens, and a monthly budget cap warns at 80 percent.
- Are the mobile devices emulators?
- No. The pool runs real Android on virtual device twins shaped like customers' phones, and physical phones with real SIMs sit on your own hosts. One prompt runs on both.
- Can it run where our data must stay?
- Yes. Private cloud engines run in your own AWS, GCP or Azure account, and on-prem engines and device hosts run inside your network through an outbound tunnel.
Other products in the family
See it on your own stack.
A short call with the team: your setup, your volumes, where the product would run.