Test a Computer Use API Controller Before Connecting a Model

A Computer Use integration has two independent questions: can the model return supported actions, and can your application execute them without mistaking a success message for a completed task? This tutorial addresses the second question first. The download contains a Python controller and 19 offline tests you can run without an API key, browser session or model charge.

Download the controller kit. Its tests passed with Python 3.9.6, including a repeat from a freshly extracted archive. The included run_live.py adapter has not been executed or validated against a real provider. There are no real API screenshots, usage measurements or endpoint-compatibility results in this article.

Run the tests before choosing an endpoint

Extract the ZIP, enter api-kit, and run:

python3 -B -m unittest discover -s . -p 'test_*.py' -v

Python 3.9 or later and the standard library are sufficient for these offline tests. There are 15 controller cases and four receiver cases. The fake runtime and synthetic model responses do not open a browser or send a request. Their screenshot placeholders are deliberately not real image files; never send them to a provider.

Separate the model, runtime and verifier

The model chooses an action from an observation. The runtime owns the browser and executes supported actions. The verifier inspects an application result independently of the model. In this lab, that result is exactly one new form record containing a unique synthetic address ending in @example.test.

Use the form from the landing-page QA lab. Read /submissions before starting and again during execution. The old record prefix must be unchanged and exactly one matching record must have been appended. An old record, a duplicate or an unrelated address does not pass. A green success banner also does not pass by itself.

The receiver verifier is specific to this fixture. A production workflow needs a result check appropriate to its own system, such as a saved draft ID. Do not generalize this form rule to arbitrary websites.

Follow one computer-tool contract

The reference code follows the structured-action path described in the OpenAI Computer Use guide. The controller sends an initial screenshot, accepts an ordered computer_call action batch, executes supported operations, and returns a screenshot as computer_call_output with the matching call_id. The next request also carries previous_response_id.

Those identifiers connect an observation to the action request it answers. A screenshot without the correct call ID is not a substitute for tool output. The source is a protocol reference, not proof that the unexecuted adapter is compatible with a specific model today.

Before live use, verify the exact endpoint, model and tool schema together. A text request succeeding on an OpenAI-compatible endpoint does not establish computer-tool support. This kit makes no claim of Ofox endpoint compatibility and selects no default provider or model.

Validate actions before execution

The controller supports a deliberately small action set: left click, bounded text input, selected single-key presses, bounded scroll and screenshot. Coordinates must fit the runtime’s screenshot dimensions; booleans, non-finite numbers and malformed payloads are rejected. Text is limited to 200 characters, a response to 12 actions, and the default run to four model turns. Unknown actions stop execution.

The controller accepts one computer call per response. It rejects repeated call IDs, incomplete responses and pending safety checks instead of silently accepting them. This is a teaching implementation; it does not implement every tool action or a complete approval system.

Completion is checked before and after each action. That matters when a submit action finishes asynchronously: the next action must not blindly click again after the record already exists. The reference browser adapter also polls the receiver briefly after click and keypress actions. None of these local checks proves that a real browser integration works; that requires a separate test.

What the offline tests establish

The tests cover malformed and out-of-bounds actions, unsupported input, call/output correlation, turn limits, repeated calls, early stopping and exact receiver changes. Regression cases check that a verified completion stops later actions in the same batch and that duplicate or wrong-address records fail verification.

For completed responses with a valid response ID, the audit retains reported usage, including a final response without actions, and records attempted actions if execution fails. Synthetic usage is test data, not observed billing. Exceptions carry the accumulated controller audit where implemented; the code does not promise an artifact for every possible startup failure.

You can inspect each test input and expected result in the source. These checks establish local controller behavior for the listed cases. They do not establish model accuracy, visual quality, browser reliability or a real-world success rate.

What is required for a real run

The unvalidated run_live.py adapter uses a fresh Playwright Chromium context, a loopback-only fixture origin and an explicitly configured HTTPS Responses endpoint. It does not attach your normal browser profile. It routes ordinary page requests through a fixture-origin allowlist and stops on an unexpected page origin or tab. This is a controlled-fixture safeguard, not a general browser security sandbox. It does not automatically retry failed model requests.

Live dependencies have not been installed and pinned as tested. Follow the Playwright installation documentation in a dedicated environment, then record the actual package and browser versions. Supply COMPUTER_RESPONSES_URL, COMPUTER_MODEL and COMPUTER_API_KEY through your environment; do not put a secret in the archive or repository.

Real execution needs permission for the browser environment and any API spending. Do not use this adapter to bypass an existing browser access denial. Use a new synthetic address and a new output directory for every attempt. The README shows the invocation shape with an explicit acknowledgement flag.

Four calls and an output-token cap are not a dollar budget. Confirm current provider rates and an account-side spending limit separately. A timeout can still correspond to billed work; inspect provider usage and receiver state before deciding whether to retry.

Accept a live integration only with its own evidence

For an eventual live test, preserve the model ID, endpoint host, runtime versions, task, redacted raw responses, call IDs, screenshots and receiver result. Review logs before sharing. Record failures as failures, including unsupported tools and incomplete tasks; do not silently switch to text-only output and call it Computer Use.

Run a second attempt from a clean environment with a different synthetic address. Report both outcomes and observed usage separately. Until those runs exist, the defensible result is narrow but useful: the downloadable controller passed 19 offline tests, and the provider/browser connection remains unverified.