DeepSeek, Gemini or Qwen for multi-turn tools? A small reproducible test

DeepSeek V4.1 Flash, Gemini 3.8 Flash and Qwen3.8 Flash all completed our synthetic order lookup and recovered from one injected retryable tool error. That is a useful starting result, but it did not produce a reliability winner. Each model ran each case once, and all final responses included Markdown fences rather than bare JSON.

The practical choice is therefore to validate the complete tool loop on your application’s contract: correct arguments, correctly matched tool results, bounded retries, an acceptable final answer and recorded usage. A model returning a tool call is only the beginning of that loop.

This article gives the task, observed results and a runnable test kit. It extends the general DeepSeek, Gemini and Qwen comparison with real, narrowly scoped API evidence. The order data and failure were fabricated; the model requests were real.

The exact workflow

The user asks for order A17. The only available tool, lookup_order, accepts a string order_id. Our local function returns a fixed, nonsensitive record:

{"order_id":"A17","status":"shipped","amount_eur":42,"next_action":"track_delivery"}

In the normal case the first call succeeds. In the fault case the first tool result is instead:

{"error":"temporarily_unavailable","retryable":true}

The next valid lookup returns the order. This is an application-level tool failure, not a simulated HTTP 429 or provider outage. Keeping that distinction prevents a successful retry here from being advertised as evidence about the model service’s incident recovery.

Our prompt asks the model to use the tool, avoid guessing, retry a retryable error exactly once and return JSON with four named fields. The harness permits at most four model requests per case. It supplies no tool that can charge a card, send a message or modify an order.

Actual model-page code interface captured October 12. This is a basic connection example, not a screenshot of the multi-turn evaluation.

What we sent and measured

All three models used Ofox’s /v1/chat/completions route with max_tokens: 1800. We used exact namespaced IDs:

deepseek/deepseek-v4.1-flash
google/gemini-3.8-flash
qwen/qwen3.8-flash

No explicit temperature or reasoning configuration was set. This tests the gateway’s current defaults; it is not a controlled comparison at equal reasoning budgets. It also does not test the providers’ direct native endpoints.

Times sum the client-observed durations of model requests in each workflow. They are not a latency percentile or pure model inference time. Token totals sum the API usage across turns, including repeated history; tokenizer and accounting differences mean fewer reported tokens do not necessarily imply less work or a cheaper bill.

An initial DeepSeek attempt encountered a transport-level proxy disconnect before the completed suite was run. It is recorded separately and excluded from this table. That exclusion must not be mistaken for a zero-failure availability result. The table describes completed cases, not every network attempt made during preparation.

What passed, and what did not

Each completed normal case made one valid lookup. Each completed fault case made two valid lookups, allowing the injected first error to recover. All final answers included the expected order values. None invented an order status before receiving the successful result.

However, all six answers were fenced as Markdown. In the DeepSeek retry case, the answer also appended a sentence explaining the retry. If the application expects to parse the entire text as one JSON value, these outputs fail that strict contract despite containing correct values.

This matters when choosing a model for automation. A human-facing support assistant may accept a fenced answer. A background worker expecting a JSON object may not. Define acceptance before measuring, and keep semantic correctness, syntax and tool behavior as separate columns.

Preserve the assistant message between turns

The harness appends the returned assistant message before appending each tool result. Do not reconstruct the history from only the final visible text: the tool-call IDs and any provider-specific continuation fields may be needed for the next turn. In this gateway test, continuation metadata was present and was retained in the next request.

The core sequence is:

message = response["choices"][0]["message"]
messages.append(message)

for call in message.get("tool_calls", []):
    # Validate name and arguments before executing any application action.
    result = lookup_validated_order(call)
    messages.append({
        "role": "tool",
        "tool_call_id": call["id"],
        "content": json.dumps(result),
    })

lookup_validated_order represents the application’s validation boundary, not arbitrary execution of a name returned by the model. Our downloadable implementation permits only lookup_order with exactly {"order_id":"A17"}. A production implementation should check the requested record against the authenticated user’s authorization too.

For native provider integration, use its own message and tool-result contract rather than blindly copying this OpenAI-compatible representation. The DeepSeek tool-call guide and Gemini function-calling documentation describe those separate interfaces. Passing this gateway test does not certify a native adapter you have not run.

Reproduce the test before expanding it

Download and unzip the test kit, inspect the script and install requests in an isolated Python environment. Set OFOX_API_KEY through your normal secret-management method. The script makes billable model requests; no free-credit assumption is built in.

Run the tool suite and save results to a new directory:

python3 -m venv .venv
source .venv/bin/activate
python -m pip install requests
python tool_suite.py --output ./tool-results

The script stops each case at the configured turn bound, records HTTP or transport failures, and retains failed cases. Read the response text and validate its structure instead of treating HTTP 200 as an application pass. Do not publish raw continuation metadata or credentials when sharing results; the supplied evidence removes reasoning fields.

Then replace the single fixture with your own labeled cases. Add unknown orders, missing arguments, multiple tool calls, invalid JSON arguments, a non-retryable error and a tool that times out. Use read-only tools while establishing the protocol. Side-effecting tools need additional controls such as idempotency and explicit authorization; a retry instruction alone does not make them safe.

Make the selection on accepted work

For a small integration trial, all three are reasonable candidates because each completed this particular loop. The observed timing order is too fragile to turn into a production ranking. Repeat tasks at the input lengths, concurrency and reasoning settings your application will use.

Record accepted-task rate, end-to-end latency, request count and reconciled spend. Include failed initial attempts, format-repair attempts and fallbacks in task cost. Keep a separate measure for user-visible correctness so cheap but invalid results cannot inflate the apparent success rate.

Before deploying, enforce maximum turns and per-tool retry limits in the application. The model followed our one-retry instruction, but your service should not depend on that happening forever. Keep provider errors separate from tool errors and output-validation failures; each calls for a different response.

If your next problem is structured extraction rather than a tool loop, the Haiku migration evaluation shows why syntax, shape and facts deserve separate checks there as well.