Production readiness

Before decision results are acted on automatically by an application, confidence thresholds must be set, cost must be assessed, and oversized input must be handled.

Setting confidence thresholds

The usual approach is to act automatically when confidence is above the threshold and, when it is below, to route the case to manual handling or to a more capable model for review. The threshold can be determined as follows:

1. Prepare 50–100 real records and label the correct answers manually.
2. Submit all records to Jev and record the answer and confidence for each (for predicate, record probability).
3. Sort by confidence in descending order and identify the cut-off point above which the results are almost all correct. This cut-off point is the threshold.
4. After deployment, periodically sample the records below the threshold that were routed to manual handling, and adjust the threshold or revise the questions as necessary.

Predicate questions can use upper and lower thresholds: for example, values above 0.9 are treated as “yes”, values below 0.1 as “no”, and values in between are routed to manual handling.

Cost and latency

• Only input tokens are billed; output is not billed. Rates are listed on the model page .
• When multiple questions are asked about the same content, placing them in the same request significantly reduces both cost and latency compared with separate calls.
• Streaming is not supported. A single request returns all answers synchronously; one question takes about 0.6 seconds.

Input limits

• Only text and JSON input are supported; images are not supported. Sending an image returns 400.
• The context limit is 64K tokens; exceeding it returns 400 max_tokens_exceeded. Split long documents first, or send only the parts relevant to the questions.
• The maximum request size is 32 MiB; choice supports up to 255 options, and score supports 2–10 levels.