Reviewing the CometAPI chat contract after an incident

Last reviewed: 2026-05-09.

Who this is for: operators, backend engineers, and incident reviewers who need to decide whether a CometAPI chat completion issue came from client code, configuration, request shape, response parsing, or an undocumented assumption.

For broader integration context, see the CometAPI tutorials index and the tutorials post archive.

Key takeaways

• Treat the public CometAPI Chat Completions API reference as the baseline contract, then compare it with the exact request and response captured during the incident.
• Preserve one redacted raw exchange before SDK retries, wrappers, or application error handlers reshape the evidence.
• Separate contract validation from availability validation: a successful retry does not prove the original request was valid.
• Do not assume rate limits, billing behavior, model availability, or guaranteed outcomes from a single endpoint reference page. Verify those in account-specific materials.
• Close the review with client-side assertions: path, auth header, required fields, response parser behavior, error classification, and logging redaction.

Concise definition

A chat completions contract is the set of API behaviors your client depends on: endpoint path, HTTP method, authentication headers, request field names and types, response shape, error behavior, and any documented usage or limit signals. During an incident review, the goal is not to prove blame; it is to identify which assumption failed.

When to use this review

Use this checklist after incidents such as:

• A deploy started returning validation errors for chat completion calls.
• A parser failed because a response field was missing, null, renamed, or shaped differently than expected.
• A retry storm followed a 4xx or 5xx response.
• Streaming and non-streaming paths behaved differently.
• A billing, token budget, or usage-reporting assumption influenced production routing.
• A model or gateway configuration change caused requests to hit a different path or base URL.

This article intentionally focuses on the API contract review, not generic smoke testing or fallback routing.

Evidence to preserve before analysis

Before changing code or replaying traffic, capture one representative failing exchange and one successful exchange from the same time window when possible.

Record:

1. UTC timestamp.
2. Environment, service name, commit SHA, and client library version if applicable.
3. HTTP method and full path after base URL expansion.
4. Request headers with secrets redacted.
5. Request JSON with user content sanitized.
6. Response status code.
7. Response headers relevant to request IDs, content type, limits, or retries if present.
8. Response body with sensitive content redacted.
9. Application-side exception class and message.
10. Retry count and whether the final user-visible result came from the first call, a retry, or a fallback.

Do not paste real customer prompts, API keys, or account identifiers into the incident ticket. Use a sanitized reproduction payload instead.

Sanitized reproduction request

Confirm the exact endpoint path and required fields against the current CometAPI reference before using this. The example uses placeholders and keeps the prompt non-sensitive.

curl -sS "${COMETAPI_BASE_URL}/v1/chat/completions" \
  -H "Authorization: Bearer ${COMETAPI_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "replace-with-approved-model",
    "messages": [
      {
        "role": "system",
        "content": "Return one short sentence."
      },
      {
        "role": "user",
        "content": "Contract probe for incident review 2026-05-09."
      }
    ],
    "stream": false
  }'

Validation notes:

• Use a non-production key or a restricted test key when possible.
• Replace the model value with one approved for your account and workflow.
• Keep optional fields out of the first reproduction unless the incident depends on them.
• If your application uses streaming, run streaming and non-streaming probes separately because the parser contract is different.
• Store the raw HTTP response before converting it into your application’s internal message object.

Contract details to verify

Incident review workflow

1. State the failed assumption

Write the suspected failed assumption as a sentence:

• “The client assumed the response always contains a non-empty assistant message.”
• “The retry layer treated a validation error as transient.”
• “The deploy assumed the configured base URL already included /v1.”
• “The usage parser assumed token accounting was always present on error responses.”

Avoid vague labels such as “CometAPI failed” or “SDK issue” until the contract comparison is complete.

2. Compare observed traffic with the documented contract

Use the CometAPI Chat Completions API reference as the public contract source. For each captured request, compare:

• HTTP method.
• Resolved path.
• Authentication header.
• Content type.
• Required JSON fields.
• Message role and content structure.
• Optional generation controls your client sends.
• Streaming flag or streaming endpoint behavior, if used.
• Expected response envelope.
• Error response handling.

If the incident involved multiple services, perform this comparison for each caller. A shared wrapper may hide divergent configuration.

3. Reproduce with the smallest safe request

Start with the smallest valid request your account supports. Then add the production fields back in groups:

1. Required request only.
2. Add application metadata or tracing fields if used.
3. Add generation controls.
4. Add tool, function, JSON-mode, or streaming features only if they were part of the incident.
5. Add the same timeout and retry policy used in production.

The first step where behavior changes is the best candidate for a contract mismatch.

4. Validate parser behavior, not just HTTP success

A 200 response can still fail the application if the parser assumes too much. Check whether the client:

• Handles empty or missing content according to documented behavior.
• Separates success responses from error responses before parsing.
• Treats usage fields as optional unless your source evidence says otherwise.
• Does not parse streaming chunks with the non-streaming parser.
• Logs enough raw shape information to debug without exposing sensitive content.

5. Classify the incident outcome

Use one of these closure labels:

Practical validation checklist

Before closing the incident, verify:

• The resolved production endpoint path matches the current public reference.
• The auth header is present, correctly formed, and redacted in logs.
• The request body contains only fields your integration intends to send.
• Required fields are validated before dispatch.
• Optional fields are covered by tests or feature flags.
• Streaming and non-streaming code paths have separate fixtures.
• The response parser handles documented success and error shapes.
• Retry policy does not retry deterministic validation failures.
• Timeout values are documented as local client settings, not API guarantees.
• Rate-limit and billing assumptions are verified from account-specific sources.
• The incident ticket includes a sanitized request, response, timestamp, and owner for each follow-up.

Sources checked

FAQ

Is this the same as a smoke test?

No. A smoke test asks, “Can one request succeed right now?” A contract incident review asks, “Which documented or assumed API behavior did the client depend on, and did the observed exchange match it?”

Should we replay the exact customer prompt?

Usually no. Preserve the original evidence securely, then create a sanitized reproduction prompt that exercises the same request shape without exposing customer data.

Can a successful retry close the incident?

Not by itself. A retry may hide a request-shape problem, parser bug, timeout, or error-classification issue. Close the incident only after identifying why the first attempt failed or documenting why it cannot be determined.

Can we assume rate limits or billing behavior from the endpoint reference?

Do not assume universal thresholds or billing behavior from a single endpoint reference unless it explicitly documents them. Check account-specific usage records, contracts, dashboards, invoices, or support responses.

What should we add to CI after this review?

Add schema checks for the request builder, response parser fixtures from sanitized real responses, configuration tests for base URL expansion, and negative-case tests for auth and validation errors.

Where should related tutorial updates go?

Use the CometAPI tutorials post archive for implementation guides and the editorial notes area for publishing or maintenance context.