devs/cproof
1
2
Fork 1
Code Issues 43 Pull Requests 6 Actions Packages Projects 1 Releases Wiki Activity

WIP: feat(ai): support both responses and chat-completions APIsfeat/ai-api-type #163

Draft
jabber.developer2 wants to merge 4 commits from feat/ai-api-type into master
pull from: feat/ai-api-type
merge into: devs:master
Conversation 0 Commits 4 Files Changed 12 +1229 -174
jabber.developer2 commented 2026-07-08 17:05:16 +00:00
Collaborator
Copy Link

Summary

The chat-completions alignment (9913344bb) fixed the customer's chat-completions-only provider but broke Perplexity, which serves only the responses API. This PR drives both flavours from one code path: requests default to /v1/responses and fall back to /v1/chat/completions when the provider reports the endpoint missing or rejects the payload shape, plus a manual per-provider override via /ai set api-type <provider> responses|chat-completions|auto.

Design

  • AIApiType enum (auto / responses / chat-completions) on the provider: api_type is the configured value (persisted as api_type= in [ai/<provider>]), resolved_api_type is a runtime-only cache of the detected flavour, and api_epoch is a generation counter bumped on any URL or api-type change so a stale in-flight detection cannot overwrite a newer decision.

  • Request flow: an explicitly pinned flavour is used as-is. In auto, the request tries the preferred flavour first and falls back to the other on an endpoint-missing status (404/405/501) or a payload-shape rejection (400/422). A previously detected flavour is treated as a first-attempt hint, not a hard pin — the other flavour stays available as a fallback, so a backend that changes flavour self-corrects within the request instead of wedging on the cache.

  • One request, one prompt: history and custom settings are serialized once per request and the per-flavour envelope is assembled per attempt under a single session-lock acquisition, so the fallback retry cannot double-count the prompt.

  • Response parsing (ai_parse_response_typed()): dispatch on the request flavour, then per-envelope extractors — chat completions anchored on choices[].message.content; responses on the output_text part with a string-aware backward "text" scan bounded to that part's own JSON object, so a reasoning summary or a truncated body is never returned as the assistant reply. ai_parse_response() is the AUTO entry point over the same ordered pipeline (no extractor runs twice).

  • Endpoint detection: wrong-model errors are classified from the structured error.code/type/message when a JSON error envelope is present (and a bare "does not exist" must mention a model), so route-missing 404s from gateways/proxies no longer suppress the fallback. When both flavours fail, the first payload-rejection error (the actionable 400/422) is preserved and surfaced instead of the second endpoint's bare 404; an unparseable 2xx re-probes.

  • Thread safety: api_url is snapshotted under settings_lock in both the request and models-fetch threads, and every resolved_api_type write goes through one locked helper with an epoch recheck — closing a use-after-free and a stale-cache race against a concurrent /ai set provider. The models-fetch provider reference is taken on the main thread (while the provider is guaranteed live) and released on every worker exit path.

  • Reserved custom-setting keys extended with input so /ai set custom cannot collide with the responses payload; store stays writable as a legitimate chat-completions parameter.

  • UX: /ai set api-type with provider-name and value autocompletion; /ai and /ai providers show the configured type, or auto (detected: …) once probed; help/synopsis/examples updated.

Commits

  1. 14f76ab1d feat(ai): support both responses and chat completions APIs — the whole feature as a single unit (enum + provider fields, the attempt loop with fallback, the parameterized payload builder, the dual-format parser, the epoch/lock protocol, persistence, command wiring and tests). Touches src/ai/ai_client.{c,h}, src/config/preferences.{c,h}, src/command/{cmd_funcs,cmd_defs,cmd_ac}.c, tests/unittests/.

  2. b3a269342 fix(ai): repair custom setting name autocompletion — an independent pre-existing bug (below), kept separate because it does not touch the dual-API path.

Follow-up commit: custom setting name autocompletion (b3a269342)

The /ai set custom <provider> <setting> name completion never worked since its introduction in 9e5dfb14f: the prefix "/ai set custom " carried a trailing space that autocomplete_param_with_ac() doubled, the chosen primitive completes the first token after the command while the setting name is the second, the num_args == 3 guard excluded partially typed names, and the static list (tools/search/memory/plugins) predated the backend. Replaced with token-position completion (autocomplete_param_no_with_func, arg 5) over a list rebuilt on demand: common payload parameters plus the provider's currently set keys via the new ai_get_provider_setting_keys() (snapshot under settings_lock). The unused parse_args() call whose NULL result hit g_strv_length() unguarded (glib CRITICAL on TAB at /ai or at the 5-arg value position) is gone with the dead branch.

Testing

  • gcc -fsyntax-only clean on all touched translation units; full make/make check not run yet — please run before merge.

  • Behavioural coverage is unit-level (parser incl. reasoning-leak / brace-in-text / logprobs-before-content / literal-"text" regressions, settings, persistence). The fallback path still needs a live smoke test against OpenAI (both endpoints), Perplexity (responses-only), and the customer's chat-completions-only provider.

Notes / open questions

  • The original spec (2026-07-04): default to responses; on 404, fall back to chat/completions; plus a manual /ai set api-type MY_PROVIDER chat-completions|responses. Implemented as specified, with one deviation to confirm: the fallback also triggers on a payload-shape rejection (400/422), not only on 404. Rationale: a flavour-specific custom setting or envelope shape would otherwise wedge every request with no recovery. If you'd rather keep the fallback strictly 404-only, this is a one-line narrowing of the endpoint-missing predicate.

  • The fallback probe costs one extra round-trip only when the first-attempt flavour is wrong; once a flavour answers it is reused as the hint for the rest of the session (not persisted across restarts).

## Summary The chat-completions alignment (`9913344bb`) fixed the customer's chat-completions-only provider but broke Perplexity, which serves only the responses API. This PR drives both flavours from one code path: requests default to `/v1/responses` and fall back to `/v1/chat/completions` when the provider reports the endpoint missing or rejects the payload shape, plus a manual per-provider override via `/ai set api-type <provider> responses|chat-completions|auto`. ## Design - **`AIApiType` enum** (`auto` / `responses` / `chat-completions`) on the provider: `api_type` is the configured value (persisted as `api_type=` in `[ai/<provider>]`), `resolved_api_type` is a runtime-only cache of the detected flavour, and `api_epoch` is a generation counter bumped on any URL or api-type change so a stale in-flight detection cannot overwrite a newer decision. - **Request flow:** an explicitly pinned flavour is used as-is. In `auto`, the request tries the preferred flavour first and falls back to the other on an endpoint-missing status (404/405/501) or a payload-shape rejection (400/422). A previously detected flavour is treated as a *first-attempt hint*, not a hard pin — the other flavour stays available as a fallback, so a backend that changes flavour self-corrects within the request instead of wedging on the cache. - **One request, one prompt:** history and custom settings are serialized once per request and the per-flavour envelope is assembled per attempt under a single session-lock acquisition, so the fallback retry cannot double-count the prompt. - **Response parsing** (`ai_parse_response_typed()`): dispatch on the request flavour, then per-envelope extractors — chat completions anchored on `choices[].message.content`; responses on the `output_text` part with a string-aware backward `"text"` scan bounded to that part's own JSON object, so a reasoning summary or a truncated body is never returned as the assistant reply. `ai_parse_response()` is the `AUTO` entry point over the same ordered pipeline (no extractor runs twice). - **Endpoint detection:** wrong-model errors are classified from the structured `error.code`/`type`/`message` when a JSON error envelope is present (and a bare "does not exist" must mention a model), so route-missing 404s from gateways/proxies no longer suppress the fallback. When both flavours fail, the first payload-rejection error (the actionable 400/422) is preserved and surfaced instead of the second endpoint's bare 404; an unparseable 2xx re-probes. - **Thread safety:** `api_url` is snapshotted under `settings_lock` in both the request and models-fetch threads, and every `resolved_api_type` write goes through one locked helper with an epoch recheck — closing a use-after-free and a stale-cache race against a concurrent `/ai set provider`. The models-fetch provider reference is taken on the main thread (while the provider is guaranteed live) and released on every worker exit path. - **Reserved custom-setting keys** extended with `input` so `/ai set custom` cannot collide with the responses payload; `store` stays writable as a legitimate chat-completions parameter. - **UX:** `/ai set api-type` with provider-name and value autocompletion; `/ai` and `/ai providers` show the configured type, or `auto (detected: …)` once probed; help/synopsis/examples updated. ## Commits 1. **`14f76ab1d` feat(ai): support both responses and chat completions APIs** — the whole feature as a single unit (enum + provider fields, the attempt loop with fallback, the parameterized payload builder, the dual-format parser, the epoch/lock protocol, persistence, command wiring and tests). Touches `src/ai/ai_client.{c,h}`, `src/config/preferences.{c,h}`, `src/command/{cmd_funcs,cmd_defs,cmd_ac}.c`, `tests/unittests/`. 2. **`b3a269342` fix(ai): repair custom setting name autocompletion** — an independent pre-existing bug (below), kept separate because it does not touch the dual-API path. ## Follow-up commit: custom setting name autocompletion (`b3a269342`) The `/ai set custom <provider> <setting>` name completion never worked since its introduction in `9e5dfb14f`: the prefix `"/ai set custom "` carried a trailing space that `autocomplete_param_with_ac()` doubled, the chosen primitive completes the first token after the command while the setting name is the second, the `num_args == 3` guard excluded partially typed names, and the static list (tools/search/memory/plugins) predated the backend. Replaced with token-position completion (`autocomplete_param_no_with_func`, arg 5) over a list rebuilt on demand: common payload parameters plus the provider's currently set keys via the new `ai_get_provider_setting_keys()` (snapshot under `settings_lock`). The unused `parse_args()` call whose NULL result hit `g_strv_length()` unguarded (glib CRITICAL on TAB at `/ai ` or at the 5-arg value position) is gone with the dead branch. ## Testing - `gcc -fsyntax-only` clean on all touched translation units; full `make`/`make check` **not run yet** — please run before merge. - Behavioural coverage is unit-level (parser incl. reasoning-leak / brace-in-text / logprobs-before-content / literal-`"text"` regressions, settings, persistence). The fallback path still needs a live smoke test against OpenAI (both endpoints), Perplexity (responses-only), and the customer's chat-completions-only provider. ## Notes / open questions - The original spec (2026-07-04): default to `responses`; on 404, fall back to `chat/completions`; plus a manual `/ai set api-type MY_PROVIDER chat-completions|responses`. Implemented as specified, **with one deviation to confirm:** the fallback also triggers on a payload-shape rejection (400/422), not only on 404. Rationale: a flavour-specific custom setting or envelope shape would otherwise wedge every request with no recovery. If you'd rather keep the fallback strictly 404-only, this is a one-line narrowing of the endpoint-missing predicate. - The fallback probe costs one extra round-trip only when the first-attempt flavour is wrong; once a flavour answers it is reused as the hint for the rest of the session (not persisted across restarts).
jabber.developer2 added 2 commits 2026-07-08 17:05:17 +00:00
feat(ai): support both responses and chat completions APIs 14f76ab1db 
Drive both OpenAI-compatible flavours from one code path. Requests
default to /v1/responses and fall back once to /v1/chat/completions when
the provider reports the endpoint missing (404/405/501) or rejects the
payload shape (400/422); the working flavour is cached for the rest of
the run as a first-attempt hint, with the other flavour kept as a
fallback so a backend change self-corrects in-request. A per-provider
override is available via
  /ai set api-type <provider> responses|chat-completions|auto
and persisted as api_type= in the provider section.

One payload builder serves both flavours (messages/stream vs
input/stream/store); history and custom settings are serialized once per
request and the per-flavour envelope is assembled per attempt under a
single lock acquisition, so the fallback retry cannot double-count the
prompt. ai_parse_response_typed() dispatches on the request flavour and
splits extraction per envelope: chat completions anchored on
choices[].message.content, responses on the output_text part with a
string-aware backward scan bounded to that part's own object so a
reasoning summary or a truncated body is never returned as the reply.

Endpoint detection classifies wrong-model errors from the structured
error code/type/message when present, so route-missing 404s from
gateways no longer suppress the fallback; it preserves and surfaces the
first payload-rejection error when the fallback also fails, and re-probes
on an unparseable 2xx. resolved_api_type is written only through a locked
helper guarded by a URL/api-type epoch, and api_url is snapshotted under
settings_lock in the request and models-fetch threads, closing
use-after-free and stale-cache races against /ai set provider. The
models-fetch provider ref is taken on the main thread and released on
every worker exit path. Reserved custom-setting keys are extended with
input (store stays writable as a legitimate chat-completions parameter).
fix(ai): repair custom setting name autocompletion
All checks were successful
CI Code / Check spelling (pull_request) Successful in 13s
Details
CI Code / Check coding style (pull_request) Successful in 23s
Details
CI Code / Code Coverage (pull_request) Successful in 3m45s
Details
CI Code / Linux (ubuntu) (pull_request) Successful in 5m24s
Details
CI Code / Linux (arch) (pull_request) Successful in 7m7s
Details
CI Code / Linux (debian) (pull_request) Successful in 8m41s
Details
b3a269342d 
The setting-name branch for /ai set custom never fired: it passed the
prefix "/ai set custom " with a trailing space to
autocomplete_param_with_ac(), which appends a space itself, so the
matched prefix contained a double space no real input ever has. Even
without that, the primitive completes the first token after the command
while the setting name is the second, and the num_args guard only held
before the name was started. The suggestion list (tools/search/memory/
plugins) predated the /ai set custom backend and matched no real
payload parameter.

Replace the branch with token-position completion (arg 5 via
autocomplete_param_no_with_func) fed by a list rebuilt on demand from
common payload parameters plus the keys already set on the provider,
exposed through the new ai_get_provider_setting_keys() (snapshot taken
under settings_lock). Drop the now-unused parse_args() call whose NULL
result was fed to g_strv_length() unguarded, spamming a glib CRITICAL
on TAB at /ai with zero or five-plus arguments.
jabber.developer2 added 2 commits 2026-07-10 09:08:11 +00:00
fix(ai): close parser and AUTO fallback gaps 3a96d0dba1 
Parser:
- Never return reasoning text as the reply: the no-output_text path
  anchors strictly on the message "content" array; a reasoning-only body
  (truncated by max_output_tokens, carries only "summary") now fails the
  parse instead of leaking the chain-of-thought. Regression test added.
- _find_json_field skips field-name occurrences that are string values
  (no ':' after), so {"type":"text","text":"hi"} parses instead of
  failing on the value of "type".
- _same_json_object tracks brace depth, so a nested object between the
  "text" key and the "output_text" tag (e.g. "annotations") no longer
  rejects the part's own text.
- Chat-completions extraction is bounded to the choices array (new
  _json_array_end helper), so "message"/"content" in a sibling object
  (e.g. a top-level "warning") is not returned as the reply.

AUTO fallback:
- Model errors are classified only from a structured JSON error
  envelope; a route-missing 404 page merely mentioning models (e.g.
  listing /v1/models) no longer suppresses the fallback. Trade-off: a
  model error in a non-JSON body costs one extra fallback attempt.
- Fall back on a curl-level failure: an endpoint that accepts the
  connection but never responds no longer burns the 60s timeout without
  trying the flavour that works.
- Fall back on an unparseable 2xx: a gateway answering unknown paths
  with a 200 error page previously reset the hint and re-probed the same
  dead flavour on every request. The body is parsed inside the attempt
  loop and no longer re-parsed after it.
- The second attempt's 401/403/429/5xx now wins over the stashed 400/422
  payload rejection, so an invalid key or rate limit is not hidden
  behind a payload-shape error; a second 404 still defers to the first.
- When the first flavour's endpoint is missing but the other answers
  (even unsuccessfully, e.g. 401), the resolved hint moves to the
  surviving flavour, so the dead endpoint is not probed first forever.

Settings:
- Reserved payload keys are rejected only when setting a value; removal
  is allowed again, so a setting persisted before its key became
  reserved (e.g. "input") is no longer stuck in the config.
fix(ai): bound response parsing and harden AUTO fallback errors
Some checks failed
CI Code / Check spelling (pull_request) Failing after 14s
Details
CI Code / Check coding style (pull_request) Successful in 30s
Details
CI Code / Linux (debian) (pull_request) Successful in 5m0s
Details
CI Code / Linux (ubuntu) (pull_request) Successful in 5m8s
Details
CI Code / Linux (arch) (pull_request) Successful in 6m36s
Details
CI Code / Code Coverage (pull_request) Successful in 3m36s
Details
cac2aa856a 
- bound _parse_responses scans to the output_text part's object and to
  the content array, so a later sibling item's "text" (e.g. a reasoning
  summary) can never be returned as the assistant reply
- do not retry the other flavour on a curl timeout: the request likely
  reached the server and may still be generating, so a re-POST of the
  conversation could trigger a second billed generation
- remember an unparseable 2xx from the first AUTO attempt and surface
  stashed first-attempt errors in both error paths, instead of showing
  only the final attempt's transport or HTTP error
- recognize Ollama's model-not-found wording in _names_model so a model
  typo is not misread as a missing endpoint
- drop the dead, racy provider-lookup fallback in the generic request
  thread: a missing provider ref is a caller bug and now fails loudly
- fix ai_providers_lists_defaults to expect the header the command
  actually prints ("Configured providers:"); the test was broken since
  its introduction but CI never ran it
- add functional test group 5 (AI command surface) to FUNC_TEST_GROUPS
  so the CI parallel target runs it; proftest.c port ranges already
  account for five groups
jabber.developer2 added 1 commit 2026-07-10 09:52:47 +00:00
style(ai): rename unparseable to unparsable for codespell
Some checks failed
CI Code / Check spelling (pull_request) Successful in 14s
Details
CI Code / Check coding style (pull_request) Failing after 23s
Details
CI Code / Code Coverage (pull_request) Successful in 3m46s
Details
CI Code / Linux (debian) (pull_request) Successful in 5m50s
Details
CI Code / Linux (ubuntu) (pull_request) Successful in 6m0s
Details
CI Code / Linux (arch) (pull_request) Successful in 7m46s
Details
e9614abb89
jabber.developer2 force-pushed feat/ai-api-type from e9614abb89 to 3f430e01d2 2026-07-10 09:56:15 +00:00 Compare
jabber.developer2 force-pushed feat/ai-api-type from 3f430e01d2 to f7a818dc1d 2026-07-10 10:32:39 +00:00 Compare
jabber.developer2 force-pushed feat/ai-api-type from f7a818dc1d to 37218d2e07 2026-07-16 14:58:54 +00:00 Compare
All checks were successful
CI Code / Check spelling (pull_request) Successful in 16s
Required
Details
CI Code / Check coding style (pull_request) Successful in 26s
Required
Details
CI Code / Code Coverage (pull_request) Successful in 9m12s
Required
Details
CI Code / Linux (ubuntu) (pull_request) Successful in 9m18s
Required
Details
CI Code / Linux (debian) (pull_request) Successful in 9m29s
Required
Details
CI Code / Linux (arch) (pull_request) Successful in 13m13s
Required
Details
This pull request is marked as a work in progress.
This branch is out-of-date with the base branch
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin feat/ai-api-type:feat/ai-api-type
git checkout feat/ai-api-type
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
Clear labels
Compat/Breaking
Breaking change that won't be backward compatible
Kind/Bug
Something is not working
Kind/Documentation
Documentation changes
Kind/Enhancement
Improve existing functionality
Kind/Feature
New functionality
Kind/Security
This is security issue
Kind/Testing
Issue or pull request related to testing
Priority
Critical
1
The priority is critical
Priority
High
2
The priority is high
Priority
Low
4
The priority is low
Priority
Medium
3
The priority is medium
Reviewed
Confirmed
1
Issue has been confirmed
Reviewed
Duplicate
2
This issue or pull request already exists
Reviewed
Invalid
3
Invalid issue
Reviewed
Won't Fix
3
This issue won't be fixed
Status
Abandoned
3
Somebody has started to work on this but abandoned work
Status
Blocked
1
Something is blocking this issue or pull request
Status
Need More Info
2
Feedback is required to reproduce issue or to continue work
No Label
Milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
admin jabber.developer jabber.developer2
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: devs/cproof#163
No description provided.
Delete Branch "feat/ai-api-type"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?