feat: discover OSS MLflow capabilities and model metadata - #334
Open
dgokeeffe wants to merge 8 commits into
Open
feat: discover OSS MLflow capabilities and model metadata#334dgokeeffe wants to merge 8 commits into
dgokeeffe wants to merge 8 commits into
Conversation
…directing HOME Pi honors the PI_CODING_AGENT_DIR env var to resolve its config directory (~/.pi/agent), so redirecting /Users/david.okeeffe to APP_DIR/pi-home was unnecessary. The HOME redirect broke macOS keychain default resolution under ucode: the Security framework looks for the login keychain under the redirected HOME, finds none, and security default-keychain returns 'A default keychain could not be found'. As a result gh auth, the git credential helper, and any keychain-backed tool failed inside pi. Setting PI_CODING_AGENT_DIR to the existing PI_CONFIG_DIR preserves config isolation (models.json/settings.json/sessions still land under APP_DIR/pi-home/.pi/agent) while leaving /Users/david.okeeffe as the user's real home, so the login keychain stays discoverable. Tests: the two pi e2e sites monkeypatched PI_UCODE_HOME/PI_CONFIG_PATH to redirect pi at a tmp home. They now also patch PI_CONFIG_DIR (read by build_runtime_env) and PI_SETTINGS_PATH/PI_SETTINGS_BACKUP_PATH (previously masked because the HOME redirect made pi read settings from the un-patched real APP_DIR path).
Expose the GLM and Kimi coding-model cohort through Pi and OpenCode with shared token limits and reasoning metadata. Keep unsupported chat models out of discovery, including Inkling until gateway issue databricks#215 is fixed, and retain the GPT-OSS Responses API routing guard.
Centralize Claude family/version parsing so Pi metadata, adaptive-thinking compatibility, and Claude Code's [1m] selector cannot drift. Cover Sonnet 4.5, Opus 4.6, future major versions, Fable fallback, and prefixed model IDs.
Configure the Databricks OpenAI Responses provider alongside the validated GLM/Kimi provider, and fall back to foundation-model serving endpoints when UC model services are unavailable.
…/consolidated-pi-opencode-upstream # Conflicts: # tests/test_e2e.py
`_pi_gpt_model_entry` declared `reasoning: True` without an off-state, so for
the thinking-off case Pi's Responses builder fell back to
`reasoning: {effort: "none"}` (pi-ai openai-responses.js, the
`thinkingLevelMap?.off !== null` branch). `"none"` is only valid on gpt-5.1+,
so every request to gpt-5, gpt-5-mini, gpt-5-nano and gpt-5-5-pro was rejected:
BAD_REQUEST: Unsupported value: 'none' is not supported with the 'gpt-5'
model. Supported values are: 'minimal', 'low', 'medium', and 'high'.
Setting `thinkingLevelMap: {"off": None}` makes Pi omit `reasoning` entirely,
which the gateway accepts for all 14 codex ids. Verified against
/ai-gateway/codex/v1/responses: effort="none" 400s on gpt-5/-mini/-nano/-5-5-pro
and 200s on gpt-5-1..-5-6; omitting `reasoning` is 200 everywhere.
`{"off": "minimal"}` was rejected as an alternative because gpt-5-5-pro 400s on
it too. Same pattern already used for the Gemini 3.x entries.
The rest of Pi's Responses payload was bisected against the gateway and is
fine: store:false, prompt_cache_key, prompt_cache_retention:"24h",
prompt_cache_options, include:["reasoning.encrypted_content"], developer role,
flat tool schemas, and the session_id / x-client-request-id affinity headers.
Regression was hard to spot because the gateway returns
{"error_code","message"} rather than OpenAI's {"error":...}, so Pi's
error-body.js recovery no-ops and every 400 renders as
"OpenAI API error (400): 400 status code (no body)". Reported upstream as
earendil-works/pi#7748.
Refs databricks#286
…i-opencode-upstream # Conflicts: # src/ucode/agents/pi.py # src/ucode/databricks.py
This was referenced Aug 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue and stack
Closes #332.
Depends on #217, #333, #223, #239, and #243. This fork-based PR must target
main; review only the capability-discovery commit relative to the pushed integrated consolidation base:dgokeeffe/ucode@review/integrated-consolidation-base...review/oss-capability-discovery
Equivalent immutable range:
168038b...347f81d. Do not merge before all prerequisites. The incremental range contains no proxy or cache-diagnostic changes.Unit 05 — OSS/MLflow capability discovery and model metadata
Objective and user-visible behavior
Dynamically discover MLflow chat-completions-only models, persist reasoning/context/output metadata, project capabilities onto UC IDs, and render bounded Pi/OpenCode model entries. Unknown or partial metadata degrades conservatively; static GLM/Kimi behavior remains intact.
Candidate:
review/oss-capability-discovery(347f81d), incremental base integrateddev(168038b). This is the capability follow-up for existing PR #217; it does not include the repair proxy.Exact scope
Production:
src/ucode/databricks.py,src/ucode/cli.py,src/ucode/agents/pi.py,src/ucode/agents/opencode.py.Tests/fixtures:
tests/conftest.py,tests/test_databricks.py,tests/test_cli.py,tests/test_agent_pi.py,tests/test_agent_opencode.py.Non-goals: SSE repair/lifecycle, cache affinity/retention, Pi home isolation, GPT off-thinking.
Before / after reproduction
Focused validation:
Full suite:
uv run --frozen pytest -q # 2 failed, 1763 passed, 36 skippedFailures: installed Claude capture and the pre-existing Pi preferred-model state assertion. Both reproduce on integrated
devwith exact focused commands; this unit does not touch Claude capture or state serialization/default selection.Live Databricks evidence (sanitized)
A live configured workspace call to
discover_oss_model_specscompleted without printing workspace, token, or model IDs:{"live_evidence":"completed","spec_count":14,"all_ids_nonempty":true,"all_reasoning_boolean":true,"context_metadata_count":8,"output_limit_count":13,"reason_redacted":null}Deterministic tests cover malformed payloads, mismatched entity capabilities, native-API exclusion, duplicate IDs, context-description parsing, UC ID projection, metadata outage fallback, conservative limit completion, and state-to-agent propagation.
Impact map
oss_model_specs; Pi/OpenCode consume it for reasoning and limits.Rollback and residual risk
Revert
347f81d; the original validated GLM/Kimi consolidation remains ondev. Residual risks: free-text context metadata can be absent; output ceilings are probed policy data and require maintenance as models evolve. The fallback direction caps conservatively rather than sending uncapped requests.Hygiene
git diff --check dev..347f81dpasses. Nine-file range; no proxy file, diagnostic scripts, generated files, credentials,uv.lock,.pi-subagents/, orgoal.md; no merge markers or unresolved index entries.