feat: cli OpenAI-compatible API response_format support

[markstur]

Misc PR

Type of PR

• Bug Fix
• New Feature
• Documentation
• Other

Description

• Link to Issue: Fixes m serve OpenAI API structured output #824

feat: cli OpenAI-compatible API `response_format` support

   - Added `JsonSchemaFormat` model to represent JSON schema definitions
   - Extended `ResponseFormat` to support `json_schema` type (in addition to existing `text` and `json_object`)
   - Used field alias to avoid conflict with Pydantic's `schema` method

   - Added `_json_schema_to_pydantic()` utility function to dynamically convert JSON schemas to Pydantic models
   - Updated `_build_model_options()` to exclude `response_format` from model options (handled separately)
   - Modified `make_chat_endpoint()` to:
     - Parse `response_format` from requests
     - Convert `json_schema` type to Pydantic models using the utility function
     - Detect if the serve function accepts a `format` parameter using `inspect.signature()`
     - Pass the generated Pydantic model as `format=` parameter to serve functions that support it
     - Handle backward compatibility with serve functions that don't accept `format`
   - Added proper error handling for invalid schemas

   - Test json_schema format is converted to Pydantic model and passed to serve
   - Test json_object format doesn't pass a schema
   - Test text format doesn't pass a schema
   - Test error handling for missing json_schema field
   - Test error handling for invalid JSON schemas
   - Test backward compatibility with serve functions without format parameter
   - Test optional fields in JSON schemas

When a client sends a request with `response_format.type = "json_schema"`, the server:
1. Extracts the JSON schema from `response_format.json_schema.schema`
2. Dynamically creates a Pydantic model from the schema
3. Passes it as the `format=` parameter to the serve function
4. The serve function can then use this for constrained decoding via Mellea's `instruct()` method

This maps OpenAI's `response_format` API to Mellea's native `format=` parameter for structured output.

Testing

• Tests added to the respective file if code was changed
• New code has 100% coverage if code as added
• Ensure existing tests and github automation passes (a maintainer will kick off the github automation when the rest of the PR is populated)

Attribution

• AI coding assistants used

[github-actions]

The PR description has been updated. Please fill out the template for your PR to be reviewed.

[planetf1]

The non-streaming path (return ChatCompletion just below, line 297) returns output.value without validating against format_model. Suggest adding before that return (also needs import json at the top and ValidationError added to the pydantic import):

if format_model is not None and output.value is not None:
    try:
        format_model.model_validate(json.loads(output.value))
    except (json.JSONDecodeError, ValidationError) as e:
        return create_openai_error_response(
            status_code=400,
            message=f"Output does not match required schema: {e!s}",
            error_type="invalid_response_error",
        )

[planetf1]

Validation runs after all content chunks are already sent (lines 68–106), so the error arrives after the client has consumed the data. A few options:

1. Buffer when format_model is set, validate, then stream or error before emitting anything.
2. Return a 400 upfront when stream=True + json_schema — simplest for now.
3. Keep post-hoc but document it — callers can pass format= to the backend for constrained decoding instead.

[planetf1]

this handles type, but will not handle enum, additionalProperties, nested types, array, $ref, allOf, anyOf

Suggest clarifying caveats in comments? or figuring out if any more validation is viable

[planetf1]

cacheable/could be done up front? Here it's done in every request but won't change?

[planetf1]

not used? See related comment - more is needed to really be strict or at least clarify behaviour?

[planetf1]

similar (not identical) code is repeated multiple times - possible opportunity for making common - minor.