Carrying the torch: 48 community PRs merged + fork fixes (v0.101.0-v0.109.0)#1028
Open
wishborn wants to merge 184 commits into
Open
Carrying the torch: 48 community PRs merged + fork fixes (v0.101.0-v0.109.0)#1028wishborn wants to merge 184 commits into
wishborn wants to merge 184 commits into
Conversation
Add comprehensive Replicate provider implementation supporting all core features: text generation, streaming (SSE), structured output, embeddings, image generation, and audio (TTS/STT). Features: - Text generation with system prompts and conversation history - Real-time SSE streaming with automatic fallback to simulated streaming - Structured output with JSON schema validation - Image generation (FLUX, Stable Diffusion XL, etc.) - Text-to-Speech with multiple voices (Kokoro-82m) - Speech-to-Text with Whisper (WAV, MP3, FLAC, OGG, M4A) - Embeddings (single and batch, 768-dimensional vectors) Implementation: - Async prediction management with configurable polling - Sync mode (Prefer: wait header) for lower latency - Comprehensive error handling with typed exceptions - Full PHPStan level 8 compliance - 21 tests with 60 assertions (100% feature coverage) - 455 lines of comprehensive documentation Files changed: 58 files, 4,444+ lines added
…chronously This adds the ability to be able to send a request to a provider to create a transcript where the provider will give you an id and then send a webhook to you in the future when the job is done with that id. This is just supplying the interface that a provider can utilize in the future.
Add comprehensive support for Alibaba Cloud's Qwen models via the DashScope native API (/api/v1), covering text generation, streaming, structured output, embeddings, image generation, and image editing. Key features: - Text generation with multi-step tool calling - Multi-modal (VL) support with automatic endpoint routing - Streaming with DashScope SSE protocol and reasoning/thinking tokens - Structured output with both JSON Object and JSON Schema modes - Embeddings with configurable dimensions - Image generation (qwen-image-max/plus) and editing (qwen-image-edit) - Region-aware configuration (International, China, US deployments) - 52 tests with real API fixtures (176 assertions) Co-authored-by: Cursor <cursoragent@cursor.com>
StreamEndEvent.usage can be null when providers don't include usage data in their final stream chunk, causing a TypeError downstream. Add `?? new Usage(0, 0)` fallback to emitStreamEndEvent() in all providers missing it, matching the existing pattern in the OpenAI stream handler.
Add an `api_format` config option to the OpenAI driver that allows switching from the default `/responses` endpoint to `/chat/completions`. This enables using Prism with OpenAI-compatible backends like vLLM, LiteLLM, and LocalAI that only implement the chat/completions API. Set `OPENAI_API_FORMAT=chat_completions` in your env to use it. Only text, structured, and stream methods dispatch conditionally — other modalities (embeddings, images, moderation, TTS, STT) already use standard endpoints that work with compatible backends as-is.
…nfigured Providers that reject unknown parameters (e.g. Perplexity via LiteLLM) return HTTP 400 when `"tools": []` is sent. Return null instead so Arr::whereNotNull() filters it out entirely.
Providers with integrated search capabilities (e.g. Perplexity, You.com) return top-level `citations` and `search_results` fields in chat/completions responses. These were previously ignored. Add ChatCompletionsCitationsMapper to map these into Prism's existing Citation infrastructure, and extract them once per stream in the ChatCompletions stream handler. Citations are passed through on the StreamEndEvent, matching the existing pattern used by the Anthropic handler.
…ob management and result handling
…h job management, result handling, and error mapping
…-anthropic-and-openai
…lue for data retrieval
… provider methods for batch management
… tool loop ## Context When Anthropic's server-side tools (like `web_search`) are used alongside regular user-defined tools, the model can do both in a single response: perform a web search, write text with citations referencing the search results, and call a regular tool. Because a regular tool was called, Prism enters its multi-step tool loop. It executes the tool, then replays the entire conversation back to the API for the next turn. The problem is that when Prism builds the replayed assistant message, it includes the text with citations but drops the `server_tool_use` and `web_search_tool_result` content blocks that the citations reference. The API validates that every citation points to an existing search result, finds none, and rejects the request with: `invalid_request_error - Could not find search result for citation index.` This only triggers when the model performs a server-side tool call AND a regular tool call in the same response. If either happens alone, everything works fine. ## Changes Both the Text and Stream handlers had the same gap in their tool loop replay logic. **Text handler (`Text.php`):** Added `extractProviderToolContent()` that pulls `server_tool_use` and `*_tool_result` content blocks from the API response and stores them in `additionalContent` as `provider_tool_calls` and `provider_tool_results`, the same keys that `MessageMap::mapAssistantMessage()` already reads and serializes back to the API. This follows the existing pattern of `extractText()`, `extractCitations()`, and `extractThinking()`. **Stream handler (`Stream.php`):** The stream state already tracked provider tool calls, provider tool results, and citations during streaming, but `handleToolCalls()` only included `thinking` and `thinking_signature` in the replayed `AssistantMessage`'s `additionalContent`. Now it also includes `citations`, `provider_tool_calls`, and `provider_tool_results`.
…delete, and metadata retrieval functionalities
…e batch job handling with inputFileId support
…xtRequest
- Use ?? [] on items to avoid passing null to buildAndUploadFile()
- Cast json_encode() result to string to satisfy non-empty-string return type
- Change clientRetry default from [] to [0] to satisfy array{0: int} type constraint
Made-with: Cursor
…s from OpenAI responses
…update tests for empty array responses
Provider-supplied output URLs are now fetched only over https from allowlisted hosts (replicate.delivery by default, configurable via prism.providers.replicate.download_hosts), with an explicit 30s timeout and a 25MB size cap. Rejected or failed downloads fall back to the URL as before. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Extends the v0.106.0 HITL mechanics (fork issue #4, phase 2) beyond Anthropic/OpenAI Responses to every Text and Structured handler with a tool loop: Azure, DeepSeek, Gemini, Groq, Mistral, Ollama, OpenRouter, Qwen, Requesty, Vertex, xAI, Z, and OpenAI ChatCompletions. Each handler resolves pending approvals before sending, stops its loop when marked tool calls are pending, and carries the approval requests on the assistant message for resume correlation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Streams now resolve pending approvals at start (yielding the resolved tool result events) and, when client-executed or approval-required tool calls are encountered, yield ToolApprovalRequestEvents and end the stream with FinishReason::ToolCalls instead of looping. Ports the upstream prism-php#932 stream fixtures and tests for both paths. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every Stream handler now resolves pending approvals at stream start and, when client-executed or approval-required tool calls are encountered, yields ToolApprovalRequestEvents and ends the stream with FinishReason::ToolCalls instead of looping — DeepSeek, Gemini, Groq, Mistral, Ollama, OpenRouter, Qwen, Requesty, Vertex, xAI, Z, and OpenAI ChatCompletions join Anthropic + OpenAI Responses. Completes the streaming item on fork issue #4. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rism-php#965) Covers both paths of the tools + response_format split: a tool call followed by the tool-less json_schema re-send, and the discard + re-send path when the model stops without calling the offered tools. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ls (prism-php#1004) Replaces the unrunnable test dropped in 5c08d92: this one ships a real two-response SSE fixture and consumes the stream before asserting the request payload omits the parameters key for a parameterless tool. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… and OpenRouter parameterless streaming Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…d Gemini These handlers reported cacheReadInputTokens while leaving the cached tokens inside promptTokens, so promptTokens + cacheReadInputTokens overstated input (double counting; upstream prism-php#1017). They now follow the convention already used by Anthropic, OpenAI (Responses) and DeepSeek: promptTokens holds the non-cached portion, guarded at zero. Gemini previously subtracted only when an explicit cachedContentName was set, missing implicit caching. Regression tests added for all three. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ders Extends the v0.108.1 usage convention (promptTokens = non-cached input) to every provider whose API reports prompt_tokens_details.cached_tokens: - OpenAI chat_completions path (Text/Structured/Stream) now has parity with the Responses path: cacheReadInputTokens populated, cached tokens excluded from promptTokens. Also applies to vLLM/LiteLLM-style backends used via api_format. - Azure, Groq, Qwen (native DashScope fields), xAI: cached-token visibility wired up (previously ignored entirely). - Z.AI and Requesty streams: fixed genuine double counting - they reported cacheReadInputTokens while leaving those tokens inside promptTokens. Their Text/Structured paths gain cache visibility. Regression tests for all seven providers (Text path), stream-level SSE fixture tests for Z.AI and Requesty, and the first Azure provider tests. Z.AI fixture-based expectations updated: the real captures contain large cached_tokens values, confirming the double counting in production responses. Removes a leftover unused providerOptions() read in the Gemini Text handler flagged by Rector. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…checker composer-require-checker surfaced undeclared direct dependencies, now declared: ext-mbstring, ext-openssl, psr/http-message, and symfony/http-foundation (all already installed transitively via laravel/framework, so this changes nothing for consumers). laravel/mcp moves to suggest for the optional LaravelMcpTool integration; its symbols plus the PHPUnit assertion used by the testing fakes are whitelisted in composer-require-checker.json. The custom ReorderMethodsRector was dev tooling shipping inside src/ (the source of phantom PhpParser/Rector symbols) - moved to dev/ under a Prism\Dev namespace, autoloaded only in autoload-dev and export-ignored from the dist package. New Require Checker CI workflow runs the tool on every push/PR. This is the proper guard for the goal upstream prism-php#1025 aimed at (catching use of undeclared symbols) without splitting laravel/framework into illuminate/* packages. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…penAI verbosity Three bug fixes from the upstream issue tracker plus one found in review: - DeepSeek and Qwen streaming handlers sent `stream: true` in the request body but never set the Guzzle `stream` transport option, so the full response body was buffered before the first event yielded — matching the other 14 providers now (upstream prism-php#990; Qwen found in review). - OpenRouter ToolCallMap eagerly ran json_decode() and passed the result to the ToolCall constructor, so malformed model JSON produced a raw TypeError (json_decode returns null) instead of a handled error. The map now stores the raw argument string; ToolCall::arguments() decodes lazily and wraps a decode failure in PrismException::malformedToolCallArguments(), which the tool-execution loop already converts into a tool result the model sees. The loop's error path is hardened to not re-throw while building that result (upstream prism-php#1006). Also adds OpenAI text_verbosity support to the structured (Responses) path and both chat/completions paths — it was previously only wired on the Responses text path (upstream prism-php#1015). Tests: OpenRouter ToolCallMap mapping + lazy-decode behavior, verbosity pass-through for structured + chat/completions, updated the ToolCall value object test to assert the handled PrismException. 1,891 passing. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…stency Per upstream prism-php#1017: promptTokens is normalized to exclude cached tokens everywhere, but completionTokens is inclusive of reasoning tokens on OpenAI/Anthropic/OpenAI-compatible providers and exclusive on Gemini/Vertex. Documents the difference rather than changing billing-relevant numbers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…s-vertex feat(vertex): added support for multi region endpoints
Introduce the Tier-1 telemetry substrate for observability (upstream prism-php#935): a provider-agnostic set of plain Laravel events emitted across the generation lifecycle, decoupled from the existing ShouldBroadcast streaming events so telemetry never forces websocket delivery. - TelemetryContext: correlation VO carrying a stable traceId plus explicit stepIndex/toolIndex ordinals, so a consumer can rebuild the span tree deterministically without ambient context surviving the tool loop. - ContextStack: container-bound ambient stack; tolerates out-of-order removal so an abandoned streaming generator cannot corrupt later calls. - Telemetry: static emission helper; a complete no-op when disabled. - Events: GenerationStarted/Completed/Failed, StepCompleted, ToolInvoked. - config: prism.telemetry.{enabled,capture_content}, both off by default (capture_content gates prompt/completion/tool-arg payloads — PII). - Bind ContextStack as a singleton. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ion paths Wire the neutral telemetry events into the provider-agnostic chokepoints so instrumentation lives in one place per path, touching no provider handler. - asText / asStructured / asEmbeddings / generate (images): start on entry, completed on success (with per-step StepCompleted for text/structured), failed on RequestException, context popped in finally. - asStream: wrap the provider generator via Telemetry::instrumentStream, which emits StepCompleted per StepFinishEvent and GenerationCompleted on StreamEndEvent while passing every event through untouched. - CallsTools: executeToolCall now measures and returns its own duration; ToolInvoked is dispatched from the in-process parent (executeToolsWithConcurrency and the approval-resume path) so listeners never fire in a forked child. - Tests: disabled no-op, started/completed, content-capture gating, per-step events, failure, streaming pass-through, and tool-invocation ordinals. Full suite green (1909 passed), PHPStan level 8 clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Security review found the capture_content privacy control was applied to GenerationStarted, StepCompleted, and GenerationCompleted but NOT to ToolInvoked, so tool arguments and tool results (which can carry user PII) were placed into the event payload even when capture_content was disabled — contradicting the documented opt-out and leaking content to whatever sink the operator wired. Fix: ToolInvoked now carries always-present scalar `toolName`/`toolCallId` (safe span metadata) while the content-bearing `toolCall`/`toolResult` are nullable and populated only when capture_content is enabled — matching the nullable-content pattern of the other three events. Added a regression test asserting content is withheld by default and present only when enabled. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ceof) The security-fix commit ran Pint but not Rector; the Formatting CI job runs `rector` + `git diff --exit-code`, which flagged the `=== null` assertions. FlipTypeControlToUseExclusiveTypeRector rewrites them to `! $e->toolCall instanceof ToolCall`, matching the repo's enforced style. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Anthropic streaming and Groq/Mistral/OpenAI audio handlers call json_validate(), a PHP 8.3 built-in, while composer.json declares "php": "^8.2" and the test matrix runs 8.2. Without a polyfill this fatals on real 8.2 in those paths, and composer-require-checker (run on 8.2) flagged json_validate as an unknown symbol — a red check on every push since the checker was added. Add symfony/polyfill-php83 to require: it defines json_validate() on 8.2, fixing the latent runtime bug and satisfying the require-checker while keeping 8.2 support (vs. a whitelist entry, which would only silence the checker). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
fix: polyfill json_validate() for PHP 8.2 support
docs: point README to the ai.particle.academy docs site
The recursive tool loop runs step N's tools before step N is recorded, so ToolInvoked carried no step ordinal and a consumer could not nest a tool span under its step. Track a per-generation step cursor on the ContextStack, advance it once per executed tool batch, and stamp it onto every ToolInvoked. No-op when telemetry is disabled.
…s + bounded content capture Adopts upstream issue prism-php#935: neutral Laravel telemetry events across the generation lifecycle (context/stack, step/tool ordinals, user/session ids), bounded opt-in content capture, and a step cursor so tool events are tagged with their owning step. Off by default and a complete no-op when disabled. Validated end-to-end (real multi-step tool generation -> OpenTelemetry bridge -> Phoenix) and security-reviewed (PASS WITH WARNINGS).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upstream has been quiet since March 2026 (v0.100.1), so particle-academy/prism — a drop-in fork,
Prism\Prismnamespace unchanged — has been absorbing the open backlog and shipping releases (context: discussion #1027). This PR offers all of that work back upstream in one piece: 166 commits, nine releases (v0.101.0–v0.109.0), gated throughout by Pest + PHPStan + Pint/Rector.If maintainership resumes, merge wholesale or tell us how you'd like it split — we're happy to break it into reviewable chunks. Either way the fork remains active.
Community PRs from this repo merged into the fork (48)
v0.101.0 — correctness fixes (17): #952, #958, #964, #971, #977, #985, #986, #987, #989, #991, #996, #1001, #1004, #1009, #1012, #1013, #1024
v0.103.0 — provider correctness / API drift (16): #949, #954, #961, #965, #975, #976, #980, #992, #993, #995, #997, #1000, #1002, #1008, #1020, #1021
v0.104.0 — features (9): #951 (batches + files APIs), #960 (xAI images), #978 (Vertex AI provider, answers #795), #988 (fine-grained tool streaming), #998 (Anthropic adaptive thinking), #1003 (pause_turn/refusal), #1014 (Mistral FIM), #1018 (provider-agnostic withReasoning()), #1026 (Requesty provider)
v0.105.0 — features + providers (6): #757 (Replicate provider), #810 (async STT interface), #835 (Azure OpenAI provider), #898 (Qwen provider), #907 (OpenAI chat/completions api_format + streaming citations, answers #900), #920 (cost tracking in Usage)
Reimplemented rather than rebased: #932 (client-executed tools + human-in-the-loop approval, answers #921) — clean-room implementation across all providers including streaming; docs at https://ai.particle.academy/docs/core-concepts/human-in-the-loop
Adjudicated, not merged (rationale posted): #950 (duplicate of #977), #937 and #1005 (superseded by an escape-based control-character fix), #999 (superseded), #1025 (rejected — a composer-require-checker CI gate solves the underlying goal properly; analysis in Particle-Academy/prism#3)
Fork-original changes
laravel/framework ^12.61.1|^13.12.0).Tool::requiresApproval(bool|Closure)/Tool::clientExecuted(), deny-by-default resume from message history, streaming approval events — text/structured/stream on all 18 providers.promptTokens= non-cached input everywhere;cacheReadInputTokenspopulated wherever the provider exposes it. Fixed silent double counting in Gemini, Vertex, OpenRouter (v0.108.1) and Z.AI + Requesty streams (v0.109.0); added cache visibility for OpenAI chat/completions, Azure, Groq, Qwen, xAI.anthropic_betaprovider option.src/.Full release notes: https://github.com/Particle-Academy/prism/releases
🤖 Generated with Claude Code