Skip to content

Commit 48c1b45

Browse files
authored
feat(integrations): extend ElevenLabs, Google Drive, Firecrawl, Pinecone, Resend, and S3 tool depth (#5270)
* feat(firecrawl): add crawl status/cancel, batch scrape + status, extract status, credit usage tools * feat(resend): add audiences, broadcasts, and cancel-email tools * feat(pinecone): add delete/update vectors, index, and stats tools * feat(google-drive): add revisions, comments, and export tools * feat(elevenlabs): add voices, settings, models, user, sound-effects, speech-to-speech, audio-isolation tools * feat(s3): add bucket CRUD, head-object, presigned-url, and batch-delete tools * chore(api-validation): bump route baseline to 873 for wave-3 internal tool routes (s3, elevenlabs, google_drive export) * docs(integrations): regenerate docs + catalog for wave-3 tools * fix(integrations): audit fixes for wave-3 - pinecone: read camelCase vectorType/deletionProtection (with snake_case fallback) so list_indexes/describe_index populate them; make describe_index_stats casing defensive - google-drive: URL-encode fileId in the export route - remove extraneous inline/section-divider comments across new blocks/tools; convert type docs to TSDoc * fix(integrations): address review — elevenlabs settings bleed, batch-scrape job-id guard, s3 head-object existence - elevenlabs: select stability/similarityBoost by operation so a stale edit-settings value can't bleed into a TTS call - firecrawl: fail fast with a clear error when batch scrape returns no job id (avoids a misleading polling timeout) - s3: head_object on a missing key now returns exists:false instead of a generic failure * fix(s3): allow exists:false in head-object response contract (schema must match the missing-object output) * fix(pinecone): guard JSON.parse of ids/filter/values/sparseValues/setMetadata Malformed JSON-string input now throws a clear '<field> must be valid JSON' error via a shared parseJsonParam helper instead of crashing the request body builder. * fix(pinecone): enforce mutual exclusivity of ids/deleteAll/filter in delete_vectors Pinecone treats these delete selectors as mutually exclusive; the tool now requires exactly one and throws a clear error otherwise, instead of sending a conflicting body. * fix(integrations): I/O fidelity vs API docs (wave-3 audit) - pinecone: normalize describe_index_stats per-namespace vector_count -> vectorCount - firecrawl: remove phantom 'sources' from extract_status, add real creditsUsed/tokensUsed; expose batch_scrape maxConcurrency/ignoreInvalidURLs - google-drive: drop undocumented supportsAllDrives from the files.export URL - elevenlabs: add next_page_token input to list_voices (fixes pagination dead-end) - resend: surface segment_id on get_broadcast; declare replyTo + segment_id in block outputs
1 parent c7bb37d commit 48c1b45

86 files changed

Lines changed: 8654 additions & 71 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

apps/docs/content/docs/en/integrations/elevenlabs.mdx

Lines changed: 212 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
---
22
title: ElevenLabs
3-
description: Convert text to speech with ElevenLabs
3+
description: Generate and transform audio with ElevenLabs
44
---
55

66
import { BlockInfoCard } from "@/components/ui/block-info-card"
@@ -27,7 +27,7 @@ In Sim, the ElevenLabs integration enables your agents to convert text to lifeli
2727

2828
## Usage Instructions
2929

30-
Integrate ElevenLabs into the workflow. Can convert text to speech.
30+
Integrate ElevenLabs into the workflow. Convert text to speech, generate sound effects, transform voices, isolate audio, and manage voices, models, and account settings.
3131

3232

3333

@@ -55,4 +55,214 @@ Convert text to speech using ElevenLabs voices
5555
| `audioUrl` | string | The URL of the generated audio |
5656
| `audioFile` | file | The generated audio file |
5757

58+
### `elevenlabs_sound_effects`
59+
60+
Generate a sound effect from a text prompt using ElevenLabs
61+
62+
#### Input
63+
64+
| Parameter | Type | Required | Description |
65+
| --------- | ---- | -------- | ----------- |
66+
| `apiKey` | string | Yes | Your ElevenLabs API key |
67+
| `text` | string | Yes | The prompt describing the sound effect \(e.g., "thunder rumbling in the distance"\) |
68+
| `modelId` | string | No | The model to use \(defaults to eleven_text_to_sound_v2\) |
69+
| `durationSeconds` | number | No | Length of the sound in seconds \(0.5-30\). Omit to auto-determine |
70+
| `promptInfluence` | number | No | How closely to follow the prompt from 0.0 to 1.0 \(default 0.3\) |
71+
| `loop` | boolean | No | Whether to generate a seamlessly looping sound effect \(default false\) |
72+
73+
#### Output
74+
75+
| Parameter | Type | Description |
76+
| --------- | ---- | ----------- |
77+
| `audioUrl` | string | URL of the generated sound effect |
78+
| `audioFile` | file | The generated sound effect file |
79+
80+
### `elevenlabs_speech_to_speech`
81+
82+
Convert audio into a chosen ElevenLabs voice while preserving content and emotion
83+
84+
#### Input
85+
86+
| Parameter | Type | Required | Description |
87+
| --------- | ---- | -------- | ----------- |
88+
| `apiKey` | string | Yes | Your ElevenLabs API key |
89+
| `voiceId` | string | Yes | The ID of the target voice to convert the audio into |
90+
| `audioFile` | file | Yes | The source audio file to convert \(e.g., MP3, WAV, M4A\) |
91+
| `modelId` | string | No | The model to use \(defaults to eleven_english_sts_v2\) |
92+
| `removeBackgroundNoise` | boolean | No | Whether to isolate the voice and remove background noise \(default false\) |
93+
94+
#### Output
95+
96+
| Parameter | Type | Description |
97+
| --------- | ---- | ----------- |
98+
| `audioUrl` | string | URL of the converted audio |
99+
| `audioFile` | file | The converted audio file |
100+
101+
### `elevenlabs_audio_isolation`
102+
103+
Remove background noise from an audio file, isolating the speech using ElevenLabs
104+
105+
#### Input
106+
107+
| Parameter | Type | Required | Description |
108+
| --------- | ---- | -------- | ----------- |
109+
| `apiKey` | string | Yes | Your ElevenLabs API key |
110+
| `audioFile` | file | Yes | The audio file to isolate speech from \(e.g., MP3, WAV, M4A\) |
111+
112+
#### Output
113+
114+
| Parameter | Type | Description |
115+
| --------- | ---- | ----------- |
116+
| `audioUrl` | string | URL of the isolated audio |
117+
| `audioFile` | file | The isolated audio file |
118+
119+
### `elevenlabs_list_voices`
120+
121+
List the voices available in your ElevenLabs account
122+
123+
#### Input
124+
125+
| Parameter | Type | Required | Description |
126+
| --------- | ---- | -------- | ----------- |
127+
| `apiKey` | string | Yes | Your ElevenLabs API key |
128+
| `search` | string | No | Search term to filter voices by name, description, labels, or category |
129+
| `category` | string | No | Filter by category: premade, cloned, generated, or professional |
130+
| `pageSize` | number | No | Number of voices to return \(1-100, default 10\) |
131+
| `nextPageToken` | string | No | Page token from a previous response to fetch the next page of voices |
132+
133+
#### Output
134+
135+
| Parameter | Type | Description |
136+
| --------- | ---- | ----------- |
137+
| `voices` | array | List of voices |
138+
|`voiceId` | string | Unique voice identifier |
139+
|`name` | string | Voice name |
140+
|`category` | string | Voice category |
141+
|`description` | string | Voice description |
142+
|`labels` | json | Voice labels \(accent, gender, age, use case\) |
143+
|`previewUrl` | string | URL to a preview audio sample |
144+
|`settings` | json | Default voice settings |
145+
| `totalCount` | number | Total number of matching voices |
146+
| `hasMore` | boolean | Whether more voices are available |
147+
| `nextPageToken` | string | Token to fetch the next page |
148+
149+
### `elevenlabs_get_voice`
150+
151+
Get metadata and settings for a specific ElevenLabs voice
152+
153+
#### Input
154+
155+
| Parameter | Type | Required | Description |
156+
| --------- | ---- | -------- | ----------- |
157+
| `apiKey` | string | Yes | Your ElevenLabs API key |
158+
| `voiceId` | string | Yes | The ID of the voice to retrieve \(e.g., "21m00Tcm4TlvDq8ikWAM"\) |
159+
160+
#### Output
161+
162+
| Parameter | Type | Description |
163+
| --------- | ---- | ----------- |
164+
| `voiceId` | string | Unique voice identifier |
165+
| `name` | string | Voice name |
166+
| `category` | string | Voice category |
167+
| `description` | string | Voice description |
168+
| `labels` | json | Voice labels \(accent, gender, age, use case\) |
169+
| `previewUrl` | string | URL to a preview audio sample |
170+
| `settings` | json | Default voice settings |
171+
| `availableForTiers` | array | Subscription tiers the voice is available on |
172+
| `highQualityBaseModelIds` | array | Model IDs that support high-quality output for this voice |
173+
| `isOwner` | boolean | Whether the current user owns this voice |
174+
175+
### `elevenlabs_get_voice_settings`
176+
177+
Get the configured settings for a specific ElevenLabs voice
178+
179+
#### Input
180+
181+
| Parameter | Type | Required | Description |
182+
| --------- | ---- | -------- | ----------- |
183+
| `apiKey` | string | Yes | Your ElevenLabs API key |
184+
| `voiceId` | string | Yes | The ID of the voice whose settings to retrieve |
185+
186+
#### Output
187+
188+
| Parameter | Type | Description |
189+
| --------- | ---- | ----------- |
190+
| `stability` | number | Voice stability \(0.0-1.0\) |
191+
| `similarityBoost` | number | Similarity boost \(0.0-1.0\) |
192+
| `style` | number | Style exaggeration \(0.0-1.0\) |
193+
| `useSpeakerBoost` | boolean | Whether speaker boost is enabled |
194+
| `speed` | number | Speech speed \(1.0 = normal\) |
195+
196+
### `elevenlabs_edit_voice_settings`
197+
198+
Update the settings for a specific ElevenLabs voice
199+
200+
#### Input
201+
202+
| Parameter | Type | Required | Description |
203+
| --------- | ---- | -------- | ----------- |
204+
| `apiKey` | string | Yes | Your ElevenLabs API key |
205+
| `voiceId` | string | Yes | The ID of the voice to update |
206+
| `stability` | number | No | Voice stability from 0.0 to 1.0 \(default 0.5\) |
207+
| `similarityBoost` | number | No | Similarity boost from 0.0 to 1.0 \(default 0.75\) |
208+
| `style` | number | No | Style exaggeration from 0.0 to 1.0 \(default 0\) |
209+
| `useSpeakerBoost` | boolean | No | Whether to enhance similarity to the original speaker \(default true\) |
210+
| `speed` | number | No | Speech speed where 1.0 is normal \(default 1.0\) |
211+
212+
#### Output
213+
214+
| Parameter | Type | Description |
215+
| --------- | ---- | ----------- |
216+
| `status` | string | Request outcome \("ok" on success\) |
217+
218+
### `elevenlabs_list_models`
219+
220+
List the models available in ElevenLabs
221+
222+
#### Input
223+
224+
| Parameter | Type | Required | Description |
225+
| --------- | ---- | -------- | ----------- |
226+
| `apiKey` | string | Yes | Your ElevenLabs API key |
227+
228+
#### Output
229+
230+
| Parameter | Type | Description |
231+
| --------- | ---- | ----------- |
232+
| `models` | array | List of available models |
233+
|`modelId` | string | Unique model identifier |
234+
|`name` | string | Model name |
235+
|`description` | string | Model description |
236+
|`canDoTextToSpeech` | boolean | Supports text-to-speech |
237+
|`canDoVoiceConversion` | boolean | Supports voice conversion |
238+
|`canUseStyle` | boolean | Supports the style parameter |
239+
|`canUseSpeakerBoost` | boolean | Supports speaker boost |
240+
|`languages` | array | Languages supported by the model |
241+
|`languageId` | string | Language code |
242+
|`name` | string | Language name |
243+
244+
### `elevenlabs_get_user`
245+
246+
Get account and subscription information for the ElevenLabs user
247+
248+
#### Input
249+
250+
| Parameter | Type | Required | Description |
251+
| --------- | ---- | -------- | ----------- |
252+
| `apiKey` | string | Yes | Your ElevenLabs API key |
253+
254+
#### Output
255+
256+
| Parameter | Type | Description |
257+
| --------- | ---- | ----------- |
258+
| `userId` | string | Unique user identifier |
259+
| `isNewUser` | boolean | Whether the user is new |
260+
| `subscription` | object | Subscription and usage details |
261+
|`tier` | string | Subscription tier |
262+
|`characterCount` | number | Characters used this period |
263+
|`characterLimit` | number | Character quota for this period |
264+
|`canExtendCharacterLimit` | boolean | Whether the character limit can be extended |
265+
|`status` | string | Subscription status |
266+
|`nextCharacterCountResetUnix` | number | Unix timestamp when the character count resets |
267+
58268

0 commit comments

Comments
 (0)