Enable internal text embedding API by ajtiwari07 · Pull Request #3441 · Azure/data-api-builder

ajtiwari07 · 2026-04-12T16:18:30Z

Summary
This PR adds configurable text chunking capabilities to the embeddings API, enabling automatic text segmentation before embedding generation. This feature supports both single-text and multi-document batch processing with runtime configuration and query parameter overrides.

Changes
Configuration
Added EmbeddingsChunkingOptions.cs - Configuration model for chunking behavior
Enabled (bool) - Enable/disable chunking
SizeChars (int) - Chunk size in characters (default: 800)
OverlapChars (int) - Overlap between chunks (default: 100)
EffectiveSizeChars property ensures minimum valid chunk size
Modified EmbeddingsOptions.cs - Added Chunking property and IsChunkingEnabled helper
Removed EmbeddingsCacheOptions.cs - Simplified configuration by removing unused cache feature

API Enhancements
Modified Controllers/EmbeddingController.cs
Auto-detects request type (single text vs. document array)
Implements overlapping text chunking algorithm
Supports query parameter overrides: $chunking.enabled, $chunking.size-chars, $chunking.overlap-chars
Returns multiple embeddings per document when chunking is enabled
Added Models/EmbedDocumentRequest.cs - Request model for document arrays
Added Models/EmbedDocumentResponse.cs - Response model with chunked embeddings
Schema (schemas)
Modified dab.draft.schema.json - Added chunking configuration schema with validation rules
Testing (UnitTests)
Added EmbeddingsChunkingOptionsTests.cs (13 tests) - Configuration validation
Added ChunkTextTests.cs (21 tests) - Chunking algorithm validation including edge cases
Modified EmbeddingControllerTests.cs (+18 tests) - API endpoint tests for chunking and document arrays
Total Test Coverage: 72 tests (48 existing + 24 new) - All passing## Why make this change?

Testing
All 72 unit tests passing
Edge cases covered: empty text, very small chunks, overlap larger than chunk size, Unicode text
Query parameter parsing validated
Backward compatibility verified

Breaking Changes
None - This is a backward-compatible addition. Existing single-text requests continue to work without modification.

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

…configuration Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

… and telemetry integration Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

…ate empty embeddings Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

…oint/health sub-objects Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

…orization Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

…lthCheckConfig in converter

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…dding controller tests, switching the dab schema for the embedding system to default to false

…icitly set

embeddings endpoint is now permanently fixed to /embed with no user-configurable path option. This removes unnecessary configuration surface since the feature has not been released yet, eliminating the need for backward compatibility. Changes: - Remove path property from dab.draft.schema.json - Remove Path, UserProvidedPath, and EffectivePath from EmbeddingsEndpointOptions - Remove EffectiveEndpointPath from EmbeddingsOptions - Remove path deserialization from EmbeddingsOptionsConverterFactory - Remove --runtime.embeddings.endpoint.path CLI option - Remove path configuration logic from ConfigGenerator - Remove endpoint path validation from RuntimeConfigValidator - Update Startup.cs logging to use DEFAULT_PATH constant - Update all tests to remove path references

…e keys

…onal

azure-pipelines · 2026-04-29T20:21:17Z

Commenter does not have sufficient privileges for PR 3441 in repo Azure/data-api-builder

…om/ajtiwari07/data-api-builder into add-internal-text-embedding-system

sayalikudale · 2026-04-29T22:09:34Z

+    }
+
+    /// <inheritdoc/>
+    public async Task<EmbeddingBatchResult> TryEmbedBatchAsync(string[] texts, CancellationToken cancellationToken = default)


nit: This has the
checks duplicated inline which are also present in the method ValidateEmbedBatchRequest. Can we use validation method here instead.

There is a difference in validations between these methods. I have extracted validation logic methods where ever applicable.

Aniruddh25 · 2026-04-29T23:47:18Z

/azp run

azure-pipelines · 2026-04-29T23:47:44Z

Azure Pipelines successfully started running 6 pipeline(s).

…om/ajtiwari07/data-api-builder into add-internal-text-embedding-system

souvikghosh04 · 2026-04-30T15:10:30Z

/azp run

azure-pipelines · 2026-04-30T15:11:05Z

Azure Pipelines successfully started running 6 pipeline(s).

…om/ajtiwari07/data-api-builder into add-internal-text-embedding-system

Aniruddh25 · 2026-05-01T17:12:00Z

/azp run

azure-pipelines · 2026-05-01T17:12:30Z

Azure Pipelines successfully started running 6 pipeline(s).

…om/ajtiwari07/data-api-builder into add-internal-text-embedding-system

ajtiwari07 · 2026-05-01T19:24:45Z

/azp run

azure-pipelines · 2026-05-01T19:24:52Z

Commenter does not have sufficient privileges for PR 3441 in repo Azure/data-api-builder

Aniruddh25 · 2026-05-02T00:34:41Z

/azp run

azure-pipelines · 2026-05-02T00:35:06Z

Azure Pipelines successfully started running 6 pipeline(s).

Aniruddh25

Main question around need for embedding cache options. Cant we reuse runtime cache options to decide what we do for embeddings, does it need to be granular?

Aniruddh25 · 2026-05-02T01:29:54Z

+                  "type": "string",
+                  "description": "Cache level (L1 for in-memory only, L1L2 for in-memory + distributed). Defaults to L1.",
+                  "enum": ["L1", "L1L2"],
+                  "default": "L1"


Shouldnt the default cache be L1L2 ? since we would like the embeddings to be stored in Redis Cache? and Redis is L2.

In this phase we only support L1 via Fusion cache. In phase 2 we will enable support for L2 with redis which would default to L1 or L1L2 like you have suggested.

Aniruddh25 · 2026-05-02T01:32:33Z

+                "size-chars": {
+                  "type": "integer",
+                  "description": "The size of each chunk in characters.",
+                  "default": 800,


PR description says default is 1000. What is the correct default?

Modified the PR description, you can refer to PRD as a SOT: #3331

Aniruddh25 · 2026-05-02T02:05:16Z

+            RuntimeEmbeddingsTimeoutMs = runtimeEmbeddingsTimeoutMs;
+            // Embeddings Endpoint
+            RuntimeEmbeddingsEndpointEnabled = runtimeEmbeddingsEndpointEnabled;
+            RuntimeEmbeddingsEndpointRoles = runtimeEmbeddingsEndpointRoles;


IS EmbeddingsEndpointPath not configurable through dab configure? Why not?

Is EmbeddingsCache not configurable? same question for chunking options.

Aniruddh25 · 2026-05-02T02:08:04Z

+                    Dimensions: dimensions,
+                    TimeoutMs: timeoutMs,
+                    Endpoint: endpointOptions,
+                    Health: healthOptions);


doesnt mention cache options, but the schema has cache options. I think we dont need cache options at the embeddings level. The global runtime cache options should be sufficient.

Aniruddh25 · 2026-05-02T02:10:49Z

+// Copyright (c) Microsoft Corporation.
+// Licensed under the MIT License.
+
+using System.Text.Json;


cache options are not deserialized either. Better to simply remove them from the dab.draft.schema

Aniruddh25 · 2026-05-02T02:25:42Z

+            // Register embedding service if configured and enabled.
+            // NOTE: IEmbeddingService is only registered when enabled to avoid constructor
+            // failures when config has empty/placeholder values for disabled embeddings.
+            // TODO: To support hot-reload for embeddings (toggling enabled on/off at runtime),


Is a separate issue created for to track this?

Aniruddh25 · 2026-05-02T04:06:54Z

+        }
+
+        // Check if embedding service is available
+        if (_embeddingService is null || !_embeddingService.IsEnabled)


Whether embeddingService is enabled or not depends on if embeddingOptions.Enabled is set or not, so why check it here again?

Aniruddh25 · 2026-05-02T04:11:11Z

+    private readonly HttpClient _httpClient;
+    private readonly EmbeddingsOptions _options;
+    private readonly ILogger<EmbeddingService> _logger;
+    private readonly IFusionCache _cache;


Do we plan to use Redis Cache in a future PR?

Copilot AI and others added 30 commits February 3, 2026 21:06

Initial plan

0dc589e

Add EmbeddingsOptions and EmbeddingProviderType configuration models

d3187bd

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Add CLI configure options for embeddings and register embedding service

6064826

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Add unit tests for embeddings and update JSON schema with embeddings …

0653f15

…configuration Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Simplify HttpClient registration for embedding service

21e81b9

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Plan for embedding service enhancements

0cd8e53

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Refactor embedding code into dedicated namespaces

7fa1c49

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Fix property renames and update tests

c3f6937

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Add EmbeddingsOptionsConverter and fix all tests

1e18c25

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Address code review feedback

857203a

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Address PR feedback: Add Azure OpenAI validation, cache key security,…

89cb2d9

… and telemetry integration Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Optimize ProviderName to avoid repeated string allocations

64b592e

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Fix schema mismatch, remove unused field, add enabled handling, valid…

e8d7238

…ate empty embeddings Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Add embedding health check execution and update JSON schema with endp…

d9c8a29

…oint/health sub-objects Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Add EmbeddingController for /embed REST endpoint with role-based auth…

3e02c0f

…orization Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

Address code review feedback for EmbeddingController

5c05464

Co-authored-by: JerryNixon <1749983+JerryNixon@users.noreply.github.com>

fix: manually deserialize EmbeddingsEndpointOptions and EmbeddingsHea…

c9eba20

…lthCheckConfig in converter

feat: add embeddings config validation and unit tests

d3a5209

Update src/Core/Services/Embeddings/EmbeddingService.cs

203e232

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/Config/Converters/EmbeddingsOptionsConverterFactory.cs

2cb3999

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update src/Service/Startup.cs

b432e4f

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Taking care of feedback

a8442f4

Merge branch 'main' into add-internal-text-embedding-system

a301290

Taking care of copilot feedback, adding embedding service tests, embe…

5f12542

…dding controller tests, switching the dab schema for the embedding system to default to false

Adding default return of application/json and only text/plain if expl…

bd84c84

…icitly set

feat(embeddings): require L2 cache and include provider/model in cach…

7100b8a

…e keys

fix(embeddings): keep L1 allow distributed cache when configured/opti…

73658de

…onal

Adding max text-count validation for embedding requests

ece97ae

Phase 1 Added Embedding Support with Chunking

d065f4b

ajtiwari07 added 2 commits April 29, 2026 13:24

Extract validations into a method

a4a1b6d

Merge branch 'add-internal-text-embedding-system' of https://github.c…

1c44f45

…om/ajtiwari07/data-api-builder into add-internal-text-embedding-system

sayalikudale reviewed Apr 29, 2026

View reviewed changes

ajtiwari07 and others added 3 commits April 29, 2026 23:08

Merge branch 'main' into add-internal-text-embedding-system

6cd5fe8

Avoid duplicate texts in embedding creation API request

437bad5

Merge branch 'add-internal-text-embedding-system' of https://github.c…

48b1020

…om/ajtiwari07/data-api-builder into add-internal-text-embedding-system

ajtiwari07 added 3 commits May 1, 2026 05:19

Fix test failures

f6b830a

Merge branch 'add-internal-text-embedding-system' of https://github.c…

2ffd643

…om/ajtiwari07/data-api-builder into add-internal-text-embedding-system

Fix formatting

fac1c38

JimRoberts-MS reviewed May 1, 2026

View reviewed changes

Comment thread src/Config/ObjectModel/Embeddings/EmbeddingsEndpointOptions.cs Outdated

Merge branch 'main' into add-internal-text-embedding-system

02b72c9

JimRoberts-MS reviewed May 1, 2026

View reviewed changes

Comment thread src/Service/Controllers/EmbeddingController.cs Outdated

ajtiwari07 and others added 4 commits May 1, 2026 11:57

Set default dev mode role and use G9 string format for embeddings

85326de

Merge branch 'add-internal-text-embedding-system' of https://github.c…

202aed4

…om/ajtiwari07/data-api-builder into add-internal-text-embedding-system

Test fix

d79c80e

Merge branch 'main' into add-internal-text-embedding-system

32448bc

Merge branch 'main' into add-internal-text-embedding-system

fc26f9e

Aniruddh25 requested changes May 2, 2026

View reviewed changes

github-project-automation Bot moved this from Todo to Review In Progress in Data API builder May 2, 2026

Conversation

ajtiwari07 commented Apr 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

azure-pipelines Bot commented Apr 29, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ajtiwari07 Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Aniruddh25 commented Apr 29, 2026

Uh oh!

azure-pipelines Bot commented Apr 29, 2026

Uh oh!

souvikghosh04 commented Apr 30, 2026

Uh oh!

azure-pipelines Bot commented Apr 30, 2026

Uh oh!

Uh oh!

Uh oh!

Aniruddh25 commented May 1, 2026

Uh oh!

azure-pipelines Bot commented May 1, 2026

Uh oh!

ajtiwari07 commented May 1, 2026

Uh oh!

azure-pipelines Bot commented May 1, 2026

Uh oh!

Aniruddh25 commented May 2, 2026

Uh oh!

azure-pipelines Bot commented May 2, 2026

Uh oh!

Aniruddh25 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

ajtiwari07 commented Apr 12, 2026 •

edited

Loading

ajtiwari07 Apr 30, 2026 •

edited

Loading