Skip to content

Nikhilc/message structure for auto#229

Merged
nikhilNava merged 8 commits intomainfrom
nikhilc/messageStructureForAuto
Apr 17, 2026
Merged

Nikhilc/message structure for auto#229
nikhilNava merged 8 commits intomainfrom
nikhilc/messageStructureForAuto

Conversation

@nikhilNava
Copy link
Copy Markdown
Contributor

@nikhilNava nikhilNava commented Apr 15, 2026

Task
Auto instrumentation should use the new message format
Langchain
AgentFramework

Result

 AgentFramework Spans

 [
   {
     "name": "chat gpt-4o-mini",
     "context": {
       "trace_id": "0x61aa296369694a6904e14842d60c1a09",
       "span_id": "0x39584a2708eea8bb",
       "trace_state": "[]"
     },
     "kind": "SpanKind.INTERNAL",
     "parent_id": "0x545d456b1ce4ebbd",
     "start_time": "2026-04-15T21:37:43.312380+00:00",
     "end_time": "2026-04-15T21:37:44.544233+00:00",
     "status": {
       "status_code": "StatusCode.UNSET",
       "description": null
     },
     "attributes": {
       "gen_ai.input.messages": "{\"messages\": [{\"role\": \"system\", \"parts\": [{\"content\": \"You are a math 
assistant. You MUST use the add_numbers function for any arithmetic. Never compute in your head.\", \"type\": 
\"text\"}]}, {\"role\": \"user\", \"parts\": [{\"content\": \"What is 15 + 27?\", \"type\": \"text\"}]}], 
\"version\": \"0.1.0\"}",
       "gen_ai.system_instructions": "[{\"type\": \"text\", \"content\": \"You are a math assistant. You MUST use 
the add_numbers function for any arithmetic. Never compute in your head.\"}]",
       "gen_ai.request.choice.count": 1,
       "gen_ai.operation.name": "chat",
       "gen_ai.provider.name": "azure.ai.openai",
       "server.address": "https://a365-hw-openai-sentinel.openai.azure.com/openai/deployments/gpt-4o-mini/",
       "gen_ai.request.model": "gpt-4o-mini",
       "gen_ai.response.id": "chatcmpl-DV2DY4F4ow7yoCqRgfhtUDRiXdrR3",
       "gen_ai.response.finish_reasons": "[\"tool_calls\"]",
       "gen_ai.response.model": "gpt-4o-mini-2024-07-18",
       "gen_ai.usage.input_tokens": 110,
       "gen_ai.usage.output_tokens": 19,
       "gen_ai.output.messages": "{\"messages\": [{\"role\": \"assistant\", \"parts\": [{\"name\": \"add_numbers\", 
\"id\": \"call_nGAQp73Qg8614ZwXUQdKPvxM\", \"arguments\": \"{\\\"a\\\":15,\\\"b\\\":27}\", \"type\": 
\"tool_call\"}], \"finish_reason\": \"tool_call\"}], \"version\": \"0.1.0\"}"
     },
     "events": null,
     "links": null,
     "resource": {
       "telemetry.sdk.language": "python",
       "telemetry.sdk.name": "opentelemetry",
       "telemetry.sdk.version": "1.39.1",
       "service.namespace": "agent365-tests",
       "service.name": "integration-test-pipeline"
     }
   },
   {
     "name": "execute_tool add_numbers",
     "context": {
       "trace_id": "0x61aa296369694a6904e14842d60c1a09",
       "span_id": "0xda89beb360c8641a",
       "trace_state": "[]"
     },
     "kind": "SpanKind.INTERNAL",
     "parent_id": "0x545d456b1ce4ebbd",
     "start_time": "2026-04-15T21:37:44.545233+00:00",
     "end_time": "2026-04-15T21:37:44.545233+00:00",
     "status": {
       "status_code": "StatusCode.UNSET",
       "description": null
     },
     "attributes": {
       "gen_ai.operation.name": "execute_tool",
       "gen_ai.tool.name": "add_numbers",
       "gen_ai.tool.call.id": "call_nGAQp73Qg8614ZwXUQdKPvxM",
       "gen_ai.tool.type": "function",
       "gen_ai.tool.description": "Add two numbers together.\n\n    Args:\n        a: First number\n        b: 
Second number\n\n    Returns:\n        The sum of a and b\n    ",
       "gen_ai.tool.call.arguments": "{\"a\": 15.0, \"b\": 27.0}",
       "gen_ai.tool.call.result": "42.0",
       "agent_framework.function.invocation.duration": 3.17e-05
     },
     "events": null,
     "links": null,
     "resource": {
       "telemetry.sdk.language": "python",
       "telemetry.sdk.name": "opentelemetry",
       "telemetry.sdk.version": "1.39.1",
       "service.namespace": "agent365-tests",
       "service.name": "integration-test-pipeline"
     }
   },
   {
     "name": "chat gpt-4o-mini",
     "context": {
       "trace_id": "0x61aa296369694a6904e14842d60c1a09",
       "span_id": "0x40a6c796122c6cc0",
       "trace_state": "[]"
     },
     "kind": "SpanKind.INTERNAL",
     "parent_id": "0x545d456b1ce4ebbd",
     "start_time": "2026-04-15T21:37:44.545233+00:00",
     "end_time": "2026-04-15T21:37:45.194249+00:00",
     "status": {
       "status_code": "StatusCode.UNSET",
       "description": null
     },
     "attributes": {
       "gen_ai.input.messages": "{\"messages\": [{\"role\": \"system\", \"parts\": [{\"content\": \"You are a math 
assistant. You MUST use the add_numbers function for any arithmetic. Never compute in your head.\", \"type\": 
\"text\"}]}, {\"role\": \"user\", \"parts\": [{\"content\": \"What is 15 + 27?\", \"type\": \"text\"}]}, {\"role\": 
\"assistant\", \"parts\": [{\"name\": \"add_numbers\", \"id\": \"call_nGAQp73Qg8614ZwXUQdKPvxM\", \"arguments\": 
\"{\\\"a\\\":15,\\\"b\\\":27}\", \"type\": \"tool_call\"}]}, {\"role\": \"tool\", \"parts\": [{\"id\": 
\"call_nGAQp73Qg8614ZwXUQdKPvxM\", \"response\": \"42.0\", \"type\": \"tool_call_response\"}]}], \"version\": 
\"0.1.0\"}",
       "gen_ai.system_instructions": "[{\"type\": \"text\", \"content\": \"You are a math assistant. You MUST use 
the add_numbers function for any arithmetic. Never compute in your head.\"}]",
       "gen_ai.request.choice.count": 1,
       "gen_ai.operation.name": "chat",
       "gen_ai.provider.name": "azure.ai.openai",
       "server.address": "https://a365-hw-openai-sentinel.openai.azure.com/openai/deployments/gpt-4o-mini/",
       "gen_ai.request.model": "gpt-4o-mini",
       "gen_ai.response.id": "chatcmpl-DV2DYcJecnQ92xqwSDUOkRLjUqdb6",
       "gen_ai.response.finish_reasons": "[\"stop\"]",
       "gen_ai.response.model": "gpt-4o-mini-2024-07-18",
       "gen_ai.usage.input_tokens": 139,
       "gen_ai.usage.output_tokens": 14,
       "gen_ai.output.messages": "{\"messages\": [{\"role\": \"assistant\", \"parts\": [{\"content\": \"The sum of 
15 and 27 is 42.\", \"type\": \"text\"}], \"finish_reason\": \"stop\"}], \"version\": \"0.1.0\"}"
     },
     "events": null,
     "links": null,
     "resource": {
       "telemetry.sdk.language": "python",
       "telemetry.sdk.name": "opentelemetry",
       "telemetry.sdk.version": "1.39.1",
       "service.namespace": "agent365-tests",
       "service.name": "integration-test-pipeline"
     }
   },
   {
     "name": "invoke_agent pipeline-test-agent",
     "context": {
       "trace_id": "0x61aa296369694a6904e14842d60c1a09",
       "span_id": "0x545d456b1ce4ebbd",
       "trace_state": "[]"
     },
     "kind": "SpanKind.CLIENT",
     "parent_id": null,
     "start_time": "2026-04-15T21:37:43.309343Z",
     "end_time": "2026-04-15T21:37:45.194249Z",
     "status": {
       "status_code": "UNSET"
     },
     "attributes": {
       "gen_ai.operation.name": "invoke_agent",
       "telemetry.sdk.name": "A365ObservabilitySDK",
       "telemetry.sdk.language": "python",
       "telemetry.sdk.version": "0.0.0",
       "gen_ai.agent.id": "3bccd52b-daaa-4b11-af40-47443852137c",
       "gen_ai.agent.name": "pipeline-test-agent",
       "gen_ai.agent.description": "Integration test agent for pipeline verification",
       "microsoft.tenant.id": "4d44f041-f91e-4d00-b107-61e47b26f5a8",
       "microsoft.session.id": "test-session-pipeline",
       "gen_ai.input.messages": "{\"messages\": [{\"role\": \"user\", \"parts\": [{\"content\": \"What is 15 + 
27?\", \"type\": \"text\"}]}], \"version\": \"0.1.0\"}"
     },
     "events": [],
     "links": [],
     "resource": {
       "attributes": {
         "telemetry.sdk.language": "python",
         "telemetry.sdk.name": "opentelemetry",
         "telemetry.sdk.version": "1.39.1",
         "service.namespace": "agent365-tests",
         "service.name": "integration-test-pipeline"
       },
       "schema_url": ""
     }
   }
 ]

And for LangChain:

 [
   {
     "name": "chat AzureChatOpenAI",
     "context": {
       "trace_id": "0x2994435e327e1493cf5964863de799f2",
       "span_id": "0x8ec377b0223df6e8",
       "trace_state": "[]"
     },
     "kind": "SpanKind.INTERNAL",
     "parent_id": "0xa41187aa9cc619bf",
     "start_time": "2026-04-15T21:30:44.111477Z",
     "end_time": "2026-04-15T21:30:44.738333Z",
     "status": {
       "status_code": "OK"
     },
     "attributes": {
       "gen_ai.operation.name": "chat",
       "gen_ai.input.messages": "{\"messages\": [{\"role\": \"system\", \"parts\": [{\"content\": \"You are a math 
assistant. You MUST use the add_numbers tool for any arithmetic. Never compute in your head.\", \"type\": 
\"text\"}]}, {\"role\": \"user\", \"parts\": [{\"content\": \"What is 15 + 27?\", \"type\": \"text\"}]}], 
\"version\": \"0.1.0\"}",
       "gen_ai.response.id": "chatcmpl-DV26mmkcyq1vUsBvqLUel8fz2B5ps",
       "gen_ai.output.messages": "{\"messages\": [{\"role\": \"assistant\", \"parts\": [{\"name\": \"add_numbers\", 
\"id\": \"call_W8xtkB6u2KrXBCjmyZQ73sZk\", \"arguments\": \"{\\\"a\\\": 15, \\\"b\\\": 27}\", \"type\": 
\"tool_call\"}], \"finish_reason\": \"tool_calls\"}], \"version\": \"0.1.0\"}",
       "gen_ai.tool.call.arguments.0": "{\"type\": \"function\", \"function\": {\"name\": \"add_numbers\", 
\"description\": \"Add two numbers together.\\n\\n    Args:\\n        a: First number\\n        b: Second 
number\\n\\n    Returns:\\n        A string describing the sum.\", \"parameters\": {\"properties\": {\"a\": 
{\"type\": \"number\"}, \"b\": {\"type\": \"number\"}}, \"required\": [\"a\", \"b\"], \"type\": \"object\"}}}",
       "gen_ai.provider.name": "azure",
       "gen_ai.request.model": "gpt-4o-mini-2024-07-18",
       "gen_ai.usage.input_tokens": 99,
       "gen_ai.usage.output_tokens": 19,
       "gen_ai.response.finish_reasons": 0
     },
     "events": [],
     "links": [],
     "resource": {
       "attributes": {
         "telemetry.sdk.language": "python",
         "telemetry.sdk.name": "opentelemetry",
         "telemetry.sdk.version": "1.39.1",
         "service.namespace": "agent365-tests",
         "service.name": "integration-test-langchain-pipeline"
       },
       "schema_url": ""
     }
   },
   {
     "name": "execute_tool add_numbers",
     "context": {
       "trace_id": "0x2994435e327e1493cf5964863de799f2",
       "span_id": "0xeb53d908c379350f",
       "trace_state": "[]"
     },
     "kind": "SpanKind.INTERNAL",
     "parent_id": "0xa41187aa9cc619bf",
     "start_time": "2026-04-15T21:30:44.739328Z",
     "end_time": "2026-04-15T21:30:44.740326Z",
     "status": {
       "status_code": "OK"
     },
     "attributes": {
       "gen_ai.operation.name": "execute_tool",
       "gen_ai.tool.type": "extension",
       "gen_ai.tool.name": "add_numbers",
       "gen_ai.tool.description": "Add two numbers together.\n\n    Args:\n        a: First number\n        b: 
Second number\n\n    Returns:\n        A string describing the sum.",
       "gen_ai.tool.call.id": "call_W8xtkB6u2KrXBCjmyZQ73sZk",
       "gen_ai.tool.call.arguments": "{'a': 15, 'b': 27}",
       "gen_ai.tool.call.result": "The sum of 15.0 and 27.0 is 42.0"
     },
     "events": [],
     "links": [],
     "resource": {
       "attributes": {
         "telemetry.sdk.language": "python",
         "telemetry.sdk.name": "opentelemetry",
         "telemetry.sdk.version": "1.39.1",
         "service.namespace": "agent365-tests",
         "service.name": "integration-test-langchain-pipeline"
       },
       "schema_url": ""
     }
   },
   {
     "name": "chat AzureChatOpenAI",
     "context": {
       "trace_id": "0x2994435e327e1493cf5964863de799f2",
       "span_id": "0x039235893d2ee73e",
       "trace_state": "[]"
     },
     "kind": "SpanKind.INTERNAL",
     "parent_id": "0xa41187aa9cc619bf",
     "start_time": "2026-04-15T21:30:44.741327Z",
     "end_time": "2026-04-15T21:30:45.551581Z",
     "status": {
       "status_code": "OK"
     },
     "attributes": {
       "gen_ai.operation.name": "chat",
       "gen_ai.input.messages": "{\"messages\": [{\"role\": \"system\", \"parts\": [{\"content\": \"You are a math 
assistant. You MUST use the add_numbers tool for any arithmetic. Never compute in your head.\", \"type\": 
\"text\"}]}, {\"role\": \"user\", \"parts\": [{\"content\": \"What is 15 + 27?\", \"type\": \"text\"}]}, {\"role\": 
\"assistant\", \"parts\": [{\"name\": \"add_numbers\", \"id\": \"call_W8xtkB6u2KrXBCjmyZQ73sZk\", \"arguments\": 
\"{\\\"a\\\": 15, \\\"b\\\": 27}\", \"type\": \"tool_call\"}]}, {\"role\": \"tool\", \"parts\": [{\"id\": 
\"call_W8xtkB6u2KrXBCjmyZQ73sZk\", \"response\": \"content='The sum of 15.0 and 27.0 is 42.0' name='add_numbers' 
tool_call_id='call_W8xtkB6u2KrXBCjmyZQ73sZk'\", \"type\": \"tool_call_response\"}]}], \"version\": \"0.1.0\"}",
       "gen_ai.response.id": "chatcmpl-DV26mbNu5Hn1OltLi4vJKBQhngTo0",
       "gen_ai.output.messages": "{\"messages\": [{\"role\": \"assistant\", \"parts\": [{\"content\": \"The sum of 
15 and 27 is 42.\", \"type\": \"text\"}], \"finish_reason\": \"stop\"}], \"version\": \"0.1.0\"}",
       "gen_ai.tool.call.arguments.0": "{\"type\": \"function\", \"function\": {\"name\": \"add_numbers\", 
\"description\": \"Add two numbers together.\\n\\n    Args:\\n        a: First number\\n        b: Second 
number\\n\\n    Returns:\\n        A string describing the sum.\", \"parameters\": {\"properties\": {\"a\": 
{\"type\": \"number\"}, \"b\": {\"type\": \"number\"}}, \"required\": [\"a\", \"b\"], \"type\": \"object\"}}}",
       "gen_ai.provider.name": "azure",
       "gen_ai.request.model": "gpt-4o-mini-2024-07-18",
       "gen_ai.usage.input_tokens": 175,
       "gen_ai.usage.output_tokens": 14,
       "gen_ai.response.finish_reasons": 0
     },
     "events": [],
     "links": [],
     "resource": {
       "attributes": {
         "telemetry.sdk.language": "python",
         "telemetry.sdk.name": "opentelemetry",
         "telemetry.sdk.version": "1.39.1",
         "service.namespace": "agent365-tests",
         "service.name": "integration-test-langchain-pipeline"
       },
       "schema_url": ""
     }
   },
   {
     "name": "invoke_agent langchain-pipeline-test-agent",
     "context": {
       "trace_id": "0x2994435e327e1493cf5964863de799f2",
       "span_id": "0xa41187aa9cc619bf",
       "trace_state": "[]"
     },
     "kind": "SpanKind.CLIENT",
     "parent_id": null,
     "start_time": "2026-04-15T21:30:44.101416Z",
     "end_time": "2026-04-15T21:30:45.553337Z",
     "status": {
       "status_code": "UNSET"
     },
     "attributes": {
       "gen_ai.operation.name": "invoke_agent",
       "telemetry.sdk.name": "A365ObservabilitySDK",
       "telemetry.sdk.language": "python",
       "telemetry.sdk.version": "0.0.0",
       "gen_ai.agent.id": "3bccd52b-daaa-4b11-af40-47443852137c",
       "gen_ai.agent.name": "langchain-pipeline-test-agent",
       "gen_ai.agent.description": "Integration test agent for LangChain pipeline verification",
       "microsoft.tenant.id": "4d44f041-f91e-4d00-b107-61e47b26f5a8",
       "microsoft.session.id": "test-langchain-pipeline",
       "gen_ai.input.messages": "{\"messages\": [{\"role\": \"user\", \"parts\": [{\"content\": \"What is 15 + 
27?\", \"type\": \"text\"}]}], \"version\": \"0.1.0\"}"
     },
     "events": [],
     "links": [],
     "resource": {
       "attributes": {
         "telemetry.sdk.language": "python",
         "telemetry.sdk.name": "opentelemetry",
         "telemetry.sdk.version": "1.39.1",
         "service.namespace": "agent365-tests",
         "service.name": "integration-test-langchain-pipeline"
       },
       "schema_url": ""
     }
   }
 ]

Copilot AI review requested due to automatic review settings April 15, 2026 21:40
@nikhilNava nikhilNava requested review from a team as code owners April 15, 2026 21:40
@github-actions
Copy link
Copy Markdown

github-actions bot commented Apr 15, 2026

⚠️ Deprecation Warning: The deny-licenses option is deprecated for possible removal in the next major release. For more information, see issue 997.

Dependency Review

The following issues were found:
  • ✅ 0 vulnerable package(s)
  • ✅ 0 package(s) with incompatible licenses
  • ✅ 0 package(s) with invalid SPDX license definitions
  • ⚠️ 3 package(s) with unknown licenses.
See the Details below.

License Issues

uv.lock

PackageVersionLicenseIssue Type
langchain-openai1.1.12NullUnknown License
langchain-core1.2.28NullUnknown License
regex2026.4.4NullUnknown License
Denied Licenses: GPL-3.0-only, AGPL-3.0-only

OpenSSF Scorecard

PackageVersionScoreDetails
pip/langchain-openai 1.1.12 UnknownUnknown
pip/langchain-core 1.2.28 UnknownUnknown
pip/regex 2026.4.4 UnknownUnknown
pip/tiktoken 0.12.0 UnknownUnknown

Scanned Files

  • uv.lock

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates observability extensions to emit A365’s versioned gen_ai.input.messages / gen_ai.output.messages structure (including tool-call message parts), and adds end-to-end integration tests to validate a single-trace span hierarchy across invoke → inference/chat → tool execution.

Changes:

  • Add LangChain message mapping to A365 versioned message format and update LangChain span attribute extraction to use it.
  • Add AgentFramework message mapping (via span enricher) to A365 versioned message format and expand pipeline/message-format integration tests.
  • Add langchain-openai to dev dependencies and update lockfile accordingly.

Reviewed changes

Copilot reviewed 14 out of 15 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
uv.lock Adds langchain-openai and its dependency graph updates (e.g., tiktoken, regex, updated langchain-core).
pyproject.toml Adds langchain-openai to workspace dev dependencies.
libraries/microsoft-agents-a365-observability-extensions-langchain/.../utils.py Switches LangChain message attribute extraction to emit A365 versioned message JSON via new mapper.
libraries/microsoft-agents-a365-observability-extensions-langchain/.../message_mapper.py New LangChain → A365 versioned message mapper (roles/parts/tool calls).
libraries/microsoft-agents-a365-observability-extensions-agentframework/.../trace_instrumentor.py Updates instrumentor _instruments minimum to agent-framework-azure-ai >= 1.0.0rc1.
libraries/microsoft-agents-a365-observability-extensions-agentframework/.../span_processor.py Clarifies that mutation happens via export-time enricher due to ReadableSpan immutability.
libraries/microsoft-agents-a365-observability-extensions-agentframework/.../span_enricher.py Adds versioned message mapping for invoke/chat spans; keeps tool attribute normalization for execute_tool spans.
libraries/microsoft-agents-a365-observability-extensions-agentframework/.../message_mapper.py New AgentFramework span-tag messages → A365 versioned message mapper.
tests/observability/extensions/langchain/integration/* New LangChain integration tests + fixtures for pipeline span hierarchy and message format capture.
tests/observability/extensions/agentframework/integration/* New AgentFramework integration tests for pipeline span hierarchy and message format capture.
tests/observability/extensions/agentframework/test_span_enricher.py Updates unit test assertions to expect A365 versioned message wrapper JSON.

Copilot AI review requested due to automatic review settings April 17, 2026 13:17
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 15 changed files in this pull request and generated 3 comments.

Copilot AI review requested due to automatic review settings April 17, 2026 14:05
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 16 out of 17 changed files in this pull request and generated 3 comments.

@nikhilNava nikhilNava merged commit 3545ab3 into main Apr 17, 2026
13 checks passed
@nikhilNava nikhilNava deleted the nikhilc/messageStructureForAuto branch April 17, 2026 19:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants