feat: add YAML parsing support for Composable Samplers#3966
feat: add YAML parsing support for Composable Samplers#3966DCchoudhury15 wants to merge 14 commits intoopen-telemetry:mainfrom
Conversation
Implements parser logic, SDK builder visitor instantiation, safe test patterns, and expands coverage for all composable sampler variants to resolve open-telemetry#3914. Signed-off-by: DCchoudhury15 <divyanshuchoudhury3@gmail.com>
92ed254 to
ae0c9dd
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3966 +/- ##
=======================================
Coverage 90.18% 90.18%
=======================================
Files 230 230
Lines 7299 7299
=======================================
Hits 6582 6582
Misses 717 717 🚀 New features to boost your workflow:
|
|
Thanks for the PR. Please check the CI logs for failures, and adjust the code accordingly. In the mean time, I will start the code review and provide feedback. Thanks for your contribution. Examples of failures below Include what you use: Build in maintainer mode Clang-tidy Download the clang-tidy report from ci, and inspect failures related to new code. |
|
Hi @marcalff, addressed all the CI failures:
|
Signed-off-by: DCchoudhury15 <divyanshuchoudhury3@gmail.com>
f5be4bf to
659aafe
Compare
| { | ||
| model = ParseTraceIdRatioBasedSamplerConfiguration(child, depth); | ||
| } | ||
| else if (name == "composable_always_off") |
There was a problem hiding this comment.
This is confusing to me - As per the config specs, the YAML shape should be something like:
tracer_provider:
sampler:
composite/development:
always_on:However in the implementation we are using composable_* sampler keys, which seems incorrect. Or am I missing something.
There was a problem hiding this comment.
Lalit is correct here, the only name to check should be "composite/development".
In this case, invoke ParseComposableSamplerConfiguration(), which will lookup the child yaml nodes.
| size_t /* depth */) const | ||
| { | ||
| auto model = std::make_unique<ComposableProbabilitySamplerConfiguration>(); | ||
| model->probability = node->GetDouble("probability", 1.0); |
There was a problem hiding this comment.
The property name in the spec is ratio.
| const std::unique_ptr<DocumentNode> & /* node */, | ||
| size_t /* depth */) const | ||
| { | ||
| return std::make_unique<ComposableParentThresholdSamplerConfiguration>(); |
There was a problem hiding this comment.
Property root is not parsed from yaml.
| const std::unique_ptr<DocumentNode> & /* node */, | ||
| size_t /* depth */) const | ||
| { | ||
| // FIXME-CONFIG: Parse rules list from YAML node here |
There was a problem hiding this comment.
To implement, parse the array of rules from yaml.
| const std::unique_ptr<DocumentNode> & /* node */, | ||
| size_t /* depth */) const | ||
| { | ||
| return std::make_unique<ComposableSamplerConfiguration>(); |
There was a problem hiding this comment.
This is an empty node.
The code needs to parse the child nodes in yaml, and create the proper sampler.
See
minProperties: 1
maxProperties: 1
properties:
always_off:
$ref: "#/$defs/ExperimentalComposableAlwaysOffSampler"
description: Configure sampler to be always_off.
defaultBehavior: ignore
always_on:
$ref: "#/$defs/ExperimentalComposableAlwaysOnSampler"
description: Configure sampler to be always_on.
defaultBehavior: ignore
parent_threshold:
$ref: "#/$defs/ExperimentalComposableParentThresholdSampler"
description: |
Configure sampler to be parent_threshold.
defaultBehavior: ignore
probability:
$ref: "#/$defs/ExperimentalComposableProbabilitySampler"
description: Configure sampler to be probability.
defaultBehavior: ignore
rule_based:
$ref: "#/$defs/ExperimentalComposableRuleBasedSampler"
description: Configure sampler to be rule_based.
defaultBehavior: ignore
marcalff
left a comment
There was a problem hiding this comment.
Thanks for the PR.
Focusing on the yaml part for now: all the yaml nodes and properties should be parsed, and represented in memory in C++ classes.
This includes rules with attribute_values, attribute_patterns and span_kinds.
| { | ||
| model = ParseTraceIdRatioBasedSamplerConfiguration(child, depth); | ||
| } | ||
| else if (name == "composable_always_off") |
There was a problem hiding this comment.
Lalit is correct here, the only name to check should be "composite/development".
In this case, invoke ParseComposableSamplerConfiguration(), which will lookup the child yaml nodes.
| sampler: | ||
| composable_always_on: |
There was a problem hiding this comment.
This is not the expected schema.
See file snippets/Sampler_rule_based_kitchen_sink.yaml in the opentelemetry-configuration repository.
tracer_provider:
processors:
- simple:
exporter:
console:
sampler:
# SNIPPET_START
composite/development:
...
2e15234 to
758e3a5
Compare
|
I have applied the changes given on the pr review , will resolve all the ci errors and commit again asap . |
Signed-off-by: DCchoudhury15 <divyanshuchoudhury3@gmail.com>
13f24d4 to
1a4617d
Compare
Signed-off-by: DCchoudhury15 <divyanshuchoudhury3@gmail.com>
e207db5 to
e22556d
Compare
| virtual void VisitParentBased(const ParentBasedSamplerConfiguration *model) = 0; | ||
| virtual void VisitTraceIdRatioBased(const TraceIdRatioBasedSamplerConfiguration *model) = 0; | ||
| virtual void VisitExtension(const ExtensionSamplerConfiguration *model) = 0; | ||
| virtual void VisitComposableAlwaysOff(const ComposableAlwaysOffSamplerConfiguration *model) = 0; |
There was a problem hiding this comment.
Provide default implementation to avoid breaking downstream users for these 2 virtual functions?
7b65db4 to
b96bd0b
Compare
Signed-off-by: DCchoudhury15 <divyanshuchoudhury3@gmail.com>
05e42ce to
db8b88f
Compare
marcalff
left a comment
There was a problem hiding this comment.
This is getting closer, thanks for the changes.
See some comments on the structure.
| auto model = std::make_unique<ComposableParentThresholdSamplerConfiguration>(); | ||
| std::unique_ptr<DocumentNode> child; | ||
|
|
||
| child = node->GetChildNode("root"); |
There was a problem hiding this comment.
Property root is required, not optional
| if (rules_node) | ||
| { | ||
| for (auto it = rules_node->begin(); it != rules_node->end(); ++it) | ||
| { |
There was a problem hiding this comment.
Please extract the body of the for loop into a separate method, ParseComposableRuleBasedSamplerRuleConfiguration()
| } | ||
|
|
||
| // parse the rule's sampler | ||
| std::unique_ptr<DocumentNode> sampler = rule_node->GetChildNode("sampler"); |
There was a problem hiding this comment.
property sampler is required, not optional.
| auto inner_child = it.Value(); | ||
|
|
||
| if (inner_name == "always_off") | ||
| model->inner = ParseComposableAlwaysOffSamplerConfiguration(inner_child, depth); |
There was a problem hiding this comment.
This returns an instance of ComposableSamplerConfiguration, which points to a different instance of ComposableAlwaysOffConfiguration, using the inner pointer.
Instead, return directly the ComposableAlwaysOffConfiguration object, as a sub class of ComposableSamplerConfiguration.
| std::unique_ptr<ComposableRuleBasedSamplerRuleAttributeValuesConfiguration> attribute_values; | ||
| std::unique_ptr<ComposableRuleBasedSamplerRuleAttributePatternsConfiguration> attribute_patterns; | ||
| std::vector<std::string> parent; | ||
| std::vector<std::string> span_kinds; |
There was a problem hiding this comment.
span_kinds is an enum in the yaml schema:
SpanKind:
type:
- string
- "null"
enum:
- internal
- server
- client
- producer
- consumer
Define a C++ enum for this, and use a vector of enums.
The code will need helpers to convert a string to the enum as well.
[edit]
Or better, define 5 booleans (match_span_kind_internal, match_span_kind_server, etc)
| std::unique_ptr<DocumentNode> av = rule_node->GetChildNode("attribute_values"); | ||
| if (av) | ||
| { | ||
| auto avc = std::make_unique<ComposableRuleBasedSamplerRuleAttributeValuesConfiguration>(); |
There was a problem hiding this comment.
Likewise, extract this code into a helper.
The philosophy is: each yaml type should have a corresponding Parse method.
This helps in the long term, because new yaml nodes can reuse existing types, and then the parsing code for the new node just reuses the existing Parse method for the type.
| model->inner = ParseComposableParentThresholdSamplerConfiguration(inner_child, depth); | ||
| else if (inner_name == "rule_based") | ||
| model->inner = ParseComposableRuleBasedSamplerConfiguration(inner_child, depth); | ||
| } |
There was a problem hiding this comment.
This code should also enforce that there is only one child, and throw yaml parsing exceptions if not.
|
|
||
| std::unique_ptr<ComposableRuleBasedSamplerRuleAttributeValuesConfiguration> attribute_values; | ||
| std::unique_ptr<ComposableRuleBasedSamplerRuleAttributePatternsConfiguration> attribute_patterns; | ||
| std::vector<std::string> parent; |
There was a problem hiding this comment.
parent should be an enum, not a string.
But, on top of that, keeping an array of enums with duplicates is not practical.
Please replace with 3 booleans:
- match_parent_none
- match_parent_remote
- match_parent_local
Signed-off-by: DCchoudhury15 <divyanshuchoudhury3@gmail.com>
8363bcc to
4d117b0
Compare
|
Hi @marcalff , thank you for the detailed review and your patience , I cleaned up the YAML parsing logic by extracting helper methods and strictly enforcing required nodes to ensure it fails fast on bad schemas. I also swapped out the string arrays for boolean flags to make the enum matching much more performant. Finally, I fixed the -Werror unused parameter warnings that were breaking all the maintainer-mode CI builds by safely commenting out those variables in the visitor header. |
Fixes #3914
Changes
This PR implements the necessary infrastructure and test coverage to fully support composable samplers. Specifically, the changes include:
For significant contributions please make sure you have completed the following items:
CHANGELOG.mdupdated for non-trivial changes