feat: add YAML parsing support for Composable Samplers#3966
feat: add YAML parsing support for Composable Samplers#3966DCchoudhury15 wants to merge 15 commits intoopen-telemetry:mainfrom
Conversation
Implements parser logic, SDK builder visitor instantiation, safe test patterns, and expands coverage for all composable sampler variants to resolve open-telemetry#3914. Signed-off-by: DCchoudhury15 <[email protected]>
92ed254 to
ae0c9dd
Compare
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3966 +/- ##
==========================================
- Coverage 90.33% 90.32% -0.01%
==========================================
Files 230 230
Lines 7299 7299
==========================================
- Hits 6593 6592 -1
- Misses 706 707 +1 🚀 New features to boost your workflow:
|
|
Thanks for the PR. Please check the CI logs for failures, and adjust the code accordingly. In the mean time, I will start the code review and provide feedback. Thanks for your contribution. Examples of failures below Include what you use: Build in maintainer mode Clang-tidy Download the clang-tidy report from ci, and inspect failures related to new code. |
|
Hi @marcalff, addressed all the CI failures:
|
Signed-off-by: DCchoudhury15 <[email protected]>
f5be4bf to
659aafe
Compare
| { | ||
| model = ParseTraceIdRatioBasedSamplerConfiguration(child, depth); | ||
| } | ||
| else if (name == "composable_always_off") |
There was a problem hiding this comment.
This is confusing to me - As per the config specs, the YAML shape should be something like:
tracer_provider:
sampler:
composite/development:
always_on:However in the implementation we are using composable_* sampler keys, which seems incorrect. Or am I missing something.
There was a problem hiding this comment.
Lalit is correct here, the only name to check should be "composite/development".
In this case, invoke ParseComposableSamplerConfiguration(), which will lookup the child yaml nodes.
marcalff
left a comment
There was a problem hiding this comment.
Thanks for the PR.
Focusing on the yaml part for now: all the yaml nodes and properties should be parsed, and represented in memory in C++ classes.
This includes rules with attribute_values, attribute_patterns and span_kinds.
| { | ||
| model = ParseTraceIdRatioBasedSamplerConfiguration(child, depth); | ||
| } | ||
| else if (name == "composable_always_off") |
There was a problem hiding this comment.
Lalit is correct here, the only name to check should be "composite/development".
In this case, invoke ParseComposableSamplerConfiguration(), which will lookup the child yaml nodes.
| sampler: | ||
| composable_always_on: |
There was a problem hiding this comment.
This is not the expected schema.
See file snippets/Sampler_rule_based_kitchen_sink.yaml in the opentelemetry-configuration repository.
tracer_provider:
processors:
- simple:
exporter:
console:
sampler:
# SNIPPET_START
composite/development:
...
2e15234 to
758e3a5
Compare
|
I have applied the changes given on the pr review , will resolve all the ci errors and commit again asap . |
Signed-off-by: DCchoudhury15 <[email protected]>
13f24d4 to
1a4617d
Compare
Signed-off-by: DCchoudhury15 <[email protected]>
e207db5 to
e22556d
Compare
| virtual void VisitParentBased(const ParentBasedSamplerConfiguration *model) = 0; | ||
| virtual void VisitTraceIdRatioBased(const TraceIdRatioBasedSamplerConfiguration *model) = 0; | ||
| virtual void VisitExtension(const ExtensionSamplerConfiguration *model) = 0; | ||
| virtual void VisitComposableAlwaysOff(const ComposableAlwaysOffSamplerConfiguration *model) = 0; |
There was a problem hiding this comment.
Provide default implementation to avoid breaking downstream users for these 2 virtual functions?
7b65db4 to
b96bd0b
Compare
Signed-off-by: DCchoudhury15 <[email protected]>
05e42ce to
db8b88f
Compare
marcalff
left a comment
There was a problem hiding this comment.
This is getting closer, thanks for the changes.
See some comments on the structure.
| auto model = std::make_unique<ComposableParentThresholdSamplerConfiguration>(); | ||
| std::unique_ptr<DocumentNode> child; | ||
|
|
||
| child = node->GetChildNode("root"); |
There was a problem hiding this comment.
Property root is required, not optional
| } | ||
|
|
||
| // parse the rule's sampler | ||
| std::unique_ptr<DocumentNode> sampler = rule_node->GetChildNode("sampler"); |
There was a problem hiding this comment.
property sampler is required, not optional.
| auto inner_child = it.Value(); | ||
|
|
||
| if (inner_name == "always_off") | ||
| model->inner = ParseComposableAlwaysOffSamplerConfiguration(inner_child, depth); |
There was a problem hiding this comment.
This returns an instance of ComposableSamplerConfiguration, which points to a different instance of ComposableAlwaysOffConfiguration, using the inner pointer.
Instead, return directly the ComposableAlwaysOffConfiguration object, as a sub class of ComposableSamplerConfiguration.
8363bcc to
4d117b0
Compare
|
Hi @marcalff , thank you for the detailed review and your patience , I cleaned up the YAML parsing logic by extracting helper methods and strictly enforcing required nodes to ensure it fails fast on bad schemas. I also swapped out the string arrays for boolean flags to make the enum matching much more performant. Finally, I fixed the -Werror unused parameter warnings that were breaking all the maintainer-mode CI builds by safely commenting out those variables in the visitor header. |
| { | ||
| public: | ||
| ComposableParentThresholdSamplerConfiguration() = default; | ||
| std::unique_ptr<SamplerConfiguration> root; |
There was a problem hiding this comment.
In the yaml spec:
ExperimentalComposableParentThresholdSampler:
type:
- object
additionalProperties: false
properties:
root:
$ref: "#/$defs/ExperimentalComposableSampler"
description: Sampler to use when there is no parent.
required:
- root
root is a ComposableSampler, meaning it allows only some types but not all samplers.
For example, root can not point to a jaeger remote sampler, which SamplerConfiguration allows.
Change root to ComposableSamplerConfiguration, which means in turn that classes like ComposableAlwaysOnSamplerConfiguration must be a subclass of ComposableSamplerConfiguration, not SamplerConfiguration.
| bool match_span_kind_producer{false}; | ||
| bool match_span_kind_consumer{false}; | ||
|
|
||
| std::unique_ptr<SamplerConfiguration> sampler; |
There was a problem hiding this comment.
sampler is a ComposableSamplerConfiguration, not just a SamplerConfiguration
| { | ||
| public: | ||
| ComposableSamplerConfiguration() = default; | ||
| std::unique_ptr<SamplerConfiguration> inner; |
| public: | ||
| ComposableSamplerConfiguration() = default; | ||
| std::unique_ptr<SamplerConfiguration> inner; | ||
| void Accept(SamplerConfigurationVisitor *visitor) const override; |
There was a problem hiding this comment.
Remove Accept.
ComposableSamplerConfiguration should be an abstract class, children will implement Accept for each composable sampler type supported.
| ParseComposableRuleBasedSamplerConfiguration(const std::unique_ptr<DocumentNode> &node, | ||
| size_t depth) const; | ||
|
|
||
| std::unique_ptr<SamplerConfiguration> ParseComposableSamplerConfiguration( |
There was a problem hiding this comment.
This should return a ComposableSamplerConfiguration.
| {} | ||
| virtual void VisitComposableRuleBased(const ComposableRuleBasedSamplerConfiguration * /*model*/) | ||
| {} | ||
| virtual void VisitComposable(const ComposableSamplerConfiguration * /*model*/) {} |
| const std::unique_ptr<DocumentNode> &node) const | ||
| { | ||
| auto model = std::make_unique<ComposableRuleBasedSamplerRuleAttributeValuesConfiguration>(); | ||
| model->key = node->GetString("key", ""); |
There was a problem hiding this comment.
key is required, use GetRequiredString.
| { | ||
| auto model = std::make_unique<ComposableRuleBasedSamplerRuleAttributeValuesConfiguration>(); | ||
| model->key = node->GetString("key", ""); | ||
| auto vals = node->GetChildNode("values"); |
| const std::unique_ptr<DocumentNode> &node) const | ||
| { | ||
| auto model = std::make_unique<ComposableRuleBasedSamplerRuleAttributePatternsConfiguration>(); | ||
| model->key = node->GetString("key", ""); |
| else if (p_str == "remote") | ||
| rule->match_parent_remote = true; | ||
| else if (p_str == "local") | ||
| rule->match_parent_local = true; |
There was a problem hiding this comment.
else ... raise an exception for illegal values that do not fit the enum.
Also, please add a unit test for illegal values, for coverage.
| else if (k_str == "producer") | ||
| rule->match_span_kind_producer = true; | ||
| else if (k_str == "consumer") | ||
| rule->match_span_kind_consumer = true; |
There was a problem hiding this comment.
Likewise:
else ... raise an exception for illegal values that do not fit the enum.
add a unit test for illegal values, for coverage.
| TestSamplerVisitor visitor; | ||
| config->tracer_provider->sampler->Accept(&visitor); | ||
| EXPECT_EQ(visitor.type_matched, SamplerType::kComposableParentThreshold); | ||
| } |
There was a problem hiding this comment.
Thanks for the unit tests.
Please also add tests to cover the rule based samplers, for coverage.
marcalff
left a comment
There was a problem hiding this comment.
Thanks for all the changes to date, the yaml parsing itself is almost there.
Typing in the model needs adjustments, the whole point of Sampler versus ComposableSampler is to have strong typing in the yaml schema, so the C++ model classes should represent that.
Fixes #3914
Changes
This PR implements the necessary infrastructure and test coverage to fully support composable samplers. Specifically, the changes include:
For significant contributions please make sure you have completed the following items:
CHANGELOG.mdupdated for non-trivial changes