aws: fix config system test (mock response + expired TLS cert)#19002
aws: fix config system test (mock response + expired TLS cert)#19002stefans-elastic wants to merge 8 commits into
Conversation
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
| {{ minify_json ` | ||
| { | ||
| "EvaluationResults": null | ||
| "EvaluationResults": [] |
There was a problem hiding this comment.
Please don't do this. This changes the behaviour of the mock to make it less restrictive, allowing tests to pass when the real data may cause the actual input to fail. The correct fix is to change the CEL program to tolerate the values that can be here. From a survey of the AWS documentation for this document, EvaluationResults does not have a required constraint, so it may be absent completely (the documentation says that this is what should happen, but by Postel, we should probably tolerate a null or a [].
Instead, drop the event.
diff --git a/packages/aws/data_stream/config/agent/stream/cel.yml.hbs b/packages/aws/data_stream/config/agent/stream/cel.yml.hbs
index faab5eb989..6101af66f5 100644
--- a/packages/aws/data_stream/config/agent/stream/cel.yml.hbs
+++ b/packages/aws/data_stream/config/agent/stream/cel.yml.hbs
@@ -322,7 +322,7 @@ program: |
}
)
:
- [{}],
+ [{"retry":true}],
"next_page": {
?"result_token": body.?NextToken,
?"rule_token": config_rules.?next_page.rule_token,
@@ -401,7 +401,8 @@ tags:
{{#contains "forwarded" tags}}
publisher_pipeline.disable_host: true
{{/contains}}
-{{#if processors}}
processors:
+- drop_event.when.equals.retry: true
+{{#if processors}}
{{processors}}
{{/if}}There was a problem hiding this comment.
@efd6 thank you for your comment. I had to tweak the fix a little as it refused working with boolean type. In the end I've done it as it is done in rapid7_insightvm package.
There was a problem hiding this comment.
The alternative is to convert the type to a dyn to get type agreement, dyn([{"retry":true}]), but this will do.
🚀 Benchmarks reportPackage
|
| Data stream | Previous EPS | New EPS | Diff (%) | Result |
|---|---|---|---|---|
rds |
41666.67 | 32258.06 | -9408.61 (-22.58%) | 💔 |
To see the full report comment with /test benchmark fullreport
💚 Build Succeeded
History
|
|
Please update the proposed commit message for the new changes. |
thanks for reminding me (I keep forgetting to do it). |
Proposed commit message
aws: fix config system test (drop empty EvaluationResults + expired TLS cert)
Fix two issues causing aws.config system test failures:
When GetComplianceDetailsByConfigRule returns a null or empty EvaluationResults field, the CEL program now emits a {"message": "retry"} sentinel and a drop_event processor discards it before indexing. Previously the sentinel reached the ingest pipeline without an event.original field, producing a pipeline_error document that broke the hit_count: 4 assertion in the system test.
The mock server response (introduced in v6.14.1 to exercise the null guard) is kept as-is; the fix is in the CEL program, not the mock.
The self-signed certificate for the aws.config mock server expired on 2026-05-06 (1-year validity). All other data streams using the same mock pattern (guardduty, inspector, securityhub) use 10-year certs. Regenerated the certificate and private key with 10-year validity (expires 2036-05-13) and updated the embedded CA in the system test config to match.
Checklist
changelog.ymlfile.Author's Checklist
How to test this PR locally
Related issues
Screenshots