SSI GitLab Test Optimization

[robertomonteromiguel]

Motivation

Changes

Workflow

1. ⚠️ Create your PR as draft ⚠️
2. Work on you PR until the CI passes
3. Mark it as ready for review
   • Test logic is modified? -> Get a review from RFC owner.
   • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

• Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
• A docker base image is modified?
  • the relevant build-XXX-image label is present
• A scenario is added, removed or renamed?
  • Get a review from R&P team

[github-actions]

CODEOWNERS have been resolved as:

.gitlab-ci.yml                                                          @DataDog/system-tests-core
.gitlab/ssi_gitlab-ci.yml                                               @DataDog/system-tests-core
utils/_context/_scenarios/docker_ssi.py                                 @DataDog/system-tests-core

[datadog-prod-us1-3]

 

✨ Fix all issues with BitsAI

⚠️ Warnings

🚦 61 Pipeline jobs failed

Testing the test | System Tests (python, dev) / End-to-end #1 / django-py3.13 1     

🔧 Fix in code (Fix with Cursor).

3 failed tests. AssertionError: Probes did not reach INSTALLED status. Also 'Snapshot was not received'.

🧪 3 Tests failed

tests.debugger.test_debugger_capture_expressions.Test_Debugger_Line_Capture_Expressions.test_log_line_capture_expressions[django-py3.13] from system_tests_suite   (Fix with Cursor)

AssertionError: assert 'Snapshot was not received' is None
 +  where 'Snapshot was not received' = <built-in method join of str object at 0x7fd6f602b720>(['Snapshot was not received'])
 +    where <built-in method join of str object at 0x7fd6f602b720> = '\n'.join
 +    and   ['Snapshot was not received'] = <tests.debugger.test_debugger_capture_expressions.Test_Debugger_Line_Capture_Expressions object at 0x7fd6ec0f14c0>.setup_failures

self = <tests.debugger.test_debugger_capture_expressions.Test_Debugger_Line_Capture_Expressions object at 0x7fd6ec0f14c0>

    def test_log_line_capture_expressions(self):
>       self._assert()

...

tests.debugger.test_debugger_capture_expressions.Test_Debugger_Method_Capture_Expressions.test_complex_capture_expressions[django-py3.13] from system_tests_suite   (Fix with Cursor)

AssertionError: assert 'Probes did not reach INSTALLED status' is None
 +  where 'Probes did not reach INSTALLED status' = <built-in method join of str object at 0x7fd6f602b720>(['Probes did not reach INSTALLED status'])
 +    where <built-in method join of str object at 0x7fd6f602b720> = '\n'.join
 +    and   ['Probes did not reach INSTALLED status'] = <tests.debugger.test_debugger_capture_expressions.Test_Debugger_Method_Capture_Expressions object at 0x7fd6ec0f1ac0>.setup_failures

self = <tests.debugger.test_debugger_capture_expressions.Test_Debugger_Method_Capture_Expressions object at 0x7fd6ec0f1ac0>

    def test_complex_capture_expressions(self):
>       self._assert()

...

View all 3 test failures

Testing the test | System Tests (python, dev) / End-to-end #1 / fastapi 1     

🔧 Fix in code (Fix with Cursor).

3 failed tests due to assertion error: Probes did not reach INSTALLED status and Snapshot was not received.

🧪 3 Tests failed

tests.debugger.test_debugger_capture_expressions.Test_Debugger_Line_Capture_Expressions.test_log_line_capture_expressions[fastapi] from system_tests_suite   (Fix with Cursor)

AssertionError: assert 'Snapshot was not received' is None
 +  where 'Snapshot was not received' = <built-in method join of str object at 0x7f82e1c2b720>(['Snapshot was not received'])
 +    where <built-in method join of str object at 0x7f82e1c2b720> = '\n'.join
 +    and   ['Snapshot was not received'] = <tests.debugger.test_debugger_capture_expressions.Test_Debugger_Line_Capture_Expressions object at 0x7f82d85114c0>.setup_failures

self = <tests.debugger.test_debugger_capture_expressions.Test_Debugger_Line_Capture_Expressions object at 0x7f82d85114c0>

    def test_log_line_capture_expressions(self):
>       self._assert()

...

tests.debugger.test_debugger_capture_expressions.Test_Debugger_Method_Capture_Expressions.test_complex_capture_expressions[fastapi] from system_tests_suite   (Fix with Cursor)

AssertionError: assert 'Probes did not reach INSTALLED status' is None
 +  where 'Probes did not reach INSTALLED status' = <built-in method join of str object at 0x7f82e1c2b720>(['Probes did not reach INSTALLED status'])
 +    where <built-in method join of str object at 0x7f82e1c2b720> = '\n'.join
 +    and   ['Probes did not reach INSTALLED status'] = <tests.debugger.test_debugger_capture_expressions.Test_Debugger_Method_Capture_Expressions object at 0x7f82d8511580>.setup_failures

self = <tests.debugger.test_debugger_capture_expressions.Test_Debugger_Method_Capture_Expressions object at 0x7f82d8511580>

    def test_complex_capture_expressions(self):
>       self._assert()

...

View all 3 test failures

Testing the test | System Tests (python, dev) / End-to-end #1 / flask-poc 1     

🔧 Fix in code (Fix with Cursor).

3 failed tests due to assertion error: 'Snapshot was not received'.

🧪 3 Tests failed

tests.debugger.test_debugger_capture_expressions.Test_Debugger_Line_Capture_Expressions.test_log_line_capture_expressions[flask-poc] from system_tests_suite   (Fix with Cursor)

AssertionError: assert 'Snapshot was not received' is None
 +  where 'Snapshot was not received' = <built-in method join of str object at 0x7f072f82b720>(['Snapshot was not received'])
 +    where <built-in method join of str object at 0x7f072f82b720> = '\n'.join
 +    and   ['Snapshot was not received'] = <tests.debugger.test_debugger_capture_expressions.Test_Debugger_Line_Capture_Expressions object at 0x7f0721a49760>.setup_failures

self = <tests.debugger.test_debugger_capture_expressions.Test_Debugger_Line_Capture_Expressions object at 0x7f0721a49760>

    def test_log_line_capture_expressions(self):
>       self._assert()

...

tests.debugger.test_debugger_capture_expressions.Test_Debugger_Method_Capture_Expressions.test_complex_capture_expressions[flask-poc] from system_tests_suite   (Fix with Cursor)

AssertionError: assert 'Snapshot was not received' is None
 +  where 'Snapshot was not received' = <built-in method join of str object at 0x7f072f82b720>(['Snapshot was not received'])
 +    where <built-in method join of str object at 0x7f072f82b720> = '\n'.join
 +    and   ['Snapshot was not received'] = <tests.debugger.test_debugger_capture_expressions.Test_Debugger_Method_Capture_Expressions object at 0x7f0721a4b6e0>.setup_failures

self = <tests.debugger.test_debugger_capture_expressions.Test_Debugger_Method_Capture_Expressions object at 0x7f0721a4b6e0>

    def test_complex_capture_expressions(self):
>       self._assert()

...

View all 3 test failures

View all 61 failed jobs.

ℹ️ Info

No other issues found (see more)

❄️ No new flaky tests detected

Useful? React with 👍 / 👎

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: f821c62 | Docs | Datadog PR Page | Give us feedback!}

[cbeauchesne]

LGTM !