Delivered
Details
Details
Assignee
Arthur She
Arthur SheReporter
Joanna Farley
Joanna FarleyUpstream
No
Priority
Checklist
Checklist
Sentry
Sentry
Created May 25, 2022 at 1:04 PM
Updated October 23, 2023 at 1:44 PM
Resolved June 8, 2022 at 2:39 PM
We have come across a situation where a TF-A Jenkins job is reporting success when an underlying test is failing and reported as failing in LAVA.
We discovered this when we were testing a patch that we expected to generate a CI failure but it did not. It does in Arm’s internal CI and in the OpenCI it is recognised as a failure by LAVA but just not cascaded up to the Jenkins job.
This job we expected to fail
It can be seen that if you inspect the lava logs for either of these jobs that tftf actually failed all tests:
and lava does pick up that the tests failed:
but seems to be returning success for the job overall anyway:
Contact for details.