kselftest: Update kselftest results parser in test-definitions - Bad test result:

Description

  • Kselftest results-parser

    • Need to improve the speed of kselftest results parser functionality currently it is taking too long to parse

    • Kselftest arm64 following two and more test cases output is matching with TAP 13 format and but needs investigation to solve the problem. 

      • sve-ptrace

      • Za-ptrace

    • The fix could be test-definitions kselftest

      • Should be pass/fail/skip here we other pattern

        • <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=arm64.syscall-abi.getpid() RESULT=SVE>

        • <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=arm64.syscall-abi.sched_yield() RESULT=SVE>

        • <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=arm64.tpidr2.skipped, RESULT=TPIDR2>

        • <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=arm64.za-ptrace.Set RESULT=VL>

        • <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=arm64.sve-ptrace.SVE RESULT=FPSIMD>

        • <LAVA_SIGNAL_TESTCASE TEST_CASE_ID=arm64.sve-ptrace.SVE RESULT=SVE_PT_VL_INHERIT>

      • Results_parser() in test-definitions to be fixed

 

Ref: LAVA job not parsing correctly:

Ref: LAVA job not parsing correctly:

 

Test results parser showing “Bad test results: “

Received signal: <TESTCASE> TEST_CASE_ID=arm64.sve-ptrace.SVE RESULT=FPSIMD
Bad test result: FPSIMD
<LAVA_SIGNAL_TESTCASE TEST_CASE_ID=arm64.sve-ptrace.SVE RESULT=FPSIMD>
Received signal: <TESTCASE> TEST_CASE_ID=arm64.sve-ptrace.SVE RESULT=get_fpsimd()
Bad test result: get_fpsimd()
<LAVA_SIGNAL_TESTCASE TEST_CASE_ID=arm64.sve-ptrace.SVE RESULT=get_fpsimd()>
Received signal: <TESTCASE> TEST_CASE_ID=arm64.sve-ptrace.SVE RESULT=SVE_PT_VL_INHERIT
Bad test result: SVE_PT_VL_INHERIT
<LAVA_SIGNAL_TESTCASE TEST_CASE_ID=arm64.sve-ptrace.SVE RESULT=SVE_PT_VL_INHERIT>
Received signal: <TESTCASE> TEST_CASE_ID=arm64.sve-ptrace.SVE RESULT=SVE_PT_VL_INHERIT
Bad test result: SVE_PT_VL_INHERIT
<LAVA_SIGNAL_TESTCASE TEST_CASE_ID=arm64.sve-ptrace.SVE RESULT=SVE_PT_VL_INHERIT>
Received signal: <TESTCASE> TEST_CASE_ID=arm64.sve-ptrace.Set RESULT=SVE
Bad test result: SVE
<LAVA_SIGNAL_TESTCASE TEST_CASE_ID=arm64.sve-ptrace.Set RESULT=SVE>
<LAVA_SIGNAL_TESTCASE TEST_CASE_ID=arm64.sve-ptrace.Set RESULT=and>
Received signal: <TESTCASE> TEST_CASE_ID=arm64.sve-ptrace.Set RESULT=and
Bad test result: and
Received signal: <TESTCASE> TEST_CASE_ID=arm64.sve-ptrace.Set RESULT=and
Bad test result: and

 

And few more test cases are having this problem,

Bad test result: Result
<LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clone3.clone3_set_tid.[1710] RESULT=Result>
Received signal: <TESTCASE> TEST_CASE_ID=clone3.clone3_set_tid.[1710] RESULT=Result
Bad test result: Result
<LAVA_SIGNAL_TESTCASE TEST_CASE_ID=clone3.clone3_set_tid.PIDs RESULT=in>
Received signal: <TESTCASE> TEST_CASE_ID=clone3.clone3_set_tid.PIDs RESULT=in
Bad test result: in

Activity

Naresh Kamboju December 28, 2022 at 11:03 AM

The reported kselftest resutls parser problem has solved now.

This is under review and shared with kernel CI IRC for upstream review.

Naresh Kamboju December 15, 2022 at 9:41 AM

Identified that PID are printing in test-case-names which is not a good idea.

Naresh Kamboju December 15, 2022 at 9:40 AM

The details of PR has been shared with kernelCI over IRC for further review.

Naresh Kamboju December 1, 2022 at 7:47 AM

Work is in progress - Test-definitions kselftests are getting modified with new results parser script.

The work is in progress under new parse-output branch.

Naresh Kamboju November 21, 2022 at 7:40 AM

I have reported this problem upstream,

Done

Details

Assignee

Reporter

Share Visibility

Anders Roxell
Benjamin Copeland
Daniel Díaz
Remi Duraffort

Original estimate

Time tracking

1w 2d logged

Sprint

Priority

Checklist

Sentry

Created November 1, 2022 at 12:12 PM
Updated December 28, 2022 at 11:15 AM
Resolved December 28, 2022 at 11:15 AM