Over the May’s week, there was preparation phase:

1. worked on planning/estimation document: .

2. I was getting initial acquaintence/experience with the ECLAIR tool - installing it locally, reviewing manual, figuring out basic licence management matters, running sample projects, etc.

There was also a few meeting with Arm and/or Bugseng to help with both planning and initial learning of the tool.

My understanding is that as of now the planning doc is ready and pending approval from Arm side. After that, specific implementation subtasks of this ticket should be created. So, setting as Blocked and reassigning to to handle further planning/scheduling work.

Some update: We had a meeting on (re)starting work on this task on Fri 2022-02-29. 's notes of this meeting:

Attendees: Madhukar, Ben, Paul, Sandrine, Don, Joanna

Goal: Lay out tasks for enabling ECLAIR into TF-A

• TFC-10 needs cleaned up and focus on TF-A

• Create a new TF-M ticket for later.

• Create subtasks for TFC-10

ACTIONS:
- Madhu provide that "top 10" configs to use
- Arm provide the current workload on the Coverity tasks. 
- Linaro lay out draft of subtasks prior to creating in Jira 
- Don, check if it's possible to run ECLAIR locally? (desktop). Check on ECLAIR licensing for use of a "Desktop" use case where pre-runs done locally prior to submitting to the CI.  
- Sandrine/Madhu: Come back with a better understanding of how often a baseline needs run (per patch? Per day?  Per something else?).
- Madhu share the zoom spin-up recorded session with Bugseng to Paul/Ben.  Also provide a pdf user guide and any other info gathered during the eval.
- Don set up follow-on in 2 weeks. Invite sent

Minutes:
Architecture of what's needed(Joanna)

• Model like how Coverity is done in Arm (Jenkins Jobs and Arm)

• Patch testing is the focus.  Part of allow CI+2 kicks off MISRA run

• Before can do that, need a baseline across the whole TF-A repo

• Ran many passes using Coverity historically.  Lots of "recommends"

• Need a baseline w/ ECLAIR,  make sure no mandate issues found.  Once get baseline (recommends), that can be used to attain deltas.

• 2 Jobs: One to redo the baseline(occasionally).  One to run w/ the patches for deltas.

• Expect multiple Build configurations.  For each build config, a baseline needs run. A daily run would be expected - becomes the golden reference.

• Sandrine: Not a single baseline, but many since many ways to build TF-A (specific platform is one, can have sub-configurations). Currently have dozens up to perhaps hundreds.  When a new patch, must test against the right baseline.  The tool can understand which baseline can use.

• Madhu: Bugseng must be able to know what baseline to use. Coverity uses a query commit to determine which baseline to use.  As of today, do all build config tests on each and every patch being analyzed. Today don't have intelligence to "know" which ones to run, so run them all.

• Sandrine: Optimization would be needed too

• What would be the Jira subtasks for TF-A enablement (TF-M to be handled later)? List below is a starting point but needs updated.
  Spin up on ECLAIR. Understand how ECLAIR was selected (criteria). 
  Madhu: Had an evaluation sync w/ Bugseng. Can we share the zoom session.
  Set up ECLAIR in the environment to run on the server/docker image for Jenkins jobs to reference
  Install and commission of the tool
  Enable baseline job
  Enable delta job
  Developer Notes
  Verify against existing runs from Coverity?  
  Validation of results? 
  Optimization 
  Ben: May need AWS to run these multiple runs.  Must store the data
  Cost analysis report
  Joanne: Need optimizations up front to not run every config.  This will be a difficult task.  May need to bring up some Python regular expression logic 
  Madhu: Initial bootup/bringup would be first task to spin up on that. 
  Madhu can share a handful of configs that would be priority. This would reduce to a dozen or less configs to prevent AWS  churn.
  Ben: Interested in workload.
  Sandrine: Can provide coverity tools locally. Never had it set up to run locally though never got there. Perhaps the CI would just have a few configurations and dev can run more configs locally, could be a hybrid solution to reduce costs.
  Ben: Even Tuxrun could help with the above.
  Sandrine: If allow to run locally, need authentication, etc.
  Madhu: Didn't evaluate using a Desktop model.... CI versus was focused on report generation.
  Joanna: The baseline is not static, it's re-generated daily or less. If failures or no patches to be caught up w/ wouldn't run at all.
  Sandrine: Can give results per patch, so must be able to compare w/ baseline from previous patch, so may be a baseline per patch. Must check.
  Madhu: Only a handful of baselines needed and put into a snapshot maintained by server.  Query the baseline using query node. 
  Paul: ECLAIR is new, perhaps an introduction of the evaluation.  Different build configs is confusing.  
  Note that Bugseng agreement has some consulting hours for initial configuration.

Perforce - the provider of Helix QAC  declined as well due to the Open Source access model.  They "operate on a named user not floating license model."

We may need to evaluate whether there is a creative way to work around this licensing issue we keep hitting.

Thanks for catching my error you are correct, it's Parasoft not Coverity in Zephyr.

And coincidentally I have a call set up today with Perforce to walk thru Helix QAC. 

> Coverity is the closest, whom are running a pilot on Zephyr...

Slight correction - it's Parasoft doing the Zephyr pilot.

Other possible alternatives include:

• Klocwork - https://www.perforce.com/products/klocwork

• Helix QAC (formerly PRQA) https://www.perforce.com/products/helix-qac

• SonarQube - https://www.sonarqube.org/features/security/