Open-CMSIS-Pack Technical Meeting 2022-07-12

Participants

Evgueni Driouk

Silvio Lucio Oliva

Joachim Krech

Daniel Brondani

David Jurajda

Luís Tonicha

Samuel Hultgren

Eric Finco

Bill Fletcher

Graham Hammond

Vincent Grenet

Reinhard Keil

Jaeden Amero

Kyle Dando

Charles Oliveira

Holt Sun

Slides

STMicroelectronics - Lockfile

Meeting Notes

Agenda:

JK: We have three items from ST: proposal on csolution lockfile, permissive project list in csolution.yml, describing parts in Open-CMSIS-Pack.

Actions from last week:

ST/NXP to progress “Rediscuss the ‘hidden’ proposal from NXP #112

No progress: Fred is on vacation, will be back on August 1^st.

Change Board:

We had yesterday our Decision Committee Meeting

We set 5 issues that were accepted: #25, #124, #126, #131, #141

Any volunteer to update the schema and documentation and create PRs is welcome.
We had 2 items being postponed: #95 and #134
We agreed on scheduling the next meeting for August 29^th.

Handover to Samuel on lockfile and optional RTE creation

SH:

Motivations for lockfile: allow loose but still reproducible/stable/shareable, provide visibility of changes (visible in git diff), simple total upgrade flow just by removing the lockfile, simplify file management in tooling

Minimum requirements:

ComponentID for each component used (component attributes + condition + pack id + relative pack path for local packs) with back-reference to make link with user-input in YML;

File set for each component used: path_in_project (e.g. for RTE), path_in_pack.

Today csolution and cbuildgen duplicates parts of the RTE processing. To me this is not optimal because if we implement some kind of flexibility on the YML side or on the PDSC side we need to implement support in cprj.

If we move to my ideal future, csolution is modified to create the lock file instead of the cprj, however cbuildgen is simplified, it takes the lock file that describes where all the files are, then it just output CMakeLists files.

I also imagine a new tool CPRJ to YML with the only purpose to create YML files.

RK: Basically it’s the rational why you have in the lock file the path to packs and things like that.

SH: There are multiple reasons, it simplifies the tooling, if you want to implement flexibility in the YML side, you don’t need to re-implement the thing in cprj. In my view cprj is deprecated.

JK: I do agree cbuildgen has capability of doing RTE again or generated components. If the csolution has done the RTE and the component generation, despite the fact that cbuildgen has the capability, it wouldn’t run anything.

SH: It would detect that the files are already there, but still we have all this logic there

JK: It’s understood, we could remove it.

SH: We saw this already with RTE property that we introduced in the YML file then we realized we also need to introduce in the cprj file.

RK: We will not achieve the simplification in one step.

SH: No, for sure. This is a long term picture. We can start implementing proper lock file support.

RK: You say the lockfile should report when there is difference to the previous runs. What would be the behaviour when a difference is detected?

SH: By default if you have any incompatibility, you have an additional optional flag --frozen which would give you an error when running the tool.

GH: Something I was thinking about whether or not we should separate the project specific elements from the actual package delivery. We can store it separately, maybe even in a different file, there was a project agnostic lock file dealing with components and packs.

The reason I was thinking about this, it would allow more people to work with the CMSIS ecosystem. If I didn’t want to use the csolution project format, they could still use the CMSIS Packs and a lot of the tooling but not necessarily specifying the project in that way. In the past that’s been criticism in the fact CMSIS ties you in using all the set of tools. The flow you’ve put down there is really great, there is knowledge in separating this two concerns out.

RK: The slides need more work in the detail to understand how the format of this lock file could look like when we remove the complexity of the cbuildgen and once we do this work we will discover it may get simpler and it may achieve what you just said but it needs a bit more work.

JK: The other aspect I found interesting you are saying you could remove the lock file but would still be able to create a project, it should still be possible to convert?

SH: In theory yes, the lock file selects specific components and assuming you have fixed all dependencies the lock file will contain components that actually work together. If you remove the lock file and the YML does not restrict the pack versions, you might select newer packs than you had before and you may end up with component dependency issues. In theory hopefully it works.

JK: I don’t want to depend on lock file making decisions that should be done in the YML file.

RK: The lock file is a relatively simple thing. When you create a lock file in the frozen mode you could make a backup and compare, if there is a mismatch you have some hiccups. Therefore the way the lock file is looped back here is a bit misleading.

SH: Not exactly, it depends on how many assumptions we will change in the tooling. Today a component identifier in YML need to be unique, if you include the exact match in the lock file we could relax that assumption.

RK: I think we need to work out the exact logic.

SH: In my view the csolution should start with the YML file to figure out components and pack list, then csolution can translate the pack lists to the versions decided by the lock file, then if you resolve components as usual, you record the answer and then you will in the lock file if you have a different component or not.

DJ: Why the conditions are part of the component identifier?

SH: Because it is really not unique if you don’t know the conditions, because you have multiple components that look identical.

RK: If you list the pack and the files that belong to the component, you don’t need the conditions.

SH: Technically yes, but I think it could simplify for the pack developer to understand which components were selected.

GH: We had the same thing, effectively if we don’t store the condition then we have to go back to the project context to effectively resolve it.

DJ: I am a bit confused about we using the term identifier, we had several discussions which elements are actually part of the identifier, I think it might introduce a bit of confusing.

SH: My point of view the identifier is not really an identifier because it is not unique because it does not include the pack and the condition.

ED: We do use the condition ID in C++ implementation to uniquely identify a component, it could be reused. It’s called full ID in contrast to the Cgroup, Csub, Cclass attributes.

JK: Great discussion, I suggest you continue discussing within issues, we can also set a separate meeting, now I would like to progress to the next point on the agenda.

SH: I shouldn’t go too long on the other issue, it’s about the things we are allowed to put in the project tag in the csolution.yml, for example to allow CMakeLists.

RK: That’s a good idea, we need to understand the implications.

JK: I was afraid of putting some project file format that the tool should ignore. I understand you want to have that as a container. Let’s discuss it in a later point.

Handover to Silvio

SLO: Concept of HW part pack: any device mounted on a board is a part. Open-CMSIS defines the schema for creating DFPs where only the MCUs are considered devices.

Proposal for <parts> element in the HW part pack (see https://github.com/Open-CMSIS-Pack/Open-CMSIS-Pack-Spec/issues/144 )

‘parts’ should be child of ‘package’ in PDSC

Attributes:

Pvendor, Pname, Pclass are mandatory

Pgroup, Psub, Pvariant and Prevision are optional

Child elements: feature, book, environment

Proposal for <mountedPart> element in the HW board pack

Open-CMSIS allows the <mountedDevice> element, similarly ST proposes <mountedPart>

RK: One aspect is not fully clear, are you planning to assign SW component against such parts? If this is the case then I believe we need to extend the component resolution, there is a conflict because there is already a Pname attribute, we need to fix the P prefix.

SLO: I am aware of the fact the P is already used. We can take into account this fix.

EF: If it makes sense to extend the specification in this direction, I would like to understand how we can organize to workout corresponding PRs.

RK: The first stage would be to update the specification, the next stage would be to update related checking tools, packchk, schemas and so on and then of course the functionality implementation inside the tools.

JK: To be fully precise we need to go through Decision Committee, other than that I fully agree.

Requests for Review:

JK: We need to look at requirements and “must fix” defects for Version 1.0.0 of csolution/cbuildgen, I will create an issue where people can comment on it. (see: https://github.com/Open-CMSIS-Pack/devtools/issues/395)

I listed a couple of new issues and some existing issues, rather than going through all of them now, I think we are seeing some discussions on cpackget, on misc settings, on compiler specific files and definitions.

Lots of things being discussed, let’s continue with that.

#107, #394, #134, #371, #356, #355

RK: I am leaning towards to implement #394 and #356, the rest needs review, because I think these are obvious improvements.

JK: I think Eric is not still satisfied with the level of explanation on the csolution element and what role the pack delivered elements of csolution really play, I think we need to better understand that to give you more input on that.

RK: I posted a comment today to this issue that clarifies it.

Wrap Up:

JK: Reminder! no meetings on 26.07, 02.08, 09.08 and 16.08 (Summer Vacation).

Next meeting on Tuesday next week. Joachim won’t be there but Reinhard, Daniel, Evgueni and Jonatan run the meeting with Bill.

Keep posting and commenting on the issues. Thanks for your collaboration.