Improve software-layer-script workflow - #147
Open
casparvl wants to merge 13 commits into
Open
Conversation
…-layer-scripts, instead of from the deployed stuff in software.eessi.io
casparvl
pushed a commit
to casparvl/software-layer-scripts
that referenced
this pull request
Jan 13, 2026
bedroge
reviewed
Feb 2, 2026
bedroge
left a comment
bedroge
left a comment
Contributor
There was a problem hiding this comment.
Just to summarize how this used to work and how it will work after merging this PR:
Previously, an updated hooks file was copied to the CVMFS repo (with the overlay) before the builds were started (by EESSI-install-software.sh), and this was only done if a PR diff file showed that the hooks file was modified by the PR.
In the new situation it will just use the version from the git clone of software-layer-scripts; the build script clones a specific commit of that repo. Additional CI will make sure that software-layer PR tarballs can only be deployed if that commit belongs to a merged PR.
One thing that maybe requires some attention is possible situations where a software layer PR modifies that software-layer-scripts commit, does the builds, and then undoes those changes regarding the used commit. A reviewer may not immediately spot that the builds were done with a different commit (the "Files changed" tab won't show it, you would have to look at individual commits or notice that there was a commit after the builds were done), but maybe the CI or bot itself could also catch that?
casparvl
commented
Aug 10, 2026
added 3 commits
August 10, 2026 15:42
…hooks being used during builds to the eb_hooks.py from the software-layer-scripts clone
… to e.g. install_cuda_and_libraries.sh. Doing it via EESSI-extend is more robust
casparvl
marked this pull request as ready for review
August 10, 2026 14:06
Contributor
Author
|
Let's prove in the build logs that we actually use the |
Contributor
Author
|
bot: build repo:eessi.io-2025.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2 |
|
New job on instance
|
Contributor
Author
|
Works like a charm: |
…in the GH repo, before we actually deploy these to CVMFS
Contributor
Author
|
CI check looks fine now: The hooks override works, the check works. Only thing left is to actually deploy this new version - which is why the CI is still failing at this time. |
added 4 commits
August 11, 2026 12:47
…loading of EESSI-extend only on the first instance, that's enough.
Contributor
|
bot: build repo:eessi.io-2023.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2 |
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
Contributor
|
bot: build repo:eessi.io-2026.06-software instance:eessi-bot-mc-aws for:arch=x86_64/amd/zen2 |
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
|
New job on instance
|
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes needed to make builds use everything from the cloned software-layer-scripts, instead of from the deployed stuff in software.eessi.io.
This is a starting point for https://gitlab.com/eessi/support/-/issues/217.
Split off from 2422804 which was tested in e.g. EESSI/software-layer#1351 (comment)
We may not want to merge this until we have sufficient checks (CI, bot) in place to ensure that tarballs can only be deployed if they were build from a merge commit of
software-layer-scripts. That means we need:software-layer-scriptscommit was merged Clone specific software-layer-commit and implement CI to check merged status software-layer#1353bot/commit_sha(i.e. the same one as the CI check uses).Edit 10-08-2026: actually, this is somewhat separate from the
software-layerwork to use a particular commit checksum. Making the build scripts use everything from thesoftware-layer-scriptsrepository makes sense in any case: it prevents us from first having to deploy certain components before we're actually able to test them. The same applies to the modification in the order in which CI steps are executed, that's also to prevent us from first having to deployEESSI-extend-easybuild.ebchanges to production before all the CI steps are run on those changes.