Conversation
…at it now creates files in an actual easystack format and that it only does one write per file instead of one write per easyconfig
eessi_software_reproduce_stack.py
Outdated
| easyblock_path = os.path.join(software_version_dir, "easybuild", "reprod", "easyblocks", "*.py") | ||
| easyconfig_path = os.path.join(software_version_dir, "easybuild", f"{software_name}-{software_version}.eb") |
There was a problem hiding this comment.
These paths are not correct, they're missing the timestamped subdir. So I guess we can use datestamp_dir_last_build, but we may have to move it out of the else clause to make sure it's defined.
There was a problem hiding this comment.
Maybe we can just move datestamp_dir_last_build up, where datestamp_dir_first_build is also initialized?
There was a problem hiding this comment.
And then we can probably just use:
| easyblock_path = os.path.join(software_version_dir, "easybuild", "reprod", "easyblocks", "*.py") | |
| easyconfig_path = os.path.join(software_version_dir, "easybuild", f"{software_name}-{software_version}.eb") | |
| easyblock_path = os.path.join(datestamp_dir_last_build, "easybuild", "reprod", "easyblocks", "*.py") | |
| easyconfig_path = os.path.join(datestamp_dir_last_build, "easybuild", f"{software_name}-{software_version}.eb") |
…dir from the reproduction dir
…systacks produced matches the original build order
…es that the easyconfig and easyblocks are used as they are specified in the easystack files - which is typically not the case if something is build as a dependency for something else
|
Just to leave it as a note here as well: in EESSI/software-layer#1412 (comment) we found that poetry was rebuilt with an additional dependency, and due to the setup of our reprod script this is not being handled well. We retain the original build order, but always build with the last used easyconfig/easyblock/EB version. In this particular case it means poetry couldn't be built, because that additional dependency ( It would be really tricky to automatically get that right. As an alternative we could simply process rebuilds afterwards, in the same way as it was originally done. That does involve more work though, which is the main reason why we had decided to process rebuilds immediately (i.e. build with the latest version). We could reconsider that, though this probably doesn't happen very often, and it can be easily fixed manually by moving some stuff around in the generated easystacks. |
2 participants
This script is the result of joint development with @bedroge on implementing what was discussed in https://gitlab.com/eessi/support/-/issues/233
In summary: it generates easystack files that will allow you to replicate the software stack installed for a reference architecture. We will use this in practice to deploy software for
zen5, which will also serve as a test for these scripts.