Skip to content

SRE-3205 ci: update for Fedora 42 #87

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
grom72 wants to merge 28 commits into
base: master
Choose a base branch
from
+261 −33
Open

SRE-3205 ci: update for Fedora 42 #87

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
ed02ef8
SRE-3205 ci: update for Fedora 42
grom72 Jul 17, 2025
e0e779f
Add Leap 15 build on Fedora latest
grom72 Jul 18, 2025
a14108d
Add EL8 build on Fedora 41
grom72 Jul 21, 2025
e74d276
Update to leap 15.6
grom72 Jul 21, 2025
5540c3a
Add leap 15.5 build explicitly
grom72 Jul 21, 2025
9f5966a
Use Fedora "latest" explicitly
grom72 Jul 22, 2025
cc7570b
Remove support for all EOL Fedoora versions
grom72 Jul 22, 2025
362ae05
Disable Leap 15.5 until SRE-3219 is fixed
grom72 Jul 22, 2025
a63357b
Fix archive removal
grom72 Jul 22, 2025
61a087c
Revert "Remove support for all EOL Fedoora versions"
grom72 Jul 24, 2025
6296cf4
Use Fedora 38 for Leap 15.5 build
grom72 Jul 24, 2025
cd93a33
Detect if archive repo is needed for older Fedora
grom72 Jul 24, 2025
050e014
Revert "Detect if archive repo is needed for older Fedora"
grom72 Jul 24, 2025
56e8740
Documentation update
grom72 Jul 24, 2025
07d08fe
Automate Fedora EOL detection
grom72 Jul 24, 2025
bf92be5
Missing jq in the base Fedora image
grom72 Jul 24, 2025
abf6b88
Missing jq in the base Fedora image - fix
grom72 Jul 24, 2025
39ba6a6
Use local artifactory as eol proxy
grom72 Jul 24, 2025
292f1e7
Use local artifactory as eol proxy - fix 1
grom72 Jul 24, 2025
12b11f4
Use local artifactory as eol proxy - fix 2
grom72 Jul 24, 2025
6ba2414
Use local artifactory as eol proxy - fix 3
grom72 Jul 24, 2025
bc3c2d9
Use local artifactory as eol proxy - fix 4
grom72 Jul 24, 2025
92b5666
jq is no longer required
grom72 Jul 24, 2025
6499193
jq is no longer required - fix 1
grom72 Jul 24, 2025
071ce35
Merge remote-tracking branch 'origin/master' into sre-3205
grom72 Apr 23, 2026
a4ad899
SRE-3223 ci: Use DAOS_HTTPS_PROXY and DAOS_NO_PROXY
grom72 Apr 23, 2026
8b0f5ad
Merge branch 'sre-3223-2nd' into sre-3205
grom72 Apr 23, 2026
f06958d
Must have +x as it is directly used from Docker
grom72 Apr 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
58 changes: 32 additions & 26 deletions Dockerfile.mockbuild
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
#
# Copyright 2018-2024 Intel Corporation
# Copyright 2025 Hewlett Packard Enterprise Development LP
# Copyright 2025-2026 Hewlett Packard Enterprise Development LP
#
# 'recipe' for Docker to build an RPM
#
Expand All @@ -12,37 +12,43 @@ FROM fedora:$FVERSION
ARG FVERSION
LABEL maintainer="[email protected]"

# Use local repo server if present
ARG REPO_FILE_URL
ARG DAOS_LAB_CA_FILE_URL
ARG REPOSITORY_NAME

# Accept DAOS_NO_PROXY at build time
# Accept DAOS_HTTP_PROXY, DAOS_HTTPS_PROXY, and DAOS_NO_PROXY at build time
ARG DAOS_NO_PROXY
# Propagate into the build environment
ENV no_proxy=${DAOS_NO_PROXY}
ENV NO_PROXY=${DAOS_NO_PROXY}
# Persist into /etc/environment for use by shells and services
RUN echo "no_proxy=${DAOS_NO_PROXY}" >> /etc/environment && \
echo "NO_PROXY=${DAOS_NO_PROXY}" >> /etc/environment

# Accept DAOS_HTTPS_PROXY at build time
ARG DAOS_HTTP_PROXY
ARG DAOS_HTTPS_PROXY

# Propagate into the build environment
ENV https_proxy=${DAOS_HTTPS_PROXY}
ENV HTTPS_PROXY=${DAOS_HTTPS_PROXY}
ENV http_proxy=${DAOS_HTTP_PROXY} \
HTTP_PROXY=${DAOS_HTTP_PROXY} \
https_proxy=${DAOS_HTTPS_PROXY} \
HTTPS_PROXY=${DAOS_HTTPS_PROXY} \
no_proxy=${DAOS_NO_PROXY} \
NO_PROXY=${DAOS_NO_PROXY}

# Persist into /etc/environment for use by shells and services
RUN echo "https_proxy=${DAOS_HTTPS_PROXY}" >> /etc/environment && \
echo "HTTPS_PROXY=${DAOS_HTTPS_PROXY}" >> /etc/environment
RUN set -e; \
if [ -n "$DAOS_HTTP_PROXY" ]; then \
echo "http_proxy=$DAOS_HTTP_PROXY" >> /etc/environment; \
echo "HTTP_PROXY=$DAOS_HTTP_PROXY" >> /etc/environment; \
fi; \
if [ -n "$DAOS_HTTPS_PROXY" ]; then \
echo "https_proxy=$DAOS_HTTPS_PROXY" >> /etc/environment; \
echo "HTTPS_PROXY=$DAOS_HTTPS_PROXY" >> /etc/environment; \
fi; \
if [ -n "$DAOS_NO_PROXY" ]; then \
echo "no_proxy=$DAOS_NO_PROXY" >> /etc/environment; \
echo "NO_PROXY=$DAOS_NO_PROXY" >> /etc/environment; \
fi

# script to install OS updates basic tools and daos dependencies
# COPY ./utils/scripts/install-fedora.sh /tmp/install.sh
# script to setup local repo if available
COPY ./packaging/scripts/repo-helper-fedora.sh /tmp/repo-helper.sh
# Use local repo server if present
ARG REPO_FILE_URL
ARG DAOS_LAB_CA_FILE_URL
ARG REPOSITORY_NAME

RUN chmod +x /tmp/repo-helper.sh && \
/tmp/repo-helper.sh && \
rm -f /tmp/repo-helper.sh
# Script to setup a local repo instead of the distro-provided one.
# Only takes effect when REPO_FILE_URL is set; otherwise it is a no-op.
RUN --mount=type=bind,source=packaging/scripts/repo-helper-fedora.sh,target=/tmp/repo-helper-fedora.sh \
/tmp/repo-helper-fedora.sh

# Install basic tools
RUN dnf -y install mock make rpm-build createrepo rpmlint git \
Expand Down
203 changes: 200 additions & 3 deletions Jenkinsfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,7 +71,7 @@ pipeline {
' --cap-add=SYS_ADMIN' +
' --privileged=true' +
' -v /scratch:/scratch'
additionalBuildArgs dockerBuildArgs() + '--build-arg PACKAGINGDIR=. '
additionalBuildArgs dockerBuildArgs() + '--build-arg FVERSION=latest --build-arg PACKAGINGDIR=. '
}
}
steps {
Expand Down Expand Up @@ -110,6 +110,55 @@ pipeline {
}
}
} //stage('Build libfabric on EL 8')
stage('Build libfabric on EL 8 (Fedora 41)') {
agent {
dockerfile {
filename 'Dockerfile.mockbuild'
label 'docker_runner'
args '--group-add mock' +
' --cap-add=SYS_ADMIN' +
' --privileged=true' +
' -v /scratch:/scratch'
additionalBuildArgs dockerBuildArgs() +
'--build-arg FVERSION=41 --build-arg PACKAGINGDIR=. '
}
}
steps {
checkoutScm url: 'https://github.com/daos-stack/libfabric.git',
checkoutDir: "libfabric",
branch: commitPragma(pragma: 'libfabric-branch', def_val: 'master')
sh label: env.STAGE_NAME,
script: updatePackaging('libfabric') + '''
rm -rf artifacts/el8/
mkdir -p artifacts/el8/
make CHROOT_NAME="rocky+epel-8-x86_64" DISTRO_VERSION=8 chrootbuild'''
}
post {
success {
sh 'ls -l /var/lib/mock/rocky+epel-8-x86_64/result/'
}
unsuccessful {
sh label: "Collect artifacts",
script: '''mockroot=/var/lib/mock/rocky+epel-8-x86_64
artdir=$PWD/libfabric/artifacts/el8
cp -af _topdir/SRPMS $artdir
(cd $mockroot/result/ &&
cp -r . $artdir)
(if cd $mockroot/root/builddir/build/BUILD/*/; then
find . -name configure -printf %h\\\\n | \
while read dir; do
if [ ! -f $dir/config.log ]; then
continue
fi
tdir="$artdir/autoconf-logs/$dir"
mkdir -p $tdir
cp -a $dir/config.log $tdir/
done
fi)'''
archiveArtifacts artifacts: 'libfabric/artifacts/el8/**'
}
}
} //stage('Build libfabric on EL 8 (Fedora 41)')
stage('Build libfabric on EL 9') {
agent {
dockerfile {
Expand Down Expand Up @@ -158,6 +207,55 @@ pipeline {
}
}
} //stage('Build libfabric on EL 9')
stage('Build libfabric on EL 9 (Fedora 42)') {
agent {
dockerfile {
filename 'Dockerfile.mockbuild'
label 'docker_runner'
args '--group-add mock' +
' --cap-add=SYS_ADMIN' +
' --privileged=true' +
' -v /scratch:/scratch'
additionalBuildArgs dockerBuildArgs() +
'--build-arg FVERSION=42 --build-arg PACKAGINGDIR=. '
}
}
steps {
checkoutScm url: 'https://github.com/daos-stack/libfabric.git',
checkoutDir: "libfabric",
branch: commitPragma(pragma: 'libfabric-branch', def_val: 'master')
sh label: env.STAGE_NAME,
script: updatePackaging('libfabric') + '''
rm -rf artifacts/el9/
mkdir -p artifacts/el9/
make CHROOT_NAME="rocky+epel-9-x86_64" DISTRO_VERSION=9 chrootbuild'''
}
post {
success {
sh 'ls -l /var/lib/mock/rocky+epel-9-x86_64/result/'
}
unsuccessful {
sh label: "Collect artifacts",
script: '''mockroot=/var/lib/mock/rocky+epel-9-x86_64
artdir=$PWD/libfabric/artifacts/el9
cp -af _topdir/SRPMS $artdir
(cd $mockroot/result/ &&
cp -r . $artdir)
(if cd $mockroot/root/builddir/build/BUILD/*/; then
find . -name configure -printf %h\\\\n | \
while read dir; do
if [ ! -f $dir/config.log ]; then
continue
fi
tdir="$artdir/autoconf-logs/$dir"
mkdir -p $tdir
cp -a $dir/config.log $tdir/
done
fi)'''
archiveArtifacts artifacts: 'libfabric/artifacts/el9/**'
}
}
} //stage('Build libfabric on EL 9 (Fedora 42)')
stage('Build mercury on EL 8') {
agent {
dockerfile {
Expand Down Expand Up @@ -216,7 +314,106 @@ pipeline {
' --privileged=true' +
' -v /scratch:/scratch'
additionalBuildArgs dockerBuildArgs() +
'--build-arg FVERSION=37 --build-arg PACKAGINGDIR=. '
'--build-arg PACKAGINGDIR=. '
}
}
steps {
checkoutScm url: 'https://github.com/daos-stack/libfabric.git',
checkoutDir: "libfabric",
branch: commitPragma(pragma: 'libfabric-branch', def_val: 'master')
sh label: env.STAGE_NAME,
script: updatePackaging('libfabric') + '''
rm -rf artifacts/leap15/
mkdir -p artifacts/leap15/
make CHROOT_NAME="opensuse-leap-15.6-x86_64" chrootbuild'''
}
post {
success {
sh 'ls -l /var/lib/mock/opensuse-leap-15.6-x86_64/result/'
}
unsuccessful {
sh label: "Collect artifacts",
script: '''mockroot=/var/lib/mock/opensuse-leap-15.6-x86_64
artdir=$PWD/libfabric/artifacts/leap15
cp -af _topdir/SRPMS $artdir
(cd $mockroot/result/ &&
cp -r . $artdir)
(if cd $mockroot/root/builddir/build/BUILD/*/; then
find . -name configure -printf %h\\\\n | \
while read dir; do
if [ ! -f $dir/config.log ]; then
continue
fi
tdir="$artdir/autoconf-logs/$dir"
mkdir -p $tdir
cp -a $dir/config.log $tdir/
done
fi)'''
archiveArtifacts artifacts: 'libfabric/artifacts/leap15/**'
}
}
} //stage('Build libfabric on Leap 15')
stage('Build libfabric on Leap 15 (Fedora 41)') {
agent {
dockerfile {
filename 'Dockerfile.mockbuild'
label 'docker_runner'
args '--group-add mock' +
' --cap-add=SYS_ADMIN' +
' --privileged=true' +
' -v /scratch:/scratch'
additionalBuildArgs dockerBuildArgs() +
'--build-arg FVERSION=41 --build-arg PACKAGINGDIR=. '
}
}
steps {
checkoutScm url: 'https://github.com/daos-stack/libfabric.git',
checkoutDir: "libfabric",
branch: commitPragma(pragma: 'libfabric-branch', def_val: 'master')
sh label: env.STAGE_NAME,
script: updatePackaging('libfabric') + '''
rm -rf artifacts/leap15/
mkdir -p artifacts/leap15/
make CHROOT_NAME="opensuse-leap-15.6-x86_64" chrootbuild'''
}
post {
success {
sh 'ls -l /var/lib/mock/opensuse-leap-15.6-x86_64/result/'
}
unsuccessful {
sh label: "Collect artifacts",
script: '''mockroot=/var/lib/mock/opensuse-leap-15.6-x86_64
artdir=$PWD/libfabric/artifacts/leap15
cp -af _topdir/SRPMS $artdir
(cd $mockroot/result/ &&
cp -r . $artdir)
(if cd $mockroot/root/builddir/build/BUILD/*/; then
find . -name configure -printf %h\\\\n | \
while read dir; do
if [ ! -f $dir/config.log ]; then
continue
fi
tdir="$artdir/autoconf-logs/$dir"
mkdir -p $tdir
cp -a $dir/config.log $tdir/
done
fi)'''
archiveArtifacts artifacts: 'libfabric/artifacts/leap15/**'
}
}
} //stage('Build libfabric on Leap 15 (Fedora 41)')
// Leap 15.5 requires Fedora 38 or older
stage('Build libfabric on Leap 15.5 (Fedora 38)') {
agent {
dockerfile {
filename 'Dockerfile.mockbuild'
label 'docker_runner'
args '--group-add mock' +
' --cap-add=SYS_ADMIN' +
' --privileged=true' +
' -v /scratch:/scratch'
additionalBuildArgs dockerBuildArgs() +
'--build-arg FVERSION=38 --build-arg PACKAGINGDIR=. '
}
}
steps {
Expand Down Expand Up @@ -254,7 +451,7 @@ pipeline {
archiveArtifacts artifacts: 'libfabric/artifacts/leap15/**'
}
}
} //stage('Build libfabric on Leap 15')
} //stage('Build libfabric on Leap 15.5')
stage('Build libfabric on Ubuntu 20.04') {
agent {
dockerfile {
Expand Down
33 changes: 29 additions & 4 deletions scripts/repo-helper-fedora.sh
100644 → 100755
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
#!/bin/bash
#
# Copyright 2025-2026 Hewlett Packard Enterprise Development LP
#
# SPDX-License-Identifier: BSD-2-Clause-Patent
#
set -uex

# This script is used by Dockerfiles to optionally use
Expand All @@ -9,9 +14,26 @@ set -uex
: "${FVERSION:=latest}"
: "${REPOSITORY_NAME:=artifactory}"
: "${archive:=}"
if [ "$FVERSION" != "latest" ]; then
archive="-archive"
Copy link
Copy Markdown

@ryon-jensen ryon-jensen Jul 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what happens when 42 gets moved to archive?

Copy link
Copy Markdown
Contributor Author

@grom72 grom72 Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Most probably we will modify this script sooner when adding support for Fedora 43, 44.

But to be honest we should remove support for any archive version of Fedora. Archive means EOL.

Copy link
Copy Markdown
Contributor Author

@grom72 grom72 Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-archive option has been removed in cc7570b

Copy link
Copy Markdown
Contributor

@JohnMalmberg JohnMalmberg Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding the archive option allows us to use older than latest in case we have to pin something to an archive release.

Fedora was used because that is the version of mock that has is kept up to date with the distros. Unfortunately fedora has been dropping support for archived EL-x and EOS leap-15.x.

At one point, only fedora/38 could build leap 15.5 rpms, and we needed to go to fedora/41 to build leap 15.6 rpms, and fedora/41 dropped building the leap 15.4 rpms.

The version of the image used in the common docker file can vary for the target of el-8, el-9, leap-15, and near future el-10.

We have fedora-latest pinned to fedora/41 currently, and I would like to remove that pin or move that pin to fedora/42.

Copy link
Copy Markdown
Contributor Author

@grom72 grom72 Jul 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I do not see a point to build anything in scope of EOL OSes.
Based on https://en.opensuse.org/Lifetime Leap 15.5 is already EOL for more than half of year.

Copy link
Copy Markdown
Contributor

@JohnMalmberg JohnMalmberg Jul 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unfortunately our paying customer has not yet migrated to SLES 15 SP6, so needs leap 15.5 built RPMs.
We run into that in a lot of cases where we can not move to the most recent version of a distro, which is one of the reasons we are building and testing on el-8.8 instead of el-8.10 and el-8.8 has been out of support for longer than leap 15.5.

Copy link
Copy Markdown
Contributor Author

@grom72 grom72 Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fedora EOL is now calculated based on date retrieved from https://endoflife.date/api/fedora.json (via artifactory)

fi

is_fedora_eol() {
local eol_url fedora_version eol_date today
if [ -n "$REPO_FILE_URL" ]; then
eol_url="${REPO_FILE_URL%repo-files/}eol-proxy/fedora.json"
fedora_version=$(grep VERSION_ID /etc/os-release | cut -d= -f2 | \
tr -d '"')
eol_date=$(curl -s "$eol_url" | sed 's/},{/}\n{/g' | \
grep "cycle\":\"$fedora_version\"" | \
sed -n 's/.*"eol":"\([^"]*\)".*/\1/p')
if [[ -z "$eol_date" ]]; then
return 1 # Assume NOT EOL if data missing
fi
today=$(date +%Y-%m-%d)
[[ "$today" > "$eol_date" ]]
return $? # Return 0 if EOL, 1 if not
else
return 1 # Assume NOT EOL if url is missing
fi
}

# shellcheck disable=SC2120
disable_repos () {
Expand Down Expand Up @@ -58,11 +80,14 @@ install_optional_ca() {
if [ -n "$REPO_FILE_URL" ]; then
install_curl
install_optional_ca
if is_fedora_eol; then
archive="-archive"
fi
mkdir -p /etc/yum.repos.d
pushd /etc/yum.repos.d/
curl -k -sSf \
-o "daos_ci-fedora${archive}-${REPOSITORY_NAME}.repo" \
"{$REPO_FILE_URL}daos_ci-fedora${archive}-${REPOSITORY_NAME}.repo"
"${REPO_FILE_URL}daos_ci-fedora${archive}-${REPOSITORY_NAME}.repo"
disable_repos /etc/yum.repos.d/
popd
fi
Expand Down