Skip to content

DM-53958: changes for matching on Nodeset #28

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
daues merged 3 commits into from
Feb 6, 2026
Merged

DM-53958: changes for matching on Nodeset #28

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c477180
DM-53958: changes for matching on Nodeset
daues Feb 3, 2026
b8a685b
DM-53958: remove unneeded symbolic links
daues Feb 5, 2026
14af1b6
Revert "DM-53958: remove unneeded symbolic links"
daues Feb 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions config
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
python/lsst/ctrl/platform/s3df/etc/config
19 changes: 15 additions & 4 deletions python/lsst/ctrl/platform/s3df/etc/templates/allocation.sh.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,15 @@ else
export RUBIN_COLLECTOR_HOST="sdfiana012.sdf.slac.stanford.edu"
fi

if [ $NODESET ]; then
echo "NODESET is specified"
echo "NODESET $NODESET"
NODESET_CLAUSE=" && (JobNodeset == \"$NODESET\")"
else
echo "NODESET is empty"
NODESET_CLAUSE=""
fi

export RUBIN_COLLECTOR_PORT=$CPORT
echo "Using RUBIN_COLLECTOR_HOST $RUBIN_COLLECTOR_HOST"
echo "Using RUBIN_COLLECTOR_PORT $RUBIN_COLLECTOR_PORT"
Expand Down Expand Up @@ -78,16 +87,18 @@ export _CONDOR_SBIN=${RELEASE_DIR}/sbin
export _CONDOR_FILETRANSFER_PLUGINS=${RELEASE_DIR}/libexec/curl_plugin

# export _CONDOR_STARTD_RESOURCE_PREFIX=slot_${VERY_RNUM}_
export _CONDOR_STARTD_RESOURCE_PREFIX=slot_${myuser}_${VERY_RNUM}_
# Add an informational slot name so that slots of different nodesets do not look the same
N10=`echo $NODESET | cut -c1-10`
export _CONDOR_STARTD_RESOURCE_PREFIX=${N10}slot_${myuser}_${VERY_RNUM}_

SPAN_MEMORY=1000
LOWER_BOUND=$((SLURM_MEM_PER_NODE - SPAN_MEMORY))
UPPER_BOUND=$((SLURM_MEM_PER_NODE + SPAN_MEMORY))
common_job_name=glide_${myuser}
common_job_name="$NODESETglide_${myuser}"
if [ $SLURM_JOB_NAME == ${common_job_name} ]; then
export _CONDOR_START="(Owner == \"${myuser}\")"
export _CONDOR_START="(Owner == \"${myuser}\")${NODESET_CLAUSE}"
else
export _CONDOR_START="(Owner == \"${myuser}\") && (RequestMemory>${LOWER_BOUND}) && (RequestMemory<${UPPER_BOUND})"
export _CONDOR_START="(Owner == \"${myuser}\")${NODESET_CLAUSE} && (RequestMemory>${LOWER_BOUND}) && (RequestMemory<${UPPER_BOUND})"
fi
echo _CONDOR_START
echo ${_CONDOR_START}
Expand Down
2 changes: 1 addition & 1 deletion python/lsst/ctrl/platform/s3df/etc/templates/generic.slurm.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
#SBATCH -A $ACCOUNT
#SBATCH --ntasks-per-node 1
#SBATCH --cpus-per-task $CPUS
#SBATCH -J glide_$USER_NAME
#SBATCH -J $NODESETglide_$USER_NAME
$QOS
$RESERVATION
$EXCLUSIVE
Expand Down
5 changes: 4 additions & 1 deletion python/lsst/ctrl/platform/s3df/etc/templates/glidein_condor_config.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,12 @@ $DYNAMIC_SLOTS_BLOCK

$PACK_BLOCK


$NODESET_BLOCK

STARTD_NOCLAIM_SHUTDOWN=$GLIDEIN_SHUTDOWN

STARTD_ATTRS = ALLOCATED_NODE_SET
STARTD_ATTRS = $(STARTD_ATTRS) Nodeset

# hold jobs that the startd detects have exceeded assigned memory
MEMORY_EXCEEDED = ((MemoryUsage > Memory) =!= TRUE)
Expand Down
1 change: 1 addition & 0 deletions templates
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
python/lsst/ctrl/platform/s3df/etc/templates
Loading