Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

htcondor: migrate to myschedd #260

Merged
merged 1 commit into from
Jul 9, 2020
Merged

htcondor: migrate to myschedd #260

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AUTHORS.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ Authors

The list of contributors in alphabetical order:

- `Adelina Lintuluoto <https://orcid.org/0000-0002-0726-1452>`_
- `Anton Khodak <https://orcid.org/0000-0003-3263-4553>`_
- `Diego Rodriguez <https://orcid.org/0000-0003-0649-2002>`_
- `Dinos Kousidis <https://orcid.org/0000-0002-4914-4289>`_
Expand Down
1 change: 1 addition & 0 deletions CHANGES.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ Version master (UNRELEASED)
- Instantiates jobs in the configured runtime namespace.
- Enables running tests locally on MacOS.
- Adds Black formatter support.
- Amends CERN HTCondor compute backend to use the new myschedd connection library.

Version 0.6.1 (2020-05-25)
--------------------------
Expand Down
8 changes: 5 additions & 3 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,8 @@ RUN if echo "$COMPUTE_BACKENDS" | grep -q "htcondorcern"; then \
export DEBIAN_FRONTEND=noninteractive ;\
apt-get -yq install wget alien gnupg2 ;\
wget -O ngbauth-submit.rpm http://linuxsoft.cern.ch/internal/repos/batch7-stable/x86_64/os/Packages/ngbauth-submit-0.23-1.el7.noarch.rpm; \
wget -O cernbatchsubmit.rpm http://linuxsoft.cern.ch/internal/repos/batch7-stable/x86_64/os/Packages/cernbatchsubmit-0.1.0-1.el7.x86_64.rpm; \
yes | alien -i cernbatchsubmit.rpm; \
wget -O myschedd.rpm http://linuxsoft.cern.ch/internal/repos/batch7-stable/x86_64/os/Packages/myschedd-1.5-1.el7.x86_64.rpm; \
yes | alien -i myschedd.rpm; \
yes | alien -i ngbauth-submit.rpm; \
wget -qO - http://research.cs.wisc.edu/htcondor/debian/HTCondor-Release.gpg.key | apt-key add -; \
echo "deb https://research.cs.wisc.edu/htcondor/debian/8.9/buster buster contrib" >>/etc/apt/sources.list; \
Expand All @@ -42,7 +42,9 @@ RUN if echo "$COMPUTE_BACKENDS" | grep -q "slurmcern"; then \
--no-install-recommends; \
fi

ADD etc/cernsubmit.yaml /etc/condor/
RUN mkdir -p /etc/myschedd
ADD etc/myschedd.yaml /etc/myschedd/

ADD etc/10_cernsubmit.config /etc/condor/config.d/

ADD etc/ngbauth-submit /etc/sysconfig/
Expand Down
9 changes: 7 additions & 2 deletions etc/10_cernsubmit.config
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
CONDOR_HOST = tweetybird04.cern.ch
LOCAL_CONFIG_FILE = /usr/bin/cernbatchsubmit|
CONDOR_HOST = tweetybird03.cern.ch, tweetybird04.cern.ch

SKIP_LOCAL_CONFIG_FILE=$ENV(SKIP_LOCAL_CONFIG_FILE:FALSE)
if ! $(SKIP_LOCAL_CONFIG_FILE)
LOCAL_CONFIG_FILE = /usr/bin/myschedd.sh |
endif

SEC_CLIENT_AUTHENTICATION_METHODS = KERBEROS
SEC_CREDENTIAL_PRODUCER = /usr/bin/batch_krb5_credential
FILESYSTEM_DOMAIN = cern.ch
Expand Down
Binary file modified etc/cernroot.crt
View file
Open in desktop
Binary file not shown.
13 changes: 0 additions & 13 deletions etc/cernsubmit.yaml
View file
Open in desktop

This file was deleted.

23 changes: 23 additions & 0 deletions etc/myschedd.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
wtfis:
endpoint: https://wtfis.cern.ch:9008
resolve_url: true
bump:
maxrunningjobs: 8000
pools:
share:
standard:
- bigbird08.cern.ch
- bigbird09.cern.ch
- bigbird10.cern.ch
- bigbird11.cern.ch
- bigbird12.cern.ch
- bigbird13.cern.ch
- bigbird14.cern.ch
- bigbird15.cern.ch
- bigbird16.cern.ch
- bigbird17.cern.ch
- bigbird18.cern.ch
- bigbird19.cern.ch
tzero:
standard:
- tbird02.cern.ch
7 changes: 5 additions & 2 deletions reana_job_controller/htcondorcern_job_manager.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
from shutil import copyfile

import classad
import htcondor
from flask import current_app
from reana_db.database import Session
from reana_db.models import Workflow
Expand Down Expand Up @@ -90,11 +89,15 @@ def __init__(
self.workflow = self._get_workflow()
self.unpacked_img = unpacked_img

# We need to import the htcondor package later during runtime after the Kerberos environment is fully initialised.
# Without a valid Kerberos ticket, importing will exit with "ERROR: Unauthorized 401 - do you have authentication tokens? Error "/usr/bin/myschedd.sh |"
initialize_krb5_token(workflow_uuid=self.workflow_uuid)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Can you please add yourself to the AUTHORS.rst file?

  • Can you add a note to the CHANGES.rst file about this change? For example: "Amends CERN HTCondor compute backend to use the new myschedd connection library"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

globals()["htcondor"] = __import__("htcondor")

@JobManager.execution_hook
def execute(self):
"""Execute / submit a job with HTCondor."""
os.chdir(self.workflow_workspace)
initialize_krb5_token(workflow_uuid=self.workflow_uuid)
job_ad = classad.ClassAd()
job_ad["JobDescription"] = (
self.workflow.get_full_workflow_name() + "_" + self.job_name
Expand Down