Add Galaxy JWD Python script #15618
Add Galaxy JWD Python script #15618
Conversation
Functionality: 1. Can get you the path of a JWD: Used when one wants to find and go to the JWD for debugging 2. Can delete JWD's of job failed within last X days: For cleaning up JWDs
There was a problem hiding this comment.
This is a nice start, but I would recommend using Galaxy's config file parsing
Galaxy's database session, Galaxy's object store instance and creating this as part of the galaxy-data package. Take a look at
galaxy/lib/galaxy/celery/__init__.py
Line 86 in 54e9768
| backends = parse_object_store(object_store_conf) | ||
|
|
||
| # Connect to Galaxy database | ||
| db = Database( |
There was a problem hiding this comment.
can you use sqlalchemy please ?
| str: Path to the object_store_conf.xml file | ||
| """ | ||
| object_store_conf = "" | ||
| with open(galaxy_config_file, "r") as config: |
There was a problem hiding this comment.
The config file is a yaml file, please parse it as such.
| object_store_conf = "" | ||
| with open(galaxy_config_file, "r") as config: | ||
| for line in config: | ||
| if line.strip().startswith("object_store_config_file"): |
There was a problem hiding this comment.
The object store file doesn't have to be an xml file, there are different ways to parse them.
| if metadata[0] not in backends_dict.keys(): | ||
| raise ValueError(f"Object store id '{metadata[0]}' does not exist in the object_store_conf.xml file") | ||
|
|
||
| jwd_path = f"{backends_dict[metadata[0]]}/0{job_id[0:2]}/{job_id[2:5]}/{job_id}" |
There was a problem hiding this comment.
That's very brittle, I think you just want to load up the object store itself.
See the inital PR: galaxyproject#15618
|
Closing this as the revised updated one part of celery task is here |
See the inital PR: galaxyproject#15618
WHAT:
Adds a new script with the following functionality:
Xdays: For cleaning up JWDsHow to test the changes?
(Select all options that apply)
gxadmin) are needed1.1 GALAXY_CONFIG_FILE: Path to the galaxy.yml file
1.2 GALAXY_LOG_DIR: Path to the Galaxy log directory
1.3 PGDATABASE: Name of the Galaxy database
1.4 PGUSER: Galaxy database user
1.5 PGHOST: Galaxy database host
~/.pgpassfile (same asgxadmin) with the following format1.1
<pg_host>:5432:*:<pg_user>:<pg_password>1.1
psycopg21.2
xml.dom.minidom1.1
python galaxy_jwd.py get_jwd 123456781.2
python galaxy_jwd.py clean_jwds --dry_run True --days 30orpython galaxy_jwd.py clean_jwds --dry_run False --days 30License