Skip to content

Releases: Barski-lab/cwl-airflow

Updated API

25 Oct 17:36
974fb9b
Compare
Choose a tag to compare
Updated API Pre-release
Pre-release

Updated API

Update to Airflow 2.1.4

21 Sep 18:55
Compare
Choose a tag to compare

Updated dependencies

  1. airflow 2.1.4
  2. cwltool 3.1.20210816212154

Fixed bugs in docker-compose file

Attached python_3.8.12_cwl_airflow_master_linux.tar.gz is a relocatable python 3.8.12 extracted from the manylinux2014 image with pre-installed CWL-Airflow (ca14232bbb78df242c841047318bcb67c775379f) built in Centos:7 docker container.

Attached python_3.8_cwl_airflow_master_macos.tar.gz is a relocatable python 3.8 from Briefcase Suite with pre-installed CWL-Airflow (ca14232bbb78df242c841047318bcb67c775379f) built in macOS 11.4.

Bug fixes. Statistics report. Initial support of conditionals

29 Jan 02:10
Compare
Choose a tag to compare
  • Proper handling of not delivered progress reports and workflow execution results
  • Initial support of conditional workflow step execution (CWL v1.2)
  • Bug fixes

All relocatable tar.gz are built from dd78057

Supports Airflow 2.0.0

31 Dec 18:40
Compare
Choose a tag to compare
  • Updated to Airflow 2.0.0
  • Fixed bug in triggering DAGs from API
  • Added DAG to optionally resend all undelivered progress reports and results

Note
Due to changes in airflow.cfg in the latest Airflow 2.0.0, when updating CWL-Airflow from the version 1.2.8 the new airflow.cfg will be created. The old one will be backed up. Please make sure that all custom properties of the original airflow.cfg have been properly transferred.

Portable versions are build from this commit

Bug fixes, added docker-compose

17 Dec 15:31
Compare
Choose a tag to compare
  • Added docker-compose with LocalExecutor
  • Switched from zlib to gzip for workflow compressing/decompressing (not breaking changes)
  • Updated packaging script for portable version for Ubuntu (runs LocalExecutor with PostgreSQL)

API updates

04 Dec 15:25
740a0e2
Compare
Choose a tag to compare
  • Added API endpoint to create and trigger DAGs in the same call
  • Added error categories reports
  • Updated macOS packaging script to use Python 3.8 by default
  • Updated clean_dag_run.py to remove outdated DAGs

removes the DAGs for the previous versions of the same workflows if they are not any running DagRuns

  • Updated docker-compose (work in progress)

use latest ubuntu 20.04 and Python 3.8
add cwl_inputs_folder volume mapping to the docker-compose
start services in the order that guaranties all databases and tables are available

All tar.gz are built from the master branch (https://github.com/Barski-lab/cwl-airflow/tree/0a80928daf55ba306805ff2f3e8eaad5e6c7397c)

Bug fixes

10 Nov 12:31
Compare
Choose a tag to compare

Fixed bugs:

  • broken dependencies (add --constraint parameter for installation)
  • workflow fails to execute if section in of the workflow step is empty
  • workflow fails to execute if workflow input has loadListing section
  • API fails to trigger multiple DAG-runs when POSTing multiple requests a once
  • init command fails to patch Airflow config file if the path to it includes spaces

Other changes:

  • refactored progress reports posting
  • added crypto installation dependency

All tar.gz are built from master (https://github.com/Barski-lab/cwl-airflow/tree/a2b9512cb3480ab005a63efc23c0421c9537915e)

Bug fixes

16 Oct 01:22
Compare
Choose a tag to compare

Updates:

  • uses apache-airflow==1.10.12 & cwltool==3.0.20200710214758
  • outputs CWL related logs into separate files
  • supports simple scatter
  • supports literal file inputs

Tested on cwl-v1.1 conformance test from Oct 7, 2020

Reference: SDDA Sprint 1

Useful features update + portable versions for Ubuntu and macOS

30 Jul 20:26
Compare
Choose a tag to compare
  • Add extras_require into setup.py for mysql, statsd, celery dependencies
  • Support workflow parameter as zlib compressed base64 encoded string
  • Update api and test to support embedded in DAG zlib compressed workflows
  • Add --upgrade argument to init command to update old formatted dags
  • Update on_kill function of CWLStepOperator to stop running docker container

Refactored CWL-Airflow (latest Airflow 1.10.11)

15 Jul 00:45
Compare
Choose a tag to compare
  1. Faster CWL parsing and validation
  2. Simplified syntax for new DAG creation
#!/usr/bin/env python3
from cwl_airflow.extensions.cwldag import CWLDAG
dag = CWLDAG(workflow="my_workflow.cwl", dag_id="my_dag")