falcon qdel termination issue

[jasmynp]

This is for a falcon/0.4.2 run. We are in the process of validating the latest 0.7.x release for our pipelines, but I was wondering if you've seen this error before? It seems to be happening sporadically for our runs but when I resubmit the same job, it usually runs ok. Any ideas? I've attached a typical falcon.log when this happens.
falcon.log.txt

[jasmynp]

Upon further investigation, it seems this keyboard interrupt is the way falcon kills jobs when it encounters an error.

[pb-cdunn]

[WARNING]'/global/projectb/scratch/qc_user/rqc/prod/pipelines/falcon/in-progress/00/00/20/96/falcon/0-rawreads/preads/c_00004_done' is missing. job: 'c_00004.sh-ct_00004-ct_00004' failed!
[INFO]Failure ('fail'). Joining 'task://localhost/ct_00004'...

I think that job failed. To learn why, look in 0-rawreads/preads/pwatcher.dir/stderr.

I guess we haven't separated the preads jobs into their own sub-dirs yet, so the stderrs might clobber each other. I'll fix that next week. (Traveling.) But with any luck, the failed job stderr is still there. Otherwise, you can try re-running it manually and maybe the same error will occur.

If you ran fc_run0 instead of the default, then you will have to look in sge_logs/ instead.