Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Strider reboot sometimes when canceling a job #1053

Open
xgalen opened this issue Feb 1, 2018 · 1 comment
Open

Strider reboot sometimes when canceling a job #1053

xgalen opened this issue Feb 1, 2018 · 1 comment

Comments

@xgalen
Copy link
Member

xgalen commented Feb 1, 2018

Hi all,

When a job gets stuck [1] (in my case, because the git clone has failed with no reason), the job remains unfinished but Strider keeps working and if you send more jobs they work as expected (or not if fails like this). After that, if you want to cancel the job, Strider reboots [2]. The cancel button works if the job is working properly but not in the case it is "blocked".

I can't getting ahead on this now due to a personal reasons but I let this here in case anyone can helps.

[1] git clone fails

Thu, 01 Feb 2018 11:09:11 GMT strider-runner-core:job Job "5a72f55516a3f3775ac70514" status: command.start { '0': 'command.start',
  '1':
   { command: 'git clone --recursive https://[username]:[password]@github.com/MYREPO.git . --branch master',
     started: 2018-02-01T11:09:11.292Z,
     plugin: 'github',
     time: 2018-02-01T11:09:11.292Z } }
Thu, 01 Feb 2018 11:09:11 GMT strider-git:worker Git Version:2.7
Thu, 01 Feb 2018 11:09:11 GMT strider-runner-core:job Job "5a72f55516a3f3775ac70514" status: stderr { '0': 'stderr', '1': 'Cloning into \'.\'...\n' }
Thu, 01 Feb 2018 11:09:13 GMT strider-runner-core:job Job "5a72f55516a3f3775ac70514" status: command.done { '0': 'command.done',
  '1': { exitCode: 0, time: 2018-02-01T11:09:13.574Z, elapsed: 2282 } }
Thu, 01 Feb 2018 11:09:13 GMT strider-runner-core:job Job "5a72f55516a3f3775ac70514" runs: github { cmd: 'git fetch origin refs/pull/1904/merge',
  cwd: '/home/ubuntu/.strider/data/MYREPO/job-5a72f55516a3f3775ac70514',
  env:
   { MOCHA_COLORS: 1,
     N_PREFIX: '/home/ubuntu/.strider/data/MYREPO/.n',
     PATH: '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/snap/bin',
     HOME: '/home/ubuntu',
     LANG: 'en_US.UTF-8',
     SSH_AUTH_SOCK: undefined,
     PAAS_NAME: 'strider' } }
Thu, 01 Feb 2018 11:09:13 GMT strider-runner-core:job Job "5a72f55516a3f3775ac70514" status: command.start { '0': 'command.start',
  '1':
   { command: 'git fetch origin refs/pull/1904/merge',
     started: 2018-02-01T11:09:13.575Z,
     plugin: 'github',
     time: 2018-02-01T11:09:13.575Z } }
Thu, 01 Feb 2018 11:09:13 GMT strider-runner-core:job Job "5a72f55516a3f3775ac70514" status: stderr { '0': 'stderr',
  '1': 'error: RPC failed; curl 18 transfer closed with outstanding read data remaining\n' }
Thu, 01 Feb 2018 11:09:13 GMT strider-runner-core:job Job "5a72f55516a3f3775ac70514" status: stderr { '0': 'stderr',
  '1': 'fatal: The remote end hung up unexpectedly\n' }
Thu, 01 Feb 2018 11:09:13 GMT strider-runner-core:job Job "5a72f55516a3f3775ac70514" status: command.done { '0': 'command.done',
  '1': { exitCode: 128, time: 2018-02-01T11:09:13.922Z, elapsed: 347 } }
Thu, 01 Feb 2018 11:09:13 GMT strider-runner-core:job Job "5a72f55516a3f3775ac70514" status: phase.errored { '0': 'phase.errored',
  '1': { time: 2018-02-01T11:09:13.922Z, exitCode: 0 } }
Thu, 01 Feb 2018 11:09:13 GMT strider-runner-core:job Job "5a72f55516a3f3775ac70514" errored: Fetch refs/pull/1904/merge failed with code 128 Error: Fetch refs/pull/1904/merge failed with code 128
    at badCode (/home/ubuntu/strider/node_modules/strider-git/worker.js:106:11)
    at /home/ubuntu/strider/node_modules/strider-git/worker.js:196:19
    at ChildProcess.<anonymous> (/home/ubuntu/strider/node_modules/strider-runner-core/lib/job.js:252:9)
    at ChildProcess.emit (events.js:160:13)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:209:12)
Thu, 01 Feb 2018 11:09:13 GMT strider-runner-core:job Job "5a72f55516a3f3775ac70514" status: stderr { '0': 'stderr',
  '1': 'An unexpected error occurred while executing this job. Please report at https://github.com/Strider-CD/strider/issues\n\nFetch refs/pull/1904/merge failed with code 128\n\nError: Fetch refs/pull/1904/merge failed with code 128\n    at badCode (/home/ubuntu/strider/node_modules/strider-git/worker.js:106:11)\n    at /home/ubuntu/strider/node_modules/strider-git/worker.js:196:19\n    at ChildProcess.<anonymous> (/home/ubuntu/strider/node_modules/strider-runner-core/lib/job.js:252:9)\n    at ChildProcess.emit (events.js:160:13)\n    at Process.ChildProcess._handle.onexit (internal/child_process.js:209:12)' }
Thu, 01 Feb 2018 11:09:13 GMT strider-simple-runner [runner:simple-runner] Job done with error. Project: MYPROJECT/MYREPO Job ID: 5a72f55516a3f3775ac70514
2018-02-01T11:09:13.923Z - ^[[31merror^[[39m: (node:30554) UnhandledPromiseRejectionWarning: Error: Command failed: git fetch origin refs/pull/1904/merge
error: RPC failed; curl 18 transfer closed with outstanding read data remaining
fatal: The remote end hung up unexpectedly

    at ChildProcess.exithandler (child_process.js:272:12)
    at ChildProcess.emit (events.js:160:13)
    at maybeClose (internal/child_process.js:943:16)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:220:5)
2018-02-01T11:09:13.923Z - ^[[31merror^[[39m: (node:30554) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 2)

[2] cancel causes reboot

I swear I clicked only once on the cancel button but it seems it sends a lot of requests.

2018-02-01T11:19:34.746Z - ^[[32minfo^[[39m: Got a request to cancel 5a72f55516a3f3775ac70514
Thu, 01 Feb 2018 11:19:34 GMT strider-runner-core:job io cancel 5a72f55516a3f3775ac70514 5a72d0d516a3f3775ac704bc
Thu, 01 Feb 2018 11:19:34 GMT strider-runner-core:job io cancel 5a72f55516a3f3775ac70514 5a72d19916a3f3775ac704c3
Thu, 01 Feb 2018 11:19:34 GMT strider-runner-core:job io cancel 5a72f55516a3f3775ac70514 5a72dadb16a3f3775ac704c7
Thu, 01 Feb 2018 11:19:34 GMT strider-runner-core:job io cancel 5a72f55516a3f3775ac70514 5a72e00d16a3f3775ac704d0
Thu, 01 Feb 2018 11:19:34 GMT strider-runner-core:job io cancel 5a72f55516a3f3775ac70514 5a72e00d16a3f3775ac704d1
Thu, 01 Feb 2018 11:19:34 GMT strider-runner-core:job io cancel 5a72f55516a3f3775ac70514 5a72e45a16a3f3775ac704dd
Thu, 01 Feb 2018 11:19:34 GMT strider-runner-core:job io cancel 5a72f55516a3f3775ac70514 5a72e69f16a3f3775ac704e4
Thu, 01 Feb 2018 11:19:34 GMT strider-runner-core:job io cancel 5a72f55516a3f3775ac70514 5a72ecda16a3f3775ac704eb
Thu, 01 Feb 2018 11:19:34 GMT strider-runner-core:job io cancel 5a72f55516a3f3775ac70514 5a72ed6016a3f3775ac704ec
Thu, 01 Feb 2018 11:19:34 GMT strider-runner-core:job io cancel 5a72f55516a3f3775ac70514 5a72f0d916a3f3775ac704fb
Thu, 01 Feb 2018 11:19:34 GMT strider-runner-core:job io cancel 5a72f55516a3f3775ac70514 5a72f0da16a3f3775ac704fc
Thu, 01 Feb 2018 11:19:34 GMT strider-runner-core:job io cancel 5a72f55516a3f3775ac70514 5a72f2a516a3f3775ac7050b
Thu, 01 Feb 2018 11:19:34 GMT strider-runner-core:job io cancel 5a72f55516a3f3775ac70514 5a72f55516a3f3775ac70514
Thu, 01 Feb 2018 11:19:34 GMT strider-runner-core:job Cancelling job "5a72f55516a3f3775ac70514"...
Thu, 01 Feb 2018 11:19:34 GMT strider-simple-runner:jobdata tried to pop a nonexistant job 5a72f55516a3f3775ac70514
Thu, 01 Feb 2018 11:19:34 GMT strider-simple-runner Job cancelled, but no data! 5a72f55516a3f3775ac70514
Thu, 01 Feb 2018 11:19:34 GMT strider-simple-runner Job cancelled, but no callback found. The queue is probably broken.
events.js:137
      throw er; // Unhandled 'error' event
      ^

TypeError: self.queue.process is not a function
    at EventEmitter.<anonymous> (/home/ubuntu/strider/node_modules/strider-simple-runner/lib/index.js:168:27)
    at EventEmitter.emit (/home/ubuntu/strider/node_modules/eventemitter2/lib/eventemitter2.js:348:22)
    at Job.cancel (/home/ubuntu/strider/node_modules/strider-runner-core/lib/job.js:113:22)
    at EventEmitter.<anonymous> (/home/ubuntu/strider/node_modules/strider-runner-core/lib/job.js:134:14)
    at EventEmitter.emit (/home/ubuntu/strider/node_modules/eventemitter2/lib/eventemitter2.js:348:22)
    at Runner.cancelJob (/home/ubuntu/strider/node_modules/strider-simple-runner/lib/index.js:208:13)
    at EventEmitter.emit (events.js:165:20)
    at /home/ubuntu/strider/lib/utils/user-socket.js:146:28
    at /home/ubuntu/strider/node_modules/mongoose/lib/query.js:3115:18
    at tryCatcher (/home/ubuntu/strider/node_modules/bluebird/js/release/util.js:16:23)
    at Promise._settlePromiseFromHandler (/home/ubuntu/strider/node_modules/bluebird/js/release/promise.js:512:31)
    at Promise._settlePromise (/home/ubuntu/strider/node_modules/bluebird/js/release/promise.js:569:18)
    at Promise._settlePromise0 (/home/ubuntu/strider/node_modules/bluebird/js/release/promise.js:614:10)
    at Promise._settlePromises (/home/ubuntu/strider/node_modules/bluebird/js/release/promise.js:693:18)
    at Async._drainQueue (/home/ubuntu/strider/node_modules/bluebird/js/release/async.js:133:16)
    at Async._drainQueues (/home/ubuntu/strider/node_modules/bluebird/js/release/async.js:143:10)
    at Immediate.Async.drainQueues [as _onImmediate] (/home/ubuntu/strider/node_modules/bluebird/js/release/async.js:17:14)
    at runCallback (timers.js:756:18)
    at tryOnImmediate (timers.js:717:5)
    at processImmediate [as _immediateCallback] (timers.js:697:5)
30554 died 1 null
11046 forked
@knownasilya
Copy link
Member

For 1, https://stackoverflow.com/questions/38618885/error-rpc-failed-curl-transfer-closed-with-outstanding-read-data-remaining looks like an error due to size. Maybe strider can pull shallow and then do another request to unshallow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@knownasilya @xgalen