Add native job support for neovim.

[okkays]

Hello!

I think I've managed to get support for neovim's jobcontrol (#125) working (I've tested it manually with vim-coverage, which wasn't working for me and caused me to try and solve this in the first place).

However, in order to get system-vimjob.vroom to succeed, I need to pass vroom --shell-delay .25 (by default vim's delay is 0.25 and neovim's is 0.0). Here's an example failure:

└─ $ vroom --neovim vroom/system-vimjob.vroom
vroom/system-vimjob.vroom
FAILED on line 45: Multiple failures:
Expected system call not received.

Got no chance to inject response:
 EXPECT:        echo hi (regex mode)

Failed command on line 45:
:let g:invocation = AsyncWait(g:syscall.CallAsync('Callback', 0), 2)

Queued system controls are:
 EXPECT:        echo hi (regex mode)

No system calls received. Perhaps your --shell is broken?

Last few commands (most recent last) were:
:if has('nvim')<CR>  set cmdheight=30<CR>endif<CR><CR>
:set nocompatible<CR>
:let g:maktabadir = fnamemodify($VROOMFILE, ':p:h:h')<CR>
:let g:bootstrapfile = g:maktabadir . '/bootstrap.vim'<CR>
:execute 'source' g:bootstrapfile<CR>
:call maktaba#system#SetUsableShellRegex('\v<shell\.vroomfaker$')<CR>
:if !has('job') && !has('nvim')<CR>  echomsg maktaba#error#MissingFeature(      'Must have +job support to run system-vimjob.vroom examples')<CR>endif<CR>
:function AsyncWait(invocation, delay) abort<CR>  let l:deadline = localtime() + a:delay " wait for at most N seconds<CR>  while !a:invocation.finished && localtime() < l:deadline<CR>    sleep 100m<CR>  endwhile<CR>  call maktaba#ensure#IsTrue(a:invocation.finished, 'Async callback was expected to complete within %d seconds', a:delay)<CR>  return a:invocation<CR>endfunction<CR>
:let g:syscall = maktaba#syscall#Create(['echo', 'hi'])<CR>
:function Callback(result) abort<CR>  let g:callback_stdout = a:result.stdout<CR>endfunction<CR>
:let g:invocation = AsyncWait(g:syscall.CallAsync('Callback', 0), 2)<CR>

Ran 1 test in vroom/system-vimjob.vroom. 0 passing, 0 errored, 1 failed.

Again, this works fine:

└─ $ vroom --neovim vroom/system-vimjob.vroom --shell-delay .25
vroom/system-vimjob.vroom
Ran 1 test in vroom/system-vimjob.vroom. 1 passing, 0 errored, 0 failed.

Any ideas on whether this is an actual problem in my use of jobcontrol, or if the async delay is just necessary for nvim in this case? I've looked into it for a bit, but this is my first adventure in maktaba/vroom.

[google-cla]

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project (if not, look below for help). Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed (or fixed any issues), please reply here with @googlebot I signed it! and we'll verify it.

---

What to do if you already signed the CLA

Individual signers

• It's possible we don't have your GitHub username or you're using a different email address on your commit. Check your existing CLA data and verify that your email is set on your git commits.

Corporate signers

• Your company has a Point of Contact who decides which employees are authorized to participate. Ask your POC to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the Google project maintainer to go/cla#troubleshoot (Public version).
• The email used to register you as an authorized contributor must be the email used for the Git commit. Check your existing CLA data and verify that your email is set on your git commits.
• The email used to register you as an authorized contributor must also be attached to your GitHub account.

ℹ️ Googlers: Go here for more info.

[okkays]

@googlebot I signed it!

[dbarnett]

Sure, I can look into why.

For starters you might hack the AsyncAwait helper in the test to change the loop from

  |  while !a:invocation.finished && localtime() < l:deadline<CR>
  |    sleep 100m<CR>
  |  endwhile<CR>

to

  |  let l:finished_log = []<CR>
  |  while !a:invocation.finished && localtime() < l:deadline<CR>
  |    call add(l:finished_log, a:invocation.finished)<CR>
  |    sleep 100m<CR>
  |  endwhile<CR>
  |  echomsg l:finished_log<CR>

and see what that spits out.

It would be strange if at_exit was getting called before the syscall process actually finished writing its files... maybe a slight bug with neovim jobcontrol?

[okkays]

finished_log contains a single 0 at that point - it looks like it's finishing within 100ms (though not before getting to that loop).

The jobcontrol docs actually explicitly mention this kind of bug when using callbacks, but suggests channel-buffered as a workaround (which I'm using).

If I look at self.stdout in maktaba#syscall#neovim#HandleJobExit, it actually contains hi - so it looks like nvim is doing what it's supposed to and populating stdout on time. That would make me think that jobstart isn't using shell.vroomfaker, but since it works fine with the delay, maybe it's a bug in how we're using vroom to capture neovim syscalls? Something there would also explain why it works fine in practice, too.

[dbarnett]

K, have you tried adding a delay control in the vroom file to just the one line to see if that takes care of failures? As long as we have the tests passing consistently and a note explaining what's going on, I'm fine with proceeding now that you've done some initial investigation.

Do consider either renaming system-vimjob.vroom or splitting into vim/neovim variants. The "vimjob" was referring specifically to +vimjob.

[okkays]

Went ahead and added some delays. I'm really liking vroom's style and syntax, nice job on that 👍

I've renamed the test to system-job.vroom (:help job will take users to the right place in both vim and nvim).

I chose to rename instead of split because of a note in the issue (#125):

Note this might complicate testing since the actual system calls would be different between vim and neovim, but a single vroom file would have to test them both.

which (barring the delays) makes sense to me, because both applications should have the same API, and this way there won't need to be special handling to skip each split file for one or the other (at least, whenever the tests are updated to use vim8). Plus, changes to that API only need to be tested in one place.

lmk if there's anything else to tweak

[dbarnett]

Went ahead and added some delays. I'm really liking vroom's style and syntax, nice job on that 👍

That's credit to @Soares who pushed for literate testing and built the initial implementation. =)

[dbarnett]

Thanks! Merging the changes.

@okkays if you could go ahead and report a bug for the mysterious delays, I do suspect we could hit on some fix we could get into the implementation so that we don't have a proliferation of weird workarounds in all the tests affected by it.