Splitting jupyterlab_robotmode in a different repository?

[SylvainCorlay]

It may be useful to have that in a standalone project that could be used independently of robotkernel.

[SylvainCorlay]

Note: @martinRenou made an update to the extension so that it works with JupyterLab 3, and the new dynamic extension system, which would allow packaging it as a pip/conda package different from robotkernel.

[datakurre]

@SylvainCorlay Thanks! I recall that @bollwyvl mentioned also being working on that.

Did I understood correctly that JupyterLab 3 could install the robotmode from PyPI and npmjs-release should be therefore deprecated?

[martinRenou]

Did I understood correctly that JupyterLab 3 could install the robotmode from PyPI

Yes, you could indeed! That makes user's experience much nicer as they don't need node installed on their machine to install the extension.

npmjs-release would not be required anymore. Although if it makes sense for the extension to be extended on the JavaScript side of the world, then it could still make sense to publish it on npm. But that might not be the case for robotmode?

[datakurre]

Thanks. I must admit that I have not been able learn JupyterLab 3 yet. A few weeks ago @bollwyvl mentioned it being ready and was planning to port his Robot Framework related stuff for it, and also create dedicated repository for jupyterlab_robotmode (which already was a fork from https://github.com/gtri/irobotframework/tree/master/src/packages/jupyterlab-robotframework).

[SylvainCorlay]

Which package is used to make releases of the jupyterlab extension then? I am confused with all the forks.

[bollwyvl]

This repo has been the source of npm releases. I've done the work for splitting this out and testing it, but haven't put it anyplace yet... It took some patches to core that have since been released, and can likely beer cleaned up. it should probably be two packages, one of which wouldn't know about lab.

[kariharju]

Hi, yeah to clarify we are kind of pushing the update to JupyterLab 3 heavily to rid of the extension building on target machines in Robocorp Lab so we are working to update to 3 as soon as it is out (or even sooner :)).
Sylvain and Martin are helping us out in that.

[bollwyvl]

I'd love to see editor syntaxes become maintained by the robot community more directly, as the pygments lexer is. We could even investigate using a textmate bundle so there is The One True Syntax... though last I checked, in lab this would require a wasm build of oniguruma, see: https://github.com/deathbeds/jupyterlab-simple-syntax <https://github.com/deathbeds/jupyterlab-simple-syntax> Another place this could land is jupyterlab-lsp, where we already use the heck out of robot: https://github.com/krassowski/jupyterlab-lsp/issues/332 We're getting more contributions there and people that "need" things on lab 3: https://github.com/krassowski/jupyterlab-lsp/issues/400 Ideally, that work will move to a new org, e.g jupyter-language-tools, etc. but seems like a JEP-level thing... But maybe lab 3, and some co-maintaining "needs" could help that along. If all of these are non-starters, anybody has our explicit permission, by license, to fork... but of course we'd prefer to work together! the maintainers of this repo use robot daily for our day jobs, we only see sporadic funding/contribution for this open source work, hence we've been liberally borrowing between repos when we'd need them.

[datakurre]

For me it sounds like forking the syntax highlighting into a separate repository is OK for everyone, if we only find the right home for it. Having the plugin closer to Robot Framework community sounds great for me too.

Could https://github.com/marketsquare /https://marketsquare.github.io/ be home for the new repo? (Including giving PyPI release maintainer rights also to some robot community member.) For me it sounds like the repo and release should be made from Martin's Jupyter Lab 3 -compatible version. We could then fork it here and propose Nick's possible refactoring later.

[aikarjal]

I think having it on marketsquare is a good idea. Someone from Robocorp can be a maintainer on PyPi release (@xylix / @mikahanninen / @osrjv for instance)

[xylix]

Sounds pretty good to me at least.

[bollwyvl]

Marketplace sounds like a good path forward. I would propose it ends up being a Lerna monorepo, e.g.

package.json
lerna.json
tsconfig.json
atest/
js_packages/  
  robotframework-syntax-common/  
  codemirror5-robotframework/  
  robotkernel-syntax-mode
py_packages/
  jupyterlab-robotframework-syntax/
    setup.py

Such that adding...

js_packages/
  monaco-robotframework/
  codemirror6-robotframework/  
<lan>_packages/
  <whatever>-robotframework

...isn't too challenging in the future: I had reached out to somebody who was maintaining a textmate bundle, but never heard back.

On the irobotframework PR that is currently shown in here, I was doing screenshot tests vs every example in the RFUG... Seems like we could apply a similar approach (albeit, more elegantly with some data files) and do a test of the pygments lexer output, e.g. Get Pygments HTML vs a live-updated CM5 vs whatever else comes up.

Another clutch thing to test, which I didn't get around to, is emulating typing out the full example, character by character... this is important to test whether the regexen ever end up in a non-halting state.

Regarding pymgents: I guess the ideal would be to someday get all these living together next to the official pygments lexer, such that a change can be co-verified between all of them (e.g the forthcoming IF THEN ELSE).

[SylvainCorlay]

Lerna has been the main source of pain for lab, jupyter widgets. If we could avoid having a monorepo for packages across several languages (py_packages, js_packages), that would be nicer IMO.

[bollwyvl]

I'm going to counter that npm/yarn/webpack (and the requirejs/unpkg hangover on widgets) are the source of pain I have observed when trying to work first- or third-party... as well as data_files, and the jupyter config system, and keeping those working on proprietary platforms. typescript is arguable, but a damned sight better than babel... I don't know how you'd be able to track some of this stuff long term without the types, whether inline or as jsdoc. I think lerna has made some of that more reasonable, as you can hoist common concerns out of individual packages.

Anyhow: i'd wager this future repo pretty much needs to be multi-language as it needs to be able to:

• do some kind of node build, test, or at least lint/typechecks
  • the straight esmodules with jsdoc could obviate this, but not for the jupyter contraption
• publish to npm that Just Work in the various flavors of distribution: "dumb" browser with globals, common, umd/amd, etc.
  • i really don't want to see lab3 lead people to writing things where the "good stuff" only works inside the (open) walled garden...
    • M$ is better at it, and winning (see earlier request for an npm package for something that's open source, but only published on the vscode marketplace).
• publish to pypi
  • this complexity needs to go down to a reproducible, pyproject-only approach... flit would be my boo, but sanely doesn't support data_files (though it's being considered for man pages, i guess)
• be able to be tested with robot in a real browser
  • i've given up on testing in "real" chrome, and tire of things that make that "easy", and am sticking to selenium/geckodriver/firefox, for all its warts

[martinRenou]

I am not a big fan of the big monorepo approach, it makes development difficult, and it makes more sense to split into separate repos in this case where users want to install the robot syntax highlighting extension but not the kernel.

[martinRenou]

I think having it on marketsquare is a good idea

Could someone with the admin rights on marketsquare create a jupyterlab_robotmode repository?

Someone from Robocorp can be a maintainer on PyPi release

Until JupyterLab 3 is released I guess it would be an NPM release only

[xylix]

Created a ticket requesting creation of the repository. Everybody who wants maintainer access to the repo, please comment at MarketSquare/MarketSquare#19

[xylix]

The repo has now been created at https://github.com/MarketSquare/jupyterlab_robotmode. Invited @martinRenou @SylvainCorlay @bollwyvl as maintainers, feel free to accept and put content there as you see fit.

[martinRenou]

Thanks a lot!

[martinRenou]

I will push the first commit if nobody is against it.

[xylix]

Can probably close this issue now? Perhaps change https://github.com/robots-from-jupyter/robotkernel/tree/master/src to use a git submodule

[datakurre]

@xylix If you don’t mind, I leave this open as the reminder to adapt this repository for the split, once I return to work on this!

Thanks all!

[datakurre]

Actually, it would be nicer from me to create a new issue about that so I close this now.

[datakurre]

@xylix Btw, whom should have rights to publish https://www.npmjs.com/package/jupyterlab_robotmode ?

[emanlove]

@datakurre @xylix If we can I would like to create a RobotFramework MarketSquare account for npmjs so there is a joint community (or at least MarketSquare admin) user in addition to a maintainer for long term account maintenance.

[bollwyvl]

npm has support for organizations, which show up as e.g. @this-organization:

Once logged in:
https://www.npmjs.com/org/create

This might be a reasonable path vs shared account, etc. I think there are also fine-grained perms on individual packages.

[emanlove]

Was reading the same. I was going to build an MarketSquare organization. I need to learn a little more about how they work.

[martinRenou]

Can probably close this issue now? Perhaps change https://github.com/robots-from-jupyter/robotkernel/tree/master/src to use a git submodule

It might be nicer to just make it a dependency?