Skip to content
This repository has been archived by the owner on Apr 9, 2024. It is now read-only.

PDF failure #13

Closed
eitanlees opened this issue Jan 3, 2020 · 61 comments
Closed

PDF failure #13

eitanlees opened this issue Jan 3, 2020 · 61 comments

Comments

@eitanlees
Copy link

Giving the package a run but failed to produce PDFs. The function runs without errors but the pdf produced is blank. I am on macOS 10.12.6

Is there a way I specify the backend or does it know to use nodejs when I ask for a PDF?

I ran npm install vega-lite vega-cli canvas which resulted in

npm WARN saveError ENOENT: no such file or directory, open '/Users/eitanlees/package.json'
npm WARN enoent ENOENT: no such file or directory, open '/Users/eitanlees/package.json'
npm WARN eitanlees No description
npm WARN eitanlees No repository field.
npm WARN eitanlees No README data
npm WARN eitanlees No license field.

+ [email protected]
+ [email protected]
+ [email protected]
updated 3 packages and audited 14777 packages in 4.667s

4 packages are looking for funding
  run `npm fund` for details

found 1 low severity vulnerability
  run `npm audit fix` to fix them, or `npm audit` for details

I am unfamiliar with node so I don't know if that is what should happen.

Here is a small script I ran in a Jupyter notebook for reproducibility.

import altair as alt
from vega_datasets import data
from altair_saver import save

chart = alt.Chart(data.iris.url).mark_point().encode(
    alt.X('petalWidth:Q'), alt.Y('sepalLength:Q')
)

save(chart, 'test.pdf')
@jakevdp
Copy link
Member

jakevdp commented Jan 3, 2020

I suspect the issue is you installed the npm packages locally, then ran the Python code from another location that could not access the local install. Try installing the npm packages globally instead:

$ npm install -g vega-lite vega-cli canvas

@jakevdp
Copy link
Member

jakevdp commented Jan 3, 2020

If there are node-related errors, they may be printed in the terminal from which you launched the notebook rather than in the notebook's stdout.

@eitanlees
Copy link
Author

hmmm, I reinstalled globally like you suggested but am still having trouble.

The terminal reads:

TypeError: Cannot read property 'getContext' of null
    at resize (/usr/local/lib/node_modules/vega/node_modules/vega-scenegraph/build/vega-scenegraph.js:3607:26)
    at CanvasRenderer.prototype$6.resize (/usr/local/lib/node_modules/vega/node_modules/vega-scenegraph/build/vega-scenegraph.js:3657:5)
    at CanvasRenderer.prototype$4.initialize (/usr/local/lib/node_modules/vega/node_modules/vega-scenegraph/build/vega-scenegraph.js:3235:17)
    at CanvasRenderer.prototype$6.initialize (/usr/local/lib/node_modules/vega/node_modules/vega-scenegraph/build/vega-scenegraph.js:3652:28)
    at initializeRenderer (/usr/local/lib/node_modules/vega/node_modules/vega-view/build/vega-view.js:632:8)
    at renderHeadless (/usr/local/lib/node_modules/vega/node_modules/vega-view/build/vega-view.js:738:12)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async View.renderToCanvas [as toCanvas] (/usr/local/lib/node_modules/vega/node_modules/vega-view/build/vega-view.js:773:15)

@jakevdp
Copy link
Member

jakevdp commented Jan 3, 2020

Ah, it looks like canvas is not correctly configured. Can you try installing the required packages listed at https://www.npmjs.com/package/canvas ?

@jakevdp
Copy link
Member

jakevdp commented Jan 3, 2020

We'll need to update the instructions in the README to point to these.

@jakevdp
Copy link
Member

jakevdp commented Jan 3, 2020

Also in the invocation we should grab stderr from the subprocess so that it actually prints in the notebook.

@jakevdp
Copy link
Member

jakevdp commented Jan 4, 2020

#14 addresses the subprocess/stderr issue.

@jakevdp
Copy link
Member

jakevdp commented Jan 4, 2020

I updated the README with instructions on installing requirements with conda. Let me know if that helps

@eitanlees
Copy link
Author

Thanks! I will give that a try when I can and let you know.

@eitanlees
Copy link
Author

Well my installation woes continue. Between conda, pip, brew, and the default system python there could be many points of confusion. Let me lay out the facts (as far as I understand them. )

  1. Am I running what I think I am running? Let's look at the paths.
$ which python
/Users/eitanlees/miniconda3/bin/python
$ python --version
Python 3.7.6
$ which npm
/Users/eitanlees/miniconda3/bin/npm
$ npm --version
6.12.0
$ which node
/Users/eitanlees/miniconda3/bin/node
$ node --version
v13.0.0

This leads me to believe I am running the correct python, npm, and node.

I followed the instructions in the README.md for installation

$ conda install -c conda-forge nodejs
$ npm install -g vega-lite vega-cli canvas

The output of the npm call is

npm WARN npm npm does not support Node.js v13.0.0
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11, 12.
npm WARN npm You can find the latest version at https://nodejs.org/
/Users/eitanlees/miniconda3/bin/vg2pdf -> /Users/eitanlees/miniconda3/lib/node_modules/vega-cli/bin/vg2pdf
/Users/eitanlees/miniconda3/bin/vg2png -> /Users/eitanlees/miniconda3/lib/node_modules/vega-cli/bin/vg2png
/Users/eitanlees/miniconda3/bin/vg2svg -> /Users/eitanlees/miniconda3/lib/node_modules/vega-cli/bin/vg2svg
/Users/eitanlees/miniconda3/bin/vl2png -> /Users/eitanlees/miniconda3/lib/node_modules/vega-lite/bin/vl2png
/Users/eitanlees/miniconda3/bin/vl2vg -> /Users/eitanlees/miniconda3/lib/node_modules/vega-lite/bin/vl2vg
/Users/eitanlees/miniconda3/bin/vl2svg -> /Users/eitanlees/miniconda3/lib/node_modules/vega-lite/bin/vl2svg
npm WARN [email protected] requires a peer of vega@^5.9.0 but none is installed. You must install peer dependencies yourself.

+ [email protected]
+ [email protected]
+ [email protected]
added 57 packages from 15 contributors and updated 3 packages in 4.775s 

I would hope now that everything should work. I start up a lab session jupyter lab from the root directory of the project (after pulling in any new upstream changes). Running the example code first posted the output is

npm WARN npm npm does not support Node.js v13.0.0
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11, 12.
npm WARN npm You can find the latest version at https://nodejs.org/
npm WARN npm npm does not support Node.js v13.0.0
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11, 12.
npm WARN npm You can find the latest version at https://nodejs.org/
TypeError: Cannot read property 'getContext' of null
    at resize (/Users/eitanlees/miniconda3/lib/node_modules/vega-cli/node_modules/vega-scenegraph/build/vega-scenegraph.js:3607:26)
    at CanvasRenderer.prototype$6.resize (/Users/eitanlees/miniconda3/lib/node_modules/vega-cli/node_modules/vega-scenegraph/build/vega-scenegraph.js:3657:5)
    at CanvasRenderer.prototype$4.initialize (/Users/eitanlees/miniconda3/lib/node_modules/vega-cli/node_modules/vega-scenegraph/build/vega-scenegraph.js:3235:17)
    at CanvasRenderer.prototype$6.initialize (/Users/eitanlees/miniconda3/lib/node_modules/vega-cli/node_modules/vega-scenegraph/build/vega-scenegraph.js:3652:28)
    at initializeRenderer (/Users/eitanlees/miniconda3/lib/node_modules/vega-cli/node_modules/vega-view/build/vega-view.js:632:8)
    at renderHeadless (/Users/eitanlees/miniconda3/lib/node_modules/vega-cli/node_modules/vega-view/build/vega-view.js:738:12)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async View.renderToCanvas [as toCanvas] (/Users/eitanlees/miniconda3/lib/node_modules/vega-cli/node_modules/vega-view/build/vega-view.js:773:15)

and a blank (zero byte) PDF is still produced. It looks like two warnings, then there is the canvas error.

I am left in the same position I was in before but now with more errors/warnings. haha.

Sorry to unload all these installation problems on you.

@jakevdp
Copy link
Member

jakevdp commented Jan 6, 2020

OK – I don't know a lot about this, but I think this indicates some kind of problem with your node-canvas installation, because this was the same error I was seeing before I upgraded canvas.

@jakevdp
Copy link
Member

jakevdp commented Jan 6, 2020

What happens if you add vega to the list of installed npm packages?

@eitanlees
Copy link
Author

I tried that as well.

$ npm install -g vega vega-lite vega-cli canvas
npm WARN npm npm does not support Node.js v13.0.0
npm WARN npm You should probably upgrade to a newer version of node as we
npm WARN npm can't make any promises that npm will work with this version.
npm WARN npm Supported releases of Node.js are the latest release of 6, 8, 9, 10, 11, 12.
npm WARN npm You can find the latest version at https://nodejs.org/
/Users/eitanlees/miniconda3/bin/vg2pdf -> /Users/eitanlees/miniconda3/lib/node_modules/vega-cli/bin/vg2pdf
/Users/eitanlees/miniconda3/bin/vg2png -> /Users/eitanlees/miniconda3/lib/node_modules/vega-cli/bin/vg2png
/Users/eitanlees/miniconda3/bin/vg2svg -> /Users/eitanlees/miniconda3/lib/node_modules/vega-cli/bin/vg2svg
/Users/eitanlees/miniconda3/bin/vl2png -> /Users/eitanlees/miniconda3/lib/node_modules/vega-lite/bin/vl2png
/Users/eitanlees/miniconda3/bin/vl2svg -> /Users/eitanlees/miniconda3/lib/node_modules/vega-lite/bin/vl2svg
/Users/eitanlees/miniconda3/bin/vl2vg -> /Users/eitanlees/miniconda3/lib/node_modules/vega-lite/bin/vl2vg
+ [email protected]
+ [email protected]
+ [email protected]
+ [email protected]
removed 57 packages and updated 4 packages in 5.026s

The results in the notebook were the same. No PDF

@jakevdp
Copy link
Member

jakevdp commented Jan 6, 2020

Can you get PNG output via Node?

@eitanlees
Copy link
Author

So I can save PNG but I don't know which method it's using.

@jakevdp
Copy link
Member

jakevdp commented Jan 6, 2020

pass method='node' to force it to use the nodejs method

@eitanlees
Copy link
Author

The PNG fails (blank image) using method='node' with the following error.

TypeError: Cannot read property 'getContext' of null
    at resize (/Users/eitanlees/miniconda3/lib/node_modules/vega/node_modules/vega-scenegraph/build/vega-scenegraph.js:3607:26)
    at CanvasRenderer.prototype$6.resize (/Users/eitanlees/miniconda3/lib/node_modules/vega/node_modules/vega-scenegraph/build/vega-scenegraph.js:3657:5)
    at CanvasRenderer.prototype$4.initialize (/Users/eitanlees/miniconda3/lib/node_modules/vega/node_modules/vega-scenegraph/build/vega-scenegraph.js:3235:17)
    at CanvasRenderer.prototype$6.initialize (/Users/eitanlees/miniconda3/lib/node_modules/vega/node_modules/vega-scenegraph/build/vega-scenegraph.js:3652:28)
    at initializeRenderer (/Users/eitanlees/miniconda3/lib/node_modules/vega/node_modules/vega-view/build/vega-view.js:632:8)
    at renderHeadless (/Users/eitanlees/miniconda3/lib/node_modules/vega/node_modules/vega-view/build/vega-view.js:738:12)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async View.renderToCanvas [as toCanvas] (/Users/eitanlees/miniconda3/lib/node_modules/vega/node_modules/vega-view/build/vega-view.js:773:15)

Same error. hmmmm

@jakevdp
Copy link
Member

jakevdp commented Jan 6, 2020

OK... I think this may be worth raising to the vega project itself. Try repoing the issue without using Altair; i.e. make certain the conversion binaries are in your path, and run the following at the command line:

$ echo '{"data": {"url": "https://vega.github.io/vega-datasets/data/cars.json"}, "mark": "point"}' | vl2vg | vg2pdf > out.pdf

The expected output file should look something like this:
out
but I suspect you'll see the same error you did above.

@eitanlees
Copy link
Author

Yes, same error! Where should I raise the issue?

@jakevdp
Copy link
Member

jakevdp commented Jan 6, 2020

Maybe the main vega repo? https://github.com/vega/vega

@eitanlees
Copy link
Author

Well I am a little embarrassed but I've figured out what was wrong. I had something in my .npmrc that was causing the installation to not work.

I am happy to report that everything works now!

@jakevdp
Copy link
Member

jakevdp commented Jan 7, 2020

Awesome! To better help future users, what was the problematic entry in npmrc?

@eitanlees
Copy link
Author

A while back I had run npm config set ignore-scripts true which set up a .npmrc file in my root directory with a single line.

ignore-scripts=true

This was the source of all of my problems. It didn't allow the installation to run properly, and as a result, nothing worked.

@braaannigan
Copy link

I had the same original error when trying to produce .png files, resolved as above by the global npm install

npm install -g vega-lite vega-cli canvas

Thanks @jakevdp

@eharkins
Copy link

eharkins commented Feb 26, 2020

@jakevdp I created a new environment using conda per the README instructions and am experiencing this issue despite having installed altair and altair_saver in my environment (see below for output of conda list --export):

(altair_min) [eharkins@stoat altair_saver_test]$ python
Python 3.6.7 | packaged by conda-forge | (default, Nov  6 2019, 16:19:42) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import altair as alt

>>> import pandas as pd
>>> from altair_saver import save
>>> source = pd.DataFrame({
...     'a': ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I'],
...     'b': [28, 55, 43, 91, 81, 53, 19, 87, 52]
... })
>>> chart = alt.Chart(source).mark_bar().encode(
...     x='a',
...     y='b'
... )
>>> save(chart, "chart.vl.json")              # Vega-Lite JSON specification
>>> save(chart, "chart.vg.json")              # Vega JSON specification
>>> save(chart, "chart.html")                 # HTML document
>>> save(chart, "chart.html", inline=True)    # HTML document with all JS code included inline
>>> save(chart, "chart.png")                  # PNG Image
TypeError: Cannot read property 'getContext' of null
    at resize (/home/eharkins/miniconda2/envs/altair_min/lib/vega-cli/node_modules/vega-scenegraph/build/vega-scenegraph.js:3377:26)
    at CanvasRenderer.prototype$6.resize (/home/eharkins/miniconda2/envs/altair_min/lib/vega-cli/node_modules/vega-scenegraph/build/vega-scenegraph.js:3427:5)
    at CanvasRenderer.prototype$4.initialize (/home/eharkins/miniconda2/envs/altair_min/lib/vega-cli/node_modules/vega-scenegraph/build/vega-scenegraph.js:2989:17)
    at CanvasRenderer.prototype$6.initialize (/home/eharkins/miniconda2/envs/altair_min/lib/vega-cli/node_modules/vega-scenegraph/build/vega-scenegraph.js:3422:28)
    at initializeRenderer (/home/eharkins/miniconda2/envs/altair_min/lib/vega-cli/node_modules/vega-view/build/vega-view.js:630:8)
    at renderHeadless (/home/eharkins/miniconda2/envs/altair_min/lib/vega-cli/node_modules/vega-view/build/vega-view.js:736:12)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async View.renderToCanvas [as toCanvas] (/home/eharkins/miniconda2/envs/altair_min/lib/vega-cli/node_modules/vega-view/build/vega-view.js:771:15)
>>> save(chart, "chart.svg")                  # SVG Image
>>> save(chart, "chart.pdf")                  # PDF Image
TypeError: Cannot read property 'getContext' of null
    at resize (/home/eharkins/miniconda2/envs/altair_min/lib/vega-cli/node_modules/vega-scenegraph/build/vega-scenegraph.js:3377:26)
    at CanvasRenderer.prototype$6.resize (/home/eharkins/miniconda2/envs/altair_min/lib/vega-cli/node_modules/vega-scenegraph/build/vega-scenegraph.js:3427:5)
    at CanvasRenderer.prototype$4.initialize (/home/eharkins/miniconda2/envs/altair_min/lib/vega-cli/node_modules/vega-scenegraph/build/vega-scenegraph.js:2989:17)
    at CanvasRenderer.prototype$6.initialize (/home/eharkins/miniconda2/envs/altair_min/lib/vega-cli/node_modules/vega-scenegraph/build/vega-scenegraph.js:3422:28)
    at initializeRenderer (/home/eharkins/miniconda2/envs/altair_min/lib/vega-cli/node_modules/vega-view/build/vega-view.js:630:8)
    at renderHeadless (/home/eharkins/miniconda2/envs/altair_min/lib/vega-cli/node_modules/vega-view/build/vega-view.js:736:12)
    at processTicksAndRejections (internal/process/task_queues.js:97:5)
    at async View.renderToCanvas [as toCanvas] (/home/eharkins/miniconda2/envs/altair_min/lib/vega-cli/node_modules/vega-view/build/vega-view.js:771:15)

Here is the output of conda list --export:

# This file may be used to create an environment using:
# $ conda create --name <env> --file <this file>
# platform: linux-64
_libgcc_mutex=0.1=main
altair=4.0.1=py_0
altair_data_server=0.4.1=py_0
altair_saver=0.1.0=py_1
altair_viewer=0.2.0=py_0
attrs=19.3.0=py_0
ca-certificates=2019.11.28=hecc5488_0
certifi=2019.11.28=py36_0
cffi=1.13.2=py36h8022711_0
cryptography=2.8=py36h72c5cf5_1
entrypoints=0.3=py36_1000
icu=64.2=he1b5a44_1
idna=2.9=py_1
importlib_metadata=1.5.0=py36_0
jinja2=2.11.1=py_0
jsonschema=3.2.0=py36_0
libblas=3.8.0=14_openblas
libcblas=3.8.0=14_openblas
libffi=3.2.1=he1b5a44_1006
libgcc-ng=9.1.0=hdf63c60_0
libgfortran-ng=7.3.0=hdf63c60_5
liblapack=3.8.0=14_openblas
libopenblas=0.3.7=h5ec1e0e_6
libstdcxx-ng=9.2.0=hdf63c60_2
libuv=1.34.0=h516909a_0
markupsafe=1.1.1=py36h516909a_0
ncurses=6.1=hf484d3e_1002
nodejs=13.9.0=h10a4023_0
numpy=1.18.1=py36h95a1406_0
openssl=1.1.1d=h516909a_0
pandas=1.0.1=py36hb3f55d8_0
pip=9.0.1=py36_1
portpicker=1.3.1=py36_0
pycparser=2.19=py_2
pyopenssl=19.1.0=py_1
pyrsistent=0.15.7=py36h516909a_0
pysocks=1.7.1=py36_0
python=3.6.7=h357f687_1006
python-dateutil=2.8.1=py_0
pytz=2019.3=py_0
readline=8.0=hf8c457e_0
selenium=3.141.0=py36h516909a_1000
setuptools=45.2.0=py36_0
six=1.14.0=py36_0
sqlite=3.30.1=hcee41ef_0
tk=8.6.10=hed695b0_0
toolz=0.10.0=py_0
tornado=6.0.3=py36h516909a_4
urllib3=1.25.7=py36_0
vega-cli=5.6.0=0
vega-lite-cli=3.4.0=0
wheel=0.34.2=py36_0
xz=5.2.4=h14c3975_4
zipp=3.0.0=py_0
zlib=1.2.11=h7b6447c_3

It would seem like an issue with canvas as you have said above.
I'm not sure if doing conda install is supposed to install node dependencies as well but:

(altair_min) [eharkins@stoat altair_saver_test]$ which npm
/home/eharkins/miniconda2/envs/altair_min/bin/npm
(altair_min) [eharkins@stoat altair_saver_test]$ npm list
/home/eharkins/altair_saver_test
└── (empty)

The README implies I don't have to do any npm installing if I'm using conda and that deps are installed

automatically when installing altair_saver via conda-forge

Thanks!

@jakevdp
Copy link
Member

jakevdp commented Feb 26, 2020

The issue is not with conda, the issue comes from how nodejs is set up on your system. Conda will automatically npm install everything that's needed, but if you have existing global npm configurations on your system, it may interfere with this process.

I don't know npm well enough to suggest where to look, but look for configurations, remove them, and then try installing the tools again.

Or, if conda is not working for you, you can install the deps yourself following the installation docs.

@jakevdp
Copy link
Member

jakevdp commented Feb 26, 2020

This site lists the possible locations of npmrc settings: https://docs.npmjs.com/configuring-npm/npmrc.html

If it's possible to force npm to ignore global settings, we might update the conda recipe to do that.

@jakevdp jakevdp reopened this Feb 26, 2020
@johnmdonich
Copy link

johnmdonich commented Mar 8, 2020

The conda environment has an outdated version of vega-lite-cli and is the likely source of the issues raised by @eharkins.

conda env file

name: altair-saver-test
channels:
  - conda-forge
  - defaults
dependencies:
  - altair_saver
  - black
  - jupyterlab
  - mypy
  - pylint
  - python=3.7

Conda env

# Name                    Version                   Build  Channel
altair                    4.0.1                      py_0    conda-forge
altair_data_server        0.4.1                      py_0    conda-forge
altair_saver              0.1.0                      py_1    conda-forge
altair_viewer             0.2.1                      py_0    conda-forge
...
nodejs                    13.9.0               hfddbe92_0    conda-forge
...
vega-cli                  5.6.0                         0    conda-forge
vega-lite-cli             3.4.0                         0    conda-forge

Notice that the vega-lite-cli is v3 while altair is v4. The v3 conversion on v4 specs can be demonstrated with the .vg.json save of the following chart with the tooltip in the mark

simple_chart = {
  "$schema": "https://vega.github.io/schema/vega-lite/v4.json",
  "description": "A simple bar chart with embedded data.",
  "data": {
    "values": [
      {"a": "A","b": 28}, {"a": "B","b": 55}, {"a": "C","b": 43},
      {"a": "D","b": 91}, {"a": "E","b": 81}, {"a": "F","b": 53},
      {"a": "G","b": 19}, {"a": "H","b": 87}, {"a": "I","b": 52}
    ]
  },
  "mark": {"type": "bar", "tooltip": True},
  "encoding": {
    "x": {"field": "a", "type": "ordinal"},
    "y": {"field": "b", "type": "quantitative"}
  }
}
altair_saver.save(simple_chart, "v4-bar.vg.json")

The vega tooltip encoding that is created is:

"encode": {
        "update": {
          "tooltip": {
            "value": true
          },
          ...

But it should be

"encode": {
        "update": {
          "tooltip": {
            "signal": "{\"a\": ''+datum[\"a\"], \"b\": format(datum[\"b\"], \"\")}"
          },

Unfortunately this will always happen with a conda env because it will always use the nodejs in the conda environment ahead of the local node installed. So even if you have the correct version of node and vega-lite-cli installed locally it will always occur further down the path than the version in the conda environment.

@jakevdp
Copy link
Member

jakevdp commented Mar 8, 2020

The vega-cli version could certainly be updated, but I don't think is the cause of the error seen by @eharkins.

In particular, the bash commands in #13 (comment) work on my computer with the current conda vega-cli, but result in the reported error for @eharkins.

@sehilyi
Copy link

sehilyi commented Apr 3, 2020

Using --force option to force installing the packages worked for me:

npm install -g --force vega-lite vega-cli canvas

@birdsarah
Copy link

@jakevdp on the linked issue you said:

The best approach is probably to use the selenium saver instead; it doesn't have the same issues that the node saver does.

Is that a solution that would work here? I'm just trying to use chart.save('a.png').

If yes, how do I use the selenium saver?

@birdsarah
Copy link

birdsarah commented Apr 17, 2020

My problem was resolved with apt install chromium-chromedriver I missed this instruction.

For extra context, the error that brought me to this issue was TypeError: Cannot read property 'getContext' of null

@jakevdp
Copy link
Member

jakevdp commented Apr 17, 2020

To use the selenium saver, you have to have all the requirements for selenium installed. See https://github.com/altair-viz/altair_saver#selenium

@birdsarah
Copy link

birdsarah commented Apr 17, 2020 via email

@jakevdp
Copy link
Member

jakevdp commented Apr 17, 2020

You can set the backend using the method argument (method='selenium' or method='node'). It defaults to what is configured on your system, and uses selenium if both are in play.

@jakevdp
Copy link
Member

jakevdp commented Apr 17, 2020

(Note that selenium doesn't support PDF output, so it's not an option to solve the original problem in this issue).

@matmel
Copy link

matmel commented May 23, 2020

Hi @jakevdp, I suffer from the same issue but I think I understand why now. The TL,DR is that at the moment, conda-forge packages vega-cli and vega-lite-cli are broken for their intended usage.

Disclaimer: I am familiar with conda packages building but not very familiar with javascript ecosystem. I did a little bit of reading though about npm and its local and global mode (https://docs.npmjs.com/cli/npm#directories) in particular. Same thing with the alternative javascript package manager yarn: https://classic.yarnpkg.com/en/docs/cli/add/#toc-caveats

Install altair, altair_saver with conda in a fresh conda environment:

conda create -n test-altair -c defaults -c conda-forge altair altair_saver

altair_saver will pull a bunch of packages: altair_data_server, altair_viewer, nodejs, vega-cli and vega-lite-cli as declared dependencies.

Once activated, the following command fails:

$ conda activate test-altair
$ echo '{"data": {"url": "https://vega.github.io/vega-datasets/data/cars.json"}, "mark": "point"}' | vl2vg | vg2pdf > out.pdf 
TypeError: Cannot read property 'length' of undefined
    at expression (/home/username/conda/test-altair/lib/vega-cli/node_modules/vega-view/node_modules/vega-runtime/build/vega-runtime.js:99:19)
    at Context.operatorExpression (/home/username/conda/test-altair/lib/vega-cli/node_modules/vega-view/node_modules/vega-runtime/build/vega-runtime.js:149:14)
    at Context.parseOperator (/home/username/conda/test-altair/lib/vega-cli/node_modules/vega-view/node_modules/vega-runtime/build/vega-runtime.js:221:27)
    at operators.forEach.entry (/home/username/conda/test-altair/lib/vega-cli/node_modules/vega-view/node_modules/vega-runtime/build/vega-runtime.js:30:36)
    at Array.forEach (<anonymous>)
    at Context.parse (/home/username/conda/test-altair/lib/vega-cli/node_modules/vega-view/node_modules/vega-runtime/build/vega-runtime.js:30:15)
    at runtime (/home/username/conda/test-altair/lib/vega-cli/node_modules/vega-view/build/vega-view.js:823:67)
    at new View (/home/username/conda/test-altair/lib/vega-cli/node_modules/vega-view/build/vega-view.js:1062:17)
    at render (/home/username/conda/test-altair/lib/vega-cli/node_modules/vega-cli/src/render.js:47:18)
    at read.then.text (/home/username/conda/test-altair/lib/vega-cli/node_modules/vega-cli/src/render.js:61:19)

My understanding of the situation is that the two conda packages vega-cli and vega-lite-cli were installed "locally" as shown in those 2 packages recipes:

Both use the command yarn add packagename in a "local" aribitrary directory of the conda environment conda_environment/lib/package_name/node-modules. I really think that in this case, we should install the npm packages "globally" because here, globally means at the root directories of the conda environment: conda_environment/lib/node_modules (note the difference: in this case package_name does not appear).

In other word. we should do it the same way as we do with pip when we create a python conda package: pip installs the python package "globally" in the conda environment.

By doing the way it is presently in both conda packages, it is like if vega-cli had its own dependencies isolated from vega-lite-cli and its dependencies because they are installed in different node-modules parent directories. And I am not even sure canvas is installed in one of those 2 node-modules which is the root cause of the error message (see this ticket in vega repo)

By removing manually the broken node-modules directories and the symlinks in conda_environment/bin directory and using npm from that environment and doing:

npm install -g vega-cli vega-lite canvas

The command

echo '{"data": {"url": "https://vega.github.io/vega-datasets/data/cars.json"}, "mark": "point"}' | vl2vg | vg2pdf > out.pdf 

works perfectly.

I will open a PR to both packages on conda forge to fix the issue.

@jakevdp
Copy link
Member

jakevdp commented May 23, 2020

Awesome - thanks for investigating!

@rn-c1
Copy link

rn-c1 commented Aug 11, 2020

Hey, just documenting a workaround relating to this issue. I was encountering the same TypeError: Cannot read property 'getContext' of null when trying to save a PNG, but had further challenges installing updated version of all the npm packages due to the need to compile stuff.

Context: this is happening in a Linux Docker container where I do not have admin rights.

Key points:

  • In conda environment, just install nodejs and build dependencies for canvas
    • note: use old version of jpeg
  • Set GCC environment variables to inside conda environment so that build dependencies are picked up.
  • Manually install npm dependencies

Steps in solution:

conda create -n myenv -c conda-forge python=3.7 jpeg=8d giflib=5.2.1 nodejs=14.7.0 -y -q 

conda activate myenv

# adjust paths if necessary - should point within your conda env
CPATH=$HOME/.conda/envs/myenv/include LIBRARY_PATH=$HOME/.conda/envs/myenv/lib \
npm install -g [email protected] [email protected] [email protected]

pip install altair==4.1.0 altair-saver==0.5.0

Possible problems you might run into:

Missing headers from dependency - probably means jpeg or giflib is missing, or GCC can't find the headers.
Make sure they're installed in the conda env, then set the CPATH environment variable accordingly to the include folder inside the conda environment.

In file included from ../src/backend/PdfBackend.h:4:0,
                 from ../src/backend/PdfBackend.cc:1:
../src/backend/../closure.h:6:10: fatal error: jpeglib.h: No such file or directory
 #include <jpeglib.h>
          ^~~~~~~~~~~

error: invalid conversion from ‘int’ to ‘boolean’ [-fpermissive] likely means your JPEG library is too new - try version 8.

...
../src/Image.cc: In member function ‘cairo_status_t Image::decodeJPEGBufferIntoMimeSurface(uint8_t*, unsigned int)’:
../src/Image.cc:929:28: error: invalid conversion from ‘int’ to ‘boolean’ [-fpermissive]
   jpeg_read_header(&args, 1);

Linker can't find libraries from dependency.
Set LIBRARY_PATH to libs path inside conda environment.

...
/usr/bin/ld: cannot find -ljpeg
/usr/bin/ld: cannot find -lgif
collect2: error: ld returned 1 exit status

@ColineG
Copy link

ColineG commented Oct 28, 2020

@jakevdp @rn-c1 I follow your step solution :

conda create -n myenv -c conda-forge python=3.7 jpeg=8d giflib=5.2.1 nodejs=14.7.0 -y -q 
conda activate myenv
pip install altair==4.1.0 altair-saver==0.5.0

I also do :

conda install -c conda-forge vega_datasets

Then I open a python console and run :

import altair as alt
... import pandas as pd
... 
... source = pd.DataFrame({
...     'a': ['A', 'B', 'C', 'D', 'E', 'F', 'G', 'H', 'I'],
...     'b': [28, 55, 43, 91, 81, 53, 19, 87, 52]
... })
... 
... plot = alt.Chart(source).mark_bar().encode(
...     x='a',
...     y='b'
... )
plot.save('bla.png')

That's return me this error :

Error: CanvasRenderer is missing a valid canvas or context
    at Object.error (/home/cgilles/PycharmProjects/metrologie_send_by_mail/node_modules/vega-util/build/vega-util.js:39:11)
    at CanvasRenderer.resize (/home/cgilles/PycharmProjects/metrologie_send_by_mail/node_modules/vega-scenegraph/build/vega-scenegraph.js:3633:28)
    at CanvasRenderer.initialize (/home/cgilles/PycharmProjects/metrologie_send_by_mail/node_modules/vega-scenegraph/build/vega-scenegraph.js:3190:19)
    at CanvasRenderer.initialize (/home/cgilles/PycharmProjects/metrologie_send_by_mail/node_modules/vega-scenegraph/build/vega-scenegraph.js:3621:30)
    at initializeRenderer (/home/cgilles/PycharmProjects/metrologie_send_by_mail/node_modules/vega-view/build/vega-view.js:643:14)
    at renderHeadless (/home/cgilles/PycharmProjects/metrologie_send_by_mail/node_modules/vega-view/build/vega-view.js:762:12)
    at processTicksAndRejections (internal/process/task_queues.js:93:5)
    at async View.renderToCanvas [as toCanvas] (/home/cgilles/PycharmProjects/metrologie_send_by_mail/node_modules/vega-view/build/vega-view.js:798:15)

I'm on linux ubuntu.

Could you please help me. Since days I'm trying to resolve it. I completly uninstall my miniconda environments already.

Thanks

@jakevdp
Copy link
Member

jakevdp commented Oct 28, 2020

I would try another means of installing node-canvas. The conda-forge route often results in these kinds of problems, and it's not clear to me how to fix it.

@ColineG
Copy link

ColineG commented Oct 28, 2020

@jakevdp Thank you I manage to save my .png on my computer !!!!

I would like also to generate a png in a server. I did the same procedure as on my computer, however I get an error :

($ npm install -g vega-lite vega-cli canvas

/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/bin/vg2pdf -> /export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/vega-cli/bin/vg2pdf
/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/bin/vg2png -> /export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/vega-cli/bin/vg2png
/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/bin/vg2svg -> /export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/vega-cli/bin/vg2svg
/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/bin/vl2png -> /export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/vega-lite/bin/vl2png
/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/bin/vl2vg -> /export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/vega-lite/bin/vl2vg
/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/bin/vl2pdf -> /export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/vega-lite/bin/vl2pdf
/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/bin/vl2svg -> /export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/vega-lite/bin/vl2svg

> [email protected] install /export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/canvas
> node-pre-gyp install --fallback-to-build

node-pre-gyp WARN Using needle for node-pre-gyp https download 
node-pre-gyp WARN Tried to download(400): https://github.com/node-gfx/node-canvas-prebuilt/releases/download/v2.6.1/canvas-v2.6.1-node-v83-linux-glibc-x64.tar.gz 
node-pre-gyp WARN Pre-built binaries not found for [email protected] and [email protected] (node-v83 ABI, glibc) (falling back to source compile with node-gyp) 
Package pixman-1 was not found in the pkg-config search path.
Perhaps you should add the directory containing `pixman-1.pc'
to the PKG_CONFIG_PATH environment variable
No package 'pixman-1' found
gyp: Call to 'pkg-config pixman-1 --libs' returned exit status 1 while in binding.gyp. while trying to load binding.gyp
gyp ERR! configure error 
gyp ERR! stack Error: `gyp` failed with exit code: 1
gyp ERR! stack     at ChildProcess.onCpExit (/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/npm/node_modules/node-gyp/lib/configure.js:351:16)
gyp ERR! stack     at ChildProcess.emit (events.js:314:20)
gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:276:12)
gyp ERR! System Linux 3.10.0-1062.9.1.el7.x86_64
gyp ERR! command "/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/bin/node" "/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "configure" "--fallback-to-build" "--module=/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/canvas/build/Release/canvas.node" "--module_name=canvas" "--module_path=/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/canvas/build/Release" "--napi_version=6" "--node_abi_napi=napi" "--napi_build_version=0" "--node_napi_label=node-v83"
gyp ERR! cwd /export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/canvas
gyp ERR! node -v v14.7.0
gyp ERR! node-gyp -v v5.1.0
gyp ERR! not ok 
node-pre-gyp ERR! build error 
node-pre-gyp ERR! stack Error: Failed to execute '/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/bin/node /export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/canvas/build/Release/canvas.node --module_name=canvas --module_path=/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/canvas/build/Release --napi_version=6 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v83' (1)
node-pre-gyp ERR! stack     at ChildProcess.<anonymous> (/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/canvas/node_modules/node-pre-gyp/lib/util/compile.js:83:29)
node-pre-gyp ERR! stack     at ChildProcess.emit (events.js:314:20)
node-pre-gyp ERR! stack     at maybeClose (internal/child_process.js:1051:16)
node-pre-gyp ERR! stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:287:5)
node-pre-gyp ERR! System Linux 3.10.0-1062.9.1.el7.x86_64
node-pre-gyp ERR! command "/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/bin/node" "/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/canvas/node_modules/.bin/node-pre-gyp" "install" "--fallback-to-build"
node-pre-gyp ERR! cwd /export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/canvas
node-pre-gyp ERR! node -v v14.7.0
node-pre-gyp ERR! node-pre-gyp -v v0.11.0
node-pre-gyp ERR! not ok 
Failed to execute '/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/bin/node /export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js configure --fallback-to-build --module=/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/canvas/build/Release/canvas.node --module_name=canvas --module_path=/export/home/cgilles/.user_conda/miniconda/envs/tests_altair/lib/node_modules/canvas/build/Release --napi_version=6 --node_abi_napi=napi --napi_build_version=0 --node_napi_label=node-v83' (1)
npm WARN [email protected] requires a peer of vega@^5.17.0 but none is installed. You must install peer dependencies yourself.

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] install: `node-pre-gyp install --fallback-to-build`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /export/home/cgilles/.npm/_logs/2020-10-28T17_18_30_189Z-debug.log

I don't understand this kind of error could you please tell me what is the issue ?
Thank you so much.

@jakevdp
Copy link
Member

jakevdp commented Oct 28, 2020

It appears this is the key clue:

npm WARN [email protected] requires a peer of vega@^5.17.0 but none is installed. You must install peer dependencies yourself.

You need to also install the vega library as part of your npm command.

@ColineG
Copy link

ColineG commented Oct 30, 2020

Ok I finally succeed in saving a plot in .png here what I did if that can help. My problem was in the installation of canvas lib so for that :
Add dependencies Ubuntu :

$ sudo apt install libpixman-1-dev libcairo2-dev libpango1.0-dev libjpeg8-dev libgif-dev

Automattic/node-canvas#1065 (comment)

Dependencies for Centos :

$ sudo yum install gcc-c++ cairo-devel libjpeg-turbo-devel pango-devel giflib-devel

https://github.com/Automattic/node-canvas/wiki/Installation%3A-Fedora-and-other-RPM-based-distributions

Creating a default package.json file at the root of the project if you have this error :

npm WARN saveError ENOENT: no such file or directory, open '/export/home/cgilles/scripts_projets/metrologie_send_by_mail/package.json'
npm WARN enoent ENOENT: no such file or directory, open '/export/home/cgilles/scripts_projets/metrologie_send_by_mail/package.json'

Then run :

$ npm init --yes

https://docs.npmjs.com/creating-a-package-json-file

Then run :

$ npm install vega
$ npm install vega-lite
$ npm install vega-embed

Then try saving your plot it should work.

@prosaic-toast
Copy link

By removing manually the broken node-modules directories and the symlinks in conda_environment/bin directory and using npm from that environment and doing:

npm install -g vega-cli vega-lite canvas

The command

echo '{"data": {"url": "https://vega.github.io/vega-datasets/data/cars.json"}, "mark": "point"}' | vl2vg | vg2pdf > out.pdf 

works perfectly.

I will open a PR to both packages on conda forge to fix the issue.

As a datapoint: I just ran into this problem with altair_saver (TypeError: Cannot read property 'getContext' of null at resize (/data/conda/lib/vega-cli/node_modules/vega-scenegraph/build/vega-scenegraph.js:3377:26)) after recently updating some conda packages. I don't understand npm, but your solution worked for me, so thanks! All I needed to remove was the conda/lib/vega-cli/node_modules directory, and do npm install -g vega-cli vega-lite canvas.

However, this may mean that whatever your vega-cli PR fixed has now been broken again... :(

Relevant conda packages:

altair                    4.1.0                      py_1    conda-forge
altair_saver              0.5.0                      py_0    conda-forge
vega-cli                  5.6.0                         0    conda-forge
vega-lite-cli             4.17.0               ha770c72_0    conda-forge

@michelledleblanc
Copy link

In case it helps anyone with this same issue: I was able to determine what was wrong with my install by running the example notebook (https://github.com/altair-viz/altair_saver/blob/master/AltairSaver.ipynb) locally. The third code cell will install the files in whatever directory you're in and then you can compare that to the environment in which conda installed the files. Similar to what's been said above, my issue was that some modules were nested.

@vision-zhao
Copy link

maybe your canvas is not installed successful, try comand"npm install -g vega-cli --canvas_binary_host_mirror=https://npm.taobao.org/mirrors/node-canvas-prebuilt/". It helps me

@joelostblom
Copy link
Member

Since Altair 5.2, the functionality of Altair Saver is now available in Altair via the vl-convert package. Most of the functionality has been available since 5.0, and the main addition in 5.2 was PDF export. See the docs on how to save charts for more details.

We are going to archive this repo, so I'm closing all the open issues and PRs before doing so. Try out the new options for saving charts mentioned above and if you run into issues, please open an issue directly in the altair or vl-convert repo.

@joelostblom joelostblom closed this as not planned Won't fix, can't repro, duplicate, stale Mar 30, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests