plotting a column name with '.' in it eventually leads to a vega error

[jowens]

I'm doing an Altair chart with a color given by color=Color('gpuinfo.name:N', .... This produces an eventual Vega error.

(Why do I have a column with a period in its name? Reading nested JSONs into pandas is nicely handled by json_normalize, which flattens nested JSON levels into dot-separated column names.)

undefined:3
return x["gpuinfo"]["name"];
                   ^

TypeError: Cannot read property 'name' of undefined
    at Object.eval [as get] (eval at <anonymous> (/Users/jowens/Documents/working/vega/node_modules/datalib/src/util.js:147:20), <anonymous>:3:20)
    at Facetor.proto._cellkey (/Users/jowens/Documents/working/vega/node_modules/datalib/src/aggregate/aggregator.js:106:23)
    at Facetor.proto._cell (/Users/jowens/Documents/working/vega/node_modules/datalib/src/aggregate/aggregator.js:114:38)
    at Facetor.proto._add (/Users/jowens/Documents/working/vega/node_modules/datalib/src/aggregate/aggregator.js:151:19)
    at add (/Users/jowens/Documents/working/vega/src/transforms/Aggregate.js:174:31)
    at Array.forEach (native)
    at Aggregate.prototype.transform (/Users/jowens/Documents/working/vega/src/transforms/Aggregate.js:190:13)
    at Aggregate.prototype.evaluate (/Users/jowens/Documents/working/vega/src/transforms/Transform.js:48:15)
    at Node.dataRef (/Users/jowens/Documents/working/vega/src/scene/Scale.js:387:15)
    at Node.ordinal (/Users/jowens/Documents/working/vega/src/scene/Scale.js:127:20)

[jakevdp]

If open an issue in Vega-Lite. Altair has no control over these sorts of details.

[jowens]

No prob. On it.

[sotte]

Just for completeness sake here are the related vega-lite issues:

• plotting a column name with '.' in it eventually leads to a vega error vega-lite#1775
• Columns not working properly in VL spec (possibly period in name of column?) vega-lite#1949

I think the relevant part from the first issue is:
"We have fixed this in Vega-Lite 2/ Vega 3 but since Altair is using the stable VL 1 and VG 2, these fixes may not work." - vega/vega-lite#1775 (comment)

[jakevdp]

Altair currently (v2.X) is built on Vega-Lite 2 and Vega 3, so those fixes should work.

[sotte]

I'm afraid that is not the case. Columns with a dot in it are not treated properly.

Code to reproduce:

import altair as alt
import pandas as pd
import numpy as np

df = pd.DataFrame({
    "x.x": np.arange(10),
    "x_x": np.arange(10),
})

# what you expect, a simple line
alt.Chart(df).mark_line().encode(x="x_x", y="x_x")
# this is an empty plot
alt.Chart(df).mark_line().encode(x="x.x", y="x.x")

That being said, I have the js warning:
The compiled spec uses Vega v3, but current version is 4.4.0.

[jakevdp]

This is expected: x.x is one way Vega-Lite uses to refer to hierarchical data. If you want to use special characters in field names, they should be escaped. e.g.

alt.Chart(df).mark_line().encode(x=r"x\.x:Q", y="x\.x:Q")

This is true regardless of the Altair/Vega-Lite version you are using.

For more information, see field under Encoding Channel Options

[sotte]

I was aware of that. Thanks. (I assume y should also be an r-string?)

I tried escaping the field names with the dot but it still produces an empty chart.

import altair as alt
import pandas as pd
import numpy as np

df = pd.DataFrame({
    "x.x": np.arange(10),
    "x_x": np.arange(10),
})
(
    alt.Chart(df).mark_line().encode(x=r"x\.x:Q", y=r"x\.x:Q") |
    alt.Chart(df).mark_line().encode(x="x_x:Q", y="x_x:Q")
)

[jakevdp]

That code works for me.

• What version of Altair are you using?
• What frontend version are you using? If Jupyterlab, what version? If Jupyter notebook, what version of the vega package do you have installed?
• Are there any errors in the javascript console?

[sotte]

I installed everything via conda.

➤ jupyter lab --version                                                                                                                        (lola) 
0.35.4

>>> import altair as alt
>>> print(alt.__version__)
2.3.0
>>>import vega
>>> print(vega.__version__)
1.4.0

If I'm not mistaken this is too old. I'll try with a clean conda env and get back to you.

[sotte]

Wow, this is freaky. New conda env, same version of jupyter lab, altair, and vega. Now it works. Sorry for the noise! Thanks @jakevdp and great work with altair!

Does altair offer an escape function that turns "x.x" into "x\\.x"? I can write one myself, I'm just wondering because it's quite common (at least for me) to have dots in the column name.

[jakevdp]

Great, glad you got it working.

Regarding auto-escaping, there's no functionality built-in for that... it would be a useful feature to have.