Windows installation guide produces failed compilation Error: invalid register for .seh_savexmm\r. '

[davidsteinar]

Description of your problem

By following the Windows installation guide and the getting started notebook I get the following error when calling map_estimate = pm.find_MAP(model=basic_model)

Please provide a minimal, self-contained, and reproducible example.

conda create -c msys2 -c conda-forge -n mypm3env python=3.8 mkl-service libpython m2w64-toolchain scipy matplotlib pandas -y
conda activate mypm3env
conda install -c conda-forge pymc3 -y

python

>>> import numpy as np
>>> import pymc3 as pm
>>> 
>>> RANDOM_SEED = 8927
>>> np.random.seed(RANDOM_SEED)
>>> 
>>> # True parameter values
>>> alpha, sigma = 1, 1
>>> beta = [1, 2.5]
>>> 
>>> # Size of dataset
>>> size = 100
>>> 
>>> # Predictor variable
>>> X1 = np.random.randn(size)
>>> X2 = np.random.randn(size) * 0.2
>>> 
>>> # Simulate outcome variable
>>> Y = alpha + beta[0] * X1 + beta[1] * X2 + np.random.randn(size) * sigma
>>> 
>>> print(f"Running on PyMC3 v{pm.__version__}")
>>> 
>>> basic_model = pm.Model()
>>> with basic_model:
>>> 
>>>     # Priors for unknown model parameters
>>>     alpha = pm.Normal("alpha", mu=0, sigma=10)
>>>     beta = pm.Normal("beta", mu=0, sigma=10, shape=2)
>>>     sigma = pm.HalfNormal("sigma", sigma=1)
>>> 
>>>     # Expected value of outcome
>>>     mu = alpha + beta[0] * X1 + beta[1] * X2
>>> 
>>>     # Likelihood (sampling distribution) of observations
>>>     Y_obs = pm.Normal("Y_obs", mu=mu, sigma=sigma, observed=Y)
>>> 
>>> map_estimate = pm.find_MAP(model=basic_model)

Please provide the full traceback.

Traceback (most recent call last):
  File ".\pipelines\mc3.py", line 37, in <module>
    map_estimate = pm.find_MAP(model=basic_model)
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\pymc3\tuning\starting.py", line 114, in find_MAP
    dlogp_func = bij.mapf(model.fastdlogp_nojac(vars))
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\pymc3\model.py", line 462, in fastdlogp_nojac
    return self.model.fastfn(gradient(self.logp_nojact, vars))
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\pymc3\theanof.py", line 131, in gradient
    return tt.concatenate([gradient1(f, v) for v in vars], axis=0)
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\pymc3\theanof.py", line 131, in <listcomp>
    return tt.concatenate([gradient1(f, v) for v in vars], axis=0)
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\pymc3\theanof.py", line 120, in gradient1
    return tt.flatten(tt.grad(f, v, disconnected_inputs="warn"))
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\gradient.py", line 639, in grad
    rval = _populate_grad_dict(var_to_app_to_idx, grad_dict, wrt, cost_name)
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\gradient.py", line 1440, in _populate_grad_dict
    rval = [access_grad_cache(elem) for elem in wrt]
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\gradient.py", line 1440, in <listcomp>
    rval = [access_grad_cache(elem) for elem in wrt]
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\gradient.py", line 1393, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\gradient.py", line 1061, in access_term_cache
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\gradient.py", line 1061, in <listcomp>
    output_grads = [access_grad_cache(var) for var in node.outputs]
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\gradient.py", line 1393, in access_grad_cache
    term = access_term_cache(node)[idx]
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\gradient.py", line 1220, in access_term_cache
    input_grads = node.op.L_op(inputs, node.outputs, new_output_grads)
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\tensor\elemwise.py", line 564, in L_op
    rval = self._bgrad(inputs, outs, ograds)
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\tensor\elemwise.py", line 668, in _bgrad
    ret.append(transform(scalar_igrad))
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\tensor\elemwise.py", line 659, in transform
    new_r = Elemwise(node.op, {})(*[transform(ipt) for ipt in node.inputs])
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\tensor\elemwise.py", line 659, in <listcomp>
    new_r = Elemwise(node.op, {})(*[transform(ipt) for ipt in node.inputs])
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\tensor\elemwise.py", line 659, in transform
    new_r = Elemwise(node.op, {})(*[transform(ipt) for ipt in node.inputs])
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\graph\op.py", line 253, in __call__
    compute_test_value(node)
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\graph\op.py", line 126, in compute_test_value
    thunk = node.op.make_thunk(node, storage_map, compute_map, no_recycling=[])
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\graph\op.py", line 634, in make_thunk
    return self.make_c_thunk(node, storage_map, compute_map, no_recycling)
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\graph\op.py", line 600, in make_c_thunk
    outputs = cl.make_thunk(
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\link\c\basic.py", line 1203, in make_thunk
    cthunk, module, in_storage, out_storage, error_storage = self.__compile__(
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\link\c\basic.py", line 1138, in __compile__
    thunk, module = self.cthunk_factory(
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\link\c\basic.py", line 1634, in cthunk_factory
    module = get_module_cache().module_from_key(key=key, lnk=self)
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\link\c\cmodule.py", line 1191, in module_from_key
    module = lnk.compile_cmodule(location)
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\link\c\basic.py", line 1543, in compile_cmodule
    module = c_compiler.compile_str(
  File "C:\Users\dsa\Anaconda3\envs\mypm3env\lib\site-packages\theano\link\c\cmodule.py", line 2546, in compile_str
    raise Exception(
Exception: ('Compilation failed (return status=1): C:\\Users\\dsa\\AppData\\Local\\Temp\\ccABu3X0.s: Assembler messages:\r. C:\\Users\\dsa\\AppData\\Local\\Temp\\ccABu3X0.s:101: Error: invalid register for .seh_savexmm\r. ', 'FunctionGraph(Elemwise{EQ}(<TensorType(float64, vector)>, <TensorType(int8, (True,))>))')

Please provide any additional information below.

Are there any other requirements that are missing in the windows installation guide that are missing?

Versions and main components

• PyMC3 Version: v3.11.2
• Aesara Version: -
• Python Version: 3.8.10
• Operating system: Windows 10
• How did you install PyMC3: conda

[davidsteinar]

@twiecki I got a working Windows conda env for pymc3.

First by installing MinGW and following these installation instructions, setting minwg bin folder to my Path environment variable.

Then I did the following:

git clone https://github.com/pymc-devs/pymc3.git
cd pymc3
cd conda-envs
conda create env -f environment-dev-py38.yml 
conda activate environment-dev-py38
conda install -c conda-forge pymc3

Cygwin was producing errors in the compilation step.

[twiecki]

Is that different from the windows installation instructions? Should we update them? CC @michaelosthege

[michaelosthege]

The Windows job in the CI pipelines for v3 also started failing recently. It's blocking PRs such as #4745 and #4741.
If you can pinpoint which dependency is incompatible and mark that in the requirements and environment YAML files that would be best!
We should not make everybody rely on our installation instructions, but instead anti-pin the I compatible dependencies so the pip/conda package managers take care of it.

For v4 the Windows job is still disabled, by the way 😬

[davidsteinar]

@twiecki Yes the steps I posted are different from the windows installation instruction. The current windows installation instruction also does not mention the requirement of MinGW.

@michaelosthege The compilation error stems from lib\site-packages\theano\. I am not sure how I can help further, but both environments are using theano-pymc 1.1.2 py38h885f38d_0 conda-forge

[michaelosthege]

I just confirmed that after installing with pip I don't get any problems.
After pip uninstall pymc3 followed by conda install -c conda-forge pymc3 it also runs fine.

However

• The conda install command produced some error messages that don't look healthy.
• I also got the WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions. in both cases.

FWIW I don't trust conda-forge when it comes to PyMC3.

[michaelosthege]

I do have Visual Studio installed, so maybe that gives me a headstart when it comes to compilation stuff.

[twiecki]

@michaelosthege maybe mingw is the safer option?

[davidsteinar]

When I do

conda create env -f environment-dev-py38.yml 
conda activate environment-dev-py38
pip install pymc3

I do not get any compilation error, so that works too.

However I always get the WARNING (theano.tensor.blas): Using NumPy C-API based implementation for BLAS functions. in any environment.

[davidsteinar]

I also have Visual Studio installed, but not with any specific C++ components since I do not work with C++

[michaelosthege]

Our install instructions include m2w64-toolchain. Correct me if I'm wrong, but that is MinGW, right?

We should specify OS-dependent dependencies in our requirements/setup.py/conda recipes.
I have no idea how, but there's probably a way.

[davidsteinar]

I am not really familiar with these compilation methods, I just wanted to share what I did to create a working environment. I am not sure if the windows installation instructions need to be updated or more drastic steps are needed to fix this issue but I am not the first to get this error 1, 2

[michaelosthege]

Okay, getting the BLAS stuff installed properly is really hard..

The sunode installation steps for Windows worked for me.
Apparently sunode is already doing a better job with the dependencies in its conda recipe.
Don't ask me why m2w64-toolchain mkl-service libpython are installed in a second step from the default channel. When I wrote those install instructions I tested countless combinations and orders of installation and this is what I ended up with.

On the plus side, by following the sunode instructions you'll get blazingly fast ODE solvers.

Getting Windows non-Linux install instructions right is unfortunately a never ending story.

[davidsteinar]

With the sunode installation I do not get the BLAS warning so that seems to be working, but still get the Error: invalid register for .seh_savexmm\r. ' error.

To get a working truly working environment I turned to the official docker image,

docker pull pymc/pymc3
docker run --rm -d  -p 8888:8888/tcp pymc/pymc3:latest
docker exec -it 054e90a4789ba0b562dc8837611239c208694b162fed4b0953d15071bb6bef1d bash 

(base) jovyan@054e90a4789b:~$ python
Python 3.9.2 | packaged by conda-forge | (default, Feb 21 2021, 05:02:46) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import pymc3
You are running the v4 development version of PyMC3 which currently still lacks key features. You probably want to use the stable v3 instead which you can either install via conda or find on the v3 GitHub branch: https://github.com/pymc-devs/pymc3/tree/v3
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/jovyan/pymc3/__init__.py", line 43, in <module>
    __set_compiler_flags()
  File "/home/jovyan/pymc3/__init__.py", line 37, in __set_compiler_flags
    import aesara
ModuleNotFoundError: No module named 'aesara'

but that also produces errors ......

[michaelosthege]

The latest Docker image builds from master/main and as you can see we're not testing it much..

But try the following. This just worked for me and this time I confirmed with the getting started notebook too. (Not sure if I did that for my last comment last night.)
These are install instructions we recently wrote for the SI of a paper.. So I sincerely hope they are reproducible.

environment.yml

name: pm3env
channels:
  - conda-forge
  - defaults
dependencies:
  - ipywidgets=7.6.3=pyhd3eb1b0_1
  - jupyter=1.0.0=py37_7
  - libpython=2.1=py37_0
  - matplotlib=3.3.2=haa95532_0
  - mkl-service=2.3.0=py37h196d8e1_0
  - numba=0.53.1=py37h4e635f9_0
  - numpy=1.19.2=py37hadc3359_0
  - pandas=1.2.1=py37hf11a4ad_0
  - pip=20.3.3=py37haa95532_0
  - python=3.7.9=h60c2a47_0
  - python-graphviz==0.16=pyh243d235_2
  - scipy=1.5.2=py37h9439919_0
  - sympy=1.7.1=py37h03978a9_1
  - pip:
    - arviz==0.11.1
    - fastprogress==1.0.0
    - h5py==3.1.0
    - openpyxl==3.0.7
    - pymc3==3.11.2
    - typing==3.7.4.3
    - watermark==2.2.0
    - xarray==0.16.2

conda create -f environment.yml
conda activate pm3env
conda install m2w64-toolchain

[davidsteinar]

When I do

conda create -f environment.yml
conda activate pm3env
conda install m2w64-toolchain

I get the same error Error: invalid register for .seh_savexmm\r. '
However, when do

conda create -f environment.yml
conda activate pm3env

I get a working env, with BLAS enabled and no compilation error.

[michaelosthege]

Not sure if that's good or bad.
Congrats to your working environment 👍

I added a warning to the wiki page and also linked to the comments above.

[michaelosthege]

I just updated the installation guide with a minimal version.
It turned out that numba and scipy should be in the dependencies at the same time. Either one individually is not enough..

[edt-andrew]

Getting this error on Windows 10 after following the guide exactly. Have tried many suggestions to fix this problem and even completely uninstalled Anaconda and re-installed to start from scratch. Any help appreciated! Let me know if any more information would be useful in resolving this problem.

Exception: ('Compilation failed (return status=1): C:\\Users\\lswad\\AppData\\Local\\Temp\\ccPHLGwt.s: Assembler messages:\r. C:\\Users\\lswad\\AppData\\Local\\Temp\\ccPHLGwt.s:2324: Error: invalid register for .seh_savexmm\r. ', 'FunctionGraph(Elemwise{EQ}(<TensorType(float64, row)>, <TensorType(int8, (True, True))>))')

[michaelosthege]

@edt-andrew I just tried with the conda create commands but couldn't replicate the problem.

I'm on Windows 10 with miniconda. Do you maybe have Cygwin or something like that installed? A quick Google search returned some results mentioning it. I don't have Cygwin on my system. Maybe that's the difference?

[edt-andrew]

I installed MinGW after seeing it suggested as a solution but didn't have that initially installed (error is the same). I don't have Visual Studio installed if that's relevant at all? Is there a parameter I can pass to pymc3 to force it to not use the compiler, even if just to check the problem is confined to compiling issues alone?

[michaelosthege]

You can set the Theano compile mode to py. That should be possible via an environment variable, but I don't remember which one.
Alternatively try theano.config.mode = theano.compile.Mode("py") before importing PyMC3.

[edt-andrew]

Imports looked like this:

import pandas as pd
import numpy as np
import theano 
theano.config.mode = theano.compile.Mode("py")
import pymc3 as pm

Same error as before...

I appreciated the quick responses! Am I to understand these issues seem to be mostly confined to windows?

[michaelosthege]

This problem in particular may be Windows specific, but more in the sense that there is something wrong with the compilation dependencies that arises from an unlucky combination of packages.. The dependency tree looks different by platform..
Do you depend on PyMC3 version 3, or could you already use the new v4? We're close to a release and because it's switching the backend you might get lucky.
Otherwise Docker or WSL are fallback options. I know it is far from ideal, but depending on what you want to do, they might bridge you to the fun parts faster.

[edt-andrew]

Can you post a link to where I can get v4? https://github.com/pymc-devs/pymc4 this url says that project has ceased?

[michaelosthege]

Can you post a link to where I can get v4? https://github.com/pymc-devs/pymc4 this url says that project has ceased?

We're referring to v4 as the next release that will originate from the main branch of https://github.com/pymc-devs/pymc3

[edt-andrew]

Hi again - that seems to have solved my compilation issues. However I seem to have a problem with DensityDist where I'm getting the below error. I note that the code example in the source code also breaks with this error (if fails on normal_dist.logp needing replaced with pm.Normal.logp first but subsequently fails with the same error). Am I correct in assuming this part of the codebase is just unfinished for the new version? Would commenting out the references to cls.rv_op.ndim_supp suffice as a quick fix? If this isn't the right place to be asking this question please to point me in the right direction! Thanks again for all your time.

---> 15     ll = pm.DensityDist('ll', logp, observed={'d': dv})
     16     trace = pm.sample(1500, cores=-1, step = pm.NUTS())
     17     trace = trace[250:]

c:\repos\pymc3\pymc3\distributions\distribution.py in __new__(cls, name, rng, dims, initval, observed, total_size, transform, *args, **kwargs)
    205         # Create the RV without specifying initval, because the initval may have a shape
    206         # that only matches after replicating with a size implied by dims (see below).
--> 207         rv_out = cls.dist(*args, rng=rng, initval=None, **kwargs)
    208         ndim_actual = rv_out.ndim
    209         resize_shape = None

c:\repos\pymc3\pymc3\distributions\distribution.py in dist(cls, dist_params, shape, size, initval, **kwargs)
    289 
    290         create_size, ndim_expected, ndim_batch, ndim_supp = find_size(
--> 291             shape=shape, size=size, ndim_supp=cls.rv_op.ndim_supp
    292         )
    293         # Create the RV with a `size` right away.

AttributeError: 'NoneType' object has no attribute 'ndim_supp'

[twiecki]

Is this on main? DensityDist hasn't been ported yet.

[michaelosthege]

DensityDist is one of the few distributions that are not yet refactored.
Also see #4831.

I am not familiar with DensityDist and can't tell how complicated the refactor is.

[edt-andrew]

Thanks guys this is all very helpful to know. Is there a vague estimate on when those distributions will be refactored, or when the v4 release is planned?

[mick-phemex]

Hi again - that seems to have solved my compilation issues. However I seem to have a problem with DensityDist where I'm getting the below error. I note that the code example in the source code also breaks with this error (if fails on normal_dist.logp needing replaced with pm.Normal.logp first but subsequently fails with the same error). Am I correct in assuming this part of the codebase is just unfinished for the new version? Would commenting out the references to cls.rv_op.ndim_supp suffice as a quick fix? If this isn't the right place to be asking this question please to point me in the right direction! Thanks again for all your time.

---> 15     ll = pm.DensityDist('ll', logp, observed={'d': dv})
     16     trace = pm.sample(1500, cores=-1, step = pm.NUTS())
     17     trace = trace[250:]

c:\repos\pymc3\pymc3\distributions\distribution.py in __new__(cls, name, rng, dims, initval, observed, total_size, transform, *args, **kwargs)
    205         # Create the RV without specifying initval, because the initval may have a shape
    206         # that only matches after replicating with a size implied by dims (see below).
--> 207         rv_out = cls.dist(*args, rng=rng, initval=None, **kwargs)
    208         ndim_actual = rv_out.ndim
    209         resize_shape = None

c:\repos\pymc3\pymc3\distributions\distribution.py in dist(cls, dist_params, shape, size, initval, **kwargs)
    289 
    290         create_size, ndim_expected, ndim_batch, ndim_supp = find_size(
--> 291             shape=shape, size=size, ndim_supp=cls.rv_op.ndim_supp
    292         )
    293         # Create the RV with a `size` right away.

AttributeError: 'NoneType' object has no attribute 'ndim_supp'

Hi, so did you solve the problem by installing the pymc4 instead of pymc3? I also met the same problem as you, and I followed up the installation guide from here, but the problem still existed.

[mick-phemex]

conda install m2w64-toolchain

I followed up the same process, but I still got some errors, could you please help me solve this problem? #4922

[gourdchen]

I have the similar problem, I think maybe I have solved it.
I try to find the code that compile the network in theano, I find it is in /site-packges/theano/gof/cmodule.py.
I guess that the cxxflags is the parameters that will be passed to gcc, I add

cxxflags.append('-fno-asynchronous-unwind-tables')

in line 2191. Maybe you can try my way.

[sreedat]

@gourdchen I tried to follow you what suggested. For the cmodule.py was in a different location. When I added the cxxflags code, that began to pick up my RTools mingw (msys-based) compiler tools. RTools uses gcc version 8.3 but is incompatible with whatever Theano-pymc needs. Back to the drawing board for me.

@mick-phemex same installation issues as you and @michaelosthege directed me to the discussion you are having with him. So I'm trying to see if my Pymc3 issues go away if Anaconda mingw compilers are set-up correctly in the User environment variables per recommendation by @michaelosthege at Pymc3 Discourse web site.

If I find a solution, I will post it here.

Sree

[sreedat]

@gourdchen @mick-phemex Following the User Path variables information that @michaelosthege had posted in his conversation with you @mick-phemex I was able to overcome two issues I was dealing with:

1. Anaconda accessing mingw compilers from RTools installation. So this meant pymc3 worked for some models and not others.
2. Another package, bambi, a really nice wrapper for pymc3, was failing even though pymc3 appeared to be working. Bambi allows non-programmers to specify Bayesian multi-level / hierarchical models using the formula specification. (as R does, bambi replicates the ease of use of a R package called brms, which is a wrapper for rstan package, first package to implement the HMC-NUTS sampler developed by Matthew Hoffman and Andrew Gelman).

Both of these issues had the same underlying cause: Anaconda packages were accessing the incorrect mingw compilers. Now that is resolved and all of my models are working.