gensim package import issues

[rsurendraidexcel]

Why pakage import issue

import gensim

[piskvorky]

@mpenkov what's the plan on that next release? Or else we will keep receiving these tickets over and over :)

[onedavidwilliams]

I have the same issue.

Edit:
A quick fix is this:

pip install scipy==1.10.1 --target=Your/Version/Directory/Packages/scipy-1.10.1

import sys
sys.path.insert(0, os.path.join(os.getcwd(), "Your/Version/Directory/Packages/scipy-1.10.1"))

make sure this is above where you from gensim import corpora, models or whatever your imports are.

The issue arises because scipy.triu_indices was depreciated in version 1.11 and removed in 1.12, you can now use numpy.triu_indices.

Note:
Would love to lend a helping hand if one is needed. Thank you for taking care of this repo!

[Crosswind]

I believe this is a duplicate to #3525 where the same import produced an error. It was then fixed in #3524 and the code was merged ;-)

gensim/gensim/matutils.py

Lines 23 to 26 in dc5b5c4

	try:
	from numpy import triu
	except ImportError:
	from scipy.linalg import triu

@piskvorky you were obviously involved. I am just pointing it out again because I feel like this is one of the 4 issues currently blocking the release and we also have huge interest in getting this fix in as we currently have our scipy version pinned to below 1.13.

[piskvorky]

Yes, you are right. We've had probably a dozen tickets for this same issue. And a few PRs too. The issue was solved a while ago => merged into master => pending a PyPI release.

@mpenkov how about another call to push the release through? Is there anything blocking it?

[lolpro11]

For those on Arch, the same issue is occurring with the AUR package. Please push an updated package to the AUR as well.
Thanks.

[albas99]

From the discussion above, it seems that this issue has been fixed and is awaiting release from PyPI, I don't know if there's a release window that we'll have to wait for but this is making gensim pretty much unusable with the latest version of scipy.

[Crosswind]

@albas99 the release window is specified by the maintainers of this repository. It is up to them to decide when a new version is released and pushed to PyPi. This has nothing to do with PyPi itself.

For my projects, we don't depend on scipy directly so we just pinned it to <1.14.0 to avoid the mentioned problems.
Another workaround is to install scipy from source which will get you the mentioned change.

I suggest this issue is closed and we let @piskvorky and @mpenkov do what's necessary to get us a new version which fixes the mentioned problem.

[mpenkov]

The blocking issue is failing wheel builds: https://github.com/piskvorky/gensim/actions/workflows/build-wheels.yml

I'm not sure what the problem is there. I've been looking at it occasionally, as time permits, but haven't pinned the cause down yet. Very little things have changed on our end, so it's possible it's a CI issue or another incompatibility between our direct/indirect dependencies.

[mpenkov]

@piskvorky We need to get the wheels building again, otherwise we simply cannot release. We could jump on a call, but there isn't much to discuss or plan: once the wheels are building, we can pull the trigger and release relatively quickly.

My intent is to continue chipping away at the problem until I work out what the cause is. It's hard to give an estimate of when that will be, both because we don't yet know what exactly the problem is, and because my personal resources are limited.

To everyone waiting on the release, I'm sorry that it's taking so long. If you're able to shed light on why the wheel builds are failing, then that would be very helpful.

[Crosswind]

I have just created a PR with a suggested fix that should make the tests green again. Hope that helps! (Waiting for an approval for the pipeline to run as I couldn't test this locally.)

[mpenkov]

Thank you @Crosswind.

I have been working on the same issue today via #3538 .

There are two problems here:

1. scipy 1.14.0 removed sparsetools, as you have mentioned
2. github CI deprecated the macos-11 runner, so we have to use a newer runner to build the macos wheels

I'm waiting for the CI to complete for #3538 - I expect the above two problems to be fixed now.

[Crosswind]

@mpenkov Apologies for missing your latest commits in #3538. Anyways, CI is looking much better now. Fingers crossed this'll go through now.

[mpenkov]

Fixed in #3538

[mpenkov]

https://pypi.org/project/gensim/4.3.3/

If you're able to confirm that the problem is fixed, please let me know.