Getting output error cause sklearn

[yochiokta]

Hi have error issue in the sklearn module, I also already check in my coastat environment this module has installed but it is still strange keep saying that there are No module named 'sklearn.neural_network.multilayer_perceptron'

System:
python: 3.8.16 (default, Jun 12 2023, 12:55:15) [Clang 14.0.6 ]
executable: /Users/yochi/opt/anaconda3/envs/coastsat/bin/python
machine: macOS-10.16-x86_64-i386-64bit

Python dependencies:
sklearn: 1.2.2
pip: 23.1.2
setuptools: 67.8.0
numpy: 1.24.3
scipy: 1.9.1
Cython: None
pandas: 2.0.2
matplotlib: 3.7.1
joblib: 1.2.0
threadpoolctl: 3.1.0

Built with OpenMP: True

Mapping shorelines:

ModuleNotFoundError Traceback (most recent call last)
Cell In[26], line 2
1 get_ipython().run_line_magic('matplotlib', 'qt')
----> 2 output = SDS_islands.extract_sand_poly(metadata, settings)

File ~/Documents/CoastSat-master/coastsat/SDS_islands.py:129, in extract_sand_poly(metadata, settings)
127 elif satname == 'S2':
128 pixel_size = 10
--> 129 clf = joblib.load(os.path.join(filepath_models, 'NN_4classes_S2.pkl'))
131 # convert settings['min_beach_area'] and settings['buffer_size'] from metres to pixels
132 buffer_size_pixels = np.ceil(settings['buffer_size']/pixel_size)

File ~/opt/anaconda3/envs/coastsat/lib/python3.8/site-packages/joblib/numpy_pickle.py:658, in load(filename, mmap_mode)
652 if isinstance(fobj, str):
653 # if the returned file object is a string, this means we
654 # try to load a pickle file generated with an version of
655 # Joblib so we load it with joblib compatibility function.
656 return load_compatibility(fobj)
--> 658 obj = _unpickle(fobj, filename, mmap_mode)
659 return obj

File ~/opt/anaconda3/envs/coastsat/lib/python3.8/site-packages/joblib/numpy_pickle.py:577, in _unpickle(fobj, filename, mmap_mode)
575 obj = None
576 try:
--> 577 obj = unpickler.load()
578 if unpickler.compat_mode:
579 warnings.warn("The file '%s' has been generated with a "
580 "joblib version less than 0.10. "
581 "Please regenerate this pickle file."
582 % filename,
583 DeprecationWarning, stacklevel=3)

File ~/opt/anaconda3/envs/coastsat/lib/python3.8/pickle.py:1212, in _Unpickler.load(self)
1210 raise EOFError
1211 assert isinstance(key, bytes_types)
-> 1212 dispatchkey[0]
1213 except _Stop as stopinst:
1214 return stopinst.value

File ~/opt/anaconda3/envs/coastsat/lib/python3.8/pickle.py:1528, in _Unpickler.load_global(self)
1526 module = self.readline()[:-1].decode("utf-8")
1527 name = self.readline()[:-1].decode("utf-8")
-> 1528 klass = self.find_class(module, name)
1529 self.append(klass)

File ~/opt/anaconda3/envs/coastsat/lib/python3.8/pickle.py:1579, in _Unpickler.find_class(self, module, name)
1577 elif module in _compat_pickle.IMPORT_MAPPING:
1578 module = _compat_pickle.IMPORT_MAPPING[module]
-> 1579 import(module, level=0)
1580 if self.proto >= 4:
1581 return _getattribute(sys.modules[module], name)[0]

ModuleNotFoundError: No module named 'sklearn.neural_network.multilayer_perceptron'

[mcuttler]

Hi - I believe this error is related to how the code is trying to open/read in the neural network classifiers that has changed with newer versions of sklearn.

I am trying to debug but am finding more issues with the underlying code than I expected - mainly due to the fact that I don't actively try to keep this up to date with new updates to the main coastsat branch. I will do my best, but this is looking like it might take significant time to debug and update.

In the interim, you might try to use the main coastsat package and see about then taking the Lines from SDS_islands (166 -265) that do the contouring and calculation of island attributes.

[kvos]

hi @mcuttler , I had the same error at a a certain point with coastsat and I needed to remake the .pkl files with the classifiers. there's two ways, one way if you have the training data, you update your version of scikit-learn then re-train and store, or there is another way which you can convert directly to the new format, Chris Leaman did it for me, maybe you can ask him. You can look in coastsat, I have two files for each classifier and then in the code I check which version of scikit-learn the user has and load the right file.

[8painkiller8]

hi @yochiokta have you found a solution yet? I am receiving the same error when running :
output = SDS_islands.extract_sand_poly(metadata, settings)

with an error :

ModuleNotFoundError: No module named 'sklearn.neural_network.multilayer_perceptron'

[mcuttler]

hi @8painkiller8 - i will try to get back to fixing this bug in the next few weeks. you might try the suggestion by @kvos above if you're actively working with the repository? otherwise, i should have time to look into this and update repository by the end of the year

[8painkiller8]

Thank you for your prompt response. I followed the suggestion provided by @kvos and updated my version of scikit-learn to resolve the issue. However, I would like clarification on the recommendation to "re-train and store" if possible. As a newcomer to Python, I am not fully clear on the process involved.

In the interim, I will make concerted efforts to find a solution to the issue. I look forward to the anticipated update of the repository and appreciate your ongoing support.

[8painkiller8]

Hi @mcuttler,

I hope you're doing well. I wanted to follow up on your previous message regarding the bug fix you were planning to address in the CoastSat repository. You mentioned that you might have time to look into this and update the repository by the end of the year.

As we're approaching that time, I was wondering if there have been any developments on this front? Your insights and updates are invaluable, especially for those of us actively using the repository. If you've had the chance to address this issue, could you kindly share any updates or modifications that have been made?

Also, if there's anything the community can assist with, please feel free to let us know. Your work is greatly appreciated, and we're eager to continue utilizing the repository with the latest improvements.

Thank you for your dedication and support.

[mcuttler]

Hi @8painkiller8, I have had no time to work on this and don't expect to in the coming weeks.

It looks like you said you were able to update your version of scikit-learn? Did that resolve your issues?

There's an example of how to re-train the classifier in the main CoastSat repository here: https://github.com/kvos/CoastSat/tree/master/classification

[8painkiller8]

Hi @mcuttler,

No worries about the timing — I completely understand that other commitments can take precedence. Hopefully, there'll be an opportunity for you to revisit this work in the near future.

I must apologize for any confusion in my previous messages. I haven't been successful in updating my version of scikit-learn as I initially thought. I've reached out to Dr. Chris, who has been kind enough to share some valuable insights and potential solutions for my situation. I'm currently exploring them.

Thank you for your patience and assistance.

[kvos]

yes if you managed to update your pickle models with the new scikit you're all good. If you train a new model just make sure you're using the latest scikit.