Fix future/deprecation warning

[EwoutH]

Fix all of the future/deprecations warnings currently present:

Done (#202, #211):

• Fix MarkerStyle(None) deprecation in matplotlib by replacing it with MarkerStyle('')
• Replace deprecated .iteritems() with .items() for dicts
• Fix Matplotlib deprecation of loc as a positional keyword in legend functions
• Import launcher from ipyparallel.cluster instead of ipyparallel.apps
• EMAworkbench/ema_workbench/em_framework/salib_samplers.py:137: DeprecationWarning: salib.sample.saltelli will be removed in SALib 1.5. Please use salib.sample.sobol
• sklearn/cluster/_agglomerative.py:983: FutureWarning: Attribute affinity was deprecated in version 1.2 and will be removed in 1.4. Use metric instead (clusterer: Update AgglomerativeClustering keyword to fix deprecation #218)
• DeprecationWarning: zmq.eventloop.ioloop is deprecated in pyzmq 17. pyzmq now works with default tornado and asyncio eventloops. (Replace deprecated zmq.eventloop.ioloop with Tornado's ioloop #334)

No action needed
The new default behaviour will be the desired behaviour:

• EMAworkbench/ema_workbench/analysis/parcoords.py:176: FutureWarning: In a future version, df.iloc[:, i] = newvals will attempt to set the values inplace instead of always setting a new array. To retain the old behavior, use either df[df.columns[i]] = newvals or, if columns are non-unique, df.isetitem(i, newvals)
• EMAworkbench/EMAworkbench/ema_workbench/analysis/logistic_regression.py:256: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
  self.peeling_trajectory = pd.concat(

To-do:

• EMAworkbench/ema_workbench/analysis/scenario_discovery_util.py:413: UserWarning: FixedFormatter should only be used together with FixedLocator
• Python/3.10.8/x64/lib/python3.10/site-packages/tornado/ioloop.py:265: DeprecationWarning: There is no current event loop
• Python/3.10.8/x64/lib/python3.10/site-packages/tornado/platform/asyncio.py:299: DeprecationWarning: There is no current event loop

[quaquel]

with respect to the open issues

1. the future behavior is the desired behavior, so in principle, we don't need to do anything.
2. not entirely sure what is going on here. We probably need to add a set_yticks next to set_yticklabels which should be relatively easy to do. The get_position in line 407 already returns this location, so most likely adding a loc list like the label list and appending the loc from get_position might be sufficient.

EMAworkbench/ema_workbench/analysis/scenario_discovery_util.py

Lines 402 to 413 in 60a45e7

	for row, ylabel in zip(grid.axes, grid.y_vars):
	if ylabel in cats:
	ax = row[0]
	labels = []
	for entry in ax.get_yticklabels():
	_, value = entry.get_position()
	try:
	label = categorical_mappings[ylabel][value]
	except KeyError:
	label = ""
	labels.append(label)
	ax.set_yticklabels(labels)

However, what concerns me is the difference with the code for handling the columns:

EMAworkbench/ema_workbench/analysis/scenario_discovery_util.py

Lines 415 to 429 in 60a45e7

	# do the xticklabeling for categorical columns
	for ax, xlabel in zip(grid.axes[-1], grid.x_vars):
	if xlabel in cats:
	labels = []
	locs = []
	mapping = categorical_mappings[xlabel]
	for i in range(-1, len(mapping) + 1):
	locs.append(i)
	try:
	label = categorical_mappings[xlabel][i]
	except KeyError:
	label = ""
	labels.append(label)
	ax.set_xticks(locs)
	ax.set_xticklabels(labels, rotation=90)

Here a mapping is being used to map labels to locations. I am not sure why this mapping is not used for the rows.

3 and 4. No idea. Where is this warning being triggered in the workbench?

5. 1.5 is not yet out, so once 1.5 is out we need to deal with this. I have 1.4.5 installed and that does not yet contain the sobol option. 1.4.6 does seem to contain it. So another option is to require 1.4.6 and just move to sobol.

[EwoutH]

1. the future behavior is the desired behavior, so in principle, we don't need to do anything.

Great, moved to the No action needed-list.

3 and 4. No idea. Where is this warning being triggered in the workbench?

3 here and 4 here and 4 here. So all in test/test_em_framework/test_ema_ipyparallel.py::TestLogWatcher::test_extract_level.

[EwoutH]

So another option is to require 1.4.6 and just move to sobol.

I think this is the most future-proof option. Do we needs some sort of validation that it doesn't (significantly) change outcomes?

[EwoutH]

[x] EMAworkbench/ema_workbench/analysis/parcoords.py:176: FutureWarning: In a future version, df.iloc[:, i] = newvals will attempt to set the values inplace instead of always setting a new array. To retain the old behavior, use either df[df.columns[i]] = newvals or, if columns are non-unique, df.isetitem(i, newvals)

Pandas 2.0.0 was released earlier today, and now the FutureWarning has disappeared. Just to doublecheck, we're sure this line of code is now (still) doing what we intended? (without us changing it)

EMAworkbench/ema_workbench/analysis/parcoords.py

Line 176 in 4463a7f

self.limits.loc[:, column] = [0, len(cats) - 1]

[quaquel]

in place is the right behavior.

[EwoutH]

@quaquel A new FutureWarning popped up in the CI:

EMAworkbench/EMAworkbench/ema_workbench/analysis/logistic_regression.py:256: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
self.peeling_trajectory = pd.concat(

It's happening here:

EMAworkbench/ema_workbench/analysis/logistic_regression.py

Line 256 in 1071ac5

self.peeling_trajectory = pd.concat(

Can you check what's the intended behavior?

[quaquel]

The behavior change should not matter. In this code, a new set of rows is added to an existing data frame, so there should not be any NA columns anyway. The new behavior is thus fine.

[EwoutH]

Relevant ones fixed, moving others to 3.0

[EwoutH]

Few FutureWarnings with Pandas 2.2 (since 2.2.0rc0) in the CI run:

/home/runner/work/EMAworkbench/EMAworkbench/ema_workbench/analysis/dimensional_stacking.py:407: FutureWarning: The default value of observed=False is deprecated and will change to observed=True in a future version of pandas. Specify observed=False to silence this warning and retain the current behavior
pvt = pd.pivot_table(x_y_concat, values=ooi_label, index=rows, columns=columns, dropna=False)

/home/runner/work/EMAworkbench/EMAworkbench/ema_workbench/analysis/logistic_regression.py:256: FutureWarning: The behavior of DataFrame concatenation with empty or all-NA entries is deprecated. In a future version, this will no longer exclude empty or all-NA columns when determining the result dtypes. To retain the old behavior, exclude the relevant entries before the concat operation.
self.peeling_trajectory = pd.concat(

/home/runner/work/EMAworkbench/EMAworkbench/test/test_analysis/test_prim.py:387: FutureWarning: ChainedAssignmentError: behaviour will change in pandas 3.0!
You are setting values through chained assignment. Currently this works in certain cases, but when using Copy-on-Write (which will become the default behaviour in pandas 3.0) this will never work to update the original DataFrame or Series, because the intermediate object on which we are setting values will behave as a copy.
A typical example is when you are setting values in a column of a DataFrame, like:

df["col"][row_indexer] = value

Use df.loc[row_indexer, "col"] = values instead, to perform the assignment in a single step and ensure this keeps updating the original df.

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

x.a[x.a > 5] = 5

/home/runner/work/EMAworkbench/EMAworkbench/ema_workbench/analysis/scenario_discovery_util.py:928: FutureWarning: ChainedAssignmentError: behaviour will change in pandas 3.0!
You are setting values through chained assignment. Currently this works in certain cases, but when using Copy-on-Write (which will become the default behaviour in pandas 3.0) this will never work to update the original DataFrame or Series, because the intermediate object on which we are setting values will behave as a copy.
A typical example is when you are setting values in a column of a DataFrame, like:

df["col"][row_indexer] = value

Use df.loc[row_indexer, "col"] = values instead, to perform the assignment in a single step and ensure this keeps updating the original df.

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

df_boxes.loc[unc][index[i]] = values.values

[EwoutH]

Back at 153 warnings again, see this run for example.