AutoSklearn2 broken on pandas 2

[PGijsbers]

Describe the bug

When having pandas 2 installed in the environment, fit calls on AutoSklearn2Classifier objects will result in an error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/bench/frameworks/autosklearn/venv/lib/python3.9/site-packages/autosklearn/experimental/askl2.py", line 472, in fit
    selection = np.argmax(selector.predict(metafeatures))
  File "/bench/frameworks/autosklearn/venv/lib/python3.9/site-packages/autosklearn/experimental/selector.py", line 25, in predict
    prediction = self._predict(X, y)
  File "/bench/frameworks/autosklearn/venv/lib/python3.9/site-packages/autosklearn/experimental/selector.py", line 366, in _predict
    prediction = self.selector.predict(X)
  File "/bench/frameworks/autosklearn/venv/lib/python3.9/site-packages/autosklearn/experimental/selector.py", line 26, in predict
    for col, series in prediction.iteritems():
  File "/bench/frameworks/autosklearn/venv/lib/python3.9/site-packages/pandas/core/generic.py", line 5989, in __getattr__
    return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'iteritems'

It looks like pandas 2 and up .items should be used instead as .iteritems was removed.

To Reproduce

Running from the framework environment in docker run -it -u 0 automlbenchmark/autosklearn:0.15.0-add_versions_2023Q2:

source frameworks/autosklearn/venv/bin/activate
python mwe.py  # ERROR

mwe.py:

from autosklearn.experimental.askl2 import AutoSklearn2Classifier
import openml
iris, *_ = openml.datasets.get_dataset(61).get_data(dataset_format='dataframe')
AutoSklearn2Classifier(time_left_for_this_task=60).fit(iris.iloc[:,:4], iris.iloc[:, 4])  # produces error

above produces the error, but after downgrading pandas it works fine:

python -m pip install "pandas<2.0"  # installs 1.5.3
python mwe.py  # OK

Expected behavior

For it not to crash.

Actual behavior, stacktrace or logfile

Provided above.

Environment and installation:

See Docker image.