Keras 2 <> Keras 3 incompatibilities

[fchollet]

Keras 3 is a major new release. It features a number of cleanups and modernizations of Keras which leads to number of breaking changes compared to Keras 2.

The list below is exhaustive to the best of our knowledge.

A small number of items are likely to affect you (jit_compile default value change, TF SavedModel support changes, usage of tf.Variable as layer attributes). The majority are very niche. All APIs that were removed were dropped due to extremely low usage.

Behavior differences between old tf.keras and Keras 3 (with TF backend)

• APIs that were previously long-deprecated or experimental are gone. compat.v1 APIs are gone (deprecated in 2019). In the case of experimental APIs, usually those are APIs that have already moved to a permanent namespace long ago (e.g. the contents of tf.keras.layers.experimental.preprocessing is now at keras.layers, since 2021), so just update the import path to the up-to-date location.
• Keras 3 has jit_compile=True by default -- this might not work with all TF ops, so with some custom models/layers you might have set jit_compile=False if you see an XLA related error.
• Saving to TF SavedModel format via model.save() is no longer supported (note: you can use tf.save_model.save(model) instead)
• Loading a TF SavedModel file via keras.models.load_model() is no longer supported (note: you can use keras.layers.TFSMLayer(filepath, call_endpoint="serving_default") to reload any TF SavedModel as a Keras layer)
• Model() can no longer be passed deeply nested inputs/outputs (nested more than 1 level deep, e.g. lists of lists of tensors)
• In old tf.keras, TF autograph is enabled by default on the call() method of custom layers. In Keras 3, it is not. This means you may have to use cond ops if you're using control flow, or alternatively you can decorate your call() method with @tf.function.
• Using a TF op on a Keras tensor during functional model construction is disallowed: "A KerasTensor cannot be used as input to a TensorFlow function". Fix: use an equivalent op from keras.ops.
• Multioutput model's evaluate() method does not return individual output losses anymore -> use the metrics argument in compile to track them
• Layer names and variable names can no longer contain the / character.
• No RaggedTensor support. We may add it back later.
• When having multiple named outputs (for example named output_a and output_b, old tf.keras adds <output_a>_loss, <output_b>_loss and so on to metrics. Keras 3 doesn't add them to metrics and needs to be done them to the output metrics by explicitly providing them in metrics list of individual outputs.
• tf.Variable objects assigned to a Layer are not tracked as part of weights. Fix: use self.add_weight() method or use a keras.Variable instead.
• None entries are not allowed as part of nested (e.g. list/tuples) tensor arguments in Layer.call(), nor as part of call() return values.
• Functional models with list outputs do not accept dict losses/metrics anymore
• Symbolic add_loss() is removed (you can still use add_loss() inside the call() method of a layer/model).
• Locally-connected layers are removed (they had ~0 usage). Fix: copy the layer implementation into your own codebase.
• Kernelized layers are removed (they had ~0 usage). Fix: copy the layer implementation into your own codebase.
• Layer attributes metrics, dynamic are removed
• constants arg in RNN layers is removed (remnant of Theano, ~0 usage)
• time_major arg in RNN layers is removed (~0 usage)
• Removed reset_metrics argument from model.*_on_batch methods (~0 usage)
• RadialConstraint constraint object is removed (~0 usage)

Present in Keras 3 standalone but will work when accessing Keras 3 via the new tf.keras

• Various (undocumented) backend functions missing, e.g. backend.random_normal
• AlphaDropout layer is removed
• ThresholdedReLU layer is removed (subsumed by ReLU)
• RandomHeight / RandomWidth layers are removed (better use RandomZoom)

[AakashKumarNain]

Various (undocumented!) backend functions missing

I checked out the issue for converting keras.io examples to keras_core examples. One important functionality that is missing from the backend is CTC. Widely used but diff frameworks implement it in different ways. If we are expecting people to use keras_core for writing framework agnostic code, we need to implement it in the backend.

[PatReis]

Hello, I wanted to ask about the plans to add a KerasRaggedTensor to Keras Core. I would be very interested to have this feature, just to feed the Tensors of shape (batch, None, C, F, ...) to a keras model. I understand that keras backend ops can not support ragged operations across platforms but simply to pass a ragged Tensor to layers as input and output and extract splits and values would be super helpful.

[fchollet]

I would be very interested to have this feature, just to feed the Tensors of shape (batch, None, C, F, ...) to a keras model

We might support RaggedTensor in the future with the TF backend specifically (such a feature does not exist with other frameworks).

However if you're just looking to have a dynamic data dimension, then you can do it just by:

• Bucketing your samples into buckets of shape (batch, A, ...), (batch, B, ...), etc. and creating batches out of the buckets, so that each batch is rectangular
• Padding/truncating your samples to a shared shape
• Some combination of the two

In general it is possible to handle any workflow using rectangular tensors. RaggedTensors are a convenience but not a blocker.

[LukeWood]

I would be very interested to have this feature, just to feed the Tensors of shape (batch, None, C, F, ...) to a keras model

We might support RaggedTensor in the future with the TF backend specifically (such a feature does not exist with other frameworks).

However if you're just looking to have a dynamic data dimension, then you can do it just by:

• Bucketing your samples into buckets of shape (batch, A, ...), (batch, B, ...), etc. and creating batches out of the buckets, so that each batch is rectangular
• Padding/truncating your samples to a shared shape
• Some combination of the two

In general it is possible to handle any workflow using rectangular tensors. RaggedTensors are a convenience but not a blocker.

No idea how it is UX or performance wise, but torch recently added https://pytorch.org/torchrec/torchrec.sparse.html

“jagged tensor” instead of “ragged”

[innat]

A small number of items are likely to affect you ..., TF SavedModel support changes,

For backend agnostic, this may reasonable change (but quite big change). What about the .h5 format? Recently while trying to save keras model, I got the following, is it intended? What the diff of .weights.h5 or only .h5? I think I could do (.h5) earlier.

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
[<ipython-input-7-694b2dd940b3>](https://localhost:8080/#) in <cell line: 14>()
     12 model.compile(optimizer='adam', loss='mse')
     13 _ = model(np.random.rand(96, 96))
---> 14 model.save_weights('model.h5')

1 frames
[/usr/local/lib/python3.10/dist-packages/keras/src/models/model.py](https://localhost:8080/#) in save_weights(self, filepath, overwrite)
    371         """
    372         if not str(filepath).endswith(".weights.h5"):
--> 373             raise ValueError(
    374                 "The filename must end in `.weights.h5`. "
    375                 f"Received: filepath={filepath}"

ValueError: The filename must end in `.weights.h5`. Received: filepath=model.h5

In the built-in model, the .h5 for weight only looks still valid (though it's from tf.keras it runs w/o raising above error).

keras.applications.EfficientNetV2B0(
    include_top=True,
    weights="imagenet",
    input_tensor=None,
    input_shape=None,
    pooling=None,
    classes=1000,
    classifier_activation="softmax",
    include_preprocessing=True,
)
output
Downloading data from https://storage.googleapis.com/tensorflow/keras-applications/efficientnet_v2/efficientnetv2-b0.h5
29403144/29403144 ━━━━━━━━━━━━━━━━━━━━ 0s 0us/step
<Functional name=efficientnetv2-b0, built=True>

[burnpanck]

I was told over at tensorflow that Keras 3 will not support nested input dictionaries. While I don't understand why that support was removed, that's definitely an incompatibility that's worth mentioning here.

[njzjz]

• compat.v1 APIs are gone (deprecated in 2019).

Hello, we are using tf.compat.v1.keras in our project, and it works in TensorFlow 2.15. Does it mean we need to change our codes for TensorFlow 2.16?

---

Update: I found yes. Use tf-keras instead.

[mocher72]

Is there any particular reason why AlphaDropout layer is removed?
Is there an alternative we can use?
I have some code examplesi am trying to convert to v3 and they use tf.keras.layers.AlphaDroput layer

[park]

Is there any particular reason why AlphaDropout layer is removed? Is there an alternative we can use? I have some code examplesi am trying to convert to v3 and they use tf.keras.layers.AlphaDroput layer

Same question here.

SELU needs AlphaDropout.

[El3ssar]

The reset_states method of a model disappeared, and now it is only available on layers. Will this be permanent? This was a nice way of resetting all internal states of all RNN layers in a model.

[edwardyehuang]

@fchollet May I ask what the consideration is for "Layer names and variable names can no longer contain the / character."? I have to make many efforts to convert my old checkpoints and weights for keras-3 compatibility.

UPDATE:

The current temporary workaround for me is to wrap the keras.layer and keras.model to replace / and then wrap the weights reader (e.g., h5) to replace '/` in weights as well.

[shkarupa-alex]

There is no more method compute_output_signature in Layer. Now it is compute_output_spec

[stellarpower]

The reset_states method of a model disappeared, and now it is only available on layers. Will this be permanent? This was a nice way of resetting all internal states of all RNN layers in a model.

I suspect something may have changed re dynamic binding of methods or some other horrible pythonic thing.If I create a Sequential model:

Sequential().reset_states
<bound method Sequential.reset_states of <Sequential name=sequential_6, built=False>>

And when I subclass Sequential, I cannot call the parent method:

class Sequential(keras.models.Sequential):
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)

    def reset_states(self):
        print(cs("Sequential::reset_states", "cyan"))
        super().reset_states()

'super' object has no attribute 'reset_states'

As a side note, please could these sorts of changes be included in formal API documentation? It seems most of the documentation on the Keras site is in the style of small tutorial examples, and a github issue to list breaking changes isn't the best way to navigate what needs to be done, especially when dealing with a scripting language and there is no compiler to help us out. Being able to navigate something like Doxygen and including breaking changes there would be a much better way of seeing hat has changed.

Thanks

[mcourteaux]

Additionally, it seems that RNNs don't support Cells with multiple inputs anymore. Really sad. The "sequences" parameter (note plural) in the rnn.py file assumes actually only one "sequence" (singular).

[Kurdakov]

While trying to run TCN with Keras 3 philipperemy/keras-tcn#256 (issue opened by another person) I found, that while Sequential model in earlier versions allows to get/set Weights in build method, it's not the case with Keras 3 (there is error that numpy is not available ("numpy() is only available when eager execution is enabled"), while previously this was not a case, also functional model somehow has no this problem) see example in first comment tensorflow/addons#2869 (the error is with tensorflow addons master branch, which is not completely ported to Keras 3, but still demonstrates a problem)

additionally: support for causal padding is removed from SeparableConv1D in Keras 3, so it's more difficult to add SeparableConv1D to TCN

[h4ck4l1]

Hello,
In undocumented backend function of tf_keras we had .set_value() and get_value() will those be added to keras 3 in future?
Thankyou.

[jonas-t-k]

The list of behavior changes between the old tf.keras and Keras 3 with TF backend is missing the redefinition of the hard_sigmoid activation function.

Old version: https://github.com/keras-team/keras/blob/v2.15.0/keras/backend.py#L5924
New version: https://github.com/keras-team/keras/blob/v3.0.0/keras/backend/tensorflow/nn.py#L53

I find it worrying that such a breaking change has been made without mentioning it anywhere. The only mention of this change I could find was this issue on the TF GitHub pages, which noted that the documentation for hard_sigmoid was incorrect.

The problem is that this change silently alters the output of every model that uses hard_sigmoid trained with old versions of tf.keras when used in TF >= 2.16, but nothing in the list of changes or migration guide indicates this.