Added SparseTensor check on convert_to_np_if_not_ragged.

[edbosne]

The problem occurs when Model.predict() returns a sparse matrix.

Example code:

import keras
import numpy as np
import tensorflow as tf


class SparseLayer(keras.layers.Layer):

    def __init__(self, **kwargs):
        super().__init__(**kwargs)

    def call(self, inputs):
        return inputs
        #return tf.sparse.to_dense(inputs)


n_batches = 4
input_shape = (3, 3)
indices = np.array([[0, 0, 0], [1, 0, 0], [2, 0, 0], [3, 0, 0]])
batch_input = tf.sparse.SparseTensor(dense_shape=(n_batches, *input_shape),
                                      indices=indices,
                                      values=tf.constant([1, 1, 1, 1], dtype=tf.dtypes.float32))

model_input = keras.Input(shape=input_shape, batch_size=1, sparse=True)
sparselayer = SparseLayer()(model_input)
model = keras.Model(inputs=model_input, outputs=sparselayer)

# works
output_call = model(batch_input)
print('output_call', output_call)

# Fails with AttributeError: 'SparseTensor' object has no attribute 'numpy'. Did you mean: '_numpy'?
output_predict = model.predict(batch_input, batch_size=1)
print('output_predict', output_predict)

Proposed solution:

check for sparce tensors in keras/src/backend/tensorflow/trainer.py,

def convert_to_np_if_not_ragged(x):
    if isinstance(x, tf.RaggedTensor):
        return x
    elif isinstance(x, tf.SparseTensor):  # added
        return x
    return x.numpy()

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 79.35%. Comparing base (befa049) to head (d6aab6a).
Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #20151   +/-   ##
=======================================
  Coverage   79.35%   79.35%           
=======================================
  Files         501      501           
  Lines       47311    47313    +2     
  Branches     8689     8690    +1     
=======================================
+ Hits        37542    37544    +2     
  Misses       8014     8014           
  Partials     1755     1755           

Flag	Coverage Δ
keras	79.20% <100.00%> (+<0.01%)	⬆️
keras-jax	62.48% <100.00%> (+<0.01%)	⬆️
keras-numpy	57.59% <100.00%> (+<0.01%)	⬆️
keras-tensorflow	63.86% <100.00%> (+<0.01%)	⬆️
keras-torch	62.51% <100.00%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[fchollet]

Would it make sense to instead return a scipy sparse matrix?

[edbosne]

In my case, this was a test to see if my layer would behave properly inside a model. So I am not using the output directly. The output will be sent to another layer.

I see no need for scipy in my use case, but it all depends on what people expect from the Model.predict() function.

What would have been really useful is that the error would say that, when using predict(), the output of the model needs to be a RaggedTensor, SparceTensor or something with a "numpy" attribute. I've spent quite some time trying to find out what I was doing wrong. However, I have no clue on where to place that check.

[fchollet]

I think it makes sense to return a TF SparseTensor in this case. The underlying issue is basically a TF limitation -- .numpy() is a standard API and it's not normal that it isn't available on a SparseTensor.

Can you add a unit test for this change?

[edbosne]

@fchollet
I've pushed some unit tests. I wasn't exactly sure about the right structure for this so I used my best judgement and did it my way.
I can change it if needed. (on Monday)

However, I don't understand the fail in the code format test. Can you help me make sense of it and understand how to fix it?