RuntimeError: Unable to create attribute (Object header message is too large)

[Miail]

I seem to get this error when I am using the callback function modelcheckpoint..

I saw from a different issue(#5253) that, that the error could be due to a large network architecture.. Which is also my case.

But how should i then save the model, I can save the model using yaml or json but what about the weight? how can i store them if the model is too large?

https://gist.github.com/Miail/00256b6ddcad65438f3d5c982037060f

Please run script as such:

python mini.py yesno 50 40 8 1

full traceback:
https://pastebin.com/raw/NdWMPfMH

• Check that you are up-to-date with the master branch of Keras. You can update with:
  pip install git+git://github.com/fchollet/keras.git --upgrade --no-deps

• If running on TensorFlow, check that you are up-to-date with the latest version. The installation instructions can be found here.

• If running on Theano, check that you are up-to-date with the master branch of Theano. You can update with:
  pip install git+git://github.com/Theano/Theano.git --upgrade --no-deps

• Provide a link to a GitHub Gist of a Python script that can reproduce your issue (or just copy the script here if it is short).

[Miail]

I made a similar post on SE, which has a reward... If that would tempt you..

https://stackoverflow.com/questions/44198201/callbackfunction-modelcheckpoint-causes-error-in-keras

[bstriner]

You can save_weights_only=True and get around that particular issue. That won't save optimizer weights, so if that is an issue you would have to write a custom callback.

It could be that there is something weird in your json. Inspect model.to_json() and see if you have something big and redundant in there, or non-printable characters or something. What is len(model.to_json())?

Cheers

[bstriner]

Just put together a PR for saving model weights and optimizer weights but not configuration. When you want to load, first instantiate and compile the model as you did when you were going to train it, then use load_all_weights to load the model and optimizer weights into that model.

keras-team/keras-contrib#90

You could use it something like this:

from keras.callbacks import LambdaCallback
from keras_contrib.utils.save_load_utils import save_all_weights, load_all_weights
# do some stuff to create and compile model
# use `save_all_weights` as a callback to checkpoint your model and optimizer weights
model.fit(..., callbacks=[LambdaCallback(on_epoch_end=lambda epoch, logs: save_all_weights(model, "checkpoint-{:05d}.h5".format(epoch))])
# use `load_all_weights` to load model and optimizer weights into an existing model
# if not compiled (no `model.optimizer`), this will just load model weights
load_all_weights(model, 'checkpoint-1337.h5')

Cheers

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed after 30 days if no further activity occurs, but feel free to re-open a closed issue if needed.

[zyavrik]

Unfixed issues shouldn't be ever marked as closed. But it seems the issue is with the h5 package.

[pGit1]

@bstriner,

How to modify your code to just save the best weights AND not every instance?

[Anurag27031994]

###USE get_weights AND set_weights TO SAVE AND LOAD MODEL, RESPECTIVELY.
#OPEN THIS LINK TO READ THE SAME CODE PROPERLY:
https://drive.google.com/open?id=1xzrqP7ExTmJiZqVt0A_G6AT69EbIjEI9tUDLD1twqj8

##############################################################################

#Assuming that this is your model architecture. However, you may use
#whatever architecture, you want to (big or small; any).
def mymodel():
inputShape= (28, 28, 3);
model= Sequential()
model.add(Conv2D(20, 5, padding="same", input_shape=inputShape))
model.add(Activation('relu'))
model.add(Flatten())
model.add(Dense(500))
model.add(Activation('relu'))
model.add(Dense(2, activation= "softmax"))
return model
model.fit(....) #paramaters to start training your model

################################################################################
################################################################################
#once your model has been trained, you want to save your model in your PC
#use get_weights() command to get your model weights
weigh= model.get_weights()

#now, use pickle to save your model weights, instead of .h5
#for heavy model architectures, .h5 file is unsupported.
pklfile= "D:/modelweights.pkl"
try:
fpkl= open(pklfile, 'wb') #Python 3
pickle.dump(weigh, fpkl, protocol= pickle.HIGHEST_PROTOCOL)
fpkl.close()
except:
fpkl= open(pklfile, 'w') #Python 2
pickle.dump(weigh, fpkl, protocol= pickle.HIGHEST_PROTOCOL)
fpkl.close()

################################################################################
################################################################################
#in future, you may want to load your model back
#use pickle to load model weights

pklfile= "D:/modelweights.pkl"
try:
f= open(pklfile) #Python 2

weigh= pickle.load(f);                
f.close();

except:

f= open(pklfile, 'rb')     #Python 3                 
weigh= pickle.load(f);                
f.close();

restoredmodel= mymodel()
#use set_weights to load the modelweights into the model architecture
restoredmodel.set_weights(weigh)

################################################################################
################################################################################
#now, you can do your testing and evaluation- predictions
y_pred= restoredmodel.predict(X)

[Anurag27031994]

open this drive link and read the same above code properly:

https://drive.google.com/open?id=1xzrqP7ExTmJiZqVt0A_G6AT69EbIjEI9tUDLD1twqj8