TypeError: Object of type Tensor is not JSON serializable

[josch]

When I run learning/main.py with Python 3 and a recent version of pytorch I get the following traceback:

Traceback (most recent call last):
  File "learning/main.py", line 388, in <module>
    main()
  File "learning/main.py", line 300, in main
    json.dump(stats, outfile)
  File "/usr/lib/python3.7/json/__init__.py", line 179, in dump
    for chunk in iterable:
  File "/usr/lib/python3.7/json/encoder.py", line 429, in _iterencode
    yield from _iterencode_list(o, _current_indent_level)
  File "/usr/lib/python3.7/json/encoder.py", line 325, in _iterencode_list
    yield from chunks
  File "/usr/lib/python3.7/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.7/json/encoder.py", line 438, in _iterencode
    o = _default(o)
  File "/usr/lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Tensor is not JSON serializable

This patch fixes the problem:

diff --git a/learning/main.py b/learning/main.py
index 2b181fa..5a56ccd 100644
--- a/learning/main.py
+++ b/learning/main.py
@@ -292,7 +292,7 @@ def main():
             acc_test, oacc_test, avg_iou_test, avg_acc_test = 0, 0, 0, 0
             print('-> Train accuracy: {}, \tLoss: {}'.format(acc, loss))
 
-        stats.append({'epoch': epoch, 'acc': acc, 'loss': loss, 'oacc': oacc, 'avg_iou': avg_iou, 'acc_test': acc_test, 'oacc_test': oacc_test, 'avg_iou_test': avg_iou_test, 'avg_acc_test': avg_acc_test})
+        stats.append({'epoch': epoch, 'acc': acc, 'loss': loss.cpu().numpy().item(), 'oacc': oacc, 'avg_iou': avg_iou, 'acc_test': acc_test, 'oacc_test': oacc_test, 'avg_iou_test': avg_iou_test, 'avg_acc_test': avg_acc_test})
 
         if epoch % args.save_nth_epoch == 0 or epoch==args.epochs-1:
             with open(os.path.join(args.odir, 'trainlog.txt'), 'w') as outfile:

[loicland]

I pushed a patch in a similar spirit. This is due to how pytorch 0.4 deals with scalar tensors.

[brando90]

When I run learning/main.py with Python 3 and a recent version of pytorch I get the following traceback:

Traceback (most recent call last):
  File "learning/main.py", line 388, in <module>
    main()
  File "learning/main.py", line 300, in main
    json.dump(stats, outfile)
  File "/usr/lib/python3.7/json/__init__.py", line 179, in dump
    for chunk in iterable:
  File "/usr/lib/python3.7/json/encoder.py", line 429, in _iterencode
    yield from _iterencode_list(o, _current_indent_level)
  File "/usr/lib/python3.7/json/encoder.py", line 325, in _iterencode_list
    yield from chunks
  File "/usr/lib/python3.7/json/encoder.py", line 405, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.7/json/encoder.py", line 438, in _iterencode
    o = _default(o)
  File "/usr/lib/python3.7/json/encoder.py", line 179, in default
    raise TypeError(f'Object of type {o.__class__.__name__} '
TypeError: Object of type Tensor is not JSON serializable

This patch fixes the problem:

diff --git a/learning/main.py b/learning/main.py
index 2b181fa..5a56ccd 100644
--- a/learning/main.py
+++ b/learning/main.py
@@ -292,7 +292,7 @@ def main():
             acc_test, oacc_test, avg_iou_test, avg_acc_test = 0, 0, 0, 0
             print('-> Train accuracy: {}, \tLoss: {}'.format(acc, loss))
 
-        stats.append({'epoch': epoch, 'acc': acc, 'loss': loss, 'oacc': oacc, 'avg_iou': avg_iou, 'acc_test': acc_test, 'oacc_test': oacc_test, 'avg_iou_test': avg_iou_test, 'avg_acc_test': avg_acc_test})
+        stats.append({'epoch': epoch, 'acc': acc, 'loss': loss.cpu().numpy().item(), 'oacc': oacc, 'avg_iou': avg_iou, 'acc_test': acc_test, 'oacc_test': oacc_test, 'avg_iou_test': avg_iou_test, 'avg_acc_test': avg_acc_test})
 
         if epoch % args.save_nth_epoch == 0 or epoch==args.epochs-1:
             with open(os.path.join(args.odir, 'trainlog.txt'), 'w') as outfile:

If like me you just wanted to save the dict of tensors in a human readable way check this out:

https://discuss.pytorch.org/t/typeerror-tensor-is-not-json-serializable/36065/3 or https://stackoverflow.com/questions/12943819/how-to-prettyprint-a-json-file/66180687#66180687 and fossasia/visdom#554.