ValueError: Variable discriminator/layer_0/batch_norm/discriminator_1/layer_0/batch_norm/moments/normalize/mean/ExponentialMovingAverage/ does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?

[ArghyaPal]

Hi,
I got the following error while trying:

python3 train.py --infogan --dataset /home/arghyapal/tinfogan/rendered_chairs/rendered_chairs/c7e590c0390e8d5debe67d9b32c3ddf8/renders --scale_dataset 64 64 --batch_size 128 --discriminator conv:4:2:64:lrelu,conv:4:2:128:lrelu,conv:4:2:256:lrelu,conv:4:1:256:lrelu,conv:4:1:256:lrelu,fc:1024:lrelu --generator fc:1024,fc:8x8x256,reshape:8:8:256,deconv:4:1:256,deconv:4:2:256,deconv:4:2:128,deconv:4:2:64,deconv:4:1:1:sigmoid --categorical_lambda 1.0 --continuous_lambda 10.0 --categorical_cardinality 20 20 20 --num_continuous 1 --style_size 128 --plot_every 400 --force_grayscale

I was trying a subset of Chair dataset:

`
Found 62 images in /home/arghyapal/tinfogan/rendered_chairs/rendered_chairs/c7e590c0390e8d5debe67d9b32c3ddf8/renders.
Loading dataset 62 100% |#######################################################################################################|Time: 0:00:00
dataset loaded.
generator architecture
Fully connected with num_outputs=1024 followed by relu
Fully connected with num_outputs=16384 followed by relu
Reshape to [8, 8, 256]
Deconvolution with nkernels=4, stride=1, num_outputs=256 followed by relu
Deconvolution with nkernels=4, stride=2, num_outputs=256 followed by relu
Deconvolution with nkernels=4, stride=2, num_outputs=128 followed by relu
Deconvolution with nkernels=4, stride=2, num_outputs=64 followed by relu
Deconvolution with nkernels=4, stride=1, num_outputs=1 followed by sigmoid

Generator produced images of shape (64, 64, 1)

discriminator architecture
Convolution with nkernels=4, stride=2, num_outputs=64 followed by lrelu
Convolution with nkernels=4, stride=2, num_outputs=128 followed by lrelu
Convolution with nkernels=4, stride=2, num_outputs=256 followed by lrelu
Convolution with nkernels=4, stride=1, num_outputs=256 followed by lrelu
Convolution with nkernels=4, stride=1, num_outputs=256 followed by lrelu
Fully connected with num_outputs=1024 followed by lrelu
`

Traceback (most recent call last): File "train.py", line 8, in <module> train() File "/home/arghyapal/tinfogan/infogan/__init__.py", line 288, in train use_batch_norm=use_batch_norm File "/home/arghyapal/tinfogan/infogan/__init__.py", line 61, in discriminator_forward debug=debug) File "/home/arghyapal/tinfogan/infogan/tf_utils.py", line 108, in run_network scope='layer_%d' % (layer_idx,) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args return func(*args, **current_args) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/contrib/layers/python/layers/layers.py", line 927, in convolution outputs = normalizer_fn(outputs, **normalizer_params) File "/home/arghyapal/tinfogan/infogan/tf_utils.py", line 41, in conv_batch_norm ema_apply_op = ema.apply([mean, variance]) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/python/training/moving_averages.py", line 375, in apply colocate_with_primary=(var.op.type in ["Variable", "VariableV2"])) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/python/training/slot_creator.py", line 174, in create_zeros_slot colocate_with_primary=colocate_with_primary) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/python/training/slot_creator.py", line 149, in create_slot_with_initializer dtype) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/python/training/slot_creator.py", line 66, in _create_slot_var validate_shape=validate_shape) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/python/ops/variable_scope.py", line 1049, in get_variable use_resource=use_resource, custom_getter=custom_getter) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/python/ops/variable_scope.py", line 948, in get_variable use_resource=use_resource, custom_getter=custom_getter) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/python/ops/variable_scope.py", line 349, in get_variable validate_shape=validate_shape, use_resource=use_resource) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/contrib/layers/python/layers/layers.py", line 1334, in layer_variable_getter return _model_variable_getter(getter, *args, **kwargs) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/contrib/layers/python/layers/layers.py", line 1326, in _model_variable_getter custom_getter=getter, use_resource=use_resource) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args return func(*args, **current_args) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/contrib/framework/python/ops/variables.py", line 262, in model_variable use_resource=use_resource) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/contrib/framework/python/ops/arg_scope.py", line 181, in func_with_args return func(*args, **current_args) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/contrib/framework/python/ops/variables.py", line 217, in variable use_resource=use_resource) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/python/ops/variable_scope.py", line 341, in _true_getter use_resource=use_resource) File "/home/arghyapal/py3_env/lib/python3.4/site-packages/tensorflow/python/ops/variable_scope.py", line 671, in _get_single_variable "VarScope?" % name) ValueError: Variable discriminator/layer_0/batch_norm/discriminator_1/layer_0/batch_norm/moments/normalize/mean/ExponentialMovingAverage/ does not exist, or was not created with tf.get_variable(). Did you mean to set reuse=None in VarScope?

I don't have any idea why this error. Please help me..!!

Regards,
Arghya

[JonathanRaiman]

I think this is due to tensor flow version changes (built this with tensor flow before 1.0 release). Probably have to update the way batch norm is done to be compatible with new tensor flow best practices.

[wxdai]

I had the same error! Have you manage to solve it?

[hyejin-min]

I also got the same error even with the version 0.12. ...

[wxdai]

@JonathanRaiman Could you let us know which specific version of tensorflow did you use?

[abner2015]

I had the same error and my tensorflow version is 1.2.1

[keevindoherty]

I was able to resolve these issues by using tensorflow version 1.0.0

[OPzealot]

in the file init.py line 285
discriminator_true = discriminator_forward( true_images, discriminator_desc, is_training=is_training_discriminator, reuse=tf.AUTO_REUSE, name="discriminator", use_batch_norm=use_batch_norm )
Just modified reuse=tf.AUTO_REUSE. Problem can be solved.

[wyxcc]

I got the error AttributeError: module 'tensorflow' has no attribute 'AUTO_REUSE'
What should I do with this error? @OPzealot Thanks!

[L1116]

modified reuse=tf.AUTO_REUSE 之后如果报错：AttributeError: module 'tensorflow' has no attribute 'AUTO_REUSE' 只需要执行pip3 install --upgrade tensorflow
更新一下tensorflow就好啦 @wyxcc