You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In other GAN papers, we usually train D for k steps and train G for only one step in each GAN training iteration In this case, the z used for training G and D are obviously different. But in this paper, we train G and D simultaneously, i.e., training G and D for both one step in a training iteration. I'm wondering if I could use the same z to train both two networks in each iteration to reduce the computational cost? Just as shown in Pytorch's official GAN tutorial (https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html#training):
# Training Loop# Lists to keep track of progressimg_list= []
G_losses= []
D_losses= []
iters=0print("Starting Training Loop...")
# For each epochforepochinrange(num_epochs):
# For each batch in the dataloaderfori, datainenumerate(dataloader, 0):
############################# (1) Update D network: maximize log(D(x)) + log(1 - D(G(z)))############################# Train with all-real batchnetD.zero_grad()
# Format batchreal_cpu=data[0].to(device)
b_size=real_cpu.size(0)
label=torch.full((b_size,), real_label, dtype=torch.float, device=device)
# Forward pass real batch through Doutput=netD(real_cpu).view(-1)
# Calculate loss on all-real batcherrD_real=criterion(output, label)
# Calculate gradients for D in backward passerrD_real.backward()
D_x=output.mean().item()
## Train with all-fake batch# Generate batch of latent vectorsnoise=torch.randn(b_size, nz, 1, 1, device=device)
# Generate fake image batch with Gfake=netG(noise)
label.fill_(fake_label)
# Classify all fake batch with Doutput=netD(fake.detach()).view(-1)
# Calculate D's loss on the all-fake batcherrD_fake=criterion(output, label)
# Calculate the gradients for this batch, accumulated (summed) with previous gradientserrD_fake.backward()
D_G_z1=output.mean().item()
# Compute error of D as sum over the fake and the real batcheserrD=errD_real+errD_fake# Update DoptimizerD.step()
############################# (2) Update G network: maximize log(D(G(z)))###########################netG.zero_grad()
label.fill_(real_label) # fake labels are real for generator cost# Since we just updated D, perform another forward pass of all-fake batch through Doutput=netD(fake).view(-1)
# Calculate G's loss based on this outputerrG=criterion(output, label)
# Calculate gradients for GerrG.backward()
D_G_z2=output.mean().item()
# Update GoptimizerG.step()
# Output training statsifi%50==0:
print('[%d/%d][%d/%d]\tLoss_D: %.4f\tLoss_G: %.4f\tD(x): %.4f\tD(G(z)): %.4f / %.4f'% (epoch, num_epochs, i, len(dataloader),
errD.item(), errG.item(), D_x, D_G_z1, D_G_z2))
# Save Losses for plotting laterG_losses.append(errG.item())
D_losses.append(errD.item())
# Check how the generator is doing by saving G's output on fixed_noiseif (iters%500==0) or ((epoch==num_epochs-1) and (i==len(dataloader)-1)):
withtorch.no_grad():
fake=netG(fixed_noise).detach().cpu()
img_list.append(vutils.make_grid(fake, padding=2, normalize=True))
iters+=1
Such that we only need to call G.forward once in each iteration. Will it affect the model performance?
The text was updated successfully, but these errors were encountered:
In other GAN papers, we usually train
D
fork
steps and trainG
for only one step in each GAN training iteration In this case, thez
used for training G and D are obviously different. But in this paper, we trainG
andD
simultaneously, i.e., trainingG
andD
for both one step in a training iteration. I'm wondering if I could use the samez
to train both two networks in each iteration to reduce the computational cost? Just as shown in Pytorch's official GAN tutorial (https://pytorch.org/tutorials/beginner/dcgan_faces_tutorial.html#training):Such that we only need to call
G.forward
once in each iteration. Will it affect the model performance?The text was updated successfully, but these errors were encountered: