Auto Byte

Science AI

# 最小二乘GAN：比常规GAN更稳定，比WGAN收敛更迅速

LSGAN 的主要思想就是在辨别器 D 中使用更加平滑和非饱和（non-saturating）梯度的损失函数。我们想要辨别器（discriminator）D 将生成器（generator）G 所生成的数据「拖」到真实数据流形（data manifold）Pdata(X)，从而使得生成器 G 生成类似 Pdata(X) 的数据。

1. 从辨别器 D 中移除对数损失

2. 使用 L2 损失代替对数损失

G = torch.nn.Sequential(

torch.nn.Linear(z_dim, h_dim),

torch.nn.ReLU(),

torch.nn.Linear(h_dim, X_dim),

torch.nn.Sigmoid()

)

D = torch.nn.Sequential(

torch.nn.Linear(X_dim, h_dim),

torch.nn.ReLU(),

# No sigmoid

torch.nn.Linear(h_dim, 1),

)

G_solver = optim.Adam(G.parameters(), lr=lr)

D_solver = optim.Adam(D.parameters(), lr=lr)

for it in range(1000000):

# Sample data

z = Variable(torch.randn(mb_size, z_dim))

X, _ = mnist.train.next_batch(mb_size)

X = Variable(torch.from_numpy(X))

# Dicriminator

G_sample = G(z)

D_real = D(X)

D_fake = D(G_sample)

# Discriminator loss

D_loss = 0.5 * (torch.mean((D_real - 1)**2) + torch.mean(D_fake**2))

D_loss.backward()

D_solver.step()

# Generator

G_sample = G(z)

D_fake = D(G_sample)

# Generator loss

G_loss = 0.5 * torch.mean((D_fake - 1)**2)

G_loss.backward()

G_solver.step()

• 论文：Least Squares Generative Adversarial Networks

1. Nowozin, Sebastian, Botond Cseke, and Ryota Tomioka.「f-GAN: Training generative neural samplers using variational divergence minimization.」Advances in Neural Information Processing Systems. 2016. arxiv (https://arxiv.org/abs/1606.00709)

2. Mao, Xudong, et al.「Multi-class Generative Adversarial Networks with the L2 Loss Function.」arXiv preprint arXiv:1611.04076 (2016).