匯入原始資料

原始資料有95筆 * 1462維資料，並且以是否為CRE細菌作為二元分類依據，我們先將資料切至95 * 50的大小並建立一簡易二元分類模型。

import pandas as pd
import numpy as np

data = pd.read_csv('C:/Users/User/OneDrive - student.nsysu.edu.tw/Educations/NSYSU/fu_chung/bacterial/123.csv')
data.head()
data_50 = data.iloc[:,0:50]
cre = data.loc[:, ['CRE']]
cre = pd.concat([data_50,cre],axis = 1,join = 'outer')
data1 = data[0:46]
data11 = data.iloc[0:46,0:50]
data2 = data[46:96]
cre

ax = pd.DataFrame(np.transpose(data.head())).plot()

訓練分類器模型

這裡使用隨機森林建立一個簡單但具分類意義的分類器。

import numpy as np
import pandas as pd
from sklearn import preprocessing
from sklearn.model_selection import train_test_split

# Build Train and Test data set
cre_X = cre.iloc[:,0:50]
cre_y = cre["CRE"]
train_X, test_X, train_y, test_y = train_test_split(cre_X, cre_y, test_size = 0.3)

# Build the random forest model
forest = ensemble.RandomForestClassifier(n_estimators = 100)
forest_fit = forest.fit(train_X, train_y)

test_y_predicted = forest.predict(test_X)
accuracy_rf = metrics.accuracy_score(test_y, test_y_predicted)
print(accuracy_rf)

0.8620689655172413

Set Up Module

%matplotlib inline
import os
import random
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns
from tqdm import tqdm_notebook as tqdm
from keras.models import Model
from keras.layers import Input, Reshape
from keras.layers.core import Dense, Activation, Dropout, Flatten
from keras.layers.normalization import BatchNormalization
from keras.layers.convolutional import UpSampling1D, Conv1D
from keras.layers.advanced_activations import LeakyReLU
from keras.optimizers import Adam, SGD
from keras.callbacks import TensorBoard

建立生成器

def get_generative(G_in, dense_dim=200, out_dim=50, lr=1e-3):
    x = Dense(dense_dim)(G_in)
    x = Activation('tanh')(x)
    G_out = Dense(out_dim, activation='tanh')(x)
    G = Model(G_in, G_out)
    opt = SGD(lr=lr)
    G.compile(loss='binary_crossentropy', optimizer=opt)
    return G, G_out

G_in = Input(shape=[10])
G, G_out = get_generative(G_in)
G.summary()

---

Layer (type) Output Shape Param #
================================================================= input_4 (InputLayer) (None, 10) 0
_________________________________________________________________ dense_5 (Dense) (None, 200) 2200
_________________________________________________________________ activation_2 (Activation) (None, 200) 0
_________________________________________________________________ dense_6 (Dense) (None, 50) 10050
================================================================= Total params: 12,250 Trainable params: 12,250 Non-trainable params: 0 _________________________________________________________________

建立判別器

def get_discriminative(D_in, lr=1e-3, drate=.25, n_channels=50, conv_sz=5, leak=.2):
    x = Reshape((-1, 1))(D_in)
    x = Conv1D(n_channels, conv_sz, activation='relu')(x)
    x = Dropout(drate)(x)
    x = Flatten()(x)
    x = Dense(n_channels)(x)
    D_out = Dense(2, activation='sigmoid')(x)
    D = Model(D_in, D_out)
    dopt = Adam(lr=lr)
    D.compile(loss='binary_crossentropy', optimizer=dopt)
    return D, D_out

D_in = Input(shape=[50])
D, D_out = get_discriminative(D_in)
D.summary()

---

Layer (type) Output Shape Param #
================================================================= input_5 (InputLayer) (None, 50) 0
_________________________________________________________________ reshape_2 (Reshape) (None, 50, 1) 0
_________________________________________________________________ conv1d_2 (Conv1D) (None, 46, 50) 300
_________________________________________________________________ dropout_2 (Dropout) (None, 46, 50) 0
_________________________________________________________________ flatten_2 (Flatten) (None, 2300) 0
_________________________________________________________________ dense_7 (Dense) (None, 50) 115050
_________________________________________________________________ dense_8 (Dense) (None, 2) 102
================================================================= Total params: 115,452 Trainable params: 115,452 Non-trainable params: 0 _________________________________________________________________

串接兩神經網路

def set_trainability(model, trainable=False):
    model.trainable = trainable
    for layer in model.layers:
        layer.trainable = trainable
        
def make_gan(GAN_in, G, D):
    set_trainability(D, False)
    x = G(GAN_in)
    GAN_out = D(x)
    GAN = Model(GAN_in, GAN_out)
    GAN.compile(loss='binary_crossentropy', optimizer=G.optimizer)
    return GAN, GAN_out

GAN_in = Input([10])
GAN, GAN_out = make_gan(GAN_in, G, D)
GAN.summary()

---

Layer (type) Output Shape Param #
================================================================= input_6 (InputLayer) (None, 10) 0
_________________________________________________________________ model_4 (Model) (None, 50) 12250
_________________________________________________________________ model_5 (Model) (None, 2) 115452
================================================================= Total params: 127,702 Trainable params: 12,250 Non-trainable params: 115,452 _________________________________________________________________

def sample_data_and_gen(G, noise_dim=10, n_samples=46):
    XT = np.array(data11)
    XN_noise = np.random.uniform(0, 1, size=[n_samples, noise_dim])
    XN = G.predict(XN_noise)
    X = np.concatenate((XT, XN))
    y = np.zeros((2*n_samples, 2))
    y[:n_samples, 1] = 1
    y[n_samples:, 0] = 1
    return X, y

def pretrain(G, D, noise_dim=10, n_samples=46, batch_size=32):
    X, y = sample_data_and_gen(G, n_samples=n_samples, noise_dim=noise_dim)
    set_trainability(D, True)
    D.fit(X, y, epochs=1, batch_size=batch_size)
    
pretrain(G, D)

def sample_noise(G, noise_dim=10, n_samples=46):
    X = np.random.uniform(0, 1, size=[n_samples, noise_dim])
    y = np.zeros((n_samples, 2))
    y[:, 1] = 1
    return X, y

Epoch 1/1 92/92 [==============================] - ETA: 0s - loss: 2.393 - 1s 6ms/step - loss: 5.6152

訓練生成對抗網路

訓練生成對抗網路，並存取最後一次生成器產生結果

def train(GAN, G, D, epochs=500, n_samples=46, noise_dim=10, batch_size=32, verbose=False, v_freq=50,):
    d_loss = []
    g_loss = []
    e_range = range(epochs)
    if verbose:
        e_range = tqdm(e_range)
    for epoch in e_range:
        X, y = sample_data_and_gen(G, n_samples=n_samples, noise_dim=noise_dim)
        set_trainability(D, True)
        d_loss.append(D.train_on_batch(X, y))
        xx,yy = X,y
        
        X, y = sample_noise(G, n_samples=n_samples, noise_dim=noise_dim)
        set_trainability(D, False)
        g_loss.append(GAN.train_on_batch(X, y))
        if verbose and (epoch + 1) % v_freq == 0:
            print("Epoch #{}: Generative Loss: {}, Discriminative Loss: {}".format(epoch + 1, g_loss[-1], d_loss[-1]))
    return d_loss, g_loss, xx, yy

d_loss, g_loss ,xx,yy= train(GAN, G, D, verbose=True)

HBox(children=(IntProgress(value=0, max=500), HTML(value=’’)))

Epoch #50: Generative Loss: 11.389734268188477, Discriminative Loss: 8.015128135681152 Epoch #100: Generative Loss: 11.560905456542969, Discriminative Loss: 8.015127182006836 Epoch #150: Generative Loss: 11.707502365112305, Discriminative Loss: 8.01512622833252 Epoch #200: Generative Loss: 11.78805160522461, Discriminative Loss: 8.01512336730957 Epoch #250: Generative Loss: 11.92248821258545, Discriminative Loss: 8.015122413635254 Epoch #300: Generative Loss: 12.040063858032227, Discriminative Loss: 8.015122413635254 Epoch #350: Generative Loss: 12.16407299041748, Discriminative Loss: 8.015120506286621 Epoch #400: Generative Loss: 12.269856452941895, Discriminative Loss: 8.015120506286621 Epoch #450: Generative Loss: 12.354406356811523, Discriminative Loss: 8.015120506286621 Epoch #500: Generative Loss: 12.472232818603516, Discriminative Loss: 8.015120506286621

損失函數

ax = pd.DataFrame(
    {
        'Generative Loss': g_loss,
        'Discriminative Loss': d_loss,
    }
).plot(title='Training loss', logy=True)
ax.set_xlabel("Epochs")
ax.set_ylabel("Loss")

Text(0, 0.5, ‘Loss’)

最後一次生成器所生成資料

pd.DataFrame(xx)

pd.DataFrame(np.transpose(xx)).plot()

<matplotlib.axes._subplots.AxesSubplot at 0x21e22a40c50>

Test the generation

使用最後一次生成結果匯入前面訓練好的分類模型，檢查是否能判別為第一類別。

test_y_predicted = forest.predict(xx)
pd.DataFrame(test_y_predicted)

判別為正確的比例：

sum(test_y_predicted)/len(test_y_predicted)

0.4673913043478261

結論

我們可以看到，生成的樣本分類率僅0.46左右，可以說是相當不理想，但換個角度思考，這當中有46%的資料是可以當作新的樣本加入訓練集，因此我們可透過事先建立的分類器來當作衡量樣本是否可當作是當的生成樣本。且若我們僅憑肉眼看生成的資料，其範圍也落在0-40000這個區間左右，矩陣稀疏情況也看似接近。當然，生成對抗網路還有許多細節可以調整，也許可以調整學習率、backpropagation、drop out等等事項來改善生成器神經網路。先前看到有多種生成對抗網路，如conditional-GAN,MCF-GAN等等，也許我可以學習其他類型架構的生成對抗網路來完成這樣工作也不一定。