# Notebook from mong-head/-Alzheimer_classification
Path: practice/alzheimer_practice_NN.ipynb

_____no_output_____# drive 저장 및 set up_____no_output_____

from google.colab import drive
drive.mount('/content/drive')Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&response_type=code&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly

Enter your authorization code:
··········
Mounted at /content/drive
import os
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import csv

from pandas import Series, DataFrame

# Python ≥3.5 is required
import sys
assert sys.version_info >= (3, 5)

# Scikit-Learn ≥0.20 is required
import sklearn
assert sklearn.__version__ >= "0.20"

try:
    # %tensorflow_version only exists in Colab.
    %tensorflow_version 2.x
except Exception:
    pass

# TensorFlow ≥2.0 is required
import tensorflow as tf
assert tf.__version__ >= "2.0"

import tensorflow as tf
from tensorflow import kerasTensorFlow 2.x selected.

# * data 처리_____no_output_____***Load TADPOLE* dataset (csv file) from Google Drive***
---
*The Alzheimer's Disease Prediction Of Longitudinal Evolution
(https://tadpole.grand-challenge.org/)

### -Subjects: 1707 (1363 Train (80%) + 344 Test (20%))
### -Features: 72
*   2 demographic feature: MMSE, ADAS13
*   70 mean values of cortical thickness_____no_output_____데이터 불러오기_____no_output_____

from google.colab import drive
drive.mount('/content/gdrive')

csv_file_train = '/content/gdrive/My Drive/BNCS401_Midterm_Project/Train_data_reupdated.csv'  # Set your path
train_data = pd.read_csv(csv_file_train)
train_data

# DXCHANGE: clinical label (1-CN, 2-MCI, 3-AD)Mounted at /content/gdrive

train_data.info()로 확인하니, 빠져있는 데이터도 있고(예측값인 ADAS13, cortical값 중에 1개), 실제로는 NaN값이지만 ' '로 채워진 데이터가 2개가 있었다.

' '로 채워져 있어 object인 두 cortical값
*   ST64TA_UCSFFSX_11_02_15_UCSFFSX51_08_01_16
*   ST123TA_UCSFFSX_11_02_15_UCSFFSX51_08_01_16

데이터 처리 순서
1. object 처리 : space to NaN
2. imputer : median으로 빠져있는 값들 채우기
3. feature scaling - RobustScaler : cortical, age 값의 범위를 같게 맞춘다.

** StandardScaler안쓰고 RobustScaler하는 이유 : 실제로 이게 더 나았음- outlier제거가 되기때문임

2,3을 pipeline으로 하고, 2 전에 AD_tr(cortical값들, age),y(reg/cls)로 나누고,

AD_tr만 pipeline해서 X로 만들고, y는 imputer만 한다.
_____no_output_____1) object처리
---_____no_output_____

#2개의 cortical feature가 float이 아니라 object이고, ' '로 채워져있다.
ob1 = train_data["ST64TA_UCSFFSX_11_02_15_UCSFFSX51_08_01_16"]
ob2 = train_data["ST123TA_UCSFFSX_11_02_15_UCSFFSX51_08_01_16"]
print(len(train_data[ob1.isnull()]), ob1.dtypes)
print(len(train_data[ob2.isnull()]),ob2.dtypes)0 object
0 object
#그냥 ' '로 뜬다.
train_data["ST64TA_UCSFFSX_11_02_15_UCSFFSX51_08_01_16"][0]_____no_output_____def object_to_float(o): #o = train_data["~~"] 형태
  for i in range(1363):
    if o[i] == ' ':
      o[i] = np.nan #nan으로 채움_____no_output_____object_to_float(ob1)
object_to_float(ob2)/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:4: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame

See the caveats in the documentation: http://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  after removing the cwd from sys.path.
print(len(train_data[ob1.isnull()]), ob1.dtypes)
print(len(train_data[ob2.isnull()]),ob2.dtypes)724 object
724 object

**train, test set나누기
---
AD_train, AD_test = _____no_output_____

#RID는 버림
train_data = train_data.drop("RID",axis=1)_____no_output_____

** trian, test set 나누기
---
train set에서 model들의 train loss와 validation loss등을 보며 model을 결정하고

test set에서 마지막으로 검사해보았을 때에도 최적의 모델이라면

그 모델을 선택하고, 마지막에는 train, test set 나누기 전의 X로 최종으로 선택했던 모델로 마지막 training시킨다._____no_output_____

from sklearn.model_selection import train_test_split

AD_train, AD_test = train_test_split(train_data,test_size=0.2,random_state= 42)_____no_output_____

** AD_tr, y나누고, y에는 imputer
---_____no_output_____RID는 버린다._____no_output_____

from sklearn.impute import SimpleImputer
imputer = SimpleImputer(strategy="median")_____no_output_____##input 해당
X_prepipeline = train_data.drop("MMSE",axis = 1).drop("ADAS13",axis = 1).drop("DXCHANGE",axis = 1)
X_train_prepipeline = AD_train.drop("MMSE",axis = 1).drop("ADAS13",axis = 1).drop("DXCHANGE",axis = 1)
X_test_prepipeline = AD_test.drop("MMSE",axis = 1).drop("ADAS13",axis = 1).drop("DXCHANGE",axis = 1)

#classification input
X_plus_prepipeline = train_data.drop("DXCHANGE",axis = 1)
X_plus_train_prepipeline = AD_train.drop("DXCHANGE",axis = 1)
X_plus_test_prepipeline = AD_test.drop("DXCHANGE",axis = 1)

##output 해당
AD_label_reg = DataFrame({"MMSE":train_data["MMSE"].copy(),
                         "ADAS13":train_data["ADAS13"].copy()})
AD_label_reg_tr = DataFrame({"MMSE":AD_train["MMSE"].copy(),
                         "ADAS13":AD_train["ADAS13"].copy()})
AD_label_reg_test = DataFrame({"MMSE":AD_test["MMSE"].copy(),
                         "ADAS13":AD_test["ADAS13"].copy()})

imputer.fit(AD_label_reg)
temp = imputer.transform(AD_label_reg)
y_reg = pd.DataFrame(temp, columns=AD_label_reg.columns,
                             index=AD_label_reg.index)

imputer.fit(AD_label_reg_tr)
temp = imputer.transform(AD_label_reg_tr)
y_reg_train = pd.DataFrame(temp, columns=AD_label_reg_tr.columns,
                             index=AD_label_reg_tr.index)
imputer.fit(AD_label_reg_test)
temp = imputer.transform(AD_label_reg_test)
y_reg_test = pd.DataFrame(temp, columns=AD_label_reg_test.columns,
                             index=AD_label_reg_test.index)

y_mmse =  DataFrame({"MMSE":y_reg["MMSE"].copy()})
y_mmse_train = DataFrame({"MMSE":y_reg_train["MMSE"].copy()})
y_mmse_test = DataFrame({"MMSE":y_reg_test["MMSE"].copy()})

y_ada =  DataFrame({"ADAS13":y_reg["ADAS13"].copy()})
y_ada_train = DataFrame({"ADAS13":y_reg_train["ADAS13"].copy()})
y_ada_test = DataFrame({"ADAS13":y_reg_test["ADAS13"].copy()})

y_cls =  DataFrame({"DXCHANGE":train_data["DXCHANGE"]})
y_cls_train = DataFrame({"DXCHANGE":AD_train["DXCHANGE"]})
y_cls_test = DataFrame({"DXCHANGE":AD_test["DXCHANGE"]})

incomplete_rows_before = AD_label_reg_tr[AD_label_reg_tr.isnull().any(axis=1)]#.head()
incomplete_rows_after = y_reg_train[y_reg_train.isnull().any(axis=1)]
print("1. before imputer:\n", incomplete_rows_before)
print("\n2. after imputer : \n", incomplete_rows_after)1. before imputer:
       MMSE  ADAS13
522     23     NaN
25      21     NaN
1311    18     NaN
999     20     NaN
426     25     NaN
946     29     NaN
431     22     NaN

2. after imputer :
 Empty DataFrame
Columns: [MMSE, ADAS13]
Index: []

2) pipeline : imputer,RobustScaler
---
* imputer - 빈데이터 채우기
* RobustScaler - 값 범위 같게 만들기, Standard보다는 outlier다루는데 더 괜찮을 거 같아서 이것을 썼다._____no_output_____

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import RobustScaler
from sklearn.impute import SimpleImputer
imputer = SimpleImputer(strategy="median")_____no_output_____#빈데이터 모두 보기
incomplete_rows = X_prepipeline[X_prepipeline.isnull().any(axis=1)]#.head()
print("빈데이터 개수: ",len(incomplete_rows))빈데이터 개수:  724
data_pipeline = Pipeline([
                     ('imputer',SimpleImputer(strategy="median")),
                     ('rb_scaler',RobustScaler()),
])_____no_output_____X = data_pipeline.fit_transform(X_prepipeline)
X_train = data_pipeline.fit_transform(X_train_prepipeline)
X_test = data_pipeline.fit_transform(X_test_prepipeline)

X_plus = data_pipeline.fit_transform(X_plus_prepipeline)
X_plus_train = data_pipeline.fit_transform(X_plus_train_prepipeline)
X_plus_test = data_pipeline.fit_transform(X_plus_test_prepipeline)_____no_output_____

# Tensorboard_____no_output_____

root_logdir = os.path.join(os.curdir, "my_logs")_____no_output_____def get_run_logdir():
    import time
    run_id = time.strftime("run_%Y_%m_%d-%H_%M_%S")
    return os.path.join(root_logdir, run_id)

run_logdir = get_run_logdir()_____no_output_____keras.backend.clear_session()
np.random.seed(42)
tf.random.set_seed(42)_____no_output_____

# 1 - Regression_____no_output_____

X.shape[1:]_____no_output_____np.random.seed(42)
tf.random.set_seed(42)_____no_output_____model = keras.models.Sequential([
    keras.layers.Dense(10, activation="relu",input_shape=X.shape[1:]),
    #keras.layers.Dense(10, activation="relu"),
    keras.layers.Dense(2, activation="relu")
])
model.compile(loss="mean_squared_error", optimizer=keras.optimizers.SGD(lr=1e-3))
model.save("keras_reg_model.h5")_____no_output_____tensorboard_cb = keras.callbacks.TensorBoard(run_logdir)
history = model.fit(X, y_reg.values, epochs=20, validation_split=0.1,batch_size=17)Train on 1226 samples, validate on 137 samples
Epoch 1/20
1226/1226 [==============================] - 2s 1ms/sample - loss: 304.6438 - val_loss: 167.6507
Epoch 2/20
1226/1226 [==============================] - 0s 206us/sample - loss: 124.7686 - val_loss: 107.0095
Epoch 3/20
1226/1226 [==============================] - 0s 216us/sample - loss: 74.7101 - val_loss: 55.4585
Epoch 4/20
1226/1226 [==============================] - 0s 213us/sample - loss: 42.4635 - val_loss: 36.7639
Epoch 5/20
1226/1226 [==============================] - 0s 205us/sample - loss: 31.9869 - val_loss: 30.2729
Epoch 6/20
1226/1226 [==============================] - 0s 204us/sample - loss: 29.4919 - val_loss: 30.1825
Epoch 7/20
1226/1226 [==============================] - 0s 209us/sample - loss: 28.2917 - val_loss: 28.5548
Epoch 8/20
1226/1226 [==============================] - 0s 227us/sample - loss: 27.5876 - val_loss: 29.3262
Epoch 9/20
1226/1226 [==============================] - 0s 201us/sample - loss: 26.9587 - val_loss: 32.3276
Epoch 10/20
1226/1226 [==============================] - 0s 209us/sample - loss: 26.9484 - val_loss: 27.8342
Epoch 11/20
1226/1226 [==============================] - 0s 215us/sample - loss: 26.3402 - val_loss: 27.7238
Epoch 12/20
1226/1226 [==============================] - 0s 218us/sample - loss: 26.1613 - val_loss: 30.1896
Epoch 13/20
1226/1226 [==============================] - 0s 200us/sample - loss: 25.8274 - val_loss: 27.4305
Epoch 14/20
1226/1226 [==============================] - 0s 204us/sample - loss: 25.5874 - val_loss: 29.6315
Epoch 15/20
1226/1226 [==============================] - 0s 219us/sample - loss: 25.3768 - val_loss: 26.8144
Epoch 16/20
1226/1226 [==============================] - 0s 213us/sample - loss: 25.2910 - val_loss: 29.3832
Epoch 17/20
1226/1226 [==============================] - 0s 203us/sample - loss: 25.1643 - val_loss: 33.1186
Epoch 18/20
1226/1226 [==============================] - 0s 204us/sample - loss: 25.0058 - val_loss: 27.6911
Epoch 19/20
1226/1226 [==============================] - 0s 201us/sample - loss: 24.9452 - val_loss: 27.7602
Epoch 20/20
1226/1226 [==============================] - 0s 208us/sample - loss: 24.7447 - val_loss: 27.9613
model = keras.models.load_model("keras_reg_model.h5")
checkpoint_cb = keras.callbacks.ModelCheckpoint("keras_reg_model.h5", save_best_only=True)
early_stopping_cb = keras.callbacks.EarlyStopping(patience=10,
                                                  restore_best_weights=True)

tensorboard_cb = keras.callbacks.TensorBoard(run_logdir)_____no_output_____history = model.fit(X, y_reg.values, epochs=30,
                    validation_split=0.1,batch_size=17,
                    callbacks=[checkpoint_cb, tensorboard_cb,early_stopping_cb])Train on 1226 samples, validate on 137 samples
Epoch 1/30
1226/1226 [==============================] - 1s 743us/sample - loss: 304.6438 - val_loss: 167.6507
Epoch 2/30
1226/1226 [==============================] - 0s 232us/sample - loss: 124.7686 - val_loss: 107.0095
Epoch 3/30
1226/1226 [==============================] - 0s 237us/sample - loss: 74.7101 - val_loss: 55.4585
Epoch 4/30
1226/1226 [==============================] - 0s 231us/sample - loss: 42.4635 - val_loss: 36.7639
Epoch 5/30
1226/1226 [==============================] - 0s 224us/sample - loss: 31.9869 - val_loss: 30.2729
Epoch 6/30
1226/1226 [==============================] - 0s 233us/sample - loss: 29.4919 - val_loss: 30.1825
Epoch 7/30
1226/1226 [==============================] - 0s 224us/sample - loss: 28.2917 - val_loss: 28.5548
Epoch 8/30
1226/1226 [==============================] - 0s 227us/sample - loss: 27.5876 - val_loss: 29.3262
Epoch 9/30
1226/1226 [==============================] - 0s 222us/sample - loss: 26.9587 - val_loss: 32.3276
Epoch 10/30
1226/1226 [==============================] - 0s 228us/sample - loss: 26.9484 - val_loss: 27.8342
Epoch 11/30
1226/1226 [==============================] - 0s 239us/sample - loss: 26.3402 - val_loss: 27.7238
Epoch 12/30
1226/1226 [==============================] - 0s 232us/sample - loss: 26.1613 - val_loss: 30.1896
Epoch 13/30
1226/1226 [==============================] - 0s 228us/sample - loss: 25.8274 - val_loss: 27.4305
Epoch 14/30
1226/1226 [==============================] - 0s 207us/sample - loss: 25.5874 - val_loss: 29.6315
Epoch 15/30
1226/1226 [==============================] - 0s 232us/sample - loss: 25.3768 - val_loss: 26.8144
Epoch 16/30
1226/1226 [==============================] - 0s 220us/sample - loss: 25.2910 - val_loss: 29.3832
Epoch 17/30
1226/1226 [==============================] - 0s 225us/sample - loss: 25.1643 - val_loss: 33.1186
Epoch 18/30
1226/1226 [==============================] - 0s 216us/sample - loss: 25.0058 - val_loss: 27.6911
Epoch 19/30
1226/1226 [==============================] - 0s 229us/sample - loss: 24.9452 - val_loss: 27.7602
Epoch 20/30
1226/1226 [==============================] - 0s 226us/sample - loss: 24.7447 - val_loss: 27.9613
Epoch 21/30
1226/1226 [==============================] - 0s 225us/sample - loss: 24.7244 - val_loss: 26.6106
Epoch 22/30
1226/1226 [==============================] - 0s 228us/sample - loss: 24.6352 - val_loss: 26.9562
Epoch 23/30
1226/1226 [==============================] - 0s 224us/sample - loss: 24.5690 - val_loss: 26.9450
Epoch 24/30
1226/1226 [==============================] - 0s 218us/sample - loss: 24.4692 - val_loss: 27.3700
Epoch 25/30
1226/1226 [==============================] - 0s 223us/sample - loss: 24.2931 - val_loss: 29.5783
Epoch 26/30
1226/1226 [==============================] - 0s 225us/sample - loss: 24.3962 - val_loss: 27.7449
Epoch 27/30
1226/1226 [==============================] - 0s 226us/sample - loss: 24.2374 - val_loss: 26.4139
Epoch 28/30
1226/1226 [==============================] - 0s 221us/sample - loss: 24.1882 - val_loss: 27.0749
Epoch 29/30
1226/1226 [==============================] - 0s 220us/sample - loss: 24.0705 - val_loss: 27.5794
Epoch 30/30
1226/1226 [==============================] - 0s 226us/sample - loss: 24.1285 - val_loss: 26.8302
model.summary()Model: "sequential"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
dense (Dense)                (None, 10)                720
_________________________________________________________________
dense_1 (Dense)              (None, 2)                 22
=================================================================
Total params: 742
Trainable params: 742
Non-trainable params: 0
_________________________________________________________________
model.predict(X)_____no_output_____loss = tf.losses.Huber(delta=1.0)
def build_model():
    model = keras.models.Sequential()
    model.add(keras.layers.Dense(10, activation="relu",input_shape=X.shape[1:]))
    model.add(keras.layers.Dense(2,activation= "relu"))
    optimizer = keras.optimizers.SGD(lr=1e-3)
    model.compile(loss="mse", optimizer=optimizer)
    return model_____no_output_____test_reg = keras.wrappers.scikit_learn.KerasRegressor(build_fn=build_model,
                                                      epochs=100,batch_size=10)_____no_output_____from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold

kfold = KFold(n_splits=5, random_state=42)
scores = cross_val_score(test_reg,X,y_reg.values,
                         scoring="neg_mean_squared_error",
                         cv=kfold)
keras_reg_scores = np.sqrt(-scores).mean()
print('keras_reg validation score(rmse) :', keras_reg_scores)Train on 1090 samples
Epoch 1/100
1090/1090 [==============================] - 1s 488us/sample - loss: 335.8716
Epoch 2/100
1090/1090 [==============================] - 0s 273us/sample - loss: 81.3029
Epoch 3/100
1090/1090 [==============================] - 0s 261us/sample - loss: 40.7986
Epoch 4/100
1090/1090 [==============================] - 0s 249us/sample - loss: 31.1064
Epoch 5/100
1090/1090 [==============================] - 0s 265us/sample - loss: 28.7177
Epoch 6/100
1090/1090 [==============================] - 0s 261us/sample - loss: 27.3328
Epoch 7/100
1090/1090 [==============================] - 0s 261us/sample - loss: 26.5178
Epoch 8/100
1090/1090 [==============================] - 0s 277us/sample - loss: 25.9202
Epoch 9/100
1090/1090 [==============================] - 0s 271us/sample - loss: 25.4792
Epoch 10/100
1090/1090 [==============================] - 0s 248us/sample - loss: 25.1202
Epoch 11/100
1090/1090 [==============================] - 0s 258us/sample - loss: 24.8458
Epoch 12/100
1090/1090 [==============================] - 0s 258us/sample - loss: 24.7765
Epoch 13/100
1090/1090 [==============================] - 0s 256us/sample - loss: 24.3302
Epoch 14/100
1090/1090 [==============================] - 0s 256us/sample - loss: 24.1942
Epoch 15/100
1090/1090 [==============================] - 0s 253us/sample - loss: 24.0402
Epoch 16/100
1090/1090 [==============================] - 0s 264us/sample - loss: 23.7036
Epoch 17/100
1090/1090 [==============================] - 0s 254us/sample - loss: 23.6958
Epoch 18/100
1090/1090 [==============================] - 0s 258us/sample - loss: 23.7157
Epoch 19/100
1090/1090 [==============================] - 0s 256us/sample - loss: 23.6536
Epoch 20/100
1090/1090 [==============================] - 0s 262us/sample - loss: 23.5082
Epoch 21/100
1090/1090 [==============================] - 0s 253us/sample - loss: 22.9129
Epoch 22/100
1090/1090 [==============================] - 0s 257us/sample - loss: 22.9616
Epoch 23/100
1090/1090 [==============================] - 0s 264us/sample - loss: 22.9901
Epoch 24/100
1090/1090 [==============================] - 0s 252us/sample - loss: 22.8724
Epoch 25/100
1090/1090 [==============================] - 0s 255us/sample - loss: 22.7633
Epoch 26/100
1090/1090 [==============================] - 0s 248us/sample - loss: 22.7069
Epoch 27/100
1090/1090 [==============================] - 0s 257us/sample - loss: 22.5744
Epoch 28/100
1090/1090 [==============================] - 0s 258us/sample - loss: 22.5338
Epoch 29/100
1090/1090 [==============================] - 0s 252us/sample - loss: 22.4520
Epoch 30/100
1090/1090 [==============================] - 0s 263us/sample - loss: 22.4668
Epoch 31/100
1090/1090 [==============================] - 0s 261us/sample - loss: 22.2346
Epoch 32/100
1090/1090 [==============================] - 0s 249us/sample - loss: 22.0118
Epoch 33/100
1090/1090 [==============================] - 0s 260us/sample - loss: 22.0659
Epoch 34/100
1090/1090 [==============================] - 0s 261us/sample - loss: 22.0340
Epoch 35/100
1090/1090 [==============================] - 0s 247us/sample - loss: 21.9264
Epoch 36/100
1090/1090 [==============================] - 0s 246us/sample - loss: 21.8389
Epoch 37/100
1090/1090 [==============================] - 0s 250us/sample - loss: 21.5979
Epoch 38/100
1090/1090 [==============================] - 0s 266us/sample - loss: 21.5976
Epoch 39/100
1090/1090 [==============================] - 0s 248us/sample - loss: 21.5910
Epoch 40/100
1090/1090 [==============================] - 0s 256us/sample - loss: 21.3505
Epoch 41/100
1090/1090 [==============================] - 0s 263us/sample - loss: 21.3945
Epoch 42/100
1090/1090 [==============================] - 0s 253us/sample - loss: 21.3568
Epoch 43/100
1090/1090 [==============================] - 0s 259us/sample - loss: 21.2775
Epoch 44/100
1090/1090 [==============================] - 0s 258us/sample - loss: 21.1418
Epoch 45/100
1090/1090 [==============================] - 0s 280us/sample - loss: 21.0339
Epoch 46/100
1090/1090 [==============================] - 0s 252us/sample - loss: 21.1170
Epoch 47/100
1090/1090 [==============================] - 0s 249us/sample - loss: 20.7896
Epoch 48/100
1090/1090 [==============================] - 0s 247us/sample - loss: 20.5942
Epoch 49/100
1090/1090 [==============================] - 0s 262us/sample - loss: 20.6155
Epoch 50/100
1090/1090 [==============================] - 0s 253us/sample - loss: 20.5923
Epoch 51/100
1090/1090 [==============================] - 0s 248us/sample - loss: 20.9118
Epoch 52/100
1090/1090 [==============================] - 0s 264us/sample - loss: 20.5306
Epoch 53/100
1090/1090 [==============================] - 0s 263us/sample - loss: 20.3612
Epoch 54/100
1090/1090 [==============================] - 0s 259us/sample - loss: 20.3830
Epoch 55/100
1090/1090 [==============================] - 0s 252us/sample - loss: 20.3039
Epoch 56/100
1090/1090 [==============================] - 0s 262us/sample - loss: 20.3318
Epoch 57/100
1090/1090 [==============================] - 0s 250us/sample - loss: 20.1646
Epoch 58/100
1090/1090 [==============================] - 0s 247us/sample - loss: 20.1122
Epoch 59/100
1090/1090 [==============================] - 0s 271us/sample - loss: 20.1879
Epoch 60/100
1090/1090 [==============================] - 0s 269us/sample - loss: 19.9720
Epoch 61/100
1090/1090 [==============================] - 0s 251us/sample - loss: 19.9722
Epoch 62/100
1090/1090 [==============================] - 0s 245us/sample - loss: 19.8850
Epoch 63/100
1090/1090 [==============================] - 0s 261us/sample - loss: 19.7366
Epoch 64/100
1090/1090 [==============================] - 0s 278us/sample - loss: 19.5298
Epoch 65/100
1090/1090 [==============================] - 0s 255us/sample - loss: 19.6525
Epoch 66/100
1090/1090 [==============================] - 0s 257us/sample - loss: 19.5912
Epoch 67/100
1090/1090 [==============================] - 0s 267us/sample - loss: 19.5346
Epoch 68/100
1090/1090 [==============================] - 0s 246us/sample - loss: 19.4070
Epoch 69/100
1090/1090 [==============================] - 0s 246us/sample - loss: 19.4436
Epoch 70/100
1090/1090 [==============================] - 0s 264us/sample - loss: 19.5145
Epoch 71/100
1090/1090 [==============================] - 0s 250us/sample - loss: 19.0292
Epoch 72/100
1090/1090 [==============================] - 0s 244us/sample - loss: 19.1920
Epoch 73/100
1090/1090 [==============================] - 0s 249us/sample - loss: 19.0616
Epoch 74/100
1090/1090 [==============================] - 0s 262us/sample - loss: 18.9851
Epoch 75/100
1090/1090 [==============================] - 0s 262us/sample - loss: 18.8995
Epoch 76/100
1090/1090 [==============================] - 0s 248us/sample - loss: 18.8099
Epoch 77/100
1090/1090 [==============================] - 0s 257us/sample - loss: 18.9455
Epoch 78/100
1090/1090 [==============================] - 0s 263us/sample - loss: 18.8997
Epoch 79/100
1090/1090 [==============================] - 0s 247us/sample - loss: 18.6093
Epoch 80/100
1090/1090 [==============================] - 0s 251us/sample - loss: 18.7327
Epoch 81/100
1090/1090 [==============================] - 0s 259us/sample - loss: 18.6760
Epoch 82/100
1090/1090 [==============================] - 0s 270us/sample - loss: 18.7278
Epoch 83/100
1090/1090 [==============================] - 0s 255us/sample - loss: 18.3679
Epoch 84/100
1090/1090 [==============================] - 0s 252us/sample - loss: 18.4630
Epoch 85/100
1090/1090 [==============================] - 0s 259us/sample - loss: 18.3706
Epoch 86/100
1090/1090 [==============================] - 0s 262us/sample - loss: 18.1305
Epoch 87/100
1090/1090 [==============================] - 0s 262us/sample - loss: 18.3170
Epoch 88/100
1090/1090 [==============================] - 0s 257us/sample - loss: 17.8707
Epoch 89/100
1090/1090 [==============================] - 0s 253us/sample - loss: 18.0565
Epoch 90/100
1090/1090 [==============================] - 0s 252us/sample - loss: 17.9522
Epoch 91/100
1090/1090 [==============================] - 0s 253us/sample - loss: 18.1533
Epoch 92/100
1090/1090 [==============================] - 0s 262us/sample - loss: 18.0634
Epoch 93/100
1090/1090 [==============================] - 0s 255us/sample - loss: 17.9320
Epoch 94/100
1090/1090 [==============================] - 0s 248us/sample - loss: 17.8901
Epoch 95/100
1090/1090 [==============================] - 0s 252us/sample - loss: 17.7975
Epoch 96/100
1090/1090 [==============================] - 0s 260us/sample - loss: 17.9024
Epoch 97/100
1090/1090 [==============================] - 0s 258us/sample - loss: 17.5982
Epoch 98/100
1090/1090 [==============================] - 0s 253us/sample - loss: 17.4361
Epoch 99/100
1090/1090 [==============================] - 0s 248us/sample - loss: 17.5776
Epoch 100/100
1090/1090 [==============================] - 0s 253us/sample - loss: 17.6304
Train on 1090 samples
Epoch 1/100
1090/1090 [==============================] - 1s 471us/sample - loss: 238.0372
Epoch 2/100
1090/1090 [==============================] - 0s 257us/sample - loss: 85.3631
Epoch 3/100
1090/1090 [==============================] - 0s 255us/sample - loss: 40.5903
Epoch 4/100
1090/1090 [==============================] - 0s 255us/sample - loss: 32.8079
Epoch 5/100
1090/1090 [==============================] - 0s 263us/sample - loss: 30.2156
Epoch 6/100
1090/1090 [==============================] - 0s 260us/sample - loss: 28.5280
Epoch 7/100
1090/1090 [==============================] - 0s 262us/sample - loss: 27.3913
Epoch 8/100
1090/1090 [==============================] - 0s 250us/sample - loss: 26.6397
Epoch 9/100
1090/1090 [==============================] - 0s 273us/sample - loss: 26.1527
Epoch 10/100
1090/1090 [==============================] - 0s 249us/sample - loss: 25.6350
Epoch 11/100
1090/1090 [==============================] - 0s 256us/sample - loss: 25.4060
Epoch 12/100
1090/1090 [==============================] - 0s 261us/sample - loss: 25.2910
Epoch 13/100
1090/1090 [==============================] - 0s 253us/sample - loss: 24.7118
Epoch 14/100
1090/1090 [==============================] - 0s 259us/sample - loss: 24.7358
Epoch 15/100
1090/1090 [==============================] - 0s 254us/sample - loss: 24.5072
Epoch 16/100
1090/1090 [==============================] - 0s 264us/sample - loss: 24.2889
Epoch 17/100
1090/1090 [==============================] - 0s 247us/sample - loss: 24.1314
Epoch 18/100
1090/1090 [==============================] - 0s 283us/sample - loss: 24.1700
Epoch 19/100
1090/1090 [==============================] - 0s 257us/sample - loss: 24.0684
Epoch 20/100
1090/1090 [==============================] - 0s 258us/sample - loss: 24.0679
Epoch 21/100
1090/1090 [==============================] - 0s 259us/sample - loss: 23.4429
Epoch 22/100
1090/1090 [==============================] - 0s 256us/sample - loss: 23.6779
Epoch 23/100
1090/1090 [==============================] - 0s 255us/sample - loss: 23.5564
Epoch 24/100
1090/1090 [==============================] - 0s 249us/sample - loss: 23.5181
Epoch 25/100
1090/1090 [==============================] - 0s 255us/sample - loss: 23.3729
Epoch 26/100
1090/1090 [==============================] - 0s 249us/sample - loss: 23.3122
Epoch 27/100
1090/1090 [==============================] - 0s 263us/sample - loss: 23.3541
Epoch 28/100
1090/1090 [==============================] - 0s 259us/sample - loss: 23.2591
Epoch 29/100
1090/1090 [==============================] - 0s 252us/sample - loss: 23.0789
Epoch 30/100
1090/1090 [==============================] - 0s 250us/sample - loss: 23.2328
Epoch 31/100
1090/1090 [==============================] - 0s 259us/sample - loss: 22.9755
Epoch 32/100
1090/1090 [==============================] - 0s 261us/sample - loss: 22.9443
Epoch 33/100
1090/1090 [==============================] - 0s 258us/sample - loss: 23.0249
Epoch 34/100
1090/1090 [==============================] - 0s 258us/sample - loss: 22.8543
Epoch 35/100
1090/1090 [==============================] - 0s 251us/sample - loss: 22.7513
Epoch 36/100
1090/1090 [==============================] - 0s 256us/sample - loss: 22.6892
Epoch 37/100
1090/1090 [==============================] - 0s 263us/sample - loss: 22.3505
Epoch 38/100
1090/1090 [==============================] - 0s 259us/sample - loss: 22.6228
Epoch 39/100
1090/1090 [==============================] - 0s 258us/sample - loss: 22.4837
Epoch 40/100
1090/1090 [==============================] - 0s 253us/sample - loss: 22.3213
Epoch 41/100
1090/1090 [==============================] - 0s 249us/sample - loss: 22.3585
Epoch 42/100
1090/1090 [==============================] - 0s 255us/sample - loss: 22.4056
Epoch 43/100
1090/1090 [==============================] - 0s 259us/sample - loss: 22.3745
Epoch 44/100
1090/1090 [==============================] - 0s 258us/sample - loss: 22.3351
Epoch 45/100
1090/1090 [==============================] - 0s 284us/sample - loss: 22.0056
Epoch 46/100
1090/1090 [==============================] - 0s 280us/sample - loss: 22.0332
Epoch 47/100
1090/1090 [==============================] - 0s 282us/sample - loss: 21.7444
Epoch 48/100
1090/1090 [==============================] - 0s 296us/sample - loss: 21.9438
Epoch 49/100
1090/1090 [==============================] - 0s 273us/sample - loss: 21.9085
Epoch 50/100
1090/1090 [==============================] - 0s 268us/sample - loss: 21.7220
Epoch 51/100
1090/1090 [==============================] - 0s 271us/sample - loss: 21.9380
Epoch 52/100
1090/1090 [==============================] - 0s 272us/sample - loss: 21.6785
Epoch 53/100
1090/1090 [==============================] - 0s 284us/sample - loss: 21.5570
Epoch 54/100
1090/1090 [==============================] - 0s 269us/sample - loss: 21.6365
Epoch 55/100
1090/1090 [==============================] - 0s 304us/sample - loss: 21.7063
Epoch 56/100
1090/1090 [==============================] - 0s 269us/sample - loss: 21.5404
Epoch 57/100
1090/1090 [==============================] - 0s 270us/sample - loss: 21.5926
Epoch 58/100
1090/1090 [==============================] - 0s 272us/sample - loss: 21.2921
Epoch 59/100
1090/1090 [==============================] - 0s 279us/sample - loss: 21.3089
Epoch 60/100
1090/1090 [==============================] - 0s 277us/sample - loss: 21.2134
Epoch 61/100
1090/1090 [==============================] - 0s 275us/sample - loss: 21.2621
Epoch 62/100
1090/1090 [==============================] - 0s 265us/sample - loss: 21.4557
Epoch 63/100
1090/1090 [==============================] - 0s 258us/sample - loss: 21.1663
Epoch 64/100
1090/1090 [==============================] - 0s 258us/sample - loss: 21.0882
Epoch 65/100
1090/1090 [==============================] - 0s 252us/sample - loss: 21.1502
Epoch 66/100
1090/1090 [==============================] - 0s 256us/sample - loss: 21.0618
Epoch 67/100
1090/1090 [==============================] - 0s 254us/sample - loss: 21.0366
Epoch 68/100
1090/1090 [==============================] - 0s 254us/sample - loss: 20.9291
Epoch 69/100
1090/1090 [==============================] - 0s 262us/sample - loss: 20.8974
Epoch 70/100
1090/1090 [==============================] - 0s 253us/sample - loss: 21.1096
Epoch 71/100
1090/1090 [==============================] - 0s 250us/sample - loss: 20.5826
Epoch 72/100
1090/1090 [==============================] - 0s 251us/sample - loss: 20.6927
Epoch 73/100
1090/1090 [==============================] - 0s 262us/sample - loss: 20.5664
Epoch 74/100
1090/1090 [==============================] - 0s 254us/sample - loss: 20.5949
Epoch 75/100
1090/1090 [==============================] - 0s 254us/sample - loss: 20.6077
Epoch 76/100
1090/1090 [==============================] - 0s 271us/sample - loss: 20.4299
Epoch 77/100
1090/1090 [==============================] - 0s 258us/sample - loss: 20.7458
Epoch 78/100
1090/1090 [==============================] - 0s 251us/sample - loss: 20.4899
Epoch 79/100
1090/1090 [==============================] - 0s 255us/sample - loss: 20.4888
Epoch 80/100
1090/1090 [==============================] - 0s 257us/sample - loss: 20.3817
Epoch 81/100
1090/1090 [==============================] - 0s 258us/sample - loss: 20.3806
Epoch 82/100
1090/1090 [==============================] - 0s 257us/sample - loss: 20.2445
Epoch 83/100
1090/1090 [==============================] - 0s 259us/sample - loss: 20.3297
Epoch 84/100
1090/1090 [==============================] - 0s 260us/sample - loss: 20.1845
Epoch 85/100
1090/1090 [==============================] - 0s 254us/sample - loss: 20.2812
Epoch 86/100
1090/1090 [==============================] - 0s 253us/sample - loss: 20.3790
Epoch 87/100
1090/1090 [==============================] - 0s 258us/sample - loss: 20.1874
Epoch 88/100
1090/1090 [==============================] - 0s 248us/sample - loss: 19.7982
Epoch 89/100
1090/1090 [==============================] - 0s 263us/sample - loss: 20.0393
Epoch 90/100
1090/1090 [==============================] - 0s 249us/sample - loss: 19.8513
Epoch 91/100
1090/1090 [==============================] - 0s 262us/sample - loss: 20.0307
Epoch 92/100
1090/1090 [==============================] - 0s 281us/sample - loss: 19.8457
Epoch 93/100
1090/1090 [==============================] - 0s 254us/sample - loss: 19.8931
Epoch 94/100
1090/1090 [==============================] - 0s 247us/sample - loss: 19.9435
Epoch 95/100
1090/1090 [==============================] - 0s 255us/sample - loss: 19.7643
Epoch 96/100
1090/1090 [==============================] - 0s 257us/sample - loss: 19.8856
Epoch 97/100
1090/1090 [==============================] - 0s 255us/sample - loss: 19.3876
Epoch 98/100
1090/1090 [==============================] - 0s 261us/sample - loss: 19.6843
Epoch 99/100
1090/1090 [==============================] - 0s 263us/sample - loss: 19.6761
Epoch 100/100
1090/1090 [==============================] - 0s 251us/sample - loss: 19.7510
Train on 1090 samples
Epoch 1/100
1090/1090 [==============================] - 1s 472us/sample - loss: 317.0615
Epoch 2/100
1090/1090 [==============================] - 0s 255us/sample - loss: 124.1055
Epoch 3/100
1090/1090 [==============================] - 0s 251us/sample - loss: 42.5644
Epoch 4/100
1090/1090 [==============================] - 0s 264us/sample - loss: 30.4484
Epoch 5/100
1090/1090 [==============================] - 0s 252us/sample - loss: 28.0201
Epoch 6/100
1090/1090 [==============================] - 0s 254us/sample - loss: 27.0589
Epoch 7/100
1090/1090 [==============================] - 0s 261us/sample - loss: 26.5594
Epoch 8/100
1090/1090 [==============================] - 0s 262us/sample - loss: 25.9593
Epoch 9/100
1090/1090 [==============================] - 0s 257us/sample - loss: 25.6269
Epoch 10/100
1090/1090 [==============================] - 0s 251us/sample - loss: 25.2206
Epoch 11/100
1090/1090 [==============================] - 0s 268us/sample - loss: 25.1941
Epoch 12/100
1090/1090 [==============================] - 0s 260us/sample - loss: 25.0543
Epoch 13/100
1090/1090 [==============================] - 0s 254us/sample - loss: 24.5646
Epoch 14/100
1090/1090 [==============================] - 0s 247us/sample - loss: 24.6978
Epoch 15/100
1090/1090 [==============================] - 0s 262us/sample - loss: 24.6997
Epoch 16/100
1090/1090 [==============================] - 0s 258us/sample - loss: 24.3461
Epoch 17/100
1090/1090 [==============================] - 0s 255us/sample - loss: 24.4227
Epoch 18/100
1090/1090 [==============================] - 0s 268us/sample - loss: 24.2196
Epoch 19/100
1090/1090 [==============================] - 0s 261us/sample - loss: 24.0903
Epoch 20/100
1090/1090 [==============================] - 0s 258us/sample - loss: 24.1485
Epoch 21/100
1090/1090 [==============================] - 0s 252us/sample - loss: 23.9244
Epoch 22/100
1090/1090 [==============================] - 0s 255us/sample - loss: 23.8325
Epoch 23/100
1090/1090 [==============================] - 0s 249us/sample - loss: 23.8695
Epoch 24/100
1090/1090 [==============================] - 0s 251us/sample - loss: 23.6415
Epoch 25/100
1090/1090 [==============================] - 0s 261us/sample - loss: 23.6279
Epoch 26/100
1090/1090 [==============================] - 0s 262us/sample - loss: 23.3196
Epoch 27/100
1090/1090 [==============================] - 0s 261us/sample - loss: 23.6976
Epoch 28/100
1090/1090 [==============================] - 0s 283us/sample - loss: 23.3455
Epoch 29/100
1090/1090 [==============================] - 0s 267us/sample - loss: 23.2456
Epoch 30/100
1090/1090 [==============================] - 0s 258us/sample - loss: 23.0641
Epoch 31/100
1090/1090 [==============================] - 0s 260us/sample - loss: 23.0690
Epoch 32/100
1090/1090 [==============================] - 0s 251us/sample - loss: 23.0364
Epoch 33/100
1090/1090 [==============================] - 0s 266us/sample - loss: 22.9672
Epoch 34/100
1090/1090 [==============================] - 0s 248us/sample - loss: 22.6062
Epoch 35/100
1090/1090 [==============================] - 0s 256us/sample - loss: 22.6944
Epoch 36/100
1090/1090 [==============================] - 0s 270us/sample - loss: 22.7007
Epoch 37/100
1090/1090 [==============================] - 0s 248us/sample - loss: 22.5130
Epoch 38/100
1090/1090 [==============================] - 0s 252us/sample - loss: 22.4972
Epoch 39/100
1090/1090 [==============================] - 0s 258us/sample - loss: 22.2039
Epoch 40/100
1090/1090 [==============================] - 0s 261us/sample - loss: 22.3539
Epoch 41/100
1090/1090 [==============================] - 0s 252us/sample - loss: 22.2883
Epoch 42/100
1090/1090 [==============================] - 0s 258us/sample - loss: 22.2817
Epoch 43/100
1090/1090 [==============================] - 0s 266us/sample - loss: 22.1560
Epoch 44/100
1090/1090 [==============================] - 0s 250us/sample - loss: 22.0481
Epoch 45/100
1090/1090 [==============================] - 0s 253us/sample - loss: 21.9184
Epoch 46/100
1090/1090 [==============================] - 0s 256us/sample - loss: 21.8969
Epoch 47/100
1090/1090 [==============================] - 0s 260us/sample - loss: 21.6651
Epoch 48/100
1090/1090 [==============================] - 0s 250us/sample - loss: 21.7248
Epoch 49/100
1090/1090 [==============================] - 0s 254us/sample - loss: 21.6959
Epoch 50/100
1090/1090 [==============================] - 0s 258us/sample - loss: 21.6594
Epoch 51/100
1090/1090 [==============================] - 0s 254us/sample - loss: 21.6206
Epoch 52/100
1090/1090 [==============================] - 0s 252us/sample - loss: 21.3549
Epoch 53/100
1090/1090 [==============================] - 0s 262us/sample - loss: 21.3316
Epoch 54/100
1090/1090 [==============================] - 0s 258us/sample - loss: 21.3767
Epoch 55/100
1090/1090 [==============================] - 0s 259us/sample - loss: 21.4221
Epoch 56/100
1090/1090 [==============================] - 0s 255us/sample - loss: 21.1575
Epoch 57/100
1090/1090 [==============================] - 0s 250us/sample - loss: 21.0886
Epoch 58/100
1090/1090 [==============================] - 0s 265us/sample - loss: 20.9794
Epoch 59/100
1090/1090 [==============================] - 0s 250us/sample - loss: 20.8848
Epoch 60/100
1090/1090 [==============================] - 0s 256us/sample - loss: 21.0170
Epoch 61/100
1090/1090 [==============================] - 0s 256us/sample - loss: 20.9159
Epoch 62/100
1090/1090 [==============================] - 0s 254us/sample - loss: 20.9163
Epoch 63/100
1090/1090 [==============================] - 0s 269us/sample - loss: 20.7118
Epoch 64/100
1090/1090 [==============================] - 0s 256us/sample - loss: 20.6621
Epoch 65/100
1090/1090 [==============================] - 0s 263us/sample - loss: 20.4232
Epoch 66/100
1090/1090 [==============================] - 0s 247us/sample - loss: 20.5354
Epoch 67/100
1090/1090 [==============================] - 0s 267us/sample - loss: 20.4189
Epoch 68/100
1090/1090 [==============================] - 0s 265us/sample - loss: 20.3006
Epoch 69/100
1090/1090 [==============================] - 0s 275us/sample - loss: 20.1646
Epoch 70/100
1090/1090 [==============================] - 0s 267us/sample - loss: 20.2975
Epoch 71/100
1090/1090 [==============================] - 0s 253us/sample - loss: 19.8865
Epoch 72/100
1090/1090 [==============================] - 0s 257us/sample - loss: 19.9355
Epoch 73/100
1090/1090 [==============================] - 0s 257us/sample - loss: 19.9330
Epoch 74/100
1090/1090 [==============================] - 0s 261us/sample - loss: 19.9326
Epoch 75/100
1090/1090 [==============================] - 0s 249us/sample - loss: 19.7571
Epoch 76/100
1090/1090 [==============================] - 0s 256us/sample - loss: 19.6404
Epoch 77/100
1090/1090 [==============================] - 0s 255us/sample - loss: 19.6131
Epoch 78/100
1090/1090 [==============================] - 0s 254us/sample - loss: 19.7175
Epoch 79/100
1090/1090 [==============================] - 0s 255us/sample - loss: 19.7308
Epoch 80/100
1090/1090 [==============================] - 0s 262us/sample - loss: 19.4163
Epoch 81/100
1090/1090 [==============================] - 0s 256us/sample - loss: 19.2540
Epoch 82/100
1090/1090 [==============================] - 0s 251us/sample - loss: 19.2614
Epoch 83/100
1090/1090 [==============================] - 0s 269us/sample - loss: 19.2687
Epoch 84/100
1090/1090 [==============================] - 0s 251us/sample - loss: 19.2890
Epoch 85/100
1090/1090 [==============================] - 0s 254us/sample - loss: 19.1219
Epoch 86/100
1090/1090 [==============================] - 0s 261us/sample - loss: 19.2922
Epoch 87/100
1090/1090 [==============================] - 0s 268us/sample - loss: 19.0258
Epoch 88/100
1090/1090 [==============================] - 0s 251us/sample - loss: 18.9135
Epoch 89/100
1090/1090 [==============================] - 0s 258us/sample - loss: 18.9703
Epoch 90/100
1090/1090 [==============================] - 0s 253us/sample - loss: 18.9106
Epoch 91/100
1090/1090 [==============================] - 0s 261us/sample - loss: 18.7400
Epoch 92/100
1090/1090 [==============================] - 0s 257us/sample - loss: 18.5174
Epoch 93/100
1090/1090 [==============================] - 0s 259us/sample - loss: 18.6575
Epoch 94/100
1090/1090 [==============================] - 0s 257us/sample - loss: 18.4756
Epoch 95/100
1090/1090 [==============================] - 0s 261us/sample - loss: 18.3682
Epoch 96/100
1090/1090 [==============================] - 0s 256us/sample - loss: 18.3143
Epoch 97/100
1090/1090 [==============================] - 0s 271us/sample - loss: 18.1198
Epoch 98/100
1090/1090 [==============================] - 0s 265us/sample - loss: 18.0379
Epoch 99/100
1090/1090 [==============================] - 0s 245us/sample - loss: 18.1227
Epoch 100/100
1090/1090 [==============================] - 0s 245us/sample - loss: 18.1793
Train on 1091 samples
Epoch 1/100
1091/1091 [==============================] - 1s 582us/sample - loss: 293.5190
Epoch 2/100
1091/1091 [==============================] - 0s 314us/sample - loss: 102.3124
Epoch 3/100
1091/1091 [==============================] - 0s 366us/sample - loss: 41.9562
Epoch 4/100
1091/1091 [==============================] - 0s 321us/sample - loss: 31.6527
Epoch 5/100
1091/1091 [==============================] - 0s 334us/sample - loss: 28.9416
Epoch 6/100
1091/1091 [==============================] - 0s 344us/sample - loss: 27.5589
Epoch 7/100
1091/1091 [==============================] - 0s 323us/sample - loss: 26.9561
Epoch 8/100
1091/1091 [==============================] - 0s 326us/sample - loss: 26.6599
Epoch 9/100
1091/1091 [==============================] - 0s 328us/sample - loss: 26.0561
Epoch 10/100
1091/1091 [==============================] - 0s 317us/sample - loss: 25.7824
Epoch 11/100
1091/1091 [==============================] - 0s 341us/sample - loss: 25.6083
Epoch 12/100
1091/1091 [==============================] - 0s 328us/sample - loss: 25.7713
Epoch 13/100
1091/1091 [==============================] - 0s 326us/sample - loss: 25.3421
Epoch 14/100
1091/1091 [==============================] - 0s 336us/sample - loss: 24.9975
Epoch 15/100
1091/1091 [==============================] - 0s 327us/sample - loss: 24.9004
Epoch 16/100
1091/1091 [==============================] - 0s 328us/sample - loss: 24.7993
Epoch 17/100
1091/1091 [==============================] - 0s 336us/sample - loss: 24.8041
Epoch 18/100
1091/1091 [==============================] - 0s 317us/sample - loss: 24.5038
Epoch 19/100
1091/1091 [==============================] - 0s 315us/sample - loss: 24.3633
Epoch 20/100
1091/1091 [==============================] - 0s 333us/sample - loss: 24.4076
Epoch 21/100
1091/1091 [==============================] - 0s 320us/sample - loss: 24.2303
Epoch 22/100
1091/1091 [==============================] - 0s 319us/sample - loss: 23.8825
Epoch 23/100
1091/1091 [==============================] - 0s 333us/sample - loss: 24.0933
Epoch 24/100
1091/1091 [==============================] - 0s 334us/sample - loss: 24.1077
Epoch 25/100
1091/1091 [==============================] - 0s 318us/sample - loss: 24.3098
Epoch 26/100
1091/1091 [==============================] - 0s 335us/sample - loss: 23.8287
Epoch 27/100
1091/1091 [==============================] - 0s 326us/sample - loss: 23.7544
Epoch 28/100
1091/1091 [==============================] - 0s 344us/sample - loss: 23.5906
Epoch 29/100
1091/1091 [==============================] - 0s 323us/sample - loss: 23.4955
Epoch 30/100
1091/1091 [==============================] - 0s 327us/sample - loss: 23.4583
Epoch 31/100
1091/1091 [==============================] - 0s 342us/sample - loss: 23.3857
Epoch 32/100
1091/1091 [==============================] - 0s 328us/sample - loss: 23.3769
Epoch 33/100
1091/1091 [==============================] - 0s 356us/sample - loss: 23.1755
Epoch 34/100
1091/1091 [==============================] - 0s 332us/sample - loss: 23.3504
Epoch 35/100
1091/1091 [==============================] - 0s 316us/sample - loss: 22.9549
Epoch 36/100
1091/1091 [==============================] - 0s 322us/sample - loss: 24.5163
Epoch 37/100
1091/1091 [==============================] - 0s 329us/sample - loss: 23.2387
Epoch 38/100
1091/1091 [==============================] - 0s 333us/sample - loss: 23.1826
Epoch 39/100
1091/1091 [==============================] - 0s 333us/sample - loss: 22.8933
Epoch 40/100
1091/1091 [==============================] - 0s 343us/sample - loss: 22.7905
Epoch 41/100
1091/1091 [==============================] - 0s 321us/sample - loss: 22.9319
Epoch 42/100
1091/1091 [==============================] - 0s 340us/sample - loss: 23.2771
Epoch 43/100
1091/1091 [==============================] - 0s 324us/sample - loss: 23.1421
Epoch 44/100
1091/1091 [==============================] - 0s 333us/sample - loss: 22.4662
Epoch 45/100
1091/1091 [==============================] - 0s 340us/sample - loss: 22.6055
Epoch 46/100
1091/1091 [==============================] - 0s 331us/sample - loss: 22.2443
Epoch 47/100
1091/1091 [==============================] - 0s 327us/sample - loss: 22.5794
Epoch 48/100
1091/1091 [==============================] - 0s 333us/sample - loss: 22.5449
Epoch 49/100
1091/1091 [==============================] - 0s 320us/sample - loss: 22.5342
Epoch 50/100
1091/1091 [==============================] - 0s 327us/sample - loss: 22.2559
Epoch 51/100
1091/1091 [==============================] - 0s 333us/sample - loss: 22.2341
Epoch 52/100
1091/1091 [==============================] - 0s 321us/sample - loss: 22.1577
Epoch 53/100
1091/1091 [==============================] - 0s 326us/sample - loss: 21.7620
Epoch 54/100
1091/1091 [==============================] - 0s 336us/sample - loss: 22.1736
Epoch 55/100
1091/1091 [==============================] - 0s 325us/sample - loss: 21.8986
Epoch 56/100
1091/1091 [==============================] - 0s 329us/sample - loss: 22.0401
Epoch 57/100
1091/1091 [==============================] - 0s 346us/sample - loss: 21.8463
Epoch 58/100
1091/1091 [==============================] - 0s 337us/sample - loss: 21.8051
Epoch 59/100
1091/1091 [==============================] - 0s 325us/sample - loss: 21.7401
Epoch 60/100
1091/1091 [==============================] - 0s 318us/sample - loss: 21.8307
Epoch 61/100
1091/1091 [==============================] - 0s 322us/sample - loss: 21.7472
Epoch 62/100
1091/1091 [==============================] - 0s 342us/sample - loss: 21.3666
Epoch 63/100
1091/1091 [==============================] - 0s 340us/sample - loss: 21.4774
Epoch 64/100
1091/1091 [==============================] - 0s 321us/sample - loss: 21.4412
Epoch 65/100
1091/1091 [==============================] - 0s 331us/sample - loss: 21.3247
Epoch 66/100
1091/1091 [==============================] - 0s 322us/sample - loss: 21.1749
Epoch 67/100
1091/1091 [==============================] - 0s 331us/sample - loss: 21.2618
Epoch 68/100
1091/1091 [==============================] - 0s 330us/sample - loss: 21.3072
Epoch 69/100
1091/1091 [==============================] - 0s 321us/sample - loss: 21.1444
Epoch 70/100
1091/1091 [==============================] - 0s 322us/sample - loss: 20.9903
Epoch 71/100
1091/1091 [==============================] - 0s 327us/sample - loss: 21.0778
Epoch 72/100
1091/1091 [==============================] - 0s 328us/sample - loss: 20.8552
Epoch 73/100
1091/1091 [==============================] - 0s 320us/sample - loss: 21.3333
Epoch 74/100
1091/1091 [==============================] - 0s 350us/sample - loss: 20.9598
Epoch 75/100
1091/1091 [==============================] - 0s 323us/sample - loss: 20.6685
Epoch 76/100
1091/1091 [==============================] - 0s 350us/sample - loss: 20.7292
Epoch 77/100
1091/1091 [==============================] - 0s 321us/sample - loss: 20.5049
Epoch 78/100
1091/1091 [==============================] - 0s 327us/sample - loss: 20.7664
Epoch 79/100
1091/1091 [==============================] - 0s 335us/sample - loss: 20.3095
Epoch 80/100
1091/1091 [==============================] - 0s 337us/sample - loss: 20.4330
Epoch 81/100
1091/1091 [==============================] - 0s 320us/sample - loss: 20.4886
Epoch 82/100
1091/1091 [==============================] - 0s 330us/sample - loss: 20.5742
Epoch 83/100
1091/1091 [==============================] - 0s 321us/sample - loss: 20.4676
Epoch 84/100
1091/1091 [==============================] - 0s 326us/sample - loss: 20.4120
Epoch 85/100
1091/1091 [==============================] - 0s 346us/sample - loss: 20.3020
Epoch 86/100
1091/1091 [==============================] - 0s 332us/sample - loss: 19.9868
Epoch 87/100
1091/1091 [==============================] - 0s 310us/sample - loss: 20.0174
Epoch 88/100
1091/1091 [==============================] - 0s 332us/sample - loss: 19.6847
Epoch 89/100
1091/1091 [==============================] - 0s 323us/sample - loss: 19.9564
Epoch 90/100
1091/1091 [==============================] - 0s 327us/sample - loss: 19.7811
Epoch 91/100
1091/1091 [==============================] - 0s 328us/sample - loss: 19.7766
Epoch 92/100
1091/1091 [==============================] - 0s 355us/sample - loss: 19.9111
Epoch 93/100
1091/1091 [==============================] - 0s 329us/sample - loss: 19.9788
Epoch 94/100
1091/1091 [==============================] - 0s 326us/sample - loss: 19.8899
Epoch 95/100
1091/1091 [==============================] - 0s 331us/sample - loss: 19.9413
Epoch 96/100
1091/1091 [==============================] - 0s 329us/sample - loss: 19.2910
Epoch 97/100
1091/1091 [==============================] - 0s 334us/sample - loss: 19.5320
Epoch 98/100
1091/1091 [==============================] - 0s 335us/sample - loss: 19.5164
Epoch 99/100
1091/1091 [==============================] - 0s 331us/sample - loss: 19.5648
Epoch 100/100
1091/1091 [==============================] - 0s 328us/sample - loss: 19.2125
Train on 1091 samples
Epoch 1/100
1091/1091 [==============================] - 1s 710us/sample - loss: 330.7261
Epoch 2/100
1091/1091 [==============================] - 0s 331us/sample - loss: 165.7597
Epoch 3/100
1091/1091 [==============================] - 0s 327us/sample - loss: 45.5473
Epoch 4/100
1091/1091 [==============================] - 0s 332us/sample - loss: 33.0987
Epoch 5/100
1091/1091 [==============================] - 0s 311us/sample - loss: 30.0707
Epoch 6/100
1091/1091 [==============================] - 0s 342us/sample - loss: 28.4020
Epoch 7/100
1091/1091 [==============================] - 0s 320us/sample - loss: 27.6146
Epoch 8/100
1091/1091 [==============================] - 0s 318us/sample - loss: 27.0097
Epoch 9/100
1091/1091 [==============================] - 0s 328us/sample - loss: 26.2880
Epoch 10/100
1091/1091 [==============================] - 0s 325us/sample - loss: 26.1073
Epoch 11/100
1091/1091 [==============================] - 0s 326us/sample - loss: 25.8519
Epoch 12/100
1091/1091 [==============================] - 0s 333us/sample - loss: 25.7252
Epoch 13/100
1091/1091 [==============================] - 0s 327us/sample - loss: 25.3867
Epoch 14/100
1091/1091 [==============================] - 0s 328us/sample - loss: 25.1208
Epoch 15/100
1091/1091 [==============================] - 0s 333us/sample - loss: 24.8806
Epoch 16/100
1091/1091 [==============================] - 0s 334us/sample - loss: 24.8233
Epoch 17/100
1091/1091 [==============================] - 0s 328us/sample - loss: 24.5801
Epoch 18/100
1091/1091 [==============================] - 0s 331us/sample - loss: 24.5226
Epoch 19/100
1091/1091 [==============================] - 0s 325us/sample - loss: 24.2111
Epoch 20/100
1091/1091 [==============================] - 0s 360us/sample - loss: 24.3547
Epoch 21/100
1091/1091 [==============================] - 0s 335us/sample - loss: 24.1348
Epoch 22/100
1091/1091 [==============================] - 0s 320us/sample - loss: 23.7303
Epoch 23/100
1091/1091 [==============================] - 0s 345us/sample - loss: 23.8608
Epoch 24/100
1091/1091 [==============================] - 0s 324us/sample - loss: 23.6721
Epoch 25/100
1091/1091 [==============================] - 0s 318us/sample - loss: 23.8080
Epoch 26/100
1091/1091 [==============================] - 0s 336us/sample - loss: 23.5089
Epoch 27/100
1091/1091 [==============================] - 0s 328us/sample - loss: 23.3230
Epoch 28/100
1091/1091 [==============================] - 0s 320us/sample - loss: 23.1545
Epoch 29/100
1091/1091 [==============================] - 0s 341us/sample - loss: 23.1081
Epoch 30/100
1091/1091 [==============================] - 0s 325us/sample - loss: 23.1231
Epoch 31/100
1091/1091 [==============================] - 0s 340us/sample - loss: 23.0129
Epoch 32/100
1091/1091 [==============================] - 0s 324us/sample - loss: 22.9187
Epoch 33/100
1091/1091 [==============================] - 0s 327us/sample - loss: 22.9058
Epoch 34/100
1091/1091 [==============================] - 0s 340us/sample - loss: 22.7379
Epoch 35/100
1091/1091 [==============================] - 0s 322us/sample - loss: 22.6239
Epoch 36/100
1091/1091 [==============================] - 0s 324us/sample - loss: 22.5113
Epoch 37/100
1091/1091 [==============================] - 0s 343us/sample - loss: 22.5344
Epoch 38/100
1091/1091 [==============================] - 0s 324us/sample - loss: 22.4370
Epoch 39/100
1091/1091 [==============================] - 0s 329us/sample - loss: 22.2197
Epoch 40/100
1091/1091 [==============================] - 0s 328us/sample - loss: 22.0829
Epoch 41/100
1091/1091 [==============================] - 0s 354us/sample - loss: 22.1682
Epoch 42/100
1091/1091 [==============================] - 0s 326us/sample - loss: 22.3550
Epoch 43/100
1091/1091 [==============================] - 0s 335us/sample - loss: 22.2626
Epoch 44/100
1091/1091 [==============================] - 0s 328us/sample - loss: 21.8886
Epoch 45/100
1091/1091 [==============================] - 0s 337us/sample - loss: 21.8527
Epoch 46/100
1091/1091 [==============================] - 0s 327us/sample - loss: 21.6807
Epoch 47/100
1091/1091 [==============================] - 0s 326us/sample - loss: 21.7083
Epoch 48/100
1091/1091 [==============================] - 0s 336us/sample - loss: 21.6705
Epoch 49/100
1091/1091 [==============================] - 0s 320us/sample - loss: 21.5014
Epoch 50/100
1091/1091 [==============================] - 0s 348us/sample - loss: 21.4420
Epoch 51/100
1091/1091 [==============================] - 0s 328us/sample - loss: 21.5332
Epoch 52/100
1091/1091 [==============================] - 0s 320us/sample - loss: 21.2957
Epoch 53/100
1091/1091 [==============================] - 0s 321us/sample - loss: 21.0995
Epoch 54/100
1091/1091 [==============================] - 0s 333us/sample - loss: 21.2802
Epoch 55/100
1091/1091 [==============================] - 0s 322us/sample - loss: 21.0299
Epoch 56/100
1091/1091 [==============================] - 0s 329us/sample - loss: 21.1524
Epoch 57/100
1091/1091 [==============================] - 0s 343us/sample - loss: 20.9769
Epoch 58/100
1091/1091 [==============================] - 0s 322us/sample - loss: 20.8613
Epoch 59/100
1091/1091 [==============================] - 0s 329us/sample - loss: 20.7930
Epoch 60/100
1091/1091 [==============================] - 0s 324us/sample - loss: 20.8217
Epoch 61/100
1091/1091 [==============================] - 0s 326us/sample - loss: 20.7406
Epoch 62/100
1091/1091 [==============================] - 0s 352us/sample - loss: 20.5065
Epoch 63/100
1091/1091 [==============================] - 0s 323us/sample - loss: 20.5210
Epoch 64/100
1091/1091 [==============================] - 0s 326us/sample - loss: 20.4952
Epoch 65/100
1091/1091 [==============================] - 0s 339us/sample - loss: 20.5468
Epoch 66/100
1091/1091 [==============================] - 0s 322us/sample - loss: 20.3620
Epoch 67/100
1091/1091 [==============================] - 0s 318us/sample - loss: 20.3941
Epoch 68/100
1091/1091 [==============================] - 0s 328us/sample - loss: 20.3338
Epoch 69/100
1091/1091 [==============================] - 0s 331us/sample - loss: 20.1615
Epoch 70/100
1091/1091 [==============================] - 0s 323us/sample - loss: 19.9829
Epoch 71/100
1091/1091 [==============================] - 0s 339us/sample - loss: 20.0384
Epoch 72/100
1091/1091 [==============================] - 0s 318us/sample - loss: 20.0639
Epoch 73/100
1091/1091 [==============================] - 0s 323us/sample - loss: 19.8535
Epoch 74/100
1091/1091 [==============================] - 0s 348us/sample - loss: 20.0235
Epoch 75/100
1091/1091 [==============================] - 0s 323us/sample - loss: 19.8053
Epoch 76/100
1091/1091 [==============================] - 0s 330us/sample - loss: 19.7179
Epoch 77/100
1091/1091 [==============================] - 0s 335us/sample - loss: 19.7592
Epoch 78/100
1091/1091 [==============================] - 0s 321us/sample - loss: 19.7173
Epoch 79/100
1091/1091 [==============================] - 0s 350us/sample - loss: 19.6548
Epoch 80/100
1091/1091 [==============================] - 0s 321us/sample - loss: 19.5355
Epoch 81/100
1091/1091 [==============================] - 0s 324us/sample - loss: 19.6989
Epoch 82/100
1091/1091 [==============================] - 0s 338us/sample - loss: 19.4739
Epoch 83/100
1091/1091 [==============================] - 0s 328us/sample - loss: 19.5300
Epoch 84/100
1091/1091 [==============================] - 0s 331us/sample - loss: 19.4741
Epoch 85/100
1091/1091 [==============================] - 0s 332us/sample - loss: 19.2986
Epoch 86/100
1091/1091 [==============================] - 0s 318us/sample - loss: 19.1341
Epoch 87/100
1091/1091 [==============================] - 0s 325us/sample - loss: 19.1268
Epoch 88/100
1091/1091 [==============================] - 0s 341us/sample - loss: 19.1423
Epoch 89/100
1091/1091 [==============================] - 0s 319us/sample - loss: 18.8871
Epoch 90/100
1091/1091 [==============================] - 0s 326us/sample - loss: 19.1115
Epoch 91/100
1091/1091 [==============================] - 0s 353us/sample - loss: 18.9430
Epoch 92/100
1091/1091 [==============================] - 0s 324us/sample - loss: 18.7555
Epoch 93/100
1091/1091 [==============================] - 0s 324us/sample - loss: 18.7939
Epoch 94/100
1091/1091 [==============================] - 0s 329us/sample - loss: 18.7888
Epoch 95/100
1091/1091 [==============================] - 0s 324us/sample - loss: 18.7112
Epoch 96/100
1091/1091 [==============================] - 0s 322us/sample - loss: 18.4436
Epoch 97/100
1091/1091 [==============================] - 0s 337us/sample - loss: 18.4395
Epoch 98/100
1091/1091 [==============================] - 0s 328us/sample - loss: 18.4737
Epoch 99/100
1091/1091 [==============================] - 0s 334us/sample - loss: 18.3219
Epoch 100/100
1091/1091 [==============================] - 0s 319us/sample - loss: 18.2769
keras_reg validation score(rmse) : 5.648804647379717

hyperparameter tuning
---_____no_output_____

keras.backend.clear_session()
np.random.seed(42)
tf.random.set_seed(42)_____no_output_____loss = tf.losses.Huber(delta=1.0)
def build_model(n_hidden=1, n_neurons=30, learning_rate=3e-3, input_shape=[71]):
    model = keras.models.Sequential()
    model.add(keras.layers.InputLayer(input_shape=input_shape))
    for layer in range(n_hidden):
        model.add(keras.layers.Dense(n_neurons, activation="relu"))
    model.add(keras.layers.Dense(2,activation= "relu"))
    optimizer = keras.optimizers.SGD(lr=learning_rate)
    model.compile(loss="mse", optimizer=optimizer)
    return model_____no_output_____keras_reg = keras.wrappers.scikit_learn.KerasRegressor(build_model)_____no_output_____checkpoint_cb = keras.callbacks.ModelCheckpoint("keras_reg_model.h5", save_best_only=True)
early_stopping_cb = keras.callbacks.EarlyStopping(patience=10,
                                                  restore_best_weights=True)

tensorboard_cb = keras.callbacks.TensorBoard(run_logdir)_____no_output_____keras_reg.fit(X, y_reg.values, epochs=100,
              validation_split=0.1,
              callbacks=[checkpoint_cb, tensorboard_cb,early_stopping_cb])Train on 1226 samples, validate on 137 samples
Epoch 1/100
1226/1226 [==============================] - 1s 416us/sample - loss: 212.4466 - val_loss: 121.4823
Epoch 2/100
1226/1226 [==============================] - 0s 135us/sample - loss: 69.9866 - val_loss: 47.5700
Epoch 3/100
1226/1226 [==============================] - 0s 136us/sample - loss: 37.8792 - val_loss: 37.5023
Epoch 4/100
1226/1226 [==============================] - 0s 138us/sample - loss: 30.9267 - val_loss: 32.3445
Epoch 5/100
1226/1226 [==============================] - 0s 133us/sample - loss: 29.0041 - val_loss: 30.3506
Epoch 6/100
1226/1226 [==============================] - 0s 141us/sample - loss: 27.8467 - val_loss: 29.9539
Epoch 7/100
1226/1226 [==============================] - 0s 137us/sample - loss: 26.7904 - val_loss: 28.6768
Epoch 8/100
1226/1226 [==============================] - 0s 125us/sample - loss: 26.1804 - val_loss: 31.1743
Epoch 9/100
1226/1226 [==============================] - 0s 126us/sample - loss: 25.8192 - val_loss: 33.3617
Epoch 10/100
1226/1226 [==============================] - 0s 144us/sample - loss: 25.6005 - val_loss: 28.1323
Epoch 11/100
1226/1226 [==============================] - 0s 127us/sample - loss: 25.1491 - val_loss: 30.1797
Epoch 12/100
1226/1226 [==============================] - 0s 129us/sample - loss: 24.4903 - val_loss: 30.5976
Epoch 13/100
1226/1226 [==============================] - 0s 134us/sample - loss: 24.3926 - val_loss: 30.1345
Epoch 14/100
1226/1226 [==============================] - 0s 128us/sample - loss: 24.0758 - val_loss: 31.2170
Epoch 15/100
1226/1226 [==============================] - 0s 140us/sample - loss: 24.2203 - val_loss: 27.6698
Epoch 16/100
1226/1226 [==============================] - 0s 122us/sample - loss: 23.6177 - val_loss: 29.0357
Epoch 17/100
1226/1226 [==============================] - 0s 126us/sample - loss: 23.7313 - val_loss: 29.1133
Epoch 18/100
1226/1226 [==============================] - 0s 139us/sample - loss: 23.2217 - val_loss: 29.6727
Epoch 19/100
1226/1226 [==============================] - 0s 130us/sample - loss: 23.6692 - val_loss: 29.6529
Epoch 20/100
1226/1226 [==============================] - 0s 125us/sample - loss: 23.0627 - val_loss: 29.7535
Epoch 21/100
1226/1226 [==============================] - 0s 121us/sample - loss: 23.2644 - val_loss: 28.3172
Epoch 22/100
1226/1226 [==============================] - 0s 126us/sample - loss: 22.9078 - val_loss: 29.0068
Epoch 23/100
1226/1226 [==============================] - 0s 122us/sample - loss: 22.7624 - val_loss: 31.0002
Epoch 24/100
1226/1226 [==============================] - 0s 127us/sample - loss: 22.4633 - val_loss: 29.6035
Epoch 25/100
1226/1226 [==============================] - 0s 137us/sample - loss: 22.2912 - val_loss: 29.1956
from sklearn.model_selection import cross_val_score
scores = cross_val_score(keras_reg,X,y_reg.values,
                         scoring="neg_mean_squared_error",cv=5)
keras_reg_scores = np.sqrt(-scores).mean()
print('keras_reg validation score(rmse) :', keras_reg_scores)Train on 1090 samples
1090/1090 [==============================] - 0s 363us/sample - loss: 244.1206
Train on 1090 samples
1090/1090 [==============================] - 1s 491us/sample - loss: 244.4997
Train on 1090 samples
1090/1090 [==============================] - 0s 360us/sample - loss: 242.4720
Train on 1091 samples
1091/1091 [==============================] - 0s 357us/sample - loss: 262.9601
Train on 1091 samples
1091/1091 [==============================] - 1s 492us/sample - loss: 240.1638
keras_reg validation score(rmse) : 11.111010090807369
train_prediction_reg = keras_reg.predict(X)
print("mmse max : ",train_prediction_reg[:,0].max())
print("mmse min :", train_prediction_reg[:,0].min())
print("adas13 max : ",train_prediction_reg[:,1].max())
print("adas13 min :", train_prediction_reg[:,1].min())
train_prediction_regmmse max :  32.84368
mmse min : 20.798416
adas13 max :  43.171127
adas13 min : 6.4930105
y_reg.values_____no_output_____np.random.seed(42)
tf.random.set_seed(42)_____no_output_____run_logdir = get_run_logdir()
tensorboard_cb = keras.callbacks.TensorBoard(run_logdir)_____no_output_____from scipy.stats import reciprocal
from sklearn.model_selection import RandomizedSearchCV

param_distribs = {
    "n_hidden": [0, 1, 2, 3],
    "n_neurons": np.arange(1, 100),
    "learning_rate": reciprocal(3e-4, 3e-2),
}

rnd_search_cv = RandomizedSearchCV(keras_reg, param_distribs, n_iter=10, cv=3, verbose=2)
rnd_search_cv.fit(X, y_reg.values, epochs=100,
                  validation_split=0.1,
                  callbacks=[tensorboard_cb,early_stopping_cb])Fitting 3 folds for each of 10 candidates, totalling 30 fits
[CV] learning_rate=0.001683454924600351, n_hidden=0, n_neurons=15 ....
Train on 817 samples, validate on 91 samples
Epoch 1/100
rnd_search_cv.best_params______no_output_____rnd_search_cv.best_score______no_output_____rnd_search_cv.best_estimator______no_output_____model_reg = rnd_search_cv.best_estimator_.model_____no_output_____run_logdir = get_run_logdir()
tensorboard_cb = keras.callbacks.TensorBoard(run_logdir)_____no_output_____model_reg.fit(X, y_reg.values, epochs=100,
                  validation_split=0.1,
                  callbacks=[tensorboard_cb,early_stopping_cb])Train on 1226 samples, validate on 137 samples
Epoch 1/100
1226/1226 [==============================] - 0s 127us/sample - loss: 25.5846 - val_loss: 26.7267
Epoch 2/100
1226/1226 [==============================] - 0s 123us/sample - loss: 25.4943 - val_loss: 26.2151
Epoch 3/100
1226/1226 [==============================] - 0s 126us/sample - loss: 25.5219 - val_loss: 26.4893
Epoch 4/100
1226/1226 [==============================] - 0s 121us/sample - loss: 25.4668 - val_loss: 26.8722
Epoch 5/100
1226/1226 [==============================] - 0s 122us/sample - loss: 25.4384 - val_loss: 26.1440
Epoch 6/100
1226/1226 [==============================] - 0s 133us/sample - loss: 25.4410 - val_loss: 26.2973
Epoch 7/100
1226/1226 [==============================] - 0s 129us/sample - loss: 25.4140 - val_loss: 26.0869
Epoch 8/100
1226/1226 [==============================] - 0s 125us/sample - loss: 25.4106 - val_loss: 26.2014
Epoch 9/100
1226/1226 [==============================] - 0s 116us/sample - loss: 25.4326 - val_loss: 27.2789
Epoch 10/100
1226/1226 [==============================] - 0s 121us/sample - loss: 25.4360 - val_loss: 26.3105
Epoch 11/100
1226/1226 [==============================] - 0s 126us/sample - loss: 25.4001 - val_loss: 26.5626
Epoch 12/100
1226/1226 [==============================] - 0s 126us/sample - loss: 25.3305 - val_loss: 27.4531
Epoch 13/100
1226/1226 [==============================] - 0s 127us/sample - loss: 25.3860 - val_loss: 26.0843
Epoch 14/100
1226/1226 [==============================] - 0s 122us/sample - loss: 25.3536 - val_loss: 26.6786
Epoch 15/100
1226/1226 [==============================] - 0s 123us/sample - loss: 25.3845 - val_loss: 26.1349
Epoch 16/100
1226/1226 [==============================] - 0s 121us/sample - loss: 25.3569 - val_loss: 26.5031
Epoch 17/100
1226/1226 [==============================] - 0s 118us/sample - loss: 25.3961 - val_loss: 26.3028
Epoch 18/100
1226/1226 [==============================] - 0s 120us/sample - loss: 25.3462 - val_loss: 26.4777
Epoch 19/100
1226/1226 [==============================] - 0s 119us/sample - loss: 25.3526 - val_loss: 26.4297
Epoch 20/100
1226/1226 [==============================] - 0s 123us/sample - loss: 25.3215 - val_loss: 26.2741
Epoch 21/100
1226/1226 [==============================] - 0s 117us/sample - loss: 25.3314 - val_loss: 26.1868
Epoch 22/100
1226/1226 [==============================] - 0s 135us/sample - loss: 25.3408 - val_loss: 26.2577
Epoch 23/100
1226/1226 [==============================] - 0s 117us/sample - loss: 25.3232 - val_loss: 26.0968
model_reg.summary()Model: "sequential_36"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
dense_78 (Dense)             (None, 2)                 144
=================================================================
Total params: 144
Trainable params: 144
Non-trainable params: 0
_________________________________________________________________
model_reg.model_____no_output_____train_prediction_reg = model_reg.predict(X)
print("mmse max : ",train_prediction_reg[:,0].max())
print("mmse min :", train_prediction_reg[:,0].min())
print("adas13 max : ",train_prediction_reg[:,1].max())
print("adas13 min :", train_prediction_reg[:,1].min())
train_prediction_reg_____no_output_____model_reg = rnd_search_cv.best_estimator______no_output_____rnd_search_cv_____no_output_____from sklearn.model_selection import cross_val_score
scores = cross_val_score(model_reg.model,X,y_reg.values,
                         scoring="neg_mean_squared_error",cv=5)
model_reg_scores = np.sqrt(-scores).mean()
print('keras_reg validation score(rmse) :', model_reg_scores)_____no_output_____

# test set(제출용)
---_____no_output_____

# Load test dataset
csv_file_test = '/content/gdrive/My Drive/BNCS401_Midterm_Project/Test_data_ageupdated.csv'  # Set your path
test_data = pd.read_csv(csv_file_test)
test_data_____no_output_____test_data = test_data.drop("RID",axis=1)_____no_output_____incomplete_rows = test_data[test_data.isnull().any(axis=1)]#.head()
incomplete_rows_____no_output_____def object_to_float(o): #o = train_data["~~"] 형태
  for i in range(343):
    if o[i] == ' ':
      o[i] = np.nan #nan으로 채움_____no_output_____object_to_float(test_data["ST64TA_UCSFFSX_11_02_15_UCSFFSX51_08_01_16"])_____no_output_____object_to_float(test_data["ST123TA_UCSFFSX_11_02_15_UCSFFSX51_08_01_16"])_____no_output_____X_ = data_pipeline.fit_transform(test_data)_____no_output_____X_.shape_____no_output_____reg_prediction = model_reg.predict(X_)
reg_prediction_____no_output_____prediction.to_csv('/content/gdrive/My Drive/Colab Notebooks/prediction_NN.csv',encoding='utf-8')_____no_output_____


# Notebook from ATrain951/01.python_intro-milaan
Path: Interpreter_Vs_Compiler.ipynb

All the IPython Notebooks in **Python Introduction** lecture series by Dr. Milaan Parmar are available @ **[GitHub](https://github.com/milaan9/01_Python_Introduction)**
_____no_output_____# Interpreter Vs Compiler : Differences Between Interpreter and Compiler

In this class, you will learn the differences between interpreters and compilers.

We generally write a computer program using a high-level language. A high-level language is one that is understandable by us, humans. This is called **source code**.

However, a computer does not understand high-level language. It only understands the program written in **0**'s and **1**'s in binary, called the **machine code**.

To convert source code into machine code, we use either a **compiler** or an **interpreter**._____no_output_____## What is Interpreter?

An interpreter is a computer program, which coverts each high-level program statement into the machine code. This includes source code, pre-compiled code, and scripts. Both compiler and interpreters do the same job which is converting higher level programming language to machine code. However, a compiler will convert the code into machine code (create an exe) before program run. Interpreters convert code into machine code when the program is run.

## What is Compiler?

A compiler is a computer program that transforms code written in a high-level programming language into the machine code. It is a program which translates the human-readable code to a language a computer processor understands (binary 1 and 0 bits). The computer processes the machine code to perform the corresponding tasks.

A compiler should comply with the syntax rule of that programming language in which it is written. However, the compiler is only a program and cannot fix errors found in that program. So, if you make a mistake, you need to make changes in the syntax of your program. Otherwise, it will not compile.

## Programming Steps Comparision:

| # | Interpreter | Compiler |
|:----:|:----|:-------|
|**1.**|Create the Program.|Create the program.|
|**2.**|No linking of files or machine code generation.|Compile will parse or analyses all of the language statements for its correctness. If incorrect, throws an error.|
|**3.**|Source statements executed line by line DURING Execution.|If no error, the compiler will convert source code to machine code.|
|**4.**|-|It links different code files into a runnable program(know as exe).|
|**5.**|-|Run the Program|_____no_output_____Both compilers and interpreters are used to convert a program written in a high-level language into machine code understood by computers. However, there are differences between how an interpreter and a compiler works.

_____no_output_____## Interpreter Vs Compiler

| Parameters | Interpreter | Compiler |
|:-----|:----|:-------|
|**Advantage**|Interpreters are easier to use, especially for beginners.|The program code is already translated into machine code. Thus, it code execution time is less.|
|**Disadvantage**|Interpreted programs can run on computers that have the corresponding interpreter.|You can't change the program without going back to the source code.|
|**Machine code**|Not saving machine code at all.|Store machine language as machine code on the disk.|
|**Running time**|Interpreters usually take less amount of time to analyze the source code. However, the overall execution time is comparatively slower than compilers.|Compilers usually take a large amount of time to analyze the source code. However, the overall execution time is comparatively faster than interpreters.|
|**Model**|It is based on interpretation method.|It is based on language translation linking-loading model.|
|**Program generation**|Do not generate output program. So they evaluate the source program at every time during execution.|Generates output program (in the form of .exe) which can be run independently from the original program.|
|**Execution**|It does not convert source code into machine code instead it scans it line by line.|It converts the source code into machine code.|
|**Memory requirement**|No Object Code is generated, hence are memory efficient.|Generates Object Code which further requires linking, hence requires more memory.|
|**Best suited for**|For web environments, where load times are important. Due to all the exhaustive analysis is done, compiles take relatively larger time to compile even small code that may not be run multiple times. In such cases, interpreters are better.|Bounded to the specific target machine and cannot be ported. C and C++ are a most popular a programming language which uses compilation model.|
|**Code Optimization**|Interpreters see code line by line, and thus optimizations are not as robust as compilers.|The compiler sees the entire code upfront. Hence, they perform lots of optimizations that make code run faster.|
|**Dynamic Typing**|Interpreted languages support Dynamic Typing|Difficult to implement as compilers cannot predict what happens at turn time.|
|**Usage**|It is best suited for the program and development environment.|It is best suited for the production environment.|
|**Error execution**|The interpreter reads a single statement and shows the error if any. You must correct the error to interpret next line.|Compiler displays all errors and warning at the compilation time. Therefore, you can't run the program without fixing errors.|
|**Input**|It takes a single line of code.|It takes an entire program.|
|**Output**|Interpreter never generate any intermediate machine code.|Compliers generates intermediate machine code.|
|**Errors**|Considering it scans code one line at a time, errors are shown line by line.|As it scans the code in one go, the errors (if any) are shown at the end together.|
|**Programming languages**| JavaScript, MATLAB, Python, Ruby, PHP, Pearl use interpreters. | C, C++, C#, Java use compilers.|_____no_output_____### HIGH-LEVEL LANGUAGES
High-level languages, like C, C++, JAVA, etc., are very near to English. It makes programming process easy. However, it must be translated into machine language before execution. This translation process is either conducted by either a compiler or an interpreter. Also known as source code.

### MACHINE CODE
Machine languages are very close to the hardware. Every computer has its machine language. A machine language programs are made up of series of binary pattern. (eg. 110110) It represents the simple operations which should be performed by the computer. Machine language programs are executable so that they can be run directly.

### OBJECT CODE
On compilation of source code, the machine code generated for different processors like Intel, AMD, an ARM is different. To make code portable, the source code is first converted to Object Code. It is an intermediary code (similar to machine code) that no processor will understand. At run time, the object code is converted to the machine code of the underlying platform.

### Java is both Compiled and Interpreted.
To exploit relative advantages of compilers are interpreters some programming language like Java are both compiled and interpreted. The Java code itself is compiled into Object Code. At run time, the JVM interprets the Object code into machine code of the target computer._____no_output_____## KEY DIFFERENCE

* Compiler transforms code written in a high-level programming language into the object code, at once, before program runs, whereas an Interpreter coverts each high-level program statement, one by one, into the object code, during program run.
* Compiled code runs faster while interpreted code runs slower.
* Compiler displays all errors after compilation, on the other hand, the Interpreter displays errors of each line one by one.
* Compiler is based on translation linking-loading model, whereas Interpreter is based on interpretation method.
* Compiler takes an entire program whereas the Interpreter takes a single line of code._____no_output_____## KEY SIMILARITIES

* Both Compiler and Interpreter translate source code for machine understanding so that your device can follow your command.
* The Compiler and the Interpreter use memory to store data.
* Interpreter and Compiler are the high-level languages.
* Java language uses both the Compiler and the Interpreter._____no_output_____## Authors ✍

I'm Dr. Milaan Parmar and I have written this tutorial. If you think you can add/correct/edit and enhance this tutorial you are most welcome🙏

See **[github's contributors page](https://github.com/milaan9/01_Python_Introduction/graphs/contributors)** for details.

If you have trouble with this tutorial please tell me about it by **[Create an issue on GitHub](https://github.com/milaan9/01_Python_Introduction/issues/new)**. and I'll make this tutorial better. This is probably the best choice if you had trouble following the tutorial, and something in it should be explained better. You will be asked to create a GitHub account if you don't already have one.

If you like this tutorial, please **[give it a ⭐ star](https://github.com/milaan9/01_Python_Introduction/blob/main/README.md#how-can-i-thank-you-for-writing-and-sharing-this-tutorial)**._____no_output_____## Licence 📜

You may use this tutorial freely at your own risk. See **[LICENSE](https://github.com/milaan9/01_Python_Introduction/blob/main/LICENSE)**._____no_output_____


# Notebook from psheehan/CIERA-HS-Program
Path: BonusProblems/Module1/BonusChallenge2.ipynb

# Calculating the Mass of The Sun

One of the most important relationships in astronomy is Kepler's laws of planetary motion, because they allow us to use orbits to calculate the masses of astronomical objects. Of particular use is Kepler's third law, which relates the period, separation, and masses of two objects:

\begin{equation}
P^2 = \frac{4 \pi^2}{G (m_1 + m_2)} a^3
\end{equation}

Here $G$ is the gravitational constant, $m_1$ and $m_2$ are the masses of the two bodies orbiting eachother, $P$ is the period of their orbit, and $a$ is the distance between them. In the case of our Solar System, because the mass of the Sun is significantly greater than the mass of any of the planets (i.e. $m_1 + m_2 \approx m_1$), this can be reduced to:

\begin{equation}
P^2 = \frac{4 \pi^2}{G \, M} a^3
\end{equation}

Because we know the periods and distances for all of the planets, we can use those values to calculate the mass of the body that they are orbiting, i.e the Sun._____no_output_____First, pick your favorite planet. Look up their orbital period and the distance from the Sun (i.e. their "semi-major axis"). You will also need to look up the value of the gravitational constant, $G$. Make sure to check your units! Define those as variables here:_____no_output_____Now, using the above equation, calculate the mass of the Sun:_____no_output_____Use what you know about string formatting to print out the mass you calculated with nice formatting, including units:_____no_output_____Just to make sure we did that correctly, pick your second favorite planet, and repeat. Do you get the same answer?_____no_output_____Kepler's laws are universal, so they don't just work for the Sun, but can be used to measure the mass of anything, if there's something orbiting it. Let's try measuring the mass of Jupiter. Look up the separation and period of Jupiter's moon, Callisto, and use Kepler's third law to calculate the mass of Jupiter in the same way:_____no_output_____Finally, astronomers think there is a *supermassive black hole* at the center of our Milky Way galaxy. Why is that? Well, we've been able to watch stars orbit something at the center of our Galaxy:

<img src="1280px-Galactic_centre_orbits.svg.png">

How massive is the object that they are orbiting? Look-up the period and distance (i.e. semi-major axis) of the star "S2", and calculate the mass of that object (*Hint1: Wikipedia knows about this star. Hint 2: The units '"', or arcseconds, that are listed there for the semi-major axis can be converted into a more useful number by multiplying by the distance to the source. What is a "pc"?*):_____no_output_____How does the mass you measured compare with the mass of our Sun:_____no_output_____Often in astronomy we will list measured masses in terms of the mass of the Sun, because units of grams or kilograms are not easy to interpret when the numbers are so large. Use string formatting to display the mass of the object at the center of our Solar System in terms of the number of "Solar masses" that it is, with the proper units:_____no_output_____