# Notebook from krentzd/ZeroCostDL4Mic
Path: Colab_notebooks/Beta notebooks/DenoiSeg_2D_ZeroCostDL4Mic.ipynb

# **Image denoising and segmentation using DenoiSeg 2D**

---

 DenoiSeg 2D is deep-learning method that can be used to jointly denoise and segment 2D microscopy images. By running this notebook, you can train your and use you own network.

 The benefits of using DenoiSeg (compared to other Deep Learning-based segmentation methods) are more prononced when only a few annotated images are available. However, the denoising part requires many images to perform well. All the noisy images don't need to be labeled to train DenoiSeg.
---

*Disclaimer*:

This notebook is part of the Zero-Cost Deep-Learning to Enhance Microscopy project (https://github.com/HenriquesLab/DeepLearning_Collab/wiki). Jointly developed by the Jacquemet (link to https://cellmig.org/) and Henriques (https://henriqueslab.github.io/) laboratories.

This notebook is largely based on the paper: **DenoiSeg: Joint Denoising and Segmentation**
Tim-Oliver Buchholz, Mangal Prakash, Alexander Krull, Florian Jug
https://arxiv.org/abs/2005.02987

And source code found in: https://github.com/juglab/DenoiSeg/wiki

**Please also cite this original paper when using or developing this notebook.**
_____no_output_____# **How to use this notebook?**

---

Video describing how to use our notebooks are available on youtube:
  - [**Video 1**](https://www.youtube.com/watch?v=GzD2gamVNHI&feature=youtu.be): Full run through of the workflow to obtain the notebooks and the provided test datasets as well as a common use of the notebook
  - [**Video 2**](https://www.youtube.com/watch?v=PUuQfP5SsqM&feature=youtu.be): Detailed description of the different sections of the notebook

---
###**Structure of a notebook**

The notebook contains two types of cell:

**Text cells** provide information and can be modified by douple-clicking the cell. You are currently reading the text cell. You can create a new text by clicking `+ Text`.

**Code cells** contain code and the code can be modfied by selecting the cell. To execute the cell, move your cursor on the `[ ]`-mark on the left side of the cell (play button appears). Click to execute the cell. After execution is done the animation of play button stops. You can create a new coding cell by clicking `+ Code`.

---
###**Table of contents, Code snippets** and **Files**

On the top left side of the notebook you find three tabs which contain from top to bottom:

*Table of contents* = contains structure of the notebook. Click the content to move quickly between sections.

*Code snippets* = contain examples how to code certain tasks. You can ignore this when using this notebook.

*Files* = contain all available files. After mounting your google drive (see section 1.) you will find your files and folders here.

**Remember that all uploaded files are purged after changing the runtime.** All files saved in Google Drive will remain. You do not need to use the Mount Drive-button; your Google Drive is connected in section 1.2.

**Note:** The "sample data" in "Files" contains default files. Do not upload anything in here!

---
###**Making changes to the notebook**

**You can make a copy** of the notebook and save it to your Google Drive. To do this click file -> save a copy in drive.

To **edit a cell**, double click on the text. This will show you either the source code (in code cells) or the source text (in text cells).
You can use the `#`-mark in code cells to comment out parts of the code. This allows you to keep the original code piece in the cell as a comment._____no_output_____# **0. Before getting started**
---

Before you run the notebook, please ensure that you are logged into your Google account and have the training and/or data to process in your Google Drive.

**it needs to have access to a paired training dataset made of images and their corresponding masks**. Information on how to generate a training dataset is available in our Wiki page: https://github.com/HenriquesLab/ZeroCostDL4Mic/wiki

**Importantly, the benefits of using DenoiSeg are more pronounced when only limited numbers of segmentation annotations are available for training. However, DenoiSeg also expects that lots of noisy raw images are available to train the denoising part. It is therefore not required for all the noisy images to be annotated to train DenoiSeg**.

**We strongly recommend that you generate extra paired images. These images can be used to assess the quality of your trained model**. The quality control assessment can be done directly in this notebook.

The data structure is important. It is necessary that all the input data are in the same folder and that all the output data is in a separate folder. The provided training dataset is already split.

Additionally, the corresponding Training_source and Training_target files need to have **the same name**.

Please note that you currently can **only use .tif files!**

You can also provide a folder that contains the data that you wish to analyse with the trained network once all training has been performed. This can include Test dataset for which you have the equivalent output and can compare to what the network provides.

Here's a common data structure that can work:
*   Experiment A
    - **Training dataset**
      - Noisy Images (Training_source)
        - img_1.tif, img_2.tif, img_3.tif, img_4.tif, ...
      - Masks (Training_target)
        - img_1.tif, img_2.tif
    - **Quality control dataset (optional, not required for training)**
     - Noisy Images
        - img_1.tif, img_2.tif
     - High SNR Images
        - img_1.tif, img_2.tif
      - Masks
        - img_1.tif, img_2.tif
    - **Data to be predicted**
    - **Results**

---
**Important note**

- If you wish to **Train a network from scratch** using your own dataset (and we encourage everyone to do that), you will need to run **sections 1 - 4**, then use **section 5** to assess the quality of your model and **section 6** to run predictions using the model that you trained.

- If you wish to **Evaluate your model** using a model previously generated and saved on your Google Drive, you will only need to run **sections 1 and 2** to set up the notebook, then use **section 5** to assess the quality of your model.

- If you only wish to **run predictions** using a model previously generated and saved on your Google Drive, you will only need to run **sections 1 and 2** to set up the notebook, then use **section 6** to run the predictions on the desired model.
---
_____no_output_____# **1. Initialise the Colab session**

---

_____no_output_____## **1.1. Check for GPU access**
---

By default, the session should be using Python 3 and GPU acceleration, but it is possible to ensure that these are set properly by doing the following:

Go to **Runtime -> Change the Runtime type**

**Runtime type: Python 3** *(Python 3 is programming language in which this program is written)*

**Accelator: GPU** *(Graphics processing unit)*
_____no_output_____

#@markdown ##Run this cell to check if you have GPU access
%tensorflow_version 1.x

import tensorflow as tf
if tf.test.gpu_device_name()=='':
  print('You do not have GPU access.')
  print('Did you change your runtime ?')
  print('If the runtime setting is correct then Google did not allocate a GPU for your session')
  print('Expect slow performance. To access GPU try reconnecting later')

else:
  print('You have GPU access')
  !nvidia-smi_____no_output_____

## **1.2. Mount your Google Drive**
---
 To use this notebook on the data present in your Google Drive, you need to mount your Google Drive to this notebook.

 Play the cell below to mount your Google Drive and follow the link. In the new browser window, select your drive and select 'Allow', copy the code, paste into the cell and press enter. This will give Colab access to the data on the drive.

 Once this is done, your data are available in the **Files** tab on the top left of notebook._____no_output_____

#@markdown ##Play the cell to connect your Google Drive to Colab

#@markdown * Click on the URL.

#@markdown * Sign in your Google Account.

#@markdown * Copy the authorization code.

#@markdown * Enter the authorization code.

#@markdown * Click on "Files" site on the right. Refresh the site. Your Google Drive folder should now be available here as "drive".

# mount user's Google Drive to Google Colab.
from google.colab import drive
drive.mount('/content/gdrive')_____no_output_____

# **2. Install DenoiSeg and Dependencies**
---_____no_output_____

#@markdown ##Install DenoiSeg and dependencies
!pip install q keras==2.2.5

# Here we enable Tensorflow 1.
%tensorflow_version 1.x
import tensorflow
print(tensorflow.__version__)

print("Tensorflow enabled.")

# Here we install Noise2Void and other required packages
!pip install denoiseg
!pip install wget
!pip install memory_profiler
%load_ext memory_profiler

print("Noise2Void installed.")

# Here we install all libraries and other depencies to run the notebook.

# ------- Variable specific to Denoiseg -------

import warnings
warnings.filterwarnings('ignore')

import numpy as np
from matplotlib import pyplot as plt
from scipy import ndimage

from denoiseg.models import DenoiSeg, DenoiSegConfig
from denoiseg.utils.misc_utils import combine_train_test_data, shuffle_train_data, augment_data
from denoiseg.utils.seg_utils import *
from denoiseg.utils.compute_precision_threshold import measure_precision, compute_labels

from csbdeep.utils import plot_history
from tifffile import imread, imsave
from glob import glob

import urllib
import os
import zipfile

# ------- Common variable to all ZeroCostDL4Mic notebooks -------
import numpy as np
from matplotlib import pyplot as plt
import urllib
import os, random
import shutil
import zipfile
from tifffile import imread, imsave
import time
import sys
import wget
from pathlib import Path
import pandas as pd
import csv
from glob import glob
from scipy import signal
from scipy import ndimage
from skimage import io
from sklearn.linear_model import LinearRegression
from skimage.util import img_as_uint
import matplotlib as mpl
from skimage.metrics import structural_similarity
from skimage.metrics import peak_signal_noise_ratio as psnr
from astropy.visualization import simple_norm
from skimage import img_as_float32

# Colors for the warning messages
class bcolors:
  WARNING = '\033[31m'
W  = '\033[0m'  # white (normal)
R  = '\033[31m' # red

#Disable some of the tensorflow warnings
import warnings
warnings.filterwarnings("ignore")

print("Libraries installed")
_____no_output_____

# **3. Select your parameters and paths**
---_____no_output_____## **3.1. Setting main training parameters**
---
 _____no_output_____ **Paths for training, predictions and results**

**`Training_source:`** These is the path to your folders containing the Training_source (noisy images). To find the path of the folder containing your datasets, go to your Files on the left of the notebook, navigate to the folder containing your files and copy the path by right-clicking on the folder, **Copy path** and pasting it into the right box below.

**`model_name`:** Use only my_model -style, not my-model (Use "_" not "-"). Do not use spaces in the name. Do not re-use the name of an existing model (saved in the same folder), otherwise it will be overwritten.

**`model_path`**: Enter the path where your model will be saved once trained (for instance your result folder).

**Training Parameters**

**`number_of_epochs`:** Input how many epochs (rounds) the network will be trained. Preliminary results can already be observed after a few (10-30) epochs, but a full training should run for 100-200 epochs. Evaluate the performance after training (see 4.3.). **Default value: 30**

**Advanced Parameters - experienced users only**

**`Priority`:** Choose how much relative the importance to assign to the denoising
and segmentation tasks by choosing an appropriate value (between 0 and 1; with 0 being only segmentation and 1 being only denoising. **Default value: 0.5**

**`number_of_steps`:** Define the number of training steps by epoch. By default this parameter is calculated so that each image / patch is seen at least once per epoch. **Default value: depends on number of patches, min 100; max 400**

**`batch_size:`** This parameter defines the number of patches seen in each training step.  Noise2Void requires a large batch size for stable training. Reduce this parameter if your GPU runs out of memory. **Default value: 128**

**`initial_learning_rate`:** Input the initial value to be used as learning rate. **Default value: 0.0004**
_____no_output_____

# create DataGenerator-object.

#@markdown ###Path to training image(s):
Training_source = "" #@param {type:"string"}
Training_target = "" #@param {type:"string"}

#@markdown ###Path to validation image(s):
#Validation_source = "/content/gdrive/My Drive/Work/manuscript/Ongoing Projects/Zero-Cost Deep-Learning to Enhance Microscopy/test folder/Training datasets/DenoiSeg/Test - Noisy" #@param {type:"string"}
#Validation_target = "/content/gdrive/My Drive/Work/manuscript/Ongoing Projects/Zero-Cost Deep-Learning to Enhance Microscopy/test folder/Training datasets/DenoiSeg/Test - Masks" #@param {type:"string"}

#@markdown ### Model name and path:
model_name = "" #@param {type:"string"}
model_path = "" #@param {type:"string"}

#@markdown ###Training Parameters
#@markdown Number of epochs:
number_of_epochs =  10#@param {type:"number"}

#@markdown ###Advanced Parameters
Use_Default_Advanced_Parameters = True#@param {type:"boolean"}

#@markdown ###If not, please input:
Priority = 0.5#@param {type:"number"}
number_of_steps = 100#@param {type:"number"}
batch_size =  128#@param {type:"number"}
percentage_validation =  10#@param {type:"number"}
initial_learning_rate = 0.0004 #@param {type:"number"}

if (Use_Default_Advanced_Parameters):
  print("Default advanced parameters enabled")
  # number_of_steps is defined in the following cell in this case
  Priority = 0.5
  batch_size = 128
  percentage_validation = 10
  initial_learning_rate = 0.0004

#here we check that no model with the same name already exist, if so delete
if os.path.exists(model_path+'/'+model_name):
  print(R + "!! WARNING: Folder already exists and has been removed !!" + W)
  shutil.rmtree(model_path+'/'+model_name)

# This will open a randomly chosen dataset input image
random_choice = random.choice(os.listdir(Training_target))
x = imread(Training_source+"/"+random_choice)

# Here we disable pre-trained model by default (in case the next cell is not ran)
Use_pretrained_model = False

# Here we disable data augmentation by default (in case the cell is not ran)
Use_Data_augmentation = True

# Here we count the number of files in the training target folder
Mask_Filelist = os.listdir(Training_target)
Mask_number_files = len(Mask_Filelist)

# Here we count the number of file to use for validation
Mask_for_validation = int((Mask_number_files)/percentage_validation)

if Mask_for_validation == 0:
  Mask_for_validation = 2
if Mask_for_validation == 1:
  Mask_for_validation = 2

# Here we count the number of files in the training target folder
Noisy_Filelist = os.listdir(Training_source)
Noisy_number_files = len(Noisy_Filelist)

# Here we count the number of file to use for validation
Noisy_for_validation = int((Noisy_number_files)/percentage_validation)

if Noisy_for_validation == 0:
  Noisy_for_validation = 1

#Here we find the noisy images that do not have masks
noisy_image_no_mask_list = list(set(Noisy_Filelist) - set(Mask_Filelist))

#Here we split the training dataset between training and validation
# Everything is copied in the /Content Folder
Training_source_temp = "/content/training_source"

if os.path.exists(Training_source_temp):
  shutil.rmtree(Training_source_temp)
os.makedirs(Training_source_temp)

Training_target_temp = "/content/training_target"
if os.path.exists(Training_target_temp):
  shutil.rmtree(Training_target_temp)
os.makedirs(Training_target_temp)

Validation_source_temp = "/content/validation_source"

if os.path.exists(Validation_source_temp):
  shutil.rmtree(Validation_source_temp)
os.makedirs(Validation_source_temp)

Validation_target_temp = "/content/validation_target"
if os.path.exists(Validation_target_temp):
  shutil.rmtree(Validation_target_temp)
os.makedirs(Validation_target_temp)

list_source = os.listdir(os.path.join(Training_source))
list_target = os.listdir(os.path.join(Training_target))

#Move files into the temporary source and target directories:

for f in os.listdir(os.path.join(Training_source)):
  shutil.copy(Training_source+"/"+f, Training_source_temp+"/"+f)

for p in os.listdir(os.path.join(Training_target)):
  shutil.copy(Training_target+"/"+p, Training_target_temp+"/"+p)

#Here we move images to be used for validation
for i in range(Mask_for_validation):
  shutil.move(Training_source_temp+"/"+list_target[i], Validation_source_temp+"/"+list_target[i])
  shutil.move(Training_target_temp+"/"+list_target[i], Validation_target_temp+"/"+list_target[i])

#Here we move a few more noisy images for validation
if noisy_image_no_mask_list:
  for y in range(Noisy_for_validation):
    shutil.move(Training_source_temp+"/"+noisy_image_no_mask_list[y], Validation_source_temp+"/"+noisy_image_no_mask_list[y])

print("Parameters initiated.")

y = imread(Training_target+"/"+random_choice)

#Here we display one image
norm = simple_norm(x, percent = 99)

f=plt.figure(figsize=(16,8))
plt.subplot(1,2,1)
plt.imshow(x, interpolation='nearest', norm=norm, cmap='magma')
plt.title('Training source')
plt.axis('off');

plt.subplot(1,2,2)
plt.imshow(y, interpolation='nearest', vmin=0, vmax=1, cmap='viridis')
plt.title('Training target')
plt.axis('off');
_____no_output_____

## **3.2. Data augmentation**
---
_____no_output_____Data augmentation can improve training progress by amplifying differences in the dataset. This can be useful if the available dataset is small since, in this case, it is possible that a network could quickly learn every example in the dataset (overfitting), without augmentation. Augmentation is not necessary for training and if your training dataset is large you should disable it.

Data augmentation is performed here by rotating the patches in XY-Plane and flip them along X-Axis (multiply the dataset by 8).

 **By default data augmentation is enabled. Disable this option is you run out of RAM during the training**.

 _____no_output_____

#Data augmentation

Use_Data_augmentation = True #@param {type:"boolean"}

if Use_Data_augmentation:
  print("Data augmentation enabled")

if not Use_Data_augmentation:
  print("Data augmentation disabled")
_____no_output_____

## **3.3. Using weights from a pre-trained model as initial weights**
---
  Here, you can set the the path to a pre-trained model from which the weights can be extracted and used as a starting point for this training session. **This pre-trained model needs to be a DenoiSeg model**.

 This option allows you to perform training over multiple Colab runtimes or to do transfer learning using models trained outside of ZeroCostDL4Mic. **You do not need to run this section if you want to train a network from scratch**.

 In order to continue training from the point where the pre-trained model left off, it is adviseable to also **load the learning rate** that was used when the training ended. This is automatically saved for models trained with ZeroCostDL4Mic and will be loaded here. If no learning rate can be found in the model folder provided, the default learning rate will be used. _____no_output_____

# @markdown ##Loading weights from a pre-trained network

Use_pretrained_model = False #@param {type:"boolean"}

pretrained_model_choice = "Model_from_file" #@param ["Model_from_file"]

Weights_choice = "last" #@param ["last", "best"]

#@markdown ###If you chose "Model_from_file", please provide the path to the model folder:
pretrained_model_path = "" #@param {type:"string"}

# --------------------- Check if we load a previously trained model ------------------------
if Use_pretrained_model:

# --------------------- Load the model from the choosen path ------------------------
  if pretrained_model_choice == "Model_from_file":
    h5_file_path = os.path.join(pretrained_model_path, "weights_"+Weights_choice+".h5")

# --------------------- Download the a model provided in the XXX ------------------------

  if pretrained_model_choice == "Model_name":
    pretrained_model_name = "Model_name"
    pretrained_model_path = "/content/"+pretrained_model_name
    print("Downloading the 2D_Demo_Model_from_Stardist_2D_paper")
    if os.path.exists(pretrained_model_path):
      shutil.rmtree(pretrained_model_path)
    os.makedirs(pretrained_model_path)
    wget.download("", pretrained_model_path)
    wget.download("", pretrained_model_path)
    wget.download("", pretrained_model_path)
    wget.download("", pretrained_model_path)
    h5_file_path = os.path.join(pretrained_model_path, "weights_"+Weights_choice+".h5")

# --------------------- Add additional pre-trained models here ------------------------

# --------------------- Check the model exist ------------------------
# If the model path chosen does not contain a pretrain model then use_pretrained_model is disabled,
  if not os.path.exists(h5_file_path):
    print(bcolors.WARNING+'WARNING: weights_last.h5 pretrained model does not exist')
    Use_pretrained_model = False

# If the model path contains a pretrain model, we load the training rate,
  if os.path.exists(h5_file_path):
#Here we check if the learning rate can be loaded from the quality control folder
    if os.path.exists(os.path.join(pretrained_model_path, 'Quality Control', 'training_evaluation.csv')):

      with open(os.path.join(pretrained_model_path, 'Quality Control', 'training_evaluation.csv'),'r') as csvfile:
        csvRead = pd.read_csv(csvfile, sep=',')
        #print(csvRead)

        if "learning rate" in csvRead.columns: #Here we check that the learning rate column exist (compatibility with model trained un ZeroCostDL4Mic bellow 1.4)
          print("pretrained network learning rate found")
          #find the last learning rate
          lastLearningRate = csvRead["learning rate"].iloc[-1]
          #Find the learning rate corresponding to the lowest validation loss
          min_val_loss = csvRead[csvRead['val_loss'] == min(csvRead['val_loss'])]
          #print(min_val_loss)
          bestLearningRate = min_val_loss['learning rate'].iloc[-1]

          if Weights_choice == "last":
            print('Last learning rate: '+str(lastLearningRate))

          if Weights_choice == "best":
            print('Learning rate of best validation loss: '+str(bestLearningRate))

        if not "learning rate" in csvRead.columns: #if the column does not exist, then initial learning rate is used instead
          bestLearningRate = initial_learning_rate
          lastLearningRate = initial_learning_rate
          print(bcolors.WARNING+'WARNING: The learning rate cannot be identified from the pretrained network. Default learning rate of '+str(bestLearningRate)+' will be used instead' + W)

#Compatibility with models trained outside ZeroCostDL4Mic but default learning rate will be used
    if not os.path.exists(os.path.join(pretrained_model_path, 'Quality Control', 'training_evaluation.csv')):
      print(bcolors.WARNING+'WARNING: The learning rate cannot be identified from the pretrained network. Default learning rate of '+str(initial_learning_rate)+' will be used instead'+ W)
      bestLearningRate = initial_learning_rate
      lastLearningRate = initial_learning_rate

# Display info about the pretrained model to be loaded (or not)
if Use_pretrained_model:
  print('Weights found in:')
  print(h5_file_path)
  print('will be loaded prior to training.')

else:
  print(bcolors.WARNING+'No pretrained nerwork will be used.')

_____no_output_____

# **4. Train the network**
---_____no_output_____## **4.1. Prepare the training data and model for training**
---
Here, we use the information from 3. to build the model and convert the training data into a suitable format for training._____no_output_____

#@markdown ##Create the model and dataset objects

# --------------------- Here we load the augmented data or the raw data ------------------------

print("In progress...")

Training_source_dir = Training_source_temp
Training_target_dir = Training_target_temp
# --------------------- ------------------------------------------------

training_images_tiff=Training_source_dir+"/*.tif"
mask_images_tiff=Training_target_dir+"/*.tif"

validation_images_tiff=Validation_source_temp+"/*.tif"
validation_mask_tiff=Validation_target_temp+"/*.tif"

train_images = imread(sorted(glob(training_images_tiff)))
val_images = imread(sorted(glob(validation_images_tiff)))

available_train_masks = imread(sorted(glob(mask_images_tiff)))
available_val_masks = imread(sorted(glob(validation_mask_tiff)))

#This allows the users to not have all their training images segmented
blank_images_train = np.zeros((train_images.shape[0]-available_train_masks.shape[0], available_train_masks.shape[1], available_train_masks.shape[2]))
blank_images_val = np.zeros((val_images.shape[0]-available_val_masks.shape[0], available_val_masks.shape[1], available_val_masks.shape[2]))
blank_images_train = blank_images_train.astype("uint16")
blank_images_val = blank_images_val.astype("uint16")

train_masks = np.concatenate((available_train_masks,blank_images_train), axis = 0)
val_masks = np.concatenate((available_val_masks,blank_images_val), axis = 0)

if not Use_Data_augmentation:
  X, Y_train_masks = train_images, train_masks

# Now we apply data augmentation to the training patches:
# Rotate four times by 90 degree and add flipped versions.
if Use_Data_augmentation:
  X, Y_train_masks = augment_data(train_images, train_masks)

X_val, Y_val_masks = val_images, val_masks

# Here we add the channel dimension to our input images.
# Dimensionality for training has to be 'SYXC' (Sample, Y-Dimension, X-Dimension, Channel)
X = X[...,np.newaxis]
Y = convert_to_oneHot(Y_train_masks)
X_val = X_val[...,np.newaxis]
Y_val = convert_to_oneHot(Y_val_masks)
print("Shape of X:     {}".format(X.shape))
print("Shape of Y:     {}".format(Y.shape))
print("Shape of X_val: {}".format(X_val.shape))
print("Shape of Y_val: {}".format(Y_val.shape))

#Here we automatically define number_of_step in function of training data and batch size
#Here we ensure that our network has a minimal number of steps
if (Use_Default_Advanced_Parameters):
  number_of_steps= max(100, min(int(X.shape[0]/batch_size), 400))

# --------------------- Using pretrained model ------------------------
#Here we ensure that the learning rate set correctly when using pre-trained models
if Use_pretrained_model:
  if Weights_choice == "last":
    initial_learning_rate = lastLearningRate

  if Weights_choice == "best":
    initial_learning_rate = bestLearningRate
# --------------------- ---------------------- ------------------------

# create a Config object

config = DenoiSegConfig(X, unet_kern_size=3, n_channel_out=4, relative_weights = [1.0,1.0,5.0],
                      train_steps_per_epoch=number_of_steps, train_epochs=number_of_epochs,
                      batch_norm=True, train_batch_size=batch_size, unet_n_first = 32,
                      unet_n_depth=4, denoiseg_alpha=Priority, train_learning_rate = initial_learning_rate, train_tensorboard=False)

# Let's look at the parameters stored in the config-object.
vars(config)

# create network model.

model = DenoiSeg(config=config, name=model_name, basedir=model_path)

# --------------------- Using pretrained model ------------------------
# Load the pretrained weights
if Use_pretrained_model:
  model.load_weights(h5_file_path)
# --------------------- ---------------------- ------------------------

print("Setup done.")
print(config)

_____no_output_____

## **4.2. Train the network**
---
When playing the cell below you should see updates after each epoch (round). Network training can take some time.

* **CRITICAL NOTE:** Google Colab has a time limit for processing (to prevent using GPU power for datamining). Training time must be less than 12 hours! If training takes longer than 12 hours, please decrease the number of epochs or number of patches. Another way circumvent this is to save the parameters of the model after training and start training again from this point.

**Of Note:** At the end of the training, your model will be automatically exported so it can be used in the CSBDeep Fiji plugin (DenoiSeg -- DenoiSeg Predict). You can find it in your model folder (export.bioimage.io.zip and model.yaml). In Fiji, Make sure to choose the right version of tensorflow. You can check at: Edit-- Options-- Tensorflow. Choose the version 1.4 (CPU or GPU depending on your system)._____no_output_____

start = time.time()

#@markdown ##Start Training
%memit

history = model.train(X, Y, (X_val, Y_val))

print("Training done.")
%memit

print("Training, done.")

threshold, val_score = model.optimize_thresholds(val_images[:available_val_masks.shape[0]].astype(np.float32), val_masks, measure=measure_precision())

print("The higest score of {} is achieved with threshold = {}.".format(np.round(val_score, 3), threshold))

# convert the history.history dict to a pandas DataFrame:
lossData = pd.DataFrame(history.history)

if os.path.exists(model_path+"/"+model_name+"/Quality Control"):
  shutil.rmtree(model_path+"/"+model_name+"/Quality Control")

os.makedirs(model_path+"/"+model_name+"/Quality Control")

# The training evaluation.csv is saved (overwrites the Files if needed).
lossDataCSVpath = model_path+'/'+model_name+'/Quality Control/training_evaluation.csv'
with open(lossDataCSVpath, 'w') as f:
  writer = csv.writer(f)
  writer.writerow(['loss','val_loss', 'learning rate','threshold'])
  for i in range(len(history.history['loss'])):
    writer.writerow([history.history['loss'][i], history.history['val_loss'][i], history.history['lr'][i], str(threshold)])

#Thresholdpath = model_path+'/'+model_name+'/Quality Control/optimal_threshold.csv'
#with open(Thresholdpath, 'w') as f1:
  #writer1 = csv.writer(f1)
  #writer1.writerow(['threshold'])
  #writer1.writerow([str(threshold)])

# Displaying the time elapsed for training
dt = time.time() - start
mins, sec = divmod(dt, 60)
hour, mins = divmod(mins, 60)
print("Time elapsed:",hour, "hour(s)",mins,"min(s)",round(sec),"sec(s)")

model.export_TF(name='DenoiSeg',
                description='DenoiSeg 2D trained using ZeroCostDL4Mic.',
                authors=["You"],
                test_img=X_val[0,...,0], axes='YX',
                patch_shape=(64, 64))

print("Your model has been sucessfully exported and can now also be used in the CSBDeep Fiji plugin")

_____no_output_____

## **4.3. Download your model(s) from Google Drive**
---

Once training is complete, the trained model is automatically saved on your Google Drive, in the **model_path** folder that was selected in Section 3. It is however wise to download the folder as all data can be erased at the next training if using the same folder._____no_output_____# **5. Evaluate your model**
---

This section allows the user to perform important quality checks on the validity and generalisability of the trained model.

**We highly recommend to perform quality control on all newly trained models.**

_____no_output_____

# model name and path
#@markdown ###Do you want to assess the model you just trained ?
Use_the_current_trained_model = True #@param {type:"boolean"}

#@markdown ###If not, please provide the path to the model folder:

QC_model_folder = "" #@param {type:"string"}

#Here we define the loaded model name and path
QC_model_name = os.path.basename(QC_model_folder)
QC_model_path = os.path.dirname(QC_model_folder)

if (Use_the_current_trained_model):
  QC_model_name = model_name
  QC_model_path = model_path

full_QC_model_path = QC_model_path+'/'+QC_model_name+'/'
if os.path.exists(full_QC_model_path):
  print("The "+QC_model_name+" network will be evaluated")
else:

  print(bcolors.WARNING + '!! WARNING: The chosen model does not exist !!')
  print('Please make sure you provide a valid model path and model name before proceeding further.')
_____no_output_____

## **5.1. Inspection of the loss function**
---

It is good practice to evaluate the training progress by comparing the training loss with the validation loss. The latter is a metric which shows how well the network performs on a subset of unseen data which is set aside from the training dataset. For more information on this, see for example [this review](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6381354/) by Nichols *et al.*

**Training loss** describes an error value after each epoch for the difference between the model's prediction and its ground-truth target.

**Validation loss** describes the same error value between the model's prediction on a validation image and compared to it's target.

During training both values should decrease before reaching a minimal value which does not decrease further even after more training. Comparing the development of the validation loss with the training loss can give insights into the model's performance.

Decreasing **Training loss** and **Validation loss** indicates that training is still necessary and increasing the `number_of_epochs` is recommended. Note that the curves can look flat towards the right side, just because of the y-axis scaling. The network has reached convergence once the curves flatten out. After this point no further training is required. If the **Validation loss** suddenly increases again an the **Training loss** simultaneously goes towards zero, it means that the network is overfitting to the training data. In other words the network is remembering the exact noise patterns from the training data and no longer generalizes well to unseen data. In this case the training dataset has to be increased._____no_output_____

#@markdown ##Play the cell to show a plot of training errors vs. epoch number

lossDataFromCSV = []
vallossDataFromCSV = []

with open(QC_model_path+'/'+QC_model_name+'/Quality Control/training_evaluation.csv','r') as csvfile:
    csvRead = csv.reader(csvfile, delimiter=',')
    next(csvRead)
    for row in csvRead:
        lossDataFromCSV.append(float(row[0]))
        vallossDataFromCSV.append(float(row[1]))

epochNumber = range(len(lossDataFromCSV))
plt.figure(figsize=(15,10))

plt.subplot(2,1,1)
plt.plot(epochNumber,lossDataFromCSV, label='Training loss')
plt.plot(epochNumber,vallossDataFromCSV, label='Validation loss')
plt.title('Training loss and validation loss vs. epoch number (linear scale)')
plt.ylabel('Loss')
plt.xlabel('Epoch number')
plt.legend()

plt.subplot(2,1,2)
plt.semilogy(epochNumber,lossDataFromCSV, label='Training loss')
plt.semilogy(epochNumber,vallossDataFromCSV, label='Validation loss')
plt.title('Training loss and validation loss vs. epoch number (log scale)')
plt.ylabel('Loss')
plt.xlabel('Epoch number')
plt.legend()
plt.savefig(QC_model_path+'/'+QC_model_name+'/Quality Control/lossCurvePlots.png')
plt.show()

_____no_output_____

## **5.2. Error mapping and quality metrics estimation**
---
**DenoiSeg** allow to both denoise and segment microscopy images. This section allow you to evaluate both tasks separetly.

**Evaluation of the denoising**

This section will display SSIM maps and RSE maps as well as calculating total SSIM, NRMSE and PSNR metrics for all the images provided in the "Source_QC_folder" and "Target_Denoising_folder" !

**1. The SSIM (structural similarity) map**

The SSIM metric is used to evaluate whether two images contain the same structures. It is a normalized metric and an SSIM of 1 indicates a perfect similarity between two images. Therefore for SSIM, the closer to 1, the better. The SSIM maps are constructed by calculating the SSIM metric in each pixel by considering the surrounding structural similarity in the neighbourhood of that pixel (currently defined as window of 11 pixels and with Gaussian weighting of 1.5 pixel standard deviation, see our Wiki for more info).

**mSSIM** is the SSIM value calculated across the entire window of both images.

**The output below shows the SSIM maps with the mSSIM**

**2. The RSE (Root Squared Error) map**

This is a display of the root of the squared difference between the normalized predicted and target or the source and the target. In this case, a smaller RSE is better. A perfect agreement between target and prediction will lead to an RSE map showing zeros everywhere (dark).

**NRMSE (normalised root mean squared error)** gives the average difference between all pixels in the images compared to each other. Good agreement yields low NRMSE scores.

**PSNR (Peak signal-to-noise ratio)** is a metric that gives the difference between the ground truth and prediction (or source input) in decibels, using the peak pixel values of the prediction and the MSE between the images. The higher the score the better the agreement.

**The output below shows the RSE maps with the NRMSE and PSNR values.**

**Evaluation of the Segmentation**

This option will calculate the Intersection over Union score for all the images provided in the Source_QC_folder and Target_Segmentation_folder ! The result for one of the image will also be displayed.

The **Intersection over Union** metric is a method that can be used to quantify the percent overlap between the target mask and your prediction output. **Therefore, the closer to 1, the better the performance.** This metric can be used to assess the quality of your model to accurately predict nuclei.

_____no_output_____

#@markdown ##Choose what to evaluate

Evaluate_Denoising = True #@param {type:"boolean"}

Evaluate_Segmentation = True #@param {type:"boolean"}

# ------------- User input ------------
#@markdown ##Choose the folders that contain your Quality Control dataset
Source_QC_folder = "" #@param{type:"string"}
Target_Denoising_folder = "" #@param{type:"string"}
Target_Segmentation_folder = "" #@param{type:"string"}

#@markdown ###If your model was trained outside of ZeroCostDl4Mic, please provide a threshold value for the segmentation (between 0-1):

threshold = 0.5 #@param {type:"number"}

# Create a quality control/Prediction Folder
if os.path.exists(QC_model_path+"/"+QC_model_name+"/Quality Control/Prediction"):
  shutil.rmtree(QC_model_path+"/"+QC_model_name+"/Quality Control/Prediction")

os.makedirs(QC_model_path+"/"+QC_model_name+"/Quality Control/Prediction")

#Activate the pretrained model.
config = None
model = DenoiSeg(config=None, name=QC_model_name, basedir=QC_model_path)

#Load the threshold value.

if os.path.exists(os.path.join(full_QC_model_path, 'Quality Control', 'training_evaluation.csv')):

  with open(os.path.join(full_QC_model_path, 'Quality Control', 'training_evaluation.csv'),'r') as csvfile:
    csvRead = pd.read_csv(csvfile, sep=',')
        #print(csvRead)

    if "threshold" in csvRead.columns: #Here we check that the learning rate column exist (compatibility with model trained un ZeroCostDL4Mic bellow 1.4)
      print("Optimal segmentation threshold found")
    #find the last learning rate
      threshold = csvRead["threshold"].iloc[-1]

# ------------- Prepare the model and run predictions ------------
# creates a loop, creating filenames and saving them

thisdir = Path(Source_QC_folder)

# r=root, d=directories, f = files
for r, d, f in os.walk(thisdir):
    for file in f:
        if ".tif" in file:
            print(os.path.join(r, file))

for r, d, f in os.walk(thisdir):
  for file in f:

#Here we load the images
    base_filename = os.path.basename(file)
    test_images = imread(os.path.join(r, file))

#Here we perform the predictions
    predicted_channels = model.predict(test_images.astype(np.float32), axes='YX')
    denoised_images= predicted_channels[...,0]
    segmented_images= (compute_labels(predicted_channels, threshold))

#Here we save the results
    io.imsave(QC_model_path+"/"+QC_model_name+"/Quality Control/Prediction"+"/"+"Predicted_denoised_"+base_filename, denoised_images)
    io.imsave(QC_model_path+"/"+QC_model_name+"/Quality Control/Prediction"+"/"+"Predicted_segmentation_"+base_filename, segmented_images)

# ------------- Here we Start assessing the denoising against GT ------------

if Evaluate_Denoising:
  def ssim(img1, img2):
    return structural_similarity(img1,img2,data_range=1.,full=True)

  def normalize(x, pmin=3, pmax=99.8, axis=None, clip=False, eps=1e-20, dtype=np.float32):
      """This function is adapted from Martin Weigert"""
      """Percentile-based image normalization."""

      mi = np.percentile(x,pmin,axis=axis,keepdims=True)
      ma = np.percentile(x,pmax,axis=axis,keepdims=True)
      return normalize_mi_ma(x, mi, ma, clip=clip, eps=eps, dtype=dtype)

  def normalize_mi_ma(x, mi, ma, clip=False, eps=1e-20, dtype=np.float32):#dtype=np.float32
      """This function is adapted from Martin Weigert"""
      if dtype is not None:
          x   = x.astype(dtype,copy=False)
          mi  = dtype(mi) if np.isscalar(mi) else mi.astype(dtype,copy=False)
          ma  = dtype(ma) if np.isscalar(ma) else ma.astype(dtype,copy=False)
          eps = dtype(eps)

      try:
          import numexpr
          x = numexpr.evaluate("(x - mi) / ( ma - mi + eps )")
      except ImportError:
          x =                   (x - mi) / ( ma - mi + eps )

      if clip:
          x = np.clip(x,0,1)

      return x

  def norm_minmse(gt, x, normalize_gt=True):
      """This function is adapted from Martin Weigert"""

      """
      normalizes and affinely scales an image pair such that the MSE is minimized

      Parameters
      ----------
      gt: ndarray
          the ground truth image
      x: ndarray
          the image that will be affinely scaled
      normalize_gt: bool
          set to True of gt image should be normalized (default)
      Returns
      -------
      gt_scaled, x_scaled
      """
      if normalize_gt:
        gt = normalize(gt, 0.1, 99.9, clip=False).astype(np.float32, copy = False)
        x = x.astype(np.float32, copy=False) - np.mean(x)
        gt = gt.astype(np.float32, copy=False) - np.mean(gt)
        scale = np.cov(x.flatten(), gt.flatten())[0, 1] / np.var(x.flatten())
        return gt, scale * x

# Open and create the csv file that will contain all the QC metrics
  with open(QC_model_path+"/"+QC_model_name+"/Quality Control/QC_metrics_Denoising_"+QC_model_name+".csv", "w", newline='') as file:
      writer = csv.writer(file)

    # Write the header in the csv file
      writer.writerow(["image #","Prediction v. GT mSSIM","Input v. GT mSSIM", "Prediction v. GT NRMSE", "Input v. GT NRMSE", "Prediction v. GT PSNR", "Input v. GT PSNR"])

    # Let's loop through the provided dataset in the QC folders

      for i in os.listdir(Source_QC_folder):
        if not os.path.isdir(os.path.join(Source_QC_folder,i)):
          print('Running QC on: '+i)
      # -------------------------------- Target test data (Ground truth) --------------------------------
          test_GT = io.imread(os.path.join(Target_Denoising_folder, i))

      # -------------------------------- Source test data --------------------------------
          test_source = io.imread(os.path.join(Source_QC_folder,i))

      # Normalize the images wrt each other by minimizing the MSE between GT and Source image
          test_GT_norm,test_source_norm = norm_minmse(test_GT, test_source, normalize_gt=True)

      # -------------------------------- Prediction --------------------------------
          test_prediction = io.imread(os.path.join(QC_model_path+"/"+QC_model_name+"/Quality Control/Prediction","Predicted_denoised_"+i))

      # Normalize the images wrt each other by minimizing the MSE between GT and prediction
          test_GT_norm,test_prediction_norm = norm_minmse(test_GT, test_prediction, normalize_gt=True)

      # -------------------------------- Calculate the metric maps and save them --------------------------------

      # Calculate the SSIM maps
          index_SSIM_GTvsPrediction, img_SSIM_GTvsPrediction = ssim(test_GT_norm, test_prediction_norm)
          index_SSIM_GTvsSource, img_SSIM_GTvsSource = ssim(test_GT_norm, test_source_norm)

      #Save ssim_maps
          img_SSIM_GTvsPrediction_32bit = np.float32(img_SSIM_GTvsPrediction)
          io.imsave(QC_model_path+'/'+QC_model_name+'/Quality Control/SSIM_GTvsPrediction_'+i,img_SSIM_GTvsPrediction_32bit)
          img_SSIM_GTvsSource_32bit = np.float32(img_SSIM_GTvsSource)
          io.imsave(QC_model_path+'/'+QC_model_name+'/Quality Control/SSIM_GTvsSource_'+i,img_SSIM_GTvsSource_32bit)

      # Calculate the Root Squared Error (RSE) maps
          img_RSE_GTvsPrediction = np.sqrt(np.square(test_GT_norm - test_prediction_norm))
          img_RSE_GTvsSource = np.sqrt(np.square(test_GT_norm - test_source_norm))

      # Save SE maps
          img_RSE_GTvsPrediction_32bit = np.float32(img_RSE_GTvsPrediction)
          img_RSE_GTvsSource_32bit = np.float32(img_RSE_GTvsSource)
          io.imsave(QC_model_path+'/'+QC_model_name+'/Quality Control/RSE_GTvsPrediction_'+i,img_RSE_GTvsPrediction_32bit)
          io.imsave(QC_model_path+'/'+QC_model_name+'/Quality Control/RSE_GTvsSource_'+i,img_RSE_GTvsSource_32bit)

      # -------------------------------- Calculate the RSE metrics and save them --------------------------------

      # Normalised Root Mean Squared Error (here it's valid to take the mean of the image)
          NRMSE_GTvsPrediction = np.sqrt(np.mean(img_RSE_GTvsPrediction))
          NRMSE_GTvsSource = np.sqrt(np.mean(img_RSE_GTvsSource))

      # We can also measure the peak signal to noise ratio between the images
          PSNR_GTvsPrediction = psnr(test_GT_norm,test_prediction_norm,data_range=1.0)
          PSNR_GTvsSource = psnr(test_GT_norm,test_source_norm,data_range=1.0)

          writer.writerow([i,str(index_SSIM_GTvsPrediction),str(index_SSIM_GTvsSource),str(NRMSE_GTvsPrediction),str(NRMSE_GTvsSource),str(PSNR_GTvsPrediction),str(PSNR_GTvsSource)])

# All data is now processed saved
  Test_FileList = os.listdir(Source_QC_folder) # this assumes, as it should, that both source and target are named the same
  norm = simple_norm(x, percent = 99)

  plt.figure(figsize=(15,15))
  #  Currently only displays the last computed set, from memory
  # Target (Ground-truth)
  plt.subplot(3,3,1)
  plt.axis('off')
  img_GT = io.imread(os.path.join(Target_Denoising_folder, Test_FileList[-1]))
  plt.imshow(img_GT, norm=norm, cmap='magma', interpolation='nearest')
  plt.title('Target',fontsize=15)

# Source
  plt.subplot(3,3,2)
  plt.axis('off')
  img_Source = io.imread(os.path.join(Source_QC_folder, Test_FileList[-1]))
  plt.imshow(img_Source, norm=norm, cmap='magma', interpolation='nearest')
  plt.title('Source',fontsize=15)

#Prediction
  plt.subplot(3,3,3)
  plt.axis('off')
  img_Prediction = io.imread(os.path.join(QC_model_path+"/"+QC_model_name+"/Quality Control/Prediction/", "Predicted_denoised_"+Test_FileList[-1]))
  plt.imshow(img_Prediction, norm=norm, cmap='magma', interpolation='nearest')
  plt.title('Prediction',fontsize=15)

#Setting up colours
  cmap = plt.cm.CMRmap

  #SSIM between GT and Source
  plt.subplot(3,3,5)
  #plt.axis('off')
  plt.tick_params(
      axis='both',      # changes apply to the x-axis and y-axis
      which='both',      # both major and minor ticks are affected
      bottom=False,      # ticks along the bottom edge are off
      top=False,        # ticks along the top edge are off
      left=False,       # ticks along the left edge are off
      right=False,         # ticks along the right edge are off
      labelbottom=False,
      labelleft=False)
  imSSIM_GTvsSource = plt.imshow(img_SSIM_GTvsSource, cmap = cmap, vmin=0, vmax=1)
  plt.colorbar(imSSIM_GTvsSource,fraction=0.046, pad=0.04)
  plt.title('Target vs. Source',fontsize=15)
  plt.xlabel('mSSIM: '+str(round(index_SSIM_GTvsSource,3)),fontsize=14)
  plt.ylabel('SSIM maps',fontsize=20, rotation=0, labelpad=75)

#SSIM between GT and Prediction
  plt.subplot(3,3,6)
  #plt.axis('off')
  plt.tick_params(
      axis='both',      # changes apply to the x-axis and y-axis
      which='both',      # both major and minor ticks are affected
      bottom=False,      # ticks along the bottom edge are off
      top=False,        # ticks along the top edge are off
      left=False,       # ticks along the left edge are off
      right=False,         # ticks along the right edge are off
      labelbottom=False,
      labelleft=False)
  imSSIM_GTvsPrediction = plt.imshow(img_SSIM_GTvsPrediction, cmap = cmap, vmin=0,vmax=1)
  plt.colorbar(imSSIM_GTvsPrediction,fraction=0.046, pad=0.04)
  plt.title('Target vs. Prediction',fontsize=15)
  plt.xlabel('mSSIM: '+str(round(index_SSIM_GTvsPrediction,3)),fontsize=14)

#Root Squared Error between GT and Source
  plt.subplot(3,3,8)
#plt.axis('off')
  plt.tick_params(
      axis='both',      # changes apply to the x-axis and y-axis
      which='both',      # both major and minor ticks are affected
      bottom=False,      # ticks along the bottom edge are off
      top=False,        # ticks along the top edge are off
      left=False,       # ticks along the left edge are off
      right=False,         # ticks along the right edge are off
      labelbottom=False,
      labelleft=False)
  imRSE_GTvsSource = plt.imshow(img_RSE_GTvsSource, cmap = cmap, vmin=0, vmax = 1)
  plt.colorbar(imRSE_GTvsSource,fraction=0.046,pad=0.04)
  plt.title('Target vs. Source',fontsize=15)
  plt.xlabel('NRMSE: '+str(round(NRMSE_GTvsSource,3))+', PSNR: '+str(round(PSNR_GTvsSource,3)),fontsize=14)

  plt.ylabel('RSE maps',fontsize=20, rotation=0, labelpad=75)

#Root Squared Error between GT and Prediction
  plt.subplot(3,3,9)
  #plt.axis('off')
  plt.tick_params(
      axis='both',      # changes apply to the x-axis and y-axis
      which='both',      # both major and minor ticks are affected
      bottom=False,      # ticks along the bottom edge are off
      top=False,        # ticks along the top edge are off
      left=False,       # ticks along the left edge are off
      right=False,         # ticks along the right edge are off
      labelbottom=False,
      labelleft=False)
  imRSE_GTvsPrediction = plt.imshow(img_RSE_GTvsPrediction, cmap = cmap, vmin=0, vmax=1)
  plt.colorbar(imRSE_GTvsPrediction,fraction=0.046,pad=0.04)
  plt.title('Target vs. Prediction',fontsize=15)
  plt.xlabel('NRMSE: '+str(round(NRMSE_GTvsPrediction,3))+', PSNR: '+str(round(PSNR_GTvsPrediction,3)),fontsize=14)

#________________________________________________________________________
# Here we start testing the differences between GT and predicted masks

if Evaluate_Segmentation:

  with open(QC_model_path+"/"+QC_model_name+"/Quality Control/Quality_Control_Segmentation for "+QC_model_name+".csv", "w", newline='') as file:
    writer = csv.writer(file)
    writer.writerow(["image","Prediction v. GT Intersection over Union"])

# define the images

    for n in os.listdir(Source_QC_folder):

      if not os.path.isdir(os.path.join(Source_QC_folder,n)):
        print('Running QC on: '+n)
        test_input = io.imread(os.path.join(Source_QC_folder,n))
        test_prediction = io.imread(os.path.join(QC_model_path+"/"+QC_model_name+"/Quality Control/Prediction","Predicted_segmentation_"+n))
        test_ground_truth_image = io.imread(os.path.join(Target_Segmentation_folder, n))

      #Convert pixel values to 0 or 255
        test_prediction_0_to_255 = test_prediction
        test_prediction_0_to_255[test_prediction_0_to_255>0] = 255

      #Convert pixel values to 0 or 255
        test_ground_truth_0_to_255 = test_ground_truth_image
        test_ground_truth_0_to_255[test_ground_truth_0_to_255>0] = 255

      # Intersection over Union metric

        intersection = np.logical_and(test_ground_truth_0_to_255, test_prediction_0_to_255)
        union = np.logical_or(test_ground_truth_0_to_255, test_prediction_0_to_255)
        iou_score =  np.sum(intersection) / np.sum(union)
        writer.writerow([n, str(iou_score)])

#Display the last image

  f = plt.figure(figsize=(25,25))

  from astropy.visualization import simple_norm
  norm = simple_norm(test_input, percent = 99)

#Input
  plt.subplot(1,4,1)
  plt.axis('off')
  plt.imshow(test_input, aspect='equal', norm=norm, cmap='magma', interpolation='nearest')
  plt.title('Input')

#Ground-truth
  plt.subplot(1,4,2)
  plt.axis('off')
  plt.imshow(test_ground_truth_0_to_255, aspect='equal', cmap='Greens')
  plt.title('Ground Truth')

#Prediction
  plt.subplot(1,4,3)
  plt.axis('off')
  plt.imshow(test_prediction_0_to_255, aspect='equal', cmap='Purples')
  plt.title('Prediction')

#Overlay
  plt.subplot(1,4,4)
  plt.axis('off')
  plt.imshow(test_ground_truth_0_to_255, cmap='Greens')
  plt.imshow(test_prediction_0_to_255, alpha=0.5, cmap='Purples')
  plt.title('Ground Truth and Prediction, Intersection over Union:'+str(round(iou_score,3)));

_____no_output_____

# **6. Using the trained model**

---

In this section the unseen data is processed using the trained model (in section 4). First, your unseen images are uploaded and prepared for prediction. After that your trained model from section 4 is activated and finally saved into your Google Drive._____no_output_____## **6.1. Generate prediction(s) from unseen dataset**
---

The current trained model (from section 4.2) can now be used to process images. If an older model needs to be used, please untick the **Use_the_current_trained_model** box and enter the name and path of the model to use. Predicted output images are saved in your **Result_folder** folder as restored image stacks (ImageJ-compatible TIFF images).

**`Data_folder`:** This folder should contains the images that you want to predict using the network that you will train.

**`Result_folder`:** This folder will contain the predicted output images._____no_output_____

import imageio

#@markdown ### Provide the path to your dataset and to the folder where the prediction will be saved, then play the cell to predict output on your unseen images.

#@markdown ###Path to data to analyse and where predicted output should be saved:
Data_folder = "" #@param {type:"string"}
Result_folder = "" #@param {type:"string"}

# model name and path
#@markdown ###Do you want to use the current trained model?
Use_the_current_trained_model = True #@param {type:"boolean"}

#@markdown ###If not, please provide the path to the model folder:

Prediction_model_folder = "" #@param {type:"string"}

#@markdown ###If your model was trained outside of ZeroCostDl4Mic, please provide a Threshold value for the segmentation (between 0-1):

threshold = 0.5 #@param {type:"number"}

#Here we find the loaded model name and parent path
Prediction_model_name = os.path.basename(Prediction_model_folder)
Prediction_model_path = os.path.dirname(Prediction_model_folder)

if (Use_the_current_trained_model):
  print("Using current trained network")
  Prediction_model_name = model_name
  Prediction_model_path = model_path

full_Prediction_model_path = Prediction_model_path+'/'+Prediction_model_name+'/'
if os.path.exists(full_Prediction_model_path):
  print("The "+Prediction_model_name+" network will be used.")
else:
  print(bcolors.WARNING +'!! WARNING: The chosen model does not exist !!')
  print('Please make sure you provide a valid model path and model name before proceeding further.')

#Activate the pretrained model.
config = None
model = DenoiSeg(config=None, name=Prediction_model_name, basedir=Prediction_model_path)

#Load the threshold value.

if os.path.exists(os.path.join(full_Prediction_model_path, 'Quality Control', 'training_evaluation.csv')):

  with open(os.path.join(full_Prediction_model_path, 'Quality Control', 'training_evaluation.csv'),'r') as csvfile:
    csvRead = pd.read_csv(csvfile, sep=',')
        #print(csvRead)

    if "threshold" in csvRead.columns: #Here we check that the learning rate column exist (compatibility with model trained un ZeroCostDL4Mic bellow 1.4)
      print("Optimal segmentation threshold found")
    #find the last learning rate
      threshold = csvRead["threshold"].iloc[-1]

# creates a loop, creating filenames and saving them

thisdir = Path(Data_folder)
outputdir = Path(Result_folder)

# r=root, d=directories, f = files
for r, d, f in os.walk(thisdir):
    for file in f:
        if ".tif" in file:
            print(os.path.join(r, file))

print("Processing...")
for r, d, f in os.walk(thisdir):
  for file in f:

#Here we load the images
    base_filename = os.path.basename(file)
    test_images = imread(os.path.join(r, file))

#Here we perform the predictions
    predicted_channels = model.predict(test_images.astype(np.float32), axes='YX')
    denoised_images= predicted_channels[...,0]
    segmented_images= (compute_labels(predicted_channels, threshold))

#Here we save the results
    io.imsave(Result_folder+"/"+"Predicted_denoised_"+base_filename, denoised_images)
    io.imsave(Result_folder+"/"+"Predicted_segmentation_"+base_filename,segmented_images)

print("Images saved into folder:", Result_folder)_____no_output_____

## **6.2. Assess predicted output**
---

_____no_output_____

# @markdown ##Run this cell to display a randomly chosen input and its corresponding predicted output.

# This will display a randomly chosen dataset input and predicted output
random_choice = random.choice(os.listdir(Data_folder))
x = imread(Data_folder+"/"+random_choice)

os.chdir(Result_folder)
y = imread(Result_folder+"/"+"Predicted_denoised_"+random_choice)
z = imread(Result_folder+"/"+"Predicted_segmentation_"+random_choice)

norm = simple_norm(x, percent = 99)

plt.figure(figsize=(30,15))
plt.subplot(1, 4, 1)
plt.imshow(x, interpolation='nearest', norm=norm, cmap='magma')
plt.axis('off');
plt.title("Input")

plt.subplot(1, 4, 2)
plt.imshow(y, interpolation='nearest', norm=norm, cmap='magma')
plt.axis('off');
plt.title("Predicted denoised image")

plt.subplot(1, 4, 3)
plt.imshow(z, interpolation='nearest', vmin=0, vmax=1, cmap='viridis')
plt.axis('off');
plt.title("Predicted segmentation")

plt.show()
_____no_output_____

## **6.3. Download your predictions**
---

**Store your data** and ALL its results elsewhere by downloading it from Google Drive and after that clean the original folder tree (datasets, results, trained model etc.) if you plan to train or use new networks. Please note that the notebook will otherwise **OVERWRITE** all files which have the same name._____no_output_____#**Thank you for using DenoiSeg!**_____no_output_____


# Notebook from thoward27/udacity-datascience
Path: investigating-titanic-dataset/Titanic Analysis.ipynb

# Titanic Data Analysis

Welcome to my analysis of the Titanic Data Set.

## Introduction
A brief introduction from [Wikipedia](https://en.wikipedia.org/wiki/RMS_Titanic):
>[The] RMS Titanic (/taɪˈtænɪk/) was a British passenger liner that sank in the North Atlantic Ocean in the early morning of 15 April 1912, after colliding with an iceberg during her maiden voyage from Southampton to New York City. Of the 2,224 passengers and crew aboard, more than 1,500 died, making it one of the deadliest commercial peacetime maritime disasters in modern history. The largest ship afloat at the time it entered service, the RMS Titanic was the second of three Olympic class ocean liners operated by the White Star Line, and was built by the Harland and Wolff shipyard in Belfast. Thomas Andrews, her architect, died in the disaster.

## Main Question
For this analysis I will endeavour to answer one simple question: is there a statistically significant difference between male and female survival rates? More specificially, if females had a statistically significant advantage over their male counterparts. The results will be held to $\alpha$ = 0.01, and tested as a 1-Tailed Z-Score.

This question, posed formally:
$$H_0: \mu_m - \mu_f = 0$$
$$H_1: \mu_m - \mu_f < 0$$
Where $\mu$ is the survival rate for each respective gender.
_____no_output_____## Table Descriptions

### VARIABLE DESCRIPTIONS:
- survival        Survival (0 = No; 1 = Yes)
- pclass          Passenger Class (1 = 1st; 2 = 2nd; 3 = 3rd)
- name            Name
- sex             Sex
- age             Age
- sibsp           Number of Siblings/Spouses Aboard
- parch           Number of Parents/Children Aboard
- ticket          Ticket Number
- fare            Passenger Fare
- cabin           Cabin
- embarked        Port of Embarkation (C = Cherbourg; Q = Queenstown; S = Southampton)

### SPECIAL NOTES:
Pclass is a proxy for socio-economic status (SES)
 1st ~ Upper; 2nd ~ Middle; 3rd ~ Lower

Age is in Years; Fractional if Age less than One (1)
 If the Age is Estimated, it is in the form xx.5

With respect to the family relation variables (i.e. sibsp and parch)
some relations were ignored.  The following are the definitions used
for sibsp and parch.

- Sibling:  Brother, Sister, Stepbrother, or Stepsister of Passenger Aboard Titanic
- Spouse:   Husband or Wife of Passenger Aboard Titanic (Mistresses and Fiances Ignored)
- Parent:   Mother or Father of Passenger Aboard Titanic
- Child:    Son, Daughter, Stepson, or Stepdaughter of Passenger Aboard Titanic

Other family relatives excluded from this study include cousins,
nephews/nieces, aunts/uncles, and in-laws.  Some children travelled
only with a nanny, therefore parch=0 for them.  As well, some
travelled with very close friends or neighbors in a village, however,
the definitions do not support such relations.

## Importing modules, data, and quick data checking_____no_output_____

#import packages
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
from scipy import stats
%matplotlib inline

#load data
df = pd.read_csv("titanic-data.csv")
df.head()_____no_output_____

### Checking Data Types
Doing this to verify variables are reading in as intended._____no_output_____

df.columns.to_series().groupby(df.dtypes).groups_____no_output_____

## Defining Functions and Data Wrangling
Some initial functions to begin working with the data. Also directly beneath my functions are any statements to modify the original data set necessary to the rest of the analysis.

*please note my naming convention for a function I plan on using in a .apply is appFunction - short for applyFunction*_____no_output_____

def appGender(row):
    """DEPRECATED thanks to helpful suggestion from first reviewer. Left for clarity of process!
    Switched to a .map function (which is way cleaner!)

    Returns 0 for male, 1 for Female

    Used in df.apply to change 'male' / 'female' to 0 / 1 respectively.
    """
    if row['Sex'] == 'male': return 0
    else: return 1_____no_output_____df['gender'] = df['Sex'].map({'male':0, 'female':1})
df['strSurvived'] = df['Survived'].map({0:'Died', 1:'Survived'})
df.head()_____no_output_____

## Wrangling and Exploring
I'm going to start this process by simply examining the full dataframe, checking for missing values, getting a basic lay of the data. Then I'll move down into descriptions of gender seperated tables.

### Population Statistics
Now that we've gotten a good clean table to work from, and added additional columns for analytical purposes, we can begin to look at how the entire population did, then subsection it down and compare. _____no_output_____

df.describe()_____no_output_____

#### Holy insights!
Let's break down some key numbers:
- There are 891 passengers in the set. Of those, 177 are missing ages. To address the missing values I will substitute the mean age for any NaN ages, after the next section. It appears that Age is the only field with missing values as the rest are consistent at 891.
- It appears that there are no other missing fields with missing values from our initial table description. If any more are uncovered they will be addressed and dealt with midstream.
- The overall survival rate was ~38% ($\mu = 38\%$)
- Of the passengers, ~35% were female.

To examine our hypothesis we we look to see if females have a abnormally larger survival rate than males:_____no_output_____

# a look at females general population statistics
df[df['gender'] == 1].describe()_____no_output_____# a look at male general population statistics
df[df['gender'] == 0].describe()_____no_output_____

#### Digging Deeper: A first look at Female v.s. Male populations
From these two tables we can get some real insight to run on, we now know:
- There were 314 females, and 577 males.
- All missing ages were from the female population.
- Survival rate = Males: 18% | Females: 74%
- Average Age = Males: 30 | Females: 27

#### Checking correlations at a higher level
A quick look at Perasons product-moment correlation coefficient to corroborate our initial findings. _____no_output_____

pearson = np.corrcoef(x=df['gender'], y=df['Survived'])
print(pearson.mean())0.771675690329

Yes, we can see that gender and survived are very positively correlated (keep in mind 1 is female and 1 is survived).

Now let's fix the missing age values:_____no_output_____

# fills df['Age'] missing values with the mean from the same column
df['Age'].fillna(df['Age'].mean())
print("Ages fixed!")Ages fixed!

### Taking it even further
Now that the data is whole, it's time to look at our data via some neat graphs. First we'll establish formatting, then examine various aspects of the data. _____no_output_____

#first we must format :)
sns.set(font_scale=1.6,
        font='calibri',
        rc={
            'grid.color': '.8',
        },
)
# naming convention for colors = {Gender (f/m)}{Class (1/2/3)}
f1 = '#c39738'
f2 = '#9D44B5'
f3 = '#B5446E'
m1 = '#525252'
m2 = '#7A5C61'
m3 = '#6874E8'
# split the df up just for plotting [NO MANIPULATION DONE WITH THESE SETS]
# sole purpose is so I don't have to waste space with gender conditions for every chart..
women = df[(df.gender == 1)]
men = df[(df.gender == 0)]_____no_output_____

### Charts!
We're formatted, we're clean, we're ready to chart!

I've decided to start with a pairplot, to examine basic relations between multiple factors in the df._____no_output_____

df.head()_____no_output_____plt.figure()
sns.pairplot(data=df,
             hue='strSurvived',
             dropna=True,
             x_vars=('Pclass', 'gender', 'Age', 'Fare'),
             y_vars=('Pclass', 'gender', 'Age', 'Fare', 'Parch', 'SibSp')
)
plt.title("Pair Plotting of Titanic Data, Colored by Survival (Survived / Died)")
plt.show()_____no_output_____

#### Results of pairplot
Few very interesting notes from the chart above are:
- Seems as though younger people had better odds than old.
- Higher fair seems to correlate slightly with survival rate.
- Younger people tended to survive regardless of fare.
- Older males may have given up seats for younger, or higher-class individuals. _____no_output_____

# establlish our subplots, going to go with a 2 by 3 grid to examine relationships between gender, class, and survival.
fig, axes = plt.subplots(2, 3, figsize=(18, 13))
# bins since they're all the same size / shape
N = 2
ind = np.arange(N)
width = .75

# Starting with women in first class -> second -> third; then men first -> second -> third
ax1 = plt.subplot2grid((2, 3), (0,0))
data = (
    women.Survived[(women.Survived == 1) & (women.Pclass == 1)].count(),
    women.Survived[(women.Survived == 0) & (women.Pclass == 1)].count(),
)
plt.bar(ind, data, width, color=f1)
plt.xticks(ind, ('Survived', 'Died'))
plt.ylabel("Number of Passengers")
plt.title("Survival of Women in First Class")

ax2 = plt.subplot2grid((2, 3), (0, 1))
data = (
    women.Survived[(women.Survived == 1) & (women.Pclass == 2)].count(),
    women.Survived[(women.Survived == 0) & (women.Pclass == 2)].count(),
)
plt.bar(ind, data, width, color=f2)
plt.xticks(ind, ('Survived', 'Died'))
plt.ylabel("Number of Passengers")
plt.title("Survival of Women in Second Class")

ax3 = plt.subplot2grid((2, 3), (0, 2))
data = (
    women.Survived[(women.Survived == 1) & (women.Pclass == 3)].count(),
    women.Survived[(women.Survived == 0) & (women.Pclass == 3)].count(),
)
plt.bar(ind, data, width, color=f3)
plt.xticks(ind, ('Survived', 'Died'))
plt.ylabel("Number of Passengers")
plt.title("Survival of Women in Third Class")

ax4 = plt.subplot2grid((2, 3), (1, 0))
data = (
    men.Survived[(men.Survived == 1) & (men.Pclass == 1)].count(),
    men.Survived[(men.Survived == 0) & (men.Pclass == 1)].count(),
)
plt.bar(ind, data, width, color=m1)
plt.xticks(ind, ('Survived', 'Died'))
plt.ylabel("Number of Passengers")
plt.title("Survival of Men in First Class")

ax5 = plt.subplot2grid((2, 3), (1, 1))
data = (
    men.Survived[(men.Survived == 1) & (men.Pclass == 2)].count(),
    men.Survived[(men.Survived == 0) & (men.Pclass == 2)].count(),
)
plt.bar(ind, data, width, color=m2)
plt.xticks(ind, ('Survived', 'Died'))
plt.ylabel("Number of Passengers")
plt.title("Survival of Men in Second Class")

ax6 = plt.subplot2grid((2, 3), (1, 2))
data = (
    men.Survived[(men.Survived == 1) & (men.Pclass == 3)].count(),
    men.Survived[(men.Survived == 0) & (men.Pclass == 3)].count(),
)
plt.bar(ind, data, width, color=m3)
plt.xticks(ind, ('Survived', 'Died'))
plt.ylabel("Number of Passengers")
plt.title("Survival of Men in Third Class")_____no_output_____# code block from reviewer, placed in here for future reference.
# first line is already achieved in survivedStr
# df['Survived'].replace({0:'Died', 1:'Survived'}, inplace=True)
ax = sns.factorplot(y='PassengerId', x='strSurvived', estimator=len, data=df, row='Sex', col='Pclass', kind='bar')
ax.set_xlabels('')
ax.set_ylabels('Number of passengers')_____no_output_____

#### Insights from Survival rates split by gender and class
From the above charts we can see a female passenger is more likely to have been saved in every class comparetively to their male counterparts. The subclass presented here with the highest odds of surviving (by no surprise) are the females of first class. The lowest odds can be found between males of second and third class, who did not fair so well.

This may be due to several causes, it could have been that women were prioritized for lifeboat positions, perhaps along with children, which of course would align to longstanding cultural ideals. To verify this suspiscion (and it's connection to my main premise) we will continue to explore the data visually, then attempt to draw concolusions statistically to affirm our hunches.

Firstly I'd like to examine how the female population presents as a stacked graph, thus showing the population as a whole and the comparisons in survival between them. _____no_output_____

fig, ax = plt.subplots(figsize=(20, 10))
# now to split the data for an intresting stacking experiment
wclass1 = (
    women.Survived[(women.Pclass == 1) & (women.Survived == 1)].count(),
    women.Survived[(women.Pclass == 1) & (women.Survived == 0)].count(),
)
wclass2 = (
    women.Survived[(women.Pclass == 2) & (women.Survived == 1)].count(),
    women.Survived[(women.Pclass == 2) & (women.Survived == 0)].count(),
)
wclass3 = (
    women.Survived[(women.Pclass == 3) & (women.Survived == 1)].count(),
    women.Survived[(women.Pclass == 3) & (women.Survived == 0)].count(),
)
# bin settings
N = 2
ind = np.arange(N)
width = .75
# stack, stack, stack em up
c3 = plt.bar(ind, wclass3, width, color=f3)
c2 = plt.bar(ind, wclass2, width, color=f2, bottom=wclass3)
c1 = plt.bar(ind, wclass1, width, color=f1, bottom=(np.array(wclass3)+np.array(wclass2)))
# add the fancy-shmancy labels so other 'hoomans' can read
plt.xticks(ind, ('Survived', 'Died'))
plt.ylabel("Number of Passengers")
plt.title("Survival of Women, Colored by Class")
plt.legend((c1[0], c2[0], c3[0]), ('First Class', 'Second Class', 'Third Class'))_____no_output_____

#### That's a very little bit of gold you have there.
I've picked gold for females of first class for this reason, it seems really as though they were treated like gold. We can see here that in the population of women first class females make an extremely small fraction of the deceased. This aligns pretty closely to the trends noted above.

Now it's time to look at the mens ratios in the same manner. _____no_output_____

fig, ax = plt.subplots(figsize=(20, 10))
# now to split the data for an intresting stacking experiment
mclass1 = (
    men.Survived[(men.Pclass == 1) & (men.Survived == 1)].count(),
    men.Survived[(men.Pclass == 1) & (men.Survived == 0)].count(),
)
mclass2 = (
    men.Survived[(men.Pclass == 2) & (men.Survived == 1)].count(),
    men.Survived[(men.Pclass == 2) & (men.Survived == 0)].count(),
)
mclass3 = (
    men.Survived[(men.Pclass == 3) & (men.Survived == 1)].count(),
    men.Survived[(men.Pclass == 3) & (men.Survived == 0)].count(),
)
# bin settings
N = 2
ind = np.arange(N)
width = .75
# stack, stack, stack em up
c3 = plt.bar(ind, mclass3, width, color=m3)
c2 = plt.bar(ind, mclass2, width, color=m2, bottom=mclass3)
c1 = plt.bar(ind, mclass1, width, color=m1, bottom=(np.array(mclass3)+np.array(mclass2)))
# add the fancy-shmancy labels so other 'hoomans' can read
plt.xticks(ind, ('Survived', 'Died'))
plt.ylabel("Number of Passengers")
plt.title("Survival of Men, Colored by Class")
plt.legend((c1[0], c2[0], c3[0]), ('First Class', 'Second Class', 'Third Class'))_____no_output_____

#### Well that's bleak
It seems that males, regardless of class were doomed...
Additionally we can also see here the only class with even a nearly even rate is first class.

But what happens if we stack them all up?_____no_output_____

fig, ax = plt.subplots(figsize=(20, 10))
# bin settings
N = 2
ind = np.arange(N)
width = .75
# stack, stack, stack em up
m3 = plt.bar(ind, mclass3, width, color=m3)
y = np.array(mclass3)
m2 = plt.bar(ind, mclass2, width, color=m2, bottom=y)
y += np.array(mclass2)
m1 = plt.bar(ind, mclass1, width, color=m1, bottom=y)
y += np.array(mclass1)
f3 = plt.bar(ind, wclass3, width, color=f3, bottom=y)
y += np.array(wclass3)
f2 = plt.bar(ind, wclass2, width, color=f2, bottom=y)
y += np.array(wclass2)
f1 = plt.bar(ind, wclass1, width, color=f1, bottom=y)
# add the fancy-shmancy labels so other 'hoomans' can read
plt.xticks(ind, ('Survived', 'Died'))
plt.ylabel("Number of Passengers")
plt.title("Survival of Passengers Colored by Gender and Class")
plt.legend(
    (f1[0], f2[0], f3[0], m1[0], m2[0], m3[0]),
    (
        'First Class Females', 'Second Class Females', 'Third Class Females',
        'First Class Males', 'Second Class Males', 'Third Class Males'
    )
)
plt.show()_____no_output_____

#### Interesting!
To start with something unexpected, I'd like to direct your attention to the brown block (second class males), what's interesting is that they are actually the smallest group of survivors! We can still clearly see women are the main survivors, no other group even compares to women of first class.

This could be a result of numerous factors, but I think the simplest solution - Chivalry - can take much of the credit for this one.

Time to look at the actual survival rates between the classes and genders. _____no_output_____

# creating a plot of chances of survival by class and gender!
data = (
    df.Survived[(df.gender == 1) & (df.Pclass == 1)].mean(),
    df.Survived[(df.gender == 1) & (df.Pclass == 2)].mean(),
    df.Survived[(df.gender == 1) & (df.Pclass == 3)].mean(),
    df.Survived[(df.gender == 0) & (df.Pclass == 1)].mean(),
    df.Survived[(df.gender == 0) & (df.Pclass == 2)].mean(),
    df.Survived[(df.gender == 0) & (df.Pclass == 3)].mean(),
)
labels = (
    "",
    "Women, First Class",
    "Women, Second Class",
    "Women, Third Class",
    "Men, First Class",
    "Men, Second Class",
    "Men, Third Class"
)
fig, ax = plt.subplots(figsize=(20, 10))
# bin settings
N = 6
ind = np.arange(N)
width = .75
# stack, stack, stack em up
plt.bar(ind, data, width)
# add the fancy-shmancy labels so other 'hoomans' can read
plt.ylabel("Survival Rate (0-1)")
plt.title("Survival of Passengers as a Proportion")
ax.set_xticklabels(labels, rotation=40, ha='right')
plt.show()_____no_output_____

#### Proportionally Safe
This paints a bleak picture for myself, a male, most likely second-class traveler, but _C'est la vie_.

## Statistical Analysis of Findings
To continue onto the numbers now; based on preliminary findings I can conclude that the female population as a whole would *not* be statistically strong enough to reject the null. Therefore I shall continue this analysis looking at the most fortunate group, females of first class; in an effort to find a single group with a statistically significant chance of survival. _____no_output_____

fem_mean = df.Survived[(df.gender == 1) & (df.Pclass == 1)].mean()
print(fem_mean)0.9680851063829787

Approximately 96% of females in first class survied the incident.

Compared to males in third class?_____no_output_____

male_mean = df.Survived[(df.gender == 0) & (df.Pclass == 3)].mean()
print(male_mean)0.13544668587896252

Well darn, ~13% are not good odds by any standard.

Let's take a look at the Z-Scores of our various subgroups of the population. _____no_output_____

fem_z = (fem_mean - df.Survived.mean()) / df.Survived.std(ddof=0)
print(fem_z)1.2013644144488933

Out of the whole population, there is a 15% chance of randomly selecting a group with the same mean as taken from the female passengers. While this margin of error will not allow me to reject the null. I'd like to compare the male group to this female subset - _____no_output_____

fem_z_toMale = (fem_mean - df.Survived[(df.gender == 0)].mean()) / df.Survived[(df.gender == 0)].std(ddof=0)
print(fem_z_toMale)1.9905621426418676

So close. A 1.99 Z-Score, rejection of the null does not make though. Still, it is evident females did have quite different odds than males on this foregone cruiseship of yore. _____no_output_____## Conclusion of main premise
From this we can conclude the condition of being female is significant, yet not significant to $\alpha = 0.01$
We are left with a 2% chance of random selection giving us this bias. I personally find this to still be significant as it passes at $\alpha = 0.05$. Unfortunately I did not set out to prove it at 5%. :(_____no_output_____## Limitations
This data set is fairly small compared to the actual number of passengers traveling on the ship. One can infer that the additional members may have allowed a statistical certainty here, but it would only be an inference, not certain by any means.

I must also note the survivor bias inherent in this data set. Since researchers were only able to survey survivors of the incident we may be lacking in information from those who perished in the incident. _____no_output_____### Resources Consulted
[Jupyter Notebook from Kaggle on Titanic Analysis](http://nbviewer.jupyter.org/github/agconti/kaggle-titanic/blob/master/Titanic.ipynb)

[Jupyter Notebook on Matplotlib](http://nbviewer.jupyter.org/github/jrjohansson/scientific-python-lectures/blob/master/Lecture-4-Matplotlib.ipynb)

[PyPlot Docs](http://matplotlib.org/api/pyplot_api.html#matplotlib.pyplot.bar)

[Pandas Docs](http://pandas.pydata.org/pandas-docs/stable/dsintro.html)_____no_output_____