# Notebook from geo-smart/ESS490-590-Spr21
Path: tutorials/Week8_neural_networks/neural_networks_PyTorch.ipynb

# Using PyTorch to build, train, and use neural networks: A short introduction_____no_output_____

import glob
import matplotlib.pyplot as plt
import numpy as np
import pandas as pd
import pickle
import torch
import torchvision
import torch.nn as nn
import torch.optim as optim
from torchvision.transforms import transforms
from torch.utils.data import DataLoader
from torch.utils.data import Dataset_____no_output_____

## Reading a dataset and converting into a format readable by PyTorch_____no_output_____We are going to do multi-class classification. We have a dataset of images of digits (0 to 9). Each image is made of 16 * 16 pixels in grey scale. The training file and the test file are made of 257 columns: the first column is the label, the next 256 columns are the corresponding value for each of the 16 * 16 = 256 pixels. There is one image per row._____no_output_____Let us start by reading the data._____no_output_____

df_train = pd.read_csv('zip.train', header=None, sep='\s+')
df_test = pd.read_csv('zip.test', header=None, sep='\s+')_____no_output_____print('There are {:d} images in the training set and {:d} images in the test set.'.format(len(df_train), len(df_test)))There are 7291 images in the training set and 2007 images in the test set.

We are going to divide the datasets into images and the corresponding labels._____no_output_____

train_labels = df_train.iloc[:, 0]
train_images = df_train.iloc[:, 1:]
test_labels = df_test.iloc[:, 0]
test_images = df_test.iloc[:, 1:]_____no_output_____print('The labels are: ', sorted(train_labels.unique().astype(int).tolist()))The labels are:  [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]

We are now going to convert the training set and the test set into a PyTorch dataset._____no_output_____

# Dataset is a class already defined in PyTorch. We are going to create a subclass of Dataset
class ZipDataset(Dataset):
    def __init__(self, images, labels=None, transforms=None):
        # Let us define the attributes of a ZipDataset
        self.X = images # that will be train_images or test_images
        self.y = labels # that will be train_labels or test_labels
        # When we create an object of class ZipDataset, we can specify the transformation to apply to the data
        self.transforms = transforms

    def __len__(self):
        # The length of the dataset is the number of images in the dataset
        return (len(self.X))

    def __getitem__(self, i):
        # How to access one pair of (image, label) from the dataset?
        # We access the ith row (which contains the 16*16 = 256 pixels of an image)
        data = self.X.iloc[i, :]
        # We tranform it into a numpy array and reshape it to an image format
        data = np.asarray(data).reshape(16, 16, 1)
        # We apply the required transformation to the image
        if self.transforms:
            data = self.transforms(data)
        # If there is a label associated to it, we return it together with the image
        if self.y is not None:
            return (data, self.y[i])
        else:
            return data_____no_output_____

Now let us create our training and test sets._____no_output_____

train_data = ZipDataset(train_images, train_labels, transforms.ToTensor())
test_data = ZipDataset(test_images, test_labels, transforms.ToTensor())_____no_output_____

Maybe we could have a look at one of the images._____no_output_____

index = 1048_____no_output_____data_point = train_data[index]
image = data_point[0]
label = data_point[1]_____no_output_____

Our image looks like this:_____no_output_____

image = image[0,:,:].numpy()
plt.imshow(image)
print('The corresponding label is ' + str(int(label)))The corresponding label is 7

Let us look at an alternative way of loading the dataset if each image is in a different file._____no_output_____

# We create another subclass of Dataset for the case where each image is in a different file
class CustomDataset(Dataset):
    def __init__(self, images_dir, labels_file=None, transforms=None):
        if labels_file is not None:
            filename = images_dir + '/' + labels_file # that will be train/label.pkl or test/label.pkl
            self.labels = pickle.load(open(filename, 'rb'))
        self.images_dir = images_dir # that will be train or test
        # When we create an object of class CustomDataset, we can specify the transformation to apply to the data
        self.transforms = transforms

    def __len__(self):
        # The length of the dataset is the number of images in the dataset
        return len(glob.glob1(self.images_dir, '*.pkl')) - 1

    def __getitem__(self, idx):
        # How to access one pair of (image, label) from the dataset?
        # We access the ith value of the label array and the ith file in images_dir
        label = self.labels[idx]
        image_path = self.images_dir + '/image_' + str(idx) + '.pkl'
        image = pickle.load(open(image_path, 'rb'))
        # We apply the required transformation to the image
        if self.transforms:
            image = self.transforms(image)
        # If there is a label associated to it, we return it together with the image
        if self.labels is not None:
            return (image, label)
        else:
            return image_____no_output_____

Now let us create our training and test sets._____no_output_____

train_data_alt = CustomDataset('train', 'label.pkl', transforms.ToTensor())
test_data_alt = CustomDataset('test', 'label.pkl', transforms.ToTensor())_____no_output_____

We can now have a look at one of the images._____no_output_____

index = 1048_____no_output_____data_point = train_data_alt[index]
image = data_point[0]
label = data_point[1]_____no_output_____

Our image looks like this:_____no_output_____

image = image[0,:,:].numpy()
plt.imshow(image)
print('The corresponding label is ' + str(int(label)))The corresponding label is 7

The simplest neural network is a logistic regression

$y = step( \sum_{i=1}^3  (w_i x_i + b_i))$,

where $y$ is the output, $w_i$, iare the weights, $b_i$ is the biases in each neuron. It is called a Threshold Logic Unit TLU because it outputs a linear combination of the inputs, and if the result exceeds a threshold, it outputs the positive class.

![Single Neuron](TLU.png)

A perceptron is a single kayer of TLUs, which each TLU connected to all the inputs.

## Logistic regression: It is just a one layer neural network
_____no_output_____

# We create a subclass of neural networks: This one will just have one layer
class NN1(nn.Module):
    """
    Logistic regression with more than 2 classes
    """
    def __init__(self, size_img, num_classes):
        # Here we define all the functions that we will use during the forward part (data -> prediction)
        super(NN1, self).__init__()
        self.flatten = nn.Flatten() # go from a 16*16 tensor to a 256*1 tensor
        self.size_img = size_img # number of pixels in an image
        self.layer1 = nn.Linear(size_img, num_classes) # y = wx + b with w = 10*256 (10 digits * 256 pixels)

    def forward(self, x):
        # Here we explain in which order to use the functions defined above
        x = self.flatten(x)
        logits = self.layer1(x)
        return logits_____no_output_____

We now create a model of the class NN1._____no_output_____

model = NN1(16*16, 10)_____no_output_____

We also need to define the loss. Remember than with two classes (binary classification), we had defined:

$\log \frac{P(Y = 1)}{P(Y = 0)} = w x + b = z$.

As $P(Y = 0) + P(Y = 1) = 1$, we get $P(Y = 1) = \frac{1}{1 + e^{-z}} = \frac{1}{1 + e^{- w x - b}}$.

We can then write the probability that $Y$ takes the value $y$ (with $y$ equal to 0 or 1):

$P(Y = y) = \left(\frac{1}{1 + e^{-z}}\right)^y \left(1 - \frac{1}{1 + e^{-z}}\right)^{1 - y}$

Finally, we defined the loss:

$\mathcal{L} = - \log P(Y = y) = - (y \log \hat{y} + (1 - y) \log (1 - \hat{y})$ with $\hat{y} = \frac{1}{1 + e^{-z}}$.

We wanted to find $w$ and $b$ such that $P(Y = y)$ is maximum, which is equivalent to minimizing the loss $\mathcal{L}$._____no_output_____Now with $K$ classes, we write similarly:

$\log \frac{P(Y = k)}{P(Y = K)} = (w_k - w_K) x + (b_k - b_K)$, which gives us:

$P(Y = k) = \frac{\exp(w_k x + b_k)}{\sum_{j = 1}^K \exp(w_j x + b_j)}$

This is called the _softmax_ function. Once we have found the values of the $w_k$ and $b_k$ for $k = 1 , \cdots , K$, we can compute the values of the $P(Y = k)$. We then look for which value of $k$ $P(Y = k)$ is maximal and we classify this sample as class $k$.

Similarly, we define the loss with $K$ classes:

$\mathcal{L} = - \sum_{k = 1}^K y_k \log \frac{\exp(w_k x + b_k)}{\sum_{j = 1}^K \exp(w_j x + b_j)}$. We have $y_k = 1$ if the true label associated with the sample $x$ is $k$, otherwise $y_k = 0$.

This is called the _cross-entropy loss_._____no_output_____

criterion = nn.CrossEntropyLoss()_____no_output_____

We also need to define the gradient descent method, that we are going to use. We need to specify the learning rate._____no_output_____

optimizer = optim.SGD(model.parameters(), lr=0.001)_____no_output_____

With stochastic gradient descent, we take the gradient and update the parameters of the model after each data sample. We need to define dataloaders that allow us to iterate over the data._____no_output_____

trainloader = DataLoader(train_data)
testloader = DataLoader(test_data)_____no_output_____

Let us now define how to train the model._____no_output_____

def train(model, n_epochs, trainloader, testloader=None):

    # Define loss and optimization method
    criterion = nn.CrossEntropyLoss()
    optimizer = optim.SGD(model.parameters(), lr=0.001)

    # Save loss and error for plotting
    loss_time = np.zeros(n_epochs)
    error_time = np.zeros(n_epochs)

    # Loop on number of epochs
    for epoch in range(n_epochs):
        # Initialize the loss
        running_loss = 0
        # Loop on samples in train set
        for data in trainloader:
            # Get the sample and modify the format for PyTorch
            inputs, labels = data[0], data[1]
            inputs = inputs.float()
            labels = labels.long()
            # Set the parameter gradients to zero
            optimizer.zero_grad()
            outputs = model(inputs)
            loss = criterion(outputs, labels)
            # Propagate the loss backward
            loss.backward()
            # Update the gradients
            optimizer.step()
            # Add the value of the loss for this sample
            running_loss += loss.item()
        # Save loss at the end of each epoch
        loss_time[epoch] = running_loss/len(trainloader)

        # After each epoch, evaluate the performance on the test set
        if testloader is not None:
            correct = 0
            total = 0
            # We evaluate the model, so we do not need the gradient
            with torch.no_grad():
                # Loop on samples in test set
                for data in testloader:
                    # Get the sample and modify the format for PyTorch
                    inputs, labels = data[0], data[1]
                    inputs = inputs.float()
                    labels = labels.long()
                    # Use model for sample in the test set
                    outputs = model(inputs)
                    # Compare predicted label and true label
                    _, predicted = torch.max(outputs.data, 1)
                    total += labels.size(0)
                    correct += (predicted == labels).sum().item()
            # Save error at the end of each epochs
            error_time[epoch] = 100 * correct / total

        # Print intermediate results on screen
        if testloader is not None:
            print('[Epoch %d] loss: %.3f - error: %.3f' %
              (epoch + 1, running_loss/len(trainloader), 100 * correct / total))
        else:
            print('[Epoch %d] loss: %.3f' %
              (epoch + 1, running_loss/len(trainloader)))

    # Save history of loss and test error
    if testloader is not None:
        return (loss_time, error_time)
    else:
        return (loss_time)_____no_output_____(loss, error) = train(model, 10, trainloader, testloader)[Epoch 1] loss: 0.592 - error: 88.142
[Epoch 2] loss: 0.292 - error: 89.586
[Epoch 3] loss: 0.246 - error: 90.284
[Epoch 4] loss: 0.222 - error: 90.433
[Epoch 5] loss: 0.207 - error: 90.732
[Epoch 6] loss: 0.196 - error: 91.031
[Epoch 7] loss: 0.188 - error: 91.281
[Epoch 8] loss: 0.181 - error: 91.480
[Epoch 9] loss: 0.175 - error: 91.679
[Epoch 10] loss: 0.170 - error: 91.629

Let us now plot the evolution of the loss and the percentage of correct predictions with time:_____no_output_____

fig, ax1 = plt.subplots()

color = 'tab:red'
ax1.set_xlabel('Epoch')
ax1.set_ylabel('Loss', color=color)
ax1.plot(np.arange(1, 11), loss, color=color)
ax1.tick_params(axis='y', labelcolor=color)

ax2 = ax1.twinx()

color = 'tab:blue'
ax2.set_ylabel('Correct predictions', color=color)
ax2.plot(np.arange(1, 11), error, color=color)
ax2.tick_params(axis='y', labelcolor=color)

fig.tight_layout()
plt.show()_____no_output_____

With stochastic gradient descent, we can also take the gradient over a subset of the sample (batch) and update the model after each batch. We can define the number of samples of the batch when we create the dataloaders_____no_output_____

trainloader = DataLoader(train_data, batch_size=128, shuffle=True)
testloader = DataLoader(test_data, batch_size=128, shuffle=True)_____no_output_____

Let us now train the model with batches. We recreate the model (otherwise, the training starts where we have ended in the previous cell)._____no_output_____

model = NN1(16*16, 10)_____no_output_____(loss, error) = train(model, 20, trainloader, testloader)[Epoch 1] loss: 2.255 - error: 28.849
[Epoch 2] loss: 2.036 - error: 37.419
[Epoch 3] loss: 1.861 - error: 44.694
[Epoch 4] loss: 1.718 - error: 51.221
[Epoch 5] loss: 1.599 - error: 57.399
[Epoch 6] loss: 1.496 - error: 62.033
[Epoch 7] loss: 1.407 - error: 65.919
[Epoch 8] loss: 1.329 - error: 68.211
[Epoch 9] loss: 1.260 - error: 70.404
[Epoch 10] loss: 1.198 - error: 72.347
[Epoch 11] loss: 1.144 - error: 73.941
[Epoch 12] loss: 1.094 - error: 75.286
[Epoch 13] loss: 1.049 - error: 76.283
[Epoch 14] loss: 1.009 - error: 77.180
[Epoch 15] loss: 0.972 - error: 78.127
[Epoch 16] loss: 0.939 - error: 78.874
[Epoch 17] loss: 0.908 - error: 79.671
[Epoch 18] loss: 0.879 - error: 80.169
[Epoch 19] loss: 0.853 - error: 80.568
[Epoch 20] loss: 0.829 - error: 80.917
fig, ax1 = plt.subplots()

color = 'tab:red'
ax1.set_xlabel('Epoch')
ax1.set_ylabel('Loss', color=color)
ax1.plot(np.arange(1, 21), loss, color=color)
ax1.tick_params(axis='y', labelcolor=color)

ax2 = ax1.twinx()

color = 'tab:blue'
ax2.set_ylabel('Correct predictions', color=color)
ax2.plot(np.arange(1, 21), error, color=color)
ax2.tick_params(axis='y', labelcolor=color)

fig.tight_layout()
plt.show()_____no_output_____

We could also use the dataset in the alternative format._____no_output_____

trainloader_alt = DataLoader(train_data_alt)
testloader_alt = DataLoader(test_data_alt)_____no_output_____model = NN1(16*16, 10)_____no_output_____(loss, error) = train(model, 10, trainloader_alt, testloader_alt)[Epoch 1] loss: 0.589 - error: 87.992
[Epoch 2] loss: 0.292 - error: 89.537
[Epoch 3] loss: 0.246 - error: 90.135
[Epoch 4] loss: 0.223 - error: 90.533
[Epoch 5] loss: 0.208 - error: 90.782
[Epoch 6] loss: 0.197 - error: 91.081
[Epoch 7] loss: 0.188 - error: 91.380
[Epoch 8] loss: 0.181 - error: 91.480
[Epoch 9] loss: 0.176 - error: 91.380
[Epoch 10] loss: 0.171 - error: 91.330
fig, ax1 = plt.subplots()

color = 'tab:red'
ax1.set_xlabel('Epoch')
ax1.set_ylabel('Loss', color=color)
ax1.plot(np.arange(1, 11), loss, color=color)
ax1.tick_params(axis='y', labelcolor=color)

ax2 = ax1.twinx()

color = 'tab:blue'
ax2.set_ylabel('Correct predictions', color=color)
ax2.plot(np.arange(1, 11), error, color=color)
ax2.tick_params(axis='y', labelcolor=color)

fig.tight_layout()
plt.show()_____no_output_____


# Notebook from cnegrelli/DSND_Term2
Path: lessons/NLP and ML pipelines/cleaning_practice.ipynb

# Cleaning Quiz: Udacity's Course Catalog
It's your turn! Udacity's [course catalog page](https://www.udacity.com/courses/all) has changed since the last video was filmed. One notable change is the introduction of  _schools_.

In this activity, you're going to perform similar actions with BeautifulSoup to extract the following information from each course listing on the page:
1. The course name - e.g. "Data Analyst"
2. The school the course belongs to - e.g. "School of Data Science"

**Note: All solution notebooks can be found by clicking on the Jupyter icon on the top left of this workspace.**_____no_output_____### Step 1: Get text from Udacity's course catalog web page
You can use the `requests` library to do this.

Outputting all the javascript, CSS, and text may overload the space available to load this notebook, so we omit a print statement here._____no_output_____

# import statements
import requests
from bs4 import BeautifulSoup_____no_output_____# fetch web page
r = requests.get("https://www.udacity.com/courses/all")_____no_output_____

### Step 2: Use BeautifulSoup to remove HTML tags
Use `"lxml"` rather than `"html5lib"`.

Again, printing this entire result may overload the space available to load this notebook, so we omit a print statement here._____no_output_____

soup = BeautifulSoup(r.text, 'lxml')_____no_output_____

### Step 3: Find all course summaries
Use the BeautifulSoup's `find_all` method to select based on tag type and class name. Just like in the video, you can right click on the item, and click "Inspect" to view its html on a web page._____no_output_____

# Find all course summaries
summaries = soup.find_all('div', class_="course-summary-card row row-gap-medium catalog-card nanodegree-card ng-star-inserted")
print('Number of Courses:', len(summaries))Number of Courses: 43

### Step 4: Inspect the first summary to find selectors for the course name and school
Tip: `.prettify()` is a super helpful method BeautifulSoup provides to output html in a nicely indented form! Make sure to use `print()` to ensure whitespace is displayed properly._____no_output_____

# print the first summary in summaries
print(summaries[0].prettify())

 <ir-catalog-card _ngcontent-sc204="" _nghost-sc207="">

       New

        School of Business

        Product Manager

         Skills Covered

          Product Strategy,

          Product Design,

          Product Development,

          Design Sprint,

          Product Launch

         beginner

       Program Details

        beginner

      Envision and execute the development of industry-defining products, and learn how to successfully bring them to market.

      Learn More

 </ir-catalog-card>

Look for selectors that contain the courses title and school name text you want to extract. Then, use the `select_one` method on the summary object to pull out the html with those selectors. Afterwards, don't forget to do some extra cleaning to isolate the names (get rid of unnecessary html), as you saw in the last video._____no_output_____

# Extract course title
summaries[0].select_one("h3 a").get_text().strip()_____no_output_____# Extract school
summaries[0].select_one("h4").get_text().strip()_____no_output_____

### Step 5: Collect names and schools of ALL course listings
Reuse your code from the previous step, but now in a loop to extract the name and school from every course summary in `summaries`!_____no_output_____

courses = []
for summary in summaries:
    # append name and school of each summary to courses list
    name = summary.select_one("h3 a").get_text().strip()
    school = summary.select_one("h4").get_text().strip()
    courses.append((name,school))_____no_output_____# display results
print(len(courses), "course summaries found. Sample:")
courses[:20]43 course summaries found. Sample:


# Notebook from MDSADABWASIM/ML-Project
Path: Code/Day41-PCA.ipynb

# Principal Component Analysis_____no_output_____PCA is a dimensionality reduction technique; it lets you distill multi-dimensional data down to fewer dimensions, selecting new dimensions that preserve variance in the data as best it can.

We're not talking about Star Trek stuff here; let's make it real - a black & white image for example, contains three dimensions of data: X position, Y position, and brightness at each point. Distilling that down to two dimensions can be useful for things like image compression and facial recognition, because it distills out the information that contributes most to the variance in the data set.

Let's do this with a simpler example: the Iris data set that comes with scikit-learn. It's just a small collection of data that has four dimensions of data for three different kinds of Iris flowers: The length and width of both the petals and sepals of many individual flowers from each species. Let's load it up and have a look:_____no_output_____

from sklearn.datasets import load_iris
from sklearn.decomposition import PCA
import pylab as pl
from itertools import cycle

iris = load_iris()

numSamples, numFeatures = iris.data.shape
print(numSamples)
print(numFeatures)
print(list(iris.target_names))150
4
['setosa', 'versicolor', 'virginica']

So, this tells us our data set has 150 samples (individual flowers) in it. It has 4 dimensions - called features here, and three distinct Iris species that each flower is classified into.

While we can visualize 2 or even 3 dimensions of data pretty easily, visualizing 4D data isn't something our brains can do. So let's distill this down to 2 dimensions, and see how well it works:_____no_output_____

X = iris.data
pca = PCA(n_components=2, whiten=True).fit(X)
X_pca = pca.transform(X)_____no_output_____

What we have done is distill our 4D data set down to 2D, by projecting it down to two orthogonal 4D vectors that make up the basis of our new 2D projection. We can see what those 4D vectors are, although it's not something you can really wrap your head around:_____no_output_____

print(pca.components_)[[ 0.36138659 -0.08452251  0.85667061  0.3582892 ]
 [ 0.65658877  0.73016143 -0.17337266 -0.07548102]]

Let's see how much information we've managed to preserve:_____no_output_____

print(pca.explained_variance_ratio_)
print(sum(pca.explained_variance_ratio_))[0.92461872 0.05306648]
0.977685206318795

That's pretty cool. Although we have thrown away two of our four dimensions, PCA has chosen the remaining two dimensions well enough that we've captured 92% of the variance in our data in a single dimension alone! The second dimension just gives us an additional 5%; altogether we've only really lost less than 3% of the variance in our data by projecting it down to two dimensions.

As promised, now that we have a 2D representation of our data, we can plot it:_____no_output_____

%matplotlib inline
from pylab import *

colors = cycle('rgb')
target_ids = range(len(iris.target_names))
pl.figure()
for i, c, label in zip(target_ids, colors, iris.target_names):
    pl.scatter(X_pca[iris.target == i, 0], X_pca[iris.target == i, 1],
        c=c, label=label)
pl.legend()
pl.show()
    _____no_output_____

You can see the three different types of Iris are still clustered pretty well. If you think about it, this probably works well because the overall size of an individual flower probably makes both the petal and sepal sizes increase by a similar amount. Although the actual numbers on this graph have no intuitive meaning, what we're probably seeing is measure of the ratio of width to height for petals and sepals - and PCA distilled our data down to that on its own._____no_output_____Our results suggest we could actually distill this data down to a single dimension and still preserve most of its variance. Try it! Do a PCA down to one component, and measure the results._____no_output_____


# Notebook from jainish4ai/pandas_exercises
Path: 08_Creating_Series_and_DataFrames/Pokemon/Exercises.ipynb

# Pokemon_____no_output_____### Introduction:

This time you will create the data.

### Step 1. Import the necessary libraries_____no_output_____

import pandas as pd_____no_output_____

### Step 2. Create a data dictionary that looks like the DataFrame below_____no_output_____

raw_data = {"name": ['Bulbasaur', 'Charmander','Squirtle','Caterpie'],
            "evolution": ['Ivysaur','Charmeleon','Wartortle','Metapod'],
            "type": ['grass', 'fire', 'water', 'bug'],
            "hp": [45, 39, 44, 45],
            "pokedex": ['yes', 'no','yes','no']
            }_____no_output_____

### Step 3. Assign it to a variable called pokemon_____no_output_____

pokemon = pd.DataFrame.from_dict(raw_data)
pokemon.head()_____no_output_____

### Step 4. Ops...it seems the DataFrame columns are in alphabetical order. Place  the order of the columns as name, type, hp, evolution, pokedex_____no_output_____### Step 5. Add another column called place, and insert what you have in mind._____no_output_____

pokemon['place'] = pd.Series(['A', 'V', 'C', 'D'])
pokemon_____no_output_____

### Step 6. Present the type of each column_____no_output_____

pokemon.dtypes_____no_output_____

### BONUS: Create your own question and answer it._____no_output_____


# Notebook from Shivangi160/DNA-Sequencing-with-Machine-Learning
Path: DNA_Sequencing.ipynb

In this Project, I will apply a classification model with Machine Learning, that can predict a gene’s function based on the DNA sequencing of the coding sequence alone._____no_output_____##Importing Libraries_____no_output_____

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt_____no_output_____

##Reading Data_____no_output_____

human_data = pd.read_table('/content/drive/My Drive/DNA Sequencing with Machine Learning/human_data.txt')
human_data.head()_____no_output_____chimp_data = pd.read_table('/content/drive/My Drive/DNA Sequencing with Machine Learning/chimp_data.txt')
chimp_data.head()_____no_output_____dog_data = pd.read_table('/content/drive/My Drive/DNA Sequencing with Machine Learning/dog_data.txt')
dog_data.head()_____no_output_____

**Treating DNA sequence as a “language”, otherwise known as k-mer counting**

A challenge that remains is that none of these above methods results in vectors of uniform length, and that is a requirement for feeding data to a classification or regression algorithm.

So with the above methods you have to resort to things like truncating sequences or padding with “n” or “0” to get vectors of uniform length.

DNA and protein sequences can be viewed metaphorically as the language of life. The language encodes instructions as well as function for the molecules that are found in all life forms.

Here I am using hexamer “words” but that is arbitrary and word length can be tuned to suit the particular situation. The word length and amount of overlap need to be determined empirically for any given application.

In genomics, we refer to these types of manipulations as “k-mer counting”, or counting the occurrences of each possible k-mer sequence. There are specialized tools for this, but the Python natural language processing tools make it super easy._____no_output_____##Let’s define a function to collect all possible overlapping k-mers of a specified length from any sequence string._____no_output_____

# function to convert sequence strings into k-mer words, default size = 6 (hexamer words)
def getKmers(sequence, size=6):
    return [sequence[x:x+size].lower() for x in range(len(sequence) - size + 1)]_____no_output_____

Now we can convert our training data sequences into short overlapping k-mers of legth 6. Lets do that for each species of data we have using our getKmers function._____no_output_____

human_data['words'] = human_data.apply(lambda x: getKmers(x['sequence']), axis=1)
human_data = human_data.drop('sequence', axis=1)
chimp_data['words'] = chimp_data.apply(lambda x: getKmers(x['sequence']), axis=1)
chimp_data = chimp_data.drop('sequence', axis=1)
dog_data['words'] = dog_data.apply(lambda x: getKmers(x['sequence']), axis=1)
dog_data = dog_data.drop('sequence', axis=1)_____no_output_____

Now, our coding sequence data is changed to lowercase, split up into all possible k-mer words of length 6 and ready for the next step. Let’s take a look._____no_output_____

human_data.head()_____no_output_____

Since we are going to use scikit-learn natural language processing tools to do the k-mer counting, we need to now convert the lists of k-mers for each gene into string sentences of words that the count vectorizer can use.

We can also make a y variable to hold the class labels. Let’s do that now._____no_output_____

human_texts = list(human_data['words'])
for item in range(len(human_texts)):
    human_texts[item] = ' '.join(human_texts[item])
y_data = human_data.iloc[:, 0].values
print(human_texts[2])atgtgt tgtgtg gtgtgg tgtggc gtggca tggcat ggcatt gcattt catttg atttgg tttggg ttgggc tgggcg gggcgc ggcgct gcgctg cgctgt gctgtt ctgttt tgtttg gtttgg tttggc ttggca tggcag ggcagt gcagtg cagtga agtgat gtgatg tgatga gatgat atgatt tgattg gattgc attgcc ttgcct tgcctt gccttt cctttc ctttct tttctg ttctgt tctgtt ctgttc tgttca gttcag ttcagt tcagtg cagtgt agtgtc gtgtct tgtctg gtctga tctgag ctgagt tgagtg gagtgc agtgct gtgcta tgctat gctatg ctatga tatgaa atgaag tgaaga gaagat aagatt agattg gattgc attgca ttgcac tgcaca gcacac cacaca acacag cacaga acagag cagagg agaggt gaggtc aggtcc ggtcca gtccag tccaga ccagat cagatg agatgc gatgca atgcat tgcatt gcattc cattcc attccg ttccgt tccgtt ccgttt cgtttt gttttg ttttga tttgag ttgaga tgagaa gagaat agaatg gaatgt aatgtc atgtca tgtcaa gtcaat tcaatg caatgg aatgga atggat tggata ggatac gataca atacac tacacc acacca caccaa accaac ccaact caactg aactgc actgct ctgctg tgctgc gctgct ctgctt tgcttt gctttg ctttgg tttgga ttggat tggatt ggattt gatttc atttca tttcac ttcacc tcaccg caccgg accggt ccggtt cggttg ggttgg gttggc ttggcg tggcgg ggcggt gcggta cggtag ggtagt gtagtt tagttg agttga gttgac ttgacc tgaccc gacccg acccgc cccgct ccgctg cgctgt gctgtt ctgttt tgtttg gtttgg tttgga ttggaa tggaat ggaatg gaatgc aatgca atgcag tgcagc gcagcc cagcca agccaa gccaat ccaatt caattc aattcg attcga ttcgag tcgagt cgagtg gagtga agtgaa gtgaag tgaaga gaagaa aagaaa agaaat gaaata aaatat aatatc atatcc tatccg atccgt tccgta ccgtat cgtatt gtattt tatttg atttgt tttgtg ttgtgg tgtggc gtggct tggctc ggctct gctctg ctctgt tctgtt ctgtta tgttac gttaca ttacaa tacaat acaatg caatgg aatggt atggtg tggtga ggtgaa gtgaaa tgaaat gaaatc aaatct aatcta atctac tctaca ctacaa tacaac acaacc caacca aaccat accata ccataa cataag ataaga taagaa aagaag agaaga gaagat aagatg agatgc gatgca atgcaa tgcaac gcaaca caacag aacagc acagca cagcat agcatt gcattt catttt attttg ttttga tttgaa ttgaat tgaatt gaattt aatttg atttga tttgaa ttgaat tgaata gaatac aatacc atacca taccag accaga ccagac cagacc agacca gaccaa accaaa ccaaag caaagt aaagtg aagtgg agtgga gtggat tggatg ggatgg gatggt atggtg tggtga ggtgag gtgaga tgagat gagata agataa gataat ataatc taatcc aatcct atcctt tccttc ccttca cttcat ttcatc tcatct catctt atcttt tcttta ctttat tttatg ttatga tatgac atgaca tgacaa gacaaa acaaag caaagg aaagga aaggag aggagg ggagga gaggaa aggaat ggaatt gaattg aattga attgag ttgagc tgagca gagcaa agcaaa gcaaac caaaca aaacaa aacaat acaatt caattt aatttg atttgt tttgta ttgtat tgtatg gtatgt tatgtt atgttg tgttgg gttgga ttggat tggatg ggatgg gatggt atggtg tggtgt ggtgtg gtgtgt tgtgtt gtgttt tgtttg gtttgc tttgca ttgcat tgcatt gcattt catttg atttgt tttgtt ttgttt tgtttt gtttta ttttac tttact ttactg tactgg actgga ctggat tggata ggatac gatact atactg tactgc actgcc ctgcca tgccaa gccaat ccaata caataa aataag ataaga taagaa aagaaa agaaag gaaagt aaagtg aagtgt agtgtt gtgttc tgttcc gttcct ttcctg tcctgg cctggg ctgggt tgggta gggtag ggtaga gtagag tagaga agagat gagata agatac gataca atacat tacata acatat catatg atatgg tatgga atggag tggagt ggagtc gagtca agtcag gtcaga tcagac cagacc agacct gacctt accttt cctttg ctttgt tttgtt ttgttt tgttta gtttaa tttaaa ttaaag taaagc aaagca aagcaa agcaat gcaatg caatga aatgac atgaca tgacag gacaga acagaa cagaag agaaga gaagat aagatg agatgg gatgga atggat tggatt ggattt gatttt attttt tttttg ttttgg tttggc ttggct tggctg ggctgt gctgta ctgtat tgtatg gtatgt tatgtt atgttc tgttca gttcag ttcaga tcagaa cagaag agaagc gaagct aagcta agctaa gctaaa ctaaag taaagg aaaggt aaggtc aggtct ggtctt gtcttg tcttgt cttgtt ttgtta tgttac gttaca ttacat tacatt acattg cattga attgaa ttgaag tgaagc gaagca aagcac agcact gcactc cactcc actccg ctccgc tccgcg ccgcga cgcgac gcgact cgactc gactcc actccc ctccct tccctt cccttt cctttt cttttt ttttta ttttaa tttaaa ttaaaa taaaag aaaagt aaagtg aagtgg agtgga gtggag tggagc ggagcc gagcct agcctt gccttt cctttt cttttc ttttct tttctt ttcttc tcttcc cttcct ttcctg tcctgg cctgga ctggac tggaca ggacac gacact acacta cactat actatg ctatga tatgaa atgaag tgaagt gaagtt aagttt agtttt gttttg ttttgg tttgga ttggat tggatt ggattt gattta atttaa tttaaa ttaaag taaagc aaagcc aagcca agccaa gccaaa ccaaat caaatg aaatgg aatggc atggca tggcaa ggcaaa gcaaag caaagt aaagtt aagttg agttgc gttgca ttgcat tgcatc gcatcc catccg atccgt tccgtg ccgtgg cgtgga gtggaa tggaaa ggaaat gaaatg aaatgg aatggt atggtt tggtta ggttaa gttaaa ttaaat taaata aaatat aatatc atatca tatcat atcatc tcatca catcac atcact tcactg cactgt actgtc ctgtcg tgtcgg gtcggg tcggga cgggat gggatg ggatgt gatgta atgtac tgtacc gtaccc tacccc acccct cccctg ccctgc cctgca ctgcac tgcacg gcacgc cacgcc acgccc cgccct gccctc ccctct cctcta ctctat tctatg ctatga tatgac atgaca tgacaa gacaat acaatg caatgt aatgtg atgtgg tgtgga gtggag tggaga ggagaa gagaaa agaaac gaaact aaactc aactct actctt ctcttt tctttc ctttcc tttcca ttccag tccagg ccaggt caggtt aggttt ggtttt gttttg ttttga tttgag ttgaga tgagat gagata agatag gataga atagaa tagaaa agaaac gaaact aaactg aactgt actgtg ctgtga tgtgaa gtgaag tgaaga gaagaa aagaac agaaca gaacaa aacaac acaacc caacct aacctc acctca cctcag ctcagg tcagga caggat aggatc ggatcc gatcct atcctt tccttt cctttt cttttt ttttta ttttaa tttaat ttaata taataa aataat ataatg taatgc aatgct atgctg tgctgt gctgta ctgtaa tgtaaa gtaaag taaaga aaagaa aagaaa agaaac gaaacg aaacgt aacgtt acgttt cgtttg gtttga tttgat ttgatg tgatga gatgac atgaca tgacag gacaga acagac cagaca agacag gacaga acagaa cagaag agaagg gaagga aaggat aggatt ggattg gattgg attggc ttggct tggctg ggctgc gctgcc ctgcct tgcctt gccttt cctttt ctttta ttttat tttatc ttatca tatcag atcagg tcaggg cagggg aggggg gggggc ggggct gggctt ggcttg gcttgg cttgga ttggac tggact ggactc gactcc actcca ctccag tccagc ccagct cagctt agcttg gcttgg cttggt ttggtt tggttg ggttgc gttgct ttgctg tgctgc gctgcc ctgcca tgccac gccact ccactc cactct actctg ctctgt tctgtt ctgttg tgttga gttgaa ttgaag tgaagc gaagca aagcag agcagc gcagct cagctg agctga gctgaa ctgaaa tgaaag gaaaga aaagaa aagaag agaagc gaagcc aagccc agccca gcccaa cccaag ccaagt caagta aagtac agtaca gtacag tacagt acagta cagtat agtatc gtatcc tatcct atcctc tcctct cctctc ctctcc tctcca ctccag tccaga ccagac cagaca agacat gacatt acattt catttg atttgc tttgca ttgcaa tgcaat gcaatt caattg aattgg attggc ttggca tggcat ggcatg gcatgg catgga atggaa tggaag ggaaga gaagac aagaca agacag gacagc acagcc cagccc agcccc gccccg ccccga cccgat ccgatt cgattt gattta atttac tttact ttactg tactgg actggc ctggct tggctg ggctgc gctgct ctgcta tgctag gctaga ctagaa tagaaa agaaag gaaagg aaaggt aaggtg aggtgg ggtggc gtggca tggcag ggcaga gcagat cagatc agatca gatcat atcata tcatat catatt atattg tattgg attgga ttggaa tggaag ggaagt gaagtg aagtga agtgaa gtgaac tgaaca gaacat aacatt acatta cattat attatg ttatga tatgaa atgaag tgaagt gaagtc aagtcc agtcct gtcctt tccttt cctttt cttttt ttttta ttttaa tttaac ttaact taactc aactct actctg ctctga tctgag ctgagg tgagga gaggaa aggaag ggaagg gaaggc aaggca aggcat ggcatt gcattc cattca attcag ttcagg tcaggc caggct aggctc ggctct gctctg ctctgg tctgga ctggat tggatg ggatga gatgaa atgaag tgaagt gaagtc aagtca agtcat gtcata tcatat catatt atattt tatttt attttc ttttcc tttcct ttcctt tccttg ccttgg cttgga ttggaa tggaaa ggaaac gaaact aaactt aactta acttat cttatg ttatga tatgac atgaca tgacat gacatt acatta cattac attaca ttacaa tacaac acaaca caacag aacagt acagtt cagttc agttcg gttcgt ttcgtg tcgtgc cgtgct gtgctt tgcttc gcttca cttcag ttcagt tcagta cagtag agtagg gtaggt taggta aggtat ggtatg gtatgt tatgta atgtat tgtatt gtattt tattta atttaa tttaat ttaatt taattt aatttc atttcc tttcca ttccaa tccaag ccaagt caagta aagtat agtata gtatat tatatt atattc tattcg attcgg ttcgga tcggaa cggaag ggaaga gaagaa aagaac agaaca gaacac aacaca acacag cacaga acagat cagata agatag gatagc atagcg tagcgt agcgtg gcgtgg cgtggt gtggtg tggtga ggtgat gtgatc tgatct gatctt atcttc tcttct cttctc ttctct tctctg ctctgg tctgga ctggag tggaga ggagaa gagaag agaagg gaagga aaggat aggatc ggatca gatcag atcaga tcagat cagatg agatga gatgaa atgaac tgaact gaactt aactta acttac cttacg ttacgc tacgca acgcag cgcagg gcaggg cagggt agggtt gggtta ggttac gttaca ttacat tacata acatat catata atatat tatatt atattt tatttt attttc ttttca tttcac ttcaca tcacaa cacaag acaagg caaggc aaggct aggctc ggctcc gctcct ctcctt tccttc ccttct cttctc ttctcc tctcct ctcctg tcctga cctgaa ctgaaa tgaaaa gaaaaa aaaaag aaaagc aaagcc aagccg agccga gccgag ccgagg cgagga gaggag aggagg ggagga gaggag aggaga ggagag gagagt agagtg gagtga agtgag gtgaga tgagag gagagg agaggc gaggct aggctt ggcttc gcttct cttctg ttctga tctgag ctgagg tgaggg gaggga agggaa gggaac ggaact gaactc aactct actcta ctctat tctatt ctattt tatttg atttgt tttgtt ttgttt tgtttg gtttga tttgat ttgatg tgatgt gatgtt atgttc tgttct gttctc ttctcc tctccg ctccgc tccgcg ccgcgc cgcgca gcgcag cgcaga gcagat cagatc agatcg gatcga atcgaa tcgaac cgaact gaacta aactac actact ctactg tactgc actgct ctgctg tgctgc gctgcc ctgccc tgccca gcccat cccatg ccatgg catggt atggtc tggtct ggtctt gtcttg tcttga cttgaa ttgaac tgaact gaactg aactga actgag ctgaga tgagag gagagt agagtc gagtcc agtccc gtccca tcccat cccatt ccattt catttc atttct tttcta ttctag tctaga ctagat tagatc agatca gatcat atcatc tcatcg catcga atcgat tcgatt cgattt gatttt attttc ttttct tttctt ttcttc tcttcc cttcct ttccta tcctat cctatt ctatta tattac attact ttactt tacttg acttgt cttgtc ttgtct tgtctc gtctct tctctg ctctgc tctgcc ctgcca tgccac gccacc ccacca caccag accaga ccagaa cagaaa agaaat gaaatg aaatga aatgag atgaga tgagaa gagaat agaatt gaattc aattcc attcca ttccaa tccaaa ccaaag caaaga aaagaa aagaat agaatg gaatgg aatggg atggga tgggat gggata ggatag gataga atagaa tagaaa agaaaa gaaaaa aaaaac aaaaca aaacat aacatc acatct catctc atctcc tctcct ctcctg tcctga cctgag ctgaga tgagag gagaga agagag gagaga agagac gagacg agacgt gacgtt acgttt cgtttg gtttga tttgag ttgagg tgagga gaggat aggatt ggattc gattcc attcca ttccaa tccaat ccaatc caatct aatctg atctga tctgat ctgata tgatac gatacc ataccc taccca acccaa cccaaa ccaaag caaaga aaagag aagaga agagat gagatt agattc gattct attctc ttctct tctctg ctctgg tctggc ctggcg tggcga ggcgac gcgacc cgacca gaccaa accaaa ccaaaa caaaag aaaaga aaagaa aagaag agaagc gaagcc aagcct agcctt gccttc ccttca cttcag ttcagt tcagtg cagtga agtgat gtgatg tgatgg gatgga atggaa tggaat ggaata gaataa aataac ataact taactt aacttc acttca cttcag ttcagt tcagtt cagtta agttaa gttaag ttaaga taagaa aagaat agaatt gaattc aattcc attcct ttcctg tcctgg cctggt ctggtt tggttt ggttta gtttaa tttaag ttaaga taagat aagatt agattt gatttt atttta ttttac tttaca ttacag tacagg acagga caggaa aggaat ggaata gaatac aatacg atacgt tacgtt acgttg cgttga gttgaa ttgaac tgaaca gaacat aacatc acatca catcag atcagg tcaggt caggtt aggttg ggttga gttgat ttgatg tgatga gatgat atgatg tgatgc gatgca atgcaa tgcaat gcaatg caatga aatgat atgatg tgatgg gatggc atggca tggcaa ggcaaa gcaaat caaatg aaatgc aatgca atgcag tgcagc gcagcc cagccc agccca gcccag cccaga ccagaa cagaaa agaaat gaaatt aaattt aatttc atttcc tttccc ttccct tccctt cccttc ccttca cttcaa ttcaat tcaata caatac aatact atactc tactcc actcct ctccta tcctaa cctaaa ctaaaa taaaac aaaacc aaacca aaccaa accaaa ccaaag caaaga aaagaa aagaag agaagg gaagga aaggat aggata ggatat gatatt atatta tattac attact ttacta tactac actacc ctaccg taccgt accgtc ccgtca cgtcaa gtcaag tcaagt caagtc aagtct agtctt gtcttt tctttg ctttga tttgaa ttgaac tgaacg gaacgc aacgcc acgcca cgccat gccatt ccatta cattac attacc ttaccc taccca acccag cccagg ccaggc caggcc aggccg ggccgg gccggg ccgggc cgggct gggctg ggctga gctgac ctgact tgactg gactgg actggc ctggct tggctg ggctga gctgag ctgagc tgagcc gagcca agccat gccatt ccatta cattac attact ttactg tactgg actgga ctggat tggatg ggatgc gatgcc atgccc tgccca gcccaa cccaag ccaagt caagtg aagtgg agtgga gtggat tggatc ggatca gatcaa atcaat tcaatg caatgc aatgcc atgcca tgccac gccact ccactg cactga actgac ctgacc tgaccc gaccct accctt cccttc ccttct cttctg ttctgc tctgcc ctgccc tgcccg gcccgc cccgca ccgcac cgcacg gcacgc cacgct acgctg cgctga gctgac ctgacc tgaccc gaccca acccac cccact ccacta cactac actaca ctacaa tacaag acaagt caagtc aagtca agtcag gtcagc tcagct cagctg agctgt gctgtc ctgtca tgtcaa gtcaaa tcaaag caaagc aaagct aagctt agctta gcttag
y_data_____no_output_____

We will perform the same steps for chimpanzee and dog_____no_output_____

chimp_texts = list(chimp_data['words'])
for item in range(len(chimp_texts)):
    chimp_texts[item] = ' '.join(chimp_texts[item])
y_chimp = chimp_data.iloc[:, 0].values  _____no_output_____dog_texts = list(dog_data['words'])
for item in range(len(dog_texts)):
    dog_texts[item] = ' '.join(dog_texts[item])
y_dog = dog_data.iloc[:, 0].values  _____no_output_____

Now we will apply the BAG of WORDS using CountVectorizer using NLP_____no_output_____

# Creating the Bag of Words model using CountVectorizer()
# This is equivalent to k-mer counting
# The n-gram size of 4 was previously determined by testing
from sklearn.feature_extraction.text import CountVectorizer
cv = CountVectorizer(ngram_range=(4,4))
X = cv.fit_transform(human_texts)
X_chimp = cv.transform(chimp_texts)
X_dog = cv.transform(dog_texts)
print(X.shape)
print(X_chimp.shape)
print(X_dog.shape)(4380, 232414)
(1682, 232414)
(820, 232414)

If we have a look at class balance we can see we have relatively balanced data set.

_____no_output_____

human_data['class'].value_counts().sort_index().plot.bar()_____no_output_____

##Splitting the human data set into the training set and test set_____no_output_____

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X,
                                                    y_data,
                                                    test_size = 0.20,
                                                    random_state=42)
print(X_train.shape)
print(X_test.shape)(3504, 232414)
(876, 232414)

A multinomial naive Bayes classifier will be created. I previously did some parameter tuning and found the ngram size of 4 (reflected in the Countvectorizer() instance) and a model alpha of 0.1 did the best._____no_output_____

# The alpha parameter was determined by grid search previously
from sklearn.naive_bayes import MultinomialNB
classifier = MultinomialNB(alpha=0.1)
classifier.fit(X_train, y_train)_____no_output_____y_pred = classifier.predict(X_test)_____no_output_____y_pred_____no_output_____

Okay, so let’s look at some model performance metrics like the confusion matrix, accuracy, precision, recall and f1 score.

We are getting really good results on our unseen data, so it looks like our model did not overfit to the training data. In a real project I would go back and sample many more train test splits since we have a relatively small data set._____no_output_____

from sklearn.metrics import accuracy_score, f1_score, precision_score, recall_score
print("Confusion matrix\n")
print(pd.crosstab(pd.Series(y_test, name='Actual'), pd.Series(y_pred, name='Predicted')))
def get_metrics(y_test, y_predicted):
    accuracy = accuracy_score(y_test, y_predicted)
    precision = precision_score(y_test, y_predicted, average='weighted')
    recall = recall_score(y_test, y_predicted, average='weighted')
    f1 = f1_score(y_test, y_predicted, average='weighted')
    return accuracy, precision, recall, f1
accuracy, precision, recall, f1 = get_metrics(y_test, y_pred)
print("accuracy = %.3f \nprecision = %.3f \nrecall = %.3f \nf1 = %.3f" % (accuracy, precision, recall, f1))Confusion matrix

Predicted   0    1   2    3    4   5    6
Actual
0          99    0   0    0    1   0    2
1           0  104   0    0    0   0    2
2           0    0  78    0    0   0    0
3           0    0   0  124    0   0    1
4           1    0   0    0  143   0    5
5           0    0   0    0    0  51    0
6           1    0   0    1    0   0  263
accuracy = 0.984
precision = 0.984
recall = 0.984
f1 = 0.984
_____no_output_____


# Notebook from EricHe98/artificial-cilantro
Path: notebooks/exploratory/soojin-NB_validation.ipynb

# Setup_____no_output_____

import numpy as np
import pandas as pd
import scipy
import matplotlib.pyplot as plt
import seaborn as sns
sns.set_style('whitegrid')

data = pd.read_csv('~/Documents/Github/data/train.csv')
val = pd.read_csv('~/Documents/Github/data/dev.csv')_____no_output_____

# Naive Bayes on text
A quick way to mine the text information is by feeding it into a Naive Bayes model, which will learn which words and tfidf word combos tend to be associated with real vs. fake reviews.

- Massive gains seen using the Tfidf-Vectorizer over the normal CountVectorizer
- Though not shown, no particular changes in performance seen swapping `n`-grams around_____no_output_____

from sklearn.metrics import precision_recall_curve, average_precision_score

def plot_pr(pred, y, label, title=None):
    plt.rcParams['figure.figsize'] = [10,6]
    precision, recall, _ = precision_recall_curve(y, pred)
    aps = average_precision_score(y_true=y, y_score=pred)

    plt.step(recall, precision, alpha=0.2,
             where='post',
             label='{}: Average Precision = {:0.3f}'.format(label, aps))
    plt.fill_between(recall, precision, step='post', alpha=0.2)

    plt.xlabel('Recall', fontsize = 12)
    plt.ylabel('Precision', fontsize = 12)
    if title is None:
        title = 'Precision-Recall Curve'
    plt.title(title)
    plt.legend(loc="lower right")
    plt.ylim([0.0, 1.05])
    plt.xlim([0.0, 1.0])_____no_output_____from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer
from sklearn.naive_bayes import MultinomialNB, ComplementNB

cv = CountVectorizer(binary=False, ngram_range=(1,2), max_df=.5, min_df=.001)
text_data_train = cv.fit_transform(data['review'].tolist())
text_data_val = cv.transform(val['review'].tolist())

print(text_data_train.shape)
print(text_data_val.shape)

import scipy
text_data = scipy.sparse.vstack([text_data_train, text_data_val])(250874, 17747)
(35918, 17747)

### Naive Bayes with CV Performance_____no_output_____

y = data['label']
nb = MultinomialNB()
nb.fit(text_data_train, y)

data['pred_nb'] = nb.predict_proba(text_data_train)[:,1]
val['pred_nb'] = nb.predict_proba(text_data_val)[:,1]
plot_pr(data['pred_nb'], data['label'], label='Naive Bayes Training')
plot_pr(val['pred_nb'], val['label'], label='Naive Bayes Validation')

plt.title('Naive Bayes with CV Performance', fontsize = 15)_____no_output_____

## Large improvements using the `TfidfVectorizer`_____no_output_____

cv = TfidfVectorizer(binary=False, ngram_range=(1,1), max_df=.5, min_df=.001)
text_data_train = cv.fit_transform(data['review'].tolist())
text_data_val = cv.transform(val['review'].tolist())

print(text_data_train.shape)
print(text_data_val.shape)

text_data = scipy.sparse.vstack([text_data_train, text_data_val])(250874, 4701)
(35918, 4701)

### Naive Bayes with Tfidf Performance_____no_output_____

y = data['label']
nb = MultinomialNB()
nb.fit(text_data_train, y)

data['pred_nb'] = nb.predict_proba(text_data_train)[:,1]
val['pred_nb'] = nb.predict_proba(text_data_val)[:,1]
plot_pr(data['pred_nb'], data['label'], label='Naive Bayes Training')
plot_pr(val['pred_nb'], val['label'], label='Naive Bayes Validation')
plt.title('Naive Bayes with Tfidf Performance', fontsize = 15)_____no_output_____

## Complement Naive Bayes_____no_output_____

cnb = ComplementNB()
cnb.fit(text_data_train, y)

data['pred_nb'] = nb.predict_proba(text_data_train)[:,1]
val['pred_nb'] = nb.predict_proba(text_data_val)[:,1]
plot_pr(data['pred_nb'], data['label'], label='Naive Bayes Training')
plot_pr(val['pred_nb'], val['label'], label='Naive Bayes Validation')
plt.title('Complement Naive Bayes Performance', fontsize = 15)_____no_output_____

## What are the largest magnitude Tfidf words for each document?
We might wonder which words see high tf-idf scores. If we leaf through the samples, we generally see that words relating to different cuisines, times, or sentiments tend to see high tf-idf scores. The lowest tfidf scores are extremely generic words. Very satisfying to observe!_____no_output_____

vocab = pd.Series(cv.vocabulary_).reset_index().set_index(0)_____no_output_____def tfidf_scores(s, vocab, n=10, ascending=False):
    row_idx, col_idx, tfidf_value = scipy.sparse.find(s)
    words = vocab.loc[col_idx].copy()
    words['tfidf'] = tfidf_value
    words = words.sort_values('tfidf', ascending=ascending)[:n]
    return words['index'].tolist()_____no_output_____row_samples = np.random.randint(0, text_data.shape[0], 10)

print('Highest TFIDF')
for row in row_samples:
    print(tfidf_scores(text_data[row,:], vocab))
print('\n\n')
print('Lowest TFIDF')
for row in row_samples:
    print(tfidf_scores(text_data[row,:], vocab, ascending=True))Highest TFIDF
['lunchtime', 'we', 'mon', 'il', 'advice', 'winner', 'were', 'clear', 'works', '14']
['hole', 'thinks', 'handmade', 'walls', 'you', 'what', 'tip', 'wall', 'tell', 'noodles']
['policy', 'inexpensive', 'somehow', 'cause', 'byob', 'mood', 'business', 'hit', 'him', 'owner']
['discovered', 'beyond', 've', 'lived', 'gone', 'recently', 'year', 'servers', 'corner', 'twice']
['breakfast', 'junior', '11am', 'pastrami', 'swiss', 'day', 'rings', 'ends', 'informed', 'onion']
['pizza', 'slice', 'star', 'really', 'not']
['chocolate', 'cake', 'carrots', 'dish', 'delicious', 'rhubarb', 'peanuts', 'candied', 'seeds', 'popcorn']
['tons', 'once', 'tvs', 'assured', 'trips', 'character', 'most', 'final', 'interested', 'hasn']
['overwhelmingly', 'sub', 'par', 'pm', 'somewhere', '40', 'dumplings', 'crowded', 'noodles', 'least']
['mignon', 'bc', 'filet', 'little', 'which', 'pretty', 'steak', 'would', 'special', 'that']

Lowest TFIDF
['that', 'my', 'good', 'they', 'great', 'there', 'as', 'all', 'very', 'just']
['just', 'like', 'if', 'back', 'go', 'will', 'don', 'order', 'know', 'tasty']
['food', 'my', 'good', 'not', 'all', 'just', 'service', 'from', 'what', 'our']
['food', 'place', 'we', 'were', 'are', 'all', 'out', 'can', 'delicious', 'from']
['that', 'on', 'at', 'be', 'all', 'if', 'really', 'by', 'well', 'ordered']
['not', 'really', 'star', 'slice', 'pizza']
['that', 'they', 'have', 'were', 'be', 'all', 'if', 'out', 'get', 'really']
['on', 'place', 'good', 'you', 'not', 'were', 'here', 'just', 'if', 'service']
['on', 'good', 'go', 'wait', 'even', 'long', 'day', 'minutes', '10', 'else']
['my', 'place', 'you', 'had', 'were', 'at', 'be', 'if', 'get', 'me']


# Notebook from wilsonjefferson/DSSC_AP
Path: exercises/python/.ipynb_checkpoints/advanced_topics-checkpoint.ipynb

## DECORATORS_____no_output_____

'''
it allows programmers to modify the behavior of function or class.
Decorators allow us to wrap another function in order to extend the
behavior of wrapped function, without permanently modifying it.

In Decorators, functions are taken as the argument into another
function and then called inside the wrapper function.
'''

'''
In the above code, decorator_1 is a callable function, will add some code
on the top of some another callable function (func), and return the wrapper function.
'''
def decorator_1(func):
    print("hello, here is dec_1")
    def inner_1():
        print("hello, here is inner_1")
        func()
    return inner_1

@decorator_1
def func():
    print("hello, here is func")

func()hello, here is dec_1
hello, here is inner_1
hello, here is func

# importing libraries
import time
import math

# decorator to calculate duration
# taken by any function.
def calculate_time(func):

    # added arguments inside the inner1,
    # if function takes any arguments,
    # can be added like this.
    def inner1(*args, **kwargs):

        # storing time before function execution
        begin = time.time()

        func(*args, **kwargs)

        # storing time after function execution
        end = time.time()
        print("Total time taken in : ", func.__name__, end - begin)

    return inner1

# this can be added to any function present,
# in this case to calculate a factorial
@calculate_time
def factorial(num):

    # sleep 2 seconds because it takes very less time
    # so that you can see the actual difference
    time.sleep(2)
    print(math.factorial(num))

# calling the function.
factorial(10) 3628800
Total time taken in :  factorial 2.0015180110931396
def dec_1(func):
    print("-"*10, "DECOR 1", "-"*10)
    def inner_1():
        print("*"*30)
        func()
        print("*"*30)
    return inner_1

def dec_2(func):
    print("-"*10, "DECOR 2", "-"*10)
    def inner_2():
        print("%"*30)
        func()
        print("%"*30)
    return inner_2

#@dec_1
#@dec_2
def func():
    print("     hello, here is func")

tmp = dec_1(dec_2(func))
print("\n")
print(tmp())---------- DECOR 2 ----------
---------- DECOR 1 ----------

******************************
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
     hello, here is func
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
******************************
None
'''
| - dec_2
| | -- dec_1
| | | --- inner_1
| | | | ---- inner_2
| | | | | ------- func
| | | | ---- inner_2
| | | --- inner_1
| |  -- dec_1
| - dec_2
'''

'''
the real decoration is applied by the inners functions
of the decs: so, in this case dec_2.inner_2 applied
decoration and than dec_1 take what is decorated and
applied it's decoration throught dec_1.inner_1
'''_____no_output_____def star(func):
    print("star")
    def inner(*args, **kwargs):
        print("*" * 30)
        func(*args, **kwargs)
        print("*" * 30)
    return inner

def percent(func):
    print("percent")
    def inner(*args, **kwargs):
        print("%" * 30)
        func(*args, **kwargs)
        print("%" * 30)
    return inner

@star
@percent
def printer(msg):
    print(msg);

printer("Hello!");percent
star
******************************
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
Hello!
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
******************************
def dec_1(func):
    print("-"*10, "DECOR 1", "-"*10)
    def inner_1(*args, **kwargs):
        print("*"*30)
        func(*args, **kwargs)
        print("*"*30)
    return inner_1

def dec_2(func):
    print("-"*10, "DECOR 2", "-"*10)
    def inner_2(*args, **kwargs):
        print("%"*30)
        func(*args, **kwargs)
        print("%"*30)
    return inner_2

def func_1(a):
    print("     hello, here is func:", a)

def func_2(a, b):
    print("     hello, here is func:", a, "and", b)

def func_3(**kwargs):
    for i in [kwargs]:
        print(i)

tmp_1 = dec_1(dec_2(func_1))
tmp_2 = dec_1(dec_2(func_2))
tmp_3 = dec_1(dec_2(func_3))

print("\n")

print(tmp_1(3))
print(tmp_2(*[1, 2]))
print(tmp_3(**{"vic" : 1, "pietro" : 2}))---------- DECOR 2 ----------
---------- DECOR 1 ----------
---------- DECOR 2 ----------
---------- DECOR 1 ----------
---------- DECOR 2 ----------
---------- DECOR 1 ----------

******************************
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
     hello, here is func: 3
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
******************************
None
******************************
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
     hello, here is func: 1 and 2
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
******************************
None
******************************
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
{'vic': 1, 'pietro': 2}
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
******************************
None
'''
if we want pass an arg to func we have to remember that
it has 2 decorations that encapsulate him, so we have to
pass the args through the decs
'''

'''
*args = get element by positional, so if we pass [1, 2] to the func
        it will consider separately 1 and 2

**kwargs = get key-value by positional, so if we pass
           {"vic": 1, "pietro" : 2} it will consider separately
           "vic": 1 and "pietro" : 2
'''(1,)

## PROPERTY_____no_output_____

class Point():
    def __init__(self, x, y):
        self._x = x
        self._y = y

    @property
    def x(self):
        return self._x

    @property
    def y(self):
        return self._y

    @x.setter
    def x(self, new_x):
        if(self._x < new_x):
            self._x = new_x

def use_point(a):
    a.x = 0

p_0 = Point(1, 2)
use_point(p_0) # ERROR: cannot modify x if you have just @property
print(p_0.x, p_0.y)1 2

## ERROR HANDLING_____no_output_____

x = -1
try:
    assert x>=0, 'x is negative'
except AssertionError as err:
    print(err)x is negative
print("# let's catch some common exceptions\n")
to_execute = ['1/0',\
              '4+unknown_var',\
              '"2"+2',\
              "int('s')",
              "print 1"]
for i in to_execute:
    try:
        print(i)
        eval(i)
        print("I am not going to be printed")
    #except Exception as err:
    #    print(type(err),":",err)
    except (ZeroDivisionError,NameError,TypeError,ValueError) as err:
        print(type(err),":",err)
    except SyntaxError as err:
        print("'print 1' was not caught because it causes SyntaxError")
        print(type(err),":",err)
    finally: # should be at the end of try statement
             # useful to make sure all resources are released
             # even if an exception occurs
             # even if no exception was caught
        print("---last but not least, finally is executed---\n")# let's catch some common exceptions

1/0
<class 'ZeroDivisionError'> : division by zero
---last but not least, finally is executed---

4+unknown_var
<class 'NameError'> : name 'unknown_var' is not defined
---last but not least, finally is executed---

"2"+2
<class 'TypeError'> : can only concatenate str (not "int") to str
---last but not least, finally is executed---

int('s')
<class 'ValueError'> : invalid literal for int() with base 10: 's'
---last but not least, finally is executed---

print 1
'print 1' was not caught because it causes SyntaxError
<class 'SyntaxError'> : Missing parentheses in call to 'print'. Did you mean print(1)? (<string>, line 1)
---last but not least, finally is executed---

## INHERITANCE_____no_output_____

class Animal:
    def __init__(self):
        self.age = 99

    def speak(self, verso = "unkown"):
        print("verse:", verso)

class Dog(Animal):
    def speak(self):
        super().speak("Bau Bau!")

a_0 = Animal()
a_0.speak()
a_0.age

a_1 = Dog()
a_1.speak()
a_1.ageverse: unkown
verse: Bau Bau!
class A:
    def __init__(self):
        self.x = 99

    def foo(self):
        print("A foo")

class B(A):
    def foo(self):
        print("B foo")

class C(A):
    def foo(self):
        print("C foo")

class D(A):
    def foo(self):
        print("D foo")
        print("x =", self.x)
        A.foo(self) # call of foo from class A

a = A()
a.foo()

b = B()
b.foo()

d = D()
d.foo()

# Method Resolution Order (MRO) is the order in which methods
# should be inherited in the presence of multiple inheritance.
# Classes Derived appears by the most "recent" to the "base" class.
D.__mro__ A foo
B foo
D foo
x = 99
A foo
print(isinstance(d, A)) # instead of type() for user-classesTrue
import abc # to define abstract methods

class Animal(abc.ABC):
    def __init__(self, class_animal = "unkown"):
        self.class_animal = class_animal

    @abc.abstractmethod
    def number_of_paws(self, number):pass

class Horse(Animal):
    def number_of_paws(self, number):
        print("This horse has", number, "paws")

'''
don't instanciate an abstract class if
you want you can do a class for Horse
'''
a = Horse() # expect error...
print("a class:", a.class_animal)
a.number_of_paws(4)

print("\n")

b = Horse("herbivorous")
print("b class:", b.class_animal)
b.number_of_paws(4)a class: unkown
This horse has 4 paws

b class: herbivorous
This horse has 4 paws

## GENERATORS_____no_output_____

class ListaSpesa:
    def __init__(self, comprare):
        self.comprare = comprare
        self.lunghezza = len(comprare)

    def __iter__(self):
        return self

    def __next__(self):
        if self.lunghezza == 0:
            print("finita spesa!")
            raise StopIteration
        else:
            self.lunghezza = self.lunghezza -1
        return self.comprare[self.lunghezza]

for i in ListaSpesa(["mele", "pere", "banane"]):
    print(i)banane
pere
mele
finita spesa!
lista = ListaSpesa(["mele", "pere", "banane", "carne"])

print(next(lista))
print(next(lista))
print(next(lista))
print(next(lista))

print("*"*10)

for i in lista:
    print(i)carne
banane
pere
mele
**********
finita spesa!
# Simply speaking, a generator is a function that returns an object
# (iterator) which we can iterate over (one value at a time)
# they don't save in memory their contents
def reverse(data):
    for index in range(len(data)-1, -1, -1): # handle in reverse order through range()
        yield data[index]

for i in reverse(("first", 'second', 3, 'IV')):
    print(i)IV
3
second
first

## CONTENT MANAGER_____no_output_____

with open("file.txt", "w") as f: # open a new file in write mode
    for i in range(10):
        print(i, file = f) # specify where the function have to print

f.closed # to check if the file is closed

with open('file.txt') as f: # read mode is default
    a = [int(i) for i in f] # read from file f and store in a lis
    print(a)_____no_output_____


# Notebook from genepattern/example-notebooks
Path: GenePattern Python Tutorial.ipynb

# GenePattern Python Library Tutorial

This is a short tutorial on how to use the [GenePattern Python library](http://www.broadinstitute.org/cancer/software/genepattern/programmers-guide#_Using_GenePattern_from_Python) in your Python scripts or in the [GenePattern Notebook](http://www.broadinstitute.org/cancer/software/genepattern/genepattern-notebooks) environment. In it we will develop a simple Python program that connects to a GenePattern Server, runs a module and loads the resulting files for further analysis. The included code can be executed directly from the notebook, so that you can try it out, modify it or create your own solutions.

This tutorial assumes that the reader is familiar with [GenePattern](http://genepattern.org) and its basic associated concepts, such as modules, jobs and result files. If you wish to learn more about GenePattern, [click here](http://www.broadinstitute.org/cancer/software/genepattern/).

### Installation

If you are using this tutorial in the GenePattern Notebook environment, the GenePattern Python library will already be installed. If you're using this notebook outside of the GenePattern Notebook environment, you may need to install the library. [Installation instructions are available here.](http://www.broadinstitute.org/cancer/software/genepattern/programmers-guide#_Using_GenePattern_from_Python)

### Compatibility

The GenePattern Python library supports both Python 2.7 and Python 3.3+._____no_output_____## Import the Library

The very first thing you will need to do is to import the GenePattern library into your script. This can be achieved by entering the following code shown below. All methods provided by the GenePattern library can then be accessed from the *gp* namespace._____no_output_____

import gp_____no_output_____

## Connect to a GenePattern Server

The next step in using the GenePattern Python library is to connect to an existing GenePattern server. This will require entering the URL of the server, as well as your username and password credentials.

The code below connects to the public GenePattern server running at the Broad Institute. If you have not used this server before, you will first need to create an account. [Click here to register an account.](http://genepattern.broadinstitute.org/gp/pages/registerUser.jsf)

Needless to say, you will need to change *myusername* and *mypassword* in the code below to your actual username and password._____no_output_____

# Create a GenePattern server proxy instance
gpserver = gp.GPServer('https://cloud.genepattern.org/gp','myusername', 'mypassword')_____no_output_____

### Query for Available Modules

If you are not aware of which modules are available on the GenePattern server, this can be programmatically explored by running the code shown below._____no_output_____

# Get the list of tasks
task_list = gpserver.get_task_list()_____no_output_____

This will return a list of GPTask objects. A GPTask object is a Python object which represents a specific module. In the case of the command above, the returned list will contain GPTask objects for every module the server makes available. These GPTask objects will provide the module name, LSID, a description and the version number of the module. More on GPTask objects is described in the next section.

This list can be iterated over like any Python list to obtain information from each GPTask object. Below is a code example that will print the name of each module._____no_output_____

for task in task_list:
    print(task.get_name())_____no_output_____

## Referencing a Module

If you already know the name or LSID of the module you want, you can also obtain a GPTask object for it directly.

Throughout this tutorial we will use the PreprocessDataset module. This module is designed to perform some basic preprocessing steps on gene expression data. For example, it can apply a ceiling, floor and fold change filter. More on this module is [available here](http://www.broadinstitute.org/cancer/software/genepattern/modules/docs/PreprocessDataset/5).

The example code below obtains a GPTask object for the PreprocessDataset module in two different ways: by name and by LSID. Either one of these methods works to obtain the reference, although specifying the LSID is more specific as it contains a specific version number for the module, whereas specifying by name always obtains the latest version of the module. See how GenePattern implements  [version numbers](https://www.genepattern.org/concepts/#_Version_Numbers) using Life Science Identifiers (LSIDs)._____no_output_____

# Obtaining GPTask by module name
module = gp.GPTask(gpserver, "PreprocessDataset")

# Obtaining GPTask by LSID
module = gp.GPTask(gpserver, "urn:lsid:broad.mit.edu:cancer.software.genepattern.module.analysis:00020")_____no_output_____

### Loading Parameter Information

At this point the GPTask object only contains some very basic data used to identify the module. Before you can use this module to run jobs, however, you will first need to load the full parameter data from the server. This can be accomplished by running the code shown below._____no_output_____

# Load the full parameter data
module.param_load()_____no_output_____

Below is code example showing how to reference the information that was loaded for the module._____no_output_____

# Print the module name
print( module.get_name() )

# Print the module LSID
print( module.get_lsid() )

# Print the module version
print( module.get_version() )

# Print the description
print( module.get_description() )_____no_output_____

After being loaded, each of the module's parameters will be represented by a GPTaskParam object. A list of each of the parameters can be obtained and explored using the example code below. Each GPTaskParam object contains a parameter name, type, description, whether it's optional and other metadata, as shown._____no_output_____

# Get the list of GPTaskParam objects
params_list = module.get_parameters()

for param in params_list:              # Loop through each parameter
    print( param.get_name() )          # Print the parameter's name
    print( param.get_type() )          # Print the parameter's type (text, number, file, etc.)
    print( param.get_description() )   # Print the parameter's description
    print( param.get_default_value() ) # Print the parameter's default value
    print( param.is_optional() )       # Print whether the parameter is optional
    print( '' )                        # Leave a blank line between printed parameters_____no_output_____

Some parameters come with a list of valid choices. These parameters can be identified by calling the is_choice_param() method. Additionally, choice parameters have a number of other methods available for working with the choice list. Calling these methods on a non-choice parameter will result in an error being thrown. An example is given below._____no_output_____

# Loop through each parameter
for param in params_list:
    if param.is_choice_param():        # If the parameter is a choice param
        print( param.get_name() )      # Print the parameter's name

        choices = param.get_choices()  # Get a list of valid choices
        for choice in choices:         # Print the label and value for each choice
            print( choice['label'] + " = " + choice['value'] )

        # Print the default selected value for each choice
        print( param.get_choice_selected_value() )_____no_output_____

## Creating a Job Specification

In order to run a GenePattern job from Python, you must first obtain a GPJobSpec object from the correct GPTask object and then set the appropriate parameters for the job. For many parameters their default values will suffice. For others, you will want to set a specific value.

Below is code showing how to obtain a GPJobSpec object and how iterate over the parameters, setting them to their default values._____no_output_____

# Create the GPJobSpec
job_spec = module.make_job_spec()

# Loop through all the parameters and set their default values
for param in module.get_parameters():
    # If the parameter has a default value, set that value
    if param.get_default_value() != None:
        # Set the default value
        job_spec.set_parameter( param.get_name(), param.get_default_value() )  _____no_output_____

To set a specific value for a parameter, the set_parameter() method should be called. In the code below you will set the *input.filename* parameter to point to a publicly available dataset. This data should suffice for the purposes of this tutorial._____no_output_____

# Attach the input file to the correct parameter
job_spec.set_parameter("input.filename", "https://datasets.genepattern.org/data/all_aml/all_aml_test.gct") _____no_output_____

Data files can be uploaded by calling GPServer.upload_file(). This will return a GPFile object, and the parameter can be set to point to the URL of this object. An example for the PreprocessDataset module is shown below. The code has been commented out, however, as it will not be used in this tutorial._____no_output_____

# Upload the input file
# uploaded_file = gpserver.upload_file("file_name", "/path/to/the/file/on/the/file/system/file_name")

# Attach the input file to the correct parameter
# job_spec.set_parameter("input.filename", 'uploaded_file.get_url()')  _____no_output_____

## Submitting Your First Job

Once the GPJobSpec is ready, it can be used to launch a GenePattern job. This will return a GPJob object, representing the specific job that was just launched. A code example of how to do this is below._____no_output_____

# This will return the job object and continue execution even if the job isn't finished
job = gpserver.run_job(job_spec, False)_____no_output_____

Why are we passing *False* in as a parameter, you ask? By default the run_job() method will halt code execution of your Python script until the job has finished running in GenePattern. For long running jobs, however, this may not be desirable. By optionally passing in *False* as a parameter, the method will return as soon as the job is submitted, allowing the Python program to continue.

For the purposes of this tutorial, it is better that you do not have to wait. If you did want to submit the job and wait for it to complete, however, the code is below (albeit commented out)._____no_output_____

#  This will halt execution until the job is complete
# job = gpserver.run_job(job_spec)_____no_output_____

### Querying for Job Status

When a GenePattern job is submitted, it passes through several states: pending, running and then either to complete or error. At any time after a job has been submitted, its status can be checked by calling *get_status_message()*. Similarly, its completion can be checked by calling *is_finished()*. Examples of both are shown below._____no_output_____

# Prints a brief description of the job's current state
print( job.get_status_message() )

# Quaries the server and returns True if the job is complete, False otherwise
print( job.is_finished() )_____no_output_____

Finally, if at any point you decide that you just want to wait until the job is complete, you can always call *wait_until_done()*._____no_output_____

job.wait_until_done()_____no_output_____

## Working with Output Files

Once the job is complete, and assuming there were no errors, a list of its output files may be obtained by making the *get_output_files()* call shown below.

This will return a list of GPFile objects, each containing methods to download or read the contents of the file._____no_output_____

# Get a list of output files
output_list = job.get_output_files()

for file in output_list:     # Loop through each output file
    print( file.get_url() )  # Print the URL to the file
    data = file.read()       # Read the data in the file _____no_output_____

Once the contents of a data file has been assigned to a variable, it may used in conjunction with other common Python libraries, such as matplotlib, pandas, numpy or scipy.

The code below will print out the contents of the last output file assigned to the *data* variable in the previous code block._____no_output_____

print ( data )_____no_output_____

This concludes the tutorial on how to work with GenePattern using the GenePattern Python library. For more information, please see the [GenePattern Programmer's Guide](http://www.broadinstitute.org/cancer/software/genepattern/programmers-guide)._____no_output_____