# Notebook from LuxTheDude/modern-ai-course
Path: lecture-08/lab.ipynb

## Train a Deep Q Learning (DQN) agent on the MSPacman-v0_____no_output_____## Mounting google drive for saving the video_____no_output_____

from google.colab import drive
drive.mount('/content/gdrive')Drive already mounted at /content/gdrive; to attempt to forcibly remount, call drive.mount("/content/gdrive", force_remount=True).

## Get the MSPackman ready from Atari_____no_output_____

import urllib.request
urllib.request.urlretrieve('http://www.atarimania.com/roms/Roms.rar','Roms.rar')
!pip install unrar
!unrar x Roms.rar
!mkdir rars
!mv HC\ ROMS.zip   rars
!mv ROMS.zip  rars
!python -m atari_py.import_roms rarsCollecting unrar
  Downloading unrar-0.4-py3-none-any.whl (25 kB)
Installing collected packages: unrar
Successfully installed unrar-0.4

UNRAR 5.50 freeware      Copyright (c) 1993-2017 Alexander Roshal

Extracting from Roms.rar

Extracting  HC ROMS.zip                                                   36%  OK
Extracting  ROMS.zip                                                      74% 99%  OK
All OK
copying adventure.bin from ROMS/Adventure (1980) (Atari, Warren Robinett) (CX2613, CX2613P) (PAL).bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/adventure.bin
copying air_raid.bin from ROMS/Air Raid (Men-A-Vision) (PAL) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/air_raid.bin
copying alien.bin from ROMS/Alien (1982) (20th Century Fox Video Games, Douglas 'Dallas North' Neubauer) (11006) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/alien.bin
copying amidar.bin from ROMS/Amidar (1982) (Parker Brothers, Ed Temple) (PB5310) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/amidar.bin
copying assault.bin from ROMS/Assault (AKA Sky Alien) (1983) (Bomb - Onbase) (CA281).bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/assault.bin
copying asterix.bin from ROMS/Asterix (AKA Taz) (07-27-1983) (Atari, Jerome Domurat, Steve Woita) (CX2696) (Prototype).bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/asterix.bin
copying asteroids.bin from ROMS/Asteroids (1981) (Atari, Brad Stewart - Sears) (CX2649 - 49-75163) [no copyright] ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/asteroids.bin
copying atlantis.bin from ROMS/Atlantis (Lost City of Atlantis) (1982) (Imagic, Dennis Koble) (720103-1A, 720103-1B, IA3203, IX-010-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/atlantis.bin
copying bank_heist.bin from ROMS/Bank Heist (Bonnie & Clyde, Cops 'n' Robbers, Hold-Up, Roaring 20's) (1983) (20th Century Fox Video Games, Bill Aspromonte) (11012) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/bank_heist.bin
copying battle_zone.bin from ROMS/Battlezone (1983) (Atari - GCC, Mike Feinstein, Brad Rice) (CX2681) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/battle_zone.bin
copying beam_rider.bin from ROMS/Beamrider (1984) (Activision - Cheshire Engineering, David Rolfe, Larry Zwick) (AZ-037-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/beam_rider.bin
copying berzerk.bin from ROMS/Berzerk (1982) (Atari, Dan Hitchens - Sears) (CX2650 - 49-75168) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/berzerk.bin
copying bowling.bin from ROMS/Bowling (1979) (Atari, Larry Kaplan - Sears) (CX2628 - 6-99842, 49-75117) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/bowling.bin
copying boxing.bin from ROMS/Boxing - La Boxe (1980) (Activision, Bob Whitehead) (AG-002, CAG-002, AG-002-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/boxing.bin
copying breakout.bin from ROMS/Breakout - Breakaway IV (Paddle) (1978) (Atari, Brad Stewart - Sears) (CX2622 - 6-99813, 49-75107) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/breakout.bin
copying carnival.bin from ROMS/Carnival (1982) (Coleco - Woodside Design Associates, Steve 'Jessica Stevens' Kitchen) (2468) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/carnival.bin
copying centipede.bin from ROMS/Centipede (1983) (Atari - GCC) (CX2676) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/centipede.bin
copying chopper_command.bin from ROMS/Chopper Command (1982) (Activision, Bob Whitehead) (AX-015, AX-015-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/chopper_command.bin
copying crazy_climber.bin from ROMS/Crazy Climber (1983) (Atari - Roklan, Joe Gaucher, Alex Leavens) (CX2683) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/crazy_climber.bin
copying defender.bin from ROMS/Defender (1982) (Atari, Robert C. Polaro, Alan J. Murphy - Sears) (CX2609 - 49-75186) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/defender.bin
copying demon_attack.bin from ROMS/Demon Attack (Death from Above) (1982) (Imagic, Rob Fulop) (720000-200, 720101-1B, 720101-1C, IA3200, IA3200C, IX-006-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/demon_attack.bin
copying donkey_kong.bin from ROMS/Donkey Kong (1982) (Coleco - Woodside Design Associates - Imaginative Systems Software, Garry Kitchen) (2451) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/donkey_kong.bin
copying double_dunk.bin from ROMS/Double Dunk (Super Basketball) (1989) (Atari, Matthew L. Hubbard) (CX26159) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/double_dunk.bin
copying elevator_action.bin from ROMS/Elevator Action (1983) (Atari, Dan Hitchens) (CX26126) (Prototype) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/elevator_action.bin
copying enduro.bin from ROMS/Enduro (1983) (Activision, Larry Miller) (AX-026, AX-026-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/enduro.bin
copying fishing_derby.bin from ROMS/Fishing Derby (1980) (Activision, David Crane) (AG-004) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/fishing_derby.bin
copying freeway.bin from ROMS/Freeway (1981) (Activision, David Crane) (AG-009, AG-009-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/freeway.bin
copying frogger.bin from ROMS/Frogger (1982) (Parker Brothers, Ed English, David Lamkins) (PB5300) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/frogger.bin
copying frostbite.bin from ROMS/Frostbite (1983) (Activision, Steve Cartwright) (AX-031) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/frostbite.bin
copying galaxian.bin from ROMS/Galaxian (1983) (Atari - GCC, Mark Ackerman, Tom Calderwood, Glenn Parker) (CX2684) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/galaxian.bin
copying gopher.bin from ROMS/Gopher (Gopher Attack) (1982) (U.S. Games Corporation - JWDA, Sylvia Day, Todd Marshall, Robin McDaniel, Henry Will IV) (VC2001) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/gopher.bin
copying gravitar.bin from ROMS/Gravitar (1983) (Atari, Dan Hitchens, Mimi Nyden) (CX2685) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/gravitar.bin
copying hero.bin from ROMS/H.E.R.O. (1984) (Activision, John Van Ryzin) (AZ-036-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/hero.bin
copying ice_hockey.bin from ROMS/Ice Hockey - Le Hockey Sur Glace (1981) (Activision, Alan Miller) (AX-012, CAX-012, AX-012-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/ice_hockey.bin
copying jamesbond.bin from ROMS/James Bond 007 (James Bond Agent 007) (1984) (Parker Brothers - On-Time Software, Joe Gaucher, Louis Marbel) (PB5110) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/jamesbond.bin
copying journey_escape.bin from ROMS/Journey Escape (1983) (Data Age, J. Ray Dettling) (112-006) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/journey_escape.bin
copying kaboom.bin from ROMS/Kaboom! (Paddle) (1981) (Activision, Larry Kaplan, David Crane) (AG-010, AG-010-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/kaboom.bin
copying kangaroo.bin from ROMS/Kangaroo (1983) (Atari - GCC, Kevin Osborn) (CX2689) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/kangaroo.bin
copying keystone_kapers.bin from ROMS/Keystone Kapers - Raueber und Gendarm (1983) (Activision, Garry Kitchen - Ariola) (EAX-025, EAX-025-04I - 711 025-725) (PAL).bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/keystone_kapers.bin
copying king_kong.bin from ROMS/King Kong (1982) (Tigervision - Software Electronics Corporation, Karl T. Olinger - Teldec) (7-001 - 3.60001 VE) (PAL).bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/king_kong.bin
copying koolaid.bin from ROMS/Kool-Aid Man (Kool Aid Pitcher Man) (1983) (M Network, Stephen Tatsumi, Jane Terjung - Kool Aid) (MT4648) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/koolaid.bin
copying krull.bin from ROMS/Krull (1983) (Atari, Jerome Domurat, Dave Staugas) (CX2682) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/krull.bin
copying kung_fu_master.bin from ROMS/Kung-Fu Master (1987) (Activision - Imagineering, Dan Kitchen, Garry Kitchen) (AG-039-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/kung_fu_master.bin
copying laser_gates.bin from ROMS/Laser Gates (AKA Innerspace) (1983) (Imagic, Dan Oliver) (720118-2A, 13208, EIX-007-04I) (PAL).bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/laser_gates.bin
copying lost_luggage.bin from ROMS/Lost Luggage (Airport Mayhem) (1982) (Apollo - Games by Apollo, Larry Minor, Ernie Runyon, Ed Salvo) (AP-2004) [no opening scene] ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/lost_luggage.bin
copying montezuma_revenge.bin from ROMS/Montezuma's Revenge - Featuring Panama Joe (1984) (Parker Brothers - JWDA, Henry Will IV) (PB5760) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/montezuma_revenge.bin
copying mr_do.bin from ROMS/Mr. Do! (1983) (CBS Electronics, Ed English) (4L4478) (PAL).bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/mr_do.bin
copying ms_pacman.bin from ROMS/Ms. Pac-Man (1983) (Atari - GCC, Mark Ackerman, Glenn Parker) (CX2675) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/ms_pacman.bin
copying name_this_game.bin from ROMS/Name This Game (Guardians of Treasure) (1983) (U.S. Games Corporation - JWDA, Roger Booth, Sylvia Day, Ron Dubren, Todd Marshall, Robin McDaniel, Wes Trager, Henry Will IV) (VC1007) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/name_this_game.bin
copying pacman.bin from ROMS/Pac-Man (1982) (Atari, Tod Frye) (CX2646) (PAL).bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/pacman.bin
copying phoenix.bin from ROMS/Phoenix (1983) (Atari - GCC, Mike Feinstein, John Mracek) (CX2673) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/phoenix.bin
copying video_pinball.bin from ROMS/Pinball (AKA Video Pinball) (Zellers).bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/video_pinball.bin
copying pitfall.bin from ROMS/Pitfall! - Pitfall Harry's Jungle Adventure (Jungle Runner) (1982) (Activision, David Crane) (AX-018, AX-018-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/pitfall.bin
copying pooyan.bin from ROMS/Pooyan (1983) (Konami) (RC 100-X 02) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/pooyan.bin
copying private_eye.bin from ROMS/Private Eye (1984) (Activision, Bob Whitehead) (AG-034-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/private_eye.bin
copying qbert.bin from ROMS/Q-bert (1983) (Parker Brothers - Western Technologies, Dave Hampton, Tom Sloper) (PB5360) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/qbert.bin
copying riverraid.bin from ROMS/River Raid (1982) (Activision, Carol Shaw) (AX-020, AX-020-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/riverraid.bin
copying road_runner.bin from patched version of ROMS/Road Runner (1989) (Atari - Bobco, Robert C. Polaro) (CX2663) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/road_runner.bin
copying robotank.bin from ROMS/Robot Tank (Robotank) (1983) (Activision, Alan Miller) (AZ-028, AG-028-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/robotank.bin
copying seaquest.bin from ROMS/Seaquest (1983) (Activision, Steve Cartwright) (AX-022) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/seaquest.bin
copying sir_lancelot.bin from ROMS/Sir Lancelot (1983) (Xonox - K-Tel Software - Product Guild, Anthony R. Henderson) (99006, 6220) (PAL).bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/sir_lancelot.bin
copying skiing.bin from ROMS/Skiing - Le Ski (1980) (Activision, Bob Whitehead) (AG-005, CAG-005, AG-005-04) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/skiing.bin
copying solaris.bin from ROMS/Solaris (The Last Starfighter, Star Raiders II, Universe) (1986) (Atari, Douglas Neubauer, Mimi Nyden) (CX26136) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/solaris.bin
copying space_invaders.bin from ROMS/Space Invaders (1980) (Atari, Richard Maurer - Sears) (CX2632 - 49-75153) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/space_invaders.bin
copying star_gunner.bin from ROMS/Stargunner (1983) (Telesys, Alex Leavens) (1005) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/star_gunner.bin
copying surround.bin from ROMS/Surround (32 in 1) (Bit Corporation) (R320).bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/surround.bin
copying tennis.bin from ROMS/Tennis - Le Tennis (1981) (Activision, Alan Miller) (AG-007, CAG-007) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/tennis.bin
copying time_pilot.bin from ROMS/Time Pilot (1983) (Coleco - Woodside Design Associates, Harley H. Puthuff Jr.) (2663) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/time_pilot.bin
copying trondead.bin from ROMS/TRON - Deadly Discs (TRON Joystick) (1983) (M Network - INTV - APh Technological Consulting, Jeff Ronne, Brett Stutz) (MT5662) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/trondead.bin
copying tutankham.bin from ROMS/Tutankham (1983) (Parker Brothers, Dave Engman, Dawn Stockbridge) (PB5340) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/tutankham.bin
copying up_n_down.bin from ROMS/Up 'n Down (1984) (SEGA - Beck-Tech, Steve Beck, Phat Ho) (009-01) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/up_n_down.bin
copying venture.bin from ROMS/Venture (1982) (Coleco, Joseph Biel) (2457) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/venture.bin
copying pong.bin from ROMS/Video Olympics - Pong Sports (Paddle) (1977) (Atari, Joe Decuir - Sears) (CX2621 - 99806, 6-99806, 49-75104) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/pong.bin
copying wizard_of_wor.bin from ROMS/Wizard of Wor (1982) (CBS Electronics - Roklan, Joe Hellesen, Joe Wagner) (M8774, M8794) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/wizard_of_wor.bin
copying yars_revenge.bin from ROMS/Yars' Revenge (Time Freeze) (1982) (Atari, Howard Scott Warshaw - Sears) (CX2655 - 49-75167) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/yars_revenge.bin
copying zaxxon.bin from ROMS/Zaxxon (1983) (Coleco) (2454) ~.bin to /usr/local/lib/python3.7/dist-packages/atari_py/atari_roms/zaxxon.bin

## Packages
First, let’s import needed packages. Firstly, we need game (gym)
neural networks (torch.nn)
optimization (torch.optim)_____no_output_____

import gym
import cv2
import torch
import torch.nn as nn
import torch.nn.functional as F
import matplotlib.pyplot as plt
from collections import deque
import numpy as np
import random
import datetime_____no_output_____reward_number = 50_____no_output_____

DQN Algorithm
-------------
We train a policy that tries to maximize the discounted,
cumulative reward
$R_{t_0} = \sum_{t=t_0}^{\infty} \gamma^{t - t_0} r_t$, where
$R_{t_0}$ is *return*. The discount, $\gamma$ is the discount, between $0$ and $1$

Q-learning tries to find a function
$Q^*: State \times Action \rightarrow \mathbb{R}$, maximizes rewards:

\begin{align}\pi^*(s) = \arg\!\max_a \ Q^*(s, a)\end{align}

However, we don't know $Q^*$. So, we use neural network as a approximators, we can simply create one and train it to resemble $Q^*$.

For our training update rule, we'll use a fact that every $Q$
function for some policy obeys the Bellman equation:

\begin{align}Q^{\pi}(s, a) = r + \gamma Q^{\pi}(s', \pi(s'))\end{align}

The difference between the two sides of the equality is known as the temporal difference error, $\delta$:

\begin{align}\delta = Q(s, a) - (r + \gamma \max_a Q(s', a))\end{align}_____no_output_____

class CNN(torch.nn.Module):
    def __init__(self):
        super(CNN, self).__init__()
        self.convolution1 = nn.Conv2d(in_channels=3, out_channels=32, kernel_size=3)
        self.convolution2 = nn.Conv2d(in_channels=32, out_channels=32, kernel_size=5)
        self.convolution3 = nn.Conv2d(in_channels=32, out_channels=64, kernel_size=7)
        self.fc1 = nn.Linear(in_features=1792, out_features=256)
        self.fc2 = nn.Linear(in_features=256, out_features=128)
        self.fc3 = nn.Linear(in_features=128, out_features=256)
        self.fc4 = nn.Linear(in_features=256, out_features=32)
        self.fc5 = nn.Linear(in_features=32, out_features=9)

    def forward(self, x):
        x = x.cuda()
        x = F.relu(F.max_pool2d(self.convolution1(x), 3))
        x = F.relu(F.max_pool2d(self.convolution2(x), 3))
        x = F.relu(F.max_pool2d(self.convolution3(x), 3, 2))
        x = x.reshape(x.size(0), -1)
        x = F.relu(self.fc1(x))
        x = F.relu(self.fc2(x))
        x = F.relu(self.fc3(x))
        x = F.relu(self.fc4(x))
        x = self.fc5(x)
        return x

        ## Quick exercise: Please draw the CNN arch._____no_output_____model = CNN()_____no_output_____

##function to ensure that our code uses the GPU if available, and defaults to using the CPU if it isn't._____no_output_____

#function to ensure that our code uses the GPU if available, and defaults to using the CPU if it isn't.
def get_default_device():
    """Pick GPU if available, else CPU"""
    if torch.cuda.is_available():
        return torch.device('cuda')
    else:
        return torch.device('cpu')

# a function that can move data and model to a chosen device.
def to_device(data, device):
    """Move tensor(s) to chosen device"""
    if isinstance(data, (list,tuple)):
        return [to_device(x, device) for x in data]
    return data.to(device, non_blocking=True)_____no_output_____

## Loss Function and Optimizer_____no_output_____

criterion = ## your loss function from nn
optimizer = ## your optimizer _____no_output_____

## DQN Agent
### Replay Memory
We’ll be using experience replay memory for training our DQN. It stores the transitions that the agent observes, allowing us to reuse this data later. By sampling from it randomly, the transitions that build up a batch are decorrelated. It has been shown that this greatly stabilizes and improves the DQN training procedure.
### Hyperparameters_____no_output_____

class DQNAgent:
    def __init__(self, action_size = 9):
        self.state_size = 3
        self.action_size = action_size
        self.memory_n = deque(maxlen=2000)
        self.memory_p = deque(maxlen=2000)
        self.gamma = 0.95    # discount rate
        self.epsilon = 1.0  # exploration rate
        self.epsilon_min = 0.2
        self.epsilon_decay = 0.995
        self.learning_rate = 0.001
        self.model = model

    def remember(self, state, action, reward, next_state, done):
        if reward == 0:
            self.memory_p.append((state, action, reward, next_state, done))
        else:
            self.memory_n.append((state, action, reward, next_state, done))
    def act(self, state):
        if np.random.rand() <= self.epsilon:
            return random.randrange(self.action_size)
        state_tensor = torch.from_numpy(state).float()
        act_values = self.model(state_tensor).cpu().detach().numpy()
        return np.argmax(act_values[0])  # returns action

    def replay(self, batch_size):
        if len(agent.memory_n) > batch_size / 2:
            print("Negative batch ready:")
            minibatch_n = random.sample(self.memory_n, 5)
            minibatch_p = random.sample(self.memory_p, 59)
            minibatch = random.sample((minibatch_p+minibatch_n), batch_size)
        else:
            minibatch = random.sample(self.memory_p, batch_size)
        for state, action, reward, next_state, done in minibatch:
            ns_model = self.model(torch.from_numpy(next_state).float()).cpu().detach().numpy()
            if reward == 0:
                reward = 1.0001
                target = reward * np.amax(ns_model[0])
                target_f = ns_model
                target_f[0][np.argmax(ns_model[0])] = target
            else:
                reward = reward_number
                target = reward * np.amin(ns_model[0])
                target_max = 0.0001 * np.amax(ns_model[0])
                target_f = ns_model
                target_f[0][action] = target
                target_f[0][random.choice([i for i in range(0,9) if i not in [action]])] = target_max
                self.train(next_state, target_f, epochs=1)
        if self.epsilon > self.epsilon_min:
            self.epsilon *= self.epsilon_decay

    def train(self, input, target, epochs = 1):
        input = torch.from_numpy(input).float().cuda()
        target = torch.from_numpy(target).float().cuda()
        y_pred = 0
        for t in range(1):
            y_pred = model(input)
            loss = - criterion(y_pred, target)
            # print(t, loss.item())
            optimizer.zero_grad()
            loss.backward()
            optimizer.step()

    def load_all(self, name):
        loaded = torch.load(name)
        self.memory_n = loaded['memory_n']
        self.memory_p = loaded['memory_p']
        self.model.load_state_dict(loaded['state'])

    def save_all(self, name):
        torch.save({'state': self.model.state_dict(),
                    'memory_n': self.memory_n,
                    'memory_p': self.memory_p
                   }, name)

    def load(self, name):
        self.model.load_state_dict(torch.load(name))

    def save(self, name):
        torch.save(self.model.state_dict(), name)

_____no_output_____env = gym.make('MsPacman-v0')
state_size = env.observation_space.shape
action_size = env.action_space.n
_____no_output_____agent = DQNAgent()_____no_output_____done = False
batch_size = 64
_____no_output_____

## Main training loop_____no_output_____

EPISODES = 100
for e in range(EPISODES):
    fourcc = cv2.VideoWriter_fourcc(*'XVID')
    vw = cv2.VideoWriter('gdrive/' + "Reward_number_" + str(reward_number) + "_" + str(e) + str(datetime.datetime.now()) +  '.avi', fourcc, 4, (160,210))
    state = env.reset()
    state = np.reshape(state, (1, 210,160,3)).transpose(0,3,1,2)/255
    for time in range(1000000000):
        print(time)
        action = agent.act(state)
        next_state, reward, done, _ = env.step(action)
        vw.write(next_state)
        reward = reward if not done else 10
        reward = reward if reward ==0 else 10
        if reward != 0:
            print("ATTENTION NEGATIVE REWARD",reward)
        next_state = np.reshape(next_state, (1, 210,160,3)).transpose(0,3,1,2)/255
        agent.remember(state, action, reward, next_state, done)
        state = next_state
        if done:
            vw.release()
            agent.save('gdrive' + "Reward_number_" + str(reward_number) + "_""Frames:_" + str(time) + "_Episode_" +str(e) + "_Date_" + str(datetime.datetime.now()) + '.pt')
            print("episode: {}/{}, score: {}, e: {:.2}".format(e, EPISODES, time, agent.epsilon))
            break
        if (len(agent.memory_p) > batch_size) & (len(agent.memory_n) > batch_size/2) :
            agent.replay(batch_size)0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
ATTENTION NEGATIVE REWARD 10
87
88
89
90
91
92
93
94
95
96
ATTENTION NEGATIVE REWARD 10
97
98
99
100
101
102
103
104
ATTENTION NEGATIVE REWARD 10
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
ATTENTION NEGATIVE REWARD 10
153
154
155
156
157
ATTENTION NEGATIVE REWARD 10
158
159
160
161
162
163
164
165
ATTENTION NEGATIVE REWARD 10
166
167
168
169
170
171
172
ATTENTION NEGATIVE REWARD 10
173
174
175
176
177
ATTENTION NEGATIVE REWARD 10
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
ATTENTION NEGATIVE REWARD 10
194
195
196
197
ATTENTION NEGATIVE REWARD 10
198
199
200
201
202
ATTENTION NEGATIVE REWARD 10
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
ATTENTION NEGATIVE REWARD 10
223
224
225
226
227
228
229
230
ATTENTION NEGATIVE REWARD 10
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
ATTENTION NEGATIVE REWARD 10
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
ATTENTION NEGATIVE REWARD 10
336
337
338
339
340
ATTENTION NEGATIVE REWARD 10
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
ATTENTION NEGATIVE REWARD 10
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
ATTENTION NEGATIVE REWARD 10
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
ATTENTION NEGATIVE REWARD 10
494
495
496
497
498
499
500
501
502
503
504
ATTENTION NEGATIVE REWARD 10
505
506
507
508
509
ATTENTION NEGATIVE REWARD 10
510
511
512
513
514
ATTENTION NEGATIVE REWARD 10
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
ATTENTION NEGATIVE REWARD 10
534
535
536
537
538
539
540
ATTENTION NEGATIVE REWARD 10
541
542
543
544
ATTENTION NEGATIVE REWARD 10
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
ATTENTION NEGATIVE REWARD 10
episode: 0/100, score: 631, e: 1.0
0
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
ATTENTION NEGATIVE REWARD 10
90
91
92
93
ATTENTION NEGATIVE REWARD 10
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
ATTENTION NEGATIVE REWARD 10
121
122
123
124
125
ATTENTION NEGATIVE REWARD 10
126
127
128
129
130
131
132
133
134
135
ATTENTION NEGATIVE REWARD 10
136
137
138
139
140
141
142
ATTENTION NEGATIVE REWARD 10
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
ATTENTION NEGATIVE REWARD 10
Negative batch ready:
177
Negative batch ready:
178
Negative batch ready:
179
Negative batch ready:
180
Negative batch ready:
181
Negative batch ready:
182
Negative batch ready:
183
Negative batch ready:
184
Negative batch ready:
185
Negative batch ready:
186
Negative batch ready:
187
Negative batch ready:
188
Negative batch ready:
189
Negative batch ready:
190
Negative batch ready:
191
Negative batch ready:
192
Negative batch ready:
193
Negative batch ready:
194
Negative batch ready:
195
Negative batch ready:
196
Negative batch ready:
197
Negative batch ready:
198
Negative batch ready:
199
Negative batch ready:
200
ATTENTION NEGATIVE REWARD 10
Negative batch ready:
201
Negative batch ready:
202
Negative batch ready:
203
Negative batch ready:
204
Negative batch ready:
205
Negative batch ready:
206
Negative batch ready:
207
Negative batch ready:
208
Negative batch ready:
209
Negative batch ready:
210
Negative batch ready:
211
Negative batch ready:
212
Negative batch ready:
213
Negative batch ready:
214
Negative batch ready:
215
Negative batch ready:
216
Negative batch ready:
217
Negative batch ready:
218
Negative batch ready:
219
Negative batch ready:
220
Negative batch ready:
221
Negative batch ready:
222
Negative batch ready:
223
Negative batch ready:
224
Negative batch ready:
225
Negative batch ready:
226
Negative batch ready:
227
Negative batch ready:
228
Negative batch ready:
229
Negative batch ready:
230
Negative batch ready:
231
Negative batch ready:
232
Negative batch ready:
233
Negative batch ready:
234
Negative batch ready:
235
Negative batch ready:
236
Negative batch ready:
237
Negative batch ready:
238
Negative batch ready:
239
Negative batch ready:
240
Negative batch ready:
241
Negative batch ready:
242
Negative batch ready:
243
Negative batch ready:
244
Negative batch ready:
245
Negative batch ready:
246
Negative batch ready:
247
Negative batch ready:
248
Negative batch ready:
249
Negative batch ready:
250
Negative batch ready:
251
Negative batch ready:
252
Negative batch ready:
253
Negative batch ready:
254
Negative batch ready:
255
Negative batch ready:
256
Negative batch ready:
257
Negative batch ready:
258
Negative batch ready:
259
Negative batch ready:
260
Negative batch ready:
261
Negative batch ready:
262
Negative batch ready:
263
Negative batch ready:
264
Negative batch ready:
265
Negative batch ready:
266
Negative batch ready:
267
Negative batch ready:
268
Negative batch ready:
269
Negative batch ready:
270
Negative batch ready:
271
Negative batch ready:
272
Negative batch ready:
273
Negative batch ready:
274
Negative batch ready:
275
Negative batch ready:
276
Negative batch ready:
277
Negative batch ready:
278
Negative batch ready:
279
Negative batch ready:
280
Negative batch ready:
281
Negative batch ready:
282
Negative batch ready:
283
Negative batch ready:
284
Negative batch ready:
285
Negative batch ready:
286
Negative batch ready:
287
Negative batch ready:
288
Negative batch ready:
289
Negative batch ready:
290
Negative batch ready:
291
Negative batch ready:
292
Negative batch ready:
293
Negative batch ready:
294
Negative batch ready:
295
Negative batch ready:
296
Negative batch ready:
297
ATTENTION NEGATIVE REWARD 10
Negative batch ready:
298
Negative batch ready:
299
Negative batch ready:
300
Negative batch ready:
301
Negative batch ready:
302
ATTENTION NEGATIVE REWARD 10
Negative batch ready:
303
Negative batch ready:
304
Negative batch ready:
305
Negative batch ready:
306
Negative batch ready:
307
ATTENTION NEGATIVE REWARD 10
Negative batch ready:
308
Negative batch ready:
309
Negative batch ready:
310
Negative batch ready:
311
Negative batch ready:
312
Negative batch ready:
313
Negative batch ready:
314
Negative batch ready:
315
Negative batch ready:
316
Negative batch ready:
317
Negative batch ready:
318
Negative batch ready:
319
Negative batch ready:
320
ATTENTION NEGATIVE REWARD 10
Negative batch ready:
321
Negative batch ready:
322
Negative batch ready:
323
Negative batch ready:
324
Negative batch ready:
325
Negative batch ready:
326
ATTENTION NEGATIVE REWARD 10
Negative batch ready:
327
Negative batch ready:
328
Negative batch ready:
329
Negative batch ready:
330
Negative batch ready:
331
ATTENTION NEGATIVE REWARD 10
Negative batch ready:
332
Negative batch ready:
333
Negative batch ready:
334
Negative batch ready:
335
Negative batch ready:
336
Negative batch ready:
337
ATTENTION NEGATIVE REWARD 10
Negative batch ready:
338
Negative batch ready:
339
Negative batch ready:
340
Negative batch ready:
341
Negative batch ready:
342
ATTENTION NEGATIVE REWARD 10
Negative batch ready:
343
Negative batch ready:
344
Negative batch ready:
345
Negative batch ready:
346
Negative batch ready:
347
Negative batch ready:
348
Negative batch ready:
349
Negative batch ready:
350
Negative batch ready:
351
Negative batch ready:
352
Negative batch ready:
353
Negative batch ready:
354
Negative batch ready:
355
Negative batch ready:
356
Negative batch ready:
357
Negative batch ready:
358
Negative batch ready:
359
Negative batch ready:
360
Negative batch ready:
361
Negative batch ready:
362
Negative batch ready:
363
Negative batch ready:
364
Negative batch ready:
365
Negative batch ready:
366
ATTENTION NEGATIVE REWARD 10
Negative batch ready:
367
Negative batch ready:
368
Negative batch ready:
369
Negative batch ready:
370
ATTENTION NEGATIVE REWARD 10
Negative batch ready:
371
Negative batch ready:
372
Negative batch ready:
373
Negative batch ready:
374
Negative batch ready:
375
Negative batch ready:
376
Negative batch ready:
377
Negative batch ready:
378
Negative batch ready:
379
Negative batch ready:
380
Negative batch ready:
381
Negative batch ready:
382
Negative batch ready:
383
Negative batch ready:
384
Negative batch ready:
385
Negative batch ready:
386
Negative batch ready:
387
Negative batch ready:
388
Negative batch ready:
389
Negative batch ready:
390
Negative batch ready:
391
Negative batch ready:
392
Negative batch ready:
393
Negative batch ready:
394
ATTENTION NEGATIVE REWARD 10
Negative batch ready:
395
Negative batch ready:
396
Negative batch ready:
397
Negative batch ready:
398
Negative batch ready:
399
Negative batch ready:
400
Negative batch ready:
401
Negative batch ready:
402
Negative batch ready:
403
Negative batch ready:
404
Negative batch ready:
405
Negative batch ready:
406
Negative batch ready:
407
Negative batch ready:
408
Negative batch ready:
409
Negative batch ready:
410
Negative batch ready:
411
Negative batch ready:
412
Negative batch ready:
413
Negative batch ready:
414
Negative batch ready:
415
Negative batch ready:
416
Negative batch ready:
417
Negative batch ready:
418
Negative batch ready:
419
Negative batch ready:
420
Negative batch ready:
421
Negative batch ready:
422
Negative batch ready:
423
Negative batch ready:
424
Negative batch ready:
425
Negative batch ready:
426
Negative batch ready:
427
Negative batch ready:
428
Negative batch ready:
429
Negative batch ready:
430
Negative batch ready:
431
Negative batch ready:
432
Negative batch ready:
433
Negative batch ready:
434
Negative batch ready:
435
Negative batch ready:
436
Negative batch ready:
437
Negative batch ready:
438
Negative batch ready:
439
Negative batch ready:
440
Negative batch ready:
441
Negative batch ready:
442
Negative batch ready:
443
Negative batch ready:
444
Negative batch ready:
445
Negative batch ready:
446
Negative batch ready:
447
Negative batch ready:
448
Negative batch ready:
449
Negative batch ready:
450
Negative batch ready:
451
Negative batch ready:
452
Negative batch ready:
453
Negative batch ready:
454
Negative batch ready:
455
Negative batch ready:
456
Negative batch ready:
457
Negative batch ready:
458
Negative batch ready:
459
Negative batch ready:
460
Negative batch ready:
461
Negative batch ready:
462
Negative batch ready:
463
Negative batch ready:
464
Negative batch ready:
465
Negative batch ready:
466
Negative batch ready:
467
Negative batch ready:
468
Negative batch ready:
469
Negative batch ready:
470
Negative batch ready:
471
Negative batch ready:
472
Negative batch ready:
473
Negative batch ready:
474
Negative batch ready:
475
Negative batch ready:
476
Negative batch ready:
477
Negative batch ready:
478
Negative batch ready:
479
Negative batch ready:
480
Negative batch ready:
481
Negative batch ready:
482
Negative batch ready:
483
Negative batch ready:
484
Negative batch ready:
485
Negative batch ready:
486
Negative batch ready:
487
Negative batch ready:
488
Negative batch ready:
489
Negative batch ready:
490
Negative batch ready:
491
Negative batch ready:
492
Negative batch ready:
493
Negative batch ready:
494
Negative batch ready:
495
Negative batch ready:
496
Negative batch ready:
497
Negative batch ready:
498
Negative batch ready:
499
Negative batch ready:
500
Negative batch ready:
501
Negative batch ready:
502
Negative batch ready:
503
Negative batch ready:
504
Negative batch ready:
505
Negative batch ready:
506
Negative batch ready:
507
Negative batch ready:
508
Negative batch ready:
509
Negative batch ready:
510
Negative batch ready:
511
Negative batch ready:
512
Negative batch ready:
513
Negative batch ready:
514
Negative batch ready:
515
Negative batch ready:
516
Negative batch ready:
517
Negative batch ready:
518
Negative batch ready:
519
Negative batch ready:
520
Negative batch ready:
521
Negative batch ready:
522
Negative batch ready:
523
Negative batch ready:
524
Negative batch ready:
525
Negative batch ready:
526
Negative batch ready:
527
Negative batch ready:
528
Negative batch ready:
529
Negative batch ready:
530
Negative batch ready:
531
Negative batch ready:
532
Negative batch ready:
533
Negative batch ready:
534
Negative batch ready:
535
Negative batch ready:
536
Negative batch ready:
537
Negative batch ready:
538
Negative batch ready:
539
Negative batch ready:
540
Negative batch ready:
541
Negative batch ready:
542
Negative batch ready:
543
Negative batch ready:
544
Negative batch ready:
545
Negative batch ready:
546
Negative batch ready:
547
Negative batch ready:
548
Negative batch ready:
549
Negative batch ready:
550
Negative batch ready:
551
Negative batch ready:
552
Negative batch ready:
553
Negative batch ready:
554
Negative batch ready:
555
Negative batch ready:
556
Negative batch ready:
557
Negative batch ready:
558
Negative batch ready:
559
Negative batch ready:
560
Negative batch ready:
561
Negative batch ready:
562
Negative batch ready:
563
Negative batch ready:
564
Negative batch ready:
565
Negative batch ready:
566
Negative batch ready:
567
Negative batch ready:
568
Negative batch ready:
569
Negative batch ready:
570
Negative batch ready:
571
Negative batch ready:
572
Negative batch ready:
573
Negative batch ready:
574
Negative batch ready:
575
Negative batch ready:
576
Negative batch ready:
577
Negative batch ready:
578
Negative batch ready:
579
Negative batch ready:
580
Negative batch ready:
581
Negative batch ready:
582
Negative batch ready:
583
Negative batch ready:
584
Negative batch ready:
585
Negative batch ready:
586
Negative batch ready:
587
Negative batch ready:
588
Negative batch ready:
589
Negative batch ready:
590
Negative batch ready:
591
Negative batch ready:
592
Negative batch ready:
593
Negative batch ready:
594
Negative batch ready:
595
Negative batch ready:
596
Negative batch ready:
597
Negative batch ready:
598
Negative batch ready:
599
Negative batch ready:
600
Negative batch ready:
601
Negative batch ready:
602
Negative batch ready:
603
Negative batch ready:
604
Negative batch ready:
605
Negative batch ready:
606
Negative batch ready:
607
Negative batch ready:
608
Negative batch ready:
609
Negative batch ready:
610
Negative batch ready:
611
Negative batch ready:
612
Negative batch ready:
613
Negative batch ready:
614
Negative batch ready:
615
Negative batch ready:
616
Negative batch ready:
617
Negative batch ready:
618
Negative batch ready:
619
Negative batch ready:
620
Negative batch ready:
621
Negative batch ready:
622
Negative batch ready:
623
Negative batch ready:
624
Negative batch ready:
625
Negative batch ready:
626
Negative batch ready:
627
Negative batch ready:
628
Negative batch ready:
629
Negative batch ready:
630
Negative batch ready:
631
Negative batch ready:
632
Negative batch ready:
633
Negative batch ready:
634
Negative batch ready:
635
Negative batch ready:
636
Negative batch ready:
637
Negative batch ready:
638
Negative batch ready:
639
Negative batch ready:
640
Negative batch ready:
641
Negative batch ready:
642
Negative batch ready:
643
Negative batch ready:
644
Negative batch ready:
645
Negative batch ready:
646
Negative batch ready:
647
Negative batch ready:
648
Negative batch ready:
649
Negative batch ready:
650
Negative batch ready:
651
Negative batch ready:
652
Negative batch ready:
653
Negative batch ready:
654
Negative batch ready:
655
Negative batch ready:
656
Negative batch ready:
657
Negative batch ready:
658
Negative batch ready:
659
Negative batch ready:
660
Negative batch ready:
661
Negative batch ready:
662
Negative batch ready:
663
Negative batch ready:
664
Negative batch ready:
665
Negative batch ready:
666
Negative batch ready:
667
Negative batch ready:
668
Negative batch ready:
669
Negative batch ready:
670
Negative batch ready:
671
Negative batch ready:
672
Negative batch ready:
673
Negative batch ready:
674
Negative batch ready:
675
Negative batch ready:
676
Negative batch ready:
677
Negative batch ready:
678
Negative batch ready:
679
Negative batch ready:
680
Negative batch ready:
681
Negative batch ready:
682
Negative batch ready:
683
Negative batch ready:
684
Negative batch ready:
685
Negative batch ready:
686
Negative batch ready:
687
Negative batch ready:
688
Negative batch ready:
689
Negative batch ready:
690
Negative batch ready:
691
Negative batch ready:
692
Negative batch ready:
693
Negative batch ready:
694
Negative batch ready:
695
Negative batch ready:
696
Negative batch ready:
697
Negative batch ready:
698
Negative batch ready:
699
Negative batch ready:
700
Negative batch ready:
701
Negative batch ready:
702
Negative batch ready:
703
Negative batch ready:
704
Negative batch ready:
705
Negative batch ready:
706
Negative batch ready:
707
Negative batch ready:
708
Negative batch ready:
709
Negative batch ready:
710
Negative batch ready:
711
Negative batch ready:
712
Negative batch ready:
713
Negative batch ready:
714
Negative batch ready:
715
Negative batch ready:
716
Negative batch ready:
717
Negative batch ready:
718
Negative batch ready:
719
Negative batch ready:
720
Negative batch ready:
721
Negative batch ready:
722
Negative batch ready:
723
Negative batch ready:
724
Negative batch ready:
725
Negative batch ready:
726
Negative batch ready:
727
Negative batch ready:
728
Negative batch ready:
729
Negative batch ready:
730
Negative batch ready:
731
Negative batch ready:
732
Negative batch ready:
733
Negative batch ready:
734
Negative batch ready:
735
Negative batch ready:
736
Negative batch ready:
737
Negative batch ready:
738
Negative batch ready:
739
Negative batch ready:
740
Negative batch ready:
741
Negative batch ready:
742
Negative batch ready:
743
Negative batch ready:
744
Negative batch ready:
745
Negative batch ready:
746
Negative batch ready:
747
Negative batch ready:
748
Negative batch ready:
749
Negative batch ready:
750
Negative batch ready:
751
Negative batch ready:
752
Negative batch ready:
753
Negative batch ready:
754
Negative batch ready:
755
Negative batch ready:
756
Negative batch ready:
757
Negative batch ready:
758
Negative batch ready:
759
Negative batch ready:
760
Negative batch ready:
761
Negative batch ready:
762
Negative batch ready:
763
Negative batch ready:
764
Negative batch ready:
765
Negative batch ready:
766
Negative batch ready:
767
Negative batch ready:
768
Negative batch ready:
769
Negative batch ready:
770
Negative batch ready:
771
Negative batch ready:
772
Negative batch ready:
773
Negative batch ready:
774
Negative batch ready:
775
Negative batch ready:
776
Negative batch ready:
777
Negative batch ready:
778
Negative batch ready:
779
Negative batch ready:
780
Negative batch ready:
781
Negative batch ready:
782
Negative batch ready:
783
Negative batch ready:
784
Negative batch ready:
785
Negative batch ready:
786
Negative batch ready:
787
Negative batch ready:
788
Negative batch ready:
789
Negative batch ready:
790
Negative batch ready:
791
Negative batch ready:
792
Negative batch ready:
793
Negative batch ready:
794
Negative batch ready:
795
Negative batch ready:
796
Negative batch ready:
797
Negative batch ready:
798
Negative batch ready:
799
Negative batch ready:
800
Negative batch ready:
801
Negative batch ready:
802
Negative batch ready:
803
Negative batch ready:
804
Negative batch ready:
805
Negative batch ready:
806
Negative batch ready:
807
Negative batch ready:


# Notebook from maccardoso/IBIP19
Path: pages/proteogenomics/Variation Analysis.ipynb

# Variation Analysis

In the [_Novel Peptides_](Novel Peptides.ipynb) chapter, novel peptides were identified by Johansson _et al._ [(2)](https://gtpb.github.io/IBIP19/pages/proteogenomics/references) using a proteogenomic search strategy. In addition, this search allowed identifying peptides presenting single amino acid variants (SAAV).

## Libraries

We will need the following libraries, please make sure that they are installed._____no_output_____

library(conflicted)
library(tidyr)
library(dplyr)
library(ggplot2)
library(scico)
library(gamlss)

theme_set(theme_bw(base_size = 11))

conflict_prefer("filter", "dplyr")
conflict_prefer("select", "dplyr")Loading required package: splines

Loading required package: gamlss.data

Attaching package: 'gamlss.data'

The following object is masked from 'package:datasets':

    sleep

Loading required package: gamlss.dist

Loading required package: MASS

Attaching package: 'MASS'

The following object is masked from 'package:dplyr':

    select

Loading required package: nlme

Attaching package: 'nlme'

The following object is masked from 'package:dplyr':

    collapse

Loading required package: parallel

 **********   GAMLSS Version 5.1-4  **********

For more on GAMLSS look at http://www.gamlss.org/

Type gamlssNews() to see new features/changes/bug fixes.

[conflicted] Will prefer dplyr::filter over any other package

[conflicted] Will prefer dplyr::select over any other package

## Genotype - peptide abundance relationship

Figure 7h displays the intensity of peptides produced by the reference and alternative alleles of corresponding SNPs for each genotype.

![Figure_7h](resources/images/Fig7h.png?raw=true "Johansson et al. Fig 7h")

> Figure 7h in Johansson _et al._ [(2)](https://gtpb.github.io/IBIP19/pages/proteogenomics/references).

##### [❔](https://github.com/GTPB/IBIP19/blob/master/pages/proteogenomics/answers.md#if-we-assume-a-linear-relationship-between-number-of-alleles-and-peptide-abundance-what-should-be-the-peptide-distribution-for-each-genotype) _If we assume a linear relationship between number of alleles and peptide abundance, what should be the peptide distribution for each genotype?_

Note that the genotyping data corresponding to the peptide intensities were not made available, but more examples can be found in [Supplementary Figure 15](../resources/Johansson_et_al_breast_cancer_quantitative_proteome_and_proteogenomic_landscape/supplementary_information.pdf).

##### 💬 Do the results presented follow a linear trend? What can affect the linearity of the relationship between number of alleles and intensity distribution?

## Abundance of variant peptides relative to the proteins

In [Supplementary Table 1](../resources/Johansson_et_al_breast_cancer_quantitative_proteome_and_proteogenomic_landscape) and [Supplementary Table 6](../resources/Johansson_et_al_breast_cancer_quantitative_proteome_and_proteogenomic_landscape), the authors provide the results of their proteogenomic analysis for genes and for peptides carrying alternative alleles of SAAVs, respectively. In addition, [Supplementary Table 1](../resources/Johansson_et_al_breast_cancer_quantitative_proteome_and_proteogenomic_landscape) contains experimental design and sample characterization information. The tables were extracted to an R-friendly text format for this tutorial, and are available in [resources/data/tumor.gz](resources/data/tumor.gz), [resources/data/proteins.gz](resources/data/proteins.gz), and [resources/data/saav.gz](resources/data/saav.gz).

##### 👨💻 Load the data in R as in the code below._____no_output_____

tumorDF <- read.table(
    file = "resources/data/tumor.gz",
    header = T,
    sep = "\t",
    comment.char = "",
    quote = "",
    stringsAsFactors = F
)

genesDF <- read.table(
    file = "resources/data/genes.gz",
    header = T,
    sep = "\t",
    comment.char = "",
    quote = "",
    stringsAsFactors = F
) %>%
    select(
        -contains("nspsm")
    )

saavDF <- read.table(
    file = "resources/data/saav.gz",
    header = T,
    sep = "\t",
    comment.char = "",
    quote = "",
    stringsAsFactors = F
)

# Set colors for each tumor type

tumorSubtypes <- unique(tumorDF$PAM50_subtype)
tumorColors <- scico(
    n = length(tumorSubtypes),
    palette = "hawaii"
)
names(tumorColors) <- tumorSubtypes
tumorDF$color <- tumorColors[tumorDF$PAM50_subtype]
_____no_output_____

Note that for the sake of speed, the last columns of the genes table were skipped.

##### [❔](https://github.com/GTPB/IBIP19/blob/master/pages/proteogenomics/answers.md#how-do-we-need-to-transform-the-tables-to-compare-saav-peptide-level-intensities-to-gene-level-intensities) _How do we need to transform the tables to compare SAAV peptide-level intensities to gene-level intensities?_

For the sake of time, this data transformation was run for you already. The proposed solution is available in [scripts/merge_table_1_6.R](resources/data/saav.gz), and the reformated SAAV peptides table is available in [resources/data/table16.gz](resources/data/table16.gz). Note that in this table one row represents a unique _{peptide sequence, gene accession}_ pair.

##### [❔](https://github.com/GTPB/IBIP19/blob/master/pages/proteogenomics/answers.md#why-can-there-be-multiple-peptide-per-gene-and-gene-per-peptide-is-it-correct-to-represent-peptides-by-their-sequence) _Why can there be multiple peptide per gene, and gene per peptide? Is it correct to represent peptides by their sequence?_

In the following, we want to compare the abundances associated to SAAV peptides to the abundances aggregated at the gene-level. For each tumor, we will therefore only retain quantified _{peptide, gene}_ pairs.

##### 👨💻 Load the reformated SAAV peptides table._____no_output_____

# Load the table

saavDF <- read.table(
    file = "resources/data/table16.gz",
    header = T,
    sep = "\t",
    comment.char = "",
    quote = "",
    stringsAsFactors = F
)

# Place holder for column names

tumorPeptideColumns <- paste0(tumorDF$tumor_id, "_saavPeptide")
tumorGeneColumns <- paste0(tumorDF$tumor_id, "_gene")
tumorColumns <- c(tumorPeptideColumns, tumorGeneColumns)

# For each tumor, keep only pairs of gene and peptide pairs

for (tumor in tumorDF$tumor_id) {

    peptideColumn <- paste0(tumor, "_saavPeptide")
    geneColumn <- paste0(tumor, "_gene")

    saavDF[[peptideColumn]][is.na(saavDF[[geneColumn]])] <- NA
    saavDF[[geneColumn]][is.na(saavDF[[peptideColumn]])] <- NA

}

# Remove lines with no quantification value

nValid <- rowSums(!is.na(saavDF[, tumorColumns]))
saavDF <- saavDF[nValid > 0,]_____no_output_____

##### 👨💻 Plot the intensity distributions as violin and box plots for the different tumors_____no_output_____

# Transform the saav data frame from wide to long format using tidyr

intensitiesDF <- saavDF %>%
    gather(
        !!tumorColumns,
        key = "sample",
        value = "intensity"
    ) %>%
    filter(
        !is.na(intensity) & !is.infinite(intensity)
    ) %>%
    separate(
        col = sample,
        into = c("tumor", "species"),
        sep = "_"
    )

# Set levels for genes and peptides

intensitiesDF$species <- factor(
    intensitiesDF$species,
    levels = c("gene", "saavPeptide")
)
levels(intensitiesDF$species) <- c("Gene", "SAAV Peptide")

# Build the plot

ggplot(
    data = intensitiesDF
) +
    geom_violin(
        mapping = aes(
            x = tumor,
            y = intensity,
            col = tumor
        ),
        fill = NA
    ) +
    geom_boxplot(
        mapping = aes(
            x = tumor,
            y = intensity,
            col = tumor
        ),
        fill = NA,
        outlier.shape = NA
    ) +
    facet_grid(
        species ~ .
    ) +
    scale_fill_manual(
        values = scico(
            n = length(unique(intensitiesDF$tumor)),
            palette = "batlow",
            begin = 0.25,
            end = 0.75
        )
    ) +
    scale_color_manual(
        values = scico(
            n = length(unique(intensitiesDF$tumor)),
            palette = "batlow",
            begin = 0.25,
            end = 0.75
        )
    ) +
    scale_y_continuous(
        name = "Intensity"
    ) +
    theme(
        legend.position = "none",
        axis.title.x = element_blank(),
        axis.text.x = element_text(
            angle = 90,
            hjust = 0,
            vjust = 0.5

        ),
        panel.grid.major.x = element_blank(),
        panel.grid.minor.x = element_blank()
    )_____no_output_____

##### [❔](https://github.com/GTPB/IBIP19/blob/master/pages/proteogenomics/answers.md#why-are-all-intensities-at-the-bottom-how-can-we-transform-the-data-to-better-visualize-these-distributions) _Why are all intensities at the bottom? How can we transform the data to better visualize these distributions?_

As you can see, the gene-level intensities are centered around one, and not for the peptide-level. You might have noticed in the paper that the gene-level data as provided in the supplementary information are centered, but this was not done for the peptides.

##### 💬 In your opinion why were the protein data centered? How would you recommend normalizing these data? Why?

Here, we hypothesize that SAAV peptides are sampled from these genes and should therefore present the same abundance distribution. Data acquisition and processing artefacts can influence the centering and scaling. To correct for this, we will normalize the abundances from all tumors using centile-based Z-scores._____no_output_____

# Placeholder for the z-transformed columns

zTumorColumns <- paste0("z_", tumorColumns)
zTumorPeptideColumns <- paste0("z_", tumorPeptideColumns)
zTumorGeneColumns <- paste0("z_", tumorGeneColumns)

# Iterate through the tumor columns

for (column in tumorColumns) {

    # Create a column name for the z-score transformed variables

    zColumn <- paste0("z_", column)

    # Create a data frame with only the columns of interest and filter out NA, zero, and infinite values

    trainingDF <- saavDF %>%
        select(
            gene, peptide_sequence, !!column
        ) %>%
        filter(
            !is.na(!!sym(column)) & !is.infinite(!!sym(column)) & !!sym(column) > 0
        )

    # Train the model

    trainingDF$x <- 0 # Note that it is possible to include covariates in the model instead of using a constant here

    model <- gamlss(
        formula = as.formula(paste0(column, " ~ x")),
        family = LOGNO,
        data = trainingDF,

    )

    # Compute the z-scores

    trainingDF[[zColumn]] <- centiles.pred(
        obj = model,
        xname = "x",
        xvalues = trainingDF$x,
        yval = trainingDF[[column]],
        type = "z-scores"
    )

    # Extract the transformed values

    trainingDF <- trainingDF %>%
        filter(
            !is.infinite(!!sym(zColumn))
        ) %>%
        select(
            gene, peptide_sequence, !!zColumn
        )

    # Merge with the saav data frame

    saavDF <- saavDF %>%
        left_join(
            trainingDF,
            by = c("gene", "peptide_sequence")
        )

}

# For each tumor, keep only pairs of gene and peptide pairs

for (tumor in tumorDF$tumor_id) {

    peptideColumn <- paste0("z_", tumor, "_saavPeptide")
    geneColumn <- paste0("z_", tumor, "_gene")

    saavDF[[peptideColumn]][is.na(saavDF[[geneColumn]])] <- NA
    saavDF[[geneColumn]][is.na(saavDF[[peptideColumn]])] <- NA

}

# Remove lines with no normalized quantification value

nValid <- rowSums(!is.na(saavDF[, zTumorColumns]))
saavDF <- saavDF[nValid > 0,]GAMLSS-RS iteration 1: Global Deviance = 2027.455
GAMLSS-RS iteration 2: Global Deviance = 2027.455
GAMLSS-RS iteration 1: Global Deviance = 1300.237
GAMLSS-RS iteration 2: Global Deviance = 1300.237
GAMLSS-RS iteration 1: Global Deviance = 1880.204
GAMLSS-RS iteration 2: Global Deviance = 1880.204
GAMLSS-RS iteration 1: Global Deviance = 1857.933
GAMLSS-RS iteration 2: Global Deviance = 1857.933
GAMLSS-RS iteration 1: Global Deviance = 2045.205
GAMLSS-RS iteration 2: Global Deviance = 2045.205
GAMLSS-RS iteration 1: Global Deviance = 1966.413
GAMLSS-RS iteration 2: Global Deviance = 1966.413
GAMLSS-RS iteration 1: Global Deviance = 1566.112
GAMLSS-RS iteration 2: Global Deviance = 1566.112
GAMLSS-RS iteration 1: Global Deviance = 1491.397
GAMLSS-RS iteration 2: Global Deviance = 1491.397
GAMLSS-RS iteration 1: Global Deviance = 1600.611
GAMLSS-RS iteration 2: Global Deviance = 1600.611
GAMLSS-RS iteration 1: Global Deviance = 1454.124
GAMLSS-RS iteration 2: Global Deviance = 1454.124
GAMLSS-RS iteration 1: Global Deviance = 1391.497
GAMLSS-RS iteration 2: Global Deviance = 1391.497
GAMLSS-RS iteration 1: Global Deviance = 1499.707
GAMLSS-RS iteration 2: Global Deviance = 1499.707
GAMLSS-RS iteration 1: Global Deviance = 1727.284
GAMLSS-RS iteration 2: Global Deviance = 1727.284
GAMLSS-RS iteration 1: Global Deviance = 1354.763
GAMLSS-RS iteration 2: Global Deviance = 1354.763
GAMLSS-RS iteration 1: Global Deviance = 1381.672
GAMLSS-RS iteration 2: Global Deviance = 1381.672
GAMLSS-RS iteration 1: Global Deviance = 1513.597
GAMLSS-RS iteration 2: Global Deviance = 1513.597
GAMLSS-RS iteration 1: Global Deviance = 1508.408
GAMLSS-RS iteration 2: Global Deviance = 1508.408
GAMLSS-RS iteration 1: Global Deviance = 1501.329
GAMLSS-RS iteration 2: Global Deviance = 1501.329
GAMLSS-RS iteration 1: Global Deviance = 3247.767
GAMLSS-RS iteration 2: Global Deviance = 3247.767
GAMLSS-RS iteration 1: Global Deviance = 3154.528
GAMLSS-RS iteration 2: Global Deviance = 3154.528
GAMLSS-RS iteration 1: Global Deviance = 2788.806
GAMLSS-RS iteration 2: Global Deviance = 2788.806
GAMLSS-RS iteration 1: Global Deviance = 3722.134
GAMLSS-RS iteration 2: Global Deviance = 3722.134
GAMLSS-RS iteration 1: Global Deviance = 3065.699
GAMLSS-RS iteration 2: Global Deviance = 3065.699
GAMLSS-RS iteration 1: Global Deviance = 3226.883
GAMLSS-RS iteration 2: Global Deviance = 3226.883
GAMLSS-RS iteration 1: Global Deviance = 2334.491
GAMLSS-RS iteration 2: Global Deviance = 2334.491
GAMLSS-RS iteration 1: Global Deviance = 2561.947
GAMLSS-RS iteration 2: Global Deviance = 2561.947
GAMLSS-RS iteration 1: Global Deviance = 3247.387
GAMLSS-RS iteration 2: Global Deviance = 3247.387
GAMLSS-RS iteration 1: Global Deviance = 3081.291
GAMLSS-RS iteration 2: Global Deviance = 3081.291
GAMLSS-RS iteration 1: Global Deviance = 2890.455
GAMLSS-RS iteration 2: Global Deviance = 2890.455
GAMLSS-RS iteration 1: Global Deviance = 3500.093
GAMLSS-RS iteration 2: Global Deviance = 3500.093
GAMLSS-RS iteration 1: Global Deviance = 3576.319
GAMLSS-RS iteration 2: Global Deviance = 3576.319
GAMLSS-RS iteration 1: Global Deviance = 3403.956
GAMLSS-RS iteration 2: Global Deviance = 3403.956
GAMLSS-RS iteration 1: Global Deviance = 4171.327
GAMLSS-RS iteration 2: Global Deviance = 4171.327
GAMLSS-RS iteration 1: Global Deviance = 3033.924
GAMLSS-RS iteration 2: Global Deviance = 3033.924
GAMLSS-RS iteration 1: Global Deviance = 2499.132
GAMLSS-RS iteration 2: Global Deviance = 2499.132
GAMLSS-RS iteration 1: Global Deviance = 3417.91
GAMLSS-RS iteration 2: Global Deviance = 3417.91
GAMLSS-RS iteration 1: Global Deviance = 3859.863
GAMLSS-RS iteration 2: Global Deviance = 3859.863
GAMLSS-RS iteration 1: Global Deviance = 2884.593
GAMLSS-RS iteration 2: Global Deviance = 2884.593
GAMLSS-RS iteration 1: Global Deviance = 3195.717
GAMLSS-RS iteration 2: Global Deviance = 3195.717
GAMLSS-RS iteration 1: Global Deviance = 3481.365
GAMLSS-RS iteration 2: Global Deviance = 3481.365
GAMLSS-RS iteration 1: Global Deviance = 3416.908
GAMLSS-RS iteration 2: Global Deviance = 3416.908
GAMLSS-RS iteration 1: Global Deviance = 4189.568
GAMLSS-RS iteration 2: Global Deviance = 4189.568
GAMLSS-RS iteration 1: Global Deviance = 2650.29
GAMLSS-RS iteration 2: Global Deviance = 2650.29
GAMLSS-RS iteration 1: Global Deviance = 2568.541
GAMLSS-RS iteration 2: Global Deviance = 2568.541
GAMLSS-RS iteration 1: Global Deviance = 2926.19
GAMLSS-RS iteration 2: Global Deviance = 2926.19
GAMLSS-RS iteration 1: Global Deviance = 693.3155
GAMLSS-RS iteration 2: Global Deviance = 693.3155
GAMLSS-RS iteration 1: Global Deviance = 1076.556
GAMLSS-RS iteration 2: Global Deviance = 1076.556
GAMLSS-RS iteration 1: Global Deviance = 195.5654
GAMLSS-RS iteration 2: Global Deviance = 195.5654
GAMLSS-RS iteration 1: Global Deviance = 865.6185
GAMLSS-RS iteration 2: Global Deviance = 865.6185
GAMLSS-RS iteration 1: Global Deviance = 1095.07
GAMLSS-RS iteration 2: Global Deviance = 1095.07
GAMLSS-RS iteration 1: Global Deviance = 126.089
GAMLSS-RS iteration 2: Global Deviance = 126.089
GAMLSS-RS iteration 1: Global Deviance = 611.7388
GAMLSS-RS iteration 2: Global Deviance = 611.7388
GAMLSS-RS iteration 1: Global Deviance = 466.5559
GAMLSS-RS iteration 2: Global Deviance = 466.5559
GAMLSS-RS iteration 1: Global Deviance = 921.7777
GAMLSS-RS iteration 2: Global Deviance = 921.7777
GAMLSS-RS iteration 1: Global Deviance = 711.1848
GAMLSS-RS iteration 2: Global Deviance = 711.1848
GAMLSS-RS iteration 1: Global Deviance = 489.9322
GAMLSS-RS iteration 2: Global Deviance = 489.9322
GAMLSS-RS iteration 1: Global Deviance = 525.162
GAMLSS-RS iteration 2: Global Deviance = 525.162
GAMLSS-RS iteration 1: Global Deviance = 637.9658
GAMLSS-RS iteration 2: Global Deviance = 637.9658
GAMLSS-RS iteration 1: Global Deviance = 574.2767
GAMLSS-RS iteration 2: Global Deviance = 574.2767
GAMLSS-RS iteration 1: Global Deviance = 521.4437
GAMLSS-RS iteration 2: Global Deviance = 521.4437
GAMLSS-RS iteration 1: Global Deviance = 759.0655
GAMLSS-RS iteration 2: Global Deviance = 759.0655
GAMLSS-RS iteration 1: Global Deviance = 826.0473
GAMLSS-RS iteration 2: Global Deviance = 826.0473
GAMLSS-RS iteration 1: Global Deviance = 650.8084
GAMLSS-RS iteration 2: Global Deviance = 650.8084
GAMLSS-RS iteration 1: Global Deviance = 1076.141
GAMLSS-RS iteration 2: Global Deviance = 1076.141
GAMLSS-RS iteration 1: Global Deviance = 346.3175
GAMLSS-RS iteration 2: Global Deviance = 346.3175
GAMLSS-RS iteration 1: Global Deviance = 2112.151
GAMLSS-RS iteration 2: Global Deviance = 2112.151
GAMLSS-RS iteration 1: Global Deviance = 1758.988
GAMLSS-RS iteration 2: Global Deviance = 1758.988
GAMLSS-RS iteration 1: Global Deviance = 1404.673
GAMLSS-RS iteration 2: Global Deviance = 1404.673
GAMLSS-RS iteration 1: Global Deviance = 827.4501
GAMLSS-RS iteration 2: Global Deviance = 827.4501
GAMLSS-RS iteration 1: Global Deviance = 746.5591
GAMLSS-RS iteration 2: Global Deviance = 746.5591
GAMLSS-RS iteration 1: Global Deviance = 1074.38
GAMLSS-RS iteration 2: Global Deviance = 1074.38
GAMLSS-RS iteration 1: Global Deviance = 1323.115
GAMLSS-RS iteration 2: Global Deviance = 1323.115
GAMLSS-RS iteration 1: Global Deviance = 1436.366
GAMLSS-RS iteration 2: Global Deviance = 1436.366
GAMLSS-RS iteration 1: Global Deviance = 1461.965
GAMLSS-RS iteration 2: Global Deviance = 1461.965
GAMLSS-RS iteration 1: Global Deviance = 924.9015
GAMLSS-RS iteration 2: Global Deviance = 924.9015
GAMLSS-RS iteration 1: Global Deviance = 1244.586
GAMLSS-RS iteration 2: Global Deviance = 1244.586
GAMLSS-RS iteration 1: Global Deviance = 899.9695
GAMLSS-RS iteration 2: Global Deviance = 899.9695
GAMLSS-RS iteration 1: Global Deviance = 1504.276
GAMLSS-RS iteration 2: Global Deviance = 1504.276
GAMLSS-RS iteration 1: Global Deviance = 1397.485
GAMLSS-RS iteration 2: Global Deviance = 1397.485
GAMLSS-RS iteration 1: Global Deviance = -116.1728
GAMLSS-RS iteration 2: Global Deviance = -116.1728
GAMLSS-RS iteration 1: Global Deviance = 861.8912
GAMLSS-RS iteration 2: Global Deviance = 861.8912
GAMLSS-RS iteration 1: Global Deviance = 1455.807
GAMLSS-RS iteration 2: Global Deviance = 1455.807
GAMLSS-RS iteration 1: Global Deviance = 973.2334
GAMLSS-RS iteration 2: Global Deviance = 973.2334
GAMLSS-RS iteration 1: Global Deviance = 618.1967
GAMLSS-RS iteration 2: Global Deviance = 618.1967
GAMLSS-RS iteration 1: Global Deviance = 416.7871
GAMLSS-RS iteration 2: Global Deviance = 416.7871
GAMLSS-RS iteration 1: Global Deviance = 565.1233
GAMLSS-RS iteration 2: Global Deviance = 565.1233
GAMLSS-RS iteration 1: Global Deviance = 1401.728
GAMLSS-RS iteration 2: Global Deviance = 1401.728
GAMLSS-RS iteration 1: Global Deviance = 982.3325
GAMLSS-RS iteration 2: Global Deviance = 982.3325
GAMLSS-RS iteration 1: Global Deviance = 516.3645
GAMLSS-RS iteration 2: Global Deviance = 516.3645
GAMLSS-RS iteration 1: Global Deviance = 997.388
GAMLSS-RS iteration 2: Global Deviance = 997.388

##### 👨💻  For a tumor and plot the normalized values against the original ones._____no_output_____

# Select a column randomly

column <- sample(
    x = tumorColumns,
    size = 1
)
zColumn <- paste0("z_", column)

# Build the plot

ggplot(
    data = saavDF
) +
    geom_point(
        mapping = aes(
            x = !!sym(column),
            y = !!sym(zColumn)
        ),
        alpha = 0.05
    )Warning message:
"Removed 1426 rows containing missing values (geom_point)."

##### [❔](https://github.com/GTPB/IBIP19/blob/master/pages/proteogenomics/answers.md#why-does-the-curve-have-this-shape) _Why does the curve have this shape?_

##### 👨💻 Plot the distributions like before after normalization._____no_output_____

# Transform the saav data frame from wide to long format using tidyr

intensitiesDF <- saavDF %>%
    select(
        -!!tumorColumns
    ) %>%
    gather(
        !!zTumorColumns,
        key = "sample",
        value = "intensity"
    ) %>%
    filter(
        !is.na(intensity) & !is.infinite(intensity)
    ) %>%
    separate(
        col = sample,
        into = c(NA, "tumor", "species"),
        sep = "_"
    )

# Set levels for the genes and peptides

intensitiesDF$species <- factor(
    intensitiesDF$species,
    levels = c("gene", "saavPeptide")
)
levels(intensitiesDF$species) <- c("Gene", "SAAV Peptide")

# Build the plot

ggplot(
    data = intensitiesDF
) +
    geom_violin(
        mapping = aes(
            x = tumor,
            y = intensity,
            col = tumor
        ),
        fill = NA
    ) +
    geom_boxplot(
        mapping = aes(
            x = tumor,
            y = intensity,
            col = tumor
        ),
        fill = NA,
        outlier.shape = NA
    ) +
    facet_grid(
        species ~ .
    ) +
    scale_fill_manual(
        values = scico(
            n = length(unique(intensitiesDF$tumor)),
            palette = "batlow",
            begin = 0.25,
            end = 0.75
        )
    ) +
    scale_color_manual(
        values = scico(
            n = length(unique(intensitiesDF$tumor)),
            palette = "batlow",
            begin = 0.25,
            end = 0.75
        )
    ) +
    scale_y_continuous(
        name = "Intensity"
    ) +
    theme(
        legend.position = "none",
        axis.title.x = element_blank(),
        axis.text.x = element_text(
            angle = 90,
            hjust = 0,
            vjust = 0.5

        ),
        panel.grid.major.x = element_blank(),
        panel.grid.minor.x = element_blank()
    )
_____no_output_____

##### 👨💻 Plot the gene- and peptide- level abundances._____no_output_____

# Transform the saav data frame from wide to long format, and separate genes and proteins using tidyr

intensitiesDF <- saavDF %>%
    select(
        -!!tumorColumns
    ) %>%
    gather(
        !!zTumorColumns,
        key = "sample",
        value = "intensity"
    ) %>%
    filter(
        !is.na(intensity) & !is.infinite(intensity)
    ) %>%
    separate(
        col = sample,
        into = c(NA, "tumor", "species"),
        sep = "_"
    ) %>%
    mutate (
        species = ifelse(species == "gene", "gene_abundance", "saav_peptide_abundance")
    ) %>%
    spread(
        key = species,
        value = intensity
    ) %>%
    filter(
        !is.na(saav_peptide_abundance) & !is.na(gene_abundance)
    ) %>%
    arrange(
        abs(saav_peptide_abundance - gene_abundance)
    )

# Run a linear model, for each tumor separately then for all values

model <- lm(
    data = intensitiesDF,
    formula = saav_peptide_abundance ~ gene_abundance
)

summary(model)

# Build the plot

axisLimit <- max(abs(c(intensitiesDF$gene_abundance, intensitiesDF$saav_peptide_abundance)))

ggplot(
    data = intensitiesDF
) +
    geom_point(
        mapping = aes(
            x = gene_abundance,
            y = saav_peptide_abundance,
            col = tumor
        ),
        alpha = 0.2
    ) +
    geom_density2d(
        mapping = aes(
            x = gene_abundance,
            y = saav_peptide_abundance,
        ),
        col = "blue"
    ) +
    geom_smooth(
        mapping = aes(
            x = gene_abundance,
            y = saav_peptide_abundance,
        ),
        method = 'lm',
        formula = y ~ x,
        col = "blue",
        linetype = "dotted"
    ) +
    scale_x_continuous(
        name = "Gene-level Abundances [Z-score]",
        limits = c(-axisLimit, axisLimit)
    ) +
    scale_y_continuous(
        name = "SAAV-peptide-level Abundances [Z-score]",
        limits = c(-axisLimit, axisLimit)
    ) +
    scale_color_manual(
        name = "Tumor Subtype",
        values = scico(
            n = length(unique(intensitiesDF$tumor)),
            palette = "batlow"
        )
    ) +
    theme(
        legend.position = "none"
    )_____no_output_____

##### 💬 What do you think about the SAAV peptide to gene agreement?

In the following, we will inspect the correlation of the profile across tumors for the abundances reported for genes and SAAV peptides. Note that we will use random values to compare our results with what one would expect by chance.

##### 👨💻 Plot a histogram of pairwise correlation between protein and peptide abundances across tumors, compare with random values._____no_output_____

# Select rows where there are at least 8 peptide-protein pairs

saavDF$nPeptideValid <- rowSums(!is.na(saavDF[, zTumorPeptideColumns]))
saavDF$nGeneValid <- rowSums(!is.na(saavDF[, zTumorGeneColumns]))
corDF <- saavDF[saavDF$nPeptideValid >= 8 & saavDF$nGeneValid >= 8,]

# Compute correlation by row

corDF$cor <- NA
corDF$rCor <- NA

for (i in 1:nrow(corDF)) {

    x <- t(corDF[i, zTumorGeneColumns])
    y <- t(corDF[i, zTumorPeptideColumns])

    corDF$cor[i] <- cor(
        x = x,
        y = y,
        use = "pairwise.complete.obs",
        method = "spearman"
    )

    n <- sum(!is.na(x) & !is.na(y))
    x <- rnorm(n)
    y <- rnorm(n)

    corDF$rCor[i] <- cor(
        x = x,
        y = y,
        use = "pairwise.complete.obs",
        method = "spearman"
    )

}

# Transform the example data frame from wide to long format

plotDF <- corDF %>%
    select(
        gene, peptide_sequence, cor, rCor
    ) %>%
    gather(
        cor, rCor,
        key = "data",
        value = "correlation"
    ) %>%
    mutate(
        data = ifelse(data == "cor", "Data", "Random")
    )

# Build plot

ggplot(
    data = plotDF
) +
    geom_histogram(
        mapping = aes(
            x = correlation,
            fill = data
        ),
        alpha = 0.8,
        bins = 20,
        position = "dodge"
    ) +
    scale_x_continuous(
        name = "Correlation [Spearman]"
    ) +
    scale_y_continuous(
        name = "# SAAV Peptide - Protein"
    ) +
    scale_fill_manual(
        values = c("darkblue", "black")
    ) +
    theme(
        legend.position = "top",
        legend.title = element_blank()
    )
_____no_output_____

##### 💬 How do you interpret this distribution? What peptides are likely to provide information on the tumors that is othogonal to the proteome?

##### 👨💻  Sample examples of peptide-protein pairs at the bottom, middle, and top of the distribution, and plot their profiles among tumors._____no_output_____

# Sample

exampleLow <- corDF %>%
    filter(
        cor < -0.5
    ) %>%
    arrange(
        desc(nPeptideValid)
    ) %>%
    slice(
        1
    ) %>%
    mutate(
        correlation_category = "Low"
    )
exampleMiddle <- corDF %>%
    filter(
        cor > -0.2 & cor < 0.2
    ) %>%
    arrange(
        desc(nPeptideValid)
    ) %>%
    slice(
        1
    ) %>%
    mutate(
        correlation_category = "Middle"
    )
exampleHigh <- corDF %>%
    filter(
        cor > 0.8
    ) %>%
    arrange(
        desc(nPeptideValid)
    ) %>%
    slice(
        1
    ) %>%
    mutate(
        correlation_category = "High"
    )

exampleDF <- rbind(
    exampleLow,
    exampleMiddle,
    exampleHigh
) %>%
    mutate(
        correlation_category = factor(
            x = correlation_category,
            levels = c("High", "Middle", "Low")
        )
    )

# Transform the example data frame from wide to long format

plotDF <- exampleDF %>%
    select(
        -!!tumorColumns
    ) %>%
    gather(
        !!zTumorColumns,
        key = "sample",
        value = "intensity"
    ) %>%
    filter(
        !is.na(intensity) & !is.infinite(intensity)
    ) %>%
    separate(
        col = sample,
        into = c(NA, "tumor", "species"),
        sep = "_"
    ) %>%
    mutate(
        species = factor(
            x = ifelse(species == "gene", "Gene", "SAAV Peptide"),
            levels = c("Gene", "SAAV Peptide")
        )
    )

# Order the tumors by category

tumorDF <- tumorDF %>%
    arrange(
        PAM50_subtype, tumor_id
    )

plotDF$tumor <- factor(
    x = plotDF$tumor,
    levels = tumorDF$tumor_id
)

# Build plot

ggplot(
    data = plotDF
) +
    geom_point(
        mapping = aes(
            x = tumor,
            y = intensity,
            col = species
        ),
        alpha = 0.8
    ) +
    facet_grid(
        correlation_category ~ .
    ) +
    scale_y_continuous(
        name = "Intensity [Z-score]"
    ) +
    scale_color_manual(
        values = scico(
            n = 2,
            palette = "cork",
            begin = 0.2,
            end = 0.8,
            direction = -1
        )
    ) +
    theme(
        legend.position = "top",
        legend.title = element_blank(),
        axis.title.x = element_blank(),
        axis.text.x = element_text(
            angle = 90,
            hjust = 0,
            vjust = 0.5,
            color = tumorDF$color
        )
    )_____no_output_____

##### 💬 How do you interpret differences in correlation between abundances reported for genes or SAAV peptides?

In the following, we investigate whether specific classes of variants influence the correlation between SAAV peptide- and gene-level abundances. For this, we query the [Ensembl Variant Effect Predictor (VEP)](ensembl.org/Homo_sapiens/Tools/VEP) using the variant identifiers reported in [Supplementary Table 6](../resources/Johansson_et_al_breast_cancer_quantitative_proteome_and_proteogenomic_landscape). For the sake of time, we will not query VEP as part of this tutorial, this was already done for you, and the VEP results are available in [resources/data/vep.txt](resources/data/vep.txt). Note that only variants with an _rsID_ were used in this analysis.

##### 👨💻 Import the VEP results and merge with the correlation data frame._____no_output_____

# Load the file, select the columns of interest, format the columns and content

vepDF <- read.table(
    file = "resources/data/vep.txt",
    header = T,
    sep = "\t",
    comment.char = "",
    quote = "",
    stringsAsFactors = F
) %>%
    filter(
        Consequence %in% c("missense_variant", "coding_sequence_variant") & BIOTYPE == "protein_coding"
    ) %>%
    select(
        SYMBOL, Uploaded_variation, MutationAssessor_pred, clinvar_clnsig, clinvar_trait, gnomAD_exomes_NFE_AF, LoFtool
    ) %>%
    rename(
        gene_variant = Uploaded_variation,
        gene = SYMBOL,
        functionality_prediction = MutationAssessor_pred,
        clinical_significance = clinvar_clnsig,
        allele_frequency = gnomAD_exomes_NFE_AF,
        loss_of_function = LoFtool
    ) %>%
    mutate(
        allele_frequency = as.numeric(allele_frequency),
        allele_frequency = ifelse(allele_frequency > 0.5, 1 - allele_frequency, allele_frequency),
        allele_frequency = log10(allele_frequency),
        functionality_prediction = ifelse(functionality_prediction == "N", "Neutral",
                                          ifelse(functionality_prediction == "L", "Low",
                                                 ifelse(functionality_prediction == "M", "Medium",
                                                        ifelse(functionality_prediction == "H", "High", NA
                                                        )))),
        functionality_prediction = factor(
            x = functionality_prediction,
            levels = c("Neutral", "Low", "Medium", "High")
        ),
        clinical_significance = ifelse(grepl(x = clinical_significance, pattern = "5"), "Pathogenic",
                                       ifelse(grepl(x = clinical_significance, pattern = "4"), "Likely pathogenic",
                                              ifelse(grepl(x = clinical_significance, pattern = "3"), "Likely benign",
                                                     ifelse(grepl(x = clinical_significance, pattern = "2"), "Benign", NA
                                                     )))),
        clinical_significance = factor(
            x = clinical_significance,
            levels = c("Benign", "Likely benign", "Likely pathogenic", "Pathogenic")
        ),
        loss_of_function = as.numeric(loss_of_function)
    ) %>%
    distinct()

# Merge with the correlation data frame

corVepDF <- corDF %>%
    select(
        peptide_sequence, gene, gene_variant, cor, rCor
    ) %>%
    left_join(
        vepDF,
        by = c("gene", "gene_variant")
    )Warning message:
"NAs introduced by coercion"
Warning message:
"NAs introduced by coercion"

##### 👨💻 Plot the distribution of the correlation against the minor allele frequency_____no_output_____

plotDF <- corVepDF %>%
    select(
        peptide_sequence, gene, gene_variant, cor, rCor, allele_frequency
    ) %>%
    filter(
        !is.na(allele_frequency)
    ) %>%
    distinct() %>%
    gather(
        cor, rCor,
        key = "data",
        value = "correlation"
    ) %>%
    mutate(
        data = ifelse(data == "cor", "Data", "Random"),
        data = factor(
            x = data,
            levels = c("Data", "Random")
        )
    ) %>%
    arrange(
        desc(data)
    )

ggplot(
    data = plotDF
) +
    geom_point(
        mapping = aes(
            x = allele_frequency,
            y = correlation,
            col = data
        ),
        alpha = 0.2
    ) +
    geom_smooth(
        mapping = aes(
            x = allele_frequency,
            y = correlation,
            col = data,
            fill = data
        ),
        method = "loess"
    ) +
    scale_x_continuous(
        name = "Minor Allele Frequency [log10]",
        breaks = -5:-1,
        labels = paste0("10e", -5:-1)
    ) +
    scale_y_continuous(
        name = "Correlation"
    ) +
    scale_color_manual(
        values = scico(
            n = 2,
            palette = "cork",
            begin = 0.2,
            end = 0.8,
            direction = -1
        )
    ) +
    scale_fill_manual(
        values = scico(
            n = 2,
            palette = "cork",
            begin = 0.2,
            end = 0.8,
            direction = -1
        )
    ) +
    theme(
        legend.position = "top",
        legend.title = element_blank()
    )Warning message:
"Removed 276 rows containing non-finite values (stat_smooth)."

##### 💬 Intuitively, what frequency do you expect variants promoting tumor survival to have?

##### 👨💻 Plot the distribution of the correlation against the functionality prediction_____no_output_____

plotDF <- corVepDF %>%
    select(
        peptide_sequence, gene, gene_variant, cor, rCor, functionality_prediction
    ) %>%
    filter(
        !is.na(functionality_prediction)
    ) %>%
    distinct() %>%
    gather(
        cor, rCor,
        key = "data",
        value = "correlation"
    ) %>%
    mutate(
        data = ifelse(data == "cor", "Data", "Random"),
        data = factor(
            x = data,
            levels = c("Data", "Random")
        )
    ) %>%
    arrange(
        desc(data)
    )

ggplot(
    data = plotDF
) +
    geom_violin(
        mapping = aes(
            x = functionality_prediction,
            y = correlation,
            col = data,
            fill = data
        ),
        alpha = 0.5,
        width = 0.8
    ) +
    geom_boxplot(
        mapping = aes(
            x = functionality_prediction,
            y = correlation,
            col = data
        ),
        fill = "white",
        alpha = 0.8,
        width = 0.8
    ) +
    scale_x_discrete(
        name = "Functionality Prediction"
    ) +
    scale_y_continuous(
        name = "Correlation"
    ) +
    scale_color_manual(
        values = scico(
            n = 2,
            palette = "cork",
            begin = 0.2,
            end = 0.8,
            direction = -1
        )
    ) +
    scale_fill_manual(
        values = scico(
            n = 2,
            palette = "cork",
            begin = 0.2,
            end = 0.8,
            direction = -1
        )
    ) +
    theme(
        legend.position = "top",
        legend.title = element_blank()
    )_____no_output_____

##### 💬 How can we interpret that protein and SAAV correlation is associated with functional prediction?

##### 👨💻 Plot the distribution of the correlation against the loss of function prediction_____no_output_____

plotDF <- corVepDF %>%
    select(
        peptide_sequence, gene, gene_variant, cor, rCor, loss_of_function
    ) %>%
    filter(
        !is.na(loss_of_function)
    ) %>%
    distinct() %>%
    gather(
        cor, rCor,
        key = "data",
        value = "correlation"
    ) %>%
    mutate(
        data = ifelse(data == "cor", "Data", "Random"),
        data = factor(
            x = data,
            levels = c("Data", "Random")
        )
    ) %>%
    arrange(
        desc(data)
    )

ggplot(
    data = plotDF
) +
    geom_point(
        mapping = aes(
            x = loss_of_function,
            y = correlation,
            col = data
        ),
        alpha = 0.2
    ) +
    geom_smooth(
        mapping = aes(
            x = loss_of_function,
            y = correlation,
            col = data,
            fill = data
        ),
        method = "loess"
    ) +
    scale_x_continuous(
        name = "Loss of Function Score"
    ) +
    scale_y_continuous(
        name = "Correlation"
    ) +
    scale_color_manual(
        values = scico(
            n = 2,
            palette = "cork",
            begin = 0.2,
            end = 0.8,
            direction = -1
        )
    ) +
    scale_fill_manual(
        values = scico(
            n = 2,
            palette = "cork",
            begin = 0.2,
            end = 0.8,
            direction = -1
        )
    ) +
    theme(
        legend.position = "top",
        legend.title = element_blank()
    )
_____no_output_____

##### 💬 Intuitively, how do you expect the correlation to evolve relatively to the loss of function score?

##### 👨💻 Plot the distribution of the correlation against the loss of clinical significance_____no_output_____

plotDF <- corVepDF %>%
    select(
        peptide_sequence, gene, gene_variant, cor, rCor, clinical_significance
    ) %>%
    filter(
        !is.na(clinical_significance)
    ) %>%
    distinct() %>%
    gather(
        cor, rCor,
        key = "data",
        value = "correlation"
    ) %>%
    mutate(
        data = ifelse(data == "cor", "Data", "Random"),
        data = factor(
            x = data,
            levels = c("Data", "Random")
        )
    ) %>%
    arrange(
        desc(data)
    )

ggplot(
    data = plotDF
) +
    geom_violin(
        mapping = aes(
            x = clinical_significance,
            y = correlation,
            col = data,
            fill = data
        ),
        alpha = 0.5,
        width = 0.8
    ) +
    geom_boxplot(
        mapping = aes(
            x = clinical_significance,
            y = correlation,
            col = data
        ),
        fill = "white",
        alpha = 0.8,
        width = 0.8
    ) +
    scale_x_discrete(
        name = "Clinical Significance"
    ) +
    scale_y_continuous(
        name = "Correlation"
    ) +
    scale_color_manual(
        values = scico(
            n = 2,
            palette = "cork",
            begin = 0.2,
            end = 0.8,
            direction = -1
        )
    ) +
    scale_fill_manual(
        values = scico(
            n = 2,
            palette = "cork",
            begin = 0.2,
            end = 0.8,
            direction = -1
        )
    ) +
    theme(
        legend.position = "top",
        legend.title = element_blank()
    )_____no_output_____

##### 💬 How do you interpret the association of correlation between gene and SAAV peptides abundances in tumors and pathogenicity? How can we take advantage of the variant association in other diseases?

## Conclusion

##### 👨💻 Can you speculate on the function or effect of these variant peptides in cancer biology? How can these be used in a clinical setup?_____no_output_____


# Notebook from gavin-bauer/earth-vital-signs
Path: notebooks/arctic sea ice.ipynb

_____no_output_____# ARCTIC SEA ICE CONCENTRATION ANALYSIS

## **About**
Below is code part of a personal project called [Earth's Vital Signs](https://github.com/gavin-bauer/earth-vital-signs/blob/master/README.md) intended to develop my proficiency in Data Science Analytics tools and work on one of the most defining issues of our time.

## **Findings**
![Arctic Sea Ice Extent](../assets/img/asie.png)

Between 1979 and 2015, the average monthly extent for September declined by 13% per decade, with a record low in 2012 when sea ice plummeted to 3.41mn square kilometers.

Moreover, it appears that low minimum extents in the Arctic have become the “new normal” since the mid-2000s.

## **Datasets**
*NB: Note to reproduce the notebook, you need to create and download the dataset from the NSIDC website below.*
* [NASA/NSIDC](https://doi.org/10.5067/8GQ8LZQVL0VL)
* [NASA](https://climate.nasa.gov/system/internal_resources/details/original/1929_Arctic_data_1979-2019.txt)_____no_output_____### **Code**_____no_output_____

# install cartopy in Google Colab
!apt-get install libproj-dev proj-data proj-bin
!apt-get install libgeos-dev
!pip install cython
!pip install cartopy_____no_output_____from google.colab import drive
drive.mount('/content/gdrive')_____no_output_____# import libs
import os
from datetime import datetime, timedelta
import numpy as np
import numpy.ma as ma
import pandas as pd
import matplotlib.pyplot as plt
plt.style.use('ggplot')
plt.rcParams['axes.facecolor']='w'
import cartopy.crs as ccrs
import imageio as io
from PIL import Image

% matplotlib inline_____no_output_____def create_path(ROOT: str, file: str):
  path = os.path.join(ROOT, file)
  return path

def reshape_asic_data(path: str):
  with open(path, 'rb') as f:
    header = f.read(300)
    data = np.fromfile(f, dtype=np.uint8)
  data = data.reshape(448,304)
  data = data / 250.
  data = ma.masked_greater(data, 1.0)
  return data

def process_date(file: str):
  date = file[3:11]
  date = datetime(year=int(date[0:4]), month=int(date[4:6]), day=int(date[6:8]))
  date = date.strftime("%Y/%m/%d")
  return date

def plot_asic_data(ROOT: str, file: str, dx: int, dy: int, x: list, y: list):
  path = create_path(ROOT, file)
  data = reshape_asic_data(path)
  date = process_date(file)
  fig, ax = plt.subplots()
  ax = plt.axes(projection=ccrs.NorthPolarStereo(central_longitude=0))
  cs = ax.coastlines(resolution='110m', linewidth=0.5)
  ax.gridlines()
  ax.set(title=f'Arctic Sea Ice Concentration in {date}')
  ax.set_extent([-180, 180, 40, 90], crs=ccrs.PlateCarree())
  kw = dict(central_latitude=90, central_longitude=-45, true_scale_latitude=70)
  cs = ax.pcolormesh(x, y, data, cmap=plt.cm.Blues,
                    transform=ccrs.Stereographic(**kw))
  return date, ax

def make_gif():
  file_names = sorted((fn for fn in os.listdir('/content') if fn.endswith('png')))
  with io.get_writer('asic.gif', mode='I', duration=0.5) as writer:
    for filename in file_names:
      image = io.imread(filename)
      writer.append_data(image)
  writer.close()

def process_arctic_data(link: str):
  data = pd.read_csv(link, sep='\t')
  data.columns = ['year', 'month', 'data_type', 'region', 'extent', 'area']
  data = data[['year', 'extent']]
  return data

def plot_arctic_data(link: str):
  data = process_arctic_data(link)
  year = pd.to_numeric(data['year'])
  extent = pd.to_numeric(data['extent'])
  fig, ax = plt.subplots()
  ax.plot(year, extent)
  ax.set(xlabel='year', ylabel='area in million sq km',
        title='Arctic Sea Ice Extent Evolution')
  ax.set_yticks(np.arange(3,8,1))
  ax.grid(color='#7f7f7f', linewidth=0.2)
  return ax_____no_output_____ROOT = 'gdrive/My Drive/datasets/asic'
files = ['nt_19790901_n07_v1.1_n.bin', 'nt_19890901_f08_v1.1_n.bin',
         'nt_19990901_f13_v1.1_n.bin', 'nt_20090901_f17_v1.1_n.bin',
         'nt_20180901_f17_v1.1_n.bin']
dx = dy = 25000
x = np.arange(-3850000, +3750000, +dx)
y = np.arange(+5850000, -5350000, -dy)_____no_output_____for file in files:
  date, ax = plot_asic_data(ROOT, file, dx, dy, x, y)
  year = date[:4]
  PATH = os.path.join('/content', f'asic_{year}')
  chart = ax.get_figure()
  chart.savefig(PATH, dpi=150)

make_gif()_____no_output_____Image.open('asic.gif')_____no_output_____link = 'https://climate.nasa.gov/system/internal_resources/details/original/1929_Arctic_data_1979-2019.txt'
PATH = os.path.join('/content', f'asic_arctic_extent')
ax = plot_arctic_data(link)
chart = ax.get_figure()
chart.savefig(PATH, dpi=150)_____no_output__________no_output_____


# Notebook from bjedwards/NetworkXTutorial
Path: II. Creating, Reading and Writing Graphs.ipynb

import networkx as nx_____no_output_____

# NetworkX Data Capabilities_____no_output_____NetworkX has many built in functions to read data from a variety of formats. Because formats can be pretty esoteric this is also the source of many bugs. Please if you find them report them. Here is a brief list of the file formats NetworkX can read and write, and their corresponding function names

| Format | Read Function | Write Function |
| - | -| -|
| Adjacency List| `nx.read_adjlist` | `nx.write_adjlist`|
| Edge List |`nx.read_edgelist` | `nx.write_edgelist`|
| Graph Exchange XML Format (GEXF)| `nx.read_gexf` |`nx.write_gexf`|
| Graph Modeling Language (GML)| `nx.read_gml` |`nx.write_gml`|
| Pickle, Python's serialization format for NetworkX graphs| `nx.read_gpickle` | `nx.write_gpickle`|
| GraphML | `nx.read_graphml` |`nx.write_graphml`|
| Leda | `nx.read_leda` | `nx.write_leda`|
| Multiline Adjacency List| `nx.read_multiline_adjlist` | `nx.write_multiline_adjlist`|
| Esri Shapefile| `nx.read_shp` |`nx.write_shp`|
| Yet Another Markup Language (YAML)| `nx.read_yaml` | `nx.write_yaml`|
| p2g (metabolic pathways)|`nx.read_p2g` | `nx.write_p2g`|
| pajek|`nx.read_pajek` | `nx.write_pajek`|
| sparse6|`nx.read_sparse6` | `nx.write_sparse6`|_____no_output_____## Exercise_____no_output_____1. What format did we save the classroom graph in the last session? Load it using the correct function.
2. Save the same graph using a different format above. Save it to the `./data/` folder_____no_output_____# Creating Graphs from Other Python Data Structures_____no_output_____## Matrices and Arrays_____no_output_____It is often useful to create graphs from other types of python data structures. In particular, graphs are often represented by matrices. If your data has a matrix represenatation, NetworkX can easily create a graph using `from_numpy_matrix`_____no_output_____

import numpy as np_____no_output_____n = 25
A = np.random.binomial(1,1.1/n,size=(n,n)) # Random 1/s with probability 1/25
G = nx.from_numpy_matrix(A)_____no_output_____G.order()_____no_output_____G.size()_____no_output_____G.degree()_____no_output_____

### Quick Quiz..._____no_output_____What kind of graph is the one above?_____no_output_____NetworkX can handle other data structures such as a list of edges (`from_edgelist`) and `scipy` sparse matrices (`scipy_sparse_matrix`). You can use the `create_using` keyword to make a `DiGraph`s or `MultiGraph`s._____no_output_____

edges = []
for u in range(n):
    for v in range(n):
        if u % 3==0 and u>v:
            edges.append((u,v))
        elif u % 3 == 1 and v < u:
            edges.append((u,v))
        _____no_output_____D = nx.from_edgelist(edges,create_using=nx.DiGraph())_____no_output_____

You can also create matrices out of already created graphs_____no_output_____

G = nx.Graph()
for u in range(5):
    G.add_edge(u,u,index=u) # Self loops!_____no_output_____nx.to_numpy_matrix(G)_____no_output_____

You can output weighted matrices too_____no_output_____

nx.to_numpy_matrix(G,weight='index')_____no_output_____

## Other Graphs_____no_output_____Another obvious way to create graphs is to use other graphs. This can be especially useful when coverting between `Graph`s and `DiGraphs` or making copies of graphs for modification._____no_output_____

D = nx.DiGraph()
D.add_star(range(5))
D.add_cycle(range(5,10))
D.add_edge(4,5)_____no_output_____G = nx.Graph(D)_____no_output_____G.edges()_____no_output_____G = nx.Graph()
G.add_star(range(5))
D = nx.DiGraph(G)_____no_output_____D.edges()_____no_output_____

Notice when we create a directed graph from a graph we get edges in both directions._____no_output_____## Graph Operations_____no_output_____Finally, NetworkX includes a number of graph operations to make combining graphs easier. First some exercises so we have some graphs to mess with.

1. Create a complete graph called `C` of five nodes
2. Create a line graph and called `L` of 10 nodes
3. Create a star graph called `S` with of 7 nodes with 0 at it's center_____no_output_____Try out these functions, what do they produce?
 - `nx.compose(C,L)`
 - `nx.cartesian_product(S,L)`
 - `nx.complement(S)`_____no_output_____### Exercises_____no_output_____To get ready for the next section let's write a function that implements the [Kronecker Graph of order k](http://arxiv.org/pdf/0812.4905v2.pdf). This is simply the `tensor_product` of a graph applied to itself k times.
That is, if we have a graph $G$, with adjacency matrix $A$ and kronecker product(tensor product) $\otimes$, the _Kronecker Graph_ of order $k$ is a graph with Adjacency matrix

$$(A)^k = \underbrace{A \otimes A \otimes \cdots \otimes A}_{ k \text{ terms}}$$

Let's also assume (as they do in the paper) that the kronecker graph always has self loops on all the nodes.

I'll give you the function stub and you can fill in the details_____no_output_____

def kronecker_graph(G,k):

    K = nx.Graph(G) # For the kronecker graph we are going to return
    B = nx.Graph(G) # For the base graph

    for n in K:
        #Add self loops to both K and B

    for i in range():# Add the number of kronecker products here
        #Create K using the appropriate networkx product function
    return K_____no_output_____

Make a kronecker graph of order $k=2$ out of the Line graph `L`.
 - How many nodes should it have?_____no_output_____

K = kronecker_graph(L,2)_____no_output_____