# Notebook from VCMason/PyGenToolbox
Path: notebooks/Lyna/CalculateRelativeBaseCoverageOfFeatureType_DNAContactsNearIESs.ipynb

<code>
%load_ext autoreload
%autoreload 2
import datetime
import os
print(datetime.datetime.now())

from pygentoolbox.CalculateRelativeBaseCoverageOfFeatureType import main
#dir(pygentoolbox.Tools)
%matplotlib inline
import matplotlib.pyplot as pltThe autoreload extension is already loaded. To reload it, use:
  %reload_ext autoreload
2021-01-14 16:59:36.884818
gff3file = '/media/sf_LinuxShare/Ciliates/Genomes/Annotations/internal_eliminated_sequence_PGM_ParTIES.pt_51_with_ies.gff3'
filelist = ['/media/sf_LinuxShare/Projects/Lyna/CharSeqPipe/PGM_L/hisat2/RNA.DNA.Contacts.pt.wthickness.Win300.MQ10.NormDPNII.txt', '/media/sf_LinuxShare/Projects/Lyna/CharSeqPipe/Nowa1_E/hisat2/RNA.DNA.Contacts.pt.wthickness.Win300.MQ10.NormDPNII.txt', '/media/sf_LinuxShare/Projects/Lyna/CharSeqPipe/EV_E_50MilReads/hisat2/Pt_51_MacAndIES/RNA.DNA.Contacts.pt.wthickness.Win300.MQ0.NormDPNII.tsv']
datatype='contacts' # options 'sam', 'contacts', 'bed'
target_feature = 'internal_eliminated_sequence'
numbins = 50

# plots histograms of all relative positions for all features of a feature type
# relative position for each gene (1st position = 1/len(gene), 5th = 5/len(gene))
# i.e. normalizes each position in feature by gene length

main(gff3file, filelist, datatype, target_feature, numbins)Reading Gff3 file: /media/sf_LinuxShare/Ciliates/Genomes/Annotations/internal_eliminated_sequence_PGM_ParTIES.pt_51_with_ies.gff3
Number of scaffolds: 511
Reading contact file: /media/sf_LinuxShare/Projects/Lyna/CharSeqPipe/PGM_L/hisat2/RNA.DNA.Contacts.pt.wthickness.Win300.MQ10.NormDPNII.txt
Gathering DNA contact position and strength
Number of total contacts: 8513261
Number of uniqe contacts: 3084594
For each position in feature: internal_eliminated_sequence, normalize by gene length, record value for histogram
Plotting histogram to outfile: /media/sf_LinuxShare/Projects/Lyna/CharSeqPipe/PGM_L/hisat2/RNA.DNA.Contacts.pt.wthickness.Win300.MQ10.NormDPNII.NormCov.internal_eliminated_sequence.pdf

</code>


# Notebook from chuangw46/Abalone_Age_Prediction
Path: src/eda/abalone_eda.ipynb

# Exploratory data analysis of the Abalone data set_____no_output_____## About the Abalone dataset

The dataset used in this project comes from an original study "The Population Biology of Abalone (_Haliotis_ species) in Tasmania. I. Blacklip Abalone (_H. rubra_) from the North Coast and Islands of Bass Strait", created by Warwick J Nash, Tracy L Sellers, Simon R Talbot, Andrew J Cawthorn and Wes B Ford (1994). It was sourced from the UCI Machine Learning Repository and can be found [here](http://archive.ics.uci.edu/ml/machine-learning-databases/abalone/). Each row in the data set represents an abalone, including the physical measurements(Sex, Length, Diameter, Height, Whole weight, etc.) and the number of rings, which gives the age in years by adding 1.5. The missing values in the original study have been removed and the range of continuous values has been scaled. Please find the detailed information [here](http://archive.ics.uci.edu/ml/datasets/Abalone?pagewanted=all)._____no_output_____## 0. Setting up_____no_output_____### 0.1 Import Packages_____no_output_____

import numpy as np
import pandas as pd
import altair as alt
from sklearn.model_selection import train_test_split
import seaborn as sns
import matplotlib.pyplot as plt_____no_output_____

### 0.2 Import Raw Data and Preprocessing_____no_output_____

# original dataset
abalone = pd.read_csv("../../data/raw/abalone.csv")

# add columns' titles to the dataframe
abalone.columns = [
    "Sex",
    "Length",
    "Diameter",
    "Height",
    "Whole Weight",
    "Shucked Weight",
    "Viscera Weight",
    "Shell Weight",
    "Rings",
]

# transform the columm `Rings` to `Age` by adding 1.5 (according to the dataset description file)
abalone["Age"] = abalone["Rings"] + 1.5
abalone = abalone.drop(columns="Rings")

# split the dataset to train set and test set
train_df, test_df = train_test_split(abalone, test_size=0.2, random_state=123)_____no_output_____

## 1. Exploratory analysis on the training data set_____no_output_____### 1.1 Check Missing Values_____no_output_____By looking at the train dataset, we can see that there aren't any missing values in the training data._____no_output_____

train_df.isnull().sum()_____no_output_____

### 1.2 Attribute Information_____no_output_____

train_df.head()_____no_output_____train_df.info()<class 'pandas.core.frame.DataFrame'>
Int64Index: 3340 entries, 2746 to 3582
Data columns (total 9 columns):
 #   Column          Non-Null Count  Dtype
---  ------          --------------  -----
 0   Sex             3340 non-null   object
 1   Length          3340 non-null   float64
 2   Diameter        3340 non-null   float64
 3   Height          3340 non-null   float64
 4   Whole Weight    3340 non-null   float64
 5   Shucked Weight  3340 non-null   float64
 6   Viscera Weight  3340 non-null   float64
 7   Shell Weight    3340 non-null   float64
 8   Age             3340 non-null   float64
dtypes: float64(8), object(1)
memory usage: 260.9+ KB

> Except `Sex`, all features and the target column `Age` are numeric._____no_output_____>  Description of each of the attributes:

- **Sex**: M, F, and I (infant)
- **Length**: Longest shell measurement in mm
- **Diameter**: perpendicular to length in mm
- **Height**: with meat in shell in mm
- **Whole Weight**: whole abalone in grams
- **Shucked Weight**: weight of meat in grams
- **Viscera Weight**: gut weight (after bleeding) in grams
- **Shell Weight**: grams after being dried in grams
- **Age**: age in years_____no_output_____

train_df.describe()_____no_output_____

### 1.3 Exploration of the target `Age`_____no_output_____#### 1). Distribution of Target Column: 'Age'_____no_output_____

train_df1 = train_df.copy()
train_df1.columns = [
    "Sex",
    "Length (mm)",
    "Diameter (mm)",
    "Height (mm)",
    "Whole Weight (g)",
    "Shucked Weight (g)",
    "Viscera Weight (g)",
    "Shell Weight (g)",
    "Age (Years)",
]
sns.set(rc={"figure.figsize": (10, 7.5)}, font_scale=1.25)
sns.set_theme(style="whitegrid")
sns.histplot(train_df1, x="Age (Years)", kde=True, bins=15)_____no_output_____

The distribution of `Age` is approximately normal with a mean of around 11.5 years. However, it does have a slight right skewness._____no_output_____#### 2). Distribution of 'Age' conditioned on 'Sex':_____no_output_____

train_df1_____no_output_____fig, ax = plt.subplots()

sns.set(rc={"figure.figsize": (10, 7.5)}, font_scale=1.25)

sns.violinplot(y="Sex", x="Age (Years)", data=train_df1, inner="quartile")

ax.set_yticklabels(["Infant", "Male", "Female"])

# ax.set_yticklabels(target_names)_____no_output_____df_i = train_df1.loc[train_df1["Sex"] == "I"]
df_m = train_df1.loc[train_df1["Sex"] == "M"]
df_f = train_df1.loc[train_df1["Sex"] == "F"]
summary = pd.concat(
    [
        train_df.describe()["Age"],
        df_i[["Age (Years)"]].describe(),
        df_m[["Age (Years)"]].describe(),
        df_f[["Age (Years)"]].describe(),
    ],
    axis=1,
)
summary.columns = ["Age", "Age - Infant", "Age - Male", "Age - Female"]
summary_____no_output_____

- The number of instances of Infants, Males and Females are pretty close to each other.
- The mean, median, Q1, and Q3 of `Age` for Infants are all lower than that for Males and Females.
- The mean, median, Q1, Q3 and standard deviation of `Age` are all very close for Males and Females._____no_output_____#### 3). Scatter Plot of 'Age' with all Numerical Columns_____no_output_____

sns.set(rc={"figure.figsize": (30, 12)}, font_scale=1.25)

fig, axs = plt.subplots(nrows=2, ncols=4)

sns.set_theme(style="whitegrid")

for i, column in enumerate(train_df1.drop(columns=["Sex"]).columns):
    sns.histplot(
        x="Age (Years)",
        y=column,
        data=train_df1,
        binwidth=(1, 0.025),
        ax=axs[(i // 4, i % 4)],
    )

fig.delaxes(axs[1, 3])_____no_output_____

> From the 2D histogram plot, we can see that the target `Age` seems to be positively correlated with all the numerical features._____no_output_____### 1.4 Exploration of all features:_____no_output_____#### 1) Distribution_____no_output_____> First, let's have a look at the distribution of the values of all features (numerical)_____no_output_____From the distribution plots below, we can see that all features values are not normally distributed. There are some outliers for the `Height` value, but we will exclude outliers from training dataset before building models. The feature "Whole Weight" distributes from 0 to 2.5, which is a wider range than other features. Other features distribute from 0 to around 1. According to the dataset information on the UCI Machine Learning Repository website, the ranges have been scaled by dividing by 200. Thus, we are going to use the numerical values as our model input, not change the scale._____no_output_____

sns.set(rc={"figure.figsize": (30, 20)}, font_scale=1.8)
fig, axs = plt.subplots(nrows=3, ncols=3)
sns.set_theme(style="whitegrid")

for i, column in enumerate(train_df1.drop(columns=["Sex"]).columns):
    sns.histplot(train_df1[column], kde=True, bins=15, ax=axs[(i // 3, i % 3)])

fig.delaxes(axs[2, 2])_____no_output_____

#### 2). Pairplot of Numerical Columns_____no_output_____> From the pairplot below, we see that all numerical features have a positive relationship between each other. We are going to use all features as our model input. _____no_output_____

sns.pairplot(
    train_df1,
    vars=list(train_df1.columns)[1:-1],
    kind="hist",
    diag_kind="kde",
    corner=True,
)_____no_output_____

#### 3). Correlation Plot_____no_output_____> From the correlation plot below, we can see that the `shell weight` is the feature that is most correlated to the target `Age` because the correlation value is largest among all features. _____no_output_____

sns.set(rc={"figure.figsize": (15, 10)})
num_data = train_df[list(train_df.columns)[1:]]
ax = sns.heatmap(
    num_data.corr(),
    annot=True,
    annot_kws={"fontsize": 15},
    vmin=-1,
    vmax=1,
    cmap="Spectral",
    linewidths=0.5,
)
ax_____no_output_____

#### 4). Numerical Columns conditioned on 'Sex'_____no_output_____

sns.set(rc={"figure.figsize": (30, 12)}, font_scale=1.25)

fig, axs = plt.subplots(nrows=2, ncols=4)

sns.set_theme(style="whitegrid")

for i, column in enumerate(train_df1.drop(columns=["Sex"]).columns):
    sns.violinplot(
        y="Sex", x=column, data=train_df1, inner="quartile", ax=axs[(i // 4, i % 4)],
    )

fig.delaxes(axs[1, 3])_____no_output_____


# Notebook from bbengfort/brookings-nlp
Path: Session 3 - Text Classification.ipynb

%matplotlib inline

import matplotlib as mpl
mpl.rcParams['figure.figsize'] = [9.0, 6.0]_____no_output_____

# Session 3: Text Classification and Topic Modeling
**July 12, 2018**_____no_output_____In this session, we'll begin using machine learning techniques to broaden our text analytics. Now that we have a corpus of documents, we can start building topic models to determine if documents are similar to each other and how to categorize our documents broadly. We can also use text classification methods to create tools that allow us to model documents and utterances more precisely. We'll take a look at a specific kind of classification - sentiment analysis and how it can be used for engaging in deeper analytics. _____no_output_____## Intro to Machine Learning

In general, a learning problem considers a set of $n$ instances (or examples) that models are trained on.

Instances are represented by a multidimensional entry (aka multivariate data) having several attributes or _features_.

The goal of the learning problem is to create a model that predicts a _target_ attribute or feature._____no_output_____![Feature Space](figures/s3_feature_space.png)

The best way to think about machine learning is problems that use mathematical and statistical methods to find patterns in _high dimensional space_._____no_output_____### Learning by Example

| Problem Domain                              | Machine Learning Class |
|---------------------------------------------|------------------------|
| Infer a function from labeled data          | Supervised learning    |
| Discover structure of data without feedback | Unsupervised learning  |
| Interact with environment towards a goal    | Reinforcement learning |

_Given examples (data) extract a meaningful pattern upon which to act.______no_output_____### Algorithms by Output

| Type of Output                              | Algorithm Category              |
|---------------------------------------------|---------------------------------|
| **Output is one or more discrete classes**  | **Classification (Supervised**) |
| **Output is continuous**                    | **Regression (Supervised)**     |
| **Output is membership in a similar group** | **Clustering (Unsupervised)**   |
| Output is the distribution of inputs        | Density Estimation              |
| Output is simplified from higher dimensions | Dimensionality Reduction        |

_Use training data to fit a model which is then used to predict incoming inputs.______no_output_____### Classification

![Classification](figures/s3_classification.png)

Given labeled input data (with two or more labels), fit a function that can determine for any input, what the label is. _____no_output_____### Regression

![Regression](figures/s3_regression.png)

Given continuous input data fit a function that is able to predict the continuous value of input given other data._____no_output_____### Clustering

![Clustering](figures/s3_clustering.png)

Given data, determine a pattern of associated data points or clusters via their similarity or distance from one another._____no_output_____## The Bag of Words Model

In order to do words on machine learning, therefore - we need to represent text numerically somehow.  _____no_output_____![Bag of Words](figures/s3_bag_of_words.png)_____no_output_____We've already noted that words co-occurring together demonstrate statistical significance that might be related to meaning. The bag-of-words model takes advantage of this to create a model that relies on numeric representations of words that are located close together. _____no_output_____## Vectorization

Also called feature-extraction; the process of transforming text documents into numeric representations (vectors) that can be used to do machine learning.

![Vector Encoding](figures/s3_vector_encoding.png)_____no_output_____The common method of vectorization is to take the _vocabulary_ of the corpus and order them lexicographically (in alphabetical order). To transform a document into a vector, we simply assign a number for each word that represents the word's relationship to the document._____no_output_____### One Hot Encoding

![One Hot Encoding](figures/s3_one_hot_encoding.png)

The simplest method is one-hot-encoding where we simply assign a 1 if the word exists in the document, or a 0 otherwise. This is a very common encoding that is generally used in artificial neural networks._____no_output_____### Frequency Encoding

![Frequency Encoding](figures/s3_frequency_encoding.png)

If we believe that the number of times a word appears in a document matters, we can simply count the number of occurrences and use that number in the word's vector position._____no_output_____### TF-IDF

![TF-IDF](figures/s3_tfidf_encoding.png)

Term-Frequency, Inverse-Document-Frequency is a measure of a word's relative importance to the document, given its frequency in the entire corpus. _____no_output_____### word2vec

![Distributed Representations](figures/s3_distributed_representation.png)

The current state of the art is called _word embeddings_. This representation computes a word vector based on word similarity; e.g. "king" and "queen" will be closer together than "red" and "banana". _____no_output_____## Scikit-Learn

Scikit-Learn is a Python package that provides an class-based API for _generalized machine learning_ -- implementing a significant number of algorithms for machine learning as well as utilities for feature extraction, evaluation, etc. _____no_output_____### Estimators

Object-oriented interface centered around the concept of an Estimator:

> “An estimator is any object that learns from data; it may be a classification, regression or clustering algorithm or a transformer that extracts/filters useful features from raw data.”

— Scikit-Learn Tutorial
_____no_output_____| Object        | Defining Method    | Description                                                       |
|---------------|--------------------|-------------------------------------------------------------------|
| Estimator     | `fit(X, y=None)`   | Train the estimator to learn from data X (and y if supervised).   |
| Transformer   | `transform(X)`     | Transform dataset `X` into `X'` (e.g. for feature extraction).    |
| Predictor     | `predict(X)`       | Given test data `X` return predictions `yhat`                     |
| Probabilistic | `predict_proba(X)` | Returns the probability for all possible `yhat` values            |
| Scorer        | `score(X, y=None)` | Given test data with known answers (y) return performance metric. |_____no_output_____- `X`: 2D array-like of shape `(n_instances, n_features)`
- `y`: 1D array-like of shape `(n_instances,)` (optional)_____no_output_____### Pipelines

![Text Pipeline](figures/s3_text_pipelines.png)_____no_output_____```python
from sklearn.naive_bayes import MultinomialNB
from sklearn.pipeline import Pipeline, FeatureUnion
from sklearn.feature_extraction.text import CountVectorizer, TfidifTransformer

pipeline = Pipeline([
  ('extract_essays', EssayExractor()),
  ('features', FeatureUnion([
    ('ngram_tf_idf', Pipeline([
      ('counts', CountVectorizer()),
      ('tf_idf', TfidfTransformer())
    ])),
    ('essay_length', LengthTransformer()),
    ('misspellings', MispellingCountTransformer())
  ])),
  ('classifier', MultinomialNB())
])
```_____no_output_____## Preliminaries for examples

1. Get `CorpusReader` to access our documents in categories
2. Create some utility functions for examples _____no_output_____

from utils.reader import PickledCorpusReader

# Create our corpus reader to access our data
corpus = PickledCorpusReader("data/corpus")_____no_output_____import random

from functools import partial
from tabulate import tabulate

tabulate = partial(tabulate, headers='firstrow', tablefmt='pipe')

def get_document(corpus, fileid):
    return next(corpus.raw_data(fileids=fileid))

def random_document(corpus, categories=None):
    fileid = random.choice(corpus.fileids(categories=categories))
    return fileid, get_document(corpus, fileid)

def fileids(index, corpus, categories=None):
    for idx in index:
        yield corpus.fileids(categories=categories)[idx]_____no_output_____

## Topic Modeling

Unsupervised methods of clustering related documents into _topics_ — e.g. find groups of related documents based on the terms that they contain. _____no_output_____![Topic Modeling Pipeline](figures/s3_topic_modeling.png)_____no_output_____## Nearest Neighbors

Similarity in feature space means _distance_ -- e.g. the closer two points are together, the more similar they are (and they are identical if they are the same point). Nearest neighbors selects the $k$ nearest documents by distance and returns them as the most similar to the given document. _____no_output_____

from sklearn.pipeline import Pipeline
from sklearn.neighbors import NearestNeighbors
from sklearn.feature_extraction.text import TfidfVectorizer

model = Pipeline([
    ('tfidf', TfidfVectorizer()),
    ('neighbors', NearestNeighbors()),
])

model.fit(corpus.raw_data(categories=["news"]))_____no_output_____# Get random document
# target_fileid, target_text = random_document(corpus, categories=["news"])

# Or use provided example
target_fileid = 'news/571c19d2c1808103a0d8de39.pickle'
target_text = get_document(corpus, target_fileid)

print(target_text)Prince ' s remains have been cremated , and a small group of family and friends held a private ceremony to honor the music legend , his publicist said Saturday .

Yvette Noel - Schure said in a statement that Prince ' s remains had been cremated without specifying when .
His body was released to his family Friday afternoon after a Minnesota medical examiner performed an autopsy .

" A few hours ago , Prince was celebrated by a small group of his most beloved : family , friends and his musicians , in a private , beautiful ceremony to say a loving goodbye ," Noel - Schure said in the statement .

The " final storage " of Prince ' s remains would remain private , but an announcement about a " musical celebration " is forthcoming , the statement said .

Prince was found dead at the age of 57 Thursday at his estate in Chanhassen , southwest of Minneapolis .
The cause of his death remains unknown .
Authorities said Friday that the results of the investigation into Prince ' s death , including the autopsy results , could take days to weeks to complete , but Noel - Schure said Saturday that autopsy results would take at least a month .

Prince ' s family and former bandmates were spotted outside the music legend ' s Paisley Park estate Saturday evening , according to The Associated Press .
Hundreds of fans have flocked to the estate to pay tribute to Prince with flowers , balloons and instruments — many of which were purple — since news of the musician ' s death broke Thursday .

A woman writes on memorial sheet adorned with the symbol Prince once used to identify himself outside Paisley Park in Chanhassen , Minn ., Saturday , April 23 , 2016 .
AP

On Saturday afternoon , Paisley Park staff dressed in black emerged from the gates of the sprawling complex and handed mourners shiny purple boxes containing CDs , t - shirts , books and other Prince memorabilia .

It is unclear who Prince left his estate to or if he had a will .

Prince told Rolling Stone in an unpublished 2014 interview that " I don ' t think about gone ."

In excerpts of the interview released online Friday , Prince said he had songs in " vaults " that no one had ever heard , but he wasn ' t saving them for when he was " gone ."
The full account will be published in the next issue .

" No , I don ' t think about gone .
I just think about in the future when I don ' t want to speak in real time ," Prince said about the unreleased music .

He said he recorded so much that he didn ' t " have time to compile everything ."
While he denied being " addicted " to recording , Prince said he was always composing new music .

" I can hear five albums in my head right now ," he said .

When asked if he would ever retire , Prince told Rolling Stone that he didn ' t understand the concept of retirement .
" There ' s always some way to serve ," Prince said .
" And I never felt like I had a job ."
# Note the use of [target_text]
target = model.named_steps['tfidf'].transform([target_text])
target_____no_output_____distances, index = model.named_steps['neighbors'].kneighbors(target)

# Unravel the single document
index = index.ravel()
distances = distances.ravel()

# Print a table with distances and index
table = list(zip(fileids(index, corpus, categories=["news"]), index, distances))
table = [["File ID", "Index", "Distance"]] + table
print(tabulate(table))| File ID                              |   Index |    Distance |
|:-------------------------------------|--------:|------------:|
| news/571c19d2c1808103a0d8de39.pickle |     736 | 3.16101e-08 |
| news/571a3601c1808103a0d61a46.pickle |     720 | 0.867588    |
| news/571947eec1808103a0d49e5a.pickle |     707 | 0.976536    |
| news/57191af9c1808103a0d45563.pickle |     703 | 0.983616    |
| news/56f96207c1808147f082c2f3.pickle |     346 | 1           |
print(get_document(corpus, 'news/571a3601c1808103a0d61a46.pickle'))Seven - time Grammy winner Prince , who was known for megahits like “ When Doves Cry ” and “ Purple Rain ,” has died at the age of 57 .

Prince was found dead in the elevator of his Paisley Park estate in Chanhassen , Minn . on Thursday .
Deputies and medical personnel found the music icon unresponsive when they arrived at his home and they performed CPR but it was unsuccessful .
He was pronounced dead at 10 : 07 a . m . local time .

A transcript of the 911 call from the compound shows confusion as an unidentified caller struggles to give the dispatcher the proper address .
The caller said he was at " Prince ' s house " but first placed it in Minneapolis .
Another person at the compound eventually gave the correct address in the suburb of Chanhassen .

The caller first says he has " someone who is unconscious " before saying " the person is dead here ."

As the dispatcher identifies the address as Paisley Park and begins to ask a question , the caller interrupts to say , " Yes , it ' s Prince ."

" It is with profound sadness that I am confirming that the legendary , iconic performer , Prince Rogers Nelson , has died at his Paisley Park residence this morning at the age of 57 ," his rep wrote in a statement .

According to a press release , " The Carver County Sheriff ’ s Office , with the assistance of the Hennepin County Sheriff ’ s Office and The Midwest Medical Examiner ’ s Office , are continuing to investigate the circumstances surrounding his death ."
An autopsy is scheduled for Friday .

Prince had been rushed to an Illinois hospital this past Friday and was released several hours later .
His rep told TMZ he had been battling the flu for several weeks .

As news of Prince ' s death spread Thursday afternoon , approximately 200 people gathered outside the Paisley Park compound to leave bouquets , balloons and signs in tribute to the singer .
Shortly before 6 p . m ., Prince ' s sister , Tyka Nelson , emerged to tell the crowd that her brother " loved all of you .
Thank you for loving him back ."

MTV canceled its regular programming Thursday and went wall - to - wall with Prince music videos following the news of his death .
President Obama released a statement calling the superstar " a creative icon ."

" Few artists have influenced the sound and trajectory of popular music more distinctly , or touched quite so many people with their talent ," Obama said .
" As one of the most gifted and prolific musicians of our time , Prince did it all .
Funk .
R & B .
Rock and roll .
He was a virtuoso instrumentalist , a brilliant bandleader , and an electrifying performer ."

As night fell across America Thursday , shock and grief gave way to celebration of Prince ' s life .
In his hometown of Minneapolis , thousands of people gathered for an all - night dance party outside the First Avenue nightclub , where part of Prince ' s hit 1984 movie " Purple Rain " was filmed , as local musicians played covers of the star ' s hits .

In Brooklyn , filmmaker Spike Lee drew a crowd of roughly 1 , 000 people for an impromptu street party that shut down an entire block .
People wearing purple items of clothing swayed and sang along , while some hoisted bouquets of purple flowers .

A vigil quickly turned into a party at Los Angeles ' Leimert Plaza Park , where dozens danced as huge speakers blasted Prince hits and deep cuts .
Some wiped away tears but most treated the gathering as a celebration .

Several prominent landmarks were tinged with Prince ' s signature color purple Thursday night , including New York ' s Madison Square Garden , the Superdome in New Orleans , the Forum in Inglewood , Calif ., city halls in Los Angeles and San Francisco , the Lowry Avenue and I - 35W bridges in Minneapolis , Terminal Tower in Cleveland , and the Zakim Bridge in Boston .

On Broadway , the stars of " The Color Purple " performed a rendition of Prince ' s hit " Purple Rain ."
The show posted a video of the performance led by Jennifer Hudson and Cynthia Erivo online late Thursday .

VIDEO

Prince broke through in the late 1970s with the hits " Wanna Be Your Lover " and soared over the following decade with the albums as " 1999 " and " Purple Rain ."

The title song from " 1999 " includes one of the most widely quoted refrains of popular culture : " Tonight I ' m gonna party like it ' s 1999 ."

RELATED STORIES

Prince : A life in photos

Celebrities react to news of Prince ' s death

Fast Facts : Prince ' s too short life

Fans react to Prince ' s sudden death

The singer , songwriter , arranger and instrumentalist was widely acclaimed as one of the most inventive musicians of his era , drawing upon influences ranging from James Brown to the Beatles to Jimi Hendrix .

In his lifetime , he sold more than 100 million records worldwide , making him one of the best - selling artists of all time .

Prince was fiercely protective of his independence , battling his record company over control of his material and even his name .
Prince once wrote " slave " on his face in protest of not owning his work and famously battled and then departed his label , Warner Bros ., before returning a few years ago .
" What ' s happening now is the position that I ' ve always wanted to be in ," Prince told The Associated Press in 2014 .
" I was just trying to get here ."

In 1993 Prince memorably changed his name to a symbol and became widely known as the artist formerly known as Prince .
When he was inducted into the Rock and Roll of Fame in 2004 , he was hailed as a musical and social trailblazer .

" He rewrote the rulebook , forging a synthesis of black funk and white rock that served as a blueprint for cutting - edge music in the Eighties ," reads the Hall ' s dedication .
" Prince made dance music that rocked and rock music that had a bristling , funky backbone .
From the beginning , Prince and his music were androgynous , sly , sexy and provocative ."

Rarely lacking in confidence , Price effortlessly absorbed the music of others and made it sound like Prince , whether the James Brown guitar riff on " Kiss " or the Beatle - esque , psychedelic pop of " Raspberry Beret ."

He also proved a source of hits for others , from Sinead O ' Connor ' s " Nothing Compares 2 U " to Cyndi Lauper ' s " When You Were Mine ."
He also wrote " Manic Monday " for the Bangles .
Prince had been touring and recording right up until his death , releasing four albums in the last 18 months , including two on the Tidal streaming service last year .
He performed in Atlanta last week as part of his " Piano and a Microphone " tour , a stripped down show that has featured a mix of his hits like " Purple Rain " or " Little Red Corvette " and some B - sides from his extensive library .

Prince debuted the intimate format at his Paisley Park studios in January , treating fans to a performance that was personal and was both playful and emotional at times .

The musician had seemed to be shedding his reclusive reputation .
He hosted several late - night jam sessions where he serenaded Madonna , celebrated the Minnesota Lynx ' s WNBA championship and showcased his latest protege , singer Judith Hill .

Ever surprising , he announced on stage in New York City last month that he was writing his memoir .
" The Beautiful Ones " was expected to be released in the fall of 2017 by publishing house Spiegel & Grau .
The publishing house has not yet commented on status of book , but a press release about the memoir says : " Prince will take readers on an unconventional and poetic journey through his life and creative work ."
It says the book will include stories about Prince ' s music and " the family that shaped him and the people , places , and ideas that fired his creative imagination ."

The Associated Press contributed to this report .

## K-Means Clustering

We can cluster our documents by separating them into $k$ groups that are relatively close together, defining the cluster by the _center_ or mean of all the vectors. We have to know how many clusters we want a priori but the algorithm is guaranteed to converge with stable centers. _____no_output_____

import numpy as np
from sklearn.cluster import KMeans
from yellowbrick.cluster import SilhouetteVisualizer

tfidf = TfidfVectorizer()
docs  = tfidf.fit_transform(corpus.raw_data())
model = KMeans(n_clusters=8)

silhouette = SilhouetteVisualizer(model)
silhouette.fit(docs)
silhouette.poof()_____no_output_____from sklearn.cluster import MiniBatchKMeans
from yellowbrick.cluster import KElbowVisualizer

elbow = KElbowVisualizer(MiniBatchKMeans(), k=(4, 12))
elbow.fit(docs)
elbow.poof()_____no_output_____

## Distance

There are more ways than one to measure distance!

![Distance Metrics](figures/s3_distance_table.png)_____no_output_____### Minkowski Distance

![Minkowski Distance](figures/s3_minkowski.png)_____no_output_____### Mahalanobis Distance

![Mahalanobis Distance](figures/s3_mahalanobis.png)_____no_output_____### Jaccard Distance

![Jaccard Distance](figures/s3_jaccard.png)_____no_output_____### Edit Distance

![Edit Distance](figures/s3_edit_distance.png)_____no_output_____### Cosine Distance

![Cosine Distance](figures/s3_cosine_distance.png)_____no_output_____## LDA

We can use Latent Dirichlet Allocation (LDA) to create a generative statistical model of similarity in text. This method is a Bayesian method that uses a sparse Dirichlet prior for a small set of topics; however the number of topics does not have to be known in advance._____no_output_____

from gensim.corpora.dictionary import Dictionary
from gensim.models.ldamodel import LdaModel
from gensim.corpora import MmCorpus
from utils.normalize import normalize_text

# Create the dictionary with preprocessed text
docs = list(normalize_text(corpus.docs()))
id2word = Dictionary(docs)
id2word.save("data/corpus.dict")
# id2word = Dictionary.load("data/corpus.dict")

# Save Matrix Market corpus to disk and load it
MmCorpus.serialize(
    "data/corpus.mm", [id2word.doc2bow(doc) for doc in docs]
)
mmcorpus = MmCorpus("data/corpus.mm")_____no_output_____# Run the LDA model
lda = LdaModel(corpus=mmcorpus, id2word=id2word, num_topics=12, update_every=1, passes=20)
lda.save("data/corpus.lda")

# lda = LdaModel.load("data/corpus.lda")_____no_output_____for topic_id, topic_terms in lda.print_topics():
    print(topic_terms + "\n")0.008*"show" + 0.006*"episode" + 0.006*"year" + 0.005*"like" + 0.005*"water" + 0.005*"new" + 0.004*"time" + 0.004*"make" + 0.004*"one" + 0.004*"book"

0.022*"clinton" + 0.021*"sanders" + 0.017*"state" + 0.017*"vote" + 0.016*"primary" + 0.014*"voter" + 0.013*"trump" + 0.012*"delegate" + 0.011*"democratic" + 0.011*"party"

0.022*"say" + 0.006*"people" + 0.005*"year" + 0.005*"state" + 0.004*"police" + 0.004*"one" + 0.004*"law" + 0.004*"case" + 0.004*"also" + 0.004*"tell"

0.012*"may" + 0.012*"june" + 0.009*"nfl" + 0.008*"april" + 0.007*"team" + 0.007*"report" + 0.006*"new" + 0.006*"1" + 0.006*"24" + 0.005*"offseason"

0.009*"get" + 0.009*"say" + 0.008*"one" + 0.008*"go" + 0.007*"like" + 0.007*"make" + 0.007*"game" + 0.006*"people" + 0.006*"know" + 0.005*"think"

0.008*"say" + 0.008*"woman" + 0.006*"film" + 0.006*"one" + 0.006*"like" + 0.005*"new" + 0.005*"time" + 0.005*"show" + 0.004*"movie" + 0.004*"make"

0.015*"say" + 0.013*"trump" + 0.007*"would" + 0.006*"president" + 0.005*"u" + 0.005*"obama" + 0.004*"year" + 0.004*"make" + 0.004*"government" + 0.004*"new"

0.006*"make" + 0.006*"one" + 0.005*"get" + 0.005*"play" + 0.005*"like" + 0.005*"would" + 0.004*"two" + 0.004*"day" + 0.004*"say" + 0.003*"take"

0.012*"company" + 0.007*"data" + 0.007*"use" + 0.006*"new" + 0.006*"make" + 0.005*"business" + 0.005*"also" + 0.005*"like" + 0.004*"$" + 0.004*"one"

0.011*"year" + 0.010*"game" + 0.009*"season" + 0.009*"team" + 0.008*"1" + 0.007*"last" + 0.006*"first" + 0.006*"2" + 0.006*"point" + 0.006*"3"

0.013*"say" + 0.008*"trump" + 0.007*"school" + 0.007*"mr" + 0.006*"new" + 0.006*"rubio" + 0.005*"news" + 0.004*"student" + 0.004*"tattoo" + 0.004*"florida"

0.007*"make" + 0.007*"food" + 0.007*"cosby" + 0.006*"2016" + 0.004*"1" + 0.004*"use" + 0.004*"add" + 0.003*"recipe" + 0.003*"one" + 0.003*"v"

**Note**: New requirement: pyLDAvis — install via:

```
$ pip install pyldavis
```

For simplicity, I've added this visualization to a separate notebook: `Session 3 Appendix: PyLDAVis`_____no_output_____```python
import pyLDAvis
import pyLDAvis.gensim

pyLDAvis.enable_notebook()
pyLDAvis.gensim.prepare(lda, mmcorpus, id2word)
```_____no_output_____![PyLDAVis](figures/s3_pyldavis.png)_____no_output_____## Text Classification

If we have documents labeled with a tag or class, we can train a model of text to identify those categorizations in other texts. E.g. we can detect bullying, positive or negative sentiment, product categories, etc. _____no_output_____![Classification Pipeline](figures/s3_classification_pipeline.png)_____no_output_____### Naive Bayes

Use Bayes rule to compute the conditional probability of a word to a class, then the product rule to chain all the probabilities of the words into a computation for the probability of the document to the class based on word features. Use a naive assumption to make the computation tractable. _____no_output_____

from sklearn.naive_bayes import MultinomialNB

model = Pipeline([
    ('tfidf', TfidfVectorizer()),
    ('clf', MultinomialNB()),
])

categories = ["cooking", "design", "tech", "books", "cinema", "data_science", "do_it_yourself", "gaming", "sports"]
model.fit(
    corpus.raw_data(categories=categories),
    np.array(list(corpus.labels(categories=categories)))
)_____no_output_____target_id, target_text = random_document(corpus, categories=categories)
print(model.predict([target_text]))
print(target_text)['sports']
Stephen Curry and the Golden State Warriors ( 70 - 9 ) are on the road to take on Zach Randolph and the Memphis Grizzlies ( 42 - 37 ) Saturday at the FedEx Forum .

The Warriors , who need to win their last three games to break the 1995 - 96 Chicago Bulls ' record of regular - season 72 wins , have outscored the Grizzlies by 66 points in two meetings .

Tipoff : 8 p . m .
ET

Coverage : ESPN

Injuries ( per CBS Sports ): Andrew Bogut , GSW ( out , rest ); Kevon Looney , GSW ( game - time decision , hip ); Jordan Adams , MEM ( out , knee ); Mike Conley , MEM ( out , Achilles ); Brandan Wright , MEM ( out , knee ); Marc Gasol , MEM ( out , foot )

Keys to the Game

The Warriors defense will have to be stout inside against Randolph , who ' s averaging 19 . 4 points and 7 . 9 rebounds since returning from injury on March 19 .

The Grizzlies , meanwhile , have to find some way to frustrate Curry .
With Tony Allen out ( per NBATV ' s Jared Greenberg ), some of the responsibility may fall to Lance Stephenson .
probas = model.predict_proba([target_text])
table = [["Class", "Probability"]] + list(zip(model.classes_, probas[0]))
print(tabulate(table))| Class          |   Probability |
|:---------------|--------------:|
| books          |    0.0164769  |
| cinema         |    0.0460481  |
| cooking        |    0.00455754 |
| data_science   |    0.00868666 |
| design         |    0.0101377  |
| do_it_yourself |    0.0361462  |
| gaming         |    0.0712569  |
| sports         |    0.677727   |
| tech           |    0.128963   |

## Evaluating Classification

Because we have the correct labels, we can evaluate the accuracy of our model; however we do not want to evaluate data we trained the model on (otherwise we might overfit our model and it will not generalize). Evaluation methods include:

- Divide data into train and test splits; fit on train and score on test.
- Cross-Validation to split into $k$ chunks, train and score multiple times and take average.
_____no_output_____

from sklearn.model_selection import train_test_split as tts

tfidf = TfidfVectorizer()
docs = tfidf.fit_transform(corpus.raw_data(categories=categories))

X_train, X_test, y_train, y_test = tts(
    docs,
    np.array(list(corpus.labels(categories=categories))),
    test_size = 0.2,
)_____no_output_____

### Precision, Recall, F1 Score

![Precision and Recall](figures/s3_precision_recall.jpg)_____no_output_____### Cross-Validation

![Cross Validation](figures/s3_cross_validation.png)_____no_output_____## The Model Selection Triple

How do you select the best classifier? It is essentially a search process where each item in the search is a triple composed of:

- Feature Engineering: extract best features (e.g. bag of words, meta features, stopwords removal, etc.)
- Algorithm Selection: Bayes, ANNs, SVMs, Decision trees, Logistic Regression ... and more.
- Hyperparameter Tuning: what hyperparamaters perform the best for the model._____no_output_____

import matplotlib.pyplot as plt

from yellowbrick.classifier import ClassificationReport
from yellowbrick.classifier import ConfusionMatrix
from yellowbrick.classifier import ROCAUC

def evaluate_model(model):
    fig, axes = plt.subplots(ncols=2, figsize=(12,6))

    model.fit(X_train, y_train)
    cr = ClassificationReport(model, ax=axes[1])
    cr.score(X_test, y_test)
    cr.finalize()

    cm = ROCAUC(model, ax=axes[0], micro=False, macro=False)
    cm.score(X_test, y_test)
    cm.finalize()_____no_output_____evaluate_model(MultinomialNB())/Users/benjamin/Workspace/ddl/brookings/venv/lib/python3.6/site-packages/sklearn/metrics/classification.py:1135: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
  'precision', 'predicted', average, warn_for)

## Support Vector Machines

Linear SVMs are widely used for text classification, though they can be slow to compute. SVMs draw "support vectors", orthogonal lines through high dimensional space against each of the classes. The use these vectors to compute a hyperplane that divides the space between the two classes with equal distance from each. _____no_output_____

from sklearn.svm import SVC
from sklearn.linear_model import SGDClassifier

# Use SGD (stochastic gradient descent) for better performance on text
evaluate_model(SGDClassifier())/Users/benjamin/Workspace/ddl/brookings/venv/lib/python3.6/site-packages/sklearn/linear_model/stochastic_gradient.py:128: FutureWarning: max_iter and tol parameters have been added in <class 'sklearn.linear_model.stochastic_gradient.SGDClassifier'> in 0.19. If both are left unset, they default to max_iter=5 and tol=None. If tol is not None, max_iter defaults to max_iter=1000. From 0.21, default max_iter will be 1000, and default tol will be 1e-3.
  "and default tol will be 1e-3." % type(self), FutureWarning)

## Logistic Regression

In text analysis is a substitute for maximum entropy models - uses the logit model to estimate the probability of a binary dependent variable. In multi-class classification uses one vs. rest or one vs. one to make computations._____no_output_____

from sklearn.linear_model import LogisticRegression

evaluate_model(LogisticRegression())/Users/benjamin/Workspace/ddl/brookings/venv/lib/python3.6/site-packages/sklearn/metrics/classification.py:1135: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
  'precision', 'predicted', average, warn_for)

## Decision Trees and Random Forests

Use an inductive method to create a graphical tree of rules that can be used to make decisions. Random forests are the modern extension that uses an ensemble of weaker multiple trees, aggregating (bagging) them together to make a better global decision, or boosting the trees with a regression (gradient boosting) to weight the performance of individual models on their random subsets of data. _____no_output_____

from sklearn.ensemble import RandomForestClassifier

evaluate_model(RandomForestClassifier())_____no_output_____

## Artificial Neural Networks

Inspired by biological neurons, ANNs are in fact universal function approximators that use a network of activating neurons based on input to make decisions. ANNs have been recently extended to multiple layers - deep learning and because of advances in computing power, are widely regarded as the state of the art for text processing._____no_output_____

from sklearn.neural_network import MLPClassifier

evaluate_model(MLPClassifier())_____no_output_____

## Other Visual Diagnostics

- Class Balance
- Class Prediction Error
- Confusion Matrix _____no_output_____

from yellowbrick.classifier import ClassBalance

viz = ClassBalance(MultinomialNB())
viz.fit(X_train, y_train)
viz.score(X_test, y_test)
viz.poof()/Users/benjamin/Workspace/ddl/brookings/venv/lib/python3.6/site-packages/sklearn/metrics/classification.py:1135: UndefinedMetricWarning: Precision and F-score are ill-defined and being set to 0.0 in labels with no predicted samples.
  'precision', 'predicted', average, warn_for)
from yellowbrick.classifier import ClassPredictionError

viz = ClassPredictionError(LogisticRegression())
viz.fit(X_train, y_train)
viz.score(X_test, y_test)
viz.poof()_____no_output_____from yellowbrick.classifier import ConfusionMatrix

viz = ConfusionMatrix(LogisticRegression())
viz.fit(X_train, y_train)
viz.score(X_test, y_test)
viz.poof()_____no_output_____

## Sentiment Analysis

Sentiment analysis is simply a classifier trained on either sentences or paragraphs whose labels express the polarity of the emotional content of the statement, e.g. either "Positive or Negative" or using a Likert Scale:

- Very Negative
- Negative
- Neutral
- Positive
- Very Positive

Sentiment/Polarity is not a language feature like gender/tense - so the best sentiment models are trained on on a corpus you've generated yourself! _____no_output_____

from nltk.corpus import movie_reviews

movie_reviews.categories()_____no_output_____# The feature data, X are the raw documents from the corpus.
X = [
    movie_reviews.raw(fileids=fileid)
    for fileid in movie_reviews.fileids()
]

# The target data, y is the labels/categories from the corpus.
y = [
    movie_reviews.categories(fileids=fileid)[0]
    for fileid in movie_reviews.fileids()
]_____no_output_____# Train a multilayer perceptron model
model = Pipeline([
    ('tfidf', TfidfVectorizer()),
    ('clf', MLPClassifier())
])

model.fit(X, y)_____no_output_____model.predict(["This film doesn't care about cleverness, wit or any other kind of intelligent humor."])_____no_output_____X_train, X_test, y_train, y_test = tts(X, y, test_size=0.2)

cr = ClassificationReport(model)
cr.fit(X_train, y_train)
cr.score(X_test,  y_test)
cr.poof()_____no_output_____

## Advanced Sentiment Analysis

![Richard Socher](figures/s3_socher_sentiment.png)_____no_output_____Unlike gender or tense, sentiment is not a language feature; please do not create a white list of positive and negative words then count them up! You can see in the example above, negation can easily flip the polarity of sentences.

For those interested, you can read Richard Socher's paper, [Recursive Deep Models for Semantic Compositionality
Over a Sentiment Treebank](https://nlp.stanford.edu/~socherr/EMNLP2013_RNTN.pdf) to see the model of sentiment he created and how it works. _____no_output_____


# Notebook from Arthur1507/introML-2021
Path: exercises/01-python/intro_to_python_Sol.ipynb

# Introduction to Python - Solutions_____no_output_____

This notebook was initially made for the CS-233 Introduction to Machine Learning course at EPFL, and adapted for the CIVIL-226 Introduction to Machine Learning for Engineers course. Many thanks to Sena Kiciroglu and the Computer Vision Lab ([CVLab](https://www.epfl.ch/labs/cvlab)) for allowing us to use this notebook.

**Author(s):** [Sena Kiciroglu](mailto:sena.kiciroglu@epfl.ch), minor changes by [Tom Winandy](mailto:tom.winandy@epfl.ch) and [David Mizrahi](mailto:david.mizrahi@epfl.ch)

Welcome to the first exercise of Introduction to Machine Learning. Today we will get familiar with Python, the language we will use for all the exercises of this course.

This week we will introduce some important concepts in the basics of Python. Next week, you will learn how to work with NumPy, a popular Python library used for scientific computing.

Python is a popular language to use for machine learning tasks. This is especially true because of the selection of **libraries and frameworks**, developed specifically for machine learning and scientific computing. To name a few, you have Keras, TensorFlow and PyTorch for developing neural networks, SciPy and NumPy used for scientific computing, Pandas for data analysis, etc. (You might also get to dabble in PyTorch in the upcoming weeks.)

Python also allows you to write quick, readable, high-level code. It's great for fast prototyping.

You can find a useful Python cheatsheet at: https://www.pythoncheatsheet.org/

Let's get into it!
_____no_output_____## 1. Anaconda and Jupyter Notebook

If you're reading this Jupyter notebook as it is intended, chances are you already installed Anaconda, a Python distribution that comes with its own package management system, `conda`. Using `conda`, you can install and upgrade software packages and libraries. It will make managing the versions of the libraries you use very convenient.

In these exercises we will use Jupyter Notebooks, which contain Python code, text explanations and visuals.

The Jupyter Notebook document (such as the one you are looking at right now) consists of cells containing Python code, text or other content. You can run each cell by clicking on the button `Run` in the top toolbar, or you can use a keyboard shortcut `Ctrl` + `Enter` (run current cell) or `Shift` + `Enter` (run current cell and move to the cell below)._____no_output_____## 2. Indentation and Control Flow_____no_output_____Finally we get to start doing some coding!

First thing to know: Python does not separate different lines of code with a semicolon `;`. So just RUN the following cell with no worries._____no_output_____

# This is a Python comment. Start the line with `#` for a comment
print("First line of code. I will declare some variables")
a = 1 # second line!!
b = 2
c = "Fish"
print(f"My variables are: a = {a}, b = {b}, c = {c}")First line of code. I will declare some variables
My variables are: a = 1, b = 2, c = Fish

Easy! However, in Python you have to be careful and have perfect indentation (a reason why Python code is so readable). The reason is, Python uses indentation to keep track of what is part of the if statement, the loops and the functions. This is different from Java (this is assuming you know Java) where you would have curly brackets `{ }` for this purpose.

Let's start with the if statement.

### 2.1. If Statement

The rule is, all indented parts after the `if condition :` belong to that branch of the if statement.

```python
if condition :
    inside the statement
    still inside the statement
elif condition:
    inside the else-if part of the statement
else:
     inside the else part of the statement
outside the statement
 ```

Let's see it in action:_____no_output_____

if a + b == 3:
    print("It's me again! We are inside the first if statement")
    print("It's optional to use parentheses for the condition a + b ==3")
    print("Don't forget to put a `:` at the end of the condition!!")
    if (c == "Fish"):
        print("This is a second if statement inside the first one")
    print("I'm out of the second if statement, but still inside the first one")
else:
    print("This is the else part of the first if statement.")
    print("These lines will never be printed!")
print("I'm not inside any of the if statements")It's me again! We are inside the first if statement
It's optional to use parentheses for the condition a + b ==3
Don't forget to put a `:` at the end of the condition!!
This is a second if statement inside the first one
I'm out of the second if statement, but still inside the first one
I'm not inside any of the if statements

#### Exercise

Let's see another if statement example. Try to figure out what the output will be **BEFORE** running the cell below.

Reminder, we declared

```python
a = 1
b = 2
c = "Fish"
 ```_____no_output_____

# Don't run me until you find the output first!
if a == 5:
    print ("1")
    if b == 1:
        print("2")
# here comes an else-if
elif a == 2 or c == "Fish":
    print("3")

    if b == 1:
        print("4")
        if b == 2:
            print("5")
    if b == 2:
        print("6")
    if c == "Fish":
        if a == 1:
            if b == 100:
                print("7")
            else:
                print("8")
    elif a == 1:
        print("9")
print ("10")3
6
8
10

### 2.3. Loops

Let's talk about loops. The syntax for a while-loop is:

```python
while condition:
    inside the loop
    inside the loop
    inside the loop
outside the loop
 ```

 A small example:_____no_output_____

count = 0
while count < 3:
    count += 1 # this is the same as count = count +1
    print(f"Count is {count}")
print("Left the loop!")Count is 1
Count is 2
Count is 3
Left the loop!

For-loops iterate through sequences, in this way:

```python
for x in sequence:
    inside the loop
    inside the loop
    inside the loop
outside the loop
```

 An example is shown below:_____no_output_____

#Here is a basic list of strings
fish_list = ["salmon", "trout", "parrot", "clown", "dory"]

#The for-loop:
for fish in fish_list:
    print(fish)
    print("*")
print("fish list over!")salmon
*
trout
*
parrot
*
clown
*
dory
*
fish list over!

An incredibly useful built-in function to use in for-loops is `range()`. Range allows you to create a sequence of integers from the start (default is 0), to the stop, with a given step size (default is 1). We can use `range()` in for-loops as shown in the example below._____no_output_____

# "default start is 0, default step size is 1"
for number in range(7):
    print (number)
print("**")

# now we also provide the start as 2.
# Default step size 1 is still used.
for number in range(2,7):
    print(number)
print("**")

# now we also provide the step size as 2.
for number in range(2,7,2):
    print(number)
print("**")

# what happens if step size is -1?
for number in range(6,-1,-1):
    print(number)0
1
2
3
4
5
6
**
2
3
4
5
6
**
2
4
6
**
6
5
4
3
2
1
0

One more useful built-in function will be `enumerate()`. Let's go back to the fish list.
_____no_output_____

for fish in fish_list:
    print(fish)salmon
trout
parrot
clown
dory

What if I also want to keep track of the index of the list element? You can use `enumerate()` which creates a sequence of 2-tuples, where each tuple contains an integer index and an actual element of the original list. Here is how it looks like:_____no_output_____

for item_index, fish in enumerate(fish_list):
    print(f"{item_index}: {fish}")0: salmon
1: trout
2: parrot
3: clown
4: dory

## 3. Data Types and Basic Operations

Python is a **dynamically typed** language. This means that the data type is inferred at run-time and can be changed during run-time. To check the type of a variable you can use the function `type()`._____no_output_____

# var_1 is first defined as an integer
var_1 = 1
print(f"{var_1} is {type(var_1)}")

# var_1's type is changed to string
var_1 = "hi!"
print(f"{var_1} is {type(var_1)}")

# more types
var_1 = 0.312
print(f"{var_1} is {type(var_1)}")
var_1 = 3.
print(f"{var_1} is {type(var_1)}")
var_1 = 3+2j
print(f"{var_1} is {type(var_1)}")
var_1 = True
print(f"{var_1} is {type(var_1)}")1 is <class 'int'>
hi! is <class 'str'>
0.312 is <class 'float'>
3.0 is <class 'float'>
(3+2j) is <class 'complex'>
True is <class 'bool'>

### 3.1. Type Casting

Some examples of type casting in Python:_____no_output_____

# From int to float
var_1 = 42
print(f"{var_1} is {type(var_1)}")
var_1 = float(var_1)
print(f"{var_1} is {type(var_1)}")
print ("**")

# From float to int
var_2 = 3.14
print(f"{var_2} is {type(var_2)}")
var_2 = int(var_2)
# This operations does FLOOR, not round!
print(f"{var_2} is {type(var_2)}")
print ("**")

# From string to int
var_3 = "100"
print(f"{var_3} is {type(var_3)}")
var_3 = int(var_3)
print(f"{var_3} is {type(var_3)}")
print("**")

# From float to string
var_4 = 1.23
print(f"{var_4} is {type(var_4)}")
var_4 = str(var_4)
print(f"{var_4} is {type(var_4)}")
print("**")
42 is <class 'int'>
42.0 is <class 'float'>
**
3.14 is <class 'float'>
3 is <class 'int'>
**
100 is <class 'str'>
100 is <class 'int'>
**
1.23 is <class 'float'>
1.23 is <class 'str'>
**

### 3.2. Basic Operations

Arithmetic operations are fairly standard. There are some examples below.
* Look out for the difference between `/` division and `//` integer division.
* `**` is used for power.
* `%` is modulo._____no_output_____

a = 50
b = 7

print(f"a + b = {a + b}")
print(f"a - b = {a - b}")
print(f"a * b = {a * b}")
print(f"a / b = {a / b}")
print(f"a // b = {a // b}") # integer divison
print(f"a ** b = {a ** b}") # power
print(f"a % b = {a % b}") # moduloa + b = 57
a - b = 43
a * b = 350
a / b = 7.142857142857143
a // b = 7
a ** b = 781250000000
a % b = 1

Boolean operations are also fairly standard:_____no_output_____

print(f"(True and False) = {True and False}")
print(f"(True or False) = {True or False}")
print(f"((True and False) or True) = {(True and False) or True}")(True and False) = False
(True or False) = True
((True and False) or True) = True

You can declare strings with a single quote `'`, a double quote `"` or a three double quotes `"""`. The string declared with `"""` is known as a *docstring*, it can span multiple lines and is usually used to comment functions and classes.

**Note:** Throughout the exercises, we will be using f-strings to format our strings nicely. You can learn more about them [here](https://realpython.com/python-f-strings/)._____no_output_____

a = 'Life\'s but a walking shadow, a poor player,'
print(a)
a = "That struts and frets his hour upon the stage,"
print(a)
a = """And then is heard no more. It is a tale
Told by an idiot, full of sound and fury,
Signifying nothing."""
print(a)Life's but a walking shadow, a poor player,
That struts and frets his hour upon the stage,
And then is heard no more. It is a tale
Told by an idiot, full of sound and fury,
Signifying nothing.
# The types of quotes do not change anything!
a = "fish" # double quote
b = 'fish' # single quote
print(a == b) # the string is the same!True

## 4. Lists

Lists are data types containing a sequence of values. The size of the list can change during run-time, as you add and remove elements from the list.

Here is how you can create lists:_____no_output_____

list_a = [] # empty
print(f"list_a {list_a}")

list_b = [1, 2, 3, 4] # 4 elements
print(f"list_b {list_b}")

list_c = [1, 'cat', 0.23] # mixed types
print(f"list_c {list_c}")

list_d = [1, ['cat', 'dog'], 2, 3] # list in list
print(f"list_d {list_d}")

list_e = [1] * 10 # a list of 1s of length 10
print(f"list_e {list_e}")

list_f = list(range(5)) # turns range object into a list
print(f"list_f {list_f}")list_a []
list_b [1, 2, 3, 4]
list_c [1, 'cat', 0.23]
list_d [1, ['cat', 'dog'], 2, 3]
list_e [1, 1, 1, 1, 1, 1, 1, 1, 1, 1]
list_f [0, 1, 2, 3, 4]

Below we introduce some common operations with lists.
* Use `len(list1)` to find the length of the list.
* `list1.append(element)` to add an element to the end of the list.
* `list1.insert(index, element)` to add an element to an index in the list
* `list1.extend(list2)` to extend the elements of list1 with the elements of list2
* `list1.pop()` removes last element from the list
* `list1.pop(index)` removes the element at the given index
* `list1.remove(element)` removes the first instance of the given element_____no_output_____

# Some common operations
b = ["great", "minds", "think", "alike"]
print(f"b: {b}")

# finding the length
print(f"length of b is {len(b)}")

# append element to list
b.append("sometimes")
print(f"b.append(\"sometimes\")= {b}")

# extend list
c = ["-", "Abraham", "Lincoln"]
b.extend(c)
print(f"c: {c}")
print(f"b.extend(c) = {b}")

# removes element and specific index
b.pop(6)
print(f"b.pop(6) = {b}")

# remove specific element
b.remove("Lincoln")
b.remove("-")
print(f"b.remove(\"Lincoln\"); b.remove(\"-\") = {b}")
b: ['great', 'minds', 'think', 'alike']
length of b is 4
b.append("sometimes")= ['great', 'minds', 'think', 'alike', 'sometimes']
c: ['-', 'Abraham', 'Lincoln']
b.extend(c) = ['great', 'minds', 'think', 'alike', 'sometimes', '-', 'Abraham', 'Lincoln']
b.pop(6) = ['great', 'minds', 'think', 'alike', 'sometimes', '-', 'Lincoln']
b.remove("Lincoln"); b.remove("-") = ['great', 'minds', 'think', 'alike', 'sometimes']

You  can also check whether an element is in a list in the following way:_____no_output_____

list_1 = ["a", "b", "c"]
if "b" in list_1:
    print("\"b\" is in list")
else:
    print("\"b\" is not in list")"b" is in list

### 4.1. List Indexing and Slicing:

You can extract a single element from a list in the following way:
`list1[index]`

In lists, the indices start from 0. You can also index elements from the end of the list to the beginning by $-1, -2, -3...$. Check out the image below for the example list:

`list_1 = ["a", "b", "c", "d", "e"]`_____no_output__________no_output_____* You can extract multiple elements by slicing. This will give you elements from the start up to **(but not including)** the end index.

  `list1[start_index:end_index]`

* If you do not specify the `start_index`, you will retrieve the elements from index $0$ up to the `end_index`.

  `list1[:end_index]` is the same as `list1[0:end_index]`

* If you do not specify the `end_index`, you will retrieve the elements from the `start_index` up to (and **including**) the end of the list.

  `list1[start_index:]`

* You can provide a step size.
  `list1[start_index:end_index:step_size]`
  _____no_output_____#### Exercise

Try to write the output of the following code **BEFORE** running the cell._____no_output_____

# Don't run BEFORE you solve it!
list_1 = ["a", "b", "c", "d", "e"]

print(f"list_1[-3] = {list_1[-3]}")
print(f"list_1[0:2] = {list_1[0:2]}")
print(f"list_1[:4:2] = {list_1[:4:2]}")
print(f"list_1[::-1] = {list_1[::-1]}")
print(f"list_1[-4:-1] = {list_1[-4:-1]}")list_1[-3] = c
list_1[0:2] = ['a', 'b']
list_1[:4:2] = ['a', 'c']
list_1[::-1] = ['e', 'd', 'c', 'b', 'a']
list_1[-4:-1] = ['b', 'c', 'd']

You can also assign new values to indices using slicing. Here is an example:_____no_output_____

list_1 = ["a", "b", "c", "d", "e"]

list_1[-1]= "<3"
print(list_1)

list_1[0:2] = ["x", "y"]
print(list_1)

list_1[::2] = [":)",":(", ":O"]
print(list_1)['a', 'b', 'c', 'd', '<3']
['x', 'y', 'c', 'd', '<3']
[':)', 'y', ':(', 'd', ':O']

### 4.2. Copying

We have one last thing to say about lists. Observe the behaviour of the following code:_____no_output_____

# Case 1:

list_1 = ["a", "b", "c", "d", "e"]
print(f"list_1 before {list_1}")

list_2 = list_1
list_2.append("Z")

print(f"list_1 after {list_1}")list_1 before ['a', 'b', 'c', 'd', 'e']
list_1 after ['a', 'b', 'c', 'd', 'e', 'Z']
# Case 2:

list_1 = ["a", "b", "c", "d", "e"]
print(f"list_1 before function {list_1}")

def function_that_changes_list(input_list):
    input_list.append("Z")

function_that_changes_list(list_1)

print(f"list_1 after function {list_1}")list_1 before function ['a', 'b', 'c', 'd', 'e']
list_1 after function ['a', 'b', 'c', 'd', 'e', 'Z']

We never changed list_1 explicitly, but the values changed anyway. What's going on?

Well, in Python, when you say `list_2 = list_1`, you are not actually creating a new list, you are only copying the **reference** to the same list. This means that they are actually two variables pointing to the same list! So when you change the values of `list_2`, the values of `list_1` also change (since they are referring to the same list). Something similar is at play when you pass this list to a function. So be careful!

If you do not want this to happen, you can use the function `.copy()` to create a new object with the same values.

#### Exercise

Change the code below and fix the two cases given above using the `.copy()` function. Make sure the contents of `list_1` do not change._____no_output_____

# Case 1
list_1 = ["a", "b", "c", "d", "e"]
print(f"list_1 before {list_1}")

list_2 = list_1.copy()
list_2.append("Z")

print(f"list_1 after {list_1}")
print("**")

# Case 2
list_1 = ["a", "b", "c", "d", "e"]
print(f"list_1 before function {list_1}")

def function_that_changes_list(input_list):
    input_list.append("Z")

list_2 = list_1.copy()
function_that_changes_list(list_2)

print(f"list_1 after function {list_1}")list_1 before ['a', 'b', 'c', 'd', 'e']
list_1 after ['a', 'b', 'c', 'd', 'e']
**
list_1 before function ['a', 'b', 'c', 'd', 'e']
list_1 after function ['a', 'b', 'c', 'd', 'e']

#### Exercise

Now that we know how lists work, here is a quick exercise for you. Fill in the function below that takes a list and returns True if it is a palindrome, False if it is not. Palindromes are defined as sequences that read the same forwards and backwards.
Examples of palindrome lists:
* ["cat", "dog", "fish", "dog", "cat"]
* [0, 1, 2, 3, 3, 2, 1, 0]
* [1]
* []

You may use a for-loop in this exercise. However, if you're feeling ambitious try to do it in 1 line, without using a for-loop (hint: use slicing)_____no_output_____

# with for-loop
def function_is_palindrome(input_list):
    is_palindrome = True

    # Your code here
    len_of_list = len(input_list) // 2
    for element in range(len_of_list):
        if input_list[element]  != input_list[-element-1]:
            is_palindrome = False
    return is_palindrome_____no_output_____# without for-loop
def function_is_palindrome(input_list):
    return input_list == input_list[::-1]_____no_output_____test_list_1 = ["cat", "dog", "fish", "dog", "cat"]
res_1 = function_is_palindrome(test_list_1)

test_list_2 = ["cat", "dog", "fish",  "bird", "dog", "cat"]
res_2 = function_is_palindrome(test_list_2)

test_list_3 = ["cat"]
res_3 = function_is_palindrome(test_list_3)

test_list_4 = ["cat", "cat"]
res_4 = function_is_palindrome(test_list_4)

if not (res_1 and not res_2 and res_3 and res_4):
    print("Test failed")
else:
    print("Correct! :)")
Correct! :)

## 5. Tuples

Tuples are similar to lists but they are fixed in size and **immutable**, which means that change is not allowed.
We declare tuples in the following way using parentheses`()`:_____no_output_____

tuple_1 = ("wash", "your", "hands", "with", "soap")

print(f"tuple_1 = {tuple_1}")tuple_1 = ('wash', 'your', 'hands', 'with', 'soap')

Since change is not allowed, observe the result of the following piece of code._____no_output_____

tuple_1[2] = ("face")_____no_output_____

You can typecast from list to tuple and vice versa! _____no_output_____

sequence_1 = ["here", "comes", "the", "sun"]
print(f"{sequence_1} is {type(sequence_1)}")

# from list to tuple
sequence_1 = tuple(sequence_1)
print(f"{sequence_1} is {type(sequence_1)}")

#from tuple to list
sequence_1 = list(sequence_1)
print(f"{sequence_1} is {type(sequence_1)}")['here', 'comes', 'the', 'sun'] is <class 'list'>
('here', 'comes', 'the', 'sun') is <class 'tuple'>
['here', 'comes', 'the', 'sun'] is <class 'list'>

## 6. Dictionaries

An incredibly useful data type to know, you might also know dictionaries as "hash maps". Dictionaries are collections of "key: value" pairs. You can access the values using the keys in $O(1)$ time.

The keys of a dictionary must be **immutable** and **unique**. Below we show how to define a dictionary.
_____no_output_____

shopping_list = {"apples": 3, "pears":2, "eggs":6, "bread":1, "yogurt":1}
print(f"shopping_list = {shopping_list}")
print("**")

book_dict = {}
print(f"book_dict = {book_dict}")
#add key value pairs
book_dict["vonnegut"] = "cat\'s cradle"
book_dict["ishiguro"] = "never let me go"
print(f"book_dict = {book_dict}")
print("**")

# we can retrieve the dict keys:
print(book_dict.keys())
# and the dict values:
print(book_dict.values())
print("**")

#we can also iterate through the dict keys and values with a for-loop
for key, value in book_dict.items():
    print(f"{key} : {value}")

print("**")
#we can modify the value of a key
book_dict["ishiguro"] = "a pale view of hills"
print(f"modified book_dict = {book_dict}")
print("**")

#and we can remove a key completely
removed_value = book_dict.pop("ishiguro")
print(f"book_dict with removed value = {book_dict}")
print(f"removed_value = {removed_value}")shopping_list = {'apples': 3, 'pears': 2, 'eggs': 6, 'bread': 1, 'yogurt': 1}
**
book_dict = {}
book_dict = {'vonnegut': "cat's cradle", 'ishiguro': 'never let me go'}
**
dict_keys(['vonnegut', 'ishiguro'])
dict_values(["cat's cradle", 'never let me go'])
**
vonnegut : cat's cradle
ishiguro : never let me go
**
modified book_dict = {'vonnegut': "cat's cradle", 'ishiguro': 'a pale view of hills'}
**
book_dict with removed value = {'vonnegut': "cat's cradle"}
removed_value = a pale view of hills

## 7. Functions

You can define a function in Python in the following way:_____no_output_____

def multiply(a, b):
    return a * b

print(f"multiply(100, 2) = {multiply(100, 2)}")multiply(100, 2) = 200

You can have default arguments by specifying their default value in the parameters._____no_output_____

def add(a, b, c=0, d=1):
    return a + b + c + d

# use no default arguments
print(f"add(1, 2, 100, 1000) = {add(1, 2, 100, 1000)}")

# use the default value of d
print(f"add(1, 2, 100) = {add(1, 2, 100)}")

# use the default value of c and d
print(f"add(1, 2) = {add(1, 2)}")

# use the default value of c
print(f"add(1, 2, d=1000) = {add(1, 2, d=1000)}")add(1, 2, 100, 1000) = 1103
add(1, 2, 100) = 104
add(1, 2) = 4
add(1, 2, d=1000) = 1003

A function can return multiple values in a tuple. You can assign the values of the tuple to separate variables. This is called **tuple unpacking**._____no_output_____

def min_max(input_list):
    return min(input_list), max(input_list)

test_list = [1,2,3,4]
min_val, max_val = min_max(test_list)
print(f"min_val: {min_val}, max_val: {max_val}")min_val: 1, max_val: 4

Note: You have seen tuple unpacking when using function `enumerate` in for-loop._____no_output_____### 7.1. Common Built-in Functions

Here we introduce some nifty commonly used built-in functions.

* You already learned `range()`, `enumerate()`!
* We have also seen `type()` to return the type of the object. We use `str()`, `int()`, `float()`, `list()`, `tuple()` for typecasting.
* The functions `len()`, `sum()`, `min()`, `max()`, `any()`, `all()`, `sorted()`, `zip()` are useful for lists and tuples.

Let's see them in action below_____no_output_____

list_1 = list(range(5))
print(f"list_1 = {list_1}")

print(f"len(list_1) = {len(list_1)}")
print(f"sum(list_1) = {sum(list_1)}")
print(f"min(list_1) = {min(list_1)}")
print(f"max(list_1) = {max(list_1)}")
print("**")

list_2 = [5,3,1,2,0,6]
print(f"list_2 = {list_2}")
print(f"sorted(list_2) = {sorted(list_2)}")
print("**")

# any checks whether there are any 1s in the list (OR)
# all checks whether all elements are 1s. (AND)
# in Python: 1 = True, 0 = False
list_3 = [1, 1, 1]
print(f"list_3 = {list_3}")
print(f"any(list_3) = {any(list_3)}")
print(f"all(list_3) = {all(list_3)}")

list_4 = [0, 1, 1]
print(f"list_4 = {list_4}")
print(f"any(list_4) = {any(list_4)}")
print(f"all(list_4) = {all(list_4)}")

list_5 = [0, 0, 0]
print(f"list_5 = {list_5}")
print(f"any(list_5) = {any(list_5)}")
print(f"all(list_5) = {all(list_5)}")
print("**")

# zip function:
x = [1,2,3]
y = [4,5,6]
zipped = zip(x,y)
print(f"x {x}")
print(f"y {y}")
print(f"zipped {list(zipped)}")list_1 = [0, 1, 2, 3, 4]
len(list_1) = 5
sum(list_1) = 10
min(list_1) = 0
max(list_1) = 4
**
list_2 = [5, 3, 1, 2, 0, 6]
sorted(list_2) = [0, 1, 2, 3, 5, 6]
**
list_3 = [1, 1, 1]
any(list_3) = True
all(list_3) = True
list_4 = [0, 1, 1]
any(list_4) = True
all(list_4) = False
list_5 = [0, 0, 0]
any(list_5) = False
all(list_5) = False
**
x [1, 2, 3]
y [4, 5, 6]
zipped [(1, 4), (2, 5), (3, 6)]

### 7.2. List Comprehensions

One of the most practical things about Python is that you can do many things on just a single line. One popular example is so called *list comprehensions*, a specific syntax to create and initalize lists of objects. Here are some examples.

A syntax for list comprehension is shown below:
`[thing for thing in list]`

Let's make it more concrete with an example._____no_output_____

list_of_numbers = [1, 2, 3, 101, 102, 103]
print(f"list_of_numbers = {list_of_numbers}")

#I want to create a new list with all these items doubled.
doubled_list = [2 * elem for elem in list_of_numbers]
print(f"doubled_list = {doubled_list}")

#A new list with all these items as floats
float_list = [float(elem) for elem in list_of_numbers]
print(f"float_list = {float_list}")list_of_numbers = [1, 2, 3, 101, 102, 103]
doubled_list = [2, 4, 6, 202, 204, 206]
float_list = [1.0, 2.0, 3.0, 101.0, 102.0, 103.0]

Let's make it more interesting by adding an if in there:

`[thing for thing in list if condition]`
_____no_output_____

#I want to create a new list with all these items doubled
#IF the element is above 100
conditional_doubled_list = [2 * elem for elem in list_of_numbers if elem > 100]
print(f"conditional_doubled_list = {conditional_doubled_list}")conditional_doubled_list = [202, 204, 206]

#### Exercise

You will be given a list of vocabulary words. Your task is to use list comprehensions to iterate through a document and create a new list including the words that are included in the vocabulary. You don't need to worry about duplicates.

Example:
```python
vocabulary = ["a" "c", "e"]
document = ["a", "b", "c", "d"]
new_list = ["a", "c"]
```_____no_output_____

vocabulary = ['epfl', 'europe', 'swiss', 'switzerland', 'best', 'education', 'high', 'higher', 'research', 'school', 'science', 'students', 'technology', 'top-tier', 'university']

document = """The École polytechnique fédérale de Lausanne (EPFL) is a research institute
and university in Lausanne, Switzerland, that specializes in natural sciences and engineering.
It is one of the two Swiss Federal Institutes of Technology, and it has three main missions:
education, research and technology transfer at the highest international level. EPFL is widely regarded
as a world leading university. The QS World University Rankings ranks EPFL 12th in the world
across all fields in their 2017/2018 ranking, whilst Times Higher Education World
University Rankings ranks EPFL as the world's 11th best school for Engineering and Technology."""
document_parsed = document.split()
document_parsed = [word.lower() for word in document_parsed]
new_list = []

# Your code here
new_list = [word for word in document_parsed if (word in vocabulary)]

# We convert the list to a set and then back to a list. We do this because converting it to a set automatically
# removes duplicates (since sets are sequences that do not contain duplicates). afterwards we sort it.
new_list = sorted(list(set(new_list)))
_____no_output_____correct_result = ['best', 'education', 'epfl', 'higher', 'research', 'school', 'swiss', 'technology', 'university']

if new_list == correct_result:
    print ("Correct! :)")
else:
    print ("Incorrect :(")
Correct! :)

## 8. Object-Oriented Programming_____no_output_____Object-oriented programming is a programming paradigm that provides a means of structuring programs so that properties and behaviors are bundled into individual objects.

For this end, we use classes. Classes are used to create user-defined data structures. Classes define functions called methods, which identify the behaviors and actions that an object created from the class can perform with its data._____no_output_____All class definitions start with the class keyword, which is followed by the name of the class and a colon. Any code that is indented below the class definition is considered part of the class’s body. To start, let's declare an `EPFL_faculty` class._____no_output_____

class EPFL_faculty:

    def __init__(self, name, number_of_students):
        self.name = name
        self.number_of_students = number_of_students

    # Instance method
    def description(self):
        return f"The faculty {self.name} has {self.number_of_students} students"_____no_output_____

While the class is the blueprint, an instance is an object that is built from a class and contains real data. `.__init__()` sets the initial state of the object by assigning the values of the object’s properties. That is, `.__init__()` initializes each new instance of the class._____no_output_____

enac = EPFL_faculty("ENAC", 500)_____no_output_____

Now that the instance `enac` has been created, we can call the method description._____no_output_____

enac.description()_____no_output_____

Inheritance is the process by which one class takes on the attributes and methods of another. Newly formed classes are called child classes, and the classes that child classes are derived from are called parent classes. Child classes inherit from the parent's attributs and methods but it can overwrite methods. Let's define a class `EPFL_section` that inherits from `EPFL_faculty` and that overwrites the method `description(self)`._____no_output_____

class EPFL_section(EPFL_faculty):

    def description(self):
        return f"The section {self.name} has {self.number_of_students} students"_____no_output_____genie_civil = EPFL_section("GC", 200)_____no_output_____genie_civil.description()_____no_output_____

__Disclaimer__: This part of the tutorial and its text was inspired by and taken from "Object-Oriented Programming (OOP) in Python 3" by David Amos. _____no_output_____### Additional OOP resources_____no_output_____For further information on object-oriented programming and classes in Python, here are two useful resources:
* Object-Oriented Programming (OOP) in Python 3: https://realpython.com/python3-object-oriented-programming/
* Classes from the official Python Tutorial: https://docs.python.org/3/tutorial/classes.html_____no_output_____## 9. (OPTIONAL) Matplotlib_____no_output_____Perhaps the most widely used plotting library in Python is Matplotlib. If you've ever used MATLAB, you'll find that the functions look pretty similar.

In the following exercise sessions, we won't ask you to do any plotting. So this part is optional for those who are interested in having a short introduction.

First, we will import Matplotlib._____no_output_____### Importing in Python

* A short note on importing: to be able to use modules in our code, we import them.

  example: `import numpy`

* We can also select a name for the imported module.

  example: `import numpy as np`. Now when we call numpy functions, we will always use `np.` as a prefix, i.e. `np.zeros()`

* You can also choose to only import selected functions/variables/classes from the module.

  example: `from numpy import arange`. Now you can use this function as `arange(5)`. You cannot use any other functions from the numpy module as you did not import them._____no_output_____

# To import Matplotlib we do:

import matplotlib.pyplot as plt_____no_output_____

Let's do some plotting!

Let's start with the simplest of plots, the good old line-plot. The function we will use is `plot()`_____no_output_____

#Let's create some data first to plot
x = list(range(10))
y = [2,3,5,1,0,2,3,0,0,1]

#first create a figure
fig = plt.figure()

#now do the plotting
#specifying a color and marker are optional.
#check out the documentation to see what else you can do with the plot function
plt.plot(x, y, marker="*", color="r")

#axis labels and title
plt.xlabel("x")
plt.ylabel("y")
plt.title("just a random plot")

#so that we see the plot
plt.show()

#close the plot
plt.close(fig)_____no_output_____

#### Exercise

You can plot two lines on top of one another by calling the `plt.plot()` function consecutively. Try to implement this! Also, specify the parameter `label` of the `plt.plot()` function and call the function `plt.legend()` to create a legend for your graph. It should look like the figure shown below._____no_output_____![result](images/two_lines_plot.png)_____no_output_____

#Let's create some data first to plot
x = list(range(10))
y1 = [2,3,5,1,0,2,3,0,0,1]
y2 = [1,2,3,5,1,0,2,3,0,0]

#your code here
#first create a figure
fig = plt.figure()

#now do the plotting
#let's add labels for the legend
p1 = plt.plot(x, y1, marker="*", color="r", label="red line")
p2 = plt.plot(x, y2, marker="^", color="b", label="green line")
plt.legend()

#axis labels and title
plt.xlabel("x")
plt.ylabel("y")
plt.title("just a random plot")

#so that we see the plot
plt.show()
#close the plot
plt.close(fig)_____no_output_____

You can create scatter plots (line plots without lines) with `scatter()`_____no_output_____

#Let's create some data first to plot
x = list(range(10))
y1 = [2,3,5,1,0,2,3,0,0,1]

#first create a figure
fig = plt.figure()

#now do the plotting
p1 = plt.scatter(x, y1, marker="*", color="r")

#axis labels and title
plt.xlabel("x")
plt.ylabel("y")
plt.title("just a random scatter plot")

#so that we see the plot
plt.show()

#close the plot
plt.close(fig)_____no_output_____

And you can read and display images with `imread()` and `imshow()`_____no_output_____

#first create a figure
fig = plt.figure()

#now do the plotting
im = plt.imread("images/krabby_patty.jpg")
plt.imshow(im)

#axis labels and title
plt.xlabel("x")
plt.ylabel("y")
plt.title("krabby patty")

#so that we see the plot
plt.show()

#close the plot
plt.close(fig)_____no_output_____

And that's all for this tutorial! If you have any problems, just ask (or even Google) them. You can check out the official Python tutorials for further learning.

https://docs.python.org/3/tutorial/_____no_output_____