# Notebook from z-haiman/MASSpy
Path: docs/tutorials/reading_writing_models.ipynb

# Reading and Writing Models

In this notebook example, the import and export capabilities of **MASSpy** are demonstrated.

**MASSpy** supports reading and writing models in the [SBML](http://sbml.org/Main_Page) and [JSON](http://www.json.org/) formats. The preferred format for general use is the SBML with the FBC (Version 2) extension and the Groups (Version 1) extension.

The JSON format may be more useful for **MASSpy** specific functionality and for visualizing networks via [Escher](https://escher.github.io/). See the [Network Visualization](./network_visualization.ipynb) section for additional details.

The **MASSpy** package also comes with models in various formats for testing purposes._____no_output_____

from os.path import join

import mass
import mass.example_data

# To view the list of available models, remove the semicolon
mass.example_data.view_example_models();_____no_output_____

## SBML_____no_output_____

from mass.io import sbml_____no_output_____

The [Systems Biology Markup Language](http://sbml.org/Main_Page) is an XML-based standard format for distributing models.

**MASSpy** supports the reading and writing of SBML Level 3. **MASSpy** attempts to convert SBML Level 1 and Level 2 models to Level 3 before loading. _____no_output_____

model = sbml.read_sbml_model(join(mass.example_data.MODELS_DIR, "textbook.xml"))
modelSet parameter Username
sbml.write_sbml_model(model, "test_textbook.xml")_____no_output_____

**MASSpy** utilizes the [libSBML](http://sbml.org/Software/libSBML/5.18.0/docs/python-api/) package to read and write SBML files, supporting both the [FBC](http://sbml.org/Software/libSBML/5.18.0/docs/python-api/group__fbc.html) (Version 2) and the [Groups](http://sbml.org/Software/libSBML/5.18.0/docs/python-api/group__groups.html) (Version 1) extensions.
When reading in a model, **MASSpy** automatically detects whether the FBC and/or Groups extensions were used.

To preserve information specific to `EnzymeModule` objects, the SBML Groups extension is used along with the notes section for SBML objects. The `use_groups_package` argument can be utilized to indicate whether to write `cobra.Group` objects to the SBML file, including `EnzymeModule` information. Disabling this extension may result in a loss of some enzyme specific information (e.g., categorized groups), but it does not prevent species and reactions of the enzyme module
from being written.

When writing a model, the `use_fbc_package` argument can be used to indicate whether to write additional model information (e.g., metabolite formula and charge, genes, reaction bounds) via the FBC extension. _____no_output_____## JSON_____no_output_____

from mass.io import json_____no_output_____

**MASSpy** models have a [JSON](http://www.json.org/) representation, allowing for interoperability with the [Escher](https://escher.github.io/).

See the [Network Visualization](./network_visualization.ipynb) section for additional details on working with **Escher**._____no_output_____

model = json.load_json_model(join(mass.example_data.MODELS_DIR, "textbook.json"))
model_____no_output_____json.save_json_model(model, "test_textbook.json")_____no_output_____

Consider having the [simplejson](https://pypi.org/project/simplejson/) package to speed up reading/writing of JSON models._____no_output_____### JSON schema
The JSON schema for **MASSpy** models is stored in [mass.io.json](../autoapi/mass/io/json/index.rst#mass.io.json.JSON_SCHEMA) as the `JSON_SCHEMA` variable. It can be accessed via the following:_____no_output_____

# To view the JSON schema, remove the semicolon
json.JSON_SCHEMA;_____no_output_____

## Converting between file formats

Often there are times where one program or package is used to execute a specific task, yet a downstream task requires the use of a different program or package. Consequently, the ability to import a model written using one format and subsequently export it to another is essential in these scenarios. The submodules of ``mass.io`` can be used to facilitate the import/export of models in different formats for this purpose.

One possible scenario in which the conversion between file formats is necessary involves the visualizion of an SBML model on a network map using the [Escher](https://escher.github.io/) (King et al., 2015) visualization tool.

See [Visualizing SBML models with Escher in Python](./network_visualization.ipynb#Visualizing-SBML-models-with-Escher-in-Python) for an example demonstrating how **MASSpy** facilitates file conversion for model exchangability._____no_output_____


# Notebook from sylvchev/muse-lsl
Path: notebooks/SSVEP with Muse.ipynb

# Steady State Visual Evoked Potential

In this experiment, participants were presented with flashing gratings of two different frequencies.

Stimuli were presented for 3 s with an intertrial interval of 600 ms and random jitter of ±100ms. The task was to passively fixate the center of the screen while the stimuli were displayed. Six blocks of 2 min were recorded for a single partipant.

The two gratings were presented in a "pattern reversal" way, i.e., the grating was alternated with a reversed version of itself. For more info, see Section 2 of:

> Danhua Zhu, Jordi Bieger, Gary Garcia Molina, and Ronald M. Aarts, "A Survey of Stimulation Methods Used in SSVEP-Based BCIs," Computational Intelligence and Neuroscience, vol. 2010, 12 pages, 2010.

This led to stimulation frequencies of 30 and 20 Hz, respectively.

Moreover, an external electrode was added to the Muse headband to collect data at POz._____no_output_____

import os
import sys
from collections import OrderedDict

import pandas as pd
import numpy as np
import seaborn as sns
from matplotlib import pyplot as plt
%matplotlib inline

sys.path.append('../muse')
import utils_____no_output_____

## Load data into MNE objects_____no_output_____

subject = 1
session = 1
raw = utils.load_data('visual/SSVEP', sfreq=256.,
                      subject_nb=subject, session_nb=session,
                      ch_ind=[0, 1, 2, 3, 4],
                      replace_ch_names={'Right AUX': 'POz'})Creating RawArray with float64 data, n_channels=6, n_times=30732
    Range : 0 ... 30731 =      0.000 ...   120.043 secs
Ready.
Creating RawArray with float64 data, n_channels=6, n_times=30720
    Range : 0 ... 30719 =      0.000 ...   119.996 secs
Ready.
Creating RawArray with float64 data, n_channels=6, n_times=30732
    Range : 0 ... 30731 =      0.000 ...   120.043 secs
Ready.
Creating RawArray with float64 data, n_channels=6, n_times=30720
    Range : 0 ... 30719 =      0.000 ...   119.996 secs
Ready.
Creating RawArray with float64 data, n_channels=6, n_times=30732
    Range : 0 ... 30731 =      0.000 ...   120.043 secs
Ready.
Creating RawArray with float64 data, n_channels=6, n_times=30732
    Range : 0 ... 30731 =      0.000 ...   120.043 secs
Ready.

## Visualizing in the frequency domain

We epoch the raw data according to the conditions, and visualize the spectrum of each condition. We expect clear peaks in the spectral domain at the stimulation frequencies of 30 and 20 Hz._____no_output_____

from mne import Epochs, find_events

events = find_events(raw)
event_id = {'30 Hz': 1, '20 Hz': 2}

epochs = Epochs(raw, events=events, event_id=event_id,
                tmin=-0.5, tmax=4, baseline=None,
                reject={'eeg': 100e-6}, preload=True,
                verbose=False, picks=[0, 1, 2, 3, 4])197 events found
Events id: [1 2]
from mne.time_frequency import psd_welch

f, axs = plt.subplots(2, 1, figsize=(10, 10))
psd1, freq1 = psd_welch(epochs['30 Hz'], n_fft=1028, n_per_seg=256 * 3)
psd2, freq2 = psd_welch(epochs['20 Hz'], n_fft=1028, n_per_seg=256 * 3)
psd1 = 10 * np.log10(psd1)
psd2 = 10 * np.log10(psd2)

psd1_mean = psd1.mean(0)
psd1_std = psd1.mean(0)

psd2_mean = psd2.mean(0)
psd2_std = psd2.mean(0)

axs[0].plot(freq1, psd1_mean[[0, 3], :].mean(0), color='b', label='30 Hz')
axs[0].plot(freq2, psd2_mean[[0, 3], :].mean(0), color='r', label='20 Hz')

axs[1].plot(freq1, psd1_mean[4, :], color='b', label='30 Hz')
axs[1].plot(freq2, psd2_mean[4, :], color='r', label='20 Hz')

axs[0].set_title('TP9 and TP10')
axs[1].set_title('POz')
axs[0].set_ylabel('Power Spectral Density (dB)')
axs[1].set_ylabel('Power Spectral Density (dB)')
axs[0].set_xlim((2, 50))
axs[1].set_xlim((2, 50))
axs[1].set_xlabel('Frequency (Hz)')
axs[0].legend()
axs[1].legend()

plt.show()Effective window size : 4.016 (s)
Effective window size : 4.016 (s)

We can see the expected peaks in both temporo-parietal and posterior electrodes.

This can also be seen using a time-frequency plot:_____no_output_____

from mne.time_frequency import tfr_morlet

frequencies = np.logspace(1, 1.75, 60)
tfr, itc = tfr_morlet(epochs['30 Hz'], freqs=frequencies,
                      n_cycles=15, return_itc=True)
tfr.plot(picks=[4], baseline=(-0.5, -0.1), mode='logratio',
         title='POz - 30 Hz stim');

tfr, itc = tfr_morlet(epochs['20 Hz'], freqs=frequencies,
                      n_cycles=15, return_itc=True)
tfr.plot(picks=[4], baseline=(-0.5, -0.1), mode='logratio',
         title='POz - 20 Hz stim');Applying baseline correction (mode: logratio)

## Filtering

We filter data between 2 and 50 Hz._____no_output_____

raw_filt = raw.copy().filter(5, 50, method='iir')Setting up band-pass filter from 5 - 50 Hz

## Epoching

Here we epoch data for -100 ms to 3 s after the stimulus. No baseline correction is needed (signal is bandpass filtered) and we reject every epochs were the signal exceeds 100 uV._____no_output_____

events = find_events(raw)
event_id = {'30 Hz': 1, '20 Hz': 2}

epochs = Epochs(raw_filt, events=events, event_id=event_id,
                tmin=-0.3, tmax=3, baseline=None,
                reject={'eeg': 100e-6}, preload=True,
                verbose=False, picks=[0,4,2,3])197 events found
Events id: [1 2]

## Epoch average
_____no_output_____

conditions = OrderedDict()
conditions['30 Hz'] = [1]
conditions['20 Hz'] = [2]

fig, ax = utils.plot_conditions(epochs, conditions=conditions,
                                ci=97.5, n_boot=1, title='',
                                diff_waveform=None, ylim=(-8, 8))_____no_output_____

The stimulation pattern can also be seen in the temporal domain._____no_output_____## Decoding

We use a filter bank approach on the original 4 Muse electrodes (to see how the headband alone without external electrodes could be used to classify SSVEP):

1. Apply bandpass filters around both stimulation frequencies
2. Concatenate bandpass-filtered channels
2. Extract epochs (from 1 to 3 s after stimulus onset, to avoid classifying the ERP)
3. Apply common classification pipelines_____no_output_____

# Bandpass filter the raw data
muse_raw = raw.drop_channels(['POz'])
raw_filt_30Hz = muse_raw.copy().filter(25, 35, method='iir')
raw_filt_20Hz = muse_raw.copy().filter(15, 25, method='iir')
raw_filt_30Hz.rename_channels(lambda x: x + '_30Hz')
raw_filt_20Hz.rename_channels(lambda x: x + '_20Hz')

# Concatenate with the bandpass filtered channels
raw_all = raw_filt_30Hz.add_channels([raw_filt_20Hz],
                                    force_update_info=True)

# Extract epochs
events = find_events(raw_all)
event_id = {'30 Hz': 1, '20 Hz': 2}

epochs_all = Epochs(raw_all, events=events, event_id=event_id, tmin=1,
                     tmax=3, baseline=None, reject={'eeg': 100e-6},
                     preload=True, verbose=False, add_eeg_ref=False)Setting up band-pass filter from 25 - 35 Hz
Setting up band-pass filter from 15 - 25 Hz
197 events found
Events id: [1 2]
epochs_all.pick_types(eeg=True)
X = epochs_all.get_data() * 1e6
times = epochs.times
y = epochs_all.events[:, -1]_____no_output_____from sklearn.pipeline import make_pipeline

from mne.decoding import Vectorizer

from sklearn.linear_model import LogisticRegression
from sklearn.preprocessing import StandardScaler
from sklearn.discriminant_analysis import LinearDiscriminantAnalysis as LDA

from sklearn.model_selection import cross_val_score, StratifiedShuffleSplit

from pyriemann.estimation import Covariances, ERPCovariances, XdawnCovariances
from pyriemann.spatialfilters import CSP
from pyriemann.tangentspace import TangentSpace
from pyriemann.classification import MDM

from collections import OrderedDict

clfs = OrderedDict()

clfs['CSP + RegLDA'] = make_pipeline(Covariances(), CSP(4), LDA(shrinkage='auto', solver='eigen'))
clfs['Cov + TS'] = make_pipeline(Covariances(), TangentSpace(), LogisticRegression())
clfs['Cov + MDM'] = make_pipeline(Covariances(), MDM())
clfs['CSP + Cov + TS'] = make_pipeline(Covariances(), CSP(4, log=False), TangentSpace(), LogisticRegression())

# define cross validation
cv = StratifiedShuffleSplit(n_splits=20, test_size=0.25,
                            random_state=42)

# run cross validation for each pipeline
auc = []
methods = []
for m in clfs:
    print(m)
    try:
        res = cross_val_score(clfs[m], X, y==2, scoring='roc_auc',
                              cv=cv, n_jobs=-1)
        auc.extend(res)
        methods.extend([m]*len(res))
    except:
        pass

results = pd.DataFrame(data=auc, columns=['AUC'])
results['Method'] = methodsCSP + RegLDA
Cov + TS
Cov + MDM
CSP + Cov + TS
fig = plt.figure(figsize=[8,4])
sns.barplot(data=results, x='AUC', y='Method')
plt.xlim(0.4, 1)
sns.despine()_____no_output_____


# Notebook from kodaim1115/test
Path: src/.ipynb_checkpoints/process_snare-seq-checkpoint.ipynb

<code>
import numpy as np
import scipy.sparse
import scipy.io
import torch

import pandas as pd
from datasets_dev import RNA_Dataset, ATAC_Dataset, read_mtx_____no_output_____path = '../data/SNARE-seq/adult'_____no_output_____rna_path = path + '/RNA-seq'
atac_path = path + '/ATAC-seq'_____no_output_____r_dataset = RNA_Dataset(rna_path, min_reads=200,min_cells=20)Loading  data ...
Original data contains 10309 cells x 33160 peaks
Finished loading takes 0.22 min
r_dataset.data.shape_____no_output_____a_dataset = ATAC_Dataset(atac_path,low=0.003, high=0.1, min_peaks=200, binarize=True)Loading  data ...
Original data contains 10309 cells x 244544 peaks
Finished loading takes 0.61 min
a_dataset.data.shape_____no_output_____torch.save(r_dataset, path + '/r_dataset.rar') #r_dataset = RNA_Dataset(rna_path, min_reads=200,min_cells=20)_____no_output_____torch.save(a_dataset, path + '/a_dataset.rar') #a_dataset = ATAC_Dataset(atac_path,low=0.005, high=1, min_peaks=200, binarize=True)_____no_output_____torch.save(a_dataset, path + '/a_dataset_.rar') #a_dataset = ATAC_Dataset(atac_path,low=0.003, high=0.1, min_peaks=200, binarize=True)_____no_output_____r_dataset = torch.load( path + '/r_dataset.rar')_____no_output_____np.round(0.1).astype('int')_____no_output_____num_cell = r_dataset.data.shape[0]_____no_output_____t_size = np.round(num_cell*0.75).astype('int')
len(t_size)_____no_output_____t_id = np.random.choice(num_cell, size=t_size, replace=False)
t_id_____no_output_____s_id = np.delete(range(num_cell),t_id)
len(s_id)_____no_output_____
</code>


# Notebook from mfernandes61/python-intro-gapminder
Path: colab/16_writing_functions.ipynb

_____no_output_____---
title: "Writing Functions"
teaching: 10
exercises: 15
questions:
- "How can I create my own functions?"
objectives:
- "Explain and identify the difference between function definition and function call."
- "Write a function that takes a small, fixed number of arguments and produces a single result."
keypoints:
- "Break programs down into functions to make them easier to understand."
- "Define a function using `def` with a name, parameters, and a block of code."
- "Defining a function does not run it."
- "Arguments in call are matched to parameters in definition."
- "Functions may return a result to their caller using `return`."
---
## Break programs down into functions to make them easier to understand.

*   Human beings can only keep a few items in working memory at a time.
*   Understand larger/more complicated ideas by understanding and combining pieces.
    *   Components in a machine.
    *   Lemmas when proving theorems.
*   Functions serve the same purpose in programs.
    *   *Encapsulate* complexity so that we can treat it as a single "thing".
*   Also enables *re-use*.
    *   Write one time, use many times.

## Define a function using `def` with a name, parameters, and a block of code.

*   Begin the definition of a new function with `def`.
*   Followed by the name of the function.
    *   Must obey the same rules as variable names.
*   Then *parameters* in parentheses.
    *   Empty parentheses if the function doesn't take any inputs.
    *   We will discuss this in detail in a moment.
*   Then a colon.
*   Then an indented block of code.

~~~
def print_greeting():
    print('Hello!')
~~~
{: .language-python}

## Defining a function does not run it.

*   Defining a function does not run it.
    *   Like assigning a value to a variable.
*   Must call the function to execute the code it contains.

~~~
print_greeting()
~~~
{: .language-python}
~~~
Hello!
~~~
{: .output}

## Arguments in call are matched to parameters in definition.

*   Functions are most useful when they can operate on different data.
*   Specify *parameters* when defining a function.
    *   These become variables when the function is executed.
    *   Are assigned the arguments in the call (i.e., the values passed to the function).
    *   If you don't name the arguments when using them in the call, the arguments will be matched to
parameters in the order the parameters are defined in the function.

~~~
def print_date(year, month, day):
    joined = str(year) + '/' + str(month) + '/' + str(day)
    print(joined)

print_date(1871, 3, 19)
~~~
{: .language-python}
~~~
1871/3/19
~~~
{: .output}

Or, we can name the arguments when we call the function, which allows us to
specify them in any order:
~~~
print_date(month=3, day=19, year=1871)
~~~
{: .language-python}
~~~
1871/3/19
~~~
{: .output}

*   Via [Twitter](https://twitter.com/minisciencegirl/status/693486088963272705):
    `()` contains the ingredients for the function
    while the body contains the recipe.

## Functions may return a result to their caller using `return`.

*   Use `return ...` to give a value back to the caller.
*   May occur anywhere in the function.
*   But functions are easier to understand if `return` occurs:
    *   At the start to handle special cases.
    *   At the very end, with a final result.

~~~
def average(values):
    if len(values) == 0:
        return None
    return sum(values) / len(values)
~~~
{: .language-python}

~~~
a = average([1, 3, 4])
print('average of actual values:', a)
~~~
{: .language-python}
~~~
average of actual values: 2.6666666666666665
~~~
{: .output}

~~~
print('average of empty list:', average([]))
~~~
{: .language-python}
~~~
average of empty list: None
~~~
{: .output}

*   Remember: [every function returns something]({{ page.root }}/04-built-in/).
*   A function that doesn't explicitly `return` a value automatically returns `None`.

~~~
result = print_date(1871, 3, 19)
print('result of call is:', result)
~~~
{: .language-python}
~~~
1871/3/19
result of call is: None
~~~
{: .output}

> ## Identifying Syntax Errors
>
> 1. Read the code below and try to identify what the errors are
>    *without* running it.
> 2. Run the code and read the error message.
>    Is it a `SyntaxError` or an `IndentationError`?
> 3. Fix the error.
> 4. Repeat steps 2 and 3 until you have fixed all the errors.
>
> ~~~
> def another_function
>   print("Syntax errors are annoying.")
>    print("But at least python tells us about them!")
>   print("So they are usually not too hard to fix.")
> ~~~
> {: .language-python}
>
> > ## Solution
> >
> > ~~~
> > def another_function():
> >   print("Syntax errors are annoying.")
> >   print("But at least Python tells us about them!")
> >   print("So they are usually not too hard to fix.")
> > ~~~
> > {: .language-python}
> {: .solution}
{: .challenge}

> ## Definition and Use
>
> What does the following program print?
>
> ~~~
> def report(pressure):
>     print('pressure is', pressure)
>
> print('calling', report, 22.5)
> ~~~
> {: .language-python}
> > ## Solution
> >
> > ~~~
> > calling <function report at 0x7fd128ff1bf8> 22.5
> > ~~~
> > {: .output}
> >
> > A function call always needs parenthesis, otherwise you get memory address of the function object. So, if we wanted to call the function named report, and give it the value 22.5 to report on, we could have our function call as follows
> > ~~~
> > print("calling")
> > report(22.5)
> > ~~~
> > {: .language-python}
> > ~~~
> > calling
> > pressure is 22.5
> > ~~~
> > {: .output}
> {: .solution}
{: .challenge}

> ## Order of Operations
>
> 1. What's wrong in this example?
>
>     ~~~
>     result = print_time(11, 37, 59)
>
>     def print_time(hour, minute, second):
>        time_string = str(hour) + ':' + str(minute) + ':' + str(second)
>        print(time_string)
>     ~~~
>     {: .language-python}
>
> 2. After fixing the problem above, explain why running this example code:
>
>     ~~~
>     result = print_time(11, 37, 59)
>     print('result of call is:', result)
>     ~~~
>     {: .language-python}
>
>     gives this output:
>
>     ~~~
>     11:37:59
>     result of call is: None
>     ~~~
>     {: .output}
>
> 3. Why is the result of the call `None`?
>
> > ## Solution
> >
> > 1. The problem with the example is that the function `print_time()` is defined *after* the call to the function is made. Python
> > doesn't know how to resolve the name `print_time` since it hasn't been defined yet and will raise a `NameError` e.g.,
> > `NameError: name 'print_time' is not defined`
> >
> > 2. The first line of output `11:37:59` is printed by the first line of code, `result = print_time(11, 37, 59)` that binds the value
> > returned by invoking `print_time` to the variable `result`. The second line is from the second print call to print the contents
> > of the `result` variable.
> >
> > 3. `print_time()` does not explicitly `return` a value, so it automatically returns `None`.
> >
> {: .solution}
{: .challenge}

> ## Encapsulation
>
> Fill in the blanks to create a function that takes a single filename as an argument,
> loads the data in the file named by the argument,
> and returns the minimum value in that data.
>
> ~~~
> import pandas as pd
>
> def min_in_data(____):
>     data = ____
>     return ____
> ~~~
> {: .language-python}
> > ## Solution
> >
> > ~~~
> > import pandas as pd
> >
> > def min_in_data(filename):
> >     data = pd.read_csv(filename)
> >     return data.min()
> > ~~~
> > {: .language-python}
> {: .solution}
{: .challenge}

> ## Find the First
>
> Fill in the blanks to create a function that takes a list of numbers as an argument
> and returns the first negative value in the list.
> What does your function do if the list is empty? What if the list has no negative numbers?
>
> ~~~
> def first_negative(values):
>     for v in ____:
>         if ____:
>             return ____
> ~~~
> {: .language-python}
> > ## Solution
> >
> > ~~~
> > def first_negative(values):
> >     for v in values:
> >         if v < 0:
> >             return v
> > ~~~
> > {: .language-python}
> > If an empty list or a list with all positive values is passed to this function, it returns `None`:
> > ~~~
> > my_list = []
> > print(first_negative(my_list))
> > ~~~
> > {: .language-python}
> > ~~~
> > None
> > ~~~
> > {: .output}
> {: .solution}
{: .challenge}

> ## Calling by Name
>
> Earlier we saw this function:
>
> ~~~
> def print_date(year, month, day):
>     joined = str(year) + '/' + str(month) + '/' + str(day)
>     print(joined)
> ~~~
> {: .language-python}
> We saw that we can call the function using *named arguments*, like this:
> ~~~
> print_date(day=1, month=2, year=2003)
> ~~~
> {: .language-python}
>
> 1.  What does `print_date(day=1, month=2, year=2003)` print?
> 2.  When have you seen a function call like this before?
> 3.  When and why is it useful to call functions this way?
>
> > ## Solution
> >
> > 1. `2003/2/1`
> > 2. We saw examples of using *named arguments* when working with the pandas library. For example, when reading in a dataset
> > using `data = pd.read_csv('data/gapminder_gdp_europe.csv', index_col='country')`, the last argument `index_col` is a
> > named argument.
> > 3. Using named arguments can make code more readable since one can see from the function call what name the different arguments
> > have inside the function. It can also reduce the chances of passing arguments in the wrong order, since by using named arguments
> > the order doesn't matter.
> {: .solution}
{: .challenge}

> ## Encapsulation of an If/Print Block
>
> The code below will run on a label-printer for chicken eggs.  A digital scale will report a chicken egg mass (in grams)
> to the computer and then the computer will print a label.
>
> Please re-write the code so that the if-block is folded into a function.
>
> ~~~
> import random
> for i in range(10):
>
>     # simulating the mass of a chicken egg
>     # the (random) mass will be 70 +/- 20 grams
>     mass = 70 + 20.0 * (2.0 * random.random() - 1.0)
>
>     print(mass)
>
>     # egg sizing machinery prints a label
>     if mass >= 85:
>        print("jumbo")
>     elif mass >= 70:
>        print("large")
>     elif mass < 70 and mass >= 55:
>        print("medium")
>     else:
>        print("small")
> ~~~
> {: .language-python}
>
>
> The simplified program follows.  What function definition will make it functional?
>
> ~~~
> # revised version
> import random
> for i in range(10):
>
>     # simulating the mass of a chicken egg
>     # the (random) mass will be 70 +/- 20 grams
>     mass = 70 + 20.0 * (2.0 * random.random() - 1.0)
>
>     print(mass, get_egg_label(mass))
>
> ~~~
> {: .language-python}
>
>
> 1. Create a function definition for `get_egg_label()` that will work with the revised program above.  Note that the `get_egg_label()` function's return value will be important. Sample output from the above program would be `71.23 large`.
> 2. A dirty egg might have a mass of more than 90 grams, and a spoiled or broken egg will probably have a mass that's less than 50 grams.  Modify your `get_egg_label()` function to account for these error conditions. Sample output could be `25 too light, probably spoiled`.
>
> > ## Solution
> >
> > ~~~
> > def get_egg_label(mass):
> >     # egg sizing machinery prints a label
> >     egg_label = "Unlabelled"
> >     if mass >= 90:
> >         egg_label = "warning: egg might be dirty"
> >     elif mass >= 85:
> >         egg_label = "jumbo"
> >     elif mass >= 70:
> >         egg_label = "large"
> >     elif mass < 70 and mass >= 55:
> >         egg_label = "medium"
> >     elif mass < 50:
> >         egg_label = "too light, probably spoiled"
> >     else:
> >         egg_label = "small"
> >     return egg_label
> > ~~~
> > {: .language-python}
> {: .solution}
{: .challenge}

> ## Encapsulating Data Analysis
>
> Assume that the following code has been executed:
>
> ~~~
> import pandas as pd
>
> df = pd.read_csv('data/gapminder_gdp_asia.csv', index_col=0)
> japan = df.loc['Japan']
> ~~~
> {: .language-python}
>
> 1. Complete the statements below to obtain the average GDP for Japan
>    across the years reported for the 1980s.
>
>     ~~~
>     year = 1983
>     gdp_decade = 'gdpPercap_' + str(year // ____)
>     avg = (japan.loc[gdp_decade + ___] + japan.loc[gdp_decade + ___]) / 2
>     ~~~
>     {: .language-python}
>
> 2. Abstract the code above into a single function.
>
>     ~~~
>     def avg_gdp_in_decade(country, continent, year):
>         df = pd.read_csv('data/gapminder_gdp_'+___+'.csv',delimiter=',',index_col=0)
>         ____
>         ____
>         ____
>         return avg
>     ~~~
>     {: .language-python}
>
> 3. How would you generalize this function
>    if you did not know beforehand which specific years occurred as columns in the data?
>    For instance, what if we also had data from years ending in 1 and 9 for each decade?
>    (Hint: use the columns to filter out the ones that correspond to the decade,
>    instead of enumerating them in the code.)
>
> > ## Solution
> >
> > 1. The average GDP for Japan across the years reported for the 1980s is computed with:
> >
> >     ~~~
> >     year = 1983
> >     gdp_decade = 'gdpPercap_' + str(year // 10)
> >     avg = (japan.loc[gdp_decade + '2'] + japan.loc[gdp_decade + '7']) / 2
> >     ~~~
> >     {: .language-python}
> >
> > 2. That code as a function is:
> >
> >     ~~~
> >     def avg_gdp_in_decade(country, continent, year):
> >         df = pd.read_csv('data/gapminder_gdp_' + continent + '.csv', index_col=0)
> >         c = df.loc[country]
> >         gdp_decade = 'gdpPercap_' + str(year // 10)
> >         avg = (c.loc[gdp_decade + '2'] + c.loc[gdp_decade + '7'])/2
> >         return avg
> >     ~~~
> >     {: .language-python}
> >
> > 3. To obtain the average for the relevant years, we need to loop over them:
> >
> >    ~~~
> >    def avg_gdp_in_decade(country, continent, year):
> >         df = pd.read_csv('data/gapminder_gdp_' + continent + '.csv', index_col=0)
> >         c = df.loc[country]
> >         gdp_decade = 'gdpPercap_' + str(year // 10)
> >         total = 0.0
> >         num_years = 0
> >         for yr_header in c.index: # c's index contains reported years
> >             if yr_header.startswith(gdp_decade):
> >                 total = total + c.loc[yr_header]
> >                 num_years = num_years + 1
> >         return total/num_years
> >     ~~~
> >     {: .language-python}
> >
> > The function can now be called by:
> >
> > ~~~
> > avg_gdp_in_decade('Japan','asia',1983)
> > ~~~
> > {: .language-python}
> >
> > ~~~
> > 20880.023800000003
> > ~~~
> > {: .output}
> {: .solution}
{: .challenge}

> ## Simulating a dynamical system
>
> In mathematics, a [dynamical system](https://en.wikipedia.org/wiki/Dynamical_system) is a system
> in which a function describes the time dependence of a point in a geometrical space. A canonical
> example of a dynamical system is the [logistic map](https://en.wikipedia.org/wiki/Logistic_map),
> a growth model that computes a new population density (between  0 and 1) based on the current
> density. In the model, time takes discrete values 0, 1, 2, ...
>
> 1. Define a function called `logistic_map` that takes two inputs: `x`, representing the current
>    population (at time `t`), and a parameter `r = 1`. This function should return a value
>    representing the state of the system (population) at time `t + 1`, using the mapping function:
>
>    `f(t+1) = r * f(t) * [1 - f(t)]`
>
> 2. Using a `for` or `while` loop, iterate the `logistic_map` function defined in part 1, starting
>    from an initial population of 0.5, for a period of time `t_final = 10`. Store the intermediate
>    results in a list so that after the loop terminates you have accumulated a sequence of values
>    representing the state of the logistic map at times `t = [0,1,...,t_final]`. Print this list to
>    see the evolution of the population.
>
> 3. Encapsulate the logic of your loop into a function called `iterate` that takes the initial
>    population as its first input, the parameter `t_final` as its second input and the parameter
>    `r` as its third input. The function should return the list of values representing the state of
>    the logistic map at times `t = [0,1,...,t_final]`. Run this function for periods `t_final = 100`
>    and `1000` and print some of the values. Is the population trending toward a steady state?
>
> > ## Solution
> >
> > 1. ~~~
> >    def logistic_map(x, r):
> >        return r * x * (1 - x)
> >    ~~~
> >    {: .language-python}
> >
> > 2. ~~~
> >    initial_population = 0.5
> >    t_final = 10
> >    r = 1.0
> >    population = [initial_population]
> >    for t in range(1, t_final):
> >        population.append( logistic_map(population[t-1], r) )
> >    ~~~
> >    {: .language-python}
> >
> > 3. ~~~
> >    def iterate(initial_population, t_final, r):
> >        population = [initial_population]
> >        for t in range(1, t_final):
> >            population.append( logistic_map(population[t-1], r) )
> >        return population
> >
> >    for period in (10, 100, 1000):
> >        population = iterate(0.5, period, 1)
> >        print(population[-1])
> >    ~~~
> >    {: .language-python}
> >    ~~~
> >    0.07508929631879595
> >    0.009485759503982033
> >    0.0009923756709128578
> >    ~~~
> >    {: .output}
> >    The population seems to be approaching zero.
> {: .solution}
{: .challenge}_____no_output_____

_____no_output_____

_____no_output_____


# Notebook from aow207/foundations_homework
Path: 04-homework/01 - SWAPI API (Star Wars).ipynb

# SWAPI (Star Wars)

Answer the following questions using the [Star Wars](https://swapi.co/). I've added three cells for each question but you're free to use more or less! Hold `Shift` and hit `Enter` to run a cell, and use the `+` on the top left to add a new cell to a notebook.

If you spend some time reading the documentation your life will probably be a little bit easier!

## 0) Import any libraries you might need

- *Tip: We're going to be downloading things from the internet, so we probably need `requests`.*_____no_output_____

import requests_____no_output_____

## 1) Make a request to the Star Wars API

I've heard there's a new Star Wars film called **Solo**, but it isn't in SWAPI!

The most recent film it contains has an ID of 7. Make a request for that film and look at the data._____no_output_____

response = requests.get('https://swapi.co/api/films/7/')_____no_output_____data = response.json()
#print(data) #Now it's a dictionary
print()
print(data.keys())
dict_keys(['title', 'episode_id', 'opening_crawl', 'director', 'producer', 'release_date', 'characters', 'planets', 'starships', 'vehicles', 'species', 'created', 'edited', 'url'])

## 2) What is that film's name, and when was it released?

Please print **only** the title and the release date. You can guess at the keys, but I recommend checking the documentation._____no_output_____

print(data['title'])The Force Awakens
print(data['release_date'])2015-12-11
print(data['characters'])['https://swapi.co/api/people/1/', 'https://swapi.co/api/people/3/', 'https://swapi.co/api/people/5/', 'https://swapi.co/api/people/13/', 'https://swapi.co/api/people/14/', 'https://swapi.co/api/people/27/', 'https://swapi.co/api/people/84/', 'https://swapi.co/api/people/85/', 'https://swapi.co/api/people/86/', 'https://swapi.co/api/people/87/', 'https://swapi.co/api/people/88/']

## 3) But I want Han Solo! Use the API to search for people named "Solo"._____no_output_____

library = []

for person in data['characters']:
    library.append(person)
    #print(person)
    response = requests.get(person)
    people = response.json()
    #print(people['name'])

    if people['name'] == "Han Solo":
        print(people['name'])
        print("Found Him:", person)
        ()Han Solo
Found Him: https://swapi.co/api/people/14/
# --- OR ---
response_Han = requests.get('https://swapi.co/api/people/?search=Solo')
Han_Solo = response_Han.json()
#print(Han_Solo)
#print(Han_Solo.keys())
print("Here he is again with the API's search:")
print(Han_Solo['results'])Here he is again with the API's search:
[{'name': 'Han Solo', 'height': '180', 'mass': '80', 'hair_color': 'brown', 'skin_color': 'fair', 'eye_color': 'brown', 'birth_year': '29BBY', 'gender': 'male', 'homeworld': 'https://swapi.co/api/planets/22/', 'films': ['https://swapi.co/api/films/2/', 'https://swapi.co/api/films/3/', 'https://swapi.co/api/films/1/', 'https://swapi.co/api/films/7/'], 'species': ['https://swapi.co/api/species/1/'], 'vehicles': [], 'starships': ['https://swapi.co/api/starships/10/', 'https://swapi.co/api/starships/22/'], 'created': '2014-12-10T16:49:14.582000Z', 'edited': '2014-12-20T21:17:50.334000Z', 'url': 'https://swapi.co/api/people/14/'}]

## 4) How many results did you get?

Show **two different ways** of displaying this number.

- *Tip: One uses the API, and one uses a Python function*_____no_output_____

len(library)_____no_output_____#API Search Result Count
print(Han_Solo['count'])1

## 5) Write me a sentence about Han Solo

Your sentence should be say `"____ has _____ hair, is _____cm tall and weighs ____kg."` - make sure you have **no spcaes** between the numbers and `cm`/`kg`._____no_output_____

response = requests.get('https://swapi.co/api/people/14/')
Han = response.json()
print(Han.keys())dict_keys(['name', 'height', 'mass', 'hair_color', 'skin_color', 'eye_color', 'birth_year', 'gender', 'homeworld', 'films', 'species', 'vehicles', 'starships', 'created', 'edited', 'url'])
print(Han['name'], 'has', Han['hair_color'], 'hair, is', Han['height'] ,'cm tall and weighs', Han['mass'] ,'kg.')Han Solo has brown hair, is 180 cm tall and weighs 80 kg.

## 5) Search for C-3PO_____no_output_____

response = requests.get('https://swapi.co/api/people/')
home_people = response.json()
everyone = home_people['results']

for person in everyone:
    #print(person)

    if person['name'] == "C-3PO":
        print("Found Him:", person['name'])
        print(person)Found Him: C-3PO
{'name': 'C-3PO', 'height': '167', 'mass': '75', 'hair_color': 'n/a', 'skin_color': 'gold', 'eye_color': 'yellow', 'birth_year': '112BBY', 'gender': 'n/a', 'homeworld': 'https://swapi.co/api/planets/1/', 'films': ['https://swapi.co/api/films/2/', 'https://swapi.co/api/films/5/', 'https://swapi.co/api/films/4/', 'https://swapi.co/api/films/6/', 'https://swapi.co/api/films/3/', 'https://swapi.co/api/films/1/'], 'species': ['https://swapi.co/api/species/2/'], 'vehicles': [], 'starships': [], 'created': '2014-12-10T15:10:51.357000Z', 'edited': '2014-12-20T21:17:50.309000Z', 'url': 'https://swapi.co/api/people/2/'}

## 6) What URL can tell me more about his species? Make a request to the corrent endpoint.

Spoiler: he's a **droid**. Are robots a species? I guess so, maybe._____no_output_____

URL = ''
for person in everyone:
    if person['name'] == "C-3PO":
        species = person['species']
        for link in species:
            #print(link)
            URL += link
print(URL)https://swapi.co/api/species/2/
response = requests.get(URL)
droid = response.json()
print(droid.keys())
#print(droid)dict_keys(['name', 'classification', 'designation', 'average_height', 'skin_colors', 'hair_colors', 'eye_colors', 'average_lifespan', 'homeworld', 'language', 'people', 'films', 'created', 'edited', 'url'])

## 7) Print out the URL of every droid

* *Tip: life will be easier if you don't say `for droid in...` or `for person in` because you aren't looping through droids or people. What are you looping through?*_____no_output_____

droids = []

for bot in droid['people']:
    print(bot)
    droids.append(bot)https://swapi.co/api/people/2/
https://swapi.co/api/people/3/
https://swapi.co/api/people/8/
https://swapi.co/api/people/23/
https://swapi.co/api/people/87/

## 8) Print out the name of every droid

* *Tip: You should start from the last answer, but add more stuff in!*
* *Tip: You'll need to make a request every time you're in the loop*
* *Tip: Be careful - f you re-use the variable name `data` you'll erase your old `data` variable*_____no_output_____

for bot in droid['people']:
    #print(bot)
    response = requests.get(bot)
    bots = response.json()
    print(bots['name'])
C-3PO
R2-D2
R5-D4
IG-88
BB8

## 9) Print out more about the droid

Count the number of films for each droid, printing `"___ was in ___ films"` for each of them

* *Tip: You might want to look at a single droid result, or print the data in the for loop, so you know how to get the films. Or read the documentation!*_____no_output_____

for bot in droid['people']:
    #print(bot)
    response = requests.get(bot)
    bots = response.json()
    #print(bots)
    link_num = 0
    for link in bots['films']:
        print(link)
        link_num += 1
    print(bots['name'], "was in", link_num, "films")
        https://swapi.co/api/films/2/
https://swapi.co/api/films/5/
https://swapi.co/api/films/4/
https://swapi.co/api/films/6/
https://swapi.co/api/films/3/
https://swapi.co/api/films/1/
C-3PO was in 6 films
https://swapi.co/api/films/2/
https://swapi.co/api/films/5/
https://swapi.co/api/films/4/
https://swapi.co/api/films/6/
https://swapi.co/api/films/3/
https://swapi.co/api/films/1/
https://swapi.co/api/films/7/
R2-D2 was in 7 films
https://swapi.co/api/films/1/
R5-D4 was in 1 films
https://swapi.co/api/films/2/
IG-88 was in 1 films
https://swapi.co/api/films/7/
BB8 was in 1 films

## 10) Let's be smart and cache the results *[DIFFICULT, maybe!]*

So far we've been using a for loop to go through each droid. Every droid, one more request. Every time we want more data, we run the loops again. I don't want that guy to get angry that we're asking for so much data!

We want to our list of **droid urls** and turn it into a **list of dictionaries about those droids**. Save it as a variable called `droids`. If you get it correct, the following code should work if you cut and paste it.

```python
for droid in droids:
    print(droid['name'])
```

* *Tip: Use a list comprehension*
* *Tip: So far we've done `response = requests.get..` and `response.json()` on separate lines. You'll need to combine them!*_____no_output_____

droids = []

for bot in droid['people']:
    #print(bot)
    response = requests.get(bot)
    bots = response.json()
    print(bots)
    droids.append(bots){'name': 'C-3PO', 'height': '167', 'mass': '75', 'hair_color': 'n/a', 'skin_color': 'gold', 'eye_color': 'yellow', 'birth_year': '112BBY', 'gender': 'n/a', 'homeworld': 'https://swapi.co/api/planets/1/', 'films': ['https://swapi.co/api/films/2/', 'https://swapi.co/api/films/5/', 'https://swapi.co/api/films/4/', 'https://swapi.co/api/films/6/', 'https://swapi.co/api/films/3/', 'https://swapi.co/api/films/1/'], 'species': ['https://swapi.co/api/species/2/'], 'vehicles': [], 'starships': [], 'created': '2014-12-10T15:10:51.357000Z', 'edited': '2014-12-20T21:17:50.309000Z', 'url': 'https://swapi.co/api/people/2/'}
{'name': 'R2-D2', 'height': '96', 'mass': '32', 'hair_color': 'n/a', 'skin_color': 'white, blue', 'eye_color': 'red', 'birth_year': '33BBY', 'gender': 'n/a', 'homeworld': 'https://swapi.co/api/planets/8/', 'films': ['https://swapi.co/api/films/2/', 'https://swapi.co/api/films/5/', 'https://swapi.co/api/films/4/', 'https://swapi.co/api/films/6/', 'https://swapi.co/api/films/3/', 'https://swapi.co/api/films/1/', 'https://swapi.co/api/films/7/'], 'species': ['https://swapi.co/api/species/2/'], 'vehicles': [], 'starships': [], 'created': '2014-12-10T15:11:50.376000Z', 'edited': '2014-12-20T21:17:50.311000Z', 'url': 'https://swapi.co/api/people/3/'}
{'name': 'R5-D4', 'height': '97', 'mass': '32', 'hair_color': 'n/a', 'skin_color': 'white, red', 'eye_color': 'red', 'birth_year': 'unknown', 'gender': 'n/a', 'homeworld': 'https://swapi.co/api/planets/1/', 'films': ['https://swapi.co/api/films/1/'], 'species': ['https://swapi.co/api/species/2/'], 'vehicles': [], 'starships': [], 'created': '2014-12-10T15:57:50.959000Z', 'edited': '2014-12-20T21:17:50.321000Z', 'url': 'https://swapi.co/api/people/8/'}
{'name': 'IG-88', 'height': '200', 'mass': '140', 'hair_color': 'none', 'skin_color': 'metal', 'eye_color': 'red', 'birth_year': '15BBY', 'gender': 'none', 'homeworld': 'https://swapi.co/api/planets/28/', 'films': ['https://swapi.co/api/films/2/'], 'species': ['https://swapi.co/api/species/2/'], 'vehicles': [], 'starships': [], 'created': '2014-12-15T12:51:10.076000Z', 'edited': '2014-12-20T21:17:50.351000Z', 'url': 'https://swapi.co/api/people/23/'}
{'name': 'BB8', 'height': 'unknown', 'mass': 'unknown', 'hair_color': 'none', 'skin_color': 'none', 'eye_color': 'black', 'birth_year': 'unknown', 'gender': 'none', 'homeworld': 'https://swapi.co/api/planets/28/', 'films': ['https://swapi.co/api/films/7/'], 'species': ['https://swapi.co/api/species/2/'], 'vehicles': [], 'starships': [], 'created': '2015-04-17T06:57:38.061346Z', 'edited': '2015-04-17T06:57:38.061453Z', 'url': 'https://swapi.co/api/people/87/'}
for droid in droids:
    print(droid['name'])C-3PO
R2-D2
R5-D4
IG-88
BB8

## 11) Get a list of every single planet in the Star Wars universe

* *Tip: You'll want to use the API documentation for this, I think*_____no_output_____

response = requests.get('https://swapi.co/api/planets/?search') #https://swapi.co/api/planets/?search=solo
#use a format
planets = response.json()
#print(planets.keys())
planets_list = planets['results']
planet_count = []

for planet in planets_list:
    #print(planet['name'])
    planet_count.append(planet['name'])_____no_output_____all_names = []
for planet in planets_list:
    for i in range (1,8):
        response = requests.get(f"https://swapi.co/api/planets/?search=&page={i}")
        next_planet = response.json()
        results = next_planet['results']
        for result in results:
            if result['name'] in all_names:
                pass
            else:
                all_names.append(result['name'])

print(len(all_names))
print(all_names)61
['Alderaan', 'Yavin IV', 'Hoth', 'Dagobah', 'Bespin', 'Endor', 'Naboo', 'Coruscant', 'Kamino', 'Geonosis', 'Utapau', 'Mustafar', 'Kashyyyk', 'Polis Massa', 'Mygeeto', 'Felucia', 'Cato Neimoidia', 'Saleucami', 'Stewjon', 'Eriadu', 'Corellia', 'Rodia', 'Nal Hutta', 'Dantooine', 'Bestine IV', 'Ord Mantell', 'unknown', 'Trandosha', 'Socorro', 'Mon Cala', 'Chandrila', 'Sullust', 'Toydaria', 'Malastare', 'Dathomir', 'Ryloth', 'Aleen Minor', 'Vulpter', 'Troiken', 'Tund', 'Haruun Kal', 'Cerea', 'Glee Anselm', 'Iridonia', 'Tholoth', 'Iktotch', 'Quermia', 'Dorin', 'Champala', 'Mirial', 'Serenno', 'Concord Dawn', 'Zolan', 'Ojom', 'Skako', 'Muunilinst', 'Shili', 'Kalee', 'Umbara', 'Tatooine', 'Jakku']

## 12) How many plants are there? Show me two different ways of displaying the answer.

* *Tip: They won't match*_____no_output_____

print(len(all_names))61
print(planets['count'])61
print(len(planet_count)) #wrong version, doesn't account for all pages10

# These questions are all list comprehension questions, if you'd like

## 13) Get the population of each of the planets

* *Tip: Only use the 10 results for now*
* *Tip: Use a list comprehension*_____no_output_____

for planet in planets_list:
    print(planet['name'], planet['population'])Alderaan 2000000000
Yavin IV 1000
Hoth unknown
Dagobah unknown
Bespin 6000000
Endor 30000000
Naboo 4500000000
Coruscant 1000000000000
Kamino 1000000000
Geonosis 100000000000
population = [planet['population'] for planet in planets_list]
print(population)['2000000000', '1000', 'unknown', 'unknown', '6000000', '30000000', '4500000000', '1000000000000', '1000000000', '100000000000']

## 14) Get the population of each of the planets, EXCEPT the ones that have a population of `'unknown'`

* *Tip: Use a list comprehension with a filter (that's the `..if..` part at the end)*_____no_output_____

for planet in planets_list:
    if planet['population'] == 'unknown':
        pass
    else:
        print(planet['population'])2000000000
1000
6000000
30000000
4500000000
1000000000000
1000000000
100000000000
population = [planet['population'] for planet in planets_list if planet['population'] != 'unknown']
print(population)['2000000000', '1000', '6000000', '30000000', '4500000000', '1000000000000', '1000000000', '100000000000']

## 14) Add up the populations

* *Tip: They're strings right now, so it isn't going to work unless you change something about them*_____no_output_____

total = 0
for pop in population:
    total += int(pop)
print(total)1107536001000
total_pop = [int(pop) for pop in population]
sum(total_pop)_____no_output_____

## 15) Print what percentage of the total population each planet has

I would like the sentences to be `"The planet ____ has a population of _____, which is ___% of the total population"`. If we don't know the planet's population, say `"We don't know the population of _____"`.

* *Tip: This finally uses a for loop, but also the result of the last question*_____no_output_____

for planet in planets_list:
    if planet['population'] == 'unknown':
        print("We don't know the population of", planet['name'])
    else:
        perc = int(planet['population'])/sum(total_pop) * 100
        percent = round(perc, 2)
        print("The planet", planet['name'], "has a population of "+ planet['population']+ ", which is "+ str(percent)+ "% of the total population")The planet Alderaan has a population of 2000000000, which is 0.18% of the total population
The planet Yavin IV has a population of 1000, which is 0.0% of the total population
We don't know the population of Hoth
We don't know the population of Dagobah
The planet Bespin has a population of 6000000, which is 0.0% of the total population
The planet Endor has a population of 30000000, which is 0.0% of the total population
The planet Naboo has a population of 4500000000, which is 0.41% of the total population
The planet Coruscant has a population of 1000000000000, which is 90.29% of the total population
The planet Kamino has a population of 1000000000, which is 0.09% of the total population
The planet Geonosis has a population of 100000000000, which is 9.03% of the total population


# Notebook from Sravanthgithub/Heart-Disease-Classification
Path: Classification.ipynb

# Predicting heart disease using machine learning
_____no_output_____**This notebook looks into using various Python-based machine learning and data science libraries in an attempt to build a machine learning model capable of predicting whether or not someone has heart disease based on their medical attributes.**
_____no_output_____
_____no_output_____

# Importing tools
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sns

#Models from Scikit-Learn
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import RandomForestClassifier

# Model Evaluations
from sklearn.model_selection import train_test_split, cross_val_score
from sklearn.model_selection import RandomizedSearchCV, GridSearchCV
from sklearn.metrics import confusion_matrix, classification_report
from sklearn.metrics import precision_score, recall_score, f1_score
from sklearn.metrics import plot_roc_curve_____no_output_____

### Loading Data
_____no_output_____

df = pd.read_csv('Data/heart-disease (2).csv')
df_____no_output_____df.shape_____no_output_____

### Let's explore the data_____no_output_____

df.head()_____no_output_____df.tail()_____no_output_____df['target'].value_counts()_____no_output_____df["target"].value_counts().plot(kind="bar", color=["salmon", "lightblue"]);_____no_output_____

Hmmmm, We can easily get more than 50% accuracy by just predicting all 1's 😉_____no_output_____

df.isna().sum()_____no_output_____

Hopefully, our data is clean. _____no_output_____#### A breif insight of our Data_____no_output_____

df.describe()_____no_output_____df.info()<class 'pandas.core.frame.DataFrame'>
RangeIndex: 303 entries, 0 to 302
Data columns (total 14 columns):
 #   Column    Non-Null Count  Dtype
---  ------    --------------  -----
 0   age       303 non-null    int64
 1   sex       303 non-null    int64
 2   cp        303 non-null    int64
 3   trestbps  303 non-null    int64
 4   chol      303 non-null    int64
 5   fbs       303 non-null    int64
 6   restecg   303 non-null    int64
 7   thalach   303 non-null    int64
 8   exang     303 non-null    int64
 9   oldpeak   303 non-null    float64
 10  slope     303 non-null    int64
 11  ca        303 non-null    int64
 12  thal      303 non-null    int64
 13  target    303 non-null    int64
dtypes: float64(1), int64(13)
memory usage: 33.3 KB

### Let's Explore our data indepthly_____no_output_____#### Heart disease frequency according to sex_____no_output_____

df.sex.value_counts()_____no_output_____df['sex'].value_counts()_____no_output_____pd.crosstab(df.target,df.sex)_____no_output_____pd.crosstab(df['target'],df['sex'])_____no_output_____pd.crosstab(df.target, df.sex).plot(kind="bar",
                                    figsize=(10, 6),
                                    color=["salmon", "lightblue"]);
plt.title("Heart Disease Frequency for Sex")
plt.xlabel("0 = No Diesease, 1 = Disease")
plt.ylabel("Amount")
plt.legend(["Female", "Male"]);
plt.xticks(rotation=0)_____no_output_____
plt.figure(figsize=(10, 6))

# Scatter with postivie examples
plt.scatter(df.age[df.target==1],
            df.thalach[df.target==1],
            c="salmon")

plt.scatter(df.age[df.target==0],
            df.thalach[df.target==0],
            c="lightblue")

plt.title("Heart Disease in function of Age and Max Heart Rate")
plt.xlabel("Age")
plt.ylabel("Max Heart Rate")
plt.legend(["Disease", "No Disease"]);_____no_output_____df['age'].plot.hist();_____no_output_____df['age']_____no_output_____

### Heart disease frequency per chest Pain Type_____no_output_____

pd.crosstab(df['cp'],df['target'])_____no_output_____pd.crosstab(df.cp, df.target).plot(kind="bar",
                                   figsize=(10, 6),
                                   color=["salmon", "lightblue"])

# Add some communication
plt.title("Heart Disease Frequency Per Chest Pain Type")
plt.xlabel("Chest Pain Type")
plt.ylabel("Amount")
plt.legend(["No Disease", "Disease"])
plt.xticks(rotation=0);_____no_output_____df.head()_____no_output_____df.corr()_____no_output_____corr_matrix = df.corr()
fig, ax = plt.subplots(figsize=(15, 10))
ax = sns.heatmap(corr_matrix,
                 annot=True,
                 linewidths=0.5,
                 fmt=".2f",
                 cmap="YlGnBu");

ax.set_ylim(bottom, top)_____no_output_____

## Data Modelling_____no_output_____

df.head()_____no_output_____x = df.drop("target", axis=1)

y = df["target"]_____no_output_____x_____no_output_____y_____no_output_____# Split data into train and test sets
np.random.seed(42)

# Split into train & test set
x_train, x_test, y_train, y_test = train_test_split(x,
                                                    y,
                                                    test_size=0.2)_____no_output_____x_train_____no_output_____x_test_____no_output_____y_train_____no_output_____y_test_____no_output_____len(y_train), len(y_test)_____no_output_____len(x_train), len(x_test)_____no_output_____

We're going to try 3 different machine learning models:

* Logistic Regression
* K-Nearest Neighbours Classifier
* Random Forest Classifier_____no_output_____

models = {
    'Logistic Regression': LogisticRegression(),
    'KNN' : KNeighborsClassifier(),
    'Random Forest' : RandomForestClassifier()
}_____no_output_____def fit_and_score(models, X_train, X_test, y_train, y_test):
    np.random.seed(42)
    model_scores = {}
    for name, model in models.items():
        model.fit(X_train, y_train)
        model_scores[name] = model.score(X_test, y_test)
    return model_scores_____no_output_____model_scores = fit_and_score(models,x_train,x_test,y_train,y_test)C:\Users\Sravanth\anaconda3\lib\site-packages\sklearn\linear_model\_logistic.py:763: ConvergenceWarning: lbfgs failed to converge (status=1):
STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.

Increase the number of iterations (max_iter) or scale the data as shown in:
    https://scikit-learn.org/stable/modules/preprocessing.html
Please also refer to the documentation for alternative solver options:
    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression
  n_iter_i = _check_optimize_result(
model_scores_____no_output_____model_compare = pd.DataFrame(model_scores,index = ['accuracy'])
model_compare.T.plot.bar();_____no_output_____

### Now Let's do these..._____no_output_____* Hypyterparameter tuning
* Feature importance
* Confusion matrix
* Cross-validation
* Precision
* Recall
* F1 score
* Classification report
* ROC curve
* Area under the curve (AUC)_____no_output_____#### Hyperparameter tuning_____no_output_____

# Tuning KNN
train_scores = []
test_scores = []

# Create a list of differnt values for n_neighbors
neighbors = range(1, 21)
knn = KNeighborsClassifier()
for i in neighbors:
    knn.set_params(n_neighbors=i)
    knn.fit(x_train, y_train)
    train_scores.append(knn.score(x_train, y_train))
    test_scores.append(knn.score(x_test, y_test))_____no_output_____train_scores_____no_output_____test_scores_____no_output_____plt.plot(neighbors, train_scores, label="Train score")
plt.plot(neighbors, test_scores, label="Test score")
plt.xticks(np.arange(1, 21, 1))
plt.xlabel("Number of neighbors")
plt.ylabel("Model score")
plt.legend()

print(f"Maximum KNN score on the test data: {max(test_scores)*100:.2f}%")Maximum KNN score on the test data: 75.41%

### Hyperparameter tuning with RandomizedSearchCV_____no_output_____We're going to tune:

* LogisticRegression()
* RandomForestClassifier()_____no_output_____

# Create a hyperparameter grid for LogisticRegression
log_reg_grid = {"C": np.logspace(-4, 4, 20),
                "solver": ["liblinear"]}

# Create a hyperparameter grid for RandomForestClassifier
rf_grid = {"n_estimators": np.arange(10, 1000, 50),
           "max_depth": [None, 3, 5, 10],
           "min_samples_split": np.arange(2, 20, 2),
           "min_samples_leaf": np.arange(1, 20, 2)}_____no_output_____

Now using RandomisedSearchCV...
_____no_output_____

#Tuning Logistic Regression

np.random.seed(42)

rs_log_reg = RandomizedSearchCV(LogisticRegression(),
                               param_distributions=log_reg_grid,
                               cv=5,
                               n_iter = 20,
                               verbose=True)
rs_log_reg.fit(x_train,y_train)Fitting 5 folds for each of 20 candidates, totalling 100 fits
rs_log_reg.best_params______no_output_____rs_log_reg.score(x_test,y_test)_____no_output_____np.random.seed(42)

# Setup random hyperparameter search for RandomForestClassifier
rs_rf = RandomizedSearchCV(RandomForestClassifier(),
                           param_distributions=rf_grid,
                           cv=5,
                           n_iter=20,
                           verbose=True)

# Fit random hyperparameter search model for RandomForestClassifier()
rs_rf.fit(x_train, y_train)Fitting 5 folds for each of 20 candidates, totalling 100 fits
rs_rf.best_params______no_output_____rs_rf.score(x_test,y_test)_____no_output_____rs_rf.score(x_train,y_train)_____no_output_____

### Hyperparaameter Tuning with GridSearchCV_____no_output_____

# Different hyperparameters for our LogisticRegression model
log_reg_grid = {"C": np.logspace(-4, 4, 30),
                "solver": ["liblinear"]}

# Setup grid hyperparameter search for LogisticRegression
gs_log_reg = GridSearchCV(LogisticRegression(),
                          param_grid=log_reg_grid,
                          cv=5,
                          verbose=True)

# Fit grid hyperparameter search model
gs_log_reg.fit(x_train, y_train);Fitting 5 folds for each of 30 candidates, totalling 150 fits
gs_log_reg.best_params______no_output_____gs_log_reg.score(x_test,y_test)_____no_output_____

### Now Let's Evaluate our trained model_____no_output_____#### Evaluaters_____no_output_____* ROC curve and AUC score
* Confusion matrix
* Classification report
* Precision
* Recall
* F1-score_____no_output_____

y_preds = gs_log_reg.predict(x_test)_____no_output_____y_preds_____no_output_____y_test_____no_output_____plot_roc_curve(gs_log_reg, x_test,y_test)_____no_output_____

**Confusion Matrix**_____no_output_____

confusion_matrix(y_test,y_preds)_____no_output_____sns.set(font_scale=1.5)

def plot_conf_mat(y_test,y_preds):
    fig, ax = plt.subplots(figsize=(3, 3))
    ax = sns.heatmap(confusion_matrix(y_test, y_preds),
                     annot=True,
                     cbar=True)
    plt.xlabel("True label")
    plt.ylabel("Predicted label")

plot_conf_mat(y_test, y_preds)_____no_output_____print(classification_report(y_test,y_preds))              precision    recall  f1-score   support

           0       0.89      0.86      0.88        29
           1       0.88      0.91      0.89        32

    accuracy                           0.89        61
   macro avg       0.89      0.88      0.88        61
weighted avg       0.89      0.89      0.89        61

### Calculating Evaluation metrics using cross-validation_____no_output_____

gs_log_reg.best_params______no_output_____clf = LogisticRegression(C=0.20433597178569418,
                         solver = "liblinear")
clf.fit(x_train,y_train)_____no_output_____clf.score(x_test,y_test)_____no_output_____# Cross-validated accuracy
cv_acc = cross_val_score(clf,
                         x,
                         y,
                         cv=5,
                         scoring="accuracy")
cv_acc_____no_output_____cv_acc = cv_acc.mean()
cv_acc_____no_output_____# Cross-validated precision
cv_precision = cross_val_score(clf,
                         X,
                         y,
                         cv=5,
                         scoring="precision")
cv_precision=np.mean(cv_precision)
cv_precision_____no_output_____# Cross-validated recall
cv_recall = cross_val_score(clf,
                         X,
                         y,
                         cv=5,
                         scoring="recall")
cv_recall = np.mean(cv_recall)
cv_recall_____no_output_____# Cross-validated f1-score
cv_f1 = cross_val_score(clf,
                         X,
                         y,
                         cv=5,
                         scoring="f1")
cv_f1 = np.mean(cv_f1)
cv_f1_____no_output_____cv_metrics = pd.DataFrame({"Accuracy": cv_acc,
                           "Precision": cv_precision,
                           "Recall": cv_recall,
                           "F1": cv_f1},
                          index=[0])

cv_metrics.T.plot.bar(title="Cross-validated classification metrics",
                      legend=False);_____no_output_____

### Feature Imporatnce

**"which features contributed most to the outcomes of the model and how did they contribute?"**_____no_output_____

df.head()_____no_output_____clf = LogisticRegression(C=0.20433597178569418,
                         solver = "liblinear")
clf.fit(x_train,y_train)_____no_output_____df.head()_____no_output_____clf.coef______no_output_____# Match coef's of features to columns
feature_dict = dict(zip(df.columns, list(clf.coef_[0])))
feature_dict_____no_output_____# Visualizing feature importance
feature_df = pd.DataFrame(feature_dict, index=[0])
feature_df.T.plot.bar(title="Feature Importance", legend=False);
_____no_output_____pd.crosstab(df['sex'],df['target'])_____no_output_____

### Finally our Accuracy - 88.52%_____no_output_____