# Notebook from crawftv/DS-Sprint-01-Dealing-With-Data
Path: module2-loadingdata/LS_DS_112_Loading_Data.ipynb

# Lambda School Data Science - Loading Data

Data comes in many shapes and sizes - we'll start by loading tabular data, usually in csv format.

Data set sources:

- https://archive.ics.uci.edu/ml/datasets.html
- https://github.com/awesomedata/awesome-public-datasets
- https://registry.opendata.aws/ (beyond scope for now, but good to be aware of)

Let's start with an example - [data about flags](https://archive.ics.uci.edu/ml/datasets/Flags)._____no_output_____## Lecture example - flag data_____no_output_____

<code>
# Step 1 - find the actual file to download

# From navigating the page, clicking "Data Folder"
flag_data_url = 'https://archive.ics.uci.edu/ml/machine-learning-databases/flags/flag.data'

# You can "shell out" in a notebook for more powerful tools
# https://jakevdp.github.io/PythonDataScienceHandbook/01.05-ipython-and-shell-commands.html

# Funny extension, but on inspection looks like a csv
#!curl https://archive.ics.uci.edu/ml/machine-learning-databases/flags/flag.data

# Extensions are just a norm! You have to inspect to be sure what something is_____no_output_____# Step 2 - load the data

# How to deal with a csv? 🐼
flag_names = ["name", "landmass", "zone", "area", "population", "language", "religion", "bars", "stripes", "colours", "red", "green", "blue", "gold", "white", "black", "orange", "mainhue", "circles", "crosses", "saltires", "quarters", "sunstars", "crescent", "triangle", "icon", "animate", "text", "topleft", "botright"]
import pandas as pd
flag_data = pd.read_csv(flag_data_url, header=None, names=flag_names )_____no_output_____# Step 3 - verify we've got *something*
flag_data.head()_____no_output_____# Step 4 - Looks a bit odd - verify that it is what we want
flag_data.count()_____no_output_____!curl https://archive.ics.uci.edu/ml/machine-learning-databases/flags/flag.data | wc  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100 15240  100 15240    0     0  45088      0 --:--:-- --:--:-- --:--:-- 44955
    194     194   15240
# So we have 193 observations with funny names, file has 194 rows
# Looks like the file has no header row, but read_csv assumes it does
help(pd.read_csv)Help on function read_csv in module pandas.io.parsers:

read_csv(filepath_or_buffer, sep=',', delimiter=None, header='infer', names=None, index_col=None, usecols=None, squeeze=False, prefix=None, mangle_dupe_cols=True, dtype=None, engine=None, converters=None, true_values=None, false_values=None, skipinitialspace=False, skiprows=None, nrows=None, na_values=None, keep_default_na=True, na_filter=True, verbose=False, skip_blank_lines=True, parse_dates=False, infer_datetime_format=False, keep_date_col=False, date_parser=None, dayfirst=False, iterator=False, chunksize=None, compression='infer', thousands=None, decimal=b'.', lineterminator=None, quotechar='"', quoting=0, escapechar=None, comment=None, encoding=None, dialect=None, tupleize_cols=None, error_bad_lines=True, warn_bad_lines=True, skipfooter=0, skip_footer=0, doublequote=True, delim_whitespace=False, as_recarray=None, compact_ints=None, use_unsigned=None, low_memory=True, buffer_lines=None, memory_map=False, float_precision=None)
    Read CSV (comma-separated) file into DataFrame

    Also supports optionally iterating or breaking of the file
    into chunks.

    Additional help can be found in the `online docs for IO Tools
    <http://pandas.pydata.org/pandas-docs/stable/io.html>`_.

    Parameters
    ----------
    filepath_or_buffer : str, pathlib.Path, py._path.local.LocalPath or any object with a read() method (such as a file handle or StringIO)
        The string could be a URL. Valid URL schemes include http, ftp, s3, and
        file. For file URLs, a host is expected. For instance, a local file could
        be file ://localhost/path/to/table.csv
    sep : str, default ','
        Delimiter to use. If sep is None, the C engine cannot automatically detect
        the separator, but the Python parsing engine can, meaning the latter will
        be used and automatically detect the separator by Python's builtin sniffer
        tool, ``csv.Sniffer``. In addition, separators longer than 1 character and
        different from ``'\s+'`` will be interpreted as regular expressions and
        will also force the use of the Python parsing engine. Note that regex
        delimiters are prone to ignoring quoted data. Regex example: ``'\r\t'``
    delimiter : str, default ``None``
        Alternative argument name for sep.
    delim_whitespace : boolean, default False
        Specifies whether or not whitespace (e.g. ``' '`` or ``'    '``) will be
        used as the sep. Equivalent to setting ``sep='\s+'``. If this option
        is set to True, nothing should be passed in for the ``delimiter``
        parameter.

        .. versionadded:: 0.18.1 support for the Python parser.

    header : int or list of ints, default 'infer'
        Row number(s) to use as the column names, and the start of the
        data.  Default behavior is to infer the column names: if no names
        are passed the behavior is identical to ``header=0`` and column
        names are inferred from the first line of the file, if column
        names are passed explicitly then the behavior is identical to
        ``header=None``. Explicitly pass ``header=0`` to be able to
        replace existing names. The header can be a list of integers that
        specify row locations for a multi-index on the columns
        e.g. [0,1,3]. Intervening rows that are not specified will be
        skipped (e.g. 2 in this example is skipped). Note that this
        parameter ignores commented lines and empty lines if
        ``skip_blank_lines=True``, so header=0 denotes the first line of
        data rather than the first line of the file.
    names : array-like, default None
        List of column names to use. If file contains no header row, then you
        should explicitly pass header=None. Duplicates in this list will cause
        a ``UserWarning`` to be issued.
    index_col : int or sequence or False, default None
        Column to use as the row labels of the DataFrame. If a sequence is given, a
        MultiIndex is used. If you have a malformed file with delimiters at the end
        of each line, you might consider index_col=False to force pandas to _not_
        use the first column as the index (row names)
    usecols : array-like or callable, default None
        Return a subset of the columns. If array-like, all elements must either
        be positional (i.e. integer indices into the document columns) or strings
        that correspond to column names provided either by the user in `names` or
        inferred from the document header row(s). For example, a valid array-like
        `usecols` parameter would be [0, 1, 2] or ['foo', 'bar', 'baz'].

        If callable, the callable function will be evaluated against the column
        names, returning names where the callable function evaluates to True. An
        example of a valid callable argument would be ``lambda x: x.upper() in
        ['AAA', 'BBB', 'DDD']``. Using this parameter results in much faster
        parsing time and lower memory usage.
    as_recarray : boolean, default False
        .. deprecated:: 0.19.0
           Please call `pd.read_csv(...).to_records()` instead.

        Return a NumPy recarray instead of a DataFrame after parsing the data.
        If set to True, this option takes precedence over the `squeeze` parameter.
        In addition, as row indices are not available in such a format, the
        `index_col` parameter will be ignored.
    squeeze : boolean, default False
        If the parsed data only contains one column then return a Series
    prefix : str, default None
        Prefix to add to column numbers when no header, e.g. 'X' for X0, X1, ...
    mangle_dupe_cols : boolean, default True
        Duplicate columns will be specified as 'X.0'...'X.N', rather than
        'X'...'X'. Passing in False will cause data to be overwritten if there
        are duplicate names in the columns.
    dtype : Type name or dict of column -> type, default None
        Data type for data or columns. E.g. {'a': np.float64, 'b': np.int32}
        Use `str` or `object` to preserve and not interpret dtype.
        If converters are specified, they will be applied INSTEAD
        of dtype conversion.
    engine : {'c', 'python'}, optional
        Parser engine to use. The C engine is faster while the python engine is
        currently more feature-complete.
    converters : dict, default None
        Dict of functions for converting values in certain columns. Keys can either
        be integers or column labels
    true_values : list, default None
        Values to consider as True
    false_values : list, default None
        Values to consider as False
    skipinitialspace : boolean, default False
        Skip spaces after delimiter.
    skiprows : list-like or integer or callable, default None
        Line numbers to skip (0-indexed) or number of lines to skip (int)
        at the start of the file.

        If callable, the callable function will be evaluated against the row
        indices, returning True if the row should be skipped and False otherwise.
        An example of a valid callable argument would be ``lambda x: x in [0, 2]``.
    skipfooter : int, default 0
        Number of lines at bottom of file to skip (Unsupported with engine='c')
    skip_footer : int, default 0
        .. deprecated:: 0.19.0
           Use the `skipfooter` parameter instead, as they are identical
    nrows : int, default None
        Number of rows of file to read. Useful for reading pieces of large files
    na_values : scalar, str, list-like, or dict, default None
        Additional strings to recognize as NA/NaN. If dict passed, specific
        per-column NA values.  By default the following values are interpreted as
        NaN: '', '#N/A', '#N/A N/A', '#NA', '-1.#IND', '-1.#QNAN', '-NaN', '-nan',
        '1.#IND', '1.#QNAN', 'N/A', 'NA', 'NULL', 'NaN', 'n/a', 'nan',
        'null'.
    keep_default_na : bool, default True
        If na_values are specified and keep_default_na is False the default NaN
        values are overridden, otherwise they're appended to.
    na_filter : boolean, default True
        Detect missing value markers (empty strings and the value of na_values). In
        data without any NAs, passing na_filter=False can improve the performance
        of reading a large file
    verbose : boolean, default False
        Indicate number of NA values placed in non-numeric columns
    skip_blank_lines : boolean, default True
        If True, skip over blank lines rather than interpreting as NaN values
    parse_dates : boolean or list of ints or names or list of lists or dict, default False

        * boolean. If True -> try parsing the index.
        * list of ints or names. e.g. If [1, 2, 3] -> try parsing columns 1, 2, 3
          each as a separate date column.
        * list of lists. e.g.  If [[1, 3]] -> combine columns 1 and 3 and parse as
          a single date column.
        * dict, e.g. {'foo' : [1, 3]} -> parse columns 1, 3 as date and call result
          'foo'

        If a column or index contains an unparseable date, the entire column or
        index will be returned unaltered as an object data type. For non-standard
        datetime parsing, use ``pd.to_datetime`` after ``pd.read_csv``

        Note: A fast-path exists for iso8601-formatted dates.
    infer_datetime_format : boolean, default False
        If True and `parse_dates` is enabled, pandas will attempt to infer the
        format of the datetime strings in the columns, and if it can be inferred,
        switch to a faster method of parsing them. In some cases this can increase
        the parsing speed by 5-10x.
    keep_date_col : boolean, default False
        If True and `parse_dates` specifies combining multiple columns then
        keep the original columns.
    date_parser : function, default None
        Function to use for converting a sequence of string columns to an array of
        datetime instances. The default uses ``dateutil.parser.parser`` to do the
        conversion. Pandas will try to call `date_parser` in three different ways,
        advancing to the next if an exception occurs: 1) Pass one or more arrays
        (as defined by `parse_dates`) as arguments; 2) concatenate (row-wise) the
        string values from the columns defined by `parse_dates` into a single array
        and pass that; and 3) call `date_parser` once for each row using one or
        more strings (corresponding to the columns defined by `parse_dates`) as
        arguments.
    dayfirst : boolean, default False
        DD/MM format dates, international and European format
    iterator : boolean, default False
        Return TextFileReader object for iteration or getting chunks with
        ``get_chunk()``.
    chunksize : int, default None
        Return TextFileReader object for iteration.
        See the `IO Tools docs
        <http://pandas.pydata.org/pandas-docs/stable/io.html#io-chunking>`_
        for more information on ``iterator`` and ``chunksize``.
    compression : {'infer', 'gzip', 'bz2', 'zip', 'xz', None}, default 'infer'
        For on-the-fly decompression of on-disk data. If 'infer' and
        `filepath_or_buffer` is path-like, then detect compression from the
        following extensions: '.gz', '.bz2', '.zip', or '.xz' (otherwise no
        decompression). If using 'zip', the ZIP file must contain only one data
        file to be read in. Set to None for no decompression.

        .. versionadded:: 0.18.1 support for 'zip' and 'xz' compression.

    thousands : str, default None
        Thousands separator
    decimal : str, default '.'
        Character to recognize as decimal point (e.g. use ',' for European data).
    float_precision : string, default None
        Specifies which converter the C engine should use for floating-point
        values. The options are `None` for the ordinary converter,
        `high` for the high-precision converter, and `round_trip` for the
        round-trip converter.
    lineterminator : str (length 1), default None
        Character to break file into lines. Only valid with C parser.
    quotechar : str (length 1), optional
        The character used to denote the start and end of a quoted item. Quoted
        items can include the delimiter and it will be ignored.
    quoting : int or csv.QUOTE_* instance, default 0
        Control field quoting behavior per ``csv.QUOTE_*`` constants. Use one of
        QUOTE_MINIMAL (0), QUOTE_ALL (1), QUOTE_NONNUMERIC (2) or QUOTE_NONE (3).
    doublequote : boolean, default ``True``
       When quotechar is specified and quoting is not ``QUOTE_NONE``, indicate
       whether or not to interpret two consecutive quotechar elements INSIDE a
       field as a single ``quotechar`` element.
    escapechar : str (length 1), default None
        One-character string used to escape delimiter when quoting is QUOTE_NONE.
    comment : str, default None
        Indicates remainder of line should not be parsed. If found at the beginning
        of a line, the line will be ignored altogether. This parameter must be a
        single character. Like empty lines (as long as ``skip_blank_lines=True``),
        fully commented lines are ignored by the parameter `header` but not by
        `skiprows`. For example, if comment='#', parsing '#empty\na,b,c\n1,2,3'
        with `header=0` will result in 'a,b,c' being
        treated as the header.
    encoding : str, default None
        Encoding to use for UTF when reading/writing (ex. 'utf-8'). `List of Python
        standard encodings
        <https://docs.python.org/3/library/codecs.html#standard-encodings>`_
    dialect : str or csv.Dialect instance, default None
        If provided, this parameter will override values (default or not) for the
        following parameters: `delimiter`, `doublequote`, `escapechar`,
        `skipinitialspace`, `quotechar`, and `quoting`. If it is necessary to
        override values, a ParserWarning will be issued. See csv.Dialect
        documentation for more details.
    tupleize_cols : boolean, default False
        .. deprecated:: 0.21.0
           This argument will be removed and will always convert to MultiIndex

        Leave a list of tuples on columns as is (default is to convert to
        a MultiIndex on the columns)
    error_bad_lines : boolean, default True
        Lines with too many fields (e.g. a csv line with too many commas) will by
        default cause an exception to be raised, and no DataFrame will be returned.
        If False, then these "bad lines" will dropped from the DataFrame that is
        returned.
    warn_bad_lines : boolean, default True
        If error_bad_lines is False, and warn_bad_lines is True, a warning for each
        "bad line" will be output.
    low_memory : boolean, default True
        Internally process the file in chunks, resulting in lower memory use
        while parsing, but possibly mixed type inference.  To ensure no mixed
        types either set False, or specify the type with the `dtype` parameter.
        Note that the entire file is read into a single DataFrame regardless,
        use the `chunksize` or `iterator` parameter to return the data in chunks.
        (Only valid with C parser)
    buffer_lines : int, default None
        .. deprecated:: 0.19.0
           This argument is not respected by the parser
    compact_ints : boolean, default False
        .. deprecated:: 0.19.0
           Argument moved to ``pd.to_numeric``

        If compact_ints is True, then for any column that is of integer dtype,
        the parser will attempt to cast it as the smallest integer dtype possible,
        either signed or unsigned depending on the specification from the
        `use_unsigned` parameter.
    use_unsigned : boolean, default False
        .. deprecated:: 0.19.0
           Argument moved to ``pd.to_numeric``

        If integer columns are being compacted (i.e. `compact_ints=True`), specify
        whether the column should be compacted to the smallest signed or unsigned
        integer dtype.
    memory_map : boolean, default False
        If a filepath is provided for `filepath_or_buffer`, map the file object
        directly onto memory and access the data directly from there. Using this
        option can improve performance because there is no longer any I/O overhead.

    Returns
    -------
    result : DataFrame or TextParser

# Alright, we can pass header=None to fix this
flag_data = pd.read_csv(flag_data_url, header=None)
flag_data.head()_____no_output_____flag_data.count()_____no_output_____flag_data.isna().sum()_____no_output_____
</code>

### Yes, but what does it *mean*?

This data is fairly nice - it was "donated" and is already "clean" (no missing values). But there are no variable names - so we have to look at the codebook (also from the site).

```
1. name: Name of the country concerned
2. landmass: 1=N.America, 2=S.America, 3=Europe, 4=Africa, 4=Asia, 6=Oceania
3. zone: Geographic quadrant, based on Greenwich and the Equator; 1=NE, 2=SE, 3=SW, 4=NW
4. area: in thousands of square km
5. population: in round millions
6. language: 1=English, 2=Spanish, 3=French, 4=German, 5=Slavic, 6=Other Indo-European, 7=Chinese, 8=Arabic, 9=Japanese/Turkish/Finnish/Magyar, 10=Others
7. religion: 0=Catholic, 1=Other Christian, 2=Muslim, 3=Buddhist, 4=Hindu, 5=Ethnic, 6=Marxist, 7=Others
8. bars: Number of vertical bars in the flag
9. stripes: Number of horizontal stripes in the flag
10. colours: Number of different colours in the flag
11. red: 0 if red absent, 1 if red present in the flag
12. green: same for green
13. blue: same for blue
14. gold: same for gold (also yellow)
15. white: same for white
16. black: same for black
17. orange: same for orange (also brown)
18. mainhue: predominant colour in the flag (tie-breaks decided by taking the topmost hue, if that fails then the most central hue, and if that fails the leftmost hue)
19. circles: Number of circles in the flag
20. crosses: Number of (upright) crosses
21. saltires: Number of diagonal crosses
22. quarters: Number of quartered sections
23. sunstars: Number of sun or star symbols
24. crescent: 1 if a crescent moon symbol present, else 0
25. triangle: 1 if any triangles present, 0 otherwise
26. icon: 1 if an inanimate image present (e.g., a boat), otherwise 0
27. animate: 1 if an animate image (e.g., an eagle, a tree, a human hand) present, 0 otherwise
28. text: 1 if any letters or writing on the flag (e.g., a motto or slogan), 0 otherwise
29. topleft: colour in the top-left corner (moving right to decide tie-breaks)
30. botright: Colour in the bottom-left corner (moving left to decide tie-breaks)
```

Exercise - read the help for `read_csv` and figure out how to load the data with the above variable names. One pitfall to note - with `header=None` pandas generated variable names starting from 0, but the above list starts from 1..._____no_output_____## Your assignment - pick a dataset and do something like the above

This is purposely open-ended - you can pick any data set you wish. It is highly advised you pick a dataset from UCI or a similar "clean" source.

If you get that done and want to try more challenging or exotic things, go for it! Use documentation as illustrated above, and follow the 20-minute rule (that is - ask for help if you're stuck).

If you have loaded a few traditional datasets, see the following section for suggested stretch goals._____no_output_____

<code>
# TODO your work here!
# And note you should write comments, descriptions, and add new
# code and text blocks as needed
computer_data_names =["vendor", "model", "myct", "mmin", "mmax", "cach", "chmin", "chmax", "prp", "erp"]
computer_data = pd.read_csv("https://archive.ics.uci.edu/ml/machine-learning-databases/cpu-performance/machine.data", header=None, names = computer_data_names)

computer_data.head(), computer_data.isna().sum()_____no_output_____
</code>

## Stretch Goals - Other types and sources of data

Not all data comes in a nice single file - for example, image classification involves handling lots of image files. You still will probably want labels for them, so you may have tabular data in addition to the image blobs - and the images may be reduced in resolution and even fit in a regular csv as a bunch of numbers.

If you're interested in natural language processing and analyzing text, that is another example where, while it can be put in a csv, you may end up loading much larger raw data and generating features that can then be thought of in a more standard tabular fashion.

Overall you will in the course of learning data science deal with loading data in a variety of ways. Another common way to get data is from a database - most modern applications are backed by one or more databases, which you can query to get data to analyze. We'll cover this more in our data engineering unit.

How does data get in the database? Most applications generate logs - text files with lots and lots of records of each use of the application. Databases are often populated based on these files, but in some situations you may directly analyze log files. The usual way to do this is with command line (Unix) tools - command lines are intimidating, so don't expect to learn them all at once, but depending on your interests it can be useful to practice.

One last major source of data is APIs: https://github.com/toddmotto/public-apis

API stands for Application Programming Interface, and while originally meant e.g. the way an application interfaced with the GUI or other aspects of an operating system, now it largely refers to online services that let you query and retrieve data. You can essentially think of most of them as "somebody else's database" - you have (usually limited) access.

*Stretch goal* - research one of the above extended forms of data/data loading. See if you can get a basic example working in a notebook. I suggset image, text, or (public) API - databases are interesting, but there aren't many publicly accessible and they require a great deal of setup._____no_output_____

<code>
import requests
url = "https://api.spacexdata.com/v3/launches/"

r = requests.get(url)
_____no_output_____data = r.json()
data[0]
_____no_output_____#from pandas.io.json import json_normalize
#spacex = json_normalize(data[0])

df =pd.DataFrame(data)
df.head()_____no_output_____list_launch_site = df['launch_site'].tolist()
list_rocket = df['rocket'].tolist()
_____no_output_____#did not help
def change_column_names(toplevel,dataframe_list):
    names = dataframe_list.columns.values.tolist()
    new = []
    for i in names:
        i = toplevel + i
        new.append(i)
    return new_____no_output_____df_launch_site = pd.DataFrame(list_launch_site)
df_rocket = pd.DataFrame(list_rocket)
_____no_output_____df_1 = pd.concat([df, df_launch_site, df_rocket], axis=1).drop(columns = ['launch_site','rocket'])
df_1.head()
_____no_output_____list_fairings = df_1['fairings'].tolist()
#df_fairings = pd.DataFrame(list_fairings)
len(list_fairings)_____no_output_____
</code>

# class notes_____no_output_____

<code>
audio_data_url = "http://archive.ics.uci.edu/ml/machine-learning-databases/audiology/audiology.standardized.data"_____no_output_____audio_data = pd.read_csv(audio_data_url, header=None)
audio_data.head() , audio_data.tail()_____no_output_____audio_data = pd.read_csv(audio_data_url, header=None)_____no_output_____import numpy as np
audio_data.replace('?', np.nan, inplace=True)
_____no_output_____audio_data.replace('f', False, inplace=True)
audio_data.replace('t', True, inplace=True)_____no_output_____audio_data.head()_____no_output_____audio_data_forward_fill = audio_data.fillna(method='ffill')_____no_output_____audio_data_forward_fill.head()_____no_output_____
_____no_output_____
</code>


# Notebook from lfunderburk/Math-Modelling
Path: modelling-covid-19-outbreak/modelling-covid-19.ipynb

![Callysto.ca Banner](https://github.com/callysto/curriculum-notebooks/blob/master/callysto-notebook-banner-top.jpg?raw=true)_____no_output_____## Modelling the COVID19 Outbreak in Canada

In this notebook, we’ll implement a “Susceptible, Exposed, Infected and Recovered” (SEIR) model used in epidemiology, the study of how disease occurs in populations.
_____no_output_____### What is a Mathematical Model

A mathematical model is a description of a system using mathematical concepts and mathematical language.

You can think of a math model as a tool to help us describe what we believe about the workings of phenomena in the world.

We use the language of mathematics to express our beliefs.

We use mathematics (theoretical and numerical analysis) to evaluate the model, and get insights about the original phenomenon._____no_output_____### How do we model a problem using mathematics?

|Step | Description |
|-|-|
|1| Choose what phenomenon you want to model|
|2| What assumptions are you making about the phenomenon|1|
|3| Use a flow diagram to help you determine the structure of your model|1|
|4| Choose equations|2|
|5| Implement equations using Python|2|
|6| Solve equations|2|
|7| Study the behaviour of the model|3|
|8| Test the model|3|
|9| Use the model|3|_____no_output_____### Our phenomenon of interest: modelling number of people affected by COVID-19

Let's turn now to an event that made headlines in 2020: the COVID-19 pandemic.

COVID-19 is a viral infection caused by a pathogen called SARS-CoV-2.

SARS-CoV-2 virus. Illustration by CDC/ Alissa Eckert, MSMI; Dan Higgins, MAMS (2020) _____no_output_____### Assumptions for a first model

1. Mode of transmission of the disease from person to person is through contact ("contact transmission") between a person who interacts with an infectious person.

2. Once a person comes into contact with the pathogen, there is a period of time (called the latency period) in which they are infected, but cannot infect others (yet!).

3. Population is not-constant (that is, people are born and die as time goes by).

4. A person in the population is either one of:
    - Susceptible, i.e. not infected but not yet exposed,
    - Exposed to the infection, i.e. exposed to the virus, but not yet infectious,
    - Infectious, and
    - Recovered from the infection.

5. People can die by "natural causes" during any of the stages. We assume an additional cause of death associated with the infectious stage.

6. People can get reinfected after they recover._____no_output_____### Fow diagram representing those assumptions

How does a person move from one stage into another? In other words, how does a person go from susceptible to exposed, to infected, to recovered?

$\Delta$: Per-capita birth rate.

$\mu$: Per-capita natural death rate.

$\alpha$: Virus-induced average fatality rate.

$\beta$: Probability of disease transmission per contact (dimensionless) times the number of contacts per unit time.

$\epsilon$: Rate of progression from exposed to infectious (the reciprocal is the incubation period).

$\gamma$: Recovery rate of infectious individuals (the reciprocal is the infectious period).

$\delta$: Rate at which a recovered person re-enters into the susceptible category

_____no_output_____### Using Mathematics & Code to Create a Simulation

Using a tool from Calculus called "Differential Equations", we can create a system that will allow us to study our model.

It will look daunting - but don't fret! This is what we refer to as using the language of mathematics to express our beliefs about a phenomenon.

Watch this YouTube Video to see how we get to the equations using our assumptions and the diagram.

_____no_output_____### The mathematical model

$N$ is updated at each time step, and infected peopel die at a higher rate.

$$ N = S + E + I + R$$

We can then express our model using differential equations

$$\frac{dS}{dt} = \Delta N - \beta \frac{S}{N}I - \mu S + \delta R$$

$$\frac{dE}{dt} = \beta \frac{S}{N}I - \mu E - \epsilon E$$

$$\frac{dI}{dt} = \epsilon E - (\gamma+ \mu + \alpha )I$$

$$\frac{dR}{dt} = \gamma I - \mu R -  \delta R$$

Also, we can keep track of people who die due to the infection.

$$\frac{dD}{dt} = \alpha I $$

We can then solve the equations to see how the values for Susceptible (S), Exposed (E), Infectious (I) and Recovered (R) change over time. _____no_output_____### Tinkering with the Parameters: $\beta$, the rate of contact

We can use Python code to solve for and plot the solutions to our system of equations.

Let's start with the rate of contact $\beta$. The more susceptible people are in contact with infectious people, the higher the value of $\beta$.

What happens when we reduce this rate?, i.e. if we find ways to reduce contact between infectious and susceptible people? Run the cells below, and use the widget to find out how the numbers change. _____no_output_____

%run ./scripts/covid19_model.py_____no_output_____interact_manual(tinker_beta,
         beta=widgets.FloatSlider(min=0, max=1, step=0.01, value=0.5,description='Beta: contact rate',style=style));_____no_output_____

When we reduce the contact between infectious and susceptible, we see that the number of new infections each infection generates is lower.

How can we reduce the contact in real life?

We can do things like social distancing, wearing masks, and using vaccines to prevent susceptible people from becoming exposed.

What is the rate $\beta$ required so that each infections generates less than 1 infection?

What happens in our plot when we enter that value for $\beta$?_____no_output_____### Tinkering with the Parameters $\beta$: the rate of contact and $\alpha$: the rate of death due COVID-19

How deadly is COVID-19? Let's tinker with a new parameter $\alpha$ - in our diagram, this corresponds to the death by COVID-19 rate.

_____no_output_____

interact_manual(tinker_beta_alpha,
    beta=widgets.FloatSlider(min=0, max=1, step=0.01, value=0.5,description='Beta: contact rate',style=style),
    alpha=widgets.FloatSlider(min=0, max=1, step=0.01, value=0.5,description='Alpha: COVID-19 death rate',style=style));_____no_output_____

Something interesting will happen now...the deadlier the virus is, the lower the number of new infections generated by each existing infection.

In the table found [here](https://coronavirus.jhu.edu/data/mortality) under "Case-fatality" there are percentages for the case fatality of COVID-19 for different countries.

If you want to try them using our notebook, recall that 1% can be represented by decimal values as 0.01, 10% as 0.1 and 100% as 1.0.

Do you think that COVID-19 is a deadly disease?_____no_output_____### Tinkering with the remaining parameters

Let's incorporate the rest of our parameters into the simulation.

$\Delta$: Per-capita birth rate.

$\mu$: Per-capita natural death rate.

$\alpha$: Virus-induced average fatality rate.

$\beta$: Probability of disease transmission per contact (dimensionless) times the number of contacts per unit time.

$\epsilon$: Rate of progression from exposed to infectious (the reciprocal is the incubation period).

$\gamma$: Recovery rate of infectious individuals (the reciprocal is the infectious period).

$\delta$: Rate at which a recovered person re-enters into the susceptible category

Run the cell below and change the values of parameters to see how the numbers change. _____no_output_____

display(tab)_____no_output_____

### Plotting number of infectious against reported cases of COVID-19 in Canada

Using COVID-19 Open Data [1], we are going to compare our model to the number of daily cases reported in Canada.

[1] COVID-19 Data Repository by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University, https://github.com/CSSEGISandData/COVID-19

In Canada, a person normally gets tested once they start displaying [symptoms of COVID-19](https://www.covid-19canada.com/#symptoms).

By the time a person starts showing symptoms, they can infect others who are in close contact with them.

Let's focus on the number of infectious people then and plot that against real data.

Run the following cell.

Play with the parameters to get a "first guess" of what the parameters in our model are. Try to get as close as possible to the curve of reported data._____no_output_____

%run -i ./scripts/open_data.py
display(tab1)_____no_output_____

### Model's Limitations:

1. Our assumes a constant contact rate - whereas we have modified our contact rate by practicing social distancing, lockdown, and easing lockdown measures.

2. Our model assumes immunity post recovery - which is yet to be proven or disproven.

3. Our model does not take into account inner circles having higher probability of exposure and infection when a member is infectious.

4. Our model does not take into account other factors, such as age, immunodeficiencies, and groups who might be more impacted than others.

5. Model is extremely sensitive to perturbations - small changes in parameters lead to significant changes in number of people in Exposed and Infected categories.

### Data's Limitations:

1. Infected individuals are those who got tested and obtained a positive result - it does not take into account actual cases that were never reported.

2. Infected individuals present symptoms - difficult to measure asymptomatic transmission.

3. Data does not represent accurately whether report is from the same individual at different times.

4. Data is based on test accuracy - a false negative means there might be infected people who tested negative, similar to a false positive, i.e. people who are not infected who test as if they were.
_____no_output_____## Further reading

Infectious Disease Modelling https://towardsdatascience.com/infectious-disease-modelling-beyond-the-basic-sir-model-216369c584c4

Model adapted from Carcione José M., Santos Juan E., Bagaini Claudio, Ba Jing, A Simulation of a COVID-19 Epidemic Based on a Deterministic SEIR Model. Frontiers in Public Health Vol 8, 2020 https://www.frontiersin.org/article/10.3389/fpubh.2020.00230   DOI=10.3389/fpubh.2020.00230   _____no_output_____[![Callysto.ca License](https://github.com/callysto/curriculum-notebooks/blob/master/callysto-notebook-banner-bottom.jpg?raw=true)](https://github.com/callysto/curriculum-notebooks/blob/master/LICENSE.md)_____no_output_____


# Notebook from FlowerPower2/Notebooks
Path: week4/1_For_Loops.ipynb

_____no_output_____# לולאת for_____no_output_____## הקדמה_____no_output_____

    פעמים רבות אנחנו מנסים לפתור בעיה, שעל־מנת להגיע לפתרון שלה אנחנו נדרשים לעבור על כל הערכים שידועים לנו במבנה מסויים:

_____no_output_____

בהנתן רשימת גבהים של האנשים בכתה – מה הגובה של התלמיד הגבוה ביותר בכתה?

איזה קלף מבין כל הקלפים שיש לי ביד הוא הקלף המתאים ביותר לשחק עכשיו?

קבל פלייליסט, והחזר את כל השירים של הלהקה Led Zeppelin.

קבל את רשימת המסעדות בצרפת והדירוגים שלהן, והחזר את 3 המסעדות בעלות הדירוג הגבוה ביותר.

_____no_output_____

    מה משותף לכל אחת מהבעיות שהוצגו מעלה?

    דרך ראויה לפתור אותן היא לולאה שתעבור על כל האיברים שהוצגו בבעיה, ותבצע על כל איבר סדרת פעולות.

    נכתוב בפסאודו קוד דוגמה לפתרון הבעיה הראשונה – מציאת הגובה של התלמיד הגבוה ביותר בכתה:

_____no_output_____

קלוט רשימת גבהים לתוך המשתנה <var>גבהים</var>.

אפס את המשתנה <var>הכי_גבוה_שמצאנו</var> כך שיכיל את הערך 0.

        עבור כל <var>גובה</var> שנמצא בתוך <var>גבהים</var>:

                אם הערך של <var>גובה</var> גדול יותר מ<var>הכי_גבוה_שמצאנו</var>:

                        שים בתוך <var>הכי_גבוה_שמצאנו</var> את הערך של <var>גובה</var>.

החזר את <var>הכי_גבוה_שמצאנו</var>.

_____no_output_____

            ממשו פונקציה שמקבלת רשימת גבהים של האנשים בכתה, ומחזירה את הגובה של התלמיד הגבוה ביותר.

            לדוגמה, עבור הרשימה [1.50, 1.84, 1.73, 1.51] החזירו <samp>1.84</samp>.

            חשוב!

            פתרו לפני שתמשיכו!

_____no_output_____### לולאות while_____no_output_____

    עד כה, אחד מהשימושים הנפוצים שעשינו בלולאת while היה לעבור על איברים ברשימה באמצעות מיקום.

    ראיתם את זה בקוד שכתבתם בתרגול למעלה, בו הייתם צריכים לעבור על גובהי התלמידים אחד אחרי השני.

    יתכן שהקוד שלכם דומה לקוד שאני כתבתי עבור התרגיל הזה:

_____no_output_____

def get_tallest(student_heights):
    current_student_index = 0
    tallest_so_far = 0
    while current_student_index < len(student_heights):
        current_student_height = student_heights[current_student_index]
        if current_student_height > tallest_so_far:
            tallest_so_far = current_student_height
        current_student_index = current_student_index + 1
    return tallest_so_far

heights = [1.50, 1.84, 1.73, 1.51]
print(get_tallest(heights))_____no_output_____

    בקוד למעלה, המשתנה <var>current_student_index</var> מחזיק את מיקום התא שמכיל את גובה התלמיד הנוכחי, אותו אנחנו משווים לתלמיד הגבוה ביותר.

    הביטוי student_heights[current_student_index] משיג את הגובה השמור באותו תא.

    תכליתה של הלולאה בקוד הכתוב מעלה היא המעבר על מספרי התאים של הרשימה, כדי להיות מסוגלים לקבל את התוכן שלהם.

    בתמצות, השתמשנו בלולאות פעמים רבות עד־כה לצורך מעבר על כל האיברים של iterable כלשהו.

_____no_output_____

    עוד לפני הלולאה שלמעלה, <var>current_student_index</var> מאותחל ל־0.

    הלולאה תרוץ כל עוד <var>current_student_index</var> קטן מגודל הרשימה, והקוד שנמצא בתוכה יקח כל פעם את האיבר במקום <var>current_student_index</var> ויבדוק האם הוא גדול מ־<var>tallest_so_far</var>.

    ניתן להגיד שאנחנו מבצעים פעולה עבור כל איבר בתוך <var>student_heights</var>.

_____no_output_____## שימוש ב־for_____no_output_____### שימוש בסיסי_____no_output_____

בפעמים שבהן אנחנו רוצים לבצע משהו עבור כל אחד מהאיברים במבנה כלשהו, נשתמש ב<dfn>לולאת for</dfn>.

נראה, לדוגמה, איך נשתמש בלולאת while על־מנת להדפיס את שמות התלמידות בכתה:

_____no_output_____

names_of_students_in_class = ['Galia', 'Hadas', 'Hen', 'Ilana', 'Ivria', 'Karin', 'Maya', 'Noa']_____no_output_____student_index = 0
while student_index < len(names_of_students_in_class):
    student_name = names_of_students_in_class[student_index]
    print(student_name)
    student_index = student_index + 1_____no_output_____

בעברית, היינו משתמשים בנוסח הבא: עבור כל שם של תלמידה בתוך שמות התלמידות שבכתה, הדפס את שם התלמידה.
_____no_output_____

    נכתוב בפייתון, הפעם בעזרת לולאת for:

_____no_output_____

for student_name in names_of_students_in_class:
    print(student_name)_____no_output_____

השוו את האלגנטיות של הקוד הזה לקוד של ה־while, ואת הדמיון בין הקוד לבין הנוסח המילולי שכתבנו.

_____no_output_____### איך זה עובד?_____no_output_____

לולאת ה־for שראיתם מתחלקת ל־3 חלקים:

_____no_output_____

ערך כלשהו, שחייב להיות iterable. אנחנו נסתכל על כל האיברים שלו לפי הסדר שלהם.

שם משתנה חדש שאנחנו ממציאים. פייתון יצור את המשתנה הזה עבור הלולאה. המשתנה הזה יצביע כל פעם על איבר אחד, לפי הסדר, מתוך ה־iterable.

הפעולה או הפעולות של מה שאנחנו רוצים לבצע עם כל אחד מהאיברים האלו.

_____no_output_____

names_of_students_in_class = ['Galia', 'Hadas', 'Hen', 'Ilana', 'Ivria', 'Karin', 'Maya', 'Noa']

#   השם שאנחנו ממציאים
#          V        Iterable, ערך שניתן לפרק לכלל איבריו
for student_name in names_of_students_in_class:
    print(student_name)  # <---- הפעולות לביצוע_____no_output_____

נבחן את הלולאה שמדפיסה את שמות התלמידות בכתה, שלב אחר שלב, כדי להבין איך לולאת for פועלת.

_____no_output_____

            כמה מקטעי הקוד הבאים לא ירוצו, מכיוון שחלק מהקוד מסומן בהערה.

            המטרה של קטעי הקוד בחלק הזה של המחברת היא להדגיש איזה קוד רץ באותו רגע.

_____no_output_____

names_of_students_in_class = ['Galia', 'Hadas', 'Hen', 'Ilana', 'Ivria', 'Karin', 'Maya', 'Noa']_____no_output_____

תוכן המשתנה <var>names_of_students_in_class</var>

        0
        1
        2
        3
        4
        5
        6
        7

            "Galia"
            "Hadas"
            "Hen"
            "Ilana"
            "Ivria"
            "Karin"
            "Maya"
            "Noa"

            -8
            -7
            -6
            -5
            -4
            -3
            -2
            -1

_____no_output_____

בחזרור הראשון, המשתנה החדש שיצרנו בשם <var>student_name</var> יצביע על הערך הראשון ברשימה, Galia.

נשים לב שהמשתנה <var>student_name</var> לא היה קיים לפני הלולאה, והלולאה היא המבנה הקסום שיוצר את המשתנה וגורם לו להצביע לערכים:

_____no_output_____

for student_name in names_of_students_in_class:  # <--- אנחנו פה
#    print(student_name)_____no_output_____

חזרור ראשון, <var>student_name</var> מצביע על "Galia"

        0
        1
        2
        3
        4
        5
        6
        7

            "Galia"
            "Hadas"
            "Hen"
            "Ilana"
            "Ivria"
            "Karin"
            "Maya"
            "Noa"

            student_name ↑
            -7
            -6
            -5
            -4
            -3
            -2
            -1

_____no_output_____

הדבר הבא שיקרה זה שיודפס התוכן שאליו <var>student_name</var> מצביע:
_____no_output_____

#for student_name in names_of_students_in_class:
    print(student_name)  # <--- אנחנו פה_____no_output_____

סיימנו את החזרור! מה עכשיו?

עולים חזרה לראש הלולאה כדי לבדוק האם נשארו עוד איברים לעבור עליהם:
_____no_output_____

for student_name in names_of_students_in_class:  # <--- אנחנו פה
#    print(student_name)_____no_output_____

המשתנה <var>student_name</var> יעבור להצביע על האיבר הבא, "Hadas":
_____no_output_____

חזרור שני, <var>student_name</var> מצביע על "Hadas"

        0
        1
        2
        3
        4
        5
        6
        7

            "Galia"
            "Hadas"
            "Hen"
            "Ilana"
            "Ivria"
            "Karin"
            "Maya"
            "Noa"

            -8
            student_name ↑
            -6
            -5
            -4
            -3
            -2
            -1

_____no_output_____

ושוב, נדפיס את המחרוזת ש־student_name מצביע אליה:

_____no_output_____

#for student_name in names_of_students_in_class:
    print(student_name)  # <---- עכשיו אנחנו פה_____no_output_____

כך נמשיך לבצע את הלולאה, עד שנגיע לאיבר האחרון ברשימה, התלמידה "Noa":

_____no_output_____

for student_name in names_of_students_in_class: # אנחנו פה, אחרי שעברנו על שמות כללל התלמידות פרט לנועה
#    print(student_name)_____no_output_____

חזרור אחרון, <var>student_name</var> מצביע על "Noa"

        0
        1
        2
        3
        4
        5
        6
        7

            "Galia"
            "Hadas"
            "Hen"
            "Ilana"
            "Ivria"
            "Karin"
            "Maya"
            "Noa"

            -8
            -7
            -6
            -5
            -4
            -3
            -2
            student_name ↑

_____no_output_____

נדפיס פעם אחרונה את המחרוזת עליה אנחנו מצביעים:

_____no_output_____

#for student_name in names_of_students_in_class:
    print(student_name)  # <---- עכשיו אנחנו פה_____no_output_____

כשנחזור לראש הלולאה, נראה שעברנו על כל האיברים.

הלולאה תפסק, והקוד שנמצא אחרי הלולאה ירוץ.

_____no_output_____

החלקים בלולאת for

            מטרה
            איפה בקוד
            דוגמה

            המבנה שאנחנו רוצים לעבור על כלל איבריו – חייב להיות iterable
            אחרי המילה in
            <var>names_of_students_in_class</var>

            שם שנמציא עבור המשתנה שבכל חזרור יכיל את הנתון שעליו נבצע את הפעולה
            אחרי המילה for, לפני המילה in
            <var>student_name</var>

            תוכן הלולאה – סדר הפעולות שנרצה לבצע עם כל נתון
            אחרי הנקודותיים, בשורה חדשה (אחת או יותר), בהזחה
            print(student_name)

_____no_output_____

            שנו את הקוד שכתבתם למציאת האדם הגבוה ביותר בכתה כך שישתמש ב־for ולא ב־while.

            טיפ: השתמשו בפסאודו קוד שניתן לכם למעלה, והשוו בין ה־for לבין ה־while בדוגמה של הדפסת שמות התלמידים.

            חשוב!

            פתרו לפני שתמשיכו!

_____no_output_____### דוגמאות מילוליות נוספות_____no_output_____

עבור כל עמוד בספר – קרא את העמוד.

עבור כל צלחת – הכנס אותה מתחת למים חמים, שפשף אותה היטב עם סקוטץ' ספוג בסבון, נגב אותה עם מגבת יבשה ושים אותה במתקן לייבוש כלים.

חשב את הגובה הממוצע של תלמיד, מתוך רשימה של 1000 תלמידים.

בליל כל הקדושים, התחפש, צא החוצה, ועבור כל בית ברחוב: לך לבית, צלצל בפעמון, אמור "ממתק או תעלול", קח ממתק ואמור תודה.

_____no_output_____### מתי להשתמש?_____no_output_____

לולאת for יוצרת מבנה אלגנטי וקריא, ומתכנתים רבים מעדיפים אותה על־פני לולאת while.

ננסה לעמוד על ההבדלים בין הלולאות:

_____no_output_____

הבדלים בין while ל־for

            נתון להשוואה
            לולאת for
            לולאת while

            מה "מניע" את הלולאה?
            iterable שהלולאה תעבור על כל האיברים שלו
            ביטוי שערכו הבוליאני שקול ל־True או ל־False

            מתי הלולאה מפסיקה
            כשהלולאה עברה על כל האברים של ה־iterable
            כשמגיעים לתנאי של הפונקציה וערכו שקול ל־False

            שימושים עיקריים
            ביצוע פעולה עבור כל ערך בסדרת ערכים,
                כמו איברי רשימה או תווים במחרוזת
            חזרה על פעולה כל עוד מצאנו שהמשימה לא הושלמה

_____no_output_____

בשלב הנוכחי בקורס, תמיד נוכל להשתמש בלולאת while במקום בלולאת for, אך לא תמיד נוכל להחליף לולאות for בלולאות while.

באופן כללי, לולאות while יכולות להוכיח את עצמן כשימושיות מאוד מפעם לפעם.

חישבו, לדוגמה, על מצב שבו אתם צריכים לקבל מהמשתמש קלט חדש כל עוד הקלט שלו לא תקין.

_____no_output_____

            כתבו 3 דוגמאות מילוליות נוספות נוספות ללולאות for.

            חשוב!

            פתרו לפני שתמשיכו!

_____no_output_____### מבנים מורכבים_____no_output_____

הרשימה הבאה מכילה tupleים בגודל 2 איברים:

_____no_output_____

words = [('star', 'rats'), ('wolf', 'flow'), ('racecar', 'racecar'), ('ekans', 'snake')]_____no_output_____

            כתבו לולאת for שתדפיס עבור כל זוג מחרוזות ברשימה: <samp dir="ltr" style="direction: ltr">Reverse "X" to get "Y"</samp>.

            לדוגמה, עבור הזוג האחרון מתוך 4 הזוגות, היא תדפיס: <samp dir="ltr" style="direction: ltr">Reverse "ekans" to get "snake"</samp>.

            חשוב!

            פתרו לפני שתמשיכו!

_____no_output_____

על־מנת לפתור את התרגיל, כתבתם קוד שהשתמש במיקום של הערך הראשון (0) ושל הערך השני (1).

הנה דוגמה לצורה שבה אני פתרתי אותו:

_____no_output_____

for word_and_reversed_word in words:
    word = word_and_reversed_word[0]
    reversed_word = word_and_reversed_word[1]
    print(f'Reverse "{word}" to get "{reversed_word}".')Reverse "star" to get "rats".
Reverse "wolf" to get "flow".
Reverse "racecar" to get "racecar".
Reverse "ekans" to get "snake".

נראה דרך נוספת לפתור את התרגיל, רק שהפעם נשתמש בטריק שנקרא <dfn>unpacking</dfn> (או "<dfn>פירוק</dfn>").

מכיוון שכל tuple ברשימת <var>words</var> מכיל 2 איברים, נוכל לתת להם שמות כבר בראש הלולאה ולחלץ אותם מה־tuple:

_____no_output_____

for word, reversed_word in words:
    print(f'Reverse "{word}" to get "{reversed_word}".')Reverse "star" to get "rats".
Reverse "wolf" to get "flow".
Reverse "racecar" to get "racecar".
Reverse "ekans" to get "snake".

בכל חזרור של הלולאה שכתובה למעלה, ה־for יעבור על tuple בודד, בדיוק כמו שקרה עד עכשיו.

השינוי הוא שבמקום שה־tuple ישמר כמו שהוא במשתנה בודד שהוגדר בראש הלולאה, הערכים שבו "יחולצו" למשתנים בראש הלולאה.

זה יכול לעבוד מכיוון שבראש הלולאה כתבנו מספר משתנים שזהה למספר הערכים בכל tuple.

_____no_output_____

לפניכם תרשים המתאר את תוכן המשתנה <var>words</var>.

הטבלה הגדולה מייצגת את הרשימה <var>words</var>, וכל אחת מהטבלאות הקטנות מייצגת tuple בתוך הרשימה.

בכל חזרור של ה־for המופיע למעלה, המשתנה <var>word</var> יקבל ערך שמסומן באדום, והמשתנה <var>reversed_word</var> יקבל את הערך הירוק התואם.

_____no_output_____

תצוגה על המשתנה <var>words</var> ועל צורת הפירוק שלו

        0
        1
        2
        3

        0
        1

            "star"
            "rats"

            -2
            -1

        0
        1

            "wolf"
            "flow"

            -2
            -1

        0
        1

            "racecar"
            "racecar"

            -2
            -1

        0
        1

            "ekans"
            "snake"

            -2
            -1

            -4
            -3
            -2
            -1

_____no_output_____### שינויים בתוך הלולאה_____no_output_____

פעמים רבות, נעדיף להמנע מלשנות רשימה בזמן לולאת for.

שינויים שכאלו עלולים לגרום להתנהגות לא צפויה, ללולאות אינסופיות ולקוד קשה במיוחד לקריאה.

_____no_output_____#### שינוי של מספר האיברים בזמן לולאה_____no_output_____

    נסתכל על קטע הקוד הזה, שעבור כל איבר ברשימה, מוציא איבר מסוף הרשימה:

_____no_output_____

numbers = ['a', 'b', 'c', 'd', 'e']
print(f"The reader expects {len(numbers)} iterations.")
for i in numbers:
    j = numbers.pop()
    print(i, j)_____no_output_____

    הלולאה נגמרה מוקדם מהרגיל, מכיוון שכשניסתה להגיע לתא שערכו 'd' הוא כבר לא היה שם.

    קוד שכזה אינו צפוי, קשה לקריאה ויוצר תקלים. מאוד לא מומלץ לשנות את כמות האיברים ב־iterable בזמן הריצה.

_____no_output_____

    פתרון אפשרי הוא ליצור עותק של הרשימה באמצעות הפעולה list.copy() ולהשתמש בו במקום:

_____no_output_____

numbers = ['a', 'b', 'c', 'd', 'e']
numbers_to_pop = numbers.copy()
print(f"The reader expects {len(numbers)} iterations.")
for i in numbers:
    j = numbers_to_pop.pop()
    print(i, j)_____no_output_____

#### עריכת הערכים שבתוך ה־iterable_____no_output_____

    ננסה להכפיל כל תא ברשימה שלנו פי 2:

_____no_output_____

numbers = [1, 3, 5]
print(f'This code will multiply every item in {numbers} by 2.')
print(f'The user expects:')
print(f'[{numbers[0] * 2}, {numbers[1] * 2}, {numbers[2] * 2}]')

for num in numbers:
    num = num * 2

print("The final result:")
print(numbers)This code will multiply every item in [1, 3, 5] by 2.
The user expects:
[2, 6, 10]
The final result:
[1, 3, 5]

    כדי להבין למה זה קרה, ננסה להזכר בשיעור על mutability.

    במהלך כל חזרור, המשתנה <var>num</var> מקבל ערך כלשהו להצביע עליו.

    לדוגמה, בחזרור הראשון <var>num</var> מצביע על numbers[0], המקום הראשון ברשימה:

_____no_output__________no_output_____

    בעצם הפעולה num * 2 אנחנו יוצרים ערך חדש שמאוחסן בכתובת שונה.

_____no_output__________no_output_____

    הפעולה num = num * 2 לא באמת "תשנה ערך בתוך <var>num</var>", אלא תגרום לו להצביע על ערך אחר.

_____no_output__________no_output_____

    נסכם כך: המשתנה <var>num</var> בהתחלה הצביע על מספר שנמצא בתוך הרשימה, ועכשיו הוא מצביע על מספר אחר.

    ההשמה, כרגיל, שינתה את המקום שעליו המשתנה מצביע, ולא את הערך שהמשתנה מצביע עליו.

    בקוד שמתואר למעלה, לא שינינו את האיבר בתוך הרשימה.

_____no_output_____### דוגמאות נוספות_____no_output_____#### סכום רשימה_____no_output_____

def total(numbers):
    total = 0
    for number in numbers:
        total = total + number
    return total

print(total([1, 2, 3]))_____no_output_____

#### ראשי תיבות_____no_output_____

def acronym(sentence):
    acronym_word = ''
    for word in sentence.split():
        if len(word) >= 1:
            acronym_word = acronym_word + word[0]
    return acronym_word

print(acronym('North Atlantic Treaty Organization'))_____no_output_____

#### סכום איברים חיוביים_____no_output_____

    קלוט מהמשתמש מספרים. אם צריך, המר את הקלט כך שיהיה מסוג רשימה, ובה יהיו מספרים שלמים. סכום את האיברים הגדולים מ־0.

_____no_output_____

def to_numbers(strings):
    numbers = []
    for semi_number in strings:
        if semi_number.isdecimal():
            numbers.append(int(semi_number))
    return numbers

def sum_positives(numbers):
    total = 0
    for number in numbers:
        if number > 0:
            total = total + number
    return total

user_numbers = input("Enter numbers seperated by comma: ")
stringy_numbers = user_numbers.replace(' ', '').split(',')
numbers = to_numbers(stringy_numbers)
print(sum_positives(numbers))_____no_output_____

#### 7 בום_____no_output_____

    סכום את האיברים שמופיעים בכל מקום שביעי ברשימה.

_____no_output_____

def sum_only_7th_places(numbers):
    total = 0
    for i in numbers[6::7]:
        total = total + i
    return total

print(sum_only_7th_places([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]))_____no_output_____

## תרגולים_____no_output_____### אקרוסטיכון_____no_output_____

<dfn>אקרוסטיכון</dfn> הוא אמצעי בו משתמשים בכתיבה ספרותית.

בשיר בו יש אקרוסטיכון, אם נסתכל על האות הראשונה בכל שורה, נקבל מסר מסויים.

ניקח לדוגמה את שירו של אדגר אלן פו, "אקרוסטיכון", אותו הוא הקדיש ללטישיה אליזבת' לאנדון ובו יש את האקרוסטיכון ELIZABETH:

_____no_output_____

    Elizabeth it is in vain you say

    Love not — thou sayest it in so sweet a way:

    In vain those words from thee or L.E.L.

    Zantippe's talents had enforced so well:

    Ah! if that language from thy heart arise,

    Breath it less gently forth — and veil thine eyes.

    Endymion, recollect, when Luna tried

    To cure his love — was cured of all beside —

    His follie — pride — and passion — for he died.

_____no_output_____

כתבו פונקציה שמקבלת שיר כמחרוזת.

החזירו את האקרוסטיכון שנוצר אם נקח את האות הראשונה מכל שורה בשיר.

טיפ: הפרידו את השורות לפי התו \n שמייצג ירידת שורה.

_____no_output_____### שעורה תרבותית_____no_output_____

    בשדה השעורה בו סיגנט הולכת יש זהב.

    בידה יש מפה, לפיה היא נמצאת כרגע בנקודה (0, 0).

    המפה מתארת אילו צעדים היא צריכה לקחת כדי להגיע למטמון.

    עזרו לסיגנט לחשב: בהנתן שהיא תלך את כל הצעדים שמופיעים במפה – באיזו נקודה ימצא המטמון?

    לדוגמה: עבור הצעדים [(1, 5), (6, -2), (4, 3)] יוחזר שהמטמון נמצא בנקודה(11, 6).

    הסבר מורחב: קבלו רשימה של צעדים המורים לסיגנט כיצד ללכת.

     כל "צעד" מורכב מזוג מספרים שלמים, שיכולים להיות שליליים – הראשון מסמל כמה צעדים ללכת ימינה, והשני מסמל כמה צעדים ללכת למעלה.

    אם המספר הראשון שלילי, יש ללכת את כמות הצעדים הזו שמאלה. אם המספר השני שלילי, יש ללכת את כמות הצעדים הזו למטה.

    כתבו פונקציה שמקבלת רשימה של צעדים ומחזירה איפה נמצא המטמון.

_____no_output_____### גבעת ווטרשיפ_____no_output_____

    בגבעת ווטרשיפ קצב ההתרבות גבוה. כל שנה נוספים עוד ועוד ארנבים לארנבייה.

    חומש הארנב החליט לנהל מעקב דמוגרפי אחרי הגידול.

    הוא מעוניין שתבנו לו פונקציה שמקבלת כפרמטר רשימה של כמות הארנבים שנולדו בכל שנה.

    הפונקציה תחזיר רשימה שבה כל תא מייצג את הכמות הנצברת של הארנבים בארנבייה עד כה.

    לדוגמה: עבור הרשימה [1, 2, 3, 4], הפונקציה תחזיר [1, 3, 6, 10].

_____no_output_____


# Notebook from Serulab/Py4Bio
Path: notebooks/Chapter 7 - Error Handling.ipynb

Python for Bioinformatics
-----------------------------

![title](https://s3.amazonaws.com/py4bio/tapabiosmall.png)

This Jupyter notebook is intented to be used alongside the book [Python for Bioinformatics](http://py3.us/)

_____no_output_____Chapter 7: Error Handling
-----------------------------_____no_output_____

!curl https://raw.githubusercontent.com/Serulab/Py4Bio/master/samples/samples.tar.bz2 -o samples.tar.bz2
!mkdir samples
!tar xvfj samples.tar.bz2 -C samples  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 16.5M  100 16.5M    0     0  14.3M      0  0:00:01  0:00:01 --:--:-- 14.3M
mkdir: cannot create directory 'samples': File exists
BLAST_output.xml
TAIR7_Transcripts_by_map_position.gz
pMOSBlue.txt
fishbacteria.csv
UniVec_Core.nsq
t3beta.fasta
PythonU.db
input4align.dnd
pdb1apk.ent.gz
readme.txt
contig1.ace
example.aln
hsc1.fasta
bioinfo/seqs/15721870.fasta
primers.txt
bioinfo/seqs/4586830.fasta
bioinfo/seqs/7638455.fasta
GSM188012.CEL
3seqs.fas
sampleX.fas
sampleXblast.xml
B1.csv
phd1
conglycinin.phy
bioinfo/seqs/218744616.fasta
spfile.txt
bioinfo/seqs/513419.fasta
bioinfo/seqs/513710.fasta
prot.fas
cas9align.fasta
seqA.fas
bioinfo/seqs/
bioinfo/
pdbaa
other.xml
vectorssmall.fasta
t3.fasta
a19.gp
data.csv
input4align.fasta
B1IXL9.txt
fasta22.fas
bioinfo/seqs/7415878.fasta
bioinfo/seqs/513718.fasta
bioinfo/seqs/513719.fasta
bioinfo/seqs/6598312.fasta
UniVec_Core.nin
Q5R5X8.fas
bioinfo/seqs/513717.fasta
BcrA.gp
bioinfo/seqs/2623545.fasta
bioinfo/seqs/63108399.fasta
conglycinin.dnd
NC2033.txt
fishdata.csv
uniprotrecord.xml
BLAST_output.html
Q9JJE1.xml
test3.csv
UniVec_Core.nhr
sampledata.xlsx
UniVec_Core
NC_006581.gb
conglycinin.multiple.phy
conglycinin.fasta

**Listing 7.1:** wotest.py: Program with no error checking_____no_output_____

with open('myfile.csv') as fh:
    line = fh.readline()
value = line.split('\t')[0]
with open('other.txt',"w") as fw:
    fw.write(str(int(value)*.2))_____no_output_____

**Listing 7.2:** LBYL.py: Error handling LBYL version_____no_output_____

import os
iname = input("Enter input filename: ")
oname = input("Enter output filename: ")
if os.path.exists(iname):
    with open(iname) as fh:
        line = fh.readline()
    if "\t" in line:
        value = line.split('\t')[0]
        if os.access(oname, os.W_OK) == 0:
            with open(oname, 'w') as fw:
                if value.isdigit():
                    fw.write(str(int(value)*.2))
                else:
                    print("Can’t be converted to int")
        else:
            print("Output file is not writable")
    else:
        print("There is no TAB. Check the input file")
else:
    print("The file doesn’t exist")    Enter input filename: hola
Enter output filename: hello
The file doesn’t exist

**Listing 7.3:** exception.py: Similar to 7.2 but with exception handling._____no_output_____

try:
    iname = input("Enter input filename: ")
    oname = input("Enter output filename: ")
    with open(iname) as fh:
        line = fh.readline()
    if '\t' in line:
        value = line.split('\t')[0]
    with open(oname, 'w') as fw:
        fw.write(str(int(value)*.2))
except NameError:
    print("There is no TAB. Check the input file")
except FileNotFoundError:
    print("File not exist")
except PermissionError:
    print("Can’t write to outfile.")
except ValueError:
    print("The value can’t be converted to int")
else:
    print("Thank you!. Everything went OK.")        Enter input filename: hola
Enter output filename: hello
File not exist

**Listing 7.4:** nested.py: Code with nested exceptions_____no_output_____

iname = input("Enter input filename: ")
oname = input("Enter output filename: ")
try:
    with open(iname) as fh:
        line = fh.readline()
except FileNotFoundError:
    print("File not exist")
if '\t' in line:
    value = line.split('\t')[0]
try:
    with open(oname, 'w') as fw:
        fw.write(str(int(value)*.2))
except NameError:
    print("There is no TAB. Check the input file")
except PermissionError:
    print("Can’t write to outfile.")
except ValueError:
    print("The value can’t be converted to int")
else:
    print("Thank you!. Everything went OK.")    Enter input filename: hola
Enter output filename: hello
File not exist
d = {"A":"Adenine","C":"Cytosine","T":"Timine","G":"Guanine"}
try:
    print(d[input("Enter letter: ")])
except:
    print("No such nucleotide")Enter letter: L
No such nucleotide
d = {"A":"Adenine", "C":"Cytosine", "T":"Timine", "G":"Guanine"}
try:
    print(d[input("Enter letter: ")])
except EOFError:
    print("Good bye!")
except KeyError:
    print("No such nucleotide")Enter letter: A
Adenine

**Listing 7.5:** sysexc.py: Using sys.exc_info()_____no_output_____

import sys

try:
    0/0
except:
    a,b,c = sys.exc_info()
    print('Error name: {0}'.format(a.__name__))
    print('Message: {0}'.format(b))
    print('Error in line: {}'.format(c.tb_lineno))Error name: ZeroDivisionError
Message: division by zero
Error in line: 4

**Listing 7.6:** sysexc2.py: Another use of sys.exc_info()_____no_output_____

import sys

try:
    x = open('random_filename')
except:
    a, b = sys.exc_info()[:2]
    print('Error name: {}'.format(a.__name__))
    print('Error code: {}'.format(b.args[0]))
    print('Error message: {}'.format(b.args[1]))Error name: FileNotFoundError
Error code: 2
Error message: No such file or directory
def avg(numbers):
    return sum(numbers)/len(numbers)_____no_output_____ avg([])_____no_output_____def avg(numbers):
    if not numbers:
        raise ValueError("Please enter at least one element")
    return sum(numbers)/len(numbers)_____no_output_____avg([])_____no_output_____