# Notebook from adityavinodk/energy_save
Path: data_inferences/television.ipynb

<code>
import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import precision_score, recall_score, f1_score_____no_output_____with open('../datasets/television_labels.txt', 'r') as f:
    lines = f.readlines()
for line in lines:
    print(line)Energy Rating Data Dictionary for Televisions (labelled) as at May 2015

Column - Column Identifier - Description

Submit_ID - Registration.pk - This is the unique registration ID record for the product and is taken from the GEMS product database

Brand_Reg - Model.brand_name - This is the manufacturers brand

Model_No - Model.model_number - This is the model number of the appliance, usually a unique field

SoldIn - Registration.selling_countries - These are the countries where the product is registered for sale and may include Australia, New Zealand and/or Fiji

Country - Registration.registrationmanufacturingcountry_set - Country of manufacture

screensize - screen_size - The diagonal measurement of the screen in cm

Screen_Area - screen_area - This is the screen area (Height x Width) in cm2

Screen_Tech - screen_tech - This is the type of screen technology (LCD, Plasma, OLED, etc…)

Pasv_stnd_power - pasv_stnd_power - This is the amount of energy used by the appliance in passive standby power mode

Act_stnd_power - act_stnd_time - This is the time allocated that the television will be in active standby mode.

Act_stnd_time - act_stnd_power - This is the amount of energy used by the appliance in active standby power mode

Avg_mode_power - avg_mode_power - This is the amount of energy used by the appliance when the television is in use.

Star - star - This is the star rating from the previous Minimum Energy Performance Standard level. This figure has been superceeded by the field Star2.

SRI - sri - The current Star Rating Index for the appliance.

CEC - cec - This is the current Comparative Energy Consumption of the product which is it's nominal energy consumption expressed as kilowatt hours per years.

SubmitStatus - Registration.get_status_display - This is the registration status of the product and must be either "Superseded" or "Approved"

ExpDate - Registration.expiry_date - This is the date that the product's registration will expire

GrandDate - Registration.superseded_date - This product can be sold after this date but can no longer be manufactured or imported from this date

Product Class - Registration.product_class - Not Applicable

Availability Status - Registration.availability_status - This is the availability status of the product and must be either "Available" or "Unavailable". This status is based on self-reporting of the registrant.

Star2 - star2 - This is the actual star rating of the television.

Product Website - Registration.get_manufacturer_company_website - This is the specific web address for the product itself

Representative Brand URL - Registration.representative_model_number.brand.url - This is the web address for the manufacturer

Star Image Large - This is the energy rating icon for the product and is based on the top half of the Energy Rating Label

Star Image Small - This is a smaller version of the energy rating icon for the product and is based on the top half of the Energy Rating Label
data = pd.read_csv('../datasets/television.csv')
print(data.info())
print(data.describe())<class 'pandas.core.frame.DataFrame'>
RangeIndex: 2513 entries, 0 to 2512
Data columns (total 31 columns):
Submit_ID                      2513 non-null int64
Brand_Reg                      2513 non-null object
Model_No                       2513 non-null object
Family Name                    755 non-null object
SoldIn                         2463 non-null object
Country                        2513 non-null object
screensize                     2513 non-null float64
Screen_Area                    2513 non-null float64
Screen_Tech                    2513 non-null object
Pasv_stnd_power                2453 non-null float64
Act_stnd_power                 2452 non-null float64
Act_stnd_time                  2452 non-null float64
Avg_mode_power                 2513 non-null float64
Star                           438 non-null float64
SRI                            438 non-null float64
CEC                            2513 non-null float64
SubmitStatus                   2513 non-null object
ExpDate                        2513 non-null object
GrandDate                      0 non-null float64
Product Class                  2513 non-null object
Availability Status            2513 non-null object
Star2                          2513 non-null float64
Product Website                651 non-null object
Representative Brand URL       1131 non-null object
Star Rating Index              2513 non-null float64
Star Image Large               2513 non-null object
Star Image Small               2513 non-null object
Power supply                   2513 non-null object
Tuner Type                     2513 non-null object
What test standard was used    2051 non-null object
Registration Number            2513 non-null object
dtypes: float64(12), int64(1), object(18)
memory usage: 608.7+ KB
None
           Submit_ID   screensize   Screen_Area  Pasv_stnd_power  \
count    2513.000000  2513.000000   2513.000000      2453.000000
mean   106159.759252   125.078469   7325.498838         0.338715
std     10256.746722    38.836596   4183.586803         0.157339
min     81191.000000    33.661000    484.110000         0.000000
25%     98454.000000   100.333000   4262.297400         0.270000
50%    107654.000000   125.731000   6754.867344         0.340000
75%    114914.000000   152.661000   9955.880000         0.480000
max    121577.000000   265.901000  27295.170000         0.880000

       Act_stnd_power  Act_stnd_time  Avg_mode_power        Star         SRI  \
count     2452.000000    2452.000000     2513.000000  438.000000  438.000000
mean         2.467913       0.202416       89.075177    7.287671    7.676926
std          8.289937       1.318672       60.544289    0.904165    0.882142
min          0.000000       0.000000        8.600000    5.000000    5.288217
25%          0.000000       0.000000       43.610000    7.000000    7.011724
50%          0.000000       0.000000       81.430000    7.000000    7.679765
75%          0.000000       0.000000      117.860000    8.000000    8.197904
max        105.030000      14.000000      649.000000   10.000000   10.014658

               CEC  GrandDate        Star2  Star Rating Index
count  2513.000000        0.0  2513.000000        2513.000000
mean    339.017509        NaN     4.553721           4.781713
std     223.141832        NaN     0.947202           0.971389
min      34.000000        NaN     1.000000           1.043541
25%     171.000000        NaN     4.000000           4.160645
50%     312.000000        NaN     4.500000           4.748753
75%     434.000000        NaN     5.000000           5.374131
max    2372.000000        NaN     8.000000           8.154055
# Let us first find see the features which we might have to drop
print(data.columns.values)

# A relatively good dataset since the important features are non-empty, we can remove the irrelevant columns
print(data.isnull().sum(), '\n')['Submit_ID' 'Brand_Reg' 'Model_No' 'Family Name' 'SoldIn' 'Country'
 'screensize' 'Screen_Area' 'Screen_Tech' 'Pasv_stnd_power'
 'Act_stnd_power' 'Act_stnd_time' 'Avg_mode_power' 'Star' 'SRI' 'CEC'
 'SubmitStatus' 'ExpDate' 'GrandDate' 'Product Class'
 'Availability Status' 'Star2' 'Product Website'
 'Representative Brand URL' 'Star Rating Index' 'Star Image Large'
 'Star Image Small' 'Power supply' 'Tuner Type'
 'What test standard was used' 'Registration Number']
Submit_ID                         0
Brand_Reg                         0
Model_No                          0
Family Name                    1758
SoldIn                           50
Country                           0
screensize                        0
Screen_Area                       0
Screen_Tech                       0
Pasv_stnd_power                  60
Act_stnd_power                   61
Act_stnd_time                    61
Avg_mode_power                    0
Star                           2075
SRI                            2075
CEC                               0
SubmitStatus                      0
ExpDate                           0
GrandDate                      2513
Product Class                     0
Availability Status               0
Star2                             0
Product Website                1862
Representative Brand URL       1382
Star Rating Index                 0
Star Image Large                  0
Star Image Small                  0
Power supply                      0
Tuner Type                        0
What test standard was used     462
Registration Number               0
dtype: int64

</code>


# Notebook from stephtdouglas/lightkurve
Path: docs/source/tutorials/1-getting-started/using-light-curve-file-products.ipynb

# Using Light Curve Files with Lightkurve_____no_output_____## Learning Goals

By the end of this tutorial, you will:

- Understand how NASA's *Kepler* Mission collected and released light curve data products.
- Be able to download and plot light curve files from the data archive using [Lightkurve](https://docs.lightkurve.org).
- Be able to access light curve metadata.
- Understand the time and brightness units._____no_output_____## Introduction

The [*Kepler*](https://archive.stsci.edu/kepler), [*K2*](https://archive.stsci.edu/k2), and [*TESS*](https://archive.stsci.edu/tess) telescopes observe stars for long periods of time, from just under a month to four years. By doing so they observe how the brightness of stars change over time. A series of these brightness observations is referred to as a [light curve](https://www.nasa.gov/kepler/education/lightcurves/) of a star.

Light curves of stars observed by the *Kepler*, *K2*, or *TESS* missions are created from the raw images collected by these telescopes using software built for this purpose by the mission teams. In this tutorial, we will learn how to use the Lightkurve package to download these preprocessed light curves from *Kepler*'s data archive, plot them, and understand their properties and units.

Much of the explanation below is inspired by [Kinemuchi et al. (2012)](https://arxiv.org/pdf/1207.3093.pdf), an excellent paper introducing and explaining the terminology surrounding the *Kepler* mission and its data. You can find detailed information on the mission and its data products in the official [*Kepler* Instrument Handbook](https://archive.stsci.edu/files/live/sites/mast/files/home/missions-and-data/kepler/_documents/KSCI-19033-002-instrument-hb.pdf) and the [*Kepler* Data Processing Handbook](https://archive.stsci.edu/files/live/sites/mast/files/home/missions-and-data/kepler/_documents/KSCI-19081-003-KDPH.pdf).

We will use the *Kepler* mission as the main example, but these tools are extensible to *TESS* and *K2* as well. For example, while in this tutorial we will learn to work with Lightkurve's [`KeplerLightCurve`](https://docs.lightkurve.org/api/lightkurve.lightcurve.KeplerLightCurve.html) objects, there are also [`TessLightCurve`](https://docs.lightkurve.org/api/lightkurve.lightcurve.TessLightCurve.html) objects that work in the same way._____no_output_____## Imports

This tutorial only requires the [**Lightkurve**](http://docs.lightkurve.org/) package, which in turn uses `matplotlib` for plotting._____no_output_____

import lightkurve as lk
%matplotlib inline_____no_output_____

## 1. About NASA's Photometric Space Telescopes_____no_output_____In order to understand the data produced by NASA's *Kepler*, *K2*, and *TESS* missions, it is useful to understand a little about how these data were obtained._____no_output_____### 1.1. [*Kepler*](https://www.nasa.gov/mission_pages/kepler/overview/index.html)

During its nominal mission, the *Kepler* telescope made observations using 21 pairs of rectangular charge-coupled device (CCD) camera chips (also called *modules*), each consisting of four 1100 x 2048 pixel *channels*. Each observed star fell on one of these 84 CCD channels. Recording the channel numbers for each star was important, because the *Kepler* spacecraft rotated by 90 degrees roughly four times a year. These rotations divide what are referred to as observing *quarters*. While the same star may be observed in multiple quarters, it may fall on a different CCD channel each time.

*Kepler* observed a single field in the sky, although not all stars in this field were recorded as light curves. Instead, pixels were selected around a predetermined list of target stars, which were then downloaded. These downloaded measurements are stored in *target pixel files* (TPFs). By adding up the flux (a measurement of an object's brightness per unit time) measured by the pixels in which a target star appears, the total brightness of a star can be measured. If you make this measurement at different times, you obtain a light curve.

*Kepler* recorded the brightness measurements at two different cadences: a Short Cadence (SC, 58.85 seconds) and a Long Cadence (LC, 29.4 minutes). For more details, read: [*Kepler* Instrument Handbook](https://archive.stsci.edu/files/live/sites/mast/files/home/missions-and-data/kepler/_documents/KSCI-19033-002-instrument-hb.pdf), Section 2.1. *Mission Overview* and 2.6. *Pixels of Interest*, and the [*Kepler* Archive Manual](https://archive.stsci.edu/files/live/sites/mast/files/home/missions-and-data/k2/_documents/MAST_Kepler_Archive_Manual_2020.pdf) Chapter 2: *Kepler* Data Products.

*Figure:* The field of view of the *Kepler* mission. The rectangles represent the CCD modules described above.
_____no_output_____### 1.2. [*K2*](https://www.nasa.gov/feature/ames/nasas-k2-mission-the-kepler-space-telescopes-second-chance-to-shine)

The *Kepler* mission ended in 2013 following the loss of two reaction wheels, leaving the spacecraft unable to stay fixed on one portion of the sky. Instead, it changed its focus to the ecliptic plane, and performed 80-day observing campaigns of 19 separate fields. *K2* data are very similar to *Kepler* data, but are subject to higher levels of instrument noise due to the increased instability of the spacecraft. For more details read the [*K2* Handbook](https://archive.stsci.edu/files/live/sites/mast/files/home/missions-and-data/k2/_documents/KSCI-19116-002.pdf), specifically *Section 2: What's New in K2*.

_____no_output_____### 1.3. [*TESS*](https://tess.mit.edu/)

The Transiting Exoplanet Survey Satellite (*TESS*) succeeded *Kepler* in 2018. The data it collects are very similar to those from *Kepler* and *K2*, but *TESS* covers a much larger area of the sky at a lower resolution. *TESS* observes large sectors of the sky for 27 days at a time. The overlap of these sectors means that stars near the ecliptic poles will receive a year of uninterrupted data, while those near the ecliptic receive only ~27 days. Compared to *Kepler*, *TESS* observes in several different cadence modes, including 20 seconds, 120 seconds, 10 minutes, and 30 minutes. For more details, see the [Mission Overview](https://tess.mit.edu/science/) and the [*TESS* Instrument Handbook](https://archive.stsci.edu/files/live/sites/mast/files/home/missions-and-data/active-missions/tess/_documents/TESS_Instrument_Handbook_v0.1.pdf), specifically *Section 2: Introduction to TESS*.

Some stars that have been observed by *TESS* will also have been observed by *Kepler*, and in some rare cases *K2*.
_____no_output_____## 2. Downloading a Light Curve File_____no_output_____The light curves of stars created by the *Kepler* mission are stored at the [Mikulksi Archive for Space Telescopes](https://archive.stsci.edu/kepler/) (MAST) archive, along with metadata about the observations, such as which CCD channel was used at each time._____no_output_____Lightkurve's built-in tools allow us to search for light curve files in the archive, and download them and their metadata. In this example, we will start by downloading one quarter of *Kepler* data for a star named [Kepler-8](http://www.openexoplanetcatalogue.com/planet/Kepler-8%20b/), a star somewhat larger than the Sun, and the host of a [hot Jupiter planet](https://en.wikipedia.org/wiki/Hot_Jupiter). _____no_output_____Using Lightkurve's [`search_lightcurve`](http://docs.lightkurve.org/api/lightkurve.search.search_lightcurve.html) function, we can find an itemized list of different light curve file products available for Kepler-8:_____no_output_____

search_result = lk.search_lightcurve("Kepler-8", mission="Kepler")
search_result_____no_output_____

In this list, each row represents a different observing period. We find that *Kepler* recorded the maxmimum of 18 quarters of data for this target across four years. The **observation** column lists the *Kepler* Quarter. The **target_name** represents the *Kepler* Input Catalogue (KIC) ID of the target, and the **productFilename** column is the name of the FITS files downloaded from MAST. The **distance** column shows the separation on the sky between the searched coordinates and the downloaded objects — this is only relevant when you pass a `radius` argument to the `search_lightcurve` function to search for targets within a given search radius around a set of coordinates.

The [`search_lightcurve`](http://docs.lightkurve.org/api/lightkurve.search.search_lightcurve.html)  function takes several additional arguments, such as the `quarter` number or the `mission` name. You can find examples of its use in the [online documentation](http://docs.lightkurve.org/api/lightkurve.search.search_lightcurve.html) for this function.

The search function returns a [`SearchResult`](https://docs.lightkurve.org/api/lightkurve.search.SearchResult.html) object which has several convenient operations. For example, we can select the fourth data product in the list as follows:_____no_output_____

search_result[4]_____no_output_____

We can download this data product using the `download()` method:_____no_output_____

klc = search_result[4].download()_____no_output_____

This instruction is identical to the following line:_____no_output_____

klc = lk.search_lightcurve("Kepler-8", mission="Kepler", quarter=4).download()
_____no_output_____

The `klc` variable we have obtained in this way is a [`KeplerLightCurve`](http://docs.lightkurve.org/api/lightkurve.lightcurve.KeplerLightCurve.html) object. This object contains time, flux, and flux error information, as well as a whole lot of data about spacecraft systematics. We can view all of them by calling the object by itself:_____no_output_____

klc_____no_output_____

This object provides a convenient way to interact with the data file that has been returned by the archive, which contains both the light curve data and metadata about the observations.

Before diving into the properties of the light curve file, we can plot the data, also using Lightkurve._____no_output_____

%matplotlib inline
klc.plot();_____no_output_____

On this plot, the y-axis is flux in electrons per second. This unit may appear counterintutive, as flux is a measure of brightness. The CCD cameras measure an electrical charge, and so light is recorded as electrons, not photons as you might expect. On the x-axis we have time in Barycentric Kepler Julian Date (BKJD). In short, the x-axis values are days since the start of the *Kepler* mission. The repeating dips in brightness are transits, the effect of a planet orbiting Kepler-8 and passing between us and the star._____no_output_____#### Note

You can also download light curve FITS files from the archive by hand, store them on your local disk, and open them using the `lk.read(<filename>)` function. This function will return a [`KeplerLightCurve`](http://docs.lightkurve.org/api/lightkurve.lightcurve.KeplerLightCurve.html)  object just as in the above example. You can find out where Lightkurve stored a light curve file using the `filename` attribute:_____no_output_____

klc.filename_____no_output_____

## 3. The SAP and PDCSAP Light Curves_____no_output_____As you can see in the Table above, there are two different types of flux stored in the [`KeplerLightCurve`](http://docs.lightkurve.org/api/lightkurve.lightcurve.KeplerLightCurve.html)  object. These correspond to different levels of data treatment performed for this star by NASA's [*Kepler* Data Processing Pipeline](https://github.com/nasa/kepler-pipeline/): the simple aperture photometry (SAP) flux, and the presearch data conditioning SAP (PDCSAP) flux.

By default, a [`KeplerLightCurve`](http://docs.lightkurve.org/api/lightkurve.lightcurve.KeplerLightCurve.html) will set the PDCSAP flux to its `.flux` property. _____no_output_____To compare the PDCSAP and the SAP flux, we can use the `column` keyword while plotting.

**Note**: alternatively, you can replace the `flux` column with the `sap_flux` column by using `klc.flux = klc['sap_flux']`.
_____no_output_____

ax = klc.plot(column='pdcsap_flux', label='PDCSAP Flux', normalize=True)
klc.plot(column='sap_flux', label='SAP Flux', normalize=True, ax=ax);_____no_output_____

In brief:

* The SAP light curve is calculated by summing together the brightness of pixels that fall within an aperture set by the *Kepler* mission. This is often referred to as the *optimal aperture*, but in spite of its name can sometimes be improved upon! Because the SAP light curve is a sum of the brightness in chosen pixels, it is still subject to systematic artifacts of the mission.

* The PDCSAP light curve is subject to more treatment than the SAP light curve, and is specifically intended for detecting planets. The PDCSAP pipeline attempts to remove systematic artifacts while keeping planetary transits intact.

Looking at the figure we made above, you can see that the SAP light curve has a long-term change in brightness that has been removed in the PDCSAP light curve, while keeping the transits at the same depth. For most inspections, a PDCSAP light curve is what you want to use, but when looking at astronomical phenomena that aren't planets (for example, long-term variability), the SAP flux may be preferred._____no_output_____For now, let's continue to use the PDCSAP flux only. Because this is the default `.flux` property of our light curve object, we don't need to change anything._____no_output_____#### Note

The `plot()` methods in Lightkurve always return a [Matplotlib](https://matplotlib.org/) object. This is useful because it lets us manipulate the plot using standard Matplotlib functions. For example, we can set the title as follows:_____no_output_____

ax = klc.plot()
ax.set_title("PDCSAP light curve of Kepler-8");_____no_output_____

And the figure can be saved as follows:_____no_output_____

ax.figure.savefig('blatmp.png')_____no_output_____

## 4. Accessing the Metadata_____no_output_____When downloading data from MAST, that data usually comes in the format of a FITS file. These FITS files carry a wealth of metadata about the observation. When these are loaded in to Lightkurve to create a [`KeplerLightCurve`](http://docs.lightkurve.org/api/lightkurve.lightcurve.KeplerLightCurve.html), all of the metadata are stored in the `.meta` property of the object.

We can view these metadata by calling this property, as follows:_____no_output_____

klc.meta_____no_output_____

As you can see, there is a lot here if you don't know what you are looking for! These metadata don't just include information about the observations, but also data from the [*Kepler* Input Catalogue](https://ui.adsabs.harvard.edu/abs/2011AJ....142..112B/abstract) (KIC) used to select observing targets, such as their magnitudes and temperature._____no_output_____The `.meta` property is a [Python dictionary](https://docs.python.org/3/tutorial/datastructures.html#dictionaries), which has some convenient features. For example, we can retrieve the value of an individual keyword as follows:_____no_output_____

klc.meta['QUARTER']_____no_output_____

Alternatively, we can use the `.get()` method, which accounts for queries that aren't in the dictionary._____no_output_____

klc.meta.get('MISSION')_____no_output_____

A feature of the [`KeplerLightCurve`](http://docs.lightkurve.org/api/lightkurve.lightcurvefile.KeplerLightCurve.html) object is that the metadata can also be accessed via user-friendly object properties for convenience. For example, the *Kepler* Quarter number is directly accessible via the `quarter` property:_____no_output_____

klc.quarter_____no_output_____

## 5. Understanding the Data Arrays and Units_____no_output_____As we saw above, the [`KeplerLightCurve`](http://docs.lightkurve.org/api/lightkurve.lightcurvefile.KeplerLightCurve.html)  object is a table that contains many arrays other than the PDCSAP and SAP fluxes. Detailed information on each of these can be found in the [*Kepler* Archive Manual](http://archive.stsci.edu/files/live/sites/mast/files/home/missions-and-data/kepler/_documents/archive_manual.pdf), *Section 2.3.1. Light Curve Files*.

The first six columns appear in all [`KeplerLightCurve`](http://docs.lightkurve.org/api/lightkurve.lightcurvefile.KeplerLightCurve.html)  objects, and contain the most commonly used information. These are:_____no_output_____- `time`: the time measurements at each cadence.
- `flux`: the flux of the target star at each time measurement. This is populated with PDCSAP flux by default.
- `flux_err`: the statistical uncertainty on each flux data point.
- `quality`: information on the data quality at each time measurement.
- `centroid_col` & `centroid_row`: the position of the target star on the CCD at each observation. This changes over time due to, for example, small jitters of the spacecraft.

The remaining columns are more detailed information on the observation. Some of these are duplicated in the first five columns described above:

- `timecorr`: correction values that allow users to revert back to non-barycentric timestamps.
- `cadenceno`: these are mission-specific identifiers of each exposure.
- `sap_flux` & `sap_flux_err`: the SAP flux and associated error.
- `sap_bkg` & `sap_bkg_err`: the calculated background (and associated error) inside the aperture used to calculate the SAP flux.
- `pdcsap_flux` & `pdcsap_flux_err`: the PDCSAP flux and associated error. Duplicated by default in `flux` and `flux_err`.
- `sap_quality`: information on the data quality at each time measurement. Duplicated in `quality`.
- `psf_centr1` & `psf_centr2` (and errors): the column and row centroid positions of a PSF model fit to the target star.
- `mom_centr1` & `mom_centr2` (and errors): the column and row centroid positions of the target star, weighted by flux. Duplicated in `centroid_col` and `centroid_row` respectively.
- `pos_corr1` & `pos_corr2`: the column and row components of the calculated image motion._____no_output_____These columns can be accessed as properties of the [`KeplerLightCurve`](http://docs.lightkurve.org/api/lightkurve.lightcurvefile.KeplerLightCurve.html) , for example, as follows:_____no_output_____

klc.sap_bkg_____no_output_____

The unit information of the arrays are stored using Astropy's [`astropy.units`](https://docs.astropy.org/en/stable/units/) module, which means that they are an Astropy [`Quantity`](https://docs.astropy.org/en/stable/api/astropy.units.Quantity.html#astropy.units.Quantity) object. We can view the units as follows:_____no_output_____

print(f'Centroid column unit: {klc.centroid_col.unit}')
print(f'Flux unit: {klc.flux.unit}')_____no_output_____

You can access the data in the form of a standard NumPy array using the `value` attribute:_____no_output_____

klc.centroid_col.value_____no_output_____

We can also plot the data using the [`KeplerLightCurve`](http://docs.lightkurve.org/api/lightkurve.lightcurvefile.KeplerLightCurve.html)'s `plot()` method by passing a `column` keyword argument:_____no_output_____

ax = klc.plot(column='mom_centr1', label='Flux-weighted column position')
klc.plot(ax=ax, column='psf_centr1', label='PSF centroid column position');_____no_output_____

Finally, the `.time` property is a little different. Instead of an Astropy `Quantity` object, it is an Astropy [`Time`](https://docs.astropy.org/en/stable/time/) object, and has some additional time scale and format information._____no_output_____

klc.time_____no_output_____print(f'Time scale: {klc.time.scale}')
print(f'Time format: {klc.time.format}')_____no_output_____

Here, the *time format* is the unit of time, in this case Barycentric Kepler Julian Date (BKJD). The *time scale* indicates how the time is measured, in this case by taking the Barycentric Dynamical Time (TDB). This detailed information may be important when comparing observations of a periodic event (such as a planet transit) with observations made with other telescopes on Earth._____no_output_____## Exercises_____no_output_____Some stars, such as Kepler-10, have been observed both with *Kepler* and *TESS*. In this exercise, download and plot the *TESS* PDCSAP flux only. You can do this by either selecting it from the `SearchResult` returned by [`search_lightcurve()`](https://docs.lightkurve.org/api/lightkurve.search.search_lightcurve.html) or by using the `mission` keyword argument when searching._____no_output_____

#search_result = lk.search_lightcurvefile(...)

_____no_output_____# Solution:
search_result = lk.search_lightcurve('Kepler-10', mission='TESS')
search_result_____no_output_____search_result.download().plot();_____no_output_____

## About this Notebook_____no_output_____**Authors:** Oliver Hall (oliver.hall@esa.int), Geert Barentsen

**Updated On**: 2020-08-31_____no_output_____## Citing Lightkurve and Astropy

If you use `lightkurve` or `astropy` for published research, please cite the authors. Click the buttons below to copy BibTeX entries to your clipboard. _____no_output_____

lk.show_citation_instructions()_____no_output_____

_____no_output_____


# Notebook from rafaelscnunes/mab786
Path: presentations/word2vec_wisard.ipynb

from gensim.models import KeyedVectors
from gensim.scripts.glove2word2vec import glove2word2vec
from WiSARD import WiSARD
import numpy as np
import pandas as pd
import math
import sys
import random
import matplotlib.pyplot as plt
import itertools
# import confusion matrix function
from sklearn.metrics import confusion_matrix
from sklearn.metrics import accuracy_score
def plot_confusion_matrix(cm, classes,
                          normalize=False,
                          title='Confusion matrix',
                          cmap=plt.cm.Blues):
    """
    This function prints and plots the confusion matrix.
    Normalization can be applied by setting `normalize=True`.
    """
    plt.imshow(cm, interpolation='nearest', cmap=cmap)
    plt.title(title)
    plt.colorbar()
    tick_marks = np.arange(len(classes))
    plt.xticks(tick_marks, classes, rotation=45)
    plt.yticks(tick_marks, classes)

    if normalize:
        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]
        print("Normalized confusion matrix")
    else:
        print('Confusion matrix, without normalization')

    print(cm)

    thresh = cm.max() / 2.
    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):
        plt.text(j, i, cm[i, j],
                 horizontalalignment="center",
                 color="white" if cm[i, j] > thresh else "black")

    plt.tight_layout()
    plt.ylabel('Classe verdadeira')
    plt.xlabel('Classe prevista')

def thermometer(word_vec, min_=-1.0, max_=1.0, n=20):
    vec = []
    for v in word_vec:
        t = (max_ - min_)/n
        rang = []
        for i in np.arange(min_ + t, max_, t):
            if v > i:
                rang.append(1)
            else:
                break
        rang.extend([0] * (n-len(rang)-1))
        vec.extend(rang)
    return np.array(vec)

def thermometer_one_hot(real_vec, min_=-1.0, max_=1.0, n=20):
    vec = []
    for v in real_vec:
        if v == max_:
            rang = [0] * n
            rang[-1] = 1
            vec.extend(rang)
        else:
            rang = [0] * n
            t = (max_ - min_)
            p = v - min_
            s = int((p / t) * n)
            rang[s] = 1
            vec.extend(rang)
    return np.array(vec)_____no_output_____model = KeyedVectors.load_word2vec_format('../glove.6B.50d.word2vec.txt')_____no_output_____

Word2vec with thermometer of 10 bits_____no_output_____

df = {
    "cooking": pd.read_csv('../dataset/processed/cooking.csv'),
    "crypto": pd.read_csv('../dataset/processed/crypto.csv'),
    "robotics": pd.read_csv('../dataset/processed/robotics.csv'),
    "biology": pd.read_csv('../dataset/processed/biology.csv'),
    "travel": pd.read_csv('../dataset/processed/travel.csv'),
    "diy": pd.read_csv('../dataset/processed/diy.csv'),
    #"physics": pd.read_csv('physics.csv'),
}

X = []
y = []
for file in df:
    for i in range(2000):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X.append(thermometer(v, n=10))
            y.append(file)

X_test = []
y_test = []
for file in df:
    for i in range(2001,2501):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X_test.append(thermometer(v, n=10))
            y_test.append(file)

w = WiSARD(2, seed=random.randint(0, 2**32-1), ignore_zero_addr=True)
#for i in range(len(X)):
#    w.fit([X[i]], [y[i]])
w.fit(X, y)
result = w.predict(X_test)

# Making the Confusion Matrix
cm = confusion_matrix(y_test, result, labels=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'])
#class_names = ['A','B','C','D','E','F','G','H']
# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cm, classes=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'], title='Matriz de Confusao')
plt.show()

print('Accuracy: '+str(accuracy_score(y_test, result)))Confusion matrix, without normalization
[[ 55  30   0   0 415   0]
 [  0  40   0   0 460   0]
 [  1  44   0   0 455   0]
 [  6  80   0  14 400   0]
 [  1   0   0   0 499   0]
 [  0   2   0   1 497   0]]

Thermometer of 10 bits predicted with NaiveBayes_____no_output_____

X = []
y = []
for file in df:
    for i in range(2000):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X.append(thermometer(v, n=10))
            y.append(file)

X_test = []
y_test = []
for file in df:
    for i in range(2001,2501):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X_test.append(thermometer(v, n=10))
            y_test.append(file)

from sklearn.naive_bayes import GaussianNB
w = GaussianNB()
w.fit(X, y)
result = w.predict(X_test)

# Making the Confusion Matrix
cm = confusion_matrix(y_test, result, labels=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'])
#class_names = ['A','B','C','D','E','F','G','H']
# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cm, classes=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'], title='Matriz de Confusao')
plt.show()

print('Accuracy: '+str(accuracy_score(y_test, result)))Confusion matrix, without normalization
[[ 79   4  28   4 381   4]
 [  1 210 156   1 132   0]
 [  2  29 334   2 133   0]
 [  6  33 163  43 255   0]
 [  1   2   8   0 489   0]
 [  1   5 136   1 349   8]]

No thermometer and prediction made with NaiveBayes_____no_output_____

X = []
y = []
for file in df:
    for i in range(2000):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X.append(v)
            y.append(file)

X_test = []
y_test = []
for file in df:
    for i in range(2001,2501):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X_test.append(v)
            y_test.append(file)

from sklearn.naive_bayes import GaussianNB
w = GaussianNB()
w.fit(X, y)
result = w.predict(X_test)

# Making the Confusion Matrix
cm = confusion_matrix(y_test, result, labels=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'])
#class_names = ['A','B','C','D','E','F','G','H']
# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cm, classes=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'], title='Matriz de Confusao')
plt.show()

print('Accuracy: '+str(accuracy_score(y_test, result)))Confusion matrix, without normalization
[[446   4   2   4   7  37]
 [  2 447  15  22  12   2]
 [  4  65 366  24   8  33]
 [ 14  36  22 371  19  38]
 [  6   9  12   7 460   6]
 [ 14   2  23  15  12 434]]

Thermometer of 50 bits and prediction made with WiSARD_____no_output_____

X = []
y = []
for file in df:
    for i in range(2000):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X.append(thermometer(v, n=50))
            y.append(file)

X_test = []
y_test = []
for file in df:
    for i in range(2001,2501):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X_test.append(thermometer(v, n=50))
            y_test.append(file)

w = WiSARD(2, seed=random.randint(0, 2**32-1), ignore_zero_addr=True)
#for i in range(len(X)):
#    w.fit([X[i]], [y[i]])
w.fit(X, y)
result = w.predict(X_test)

# Making the Confusion Matrix
cm = confusion_matrix(y_test, result, labels=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'])
#class_names = ['A','B','C','D','E','F','G','H']
# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cm, classes=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'], title='Matriz de Confusao')
plt.show()

print('Accuracy: '+str(accuracy_score(y_test, result)))Confusion matrix, without normalization
[[ 87  11  17   2 383   0]
 [  0  60   3   2 435   0]
 [  1  98   5   1 395   0]
 [ 14  66  29  30 360   1]
 [  0   0   1   0 499   0]
 [  8   5   3   2 482   0]]

Thermometer of 50 bitd and predicted with NaiveBayes_____no_output_____

X = []
y = []
for file in df:
    for i in range(2000):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X.append(thermometer(v, n=50))
            y.append(file)

X_test = []
y_test = []
for file in df:
    for i in range(2001,2501):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X_test.append(thermometer(v, n=50))
            y_test.append(file)

from sklearn.naive_bayes import GaussianNB
w = GaussianNB()
w.fit(X, y)
result = w.predict(X_test)

# Making the Confusion Matrix
cm = confusion_matrix(y_test, result, labels=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'])
#class_names = ['A','B','C','D','E','F','G','H']
# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cm, classes=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'], title='Matriz de Confusao')
plt.show()

print('Accuracy: '+str(accuracy_score(y_test, result)))Confusion matrix, without normalization
[[193  61   2   7 111 126]
 [  0 483   1   6   9   1]
 [  3 390  80   4  11  12]
 [  7 183  83 133  68  26]
 [  1  73   4   0 419   3]
 [  3 141  16   3  94 243]]

Image comparison of 10 bits regular thermometer and proposed_____no_output_____

from matplotlib import pyplot

X = []
y = []
for file in df:
    for i in range(2000):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X.append(thermometer(v, n=10))
            print(doc)
            y.append(file)
            break

pixels = np.array(X[0], dtype='uint8')
# Reshape the array into 28 x 28 array (2-dimensional array)
pixels = pixels.reshape((50, 9))

# Plot
pyplot.imshow(pixels, cmap='gray', interpolation='nearest')
pyplot.show()

X = []
y = []
for file in df:
    for i in range(2000):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X.append(thermometer_one_hot(v, n=10))
            #print(doc)
            y.append(file)
            break

pixels = np.array(X[0], dtype='uint8')
# Reshape the array into 28 x 28 array (2-dimensional array)
pixels = pixels.reshape((50, 10))

# Plot
pyplot.imshow(pixels, cmap='gray', interpolation='nearest')
pyplot.show()fianc and are looking for good caribbean cruise october and were wondering which islands are best see and which cruise line take seems like lot the cruises don run this month due hurricane season looking for other good options edit travelling
why use table the first step des algorithm and one the end the algorithm
imagine wheel soccer robot what type controller would you use for spinning pid the goal for this controller that should make the robot stand defined angle degree and turn back rotated hand other robot use stepper motors for robot and not servos need implement this software have written sample type controller already and the movement fairly good but would like make better possible the code follows correction range which robot has movement degree number between and which returned from the compass motorspeed number between and which applied the pwm
chocolate chips cookies are always too crisp how can get chewy cookies like those starbucks thank you everyone who has answered far the tip that had the biggest impact was chill and rest the dough however also increased the brown sugar ratio and increased bit the butter also adding maple syrup helped
how critical for efficient the location the ribosome binding site relative the start codon ideally supposed away from the start how about bases away even more will this have observable effect
looking finish basement and simply want wall off the concrete blocks that make the wall want insulate well that becomes usable family room when done how about installing wall given wooden beams the ceiling and concrete floor

Proposed thermometer with 10 bits WiSARD_____no_output_____

X = []
y = []
for file in df:
    for i in range(2000):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X.append(thermometer_one_hot(v, n=10))
            y.append(file)

X_test = []
y_test = []
for file in df:
    for i in range(2001,2501):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X_test.append(thermometer_one_hot(v, n=10))
            y_test.append(file)

w = WiSARD(2, seed=random.randint(0, 2**32-1), ignore_zero_addr=True)
#for i in range(len(X)):
#    w.fit([X[i]], [y[i]])
w.fit(X, y)
result = w.predict(X_test)

# Making the Confusion Matrix
cm = confusion_matrix(y_test, result, labels=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'])
#class_names = ['A','B','C','D','E','F','G','H']
# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cm, classes=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'], title='Matriz de Confusao')
plt.show()

print('Accuracy: '+str(accuracy_score(y_test, result)))Confusion matrix, without normalization
[[241   2   2   5   1 249]
 [  0 220   2   2  12 264]
 [  2  53  68   4   5 368]
 [ 13  32   2 186  14 253]
 [  9  27   2   4 276 182]
 [  1   0   6   3   2 488]]

Proposed with NaiveBayes_____no_output_____

X = []
y = []
for file in df:
    for i in range(2000):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X.append(thermometer_one_hot(v, n=10))
            y.append(file)

X_test = []
y_test = []
for file in df:
    for i in range(2001,2501):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X_test.append(thermometer_one_hot(v, n=10))
            y_test.append(file)

from sklearn.naive_bayes import GaussianNB
w = GaussianNB()
w.fit(X, y)
result = w.predict(X_test)

# Making the Confusion Matrix
cm = confusion_matrix(y_test, result, labels=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'])
#class_names = ['A','B','C','D','E','F','G','H']
# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cm, classes=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'], title='Matriz de Confusao')
plt.show()

print('Accuracy: '+str(accuracy_score(y_test, result)))Confusion matrix, without normalization
[[161  10  41   2 273  13]
 [  1 321 109   3  66   0]
 [  2  59 380   5  52   2]
 [  7  68 159  77 187   2]
 [  1  10  12   2 475   0]
 [  1  10 141   5 252  91]]

Proposed with 50 bits and n=4 WiSARD_____no_output_____

X = []
y = []
for file in df:
    for i in range(2000):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X.append(thermometer_one_hot(v, n=50))
            y.append(file)

X_test = []
y_test = []
for file in df:
    for i in range(2001,2501):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X_test.append(thermometer_one_hot(v, n=50))
            y_test.append(file)

w = WiSARD(4, seed=random.randint(0, 2**32-1), ignore_zero_addr=True)
#for i in range(len(X)):
#    w.fit([X[i]], [y[i]])
w.fit(X, y)
result = w.predict(X_test)

# Making the Confusion Matrix
cm = confusion_matrix(y_test, result, labels=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'])
#class_names = ['A','B','C','D','E','F','G','H']
# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cm, classes=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'], title='Matriz de Confusao')
plt.show()

print('Accuracy: '+str(accuracy_score(y_test, result)))Confusion matrix, without normalization
[[442   6  10   5  10  27]
 [  2 384  73  27   9   5]
 [  5  50 382  30   4  29]
 [ 17  34  38 362  19  30]
 [  6  13  17   6 449   9]
 [ 21   5  50  11  25 388]]

For n=8_____no_output_____

X = []
y = []
for file in df:
    for i in range(2000):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X.append(thermometer_one_hot(v, n=50))
            y.append(file)

X_test = []
y_test = []
for file in df:
    for i in range(2001,2501):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X_test.append(thermometer_one_hot(v, n=50))
            y_test.append(file)

w = WiSARD(8, seed=random.randint(0, 2**32-1), ignore_zero_addr=True)
#for i in range(len(X)):
#    w.fit([X[i]], [y[i]])
w.fit(X, y)
result = w.predict(X_test)

# Making the Confusion Matrix
cm = confusion_matrix(y_test, result, labels=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'])
#class_names = ['A','B','C','D','E','F','G','H']
# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cm, classes=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'], title='Matriz de Confusao')
plt.show()

print('Accuracy: '+str(accuracy_score(y_test, result)))Confusion matrix, without normalization
[[435   2   6   8   9  40]
 [  3 365  83  32   8   9]
 [  4  42 372  36   7  39]
 [ 16  26  28 377  17  36]
 [  6  13  18   6 442  15]
 [ 19   2  45  13  18 403]]

Proposed thermometer with 100 bits, n=4 and WiSARD_____no_output_____

X = []
y = []
for file in df:
    for i in range(2000):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X.append(thermometer_one_hot(v, n=100))
            y.append(file)

X_test = []
y_test = []
for file in df:
    for i in range(2001,2501):#range(df[file].shape[0]):
        #data.append(df[file].iloc[i]['title'])
        #data.append(df[file].iloc[i]['content'])
        if type(df[file].iloc[i]['content']) is str:
            doc = df[file].iloc[i]['content']
            v = np.array([0] * 50)
            w = doc.split(' ')
            for j in w:
                if j in model:
                    v = np.add(v, model[j])
            v = v / np.linalg.norm(v) # normalized
            X_test.append(thermometer_one_hot(v, n=100))
            y_test.append(file)

w = WiSARD(4, seed=random.randint(0, 2**32-1), ignore_zero_addr=True)
#for i in range(len(X)):
#    w.fit([X[i]], [y[i]])
w.fit(X, y)
result = w.predict(X_test)

# Making the Confusion Matrix
cm = confusion_matrix(y_test, result, labels=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'])
#class_names = ['A','B','C','D','E','F','G','H']
# Plot non-normalized confusion matrix
plt.figure()
plot_confusion_matrix(cm, classes=['cooking', 'crypto', 'robotics', 'biology', 'travel', 'diy'], title='Matriz de Confusao')
plt.show()

print('Accuracy: '+str(accuracy_score(y_test, result)))Confusion matrix, without normalization
[[446   4   8   5   6  31]
 [  3 394  48  39   8   8]
 [  5  69 341  39   6  40]
 [ 13  33  13 399  15  27]
 [  7   9  16  13 444  11]
 [ 19   9  33  12  19 408]]


# Notebook from nick87ds/MaterialeSerate
Path: AnalisiTesto/Extra/WikiClassifier.ipynb

%config IPCompleter.greedy=True
%config Completer.use_jedi = False_____no_output_____import glob as glob
import re
import string
import numpy as np
import pandas as pd
import json_____no_output_____import spacy_____no_output_____spacy.__version_______no_output_____nlp = spacy.load("it_core_news_sm")

frase_esempio = "Mi piace New York"

doc = nlp(frase_esempio)

for n, token in enumerate(doc):
    print(f"{n+1}) {token}")1) Mi
2) piace
3) New
4) York
nlp.add_pipe("merge_entities")

doc = nlp(frase_esempio)

for n, token in enumerate(doc):
    print(f"{n+1}) {token}")1) Mi
2) piace
3) New York

## Generator articoli_____no_output_____Quando il dataset si compone di moltissimi articoli diventa dispendioso, in termini di tempo, costruire prima una lista con i nomi dei files e poi ciclare nuovamente quest'ultima per elaborarne il contenuto.

A questo scopo è utile utilizzare la libreria "glob" la quale permette anche di "filtrare" gli articoli in base al nome ed all'estensione._____no_output_____

articoli = iter(glob.glob("archivio/*.txt"))_____no_output_____next(articoli)_____no_output_____

##### La classe di appartenenza dell'articolo è inserita nel nome del file tra due "_", la estrarremo attraverso una espressione regolare_____no_output_____Strumenti come Spacy e le regular expression possono essere molto utili per costruire un dataset strutturato partendo da dati semi-strutturati o, peggio, non strutturati._____no_output_____

keywords = re.compile(r"_(\w+)_")_____no_output_____

#### Anteprima file_____no_output_____Prima di costruire la funzione per estrarre il contenuto del file guardiamo un'anteprima_____no_output_____

articolo = next(articoli)

with open(articolo, 'r') as f:
    testo = f.readlines()
    testo = "".join(testo)

print(testo[:100])
print("-"*100)
print(testo[-100:])
<title>Potenza Calcio</title>Il Potenza Calcio S.r.l., meglio noto come Potenza, è un
----------------------------------------------------------------------------------------------------
'Internet Archive", 'Categoria:Voci con modulo citazione e parametro pagine']</categories>

Il contenuto del file è organizzato in tag annidati, attraverso le regular expressions possiamo estrarre queste informazioni in modo strutturato.

In particolare possiamo notare che il tag "categories" necessita di particolare "pulizia"._____no_output_____##### Altra regular expression per estrarre i dati in formato dictionary_____no_output_____

tags = re.compile(r"<title>(?P<titolo>.*)</title>(?P<corpo>.*)<categories>(?P<categorie>.*)</categories>")_____no_output_____

##### Estrazione e Primo preprocessing_____no_output_____

def estrati_dati(articolo):

    # Identificazione della class/keywords estraendola dal nome del file
    kw = keywords.findall(articolo)[0]

    with open(articolo, 'r') as f:
        testo = f.readlines()

    testo = "".join(testo)
    testo = testo.replace("\n","")

    # Estrazione dei campi di interesse per ottenere il dictionary
    m = tags.search(testo)
    dati = m.groupdict()

    # Qui applicheremo la pulizia "speciale" per il testo
    # del tag categorie (se presente)
    if 'categorie' in dati.keys():

        get_text = re.compile("\'|\\[|\\]|Categoria:")

        categorie = m.group('categorie')
        elenco_categorie = []

        for categ in get_text.sub("", categorie).split(","):
            elenco_categorie.append(categ.strip())

        dati['categorie'] = ",".join(elenco_categorie)

    # Rimozione della keyword, che identifica la classe, per aumentare
    # la capacità di generalizzare del classificatore
    # Ho utilizzato un'assegnazione dinamica del valore "wk" per otterene:
    #                                                  r"\bsalute\b" - r"\btecnologia\b" - r"\bsport\b"
    remove_kw = re.compile(fr"\b{kw}\b", re.IGNORECASE)

    testo = " ".join([txt for k, txt in dati.items()])
    testo = re.sub("\W|\d", " ", testo)
    testo = re.sub("[ ]+", " ", testo)
    testo = remove_kw.sub("", testo)

    return [testo, kw]
    _____no_output_____

##### Un secondo preprocessing consiste nel lemming delle parole_____no_output_____Il lemming () permette una riduzione dello spazio dimensionale delle variabili/dei parametri elaborati dall'algoritmo. Questo implica, tra l'altro, un minor tempo di elaborazione.

Questo processo assomiglia un poco alla standardizzazione che si applica alle variabili quantitative prima di una regressione statistica._____no_output_____

articolo = next(articoli)

testo, classe = estrati_dati(articolo)
doc = nlp(testo)

print(f"Classe {classe} \n")

print(f"Frase originale contiene n.{len(testo.split(' '))} parole diverse:\n\n", estrati_dati(articolo)[0][:250], "\n")

lemma_text = " ".join(token.lemma_ for token in doc if not token.is_stop)

print(f"Frase lemmizzata senza stop_words contiente n.{len(lemma_text.split(' '))} parole diverse:\n\n",lemma_text[:250])Classe tecnologia

Frase originale contiene n.1215 parole diverse:

  di Star Trek L universo di Star Trek la serie televisiva fantascientifica originariamente ideata da Gene Roddenberry nel è inscindibilmente legato ad una immagine di avanzato sviluppo tecnologico Il divario tecnologico è uno degli aspetti che più sp

Frase lemmizzata senza stop_words contiente n.777 parole diverse:

   Star Trek L universo di Star Trek serio televisivo fantascientifico originariamente ideare Gene Roddenberry essere inscindibilmente legare immagine avanzare sviluppare tecnologico Il divario tecnologico essere aspetto emergere puntato Star Trek voc

### Un esempio pratico_____no_output_____##### Dati senza lemming:_____no_output_____PAROLE|CLASSE
---------|-------
 |
**TRAINING**|
corrono|sport
correva|sport
corro|sport
 |
**TEST**|
corre|**???**_____no_output_____I dati test potrebbero contenere parole che l'algoritmo non conosce, ad esempio "corre". In questo caso l'algoritmo li tratterebbe come outliers (valori anomali) non assegnando ad essi nessun valore a discapito della precisione nella classificazione.

**Possiamo dire che, in questo caso, il classificatore invece di generalizzare si addestra a riconoscere una serie di eccezioni... quindi potrebbe anche funzionare ma non perchè lo stiamo applicando nel modo corretto.**_____no_output_____

parole = nlp("corrono correva corro corre")

for parola in parole:
    print(f"{parola} ---> {parola.lemma_}")corrono ---> correre
correva ---> correre
corro ---> correre
corre ---> correre

Dall'esempio di cui sopra si può notare come, attraverso, il lemming abbiamo bisogno di meno parametri, le 3 diverse parole (corrono, correva, corro) diventano solo 1 (correre) senza che ciò comporti una sensibile perdita di informazione che solitamente avviene quando si applicano tecniche di riduzione._____no_output_____### Introduzione del Vocabolario_____no_output_____Il vocabolario è l'insieme di parole "conosciute" dall'algoritmo e della frequenze assolute con la quale esse sono presenti in tutti gli articoli processati.

Per costruirlo utilizzeremo un particolare tipo di dictionary: il default dictionary (defaultdict).

Questo tipo di dictionary ha una particolarità non "fallisce" quando cerchiamo in esso un indice non presente... ma vediamo un esempio prima con un normale dictionary._____no_output_____

d = dict()_____no_output_____lista_esempio = ["parola 1","parola 1","parola 1","New York","parola2"]_____no_output_____for parola in lista_esempio:

    d[parola] += 1_____no_output_____from collections import defaultdict
import json_____no_output_____dd = defaultdict(int)

for parola in lista_esempio:

    dd[parola] += 1

dd_____no_output_____

##### Per la costruzione del vocabolario ho costruito una classe che oltre al conteggio delle frequenze mi aiuti a fare anche altre operazioni._____no_output_____

class Vocabulary():

    def __init__(self, nome_file = "", tokens = False):
        self.nome_file = nome_file
        self.vocab = defaultdict(int)
        self.tokens = tokens
        self.word2idx = None
        self.idx2word = None
        self.stopwords = []

        if self.nome_file != "":
            self.load(self.nome_file, self.tokens)

    def import_stopwords(self, nome_file):
        with open(nome_file, 'r') as f:
            for line in f.readlines():
                self.stopwords.append(line.replace("\n", ""))

    def builder(self, token):

        """
        Aggiuge il conteggio dei token al vocabolario
        """

        self.vocab[str(token)] += 1

    def reduce(self, filter_func = lambda x, y=5: x>y):

        """
        Trasforma in vocabolario senza frequenze
        """

        self.word2idx = defaultdict(int)
        self.idx2word = defaultdict()

        self.word2idx['OOV'] = 0
        self.idx2word['0'] = 'OOV'

        i = 1

        for k in self.vocab.keys():

            if filter_func(self.vocab[k]):

                self.word2idx[k] = i
                self.idx2word[str(i)] = k
                i += 1

        print(f"Dopo la riduzione ci sono n.{len(self.word2idx)} elementi")

    def delete(self):

        del self.vocab, self.word2idx, self.idx2word
        self.vocab = defaultdict(int)
        self.word2idx = None
        self.idx2word = None

    def save(self, nome_file, tokens = False):

        """
        Con tokens = True salva il vocabolario senza frequenze
        """

        dati = self.vocab

        if tokens:
            self.reduce()
            dati = self.word2idx

        with open(nome_file, 'w', encoding='utf-8') as f:
            json.dump(dati, f, ensure_ascii=False, indent=4)

    def load(self, nome_file, tokens = False):

        """
        Con tokens = True carica il vocabolario senza frequenze
        """

        with open(nome_file, 'r') as f:
            self.vocab = json.load(f)

            print(f"Sono stati caricati n.{len(self.vocab)} elementi")

        if self.tokens:
            self.reduce()

    def get(self):

        """
        Restituisce il vocabolario
        """

        return self.vocab

    def get_tokens(self):

        """
        Restituisce il dict con gli indici
        """

        if self.word2idx is None or self.idx2word is None:
            self.reduce()

        return self.word2idx, self.idx2word

    def get_stats(self):

        data = [v for v in self.vocab.values()]

        self.stats = {}
        self.stats['min'] = np.min(data)
        self.stats['media'] = np.mean(data)
        self.stats['max'] = np.max(data)

        return self.stats_____no_output_____

##### Esempio uso classe_____no_output_____

vocab = Vocabulary()_____no_output_____for parola in lista_esempio:
    vocab.builder(parola.replace(" ","_"))_____no_output_____vocab.vocab_____no_output_____vocab.get_stats()_____no_output_____

##### Il metodo reduce esegue una funzione che può essere assegnata direttamente dall'utente_____no_output_____Questo permette di "personalizzare" la riduzione a seconda dei casi.

Ad esempio andremo ad escludere le parole più frequenti perchè, in questo caso, potrebbero essere parole contenute in tutte e 3 le classi e pertanto potrebbero non avere potere esplicativo.
Questo è un altro metodo di riduzione dimensionale dei parametri... da usare però con il condizionale!!!_____no_output_____

vocab.reduce(lambda x,y=1,z=2: x>=y and x<=z)Dopo la riduzione ci sono n.3 elementi

##### Gli algoritmi lavorano con i numeri quindi la classe ci restituisce 2 dizionari che ci permettono di "trasformare" le parole in numeri e viceversa._____no_output_____

vocab.get_tokens()_____no_output_____vocab = Vocabulary('vocab.json', tokens = False)
vocab.import_stopwords("stopwords_ita.txt")Sono stati caricati n.81039 elementi
vocab.get_stats()_____no_output_____vocab.reduce(lambda x,y=5,z=18*100: x>=y and x<=z)Dopo la riduzione ci sono n.20426 elementi

##### Le seguenti variabili saranno usate per la costruzione del WordCloud_____no_output_____

salute = ""
tecnologia = ""
sport = ""_____no_output_____

#####  La seguente porzione di codice serve per costruire il vocaboraio e conteggiare la frequenza del numero di parole_____no_output_____Andiamo quindi ad impiegare tutte le funzioni vista in precedenza_____no_output_____

articoli = iter(glob.glob("archivio/*.txt"))_____no_output_____# Inizializzazione della classe
vocabs = Vocabulary()

# Importazione di un elenco personalizzato di stopwords
vocabs.import_stopwords("stopwords_ita.txt")

i = 0

while True:

    i += 1

#     if i == 10:
#         break

    try:

        articolo = next(articoli)
        estrati_dati(articolo)

        dati = estrati_dati(articolo)
        classe = dati[1]

        doc = nlp(dati[0])

        # Filtro dei tokens (lemmizzati) che non sono stop_words
        tokens = [token.lemma_.lower() for token in doc if not (token.lemma_ in vocabs.stopwords)]

        for token in tokens:

            # Ulteriore filtro è sulla lunghezza della parola
            if len(str(token)) > 3:

                vocabs.builder(token.replace(" ","_"))

                """
                Il seguente codice servirà per la costruzione del WordCloud

                Qui andiamo ad "accodare", in 3 distinte variabili,
                le parole che compongono i vari articoli
                """

                if classe == "salute":

                    salute += " " + str(token)

                elif classe == "sport":

                    sport += " " + str(token)

                else:

                    tecnologia += " " + str(token)

    except StopIteration:
        print("Fine")
        breakFine
from wordcloud import WordCloud, STOPWORDS, ImageColorGenerator
import matplotlib.pyplot as plt_____no_output_____wordcloud = WordCloud(max_font_size=50, max_words=25, background_color='blue', colormap='autumn')
wordcloud.generate(salute)

plt.figure()
plt.title("Salute")
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off')
plt.show()_____no_output_____wordcloud = WordCloud(max_font_size=50, max_words=25, background_color='blue', colormap='autumn')
wordcloud.generate(tecnologia)

plt.figure()
plt.title("Tecnologia")
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off')
plt.show()_____no_output_____wordcloud = WordCloud(max_font_size=50, max_words=25, background_color='blue', colormap='autumn')
wordcloud.generate(sport)

plt.figure()
plt.title("Sport")
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off')
plt.show()_____no_output_____

### Costruire WordCloud senza common words_____no_output_____

wordcloud = WordCloud(max_font_size=50, max_words=25, background_color='blue', colormap='autumn')
wordcloud.generate(salute_wc)

plt.figure()
plt.title("Salute")
plt.imshow(wordcloud, interpolation='bilinear')
plt.axis('off')
plt.show()_____no_output_____

### Scikit learn (sklearn)... una libreria indispensabile per il machine learning con Python_____no_output_____All'interno di sklearn si trovano molti "strumenti" già pronti all'uso per effettuare molte operazioni comuni nel trattamento dei dati in ambito machine learning come ad esempio:

- la standardizzazione delle variabili ---> sklearn.preprocessing.StandardScaler
- suddivisione del dataset in train e test set ---> sklearn.model_selection.train_test_split

Oltre a tanti algoritmi di ML naturalmente...:

- Regressione logistica ---> sklearn.linear_model.LogisticRegression
- Eccetera eccetera
_____no_output_____

from sklearn.feature_extraction.text import CountVectorizer_____no_output_____

##### Spiegazione_____no_output_____CountVectorizer costruisce un dizionario con la frequenza delle parole e permette, inoltre, alcune operazioni con i dati.

In particolare andremo a costruire, frase per frase, un array di zeri (0) di lunghezza pari al numero di parole presenti nel vocabolario.

Siccome le parole rappresentano l'indice dell'array, per ogni parola presente nella frase l'array avrà valore uno (1)._____no_output_____##### Esempio pratico_____no_output_____

cv_esempio = CountVectorizer()_____no_output_____testo_prova = "primo testo di prova"

testo_test = "secondo testo di prova"_____no_output_____

In questo caso la lunghezza dell'array sarà di 4_____no_output_____

cv_esempio.fit(testo_prova.split())

cv_esempio.get_feature_names()_____no_output_____cv_esempio.get_feature_names().index('testo')_____no_output_____print(testo_prova)

cv_esempio.transform(testo_prova.split()).toarray()primo testo di prova
np.max(cv_esempio.transform(testo_prova.split()).toarray(), axis=0)_____no_output_____df = pd.DataFrame(data = np.max(cv_esempio.transform(testo_prova.split(" ")).toarray(), axis=0).reshape((1,4)), \
                  columns=cv_esempio.get_feature_names())
df_____no_output_____

Cosa accade quando c'è una parola che non conosce?_____no_output_____

print(testo_test)

cv_esempio.transform(testo_test.split()).toarray()secondo testo di prova

Semplicemente la scarta!_____no_output_____

np.max(cv_esempio.transform(testo_test.split()).toarray(), axis=0)_____no_output_____df = pd.DataFrame(data = np.max(cv_esempio.transform(testo_test.split()).toarray(), axis=0).reshape((1,4)), \
                  columns=cv_esempio.get_feature_names())
df_____no_output_____vocab = Vocabulary('vocab.json', tokens = True)
vocab.import_stopwords("stopwords_ita.txt")Sono stati caricati n.81039 elementi
Dopo la riduzione ci sono n.18101 elementi

##### Applicazione_____no_output_____Con questa operazione importiamo direttamente il dizionario nel CountVectorizer senza che quest'ultimo processi prima i vari articoli._____no_output_____

cv = CountVectorizer(vocabulary = vocab.word2idx)_____no_output_____

Altro strumento molto importante che troviamo nella libreria sklearn è LabelEncoder.

Come dice il nome ci permette di codificare le etichette, ovvero trasformarle in numeri. La trasformazione di stringhe in numeri è sempre un passaggio obbligato dato che gli algoritmi di ML lavorano con i numeri non con le parole.

E' solitamente utilizzata per la codifica delle variabili risposta in problemi di classificazione (classi).

Si compone di un metodo "fit" che addestra l'oggetto e di un metodo "tranform" che le codifica in numeri._____no_output_____

from sklearn.preprocessing import LabelEncoder
labelencoder = LabelEncoder()

labelencoder.fit(['salute','tecnologia','sport'])

classes = labelencoder.transform(['salute','tecnologia','sport'])

classes_____no_output_____

Codifica in indice_____no_output_____

labelencoder.transform(['salute'])_____no_output_____

Decodifica in classe_____no_output_____

labelencoder.classes_[0]_____no_output_____

##### Mettiamo tutto insieme..._____no_output_____

def preparazione_dati(articolo):

        estrati_dati(articolo)

        dati = estrati_dati(articolo)

        y = dati[1]

        y = labelencoder.transform([y])[0]

        doc = nlp(dati[0])

        """
        Esclude le parole che sono stopwords e
        trasforma in OOV le parole non comprese nel vocabolario

        OOV = Out Of Vocabulary

        """

        tokens = [token.lemma_.lower() if token.lemma_ in vocab.word2idx.keys() else 'OOV' for token in doc if not (token.lemma_ in vocab.stopwords)]

        testo = ""

        for token in tokens:

            testo += " " + token

        X = cv.transform(testo.split()).toarray()
        X = np.max(X, axis=0).tolist()

        return X, y_____no_output_____

#### Carichiamo l'algoritmo di ML_____no_output_____

from sklearn.linear_model import SGDClassifier

classifier = SGDClassifier(fit_intercept=False)_____no_output_____

Ho scelto quest'algoritmo semplicemente perchè ha un metodo che mi permette di estrarre più facilmente l'importanza associata ad ogni parola. Anche se non è l'unico che permette di fare questo._____no_output_____Come per il LabelEncoder gli algoritmi di ML devono essere prima addestrati passando loro un dataset *(train set)* che comprende il valore della variabile risposta (in questo caso la classe) e poi testati su un dataset *(test set)* che contiene solo le varibili esplicative._____no_output_____

articoli = iter(glob.glob("archivio/*.txt"))_____no_output_____arr_X, arr_y = [], []

i = 0

# Rappresenta la dimension (numero di articoli)
# che l'algoritmo prcocessa ad ogni step
batch_size = 64

while True:

    try:

        i += 1

        # Faremo stoppare l'addestramento del modello (fitting)
        # dopo i primi 1024 articoli così da usare i rimanenti
        # come test set.
        if i > 1024:
            raise StopIteration

        articolo = next(articoli)

        # La variabile risp contiene
        # indice 0 l'array di dati
        # indice 1 l'array della classe di appartenenza
        risp = preparazione_dati(articolo)

        # Qui andiamo a costruire la matrice (X) e l'array (y)
        # che saranno processati dal classificatore
        arr_X.append(risp[0])
        arr_y.append(risp[1])

        # Il classificatore effettua i suoi calcoli solo quando
        # la dimensione dell'array X è pari al batch size
        if i % batch_size == 0:

            classifier.partial_fit(arr_X, arr_y, classes)

            # Ad ogni step di addestramento azzeriamo
            # i valori per caricarli nuovamente per il
            # nuovo batch
            arr_X, arr_y = [], []

    except StopIteration:
        print("Fine")
        break
Fine

#### Come si valuta la bontà del modello?_____no_output_____Se si trattasse di statistica inferenziale avremmo la possibilità di applicare dei test statistici che, attraverso l'esame del valore soglia, ci dicono se possiamo accettare o meno il modello... ma in questo caso ?_____no_output_____citazione: _**"TUTTI I MODELLI SONO SBAGLIATI... ALCUNI SONO UTILI"**_._____no_output_____Nei problemi di classificazione (come questo) **uno dei modi** per valutare la bontà del modello è misurare quante volte classifica correttamente gli articoli del test_set diviso il totale degli articoli che compongono il test_set. Questa misura si chiama "accuracy". Si confronta, poi, il valore ottenuto con l'accuracy del trainset._____no_output_____**Se i due valori sono elevati e, soprattutto, sono abbastanza vicino tra loro possiamo dire che l'algoritmo è stato in grado di generalizzare** ovvero di interpolare la regola che determina il fenomeno osservato._____no_output_____Ricordo che il test set potrebbe/dovrebbe contenere parole sulle quali l'algorimo non è stato addestrato._____no_output_____#### Applichiamo il classificatore al test set_____no_output_____

articolo = next(articoli)
risp = preparazione_dati(articolo)_____no_output_____# Classe di appartenenza dell'articolo
risp[1]_____no_output_____# Classe prevista dal classificatore
classifier.predict([risp[0]])[0]_____no_output_____

### Costruzione dell'accuracy per i test set_____no_output_____

n = 0

# Totale articoli processati
total = 0

# Numero articoli correttamente classificati
correct = 0

while True:

    n += 1

    try:

        total += 1

        articolo = next(articoli)

        risp = preparazione_dati(articolo)

        """
        Costruzione manuale dell'accuracy
        """

        # Se l'algoritmo classifica correttamente allora
        # incrementare la varibile correct
        if risp[1] == classifier.predict([risp[0]])[0]:

            correct += 1

        if n % 100 == 0:
            print(f"partial {int(correct/total * 100)}%")

    except StopIteration:
        # Qui stampa l'accuracy finale
        print(f"Fine: {correct/total*100:0.2f}%")
        breakpartial 80%
partial 83%
partial 84%
partial 85%
Fine: 85.71%

##### Qualche dettaglio sul classificatore_____no_output_____L'indice che identifica la classe "salute" è:_____no_output_____

labelencoder.transform(['salute'])[0]_____no_output_____

I pesi che l'algoritmo assegna a questa classe si estraggono attraverso il metodo coef_ _(Ecco perchè avevo scelto quest'algoritmo)_.

Come si può notare, la lunghezza di questi pesi corrisponde alla lunghezza delle parole del vocabolario (o meglio dei tokens)._____no_output_____

print(len(classifier.coef_[0]))
print(len(vocab.idx2word))18101
18101

Estraiamo adesso il valore massimo assunto dai pesi e poi l'indice associato._____no_output_____

# Valore max
np.max(classifier.coef_[0])_____no_output_____# Indice della parola con valore max
indice = str(np.argmax(classifier.coef_[0]))
print(indice)1559

L'indice ci serve per individuare qual'è la parola, per la classe "salute", alla quale viene attribuita l'importanza maggiore._____no_output_____

# Parola con valore max
vocab.idx2word[indice]_____no_output_____

#### Un DataFrame per meglio visualizzare..._____no_output_____

tot = {k:v for k,v in zip(vocab.word2idx.keys(), classifier.coef_[0])}_____no_output_____tot_salute = {k:v for k,v in sorted(tot.items(), key=lambda item: item[1], reverse=True)}_____no_output_____df = pd.DataFrame()
df = df.from_dict(tot_salute, orient='index')

print(f"Prime 10 parole:\n {df.head(10)}\n\n")

print(f"Ultime 10 parole:\n {df.tail(10)}")Prime 10 parole:
                      0
malattia     74.147306
marzo        69.204152
sanitario    59.317845
ospedale     59.317845
attribuire   59.317845
ottenere     54.374691
strutturare  54.374691
ricordare    54.374691
quotidiano   49.431537
riferimento  49.431537

Ultime 10 parole:
                      0
campo       -44.488384
gara        -49.431537
ormai       -49.431537
ingegneria  -54.374691
sportivo    -59.317845
salute      -59.317845
tecnologia  -69.204152
società     -69.204152
tecnologico -74.147306
tecnico     -79.090460


# Notebook from ShepherdCode/Soars2021
Path: Notebooks/ORF_MLP_103.ipynb

# ORF recognition by MLP

The MLP cannot read RNA sequences.
Here, the inputs are codon frequencies.
Measure a simple MLP on simulated RNA with and without ORF._____no_output_____

<code>
import time
t = time.time()
time.strftime('%Y-%m-%d %H:%M:%S %Z', time.localtime(t))_____no_output_____PC_SEQUENCES=20000   # how many protein-coding sequences
NC_SEQUENCES=20000   # how many non-coding sequences
PC_TESTS=1000
NC_TESTS=1000
BASES=1000            # how long is each sequence
ALPHABET=4          # how many different letters are possible
CODON_SPACE=64
INPUT_SHAPE_2D = (BASES,ALPHABET,1) # Conv2D needs 3D inputs
INPUT_SHAPE = (BASES,CODON_SPACE) # Conv1D needs 2D inputs
#FILTERS = 32   # how many different patterns the model looks for
#WIDTH = 3   # how wide each pattern is, in bases
#STRIDE_2D = (1,1)  # For Conv2D how far in each direction
#STRIDE = 1 # For Conv1D, how far between pattern matches, in bases
NEURONS = 32
#DROP_RATE = 0.01
EPOCHS=50  # how many times to train on all the data
SPLITS=5  # SPLITS=3 means train on 2/3 and validate on 1/3
FOLDS=5  # train the model this many times (range 1 to SPLITS)_____no_output_____import sys
try:
    from google.colab import drive
    IN_COLAB = True
    print("On Google CoLab, mount cloud-local file, get our code from GitHub.")
    PATH='/content/drive/'
    #drive.mount(PATH,force_remount=True)  # hardly ever need this
    #drive.mount(PATH)    # Google will require login credentials
    DATAPATH=PATH+'My Drive/data/'  # must end in "/"
    import requests
    r = requests.get('https://raw.githubusercontent.com/ShepherdCode/Soars2021/master/SimTools/RNA_gen.py')
    with open('RNA_gen.py', 'w') as f:
        f.write(r.text)
    from RNA_gen import *
    r = requests.get('https://raw.githubusercontent.com/ShepherdCode/Soars2021/master/SimTools/RNA_describe.py')
    with open('RNA_describe.py', 'w') as f:
        f.write(r.text)
    from RNA_describe import *
    r = requests.get('https://raw.githubusercontent.com/ShepherdCode/Soars2021/master/SimTools/RNA_prep.py')
    with open('RNA_prep.py', 'w') as f:
        f.write(r.text)
    from RNA_prep import *
except:
    print("CoLab not working. On my PC, use relative paths.")
    IN_COLAB = False
    DATAPATH='data/'  # must end in "/"
    sys.path.append("..") # append parent dir in order to use sibling dirs
    from SimTools.RNA_gen import *
    from SimTools.RNA_describe import *
    from SimTools.RNA_prep import *

MODELPATH="BestModel"  # saved on cloud instance and lost after logout
#MODELPATH=DATAPATH+MODELPATH  # saved on Google Drive but requires login

if not assert_imported_RNA_gen():
    print("ERROR: Cannot use RNA_gen.")
if not assert_imported_RNA_prep():
    print("ERROR: Cannot use RNA_prep.")On Google CoLab, mount cloud-local file, get our code from GitHub.
from os import listdir
import csv
from zipfile import ZipFile

import numpy as np
import pandas as pd
from scipy import stats  # mode

from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import KFold
from sklearn.model_selection import cross_val_score

from keras.models import Sequential
from keras.layers import Dense,Embedding,Dropout
from keras.layers import Conv1D,Conv2D
from keras.layers import Flatten,MaxPooling1D,MaxPooling2D
from keras.losses import BinaryCrossentropy
# tf.keras.losses.BinaryCrossentropy

import matplotlib.pyplot as plt
from matplotlib import colors
mycmap = colors.ListedColormap(['red','blue'])  # list color for label 0 then 1
np.set_printoptions(precision=2)
_____no_output_____# Use code from our SimTools library.
def make_generators(seq_len):
    pcgen = Collection_Generator()
    pcgen.get_len_oracle().set_mean(seq_len)
    pcgen.set_seq_oracle(Transcript_Oracle())
    ncgen = Collection_Generator()
    ncgen.get_len_oracle().set_mean(seq_len)
    return pcgen,ncgen

pc_sim,nc_sim = make_generators(BASES)
pc_train = pc_sim.get_sequences(PC_SEQUENCES)
nc_train = nc_sim.get_sequences(NC_SEQUENCES)
print("Train on",len(pc_train),"PC seqs")
print("Train on",len(nc_train),"NC seqs")Train on 20000 PC seqs
Train on 20000 NC seqs
# Use code from our SimTools library.
X,y = prepare_inputs_codon_frequency(pc_train,nc_train) # shuffles
print("X shape:",X.shape)X shape: (40000, 64)
def make_DNN():
    print("make_DNN")
    print("input shape:",INPUT_SHAPE)
    dnn = Sequential()
    dnn.add(Dense(NEURONS,activation="sigmoid",dtype=np.float32))
    dnn.add(Dense(NEURONS,activation="sigmoid",dtype=np.float32))
    dnn.add(Dense(NEURONS,activation="sigmoid",dtype=np.float32))
    dnn.add(Dense(NEURONS,activation="sigmoid",dtype=np.float32))
    dnn.add(Dense(1,activation="sigmoid",dtype=np.float32))
    dnn.compile(optimizer='adam',
                loss=BinaryCrossentropy(from_logits=False),
                metrics=['accuracy'])   # add to default metrics=loss
    dnn.build(input_shape=INPUT_SHAPE)
    #ln_rate = tf.keras.optimizers.Adam(learning_rate = LN_RATE)
    #bc=tf.keras.losses.BinaryCrossentropy(from_logits=False)
    #model.compile(loss=bc, optimizer=ln_rate, metrics=["accuracy"])
    return dnn
model = make_DNN()
print(model.summary())make_DNN
input shape: (1000, 64)
Model: "sequential_17"
_________________________________________________________________
Layer (type)                 Output Shape              Param #
=================================================================
dense_80 (Dense)             (1000, 32)                2080
_________________________________________________________________
dense_81 (Dense)             (1000, 32)                1056
_________________________________________________________________
dense_82 (Dense)             (1000, 32)                1056
_________________________________________________________________
dense_83 (Dense)             (1000, 32)                1056
_________________________________________________________________
dense_84 (Dense)             (1000, 1)                 33
=================================================================
Total params: 5,281
Trainable params: 5,281
Non-trainable params: 0
_________________________________________________________________
None
from keras.callbacks import ModelCheckpoint
def do_cross_validation(X,y):
    cv_scores = []
    fold=0
    mycallbacks = [ModelCheckpoint(
        filepath=MODELPATH, save_best_only=True,
        monitor='val_accuracy', mode='max')]
    splitter = KFold(n_splits=SPLITS)  # this does not shuffle
    for train_index,valid_index in splitter.split(X):
        if fold < FOLDS:
            fold += 1
            X_train=X[train_index] # inputs for training
            y_train=y[train_index] # labels for training
            X_valid=X[valid_index] # inputs for validation
            y_valid=y[valid_index] # labels for validation
            print("MODEL")
            # Call constructor on each CV. Else, continually improves the same model.
            model = model = make_DNN()
            print("FIT")  # model.fit() implements learning
            start_time=time.time()
            history=model.fit(X_train, y_train,
                    epochs=EPOCHS,
                    verbose=1,  # ascii art while learning
                    callbacks=mycallbacks,   # called at end of each epoch
                    validation_data=(X_valid,y_valid))
            end_time=time.time()
            elapsed_time=(end_time-start_time)
            print("Fold %d, %d epochs, %d sec"%(fold,EPOCHS,elapsed_time))
            # print(history.history.keys())  # all these keys will be shown in figure
            pd.DataFrame(history.history).plot(figsize=(8,5))
            plt.grid(True)
            plt.gca().set_ylim(0,1) # any losses > 1 will be off the scale
            plt.show()
_____no_output_____do_cross_validation(X,y)MODEL
make_DNN
input shape: (1000, 64)
FIT
Epoch 1/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6984 - accuracy: 0.4998 - val_loss: 0.6945 - val_accuracy: 0.4999
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 2/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6939 - accuracy: 0.5040 - val_loss: 0.6986 - val_accuracy: 0.4999
Epoch 3/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6938 - accuracy: 0.5026 - val_loss: 0.6931 - val_accuracy: 0.4999
Epoch 4/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6937 - accuracy: 0.4990 - val_loss: 0.6932 - val_accuracy: 0.4999
Epoch 5/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6941 - accuracy: 0.4998 - val_loss: 0.6930 - val_accuracy: 0.4999
Epoch 6/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6936 - accuracy: 0.4994 - val_loss: 0.6936 - val_accuracy: 0.4999
Epoch 7/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6933 - accuracy: 0.5062 - val_loss: 0.6917 - val_accuracy: 0.5001
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 8/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6882 - accuracy: 0.5477 - val_loss: 0.6687 - val_accuracy: 0.5776
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 9/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6509 - accuracy: 0.6342 - val_loss: 0.6406 - val_accuracy: 0.6265
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 10/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6183 - accuracy: 0.6561 - val_loss: 0.6217 - val_accuracy: 0.6536
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 11/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6138 - accuracy: 0.6649 - val_loss: 0.6144 - val_accuracy: 0.6612
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 12/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6111 - accuracy: 0.6609 - val_loss: 0.6193 - val_accuracy: 0.6559
Epoch 13/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6096 - accuracy: 0.6665 - val_loss: 0.6133 - val_accuracy: 0.6618
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 14/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6086 - accuracy: 0.6691 - val_loss: 0.6168 - val_accuracy: 0.6580
Epoch 15/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6034 - accuracy: 0.6734 - val_loss: 0.6131 - val_accuracy: 0.6603
Epoch 16/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6031 - accuracy: 0.6736 - val_loss: 0.6093 - val_accuracy: 0.6634
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 17/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6040 - accuracy: 0.6714 - val_loss: 0.6129 - val_accuracy: 0.6654
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 18/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6078 - accuracy: 0.6688 - val_loss: 0.6107 - val_accuracy: 0.6661
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 19/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6030 - accuracy: 0.6751 - val_loss: 0.6122 - val_accuracy: 0.6666
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 20/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6043 - accuracy: 0.6717 - val_loss: 0.6207 - val_accuracy: 0.6585
Epoch 21/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6026 - accuracy: 0.6757 - val_loss: 0.6165 - val_accuracy: 0.6594
Epoch 22/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6000 - accuracy: 0.6757 - val_loss: 0.6041 - val_accuracy: 0.6675
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 23/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6002 - accuracy: 0.6741 - val_loss: 0.6089 - val_accuracy: 0.6675
Epoch 24/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6002 - accuracy: 0.6777 - val_loss: 0.6041 - val_accuracy: 0.6685
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 25/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6012 - accuracy: 0.6736 - val_loss: 0.6047 - val_accuracy: 0.6680
Epoch 26/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6032 - accuracy: 0.6728 - val_loss: 0.6127 - val_accuracy: 0.6630
Epoch 27/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.5997 - accuracy: 0.6748 - val_loss: 0.6145 - val_accuracy: 0.6624
Epoch 28/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6056 - accuracy: 0.6692 - val_loss: 0.6032 - val_accuracy: 0.6708
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 29/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.5990 - accuracy: 0.6778 - val_loss: 0.6039 - val_accuracy: 0.6720
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 30/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6026 - accuracy: 0.6728 - val_loss: 0.6022 - val_accuracy: 0.6678
Epoch 31/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6013 - accuracy: 0.6729 - val_loss: 0.6061 - val_accuracy: 0.6712
Epoch 32/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6052 - accuracy: 0.6675 - val_loss: 0.6024 - val_accuracy: 0.6718
Epoch 33/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6015 - accuracy: 0.6712 - val_loss: 0.6103 - val_accuracy: 0.6645
Epoch 34/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.5981 - accuracy: 0.6751 - val_loss: 0.6017 - val_accuracy: 0.6685
Epoch 35/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6020 - accuracy: 0.6708 - val_loss: 0.6022 - val_accuracy: 0.6729
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 36/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6008 - accuracy: 0.6732 - val_loss: 0.6016 - val_accuracy: 0.6709
Epoch 37/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.5961 - accuracy: 0.6822 - val_loss: 0.6030 - val_accuracy: 0.6720
Epoch 38/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.5972 - accuracy: 0.6785 - val_loss: 0.6025 - val_accuracy: 0.6736
INFO:tensorflow:Assets written to: BestModel/assets
Epoch 39/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6008 - accuracy: 0.6713 - val_loss: 0.6016 - val_accuracy: 0.6711
Epoch 40/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.5971 - accuracy: 0.6773 - val_loss: 0.6034 - val_accuracy: 0.6712
Epoch 41/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6021 - accuracy: 0.6721 - val_loss: 0.6048 - val_accuracy: 0.6716
Epoch 42/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.5959 - accuracy: 0.6783 - val_loss: 0.6016 - val_accuracy: 0.6725
Epoch 43/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6000 - accuracy: 0.6736 - val_loss: 0.6018 - val_accuracy: 0.6726
Epoch 44/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.5966 - accuracy: 0.6771 - val_loss: 0.6015 - val_accuracy: 0.6725
Epoch 45/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.5920 - accuracy: 0.6809 - val_loss: 0.6013 - val_accuracy: 0.6719
Epoch 46/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.6006 - accuracy: 0.6753 - val_loss: 0.6017 - val_accuracy: 0.6731
Epoch 47/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.5966 - accuracy: 0.6774 - val_loss: 0.6067 - val_accuracy: 0.6696
Epoch 48/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.5988 - accuracy: 0.6780 - val_loss: 0.6067 - val_accuracy: 0.6697
Epoch 49/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.5960 - accuracy: 0.6774 - val_loss: 0.6011 - val_accuracy: 0.6720
Epoch 50/50
1000/1000 [==============================] - 2s 2ms/step - loss: 0.5971 - accuracy: 0.6758 - val_loss: 0.6034 - val_accuracy: 0.6735
Fold 1, 50 epochs, 112 sec
from keras.models import load_model
pc_sim.set_reproducible(True)
nc_sim.set_reproducible(True)
pc_test = pc_sim.get_sequences(PC_TESTS)
nc_test = nc_sim.get_sequences(NC_TESTS)
X,y = prepare_inputs_codon_frequency(pc_train,nc_train) # shuffles
print("X shape:",X.shape)
best_model=load_model(MODELPATH)
scores = best_model.evaluate(X, y, verbose=0)
print("The best model parameters were saved during cross-validation.")
print("Best was defined as maximum validation accuracy at end of any epoch.")
print("Now re-load the best model and test it on previously unseen data.")
print("Test on",len(pc_test),"PC seqs")
print("Test on",len(nc_test),"NC seqs")
print("%s: %.2f%%" % (best_model.metrics_names[1], scores[1]*100))
X shape: (40000, 64)
The best model parameters were saved during cross-validation.
Best was defined as maximum validation accuracy at end of any epoch.
Now re-load the best model and test it on previously unseen data.
Test on 1000 PC seqs
Test on 1000 NC seqs
accuracy: 67.82%
from sklearn.metrics import roc_curve
from sklearn.metrics import roc_auc_score
ns_probs = [0 for _ in range(len(y))]
bm_probs = best_model.predict(X)
ns_auc = roc_auc_score(y, ns_probs)
bm_auc = roc_auc_score(y, bm_probs)
ns_fpr, ns_tpr, _ = roc_curve(y, ns_probs)
bm_fpr, bm_tpr, _ = roc_curve(y, bm_probs)
plt.plot(ns_fpr, ns_tpr, linestyle='--', label='Guess, auc=%.4f'%ns_auc)
plt.plot(bm_fpr, bm_tpr, marker='.', label='Model, auc=%.4f'%bm_auc)
plt.title('ROC')
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.legend()
plt.show()
print("%s: %.2f%%" %('AUC',bm_auc*100.0))
_____no_output_____t = time.time()
time.strftime('%Y-%m-%d %H:%M:%S %Z', time.localtime(t))_____no_output_____
</code>

## Conclusion
The MLP achieved only about 65% accuracy in classifying simulated RNA with and without ORFs. Since MLPs do better on real RNA, we learn that there is more to classification than ORF recognition._____no_output_____