# Notebook from drammock/mne-tools.github.io
Path: 0.12/_downloads/plot_object_epochs.ipynb

%matplotlib inline_____no_output_____

.. _tut_epochs_objects:

The :class:`Epochs <mne.Epochs>` data structure: epoched data
=============================================================

_____no_output_____

from __future__ import print_function

import mne
import os.path as op
import numpy as np
from matplotlib import pyplot as plt_____no_output_____

:class:`Epochs <mne.Epochs>` objects are a way of representing continuous
data as a collection of time-locked trials, stored in an array of
`shape(n_events, n_channels, n_times)`. They are useful for many statistical
methods in neuroscience, and make it easy to quickly overview what occurs
during a trial.

:class:`Epochs <mne.Epochs>` objects can be created in three ways:
 1. From a :class:`Raw <mne.io.RawFIF>` object, along with event times
 2. From an :class:`Epochs <mne.Epochs>` object that has been saved as a
    `.fif` file
 3. From scratch using :class:`EpochsArray <mne.EpochsArray>`. See
    :ref:`tut_creating_data_structures`

_____no_output_____

data_path = mne.datasets.sample.data_path()
# Load a dataset that contains events
raw = mne.io.RawFIF(
    op.join(data_path, 'MEG', 'sample', 'sample_audvis_raw.fif'))

# If your raw object has a stim channel, you can construct an event array
# easily
events = mne.find_events(raw, stim_channel='STI 014')

# Show the number of events (number of rows)
print('Number of events:', len(events))

# Show all unique event codes (3rd column)
print('Unique event codes:', np.unique(events[:, 2]))

# Specify event codes of interest with descriptive labels.
# This dataset also has visual left (3) and right (4) events, but
# to save time and memory we'll just look at the auditory conditions
# for now.
event_id = {'Auditory/Left': 1, 'Auditory/Right': 2}_____no_output_____

Now, we can create an :class:`mne.Epochs` object with the events we've
extracted. Note that epochs constructed in this manner will not have their
data available until explicitly read into memory, which you can do with
:func:`get_data <mne.Epochs.get_data>`. Alternatively, you can use
`preload=True`.

Expose the raw data as epochs, cut from -0.1 s to 1.0 s relative to the event
onsets

_____no_output_____

epochs = mne.Epochs(raw, events, event_id, tmin=-0.1, tmax=1,
                    baseline=(None, 0), preload=True)
print(epochs)_____no_output_____

Epochs behave similarly to :class:`mne.io.Raw` objects. They have an
:class:`info <mne.Info>` attribute that has all of the same
information, as well as a number of attributes unique to the events contained
within the object.

_____no_output_____

print(epochs.events[:3], epochs.event_id, sep='\n\n')_____no_output_____

You can select subsets of epochs by indexing the :class:`Epochs <mne.Epochs>`
object directly. Alternatively, if you have epoch names specified in
`event_id` then you may index with strings instead.

_____no_output_____

print(epochs[1:5])
print(epochs['Auditory/Right'])_____no_output_____

It is also possible to iterate through :class:`Epochs <mne.Epochs>` objects
in this way. Note that behavior is different if you iterate on `Epochs`
directly rather than indexing:

_____no_output_____

# These will be epochs objects
for i in range(3):
    print(epochs[i])

# These will be arrays
for ep in epochs[:2]:
    print(ep)_____no_output_____

You can manually remove epochs from the Epochs object by using
:func:`epochs.drop(idx) <mne.Epochs.drop>`, or by using rejection or flat
thresholds with :func:`epochs.drop_bad(reject, flat) <mne.Epochs.drop_bad>`.
You can also inspect the reason why epochs were dropped by looking at the
list stored in ``epochs.drop_log`` or plot them with
:func:`epochs.plot_drop_log() <mne.Epochs.plot_drop_log>`. The indices
from the original set of events are stored in ``epochs.selection``.

_____no_output_____

epochs.drop([0], reason='User reason')
epochs.drop_bad(reject=dict(grad=2500e-13, mag=4e-12, eog=200e-6), flat=None)
print(epochs.drop_log)
epochs.plot_drop_log()
print('Selection from original events:\n%s' % epochs.selection)
print('Removed events (from numpy setdiff1d):\n%s'
      % (np.setdiff1d(np.arange(len(events)), epochs.selection).tolist(),))
print('Removed events (from list comprehension -- should match!):\n%s'
      % ([li for li, log in enumerate(epochs.drop_log) if len(log) > 0]))_____no_output_____

If you wish to save the epochs as a file, you can do it with
:func:`mne.Epochs.save`. To conform to MNE naming conventions, the
epochs file names should end with '-epo.fif'.

_____no_output_____

epochs_fname = op.join(data_path, 'MEG', 'sample', 'sample-epo.fif')
epochs.save(epochs_fname)_____no_output_____

Later on you can read the epochs with :func:`mne.read_epochs`. For reading
EEGLAB epochs files see :func:`mne.read_epochs_eeglab`. We can also use
``preload=False`` to save memory, loading the epochs from disk on demand.

_____no_output_____

epochs = mne.read_epochs(epochs_fname, preload=False)_____no_output_____

If you wish to look at the average across trial types, then you may do so,
creating an :class:`Evoked <mne.Evoked>` object in the process. Instances
of `Evoked` are usually created by calling :func:`mne.Epochs.average`. For
creating `Evoked` from other data structures see :class:`mne.EvokedArray` and
:ref:`tut_creating_data_structures`.

_____no_output_____

ev_left = epochs['Auditory/Left'].average()
ev_right = epochs['Auditory/Right'].average()

f, axs = plt.subplots(3, 2, figsize=(10, 5))
_ = f.suptitle('Left / Right auditory', fontsize=20)
_ = ev_left.plot(axes=axs[:, 0], show=False)
_ = ev_right.plot(axes=axs[:, 1], show=False)
plt.tight_layout()_____no_output_____

To export and manipulate Epochs using Pandas see :ref:`tut_io_export_pandas`.

_____no_output_____


# Notebook from sophiabrandt/ZTM-Complete-Machine-Learning-Data-Science
Path: notebooks/heart-disease-project/milestone-project-1.ipynb

# Predicting heart disease using machine learning

This notebook looks into using various Python-based machine learning and data science libraries in an attempt to build a machineLearning model capable of predicting whether or not someone has heart disease based on their medical attributes.

**Approach**:
1. Problem definition
2. Data
3. Evaluation
4. Features
5. Modelling
6. Experimentation

## 1. Problem Definition
In a statement,
> Given clinical parameters about a patient, can we predict whether or not they have heart disease?

## 2. Data
Dua, D. and Graff, C. (2019). UCI Machine Learning Repository [http://archive.ics.uci.edu/ml]. Irvine, CA: University of California, School of Information and Computer Science.
[Link](https://archive.ics.uci.edu/ml/datasets/Heart+Disease)

## 3. Evaluation

> If we can reach 95% accuracy at predicting whether or not a patient has heart disease during the proof of concept stage, we'll pursue the project.

## 4. Features

This is where you'll get different information about each of the features in your data.

**Create data dictionary**

* age age in years
* sex(1 = male; 0 = female)
* cp chest pain type
-- Value 1: typical angina
-- Value 2: atypical angina
-- Value 3: non-anginal pain
-- Value 4: asymptomatic
* trestbps resting blood pressure (in mm Hg on admission to the hospital)
* cholserum cholestoral in mg/dl
* fbs(fasting blood sugar > 120 mg/dl) (1 = true; 0 = false)
* restecgresting electrocardiographic results
* thalach maximum heart rate achieved
* exangexercise induced angina (1 = yes; 0 = no)
* oldpeakST depression induced by exercise relative to rest
* slopethe slope of the peak exercise ST segment
* canumber of major vessels (0-3) colored by flourosopy
* thal3 = normal; 6 = fixed defect; 7 = reversable defect
* target1 or 0_____no_output_____## Preparing the tools

Importing all needed libraries_____no_output_____

# Regular EDA (exploratory data analysis) and plotting libraries

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns; sns.set()

%matplotlib inline

# Models from Scikit-Learn
from sklearn.linear_model import LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.ensemble import RandomForestClassifier

# Model Evaluations
from sklearn.model_selection import cross_val_predict, cross_val_score, train_test_split, RandomizedSearchCV, GridSearchCV
from sklearn.metrics import confusion_matrix, classification_report, precision_score, f1_score, roc_curve, plot_roc_curve_____no_output_____

## Load Data_____no_output_____

df = pd.read_csv("../../data/heart-disease.csv")
df.shape_____no_output_____

### Data Exploration (exploratory data analysis (EDA))

Become a subject matter export on the dataset you're working with

1. What question(s) are you trying to solve?
2. Wnat kind of data do we have and how do we treat different types?
3. What's missing from the data and how do you deal with it?
4. Where areThe outliners and why should you care about them?
5. How can ou add, change or remove features to get more out of your data?_____no_output_____

df.head()_____no_output_____# Find out how many target values exist
df.target.value_counts()_____no_output_____

**Note:** This is a balanced classification problem._____no_output_____

df.target.value_counts()_____no_output_____df.target.value_counts().plot(kind='bar',
                              color=['salmon', 'lightblue'])
plt.ylabel('No.of Cases')
plt.title('Heart Disease Target')
plt.xticks([0,1], ('Positive', 'Negative'));_____no_output_____df.info()<class 'pandas.core.frame.DataFrame'>
RangeIndex: 303 entries, 0 to 302
Data columns (total 14 columns):
 #   Column    Non-Null Count  Dtype
---  ------    --------------  -----
 0   age       303 non-null    int64
 1   sex       303 non-null    int64
 2   cp        303 non-null    int64
 3   trestbps  303 non-null    int64
 4   chol      303 non-null    int64
 5   fbs       303 non-null    int64
 6   restecg   303 non-null    int64
 7   thalach   303 non-null    int64
 8   exang     303 non-null    int64
 9   oldpeak   303 non-null    float64
 10  slope     303 non-null    int64
 11  ca        303 non-null    int64
 12  thal      303 non-null    int64
 13  target    303 non-null    int64
dtypes: float64(1), int64(13)
memory usage: 33.3 KB
# Missing values?
df.isna().sum()_____no_output_____df.describe()_____no_output_____# Visualize dataset
from sklearn.decomposition import PCA

# instantiate model with 2 dimensions
pca = PCA(2)

# project from 14 to 2 dimensions
projected = pca.fit_transform(df)

# plot
plt.scatter(projected[:,0], projected[:,1],
            c=df.target, edgecolor='none', alpha=0.5,
            cmap=plt.get_cmap('coolwarm', 10))
plt.xlabel('PCA1')
plt.ylabel('PCA2')
plt.colorbar();_____no_output_____

### Heart Disease Frequency according to Gender_____no_output_____

# 1 is male, 0 is female
df.sex.value_counts()_____no_output_____# Compare target column to sex column
pd.crosstab(df.target, df.sex)_____no_output_____# Create a pot of crosstab
pd.crosstab(df.target, df.sex).plot(kind='bar',
                                   figsize=(10,6),
                                   color=['salmon', 'lightblue'])
plt.xticks([0,1], ('Negative', 'Positive'))
plt.legend(labels=['Female', 'Male'])
plt.title('Heart Disease Frequency for Sex');_____no_output_____df.thalach.value_counts()_____no_output_____

## Age vs. Max Heart Rate for Heart Disease_____no_output_____

plt.figure(figsize=(10,6))

# Scatter with positive samples
plt.scatter(df.age[df.target==1],
            df.thalach[df.target==1],
            label='Positive Samples',
            c='salmon')

# Scatter with negative samples
plt.scatter(df.age[df.target==0],
            df.thalach[df.target==0],
            label='Negative Samples',
            c='lightblue')

plt.axhline(df.thalach.mean(), linestyle='--', color='grey')
plt.title('Age vs. Max Heart Rate for Heart Disease', fontdict={'fontsize': 16})
plt.legend(labels=['Average Max Heart Rate', 'Disease', 'No Disease'])
plt.xlabel('Age')
plt.ylabel('Max Heart Rate');_____no_output_____# Check the distribution of the age column with a histogram
df.age.plot.hist();_____no_output_____

### Heart Disease Frequency per Chest Pain Type

cp: chest pain type
- Value 1: typical angina
- Value 2: atypical angina
- Value 3: non-anginal pain
- Value 4: asymptomatic_____no_output_____

pd.crosstab(df.cp, df.target)_____no_output_____# Make the crosstab more visual
pd.crosstab(df.cp, df.target).plot(kind='bar',
                                   figsize=(10,6),
                                   color=['lightblue', 'salmon'])
plt.title('Heart Disease Frequence per Chest Pain Type')
plt.xticks(np.arange(4), ('Typical Angina', 'Atypical Angina', 'Non-anginal', 'Asymptomatic'),
           rotation=0)
plt.xlabel('Chest Pain')
plt.ylabel('Patients')
plt.legend(labels=['No Disease', 'Disease']);_____no_output_____# Make a correlation matrix
df.corr()_____no_output_____# Use heatmap to visualize correlation table
corr_m = df.corr()
fig, ax = plt.subplots(figsize=(15,10))
ax = sns.heatmap(corr_m,
                 annot=True,
                 linewidth=0.5,
                 fmt='.2f',
                 cmap='YlOrRd',
                 cbar=True);_____no_output_____

## 5. Modelling_____no_output_____

df.head()_____no_output_____# Split data into X and y
X = df.drop('target', axis=1)
y = df.target
X.shape, y.shape_____no_output_____# Split data into train and test sets
np.random.seed(44)

Xtrain, Xtest, ytrain, ytest = train_test_split(X, y, test_size=0.2)_____no_output_____

We're going to try _three_ different machine learning models:

1. Logistic Regression
2. K-Nearest Neighbors
3. Random Forest_____no_output_____

# Put models in a dictionary
models = {'Logistic Regression': LogisticRegression(max_iter=1000),
          'KNN': KNeighborsClassifier(),
          'Random Forest': RandomForestClassifier()}

# Create a function to fit and score models
def fit_and_score(models, Xtrain, Xtest, ytrain, ytest):
    """
    Fits and evaluates given machine learning models.
    models: a dict of different Scikit-Learn machine learning models
    Xtrain: training data (no labels)
    Xtest: testing data (no labels)
    ytrain: training labels
    ytest: test labels
    """
    # Set random seed
    np.random.seed(44)
    # Make a dictionary to keep model scores
    model_scores = {}
    for name, model in models.items():
        # Fit the model to the data
        model.fit(Xtrain, ytrain)
        # evaluate model and append score
        model_scores[name] = model.score(Xtest, ytest)
    return model_scores_____no_output_____model_scores = fit_and_score(models=models,
                             Xtrain=Xtrain,
                             Xtest=Xtest,
                             ytrain=ytrain,
                             ytest=ytest)
model_scores_____no_output_____

## Model Comparison_____no_output_____

model_compare = pd.DataFrame(model_scores, index=['accuracy'])
model_compare.T.plot.bar()
plt.xticks(rotation=1);_____no_output_____

How to improve the baseline metrics?

| classification | regression                     |
| ---------------| -------------------------------|
| **accuracy**   | **r^2**                        |
| precision      | mean absolute error            |
| recall         | maen squared error (MSE)       |
| f1             | root mean squared error (RMSE) |

Ideas:

* hyperparameter tuning
* feature importance
* confusion matrix
* precision
* recall
* f1 score
* classification report
* roc curve
* area under the curve (AUC)_____no_output_____## Hyperparameter Tuning (by Hand)_____no_output_____

# Tune KNN
np.random.seed(44)

train_scores = []
test_scores = []

# Create a list of different values for n_neighbors
neighbors = range(1,21)

# Setup KNN instance
knn = KNeighborsClassifier()

# Loop through different n_neighbors
for i in neighbors:
    knn.set_params(n_neighbors=i)

    # Fit algorithm
    knn.fit(Xtrain, ytrain)

    # Update training scores list
    train_scores.append(knn.score(Xtrain, ytrain))

    # Update test_scores list
    test_scores.append(knn.score(Xtest, ytest))_____no_output_____train_scores_____no_output_____test_scores_____no_output_____plt.plot(neighbors, train_scores, label='Train Scores')
plt.plot(neighbors, test_scores, label='Test Scores')
plt.xticks(np.arange(1,21,1))
plt.xlabel('Number of neighbors')
plt.ylabel('Model Score')
plt.legend();

print(f"Maximum KNN score on test data: {max(test_scores)*100:.2f}%")Maximum KNN score on test data: 73.77%

## Hyperparamter Tuning with RandomizedSearchCV

We're going to tune:
* LogisticRegression
* RandomForestClassifier_____no_output_____### Tuning Logistic Regression_____no_output_____

# Create a hyperparameter grid for LogisticRegression
log_reg_grid = {'C': np.logspace(-4, 4, 20),
                'penalty': ['l1', 'l2'],
                'solver': ['liblinear']}

# Create a hyperparameter grid for RandomClassifier
rf_grid = {'n_estimators': np.arange(10, 1000, 50),
           'max_depth': [None, 3, 5, 10],
           'min_samples_split': np.arange(2, 20, 2),
           'min_samples_leaf': np.arange(1, 20, 2)}_____no_output_____# Tune Logistic Regression

np.random.seed(44)

# Setup random hyperparameter search for LogisticRegression
rs_log_reg = RandomizedSearchCV(LogisticRegression(),
                                param_distributions=log_reg_grid,
                                cv=5,
                                n_iter=20,
                                verbose=True)

# Fit random hyperparameter search model for Logistic Regression
rs_log_reg.fit(Xtrain, ytrain)Fitting 5 folds for each of 20 candidates, totalling 100 fits
rs_log_reg.best_params______no_output_____rs_log_reg.score(Xtest, ytest)_____no_output_____

### Tuning Random Forest_____no_output_____

# Tune Random Forest

np.random.seed(44)

# Setup random hyperparameter search for LogisticRegression
rs_rf_reg = RandomizedSearchCV(RandomForestClassifier(),
                               param_distributions=rf_grid,
                               cv=5,
                               n_iter=10,
                               verbose=True)

# Fit random hyperparameter search model for Logistic Regression
rs_rf_reg.fit(Xtrain, ytrain)[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
rs_rf_reg.best_params______no_output_____# Evaluate the tuned model
rs_rf_reg.score(Xtest, ytest)_____no_output_____

### Compare hypertuned models_____no_output_____

tuned_model_scores = fit_and_score(models={'RS Logistic Regression': rs_log_reg,
                                           'RS Random Forest': rs_rf_reg},
                                   Xtrain=Xtrain,
                                   Xtest=Xtest,
                                   ytrain=ytrain,
                                   ytest=ytest)
tuned_model_scoresFitting 5 folds for each of 20 candidates, totalling 100 fits
tuned_model_compare = pd.DataFrame(tuned_model_scores, index=['accuracy'])
tuned_model_compare.T.plot.bar()
plt.xticks(rotation=1);_____no_output_____

## Hyperparameter Tuning with GridSearchCV

Since our Logisti Regression model provides the best scores so far, we'll try to improve the model further using GridSearchCV_____no_output_____

# Create a hyperparameter grid for LogisticRegression with liblinear solver
log_reg_libnear_grid = {'C': np.logspace(-4, 4, 30),
                'penalty': ['l1', 'l2'],
                'solver': ['liblinear']}

# Setup grid hyperparameter search for LogisticRegression
gs_log_reg_liblinear = GridSearchCV(LogisticRegression(),
                          param_grid=log_reg_grid,
                          cv=5,
                          verbose=True)

# Fit model
gs_log_reg_liblinear.fit(Xtrain, ytrain)Fitting 5 folds for each of 40 candidates, totalling 200 fits
# Check best parameters
gs_log_reg_liblinear.best_params______no_output_____# Evaluate grid search Logistic Regression model
gs_log_reg_liblinear.score(Xtest, ytest)_____no_output_____# Create a hyperparameter grid for LogisticRegression
log_reg_libnear_grid = {'C': np.logspace(-4, 4, 30),
                'penalty': ['l1', 'l2'],
                'solver': ['liblinear']}

# Setup grid hyperparameter search for LogisticRegression
gs_log_reg_liblinear = GridSearchCV(LogisticRegression(),
                          param_grid=log_reg_grid,
                          cv=5,
                          verbose=True)

# Fit model
gs_log_reg_liblinear.fit(Xtrain, ytrain)[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
# Create a hyperparameter grid for LogisticRegression with lbfgs solver
log_reg_libnear_grid = {'C': np.logspace(-4, 4, 30),
                'penalty': [None, 'l2'],
                'solver': ['lbfgs']}

# Setup grid hyperparameter search for LogisticRegression
gs_log_reg_lbfgs = GridSearchCV(LogisticRegression(),
                          param_grid=log_reg_grid,
                          cv=5,
                          verbose=True)

# Fit model
gs_log_reg_lbfgs.fit(Xtrain, ytrain)[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
gs_log_reg_lbfgs.best_params______no_output_____gs_log_reg_lbfgs.score(Xtest, ytest)_____no_output_____rs_model_scores = fit_and_score(models={'Baseline Logistic Regression': LogisticRegression(max_iter=1000),
                                        'RS Log. Regression': rs_log_reg,
                                        'Grid Search with liblinear': gs_log_reg_liblinear,
                                        'Grid Search with lbfgs': gs_log_reg_lbfgs},
                                   Xtrain=Xtrain,
                                   Xtest=Xtest,
                                   ytrain=ytrain,
                                   ytest=ytest)
rs_model_scores[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.
rs_model_compare = pd.DataFrame(rs_model_scores, index=['accuracy'])
rs_model_compare.T.plot.bar();_____no_output_____

## Evaluating tuned machine learning classifier, beyond accuracy

* ROC curve, AUC curve
* Confusion matrix
* Classificatoin report
* Precision
* Recall
* F1-score

... with cross-validation where possible.

To make comparions and evaluate our trained model, we need to make predictions first._____no_output_____### ROC curve and AUC metric_____no_output_____

# Make predictions with best tuned model on test/train-split data-
ypreds = rs_log_reg.predict(Xtest)_____no_output_____# Plot ROC curve and calculate AUC metric
plot_roc_curve(rs_log_reg, Xtest, ytest);_____no_output_____

### Confusion Matrix_____no_output_____

# Confusion matrix
cm = confusion_matrix(ytest, ypreds)_____no_output_____# Plot confusion matrix
sns.set(font_scale=1.4)
sns.heatmap(cm.T, annot=True, fmt='d', annot_kws={"size": 16}, square=True, cbar=False)
plt.xlabel('actual')
plt.ylabel('predictions');_____no_output_____

### Classifiction Report_____no_output_____

# Classification Report on train/test-split data
print(classification_report(ytest, ypreds))              precision    recall  f1-score   support

           0       0.88      0.79      0.83        28
           1       0.83      0.91      0.87        33

    accuracy                           0.85        61
   macro avg       0.86      0.85      0.85        61
weighted avg       0.85      0.85      0.85        61

### Cross-validated evaluation metrics_____no_output_____

# Check best hyperparameters
rs_log_reg.best_params______no_output_____# Create a new classifier with best parameters
clf = LogisticRegression(C=0.08858667904100823,
                         solver='liblinear',
                         penalty='l2')_____no_output_____def average_metric(clf, X, y, scoring):
    """
    Returns the average score on each scoring metric, rounded to 2 decimals.
    clf: scikit-learn classification
    X: features (no labels)
    y: labels
    scoring: scoring metric
    """
    return round((np.mean(cross_val_score(clf, X, y, cv=5, scoring=scoring))), 2)

def average_score_on_cross_val_classification(clf, X, y):
    """
    Evaluates a given model/estimator using cross-validation (5 folds)
    and returns a dict containing the average (mean) scores
    for classification models

    clf: scikit-learn classification
    X: features (no labels)
    y: labels
    """
    metrics = ['accuracy', 'precision', 'recall', 'f1']
    scores = {metric: average_metric(clf, X, y, metric) for metric in metrics}

    return scores_____no_output_____cv_metrics = average_score_on_cross_val_classification(clf, X, y)
cv_metrics_____no_output_____# Create visualization
cv_metrics_df = pd.DataFrame(cv_metrics, index=[0])

cv_metrics_df.T.plot.bar(title='Cross-validated classification metrics',
                         legend=False);_____no_output_____

### Feature Importance

“Which features contributed most to the outcomes ofThe model and how did they contribute?”

Finding feature importance is different for each machine learning model._____no_output_____

df.head()_____no_output_____# Fit hypertuned Logistic Regression (Random Search CV)
clf.fit(Xtrain, ytrain)_____no_output_____# Find coefficients
clf.coef______no_output_____# Match coefficents to feature names
feature_dict = dict(zip(df.columns, list(clf.coef_[0])))
feature_dict_____no_output_____feature_importance = pd.DataFrame(feature_dict, index=['importance']).T
feature_importance_____no_output_____sorted_feature_importance = feature_importance.sort_values(by='importance', ascending=False)
sorted_feature_importance_____no_output_____sorted_feature_importance.plot.bar(title='Feature Importance');_____no_output_____pd.crosstab(df['cp'], df['target'])_____no_output_____pd.crosstab(df['ca'], df['target'])_____no_output_____

## Experiment With Removing Features With Low Coefficiennt_____no_output_____

df.head()_____no_output_____reduced_df = df.drop(['thal', 'chol', 'age', 'trestbps', 'fbs'], axis=1)_____no_output_____reduced_df.head()_____no_output_____# Train LogisticRegression model on reduced data set

np.random.seed(44)

X = reduced_df.drop('target', axis=1)
y = reduced_df['target']

Xtrain, Xtest, ytrain, ytest = train_test_split(X, y, test_size=0.2)_____no_output_____# Create a new classifier with best parameters from previous run
reduced = LogisticRegression(C=0.08858667904100823,
                         solver='liblinear',
                         penalty='l2')_____no_output_____reduced.fit(Xtrain, ytrain)_____no_output_____reduced.score(Xtest, ytest)_____no_output_____average_score_on_cross_val_classification(reduced, X, y)_____no_output_____# Create a hyperparameter grid for LogisticRegression
log_reg_grid_reduced = {'C': np.logspace(-4, 4, 20),
                        'penalty': ['l1', 'l2'],
                        'solver': ['liblinear']}_____no_output_____# Tune Logistic Regression

np.random.seed(44)

# Setup random hyperparameter search for LogisticRegression
rs_log_reg_reduced = RandomizedSearchCV(LogisticRegression(),
                                param_distributions=log_reg_grid_reduced,
                                cv=5,
                                n_iter=20,
                                verbose=True)

# Fit random hyperparameter search model for Logistic Regression
rs_log_reg_reduced.fit(Xtrain, ytrain)Fitting 5 folds for each of 20 candidates, totalling 100 fits
rs_log_reg_reduced.best_params______no_output_____# Create a new classifier with best parameters from previous run
reduced = LogisticRegression(C=0.23357214690901212,
                         solver='liblinear',
                         penalty='l2')_____no_output_____# Evaluate the tuned model
reduced.fit(Xtrain, ytrain)
cv_metrics_reduced = average_score_on_cross_val_classification(reduced, X, y)
cv_metrics_reduced_____no_output_____# Create visualization
compare_metrics_df = pd.DataFrame({'all features': cv_metrics,
                                   'reduced features': cv_metrics_reduced},
                                  index=cv_metrics.keys())

compare_metrics_df.plot.bar(title='Classifcation Metrics for Logistic Regression',
                            rot=0,
                            color=['lightblue', 'salmon'])_____no_output_____cv_metrics_____no_output_____cv_metrics_reduced_____no_output_____

**Result**: Slight improvment by reducing features_____no_output_____## 6. Experimentation

If you haven't hit your evaluation metric yet, ask yourself:

* Could we collect more data?
* Could we try a better model? Like CatBoostor XGBoost?
* Could we improve the curent models (beyond what we've done so far)?_____no_output_____