# Notebook from silviaclaire/recommendations-with-IBM
Path: Recommendations_with_IBM.ipynb

# Recommendations with IBM

In this notebook, you will be putting your recommendation skills to use on real data from the IBM Watson Studio platform.

You may either submit your notebook through the workspace here, or you may work from your local machine and submit through the next page.  Either way assure that your code passes the project [RUBRIC](https://review.udacity.com/#!/rubrics/2322/view).  **Please save regularly.**

By following the table of contents, you will build out a number of different methods for making recommendations that can be used for different situations.

## Table of Contents

I. [Exploratory Data Analysis](#Exploratory-Data-Analysis)

II. [Rank Based Recommendations](#Rank)

III. [User-User Based Collaborative Filtering](#User-User)

IV. [Content Based Recommendations (EXTRA - NOT REQUIRED)](#Content-Recs)

V. [Matrix Factorization](#Matrix-Fact)

VI. [Extras & Concluding](#conclusions)

At the end of the notebook, you will find directions for how to submit your work.  Let's get started by importing the necessary libraries and reading in the data._____no_output_____

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import project_tests as t
import pickle

%matplotlib inline

df = pd.read_csv('data/user-item-interactions.csv')
df_content = pd.read_csv('data/articles_community.csv')
del df['Unnamed: 0']
del df_content['Unnamed: 0']

# Show df to get an idea of the data
df.head()_____no_output_____# Show df_content to get an idea of the data
df_content.head()_____no_output_____

### Part I : Exploratory Data Analysis

Use the dictionary and cells below to provide some insight into the descriptive statistics of the data.

`1.` What is the distribution of how many articles a user interacts with in the dataset?  Provide a visual and descriptive statistics to assist with giving a look at the number of times each user interacts with an article._____no_output_____

n_views_by_user = df.groupby('email').count()['article_id']
n_views_by_user_____no_output_____n_views_by_user.hist(bins=30)
plt.xlabel("number of article viewed by a user")
plt.ylabel("number of users")
plt.title("Distribution of the Number of Articles Viewed by a User");_____no_output_____# Fill in the median and maximum number of user_article interactios below
median_val = n_views_by_user.median()
max_views_by_user = n_views_by_user.max()

print(f'50% of individuals interact with {median_val} articles or fewer.')
print(f'The maximum number of user-article interactions by any 1 user is {max_views_by_user}.')50% of individuals interact with 3.0 articles or fewer.
The maximum number of user-article interactions by any 1 user is 364.

`2.` Explore and remove duplicate articles from the **df_content** dataframe._____no_output_____

# Find and explore duplicate articles
df_content[df_content.duplicated(subset=['article_id'], keep=False)].sort_values(by=['article_id'])_____no_output_____# Remove any rows that have the same article_id - only keep the first
df_content = df_content.drop_duplicates(subset=['article_id'], keep='first')
assert len(df_content) == len(df_content['article_id'].unique())_____no_output_____

`3.` Use the cells below to find:

**a.** The number of unique articles that have an interaction with a user.
**b.** The number of unique articles in the dataset (whether they have any interactions or not).

**c.** The number of unique users in the dataset. (excluding null values)

**d.** The number of user-article interactions in the dataset._____no_output_____

unique_articles = len(df['article_id'].unique())
total_articles = len(df_content)
unique_users = len(df['email'].dropna().unique())
user_article_interactions = len(df)

print(f'The number of unique articles that have at least one interaction: {unique_articles}')
print(f'The number of unique articles on the IBM platform: {total_articles}')
print(f'The number of unique users: {unique_users}')
print(f'The number of user-article interactions: {user_article_interactions}')The number of unique articles that have at least one interaction: 714
The number of unique articles on the IBM platform: 1051
The number of unique users: 5148
The number of user-article interactions: 45993

`4.` Use the cells below to find the most viewed **article_id**, as well as how often it was viewed.  After talking to the company leaders, the `email_mapper` function was deemed a reasonable way to map users to ids.  There were a small number of null values, and it was found that all of these null values likely belonged to a single user (which is how they are stored using the function below)._____no_output_____

most_viewed_article = df.groupby('article_id').count().sort_values('email', ascending=False).iloc[0]
most_viewed_article_____no_output_____most_viewed_article_id = str(most_viewed_article.name)
max_views = most_viewed_article.email

print(f'The most viewed article in the dataset: {most_viewed_article_id}')
print(f'The most viewed article in the dataset was viewed {max_views} times')The most viewed article in the dataset: 1429.0
The most viewed article in the dataset was viewed 937 times
## No need to change the code here - this will be helpful for later parts of the notebook
# Run this cell to map the user email to a user_id column and remove the email column

def email_mapper():
    coded_dict = dict()
    cter = 1
    email_encoded = []

    for val in df['email']:
        if val not in coded_dict:
            coded_dict[val] = cter
            cter+=1

        email_encoded.append(coded_dict[val])
    return email_encoded

email_encoded = email_mapper()
del df['email']
df['user_id'] = email_encoded

# show header
df.head()_____no_output_____## If you stored all your results in the variable names above,
## you shouldn't need to change anything in this cell

sol_1_dict = {
    '`50% of individuals have _____ or fewer interactions.`': median_val,
    '`The total number of user-article interactions in the dataset is ______.`': user_article_interactions,
    '`The maximum number of user-article interactions by any 1 user is ______.`': max_views_by_user,
    '`The most viewed article in the dataset was viewed _____ times.`': max_views,
    '`The article_id of the most viewed article is ______.`': most_viewed_article_id,
    '`The number of unique articles that have at least 1 rating ______.`': unique_articles,
    '`The number of unique users in the dataset is ______`': unique_users,
    '`The number of unique articles on the IBM platform`': total_articles
}

# Test your dictionary against the solution
t.sol_1_test(sol_1_dict)It looks like you have everything right here! Nice job!

### Part II: Rank-Based Recommendations

Unlike in the earlier lessons, we don't actually have ratings for whether a user liked an article or not.  We only know that a user has interacted with an article.  In these cases, the popularity of an article can really only be based on how often an article was interacted with.

`1.` Fill in the function below to return the **n** top articles ordered with most interactions as the top. Test your function using the tests below._____no_output_____

def get_top_articles(n, df=df):
    '''
    INPUT:
    n - (int) the number of top articles to return
    df - (pandas dataframe) df as defined at the top of the notebook

    OUTPUT:
    top_articles - (list) A list of the top 'n' article titles

    '''
    # get the top n article ids
    top_articles_id = get_top_article_ids(n, df)

    # get the top n article titles
    top_articles = df[df['article_id'].isin(top_articles_id)]['title'].unique().tolist()

    return top_articles # Return the top article titles from df (not df_content)

def get_top_article_ids(n, df=df):
    '''
    INPUT:
    n - (int) the number of top articles to return
    df - (pandas dataframe) df as defined at the top of the notebook

    OUTPUT:
    top_articles_id - (list) A list of the top 'n' article ids

    '''
    # sort articles by times of interactions
    sorted_articles = df.groupby('article_id').count().sort_values('user_id', ascending=False)

    # get the top n article ids
    top_articles_id = [str(e) for e in sorted_articles.iloc[0: n].index.tolist()]

    return top_articles_id # Return the top article ids_____no_output_____print(get_top_articles(10))
print(get_top_article_ids(10))['healthcare python streaming application demo', 'use deep learning for image classification', 'apache spark lab, part 1: basic concepts', 'predicting churn with the spss random tree algorithm', 'analyze energy consumption in buildings', 'visualize car data with brunel', 'use xgboost, scikit-learn & ibm watson machine learning apis', 'gosales transactions for logistic regression model', 'insights from new york car accident reports', 'finding optimal locations of new store using decision optimization']
['1429.0', '1330.0', '1431.0', '1427.0', '1364.0', '1314.0', '1293.0', '1170.0', '1162.0', '1304.0']
# Test your function by returning the top 5, 10, and 20 articles
top_5 = get_top_articles(5)
top_10 = get_top_articles(10)
top_20 = get_top_articles(20)

# Test each of your three lists from above
t.sol_2_test(get_top_articles)Your top_5 looks like the solution list! Nice job.
Your top_10 looks like the solution list! Nice job.
Your top_20 looks like the solution list! Nice job.

### Part III: User-User Based Collaborative Filtering

`1.` Use the function below to reformat the **df** dataframe to be shaped with users as the rows and articles as the columns.

* Each **user** should only appear in each **row** once.

* Each **article** should only show up in one **column**.

* **If a user has interacted with an article, then place a 1 where the user-row meets for that article-column**.  It does not matter how many times a user has interacted with the article, all entries where a user has interacted with an article should be a 1.

* **If a user has not interacted with an item, then place a zero where the user-row meets for that article-column**.

Use the tests to make sure the basic structure of your matrix matches what is expected by the solution._____no_output_____

# create the user-article matrix with 1's and 0's

def create_user_item_matrix(df):
    '''
    INPUT:
    df - pandas dataframe with article_id, title, user_id columns

    OUTPUT:
    user_item - user item matrix

    Description:
    Return a matrix with user ids as rows and article ids on the columns with 1 values where a user interacted with
    an article and a 0 otherwise
    '''
    # group and unstack to create matrix
    user_item = df.groupby(['user_id','article_id']).any().unstack(level=1)

    # fill nan with 0
    user_item = user_item.fillna(0)

    # transform True to 1
    user_item = user_item.astype(int)

    # remove unused level
    user_item.columns = user_item.columns.droplevel(0).astype(str)

    return user_item # return the user_item matrix

user_item = create_user_item_matrix(df)_____no_output_____## Tests: You should just need to run this cell.  Don't change the code.
assert user_item.shape[0] == 5149, "Oops!  The number of users in the user-article matrix doesn't look right."
assert user_item.shape[1] == 714, "Oops!  The number of articles in the user-article matrix doesn't look right."
assert user_item.sum(axis=1)[1] == 36, "Oops!  The number of articles seen by user 1 doesn't look right."
print("You have passed our quick tests!  Please proceed!")You have passed our quick tests!  Please proceed!

`2.` Complete the function below which should take a user_id and provide an ordered list of the most similar users to that user (from most similar to least similar).  The returned result should not contain the provided user_id, as we know that each user is similar to him/herself. Because the results for each user here are binary, it (perhaps) makes sense to compute similarity as the dot product of two users.

Use the tests to test your function._____no_output_____

def calc_similarity(a, b):
    '''
    INPUT:
    a, b - (pandas Series) a pair of vectors

    OUTPUT:
    sim - (int) similarity for given users

    Description:
    Return the similarity of given pair of vectors
    '''
    # manually compute dot similarity
    sim = np.dot(a, b)

    return sim_____no_output_____def find_similar_users(user_id, user_item=user_item):
    '''
    INPUT:
    user_id - (int) a user_id
    user_item - (pandas dataframe) matrix of users by articles:
                1's when a user has interacted with an article, 0 otherwise

    OUTPUT:
    similar_users - (list) an ordered list where the closest users (largest dot product users)
                    are listed first

    Description:
    Computes the similarity of every pair of users based on the dot product
    Returns an ordered list of similar users' ids

    '''
    # compute similarity of each user to the provided user
    similarities = pd.Series(index=user_item.index)
    for a_user_id, a_user_vec in user_item.iterrows():
        similarities[a_user_id] = calc_similarity(user_item.loc[user_id], a_user_vec)

    # sort by similarity
    similarities = similarities.sort_values(ascending=False)

    # create list of just the ids (remove the own user's id)
    most_similar_users = similarities.drop(user_id).index.tolist()

    return most_similar_users # return a list of the users in order from most to least similar_____no_output_____# Do a spot check of your function
print("The 10 most similar users to user 1 are: {}".format(find_similar_users(1)[:10]))
print("The 5 most similar users to user 3933 are: {}".format(find_similar_users(3933)[:5]))
print("The 3 most similar users to user 46 are: {}".format(find_similar_users(46)[:3]))The 10 most similar users to user 1 are: [3933, 23, 3782, 203, 4459, 131, 3870, 46, 4201, 5041]
The 5 most similar users to user 3933 are: [1, 23, 3782, 4459, 203]
The 3 most similar users to user 46 are: [4201, 23, 3782]

`3.` Now that you have a function that provides the most similar users to each user, you will want to use these users to find articles you can recommend.  Complete the functions below to return the articles you would recommend to each user. _____no_output_____

def get_article_names(article_ids, df=df):
    '''
    INPUT:
    article_ids - (list) a list of article ids
    df - (pandas dataframe) df as defined at the top of the notebook

    OUTPUT:
    article_names - (list) a list of article names associated with the list of article ids
                    (this is identified by the title column)
    '''
    article_ids = [str(e) for e in article_ids]
    df_dropped_dupl_articles = df.drop_duplicates(subset=['article_id'])[['article_id', 'title']]
    article_names = [row['title'] for _, row in df_dropped_dupl_articles.iterrows() if str(row['article_id']) in article_ids]

    return article_names # Return the article names associated with list of article ids

def get_user_articles(user_id, user_item=user_item):
    '''
    INPUT:
    user_id - (int) a user id
    user_item - (pandas dataframe) matrix of users by articles:
                1's when a user has interacted with an article, 0 otherwise

    OUTPUT:
    article_ids - (list) a list of the article ids seen by the user
    article_names - (list) a list of article names associated with the list of article ids
                    (this is identified by the doc_full_name column in df_content)

    Description:
    Provides a list of the article_ids and article titles that have been seen by a user
    '''
    article_ids = [col_name for col_name, val in user_item.loc[user_id].iteritems() if val == 1]
    article_names = get_article_names(article_ids)

    return article_ids, article_names # return the ids and names

def user_user_recs(user_id, m=10):
    '''
    INPUT:
    user_id - (int) a user id
    m - (int) the number of recommendations you want for the user

    OUTPUT:
    recs - (list) a list of recommendations for the user

    Description:
    Loops through the users based on closeness to the input user_id
    For each user - finds articles the user hasn't seen before and provides them as recs
    Does this until m recommendations are found

    Notes:
    Users who are the same closeness are chosen arbitrarily as the 'next' user

    For the user where the number of recommended articles starts below m
    and ends exceeding m, the last items are chosen arbitrarily

    '''
    # get most similar users for the given user_id
    most_similar_users = find_similar_users(user_id)

    # get articles the user has seen before
    article_ids_already_read, _ = get_user_articles(user_id)

    # initialize a empty set of recommendations for the user
    recs = set()

    for user in most_similar_users:
        # find articles in similar users that the user hasn't seen before
        article_ids, _ = get_user_articles(user)
        rec = [id_ for id_ in article_ids if id_ not in article_ids_already_read]
        recs.update(rec)
        if len(recs) >= m:
            break

    recs = list(recs)[:m]

    return recs # return your recommendations for this user_id_____no_output_____# Check Results
get_article_names(user_user_recs(1, 10)) # Return 10 recommendations for user 1_____no_output_____# Test your functions here - No need to change this code - just run this cell
assert set(get_article_names(['1024.0', '1176.0', '1305.0', '1314.0', '1422.0', '1427.0'])) == set(['using deep learning to reconstruct high-resolution audio', 'build a python app on the streaming analytics service', 'gosales transactions for naive bayes model', 'healthcare python streaming application demo', 'use r dataframes & ibm watson natural language understanding', 'use xgboost, scikit-learn & ibm watson machine learning apis']), "Oops! Your the get_article_names function doesn't work quite how we expect."
assert set(get_article_names(['1320.0', '232.0', '844.0'])) == set(['housing (2015): united states demographic measures','self-service data preparation with ibm data refinery','use the cloudant-spark connector in python notebook']), "Oops! Your the get_article_names function doesn't work quite how we expect."
assert set(get_user_articles(20)[0]) == set(['1320.0', '232.0', '844.0'])
assert set(get_user_articles(20)[1]) == set(['housing (2015): united states demographic measures', 'self-service data preparation with ibm data refinery','use the cloudant-spark connector in python notebook'])
assert set(get_user_articles(2)[0]) == set(['1024.0', '1176.0', '1305.0', '1314.0', '1422.0', '1427.0'])
assert set(get_user_articles(2)[1]) == set(['using deep learning to reconstruct high-resolution audio', 'build a python app on the streaming analytics service', 'gosales transactions for naive bayes model', 'healthcare python streaming application demo', 'use r dataframes & ibm watson natural language understanding', 'use xgboost, scikit-learn & ibm watson machine learning apis'])
print("If this is all you see, you passed all of our tests!  Nice job!")If this is all you see, you passed all of our tests!  Nice job!

`4.` Now we are going to improve the consistency of the **user_user_recs** function from above.

* Instead of arbitrarily choosing when we obtain users who are all the same closeness to a given user - choose the users that have the most total article interactions before choosing those with fewer article interactions.

* Instead of arbitrarily choosing articles from the user where the number of recommended articles starts below m and ends exceeding m, choose articles with the articles with the most total interactions before choosing those with fewer total interactions. This ranking should be  what would be obtained from the **top_articles** function you wrote earlier._____no_output_____

def get_top_sorted_users(user_id, df=df, user_item=user_item):
    '''
    INPUT:
    user_id - (int)
    df - (pandas dataframe) df as defined at the top of the notebook
    user_item - (pandas dataframe) matrix of users by articles:
            1's when a user has interacted with an article, 0 otherwise

    OUTPUT:
    neighbors_df - (pandas dataframe) a dataframe with:
                    neighbor_id - a neighbor user_id
                    similarity - measure of the similarity of each user to the provided user_id
                    num_interactions - the number of articles viewed by the user

    Other Details - sort the neighbors_df by the similarity and then by number of interactions where
                    highest of each is higher in the dataframe

    '''
    # compute similarity of each user to the provided user from user_item
    similarities = pd.Series(index=user_item.index)
    for a_user_id, a_user_vec in user_item.iterrows():
        similarities[a_user_id] = calc_similarity(user_item.loc[user_id], a_user_vec)

    # calculate the number of interactions of each user from df
    interactions = df.groupby('user_id').count()['article_id']

    # concat similarities and interactions
    neighbors_df = pd.concat([similarities, interactions], axis=1)
    neighbors_df.columns = ['similarity', 'num_interactions']

    # remove the own user's id
    neighbors_df = neighbors_df.drop(user_id)

    # sort by similarity then num_interactions
    neighbors_df = neighbors_df.sort_values(['similarity', 'num_interactions'], ascending=False)

    return neighbors_df # Return the dataframe specified in the doc_string

def get_top_sorted_user_articles(user_id, df=df, user_item=user_item):
    '''
    INPUT:
    user_id - (int) a user id
    df - (pandas dataframe) df as defined at the top of the notebook
    user_item - (pandas dataframe) matrix of users by articles:
                1's when a user has interacted with an article, 0 otherwise

    OUTPUT:
    user_articles_df - (pandas dataframe) a dataframe of the articles that have been seen by a user
                        with:
                        article_id - an article's id
                        title - title of the article
                        total_interactions - total interactions of the article

    Other Details - sort the user_articles_df by number of total interactions where
                    highest of each is higher in the dataframe
    '''
    # get a list of the article ids seen by the user
    article_ids = [col_name for col_name, val in user_item.loc[user_id].iteritems() if val == 1]

    # get a list of article names associated with the list of article ids
    article_names = pd.Series(zip(article_ids, get_article_names(article_ids)), index=article_ids)

    # calculate the number of interactions of each user from df
    interactions = df[df['article_id'].isin(article_ids)].groupby('article_id').count()['title']

    # concat article_names and interactions
    user_articles_df = pd.concat([article_names, interactions], axis=1)
    user_articles_df.columns = ['title', 'total_interactions']
    user_articles_df.index = user_articles_df.index.astype(str)

    # sort by total_interactions
    user_articles_df = user_articles_df.sort_values('total_interactions', ascending=False)

    return user_articles_df

def user_user_recs_part2(user_id, m=10):
    '''
    INPUT:
    user_id - (int) a user id
    m - (int) the number of recommendations you want for the user

    OUTPUT:
    recs - (list) a list of recommendations for the user by article id
    rec_names - (list) a list of recommendations for the user by article title

    Description:
    Loops through the users based on closeness to the input user_id
    For each user - finds articles the user hasn't seen before and provides them as recs
    Does this until m recommendations are found

    Notes:
    * Choose the users that have the most total article interactions
    before choosing those with fewer article interactions.

    * Choose articles with the articles with the most total interactions
    before choosing those with fewer total interactions.

    '''
    # get most similar users for the given user_id
    most_similar_users = get_top_sorted_users(user_id).index.tolist()

    # get articles the user has seen before
    article_ids_already_read, _ = get_user_articles(user_id)

    # initialize a empty set of recommendations for the user
    recs = set()

    for user in most_similar_users:
        # find articles in similar users that the user hasn't seen before
        article_ids = get_top_sorted_user_articles(user).index.tolist()
        rec = [id_ for id_ in article_ids if id_ not in article_ids_already_read]
        recs.update(rec)
        if len(recs) >= m:
            break

    recs = list(recs)[:m]
    rec_names = get_article_names(recs)

    return recs, rec_names_____no_output_____# Quick spot check - don't change this code - just use it to test your functions
rec_ids, rec_names = user_user_recs_part2(20, 10)
print("The top 10 recommendations for user 20 are the following article ids:")
print(rec_ids)
print()
print("The top 10 recommendations for user 20 are the following article names:")
print(rec_names)The top 10 recommendations for user 20 are the following article ids:
['730.0', '205.0', '362.0', '1409.0', '1324.0', '142.0', '1420.0', '1356.0', '1296.0', '1170.0']

The top 10 recommendations for user 20 are the following article names:
['apache spark lab, part 1: basic concepts', 'dsx: hybrid mode', 'ibm watson facebook posts for 2015', 'use apache systemml and spark for machine learning', 'developing for the ibm streaming analytics service', 'fortune 100 companies', "a beginner's guide to variational methods", 'occupation (2015): united states demographic measures', 'neural networks for beginners: popular types and applications', 'uci: red wine quality']

`5.` Use your functions from above to correctly fill in the solutions to the dictionary below.  Then test your dictionary against the solution.  Provide the code you need to answer each following the comments below._____no_output_____

### Tests with a dictionary of results

user1_most_sim = get_top_sorted_users(1).index.tolist()[0]
user131_10th_sim = get_top_sorted_users(131).index.tolist()[9]
print(f'The user that is most similar to user 1 : {user1_most_sim}')
print(f'The user that is the 10th most similar to user 131: {user131_10th_sim}')The user that is most similar to user 1 : 3933
The user that is the 10th most similar to user 131: 242
## Dictionary Test Here
sol_5_dict = {
    'The user that is most similar to user 1.': user1_most_sim,
    'The user that is the 10th most similar to user 131': user131_10th_sim,
}

t.sol_5_test(sol_5_dict)This all looks good!  Nice job!

`6.` If we were given a new user, which of the above functions would you be able to use to make recommendations?  Explain.  Can you think of a better way we might make recommendations?  Use the cell below to explain a better method for new users._____no_output_____The function I would use to make recommendation for new users: `get_top_articles`

**Reason:** Since new users has no ratings, any existing user would have a 0 cosine similarity with him/her. Instead of finding a similar user with user-user based collaborative filtering method, it makes more sense to use a rank-based recommendation method.

A better way to make recommendations: provide somd lists of the highest ranked articles associated with some terms.

Examples:
- top articles for ensemble learning
- top articles for hierarchical clustering_____no_output_____`7.` Using your existing functions, provide the top 10 recommended articles you would provide for the a new user below.  You can test your function against our thoughts to make sure we are all on the same page with how we might make a recommendation._____no_output_____

new_user = '0.0'

# What would your recommendations be for this new user '0.0'?  As a new user, they have no observed articles.
# Provide a list of the top 10 article ids you would give to
new_user_recs = get_top_article_ids(10)_____no_output_____assert set(new_user_recs) == set(['1314.0','1429.0','1293.0','1427.0','1162.0','1364.0','1304.0','1170.0','1431.0','1330.0']), "Oops!  It makes sense that in this case we would want to recommend the most popular articles, because we don't know anything about these users."

print("That's right!  Nice job!")That's right!  Nice job!

### Part IV: Content Based Recommendations (EXTRA - NOT REQUIRED)

Another method we might use to make recommendations is to perform a ranking of the highest ranked articles associated with some term.  You might consider content to be the **doc_body**, **doc_description**, or **doc_full_name**.  There isn't one way to create a content based recommendation, especially considering that each of these columns hold content related information.

`1.` Use the function body below to create a content based recommender.  Since there isn't one right answer for this recommendation tactic, no test functions are provided.  Feel free to change the function inputs if you decide you want to try a method that requires more input values.  The input values are currently set with one idea in mind that you may use to make content based recommendations.  One additional idea is that you might want to choose the most popular recommendations that meet your 'content criteria', but again, there is a lot of flexibility in how you might make these recommendations.

### This part is NOT REQUIRED to pass this project.  However, you may choose to take this on as an extra way to show off your skills._____no_output_____

def make_content_recs():
    '''
    INPUT:

    OUTPUT:

    '''_____no_output_____

`2.` Now that you have put together your content-based recommendation system, use the cell below to write a summary explaining how your content based recommender works.  Do you see any possible improvements that could be made to your function?  Is there anything novel about your content based recommender?

### This part is NOT REQUIRED to pass this project.  However, you may choose to take this on as an extra way to show off your skills._____no_output_____**Write an explanation of your content based recommendation system here.**_____no_output_____`3.` Use your content-recommendation system to make recommendations for the below scenarios based on the comments.  Again no tests are provided here, because there isn't one right answer that could be used to find these content based recommendations.

### This part is NOT REQUIRED to pass this project.  However, you may choose to take this on as an extra way to show off your skills._____no_output_____

# make recommendations for a brand new user

# make a recommendations for a user who only has interacted with article id '1427.0'

_____no_output_____

### Part V: Matrix Factorization

In this part of the notebook, you will build use matrix factorization to make article recommendations to the users on the IBM Watson Studio platform.

`1.` You should have already created a **user_item** matrix above in **question 1** of **Part III** above.  This first question here will just require that you run the cells to get things set up for the rest of **Part V** of the notebook. _____no_output_____

# Load the matrix here
user_item_matrix = pd.read_pickle('user_item_matrix.p')_____no_output_____# quick look at the matrix
user_item_matrix.head()_____no_output_____

`2.` In this situation, you can use Singular Value Decomposition from [numpy](https://docs.scipy.org/doc/numpy-1.14.0/reference/generated/numpy.linalg.svd.html) on the user-item matrix.  Use the cell to perform SVD, and explain why this is different than in the lesson._____no_output_____

# Perform SVD on the User-Item Matrix Here
u, s, vt = np.linalg.svd(user_item_matrix)
u.shape, vt.shape, s.shape_____no_output_____

**Answer:** SVD works because this time there is no missing values in user-item matrix._____no_output_____`3.` Now for the tricky part, how do we choose the number of latent features to use?  Running the below cell, you can see that as the number of latent features increases, we obtain a lower error rate on making predictions for the 1 and 0 values in the user-item matrix.  Run the cell below to get an idea of how the accuracy improves as we increase the number of latent features._____no_output_____

num_latent_feats = np.arange(10,700+10,20)
sum_errs = []

for k in num_latent_feats:
    # restructure with k latent features
    s_new, u_new, vt_new = np.diag(s[:k]), u[:, :k], vt[:k, :]

    # take dot product
    user_item_est = np.around(np.dot(np.dot(u_new, s_new), vt_new))

    # compute error for each prediction to actual value
    diffs = np.subtract(user_item_matrix, user_item_est)

    # total errors and keep track of them
    err = np.sum(np.sum(np.abs(diffs)))
    sum_errs.append(err)

plt.plot(num_latent_feats, 1 - np.array(sum_errs)/df.shape[0]);
plt.xlabel('Number of Latent Features');
plt.ylabel('Accuracy');
plt.title('Accuracy vs. Number of Latent Features');_____no_output_____

`4.` From the above, we can't really be sure how many features to use, because simply having a better way to predict the 1's and 0's of the matrix doesn't exactly give us an indication of if we are able to make good recommendations.  Instead, we might split our dataset into a training and test set of data, as shown in the cell below.

Use the code from question 3 to understand the impact on accuracy of the training and test sets of data with different numbers of latent features. Using the split below:

* How many users can we make predictions for in the test set?
* How many users are we not able to make predictions for because of the cold start problem?
* How many articles can we make predictions for in the test set?
* How many articles are we not able to make predictions for because of the cold start problem?_____no_output_____

df_train = df.head(40000)
df_test = df.tail(5993)

def create_test_and_train_user_item(df_train, df_test):
    '''
    INPUT:
    df_train - training dataframe
    df_test - test dataframe

    OUTPUT:
    user_item_train - a user-item matrix of the training dataframe
                      (unique users for each row and unique articles for each column)
    user_item_test - a user-item matrix of the testing dataframe
                    (unique users for each row and unique articles for each column)
    test_idx - all of the test user ids
    test_arts - all of the test article ids

    '''
    user_item_train = create_user_item_matrix(df_train)
    user_item_test = create_user_item_matrix(df_test)
    test_idx = user_item_test.index.tolist()
    test_arts = user_item_test.columns.tolist()

    return user_item_train, user_item_test, test_idx, test_arts

user_item_train, user_item_test, test_idx, test_arts = create_test_and_train_user_item(df_train, df_test)_____no_output_____print(user_item_train.shape)
print(user_item_test.shape)

# users we can make predictions for
valid_idx = sorted(list(set(user_item_train.index) & set(test_idx)))
print(len(valid_idx))

# articles we can make predictions for
valid_arts = sorted(list(set(user_item_train.columns) & set(test_arts)))
print(len(valid_arts))(4487, 714)
(682, 574)
20
574
# Replace the values in the dictionary below
a = 662
b = 574
c = 20
d = 0

sol_4_dict = {
    'How many users can we make predictions for in the test set?': c,
    'How many users in the test set are we not able to make predictions for because of the cold start problem?': a,
    'How many articles can we make predictions for in the test set?': b,
    'How many articles in the test set are we not able to make predictions for because of the cold start problem?': d
}

t.sol_4_test(sol_4_dict)Awesome job!  That's right!  All of the test movies are in the training data, but there are only 20 test users that were also in the training set.  All of the other users that are in the test set we have no data on.  Therefore, we cannot make predictions for these users using SVD.

`5.` Now use the **user_item_train** dataset from above to find U, S, and V transpose using SVD. Then find the subset of rows in the **user_item_test** dataset that you can predict using this matrix decomposition with different numbers of latent features to see how many features makes sense to keep based on the accuracy on the test data. This will require combining what was done in questions `2` - `4`.

Use the cells below to explore how well SVD works towards making predictions for recommendations on the test data.  _____no_output_____

# fit SVD on the user_item_train matrix
u_train, s_train, vt_train = np.linalg.svd(user_item_train)
u_train.shape, vt_train.shape, s_train.shape_____no_output_____# Use these cells to see how well you can use the training
# decomposition to predict on test data

def predict_user_item(k, user_item_test=user_item_test, valid_users=valid_idx, valid_items=valid_arts):
    """
    INPUT:
    k - the number of latent features
    user_item_test - a user-item matrix of the testing dataframe
                    (unique users for each row and unique articles for each column)

    OUTPUT:
    user_item_est - a user-item matrix estimated from k latent features
    train_err - total error between user_item_est and user_item_train
    val_err - total error between user_item_est and user_item_test
    """
    # restructure with k latent features
    s_new, u_new, vt_new = np.diag(s_train[:k]), u_train[:, :k], vt_train[:k, :]

    # take dot product
    user_item_est = pd.DataFrame(np.around(np.dot(np.dot(u_new, s_new), vt_new)))
    user_item_est.index = user_item_train.index
    user_item_est.columns = user_item_train.columns

    # compute error for each prediction to actual value
    train_diffs = np.subtract(user_item_train.loc[valid_idx, valid_arts], user_item_est.loc[valid_idx, valid_arts])
    val_diffs = np.subtract(user_item_test.loc[valid_idx, valid_arts], user_item_est.loc[valid_idx, valid_arts])

    # total errors and keep track of them
    train_err = np.sum(np.sum(np.abs(train_diffs)))
    val_err = np.sum(np.sum(np.abs(val_diffs)))

    return user_item_est, train_err, val_err_____no_output_____#num_latent_feats = np.arange(1, 700+10, 50)
num_latent_feats = np.arange(1, 30)
train_errs = []
val_errs = []

for k in num_latent_feats:
    _, train_err, val_err = predict_user_item(k)
    train_errs.append(train_err)
    val_errs.append(val_err)
    print(f'k:{k} \t train_err:{train_err} \t val_err:{val_err}')

plt.plot(num_latent_feats, 1 - np.array(train_errs)/user_item_train.shape[0])
plt.plot(num_latent_feats, 1 - np.array(val_errs)/user_item_train.shape[0])
plt.xlabel('Number of Latent Features');
plt.ylabel('Accuracy');
plt.title('Accuracy vs. Number of Latent Features');k:1 	 train_err:243.0 	 val_err:221.0
k:2 	 train_err:237.0 	 val_err:225.0
k:3 	 train_err:237.0 	 val_err:227.0
k:4 	 train_err:237.0 	 val_err:229.0
k:5 	 train_err:228.0 	 val_err:232.0
k:6 	 train_err:225.0 	 val_err:239.0
k:7 	 train_err:214.0 	 val_err:240.0
k:8 	 train_err:214.0 	 val_err:238.0
k:9 	 train_err:209.0 	 val_err:249.0
k:10 	 train_err:206.0 	 val_err:248.0
k:11 	 train_err:205.0 	 val_err:249.0
k:12 	 train_err:201.0 	 val_err:249.0
k:13 	 train_err:195.0 	 val_err:253.0
k:14 	 train_err:192.0 	 val_err:256.0
k:15 	 train_err:194.0 	 val_err:252.0
k:16 	 train_err:191.0 	 val_err:257.0
k:17 	 train_err:187.0 	 val_err:259.0
k:18 	 train_err:187.0 	 val_err:259.0
k:19 	 train_err:187.0 	 val_err:257.0
k:20 	 train_err:184.0 	 val_err:260.0
k:21 	 train_err:185.0 	 val_err:261.0
k:22 	 train_err:183.0 	 val_err:261.0
k:23 	 train_err:173.0 	 val_err:269.0
k:24 	 train_err:173.0 	 val_err:269.0
k:25 	 train_err:173.0 	 val_err:269.0
k:26 	 train_err:173.0 	 val_err:269.0
k:27 	 train_err:164.0 	 val_err:272.0
k:28 	 train_err:162.0 	 val_err:270.0
k:29 	 train_err:160.0 	 val_err:270.0

`6.` Use the cell below to comment on the results you found in the previous question. Given the circumstances of your results, discuss what you might do to determine if the recommendations you make with any of the above recommendation systems are an improvement to how users currently find articles? _____no_output_____**Answer:**

Based on the accuracy on the test data, it should be enough to keep just 4 latent features.

To evaluate how well the recommendation engine is working in practice:

1. Predict a list of recommended articles for a given user who has viewed a few existing articles.
2. Compare to the actual list of articles the user has viewed.
3. If the percentage of common articles is high for every given user, the recommendation engine is working well._____no_output_____

def validation_comparison(df_valid, k):
    '''
    INPUT:
    df_valid - validation dataframe
    k - the number of latent features

    OUTPUT:
    err - totol error of how far off each value is from it's predicted value
    accs - prediction accuracy (percent of matched articles)
    preds - a list of predicted articles for each user
    acts - a list of actually viewed articles for each user
    matches - a list of matched articles for each user
    valid_users - users we can make predictions for
    '''
    # create user-item matrix of the validation dataframe
    user_item_valid = create_user_item_matrix(df_valid)

    # users we can make predictions for
    valid_users = sorted(list(set(user_item_train.index) & set(user_item_valid.index)))

    # articles we can make predictions for
    valid_items = sorted(list(set(user_item_train.columns) & set(user_item_valid.columns)))

    if len(valid_users)==0 or len(valid_items)==0:
        print('no common users/items between training set and validation set.')
        return

    user_item_est, _, err = predict_user_item(k, user_item_valid, valid_users, valid_items)

    preds, acts, matches, accs = [], [], [], []
    for user_id in valid_users:
        _, pred = get_user_articles(user_id, user_item_est)
        _, act = get_user_articles(user_id, user_item_train)

        match = [item for item in pred if item in act]
        acc = len(match)/len(act)

        preds.append(pred)
        acts.append(act)

        matches.append(match)
        accs.append(acc)

    return err, matches, accs, preds, acts, valid_users_____no_output_____k = 4
err, matches, accs, preds, acts, valid_users = validation_comparison(df_test, k)_____no_output_____print(f'err:{err}')
for idx, user_id in enumerate(valid_users):
    if len(preds[idx]) > 0:
        print(f'user:{user_id} \t accuracy:{accs[idx]}')
        print(f'pred:\n{preds[idx]}')
        print(f'act:\n{acts[idx]}')
        print(f'matches:\n{matches[idx]}')err:229.0
user:3024 	 accuracy:0.19230769230769232
pred:
['use deep learning for image classification', 'apache spark lab, part 1: basic concepts', 'predicting churn with the spss random tree algorithm', 'jupyter notebook tutorial', 'python machine learning: scikit-learn tutorial', 'deep learning with tensorflow course by big data university']
act:
['use deep learning for image classification', 'apache spark lab, part 1: basic concepts', 'visualize car data with brunel', 'apache spark lab, part 3: machine learning', 'welcome to pixiedust', 'insights from new york car accident reports', 'jupyter notebook tutorial', 'python machine learning: scikit-learn tutorial', 'deep learning with tensorflow course by big data university', 'apache spark lab, part 2: querying data', 'using brunel in ipython/jupyter notebooks', 'deep learning trends and an example', 'working interactively with rstudio and notebooks in dsx', '1357    what i learned implementing a classifier from ...\nName: title, dtype: object', 'learn basics about notebooks and apache spark', 'access ibm analytics for apache spark from rstudio', 'interactive web apps with shiny cheat sheet', 'sector correlations shiny app', 'blogging with brunel', 'machine learning exercises in python, part 1', 'practical tutorial on random forest and parameter tuning in r', 'spark-based machine learning tools for capturing word meanings', 'the machine learning database', 'machine learning for the enterprise.', 'flightpredict ii: the sequel  – ibm watson data lab', 'using machine learning to predict parking difficulty']
matches:
['use deep learning for image classification', 'apache spark lab, part 1: basic concepts', 'jupyter notebook tutorial', 'python machine learning: scikit-learn tutorial', 'deep learning with tensorflow course by big data university']
user:3684 	 accuracy:0.05714285714285714
pred:
['healthcare python streaming application demo', 'use xgboost, scikit-learn & ibm watson machine learning apis', 'insights from new york car accident reports']
act:
['healthcare python streaming application demo', 'use deep learning for image classification', 'ml optimization using cognitive assistant', 'classify tumors with machine learning', 'apache spark lab, part 1: basic concepts', 'predicting churn with the spss random tree algorithm', 'use xgboost, scikit-learn & ibm watson machine learning apis', 'putting a human face on machine learning', 'gosales transactions for naive bayes model', 'uci ml repository: chronic kidney disease data set', 'apache spark as the new engine of genomics', 'analyze accident reports on amazon emr spark', 'quick guide to build a recommendation engine in python', 'analyzing data by using the sparkling.data library features', 'breast cancer wisconsin (diagnostic) data set', 'use r dataframes & ibm watson natural language understanding', 'deep learning trends and an example', 'deep learning from scratch i: computational graphs', 'working interactively with rstudio and notebooks in dsx', 'data model with streaming analytics and python', 'movie recommender system with spark machine learning', 'deep learning with data science experience', 'learn basics about notebooks and apache spark', 'data science in the cloud', 'improving real-time object detection with yolo', 'real-time sentiment analysis of twitter hashtags with spark (+ pixiedust)', 'access postgresql with python', 'optimization for deep learning highlights in 2017', 'the difference between ai, machine learning, and deep learning?', 'laplace noising versus simulated out of sample methods (cross frames)', 'perform sentiment analysis with lstms, using tensorflow', 'visualize the 1854 london cholera outbreak', 'process events from the watson iot platform in a streams python application', 'generalization in deep learning', 'develop a scala spark model on chicago building violations']
matches:
['healthcare python streaming application demo', 'use xgboost, scikit-learn & ibm watson machine learning apis']
user:3968 	 accuracy:0.05555555555555555
pred:
['insights from new york car accident reports']
act:
['healthcare python streaming application demo', 'the nurse assignment problem', 'data science for real-time streaming analytics', 'intents & examples for ibm watson conversation', 'access mysql with r', 'insights from new york car accident reports', 'apache spark as the new engine of genomics', 'developing for the ibm streaming analytics service', 'quick guide to build a recommendation engine in python', 'breast cancer wisconsin (diagnostic) data set', 'analyze traffic data from the city of san francisco', 'analyzing streaming data from kafka topics', 'data model with streaming analytics and python', 'variational auto-encoder for "frey faces" using keras', 'build a python app on the streaming analytics service', 'apache spark™ 2.0: extend structured streaming for spark ml', 'ml algorithm != learning machine', 'automating web analytics through python']
matches:
['insights from new york car accident reports']
user:4293 	 accuracy:0.15384615384615385
pred:
['healthcare python streaming application demo', 'predicting churn with the spss random tree algorithm', 'gosales transactions for logistic regression model', 'insights from new york car accident reports', 'total population by country', 'customer demographics and sales']
act:
['healthcare python streaming application demo', 'use deep learning for image classification', 'visualize data with the matplotlib library', 'classify tumors with machine learning', 'ibm watson facebook posts for 2015', 'visualize car data with brunel', 'analyze open data sets with pandas dataframes', 'intents & examples for ibm watson conversation', 'welcome to pixiedust', 'insights from new york car accident reports', 'analyze open data sets with spark & pixiedust', 'breast cancer wisconsin (diagnostic) data set', 'total population by country', 'data model with streaming analytics and python', 'movie recommender system with spark machine learning', 'programmatic evaluation using watson conversation', 'annual % population growth by country', 'customer demographics and sales', 'united states demographic measures: zip code tabulation areas (zctas)', 'united states demographic measures: population and age', 'employed population by occupation and age', 'access postgresql with r', 'how ibm builds an effective data science team', 'deep learning achievements over the past year ', 'aspiring data scientists! start to learn statistics with these 6 books!', 'introducing ibm watson studio ']
matches:
['healthcare python streaming application demo', 'insights from new york car accident reports', 'total population by country', 'customer demographics and sales']

### Extras
Using your workbook, you could now save your recommendations for each user, develop a class to make new predictions and update your results, and make a flask app to deploy your results.  These tasks are beyond what is required for this project.  However, from what you learned in the lessons, you certainly capable of taking these tasks on to improve upon your work here!

## Conclusion

> Congratulations!  You have reached the end of the Recommendations with IBM project!

> **Tip**: Once you are satisfied with your work here, check over your report to make sure that it is satisfies all the areas of the [rubric](https://review.udacity.com/#!/rubrics/2322/view). You should also probably remove all of the "Tips" like this one so that the presentation is as polished as possible.

## Directions to Submit

> Before you submit your project, you need to create a .html or .pdf version of this notebook in the workspace here. To do that, run the code cell below. If it worked correctly, you should get a return code of 0, and you should see the generated .html file in the workspace directory (click on the orange Jupyter icon in the upper left).

> Alternatively, you can download this report as .html via the **File** > **Download as** submenu, and then manually upload it into the workspace directory by clicking on the orange Jupyter icon in the upper left, then using the Upload button.

> Once you've done this, you can submit your project by clicking on the "Submit Project" button in the lower right here. This will create and submit a zip file with this .ipynb doc and the .html or .pdf version you created. Congratulations! _____no_output_____

from subprocess import call
call(['python', '-m', 'nbconvert', 'Recommendations_with_IBM.ipynb'])_____no_output_____


# Notebook from lmcguire/JupyterAlluvialChannels
Path: AlluvialChannels.ipynb

# Modeling Alluvial Channel Evolution

### Overview
In class, we discussed an equation that can be used to determine the unit sediment flux ($q_s$) in an alluvial channel. The equation is given by

$$q_s=\kappa(\tau-\tau_c)^{3/2}$$

where $\tau=\rho g h S$, $\tau_c$ is the critical shear stress required for sediment transport, and $\kappa$ is an empirical coeficient. This is a powerful relationship, but it does not directly tell us anything about how elevation will change with time at a particular location on the landscape. The quantify changes in elevation with time, we need to compute the derivative of the sediment flux in the along-channel direction. Fundamentally, our reason for doing this is that changes in elevation are related to spatial variations in sediment flux (change in elevation = flux in - flux out). Therefore, the model that we use to determine changes in elevation ($z$) with time ($t$) can be written as

$$ \frac{\partial z}{\partial t}=\kappa\frac{\partial q_s}{\partial x}$$

Below, we use this equation to simulate how an alluvial channel will respond to a rapid change in base level.

To visualize the scenario that we will consider, examine the figure above. It shows a high-order channel in blue as well as a low order tributary (red). Imagine that the high-order channel rapidly incises by 5 m over a short period of time. The direct result of this rapid incising is that its tributary (red) will experience a sudden 5 m drop in base level. **Run the code block below, which is set up to plot the longitudinal profile of the tributary channel immediately before and immediately after the drop in base level.**

_____no_output_____

time=0.01                          # Time [kyr]

# !! YOU DO NOT NEED TO MODIFY THE CODE BELOW THIS LINE !!
kappa=20                          # Erodibility coefficient
deltaz=5                          # Base level drop [m]
uplift=0.02                       # Uplift rate [m/kyr]

import numpy as np
from alluvialchannelevolution import alluvialchannel
[distance,area,elevationi,slope]=alluvialchannel(time,uplift,kappa,kappa,deltaz)

itopo=np.loadtxt('steadystatealluvial_U0pt02.txt')

import matplotlib.pyplot as plt
plt.rc('font', size=18)          # controls default text sizes
fig = plt.figure(figsize=(15, 8))
# Adds subplot on position 1
ax1 = fig.add_subplot(121)
# Adds subplot on position 2
ax2 = fig.add_subplot(122)
fig.tight_layout(pad=4.0)
ax1.plot(distance,elevationi,'k-',label="Modeled Topography")
ax1.plot(distance,itopo,'r--',label="Initial Topography")
ax1.legend()
ax1.set(xlabel='Distance (m)', ylabel='Elevation (m)', title='t = '+str(round(time,2)))
ax2.plot(distance,elevationi,'k-',label="Modeled Topography")
ax2.plot(distance,itopo,'r--',label="Initial Topography")
ax2.set_xlim([1500, 2000])
ax2.set_ylim([-5, 10])
ax2.legend()
ax2.set(xlabel='Distance (m)', ylabel='Elevation (m)', title='Close-up view at t = '+str(round(time,2)))
plt.show()_____no_output_____

The figures that are generated by the model show the 5 m drop in elevation where the tributary meets the main channel. How will the elevation in the tributary change in response to this drop in base level? **In the code block below, change the simulation time from 0.01 kyr to 1 kyr to see a model prediction for channel profile after 1 kyr of time has passed.**_____no_output_____

time=1                            # Time [kyr]

# !! YOU DO NOT NEED TO MODIFY THE CODE BELOW THIS LINE !!
kappa=20                          # Erodibility coefficient
deltaz=5                          # Base level drop [m]
uplift=0.02                       # Uplift rate [m/kyr]

import numpy as np
from alluvialchannelevolution import alluvialchannel
[distance,area,elevation1,slope]=alluvialchannel(time,uplift,kappa,kappa,deltaz)

itopo=np.loadtxt('steadystatealluvial_U0pt02.txt')

import matplotlib.pyplot as plt
plt.figure(figsize=(15, 9))
plt.rc('font', size=18)          # controls default text sizes
plt.plot(distance,elevationi,'k-',label="t=0.01 kyr")
plt.plot(distance,elevation1,'c-',label="t=1 kyr")
plt.plot(distance,itopo,'r--',label="Initial Topography")
plt.xlabel('Distance (m)', fontsize=18)
plt.ylabel('Elevation (m)', fontsize=18)
plt.title('t = '+str(round(time,1)), fontsize=18)
plt.legend()
plt.rc('legend', fontsize=18)    # legend fontsize
plt.show()_____no_output_____

**In the code block below, change the simulation time from 0.01 kyr to 10 kyr to see a model prediction for channel profile after 10 kyr of time has passed.**_____no_output_____

time=10                          # Time [kyr]

# !! YOU DO NOT NEED TO MODIFY THE CODE BELOW THIS LINE !!
kappa=20                          # Erodibility coefficient
deltaz=5                          # Base level drop [m]
uplift=0.02                       # Uplift rate [m/kyr]

import numpy as np
from alluvialchannelevolution import alluvialchannel
[distance,area,elevation10,slope]=alluvialchannel(time,uplift,kappa,kappa,deltaz)

itopo=np.loadtxt('steadystatealluvial_U0pt02.txt')

import matplotlib.pyplot as plt
plt.figure(figsize=(15, 9))
plt.rc('font', size=18)          # controls default text sizes
plt.plot(distance,elevationi,'k-',label="t=0.01 kyr")
plt.plot(distance,elevation1,'c-',label="t=1 kyr")
plt.plot(distance,elevation10,'g-',label="t=10 kyr")
plt.plot(distance,itopo,'r--',label="Initial Topography")
plt.xlabel('Distance (m)', fontsize=18)
plt.ylabel('Elevation (m)', fontsize=18)
plt.title('t = '+str(round(time,1)), fontsize=18)
plt.legend()
plt.rc('legend', fontsize=18)    # legend fontsize
plt.show()_____no_output_____

**In the code block below, change the simulation time from 0.01 kyr to 1000 kyr to see a model prediction for channel profile after 1000 kyr of time has passed.**_____no_output_____

time=1000                          # Time [kyr]

# !! YOU DO NOT NEED TO MODIFY THE CODE BELOW THIS LINE !!
kappa=20                          # Erodibility coefficient
deltaz=5                          # Base level drop [m]
uplift=0.02                       # Uplift rate [m/kyr]

import numpy as np
from alluvialchannelevolution import alluvialchannel
[distance,area,elevation1000,slope]=alluvialchannel(time,uplift,kappa,kappa,deltaz)

itopo=np.loadtxt('steadystatealluvial_U0pt02.txt')

import matplotlib.pyplot as plt
plt.figure(figsize=(15, 9))
plt.rc('font', size=18)          # controls default text sizes
plt.plot(distance,elevationi,'k-',label="t=0.01 kyr")
plt.plot(distance,elevation1,'c-',label="t=1 kyr")
plt.plot(distance,elevation10,'g-',label="t=10 kyr")
plt.plot(distance,elevation1000,'r-',label="t=1000 kyr")
plt.plot(distance,itopo,'r--',label="Initial Topography")
plt.xlabel('Distance (m)', fontsize=18)
plt.ylabel('Elevation (m)', fontsize=18)
plt.title('t = '+str(round(time,1)), fontsize=18)
plt.legend()
plt.rc('legend', fontsize=18)    # legend fontsize
plt.show()_____no_output_____

After 1000 kyr of time has passed following the change in base level, the channel has adjusted everywhere to be approximately 5 m lower in elevation. In other words, the overall shape of **the channel profile looks exactly the same as it did before the drop in base level (it is just shifted down by 5 m)**. It has reached a new "steady state" and will continue to maintain this same shape unless some other change in base level, climate, or uplift occurs._____no_output_____


# Notebook from Jun-NIBS/bmtk
Path: docs/tutorial/01_single_cell_clamped.ipynb

# Chapter 1: Single cell simulation with current injection (with BioNet)

In this example we will build a network consisting of a single biophysically detailed cell. Then we will run a short simulation with a current injection at the soma of the cell, then look at the output of spikes, membrane potential and calcium flux.

**Note** - scripts and files for running this tutorial can be found in the directory [sources/chapter01/](sources/chapter01)

**requirements:**
* Python 2.7, 3.6+
* bmtk
* NEURON 7.4+_____no_output_____## 1. Building the network

First step is to use the bmtk Network Builder to create and save the network. First we instantiate a network with a name or our choosing (since throughout this tutorial we will use cell models that from the mouse cortex, let's call our network 'mcortex').

Once we have a network, we can add a single node by calling the add_nodes method._____no_output_____

from bmtk.builder.networks import NetworkBuilder

net = NetworkBuilder('mcortex')
net.add_nodes(cell_name='Scnn1a_473845048',
              potental='exc',
              model_type='biophysical',
              model_template='ctdb:Biophys1.hoc',
              model_processing='aibs_perisomatic',
              dynamics_params='472363762_fit.json',
              morphology='Scnn1a_473845048_m.swc')_____no_output_____

Some of the parameters used to create the node are optional and only for our benefit. Others are necessary for when we will eventually run a simulation:
* *cell_name* (optional) - Name/type of cell we will be modeling.
* *potential* (optional) - Use to indicate that it is an excitatory type cell.
* *model_type* - Used by the simulator to indicate that we are using a biophysical cell.
* *dynamics_params* - Model parameters. File will be downloaded from the Allen Cell Types Database.
* *morphology* - Model morphology. File will be downloaded from the Allen Cell Types Database.
* *model_processing* - A custom function used by the simulator to load the model into NEURON using Allen Cell-Types files for perisomatic models

#### Building and saving
The final thing to do is to build and save the network. If successful, we should see a combination of hdf5 and csv files in the './network' directory, these files are used describe the network, and can be saved, stored and run at a later date.

First, it's a good idea to remove any old files in the "network" folder so they don't interfere with the current simulation._____no_output_____

%%bash
rm network/*_____no_output_____

If you get the output:

rm: cannot remove 'network/*': No such file or directory

It's OK. Keep going._____no_output_____

net.build()
net.save_nodes(output_dir='network')_____no_output_____

Use the NetworkBuilder nodes() method to show that a node of our parameters was created_____no_output_____

from pprint import pprint
for node in net.nodes():
    pprint(node){'node_type_id': 100, 'dynamics_params': '472363762_fit.json', 'potental': 'exc', 'morphology': 'Scnn1a_473845048_m.swc', 'node_id': 0, 'model_template': 'ctdb:Biophys1.hoc', 'model_type': 'biophysical', 'model_processing': 'aibs_perisomatic', 'cell_name': 'Scnn1a_473845048'}

## 2. Setting up the simulator enviornment

Now that the network has been built, we can use the BioNet simulator to setup and run it using NEURON. We will need to set-up the paths to the configuration and parameters files. We can do this by either:
1. Copying the directory tree from LOCATION into the current directory. Then compiling the extra NEURON mechanisms used by the Allen Cell Models by running the command
```bash
$ cd components/mechanism
$ nrnivmodl modfiles
```
2. We can build the enviornment by scratch, which we can do from a command-line
```bash
$ python -m bmtk.utils.sim_setup -n network --membrane_report-vars v,cai --membrane_report-sections soma --tstop 2000.0 --dt 0.1 bionet
```
3. Call the function directly in python_____no_output_____

from bmtk.utils.sim_setup import build_env_bionet
build_env_bionet(network_dir='network', tstop=2000.0, dt=0.1,
                 reports={'membrane_report': {
                         "module": "membrane_report",
                         'cells': 'all',
                         'variable_name': ['cai', 'v'],
                         "file_name": "cell_vars.h5",
                         "sections": "soma",
                     }})/home/ben/bmtk/docs/tutorial/biophys_components/mechanisms

This creates standard SONATA configuration files for BioNet, using the built network files in ./network and having a simulation run-time of 2 seconds. The membrane_report-vars and membrane_report-sections sets it so that we are recording membrane potential (v) and calcium concentration (cai) at the soma.:_____no_output_____#### Set the model files
The cell will be modeled from a V1, layer 4 Scnn1a mouse cell. Download the model file from http://celltypes.brain-map.org/neuronal_model/download/482934212, or by searching the Allen Cell-Types Database. We will need to unzip the downloaded package and move the parameters and morphology files into our simulator enviornment:
* Rename *fit_parameters.json* to *472363762_fit.json* and place in ./components/biophysical/electrophysiology/
* Rename *reconstruction.swc* to *Scnn1a_473845048_m.swc* and place in ./components/biophysical/morphologies/ _____no_output_____

%%bash
cp sources/bionet_files/components/biophysical/electrophysiology/472363762_fit.json biophys_components/biophysical_neuron_templates/
cp sources/bionet_files/components/biophysical/morphology/Scnn1a_473845048_m.swc biophys_components/morphologies/_____no_output_____

#### Checking the simulation_config.json

The file ./simulation_config.json is used to set network, run-time and output parameters during a simulation. In the prefered bmtk workflow the config file is used to update and adjust parameters on different simulations runs, eliminating the need for complicated programs and allows modelers and experimentalist to run their models with little-to-no programming required.

More about the configuration file will be explained throughout the tutorials. However current what is missing is any input stimulus. Without it the simulation will run, but not do anything interesting. To add a current injection into the soma of our cell, we must open simulation_config.json with a text editor and modify the **input** section to:
```json
"inputs": {
 	"current_clamp": {
      "input_type": "current_clamp",
      "module": "IClamp",
      "node_set": "all",
      "amp": 0.120,
      "delay": 500.0,
      "duration": 1000.0
    }
},
```

This will inject a current (amp) of .12 mV with a delay of 500 ms and a duration of 1000 ms (i.e from times 500 - 1500 ms).
_____no_output_____## 3. Running the simulation

Once our config file is setup we can run a simulation either through the command line:
```bash
$ python run_bionet.py simulation_config.json
```

or through the script_____no_output_____

from bmtk.simulator import bionet

conf = bionet.Config.from_json('simulation_config.json')
conf.build_env()
net = bionet.BioNetwork.from_config(conf)
sim = bionet.BioSimulator.from_config(conf, network=net)
sim.run()2019-02-22 13:15:18,782 [INFO] Created log file
Mechanisms already loaded from path: ./biophys_components/mechanisms.  Aborting.
2019-02-22 13:15:18,796 [INFO] Building cells.

A quick breakdown of the script:
```python
conf = config.from_json('config.json')
io.setup_output_dir(conf)
nrn.load_neuron_modules(conf)
```
This section loads the configuration file, it setups the output directory and files for writing during the simulation, and loads NEURON mechanisms needed by the cell model(s) during the simulation.

```python
net = bionet.BioNetwork.from_config(conf)
```
Creates a NEURON representation of the network, including cell models that have been converted into their NEURON equivelents.

```python
sim = Simulation.from_config(conf, network=net)
sim.run()
```
Sets up and runs the NEURON simulation. When finished the output - spike times, membrane potential and Calcium influx, will be saved into the output directory as specified in the config.

_____no_output_____## 4. Analyzing the run

The results of the simulation are placed into various files as specified in the "output" section of the config file. We can change this before run-time if required.

All simulations will save the spike times of the network cells. These are saved in csv format (output/spikes.txt) or hdf5 format(output/spikes.h5). To get the table of spike times for our single-cell network we can run the following method from the analyzer (gid 0 corresponds to our single cell)._____no_output_____

from bmtk.analyzer.spike_trains import to_dataframe
to_dataframe(config_file='simulation_config.json')_____no_output_____

When setting up the enviornment and config file we specified cell_vars=['v', 'cai']. This indicates to the simulator to also record membrane potential and calcium diffusion (and we can also specify other variables as long as they are supported in NEURON). The recordings are stored in hdf5 format in the file output/cellvars/0.h5 (0 for the gid of our first and only cell)._____no_output_____

from bmtk.analyzer.cell_vars import plot_report

plot_report(config_file='simulation_config.json')_____no_output_____

## 5. Additional Information

### Changing run-time parameters.
By making changes to the config file, we can change the conditions and simulation parameters without having to rebuild the network, modify parameter files, or change our run_bionet script. In fact we can iteratively run multiple simulations without any extra coding, only a text editor to change the json file.

The run section of the config.json contains most of the parameters unique to the simulation:
```json
"run": {
    "dL": 20,
    "nsteps_block": 5000,
    "spike_threshold": -15,
    "tstop": 2000.0,
    "dt": 0.1
}
```
* tstop - simulation runtime in milliseconds.
* dt - the time steps of the simulation. decreasing dt should increase accuracy of firing dynamics, but also increase time it takes to complete.
* spike_threshold - used to determine when to count a action potential
* dL - length of segments in a section. the number of segments will be forced to an odd number so the actual length may vary.

Through the conditions section we can adjust simulation temperature (C) and the initial membrane potential of the cells:
```json
"conditions": {
    "celsius": 34.0,
    "v_init": -80
}
```

And lastly, the input section lets us control stimulus onto the network. There are a number of different options which will be explained in the following tutorials. But even with a simple current injection we can adjust amplitude, delay and stimulation duration and measure the effect on the cell.
```json
"inputs": {
 	"current_clamp": {
      "input_type": "current_clamp",
      "module": "IClamp",
      "node_set": "all",
      "amp": 0.120,
      "delay": 500.0,
      "duration": 1000.0
    }
}
```
We can even add multiple injections
```json
"inputs": {
 	"cclamp1": {
      "input_type": "current_clamp",
      "module": "IClamp",
      "node_set": "all",
      "amp": 0.150,
      "delay": 0.0,
      "duration": 400.0
    }

    "cclamp2": {
      "input_type": "current_clamp",
      "module": "IClamp",
      "node_set": "all",
      "amp": 0.300,
      "delay": 500.0,
      "duration": 400.0
    }

    "cclamp3": {
      "input_type": "current_clamp",
      "module": "IClamp",
      "node_set": "all",
      "amp": 0.450,
      "delay": 1000.0,
      "duration": 400.0
    }
}
```_____no_output_____### Changing cell models
When building the network we defined the cell model and morphology through the 'dynamics_params' and 'morphology_file' options. After building and saving the network, these values were saved in the node-types csv file._____no_output_____

import pandas as pd
pd.read_csv('network/mcortex_node_types.csv', sep=' ')_____no_output_____

If we want to run the simulation on a different cell model, all we have to do is
1. Download new parameters.json and morphology.swc into components/biophysical
2. Open mcortex_node_types.csv in a text editor and update 'morphology_file' and 'params_file' accordingly.

In our simple one-cell example, it is likely faster to just rebuild the network. However the advantage of the use of the node types becomes clear once we start dealing with a larger network. For example we may have a network of hundreds of thousands of individual cells with tens of thousands of Scnn1a type cells. The process of adjusting/changing the Scnn1a parameter in the csv then starting another simulation only takes seconds, whereas rebuilding the entire network may take hours._____no_output_____


# Notebook from flyjump/PythonDataMining
Path: BOOK_CODE/Chapter 4/ch4 Affinity Analysis.ipynb

<code>
# data from http://grouplens.org/datasets/movielens/_____no_output_____import os
data_folder = os.path.join(os.path.expanduser("~"), "Data", "ml-100k")
ratings_filename = os.path.join(data_folder, "u.data")_____no_output_____import pandas as pd_____no_output_____all_ratings = pd.read_csv(ratings_filename, delimiter="\t", header=None, names = ["UserID", "MovieID", "Rating", "Datetime"])
all_ratings["Datetime"] = pd.to_datetime(all_ratings['Datetime'],unit='s')
all_ratings[:5]_____no_output_____# As you can see, there are no review for most movies, such as #213
all_ratings[all_ratings["UserID"] == 675].sort("MovieID")  _____no_output_____# Not all reviews are favourable! Our goal is "other recommended books", so we only want favourable reviews
all_ratings["Favorable"] = all_ratings["Rating"] > 3
all_ratings[10:15]_____no_output_____all_ratings[all_ratings["UserID"] == 1][:5]_____no_output_____# Sample the dataset. You can try increasing the size of the sample, but the run time will be considerably longer
ratings = all_ratings[all_ratings['UserID'].isin(range(200))]  # & ratings["UserID"].isin(range(100))]_____no_output_____# We start by creating a dataset of each user's favourable reviews
favorable_ratings = ratings[ratings["Favorable"]]
favorable_ratings[:5]_____no_output_____# We are only interested in the reviewers who have more than one review
favorable_reviews_by_users = dict((k, frozenset(v.values)) for k, v in favorable_ratings.groupby("UserID")["MovieID"])
len(favorable_reviews_by_users)_____no_output_____# Find out how many movies have favourable ratings
num_favorable_by_movie = ratings[["MovieID", "Favorable"]].groupby("MovieID").sum()
num_favorable_by_movie.sort("Favorable", ascending=False)[:5]_____no_output_____from collections import defaultdict

def find_frequent_itemsets(favorable_reviews_by_users, k_1_itemsets, min_support):
    counts = defaultdict(int)
    for user, reviews in favorable_reviews_by_users.items():
        for itemset in k_1_itemsets:
            if itemset.issubset(reviews):
                for other_reviewed_movie in reviews - itemset:
                    current_superset = itemset | frozenset((other_reviewed_movie,))
                    counts[current_superset] += 1
    return dict([(itemset, frequency) for itemset, frequency in counts.items() if frequency >= min_support])_____no_output_____import sys
frequent_itemsets = {}  # itemsets are sorted by length
min_support = 50

# k=1 candidates are the isbns with more than min_support favourable reviews
frequent_itemsets[1] = dict((frozenset((movie_id,)), row["Favorable"])
                                for movie_id, row in num_favorable_by_movie.iterrows()
                                if row["Favorable"] > min_support)

print("There are {} movies with more than {} favorable reviews".format(len(frequent_itemsets[1]), min_support))
sys.stdout.flush()
for k in range(2, 20):
    # Generate candidates of length k, using the frequent itemsets of length k-1
    # Only store the frequent itemsets
    cur_frequent_itemsets = find_frequent_itemsets(favorable_reviews_by_users, frequent_itemsets[k-1],
                                                   min_support)
    if len(cur_frequent_itemsets) == 0:
        print("Did not find any frequent itemsets of length {}".format(k))
        sys.stdout.flush()
        break
    else:
        print("I found {} frequent itemsets of length {}".format(len(cur_frequent_itemsets), k))
        #print(cur_frequent_itemsets)
        sys.stdout.flush()
        frequent_itemsets[k] = cur_frequent_itemsets
# We aren't interested in the itemsets of length 1, so remove those
del frequent_itemsets[1]There are 16 movies with more than 50 favorable reviews
I found 93 frequent itemsets of length 2
I found 295 frequent itemsets of length 3
I found 593 frequent itemsets of length 4
I found 785 frequent itemsets of length 5
I found 677 frequent itemsets of length 6
I found 373 frequent itemsets of length 7
I found 126 frequent itemsets of length 8
I found 24 frequent itemsets of length 9
I found 2 frequent itemsets of length 10
Did not find any frequent itemsets of length 11
print("Found a total of {0} frequent itemsets".format(sum(len(itemsets) for itemsets in frequent_itemsets.values())))Found a total of 2968 frequent itemsets
# Now we create the association rules. First, they are candidates until the confidence has been tested
candidate_rules = []
for itemset_length, itemset_counts in frequent_itemsets.items():
    for itemset in itemset_counts.keys():
        for conclusion in itemset:
            premise = itemset - set((conclusion,))
            candidate_rules.append((premise, conclusion))
print("There are {} candidate rules".format(len(candidate_rules)))There are 15285 candidate rules
print(candidate_rules[:5])[(frozenset({79}), 258), (frozenset({258}), 79), (frozenset({50}), 64), (frozenset({64}), 50), (frozenset({127}), 181)]
# Now, we compute the confidence of each of these rules. This is very similar to what we did in chapter 1
correct_counts = defaultdict(int)
incorrect_counts = defaultdict(int)
for user, reviews in favorable_reviews_by_users.items():
    for candidate_rule in candidate_rules:
        premise, conclusion = candidate_rule
        if premise.issubset(reviews):
            if conclusion in reviews:
                correct_counts[candidate_rule] += 1
            else:
                incorrect_counts[candidate_rule] += 1
rule_confidence = {candidate_rule: correct_counts[candidate_rule] / float(correct_counts[candidate_rule] + incorrect_counts[candidate_rule])
              for candidate_rule in candidate_rules}_____no_output_____# Choose only rules above a minimum confidence level
min_confidence = 0.9_____no_output_____# Filter out the rules with poor confidence
rule_confidence = {rule: confidence for rule, confidence in rule_confidence.items() if confidence > min_confidence}
print(len(rule_confidence))5152
from operator import itemgetter
sorted_confidence = sorted(rule_confidence.items(), key=itemgetter(1), reverse=True)_____no_output_____for index in range(5):
    print("Rule #{0}".format(index + 1))
    (premise, conclusion) = sorted_confidence[index][0]
    print("Rule: If a person recommends {0} they will also recommend {1}".format(premise, conclusion))
    print(" - Confidence: {0:.3f}".format(rule_confidence[(premise, conclusion)]))
    print("")Rule #1
Rule: If a person recommends frozenset({64, 56, 98, 50, 7}) they will also recommend 174
 - Confidence: 1.000

Rule #2
Rule: If a person recommends frozenset({98, 100, 172, 79, 50, 56}) they will also recommend 7
 - Confidence: 1.000

Rule #3
Rule: If a person recommends frozenset({98, 172, 181, 174, 7}) they will also recommend 50
 - Confidence: 1.000

Rule #4
Rule: If a person recommends frozenset({64, 98, 100, 7, 172, 50}) they will also recommend 174
 - Confidence: 1.000

Rule #5
Rule: If a person recommends frozenset({64, 1, 7, 172, 79, 50}) they will also recommend 181
 - Confidence: 1.000

# Even better, we can get the movie titles themselves from the dataset
movie_name_filename = os.path.join(data_folder, "u.item")
movie_name_data = pd.read_csv(movie_name_filename, delimiter="|", header=None, encoding = "mac-roman")
movie_name_data.columns = ["MovieID", "Title", "Release Date", "Video Release", "IMDB", "<UNK>", "Action", "Adventure",
                           "Animation", "Children's", "Comedy", "Crime", "Documentary", "Drama", "Fantasy", "Film-Noir",
                           "Horror", "Musical", "Mystery", "Romance", "Sci-Fi", "Thriller", "War", "Western"]_____no_output_____def get_movie_name(movie_id):
    title_object = movie_name_data[movie_name_data["MovieID"] == movie_id]["Title"]
    title = title_object.values[0]
    return title_____no_output_____get_movie_name(4)_____no_output_____for index in range(5):
    print("Rule #{0}".format(index + 1))
    (premise, conclusion) = sorted_confidence[index][0]
    premise_names = ", ".join(get_movie_name(idx) for idx in premise)
    conclusion_name = get_movie_name(conclusion)
    print("Rule: If a person recommends {0} they will also recommend {1}".format(premise_names, conclusion_name))
    print(" - Confidence: {0:.3f}".format(rule_confidence[(premise, conclusion)]))
    print("")Rule #1
Rule: If a person recommends Shawshank Redemption, The (1994), Pulp Fiction (1994), Silence of the Lambs, The (1991), Star Wars (1977), Twelve Monkeys (1995) they will also recommend Raiders of the Lost Ark (1981)
 - Confidence: 1.000

Rule #2
Rule: If a person recommends Silence of the Lambs, The (1991), Fargo (1996), Empire Strikes Back, The (1980), Fugitive, The (1993), Star Wars (1977), Pulp Fiction (1994) they will also recommend Twelve Monkeys (1995)
 - Confidence: 1.000

Rule #3
Rule: If a person recommends Silence of the Lambs, The (1991), Empire Strikes Back, The (1980), Return of the Jedi (1983), Raiders of the Lost Ark (1981), Twelve Monkeys (1995) they will also recommend Star Wars (1977)
 - Confidence: 1.000

Rule #4
Rule: If a person recommends Shawshank Redemption, The (1994), Silence of the Lambs, The (1991), Fargo (1996), Twelve Monkeys (1995), Empire Strikes Back, The (1980), Star Wars (1977) they will also recommend Raiders of the Lost Ark (1981)
 - Confidence: 1.000

Rule #5
Rule: If a person recommends Shawshank Redemption, The (1994), Toy Story (1995), Twelve Monkeys (1995), Empire Strikes Back, The (1980), Fugitive, The (1993), Star Wars (1977) they will also recommend Return of the Jedi (1983)
 - Confidence: 1.000

# Evaluation using test data
test_dataset = all_ratings[~all_ratings['UserID'].isin(range(200))]
test_favorable = test_dataset[test_dataset["Favorable"]]
#test_not_favourable = test_dataset[~test_dataset["Favourable"]]
test_favorable_by_users = dict((k, frozenset(v.values)) for k, v in test_favorable.groupby("UserID")["MovieID"])
#test_not_favourable_by_users = dict((k, frozenset(v.values)) for k, v in test_not_favourable.groupby("UserID")["MovieID"])
#test_users = test_dataset["UserID"].unique()_____no_output_____test_dataset[:5]_____no_output_____correct_counts = defaultdict(int)
incorrect_counts = defaultdict(int)
for user, reviews in test_favorable_by_users.items():
    for candidate_rule in candidate_rules:
        premise, conclusion = candidate_rule
        if premise.issubset(reviews):
            if conclusion in reviews:
                correct_counts[candidate_rule] += 1
            else:
                incorrect_counts[candidate_rule] += 1_____no_output_____test_confidence = {candidate_rule: correct_counts[candidate_rule] / float(correct_counts[candidate_rule] + incorrect_counts[candidate_rule])
                   for candidate_rule in rule_confidence}
print(len(test_confidence))5152
sorted_test_confidence = sorted(test_confidence.items(), key=itemgetter(1), reverse=True)
print(sorted_test_confidence[:5])[((frozenset({64, 1, 7, 172, 79, 50}), 174), 1.0), ((frozenset({64, 258, 98, 7, 174, 181}), 172), 1.0), ((frozenset({64, 1, 98, 7, 79, 181, 56}), 174), 1.0), ((frozenset({64, 1, 98, 7, 172, 79, 181}), 174), 1.0), ((frozenset({64, 258, 98, 7, 174, 50, 181}), 172), 1.0)]
for index in range(10):
    print("Rule #{0}".format(index + 1))
    (premise, conclusion) = sorted_confidence[index][0]
    premise_names = ", ".join(get_movie_name(idx) for idx in premise)
    conclusion_name = get_movie_name(conclusion)
    print("Rule: If a person recommends {0} they will also recommend {1}".format(premise_names, conclusion_name))
    print(" - Train Confidence: {0:.3f}".format(rule_confidence.get((premise, conclusion), -1)))
    print(" - Test Confidence: {0:.3f}".format(test_confidence.get((premise, conclusion), -1)))
    print("")Rule #1
Rule: If a person recommends Shawshank Redemption, The (1994), Pulp Fiction (1994), Silence of the Lambs, The (1991), Star Wars (1977), Twelve Monkeys (1995) they will also recommend Raiders of the Lost Ark (1981)
 - Train Confidence: 1.000
 - Test Confidence: 0.909

Rule #2
Rule: If a person recommends Silence of the Lambs, The (1991), Fargo (1996), Empire Strikes Back, The (1980), Fugitive, The (1993), Star Wars (1977), Pulp Fiction (1994) they will also recommend Twelve Monkeys (1995)
 - Train Confidence: 1.000
 - Test Confidence: 0.609

Rule #3
Rule: If a person recommends Silence of the Lambs, The (1991), Empire Strikes Back, The (1980), Return of the Jedi (1983), Raiders of the Lost Ark (1981), Twelve Monkeys (1995) they will also recommend Star Wars (1977)
 - Train Confidence: 1.000
 - Test Confidence: 0.946

Rule #4
Rule: If a person recommends Shawshank Redemption, The (1994), Silence of the Lambs, The (1991), Fargo (1996), Twelve Monkeys (1995), Empire Strikes Back, The (1980), Star Wars (1977) they will also recommend Raiders of the Lost Ark (1981)
 - Train Confidence: 1.000
 - Test Confidence: 0.971

Rule #5
Rule: If a person recommends Shawshank Redemption, The (1994), Toy Story (1995), Twelve Monkeys (1995), Empire Strikes Back, The (1980), Fugitive, The (1993), Star Wars (1977) they will also recommend Return of the Jedi (1983)
 - Train Confidence: 1.000
 - Test Confidence: 0.900

Rule #6
Rule: If a person recommends Toy Story (1995), Silence of the Lambs, The (1991), Fargo (1996), Raiders of the Lost Ark (1981), Godfather, The (1972) they will also recommend Pulp Fiction (1994)
 - Train Confidence: 1.000
 - Test Confidence: 0.750

Rule #7
Rule: If a person recommends Godfather, The (1972), Silence of the Lambs, The (1991), Empire Strikes Back, The (1980), Raiders of the Lost Ark (1981), Twelve Monkeys (1995) they will also recommend Shawshank Redemption, The (1994)
 - Train Confidence: 1.000
 - Test Confidence: 0.854

Rule #8
Rule: If a person recommends Pulp Fiction (1994), Toy Story (1995), Shawshank Redemption, The (1994), Godfather, The (1972) they will also recommend Silence of the Lambs, The (1991)
 - Train Confidence: 1.000
 - Test Confidence: 0.870

Rule #9
Rule: If a person recommends Shawshank Redemption, The (1994), Fargo (1996), Return of the Jedi (1983), Raiders of the Lost Ark (1981), Fugitive, The (1993) they will also recommend Pulp Fiction (1994)
 - Train Confidence: 1.000
 - Test Confidence: 0.756

Rule #10
Rule: If a person recommends Silence of the Lambs, The (1991), Fargo (1996), Empire Strikes Back, The (1980), Raiders of the Lost Ark (1981), Fugitive, The (1993), Star Wars (1977), Return of the Jedi (1983) they will also recommend Pulp Fiction (1994)
 - Train Confidence: 1.000
 - Test Confidence: 0.756

</code>