# Notebook from SauravMaheshkar/Trax-Examples
Path: NLP/NER using Reformer.ipynb

# Install Dependencies
_____no_output_____Install the latest version of the [Trax](https://github.com/google/trax) Library._____no_output_____

!pip install -q -U traxERROR: After October 2020 you may experience errors when installing or updating packages. This is because pip will change the way that it resolves dependency conflicts.

We recommend you use --use-feature=2020-resolver to test your packages with the new resolver before it becomes the default.

pytorch-lightning 0.9.0 requires tensorboard==2.2.0, but you'll have tensorboard 2.3.0 which is incompatible.
kfac 0.2.3 requires tensorflow-probability==0.8, but you'll have tensorflow-probability 0.7.0 which is incompatible.
WARNING: You are using pip version 20.2.3; however, version 20.2.4 is available.
You should consider upgrading via the '/opt/conda/bin/python3.7 -m pip install --upgrade pip' command.

# Introduction
_____no_output_____---

**Named-entity recognition** (NER) is a subtask of *information extraction* that seeks to locate and classify named entities mentioned in unstructured text into pre-defined categories such as person names, organizations, locations, medical codes, time expressions, quantities, monetary values, percentages, etc.

To evaluate the quality of a NER system's output, several measures have been defined. The usual measures are called **Precision**, **recall**, and **F1 score**. However, several issues remain in just how to calculate those values. State-of-the-art NER systems for English produce near-human performance. For example, the best system entering MUC-7 scored 93.39% of F-measure while human annotators scored 97.60% and 96.95%._____no_output_____# Importing Packages_____no_output_____

import trax # Our Main Library
from trax import layers as tl
import os # For os dependent functionalities
import numpy as np # For scientific computing
import pandas as pd # For basic data analysis
import random as rnd # For using random functions_____no_output_____

# Pre-Processing_____no_output_____## Loading the Dataset_____no_output_____Let's load the `ner_dataset.csv` file into a dataframe and see what it looks like_____no_output_____

data = pd.read_csv("/kaggle/input/entity-annotated-corpus/ner_dataset.csv",encoding = 'ISO-8859-1')
data = data.fillna(method = 'ffill')
data.head()_____no_output_____

## Creating a Vocabulary File_____no_output_____We can see there's a column for the words in each sentence. Thus, we can extract this column using the [`.loc()`](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.loc.html) and store it into a `.txt` file using the [`.savetext()`](https://numpy.org/doc/stable/reference/generated/numpy.savetxt.html) function from numpy._____no_output_____

## Extract the 'Word' column from the dataframe
words = data.loc[:, "Word"]

## Convert into a text file using the .savetxt() function
np.savetxt(r'words.txt', words.values, fmt="%s")_____no_output_____

## Creating a Dictionary for Vocabulary_____no_output_____Here, we create a Dictionary for our vocabulary by reading through all the sentences in the dataset._____no_output_____

vocab = {}
with open('words.txt') as f:
  for i, l in enumerate(f.read().splitlines()):
    vocab[l] = i
  print("Number of words:", len(vocab))
  vocab['<PAD>'] = len(vocab)Number of words: 35178

## Extracting Sentences from the Dataset_____no_output_____For extracting sentences from the dataset and creating (X,y) pairs for training._____no_output_____

class Get_sentence(object):
    def __init__(self,data):
        self.n_sent=1
        self.data = data
        agg_func = lambda s:[(w,p,t) for w,p,t in zip(s["Word"].values.tolist(),
                                                     s["POS"].values.tolist(),
                                                     s["Tag"].values.tolist())]
        self.grouped = self.data.groupby("Sentence #").apply(agg_func)
        self.sentences = [s for s in self.grouped]_____no_output_____getter = Get_sentence(data)
sentence = getter.sentences_____no_output_____words = list(set(data["Word"].values))
words_tag = list(set(data["Tag"].values))

word_idx = {w : i+1 for i ,w in enumerate(words)}
tag_idx =  {t : i for i ,t in enumerate(words_tag)}_____no_output_____X = [[word_idx[w[0]] for w in s] for s in sentence]
y = [[tag_idx[w[2]] for w in s] for s in sentence]_____no_output_____

## Making a Batch Generator_____no_output_____Here, we create a batch generator for training. _____no_output_____

def data_generator(batch_size, x, y,pad, shuffle=False, verbose=False):

    num_lines = len(x)
    lines_index = [*range(num_lines)]
    if shuffle:
        rnd.shuffle(lines_index)

    index = 0
    while True:
        buffer_x = [0] * batch_size
        buffer_y = [0] * batch_size

        max_len = 0
        for i in range(batch_size):
            if index >= num_lines:
                index = 0
                if shuffle:
                    rnd.shuffle(lines_index)

            buffer_x[i] = x[lines_index[index]]
            buffer_y[i] = y[lines_index[index]]

            lenx = len(x[lines_index[index]])
            if lenx > max_len:
                max_len = lenx

            index += 1

        X = np.full((batch_size, max_len), pad)
        Y = np.full((batch_size, max_len), pad)

        for i in range(batch_size):
            x_i = buffer_x[i]
            y_i = buffer_y[i]

            for j in range(len(x_i)):

                X[i, j] = x_i[j]
                Y[i, j] = y_i[j]

        if verbose: print("index=", index)
        yield((X,Y))_____no_output_____

## Splitting into Test and Train _____no_output_____

from sklearn.model_selection import train_test_split
x_train,x_test,y_train,y_test = train_test_split(X,y,test_size = 0.1,random_state=1)_____no_output_____

# Building the Model_____no_output_____## The Reformer Model_____no_output_____In this notebook, we use the Reformer, which is a more efficient of Transformer that uses reversible layers and locality-sensitive hashing. You can read the original paper [here](https://arxiv.org/abs/2001.04451).
_____no_output_____### Locality-Sensitive Hashing
_____no_output_____---
The biggest problem that one might encounter while using Transformers, for huge corpora is the handling of the attention layer. Reformer introduces Locality Sensitive Hashing to solve this problem, by computing a hash function that groups similar vectors together. Thus, a input sequence is rearranged to bring elements with the same hash together and then divide into segments(or *chunks*, *buckets*) to enable parallel processing. Thus, we can apply Attention to these chunks (rather than the whole input sequence) to reduce the computational load._____no_output_____![Reformer LSH.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAoAAAAGbCAMAAABj6tHUAAAAA3NCSVQICAjb4U/gAAAA4VBMVEX///90dHTKysr///+ioqIiIiL///////8PDxBCPyn///87nOPq3mamQ2IAAADCwsK7u7vy8vLd3d38/Pzj4+M8oOnV1dX4+Pjr6+uWlpazs7PMzMzv42jFxcWRkZKamZqqqqrQ0NB0dHQWb6egoKBpam3JycmtpENpZ2ReXl9EREmLi4uGhoZ+fn0AKlp0Hj2Ojo5SVFM5AAEAAC84ltrz5mo7Mw8yick7OzvNw1hQTEaMNlFlXgBaUwAvLy4TEhIiIiIASXmKj5SXTWCOhSkeCwBnCjGGfiLd0WAeJDteVwBQ5b7EAAAACnRSTlPBTEpMSko3SvfnLm/tgQAAAF96VFh0UmF3IHByb2ZpbGUgdHlwZSBBUFAxAAAImeNKT81LLcpMVigoyk/LzEnlUgADYxMuE0sTS6NEAwMDCwMIMDQwMDYEkkZAtjlUKNEABZiYm6UBoblZspkpiM8FAE+6FWgbLdiMAAAgAElEQVR4nO2dC5erNrbnz8zcO2ctpa9AQJCA8HIwsct2t5100pXq7qrq1a879/t/oNlb+IXNFmC7yo/wT06Vy0JCwI+t15b05f9+GTToivo/bNCg6+nLV8byLMuvnY9Bv1EBgFO+WvEXee2cDPpN6svXjCvGBJ9fOyeksoSUL5Zk2JKVdESb0WFLz6MDAxbSgZFryI706YgZK+hAR9GpzphFR7TYjM6OcuiIhfGeS8NFuhEdMWx8ul++BlP9lLnLmLNMfPzDXwaqzJkoBKt+sHiRwS+VSXseoq2Utv4CAuaF+mAAV+O0WQW3BQ+IwCVnxWtBBK4WjC/IVB2fUxEXnC2mRFj6mrl8SYQFXNhkqpMpex+R2Sk9Tp1xzlnwRKRaPCWMz8lUvYjMzoiz6YSMGNL3POEye6dSnY4IALUFZBWEoykHTkP+tOA8Yx73mP7hrvj8HX77/HW14BMgkfP5K8Zb4W/ngwG0qJCpJTgVBiHFjArMAEAy1wggFeZwNmp+k0ELAFCQqQp7QoVZU/ZG1sJfAUAqTAGAKRWYAoCkcQAA36mwHAC0qcAxvPQuESa5C3eWUEgByMZ8FqFVc3iO8AnBU7y0PQALDuHwmjqIZww3efEMx6+mgCzc8NGUOudlRAO4suibAc+mIKsVJwMIIaOMCjQBKE0A2gBgSQWaAISQIKACEUCPCjQBGJkAnACAVHtBnAYgi7S9A5o8z8vx9mO2X/YA5HPlqYwDgA4+A+EiiMwJ2cuoCvhQDQA+BIAZDSBjLtRPGGCI8n28i24NwEpiB2CkI8vXKoCC4DJ6GQCkwu4JQMoC/kemn0TBWbpSIOH6WHOoA5gKDNmzgDp/LnsNMOCDWyGDBXxsAN/1Y0o4s3SV1YOrgrO7rwigXwE4xlqe3AHIVngasKm6Luh+MICDBXwIAMkiOOejKEqgEug+cStfYBODh/kEvpDvT7nFOSI5jeLn6Q7AnC/zAjhVfFqWzyvqnJfRYAEfAkC6EeItXl9X2Avgpq/vIzRn6etrtoL77E3fn/x3uAg1en9NXea8ethYhpvsr95fsB6oRq+v6QcPogwW8MEBbNYLeZ8/W4MFfAgAja3gY7lPNwPgYAEfAsCeFtCd3gyAgwX8LQJ4Qxos4EMA2LMIviENFvAhABws4J4GAG8TQFGe3acsSHeQkzVYwIcAsEMRHPNObqnOxIDp8qlDCv00WMCHALDdAsq3OeviVuDTlwVXfXnfrMECPjyA1UOUPN5+ffxYj75xGwP2ADyIcbLLzGABHwJAughWs6dJCPd2xKejKsNq/jSKFw7LlvCHNYdnHI2f5lDwhqn1miscroNDxnjPvNnzZD+zCKAzApLL8dNMMXuBnvwjhznz53HclIF2DRbwIQAkLaDgUzvjS+YVfJHp2p3k73bIuc/mbwz9VCWzeGFP4eYmfGx76B8j+MQKeAm/x1aBztOvzyA4AgBUfAz3FGKs3qWDnoMZh+8WVsJJkowaLOBjAxi86XN6cI/WTzrE2+gggK9MT1eS6DDNeMgSdL9CALUf1nIFZEJmihWTcQmKJUsnio8wK2EV430GnCyZ9ppekJkzarCADwEgVQT/x7O+oYCY2AA4wTvl7QHo8DC2yvcFS6Y6RLnP89iKZ1yi+ay1idPV8wtDfDM44DXA2AJS8sBSntrFM1jAhwCQsoD/qW0V4+kOwCkUofsAQnmqNWfJSocosXbSl0wFnK8UU1PUCxTB8K2DJfA7hqfAgW9jhr0Z55PT+ggHC/jQAP6HdnvBaUZbAEeVndu3gGvrtQHQfV3PTcTHr1ZvTNhaUAS/ycmbrLz5tcbpJKiO897JJ2DUimy8PMq0TPtDpmUWdzItU0+txMnBWwBjfDgxABgi6wvuurqDGsrQNYAeW2IkL6rmZRZ7NwgaIQIqgYLPqhhQS8TUZpjpxWmu06tJ0KwUJ6YnROAMJ6anREQ9MZ1MFQAkIgY4MX1FpfoOAM6IiAlOTKci6onpZHZwYjoRqCemU6nqielkql5EpooT06dUqjgxnb7nMnunIpIT091nPhthOSz4BvsVn4+wFaz4y2yFBW3OV8sJgDR7WgMon/l8oYvtd4hc7NJDRH1ICGNMET3Bq4bL03LMydfKqIy4XJAvqHsBj5xFdESb0WGJ8ujAlNl0YOQasiN9OmLICjrQMV5kTEe0mCGicOiIBQvpQF8aUnVzOiK1NAdz7XmB5ZEMN8WmGy5CX89ISueWE7q4KIfuosnRRgpclAMipVhqSHuR+nvp6SPiEArhbBHqAuBF4ynCRfHBKygMukeR3jB6StwFpEwDd4N+8/pgAGU+fbpAMoMeViSAanIJy6X4ZFh4cJBBd+CQOuiRNQA46KoaABx0VQ0ADrqqBgAHXVUDgIOuqgHAQVfVAOCgq2oAcNBVNQA46KoaABx0VQ0ADrqqBgAHXVUDgIOuqgHAQVfVAOCgq+rmAZRZSCmTDhkWWsymAz1hSJXldMScGSIKj45oM4sOdEwX6fp0xJjRYaFSposs6Yi+6SJN99w23XNiRtDtA8h/TwlnOv6F0C9T9vYTGTHyyIh/5Sx5GxF6C5ghO17JqYirNzals2MLQ6puRqb6OmL8r9SFcN/hPxL6J2dzMpAXjE+oU3Jl0RFX7PkXMjvE8gJ3AOAfv23WNwDg//z8XaN+/mHMfvkTEfHbNwCQivjfACC9BS8ASKbKveiVipi/scn3VMTfA4DfEGF/bJnrzX+lLgQB/C9CfwYA/0UF/pgx0/x6659UxD9M2dMPVHZ+uWMAv2nWHwHAv373u0Z9hwB+/21zxG/RAhIRf/erCcAAASSyYwQw0gASERFA+iLN6/3wX4kLOQ9AakUFtxVA6oEMAA4ADgCeIhLAb00A/u5UAL/7GADLDwEwHAD8eA0W8OEt4LbC6XqRwoWP1lstrH+5C2p5tGjaNOfXv+z+6acBOFjA+wGw3Kxt5eGSfs8eLkel/x69apTcZ2pBPPtwoThH4QJmEXW/TtJgAR/dAkZ8k/ZYumIELXC9QCVji6c1gNSCeIcAunqZrPiySyEMdcBHt4AbAHN9Vo/nxwBKK4KPwge0JP5gnv5CA+jgEjKeDT9Fzme+lL5gymF+tXKvG1tSnbUq1mABfysW0KuW7/OOLODLmD/zd8VKrqq1y9kMvnjzNIA2xkrh7yfgD+QhwRYfwUcLl4aBEGJlwo4a6oC/FQsIWC0rWzVd+Xme+9OXCsAVV67gCxZtALThn1yt4Lcb4cqWNneY4An8K6ReVSvmoesuIN3Jq4Avuuz/RWqwgL8VCwiF8IrzDKCbrpcgX+3VAW0utgCu0KT5CGCp92+Y4hZfIZcu/lUBKNe7g+DyzqPTdmdYa6gD/mYsIEjZHMzeusysNULiHYBys9QuLp4PX4m3ilexB6BYA4gm9YMAHCzgg1lAR5e/2IvS1AreA5Btlsi3efGywlWg0zwCuTcE4GAB7w/AQneqKDBvBwA+YedKxqWP2UK+JrjavbPARoiCqt/GYrIjAKv9kiZDHfD6FvDvJgDrgT0ArEc8GUChQFLwlSe8EZz7AMAVj6CFWwAKc+FNgS+fZ8J51wBC0ziC1kmh1AIoXC2Euw8gS7gv7A9qhAx1wAexgGVVg7OYt4BfU+doJOQ1gMYJbplpQfgMu2Gw8jeTVT9gApmNMaKnd7xed8OsAYSmMB9PBwt4fQtYt1g3BSCTLkgPX7iy+lWN5q5/4S5H1fcQ7rK9A6tI8N0mov5GrgPcTVovQx3woS3gthy+QW+YIJAyOnF/mrUGC/hZreANSI/kjuU8Q+FctB9n0FAHvHULeMsAMlf5p22RudVgAR+9H5C6uhvRUAccLOBVNVjAwQJeVadNy/zOOC2TG6ZlGgE8eVpm9HHTMokbMEzLvJAk/54SAPjPHwj9ZcLe/0ZGBACpiP8AAKfU7P4pAEin6kGDn4g4e2cTOiIASF+kmz1R2RkBgP+mLgQB/AOhfwGAP1KBHAAsqFPixHQq4o8r9kRn524BrE3SX+z/MRHleDQaj8aVRisI3XwepWxOze4fjX1VizjdxoOvJywc7+tpskt2HLJadmp/jJVfi/iCoZs/5izYv4xpLWIkxmSqE9eqp7pLczwu2Gh3HSO4ju1lwH+OV0u1fgeg+CZls0ntlKP9P0S+l+qifoaEzXRu8H84x2ovq6MxsSX8zQNYV2kKLCy6xW2bpkrZ1LQrkEwzyyHStYw9nEVmec3TE4Rl6JryMuOkmiIkN5EsqbkTKNuw+anITNMoIvqmq4yqLErPyjr1v90XgHJh6tTxsllGPJ04MSXrFQnxfjKciWCnyyQrj2DKk4K6/euIYbJMbecwniGbzI2CZWzsuFJ2kjQy6Ccpvb+pmwV0XtUyNL2e3pIAW4Qzq4lc6Vhwx8JuHXB3BuDcfFHKXhZOw80Us5Z5KS0wCS/Ognlh+WKbuDJBu8uQH6bzIt7B6zrF0qZOJeJZmrfP6XLCZRAfpKGypcFwyqKgb5wzMxQAOu2gCVA3nzfYP+nFxTwIfeOrua/HAhCf4TKNjp6hTc0s3cWz57b50UuVA0xpGCkc8IbDO/awSxWFy2VlQWWU0gZO2bPGt6cpt342z/JdQq41Cw3ZEUVBX5s/M9ZrdPz0uIyGUuOwUHe9MoMr1Teosx4NQNykPUkOKoNq3mHz7YY7eiwBpcssyKzUXPoeCh5NsUxDO03JafumgrkxJ2WwzNa8OkFgiqoCw/sXzduvGixoWodKhrOD91VFWTArSq/vsgSPByDIKebhPiCZqXa+U7TshlWUBEWS2r7qdbO9NClSKJxEU3FmKphJKTuY254rsllsPPHM0Fay2iona9XrkPls/065UNudLfcrGj30kADCTQ/nO3vizTrGEuG8sVZdz0M8s4WrfBvrOqXXNUPZDJoOYECLOaBbjyTzNDG3PMi8OHZQBIb6HcMqngFPe9nV6obJ7n7ulxXYRoPaMdVR0KoHBRDNw7xY36aiuz+YlwQtFmGvuJNOWcyTwm6/+X4x3xo44djLWbqLJKwk6dDyoDIcpFlSGOL7c0PjJEu6m11rXY+R9ixcnw74T2fLDtdv0MMCCAdbyRKfjD/r83jLZWaqzh81NrGdu5yFOV35kTk0WutZgDIrmAWWI/V7cvrKEVj6ulAdTOcZ0XwpDVW8o5qdWVVt0V+XvtKJ4RI69rUY9MAAYk8BNjrTDt0lexLhjFreBkrfxsYmlKzZcpbFTaZAWMvmrhVsVCfFMujQBiDkRrNNR4hnBUu74R2wDM0vkRraxk3yZ7nKsPSVXpxB/SPvXWtt0EMDqGv3ibGF2CQnSBuNkpMaymfpReFsVhz0gBm7VrDlESZBFvVuOmpBVWwvO64TztPD3sG9ituRRJL1Pa2P/QvQ3E3mPftaDHpwACEKWJle/RuoeHZcDrc1NlHQNEmTJRBVPR1T1wr2FmlL65UFROltTbAj5AAg6Rc1a+tmKX27vGXvmRJ+mgYFWO247+006eEBtApthnoumymyeVmL4ZZzU1fvftT1qInjFAE5OI395TtUcOB0ljT30DTLzWeNw7ACmkXZevDvvOGPozOqbAnWujAwfZIeHUA1w9e1/rw7yUn2GwdOrbhrlRRxspwtC7+5oGoqmLGLe552bFF6xZKqOrrKWia21zx8sZU/67WMKLTci5nuaxGF0XGhvx4dwHA9CODmwdLgLNOgveEtHHXvZUFlHKQ5dvotocA69EagC2Z4zkGT+8Jh6kfDEAdywlkap4be907DH5uzAXxBsizWWTba1RP04AB6811BdTg+0iqVaTuBjc1ep1XhtmsFmiZZqrue18TgmIcxF0fuC0fyZ+0DNq69TPYHi+vqOvxRuRaktpUGewbT5FjTXw8OYL0PujY+0kV+Unidxoj3T5LN6+5SUuhRk6z0hMyLDmMe6PYA7ZJmCL2MLH335MBpymIWNhrTbsMf2NeyTMJcyCipV39NbeveemwAneXB8br21ScFO0l6tRadYtY4qIumMAmCjqPNWJWLoF2SRQeHu7thCJPWwx9wtbo6WE8iPLwpDdJ9LYXuH3LSo36szha0gx4aQDc97jiRcZIc+2tRypdZ1sXvb512tO5aaZKwgaewWC7Drn1oLhSAySzbc3rwt1Uxo/aqeA7Y0lrvYOvwhwuVgBn2teizNnc+mcZXeuqhAcyXTU2H7mP/VemL5XCXo6HlSze1ofSvCmYcNZktm0dNmhL1rK2Dp+r4LtQNFPoOFtvqYMvwx9q1YOtiURLVX3/eb3iJ1iMDKKnnBS2BWXt7RG58VGWrsyrDloehcN93RmBVeTxbpl39hqVjJbPCtpP2XKDs2WGyIkLfQf0poIc/BLoWJPsdQR7d+eTNW3vlu+mRATRN/VGt/p/5Xn1NFUvzG9825tHg5YXGBgrYqJsXncyWQZJabT00OPyRNN0jZQcz21PU8AcwHszSumsBdj7Rp1HUTJGeemAAWyaCmMdHDou7PCEngDHXJ1oeOhemPnAcNUm0N11LNyNkJ6rmBJBT7SrR3XQuVAcbO3CaXQvcaElfsc6SwZj2kAZQGB+radIiw16Ghi8F2dqLz1msrReArRNBaDag0D1sbIqQKIdlft48D93pMk9NDl27PnFX5dg4Jj3fjcMfzszarw5W6ZUZmOH8qFnUofPJeK7OAgAjXCC1YWLK5hpH4+OwZLMIhfMKkdEY8+pxv1bLWuA6qc2aT4nTdFEfANVRVehYxPhIc7NDNT0TYS2D/Px5HjhqkiyPR020nKA+4ZJ2X1CJYYKlbrqKMp1tfAcB5mVSlA05bPB0aJC8xMDwl68+LzwvO145IeLrnC0adjqabZbW5zPHs3EJwPUygHwDIPX+zA5wVrxHr2YfADtOBPGzw+KTnuF4VCope55R5bxLdQlSQocuPWpSN0eNHSG6l3gZ5vWi2+jhsm0bKytNbM+n+7qx86lTxt2wh0c1oS9f02f8vXjDn56GAQpVpaTF/Sp3AKD+Xo8myeqMs7cqeqaX1i+4aABQv826gK5KaaG/0ADqMl/gDZEO7+FU2wNAr/Oh9fER1zK4veCkkd0jPxrz2JOxYKa1GzXZDN2V5IRL3UOz33DoOvtDQN5my6QkClCDp8ORzh8UAQAr9CDv6gmKUx8XI1/wxRIXHteHLCZT+CjYO1rCtemrA6hieQTgrCqa9QERrrY0r75AADM8y1KvjI4Lo68657YHgEUPdyNo723mj/iBYX0BhnPbNi7M0PIgSTUWzK3CEdhloj1jnLTZN3Z76J77gtHDZTP7A6dxLANI2iemkrhtng51nT0o8uWrx0frTm/4oBIoES2eKKFCvnY8XHBbOHzEbK6YXFf1NgAqvlpXW3gqUFsAV47KoCK4BXDOc2HzGAG0sbzHHRwy7qiSW91fou4A+kmvCrKIgyCSDVM+jhXhpBEZBQHd8gjPmedRCccjgMJl2u4tLSr3hXhmmv2B3c9ybTLXxea+7+BWXTwdaurjWNMkaIQ4E/6MbfdQrwrH59UuC+vtWxlaQKb3ihPAjc/rRTDu7fCmvSPW+8vx/UbIa7IFUOilyucLNhtZ+FHpZs+q+KA6oBv03TZb5kVqJ12cTkWY2Ck9EW075nG24mUYY5u3fc6jK6A1lVBVOnTiEwqTCvPaXGZXWUltsLi1v7NBxml37dLdMGDtuMOWuigMpgAgElVvhDjw3XjB0nVxuQUQxzifcE9CXrgSVGuEpNMtgL7etct12fKVV+lNx+Px23i9nVdHdQYwCvoXgE4C1fpOAC6X5AhBb58vUl6qT9Jx1MSeKRVl8yQrj48TaVIEOI2jcUp8ONssNNPR0+FQzrync3VNX75W24GMn9et3WJFAhhzl6+blhsAKze3ZWMjhCGudQD1sQvk1eHLAhR+DIBi2bsI1I1NL2ztN6lmmDdNGmlxRuiZnZoPLDZNlvOGFbrWcrOgyg720MyzPTuHRfkypTsOmZ5KMktz0dXT4Vjk+EoXffnPiqmM44ar8OF10Qgg7r4l+HJTLG8A5EumA1UjgKs5pOtiaS4UR0PtR2y2Yikcvu2n+ZAi2DQI16htY9Pcc7zrWlHhrDw4zOiM0FMNHSF61GRWW6Fro9rwh55gUk3YxenHy2DZ/k6IqEiK5cmju8buxxZ9+ZogCup9CigEWNfL1wDmWwDhg3zGFvGcb3qRNwAWaM3E9P24FbxALi3A2GFyAvROnnHv6gwbIe7TM8D5DE2aRQ5n7VHr6AigmPV8lZ10V6biGmlR4/10o3S2a3k49fX4eqxt1SpFdYToUZPZ4aiJSIvD+XFeHCRFkQSxV7Y3q1D2MoN296l11+McdBbUAbF/hE8Vq4ZEMrYGkG36R0bYVaK/8rebHs3XWwSDNQO9eruO6GolSGha435y8NAm2M+C5hP7eObVSAhury5f4O8nxdwpf+mc244AduyD3ginotfq5nk6Ox4fERYUVLXD9qap913byqS2KR961GRWbB26xJFbAK4QmRS2XQRF2Ik/Rw/9OCHUIE6rwMosONH2YyNEOdVewXBpuqorqjdZ+NUt9YT0qz5pZ7t6OkYBuXjweqPhKufrX67PPL9K1PE9qRN0dHq6T9vD9JzqAOlfuhvG6zAIt9Pe+gI7Odm8PorROOaBk0bcFmeE3upUFdsbNfGWtTaA0NM41m0WhSXrnntfs0S2XI/ESv9wsLirTh4UqbxhOtrPTRv44NwnnfgkdQMw61MnpnzeajaNNHBoOvyTxjwIqcLQnVcXjpqksyJIt00T6R24FtiJ50KpPaN7aOqGnNUHi3vJXp5EYA93rGjVp8H6IeoEoDPvXhqYRt2FPUs1DH46I5dDFcHy9Mr7cXZ6d4RESztLEnToUjl+2J9EslsYGqd44IINDQk4wdF6I8pOlid0ZVrzU/DoAaDfZZ3Rj1UXAN20dcnZ7aHE+gIbCZzem6eGMQ97lvlZcvqoW139O0JKbS89IGY+WwZlLZ9118CN+8Kxp0Nz7+DxMtStOmlQ5PEcUvPO1eEOo+4qnc1S6kFsnBFO7kA7OFc26zsMYSUe02N2Rew5AFiQ7ep7DbM/Dsbi6Ckf7HAqSTd1WG76SI8HYNc6FBR3baPuKlwWDjWTfM8ZoUNSrUKn075phIltadeCjeuMF4GVqwijHJYr9wU9YdhLzeuN6KkkvYxa7xVnHhDArn3Q7aPuW2+rhvnsMq87I/RoOzTL6W1FpYfupMcrRCrHxrmfgWH2m3DCdFlks/buYwXNnKZ1B8njew+KPBqAHQfhVNYy6l7vWjmYP9I05tHViZPIzqy575sQVugSXPuIiCTtJa5Zb9NzTfCIAhvHbTdU6oVmOl9a854iBj0agHYXA9g66u5G6bxeoxflfDM+AgVzk5NeNzf2Rpl3+TgU9rXMMysx7v0RVaPAsyDMmyyrLn1xgsm82Se/JhwsNi1DXZMI+s0UeTAA1bzDuwqlr/Ge6zGP46+r8RHDmId3WjnsmPbYqqtCJgNkRNBp8Ss9ajLHuSa1sbs9x+5qwdO8ZcsJETX4DhJ57DdT5MEA7DAI17a+AHatEB2xTpZkxv08TiiHcRiik+HEHuVgtu5R7jb7oxKOmgTzYjfX5DCXWKAf+Qoen55Yhvr4yLDP8ll3BaBg88ZJoGsp0b4jSMumVtjyoHcSzIsgM3fR9l5JsNmv6zhd9HoOtvPSHXJdArdpZQS9l4ydztEtS1brjB9KOnE6T82L3utlqK0Ocw31oEjH23BXANpZYDJxTlYYByWUaUtJfb/8dNk45oGBa2cEYUMN0GXkDYbala9P1SqF25KYp3xoCcfWu25uMhYaVmYRFmteGQHlRtB0SZYJ1bun54skxkVaXe072FoUgw2WHZvDdwWgWKamAcc8CUy74ua2YcqHiI+6VvZiqn1nBJw/As+AHB7BSSN2e5eyEx564dQzVP10Dma+aecXuv88SjPDAqaFcIIs11s8bVfocus5qCaYbCYBNK45UBbzqhvRAGq0DDvuQXJXALLMtO0eQx84+qaooKCnfMgii5PAb75nzrKse0pLP0jKjGyFijApWqtBIk0Dw2ofUNA3z/1lVpCQvb1uYdq2EBrOuvrraqw3c02OqgCuyMNltfWb19ynomxchtoxNTbCoOOqdvcFoGNsZtrGXlB4NmTxjQ+OnMgG5WRyVDA7EINeGyWE5AxZQWWB4WVSRWJly+P1KRl2cwRJQI14+UlAb3yM17G7Rhw1SZZB6BeNm59U7gsWmZoXJkFADlLiWxJ081G9LwCZ8aIy0ztnz4IiIwpgFx72kuoZ9JZQbzriXobwLUVglBWBaYtKho0PyA7J0SxIqZ1P44D2W3bTIqIn1c+SNDsYVMQtFwOo9zZeOiBapAH1yrsiDQJ6rV8XCO3UJ3VnABpbjKZST/qKXoFK+b7vUHVvCPO9oy4w5TjwPV2ECkWU5+twyA4ZruysWFJVKNPaWMJ0Bzyv8Yz5LClC4s2VxbLIqG6nOLSsklpdQcfu1Lt5ZwD+ZuSqSy5FT0v6BqJVvx2RT9IA4KCr6uYBdB1arqDDPGaIKKQhcDPhpUnKlKo0ZMdhniE7pos0ZeeMizRkx3iR5ntuusjm53vzAApOS4R02JTRYbz0DIEsocMSU6peaUp1SofZpot0MzpsZMqO75iys6DDClOqyqbDXtgzHUj0Hd08gJL/8dtmfcOF/Xsi7NvvJ+yXP1GBP0Uep8L+xFnwNyrwbynj//0zIe5F/6TCfv1/bPoDFfg/AODviLD/BgB//C9Cf1gw/mcqEAGkwv4MAJKdQDMAkGzscWEdrjC6VQwAkpNAX+4YwG+a9UcNYHOYBvB7KpAjgERYBSARiAD++rtmfYcAfkeE/YAAUoF/RQAJGQH814cAOAcAqRaQCwCSq+lZACA5Fvr02wLwm1MB/OZDAPx5APA3BuDJFvBjABws4AMC+O2HWMCPKYIHC/iAALT03kkAACAASURBVA4WcADwM/TJdcDBAg4A1jVYwAHAq2qoAw4AXlWDBRwAvKqGOuAA4FU1WMABwKtqqAMOAF5VgwUcALyqhjrgAOBVNVjAHYB/3/t5BOBhYA3Ag8ABwO4a6oCDBbyqBgs4AHhVDXXAAcCrarCAA4BX1VAHHAC8qgYLOAB4VQ11wAHAqwqnZTbCsJ6WSaDygQBSHOlpmUSYCcCfNYDNgd9dYVrm/NRpmQgguVPNHQP4J0oA4E9U2N8m7PV7MiIASIV9jwCSqQKAP/xKCADkVNgPr2zyDyrwFwCQCvsVAfwzIbSAf6ACEUAi6O9/4GyeEKsYeCMAMKeWOAAAn6iI2Yo9Z1Tg670CeOrKCKvTV0ZY0mHL01dGWNFh4cesjOCbsvMRKyM8sXc68F4BHPTYGgAcdFUNAA66qgYAB11VNw+g8j5nqdBKrudJ5qrWLfwuKH2BwvuExUg3usJFwgUSmyxVAFbLJzvkyu5R+w5YF4p0nIpFrvj1ARK2rZgb26dsPn+icquEW29b52443F36IqX1mRfJPOoCEUBs54/hNbQ5FX1Ghhi0OCXSkfLPBdCyEEDrUwGEC3Ss+BMBxIuUsXWJbd67yqMuEACcc9vxp1wYAEzeWs/AZ4ffzN/75JDS5wIoBwA/RgYAPY5PWPACAZS6pNbr+eMPAfUFrCksEUCxqTOodc5lVYuA8tuT8m2udwQV3vaY2R0COFjAD5IBwKKye0UCANqcv/lgzXCccJUyh2PpHFYW0Fr3ZYsJ5+8IBYbNXOa+L5657h73mZxDGO5PAr+nk9dL5HwA8OK6MQBn28Flm088b/XkVgA+BQDgk69mUDgDgBFfj15PVx58p1jIbZXzOWNvPFLCeVs4ko3eHZXA8QGPhcUHALvoNw/geLz5rOuANvCjWXtBAH0snH2WPPt800RG5mTgybeUoVmEo3XIWwJn4fjguF19NdQBO+k3D+DiEEDVAOA737OTT7jJreLoeSOhArkGcIlBi9FowQOhQbxHAAcL+EEy1QGrFq4fmQDkC77dfM8LV4CjwjAA0NoDMONpAIoVx2sbAOyk3zyAceX++r7YAxCRWtUABLiq/ErcWtHhscTGCfM3RzOe4F/rhrJurwyt4E76zQPI3lYQVECpuQVwNYH7wtM9AMFKTt70cIlbkRYDlHAVzyu2BvBlgfZwIplYOGz8LJh6vsNGyBXqgP4awM8birs1AMX72l9wC2AOf78DaFsAcSRE8IWOgYF8BM9qAr9f4XFxbI1APDhOYRjwLHDLpucLjYTEfu6f8v8pkfI1gOVJsfVZ+0aI1wB+4FU1XSQAGH3ijY0MAOotcXXXM9bq9D63Cr5wPCZ9gVvbSuahRVB5lYTw/cpAOL7ekni9a66T64OrMPgtxEXMSGSdqDA+LZ4G8MS4p0kD2FXhRU6pATxVJ2bBAODH6DJlih+dpDxLs/KUiCUAmJ92ziiK08LqHwteY69jXq0iPeEERyoFkyfHhTt70g36dACvKmV9at1xd9aP9XGC6hs57+yzVF70Ih8UQChFP9G/aXtWKEk//gyf6UXVJHwJLtdielAAfetT23gblR9vdi9rf04SvAT+xRJ7TAChKMyvcd5PME+XtT+nKcJWzIX0kABKss31sRIXfDCkLmp/ThPe30u9BA8JYP4ZIDTI+xTuL2l/TpR3uZfgEQG84O3pJ/9TxjPAzn7isEmzLveKPyCA8moPqPycmufVXrCdZHypwesHBPBqRZT8rDHka1Ux9uRdqpl3MQDl6Br9Hg26nn1Qn8XF9Wz8TvmFOrouByC/drFQCWtIVzq182lt74vZn9N1qa5+ANCfTwN8c2U2XWA/qptN507gszyFE3iJwB+rAHCPQ2cx0b5X+WyaYhQVrJLqPbCWfAFBXjqdX+3WaKMQXaejFs/tf94w2dr+XNMMbvpaz8zDl68+T7IVOqU+P2dL9Hse8SLj8CFE91L0OPX4KJvA74K/ZykPcCbIPJyuA8aVo2qsAfT4NJvzi6yIcIIcoa3QVYaqRO5+VhsEtW4E+Nfo7dyoGm1SZxbEX74GK/QyzVjGtV8qIKedALcACvYyggOfUgiEL+bPjCGE7mvA9Hym17lOSOI8kPELY1VC11AeXW+wXlqx0g/kk8yvtj/CvuaoXFUIl2fWvL58tXnhIDFPUz/3Qy6recJ7ACqeQcjimemQ9Am+QyPjOIKnEDCrFq0W2h/V0p+uVAmLLe96I6Vw5tjyc/uzasJof8qrDHhvBW975Jz7wkMd0J5ynrnseb0E6xGAThWAALoawHw9PWS9sOxKVn/4bLPIwnXKYKndSK/1SHwrjuPPtL94tiv3B1ZutOeloVvB0gZmnhdMStdlRwACVu46pALQ4VU1C8tpHVD9gRYwZtezgEoj8KmLju3JA/zh9J91dqmQ+Os6BwpP3/HzLvnLf/Al00sLFdgQETngFeGsX5vp6XJI4TNW9Ry1BVDXAdnrjE1xsrCzMYdQAx9BBREOu04d0KkQsK/TSYv4f5799W0LH35sX68drOBOx+dfM07LnBdjsGnymacpX+GyavAbjRt/KhZQJgOSq2IJNlIDGLyC9YcW73QdkKyX7JDQAIYyeFXMdzOIP1d5dT/86zwTiQB+nhO2Kj/X4jZI+lUWzqsGYD9gMsWlDpgMRwt8o9xwsvSQIi+ZZH6B/YDFZAn2LULUSvzhBxP8nqliMtt0PTijQH+RXKteovGLrzZGFX+uEzb6RsefO7XySCo+vxpAjYRcy4ydLt0GuaK7+qXGpjpLoBG88vCTj2/BWa/d4wCI05Cu2S3mfD4MzvWGHTcS53Z8UQA+XWFO2Xly7Ot6Q6grOGHL6Cqe3/tyz7zvj+OOddVhKZC4jhP2tWcowYWfVe95HAB/o7q2W9a5GgAcdFUNAA66qgYAB11VA4CDrqpWAOczxp723FsWCWPEQkeDBvVWK4DTyWYN1EovI+Zczed50MOpHcBxHcDViDFnsICDLqQvX5Ujo0h3osoc10pkIpeRw5gX6ZGtLYCiOgoAdHPB/ByFX/jRbUyHG3SX+vI1febvHDf9cHBh6Cn6oD7zBQvw22gHYInOz6UGEHdomGpn6Iy5YzjuZbCIg04U7hXnMHeEfqcjhK+EfzZ+yKsNV9cA4maG2tV0DSBq8S71RCXByY2GBw0ya71RDXrZCxzP5Fa124cErgRuxLUGMERHffR63gGYoRerbo9kq6tew6A71ma3TB4zsai2a6i2myn15g1bAJNqAlK0A9BH9mT19bUmYg66e60BdAHA1+fc8TcA+jx1vHAHYPqM+0P4YgugwE0LAcBCf3/tyxh0r1oDWHJP768lNwBm1bdbAG1tFuW2EcLGTxjdfQ+umftBdy8EsPB8bP3ysecsNgBa3FaWLoKrjmgXtw4OcRuvCsCCh7gBDoCZec50fu3LGHSvAgBfcXtzXIfoifNslazrgAXnK9ygcDyqumHEiPP3GFrFC4a7sT6/YtXvFUiFuuJoqAIOOlG6Fbxx5lWCScncyskWvxQufoG/2Mbld/23FCgEz1XXd8oddLfatIJvR8kLpafQfSUDn0X8RAbO2ZiOWJ/94j2TR76y7IlUwV7JsGfPfyYDp/XLn9GnsOTzitKrtJ+osJcZI+OtnusNSOeZ0tMrK57I0AwunwxUEZ3q7QG4+Nv3hH7KXE6Ffc+F/RMV9rcJeyVT5fV5rZ7hFCz9nx8I/SVhnAr7gXs5GfiPn+qXP/43deQvoeR/oMRlSAX++ccR4/8iI9aXlHN4FDcLmqWzUdkcVk5SxkMiYsxVzKmIyZev8trLDR9o8f23hP4GAP6RCPsGAPw9FfH7CfuFTPWnQwCpA/8EAP775++++7npv38AgL82B333MwJIRfz1EMAfqCP/AgD+FyUA8Ecq7A8A4J/JiIcAUs/FAwDJEa8EACQ5AgCfqLD49hxSAcBvGlUB2Bz2zR81gETECkAi8BhA4kAN4He/a9R3GsDmsN99hwBSEX84ApA68soAOh8CoPUoAH7z4QB+MwD4Gwfwm0e0gD8PAFJB19JgAU8A8O8DgBfT5S3gN4MFHADsrsECDgBeVYMFHAC8qgYLeAKAQx3wchos4AkADhbwcrrVjuibtoADgJfTSRbw28ECDgBeSIMFPAHAoQ54OQ0W8AQABwt4OQ0WsAuAfzcCWA+tA3gQcwDwUIMF7ALgIU19LeAu4gDggQYL2APALU9NAG4gG4rgXhos4AkADnXAy2mwgAOAV9VgAQcAr6rBAp4A4NAPeDkNFvAEAAcLeDkNFnAA8Kr6oGmZfzp7WuYfAcB//Pxdo6ppmVTgelpmY1jjtMzGI295WmbwQNMyR/wnQhwApMJ+AgDpiFP2TgfWdzz1DKdgAf+FEF8yMuwX7kV0xIMnPqaPDDerMTZJhnTYmBki1heGcPgLtbwDAEgFvnAAkFr84QUnplOpvt8egHoPZKvpP8thFogIlJ4VN0eMrYjFdKr1N1dQx8GRuEEvJZ/ZdKBQdNjBjr85faTnWtTiA5blenRgzmI68PDySdn4aChVj+aEy789AAf9pjQAOOiqGgAcdFUNAA66qgYAB11VA4CDrqoBwEFX1QDgoKtqAHDQVTUAOOiqGgAcdFUNAA66qgYAB11V9w+gBAn4T7rXzkkH3VVm+8jdXFjfXdvuGUApPD+KLcvWsqwy8m9327CGzHo3m9k+cvWFxdsLi/HCur9fdwugcOBplr6nRPXSSSGUk8N30Q0+V53Z+E4y20fSieDCcmd7YVIoz8cn43S8sPsEUPj62TW8Z1I5pR07t7R9511lto/gtbIjRzVcmCsQTL8Lg30AdE0J9rC6Z8r1YjtXhqcmvdKObmQDsrvKbC+pyI4804Wp3C69Vio0gMLuRI/iztF3mb8J86oPDjlx5TJyHcsyXXcl4dux97EZ6aJumZW3kdle8roYOAmX77SgpQFMuaWPL/A2FD51bAOAkmeM2fEegAX/yBLFdeyym7mAZx9f2bDcVWZ7yWsHa3NkbJvfLQTQ5c8r/Kz0DCkeUsc2WUCkbTLfA5B9ZK1axT2ek3Ts6JpV/LvKbB+JqCt+KM/8biGAES8Rn3zOR4Wf8HEKVJWjBVhCdybscYDxZTbJvArAPIEftg2hRcSKXATPL4EAOO1JCrfQT6Fczp3FgjSkJ0vmdo8LxwiRffzKfJL6Zza/XmZ7ybHzfoWcb/v0nUAAJwv2DthYU75aRBP+MhZswWcLKF1dzqfpG3eZ+8yTEa8A1Bzyd8kE912eqcn760Qp/jYJcI5rBv+e+Hs6PZjxfL6UFfUu3ZUVX6eNqazyfjLbR6KPXV9LlhYZBwAUgEqINTelmcEiOEKLGHLBeKo30YbP8IVVAei+2/AdfLa562IdcDzDuBkeiSkxtnoC4t8v3BpxWioTzQK7co3KlXOSNXOvk9k+8kzWjBZ9PwBAREYiewp3r3cRwBn3HAcpxFqhBz8mWEnc1AEXU5YtZgVL5lUjZLypAyoe69ReFnDU9KIAupF1YhXJ+/yS7a4y20v+SVaAoXUnCkQA8OnJDu23xT6Ai2rVhpzhNwjgdMx2AObcXZXlO4O2cx1AD77ZAri6JIAyjk7uZySv/aMk4/J+MttHbn7qi4U3pbl+8eWrxxdjEJS3ewA+SwGCEnYD4ITtAJQcCl+oAHL1WQCKzo/FxVFx6dYAEHH+mWP/d5XZPnKjznVUfWF1jwsi9pevhV4dRwJ3uhtGA2jrpY4ctgUwwzqiv+mGeXke4yJC03U/4HjxwQAKq0uTGof7y3i94EptRPwc+9lbd5XZPnK7Gfa104W+sDJ39kaBGu3nl/+FCIHmAM772JLsaWpLd8VDawXQbQCU/NnO+AZAGyG1MKIGcM5tsQVQt4JH7JIAyg6PVHrR1jcBh/r1iHi0GQmCh3qpzLRm5J4y20du1IE/lW+dLiq/hBIudDsWlFvHNvBLMK4aXs7UY/l0IpkzWQnmFtNpAr9WcDfVBH8spiM8pDrNFFAWE/jLHVnwafSkxAobIdOcxVBY63XklmSHdk91eKQqt2PncDTaVX5sr8eLZPxJVau7ymwvRXHbEdK3LX9j8TbX5wpnN9idx0cMf/m/F8vhB8kt2x6HKvWja3o/4WFXAwzyc5qXbtlmvW4os73kN1ivmoRv502eMWxvZL7h9ty+O1be8uaJyDY5NMm86rgXn9LF5h+/4jW1Zda3o8/LbB95trn96wJ+hiNcx44x+LiAuHkAnZY3z7H9tlezGpBru4WX0AUym39aZvuo7Y3wWkey3WpAThx2JN46gC1XLgyDPDt5ehDPL9uPPE93ldk+cmNjzdbNu/RPixiNoHfwjt46gKXxypXdrdNM4tCEa310zaols9ZNZbaPfGM1SGqyuiSDnOb1V+vGAXQsU2iPgSsfjJP64HLtYpl1ILPihgph840TdudGu7J9qAbWrOVtA2gu05w+VXUP3r6PLdfaMttjFNX58Mz2Umx6dVSfJruCRki9EL5tAHPTu+X1ayrC4W6XOtjJumRm1Udnto88y1B1EP26jPDwWoXypgE0FkM9H6luWXrGQvI8GTOrbiyzfXRQZtYl+tZVgcDaneoDoJhTd9HKqChnvcWRoVJ/1J5vl2OJD5z8c1eZ7SPH0AJxOw181wTvor9XVvQBsGlOSKX0pf633HijT86ZoqRsOnJLx0Cz8shr6Sg+XcLQBXhzme0jY1UgP8HxzLHUnlfCZQAsVvW/fb4+g3fOuHpueGx5lyq6UJ7jeLvlOmTsX96qSM+PLMsqMo98UB+bWVFl4KOW+vAMBtAzmIitDi+MRdFeaQEAut56rFhFaLhc5YocPoioMmPS3wweKe4pfajc/nC9EhPWACr8wsMecRHz9c0Q8EtJmfebn7POeL1S5Uq8ELgSdDNr71Fxvdy24ijPo9jaDlIqK7ps2xLnnaNbi/KyPLLs5qmyH5lZnHobYQYqz5MPWGWhqgFWN99RNScKY+2wkvItGy7Mj0prO49dWtGuuPjyVU4553OGfvicPysm+Bg+jCz4sYKzxfD7rbJ8io/gj4BV3oHodyWe4IukAjDlPtNpZQx/VnvwzZ8Z4+hf/dz/zjj71lP4MVyI7/sR/PaVsWOAVTPCfbE+p4S7sJ5ICB8uaSYcK17fVMwsrgXQMCDqflxmXb+2OoH0otaxvr5S0ATGNTgsvPl5jP4u27C8rXyD19NXa79UiQvkVLnzrB256JAqWcRtZkMBK16XAOBIAnbPAgrSWE82cscvOg3FFy6zcIoS4oSep5NXPMpGADOcVTKCvyNIJ+frtziZAoCvHhzUv0m393qp0s63NxqecxaYGzfqcAtCdz07VVoXdDQRezv9rTMrGuZWtrVmycy6bZlV1tEQhDjcffBc+b7cX4MD3o/tMg5tfeWQl4OST5XVfYrCLboIICSTe1haSpZwABBvGC/gZO82W6Ilc6oqXVUHBGu5AdDTM9mLjBWTrJq+hM2b12BXB1xOd0n1lNqaaZkfvNeuHRunaPpND86xseLhhBfr3PDsXU1ml1koOOs3va2cOj2zzVPNTpuQR0la8WHFAgy2rne1jDw2T6Dz9O0RdrhJ88tXKEanBf5pYcn5CgAiK5qaN5uN9E6v1RzfCsDRYgugs1uPg2M72Oe4BSyf7QBMTgdw21aHR3rwpkG9WETkm+5GzUOTeq6Ga2UXKoNrQxvOrmNBlvXGq6kSb8ys25JZnyigu80J6CgvKY/PIvWgrjC3QIhxKqmnhkTZ5i3BVrCTvQJJFk8jZ/F2COCzXnlwzwI2A1hghJyHeLB/EQu4MfQNQz3appBjWxHVe6E9jZ3iMgai3g++3151y1rbwWwAT88sxd9FCRRFs187Dmz7xiFgOns5EKiyzWv55X9rYw/l6gonvgX8AMBgb0PvdRE8q6axA4BK7zYeR1gHjKEMVpvtty9gATf9at4xZ+vyriEERT7Syh1SZhcpg+tDG/VOwFq3nzK6CLZmlrxtnqGBckK/N5VQRpQzwqYJQzmG+iE699vF+sK/fF28ecrnIdT2HBEfFcEenysVvuubqPhKiUK3MWzhvHKcjuSIEuqM2ApeclzRw1feMzZd1uc/3QKuS66me7npH2z02jz0N6sJU4uLC7QTD7ylDtoZ+5k2dWaekVmz58OF/H6k1TSNaJ2BzDTnx5g9fEGddH3Al68Se11m0NZeQUVujBYQU8Y1OXCqJhgzUAWD4iF8xtA5/MbmiTuqvgheEddn18UelznkeczfdBTdDROsk+ol39/k9Sj/29qfczxKLs2Drp7leskFmolO3XL5B7ncDd8bxxHOyGxkdoFq7SDppChXdE+kbXqRzUM/0HyWwfoIrAPK9YRA1Xyxta83LfDNK+4eGCh3nVb7sowtKnXCTZ6QaofdcTus7c6XvhucX0M6JKc8NNPbrj/j2p+nZ9Y0SolyLzGpBF6Awzdr7wyWRefe6D/D9GMN10XirXrDSF3DaOxp2uugPgpvdeNUtnuBSqBfv/fyqDq0rfk5BsbOyGzZ1pJyzh/xcaEKcPRmbaUsQRv3thESeD+i9bTxWwWwshyNDa1o7+YfHmBumaFKr6RcdzrKPSpXG8zc5hFEBlJMcFYqvagxsy09IKwD3O1ChmlD6kU05Kr1FXdyL6ku4TYBlAKuQTYWJMqN1W41ldqTcBVrHwbwYic4z8nEzaHtsPd415mtq4Krntl6KqdnVjhOJNoINJHfTZYnXJvYeEa4uUNBrjCsRdBIWVZXf5sAshjkCK88fnoe1D32zFytjPBjy+SVpyUtZ35m/TSy4tqtx8wesOQKXQs6yGxNvmUZW8goaXkNmXVx05G2l8jc/90qqeCt8q2YGO0QEBITLXzPiqMOtj2pntyNAohPzrXi+HiZUWnF+10DtWJMdXgw0D6YnVk64RLdB38fFUauVc3DjsnesjMy6x+OHjdItLQD2gQ3P3Lq93pfJfoTNEPuGi56K99JK3pvFECJtzi3mvy9o1o/f626Ia12/yCW58GZ3bTCqtdxZAMPbmWaI3pQ4ozMeh0WgDtuGPUTQO4DZlRh6uA7J5urooaL3srLs+qlvVEA9YIcXqOJqDtB1u9zu+lHm3n2YNwhb02rh1T1IJPH5umZJR58XWd6xYCBloaVHnRPs9u8XHTb+hA6+TisbtqtAqg8qjtd1l4vtwZglw3KPCs9F8BDo6QarJSvTyINtsDptExCY2a7XMCZG49INH6G7Ou3J2q04apD9qS97mDqAaAg131XrVP+DT3qzdKtj2YTUT9X7UXv0rqQqus+ep1FNHR1kCEjHVI+I7PnzifRz5TOow4hmtodTu0qr3cruOTUO2W3Tj2yeMsBjeq0IOctTNx5RHW5r713Bz5WDwAjGsC3tozEJwE46PH1SQCWA4CDGgUAOk+cpy4wArWNAFd35jZ7tuacb0aBvBfOCxcAjN/5K5T60xS+HAdM8HjC0XkLAfRwchLK5hyd+h2eP3OOldic81XWHcAInVqb/zkspILsUDp0YMzINO2wXpEWhlMwP9S/m37kjA4LhaIDD/zUyur8DYeGjhvalELXIQPDkpHx7LBuVVQYwpma/7GcDvNZSMcUHh2G2zTMnZiPcOMt3JoL2t7cYc88c4q1e6niIz/iMwCQ5/4KN0JC1KYJxOC2M+ceAOgKPqquIIPvcIKSw3nkj7kLH2aOxbsDuOC/J8Qzlwz7PRc2GfjTlL3/REas96F4hlOwgP/PXxv1P3zJeHMQBnoRFfGvvxzcmfE/yVOEkv/zx2b9k8uQE2E/wqPhdGC9pefw8aRZ8ORnVOCYp4xPiYgTrmIy1acvX5MnVu3A8Joy//UtAhPG2CsyxiuHhRRvkcWF3r8LNwfZARginj6zX7338foKcO9WBtZQu+sjyyNM3+4B4PffEvobAPhHIuwbAPD3VMTvJ+yXP1GBP9Ub6B6nDvwTZ+k/fv6uUT//I2H8VyqQezmnwn796QDAH6gj/wIA/hclAPBHKuwPAOCfyYj1Pm2HfFAerxafb1QAAJLdPgDgExUWf/lf77qkhfI2W7E0KEZstAQAC9wxpCqDJ0v86VZ1QAd42wGIS+QjgG9Pu3yrcJaAPdAz6bQxDVivRggA+E2zNIBE2B81gM1hGkAq1QYAiQMRwH9/97tGfacBbA773XcIIBXxhyMAqSOvDKBjAjA5FUDry39WvspgQz3uch+4QaO8B6D7tK4LNgOoLSB/4puqjMVfFnMNoFoDiLm+BIDfXhfAbwYAPwTA/73CJoUEgORbglvThFBvq1nAxRQPcPYAnMEXkxqAIluvG+M+ob183wPwCVdd6NEKPskCfmsC8JsbtoA//+YB/Iq7skI9T+BMD2hIJLo1sQ+g3rer4HIL4PgZiQ32AHyXbPVarSSDMdW+BSzww2ywgETYZQD8+x0DyCZ8Uk07yrHw9bdzktx1IwTapdN3CCo3APr8fcrfltX8ucoC4u7VVTMk5E8vnC90a6XaXe6Jj/nrYAGbwwYL+BXAK6rZnzLE/mS9aIKeDL6dEV4doDBYYLBTFMrJ4XgPIynmoUu6s15twSlCEcVw4OZoaQe56D4pbrCAvzkAb0uDBRwAvKoGC3gCgHddB7wxDRbwBAAHC3g5DRZwAPCqGizgAOBVNVjADgD+fftj/XcNwL2AIwD/fnDEAOChBgvYAcAuFnAL2WABe2mwgBcCsNkCPiyA9PSAnhMHBgs4ALh/QNfpbNmKmsUymndMotJgAU8A8GH7Ad13MuqBsicKwPGiYxKVBgt4AoCPawE7T3rMXqiQ0YUAHCzgYwKYW16SYC6yMl8o5tqzQjGVomdBaDMb56976QwXg/BwqTCFi/t6xSzc1OxENsNdHrKVCpYx1PgKD7DNfOYUMpuFbgWgKjrPTB8sYB8AN03dOwaw4LxYoQ8W56+Fcp948c49dNHX7laTBToETgv+m7xpOgAADVNJREFUIlmJrqroYeXgF68VgQqiv0GEjPNgDBG0Q7WE6BbnixnuAQYAyqf3zi2RwQL2AXDD4V0DiNMxIbqeVxQgZNMRS1foR+9iBU6gX6DgCYsqAD32MkLwKif8CRa9TyMA0MOtuuDAvAIwxg/o7QqNkKen7i3hwQKeAOBdW0Bczx7dSt9wqsdzIJQquIsOzYu5bkHEeuGNgu8AXPJIrXtXqq1CfBunNDE2f9sHkCHDCizg4r3HAieDBWwH8O/HHO0BeBhaB/Ag9AYA1KcEjhBAxStJ9hwynFYOAOppmbi6yxZAGXA+jdbPa30BGW5oOOdNAHL+3GMBlw+alkmm2mta5r+N0zKbw4zTMr97jGmZJwMYry2g5KUGUGx3tcyecrR8AGCo85TtAYgZTasZzYqvn58JQGcza72LFr//ntBPACAV9j1OTKfC/jZhr38jIx4CSJ+CpX/9gdBfAcB/UIE4MZ0K+/chgH+hjvwFAPwDJZyYToX9CAD+i4x4CKBFLKEAhmo2ptZXmAKAGRWIE9OpsNnaAoYAii6Cx8gq7kWt+Agn9EILwtOl7GoEHxycOAItFHSwf6+87PWkt2C0BZBVNcZ9AOdwWd1d8gtqgv1kYrvk7PvJVJR0YMIWdMT6E1B0KisWTsaEJhlbUWHjqfKnZODBuxmQp5jGkk5lKi06bykb0RHrSwB6U0pw+RkZOA3ZakIGqpxOFQEc+yGfATlLbQAmefyGPXdTPdESe5FnPPTHWPLyFRwJXy955sx4lfOIL7U11B3RCOCIx9HLBkALAMQkNrM2719u439nB3Y+xaMJAHwGqHBu8FSX8B5U2TJsX8Q4HZgt0AzCASO9QeUTX5Vv8Cl8AxbXKeQv/Ak+h9iGTqFJLBec2yMbyIQvolfBZgj24uVhb+Ggc7RphAwadBUNAA66qm4UQNNGQF36FB3P+dAi/+r5M52kSwZ8z1gp/+Dk93R7DqlaTi3/taftxvWH37xVRR6bK+5dRmaE8qkkHPr+HuavUTp/56ikMZBdNmO0zTt1xobdsDtsweS2JL+nGwWwtkOIrHWUiINdWvKG5+2WPiRg2gehy/2JwzHFkmGTo8P8NanK3xmSpr1S7XbrIy3fdAdM10BsVH+YfPtNqHSbAIraHkn1jU+8+r5Esul+SAt3cozpm9BpU/vSGlH7lTfu4dSYv0ZV+TtDppP4VvtGMSKmdwKukidNYG61m3hVGpPf120C6FjW3lZncW3rp7y+x5rTZIyE5UVICb3ZaIeHJGNrQRxUz19deYc94HT+zlHUsD3dWriVYav5cSLfZCZLejdq17Lad+Jy8vYNM9e6SQDdyC+drYkStS3z3NiPvT2wyqYt15wc3kCHfkhISetdhPPOuuTvIOwgf43S+TtDsowiagcbFUW5076Za/MmR5UEJnFO8qVTdt2n6TYBdPf3evTjeG8TVunKfeMFlETHteJYlR7uokrdBtyIM29rzTmRHXTJX10H+WuWzt8ZkobdWCVTrc0EaSnTXoYu88hX13TqjsnXdJMAgvY25BXyYNvk/f3qBW7KdggDbicvpIAHQWAihGrfzTTyUnpvddOG5GVb8aPz13Z6sxo2ad9KtlZCndgUn5k3e+2QfNmS/J5uFcDaLmQHT1vVTIxzXOEpfX0DjZtKtu6VKmw5sslQ04bkqs0ErvN3jlwTBU6LjYLILXVQ1/SC+C0NKCDU6VzFvVUAa1dwuPFjbRvv4wIHAKgamaYNI1ubobkvJnRFzXiHiW3GN9rm7xyZ3gDZ8nYBoKboKJMVbzOBAGhrIbDVrQIo9q3XYYW5to/rkbXHjUarKpappt92F4UtnAn9HIXJujbvM3ucv3NkrIl5RhsM2XNNVQidgukNMW8IDHdOtiW/060CWHtERyDt34Gj0iIqN3gought3VY59pm9MFX0TAgZ097l7xyZETJthi0tv70hYUaoNCbvmPGt62YB3K/HHFdY8r3O0IPSxIeHv+7mN1aUWGTqLfMtV6apoaJmrmfldGftXv7OUm4q5yXdCe8iPe1FpLGMlnQnlsTk4+7enzcLoLtXjIkjS+aWuydcr4758Opu45ofEj2awTxIxZmZ3LhdYzG7n7+6avk7R8poRcnBNMiZ28UCm7sJBEWgxOTNWavrZgHctxKyoU2Wbx/ifiVQllj4ba2T+UYLm+LTg8Rda2YcUGixYnkjZAf5O0tmKybjRocCYUVwt/IOI7VmKyatsunWChs3Wo969LLfLoBy7wk2Vbgc269uwV5B69nomuDuGoHmuwhmouk1dx0cKfayxGinZIsZ2+ZvT0f5O0fK3Nfm5vaRy5fra0cF0aWN4Jl7iiD5o6uQVfJ9DOANA7jf3dToWyFKyxd7odKLqye7Z2BauuRE3NBTA+lisnFWmO9jW3fYNn9rNeXvLLX09sDFx7VBM+lYlVVsi1iprSIHyXv7yQvHKnskv9YNA7jXm6WaXdxUaceOkkCZVE5kx9XxtU6Qtpvh2FHdGgjf1l6Anp2Z2tCsvbdtkz/hMpfM3zlqN2RebJeekpABoZzYXr9t5k6arVRb8i4kH+0lX/ZKfq0bBhAY2FxJUyWwCvDy2LJnmRXvBufLfXPZehcBuNJZOzhL4ZVrIKUVZ20vstde1FT5s20yf2epzQYzNEvRYQbaX5y18j7J594m+S6ebjvdMoBs51ZrqHC7UvqW3NVXDvrg2h+S9CJ8PlFUWmCwxCZanLUaqk5uv5A/EJm/c+S2u/Q0ZMDUR1iTqZvgAslXumkA3e0dMPds7rcH1EEB53a5i1CEeJ6HheXmfJYTto+nd0r7QIf5O0unlOY9XoBT8ur3HOW+aQB3jsstYzu7LhFx5I9+yl2EYiQKO1iXTo7VBzEuOkPf6z7mtY3R43acknzPGLcN4O52mQfPt3w2dZD2L/Pc2Fd2pwfV63EyQwfuqfJ7+nWpfq/MByfPbh5AaKVumlbGw9Y1PdnY/28YF2uUG5WsbPerr+Wvm5rzd5byXoiovgY47/X29k7+9gHcTMIyD+syVx9FjT9FvQh0y5h5cdfhzC6TxDbqMmGut/wehV6XCXOHyfcA/ITkbx/AzVvVMvKF/fYeObTWh0AcxoXmZec3v/tbT+fvLDmdX4HuR+7J7xyp+5F7un0AcfhStve6lr7h+t3uBZWIS8nyqIe/SpW/dp30fLqoI9gy6t2k6Jf8SdMM7gBAJvWwo9GzhaHviun6u778ys7RnHafVLPNX1vClmG1gTMl4/apyMBRt/fkWKJb8o3j6q26BwCRiFIZ5/GI3LLNHVCqi52S2kpJy+s+p2GbP+MBIrfJZT4uIcdumeSnyjM8IFzf9j8q+fsAEMnI6ZJA+DiA1jLEBUVEW2XNq6xUmbd5ujTlL6KjVPn7UEH5amBERQ2uOX0kPiz5OwEQH3G4bLxKdzN+a3DTraQsyzOYIbV2VvHjUzyWpX/ge3KUvw8WMBI1Xp704jPx2ySvGpN3YuuM5O8GQOxqKazIUXvDjug9kNvx5t1sHZdwPcsiZvxLB4J0yo4lOvnLHSeBpJny99ESTmzn3v7KceiFU9pl60oGnSR9InnvnOTvCEAcHIfnacWR76D8vLTQk2X3wNtH3bQ58g6BEB76SlW3Ecc2+nj01hNyjPn7cLnCiWyrzNcZQE8VcgmPUySAt/3kbZtcwqOr7gnAypNTKM/PUb7jHT7cLiNjWGRYke8pgVLQ3rCsrROM7lfu4GZlkCl/nyFXeM4mA+rytncvee8Syd8VgDhX0qxuvS2ucvwSMATFZQ6l5jYE+evp0DboLN0XgO3uRz1GxlyQrJsoHbunQ9ugs3RfAEILoa1QO2E8fCsf+T6vAB7UU3cGYLs/u1tNPDxBssRewP4ufoPO0b0B2MFNXEYdRo6OVQ2VuK3VzEEX1b0B2MnB2envdrKeMcuictjR6VN1dwACXe2N/07D5/taD8JdcsbQoE66PwCZ38W572i+r0liM0zcfw7EoDN1hwB2KybRfaEbTbuh9P4zGgadq3sEkMWdKmoitzus1e7tFtAY+LuC7hJAvcRYB6GHgW8YDnOVb+9cWAb+rqG7BJDJsmNj1VW5beVH3gdMj2nmlu3vPIz6zC4adDHdJ4CGBSCPD1W+ZVklDp3rRSQkeiD4pWXF/r5/28DfdXSnAOICkH1mbSjtm2TZ+j90mPJUHWD/kitmDOquuwXwhDmGcq3j0rvHrLlBl9X9AnjBabZ6Kuagq+iOAcT1TS9it04YuRt0Kd0zgDiAe37L4UTfhUGX0V0DiC4shr3rO8npuK7BoI/RnQPouv5ZM75FfJkV6wedqjsHkFWT8k9EUDTsZDDoc3X/AKIvlXUKR6Kzu8Kgj9MjAFgh2LMmp/JPWa9gUIseA0B0amlbP2dfuGfMZ61XMMioRwFQl6hxp2n6EpfLOGs5iUGX0+MAiOtuRLblKyNawouswfjdkB4JQKZdX2KA0FPHA764nRQunHLJtVIGna0HA5Dhkk2eH1nVGjqepxRuQePrdXri2iocg25CjweglivR6c/PoxKk1wlS11goaFCr/j/2lR9rF+C33gAAAABJRU5ErkJggg==)_____no_output_____### Reversible Layers_____no_output_____---

Using Locality Sensitive Hashing, we were able to solve the problem of computation but still we have a memory issue. Reformer implements a novel approach to solve this problem, by recomputing the input of each layer on-demand during back-propagation, rather than storing it in memory. This is accomplished by using Reversible Layers (*activations from last layers are used to recover activations from any intermediate layer*).

Reversible layers store two sets of activations for each layer.

- One follows the standard procedure in which the activations are added as they pass through the network

- The other set only captures the changes. Thus, if we run the network in reverse, we simply subtract the activations applied at each layer._____no_output_____![Reformer Reversible.png](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAgAAAAFjCAIAAADEkiTEAAAAA3NCSVQICAjb4U/gAAAAX3pUWHRSYXcgcHJvZmlsZSB0eXBlIEFQUDEAAAiZ40pPzUstykxWKCjKT8vMSeVSAANjEy4TSxNLo0QDAwMLAwgwNDAwNgSSRkC2OVQo0QAFmJibpQGhuVmymSmIzwUAT7oVaBst2IwAACAASURBVHic7N13WBRHGwDwd3b37kBAQKoUFRURG/aOgqKxYTcmMfaYGHtvUWMv2NDYTawxiahorNH4YcUCdrCggAgIiNJE6m35/ph4IYgICNzBvr8nT567vd2992R23tmZ2V0iSRIghBCSH0bbASCEENIOTAAIISRTmAAQQkimMAEghJBMYQJACCGZwgSAEEIyhQkAIYRkChMAQgjJFCYAhBCSKUwACCEkU5gAEEJIpjhtB4AK4cGDB0ePHmUYHU3bkiS5urq6urpqOxBUgiIjI/fu3cuyrLYDyZskSc2bN/fw8NB2IGUDJoCy5PXr13Pnzt2xY4dardZ2LLlxHLdx40Y7OztMAOXbmzdv5s2bt2nTJkKItmPJjeM4Hx+fihUrYgIoIEwAZQlt+3/++edZWVnajiU3pVIZEhKig5UCKl70T9yvXz+WZXXtXsIKhSIhIQELYcFhAih7eJ7neV7bUeRGCBEEQdtRoFLC87wkSbqWAAghOnho6DId7U1GCCFU0vAMoDwghCgUCp7nRVEs6e9iWZZhGEEQSuG7UBlSaoWQYRjNEDQ9ESnRryvf8AygzCOEZGVlmZubBwUFlfTcDI7jgoKCli5d+vz5c52djIRKHyEkMTHR3Nz8xYsXJVowGIZ5/vz5li1bli1btnPnztTUVCyHnwL/7coDOurFcXmczzEMwzCMZliMvs25oeYtfZ3np3QPSqXy+++/nzZtmre3d2pqKg61oZxoecizCcK8k/Ntzg1zvs1VYnMWQoZhEhMTW7RoYWtr6+rqevv27Vq1amVkZGBRLDJMAOUWPXKePXv29OnT5ORkhULBMExYWFhcXBw9ogghKSkpT548oefUoiiGhoaGhYWxLEsIIYRkZGSEhoYqFIoXL14kJSUJgjBt2rQzZ85o+5ehskShUERHRz99+jQuLo4WwhcvXmjOIGkxCwkJEQSBYRiO42iJ5XmelsM3b948e/ZMpVJFRESkpqZWqlQpMjKyZ8+ebdu2Xbt2LQAEBATo7EUJug8TgI5KSkpauXJlbGxs0Tanx5W1tbWXl9eWLVucnZ39/PxUKtX169cbNmxIW0wKhWL16tW///67UqmMioqytbXdvHnz7NmzraysMjMzaWurbdu2Bw4caNKkyd27dwkhDg4OONVHPpKTk+fOnRsdHV3kPbAsa2VlNXHixN27dzds2HDr1q1KpTImJqZFixa05c6y7NmzZ9u3b69UKtPS0iwtLefNm7dx40Z7e/uYmBiO454/fz506NCtW7e2bNny2bNndKRBrVbzPK9Sqdq3b5+amlqMP1lucBC4lAQHB0dFRRVwZdoAnzVr1qxZs77//vthw4Y1atRIoVAU/OskSapQoUJoaGilSpVYlh00aFD37t2jo6M9PT2nTp0aHh5evXr11NTUn3/+2d/fn+f5pk2bHjlypEOHDqIozp07d+/evePGjaNttGvXrj179kypVBZ8gh3P86dOnSpUwJofLghC9+7dC7shKojHjx8/e/as4Ovr6ektXbp06dKlAwYMmDRpUpMmTVQqVaG+URTFp0+fmpiYsCw7YcIEFxeXPn36NGvWDABu3LjRoUMHhmFGjx69f/9+PT29yZMnr1q1atSoUYQQd3f3adOm+fj4cBz35MmTpKSkyMhIjuM07Q+GYV69enXx4sW1a9d+aNj59OnThYqWooWwU6dOSqWyCJuXLZgASsnFixfHjRs3atSoglSjhJAKFSrY2tq+ePFiy5YtDg4O1apVs7S0LNQ3SpJkamqampoaExNDZ0qkp6ebmZl9+eWX586dGzduXEBAAAA4OTkFBwcDQP369V++fMkwTKdOnXr37j127Fh6ojB27FgDA4NCTa+WJOnAgQMqlapQnbOEkNTUVB8fH5zXUUICAgKGDh1a8EKop6dXvXr18PDwgwcP1q5du2bNmkUohMbGxllZWVFRUZmZmQDw+vVrGxubdevWrV+/vkuXLuHh4QDQsmXLuLi43377bfLkya9evQKARo0ajR49OjMzkxahESNGqFSqnGefCoVi8eLF48ePr1KlyocujD948KAkSYUaJSaEvH379sCBA2/evMEEgIoNIWTLli2jR48u4PrJyckbN248d+5c+/bt8xzd/ejXCYKwevXq5cuXT5gwwdbWli4XBGHEiBGdOnX67rvvfH19t2/fzjAMPYl2cXFJS0sDAFtbWw8Pj+zsbLqJubl5Ybt9FArF/v37CxszACQlJfn4+BRhQ1QQhJCFCxfOnz+/gOsLgsBx3LFjxzp37lzYtj/FsqyPj8+YMWMGDRrUokULeHetVufOnSdPnhwTE3P9+vXvvvuuUqVKERERAODp6RkXFwcA9Mw1PT2d7qdSpUo5C6FKpdqyZYuPj09EREQ+yWznzp1FiDklJeXAgQNF2LAswgRQegp1Ax+VSvUpDWGWZY8ePXru3Lm4uDilUpmUlDRz5kwAEEWxXr16AHDjxo1du3ZNmzZNEAQ6hvbixYucx5haraYBlGZ7XAfvcVTOFOpMjl7tUeQ5NizL3r17d8yYMbQrkuf5CRMm0OuHK1eu3L9//6tXr+7bt2/RokU8z2v6G01MTDRdOoIg0I7TnIVQqVRu37597ty54eHhHMcV+2UHmqaPHOAgsI7S19cv7CYKhUKhUCjfCQkJcXJy0tfXJ4ScP38eAAghkiRxHLdp0yZPT8+RI0daWVnxPF+3bl0AOHjwIF1BFMW3b98W/09CZY1SqSxs7Z+zBNL5PwBgZGRECAkKCoJ397NSq9UjR4785ptvrl271qBBA57nbWxsmjVrtnXrVtrykCQpLS3t/cYHrf0PHz4cHR1tYmLCcVwRhpqQRt4JYOvWrZGRkXl+dPLkycDAwJIMCRUaPU7atm1bqVIlc3Nzc3PzYcOGff/997/++quxsbGpqSlt1Lx9+5Z2Dbm7uwPAwIEDBUGQJElPT+/mzZsLFy6sVKlSy5YtLSws6NQ62rbKdRDStzgXCOVCC0bdunU1hdDLy6tLly6tW7e2srIyNTU9f/589erVk5OTAUAUxSZNmgDA1q1bGYaRJEkQhP379z99+tTS0rJ169bm5ub79u3LOeoLABzH/fnnn3PmzLl586adnR39on79+smhs76E5NEF9ODBg5MnT36ot9rV1dXY2FitVhehYxqVBEmSlErlo0ePci5kWdbY2DgqKio1NVWlUhkbG7u5uRkYGNCV79+/DwANGzakHQKCINjZ2d26dSspKUkURZVKZWRkpFarraysHjx4YGRklDMHEEIePXpUoUIFvBUE0qCDvQ8fPsx5xqBQKFiWPXz4cFJSEsuy5ubmgwcPpp02LMuGhYUBgIeHBy2EoigaGRlt27Zt+fLldOzBxMQkOzu7Vq1aDx480DQ73NzcchV1hmGw57DI8qjER48evW3btg9tULFiRS8vr7/++qtHjx4lGRgqHBMTk1xL6FFkamoKANnZ2fQFy7K3bt364osvLly4kLNap7W5sbExfUuPSZZlK1Wq9P5pOP0unKuDcmIYhpaxnGi5osuzsrIqVqxI13zx4kWrVq127txJW5N0ZUmS1Go1XQfeFUKlUqkphHRyc4UKFXJ9CxbFIsudAJKTk69cueLs7EzfJiYmPn36VBRFBwcHa2trutDT07N79+6YAHRKnsdAzoWSJNHav2vXrgcPHqxbt+777ab3d/LR3SKk8aGCoVlOJ2XGx8c3atRo1apVnp6eHy2E+b9Fnyj3GEBYWFjXrl3peVx0dLSZmdnVq1fPnTtXuXJlTdd/tWrVwsPDNTO0UFkhiqKzs/OjR4/c3NzwrBlphSiKlSpVunfv3vDhw7EQal3uM4DIyMhu3brR13Z2dpp826RJk+bNm9O3enp6AJCRkfH+uRgqBbnupVUo+vr6+vr69L4ruhMVKnPon7vI7fHKlStrpn4We1TFu8/yLXcCSE9P13QEU9HR0TExMUlJSbnWxHMxbYmMjNTBphPHcYmJiXhfRpmIjo7WwaqW47iEhIT3x8PQh+ROAPr6+vSSPABITEzs2rUrHbt//yG0eKiXPlrvu7i4aDuQD8Inwpd7dF5m8+bNtR3IB9G7hKKCyJ0AHBwc1q9fP2XKFACYM2fO119/PX78eACgc7YoepWQoaFhKcaJAADc3d11fAI+NgvKvfr162MhLDdyJwBHR8dLly6p1WqFQnHv3j1HR0cAEARh+/btmnVCQ0O7detWtHuDoE9B7xKq7SiQrJX7Qkh/Xfn+jRq5e/EMDQ0HDhx49+5dAPDx8Zk2bRohpFKlSh4eHvCuC8LHx2fq1KmlHysq35KTk+lRJ0kS3osCaUViYqLmorOUlBRth1Pi8hjGWb58Oe3gs7e3lySJ5/mUlJROnTrxPK9QKGJjY5cvX96hQ4dSDxWVc3fu3KF3G2YYRg7HHtJBUVFRtBCamJjEx8drO5wSl0cCcHBwOH36tOYxQJrHrdEXYWFhoaGhpRYfkg96hyIAmDRpkub+1QiVJhcXF/q8ml69etEO8PIt74lcXbp0sbOzy/Ojtm3b1qhRoyRDQvL1xx9/AMDIkSO1HQiSr+XLlwMAvX16uadzM3mRnHXp0gUA6BMLENIKNzc30O15rsUIEwDSIcbGxtjBiLSLZdmIiAhN13f5hgkA6RbsYERaV7VqVW2HUEowASCEkEzhQ12QThBF8fXr1y/j4oAQa2trMzMzHbzVDCrfJElKSEiIjY2VJMnS0tLCwqLcdwThMYa07NWrV96rV7Msa2Vl1cDFpUGDBpaWlizLbvD2Tk1N1XZ0SBbS09O3bt7EMIyFhUWDBg1cXFwqV67McdzqFSvi4uK0HV0JwgSAtOnPo0ctLS3nhoYNunVnckbWVLUwlRcnpWUMunl74q07FStWPHXypLZjROXcub//NjAw+P6c35cBNye+TZ/Ki1N5cXJm9uB791fFvaxcufIfv/+u7RhLCiYApDWrvLx69+kz5O79cVu32DRqyCgUQAgAsCqVTeNGs/bt+epGYPcePX5av17bkaJya9fOnZ06dx54xX+m72G7pk0Uenp0OcNx1vXrD/deNyz4wZdffbVg/nztxllCcAwAaceJEydmzJw5Ji7ewMqC5/95vjzDMQAg8qIoSCKAXfOmo549n+BQ1blOHY9OnbQaLyqHrly5MmLkyJFPQs0ca3yoEFrUrTMuIWmhmWndevUGfP65VuMtfngGgLQgNTXV09NzUOAtQysL8d2BB4QE7//t3u499AgEAIEXTatVGXjlaqfOnfERpKh4ZWVlubq69v37nJljDSFH7R/8+x/39+zVrCbyon4lk6FBwZ8PHPj69WstBVtSylIC4Hk+LS1N85beM1IUxXw2Qbrp1z17YPAQ26aNNQceABAA/+XLzg4flrNQCrzo0KYVeHQ65ONT+nHmIy0tLedt8dVqdUZGhhbjQYV14vhxaOBS06NjrkIYsG7dmWFD370DABB50apeXauJk3bt2KGNSAsqPT09I4eCPDewLCUAADA0NNyzZw99PWfOnFGjRpWhyYIyucN4ToIg5MzZGmPGj/9izNh/nylKCP1P39ycpc/zo0voTgD6zfnBe/jwUgkZAKAg9yLt3LnzokWLNG+bNGmS85kZSHeIopjn3cV/mjfXc/mKXIWQAOiZmFSoWRPoUw/eFUJJgs7Dh8+YM6cUAk5OTi7CVllZWQYGBhVy+Pvvvz+6VZmpPQGA47jw8PBhw4a9evXq4cOHK1as2L17t7aDKiiWZcv9nOL3sSzbp0+fDRs25LyzbmJiIgBUql5dEgEACCGSIEiiKIkCIYTR05cAJFGQ3rWvJRGs6ta9A5CdnV06Ye/Zs2fSpElPnz7NZ50NGzYsWrSI53kASExMDAoKGjBgQOmEV2Qcx3Gc7Ib9GIaZMGHCihUrNHc4BgBJki4+DrFp0EB61/qXBEESBREAGMJwnCQIoihKgvDPMypEybhKVQCIjY0t6YBXrlw5f/788PDwQm2lUqmkdzZv3tygQYNu3bp9dCtS5p7t/ssvv5w6deru3bs+Pj5NmjQp4FapqakdO3a0trbWSjOcZdkjR44AQN++fWmVIRP0ARJXr14FgN69ey9btszZ2fnly5fW1tbTRUkURMIysTdv7m/eLM/NR4Q8qVTTUZIkMTt7XQW9qKgoOzu7W7dujRs3jt60vSQwDJORkXHmzBkA0NPT8/X17dq16/urSZLEMMyVK1fatGlz7NixtWvXXrhw4UP7TE5O1vptdFmWPXr0qCRJciuELMsmJCRcunQJANq2bbtmzZrmzZvHx8dbWVmNT36jMjQEQlJfRG+rYp/n5gMvXqri6ioKImGY1Sx5HhFRpWrV8PDwzz//vCRuWs4wjFqtPnnyJADUr19/69atrVu3LtQebty40bJly6SkJBN6Mp2vstccGDly5DfffNO3b9+C1/4AoFAoli1bpsXmz6RJkwBAbiMWSqVy27ZtV69e7dGjx6RJk6pVqwbvusJEQQRCQIKKdvZuGzZyeiogxH/Z0oznzzv/slNUq/nMLFVF4392RKeHsiwAWFtbL1mypOROpziOCwgIOHPmDMuy+/fvb9myZZ6rEUJ27NixadOmNm3aeHl5LV68OJ996unpeXt7a731Lc9CqFAo/vzzz0uXLrVu3Xru3Ll169aFd2VJ07+jqGDgtuEnTk+P4RT+K5ZlxMV1WL0GAPisLBOH6pIEQIgk/nPGCgCmpqYrV64siULIcZyfn9/Jkydr1669bNmywt4ZNyUlpWXLlleuXClI7Q9l8Qzg4MGDPj4+hw4dunPnTsOGDbUdDsqPJEk//vjj8OHDHRwcNAuTk5NNTU1Hv4g1tLaWRJEw/47j7O3QMeHhg2lxcbT3RxSBrkAbaGq1unTq0B07dri4uHz0hsAJCQnm5uZPnz51dHTMzs5WKBSlEBsqggULFnzxxRe1a9fOuZAQ8k1omIlDdUkUCSEMSwCABdjXtVtyxLPvHz2i6UFTCDMSEzdZmL18+bLkzj6pJUuW9OzZs0GDBkXYtnPnzq1bt16wYEEB1y9jZwAxMTGff/55YmLid99916hRI57nZdixXoYQQnKOlFImJiZ1AeIePHC0sZZEkETa8wqEIZLAC5mZIsB/JmYwEH33Xm93t1JrQY8aNaogq5mZmQ0ePNjZ2XnFihVY++uyPCvEof37R9y81bhGdUEESZIEXgIA4BhJEEReLfIiAAH4p31MGHj1+BEAlHTtDwBz584t2oY7d+58/Pjx2bNnC75JWRoEFgTB1tZ2586dpqamHh4ebm5u8+bN03ZQqCgWHzp0dPKk3EslSImMFPOagXOyZ4+Rk6eURmSFNGnSJJ7n+/fvr+1AUKENGT363Bef89k8/HdcMDU2Jj00LNfKBMBn8eJf9+4FXfX06dORI0d26dJlzZo1K1asWLRo0bVr1z66VVnqAkpKSgoPD2/UqBHtMkhOTg4LC6tfv75SqdR2aKhwBEHgOK7n8RPOPbprrsAkhEReuUIIqeLa9p+TAgCOY+75HPxr4Oe6ebZ3/PjxdevW+fn5aTsQVBQGhLT4ZWfzEcP/vQyYZaKvX+czMqq0d4N3dSPLMWH/8zvs0TE9PV1fX1978eYnKSkpKiqKEELHeERRtLKysrGxyX+rspQAUHly9+7dRo0aDQoItGvWlBckerAxLAN0fBgACOFYEnHF/4Br2wcPHtSpU0e7Ab/vyZMnTk5OT58+rVmzprZjQUURGhrq6OjY79z/anTsIOQshAT+uUCdEI4lcUEP9jSod+3atQ/NCCi7ylIXECpPGjZsePLEif3Nm90/eJgQwnIMYRlJAkkCwjIsxxBCbu379YBr24sXLuhg7X/mzBknJ6fTp09j7V921axZMyAg4LBHx4CffwFJ+rcQikBYhuEYhiEP/zy+p0G9Q4cOlb/aH/AMAGnXo0ePaOXu8fuBKk2aKA0NCGEyU5Kf37rlN+grAHj8+LGTk5O2w8xDTEwMz/NVqlTRdiDoU4WFhbWqWfMVgNuefdWaN9M3MZEkKTst7cX9+2f69QWAcjzhEBMA0j6///1v37Ztuw8e1CwZPnDg0NGj27u5aS8oJC9X/f13b9myY/9+zZIvPT2HjR3r0alTGbrfTGFhAkC6Iisri97sQaVS4cA+0gq1Wp2VlSVJklKpVKlU2g6nxGECQAghmSq3pzYIIYTyhwkAIYRkChMAQgjJFCYAhBCSKUwACCEkU5gAEEJIpjABIISQTGECQAghmcIEgBBCMoUJACGEZAoTAEIIyRQmAIQQkilMAAghJFOYABBCSKYwASCEkExhAkAIIZnCBIAQQjKFCQAhhGQKEwBCCMkUJgCEEJIpTAAIISRTmAAQQkimMAEghJBMYQJACCGZwgSAEEIyhQkAIYRkChMAQgjJFCYAhBCSKUwACCEkU5gAEEJIpjABIISQTGECQAghmcIEgBBCMoUJACGEZAoTAEIIyRQmAIQQkilMAAghJFOYABBCSKYwASCEkExhAkAIIZnCBIAQQjKFCQAhhGQKEwBCCMkUJgCEEJIpTAAIISRTmAAQQkimMAEghJBMYQJACCGZwgSAEEIyhQkAIYRkChMAQgjJFCYAhBCSKUwACCEkU5gAEEJIpjhtB4AQQjokIyNDrVYDAMMwhoaG2g6nZOEZAEIIAQCc9/Mb2L1bhQoVjI2NjY2NjYyMWlla7Nm1SxAEbYdWUogkSdqOASGEtCk8PLxGjRoA0PngIftGjQ2trVhOkfbq1evQ0MOrvODUyb9On/6sSxdth1n8MAEghGTN39+/bdu2HXbtbjx4CMMSUQSQJAmAEEIYIABR12/83qrlmlWrpkybpu1gi1nZTgD37t0LCgoaOHCgQqHQdixI7m7cuPH06dOvv/5a24GgQoiMjKxatWq/v8/V9OjICxJIEgAwLAMERF6k67AckxoXv6Wyle/hw3369tVqvIVw+vRplUrVoUOHfNYpqwkgOzu7U6dOly5dAoCMjAw9PT1tR4TkKyUlxcTEhL4uoweUbNUgxHbLttajv+XfVfcMyzy/fDk5PLz+14M1qzEc8/J+8F6X+i9fvrS0tNRSsAUVEhJSu3ZtABg1atT27dvzWVPXB4F5nk9MTOR5nr4VRZG+ZVl21apVCQkJ2g0PyUpSUlJmZqbmbVZWVnJyMgCoVKrbt2+HhYVpLzRUFDeuXw8HaD5yJC/8m7YJgYhz584OHwaSBITQhSIvVm5Qr8q06Tu2bNFSsHlLTk5OySEjIwMAjIyMoqKiTp8+/dHmiK4nAI7jmjVrtmTJEvp2w4YNbdu25TiOZdnmzZuX49F5pIMWLVo0aNAgzdspU6YsXboUAPT09Bo1akTnDqIy5OD+/W2272AVLPy3omSVyvdXFiRwGzp07oIF2dnZpRXgx5mamprk4OPjAwA2NjZ2dnY0GeRP1xMAANy+fXvhwoUPHjyIioqaPHnytWvXNB/h6TYqTZMmTfL19aXnnenp6Zs3b/7222+1HRQqkNmzZ/v7++dcIgjCmo0ba7RqLdJahBCWY1iOYQEIxwEAo2BZlrAcQxgGACRRsnSuAwAvX74s6WhXr17t5+cniuJH15TeOXDgAMuygwcP/ugmOZWBC8GMjY3Pnj3bo0cPBwcHPz8/Y2NjbUeEypX09PSFCxeOHj3awcEh/zWrVq3aoEGDixcv9u3b986dOwDg6OhY8C8SRdHX11eZV+sSlSiO41QqVdu2bQFg//79HTt2tLKyomdsesbGIAEQwmdkXFqxXGFoyHDcs3N/A8DVFSsBgM/MrDNwoJlTbUkUGY4BgNjYWHt7+6SkJD8/v5KYfsKyrCAIHTt2BIBffvmlS5cuNjY2+W/y4MGDgQMHvnjxgmEK16YvAwkAADp16pSenn7+/Hk/P7+i7SEtLe3o0aNaPPboH6YgKR2VMqVS6ePj4+Xl1bVr1+nTp7dq1SqfOQUrV67s2rWrJEm//fbboUOHCvVFkiRdvnwZWzClj2XZ2NhY+trX19fR0dHKyoq+pZ38hJDs1NTApUtybuU/exZ9Yd/W1dy5tqQ5diUJANLS0q5evWpgYFDs0TIMExMTQ18fPHiwevXq+SeA9PT0evXqnTx58qN54n1lYxbQmTNntm3bduTIkZMnT3br1k2zPD4+3srKqiCzgNLT04cPH66tY4/juNOnT4ui2L17d82ANtIRenp6x48fj4iIAIDFixd/88031tbWH1o5KytLT0/v0aNHzs7OiYmJpqammo/o1IsycUDJ0MyZM62trQcNGqSZwyMIAsdxQ4IeWNatIwkiIYSwBABYgEvLV1ydM3u6JBEACUASQRJFIIQQspolERERVatWLdFovb29MzIyhgwZYmtr+9GVR44cybLs+7N9jhw5curUqR07duSzbRk4A0hOTu7SpUtERMTixYvr1auXlJSkmXJXcBUqVDhw4EBJhFdA27dvF0Vx9OjRWowB5UmtVp88efLUqVPu7u4fbUmoVKoVK1Y4OztPnjw5Z+2PdNz8+fNztdZZlh33zTf3bwZa16sj0M50XgIAwjGSIACAyItAiGZ8mDAk4ckTAMinfVBcRo8eXcCp7b6+vjt37ixys6MMJIA2bdp4eXnRlDtlypRevXpdvHiRfkR/dploc/E8j3OWdBPDMKGhoeTdhL+P+uKLL2bNmvX+aBv+fXVZnn01XwwdutHVtfmQoTkregAQ6ISuHNNAAYAlcHnv3nmzZ6tUqpKOtoC1/8uXL/v16+fp6blgwYLs7Gy1Wt2uXTtPT0/6qVqtTkpKyn8Pup4A0tLSfHx8nJyc6NslS5aEhYWlpaXRP6epqWlwcHAp/D1QOcaybKHWf/78OQA0atQo1/KqVasGBwcXW1io5LVp2xYA7vy6r+mQwTz/TwKQAOzbulb4aSMAaLICwzHxj5+ELlv6XXS0loLNg5GR0ePHjwkhdHBRkqSKZcl0cQAAIABJREFUFStqPu3QoUPz5s3z30PZGAMoBzZv3iwIwvjx47UdCPok9K5hd+/edXFx0XYsqBg8efLEycnpiyv+Vdu0/vdiYI4B+PdWEAzHpMW/2mxluWf37iFDh2ot1hJQBq4DQEhHXL9+vUaNGgcOHMDav9yoVavWhQsX/mjbJuiwL8sxDMcQhhEFSRQkQgjDMgqOiQ9+sNnKct4PP5Sz2h90vwsIId1Ru3btUpgBgkpZ+/btg4KC6tevfwqg15mzNg0aVDC3AIDs9PT4x4/+2LYNdv7i4+MzYMAAbUda/DABIFRQ9Gp7bUeBil+9evXUavWxP//cMn3qlvtBOT/6ydt7YHy8hYWFtmIrUZgAEEIIOI7r269f3379MjMzX758yfO8paWloaFhwaeHlUWYABBC6F96enry6eXDQWCEEJIpTAAIIZTbkydPtB1CacAEgBBC/yGKopOTU3p6urYDKXGYABBC6D/oQ0fOnj2r7UBKHCYAhBD6jxUrVgBAnz59tB1IicMEgBBC/woJCTlx4gR9ffXqVe0GU9JwGihCCP2rcuXKWVlZKpUqPj6+gHflLLvwDAAhhP5VsWJFenNNQRCMjIy0HU7JwgSAEEL/IZ97JGMCQAghmcIEgBBCMoWDwGWMjp+clu87ZyFKxwshYDksMDwDKEtu3LjB6LYDBw5o+x8Jlazg4GBtl7KP2LJli7b/kcoMPAMoS7KysgDgypUravrQal3CcdzGjRt1MDBUvGjj+n//+x/D6FzzkeO43377TdtRlCWYAMoSeuzVq1ePZgKdolAobG1tdb9zABULZ2dnlmW1HUVuCoXCzs4OC2HBYQIoewRBEARB21HkxjAMnT2N5EAURUKIrlW1WAgLK++TuH79+kVFRUVERBBC3j+pz8jIIIRkZ2eXfHiooFiWLbWBLxxhQ+8jhJRaISSEYCEsFnkkgIsXL9aqVcve3v5D/bn6+vq+vr4//fRTCceGCkoUxdu3b79586akjwqGYTiOU6vVutb0Q9pFCMnKyrp58yZtHZboFykUClEUeZ6XJInjsA/jk+SRANzc3L777jvIt6HXrVu3adOm6WBPtAwRQnie79q1a3R0dImOyzEMExcXt2DBgipVqjx48EAHu4CRFqWlpXXv3j05ObnkEgDtkNi0aZOtra29vb2tre358+cVCkUJfZ0c5K4vXr58CQDVqlX7dw2GuXDhwu7du8+dO6dZqFKpRowYcf/+/VIJEn0EPeTer/1ZllUoFAqFguM4ug7HcTkbTXSFXCtr9kNbW/T/LMtyHDds2LA2bdqwLIu1P8rlQ4WQYRjFO3SdnKVOs4Jmc7qmpoBpPqUlk3ZLPHnyJDo6+uDBgwMHDnz+/LkOzkcqK3L/w4WGhn7xxRc5lwwYMCA8PNzOzq5Tp06fffaZZnmnTp0CAgJKI0ZZKpZh3lu3bq1fv97Ly+vIkSOCICgUioMHDx49epQefizLBgYGbt68mR5yt2/fXr169Zo1a0JDQ+kh9/r16w0bNmRkZGzduvXevXuCIJw9e9bDw0MQBOwCkglRFD9lWJVhmCdPnmzbts3Ly2vPnj1paWkKheLq1aubNm2iDRGGYZ4/f75y5crs7GyFQvHs2TNvb28vL6/AwECO4xiGefbs2c6dO9PS0tauXRseHm5oaDhmzBgjIyOWZd3c3OrXr3/r1i1MAEWW+x8uNja2Q4cOOZeMHTt2xIgRHh4e6enpZ8+epc/KAQBLS8ukpKRSCrNcKNS5ampqqq2t7dmzZ4s2s56O0h87dqxVq1YeHh6//vrrkiVLWJa1sbH59ttv09LS6DDarFmzLCwsOI47fPhw165dO3bs2LRp07Zt24aEhLAsm52dvXjx4nHjxkVFRSmVSkmSdHMCEio5DMNUq1bt8OHDqampRdvDn3/+6ezs/Nlnn4WHh9esWVOSJHt7+wULFtAeS47jDh06lJSUVLFixTt37rRq1app06YdO3bs0aPH33//rVAo0tLStm7d+vnnn6enpysUCkmSaO8/LY1BQUFmZmbF3hxRKpXFu0OdlXsIJSsrK9ctsF1dXekLfX3977//3t/fv1WrVnQJzrgqlO+///7KlSsFrEBVKlVMTAw95VqxYgVt9RT8uyRJUqlUS5YsoXMzfvrpp4YNG06fPr1ly5YAcO/evdatW8fGxgYHB3t4eCQkJIwePfrJkycWFhYsy/7888/r16/ftm0bPSvv3r37V199lZWVVfA/tyAIX3/9tSiKhW2aMQyTkZHh6+tbqK1Qwd2+fXvlypUF/7solUoTE5P+/fsDwJw5c0aNGlWo3j9RFGfNmkUv0HV2dt68eXNoaKiTk1P79u39/PyGDRuWlZW1atWq8+fPC4LQuXNnPz+/xo0bA8D58+fd3d1fv35NTwJWrVrVsWPHnIWQ47igoCAAaNy4cT4l08vLKy0trbDlkLa65HCamzsBGBkZ3bt3L+eSnH9vW1tbzexPQRDkkyc/XcuWLX18fAo4PkZX27NnDwAMHTrU3d3dwMCgsN/IMExqauq1a9cePnxIm2+0F2jt2rV79uxp37799evXP//8c3Nz8xs3bgCAv78/z/MMw8TGxh48eHDDhg00jPbt2xeq9gcAlmUHDhyoVqsLNR7IMExiYuKoUaMK+UNRIURHR/v4+Bw9erSAZ5YKhSIuLi4oKMjNzc3Dw8POzu7JkycF/zraax8YGHj79m36jPXs7GxJkqZNm+bp6TlkyJBHjx4BQJ06dWJiYgDg8ePH4eHh8K4K1gwpt2jRIjMzU1MjsyybmprasWNHX19fIyOjfH6LtbW1JElFGJfeuXNnhQoVCrtVmZM7AdSsWXPOnDnz5s3TLImPj69cuTJ9PXfu3L///pu+vnXrlrOzc+lEWQ40btyYNm0K6PXr10uXLh02bJiNjU0Rvo5hmJiYmEaNGnl5eXXq1EmhUKxfvx4ABEHo0KHDlClTVq9evXTp0s2bN0uSRI8fpVJJk72Dg0POXEU7fwobQO/evYsQNiaAkkYI6d69e69evQq4viAIly9f3rRpU82aNemSghcGQkhmZma1atUmTpzYrVu3ypUrr1q1ihAiCELDhg0BIDQ01M/Pz9vbm+M4nucBQE9Pj/bzqFSqffv2qVQquis69ZO+ZhgmLS2tRo0aXl5erq6u+WeyIUOGFDBaecqdABwdHR88eJCRkaGvr0//0gMGDPDx8TEwMNi8eTMAtGvXjq45e/bsyMjIUg5XPszNzefMmVOoTRiG0Zyu0ZuizJw5c/To0Wq1Oj4+ni4XRdHW1rZr16779+9//vx5gwYNeJ6vVKkSALi5uenr69PDjHaw0gJQmifCtBZAJapQ/8gsy65evbrg69NCqJntc/bs2Tp16ixcuJDn+Zw1tb6+/qJFi3x8fLy9vYOCgniepz2cDRs2dHBw0HSTvl8IaSdh9erVly9fPmrUqOzsbLz691PkTgAKhWLu3LmXLl367LPPRFHs1KnTzz//PHz48LNnz86YMePNmze02yciIkKpVNrb22sjZpS3kJAQnudFUZQkydjYuFq1amPHju3bt68kSfQYpoelKIoTJ07s0qXLypUr9fX1s7Oza9WqNXjwYE9Pz3Xr1hkaGiYkJBgbGzs4OOT5LXLoGEVF9vDhw/j4eDpIa2ZmZmVl9fDhwxs3bpiZmR08eFCzGs/zffr0qV+//oABAypXrpydnW1qarp69erGjRv/9ddflpaWKSkpoig2aNAgZ3kjhKSkpDg5OXl6erq6ut68eVOSJH19fUdHR8wBRZPHdXQTJ060sLDIzs52cnI6e/YsAJw5cybXOsOGDdNMB0K64Ouvvz537typU6cAQK1WN2nSZPz48RkZGT/++GO1atV+/PHHjIyMN2/eGBkZ0WkYANClSxfaGBQEwcvL6+zZs+vXr1er1TY2NiNGjKCXWbZu3fr9AbTBgwerVCrMBCgnlmX79+9/6NAhWqgyMzM9PT2//PLLffv2bdy4UV9ff/LkyUql8u3btwAgSZKlpSUAjBgxgrb3s7OzhwwZ4uTktG/fvuTkZDMzsy+//BIAVCpV69at6VcwDHPz5s0BAwbo6emtWbOGzgiqV6/e1KlTMQEUTd63c3r06JG5ubmFhUWe22RkZAQHBzdr1qyEY0O5Xb58uV27dgkJCbmuwaaXyeRcQnv26XR+2p9Du1klSVIqlT/++KNarV62bJlmP3SyEK3r6fp0Dk+eN35QKpX0VCPnQoVCsXTp0saNGw8ePLjIPzA+Pt7KygpTS8k5fvz4pk2b/vrrryLv4cGDB/Xq1YuOjmZZNlfzPFchpDdsoNP5AUAQBNpdI4qinp7eH3/8MX369NDQ0JwFKec1hnTOMS2Emrkn71+EqBnEAgCFQrF161ZTU9OxY8cW+QfKSt530sh/dFdfXx9rf50iSVKe9+bL2euanZ1NCElOTl69evUvv/zy7NmznJvQxlSuzUVRzHO3eB9A9L4PFcKc5UoQBDoyvGvXrjlz5ty+fTvX/Jz3LzTJVQjxSpTihVfQyQghJCEhITU1NSQkpEKFCtjQRlqhVqt9fX0DAwPt7OywNtcuvJeejIiiWKNGjY0bN4qiiAce0gpJkoyMjE6dOkU7iLQdjtxhAih7OI77xOqbXplZXPFQHMfhHeLk49NvCIiFUBdgAihLaKfN33//rYOP3uU47s6dOy4uLtoOBJUsWgj9/Px08BZsLMvevn3bzc1N24GUGZgAyhJjY+O+ffseOHBA24HkzdTU9EMzx1C5oVKpevfu/eeff+rmGBIhBAthwWECKEtcXFwOHz6s7SiQrDk6Oh45ckTbUaDioXMncUjO8EGvCJUmPANAuuLmzZv6+voAEB0dnZGR4ejoqO2IECrnMAEgXWFsbFyrVi0AsLe3T0tL03Y4CJV/2AWEdIWjo2PXrl0BwNvbWw63YkdI6zABIB0yc+ZMAKDPn0IIlTRMAEiHtGnTpnfv3ra2ttoOBCFZwASAdAjHcX/88Ye2o0BILjABIN2ieQogQqikYQJACCGZwmmgSCdERkaGPnmS+uaNIAimZmaOtWrZ2dlpOyiEyjlMAEjLLl28ONrN7REAVK1aZdS3BOD5j1+BwNcF2BUQgI8eQqjkYAJAWiOK4qhhw3bu29fj2PHPOnqoKujRu4uRH+ZkpqU/+euv5s2bT5040WvtWh288SRC5QAmAKQdoij27979SFr6hJRUvYqGoiDx/L/PhlXo6dfv17dGQtKavn3Cu3f3PX1ai6EiVF6V1YZVSkrKqlWrCCGEEG9v71wPKEe6b+Xy5Udu3p7y9zmloaHAi/TewizHsNy7B9Pzop6pyZSzfx/566/169ZpO978xMfHz5kzh5bGvXv3ajscJHePHj0aOnQoIcTS0tLPzy+fNctqAjh69KiFhUVUVNTjx49Xrlw5bdo0bUeECiEsLGzO3Lnf37/HKBXSu+RNWGZvh477u3SlOQAAJEFkFdy3z6MmTZkSFRWlvXg/YuvWrS1btoyOjg4KCho6dOjWrVu1HRGStXnz5o0ePTo6Ovr333/v2LHj9evXP7Qm0c2nOmio1erjx4+3aNGCXh366tWrS5cu9ejRI+ds8Xv37jVs2DA9PZ3eSxLpvnmzZv1pbNJt9qyc3T6EkD3tXV/5+8+U/tMdxHHM/ukzJluYT5kxQxvB/uvcuXOWlpYNGjSgb0NCQqKiojw8PHKuc/bs2c8++0wURd28tfXx48c3bdr0119/aTsQVDyOHDlCK0N6Ampvb1+3bt2cKyxcuDApKcnb2zvPzXX9DEChUMTFxWlmBHbr1i0+Pj7XtUK03sdxQh2kVqvv3r2ba6EoiktWrmzcq5fwz5gvYViGYRnCElah4MzNCQBdAoQAgChB+y+/mjpzZgkF+fbt27CwsIKs+fLly5zPvBwwYEB0dHSudQwNDUGHH2xAqwltR4E+4v79+wVcMyoqKiYm5uXLl48ePeratWtCQkKuFczMzJRK5Yc2LwODwGPGjFm4cOHBgwfNzMwSExO///77XCv4+PhMnDgx/ytIJUnKzs7Gol/KFArFmjVrLl26tHPnznbt2ikUCgBITEwEgIo2tiBKQIioVscFBTEcBwSy3ryR1Oq4Bw+F7GxJECrVqqUwMJREyaxmDQBITU01MjISBEEQhOKKkBCiUqlq1qzZt2/fGTNmNG/ePJ9C4unpCQBPnz51dHSMi4sLCgrq27dvrnXWrl27ZcuWD+0hKytLi4WQ4zjN06Szs7O1FQbKn0KhuHfvnouLi4+PT6dOnUxMTPJZecKECfTFV199tWTJknbt2uX8VBCE8ePH59MFBFJZoOn/jYmJyfXRnTt3ACApKSn/PcTHx3/iXwV9uqtXr0qSFBcXBwDTJWmqWpgmSoNuBH5o/REhT6cJ0lRenJyeCQCvXr2SJOl0Cc8ICgkJyacgTZ061cvLS5KkgwcP9u7dO9enZ86cAYDMzMw8t9VUvggV3KZNm/Kv3CRJ2rVrFwDwPJ9r+ZIlS/r165fPhro+BkDxPE8bj9nZ2fQFFR4eXqNGjTt37jRs2PCjO4mOjsZuolKmUCjmzJnz888/79y5s2fPnmZmZgDw6tUrS0vLSemZrFIJACLPvwoOYlgOCDk+cvib0NDBV6+L2dmiIFRydFQYGACAkJ3tXUHv1atX5ubmarX65cuXxfinZBimcuXKAODr69upUyfah/MhwcHB9evXlySpadOmP/30U6tWrTQf3bx5s1mzZpGRkfb29h/aPCoqimXZ4oq8CARBUKvVenp6WowB5Y/juOPHj3/zzTeLFi36+uuvHRwc8l8/JCSkdu3aUVFRuS6e37lz58iRI/MfHC0bCWD+/PkWFhahoaEVK1ZcvHgxXUhr/8DAwKZNm2o3PPQhgiD873//69y5c86FaWlphoaGo549N65SRRJFIIRhCAAAgX3u7q+Dg6e9ekWHB0RRAkkiLJPw5Mmu2k4lNLL65s2b+/fvt23btoDrE0J8fX379u3L87ymNg8ICGjRokVYWFj16tWLPUIkN+fPn2/WrFn+bREqMzNTX1//2LFjtH9SY9euXSNGjEhLS8v/2UplYAzgzp07ixcvzsjIyMrKMjEx6d+/v4uLy5MnT5ycnI4fP+7s7JyYmCiKYsWKFfMZ60BawbJsrtofAAwMDD7v1vXZtWuNq1URRABJEgUJAAghkiiK2WoJQBT+nQXEEAj1vzpu5MgS6j2vWLFiwWt/AKC1/5YtWzS1f2BgYIsWLfz9/S0tLWlpNDU11W5LH5Vp7u7uBVxz4sSJY8aM6dGjh1qtliSJZVmWZffu3TtixIiIiAi1Wk2HhenJ9/t0/Qzg7du3RkZGfn5+9F/kyJEjffv2TU9Pfz+t3bx5s0mTJtqIERXa5cuX27VrNyUrm2E5TQkkhKzW50At5pwGSggR1Op1+qpbt241btxYeyH/Kz4+3srKSnPGTU9ocq0TERFRtWpVbUSHZOTp06f0Mdoaa9eu7dWrV40aNXKtmZWVlWf7WNcTACqv3Bq6JPbp1/XH+YIggSQBAGGYt3FxrFKpb2r6T7EkhGPJ4RkzXB4+PHTihJYjfmfVqlWRkZE//fSTtgNB6FNhAkDaQdvRbtt3NB/1jSgCvR6YMAydlwYAhGUYAje277j43bfJycnGxsbaDhkyMzP379//zTff6Eg8CH0iTABIa2JiYmxtbSsMHzFkyVJjG2sRAP65NAwYgOQXMdsmToTDh96f3qAt/v7+bdu2jY6OxqcWo/IBp0UirbGxsUlLSxtnarLVtvLKAZ/f+uNAxI0bz28EBO7/bWXvPtvsbBfXr5eRkaEjtT8AtG7dWpIkrP1RuYFnAEj7YmNjr1y6dP3Spbhnz7IyMqo3qN/StV0bV1crKytth4ZQeYYJACGEZAq7gBBCSKYwASCEkExhAkAIIZnCBIAQQjKFCQAhhGQKEwBCCMkUJgCEEJIpTAAIISRTmAAQQkimMAEghJBMYQJACCGZwgSAEEIyhQkAIYRkChMAQgjJFCYAhBCSKUwACCEkU5gAEEJIpjABIISQTGECQAghmcIEgBBCMoUJACGEZAoTAEIIyRQmAIQQkilMAAghJFOYABBCSKYwASCEkExhAkAIIZnCBIAQQjKFCQAhhGQKEwBCCMkUJgCEEJIpTAAIISRTmAAQQkimMAEghJBMYQJACCGZwgSAEEIyhQkAIYRkChMAQgjJFCYAhBCSKUwACCEkU5gAEEJIpjABIISQTGECQAghmcIEgBBCMoUJACGEZAoTAEIIyRQmAIQQkilMAAghJFOYABBCSKYwASCEkExhAkAIIZnCBIAQQjKFCQAhhGQKEwBCCMkUp+0AEEIlThTF0NDQl7GxaWlvVSo9M3PzWk5Oenp62o4LaRmeASBUngmCsG/vXpZlnZyc2q1d63n+Yof9v7k0bKivr79qxYrk5GRtB4i0iUiSpO0YEEIlIjY21sbGBuzsvzh0yLZJU4ZjCAAAiBIkPHm6a+kS2Lf3+vXrLVq00HKgSEswASBUPkVHR9vb27fduLnV2O8BQBQkyHGwE5ZhCYT+z++wR8fz58+7ublpLVCkPWW4CygzM3Pjxo0uLi5ZWVnajgXJXUpKyrJly+rVq6ftQP6hVqvt7e3bbdnaZuz3oiCJvAj/bepJgsjzYs2OHb4KCHR3d4+KitJWqKgkPH/+/Ntvv12+fHn+q5XVBJCWlqavr//27dv79+/jSQzSrri4OBMTE0mSHjx4oO1Y/rF961Zw79Bi9Hf8fxv+ufC8aN+sabPVayZ/921phodK1OXLl6tVq2ZiYvL8+fP819T1LiC1Wn3hwgUXFxdLS0sASEpKCgwMbNeunZ6eXlpamlqtNjU1zcjIwPkMqBT4+/tbWFjUqlWLvo2IiIiJiWndurUkSVlZWTExMTVq1NCFAyo7O1ulUg0LfmhRx1kUxPxXJoRkp6dtqGgUFhZWvXr10okQFZdz584plUoAIIQQQqytrWvWrJmRkaGvr3/8+PFjx47t2LEjn811/QxAoVDcuXOna9eu9O2QIUPu3r1Lq3sDA4Ps7Oyi7fbVq1fFFiKSjZCQECcnJ83br7/+Ojg4GAAIIXp6emq1uuC7ev36dfHH905wUBAAmBeg9gcASZL0jAyNJky8fOHC+58mJCQUe3ioGF2+fDkgIODmzZt+fn6urq60ya+vrw8APM9/dHNdTwAAMH369Nu3b588efLatWsnTpyYMWPGp+wtICCgW7duly9fLq7wkHz06dMHAJ49ewYAr1+/9vf379+/f2F3EhgY6OnpefHixeKP752IZ8/g2+/+mfEDQFiGMHn/R1eQAFq5d3h4965mD5IkXb16tXHjxvfu3Su5ONGnW7hw4bRp06ZMmfL69esZM2Z07NixUJuXgQvBCCFhYWE1atQAgI92aeVj7969Q4cOpa/VavWyZctE8ePtI1RqCCE8z9+/f7958+aFak0Xr4SEhIkTJ06ZMuX9j0xNTceMGXP8+PEJEyZcuXLFw8OjUqVKBd/zvn37hgwZQl+npaUtX768eEvgnTt3aAdURtrbxo2b0IUsx1xZsfLq7Fl5bjJTknheBIAKxhVfRkcDgCAIu3btGjVqFF1h0aJFM2bMwCNFWyRJunv37kf7FQ8cOLBx48Yi9IiUgQQAAA4ODvSFnZ1dkXfSo0ePQ4cO0Sbbl19+6ezsjMVapxBCEhISevbs6e3trcU/jSRJFStW/NCno0aNatSo0YQJE7y9vRctWlSoPffo0ePw4cP9+vUDgK+//rpOnTqCIHxquDlosianUCRFx9DXEkDlxk2a/7iAVSj+s7YkAcNo6hVJlBiWBQCWZfv06VOhQoVBgwYBwJgxY+zt7fFI0aKPNoaePXv2xRdfhIaGKnL9iQtA1weBqVWrVgmCEBIS4uzsnLMLKD4+3srKqrCDwDdu3EhJSencuXMJRIo+SUJCgrm5uY6XSULIqVOnunXrlpWVRcffqJCQkNq1axck+ICAgKdPn9IatiT8ffZs56++nvk6nrbuNdd/vU/gRQBgOebMmrWDsjKnz5mT89NLly5lZGR89tlnJRQn+nQ8zysUit9+++3LL7/M9dGRI0dOnTqV/yBwGTgDePjw4YwZM96+fZuVlWVmZubp6ens7PwpO8TrHnVW8baIS8gff/zRrVu3DRs25Kz9C6V58+bNmzcv3qhycmnYEBJepb5KqGBmJomiyH+k/S5KcG/a1C3+/rmWt2vXrsRiRMVj9uzZw4cP//LLL+lZGp0LVPDNdX0QOD09vW7duqdPnzYwMKhUqdL+/fvr1KmjufKL/mYdbzCicoZeNNuzZ89cywsy6aJ0WFpa9uzQ4d5BH7YAxzfDMVHXrgNA02bNSjwyVKzCwsJWr169a9cuQgjLsizLrlu3TvNpZmZmTExM/nvQ9S4gSZLeT2h5LkTlAO3T0/EyuWXLlhs3buzevVvbgeQnLCysZs2aIx6FmNWulc8ZAGEZPj3D27DC8WPHenh6lmaESBfoehdQnhU91v5IK3ie9/X1HTNmTInO4i8WNWrU2P/rr4Ocnb599tykWhUhr+uBGY7JTktf37XruNGjsfaXJ13vAkJId1y7dm3gwIGhoaFmZmbajuXjvho06Jeff97uUPXxiZOEEJZjGI4hLMOwDMsxLMe8CLy53tBgVHWH9Zs2aTtYpB263gWEZEXHu4AEQWBZVttRFE7AjRstWrYEgKbrN1Rt1FjfxJjPzHwRHHx5xHAAOHTwYL/CX8uGyg1MAEiH6HgCKKMkSQoICPjr+PG7ly8lhYcZWFk7NW/RpVev9m5uKpVK29EhbcIEgHQIJgCEShOOASCEkExhAkAIIZnCBICQjKSkpPi/d8Uvki1MAAjJyF9//TV9+nRtR4F0BQ4CIx2Cg8AlzczMLDExMTw8XHOHXSRneAaAkFwEBgYmJiYCwO8c7xVKAAAVzUlEQVS//67tWJBOwASAkFz4+fn98MMPHTp04Hk+OTlZ2+Eg7cMuIKRDsAuopJ0+fXrjxo0nT57UdiBIJ+AZAEIywvN8mXjoAiodmAAQQkimdP120Cin2NjYkJAQhtHRtC1Jkp2dXY0aNbQdCCpBiYmJ9+/f1+VCaG1t7eTkpO1AygZMAGVJSEiIu7u7q6trdna2tmPJTaFQXLlyZffu3ZgAyrcXL164u7u3bNkSdO9hfBzH+fv7r1+/HhNAAWECKEvovYhPnDihmwlgyZIlOtswRMWF/omPHj2qg3fG5jhu69atOhiYzsIEUPbwPK9Wq7UdRW6EEN15KC4qaTzPS5Kka2cAAIBD3IWSd3vt1atXKSkp7y+XJCksLKyEQ0Jlkg7WBUiGsBwWSh4JID093dLSMs/TKELIzp07T506VfKBoUIghCiVytLpfiGEsCyLj2VGudBCWAoFg5ZAjuOwq+fT5VFl7N69+9ChQ4aGhnluMGPGjO7du2Oa1R2EELVaPWXKlGfPnpV0DlAqlampqRcuXEhISMAcgDQIIW/evBk9enRJFwyGYdRq9c2bNy9cuHD37l2GYbAcfoo86ouxY8d6eHh8aANjY+NRo0Zdv369JKNChSNJ0t69ezMyMvI8GIrrCOE4bv/+/bVq1Ro4cGBsbCyO95ZRJVRjCoJw6NAhtVpdhEKY69MPrUwISUpKqlKlyt69ewMDAz/77LPRo0d/Sswo9zEcFhbGMIyxsTF9m56evnbtWkIIIcTCwoIu7N27t5+fX6mGiQrg/aNIoVAAgCiKDMPQ82VRFHOuSQjRLOE4jg7rcdy/UwMkScp5om1sbBwaGvr+dyFEvV8wNIWQljHIqxDSkducJZbjOM0KmjIpSZKxsXFgYODmzZtnzZoVEhJy9OjRW7duYV9QkeVOAE+fPp02bRp9zfO8gYFBUlJSUlJSenr6zZs36XInJ6f169eXapiokAghmZmZ48aNs7GxsbW1tba2joiI0NPTW758eZ8+fehhplQqN2/e3K9fP5VKlZmZOX36dFtbW1tb2/Xr1zMMwzBMdHS0ra3t/fv3bWxszpw5AwDdunUzMDAoeBiJiYlBQUGFCruwvxQVVmn+IzMM4+XlRQth5cqVb9y4oaend+bMGRsbG0mSaEq4dOmSnZ0dzRCbNm2iK0+cODEzM5PjuHv37lWpUiUgIMDGxubOnTssy9rb22dlZWVlZZmbm3t4eERGRn70F2VlZV24cKFUfnEZk3saaFJSUr169ehrOti7ePFi+rZq1ar0hUKhePXqVWlFWE4kJCSkp6cXYUNJkipVqvShIZl8ttLT0/vuu+/WrFnDsuzhw4dbtGjx8uXL4cOHb9q0KSYmxtLSMisra/HixSdOnACAiRMnNm3aNC4uLisrq2fPng4ODr1796bNrvHjx1+4cMHW1lYQhIKP/YSGhv7xxx/z5s37888/bWxsCjI5j2GY+Pj4Qv1MVARpaWnJycmFvZREkiRDQ8NCpX+qR48eY8eOValUt2/f7t69e3h4eMeOHQHg3r17jRs35jhu27Zt69atMzIy2rhxY3BwcFRUFMdxy5cv9/b2nj9/vkqlEgTBy8vL39/fxsZGFEVaCAkhb9++PXfu3Jw5c/IpljExMb6+vuPHj//pp58aNGhQ8JnKoihaW1sX9seWOXlcB0BP0ADg8ePH8+fPL914yq3ffvttwoQJRdvWz8/P3d29sFtJktSgQQNJknieb9euHQCkpKQ4ODjUrVv3ypUrAwcOfPz4MQA0adLk2bNnx44d27FjByHE0NBw2bJlM2fO7Nu3L93Phg0b6tWrp1arC177T5o0SXOO2KtXr8JGjkoOIeTSpUumpqZF2Hbjxo1jx44t1CaiKNapU4fOU6hfvz4AREZGuri4zJ4928fHp0WLFvHx8X5+fuvXr09JSfnhhx+Cg4P19PQA4JtvvmnQoMEPP/xA97N27dqqVavSiw/oEqVS+euvv9apU6devXofqta3b9/+3Xff0dfjx48fP358oYJ/8eKFjY1NoTYpc3InAEtLy99++23o0KEAwLLsmzdv3t8mNTW1adOmpRFdOcKy7Nq1aydOnFiEa7hydsoX6hsvX768ZMmSW7duubq60oWiKM6fP3/gwIFffPHFuXPnli5dqlKpYmNjAaB27dpJSUmazTVDytWqVStU7Q8A3t7eU6dOPXLkyMSJE48fP96+ffsCXp6jm9cWlSfdu3dPTEwsQi+QJEn6+vqF3YrjuIcPH65bt+7o0aO0EUMvGOzZs2erVq1+/PHHO3fuuLm52dnZRUREAMBnn3324sULeHfRe0JCAp1rYGdnl/PAUalUV69enT59+oMHDzQN1vd9++23/fr1O378+PDhw3/66adhw4YV5AyAEJKenm5jYyOHaQ65axZHR8edO3f+8ssvANCmTZtWrVrNnj3b0tIy5zqPHz8eNmxYqYVYbigUCoZhVCpVKXwXy7JXr17t06fPvXv3rKysMjIy6CMABUFo0aIFAISFhS1ZsiQgIEBTNd+9e5d28tAxfzomDEW9ssbe3n7ChAkjRoyIiooyMjIqvl+GPgkhpGjN/yJgGOb58+ft27f38/PbvHmzUqk0NzeXJEkURUdHR3Nz8/v37x85cmTs2LGaevnMmTMGBgZ0PAAADAwMaOskZyFUqVTXr1/39PS8du2ahYVF/nW6mZnZsGHDhgwZEhAQUPB+VHoWIge5U1yVKlUAICYmBgBatmw5b948Kyurn3/+2dfXd+LEiXSdlStXFqFHApV021ahUCgUCqVSqVQqVSrVyZMnZ86c6eDgoFKpwsPDNTEYGhouWLBg8ODB7du3r1atmiAI9PZtkZGRpqamxsbGFStW1NfXL5ZoDQ0NnZ2dP30/qKzIWQiVSmVgYKCLi0vjxo319PRevnwJ74agRVHcuHHjsGHDfHx8WrZsyfO8lZUVANy9e9fExIQWwjzHG5RK5bVr17p3737nzh1nZ2eGYeiMhvwxDEPvXldA+ZxVlDN59C0cPXr0wIEDkydPBoBFixYNHjz44cOHADB8+HAAiIyMDAwMrFOnTikHivJBK+sVK1bY2tqKosjzfP369YcMGdKuXTuO43iep4P2tK3E83yPHj0WLFiwcOFCABBF0cLCwsfHx9XVdcKECTY2No8fP+7Ro0f79u3zPAOgb+VzhKACogVj8eLFFStWlCQpOzvb3d3d3d19zJgxM2fOtLGxuXfvHgDQwWdBEFq3bp2YmLho0SI9PT21Wk17LF1dXb/66isXF5fIyMjq1asPGzYsZ0ljWTYgIKBHjx4tWrTYvXt3enq6IAg1a9YcNWqUDt4dq0zIIwH06tWLENK/f397e3sAcHR0dHR01Hzas2fP4ODg0gsQfQydJX3ixAlNBzpt5tetW/f27dvPnz83NjauV69ez549aeOLYRg6H6l169a0/0etVru7u4eEhDx58kQUxebNmzs6OvI8b2Ji4uvrm6vnlxBy4sQJOm9PGz8X6SgDA4Njx44RQjSF0MzMzNzc/PHjxyEhIRz3//buPaiJOw8A+G83SxJSBBKIqBn6OF5Tr1jgeBb0qK1ArSDMWewU7EwrXqzYK1arOTg8dKAORVt1zgeK5ZyL2laLepRSbxRpe5USa0VSFCwIOiACCjbhcUn2cX+sl3LhUU1JNnG/n/EPdrML34Gfv+/+XvsjlErlqlWrPD092W5Gg8GAEEpOTmYLIUVRgYGB169fv3z5ssFgCAkJCQgIMJlMs2bNqqioYH8EwzDe3t6VlZXm9SsMw0gkEiiKVht/dHGS5n9DQ4PNggFWwnE8IiLC4qTBYGCnVCOESJKMiYmhaVogEAwMDMybN0+tVrNPXuzFJpPJ3d09MjKSPWQHAyQSSWxsrHninVlERMTYk4DPGIYRiUTs8NLokyaTycPDg+1+oWk6ODiYLTYURS1fvjw3N9fX19c8IZWiKKFQGBYWxh7SNE3TtFQqjY2NZZMETdOPPvqoeT66GbwB1GoP/zA3T1BjIIRomjZ/TVEUhmHNzc1BQUEqlSoxMdGi1cwwzOh7zWfGVvQPtCAA8MTo8sMyP6Szh+bi1N/fr1AoXF1ds7OzJymE5pbE6PrdXKQtijqwDuwHwCNsC7qqqioiIgL6TAFX3Nzc2DEnBG9v5hokAKdk9Wp+mUzm5eXFtgamPCR4kQPfWPcXd3FxiY+PN7//xxFC4i1IAM6HnUnNdRSWYFSAV5j/se52GxVgWEj4oCABOBN2Hqe3tzfXgUyovLyc6xCAbbF1NztF0DFt376d6xCcBiQAZxIaGtrc3OywK9QZhvH09OQ6CmBbTzzxhIMXQnd3d66jcBqQAJyJp6cn1LCAW25ubkFBQVxHAaaGg6ZxAAAAtgYJAAAAeAq6gOzEYftMAQC8BbWSzVEUVVhY2NLS8uOPP7LvXwMAAEcALQCbEwgE/v7++fn5CKGPP/6Y63AAAOAeDNZN2IFer2enpun1+gfd3RcAYGcjIyMSiaS7u/uh3xYYWgD2MG3aNJVKJRaLofYHADgOSAB28uqrr7LbnAIAgIOABGAnsDMiAMDRwCwgAADgKUgAAADAU9AFZFsURZ3XaM588UXT+fOa6urfJSyYHR7x/MKFkZGRLi4uXEcHAOA1aAHY0LlvviEIIuaZZyqkMmNuXkT9eXrjpqqZs+Li4oRC4dmaGq4DBADwGqwDsJXiLVtUubmpX/wrIHEBhhCDEGIQwhC7X9G1s18emx+fp1IVbtnCbZwAAAuwDsA53L17l2EYqVTKdSCWtn/wgSo3d2XnTXfFTIocZ/Oj3zz7++y+O0VyL5FIlF9QYPcAwRRjGObu3bsYhsH7uoEjMBqNQ0NDOI57eHhMcpmzdgF9//33GIZJpVKZTObi4tLT08N1RD/TarVr3n5beaNz2gS1P0KIImmJt+yN7p6NmzZpNBo7RwimVk1NDY7jMplMKpUmJCTo9XquIwK8tm/fPpFIJJPJPD0933nnnUmudNYEYDQaW1pa2C1A16xZs2DBAq4j+lnWwheeP3TYw1dBT1D7s2iSnjZj+qIT/4yKirJbbMAWSJLs6upi92oWCoUbN27kOiLAa1KpVK/XMwzT39+/devWioqKia509DEAiqJef/31rKysuXPnIoQuXLiwY8eOsrIyoVBovqatrc3f3/+nn35yhK3g2GByhkYIkeiXf7cYxtD0+0LiB632t089ZZcAgfWKi4sVCkVmZiZ7eOLEiatXr65fv370NRqNJioqiiTJSRZ+X7p0qb29PTU11bbhAmuZTCahUNjb2yuXy7mO5RckJiYqFAqEkFgsxnE8MTExOTl59AWlpaVff/21Wq0e93ZHHwMQCARKpTI2NnZwcFAkEoWHh587d2507Y8QGhgYQAjdf+1/4cKFtra29PT0qQ8XIe2lS2jFH4USMdv5gxM4Nt5lDEI0SSOGIVwEsrXrGhsaLBJATU2Nu7t7eHi4LYIE1omMjJw/f/7LL79MEARCKC0t7fTp0xbX9PX1RUZGjlv7Dw8P19TUqFSqpqam8vJyjUZDUZQ94gYPAsMwg8GAEKqvr5fL5TQ9WTveDkiSZB9/x7V161aCIAiCaGxsXLJkydgOn46OjtDQ0Ilud/QWAGvDhg0ikcjHx6enp2fz5s0Wny5atCg1NTUrK2vyb6LX68+ePbt48WKE0EcfffTkk09O4X8/gUAwZ84chFBRQcFxmddzf3qTImmcwLXqQ0M9Pdj/7wbD0PQj06cHZ2bSFI0TuObgP0K/PffBnj0IoTt37lRXVy9btgwhdPr0aW9vb87LH98wDOPm5hYYGDj2I4qiCIKoq6uLjo5ubW0NCAgY+6SPYVh1dXVSUtLY23ft2rV69Wr2a7lc3tfXZ4v4wZSYM2dOY2Mj11Hc84u1NEmSLi4uYwve7du35XJ5e3v7448/Pu6Njt4CYBUWFrJP/SRJWnx08ODBqqqqzz77bPLvQNP0kSNHlEole1hWVjY4ODhVdavJZPLz8zt69ChCyGQ0ih9xY/9cGEIdZ840/7187C1+6enBy+71JBAi4aBOhxAaGRkpLS3Ny8tjzxcXF+t0OqfI0A8TnU732muvWXTssAQCwe7du/fv3x8dHX3q1KlNmzZZ1P6bN29OT08ft/ZHCGVnZ69ataquri4/Pz8nJ8eiqQ6A1ZRK5erVqy0KHsMw8fHxe/funaj2R87SAujp6WEn5HZ2drIdXqxPPvlk6dKlAwMD9zn3jqbp+vr6t956Ky8vj20KTLn333vvQ5p5UbWBImmEYQxFMeOlGQzDMIJADCMg8H+XfZjQ3vbXoiL2I4PBUFtbm5SU1NDQ8PTTT9siSGC1zs5OX1/f4eFhiUTS0tIyuqGwY8eOnJwco9F4P2u8u7q6RpdkAKxWWVmZkpJiMBgs+saTkpICAwN37tw5yb3OkQDkcvmRI0du3rxZXFzc1NTEnjx69Gh6erp1W6zcunXLRks8vqytjX8je/2VpnsTQLFxhwAQQggxDEKIIPDiuLmf/1n1wosv2i1I8GukpKSEhISo1epr166ZT27btu3AgQNarRZe+g3s6caNG4899lhra6ufn9/o86mpqbNnz3733Xcnv90JEkB5eXlRUVFrayvDMDiOq9XqjIyMw4cPZ2RkHDp0yNfX12QykSQZFhbm7e3NdbCov7/fy8tLeaNzmkIx7rP/aBiOD/f17Z4x/VZ3tw/U9U6ivr4+Ojr6+PHj5mk827ZtW7du3cmTJz08PCiKMplMsbGxsPkPsDV2ttLatWtXrlxpNBppmpZKpT4+PkuXLq2oqPjqq69IkmRr+Pj4eBwfZ9K/o48B6HS6pqamM2fOIIQwDLty5cqBAwfS0tKuX79eUlLS0dHR2tqKEBoZGfH393eEBCCTyf6iUv2tqFC5d8+kywAQQgjHUe3uXcszM6H2dyIzZ85ECJmXnoyMjAwODpaUlGi1WnZawdDQUFhYGCQAYGu9vb0FBQVisZid5ck+eYSGhgYHB8fExNTW1pqf7+fNmzduAnCCFoDT0el0Hh4eySdOzl6cQk6cBAgCb6s5e+y5+bdv3/by8rJnhODXSEhIeOmll1asWMF1IAD8Ws66EtiRubu7X7x4sTJ1sfbTCpzAcYHlLxkX4AICb6k+dey5+d/W1UHt7yw6Ozvj4uL6+vqg9gcPB0gANhESEqLVaj9f8oeS1LSeH37ACVww6l/v5SvvvZJxYmHSxYsXo6KjuQ4W3C+9Xp+Wlvbdd99xHQgAUwO6gGzIaDTu27PnzZwchBBKTlGER3Q1XkKfHkMIlZWWvrJsmaurK8chAgB4DBKAzRmNxqstLd3d3f8ZHha7uk6fMSMoKEgsFnMdFwCA7yABAAAAT8EYAAAA8BQkAAAA4ClIAAAAwFOQAAAAgKcgAQAAAE9BAgAAAJ6CBAAAADwFCQAAAHgKEgAAAPDUfwGa6zjnNUkFIQAAAABJRU5ErkJggg==)_____no_output_____## Model Architecture_____no_output_____We will perform the following steps:

* Use input tensors from our data generator

* Produce Semantic entries from an Embedding Layer

* Feed these into our Reformer Language model

* Run the Output through a Linear Layer

* Run these through a log softmax layer to get predicted classes_____no_output_____We use the:

1. [`tl.Serial()`](https://trax-ml.readthedocs.io/en/latest/trax.layers.html#trax.layers.combinators.Serial): Combinator that applies layers serially(by function composition). It's commonly used to construct deep networks. It uses stack semantics to manage data for its sublayers
2. [`tl.Embedding()`](https://trax-ml.readthedocs.io/en/latest/trax.layers.html#trax.layers.core.Embedding): Initializes a trainable embedding layer that maps discrete tokens/ids to vectors

3. [`trax.models.reformer.Reformer()`](https://trax-ml.readthedocs.io/en/latest/trax.models.html#trax.models.reformer.reformer.Reformer): Creates a Reversible Transformer encoder-decoder model.

4. [`tl.Dense()`](https://trax-ml.readthedocs.io/en/latest/trax.layers.html#trax.layers.core.Dense): Creates a Dense(*fully-connected, affine*) layer

5. [`tl.LogSoftmax()`](https://trax-ml.readthedocs.io/en/latest/trax.layers.html#trax.layers.core.LogSoftmax): Creates a layer that applies log softmax along one tensor axis.

_____no_output_____

def NERmodel(tags, vocab_size=35181, d_model = 50):

  model = tl.Serial(
    # tl.Embedding(vocab_size, d_model),
    trax.models.reformer.Reformer(vocab_size, d_model, ff_activation=tl.LogSoftmax),
    tl.Dense(tags),
    tl.LogSoftmax()
  )

  return model_____no_output_____model = NERmodel(tags = 17)

print(model)Serial_in2_out2[
  Serial_in2_out2[
    Select[0,1,1]_in2_out3
    Branch_out2[
      []
      [PaddingMask(0), Squeeze]
    ]
    Serial_in2_out2[
      Embedding_35181_512
      Dropout
      PositionalEncoding
      Dup_out2
      ReversibleSerial_in3_out3[
        ReversibleHalfResidual_in3_out3[
          Serial[
            LayerNorm
          ]
          SelfAttention_in2
        ]
        ReversibleSwap_in2_out2
        ReversibleHalfResidual_in2_out2[
          Serial[
            LayerNorm
            Dense_2048
            Dropout
            LogSoftmax
            Dense_512
            Dropout
          ]
        ]
        ReversibleSwap_in2_out2
        ReversibleHalfResidual_in3_out3[
          Serial[
            LayerNorm
          ]
          SelfAttention_in2
        ]
        ReversibleSwap_in2_out2
        ReversibleHalfResidual_in2_out2[
          Serial[
            LayerNorm
            Dense_2048
            Dropout
            LogSoftmax
            Dense_512
            Dropout
          ]
        ]
        ReversibleSwap_in2_out2
        ReversibleHalfResidual_in3_out3[
          Serial[
            LayerNorm
          ]
          SelfAttention_in2
        ]
        ReversibleSwap_in2_out2
        ReversibleHalfResidual_in2_out2[
          Serial[
            LayerNorm
            Dense_2048
            Dropout
            LogSoftmax
            Dense_512
            Dropout
          ]
        ]
        ReversibleSwap_in2_out2
        ReversibleHalfResidual_in3_out3[
          Serial[
            LayerNorm
          ]
          SelfAttention_in2
        ]
        ReversibleSwap_in2_out2
        ReversibleHalfResidual_in2_out2[
          Serial[
            LayerNorm
            Dense_2048
            Dropout
            LogSoftmax
            Dense_512
            Dropout
          ]
        ]
        ReversibleSwap_in2_out2
        ReversibleHalfResidual_in3_out3[
          Serial[
            LayerNorm
          ]
          SelfAttention_in2
        ]
        ReversibleSwap_in2_out2
        ReversibleHalfResidual_in2_out2[
          Serial[
            LayerNorm
            Dense_2048
            Dropout
            LogSoftmax
            Dense_512
            Dropout
          ]
        ]
        ReversibleSwap_in2_out2
        ReversibleHalfResidual_in3_out3[
          Serial[
            LayerNorm
          ]
          SelfAttention_in2
        ]
        ReversibleSwap_in2_out2
        ReversibleHalfResidual_in2_out2[
          Serial[
            LayerNorm
            Dense_2048
            Dropout
            LogSoftmax
            Dense_512
            Dropout
          ]
        ]
        ReversibleSwap_in2_out2
      ]
      XYAvg_in2
      LayerNorm
    ]
    Select[2,0,1]_in3_out3
    ShiftRight(1)
    Embedding_50_512
    Dropout
    PositionalEncoding
    Dup_out2
    ReversibleSerial_in4_out4[
      ReversibleHalfResidual_in2_out2[
        Serial[
          LayerNorm
        ]
        SelfAttention
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in4_out4[
        Serial[
          LayerNorm
        ]
        EncDecAttention_in3
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in2_out2[
        Serial[
          LayerNorm
          Dense_2048
          Dropout
          LogSoftmax
          Dense_512
          Dropout
        ]
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in2_out2[
        Serial[
          LayerNorm
        ]
        SelfAttention
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in4_out4[
        Serial[
          LayerNorm
        ]
        EncDecAttention_in3
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in2_out2[
        Serial[
          LayerNorm
          Dense_2048
          Dropout
          LogSoftmax
          Dense_512
          Dropout
        ]
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in2_out2[
        Serial[
          LayerNorm
        ]
        SelfAttention
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in4_out4[
        Serial[
          LayerNorm
        ]
        EncDecAttention_in3
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in2_out2[
        Serial[
          LayerNorm
          Dense_2048
          Dropout
          LogSoftmax
          Dense_512
          Dropout
        ]
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in2_out2[
        Serial[
          LayerNorm
        ]
        SelfAttention
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in4_out4[
        Serial[
          LayerNorm
        ]
        EncDecAttention_in3
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in2_out2[
        Serial[
          LayerNorm
          Dense_2048
          Dropout
          LogSoftmax
          Dense_512
          Dropout
        ]
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in2_out2[
        Serial[
          LayerNorm
        ]
        SelfAttention
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in4_out4[
        Serial[
          LayerNorm
        ]
        EncDecAttention_in3
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in2_out2[
        Serial[
          LayerNorm
          Dense_2048
          Dropout
          LogSoftmax
          Dense_512
          Dropout
        ]
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in2_out2[
        Serial[
          LayerNorm
        ]
        SelfAttention
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in4_out4[
        Serial[
          LayerNorm
        ]
        EncDecAttention_in3
      ]
      ReversibleSwap_in2_out2
      ReversibleHalfResidual_in2_out2[
        Serial[
          LayerNorm
          Dense_2048
          Dropout
          LogSoftmax
          Dense_512
          Dropout
        ]
      ]
      ReversibleSwap_in2_out2
    ]
    XYAvg_in2
    LayerNorm
    Select[0]_in3
    Dense_50
    LogSoftmax
  ]
  Dense_17
  LogSoftmax
]

# Train the Model_____no_output_____

from trax.supervised import training

rnd.seed(33)

batch_size = 64

train_generator = trax.data.inputs.add_loss_weights(
    data_generator(batch_size, x_train, y_train,vocab['<PAD>'], True),
    id_to_mask=vocab['<PAD>'])

eval_generator = trax.data.inputs.add_loss_weights(
    data_generator(batch_size, x_test, y_test,vocab['<PAD>'] ,True),
    id_to_mask=vocab['<PAD>'])_____no_output_____def train_model(model, train_generator, eval_generator, train_steps=1, output_dir='model'):
    train_task = training.TrainTask(
      train_generator,
      loss_layer = tl.CrossEntropyLoss(),
      optimizer = trax.optimizers.Adam(0.01),
      n_steps_per_checkpoint=10
    )

    eval_task = training.EvalTask(
      labeled_data = eval_generator,
      metrics = [tl.CrossEntropyLoss(), tl.Accuracy()],
      n_eval_batches = 10
    )

    training_loop = training.Loop(
        model,
        train_task,
        eval_tasks = eval_task,
        output_dir = output_dir)

    training_loop.run(n_steps = train_steps)
    return training_loop_____no_output_____train_steps = 100
training_loop = train_model(model, train_generator, eval_generator, train_steps)
Step      1: Ran 1 train steps in 815.40 secs
Step      1: train CrossEntropyLoss |  2.97494578
Step      1: eval  CrossEntropyLoss |  5.96823492
Step      1: eval          Accuracy |  0.85458949

Step     10: Ran 9 train steps in 6809.59 secs
Step     10: train CrossEntropyLoss |  5.27117538
Step     10: eval  CrossEntropyLoss |  5.19212604
Step     10: eval          Accuracy |  0.85005882

Step     20: Ran 10 train steps in 5372.06 secs
Step     20: train CrossEntropyLoss |  6.68565750
Step     20: eval  CrossEntropyLoss |  4.00950582
Step     20: eval          Accuracy |  0.81635543

Step     30: Ran 10 train steps in 1040.84 secs
Step     30: train CrossEntropyLoss |  3.92878985
Step     30: eval  CrossEntropyLoss |  3.32506871
Step     30: eval          Accuracy |  0.78096363

Step     40: Ran 10 train steps in 3624.02 secs
Step     40: train CrossEntropyLoss |  3.41684675
Step     40: eval  CrossEntropyLoss |  3.47973170
Step     40: eval          Accuracy |  0.84054841

Step     50: Ran 10 train steps in 195.43 secs
Step     50: train CrossEntropyLoss |  2.64065409
Step     50: eval  CrossEntropyLoss |  2.21273057
Step     50: eval          Accuracy |  0.84472065

Step     60: Ran 10 train steps in 1060.08 secs
Step     60: train CrossEntropyLoss |  2.35068488
Step     60: eval  CrossEntropyLoss |  2.66343498
Step     60: eval          Accuracy |  0.84561690

Step     70: Ran 10 train steps in 1041.36 secs
Step     70: train CrossEntropyLoss |  2.30295134
Step     70: eval  CrossEntropyLoss |  1.31594980
Step     70: eval          Accuracy |  0.84971260

Step     80: Ran 10 train steps in 1178.78 secs
Step     80: train CrossEntropyLoss |  1.15712142
Step     80: eval  CrossEntropyLoss |  1.15898243
Step     80: eval          Accuracy |  0.84357584

Step     90: Ran 10 train steps in 2033.67 secs
Step     90: train CrossEntropyLoss |  1.06345284
Step     90: eval  CrossEntropyLoss |  0.93652567
Step     90: eval          Accuracy |  0.84781972

Step    100: Ran 10 train steps in 2001.96 secs
Step    100: train CrossEntropyLoss |  1.04488492
Step    100: eval  CrossEntropyLoss |  1.02899926
Step    100: eval          Accuracy |  0.85163420

# References_____no_output_____---

* [Google AI Blog- Reformer: The Efficient Transformer](https://ai.googleblog.com/2020/01/reformer-efficient-transformer.html)

* [Google AI Blog- Transformer: A Novel Neural Network Architecture for Language Understanding](https://ai.googleblog.com/2017/08/transformer-novel-neural-network.html)

* [Trax: Deep Learning with Clear Code and Speed](https://github.com/google/trax)

* [The Illustrated Transformer](http://jalammar.github.io/illustrated-transformer/)

* [Attention Is All You Need](https://arxiv.org/abs/1706.03762)

* [Illustrating the Reformer](https://towardsdatascience.com/illustrating-the-reformer-393575ac6ba0)_____no_output_____


# Notebook from abhatia25/Beaverworks-Racecar
Path: PythonHW-1.2.4/heat_dispersion/HW_heat_dispersion.ipynb

# Simulating The Spread of Heat Through a Material
> Written by David Mascharka and Ryan Soklaski

## Understanding the Heat Propagation
In this problem, we will learn about a simple algorithm for numerically simulating the spread of heat through a material. We will want to use vectorization to write an efficient algorithm.

Imagine that we have a rectangular piece of steel. For now, let's treat this piece of steel as a 5x5 grid - we are only able to measure the average temperature of each of these 25 grid regions. Let's assume that steel starts off at a uniform 0-degrees. Thus, our temperature readout for each of its grid positions is:

```
                                0     0     0     0     0
                                0     0     0     0     0
                                0     0     0     0     0
                                0     0     0     0     0
                                0     0     0     0     0
```

Now, we will clamp hot contacts, which are always at a constant 100-degrees, along the outer edges of the steel. Upon clamping these contacts, our temperature readout at time-0 becomes:

```
                               100   100   100   100   100
                               100    0     0     0    100
                               100    0     0     0    100
                               100    0     0     0    100
                               100   100   100   100   100
```

We will adopt the same indexing scheme as a 2D NumPy array. That is, element (i,j) of this grid is row-i, column-j in the grid. The top-left corner is located at (0, 0), and has a temperature of 100.

Moving forward, we want to describe, numerically, how the heat from the contacts will spread through the material as time carries on. The heat equation is a partial differential equation that describes the flow of heat through space and time. In the following equation, the function $u(x, y, t)$ describes how much heat resides at the location $(x, y)$ at time $t$:

\begin{equation}
\frac{\partial u}{\partial t} - \alpha \left(\frac{\partial^{2} u}{\partial x^{2}} + \frac{\partial^{2} u}{\partial y^{2}} \right)= 0
\end{equation}

Do not worry if you have no clue what a partial differential equation is! You do not need to know anything about the heat equation, we are simply providing some background here.

What this equation ultimately says is that heat will spread such that a point will take on the average amount of heat among its neighboring points. Numerically, we can write this out as:

\begin{equation}
u^{(t)}_{ij} = \frac{u^{(t-1)}_{i+1,j} + u^{(t-1)}_{i-1,j} + u^{(t-1)}_{i,j+1} + u^{(t-1)}_{i,j-1}}{4}
\end{equation}

That is, $u^{(t)}_{ij}$ is the heat at grid-location $(i, j)$ at time-step $t$. It's value is given by the average of the heat of all four of its neighboring grid positions from time-step $t-1$. See that the right side of the equation averages the heat from above, below, left-of, and right-of $(i, j)$, at time-step $t-1$. This means of evolving the heat through our gridded material is known as the *finite difference method*.

Let's using the finite difference method to figure out what the distribution of heat looks like throughout our steel square at time-step 1. Keep in mind that we need not update any of the outer-edges of the steel - those positions are held at a fixed heat. We'll start at the upper-left corner and get

\begin{equation}
u^{t=1}_{1,1} = \frac{0 + 100 + 0 + 100}{4} = 50\\
u^{t=1}_{1,2} = \frac{0 + 100 + 0 + 0}{4} = 25\\
\end{equation}

and so on, yielding the heat distribution at timestep-1 of:
```
                                100   100   100   100   100
                                100    50    25    50   100
                                100    25     0    25   100
                                100    50    25    50   100
                                100   100   100   100   100
```

Repeating this process again will produce the heat distribution at timestep-2, and so on. After many iterations, we see the entire region becomes 100 degrees. This is because the heat from the edges flows inward until everything is the same temperature. This stabilized distribution of heat is known as the *steady state*. If we change the boundary conditions, i.e. change what we clamp to the edges of our steel, we will observe different steady states.
_____no_output_____## Problem 1: A Simple Implementation of Finite Differences
Write a Python function that takes in a 2-dimensional numpy-array containing heat-values, and uses the finite difference method to produce the heat distribution for that material at the next time-step. Do this using simple for-loops to iterate over the values of the array.

Assume that the boundary-values of the array are fixed, so you need not update them. However, do *not* assume that the boundary values are all the same as one another, as they were in the preceding example.

Also, be careful not to change the content of the array that your function is given. You need to use the values in that array, unchanged, to compute the new heat distribution. Consider making use of `np.copy` to create a copy of the input array (so that your new array will have the appropriate boundary values)._____no_output_____

# make sure to execute this cell so that your function is defined
# you must re-run this cell any time you make a change to this function
import numpy as np
def evolve_heat_slow(u):
    """ Given a 2D array of heat-values (at fixed boundary), produces
        the new heat distribution after one iteration of the finite
        difference method.

        Parameters
        ----------
        u : numpy.ndarray shape=(M, N)
            An MxN array of heat values at time-step t-1.
            (M and N are both at least 2)

        Returns
        -------
        numpy.ndarray, shape=(M, N)
            An MxN array of heat values at time-step t.
        """
    new = np.copy(u)
    size = np.size(u)
    rows = np.size(u,0)
    columns = np.size(u,1)
    for r in range(1, rows - 1):
        for c in range(1, columns - 1):
            top = u[r-1, c]
            left = u[r, c-1]
            right = u[r, c+1]
            bottom = u[r+1, c]
            new[r,c] = (top+left+right+bottom)/4
    return new

    # student code goes here_____no_output_____from bwsi_grader.python.heat_dispersion import grader
grader(evolve_heat_slow, grade_ver=1)
============================== ALL TESTS PASSED! ===============================
Your submission code: bw4c328516b4c813fe5f8394f4f6b83799336f22f27044cb26efb23cf8
================================================================================

Armed with this function, we will find the steady state of a more finely-gridded sheet of steel, with a less trivial set of boundary heat-values.

We will create an 80x96 grid with the following boundary conditions:

- Along the top row, we linearly increase the heat from 0 to 300 degrees from left to right
- Along the bottom row, we fade from 0 to 80 degrees at the middle and back to 0 on the right
- Along the left side, we linearly increase from 0 degrees at the bottom to 90 at the top (note that the very corner point is 0 from the 0 -> 300 continuum above)
- Along the right side, we linearly increase the heat from 0 to 300 degrees from bottom to top_____no_output_____

# creating the 80x96-grid sheet with the non-trivial boundary conditions
# simply execute this cell; you need not change anything.

import numpy as np
# discretize our rectangle into an 80x96 grid
rows = 80
columns = 96
u = np.zeros((rows, columns))

# set up the boundary conditions
u[0] = np.linspace(0, 300, columns)                # top row runs 0 -> 300
u[1:,0] = np.linspace(90, 0, rows-1)               # left side goes 0 -> 90 bottom to top
u[-1,:columns//2] = np.linspace(0, 80, columns//2) # 0 (left) to 80 (middle) along the bottom
u[-1,columns//2:] = np.linspace(80, 0, columns//2) # 80 (middle) to 0 (left) along the bottom
u[:,-1] = np.linspace(300,0,rows)                  # 0 -> 300 bottom to top along the right_____no_output_____

Let's plot the initial condition for this steel sheet. You should see a "hot spot" in the top-right corner, and varying amounts of heat elsewhere on the boundary. Check that this corresponds to the boundary conditions that we imposed._____no_output_____

# execute this cell

# matplotlib is a Python library used for visualizing data
import matplotlib.pyplot as plt
from matplotlib.animation import FuncAnimation
%matplotlib notebook

fig, ax = plt.subplots()
ax.imshow(u, cmap='hot');_____no_output_____

Now, we will make an animation of the heat spreading through this material. However, our current implementation is too slow - let's time the amount of time required to evolve the heat in the material for 5000 iterations. This should take 20 sec - 1 minute._____no_output_____

import time
slow = u.copy()
start = time.time()
for _ in range(5000): # perform 5000 iterations to reach a steady state
    slow = evolve_heat_slow(slow)
t = round(time.time() - start, 1)
print("`evolve_heat_slow` took {} seconds to complete 5000 iterations".format(t))`evolve_heat_slow` took 45.3 seconds to complete 5000 iterations

# Problem 2: A Vectorized Version of Finite Differences
Use NumPy array arithmetic to vectorize the finite-difference method that you implemented in problem #1. Your code should not utilize any for-loops._____no_output_____

import numpy as np
def evolve_heat_fast(u):
    """ Given a 2D array of heat-values (at fixed boundary), produces
        the new heat distribution after one iteration of the finite
        difference method.

        Parameters
        ----------
        u : numpy.ndarray shape=(M, N)
            An MxN array of heat values at time-step t-1.
            (M and N are both at least 2)

        Returns
        -------
        numpy.ndarray, shape=(M, N)
            An MxN array of heat values at time-step t.
        """
    old_temps = np.copy(u)
    new_temps = np.copy(u)
    new_temps[1:-1,1:-1] = (1/4) * (old_temps[2:,1:-1] + old_temps[:-2,1:-1] + old_temps[1:-1,:-2] + old_temps[1:-1,2:])
    return new_temps_____no_output_____from bwsi_grader.python.heat_dispersion import grader
grader(evolve_heat_fast, grade_ver=2)
============================== ALL TESTS PASSED! ===============================
Your submission code: bw95ca3a7cfec977141125d15af9059617240297927b952f8dfae97d2a
================================================================================

Now let's use our vectorized code to perform 5000 iterations to evolve the heat in our system. This should be nearly 100-times faster than our original implementation._____no_output_____

# execute this cell

import time
fast = u.copy()
start = time.time()
all_frames = []
for _ in range(5000):
    all_frames.append(fast.copy())
    fast = evolve_heat_fast(fast)
t = round(time.time() - start, 1)
print("`evolve_heat_fast` took {} seconds to complete 5000 iterations".format(t))_____no_output_____

Plotting the distribution of heat after 5000 time-steps of evolution. _____no_output_____

# execute this cell

fig, ax = plt.subplots()
ax.imshow(fast, cmap='hot');_____no_output_____

Even better, let's plot an animation of the heat spreading through the steel sheet! The animation will loop back to the beginning after playing through._____no_output_____

# execute this cell

fig = plt.figure()
t = u.copy()
im = plt.imshow(t, animated=True, cmap='hot')

def updatefig(*args):
    im.set_array(all_frames[args[0]])
    return im,

ani = FuncAnimation(fig, updatefig, range(5000), interval=1, blit=True, repeat=True,
                    repeat_delay=1000)
plt.show()_____no_output_____

Try creating your own boundary conditions for the temperature distribution on this steel sheet. Reuse the code from above to animate how the heat spreads through it. Also congratulate yourself for numerically solving a fixed-boundary partial differential equation :)_____no_output_____