# Notebook from PaMcD/Create-Your-Own-Image-Classifier
Path: Image Classifier Project.ipynb

# Developing an AI application

Going forward, AI algorithms will be incorporated into more and more everyday applications. For example, you might want to include an image classifier in a smart phone app. To do this, you'd use a deep learning model trained on hundreds of thousands of images as part of the overall application architecture. A large part of software development in the future will be using these types of models as common parts of applications.

In this project, you'll train an image classifier to recognize different species of flowers. You can imagine using something like this in a phone app that tells you the name of the flower your camera is looking at. In practice you'd train this classifier, then export it for use in your application. We'll be using [this dataset](http://www.robots.ox.ac.uk/~vgg/data/flowers/102/index.html) of 102 flower categories, you can see a few examples below.

The project is broken down into multiple steps:

* Load and preprocess the image dataset
* Train the image classifier on your dataset
* Use the trained classifier to predict image content

We'll lead you through each part which you'll implement in Python.

When you've completed this project, you'll have an application that can be trained on any set of labeled images. Here your network will be learning about flowers and end up as a command line application. But, what you do with your new skills depends on your imagination and effort in building a dataset. For example, imagine an app where you take a picture of a car, it tells you what the make and model is, then looks up information about it. Go build your own dataset and make something new.

First up is importing the packages you'll need. It's good practice to keep all the imports at the beginning of your code. As you work through this notebook and find you need to import a package, make sure to add the import up here._____no_output_____

# Imports here
import torch
import torchvision
from torch import nn, optim
from torchvision import transforms
from torchvision.datasets import ImageFolder
from torch.utils import data
from PIL import Image
import numpy as np
import os, random
import json
import signal

from contextlib import contextmanager

import requests

import matplotlib
import matplotlib.pyplot as plt

%config IPCompleter.greedy=True_____no_output_____

## Load the data

Here you'll use `torchvision` to load the data ([documentation](http://pytorch.org/docs/0.3.0/torchvision/index.html)). The data should be included alongside this notebook, otherwise you can [download it here](https://s3.amazonaws.com/content.udacity-data.com/nd089/flower_data.tar.gz). The dataset is split into three parts, training, validation, and testing. For the training, you'll want to apply transformations such as random scaling, cropping, and flipping. This will help the network generalize leading to better performance. You'll also need to make sure the input data is resized to 224x224 pixels as required by the pre-trained networks.

The validation and testing sets are used to measure the model's performance on data it hasn't seen yet. For this you don't want any scaling or rotation transformations, but you'll need to resize then crop the images to the appropriate size.

The pre-trained networks you'll use were trained on the ImageNet dataset where each color channel was normalized separately. For all three sets you'll need to normalize the means and standard deviations of the images to what the network expects. For the means, it's `[0.485, 0.456, 0.406]` and for the standard deviations `[0.229, 0.224, 0.225]`, calculated from the ImageNet images.  These values will shift each color channel to be centered at 0 and range from -1 to 1.
 _____no_output_____

data_dir = 'flowers'
train_dir = data_dir + '/train'
valid_dir = data_dir + '/valid'
test_dir = data_dir + '/test'_____no_output_____# TODO: Define your transforms for the training, validation, and testing sets
train_transforms = transforms.Compose([transforms.RandomRotation(30),
                                     transforms.RandomResizedCrop(224),
                                     transforms.RandomHorizontalFlip(),
                                     transforms.ToTensor(),
                                     transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
                                     ])

valid_and_test_transform = transforms.Compose([transforms.Resize(255),
                                     transforms.CenterCrop(224),
                                     transforms.ToTensor(),
                                     transforms.Normalize([0.485, 0.456, 0.406], [0.229, 0.224, 0.225])
                                     ])

# TODO: Load the datasets with ImageFolder
# image_datasets =
train_data = ImageFolder(root=train_dir, transform=train_transforms)
valid_data = ImageFolder(root=valid_dir, transform=valid_and_test_transform)
test_data = ImageFolder(root=test_dir, transform=valid_and_test_transform)

# TODO: Using the image datasets and the trainforms, define the dataloaders
# dataloaders =
train_data_loader = data.DataLoader(train_data, batch_size=64, shuffle=True)
valid_data_loader = data.DataLoader(valid_data, batch_size=64)
test_data_loader = data.DataLoader(test_data, batch_size=64)
_____no_output_____

### Label mapping

You'll also need to load in a mapping from category label to category name. You can find this in the file `cat_to_name.json`. It's a JSON object which you can read in with the [`json` module](https://docs.python.org/2/library/json.html). This will give you a dictionary mapping the integer encoded categories to the actual names of the flowers._____no_output_____

with open('cat_to_name.json', 'r') as f:
    cat_to_name = json.load(f)
_____no_output_____

# Building and training the classifier

Now that the data is ready, it's time to build and train the classifier. As usual, you should use one of the pretrained models from `torchvision.models` to get the image features. Build and train a new feed-forward classifier using those features.

We're going to leave this part up to you. Refer to [the rubric](https://review.udacity.com/#!/rubrics/1663/view) for guidance on successfully completing this section. Things you'll need to do:

* Load a [pre-trained network](http://pytorch.org/docs/master/torchvision/models.html) (If you need a starting point, the VGG networks work great and are straightforward to use)
* Define a new, untrained feed-forward network as a classifier, using ReLU activations and dropout
* Train the classifier layers using backpropagation using the pre-trained network to get the features
* Track the loss and accuracy on the validation set to determine the best hyperparameters

We've left a cell open for you below, but use as many as you need. Our advice is to break the problem up into smaller parts you can run separately. Check that each part is doing what you expect, then move on to the next. You'll likely find that as you work through each part, you'll need to go back and modify your previous code. This is totally normal!

When training make sure you're updating only the weights of the feed-forward network. You should be able to get the validation accuracy above 70% if you build everything right. Make sure to try different hyperparameters (learning rate, units in the classifier, epochs, etc) to find the best model. Save those hyperparameters to use as default values in the next part of the project.

One last important tip if you're using the workspace to run your code: To avoid having your workspace disconnect during the long-running tasks in this notebook, please read in the earlier page in this lesson called Intro to
GPU Workspaces about Keeping Your Session Active. You'll want to include code from the workspace_utils.py module.

**Note for Workspace users:** If your network is over 1 GB when saved as a checkpoint, there might be issues with saving backups in your workspace. Typically this happens with wide dense layers after the convolutional layers. If your saved checkpoint is larger than 1 GB (you can open a terminal and check with `ls -lh`), you should reduce the size of your hidden layers and train again._____no_output_____

# TODO: Build and train your network
model = torchvision.models.vgg16(pretrained=True)

# freeze model parameters
for param in model.parameters():
    param.requires_grad = False

in_features_of_pretrained_model = model.classifier[0].in_features

# alter the classifier so that it has 102 out features (i.e. len(cat_to_name.json))
number_of_flower_classes = len(train_data.classes)
classifier = nn.Sequential(nn.Linear(in_features=in_features_of_pretrained_model, out_features=2048, bias=True),
                           nn.ReLU(inplace=True),
                           nn.Dropout(p=0.2),
                           nn.Linear(in_features=2048, out_features=number_of_flower_classes, bias=True),
                           nn.LogSoftmax(dim=1)
                          )

model.classifier = classifier

_____no_output_____

DELAY = INTERVAL = 4 * 60  # interval time in seconds
MIN_DELAY = MIN_INTERVAL = 2 * 60
KEEPALIVE_URL = "https://nebula.udacity.com/api/v1/remote/keep-alive"
TOKEN_URL = "http://metadata.google.internal/computeMetadata/v1/instance/attributes/keep_alive_token"
TOKEN_HEADERS = {"Metadata-Flavor":"Google"}

def _request_handler(headers):
    def _handler(signum, frame):
        requests.request("POST", KEEPALIVE_URL, headers=headers)
    return _handler

@contextmanager
def active_session(delay=DELAY, interval=INTERVAL):
    """
    Example:

    from workspace_utils import active session

    with active_session():
        # do long-running work here
    """
    token = requests.request("GET", TOKEN_URL, headers=TOKEN_HEADERS).text
    headers = {'Authorization': "STAR " + token}
    delay = max(delay, MIN_DELAY)
    interval = max(interval, MIN_INTERVAL)
    original_handler = signal.getsignal(signal.SIGALRM)
    try:
        signal.signal(signal.SIGALRM, _request_handler(headers))
        signal.setitimer(signal.ITIMER_REAL, delay, interval)
        yield
    finally:
        signal.signal(signal.SIGALRM, original_handler)
        signal.setitimer(signal.ITIMER_REAL, 0)

def keep_awake(iterable, delay=DELAY, interval=INTERVAL):
    """
    Example:

    from workspace_utils import keep_awake

    for i in keep_awake(range(5)):
        # do iteration with lots of work here
    """
    with active_session(delay, interval): yield from iterable_____no_output_____
# train

# specify criterion
criterion = nn.NLLLoss()

# specify optimizer, using only classifer params. don't want to change the rest of VGG19
optimizer = optim.Adam(model.classifier.parameters(), lr=0.003)

# move device to cpu, before moving it to cuda/gpu
device = 'cuda'
model.to(device)

# init variables for tracking loss/steps etc.
epochs = 5
print_every = 20

with active_session():

    # for each epoch
    for e in range(epochs):
        step = 0
        running_train_loss = 0
        running_valid_loss = 0

        # for each batch of images
        for images, labels in train_data_loader:
            step += 1

            # turn model to train mode
            model.train()

            # move images and model to device
            images, labels = images.to(device), labels.to(device)

            # zeroise grad
            optimizer.zero_grad()

            # forward
            outputs = model(images)

            # loss
            train_loss = criterion(outputs, labels)

            # backward
            train_loss.backward()

            # step
            optimizer.step()

            running_train_loss += train_loss.item()

            if step % print_every == 0 or step == 1 or step == len(train_data_loader):
                print("Epoch: {}/{} Batch % Complete: {:.2f}%".format(e+1, epochs, (step)*100/len(train_data_loader)))

        # validate
        # turn model to eval mode
        # turn on no_grad

        model.eval()
        with torch.no_grad():
            # for each batch of images
            running_accuracy = 0
            running_valid_loss = 0
            for images, labels in valid_data_loader:

                # move images and model to device
                images, labels = images.to(device), labels.to(device)

                # forward
                outputs = model(images)

                # loss
                valid_loss = criterion(outputs, labels)
                running_valid_loss += valid_loss.item()

                # accuracy
                ps = torch.exp(outputs)
                top_p, top_class = ps.topk(1, dim=1)
                equals = top_class == labels.view(*top_class.shape)
                running_accuracy += torch.mean(equals.type(torch.FloatTensor)).item()

        # print stats
        average_train_loss = running_train_loss/len(train_data_loader)
        average_valid_loss = running_valid_loss/len(valid_data_loader)
        accuracy = running_accuracy/len(valid_data_loader)
        print("Train Loss: {:.3f}".format(average_train_loss))
        print("Valid Loss: {:.3f}".format(average_valid_loss))
        print("Accuracy: {:.3f}%".format(accuracy*100))

Epoch: 1/5 Batch % Complete: 0.97%
Epoch: 1/5 Batch % Complete: 19.42%
Epoch: 1/5 Batch % Complete: 38.83%
Epoch: 1/5 Batch % Complete: 58.25%
Epoch: 1/5 Batch % Complete: 77.67%
Epoch: 1/5 Batch % Complete: 97.09%
Epoch: 1/5 Batch % Complete: 100.00%
Train Loss: 5.458
Valid Loss: 1.503
Accuracy: 60.356%
Epoch: 2/5 Batch % Complete: 0.97%
Epoch: 2/5 Batch % Complete: 19.42%
Epoch: 2/5 Batch % Complete: 38.83%
Epoch: 2/5 Batch % Complete: 58.25%
Epoch: 2/5 Batch % Complete: 77.67%
Epoch: 2/5 Batch % Complete: 97.09%
Epoch: 2/5 Batch % Complete: 100.00%
Train Loss: 1.950
Valid Loss: 1.170
Accuracy: 69.827%
Epoch: 3/5 Batch % Complete: 0.97%
Epoch: 3/5 Batch % Complete: 19.42%
Epoch: 3/5 Batch % Complete: 38.83%
Epoch: 3/5 Batch % Complete: 58.25%
Epoch: 3/5 Batch % Complete: 77.67%
Epoch: 3/5 Batch % Complete: 97.09%
Epoch: 3/5 Batch % Complete: 100.00%
Train Loss: 1.774
Valid Loss: 1.001
Accuracy: 74.413%
Epoch: 4/5 Batch % Complete: 0.97%
Epoch: 4/5 Batch % Complete: 19.42%
Epoch: 4/5 Batch % Complete: 38.83%
Epoch: 4/5 Batch % Complete: 58.25%
Epoch: 4/5 Batch % Complete: 77.67%
Epoch: 4/5 Batch % Complete: 97.09%
Epoch: 4/5 Batch % Complete: 100.00%
Train Loss: 1.634
Valid Loss: 0.967
Accuracy: 72.418%
Epoch: 5/5 Batch % Complete: 0.97%
Epoch: 5/5 Batch % Complete: 19.42%
Epoch: 5/5 Batch % Complete: 38.83%
Epoch: 5/5 Batch % Complete: 58.25%
Epoch: 5/5 Batch % Complete: 77.67%
Epoch: 5/5 Batch % Complete: 97.09%
Epoch: 5/5 Batch % Complete: 100.00%
Train Loss: 1.629
Valid Loss: 0.951
Accuracy: 76.163%

## Testing your network

It's good practice to test your trained network on test data, images the network has never seen either in training or validation. This will give you a good estimate for the model's performance on completely new images. Run the test images through the network and measure the accuracy, the same way you did validation. You should be able to reach around 70% accuracy on the test set if the model has been trained well._____no_output_____

# TODO: Do validation on the test set

model.eval()
with torch.no_grad():
    # for each batch of images
    running_accuracy = 0
    running_loss = 0
    for images, labels in test_data_loader:

        # move images and model to device
        images, labels = images.to(device), labels.to(device)

        # forward
        outputs = model(images)

        # loss
        loss = criterion(outputs, labels)
        running_loss += loss.item()

        # accuracy
        ps = torch.exp(outputs)
        top_p, top_class = ps.topk(1, dim=1)
        equals = top_class == labels.view(*top_class.shape)
        accuracy += torch.mean(equals.type(torch.FloatTensor)).item()

average_loss = running_loss/len(valid_data_loader)
total_accuracy = accuracy/len(valid_data_loader)
print("Test Loss: {:.3f}".format(average_loss))
print("Test Accuracy: {:.3f}".format(total_accuracy*100))       Test Loss: 1.022
Test Accuracy: 81.266

## Save the checkpoint

Now that your network is trained, save the model so you can load it later for making predictions. You probably want to save other things such as the mapping of classes to indices which you get from one of the image datasets: `image_datasets['train'].class_to_idx`. You can attach this to the model as an attribute which makes inference easier later on.

```model.class_to_idx = image_datasets['train'].class_to_idx```

Remember that you'll want to completely rebuild the model later so you can use it for inference. Make sure to include any information you need in the checkpoint. If you want to load the model and keep training, you'll want to save the number of epochs as well as the optimizer state, `optimizer.state_dict`. You'll likely want to use this trained model in the next part of the project, so best to save it now._____no_output_____

# TODO: Save the checkpoint
model.class_to_idx = train_data.class_to_idx
checkpoint = {'classifier': model.classifier,
              'state_dict': model.state_dict(),
              'epochs': epochs,
              'optim_stat_dict': optimizer.state_dict(),
              'class_to_idx': train_data.class_to_idx
             }

torch.save(checkpoint, 'checkpoint.pth')_____no_output_____

## Loading the checkpoint

At this point it's good to write a function that can load a checkpoint and rebuild the model. That way you can come back to this project and keep working on it without having to retrain the network.
_____no_output_____

# TODO: Write a function that loads a checkpoint and rebuilds the model

def load_checkpoint(filepath):
    checkpoint = torch.load(filepath)
    model = torchvision.models.vgg16(pretrained=True)
    # freeze model parameters
    for param in model.parameters():
        param.requires_grad = False

    model.classifier = checkpoint['classifier']
    model.load_state_dict(checkpoint['state_dict'])
    model.class_to_idx = checkpoint['class_to_idx']

    return model_____no_output_____

# Inference for classification

Now you'll write a function to use a trained network for inference. That is, you'll pass an image into the network and predict the class of the flower in the image. Write a function called `predict` that takes an image and a model, then returns the top $K$ most likely classes along with the probabilities. It should look like

```python
probs, classes = predict(image_path, model)
print(probs)
print(classes)
> [ 0.01558163  0.01541934  0.01452626  0.01443549  0.01407339]
> ['70', '3', '45', '62', '55']
```

First you'll need to handle processing the input image such that it can be used in your network.

## Image Preprocessing

You'll want to use `PIL` to load the image ([documentation](https://pillow.readthedocs.io/en/latest/reference/Image.html)). It's best to write a function that preprocesses the image so it can be used as input for the model. This function should process the images in the same manner used for training.

First, resize the images where the shortest side is 256 pixels, keeping the aspect ratio. This can be done with the [`thumbnail`](http://pillow.readthedocs.io/en/3.1.x/reference/Image.html#PIL.Image.Image.thumbnail) or [`resize`](http://pillow.readthedocs.io/en/3.1.x/reference/Image.html#PIL.Image.Image.thumbnail) methods. Then you'll need to crop out the center 224x224 portion of the image.

Color channels of images are typically encoded as integers 0-255, but the model expected floats 0-1. You'll need to convert the values. It's easiest with a Numpy array, which you can get from a PIL image like so `np_image = np.array(pil_image)`.

As before, the network expects the images to be normalized in a specific way. For the means, it's `[0.485, 0.456, 0.406]` and for the standard deviations `[0.229, 0.224, 0.225]`. You'll want to subtract the means from each color channel, then divide by the standard deviation.

And finally, PyTorch expects the color channel to be the first dimension but it's the third dimension in the PIL image and Numpy array. You can reorder dimensions using [`ndarray.transpose`](https://docs.scipy.org/doc/numpy-1.13.0/reference/generated/numpy.ndarray.transpose.html). The color channel needs to be first and retain the order of the other two dimensions._____no_output_____

def process_image(image_path):
    ''' Scales, crops, and normalizes a PIL image for a PyTorch model,
        returns an Numpy array
    '''
    pil_image = Image.open(image_path)

    # TODO: Process a PIL image for use in a PyTorch model

    # reize
    pil_image.resize((256,256))

    # centre crop
    width, height = pil_image.size   # Get dimensions
    new_width, new_height = 224, 224

    left = round((width - new_width)/2)
    top = round((height - new_height)/2)
    x_right = round(width - new_width) - left
    x_bottom = round(height - new_height) - top
    right = width - x_right
    bottom = height - x_bottom

    # Crop the center of the image
    pil_image = pil_image.crop((left, top, right, bottom))

    # convert colour channel from 0-255, to 0-1
    np_image = np.array(pil_image)/255

    # normalize for model
    mean = np.array([0.485, 0.456, 0.406])
    std = np.array([0.229, 0.224, 0.225])
    np_image = (np_image - mean)/std

    # tranpose color channge to 1st dim
    np_image = np_image.transpose((2 , 0, 1))

    # convert to Float Tensor
    tensor = torch.from_numpy(np_image)
    tensor = tensor.type(torch.FloatTensor)

    # return tensor
    return tensor
    _____no_output_____

To check your work, the function below converts a PyTorch tensor and displays it in the notebook. If your `process_image` function works, running the output through this function should return the original image (except for the cropped out portions)._____no_output_____

def imshow(image, ax=None, title=None):
    """Imshow for Tensor."""
    if ax is None:
        fig, ax = plt.subplots()

    # PyTorch tensors assume the color channel is the first dimension
    # but matplotlib assumes is the third dimension
    image = image.numpy().transpose((1, 2, 0))

    # Undo preprocessing
    mean = np.array([0.485, 0.456, 0.406])
    std = np.array([0.229, 0.224, 0.225])
    image = std * image + mean

    # Image needs to be clipped between 0 and 1 or it looks like noise when displayed
    image = np.clip(image, 0, 1)

    ax.imshow(image)

    return ax_____no_output_____

## Class Prediction

Once you can get images in the correct format, it's time to write a function for making predictions with your model. A common practice is to predict the top 5 or so (usually called top-$K$) most probable classes. You'll want to calculate the class probabilities then find the $K$ largest values.

To get the top $K$ largest values in a tensor use [`x.topk(k)`](http://pytorch.org/docs/master/torch.html#torch.topk). This method returns both the highest `k` probabilities and the indices of those probabilities corresponding to the classes. You need to convert from these indices to the actual class labels using `class_to_idx` which hopefully you added to the model or from an `ImageFolder` you used to load the data ([see here](#Save-the-checkpoint)). Make sure to invert the dictionary so you get a mapping from index to class as well.

Again, this method should take a path to an image and a model checkpoint, then return the probabilities and classes.

```python
probs, classes = predict(image_path, model)
print(probs)
print(classes)
> [ 0.01558163  0.01541934  0.01452626  0.01443549  0.01407339]
> ['70', '3', '45', '62', '55']
```_____no_output_____

def predict(image_path, model, topk=5, device="cuda"):
    ''' Predict the class (or classes) of an image using a trained deep learning model.
    '''
    image = process_image(image_path)
    image = image.unsqueeze(0)

    # move to device
    image = image.to(device)
    model.eval()
    with torch.no_grad():
        ps = torch.exp(model(image))

    idx_to_flower = {v:cat_to_name[k] for k, v in model.class_to_idx.items()}

    top_ps, top_classes = ps.topk(topk, dim=1)
    predicted_flowers = [idx_to_flower[i] for i in top_classes.tolist()[0]]

    return top_ps.tolist()[0], predicted_flowers

    # TODO: Implement the code to predict the class from an image file_____no_output_____

## Sanity Checking

Now that you can use a trained model for predictions, check to make sure it makes sense. Even if the testing accuracy is high, it's always good to check that there aren't obvious bugs. Use `matplotlib` to plot the probabilities for the top 5 classes as a bar graph, along with the input image. It should look like this:

You can convert from the class integer encoding to actual flower names with the `cat_to_name.json` file (should have been loaded earlier in the notebook). To show a PyTorch tensor as an image, use the `imshow` function defined above._____no_output_____

# TODO: Display an image along with the top 5 classes
# load the model
model = load_checkpoint("checkpoint.pth")
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model = model.to(device)

# print(device)_____no_output_____

def classify_random_image():
    # pick random image from test folder
    random_label_idx = random.choice(os.listdir(test_dir))
    random_image_file = random.choice(os.listdir("{}/{}".format(test_dir, random_label_idx)))
    image_path = os.path.join(test_dir, random_label_idx, random_image_file)
    actual_flower_name = cat_to_name[random_label_idx]

    # process image
    image = process_image(image_path)
    top_ps, top_classes = predict(image_path, model)

    top1_flower_name = top_classes[0]

    # graph
    if top1_flower_name == actual_flower_name:
        classified_correctly = "Yes, Top Prediction"
    elif actual_flower_name in top_classes:
        classified_correctly = "Yes in Top 5 Prediction"
    else:
        classified_correctly = "No"

    plt.barh(top_classes, top_ps)
    plt.title("Actual Class = {} \n Correctly Classified = {}".format(actual_flower_name, classified_correctly))
    plt.show()
    imshow(image)

_____no_output_____classify_random_image()_____no_output_____


# Notebook from whitead/hoomd-tf
Path: examples/08. N-Body Neural Force Fields.ipynb

## 08. N-Body Neural Force Fields_____no_output_____

# disable GPU. Remove this if you've compiled HOOMD for GPU
import os
os.environ['CUDA_VISIBLE_DEVICES'] = '-1'

# import the hoomd, htf packages
import hoomd
import hoomd.htf as htf
import tensorflow as tf
import matplotlib.pyplot as plt
from MDAnalysis import Universe_____no_output_____

## Build the SimModel

Here we build a 2 hidden-layer N-body neural force fields. The inputs are the nearest N neighbors._____no_output_____

class NlistNN(htf.SimModel):
    def setup(self, dim, top_neighs):
        self.dense1 = tf.keras.layers.Dense(dim)
        self.dense2 = tf.keras.layers.Dense(dim)
        self.last = tf.keras.layers.Dense(1)
        self.top_neighs = top_neighs

    def compute(self, nlist, positions, box):
        rinv = htf.nlist_rinv(nlist)
        # closest neighbors have largest value in 1/r, take top
        sorted_n = tf.reshape(tf.sort(rinv, axis=1, direction='DESCENDING'), [-1, self.nneighbor_cutoff])
        top_n = sorted_n[:, :self.top_neighs]
        # run through NN
        x = self.dense1(top_n)
        x = self.dense2(x)
        pair_energy = self.last(x)
        # get per-particle energy
        energy = tf.reduce_sum(pair_energy, axis=1)
        forces = htf.compute_nlist_forces(nlist, energy)
        # don't output last column of forces, pairwise energy, since it's meaningless here
        return forces[:,:3], energy
_____no_output_____

## Training from a Trajectory File

We will compile our model and then train against a trajectory. If this were a real example, we would be training against the forces in the trajectory but this trajectory (to save space) has no forces. Instead, we will train to make our forces match the positions of the trajctory, which makes no physical sense, but it has the correct dimensions._____no_output_____

model = NlistNN(128, dim=16, top_neighs=8)
# when we compile, add a None loss so as to not train the second output
# which is energy
model.compile('Adam', ['MeanSquaredError', None])
universe = Universe('test_topol.pdb', 'test_traj.trr')
losses = []
for epoch in range(3):
    for inputs, ts in htf.iter_from_trajectory(128, universe, r_cut=25, period=5):
        #labels = ts.forces
        labels = ts.positions
        loss = model.train_on_batch(inputs, labels)
        losses.append(loss)100%|██████████| 801/801 [00:02<00:00, 292.22it/s]
100%|██████████| 801/801 [00:01<00:00, 548.24it/s]
100%|██████████| 801/801 [00:01<00:00, 537.72it/s]
plt.plot(losses)
plt.show()_____no_output_____

## Inferrence with the Model

Now we will run a particle simulation with our model._____no_output_____

########### Hoomd-Sim Code ################
hoomd.context.initialize('--mode=cpu')

tfcompute = htf.tfcompute(model)

# create a square lattice
system = hoomd.init.create_lattice(unitcell=hoomd.lattice.sq(a=1.2),
                                    n=[16,16])
nlist = hoomd.md.nlist.cell()
# NVT ensemble
hoomd.md.integrate.mode_standard(dt=1e-4)
hoomd.md.integrate.nvt(group=hoomd.group.all(), kT=0.1, tau=0.5).randomize_velocities(seed=1)
tfcompute.attach(nlist, r_cut=5, save_output_period=5)
#run with our silly model
hoomd.run(1e3)HOOMD-blue v2.9.0 CUDA (10.1) DOUBLE HPMC_MIXED SSE SSE2 SSE3 SSE4_1 SSE4_2 AVX AVX2
Compiled: 03/03/2021
Copyright (c) 2009-2019 The Regents of the University of Michigan.
-----
You are using HOOMD-blue. Please cite the following:
* J A Anderson, C D Lorenz, and A Travesset. "General purpose molecular dynamics
  simulations fully implemented on graphics processing units", Journal of
  Computational Physics 227 (2008) 5342--5359
* J Glaser, T D Nguyen, J A Anderson, P Lui, F Spiga, J A Millan, D C Morse, and
  S C Glotzer. "Strong scaling of general-purpose molecular dynamics simulations
  on GPUs", Computer Physics Communications 192 (2015) 97--107
-----
no CUDA-capable device is detected
HOOMD-blue is running on the CPU
notice(2): Group "all" created containing 256 particles
notice(2): Force mode is FORCE_MODE.tf2hoomd
notice(2): Starting TensorflowCompute
notice(2): completed reallocate
notice(2): Setting flag indicating virial modification will occur
notice(2): -- Neighborlist exclusion statistics -- :
notice(2): Particles with 0 exclusions             : 256
notice(2): Neighbors included by diameter          : no
notice(2): Neighbors excluded when in the same body: no
** starting run **
Time 00:00:05 | Step 1000 / 1000 | TPS 179.119 | ETA 00:00:00
Average TPS: 179.054
---------
-- Neighborlist stats:
0 normal updates / 10 forced updates / 0 dangerous updates
n_neigh_min: 62 / n_neigh_max: 68 / n_neigh_avg: 64.4219
shortest rebuild period: 100
-- Cell list stats:
Dimension: 3, 3, 1
n_min    : 25 / n_max: 34 / n_avg: 28.4444
** run complete **

## Analysis

This model is not properly trained so there is not much analysis to do. We'll just take a look at the energy_____no_output_____

e = tf.reduce_sum(tfcompute.outputs[0], axis=1)
plt.plot(e)
plt.show()_____no_output_____