# Notebook from BohanMeng/storytelling-with-data
Path: data-stories/VT-DoH/fanrui/ignored_treatment_md.ipynb

_____no_output_____#link to youtube
[ignored_treatment](https://www.youtube.com/watch?v=XJc5zbxlO0k)_____no_output_____#overview

1. _What was your main question?_
Are other diseases or medical appointment canceled or retarded due to the pendamic? How?

2. _How did you approach exploring and/or answering your question?
What data did you use?_
I tell my life story and use data from two sources: German questionaire answered by doctors in 2020, regarding the number of patients/appointments they have; a larger data set about the expected and actual Medicare physician fee schedule spending in US 2020.

3. _What sorts of data science tools did you use?_
Pandas and Plotly

4. _What did you find or accomplish (be brief)? For example, did you find any interesting results or insights? Or did you solve an interesting problem that might be useful in another project?_
Yes, the part on what kinds of medicare service is more likely to be ignored under pendamic makes sense. But the between-states difference needs more detailled data and analysis._____no_output_____

# Below is my outline_____no_output_____

#Outline

1.	Story of my surgery before covid, and after removal all the suture, I came to France, where I found that they missed one suture. What to do?
2.	The hospitals are closed, and I don’t have a local health care
3.	Indeed, the incidence like such is common during pandemic, for example…
4.	The lack of data seems to tell us the impact of pandemic that takes the resources from other medical treatments(1.5min)
5.	Graphs (3min)
6.	Back to the first story, how I remove the suture by myself
_____no_output_____

#below is my narration text_____no_output_____

#Written narrative

In december 2019, I had a small surgery on the left side of my nose in my hometown in China. The suture was removed in a week and I flied to France for a language exam. After the exam, I went to visit friends in Germany, and finally had time to look at myself in the mirror. I suddenly noticed a blue thread on my surgical wound. I looked **5% like a modern Frankenstein**.  I didn’t know what to do.
Covid had its first outburst in China that the hospitals, schools and airports were closed. it was unlikely that my parents could go to the hospital to the nurse or the doctor that was responsible for me.
I was not able to go back home and did not have a French or German health insurance, neither did I want to go to a hospital to expose myself to potential virus.

During the pandemic, there are other patients like me who is not able to or unwilling to go to an emergency room or a scheduled appointment.
In Germany, where I noticed my unremoved suture thread, doctors reported the general trend of cancelation in selective surgeries.
in the same survey, more than half of the doctors observed a decrease in oncological patients coming for consultation.
And there were also less emergent surgeries, which may tell us that home is a safe place? when people stay at home, they seem less likely to be injured.

In the US, we see a similar pattern from the Medicare physician fee schedule **(MPFS)** services. Every year every state has a MPFS spending budget for each specialties in medicare, and at the end of the year, we calculate the actual spending. In 2020, the estimated shortfall in MPFS spending associated with the pandemic was **$13.9 billion (14 percent)**. And we witnessed reductions for all states and every major specialty. Yet, as you can imagine, the percentage decrease depends on the specialties. The selective treatments are more likely to be ignored compared to more life-threatening problems.
The following three specialties are the ones with the highest percentage reduction in spending: **physical therapy** – it’s reasonable that people stopped to do physical exercise at the beginning of pandemic; **optometry** – I have to admit I still wear my old glasses that I got 3 years ago. And **otolaryngology**, namely related to nose, throat and ear, the domain that my surgery belongs.  The three specialties’ actual spending is **less than 80 % of the expected**.
In comparison, The least influenced specialties are **infectious disease, interventional radiology, and nephrology**. does not belong to the specialty of infectious disease, as it was not expected in the spending before 2020.
Like I mentioned before, the MPFS spending dropped in in every state, while the percentage drop is more striking in some states than the other. In this graph, the states on the left side has the highest percentage drop, while the ones on the right have the lowest percentage drop.
How can we explain the difference bt states?
I mean, for example, a small state’s non-covid medicare spending can be more influenced by covid, for an easier application of quarantine policy, and a lower budget. However, on this graph, we see that the states with a higher MPFS spending, in red or yellow, like Minnesota Okahoma, and South Carolina have quite different percentage drops. Minnesota dropped by **22%**, while South Caroline only **8%**. Same for Vermont, NH and Idaho. Therefore, to think about difference bt states, we need other insights or to look at specific difference and relate them to local quarantine policies etc.

At this point, If you still remember the blue thread on my nose, I can tell you my solution at that time. From my senior neuroscience lab class, I learned basic suture. My knowledge was telling me that it’s an undissovable thread hidden by blood scab and forgotten by the nurse. So I spent 2 hours to removed it by myself. Happy ending. But I was actually lucky that I safely removed the thread and it was not connected to a knot or other delicate suture. to be honest, I hope people can get prompt treatment and diagnostic in the post-pandemic time.
_____no_output_____#Downloading the data

data from:

[German non-university questionaire 2020](https://bmcsurg.biomedcentral.com/articles/10.1186/s12893-020-00970-x/tables/4)

[US MPFS spending 2020](https://www.ama-assn.org/system/files/2020-prp-covid-impact-medicare-physician-spending.pdf)_____no_output_____#Running the code

[code in colab](https://colab.research.google.com/drive/1kCgP697FHf2-I-0C3Sd_95oz5TMAS4pq)_____no_output_____#Contributing to the code

Tell other people how they can contribute to the project you've started. Specifically:

1. _What are the most obvious next steps?_
To look at detailed spending in a specific region, instead of national data. To find a rawer but more detailed dataset.

2. _What are some questions that your work raises?_
How do states differ in their MPFS spending? Why and where does the difference exist? Why are some medicare specialties more impacted by covid than the others?

3. _What challenges remain?_
There are complexer and other factors contribute to the difference in spending in medicare that may or maynot be related to covid. It's hard to draw a conclusion.

4. _Are there any known bugs or problems with your approach that someone continuing your project should be aware of?_
I am intimidated to draw any conclusion in this project, as my data are more exploratory than complete. I think in the future I will look at smaller dataset, or my personal data that I know better, and feel more confident to make observations and suggestions._____no_output_____#Acknowledgements

Thanks for Jeremy's office hour and the two debug sessions that helped my coding process. _____no_output_____


# Notebook from tjustorm/Multi-resBind
Path: jupyter_notebooks/Fig5.ipynb

<code>
import os
import numpy as np
import h5py
import scipy.io
np.random.seed(7)  # for reproducibility

import keras.backend as K
from keras.models import Model, load_model

import matplotlib.pyplot as plt
import matplotlib
# from concise.utils.plot import seqlogo, seqlogo_fig

import sys
sys.path.append("../Scripts/")
from IntegratedGradients import *
from util_funcs import *
from plotseqlogo import seqlogo, seqlogo_fig
Using TensorFlow backend.
../Scripts/plotseqlogo.py:41: FutureWarning: arrays to stack must be passed as a "sequence" type such as list or tuple. Support for non-sequence iterables such as generators is deprecated as of NumPy 1.16 and will raise an error in the future.
  min_coords = np.vstack(data.min(0) for data in polygons_data).min(0)
../Scripts/plotseqlogo.py:42: FutureWarning: arrays to stack must be passed as a "sequence" type such as list or tuple. Support for non-sequence iterables such as generators is deprecated as of NumPy 1.16 and will raise an error in the future.
  max_coords = np.vstack(data.max(0) for data in polygons_data).max(0)
def load_data_PU(path_to_data):

    data = h5py.File(path_to_data,'r')
    X_test_seq = np.transpose(np.array(data['test_in_seq']),axes=(0,2,1))
    X_test_annotation = np.transpose(np.array(data['test_in_annotation']),axes=(0,2,1))
    X_test_region = np.transpose(np.array(data['test_in_region']),axes=(0, 2, 1))
    y_test_RBP = np.array(data['test_out'])
    y_test_name= np.array(data['test_name'])
    y_train= np.array(data['train_out'])
    data.close()

    return X_test_seq, X_test_annotation, X_test_region, y_test_RBP, y_test_name, y_train
_____no_output_____# add the absolute path of data directory to the path_to_data
path_to_data = "/Users/storm/git/Multi-resBind/Data/"
path_to_datalow = path_to_data + "data_RBPslow_RNAplfold.h5"
X_test_seq_low,X_test_annotation_low, X_test_region_low, y_test_RBP_low, y_test_name_low, y_train_low = load_data_PU(path_to_datalow)
print (X_test_seq_low.shape)
print (X_test_annotation_low.shape)
print (X_test_region_low.shape)
print (y_test_RBP_low.shape)
print (y_test_name_low.shape)
print (y_train_low.shape)(7969, 150, 4)
(7969, 150, 2)
(7969, 250, 4)
(7969, 27)
(7969,)
(56547, 27)
seq_struc = np.concatenate((X_test_seq_low, X_test_annotation_low), axis=2)
print (seq_struc.shape)(7969, 150, 6)
#load models and obtain prediction and integrated_gradients
# add the absolute path of results directory to the path_to_model
path_to_model = "/Users/storm/git/Multi-resBind/results/input_combination_RNAplfold/"
path_to_model_seq_struc= path_to_model + "m_resbind_input_PU_4_model.h5"_____no_output_____model_seq_struc=load_model(path_to_model_seq_struc, custom_objects={'precision': precision,'recall': recall })
pred_seq_struc=model_seq_struc.predict(seq_struc)WARNING:tensorflow:From /anaconda3/lib/python3.7/site-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
#RBPnames for each model
RBPnames_low=np.array(['MBNL1', 'P53_NONO', 'PUM2', 'QKI', 'AGO3', 'FUS', 'TAF15', 'ZFP36', 'DICER1', 'EIF3A', 'EIF3D', 'EIF3G', 'SSB', 'PAPD5', 'CPSF4', 'CPSF3', 'RTCB', 'FXR1', 'NOP58', 'NOP56', 'FBL', 'LIN28A', 'LIN28B', 'UPF1', 'G35', 'G45', 'XPO5'])
#number of tasks for each model
num_task_low=len(RBPnames_low)_____no_output_____# one example from the low models, you can change the RBPname to see the attribution map of other RBPs in low models
X_test_seq = X_test_seq_low
X_test_struc = X_test_annotation_low
X_test_region = X_test_region_low
y_test_RBP = y_test_RBP_low_____no_output_____RBPnames = RBPnames_low
pred = pred_seq_struc
igres = integrated_gradients(model_seq_struc)Evaluated output channel (0-based index): All
Building gradient functions
Progress: 100.0%
Done.
# plot the Fig.5 (b) in the paper_____no_output_____RBPname = "MBNL1"
RBP_index = np.where(RBPnames == RBPname)[0][0]
#print (RBP_index)
ind = [i[0] for i in sorted(enumerate(pred[:, RBP_index]), key=lambda x: x[1],
                  reverse=True) if y_test_RBP[i[0], RBP_index] == 1 and pred[i[0], RBP_index] > 0.50][1:2]
#print (pred[ind,RBP_index])
ex_seq = np.array([igres.explain(seq_struc[i], outc=RBP_index, reference=False) for i in ind])
#print (ex_seq.shape)
seq_log = np.multiply(ex_seq, seq_struc[ind])
#from matplotlib.backends.backend_pdf import PdfPages
#pdf = PdfPages('motif_MBNL1_seq_struc_region_zero.pdf')
fig = plt.figure(figsize=(9,3))
ax1 = plt.subplot(2, 1, 1)
seqlogo(np.transpose(seq_log[:, 50:100, :4], axes=(1, 2, 0)), "RNA", ax1)
ax2 = plt.subplot(2, 1, 2)
seqlogo(np.transpose(seq_log[:, 50:100, 4:], axes=(1, 2, 0)), "RNAStruct", ax2)
fig.suptitle('MBNL1 (b)', fontsize=12)
plt.show()
#plt.savefig('MBNL1_seq_struc_contribution_map_b.eps', format='eps',bbox_inches='tight')../Scripts/plotseqlogo.py:153: MatplotlibDeprecationWarning: Passing the minor parameter of set_ticks() positionally is deprecated since Matplotlib 3.2; the parameter will become keyword-only two minor releases later.
  ax.set_xticks([],[])
RBPname = "MBNL1"
RBP_index = np.where(RBPnames == RBPname)[0][0]
#print (RBP_index)
ind = [i[0] for i in sorted(enumerate(pred[:, RBP_index]), key=lambda x: x[1],
                  reverse=True) if y_test_RBP[i[0], RBP_index] == 1 and pred[i[0], RBP_index] > 0.50][1:2]
#print (pred[ind,RBP_index])
print (ind)
ref_ind = [i[0] for i in sorted(enumerate(pred[:, RBP_index]), key=lambda x: x[1],
                      reverse=False) if y_test_RBP[i[0], RBP_index] == 0][0:2000]
#print (ref_ind)
ref = np.mean(seq_struc[ref_ind], axis=0)
#print (ref.shape)
ex_seq = np.array([igres.explain(seq_struc[i], outc=RBP_index, reference=ref) for i in ind])
#print (ex_seq.shape)
seq_log = np.multiply(ex_seq, seq_struc[ind])
#from matplotlib.backends.backend_pdf import PdfPages
#pdf = PdfPages('motif_MBNL1_seq_struc_region_zero.pdf')
fig = plt.figure(figsize=(9,3))
ax1 = plt.subplot(2, 1, 1)
seqlogo(np.transpose(seq_log[:, 50:100, :4], axes=(1, 2, 0)), "RNA", ax1)
ax2 = plt.subplot(2, 1, 2)
seqlogo(np.transpose(seq_log[:, 50:100, 4:], axes=(1, 2, 0)), "RNAStruct", ax2)
fig.suptitle('MBNL1 (a)', fontsize=12)
plt.show()
#plt.savefig('MBNL1_seq_struc_contribution_map_a.eps', format='eps',bbox_inches='tight')[3386]

</code>


# Notebook from monocongo/datascience_portfolio
Path: dataquest/notebooks/project_visualization_gender_gap/Visualizing Gender Gap in College Degrees.ipynb

<code>
%matplotlib inline
import pandas as pd
import matplotlib.pyplot as plt

# read the CSV values into a Pandas DataFrame
women_degrees = pd.read_csv('percent-bachelors-degrees-women-usa.csv')

# set tuples for colors from Color Blind 10 palette
cb_dark_blue = (0/255, 107/255, 164/255)
cb_orange = (255/255, 128/255, 14/255)

# major categories
stem_cats = ['Psychology',
             'Biology',
             'Math and Statistics',
             'Physical Sciences',
             'Computer Science',
             'Engineering']
lib_arts_cats = ['Foreign Languages',
                 'English',
                 'Communications and Journalism',
                 'Art and Performance',
                 'Social Sciences and History']
other_cats = ['Health Professions',
              'Public Administration',
              'Education',
              'Agriculture',
              'Business',
              'Architecture']

# put category lists into a list corresponding to the columns we'll display
cat_cols = [stem_cats, lib_arts_cats, other_cats]

# create a matplotlib Figure to which we'll add subplots
fig = plt.figure(figsize=(18, 15))

for col in range(3):
    cats = cat_cols[col]
    for row in range(6):
        plot_number = (row * 3) + col + 1
        if plot_number != 17:
            ax = fig.add_subplot(6, 3, plot_number)
            ax.plot(women_degrees['Year'],
                    women_degrees[cats[row]],
                    c=cb_dark_blue,
                    label='Women',
                    linewidth=3)
            ax.plot(women_degrees['Year'],
                    100-women_degrees[cats[row]],
                    c=cb_orange,
                    label='Men',
                    linewidth=3)
            ax.spines["right"].set_visible(False)
            ax.spines["left"].set_visible(False)
            ax.spines["top"].set_visible(False)
            ax.spines["bottom"].set_visible(False)
            ax.set_xlim(1968, 2011)
            ax.set_ylim(0,100)
            ax.set_title(stem_cats[row])

            # remove the axis ticks
            ax.tick_params(bottom="off",
                           top="off",
                           left="off",
                           right="off")

            # use only the range limits for the Y-axis
            ax.set_yticks([0,100])

            # remove the bottom label unless it's the last plot in the column
            if row != len(cats) - 1:
                ax.tick_params(labelbottom='off')

            # add a faint horizontal line at the 50% level
            ax.axhline(50,
                       c=(171/255, 171/255, 171/255),
                       alpha=0.3)

            # annotate the top and bottom plots of each column with labels on the lines
            if row == 0:
                ax.text(2005, 87, 'Men')
                ax.text(2002, 8, 'Women')
            elif row == 5:
                ax.text(2005, 62, 'Men')
                ax.text(2001, 35, 'Women')

# export the figure as an image file
plt.savefig('gender_degrees.png')

# display the figure
plt.show()_____no_output_____
</code>


# Notebook from annacuomo/CellRegMap_analyses
Path: neuroseq/preprocessing/scanpy_merge_all_da_cells.ipynb

<code>
import scanpy as sc/nfs/software/stegle/users/acuomo/conda-envs/myscanpy/lib/python3.7/site-packages/anndata/_core/anndata.py:21: FutureWarning: pandas.core.index is deprecated and will be removed in a future version.  The public classes are available in the top-level namespace.
  from pandas.core.index import RangeIndex
mydir = "/nfs/leia/research/stegle/dseaton/hipsci/singlecell_neuroseq/data/data_processed/"_____no_output_____#### Day 30
file0 = mydir+"pool1_13_noddd_D30/pool1_13_noddd_D30.scanpy.w_metadata.w_celltype.scanpy.h5"
d30 = sc.read(file0)WARNING: Your filename has more than two extensions: ['.scanpy', '.w_metadata', '.w_celltype', '.scanpy', '.h5'].
Only considering the two last: ['.scanpy', '.h5'].
WARNING: Your filename has more than two extensions: ['.scanpy', '.w_metadata', '.w_celltype', '.scanpy', '.h5'].
Only considering the two last: ['.scanpy', '.h5'].
d30.n_obs_____no_output_____d30.obs.head()_____no_output_____### select DA only
d30_da = d30[d30.obs['celltype'] == 'DA']
d30_da.n_obs_____no_output_____d30_da.var_____no_output_____#### Day 52
file1 = mydir+"pool1_17_D52/pool1_17_D52.scanpy.w_metadata.w_celltype.scanpy.h5"
d52 = sc.read(file1)WARNING: Your filename has more than two extensions: ['.scanpy', '.w_metadata', '.w_celltype', '.scanpy', '.h5'].
Only considering the two last: ['.scanpy', '.h5'].
WARNING: Your filename has more than two extensions: ['.scanpy', '.w_metadata', '.w_celltype', '.scanpy', '.h5'].
Only considering the two last: ['.scanpy', '.h5'].
d52.n_obs_____no_output_____d52.obs.head()_____no_output_____### select DA only
d52_da = d52[d52.obs['celltype'] == 'DA']
d52_da.n_obs_____no_output_____d52_da.var = d30_da.var
d52_da.var_____no_output_____d30_da_____no_output_____d52_da_____no_output_____d30_da = d30_da.copy()_____no_output_____# d52_da = d52_da.copy()_____no_output_____all_da = d30_da.concatenate(d52_da, join="outer") # (148,808 cells x 32,738 genes)
all_da_____no_output_____all_da.write("/hps/nobackup/stegle/users/acuomo/all_scripts/struct_LMM2/sc_neuroseq/all_da_cells.h5")... storing 'donor_id' as categorical
... storing 'pool_id' as categorical
... storing 'sample_id' as categorical
... storing 'time_point' as categorical
... storing 'treatment' as categorical

</code>


# Notebook from divshacker/IBM_Quantum_Challenge_2021_Solutions
Path: Solutions/ex5/ex5.ipynb

# Exercise 5 - Variational quantum eigensolver

## Historical background

During the last decade, quantum computers matured quickly and began to realize Feynman's initial dream of a computing system that could simulate the laws of nature in a quantum way. A 2014 paper first authored by Alberto Peruzzo introduced the **Variational Quantum Eigensolver (VQE)**, an algorithm meant for finding the ground state energy (lowest energy) of a molecule, with much shallower circuits than other approaches.[1] And, in 2017, the IBM Quantum team used the VQE algorithm to simulate the ground state energy of the lithium hydride molecule.[2]

VQE's magic comes from outsourcing some of the problem's processing workload to a classical computer. The algorithm starts with a parameterized quantum circuit called an ansatz (a best guess) then finds the optimal parameters for this circuit using a classical optimizer. The VQE's advantage over classical algorithms comes from the fact that a quantum processing unit can represent and store the problem's exact wavefunction, an exponentially hard problem for a classical computer.

This exercise 5 allows you to realize Feynman's dream yourself, setting up a variational quantum eigensolver to determine the ground state and the energy of a molecule. This is interesting because the ground state can be used to calculate various molecular properties, for instance the exact forces on nuclei than can serve to run molecular dynamics simulations to explore what happens in chemical systems with time.[3]

### References

1. Peruzzo, Alberto, et al. "A variational eigenvalue solver on a photonic quantum processor." Nature communications 5.1 (2014): 1-7.
2. Kandala, Abhinav, et al. "Hardware-efficient variational quantum eigensolver for small molecules and quantum magnets." Nature 549.7671 (2017): 242-246.
3. Sokolov, Igor O., et al. "Microcanonical and finite-temperature ab initio molecular dynamics simulations on quantum computers." Physical Review Research 3.1 (2021): 013125.

## Introduction

For the implementation of VQE, you will be able to make choices on how you want to compose your simulation, in particular focusing on the ansatz quantum circuits.
This is motivated by the fact that one of the important tasks when running VQE on noisy quantum computers is to reduce the loss of fidelity (which introduces errors) by finding the most compact quantum circuit capable of representing the ground state.
Practically, this entails to minimizing the number of two-qubit gates (e.g. CNOTs) while not loosing accuracy.

Goal

Find the shortest ansatz circuits for representing accurately the ground state of given problems. Be creative!

Plan

First you will learn how to compose a VQE simulation for the smallest molecule and then apply what you have learned to a case of a larger one.

**1. Tutorial -  VQE for H$_2$:** familiarize yourself with VQE and select the best combination of ansatz/classical optimizer by running statevector simulations.

**2. Final Challenge - VQE for LiH:** perform similar investigation as in the first part but restricting to statevector simulator only. Use the qubit number reduction schemes available in Qiskit and find the optimal circuit for this larger system. Optimize the circuit and use your imagination to find ways to select the best building blocks of parameterized circuits and compose them to construct the most compact ansatz circuit for the ground state, better than the ones already available in Qiskit.

Below is an introduction to the theory behind VQE simulations. You don't have to understand the whole thing before moving on. Don't be scared!

_____no_output_____## Theory

Here below is the general workflow representing how the molecular simulations using VQE are performed on quantum computers.

The core idea hybrid quantum-classical approach is to outsource to **CPU (classical processing unit)** and **QPU (quantum processing unit)** the parts that they can do best. The CPU takes care of listing the terms that need to be measured to compute the energy and also optimizing the circuit parameters. The QPU implements a quantum circuit representing the quantum state of a system and measures the energy. Some more details are given below:

**CPU** can compute efficiently the energies associated to electron hopping and interactions (one-/two-body integrals by means of a Hartree-Fock calculation) that serve to represent the total energy operator, Hamiltonian. The [Hartree–Fock (HF) method](https://en.wikipedia.org/wiki/Hartree%E2%80%93Fock_method#:~:text=In%20computational%20physics%20and%20chemistry,system%20in%20a%20stationary%20state.) efficiently computes an approximate grounds state wavefunction by assuming that the latter can be represented by a single Slater determinant (e.g. for H$_2$ molecule in STO-3G basis with 4 spin-orbitals and qubits, $|\Psi_{HF} \rangle = |0101 \rangle$ where electrons occupy the lowest energy spin-orbitals). What QPU does later in VQE is finding a quantum state (corresponding circuit and its parameters) that can also represent other states associated missing electronic correlations (i.e. $\sum_i c_i |i\rangle$ states in  $|\Psi \rangle = c_{HF}|\Psi_{HF} \rangle + \sum_i c_i |i\rangle $ where $i$ is a bitstring).

After a HF calculation, operators in the Hamiltonian are mapped to measurements on a QPU using fermion-to-qubit transformations (see Hamiltonian section below). One can further analyze the properties of the system to reduce the number of qubits or shorten the ansatz circuit:

- For Z2 symmetries and two-qubit reduction, see [Bravyi *et al*, 2017](https://arxiv.org/abs/1701.08213v1).
- For entanglement forging, see [Eddins *et al.*, 2021](https://arxiv.org/abs/2104.10220v1).
- For the adaptive ansatz see, [Grimsley *et al.*,2018](https://arxiv.org/abs/1812.11173v2), [Rattew *et al.*,2019](https://arxiv.org/abs/1910.09694), [Tang *et al.*,2019](https://arxiv.org/abs/1911.10205). You may use the ideas found in those works to find ways to shorten the quantum circuits.

**QPU** implements quantum circuits (see Ansatzes section below), parameterized by angles $\vec\theta$, that would represent the ground state wavefunction by placing various single qubit rotations and entanglers (e.g. two-qubit gates). The quantum advantage lies in the fact that QPU can efficiently represent and store the exact wavefunction, which becomes intractable on a classical computer for systems that have more than a few atoms.  Finally, QPU measures the operators of choice (e.g. ones representing a Hamiltonian).

Below we go slightly more in mathematical details of each component of the VQE algorithm. It might be also helpful if you watch our [video episode about VQE](https://www.youtube.com/watch?v=Z-A6G0WVI9w).

### Hamiltonian

Here we explain how we obtain the operators that we need to measure to obtain the energy of a given system.
These terms are included in the molecular Hamiltonian defined as:
$$
\begin{aligned}
\hat{H} &=\sum_{r s} h_{r s} \hat{a}_{r}^{\dagger} \hat{a}_{s} \\
&+\frac{1}{2} \sum_{p q r s} g_{p q r s} \hat{a}_{p}^{\dagger} \hat{a}_{q}^{\dagger} \hat{a}_{r} \hat{a}_{s}+E_{N N}
\end{aligned}
$$
with
$$
h_{p q}=\int \phi_{p}^{*}(r)\left(-\frac{1}{2} \nabla^{2}-\sum_{I} \frac{Z_{I}}{R_{I}-r}\right) \phi_{q}(r)
$$
$$
g_{p q r s}=\int \frac{\phi_{p}^{*}\left(r_{1}\right) \phi_{q}^{*}\left(r_{2}\right) \phi_{r}\left(r_{2}\right) \phi_{s}\left(r_{1}\right)}{\left|r_{1}-r_{2}\right|}
$$

where the $h_{r s}$  and $g_{p q r s}$ are the one-/two-body integrals (using the Hartree-Fock method) and $E_{N N}$ the nuclear repulsion energy.
The one-body integrals represent the kinetic energy of the electrons and their interaction with nuclei.
The two-body integrals represent the electron-electron interaction.
The $\hat{a}_{r}^{\dagger}, \hat{a}_{r}$ operators represent creation and annihilation of electron in spin-orbital $r$ and require mappings to operators, so that we can measure them on a quantum computer.
Note that VQE minimizes the electronic energy so you have to retrieve and add the nuclear repulsion energy $E_{NN}$ to compute the total energy.

So, for every non-zero matrix element in the $ h_{r s}$ and $g_{p q r s}$ tensors, we can construct corresponding Pauli string (tensor product of Pauli operators) with the following fermion-to-qubit transformation.
For instance, in Jordan-Wigner mapping for an orbital $r = 3$, we obtain the following Pauli string:
$$
\hat a_{3}^{\dagger}= \hat \sigma_z \otimes \hat \sigma_z \otimes\left(\frac{ \hat \sigma_x-i \hat \sigma_y}{2}\right) \otimes 1 \otimes \cdots \otimes 1
$$
where $\hat \sigma_x, \hat \sigma_y, \hat \sigma_z$ are the well-known Pauli operators. The tensor products of $\hat \sigma_z$ operators are placed to enforce the fermionic anti-commutation relations.
A representation of the Jordan-Wigner mapping between the 14 spin-orbitals of a water molecule and some 14 qubits is given below:

Then, one simply replaces the one-/two-body excitations (e.g. $\hat{a}_{r}^{\dagger} \hat{a}_{s}$, $\hat{a}_{p}^{\dagger} \hat{a}_{q}^{\dagger} \hat{a}_{r} \hat{a}_{s}$) in the Hamiltonian by corresponding Pauli strings (i.e. $\hat{P}_i$, see picture above). The resulting operator set is ready to be measured on the QPU.
For additional details see [Seeley *et al.*, 2012](https://arxiv.org/abs/1208.5986v1).

### Ansatzes

There are mainly 2 types of ansatzes you can use for chemical problems.

- **q-UCC ansatzes** are physically inspired, and roughly map the electron excitations to quantum circuits. The q-UCCSD ansatz (`UCCSD`in Qiskit) possess all possible single and double electron excitations. The paired double q-pUCCD (`PUCCD`) and singlet q-UCCD0 (`SUCCD`) just consider a subset of such excitations (meaning significantly shorter circuits) and have proved to provide good results for dissociation profiles. For instance, q-pUCCD doesn't have single excitations and the double excitations are paired as in the image below.
- **Heuristic ansatzes (`TwoLocal`)** were invented to shorten the circuit depth but still be able to represent the ground state.
As in the figure below, the R gates represent the parametrized single qubit rotations and $U_{CNOT}$ the entanglers (two-qubit gates). The idea is that after repeating certain $D$-times the same block (with independent parameters) one can reach the ground state.

For additional details refer to [Sokolov *et al.* (q-UCC ansatzes)](https://arxiv.org/abs/1911.10864v2) and [Barkoutsos *et al.* (Heuristic ansatzes)](https://arxiv.org/pdf/1805.04340.pdf).

### VQE

Given a Hermitian operator $\hat H$ with an unknown minimum eigenvalue $E_{min}$, associated with the eigenstate $|\psi_{min}\rangle$, VQE provides an estimate $E_{\theta}$, bounded by $E_{min}$:

\begin{align*}
    E_{min} \le E_{\theta} \equiv \langle \psi(\theta) |\hat H|\psi(\theta) \rangle
\end{align*}

where $|\psi(\theta)\rangle$ is the trial state associated with $E_{\theta}$. By applying a parameterized circuit, represented by $U(\theta)$, to some arbitrary starting state $|\psi\rangle$, the algorithm obtains an estimate $U(\theta)|\psi\rangle \equiv |\psi(\theta)\rangle$ on $|\psi_{min}\rangle$. The estimate is iteratively optimized by a classical optimizer by changing the parameter $\theta$ and minimizing the expectation value of $\langle \psi(\theta) |\hat H|\psi(\theta) \rangle$.

As applications of VQE, there are possibilities in molecular dynamics simulations, see [Sokolov *et al.*, 2021](https://arxiv.org/abs/2008.08144v1), and excited states calculations, see [Ollitrault *et al.*, 2019](https://arxiv.org/abs/1910.12890) to name a few.

 References for additional details

For the qiskit-nature tutorial that implements this algorithm see [here](https://qiskit.org/documentation/nature/tutorials/01_electronic_structure.html)
but this won't be sufficient and you might want to look on the [first page of github repository](https://github.com/Qiskit/qiskit-nature) and the [test folder](https://github.com/Qiskit/qiskit-nature/tree/main/test) containing tests that are written for each component, they provide the base code for the use of each functionality.

_____no_output_____## Part 1: Tutorial - VQE for H$_2$ molecule

In this part, you will simulate H$_2$ molecule using the STO-3G basis with the PySCF driver and Jordan-Wigner mapping.
We will guide you through the following parts so then you can tackle harder problems.

#### 1. Driver

The interfaces to the classical chemistry codes that are available in Qiskit are called drivers.
We have for example `PSI4Driver`, `PyQuanteDriver`, `PySCFDriver` are available.

By running a driver (Hartree-Fock calculation for a given basis set and molecular geometry), in the cell below, we obtain all the necessary information about our molecule to apply then a quantum algorithm._____no_output_____

from qiskit_nature.drivers import PySCFDriver

molecule = 'Li 0.0 0.0 0.0; H 0.0 0.0 1.5474'
driver = PySCFDriver(atom=molecule)
qmolecule = driver.run()_____no_output_____

 Tutorial questions 1

Look into the attributes of `qmolecule` and answer the questions below.

1. We need to know the basic characteristics of our molecule. What is the total number of electrons in your system?
2. What is the number of molecular orbitals?
3. What is the number of spin-orbitals?
3. How many qubits would you need to simulate this molecule with Jordan-Wigner mapping?
5. What is the value of the nuclear repulsion energy?

You can find the answers at the end of this notebook.

_____no_output_____

# WRITE YOUR CODE BETWEEN THESE LINES - START
n_el = qmolecule.num_alpha + qmolecule.num_beta

n_mo = qmolecule.num_molecular_orbitals

n_so = 2 * qmolecule.num_molecular_orbitals

n_q = 2* qmolecule.num_molecular_orbitals

e_nn = qmolecule.nuclear_repulsion_energy
# WRITE YOUR CODE BETWEEN THESE LINES - END_____no_output_____

#### 2. Electronic structure problem

You can then create an `ElectronicStructureProblem` that can produce the list of fermionic operators before mapping them to qubits (Pauli strings)._____no_output_____

from qiskit_nature.problems.second_quantization.electronic import ElectronicStructureProblem
from qiskit_nature.transformers import FreezeCoreTransformer
freezeCoreTransfomer = [FreezeCoreTransformer(freeze_core=True,remove_orbitals=[3,4])]
problem = ElectronicStructureProblem(driver,q_molecule_transformers=freezeCoreTransfomer)

# Generate the second-quantized operators
second_q_ops = problem.second_q_ops()

# Hamiltonian
main_op = second_q_ops[0]

_____no_output_____

#### 3. QubitConverter

Allows to define the mapping that you will use in the simulation. You can try different mapping but
we will stick to `JordanWignerMapper` as allows a simple correspondence: a qubit represents a spin-orbital in the molecule._____no_output_____

from qiskit_nature.mappers.second_quantization import ParityMapper, BravyiKitaevMapper, JordanWignerMapper
from qiskit_nature.converters.second_quantization.qubit_converter import QubitConverter

# Setup the mapper and qubit converter
mapper_type = 'ParityMapper'

if mapper_type == 'ParityMapper':
    mapper = ParityMapper()
elif mapper_type == 'JordanWignerMapper':
    mapper = JordanWignerMapper()
elif mapper_type == 'BravyiKitaevMapper':
    mapper = BravyiKitaevMapper()

converter = QubitConverter(mapper=mapper,two_qubit_reduction=True)

# The fermionic operators are mapped to qubit operators
num_particles = (problem.molecule_data_transformed.num_alpha,
             problem.molecule_data_transformed.num_beta)
qubit_op = converter.convert(main_op, num_particles=num_particles)
#qubit_op=qubit_op[0]+qubit_op[1]+qubit_op[3]+qubit_op[1]_____no_output_____

#### 4. Initial state
As we described in the Theory section, a good initial state in chemistry is the HF state (i.e. $|\Psi_{HF} \rangle = |0101 \rangle$). We can initialize it as follows:_____no_output_____

from qiskit_nature.circuit.library import HartreeFock

num_particles = (problem.molecule_data_transformed.num_alpha,
             problem.molecule_data_transformed.num_beta)
num_spin_orbitals = 2 * problem.molecule_data_transformed.num_molecular_orbitals

init_state = HartreeFock(num_spin_orbitals, num_particles, converter)
init_state.draw('mpl')_____no_output_____

#### 5. Ansatz
One of the most important choices is the quantum circuit that you choose to approximate your ground state.
Here is the example of qiskit circuit library that contains many possibilities for making your own circuit._____no_output_____

from qiskit.circuit.library import TwoLocal
from qiskit_nature.circuit.library import UCCSD, PUCCD, SUCCD
from qiskit import transpile
# Choose the ansatz
ansatz_type = "Custom"
# Parameters for q-UCC antatze
num_particles = (problem.molecule_data_transformed.num_alpha,
             problem.molecule_data_transformed.num_beta)
num_spin_orbitals = 2 * problem.molecule_data_transformed.num_molecular_orbitals

# Put arguments for twolocal
if ansatz_type == "TwoLocal":
    # Single qubit rotations that are placed on all qubits with independent parameters
    rotation_blocks = ['ry','rz']
    # Entangling gates
    entanglement_blocks = 'cx'
    # How the qubits are entangled
    entanglement = 'linear'
    # Repetitions of rotation_blocks + entanglement_blocks with independent parameters
    repetitions = 1
    # Skip the final rotation_blocks layer
    skip_final_rotation_layer = False
    ansatz = TwoLocal(qubit_op.num_qubits, rotation_blocks, entanglement_blocks, reps=repetitions,
                      entanglement=entanglement, skip_final_rotation_layer=skip_final_rotation_layer)
    # Add the initial state
    ansatz.compose(init_state, front=True, inplace=True)
elif ansatz_type == "UCCSD":
    ansatz = UCCSD(converter,num_particles,num_spin_orbitals,initial_state = init_state)
elif ansatz_type == "PUCCD":
    ansatz = PUCCD(converter,num_particles,num_spin_orbitals,initial_state = init_state)
elif ansatz_type == "SUCCD":
    ansatz = SUCCD(converter,num_particles,num_spin_orbitals,initial_state = init_state)
elif ansatz_type == "Custom":
    # Example of how to write your own circuit
    from qiskit.circuit import Parameter, QuantumCircuit, QuantumRegister
    # Define the variational parameter
    #

    n = qubit_op.num_qubits
    #theta = Parameter('θ')
    # Make an empty quantum circuit
    qc = QuantumCircuit(qubit_op.num_qubits)
    qubit_label = 0
    # Place a Hadamard gate
    #qc.h(qubit_label)
    theta = Parameter('θ[0]')
    qc.ry(theta,0)
    theta = Parameter('θ[1]')
    qc.ry(theta,1)
    theta = Parameter('θ[2]')
    qc.ry(theta,2)
    theta = Parameter('θ[3]')
    qc.ry(theta,3)
    theta = Parameter('θ[4]')
    qc.rz(theta,0 )
    theta = Parameter('θ[5]')
    qc.rz(theta,1 )
    theta = Parameter('θ[6]')
    qc.rz(theta,2 )
    theta = Parameter('θ[7]')
    qc.rz(theta,3 )
    # Place a CNOT ladder
    for i in range(n-1):
        qc.cx(i, i+1)
    # Visual separator
    #qc.barrier()
    # rz rotations on all qubits
    #qc.rz(theta, range(n))
    theta = Parameter('θ[8]')
    qc.ry(theta,0)
    theta = Parameter('θ[9]')
    qc.ry(theta,1)
    theta = Parameter('θ[10]')
    qc.ry(theta,2)
    theta = Parameter('θ[11]')
    qc.ry(theta,3)
    theta = Parameter('θ[12]')
    qc.rz(theta,0 )
    theta = Parameter('θ[13]')
    qc.rz(theta,1 )
    theta = Parameter('θ[14]')
    qc.rz(theta,2 )
    theta = Parameter('θ[15]')
    qc.rz(theta,3 )
    #for i in range(n-1):
        #qc.cx(i, i+1)
    ansatz = qc
    ansatz.compose(init_state, front=True, inplace=True)
#ansatz = transpile(ansatz,  basis_gates=['cx', 'u'], optimization_level=3)
#print(ansatz)
ansatz.draw('mpl')_____no_output_____from qiskit import Aer
backend = Aer.get_backend('statevector_simulator')_____no_output_____

#### 7. Optimizer

The optimizer guides the evolution of the parameters of the ansatz so it is very important to investigate the energy convergence as it would define the number of measurements that have to be performed on the QPU.
A clever choice might reduce drastically the number of needed energy evaluations._____no_output_____

from qiskit.algorithms.optimizers import COBYLA, L_BFGS_B, SPSA, SLSQP

optimizer_type = 'SLSQP'

# You may want to tune the parameters
# of each optimizer, here the defaults are used
if optimizer_type == 'COBYLA':
    optimizer = COBYLA(maxiter=1000)
elif optimizer_type == 'L_BFGS_B':
    optimizer = L_BFGS_B(maxfun=500)
elif optimizer_type == 'SPSA':
    optimizer = SPSA(maxiter=500)
elif optimizer_type == 'SLSQP':
    optimizer = SLSQP(maxiter=100)_____no_output_____

#### 8. Exact eigensolver
For learning purposes, we can solve the problem exactly with the exact diagonalization of the Hamiltonian matrix so we know where to aim with VQE.
Of course, the dimensions of this matrix scale exponentially in the number of molecular orbitals so you can try doing this for a large molecule of your choice and see how slow this becomes.
For very large systems you would run out of memory trying to store their wavefunctions._____no_output_____

from qiskit_nature.algorithms.ground_state_solvers.minimum_eigensolver_factories import NumPyMinimumEigensolverFactory
from qiskit_nature.algorithms.ground_state_solvers import GroundStateEigensolver
import numpy as np

def exact_diagonalizer(problem, converter):
    solver = NumPyMinimumEigensolverFactory()
    calc = GroundStateEigensolver(converter, solver)
    result = calc.solve(problem)
    return result

result_exact = exact_diagonalizer(problem, converter)
exact_energy = np.real(result_exact.eigenenergies[0])
print("Exact electronic energy", exact_energy)
print(result_exact)

# The targeted electronic energy for H2 is -1.85336 Ha
# Check with your VQE result.Exact electronic energy -1.0887060157347395
=== GROUND STATE ENERGY ===

* Electronic ground state energy (Hartree): -8.907396311316
  - computed part:      -1.088706015735
  - FreezeCoreTransformer extracted energy part: -7.818690295581
~ Nuclear repulsion energy (Hartree): 1.025934879643
> Total ground state energy (Hartree): -7.881461431673

=== MEASURED OBSERVABLES ===

  0:  # Particles: 2.000 S: 0.000 S^2: 0.000 M: 0.000

=== DIPOLE MOMENTS ===

~ Nuclear dipole moment (a.u.): [0.0  0.0  2.92416221]

  0:
  * Electronic dipole moment (a.u.): [0.0  0.0  4.76300889]
    - computed part:      [0.0  0.0  4.76695575]
    - FreezeCoreTransformer extracted energy part: [0.0  0.0  -0.00394686]
  > Dipole moment (a.u.): [0.0  0.0  -1.83884668]  Total: 1.83884668
                 (debye): [0.0  0.0  -4.67388163]  Total: 4.67388163

#### 9. VQE and initial parameters for the ansatz
Now we can import the VQE class and run the algorithm._____no_output_____

from qiskit.algorithms import VQE
from IPython.display import display, clear_output

# Print and save the data in lists
def callback(eval_count, parameters, mean, std):
    # Overwrites the same line when printing
    display("Evaluation: {}, Energy: {}, Std: {}".format(eval_count, mean, std))
    clear_output(wait=True)
    counts.append(eval_count)
    values.append(mean)
    params.append(parameters)
    deviation.append(std)

counts = []
values = []
params = []
deviation = []

# Set initial parameters of the ansatz
# We choose a fixed small displacement
# So all participants start from similar starting point
try:
    initial_point = [0.01] * len(ansatz.ordered_parameters)
except:
    initial_point = [0.01] * ansatz.num_parameters

algorithm = VQE(ansatz,
                optimizer=optimizer,
                quantum_instance=backend,
                callback=callback,
                initial_point=initial_point)

result = algorithm.compute_minimum_eigenvalue(qubit_op)

print(result){   'aux_operator_eigenvalues': None,
    'cost_function_evals': 786,
    'eigenstate': array([ 1.39068158e-03+1.62802133e-04j, -4.80488330e-03-1.47021752e-04j,
        2.62469948e-02+8.47501878e-04j, -9.91330223e-01+1.36880475e-02j,
       -5.37441452e-02-3.77341078e-03j, -3.68884683e-04-8.91109049e-05j,
        8.79392425e-04-4.46248052e-06j, -2.32771036e-02+1.49206277e-03j,
       -2.67365952e-03-7.05207349e-05j, -1.11750542e-05-8.32246629e-06j,
        3.66657008e-06+2.34886628e-05j,  3.10027542e-04-9.10842186e-04j,
        1.13572522e-01-2.69542887e-03j,  5.54608320e-04+1.41009949e-04j,
       -5.49601866e-04+3.83368293e-05j, -2.71585836e-05-1.95987447e-05j]),
    'eigenvalue': -1.0863472018960634,
    'optimal_parameters': {   Parameter(θ[12]): -0.045879856802719214,
                              Parameter(θ[1]): -0.005430075170793269,
                              Parameter(θ[10]): 3.1886690642436473,
                              Parameter(θ[15]): -0.09609215948776839,
                              Parameter(θ[14]): -0.050142793214078746,
                              Parameter(θ[6]): -0.0702238904491193,
                              Parameter(θ[7]): 0.004098714786970228,
                              Parameter(θ[8]): 0.052287523816089615,
                              Parameter(θ[11]): 0.4424435814202906,
                              Parameter(θ[2]): -3.1415239835860964,
                              Parameter(θ[9]): 3.1318832114230233,
                              Parameter(θ[5]): 0.02860990134284915,
                              Parameter(θ[0]): 0.2521541561209351,
                              Parameter(θ[4]): -0.04518947630263523,
                              Parameter(θ[3]): -2.6983242252300936,
                              Parameter(θ[13]): -0.04486848130903857},
    'optimal_point': array([ 0.25215416,  3.18866906,  0.44244358, -0.04587986, -0.04486848,
       -0.05014279, -0.09609216, -0.00543008, -3.14152398, -2.69832423,
       -0.04518948,  0.0286099 , -0.07022389,  0.00409871,  0.05228752,
        3.13188321]),
    'optimal_value': -1.0863472018960634,
    'optimizer_evals': 786,
    'optimizer_time': 8.068114042282104}

### 9. Scoring function
We need to judge how good are your VQE simulations, your choice of ansatz/optimizer.
For this, we implemented the following simple scoring function:

$$ score = N_{CNOT}$$

where $N_{CNOT}$ is the number of CNOTs.
But you have to reach the chemical accuracy which is $\delta E_{chem} = 0.004$ Ha $= 4$ mHa, which may be hard to reach depending on the problem.
You have to reach the accuracy we set in a minimal number of CNOTs to win the challenge.
The lower the score the better!_____no_output_____

# WRITE YOUR CODE BETWEEN THESE LINES - START
from qc_grader import grade_ex5
freeze_core = True# change to True if you freezed core electrons
grade_ex5(ansatz,qubit_op,result,freeze_core)

# WRITE YOUR CODE BETWEEN THESE LINES - ENDGrading your answer for ex5. Please wait...

Congratulations 🎉! Your answer is correct.
Your cost is 3.
Feel free to submit your answer.

from qc_grader import submit_ex5
submit_ex5(ansatz,qubit_op,result,freeze_core)Submitting your answer for ex5. Please wait...
Success 🎉! Your answer has been submitted.
# Store results in a dictionary
from qiskit.transpiler import PassManager
from qiskit.transpiler.passes import Unroller

# Unroller transpile your circuit into CNOTs and U gates
pass_ = Unroller(['u', 'cx'])
pm = PassManager(pass_)
ansatz_tp = pm.run(ansatz)
cnots = ansatz_tp.count_ops()['cx']
score = cnots

accuracy_threshold = 4.0 # in mHa
energy = result.optimal_value

if ansatz_type == "TwoLocal":
    result_dict = {
        'optimizer': optimizer.__class__.__name__,
        'mapping': converter.mapper.__class__.__name__,
        'ansatz': ansatz.__class__.__name__,
        'rotation blocks': rotation_blocks,
        'entanglement_blocks': entanglement_blocks,
        'entanglement': entanglement,
        'repetitions': repetitions,
        'skip_final_rotation_layer': skip_final_rotation_layer,
        'energy (Ha)': energy,
        'error (mHa)': (energy-exact_energy)*1000,
        'pass': (energy-exact_energy)*1000 <= accuracy_threshold,
        '# of parameters': len(result.optimal_point),
        'final parameters': result.optimal_point,
        '# of evaluations': result.optimizer_evals,
        'optimizer time': result.optimizer_time,
        '# of qubits': int(qubit_op.num_qubits),
        '# of CNOTs': cnots,
        'score': score}
else:
    result_dict = {
        'optimizer': optimizer.__class__.__name__,
        'mapping': converter.mapper.__class__.__name__,
        'ansatz': ansatz.__class__.__name__,
        'rotation blocks': None,
        'entanglement_blocks': None,
        'entanglement': None,
        'repetitions': None,
        'skip_final_rotation_layer': None,
        'energy (Ha)': energy,
        'error (mHa)': (energy-exact_energy)*1000,
        #'pass': (energy-exact_energy)*1000 <= accuracy_thresholdaccuract_thres,
        '# of parameters': len(result.optimal_point),
        'final parameters': result.optimal_point,
        '# of evaluations': result.optimizer_evals,
        'optimizer time': result.optimizer_time,
        '# of qubits': int(qubit_op.num_qubits),
        '# of CNOTs': cnots,
        'score': score}

# Plot the results
import matplotlib.pyplot as plt

fig, ax = plt.subplots(1, 1)
ax.set_xlabel('Iterations')
ax.set_ylabel('Energy')
ax.grid()
fig.text(0.7, 0.75, f'Energy: {result.optimal_value:.3f}\nScore: {score:.0f}')
plt.title(f"{result_dict['optimizer']}-{result_dict['mapping']}\n{result_dict['ansatz']}")
ax.plot(counts, values)
ax.axhline(exact_energy, linestyle='--')
fig_title = f"\
{result_dict['optimizer']}-\
{result_dict['mapping']}-\
{result_dict['ansatz']}-\
Energy({result_dict['energy (Ha)']:.3f})-\
Score({result_dict['score']:.0f})\
.png"
fig.savefig(fig_title, dpi=300)

# Display and save the data
import pandas as pd
import os.path
filename = 'results_h2.csv'
if os.path.isfile(filename):
    result_df = pd.read_csv(filename)
    result_df = result_df.append([result_dict])
else:
    result_df = pd.DataFrame.from_dict([result_dict])
result_df.to_csv(filename)
result_df[['optimizer','ansatz', '# of qubits', '# of parameters','rotation blocks', 'entanglement_blocks',
    'entanglement', 'repetitions', 'error (mHa)', 'pass', 'score']]_____no_output_____

Tutorial questions 2

Experiment with all the parameters and then:

1. Can you find your best (best score) heuristic ansatz (by modifying parameters of `TwoLocal` ansatz) and optimizer?
2. Can you find your best q-UCC ansatz (choose among `UCCSD, PUCCD or SUCCD` ansatzes) and optimizer?
3. In the cell where we define the ansatz,
   can you modify the `Custom` ansatz by placing gates yourself to write a better circuit than your `TwoLocal` circuit?

For each question, give `ansatz` objects.
Remember, you have to reach the chemical accuracy $|E_{exact} - E_{VQE}| \leq 0.004 $ Ha $= 4$ mHa.

_____no_output_____

# WRITE YOUR CODE BETWEEN THESE LINES - START
from qc_grader import grade_ex5
freeze_core = True# change to True if you freezed core electrons
grade_ex5(ansatz,qubit_op,result,freeze_core)

# WRITE YOUR CODE BETWEEN THESE LINES - ENDGrading your answer for ex5. Please wait...

Congratulations 🎉! Your answer is correct.
Your cost is 3.
Feel free to submit your answer.

## Part 2: Final Challenge - VQE for LiH molecule

In this part, you will simulate LiH molecule using the STO-3G basis with the PySCF driver.

Goal

Experiment with all the parameters and then find your best ansatz. You can be as creative as you want!

For each question, give `ansatz` objects as for Part 1. Your final score will be based only on Part 2.

Be aware that the system is larger now. Work out how many qubits you would need for this system by retrieving the number of spin-orbitals.

### Reducing the problem size

You might want to reduce the number of qubits for your simulation:
- you could freeze the core electrons that do not contribute significantly to chemistry and consider only the valence electrons. Qiskit  already has this functionality implemented. So inspect the different transformers in `qiskit_nature.transformers`  and find the one that performs the freeze core approximation.
- you could use `ParityMapper` with `two_qubit_reduction=True` to eliminate 2 qubits.
- you could reduce the number of qubits by inspecting the symmetries of your Hamiltonian. Find a way to use `Z2Symmetries` in Qiskit.

### Custom ansatz

You might want to explore the ideas proposed in [Grimsley *et al.*,2018](https://arxiv.org/abs/1812.11173v2), [H. L. Tang *et al.*,2019](https://arxiv.org/abs/1911.10205), [Rattew *et al.*,2019](https://arxiv.org/abs/1910.09694), [Tang *et al.*,2019](https://arxiv.org/abs/1911.10205).
You can even get try machine learning algorithms to generate best ansatz circuits.

### Setup the simulation

Let's now run the Hartree-Fock calculation and the rest is up to you!

Attention

We give below the `driver`, the `initial_point`, the `initial_state` that should remain as given.
You are free then to explore all other things available in Qiskit.
So you have to start from this initial point (all parameters set to 0.01):

`initial_point = [0.01] * len(ansatz.ordered_parameters)`
    or
`initial_point = [0.01] * ansatz.num_parameters`

and your initial state has to be the Hartree-Fock state:

`init_state = HartreeFock(num_spin_orbitals, num_particles, converter)`

For each question, give `ansatz` object.
Remember you have to reach the chemical accuracy $|E_{exact} - E_{VQE}| \leq 0.004 $ Ha $= 4$ mHa.

_____no_output_____

from qiskit_nature.drivers import PySCFDriver

molecule = 'Li 0.0 0.0 0.0; H 0.0 0.0 1.5474'
driver = PySCFDriver(atom=molecule)
qmolecule = driver.run()_____no_output_____# WRITE YOUR CODE BETWEEN THESE LINES - START

# WRITE YOUR CODE BETWEEN THESE LINES - END_____no_output_____# Check your answer using following code
from qc_grader import grade_ex5
freeze_core = True# change to True if you freezed core electrons
grade_ex5(ansatz,qubit_op,result,freeze_core)Grading your answer for ex5. Please wait...

Congratulations 🎉! Your answer is correct.
Your cost is 3.
Feel free to submit your answer.

# Submit your answer. You can re-submit at any time.
from qc_grader import submit_ex5
submit_ex5(ansatz,qubit_op,result,freeze_core)Submitting your answer for ex5. Please wait...
Success 🎉! Your answer has been submitted.

## Answers for Part 1

Questions

Look into the attributes of `qmolecule` and answer the questions below.

1. We need to know the basic characteristics of our molecule. What is the total number of electrons in your system?
2. What is the number of molecular orbitals?
3. What is the number of spin-orbitals?
3. How many qubits would you need to simulate this molecule with Jordan-Wigner mapping?
5. What is the value of the nuclear repulsion energy?

Answers

1. `n_el = qmolecule.num_alpha + qmolecule.num_beta`

2. `n_mo = qmolecule.num_molecular_orbitals`

3. `n_so = 2 * qmolecule.num_molecular_orbitals`

4. `n_q = 2* qmolecule.num_molecular_orbitals`

5. `e_nn = qmolecule.nuclear_repulsion_energy`

_____no_output_____## Additional information

**Created by:** Igor Sokolov, Junye Huang, Rahul Pratap Singh

**Version:** 1.0.0_____no_output_____


# Notebook from msednev/DZ-seq
Path: notebooks/10_validation.ipynb

<code>
import numpy as np
import pandas as pd
from matplotlib import pyplot as plt
import seaborn as sns
from scipy.stats import linregress_____no_output_____seq_data = pd.read_csv('../output/datasets/cleavage_filtered.csv.gz')
seq_data = seq_data.groupby(['core', 'rna', 'sub_mod']).agg(FC=('FC', np.mean),
                                                                       FC_sd=('FC', np.std),
                                                                       N=('core', np.count_nonzero))
page_data = pd.read_csv('../input/kinetics_PAGE.csv')
page_data = page_data.rename(columns={'sequence': 'core'})
page_data = page_data.groupby(['core', 'rna', 'sub_mod']).agg(FC=('FC', np.mean),
                                                                       FC_sd=('FC', np.std),
                                                                       N=('core', np.count_nonzero))_____no_output_____joined = seq_data.join(page_data,
                       on=['core', 'sub_mod', 'rna'], how='inner',
                       lsuffix='_seq').reset_index()_____no_output_____slope, intercept, r, p, se = linregress(joined['FC'], joined['FC_seq'])_____no_output_____print(f'slope: {slope}')
print(f'intercept: {intercept}')
print(f'r squared: {r ** 2}')
print(f'p-value: {p}')
print(f'standard error: {se}')slope: 1.3829580219488784
intercept: -0.06322211731313765
r squared: 0.910039221967861
p-value: 1.8592159020728688e-05
standard error: 0.15373068080608446
plt.figure(figsize=(5, 5))
plt.errorbar(x=joined['FC'],
             y=joined['FC_seq'],
             fmt='o',
             yerr=joined['FC_sd_seq'],
             xerr=joined['FC_sd'],
             ms=10,
             capsize=5)
plt.plot(np.linspace(0, 1), intercept + slope * np.linspace(0, 1),
         linestyle='--',
         linewidth=2.5,
         zorder=0)
plt.xlim(0, 1)
plt.ylim(0, 1)
plt.xlabel('FC by PAGE', fontsize=14)
plt.ylabel('FC by DZ-Seq', fontsize=14)
plt.gca().spines['top'].set_visible(False)
plt.gca().spines['right'].set_visible(False)
plt.gca().spines['bottom'].set_linewidth(2.5)
plt.gca().spines['left'].set_linewidth(2.5)
plt.gca().tick_params(width=3, size=6, labelsize=14)_____no_output_____
</code>


# Notebook from saketkc/re-ribo-smk
Path: notebooks/Mmul8-ortho-DESeq.ipynb

<code>
suppressMessages(library(readr))
suppressMessages(library(DESeq2))
suppressMessages(library(RColorBrewer))
suppressMessages(library(ggplot2))
suppressMessages(library(BiocParallel))
suppressMessages(library(pheatmap))
suppressMessages(library(sva))
suppressMessages(library(reshape2))
suppressMessages(library(edgeR))
suppressMessages(library(cowplot))
suppressMessages(library(riborex))
suppressMessages(library(ggrepel))
suppressMessages(library(fdrtool))
suppressMessages(library(grid))

'%notin%' <- function(x,y)!('%in%'(x,y))
cbbPalette <- c("#999999", "#E69F00", "#56B4E9", "#009E73", "#F0E442", "#0072B2", "#D55E00", "#CC79A7")

colors <- list(species = c("GRCh38" = cbbPalette[5],
                            "Mmul8" =  cbbPalette[6],
                             "panTro3" =  cbbPalette[7]),
              assay = c("ribo" = cbbPalette[3],
                       "rna" =  cbbPalette[4]))

write_results <- function(df, results.dir, prefix){
  df<- as.data.frame(df)
  df <- df[order(df$padj),]

  df.sig <- subset(df, padj<0.05)
  df.sig.up <- subset(df.sig, log2FoldChange>0)
  df.sig.down <- subset(df.sig, log2FoldChange<0)
  write.table(df, file = file.path(results.dir,
                                   paste(prefix, 'tsv', sep='.')), sep = '\t')

  write.table(df.sig, file = file.path(results.dir,
                                   paste(prefix, 'sig', 'tsv', sep='.')), sep = '\t')
  write.table(df.sig.up,  file = file.path(results.dir,
                                       paste(prefix, 'sig', 'up', 'tsv', sep='.')), sep = '\t')
  write.table(df.sig.down,  file = file.path(results.dir,
                                         paste(prefix, 'sig', 'down', 'tsv', sep='.')), sep = '\t')
  return (df.sig)
}

plotHeatMap <- function(rlogdist, filename=NULL){
  sampleDists <- dist(t(assay(rlogdist)))
  sampleDistMatrix <- as.matrix(sampleDists)
  rownames(sampleDistMatrix) <- colnames(rlogdist)#paste(rlogdist$condition, colnames(rlogdist), sep="-")
  colnames(sampleDistMatrix) <- NULL
  colors <- colorRampPalette( rev(brewer.pal(9, "Blues")) )(255)
  pheatmap(sampleDistMatrix,
           clustering_distance_rows=sampleDists,
           cellwidth=10,
           cellheight=10,
          clustering_distance_cols=sampleDists,
           col=colors,)
  if (!is.null(filename)) {
      pheatmap(sampleDistMatrix,
               cellwidth=20,
               cellheight=20,
               clustering_distance_rows=sampleDists,
               clustering_distance_cols=sampleDists,
               col=colors, filename=filename)
  }
}

metadata <- read.csv('../../re-ribo-smk/data/ortho-datasets-metadata/rna_ribo_metadata.tsv', sep='\t',stringsAsFactors=F)
metadata.Mmul8 <- subset(metadata, metadata$species == 'Mmul8')
sex <- factor(metadata.Mmul8$sex, levels=c("male", "female"))
assay <- factor(metadata.Mmul8$assay, levels=c("rna", "ribo"))

Mmul8.cds.counts <- read.csv('../../re-ribo-smk/data/ortho-datasets-counts/Mmul8_cds.tsv', sep='\t', row.names=1)
Mmul8.cds.counts <- Mmul8.cds.counts[, metadata.Mmul8$experiment_accession]

coldata <- data.frame(row.names=colnames(Mmul8.cds.counts), sex, assay)
dds <- DESeqDataSetFromMatrix(countData=Mmul8.cds.counts,
                              colData=coldata,
                              design=~assay)
dds <- DESeq(dds)
dds <- dds[apply(counts(dds), 1, function(row) all(row !=0 )),]
rld  <- rlogTransformation(dds , blind=TRUE)

annotation.df <- as.data.frame(colData(dds)[, c("assay", "sex")])
rownames(annotation.df) <- rownames(colData(dds))
colnames(annotation.df) <- c("assay", "sex")
pheatmap(assay(rld), show_rownames = F, cluster_rows = F, annotation_col=annotation.df, annotation_colors = colors)
pheatmap(assay(rld), show_rownames = F, cluster_rows = F, annotation_col=annotation.df, annotation_colors = colors, file='../../re-ribo-smk/plots/Mmul8/Mmul8_cds_rna_ribo_heatmap.pdf')

data <- plotPCA(rld, intgroup = c("assay", "sex"), returnData=TRUE)
percentVar <- round(100 * attr(data, "percentVar"))

ggplot(data, aes(PC1, PC2, color=assay, shape=sex, label = rownames(data))) +
      scale_color_manual("assay", values=colors$assay) +
geom_text_repel() +
geom_point(size=3) +
xlab(paste0("PC1: ",percentVar[1],"% variance")) +
ylab(paste0("PC2: ",percentVar[2],"% variance")) +
coord_fixed() + #ggtitle('PCA -- all samples') +
  theme(text = element_text(size=12))
ggsave('../../re-ribo-smk/plots/Mmul8/Mmul8_cds_rna_ribo_pca.pdf')

results.Mmul8.TE <- results(dds)
write_results(results.Mmul8.TE,
              "/home/cmb-panasas2/skchoudh/github_projects/re-ribo-smk/data/ortho-datasets-translational-efficiency/",
              "Mmul8_TE")

Mmul8.uORF.counts <- read.csv('../../re-ribo-smk/data/ortho-datasets-counts/Mmul8_uORF.tsv', sep='\t', row.names=1)
Mmul8.uORF.counts <- Mmul8.uORF.counts[, metadata.Mmul8$experiment_accession]

coldata <- data.frame(row.names=colnames(Mmul8.uORF.counts), sex, assay)
dds <- DESeqDataSetFromMatrix(countData=Mmul8.uORF.counts,
                              colData=coldata,
                              design=~assay)
dds <- DESeq(dds)
dds <- dds[apply(counts(dds), 1, function(row) all(row !=0 )),]
rld  <- rlogTransformation(dds , blind=TRUE)

annotation.df <- as.data.frame(colData(dds)[, c("assay", "sex")])
rownames(annotation.df) <- rownames(colData(dds))
colnames(annotation.df) <- c("assay", "sex")
pheatmap(assay(rld), show_rownames = F, cluster_rows = F, annotation_col=annotation.df, annotation_colors = colors)
pheatmap(assay(rld), show_rownames = F, cluster_rows = F, annotation_col=annotation.df, annotation_colors = colors, file='../../re-ribo-smk/plots/Mmul8/Mmul8_uORF_rna_ribo_heatmap.pdf')

data <- plotPCA(rld, intgroup = c("assay", "sex"), returnData=TRUE)
percentVar <- round(100 * attr(data, "percentVar"))

ggplot(data, aes(PC1, PC2, color=assay, shape=sex, label = rownames(data))) +
      scale_color_manual("assay", values=colors$assay) +
geom_text_repel() +
geom_point(size=3) +
xlab(paste0("PC1: ",percentVar[1],"% variance")) +
ylab(paste0("PC2: ",percentVar[2],"% variance")) +
coord_fixed() + #ggtitle('PCA -- all samples') +
  theme(text = element_text(size=12))
ggsave('../../re-ribo-smk/plots/Mmul8/Mmul8_uORF_rna_ribo_pca.pdf')

results.Mmul8.dORF <- results(dds)
write_results(results.Mmul8.dORF,
              "/home/cmb-panasas2/skchoudh/github_projects/re-ribo-smk/data/ortho-datasets-translational-efficiency/",
              "Mmul8_dORF")
estimating size factors
estimating dispersions
gene-wise dispersion estimates
mean-dispersion relationship
final dispersion estimates
fitting model and testing

</code>