# Notebook from mora-lab/benchmarks
Path: time-course/workflows/TcGSA/TcGSA.ipynb

The TcGSA (Time-course Gene Set Analysis Hejblum et al., 2015) R package tests gene expression dynamics for significance in gene sets.

TcGSA relies on linear mixed model to take into account the potential heterogeneity of expression within a gene set._____no_output_____Run TcGSA need to requird:

    The gene set object

    The gene expression matrix

    The design data matrix_____no_output_____##### The gene set object_____no_output_____The GMT file format is a tab delimited file format that describes gene sets. In the GMT format, each row represents a gene set; in the GMX format, each column represents a gene set. The GMT file format is organized as follows:_____no_output_____![Gmt_format_snapshot.gif](picture/Gmt_format_snapshot.gif)_____no_output_____##### The gene expression matrix_____no_output_____This matrix contains the gene expression (in cells) for each gene (in rows) of each sample (in columns) gathered from microarray measurements. The gene expression should already be normalized before using TcGSA. In the rownames, the name of each probe/gene must match with the name of probes/genes in the gmt object._____no_output_____

![TcGSA2.png](picture/TcGSA2.png)_____no_output_____##### Design data matrix_____no_output_____The design data matrix contains for each sample (in row), several variables (in column). The variables required for the matrix are:

    Sample names
    Patient identifiers
    Time measurements
    In case of multiple treatments, the name of treatment_____no_output_____

![TcGSA3.png](picture/TcGSA3.png)_____no_output_____# Install TcGSA and GROquery_____no_output_____

if (!requireNamespace("GEOquery", quietly = TRUE)) {
    if (!requireNamespace("BiocManager", quietly = TRUE)){
        install.packages("BiocManager")
    }
    BiocManager::install("GEOquery")
}_____no_output_____if (!requireNamespace("TcGSA", quietly = TRUE)) {
    if (!requireNamespace("BiocManager", quietly = TRUE)){
        install.packages("BiocManager")
    }
    BiocManager::install("TcGSA")
}_____no_output_____

# Download gmts from Hejblum_____no_output_____

temp <- tempfile()
utils::download.file("http://doi.org/10.1371/journal.pcbi.1004310.s007", destfile = temp, mode = "wb")
load(unz(temp, "ReproducibleRFiles/GMTs_PLOScb.RData", open = "r"))
unlink(temp)
rm(temp)_____no_output_____

# one treatment group_____no_output_____## Import of data files._____no_output_____

GEOquery::getGEOSuppFiles('GSE46734', filter_regex="(*NonParamCombat*)|(*DESIGN*)")Setting options('download.file.method.GEOquery'='auto')
Setting options('GEOquery.inmemory.gpl'=FALSE)

## Design data matrix_____no_output_____In this example, we need import the supplementary files available on GEOwith the getGEOSuppFiles function (we only need the three following files, hence the regular expression filter: gene expression pre-ATI, gene expression post-ATI, experimental design):_____no_output_____

design <- read.delim(gzfile("GSE46734/GSE46734_DALIA1longitudinalTranscriptome_DESIGN_anonym.txt.gz"))
design_preATI <- design[-which(design$TimePoint<0 | design$TimePoint==16 | design$TimePoint>22), ]
head(design_preATI,5)_____no_output_____

## Gene expression matrix_____no_output_____

expr_preATI <- read.delim(gzfile("GSE46734/GSE46734_DALIA1longitudinalTranscriptome_PALO01_PreATI_NEQC_NonParamCombat.txt.gz"))
rownames(expr_preATI) <- expr_preATI$PROBE_ID
expr_preATI <- expr_preATI[,as.character(design_preATI$Sample_name)]

expr_preATI[1:4,1:4]_____no_output_____

## Likelihood ratios test_____no_output_____This function provides the result of likelihood ratio test using the linear mixed model for each gene set. For this example, we use gene sets data from Chaussabel s modules (Chaussabel et al., 2008). TcGSA.LR function requires

 这个函数使用线性混合模型为每个基因集提供似然比检验的结果，在本例中，我们使用了来自Chaussabel模块，TcGSA.LR()需要：

    expr:           name of the gene expression matrix
                  基因表达矩阵
    gmt:            name of the gmt gene set object
                  基因集对象
    design:          name of the design data matrix
                  数据设计矩阵
    subject_name:      name of the identification of patients in the design data matrix
                  在数据设计矩阵中的病人ID
    time_name:        name of the time measurements in the design data matrix
                  数据设计矩阵的时间测量名称_____no_output_____

tcgsa_result <- TcGSA::TcGSA.LR(expr = expr_preATI,
                                    gmt = gmt_modulesV2,
                                    design = design_preATI,
                                    subject_name = "Patient_ID",
                                    time_name = "TimePoint")boundary (singular) fit: see ?isSingular
1/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
2/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
3/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
4/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00315548 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
5/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00627848 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
6/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
7/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
8/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
9/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
10/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
11/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
12/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
13/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.0111042 (tol = 0.002, component 1)"14/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
15/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.0044986 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
16/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
17/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
18/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
19/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
20/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
21/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
22/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
23/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
24/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
25/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
26/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
27/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.0084516 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
28/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00730001 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
29/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
30/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
31/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
32/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
33/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
34/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
35/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00247216 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
36/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
37/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
38/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
39/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00214175 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
40/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
41/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
42/260 gene sets analyzed

43/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
44/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
45/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
46/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
47/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
48/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
49/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
50/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00797107 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
51/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
52/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
53/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
54/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
55/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
56/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
57/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
58/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
59/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00363796 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
60/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
61/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
62/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
63/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
64/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
65/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
66/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00513997 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
67/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
68/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
69/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
70/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
71/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
72/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00705769 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
73/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
74/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
75/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
76/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
77/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00223095 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
78/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
79/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
80/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
81/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
82/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00401723 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
83/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
84/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
85/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.0121487 (tol = 0.002, component 1)"86/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
87/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
88/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00690124 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
89/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
90/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
91/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00205648 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
92/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
93/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
94/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
95/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
96/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
97/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
98/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
99/260 gene sets analyzed

Warning message in TcGSA::TcGSA.LR(expr = expr_preATI, gmt = gmt_modulesV2, design = design_preATI, :
"The size of the gene set M8.100is problematic (too many or too few genes)
"100/260 gene sets analyzed

Warning message in TcGSA::TcGSA.LR(expr = expr_preATI, gmt = gmt_modulesV2, design = design_preATI, :
"The size of the gene set M8.101is problematic (too many or too few genes)
"101/260 gene sets analyzed

Warning message in TcGSA::TcGSA.LR(expr = expr_preATI, gmt = gmt_modulesV2, design = design_preATI, :
"The size of the gene set M8.102is problematic (too many or too few genes)
"102/260 gene sets analyzed

Warning message in TcGSA::TcGSA.LR(expr = expr_preATI, gmt = gmt_modulesV2, design = design_preATI, :
"The size of the gene set M8.103is problematic (too many or too few genes)
"103/260 gene sets analyzed

Warning message in TcGSA::TcGSA.LR(expr = expr_preATI, gmt = gmt_modulesV2, design = design_preATI, :
"The size of the gene set M8.104is problematic (too many or too few genes)
"104/260 gene sets analyzed

Warning message in TcGSA::TcGSA.LR(expr = expr_preATI, gmt = gmt_modulesV2, design = design_preATI, :
"The size of the gene set M8.105is problematic (too many or too few genes)
"105/260 gene sets analyzed

Warning message in TcGSA::TcGSA.LR(expr = expr_preATI, gmt = gmt_modulesV2, design = design_preATI, :
"The size of the gene set M8.106is problematic (too many or too few genes)
"106/260 gene sets analyzed

Warning message in TcGSA::TcGSA.LR(expr = expr_preATI, gmt = gmt_modulesV2, design = design_preATI, :
"The size of the gene set M8.107is problematic (too many or too few genes)
"107/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
108/260 gene sets analyzed

Warning message in TcGSA::TcGSA.LR(expr = expr_preATI, gmt = gmt_modulesV2, design = design_preATI, :
"The size of the gene set M8.109is problematic (too many or too few genes)
"109/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
110/260 gene sets analyzed

Warning message in TcGSA::TcGSA.LR(expr = expr_preATI, gmt = gmt_modulesV2, design = design_preATI, :
"The size of the gene set M8.110is problematic (too many or too few genes)
"111/260 gene sets analyzed

Warning message in TcGSA::TcGSA.LR(expr = expr_preATI, gmt = gmt_modulesV2, design = design_preATI, :
"The size of the gene set M8.111is problematic (too many or too few genes)
"112/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
113/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
114/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
115/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
116/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00235917 (tol = 0.002, component 1)"117/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
118/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
119/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
120/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
121/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00663961 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
122/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
123/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
124/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
125/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
126/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
127/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
128/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00443147 (tol = 0.002, component 1)"129/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
130/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
131/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
132/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
133/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
134/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
135/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.0917374 (tol = 0.002, component 1)"136/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
137/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00219818 (tol = 0.002, component 1)"138/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
139/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
140/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
141/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00205355 (tol = 0.002, component 1)"boundary (singular) fit: see ?isSingular
142/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
143/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
144/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00615577 (tol = 0.002, component 1)"145/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
146/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
147/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
148/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
149/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
150/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
151/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
152/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
153/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
154/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
155/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
156/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
157/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
158/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
159/260 gene sets analyzed

160/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
161/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
162/260 gene sets analyzed

163/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
164/260 gene sets analyzed

Look at the result:_____no_output_____

tcgsa_result_____no_output_____

Now `tcgsa_result` is a `tcgsa` object containing, in addition to the likelihood ratio test results:

    the form of tested time trends (default option is linear)

    the number of treatment groups and the number of gene sets tested (depending on which gene set base is defined)

    a few other information_____no_output_____## To get the number of significant gene sets_____no_output_____To get the number of significant gene sets, one can use summary function on a `tcgsa` object:_____no_output_____

summary(tcgsa_result)_____no_output_____

## To get more details on the significant gene sets_____no_output_____To get more details on the significant gene sets, use the `signifLRT.TcGSA()` function._____no_output_____

TcGSA::signifLRT.TcGSA(tcgsa_result)_____no_output_____

## To get the likelihood ratios, the raw and adjusted p-values_____no_output_____You can also use the multtest.TcGSA function to provide the likelihood ratios, the raw and adjusted p-values for the whole gene sets with 5% threshold. Below is an example displaying only for five results:_____no_output_____

head(TcGSA::multtest.TcGSA(tcgsa_result))_____no_output_____

## Graphical outputs_____no_output_____plot1GS() plots the different representations of gene expression in a specific gene set of interest (specified by the  geneset.name argument). This function requires the following:

 plot1GS()对感兴趣的特定基因集中基因表达的不同表达（由geneset name 参数指定）。参数如下：

    expr:        either the name of the gene expression matrix or the estimations of linear mixed model (in this example, we used the raw data from the gene expression matrix)
               要么是基因表达矩阵要么是线性混合模型的估计值（在本例中，使用原始数据基因表达矩阵

    gmt:         the name of the gmt gene set object
               gene set对象名称

    Subject_ID:    the name of the identification of patients in the design data matrix
               数据设计矩阵中病人id的名称

    TimePoint:     the name of the time measurements of the design data matrix
                数据设计矩阵的时间测量名称

    geneset.name:   the name of gene set (significant ones can be found with signifLRT.TcGSA(tcgsa_result)$mixedLRTadjRes)
                gene set 名称

    time_unit:      string to be displayed before to the values of TimePoint on the x-axis (such as ‘D’ for ‘days’ for instance - optional)
                要在此之前显示到x轴上的时间点值的字符串_____no_output_____

TcGSA::plot1GS(expr = expr_preATI,
               #plot1GS(expr = tcgsa_result$Estimations,
               gmt = gmt_modulesV2,
               Subject_ID = design_preATI$Patient_ID,
               TimePoint = design_preATI$TimePoint,
               clustering = FALSE,
               time_unit = "W",
               geneset.name = "M3.2",
               title="",
               margins=0.4,
               lab.cex=0.37,
               axis.cex=0.37,
               line.size=0.45,
               gg.add=list(ggplot2::theme(legend.position="none"),
                        ggplot2::ylim(-1.26,1.26)
               ))Optimally clustering...

DONE

Scale for 'y' is already present. Adding another scale for 'y', which will
replace the existing scale.

## Make clusters from tcgsa object_____no_output_____clustTrend builds clusters of genes from their trends dynamics. clustTrend function requires:

TcGSA::clustTrend() 从基因的动态趋势中构建基因族，参数如下：

    tcgs: your TcGSA object
        TcGSA对象

    expr: estimation of gene expressions with linear mixed model from TcGSA object
        TcGSA对象利用线性混合模型后的基因表达的估计值

    Subject_ID: name of the identification of patients in the design data matrix
            数据设计矩阵中的病人ID
    TimePoint: name of the time measurements in the design data matrix
            数据设计矩阵中时间测量的方法的名称
    baseline (optional): value of TimePoint used to be the reference
                   时间点的值
    group_of_interest: name of a treatment in the design data matrix
                数据设计矩阵中处理组的名称_____no_output_____

clust <- TcGSA::clustTrend(tcgs = tcgsa_result,
                           expr=tcgsa_result$Estimations,
                           Subject_ID=design_preATI$Patient_ID,
                           TimePoint=design_preATI$TimePoint)1/17 gene sets clustered

2/17 gene sets clustered

3/17 gene sets clustered

4/17 gene sets clustered

5/17 gene sets clustered

6/17 gene sets clustered

7/17 gene sets clustered

8/17 gene sets clustered

9/17 gene sets clustered

10/17 gene sets clustered

11/17 gene sets clustered

12/17 gene sets clustered

13/17 gene sets clustered

14/17 gene sets clustered

15/17 gene sets clustered

16/17 gene sets clustered

17/17 gene sets clustered

## Heatmap of significant gene sets_____no_output_____plot draws different kinds of graphics, but we focus on heatmap graphics. This function requires:

    x: a tcgsa object
    tcgsa 对象

    expr: estimation of gene expressions with linear mixed model from a tcgsa object
    TcGSA对象利用线性混合模型后的基因表达的估计值

    Subject_ID: name of the subject identifier variable in the design data matrix
    数据设计矩阵中的病人ID

    TimePoint: name of the time measurement variable in the design data matrix
    数据设计矩阵中时间测量的方法的名称

    group_of_interest: name of the treatment factor variable in the design data matrix
    设计数据矩阵中处理因子变量的名称

    clust_trends: cluster object with the clusters of genes from their trends dynamics
    聚类对象：从它们的动态变化趋势聚类的基因_____no_output_____

plot(x=tcgsa_result, expr=tcgsa_result$Estimations,
     Subject_ID=design_preATI$Patient_ID,
     TimePoint=design_preATI$TimePoint,
     clust_trends=clust,
     legend.breaks=seq(from=-2,to=2, by=0.01), time_unit="D",
     subtitle="HIV", cex.label.row=0.5, cex.label.col=1, cex.main=0.7,
     heatmap.width=0.2, dendrogram.size=0.3, margins=c(2,3),
     heatKey.size=0.8)_____no_output_____

# several treatment group_____no_output_____## Data preparation_____no_output_____download the data files and import them with `getGEO()` function:_____no_output_____

#utils::download.file("ftp://ftp.ncbi.nlm.nih.gov/geo/series/GSE30nnn/GSE30101/soft/GSE30101_family.soft.gz", destfile = "GSE30101_family.soft.gz", mode = "wb", cacheOK = FALSE)
gse.soft <- GEOquery::getGEO(filename="GSE30101_family.soft.gz")Setting options('download.file.method.GEOquery'='auto')
Setting options('GEOquery.inmemory.gpl'=FALSE)
Reading file....
Parsing....
Found 694 entities...
GPL6947 (1 of 695 entities)
GSM744141 (2 of 695 entities)
GSM744142 (3 of 695 entities)
GSM744143 (4 of 695 entities)
GSM744144 (5 of 695 entities)
GSM744145 (6 of 695 entities)
GSM744146 (7 of 695 entities)
GSM744147 (8 of 695 entities)
GSM744148 (9 of 695 entities)
GSM744149 (10 of 695 entities)
GSM744150 (11 of 695 entities)
GSM744151 (12 of 695 entities)
GSM744152 (13 of 695 entities)
GSM744153 (14 of 695 entities)
GSM744154 (15 of 695 entities)
GSM744155 (16 of 695 entities)
GSM744156 (17 of 695 entities)
GSM744157 (18 of 695 entities)
GSM744158 (19 of 695 entities)
GSM744159 (20 of 695 entities)
GSM744160 (21 of 695 entities)
GSM744161 (22 of 695 entities)
GSM744162 (23 of 695 entities)
GSM744163 (24 of 695 entities)
GSM744164 (25 of 695 entities)
GSM744165 (26 of 695 entities)
GSM744166 (27 of 695 entities)
GSM744167 (28 of 695 entities)
GSM744168 (29 of 695 entities)
GSM744169 (30 of 695 entities)
GSM744170 (31 of 695 entities)
GSM744171 (32 of 695 entities)
GSM744172 (33 of 695 entities)
GSM744173 (34 of 695 entities)
GSM744174 (35 of 695 entities)
GSM744175 (36 of 695 entities)
GSM744176 (37 of 695 entities)
GSM744177 (38 of 695 entities)
GSM744178 (39 of 695 entities)
GSM744179 (40 of 695 entities)
GSM744180 (41 of 695 entities)
GSM744181 (42 of 695 entities)
GSM744182 (43 of 695 entities)
GSM744183 (44 of 695 entities)
GSM744184 (45 of 695 entities)
GSM744185 (46 of 695 entities)
GSM744186 (47 of 695 entities)
GSM744187 (48 of 695 entities)
GSM744188 (49 of 695 entities)
GSM744189 (50 of 695 entities)
GSM744190 (51 of 695 entities)
GSM744191 (52 of 695 entities)
GSM744192 (53 of 695 entities)
GSM744193 (54 of 695 entities)
GSM744194 (55 of 695 entities)
GSM744195 (56 of 695 entities)
GSM744196 (57 of 695 entities)
GSM744197 (58 of 695 entities)
GSM744198 (59 of 695 entities)
GSM744199 (60 of 695 entities)
GSM744200 (61 of 695 entities)
GSM744201 (62 of 695 entities)
GSM744202 (63 of 695 entities)
GSM744203 (64 of 695 entities)
GSM744204 (65 of 695 entities)
GSM744205 (66 of 695 entities)
GSM744206 (67 of 695 entities)
GSM744207 (68 of 695 entities)
GSM744208 (69 of 695 entities)
GSM744209 (70 of 695 entities)
GSM744210 (71 of 695 entities)
GSM744211 (72 of 695 entities)
GSM744212 (73 of 695 entities)
GSM744835 (74 of 695 entities)
GSM744836 (75 of 695 entities)
GSM744837 (76 of 695 entities)
GSM744838 (77 of 695 entities)
GSM744839 (78 of 695 entities)
GSM744840 (79 of 695 entities)
GSM744841 (80 of 695 entities)
GSM744842 (81 of 695 entities)
GSM744843 (82 of 695 entities)
GSM744844 (83 of 695 entities)
GSM744845 (84 of 695 entities)
GSM744846 (85 of 695 entities)
GSM744847 (86 of 695 entities)
GSM744848 (87 of 695 entities)
GSM744849 (88 of 695 entities)
GSM744850 (89 of 695 entities)
GSM744851 (90 of 695 entities)
GSM744852 (91 of 695 entities)
GSM744853 (92 of 695 entities)
GSM744854 (93 of 695 entities)
GSM744855 (94 of 695 entities)
GSM744856 (95 of 695 entities)
GSM744857 (96 of 695 entities)
GSM744858 (97 of 695 entities)
GSM744859 (98 of 695 entities)
GSM744860 (99 of 695 entities)
GSM744861 (100 of 695 entities)
GSM744862 (101 of 695 entities)
GSM744863 (102 of 695 entities)
GSM744864 (103 of 695 entities)
GSM744865 (104 of 695 entities)
GSM744866 (105 of 695 entities)
GSM744867 (106 of 695 entities)
GSM744868 (107 of 695 entities)
GSM744869 (108 of 695 entities)
GSM744870 (109 of 695 entities)
GSM744871 (110 of 695 entities)
GSM744872 (111 of 695 entities)
GSM744873 (112 of 695 entities)
GSM744874 (113 of 695 entities)
GSM744875 (114 of 695 entities)
GSM744876 (115 of 695 entities)
GSM744877 (116 of 695 entities)
GSM744878 (117 of 695 entities)
GSM744879 (118 of 695 entities)
GSM744880 (119 of 695 entities)
GSM744881 (120 of 695 entities)
GSM744882 (121 of 695 entities)
GSM744883 (122 of 695 entities)
GSM744884 (123 of 695 entities)
GSM744885 (124 of 695 entities)
GSM744886 (125 of 695 entities)
GSM744887 (126 of 695 entities)
GSM744888 (127 of 695 entities)
GSM744889 (128 of 695 entities)
GSM744890 (129 of 695 entities)
GSM744891 (130 of 695 entities)
GSM744892 (131 of 695 entities)
GSM744893 (132 of 695 entities)
GSM744894 (133 of 695 entities)
GSM744895 (134 of 695 entities)
GSM744896 (135 of 695 entities)
GSM744897 (136 of 695 entities)
GSM744898 (137 of 695 entities)
GSM744899 (138 of 695 entities)
GSM744900 (139 of 695 entities)
GSM744901 (140 of 695 entities)
GSM744902 (141 of 695 entities)
GSM744903 (142 of 695 entities)
GSM744904 (143 of 695 entities)
GSM744905 (144 of 695 entities)
GSM744906 (145 of 695 entities)
GSM744907 (146 of 695 entities)
GSM744908 (147 of 695 entities)
GSM744909 (148 of 695 entities)
GSM744910 (149 of 695 entities)
GSM744911 (150 of 695 entities)
GSM744912 (151 of 695 entities)
GSM744913 (152 of 695 entities)
GSM744914 (153 of 695 entities)
GSM744915 (154 of 695 entities)
GSM744916 (155 of 695 entities)
GSM744917 (156 of 695 entities)
GSM744918 (157 of 695 entities)
GSM744919 (158 of 695 entities)
GSM744920 (159 of 695 entities)
GSM744921 (160 of 695 entities)
GSM744922 (161 of 695 entities)
GSM744923 (162 of 695 entities)
GSM744924 (163 of 695 entities)
GSM744925 (164 of 695 entities)
GSM744926 (165 of 695 entities)
GSM744927 (166 of 695 entities)
GSM744928 (167 of 695 entities)
GSM744929 (168 of 695 entities)
GSM744930 (169 of 695 entities)
GSM744931 (170 of 695 entities)
GSM744932 (171 of 695 entities)
GSM744933 (172 of 695 entities)
GSM744934 (173 of 695 entities)
GSM744935 (174 of 695 entities)
GSM744936 (175 of 695 entities)
GSM744937 (176 of 695 entities)
GSM744938 (177 of 695 entities)
GSM744939 (178 of 695 entities)
GSM744940 (179 of 695 entities)
GSM744941 (180 of 695 entities)
GSM744942 (181 of 695 entities)
GSM744943 (182 of 695 entities)
GSM744944 (183 of 695 entities)
GSM744945 (184 of 695 entities)
GSM744946 (185 of 695 entities)
GSM744947 (186 of 695 entities)
GSM744948 (187 of 695 entities)
GSM744949 (188 of 695 entities)
GSM744950 (189 of 695 entities)
GSM744951 (190 of 695 entities)
GSM744952 (191 of 695 entities)
GSM744953 (192 of 695 entities)
GSM744954 (193 of 695 entities)
GSM744955 (194 of 695 entities)
GSM744956 (195 of 695 entities)
GSM744957 (196 of 695 entities)
GSM744958 (197 of 695 entities)
GSM744959 (198 of 695 entities)
GSM744960 (199 of 695 entities)
GSM744961 (200 of 695 entities)
GSM744962 (201 of 695 entities)
GSM744963 (202 of 695 entities)
GSM744964 (203 of 695 entities)
GSM744965 (204 of 695 entities)
GSM744966 (205 of 695 entities)
GSM744967 (206 of 695 entities)
GSM744968 (207 of 695 entities)
GSM744969 (208 of 695 entities)
GSM744970 (209 of 695 entities)
GSM744971 (210 of 695 entities)
GSM744972 (211 of 695 entities)
GSM744973 (212 of 695 entities)
GSM744974 (213 of 695 entities)
GSM744975 (214 of 695 entities)
GSM744976 (215 of 695 entities)
GSM744977 (216 of 695 entities)
GSM744978 (217 of 695 entities)
GSM744979 (218 of 695 entities)
GSM744980 (219 of 695 entities)
GSM744981 (220 of 695 entities)
GSM744982 (221 of 695 entities)
GSM744983 (222 of 695 entities)
GSM744984 (223 of 695 entities)
GSM744985 (224 of 695 entities)
GSM744986 (225 of 695 entities)
GSM744987 (226 of 695 entities)
GSM744988 (227 of 695 entities)
GSM744989 (228 of 695 entities)
GSM744990 (229 of 695 entities)
GSM744991 (230 of 695 entities)
GSM744992 (231 of 695 entities)
GSM744993 (232 of 695 entities)
GSM744994 (233 of 695 entities)
GSM744995 (234 of 695 entities)
GSM744996 (235 of 695 entities)
GSM744997 (236 of 695 entities)
GSM744998 (237 of 695 entities)
GSM744999 (238 of 695 entities)
GSM745000 (239 of 695 entities)
GSM745001 (240 of 695 entities)
GSM745002 (241 of 695 entities)
GSM745003 (242 of 695 entities)
GSM745004 (243 of 695 entities)
GSM745005 (244 of 695 entities)
GSM745006 (245 of 695 entities)
GSM745007 (246 of 695 entities)
GSM745008 (247 of 695 entities)
GSM745009 (248 of 695 entities)
GSM745010 (249 of 695 entities)
GSM745011 (250 of 695 entities)
GSM745012 (251 of 695 entities)
GSM745013 (252 of 695 entities)
GSM745014 (253 of 695 entities)
GSM745015 (254 of 695 entities)
GSM745016 (255 of 695 entities)

## gene expression matrix_____no_output_____

probesIDs <- GEOquery::Table(GEOquery::GSMList(gse.soft)[[1]])$ID
data.matrix <- do.call('cbind', lapply(GEOquery::GSMList(gse.soft),
                                       function(x) {
                                        tab <- GEOquery::Table(x)
                                        mymatch <- match(probesIDs,tab$ID_REF)
                                        return(tab$VALUE[mymatch])
                                       }))
rownames(data.matrix) <- probesIDs
expr.All.ChaussVac <- apply(X = data.matrix, MARGIN = 2, FUN = as.numeric)
rownames(expr.All.ChaussVac) <- probesIDs_____no_output_____

## design data matrix_____no_output_____

design_list <- lapply(GEOquery::GSMList(gse.soft),
                      function(x){GEOquery::Meta(x)$characteristics_ch1})

#为design添加行名
design <- data.frame(row.names = names(design_list))

#为design添加sample_ID这一列
design$sample_ID <- names(design_list)

#为design添加Subject_ID这一列
s_id <- unlist(lapply(design_list,
                      function(x){
                        gsub("subject id: ", "",
                             #gsub用于字段的删减、增补、替换和切合，可以处理一个字段也可以处理由字段组成的向量。
                             #gsub("目标字符","替换字符",对象)
                             x[grep("subject id: ", x)])}))
                            #grep()输出向量下标或值，通过正则表达式在给定对象中搜索文本
design$Subject_ID <- as.character(paste("P", s_id[design$sample_ID], sep=""))

#为design添加time这一列
time <- unlist(lapply(design_list, function(x){gsub("day: ", "", x[grep("day: ", x)])}))
time[which(time %in% c("-7", "0.5", "1", "7", "10", "14",  "21", "28"))] <-
    paste("D", time[which(time %in% c("-7", "0.5", "1", "7", "10", "14", "21", "28"))], sep="")
time[which(time %in% c("-168", "1.5", "6", "9", "12", "15", "24", "36", "48"))] <-
    paste("H", time[which(time %in% c("-168", "1.5", "6", "9", "12", "15", "24", "36", "48"))], sep="")
design$Time <- as.character(time[design$sample_ID])

#为design添加Vaccine这一列
vac <- unlist(lapply(design_list, function(x){
    gsub("vaccine: ", "", x[grep("vaccine: ", x)])
}))
vac <- as.factor(vac)
levels(vac) <- c("influenza", "influenza", "influenza", "influenza", "saline",
                 "pneumo", "pneumo", "pneumo", "saline", "saline")
design$Vaccine <- as.character(vac[design$sample_ID])

#为design添加sampSet这一列
sampSet <- unlist(lapply(design_list, function(x){
    gsub("sample set: ", "", x[grep("sample set: ", x)])
}))
design$sampSet <- as.character(sampSet[design$sample_ID])

#为design中的Time这一列更改
design$Time[which(design$sampSet=="Training_Set_Vein" & design$Time %in% c("0", "3"))] <-
    paste("D", design$Time[which(design$sampSet=="Training_Set_Vein" & design$Time %in% c("0", "3"))], sep="")
design$Time[which(design$sampSet=="Training_Set_Finger" & design$Time %in% c("0", "3"))] <-
    paste("H", design$Time[which(design$sampSet=="Training_Set_Finger" & design$Time %in% c("0", "3"))], sep="")
design$Time[which(design$sampSet=="Test_Set_Vein" & design$Time %in% c("0", "3"))] <-
    paste("D", design$Time[which(design$sampSet=="Test_Set_Vein" & design$Time %in% c("0", "3"))], sep="")
design$Time[which(design$sampSet=="Test_Set_Finger" & design$Time %in% c("0", "3"))] <-
    paste("D", design$Time[which(design$sampSet=="Test_Set_Finger" & design$Time %in% c("0", "3"))], sep="")
design$Time[which(design$sampSet=="Validation_Vein" & design$Time %in% c("0", "3"))] <-
    paste("D", design$Time[which(design$sampSet=="Validation_Vein" & design$Time %in% c("0", "3"))], sep="")

#为design添加Day这一列
design$Day <- gsub("D", "", design$Time)
design$Day[grep("H", design$Day)] <- as.numeric(gsub("H", "", design$Day[grep("H", design$Day)]))/24
design$Day <- as.numeric(design$Day)

design.All.ChaussVac <- design

# Avg Baseline -----
design.All.ChaussVac.trainSetVein <- design.All.ChaussVac[which(design.All.ChaussVac$sampSet=="Training_Set_Vein"),]
samplesSaline2rmv <- design.All.ChaussVac.trainSetVein[162:214,"sample_ID"]
design.All.ChaussVac.noDup <- design.All.ChaussVac.trainSetVein[-which(design.All.ChaussVac.trainSetVein$sample_ID%in%samplesSaline2rmv),]

design.All.ChaussVac.AvgBl <- design.All.ChaussVac.noDup[which(design.All.ChaussVac.noDup$Day!=0),]
design.All.ChaussVac.AvgBl[which(design.All.ChaussVac.AvgBl$Day==-7),"Day"] <- 0
design.All.ChaussVac.AvgBl[which(design.All.ChaussVac.AvgBl$Time=="D-7"),"Time"] <- "D0"

expr.All.ChaussVac.AvgBl <- expr.All.ChaussVac[, design.All.ChaussVac.AvgBl$sample_ID]
for(p in unique(design.All.ChaussVac.AvgBl$Subject_ID)){
    if(length(which(design.All.ChaussVac.noDup$Subject_ID==p & (design.All.ChaussVac.noDup$Day==0 | design.All.ChaussVac.noDup$Day==-7)))>1){
        expr.All.ChaussVac.AvgBl[, which(design.All.ChaussVac.AvgBl$Subject_ID==p & design.All.ChaussVac.AvgBl$Day==0)] <-
            apply(X=cbind(expr.All.ChaussVac[, design.All.ChaussVac.noDup[which(design.All.ChaussVac.noDup$Subject_ID==p & design.All.ChaussVac.noDup$Day==0), "sample_ID"]],
                          expr.All.ChaussVac[, design.All.ChaussVac.noDup[which(design.All.ChaussVac.noDup$Subject_ID==p & design.All.ChaussVac.noDup$Day==-7), "sample_ID"]]),
                  MARGIN=1, FUN=mean, na.rm=TRUE)
    }
}
rownames(expr.All.ChaussVac.AvgBl) <- probesIDs

if(!all.equal(as.character(design.All.ChaussVac.AvgBl$sample_ID), colnames(expr.All.ChaussVac.AvgBl))){stop("\n\n\nWARNING: EXPRESSION FILE ORDER NOT MATCHING DESIGN FILE\n\n\n")}
design.All.ChaussVac.AvgBl$Subject_ID <- as.factor(design.All.ChaussVac.AvgBl$Subject_ID)

design.PNEUMOvsSALINE.ChaussVac.AvgBl <- design.All.ChaussVac.AvgBl[which(design.All.ChaussVac.AvgBl$Vaccine!="influenza"), ]
design.PNEUMOvsSALINE.ChaussVac.AvgBl$Vaccine <- as.factor(as.character(design.PNEUMOvsSALINE.ChaussVac.AvgBl$Vaccine))

expr.PNEUMOvsSALINE.ChaussVac.AvgBl <- expr.All.ChaussVac.AvgBl[,design.PNEUMOvsSALINE.ChaussVac.AvgBl$sample_ID]Your code contains a unicode char which cannot be displayed in your
current locale and R will silently convert it to an escaped form when the
R kernel executes this code. This can lead to subtle errors if you use
such chars to do comparisons. For more information, please see
https://github.com/IRkernel/repr/wiki/Problems-with-unicode-on-windows

## Likelihood ratio tests_____no_output_____

tcgsa_result_MT <- TcGSA::TcGSA.LR(expr = expr.PNEUMOvsSALINE.ChaussVac.AvgBl,
                                   gmt = gmt_modulesV2,
                                   design = design.PNEUMOvsSALINE.ChaussVac.AvgBl,
                                   subject_name = "Subject_ID",
                                   time_name = "Day",
                                   group_name = "Vaccine")boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
1/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
2/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
3/260 gene sets analyzed

4/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
5/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
6/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
7/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
8/260 gene sets analyzed

9/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
10/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
11/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
12/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
13/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
14/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
15/260 gene sets analyzed

16/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
17/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
18/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
19/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
20/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00204635 (tol = 0.002, component 1)"21/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00779801 (tol = 0.002, component 1)"22/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
23/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00703905 (tol = 0.002, component 1)"24/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
25/260 gene sets analyzed

26/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
27/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.0052332 (tol = 0.002, component 1)"28/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00226821 (tol = 0.002, component 1)"29/260 gene sets analyzed

30/260 gene sets analyzed

31/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
32/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.0201355 (tol = 0.002, component 1)"33/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
34/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
35/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.0325874 (tol = 0.002, component 1)"36/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
37/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
38/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
39/260 gene sets analyzed

40/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00253541 (tol = 0.002, component 1)"41/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
42/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
43/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.0020418 (tol = 0.002, component 1)"44/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
45/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
46/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
47/260 gene sets analyzed

48/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
49/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
50/260 gene sets analyzed

51/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
52/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
53/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
54/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00414656 (tol = 0.002, component 1)"55/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00655315 (tol = 0.002, component 1)"56/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00204241 (tol = 0.002, component 1)"57/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
58/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00571298 (tol = 0.002, component 1)"59/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
60/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
61/260 gene sets analyzed

62/260 gene sets analyzed

63/260 gene sets analyzed

64/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
65/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
66/260 gene sets analyzed

67/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00341143 (tol = 0.002, component 1)"68/260 gene sets analyzed

69/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
70/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.00336097 (tol = 0.002, component 1)"71/260 gene sets analyzed

72/260 gene sets analyzed

boundary (singular) fit: see ?isSingular
boundary (singular) fit: see ?isSingular
73/260 gene sets analyzed

Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
"Model failed to converge with max|grad| = 0.0188226 (tol = 0.002, component 1)"Warning message in checkConv(attr(opt, "derivs"), opt$par, ctrl = control$checkConv, :
summary(tcgsa_result_MT)_____no_output_____

## To get more details on the significant gene sets_____no_output_____

TcGSA::signifLRT.TcGSA(tcgsa_result_MT)_____no_output_____

## Make clusters from tcgsa object_____no_output_____

clust <- TcGSA::clustTrend(tcgs = tcgsa_result_MT,
                           expr=tcgsa_result_MT$Estimations,
                           Subject_ID=design.PNEUMOvsSALINE.ChaussVac.AvgBl$Patient_ID,
                           TimePoint=design.PNEUMOvsSALINE.ChaussVac.AvgBl$Day,
                           group.var = design.PNEUMOvsSALINE.ChaussVac.AvgBl$Vaccine,
                           group_of_interest="pneumo",
                           ref="saline")1/36 gene sets clustered

2/36 gene sets clustered

3/36 gene sets clustered

4/36 gene sets clustered

5/36 gene sets clustered

6/36 gene sets clustered

7/36 gene sets clustered

8/36 gene sets clustered

9/36 gene sets clustered

10/36 gene sets clustered

11/36 gene sets clustered

12/36 gene sets clustered

13/36 gene sets clustered

14/36 gene sets clustered

15/36 gene sets clustered

16/36 gene sets clustered

17/36 gene sets clustered

18/36 gene sets clustered

19/36 gene sets clustered

20/36 gene sets clustered

21/36 gene sets clustered

22/36 gene sets clustered

23/36 gene sets clustered

24/36 gene sets clustered

25/36 gene sets clustered

26/36 gene sets clustered

27/36 gene sets clustered

28/36 gene sets clustered

29/36 gene sets clustered

30/36 gene sets clustered

31/36 gene sets clustered

32/36 gene sets clustered

33/36 gene sets clustered

34/36 gene sets clustered

35/36 gene sets clustered

36/36 gene sets clustered

clust_____no_output_____

## Heatmap of significant gene sets_____no_output_____

plot(x=tcgsa_result_MT, expr=tcgsa_result_MT$Estimations,
     Subject_ID=design.PNEUMOvsSALINE.ChaussVac.AvgBl$Patient_ID,
     TimePoint=design.PNEUMOvsSALINE.ChaussVac.AvgBl$TimePoint,
     group_of_interest="pneumo",
     clust_trends=clust,
     legend.breaks=seq(from=-2,to=2, by=0.01), time_unit="D",
     subtitle="Pneumo vs Saline", cex.label.row=0.5, cex.label.col=1, cex.main=0.7,
     heatmap.width=0.2, dendrogram.size=0.3, margins=c(2,3),
     heatKey.size=0.8)_____no_output_____

# Reference_____no_output_____[Time-Course Gene Set Analysis for Longitudinal Gene Expression Data](https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1004310)_____no_output_____[User guide to the TcGSA R package](https://cran.r-project.org/web/packages/TcGSA/vignettes/TcGSA_userguide.html)_____no_output_____


# Notebook from cumc/xqtl-pipeline
Path: code/molecular_phenotypes/calling/splicing_calling.ipynb

# Quantifying alternative splicing from RNA-seq data

This pipeline implements our pipeline to call alternative splicing events from RNA-seq data, using [`leafcutter`](https://www.nature.com/articles/s41588-017-0004-9) and [`psichomics`](https://academic.oup.com/nar/article/47/2/e7/5114259) to call the RNA-seq data from original `fastq.gz` data. It implements the GTEx pipeline for GTEx/TOPMed project. Please refer to [this page](https://github.com/broadinstitute/gtex-pipeline/blob/master/TOPMed_RNAseq_pipeline.md) for detail. The choice of pipeline modules in this project is supported by internal (unpublished) benchmarks from GTEx group.

**Various reference data needs to be prepared before using this workflow**. [Here we provide a module](https://cumc.github.io/xqtl-pipeline/code/data_preprocessing/reference_data.html) to download and prepare the reference data.

The product of this workflow can be used in generating phenotype tables using /molecular_phenotyles/QC/splicing_normalization.ipynb._____no_output_____## Methods overview

There are many types of alternative splicing events. See [Wang et al (2008)](https://pubmed.ncbi.nlm.nih.gov/18978772/) and [Park et al (2018)](https://pubmed.ncbi.nlm.nih.gov/29304370/) for an illustration on different events and how splicings are controlled. We will apply two methods to quantify alternative splicing:

1. [`psichomics`](https://academic.oup.com/nar/article/47/2/e7/5114259) that quantifies each specific event. In particular the exon skipping event which is used also in GTEx sQTL analysis.
2. [`leafcutter`](https://www.nature.com/articles/s41588-017-0004-9) to quantify the usage of alternatively excised introns. This collectively captures skipped exons, 5’ and 3’ alternative splice site usage and other complex events. The method was previously applied to ROSMAP data as part of the Brain xQTL version 2.0. _____no_output_____## Input

### `leafcutter`

The bam file can be generated by `the STAR_align` workflow from our RNA_calling.ipynb module.

A meta-data file, white space delimited without header, containing 3 columns: sample ID, RNA strandness and path to the BAM file:

```
sample_1 rf samp1.bam
sample_2 fr samp2.bam
sample_3 unstranded samp3.bam
```

All the BAM files should be available under specified folder (default assumes the same folder as where the meta-data file is).

If intend to blacklist some chromosomes and not analyze it, add one text file named black_list.txt with one chromosome name per line in the same directory of the meta-data file.

### `psichomics`

_____no_output_____## Output

### `leafcutter`

`{sample_list}` below refers to the name of the meta-data file input.

Main output include:

- `{sample_list}_intron_usage_perind.counts.gz` file with row id in format: "chromosome:intron_start:intron_end:cluster_id", column labeled as input sample names and each type of intron usage ratio under each sample (i.e. #particular intron in a sample / #total introns classified in the same cluster in a sample) in each cells.
- `{sample_list}_intron_usage_perind_numers.counts.gz` file with the same row and column label but the count of each intron in each cells.

### `psichomics`_____no_output_____## Minimal working example
_____no_output_____### For `leafcutter`
A minimal working example is uploaded in the [google drive](https://drive.google.com/drive/folders/1lpcx3eKG2UpauntLUuJ6bMBjHyIhWW_R?usp=sharing)_____no_output_____

sos run pipeline/splicing_calling.ipynb leafcutter \
    --cwd output/ \
    --samples sample_bam.list \
    --container containers/leafcutter.sif _____no_output_____

### For `psichomics`_____no_output_____

sos run splicing_calling.ipynb psichomics \
    --cwd output/rnaseq/splicing \
    --samples data/sample_bam.list \
    --data-dir data \
    --container container/splicing.sif_____no_output_____

## Command interface_____no_output_____

sos run splicing_calling.ipynb -husage: sos run splicing_calling.ipynb
               [workflow_name | -t targets] [options] [workflow_options]
  workflow_name:        Single or combined workflows defined in this script
  targets:              One or more targets to generate
  options:              Single-hyphen sos parameters (see "sos run -h" for details)
  workflow_options:     Double-hyphen workflow-specific parameters

Workflows:
  leafcutter

Global Workflow Options:
  --cwd output (as path)
                        The output directory for generated files.
  --samples VAL (as path, required)
                        Sample meta data list
  --data-dir  path(f"{samples:d}")

                        Raw data directory, default to the same directory as
                        sample list
  --job-size 1 (as int)
                        For cluster jobs, number commands to run per job
  --walltime 5h
                        Wall clock time expected
  --mem 16G
                        Memory expected
  --numThreads 8 (as int)
                        Number of threads
  --container ''
                        Software container option

Sections
  leafcutter_1:
    Workflow Options:
      --anchor-len 8 (as int)
      --min-intron-len 50 (as int)
      --max-intron-len 500000 (as int)
      --strandness 1 (as int)
  leafcutter_2:
    Workflow Options:
      --min-split-reads 50 (as int)
      --max-intron-len 500000 (as int)
  leafcutter_3:
    Workflow Options:
      --npc 10 (as int)
      --chr-blacklist  f'{samples:d}/black_list.txt'

  leafcutter_4:

## Setup and global parameters_____no_output_____

[global]
# The output directory for generated files.
parameter: cwd = path("output")
# Sample meta data list
parameter: samples = path
# Raw data directory, default to the same directory as sample list
parameter: data_dir = path(f"{samples:d}")
# For cluster jobs, number commands to run per job
parameter: job_size = 1
# Wall clock time expected
parameter: walltime = "5h"
# Memory expected
parameter: mem = "16G"
# Number of threads
parameter: numThreads = 8
# Software container option
parameter: container = ""
from sos.utils import expand_size
cwd = path(f'{cwd:a}')

def get_samples(fn, dr):
    import os
    samples = [x.strip().split() for x in open(fn).readlines()]
    names = []
    strandness = []
    files = []
    bams = []

    for i, x in enumerate(samples):
        if len(x)<3:
            raise ValueError(f"Line {i+1} of file {fn} must have 3 columns")
        names.append(x[0])
        strandness.append(x[1])
        files.append(x[2])

    for j in range(len(strandness)):
        # for regtools command usage, replace 0 = unstranded/XS, 1 = first-strand/RF, 2 = second-strand/FR
        if strandness[i] == 'rf':
            strandness[i] = 1
        if strandness[i] == 'fr':
            strandness[i] = 2
        if strandness[i] == 'unstranded':
            strandness[i] = 0

    for y in files:
        y = os.path.join(dr, y)
        if not os.path.isfile(y):
            raise ValueError(f"File {y} does not exist")
        bams.append(y)

    if len(files) != len(set(files)):
        raise ValueError("Duplicated files are found (but should not be allowed) in BAM file list")

    return names, bams, strandness

sample_id, bam, strandness = get_samples(samples, data_dir)_____no_output_____

## `leafcutter`

Documentation: [`leafcutter`](https://davidaknowles.github.io/leafcutter/index.html). The choices of regtool parameters are [discussed here](https://github.com/davidaknowles/leafcutter/issues/127).

### Parameter Annotations

* anchor_len: anchor length
* min_intron_len: minimum intron length to be analyzed
* max_intron_len: maximum intron length to be analyzed
* min_split_reads: minimal split reads allows to form a cluster

### Things to keep in mind

* If .bam.bai index files of the .bam input are ready before using leafCutter, it can be placed in the same directory with input .bam files and the "samtools index ${_input}" line can be skipped.
_____no_output_____

[leafcutter_1]
parameter: anchor_len = 8
parameter: min_intron_len = 50
parameter: max_intron_len = 500000
input: bam, group_by = 1, group_with = "strandness"
output: f'{cwd}/{_input:bn}.junc'
task: trunk_workers = 1, trunk_size = job_size, walltime = walltime, mem = mem, cores = numThreads
bash: expand= "${ }", stderr = f'{_output:n}.stderr', stdout = f'{_output:n}.stdout', container=container
    samtools index ${_input}
    regtools junctions extract -a ${anchor_len} -m ${min_intron_len} -M ${max_intron_len} -s ${_strandness} ${_input} -o ${_output}_____no_output_____[leafcutter_2]
parameter: min_split_reads = 50
parameter: max_intron_len = 500000
input: group_by = 'all'
output: f'{cwd}/{samples:bn}_intron_usage_perind.counts.gz'
task: trunk_workers = 1, trunk_size = job_size, walltime = walltime, mem = mem, cores = numThreads
bash: expand= "${ }", stderr = f'{_output[0]:n}.stderr', stdout = f'{_output[0]:n}.stdout', container=container
    rm -f ${_output:nn}.junc
    for i in ${_input:r}; do
    echo $i >> ${_output:nn}.junc ; done
    python /opt/leafcutter/clustering/leafcutter_cluster_regtools.py -j ${_output:nn}.junc -o ${f'{_output:bnn}'.replace("_perind","")} -m ${min_split_reads} -l ${max_intron_len} -r ${cwd}_____no_output_____

## `psichomics`

Documentation: [`psichomics`](http://bioconductor.org/packages/release/bioc/html/psichomics.html)_____no_output_____


# Notebook from mtreinish/qiskit-tutorial
Path: qiskit/aqua/chemistry/dissociation_profile_of_molecule.ipynb

 Trusted Notebook" width="500 px" align="left">_____no_output_____# _*Qiskit Aqua Chemistry: Experimenting with molecular ground state energy with variational quantum eigensolver*_

The latest version of this notebook is available on https://github.com/qiskit/qiskit-tutorial.

***
### Contributors
Antonio Mezzacapo[1], Richard Chen[1], Marco Pistoia[1], Shaohan Hu[1], Peng Liu[1], Stephen Wood[1], Jay Gambetta[1]
### Affiliation
- [1]IBMQ_____no_output_____### Introduction

One of the most compelling possibilities of quantum computation is the the simulation of other quantum systems. Quantum simulation of quantum systems encompasses a wide range of tasks, including most significantly:

1. Simulation of the time evolution of quantum systems.

2. Computation of ground state properties.

These applications are especially useful when considering systems of interacting fermions, such as molecules and strongly correlated materials. The computation of ground state properties of fermionic systems is the starting point for mapping out the phase diagram of condensed matter Hamiltonians. It also gives access to the key question of electronic structure problems in quantum chemistry - namely, reaction rates. The focus of this notebook is on molecular systems, which are considered to be the ideal bench test for early-stage quantum computers, due to their relevance in chemical applications despite relatively modest sizes. Formally, the ground state problem asks the following:

For some physical Hamiltonian *H*, find the smallest eigenvalue $E_G$, such that $H|\psi_G\rangle=E_G|\psi_G\rangle$, where $|\Psi_G\rangle$ is the eigenvector corresponding to $E_G$.

It is known that in general this problem is intractable, even on a quantum computer. This means that we cannot expect an efficient quantum algorithm that prepares the ground state of general local Hamiltonians. Despite this limitation, for specific Hamiltonians of interest it might be possible, given physical constraints on the interactions, to solve the above problem efficiently. Currently, at least four different methods exist to approach this problem:

1. Quantum phase estimation: Assuming that we can approximately prepare the state $|\psi_G\rangle$, this routine uses controlled implementations of the Hamiltonian to find its smallest eigenvalue.
2. Adiabatic theorem of quantum mechanics: The quantum system is adiabatically dragged from being the ground state of a trivial Hamiltonian to the one of the target problem, via slow modulation of the Hamiltonian terms.
3. Dissipative (non-unitary) quantum operation: The ground state of the target system is a fixed point. The non-trivial assumption here is the implementation of the dissipation map on quantum hardware.
4. Variational quantum eigensolvers: Here we assume that the ground state can be represented by a parameterization containing a relatively small number of parameters.

In this notebook we focus on the last method, as this is most likely the simplest to be realized on near-term devices.

The general idea is to define a parameterization $|\psi(\boldsymbol\theta)\rangle$ of quantum states, and minimize the energy

$$E(\boldsymbol\theta) = \langle \psi(\boldsymbol\theta)| H |\psi(\boldsymbol\theta)\rangle,$$

The key ansatz is that the number of parameters $|\boldsymbol\theta^*|$ that minimizes the energy function scales polynomially with the size (e.g., number of qubits) of the target problem.

Then, any local fermionic Hamiltonian can be mapped into a sum over Pauli operators $P_i$,

$$H\rightarrow H_P = \sum_i^M w_i P_i,$$

and the energy corresponding to the state $|\psi(\boldsymbol\theta\rangle$, $E(\boldsymbol\theta)$, can be estimated by sampling the individual Pauli terms $P_i$ (or sets of them that can be measured at the same time) on a quantum computer:

$$E(\boldsymbol\theta) = \sum_i^M w_i \langle \psi(\boldsymbol\theta)| P_i |\psi(\boldsymbol\theta)\rangle.$$

Last, some optimization technique must be devised in order to find the optimal value of parameters $\boldsymbol\theta^*$, such that $|\psi(\boldsymbol\theta^*)\rangle\equiv|\psi_G\rangle$.

### Fermionic Hamiltonians

The Hamiltonians describing systems of interacting fermions can be expressed in second quantization language, considering fermionic creation (annihilation) operators $a^\dagger_\alpha(a_\alpha)$, relative to the $\alpha$-th fermionic mode. In the case of molecules, the $\alpha$ labels stand for the different atomic or molecular orbitals. Within the second-quantization framework, a generic molecular Hamiltonian with $M$ orbitals can be written as
$$H =H_1+H_2=\sum_{\alpha, \beta=0}^{M-1} t_{\alpha \beta} \, a^\dagger_{\alpha} a_{\beta} +\frac{1}{2}  \sum_{\alpha, \beta, \gamma, \delta = 0}^{M-1} u_{\alpha \beta \gamma \delta}\, a^\dagger_{\alpha} a^\dagger_{\gamma} a_{\delta} a_{\beta},$$
with the one-body terms representing the kinetic energy of the electrons and the potential energy that they experience in the presence of the nuclei,
$$ t_{\alpha\beta}=\int d\boldsymbol x_1\Psi_\alpha(\boldsymbol{x}_1) \left(-\frac{\boldsymbol\nabla_1^2}{2}+\sum_{i} \frac{Z_i}{|\boldsymbol{r}_{1i}|}\right)\Psi_\beta (\boldsymbol{x}_1),$$
and their interactions via Coulomb forces
$$ u_{\alpha\beta\gamma\delta}=\int\int d \boldsymbol{x}_1 d \boldsymbol{x}_2 \Psi_\alpha^*(\boldsymbol{x}_1)\Psi_\beta(\boldsymbol{x}_1)\frac{1}{|\boldsymbol{r}_{12}|}\Psi_\gamma^*(\boldsymbol{x}_2)\Psi_\delta(\boldsymbol{x}_2),$$
where we have defined the nuclei charges $Z_i$, the nuclei-electron and electron-electron separations $\boldsymbol{r}_{1i}$ and $\boldsymbol{r}_{12}$, the $\alpha$-th orbital wavefunction $\Psi_\alpha(\boldsymbol{x}_1)$, and we have assumed that the spin is conserved in the spin-orbital indices $\alpha,\beta$ and $\alpha,\beta,\gamma,\delta$.

### Molecules considered in this notebook and mapping to qubits

We consider in this notebook the optimization of two potential energy surfaces, for the hydrogen and lithium hydride molecules, obtained using the STO-3G basis. The molecular Hamiltonians are computed as a function of their interatomic distance, then mapped to two-(H$_2$) and four-(LiH$_2$) qubit problems, via elimination of core and high-energy orbitals and removal of $Z_2$ symmetries.

### Approximate universal quantum computing for quantum chemistry problems

In order to find the optimal parameters $\boldsymbol\theta^*$, we set up a closed optimization loop with a quantum computer, based on some stochastic optimization routine. Our choice for the variational ansatz is a deformation of the one used for the optimization of classical combinatorial problems, with the inclusion of $Z$ rotation together with the $Y$ ones. The optimization algorithm for fermionic Hamiltonians is similar to the one for combinatorial problems, and can be summarized as follows:

1. Map the fermionic Hamiltonian $H$ to a qubit Hamiltonian $H_P$.
2. Choose the maximum depth of the quantum circuit (this could be done adaptively).
3. Choose a set of controls $\boldsymbol\theta$ and make a trial function $|\psi(\boldsymbol\theta)\rangle$. The difference with the combinatorial problems is the insertion of additional parameterized $Z$ single-qubit rotations.
4. Evaluate the energy $E(\boldsymbol\theta) = \langle\psi(\boldsymbol\theta)~|H_P|~\psi(\boldsymbol\theta)\rangle$ by sampling each Pauli term individually, or sets of Pauli terms that can be measured in the same tensor product basis.
5. Use a classical optimizer to choose a new set of controls.
6. Continue until the energy has converged, hopefully close to the real solution $\boldsymbol\theta^*$, and return the last value of $E(\boldsymbol\theta)$.

Note that, as opposed to the classical case, in the case of a quantum chemistry Hamiltonian one has to sample over non-computational states that are superpositions, and therefore take advantage of using a quantum computer in the sampling part of the algorithm. Motivated by the quantum nature of the answer, we also define a variational trial ansatz in this way:

$$|\psi(\boldsymbol\theta)\rangle  = [U_\mathrm{single}(\boldsymbol\theta) U_\mathrm{entangler}]^m |+\rangle$$

where $U_\mathrm{entangler}$ is a collection of cPhase gates (fully entangling gates), $U_\mathrm{single}(\boldsymbol\theta) = \prod_{i=1}^n Y(\theta_{i})Z(\theta_{n+i})$ are single-qubit $Y$ and $Z$ rotation, $n$ is the number of qubits and $m$ is the depth of the quantum circuit.

References and additional details:

[1] A. Kandala, A. Mezzacapo, K. Temme, M. Takita, M. Brink, J. M. Chow, and J. M. Gambetta, *Hardware-efficient Variational Quantum Eigensolver for Small Molecules and Quantum Magnets*, Nature 549, 242 (2017), and references therein.
_____no_output_____

# useful additional packages
import matplotlib.pyplot as plt
%matplotlib inline
import numpy as np

from qiskit_aqua_chemistry import AquaChemistry
import warnings
warnings.filterwarnings('ignore')

# setup aqua_chemistry logging
import logging
from qiskit_aqua_chemistry._logging import set_logging_config, build_logging_config
# set_logging_config(build_logging_config(logging.DEBUG))  # choose INFO, DEBUG to see the log_____no_output_____

### [Optional] Setup token to run the experiment on a real device
If you would like to run the experiement on a real device, you need to setup your account first.

Note: If you do not store your token yet, use `IBMQ.save_accounts()` to store it first._____no_output_____

from qiskit import IBMQ
IBMQ.load_accounts()_____no_output_____

## Optimization of H$_2$ at bond length

In this first part of the notebook we show the optimization of the H$_2$ Hamiltonian in the STO-3G basis at the bond length of 0.735 Angstrom. After mapping it to a four-qubit system with a parity transformation, two spin-parity symmetries are modded out, leading to a two-qubit Hamiltonian. The energy of the mapped Hamiltonian obtained is then minimized using the variational ansatz described in the introduction, and a stochastic perturbation simultaneous approximation (SPSA) gradient descent method. We stored the precomputed one- and two-body integrals and other molecular information in the hdf5 file._____no_output_____

# First, we use classical eigendecomposition to get ground state energy (including nuclear repulsion energy) as reference.
aqua_chemistry_dict = {
    'driver': {'name': 'HDF5'},
    'HDF5': {'hdf5_input': 'H2/H2_equilibrium_0.735_sto-3g.hdf5'},
    'operator': {'name':'hamiltonian',
                 'qubit_mapping': 'parity',
                 'two_qubit_reduction': True},
    'algorithm': {'name': 'ExactEigensolver'}
}
solver = AquaChemistry()
result = solver.run(aqua_chemistry_dict)
print('Ground state energy (classical): {:.12f}'.format(result['energy']))

# Second, we use variational quantum eigensolver (VQE)
aqua_chemistry_dict['algorithm']['name'] = 'VQE'
aqua_chemistry_dict['optimizer'] = {'name': 'SPSA', 'max_trials': 350}
aqua_chemistry_dict['variational_form'] = {'name': 'RYRZ', 'depth': 3, 'entanglement':'full'}
aqua_chemistry_dict['backend'] = {'name': 'statevector_simulator'}

solver = AquaChemistry()
result = solver.run(aqua_chemistry_dict)
print('Ground state energy (quantum)  : {:.12f}'.format(result['energy']))
print("====================================================")
# You can also print out other info in the field 'printable'
for line in result['printable']:
    print(line)Ground state energy (classical): -1.137306035753
Ground state energy (quantum)  : -1.137160612416
====================================================
=== GROUND STATE ENERGY ===

* Electronic ground state energy (Hartree): -1.857129603695
  - computed part:      -1.857129603695
  - frozen energy part: 0.0
  - particle hole part: 0.0
~ Nuclear repulsion energy (Hartree): 0.719968991279
> Total ground state energy (Hartree): -1.137160612416
  Measured:: Num particles: 2.000, S: 0.000, M: 0.00000

=== DIPOLE MOMENT ===

* Electronic dipole moment (a.u.): [0.0  0.0  -0.01520674]
  - computed part:      [0.0  0.0  -0.01520674]
  - frozen energy part: [0.0  0.0  0.0]
  - particle hole part: [0.0  0.0  0.0]
~ Nuclear dipole moment (a.u.): [0.0  0.0  0.0]
> Dipole moment (a.u.): [0.0  0.0  -0.01520674]  Total: 0.01520674
               (debye): [0.0  0.0  -0.03865167]  Total: 0.03865167

## Optimizing the potential energy surface

The optimization considered previously is now performed for two molecules, H$_2$ and LiH, for different interatomic distances, and the corresponding nuclei Coulomb repulsion is added in order to obtain a potential energy surface. _____no_output_____

# select H2 or LiH to experiment with
molecule='H2'

aqua_chemistry_dict = {
    'driver': {'name': 'HDF5'},
    'HDF5': {'hdf5_input': ''},
    'operator': {'name':'hamiltonian',
                 'qubit_mapping': 'parity',
                 'two_qubit_reduction': True},
    'algorithm': {'name': ''},
    'optimizer': {'name': 'SPSA', 'max_trials': 350},
    'variational_form': {'name': 'RYRZ', 'depth': 3, 'entanglement':'full'}
}

# choose which backend want to use
# backend = {'name': 'statevector_simulator'}
backend = {'name': 'qasm_simulator', 'shots': 1024}
algos = ['ExactEigensolver', 'VQE']

if molecule == 'LiH':
    mol_distances = np.arange(0.6, 5.1, 0.1)
    aqua_chemistry_dict['operator']['freeze_core'] = True
    aqua_chemistry_dict['operator']['orbital_reduction'] = [-3, -2]
    aqua_chemistry_dict['optimizer']['max_trials'] = 2500
    aqua_chemistry_dict['variational_form']['depth'] = 5
else:
    mol_distances = np.arange(0.2, 4.1, 0.1)

energy = np.zeros((len(algos), len(mol_distances)))

for j, algo in enumerate(algos):
    aqua_chemistry_dict['algorithm']['name'] = algo
    if algo == 'ExactEigensolver':
        aqua_chemistry_dict.pop('backend', None)
    elif algo == 'VQE':
        aqua_chemistry_dict['backend'] = backend
    print("Using {}".format(algo))
    for i, dis in enumerate(mol_distances):
        print("Processing atomic distance: {:1.1f} Angstrom".format(dis), end='\r')
        aqua_chemistry_dict['HDF5']['hdf5_input'] = "{}/{:1.1f}_sto-3g.hdf5".format(molecule, dis)
        result = solver.run(aqua_chemistry_dict)
        energy[j][i] = result['energy']
    print("\n")Using ExactEigensolver
Processing atomic distance: 4.0 Angstrom

Using VQE
Processing atomic distance: 4.0 Angstrom

for i, algo in enumerate(algos):
    plt.plot(mol_distances, energy[i], label=algo)
plt.xlabel('Atomic distance (Angstrom)')
plt.ylabel('Energy')
plt.legend()
plt.show()_____no_output_____


# Notebook from bwlang/SARS-CoV-2
Path: genomics/4-Variation/ADAR activity.ipynb

# ADAR activity_____no_output_____The ADAR enzyme deaminate adenosine residues to inosine in double stranded RNA sequences. This has been implicated both as an antiviral defense mechanism (it may destroy viral genomes / transcripts) and as a means to block the RNAi machinery (mismatches may not be efficiently processed). In Illumina sequencing the inosine is read out as G, leaving behind a detectable A to G transition. Do SARS-CoV-2 datasets show any of these signs ? _____no_output_____

<code>
import os

import matplotlib.pyplot as plt
import pandas as pd
import seaborn as sns_____no_output_____variants = pd.read_csv('variant_list.05.tsv', sep='\t')
variants_no_filter = pd.read_csv('variant_list.tsv', sep='\t')_____no_output_____def plot_transitions(variants):
    interesting_samples = variants.Sample.value_counts() >= 20
    interesting_samples = interesting_samples.index[interesting_samples]
    filtered_df = variants[variants.Sample.isin(interesting_samples)]
    s = filtered_df.groupby(['Sample', 'REF', 'ALT']).count()['CHROM']
    df = s.reset_index()
    df['transition'] = df.REF + '>' + df.ALT
    fig, axes = plt.subplots(ncols=1, nrows=24, figsize=(5, 40))
    for sample, ax in zip(df.Sample.unique(), axes):
        subset = df[df.Sample == sample]
        subset.plot(kind='bar', x='transition', y='CHROM', legend=False, title=sample, ax=ax)
    plt.tight_layout()
    return fig_____no_output_____plot_transitions(variants).savefig('NT transition 0.05 AF filter.pdf', bbox_inches='tight')/Users/mvandenb/src/galaxy/.venv/lib/python3.6/site-packages/pandas/plotting/_matplotlib/tools.py:298: MatplotlibDeprecationWarning:
The rowNum attribute was deprecated in Matplotlib 3.2 and will be removed two minor releases later. Use ax.get_subplotspec().rowspan.start instead.
  layout[ax.rowNum, ax.colNum] = ax.get_visible()
/Users/mvandenb/src/galaxy/.venv/lib/python3.6/site-packages/pandas/plotting/_matplotlib/tools.py:298: MatplotlibDeprecationWarning:
The colNum attribute was deprecated in Matplotlib 3.2 and will be removed two minor releases later. Use ax.get_subplotspec().colspan.start instead.
  layout[ax.rowNum, ax.colNum] = ax.get_visible()
/Users/mvandenb/src/galaxy/.venv/lib/python3.6/site-packages/pandas/plotting/_matplotlib/tools.py:304: MatplotlibDeprecationWarning:
The rowNum attribute was deprecated in Matplotlib 3.2 and will be removed two minor releases later. Use ax.get_subplotspec().rowspan.start instead.
  if not layout[ax.rowNum + 1, ax.colNum]:
/Users/mvandenb/src/galaxy/.venv/lib/python3.6/site-packages/pandas/plotting/_matplotlib/tools.py:304: MatplotlibDeprecationWarning:
The colNum attribute was deprecated in Matplotlib 3.2 and will be removed two minor releases later. Use ax.get_subplotspec().colspan.start instead.
  if not layout[ax.rowNum + 1, ax.colNum]:
plot_transitions(variants_no_filter).savefig('NT transition no filter.pdf', bbox_inches='tight')/Users/mvandenb/src/galaxy/.venv/lib/python3.6/site-packages/pandas/plotting/_matplotlib/tools.py:298: MatplotlibDeprecationWarning:
The rowNum attribute was deprecated in Matplotlib 3.2 and will be removed two minor releases later. Use ax.get_subplotspec().rowspan.start instead.
  layout[ax.rowNum, ax.colNum] = ax.get_visible()
/Users/mvandenb/src/galaxy/.venv/lib/python3.6/site-packages/pandas/plotting/_matplotlib/tools.py:298: MatplotlibDeprecationWarning:
The colNum attribute was deprecated in Matplotlib 3.2 and will be removed two minor releases later. Use ax.get_subplotspec().colspan.start instead.
  layout[ax.rowNum, ax.colNum] = ax.get_visible()
/Users/mvandenb/src/galaxy/.venv/lib/python3.6/site-packages/pandas/plotting/_matplotlib/tools.py:304: MatplotlibDeprecationWarning:
The rowNum attribute was deprecated in Matplotlib 3.2 and will be removed two minor releases later. Use ax.get_subplotspec().rowspan.start instead.
  if not layout[ax.rowNum + 1, ax.colNum]:
/Users/mvandenb/src/galaxy/.venv/lib/python3.6/site-packages/pandas/plotting/_matplotlib/tools.py:304: MatplotlibDeprecationWarning:
The colNum attribute was deprecated in Matplotlib 3.2 and will be removed two minor releases later. Use ax.get_subplotspec().colspan.start instead.
  if not layout[ax.rowNum + 1, ax.colNum]:

</code>

# For some of the samples this really looks like ADAR editing, especially for lower AF variants_____no_output_____