# Notebook from krassowski/meningitis-integration
Path: analyses/rnaseq_vs_clinical/_Detailed_DESeq2_-_limma_comparison.ipynb

from helpers.utilities import *
%run helpers/notebook_setup.ipynb_____no_output_____%R source('plots/colors.R');_____no_output_____%store -r trend_tmm
%store -r trend_rle
%store -r voom_rle
%store -r voom_tmm
%store -r reference_definite_tbm_cm_deg_____no_output_____

### Quantification of the similarities and differences_____no_output_____Important reading for filtering limma results using genefilter:
   - https://stat.ethz.ch/pipermail/bioconductor/2012-March/043895.html
   > Clearly filtering should be done before topTable(), because there is no
point in conducting multiple testing adjustments over probes that are to
be filtered.

   > Whether to filter before for after eBayes() is not so clear.  I would
    suggest filtering before, assuming that the probes after filtering are
    still representative of the whole genome.

   > It makes no difference whether filtering is done before or after lmFit().

   - https://stat.ethz.ch/pipermail/bioconductor/2009-October/030062.html

   > You are correct that including a bunch of unexpressed genes when
adjusting for multiplicity will reduce power. However, with limma you
don't want to remove the 'unexpressed' genes too early (this doesn't
apply to 'bad' data, where the spots are demonstrably unreliable for
some reason or another).

   > You have to remember that the eBayes() step adjusts the denominator of
the t-statistic based on a prior variance estimate that is calculated
from all the genes under consideration. If you filter out genes prior to
this step you can bias this estimate.

   > So the recommended method is to perhaps remove demonstrably bad spots
first, do the normalization, model fitting, etc, and then filter out
those genes you consider unexpressed before doing the multiplicity
adjustment.

  - and ofcourse filtering by variance is not advised and wrong for limma: https://support.bioconductor.org/p/59747/_____no_output_____

from scipy.stats import spearmanr_____no_output_____def compare_results(a, b, key=None, key_a=None, key_b=None, function=spearmanr, from_quantile=0, up_to_quantile=1, drop_na=True):
    if key:
        key_a = key
        key_b = key
    if key_a:
        a = a[key_a]
    if key_b:
        b = b[key_b]

    a = a.sort_values()
    b = b.sort_values()

    a = a.iloc[int(from_quantile * len(a)):int(up_to_quantile * len(a))]
    b = b.iloc[int(from_quantile * len(b)):int(up_to_quantile * len(b))]

    if drop_na:
        a = a.dropna()
        b = b.dropna()

    intersection = list(set(a.index) & set(b.index))

    a_values = a.loc[intersection]
    b_values = b.loc[intersection]
    return function(a_values, b_values)

def get_rank_from_ordered(x):
    return x.reset_index().index

compare_results(trend_tmm, trend_rle, key='P.Value')_____no_output_____# should be close to 1
compare_results(trend_tmm, trend_rle, key='logFC').correlation_____no_output_____deseq = reference_definite_tbm_cm_deg.set_index('ensembl_id')_____no_output_____# should be close to 1 as well...
compare_results(trend_tmm, deseq, key_a='logFC', key_b='log2FoldChange').correlation_____no_output_____

This one should actually be greater than for TMM..._____no_output_____

compare_results(trend_rle, deseq, key_a='logFC', key_b='log2FoldChange').correlation_____no_output_____# this wil be interesting
compare_results(trend_tmm, deseq, key_a='P.Value', key_b='pvalue').correlation_____no_output_____

#### Let's dive into rank comparison

I have claimed that the results are comparable when it comes to ordering. Here is the evidence:_____no_output_____Lemma 1: I can use pvalue instead of padj for the non-filtered subset of DESeq2 results:_____no_output_____

compare_results(deseq, deseq, key_a='pvalue', key_b='padj', drop_na=True).correlation_____no_output_____

And for limma:_____no_output_____

compare_results(trend_tmm, trend_tmm, key_a='P.Value', key_b='adj.P.Val').correlation_____no_output_____def calc_rank_correlations(a, b, start=0, step=0.01, end=1, function=spearmanr, window=None, window_start_edge='left', trim=False):
    """
    Use first n (up to given quantile) if no window size ('window') is given, otherwise compute rolling correlation wisth sliding window of given size

    Args:
        window_start_edge: 'left' to set the first window as (start, start + window_size),
                           'right' to set the first window as (start - window_size, start)
        trim: if using window, will trim the results so that no results for partial (not fully covering requested size) windows are shown
    """

    compare_ab = partial(compare_results, a, b, function=function)

    calc_quantile_ranges = (
        (

            (lambda q: dict(from_quantile=q, up_to_quantile=q + window))
            if window_start_edge == 'left' else
            (lambda q: dict(from_quantile=max(start, q - window), up_to_quantile=q))
        )
        if window else
        (lambda q: dict(from_quantile=0, up_to_quantile=q))
    )

    mode = 'sliding_window' if window else 'up_to_quantile'
    if window:
        mode += '_start' if window_start_edge == 'left' else '_end'

    if not window:
        start = start + step

    quantiles = np.linspace(start, end, int((end-start)/step))

    return DataFrame([
        {
            'range': '{from_quantile:.2f}:{up_to_quantile:.2f}'.format(**ranges),
            **compare_ab(**ranges)._asdict(),
            **ranges
        }
        for ranges in map(calc_quantile_ranges, quantiles)
        if (
            not trim or
            (window_start_edge == 'left' and ranges['from_quantile'] < end - window) or
            (window_start_edge == 'right' and ranges['up_to_quantile'] > start + window)
        )
    ])_____no_output_____def compare_cumulative_and_sliding(a, b, step=0.001, window=0.1):
    cumulative = calc_rank_correlations(a, b, step=step)
    sliding = calc_rank_correlations(a, b, step=step, window=window, trim=T)

    return concat([
        cumulative.assign(mode='Cumulative - up to n-th quantile'),
        sliding.assign(mode=f'Sliding window of {window}, ending at n-th qunatile')
    ])_____no_output_____fdr_qrc_between_normalizations = compare_cumulative_and_sliding(trend_rle['adj.P.Val'], trend_tmm['adj.P.Val'])_____no_output_____%%R -i fdr_qrc_between_normalizations -w 800 -h 600
(
    ggplot(fdr_qrc_between_normalizations, aes(x=up_to_quantile, y=correlation, color=pvalue))
    + facet_wrap(~mode, ncol=1, scale='free_y')
    + geom_point() + nice_theme + ggtitle('Spearman correlations between FDR adjusted p-values of Limma with TMM vs RLE normalization')
)_____no_output_____fdr_qrc_limma_rle_vs_deseq = compare_cumulative_and_sliding(trend_rle['adj.P.Val'], deseq['padj'])_____no_output_____%%R -i fdr_qrc_limma_rle_vs_deseq -w 800 -h 600
(
    ggplot(fdr_qrc_limma_rle_vs_deseq, aes(x=up_to_quantile, y=correlation, color=pvalue))
    + facet_wrap(~mode, ncol=1, scale='free_y')
    + geom_point() + nice_theme + ggtitle('Spearman correlations between FDR adjusted p-values of Limma-voom (RLE) vs DESeq2')
)_____no_output_____

The cut in half is the effect of the independent filtering._____no_output_____

pval_qrc_limma_rle_vs_deseq = compare_cumulative_and_sliding(trend_rle['P.Value'], deseq['pvalue'])_____no_output_____%%R -i pval_qrc_limma_rle_vs_deseq -w 800 -h 600
(
    ggplot(pval_qrc_limma_rle_vs_deseq, aes(x=up_to_quantile, y=correlation, color=pvalue))
    + facet_wrap(~mode, ncol=1, scale='free_y')
    + geom_point() + nice_theme + ggtitle('Spearman correlations between p-values of Limma-voom (RLE) vs DESeq2')
)_____no_output_____compare_results(trend_rle['P.Value'], deseq['pvalue'], up_to_quantile=0.01).correlation_____no_output_____compare_results(trend_rle['P.Value'], deseq['pvalue'], up_to_quantile=0.05).correlation_____no_output_____compare_results(trend_rle['P.Value'], deseq['pvalue'], up_to_quantile=0.1).correlation_____no_output_____

- the very first few genes have low correlations, but
- The top 0.01 quantile agrees at a level of r=0.7
- The top 0.05 quantile agrees at a level of r=0.6
- The top 0.1 quantile agrees at a level of r=0.4
- overall the agreement is at a Spearman correlation of 0.5_____no_output_____## What about the DESeq2 with variance stabilization?_____no_output_____as suggested here: https://support.bioconductor.org/p/52228/_____no_output_____### Sanity check: reproducibility of the preliminary DESeq2 results_____no_output_____

%store -r by_condition_and_tb_subset
%store -r reference_samples
%store -r raw_counts_trimmed_____no_output_____%%R -i reference_samples -i raw_counts_trimmed -i by_condition_and_tb_subset
# raw_counts_matrix_no_outliers, by_condition_and_tb_subset_no_outliers
conditions_for_deseq = by_condition_and_tb_subset[reference_samples, drop=F]

colnames(raw_counts_trimmed) = colnames(by_condition_and_tb_subset)
#print(by_condition_and_tb_subset)

dds <- DESeq2::DESeqDataSetFromMatrix(
    countData = round(raw_counts_trimmed),
    colData = as.data.frame(conditions_for_deseq),
    design = ~ conditions_for_deseq
)

keep <- rowSums(DESeq2::counts(dds)) >= 10
dds <- dds[keep,]

dds <- DESeq2::DESeq(dds)_____no_output_____%%R -o my_deseq2
my_deseq2 = as.data.frame(
    DESeq2::results(dds, contrast=c('conditions_for_deseq', 'Definite tuberculosis', 'Cryptococcal'))
)_____no_output_____%%R
DESeq2::plotDispEsts(dds)_____no_output_____my_deseq2 = my_deseq2.sort_values('padj')_____no_output_____assert not set(deseq.index) - set(my_deseq2.index)_____no_output_____

There are many more genes in my results:_____no_output_____

my_deseq2.loc[deseq.index]_____no_output_____

But overall, the p-values (for these genes that overlap) are highly correlated:_____no_output_____

compare_results(deseq['pvalue'], my_deseq2['pvalue'])_____no_output_____pval_qrc_deseq_reproducibility = compare_cumulative_and_sliding(deseq['pvalue'], my_deseq2['pvalue'])_____no_output_____%%R -i pval_qrc_deseq_reproducibility -w 800 -h 600
(
    ggplot(pval_qrc_limma_rle_vs_deseq, aes(x=up_to_quantile, y=correlation, color=pvalue))
    + facet_wrap(~mode, ncol=1, scale='free_y')
    + geom_point() + nice_theme + ggtitle('Spearman correlations between p-values of DESeq2 (preliminary) and DESeq2 (mine)')
)_____no_output_____

### The variance stabilization with log-transformation_____no_output_____

%%R
# raw_counts_matrix_no_outliers, by_condition_and_tb_subset_no_outliers
conditions_for_deseq = by_condition_and_tb_subset[reference_samples, drop=F]

dds <- DESeq2::DESeqDataSetFromMatrix(
    countData = round(raw_counts_trimmed),
    colData = as.data.frame(conditions_for_deseq),
    design = ~ conditions_for_deseq
)

keep <- rowSums(DESeq2::counts(dds)) >= 10
dds <- dds[keep,]
mean(keep)[1] 1
%%R
dds$conditions_for_deseq <- relevel(factor(dds$conditions_for_deseq), ref = "Cryptococcal")_____no_output_____%%R
dds <- DESeq2::DESeq(dds)_____no_output_____%%R -o my_deseq2
my_deseq2 = as.data.frame(
    DESeq2::results(dds, contrast=c('conditions_for_deseq', 'Definite tuberculosis', 'Cryptococcal'))
)_____no_output_____

Reproducibility of p-values is high:_____no_output_____

compare_results(deseq['pvalue'], my_deseq2['pvalue'])_____no_output_____

But the dimensions differ and there are some nulls in my results:_____no_output_____

deseq.shape, my_deseq2.shape_____no_output_____outliers = my_deseq2[my_deseq2.pvalue.isnull()]
print(len(outliers))
outliers.head()227

> If a row contains a sample with an extreme count outlier then the p value and adjusted p value will be set to NA. These outlier counts are detected by Cook’s distance. Customization of this outlier filtering and description of functionality for replacement of outlier counts and refitting is described below
(https://www.bioconductor.org/packages/devel/bioc/vignettes/DESeq2/inst/doc/DESeq2.html#more-information-on-results-columns)_____no_output_____

assert not outliers.baseMean.isnull().any()_____no_output_____assert not deseq.pvalue.isnull().any()_____no_output_____

A few outliers made it into the preliminary results:_____no_output_____

deseq[deseq.index.isin(outliers.index)]_____no_output_____deseq2_without_outliers = my_deseq2[~my_deseq2.pvalue.isnull()]_____no_output_____

### With pre-selected samples?_____no_output_____

%%R
conditions_for_deseq = by_condition_and_tb_subset[reference_samples, drop=F]
conditions_subset = conditions_for_deseq %in% c('Definite tuberculosis', 'Cryptococcal')

conditions_for_deseq = conditions_for_deseq[conditions_subset]

dds <- DESeq2::DESeqDataSetFromMatrix(
    countData = round(raw_counts_trimmed[,conditions_subset]),
    colData = as.data.frame(conditions_for_deseq),
    design = ~ conditions_for_deseq
)

keep <- rowSums(DESeq2::counts(dds)) >= 10
dds <- dds[keep,]
mean(keep)[1] 0.9997931
%%R
dds$conditions_for_deseq <- relevel(factor(dds$conditions_for_deseq), ref = "Cryptococcal")_____no_output_____%%R
dds <- DESeq2::DESeq(dds)_____no_output_____%%R -o deseq_preselected
deseq_preselected = as.data.frame(
    DESeq2::results(dds, contrast=c('conditions_for_deseq', 'Definite tuberculosis', 'Cryptococcal'))
)_____no_output_____compare_results(deseq['pvalue'], deseq_preselected['pvalue']).correlation - compare_results(deseq['pvalue'], my_deseq2['pvalue']).correlation_____no_output_____deseq_preselected.sort_values('pvalue')_____no_output_____deseq_preselected.pvalue_____no_output_____sum(deseq_preselected.pvalue.isnull())_____no_output_____

if (!requireNamespace("BiocManager", quietly = TRUE))
    install.packages("BiocManager")

BiocManager::install("apeglm")_____no_output_____### Effect of shrinkage method_____no_output_____

%%R -o ashr -o normal
definit_tbm_vs_crypto_contrast = c('conditions_for_deseq', 'Definite tuberculosis', 'Cryptococcal')
ashr <- as.data.frame(DESeq2::lfcShrink(dds, contrast=definit_tbm_vs_crypto_contrast, type="ashr"))
normal <- as.data.frame(DESeq2::lfcShrink(dds, contrast=definit_tbm_vs_crypto_contrast, type="normal"))_____no_output_____%%R -o apeglm
apeglm <- as.data.frame(DESeq2::lfcShrink(dds, coef='conditions_for_deseq_Definite.tuberculosis_vs_Cryptococcal', type="apeglm"))_____no_output_____compare_results(deseq['pvalue'], normal['pvalue'])_____no_output_____compare_results(deseq['pvalue'], ashr['pvalue'])_____no_output_____compare_results(deseq['pvalue'], apeglm['pvalue'])_____no_output_____compare_results(trend_rle['P.Value'], ashr['pvalue'])_____no_output_____compare_results(trend_rle['P.Value'], apeglm['pvalue'])_____no_output_____compare_results(trend_rle['P.Value'], deseq['pvalue'])_____no_output_____shrinkages = {'apeglm': apeglm, 'normal': normal, 'ashr': ashr}_____no_output_____

Different shrinkage algorithms change the final log fold change visibly:_____no_output_____

lfc_by_shrinkage = concat([
    table[['log2FoldChange']].assign(shrinkage=name)
    for name, table in shrinkages.items()
]).rename_axis('gene').reset_index()_____no_output_____%%R -i lfc_by_shrinkage -h 800
(
    ggplot(lfc_by_shrinkage, aes(x=shrinkage, y=log2FoldChange, fill=shrinkage, color=shrinkage))
    + nice_theme
    + geom_point(alpha=0.01)
    + geom_line(aes(group=gene), color='grey', alpha=0.01)
    + geom_boxplot(alpha=0.1)
    + scale_y_log10()
)_____no_output_____

But it does not influence the number or composition of genes marked as significant:_____no_output_____

from helpers.venn import venn3_____no_output_____venn3({
    name: set(table.query('padj < 0.05').index)
    for name, table in shrinkages.items()
});_____no_output_____len(deseq.query('padj < 0.05'))_____no_output_____

### Reproducing the adjusted p-value calculation_____no_output_____Works easily for limma, this is simple BH FDR:_____no_output_____

%%R -o fdr_trend_rle -i trend_rle
fdr_trend_rle = p.adjust(trend_rle$P.Value, 'BH')
head(fdr_trend_rle)[1] 0.06850822 0.06850822 0.10902677 0.10902677 0.10902677 0.10902677
assert np.isclose(trend_rle['adj.P.Val'], fdr_trend_rle).all()_____no_output_____

For DESeq2: this is not as easy (as it uses independent filtering):_____no_output_____

%%R -o fdr_deseq2 -o deseq2_order
deseq2_order <- normal[order(normal$pvalue),]
fdr_deseq2 = p.adjust(deseq2_order$pvalue, 'BH')
head(fdr_deseq2)[1] 0.004558102 0.004558102 0.004558102 0.004558102 0.004558102 0.004558102
np.isclose(deseq2_order['padj'], fdr_deseq2).all()/home/krassowski/.pyenv/versions/3.7.2/envs/meningitis/lib/python3.7/site-packages/numpy/core/numeric.py:2373: RuntimeWarning: invalid value encountered in multiply
  x = x * ones_like(cond)

But if we used simple FDR, the number of significant genes would still be much higher in DESeq2:_____no_output_____

%R (sum(fdr_deseq2 < 0.05, na.rm=T))_____no_output_____

Still, the independent filtering must have an important impact on the result! But why I did not see such impact when filtering limma results? Let's try to reproduce from both sides:_____no_output_____

%R import::here(filter_by_mean_expression, .from='helpers/independent_hypothesis.R')_____no_output_____%%R -w 800
par(mfrow=c(1, 2))
filter_by_mean_expression(trend_rle); title('limma-trend RLE')
filter_by_mean_expression(normal, expression_col='baseMean', pvalue_col='pvalue'); title('DESeq2')_____no_output_____

This does not seem to be due to differences in mean expression calculation:_____no_output_____

deseq_vs_limma_mean = compare_cumulative_and_sliding(trend_rle['AveExpr'], my_deseq2['baseMean'])/home/krassowski/.pyenv/versions/3.7.2/envs/meningitis/lib/python3.7/site-packages/numpy/lib/function_base.py:2400: RuntimeWarning: invalid value encountered in true_divide
  c /= stddev[:, None]
/home/krassowski/.pyenv/versions/3.7.2/envs/meningitis/lib/python3.7/site-packages/scipy/stats/_distn_infrastructure.py:879: RuntimeWarning: invalid value encountered in greater
  return (self.a < x) & (x < self.b)
/home/krassowski/.pyenv/versions/3.7.2/envs/meningitis/lib/python3.7/site-packages/scipy/stats/_distn_infrastructure.py:879: RuntimeWarning: invalid value encountered in less
  return (self.a < x) & (x < self.b)
/home/krassowski/.pyenv/versions/3.7.2/envs/meningitis/lib/python3.7/site-packages/scipy/stats/_distn_infrastructure.py:1821: RuntimeWarning: invalid value encountered in less_equal
  cond2 = cond0 & (x <= self.a)
%%R -i deseq_vs_limma_mean -w 800 -h 600
(
    ggplot(deseq_vs_limma_mean, aes(x=up_to_quantile, y=correlation, color=pvalue))
    + facet_wrap(~mode, ncol=1, scale='free_y')
    + geom_point() + nice_theme + ggtitle('Spearman correlations between base/mean expression of Limma-voom (RLE) vs DESeq2')
)_____no_output_____import random
random.seed(0)

trend_rle_random_200 = trend_rle.sample(n=200)

mean_by_method = concat([
    trend_rle_random_200[['AveExpr']].assign(method='limma-trend RLE'),
    my_deseq2[['baseMean']].loc[trend_rle_random_200.index].rename(columns={'baseMean': 'AveExpr'}).assign(method='DESeq2')
])_____no_output_____%%R -i mean_by_method -h 800
(
    ggpubr::ggpaired(
        mean_by_method, x='method', y='AveExpr',
        color='method', line.color='gray', xlab='Method'
    )
    + scale_y_log10()
)_____no_output_____

#### Using the DESeq2-calculated mean for filtering:_____no_output_____

trend_rle['mean_count_expr'] = my_deseq2['baseMean'].loc[trend_rle.index]_____no_output_____subset_trend_rle = trend_rle[
    (~trend_rle.mean_count_expr.isnull())
    &
    trend_rle.index.isin(deseq2_without_outliers.index)
]_____no_output_____%%R -i subset_trend_rle
filter_by_mean_expression(subset_trend_rle, expression_col='mean_count_expr')_____no_output_____

Changes the result slightly (one DE gene more)._____no_output_____### Comparing raw p-values:_____no_output_____So it's on on the raw p-values again..._____no_output_____

p_value_comparison = concat([
    subset_trend_rle[['P.Value']].rename({'P.Value': 'pvalue'}, axis=1).assign(method='limma-RLE'),
    deseq2_without_outliers.loc[subset_trend_rle.index][['pvalue']].assign(method='DESeq2'),
]).rename_axis('gene').reset_index()_____no_output_____%%R -i p_value_comparison
(
    ggplot(p_value_comparison, aes(x=log10(pvalue), fill=method, color=method))
    + nice_theme
    + geom_density(data=p_value_comparison[p_value_comparison$method=='DESeq2',], alpha=0.2)
    + geom_density(data=p_value_comparison[p_value_comparison$method=='limma-RLE',], alpha=0.2)
    + scale_y_sqrt()
)_____no_output_____%%R -i p_value_comparison -w 800
(
    ggplot(p_value_comparison, aes(x=log10(pvalue), fill=method))
    + geom_histogram()
    + facet_wrap(~ method)
    + scale_y_log10()
    + stat_bin(aes(y=..count.., label=..count..), geom="text", vjust=-.5)
    + nice_theme
)_____no_output_____%%R -i p_value_comparison -w 800

up_to_m4 = p_value_comparison[p_value_comparison$pvalue<0.5*10e-4,]
up_to_m4$method = as.factor(up_to_m4$method)

(
    ggpubr::ggpaired(
        up_to_m4, x='method', y='-pvalue',
        color='method', line.color='gray', xlab='Method'
    )
    + scale_x_discrete(breaks=up_to_m4$method)
)
Error in `$<-.data.frame`(`*tmp*`, "id", value = c(1L, 2L, 3L, 4L, 5L,  :
  replacement has 186 rows, data has 187
Calls: <Anonymous> ... <Anonymous> -> <Anonymous> -> .f -> $<- -> $<-.data.frame

> "In DESeq2 the p-value is from a test that the absolute value of the LFC is greater than 2, while in edgeR the p-value is from a test that the absolute value of the LFC is greater than 0. For comparison can you try" prof Mike Love [at seqanswers](http://seqanswers.com/forums/showthread.php?t=49495)_____no_output_____### Gene sets_____no_output_____Let's compare gene sets:_____no_output_____

%R calc_de_tb_subsets('Definite tuberculosis', 'Cryptococcal', camera=T)
Error in calc_de_tb_subsets("Definite tuberculosis", "Cryptococcal", camera = T) :
  could not find function "calc_de_tb_subsets"
Calls: <Anonymous> -> <Anonymous> -> withVisible
%%R -i reference_definite_tbm_cm_deg
reference_definite_tbm_cm_deg = reference_definite_tbm_cm_deg[complete.cases(enembl_to_entrez[reference_definite_tbm_cm_deg$ensembl_id,]),]
rownames(reference_definite_tbm_cm_deg) = enembl_to_entrez[reference_definite_tbm_cm_deg$ensembl_id,]
Error in complete.cases(enembl_to_entrez[reference_definite_tbm_cm_deg$ensembl_id,  :
  object 'enembl_to_entrez' not found
Calls: <Anonymous> ... <Anonymous> -> withVisible -> [ -> [.data.frame -> complete.cases
%R limma::cameraPR(setNames(reference_definite_tbm_cm_deg$stat, rownames(reference_definite_tbm_cm_deg)), collection)
Error in cameraPR.default(setNames(reference_definite_tbm_cm_deg$stat,  :
  object 'collection' not found
Calls: <Anonymous> ... <Anonymous> -> withVisible -> <Anonymous> -> cameraPR.default
%R limma::cameraPR(setNames(reference_definite_tbm_cm_deg$log2FoldChange, rownames(reference_definite_tbm_cm_deg)), collection)
Error in cameraPR.default(setNames(reference_definite_tbm_cm_deg$log2FoldChange,  :
  object 'collection' not found
Calls: <Anonymous> ... <Anonymous> -> withVisible -> <Anonymous> -> cameraPR.default
top_reference.map(get_gene).map(lambda gene: gene.name)_____no_output_____list(top_limma.map(get_gene).map(lambda gene: gene.name))_____no_output_____%R source('plots/volcano.R');_____no_output_____%%R
c_definite_tb_cm$protein = rownames(c_definite_tb_cm)
volcano_plot(c_definite_tb_cm)_____no_output_____

I do not like this plot... It should not look like that..._____no_output_____- https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4927907/ [Parkinson's] used EdgeR (likely the default - TMM)
- https://www.biorxiv.org/content/biorxiv/early/2018/12/07/490045.full.pdf [pneumococcal meningitis IL-17] used DESeq2 (probably RLE); they have beautiful volcano plots but that's cause they have survivors vs non-survivors; this study is thus free of this problem

okey, I cannot quickly find comparable study with a statistician engaged in the data processing..._____no_output_____Literature:_____no_output_____> Different studies (e.g., [44, 61]) indicated TMM and ‘DESeq’ methods as the most effective approaches for library size normalization. However, if the common assumption that the compared samples contain similar amount of RNA does not hold, count normalization methods are ineffective, and calibration techniques leveraging on spike-in RNA measurements can be used [63]. [(Finotello and Camillo, 2014)](https://academic.oup.com/bfg/article/14/2/130/257370)_____no_output_____Well we don't have synthetic spike-in RNAs, do we?

We probably cannot relay on the housekeping genes neither...

Do we have some genes that must be there? Like genes which are always present in meningitis?_____no_output_____[(Otake et al, 2019)](https://bmcmedgenomics.biomedcentral.com/articles/10.1186/s12920-019-0473-z) used DESeq2 for exosomal RNASeq in CSF (healthy vs amyotrophic lateral sclerosis, ALS) and found some of the same issues:

> Based on the hypothesis that disease conditions in ALS would be different from healthy conditions, we did not exclude the uncommon genes in NH group in the following DESeq2 analysis. In false discovery rate (FDR) correction by Benjamini-Hochberg method, 5006 genes could be calculated for adjusted p-values (data not shown, see Additional file 3). Thus, the genes with no detection in any specimens or with outliers in some specimens were not calculated for adjusted p-values. DESeq2 analysis resulted in 543 genes that were significantly changed between NH donors and ALS patients groups, with adjusted p-values less than 0.05 (Fig. 4a).

> There existed two limitations in this study. One is low statistical power due to small sample size. It was considered as a main reason why so many genes were determined as DEGs. In DESeq2, FDR correction can be applied to the genes without no outlier identified cook’s distance. Small sample size might not be enough to identify true outliers. The other is extremely low abundance of exosomal mRNAs that might cause the large variation of gene expression among four NH specimens

(I disagree with their reasoning on the FDR)

> We reported that more than 4500 genes were commonly detected in exosomes in CSF from NH donors, and also showed that housekeeping genes such as ACTB and GAPDH in the exosomes were also abundant and consistent among four individuals. DESeq2 can be applied to RNA-seq with normalization by logarithm of geometric mean of all genes. Upon further validation in RT-PCR, how the data is normalized should be considered because there is no information in housekeeping genes in exosomes_____no_output_____