# Notebook from zoeydy/CMEESamraat
Path: content/_build/jupyter_execute/notebooks/07-R.ipynb

library(repr) ; options(repr.plot.width=6, repr.plot.height= 6) # Change plot sizes (in cm)_____no_output_____

# Biological Computing in R_____no_output_____## Introduction

The R part of this course aims to lay down the foundations for you to use R for scientific computing in biology by exploiting it full potential as a fully featured object-oriented programming language. Specifically, this chapter aims at teaching you:

* Basic R syntax and programming conventions, assuming you have never set your eyes on R
* Principles of data processing and exploration (including visualization) using R
* Principles of clean and efficient programming using R
* To generate publication quality graphics in R
* To develop reproducible data analysis "work flows" so you (or anybody else) can run and re-run your analyses, graphics outputs and all, in R
* To make R simulations more efficient using vectorization
* To find and fix errors in R code using debugging
* T make data wrangling and analyses more efficient and convenient using custom tools such as `tidyr`
* Some additional tools and topics in R (accessing databases, building your own packages, etc.).

### What is R?

R is a freely available statistical software with strong  programming capabilities widely used by professional scientists around  the world. It was based on the commercial statistical software `S` by Robert Gentleman and Ross Ihaka. The first stable version appeared  in 2000.

R was essentially designed for *programming* statistical  analysis and data-mining. It became the standard tool for data  analysis and visualization in biology in a matter of just 10 years or so. It is also increasingly being used for modelling in biology. This is because R has:

* Many tried and tested packages to perform practically all statistical analysis
* Numerous packages for data-handling and processing
* Excellent graphing and visualization capabilities
* Good capabilities for mathematical calculations, including matrix algebra

### Why R?

There are many commercial statistical (minitab, SPSS, etc) software  packages in the world that are mouse-driven, warm, and friendly, and  have lots of statistical tests and plotting/graphing capabilities. Why not just use them? Here are some very good reasons:

* R is scriptable, so you can build a perfectly repeatable record of your analysis. This in itself has several advantages:
   * You can never replicate *exactly* the same analysis with all the same steps using a point-and-click approach/software. With R you can reproduce your full analysis for yourself (in the future!), your colleagues, your supervisor/employer, and any journal you might want to submit your work to.
   * You may need to rerun your analysis every time you get new data. Once you have it all in a R script, you can just rerun your     analysis and go home!
   * You may need to tweak your analysis many times (new data, supervisor changes mind, you change mind, paper reviewers want you do something differently). Having the analysis recorded as script then allows you to do so by revising the relevant parts of your analysis with relatively little pain.
* R provides basically every statistical test you'll ever need and is constantly being improved –  you can tailor your analyses rather than trying to use the more limited options each statistical software package can offer.
* R can produce publication-quality graphics that can be re-produced with scripts – you won't get [RSI](https://en.wikipedia.org/wiki/Repetitive_strain_injury) mouse-clicking your way though graphing and re-graphing your data every time you change your analysis!
* R is freely available for all common computer operating systems – if you want a copy on your laptop, help yourself  at the [CRAN website](https://cran.r-project.org).
* Being able to program in R means you can develop and automate your own data handling, statistical analysis, and graphing/plotting, a set of skills you are likely to need in many, if not most careers paths!

#### Would you ever need anything other than R?

Being able to program R means you can develop and automate your statistical analyses and the generation of figures into a reproducible work flow. For many of you, using R as your only programming language will do the job. However, if your work also includes extensive numerical simulations, manipulation of very large matrices, bioinformatics, relational database access and manipulation, or web development, you will be better-off *also* knowing another programming language that is more versatile and computationally efficient (like Python, C, or Julia).

### Installing R

If you are using a college computer, R will likely already be available.

Otherwise you can follow [these instructions](https://imperial-fons-computing.github.io/rstudio.html) to install R on your own computer.

In particular, on Ubuntu Linux, it is as simple as typing the following in terminal:

```bash
sudo apt install r-base r-base-dev
```

## Getting started

You should be using an IDE for R. Please re-visit the ["To IDE or not to IDE" section of the introduction](../intro.md) if you are not familiar with IDEs.

Let's briefly look at the bare-bones R interface and command line  interface (CLI), and then switch to an IDE like VS Code or RStudio.

Launch R (From Applications menu on Window or Mac, from terminal in  Linux/Ubuntu) — it should look something like this (on Linux/Ubuntu or Mac terminal):

![image](./graphics/R_Linux.png)

Or like this (Windows "console", similar in Mac):

![image](./graphics/R_terminal.jpg)

## R basics

Lets get started with some R basics. You will be working by entering R commands interactively at the R user prompt (`>`). Up and down arrow keys scroll through your command history.

### Useful R commands

|Command| What it does|
|:-|:-|
| `ls()`| list all the variables in the work space |
| `rm('a', 'b')`| remove variable(s) `a` and `b`|
| `rm(list=ls())`| remove all variable(s)|
| `getwd()`| get current working directory |
| `setwd('Path')`| set working directory to `Path`|
| `q()`| quit R |
| `?Command`| show the documentation of `Command`|
| `??Keyword`| search the all packages/functions with 'Keyword', "fuzzy search"|

### Baby steps

Like in any programming language, you will need to use "variables" to store information in a R session's workspace. Each variable has a reserved location in your [RAM](https://en.wikipedia.org/wiki/Random-access_memory), and takes up "real estate" in it — that is when you create a variable you reserve some space in your computer's memory.

$\star$ Now, let's try assigning a few variables in R and doing things to them:_____no_output_____

a <- 4  # store 4 as variable a_____no_output_____a # display it_____no_output_____a * a # product_____no_output_____

Store a variable:_____no_output_____

a_squared <- a * a _____no_output_____

```{note}
Unlike Python or most other programming languages, R uses the `<-` [operator to assign variables](https://stat.ethz.ch/R-manual/R-devel/library/base/html/assignOps.html). You can use `=` as well, but it does not work everywhere, so better to stick with `<-`.
```_____no_output_____

sqrt(a_squared) # square root_____no_output_____

Build a vector with `c` (stands for "`c`oncatenate"): _____no_output_____

v <- c(0, 1, 2, 3, 4)_____no_output_____v # Display the vector-valued variable you created_____no_output_____

Note that any text after a "#" is ignored by R, like in many other languages — handy for commenting.

*In general, please comment your code and scripts, for *everybody's* sake. You will be amazed by how difficult it is to read and understand what a certain R script does (or any other script, for that matter) without judicious comments — even scripts you yourself wrote not so long ago!

```{tip}

 **The Concatenate function:** `c()`(concatenate) is one of the most commonly used R functions because it is the default method for combining multiple arguments into a vector. To learn more about it, type `?c` at the R prompt and hit enter.
```_____no_output_____

is.vector(v) # check if v's a vector_____no_output_____mean(v) # mean_____no_output_____

Thus, a *vector* is like a single column or row in a *spreadsheet* (just like it is in [Python](05-Python_I.ipynb)). Multiple vectors can be combined to make a matrix (the full spreadsheet).

This is one of many ways R stores and processes data. More on R data types and objects below.

A single value (any kind) is a vector object of length 1 by default. That's why in the R console you see ` [1]` before any single-value output (e.g., type `8`, and you will see `[1] 8`)._____no_output_____Some examples of operations on vectors: _____no_output_____

var(v) # variance_____no_output_____median(v) # median_____no_output_____sum(v) # sum all elements_____no_output_____prod(v + 1) # multiply_____no_output_____length(v) # how many elements in the vector_____no_output_____

### Variable names and Tabbing

In R, you can name variables in the following way to keep track of related variables:_____no_output_____

wing.width.cm <- 1.2 #Using dot notation_____no_output_____wing.length.cm <- c(4.7, 5.2, 4.8)_____no_output_____

This can be handy; type:

```r
wing.
```

And then hit the ` tab`key to reveal all variables in that  category. This is nice — variable names should be as obvious as possible. However, they should not be over-long either! Good style and readability is more important than just convenient variable names.

In fact, R allows dots to be sued in variable names of all objects (not just objects that are variables). For example, functions names can have dots in them as well, as you will see below with the `is.*` family (e.g., `is.infinite()`, `is.nan()`, etc.). _____no_output_____### Operators
The usual operators are available in R:

|Operator||
|:-|:-|
| ` +`| Addition |
| ` -`| Subtraction|
| ` *`| Multiplication|
| ` /`| Division|
| `^`| Power|
| ` %%`| Modulo|
| ` %/%`| Integer division|
| ` ==`| Equals|
| ` !=`| Differs|
| `>`| Greater|
| `>=`| Greater or equal|
| ` &`| Logical and|
| $|$ |  Logical or|
| ` !`| Logical not|_____no_output_____### When things go wrong

Here are some common [syntax errors](https://en.wikipedia.org/wiki/Syntax_error) that you might run into in R, especially when you just beginning to learn this language:

* Missing a closing bracket leads to continuation line, which looks something like this, with a `+` at the end:

```r
x <- (1 + (2 * 3)
+
```

Hit ` Ctrl-C`(UNIX terminal or base R command line) or ESC (in in RStudio) or keep typing!

* Too many parentheses; for example, `2 + (2*3))`

* Wrong or mismatched brackets (see next subsection).

* Mixing double and single quotes will also give you an error

When things are taking too long and the R consile seems frozen, try `Ctrl + C` (UNIX terminal or base R command line) or ESC (in RStudio) to force an exit from whatever is going on._____no_output_____### Types of parentheses

R has specific uses for different types of Parentheses type that you need to get used to:

| Parentheses type| What it does|
|:-|:-|
|`f(3,4)`| call the function (or command) f, with the arguments 3 & 4. |
|`a + (b*c)`| to enforce order over which statements or calculations are executed. Here `(b*c)`is executed before adding to ` a`. Here is an alternative order: ` (a + b)*c`|
|`{expr1; expr2;...exprn}` | group a set of expressions or commands into one compound expression. Value returned is value of last expression; used in building function, loops, and conditionals (more on these soon!).|
|`x[4]`| get the 4th element of the vector ` x`.|
| `li[[3]]`| get the 3rd element of some list ` li`, and return it.(compare with `li[3]`, which returns a list with just the 3rd element inside). More on lists in next section._____no_output_____

li = list(c(1,2,3))_____no_output_____class(li)_____no_output_____

## Variable types

There are different kinds of data variable types in R, but you will basically need to know four for most of your work: integer, float (or "numeric", including real numbers), string (or "character", e.g.,
text), and Boolean ("logical"; ` True`or ` FALSE`).

Try this:_____no_output_____

v <- TRUE_____no_output_____v_____no_output_____class(v)_____no_output_____

The `class()` function tells you what type of variable any object in the workspace is._____no_output_____

v <- 3.2
class(v)_____no_output_____v <- 2L
class(v)_____no_output_____v <- "A string"
class(v)_____no_output_____b <- NA
class(b)_____no_output_____

Also, the `is.*` family of functions allow you to check if a variable is a specific in R: _____no_output_____

is.na(b)_____no_output_____b <- 0/0
b_____no_output_____is.nan(b)_____no_output_____b <- 5/0
b_____no_output_____is.nan(b)_____no_output_____

```{tip}
Beware of the difference between ` NA`(` N`ot `A`vailable) and `NaN`(` N`ot `a N`umber). R will use ` NA`to represent/identify missing values in data or outputs, while  ` NaN`represent nonsense values (e.g., 0/0) that cannot be represented as a number or some other data type.

See what R has to say about this: try ` ?is.nan`, ` ?is.na`, ` ?NA`, ` ?NaN`in the R commandline (one at a time!).

There are also ` Inf`(Infinity, e.g., 1/0), and ` NULL` (variable not set) value types. Look these up as well using ` ?`.
```_____no_output_____

is.infinite(b)_____no_output_____is.finite(b)_____no_output_____is.finite(0/0)_____no_output_____

### Type Conversions and Special Values

The `as.*` commands all convert a variable from one type to another. Try out the following examples:_____no_output_____

as.integer(3.1)_____no_output_____as.numeric(4)_____no_output_____as.roman(155)_____no_output_____as.character(155)  # same as converting to string_____no_output_____as.logical(5)_____no_output_____

*What just happened?!* R maps all values other than `0` to logical `True`, and `0` to `False`. This can be useful in some cases, for example, when you want to convert all your data to Presence-Absence only._____no_output_____

as.logical(0)_____no_output_____

Also, *keep an eye out for [E notation](https://en.wikipedia.org/wiki/Scientific_notation) in outputs of R functions for statistical analyses, and learn to interpret numbers formatted this way.* R uses E notation in outputs of statistical tests to display  very large or small numbers. If you are not used to different  representations of long numbers, the E notation might be  confusing.

Try this:_____no_output_____

1E4_____no_output_____1e4_____no_output_____5e-2_____no_output_____1E4^2_____no_output_____1 / 3 / 1e8_____no_output_____

```{tip}
**Boolean arguments in R**: In `R`, you can use `F` and `T` for boolean `FALSE` and `TRUE` respectively. To see this, type

`a <- T`

in the R commandline, and then see what R returns when you type `a`. Using `F` and `T` for boolean `FALSE` and `TRUE` respectively is not necessarily good practice, but be aware that this option exists.
```_____no_output_____## R Data Structures

R comes with different built-in structures (objects) for data storage and manipulation. Mastering these, and knowing which one to use when will help you write better, more efficient programs and also handle diverse datasets (numbers, counts, names, dates, etc).

```{note}
**Data "structures" vs. "objects" in R**: You will often see the terms "object" and "data structure" thrown around in this and other chapters. These two have a very distinct meaning in object-oriented programming (OOP) languages like R and Python. A data structure is just a "dumb" container for data (e.g., a vector). An object, on the other hand can be a data structure, but also any other variable or a function in your R environment. R, being an OOP language, converts everything in the current environment to an object so that it knows what to do with each such entity — each object type has its own set of rules for operations and manipulations that R uses when interpreting your commands.
```

### Vectors
The Vector, which you have already seen above, is a fundamental data object / structure in R. Scalars (single data values) are treated as vector of length 1. *A vector is like a single column or row in a spreadsheet.*

$\star$ Now get back into R (if you somehow quit R using ` q()`or something else), and try this:_____no_output_____

a <- 5
is.vector(a)_____no_output_____v1 <- c(0.02, 0.5, 1)
v2 <- c("a", "bc", "def", "ghij")
v3 <- c(TRUE, TRUE, FALSE)_____no_output_____v1;v2;v3_____no_output_____

R vectors can only store data of a single type (e.g., all numeric or all character). If you try to combine different types, R will homogenize everything to the same data type. To see this, try the following:_____no_output_____

v1 <- c(0.02, TRUE, 1)
v1_____no_output_____

TRUE gets converted to 1.00!_____no_output_____

v1 <- c(0.02, "Mary", 1)
v1_____no_output_____

Everything gets converted to text!

Basically, the function `c` "coerces" arguments that are of mixed types (strings/text, real numbers, logical arguments, etc) to a common type.

(R-matrices)=
### Matrices and arrays

A R "matrix" is a 2 dimensional vector (has both rows and columns) and a "array" can store data in more than two dimensions (e.g., a stack of 2-D matrices).

R has many functions to build and manipulate matrices and arrays.

Try this:_____no_output_____

mat1 <- matrix(1:25, 5, 5)
mat1_____no_output_____

Note that the output in your terminal or console will be more like:_____no_output_____

print(mat1)     [,1] [,2] [,3] [,4] [,5]
[1,]    1    6   11   16   21
[2,]    2    7   12   17   22
[3,]    3    8   13   18   23
[4,]    4    9   14   19   24
[5,]    5   10   15   20   25

(even if you don't use the `print()` command)

The output looks different above because the R code is running in a jupyter notebook.

Now try this:_____no_output_____

mat1 <- matrix(1:25, 5, 5, byrow=TRUE)
mat1_____no_output_____

That is, you can order the elements of a matrix by row instead of column (default)._____no_output_____

dim(mat1) #get the size of the matrix_____no_output_____

To make an array consisting of two 5$\times$5 matrices containing the integers 1-50:_____no_output_____

arr1 <- array(1:50, c(5, 5, 2))
arr1[,,1]_____no_output_____print(arr1), , 1

     [,1] [,2] [,3] [,4] [,5]
[1,]    1    6   11   16   21
[2,]    2    7   12   17   22
[3,]    3    8   13   18   23
[4,]    4    9   14   19   24
[5,]    5   10   15   20   25

, , 2

     [,1] [,2] [,3] [,4] [,5]
[1,]   26   31   36   41   46
[2,]   27   32   37   42   47
[3,]   28   33   38   43   48
[4,]   29   34   39   44   49
[5,]   30   35   40   45   50

arr1[,,2]_____no_output_____

Just like R vectors, R matrices and arrays have to be of a homogeneous type, and R will do the same sort of type homogenization you saw for R vectors above (try inserting a text value in `mat1`and see what happens). Thus R's matrix and array are similar to Python's `numpy` array and matrix data structures.

### Data frames

This is a very important data structure in R. Unlike matrices and vectors, R data frames can store data in which each column contains a different data type (e.g., numbers, strings, boolean) or even a combination of data types, just like a standard spreadsheet. Indeed, the dataframe data type was built to emulate some of the convenient properties of spreadsheets. Many statistical and plotting functions and packages in R naturally use data frames.

Let's build and manipulate a dataframe. First create three vectors:_____no_output_____

Col1 <- 1:10
Col1_____no_output_____Col2 <- LETTERS[1:10]
Col2_____no_output_____Col3 <- runif(10) # 10 random numbers from a uniform distribution
Col3_____no_output_____

Now combine them into a dataframe:_____no_output_____

MyDF <- data.frame(Col1, Col2, Col3)
MyDF_____no_output_____

Again, this output looks different from your terminal/console output because these commands are running in a Jupyter notebook.

Your output will look like this:_____no_output_____

print(MyDF)   Col1 Col2       Col3
1     1    A 0.96037682
2     2    B 0.64423265
3     3    C 0.98943218
4     4    D 0.68588555
5     5    E 0.55279158
6     6    F 0.20148309
7     7    G 0.53286961
8     8    H 0.09128843
9     9    I 0.22414756
10   10    J 0.40348969

You can easily assign names to the columns of dataframes:_____no_output_____

names(MyDF) <- c("MyFirstColumn", "My Second Column", "My.Third.Column")
MyDF_____no_output_____

And unlike matrices, you can access the contents of data frames by naming the columns directly using a $ sign:_____no_output_____

MyDF$MyFirstColumn_____no_output_____

But don't use spaces in column names! Try this: _____no_output_____

MyDF$My Second Column_____no_output_____

That gives an error, because R cannot handle spaces in column names easily.

So replace that column name using the `colnames` function:_____no_output_____

colnames(MyDF)_____no_output_____colnames(MyDF)[2] <- "MySecondColumn"
MyDF_____no_output_____

But using dots in column names is OK (just as it is for variable names): _____no_output_____

MyDF$My.Third.Column_____no_output_____

You can also access elements by using numerical indexing:   _____no_output_____

MyDF[,1]_____no_output_____

That is, you asked R to return values of `MyDF` in all Rows (therefore, nothing before the comma), and the first column (`1` after the comma)._____no_output_____

MyDF[1,1]_____no_output_____MyDF[c("MyFirstColumn","My.Third.Column")] # show two specific columns only_____no_output_____

You can check whether a particular object is a dataframe data structure with:_____no_output_____

class(MyDF)_____no_output_____

You can check the structure of a dataframe with `str()`:_____no_output_____

str(MyDF)'data.frame':	10 obs. of  3 variables:
 $ MyFirstColumn  : int  1 2 3 4 5 6 7 8 9 10
 $ MySecondColumn : chr  "A" "B" "C" "D" ...
 $ My.Third.Column: num  0.96 0.644 0.989 0.686 0.553 ...

You can print the column names and top few rows with `head()`:_____no_output_____

head(MyDF)_____no_output_____

And the bottom few rows with `tail()`:_____no_output_____

tail(MyDF)_____no_output_____

### Lists

A list is used to collect a group of data objects of different sizes and types (e.g., one whole data frame and one vector can both be in a single list). It is simply an ordered collection of objects (that can be variables).

```{note}
The outputs of many statistical functions in R are lists (e.g. linear model fitting using ` lm()`), to return all relevant information in one output object. So you need to know how to unpack and manipulate lists.
```

As a budding multilingual quantitative biologist, you should not be perturbed by the fact that a `list` is a very different data structure in python vs R. It will take some practice — sometimes the same word means different things in different human languages too!

Try this:_____no_output_____

MyList <- list(species=c("Quercus robur","Fraxinus excelsior"), age=c(123, 84))
MyList_____no_output_____

You can access contents of a list item using number of the item instead of name using nested square brackets:_____no_output_____

MyList[[1]]_____no_output_____MyList[[1]][1]_____no_output_____

or using the name in either this way:_____no_output_____

MyList$species_____no_output_____

or this way:  _____no_output_____

MyList[["species"]]_____no_output_____

And to access a specific element inside a list's item _____no_output_____

MyList$species[1]_____no_output_____

A more complex list:_____no_output_____

pop1<-list(species='Cancer magister',
           latitude=48.3,longitude=-123.1,
           startyr=1980,endyr=1985,
           pop=c(303,402,101,607,802,35))
pop1_____no_output_____

You can build lists of lists too:_____no_output_____

pop1<-list(lat=19,long=57,
           pop=c(100,101,99))
pop2<-list(lat=56,long=-120,
           pop=c(1,4,7,7,2,1,2))
pop3<-list(lat=32,long=-10,
           pop=c(12,11,2,1,14))
pops<-list(sp1=pop1,sp2=pop2,sp3=pop3)
pops_____no_output_____pops$sp1 # check out species 1_____no_output_____pops$sp1["pop"] # sp1's population sizes_____no_output_____pops[[2]]$lat #latitude of second species_____no_output_____pops[[3]]$pop[3]<-102 #change population of third species at third time step
pops_____no_output_____

Maybe you have guessed by now that R dataframes are actually a kind of list.

### Matrix vs Dataframe

If dataframes are so nice, why use R matrices at all? The problem is that dataframes can be too slow when large numbers of mathematical calculations or operations (e.g., matrix - vector multiplications or other linear algebra operations) need to be performed. In such cases, you will need to convert a dataframe to a matrix. But for statistical analyses, plotting, and writing output of standard R analyses to a file, data frames are more convenient. Dataframes also allow you to refer to columns by name (using `$`), which is often convenient.

To see the difference in memory usage of matrices vs dataframes, try this:_____no_output_____

MyMat = matrix(1:8, 4, 4)
MyMat_____no_output_____MyDF = as.data.frame(MyMat)
MyDF_____no_output_____object.size(MyMat) # returns size of an R object (variable) in bytes_____no_output_____object.size(MyDF)_____no_output_____

Quite a big difference!_____no_output_____## Creating and manipulating data

### Creating sequences

The `:` operator creates vectors of sequential integers:_____no_output_____

years <- 1990:2009
years_____no_output_____years <- 2009:1990 # or in reverse order
years_____no_output_____

For sequences of float numbers, you have to use `seq()`:_____no_output_____

seq(1, 10, 0.5)_____no_output_____

```{tip}
Don't forget, you can get help on a particular R command by prefixing it with `?`. For example, try:

`?seq`
```_____no_output_____You can also use `seq(from=1,to=10, by=0.5) `OR` seq(from=1, by=0.5, to=10)` with the same effect (try it). This explicit, "argument matching" approach is partly what makes R so popular and accessible to a wider range of users._____no_output_____### Accessing parts of data stuctures: Indices and Indexing

Every element (entry) of a vector in R has an order (an "index" value): the first value, second, third, etc. To illustrate this, let's create a simple vector:_____no_output_____

MyVar <- c( 'a' , 'b' , 'c' , 'd' , 'e' )_____no_output_____

Then, square brackets extract values based on their numerical order in the vector:_____no_output_____

MyVar[1] # Show element in first position _____no_output_____MyVar[4] # Show element in fourth position _____no_output_____

The values in square brackets are called "indices" — they give the index (position) of the required value. We can also select sets of values in different orders, or repeat values:_____no_output_____

MyVar[c(3,2,1)] # reverse order_____no_output_____MyVar[c(1,1,5,5)] # repeat indices_____no_output_____

You can also manipulate data structures/objects by indexing:_____no_output_____

v <- c(0, 1, 2, 3, 4) # Create a vector named v_____no_output_____v[3] # access one element_____no_output_____v[1:3] # access sequential elements_____no_output_____v[-3] # remove elements_____no_output_____v[c(1, 4)] # access non-sequential indices_____no_output_____

For matrices, you need to use both row and column indices:_____no_output_____

mat1 <- matrix(1:25, 5, 5, byrow=TRUE) #create a matrix
mat1_____no_output_____mat1[1,2]_____no_output_____mat1[1,2:4]_____no_output_____mat1[1:2,2:4]_____no_output_____

And to get all elements in a particular row or column, you need to leave the value blank:_____no_output_____

mat1[1,] # First row, all columns_____no_output_____mat1[,1] # First column, all rows_____no_output_____

(R-recycling)=
### Recycling

When vectors are of different lengths, R will recycle the shorter one to make a vector of the same length:_____no_output_____

a <- c(1,5) + 2
a_____no_output_____x <- c(1,2); y <- c(5,3,9,2)
x;y_____no_output_____x + y_____no_output_____

Strange! R just recycled `x` (repeated `1,2` twice) so that the two vectors could be summed! here's another example:_____no_output_____

x + c(y,1)Warning message in x + c(y, 1):
“longer object length is not a multiple of shorter object length”

*Think about what happened here*. R is clearly not comfortable doing this, so it warns you! Recycling could be convenient at times, but is dangerous!_____no_output_____### Basic vector-matrix operations

You can perform the usual vector matrix operations on R `vectors` _____no_output_____

v <- c(0, 1, 2, 3, 4)
v2 <- v*2 # multiply whole vector by 2
v2_____no_output_____v * v2 # element-wise product_____no_output_____t(v) # transpose the vector_____no_output_____v %*% t(v) # matrix/vector product_____no_output_____v3 <- 1:7 # assign using sequence
v3_____no_output_____v4 <- c(v2, v3) # concatenate vectors
v4_____no_output_____

### Strings and Pasting

It is important to know how to handle strings in R for two main reasons:

* To deal with text data, such as names of experimental treatments
* To generate appropriate text labels and titles for figures

Let's try creating and manipulating strings:_____no_output_____

species.name <- "Quercus robur" #You can alo use single quotes
species.name_____no_output_____

To combine to strings: _____no_output_____

paste("Quercus", "robur")_____no_output_____paste("Quercus", "robur",sep = "") #Get rid of space_____no_output_____paste("Quercus", "robur",sep = ", ") #insert comma to separate_____no_output_____

As you can see above, both double and single quotes work, but using double quotes is better because it will allow you to define strings that contain a single quotes, which is often necessary.

And as is the case with so many R functions, pasting works on vectors:_____no_output_____

paste('Year is:', 1990:2000)_____no_output_____

Note that this last example creates a vector of 11 strings as it is 1990:2000 *inclusive*. _____no_output_____## Useful R functions

There are a number of very useful functions available by default (in the "base packages"). Some particularly useful ones are listed below.

### For manipulating strings

|Function||
|:-|:-|
|`strsplit(x,';')`| Split the string `x` at ';' |
|`nchar(x)`| Number of characters in string `x`|
|`toupper(x)`| Set string `x` to upper case|
|`tolower(x)`| Set string `x` to lower case|
|`paste(x1,x2,sep=';')`| Join the two strings using ';'|

### Mathematical

|Function||
|:-|:-|
|`log(x)`| Natural logarithm of the number (or every number in the vector or matrix) `x`|
|`log10(x)`| Logarithm in base 10 of the number (or every number in the vector or matrix) `x`|
|`exp(x)`| Exponential of the number (or every number in the vector or matrix) `x` ($e^x$)|
|`abs(x)`| Absolute value of the number (or every number in the vector or matrix) `x`|
|`floor(x)`| Largest integer smaller than the number (or every number in the vector or matrix) `x`|
|`ceiling(x)`| Smallest integer greater than the number (or every number in the vector or matrix) `x`|
|`sqrt(x)`| Square root of the number (or every number in the vector or matrix) `x` ($\sqrt{x}$)|
|`sin(x)`| Sine function of the number (or every number in a vector or matrix) `x`|
|`pi`| Value of the constant $\pi$|

### Statistical

|Function||
|:-|:-|
|`mean(x)`| Compute mean of (a vector or matrix) `x`|
|`sd(x)`| Standard deviation of (a vector or matrix) `x`|
|`var(x)`| Variance of (a vector or matrix) `x`|
|`median(x)`| Median of (a vector or matrix) `x`|
|`quantile(x,0.05)`| Compute the 0.05 quantile of (a vector or matrix) `x`|
|`range(x)`| Range of the data in (a vector or matrix) `x`|
|`min(x)`| Minimum of (a vector or matrix) `x`|
|`max(x)`| Maximum of (a vector or matrix) `x`|
|`sum(x)`| Sum all elements of (a vector or matrix) `x`|
|`summary(x)`| Summary statistics for (a vector or matrix) `x`|

### Sets

|Function||
|:-|:-|
|`union(x, y)` | Union of all elements of two vectors x & y|
|`intersect(x, y)`|Elements common to two vectors x & y|
|`setdiff(x, y)`|Elements unique to two vectors x & y|
|`setequal(x, y)`|Check if two vectors x & y are the same set (have same unique elements)|
|`is.element(x, y)`|Check if an element x is in vector y (same as `x %in% y`)|

$\star$ *Try out the above commands in your R console by generating the appropriate data*. For example, _____no_output_____

strsplit("String; to; Split",';')# Split the string at ';'_____no_output_____

(R-random-numbers)=
### Generating Random Numbers

You will probably need to generate random numbers at some point as a quantitative biologist.

R has many routines for generating random samples from various probability distributions. There are a number of random number distributions that you can sample or generate random numbers from:

|Function||
|:-|:-|
|`rnorm(10, m=0, sd=1)`| Draw 10 normal random numbers with mean=0 and standard deviation = 1|
|`dnorm(x, m=0, sd=1)` | Density function|
|`qnorm(x, m=0, sd=1)` | Cumulative density function|
|`runif(20, min=0, max=2)` | Twenty random numbers from uniform `[0,2]`|
|` rpois(20, lambda=10)` | Twenty random numbers from Poisson (with mean $\lambda$)|

#### "Seeding" random number generators

Computers *can't* generate *true* mathematically random numbers. This may seem surprising, but basically a computer cannot be programmed to do things purely by chance; it can only follow given instructions blindly and is therefore completely predictable. Instead, computers have algorithms called "pseudo-random number generators" that generate *practically random* sequences of numbers. These are typically based on a iterative formula that generates a sequence of random numbers, starting with a first number called the "**seed**". This sequence is completely "deterministic", that is, starting with a particular seed yields exactly the same sequence of pseudo-random numbers every time you re-run the generator.

Try this:_____no_output_____

set.seed(1234567)
rnorm(1)_____no_output_____

Everybody in the class  will get the same answer!

Now try and compare the results with your neighbor:_____no_output_____

rnorm(10)_____no_output_____

And then the whole sequence of 11 numbers you generated:_____no_output_____

set.seed(1234567)
rnorm(11)_____no_output_____

Thus, setting the seed allows you to reliably generate the identical sequence of "random" numbers. These numbers are not truly random, but have the properties of random numbers. Note also that pseudo-random number generators are periodic, which means that the sequence will eventually repeat itself. However, this  period is so long that it can be ignored for most practical purposes. So effectively, `rnorm` has an enormous list that it cycles through. The random seed starts the process, i.e., indicates where in the list to start. This is usually taken from the clock when you start R.

But why bother with random number seeds? Setting a particular seed can be useful when debugging programs (coming up below). Bugs in code can be hard to find — harder still if you are generating random numbers, so repeat runs of your code may or may not all trigger the same behavior. You can set the seed once at the beginning of the code — ensuring repeatability, retaining (pseudo) randomness. Once  debugged, if you want, you can remove the set seed line._____no_output_____## Your analysis workflow

In using R for an analysis, you will likely use and create several files. As in the case of bash and Python based projects, in R projects as well, you should keep your workflow well organized. For example, it is sensible to create a folder (directory) to keep all code files together. You can then set R to work from this directory, so that files are easy to find and run — this will be your "working directory" (more on this below). Also, you don't want to mix code files with data and results files. So you should create separate directories for these as well.

Thus, your typical R analysis workflow will be:

---
![image](./graphics/RWorkflow.png)
Your R project. Keep it neat and organized!

---

Some details on each kind of file:

* *R script files*: These are plain text files containing all the R code needed for an analysis. These should  be created with a text editor, typically part of some smart code editor like vscode, or RStudio, and saved with the extension `*.R`. You should *never* use Word to save or edit these files as R can only read code from plain text files.

* *Text data files* These are files of data in plain text format containing one or more columns of data (numbers, strings, or both). Although there are several format options, we will typically be using `csv` files, where the data entries are separated by commas. These are easy to create and export from Excel (if that's what you use...).

* *Results output files* These are a plain text files containing your results, such the the summary of output of a regression or ANOVA analysis. Typically, you will output your results in a table format where the columns are separated by commas (csv) or tabs (tab-delimited).

* *Graphics files* R can export graphics in a wide range of formats. This can be done automatically from R code and we will look at this later but you can also select a graphics window (e.g., in RStudio) and click `File` $\triangleright$ `Save as...`.

* *Rdata files* You can save any data loaded or created in R, including outputs of statistical analyses and other things, into a single`Rdata` file. These are not plain text and can only be read by R, but can hold all the data from an analysis in a single handy location. We will not use these much in this course.

So let's build your R analysis project structure.

Do the following:

$\star$ Create a sensibly-named directory (e.g., `MyRCoursework`, ` week3`, etc) in an appropriate location on your computer. If you are using a college Windows computer, you may need to create it in your `H:` drive. Avoid including spaces in your file or directory names, as this will often create problems when you share your file or directory with somebody else. Many software programs do not handle spaces in file/directory names well. Use underscores instead of spaces. For example, instead of ` My R Coursework`, use ` My_R_Coursework` or ` MyRCoursework`.

$\star$ Create subdirectories *within this directory* called `code`, `data`, and `results`. Remember,  commands in all programming languages are case-sensitive when it comes to reading directory path names, so `code` is not the same as `Code`!

You can create directories using `dir.create()`within R (or if on Mac/Linux, with the usual `mkdir` from the bash terminal):

```R
dir.create("MyRCoursework")
dir.create("MyRCoursework/code")
dir.create("MyRCoursework/data")
dir.create("MyRCoursework/results")
```_____no_output_____### The R Workspace and Working Directory

R has a "workspace" – a current working environment that includes any user-defined data structures objects (vectors, matrices, data frames, lists) as well as other objects (e.g., functions). At the end of an R session, the user can save an image of the current workspace that is automatically reloaded the next time R is started. Your workspace is saved in your "Working Directory", which has to be set manually.

So before we go any further, let's get sort out where your R "Working Directory" should be and how you should set it. R has a default location where it assumes your working directory is.

* In UNIX/Linux, it is whichever directory you are in when you launch R.

* In Mac, it is ` /User/YourUserName` or similar.

* In Windows, it is ` C:/Windows/system32`or similar.

To see where your current working directory is, at the R command prompt, type:

`getwd()`

This tells you what the current `w`orking `d`irectory is.

Now, set the working directory to be ` MrRCourseworkcode`. For example, if you created ` MrRCoursework`directly in your `H:\`, the you would use:

`setwd("H:/MrRCourseworkcode")`

`dir()` #check what's in the current working directory

On your own computer, you can also change R's default to a particular working directory where you would like to start (easily done in RStudio):

* In Linux, you can do this by editing the `Rprofile.site`site with `sudo gedit /etc/R/Rprofile.site`. In that file, you would add your start-up parameters between the lines

    `.First <- function() cat("\n Welcome to R!\n\n")`

and

    `.Last <- function() cat("\n   Goodbye! \n\n")`.

Between these two lines, insert:
    `setwd("/home/YourName/YourDirectoryPath")`

* In Windows and Macs, you can find the ` Rprofile.site`file by searching for it. On Windows, it should be at `C:\Program Files\R\R-x.x.x\etc` directory, where `x.x.x` is your R version.

* If you are using RStudio, you can change the default working directory through the RStudio "Options" dialog._____no_output_____## Importing and Exporting Data

We are now ready to see how to import and export data in R, typically the first step of your analysis. The best option is to have your data in a ` c`omma ` s`eparated ` v`alue (` csv`) text file or in a tab separated text file. Then, you can use the function ` read.csv`(or ` read.table`) to import your data. Now, lets get some data into your ` Data`directory.

$\star$ Go to the [TheMulQuaBio git repository](https://github.com/mhasoba/TheMulQuaBio) and navigate to the [`data` directory](https://github.com/mhasoba/TheMulQuaBio/tree/master/content/data).

$\star$ Download and copy the file [`trees.csv`](https://raw.githubusercontent.com/mhasoba/TheMulQuaBio/master/content/data/trees.csv) into your own `data` directory.

Alternatively, of you may download the whole repository to your computer and then grab the file from the place where you downloaded it.

Now, import the data:_____no_output_____

setwd("../code/")_____no_output_____MyData <- read.csv("../data/trees.csv")_____no_output_____ls() #Check that MyData has appeared_____no_output_____

Your output may be somewhat different, depending on what what variables and other objects have been created in
your R Workspace during the current R session. But the main thing is, you should be able to see a `MyData` in the list of objects printed.  _____no_output_____Note that the resulting `MyData` object in your workspace is a R dataframe:_____no_output_____

class(MyData)_____no_output_____

### Relative paths

Note the UNIX-like path to the file we used in the `read.csv()` command above (using forward slashes; Windows uses back slashes).

The `../` in `read.csv("../data/trees.csv")` above signifies a "relative" path. That is, you are asking R to load data that lies in a different directory (folder) *relative* your current location (in this case, you are in your ` Code`directory). In other, more technical words, `../data/trees.txt`points to a file named `trees.txt`located in the "parent" of the current directory.

*What is an absolute path?* — one that specifies the whole path on your computer, say from ` C:\`"upwards" on Windows, `/Users/` upwards on Mac, and `/home/` upwards on Linux. Absolute paths are specific to each computer, so should be avoided. So to import data and export results, your script should *not* use absolute paths. Also, *AVOID putting a `setwd()`command at the start of your R script*, because setting the working directory requires an absolute directory path, which will differ across computers, platforms, and users. Let the end users set the working directory on their machine themselves.

Using relative paths in in your R scripts and code will make your code computer-independent and easier for others to use your code. The relative path way should always be the way you load data in your analyses scripts — it will guarantee that your analysis works on every computer, not just your own or college computer.

```{tip}
If you are using a computer from elsewhere in the EU, Excel may use a comma (e.g., $\pi=3,1416$) instead of a decimal point ($\pi=3.1416$). In this case, `csv`files may use a semi-colon to separate columns and you can use the alternative function `read.csv2()` to read them into the R workspace.

```_____no_output_____

head(MyData) # Have a quick look at the data frame_____no_output_____

You can also have a more detailed look at the data you imported:_____no_output_____

str(MyData) # Note the data types of the three columns'data.frame':	120 obs. of  3 variables:
 $ Species      : chr  "Populus tremula" "Quercus robur" "Ginkgo biloba" "Fraxinus excelsior" ...
 $ Distance.m   : num  31.7 46 31.2 34.6 45.5 ...
 $ Angle.degrees: num  41.3 44.5 25.1 23.3 38.3 ...
MyData <- read.csv("../data/trees.csv", header = F) # Import ignoring headers_____no_output_____head(MyData)_____no_output_____

Or you can load data using the more general `read.table` function:_____no_output_____

MyData <- read.table("../data/trees.csv", sep = ',', header = TRUE) #another way_____no_output_____

With `read.table` you need to specify whether there is a header row that needs to be imported as such. _____no_output_____

head(MyData)_____no_output_____MyData <- read.csv("../data/trees.csv", skip = 5) # skip first 5 lines_____no_output_____

### Writing to and saving files

You can also save your data frames using `write.table` or `write.csv`:_____no_output_____

write.csv(MyData, "../results/MyData.csv")_____no_output_____dir("../results/") # Check if it worked_____no_output_____write.table(MyData[1,], file = "../results/MyData.csv",append=TRUE) # appendWarning message in write.table(MyData[1, ], file = "../results/MyData.csv", append = TRUE):
“appending column names to file”

You get a warning with here because R thinks it is strange that you are appending headers to a file that already has headers!_____no_output_____

write.csv(MyData, "../results/MyData.csv", row.names=TRUE) # write row names_____no_output_____write.table(MyData, "../results/MyData.csv", col.names=FALSE) # ignore col names_____no_output_____

## Writing R code

Typing in commands interactively in the R console is good for starters, but you will want to switch to putting your sequence of commands into a script file, and then ask R to run those commands.

* Open a new text file, call it `basic_io.R`, and save it to your `code`directory.
* Write the above input-output commands in it:

```R
# A simple script to illustrate R input-output.
# Run line by line and check inputs outputs to understand what is happening

MyData <- read.csv("../data/trees.csv", header = TRUE) # import with headers

write.csv(MyData, "../results/MyData.csv") #write it out as a new file

write.table(MyData[1,], file = "../results/MyData.csv",append=TRUE) # Append to it

write.csv(MyData, "../results/MyData.csv", row.names=TRUE) # write row names

write.table(MyData, "../results/MyData.csv", col.names=FALSE) # ignore column names

```

* Now place the cursor on the first line of code in the script file and run it by pressing the appropriate keyboard shortcut (e.g., PC: ctrl+R, Mac: command+enter, Linux: ctrl+enter are the usual shortcuts for doing this).
* Check after every line that you are getting the expected result._____no_output_____## Running R code

But even writing to a script file and running the code line-by-line or block-by-block is not your ultimate goal. What you would really like to do is to just run your full analysis and output all the results. There are two main approaches for running R script/code.

### Using `source`

You can run all the contents of a `*.R`script file from the R command line by using `source()`.

$\star$ Try sourcing `basic_io.R` (you will need to make sure you have `setwd` to your code directory):_____no_output_____

source("basic_io.R")Warning message in write.table(MyData[1, ], file = "../results/MyData.csv", append = TRUE):
“appending column names to file”

That has run OK, warning and all.

* If you get errors, read and then try to fix them. The most common problem is likely to be that you have not `setwd()` to the `code` directory.

Alternatively, you can run the script from wherever (e.g., `data` directory) by adding the directory path to the script file name, if the script file is not in your working directory and you don't want to change your working directory. For example, you will need `source("../code/control.R` if your working directory is `data` and not `code` (using a relative path).

*Do not* put a ` source()`command inside the script file you are sourcing, as it is then trying to run itself again and again and that's just cruel!*

```{tip}
The command ` source()`has a `chdir`argument whose default value is FALSE. When set to TRUE, it will change the working directory to the directory of the file being sourced.

```

Also, if you have `source`ed a script successfully, you will see no output on in R console/terminal unless there was an error or warning, or if you explicitly asked for something to be printed. So it can be useful to add a line at the end of the script saying something like `print("Script complete!")`.

### Using ` Rscript`

You can also run R script from the UNIX/Linux terminal by calling `Rscript`. That is, while you have to be inside an R session to use the ` source` command to run a script, you can run a R script directly from the UNIX/Linux terminal by calling `Rscript`.

This allows you to easily automate execution of your R scripts (e.g., by writing a bash script) and integrate R into a bigger computing pipeline/workflow by calling it through other tools or languages (e.g., see the [Python Chapter II](06-Python_II.ipynb)).

If you are on Linux, try using `Rscript` to run `basic_io.R`:

* Exit from the R console using `ctrl+D`, or open a new bash terminal

* `cd` to the location of ` basic_io.R` (e.g., `week3/code`)

* Then run the script using `Rscript basic_io.R`

Also, please have a look at `man Rscript` in a bash terminal.

### Running R in batch mode

In addition to ` Rscript`, there is another way to run you R script without opening the R console. In Mac or Linux, you can do so by typing:

`R CMD BATCH MyCode.R MyResults.Rout`

This will create an `MyResults.Rout`file containing all the output. On Microsoft Windows, it's more complicated — change the path to `R.exe`and output file as needed:

`"C:\Program Files\R\R-4.x.x\bin\R.exe" CMD BATCH -vanilla -slave "C:\PathToMyResults\Results\MyCode.R"`

Here, replace 4.x.x with the R version you have._____no_output_____## Control flow tools

In R, you can write `if`, `then`, `else` statements, and `for` and `while` loops like any programming language to give you finer control over your program's "control flow". Such statements are useful to include in functions and scripts because you may only want to do certain calculations or other tasks, under certain conditions (e.g., ` if` the dataset is from a particular year, do something different).

Let's look at some examples of code exemplifying control flow tools in R.

$\star$ Type each of the following blocks of code in a script file called `control_flow.R` and save it in your `code` directory. then run each block *separately* by sending or pasting into the R console. _____no_output_____### `if` statements_____no_output_____

a <- TRUE
if (a == TRUE){
    print ("a is TRUE")
    } else {
    print ("a is FALSE")
}[1] "a is TRUE"

You can also write an `if` statement on a single line:_____no_output_____

z <- runif(1) ## Generate a uniformly distributed random number
if (z <= 0.5) {print ("Less than a half")}[1] "Less than a half"

But code readability is important, so avoid squeezing control flow blocks like this into a single line.

```{tip}
Please indent your code for readability, even if its not strictly necessary (unlike [Python](./05-Python_I.ipynb)). Indentation helps you see the flow of the logic, rather than flattened version, which is hard for you and anybody else to read. For example, the following code block is so much much readable than the one above:

```r
z <- runif(1)
if (z <= 0.5) {
    print ("Less than a half")
   }
```_____no_output_____### `for` loops_____no_output_____[Loops](https://en.wikipedia.org/wiki/Control_flow#Loops) are really useful to repeat a task over some range of input values.

The following code "loops" over a range of numbers, squaring each, and then printing the result:_____no_output_____

<code>
for (i in 1:10){
    j <- i * i
    print(paste(i, " squared is", j ))
}[1] "1  squared is 1"
[1] "2  squared is 4"
[1] "3  squared is 9"
[1] "4  squared is 16"
[1] "5  squared is 25"
[1] "6  squared is 36"
[1] "7  squared is 49"
[1] "8  squared is 64"
[1] "9  squared is 81"
[1] "10  squared is 100"

</code>

What exactly is going on in the piece of code above? What are `i` and `j`? Let's break it down:

Firstly The `1:10` part simply generates a sequence, as you [learned above](#Creating-sequences):  _____no_output_____

<code>
1:10_____no_output_____
</code>

This is the same as using `seq(10)` (try substituting this instead of `1:10` in the above block of code).

Then, `j` is a temporary variable that stores the value of the number (the other temporary variable, `i`) in that iteration of the loop.

```{note}
Using the `:` operator or the `seq()` function pre-generates the sequence and stores it in memory, so is less efficient that Python's [`range` function](./05-Python_I.ipynb), which generates numbers in a sequence on a "need-to" basis.
```

You can also loop over a vector of strings:_____no_output_____

<code>
for(species in c('Heliodoxa rubinoides',
                 'Boissonneaua jardini',
                 'Sula nebouxii')){
  print(paste('The species is', species))
}[1] "The species is Heliodoxa rubinoides"
[1] "The species is Boissonneaua jardini"
[1] "The species is Sula nebouxii"

</code>

These is a random assortment of birds!

And here's a for loop using apre-existing vector:_____no_output_____

<code>
v1 <- c("a","bc","def")
for (i in v1){
    print(i)
}[1] "a"
[1] "bc"
[1] "def"

</code>

### `while` loops

If you want to perform an operation till some condition is met, use a `while` loop:_____no_output_____

<code>
i <- 0
while (i < 10){
    i <- i+1
    print(i^2)
}[1] 1
[1] 4
[1] 9
[1] 16
[1] 25
[1] 36
[1] 49
[1] 64
[1] 81
[1] 100

</code>

$\star$ Now test `control_flow.R`. That is, run the script file  using `source` (and `Rscript` if on Linux).

If you get errors, read them carefully and fix them (this is going to be your mantra henceforth!)._____no_output_____### Some more control flow tools

Let's look at some more control tools that are less commonly used, but can be useful in certain scenarios.

#### `break`ing out of loops
Often it is useful (or necessary) to `break` out of a loop when some condition is met. Use ` break` (like in pretty much any other programming language, like Python) in situations when you cannot set a target number of iterations and would like to stop the loop execultion once some condition is met (as you would with a ` while` loop).

Try this (type into `break.R` and save in `code`):_____no_output_____

<code>
i <- 0 #Initialize i
    while(i < Inf) {
        if (i == 10) {
            break
             } # Break out of the while loop!
        else {
            cat("i equals " , i , " \n")
            i <- i + 1 # Update i
    }
}i equals  0
i equals  1
i equals  2
i equals  3
i equals  4
i equals  5
i equals  6
i equals  7
i equals  8
i equals  9

</code>

#### Using `next`

You can also skip to next iteration of a loop. Both `next` and `break` can be used within other loops (` while`, ` for`). Try this (type into `next.R` and save in `code`):_____no_output_____

<code>
for (i in 1:10) {
  if ((i %% 2) == 0) # check if the number is odd
    next # pass to next iteration of loop
  print(i)
}[1] 1
[1] 3
[1] 5
[1] 7
[1] 9

</code>

This code checks if a number is odd using the "modulo" operation and prints it if it is._____no_output_____## Writing R Functions

A function is a block of re-useable code that takes an input, does something with it (or to it!), and returns the result. Like any other programming language, R lets you write your own functions. All the "commands" that you have been using, such as ` ls()`, `mean()`, `c()`, etc are basically functions. You will want to write your own function for every scenario where a particular task or set of analysis steps need to be performed again and again.
The syntax for R functions is quite simple, with each function accepting "arguments" and "returning" a value.

### Your first R function

$\star$ Type the following into a script file called  `boilerplate.R`and save it in your `code` directory:

```R

# A boilerplate R script

MyFunction <- function(Arg1, Arg2){

  # Statements involving Arg1, Arg2:
  print(paste("Argument", as.character(Arg1), "is a", class(Arg1))) # print Arg1's type
  print(paste("Argument", as.character(Arg2), "is a", class(Arg2))) # print Arg2's type

  return (c(Arg1, Arg2)) #this is optional, but very useful
}

MyFunction(1,2) #test the function
MyFunction("Riki","Tiki") #A different test
```
Note the curly brackets – these are necessary for R to know where the specification of the function starts and ends. Also, note the indentation. Not necessary (unlike Python), but recommended to make the code more readable.

Now enter the R console and source the script:_____no_output_____

<code>
source("boilerplate.R")[1] "Argument 1 is a numeric"
[1] "Argument 2 is a numeric"
[1] "Argument Riki is a character"
[1] "Argument Tiki is a character"

</code>

This will run the script, and also save your function `MyFunction` as an object into your workspace (try `ls()`, and you will see `MyFunction` appear in the list of objects):_____no_output_____

<code>
ls()_____no_output_____
</code>

Again, you output will likely be a bit different as you have been different things in your R workspace/session than I have. What matters is that you see `MyFunction` in the list of objects above.

Now try: _____no_output_____

<code>
class(MyFunction)_____no_output_____
</code>

So, yes, `MyFunction`is a `function` object! _____no_output_____### Functions with conditionals

Here are some examples of functions with conditionals:_____no_output_____

<code>
# Checks if an integer is even
is.even <- function(n = 2){
  if (n %% 2 == 0)
  {
    return(paste(n,'is even!'))
  }
  return(paste(n,'is odd!'))
}

is.even(6)_____no_output_____# Checks if a number is a power of 2
is.power2 <- function(n = 2){
  if (log2(n) %% 1==0)
  {
    return(paste(n, 'is a power of 2!'))
  }
  return(paste(n,'is not a power of 2!'))
}

is.power2(4)_____no_output_____# Checks if a number is prime
is.prime <- function(n){
  if (n==0){
    return(paste(n,'is a zero!'))
  }
  if (n==1){
    return(paste(n,'is just a unit!'))
  }
  ints <- 2:(n-1)
  if (all(n%%ints!=0)){
    return(paste(n,'is a prime!'))
  }
  return(paste(n,'is a composite!'))
}

is.prime(3)_____no_output_____
</code>

$\star$ Run the three blocks of code in R one at a time and make sure you understand what each function is doing and how. Save all three blocks to a single script file called `R_conditionals.R` in your `code` directory, and make sure it runs using `source` and `Rscript` (in the Linux/Ubuntu terminal) . _____no_output_____### An example utility function

Now let's write a script containing a more useful function:

$\star$ In your text editor type the following in a file called `TreeHeight.R`, and save it in your `code`directory:

```R
# This function calculates heights of trees given distance of each tree
# from its base and angle to its top, using  the trigonometric formula
#
# height = distance * tan(radians)
#
# ARGUMENTS
# degrees:   The angle of elevation of tree
# distance:  The distance from base of tree (e.g., meters)
#
# OUTPUT
# The heights of the tree, same units as "distance"

TreeHeight <- function(degrees, distance){
    radians <- degrees * pi / 180
    height <- distance * tan(radians)
    print(paste("Tree height is:", height))

    return (height)
}

TreeHeight(37, 40)
```

* Run `TreeHeight.R`'s two blocks (the `TreeHeight` function, and the call to the function, `TreeHeight(37, 40)`) by pasting them sequentially into the R console. Try and understand what each line is doing.
* Now test the whole `TreeHeight.R` script at one go using `source` and/or `Rscript` (in Linux/Ubuntu)._____no_output_____## Practicals

### Tree heights

Modify the script `TreeHeight.R` so that it does the following:
  * Loads `trees.csv` and calculates tree heights for all trees in the data. Note that the distances have been measured in meters. (Hint: use relative paths)).
  * Creates a csv output file called `TreeHts.csv` in `results`that contains the calculated tree heights along with the original data in the following format (only first two rows and headers shown):
    ```bash
    "Species","Distance.m","Angle.degrees","Tree.Height.m"
    "Populus tremula",31.6658337740228,41.2826361937914,27.8021161438536
    "Quercus robur",45.984992608428,44.5359166583512,45.2460250644405
    ```
This script should work using either `source` or `Rscript` in Linux._____no_output_____## Vectorization

R is relatively slow at cycling through a data structure such as a dataframe or matrix (e.g., by using a `for` loop) because it is a *high-level, interpreted computer language*.

That is, when you execute a command in R, it needs to "read" and interpret the necessary code from scratch every single time the command is called. On the other hand, compiled languages like C know exactly what the flow of the program is because the code is pre-interpreted and ready to go before execution (i.e., the code is "compiled").

For example, when you assign a new variable in R:_____no_output_____

<code>
a <- 1.0
class(a)_____no_output_____
</code>

R automatically figures out that `1.0` is an floating point number, and finds a place in the system memory for it with `a` registered as a "pointer" to it.

In contrast, in C, for example, you would have to do so manually by giving `a` an address and space in memory:

```C
float a
a = 1
```

*Vectorization is an approach where you directly apply compiled, optimized code to run an operation on a vector, matrix, or an higher-dimensional data structure (like an R array), instead of performing the operation element-wise (each row or column element one at a time) on the data structure*.

Apart from computational efficiency vectorization makes writing code more concise, easy to read, and less error prone.

Let's try an example that illustrates this point.

$\star$ Type (save in `Code`) as `Vectorize1.R` the following script, and run it (it sums all elements of a matrix):_____no_output_____

<code>
M <- matrix(runif(1000000),1000,1000)

SumAllElements <- function(M){
  Dimensions <- dim(M)
  Tot <- 0
  for (i in 1:Dimensions[1]){
    for (j in 1:Dimensions[2]){
      Tot <- Tot + M[i,j]
    }
  }
  return (Tot)
}

print("Using loops, the time taken is:")
print(system.time(SumAllElements(M)))

print("Using the in-built vectorized function, the time taken is:")
print(system.time(sum(M)))[1] "Using loops, the time taken is:"
   user  system elapsed
  0.101   0.000   0.101
[1] "Using the in-built vectorized function, the time taken is:"
   user  system elapsed
  0.002   0.000   0.002

</code>

Note the `system.time` R function: it calculates how much time your code takes. This time will vary with every run, and every computer that this code is run on (so your times will be a bit different than what I got).

Both `SumAllElements()` and `sum()` approaches are correct, and will give you the right answer. However, the inbuilt function `sum()`  is about 100 times faster than the other, because it uses vectorization, avoiding the amount of looping that `SumAllElements()` uses.

In effect, of course, the computer still has to run loops. However, this running of the loops is encapsulated in a pre-complied program that R calls. These programs are written in more primitive (and therefore, faster) languages like Fortran and C. For example `sum` is actually written in C if you look under the hood.

In R, even if you should try to avoid loops, in practice, it is often much easier to throw in a `for` loop, and *then* "optimize" the code to avoid the loop if the running time is not satisfactory. Therefore, it won't hurt you to become really familiar with loops and looping as you learned in the sections above.

### Pre-allocation

And if you are using loops, one operation that is slow in R (and somewhat slow in all languages) is memory allocation for a particular variable that will change during loping (e.g., a variable that is a dataframe). So writing a for loop that resizes a vector repeatedly makes R re-allocate memory repeatedly, which makes it slow. Try this:_____no_output_____

<code>
NoPreallocFun <- function(x){
    a <- vector() # empty vector
    for (i in 1:x) {
        a <- c(a, i)
        print(a)
        print(object.size(a))
    }
}

system.time(NoPreallocFun(10))[1] 1
56 bytes
[1] 1 2
56 bytes
[1] 1 2 3
64 bytes
[1] 1 2 3 4
64 bytes
[1] 1 2 3 4 5
80 bytes
[1] 1 2 3 4 5 6
80 bytes
[1] 1 2 3 4 5 6 7
80 bytes
[1] 1 2 3 4 5 6 7 8
80 bytes
[1] 1 2 3 4 5 6 7 8 9
96 bytes
 [1]  1  2  3  4  5  6  7  8  9 10
96 bytes

</code>

Here, in each repetition of the for loop, R has to re-size the vector and re-allocate memory. It has to find the vector in memory, create a new vector that will fit more data, copy the old data over, insert the new data, and erase the old vector. This can get very slow as vectors get big.

On the other hand, if you "pre-allocate" a vector that fits all the values, R doesn't have to re-allocate memory each iteration, and the results can be much faster, especially when the vector lets large. Here's how you'd do that for the above case:_____no_output_____

<code>
PreallocFun <- function(x){
    a <- rep(NA, x) # pre-allocated vector
    for (i in 1:x) {
        a[i] <- i
        print(a)
        print(object.size(a))
    }
}

system.time(PreallocFun(10)) [1]  1 NA NA NA NA NA NA NA NA NA
96 bytes
 [1]  1  2 NA NA NA NA NA NA NA NA
96 bytes
 [1]  1  2  3 NA NA NA NA NA NA NA
96 bytes
 [1]  1  2  3  4 NA NA NA NA NA NA
96 bytes
 [1]  1  2  3  4  5 NA NA NA NA NA
96 bytes
 [1]  1  2  3  4  5  6 NA NA NA NA
96 bytes
 [1]  1  2  3  4  5  6  7 NA NA NA
96 bytes
 [1]  1  2  3  4  5  6  7  8 NA NA
96 bytes
 [1]  1  2  3  4  5  6  7  8  9 NA
96 bytes
 [1]  1  2  3  4  5  6  7  8  9 10
96 bytes

</code>

You can't really see the difference in timing here using `system.time()` because the vector is really small (just 10 elements). To really see the difference, increase the iterations from 10 to, say, 100000.

$\star$ Write the above two blocks of code into a script called `preallocate.R`.

Fortunately, R has several functions that can operate on entire vectors and matrices without requiring looping (Vectorization). That is, vectorizing a computer program means you write it such that as many operations as possible are applied to whole data structure (vectors, matrices, dataframes, lists, etc) at one go, instead of its individual elements.

You will learn about some important R functions that allow vectorization in the following sections.

### The ` *apply` family of functions

There are a family of functions called ` *apply` in R that vectorize your code for you. These functions are described in the help files (e.g. `?apply`).

For example, ` apply` can be used when you want to apply a function to the rows or columns of a matrix (and higher-dimensional analogues – remember arrays!). This is not generally advisable for data frames as it will first need to coerce the data frame to a matrix first.

Let us try using applying the same function to rows/colums of a matrix using `apply`.

$\star$ Type the following in a script file called `apply1.R`, save it to your ` Code` directory, and run it:_____no_output_____

<code>
## Build a random matrix
M <- matrix(rnorm(100), 10, 10)

## Take the mean of each row
RowMeans <- apply(M, 1, mean)
print (RowMeans) [1] -0.5084785  0.3905512 -0.4905091 -0.4291110  0.2184447  0.2383627
 [7] -0.5632035 -0.1124572 -0.6899271  0.4223239
## Now the variance
RowVars <- apply(M, 1, var)
print (RowVars) [1] 1.4307387 0.9763840 0.9974280 0.4736560 1.5676097 1.2434270 0.7080509
 [8] 1.4910468 1.1534264 0.9555595
## By column
ColMeans <- apply(M, 2, mean)
print (ColMeans) [1]  0.01831564 -0.18788193 -0.05712848 -0.39083201 -0.19890709 -0.50880697
 [7] -0.24618503 -0.02318715 -0.18383610  0.25444544

</code>

That was using `apply` on some of R's inbuilt functions. You can use apply to define your own functions. Let's try it.

$\star$ Type the following in a script file called `apply2.R`, save it to your ` Code` directory, and run it:_____no_output_____

<code>
SomeOperation <- function(v){ # (What does this function do?)
  if (sum(v) > 0){ #note that sum(v) is a single (scalar) value
    return (v * 100)
  }
  return (v)
}

M <- matrix(rnorm(100), 10, 10)
print (apply(M, 1, SomeOperation))             [,1]       [,2]        [,3]          [,4]       [,5]        [,6]
 [1,]   59.037730  1.0193006 -0.42406382 -150.31488779 104.486655  232.918824
 [2,]  149.079432 -1.0742137  0.16003235    6.47357573 -85.357863    9.165404
 [3,]  168.793246 -0.3790472  0.07660339  -14.88413915  61.046094   -1.494568
 [4,]  -72.407045  0.7915568  0.57535908    0.05333914  81.383348  123.537118
 [5,]   44.097495 -0.9483649  0.19647936  177.51480127 237.328482  -54.770861
 [6,] -257.769842 -0.2950781 -0.04929601  -37.38082848   7.315221  -49.281337
 [7,]  -78.735604 -0.8216683 -0.63592054  186.27053456 228.318262 -126.178703
 [8,]  115.460074 -0.3538057 -0.80311028   20.41902905  53.285407  108.196509
 [9,]    5.645586  0.8517810 -1.40567527   -9.35426251 -60.251690  -26.728866
[10,]  -24.678922  0.6002191  0.32448944  -24.35401475  25.634972  134.253731
             [,7]        [,8]        [,9]       [,10]
 [1,]  0.14155933 -202.816173  117.285373   99.385301
 [2,]  0.63138499  -58.652756 -122.694643 -153.806131
 [3,] -0.13631282    7.724292  175.594460   84.590716
 [4,] -0.03627231   42.484761  -44.486794  190.502526
 [5,]  1.35888505   22.887491   29.044510   45.574454
 [6,] -1.52877260  -23.205137   37.307916    2.558489
 [7,] -1.43995781   25.144986   77.105961  190.098920
 [8,] -1.81514392  139.239781   -7.564196   21.329770
 [9,]  0.67357108  -30.918732   73.772007   40.681019
[10,] -1.00334974  157.564039 -247.305894  -54.891711

</code>

Thus, the function `SomeOperation` takes as input `v`. Then if the sum of v is greater than zero, it multiplies that value by 100. So if `v` has positive and negative numbers, and the sum comes out to be positive, only then does it multiply all the values in `v` by 100 and return them.

There are many other methods: ` lapply`, ` sapply`, ` eapply`, etc. Each is best for a given data type. For example, ` lapply` is designed for R lists. Have a look at [this Stackoveflow thread](https://stackoverflow.com/questions/3505701/grouping-functions-tapply-by-aggregate-and-the-apply-family)
for some guidelines.

#### A vectorization example

Let's try an example of vectorization involving `lapply` and `sapply`. We will also learn how sampling random numbers works.

$\star$ Type the following blocks of code into a single script called `sample.R` and save in `code`.

First some functions:_____no_output_____

<code>
######### Functions ##########

## A function to take a sample of size n from a population "popn" and return its mean
myexperiment <- function(popn,n){
    pop_sample <- sample(popn, n, replace = FALSE)
    return(mean(pop_sample))
}

## Calculate means using a for loop without preallocation:
loopy_sample1 <- function(popn, n, num){
    result1 <- vector() #Initialize empty vector of size 1
    for(i in 1:num){
        result1 <- c(result1, myexperiment(popn, n))
    }
    return(result1)
}

## To run "num" iterations of the experiment using a for loop on a vector with preallocation:
loopy_sample2 <- function(popn, n, num){
    result2 <- vector(,num) #Preallocate expected size
    for(i in 1:num){
        result2[i] <- myexperiment(popn, n)
    }
    return(result2)
}

## To run "num" iterations of the experiment using a for loop on a list with preallocation:
loopy_sample3 <- function(popn, n, num){
    result3 <- vector("list", num) #Preallocate expected size
    for(i in 1:num){
        result3[[i]] <- myexperiment(popn, n)
    }
    return(result3)
}

## To run "num" iterations of the experiment using vectorization with lapply:
lapply_sample <- function(popn, n, num){
    result4 <- lapply(1:num, function(i) myexperiment(popn, n))
    return(result4)
}

## To run "num" iterations of the experiment using vectorization with lapply:
sapply_sample <- function(popn, n, num){
    result5 <- sapply(1:num, function(i) myexperiment(popn, n))
    return(result5)
}_____no_output_____
</code>

*Think about what each of these functions does*

Now generate a population:_____no_output_____

<code>
popn <- rnorm(1000) # Generate the population
hist(popn)_____no_output_____
</code>

Now run and time the different functions:_____no_output_____

<code>
n <- 20 # sample size for each experiment
num <- 1000 # Number of times to rerun the experiment

print("The loopy, non-preallocation approach takes:" )
print(system.time(loopy_sample1(popn, n, num)))

print("The loopy, but with preallocation approach takes:" )
print(system.time(loopy_sample2(popn, n, num)))

print("The loopy, non-preallocation approach on a list takes:" )
print(system.time(loopy_sample3(popn, n, num)))

print("The vectorized sapply approach takes:" )
print(system.time(sapply_sample(popn, n, num)))

print("The vectorized lapply approach takes:" )
print(system.time(lapply_sample(popn, n, num)))[1] "The loopy, non-preallocation approach takes:"
   user  system elapsed
  0.019   0.000   0.018
[1] "The loopy, but with preallocation approach takes:"
   user  system elapsed
  0.023   0.000   0.022
[1] "The loopy, non-preallocation approach on a list takes:"
   user  system elapsed
  0.021   0.000   0.021
[1] "The vectorized sapply approach takes:"
   user  system elapsed
  0.013   0.000   0.013
[1] "The vectorized lapply approach takes:"
   user  system elapsed
  0.011   0.000   0.012

</code>

$\star$ Run`sample.R` using `source` and/or `Rscript` and make sure it works._____no_output_____#### The `tapply` function

Let's look at `tapply`, which is particularly useful because it allows you to apply a function to subsets of a vector in a dataframe, with the subsets defined by some other vector in the same dataframe, usually a factor (this could be useful for pound hill data analysis that's coming up, for example).

This makes it a bit of a different member of the ` *apply` family. Try this:_____no_output_____

<code>
x <- 1:20 # a vector
x_____no_output_____
</code>

Now create a `factor` type variable (of the same length) defining groups:_____no_output_____

<code>
y <- factor(rep(letters[1:5], each = 4))
y_____no_output_____
</code>

Now add up the values in x within each subgroup defined by y:_____no_output_____

<code>
tapply(x, y, sum)_____no_output_____
</code>

```{note}
**The Factor data type**: R has a special data type called "factor". Different values in this data type are called "levels". This data type is used to identify "grouping variables" such as columns in a dataframe  that contain  experimental treatments. This is convenient for statistical analyses using one of the many plotting and statistical commands or routines available in R, which have been written to interpret the `factor` data type as such and use it to automatically compare subgroups in the data. More on this later, when we delve into analyses (including visualization) in R.
```_____no_output_____### Using ` by`

You can also do something similar to `tapply` with the `by` function, i.e., apply a function to a dataframe using some factor to define the subsets. Try this:

First import some data:_____no_output_____

<code>
attach(iris)
iris_____no_output_____
</code>

Now run the `colMeans` function (it is better for dataframes than just mean) on multiple columns:_____no_output_____

<code>
by(iris[,1:2], iris$Species, colMeans)_____no_output_____by(iris[,1:2], iris$Petal.Width, colMeans)_____no_output_____
</code>

### Using ` replicate`

The `replicate` function is useful to avoid a loop for function that typically involves random number generation (more on this below). For example:_____no_output_____

<code>
replicate(10, runif(5))_____no_output_____
</code>

That is, you just generated 10 sets (columns) of 5 uniformly-distributed random numbers (a 10 $\times$ 5 matrix). _____no_output_____### Using `plyr` and `ddply`

The `plyr` package combines the functionality of the `*apply` family, into a few handy functions. Look up the [web page](http://plyr.had.co.nz/).

In particular, ` ddply` is very useful, because for each subset of a data frame, it applies a function and then combines results into another data frame. In other words, "ddply" means: take a data frame, split it up, do something to it, and return a data frame. Look up [this](http://seananderson.ca/2013/12/01/plyr.html) and
[this](https://www.r-bloggers.com/transforming-subsets-of-data-in-r-with-by-ddply-and-data-table/)
for examples.    There you will also see a comparison of speed of `ddply` vs `by` at the latter web page; `ddply` is actually slower than other vectorized methods, as it trades-off compactness of use for some of the speed of vectorization! Indeed, overall functions in ` plyr` can be slow if you are working with very large datasets that involve a lot of subsetting (analyses by many groups or grouping variables).

The base ` *apply` functions remain useful and worth knowing even if you do get into ` plyr` or better still, `
dplyr`, which we will see in the [Data chapter](08-Data_R.ipynb).

## Practicals

### A vectorization challenge

The Ricker model is a classic discrete population model which was introduced in 1954 by Ricker to model recruitment of stock in fisheries. It gives the expected number (or density) $N_{t+1}$ of individuals in generation $t + 1$ as a function of the number of individuals in the previous generation $t$:

$$ N_{t+1}= N_t e^{r\left(1-\frac{N_t}{k}\right)} $$

Here $r$ is intrinsic growth rate and $k$ as the carrying capacity of the environment. Try this script (call it `Ricker.R` and save it to `code`) that runs it:_____no_output_____

<code>
Ricker <- function(N0=1, r=1, K=10, generations=50)
{
  # Runs a simulation of the Ricker model
  # Returns a vector of length generations

  N <- rep(NA, generations)    # Creates a vector of NA

  N[1] <- N0
  for (t in 2:generations)
  {
    N[t] <- N[t-1] * exp(r*(1.0-(N[t-1]/K)))
  }
  return (N)
}

plot(Ricker(generations=10), type="l")_____no_output_____
</code>

Now open and run the script `Vectorize2.R` (available on the TheMulQuaBio repository). This is the stochastic Ricker model (compare with the above script to see where the stochasticity (random error) enters). Now modify the script to complete the exercise given.
*Your goal is to come up with a solution better than mine!*

### Groupwork Practical on Vectorization

Implement the python versions of `Vectorize1.R`and ` Vectorize2.R` (call them `Vectorize1.py` and `Vectorize2.py` respectively). Then write a bash script that compares the computational speed of the four scripts. the bash script should display meaningful summary of the results in the terminal.       _____no_output_____## Errors and Debugging

As we learned in {ref}`this section of the Python Chapter I <Python:errors>`, dealing with errors is a fundamental programming task that never really goes away!

### Unit testing

We will not cover unit testing here, which we {ref}`did in the Python Chapter I <Python:errors>` (reab about the concept there). Unit testing in R is equally recommended if you are going to develop complex code and packages.

```{tip}
A very convenient tool for R unit testing is [`testthat`](https://testthat.r-lib.org).
```

In addition, you can use other "[defensive programming](https://en.wikipedia.org/wiki/Defensive_programming)" methods in R to keep an eye on errors that might arise in special circumstances in an complex program. A good option for this is to convert warnings to errors using the [`stopifnot()`](https://stat.ethz.ch/R-manual/R-devel/library/base/html/stopifnot.html). This is a bit like `try` (which we will cover below, and we also covered in {ref}`Python Chapter I <Python:errors>`). _____no_output_____### Debugging

Once you have found an error, you need to debug it. Assuming you have an idea about where (or the general area) in your code the error is, a simple way to debug is to use the  `browser()` function.

The `browser()` function allows you to insert a breakpoint in your script (where the running of the script is stopped) and then step through your code. Place it within your script at the point you want to examine local variables  (e.g. inside a `for` loop).

Let's look at an example usage of `browser()`.

$\star$ Type the following in a script file called `browse.R` and save in `code`:

```R
Exponential <- function(N0 = 1, r = 1, generations = 10){
  # Runs a simulation of exponential growth
  # Returns a vector of length generations

  N <- rep(NA, generations)    # Creates a vector of NA

  N[1] <- N0
  for (t in 2:generations){
    N[t] <- N[t-1] * exp(r)
    browser()
  }
  return (N)
}

plot(Exponential(), type="l", main="Exponential growth")
```
The script will be run till the first iteration of the `for` loop and the console will enter the browser mode, which looks like this:

```R
Browse[1]>
```

Now, you can examine the variables present at that point. Also, at the browser console, you can enter expressions as normal, or use a few particularly useful debug commands (similar to the Python debugger):

* `n`: single-step
* `c`: exit browser and continue
* `Q`: exit browser and abort, return to top-level.

```{tip}
We will not cover advanced debugging here as we did {ref}`in Python Chapter I <Python:errors>`, but look up `traceback()` (to find where the errors(s) are when a program crashes), and `debug()` (to debug a whole function).
```_____no_output_____### "Catching" errors

Often, you don't know if a simulation or a R function will work on a particular data or variable, or a value of a variable (can happen in many statistical functions).

Indeed, as most of you must have already experienced by now, there can be frustrating, puzzling bugs in programs that lead to mysterious errors. Often, the error and warning messages you get are un-understandable, especially in R!

Rather than having R throw you out of the code, you would rather catch the error and keep going. This can be done by using the `try` keyword.

Lets try `try`!

First, let's write a function:_____no_output_____

<code>
doit <- function(x){
    temp_x <- sample(x, replace = TRUE)
    if(length(unique(temp_x)) > 30) {#only take mean if sample was sufficient
         print(paste("Mean of this sample was:", as.character(mean(temp_x))))
        }
    else {
        stop("Couldn't calculate mean: too few unique values!")
        }
    }_____no_output_____
</code>

This function runs a simulation that involves sampling from a synthetic population with replacement and takes its mean, but *only* if at least 30 unique samples are obtained. Note that we have used another keyword here: `stop()`. Pay close attention to `sample()`, and `stop()` in the above script (check out what they are by using R help or searching online).

Now, generate your population:_____no_output_____

<code>
popn <- rnorm(50)

hist(popn)_____no_output_____
</code>

Now try running it using `lapply` as you did above, repeating the sampling exercise 15 times:_____no_output_____

<code>
lapply(1:15, function(i) doit(popn))[1] "Mean of this sample was: -0.0819495413387692"
[1] "Mean of this sample was: 0.0239552450301257"

</code>

Your result will be different than this (you might get between 0 - 15 samples; here I got only two means). But in most cases, the script will fail because of the `stop` command (on line 7 of the above function) at some iteration, returning less than the requested 15 mean values, followed by an error stating that the mean could not be calculated because too few unique values were sampled._____no_output_____Now try doing the same using `try`:_____no_output_____

<code>
result <- lapply(1:15, function(i) try(doit(popn), FALSE))[1] "Mean of this sample was: -0.121221064404979"
Error in doit(popn) : Couldn't calculate mean: too few unique values!
[1] "Mean of this sample was: -0.173759374437533"
Error in doit(popn) : Couldn't calculate mean: too few unique values!
[1] "Mean of this sample was: 0.00274247326407021"
[1] "Mean of this sample was: -0.193057628828299"
Error in doit(popn) : Couldn't calculate mean: too few unique values!
[1] "Mean of this sample was: -0.259135797305263"
[1] "Mean of this sample was: -0.177113866182732"
[1] "Mean of this sample was: -0.15086835618164"
Error in doit(popn) : Couldn't calculate mean: too few unique values!
[1] "Mean of this sample was: -0.177809532764058"
[1] "Mean of this sample was: -0.0554061223415571"
[1] "Mean of this sample was: -0.144011561341331"
[1] "Mean of this sample was: -0.186395187133434"

</code>

In this run, you again asked for the means of 15 samples, and again you (most likely) got less than that (see the above example output), but without any error! The `FALSE` modifier for the `try` command suppresses any error messages, but `result` will still contain them so that you can inspect them later (see below). *Please don't forget to  check the help on inbuilt R commands like `try`.*

The errors are stored in the object `result`:_____no_output_____

<code>
class(result)_____no_output_____
</code>

This is a list that stores the result of each of the 15 runs, including the ones that ran into an error. Have a look at it:   _____no_output_____

<code>
result_____no_output_____
</code>

That's a lot output! But basically it tells you which runs ran into and error and why (no surprises here, of course!). You can also store the results "manually" by using a loop to do the same: _____no_output_____

<code>
result <- vector("list", 15) #Preallocate/Initialize
for(i in 1:15) {
    result[[i]] <- try(doit(popn), FALSE)
    }Error in doit(popn) : Couldn't calculate mean: too few unique values!
[1] "Mean of this sample was: 0.279031226403986"
Error in doit(popn) : Couldn't calculate mean: too few unique values!
Error in doit(popn) : Couldn't calculate mean: too few unique values!
[1] "Mean of this sample was: -0.0219088573358361"
Error in doit(popn) : Couldn't calculate mean: too few unique values!
Error in doit(popn) : Couldn't calculate mean: too few unique values!
Error in doit(popn) : Couldn't calculate mean: too few unique values!
[1] "Mean of this sample was: 0.109154079397255"
[1] "Mean of this sample was: 0.259256957371067"
Error in doit(popn) : Couldn't calculate mean: too few unique values!
Error in doit(popn) : Couldn't calculate mean: too few unique values!
[1] "Mean of this sample was: 0.211124799765074"
[1] "Mean of this sample was: -0.015175901793773"
[1] "Mean of this sample was: 0.262309113486109"

</code>

Now have a look at the new `result`; it will have similar content as the one you got from using `lapply`.

```{tip}
Also check out `tryCatch()` as an alternative to `try()`.
```_____no_output_____$\star$ Type the above blocks of code illustrating `try` into a script file called `try.R` and save in `code`. The script, when `source`d, should run without errors (that is, don't run the sampling function without `try`!). _____no_output_____## Packages

The big strength of R is that users can easily build and share packages through [cran.r-project.org](https://cran.r-project.org).

There are R packages for practically all statistical and mathematical analysis you might conceive, so check them out before reinventing the wheel. Visit [cran.r-project.org](https://cran.r-project.org) and go to packages to see a list and a brief description.

In Windows and Macs and Linux, you can install a package within R by using the `install.packages()`command.

For example:

```r
install.packages(c("tidyverse"))
```
You can also install multiple packages by concatenating their names: `install.packages(c("pack1","pack2","pack3"))`
(three packages in this hypothetical example)

In Ubuntu, you will have to launch a ` sudo R`session to get `install.packages()`to work properly. Otherwise, you be forced to install the package in a non-standard location on your drive that does not require `sudo` privileges (nor recommended).

You can also use the RStudio GUI to install packages using your mouse and menu.

In Ubuntu, you can also use the bash terminal:

```bash
sudo apt install r-cran-tidyverse
```

$\star$ Go ahead and install `tidyverse` if you don't have it yet. We will be using it [soon](./08-Data_R.ipynb)!

### Building your own

You can combine your code, data sets and documentation to make a *bona fide* R package. You may wish to do this for particularly large projects that you think will be useful for others. Read the [*Writing R Extensions*](https://cran.r-project.org/doc/manuals/r-release/R-exts.html) manual and see *package.skeleton* to get started.

The R tool set [EcoDataTools](https://github.com/DomBennett/EcoDataTools) and the package [cheddar](https://github.com/quicklizard99/cheddar) were written by Silwood Grad Students!_____no_output_____## Practicals

### Autocorrelation in weather

This Practical assumes you know about correlation coefficients and p-values.

Your goal is to write an R script that will help answer the question: *Are temperatures of one year significantly correlated with the next year (successive years), across years in a given location*? For this, you need to calculate the correlation between $n-1$ pairs of years, where $n$ is the total number of years. However, you can't use the standard p-value calculated for a correlation coefficient, because measurements of climatic variables in successive time-points in a time series (successive seconds, minutes, hours, months, years, etc.) are *not independent*. The general guidelines are:

 * Make a new script named ` TAutoCorr.R`, and save in ` code`directory
 * At the start of the script, load and examine and plot `KeyWestAnnualMeanTemperature.Rdata`, using ` load()`. This is the temperature in Key West, Florida for the 20th century.
 * Now extend the script to:
     * Compute the appropriate correlation coefficient between successive years and store it (look at the help file for `cor()`
     * Repeat this calculation 10000 times by – randomly permuting the time series, and then recalculating the correlation coefficient for each randomly permuted year sequence and storing it.
     *Hint*: you can use the `sample` function that we learned about). Read the help file for this function and experiment with it.
     *  Then calculate what fraction of the correlation coefficients from the previous step were greater than that from step 1 (this is your approximate p-value).
 * *How do you interpret these results? Why?* Present your results and their interpretation in a pdf document written in $\LaTeX$ (please include the the document's source code as well).

### Groupwork Practical on Tree Heights

The goal of this practical is to make the `TreeHeight.R` Groupwork script more general, so that it could be used for other datasets, not just `trees.csv`.

Guidelines:

  * Write another R script called `get_TreeHeight.R` that takes a csv file name from the command line (e.g., `get_TreeHeight.R Trees.csv`) and outputs the result to a file just like ` TreeHeight.R`above, but this time includes the input file name in the output file name as ` InputFileName_treeheights.csv`. Note that you will have to strip the `.csv`or whatever the extension is from the filename, and also ` ../`etc., if you are using relative paths. (Hint: Command-line parameters are accessible within the R running environment via `commandArgs()` — so `help(commandArgs)` might be your starting point.)
  * Write a Unix shell script called ` run_get_TreeHeight.sh` that tests ` get_TreeHeight.R`. Include `trees.csv`as your example file. Note that     ` source`will not work in this case as it does not allow scripts with arguments to be run; you will have to use ` Rscript`instead.

### Groupwork Practical  on Tree Heights  2

Assuming you have already worked through [Python Chapter I](./05-Python_I.ipynb), write a Python version of `get_TreeHeight.R` (call it ` get_TreeHeight.py`). Include a test of this script into ` run_get_TreeHeight.sh`.

### Practicals wrap-up

* Review and make sure you can run all the commands, code fragments, and named scripts we have built till now and get the expected outputs.

* Annotate/comment your code lines as much and as often as necessary using #.

* Keep all files organized in ` code`, ` data`and `results` directories

* (If you are required to as part of your course) `git add`, `commit` and `push` all your code and data from this chapter to your git repository by the given deadline._____no_output_____## Readings and Resources

Check the readings under the R directory in the TheMulQuaBio repository. Also, search online for "R tutorial", and plenty will pop up. Choose ones that seem the most intuitive to you. * *Remember, all R packages come with pdf guides/documentation!*

### General

* The Use R! series (all yellow books) by Springer are really good. In particular, consider: "A Beginner's Guide to R", "R by Example", "Numerical Ecology With R", "ggplot2" (coming up in the [Visualization Chapter](08-Data_R.ipynb), "A Primer of Ecology with R", "Nonlinear Regression with R", and "Analysis of Phylogenetics and Evolution with R".
* For more focus on dynamical models: Soetaert & Herman. 2009 "A practical guide to ecological modelling: using R as a simulation platform".
* There are excellent websites besides cran. In particular, check out
[statmethods.net](https://www.statmethods.net/) and the [R Wiki](https://en.wikibooks.org/wiki/R_Programming).

### Debugging

* `testthat`(https://testthat.r-lib.org/): Unit Testing for R by Hadley Wickham (also developer of the `tidyverse` package, including `ggplot`).
* [Notes on debugging in R by Wickham](https://adv-r.hadley.nz/debugging.html)
* A good [overview of debugging methods in R with examples](https://data-flair.training/blogs/debugging-in-r-programming)
*  [An introduction to the Interactive Debugging Tools in R](http://www.biostat.jhsph.edu/~rpeng/docs/R-debug-tools.pdf) by Roger D Peng.
* If you are using RStudio, [see this](https://support.rstudio.com/hc/en-us/articles/205612627-Debugging-with-RStudio).

### Sweave and knitr

Sweave and knitr are tools that allows you to write your Dissertation Report or some other document such that it can be updated automatically if data or R analysis change. Instead of inserting a prefabricated graph or table into the report, the master document contains the R code necessary to obtain it. When run through R, all data analysis output (tables, graphs, etc.) is created on the fly and inserted into a final document, which can be written using $\LaTeX$, LyX, HTML, or Markdown. The report can be automatically updated if data or analysis change, which allows for truly reproducible research.

* For a practical intro to Sweave and knitr, see [this](https://support.rstudio.com/hc/en-us/articles/200552056-Using-Sweave-and-knitr) and [this](http://yihui.name/knitr/)._____no_output_____

STAR bioinformatics evolution biology phylogenetics ecology