A brief introduction to using R for high-performance computing

Agenda

1. High-Performance Computing: An overview

2. Parallel computing in R

3. Extended examples

High-Performance Computing: An overview

Loosely, from R’s perspective, we can think of HPC in terms of two, maybe three things:

1. Big data: How to work with data that doesn’t fit your computer

2. Parallel computing: How to take advantage of multiple core systems

3. Compiled code: Write your own low-level code (if R doesn’t has it yet…)

(Checkout CRAN Task View on HPC)

Big Data

• Buy a bigger computer/RAM memory (not the best solution!)

• Use out-of-memory storage, i.e., don’t load all your data in the RAM. e.g. The bigmemory, data.table, HadoopStreaming R packages

• Efficient algorithms for big data, e.g.: biglm, biglasso

• Store it more efficiently, e.g.: Sparse Matrices (take a look at the dgCMatrix objects from the Matrix R package)

Parallel computing

Flynn’s Classical Taxonomy (Blaise Barney, Introduction to Parallel Computing, Lawrence Livermore National Laboratory)

We will be focusing on the Single Instruction stream Multiple Data stream

Parallel computing

Serial vs Parallel

source: Blaise Barney, Introduction to Parallel Computing, Lawrence Livermore National Laboratory

Parallel computing

source: Blaise Barney, Introduction to Parallel Computing, Lawrence Livermore National Laboratory

Some vocabulary for HPC

In raw terms

• Supercomputer: A single big machine with thousands of cores/gpus.

• High Performance Computing (HPC): Multiple machines within a single network.

• High Throughput Computing (HTC): Multiple machines across multiple networks.

You may not have access to a supercomputer, but certainly HPC/HTC clusters are more accessible these days, e.g. AWS provides a service to create HPC clusters at a low cost (allegedly, since nobody understands how pricing works)

GPU vs CPU

NVIDIA Blog

• Why use OpenMP if GPU is suited to compute-intensive operations? Well, mostly because OpenMP is VERY easy to implement (easier than CUDA, which is the easiest way to use GPU).

Let’s think before we start…