Introduction to High Performance Computing

The Jackson Laboratory

March 6, 2020

9:00 am - 3:00 pm

Instructors: Jason Macklin, Aaron McDivitt, Matt Bradley

Helpers: Sue McClatchy

General Information

This workshop introduces high performance computing using the Jackson Laboratory’s computing cluster. The goal is to teach novice programmers to use powerful tools and computing resources, and to engage in best practices for using these resources. The emphasis of this workshop is to give attendees a strong foundation in the fundamentals of the bash shell and to teach best practices for scientific computing in high performance computing environments. By the end of this workshop, participants will be able to:

Who: The course is aimed at graduate students and other researchers at the Jackson Laboratory. For IT security reasons, external participants will not be admitted. You can attend in classrooms in Bar Harbor or Farmington, however, remote participation from desks, home, or away will not be supported. You don't need to have any previous knowledge of the tools that will be presented at the workshop.

Where: Holt Conference Room, 10 Discovery Dr, Farmington CT | Bioinformatics Training Room, 600 Main St, Bar Harbor ME. Get directions with OpenStreetMap or Google Maps.

When: March 6, 2020. Add to your Google Calendar.

Requirements: Participants must bring a laptop with a Mac or Windows operating system (not a tablet, Chromebook, etc.). Windows users should have specific software installed (listed below).

Accessibility: We are committed to making this workshop accessible to everybody. The workshop organizers have checked that:

Materials will be provided in advance of the workshop and large-print handouts are available if needed by notifying the organizers in advance. If we can help making learning easier for you (e.g. sign-language interpreters, lactation facilities) please get in touch (using contact details below) and we will attempt to provide them.

Contact: Please email susan.mcclatchy@jax.org for more information.


Collaborative Notes

We will use this collaborative document for chatting, taking notes, and sharing URLs and bits of code.



Schedule

Friday Mar 6

09:00 Automating Tasks with the Unix Shell
10:30 Morning break
11:00 Automating Tasks with the Unix Shell (Continued)
12:00 Lunch break
13:00 Containers with Singularity
14:50 Wrap-up
15:00 END

Syllabus

The Unix Shell

  • Files and Directories
  • History and Tab Completion
  • Pipes and Redirection
  • Looping Over Files
  • Creating and Running Shell Scripts
  • Finding Things
  • Reference...

High Performance Computing

  • Connect to a cluster
  • Use a scheduler to submit and manage jobs
  • Transfer files
  • Use software through environment modules
  • Reference...

Containers with Singularity

  • Introduction to Singularity containers
  • Interactive Exploration of Containers
  • Build Your Own Containers
  • Reference...

Setup

To participate in a Software Carpentry workshop, you will need access to the software described below. In addition, you will need an up-to-date web browser.

We maintain a list of common issues that occur during installation as a reference for instructors that may be useful on the Configuration Problems and Solutions wiki page.

The Bash Shell

Bash is a commonly-used shell that gives you the power to do simple tasks more quickly.

  1. Download and install MobaXterm or PuTTY.
  2. If your "HOME" environment variable is not set (or you don't know what this is):
    1. Open command prompt (Open Start Menu then type cmd and press [Enter])
    2. Type the following line into the command prompt window exactly as shown:

      setx HOME "%USERPROFILE%"

    3. Press [Enter], you should see SUCCESS: Specified value was saved.
    4. Quit command prompt by typing exit then pressing [Enter]

The default shell in some versions of macOS is Bash, and Bash is available in all versions, so no need to install anything. You access Bash from the Terminal (found in /Applications/Utilities). See the Git installation video tutorial for an example on how to open the Terminal. You may want to keep Terminal in your dock for this workshop.

To see if your default shell is Bash type echo $SHELL in Terminal and press the enter/return key. If the message printed does not end with '/bash' then your default is something else and you can run Bash by typing bash.

The default shell is usually Bash and there is usually no need to install anything.

To see if your default shell is Bash type echo $SHELL in a terminal and press the enter/return key. If the message printed does not end with '/bash' then your default is something else and you can run Bash by typing bash.