Find us on GitHub

USDA-ARS Southeast Area, Stoneville, MS

Jan 19-20, 2016

9:00 am - 4:30 pm

Instructors: John Moreau, Zhuo Fu

Helpers: Mathew Seymour

General Information

Data Carpentry: develops and teaches workshops on the fundamental data skills needed to conduct research. Our mission is to provide researchers high-quality, domain-specific training covering the full lifecycle of data-driven research. Data Carpentry is a sibling organization of Software Carpentry. Where Software Carpentry teaches best practices in software development, our focus is on the introductory computational skills needed for data management and analysis in all domains of research. Our lessons are domain specific, from life and physical sciences to social science and build on the existing knowledge of learners to enable them to quickly apply skills learned to their own research. For more information on what we teach and why, please see our paper "Best Practices for Scientific Computing".

Who: The course is aimed at scientists (Ph. D.), IT specialists and technicians (BS/MS level). Attendees are learners who have little to no prior computational experience. We create a friendly environment for learning to empower researchers and enable data driven discovery.

Where: Charles W. Capps, Jr. Center classroom, Jamie Whitten Delta States Research Center, 141 Experiment Station Road, Stoneville, MS. (Get directions with OpenStreetMap or Google Maps.)

Requirements: Participants must bring a laptop with a few specific software packages installed (listed below). They are also required to abide by Software Carpentry's Code of Conduct.

Contact: Please mail op.perera@ars.usda.gov for more information.


Schedule

Day 1

Arrival Pre-workshop Survey
09:00 Data organization in spreadsheets and Open refine
10:30 Coffee
12:00 Lunch break
13:00 R basics
14:30 Coffee
16:00 Wrap-up

Day 2

09:00 Data analysis and visualization in R
10:30 Coffee
12:00 Lunch break
13:00 Data management with SQL
14:30 Coffee
16:00 Wrap-up
Dismissal Post-workshop Survey

Syllabus

Data organization in spreadsheets

  • Introduction
  • Formatting data
  • Common formatting problems
  • Dates as data
  • Quality control
  • Exporting data
  • Data Format Caveats

Data cleaning with OpenRefine

  • Getting Started with OpenRefine
  • Working with OpenRefine - faceting, clustering & splitting
  • Scripts
  • Saving and Exporting Projects and Files

Data management with SQL

  • Introduction to SQL
  • Basic queries
  • Aggregation
  • Joins and aliases

Data analysis and visualization in R

  • Before we start
  • Introduction to R
  • Starting with data
  • Data visualisation with ggplot2
  • Aggregating and analyzing data with dplyr
  • R and SQL

Setup

To participate in a Data Carpentry workshop, you will need access to the software described below. In addition, you will need an up-to-date web browser.

We maintain a list of common issues that occur during installation as a reference for instructors that may be useful on the Configuration Problems and Solutions wiki page.

R

R is a programming language that is especially powerful for data exploration, visualization, and statistical analysis. To interact with R, we use RStudio.

Windows

Install R by downloading and running this .exe file from CRAN. Also, please install the RStudio IDE.

Mac OS X

Install R by downloading and running this .pkg file from CRAN. Also, please install the RStudio IDE.

Linux

You can download the binary files for your distribution from CRAN. Or you can use your package manager (e.g. for Debian/Ubuntu run sudo apt-get install r-base and for Fedora run sudo yum install R). Also, please install the RStudio IDE.

SQLite

SQL is a specialized programming language used with databases. We use a simple database manager called SQLite in our lessons.

Windows

The Software Carpentry Windows Installer installs SQLite for Windows. If you used the installer to configure nano, you don't need to run it again.

Mac OS X

SQLite comes pre-installed on Mac OS X.

Linux

SQLite comes pre-installed on Linux.