Could your Gateway or other computational work benefit from the ability to concurrently run hundreds or thousands of independent computations, for free? The Open Science Grid (OSG) facilitates distributed high-throughput computing (dHTC) via a partnership of national labs, universities, and other organizations who contribute and share computing capacity for use by researchers across and beyond the United States. Individual researchers, institutions, or multi-institutional collaborations can access OSG via local submission points or through the OSG Connect service (freely available to U.S. academic, government, and non-profit researchers).
The OSG is perhaps the most scalable resource for computational work that can be run as numerous independent jobs, making it an ideal fit for many existing and future gateways. Individual users regularly occupy thousands of CPU cores across jobs when each runs for less than a day on a single core, achieving greater parallelization than on any individual cluster. Computational work runs in the OSG via the HTCondor job scheduler, which can be integrated with numerous workflow tools (Pegasus, TOIL, CCTools, HTCondor's own DAG Manager, etc.) and made interchangeable with submission to other schedulers. Available capacity includes not only the significant CPU and data storage in OSG, but also GPUs and seamless expansion to cloud resources. The OSG offers multiple services in support of the Science Gateway Community, including consulting on workflow design/optimization, data handling, and software solutions via help@opensciencegrid.org.
During this 3-hour tutorial, you'll learn to run examples of large HTC workloads and multi-step workflows via the OSG Connect service, including discussion of the support available to gateway developers through OSG. If time permits, the OSG User Support team will also help you run your own sample workload on OSG.
Skill Level : Intermediate
Prerequisites : Familiarity with the unix command-line; familiarity with SSH connection to a remote server
Requirements : Participants will need to bring a laptop with WiFi and SSH capabilities (e.g. Putty for Windows or Terminal for Mac/Linux laptops); learning accounts will be distributed to participants; no additional software will need to be installed ahead of the tutorial.
