1. 01-Introduction

header.png

Author: Tim Kreuzer

Index

Jupyter-JSC

About

Jupyter-JSC is designed to provide the rich high performance computing (HPC) ecosystem to the world's most popular software: web browsers. JupyterLab is a web-based interactive development environment for Jupyter notebooks, code, and data. JupyterLab is flexible to support a wide range of workflows in data science, scientific computing, and machine learning.

Setup

Webservice

Jupyter-JSC is a customized JupyterHub running on the HDF-Cloud. It is deployed in a kubernetes cluster.
There are four basic components to serve Jupyter-JSC:

  • JuypterHub
    • Interacts with the user. Handles authentication and communication between JupyterHub and the user's services.
  • Backend
    • Starting / stopping services for the user on any system. Called by JupyterHub Spawner. Communicating with UNICORE and UserLab Manager.
  • Tunneling
    • Secure port forwarding between JupyterHub and the user's service. Required to reach compute nodes without access to the internet.
  • UserLab Manager
    • Starting / stopping services on the cloud systems, where UNICORE is not an option.

Authentication

If you visit Jupyter-JSC and hit the Login button, you will be redirected to our authentication service. This webservice, called Unity-IdM, is connected to the JSC WebLDAP and the Helmholtz AAI. You can use your JSC account, also used at JuDoor, GitLab and more, or the Helmholtz AAI, to sign in with your local identity provider (IdP). After your successful login, you will be redirected to Jupyter-JSC.
jupyter-jsc-authentication.png

2-Factor-Authorization

Authorization

System access

Jupyter-JSC gets the information about the users HPC accounts and projects directly from JuDoor.
While the cloud systems are available for everyone, the resources and quotas may differ for each virtual organization. Both are also dependent on the use of the system and may change. You can see your current quota while your Service is starting.

Virtual organizations

Jupyter-JSC supports multiple virtual organizations (VO) in Jupyter-JSC. This allows us to offer specific resources and quotas for different communities or workshops. If you have multiple VOs you can choose your active VO in the top right corner of the website.

Service Start

Jupyter-JSC uses UNICORE to start jobs on the HPC-Systems. These jobs contain all the information to start your service.
On Cloud-Systems, where UNICORE is not available, Jupyter-JSC uses an internal solution (UserLab Manager) to start a Kubernetes Pod for each of the users services.
During the startup process of the users service, Jupyter-JSC creates a secure ssh-connection between JupyterHub and the users service.

jupyter-jsc-start-01.png