Remote editing with VS Code
Visual Studio Code is a widely-used, cross-platform Integrated Development Environment (IDE) that supports numerous programming languages and offers a vast array of extensions to enhance its functionality.
One notable extension enables development on remote machines via SSH, providing integrated access to a file explorer, terminal, and text editor on the configured remote system. This makes it a strong alternative to JupyterLab as a remote editor for e.g., the ETROFARM Slurm cluster.
Configuring Visual Studio Code extensions for working on ETROFARM Slurm cluster
Open the Extensions tab using the corresponding icon in the left toolbar.
Search for “SSH” and subsequently select the topmost “Remote – SSH” extension. Install this extension.
Depending on your programming language of choice, you might also be interested in extensions such as “Python”, “Python Debugger”, “Ruff” (a Python code linter) etc.
With the “Remote – SSH” extension installed, a new tab “Remote explorer” has been added to the left toolbar.
Add a new remote by pressing the + icon.
When asked for the SSH Connection Command:
ssh <username>@etroflock.etrovub.be
Secondly, it will ask where to store this information. This can be the default option.
The etroflock.etrovub.be remote has been created. Time to connect by triggering one of the 2 corresponding buttons.
During a brief instant the option will be displayed to edit the configuration. If you have missed it you can find this file via the gear button next to remote explorer – remote tunnels – ssh
Your config file should look like
Host etroflock.etrovub.be
HostName etroflock.etrovub.be
User jdoe
IdentityFile C:\users\jdoe\.ssh\id_rsaIf you have no experience with encryption you can e.g. use a rsa 2048 type of key. Please make sure you are using a private key in openssh format.
Upon our first connection attempt, it requests the platform of the remote host, being the Slurm cluster’s login node we are connecting to. This is a Linux machine.
It will also ask to confirm the SSH public key credential of the server.
We are now connected to the remote server. This can be seen in the Remote explorer tab as well as in the left corner of the bottom toolbar.
Configuring the file explorer and text editor on the remote system
Time to open our file explorer via the “Explorer” tab in the left toolbar (Ctrl + Shift + E). Press the “Open Folder” button. It should by default suggest to open your home folder on the Slurm cluster (currently on /FARM/<username>).
If prompted, confirm that the remote server is (again) a Linux platform. Lastly, confirm that you trust the authors of the files in this folder as this is your own home folder.
Congratulations! Your remote file explorer and text editor on the Slurm cluster is now operational.
Remote terminal and running code on the remote system
A remote terminal can be opened using Terminal -> New Terminal in the top toolbar, or via the Ctrl + Shift + ` shortcut.
An interactive terminal session is opened on the remote host as if it was a PuTTY (or other) SSH session.
Running code remotely with VScode (ideally on Slurm compute nodes)
With the remote file explorer, remote text editor and remote terminal sessions available, it is a logical next step to focus on running our code remotely on the machine. Luckily, this is typically as straightforward as pressing the “Run file” button on the active (Python) file.
We can observe that the code has indeed been executed on the remote machine. However, the configured remote machine is ETROflock, ETRO’s Slurm cluster’s login server that is scarce in compute resources and lacks and GPU’s.
Running our code on a Slurm compute node is more complicated as it involves requesting a Slurm job. This is currently a manual process but we are investigating if this can be automated in VScode by using a custom launch script.
For now, there are two possibilities to run code on the Slurm cluster from within VScode:
A Slurm job can be requested that immediately runs the code until completion (or an error or timeout). This is the recommended default approach for running Slurm jobs.
E.g. the same test.py code is run as a Slurm job by using the srun command.
We observe that the first command is run on ETROFLOCK (the Slurm login node) because that is immediately launched on the remote system. The second command is scheduled as a Slurm job and is run on ETROFARM (a Slurm compute node).
The second possibility involves starting a Slurm job with an interactive shell. Once this interactive shell is running on a compute node, we can manually launch the desired code within that shell. This solution might be preferred when developing and testing the functionality of the code as a Slurm job (and potential queue) must only be requested once per session.
Launching a Slurm job with interactive shell is possible using the following srun command parameters:
srun –pty bash -i
E.g. the same test.py code is run after an interactive shell has been requested using the srun command and has started.
Observe that after requesting the slurm job with interactive terminal we change from “steffen@ETROFLOCK” to “steffen@ETROFARM”. This indicates our interactive shell is indeed running on a compute node instead of the login node.
Executing the test.py script from within this shell again confirms that the code is indeed run on the compute node with hostname ETROFARM.
When finished, one should use the “exit” command. This closes the interactive shell and terminates the Slurm job, thereby releasing the allocated resources for new jobs.
