Remote PyTorch Execution

The workstations in the JMU CS labs have NVIDIA GPUs, which train convolutional networks many times faster than a laptop CPU. You can use them remotely, either through VS Code or from a terminal.

A few things apply either way:

Possible lab computers are L25001 through L25030. Pick one at random so that the class spreads out. Running nvidia-smi on the lab machine shows whether anyone else is already using its GPU; if so, try another machine.
Your home directory on the lab machines is the same as on stu, so your files are available on every lab machine.
The course environment on the lab machines is env445gpu. It has the same packages as env445, but with GPU support. When you run cifar_challenge.py, the first line of output should be Using cuda device. The script’s image windows can’t open on a remote machine, so they are skipped. Use TensorBoard to watch training.

Option 1: VS Code

One-time setup

In VS Code, install the Remote - SSH extension (published by Microsoft).
Open the Command Palette (Ctrl+Shift+P, or Cmd+Shift+P on a Mac), run Remote-SSH: Open SSH Configuration File…, and pick the file in your own home folder (~/.ssh/config on Mac and Linux, C:\Users\YOUR_NAME\.ssh\config on Windows). Add these lines:
```
Host cs445gpu
    HostName L25017.cs.jmu.edu
    User EID
    ProxyJump EID@stu.cs.jmu.edu
```
Replace EID (twice) with your JMU eID, and replace L25017 with a lab machine you picked at random. The ProxyJump line connects through stu, so the same settings work on and off campus. To switch machines later, just edit the HostName line.

Connecting

Run Remote-SSH: Connect to Host… from the Command Palette and choose cs445gpu. Enter your password when prompted; you will be asked twice, once for stu and once for the lab machine. If VS Code asks for the remote platform, choose Linux.

The first connection takes a minute while VS Code installs its server in your home directory. Once you are connected, the bottom left corner of the window shows SSH: cs445gpu.
Use File > Open Folder… to open the folder that holds your code. To copy files from your own computer, drag them into the VS Code Explorer.
Extensions run on the remote machine, so install the Python extension again there: the Extensions view shows an Install in SSH: cs445gpu button.
Run Python: Select Interpreter, choose Enter interpreter path…, and enter:
```
/cs/shr/cs445/miniconda3/envs/env445gpu/bin/python
```
VS Code remembers this choice for the folder.

Training

For a quick run, the usual Run Python File button works.
For anything long, use a terminal (Terminal > New Terminal, which opens a shell on the lab machine) and run inside tmux so that training keeps going if your connection drops or you close VS Code:
```
tmux
source /cs/shr/cs445/miniconda3/bin/activate
conda activate env445gpu
python cifar_challenge.py
```
If you get disconnected, open a new terminal and run tmux attach to get back to your training run.

TensorBoard

In a second VS Code terminal, activate the environment as above and run:
```
tensorboard --logdir runs --port 6006
```
VS Code notices the address that TensorBoard prints and forwards the port to your computer automatically. Click Open in Browser in the pop-up, or open the Ports tab next to the terminal and click the globe icon next to port 6006.

If someone else on the same machine is already using port 6006, TensorBoard will fail to start. Use another port, such as 6007; VS Code forwards it the same way.

Option 2: Terminal

Connect to a lab machine with port forwarding (the port forwarding is for TensorBoard):

ssh -L 16006:127.0.0.1:6006 EID@LAB_COMPUTER_ID.cs.jmu.edu

You can do this from off campus as well:

ssh -L 16006:127.0.0.1:6006 -J EID@stu.cs.jmu.edu EID@LAB_COMPUTER_ID.cs.jmu.edu

To copy files from your own computer, use scp (to stu or to the lab machine; it’s the same home directory).
Once you are logged in, it can be handy to access multiple terminals on the remote machine without logging in multiple times. The tmux tool (terminal multiplexer) makes this possible. Once you have activated tmux, Ctrl-b c creates a new window and Ctrl-b n moves to the next window. Training also keeps running inside tmux if your connection drops; log back in and run tmux attach to return to it.

To activate the course environment:

source /cs/shr/cs445/miniconda3/bin/activate
conda activate env445gpu

To start TensorBoard:
```
tensorboard --logdir runs --port 6006
```
Now you can view TensorBoard in your local browser at http://localhost:16006.

If someone else on the same machine is already using port 6006, TensorBoard will fail to start. Pick another port, such as 6007, and use it both in the ssh -L command (16006:127.0.0.1:6007) and in the --port option.