HPC/Network Access/SSH Tunnel Setup and Use

The following sections document how to establish ssh tunneling on machines running OpenSSH or close relatives. OpenSSH is available on all major operating systems: Linux, macOS, Windows, and on mobile.

Manual setup

On your computer, run the following commands:

0. (Windows only) Start your X11 server if you have one.

1. Open a terminal window (such as Mac Terminal, Cygwin, PuTTY, or xterm).

2. Type:

ssh -v -L 33301:carbon:22  username@mega.cnm.anl.gov

Do not type any further commands in this window.

3. Minimize that window - no further commands need be typed there.

4. Open another terminal window.

5. Type:

ssh -Y -p 33301 username@localhost

• Replace username by the name of your Argonne account name.
• Steps 4 and 5 can be repeated as often as desired.

Do not type any commands at a prompt on mega.

Using an ssh config file

To simplify making connections from your computer, create or use an ssh configuration file on it.

Config file contents

On your computer:

• Close any Carbon windows opened previously.
• Open a new terminal window.
  • Linux: xterm
  • Mac: Terminal.app
  • Windows: a Cygwin window.
• Copy&paste the following text into your ~/.ssh/config file.

NoHostAuthenticationForLocalhost yes

Host mega
        Hostname mega.cnm.anl.gov
        User              argonne_user_name
        LocalForward      33301  carbon:22
        LocalForward      33380  carbonmgmt:80

Host carbon
        Hostname localhost
        User              argonne_user_name
        Port              33301
        ForwardX11        yes
        ForwardX11Trusted yes
        AddKeysToAgent    yes

• Replace the string argonne_user_name with the name of your Argonne domain account.
• Remove write permissions to the config file, its parent directory, and your home for anyone but you:

chmod go-rwx ~/.ssh ~/.ssh/config
chmod go-w ~

• Note for Cygwin users: To paste text into a Cygwin window, right-click with your mouse or use the Shift-Ins keyboard shortcut.
• Simply Copy & Paste the line from this browser window to your terminal.

• Advanced users: See the ssh_config man page for syntax and options of the config file.

Connecting with a config file

With the config file in place, the commands in section #Manual setup above simplify to:

1. In one window, run the following command, then minimize the window:

ssh -v mega

Do not type any further commands here. (You can use the -N option to intentionally suppress a command prompt.)

2. Use other terminals as often as needed to log in interactively:

ssh carbon

Advanced: Connecting to a specific login node

The configuration described above uses a generic host name carbon, which at present will lead to one specific login node. However, you may want to connect to another node that you have used previously, such as to reconnect to a VNC sesssion. To do so, change your .ssh/config to forward additional ports and set up more specific Host aliases.

Modify configuration file

• Similar to the above, copy&paste the following text into your ~/.ssh/config file:

NoHostAuthenticationForLocalhost yes

Host mega
        Hostname          mega.cnm.anl.gov
        User              argonne_user_name
        LocalForward      33301  carbon:22
        LocalForward      33305  clogin5:22
        LocalForward      33306  clogin6:22
        LocalForward      33380  carbonmgmt:80

Host carbon
        Port              33301
Host clogin5
        Port              33305
Host clogin6
        Port              33306
Host carbon clogin*
        …

You can then connect to either carbon, or specific host names clogin5 etc., as follows:

1. Run once in your desktop session:

ssh -v mega

2. Run as often as needed:

ssh clogin
ssh clogin5
ssh clogin6
scp …      # (to or from clogin)
…

Test remote graphics using X11

1. In a terminal connected to a Carbon login node, run the command:

   echo $DISPLAY
   

   You should get something like:

   localhost:17.0

   The actual number will vary.

   

2. Start a simple X11 program:

   xload
   

   You should see a small window pop up on your screen, labeled "login5" or "login6".

3. Close the window using its close button or press Ctrl-C in the terminal window.
4. Now use graphical applications on Carbon, for instance:
   

   gnuplot
   

   This will start the gnuplot interpreter in the terminal.

   • Create a sample plot. At the gnuplot> prompt, type:

     plot sin(x)
     

   • Exit the program:

     exit
     

File transfer

• There are many ways to transfer files through an ssh tunnel. However, you must always initiate the connection from the remote machine (your home computer), and either push files to Carbon or pull files from Carbon.
• With the above tunnel setup you cannot initiate a file transfer from Carbon or mega.
• Do not use mega as an intermediate. Files stored there wil be deleted.

Prerequisite

The command examples below require that a tunnel-establishing connection to mega is open. Follow section #Manual setup or #Connecting with a config file above to open or re-open this connection if needed.

Without SSH config file

Run the following example commands on your machine.

Copy to Carbon – "push"

scp -P 33301 -p  'foofile'  argonne_user_name@localhost:'dir/on/carbon/'

Copy from Carbon – "pull"

scp -P -p  argonne_user_name@localhost:'dir/on/carbon/name_on_carbon'  destination_on_your_machine/

• Type localhost as written. Together with the option -P 33301 (or -o port=number for older scp versions) it refers to the tunnel that you have set up earlier.
• Replace argonne_user_name by the user name you have at Argonne.

With an SSH config file

Run the following example commands on your machine.

Copy a file to Carbon (push from outside)

• With the above ssh/.config file in place:

scp -p 'foofile' carbon:'dir/on/carbon/'

Copy a whole directory to Carbon

Short of using tar or zip:

scp -rp 'local/path/' carbon:'dir/on/carbon/'

This method does not preserve symbolic links.

Copy a file from Carbon (pull from outside)

scp -p carbon:remote/path/foo  local/path/
scp -p carbon:remote/path/foo  .

Note the trailing "." in the last example to copy to your current directory.

Reverse tunneling

The file copy operations above must all be initiated on the outside host. It is possible to set up tunnels to be able to initiate file operations on the inside. However, this is more complex and will be left as an exercise for advanced users. It involves setting up:

1. agent forwarding
2. a .ssh/config file on Carbon, and
3. specifying -R command line options or RemoteForward config file directives when accessing carbon at the second stage.

Bonus – Intranet web access

Open the following links in a browser on your external workstation:

https://localhost:33343/cnm/HPC
http://localhost:33380/

Copy a file to Carbon from a third host

0. Similar to the procedure in section #Manual setup above, open two interactive sessions to the remote computer.

1. From the first session, open a connection to Mega and forward a port:

ssh -v -L 33301:carbon:22  username@mega.cnm.anl.gov

• Do not type any further commands there.

2. In the second session, test if you can reach Carbon interactively.

ssh -Y -p 33301 -o NoHostAuthenticationForLocalhost=yes username@localhost

3. Close the Carbon connection from step 2.

exit

You should arrive back at the prompt of the third host.

4. From the second session again, initiate the file transfer as:

scp -v -p -o port=33301 -o NoHostAuthenticationForLocalhost=yes \
        filename_remote username@localhost:[target_directory/][filename_on_carbon]

• Replace username by the name of your Argonne account name.
• Note the ":" at the end of the host name, which tells the scp command that the copy target is on a remote host.
• The file named filename_remote will arrive in your Carbon home directory under this name, unless you specify a target_directory and/or a filename_on_carbon.

5. When done, press Ctrl-C in the first session to terminate the connection to mega.

Advanced: Speeding up logins using ssh keys

Our SSH gatway host itself does not accept SSH keys – you must always type your password for the initial connection to Mega.

Once you have established this initial connection and your tunnels are configured, you can use SSH keys to log in to Carbon login nodes without further passwords.

Create a key

• Create an ssh key pair key if you do not already have one :

yourcomputer$ ssh-keygen -t rsa

The output will look like:

Generating public/private rsa key pair.
Enter file in which to save the key (/Users/home_id/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase):              <-- Choose and enter a password.
Enter same passphrase again: 

Your identification has been saved in /Users/home_id/.ssh/id_rsa.
Your public key has been saved in /Users/home_id/.ssh/id_rsa.pub.
The key fingerprint is:
42:c3:72:fb:4c:be:c6:80:81:bc:8b:73:d8:88:84:2e home_id@yourcomputer.local

• Inspect the .ssh directory on your computer. It should contain the following files:

$ ls -1 ~/.ssh
config
id_rsa
id_rsa.pub
known_hosts
random_seed

id_* are the just created key files.

Using an ssh-agent

On modern systems, the mere presence of private keys with their default names triggers handling through ssh-agent and brings up a passphrase dialog at the start of the desktop session or when a key is first requested for an outgoing connection. This is the case on MacOS X since 10.5 (Leopard), where key management is secure and largely automated through its general keychain concept.

On older systems, you need to (a) set up the session to run under ssh agent and (b) request the passphrase upon login. Circumstances vary widely. Ask Google or your local admin for help. See also the page Getting started with SSH.

Place your public key on Carbon

Password dialog on connecting to carbon once keys are in place.

To productively use your ssh key pair, you need to transfer the public half to Carbon.

Requirements:

• you have a public/private ssh key pair on your machine (called yourcomputer in this example).
• the public file on yourcomputer is named ~/.ssh/id_rsa.pub or ~/.ssh/id_dsa.pub (either one will work).
• the ssh host alias mega has been set up on yourcomputer in ~/.ssh/config and defines tunnels using LocalForward declarations.
• the tunnel connection for mega is open, i.e., on yourcomputer, ssh mega runs in another terminal.
• the ssh alias carbon has been set up as described above.
• an interactive session ssh carbon succeeds.

Then, to transfer your public key from yourcomputer to the Carbon login nodes, type:

cat ~/.ssh/id_rsa.pub | ssh carbon "umask 033; mkdir -p .ssh; cat >> .ssh/authorized_keys"

This command will append the contents of your ssh public key on yourcomputer to an authorized_keys file on Carbon. The command above is more reliable than cut&paste across interactive terminal sessions. If you created a dsa-type key, or chose your own name for the key pair, you'll have to cat the appropriate file instead. Remember to use the public part of the key, i.e. the file id_foo.pub.

Troubleshooting

General connection issues

Run ssh with one or more -v options ("verbosity"), and record the output for diagnostic (copy&paste into a file). E.g.:

ssh -v -v [other options and arguments …]

If you cannot resolve the issue yourself, follow HPC/Support.

Public keys

If public keys do not work, use your password to gain access, then check the following:

Permissions

The directories and the authorized_keys file must, at the least, not be writable by anyone except yourself; other permissions are recommended to be restrictive as well:

login5$ ls -ld ~ ~/.ssh ~/.ssh/authorized_keys
drwxr-xr-x 28 argonne_id users 4096 Feb 22 13:52 /home/argonne_id
drwx------  4 argonne_id argonne_id 4096 Feb 18 21:25 /home/argonne_id/.ssh
-rw-------  1 argonne_id argonne_id  829 Oct  9 10:55 /home/argonne_id/.ssh/authorized_keys

To change:

login5$ chmod go-w ~
login5$ chmod go-rwx ~/.ssh ~/.ssh/authorized_keys

Corruption of authorized_keys file

The file should contain public keys in OpenSSH format, one per line, without additional line breaks. One way to check is:

yourcomputer$ cut -c1-50 ~/.ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEjdsfjJSAHkjh88y
ssh-dss AAAAB3NzaC1kc3MAAACBAkkjsadfJFg7EFjDJhWEjk

The output should look similar to the sample above. Use your favorite text editor to join erroneously broken lines. Note that there should not be anything like BEGIN RSA PRIVATE KEY, which would obviously indicate a private, rather than public key.

Verify that an ssh-agent is running

To verify that an ssh-agent is running and visible to your shell, check for the presence of the environment variable SSH_AUTH_SOCK. To list the keys it has loaded, use ssh-add -l (lowercase "L"). Typical output is as follows:

yourcomputer$ echo $SSH_AUTH_SOCK
/tmp/ssh-HCvZKt5478/agent.5478
yourcomputer$ ssh-add -l
1024 b9:65:98:c0:7c:25:da:b6:86:48:98:6b:2a:5a:9d:21 /Users/home_id/.ssh/id_rsa (RSA)