HPC/Network Access/SSH Tunnel Setup on Linux and MacOS: Difference between revisions

From CNM Wiki
Jump to navigation Jump to search
m (→‎Troubleshooting: change group name)
 
(127 intermediate revisions by the same user not shown)
Line 1: Line 1:
The following sections document how to establish ssh tunneling on a Linux or MacOS X machine running [http://en.wikipedia.org/wiki/OpenSSH OpenSSH].  Numerous [http://www.google.com/search?hl=en&q=SSH+Windows Windows solutions] are available to provide the same functionality, but details vary considerably.
The following sections document how to establish ssh tunneling on a Linux or MacOS X machine running [http://en.wikipedia.org/wiki/OpenSSH OpenSSH].  Numerous [http://www.google.com/search?hl=en&q=SSH+Windows Windows solutions] are available to provide the same functionality, but details vary considerably.


== First steps ==
== Manual setup ==
=== Assumptions ===
[[Image:HPC 2012-02-06 screenshot sketch ssh tunneling windows.png|350px|right]]
For the purpose of the following examples, I'll use the following names:
'''On your computer''', run the following commands:
* your external system is '''homebase''',
* your user-id on homebase is '''home_id''',
* your Argonne domain login is '''argonne_id''',
* your home directory on your machine is '''/Users/home_id'''.


=== Registering the gateway ===
0. (Windows only) Start your X11 server if you have one.
First, test that you can reach the gateway from your current machine (homebase).
More importantly, this command will fetch and store the host key of mega on homebase:
homebase:/Users/home_id$ <strong>ssh [email protected]</strong>
The authenticity of host 'mega.cnm.anl.gov (146.139.1.6)' can't be established.
RSA key fingerprint is 0e:c5:2a:49:e9:27:6a:2e:b0:9c:ed:3b:5e:8f:28:4c.
Are you sure you want to continue connecting (yes/no)? <strong>yes</strong>
Warning: Permanently added 'mega.cnm.anl.gov,146.139.1.6' (RSA) to the list of known hosts.
 
                        NOTICE TO USERS
                [... DOE warning banner ...]
 
Password: <strong>*********</strong>
Last login: Mon Feb 18 16:57:55 2008 from 1234.comcast.net
Could not chdir to home directory /Users/argonne_id: No such file or directory          <font color="blue"><strong>&lt;-- This is expected.</strong></font>
mega:/ argonne_id$ <strong>exit</strong>
This step is only needed once for each computer you try to connect from.
The public key for mega is stored on homebase
in home_id's <code>.ssh/known_hosts</code> file
and will be verified at each subsequent connection.
There is no need to be alarmed by the ''absence of a home directory''.


=== Establishing a test tunnel ===
1. Open a terminal window (such as Mac Terminal, Cygwin, <strike>PuTTY</strike>, or xterm).
To establish a tunnel ad hoc or for testing purposes, use the following commands on your workstation, best in a newly opened terminal window:
 
  homebase:/Users/home_id$ <strong>ssh -v -L ''33301'':clogin:22  argonne_id@mega.cnm.anl.gov</strong>
2. Type:
 
  <font color="darkred">ssh -v -L 33301:carbon:22  ''username''@mega.cnm.anl.gov</font>
                        NOTICE TO USERS
: '''Do not type any further commands in this window.'''
                [… DOE warning banner …]
 
… 
3. Minimize that window - no further commands need be typed there.
  debug1: Next authentication method: keyboard-interactive
 
Password: <strong>*********</strong>
4. Open another terminal window.
 
 
  debug1: Local connections to LOCALHOST:33301 forwarded to remote address clogin1:22
5. Type:
  debug1: Local forwarding listening on ::1 port 33301.
  <font color="darkgreen">ssh -Y -p 33301 ''username''@localhost</font>
 
:* Replace <code>''username''</code> by the name of your Argonne account name.
  debug1: Entering interactive session.
:* Steps 4 and 5 can be repeated as often as desired.
Could not chdir to home directory /Users/argonne_id: No such file or directory
 
mega:/ argonne_id$ _
Do not type any commands at a prompt on mega.
 
== Using an ssh config file ==
To simplify making connections from your computer, create or use an ssh configuration file on it.
 
=== Config file contents ===
'''On your computer''':
* Close any Carbon windows opened previously.
* Open a new terminal window.
** Linux: xterm
** Mac: Terminal.app
** Windows: a Cygwin window.
* Copy&paste the following text into your ~/.ssh/config file.
<!-- You can use the following command to create the file from scratch:
cat > ~/.ssh/config
: Copy&paste the text below, then press Ctrl-D to finish.
: '''Warning:''' This command will override an existing file.
-->
  NoHostAuthenticationForLocalhost yes
  Host mega
        Hostname mega.cnm.anl.gov
        User              ''argonne_user_name''
        LocalForward      33301 carbon:22
        LocalForward      33380 carbonmgmt:80
   
  Host carbon
        Hostname localhost
        User              ''argonne_user_name''
        Port              33301
        ForwardX11        yes
        ForwardX11Trusted yes
        AddKeysToAgent    yes
* '''Replace''' the string ''argonne_user_name'' with the name of your Argonne domain account.
* Remove write permissions to the config file, its parent directory, and your home for anyone but you:
<syntaxhighlight lang="bash">
chmod go-rwx ~/.ssh ~/.ssh/config
chmod go-w ~
</syntaxhighlight>
:* '''Note for Cygwin users:''' To paste text into a Cygwin window, right-click with your mouse or use the Shift-Ins keyboard shortcut.
<!--
* Run this command:
<syntaxhighlight lang="bash">
read -p "your Argonne user name: "
sed -i "s/
</syntaxhighlight>
-->
:* Simply Copy & Paste the line from this browser window to your terminal.
* Advanced users: See the ssh_config man page for syntax and options of the config file.
 
=== Connecting with a config file ===
With the config file in place, the commands in section [[#Manual setup]] above simplify to:


==== Note for experts ====
: 1. In one window, run the following command, then minimize the window:
ssh can use the <code>-N</code> option to run without a shell and the <code>-f</code> option to background itself after the password request.
<font color="darkred">ssh -v mega</font>
Note, however, that the use of these options tends to leave a bunch of forgotten <code>ssh</code> processes hanging around which hog ports.
: Do not type any further commands here. (You can use the -N option to intentionally suppress a command prompt.)
These processes would have to be tracked down by ps and be killed manually.


=== Connecting through the test tunnel ===
: 2. Use other terminals as often as needed to log in interactively:
Now you are ready to connect through the tunnel. This means opening an ssh session ''to localhost'', which may look odd at first. However, rather than using the standard port (which would get you right back into homebase), use the port specified above at the <code>-L</code> option.  Roughly speaking, that will kick <code>sshd</code> on mega into gear to initiate the connection to the internal tunnel target (clogin) on your behalf, and then hand the connection over to your workstation.
  <font color="darkgreen">ssh carbon</font>
  homebase:/Users/home_id$ <strong>ssh -p ''33301'' argonne_id@localhost </strong>
  argonne_id@localhost's password: <strong>*********</strong>
  Last login: Mon Feb 18 18:56:32 2008 from mega.cnm.anl.gov
  *******************************************************
  For documentation, see:
  http://wiki.inside.anl.gov/cnm/HPC
  *******************************************************
  [21:24] argonne_id@login2:/home/argonne_id$


The port number (''33301'' above) must of course agree between the two commands, but is otherwise largely arbitrary within the range of about 1024 … 65000.  It refers to a port on ''your'' machine (homebase). If, at the first command, you get a message "Address already in use", check with <code>ps -ef</code> if perhaps this is due to one of your own, still active, tunnels.  If not, choose another port.  Do not change the clogin port; it is always 22.
=== Advanced: Connecting to a specific login node ===
The configuration described above uses a generic host name {{host|carbon}}, which at present will lead to one specific login node.
However, you may want to connect to another node that you have used previously, such as to reconnect to a VNC sesssion.
To do so, change your .ssh/config to forward additional ports and set up more specific <code>Host</code> aliases.


When done, exit the interactive shell.
* Similar to the above, copy&paste the following text into your ~/.ssh/config file.
[21:54] argonne_id@login2:/home/argonne_id$ <strong>exit</strong>
logout
Connection to localhost closed.


Then, exit the original connection to mega, which hosted the tunnel:
  NoHostAuthenticationForLocalhost yes
  [21:00] argonne_id@mega:/Users/argonne_id$ <strong>exit</strong>
   
  logout
  Host mega
Connection to mega.cnm.anl.gov closed.
        Hostname          mega.cnm.anl.gov
You may also choose to leave it running in the background to connect back later.
        User              ''argonne_user_name''
        LocalForward      33301  carbon:22
        '''LocalForward      33305  clogin5:22'''
        '''LocalForward      33306  clogin6:22'''
        LocalForward      33380  carbonmgmt:80
Host carbon
        Port              33301
'''Host clogin5'''
        '''Port              33305'''
'''Host clogin6'''
        '''Port              33306'''
Host carbon clogin*
        Hostname localhost
        User ''argonne_user_name''
        ForwardX11 yes
        ForwardX11Trusted yes
        AddKeysToAgent yes
<!--
        '''LocalForward      33307  clogin7:22'''
        '''LocalForward      33308  clogin8:22'''


== Permanent setup ==
'''Host clogin7'''
For more routine access, I recommend using ''ssh public keys'' and the ssh ''configuration file''.  Run the following commands on your workstation, homebase:
        '''Port              33307'''
=== Create your ssh keys ===
'''Host clogin8'''
If you do not already have an ssh private/public key pair, create one:
        '''Port              33308'''
  homebase:/Users/home_id$ <strong>ssh-keygen -t rsa</strong>
-->
  Generating public/private rsa key pair.
With this modified config file in place, you can connect to the network-default {{host|carbon}}, but also to the specific login hosts you configured.
  Enter file in which to save the key (/Users/home_id/.ssh/id_rsa):  
: 1. Run once in your desktop session:
  Enter passphrase (empty for no passphrase): <strong>************</strong>
<font color="darkred">ssh -v mega</font>
  Enter same passphrase again: <strong>************</strong>
: 2. Run as often as needed, any one of these commands:
  Your identification has been saved in /Users/home_id/.ssh/id_rsa.
<font color="darkgreen">ssh clogin5</font>
  Your public key has been saved in /Users/home_id/.ssh/id_rsa.pub.
<font color="darkgreen">ssh clogin6</font>
  The key fingerprint is:
 
  42:c3:72:fb:4c:be:c6:80:81:bc:8b:73:d8:88:84:2e [email protected]
On modern systems, the mere presence of private keys with their default names triggers handling through ''ssh-agent'' and brings up a passphrase dialog at the start of the desktop session or when a key is first requested for an outgoing connection. This is the case on MacOS X Leopard, where key management is largely automated through its general ''keychain'' concept; on Tiger, use [http://www.sshkeychain.org/ SSHKeychain.app].


To verify that an ssh-agent is running and visible to your shell, check for the presence of the environment variable <code>SSH_AUTH_SOCK</code>.  To list the keys it has loaded, use <code>ssh-add -l</code> (lowercase "L"). Typical output is as follows:
== Test remote graphics using X11 ==
; Linux or MacOS X Leopard:
# In a terminal '''connected to a Carbon login node''', run the command:
  homebase:/Users/home_id$ <strong>echo $SSH_AUTH_SOCK</strong>
#:: <source lang="bash">echo $DISPLAY</source>
  /tmp/ssh-HCvZKt5478/agent.5478
#:: You should get something like:
  homebase:/Users/home_id$ <strong>ssh-add -l</strong>
#:: <code>localhost:''17.0''</code>
  1024 b9:65:98:c0:7c:25:da:b6:86:48:98:6b:2a:5a:9d:21 /Users/home_id/.ssh/id_rsa (RSA)
#: The actual number will vary. [[Image:HPC 2011-06-23 xload example.png|right|60px]]
; MacOS X Tiger using [http://www.sshkeychain.org/ SSHKeychain.app]:
# Start a simple X11 program:
  homebase:/Users/home_id $ <strong>echo $SSH_AUTH_SOCK</strong>
#:: <source lang="bash">xload</source>
  /tmp/55522/SSHKeychain.socket
#: You should see a small window pop up on your screen, labeled "login5" or "login6".
  homebase:/Users/home_id$ <strong>ssh-add -l</strong>
# Close the window using its close button or press Ctrl-C in the terminal window.
  1024 b9:65:98:c0:7c:25:da:b6:86:48:98:6b:2a:5a:9d:21 /Users/home_id/.ssh/id_rsa (RSA)
# Now use graphical applications on Carbon, for instance: [[Image:HPC 2011-06-23 gnuplot example.png|right|300px]]
 
#:: <source lang="bash">gnuplot</source>
On older systems, you need to (a) set up the session to run under ''ssh agent'' and (b) request the passphrase upon login.  Circumstances vary widely.  Ask [http://www.google.com/search?hl=en&q=ssh-agent Google] or your local admin for help.  See also the page [http://kimmo.suominen.com/docs/ssh/ ''Getting started with SSH''].
#: This will start the gnuplot interpreter in the terminal.
#* Create a sample plot. At the <code>gnuplot&gt; </code> prompt, type:
#*: <source lang="bash">plot sin(x)</source>
#* Exit the program:
#*: <source lang="bash">exit</source>


=== Configure <code>~/.ssh/config</code> ===
== File transfer ==
Create your <code>~/.ssh/config</code> file or, if you already have one (in which case you might already know the procedure anyway), add the following lines.  The first line, to be placed before any other <code>Host</code> entries, is optional. Use it if your home is shared across machines or you use many of these type of forwards.
* There are many ways to transfer files through an ssh tunnel. However, you must always initiate the connection from the remote machine (your home computer), and either ''push'' files to Carbon or ''pull'' files from Carbon.
  # optional
* With the above tunnel setup you cannot initiate a file transfer from Carbon or mega.
  NoHostAuthenticationForLocalhost yes
* Do not use mega as an intermediate. Files stored there wil be deleted.


  Host mega
; Prerequisite: The command examples below require that a tunnel-establishing '''''connection to <code>mega</code> is open'''''. Follow section [[#Manual setup]] or [[#Connecting with a config file]] above to open or re-open this connection if needed.
        Hostname mega.cnm.anl.gov
        User  <strong>argonne_id</strong>
        LocalForward  <strong>33301</strong>  clogin:22
        LocalForward  <strong>33343</strong>  wiki.inside.anl.gov:443
        LocalForward  <strong>33380</strong>  cmgmt1:80
 
  Host carbon
        Hostname localhost
        User <strong>argonne_id</strong>
        Port          <strong>33301</strong>
        ForwardX11Trusted yes
As with ''33301'', the other '''<code>port numbers</code>''' refer to ports on your machine (homebase) and are mostly arbitrary. If you are on a shared machine, it is entirely possible that these ports are busy, especially if someone else follows these very instructions.  Consult <code>/etc/services</code> or [http://www.iana.org/assignments/port-numbers IANA] and pick ports above 1024 that are unassigned.


=== Establish the tunnel - simple version ===
=== Without SSH config file ===
Open a separate terminal window:
<!-- * Without a config file (''roughing it'') -->
<strong>ssh mega</strong>
This command now uses the ''host alias'' <code>mega</code>
from your <code>.ssh/config</code> file to address the gateway and even set your remote user name.
You can minimize the window at this point and essentially forget about it until you log out.


Alternatively, you may choose to run:
Run the following '''example commands on your machine'''.
<strong>ssh -N mega</strong>
This will give no more output after the password prompt, but will disallow adding further tunneled connections.


=== Place your public key on Carbon ===
; Copy ''to'' Carbon – "push":
To put your ssh key pair to use, you need to append your ssh public key to an <code>authorized_keys</code> file on Carbon.  To do this, while you are on your external workstation, type:
<font color="darkgreen">scp -P 33301 -p  'foofile' ''argonne_user_name''@localhost:'dir/on/carbon/'</font>
  <strong>cat ~/.ssh/id_rsa.pub | ssh carbon "umask 033; mkdir -p .ssh; cat >> .ssh/authorized_keys"</strong>
The command above is more reliable than cut&paste across interactive terminal sessions. If you created a dsa-type key, or chose your own name for the key pair, you'll have to cat the appropriate file instead. Remember to use the ''public'' part of the key, i.e. the file <code>id_foo.pub</code>.


==== Troubleshooting ====
; Copy ''from'' Carbon – "pull":
<font color="darkgreen">scp -P -p  ''argonne_user_name''@localhost:'dir/on/carbon/name_on_carbon'  destination_on_your_machine/</font>


If public keys do not work, use your password to gain access, then check the following:
* Type <code>localhost</code> as written. Together with the option <code>-P 33301</code> (or <code>-o port=''number''</code> for older scp versions) it refers to the tunnel that you have set up earlier.
; Permissions: The directories and the <code>authorized_keys</code> file must, at the least, not be writable by anyone except yourself; other permissions are recommended to be restrictive as well:
* Replace ''<code>argonne_user_name</code>'' by the user name you have at Argonne.
  /home/argonne_id$ <strong>ls -ld ~ ~/.ssh ~/.ssh/authorized_keys</strong>
  drwxr<font color="red">-</font>xr<font color="red">-</font>x 28 argonne_id users 4096 Feb 22 13:52 /home/argonne_id
  drwx<font color="red">------</font> 4 argonne_id argonne_id 4096 Feb 18 21:25 /home/argonne_id/.ssh
  -rw-<font color="red">------</font> 1 argonne_id argonne_id  829 Oct  9 10:55 /home/argonne_id/.ssh/authorized_keys
: To change:
  /home/argonne_id$ <strong>chmod <font color="red">go-w</font> ~</strong>
  /home/argonne_id$ <strong>chmod <font color="red">go-rwx</font> ~/.ssh ~/.ssh/authorized_keys</strong>
; Corruption of <code>authorized_keys</code> file: The file should contain public keys in OpenSSH format, ''one per line'', without additional line breaks.  One way to check is:
  /home/argonne_id $ <strong>cut -c1-50 ~/.ssh/authorized_keys</strong>
  ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEjdsfjJSAHkjh88y
  ssh-dss AAAAB3NzaC1kc3MAAACBAkkjsadfJFg7EFjDJhWEjk
The output should ''look similar'' to the sample above.  Use your favorite text editor to join erroneously broken lines.
Note that there should '''not''' be anything like <code>BEGIN RSA PRIVATE KEY</code>, which would obviously indicate a private, rather than public key.


== Routine use ==
=== With an SSH config file ===
The commands above describe a one-off setup process. Once all the necessary files are in place, all that is needed for routine access is to open the tunnel once per login session, and then use it as often as needed.  The host aliases defined in the config file allow the tunnel "entrance" to appear like a regularly named machine.
Run the following '''example commands on your machine'''.
=== I - Establish the tunnel - advanced version ===
If a tunnel session is not already running, open one.
You can use one of the ways described above, or add a twist as follows:
  <strong>ssh -v mega  iostat -w 60</strong>
This command will give you a life sign from the tunnel machine every minute.  This may be helpful with choppy network connections.
The actual output (the load average on the gateway machine) is useful only for an admin who will note if the machine should get bogged down.
The <code>-v</code> option will give you some ssh chatter, especially when the tunnel is being used.
As before, you can still minimize this window and forget about it until the end of your desktop session.
Note that you will ''always'' have to type your domain password to reach mega.


Keep in mind that either one of the above methods to establish the tunnel
==== Copy a file ''to'' Carbon (''push'' from outside) ====
is necessary only ''once'' for each desktop session on your workstation.
* With the above ssh/.config file in place:
<font color="darkgreen">scp -p 'foofile' carbon:'dir/on/carbon/'</font>


=== II - Access the cluster ===
==== Copy a whole directory ''to'' Carbon ====
Once the tunnel is established, you can access the cluster through the tunnel as often as you like.
If you've set up the ''ssh agent'' on your external machine, you no longer need to type your domain password
for any of the following commands.  These commands are for use on your ''external'' workstation.
; Open an interactive shell:
  <strong>ssh carbon</strong>
; Execute a single remote command:
  <strong>ssh carbon qstat</strong>
; Copy a file ''to'' Carbon (''push'' from outside):
  <strong>scp -p foo carbon:remote/path/</strong>
; Copy a whole directory ''to'' Carbon:
Short of using tar or zip:
Short of using tar or zip:
  <strong>scp -rp local/path/ carbon:remote/path/</strong>
<font color="darkgreen">scp -rp 'local/path/' carbon:'dir/on/carbon/'</font>
This method does not preserve symbolic links.
This method does not preserve symbolic links.
; Copy a file ''from'' Carbon (''pull'' from outside):
 
  <strong>scp -p carbon:remote/path/foo  local/path/</strong>
==== Copy a file ''from'' Carbon (''pull'' from outside) ====
  <strong>scp -p carbon:remote/path/foo  .</strong>
<font color="darkgreen">scp -p carbon:remote/path/foo  local/path/</font>
<font color="darkgreen">scp -p carbon:remote/path/foo  .</font>
Note the trailing "." in the last example to copy to your current directory.
Note the trailing "." in the last example to copy to your current directory.
; Reverse tunneling
; Reverse tunneling
Line 199: Line 194:
# agent forwarding
# agent forwarding
# a <code>.ssh/config</code> file on Carbon, and
# a <code>.ssh/config</code> file on Carbon, and
# specifying <code>-R</code> command line options or <code>RemoteForward</code> config file directives when accessing carbon at the second stage.
# specifying <code>-R</code> command line options or <code>RemoteForward</code> config file directives when accessing {{host|carbon}} at the second stage.
; Bonus – Intranet web access:
 
==== Bonus – Intranet web access ====
Open the following links in a browser ''on your external workstation'':
Open the following links in a browser ''on your external workstation'':
  https://localhost:33343/cnm/HPC
https://localhost:33343/cnm/HPC
  http://localhost:33380/ganglia
http://localhost:33380/
 
==== Copy a file ''to'' Carbon from a ''third host'' ====
0. Similar to the procedure in section [[#Manual setup]] above, open two interactive sessions to the remote computer.
 
1. From the first session, open a connection to Mega and forward a port:
<font color="darkred">ssh -v -L 33301:carbon:22  ''username''@mega.cnm.anl.gov</font>
:* '''Do not type any further commands there.'''
 
2. In the second session, test if you can reach Carbon interactively.
<font color="darkgreen">ssh -Y -p 33301 -o NoHostAuthenticationForLocalhost=yes username@localhost</font>
 
3. Close the Carbon connection from step 2.
exit
: You should arrive back at the prompt of the third host.
 
4. From the second session again, initiate the file transfer as:
<font color="darkgreen">scp -v -p -o port=33301 -o NoHostAuthenticationForLocalhost=yes \
        ''filename_remote'' ''username''@localhost''':'''[''target_directory/''][''filename_on_carbon'']</font>
:* Replace <code>''username''</code> by the name of your Argonne account name.
:* Note the ":" at the end of the host name, which tells the <code>scp</code> command that the copy target is on a remote host.
:* The file named <code>''filename_remote''</code> will arrive in your Carbon home directory under this name, unless you specify a <code>''target_directory''</code> and/or a <code>''filename_on_carbon''</code>.
 
5. When done, press Ctrl-C in the first session to terminate the connection to mega.


--[[User:Stern|stern]] 00:24, February 20, 2008 (CST)
== Advanced: Speeding up logins using ssh keys ==
 
Our SSH gatway host itself does not accept SSH keys – you must always type your password for the initial connection to Mega.
 
Once you have established this initial connection and your tunnels are configured, you can use SSH keys to log in to Carbon login nodes without further passwords.
 
=== Create a key ===
* Create an ssh key pair key ''if you do not already have one'' :
<font color="green">yourcomputer</font>$ '''ssh-keygen -t rsa'''
 
The output will look like:
Generating public/private rsa key pair.
Enter file in which to save the key (/Users/home_id/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):              <font color="blue"><strong>&lt;-- Choose and enter a password.</strong></font>
Enter same passphrase again:
Your identification has been saved in /Users/home_id/.ssh/id_rsa.
Your public key has been saved in /Users/home_id/.ssh/id_rsa.pub.
The key fingerprint is:
42:c3:72:fb:4c:be:c6:80:81:bc:8b:73:d8:88:84:2e home_id@<font color="green">yourcomputer</font>.local
 
* Inspect the .ssh directory '''on your computer'''. It should contain the following files:
$ ls -1 ~/.ssh
config
'''id_rsa'''
'''id_rsa.pub'''
known_hosts
random_seed
: id_* are the just created key files.
 
=== Using an ssh-agent ===
On modern systems, the mere presence of private keys with their default names triggers handling through ''ssh-agent'' and brings up a passphrase dialog at the start of the desktop session or when a key is first requested for an outgoing connection.  This is the case on MacOS X since 10.5 (Leopard), where key management is secure and largely automated through its general ''keychain'' concept.
 
On older systems, you need to (a) set up the session to run under ''ssh agent'' and (b) request the passphrase upon login.  Circumstances vary widely.  Ask [http://www.google.com/search?hl=en&q=ssh-agent Google] or your local admin for help.  See also the page [http://kimmo.suominen.com/docs/ssh/ ''Getting started with SSH''].
 
=== Place your public key on Carbon ===
[[Image:HPC 2012-02-06 screenshot ssh agent password request.png|350px|thumb|Password dialog on connecting to {{host|carbon}} once keys are in place.]]
To productively use your ssh key pair, you need to transfer the public half to Carbon.
 
Requirements:
* you have a public/private ssh key pair on your machine (called '''<font color="green">yourcomputer</font>''' in this example).
* the public file on '''<font color="green">yourcomputer</font>''' is named <code>~/.ssh/id_rsa.pub</code> or  <code>~/.ssh/id_dsa.pub</code> (either one will work).
* the ssh host alias <code><font color="red">mega</font></code> has been set up on '''<font color="green">yourcomputer</font>''' in <code>~/.ssh/config</code> and defines tunnels using <code>LocalForward</code> declarations.
* the tunnel connection for '''<font color="red">mega</font>''' is open, i.e., on '''<font color="green">yourcomputer</font>''', <code>ssh mega</code> runs in another terminal.
* the ssh alias {{host|carbon}} has been set up as described above.
* an interactive session <code>ssh carbon</code> succeeds.
 
<br clear="all" />
Then, to transfer your public key from '''<font color="green">yourcomputer</font>''' to the Carbon login nodes, type:
cat ~/.ssh/id_rsa.pub | ssh carbon "umask 033; mkdir -p .ssh; cat >> .ssh/authorized_keys"
This command will append the contents of your ssh public key on '''<font color="green">yourcomputer</font>''' to an <code>authorized_keys</code> file on Carbon.  The command above is more reliable than cut&paste across interactive terminal sessions.  If you created a dsa-type key, or chose your own name for the key pair, you'll have to cat the appropriate file instead.  Remember to use the ''public'' part of the key, i.e. the file <code>id_foo.pub</code>.
 
== Troubleshooting ==
 
=== General connection issues ===
Run <code>ssh</code> with one or more <code>-v</code> options ("verbosity"), and record the output for diagnostic (copy&paste into a file). E.g.:
<source lang="bash">
ssh -v -v [other options and arguments …]
</source>
If you cannot resolve the issue yourself, follow [[HPC/Support]].
 
=== Public keys ===
If public keys do not work, use your password to gain access, then check the following:
; Permissions: The directories and the <code>authorized_keys</code> file must, at the least, not be writable by anyone except yourself; other permissions are recommended to be restrictive as well:
<font color="blue">login5</font>$ <strong>ls -ld ~ ~/.ssh ~/.ssh/authorized_keys</strong>
drwxr<font color="red">-</font>xr<font color="red">-</font>x 28 argonne_id users 4096 Feb 22 13:52 /home/argonne_id
drwx<font color="red">------</font>  4 argonne_id argonne_id 4096 Feb 18 21:25 /home/argonne_id/.ssh
-rw-<font color="red">------</font>  1 argonne_id argonne_id  829 Oct  9 10:55 /home/argonne_id/.ssh/authorized_keys
: To change:
<font color="blue">login5</font>$ '''chmod <font color="red">go-w</font> ~'''
<font color="blue">login5</font>$ '''chmod <font color="red">go-rwx</font> ~/.ssh ~/.ssh/authorized_keys'''
; Corruption of <code>authorized_keys</code> file: The file should contain public keys in OpenSSH format, ''one per line'', without additional line breaks.  One way to check is:
<font color="green">yourcomputer</font>$ <strong>cut -c1-50 ~/.ssh/authorized_keys</strong>
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEjdsfjJSAHkjh88y
ssh-dss AAAAB3NzaC1kc3MAAACBAkkjsadfJFg7EFjDJhWEjk
The output should ''look similar'' to the sample above.  Use your favorite text editor to join erroneously broken lines.
Note that there should '''not''' be anything like <code>BEGIN RSA PRIVATE KEY</code>, which would obviously indicate a private, rather than public key.
; Verify that an ssh-agent is running:
To verify that an ssh-agent is running and visible to your shell, check for the presence of the environment variable <code>SSH_AUTH_SOCK</code>.  To list the keys it has loaded, use <code>ssh-add -l</code> (lowercase "L").  Typical output is as follows:
<font color="green">yourcomputer</font>$ '''echo $SSH_AUTH_SOCK'''
/tmp/ssh-HCvZKt5478/agent.5478
<font color="green">yourcomputer</font>$ '''ssh-add -l'''
1024 b9:65:98:c0:7c:25:da:b6:86:48:98:6b:2a:5a:9d:21 /Users/home_id/.ssh/id_rsa (RSA)


{{Template:Standard Footer}}
[[Category:HPC|Network]]

Latest revision as of 21:11, March 14, 2024

The following sections document how to establish ssh tunneling on a Linux or MacOS X machine running OpenSSH. Numerous Windows solutions are available to provide the same functionality, but details vary considerably.

Manual setup

HPC 2012-02-06 screenshot sketch ssh tunneling windows.png

On your computer, run the following commands:

0. (Windows only) Start your X11 server if you have one.

1. Open a terminal window (such as Mac Terminal, Cygwin, PuTTY, or xterm).

2. Type:

ssh -v -L 33301:carbon:22  username@mega.cnm.anl.gov
Do not type any further commands in this window.

3. Minimize that window - no further commands need be typed there.

4. Open another terminal window.

5. Type:

ssh -Y -p 33301 username@localhost
  • Replace username by the name of your Argonne account name.
  • Steps 4 and 5 can be repeated as often as desired.

Do not type any commands at a prompt on mega.

Using an ssh config file

To simplify making connections from your computer, create or use an ssh configuration file on it.

Config file contents

On your computer:

  • Close any Carbon windows opened previously.
  • Open a new terminal window.
    • Linux: xterm
    • Mac: Terminal.app
    • Windows: a Cygwin window.
  • Copy&paste the following text into your ~/.ssh/config file.
NoHostAuthenticationForLocalhost yes

Host mega
        Hostname mega.cnm.anl.gov
        User              argonne_user_name
        LocalForward      33301  carbon:22
        LocalForward      33380  carbonmgmt:80

Host carbon
        Hostname localhost
        User              argonne_user_name
        Port              33301
        ForwardX11        yes
        ForwardX11Trusted yes
        AddKeysToAgent    yes
  • Replace the string argonne_user_name with the name of your Argonne domain account.
  • Remove write permissions to the config file, its parent directory, and your home for anyone but you:
chmod go-rwx ~/.ssh ~/.ssh/config
chmod go-w ~
  • Note for Cygwin users: To paste text into a Cygwin window, right-click with your mouse or use the Shift-Ins keyboard shortcut.
  • Simply Copy & Paste the line from this browser window to your terminal.
  • Advanced users: See the ssh_config man page for syntax and options of the config file.

Connecting with a config file

With the config file in place, the commands in section #Manual setup above simplify to:

1. In one window, run the following command, then minimize the window:
ssh -v mega
Do not type any further commands here. (You can use the -N option to intentionally suppress a command prompt.)
2. Use other terminals as often as needed to log in interactively:
ssh carbon

Advanced: Connecting to a specific login node

The configuration described above uses a generic host name carbon, which at present will lead to one specific login node. However, you may want to connect to another node that you have used previously, such as to reconnect to a VNC sesssion. To do so, change your .ssh/config to forward additional ports and set up more specific Host aliases.

  • Similar to the above, copy&paste the following text into your ~/.ssh/config file.
NoHostAuthenticationForLocalhost yes

Host mega
        Hostname          mega.cnm.anl.gov
        User              argonne_user_name
        LocalForward      33301  carbon:22
        LocalForward      33305  clogin5:22
        LocalForward      33306  clogin6:22
        LocalForward      33380  carbonmgmt:80

Host carbon
        Port              33301

Host clogin5
        Port              33305

Host clogin6
        Port              33306

Host carbon clogin*
        Hostname		localhost
        User			argonne_user_name
        ForwardX11		yes
        ForwardX11Trusted	yes
        AddKeysToAgent		yes

With this modified config file in place, you can connect to the network-default carbon, but also to the specific login hosts you configured.

1. Run once in your desktop session:
ssh -v mega
2. Run as often as needed, any one of these commands:
ssh clogin5
ssh clogin6

Test remote graphics using X11

  1. In a terminal connected to a Carbon login node, run the command:
    echo $DISPLAY
    
    You should get something like:
    localhost:17.0
    The actual number will vary.
    HPC 2011-06-23 xload example.png
  2. Start a simple X11 program:
    xload
    
    You should see a small window pop up on your screen, labeled "login5" or "login6".
  3. Close the window using its close button or press Ctrl-C in the terminal window.
  4. Now use graphical applications on Carbon, for instance:
    HPC 2011-06-23 gnuplot example.png
    gnuplot
    
    This will start the gnuplot interpreter in the terminal.
    • Create a sample plot. At the gnuplot> prompt, type:
      plot sin(x)
      
    • Exit the program:
      exit
      

File transfer

  • There are many ways to transfer files through an ssh tunnel. However, you must always initiate the connection from the remote machine (your home computer), and either push files to Carbon or pull files from Carbon.
  • With the above tunnel setup you cannot initiate a file transfer from Carbon or mega.
  • Do not use mega as an intermediate. Files stored there wil be deleted.
Prerequisite
The command examples below require that a tunnel-establishing connection to mega is open. Follow section #Manual setup or #Connecting with a config file above to open or re-open this connection if needed.

Without SSH config file

Run the following example commands on your machine.

Copy to Carbon – "push"
scp -P 33301 -p  'foofile'  argonne_user_name@localhost:'dir/on/carbon/'
Copy from Carbon – "pull"
scp -P -p  argonne_user_name@localhost:'dir/on/carbon/name_on_carbon'  destination_on_your_machine/
  • Type localhost as written. Together with the option -P 33301 (or -o port=number for older scp versions) it refers to the tunnel that you have set up earlier.
  • Replace argonne_user_name by the user name you have at Argonne.

With an SSH config file

Run the following example commands on your machine.

Copy a file to Carbon (push from outside)

  • With the above ssh/.config file in place:
scp -p 'foofile' carbon:'dir/on/carbon/'

Copy a whole directory to Carbon

Short of using tar or zip:

scp -rp 'local/path/' carbon:'dir/on/carbon/'

This method does not preserve symbolic links.

Copy a file from Carbon (pull from outside)

scp -p carbon:remote/path/foo  local/path/
scp -p carbon:remote/path/foo  .

Note the trailing "." in the last example to copy to your current directory.

Reverse tunneling

The file copy operations above must all be initiated on the outside host. It is possible to set up tunnels to be able to initiate file operations on the inside. However, this is more complex and will be left as an exercise for advanced users. It involves setting up:

  1. agent forwarding
  2. a .ssh/config file on Carbon, and
  3. specifying -R command line options or RemoteForward config file directives when accessing carbon at the second stage.

Bonus – Intranet web access

Open the following links in a browser on your external workstation:

https://localhost:33343/cnm/HPC
http://localhost:33380/

Copy a file to Carbon from a third host

0. Similar to the procedure in section #Manual setup above, open two interactive sessions to the remote computer.

1. From the first session, open a connection to Mega and forward a port:

ssh -v -L 33301:carbon:22  username@mega.cnm.anl.gov
  • Do not type any further commands there.

2. In the second session, test if you can reach Carbon interactively.

ssh -Y -p 33301 -o NoHostAuthenticationForLocalhost=yes username@localhost

3. Close the Carbon connection from step 2.

exit
You should arrive back at the prompt of the third host.

4. From the second session again, initiate the file transfer as:

scp -v -p -o port=33301 -o NoHostAuthenticationForLocalhost=yes \
        filename_remote username@localhost:[target_directory/][filename_on_carbon]
  • Replace username by the name of your Argonne account name.
  • Note the ":" at the end of the host name, which tells the scp command that the copy target is on a remote host.
  • The file named filename_remote will arrive in your Carbon home directory under this name, unless you specify a target_directory and/or a filename_on_carbon.

5. When done, press Ctrl-C in the first session to terminate the connection to mega.

Advanced: Speeding up logins using ssh keys

Our SSH gatway host itself does not accept SSH keys – you must always type your password for the initial connection to Mega.

Once you have established this initial connection and your tunnels are configured, you can use SSH keys to log in to Carbon login nodes without further passwords.

Create a key

  • Create an ssh key pair key if you do not already have one :
yourcomputer$ ssh-keygen -t rsa

The output will look like:

Generating public/private rsa key pair.
Enter file in which to save the key (/Users/home_id/.ssh/id_rsa): 
Enter passphrase (empty for no passphrase):              <-- Choose and enter a password.
Enter same passphrase again: 

Your identification has been saved in /Users/home_id/.ssh/id_rsa.
Your public key has been saved in /Users/home_id/.ssh/id_rsa.pub.
The key fingerprint is:
42:c3:72:fb:4c:be:c6:80:81:bc:8b:73:d8:88:84:2e home_id@yourcomputer.local
  • Inspect the .ssh directory on your computer. It should contain the following files:
$ ls -1 ~/.ssh
config
id_rsa
id_rsa.pub
known_hosts
random_seed
id_* are the just created key files.

Using an ssh-agent

On modern systems, the mere presence of private keys with their default names triggers handling through ssh-agent and brings up a passphrase dialog at the start of the desktop session or when a key is first requested for an outgoing connection. This is the case on MacOS X since 10.5 (Leopard), where key management is secure and largely automated through its general keychain concept.

On older systems, you need to (a) set up the session to run under ssh agent and (b) request the passphrase upon login. Circumstances vary widely. Ask Google or your local admin for help. See also the page Getting started with SSH.

Place your public key on Carbon

Password dialog on connecting to carbon once keys are in place.

To productively use your ssh key pair, you need to transfer the public half to Carbon.

Requirements:

  • you have a public/private ssh key pair on your machine (called yourcomputer in this example).
  • the public file on yourcomputer is named ~/.ssh/id_rsa.pub or ~/.ssh/id_dsa.pub (either one will work).
  • the ssh host alias mega has been set up on yourcomputer in ~/.ssh/config and defines tunnels using LocalForward declarations.
  • the tunnel connection for mega is open, i.e., on yourcomputer, ssh mega runs in another terminal.
  • the ssh alias carbon has been set up as described above.
  • an interactive session ssh carbon succeeds.


Then, to transfer your public key from yourcomputer to the Carbon login nodes, type:

cat ~/.ssh/id_rsa.pub | ssh carbon "umask 033; mkdir -p .ssh; cat >> .ssh/authorized_keys"

This command will append the contents of your ssh public key on yourcomputer to an authorized_keys file on Carbon. The command above is more reliable than cut&paste across interactive terminal sessions. If you created a dsa-type key, or chose your own name for the key pair, you'll have to cat the appropriate file instead. Remember to use the public part of the key, i.e. the file id_foo.pub.

Troubleshooting

General connection issues

Run ssh with one or more -v options ("verbosity"), and record the output for diagnostic (copy&paste into a file). E.g.:

ssh -v -v [other options and arguments …]

If you cannot resolve the issue yourself, follow HPC/Support.

Public keys

If public keys do not work, use your password to gain access, then check the following:

Permissions
The directories and the authorized_keys file must, at the least, not be writable by anyone except yourself; other permissions are recommended to be restrictive as well:
login5$ ls -ld ~ ~/.ssh ~/.ssh/authorized_keys
drwxr-xr-x 28 argonne_id users 4096 Feb 22 13:52 /home/argonne_id
drwx------  4 argonne_id argonne_id 4096 Feb 18 21:25 /home/argonne_id/.ssh
-rw-------  1 argonne_id argonne_id  829 Oct  9 10:55 /home/argonne_id/.ssh/authorized_keys
To change:
login5$ chmod go-w ~
login5$ chmod go-rwx ~/.ssh ~/.ssh/authorized_keys
Corruption of authorized_keys file
The file should contain public keys in OpenSSH format, one per line, without additional line breaks. One way to check is:
yourcomputer$ cut -c1-50 ~/.ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAIEjdsfjJSAHkjh88y
ssh-dss AAAAB3NzaC1kc3MAAACBAkkjsadfJFg7EFjDJhWEjk

The output should look similar to the sample above. Use your favorite text editor to join erroneously broken lines. Note that there should not be anything like BEGIN RSA PRIVATE KEY, which would obviously indicate a private, rather than public key.

Verify that an ssh-agent is running

To verify that an ssh-agent is running and visible to your shell, check for the presence of the environment variable SSH_AUTH_SOCK. To list the keys it has loaded, use ssh-add -l (lowercase "L"). Typical output is as follows:

yourcomputer$ echo $SSH_AUTH_SOCK
/tmp/ssh-HCvZKt5478/agent.5478
yourcomputer$ ssh-add -l
1024 b9:65:98:c0:7c:25:da:b6:86:48:98:6b:2a:5a:9d:21 /Users/home_id/.ssh/id_rsa (RSA)