Transferring Files: Difference between revisions

From TRACC Wiki
Jump to navigation Jump to search
No edit summary
 
(2 intermediate revisions by the same user not shown)
Line 1: Line 1:


* Files can be transferred to and from the cluster using a client that supports the SFTP (SSH File Transfer Protocol) or with scp (Secure Copy Protocol). [[Free SSH clients | Free SFTP clients]] are available for Windows, and are included with or available for MacOS, and most varieties of Linux/UNIX. Be sure your client is using port 22.
* Files can be transferred to and from the cluster using a client that supports the SFTP (SSH File Transfer Protocol) or with scp (Secure Copy Protocol). Free SFTP clients are available from https://www.putty.org for Windows, and are included with or available for MacOS, and most varieties of Linux/UNIX. Be sure your client is using port 22.


*Popular file transfer clients are WinSCP and FileZilla. You can download WinSCP from http://winscp.net and FileZilla from http://filezilla-project.org.
*Popular file transfer clients are WinSCP and FileZilla. You can download WinSCP from http://winscp.net and FileZilla from http://filezilla-project.org.
Line 20: Line 20:
Windows text files will have a ^M at the end of each line.
Windows text files will have a ^M at the end of each line.
<!--
<!--
=== Transferring files from Phoenix to Zephyr ===
I don't think we need to transfer files within the cluster anymore so I commented out all the code. Take a look at what is commented out and if we all agree we can remove this whole section........Larry ...........I agree, Steve
An easy and efficient way to transfer files is using the tool '''rsync''' over ssh.
The following example creates a "jobs" directory on Zephyr and transfers the '''hpl_benchmark''' directory along with its content from the jobs directory located in the users "home" directory on the Phoenix cluster to the created "jobs" directory on the Zephyr cluster.
# Login to login.zephyr.tracc.anl.gov
# rsync -avze ssh login.phoenix.tracc.anl.gov:~/jobs/hpl_benchmark ~/jobs/
For more complicated transfers, i.e collaborative groups, please contact us. 
The following example shows how to transfer a project file called BridgeModel.cad. First login to Zephyr:
'''Then you must sftp to:'''
    login.phoenix.tracc.anl.gov
'''To Establish the connection:'''
Connecting to login.phoenix.tracc.anl.gov... <br>
[email protected]'s password: <br>
'''Once connected, change to desired directory:''' <br>
sftp> cd project <br>
sftp> ls <br>
BridgeModel.cad <br>
sftp> get BridgeModel.cad <br>
Fetching /mnt/gluster/home/jdoe/project/BridgeModel.cad to BridgeModel.cad <br>
/mnt/gluster/home/jdoe/project/BridgeModel.cad 100% 100KB 100.0KB/s 00:00 <br>
sftp> quit <br>
[jdoe@login1 ~]$ ls -l BridgeModel.cad <br>
-rw-r----- 1 jdoe jdoe 102400 Sep 24 20:30 BridgeModel.cad
-->

Latest revision as of 17:41, June 9, 2021

  • Files can be transferred to and from the cluster using a client that supports the SFTP (SSH File Transfer Protocol) or with scp (Secure Copy Protocol). Free SFTP clients are available from https://www.putty.org for Windows, and are included with or available for MacOS, and most varieties of Linux/UNIX. Be sure your client is using port 22.
  • Some users have also reported success with cwrsync, a prepackaged version of Cgywin and rsync.

Windows text files

Note that text files created under Windows may cause problems on the cluster. This is because Windows and Linux have different conventions for representing a newline in ASCII text files. Some Windows text editors will allow you to save a file in the UNIX/Linux format before transferring the file to the cluster. Alternatively, you can use the Linux command dos2unix to remove the CR characters that Windows uses at the ends of lines: 

dos2unix somefile

dos2unix will have no effect on a Linux text file, so it's safe to use on any text file.

You can determine whether a file is in Windows or Linux format with the command 

cat -v somefile

Windows text files will have a ^M at the end of each line.

Retrieved from "https://wiki.anl.gov/wiki_tracc/index.php?title=Transferring_Files&oldid=1930"