Transferring Files - Revision history

Ley: /* Text File Compatibility */

2025-12-06T01:04:20Z

Text File Compatibility

← Older revision		Revision as of 01:04, December 6, 2025
Line 13:		Line 13:

	dos2unix will have no effect on a Linux text file, so it's safe to use on any text file.		dos2unix will have no effect on a Linux text file, so it's safe to use on any text file.

			==GLOBUS Connect==

			This is being evaluated and tested as of late 2025.

Ley at 04:28, October 29, 2025

2025-10-29T04:28:59Z

← Older revision		Revision as of 04:28, October 29, 2025
Line 1:		Line 1:

			==File Transfer==
	Files can be transferred to and from the cluster using a client that supports the SFTP (SSH File Transfer Protocol) or with scp (Secure Copy Protocol). Free SSH or SFTP clients are available from https://www.putty.org for Windows, and are included with or available for MacOS, and most varieties of Linux/UNIX. Be sure your client is using port 22 on the login node to connect. This should be the default for most terminal and file transfer applications, but has proven to be a problem for some users unaware of the port number.		Files can be transferred to and from the cluster using a client that supports the SFTP (SSH File Transfer Protocol) or with scp (Secure Copy Protocol). Free SSH or SFTP clients are available from https://www.putty.org for Windows, and are included with or available for MacOS, and most varieties of Linux/UNIX. Be sure your client is using port 22 on the login node to connect. This should be the default for most terminal and file transfer applications, but has proven to be a problem for some users unaware of the port number.

Line 6:		Line 7:
	Some users have also reported success with [http://www.itefix.no/i2/cwrsync cwrsync], a prepackaged version of Cgywin and rsync on Windows. There are numerous other SSH and SFTP clients available, and all of them should work with the login nodes.		Some users have also reported success with [http://www.itefix.no/i2/cwrsync cwrsync], a prepackaged version of Cgywin and rsync on Windows. There are numerous other SSH and SFTP clients available, and all of them should work with the login nodes.

			==Text File Compatibility==
	Note that text files created under Windows may cause subtle issues. This is because Windows, MacOS, and Linux have different conventions for representing a [http://en.wikipedia.org/wiki/Newline newline] in ASCII text files. Some Windows text editors will allow you to save a file in the UNIX/Mac/Linux format before transferring the file to the cluster. Alternatively, you can use the Linux command dos2unix to remove the CR characters that Windows uses at the ends of lines, or the reverse utility unix2dos:		Note that text files created under Windows may cause subtle issues. This is because Windows, MacOS, and Linux have different conventions for representing a [http://en.wikipedia.org/wiki/Newline newline] in ASCII text files. Some Windows text editors will allow you to save a file in the UNIX/Mac/Linux format before transferring the file to the cluster. Alternatively, you can use the Linux command dos2unix to remove the CR characters that Windows uses at the ends of lines, or the reverse utility unix2dos:

Ley at 03:57, October 29, 2025

2025-10-29T03:57:40Z

← Older revision		Revision as of 03:57, October 29, 2025
Line 1:		Line 1:

	* Files can be transferred to and from the cluster using a client that supports the SFTP (SSH File Transfer Protocol) or with scp (Secure Copy Protocol). Free SFTP clients are available from https://www.putty.org for Windows, and are included with or available for MacOS, and most varieties of Linux/UNIX. Be sure your client is using port 22.		Files can be transferred to and from the cluster using a client that supports the SFTP (SSH File Transfer Protocol) or with scp (Secure Copy Protocol). Free SSH or SFTP clients are available from https://www.putty.org for Windows, and are included with or available for MacOS, and most varieties of Linux/UNIX. Be sure your client is using port 22 on the login node to connect. This should be the default for most terminal and file transfer applications, but has proven to be a problem for some users unaware of the port number.

	*Popular file transfer clients are WinSCP and FileZilla. You can download WinSCP from http://winscp.net and FileZilla from http://filezilla-project.org.		Popular file transfer clients are WinSCP and FileZilla. You can download WinSCP from http://winscp.net and FileZilla from http://filezilla-project.org. Even the "rsync" utility on a Linux or WSL machine can be used with the appropriate options. Contact us to learn more about the various options for file transfers.

	* Some users have also reported success with [http://www.itefix.no/i2/cwrsync cwrsync], a prepackaged version of Cgywin and rsync.		Some users have also reported success with [http://www.itefix.no/i2/cwrsync cwrsync], a prepackaged version of Cgywin and rsync on Windows. There are numerous other SSH and SFTP clients available, and all of them should work with the login nodes.

	~~=== Windows text files ===~~		Note that text files created under Windows may cause subtle issues. This is because Windows, MacOS, and Linux have different conventions for representing a [http://en.wikipedia.org/wiki/Newline newline] in ASCII text files. Some Windows text editors will allow you to save a file in the UNIX/Mac/Linux format before transferring the file to the cluster. Alternatively, you can use the Linux command dos2unix to remove the CR characters that Windows uses at the ends of lines, or the reverse utility unix2dos:

	Note that text files created under Windows may cause ~~problems on the cluster~~. This is because Windows and Linux have different conventions for representing a [http://en.wikipedia.org/wiki/Newline newline] in ASCII text files. Some Windows text editors will allow you to save a file in the UNIX/Linux format before transferring the file to the cluster. Alternatively, you can use the Linux command dos2unix to remove the CR characters that Windows uses at the ends of lines:

	dos2unix somefile		dos2unix somefile

	dos2unix will have no effect on a Linux text file, so it's safe to use on any text file.		dos2unix will have no effect on a Linux text file, so it's safe to use on any text file.

	~~You can determine whether a file is in Windows or Linux format with the command~~

	~~cat -v somefile~~

	~~Windows text files will have a ^M at the end of each line.~~
	~~<p>~~

Amiot at 21:40, April 14, 2025

2025-04-14T21:40:35Z

← Older revision		Revision as of 21:40, April 14, 2025
Line 20:		Line 20:
	Windows text files will have a ^M at the end of each line.		Windows text files will have a ^M at the end of each line.
	<p>		<p>

	~~:::::::::::::::[[New main\|'''Return To Main Page''']]<br >~~

Amiot at 01:23, March 9, 2025

2025-03-09T01:23:20Z

← Older revision		Revision as of 01:23, March 9, 2025
Line 19:		Line 19:

	Windows text files will have a ^M at the end of each line.		Windows text files will have a ^M at the end of each line.
	~~<!--~~
	<p>		<p>

	:::::::::::::::[[New main\|'''Return To Main Page''']]<br >		:::::::::::::::[[New main\|'''Return To Main Page''']]<br >

Amiot at 01:21, March 9, 2025

2025-03-09T01:21:59Z

← Older revision		Revision as of 01:21, March 9, 2025
Line 20:		Line 20:
	Windows text files will have a ^M at the end of each line.		Windows text files will have a ^M at the end of each line.
	<!--		<!--
			<p>

	:::::::::::::::[[New main\|'''Return To Main Page''']]<br >		:::::::::::::::[[New main\|'''Return To Main Page''']]<br >

Amiot at 01:20, March 9, 2025

2025-03-09T01:20:38Z

← Older revision		Revision as of 01:20, March 9, 2025
Line 20:		Line 20:
	Windows text files will have a ^M at the end of each line.		Windows text files will have a ^M at the end of each line.
	<!--		<!--


			:::::::::::::::[[New main\|'''Return To Main Page''']]<br >

Amiot at 17:41, June 9, 2021

2021-06-09T17:41:07Z

← Older revision		Revision as of 17:41, June 9, 2021
Line 1:		Line 1:

	* Files can be transferred to and from the cluster using a client that supports the SFTP (SSH File Transfer Protocol) or with scp (Secure Copy Protocol). [[Free https://www.putty.org ~~\| Free SFTP clients]] are available~~ for Windows, and are included with or available for MacOS, and most varieties of Linux/UNIX. Be sure your client is using port 22.		* Files can be transferred to and from the cluster using a client that supports the SFTP (SSH File Transfer Protocol) or with scp (Secure Copy Protocol). Free SFTP clients are available from https://www.putty.org for Windows, and are included with or available for MacOS, and most varieties of Linux/UNIX. Be sure your client is using port 22.

	*Popular file transfer clients are WinSCP and FileZilla. You can download WinSCP from http://winscp.net and FileZilla from http://filezilla-project.org.		*Popular file transfer clients are WinSCP and FileZilla. You can download WinSCP from http://winscp.net and FileZilla from http://filezilla-project.org.

Amiot at 17:39, June 9, 2021

2021-06-09T17:39:11Z

← Older revision		Revision as of 17:39, June 9, 2021
Line 1:		Line 1:

	* Files can be transferred to and from the cluster using a client that supports the SFTP (SSH File Transfer Protocol) or with scp (Secure Copy Protocol). [[Free ~~SSH clients~~ \| Free SFTP clients]] are available for Windows, and are included with or available for MacOS, and most varieties of Linux/UNIX. Be sure your client is using port 22.		* Files can be transferred to and from the cluster using a client that supports the SFTP (SSH File Transfer Protocol) or with scp (Secure Copy Protocol). [[Free https://www.putty.org \| Free SFTP clients]] are available for Windows, and are included with or available for MacOS, and most varieties of Linux/UNIX. Be sure your client is using port 22.

	*Popular file transfer clients are WinSCP and FileZilla. You can download WinSCP from http://winscp.net and FileZilla from http://filezilla-project.org.		*Popular file transfer clients are WinSCP and FileZilla. You can download WinSCP from http://winscp.net and FileZilla from http://filezilla-project.org.

Amiot at 19:52, May 18, 2021

2021-05-18T19:52:50Z

← Older revision		Revision as of 19:52, May 18, 2021
Line 20:		Line 20:
	Windows text files will have a ^M at the end of each line.		Windows text files will have a ^M at the end of each line.
	<!--		<!--
	~~=== Transferring files from Phoenix to Zephyr ===~~
	I don't think we need to transfer files within the cluster anymore so I commented out all the code. Take a look at what is commented out and if we all agree we can remove this whole section........Larry ...........I agree, Steve


	~~An easy and efficient way to transfer files is using the tool '''rsync''' over ssh.~~

	The following example creates a "jobs" directory on Zephyr and transfers the '''hpl_benchmark''' directory along with its content from the jobs directory located in the users "home" directory on the Phoenix cluster to the created "jobs" directory on the Zephyr cluster.

	~~# Login to login.zephyr.tracc.anl.gov~~
	~~# rsync -avze ssh login.phoenix.tracc.anl.gov:~/jobs/hpl_benchmark ~/jobs/~~

	~~For more complicated transfers, i.e collaborative groups, please contact us.~~



	~~The following example shows how to transfer a project file called BridgeModel.cad. First login to Zephyr:~~

	~~ssh userid@login.zephyr.tracc.anl.gov~~

	~~'''Then you must sftp to:'''~~

	~~login.phoenix.tracc.anl.gov~~


	~~'''To Establish the connection:'''~~

	~~sftp jdoe@login.phoenix.tracc.anl.gov <br>~~
	~~Connecting to login.phoenix.tracc.anl.gov... <br>~~
	~~jdoe@login.phoenix.tracc.anl.gov's password: <br>~~

	~~'''Once connected, change to desired directory:''' <br>~~
	~~sftp> cd project <br>~~
	~~sftp> ls <br>~~
	~~BridgeModel.cad <br>~~
	~~sftp> get BridgeModel.cad <br>~~
	~~Fetching /mnt/gluster/home/jdoe/project/BridgeModel.cad to BridgeModel.cad <br>~~
	~~/mnt/gluster/home/jdoe/project/BridgeModel.cad 100% 100KB 100.0KB/s 00:00 <br>~~
	~~sftp> quit <br>~~

	~~[jdoe@login1 ~]$ ls -l BridgeModel.cad <br>~~
	~~-rw-r----- 1 jdoe jdoe 102400 Sep 24 20:30 BridgeModel.cad~~
	~~-->~~