HPC/Linux Guides: Difference between revisions

From CNM Wiki
< HPC
Jump to navigation Jump to search
 
(28 intermediate revisions by the same user not shown)
Line 1: Line 1:
To use ''Carbon'' and other HPC systems, you will need to become familiar with many Unix/Linux concepts and ways of doing things.
{| class="wikitable" cellpadding="1" style="float: right; text-align:center;"
You will be navigating directories, create, view, and edit files, and run commands on the ''shell'' command line and in scripts.
| __TOC__
As you become comfortable and learn about more commands and shell features, you will find that you will work increasingly efficiently.
|}
To use ''Carbon'' and other HPC systems, you will need to become familiar with the Unix/Linux ways of doing things.


There are numerous resources to get you started, and some are shown below. Free web pages and PDFs provide the extremes of coverage and style – both short introductions and good reference materials, but for the purpose of active learning it will be best to purchase a book in print or ebook form, or attend an instructor-led class.
You will be navigating directories, create, view, and edit various text files, and run commands on the shell command line and in scripts.
As you become comfortable and learn more about commands and shell features you will find that you get much more efficient.
 
There are numerous resources to get you started; some are shown below.
 
; Free vs. paid content – Choose wisely!:
:* Web pages and PDFs that are available at no cost usually contain either short ''introductions'' or more or less detailed ''reference'' materials. Those are relatively straightforward to put together from existing sources, but when compared to a textbook or course on the subject, they would constitute only the first and last chapters.
:* Materials created for ''teaching'', by contrast, will present the entirety of the subject matter, and at a gradual learning curve. This requires a great deal more effort and time to assemble, and thus, these materials are usually available upon purchase.


== Beginners ==
== Beginners ==
<br>
[[Image:HPC 2012-02-04 Book cover small - The Linux Command Line.png|100px|right]]
[[Image:HPC 2012-02-04 Book cover small - The Linux Command Line.png|100px|right]]
; [http://nostarch.com/tlcl.htm The Linux Command Line – A Complete Introduction]:
; [https://nostarch.com/tlcl2 The Linux Command Line, 2nd Edition – A Complete Introduction]:
:* By William E. Shotts Jr.; No Starch Press, January 2012; about $30 – $40 in print or as ebook.
:* by William Shotts; No Starch Press, March 2019, 504 pp.; ISBN-13: 9781593279523; about $25 – $30 as ebook ([https://nostarch.com/tlcl2 from the publisher] or [https://books.apple.com/us/book/the-linux-command-line-2nd-edition/id1438349057 here]), or about $40 in print.
: Fairly gentle tutorial introducing Linux in general and command line usage in particular (brand new as of 2012, and promising). --[[User:Stern|stern]]
:–– Fairly gentle tutorial introducing Linux in general and command line usage in particular.
<!--
<!--
; [http://en.wikibooks.org/wiki/Guide_to_Unix Guide to Unix Computing (Wikibook)], section [http://en.wikibooks.org/wiki/Guide_to_Unix/Explanations/Shell_Prompt Shell Prompt]:
; [http://en.wikibooks.org/wiki/Guide_to_Unix Guide to Unix Computing (Wikibook)], section [http://en.wikibooks.org/wiki/Guide_to_Unix/Explanations/Shell_Prompt Shell Prompt]:
: "… describes Unix and Unix-like systems for users and system administrators. We include a guide to commands which lists several of the common shell commands."
: "… describes Unix and Unix-like systems for users and system administrators. We include a guide to commands which lists several of the common shell commands."
-- incomplete and quickly hairy past the first page (stern)
:–– incomplete and quickly hairy past the first page.
-->
-->
; [http://moo.nac.uci.edu/~hjm/ManipulatingDataOnLinux.html Manipulating Data on Linux] – [https://hjmangalam.wordpress.com/2009/09/14/manipulating-data-on-linux/. (alternate link)]:
:* By [http://moo.nac.uci.edu/~hjm/Mangalam_2014.html Harry Mangalam], UC Irvine.
:–– Very good introduction to ''concepts'' and ''tools'', especially for persons coming from a Windows background.
; [http://vic.gedris.org/Manual-ShellIntro/1.2/ShellIntro.pdf Introduction to the Linux Command Shell For Beginners (pdf)]:
; [http://vic.gedris.org/Manual-ShellIntro/1.2/ShellIntro.pdf Introduction to the Linux Command Shell For Beginners (pdf)]:
: Provides a "fast and simple introduction to using the Linux command shell and some of its basic utilities. Assumes very little or even no prior exposure to the Linux command prompt. References to DOS commands are made where appropriate. This document was meant to accompany an instructor-led tutorial on this subject, and therefore some details have been left out."
: Provides a "fast and simple introduction to using the Linux command shell and some of its basic utilities. Assumes very little or even no prior exposure to the Linux command prompt. References to DOS commands are made where appropriate. This document was meant to accompany an instructor-led tutorial on this subject, and therefore some details have been left out."
: Very short, but does cover the basics of managing files and directories, and touches on key concepts like redirections and pipes. --[[User:Stern|stern]]
:–– Very short, but does cover the basics of managing files and directories, and touches on key concepts like redirections and pipes.
; [http://www.gnu.org/software/bash/manual/ Bash Reference Manual]:
 
: "A brief introduction to features found in Bash." For a definitive reference on shell behavior please review the Bash manual page.
 
[[Image:HPC 2012-02-04 Book cover small - Learning the bash.gif|100px|right]]
; [http://my.safaribooksonline.com/book/operating-systems-and-server-administration/unix/0596009658 Learning the bash Shell – Unix Shell Programming]:
:* By Cameron Newham; O'Reilly, March 2005; about $30–$50.
:–– Valuable introduction, with exercises.
 
 
; [http://tldp.org/LDP/Bash-Beginners-Guide/html/ The Bash Guide for Beginners]:
; [http://tldp.org/LDP/Bash-Beginners-Guide/html/ The Bash Guide for Beginners]:
: "Gets you started with Bash scripting…. Everybody who wants to make life easier on themselves, power users and sysadmins alike, can benefit from reading this practical course. The guide contains lots of examples and exercises at the end of each chapter, demonstrating the theory and helping you practice."
:–– "Gets you started with Bash scripting…. Everybody who wants to make life easier on themselves, power users and sysadmins alike, can benefit from reading this practical course. The guide contains lots of examples and exercises at the end of each chapter, demonstrating the theory and helping you practice."
 
 
<!--
=== Classes ===
Onsite users can benefit from classes occasionally offered by CIS.
* [http://inside.anl.gov/event/beginning-unix-cis-564 Beginning UNIX (CIS 564)]
* [http://inside.anl.gov/event/vi-editor-unix-cis-567 vi Editor in UNIX (CIS 567)]
-->


== Intermediate ==
== Intermediate ==
; <code>man bash</code>:
; <code>man bash</code>
: The Bash manual page is the definitive resource on capabilities and syntax. Covers all details of bash precisely and concisely.
; <code>help ''cmd''</code>:
: ''Version note:'' The man command as shown gives the version relevant for your current system. See http://bashcookbook.com/bashinfo/#bash for other versions. Simple scripting concepts are applicable for all versions, but more recent ones add capabilities to make scripting more flexible, such as regular expression matching, array variables, and arithmetic loops. --[[User:Stern|stern]]
: The Bash manual page is the definitive resource on capabilities and syntax. It covers all details of bash precisely and concisely. The <code>help</code> command gives the paragraph from the man page (or a close version thereof) relevant to the bash builtin ''cmd''.
: ''Version note:'' A couple of major versions of Bash are in concurrent popular use, mostly Bash-3, phasing in Bash-4. While simple scripting concepts are applicable to all versions, each version introduces capabilities for more versatile scripting, such as array variables, arithmetic loops, and regular expression matching. The <code>man</code> command always shows the capabilities of the version in use on your system. See http://bashcookbook.com/bashinfo/#bash for other versions. --[[User:Stern|stern]]
; [http://www.gnu.org/software/bash/manual/ Bash Reference Manual]:
: "A brief introduction to features found in Bash." For a definitive reference on shell behavior please review the Bash manual page.
[[Image:HPC 2012-02-04 Book cover small - bash Cookbook.gif|100px|right]]
[[Image:HPC 2012-02-04 Book cover small - bash Cookbook.gif|100px|right]]
; [http://tldp.org/LDP/abs/html/  Advanced Bash-Scripting Guide]:
: "This document is both a tutorial and a reference on shell scripting with Bash. It assumes no previous knowledge of scripting or programming, but progresses rapidly toward an intermediate/advanced level of instruction. The exercises and heavily-commented examples invite active reader participation."
; [http://oreilly.com/catalog/bashckbk/ bash Cookbook – Solutions and Examples for bash Users]:
; [http://oreilly.com/catalog/bashckbk/ bash Cookbook – Solutions and Examples for bash Users]:
:* By Carl Albing, JP Vossen, Cameron Newham; O'Reilly, May 2007; about $30–$50.
:* By Carl Albing, JP Vossen, Cameron Newham; O'Reilly, May 2007; about $30–$50.
:* related materials at http://bashcookbook.com/
:* Related materials at http://bashcookbook.com/


== Advanced ==
== Advanced ==
; [http://tldp.org/LDP/abs/html/  Advanced Bash-Scripting Guide]:
: "This document is both a tutorial and a reference on shell scripting with Bash. It assumes no previous knowledge of scripting or programming, but progresses rapidly toward an intermediate/advanced level of instruction. The exercises and heavily-commented examples invite active reader participation."
[[Image:HPC 2012-02-04 Book cover small - Unix Power Tools.jpg|100px|right]]
[[Image:HPC 2012-02-04 Book cover small - Unix Power Tools.jpg|100px|right]]
; [http://my.safaribooksonline.com/book/operating-systems-and-server-administration/unix/0596003307 Unix Power Tools, Third Edition]:
; [http://my.safaribooksonline.com/book/operating-systems-and-server-administration/unix/0596003307 Unix Power Tools]:
:* By Jerry Peek, Shelley Powers, Tim O'Reilly, Mike Loukides; O'Reilly, 2002. About $40–$70 in print or as ebook.
:* By Jerry Peek, Shelley Powers, Tim O'Reilly, Mike Loukides; O'Reilly, Third Edition 2002. About $40–$70 in print or as ebook.
; [http://bashcookbook.com/bashinfo/ The Comprehensive List of bash Reference Documentation and Examples]:
; [http://bashcookbook.com/bashinfo/ The Comprehensive List of bash Reference Documentation and Examples]:
: Various documentation resources, sources, snippets, metadata across multiple versions (from the [http://bashcookbook.com/ bash Cookbook wiki]).
: Various documentation resources, sources, snippets, metadata across multiple versions (from the [http://bashcookbook.com/ bash Cookbook wiki]).
; [http://stackoverflow.com/a/1220118/5999218 What's your most productive shortcut with vim?]
: A question asked on [http://stackoverflow.com Stackoverflow], with an enlightening answer.

Latest revision as of 02:52, October 10, 2020

To use Carbon and other HPC systems, you will need to become familiar with the Unix/Linux ways of doing things.

You will be navigating directories, create, view, and edit various text files, and run commands on the shell command line and in scripts. As you become comfortable and learn more about commands and shell features you will find that you get much more efficient.

There are numerous resources to get you started; some are shown below.

Free vs. paid content – Choose wisely!
  • Web pages and PDFs that are available at no cost usually contain either short introductions or more or less detailed reference materials. Those are relatively straightforward to put together from existing sources, but when compared to a textbook or course on the subject, they would constitute only the first and last chapters.
  • Materials created for teaching, by contrast, will present the entirety of the subject matter, and at a gradual learning curve. This requires a great deal more effort and time to assemble, and thus, these materials are usually available upon purchase.

Beginners


HPC 2012-02-04 Book cover small - The Linux Command Line.png
The Linux Command Line, 2nd Edition – A Complete Introduction
  • by William Shotts; No Starch Press, March 2019, 504 pp.; ISBN-13: 9781593279523; about $25 – $30 as ebook (from the publisher or here), or about $40 in print.
–– Fairly gentle tutorial introducing Linux in general and command line usage in particular.


Manipulating Data on Linux – (alternate link)
–– Very good introduction to concepts and tools, especially for persons coming from a Windows background.


Introduction to the Linux Command Shell For Beginners (pdf)
Provides a "fast and simple introduction to using the Linux command shell and some of its basic utilities. Assumes very little or even no prior exposure to the Linux command prompt. References to DOS commands are made where appropriate. This document was meant to accompany an instructor-led tutorial on this subject, and therefore some details have been left out."
–– Very short, but does cover the basics of managing files and directories, and touches on key concepts like redirections and pipes.


HPC 2012-02-04 Book cover small - Learning the bash.gif
Learning the bash Shell – Unix Shell Programming
  • By Cameron Newham; O'Reilly, March 2005; about $30–$50.
–– Valuable introduction, with exercises.


The Bash Guide for Beginners
–– "Gets you started with Bash scripting…. Everybody who wants to make life easier on themselves, power users and sysadmins alike, can benefit from reading this practical course. The guide contains lots of examples and exercises at the end of each chapter, demonstrating the theory and helping you practice."


Intermediate

man bash
help cmd
The Bash manual page is the definitive resource on capabilities and syntax. It covers all details of bash precisely and concisely. The help command gives the paragraph from the man page (or a close version thereof) relevant to the bash builtin cmd.
Version note: A couple of major versions of Bash are in concurrent popular use, mostly Bash-3, phasing in Bash-4. While simple scripting concepts are applicable to all versions, each version introduces capabilities for more versatile scripting, such as array variables, arithmetic loops, and regular expression matching. The man command always shows the capabilities of the version in use on your system. See http://bashcookbook.com/bashinfo/#bash for other versions. --stern
Bash Reference Manual
"A brief introduction to features found in Bash." For a definitive reference on shell behavior please review the Bash manual page.
HPC 2012-02-04 Book cover small - bash Cookbook.gif
Advanced Bash-Scripting Guide
"This document is both a tutorial and a reference on shell scripting with Bash. It assumes no previous knowledge of scripting or programming, but progresses rapidly toward an intermediate/advanced level of instruction. The exercises and heavily-commented examples invite active reader participation."
bash Cookbook – Solutions and Examples for bash Users
  • By Carl Albing, JP Vossen, Cameron Newham; O'Reilly, May 2007; about $30–$50.
  • Related materials at http://bashcookbook.com/

Advanced

HPC 2012-02-04 Book cover small - Unix Power Tools.jpg
Unix Power Tools
  • By Jerry Peek, Shelley Powers, Tim O'Reilly, Mike Loukides; O'Reilly, Third Edition 2002. About $40–$70 in print or as ebook.
The Comprehensive List of bash Reference Documentation and Examples
Various documentation resources, sources, snippets, metadata across multiple versions (from the bash Cookbook wiki).
What's your most productive shortcut with vim?
A question asked on Stackoverflow, with an enlightening answer.
Retrieved from "https://wiki.anl.gov/wiki_cnm/index.php?title=HPC/Linux_Guides&oldid=6584"