Files & Storage
- Each node in the cluster provides access to local disk storage and to a globally accessible filesystem mounted as /mnt/lustre on ARROW.
- Your home directory, the directory you are placed in when you log in, is /mnt/lustre/home/<user> on ARROW. Your home directory is accessible from all the login nodes and compute nodes. You should use this space for storing files you want to keep long term, such as source code, scripts, input data sets, etc.
Collaborative groups
A collaborative group can be set up on the TRACC cluster as an easy way for the group to create and manage a shared directory tree of files and programs. A TRACC systems administrator will create the base directory for the directory structure shared by the group. The group base directory will have permissions set to populate the group file structure with directories and files that can be read by all group members, yet still retain some security against accidental modification or deletion of files not owned by a group member working in a group directory.
An example base group directory has the following permissions set:
drwsrws--T 3 joe-user joes-group 32768 Oct 17 17:28 tfhrc
Note that the setuid and setgid flags, “s,” are in the directory permissions in the position where the execute, “x,” flag would normally appear, allowing the directory owner and group members to list (“ls”) the directory and change to the directory (“cd”). The setuid flag is ignored by Linux. The setgid flag causes new files and subdirectories created within the directory to inherit its group ID rather than the group ID of the user who created the file or subdirectory. This inheritance feature allows users, without first changing their group ID with the “newgrp” command, to create files and subdirectories that have group ID of the parent directory instead of the group ID of the user.
In short, the setgid flag allows group members to work in the group directory without doing anything special when creating files; they will be automatically readable by other group members.
The group directory also has the sticky bit set, indicated by a capitol T in the others execute flag position. When the sticky bit is set on a directory, and the group write permission flag is set, group members may create files in the directory, including new subdirectories, and modify and delete the files that they own, but they will not be able to change or delete the files owned by other group members.
In short, the sticky bit set on a group directory, allows users to work safely in collaboration in a group directory: they can create and change their own files, read and copy the files of others, but they cannot change or delete the files of other group members.
When a user creates a subdirectory in the group directory, it inherits the group ID, but the permissions will be assigned based on the user’s mask (umask). The default umask on the TRACC cluster sets read and execute permissions for group members and no permissions for others. It does not set the group write permission. When a group member creates a subdirectory in the group directory, other group members will be able to change into that directory, read the files in the directory, copy the files into another directory, but they will not be able to create files in a new subdirectory that they do not own, unless the owner sets the group write permission for the directory. In that case, it is good practice for the directory owner to also set the sticky bit on the directory to allow other group members to add files to the directory, but prevent them from accidentally changing or deleting files belonging to other group members (i.e. files that they do not own).
In short, to allow other group members to create and modify their own files in a new group subdirectory, but not the files of others do:
chmod g+w subdirectory.name
chmod +t subdirectory.name
Group membership and the group directory provide a relatively easy way for a collaborative group to share data and programs.