Data Storage: Difference between revisions

From HELIOS Digital DAQ
Jump to navigation Jump to search
 
(4 intermediate revisions by the same user not shown)
Line 5: Line 5:


at Nov-11-2019, there are 3 Hard disks
at Nov-11-2019, there are 3 Hard disks
  /media/DIGIOSDATA4 -- 3TB, used
  /media/DIGIOSDATA4 -- 3TB, used (245GB left, 2020-10, after 16N isomer exp.)
  /media/DIGIOSDATA5 -- 6TB, used (from H077_29Al, 2020-11)
  /media/DIGIOSDATA5 -- 6TB, used (from H077_29Al, 2020-11)
  /media/DIGIOSDATA6 -- 6TB, empty (borrow to MUSIC, 2020-11-21)
  /media/DIGIOSDATA6 -- 6TB, empty (borrow to MUSIC, 2020-11-21)


After each run, the previous run will be push to LCRC. see [[Globus]]
After each run, the previous run will be push to LCRC. see [[Globus]]
== mount disk ==
This is a temporary mount disk method, the disk will not be mounted after restart.
Suppose the Harddisk appeared on /dev/sdd
you wish to mount to /mount/DIGIOSDATA6 (assume you created a mount point by mkdir)
the HD is formatted as ext4
the command to mount is:
sudo mount -t ext4 -o rw /dev/sdd /mount/DIGIOSDATA6
if you mount as
sudo mount /dev/sdd /mount/DIGIOSDATA6
the HD is only read but NOT write.
== mount disk at startup ==
1) Get name, UUID, file system Type of detected disks
[root@digios1 helios]# blkid
/dev/sda1: UUID="6afd3d3e-d944-44d8-95cf-64d76118763a" TYPE="ext4"
/dev/sda2: UUID="99b398a6-20c6-4f2b-b680-a4bc8afce1c4" TYPE="ext4"
/dev/sda3: LABEL="_SL-62-i386-Live" UUID="dbccbb90-6026-4913-931a-b71e169d17ed" TYPE="ext4"
/dev/sda5: UUID="de9962db-d5ec-4f07-8da7-3ef50c257135" TYPE="swap"
/dev/sda6: UUID="958f0586-b338-4255-ab62-a86447e549e4" TYPE="ext4"
/dev/sdc: LABEL="DIGIOSDATA5" UUID="48e123eb-7613-4ef4-b9e4-eb53283ddcdf" TYPE="ext4"
/dev/sdb: UUID="575cde2b-1adf-4bf0-af91-74601ef294c1" TYPE="ext4"
2) make mount point using mkdir
/media/DIGIOSDATA<X>
make sure chmod, such as
drwxrwxrwx. 14 root  root  4.0K Aug 25 12:06 DIGIOSDATA4
drwxrwxrwx.  5 helios helios 4.0K Nov 11 21:07 DIGIOSDATA5
3) edit /etc/fstab, here is the content of the file
#
# /etc/fstab
# Created by anaconda on Wed Jan 30 13:24:43 2013
# Last edit by Ryan on Sat Nov 21 16:42 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=dbccbb90-6026-4913-931a-b71e169d17ed /                      ext4    defaults        1 1
UUID=6afd3d3e-d944-44d8-95cf-64d76118763a /boot                  ext4    defaults        1 2
UUID=958f0586-b338-4255-ab62-a86447e549e4 /global                ext4    defaults        1 2
UUID=99b398a6-20c6-4f2b-b680-a4bc8afce1c4 /home                  ext4    defaults        1 2
UUID=de9962db-d5ec-4f07-8da7-3ef50c257135 swap                    swap    defaults        0 0
UUID=575cde2b-1adf-4bf0-af91-74601ef294c1 /media/DIGIOSDATA4      ext4    defaults        0 0
UUID=48e123eb-7613-4ef4-b9e4-eb53283ddcdf /media/DIGIOSDATA5      ext4    defaults        0 0
#UUID=43f42e85-fd83-42d7-8e4b-46323a858cc3 /media/DIGIOSDATA6      ext4    defaults        0 0
tmpfs                  /dev/shm                tmpfs  defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                  /sys                    sysfs  defaults        0 0
proc                    /proc                  proc    defaults        0 0
/dev/sdb                /media/DIGIOSDATA4      ext4    users,rw,owner,umask=000 0 0
/dev/sdc                /media/DIGIOSDATA5      ext4    users,rw,owner,umask=000 0 0
#/dev/sdd              /media/DIGIOSDATA6      ext4    users,rw,owner,umask=000 0 0
#/dev/sdd              /media/DIGIOSDATA3      vfat    users,rw,owner,umask=000 0 0


= in MAC =  
= in MAC =  

Latest revision as of 22:56, November 21, 2020

in DAQ

The raw data is stored in

/media/

at Nov-11-2019, there are 3 Hard disks

/media/DIGIOSDATA4 -- 3TB, used (245GB left, 2020-10, after 16N isomer exp.)
/media/DIGIOSDATA5 -- 6TB, used (from H077_29Al, 2020-11)
/media/DIGIOSDATA6 -- 6TB, empty (borrow to MUSIC, 2020-11-21)

After each run, the previous run will be push to LCRC. see Globus

mount disk

This is a temporary mount disk method, the disk will not be mounted after restart.

Suppose the Harddisk appeared on /dev/sdd

you wish to mount to /mount/DIGIOSDATA6 (assume you created a mount point by mkdir)

the HD is formatted as ext4

the command to mount is:

sudo mount -t ext4 -o rw /dev/sdd /mount/DIGIOSDATA6

if you mount as

sudo mount /dev/sdd /mount/DIGIOSDATA6

the HD is only read but NOT write.

mount disk at startup

1) Get name, UUID, file system Type of detected disks

[root@digios1 helios]# blkid
/dev/sda1: UUID="6afd3d3e-d944-44d8-95cf-64d76118763a" TYPE="ext4" 
/dev/sda2: UUID="99b398a6-20c6-4f2b-b680-a4bc8afce1c4" TYPE="ext4" 
/dev/sda3: LABEL="_SL-62-i386-Live" UUID="dbccbb90-6026-4913-931a-b71e169d17ed" TYPE="ext4" 
/dev/sda5: UUID="de9962db-d5ec-4f07-8da7-3ef50c257135" TYPE="swap" 
/dev/sda6: UUID="958f0586-b338-4255-ab62-a86447e549e4" TYPE="ext4" 
/dev/sdc: LABEL="DIGIOSDATA5" UUID="48e123eb-7613-4ef4-b9e4-eb53283ddcdf" TYPE="ext4" 
/dev/sdb: UUID="575cde2b-1adf-4bf0-af91-74601ef294c1" TYPE="ext4" 

2) make mount point using mkdir

/media/DIGIOSDATA<X>

make sure chmod, such as

drwxrwxrwx. 14 root   root   4.0K Aug 25 12:06 DIGIOSDATA4
drwxrwxrwx.  5 helios helios 4.0K Nov 11 21:07 DIGIOSDATA5

3) edit /etc/fstab, here is the content of the file

#
# /etc/fstab
# Created by anaconda on Wed Jan 30 13:24:43 2013
# Last edit by Ryan on Sat Nov 21 16:42 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
UUID=dbccbb90-6026-4913-931a-b71e169d17ed /                       ext4    defaults        1 1
UUID=6afd3d3e-d944-44d8-95cf-64d76118763a /boot                   ext4    defaults        1 2
UUID=958f0586-b338-4255-ab62-a86447e549e4 /global                 ext4    defaults        1 2
UUID=99b398a6-20c6-4f2b-b680-a4bc8afce1c4 /home                   ext4    defaults        1 2
UUID=de9962db-d5ec-4f07-8da7-3ef50c257135 swap                    swap    defaults        0 0
UUID=575cde2b-1adf-4bf0-af91-74601ef294c1 /media/DIGIOSDATA4      ext4    defaults        0 0
UUID=48e123eb-7613-4ef4-b9e4-eb53283ddcdf /media/DIGIOSDATA5      ext4    defaults        0 0
#UUID=43f42e85-fd83-42d7-8e4b-46323a858cc3 /media/DIGIOSDATA6      ext4    defaults        0 0

tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0 

/dev/sdb                /media/DIGIOSDATA4      ext4    users,rw,owner,umask=000 0 0
/dev/sdc                /media/DIGIOSDATA5      ext4    users,rw,owner,umask=000 0 0
#/dev/sdd               /media/DIGIOSDATA6      ext4    users,rw,owner,umask=000 0 0
#/dev/sdd               /media/DIGIOSDATA3      vfat    users,rw,owner,umask=000 0 0

in MAC

The data will be donwloaded at

~/digios/analysis/data

in LCRC

The experimental data is stored at

/lcrc/project/HELIOS/experimentsData/

= Current folder structure

Each experiment data will be stored inside the folder with the experiment ID.

For example, h070_146Nd, in DAQ will be stored in

/media/DIGIOSDATA4/h070_146Nd/

Under this folder, there are 3 sub-folders

/media/DIGIOSDATA4/h070_146Nd/data
/media/DIGIOSDATA4/h070_146Nd/merged_data
/media/DIGIOSDATA4/h070_146Nd/root_data