Computers: Difference between revisions

From GRETINA at ANL
Jump to navigation Jump to search
No edit summary
Line 95: Line 95:
The computers above get their password file via NIS from a1.gam. Not all computers exists
The computers above get their password file via NIS from a1.gam. Not all computers exists


==Digiticer IOCs==
==Digitizer IOCs==


  10.0.3.1 vme01
  10.0.3.1 vme01

Revision as of 20:55, October 4, 2020

NAT

10.0.0.1 # cisco firewall, default gateway

defines the 10.0.x.x private network for GT (netmask 255.255.0.0 or /16). The WAN name for this NAT is gt1.phy.anl.gov. Note: this NATBOX is not the standard GT ASA box; but the Cisco NAT box we use many places at ANL.

Decomposition computers

10.0.1.100              n0000 n0000.gam
10.0.1.1                n0001 n0001.gam
.
.
10.0.1.62               n0062 n0062.gam

n0000 is the Global Event Builder (GEB). The other ~60 nodes are decomposition computers. A few of them are dead.

You can only ssh to them from certain machines, like a1 and a2, and you must be user 'gretina'

mysterious IP addresses

10.2.0.1   ipmi-master_eth
10.2.0.200 ipmi-master
10.2.0.98  ipmi-a1.gam
10.2.0.52  fs1-bmc
10.2.0.154 ipmi-r610

# IPMI
#
10.2.1.100              ipmi-n0000
10.2.1.1                ipmi-n0001
10.2.1.2                ipmi-n0002
10.2.1.3                ipmi-n0003
.
.
10.2.1.58               ipmi-n0058
10.2.1.59               ipmi-n0059
10.2.1.60               ipmi-n0060
10.2.1.61               ipmi-n0061
10.2.1.62               ipmi-n0062

storage nodes

10.0.1.168		n0168 n0168.gam #storage node
10.0.1.169		n0169 n0169.gam #storage node

service/analysis computers

10.0.0.5	dell-6248-master # Dell swith, master, middle rack
10.0.0.7	dell-6248-shack # Dell swith, up on the shack

10.0.0.79	a0.gam a0 a0.gam.nscl.msu.edu
10.0.0.98	a1.gam.nscl.msu.edu a1.gam
10.0.0.99	a2.gam a2 a2.gam.nscl.msu.edu

The role of a1.gam is that of the NIS server, but like a2.gam it is also used for general software developments. At the moment we run all the DAQ control screens (and soft IOCs) from a1.gam 

10.0.0.10	dogs.gam dogs dogs.gam.nscl.msu.edu
10.0.0.14      master1
10.0.0.15      master2

the decomposition machines get their systems from dogs (true?) - master1 I think 

10.0.0.50	fs.gam fs fs.gam.nscl.msu.edu
10.0.0.51	fs2.gam fs2 fs2.gam.nscl.msu.edu
10.0.0.52	fs1.gam fs1 fs1.gam.nscl.msu.edu

the root host machine for fs1.gam is a1 (we think) 

10.0.0.70	spdaq44 spdaq44.gam
10.0.0.71	spdaq38 spdaq38.gam
10.0.0.85	op0.gam op0 op0.gam.nscl.msu.edu
10.0.0.86	op1.gam op1 op1.gam.nscl.msu.edu
10.0.0.29	op2.gam op2 op2.gam.nscl.msu.edu
10.0.0.30	op3.gam op3 op3.gam.nscl.msu.edu
10.0.0.31	op4.gam op4 op3.gam.nscl.msu.edu
10.0.0.32	op5.gam op5 op5.gam.nscl.msu.edu
10.0.0.33	op6.gam op6 op6.gam.nscl.msu.edu
10.0.0.34	op7.gam op7 op7.gam.nscl.msu.edu
10.0.0.35	op8.gam op8 op8.gam.nscl.msu.edu
10.0.0.36	op9.gam op9 op9.gam.nscl.msu.edu
10.0.0.37	e1.gam e1 e1.gam.nscl.msu.edu
10.0.0.38	e2.gam e2 e2.gam.nscl.msu.edu
10.0.0.39	m1.gam m1 m1.gam.nscl.msu.edu
10.0.0.40	m2.gam m2 m2.gam.nscl.msu.edu
10.0.0.60	metal.gam metal metal.gam.nscl.msu.edu

10.0.0.11	gret-ts.gam gret-ts gret-ts.gam.nscl.msu.edu

10.0.0.110	sol.gam sol.gam.nscl.msu.edu sol

sol is the boothost for all the Digiticer IOCs

The computers above get their password file via NIS from a1.gam. Not all computers exists

Digitizer IOCs

10.0.3.1		vme01
10.0.3.2		vme02
.
.
10.0.3.47		vme47
10.0.3.48		vme48
10.0.3.49		vme49

User and access computers

10.0.10.11     reserved for tl laptop
10.0.10.12     ws1.gam ws1
10.0.10.13     ws2.gam ws2
10.0.10.14     ws3 gam ws3

these computers have their own password files independent of the NIS server a1.gam. In particular, access to ws1 is restricted to personel that need outside access to the GRETINA network at anl. (ws1), ws2 and ws3 are general purpose linux workstations for users to use and sort data on during experiments.

LN fill computers

10.0.4.11		gln1
10.0.4.12		gln2

misc other computers we know of

10.0.4.1       caenps01
10.0.4.2       caenps02
10.0.8.11	jays-happy-place
10.0.8.12	daqtape
10.0.9.11	middlemac

computer shutdown procedure

(for computers in g126 computer room) 

- log in to ws1
  umount anything from fs3.gam (or you will have
  problems shutting down ws1) 
- a2.gam  (shutdown -h now)
- sol.gam (shutdown -g0 now) turn power off by hand
- ssh [email protected]
    pdsh -w n00[00-89] shutdown -h now
    shutdown master1.dogs (shutdown -h now)
- ssh [email protected] (shutdown -h now)
- ssh root@n0168 - different root password (shutdown -h now)
- ssh root@n0169 (shutdown -h now)
- ssh [email protected] (shutdown -h now)
- Shutdown the NetApp
  - Connect to the fs3-01 via serial port
    - halt local -inhibit-takeover true (then answer 'y')
    - Wait to see "LOADER-A> " prompt
  - Connect to the fs3-02 via serial port
    - halt local -inhibit-takeover true -skip-lif-migration-before-shutdown (then answer 'y')
    - Wait to see "LOADER-B> " prompt
  - Physically turn off the file server head
  - Physically turn off the file server shelves
  - Physically unplug the power cables for both head and shelves
- 2 switches in racks sw02  & sw03
  - Use the LCD panel -> System Maintenance -> halt
- power off gt1.phy.anl.gov (ws1) (shutdown -h now)
- power off NAT box
- throw 5 three phase switches in control panel

computer powerup procedure

- Make sure switches are up
- Boot up gt1.phy.anl.gov and NAT box
- fileserver - fs3.gam
   - Reconnect all the cables if any of them got disconnected(e.g. power cables)
   - Turn on fs3.gam shelf and wait for 30 seconds
   - Turn on fs3.gam head
   - Log into fs3.gam with serial port
       - Run "cluster show" and "storage failover show" to confirm CFO/SFO is configured/enabled.
- NIS master - a1.gam
- the super nodes (n0168, n0169)
   - Make sure the mounts are all available. Check /var/log/messages if run into issues
- a2.gam
- sol.gam
- dogs cluster
   - Power on master1.dogs and master2.dogs
   - 'sh /root/ipmi/ipmi.poweron' to use IPMI to power on all compute nodes
   - Check the physical nodes and boot the nodes that impi didn't reach
- 30 VMEs
- op3.gam
- op4.gam
- op5.gam

dFMA info

The dFMA DAQ system is using non standard EPICS ports so that it can coexist with the DGS and GT acquisition systems. The ports the DFMA DAQ are using are 

  export EPICS_CA_SERVER_PORT=5068
  export EPICS_CA_REPEATER_PORT=5069

How to move dFMA DAQ to the GT network

To move the digital FMA DAQ (dFMA) from onenet network to the GT network, a number of steps must be followed. They are outlined below.

On the GRETINA network, we will used these IP addresses for dFMA 

10.0.50.189   dfma_darek  'set manually'
10.0.50.190   dfma_ts     'set manually'
10.0.50.191   dfma_ioc0   telnet dfma_ts 2004 TRIGGER VME32.cmd
10.0.50.192   dfma_ioc1   telnet dfma_ts 2001
10.0.50.193   dfma_ioc2   telnet dfma_ts 2002
10.0.50.194   dfma_ioc3   telnet dfma_ts 2003
10.0.50.195   dfma_ioc4   telnet dfma_ts 2005
10.0.50.196   dfma_ioc5   telnet dfma_ts 2012
10.0.50.197   dfma_ioc6   telnet dfma_ts 2007
10.0.50.198   dfma_ioc7   telnet dfma_ts 2008
10.0.50.199   dfma_ioc8   telnet dfma_ts 2009
10.0.50.200   dfma_ioc9   not therdfma_tse
10.0.50.201   dfma_ioc10  telnet dfma_ts 2010
10.0.50.202   dfma_ioc11  telnet dfma_ts 2011
10.0.50.203   dfma_ioc12  not there
10.0.50.204   dfma_ioc13  not there
10.0.50.205   dfma_ioc14  not there
10.0.50.205   dfma_ioc15  not there

Also shown in the list are the terminal access commands (they may change in the future).

- First access all the OICs and reboot them, but leave them at the boot prompt. Type 'ctrl x' and then hit any key when they try to boot.

- The GRETINA computers now should provide DHCP service on the GRETINA network, so the step below should nolonger be necessary. Just restart dFMA_darek and dfma after the network has bee changed

- [THIS SHOULD NOT BE NECESSARY, DHCP should take care of it] If there is no DHCP service for the dFMA DAQ, you can set the IP for the dFMA_darek and dfma_ts as: log into the terminal server through it's web interface 'firefox http://192.168.203.190'. The username is 'root' and the password is xxx. Now do this 

IP setting
  change IP to 10.0.50.190
  with netmask 255.255.0.0
  Gateway 10.0.0.1
reboot and turn off

-After this step, the terminal server does not function on onenet anymore, but it should come alive on the GT network (see below). Be careful about this step!! If you do it wrong, you may have trouble getting to the terminal server again.


-[THIS SHOULD NOT BE NECESSARY, DHCP should take care of it] change the IP and netmask on the dfma_darek computer from DHCP to 10.0.50.189 and 255.255.0.0. Specify 10.0.0.1 for the router as well as DNS.

-now switch the WAN connection on the switch from onenet to GT

-reboot dfma_darek, or restart the network if you just stopped it. The machine should come up on the GT network.

-turn on the dfma_ts terminal server. If the above steps were done properly, it should come up on the GT network

-on dfma_darek, change the entries in the /etc/hosts to have these entries rather than the onenet IPs 

 10.0.50.189 dfma_darek
 10.0.50.190 dfma_ts
 10.0.50.191 dfma_ioc0
 10.0.50.192 dfma_ioc1
 10.0.50.193 dfma_ioc2
 10.0.50.194 dfma_ioc3
 10.0.50.195 dfma_ioc4
 10.0.50.196 dfma_ioc5
 10.0.50.197 dfma_ioc6
 10.0.50.198 dfma_ioc7
 10.0.50.199 dfma_ioc8
 10.0.50.200 dfma_ioc9
 10.0.50.201 dfma_ioc10
 10.0.50.202 dfma_ioc11
 10.0.50.203 dfma_ioc12
 10.0.50.204 dfma_ioc13
 10.0.50.205 dfma_ioc14
 10.0.50.205 dfma_ioc15

You can accomplish that by 

 cd /etc
 cp hosts.GT hosts

note that we do not have to change the .rhosts file in the dgs account because it is using the IP information from the /etc/hosts file.

-now, through the terminal server, access the IOCs, one by one, and change the IP 

 from
   192.168.203.n:ffffff00
 to
   10.0.50.n:ffff0000

where n is the one from onenet (same last number on the GT network). Also change the host 

 from
   192.168.203.189
 to 
   10.0.50.189

You should now be able to boot the IOC (type @)

- once all IOCs are rebooted, you should be able to start the dFMA DAQ by typing this command 

 (cd; dfmaCSS)

(remember to start the soft IOC).

NOTE: the dFMA DAQ can function on the onenet with the dGS and with GRETINA on the GT network because the dFMA DAQ is using these non standard EPICS ports 

 export EPICS_CA_SERVER_PORT=5068
 export EPICS_CA_REPEATER_PORT=5069

Although any linux box on the GT network in principle can access the dFMA, only dfma_darek is set up for it.

How to move dFMA DAQ to the dGS network (onenet)

To move the digital FMA DAQ (dFMA) from the GT network to the onenet network, a number of steps must be followed. They are outlined below

On the onenet network, we will used these IP addresses for dFMA 

192.168.203.189   dfma_darek  DHCP
192.168.203.190   dfma_ts     DHCP
192.168.203.191   dfma_ioc0   telnet dfma_ts 2004 TRIGGER VME32.cmd
192.168.203.192   dfma_ioc1   telnet dfma_ts 2001
192.168.203.193   dfma_ioc2   telnet dfma_ts 2002
192.168.203.194   dfma_ioc3   telnet dfma_ts 2003
192.168.203.195   dfma_ioc4   telnet dfma_ts 2005
192.168.203.196   dfma_ioc5   telnet dfma_ts 2012
192.168.203.197   dfma_ioc6   telnet dfma_ts 2007
192.168.203.198   dfma_ioc7   telnet dfma_ts 2008
192.168.203.199   dfma_ioc8   telnet dfma_ts 2009
192.168.203.200   dfma_ioc9   not there
192.168.203.201   dfma_ioc10  telnet dfma_ts 2010
192.168.203.202   dfma_ioc11  telnet dfma_ts 2011
192.168.203.203   dfma_ioc12  not there
192.168.203.204   dfma_ioc13  not there
192.168.203.205   dfma_ioc14  not there
192.168.203.205   dfma_ioc15  not there

you can accomplish that by: 

 cd /etc
 cp hosts.onenet hosts

Also shown in the list above are the terminal access commands (subject to change in the future)

- First access all the OICs and reboot them, but leave them at the boot prompt. Type 'ctrl x' and then hit any key when they try to boot.

- [this step should no longer be necessary. The TS should already be in the default DHCP mode] now log into the terminal server through it's web interface 'firefox http://10.0.50.190'. The username is 'root' and the password is xxx. Now do this 

change to use DHCP
reboot and turn off

-After this step, the terminal server does not function on the GT network anymore, but it should come alive on the onenet network (see below). Be careful about this step!! If you do it wrong, you may have trouble getting to the terminal server again.

-change the IP and netmask on the dfma_darek computer to use DHCP .

-[this step should no longer be necessary. The computer should already be in the default DHCP mode] now switch the WAN connection on the switch from GT to onenet

-reboot dfma_darek, or restart the network if you just stopped it. The machine should come up on the onenet network.

-turn on the dfma_ts terminal server. If the above steps were done properly, it should come up on the onenet network

-on dfma_darek, change the entries in the /etc/hosts to have these entries rather than the onenet IPs 

192.168.203.189   dfma_darek
192.168.203.190   dfma_ts
192.168.203.191   dfma_ioc0
192.168.203.192   dfma_ioc1
192.168.203.193   dfma_ioc2
192.168.203.194   dfma_ioc3
192.168.203.195   dfma_ioc4
192.168.203.196   dfma_ioc5
192.168.203.197   dfma_ioc6
192.168.203.198   dfma_ioc7
192.168.203.199   dfma_ioc8
192.168.203.200   dfma_ioc9
192.168.203.201   dfma_ioc10
192.168.203.202   dfma_ioc11
192.168.203.203   dfma_ioc12
192.168.203.204   dfma_ioc13
192.168.203.205   dfma_ioc14
192.168.203.205   dfma_ioc15

note that we do not have to change the .rhosts file in the dgs account because it is using the IP information from the /etc/hosts file.

-now, through the terminal server, access the IOCs, one by one, and change the IP 

 from
   10.0.50.n:ffff0000
 to
   192.168.203.n:ffffff00


where n is the one from the GT nat (same last number as on the onenet network). Also change the host 

 from
   10.0.50.189
 to 
   192.168.203.189

You should now be able to boot the IOC (type @)

- once all IOCs are rebooted, you should be able to start the dFMA DAQ by typing this command 

 (cd; dfmaCSS)

(remember to start the soft IOC).

Although any linux box on the onenet network in principle can access the dFMA, only dfma_darek is set up for it.

reset TS16 to factory defaults

If you loose contact with the DIGI TS16 terminal server, bring it up to the 'bench' private network in the f-wing terninal room. 

 Power off the unit.
 Hold in the Reset button (white next to LEDs in front).
 While keeping the Reset button depressed, power the unit back on.
 Continue holding in the Reset button for 45 seconds, at which point you can release it.

The default username is root and the default password is dbps. The TS comes up in DHCP mode, so you just have to find it on the private network under 'bench' and set it up again. After the reset, set up port 1 to use TCP sockets, go to advanced and allow 4 connections to the port. Use copy to set all the other ports up the same way.

Retrieved from "https://wiki.anl.gov/wiki_gretina_at_anl/index.php?title=Computers&oldid=1975"