Editing Scanco Administration (section)

=Process, Printer and System=

==Processes and Jobs==

===Manage jobs in the queue===
Use the DELETE/ENTRY and SET ENTRY commands to manage a batch job.

<pre>
$ SH QUE                                        !Get process info
$ SH QUE /all/by_job_status=executing           !Get all executing process info
$ SHOW ENTRY 1131 /full                         !Get full process info of entry 1131
$ SE 1311                                       !Get full process info of entry 1131

$ DELETE /ENTRY=251                             !Deletes batch job number 251.

$ SET ENTRY 251/PRIORITY=50                     !Changes the priority of batch job number 251. 
                                                !Default priority is 100 for all jobs, max is 200.
$ SET ENTRY 251/REQUEUE=SCEVAL_FAST             !Moves to queue sceval_fast. 
                                                !The job cannot be executing to do this, has to be in pending status.
$ SET ENTRY 251/AFTER=TOMORROW                  !Holds entry 251 until the following day (midnight).

$ HELP SET ENTRY                                !More information on the SET ENTRY command.
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Control processors # on multi-processor machines===

For example, on the itanium there are two processors.  By default, only one takes reconstruction jobs.  The other is left for so that evaluations can take place without competing for the CPU. 

But if a bottleneck is the reconstruction, it can be switched as user <code>system</code>:

<pre>
$ set que sceva2_slow/job=2
</pre>

so that it will do two reconstruction jobs simultaneously.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Create a new queue===
This may be useful to separate jobs for a project.  The following will create a queue called BUIE$FAST, which will submit jobs to SCEVAL_FAST and SC3309_FAST.

 $ initialize/queue/GENERIC=(SCEVAL_FAST,SC3309_FAST)/OWNER=[SYSTEM] -
   /PROTECTION=(S:M,O:D,G:R,W:S)/BATCH/START BUIE$FAST


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Clear out all entries in a queue===
First create a dummy queue, then merge (i.e., move) all the entries from the queue to be cleared into it, then delete the dummy queue.

Example:  Clear out the printer queue.
 $ stop/reset printer1
 $ initialize/que dummy
 $ assign/merge dummy printer1
 $ delete/que dummy
 
Example:  Clear out a batch queue, e.g., sys$fast.
 $ initialize/que/batch dummy
 $ assign/merge dummy sys$fast
 $ delete/que dummy


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Stop and restart queues===
There are three methods.  First the nice method.

1. As for the printer queue, a normal user can stop and start other queues.

<pre>
$ stop/que/reset sceval_slow
$ start sceval_slow
</pre>

This kills the current process running.  If you want to let the current process complete before stopping the queue, use:
<pre>
$ stop/que/next sceval_slow
</pre>

2. Use the web browser Mozilla.

:*Open mozilla
:*'batch queues'

:*Resize the window so it occupies the right half of the screen

:*Click on 'delete job' once with the MIDDLE mouse button, this will open a new window on the right half of the screen. Ensure the two windows do not overlap, if they do resize the window which contains the batch queues list. Once window locations are correct MIDDLE click on the rest of the jobs which you wish to delete (click on 'Delete Job' in the batch queue list). The reason for locating the widows is because each successive click on 'Delete Job' will open a new window and you don't want it to cover the batch queues list.

:*Then Exit from mozilla, this will delete all the open windows, it may take a bit if you are deleting lots.

3. Now the harsh method.

[[Image:Bones2.jpg|20px]] <b>Warning:</b> Extreme caution!  Explicit permission for this change or similar changes <b>must</b> be obtained from Dr. Steven Boyd.

Completely remove the queue and re-generate, as user <code>system</code>

<pre>
$ stop/queue/reset sc3309_slow
$ delete/queue sc3309_slow
</pre>

Then you must restart the queue.  To learn how to do this refer to:
<pre>
$ eve sys$startup:dcps_hpljet_ip.com
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Kill many jobs on a queue===

Sometimes somebody can submit a recursive job that when executing can resubmit the job. It results in an endless loop and backs up the whole system with jobs.
It is hard to delete the jobs one at a time if there are millions of jobs. So the solution is to create a temporary queue, merge the jobs that need to be killed onto that queue, then delete the temporary queue.

In this example, we had >1000000 jobs on <code>sys$fast</code> so first I stopped all the queues:

<pre>
stop/queue/next SCEVA1_FAST
stop/queue/next SCEVA2_FAST
stop/queue/next SCEVAL_FAST
stop/queue/next SCSRV1_FAST
stop/queue/next SCSRV2_FAST
stop/queue/next SCSRV3_FAST
</pre>

Then I created the temporary queue, merged, then deleted:

<pre>
initialize/queue/OWNER=[SYSTEM] /PROTECTION=(S:M,O:D,G:R,W:S)/BATCH/START JUNKQ
assign/merge JUNKQ sys$fast
stop/queue/next JUNKQ
delete/queue JUNKQ
</pre>

Then restart the stopped queues:
<pre>
start SCEVA1_FAST
start SCEVA2_FAST
start SCEVAL_FAST
start SCSRV1_FAST
start SCSRV2_FAST
start SCSRV3_FAST
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Kill hunging processes===

Sometimes jobs need to be stopped manually, such as when FTP or TCPIP is hung. This is done as follows:

Checked processes with  
  $ show sys
where the FTP jobs are running. Doing it as <b>system</b> with these commands would show all the FTP jobs on all nodes:
  $ mcr sysman
  SYSMAN> set env /cluster
  SYSMAN> do pipe sh sys|sear sys$pipe ftpc
It showed that they all were running on SCEVA1 only. So I logged in there and using the IDs given at the list from above's command:
  $ stop/id=29804056
  ...


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Check CPU usage and kill a process===
Determine what process is using all the CPU (e.g., SYMBIONT, a print process) using the monitor command, then use the show system command to determin the Pid, which can be used to stop the process.

The print (SYMBIONT) process is known to consume excessive CPU when the clocks are advanced for daylight savings on the alpha systems.
<pre>
$ monitor process/topcpu
$ show system
OpenVMS V7.3-2 on node SCEVAL 6-APR-2009 12:42:22.82 Uptime 124 23:16:08
Pid      Process Name     State  Pri  I/O         CPU         Page flts Pages
20C00081 SWAPPER          HIB    16   0      0 01:07:05.63     0         0
20C00087 CLUSTER_SERVER   HIB    14   2218   0 00:00:00.54    63        77
20C00088 CONFIGURE        HIB    10   23     0 00:00:00.02    54        20
20C00089 LANACP           HIBO   13   -- swapped out --                 32
20C0008B FASTPATH_SERVER  HIB    10   9      0 00:00:00.01    79        94
20C0008C IPCACP           HIB    10   10     0 00:00:00.02    37        28
$ show process/all/id=Pid
$ set host 0
     user:  system
     password:
$ stop/id=Pid
</pre>
*Process Name = BATCH_122  means entry 122 in the queue


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

==Relay for pump==
In Houston we installed a relay that allows us to remotely turn on and off the power to the gantry. There are scripts in the <code>COM</code> directory called <code>RELAY_PUMP_...</code>.

Install the special <code>TELNET_CLIENT.EXE</code> as user <code>system</code> into <code>DISK0:[SYS0.SYSCOMMON.SYSEXE]</code> so that it is accessible and then set the permissions:

 $ set def disk0:[sys0.syscommon.sysexe]
 $ set prot=(s:rwed,o:rwed,g:re,w:re) telnet_client.exe

You need to set a logical (can be set permanently in <code>LOGIN.COM</code>):

 $ DEFINE TELNET_SERVER_LINE "TCPIP$DEVICE:172.16.1.2:9500"

Then you can turn the gantry on or off with

 $ mcr telnet_client -on
 $ mcr telnet_client -off

And you can check the current status of the gantry by pinging it:

 $ TCPIP PING 192.168.34.18

The output for when the gantry is <code>on</code> looks like this:

  $ TCPIP PING 192.168.34.18
  PING 192.168.34.18 (192.168.34.18): 56 data bytes 
  64 bytes from 192.168.34.18: icmp_seq=0 ttl=255 time=1 ms
  64 bytes from 192.168.34.18: icmp_seq=1 ttl=255 time=0 ms
  64 bytes from 192.168.34.18: icmp_seq=2 ttl=255 time=0 ms
  64 bytes from 192.168.34.18: icmp_seq=3 ttl=255 time=0 ms
  
  
  ----192.168.34.18 PING Statistics----
  4 packets transmitted, 4 packets received, 0% packet loss
  round-trip (ms)  min/avg/max = 0/0/1 ms

And when it is <code>off</code> it looks like this:

  $ TCPIP PING 192.168.34.18
  PING 192.168.34.18 (192.168.34.18): 56 data bytes
  ping: sendto: network is down 
  ping: wrote 192.168.34.18 64 chars, ret=-1
  ping: sendto: network is down 
  ping: wrote 192.168.34.18 64 chars, ret=-1
  ping: sendto: network is down 
  ping: wrote 192.168.34.18 64 chars, ret=-1
  ping: sendto: network is down 
  ping: wrote 192.168.34.18 64 chars, ret=-1
  
  
  ----192.168.34.18 PING Statistics----
  4 packets transmitted, 0 packets received, 100% packet loss
  %SYSTEM-F-TIMEOUT, device timeout

There is a script called <code>RELAY_PUMP_10_MIN_ON.COM</code> that self-submits and automatically turns on and off the gantry on Sunday at 2am.

==Printer==

===Set up TCPIP printer queue on a VMS workstation===
  $ stop/que/res hpljet
  $ tcpip set nohost hpljet
  $ tcpip set host "hpljet" /address= xxx.xxx.xxx.xxx
  $ start hpljet

Note that many HP printers will not work with OpenVMS. Checking for compatibility is important.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Stop and restart the printer queue===

How to fix the printer queue when hung, not working or giving an error.
A full printer queue can seriously affect system performance.  
Be sure that the queue is cleaned up from time to time.

<pre>$ show queue</pre>

to first find out the real name of your printer queue. 
* Replace "hpljet" and "...dcps_hpljet..." by your own queue (i.e., PRINTER1 for XtremeCT) name below. 
* First approach, as normal user <code>microct</code>: 
<pre>
$ stop/queue/reset hpljet
$ start/queue hpljet
</pre>

If you need to release a job
  $ set entry/release 123


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

==System==

===Monitor System===
====Monitor system usage====

<pre>
$ monitor/interval=1 processes/topcpu
$ monitor/interval=1 system
$ show error
$ show system
$ @SCOM:SHOW_DISKS.COM (or simply $ DISKS)
</pre>

The command <code>show error</code> may indicate errors.  One or two errors is OK.

To see what processes are running (specific to the machine, not cluster wide).

<pre>
$ show device disk2/file
$ show user /full
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Check status of OpenVMS system====

If when rebooting a VMS system there is a failure, sometimes a message can be repeated many times. The way to stop this repeating message is to press <code>ctrl-C</code> or <code>ctrl-P</code> to get to the <code>>>></code> prompt.

Then show some of the device information, either specifically

  >>> show device py*
  >>> show device dy*

or in general

  >>> show device

which might show more than needed. The output of

  >>> show bootdef_dev

can also be useful.

Once settings are all good, you can reboot the system with
  >>> init
  >>> set bootbios pyb0
  >>> boot


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Check disk status====
Login as the system user on the workstation connected to the disk array that you want to check.

 $ run sys$system:msa$util.exe
 MSA> set controller

Show the raid volumes (e.g., to check if it failed or is recovering).
 MSA> show unit

Locate disks associated with a raid volume (e.g., unit0).
 MSA> locate unit0

Accept a failed unit (e.g., unit0) - only for itanium workstations, which use a newer version of OpenVMS.
 MSA> accept unit0

Show disk information.
 MSA> show disks

Get information on MSA utility functions.
 MSA> help

Exit the MSA utility.
 MSA> exit


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Check status of disk array====
To look at the disks in an MSA30 tower:
There are several options to check the status of the disks. One is, if it is installed, to use the command-line utility
<pre>$ mcr msa$util</pre>
as "system". Enter
<pre>
MSA> set controller
MSA> show unit
MSA> show disks
MSA> exit
</pre>

Another is to start the WBEM web interface, which shows the status of the hardware graphically.

It is possible to fix a disk array that hasn't booted by doing the following:

  At the login prompt, type 'shutdown'. This will get you to the <code>>>></code> prompt.
There should be a RAID card, for example on Percy, named <code>pya0</code>. Try:
  >>> show device
  >>> set auto_action halt
  >>> init
  >>> run bios pya0
  >>> set auto_action restart
  >>> boot


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Check disks status in enclosure====

Periodically connect to http://10.148.64.254/ (user name: manage, password same as system password) and check if there are any errors or warnings, like failed disks. You can easily see which disk has failed in the graphical view of "Physical" -> "Enclosure X"

Consider not all web browsers allow connection to this rather old web server. Firefox and Mozilla on the workstations do work fine.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Recover System===
====Consistently needs warm-up scanner - Shutter-failsave-Job====

If the scanner is consistently needing to warm-up on consecutive days, and was not powered off, this is likely due to the Shutter-failsave-Job disappearing from the 3309-Batch queue (for XT1). The job must be re-submitted. 

The executing file is located in the CALIB3309 ($ um:submit_shutter_test.com) and the command to submit the job is:
<pre>
$ submit/after="tomorrow"/noprint/que=uct_measurement_queue um:submit_shutter_test.com
</pre>

You can check the status by typing: 
<pre>
$ show que *3309_batch*/all
</pre>

This job tends to disappear now and then, perhaps because of an abrupt shutdown of the workstation or maybe when stop/pause/re-start the queue. Contact Scanco if this does not resolve.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Disk not mounted/unavailable, directory not found, volume not enabled====
Disk is not mounted or is unavailable.  Error opening ...., directory not found, volume is not software enabled.
<pre>
%DIRECT-E-OPENIN, error opening DISK4:[MICROCT.PROJECTS]*.*;* as input
-RMS-E-DNF, directory not found
-SYSTEM-F-VOLINV, volume is not software enabled
</pre>
This can happen if communication is broken between workstations, causing the disk mount to time out.  Check if this is by showing the devices, then dismount and remount the disk.
<pre>
$ show dev

Device                  Device           Error    Volume         Free  Trans Mnt
 Name                   Status           Count     Label         Space Count Cnt
SC3309$DKA100:          Online               0
SC3309$DKD0:            Mounted              0  ALPHASYS        3.18GB   642   2
SC3309$DKD1:            Mounted              0  AUSER_01       42.84GB     4   2
SC3309$DKD2:            Mounted              0  ADATA_01      173.00GB     4   2
SC3309$DKD3:            Mounted              0  ADATA_02A     215.80GB     1   2
SC3309$DKD4:            Mounted              0  ADATA_02B     215.85GB     1   2
SC3309$DKD5:            Mounted              0  ADATA_02C     215.81GB     1   2
SC3309$DQA0:            Online               0
SC3309$DQA1:            Offline              1
SC3309$DQB0:            Offline              1
SC3309$DQB1:            Offline              1
SCEVAL$DKA0:            Online               0
SCEVAL$DKA1500:         Mounted              0  (remote mount)                 1
SCEVAL$DKC0: MntVerifyTimeout 0 ADATA_03A 541.62GB 1 2 
SCEVAL$DKC1: MntVerifyTimeout 0 ADATA_03B 541.38GB 1 2 
SCEVAL$DKC2: MntVerifyTimeout 0 ADATA_03C 541.60GB 1 2 
SCEVAL$DQA0:            Online               0


$ set host 0
system

$ dismount SCEVAL$DKC0:
$ mount/cluster SCEVAL$DKC0: ADATA_03A
%MOUNT-I-MOUNTED, ADATA_03A mounted on _SCEVAL$DKC0:
$ mount/cluster SCEVAL$DKC1: ADATA_03B
%MOUNT-I-MOUNTED, ADATA_03B mounted on _SCEVAL$DKC1:
$ mount/cluster SCEVAL$DKC2: ADATA_03C
%MOUNT-I-MOUNTED, ADATA_03C mounted on _SCEVAL$DKC2:
$
</pre>

If this does not restore the disk, then shutdown the computer.  At the blue screen...
<code>>>> init</code>
Wait for blue screen to return.
<code>>>> run bios pya0</code>
Press F2 at the prompt to accept potential data loss.
Wait for blue screen to return.
<code>>>></code>
Press power button to turn off.
Press power button to turn on.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Kill hanging program====

If a program is hunging, it may not be necessary to shut down the system and restart. For example, if the <code>uct_evaluation</code>
program is hung, do the following

<pre>
$ show dev/full microct_scsi

$ stop/id=<Owner process ID> (Replace the <> with the actual ID)

for example: 

$ stop/id=2280091C 
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Reset the display/GUI====
These commands will terminate the current login session and may restore a frozen or messed up screen.  You will need to remotely access the problem workstation either from another micro-ct workstation or using a personal computer.

Use the set host command to connect to the problem workstation (e.g., thomas) using the node name (e.g., sc3309) or the IP address (e.g., thomas.kin.ucalgary.ca). 

 $ set host sc3309
 login:  system
 password: 

Issue the reset display command.

 $ @sys$startup:decw$startup restart

At the prompt, type "y".


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Restart screen server====

If the monitor fails to show the login prompt and displays a blank gray screen...telnet to the computer from another computer (e.g. a PC or so) and login as user system. Run the restart command:

<pre>
$set host gordon.ucalgary.ca.    [or other computer name]
$username: system
$password: [system password]
$@sys$startup:decw$startup restart</pre>
confirm (answer "yes" at the prompt) and logout.
<pre>
$y
$logout</pre>

This will kill all programs on your screen, but it will leave all batch jobs running, so if you do a BATCH measurement, this will survive! (but interactive scan will be killed).

If you are restarting a monitor connected to a KVM box, unplug and re-plug in the box.

This command also helps when the mouse/keyboard freezes.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Restart Alpha stations====

[[Image:Bones2.jpg|20px]] <b>Warning:</b> Do not do this unless absolutely necessary.

For shutdown

At the login screen, type "shutdown"/no password and hit enter. 
Then, you can turn the computer off at the prompt.

Then, you may turn the disk array off (unplug it as there is no power switch).

After you move the computer, power on the disk array and then the computer.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Slow/non-responsive computer====

This is usually caused by people logging in via SSH multiple times.  The solution involves resetting the SSH/TCPIP daemon.

:*If you are local at the computer:

You will need to log in as user <code>system</code>.  ONLY EXPERIENCED USERS SHOULD EVER LOG IN AS SYSTEM!

The system will be extremely slow (it will take 5 minutes just to get a log in prompt).  This is expected.

<pre>
$ set host 0   ! user name 'system' and password	
$
$ @sys$startup:tcpip$ssh_shutdown    ! shutdown SSH
$
$ set host 0   ! user name 'system' and password	
$
$ @sys$startup:tcpip$ssh_startup     ! restart SSH
</pre>

Alternatively, you can reboot the system to reset the SSH.  This is not an appropriate solution and it is solving the problem by hitting it with a jack hammer.  Sometimes it is the only choice though.

:*If you are remotely logged in:

You will have to log in as user <code>system</code> and run a script called STOP_START_TCPIP.COM.  

This works essentially the same as for the above method, except that since it would be impossible to restart SSH remotely once it is shut down, two separate scripts are called and submitted on SCEVAL_FAST (make sure the queue isn't clogged) separated by a 1 minute delay. The first script called is STOP_TCPIP.COM and the second is START_TCPIP.COM.

To execute this process, you must log in as system, go to the SYS$SCRATCH directory, and interactively execute the script:
<pre>
$ @STOP_START_TCPIP.COM
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Shutdown a workstation locally====
These instructions were provided by Pascal in May 2026:

    - end the session
    - enter 'shutdown' as username and hit enter key for a few times (no password needed)
    - the screen eventually turns black/blank
    - just give it another minute or so, then press and hold the power button until the workstation turns off
    - remove both power cords for a few minutes

    - reconnect power cords
    - wait for at least 30 seconds
    - press and release again the power button; the workstation will then turn on with a slight delay

The workstation should then be back online after a few minutes.

====Shutdown a workstation remotely====
If a workstation becomes unresponsive, it may be possible to shut it down remotely from another workstation or personal computer, which is preferable over doing a hard power off.  These instructions were provided by Mark Blanchard.

'''1. Remote shutdown from another micro-ct workstation'''.  MAY NOT WORK FOR CLUSTER SERVER

Login as the system user.

 $ set host 0
 user:  system
 password:

Start the system manager.
 $ mcr sysman

Set the environment to the workstation to be shutdown (e.g., thomas) using its node name (e.g., sc3309).
 SYSMAN>< set env/node=sc3309

Issue the shutdown command.  Add a time delay, e.g., 5 min, if you want to give users time to disconnect and the automatic reboot flag if you want to restart.
 SYSMAN> shutdown node /min=5 /automatic_reboot

*You need to remain logged into the session running the shutdown until the other node has been shutdown completely!!!

'''2. Remote shutdown from a personal computer'''.

Begin a remote session to the workstation and login as the system user.
 
Create a detached terminal.
 $ create/terminal/detach
 
On the PC, port 6000 needs to be accessible to the micro-ct workstation and in the X-server's configuration (e.g., in file X0.hosts for Xming). 
 
Issue the shutdown command.
 $ shutdown


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Slow computers - Change RAID battery for Itanium computers====

If computers become sluggish, particularly when saving data after a scan, have Scanco check the batteries in the RAID drive. The batteries in SCEVA3 were changed 2014-11-10. There are two batteries. Order the HP brand (should look like the ones in the image).
*Instructions to open the Itanium work station and access the PCI card: [[Media:HP_rx2660_PCI_card.pdf |'''Change PCI card''']].
*Instructions to change the batteries after accessing the PCI card: [[Media:HP_SA6402_Batteries.pdf |'''Change batteries''']].


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Frozen computer - "bootstrap failure" on boot up==== 
The micro-CT computer is frozen or there is a "bootstrap failure" on boot up, This happens to the client computers (those connected to the scanners) when there is interruption in the network.  The only option is to restart the computer.

====Shutdown via iLO====

Connecting the to the iLO of SCEVAL is quite simple.  The IP to be used within Putty is 10.148.64.233. If you are on a Mac, find a linux workstation (i.e., Winston) to connect as the security is too old for Mac to connect these days.

Once the connection has been opened, click into the terminal and press the enter key unit you get asked for a username:
    Username (case sensitive):    Administrator
    password:    ---

At the main menu, enter 'co' in order to switch to the console (CTRL+B always gets you back one menu layer if needed). 

Then hit the enter key again and provide username and password for the OpenVMS user 'system'. 

You then are connected to the workstation as system user and can shut it down properly (assuming there are no jobs running anymore on that node):
    $ shutdown
The progess of the shut down procedure will then be shown in the terminal. 

The following line will then indicate that the shutdown procedure has been completed:
    P00>>
To turn it off remotely (it does not automatically turn off after a shutdown), just press CTRL+B until you are back at the MP Main Menu. 

The enter 'cm'. To turn off the workstation, just enter the following command then:
    CM>    pc -nc -off
Turning back on the workstation works pretty much the same (with a slightly different command):
    CM> pc -nc -on

====Crash of XtremeCT II workstation computer====

For a non-responsive monitor and/or a mouse and/or a keyboard of the XtremeCT II workstation (sometimes caused by a lost of connection to server, or an internet outage):

:*1. The first thing to do is to check system logs for any indication of the cause of the issue, if you have the "Administrator" user privilege.
:*2. The next thing to do is to soft-reboot by remote-login and using command line to reboot the workstation, if you have the "system" user privilege.
:*3. If no success, try hard-reboot by pressing the power button as indicated below.
:*4. If no success, try hard-reboot by unplugging both power cords as indicated below.
:*5. If still no success, contact Scanco immediately.
:*6. The last thing to do is to record the issue and any possible reasons.

1.  Checking system logs:
 
:*1) Remotely log into the iLO (the management console of the workstation) using SSH as user "Administrator" with the system password. 
Both user/pw are case sensitive. For XtremeCT II workstation (James) it is SSH to 10.148.64.231. For others, consult the network diagram. 

A trick: This connection only succeeds through a MAC or PC (via putty) direct login, not through an OpenVMS or Scanco system.
   $ ssh -l Administrator 10.148.64.231
:*2) Then you get to a   MP>   prompt here:

   MP MAIN MENU:
         CO: Console
        VFP: Virtual Front Panel
         CM: Command Menu
         CL: Console Log
         SL: Show Event Logs
         HE: Main Help Menu
          X: Exit Connection
   [sc3401-mp]</> hpiLO->

:*3) Type in 'SL' to show event logs, then you could choose 'E' for system event:

   [sc3401-mp]</> hpiLO-> SL
     Event Log Viewer Menu:
       Log Name            Entries    % Full      Latest Timestamped Entry
  -------------------------------------------------------------------------------
   E - System Event          1661       58 %        13 Sep 2018 11:17:43
   F - Forward Progress     20000      100 %        13 Sep 2018 11:17:43
   I - iLO Event              500      100 %        18 Sep 2018 07:46:42
   C - Clear SEL and FPL
   L - Live Events

   Enter menu item or [Ctrl-B] to Quit: E
       Log Name            Entries    % Full      Latest Timestamped Entry
  -------------------------------------------------------------------------------
   E - System Event          1661       58 %        13 Sep 2018 11:17:43
     Event Log Navigation Help:
   +       View next block     (forward in time,  e.g. from 3 to 4)
   -       View previous block (backward in time, e.g. from 3 to 2)
   <CR>    Continue to the next or previous block
   D       Dump the entire log
   F       First entry
   L       Last entry
   J       Jump to entry number
   H       View mode configuration - Hex
   K       View mode configuration - Keyword
   T       View mode configuration - Text
   A       Alert Level Filter options
   U       Alert Level Unfiltered
   ?       Display this Help menu
   Q       Quit and return to the Event Log Viewer Menu
   Ctrl-B  Exit command, and return to the MP Main Menu

:*4) From here you can use -  and  +  to go through the log history back and forth. Not all logs do make a lot of sense though, as they are mainly meant for HPE engineers. 
:*5) When finished, use  CTRL + b to get back to the main MP> prompt.
:*6) Use  MP> co   to get to the console, where you see at what stage the booting process is, in case it does not boot properly.
:*7) When finished, use  CTRL + b to get back to the main MP> prompt.
:*8) Use  MP> x   to end the connection.

2. Soft-reboot by remote login:

If a workstation becomes unresponsive, it may be possible to shut it down remotely from another workstation or personal computer, which is preferable over doing a hard-reboot.  These instructions were provided by Mark Blanchard.

:*2.A) Remote shutdown from another micro-ct workstation.  MAY NOT WORK FOR CLUSTER SERVER

Login as the system user.
   $ set host 0
   user:  system
   password:

Start the system manager.
   $ mcr sysman

Set the environment to the workstation to be shutdown (e.g., thomas) using its node name (e.g., sc3309).
   SYSMAN>< set env/node=sc3309

Issue the shutdown command.  Add a time delay, e.g., 5 min, if you want to give users time to disconnect and the automatic reboot flag if you want to restart.
   SYSMAN> shutdown node /min=5 /automatic_reboot

You need to remain logged into the session running the shutdown until the other node has been shutdown completely!!!

:*2.B) Remote shutdown from a personal computer.
Begin a remote session to the workstation and login as the system user.

Create a detached terminal.
   $ create/terminal/detach
 
On the PC, port 6000 needs to be accessible to the micro-ct workstation and in the X-server's configuration (e.g., in file X0.hosts for Xming).  

Issue the shutdown command.
   $ shutdown

3. Hard-reboot by power button:

:*1) Hold the power button until it turns yellow.
:*2) Wait a minutes or two, then press the power button only once and shortly. You will not see anything for a while. Pressing the power button again during this time will turn off the workstation again !
:*3) After about 10~15 min, the workstation would boot up properly. 
  
4. Hard-reboot by unplugging both power cords:

:*1) Hard-reboot by unplugging both the power cords. This will reset the console and controllers.
:*2) Wait for 5~10 minutes, then press the power button only once and shortly. You will not see anything for a while. Pressing the power button again during this time will turn off the workstation again !
:*3) After about 10~15 min, the workstation would boot up properly.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Crash of XtremeCT II workstation at Johnson Space Center====

Take a look at the log files by connecting to the iLO. Since we don't have an ethernet connection to the iLO at JSC, but we are able to log in on the LAN, we can log in as per normal and then connect:

 $ ssh "Administrator@192.68.196.57"

Use the <code>system</code> password to connect.

<pre>
hpiLO>    sl

                        e

                        t

                        a    3

                        d
</pre>

This then should create a dump in text form of all entries in the system log with an alarm level 3 (critical) and higher. Copy from the terminal and share with Scanco (e.g., Pascal).

[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]