Editing Scanco Administration (section)

=Data Management=
==Lab’s Data Responsibility==
*Weekly 
#Clean up sys$scratch directory and only keep last two week's data.
#Archive RSQ to corresponding tapes.
#*RSQ from temporary tests and non-project (not a formally established project) will NOT be archived unless specifically notified to lab personal.
#Database backup to disks.
*Monthly
#Database backup to corresponding tapes.
#Archive QC RSQ to corresponding tapes.
#Delete QC IMA and only keep last month's data.
*6-monthly
#System backup to corresponding tapes when prompted.
*Yearly
#Data Disk clean up when usage is over 90%.
#*Temporary tests and non-project RSQ and IMA data will be deleted during check up.  
#Project Disk clean up when usage is over 90%.
#Winston backups clean up.  
*Whenever requested
#Projects backup to Winston server and restoration.
#Projects archive to HOG server and restoration.

[[File:BoneLabDataManagement.JPG|800px]]

[[Image:Bones2.jpg|20px]] <b>Warning:</b> STRONGLY URGE TO DO THE WEEKLY and MONHTLY BACKUP. Two users DID experience disk crashes before, and without the weekly backup, you may still have all scans on the data disk, but without the databases you cannot tell which scans belong the which sample.

*Refer to [[Media:Microct_management.pdf|'''XtremeCT Micro-CT Management SOP''']] or [[Media:Microct_management.docx|'''its Word format''']] for references.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

==HP Server and Disk Array==
===HP References===
*[[Media:Hp-quick-specs-msa30.pdf|'''MSA30 Disk Array Quick Specs''']]
*[[Media:Hp-integrity-rx2620-sceva2-user-service-guide.pdf|'''Integrity RX2620 User Guide''']]
*[[Media:Hp-integrity-rx2660-sceva3-user-service-guide.pdf|'''Integrity RX2660 User Guide''']]


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===HP FAQ===
====Change disk description for $ disks====
Login as the system user and change to the system commands directory.

 $ cd scom
 $ eve show_disks.com


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

==Tape==
[[Image:Bones2.jpg|20px]] <b>Warning:</b> While the tape is running and writing to it, DO NOT LOG OUT, DO NOT INTERRUPT THE PROGRAM! This would make that tape unusable from this point on - you could still read from that tape up to this point, but could not add more scans to this tape.

===Tape Label Convention===
Tape labels are 6 characters in length.  For the Bone Imaging Lab projects, the first three letters indicate if the tape is for raw (RAW) or image (IMA) data, and the last three digits are for the tape number.

<pre>
RAW040 = RAW date, tape #040
IMA001 = IMAGE data, tape #001
</pre>

Alternatively, the first two characters may be set to a PI's initials, the third character indicates if the tape is for raw (R) or image (I) data, and the last three digits are for the tape number.

<pre>
SBR040 = Steve Boyd RAW, tape #040
SBI001 = Steve Boyd IMA, tape #001
BHR006 = Benedikt Hallgrimsson RAW, tape #006
BHI017 = Benedikt Hallgrimsson IMA, tape #017
FJR012 = Frank Jirik RAW, tape #012
FJI003 = Frank Jirik IMA, tape #003
</pre>

CaMos raw data is backed up with the following naming:
<pre>
CAMOS1 = CaMos RAW, tape #1
</pre>

When a tape becomes full, start (initialize) a new tape, increasing the tape number by one so that each tape label is unique.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Initialize a New Tape===
Put a new tape into the tape drive, then press "Initialize", this will format (erase) everything on the tape.  When prompted for a label for the tape, follow the [[Scanco_Administration#Tape_Label_Convention|'''Tape Label Convention''']].  The tape can now be mounted for use.<br>

[[File:Uct_backup_pic02.jpg|400px]]


===Full Tape===
A tape automatically ejects during a backup procedure if it becomes full.  
*Remove the tape and write protect it (orange indicator will be displayed).  
*Exit the disk-utility program. 
:*Hit <code>ctrl/c</code> to interrupt the program if needed. 
*Start a new instance of the program.  
*[[Scanco_Administration#Initialize_a_New_Tape|'''Initialize a New Tape''']].  
*Backup any remaining files.

[[File:Uct_backup_pic03.jpg]]


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Tape Drive Setup on XCT cluster===
Both the old and new tape drives are connected to SC3309 (XCT1) and powered on. You do not need to power off one to use another.


New tape in new drive <--> microct, xtremect2, xt2recalib, calib3401, calib3309

:*The 'new' Ultrium 3000 tape drive is for the new tapes and the default tape drive for all other accounts (microct, xtremect2, xt2recalib, calib3309, calib3401). 
:*Use the following command to use the old SDLT tape drive no matter which account on the command line:
<pre>
$ uct_backup_sdlt
</pre>

Old tape in old drive <-->  xt1recalib  

:*The 'old' SDLT tape drive is for the old tapes. You have to log into the default xt1recalib account to recognize this tape drive. 
:*Restoring from old tapes in xt1recalib account will restore your data to DISK6:[MICROCT.DATA]. 
:*By default, xt1recalib account can only see old tape drive, not the new tape drive.
:*Use the following command to use the new tape drive no matter which account on the command line:
<pre>
$ uct_backup_sas
</pre>

*With either of the above commands, you must not "submit" the tape usage jobs to the queue (i.e., cannot run a batch backup from a script), but only run them interactively in the 
software. Any job in the queue will use the default tape drive again.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Extract from Tape===

====Scripts to Extract from Tape====
Generally, all files on the tape can be extracted, or only specific files.
To extract all files on a tape, for example, a tape received from a collaborator, insert the tape into the drive and do the following:
  $ cd ur
  $ create/dir [.collaborator]
  $ create/dir [.collaborator.com]
  $ create/dir [.collaborator.log]
  $ create/dir [.collaborator.models]
  $ cd ur collaborator models
  $ backup/log tp0:*.sav []/by=parent  ! be sure you are the user (i.e. 'microct' or 'xtremect2') that has permission
  $ backup/log tp0:*.sav DISK4:[BONELAB.PROJECTS.FRAMINGHAM.MODELS...]/by=parent ! The "..." keeps the directory structure. The /by=parent ensures owner of the files is same as parent directory

This will save all the files on the tape to the current directory. Alternatively, if you only want a certain file extracted from the tape, such as an RSQ file, do essentially the same as per above, except with a different final command:
  $ backup/log tp0:*.sav/select=*.rsq []/by=parent

The <code>/by=parent</code> part of the command is a qualified for the file owner, and it makes the files that are copied the same owner as the parent directory (nothing to do with how directories are structured). The <code>[...]</code> generates sub-directories.


A custom script is required when extracting data from tape from another micro-CT system or to extract the data to a central location (e.g., a user's directory).  
*The script is then submitted to the backup queue of the workstation the tape drive is connected to (i.e., SCEVA3_BACKUP)
*Some example scripts are provided below.

*Extract image data for three scans to a custom folder, maintaining the sample and measurement directory structure:
<pre>
$ mount/foreign tp0
$ backup/log tp0:00001234_IMA.sav/select=[microct.data...]  disk3:[microct.projects.buie...]
$ backup/log tp0:00001245_IMA.sav/select=[microct.data...]  disk3:[microct.projects.buie...]
$ backup/log tp0:00001238_IMA.sav/select=[microct.data...]  disk3:[microct.projects.buie...]
$ dismount tp0
</pre>
 
*Extract segmented AIM files for three scans to a custom folder without the original directory structure:
<pre>
$ mount/foreign tp0
$ backup/log tp0:00001234_IMA.sav/select=[microct.data...]*seg.aim  disk3:[microct.projects.buie]
$ backup/log tp0:00001245_IMA.sav/select=[microct.data...]*seg.aim  disk3:[microct.projects.buie]
$ backup/log tp0:00001238_IMA.sav/select=[microct.data...]*seg.aim  disk3:[microct.projects.buie]
$ dismount tp0
</pre>
 
*Extract segmented AIM files for all scans on tape to a custom folder:
<pre>
$ mount/foreign tp0
$ backup/log tp0:*_IMA.sav/select=[microct.data...]*seg.aim  disk3:[microct.projects.buie]
$ dismount tp0
</pre>
 
*Extract raw data for three scans to a custom folder, without the original directory structure:
<pre>
$ mount/foreign tp0
$ backup/log tp0:00001234_IMA.sav/select=[microct.data...]  disk3:[microct.projects.buie]
$ backup/log tp0:00001245_IMA.sav/select=[microct.data...]  disk3:[microct.projects.buie]
$ backup/log tp0:00001238_IMA.sav/select=[microct.data...]  disk3:[microct.projects.buie]
$ dismount tp0
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Install Tape Drive===

In order replace a faulty tape drive first shut down the workstation. Then exchange the tape drives and reboot the workstation (make sure the cable connecting the tape drive to the workstation is properly connected at both ends).

Once logged in using the regular account, enter the following commands to activate the new tape drive:

  $ set ho 0

username: system,  password: xxx

  $ set def sys$startup
  $ mcr sysman io list_wwid

This then should list some info about the new tape drive; at the end should be listed its unique WWID

  $ mcr sysman io replace_wwid $2$mga0 /WWID=... 

Enter the WWID that was displayed after entering the previous command, providing the entire string. Something like this <code>02000008:5140-2EC0-1208-5F00</code>

  $ mcr sysman io auto/log
  $ @sys$startup:scanco_setup_devices
  $ logout

The new tape drive should then be operable.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Mounting a Tape Using the SDLT 600===

To mount a tape, specific to the SDLT 600 drive (the old one), you must use a variation of the <code>uct_backup</code> software on the workstation that sets the correct <code>TP0</code> symbol. Insert the tape into the drive and then log into the SC3505 workstation (aka Toby) as user <code>microct2</code>. After this step, open a new terminal and enter the following:

 $ uct_backup_sdlt

This will then start the archiving software and also assign the old tape drive to be the primary archiving device. If you don't want to use <code>uct_backup</code> you can mount the drive from the command line:

 $ mount/foreign tp1

This should then mount whatever tape you have inserted in the SDLT 600 drive and you can extract the data. Don't forget to unmount when you're done (instructions for mounting, unmounting and extracting data are elsewhere).

This was successfully done in December 2025. Make sure to watch out for bad tapes, as these cannot be mounted.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Tape FAQ===
====Error: Can't mount tape, Medium offline====
This can happen if a tape is interrupted during use (e.g., due to a power failure or system restart while the tape drive is in operation).

#Use the manual eject button to remove the tape from the drive.
#Turn off the tape drive, the switch is on the back of the drive.
#Shutdown the whole micro-CT system.
#Turn on the tape drive.
#Startup the micro-CT system, starting with the server.
#Try to mount the same tape again.

If the above does not work then either the tape or the tape drive is damaged.  To determine if it is the drive, take a brand new tape and try to initialize it.  Do not use a tape with data on it in the drive unless you are certain the drive is okay.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Error: positioning====
This can have two reasons usually:

#The tape is just faulty. Try with a fresh tape.
#A previous tape write operation was interrupted (e.g., a user did an 'End Session', the power went out, or the eject button was pressed on the tape drive, which killed the backup program). In this case the drive did not write the very important 'End of Tape' marker, and now, the tape drive cannot find the current place where it can start adding new data. 
*Solution: either take a fresh tape, or if there were only a few scans are on the 'bad' tape, 'Move to Hard Disk' of ALL the scans on that tape, check that the scans have been retrieved well, then initialize that tape again (which will erase everything on the tape!) and archive those scans again. 
*IMPORTANT: never interrupt the tape drive while it is running!


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Error: parity====
Either tape or the tape drive is faulty. Take a fresh tape (potentially from a new batch of tapes), and see if the error persists. 
*If yes: have the tape drive repaired. 
*Further things to check: for DLT and SDLT tape drives: is the terminator ('dead end') connected on the second SCSI port in the back of the tape drive.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Fix: non-responsive tape drive====
This can occur if the user ejects the tape with the little button (and not with the recommended "dismount" button in the uct_backup program).  This can also occur if a tape is mounted immediately after one is dismounted.  You get the same error for both cases, "medium offline".  

:*Check all that all cables are connected.

:*Check that the number on the back of the drive says "5" (use +/- to adjust if necessary)

:*Check that the top connector has a cable connecting to the computer and the bottom one has a terminator.

:*Try turning the tape drive on and off again.

:*Log in as user <code>system</code> (you will need a password from Steve):
<pre>
$ mcr sysman io auto/sele=mk
$ @sys$startup:scanco_setup_devices.com

$ show device tp0
</pre>

If it's online again, then great.  If not, then the last resort is to reboot the entire system.  If this doesn't work, then shut down the system (see user manual for proper steps) and eject the tape.  You may need to turn the computer back on before it will eject.  The tape drive should be turned on first (no tape), then the server, and any other computers.  Check that the drive is online ($ show device mk).  If it is, then test the drive by initializing, mounting and dismounting a brand new tape.  If the drive is not online, do not insert any tapes or they will likely get stuck.  Log in as system and re-try the command lines shown above.

NOTE: See "Fix a stubborn tape drive" elsewhere in this wiki.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Fix: stubborn tape drive====
If the drive cannot be mounted, or it is not visible with
<pre>
$ show device tp0
</pre>
then something went wrong.  For example, the server computer was booted while the tape was turned off. It then does not load the driver.

You can 'force load' it - but this will disconnect the scanner for a moment, so only do it while no scan is running:

as user <code>system</code>, on sc3309:
<pre>
$ mcr sysman io auto

$ show dev tp0
</pre>

and you should see it now (otherwise turn on the tape drive, and do again)

Then you'll also have to make it available to all users:
<pre>
$ @sys$startup:scanco_setup_devices.com
</pre>

NOTE: See "Fix a non-responsive tape drive" elsewhere in this wiki.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====SDLT320 Tape and Tape Drive FAQ====
Please refer to [[Media:Hp-faq-sdlt320-tape-drive-and-sdlt-tapes.pdf|'''SDLT320 Tape and Tape Drive FAQ''']] for details.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

==USB Drive==

===Use a USB drive===
Connect USB to VMS system and format. Transfer files from the terminal after plugging in the USB device.
  $ show device dna
Usually the device is dna1 or dna2
  $ mount /over=id dna2
  $ create/dir dna2:[microct]
  $ backup/log disk1:[microct]login.com dna2:[microct]
  $ dismount dna2
Transfer files back again
  $ mount /over=id dna2
  $ dir dna2:[microct]
  $ copy dna2:[microct]login.com disk1:[microct]
I'm not sure the difference between using <code>copy</code> and <code>backup</code>.

You can initialize a USB drive using <code>uct_backup</code>. Or you can do it from the command line:
  $ set host 0                      ! login as system
  $ init dna7: raw001/own=microct   ! <b>deletes</b> anything on the drive
  $ mount dna7: raw001 
  $ create/dir dna7:[microct]/own=microct

Or in case the directory already exists:

  $ set file dna7:[000000]microct.dir/own=microct
  $ dismount dna7:

For manual backup of files, you need to mount the filesystem (without the /foreign):

  $ mount dna7: raw001
  $ backup login.com dna7:[microct]/log
  $ dismount dna7:

[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Trouble shooting a USB drive===

Sometimes we have problems with USB devices. Check the status of the device to find if it has been allocated already and that there are no current jobs running that may have hung.

The device may look like this:

<pre>
$ sh dev dna

Device                  Device           Error    Volume         Free  Trans Mnt
 Name                   Status           Count     Label         Space Count Cnt
SC3418$DNA7:            Online alloc         0
</pre>

The <code>Online alloc</code> instead of  <code>Online</code> suggests the device is already allocated. So we get a more full output:

<pre>
$ sh dev dna/full

Disk SC3418$DNA7:, device type ST1500LM 003-9YH148 CC9F, is online, allocated,
    file-oriented device, shareable, available to cluster, error logging is
    enabled, device is busy.

    Error count                    0    Operations completed            1424772
    Owner process        "BATCH_357"    Owner UIC                      [SYSTEM]
    Owner process ID        0000A94A    Dev Prot         S:RWPL,O:RWPL,G:R,W:RW
    Reference count                3    Default buffer size                 512
    Total size                1.36TB    Sectors per track                     1
    Total cylinders            31536    Tracks per cylinder                   1
</pre>

This suggests that the owner is process <code>BATCH_357</code>. The <code>357</code> refers to the job that ran on the queue.

<pre>
$ sh que/all
Printer queue HPLJET, idle, on SC3418::"IP_LPD/hpljet", 
mounted form DCPS$DEFAULT (stock=DEFAULT)

Batch queue SYS$ADMIN, idle, on SC3418::

  Entry  Jobname         Username             Status
  -----  -------         --------             ------
    752  RELAY_PUMP_10_MIN_ON
                         MICROCT              Holding until 31-JAN-2025 02:00:00

Batch queue SYS$BACKUP, idle, on SC3418::

Batch queue SYS$BATCH, idle, on SC3418::

Batch queue SYS$FAST, idle, on SC3418::

Batch queue SYS$FE, idle, on SC3418::

Batch queue SYS$SLOW, idle, on SC3418::
</pre>

But there are no jobs running. So let's try manually dismounting the device, but do so as user <code>system</code>:

<pre>
$ dism/abort dna7:
%SYSTEM-F-DEVNOTMOUNT, device is not mounted
</pre>

Well, that didn't work, so let's see if there is a list of running processes, again as user <code>system</code>:

<pre>
$ show user/full
      OpenVMS User Processes at 30-JAN-2025 15:20:04.96
    Total number of users = 2,  number of processes = 3

 Username Process Name    PID     Terminal
 MICROCT  BATCH_357     0000A94A  (Batch)
 MICROCT  MICROCT       0000D2A3  FTA154:  
 SYSTEM   SYSTEM        0000D2A4  RTA1:    (SC3418::MICROCT)
</pre>

Stop the process associated with running the USB device:

<pre>
$ stop/id=0000A94A
$ sh user /full
      OpenVMS User Processes at 30-JAN-2025 18:28:23.65
    Total number of users = 2,  number of processes = 3

 Username Process Name    PID     Terminal
 MICROCT  BATCH_357     0000A94A  (Batch)
 MICROCT  MICROCT       0000D2A6  FTA155:  
 SYSTEM   SYSTEM        0000D2A7  RTA1:    (SC3418::MICROCT)
</pre>

Ok, did that still not work? Try unplugging the USB drive and waiting 30 seconds before plugging it back in. Perhaps that will force the job to end.

<pre>
$ sh dev dna

Device                  Device           Error    Volume         Free  Trans Mnt
 Name                   Status           Count     Label         Space Count Cnt
SC3418$DNA7:            Online               0
</pre>

That seems to have worked!


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===USB Device Usage with Scanco Backup/Archive Software===

For illustrated help on USB device usage on Itanium systems, please refer to [[Media:Scanco_TD387_USB_Device_Usage.pdf ‎ | '''Scanco_TD387_USB_Device_Usage.pdf ‎''' ]]
 
====Initialize procedure====
If an external USB device is to be used to archive measurement data, the USB device must be initialized by Open-
VMS. The data will not be accessible from any other operating system.

- Attach the USB device to the front USB port of the Itanium rx2660.
(note: Itanium rx2620 only supports USB1.1, which will have a very slow transfer rate)

- Open the Scanco backup/archive software.

- Select the correct USB device with the arrow buttons in the “USB Device” section.

- Use the “Initialize<” button in the “USB Device” section to initialize the USB device.

- Confirm the Warning message. Note: All data currently on the device will be erased!

- Enter a six-character label

The USB device will be initialized and mounted.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Archive procedure====

- Attach an OpenVMS formatted USB device to the front USB port of the Itanium.
(note: Itanium rx2620 only supports USB1, which will have a very slow transfer rate)

- Open the Scanco backup/archive software.

- Select the correct USB device with the arrow buttons in the “USB Device” section.

- Use the “Mount” button in the “USB Device” section to mount the USB device.

The device label of the mounted USB device will be shown in the “USB Device” section.

- Select the measurement data to be archived.

- Select “Move To:” and “USB/VMS” and click “Start<” to archive the selected data to the USB device.

After the move procedure, the new current location of the measurement data will be shown next to the files path.

- “Dismount” the USB device when done with archiving.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Manual file copy procedure====
An OpenVMS formatted USB device can also be mounted and used for file storage using commands in a DECterm.

- Attach an OpenVMS formatted USB device to the front USB port of the Itanium.
(note: Itanium rx2620 only supports USB1, which will have a very slow transfer rate)

- List available devices. The last shown device is usually the attached USB device; dna12 in this example.
    $ show device dna

- Mount the USB device using the correct device name
    $ mount /over=id dna12

- Create a root directory. Ignore any messages if the directory already exists.
    $ create /dir dna12:[microct]

- Copy any wished files to the USB device
    $ copy {source files} dna12:[microct] /log

- Dismount the USB device
    $ dismount dna12


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Exporting Files / FAT file system====
If an external USB device is to be used to export measurement data to a Windows or MAC OS, the USB device must be formatted as FAT file system.

- Attach the USB device to the front USB port of the Itanium.
(note: Itanium rx2620 only supports USB1, which will have a very slow transfer rate)

- Open the Scanco backup/archive software.

- Do not “Initialize<” or “Mount” the USB device !

- Select the measurement data to be copied to the USB device, may be even sub-select the wished file type
(TIF files in the example below).

- Select “Copy To:” and “USB/FAT” and click “Start<” to copy the selected data to the USB device.

Once the USB device is connected to Windows or MAC OS, it will show a ZIP file named like the copied measurement
number. This ZIP file contains the same directory structure like the data disk on OpenVMS: 

data\sample number\measurement number

In the lowest sub-directory are the copied files, as selected in the backup/archive software (TIF files in this example).


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Problems with connected USB devices===

Sometimes USB configurations can be problematic, in which case log in as "system" on the server and run

  $ dir/date SYS$SYSROOT:[SYSEXE]USB$UCM_DEVICES.DAT

to see whether a file was created recently. If one has, delete that file and reboot the server.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

==CDROM==
===Mount a CDROM===

Device could be <code>dqa0</code> or <code>dqb0</code>.  I'm not sure which works:
<pre>
$ mount/foreign dqa0:
</pre>
OR
<pre>
$ mount/override=id dqa0:
</pre>
Then
<pre>
$ dismount dqa0:
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

==Saveset and Zip files==

===Installation===

The standard <code>ZIP</code> and <code>UNZIP</code> tools on the VMS system cannot manage large file sizes. Therefore it is required to install a version that Mark Blanchard (February 2024) has compiled that specifically manages these large sizes.

Extract from <code>ZIP30L.ZIP</code> and <code>UNZIP60L.ZIP</code> the EXE files <code>ZIP.EXE</code> and <code>UNZIP.EXE</code>. Then as user "system" copy these into <code>DISK0:[SYS0.SYSCOMMON.SYSEXE]</code>. You need to ensure the permissions are set properly too:

 $ set def disk0:[sys0.syscommon.sysexe]
 $ set prot=(s:rwed,o:rwed,g:re,w:re) telnet_client.exe

Previous versions of <code>ZIP.EXE</code> and <code>UNZIP.EXE</code> are stored in the same directory in case it is necessary to revert to an older version.

===Saveset===

====Saveset Naming Convention====
The Scanco backup program stores raw (RAW) and image (IMA) data as save sets (SAV).  
*RAW save sets contain RSQ files and IMA savesets contain all other file types.  
*The saveset filename is composed of the 8-digit measurement number plus "_RAW.SAV" for raw data and "_IMA.SAV" for image data.  
*For example, the data for a scan with measurement number 1234 would be stored as:
<pre>
00001234_RAW.SAV  ?  RSQ
00001234_IMA.SAV  ?  all other files, ISQ, AIM, GOBJ, TXT, SCV, etc.
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Create and Unpack Saveset====
Savesets are OpenVMS backup files, similar to a zip.  Use the backup command.
<pre>
$ BACKUP/LOG disk3:[microct.projects.buie...] disk3:[microct.projects]buie2012.sav/sav	
                !Creates a saveset file of disk3:[microct.projects.buie].  
                !The ... after the folder indicates to include all subfolders and files.
$ BACKUP/LOG mysaveset.sav/sav disk3:[microct.projects.buie]
                !Unpacks the whole saveset to disk3:[microct.projects.buie]
$ BACKUP/LOG mysaveset.sav/sav disk3:[microct.projects.buie...]	
                !The ... at the end of the output location indicates to unpack the saveset and maintain the original directory structure.
$ BACKUP/LOG mysaveset.sav/sav/select=*.aim disk3:[microct.projects.buie]	
                !The select flag is used to indicate extraction of only aim file types.
$ BACKUP/LOG mysaveset.sav/sav/select=[microct.data...] disk3:[microct.projects.buie]	
                !The select flag is used to indicate extraction of the folder data and all subfolders and files.
$ HELP BACKUP   !More information on the backup command.
</pre>

You may want to exclude some files:
<pre>
$ set default DK0:[MICROCT.DATA]
$ BACK/LOG [.00000031.00000600]/EXCL=(*_TRNSFR.COM,*.SAV,*.ZIP,ZI*,*.RSQ) DK0:[MICROCT.DATA.00000031.00000600]3418_00000031_00000600_IMA.SAV/SAV
</pre>

[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====View Saveset Contents====
To view contents of a saveset (SAV) on disk:
<pre>
$ backup/list upat_print_list_nomatch.sav/sav
</pre>

To view the savesets (SAV) on a tape:
<pre>
$ mount/ov=id tp0:
$ dir/date tp0:*.*
$ dismount tp0:
</pre>

To view contents of a saveset (SAV) on a tape:
<pre>
$ mount/foreign tp0:
$ backup/list tp0:*.*  ! OR, backup/list tp0:*.SAV
or
$ backup/list=results.txt tp0:*.*  ! save output in results.txt
$ dismount tp0:
</pre>

An idea to get the results of the backup/list into a text file: Run the procedure in a submitted script and save the resulting log file with all the output.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Extract a single file from SaveSet====

In a large SaveSet, sometimes we only want one file extracted, not the whole archive extracted. This can be done with

  $ backup /log /select=myfile.txt mysaveset.sav /sav disk1:[microct.scratch]

or []  as destination will put the file in the current directory.

This can take a while as it looks through all the saveset. Once it logs that the file has been restored, you may interrupt with F6. Else it goes on looking through the rest of the saveset.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Transfer Saveset from microCT to XtremeCT====

You can create a SAV set of your project on the VivaCT and transfer it to the Xtreme system, or vice-versa.
First, create the SAV set using the command:
<pre>
For example:
$ BACKUP/LOG DISK3:[MICROCT.PROJECTS.SB.MYPROJECT...]*.*;* -
     SYS$SCRATCH:MYPROJECT.SAV/SAV
</pre>
This creates the MYPROJECT.SAV file in the scratch directory.  Now zip the file:
<pre>
$ cd sys$scratch
$ zip "-V" MYPROJECT.ZIP MYPROJECT.SAV  (NOTE: must use quotes and a capital V!!!!)
</pre>
FTP the .ZIP file to SODOR ( in binary mode!!!), and then FTP it off onto your destination system (Viva or Xtreme).  See the appropriate HowTo for FTP instructions.  Once you've FTP'd it, unzip the file:
<pre>
$ unzip MYPROJECT.ZIP
</pre>
Finally, unpack the contents of the .SAV file:
<pre>
$ BACKUP/LOG MYPROJECT.SAV/SAV DISK3:[MICROCT.PROJECTS.MYPROJECT] <--- or whatever destination directory
</pre>
NOTE:  transferring a .SAV directly from VMS to VMS should work, but not via any third operating system (like SODOR) - so zipping is always the safe way to make sure it works.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Zip===

====Zip and Unzip files====
Use the zip program to add files to a zip archive. Make sure you are using version 3 if you want to Zip files greater than 2GB (contact Scanco to install v3 if necessary).
<pre>
$ ZIP myzip.zip myfile.txt;1	                !Adds myfile.txt to myzip.zip.
$ ZIP "-V" myzip.zip myfile.txt;1	        !Adds myfile.txt;1 to myzip.zip.  The "-V" maintains the OpenVMS version number.
$ ZIP "-V" myzip.zip [.subdir...]*.*;*	        !Adds all files within subdir to myzip.zip, maintaining their version number.
$ ZIP	                                        !Displays the help for the zip command.
</pre>

Use the unzip program to unzip an archive.
<pre>
$ UNZIP "-V" myzip.zip		                !Extracts myzip.zip, maintaining the OpenVMS structure.
$ UNZIP		                                !Displays the help for the unzip command.
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Transfer Zip between Scanco systems====

Files must be zipped before transfer by e-mail or ftp because important attributes will be lost in the transfer otherwise.

To transfer files from system A to B, do the following.

On system A:
<pre>
$ zip "-V" for_calgary.zip uctd:UCT_SAMPLES_TORONTOXT.DAT  ! for each filename
$ zip "-V" for_calgary.zip uctd:UCT_MEASUREMENTS_TORONTOXT.DAT
</pre>
This will add both files to the same zip file.

Note: "-V" is very important, it must be capital V!

Transfer the file by e-mail, or by ftp.

On system B:
<pre>
$ unzip -j for_calgary.zip  ! to unzip on your end them (-j to 'junk' the original file path)
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Unpack Zip sent by Scanco====
In the .zip file, you'll find a .sav file. Unzip the .zip just by saying

<pre>$ unzip xxxx.zip</pre>

Then:

<pre>$ backup/log xxxx.sav/sav disk3:[steve.devel.temp]</pre>

and it will unpack everything into <code>disk3:[steve.devel.temp]</code>

Or:

<pre>$ backup/list xxxx.sav/sav</pre>

and you see what's in the .sav.

Then:

<pre>$ backup/log xxx.sav/sav/sel=[andres.uct.ipl_for_cust...] disk3:[steve.devel.cipl...]</pre>

and it will create the same directory tree in <code>steve.devel.cipl</code> as it was when I 
packed it into the .sav.

[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Tips for making and unpacking SAVSETS====
Separate all "input" files with comma, and no space in between (space separated parameters) :

<pre>
$ backup/log D0014097.RAD,D0014097.SCV myfiles.sav/sav
</pre>

You may want to consider to only take the newest/highest file version:

<pre>
$ backup/log D0014097.RAD;,D0014097.SCV; myfiles.sav/sav
</pre>

When unpacking a SAVSET, if you want the directory structure to be maintained, do something like this:
<pre>
$ BACKUP/LOG ACKNEE_HUMAN_2020-02-18.SAV/SAV/SELECT=([BONELAB.PROJECTS.ACKNEE...]*.*) DISK4:[BONELAB.PROJECTS.ACKNEE...]
</pre>

Extracting an archived SAVESET back onto the VMS system in location different from where it was originally located, for example putting it
on DISK7:[XTREMECT2.EXTERNAL] instead of DISK4:[BONELAB.PROJECTS], do the following:
<pre>
$ SET FILE/ATTR=(RFM=FIX,LRL=32256) UBC_HUMAN_2024-03-25.SAV
$ BACKUP/LOG UBC_HUMAN_2024-03-25.SAV/SAV/SELECT=([BONELAB.PROJECTS.UBC...]*.*) DISK7:[XTREMECT2.EXTERNAL.UBC...]
</pre>
[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

==Backup and Archive==

===UI Procedures===

====µCT Disk-Utility====
The µCT Disk-Utility Program is used to archive and restore scan data. To start the program type <code>$ uct_backup</code> in a DEC terminal, or use the disk button.

[[File:Uct_backup.jpg|40px]]
*The backup captures both the programs and databases.
*A little helper terminal comes up and gives feedback of every file backed up.
[[File:Uct_backup_pic01.jpg|600px]] 


Two possible ways to use <code>UCT_BACKUP</code>:
:* COPY TO -> this does not update the database.
:* MOVE TO -> this updates the database.

Scenario 1: 
<pre>
1. Use UCT_BACKUP to 'move' the RSQ to 'tape'
2. Use UCT_BACKUP to 'move' the ISQ to 'garbage'
3. Use UCT_BACKUP to 'move' the remainder (AIM/GOBJ) to 'tape'
</pre>
*At operation 3, the measurement directory will be deleted from the measurement disk.  
*The directory will automatically be restored if you restore your data from tape.

Scenario 2:
<pre>
1. Use UCT_BACKUP to 'move' the RSQ to 'tape'
2. Use UCT_BACKUP to 'move' the ISQ to 'garbage'
3. Move all the GOBJ/AIM/etc to a project directory.
4. Generate a .SAV file and put on the IMA tape.
</pre>
*You will have to delete the measurement directories (using UCT_BACKUP if you like). 
*The disadvantage of this scenario is that the image data cannot be restored using UCT_BACKUP.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Raw Data Backup====
*Ideally 1-2 week after scanning.
*RSQ files are backed up onto RAW tapes (see [[Scanco_Administration#Tape_Label_Convention|'''Tape Label Convention''']]).  Put the most recent RAW tape into the tape drive, then press "Mount". 
*Show "Current (with RAW)", use the file filter (Measurement:     ) to search for your files (e.g., CAMOS) and press enter or press "Update List".  
*Select all the files to backup by highlighting them using the left mouse button and ctrl to select individual files or shift to select a range.  
*Highlight RAW, choose "Move To" and "Tape".  Press "Start...".  
*When the backup is complete press "Dismount" to eject the tape. 
*If the tape auto-ejected without pressing "Dismount" then the tape is full (see [[Scanco_Administration#Full_Tape|'''Full Tape''']]).
*If you select multiple files, be sure they all need to be backed up, otherwise you will get an error.
:[[File:Uct_backup_pic04.jpg|500px]]


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Image Data Backup====
*Ideally 1-2 months after scanning, after contouring, onsite analysis and copied image data to project disk.

#'''Delete ISQ After Evaluation''' 
#*ISQ files are normally deleted after completing your evaluation since they are very large files and can be reconstructed from the RSQ file. 
#*Show "Current (on Disk)", type the beginning of your sample name (e.g., CAMOS) in the file filter and press enter or press "Update List".
#*Select all the files for which you have completed the evaluation (there will be a + sign next to any scan that has been evaluated).  #*Highlight IMA, choose "Move To" and "Trash".  
#*[[File:Important.JPG|20px]] After you choose the trash symbol, you will now be given the option to select specific file types, choose "Select" and "ISQ".  Press "Start...".
#*Only choose one file type to delete at a time. The wildcard (*) indicate all files other than "ISQ" and "AIM" files.
#*Once you have deleted all of the different file types the directory itself needs to be deleted, which can be done by deleting "all file types"
#:[[File:Uct_backup_pic05.jpg|500px]]
#:<br>
#'''Backup IMAGE Data''' 
#*AIM, GOBJ and other evaluation files are backed up onto IMA tapes (see [[Scanco_Administration#Tape_Label_Convention|'''Tape Label Convention''']]).  
#*Put the most recent IMA tape into the tape drive, then press "Mount". 
#*Show "Current (on Disk)", type in the beginning of your sample names (e.g., CAMOS) in the file filter and press enter or press "Update List". 
#*Select all the files to backup by highlighting them using the left mouse button and ctrl to select individual files or shift to select a range.  
#*Highlight IMA, choose "Move To" and "Tape".  Press "Start...".   
#*When the backup is complete press "Dismount" to eject the tape.  
#*If the tape auto-ejected without pressing "Dismount" then the tape is full (see [[Scanco_Administration#Full_Tape|'''Full Tape''']]).
#:[[File:Uct_backup_pic06.jpg|500px]]


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Use 'Script...' button====
Usually, you will start the move/copy by clicking on <code>'Start...'</code> button. The <code>'Script...'</code> button is for advanced users.

*Use of the 'Script...' button will generate a script <code>sys$scratch:archive.com</code>, that you may submit it in batch mode later (e.g. over night) by typing: <pre>$ scanco_submit_command sys$scratch:archive.com/after=23:59</pre>
*In a multi-computer cluster setup, you may want to specify the exact queue name of the computer that has the tape hooked up by giving e.g. <code>/que=sceval_fast</code>.
*Insert a tape, but do NOT mount the tape in the normal program. If necessary: if it's a new tape, [[Scanco_Administration#Initialize_a_New_Tape|'''initialize''']] it as usual. 
*The batch job will be started one minute before midnight in the example above. <code>$ show entry</code> will show you that it is holding until then.
*See <code>$ help submit /after</code> and <code>$ help dcl_tips date_time example</code> for more help on this.
*Sample script file (archive.com) restoring the RSQs of measurements 6452 and 6580:
<pre>
$ MOUNT/FOREIGN TP0:
$ UCT_BACKUP_BATCH
ALL
RAW
COPY
DISK
! Meas: SB_BUIE_EX_CTRL_PH_01 (6452)
MEAS
6452
EXECUTE
! Meas: SB_BUIE_EX_CTRL_PH_01 (6580)
MEAS
6580
EXECUTE
EXIT
$ DISMOUNT TP0:
$ EXIT 
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Database Backup====

The database (DB) backup should be performed for each user on the system. It should not be performed as user 'calibration' or some derivative. In the case of the XtremeCT2 and XtremeCT cluster, both the 'microct' and 'xtremect2' user accounts must be used to backup their respective DB. 

By performing the DB backup, the warning dialog that appears when logging into the system should go away.  


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Operating System (OS) Backup====
If the following error-message appears on the screen then you can click OK to continue your work. 

     MESSAGE: No recent OS Backup found!
     Please perform OS backup (DISK0) in privileged account!
     %RMS-W-FNF file not found.
     CODE: 268534416
     Facility RTL

This message means that no operating system (OS) backup has been performed within the last 6 months. Please perform an OS backup as soon as possible by following the steps below:

1. Make sure there are no scans in progress and that the que is halted from all activity.

2. Log in to the calibration account: Since we have two scanners on the same system, login as either "calib3401" or "calib3309", depending if you need to backup the XtremeCT2 or XtremeCT, respectively. For microCT clusters, (best to do this on SCEVA3) using the username calib5019 and the password (ask Steve for this).

3. Type in the DECterm window:
          $ set proc/priv=bypass
Or Set priviledge to bypass by <code>$ spb</code>
          $ r um:uct_b     Or $ uct_backup

4. You will now see 2 additional buttons on the bottom left that is not seen when logged in as microct user. Push either the button "OS Backup to Disk DK0" or mount the specific OS backup tape and push the "OS Backup to Tape" button.
* A system backup to tape and to DK0 should both be performed. 

5. Check the command line & queue to ensure that the backup is being carried out. If you submit the job as 'batch mode' the backup is submitted to SYS$BACKUP queue and allows you to close the backup program and 'end session' without interrupting a running backup. Check with 'show queue' to see the backup job on the queue.

6. Exit the Backup Program and log out of the calibration account.

After a successful OS backup the message above will not appear for the next six months.

Alternatively a system backup can be completed using the command line, without using the uct_backup program. The instructions from Scanco can be found here. The advantage to this method is that you will be able to name the specific backup save set.

[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

===Using Scripts for Backup/Archive===

====Project Backup/Archive====
*Refer to [[Media:Microct_project_backup.pdf|'''Project Backup SOP''']] or [[Media:Microct_project_backup.docx|''' its Word format''']] for reference.


=====Backup Project=====
Regularly backup projects on a monthly basis using command <code>$ @com:BACKUP_PROJECT.COM</code>
*The script is submitted to the queue and upon completion the log is emailed to the user.
*a SAVset (an OpenVMS file type similar to a zip file), a project info file and a file list are created in DISK5:[BONELAB.PROJECTS.BACKUP] for the microCTs and DISK4:[BONELAB.PROJECTS.ARCHIVE.ARCHIVE] for the XtremeCTs.
*The saveset along with two TXT files created will be automatically SFTP to Winston:\\Backups\backup\
*After the transfer, the saveset file on disk are deleted, leaving only two TXT files on the project disk.
*Only keep the two most recent backups for each project, A or B version name appended on the project name:
:*Backup version "_A" for odd months (Jan, Mar, May, Jul, Sep, Nov), e.g. YZHU_A_OTHER_2017-01-12.SAV
:*Backup version "_B" for even months (Feb, Apr, Jun, Aug, Oct, Dec), e.g. YZHU_B_OTHER_2020-04-08.SAV
*The user MUST verify that the backup completed successfully.
*SAVsets greater than 250 GB cannot be transferred by automatic SFTP, use interactive/manual SFTP instead to transfer to Winston. 
*If you are backing up a very large project, please check the server disk space before starting the backup. 

<pre>
$ @com:BACKUP_PROJECT.COM
===========================================================
BACKUP PROJECT TOOL V1.2 (XtremeCT)
 P1:LOCATION        DISK4:[BONELAB.PROJECTS.DUNCAN]
 P2:TYPE            1=MOUSE,2=RAT,3=HUMAN,4=OTHER
 P3:PROJECT         CAMOS
 P4:PERSON          Ying Zhu
 P5:DESCRIPTION     A monthly backup
 P6:EMAIL           me@ucalgary.ca
 P7:SFTP            y,n
 P8:START           y,n
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

=====Restore Project=====
In the commands directory (com) is a BACKUP_RESTORE.COM script which will automatically generate a script to restore a user's project.  
*The saveset file has to be moved to Winston:\\Backups\archive folder to be restored.
*The user will be asked if they wish to restore to the original location, or to a new location.
*The script is submitted to the queue and the log from the restore job is emailed to the user. 
*The user MUST verify that the job completed successfully.
<pre>
$ @COM:BACKUP_RESTORE.COM
</pre>
 

=====Delete Project=====
In the commands directory (com) is a BACKUP_DELETE.COM script which will automatically generate a script to delete all the files and subdirectories of a specified path.
*The command script is opened so that the user can check the files to be removed to go ahead.
*You will need to manually delete the root project directory after it's empty.
<pre>
$ @COM:BACKUP_DELETE.COM
</pre>
 

=====Known Bugs in Project Tools=====
*Long file name won't work (28 letters work but not 32 letters).
*if the user specifies too long a file name, the Project Backup Scripts will fail with the following error:
     RMS-E-CRE, ACP file create failed
     SYSTEM-W-BADFILENAME, bad file name syntax


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

=====Moving data disk directories=====

Sometimes when the data disk (DISK2) is getting too full measurement directories can be moved to a temporary location. We can move from DISK2 to another location:

 BACKUP DISK2:[XTREMECT2.DATA.00002784...] DISK7:[XTREMECT2.DRF_STORE.00002784...]/LOG/VERIFY

and back again:

 BACKUP DISK7:[XTREMECT2.DRF_STORE.00002784...] DISK2:[XTREMECT2.DATA.00002784...]/LOG/VERIFY

Check out the options for the <code>BACKUP</code> command for more options. One that might be useful is the <code>/DELETE</code> option, which deletes the files after verifying the move.

Sometimes the <code>uct_backup</code> tool loses information about where the data is stored after it's been moved back to DISK2. In this case you can reset the location as follows:

:*Start <code>uct_backup</code>
:*Navigate to the scan in question and you'll see the IMA section is blank
:*While holding the <code>SHIFT</code> key click on the radio button for IMA
:*In the blank text box, type "DK0" and the rest of the path should automatically fill out
:*Save the new path location information

Now you can run <code>UCT_EVALUATION</code> as you normally would. Do not use <code>DEF UCT_DEFAULT_ISQ DISK2:[XTREMECT2.DATA.00002784.00012345]D0013456.ISQ</code> because the evaluation will not complete properly because information about sample/measurement will not be available.

[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

=====Archive Project Saveset to Tape=====
Projects should be stored safely onto a backup tape once complete.  
Prior to archiving a project please create a directory in DISK4:[BONELAB.PROJECTS.COM] of the project name. Please copy over the com.dir so we can keep all of the scripts on the openVMS system 

<b>Summary</b>

The following steps summarize the process:
:* Archive the project directory to the tape (detailed instructions below).
:* Change tape to "read only" to protect contents
:* Check tape saveset to ensure proper backup
:* Write name of tape on the project LOG sheet
:* Copy project LOG sheet and bring to Bone Imaging Laboratory "completed" project book
:* Bring copy of tape to the Bone Imaging Laboratory for safe keeping
:* <b>Delete</b> the contents of the project from the disk

In the project, there should be the regular set of directories including DOC, COM, MODELS, LOG.  Once the project is stored to tape, ensure that the log sheet for that project include a note stating which tape it was backed up to.  Take a photocopy of the log sheet and add it to our Bone Imaging Laboratory project book for COMPLETED projects.

In the following example, a project titled BERN is backed up onto tape.  Ideally, the name of the SAV file on tape is the same as the project (i.e., BERN).  Note that <b>SAV files must have names with no more than 13 characters (including the .SAV)!</b>

The following example backs the BERN project/directory onto tape IMA001.  For large projects and/or critical projects, it is advisable to dedicate an entire tape.  The three dots (...) indicate that ALL subfolders of DK0:[MICROCT.PROJECTS.BERN] will be included.

<pre>
$ mount/foreign tp0:

$ backup/log dk0:[microct.projects.bern...]*.*;* tp0:proj_bern.sav -  ! writes a saveset to tape
        /block=65534 /label=IMA001 -
        /media_format=nocompaction     ! NEVER use the /rewind option here

$ backup/log tp0:*.sav/select=*.isq DISK4:[BONELAB.PROJECTS.TBONE.ISQ_FROM_TAPE...]/by=parent  ! extracts all ISQ files from the tape

$ dismount tp0:  
$ dismount tp0: /nounload ! same as previous command, but does not eject the tape (i.e. if you want to read the tape once more)
</pre>

Note that the <code>/rewind</code> option should never be used for backup as it instructs the tape drive to rewind after backup is complete causing a major risk of over writing.

Sometimes it might be handy to exclude certain files.  For example, to exclude RSQ files: <code>/excl=*.rsq</code>

A good reference for backup commands and ideas are available in the <code>UM:UCT_MOVE_FILES.COM</code> script which is used by the <code>UCT_BACKUP</code> program.  See also the <code>help backup</code> instructions:
<pre>
$ help backup
</pre>

To view the contents of a tape to ensure you've successfully stored your project use the following commands (further details in "HowTo: View save set (SAV) contents"):

To just view if the saveset is on tape (but not its contents):
<pre>
$ mount/ov=id tp0:
$ dir/date tp0:*.*
$ dismount tp0:
</pre>

Note: you can also save the saveset list in a text file using the /output qualifier.

To view the files within a saveset on tape, and save the results in a file called <code>results.txt</code> This .txt file can be found in IDISK1[XTREMECT2]:
<pre>
$ mount/foreign tp0:
$ backup/list=results.txt tp0:*.*
$ dismount tp0:
</pre>

To delete a set of directories, use the <code>dl</code> command.
<pre>
$ cd ur BERN
$ del *.*;*   ! Deletes ALL files in a directory
$ dl *.dir;*  ! Will only delete empty directories (not nested directories)
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

=====Extract Project Saveset from Tape=====
To restore a project, you have two choices:
*1. Login as user <code>calibration</code> and then go to the projects directory. From there:
<pre>
$ spb (bypass priv)
$ backup/log tp0:proj_davison.sav/rewind [*...]
</pre>
This puts all the files back in their original directory structure.

*2. As user <code>microct</code>
<pre>
$ mount/foreign tp0:
 
$ backup/log tp0:proj_davison.sav/rewind/select=[microct.projects.davison...]
     disk2:[microct.projects.davison...]

OR

$ backup/log tp0:*.*/rewind disk3:[microct.projects.vitamink.models]
</pre>
Make sure to write the <b>same</b> path in the /select as the output (except for the 
actual disk2 portion).  Alternatively, you could unpack it into a slightly different root
directory (i.e., TMP) by changing the output path.

The three dots infers to use the same paths in the saveset as the file structure put onto disk.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Scripts to Backup RAW and IMA====
A script can be output from the backup tool to backup and restore data using the backup program.  
*When restoring data using this method, choose the COPY to DISK option. The data will be restored to its original location on the data disk.  
*You must manually submit the script to the backup queue.  
*The command procedures are written to SYS$SCRATCH:ARCHIVE.COM.  
*Here is the sample script for backing up RAW data:
<pre>
$ MOUNT/FOREIGN TP0:
$ UCT_BACKUP_BATCH
ALL
RAW
MOVE
TAPE
! Meas: ATHLETE_126 (10565)
MEAS
10565
EXECUTE
! Meas: ATHLETE_126 (10566)
MEAS
10566
EXECUTE
! Meas: ATHLETE_126 (10567)
MEAS
10567
EXECUTE
EXIT
$ DISMOUNT TP0:
$ EXIT
</pre>
*Here is the sample script for backing up IMA data:
<pre>
$ MOUNT/FOREIGN TP0:
$ UCT_BACKUP_BATCH
ALL
IMA
MOVE
TAPE
! Meas: ATHLETE_126 (10565)
MEAS
10565
EXECUTE
! Meas: ATHLETE_126 (10566)
MEAS
10566
EXECUTE
! Meas: ATHLETE_126 (10567)
MEAS
10567
EXECUTE
EXIT
$ DISMOUNT TP0:
$ EXI
</pre>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Disk Usage information====
Run the <code>DISK_MOUNTIES.COM</code> script and it will produce a file in the scratch directory called <code>DISK_MOUNTIES_OUTPUT.TXT</code> that contains a row entry for each directory on the disk. It'll include the size and number of RSQ, ISQ, etc, files. If running for the first time, it is suggested you run as follows to learn about the options:
<pre>
$ @COM:DISK_MOUNTIES.COM HELP
</pre>
Probably using the default SF is what you will need. It can be run like this:
<pre>
$ @COM:DISK_MOUNTIES.COM
</pre>
At the end of the script it calls <code>SUMMARIZE_DISK_MOUNTIES.COM</code> that will take the resulting <code>DISK_MOUNTIES_OUTPUT.TXT</code> and provide a summary outcome. It seems that this script is not working because it uses a specific <code>ICALC</code> function that is only available on Alpha stations (which we no longer use). To date (January, 2022) there is no solution to this problem.

You can find the output file (not the summarized version) here:
<pre>
$ eve sys$scratch:DISK_MOUNTIES_OUTPUT_SUMMARY.TXT
</pre>

The template file for summarizing the disk mounties results: [[Media:BoneLab_Data_Disc_Usage.xlsx|'''BoneLab_Data_Disc_Usage.xlsx''']]



To look at the RSQs in the database
<pre>
$ dir/siz dk0:[microct.data...]*.rsq
</pre>
the three dots means all subdirectories.

Other useful commands:

<code>$ ctheader xxxxxx.isq</code>

or

<code>$ ctheader xxxxxx.rsq</code>

shows you header.
<pre>
$ fda command file-wildcard
</pre>
does a 'file do all' of the command on all the given wildcard filenames.
<pre>
$ pipe xxxx | search sys$pipe "patient"
</pre>
is the pipe command in VMS.

So all three combined:
<pre>
$ pipe fda ctheader dk0:[microct.data...]*.rsq | search sys$pipe "patient name"
</pre>
or any other field you want to search for! 

If you have a more complex program to run (e.g. with an argument), use the double quotes as follows:
<pre>
pipe fda "aix -l" *.AIM | search sys$pipe "Patient Name"
</pre>

'''Other ideas''':

<pre>$ write_info</pre>

gives you a file <code>sys$scratch:info.txt</code> with info in it (leave defaults to get all samples ever scanned, otherwise 'zoom' in on some samples.)

Check out the <code>search</code> command:
<pre>
$ help search
</pre>
and <code>/match=and</code>
or <code>/match=or</code>


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]

====Disk Backup====
In the commands directory (com) is a BACKUPDISK script which will automatically generate scripts to backup all the directories on the projects disk.  
*As the backup requires significant amounts of space, please check the server disk space before starting the backup. 
*Each directory is saved as a separate save set (an OpenVMS file type similar to a zip file) on Winston://Bonelab/Backup/.

<pre>
$ @COM:BACKUPDISK
</pre>

*Refer to [[Media:Microct_disk_backup.pdf|'''Disk Backup SOP''']] or [[Media:Microct_disk_backup.docx|''' its Word format''']] for reference.


[[Image:Redirect.jpg|15px]] [[#top|''Return to Contents'']] or [[Main_Page|''Return to Main Page'']]