[90:114] Cannot unload medium, target slot (255) appears to be occupied

[Warning] From: BMA@linuxsrv155.com "ESL01_D17"  Time: 2/9/2015 1:13:44 PM
[90:54]   /dev/nst1
Cannot open device (Unit not ready.)

[Major] From: BMA@linuxsrv155.com "ESL01_D17"  Time: 2/9/2015 1:13:55 PM
[90:114]   By: UMA@mediasrv01.com@/dev/rchgr/autoch3
Cannot unload medium, target slot (255) appears to be occupied

Solution:

> The above error is due to DP couldn't unload the media from where it was actually loaded to the drive during backup/restore.

> Move the tape manually from the drive to any free slot using UMA by following the steps below. 

> Login to the media server from where the tape library/autoloader is connected.

>> run devbra-dev, the output will be the drives and exchanger detected from the device manager.

>> copy the scsi path of exchanger (here it is /dev/rchgr/autoch3)
>> execute the below command and move the media to any empty slot.
>uma -ioctl /dev/rchgr/autoch3

/dev/rchgr/autoch3> move D17 S13

> If there are no empty slots available, eject full media to I/O slots and move the media stuck in drive to free slot.
> Then retry the backup, which should be successful.

Hope this helps!

[Major] Host for device "ESL01_D01" not found.

[Normal] From: BSM@cellsrv01.in.com "Backupspec_win1"  Time: 3/21/2014 6:04:27 PM

 Backup session 2014/03/21-104 started.

[Major] From: BSM@cellsrv01.in.com "Backupspec_win1"  Time: 3/21/2014 6:04:32 PM

 Host for device ESL01_D01 not found.

[Major] From: BSM@cellsrv01.in.com "Backupspec_win1"  Time: 3/21/2014 6:04:32 PM

 Host for device ESL01_D02 not found.

[Major] From: BSM@cellsrv01.in.com "Backupspec_win1"  Time: 3/21/2014 6:04:32 PM

 Host for device ESL01_D03 not found.

[Normal] From: BSM@cellsrv01.in.com "Backupspec_win1"  Time: 3/21/2014 6:04:32 PM

 Backup Statistics:

         

  Session Queuing Time (hours)         0.00      

  -------------------------------------------    

  Completed Disk Agents ........          0        

  Failed Disk Agents ...........          7        

  Aborted Disk Agents ..........          0        

  -------------------------------------------    

  Disk Agents Total  ...........          7        

  ===========================================    

  Completed Media Agents .......          0        

  Failed Media Agents ..........          0        

  Aborted Media Agents .........          0        

  -------------------------------------------    

  Media Agents Total  ..........          0        

  ===========================================    

  Mbytes Total .................       0 MB      

  Used Media Total .............          0        

  Disk Agent Errors Total ......          0  

This error occurs when the backup device wasn't removed from the devices context following the media server deletion/migration to other cell.

In other words, the media server could have moved to another cell server or might have been decommissioned and by mistake the devices were not removed from the devices list.

Check the media server's existence, if available;

Check the cell server name, it is pointing to;

Mostly, if the orphan device entry is removed, this error will disappear.

[90:1004] Device address not found.

[Normal] From: BSM@cellsrv01.in.com "Backup_Spec_Win1"  Time: 4/16/2014 12:08:46 PM

 Backup session 2014/04/16-256 started.

[Normal] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:08:55 PM

 STARTING Media Agent "AUT01_D01"

[Normal] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:09:01 PM

 By: UMA@winsrv01.in.com@Changer0:7:0:1

 Loading medium from slot 8 to device Tape1:7:0:0C

[Warning] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:09:57 PM

 The device "AUT01_D01" could not be opened("Device could not be accessed")

[Normal] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:09:57 PM

 Starting the device path discovery process.

[Critical] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:10:00 PM

[90:1004]  Device address not found.

[Normal] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:10:00 PM

 Device path discovery process finished.

[Normal] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:10:00 PM

 By: UMA@winsrv01.in.com@Changer0:7:0:1

 Unloading medium to slot 8 from device Tape1:7:0:0C

[Normal] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:10:39 PM

 ABORTED Media Agent "AUT01_D01"

[Normal] From: BSM@cellsrv01.in.com "Backup_Spec_Win1"  Time: 4/16/2014 12:10:39 PM

 Backup Statistics:

         

  Session Queuing Time (hours)         0.00      

  -------------------------------------------    

  Completed Disk Agents ........          0        

  Failed Disk Agents ...........          4        

  Aborted Disk Agents ..........          0        

  -------------------------------------------    

  Disk Agents Total  ...........          4        

  ===========================================    

  Completed Media Agents .......          0        

  Failed Media Agents ..........          1        

  Aborted Media Agents .........          0        

  -------------------------------------------    

  Media Agents Total  ..........          1        

  ===========================================    

  Mbytes Total .................       0 MB      

  Used Media Total .............          0        

  Disk Agent Errors Total ......          0  

Troubleshooting steps as follows:

>> Logged into the media server and checked for devices claimed in Device Manager. Found the devices.

>> Ran devbra -dev to determine the SCSI address.

>> Found N/A for drive

C:\>devbra -dev

Exch    HP:1x8 G2 AUTOLDR  Path: "Changer0:0:0:1"  SN: "AABBCCDD1E"

        Description: CLAIMED:HP StorageWorks 1x8 Cartridge Autoloader

        Revision: 4.20  Flags: 0x0016  Slots: 8  Drives: 1

        Drive(s) SN:

                "ABCDEFGHIJ"

Tape    HP:Ultrium 3-SCSI  Path: "Tape0:0:0:0"  SN: "N/A"

        Description: CLAIMED:HP LTO3 Drive

        Revision: Q51W  Device type: lto [13]  Flags: 0x0011

>> Checked if the drive is locked by DP

[root@cellsrv01:/root]

# omnimm -show_locked_devs | grep AUT01_D01

[root@ cellsrv01:/root]

>> Stopped DP Inet from Services.msc and Ran LTT.

>> Got error message that "mma.exe" process is accessing the device. Killed the mma.exe process from the Task Manager.

>> rescanned the devices in LTT. Able to detect both autoloader and it's drive this time.

>> Ran devbra -dev, which got the SCSI address.

C:\>devbra -dev

Exch    HP:1x8 G2 AUTOLDR  Path: "Changer0:7:0:1"  SN: "AABBCCDD1E "

        Description: CLAIMED:HP StorageWorks 1x8 Cartridge Autoloader

        Revision: 4.20  Flags: 0x0016  Slots: 8  Drives: 1

        Drive(s) SN:

                "ABCDEFGHIJ "

Tape    HP:Ultrium 3-SCSI  Path: "Tape1:7:0:0C"  SN: "ABCDEFGHIJ "

        Description: CLAIMED:HP LTO3 Drive

        Revision: Q51W  Device type: lto [13]  Flags: 0x0011

>> Ran the backup successfully.

Cause :

The DP media agent was accessing the drive and didn't let any other process to send commands. After the hung process is killed, the drive was accessible for normal operations.

[90:801] Active Removable Storage Manager (RSM) service found on local system.

[Normal] From: BMA@mediasrv01.in.com "MSL2024_D02"  Time: 3/23/2014 2:57:16 PM

 STARTING Media Agent "MSL2024_D02"

[Warning] From: BMA@mediasrv01.in.com "MSL2024_D02"  Time: 3/23/2014 2:57:16 PM

[90:801]  Active Removable Storage Manager (RSM) service found on local system.

 [Warning] From: UMA@mediasrv01.in.com "MSL2024"  Time: 3/23/2014 2:57:18 PM

[90:59]   Changer0:0:0:0

 Cannot open exchanger control device ([2] The system cannot find the file specified. )

[Warning] From: UMA@mediasrv01.in.com "MSL2024"  Time: 3/23/2014 2:57:18 PM

 The device "MSL2024" could not be opened ("Device could not be accessed") 

[Normal] From: UMA@mediasrv01.in.com "MSL2024"  Time: 3/23/2014 2:57:18 PM

 Starting the device path discovery process. 

[Warning] From: UMA@mediasrv01.in.com "MSL2024"  Time: 3/23/2014 2:57:19 PM

[90:59]   Changer0:0:0:0

 Cannot open exchanger control device ([2] The system cannot find the file specified. )

[Normal] From: BMA@ mediasrv01.in.com "MSL2024"  Time: 3/23/2014 2:57:21 PM ABORTED Media Agent " MSL2024_D02"

Solution:

Removable Storage Manager (RSM) service on MS Windows 2003 might cause device issues in backup environment.

It is recommended to stop and disable the RSM service on all MS Windows 2003 systems.

RSM Service in Started status

[61:2015] Timeout waiting for the devices to get free.

[Critical] From: BSM@cellsrv01.in.com "cellsrv01_IDB"  Time: 02/27/14 02:00:31

[61:2015]  Timeout waiting for the devices to get free.

The session will terminate.

Ø  It is very common in every backup environment to share the same device for different backups. This is an error message due to device contention issue.

Ø  The backup device selected in the backup specification is unavailable for the backup to start. It is in use by another process or by another backup/restore/copy sessions.

Ø   Check the device status using the lock name. (Check Here for the commands used to find a locked device). Wait for the device to be free.

Ø  The backup will be queued for global timeout seconds and will fail if no device is freed / allocated to the backup session.

Ø  The queuing time can be found at the end of backup session from backup statistics. Shown below

  

Backup Statistics:

          

                   Session Queuing Time (hours)         0.00        

                   -------------------------------------------      

                   Completed Disk Agents ........          5          

                   Failed Disk Agents ...........          0          

                   Aborted Disk Agents ..........          0          

                   -------------------------------------------      

                   Disk Agents Total  ...........          5          

                   =====================================     

                   Completed Media Agents .......          1          

                   Failed Media Agents ..........          0          

                   Aborted Media Agents .........          0          

                   -------------------------------------------      

                   Media Agents Total  ..........          1          

                   ===========================================      

                   Mbytes Total .................   17985 MB        

                   Used Media Total .............          1          

                   Disk Agent Errors Total ......          0    

[90:63] Cannot load exchanger medium (Medium error.)

[Normal] From: BMA@winsrv01.in.com "MSL_D01"  Time: 3/24/2014 10:53:43 PM

 By: UMA@winsrv01.in.com@Changer0:0:0:0

 Loading medium from slot 9 to device Tape0:0:0:0

[Major] From: BMA@winsrv01.in.com "MSL_D01"  Time: 3/24/2014 11:03:16 PM

[90:63]   By: UMA@winsrv01.in.com@Changer0:0:0:0

 Cannot load exchanger medium (Medium error.)

[Normal] From: BMA@winsrv01.in.com "MSL_D01"  Time: 3/24/2014 11:03:16 PM

 ABORTED Media Agent "MSL_D01"

[Normal] From: BMA@winsrv01.in.com " MSL_D02"  Time: 3/24/2014 11:03:21 PM

 STARTING Media Agent " MSL_D02"

[Normal] From: BMA@winsrv01.in.com " MSL_D02"  Time: 3/24/2014 11:03:26 PM

 By: UMA@winsrv01.in.com@Changer0:0:0:0

 Loading medium from slot 9 to device Tape1:0:0:0

[Major] From: BMA@winsrv01.in.com " MSL_D02"  Time: 3/24/2014 11:13:10 PM

[90:63]   By: UMA@winsrv01.in.com@Changer0:0:0:0

 Cannot load exchanger medium (Medium error.)

[Normal] From: BMA@winsrv01.in.com " MSL_D02"  Time: 3/24/2014 11:13:10 PM

 ABORTED Media Agent " MSL_D02"

Solution:

Exchanger medium is failing while handling media from particular slot (here it is slot 9). The media has errors that is residing slot 9.

Move to 'Devices & Media' context -> Click Library -> Click Slot -> move to Slot 9.

--- Check the status of media in slot 9.

--- It was poor media. Isolate the media by moving it to separate media pool and unload it from library. Wait for the media to expire and reformat it. If the status is still shows poor even after formatting, it is not advisable to use it henceforth.

--- Clean the affected drives with good cleaning tape and run a test backup. They should be good for backups.

[61:4006] Couldnot connect to inet in order to start BMA@ "Device_name".

Backup Error

[Major] From: BSM@cellsrv01.in.com "winsrvspec01"  Time: 2/28/2014 6:53:57 PM
 [61:4006]   Couldnot connect to inet in order to start BMA@Mediasrv02.in.com "MSL2024_D1".

[Critical] From: BMA@cellsrv01.in.com "MSL2024_D1"  Time: 2/28/2014 6:54:55 PM
[90:1004] Device address not found.

Solution

>> The DP Inet service running in the media server is not reachable from the cell server and the Media Agent couldn't be started. When checked the media server is 'Removed' from production and was shutdown.

>> In this case, the device mapping of the drive should be removed/disabled 'MSL2024_D1' from DP and the backup spec need to be modified accordingly to eliminate failures in future.

>> You can verify the media server reachability from cell manager by pinging, telnet the DP port 5555, can also use omnitcpchk commands.

For ex:

[root@ cellsrv01:/root]

# ping Mediasrv02.in.com 

PING Mediasrv02.in.com: 64 byte packets

----Mediasrv02.in.com PING Statistics----

11 packets transmitted, 0 packets received, 100% packet loss

[root@ cellsrv01:/root]

# /opt/omni/sbin/utilns/omnitcpchk -host Mediasrv02.in.com 

Testing connection Mediasrv02.in.com <---> cellsrv01.in.com....

ERROR!

==============================

TcpCheck failed for host pair:

Mediasrv02.in.com <---> cellsrv01.in.com

[root@ cellsrv01:/root]

# telnet Mediasrv02.in.com 5555

Trying...

telnet: Unable to connect to remote host: Connection timed out

>> The backup would complete using the next device in line, if there are more than one device selected in the backup specification.

  

>> Optionally, the backup would also run with the next zoned drive that has correct SCSI path and available for this backup session.