Tuesday, April 22, 2014

[90:1004] Device address not found.

[Normal] From: BSM@cellsrv01.in.com "Backup_Spec_Win1"  Time: 4/16/2014 12:08:46 PM
 Backup session 2014/04/16-256 started.

[Normal] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:08:55 PM
 STARTING Media Agent "AUT01_D01"

[Normal] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:09:01 PM
 By: UMA@winsrv01.in.com@Changer0:7:0:1
 Loading medium from slot 8 to device Tape1:7:0:0C

[Warning] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:09:57 PM
 The device "AUT01_D01" could not be opened("Device could not be accessed")

[Normal] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:09:57 PM
 Starting the device path discovery process.

[Critical] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:10:00 PM
[90:1004]  Device address not found.

[Normal] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:10:00 PM
 Device path discovery process finished.

[Normal] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:10:00 PM
 By: UMA@winsrv01.in.com@Changer0:7:0:1
 Unloading medium to slot 8 from device Tape1:7:0:0C

[Normal] From: BMA@winsrv01.in.com "AUT01_D01"  Time: 4/16/2014 12:10:39 PM
 ABORTED Media Agent "AUT01_D01"

[Normal] From: BSM@cellsrv01.in.com "Backup_Spec_Win1"  Time: 4/16/2014 12:10:39 PM

 Backup Statistics:
         
  Session Queuing Time (hours)         0.00      
  -------------------------------------------    
  Completed Disk Agents ........          0        
  Failed Disk Agents ...........          4        
  Aborted Disk Agents ..........          0        
  -------------------------------------------    
  Disk Agents Total  ...........          4        
  ===========================================    
  Completed Media Agents .......          0        
  Failed Media Agents ..........          1        
  Aborted Media Agents .........          0        
  -------------------------------------------    
  Media Agents Total  ..........          1        
  ===========================================    
  Mbytes Total .................       0 MB      
  Used Media Total .............          0        
  Disk Agent Errors Total ......          0  


Troubleshooting steps as follows:

>> Logged into the media server and checked for devices claimed in Device Manager. Found the devices.
>> Ran devbra -dev to determine the SCSI address.
>> Found N/A for drive

C:\>devbra -dev

Exch    HP:1x8 G2 AUTOLDR  Path: "Changer0:0:0:1"  SN: "AABBCCDD1E"
        Description: CLAIMED:HP StorageWorks 1x8 Cartridge Autoloader
        Revision: 4.20  Flags: 0x0016  Slots: 8  Drives: 1
        Drive(s) SN:
                "ABCDEFGHIJ"

Tape    HP:Ultrium 3-SCSI  Path: "Tape0:0:0:0"  SN: "N/A"
        Description: CLAIMED:HP LTO3 Drive
        Revision: Q51W  Device type: lto [13]  Flags: 0x0011

>> Checked if the drive is locked by DP

[root@cellsrv01:/root]
# omnimm -show_locked_devs | grep AUT01_D01

[root@ cellsrv01:/root]

>> Stopped DP Inet from Services.msc and Ran LTT.
>> Got error message that "mma.exe" process is accessing the device. Killed the mma.exe process from the Task Manager.

>> rescanned the devices in LTT. Able to detect both autoloader and it's drive this time.

>> Ran devbra -dev, which got the SCSI address.

C:\>devbra -dev

Exch    HP:1x8 G2 AUTOLDR  Path: "Changer0:7:0:1"  SN: "AABBCCDD1E "
        Description: CLAIMED:HP StorageWorks 1x8 Cartridge Autoloader
        Revision: 4.20  Flags: 0x0016  Slots: 8  Drives: 1
        Drive(s) SN:
                "ABCDEFGHIJ "

Tape    HP:Ultrium 3-SCSI  Path: "Tape1:7:0:0C"  SN: "ABCDEFGHIJ "
        Description: CLAIMED:HP LTO3 Drive
        Revision: Q51W  Device type: lto [13]  Flags: 0x0011

>> Ran the backup successfully.

Cause :

The DP media agent was accessing the drive and didn't let any other process to send commands. After the hung process is killed, the drive was accessible for normal operations.





Sunday, April 13, 2014

[61:12500] cannot connect to inet for getting file system list on host

Error:

[Critical]   From: BSM@cellsrv01.in.com "backup_spec_01” Time: 8/16/2014
[
61:12500] cannot connect to inet for getting file system list
              on host" winclnt01.in.com".
 
Reason:

Host may be down or Data protector client services (Inet) down.


Troubleshooting steps for Windows clients are as follows:

1. Check if the client (i.e. winclnt01.in.com in above error) is pingable from the cell manager, if not check the issue with platform team.

2. If the server is pingable, telnet the client using port 5555 (eg: #Telnet <Client name> 5555)

3. If the telnet failed, login to the windows client and check whether “Data protector Inet” service is running or not. If not start the “Data Protector Inet” service, Just by right clicking the service and restart it.

4. If the Data protector service is not listed in the “services.msc”, the DP agent might be removed from the client. Push the DP agents on this client from cell server.

5. If the telnet failed using port 5555 even data protector service running, check with platform team to open the port.

Troubleshooting steps for UNIX clients are as follows:

1. Follow the steps 1 and 2 as explained above.

2. If the telnet failed, login to the UNIX client and check for the Omni entries in /etc/services and /etc/inetd.conf using the commands below. If not found, DP client needs to be reinstalled.

[root@unxclnt01:/root]
# egrep omni /etc/services
omni  5555/tcp     # DATA-PROTECTOR

[root@unxclnt01:/root]
# egrep omni /etc/inetd.conf
omni stream tcp nowait root /opt/omni/lbin/inet inet -log /var/opt/omni//log/inet.log

3. Check if the port 5555 is listening using the command below. If not, raise a request to platform team to enable the port and start the inetd daemon.

[root@unxclnt01:/root]
# netstat -an | grep 5555
tcp        0      0  *.5555                 *.*                     LISTEN

4. Check if the ‘inetd’ daemon is running, if not start it using the below commands:

[root@unxclnt01:/root]
# ps -ef | grep inetd
    root  2144     1  0  Aug 10  ?         7:20 /usr/sbin/inetd

# /etc/inetd –s




Tuesday, April 1, 2014

Virtual Device Interface reported error: The object was not open.

Backup of SQL Database fails with the following errors:

[Critical] From: OB2BAR_Main@sqlsrv01.in.com "SIDDBS01"  Time: 3/9/2014 6:31:55 PM
 Error has occurred while executing a SQL statement.
 Error message: 'SQLSTATE:[08004] CODE:(911) MESSAGE:[Microsoft][ODBC SQL Server Driver][SQL Server]Database 'DBN01' does not exist. Make sure that the name is entered correctly.
SQLSTATE:[42000] CODE:(3013) MESSAGE:[Microsoft][ODBC SQL Server Driver][SQL Server]BACKUP DATABASE is terminating abnormally.'

[Critical] From: OB2BAR_Main@sqlsrv01.in.com "SIDDBS01"  Time: 3/9/2014 6:31:55 PM
 Virtual Device Interface reported error:
The object was not open.

 See also Data Protector debug.log and SQL Server error log for details.

[Normal] From: OB2BAR_DBN01@sqlsrv01.in.com "SIDDBS01"  Time: 3/9/2014 6:31:57 PM
 Completed OB2BAR Backup: sqlsrv01.in.com:/SIDDBS01/DBN01/0 "MSSQL"

[Major] From: OB2BAR_DBN01@sqlsrv01.in.com "SIDDBS01"  Time: 3/9/2014 6:31:57 PM

Aborting connection to BSM. Abort code -2.

Solution:

>> Remove the DB 'DBN01' from backup spec, if it doesn't exists in the SQL server.

[OR]

>> Check if the DB was dismounted, If so mount the database 'DBN01' and bring it online.

Trigger the backup and see if it's successful.