Thursday, October 24, 2013

ASM-00001: Could not open device

While doing ASM installation on Windows System , we faced the following issues.

OS version : Windows 2008 R2
Grid version: 11.2.0.3

When  I was trying to run the "asmtoolg" command to stamp the ASM disks, we were getting the following error. Tried many things , including dropping and recreating the LUNS  , still we could not resolve it.




Solution :

Solution  of the problem is very simple. when you are invoking the asmtoolg.exe or setup.exe , just right click and "Run as Administrator". Thats it , issue resolved


Cause:
On windows 7 and Windows Server 2008 , is spite the user having Administrator Priviedges,
By default all the programs are run as " Non Admin" privilege and we need specifically "run as" administrator if needed.

Tuesday, October 22, 2013

One or more storage server has test stateless alerts with null "examinedby" fields

This error gets reported in exacheck for exadata storage nodes regularly.
Most of the error is just a test trap. We have to update the "examined by " to clean it.

Its Very simple !

Just login to exadata storage node as root or cell admin

Just use the command "alter alerthistory 64 examinedby="sameer";"
where 64 - is the alert number reported in exacheck
Sameer - is the name of the DBA addressing it

[root@storage01 ~]# cellcli
CellCLI: Release 11.2.3.2.1 - Production on Tue Oct 22 21:06:43 GMT+05:30 2013

Copyright (c) 2007, 2012, Oracle.  All rights reserved.
Cell Efficiency Ratio: 5,443

CellCLI> list alerthistory
         53_1    2013-07-17T02:01:37+05:30       info    "The disk controller battery is executing a learn cycle and may temporarily enter WriteThrough Caching mode as part of the learn cycle. Disk write throughput might be temporarily lower during this time. The flash drives are not affected. The battery learn cycle is a normal maintenance activity that occurs quarterly and runs for approximately 1 to 12 hours.  Note that many learn cycles do not require entering WriteThrough caching mode.  When the disk controller cache returns to the normal WriteBack caching mode, an additional informational alert will be sent.  Battery Serial Number : 1528  Battery Type          : iBBU08  Battery Temperature   : 45 C  Full Charge Capacity  : 1384 mAh  Relative Charge       : 96 %  Ambient Temperature   : 26 C"
         53_2    2013-07-17T08:04:57+05:30       clear   "All disk drives are in WriteBack caching mode.  Battery Serial Number : 1528  Battery Type          : iBBU08  Battery Temperature   : 49 C  Full Charge Capacity  : 1379 mAh  Relative Charge       : 50 %  Ambient Temperature   : 25 C"
         54      2013-07-29T15:52:26+05:30       info    "This is a test trap"
         55      2013-08-06T11:33:16+05:30       info    "This is a test trap"
         56      2013-08-06T12:52:30+05:30       info    "This is a test trap"
         57      2013-08-06T18:22:19+05:30       info    "This is a test trap"
         58      2013-08-22T18:41:53+05:30       info    "This is a test trap"
         59      2013-08-23T17:00:43+05:30       info    "This is a test trap"
         60      2013-08-27T15:11:24+05:30       info    "This is a test trap"
         61      2013-09-13T17:39:32+05:30       info    "This is a test trap"
         62      2013-09-17T23:50:01+05:30       info    "This is a test trap"
         63_1    2013-10-17T02:00:32+05:30       info    "The disk controller battery is executing a learn cycle and may temporarily enter WriteThrough Caching mode as part of the learn cycle. Disk write throughput might be temporarily lower during this time. The flash drives are not affected. The battery learn cycle is a normal maintenance activity that occurs quarterly and runs for approximately 1 to 12 hours.  Note that many learn cycles do not require entering WriteThrough caching mode.  When the disk controller cache returns to the normal WriteBack caching mode, an additional informational alert will be sent.  Battery Serial Number : 1528  Battery Type          : iBBU08  Battery Temperature   : 44 C  Full Charge Capacity  : 1376 mAh  Relative Charge       : 97 %  Ambient Temperature   : 29 C"
         63_2    2013-10-17T07:49:29+05:30       clear   "All disk drives are in WriteBack caching mode.  Battery Serial Number : 1528  Battery Type          : iBBU08  Battery Temperature   : 49 C  Full Charge Capacity  : 1367 mAh  Relative Charge       : 50 %  Ambient Temperature   : 28 C"
         64      2013-10-22T20:39:06+05:30       info    "This is a test trap"

CellCLI>; alter alerthistory 64 examinedby="sameer";
Alert 64 successfully altered



Thursday, October 17, 2013

Failed to create keys in the OLR, rc = 127, Message:

Grid Version : 11.2.0.3
OS : RHEL 6.3


While running root.sh , we got below error , after little bit of googling I found that we need more packages
Note:  there was no error reported regarding missing packages


[root@host dev]# /opt/app/11.2.0.3/grid/root.sh
Performing root user operation for Oracle 11g

The following environment variables are set as:
    ORACLE_OWNER= oracle
    ORACLE_HOME=  /opt/app/11.2.0.3/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:
   Copying dbhome to /usr/local/bin ...
   Copying oraenv to /usr/local/bin ...
   Copying coraenv to /usr/local/bin ...


Creating /etc/oratab file...
Entries will be added to the /etc/oratab file as needed by
Database Configuration Assistant when a database is created
Finished running generic part of root script.
Now product-specific root actions will be performed.
Using configuration parameter file: /opt/app/11.2.0.3/grid/crs/install/crsconfig_params
Creating trace directory
User ignored Prerequisites during installation
Failed to create keys in the OLR, rc = 127, Message:
  /opt/app/11.2.0.3/grid/bin/clscfg.bin: error while loading shared libraries: libcap.so.1: cannot open shared object file: No such file or directory

Failed to create keys in the OLR at /opt/app/11.2.0.3/grid/crs/install/crsconfig_lib.pm line 7497.
/opt/app/11.2.0.3/grid/perl/bin/perl -I/opt/app/11.2.0.3/grid/perl/lib -I/opt/app/11.2.0.3/grid/crs/install /opt/app/11.2.0.3/grid/crs/install/rootcrs.pl execution failed


What I found that we have these two packages installed 

[root@host dev]# yum list installed | grep -i libcap
Unable to read consumer identity
libcap.x86_64                      2.16-5.5.el6         @anaconda-RedHatEnterpriseLinux-201206132210.x86_64/6.3
libcap-ng.x86_64                   0.6.4-3.el6_0.1      @anaconda-RedHatEnterpriseLinux-201206132210.x86_64/6.3

I went ahead and installed these two packages and root.sh went fine.

[root@host dev]# yum install compat-libcap1.i686
[root@host dev]# yum install compat-libcap1.x86_64 -y



[root@host dev]# yum list installed | grep -i libcap
Unable to read consumer identity
compat-libcap1.i686                1.10-1               @nielsen-rhel-6.3-x86_64
compat-libcap1.x86_64              1.10-1               @nielsen-rhel-6.3-x86_64
libcap.x86_64                      2.16-5.5.el6         @anaconda-RedHatEnterpriseLinux-201206132210.x86_64/6.3
libcap-ng.x86_64                   0.6.4-3.el6_0.1      @anaconda-RedHatEnterpriseLinux-201206132210.x86_64/6.3

Friday, October 11, 2013

Reclaim DBFS space by Reorg of Secure File LOB

What we wanted to achieve:
We wanted reclaim the space , what DBFS is using in ASM. but we didn't want to deconfigure the DBFS , so we can use it any time in future.

Standard steps would be :
1.Delete all the files
2.Unmount the dbfs mount point (stop the crs resource)
3.Resize the datafile to minimum value .
4.Shut down the DBFS database

But on step 3 we hit the road block:

SQL> alter database datafile '+DATA_PROD/dbfs/datafile/dbfsts.532.817432735' resize 100g;
alter database datafile '+DATA_PROD/dbfs/datafile/dbfsts.532.817432735' resize 100g
*
ERROR at line 1:
ORA-03297: file contains used data beyond requested RESIZE value

Looking into the segments of the tablespace , I noticed

SEGMENT_NAME                                       SEGMENT_TYPE                         BYTES/1024/1024/1024
-------------------------------------------------- ------------------------------------ --------------------
T_FS2                                              TABLE                                          .000061035
IG_SFS$_FST_1731                                   INDEX                                          .000061035
SYS_IL0000140011C00007$$                           LOBINDEX                                       .000061035
IG_SFS$_FST_1823                                   INDEX                                          .000061035
SYS_IL0000140348C00007$$                           LOBINDEX                                       .000061035
LOB_SFS$_FST_1823                                  LOBSEGMENT                                     1230.59357
IP_SFS$_FST_1823                                   INDEX                                          .000061035
IPG_SFS$_FST_1823                                  INDEX                                          .000061035
SYS_IL0000148710C00007$$                           LOBINDEX                                       .074035645
SYS_LOB0000148710C00007$$                          LOBSEGMENT                                     87.8859863
T_DUMP                                             TABLE                                           .00012207
INTERIM                                            TABLE                                          .000183105
LOB_SFS$_FST_1731                                  LOBSEGMENT                                      .00012207
IP_SFS$_FST_1731                                   INDEX                                          .000061035
IPG_SFS$_FST_1731                                  INDEX                                          .000061035
SFS$_FSTP_1731                                     TABLE                                          .000061035
SYS_C0039855                                       INDEX                                          .000061035
T_FS3                                              TABLE                                          .000061035
IG_SFS$_FST_1777                                   INDEX                                          .000061035
SYS_IL0000140075C00007$$                           LOBINDEX                                       .000061035
LOB_SFS$_FST_1777                                  LOBSEGMENT                                      .00012207
IP_SFS$_FST_1777                                   INDEX                                          .000061035
IPG_SFS$_FST_1777                                  INDEX                                          .000061035
SFS$_FSTP_1777                                     TABLE                                          .000061035
SYS_C0039985                                       INDEX                                          .000061035

We see the LOB segments which are occupying the space in tablespace , We learned that we have to perform a reorg to reclaim the space. We used online dbms_redef to accomplish this.

This Note Id is very helpful -How to Shrink a SECUREFILE LOB Using Online Redefinition (DBMS_REDEFINITION)? (Doc ID 1394613.1)

Step 1.DETERMINE IF THE TABLE CAN BE REDEFINED ONLINE

SET SERVEROUTPUT ON
BEGIN
    DBMS_REDEFINITION.CAN_REDEF_TABLE('DBFS_USER','T_DUMP', DBMS_REDEFINITION.CONS_USE_ROWID);
END;
/

PL/SQL procedure successfully completed.

Step 2. CREATE THE INTERIM TABLE 


SQL> CREATE TABLE DBFS_USER.INTERIM AS SELECT * FROM DBFS_USER.T_DUMP WHERE 1=2;

Table created.

Step 3 : START THE REDEFINITION 
this step can take some time

SQL> BEGIN
   DBMS_REDEFINITION.START_REDEF_TABLE(
                uname => 'DBFS_USER',
                orig_table => 'T_DUMP',
                int_table => 'INTERIM',
                options_flag => DBMS_REDEFINITION.CONS_USE_ROWID);
END;
/  2    3    4    5    6    7    8

PL/SQL procedure successfully completed.



Step 4: FINISH THE REDEFINITION

SQL> exec DBMS_REDEFINITION.FINISH_REDEF_TABLE('DBFS_USER','T_DUMP','INTERIM');

PL/SQL procedure successfully completed.


Step 5 : DROP INTERIM TABLE (this is not there in the note ID , but its pretty much self explanatory)

SQL> drop table dbfs_user.INTERIM cascade constraints;

Table dropped.


Purge recyclebin if you have it enabled.


Thats it, go ahead and resize the file , 

SQL> alter database datafile '+DATA_PROD/dbfs/datafile/dbfsts.532.817432735' resize 10g;

Database altered.