RE: Oracle ASM disk corruption

From: Hameed, Amir <Amir.Hameed_at_xerox.com>
Date: Mon, 27 Jul 2020 03:03:50 +0000
Message-ID: <DM6PR11MB348374186CCF78C430D29D22F4720_at_DM6PR11MB3483.namprd11.prod.outlook.com>

Hi Mladen!
Thank you for your input. I already tried that and got the following result.

---

SQL> ALTER DISKGROUP GRID
ADD DISK '/dev/oracleasm/grid/asmgrid01' NAME GRID_0002 /
ALTER DISKGROUP GRID
*
ERROR at line 1:

ORA-15032: not all alterations performed
ORA-15033: disk '/dev/oracleasm/grid/asmgrid01' belongs to diskgroup "GRID"
-----

I also opened an SR and the analyst suggested the following action: Closed and member status of the disk means that the disk is already dropped from asm. The only thing you can do at this point is to format that disk and then add it back to asm.

Since it is a block device, I was thinking that overwriting the device header would reinitialize it? (I am using UDEV and not using ASMLIB. The disk is not partitioned).

Thank you,
Amir
From: oracle-l-bounce_at_freelists.org <oracle-l-bounce_at_freelists.org> On Behalf Of Mladen Gogala Sent: Sunday, July 26, 2020 10:44 PM
To: oracle-l_at_freelists.org
Subject: Re: Oracle ASM disk corruption

Hi Amir!

The status of CLOSED means that the disk is not being used by the ASM instance:

https://docs.oracle.com/en/database/oracle/oracle-database/12.2/refrn/V-ASM_DISK.html#GUID-8E2E5721-6D4E-48C2-8DF3-A0EEBD439606

MOUNT_STATUS VARCHAR2(7) Per-instance status of the disk relative to group mounts:

• MISSING - Oracle ASM metadata indicates that the disk is known to be part of the Oracle ASM disk group but no disk in the storage system was found with the indicated name
• CLOSED - Disk is present in the storage system but is not being accessed by Oracle ASM
• OPENED - Disk is present in the storage system and is being accessed by Oracle ASM. This is the normal state for disks in a database instance which are part of a disk group being actively used by the instance.
• CACHED - Disk is present in the storage system and is part of a disk group being accessed by the Oracle ASM instance. This is the normal state for disks in an Oracle ASM instance which are part of a mounted disk group.
• IGNORED - Disk is present in the system but is ignored by Oracle ASM because of one of the following:
• The disk is detected by the system library but is ignored because an Oracle ASM library discovered the same disk
• Oracle ASM has determined that the membership claimed by the disk header is no longer valid
• CLOSING - Oracle ASM is in the process of closing this disk

So, the disk is there but it's not used by ASM. You can add it to one of your disk groups or leave it as a reserve for the rainy days, whatever suits you better. No action is necessary, this is no error condition.

Regards

On 7/26/20 10:09 PM, Hameed, Amir wrote: Hi,
I have an Oracle 12.1.0.2 Grid Infrastructure setup with three-nodes. There exist multiple ASM disk groups that are managed by this setup. One of the disk groups is called GRID and it hosts the OCR and voting disks. Recently I have noticed that one of the ASM disks in this group has MOUNT_STATUS='CLOSED" and HEADER_STATUS='MEMBER' as shown below:

The following data was captured from V$ASM_DISK but it is consistent on all nodes if queried from GV$ASM_DISK:

                                                 OS disk Space   Space              Disk
           Mount   Header       Mode    Disk     Size    Total   Free    ASM Disk   Failgroup                                 Vote
Grp# Disk# Status  Status       Status  State    (MB)    (MB)    (MB)    Name       Name       Disk path                      file
---- ----- ------- ------------ ------- -------- ------- ------- ------- ---------- ---------- ------------------------------ ----
   0     0 CLOSED  MEMBER       ONLINE  NORMAL    20,490       0       0                       /dev/oracleasm/grid/asmgrid01  Y
   2     0 CACHED  MEMBER       ONLINE  NORMAL    20,490  20,480   9,987 GRID_0000  GRID_0000  /dev/oracleasm/grid/asmgrid03  Y
   2     1 CACHED  MEMBER       ONLINE  NORMAL    20,490  20,480   9,987 GRID_0001  GRID_0001  /dev/oracleasm/grid/asmgrid02  Y

The disk that is not showing up is GRID_0002 and the block device name is /dev/oracleasm/grid/asmgrid01. The only change that has been made recently was that the OS on all three nodes was upgraded from RHEL6 to RHEL7. I have tried to drop this disk from the DG but that didn't work and I got the message that this disk is not part of the GRID DG.

What is the best way to resolve this issue? Should I overwrite the header of this device using dd so that it becomes a candidate disk? Any help will be appreciated.

Thank you,
Amir

--

Mladen Gogala

Database Consultant

Tel: (347) 321-1217

--
http://www.freelists.org/webpage/oracle-l

Received on Mon Jul 27 2020 - 05:03:50 CEST