Re: A beginners guide to implementing ASM

>

I strongly recommend you look at ASM Filter Driver, rather than ASMLib
>

W.r.t. AFD, I actually hit an issue a while ago when AFD did not see the modified EBS volume size. I increased the EBS volumes, and the new size was visible fine from the OS (parted/fdisk). Had to reload the AFD driver to see the new size with downtime (it was a test system). afd_scan/afd_refresh(-all) did not help too. There was an SR open with Oracle support.
I was on 19.6 at that time. There were several similar AFD issues across multiple versions, so it seems to be a recurring issue.

Another AFD issue that I hit happened with NVMe EBS volumes on Nitro instances:
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/device_naming.html

> * The device names that you specify for NVMe EBS volumes in a block device
> mapping are renamed using NVMe device names (/dev/nvme[0-26]n1). The
> block device driver can assign NVMe device names in a different order than
> you specified for the volumes in the block device mapping.
>

Basically, AFD did not handle that different order gracefully. The AFD disks were missing randomly on reboot: Example: two devices with the same AFD label after OS startup: the TEST labels should go from TEST01 to TEST10 (10 disks)

> [root_at_oracle-sandbox ~]# blkid /dev/nvme?n1p1
> /dev/nvme0n1p1: LABEL="TEST04" TYPE="oracleasm" PARTLABEL="primary"
> PARTUUID="91f987f4-9622-4bf7-a6cd-f20e81759a0c"
> /dev/nvme1n1p1: LABEL="TEST06" TYPE="oracleasm" PARTLABEL="primary"
> PARTUUID="d5101c67-7a5b-4e3d-89aa-41c1cebe3ea1"
> /dev/nvme2n1p1: LABEL="TEST03" TYPE="oracleasm" PARTLABEL="primary"
> PARTUUID="bf2e20e7-bb46-45f8-bb8a-8f0e192fe517"
> /dev/nvme3n1p1: LABEL="TEST08" TYPE="oracleasm" PARTLABEL="primary"
> PARTUUID="584f374c-157d-468c-b6d6-7a7b5687d524"
> /dev/nvme4n1p1: LABEL="TEST01" TYPE="oracleasm" PARTLABEL="primary"
> PARTUUID="0798e70c-550d-49eb-8a1d-67e521ad8288"
> /dev/nvme5n1p1: LABEL="TEST05" TYPE="oracleasm" PARTLABEL="primary"
> PARTUUID="5fedf86c-ab39-421d-91e6-2b6cee01fad3"
> /dev/nvme6n1p1: LABEL="TEST08" TYPE="oracleasm" PARTLABEL="primary"
> PARTUUID="98879ff5-46c0-4231-ae52-b22e8ec16656"
> /dev/nvme7n1p1: LABEL="TEST09" TYPE="oracleasm" PARTLABEL="primary"
> PARTUUID="ea42671c-41f9-4af2-8562-1bb7201ce67f"
> /dev/nvme8n1p1: LABEL="TEST02" TYPE="oracleasm" PARTLABEL="primary"
> PARTUUID="a19e8a16-892b-414c-a5d8-e9e159bb0b5a"
> /dev/nvme9n1p1: LABEL="TEST07" TYPE="oracleasm" PARTLABEL="primary"
> PARTUUID="3880d67a-ee2d-444f-b1a4-8aa0375128c9"
>
> Missing disk TEST10 in this example (there were several disks missing
sometimes):

> [root_at_oracle-sandbox ~]# asmcmd afd_lsdsk
>
> --------------------------------------------------------------------------------
> Label Filtering Path
>
> ================================================================================
> DATA01 ENABLED /dev/nvme11n1p1
> TEST01 ENABLED /dev/nvme4n1p1
> TEST02 ENABLED /dev/nvme8n1p1
> TEST03 ENABLED /dev/nvme2n1p1
> TEST04 ENABLED /dev/nvme0n1p1
> TEST05 ENABLED /dev/nvme5n1p1
> TEST06 ENABLED /dev/nvme1n1p1
> TEST07 ENABLED /dev/nvme9n1p1
> TEST08 ENABLED /dev/nvme6n1p1
> TEST09 ENABLED /dev/nvme7n1p1
>

No TEST05 in afd_lslbl (the commands run one after another):

> [root_at_oracle-sandbox ~]# asmcmd afd_lslbl
>
> --------------------------------------------------------------------------------
> Label Duplicate Path
>
> ================================================================================
> DATA01 /dev/nvme11n1p1
> TEST01 /dev/nvme4n1p1
> TEST02 /dev/nvme8n1p1
> TEST03 /dev/nvme2n1p1
> TEST04 /dev/nvme0n1p1
> TEST06 /dev/nvme1n1p1
> TEST07 /dev/nvme9n1p1
> TEST08 /dev/nvme6n1p1
> TEST09 /dev/nvme7n1p1
> TEST10 /dev/nvme3n1p1
>

I had to run afd_scan manually each time, or create a systemd service for that. Oracle support sent me to AWS support. I actually hoped that Oracle would be interested in getting AFD work properly on AWS Nitro, but to no avail. AWS support came up with UDEV rules setting fixed names for EBS volumes, and AFD devices on top of them. That turned out to be pretty stable.
I did not experience the same issue with ASMLib in this environment.

On Fri, 26 Feb 2021 at 22:18, Mark J. Bobak <mark_at_bobak.net> wrote:

> I agree with Michael.
>
> Also, it's not a required component, but I strongly recommend you look at
> ASM Filter Driver, rather than ASMLib or Linux native methods for device
> labeling. (Particlarly since you're on 19c.)
>
> -Mark
>
> On Fri, Feb 26, 2021 at 9:46 AM Michael McMullen <ganstadba_at_hotmail.com>
> wrote:
>
>> You wouldn't extend the disk, you would add a new block device of the
>> same size and then present it to asm. Create a disk for FRA and one for
>> DATA. Use external redundancy as your SAN would take care of that piece.
>> I don' t have much more information for you. We did follow this document
>> when setting up back in the day. I don't know if it's relevant anymore.
>>
>>
>> https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/solutions/vmware-oracle-databases-on-vmware-best-practices-guide.pdf
>>
>>
>> Oracle Databases on VMware Best Practices Guide
>> <https://www.vmware.com/content/dam/digitalmarketing/vmware/en/pdf/solutions/vmware-oracle-databases-on-vmware-best-practices-guide.pdf>
>> © 2016 VMware, Inc. All rights reserved. Page 1 of 81 VMware Hybrid Cloud
>> Best Practices Guide for Oracle Workloads Version 1.0 May 2016
>> www.vmware.com
>>
>> ------------------------------
>> *From:* oracle-l-bounce_at_freelists.org <oracle-l-bounce_at_freelists.org> on
>> behalf of Steve Wales (AddOns) <steve.wales_at_addonsinc.com>
>> *Sent:* February 24, 2021 5:51 PM
>> *To:* oracle-l_at_freelists.org <oracle-l_at_freelists.org>
>> *Subject:* A beginners guide to implementing ASM
>>
>>
>> Setup:
>>
>>
>>
>> Linux 7.8
>>
>> Oracle 19 (mostly, still got a couple I’m in the process of upgrading
>> from 18)
>>
>>
>>
>> The environments I work in are primarily all Linux on Oracle VM’s.
>>
>>
>>
>> Our config is a stack or Oracle Hosts, running virtual machines through
>> Oracle VM Manager (OVM).
>>
>> The storage appliance is a ZFS Appliance.
>>
>>
>>
>> Currently all of my databases use Direct NFS for database data file
>> access.
>>
>> When building a new machine the template is set up with a couple of
>> virtual disks assigned to the VM that were defined to Linux by using ssm
>> with something along the way of:
>>
>>
>>
>> ssm create -s 59G -n lv_u01 –fstype xfs -p vg_u01 /dev/xvdb /u01
>>
>>
>>
>> Then we create shares using the ZFS appliance console, and mount them to
>> the server and manage those via dNFS.
>>
>>
>>
>> I want to start looking into ASM. I’d like to run some performance
>> tests on ASM vs dNFS.
>>
>>
>>
>> I’ve started reading the ASM documentation and notes that I’ve been able
>> to find on the Oracle Support site – and my first step appears to be to try
>> to create disk presented to the VM and then configure it on Linux as a raw
>> disk partition. Doc Id 452924.1 offers up a few examples of disk type to
>> use and Raw disk partition is #1 on that list.
>>
>>
>>
>> I guess related to that for down the road type stuff, how easy is it to
>> extend a raw partition if you need to grow it down the road ?
>>
>>
>>
>> Found an article on fedoraproject.org using parted, but don’t want to
>> muck around too much without knowing what top expect, so I’m hoping to get
>> some pointers from here.
>>
>>
>>
>> I am not a Linux Sysadmin (I know enough to be dangerous) and I am not a
>> storage engineer. Our previous Linux guy has just resigned and we’re still
>> interviewing replacements.
>>
>>
>>
>> So, anyone able to offer me something from their experience that might
>> shed some light here. I don’t have a lot of spare time at the moment for a
>> lot of the learn by playing around stuff so was hoping that someone might
>> have some real life tales to share or pointers to links for my reading
>> (since my Google-fu has found a bunch of stuff for mucking around in
>> VirtualBox but that’s not what I’m using here.
>>
>>
>>
>> Thanks
>>
>> Steve
>>
>>
>>
>> *Disclaimer*
>>
>> The information contained in this communication from the sender is
>> confidential. It is intended solely for use by the recipient and others
>> authorized to receive it. If you are not the recipient, you are hereby
>> notified that any disclosure, copying, distribution or taking action in
>> relation of the contents of this information is strictly prohibited and
>> may be unlawful.
>>
>> This email has been scanned for viruses and malware, and may have been
>> automatically archived.
>>
>

--
http://www.freelists.org/webpage/oracle-l