Re: dNFS is kicking my ass

Doug,

I agree with Mark, Oracle apparently did not bother to build much intelligence or flexibility into parsing the oranfstab file.  I'd remove comments and instead keep older versions of the files around renamed, instead.

The parsing of oranfstab is even so inflexible that accidentally messing up the ordering of parameter=value settings can screw everything up;  check out my blog post at "http://evdbt.com/oracle-dnfs-on-nfs-v4-1/".

Hope this helps?

Thanks!

-Tim

On 10/23/2023 5:30 PM, Mark Burgess wrote:
> Doug,
>
> a couple of things to try with the formatting of the oranfstab as follows:
>
> * Put the local and path parameters on seperate lines.
> * Add a colon after the nfs_version parameter
>
>
> See how that goes.
>
> server: unity_sp_a# oradatalocal: 10.1.x.10 path: 10.1.x.40#local:
> 10.1.x.210 path: 10.1.x.240nfs_version: NFSv4.1export: /u02 mount:
> /u02#server: unity_sp_b# fralocal: 10.1.x.10 path:
> 10.1.x.41#local: 10.1.x.210 path: 10.1.x.241nfs_version:
> NFSv4.1export: /flash_recovery_area mount:
> /u01/app/oracle/fast_recovery_area
>
>
> On 24 Oct 2023 at 7:29 AM +1100, DOUG KUSHNER <dougk5_at_cox.net>, wrote:
>> Using NFSv4.1.  I have dumbed this down to just a single path to the
>> storage for testing, but still no joy.
>> ls -l $ORACLE_HOME/rdbms/lib/odm
>> -rw-r--r-- 1 oracle oinstall 59600 Oct 22 20:42 libnfsodm19.so
>> ls -l $ORACLE_HOME/bin/oradism
>> -rwsr-x--- 1 root oinstall 147848 Apr 17  2019
>> /u01/app/oracle/product/19.0.0.0/dbhome_1/bin/oradism
>> Each SP on the unity is independent, so have configured each as a
>> server in oranfstab.  Have temporarily commented out the alternative
>> path, so the only path in this file is the same path that is in
>> /etc/fstab.
>> server: unity_sp_a
>> # oradata
>> local: 10.1.x.10  path: 10.1.x.40
>> #local: 10.1.x.210 path: 10.1.x.240
>> nfs_version nfsv4.1
>> export: /u02                    mount: /u02
>> #
>> server: unity_sp_b
>> # fra
>> local: 10.1.x.10  path: 10.1.x.41
>> #local: 10.1.x.210 path: 10.1.x.241
>> nfs_version nfsv4.1
>> export: /flash_recovery_area    mount: /u01/app/oracle/fast_recovery_area
>> BTW, 10.1.x.10 & 41 are on a different vlan than 10.1.x.210 & 241. 
>> There are routing rules in place.
>> Some curious lines from alert and trace files:
>> Direct NFS: NFSERR 22 Invalid or unsupported argument. Server
>> unity_sp_b path 10.1.7.41  (Did not get a similar error for unity_sp_a)
>> skgnfs_get_sinfo_from_otab(): dir /u01/app/oracle/audit/DBNAME mount /u02
>> skgnfs_get_sinfo_from_otab(): dir /u01/app/oracle/audit/DBNAME mount
>> /u01/app/oracle/fast_recovery_area
>> skgnfs_get_sinfo_from_otab: Looking in ?=/dbs/oranfstab for
>> /u01/app/oracle/audit/DBNAME
>> skgnfs_get_sinfo_from_otab(): server for dir
>> /u01/app/oracle/audit/DBNAME not found
>> During mount phase, why would dNFS be looking for a directory that is
>> on a local file system?
>> ls -ld /u01/app/oracle/audit/DBNAME
>> drwxr-x--- 2 oracle oinstall 4096 Oct 22 20:39
>> /u01/app/oracle/audit/DBNAME
>> TIA for your assistance.
>> Doug
>>> On 10/23/2023 12:36 PM MST Tim Gorman <tim.evdbt_at_gmail.com> wrote:
>>> Doug,
>>>
>>> What version of NFS?  NFSv3?  NFSv4.0?  NFSv4.1?  NFSv4.2? pNFS?
>>>
>>> If you're using other than NFSv3, can you share the "oranfstab" file
>>> you have configured, and in what directory it is located?
>>>
>>> Can you share output from the following commands...
>>>
>>> ls -l $ORACLE_HOME/rdbms/lib/odm
>>> ls -l $ORACLE_HOME/bin/oradism
>>>
>>>
>>> I'd ask for lines from your alert.log file, but perhaps that
>>> premature, plus that may not be something you want to share with the
>>> open internet.
>>>
>>> Hope this helps?
>>>
>>> Thanks!
>>>
>>> -Tim
>>>
>>>
>>>
>>> On 10/23/2023 12:30 PM, DOUG KUSHNER wrote:
>>>> Most  of my time is spent on Exadata, but have a need to set up
>>>> 19.20 on OL8.8 with OFA on file systems mounted from a Dell Unity. 
>>>> We have followed Dell's Oracle Best Practices guide, Oracle's
>>>> documentation and everything that I could Google on the subject.
>>>> After 2 days of working with Oracle Support, their conclusion is
>>>> that my setup is ok and are questioning the storage setup.  There
>>>> don't appear to be any known dNFS bugs that haven't been fixed in
>>>> 19.20.
>>>> There are 2 paths to the storage from the server and each path is
>>>> mountable with kNFS, but not with dNFS.  The control channel has
>>>> not been assigned to a separate interface.  The database was
>>>> created with dbca over kNFS.
>>>> mount points are /u02 for oradata and
>>>> /u01/app/oracle/fast_recovery_area for the FRA.
>>>> Can anyone share some troubleshooting tips?  I set events in a
>>>> pfile and attempted to mount the database, but the resulting debug
>>>> info was inconclusive.
>>>> event="10298 trace name context forever, level 1"
>>>> event="19392 trace name context forever, level 8"
>>>> event="19394 trace name context forever, level 8"
>>>> event="19396 trace name context forever, level 2"
>>>> event="19398 trace name context forever, level 128"
>>>> Regards,
>>>> Doug

--
http://www.freelists.org/webpage/oracle-l