Re: dNFS vs NFS

I asked Dell/EMC about the kNFS nconnect parameter, since it was not mentioned in their documentation and the support rep was unaware of it. LOL    

> On 10/24/2023 6:15 AM MST Tim Gorman <tim.evdbt_at_gmail.com> wrote:
>
>
>
> Clay,
>
> Yes, typically there is a performance improvement, especially when high I/O throughput is desirable. Kernel-based NFS employs UNIX/Linux remote procedure calls (RPCs) locally on the NFS client, with the result that all I/O requests from the database go through this local subsystem on the NFS client including queuing. Oracle direct NFS has each Oracle server/background process connect directly to the NFS server, presumably increasing bandwidth.
>
> However, I advise folks to test comparatively (with SLOB at least) to verify any improvements. With the ability of kernel-based NFS to create more TCP connections using nconnect in RHEL8+/OEL8+, I am pretty sure dNFS is probably unnecessary in most cases. Again, test your application under load if possible, using SLOB to push high workloads if not.
>
> dNFS had some persistent bugs in the 19c timeframe for NFS v4.1, all the way to the 19.19 and 19.20 patchsets. Oracle created one-off patch 35450949 to be applied atop those patchsets. Otherwise, the workaround was to continue using NFS v3 with dNFS. We later learned through testing on RHEL8.6 that using kNFS with nconnect=8 yielded better performance than dNFS, so the customer dropped dNFS and went with nconnect.
>
> Hope this helps,
>
> -Tim
>
>
>
> On 10/23/2023 1:55 PM, Clay Jackson (cjackson) wrote:
>
> >
> > I’m pretty ignorant on Direct NFS. Is there significant gain over “plain ol’ NFS”?
> >
> >
> >
> > Thanks in advance!
> >
> >
> >
> >
> >
> > Clay Jackson
> >
> > Database Solutions Sales Engineer https://www.quest.com/solutions/database-performance-monitoring/
> >
> > clay.jackson_at_quest.com mailto:clay.jackson_at_quest.com
> >
> > office 949-754-1203 mobile 425-802-9603
> >
> >
> >
> > From: oracle-l-bounce_at_freelists.org mailto:oracle-l-bounce_at_freelists.org <oracle-l-bounce_at_freelists.org> mailto:oracle-l-bounce_at_freelists.org On Behalf Of DOUG KUSHNER
> > Sent: Monday, October 23, 2023 1:29 PM
> > To: tim.evdbt_at_gmail.com mailto:tim.evdbt_at_gmail.com; oracle-l_at_freelists.org mailto:oracle-l_at_freelists.org
> > Subject: Re: dNFS is kicking my ass
> >
> >
> >
> > CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.
> >
> >
> >
> > Using NFSv4.1. I have dumbed this down to just a single path to the storage for testing, but still no joy.
> >
> >
> >
> > ls -l $ORACLE_HOME/rdbms/lib/odm
> >
> > -rw-r--r-- 1 oracle oinstall 59600 Oct 22 20:42 libnfsodm19.so
> >
> >
> >
> > ls -l $ORACLE_HOME/bin/oradism
> >
> > -rwsr-x--- 1 root oinstall 147848 Apr 17 2019 /u01/app/oracle/product/19.0.0.0/dbhome_1/bin/oradism
> >
> >
> >
> > Each SP on the unity is independent, so have configured each as a server in oranfstab. Have temporarily commented out the alternative path, so the only path in this file is the same path that is in /etc/fstab.
> >
> >
> >
> > server: unity_sp_a
> > # oradata
> > local: 10.1.x.10 path: 10.1.x.40
> > #local: 10.1.x.210 path: 10.1.x.240
> > nfs_version nfsv4.1
> > export: /u02 mount: /u02
> > #
> > server: unity_sp_b
> > # fra
> > local: 10.1.x.10 path: 10.1.x.41
> > #local: 10.1.x.210 path: 10.1.x.241
> > nfs_version nfsv4.1
> > export: /flash_recovery_area mount: /u01/app/oracle/fast_recovery_area
> >
> >
> >
> > BTW, 10.1.x.10 & 41 are on a different vlan than 10.1.x.210 & 241. There are routing rules in place.
> >
> >
> >
> > Some curious lines from alert and trace files:
> >
> > Direct NFS: NFSERR 22 Invalid or unsupported argument. Server unity_sp_b path 10.1.7.41 (Did not get a similar error for unity_sp_a)
> >
> >
> >
> > skgnfs_get_sinfo_from_otab(): dir /u01/app/oracle/audit/DBNAME mount /u02
> >
> > skgnfs_get_sinfo_from_otab(): dir /u01/app/oracle/audit/DBNAME mount /u01/app/oracle/fast_recovery_area
> >
> > skgnfs_get_sinfo_from_otab: Looking in ?=/dbs/oranfstab for /u01/app/oracle/audit/DBNAME
> >
> > skgnfs_get_sinfo_from_otab(): server for dir /u01/app/oracle/audit/DBNAME not found
> >
> >
> >
> > During mount phase, why would dNFS be looking for a directory that is on a local file system?
> >
> > ls -ld /u01/app/oracle/audit/DBNAME
> > drwxr-x--- 2 oracle oinstall 4096 Oct 22 20:39 /u01/app/oracle/audit/DBNAME
> >
> >
> >
> >
> >
> > TIA for your assistance.
> >
> > Doug
> >
> >
> >
> > >
> > > On 10/23/2023 12:36 PM MST Tim Gorman <tim.evdbt_at_gmail.com mailto:tim.evdbt_at_gmail.com> wrote:
> > >
> > >
> > >
> > >
> > >
> > > Doug,
> > >
> > > What version of NFS? NFSv3? NFSv4.0? NFSv4.1? NFSv4.2? pNFS?
> > >
> > > If you're using other than NFSv3, can you share the "oranfstab" file you have configured, and in what directory it is located?
> > >
> > > Can you share output from the following commands...
> > >
> > > >
> > > > ls -l $ORACLE_HOME/rdbms/lib/odm
> > > > ls -l $ORACLE_HOME/bin/oradism
> > > >
> > >
> > >
> > > I'd ask for lines from your alert.log file, but perhaps that premature, plus that may not be something you want to share with the open internet.
> > >
> > > Hope this helps?
> > >
> > > Thanks!
> > >
> > > -Tim
> > >
> > >
> > >
> > > On 10/23/2023 12:30 PM, DOUG KUSHNER wrote:
> > >
> > > >
> > > > Most of my time is spent on Exadata, but have a need to set up 19.20 on OL8.8 with OFA on file systems mounted from a Dell Unity. We have followed Dell's Oracle Best Practices guide, Oracle's documentation and everything that I could Google on the subject.
> > > >
> > > >
> > > >
> > > > After 2 days of working with Oracle Support, their conclusion is that my setup is ok and are questioning the storage setup. There don't appear to be any known dNFS bugs that haven't been fixed in 19.20.
> > > >
> > > >
> > > >
> > > > There are 2 paths to the storage from the server and each path is mountable with kNFS, but not with dNFS. The control channel has not been assigned to a separate interface. The database was created with dbca over kNFS.
> > > >
> > > >
> > > >
> > > > mount points are /u02 for oradata and /u01/app/oracle/fast_recovery_area for the FRA.
> > > >
> > > >
> > > >
> > > > Can anyone share some troubleshooting tips? I set events in a pfile and attempted to mount the database, but the resulting debug info was inconclusive.
> > > >
> > > >
> > > >
> > > > event="10298 trace name context forever, level 1"
> > > > event="19392 trace name context forever, level 8"
> > > > event="19394 trace name context forever, level 8"
> > > > event="19396 trace name context forever, level 2"
> > > > event="19398 trace name context forever, level 128"
> > > >
> > > >
> > > >
> > > > Regards,
> > > >
> > > > Doug
> > > >
> > >
> >
>

--
http://www.freelists.org/webpage/oracle-l