RE: Asynchronous I/O on Solaris

Hi All,

It seems that the issue of asynchronous I/O on Solaris is still alive and kicking. Here is an attempt to pull the threads together.

Oracle 7 had an 'async_write' parameter that used to default to TRUE. If set to FALSE, Oracle would use the standard 'write()' system call to do synchronous writes. So DBWR for example would have to wait for each physical I/O operation to complete before it could do anything else. So if DBWR needed to write 1 block to each of 10 disks, all those write operations would be done in series. The intent of the default use of asynchronous I/O was to allow multiple writes to distinct disks to be performed in parallel by a single process.

Up to Solaris 2.3, there was only a threads based implementation of asynchronous I/O. For every concurrent asynchronous I/O operation, the 'aiowrite()' system call would allocate or spawn a light-weight process to perform an ordinary synchronous write. This achieved I/O parallelism at the expense of additional CPU usage associated with thread creation and extra context switching overheads. Under moderate to heavy write loads it was actually more efficient to set
'async_write' to FALSE and to configure a modest number of DBWR slave processes
using the 'db_writers' parameter. The performance gain here was commonly of the order of 2% to 5%.

Solaris 2.4 introduced kernelized asynchronous I/O (KAIO) for raw devices. This is an alternative implementation of the 'aiowrite()' system call that avoids the unwanted performance costs of the threads based implementation. Because KAIO only works for raw devices, the 'aiowrite()' system call was changed to use KAIO on raw devices and to use the threads based implementation for file system I/O.

With the introduction of Quick I/O with Solaris 2.5.1, it became possible to use KAIO against file system based datafiles. Thus the 'aiowrite()' system call was changed to attempt to use KAIO even on file systems, and then to revert to threads based asynchronous I/O if that fails. This increased the overhead of asynchronous I/O even further, so that more sites needed to disable asynchronous I/O using Oracle parameters than before. From Oracle8 this is done using the
'disk_asynch_io' parameter, normally in conjunction with either the
'db_writer_processes' or 'dbwr_io_slaves' parameters.

The situation was confused somewhat by bug 4222164 in Solaris 2.6, that affected patchsets below 105181-19 and greatly exaggerated the performance cost of the threads based implementation of asynchronous I/O. However, the basic problem of the performance overhead of file system based asynchronous I/O remains unchanged despite the resolution of this bug.

Oracle have responded at 8.1 by avoiding the use of 'aiowrite()' for UFS based datafiles. That has confined the problem to VxFS based datafiles for which Quick I/O is not enabled -- still a large number of sites. A more general solution is promised with 8.1.7 in which Oracle will attempt to recognise whether KAIO is available for file system based datafiles, and decide dynamically whether to use
'aiowrite()' or not. There may also be a new '_filesystemio_options' parameter
to override the default intelligence if necessary.

@   Regards,
@   Steve Adams
@   http://www.ixora.com.au/