Re: new database server build - sanity check

Having just completed a similar project, I would suggest if you're not bound to Solaris, go Linux instead. Otherwise, avoid Solaris 11 and stick to Solaris 10 if possible. From a sysadmin and user perspective, Solaris 11 is a completely new operating system that only superficially resembles earlier Solaris versions. I worked with it for the last 10 months, both as the sysadmin and DBA, and I honestly cannot recommend it.

Amongst the features that made Solaris 11 an unfortunate choice were:

• Solaris 11 local zones are integrated into the global operating system, much like a chroot jail and completely unlike a VM. This means that when the global zone is updated, all local zones are updated at the same time. This negates the possibility to apply security patches to the test zones before rolling them out into production.
• The new IPS package installation system is next to unusable, not to mention badly documented and counter-intuitive.
• All system management is done via completely new tools and stored in a Microsoft-like registry system, and badly documented. There is a lot to find out, mostly via the Solaris Express developers forums. Some (but not all) traditional configuration files are ignored, or even regenerated from the registry.
• The familiar "recommended patch sets" no longer exist, all patching must be done online from ( a downloaded copy of ) the Oracle support package repository which requires either a direct internet connection or a dedicated server to host a copy of the entire repository.
• Solaris 11 local zones are bare minimum installations and require a list of other packages to be installed before they are usable. By default no user interface is provided. Try installing gnome by hand.. :(
• Resource monitoring within the zone is confusing, to say the least, as zone capped resources (CPU, memory) and system wide resources (swap, load) are reported by tools in the zone. For instance, given total virtual memory of 160G, a capped memory allocation for the zone of 72G, monitoring inside the zone may show total memory as the capped 72G, but available virtual memory as 160G! Trying to allocate beyond 72G fails due to the resource cap. Similarly the load (runqueue), which even inside a zone is the global figure.

Only Oracle 11.2 from 11.2.0.3 upward is supported on Solaris 11.

If you must use Solaris 11 zones, be sure to use exclusive network interfaces. When using shared interfaces, the routing table of the global zone is pushed into the local zones, because the TCP stack is shared. This can lead to difficult to debug connection failures if the local zones cannot see all interfaces that the global zone can see. For instance, we had 6 NICs installed, two of which had gigabit network and were to be used by the production zones. The other NICs were connected to the same subnets, but via slower switches. The production zones would intermittently loose connectivity because the global zone would shift the default route to one of the NICs that were invisible to the production zones.

The solution is to create a VNIC on the physical NIC or aggregate that you want the zone to use and assign that VNIC as exclusive NIC to the zone. This works also for Solaris10 branded zones in a Solaris 11 global zone.

We decided against using ZFS for any data systems, as ZFS (even more than VxFS) is a first and foremost a volume manager, more so than a file system. It duplicates functionality found in the Oracle instance (e.g. block caching, predictive read-ahead) as well as the SAN (raid, snapshots, predictive read-ahaed, write-behind). It is also wasteful, as it needs ~25% of free disk space for internal workings, compared with only 1-5% for UFS. "Tuning" ZFS for Oracle database involves disabling as good as possible all volume managing and caching capabilities of ZFS, which makes implementing it rather a frustrating and ultimately futile exercise.

Instead, we used ASM for the database and FRA, and UFS for RMAN backup sets. UFS was chosen to avoid contention problems with RMAN's write sizes and the ZFS preferred block size.

For the root partition, you have no choice: Solaris 11 must be installed on a ZFS partition.

As always, your mileage may vary ;)

Cheers,
Tony

On 25/01/13 10:14, ~Jeff~ wrote:
> Hi all
> Your opinions please!
>
> We are embarking on a new database server build - the sysadmin is keen on
> Solaris 11.1 and ZFS on the T4-1 server. We will use zones for environment
> segregation and license mitigation.
>
> While I've used sol9 and 10 without absolutely no issues before, using such
> a new OS version with a relatively non-mainstream filesystem seems ...
> adventurous.
>
> Am I being too apprehensive, or is this a reasonable concern ? I'm
> interested in any real-life experience, rather than the marketing bullet
> points. Discuss. :)
>
> thanks in advance -
> Jeff
>
>
> --
> http://www.freelists.org/webpage/oracle-l
>
>
>

--
http://www.freelists.org/webpage/oracle-l