If Your Database Fails, It must be Friday!

A couple of days ago I attempted to upgrade the GI on a standalone ASM server to 12.2 from 12.1.

During the rootupgrade.sh the process went off the rails. It seems that if your hostname starts with a number, you need to 'pre-patch' the software.

After following all the instructions for actions you need to take when the rootupgrade fails, I got the database back up. I then installed a 'golden' copy of the GI binaries - per instructions - and pre-patched the installation.

When I tried the installer yesterday, it told me that the NEW path was not a valid ORACLE_HOME. It turns out /etc/oracle/olr.loc and /etc/oracle/ocr.loc had been changed to point to the new home. So I saved them off and told them to point to the old home. The install then passed the initial check but when I hit the 'next' button, it just gave me a big red circle 'X' with no other details.

Looking at the logs however, it shows ORA-01017: invalid username/password; logon denied, which makes sense since the diskgroups are no longer showing up in the crs status.

I've restored the olr, ocr and inventory.xml from backups/saved copies and have tried relinking the binaries and performing a number of other checks and suggestions from both Mr. Google and Metalink, but no luck.

The disks are visible in /dev/oracleasm/disks/. It's just that they're no longer defined in the OLR? or something.

Anybody have any ideas? I probably can just reinstall the HA stack, but I'd really like to know why this isn't working.

This isn't super critical, it's our DR database and school's out (we're an education company) and I do have good restorable backups. Just can't figure out what suddenly went awry.

Thanks.