Last week at the upgrade workshop in Budapest a customer had a interesting and – I believe – not uncommon question.
“How can I upgrade my Linux Operating System in my RAC environment without taking the entire cluster down?”
In this specific case the customer wanted to upgrade from RHEL 5.10 to RHEL6 or RHEL7.
Let’s assume it’s the typical 2-node-RAC where one wants to upgrade in a rolling fashion. And the data is stored within ASM.
- Drain Node 1 (i.e. take the workload off) – this will be the node getting upgraded first
- Remove Node 1 from the cluster (deleteNode procedure)
- Upgrade the OS (which is most likely a reimaging of the node). If the OS upgrade does not wipe out the entire server you can follow MOS Note:1559762.1 as it shows an OS upgrade from OL 6.2 to 6.4 which leave the Oracle installations intact)
- Add Node 1 back to the cluster (addNode procedure)
- Extend the Database Home to Node 1 using either cloning or addNode
- Make the database available on the newly added Node 1 and drain Node 2
- Repeat steps 2-6 for Node 2
- Ideally now you’ll perform a rolling upgrade of Oracle GI to Oracle Database 12c. Please apply the most recent PSU as well.
- Furthermore you may now look into upgrading your databases to Oracle Database 12.1.0.2 as well.
See the following documentation:
- Adding and Deleting Oracle RAC from Nodes on Linux and UNIX Systems
- Cloning Oracle RAC to Nodes in a New Cluster
- Using Cloning to Add Nodes to a Cluster
–Mike
Thank you for share your knowledge !
Thanks for sharing the info.
When we do addnode in step 4, "Node 1" will be of upgraded OS and so will be upgraded ASMLIB kernel module.
Will it be ok to run cluster with different nodes having different kernel module ?
Yes it will as far as I understand the procedure.
Cheers
Mike
we need to upgrade our kernel from 2.6.18-308.el5 to kernel-2.6.18-308.20.1.el5 upon a race condition in the direct I/O functions of the kernel where the value of a data structure changed unexpectedly, leading to a kernel crash.
IN case of RAC we need to relink or not ?
Regards
Maurizio
Maurizio,
no relink is necessary in this case.
Scroll down to the answer with the GREEN CHECKED icon – it lists all the cases where you will have to relink plus the correct source in MOS or Certify:
http://serverfault.com/questions/37557/when-to-relink-oracle
Cheers
Mike
thxs!
regards
Maurizio
in case of the mentioned kernel upgrade shloud we install the oracleasm-2.6.18-308.20.1.el5 ? in oracle site we can’t find any related oracleasm kernel drive. http://www.oracle.com/technetwork/server-storage/linux/downloads/rhel5-084877.html where says :
[oracleasm]
name=Oracle ASMLib Kernel Drivers for Red Hat Enterprise Linux Server 5
baseurl=http://yum.oracle.com/repo/OracleASM/RHEL5/$basearch/
but unfortunately http://yum.oracle.com/repo/OracleASM/RHEL5 doesn’t exist anymore …
Regards,
Maurizio
Maurizio,
sorry for the inconvenience but I fear you either have to open an SR or ask this in the Oracle Linux forum. I’m not an OL expert and can’t answer your question.
https://community.oracle.com/community/server_%26_storage_systems/linux/oracle_linux
Cheers
Mike
Hi,
Thanks for sharing this note. We are also trying something similar. We are migrating the RH5 to RH6 environment. On both the environment we installed oracleasm lib and disks are visible on both RH5 (as mounted) and RH6 (as unmounted) . During this migration we are also upgrading from 11gR2 GI to 12cR1 GI.
I did not find any good documentation on whether we can mount ASM diskgroup allocated to 11g GI to 12C GI. Just wanted to confirm our procedure or if you have any document can you please share.
1) Shutdown all services on 11g GI RH5 environment
2) Mount the disks to 12c GI RH6 environment. Verify if we notice any issue and take action on the same.
3) Once the disks are mounted, add database service in cluster and open the database.
So far my worry area is whether step 2 will be successful or not as I did not find any good document on the same
That step (2) should be successful – it won’t if you change the OS architecture but Linux-to-Linux will work.
Cheers
Mike
We are planning to upgrade from RHEL 6.8 to 6.9 on a 2-node RAC that is running Oracle database 12.1.0.2. We are able to accomodate a downtime for the database. In this scenario, can we simply shutdown the grid infra and database, let the UNIX admin perform the o/s upgrade, and then restart everything?? Do we need to relink oracle or any other executables? We are on netapp storage with cooked filesystems and no ASM. Thanks in advance!! BTW I have checked the certifications tab on MOS and I see that RHEL 6 (and even 7) is certified for Oracle 12.1.
Raj,
it is always wise to relink in such a case – but technically I doubt that it is really necessary when you jump from 6.8 to 6.9.
But the official advice would be: Yes.
Cheers
Mike
Raj,
RHEL 6.8 can be upgraded to 6.9 through yum update (I hope you have a working repo, if not then you can setup even locally).
Reallocate SCAN/VIP to node 2, shutdown CRS on node 1, perform yum update on node 1, restart sever.
DB and all services should come on node 1 automatically with server boot.
Hello Mike,
We are planning to upgrade RHL 5.2 to 7.4 for our Oracle 12 RAC servers. If we create RAC node three with RHL 7, do we need another RAC node license? After testing and completely cloning each and every component of the server, we shall shutdown node 2 and repeat the same for Node 1.
Thanks,
S K
S K,
I “THINK” the correct path would be getting in touch with your Oracle sales contact and ask for TEST licenses. This way you’d communicate (and receive an agreement) for a certain period. But I can’t give license recommendations or explanations here. So please, the best is to contact your Oracle contact in sales.
Cheers,
Mike
Hi.
We have a 2Node RAC with ASM on OL 5.11 and we have to upgrade OS to OL 7.8.We are planning to do below steps to reduce downtime. Please advise it is possible or not
1) Add 3rd and 4th Nodes with OL7.8 to the existing RAC
2) Remove old OL 5.11 1st and 2nd Nodes from the existing RAC.
Regards,
Salim
Which GI version are you using?
Are you going to upgrade GI as well?
Technically you can have such a mix for up to 24 hours in a supported fashion. But I fear that you are either using a very old database software or you may get in trouble with the OS version.
Cheers,
Mike
Hi.
I am completely stuck in oracle rac upgrade. It is possible to carry out an upgrade of oracle grid infrastructure (2 nodes) while one of the nodes are down ? I got it while both nodes are up and running. But I am trying to do it while one node up, and the other down. I mean I am trying a “non-rolling upgrade”
I can’t find any documentation related to non-rolling upgrade, for example, oracle rac release 12 to 18
Is out there any documentation available ?
Thank you
You please need to check with MOS or Oracle Support – or ping the RAC PM on Twitter (Anil Nair) – he may know.
Cheers, and sorry about the inconvenience 🙁
Mike
Hello Mike,
We are planning to update RHEL 6.5 to 6.10 on a 2-node RAC with Grid and Oracle database (on ASM) 11.2 versions.
In this scenario, can we simply shutdown the grid infra and database on first node, perform the OS update, relink oracle (1467060.1) and grid (1536057.1), startup grid and db, and do the same on the second node?
We tested this in a test environment.
Regards,
Alex
Hi Alex,
in many cases you don’t even have to shutdown. But yes, you are fully right. Just keep in the back of your head that 19c would require RHEL 7+ – and 11.2.0.4 is even out of paid extended support since 4 months now.
Cheers,
Mike
Hi Mike,
We need upgrade Solaris Sparc from 11.3 to 11.4. Our GI version is 11.2 and not upgrade at the same time.
Do we need relink the GI/RDBMS binaries? I saw this in some Oracle Note, but in the install/upgrade doc for Solaris didn’t see it.
Do we need delete/add node steps? What if we didn’t do this step?
Thanks,
Adam
Hi Adam,
I don’t think so – if the doc does not say anything, I guess this is not required.
I upgraded my Linux environments from 7.4 to 7.5 to … and I never relinked anything. It would be different if I’d go to Linux 8 since there are new libs etc.
Cheers,
Mike
Hi Mike,
my customer has two node RAC Cluster version 12.2 on OEL 7.3. We want to upgrade OEL from version 7.3 to 7.9. Is in this case enough to shutdown all Oracle services, upgrade OEL, relink all and then start all services (grid and database). Can we perform this OEL upgrade with yum update?
Thanks
Zoran
Hi Zoran,
going from OL7.3 to 7.9 is not an upgrade but a patch, and does not IMHO require a relink or so.
I patch in my env without any sort of downtime.
Only if you’d go from OL7 to OL8, this will require a different approach since this is an UPGRADE, but not a patch anymore.
Cheers
Mike