A customer of mine hit an issue recently when upgrading to Oracle Database 12.2.0.1. They did everything correctly but received a ton of issues from the Data Guard Broker. A rule says: When you upgrade, disable the Data Guard Broker. But I can’t blame the customer as this “rule” is well hidden in the documentation.
When you upgrade, disable the Data Guard Broker
First of all, the Data Guard Broker is required if you would like to administer your databases in Oracle Cloud Control. Otherwise you can’t switchover or failover within Enterprise Manager. But once you approach a database upgrade please disable the Data Guard Broker while you do the upgrade. Enable the Broker after the upgrade again.
In an Oracle Data Guard configuration the Broker is always the boss. If you do something the Broker does not like – and you did it via SQL Plus and not via the Broker’s command line DGMGRL tool – the Broker will change your change. Bet with me.
From Oracle 12.1 High Availability Guide – can be found at: https://docs.oracle.com/database/121/DGBKR/configure.htm#DGBKR3659
Therefore you must disable the Data Guard Broker when you upgrade your primary database. Otherwise the Broker will interfere with the upgrade and cause trouble.
Where is this documented?
The crucial point is that you most likely won’t be able to find this piece of information in the documentation unless you studied the Data Guard’s Broker manual carefully. When you read through the steps to upgrade a database in a physical standby environment you were not able to find a hint.
Thanks to my colleagues from the MAA / Data Guard Team we updated the documentation.
- If you are using the Oracle Data Guard broker to manage your configuration, follow the instructions in the Oracle Data Guard Broker manual for information about removing or disabling the broker configuration.
It will guide you to this section explaining clearly how to disable (and enable afterwards) the Data Guard Broker while you upgrade your primary database.
How to upgrade your database with a physical standby in place?
I won’t replicate the documentation as the Data Guard documentation is usually very good and precise. But it is important that you don’t skip this part:
Then you proceed with:
In brief a very short summary:
- Disable the Data Guard Broker configuration
- Install the new software on the standby host and patch it with the newest RU
- Install the new software on the primary host as well and patch it with the newest RU
- Download the most recent
preupgrade.jar(for instance the September 2017 one) and execute it on your primary - Shutdown everything on the standby and restart listeners etc with the new environment
- Mount the physical standby and start redo apply
- Upgrade the primary database as explained in the Database Upgrade Guide
- After the upgrade has been completed the standby has been upgraded via log apply as well
- Enable the Data Guard Broker configuration
An additional comment
I received this comment via LinkedIn and it’s worth to mention:
–Mike
Having performed many such upgrades, I have found the safest method is to defer log shipping from the Primary to stop alerts and keep the Standby shut down until after the Primary is successfully upgraded. Only after upgrade mount the Standby and enable configuration (enable shipping and start redo apply). That way a failed Primary upgrade would not be replicated to the Standby.