Actually I really refused to blog about this for a few weeks now. But as I receive more questions and read more tweets, I think I need to shed some light and help avoiding confusion. And as readmes and MOS notes are not entirely in sync, you may get confused if you don’t read the “correct” document.
Oracle 19.3.0 Update (RU)
In April, Oracle released the 19.3.0 Release Update (RU) on Linux and Solaris only. As you can read in my blog post from April 25, 2019, this is a patch bundle meant for customers who installed 19.2.0 on Exadata or Solaris.
On June 5, 2019, a colleague from German ACS Support mailed me and asked if I know the reason why RU 19.3.0 has been withdrawn as he read in MOS Note: 2511334.1 (Oracle Database / Grid Infrastructure / OJVM Release Update and Release Update Revision R19 Apr 2019 Known Issues):
NOTE: A regression in DB RU 19.3.0 has been identified for updated Exadata on-premise systems. New installations of 19.3.0 are not involved.
The download of April 2019 patches that involve the DB RU 19.3.0 have been suspended.
Oracle Database Development recommends that all customers who have installed DB RU 19.3.0 or GI RU 19.3.0 instead install DB RUR 19.3.1 or GI RUR 19.3.1 respectively.
Well, I’ve had no idea. But I started to investigate.
Oracle 19.3.1 Revision (RUR)
First of all, I was a bit puzzled by the existence of 19.3.1 as an RUR. As I present about the release and patching model for quite a while now, I remember that there was no RUR planned for the initial on-prem release. In addition, an RUR should be released at quarterly intervals on the typical patch days. You can counter the first argument with “19.2.0 on Exadata was on-prem already”. And of course, it does not make sense to discuss this here. And maybe it makes more sense to release RURs as “emergency” patch bundles instead of having them on a quarterly schedule as well. But this gets decided on higher levels.
Nevertheless, my first step is to check the readme of the 19.3.1 revision:
The readme does not give you any indication about whether 19.3.1 is meant for you or not. It is a standard readme.
Only if you (hopefully) click on this link:
- Document 2511334.1 Oracle DB/GI/OJVM Update/Revision R19 Apr 2019 Known Issues
you’ll see the above text my colleague sent to me in early June. And here you will find more information:
1.1 DB RU 19.3.0 and GI RU 19.3.0 replaced with DB RUR 19.3.1 and GI RUR 19.3.1
A regression in DB RU 19.3.0 has been identified for updated Exadata on-premise systems. New installations of 19.3.0 are not involved. The download of April 2019 patches that involve the DB RU 19.3.0 have been suspended.
Oracle Database Development recommends all customers who have installed DB RU 19.3.0 or GI RU 19.3.0 to instead install DB RUR 19.3.1 Patch 29799057 or GI RUR 19.3.1 Patch 29800658, respectively. All 19.3.1 patches are available as of 4-June-2019. Please see the patch availability section of MOS Note 2498664.1.
If you ask what issue caused the withdrawal of RU 19.3.0, I can’t tell you as long as it is not mentioned in MOS Note: 2523220.1 (Database 19 Release Updates and Revisions Bugs Fixed Lists). But you won’t need 19.3.1 if you started with a standard 19.3.0 installation on any of the OS platforms. Only those you went from 19.2.0 to 19.3.0 RU will need to apply the 19.3.1 RUR. And this is also the reason why we didn’t release 19.3.1 RUR on HP-UX or AIX or zLinux. There was no 19.3.0 RU for any of these platforms. You can safely ignore the Revision 19.3.1.
And by the way, customers who downloaded RU 19.3.0 from MOS received an email alert to use now RUR 19.3.1 on top. Finally, if you are using still using 19.2.0 you can go straight to RUR 19.3.1. And once RU 19.4.0 is available, you can jump back on the RU train again.
- Oracle 19.3.0 for Linux is available for download
- Oracle 19.2.0 for Exadata is available for download
- MOS Note: 2511334.1 (Oracle Database / Grid Infrastructure / OJVM Release Update and Release Update Revision R19 Apr 2019 Known Issues)
- MOS Note: 2523220.1 (Database 19 Release Updates and Revisions Bugs Fixed Lists)
- Why Release Update Revisions (RUR) are tricky
- When you patch, use Updates, not Revisions
- MOS Note: 2498664.1 (Critical Patch Update (CPU) Program April 2019 Patch Availability Document (PAD))
- MOS Note: 2521164.1 (Database 19 Proactive Patch Information)
Funny that this RUR is not included into the “Download Assistant for RUs, RURs…”
Well, that is a different story.
First of all, logically you are right – and I made the same comment internally.
But second, the download assistant is meant for all on-prem customers and does not allow to downselect “Exadata only”.
Hence, including it here would mean that we would drive a lot of people with 19.3.0 NEW on-prem to apply the 19.3.1 RUR which they won’t need.
That’s why it is not there.
Hope this helps – cheers,
Hi Mike, I am in the process of testing upgrades from 220.127.116.11 to 19.3.0 using copies of production databases (using RMAN duplicate). Should I continue with 19.3.0 or install 19.3.1 and go with that? I did one upgrade test and it failed with errors about:
ORA-20001: Latest xml inventory is not loaded into table
ORA-29913: error in executing ODCIEXTTABLEFETCH callout
ORA-29400: data cartridge error
What I found on Oracle support was about Windows system but we are running RHEL7. So Should I go with 19.3.1?
at first – NO, you don’t need 19.3.1. It is meant only for Exadata customers who patched from 19.2.0 to 19.3.0.
All on-prem customers who started with 19.3.0 as a fresh/new install don’t need 19.3.1. It won’t harm, but it’s wasting patching time and effort.
Now at second, regarding the error, WHERE did you get this error?
In the upg_summary.log at the end of the upgrade?
My guess is that this has happened during the execution of datapatch during the upgrade.
Can you please check and let me know?
Check the logfiles please for things such as:
Bootstrapping registry and package to current versions...done
verify_queryable_inventory returned ORA-20001: Latest xml inventory is not loaded into table
Queryable inventory could not determine the current opatch status.
Execute 'select dbms_sqlpatch.verify_queryable_inventory from dual
MOS Note:1602089.1 – Queryable Patch Inventory – Issues/Solutions for ORA-20001: Latest xml inventory is not loaded into table
And point the support engineer to it as well.
Those ORA- errors that I mentioned are in the files sqlpatch_debug.log and sqlpatch_invocation.log.
I did another test (Test number 2) by deleting the whole instance and starting over with RMAN duplicate wiith the exact same backup. This time it worked. But I did something slightly different.
Test number 1: I installed 19c under …/product/18.104.22.168. When entering sqlplus I saw version 22.214.171.124 so I also created a symbolic link pointing 126.96.36.199 to 188.8.131.52 and set the ORACLE_HOME to 184.108.40.206. This is the upgrade that failed.
Test number 2: Did not touch anything except modified bash_profile to point ORACLE_HOME to 220.127.116.11
Do you think that the whole problem was caused by the symbolic link?
I will finish my upgrade (I am at the time zone upgrade) just to complete the whole process and do another test (number 3) the same way I did number 2 just to make sure that it wasn’t a fluke.
yes, I think this caused the issue. I remember that there were MOS notes about the usage of sym links causing trouble.
And I’m not sure but it could be that this is not supported (but just my bad memory).
Thank You, first lucid explanation of this issue for anyone on AIX.