I have to confess: Time didn’t allow to rearrange my lab yet to have GI and ASM. Hence, this will be another blog post about Patching all my environments with the October 2020 Patch Bundles with only non-RAC patching. But it is in the works …
Photo by Olga Guryanova on Unsplash
As usual, an important annotation upfront: I patch in-place due to space issues. But in reality, you please patch always out-of-place with a separate home. Please see this blog post about how to apply the RU directly when you provision a new home with OUI.
Security Alert October 2020
You can find the Security Alert for the October 2020 listing all affected platforms and products. And of course, I look for Oracle Database Server, versions 11.2.0.4, 12.1.0.2, 12.2.0.1, 18c, 19c. From here you’ll be guided to the Risk Matrix.
It lists 28 new security patches for Oracle Database Products divided as follows:
- 18 new security patches for Oracle Database Products
- 1 new security patch for Oracle Big Data Graph
- 5 new security patches for Oracle REST Data Services
- 4 new security patches for Oracle TimesTen In-Memory Database
This is quite a similar number as in the July 2020 risk matrix.
I’m adding only a screenshot for CVEs with risk score > 6.0 here. Please access the Risk Matrix to check all of the listed fixes.
Risk Matrix – October 2020 – only Base Score’s > 6.0 shown here
But regardless of what’s included, you should apply the RUs anyway as there are a lot more fixes in it.
Database Patch Bundles
You will find the links to the database patches in MOS Note: 2694898.1 – Critical Patch Update (CPU) Program Oct 2020 Patch Availability Document (PAD). In my case, this is the list of bundles:
- Oracle Database 19c
- Database Release Update 19.9.0.0.201020 Patch 31771877 for UNIX (1.3GB on Linux)
- Readme
- List of fixes: MOS Note: 2523220.1 (does not include any 19.9.0 fixes yet unfortunately)
- Database Release Update 19.9.0.0.201020 Patch 31771877 for UNIX (1.3GB on Linux)
- Oracle Database 12.2.0.1
- Database Oct2020 Release Update 12.2.0.1.201020 Patch 31741641 for UNIX (780MB on Linux)
- Readme
- Database Oct2020 Release Update 12.2.0.1.201020 Patch 31741641 for UNIX (780MB on Linux)
- Oracle Database 11.2.0.4
- Database PSU 11.2.0.4.201020 Patch 31537677 for UNIX (374MB on Linux)
- Readme
- Database PSU 11.2.0.4.201020 Patch 31537677 for UNIX (374MB on Linux)
Unfortunately, as usual MOS Note: 2118136.2 – Assistant: Download Reference for Oracle Database/GI Update, Revision, PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases hasn’t been updated yet with the links for the October bundles by the time I write this.
Do I need a new OPatch?
First check while the download is progressing: Do I need to refresh my OPatch versions?
- 19.9.0 requires opatch 12.2.0.1.19 or later (I seriously doubt that as the July required 21 already – I guess this is one of the common copy&paste errors – I won’t verify this but rather use the opatch from the previous July cycle)
- 12.2.0.1 requires opatch 12.2.0.1.21 or later (same as in July)
- 11.2.0.4 requires opatch 11.2.0.3.23 or later (same as in July)
Actually, the 21 version for 12.2.0.1, 18c and 19c is the newest available version on MOS anyways. You can download OPatch via: 6880880. I could wipe out my current OPatch directories in all my three homes. Once the new OPatch bundles would have been unzipped into each home, I could proceed. But this is not needed here in my case.
Applying RU 19.9.0 to my 19c home
At first, I unzip the patch into a separate directory and place myself into this directory ~/31741641 .
- opatch conflict check
$ cd 31771877/ [CDB2] oracle@hol:~/31771877 $ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./ Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /u01/app/oracle/product/19 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/19/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.2.0.7.0 Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2020-10-21_11-40-19AM_1.log Invoking prereq "checkconflictagainstohwithdetail" Prereq "checkConflictAgainstOHWithDetail" passed. OPatch succeeded. [CDB2] oracle@hol:~/31771877
- opatch apply
$ $ORACLE_HOME/OPatch/opatch apply Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/oracle/product/19 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/19/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.2.0.7.0 Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2020-10-21_11-42-01AM_1.log Verifying environment and performing prerequisite checks... -------------------------------------------------------------------------------- Start OOP by Prereq process. Launch OOP... Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/oracle/product/19 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/19/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.2.0.7.0 Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2020-10-21_11-42-54AM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: 31771877 Do you want to proceed? [y|n] y User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/u01/app/oracle/product/19') Is the local system ready for patching? [y|n] y User Responded with: Y Backing up files... Applying interim patch '31771877' to OH '/u01/app/oracle/product/19' ApplySession: Optional component(s) [ oracle.network.gsm, 19.0.0.0.0 ] , [ oracle.rdbms.ic, 19.0.0.0.0 ] , [ oracle.rdbms.tg4db2, 19.0.0.0.0 ] , [ oracle.tfa, 19.0.0.0.0 ] , [ oracle.options.olap.awm, 19.0.0.0.0 ] , [ oracle.net.cman, 19.0.0.0.0 ] , [ oracle.xdk.companion, 19.0.0.0.0 ] , [ oracle.oraolap.mgmt, 19.0.0.0.0 ] , [ oracle.network.cman, 19.0.0.0.0 ] , [ oracle.options.olap, 19.0.0.0.0 ] , [ oracle.assistants.usm, 19.0.0.0.0 ] , [ oracle.assistants.asm, 19.0.0.0.0 ] , [ oracle.sqlj, 19.0.0.0.0 ] , [ oracle.jdk, 1.8.0.191.0 ] not present in the Oracle Home or a higher version is found. Patching component oracle.rdbms.rsf, 19.0.0.0.0... Patching component oracle.rdbms, 19.0.0.0.0... Patching component oracle.rdbms.util, 19.0.0.0.0... Patching component oracle.assistants.acf, 19.0.0.0.0... Patching component oracle.assistants.deconfig, 19.0.0.0.0... Patching component oracle.assistants.server, 19.0.0.0.0... Patching component oracle.buildtools.rsf, 19.0.0.0.0... Patching component oracle.ctx, 19.0.0.0.0... Patching component oracle.dbjava.ic, 19.0.0.0.0... ... Patching component oracle.dbdev, 19.0.0.0.0... Patching component oracle.rdbms.install.common, 19.0.0.0.0... Patching component oracle.sdo.locator, 19.0.0.0.0... Patching component oracle.duma, 19.0.0.0.0... Patching component oracle.sqlplus.ic, 19.0.0.0.0... Patching component oracle.xdk.rsf, 19.0.0.0.0... Patching component oracle.xdk.parser.java, 19.0.0.0.0... Patching component oracle.rdbms.rsf.ic, 19.0.0.0.0... Patching component oracle.precomp.common, 19.0.0.0.0... Patching component oracle.precomp.lang, 19.0.0.0.0... Patching component oracle.jdk, 1.8.0.201.0... Patch 31771877 successfully applied. Sub-set patch [31281355] has become inactive due to the application of a super-set patch [31771877]. Please refer to Doc ID 2161861.1 for any possible further required actions. Log file location: /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2020-10-21_11-42-54AM_1.log OPatch succeeded.
- datapatch
As final action, I will start my database and all pluggable databases again. The next step,datapatch, requires my databases and PDBs to be open.SQL*Plus: Release 19.0.0.0.0 - Production on Wed Oct 21 11:52:07 2020 Version 19.9.0.0.0 Copyright (c) 1982, 2020, Oracle. All rights reserved. Connected to an idle instance. SQL> startup ORACLE instance started. Total System Global Area 1577055360 bytes Fixed Size 9135232 bytes Variable Size 402653184 bytes Database Buffers 1157627904 bytes Redo Buffers 7639040 bytes Database mounted. Database opened. SQL> alter pluggable database all open; Pluggable database altered. SQL> exit Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.9.0.0.0 [CDB2] oracle@hol:~/31771877 $ $ORACLE_HOME/OPatch/datapatch -verbose SQL Patching tool version 19.9.0.0.0 Production on Wed Oct 21 11:52:34 2020 Copyright (c) 2012, 2020, Oracle. All rights reserved. Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_15467_2020_10_21_11_52_34/sqlpatch_invocation.log Connecting to database...OK Gathering database info...done Note: Datapatch will only apply or rollback SQL fixes for PDBs that are in an open state, no patches will be applied to closed PDBs. Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation (Doc ID 1585822.1) Bootstrapping registry and package to current versions...done Determining current state...done Current state of interim SQL patches: No interim patches found Current state of release update SQL patches: Binary registry: 19.9.0.0.0 Release_Update 200930183249: Installed PDB CDB$ROOT: Applied 19.8.0.0.0 Release_Update 200703031501 successfully on 15-JUL-20 10.29.43.549256 AM PDB PDB$SEED: Applied 19.8.0.0.0 Release_Update 200703031501 successfully on 15-JUL-20 10.29.44.668492 AM Adding patches to installation queue and performing prereq checks...done Installation queue: For the following PDBs: CDB$ROOT PDB$SEED No interim patches need to be rolled back Patch 31771877 (Database Release Update : 19.9.0.0.201020 (31771877)): Apply from 19.8.0.0.0 Release_Update 200703031501 to 19.9.0.0.0 Release_Update 200930183249 No interim patches need to be applied Installing patches... Patch installation complete. Total patches installed: 2 Validating logfiles...done Patch 31771877 apply (pdb CDB$ROOT): SUCCESS logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/31771877/23869227/31771877_apply_CDB2_CDBROOT_2020Oct21_11_55_46.log (no errors) Patch 31771877 apply (pdb PDB$SEED): SUCCESS logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/31771877/23869227/31771877_apply_CDB2_PDBSEED_2020Oct21_11_56_21.log (no errors) SQL Patching tool complete on Wed Oct 21 11:56:39 2020
Everything looks good.
Applying RU 12.2.0.1.201020 to my 12.2.0.1 home
I won’t copy/paste all steps as the output is similar to the above.
- opatch conflict check
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./
- opatch apply
$ORACLE_HOME/OPatch/opatch apply
Then I will need to startup my databases and all PDBs.
- datapatch -verbose
Final steps is “datapatch”. It requires the databases and all its PDBs to be up and running.$ORACLE_HOME/OPatch/datapatch -verbose
Applying PSU 11.2.0.4.201020 to my Oracle 11.2 home
For Oracle 11.2 I can only take the PSU. Bundle patches (BP) in Oracle 11.2 were only meant for Exadata systems. But the path to apply them is the same as for later bundles.
Again, I won’t copy/paste all steps as the output is similar to the above.
- opatch conflict check
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./
- opatch apply
$ORACLE_HOME/OPatch/opatch apply
Then I will need to startup my databases (
FTEXandUPGR). - catbundle.sql
Final steps is “catbudle.sql”. It requires the databases to be up and running.cd $ORACLE_HOME/rdbms/admin sqlplus / as sysdba @?/rdbms/admin/catbundle.sql psu apply
Everything fine now. No major issues seen. But as you know, my environment is simple and doesn’t include RAC or standbys.
Additional Tools and Cleanup
At this point, it is always time for me to check whether there are newer versions of AutoUpgrade, preupgrade.jar and SQL Developer available. Please note that of course I’m doing no upgrade here but still want the most recent versions of these (at least for my daily work) essential tools on disk.
- AutoUpgrade
As there is a new AutoUpgrade 19.9.2 available via MOS Note: 2485457.1 – AutoUpgrade Tool, I apply this right away to my home as well. And of course, if you have no intention to do any database upgrade on this host, you can safely skip this step.$ cp /media/sf_TEMP/Oct2020/autoupgrade1992/autoupgrade.jar $OH19/rdbms/admin $ java -jar autoupgrade.jar -version build.hash bf4ccd4 build.version 19.9.2 build.date 2020/08/31 13:47:51 build.max_target_version 19 build.supported_target_versions 12.2,18,19 build.type production
- opatch util cleanup
As somebody asked me the other day about whether it is ok to remove the directory where the patch has been unziped into? Yes, of course, this is ok. The information which is necessary to rollback the patch is kept in .patch_storage. And you can cleanup a little bit of it with “opatch util cleanup” – but be aware to be on a newer version of opatch.$ $ORACLE_HOME/OPatch/opatch util cleanup Oracle Interim Patch Installer version 12.2.0.1.21 Copyright (c) 2020, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/oracle/product/19 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/19/oraInst.loc OPatch version : 12.2.0.1.21 OUI version : 12.2.0.7.0 Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2020-10-21_12-02-44PM_1.log Invoking utility "cleanup" OPatch will clean up 'restore.sh,make.txt' files and 'scratch,backup' directories. You will be still able to rollback patches after this cleanup. Do you want to proceed? [y|n] y User Responded with: Y Backup area for restore has been cleaned up. For a complete list of files/directories deleted, Please refer log file. OPatch succeeded.
Don’t expect miracles please. And repeat this for all your homes.
- SQL Developer and SQLcliThis is the download location for both, SQL Developer and SQLcli for the new current 20.2 versions.
But my Patch Bundle isn’t there!
By the time I write this blog post, a lot of the bundles aren’t available yet. For instance, the 19.9.0 RU was available for Linux and AIX only on Oct 21, 2020. Please check out this section of MOS Note: 2694898.1.
Section 2.2 Post Release Patches – MOS Note: 2694898.1
You will quickly spot that – with some exceptions – almost all patch bundles may be available not earlier than 10 days after the actual quarterly patching date. Don’t shoot the messenger please!
And please be aware that the patch numbers are already there for all patch bundles. But for instance, if you try to download Microsoft Windows 32-Bit and x86-64 BP 19.9.0.0.201020 Patch 31719903, you will receive this:
This simply means “Patch is not there yet”. Of course, in an ideal world, MOS would tell you that the patch is not there yet and supposed to be available on day X.Y.Z. Well …
Further Links and Information
- Security Alert for the October 2020
- October 2020: Oracle Database Server, versions 11.2.0.4, 12.1.0.2, 12.2.0.1, 18c, 19c
- October 2020: Risk Matrix
- MOS Note: 2694898.1 – Critical Patch Update (CPU) Program Oct 2020 Patch Availability Document (PAD)
- MOS Note: 2523220.1 – List of Fixes included in 19c Bundle Patches
- Apply the RU directly when you provision a new home with OUI
- Patching all my environments with the July 2020 Patch Bundles
–Mike
Hi Mike,
There’s even version 19.9.2 for the autoupgrade tool in the attachments, which is with date 31.08.
Thanks for the hint – I’Ve had the right code example taken from the tool in my env (with 19.9.2) but the wrong text mentioning 19.9.1.
Classical copy/paste dummy error 🙁
Thanks for the hint!
Mike
In effect I was afraid about knowing that many OEM patches weren’t yet available yesterday, even if the vulnerability matrix are so horrible 🙁
Mike,
Take your time with the GI and RAC. No pressure. We’re just glad you exist! 🙂
Hi.
I download the latest OPatch and PreUpgrade.jar (for the hell of it) each quarter for each release I’m managing, even if they haven’t changed. That way I’ve got a consistent point in time for each quarterly patch, as I used it. What’s more, I don’t have to check for specific OPatch requirements, because I’ve always got the latest for that release.
Cheers
Tim…
Mike, I love your work! Makes my job so much easier. I’d like to suggest that for 19c, and probably 12.2, you include an additional step in between the opatch check conflict and the opatch apply steps.
I would add the check storage step, i.e,
opatch prereq CheckSystemSpace -phBaseFile where is the full directory path for a file that contains only the full directory path(s) for the patch(es) about to be applied, for example, the DB RU only, or both the DB RU and OJVM RU in the case of a combo patch.
Keep up the great work!
Thanks Ed – will do this the next time 🙂
Cheers,
Mike
Hi Mike,
We had applied 19.9 DB RU, OPatch completed success. But datapatch is failing with “ORA-00060: deadlock detected while waiting for resource” for Container Database.
Thanks,
Karthik
Please open an SR and upload all the logs – I can’t unfortunately analyze datapatch issues via the blog 🙁
Sorry for the inconvenience!
Cheers,
Mike
Hi Mike,
Thanks for the update.
We had raised SR for the same. But not sure whether it’s going in right direction.
Regards,
Karthik
Mike,
MOS has conflict checker tool to analyze for any existing issues with patching, like a one-off requires porting or a conflict. Question is: How do I use the conflict checker with QFSDP? The Oct 2020 QFSDP is Patch 31721191. If I select this patch, select Analyze with OPatch and then upload the lsinventory from GI_HOME or 19c DB_HOME or 12c DB_HOME, and check for conflicts, is that the right way to go? Does the conflict checker drill down into all the patches within QFSDP and select appropriate patches and check for conflict against the lsinventory output? Any insight will be helpful.
Thanks,
Arun Gupta
Hi Arun,
you please need to raise this question with Oracle Support. I really don’t know.
Cheers,
Mike
Hi Mike,
FYI (and I will open a SR)
if I install this patch on top of 19.7. RU I got:
Please rebuild the superset patch [31771877] to make sure it supersedes all the relevant patch(es) [30869156,31088341,31053402].
MOS Doc ID 2477824.1 advices to rollback 19.7 wich includes 30869156,31088341,31053402.
The download page of “Patch 30869156: DATABASE RELEASE UPDATE 19.7.0.0.0” says
“This patch was originally replaced by patch 31281355. The most recent replacement for this patch is 31771877.” 31771877 is the actual RU. But I can not apply this patch. It seems the actual RU doesn’t superseed 19.7.
Cheers Peter
Peter,
the “superseeds” message is pretty misleading on MOS.
It basically simply means: A newer patch from the same family tree is available.
A one-off gets always build on top of an RU. Hence, a fix for 19.7. can only be applied on-top of 19.7..
Cheers,
Mike
Hi Mike,
I was able to apply Oct 2020 RU to Oracle 19c (i.e. install first and then apply RU). Then, I ran into another note about installing and applying it in one shot with command and so I try the following in a new ORACLE_HOME
./runInstaller -applyRU patch/31771877/31771877
This gives me an error. I wonder if you’ve the same issue as me.
[/oracle/product/19/db_9] ./runInstaller -applyRU patch/31771877/31771877
Preparing the home to patch…
Applying the patch patch/31771877/31771877…
OPatch command failed while applying the patch. For details look at the logs from /oracle/product/19/db_9/cfgtoollogs/opatchauto/.
[Nov 4, 2020 4:56:43 PM] [INFO] The details are:
Exception occured : Invalid Home : /oracle/product/19/db_9
Summary of Conflict Analysis:
There are no patches that can be applied now.
Regards,
Jennifer
Hi Jennifer,
I blindly guess that the directory structure is different in your case. Where did you unzip it to?
Did you check this blog post:
https://mikedietrichde.com/2020/07/28/install-and-patch-in-one-single-action-with-oui/
I tried to point out the pitfalls.
Cheers,
Mike
Thanks for looking into this. I followed the instructions and unpacked it under the $ORACLE_HOME/patch directory.
Anyways , I used the traditional way (install then patch). Will try again with the next RU and hope that will work for me.
Hi Jennifer,
recently I stuck into the same situation. In my case oraInventory was corrupted (oraInst.loc was absent). I renamed oraInventory, recreated ORACLE_HOME etc. and installation was successful.
Dmitry
Hello Dmitry,
I have the same situation. Can you explain in detail what you did when you said “I renamed oraInventory, recreated ORACLE_HOME etc”?
Thank you in advance.
The latest DB RUR on October 20, 2020 is 19.8.1 and the latest OJVM is 19.9.0. Is there any compatibility issues between the 19.8.1 DB and 19.9.0 OJVM versions? I read the README for OJVM 19.9.0 and it just states “Oracle Database Release 19c”.
Can any 19.X DB RUR run with any 19.X OJVM patch release? Or is there a compatibility matrix somewhere that I’m not finding? Thanks!
You can combine them.
But my clear advice is to stay away from RURs.
https://mikedietrichde.com/2018/11/08/why-release-update-revisions-rur-are-tricky/
Cheers,
Mike
Hi Mike,
After apply PSU DATABASE PATCH 201020 the OEM13c Inventory and Usage Details does not show real database versions, remains with the old version 190416 and 181016 in other cases. The database version is 11.2.0.4 and the only one post-installation step was to run catbundle psu apply. In 12c version this is resolved by executing ./datapatch -verbose but this is not present in the OPatch version used. Please could you advise me in what would by missing here.
Regards,
Samir
Hi Samir,
check the patching tables in 11.2 (dba_registry_history and dba_registry_sqlpatch).
You may need to open an SR and check with Oracle Support please.
Cheers,
Mike
Mike,
I am just curious that if support for 12.1.0.2 is extended, then why do I see the following message when I try to download the 12.1.0.2 Oct 2020 patch:
You do not have privileges to download Software Extended Support patches.
The download link is also disabled. It is not an issue for me as we have Exadata and the QFSDP contains all the patches, but just curious about the meaning of this message.
Thanks,
Arun
Hi Arun,
this happens simply if you don’t have Extended Support.
If you have either Extended Support paid for, or you have an ULA or PULA license agreement (it may apply to Exadata systems as well), you need to please alert your Oracle contacts. They need to get in touch with the support owners – it is a simple flag which needs to be released in MOS as this currently prevents you from downloading if you are eligible for it.
Cheers,
Mike