It’s patching time. When I checked an SR and had a quick discussion with a customer yesterday, Peter reminded me that it’s patching day. And at about 20:00h CET the new patch bundles appeared on MOS. So let me show you again Patching all my environments with the January 2022 Patch Bundles for 11.2.0.4, 12.2.0.1, 19c and 21c.

Photo by Mehmet Turgut Kirkgoz on Unsplash
As usual, an important annotation upfront: I patch in-place due to space issues. But in reality, you please patch always out-of-place with a separate home. Please see this blog post about how to apply the RU directly when you provision a new home with OUI.
Security Alert January 2022
You will find the January 2022 Security Alerts for all products here. And as usual, please pay close attention to the Database Server Products Risk Matrix. You will find the usual suspects but this time not many fixes and relatively low risk scores. Still, my advice is be to apply the patch bundles to your environments if you haven’t done so already.
Regarding the log4j topic, please find further information in the alert.
You may realize that neither 11.2.0.4 nor 18c are mentioned in the risk matrix anymore. As both are out of Premier or Extended Support, there are no regular patch bundles anymore. But the issues noted in the risk matrix may happen in both these releases, and potentially in older releases as well.
Database Patch Bundles
You will find the links to the individual patch bundles in MOS Note: 2817011.1 – Critical Patch Update (CPU) Program Jan 2022 Patch Availability Document (PAD). But you can use as well MOS Note: 2118136.2 – Assistant: Download Reference for Oracle Database/GI Update, Revision, PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases to download “your” bundle.
- Oracle Database 21c
- Database Release Update 21.5.0.0.220118 Patch 33516412 for UNIX
- README
- List of fixes: MOS Note: 2523220.1
- New fixes: 343
- Database Release Update 21.5.0.0.220118 Patch 33516412 for UNIX
- Oracle Database 19c
- Database Release Update 19.14.0.0.220118 Patch 33515361 for UNIX
- README
- List of fixes: MOS Note: 2523220.1
- New fixes: 502
- OJVM Release Update 19.14.0.0.220118 Patch 33561310 for all platforms
- README
- New fixes: 3
- Database Release Update 19.14.0.0.220118 Patch 33515361 for UNIX
- Oracle Database 12.2.0.1
- Database Jan2022 Release Update 12.2.0.1.220118 Patch 33587128 for UNIX
- README
- List of fixes: MOS Note: 2245178.1
- New fixes: 24
- Database Jan2022 Release Update 12.2.0.1.220118 Patch 33587128 for UNIX
- Oracle Database 11.2.0.4
- Database Patch Set Update 11.2.0.4.220118 Patch 33477185 for UNIX
- README
- List of fixes: MOS Note:1611785.1
- Database Patch Set Update 11.2.0.4.220118 Patch 33477185 for UNIX
Do I need a new OPatch?
And then it is time again for the opatch check. Without even seeing the number, I can tell you that you MUST refresh your opatch since there is a significant performance improvement in the 28 version on all platforms. But let me double check with the READMEs:
- 21.5.0 requires opatch 12.2.0.1.28 or later
- 19.14.0 requires opatch 12.2.0.1.28 or later
- 12.2.0.1 January 2022 requires opatch 12.2.0.1.28 or later
- 11.2.0.4 January 2022 requires opatch 11.2.0.3.32 or later
Hence, I will update opatch in all my homes. The 6880880 link from the readmes takes you directly to the correct download. And it may not wonder you, all three opatch releases for 12.2 – 21c are identical even though all sail with different display labels on MOS. So it is fairly enough in my case to download it only once.
Wipe out your current OPatch
directory in your homes. Once you unzip the new OPatch bundles. you can proceed.
Applying RU 21.5.0 to my 21c home
I always do something you shouldn’t do. I apply the patch to my existing home. This has to do with the available space in my lab environment. You please provide a new home and apply the patch(es) right away as I describe here.
Patching my 21c database home to RU 21.5.0 is pretty straight forward.
- Conflict and space checks
$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./ Oracle Interim Patch Installer version 12.2.0.1.28 Copyright (c) 2022, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /u01/app/oracle/product/21 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/21/oraInst.loc OPatch version : 12.2.0.1.28 OUI version : 12.2.0.9.0 Log file location : /u01/app/oracle/product/21/cfgtoollogs/opatch/opatch2022-01-19_13-32-26PM_1.log Invoking prereq "checkconflictagainstohwithdetail" Prereq "checkConflictAgainstOHWithDetail" passed. OPatch succeeded. $ $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -ph ./ Oracle Interim Patch Installer version 12.2.0.1.28 Copyright (c) 2022, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /u01/app/oracle/product/21 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/21/oraInst.loc OPatch version : 12.2.0.1.28 OUI version : 12.2.0.9.0 Log file location : /u01/app/oracle/product/21/cfgtoollogs/opatch/opatch2022-01-19_13-33-25PM_1.log Invoking prereq "checksystemspace" Prereq "checkSystemSpace" passed. OPatch succeeded.
All set – we are good to go.
- opatch apply
$ $ORACLE_HOME/OPatch/opatch apply Oracle Interim Patch Installer version 12.2.0.1.28 Copyright (c) 2022, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/oracle/product/21 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/21/oraInst.loc OPatch version : 12.2.0.1.28 OUI version : 12.2.0.9.0 Log file location : /u01/app/oracle/product/21/cfgtoollogs/opatch/opatch2022-01-19_14-17-44PM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: 33516412 Do you want to proceed? [y|n] y User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/u01/app/oracle/product/21') Is the local system ready for patching? [y|n] y User Responded with: Y Backing up files... Applying interim patch '33516412' to OH '/u01/app/oracle/product/21' ApplySession: Optional component(s) [ oracle.network.gsm, 21.0.0.0.0 ] , [ oracle.tfa, 21.0.0.0.0 ] , [ oracle.net.cman, 21.0.0.0.0 ] , [ oracle.ons.cclient, 21.0.0.0.0 ] , [ oracle.rdbms.ic, 21.0.0.0.0 ] , [ oracle.sdo.companion, 21.0.0.0.0 ] , [ oracle.duma, 21.0.0.0.0 ] , [ oracle.network.cman, 21.0.0.0.0 ] , [ oracle.ons.eons.bwcompat, 21.0.0.0.0 ] , [ oracle.jdk, 1.8.0.271.00 ] not present in the Oracle Home or a higher version is found. Patching component oracle.rdbms.rsf, 21.0.0.0.0... Patching component oracle.network.rsf, 21.0.0.0.0... Patching component oracle.rdbms, 21.0.0.0.0... Patching component oracle.dbjava.ic, 21.0.0.0.0... Patching component oracle.dbjava.jdbc, 21.0.0.0.0... Patching component oracle.dbjava.ucp, 21.0.0.0.0... Patching component oracle.has.common.cvu, 21.0.0.0.0... Patching component oracle.ldap.owm, 21.0.0.0.0... Patching component oracle.ldap.rsf, 21.0.0.0.0... Patching component oracle.ldap.rsf.ic, 21.0.0.0.0... Patching component oracle.ldap.security.osdt, 21.0.0.0.0... Patching component oracle.nlsrtl.rsf, 21.0.0.0.0... Patching component oracle.oraml.server, 21.0.0.0.0... Patching component oracle.rdbms.dbscripts, 21.0.0.0.0... Patching component oracle.rdbms.deconfig, 21.0.0.0.0... Patching component oracle.rdbms.rman, 21.0.0.0.0... Patching component oracle.rdbms.rsf.ic, 21.0.0.0.0... Patching component oracle.sqlplus, 21.0.0.0.0... Patching component oracle.tfa.db, 21.0.0.0.0... Patching component oracle.ons, 21.0.0.0.0... Patching component oracle.ctx.rsf, 21.0.0.0.0... Patching component oracle.sqlplus.ic, 21.0.0.0.0... Patching component oracle.sdo.locator, 21.0.0.0.0... Patching component oracle.assistants.server, 21.0.0.0.0... Patching component oracle.sdo.locator.jrf, 21.0.0.0.0... Patching component oracle.ldap.ssl, 21.0.0.0.0... Patching component oracle.rdbms.dv, 21.0.0.0.0... Patching component oracle.rdbms.scheduler, 21.0.0.0.0... Patching component oracle.xdk.parser.java, 21.0.0.0.0... Patching component oracle.xdk.rsf, 21.0.0.0.0... Patching component oracle.ctx, 21.0.0.0.0... Patching component oracle.rdbms.crs, 21.0.0.0.0... Patching component oracle.sdo, 21.0.0.0.0... Patching component oracle.rdbms.util, 21.0.0.0.0... Patching component oracle.ons.ic, 21.0.0.0.0... Patching component oracle.javavm.server, 21.0.0.0.0... Patching component oracle.javavm.server.core, 21.0.0.0.0... Patching component oracle.xdk, 21.0.0.0.0... Patching component oracle.assistants.acf, 21.0.0.0.0... Patching component oracle.rdbms.oci, 21.0.0.0.0... Patching component oracle.assistants.deconfig, 21.0.0.0.0... Patching component oracle.precomp.common, 21.0.0.0.0... Patching component oracle.precomp.lang, 21.0.0.0.0... Patching component oracle.jdk, 1.8.0.291.09... Patch 33516412 successfully applied. Sub-set patch [33197565] has become inactive due to the application of a super-set patch [33516412]. Sub-set patch [33239276] has become inactive due to the application of a super-set patch [33516412]. Please refer to Doc ID 2161861.1 for any possible further required actions. Log file location: /u01/app/oracle/product/21/cfgtoollogs/opatch/opatch2022-01-19_14-17-44PM_1.log OPatch succeeded.
Please note that with Oracle Database 21c there is no separate OJVM bundle anymore needed. Everything is included into the regular RU.
- datapatch
$ $ORACLE_HOME/OPatch/datapatch -verbose SQL Patching tool version 21.5.0.0.0 Production on Wed Jan 19 14:25:25 2022 Copyright (c) 2012, 2022, Oracle. All rights reserved. Log file for this invocation: /u01/app/oracle/homes/OraDB21Home1/cfgtoollogs/sqlpatch/sqlpatch_8201_2022_01_19_14_25_25/sqlpatch_invocation.log Connecting to database...OK Gathering database info...done Note: Datapatch will only apply or rollback SQL fixes for PDBs that are in an open state, no patches will be applied to closed PDBs. Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation (Doc ID 1585822.1) Bootstrapping registry and package to current versions...done Determining current state...done Current state of interim SQL patches: No interim patches found Current state of release update SQL patches: Binary registry: 21.5.0.0.0 Release_Update 220107185933: Installed PDB CDB$ROOT: Applied 21.4.0.0.0 Release_Update 211006052306 successfully on 15-DEC-21 10.22.32.516540 PM PDB PDB$SEED: Applied 21.4.0.0.0 Release_Update 211006052306 successfully on 15-DEC-21 10.22.32.914473 PM Adding patches to installation queue and performing prereq checks...done Installation queue: For the following PDBs: CDB$ROOT PDB$SEED No interim patches need to be rolled back Patch 33516412 (Database Release Update : 21.5.0.0.220118 (33516412)): Apply from 21.4.0.0.0 Release_Update 211006052306 to 21.5.0.0.0 Release_Update 220107185933 No interim patches need to be applied Installing patches... Patch installation complete. Total patches installed: 2 Validating logfiles...done Patch 33516412 apply (pdb CDB$ROOT): SUCCESS logfile: /u01/app/oracle/homes/OraDB21Home1/cfgtoollogs/sqlpatch/33516412/24589362/33516412_apply_CDB3_CDBROOT_2022Jan19_14_25_36.log (no errors) Patch 33516412 apply (pdb PDB$SEED): SUCCESS logfile: /u01/app/oracle/homes/OraDB21Home1/cfgtoollogs/sqlpatch/33516412/24589362/33516412_apply_CDB3_PDBSEED_2022Jan19_14_26_39.log (no errors) SQL Patching tool complete on Wed Jan 19 14:27:24 2022
Don’t forget to startup the database and all PDBs at first.
- Enabling new optimizer fixes
SQL*Plus: Release 21.0.0.0.0 - Production on Wed Jan 19 14:28:39 2022 Version 21.5.0.0.0 Copyright (c) 1982, 2021, Oracle. All rights reserved. Connected to: Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production Version 21.5.0.0.0 SQL> set serveroutput on; SQL> execute dbms_optim_bundle.getBugsforBundle; 21.5.0.0.220118DBRU: Bug: 32913527, fix_controls: 32913527 Bug: 32766397, fix_controls: 32766397 Bug: 31912834, fix_controls: 31912834 Bug: 33145153, fix_controls: 33145153 Bug: 31843716, fix_controls: 31843716 Bug: 32212062, fix_controls: 32212062 Bug: 33613512, fix_controls: 31880080 PL/SQL procedure successfully completed. SQL> execute dbms_optim_bundle.enable_optim_fixes('ON','BOTH', 'YES') DBMS_OPTIM command: dbms_optim_bundle.enable_optim_fixes('ON', 'BOTH', 'YES') Bundle's _fix_control setting as per action:ON 31988833:1 32800137:0 32408640:1 29738374:1 33325981:1 32913527:0 32766397:0 31912834:1 33145153:1 31843716:0 32212062:0 31880080:0 Taking current instance CDB3 as base, details on _fix_control setting for CON_ID 1 : 1) Current _fix_control setting for spfile: 31988833:1 32800137:0 32408640:1 29738374:1 33325981:1 2) Final _fix_control setting for spfile considering current_setting_precedence is YES 31988833:1 32800137:0 32408640:1 29738374:1 33325981:1 32913527:0 32766397:0 31912834:1 33145153:1 31843716:0 32212062:0 31880080:0 3) Current _fix_control setting in memory: 31988833:1 32800137:0 32408640:1 29738374:1 33325981:1 32913527:0 32766397:0 31912834:0 33145153:0 31843716:0 32212062:0 31880080:0 4) Final _fix_control setting for memory considering current_setting_precedence is YES 31988833:1 32800137:0 32408640:1 29738374:1 33325981:1 32913527:0 32766397:0 31912834:0 33145153:0 31843716:0 32212062:0 31880080:0 WARNING: final _fix_control setting for memory is not same as final _fix_control setting for spfile. Please look at point 2 and 4 above to see the differences. PL/SQL procedure successfully completed.
New optimizer fixes are there for a reason. But it is your choice whether you’d like to enable them after patching, or keep them disabled. Just make sure when you upgrade or create a new database to always enable them.
Applying RU 19.14.0 to my 19c home
Next step is patching my 19c environment. And again I see the same waiting times in opatch I have seen during Patching my environments with the October 2021 Bundle Patches when opatch does “Verifying environment and performing prerequisite checks…“. And badly enough, this check happens twice for no deeeper reason during the opatch apply run – it wastes about 10 minutes in my environment.
- Conflict and space checks
$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./ Oracle Interim Patch Installer version 12.2.0.1.28 Copyright (c) 2022, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /u01/app/oracle/product/19 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/19/oraInst.loc OPatch version : 12.2.0.1.28 OUI version : 12.2.0.7.0 Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2022-01-19_21-08-02PM_1.log Invoking prereq "checkconflictagainstohwithdetail" Prereq "checkConflictAgainstOHWithDetail" passed. OPatch succeeded. $ $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -ph ./ Oracle Interim Patch Installer version 12.2.0.1.28 Copyright (c) 2022, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /u01/app/oracle/product/19 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/19/oraInst.loc OPatch version : 12.2.0.1.28 OUI version : 12.2.0.7.0 Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2022-01-19_21-19-06PM_1.log Invoking prereq "checksystemspace" Prereq "checkSystemSpace" passed. OPatch succeeded.
- opatch apply
$ $ORACLE_HOME/OPatch/opatch apply Oracle Interim Patch Installer version 12.2.0.1.28 Copyright (c) 2022, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/oracle/product/19 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/19/oraInst.loc OPatch version : 12.2.0.1.28 OUI version : 12.2.0.7.0 Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2022-01-19_21-25-44PM_1.log Verifying environment and performing prerequisite checks... -------------------------------------------------------------------------------- Start OOP by Prereq process. Launch OOP... Oracle Interim Patch Installer version 12.2.0.1.28 Copyright (c) 2022, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/oracle/product/19 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/19/oraInst.loc OPatch version : 12.2.0.1.28 OUI version : 12.2.0.7.0 Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2022-01-19_21-31-23PM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: 33515361 Do you want to proceed? [y|n] y User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/u01/app/oracle/product/19') Is the local system ready for patching? [y|n] y User Responded with: Y Backing up files... Applying interim patch '33515361' to OH '/u01/app/oracle/product/19' ApplySession: Optional component(s) [ oracle.network.gsm, 19.0.0.0.0 ] , [ oracle.rdbms.ic, 19.0.0.0.0 ] , [ oracle.rdbms.tg4db2, 19.0.0.0.0 ] , [ oracle.tfa, 19.0.0.0.0 ] , [ oracle.options.olap.api, 19.0.0.0.0 ] , [ oracle.ons.cclient, 19.0.0.0.0 ] , [ oracle.network.cman, 19.0.0.0.0 ] , [ oracle.rdbms.tg4sybs, 19.0.0.0.0 ] , [ oracle.rdbms.tg4msql, 19.0.0.0.0 ] , [ oracle.rdbms.tg4ifmx, 19.0.0.0.0 ] , [ oracle.oid.client, 19.0.0.0.0 ] , [ oracle.rdbms.tg4tera, 19.0.0.0.0 ] , [ oracle.net.cman, 19.0.0.0.0 ] , [ oracle.xdk.companion, 19.0.0.0.0 ] , [ oracle.ons.eons.bwcompat, 19.0.0.0.0 ] , [ oracle.options.olap, 19.0.0.0.0 ] , [ oracle.jdk, 1.8.0.191.0 ] not present in the Oracle Home or a higher version is found. Patching component oracle.help.ohj, 11.1.1.7.0... Patching component oracle.perlint, 5.28.1.0.0... Patching component oracle.rdbms.locator, 19.0.0.0.0... Patching component oracle.perlint.expat, 2.0.1.0.4... ... Patching component oracle.rdbms.rsf.ic, 19.0.0.0.0... Patching component oracle.precomp.common, 19.0.0.0.0... Patching component oracle.precomp.lang, 19.0.0.0.0... Patching component oracle.jdk, 1.8.0.201.0... Patch 33515361 successfully applied. Sub-set patch [33197296] has become inactive due to the application of a super-set patch [33515361]. Sub-set patch [33192793] has become inactive due to the application of a super-set patch [33515361]. Please refer to Doc ID 2161861.1 for any possible further required actions. Log file location: /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2022-01-19_21-31-23PM_1.log OPatch succeeded.
- datapatch
$ $ORACLE_HOME/OPatch/datapatch -verbose SQL Patching tool version 19.14.0.0.0 Production on Wed Jan 19 21:46:51 2022 Copyright (c) 2012, 2021, Oracle. All rights reserved. Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_29286_2022_01_19_21_46_51/sqlpatch_invocation.log Connecting to database...OK Gathering database info...done Note: Datapatch will only apply or rollback SQL fixes for PDBs that are in an open state, no patches will be applied to closed PDBs. Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation (Doc ID 1585822.1) Bootstrapping registry and package to current versions...done Determining current state...done Current state of interim SQL patches: Interim patch 33192694 (OJVM RELEASE UPDATE: 19.13.0.0.211019 (33192694)): Binary registry: Installed PDB CDB$ROOT: Applied successfully on 16-DEC-21 12.10.22.436110 AM PDB PDB$SEED: Applied successfully on 16-DEC-21 12.10.22.446152 AM Current state of release update SQL patches: Binary registry: 19.14.0.0.0 Release_Update 211225122123: Installed PDB CDB$ROOT: Applied 19.13.0.0.0 Release_Update 211004165050 successfully on 15-DEC-21 11.32.48.559776 PM PDB PDB$SEED: Applied 19.13.0.0.0 Release_Update 211004165050 successfully on 15-DEC-21 11.32.49.191356 PM Adding patches to installation queue and performing prereq checks...done Installation queue: For the following PDBs: CDB$ROOT PDB$SEED No interim patches need to be rolled back Patch 33515361 (Database Release Update : 19.14.0.0.220118 (33515361)): Apply from 19.13.0.0.0 Release_Update 211004165050 to 19.14.0.0.0 Release_Update 211225122123 No interim patches need to be applied Installing patches... Patch installation complete. Total patches installed: 2 Validating logfiles...done Patch 33515361 apply (pdb CDB$ROOT): SUCCESS logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/33515361/24589353/33515361_apply_CDB2_CDBROOT_2022Jan19_21_47_29.log (no errors) Patch 33515361 apply (pdb PDB$SEED): SUCCESS logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/33515361/24589353/33515361_apply_CDB2_PDBSEED_2022Jan19_21_47_54.log (no errors) SQL Patching tool complete on Wed Jan 19 21:48:22 2022
- Enabling optimizer fixes
SQL*Plus: Release 19.0.0.0.0 - Production on Wed Jan 19 21:52:50 2022 Version 19.14.0.0.0 Copyright (c) 1982, 2021, Oracle. All rights reserved. Connected to: Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.14.0.0.0 SQL> set serveroutput on SQL> execute dbms_optim_bundle.getBugsforBundle; 19.14.0.0.220118DBRU: Bug: 31945701, fix_controls: 31945701 Bug: 32212062, fix_controls: 32212062 Bug: 32766397, fix_controls: 32766397 Bug: 32508585, fix_controls: 32508585 Bug: 29651517, fix_controls: 29651517 Bug: 31912834, fix_controls: 31912834 Bug: 33145153, fix_controls: 33145153 Bug: 33613512, fix_controls: 31880080 PL/SQL procedure successfully completed. SQL> execute dbms_optim_bundle.enable_optim_fixes('ON','BOTH', 'YES') DBMS_OPTIM command: dbms_optim_bundle.enable_optim_fixes('ON', 'BOTH', 'YES') Bundle's _fix_control setting as per action:ON 29331066:1 28965084:1 28776811:1 28498976:1 28567417:1 28558645:1 29132869:1 29450812:1 29687220:1 29939400:1 30232638:1 30001331:0 29304314:1 29930457:1 30028663:1 28144569:1 28776431:1 27261477:1 31069997:1 31077481:1 28602253:1 29653132:0 29937655:1 30347410:1 30602828:1 30896685:0 29487407:1 30998035:1 30786641:1 31444353:0 30486896:1 28999046:1 30902655:1 30681521:1 29302565:1 30972817:1 30222669:1 31668694:1 31001490:1 30198239:7 30980115:1 30616738:0 31895670:0 19138896:1 31670824:0 9876287:1 30564898:1 32075777:0 30570982:1 32037237:1 30927440:1 30822446:1 24561942:1 31625959:1 31579233:1 29696242:1 31626438:1 30228422:1 17295505:1 29725425:1 30618230:1 30008456:1 30537403:1 30235878:1 30646077:1 29657973:1 29712727:1 20922160:1 30006705:1 29463553:1 30751171:1 31009032:1 30063629:1 30207519:1 31517502:1 30617002:1 30483217:1 30235691:1 30568514:1 28414968:3 32014520:1 30249927:1 31580374:1 29590666:0 29435966:1 28173995:1 29867728:1 30776676:1 26577716:1 30470947:1 30979701:1 30483184:1 31001295:1 31191224:1 31974424:1 29385774:1 28234255:3 31459242:0 31082719:1 28708585:1 31821701:1 32107621:1 26758837:1 31558194:1 30781970:0 30142527:1 31143146:1 31961578:0 31496840:1 22387320:1 30652595:1 25979242:1 32578113:1 32205825:1 32408640:1 31988833:1 32800137:0 31360214:1 32913527:0 29738374:1 33325981:1 31945701:0 32212062:0 32766397:0 32508585:1 29651517:1 31912834:1 33145153:1 31880080:0 ,,, WARNING: final _fix_control setting for memory is not same as final _fix_control setting for spfile. Please look at point 2 and 4 above to see the differences. PL/SQL procedure successfully completed.
And finally …
- OJVM patch
Since I have the JavaVM in my 19c environment, I add the OJVM RU from January 2022 into this home, too.$ $ORACLE_HOME/OPatch/opatch apply Oracle Interim Patch Installer version 12.2.0.1.28 Copyright (c) 2022, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/oracle/product/19 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/19/oraInst.loc OPatch version : 12.2.0.1.28 OUI version : 12.2.0.7.0 Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2022-01-19_22-06-05PM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: 33561310 Do you want to proceed? [y|n] y User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/u01/app/oracle/product/19') Is the local system ready for patching? [y|n] y User Responded with: Y Backing up files... Applying interim patch '33561310' to OH '/u01/app/oracle/product/19' Patching component oracle.javavm.server, 19.0.0.0.0... Patching component oracle.javavm.server.core, 19.0.0.0.0... Patching component oracle.rdbms.dbscripts, 19.0.0.0.0... Patching component oracle.rdbms, 19.0.0.0.0... Patching component oracle.javavm.client, 19.0.0.0.0... Patch 33561310 successfully applied. Sub-set patch [33192694] has become inactive due to the application of a super-set patch [33561310]. Please refer to Doc ID 2161861.1 for any possible further required actions. Log file location: /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2022-01-19_22-06-05PM_1.log OPatch succeeded. [CDB2] oracle@hol:~/33561310 $ s SQL*Plus: Release 19.0.0.0.0 - Production on Wed Jan 19 22:13:08 2022 Version 19.14.0.0.0 Copyright (c) 1982, 2021, Oracle. All rights reserved. Connected to an idle instance. SQL> startup ORACLE instance started. Total System Global Area 1577055352 bytes Fixed Size 9135224 bytes Variable Size 570425344 bytes Database Buffers 989855744 bytes Redo Buffers 7639040 bytes Database mounted. Database opened. SQL> alter pluggable database all open; Pluggable database altered. SQL> exit Disconnected from Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production Version 19.14.0.0.0 [CDB2] oracle@hol:~/33561310 $ $ORACLE_HOME/OPatch/datapatch -verbose SQL Patching tool version 19.14.0.0.0 Production on Wed Jan 19 22:13:40 2022 Copyright (c) 2012, 2021, Oracle. All rights reserved. Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_485_2022_01_19_22_13_40/sqlpatch_invocation.log Connecting to database...OK Gathering database info...done Note: Datapatch will only apply or rollback SQL fixes for PDBs that are in an open state, no patches will be applied to closed PDBs. Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation (Doc ID 1585822.1) Bootstrapping registry and package to current versions...done Determining current state...done Current state of interim SQL patches: Interim patch 33192694 (OJVM RELEASE UPDATE: 19.13.0.0.211019 (33192694)): Binary registry: Not installed PDB CDB$ROOT: Applied successfully on 16-DEC-21 12.10.22.436110 AM PDB PDB$SEED: Applied successfully on 16-DEC-21 12.10.22.446152 AM Interim patch 33561310 (OJVM RELEASE UPDATE: 19.14.0.0.220118 (33561310)): Binary registry: Installed PDB CDB$ROOT: Not installed PDB PDB$SEED: Not installed Current state of release update SQL patches: Binary registry: 19.14.0.0.0 Release_Update 211225122123: Installed PDB CDB$ROOT: Applied 19.14.0.0.0 Release_Update 211225122123 successfully on 19-JAN-22 09.48.19.540225 PM PDB PDB$SEED: Applied 19.14.0.0.0 Release_Update 211225122123 successfully on 19-JAN-22 09.48.19.756424 PM Adding patches to installation queue and performing prereq checks...done Installation queue: For the following PDBs: CDB$ROOT PDB$SEED The following interim patches will be rolled back: 33192694 (OJVM RELEASE UPDATE: 19.13.0.0.211019 (33192694)) No release update patches need to be installed The following interim patches will be applied: 33561310 (OJVM RELEASE UPDATE: 19.14.0.0.220118 (33561310)) Installing patches... Patch installation complete. Total patches installed: 4 Validating logfiles...done Patch 33192694 rollback (pdb CDB$ROOT): SUCCESS logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/33192694/24421575/33192694_rollback_CDB2_CDBROOT_2022Jan19_22_14_08.log (no errors) Patch 33561310 apply (pdb CDB$ROOT): SUCCESS logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/33561310/24538862/33561310_apply_CDB2_CDBROOT_2022Jan19_22_14_43.log (no errors) Patch 33192694 rollback (pdb PDB$SEED): SUCCESS logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/33192694/24421575/33192694_rollback_CDB2_PDBSEED_2022Jan19_22_14_44.log (no errors) Patch 33561310 apply (pdb PDB$SEED): SUCCESS logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/33561310/24538862/33561310_apply_CDB2_PDBSEED_2022Jan19_22_14_44.log (no errors) SQL Patching tool complete on Wed Jan 19 22:14:45 2022
All set now.
Applying RU 12.2.0.1.220118 to my 12.2 home
Next step is patching my 12.2.0.1 environment. I won’t print out the entire output for 12.2.0.1 anymore.
- Conflict and space checks
$ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./ $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -ph ./
Let me just point out that patching my 12.2.0.1 environment does not have these minute-long hangs or waits as I see with 19c.
- opatch apply
$ORACLE_HOME/OPatch/opatch apply
- datapatch
$ORACLE_HOME/OPatch/datapatch -verbose
- Enabling optimizer fixes
SQL*Plus: Release 12.2.0.1.0 Production on Wed Jan 19 22:26:22 2022 Copyright (c) 1982, 2016, Oracle. All rights reserved. Connected to: Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production SQL> set serveroutput on SQL> execute dbms_optim_bundle.getBugsforBundle; 12.2.0.1.210119DBRU: Bug: 32234161, fix_controls: 32075777 PL/SQL procedure successfully completed. SQL> execute dbms_optim_bundle.enable_optim_fixes('ON','BOTH', 'YES') DBMS_OPTIM command: dbms_optim_bundle.enable_optim_fixes('ON', 'BOTH', 'YES') Bundle's _fix_control setting as per action:ON 25476149:1 23249829:1 26019148:1 23643560:1 26986173:1 27466597:1 20107874:1 27321179:1 25120742:1 26536320:1 26423085:1 28072567:1 8932139:1 25405100:1 24745366:1 24926999:1 25643889:0 23738553:1 28201419:1 25575369:1 22070473:1 24841671:1 24573561:1 27000158:1 29450812:1 25926263:1 22174392:1 23738304:1 28835937:1 29687220:1 25792706:1 29930457:1 28776431:1 28602253:1 31310771:0 32075777:0 ... PL/SQL procedure successfully completed.
Applying PSU 11.2.0.4.220118 to my 11.2 home
Next step is patching my 19c environment.
- Conflict and space checks
$ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./ Oracle Interim Patch Installer version 11.2.0.3.32 Copyright (c) 2022, Oracle Corporation. All rights reserved. PREREQ session Oracle Home : /u01/app/oracle/product/11.2.0.4 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/11.2.0.4/oraInst.loc OPatch version : 11.2.0.3.32 OUI version : 11.2.0.4.0 Log file location : /u01/app/oracle/product/11.2.0.4/cfgtoollogs/opatch/opatch2022-01-19_22-36-22PM_1.log Invoking prereq "checkconflictagainstohwithdetail" Prereq "checkConflictAgainstOHWithDetail" passed. OPatch succeeded.
- opatch apply
$ $ORACLE_HOME/OPatch/opatch apply Oracle Interim Patch Installer version 11.2.0.3.32 Copyright (c) 2022, Oracle Corporation. All rights reserved. Oracle Home : /u01/app/oracle/product/11.2.0.4 Central Inventory : /u01/app/oraInventory from : /u01/app/oracle/product/11.2.0.4/oraInst.loc OPatch version : 11.2.0.3.32 OUI version : 11.2.0.4.0 Log file location : /u01/app/oracle/product/11.2.0.4/cfgtoollogs/opatch/opatch2022-01-19_22-36-36PM_1.log Verifying environment and performing prerequisite checks... OPatch continues with these patches: 31983472 32328626 32758711 33128584 33477185 Do you want to proceed? [y|n] y User Responded with: Y All checks passed. Please shutdown Oracle instances running out of this ORACLE_HOME on the local system. (Oracle Home = '/u01/app/oracle/product/11.2.0.4') Is the local system ready for patching? [y|n] y User Responded with: Y Backing up files... Applying sub-patch '31983472' to OH '/u01/app/oracle/product/11.2.0.4' Patching component oracle.rdbms, 11.2.0.4.0... Patching component oracle.ctx, 11.2.0.4.0... Applying sub-patch '32328626' to OH '/u01/app/oracle/product/11.2.0.4' Patching component oracle.rdbms, 11.2.0.4.0... Patching component oracle.rdbms.dbscripts, 11.2.0.4.0... Patching component oracle.rdbms.rsf, 11.2.0.4.0... Patching component oracle.rdbms.dv, 11.2.0.4.0... Patching component oracle.rdbms.rman, 11.2.0.4.0... Patching component oracle.ldap.rsf, 11.2.0.4.0... Patching component oracle.ldap.rsf.ic, 11.2.0.4.0... Applying sub-patch '32758711' to OH '/u01/app/oracle/product/11.2.0.4' Patching component oracle.rdbms, 11.2.0.4.0... Patching component oracle.rdbms.dbscripts, 11.2.0.4.0... Patching component oracle.ctx, 11.2.0.4.0... Patching component oracle.network.rsf, 11.2.0.4.0... Applying sub-patch '33128584' to OH '/u01/app/oracle/product/11.2.0.4' Patching component oracle.rdbms, 11.2.0.4.0... Patching component oracle.rdbms.rsf, 11.2.0.4.0... Patching component oracle.ctx, 11.2.0.4.0... Applying sub-patch '33477185' to OH '/u01/app/oracle/product/11.2.0.4' Patching component oracle.rdbms, 11.2.0.4.0... Patching component oracle.rdbms.dbscripts, 11.2.0.4.0... Patching component oracle.rdbms.rsf, 11.2.0.4.0... Patching component oracle.network.rsf, 11.2.0.4.0... Patching component oracle.rdbms.dv, 11.2.0.4.0... Composite patch 33477185 successfully applied. Log file location: /u01/app/oracle/product/11.2.0.4/cfgtoollogs/opatch/opatch2022-01-19_22-36-36PM_1.log OPatch succeeded.
- catbundle
$ cd $ORACLE_HOME/rdbms/admin [FTEX] oracle@hol:/u01/app/oracle/product/11.2.0.4/rdbms/admin $ s SQL*Plus: Release 11.2.0.4.0 Production on Wed Jan 19 22:39:52 2022 Copyright (c) 1982, 2013, Oracle. All rights reserved. Connected to an idle instance. SQL> startup ORACLE instance started. Total System Global Area 1152450560 bytes Fixed Size 2252584 bytes Variable Size 335544536 bytes Database Buffers 805306368 bytes Redo Buffers 9347072 bytes Database mounted. Database opened. SQL> @catbundle.sql psu apply
But at this point I see a number of errors such as:
CREATE OR REPLACE LIBRARY xdb.DBMS_XDB_LIB wrapped * ERROR at line 1: ORA-01435: user does not exist CREATE OR REPLACE FUNCTION xdb.get_xdb_tablespace wrapped * ERROR at line 1: ORA-01435: user does not exist SQL> grant execute on xdb.get_xdb_tablespace to public; grant execute on xdb.get_xdb_tablespace to public * ERROR at line 1: ORA-04042: procedure, function, package, or package body does not exist SQL> grant execute on sys.check_upgrade to xdb; grant execute on sys.check_upgrade to xdb * ERROR at line 1: ORA-01917: user or role 'XDB' does not exist
That’s expected since my database has no user schema XDB.
Your patch is not available?
In case you miss the patch bundle for your release and platform, please read this blog post.
Further Links and Information
- January 2022 Security Alerts for all products
- Database Server Products Risk Matrix
- MOS Note: 2817011.1 – Critical Patch Update (CPU) Program Jan 2022 Patch Availability Document (PAD)
- MOS Note: 2118136.2 – Assistant: Download Reference for Oracle Database/GI Update, Revision, PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases
–Mike
Hi Mike,
Thanks for the article. I’ve been applying 19.14 on RAC and it fails on crs. Then I tried to install each of the sub-patches from the RU and found that it fails during ACFSRU. So
33515361- ok
33529556- ok
33534448- fails
33239955- ok
33575402- ok
These are the error msgs from opatchauto (on post-patch stage):
2022/01/20 12:46:28 CLSRSC-400: A system reboot is required to continue installing.
Argument “” isn’t numeric in numeric ne (!=) at /u01/app/grid/19/grid/lib/acfslib.pm line 347.
CLSRSC-400: A system reboot – actually doesnt work.
Have you heard smth about this?
p.s. applying 19.13 on the same env goes flawless
Hi Nariman,
I see and hear of several issues – but I haven’t seen specifically that the ACFSRU does not apply well.
I know that it requires a cluster stop and can’t be applied rolling.
Did you open an SR? If yes, please share the SR number with me.
Cheers,
Mike
Dear Mike,
I’ve already solved this and now sharing it here so our readers could also benefit.
As soon as I updated kernel, it was upgraded from “4.18.0-305.19.1.el8_4.x86_64” to “4.18.0-348.12.2.el8_5.x86_64”
and the patching went perfect!
Regards, Nariman
Thank you very much, Nariman!!
And happy that you could fix it even though I would note have considered a kernel update being the solution to it.
Thanks for sharing!
Cheers,
Mike
In addition, I checked the bug database – you may need to open an SR please.
I could find for instance
BUG 31240145 – 19C GRID UPGRADE FAILED WITH CLSRSC-400: A SYSTEM REBOOT IS REQUIRED TO CONTINUE INSTALLING
but this is supposed to be fixed in 19.14.0.
As an alternative, you may instead use out-of-place patching, even for GI.
Cheers,
Mike
Hi Mike! Thank you for the post! Is it still necessary to install OJVM patch separately from RU in fresh installation with runinstaller? Bug is not fixed?
Hi Vadim,
I fear: yes, still a separate install. The bug I filed in April 2021 is still a Sev.1, and still not fixed despite having a high customer count associated with it. But I hope that we’ll get some progress soon.
Sorry for the inconvenience 🙁
Mike
Hi Mike,
I have filed a Service Request with Oracle Support as the Combo patch for AIX seems to be missing the OJVM patch. It only contains one sub directory containing the DB patch (33515361) whereas the same file for Linux contains two sub directories
Thanks for the update – and this is not good 🙁
Cheers,
Mike
wow, thanks. After reading your comment i had to check the Linux x86-64 patch file (33567270) i downloaded yesterday at around 10 am and found out that it only had the OJVM patch included. So the other way around! I downloaded it again right now and now both patches (33515361 + 33561310) are inside.
But for the AIX patch, this one is still missing the OJVM patch.
Hi Mathias,
the best is to open an SR and point support to it. I will drop the responsible person meanwhile a message, too.
Thanks and kind regards,
Mike
Hi Mike,
I noticed in your post that you applied the Database RU, ran datapatch, then applied the OJVM patch and ran datapatch. MOS usually recommends applying both patches and running datapatch one time after both are applied. Is one method better than the other, or are they essentially the same?
And thank you for all that you do for Oracle and the user community!
Tom Raffa
Hi Thomas,
it is fine and enough to run datapatch at the end, only once.
It will update everything then.
Cheers,
Mike
Hello Mike,
just a quick question regarding the Application Express content in this Critical Patch Update:
Wouldn’t it be necessary to additionally install Patch 32598392 (APEX 21.1 Bundle Patch) to fix the APEX CVEs (CVE-2021-37695, CVE-2021-32723, CVE-2021-32808, CVE-2021-32809) mentioned in the database risk matrix ?
And if the answer is yes, what is recommended if someone still needs older APEX versions (e.g. APEX 19.1) ?
Thanks, Juergen
Hi Juergen,
that is a very good and reasonable question.
And yes, if you use APEX, you must patch APEX as well. But since APEX is not a standard install in any of my Hands-On Lab databases (I use it quite frequently on apex.oracle.com), I don’t need to apply anything for APEX. But if you use APEX, you should definitely.
Cheers,
Mike
Hi Mike
12.2.0.1.220118 RU is reintroducing vulnerable log4j files under md folder:
33587128/files/md/jlib/log4j-core-2.9.1.jar
33587128/files/md/jlib/log4j-api-2.9.1.jar
Opening SR with Oracle.
Hi Ashish,
I can’t comment regarding log4j – but I have seen others complaining, too.
Opening an SR is the best way.
Thank you, and I feel really sorry for the inconvenience,
Mike
Hi Mike.
I’m about to start patching 12.2 DB and I noticed with the DB-RU patch (33587128) there are 3 log4j v2.9.1 jar files packaged with the patch (located under ./33587128/files/md/jlib/). Any info on whether or not these are needed/at issue with the log4j security issues from Dec/Jan?
Thanks
Hi Mike.
I am posted my question again about the inclusion of log4j v2.9.1 in the January 2022 12.2 DB-RU (33587128) as Oracle Support is not addressing the issue stating that they don’t need to as 12.2 is in Extended Support.
Any guidance would be greatly appreciated as we need to keep our databases patched but patching with known vulnerabilities is not an acceptable solution.
Thanks!
Hi Devin,
would you mind to share the SR with me, please?
To be clear: Oracle Database 12.2.0.1 is under Limited Error Correction Support which includes Security and Sev,1 bug fixes until end of March 2022.
So you should be able to get fixes for 12.2.0.1.
Cheers,
Mike
Hi,
I want to comment as I noticed this as well when installing January 2022 12.2 DBRU patch set, that those apparently vulnerable log4j files are included in patch set. In MOS Note Doc ID 2828303.1 regarding the Spatial and Graph vulnerabilies, Oracle has stated this:
“Patch 33695048 is only applicable via OPatch on the October 2021 DBRU of the 12.2 and 21.4 releases and the Apr 2021 DBRU of the 18c release. The patch will be included in subsequent DBRUs of each release.”
So I was very surprised to find out that this statement was not true after all. I created SR 3-28339618461 about this last week and got an answer that this was mitigated to development. At that point there was not even this interim patch 33695048 available for Jan 2022 12.2 DBRU, but now I actually see that Oracle has released that on Friday, so now it’s possible to install this interim patch on top of Jan 2022 12.2 DBRU to fix log4j vulnerability.
Maybe it was just a mistake to not include this fix in DBRU, though I think quite a big one thinking about the criticality of log4j and also the statement that was made in MOS Note.
Regards,
Janne
Thanks for the information, Janne – and I agree with you.
But I will keep my hands off the log4j topic.
Still read your SR, and my guess is that it has happened as a mix off code freeze dates and human failure.
Cheers,
Mike
Hi Mike, Your all patching procedures are for patching of standalone databases in manual mode i.e using opatch apply. Nowadays most of the production environments are RAC. Have you posted any separate post to patch GI+DB in RAC environment for January 2022 Patch? If not, are you going to post it soon?
Hi Devi,
if you give me more time and more system resources, then I will happily patch GI etc as well.
It’s on my list of to-dos for a long time …
Cheers,
Mike
Hi Mike, adding to my previous comment, instead of using opatch apply, why don’t you use opatchauto apply?
Experience 🙂
Our ACS engineers usually use opatch instead of opatchapply.
They may know why 🙂
Cheers,
MIke
Hi Mike,
just found this importrant alert for the GI-RU 14
ALERT: Database Service Fails to Start With “CRS-5037: Database does not have the required fix for bug ‘31143870’” AFTER Grid Infrastructure Patching or Upgrade to 19.14 RU (Doc ID 2835152.1)
DB releases prior 18c wont’t start on GI RU 14 anymore.
Cheers Peter
Thanks Peter – I added this to my follow up blog post.
Cheers,
Mike
Dear Mike,
today I downloaded OPatch 12.2.0.1.28 for DB 19.0.0.0.0 (Linux x86-64) from the site you linked above but after unzipping the file I found out that it is Version 12.2.0.1.27 (double checked). Do you know if there are any problems with Version 12.2.0.1.28?
Best,
Boris
Hi Boris,
never seen this before – it should be definitely 28 – and I haven’t heard of issues yet.
Cheers,
Mike
Hi Mike,
thanks for your response. I downloaded the files on the next day again and it was the correct version. Really don’t know what was the problem.
Best,
Boris
Thanks for the feedback, Boris!
Cheers,
Mike
Hi Mike. My SR is 3-28362742811. After digging into this more and applying the patch there are 3 log4j 2.9.1 files being installed (if they didn’t exist already. We removed the files in December per our Cyber Security Teams direction to mitigate all log4j affected versions).
Something I have been noticing in our environment lately is the amount of time that it takes to complete OPatch operations. Rolling back a simple one-off patch takes over 6 minutes.
Is this something others have seen?
bddodb202> time opatch rollback -silent -id 33446866
Oracle Interim Patch Installer version 12.2.0.1.28
Copyright (c) 2022, Oracle Corporation. All rights reserved.
Oracle Home : /oracle/app/oracle/product/19
Central Inventory : /opt/oraInventory
from : /oracle/app/oracle/product/19/oraInst.loc
OPatch version : 12.2.0.1.28
OUI version : 12.2.0.7.0
Log file location : /oracle/app/oracle/product/19/cfgtoollogs/opatch/opatch2022-01-26_12-23-45PM_1.log
Patches will be rolled back in the following order:
33446866
The following patch(es) will be rolled back: 33446866
Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
(Oracle Home = ‘/oracle/app/oracle/product/19’)
Is the local system ready for patching? [y|n]
Y (auto-answered by -silent)
User Responded with: Y
Rolling back patch 33446866…
RollbackSession rolling back interim patch ‘33446866’ from OH ‘/oracle/app/oracle/product/19’
Patching component oracle.rdbms, 19.0.0.0.0…
RollbackSession removing interim patch ‘33446866’ from inventory
Log file location: /oracle/app/oracle/product/19/cfgtoollogs/opatch/opatch2022-01-26_12-23-45PM_1.log
OPatch succeeded.
real 6m39.392s
user 6m53.835s
sys 0m7.706s
Hi Joseph.
I have the same findings – and others, too.
Please open an SR and upload all opatch related logs. My suspicion is that this has to do with opatch reading the inventory. I have environments where this is super-fast, but especially in my 19c environment where I have every RU applied since 19.4.0 it takes an awful long time with visibly no progress for minutes.
Cheers,
Mike
Hi Joseph,
we and other customers have the same issue with opatchauto and there is already a bug regarding this: Bug 33425636 – OPATCHAUTO VER 27 BINARIES APPLY TAKES LONG TIME 19.12
Please also open a SR because the more customers complain about this issue the faster it might get solved.
Best regards
Sven
Hi Mike,
I noticed that when applying a database RU, opatch only updates the Database binaries but not the OCW (see example below for 19.8); in our case we are patching a database OH; ASM and CI are in a different home and could be at a higher or lower version than 19.8.
So, what do we do with the OCW component placed in the database Oracle Home>? What version should it be at ? Is there any documentation/notes on this scenario?
Regards,
Pedro
$ORACLE_HOME/OPatch/opatch lspatches
29585399;OCW RELEASE UPDATE 19.3.0.0.0 (29585399)
31281355;Database Release Update : 19.8.0.0.200714 (31281355)
Hi Pedro,
see here please:
https://mikedietrichde.com/2022/02/07/adding-the-oracle-19-14-0-ocw-gi-bundle-patch-to-my-database-home/
Thanks,
Mike
Mike,
We are currently on Database 19.13.0. I’m not quite sure I understand the distinction between applying RU 19.14, versus Update Revision 19.13.1… What’s the difference between the two, and why would we choose one over the other?
Hi Benoit,
please take 20mins and watch this excerpt from our Virtual Classroom Seminar 1:
https://videohub.oracle.com/media/Upgrade+to+19c+Virtual+Classroom+SeriesA+Release+Strategy+and+Patching+Best+Practices/1_5acfk3kk?st=1444
There we are explaining everything in details.
Cheers,
Mike
Mike, I just happened to go back to the patching website, and noticed that the windows quarterly db patch now has a more recent update date than when I downloaded and applied. The zip file is also a different size. I didnt see any announcement about it from Oracle support. Any idea what is going on?
The file i downloaded is 1,285,964,483 p33575656_190000_MSWIN-x86-64.zip
Thanks Ted
from the patchsite
Description WINDOWS DATABASE BUNDLE PATCH 19.14.0.0.220118
Product Oracle Database Family
Release
Oracle Database 19.0.0.0.0
Platform or Language Click for more information about this option
Last Updated 16-FEB-2022
Size 1.2G (1285680445 bytes)
Hi Ted,
no, unfortunately I don’t know. Is there anything at the end of the README?
I have just seen that the same seemed to have happened on AIX as well since there were things missing. I can just guess …
But you may need to check with Support please.
Thanks,
Mike
Nothing in comments Mike,thanks for responding.
I will open an SR.
Ted
Thanks Ted!
Cheers,
Mike
Hi Mike,
Just want to know if there is an official MOS note for doing update/upgrade from 19.12 to 19.14 where the 19.14 is a new home install?
Scenario is dbhome_1 is at 19.12 and then we installed 19.14 to dbhome_2 and we want to move some database that is running on dbhome_1 to start using dbhome_2, i.e. some, not all as of yet.
Hi Edwin,
just see the README of 19.14. please. It explains how to do it.
Basically it is very straight forward. Once you have the new home, you stop your database in the old home, change your environment, start it in the new home and invoke $ORACLE_HOME/OPatch/datapatch -verbose
Make sure to adjust /etc/oratab if you use it.
Cheers,
Mike
Mike,
Is there any recommendation as to the software owner in this scenario – for example, the 19.13 ORACLE_HOME being owned by OS user ora1913, and the 19.14 ORACLE_HOME being owned by OS user ora1914? If so, any special consideration when restarting the database under the new ORACLE_HOME?
Hi Ben,
I don’t use different owners since it will be the home before and after. I don’t see the benefit of having different owners.
But if you do so, you need to switch the user before you start the db in the new environment and invoke datapatch.
Cheers,
Mike
Hello Mike,
Great blog and source of information.
I just wanted to ask your opinion regarding the ‘Enabling optimizer fixes’ – is it recommended to run it by default or it should be used only to enable fixes when specific issues are encountered?
Thank you for your time.
Hi Milan,
we have a clear recommendation:
1. You create a new database: Do it.
2. You upgrade to a higher release: Do it.
3. You patch to a higher RU: Evaluate it. I tend to YES, but I see when testing is not possible to some extent that people may be shy and cautious.
But no, you don’t “have to”.
Thanks,
Mike
Hello Mike,
After applying RU 19.14.0 to my 19c home, oracle core dump when lsnrctl starts.
lsnrctl start
LSNRCTL for Linux: Version 19.0.0.0.0 – Production on 15-MAR-2022 23:48:01
Copyright (c) 1991, 2021, Oracle. All rights reserved.
Starting /oracle/product/19.3.0/db_1/bin/tnslsnr: please wait…
TNSLSNR for Linux: Version 19.0.0.0.0 – Production
System parameter file is /oracle/product/19.3.0/db_1/network/admin/listener.ora
Log messages written to /oracle/diag/tnslsnr/uui97/listener/alert/log.xml
Listening on: (DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=10.22.21.97)(PORT=2484)))
Connecting to (DESCRIPTION=(ADDRESS=(PROTOCOL=TCPS)(HOST=10.22.21.97)(PORT=2484)))
Aborted (core dumped)
Only use TCPS:
(ADDRESS = (PROTOCOL = TCPS)(HOST = 10.22.21.97)(PORT = 2484))
Alert log:
TNS-00542: SSL Handshake failed
TNS-12560: TNS:protocol adapter error
Trace log:
NET-7445 [4] [[si_signo=4] [si_errno=0] [si_code=2] [si_int=38999456] [si_ptr=0x25315a0] [si_addr=0x7f14e162ed23]] [] [] [] [] [] [] [] [] [] []
========= Dump for incident 17 (NET 7445) ========
Starting a Diag Context default dump (level=3)
at 0x7ffe539c2078 placed dbge.c@1317
—– Call Stack Trace —–
calling call entry argument values in hex
location type point (? means dubious value)
——————– ——– ——————– —————————-
dbgc_dmp()+127 call kgdsdst() 7FFE539C0560 000000003
7FFE539BA8A0 ? 7FFE539BA9B8 ?
000000000 000000083 ?
dbgexPhaseII()+2092 call dbgc_dmp() 7F14E4D0F7B0 7F14E4D0FA80
7FFE539BA8A0 ? 7FFE539BA9B8 ?
000000000 ? 000000083 ?
dbgexProcessError() call dbgexPhaseII() 002441230 002466200
+1871 7FFE539C1801 7FFE539BA9B8 ?
000000000 ? 000000083 ?
dbgePostErrorDirect call dbgexProcessError() 002441230 002466200 000000000
VaList_int()+1672 000000000 000000000 ?
000000083 ?
dbgePostErrorDirect call dbgePostErrorDirect 002441230 7F14E54A1EE4
()+408 VaList_int() 000001D15 000000001 000000002
7FFE539C2530
nlddSigHandler()+32 call dbgePostErrorDirect 002441230 ? 7F14E54A1EE4 ?
6 () 000001D15 ? 000000001 ?
000000002 ? 000000000
skgesig_sigactionHa call nlddSigHandler() 7379735F61726F2F
ndler()+258 676169642F6D6574 000001D15 ?
000000001 ? 000000002 ?
000000000 ?
__sighandler() call skgesig_sigactionHa 000000008 000000000 000000000
ndler() 000000001 ? 000000002 ?
000000000 ?
r0_aes_set_key_x86_ signal __sighandler() 002530EB0 00252EE50 00000000E
intel()+1235 002530F40 ? 000000100 ?
63AE72DBDB20E53E ?
r0_cipher_aes_set_k call r0_aes_set_key_x86_ 002530EB0 ? 00252EE50 ?
ey_x86_intel_enc()+ intel() 00000000E ? 002530F40 ?
56 000000100 ?
63AE72DBDB20E53E ?
r0_cipher_aes_set_k call r0_aes_set_key_x86_ 002530EB0 ? 00252EE50 ?
ey_x86_intel_enc()+ intel() 00000000E ? 002530F40 ?
56 000000100 ?
63AE72DBDB20E53E ?
R1_CIPH_CTX_set_key call r0_cipher_aes_set_k 002530EB0 ? 00252EE50 ?
_bytes_state()+460 ey_x86_intel_enc() 00000000E ? 002530F40 ?
000000100 ?
63AE72DBDB20E53E ?
r0_gcm_imp_key_byte call R1_CIPH_CTX_set_key 002530E60 ? 00252EE50 ?
s()+165 _bytes_state() 00000000E ? 002530F40 ?
000000100 ?
63AE72DBDB20E53E ?
R1_CIPH_CTX_set_key call r0_gcm_imp_key_byte 002530C50 ? 00252EE50 ?
_bytes_state()+460 s() 000000020 ? 002530F40 ?
000000100 ?
63AE72DBDB20E53E ?
r_ck_cipher_init_ba call R1_CIPH_CTX_set_key 002530C50 ? 00252EE50 ?
se()+247 _bytes_state() 000000020 ? 002530F40 ?
000000100 ?
63AE72DBDB20E53E ?
r_ck_cipher_init()+ call r_ck_cipher_init_ba 00252BB00 ? 7FFE539C4470 ?
60 se() 000000001 ? 000000000 ?
000000100 ?
63AE72DBDB20E53E ?
ri_cr_ciph_init()+2 call r_ck_cipher_init() 00252BB00 ? 7FFE539C4470 ?
01 7FFE539C4580 ? 000000001 ?
000000100 ?
63AE72DBDB20E53E ?
R_CR_encrypt_init() call ri_cr_ciph_init() 00252BB00 ? 7F14E1A4B000
+56 7FFE539C44F0 ? 000000001 ?
000000100 ?
63AE72DBDB20E53E ?
ri_tls1_enc_aead()+ call R_CR_encrypt_init() 00252BB00 ? 002530650
769 7FFE539C4580 000000001 ?
000000100 ?
63AE72DBDB20E53E ?
——————— Binary Stack Dump ———————
……..
========== FRAME [1] (dbgc_dmp()+127 -> kgdsdst()) ==========
defined by frame pointers 0x7ffe539c0640 and 0x7ffe539c0550
CALL TYPE: call ERROR SIGNALED: no CALLER: dbgc_dmp
RDI 00007FFE539C0560 RSI 0000000000000003 RDX 00007FFE539BA8A0
RCX 00007FFE539BA9B8 R8 0000000000000000 R9 0000000000000083
RAX 0000000000000000 RBX 0000000002444230 RBP 00007FFE539C0640
R10 00007FFE539BF6F0 R11 0000000000000000 R12 0000000000000003
R13 0000000000000001 R14 0000000002441230 R15 0000000002466200
RSP 00007FFE539C0560 RIP 00007F14E4D0FB8F
You please need to open an SR and check with Oracle Support – I can’t debug such issues on the blog here.
Thanks,
Mike
Hi Mike,
we’ve applied GI RU14 patch 33509923 to our ORACLE
SPARC 19.12 GI successfully, then when tried to apply
the recommended patch 33733953 on top of RU14 (Note 555.1)
we’ve got Patch conflict with 33529556.
Any recommendation ?
Best Regards,
Thanasis
Yes, you need to open an SR please, upload your “opatch lsinventory” and check with Support.
Thanks,
Mike
Hi Mike,
Thanks for the article.
When upgrade mode is required to datapatch ? and why ?
How to do when we have too many databases to datapatch ?
And thank you for all that you do for Oracle and the user community!
Kais
Hi Kais,
datapatch does NOT require STARTUP UPGRADE unless there as a patch mentioning it explicitly.
Since RUs are always RAC ROLLING, there is no STARTUP UPGRADE, and hence datapatch does work in normal mode.
Cheers,
Mike
How long a datapatch (after binary patching is applied) can be delayed while the database is open?
there’s an installation with 300 databases and datapatch can not be applied to all of them at the same time
by example: can de datapatch be applied one month after the binary patching was done?
Hi Guido,
clearly NO. When you apply the binary patch to an environment, then you must make sure that you run datapatch as quickly as possible since your packages etc may now mismatch with the binary changes. In reality, it depends a lot on what you do. But given the number of fixes in e.g. 19.18.0 you should definitely execute datapatch directly after you startup the DBs.
And I don’t see any legit reason why you should not be able to run datapatch in those databases. It is not downtime.
Cheers
Mike
it’s just because as soon as I move all the databases (300) to the new patched oracle home, its resource wise not possible to run datapatch on all 300 databases at the same time, therefore some of the databases will have to wait until the datapatch is run in batches.
so the solution is to start moving to the new home in small batches applying immediately datapatch).. that for a production system with 300 databases can result in a lot of time of planing having to schedule database downtime (usually on the weekends) to restart the database in the new home and apply datapatch.
thanks.
Hi Guido,
I guess you won’t relocate 300 databases all at the same moment in time, right?
Normally, database 1-20 go on day 1, databases 21-50 go on day 2 etc etc.
Or are you saying that you have common downtime for all 300 databases on a single server??
This would be very unusual but of course not unrealistic.
So my advice is:
When you relocate databases, do this in batches – and invoke datapatch after you start them up. Datapatch runs online. The recompilation can consume significant CPU, that is right. So don’t do it for too many at the same time.
In case you have a RAC, this can be done node-by-node in a rolling fashion.
Cheers,
Mike