Patching all my environments with the April 2021 Patch Bundles

Oh well, time flies. And it is April 2021, and hence I will start Patching all my environments with the April 2021 Patch Bundles. In my case, this will be 19.11.0 and 12.2.0.1 Release Updates. But there will be an additional blog post for the OJVM bundle, too.

As usual, an important annotation upfront: I patch in-place due to space issues. But in reality, you please patch always out-of-place with a separate home. Please see this blog post about how to apply the RU directly when you provision a new home with OUI.

Security Alert April 2021

Find the Security Alert for April 2021 here. And don’t forget to take a look at the Oracle Database Server Risk Matrix for April 2021. This time the risk score for core database products is relatively low – and also the usual OJVM issue is leveled at 5.3 “only” on a scale up to 10.0 max.

Patching all my environments with the April 2021 Patch Bundles

Risk Matrix Oracle Database Server Products – April 2021 -Source: https://www.oracle.com/security-alerts/cpuapr2021.html#AppendixDB

But to be frank, the reason why 19.11.0 is so important is the number of fixes included into it. But the list of Fixed Bugs (MOS Note: 2523220.1) hasn’t been updated yet while I’m writing this article.

Database Patch Bundles

You will find the links to the individual patch bundles in MOS Note: 2749094.1 – Critical Patch Update (CPU) Program Apr 2021 Patch Availability Document (PAD). And please note that a patch number in the document does not necessarily mean that your patch is available already. Please find a discussion about this recurring topic here: Why is the release update not available on my platform yet? Check especially the Section 2.2 (Post Release Patches).

One of the most important notes, MOS Note: 2118136.2 – Assistant: Download Reference for Oracle Database/GI Update, Revision, PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases has been updated already with the links for the April 2021 bundles. Just be aware that some platforms are delayed and check section 2.2 of MOS Note: 2725756.1. For instance, all ports except Linux have currently a projected availability date of April 30, 2021 for the RU 19.11 including Windows.

Do I need a new OPatch?

First check while the download is progressing: Do I need to refresh my OPatch versions?

  • 19.11.0 requires opatch 12.2.0.1.24 or later
  • 12.2.0.1 April 2021 requires opatch 12.2.0.1.23 or later

In my case I need to update opatch in my 19c home. The 6880880 link from the 19.11.0 Readme brings you directly to the correct download:

Download OPatch via 6880880. Wipe out your current OPatch directory in your homes. Once you unzip the new OPatch bundles. you can proceed.

Applying RU 19.11.0 to my 19c home

At first, I unzip the patch into a separate directory and place myself into this directory ~/32545013 . This time my apply run will be a bit more interesting since I have one-off patches for Blockchain Tables and for DBMS_OPTIM_BUNDLE in my 19.10.0 home. But you will see below under 2. that opatch solves this flawlessly.

And just to be clear, I install the RUs in-place only because I have a toy-environment and not much space left. In your real wold environments you please always either clone the home and apply the new RU, or you install 19.3.0 with the newest RU on top in one pass. See this blog post about how to apply the RU directly when you provision a new home with OUI for more information.

  1. Patch conflict check

    $  $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./
    Oracle Interim Patch Installer version 12.2.0.1.24
    Copyright (c) 2021, Oracle Corporation.  All rights reserved.
    
    PREREQ session
    
    Oracle Home       : /u01/app/oracle/product/19
    Central Inventory : /u01/app/oraInventory
       from           : /u01/app/oracle/product/19/oraInst.loc
    OPatch version    : 12.2.0.1.24
    OUI version       : 12.2.0.7.0
    Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2021-04-21_13-47-57PM_1.log
    
    Invoking prereq "checkconflictagainstohwithdetail"
    
    Prereq "checkConflictAgainstOHWithDetail" passed.
    
    OPatch succeeded.

    In addition, a quick space check:

    $  $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -ph ./
    Oracle Interim Patch Installer version 12.2.0.1.24
    Copyright (c) 2021, Oracle Corporation.  All rights reserved.
    
    PREREQ session
    
    Oracle Home       : /u01/app/oracle/product/19
    Central Inventory : /u01/app/oraInventory
       from           : /u01/app/oracle/product/19/oraInst.loc
    OPatch version    : 12.2.0.1.24
    OUI version       : 12.2.0.7.0
    Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2021-04-21_14-04-51PM_1.log
    
    Invoking prereq "checksystemspace"
    
    Prereq "checkSystemSpace" passed.
    
    OPatch succeeded.
    

    All looks good, my checks ran fine.
    .

  2. opatch apply

    $  $ORACLE_HOME/OPatch/opatch apply
    Oracle Interim Patch Installer version 12.2.0.1.24
    Copyright (c) 2021, Oracle Corporation.  All rights reserved.
    
    
    Oracle Home       : /u01/app/oracle/product/19
    Central Inventory : /u01/app/oraInventory
       from           : /u01/app/oracle/product/19/oraInst.loc
    OPatch version    : 12.2.0.1.24
    OUI version       : 12.2.0.7.0
    Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2021-04-21_14-08-04PM_1.log
    
    Verifying environment and performing prerequisite checks...
    
    Conflicts/Supersets for each patch are:
    
    Patch : 32545013
    
    	Bug Superset of 32431413
    	Super set bugs are:
    	32431413
    
    	Bug Superset of 31862593
    	Super set bugs are:
    	31862593
    
    
    Patches [   32431413   31862593 ] will be rolled back.
    
    
    --------------------------------------------------------------------------------
    Start OOP by Prereq process.
    Launch OOP...
    
    Oracle Interim Patch Installer version 12.2.0.1.24
    Copyright (c) 2021, Oracle Corporation.  All rights reserved.
    
    
    Oracle Home       : /u01/app/oracle/product/19
    Central Inventory : /u01/app/oraInventory
       from           : /u01/app/oracle/product/19/oraInst.loc
    OPatch version    : 12.2.0.1.24
    OUI version       : 12.2.0.7.0
    Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2021-04-21_14-10-26PM_1.log
    
    Verifying environment and performing prerequisite checks...
    
    Conflicts/Supersets for each patch are:
    
    Patch : 32545013
    
    	Bug Superset of 31862593
    	Super set bugs are:
    	31862593
    
    	Bug Superset of 32431413
    	Super set bugs are:
    	32431413
    
    
    Patches [   32431413   31862593 ] will be rolled back.
    
    OPatch continues with these patches:   32545013  
    
    Do you want to proceed? [y|n]
    y
    User Responded with: Y
    All checks passed.
    
    Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
    (Oracle Home = '/u01/app/oracle/product/19')
    
    
    Is the local system ready for patching? [y|n]
    y
    User Responded with: Y
    Backing up files...
    Applying interim patch '32545013' to OH '/u01/app/oracle/product/19'
    ApplySession: Optional component(s) [ oracle.network.gsm, 19.0.0.0.0 ] , [ oracle.rdbms.ic, 19.0.0.0.0 ] , [ oracle.rdbms.tg4db2, 19.0.0.0.0 ] , [ oracle.tfa, 19.0.0.0.0 ] , [ oracle.options.olap.api, 19.0.0.0.0 ] , [ oracle.ons.cclient, 19.0.0.0.0 ] , [ oracle.options.olap, 19.0.0.0.0 ] , [ oracle.network.cman, 19.0.0.0.0 ] , [ oracle.oid.client, 19.0.0.0.0 ] , [ oracle.ons.eons.bwcompat, 19.0.0.0.0 ] , [ oracle.net.cman, 19.0.0.0.0 ] , [ oracle.xdk.companion, 19.0.0.0.0 ] , [ oracle.jdk, 1.8.0.191.0 ]  not present in the Oracle Home or a higher version is found.
    Rolling back interim patch '32431413' from OH '/u01/app/oracle/product/19'
    
    Patching component oracle.rdbms, 19.0.0.0.0...
    
    Patching component oracle.rdbms.rsf, 19.0.0.0.0...
    RollbackSession removing interim patch '32431413' from inventory
    Rolling back interim patch '31862593' from OH '/u01/app/oracle/product/19'
    
    Patching component oracle.rdbms, 19.0.0.0.0...
    RollbackSession removing interim patch '31862593' from inventory
    
    
    OPatch back to application of the patch '32545013' after auto-rollback.
    
    
    Patching component oracle.rdbms.rsf, 19.0.0.0.0...
    
    Patching component oracle.rdbms.util, 19.0.0.0.0...
    
    Patching component oracle.rdbms, 19.0.0.0.0...
    
    [... cut and shorten output ...]
    
    Patching component oracle.ovm, 19.0.0.0.0...
    
    Patching component oracle.rdbms.rsf.ic, 19.0.0.0.0...
    
    Patching component oracle.precomp.common, 19.0.0.0.0...
    
    Patching component oracle.precomp.lang, 19.0.0.0.0...
    
    Patching component oracle.jdk, 1.8.0.201.0...
    Patch 32545013 successfully applied.
    Sub-set patch [32218454] has become inactive due to the application of a super-set patch [32545013].
    Please refer to Doc ID 2161861.1 for any possible further required actions.
    OPatch Session completed with warnings.
    Log file location: /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2021-04-21_14-10-26PM_1.log
    
    OPatch completed with warnings.
    

    Ok, the two one-offs seem to be included in the RU 19.11.0 as there was no conflict flagged. But what do I do with the warning at the end of the opatch run about Sub-set patch [32218454]?

    In such situations I know why you all love patching so much. A quick check with 32218454 on MOS reveals: This is the 19.10.0 RU. No idea why this gets flagged as a warning as it is expected certainly that 19.11.0 invalidates and obsoletes 19.10.0 – especially since RUs are cumulative.
    .

  3. datapatch

    $  $ORACLE_HOME/OPatch/datapatch -verbose
    SQL Patching tool version 19.11.0.0.0 Production on Wed Apr 21 14:40:26 2021
    Copyright (c) 2012, 2021, Oracle.  All rights reserved.
    
    Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_2302_2021_04_21_14_40_26/sqlpatch_invocation.log
    
    Connecting to database...OK
    Gathering database info...done
    
    Note:  Datapatch will only apply or rollback SQL fixes for PDBs
           that are in an open state, no patches will be applied to closed PDBs.
           Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
           (Doc ID 1585822.1)
    
    Bootstrapping registry and package to current versions...done
    Determining current state...done
    
    Current state of interim SQL patches:
      No interim patches found
    
    Current state of release update SQL patches:
      Binary registry:
        19.11.0.0.0 Release_Update 210413004009: Installed
      PDB CDB$ROOT:
        Applied 19.10.0.0.0 Release_Update 210108185017 successfully on 20-JAN-21 01.03.57.749255 AM
      PDB PDB$SEED:
        Applied 19.10.0.0.0 Release_Update 210108185017 successfully on 20-JAN-21 01.03.58.234326 AM
      PDB PDB1:
        Applied 19.10.0.0.0 Release_Update 210108185017 successfully on 20-JAN-21 01.03.58.234326 AM
    
    Adding patches to installation queue and performing prereq checks...done
    Installation queue:
      For the following PDBs: CDB$ROOT PDB$SEED PDB1
        No interim patches need to be rolled back
        Patch 32545013 (Database Release Update : 19.11.0.0.210420 (32545013)):
          Apply from 19.10.0.0.0 Release_Update 210108185017 to 19.11.0.0.0 Release_Update 210413004009
        No interim patches need to be applied
    
    Installing patches...
    Patch installation complete.  Total patches installed: 3
    
    Validating logfiles...done
    Patch 32545013 apply (pdb CDB$ROOT): SUCCESS
      logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/32545013/24175065/32545013_apply_CDB2_CDBROOT_2021Apr21_14_41_01.log (no errors)
    Patch 32545013 apply (pdb PDB$SEED): SUCCESS
      logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/32545013/24175065/32545013_apply_CDB2_PDBSEED_2021Apr21_14_42_22.log (no errors)
    Patch 32545013 apply (pdb PDB1): SUCCESS
      logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/32545013/24175065/32545013_apply_CDB2_PDB1_2021Apr21_14_42_22.log (no errors)
    SQL Patching tool complete on Wed Apr 21 14:43:47 2021

    This worked fine, too. And as always, please ensure that all your PDBs are open read-write when you execute datapatch.

 

Applying RU 12.2.0.1.210420 to my 12.2.0.1 home

I won’t copy/paste all steps as the output is similar to the above. And I can still use the opatch from my January 2021 patch apply operation.

  1. Patch conflict check
    $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./
  2. opatch apply
    $ORACLE_HOME/OPatch/opatch apply
  3. datapatch
    $ORACLE_HOME/OPatch/datapatch -verbose

All steps worked fine and flawless.

But there may be a bit more to do in addition.

Important Fixes on top?

Please check MOS Note: 555.1 (Oracle Database 19c Important Recommended One-off Patches) from time to time. Currently there are no recommended one-off patches on top of 19.11.0 noted. Even though, the Data Pump Super Patch is coming to my mind immediately. And it should be available for 19.11.0 soon as well under patch 32551008.

For Grid Infrastructure 19.11.0 you definitely should apply the fix mentioned in MOS Note: 2774284.1 – ALERT: While applying or after applying 19.11 GI RU, “crsctl start crs -wait” hangs or databases fail to start. While I’m adding this on May 17, 2021, the one-off patch 32847378 for Linux has been made available already.

Enabling optimizer fixes

As you can see in my previous blog post, in 19.10.0 an extra patch was necessary to tell DBMS_OPTIM_BUNDLE about the existing optimizer fixes turned off by default.

But this time all seems to work fine and as expected:

SQL> set serveroutput on;
SQL> execute dbms_optim_bundle.getBugsforBundle;

19.11.0.0.210420DBRU:
    Bug: 32037237,  fix_controls: 32037237
    Bug: 30927440,  fix_controls: 30927440
    Bug: 31788104,  fix_controls: 30822446
    Bug: 24561942,  fix_controls: 24561942
    Bug: 31625959,  fix_controls: 31625959
    Bug: 31976303,  fix_controls: 31579233
    Bug: 29696242,  fix_controls: 29696242
    Bug: 31626438,  fix_controls: 31626438
    Bug: 30228422,  fix_controls: 30228422
    Bug: 32122574,  fix_controls: 17295505
    Bug: 29725425,  fix_controls: 29725425
    Bug: 30618230,  fix_controls: 30618230
    Bug: 30008456,  fix_controls: 30008456
    Bug: 30537403,  fix_controls: 30537403
    Bug: 30235878,  fix_controls: 30235878
    Bug: 30646077,  fix_controls: 30646077
    Bug: 29657973,  fix_controls: 29657973
    Bug: 30527198,  fix_controls: 29712727
    Bug: 20922160,  fix_controls: 20922160
    Bug: 30006705,  fix_controls: 30006705
    Bug: 29463553,  fix_controls: 29463553
    Bug: 30751171,  fix_controls: 30751171
    Bug: 31009032,  fix_controls: 31009032
    Bug: 30207519,  fix_controls: 30063629, 30207519
    Bug: 31517502,  fix_controls: 31517502
    Bug: 30617002,  fix_controls: 30617002
    Bug: 30483217,  fix_controls: 30483217
    Bug: 30235691,  fix_controls: 30235691
    Bug: 30568514,  fix_controls: 30568514
    Bug: 28414968,  fix_controls: 28414968
    Bug: 32014520,  fix_controls: 32014520
    Bug: 30249927,  fix_controls: 30249927
    Bug: 31580374,  fix_controls: 31580374
    Bug: 29590666,  fix_controls: 29590666
    Bug: 29435966,  fix_controls: 29435966
    Bug: 29867728,  fix_controls: 28173995, 29867728
    Bug: 30776676,  fix_controls: 30776676
    Bug: 26577716,  fix_controls: 26577716
    Bug: 30470947,  fix_controls: 30470947
    Bug: 30979701,  fix_controls: 30979701
    Bug: 31435308,  fix_controls: 30483184, 31001295
    Bug: 31191224,  fix_controls: 31191224
    Bug: 31974424,  fix_controls: 31974424
    Bug: 29385774,  fix_controls: 29385774
    Bug: 28234255,  fix_controls: 28234255

PL/SQL procedure successfully completed.

So I can easily enable the fixes:

SQL> execute dbms_optim_bundle.enable_optim_fixes('ON','BOTH', 'YES')

All set.

Further Links and Information

–Mike

Share this: