Patching all my environments with the July 2021 Patch Bundles

It’s patching day again. Hurray! Or not. I realize that at patching day, the 19c bundles are all missing. So wrote this blog post a bit after the usual release day. In my case this will include Oracle 19.12.0 RU and the July 2021 RU for Oracle 12.2.0.1. Please find the details about Patching all my environments with the July 2021 Patch Bundles below.

As usual, an important annotation upfront: I patch in-place due to space issues. But in reality, you please patch always out-of-place with a separate home. Please see this blog post about how to apply the RU directly when you provision a new home with OUI.

Security Alert July 2021

You will find the July 2021 Security Alerts for all products here. And as usual, please pay close attention to the Database Server Products Risk Matrix. There are several 7+ issues with Oracle Advanced Networking, Oracle TEXT and XML DB. So it is highly advised to apply this patch bundle.

You may realize that neither 11.2.0.4 nor 18c are mentioned in the risk matrix anymore. As both are out of Premier or Extended Support, there are no regular patch bundles anymore. But the issues noted in the risk matrix may happen in both these releases, and potentially in older releases as well.

Patching all my environments with the July 2021 Patch Bundles

In addition, Oracle 19.12.0 delivers again a very high number of fixes.

Database Patch Bundles

You will find the links to the individual patch bundles in MOS Note: 2773670.1 – Critical Patch Update (CPU) Program Jul 2021 Patch Availability Document (PAD). But you can use as well MOS Note: 2118136.2 – Assistant: Download Reference for Oracle Database/GI Update, Revision, PSU, SPU(CPU), Bundle Patches, Patchsets and Base Releases to download “your” bundle.

Do I need a new OPatch?

First check while the download is progressing: Do I need to refresh my OPatch versions?

  • 19.12.0 requires opatch 12.2.0.1.25 or later
  • 12.2.0.1 July 2021 requires opatch 12.2.0.1.25 or later

In my case I need to update opatch in both homes. The 6880880 link from the readmes takes you directly to the correct download. And it may not wonder you, but both opatch releases are identical even though both sail with different display labels on MOS. So it is fairly enough in my case to download it only once.

Wipe out your current OPatch directory in your homes. Once you unzip the new OPatch bundles. you can proceed.

Applying RU 19.12.0 to my 19c home

I always do something you shouldn’t do. I apply the patch to my existing home. This has to do with the available space in my lab environment. You please provide a new home and apply the patch(es) right away as I describe here.

Applying RU 12.2.0.1 July 2021 to my 12.2.0.1 home

I won’t copy/paste all steps as the output is similar to the above. And I can still use the opatch from my January 2021 patch apply operation.

  1. Patch conflict and space checks
    $ $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./
    Oracle Interim Patch Installer version 12.2.0.1.25
    Copyright (c) 2021, Oracle Corporation.  All rights reserved.
    
    PREREQ session
    
    Oracle Home       : /u01/app/oracle/product/19
    Central Inventory : /u01/app/oraInventory
       from           : /u01/app/oracle/product/19/oraInst.loc
    OPatch version    : 12.2.0.1.25
    OUI version       : 12.2.0.7.0
    Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2021-07-26_23-01-11PM_1.log
    
    Invoking prereq "checkconflictagainstohwithdetail"
    
    Prereq "checkConflictAgainstOHWithDetail" passed.
    
    OPatch succeeded.
    $ $ORACLE_HOME/OPatch/opatch prereq CheckSystemSpace -ph ./
    Oracle Interim Patch Installer version 12.2.0.1.25
    Copyright (c) 2021, Oracle Corporation.  All rights reserved.
    
    PREREQ session
    
    Oracle Home       : /u01/app/oracle/product/19
    Central Inventory : /u01/app/oraInventory
       from           : /u01/app/oracle/product/19/oraInst.loc
    OPatch version    : 12.2.0.1.25
    OUI version       : 12.2.0.7.0
    Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2021-07-26_23-05-55PM_1.log
    
    Invoking prereq "checksystemspace"
    
    
    Prereq "checkSystemSpace" passed.
    
    OPatch succeeded.

    At least in my environment, both checks with the new opatch 25 take an awful long time. I think I waited 3 minutes for each check to return back to the command prompt. And since opatch does the checks again when I call the apply in the next stage, the wait time happens again when it tells me “Verifying environment and performing prerequisite checks…“.

    Now I was curious and checked the logfile /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2021-07-26_23-14-10PM_1.log:

    [Jul 26, 2021 11:14:23 PM] [INFO]   [OPSR-TIME] Loading cooked inventory
    [Jul 26, 2021 11:14:23 PM] [INFO]   [OPSR-MEMORY] : Loading cooked one offs. Heap memory used 514 (MB)
    [Jul 26, 2021 11:14:28 PM] [INFO]   [OPSR-MEMORY] : Loaded cooked oneoffs. Heap memory used : 348 (MB)
    [Jul 26, 2021 11:14:28 PM] [INFO]   [OPSR-TIME] Cooked inventory loaded successfully
    [Jul 26, 2021 11:14:28 PM] [INFO]   CUP_LOG: Found poh CUP 30557433 is a subset of other poh CUP: 30869156
    [Jul 26, 2021 11:14:28 PM] [INFO]   CUP_LOG: Found poh CUP 30557433 is a subset of other poh CUP: 31281355
    [Jul 26, 2021 11:14:28 PM] [INFO]   CUP_LOG: Found poh CUP 32218454 is a subset of other poh CUP: 32545013
    [Jul 26, 2021 11:17:43 PM] [INFO]   Checking if Oracle Home has components required by patches...
    [Jul 26, 2021 11:17:43 PM] [INFO]   CheckMissingComps: Cached file does not exist or is invalid, re-build prereq result.
    [Jul 26, 2021 11:17:43 PM] [INFO]   Checking conflict among patches...
    [Jul 26, 2021 11:17:43 PM] [INFO]   Running prereq checkConflictAmongPatchesWithDetail
    [Jul 26, 2021 11:17:44 PM] [INFO]   Following patches can be applied:  32904851

    No idea what it takes over 3 minutes to check for Oracle components which require patching. My guess is that is has to do with updating the inventory as you can see below.

    Just a hint: It is always good to tail -f this logfile because it tells you much more about what opatch is doing than you’ll see on the command prompt.

  2. opatch apply
    $ $ORACLE_HOME/OPatch/opatch apply
    Oracle Interim Patch Installer version 12.2.0.1.25
    Copyright (c) 2021, Oracle Corporation.  All rights reserved.
    
    
    Oracle Home       : /u01/app/oracle/product/19
    Central Inventory : /u01/app/oraInventory
       from           : /u01/app/oracle/product/19/oraInst.loc
    OPatch version    : 12.2.0.1.25
    OUI version       : 12.2.0.7.0
    Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2021-07-26_23-10-05PM_1.log
    
    Verifying environment and performing prerequisite checks...
    
    --------------------------------------------------------------------------------
    Start OOP by Prereq process.
    Launch OOP...
    
    Oracle Interim Patch Installer version 12.2.0.1.25
    Copyright (c) 2021, Oracle Corporation.  All rights reserved.
    
    
    Oracle Home       : /u01/app/oracle/product/19
    Central Inventory : /u01/app/oraInventory
       from           : /u01/app/oracle/product/19/oraInst.loc
    OPatch version    : 12.2.0.1.25
    OUI version       : 12.2.0.7.0
    Log file location : /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2021-07-26_23-14-10PM_1.log
    
    Verifying environment and performing prerequisite checks...
    OPatch continues with these patches:   32904851  
    
    Do you want to proceed? [y|n]
    y
    User Responded with: Y
    All checks passed.
    
    Please shutdown Oracle instances running out of this ORACLE_HOME on the local system.
    (Oracle Home = '/u01/app/oracle/product/19')
    
    
    Is the local system ready for patching? [y|n]
    y
    User Responded with: Y
    Backing up files...
    Applying interim patch '32904851' to OH '/u01/app/oracle/product/19'
    ApplySession: Optional component(s) [ oracle.network.gsm, 19.0.0.0.0 ] , [ oracle.rdbms.ic, 19.0.0.0.0 ] , [ oracle.rdbms.tg4db2, 19.0.0.0.0 ] , [ oracle.tfa, 19.0.0.0.0 ] , [ oracle.oid.client, 19.0.0.0.0 ] , [ oracle.network.cman, 19.0.0.0.0 ] , [ oracle.options.olap, 19.0.0.0.0 ] , [ oracle.options.olap.api, 19.0.0.0.0 ] , [ oracle.xdk.companion, 19.0.0.0.0 ] , [ oracle.rdbms.tg4ifmx, 19.0.0.0.0 ] , [ oracle.ons.eons.bwcompat, 19.0.0.0.0 ] , [ oracle.rdbms.tg4tera, 19.0.0.0.0 ] , [ oracle.ons.cclient, 19.0.0.0.0 ] , [ oracle.rdbms.tg4sybs, 19.0.0.0.0 ] , [ oracle.net.cman, 19.0.0.0.0 ] , [ oracle.rdbms.tg4msql, 19.0.0.0.0 ] , [ oracle.jdk, 1.8.0.191.0 ]  not present in the Oracle Home or a higher version is found.
    
    Patching component oracle.perlint, 5.28.1.0.0...
    
    Patching component oracle.rdbms.locator, 19.0.0.0.0...
    
    Patching component oracle.rdbms.rsf, 19.0.0.0.0...
    
    Patching component oracle.rdbms.util, 19.0.0.0.0...
    
    Patching component oracle.rdbms, 19.0.0.0.0...
    
    [...]
    
    Patching component oracle.mgw.common, 19.0.0.0.0...
    
    Patching component oracle.rdbms.rsf.ic, 19.0.0.0.0...
    
    Patching component oracle.precomp.common, 19.0.0.0.0...
    
    Patching component oracle.precomp.lang, 19.0.0.0.0...
    
    Patching component oracle.jdk, 1.8.0.201.0...
    Patch 32904851 successfully applied.
    Sub-set patch [32545013] has become inactive due to the application of a super-set patch [32904851].
    Please refer to Doc ID 2161861.1 for any possible further required actions.
    Log file location: /u01/app/oracle/product/19/cfgtoollogs/opatch/opatch2021-07-26_23-14-10PM_1.log
    
    OPatch succeeded.
    

    This part of the patch (cut from the logfile) took so long – see the timestamps:

    [Jul 26, 2021 11:29:28 PM] [INFO]   [OPSR-MEMORY] : Loaded cooked oneoffs. Heap memory used : 959 (MB)
    [Jul 26, 2021 11:29:28 PM] [INFO]   [OPSR-TIME] Cooked inventory loaded successfully
    [Jul 26, 2021 11:29:28 PM] [INFO]   CUP_LOG: Found poh CUP 30557433 is a subset of other poh CUP: 30869156
    [Jul 26, 2021 11:29:28 PM] [INFO]   CUP_LOG: Found poh CUP 30557433 is a subset of other poh CUP: 31281355
    [Jul 26, 2021 11:29:28 PM] [INFO]   CUP_LOG: Found poh CUP 32218454 is a subset of other poh CUP: 32545013
    [Jul 26, 2021 11:29:28 PM] [INFO]   CUP_LOG: Found poh CUP 32218454 is a subset of other poh CUP: 32904851
    [Jul 26, 2021 11:29:28 PM] [INFO]   CUP_LOG: Found poh CUP 32545013 is a subset of other poh CUP: 32904851
    [Jul 26, 2021 11:33:47 PM] [INFO]   [OPSR-TIME] Patch 32904851 saved to inventory
    [Jul 26, 2021 11:33:47 PM] [INFO]   ApplySession: Skip patch verification.
    [Jul 26, 2021 11:33:47 PM] [INFO]   [OPSR-TIME] Finished applying patch "32904851" to local system
    [Jul 26, 2021 11:33:48 PM] [INFO]   The following re-links will eventually be skipped because they are duplicates: client_sharedlib, iorion, libasmclntsh19.ohso, itnsping,
    
  3. datapatch
    $ $ORACLE_HOME/OPatch/datapatch -verbose
    SQL Patching tool version 19.12.0.0.0 Production on Mon Jul 26 23:38:30 2021
    Copyright (c) 2012, 2021, Oracle.  All rights reserved.
    
    Log file for this invocation: /u01/app/oracle/cfgtoollogs/sqlpatch/sqlpatch_18299_2021_07_26_23_38_30/sqlpatch_invocation.log
    
    Connecting to database...OK
    Gathering database info...done
    
    Note:  Datapatch will only apply or rollback SQL fixes for PDBs
           that are in an open state, no patches will be applied to closed PDBs.
           Please refer to Note: Datapatch: Database 12c Post Patch SQL Automation
           (Doc ID 1585822.1)
    
    Bootstrapping registry and package to current versions...done
    Determining current state...done
    
    Current state of interim SQL patches:
      No interim patches found
    
    Current state of release update SQL patches:
      Binary registry:
        19.12.0.0.0 Release_Update 210716141810: Installed
      PDB CDB$ROOT:
        Applied 19.11.0.0.0 Release_Update 210413004009 successfully on 21-APR-21 02.43.40.035316 PM
      PDB PDB$SEED:
        Applied 19.11.0.0.0 Release_Update 210413004009 successfully on 21-APR-21 02.43.41.532161 PM
    
    Adding patches to installation queue and performing prereq checks...done
    Installation queue:
      For the following PDBs: CDB$ROOT PDB$SEED
        No interim patches need to be rolled back
        Patch 32904851 (Database Release Update : 19.12.0.0.210720 (32904851)):
          Apply from 19.11.0.0.0 Release_Update 210413004009 to 19.12.0.0.0 Release_Update 210716141810
        No interim patches need to be applied
    
    Installing patches...
    Patch installation complete.  Total patches installed: 2
    
    Validating logfiles...done
    Patch 32904851 apply (pdb CDB$ROOT): SUCCESS
      logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/32904851/24343243/32904851_apply_CDB2_CDBROOT_2021Jul26_23_39_36.log (no errors)
    Patch 32904851 apply (pdb PDB$SEED): SUCCESS
      logfile: /u01/app/oracle/cfgtoollogs/sqlpatch/32904851/24343243/32904851_apply_CDB2_PDBSEED_2021Jul26_23_41_03.log (no errors)
    SQL Patching tool complete on Mon Jul 26 23:42:00 2021
  4. Enabling new optimizer fixes
    Finally I will enable the new optimizer fixes which are disabled by default with DBMS_OPTIM_BUNDLE.

    SQL> set serveroutput on;
    SQL> execute dbms_optim_bundle.getBugsforBundle;
    
    19.12.0.0.210720DBRU:
        Bug: 31459242,  fix_controls: 31459242
        Bug: 31082719,  fix_controls: 31082719
        Bug: 28708585,  fix_controls: 28708585
        Bug: 31821701,  fix_controls: 31821701
        Bug: 32107621,  fix_controls: 32107621
        Bug: 26758837,  fix_controls: 26758837
        Bug: 31558194,  fix_controls: 31558194
        Bug: 30781970,  fix_controls: 30781970
        Bug: 30142527,  fix_controls: 30142527
        Bug: 31143146,  fix_controls: 31143146
        Bug: 31961578,  fix_controls: 31961578
        Bug: 31496840,  fix_controls: 31496840
        Bug: 22387320,  fix_controls: 22387320
    
    PL/SQL procedure successfully completed.

    The list is much shorter than in 19.11.0.

    I will enable them:

    SQL> execute dbms_optim_bundle.enable_optim_fixes('ON','BOTH', 'YES')

    Afterwards I have this entry in my SPFILE:

    *._fix_control='29331066:1','28965084:1','28776811:1','28498976:1','28567417:1','28558645:1','29132869:1','29450812:1','29687220:1','29939400:1','30232638:1','30001331:0','29304314:1','29930457:1','30028663:1','28144569:1','28776431:1','27261477:1','31069997:1','31077481:1','28602253:1','29653132:0','29937655:1','30347410:1','30602828:1','30896685:0','29487407:1','30998035:1','30786641:1','31444353:0','30486896:1','28999046:1','30902655:1','30681521:1','29302565:1','30972817:1','30222669:1','31668694:1','31001490:1','30198239:7','30980115:1','30616738:0','31895670:0','19138896:1','31670824:0','9876287:1','30564898:1','32075777:0','30570982:1','32037237:1','30927440:1','30822446:1','24561942:1','31625959:1','31579233:1','29696242:1','31626438:1','30228422:1','17295505:1','29725425:1','30618230:1','30008456:1','30537403:1','30235878:1','30646077:1','29657973:1','29712727:1','20922160:1','30006705:1','29463553:1','30751171:1','31009032:1','30063629:1','30207519:1','31517502:1','30617002:1','30483217:1','30235691:1','30568514:1','28414968:3','32014520:1','30249927:1','31580374:1','29590666:0','29435966:1','28173995:1','29867728:1','30776676:1','26577716:1','30470947:1','30979701:1','30483184:1','31001295:1','31191224:1','31974424:1','29385774:1','28234255:3','31459242:0','31082719:1','28708585:1','31821701:1','32107621:1','26758837:1','31558194:1','30781970:0','30142527:1','31143146:1','31961578:0','31496840:1','22387320:1'#added through dbms_optim_bundle package
  5. Additional Patches
    In addition, you may please consider the patches mentioned on top of 19.12.0 in the MOS Notes I listed here:
    Some very important MOS Notes when you upgrade and patch including MOS Note:555.1.

Applying RU 12.2.0.1.210720 to my 12.2.0.1 home

I won’t copy/paste all steps as the output is similar to the above. As I wrote above, I had to exchange my opatch with a newer one.

  1. Patch conflict check
    $ORACLE_HOME/OPatch/opatch prereq CheckConflictAgainstOHWithDetail -ph ./
  2. opatch apply
    $ORACLE_HOME/OPatch/opatch apply
  3. datapatch
    $ORACLE_HOME/OPatch/datapatch -verbose

All steps worked fine and flawless.

But there may be a bit more to do in addition.

Further Links and Information

–Mike

Share this: