The new AutoUpgrade utility in Oracle 19c

The new AutoUpgrade utility in Oracle 19cOracle Database 19c on premises is available – and one of the key features is the new AutoUpgrade utility. In the following days I will publish several blog posts explaining and showcasing the new AutoUpgrade.

What is the AutoUpgrade?

The Oracle Database AutoUpgrade utility is a new tiny little command line tool which allows you to upgrade your databases in an unattended way. I call it the Hands-Free Upgrade. TThe new AutoUpgrade utility in Oracle 19che idea of the tool is to run the prechecks against multiple databases, fix 99% of the potential issues, set a restore point in case something goes wrong – and then upgrade your databases. And of course, do the postupgrade, recompilation and time zone adjustment.

The only thing you need to provide is a config file in text format.

Where do you get the AutoUpgrade?

You get it when you install Oracle Database 19c (19.3) or newer. Or – and this is the recommended source, you download the most recent version from MOS Note: 2485457.1 – AutoUpgrade Tool:

The new AutoUpgrade utility in Oracle 19c

Where do you find the AutoUpgrade documentation?

It is all here included in the Oracle Database 19c Upgrade Guide:

Which database releases are supported?

As source, the minimum version is Oracle Database 11.2.0.4. Fair enough, isn’t it?

And as target (and yes, we heard your feedback at OOW18) we will support upgrade to:

  • Oracle Database 19.3.0 and newer
  • Oracle Database 18.5.0 and newer
  • Oracle Database 12.2.0.1 with Jan 2019 RU and newer

More Information

Within the next days, I will blog more about the AutoUpgrade and add the links here. If you’d like to get an introduction, please see our OOW 2018 presentation together with Daniel Overby Hansen from SimCorp.

–Mike

Share this:

31 thoughts on “The new AutoUpgrade utility in Oracle 19c

  1. Does AutoUpgrade utility handle RAC database too? I don’t see any reference where it say’s it can handle RAC or otherwise.

    • Yes, it does – when you set CLUSTER_DATABASE=FALSE (the documentation tells you about it).
      What we don’t do at the moment: We don’t do the SRVCTL configuration after the upgrade as the DBUA did in the past.
      This is an enhancement we are working on right now.

      Cheers,
      Mike

  2. I have one doubt, if I have version 11.2.0.4 the autoupgrade updates to version 18.3 and then I have to install the patch to be at 18.5?
    Otherwise it is not clear to me how you do to update to version 18.5
    I have to put the patch in some directory to install it?
    thanks a lot!

    • Hi Daniel,

      you need to install (aka Patch) to 18.5.0 or 18.6.0 at first.
      And then you download the AutoUpgrade from the MOS note. If you try it with 18.3.0 for instance it will fail.

      Cheers,
      Mike

      • Mike,

        Support is telling me that the source has to be 18.5+. That doesn’t align with what you have said about 11.2.0.4 as the source. I think that they really mean for the target. That being said, autoupgrade can not determine if you are running Standard Edition on 11.2.0.4 due to Standard not being in v$version.

        Mark

        • Mark,

          please send me the SR number and I will explain the support people.

          I’m right – they are wrong. Unfortunately …

          Source can be as low as 11.2.0.4

          Upgrade target can be: 12.2.0.1 with Jan 2019 RU or newer
          18.5.0 or newer
          19.3.0 or newer

          Trust me – we developed it!

          Cheers,
          Mike

  3. Hi Mike,

    I did a fresh install of a 19c home but invoking autoupgrade.jar fails with:
    Exception in thread “main” java.lang.ExceptionInInitializerError
    at oracle.upgrade.autoupgrade.boot.AutoUpgMain.(AutoUpgMain.java:114)
    at oracle.upgrade.autoupgrade.boot.AutoUpgMain.newInstance(AutoUpgMain.java:142)
    at oracle.upgrade.autoupgrade.boot.Boot.main(Boot.java:39)
    Caused by: java.lang.NullPointerException
    at java.io.Reader.(Reader.java:78)
    at java.io.InputStreamReader.(InputStreamReader.java:113)
    at oracle.upgrade.commons.lang.LangSettings.resourceBundleFromFile(LangSettings.java:213)
    at oracle.upgrade.commons.lang.LangSettings.loadLanguageInfo(LangSettings.java:71)
    at oracle.upgrade.commons.lang.LangSettings.(LangSettings.java:64)
    at oracle.upgrade.commons.lang.LangSettings.(LangSettings.java:54)
    at oracle.upgrade.commons.context.AppContext.(AppContext.java:47)

    JDK is
    java version “1.8.0_201”
    Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
    Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)

    This happens with the included jar as well as a fresh download from MoS. What am I doing wrong?

      • Hi Mike,

        thank you for taking the time to look into this. Of course I knew what I did so I expected the same from you 😉
        I did the following:
        $ORACLE_HOME/jdk/bin/java -version
        java version “1.8.0_201”
        Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
        Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)

        $ORACLE_HOME/jdk/bin/java -jar $ORACLE_HOME/rdbms/admin/autoupgrade.jar -version
        Exception in thread “main” java.lang.ExceptionInInitializerError
        at oracle.upgrade.autoupgrade.boot.AutoUpgMain.(AutoUpgMain.java:114)
        at oracle.upgrade.autoupgrade.boot.AutoUpgMain.newInstance(AutoUpgMain.java:142)
        at oracle.upgrade.autoupgrade.boot.Boot.main(Boot.java:39)
        Caused by: java.lang.NullPointerException
        at java.io.Reader.(Reader.java:78)
        at java.io.InputStreamReader.(InputStreamReader.java:113)
        at oracle.upgrade.commons.lang.LangSettings.resourceBundleFromFile(LangSettings.java:213)
        at oracle.upgrade.commons.lang.LangSettings.loadLanguageInfo(LangSettings.java:71)
        at oracle.upgrade.commons.lang.LangSettings.(LangSettings.java:64)
        at oracle.upgrade.commons.lang.LangSettings.(LangSettings.java:54)
        at oracle.upgrade.commons.context.AppContext.(AppContext.java:47)

        Oracle Home is a freshly installed 19c.

        Cheers,
        Holger

    • Hi Holger,
      we hit the same issue on SPARC Solaris x64 today. Please check
      # locale
      If it’s not saying like
      LC_NUMERIC = “C”
      but something like
      LC_NUMERIC = de_DE.UTF-8
      then do a
      # export LC_ALL=C
      and try again.
      BTW, this issue comes to us after an Solaris Update. We didn’t recognized that until we did a RAC upgrade and fired a rootupgrade.sh script with wrong locale settings. This broke us the whole cluster. 🙁
      Well, lessons learned. Always check “locale”.
      Regards
      Kay.

          • I’m glad I could help.
            But please keep in mind that this issue will hit you when doing things like RAC installation or upgrade and running root.sh or rootupgrade.sh. With wrong LOCALE settings this will fail and any attempt to fix that (e.g. with a downgrade) will cause the cluster to be left in an inconsistant state. It can help to set LC_ALL=C and try root.sh or rootupgrade.sh again.
            Mike, if any of your flights is delayed again and you don’t know what to do… try this out — it’s really ugly. 😉

  4. Hi Mike,
    playing around with autoupgrade.jar I have some clarification for the documentation:
    The dbname parameter in the config file does not reflect the dbname parameter of the database, but the db_unique_name. Needed some time with trial and error to find out what is meant here. 😉
    But … cool tool! 🙂
    Regards
    Kay.

  5. Hi Mike!

    I got this error on 2 different servers when trying to upgrade using autoupgrade.jar (build.version 20190513).

    Source 12.2.0.1 (Including Golden Gate)
    Target 19.3.0.0

    Debug output:
    “Database check with running exception” (conName=”PDB$SEED”,stage=”PRECHECKS”,checkName=”STREAMS_SETUP”)

    Iogfile output:
    ERROR The dispatcher has failed due to: AutoUpgException [UPG-1316] – AutoUpgDispatcher.runDispatcher
    oracle.upgrade.autoupgrade.utils.errors.AutoUpgException: AutoUpgException [UPG-1316]

    Any Idea what this means?

    Regards

    /Jocke

    • Jocke,

      I think this may have to do with Streams in your database. There may be either leftovers or objects. As we remove Streams the alarm detects it.
      Can you please use the preupgrade.jar from 884522.1 and check it manually with “perl preupgrade.jar TEXT TERMINAL”?

      The logs of the current autoupgrade will tell you as well – go to your log directory you defined (for instance, /home/oracle/log), then to .///preupgrade and check the preupgrade.log or the HTML file in the same dir.

      Cheers,
      Mike

      Thanks,
      Mike

      • Hi again Mike!

        Thx for taking you time and answering our questions, your blog is very helpful to all of us users.

        I did as you suggested, downloaded the latest preupgrade.jar file and run the catcon.pl to do the preupgrade fixes.
        The auotupgrade went fine until the last postupgrade step where this error showed up. Although everything seems to be fine with the databases afterwards.

        Regards

        /Jocke

        Tail from the dbupgrade_20190524_user.log:

        |CONTAINER| PERCENTAGE|
        +———+—————————————–+
        | CDB$ROOT|SUCCESSFULLY UPGRADED [int904c1-cdb$root]|
        | PDB$SEED|SUCCESSFULLY UPGRADED [int904c1-pdb$seed]|
        | RSA01INT|SUCCESSFULLY UPGRADED [int904c1-rsa01int]|
        +———+—————————————–+
        2019-05-24 13:45:58.805 INFO SUCCESSFULLY UPGRADED [int904c1]
        2019-05-24 13:45:58.805 INFO End Upgrade on Database [int904c1]
        2019-05-24 13:45:58.807 INFO SUCCESSFULLY UPGRADED [int904c1]
        2019-05-24 13:45:58.827 INFO int904c1 Return status is SUCCESS
        2019-05-24 13:48:00.971 INFO Analyzing INT904C1, 18 checks will run using 12 threads
        2019-05-24 13:48:07.211 INFO Using /orabck/DB_ADM/19c_logs/104/postchecks/int904c1_checklist.cfg as reference to determine the fixups which will be executed
        2019-05-24 14:03:41.835 INFO Analyzing INT904C1, 18 checks will run using 12 threads
        2019-05-24 14:04:37.364 ERROR [Unexpected Exception Error]
        2019-05-24 14:04:37.366 INFO Starting error management routine
        2019-05-24 14:04:37.366 INFO Ended error management routine

        Tail from the autoupgrade_err.log:

        2019-05-24 14:04:37.364 ERROR [Unexpected Exception Error]
        2019-05-24 14:04:37.366 INFO Starting error management routine
        2019-05-24 14:04:37.366 INFO Ended error management routine
        labls1175_oracle% cat autoupgrade_err.log
        2019-05-24 14:04:37.363 ERROR The dispatcher has failed due to: Index: 0, Size: 0 – AutoUpgDispatcher.runDispatcher
        java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
        at java.util.ArrayList.rangeCheck(ArrayList.java:657)
        at java.util.ArrayList.get(ArrayList.java:433)
        at oracle.upgrade.commons.sql.ExecuteSql.doSqlCmds(ExecuteSql.java:626)
        at oracle.upgrade.autoupgrade.utils.schema.Database.restartDB(Database.java:1264)
        at oracle.upgrade.autoupgrade.utils.schema.Database.restartDB(Database.java:1226)
        at oracle.upgrade.autoupgrade.dispatcher.helper.DeployExecuteHelper.executeStages(DeployExecuteHelper.java:409)
        at oracle.upgrade.autoupgrade.dispatcher.strategy.strategies.DeployDBS.exec(DeployDBS.java:165)
        at oracle.upgrade.autoupgrade.dispatcher.strategy.strategies.Context.executeCommand(Context.java:45)
        at oracle.upgrade.autoupgrade.dispatcher.helper.DispatcherExecuteContext.callExecuteContext(DispatcherExecuteContext.java:122)
        at oracle.upgrade.autoupgrade.dispatcher.helper.DispatcherExecuteContext.executeDispatcher(DispatcherExecuteContext.java:110)
        at oracle.upgrade.autoupgrade.dispatcher.AutoUpgDispatcher.runDispatcher(AutoUpgDispatcher.java:314)
        at oracle.upgrade.autoupgrade.dispatcher.AutoUpgDispatcher.run(AutoUpgDispatcher.java:181)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

        2019-05-24 14:04:37.364 ERROR [Unexpected Exception Error] – AutoUpgDispatcher.run
        oracle.upgrade.autoupgrade.utils.errors.AutoUpgException: AutoUpgException [UPG-1601#The dispatcher has failed due to: Index: 0, Size: 0]
        at oracle.upgrade.autoupgrade.dispatcher.AutoUpgDispatcher.runDispatcher(AutoUpgDispatcher.java:324)
        at oracle.upgrade.autoupgrade.dispatcher.AutoUpgDispatcher.run(AutoUpgDispatcher.java:181)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        at java.lang.Thread.run(Thread.java:748)

        2019-05-24 14:04:37.367 ERROR The dispatcher has failed due to:
        ERROR UPG-1601
        The dispatcher has failed due to: Index: 0, Size: 0
        Cause: An error occurred while trying to read the error properties file

        • Hi Mike,

          I’m seeing the same errors after successfull upgrade from 12.2.0.1 to 19.3. Can you confirm that the upgrade process finished at this time, or are there any relevant steps missing?

          build.version:20190513
          build.date:2019/05/13 16:59:48
          build.label:RDBMS_PT.AUTOUPGRADE_LINUX.X64_190513.1404

          regards,
          Erwin

          — autoupgrade_20190617_user.log
          2019-06-17 14:09:59.966 INFO Copying/merging file /u01/app/oracle/product/19.3.0/dbhome_1/network/admin/tnsnames.ora.tmp ended
          2019-06-17 14:09:59.966 INFO Copying/merging file sqlnet.ora ended
          2019-06-17 14:09:59.970 INFO File [/u01/app/oracle/product/19.3.0/dbhome_1/dbs/orapwauredt] already exists.
          2019-06-17 14:09:59.979 INFO Return status is SUCCESS
          2019-06-17 14:09:59.985 INFO Update of oratab [AUREDT]
          [/etc/oratab] [SUCCESS] [None]

          Network Files [AUREDT]
          [/u01/app/oracle/product/19.3.0/dbhome_1/network/admin/tnsnames.ora] [SUCCESS] [None]
          [/u01/app/oracle/product/19.3.0/dbhome_1/network/admin/listener.ora] [SUCCESS] [None]
          [/u01/app/oracle/product/19.3.0/dbhome_1/network/admin/sqlnet.ora] [SUCCESS] [None]
          Copy of password file [AUREDT]
          [/u01/app/oracle/product/19.3.0/dbhome_1/dbs/orapwauredt] [SUCCESS] [None]
          Database State
          Resetting the database’s state: [SUCCESS] [None]
          2019-06-17 14:10:41.168 INFO Starting error management routine
          2019-06-17 14:10:41.169 INFO Ended error management routine
          2019-06-17 14:10:41.185 ERROR Error occurred while running the dispatcher for job 100
          Cause: An error occurred while trying to read the error properties file

          — autoupgrade_err.log
          2019-06-17 14:10:41.167 ERROR The dispatcher has failed due to: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 – AutoUpgDispatcher.runDispatcher
          java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
          at java.util.ArrayList.rangeCheck(ArrayList.java:653)
          at java.util.ArrayList.get(ArrayList.java:429)
          at oracle.upgrade.commons.sql.ExecuteSql.doSqlCmds(ExecuteSql.java:699)
          at oracle.upgrade.autoupgrade.utils.schema.Database.restartDB(Database.java:1352)
          at oracle.upgrade.autoupgrade.utils.schema.Database.restartDB(Database.java:1314)
          at oracle.upgrade.autoupgrade.dispatcher.helper.DeployExecuteHelper.executeStages(DeployExecuteHelper.java:413)
          at oracle.upgrade.autoupgrade.dispatcher.strategy.strategies.DeployDBS.exec(DeployDBS.java:180)
          at oracle.upgrade.autoupgrade.dispatcher.strategy.strategies.Context.executeCommand(Context.java:45)
          at oracle.upgrade.autoupgrade.dispatcher.helper.DispatcherExecuteContext.callExecuteContext(DispatcherExecuteContext.java:116)
          at oracle.upgrade.autoupgrade.dispatcher.helper.DispatcherExecuteContext.executeDispatcher(DispatcherExecuteContext.java:105)
          at oracle.upgrade.autoupgrade.dispatcher.AutoUpgDispatcher.runDispatcher(AutoUpgDispatcher.java:405)
          at oracle.upgrade.autoupgrade.dispatcher.AutoUpgDispatcher.run(AutoUpgDispatcher.java:254)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.run(FutureTask.java:266)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at java.lang.Thread.run(Thread.java:745)

          2019-06-17 14:10:41.168 INFO Starting error management routine – ErrorParser.getError
          2019-06-17 14:10:41.169 INFO Ended error management routine – ErrorParser.getError
          2019-06-17 14:10:41.185 ERROR Error occurred while running the dispatcher for job 100
          Cause: An error occurred while trying to read the error properties file – AutoUpgDispatcher.run
          2019-06-17 14:10:41.186 ERROR The dispatcher has failed due to:
          Error: UPG-1601
          The dispatcher has failed due to: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0]
          Cause: An error occurred while trying to read the error properties file

          • Hi Erwin,

            I think this is a known issue of the current tool. The next version should fix this.
            If the database shows everything VALID afterwards, all is fine.

            Cheers,
            Mike

  6. Mike,
    We are seeing the below error using analyze stage of autoupgrade :

    Ensure there is additional disk space in LOG_ARCHIVE_DEST_1 for at least 3291480064 of archived logs. Check alert log during the upgrade that there is no write error to the destination due to lack of disk space.

    The database has archiving enabled. The upgrade process will need free disk space in the archive log destination(s) to generate archived logs to.

    Archiving cannot proceed if the archive log destination is full during upgrade.

    Archive Log Destination:
    Parameter : LOG_ARCHIVE_DEST_1
    Destination : +LINUX_MIGRATION_FRA

    +LINUX_MIGRATION_FRA shows 150G free :

    ASMCMD> lsdg LINUX_MIGRATION_FRA
    State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
    MOUNTED EXTERN N 512 512 4096 1048576 204801 152269 0 152269

    Please advise

    • Hi Sunil,

      please do me a favor and open an SR. Upload the ENTIRE logging directory as you defined it in your config.cfg file.
      Then share the SR number with me. My team mates will have a look at it.

      Cheers,
      Mike

  7. Mike,
    Thanks.

    I have updated the SR.
    Looks like the issue is Alternate Log_archive_dest settings:
    Autoupgrade Analyze is failing with errors for settings:

    log_archive_dest_1 string LOCATION=+LINUX_MIGRATION_FRA MAX_FAILURE=1 REOPEN=10
    ALTERNATE=LOG_ARCHIVE_DEST_2

    log_archive_dest_2 string LOCATION=+LINUX_MIGRATION_DG

    But no errors if settings are:
    log_archive_dest_1 string LOCATION=+LINUX_MIGRATION_FRA MAX_FAILURE=1 REOPEN=10
    log_archive_dest_2 string

    Still verifying with other options. can you please confirm this

    Thanks
    Sunil

  8. Mike,

    I found that Restore point was not dropped after the upgrade. Does that mean the upgrade is not completed successfully?

    select * from v$restore_point;

    SCN DATABASE_INCARNATION# GUA STORAGE_SIZE TIME
    ———- ——————— — ———— —————————————————————————
    RESTORE_POINT_TIME PRE
    ————————————————————————— —
    NAME PDB CLE
    ——————————————————————————————————————————– — —
    PDB_INCARNATION# CON_ID
    —————- ———-
    1.5459E+13 4 YES 3670016000 13-JUN-19 04.01.43.000000000 PM
    YES
    AUTOUPGRADE_221145114461854_DDRMAN NO NO
    0 0

    Please advise

    • Hi Sunil,

      the newer version – uploaded already – has this enhancement:

      AUPG-779 Add option to drop Grp after upgrade

      I’m working on the explanation blog posts.

      At the moment it needs to be dropped manually.

      Cheers,
      Mike

    • Hi Christoph,

      it works – but you need to disable the Broker and defer the log transport manually at the moment.
      We are working on the inclusion of this as well.

      Cheers,
      Mike

Leave a Reply

Your email address will not be published. Required fields are marked *

* Checkbox to comply with GDPR is required

*

I agree

This site uses Akismet to reduce spam. Learn how your comment data is processed.