Oracle Database 19c on premises is available – and one of the key features is the new AutoUpgrade utility. In the following days I will publish several blog posts explaining and showcasing the new AutoUpgrade.
What is the AutoUpgrade?
The Oracle Database AutoUpgrade utility is a new tiny little command line tool which allows you to upgrade your databases in an unattended way. I call it the Hands-Free Upgrade. T
he idea of the tool is to run the prechecks against multiple databases, fix 99% of the potential issues, set a restore point in case something goes wrong – and then upgrade your databases. And of course, do the postupgrade, recompilation and time zone adjustment.
The only thing you need to provide is a config file in text format.
Where do you get the AutoUpgrade?
You get it when you install Oracle Database 19c (19.3) or newer. Or – and this is the recommended source, you download the most recent version from MOS Note: 2485457.1 – AutoUpgrade Tool:
Where do you find the AutoUpgrade documentation?
It is all here included in the Oracle Database 19c Upgrade Guide:
Which database releases are supported?
As source, the minimum version is Oracle Database 11.2.0.4. Fair enough, isn’t it?
And as target (and yes, we heard your feedback at OOW18) we will support upgrade to:
- Oracle Database 19.3.0 and newer
- Oracle Database 18.5.0 and newer
- Oracle Database 12.2.0.1 with Jan 2019 RU and newer
More Information
Within the next days, I will blog more about the AutoUpgrade and add the links here. If you’d like to get an introduction, please see our OOW 2018 presentation together with Daniel Overby Hansen from SimCorp.
AutoUpgrade – Step-by-step
- The new AutoUpgrade Utility – Download, documentation and supported versions
- Create and adjust the config file for AutoUpgrade
- Config file for AutoUpgrade – Advanced options
- Config file for AutoUpgrade – Tweaking init parameters
- AutoUpgrade: ANALYZE, FIXUPS, UPGRADE and DEPLOY modes
- AutoUpgrade: Where do you find all the logfiles?
- UPG: The AutoUpgrade Command Line Interface
- Upgrading Multitenant databases with AutoUpgrade
- Moving to a new server with AutoUpgrade
- How to tweak the hidden settings in AutoUpgrade
- AutoUpgrade and Data Guard, RAC, Restart and non-CDB to PDB
- AutoUpgrade and Wallets
–Mike
Does AutoUpgrade utility handle RAC database too? I don’t see any reference where it say’s it can handle RAC or otherwise.
Yes, it does – when you set CLUSTER_DATABASE=FALSE (the documentation tells you about it).
What we don’t do at the moment: We don’t do the SRVCTL configuration after the upgrade as the DBUA did in the past.
This is an enhancement we are working on right now.
Cheers,
Mike
I have one doubt, if I have version 11.2.0.4 the autoupgrade updates to version 18.3 and then I have to install the patch to be at 18.5?
Otherwise it is not clear to me how you do to update to version 18.5
I have to put the patch in some directory to install it?
thanks a lot!
Hi Daniel,
you need to install (aka Patch) to 18.5.0 or 18.6.0 at first.
And then you download the AutoUpgrade from the MOS note. If you try it with 18.3.0 for instance it will fail.
Cheers,
Mike
thanks a lot mike!!!!
Mike,
Support is telling me that the source has to be 18.5+. That doesn’t align with what you have said about 11.2.0.4 as the source. I think that they really mean for the target. That being said, autoupgrade can not determine if you are running Standard Edition on 11.2.0.4 due to Standard not being in v$version.
Mark
Mark,
please send me the SR number and I will explain the support people.
I’m right – they are wrong. Unfortunately …
Source can be as low as 11.2.0.4
Upgrade target can be: 12.2.0.1 with Jan 2019 RU or newer
18.5.0 or newer
19.3.0 or newer
Trust me – we developed it!
Cheers,
Mike
Hi Mike,
I did a fresh install of a 19c home but invoking autoupgrade.jar fails with:
Exception in thread “main” java.lang.ExceptionInInitializerError
at oracle.upgrade.autoupgrade.boot.AutoUpgMain.(AutoUpgMain.java:114)
at oracle.upgrade.autoupgrade.boot.AutoUpgMain.newInstance(AutoUpgMain.java:142)
at oracle.upgrade.autoupgrade.boot.Boot.main(Boot.java:39)
Caused by: java.lang.NullPointerException
at java.io.Reader.(Reader.java:78)
at java.io.InputStreamReader.(InputStreamReader.java:113)
at oracle.upgrade.commons.lang.LangSettings.resourceBundleFromFile(LangSettings.java:213)
at oracle.upgrade.commons.lang.LangSettings.loadLanguageInfo(LangSettings.java:71)
at oracle.upgrade.commons.lang.LangSettings.(LangSettings.java:64)
at oracle.upgrade.commons.lang.LangSettings.(LangSettings.java:54)
at oracle.upgrade.commons.context.AppContext.(AppContext.java:47)
JDK is
java version “1.8.0_201”
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)
This happens with the included jar as well as a fresh download from MoS. What am I doing wrong?
Holger,
I can’t tell you because I don’t know what you are doing.
Can you please copy/paste the call you are using?
Thanks!
Mike
Hi Mike,
thank you for taking the time to look into this. Of course I knew what I did so I expected the same from you 😉
I did the following:
$ORACLE_HOME/jdk/bin/java -version
java version “1.8.0_201”
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)
$ORACLE_HOME/jdk/bin/java -jar $ORACLE_HOME/rdbms/admin/autoupgrade.jar -version
Exception in thread “main” java.lang.ExceptionInInitializerError
at oracle.upgrade.autoupgrade.boot.AutoUpgMain.(AutoUpgMain.java:114)
at oracle.upgrade.autoupgrade.boot.AutoUpgMain.newInstance(AutoUpgMain.java:142)
at oracle.upgrade.autoupgrade.boot.Boot.main(Boot.java:39)
Caused by: java.lang.NullPointerException
at java.io.Reader.(Reader.java:78)
at java.io.InputStreamReader.(InputStreamReader.java:113)
at oracle.upgrade.commons.lang.LangSettings.resourceBundleFromFile(LangSettings.java:213)
at oracle.upgrade.commons.lang.LangSettings.loadLanguageInfo(LangSettings.java:71)
at oracle.upgrade.commons.lang.LangSettings.(LangSettings.java:64)
at oracle.upgrade.commons.lang.LangSettings.(LangSettings.java:54)
at oracle.upgrade.commons.context.AppContext.(AppContext.java:47)
Oracle Home is a freshly installed 19c.
Cheers,
Holger
Hi Holger,
we hit the same issue on SPARC Solaris x64 today. Please check
# locale
If it’s not saying like
LC_NUMERIC = “C”
but something like
LC_NUMERIC = de_DE.UTF-8
then do a
# export LC_ALL=C
and try again.
BTW, this issue comes to us after an Solaris Update. We didn’t recognized that until we did a RAC upgrade and fired a rootupgrade.sh script with wrong locale settings. This broke us the whole cluster. 🙁
Well, lessons learned. Always check “locale”.
Regards
Kay.
Hi KAy,
perfect! That’s what caused the error message.
Thanks a lot.
Holger
Thanks Kay – happy and glad to see that you help each other!!!
Cheers,
Mike
I’m glad I could help.
But please keep in mind that this issue will hit you when doing things like RAC installation or upgrade and running root.sh or rootupgrade.sh. With wrong LOCALE settings this will fail and any attempt to fix that (e.g. with a downgrade) will cause the cluster to be left in an inconsistant state. It can help to set LC_ALL=C and try root.sh or rootupgrade.sh again.
Mike, if any of your flights is delayed again and you don’t know what to do… try this out — it’s really ugly. 😉
Hi Mike,
playing around with autoupgrade.jar I have some clarification for the documentation:
The dbname parameter in the config file does not reflect the dbname parameter of the database, but the db_unique_name. Needed some time with trial and error to find out what is meant here. 😉
But … cool tool! 🙂
Regards
Kay.
Thanks a lot, Kay!
I sent it to the team for clarification!
Cheers,
Mike
Hi Mike!
I got this error on 2 different servers when trying to upgrade using autoupgrade.jar (build.version 20190513).
Source 12.2.0.1 (Including Golden Gate)
Target 19.3.0.0
Debug output:
“Database check with running exception” (conName=”PDB$SEED”,stage=”PRECHECKS”,checkName=”STREAMS_SETUP”)
Iogfile output:
ERROR The dispatcher has failed due to: AutoUpgException [UPG-1316] – AutoUpgDispatcher.runDispatcher
oracle.upgrade.autoupgrade.utils.errors.AutoUpgException: AutoUpgException [UPG-1316]
Any Idea what this means?
Regards
/Jocke
Jocke,
I think this may have to do with Streams in your database. There may be either leftovers or objects. As we remove Streams the alarm detects it.
Can you please use the preupgrade.jar from 884522.1 and check it manually with “perl preupgrade.jar TEXT TERMINAL”?
The logs of the current autoupgrade will tell you as well – go to your log directory you defined (for instance, /home/oracle/log), then to .///preupgrade and check the preupgrade.log or the HTML file in the same dir.
Cheers,
Mike
Thanks,
Mike
Hi again Mike!
Thx for taking you time and answering our questions, your blog is very helpful to all of us users.
I did as you suggested, downloaded the latest preupgrade.jar file and run the catcon.pl to do the preupgrade fixes.
The auotupgrade went fine until the last postupgrade step where this error showed up. Although everything seems to be fine with the databases afterwards.
Regards
/Jocke
Tail from the dbupgrade_20190524_user.log:
|CONTAINER| PERCENTAGE|
+———+—————————————–+
| CDB$ROOT|SUCCESSFULLY UPGRADED [int904c1-cdb$root]|
| PDB$SEED|SUCCESSFULLY UPGRADED [int904c1-pdb$seed]|
| RSA01INT|SUCCESSFULLY UPGRADED [int904c1-rsa01int]|
+———+—————————————–+
2019-05-24 13:45:58.805 INFO SUCCESSFULLY UPGRADED [int904c1]
2019-05-24 13:45:58.805 INFO End Upgrade on Database [int904c1]
2019-05-24 13:45:58.807 INFO SUCCESSFULLY UPGRADED [int904c1]
2019-05-24 13:45:58.827 INFO int904c1 Return status is SUCCESS
2019-05-24 13:48:00.971 INFO Analyzing INT904C1, 18 checks will run using 12 threads
2019-05-24 13:48:07.211 INFO Using /orabck/DB_ADM/19c_logs/104/postchecks/int904c1_checklist.cfg as reference to determine the fixups which will be executed
2019-05-24 14:03:41.835 INFO Analyzing INT904C1, 18 checks will run using 12 threads
2019-05-24 14:04:37.364 ERROR [Unexpected Exception Error]
2019-05-24 14:04:37.366 INFO Starting error management routine
2019-05-24 14:04:37.366 INFO Ended error management routine
Tail from the autoupgrade_err.log:
2019-05-24 14:04:37.364 ERROR [Unexpected Exception Error]
2019-05-24 14:04:37.366 INFO Starting error management routine
2019-05-24 14:04:37.366 INFO Ended error management routine
labls1175_oracle% cat autoupgrade_err.log
2019-05-24 14:04:37.363 ERROR The dispatcher has failed due to: Index: 0, Size: 0 – AutoUpgDispatcher.runDispatcher
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:657)
at java.util.ArrayList.get(ArrayList.java:433)
at oracle.upgrade.commons.sql.ExecuteSql.doSqlCmds(ExecuteSql.java:626)
at oracle.upgrade.autoupgrade.utils.schema.Database.restartDB(Database.java:1264)
at oracle.upgrade.autoupgrade.utils.schema.Database.restartDB(Database.java:1226)
at oracle.upgrade.autoupgrade.dispatcher.helper.DeployExecuteHelper.executeStages(DeployExecuteHelper.java:409)
at oracle.upgrade.autoupgrade.dispatcher.strategy.strategies.DeployDBS.exec(DeployDBS.java:165)
at oracle.upgrade.autoupgrade.dispatcher.strategy.strategies.Context.executeCommand(Context.java:45)
at oracle.upgrade.autoupgrade.dispatcher.helper.DispatcherExecuteContext.callExecuteContext(DispatcherExecuteContext.java:122)
at oracle.upgrade.autoupgrade.dispatcher.helper.DispatcherExecuteContext.executeDispatcher(DispatcherExecuteContext.java:110)
at oracle.upgrade.autoupgrade.dispatcher.AutoUpgDispatcher.runDispatcher(AutoUpgDispatcher.java:314)
at oracle.upgrade.autoupgrade.dispatcher.AutoUpgDispatcher.run(AutoUpgDispatcher.java:181)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-05-24 14:04:37.364 ERROR [Unexpected Exception Error] – AutoUpgDispatcher.run
oracle.upgrade.autoupgrade.utils.errors.AutoUpgException: AutoUpgException [UPG-1601#The dispatcher has failed due to: Index: 0, Size: 0]
at oracle.upgrade.autoupgrade.dispatcher.AutoUpgDispatcher.runDispatcher(AutoUpgDispatcher.java:324)
at oracle.upgrade.autoupgrade.dispatcher.AutoUpgDispatcher.run(AutoUpgDispatcher.java:181)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
2019-05-24 14:04:37.367 ERROR The dispatcher has failed due to:
ERROR UPG-1601
The dispatcher has failed due to: Index: 0, Size: 0
Cause: An error occurred while trying to read the error properties file
Hi Mike,
I’m seeing the same errors after successfull upgrade from 12.2.0.1 to 19.3. Can you confirm that the upgrade process finished at this time, or are there any relevant steps missing?
build.version:20190513
build.date:2019/05/13 16:59:48
build.label:RDBMS_PT.AUTOUPGRADE_LINUX.X64_190513.1404
regards,
Erwin
— autoupgrade_20190617_user.log
2019-06-17 14:09:59.966 INFO Copying/merging file /u01/app/oracle/product/19.3.0/dbhome_1/network/admin/tnsnames.ora.tmp ended
2019-06-17 14:09:59.966 INFO Copying/merging file sqlnet.ora ended
2019-06-17 14:09:59.970 INFO File [/u01/app/oracle/product/19.3.0/dbhome_1/dbs/orapwauredt] already exists.
2019-06-17 14:09:59.979 INFO Return status is SUCCESS
2019-06-17 14:09:59.985 INFO Update of oratab [AUREDT]
[/etc/oratab] [SUCCESS] [None]
Network Files [AUREDT]
[/u01/app/oracle/product/19.3.0/dbhome_1/network/admin/tnsnames.ora] [SUCCESS] [None]
[/u01/app/oracle/product/19.3.0/dbhome_1/network/admin/listener.ora] [SUCCESS] [None]
[/u01/app/oracle/product/19.3.0/dbhome_1/network/admin/sqlnet.ora] [SUCCESS] [None]
Copy of password file [AUREDT]
[/u01/app/oracle/product/19.3.0/dbhome_1/dbs/orapwauredt] [SUCCESS] [None]
Database State
Resetting the database’s state: [SUCCESS] [None]
2019-06-17 14:10:41.168 INFO Starting error management routine
2019-06-17 14:10:41.169 INFO Ended error management routine
2019-06-17 14:10:41.185 ERROR Error occurred while running the dispatcher for job 100
Cause: An error occurred while trying to read the error properties file
— autoupgrade_err.log
2019-06-17 14:10:41.167 ERROR The dispatcher has failed due to: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 – AutoUpgDispatcher.runDispatcher
java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:653)
at java.util.ArrayList.get(ArrayList.java:429)
at oracle.upgrade.commons.sql.ExecuteSql.doSqlCmds(ExecuteSql.java:699)
at oracle.upgrade.autoupgrade.utils.schema.Database.restartDB(Database.java:1352)
at oracle.upgrade.autoupgrade.utils.schema.Database.restartDB(Database.java:1314)
at oracle.upgrade.autoupgrade.dispatcher.helper.DeployExecuteHelper.executeStages(DeployExecuteHelper.java:413)
at oracle.upgrade.autoupgrade.dispatcher.strategy.strategies.DeployDBS.exec(DeployDBS.java:180)
at oracle.upgrade.autoupgrade.dispatcher.strategy.strategies.Context.executeCommand(Context.java:45)
at oracle.upgrade.autoupgrade.dispatcher.helper.DispatcherExecuteContext.callExecuteContext(DispatcherExecuteContext.java:116)
at oracle.upgrade.autoupgrade.dispatcher.helper.DispatcherExecuteContext.executeDispatcher(DispatcherExecuteContext.java:105)
at oracle.upgrade.autoupgrade.dispatcher.AutoUpgDispatcher.runDispatcher(AutoUpgDispatcher.java:405)
at oracle.upgrade.autoupgrade.dispatcher.AutoUpgDispatcher.run(AutoUpgDispatcher.java:254)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
2019-06-17 14:10:41.168 INFO Starting error management routine – ErrorParser.getError
2019-06-17 14:10:41.169 INFO Ended error management routine – ErrorParser.getError
2019-06-17 14:10:41.185 ERROR Error occurred while running the dispatcher for job 100
Cause: An error occurred while trying to read the error properties file – AutoUpgDispatcher.run
2019-06-17 14:10:41.186 ERROR The dispatcher has failed due to:
Error: UPG-1601
The dispatcher has failed due to: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0]
Cause: An error occurred while trying to read the error properties file
Hi Erwin,
I think this is a known issue of the current tool. The next version should fix this.
If the database shows everything VALID afterwards, all is fine.
Cheers,
Mike
Mike,
We are seeing the below error using analyze stage of autoupgrade :
Ensure there is additional disk space in LOG_ARCHIVE_DEST_1 for at least 3291480064 of archived logs. Check alert log during the upgrade that there is no write error to the destination due to lack of disk space.
The database has archiving enabled. The upgrade process will need free disk space in the archive log destination(s) to generate archived logs to.
Archiving cannot proceed if the archive log destination is full during upgrade.
Archive Log Destination:
Parameter : LOG_ARCHIVE_DEST_1
Destination : +LINUX_MIGRATION_FRA
+LINUX_MIGRATION_FRA shows 150G free :
ASMCMD> lsdg LINUX_MIGRATION_FRA
State Type Rebal Sector Logical_Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name
MOUNTED EXTERN N 512 512 4096 1048576 204801 152269 0 152269
Please advise
Hi Sunil,
please do me a favor and open an SR. Upload the ENTIRE logging directory as you defined it in your config.cfg file.
Then share the SR number with me. My team mates will have a look at it.
Cheers,
Mike
Thanks
I have opened SR with Oracle Support
SR# 3-20291517071
Thanks Sunil,
my team mates are looking into it.
Cheers,
Mike
Mike,
Thanks.
I have updated the SR.
Looks like the issue is Alternate Log_archive_dest settings:
Autoupgrade Analyze is failing with errors for settings:
log_archive_dest_1 string LOCATION=+LINUX_MIGRATION_FRA MAX_FAILURE=1 REOPEN=10
ALTERNATE=LOG_ARCHIVE_DEST_2
log_archive_dest_2 string LOCATION=+LINUX_MIGRATION_DG
But no errors if settings are:
log_archive_dest_1 string LOCATION=+LINUX_MIGRATION_FRA MAX_FAILURE=1 REOPEN=10
log_archive_dest_2 string
Still verifying with other options. can you please confirm this
Thanks
Sunil
Hi Sunil,
we are discussing and checking this.
Thanks
Mike
Mike,
I found that Restore point was not dropped after the upgrade. Does that mean the upgrade is not completed successfully?
select * from v$restore_point;
SCN DATABASE_INCARNATION# GUA STORAGE_SIZE TIME
———- ——————— — ———— —————————————————————————
RESTORE_POINT_TIME PRE
————————————————————————— —
NAME PDB CLE
——————————————————————————————————————————– — —
PDB_INCARNATION# CON_ID
—————- ———-
1.5459E+13 4 YES 3670016000 13-JUN-19 04.01.43.000000000 PM
YES
AUTOUPGRADE_221145114461854_DDRMAN NO NO
0 0
Please advise
Hi Sunil,
the newer version – uploaded already – has this enhancement:
AUPG-779 Add option to drop Grp after upgrade
I’m working on the explanation blog posts.
At the moment it needs to be dropped manually.
Cheers,
Mike
Hi Mike,
Does autoupgrade support Data Guard setups as well?
Chhers,
Chris
Hi Christoph,
it works – but you need to disable the Broker and defer the log transport manually at the moment.
We are working on the inclusion of this as well.
Cheers,
Mike
Hi Mike,
I’m trying to upgrade a (fresh installed) 12.2 CDB to 19.3 with the autoupgrade tool (on a AIX 7.2 server).
(I downloaded the latest version: 20190620).
After the common RTFM-related issues, the upgrade now fails with:
2019-06-27 12:02:30.990 ERROR The following checks have ERROR severity and their current fixup is not available or
the fixup failed to resolve the issue. Please fix them manually before continuing:
CDB03D_APD MIN_RECOVERY_AREA_SIZE
The FRA first was 20 Gb (19 Gb free), and I raised it to 80 Gb now (76 Gb free), but no succes.
Any idea how to solve this ?
I raised an SR: 3-20440526811
Best regards,
Jan Willem
Jan Willem,
we are aware of this issue – currently we are working on the algorithm.
I drop you an email in parallel.
Cheers,
Mike
Did this issue MIN_RECOVERY_AREA_SIZE got resolved ? I am getting same issue when I upgraded from 12.1 to 19.3.
Hi Carol,
are you really upgrading to 19.3 or to a newer RU?
We solved it but I saw one new issue with it in case you use a different OS LOCALE or NLS_LANG setting.
What’s your settings?
Thanks,
Mike
Did you get this issue resolved ? I am getting same even with latest autoupgrade.jar
Which one?
Is this MIN_RECOVERY_AREA_SIZE issue resolved, What is the resolution.
I too get this issue
Which version of AU are you using?
Cheers,
Mike
Hi,
As part of AutoUpgrade tool it must remove the unsupported parameters before doing the upgrade. for example, if you try to upgrade one database from 12.1.0.2 with PARALLEL_ADAPTIVE_MULTI_USER set, the tool must identify and fix it based on a dictionary of unsupported parameters of target version, for instance, if target version is 18.6, the autoupgrade fail because cannot open the database in new version to perform the upgrade.
Regarding RAC support, with lastest version of the tool it does not end with success. issues: cluster_database; SPFILE localtion no $OH/dbs not in ASM shared DG; parameters collected are only from connected instance, not collected all parameters specific from other Oracle instances.
Hi Jose,
this problem is fixed with the newest tool. We were aware of it and fixed it with the July 15, 2019 version.
https://support.oracle.com/epmos/faces/DocumentDisplay?id=2485457.1
I just tried it before posting a reply to your comment with PARALLEL_ADAPTIVE_MULTI_USER=TRUE before the upgrade. It errored out during the time zone change before with the June version – and it completes successfully with the July version:
upg> lsj+----+-------+----------+---------+-------+--------------+--------+--------+---------------+
|Job#|DB_NAME| STAGE|OPERATION| STATUS| START_TIME|END_TIME| UPDATED| MESSAGE|
+----+-------+----------+---------+-------+--------------+--------+--------+---------------+
| 101| DB12|POSTFIXUPS|EXECUTING|RUNNING|19/07/16 21:11| N/A|21:52:27|Loading DB info|
+----+-------+----------+---------+-------+--------------+--------+--------+---------------+
Total jobs 1
upg> Job 101 completed
------------------- Final Summary --------------------
Number of databases [ 1 ]
Jobs finished successfully [1]
Jobs failed [0]
Jobs pending [0]
------------- JOBS FINISHED SUCCESSFULLY -------------
Job 101 FOR DB12
---- Drop GRP at your convenience once you consider it is no longer needed ----
And afterwards, a check and restart:
SQL*Plus: Release 19.0.0.0.0 - Production on Tue Jul 16 22:05:30 2019Version 19.3.0.0.0
Copyright (c) 1982, 2019, Oracle. All rights reserved.
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0
SQL> show parameter PARALLEL_ADAPTIVE_MULTI_USER
NAME TYPE VALUE
------------------------------------ ----------- ------------------------------
parallel_adaptive_multi_user boolean TRUE
SQL> startup force
ORA-32004: obsolete or deprecated parameter(s) specified for RDBMS instance
Regarding RAC:
Please see my other post getting published on Thursday most likely.
But the above fix fixes also the problem with cluster_database=TRUE set too early. It now completes. But you will have to do the srvctl part.
Cheers,
Mike
Can I use autoupgrade to apply a RU, moving the ORACLE_HOME?
In details: I have a CDB database, version 19.3.0, installed on ../19.3.0 OH. I created a ../19.4.0 OH. I did this installing the 19.3.0 base version and upgraded it to 19.4.0 RU (patch 29834717which has just released on 16-Jul-2019).
After this I created a config file that contains:
upg1.source_home=/tools/oracle/19.3.0
upg1.target_home=/tools/oracle/19.4.0
But when runing the analyze phase I got:
Errors in database [P04CDB]
Stage [SETUP]
Operation [STOPPED]
Status [ERROR]
Info [
Error: UPG-1606
An unsupported source database version [19.0.0.0 when upgrading to target database version [19.4.0.0.0] has been found.
Cause: The upgrade source version cannot be upgraded to the specified target version.
For further details, see the log file located at /tools/oracle/19.4.0/rdbms/admin/upg_logs/P04CDB/P04CDB/100/autoupgrade_20190719_user.log
I forgot to mention, but I’m of course usign the latest version:
[oracle@brux2490 OHOME1940 /tools/oracle/19.4.0/rdbms/admin]$ $ORACLE_HOME/jdk/bin/java -jar autoupgrade.jar -version
build.version 20190715
build.date 2019/07/15 12:45:48
No.
An UPGRADE is going from for example 18.6.0 to 19.4.0.
A PATCH is going from 19.3.0 to 19.4.0
For the UPGRADE you can take AutoUpgrade.
For the patch, you must use opatch and datapatch.
Nothing has changed here.
Thanks,
Mike
Ok, thanks for the reply. But this would be a great improvement for autoupgrade, despite the different names semantics.
With opatch and datapatch you can’t easily move an instance from one ORACLE_HOME to other.
When we apply RU or RUR (19.3.0 to 19.4.0 for example) we install the base release into a new empty ORACLE_HOME (no instances) and use opatch. After this we user dbua to change the ORACLE_HOME (which automatically run datapatch, since 12.2).
If I use the old way to upgrade from 18.6.0 to 19.4.0 I would do the same: install a new base release 19.3.0 ORACLE_HOME and update it to 19.4.0 using opatch. And use dbua from the new 19.4.0 OH to upgrade from 18.6.0 to 19.4.0 after this.
If I use the new way to upgrade from 18.6.0 to 19.4.0 I would do the same: install a new base release 19.3.0 ORACLE_HOME and update it to 19.4.0 using opatch. And use the latest autoupgrade after this.
So autoupgrade is a dbua replacement. So it could be also used to apply RU and RUR.
Hi Luis,
I see your point – but opatch is not meant to “flip” homes. If you’d like to automate this, then RHP (Rapid Home Provision) aka FPP (new name: Fleet Patch and Provisioning) are the tools who do this in interaction with opatch.
I know that the DBUA “can” do this move.
But I do the same with 2 steps more but instead see the logs directly. I prefer calling “datapatch -verbose” by myself.
Of course feel free to do it with the DBUA.
But in any case, you need OPATCH – DBUA does not run opatch for you but only does the datapatch part – and changes your home settings (which is without a doubt a nice thing).
Hence, patch with opatch, upgrade with autoupgrade (or dbupgrade or dbua). And you can run datapatch either manually (which I prefer) or ask DBUA to do this for you.
Cheers,
Mike
Thanks Mike and Luis, I ran into the same error and saw this message explains what happened.
[oracle@oel2 198]$ java -jar autoupgrade.jar -config sample_config.cfg -mode analyze
AutoUpgrade tool launched with default options
Processing config file …
There were conditions found preventing AutoUpgrade from successfully running
*Target compatibility
po19 An unsupported source database version [19.7.0.0.0] when upgrading to target database version [19.8.0.0.0] has been found
[oracle@oel2 198]$
I will continue to use opatch to apply new RU.
Hello Mike,
In mode “analyze” I recive following error:
CheckName: MIN_RECOVERY_AREA_SIZE FixUp Available: YES Severity: ERROR Stage: PRECHECKS
The database has {1} enabled, and the upgrade process will need free space to generate {2} to the recovery area specified by initialization parameter DB_RECOVERY_FILE_DEST. The logs generated must not overflow the limit set by DB_RECOVERY_FILE_DEST_SIZE, as that can cause the upgrade to not proceed.
DB_RECOVERY_FILE_DEST_SIZE is set at {4}. There is currently {6} of free space remaining, which may not be adequate for the upgrade.
Currently DB_RECOVERY_FILE_DEST_SIZE is 190 G and 187 G is free.
I think the error must be somthing else – have you any idea?
Thanks Philipp
Philipp,
can you please contact me via email and send me the entire log directory (zip or gzip) plus your config file.
I assume you are in ASM, right?
And you use the tool from July.
Cheers,
Mike
Please update the fix. I am running into the same issue.
Hi Moshe,
which version of the tool are you using? The newer ones shouldn’t cause this anymore.
Thanks,
Mike
I’m having the same issue with July 2020 version:
Err message: Invalid directory +RECOC1
Please open an SR and check with Oracle Support.
Thanks,
Mike
Hi Mike,
i did a upgrade with the autoupdater tool for a Windows database from 12.1.0.2 to 12.2.0.1. The DB Upgrade process itself works fine, the upgrade was a success. But it would be nice if the tool could remove the old windows service and creates a new one with the new Oracle Home set. Maybe the same with the listener.
Also it would be nice if it could check if the new OH is in front of the old OH in the PATH variable. After installing the new OH this is default i know but in my case i move the new OH at the end of the list to ensure that the environment will work for the next few weeks because i didn´t upgrade directly on that day where i installed the new OH.
Just some ideas. Maybe i missed some things in the documentation 😉
Thanks
Sven
Hi Sven,
sorry for the inconvenience – this is clearly a lock of documentation.
We can’t change the service as we would need access to the password. When you create a new service with ORADIM you need to specify the PW – and that’s what we can’t do. Furthermore, if we’d throw away the previous service, we would block your fallback unless we’d recreate it.
So there are reasons – but I see that this is not documented at all. And I will add this to the blog asap (which means in 2-3 weeks).
Cheers,
Mike
Hi Mike,
thanks for your answer.
That makes sense. No problem to make it manual was just an idea 😉
Thanks for adding it into the documentation.
Cheers
Sven
Hi Mike,
we have made successful with autoupgrade (build.version:20190715) upgrades on Linux and exadata.
But autoupgrade stopped during fixups on AIX (from 11.2.0.4 to 19.3) with the following error (SORA is the database name):
2019-07-29 13:40:38.613 ERROR The following checks have ERROR severity and their current fixup is not available or
the fixup failed to resolve the issue. Please fix them manually before continuing:
SORA PURGE_RECYCLEBIN
In prefixups_sora.log I found:
2019-07-29 13:40:21.213 INFO # PURGE_RECYCLEBIN – Check.runFix
2019-07-29 13:40:21.213 INFO Finished FIXUP [PURGE_RECYCLEBIN][SORA][SUCCESSFUL] – FixUpTrigger.executeFixUp
Also manuell purge of RECYCLEBIN doesn’t help.
What can I do ?
Hi Juergen,
please open an SR and upload the entire log directory from autoupgrade as you defined it in your config file.
Upload also the config file itself and an output log from DBA_RECYCLEBIN.
Then send me the SR number please (either email: mike.dietrich –at– oracle.com or here via comment).
My team mates will look into the logs.
And the SR is the best way to exchange the logs 🙂
Cheers,
Mike
Jurgen,
Refer to following Doc ID. I had the same issue.
Unable To Empty or delete rows from Sys.recyclebin$ which is causing dbua (upgrade) stopped and purge dba_recyclebin not helping (Doc ID 1910945.1)
Thanks Nishant – but with AutoUpgrade, this wouldn’t have happened 🙂
And with command line upgrade (dbupgrade) you wouldn’t hit this either.
The recyclebin should be empty. We have seen XDB upgrade issues in the past. But the DBUA has this check hard coded and does not progress when it is not satisfied.
Cheers,
Mike
Hi Mike,
thanks a lot for Your blog about autoupgrade Tool:-)
I was trying the autoupgrade from a multitenent 12.2 database (with only one PDB inside) to the newest EE 19c on the same host.
All was working fine as expected..;-)
But the compatible=’19.0.0″ parameter in my global_add_after.ora File goes wrong because autoupgrade was created a global restore point
as you can see below:
2019-08-28 15:16:28.435 ERROR
DATABASE NAME: cdb05
CAUSE: ERROR at Line 7 in [Buffer]
REASON: ORA-38880: Cannot advance compatibility from 12.2.0.0.0 to 19.0.0.0.0 due to guaranteed restore points
ACTION: [MANUAL]
DETAILS: 38880, 00000, “Cannot advance compatibility from %s to %s due to guaranteed restore points”
// *Cause: Flashback database cannot undo the advance of database
// compatibility. Therefore, one cannot advance the compatibility
// of the database while there are guaranteed restore points in the
// database.
// *Action: Drop all guaranteed restore points first and retry, or delay
// the advance of database compatibility to a later time.
Sure, I can shutdown the database, change the compatible parameter, create a new spfile and startup the database again.
So, in this case all is working fine, but i need a little bit more downtime…
Have You any ideas how to resolve the “problem”? Maybe another approach for doing this?
Thanks a lot again!
best regards
Stefan
Hi Stefan,
this is expected. We don’t want you to change COMPATIBLE as part of the upgrade but manually afterwards.
While I see that it may be desired, you can only combine it when you use RESTORATION=NO.
But I’m with you – this is not documented – and we don’t check this.
I share this with the team for discussion.
Cheers,
Mike
Hi Mike,
I noticed something strange
I am upgrading database from 18.7 to 19.4 using Autoupgrade Utility 20190715.
I ran $18c/jdk/bin/java -jar $19c/rdbms/admin/autoupgrade.jar -config $loghome/sid_upgrade19c.cfg -mode analyze which ran successfully
Then I ran $18c/jdk/bin/java -jar $19c/rdbms/admin/autoupgrade.jar -config $loghome/sid_upgrade19c.cfg -mode deploy
and prompted an error as below
2019-09-02 01:42:33.693 ERROR Unable to determine the amount of usable free space available on /archpoint/arch_ – Utilities.getUsableSpace
2019-09-02 01:42:41.495 ERROR The following checks have ERROR severity and their current fixup is not available or
the fixup failed to resolve the issue. Please fix them manually before continuing:
SID MIN_ARCHIVE_DEST_SIZE – PreChecks.executeChecks
2019-09-02 01:42:41.509 ERROR DB checks ended unsuccessfully – ExecuteDbChecks.executeStage
2019-09-02 01:42:41.509 ERROR Upgrade Dispatcher failed. Cause: AutoUpgException [UPG-1303] – DispatcherExecuteContext.callExecuteContext
oracle.upgrade.autoupgrade.utils.errors.AutoUpgException: AutoUpgException [UPG-1303]
I fixed the spfile for the parameter log_archive_dest_1, restarted the database and wanted to check if the error still exists and hence ran in mode analyze again
$18c/jdk/bin/java -jar $19c/rdbms/admin/autoupgrade.jar -config $loghome/sid_upgrade19c.cfg -mode analyze
But the utility proceeded to upgrade the database
How do I do the following?
1. Completely stop the utility?
2. Recognize the failed job?
3. Restart the utility after fixing the failed job?
Thanks,
Tanveer
Hi Tanveer,
you used “-mode deploy” and hence, the upgraded proceeded. The error is a java error from the tool but the process will go on.
Just hit RETURN when such an error happens and you should be back on the UPG> prompt.
Please, if you have time, would you mind to file an SR, upload the entire log directory.
Then send me the SR number and I can share it with the developers.
Cheers,
Mike
PS: I assume you are using the newest tool, right?
Hi Mike,
just downloaded the 20190823 version of autoupgrade.jar.
During analyze run I got the error UPG-1316.
In the log file:
2019-09-10 13:26:42.400 ERROR
============================ check info ============================
[LXTOID2][DISK_SPACE_FOR_RECOVERY_AREA][ERROR]
============================ check info ============================
=========================== trace start ==============================
Exception: IOException
Err message: Unable to determine the amount of usable free space available on +lxtoid2_fra_dg
oracle.upgrade.commons.helpers.Utilities.getUsableSpace(Utilities.java:682)
oracle.upgrade.commons.dbinspector.checks.disk_space_for_recovery_area.checkCode(disk_space_for_recovery_area.java:89)
oracle.upgrade.commons.dbinspector.Check.presentInDb(Check.java:227)
oracle.upgrade.commons.dbinspector.CheckTrigger.call(CheckTrigger.java:72)
oracle.upgrade.commons.dbinspector.CheckTrigger.call(CheckTrigger.java:40)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
With version 20190715, build date 2019/07/15 12:45:48 it works.
???
Hi Bernd,
thanks for letting us know. Would you please do us a big favor and open an SR, upload the entire log directory and send me the SR number, either via the blog or via email to mike.dietrich — at — oracle.com?
That is by far the easiest way to deal with the logs.
Cheers and sorry for the inconvenience,
Mike
I had a similar issue. I decompiled the java class
oracle.upgrade.commons.helpers.Utilities.getUsableSpace
and I found a piece of code on a corresponding line:
if (path.indexOf(“+”) == 0) {
final int mb = 1048576;
final String query = “select USABLE_FILE_MB from v$asm_diskgroup where upper(name)=upper(substr(‘” + path + “‘,2));”;
There is hard coded SQL query againt an ASM view v$asm_diskgroup . Try to run query with SQLplus and compare the result.
Hi Jiri,
to which version of the tool does this apply? The Oct 24 autoupgrade?
Cheers,
Mike
Hi Mike,
Does the database have to be in archivelog mode to run the autoupgrade tool?
Thanks,
Juhan
Please read the blog post series – that’s why I started it 🙂
See here:
https://mikedietrichde.com/2019/06/28/config-file-for-autoupgrade-19c-advanced-options/
Cheers,
Mike
Hi,
Is there a configuration parameter to run upgrade tool while database is in noarchivelog mode?
Thanks,
Juhan
Juhan,
please see the blog post series (or the documentation which is even linked from the artical you were commenting on):
https://mikedietrichde.com/2019/06/28/config-file-for-autoupgrade-19c-advanced-options/
Cheers,
Mike
Hello Mike,
I’m using the newest autoupgrade.jar on SUSE Linux 12 for autoupgrading 12.2.0.1 to 19c.
Question 1:
The running time for the Stage Postfixups needs more than 30 minutes with message “Remaining 3/7”.
Is this a normal behaviour?
upg> lsj
Job#|DB_NAME| STAGE |OPERATION| STATUS | START_TIME |END_TIME | UPDATED| MESSAGE|
104 | CDB06C |POSTFIXUPS|EXECUTING |RUNNING|19/10/16 17:11 | N/A |10:17:23 |Remaining 3/7|
Question2:
Sometimes autoupgrade has an error like:
2019-10-16 18:09:07.322 INFO SUCCESSFULLY UPGRADED [cdb06c-chpoli1p]
2019-10-16 18:09:07.322 INFO End Upgrade on Database [cdb06c-chpoli1p]
2019-10-16 18:09:11.858 ERROR
DATABASE NAME: cdb06c-CHPOLI1P
CAUSE: ERROR at Line 4 in [Buffer]
REASON: ORA-44309: unknown failure
ACTION: [MANUAL]
DETAILS: 44309, 0000, “unknown failure”
// *Document: No
// *Cause: There was an unknown failure.
// *Action: Contact Oracle Support Services.
2019-10-16 18:09:11.917 ERROR
DATABASE NAME: cdb06c-CHPOLI1P
CAUSE: ERROR at Line 5 in [Buffer]
REASON: ORA-44777: Pluggable database service cannot be started.
ACTION: [MANUAL]
DETAILS: 44777, 0000, “Pluggable database service cannot be started.”
// *Document: Yes
// *Cause: The attempted pluggable database operation failed.
// *Action: Check the trace file for details.
2019-10-16 18:09:11.922 ERROR Database Open Failed for CHPOLI1P alter pluggable database CHPOLI1P open read write restricted
*
ERROR at line 1:
ORA-44309: unknown failure
ORA-44777: Pluggable database service cannot be started.
My workaround is:
alter pluggable database CHPOLI1P close;
alter pluggable database CHPOLI1P unplug into ‘/tmp/chpoli1p.xml’;
drop pluggable database CHPOLI1P;
create pluggable database CHPOLI1P using ‘/tmp/chpoli1p.xml’ NOCOPY TEMPFILE REUSE;
alter pluggable database CHPOLI1P open restricted;
after this steps I resume the stpped job with:
upg> resume -job 104
Now all was working fine as expected:-)
Any ideas whats the reason?
Question3:
Is it possible to use autoupgrade in “parallel-mode” so that the upgrading steps are a little bit faster..?
Thank you very much for your investigations and feedback:-)
Congratulations! autoupgrade is a very great tool!:-)
with best regards
Stefan
Hi Stefan,
regarding your questions:
1. What is “status -job 104” telling you when it takes so long? The command will tell you.
2. This shouldn’t have to do with the upgrade itself. Check the alert.log – and I think you please should open an SR for this. You can share the SR number with me.
3. AutoUpgrade runs the database upgrade in parallel as underneath the covers, catctl.pl with 4 default threads will be used. But still a number of things in the upgrade are serial. For things such as stats refresh, AutoUpgrade relies on the features the stats collection offers. We can’t speed this up. But AutoUpgrade can upgrade many databases in parallel and leverage the CPU power you offer.
Cheers,
Mike
Hi Mike,
thanks you so much for Your fast respone.
best regards
Stefan
Hi Mike,
thanks for the tool. It’s possible to restart the console and after the restart autoupgrade doesn’t start any new task?
Uwe, can you explain a but more what you are doing and what doesn’t give you the expected results?
I’m a bit unsure.
Thanks,
Mike
Hi Mike,
Example:
The Upgrade with autoupgrade failed. I have to wait for an answer from Oracle Support. I left the console with “exit”. The next day I want to use the console to reset the database to the state before the update.
My questions are:
1. How can I start the console to get to the point where I left it?
2. What happens if the client loses its connection to the server during a console upgrade?
Thank you.
Best regards
Uwe
Hi Uwe,
sorry, I missed your reply. You can call the console with java -jar autoupgrade.jar -config … -mode deploy
the same way as before. If the upgrade has halted, it will resume it – and if it is running, it will reattach.
Cheers,
Mike
Hi Mike,
Thanks for your fast response.
Example:
autoupgrade terminated with an upgrade error and a service request was opened in MOS. The console was left with “exit”. The next day, the database must be reset to the state before the update. For that I want to use the console.
My questions are:
1. How do I start the console to have the state I left the console before?
2. What happens if, during an upgrade with autoupgrade, the client on which the console is running loses the connection to the Linux database server?
Best regards
Uwe
Hi Mike,
I have created SR 3-21509138221 for the following issue :
I’m getting an Internal Error when executing autoupgrade in mode analyze
AutoUpgrade tool launched with default options
Internal Error has occurred. Please contact Oracle support.
java.lang.StringIndexOutOfBoundsException: String index out of range: -1
at java.lang.String.substring(String.java:1967)
at oracle.upgrade.autoupgrade.config.jvalid.NonCDBConversion.isValidSetup(NonCDBConversion.java:51)
at oracle.upgrade.autoupgrade.config.links.JobRulesValidator.process(JobRulesValidator.java:65)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.SettingsMaker.process(SettingsMaker.java:80)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.UpgradeConfigMaker.process(UpgradeConfigMaker.java:669)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.InternalSettingsParser.process(InternalSettingsParser.java:124)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.UserConfigFileParser.process(UserConfigFileParser.java:180)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.ContextFinder.process(ContextFinder.java:133)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.LoggerMaker.process(LoggerMaker.java:68)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.AutoUpgradeLocker.process(AutoUpgradeLocker.java:73)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.GlobalConstants.process(GlobalConstants.java:146)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.CLIOptionsParser.process(CLIOptionsParser.java:136)
at oracle.upgrade.autoupgrade.config.CLIProcessor.processCLIParams(CLIProcessor.java:117)
at oracle.upgrade.autoupgrade.boot.AutoUpgMain.(AutoUpgMain.java:124)
at oracle.upgrade.autoupgrade.boot.AutoUpgMain.newInstance(AutoUpgMain.java:141)
at oracle.upgrade.autoupgrade.boot.Boot.main(Boot.java:43)
with the latest version :
java -jar ./autoupgrade.jar -version
build.hash 9c85160
build.version 20191024
build.date 2019/10/24 12:47:31
build.max_target_version 19
build.type production
The database is running on Oracle Linux 7.6 in OVM on Exadata.
config file :
#
# sample config file
#
# build version 20191024
# build date 2019/10/24 12:47:31
#
#
# Global configurations
#
# This directory will include the following
# (1) AutoUpgrade’s global directory
# (2) Any logs, not directly tied to a job
# (3) Config files
# (4) progress.json and status.json
global.autoupg_log_dir=/home/oracle/upgrade/upg_logs
#
# Database number 1
#
upg1.dbname=dbupg
upg1.start_time=NOW
upg1.source_home=/u02/app/oracle/product/12.1.0.2/dbhome_31
upg1.target_home=/u02/app/oracle/product/19.3.0/dbhome_1
upg1.sid=dbupg
upg1.log_dir=/home/oracle/upgrade/upg_logs/dbupg
upg1.upgrade_node=
upg1.target_version=19
#upg1.run_utlrp=yes
#upg1.timezone_upg=yes
Using the autoconfig.jar file which is installed with version 19.3.0 does work, in analyze mode (so this indicates that it’s not the config file).
However the deploy mode just returns, without doing anything. But that’s another issue. I would like to use the latest version anyhow.
Any idea ?
Thanks,
Bart
I found the cause. I made a type in the target ORACLE_HOME.
This wasn’t clear from the error message, unfortunately.
Hi Bart,
thanks for the update – I will share with the developers.
Cheers,
Mike
Just to inform you : I still have a problem using autoupgrade on an 11.2.0.4 database to 19.3.0.
In the same environment as above, with practically the same configuration file (except for the source version and source database name).
java -jar $ORACLE_HOME/rdbms/admin/autoupgrade.jar -config au.cfg -mode analyze
AutoUpgrade tool launched with default options
There was an error initializing the patching information for entry srcdb
Config file :
# config file
global.autoupg_log_dir=/home/oracle/upgrade/srcdb_logs
global.target_home=/u02/app/oracle/product/19.3.0.0/dbhome_1
global.target_version=19
global.drop_grp_after_upgrade=no
global.del_during_upgrade_pfile=/home/oracle/upgrade/inactive_params.ora
global.del_after_upgrade_pfile=/home/oracle/upgrade/inactive_params.ora
#
# Database number 1 – srcdb
#
srcdb.dbname=srcdb
srcdb.start_time=NOW
srcdb.source_home=/u02/app/oracle/product/11.2.0.4/dbhome_31
srcdb.sid=srcdb
srcdb.log_dir=/home/oracle/upgrade/srcdb_logs/srcdb
srcdb.upgrade_node=exa02adm01vm02.admbnet.be
srcdb.run_utlrp=yes
srcdb.timezone_upg=yes
The autoupgrade_err.log shows
2019-11-19 15:20:22.827 ERROR 4 – UpgradeConfigDBValidator.findPatchInfo
java.lang.ArrayIndexOutOfBoundsException: 4
at oracle.upgrade.autoupgrade.config.UpgradeConfigDBValidator.findContainerPatches(UpgradeConfigDBValidator.java:396)
at oracle.upgrade.autoupgrade.config.UpgradeConfigDBValidator.findPatchInfo(UpgradeConfigDBValidator.java:329)
at oracle.upgrade.autoupgrade.config.links.UpgradeConfigMaker.process(UpgradeConfigMaker.java:605)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.InternalSettingsParser.process(InternalSettingsParser.java:124)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.UserConfigFileParser.process(UserConfigFileParser.java:180)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.ContextFinder.process(ContextFinder.java:133)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.LoggerMaker.process(LoggerMaker.java:68)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.AutoUpgradeLocker.process(AutoUpgradeLocker.java:73)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.GlobalConstants.process(GlobalConstants.java:146)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.CLIOptionsParser.process(CLIOptionsParser.java:136)
at oracle.upgrade.autoupgrade.config.CLIProcessor.processCLIParams(CLIProcessor.java:117)
at oracle.upgrade.autoupgrade.boot.AutoUpgMain.(AutoUpgMain.java:124)
at oracle.upgrade.autoupgrade.boot.AutoUpgMain.newInstance(AutoUpgMain.java:141)
at oracle.upgrade.autoupgrade.boot.Boot.main(Boot.java:43)
All information is added to SR 3-21509138221.
Bart,
I guess you are hitting the same issue as others commented here:
https://mikedietrichde.com/2019/11/25/autoupgrade-may-fail-when-patch-id-column-is-null/
Please check the ID column is registry$history – and if an entry is NULL, change it as I explained in my blog post.
Cheers,
Mike
Hi Mike,
can’t reply to your message, but it was indeed caused by a NULL value in registry$history.
Many thanks !
Bart
Hello Mike,
I am trying to upgrade 12.2 ( 2 node RAC database) to 19c using autoupgrade utility, however I am always having error MIN_RECOVERY_AREA_SIZE precheks error.
I almost keep 100gb recovery destination size, still I am seeing same error message.
please let us know how we can proceed on this.
Regards,
Prashant
Hi Prashant,
what does “java -jar autoupgrade.jar -version” tells you?
Thanks,
Mike
Hi Mike,
Thanks for the reply.
Issue is resolved by downloading latest autoupgrade jar utility.
Regards,
Prashant Dhoble
Awesome – thanks!
Mike
Seeing this error with latest autoupgrade.jar also 🙁
[oracle@casmvlpe1ora02 ~]$ java -jar autoupgrade.jar -version
build.hash 04dd9f2
build.version 19.7.5
build.date 2020/02/11 15:28:49
build.max_target_version 19
build.type production
Which error?
Hi
Mike, is it possible to change default number of parallel threads from 4 to something higher ? What is parallel degree of invalid objects recompilation, can it be set separately ?
Regards,
Artem
Hi Artem,
no, on purpose this isn’t possible. But I’ll explain, why we set it this way.
1. For the non-CDB (or CDB$ROOT) upgrade itself, n=4 is a good value. The maximum workers of n=8 in this case would add very little performance benefit. with 2 workers per PDB, e.g. your CPU_COUNT=32, then we will use 32 workers for the PDBs, 2 per PDB => 16 PDBs can be worked in parallel. is taken into calculation by default as in every typical upgrade, i.e. in the above example you’d get 63 parallel recompilation jobs and one coordinator. This sometimes is a bit too much, especially when Virtual CPUs get displayed as on SUN Tx CPUs. That’s why you can skip the recompilation, and compile later by yourself with your desired degree (which answers your 2nd question).
2. For PDBs, we use n=
3. For the recompilation, the
Cheers,
Mike
Hi Mike,
I’m trying to upgrade our singletenant databases with autoupgrade (from 12.2 to 19.5). In the most cases, ROOT_CONTAINER and SEED Databases are upgraded successfully. But sometimes i’m a liitel confused about the messages in the autoupgrade_20191209_user.log logfile as you can see in the snippet below:
When the ROOT_CONTAINER was upgraded, the SEED and the Pluggable Database JANUS1P are in state “UPGRADE PENDING”….thats correct. But few seconds later the state for the PDB JANUS1P was changed to ” SUCCESSFULLY UPGRADED [cdb15c-janus1p]” ..but this message is wrong…3 mins later the state for PDB JANUS1P was updated with “JANUS1P UPGRADE 6%” like the SEED database. Sounds better..;-)
Can Yo tell me what is wrong in this case? From where is the wrong information?
Thanks a lot for Your investigations and feedback!
with best regards
Stefan
+———+—————+
2019-12-09 16:46:32.827 INFO [Compiling] is [99%] completed for [cdb15c-cdb$root] objects remaining is [20]
+———+—————+
|CONTAINER| PERCENTAGE|
+———+—————+
| CDB$ROOT| COMPILE [99%]|
| PDB$SEED|UPGRADE PENDING|
| JANUS1P|UPGRADE PENDING|
+———+—————+
2019-12-09 16:47:32.610 INFO End Compiling Invalid Objects on Database [CDB15C-cdb$root]
2019-12-09 16:47:32.611 INFO SUCCESSFULLY COMPILED [CDB15C-cdb$root]
2019-12-09 16:47:32.830 INFO [Upgrading] is [100%] completed for [cdb15c-cdb$root]
+———+—————————————+
|CONTAINER| PERCENTAGE|
+———+—————————————+
| CDB$ROOT|SUCCESSFULLY UPGRADED [cdb15c-cdb$root]|
| PDB$SEED| UPGRADE PENDING|
| JANUS1P| UPGRADE PENDING|
+———+—————————————+
2019-12-09 16:47:34.070 INFO Creating spfile completed with success
2019-12-09 16:47:36.544 INFO Total Number of upgrade phases is 108
2019-12-09 16:47:36.545 INFO Total Number of upgrade phases is 108
2019-12-09 16:47:41.779 INFO Begin Upgrade on Database [cdb15c-pdb$seed]
2019-12-09 16:47:42.860 INFO Begin Upgrade on Database [cdb15c-janus1p]
2019-12-09 16:47:51.504 INFO [Upgrading] is [100%] completed for [cdb15c-janus1p]
+———+—————————————+
|CONTAINER| PERCENTAGE|
+———+—————————————+
| CDB$ROOT|SUCCESSFULLY UPGRADED [cdb15c-cdb$root]|
| PDB$SEED| UPGRADE PENDING|
| JANUS1P| SUCCESSFULLY UPGRADED [cdb15c-janus1p]|
+———+—————————————+
2019-12-09 16:47:51.515 INFO [Upgrading] is [0%] completed for [cdb15c-pdb$seed]
+———+—————————————+
|CONTAINER| PERCENTAGE|
+———+—————————————+
| CDB$ROOT|SUCCESSFULLY UPGRADED [cdb15c-cdb$root]|
| PDB$SEED| UPGRADE [0%]|
| JANUS1P| SUCCESSFULLY UPGRADED [cdb15c-janus1p]|
+———+—————————————+
2019-12-09 16:50:53.985 INFO [Upgrading] is [6%] completed for [cdb15c-janus1p]
+———+—————————————+
|CONTAINER| PERCENTAGE|
+———+—————————————+
| CDB$ROOT|SUCCESSFULLY UPGRADED [cdb15c-cdb$root]|
| PDB$SEED| UPGRADE [0%]|
| JANUS1P| UPGRADE [6%]|
+———+—————————————+
2019-12-09 16:50:53.999 INFO [Upgrading] is [6%] completed for [cdb15c-pdb$seed]
+———+—————————————+
|CONTAINER| PERCENTAGE|
+———+—————————————+
| CDB$ROOT|SUCCESSFULLY UPGRADED [cdb15c-cdb$root]|
| PDB$SEED| UPGRADE [6%]|
| JANUS1P| UPGRADE [6%]|
+———+—————————————+
Hi Stefan,
I’m almost 100% sure that I have seen this by myself – and the folks fixed it already.
Not sure if it is fixed already in the November 2019 autoupgrade – or if it will be in a future one.
Cheers,
Mike
Hi Mike,
thanks a lot for Your Feedback. I’m using the latest autoupgrade.jar from November 2019. So I think the fix is available in a future release. Sure, it’s not a critical issue:-) I think we will use autoupgrade in the next year for patching all our database servers with dataguard:-)
Have a nice day and thanks again.
Best regards
Stefan
Hi Mike,
in Doc ID 2485457.1 the DB version 12.1 is not listed as supported version; In this thread people use it for upgrade to 18c/12.2 – Can we upgrade from 12.1.0.2 to 19c using autoupgrade ? Is it supported?
Hi Christian,
I wrote:
“As source, the minimum version is Oracle Database 11.2.0.4. Fair enough, isn’t it?”
In above note, you refer to, it says:
“For supported source Oracle Databases releases to be upgraded to above target releases, refer to Database Server Upgrade/Downgrade Compatibility Matrix (Doc ID 551141.1)”
And this MOS note gives 11.2.0.4 as the minimum source release to upgrade to 19c.
So yes, of course, this is supported 🙂
Cheers,
Mike
Hi Mike,
First of all – that a lot for your blog!
~
Do you have any suggestion how one can make clear to Oracle Support Engineer that the reason for an SR is the AutoUpgrade tool and not the upgrade itself?
It seems that not everyone at Support is aware of this tool…
Regards,
jay
Hi Jay,
the best thing is to upload the entire log directory plus the alert.log.
Then the logs contain the autoupgrade logs PLUS the entire regular upgrade logs, too.
It should be not too complicated for the folks to determine if it’s an issue of the tool or a regular upgrade problem.
Cheers,
Mike
Hi Mike,
Thanks for the tool, we use it to upgrade from 12.1 to 19, mainly non-CDB.
With the latest version 20191125 several things work fine even in RAC/RACOneNode environment:
– spfile management in ASM
– cluster_database management even for timezone upgrade
– srvctl upgrade
Unfortunately, the progress in main phase of the upgrade is only updated for CDBs but not for non-CDBs anymore:
upg> lsj
+—-+———-+———+———+——-+————–+——–+——–+——————-+
|Job#| DB_NAME| STAGE|OPERATION| STATUS| START_TIME|END_TIME| UPDATED| MESSAGE|
+—-+———-+———+———+——-+————–+——–+——–+——————-+
| 100|CDBTWAC2_1|DBUPGRADE|EXECUTING|RUNNING|19/12/19 12:21| N/A|12:32:10|6%Upgraded CDB$ROOT|
| 101| TWAC1_1|DBUPGRADE|EXECUTING|RUNNING|19/12/19 12:23| N/A|12:28:05| Running|
+—-+———-+———+———+——-+————–+——–+——–+——————-+
Total jobs 2
upg> ————————————————-
job 101 has not shown progress in last 10 minutes
An error appears at the end of the main upgrade phase:
upg> java.sql.SQLException: Errors executing [SELECT OPEN_MODE FROM SYS.V$DATABASE;
SELECT OPEN_MODE FROM SYS.V$DATABASE
*
ERROR at line 1:
ORA-01034: ORACLE not available
Process ID: 0
Session ID: 0 Serial number: 0
After the error, everything looks fine:
upg> lsj
+—-+———-+———+———+——-+————–+——–+——–+——————–+
|Job#| DB_NAME| STAGE|OPERATION| STATUS| START_TIME|END_TIME| UPDATED| MESSAGE|
+—-+———-+———+———+——-+————–+——–+——–+——————–+
| 100|CDBTWAC2_1|DBUPGRADE|EXECUTING|RUNNING|19/12/19 12:21| N/A|12:56:27|93%Upgraded CDB$ROOT|
| 101| TWAC1_1|DBUPGRADE|EXECUTING|RUNNING|19/12/19 12:23| N/A|12:58:05| Running|
+—-+———-+———+———+——-+————–+——–+——–+——————–+
Total jobs 2
upg> lsj
+—-+———-+———-+———+——-+————–+——–+——–+——————-+
|Job#| DB_NAME| STAGE|OPERATION| STATUS| START_TIME|END_TIME| UPDATED| MESSAGE|
+—-+———-+———-+———+——-+————–+——–+——–+——————-+
| 100|CDBTWAC2_1| DBUPGRADE|EXECUTING|RUNNING|19/12/19 12:21| N/A|13:08:30|6%Upgraded PDB$SEED|
| 101| TWAC1_1|POSTFIXUPS|EXECUTING|RUNNING|19/12/19 12:23| N/A|13:09:51| Remaining 2/3|
+—-+———-+———-+———+——-+————–+——–+——–+——————-+
Total jobs 2
Thanks,
Christian
Hi Christian,
would you please mind to open an SR, upload the entire log tree and the alert.log and send me the SR number (ideally via email – mike.dietrich — at — oracle.com)?
Then I can ask my team mates to have a look.
Cheers,
Mike
Hi Mike,
We’re trying to autoupgrade the year 2019 🙂 (or at least a database).
In a test environment, we could successfully run autoupgrade to upgrade a 11.2.0.4 db to 19.3 (after solving the issue with registry$history you commented on earlier).
However, in the production environment, we’re facing a new issue.
Running autoupgrade in mode analyze, fails with
2019-12-31 09:16:29.089 DEBUG deriving source oracle_base [isadev] – UpgradeConfigMaker.process
2019-12-31 09:16:29.090 DEBUG validating source ORACLE_BASE directory [isadev] – UpgradeConfigMaker.process
2019-12-31 09:16:29.091 DEBUG looking for before_upgrade action[isadev] – UpgradeConfigMaker.process
2019-12-31 09:16:29.092 DEBUG looking for after_upgrade action[isadev] – UpgradeConfigMaker.process
2019-12-31 09:16:29.092 DEBUG finding database version [isadev] – UpgradeConfigMaker.process
It was not possible to determine the database version for entry isadev
The error log shows :
2019-12-31 09:16:28.653 ERROR It was not possible to determine the version value of database isadev from the returned value select version_full from sys.v$instance. – UpgradeConfigDBValidator.findDBVersion
In the test environment, the same select statement also failed, but the error was only shown as an informational message in the autoupgrade.log (not in autoupgrade_err.log) :
2019-11-26 15:29:54.605 INFO Executing SQL [select version_full from sys.v$instance;] in [horus, container:null] – ExecuteSql$SQLClient.run
2019-11-26 15:29:54.638 INFO The query [select version_full from sys.v$instance;] [null] finished with the following message [[00904] “VERSION_FULL”: invalid identifier] – ExecuteSql$SQLClient.run
This happens with the latest autoupgrade version :
build.hash 04a58df
build.version 20191125
build.date 2019/11/25 17:49:18
build.max_target_version 19
build.type production-19.7.2
and also with a previous version
build.hash 9c85160
build.version 20191024
build.date 2019/10/24 12:47:31
build.max_target_version 19
build.type production
I’ve created SR 3-21920667731, but I’m afraid that autoupgrade isn’t very well known yet …
Kind Regards, and a happy New Year !
Hi Bart,
I will check with the developers and get back to you then – thanks!
Mike
Hi there, I am also hitting the “version_full” issue, please see below:
13:08:40 SQL> Rem =====================================================================
13:08:40 SQL> Rem The following statement confirms that the script version identified by
13:08:40 SQL> Rem the &C_ORACLE_HIGH_ sqlplus variables matches the
13:08:40 SQL> Rem server version full value from v$instance. The C_ORACLE_HIGH
13:08:40 SQL> Rem sqlplus variables are defined in dbms_registry_basic.sql and contains
13:08:40 SQL> Rem a version value like 18.10.2.0.0. This value will be
13:08:40 SQL> Rem substituted in the query at run time, for example,
13:08:40 SQL> Rem TO_NUMBER(‘MUST_BE_18.10.2’)
13:08:40 SQL> Rem =====================================================================
13:08:40 SQL>
13:08:40 SQL> SELECT TO_NUMBER(
13:08:40 2 ‘MUST_BE_&C_ORACLE_HIGH_MAJ..&C_ORACLE_HIGH_RU..&C_ORACLE_HIGH_RUR’)
13:08:40 3 FROM v$instance
13:08:40 4 WHERE substr(version_full,1,instr(version,’.’,1,3)-1) !=
13:08:40 5 ‘&C_ORACLE_HIGH_MAJ..&C_ORACLE_HIGH_RU..&C_ORACLE_HIGH_RUR’;
old 2: ‘MUST_BE_&C_ORACLE_HIGH_MAJ..&C_ORACLE_HIGH_RU..&C_ORACLE_HIGH_RUR’)
new 2: ‘MUST_BE_19.3.0’)
old 5: ‘&C_ORACLE_HIGH_MAJ..&C_ORACLE_HIGH_RU..&C_ORACLE_HIGH_RUR’
new 5: ‘19.3.0’
WHERE substr(version_full,1,instr(version,’.’,1,3)-1) !=
*
ERROR at line 4:
ORA-00904: “VERSION_FULL”: invalid identifier
Disconnected from Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 – 64bit Production
With the Partitioning, OLAP, Advanced Analytics and Real Application Testing options
Hi Francious,
did you open an SR for this?
To which RU are you upgrading?
Which AutoUpgrade are you using?
Thanks,
Mike
Hello Mike ,
We had completely automated our database upgrades using the transient logical standby method and using the autoupgrade.jar utility to upgrade the logical standby database .But the autoupgrade tool is collecting the dictionary stats before and after upgrade (SYS.DBMS_STATS.GATHER_DICTIONARY_STATS) . We have some huge databases, where this stats collection is running for a long time which affect the total upgrade timelines .
Is there any way to disable the stats gathering from the autoupgrade utility
Regards
Anjith
Hi Anjith,
glad to hear that you’ve automated this process.
Currently the only way to workaround the stats collection is to manipulate a file entry on disk.
Try this please:
In case you need to overwrite this particular fixup, do this:
prefix_logdir/sid/jobno/prechecks/sid_checklist.cfg
[checkname] DICTIONARY_STATS
[stage] PRECHECKS
[fixup_available] YES
[runfix] YES >>>> NO
[severity] RECOMMEND
You need to change [runfix] attribute from YES to NO.
Thanks,
Mike
Hello Mike, I am wanting to disable pre/post POST_DICTIONARY checks, I have read you are able to use a specific file .cfg. I am wondering how to do this I have read you can put it in the driving .cfg file that you pass in when you run the deploy is this correct? What is the correct syntax for adding this, also will you need to specifically run the analyze everytime… say if i ran it once and created a template for the .cfg i wanted to run pre/post etc?
Hi Colby,
Try this please:
In case you need to overwrite this particular fixup, do this:
prefix_logdir/sid/jobno/prechecks/sid_checklist.cfg
[checkname] DICTIONARY_STATS
[stage] PRECHECKS
[fixup_available] YES
[runfix] YES >>>> NO
[severity] RECOMMEND
You need to change [runfix] attribute from YES to NO.
Thanks,
Mike
Mike , i’m testing the autoupgrade . I’m hitting the following
Error: UPG-1401
Opening Database indmgphs in upgrade mode failed
When I look at the log , I see
REASON: ORA-16026: parameter LOG_ARCHIVE_DEST_1 contains an invalid attribute value
When I manually startup the db with oracle 19 in upgrade mode , it startups up fine . So not sure why autoupgrade is having an issue starting up .
I’ve created an SR 3-21974932511 . Could someone in your team take a look ? Thanks .
I found the issue . Looks like a bug with autoupgrade . I’m using the latest version of autoupgrade .
During the autoupgrade it startups the db with version 19 , using the following pfile .
/opt/oracle/data/tmp/INDMGPHS/temp/during_upgrade_pfile_INDMGPHS.ora
The 11g had a parameter :
*.log_archive_dest_1=’LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES,ALL_ROLES) DB_UNIQUE_NAME=INDMGPHS’
in the following file during_upgrade_pfile_INDMGPHS.ora ( generated by autoupgrade ) it had
log_archive_dest_1=’LOCATION=USE_DB_RECOVERY_FILE_DEST VALID_FOR=(ALL_LOGFILES’,’ALL_ROLES) DB_UNIQUE_NAME=INDMGPHS’
Notice its has put ‘ here ( ALL_LOGFILES’,’ALL_ROLES ) for some reason and failing during startup upgrade
This looks like a bug . Can you please validate and possibly file it as such . I’ve also updated the SR
See my other reply 🙂
Somebody from our team is looking into this already.
Cheers,
Mike
Great, thanks . Just an update , I created an spfile from the same pfile , like we have for all our dbs , and the upgrade was successful . I’m guessing this only happens when using a pfile to start the db .
Hi Dinesh,
Did you use ../temp/after_upgrade….ora pfile to create spfile? What was your steps?
Hi Dinesh,
somebody from the team is already looking into this – Support contacted us.
Cheers,
Mike
Hi Mike
I download the latest autoupgrade.ja tool, i want to upgrade from 19c to 21c but still got this error
Hi Mike, I am trying to upgrade a single database, and am getting the following error.. Only the global folder is getting created the DB specific folder is not getting created.. any ideas?
2020-01-09 13:28:36.081 ERROR Problem detected in configuration file, there is one prefix for more than one database entry [DBNAME], fix it before continuing – UserConfigFileParser.findDuplicateEntries
2020-01-09 13:28:36.084 ERROR There are duplicate prefixes defined in the user configuration file, stopping execution /amp/oradepot/dbupgrade/transient/autoupgrade/configfiles/DBNAME_config.cfg – UserConfigFileParser.process
Any help is greatly appreciated. Cheers,
Colby
Colby,
please share your config file.
Cheers,
Mike
Hello Mike,
Unfortunately we have same problem with 19.9 AutoUpgrade version. What wold be solution for that?
Regards,
Marcin.
My config file looks following:
# global
global.autoupg_log_dir=/oraacfs/upgrade/ICTEST1
global.drop_grp_after_upgrade=no
# database ICTEST1
# upg1.dbname=ICTEST1_011
upg1.start_time=NOW
upg1.source_home=/u01/app/oracle/product/12.1.0.2/dbhome_1
upg1.target_home=/u01/app/oracle/product/19.0.0.0/dbhome_1
upg1.sid=ICTEST1_0111
upg1.log_dir=/oraacfs/upgrade/ICTEST1
upg1.upgrade_node=sidm01db01.corproot.net
upg1.target_version=19
upg1.run_utlrp=yes
upg1.timezone_upg=yes
Hi Macin,
do you have an SR number for me please?
Thanks,
Mike
Dear Mike,
a customer of mine is facing the same issue with autoupgrade.jar 19.9.0 (he has this problem only with one database, the other ones worked fine) on Windows. The config file looks good.
Is there already a solution/workaround available?
Best regards and thanks a lot
Joerg
Hi Joerg,
as discussed via email, 19.9.1 solved the problem. SR was closed by your customer in 2:45 hrs after it was opened 🙂
Cheers,
Mike
Hi Mike
What do you think about this affirmation: I saw it somewhere
/** there is really no reason for using DBUA anymore. It is a remnant from the past, and in many other situations you would be much better off with AutoUpgrade.” **/
Is it true that dbua is for the past?
Do you have any recommendation about using dbua or autoupgrade
Thanks
I personally have a clear recommendation – and this is not because our teams owns “autoupgrade”.
It is simply because of all the issues I saw with DBUA over the years leaving customers frustrated with a not-so-good upgrade experience.
Then people complained to me saying “Upgrade is not working” etc. Well, just do a quick search on the blog for “DBUA” and you will see what I mean.
And I didn’t even blog about all the issues lately anymore.
I’m telling you this from hundreds of workshops over the past 10 years with over 10,000 customers.
Cheers,
Mike
Hi Mike,
I am exploring autoupgrade utility to upgrade from 12c to 19c for both Linux as well as Windows platform. Below are the issues I am facing
For Linux: Process gets in hang state in analyze mode. Please see below snip from log
2020-02-26 03:26:21.294 WARNING The process ended with an exit value 1 and the following output
[AutoUpgrade detected no process stdout or stderr] – ExecuteProcess.doCmds
2020-02-26 03:26:21.298 WARNING $ORACLE_HOME/bin/orabase execution failed due to $ORACLE_HOME/install/orabasetab not containing the proper content. – OracleHomeValidator.checkOraBaseTab
2020-02-26 03:37:23.792 WARNING ————————————————-
job 100 has not shown progress in last 10 minutes
I see below message in autoupgrade_user.log
2020-02-26 03:26:20.233 INFO
build.version:19.7.5
build.hash:04dd9f2
build.date:2020/02/11 15:28:49
build.max_target_version:19
build.type:production
build.label:HEAD
2020-02-26 03:26:20.299 INFO Loading user config file metadata
2020-02-26 03:26:21.297 WARNING $ORACLE_HOME/bin/orabase execution failed due to $ORACLE_HOME/install/orabasetab not containing the proper content.
2020-02-26 03:26:21.299 INFO Oracle Base utility failed to determine ORACLE_BASE for ORACLE_HOME /opt/aptare/oracle19c
2020-02-26 03:26:21.943 INFO The target_version parameter was updated from 19 to 19.3.0.0.0 due to finding a more accurate value.
2020-02-26 03:26:21.945 INFO This database does not support restoration using a GRP. Restoration support is being disabled.
2020-02-26 03:26:22.369 INFO Finished processing dbEntry upg1
2020-02-26 03:26:22.426 INFO Current settings Initialized
2020-02-26 03:26:22.589 INFO Starting
2020-02-26 04:07:02.567 INFO Abort job: [100][scdb]
2020-02-26 04:07:02.572 WARNING Executing kill switch of [100][scdb]
2020-02-26 04:07:02.610 ERROR Unexpected exception aborting job [100][scdb][null]
2020-02-26 04:07:07.062 INFO Closing
Please note – I am not a DBA, I am exploring this to automate our upgrade process. Also, we use SE in our product.
For Windows, getting below erorr
AutoUpgrade tool launched with default options
Processing config file …
Unable to determine DB unique name for entry upg1
Thanks,
Mangesh
Hi Mangesh,
please open an SR – unfortunately I can deliver what Oracle Support does.
Thanks and sorry for the inconvenience
Mike
PS: The error on Window most likely is either coming from using the wrong database name in the config file (it has to be the DB_UNIQUE_NAME) or from a user context issue by using AutoUpgrade with the wrong OS user.
Thank you Mike for your prompt response. I will open SR to Oracle support. Just to inform you that I am able to upgrade to 19c using DBUA successfully.
Hi Mike, I have some general questions about Autoupgrade tool.
1- In case of CDB and PDBS – Do I need to provide only CDB related parameter in UPG1 local section?
( autoupgrade will auto upgrade CDB including ALL PDBs, right?)
2- If I have multiple DBs ( CDB or non-CDB) running on same server ( does not matter sharing same Oracle home or using different Oracle Homes) , I can provide individual DBs as UPG1, UPG2 etc… and It will upgrade all in parallel?
3- In single config file, can we upgrade multiple DBs ( does not matter using same or different Oracle homes) to different Target version, like some to 18.5, some to 19.3, or some to 19.5 etc?
5- Lastly, is Standby upgrade supported in current Autoupgrade release?
Thanks
John
Hi John,
1- no you don’t have to – and currently everything is treated for the entire piece, CDB with all PDBs. And AutoUpgrade will upgrade everything.
2- yes, it will – just use the different SIDs for each of them
3- yes, you can
4/5- yes, it is – but you need to disable the DG broker manually, and ideally defer the log transport, then after everything is done, enable the log transport and enable the DG Broker, too.
Cheers,
Mike
Hi Mike,
is there any option to manually overwrite the checks being executed during the “prechecks” and “postchecks” ? We have an option for prefixups and postfixups but we don’t have any control over the prechecks and postchecks.
The issue we sessing is, one of the sql was taking longer time after the upgrade to 19c from 11204, this sql is being executed as part of precheck and postcheck phase. SQL execution plan got changed after 19c upgrade and it was taking around 4 hours, while the same sql was taken only 40 minutes during the precheck phase.
I don’t see any benefit of running this check, actually it is being executed 4 times (twice during prechk and twice during post checks)
We endedup killing the job and created a sql profile and restarted, but we lost almost 3 hours of valuable downtime during the change window due to this issue.
with segments as (select tablespace_name, inuse from( select ds.tablespace_name, row_number() over (partition by tablespace_name order by 1) rn, round(nvl(sum(ds.bytes) over (partition by ds.tablespace_name),0)/1024,2) as inuse from sys.dba_segments ds) where rn=1),tmp_ts_query as ( select null group_name, tablespace_name, bytes, maxbytes from sys.dba_temp_files where tablespace_name = ‘TEMP’ ),ts_qresult as ( SELECT /*+ MATERIALIZE */ nvl(dtf.group_name, dt.tablespace_name) as name, dt.contents as contents, decode(dt.contents,’TEMPORARY’,1,0) as temporary, decode(dt.extent_management,’LOCAL’,1,0) as localmanaged, nvl(ds.inuse,0) as inuse, nvl(round(sum(case dt.contents when ‘TEMPORARY’ then dtf.bytes else ddf.bytes end)/1024, 2),0) as alloc, nvl(round(sum(case dt.contents when ‘TEMPORARY’ then decode(dtf.maxbytes, 0, 0, dtf.maxbytes-dtf.bytes) else decode(ddf.maxbytes, 0, 0, ddf.maxbytes-ddf.bytes) end)/1024,2),0) as auto FROM sys.dba_tablespaces dt left join segments ds on (dt.tablespace_name = ds.tablespace_name) left join sys.dba_data_files ddf on (ddf.tablespace_name = dt.tablespace_name) left join tmp_ts_query dtf on (dtf.tablespace_name = dt.tablespace_name) WHERE (dt.tablespace_name in (‘UNDOTBS1’, ‘SYSTEM’,’SYSAUX’)) or (dt.tablespace_name in ( SELECT distinct T.tablespace_name FROM sys.dba_queues Q, sys.dba_tables T WHERE Q.queue_table=T.table_name AND Q.owner = T.owner)) or dtf.tablespace_name is not null group by nvl(dtf.group_name, dt.tablespace_name), dt.contents, dt.extent_management, ds.inuse )select tq.name || ‘#’ || tq.contents || ‘#’ || tq.temporary || ‘#’ || tq.localmanaged || ‘#’ || tq.inuse || ‘#’ || tq.alloc || ‘#’ || tq.auto || ‘#’ || (tq.alloc+tq.auto) avail from ts_qresult tq order by tq.name;
Hi Rakesh,
I’ve sent the offending query to the team asking if somebody has seen this already. But I may ask you to upload your logs to an SR as well. So please stay tuned.
Regarding overriding of checks, yes, you can.
Please see slide 38 of the AutoUpgrade talk I’d given at UKOUG and DOAG:
https://mikedietrichde.com/slides/#DOAG19
Cheers,
Mike
Hi Rakesh,
can you please open an SR and upload the entire log directory?
We’ve had some issues with this query in the past and will get back to you directly then.
Just mail me the SR number once you uploaded the log tree:
mike.dietrich .——… oracle.com
I will get back to you then directly.
Cheers,
Mike
Hi Mike,
Thanks for replying. I already opened an SR and uploaded the entire directory.
I don’t see any option to check and manage the checks run as part of the “precheck” and “postcheck” phase. I see we can manage the “prefixups” and “postfixups” but not the checks run during the “precheck/postcheck”
The above query was running as part of “precheck” so currently there is no option for us to disable it.
Can you provide me your email address so that i can email the SR number. The email address you mentioned above is not valid one.
Sorry, i think i got your email address and sent an email with SR information.
-Thanks,
Rakesh
Hi Mike, Rakesh
I am seeing same issue after upgrading 12c1 to 19.10 during running postupgrade_fixups.sql. Same query is running now since 3 hours. I not received any error yet as session is still running.
May I know if you have any solution on this.
Also, I found this note “19c :postupgrade_fixups.sql fails with error “ORA-04023: Object with segments as ..” (Doc ID 2650211.1)” where they are suggesting to apply patch 30576112 to 19c ORACLE_HOME but this patch is already applied.
in same note suggesting to use “_optimizer_ansi_rearchitecture to TRUE” and then run fixup script.
DB is on exadata without standby.
Hi Jagdish,
did you try the underscore?
Otherwise we need the SR number and all autoupgrade logs which you collect with
java -jar autoupgrade.jar -config yourconfig.cfg -zip
Thanks,
Mike
Mike,
I have a question about relink: Can I use “relink as_installed” with RAC databases. It throws ORA-439 in 18c(12.2) and above. It was fine till 12.1
Where can I find the differences between Relink all and relink as_installed
Is relink as_installed only meant to relink client installation in 12.2 and above ?
Hi Sunil,
please open an SR – I think I saw this on an email as well but as far as I’m aware something has been changed.
Thanks
Mike
Hi Mike,
I’m performing an Oracle upgrade from 12.1.0.2 to 19 (including a server move) on Windows. On the target server I have started autoupgrade and am getting an error “Opening Database in upgrade mode failed”.
When I try to connect to the database using “sqlplus / as sysdba” (under a domain user in the local ORA_DBA group) I get a “ORA-12638: Credential retrieval failed” error. I see the AutoUpgrade has created a new Windows service called “AutoUpgradeOracleService” (which is running instead of the usual OracleService) and this new service is running as a Local System account. If I change it do run as my domain Oracle Home user account and restart it, then the error from “sqlplus / as sysdba” goes away and I am able to successfully start the database in upgrade mode. However after doing this, if I restart AutoUpgrade it shuts down the database, delete the service and recreates it using a Local System user account again and then runs into the same error.
I haven’t been able to find any documentation on how to get Autoupgrade to use a domain Oracle Home user account. Additionally I’m not sure how to resolve the ORA-12638 error with the service running as a Local System account.
I’ve tried having setting SQLNET.AUTHENTICATION_SERVICES to NONE, ALL and NTS and it doesn’t work with each. With NONE the error message changed to a ORA-1017 instead of ORA-12638 but autoupgrade still failed.
Any suggestions?
Hi Jason,
let me circle this through the team and then get back to you.
Cheers,
Mike
Hi Mike,
I am also facing the same issue. Hopefully we get solution quickly.
Thanks Jason for posting this query
Thank you,
Mangesh
Mangesh,
we are checking – and I will put it on the blog.
THanks,
Mike
Hi Mike,
When can we expect blog on this. I am stuck and have no clue to move further for Windows. Also, i see very less examples are for Windows so it is difficult to understand what is going wrong when using AutoUpgrade utility for windows server. Appreciate your help here 🙂
Hi Mangesh,
I have no Windows system but I will ask Daniel to do a blog post about it. But it may take a bit.
In between, please raise an SR.
Thanks,
Mike
I hope you have solved your issues by now but experienced the same exact issue on one of my test DBs.
Not sure if Mike’s colleague managed to create a blog related to this but the only way I figured out to get out of it was to change the Log On As of the OracleService to Local System and run autoupgrade using the -deploy mode again.
Once it’s ready, you still need to create the new service using oradim.exe and there it will ask you for the password for the domain oraclehomeuser. The new 19c OracleService will be created with the domain user then
Did this issue ever get resolved? I get the ORA-12638, but during the “dbupgrade” stage, which fails as a result. AutoUpgrade works when I set Oracle Home User to “LocalSystem” during the initial install, but then can’t ever set it to anything else after that. Running “oradmin” to create the “real” service doesn’t work/prompt because OHU is “LocalSystem”.
Do you have more detailed examples of the 11g to 19c Autoupgrade with a dataguarg installed?
Hi Griffin,
for the database it is very simple:
1. Disable the Data Guard Broker
====> https://mikedietrichde.com/2017/09/12/upgrade-disable-data-guard-broker/
2. Defer the log transport (see the doc links)
====> https://mikedietrichde.com/2017/09/12/upgrade-disable-data-guard-broker/
So the list would look like:
1. Install the new software on the standby host and patch it with the newest RU
2. Install the new software on the primary host as well and patch it with the newest RU
3. Download the most recent autoupgradejar – run -mode analyze on source
4. Disable the Data Guard Broker configuration
5. Shutdown everything on the standby and restart listeners etc with the new environment
6. Defer the log transport
7. Mount the physical standby and start redo apply
8. Use autoupgrade to upgrade production
9. Enable log transport
10. Enable the DG Broker again
Cheers,
Mike
Mike,
I contacted you before regarding an 18c issue. Since my last contact we have decided to upgrade to 19c. I was able to install the Oracle 19c binaries from the downloaded Solaris.zip file. I successfully extracted the .zip files into my new 19c ORACLE_HOME. Unfortunately, when using the autoupgrade.jar I am having a problem. The upgrade fails with error ” “It is not possible to determine the target ORACLE_BASE value to RMAN because the target ORACLE_HOME directory does not exist for database. It was not possible to determine the target ORACLE_BASE for entry upg1”. I filed SR 3-22988707091 and I am still waiting on some direction as to how to proceed. Also, if this is a bug, I am concerned how I can do an upgrade using the autoupgrade.jar utility in my data gaurd environment.
I imagine you have resolved it by now, but in my case, I had a typo in the config file so the $ORACLE_HOME for upg1.target_home did not match what was in (19c_ORA_HOME)/install/orabasetab, it is a very obscure and terribly worded error. Orabasetab gets set at install with information from the inventory.
Cheers
I have a question.
Our Grid binaries are still in 12.1. so when we tried installing the oracle binaries it didn’t like it
[FATAL] [INS-35363] Incompatible version of Oracle Grid Infrastructure detected.
CAUSE: The active version of Oracle Grid Infrastructure detected on the system (12.1.0.2.0), is lower than the version of Oracle Real Application Database software that you were attempting to install(19.0.0.0.0).This is not supported.
ACTION: If Oracle Grid Infrastructure is of older version, only Oracle Single Instance Database software can be installed. If you want to install Oracle Real Application Database software,you should upgrade Oracle Grid Infrastructure to same or higher version first.
My question is
Is there anyway to run autoupgrade analyze so we do some preparation before the grid upgrade
AutoUpgrade does not handle GI upgrades. You need to upgrade GI always at first. You can’t upgrade your database with AutoUpgrade to 19c when your GI is 12.1.0.2.
If that is an ongoing issue, please work with Oracle Support to get this fixed.
Cheers,
Mike
Downloaded the latest version
The manual upgrade went fine for me but when I was testing autoupgrade — I am getting below message – ANALYZE phase
Session ID: 31 Serial number: 31982
] [PDB$SEED]
at oracle.upgrade.commons.sql.ExecuteSql.quickSQL(ExecuteSql.java:629)
at oracle.upgrade.commons.sql.ExecuteSql.quickSQL(ExecuteSql.java:569)
at oracle.upgrade.commons.dbinspector.tools.Check.presentInDb(Check.java:251)
at oracle.upgrade.commons.dbinspector.tools.CheckTrigger.call(CheckTrigger.java:73)
at oracle.upgrade.commons.dbinspector.tools.CheckTrigger.call(CheckTrigger.java:41)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
[oracle@devops auto]$ locale
LANG=en_US.UTF-8
LC_CTYPE=”C”
LC_NUMERIC=”C”
LC_TIME=”C”
LC_COLLATE=”C”
LC_MONETARY=”C”
LC_MESSAGES=”C”
LC_PAPER=”C”
SQL> show pdbs
CON_ID CON_NAME OPEN MODE RESTRICTED
———- —————————— ———- ———-
2 PDB$SEED READ ONLY NO
3 KACHU READ WRITE NO
Please open an SR. Seriously – I can’t do debugging via the blog.
See here:
https://mikedietrichde.com/2020/04/08/troubleshooting-restoring-and-restarting-autoupgrade/
Cheers,
Mike
Mike,
I am using the new 19c autoupgrade feature to upgrade from 12.2.0 to 19.3.0. The binaries for 19.3.0 I am using was downloaded in a .zip from Oracle. I was able to successfully unzip the binaries into the new ORACLE_HOME for a)rcat, b)primary database, c) physical standby database. I followed your documentation to disable the data gaurd broker and set dg_broker_start=false. I prepared each database server for the upgrade by following the upgrade guide and ensuring all Solaris Sparc packages were installed and that all parameters were set correctly. Next, I create a autoupgrade cfg file so that I could a)upgrade the rcat db, b) upgrade the primary database. I ran the autoupgrade update command after doing an analyze and prefixs. The upgrade failed due to ora-20001. I was able to work through error, Per Note: (Doc ID 1602089.1), I had to a)Register oracle home using runInstaller or run $ORACLE_HOME/oui/bin/attachHome.sh
b) Re-run Datapatch
I then ran the autoupgrade command again and once again, the message came back that only 1 database will be upgrade. The command ran to completion.
I have filed SR 3-22988707091. I understand that the installation engineers are online later in the afternoon however, I am really pressed to address this as soon as possilble.
a. Do I need to attach the new ORACLE_HOME when using the downloaded .zip file from Oracle prior to running autoupgrade?
b. Why isn’t autoupgrade seeing the 2nd database?
I have placed my autoupgrade config file contents in the SR.
Thanks!
Ken
Hi Ken,
could you solve this with support? Sorry for my late response, but at the moment I “drown”.
Cheers,
Mike
Hi Mike,
How AutoUpgrade deal with wallet ? Is there any suggestion if i am going to upgrade 12c database to19c using AutoUpgrade tool with walllet ?
Hi Sandeep,
wallets need to be autologon – and I will put out the blog post in a few days (working on it).
Cheers,
Mike
Hi Mike,
I used one of your Hands on lab. Autoupgrade config file and was able to upgrade and plug the database as a PDB on an existing CDB. This does not copy or move the datafiles under CDB GUID. Is there an autoupgrade config file setting that I can use to copy or move the datafiles under PDB GUID.
upg2.dbname=upgd05t
upg2.start_time=NOW
upg2.source_home=/u01/app/oracle/product/12.1.0.2.200114
upg2.target_home=/u01/app/oracle/product/19.6.0.0.200114
upg2.sid=upgd05t
upg2.log_dir=/home/oracle/upg_logs
upg2.upgrade_node=localhost
upg2.target_version=19
upg2.timezone_upg=no
upg2.restoration=no
upg2.target_cdb=CDB2
I would like to have the PDB datafile look like :
+APP/CDB2//DATAFILE/*******
Thanks
Vaithianathan
See the blog post being published today for the new (and now officially available and supported) options:
https://mikedietrichde.com/2020/05/20/autoupgrade-and-plug-in-to-a-cdb-with-a-single-command/
Thanks,
Mike
Hi Mike,
Using autoupgrade version 19.9, dated 2020/04/23 on AIX 7.1. Done a few successful upgrades – thanks 🙂
This one is failing in the analyse phase with:
Database check with running exception” (conName=”monproda”,stage=”PRECHECKS”,checkName=”DISK_SPACE_FOR_RECOVERY_AREA”)
and in the log file i see:
2020-06-03 16:22:51.354 INFO Analyzing monproda, 78 checks will run using 32 threads
2020-06-03 16:22:53.407 ERROR Unable to determine the amount of usable free space available on
2020-06-03 16:22:54.275 ERROR
============================ check info ============================
[monproda][DISK_SPACE_FOR_RECOVERY_AREA][ERROR]
============================ check info ============================
=========================== trace start ==============================
Exception: IOException
Err message: Invalid directory
java.io.IOException: Invalid directory
at oracle.upgrade.commons.helpers.Utilities.getUsableSpace(Utilities.java:608)
at oracle.upgrade.commons.dbinspector.checks.disk_space_for_recovery_area.checkCode(disk_space_for_recovery_area.java:121)
…
Recovery area is not used at this customer but I created it to do the upgrade, same way as the previous successful upgrades.
The directory is created, readable and writable by oracle. In fact I used the same directory as the datafiles so it must be ok.
12.1.0.2 stand-alone, non-db database with very few options.
‘df -g’ shows plenty of space available.
Querying v$recovery_file_dest and v$recovery_area_usage succeeds.
Any ideas?
Thanks
Stuart
Hi Stuart,
can you try creating a GRP by yourself, and drop it afterwards?
Can you check parameters db_recovery_file_dest_size and db_recovery_file_dest?
Do you use upg1.restoration=NO explicitly, or do you run with the default assuming that the database in in ARCHIVELOG mode?
And finally, do you have an SR open where we can check the logs?
Thanks,
Mike
Hi Mike, Thank you for the reply. It was indeed the db_recovery_file_dest parameter. There was a non-printable character in it ! After resetting this, it worked perfectly.
Thanks,
Stuart
Hi Stuart,
awesome – thanks for getting back, and happy that you solved it so quickly.
Thanks,
Mike
Hi Mike,
I am using latest version of autoupgrade from April 2020 but still getting below error even i have enough free space on FRA.
2020-06-04 08:50:16.373 ERROR The following checks have ERROR severity and no fixup is available or
the fixup failed to resolve the issue. Fix them manually before continuing:
emprod MIN_RECOVERY_AREA_SIZE
2020-06-04 08:50:16.401 ERROR Dispatcher failed: AutoUpgException [UPG-1312]
2020-06-04 08:50:16.404 INFO Starting error management routine
2020-06-04 08:50:16.406 INFO Ended error management routine
2020-06-04 08:50:16.407 ERROR Error running dispatcher for job 102
Hi Manoj,
we need an SR please. I can’t debug all potential issues via my own upgrade blog.
Thanks,
Mike
Hello Mike i have got
Cause: Database post upgrade failed
2020-06-05 03:54:50.170 ERROR Dispatcher unable to recover from error
Error: UPG-1500
I can’t find any info about this error
Yuriy,
please open an SR and upload all logs:
https://mikedietrichde.com/2020/04/08/troubleshooting-restoring-and-restarting-autoupgrade/
Cheers,
Mike
Hi Mike
I’m trying to do an upgrade from
Source: 11.2.0.4
Target: 19.7
on a two node RAC + dataguard (also in RAC).
while running the analyze step, i encountered the following error:
2020-06-10 08:01:04.170 ERROR Error occurred while running the dispatcher for job 101
Cause: Loading the current state of the database failed. – AutoUpgDispatcher.run
2020-06-10 08:01:04.170 ERROR The dispatcher has failed due to:
Error: UPG-1319
[Unexpected Exception Error]
Cause: Loading the current state of the database failed.
as i check the details from the logs, the error started here
2020-06-10 08:01:03.947 INFO Executing SQL [SELECT dest_name || ‘#’ || destination || ‘#’ || status FROM sys.v$archive_dest WHERE upper(status) = ‘VALID’ AND destination IS NOT NULL AND destination ‘USE_DB_RECOVERY_FILE_DEST’ AND destination NOT IN ((select value from sys.v$parameter where name = ‘db_recovery_file_dest’ AND value IS NOT NULL) union (select ‘+’||name from sys.v$asm_diskgroup where usable_file_mb*1024*1024 >= 3662675968 )) AND target=’PRIMARY’ AND rownum = 1;] in [DUKRZTES1, container:null] – ExecuteSql$SQLClient.run
2020-06-10 08:01:04.157 ERROR null – ChecksToolBox.initializeDBUI
java.lang.NullPointerException
at oracle.upgrade.commons.dbinspector.tools.SteadyData.find_archive_dest_info(SteadyData.java:473)
at oracle.upgrade.commons.dbinspector.tools.SteadyData.initTablespaceInfo(SteadyData.java:336)
I cant seem to find any reference that points to UPG-1319 error. Can help to check why I’m encountering this error?
also, just to check about 11.2.0.4 upgrade to 19c, will it become multi-tenant database (CDB + PDB for the application data) after the upgrade?
Hi Erwin,
please open an SR, upload the logs as I describe here:
https://mikedietrichde.com/2020/04/08/troubleshooting-restoring-and-restarting-autoupgrade/
and send me the SR number please, either via the blog or via email (mike.dietrich —- at —– oracle.com)
Cheers,
Mike
Hi there!
Here is a hint for everyone who may end up the same googling (well, to be exact it was duckduckgoing) than I:
The configuration file example in this: https://docs.oracle.com/en/database/oracle/oracle-database/19/upgrd/using-autoupgrade-utility-perform-checks.html#GUID-2D8ACE4F-1434-4059-AE00-3B73AB7EFB56
documentaion has a typo.
> global.autoup_log_dir=/home/oracle/autoupgrade is missing a _g_ in autoupg
Valid: global.autoupg_log_dir
autoupgrade.jar in version 19.9.0 wasn’t very helpful with its error messages (this output is with debug parameter):
_____________________________________
oracle.upgrade.autoupgrade.config.links.CLIOptionsParser
oracle.upgrade.autoupgrade.config.links.GlobalConstants
Internal Error has occurred. Please contact Oracle support.
java.lang.NullPointerException
at java.io.File.(File.java:277)
at oracle.upgrade.autoupgrade.config.links.GlobalConstants.process(GlobalConstants.java:160)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.CLIOptionsParser.process(CLIOptionsParser.java:165)
at oracle.upgrade.autoupgrade.config.CLIProcessor.processCLIParams(CLIProcessor.java:127)
at oracle.upgrade.autoupgrade.boot.AutoUpgMain.(AutoUpgMain.java:124)
at oracle.upgrade.autoupgrade.boot.AutoUpgMain.newInstance(AutoUpgMain.java:141)
at oracle.upgrade.autoupgrade.boot.Boot.main(Boot.java:43)
____________________________________
Eventually I downgraded autoupgrade to 19.8.1 which gave a more specific error message:
___________________________________
oracle.upgrade.autoupgrade.config.links.CLIOptionsParser
oracle.upgrade.autoupgrade.config.links.GlobalConstants
No parameter ‘global.autoupg_log_dir’ found in config file, using /ACFS/home/hlaoa01/autoupg_default
oracle.upgrade.autoupgrade.config.links.ZipMaker
oracle.upgrade.autoupgrade.config.links.LoggerMaker
2020-06-16 15:21:59.864 DEBUG oracle.upgrade.autoupgrade.config.links.ContextFinder – ContextFinder.process
2020-06-16 15:22:00.076 DEBUG oracle.upgrade.autoupgrade.config.links.UserConfigFileParser – UserConfigFileParser.process
[…..]
__________________________________
I’ll open a SR reagrding the documentation as soon as autoupgrade is running in deploy mode. 😉
Best Regards,
Benedikt
Hi Benedikt,
sorry sorry sorry … and I’ll send this straight to our doc writer.
This one has clearly slipped through the cracks.
I’m happy that my example has it right 🙂
https://mikedietrichde.com/2019/06/13/create-and-adjust-the-config-file-for-autoupgrade-19c/
Cheers,
Mike
And no need to open an SR – I sent it to our doc writer.
Thanks again
Mike
Hi Mike,
after an error during the run of the Unix shell script of the user-defined postupgrade action, autoupgrade starts the script two more times. Autoupgrade version is 19.9.0.
2020-06-18 14:21:03.591 ERROR Error running dispatcher for job 101
Cause: Error during user-defined postupgrade action
2020-06-18 14:21:03.592 ERROR Dispatcher unable to recover from error
Error: UPG-1511
[Unexpected exception error]
Cause: Error during user-defined postupgrade action
For further details, see the log file located at …/101/autoupgrade_20200618_user.log more than three times,
Review the logs in …/autoupgrade/upg_logs/T922/101 for job 101
In my opinion, the script should only be started once by autoupgrade.
Works as designed or a bug?
Regards
Uwe
Uwe,
sounds like a unwanted issue.
Is this a CDB/PDB?
Would you mind to zip all the logs together and upload them to an SR and mail me the SR number?
Thanks,
Mike
Hi Mike,
this is a Non-CDB database. You have mail.
Regards
Uwe
Thanks Uwe – I will get back to you via email once I checked the logs.
Cheers,
Mike
All autoupgrade.jar files newer than version 19.6 (20190823) immediately fails to execute with the following error:
There are duplicate prefixes defined in the user configuration file, stopping execution
All the autoupgrade.jar files newer than version 19.6 generate the following error immediately after the run:
There are duplicate prefixes defined in the user configuration file, stopping execution.
I opened a ticket with Oracle Support but they mentioned that they do not have much knowledge on the utility.
Any advice why that issue happening in new releases?
Can you give me the SR number please?
Thanks,
Mike
SR 3-23338707681 : There are duplicate prefixes defined in the user configuration file, stopping execution
Thanks – I think the guys working on it already.
Cheers,
Mike
Hi Mike,
Is it possible to show the upgrade progress in percentage when using utility in noconsole mode? With current message we are not able to understand how tool is progressing without checking the log
Thanks,
Mangesh
Hi Mangesh,
you can tail the status log which (if I remember correctly) or use an app to read the JSON log.
Cheers,
Mike
Hi Mike,
I want to upgrade several databases running on one server with one autoupgrade run. The before action linux shell script is identical for all databases, but it must run in the environment (“$ORACLE_HOME”, “$ORACLE_SID”, etc) of the database.
I use the local parameter “before_action” in the config file, because I find this in the documentation in the upgrade guide:
“In contrast to the global before_action parameter, the local before_action parameter can specify a SQL script, which can run on the database in the source database Oracle home, using the earlier release Oracle Database binaries.”
Example config.cfg
==================
#
# Database number 1
#
upg1.dbname=DB1_SITE1
upg1.start_time=19/06/2020 19:00:00
upg1.source_home=/u01/app/oracle/db1/product/12.2.0.1
upg1.target_home=/u01/app/oracle/db1/product/19.0.0
upg1.target_version=19
upg1.sid=DB1
upg1.log_dir=/u01/app/oracle/db1/admin/upgrade_19c/autoupgrade/upg_logs/
upg1.upgrade_node=localhost
upg1.before_action=/scripts/autoupgrade/19c_before_upgrade.sh Y
…
#
# Database number 2
#
upg2.dbname=DB2_SITE1
upg2.start_time=19/06/2020 20:00:00
upg2.source_home=/u01/app/oracle/db2/product/12.2.0.1
upg2.target_home=/u01/app/oracle/db2/product/19.0.0
upg2.target_version=19
upg2.sid=DB2
upg2.log_dir=/u01/app/oracle/db2/admin/upgrade_19c/autoupgrade/upg_logs/
upg2.upgrade_node=localhost
upg2.before_action=/scripts/autoupgrade/19c_before_upgrade.sh Y
…
My questions are:
1. If I use a Unix shell script here, does the script run in the environment of the database I‘m upgrading?
I mean knows the script “$ORACLE_HOME”, “$ORACLE_SID” etc.? With an SQL script it works according to the documentation in the upgrade guide.
It wasn’t like that in my tests with the before action script. $ORACLE_SID and $ORACLE_HOME wasn’t set in the shell environment before the autoupgrade run.
Did I do something wrong or is it an issue?
2. If not, how can I use the script to find out which database is being upgraded?
3. Can I pass parameters to the script in the config file?
In this example „DB1“ is the SID, „Y“ is the flag to stop autoUpgrad if the script fails.
upg1.before_action=/u01/app/oracle/admin/upgrade_19c/autoupgrade/’19c_upg1_before_upgrade.sh DB1′ Y
I couldn’t do that in my tests.
Thank you.
Regards
Uwe
Hi Uwe,
sorry for getting back so late to you. I have a huge backlog …
See this blog post:
https://mikedietrichde.com/2020/03/18/autoupgrade-and-the-compatible-parameter/
I explain how setting the environment works. And I link also the Y option, and how to set your environment.
Cheers,
Mike
Hi Mike,
Thank you.
As already written, the Unix script does not know which database to upgrade and therefore cannot set the environment.
In my opinion, autoupgrade should start the Unix shell scripts, which are determined with the local parameters “before_action” and “after_action”, also in the environment of the source or target environment. According to the documentation in the upgrade guide, this was implemented for the SQL scripts.
Regards
Uwe
Hi Uwe,
yes, but you can have hundreds of databases within the same source environment. That is not going to fly I’d guess.
Cheers,
Mike
Hi Mike,
that’s right. But what changes with every upgrade is $ORACLE_SID. If the script had this information, it would be easy to load the rest of the environment.
Regards
Uwe
Hi Mike,
Am getting below error while doing analyze and am using latest autoupgrade jar file
am trying to upgrade 12.1 to 19.3
[oracle@racdb01 bin]$ ./java -version
java version “1.8.0_201”
Java(TM) SE Runtime Environment (build 1.8.0_201-b09)
Java HotSpot(TM) 64-Bit Server VM (build 25.201-b09, mixed mode)
[oracle@racdb01 bin]$ pwd
/u01/app/oracle/product/19.0.0.0/db_1/jdk/bin
[oracle@racdb01 bin]$ ./java -jar /u01/app/oracle/product/19.0.0.0/db_1/rdbms/admin/autoupgrade.jar -config /u01/software/sample_config.cfg -mode analyze
Previous execution found loading latest data
Internal Error has occurred. Please contact Oracle support.
java.lang.IllegalStateException
at oracle.upgrade.autoupgrade.utils.collector.state.KaiserWriter.(KaiserWriter.java:90)
at oracle.upgrade.autoupgrade.utils.collector.state.KaiserWriter.newRecoveryInstance(KaiserWriter.java:99)
at oracle.upgrade.autoupgrade.config.links.ContextFinder.recoverData(ContextFinder.java:176)
at oracle.upgrade.autoupgrade.config.links.ContextFinder.findPreviousJobs(ContextFinder.java:163)
at oracle.upgrade.autoupgrade.config.links.ContextFinder.process(ContextFinder.java:87)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.LoggerMaker.process(LoggerMaker.java:68)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.AutoUpgradeLocker.process(AutoUpgradeLocker.java:74)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.ZipMaker.process(ZipMaker.java:56)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.GlobalConstants.process(GlobalConstants.java:202)
at oracle.upgrade.autoupgrade.config.CLILink.nextLink(CLILink.java:53)
at oracle.upgrade.autoupgrade.config.links.CLIOptionsParser.process(CLIOptionsParser.java:165)
at oracle.upgrade.autoupgrade.config.CLIProcessor.processCLIParams(CLIProcessor.java:127)
at oracle.upgrade.autoupgrade.boot.AutoUpgMain.(AutoUpgMain.java:124)
at oracle.upgrade.autoupgrade.boot.AutoUpgMain.newInstance(AutoUpgMain.java:141)
at oracle.upgrade.autoupgrade.boot.Boot.main(Boot.java:43)
[oracle@racdb01 bin]$
[oracle@racdb01 bin]$ ./java -jar /u01/app/oracle/product/19.0.0.0/db_1/rdbms/admin/autoupgrade.jar -version
build.hash 255dd7d
build.version 19.9.0
build.date 2020/04/23 15:01:36
build.max_target_version 19
build.supported_target_versions 12.2,18,19
build.type production
[oracle@racdb01 bin]$
My recommendation:
Patch your database home instead of using 19.3.0.
Then cleanout the information from previous runs:
https://mikedietrichde.com/2020/04/08/troubleshooting-restoring-and-restarting-autoupgrade/
Cheers,
Mike
Hi Mike, even with latest autoupgrade.jar (20200710) I still get following message when trying to upgrade from 12201 to 19c (both HOMES are patched to latest patch bundle level):
$ $ORACLE_HOME/jdk/bin/java -jar autoupgrade.jar -config upg_hvbpgva.cfg -mode analyze
AutoUpgrade tool launched with default options
Processing config file …
It was not possible to determine the database version for entry upg1
When I look in the error log file I see:
2020-07-21 11:21:42.013 ERROR It was not possible to determine the version value
of database hvbpgva from the returned value select version_full from sys.v$inst
ance – UpgradeConfigDBValidator.findDBVersion
AND THIS IS CORRECT
There is no field VERSION_FULL in SYS.V$INSTANCE in Oracle 12c !
This is my config file used:
global.autoupg_log_dir=/oradumps/upgrade/log
#
# Database number 1
#
upg1.dbname=mydb
upg1.sid=mydb
upg1.start_time=NOW
upg1.target_home=/u01/app/oracle/product/itc19c
upg1.source_home=/u01/app/oracle/product/itc12201_ee
upg1.target_version=19
upg1.log_dir=/oradumps/upgrade/log
upg1.upgrade_node=mysystem.mybusiness.nl
upg1.run_utlrp=yes
upg1.timezone_upg=yes
upg1.restoration=no
upg1.target_cdb=cupgdb
upg1.target_pdb_name=PMYDB
upg1.target_pdb_copy_option=file_name_convert=(‘/u02/oradata/mydb’,’/u02/oradata
/cupgdb/pmydb’)
Any help would be appreciated !
Thanks
Thanks John – I will send this to the team rapidly.
Cheers,
Mike
Hi John,
may I ask you to open an SR and upload all the logs please?
Use “java -jar autoupgrade.jar -config yourconfig.cfg -zip” please.
Just paste the SR number here into comments.
Something strange happened here as the query has a fallback in case the column does not exist. And I don’t see this error in any of my environments.
Thanks,
Mike
Hi there, I manage to solve this by following the soltion on MOS Document 2579192.1
Cheers!
Thanks 🙂 Ignore my previous reply.
Cheers,
Mike
Mike,
Could you please explain the difference between Autoupgrade and Fleet PP? If we use FPP then we use update_db to upgrade/patch databases. How does Autoupgrade differ?
Hi Jennifer,
FPP (or previously, RHP) used (as far as I know) DBUA for the database upgrade. This may work, or not.
AutoUpgrade is supposed to be included into FPP instead of DBUA the sooner or later – but it may take a while.
Cheers,
Mike
We are observing two issues while upgrading 11.2.0.4 and 12.1.0.2 Standard Edition versions to Oracle 19c SE (19.3) using Autoupgrade utility on Linux Platform
Issue 1: While accessing sqlplus from newly installed oracle19c binaries we get below issue
“Error in `/opt/aptare/oracle19c/bin/sqlplus’: free(): invalid pointer: 0x000000000244c090”
Details are as below
bash-4.2$ /opt/aptare/oracle19c/bin/sqlplus
/opt/aptare/oracle19c/bin/sqlplus: error while loading shared libraries: libclntsh.so.19.1: cannot open shared object file: No such file or directory
-bash-4.2$ export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/opt/aptare/oracle19c/lib
-bash-4.2$ /opt/aptare/oracle19c/bin/sqlplus
*** Error in `/opt/aptare/oracle19c/bin/sqlplus’: free(): invalid pointer: 0x000000000244c090 ***
It got resolve when using below before calling autoupgrade utility
export MALLOC_CHECK_=0
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/opt/aptare/oracle19c/lib
Use of MALLOC_CHECK is not good idea so just want to check with you any better solution
Issue 2: Autopgrade Utitility fails in deploy mode in dbupgrade step
before calling autoupgrade utility I just set below ev and autoupgrade execution went well till 97%
export MALLOC_CHECK_=0
export LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/opt/aptare/oracle19c/lib
After completion of 97% upgrade process fails with Oracle internal error
2020-07-21 03:00:08.423 INFO End Oracle Home=/opt/aptare/oracle19c Oracle Sid=N/A Run Command =[/opt/aptare/oracle19c/bin/oerr, ORA, 00600] Command=None Filename=None – ExecuteProcess.doCmds
2020-07-21 03:00:08.423 INFO The process ended with an exit value 0 and the following output
00600, 00000, “internal error code, arguments: [%s], [%s], [%s], [%s], [%s], [%s], [%s], [%s], [%s], [%s], [%s], [%s]”
SR is raised for the same – SR 3-23661908481
Are you really upgrading to 19.3.0 (without any RU)?
If this is the case, please install the most recent RU for your platform at first:
https://mikedietrichde.com/2017/10/13/download-assistant-for-rus-rurs-bps-psus-patch-sets-and-releases/
Then try again.
Cheers,
Mike
Thank you Mike for your quick response
May be a silly question, are you suggesting to apply latest RU on 19.3.0? Or We need to apply latest patch for 11.2.0.4 and 12.1.0.2. I am not DBA but we have automated the autoupgrade process and it is perfectly working for 12.2.0.1
Regards,
Mangesh
Hi Mangesh,
why had you applied the latest patch to 11.2.0.4 and 12.1.0.2 BEFORE upgrade?
This shouldn’t be the case.
But for sure you should apply the latest RU to your 19.3.0 home BEFORE upgrade for several reasons:
1. There are a ton of security patches included on top of 19.3.0
2. There are hundreds of bug fixes on top of 19.3.0
3. We see especially a good number of performance topics fixed with 19.8.0.
You just ease your life, make your database more secure and avoid hitting known issues.
Cheers,
Mike
Hi Mike, can one use autoupgrade to apply latest patchset to more than one database at a time?
thanks
Christian
Hi Christian,
not yet – we are working on this.
At the moment, you need to call “datapatch -verbose” by yourself after you started the DB with the new home.
But AU will support this soon.
Thanks,
Mike
Hi Mike ,
I Ran the autoupgrade script with Deploy mode and Fixup failed with MIN_RECOVERY_AREA_SIZE being less than expected .
However the autoupgrade should exit the tool and give back the OS command prompt which i do not see happening and where as it still hangs
Push RETURN – then it gets you back to the command prompt UPG>, and then you can “exit”.
Cheers,
Mike
I understand that … My Say is that i do not see this behaviour with analyze or fixup mode
I am actually try to run the command from script and my script hangs because of it
Hello Mike,
I upgraded my 12.1.0 EE database to 19.5.0 using AutoUpgrade utility.
The data base seems fine, i have no error during upgrade, but i have ora-01017 (i can not connect externally to my database).
The parameter remote_os_authent and SEC_CASE_SENSITIVE_LOGON are true.
Have you an idea concerning this issue, it’s the most important database in the company 🙁
Thanks and best regards,
Houda
See here please:
https://mikedietrichde.com/2017/04/24/having-some-fun-with-sec_case_sensitive_logon-and-ora-1017/
Cheers,
Mike
Hi Mike,
It seems autoupgrade is not ready to work with threaded_execution. When you run autoupgrade in any mode, it does a “sqlplus / as sysdba” connection what it not allowed with threaded_execution (MoS 1639445.1). Although workaround is pretty simple (disable threaded execution) it would be a desirable feature for future versions ( por example, allowing specify connection mode/string in local parameters)
Thx and best regards
Alfredo.
Hi Alfredo,
I will share with the developers.
Thanks!
Mike
Hi Alfredo,
AU uses “/ as sysdba” strictly as it does not ask for a password in order to be fully automated.
When you check for:
https://docs.oracle.com/database/121/REFRN/GUID-7A668A49-9FC5-4245-AD27-10D90E5AE8A8.htm#REFRN10335
then you’ll see that threaded_execution=true does not allow “/ as sysdba” authentication. Hence, AU will fail.
We will most likely include a preupgrade check to detect such situations and avoid to fail as we will recommend to disable it.
Cheers,
Mike
Thank you very much Mike
Hi Mike,
Take a look at this blog: https://dohdatabase.com/2020/09/16/how-to-gather-fixed-object-statistics-after-upgrade-with-autoupgrade/
I thing that can be a new feature to include on autoupgrade tool: Create a Schedule to Gather Fixed Stats after a few days from the upgrade, with default 7 days after, but can be customized on config files.
We are discussing this internally, José – Daniel and I work in the same team 🙂
Cheers,
Mike
How to manage parallel threads using autoupgrade.jar
Please search for “autoupgrade parallel” on the blog 🙂
It would bring you to:
https://mikedietrichde.com/2019/11/28/autoupgrade-tips-running-two-or-more-session-in-parallel/
Cheers,
Mike
Hi Mike,
Thanks for that tool, it’s really brilliant and makes our life easier. I just wanted to let you know that I successfully upgraded 11.2.0.3 RAC database ( 11.2.0.3.15 (20760997) to 18c using autoupgrade.
Thanks a lot, Adam!
And glad to hear that.
Cheers,
Mike
Hi Mike,
Thanks for the autoupgrade tool.
I am using latest autoupgrade.jar version (oct 2020).
I was able to upgrade the database from 11.2.0.4 to 19.4.0 succesfully. However, in the postfixups, I ran into issue. After timezone was upgraded successfully, logs showed that Oracle database could not be opened due to “REASON: ORA-12578: TNS:wallet open failed”.
In our environment, we have sqlnet.ora (customized one) placed in common location. On source DBHOME, We are just pointing sqlnet.ora to the common location from 11.2.0.4 ORACLE_HOME using symbolic link.
sqlnet.ora file contains wallet entries for ZDLRA backup only. Database is neither configured for TDE wallet nor using it.
So as per autoupgrade log, noticed that autoupgrade did copy and merged the sqlnet.ora (among other network related config files) contents and messed up the order of wallet_location entries causing these errors.
1)
can you please advise, Is there any option in autoupgrade config I can use to skip copying network related files from OLD_DBHME to 19.4 DBHOME to avoid this issue ?
2) Another question, if we upgrade multiple databases from 11.2.0.4 DBHOME to 19.4,, Is autoupgrade going to copy $ORACLE_HOME/network/admin files every time ?
If it does, I am afraid it could affect other already migrated database to 19.4 ORACLE_HOME from sqlnet.ora perspective.
Thanks,
Paul
Hi Paul,
we are aware of this issue, a fix should be in place soon.
In your case, using “upg1.timezone_upg=no” and a manual execution of utltz_upg_check.sql and utltz_upg_apply.sql will do the job.
I need to check what happens with the network files.
Are you sure you will use 19.4.0? This is a patch bundle from July 2019 – 15 months old.
Cheers,
Mike
Hi Mike,
Thanks for the newest Autoupgrade version 21.1.1.
Just wanted to quick follow-up to my earlier posted question. Basically, I want to skip copying $ORACLE_HOME/network/admin files from OLD HOME to new HOME.
Does the new Autoupgrade version have any fix related to this .?
we have sqlnet.ora stored in common location and we are just pointing link from Database Home to common location.
“With prior version, as per autoupgrade log, noticed that autoupgrade did copy and merged the sqlnet.ora (among other network related config files) contents and messed up the order of wallet_location entries causing these errors.”
1)
can you please advise, if new version has option to skip copying network related files from OLD_DBHME to 19.4 DBHOME to avoid this issue ?
Hi Paul,
I know that we handle some things differently – but please test it by yourself.
Cheers,
Mike
Hi Mike,
Thanks for the quick reply. Appreciated. I will use the workaround “upg1.timezone_upg=no” and a manual execution of utltz_upg_check.sql and utltz_upg_apply.sql to avoid this issue.
Good point. We are considering to use Oracle 19.8 for 19c upgrade.
Thanks,
Paul
Hi Mike,
Could You please clarify below poin
1.For exacc rac database tool is disabling service which we don’t need as we are going to upgarde few pdb only.
2.Even though i am using copy option from source pdb(12c) to target (19.8) source pdb is getting dropped .We don’t want as in case of upgrade fail we will just close and start the source pdb.
upg.pdbs=PDBUPG # Comma delimited list of pdb names that will be upgraded and moved to the target CDB
upg.target_pdb_name.PDBUPG=PDBUPG # Optional. Name of the PDB to be created on the target CDB
upg.target_pdb_copy_option.PDBUPG=file_name_convert=(‘+DATAC3′,’+DATAC3’) # Optional. file_name_convert option used when creating the PDB on the target CDB
Hi Vinod,
can you please open an SR and upload the entire log tree:
java -jar autoupgrade.jar -config yourconfig.cfg -zip
and pass on the SR number.
Thanks,
Mike
Hi Mike,
We have automated the Oracle 19c upgrade process for our Linux and Windows standard and enterprise edition product using Autoupgrade utility. So far we did upgrade for at least 40+ customers’ UAT and production environment and we see no issues and upgrade process is very smooth including conversion of non-CDB to CDB. However we observe one issue last week where utility failed after 93%. Upon initial investigation we observed that if existing and new Oracle Home are symbolic links the it fails. So just wanted to check with you if my observation is correct. If yes, what is the solution? If no, how I should debug the issue so that it can be fixed?
Thank you and appreciate your effort on such a cool tool.
Regards,
Mangesh
Hi Mangesh,
thanks for your feedback – and happy to read about your success!
I think symbolic links are always an issue. And with 19c, symbolic links are generally not allowed anymore, at least not within the database.
https://mikedietrichde.com/2019/07/15/behavior-change-in-oracle-18c-19c-no-symbolic-links-for-data-pump-directories/
I can’t tell you if this is related – but it may be worth a test if the underscore I reference in this blog post set in your spfile before the upgrade will allow you to go forward.
But this may be not helpful at all.
Cheers,
Mike
Hi Mike,
the upgrade is going well but the postfixups taking a long time (more then 3 hours) to do the following:
Additional information
———————————–
Details:
+——–+—————–+——-+
|DATABASE| FIXUP | STATUS|
+——–+—————–+——-+
| ssspat|AWR_DBIDS_PRESENT | STARTED|
+——–+—————–+——-+
is there to skip it?
thanks,
Jonathan
Hi Jonathan,
we are aware of this – and actually I’ve had several customers being hit by this.
Would you please mind to open an SR as this needs to be tracked by support.
What you can do as a workaround:
You could override the fixup – and then run it afterwards by yourself.
https://mikedietrichde.com/2020/07/22/how-to-skip-a-fixup-in-autoupgrade/
Cheers,
Mike
Hi,
First time I had this, I decided to fix the issue before I ran AutoUpgrade by removing all awr history from previous incarnations.
If this query gives a result:
select dbid
from wrm$_wr_control wwc
where wwc.dbid != (select dbid from v$database)
/
then run this:
exec dbms_swrf_internal.cleanup_database;
Run it sometime (days) before you do the AutoUpgrade as it might take a while depending on how much data there is.
But then AutoUpgrade will run without the issue.
Cheers,
Stuart
Thanks Stuart!
Cheers,
Mike
Hi
I have been using the autoupgrade tool to convert my database from 12c to 19c .
It is an excellent tool and saves me time.
unfortunately there is a bug with autoupgrade where it changes the description of listeners which i don’t want to upgrade , the reason for this is that we use a symlink which containes all the listeners on a server.
Oracle support say they they do not want to help and say we should not use symlinks. Obviously this is a bug and should be fixed. I can give you the SR details if required.
thanks
Ansar
Hi Ansar,
if you think this is a bug and you have an SR, then please ask the support engineer to file a bug.
This would be the best, most efficient and correct way.
Thanks,
Mike
Hi Ansar,
I’m using “global.manage_network_files=SKIP” in my autoupgrade config file. In this case, my autoupgrade works as expected without any merges from Listener.ora and Tnsnames.ora files.
best regards
Stefan
Thanks Stefan – and I have to confess that I wasn’t aware of this parameter.
Thanks for sharing!!
Cheers,
Mike
Thanks from me also Stefan. Fixes an issue I have with every upgrade where the static connections all get their home updated – even the ones that were not upgraded.
Cheers,
Stuart
Hi Stuart,
yes, this is the case where it was made for actually.
Cheers,
Mike
Hey Mike!
RE: the latest January 2021 CPUs for 19c on Windows servers…
I just bashed my head against the wall for a few hours when I started receiving “ORA-12638: Credential retrieval failed” when selecting data through a db_link to a database on a remote server that was working fine before the patching.
Turns out with CPUJAN2021, Oracle changed the default for NO_NTLM from FALSE to TRUE when SQLNET.AUTHENTICATION_SERVICES=NTS in the sqlnet.ora on Windows servers!
Here’s the MOS DocID:
Windows: While Connect to Database Getting ORA-12638 After Applying Jan 2021 WINDBBP 19.10.0.0.210119 On Windows 6432 Bit Database Client (Doc ID 2757734.1)
You might want to let your readers know about this! It’s not published anywhere yet outside of MOS, and it’s NOT in the Critical Issues doc for 19c for this CPUJAN2021 either!
Oracle Database 19c Release Update & Release Update Revision January 2021 Critical Issues (Doc ID 2725758.1)
Thanks!
Thanks again, Ernst – and as you saw already, it lead to a blog post!
Thanks so much!!!
Mike
Hi Mike,
We have been using AutoUpgrade tool for some time and its an wonderful tool for upgrading to newer version of databases. Can we use this tool to upgrade Databases that are associated with Oracle E-Business Suite Application like R12.1.3 or R12.2 instances.
Thanks for your feedback – and unfortunately the EBS team has settled on DBUA and a ton of self-written scripts to make the plugin possible.
See here: https://mikedietrichde.com/2020/06/30/collection-of-ebs-upgrade-information-for-oracle-database-19c/
Cheers,
Mike
Can we use this tool to upgrade a Database that’s used for Ebusiness Suite?
Hi,
see my other reply please.
Cheers,
Mike
HI Mike
I installed EPPM for a customer with an 18.4 XE database. Now they’ve have found some money for a licence and wish to upgrade to SE2. Will Autoupgrade work from 18.4XE to 19c?
I read that to upgrade from XE the database needs to initially be upgraded to the same version database on EE or SE2. But this wasn’t using AutoUpgrade so thought i would ask !
Thanks
Hi Darren,
as far as I’m aware, XE is not upgrade-supported. This means, you would end up with a ton of options which don’t exist in SE2.
So please use expdp/impdp.
Cheers,
Mike
Hi Mike
I’ve been told to manually copy the schemas following:
“How to Move a P6 Enterprise Oracle Database to a New Oracle Database Server (Doc ID 2431072.1)”
I’m interested to know what the difference is between this manual process and migrate.bat which is an automated process in EPPM source files. Any ideas?
Regards
Hi Darren,
the note you refer to is a schema export/import guideline. But usually, you don’t want to go the expdp/impdp route when your database is bigger. The note is fine for either smaller databases or isolated and not too big schemas. But as soon as it gets in the hundreds of GB and larger, you drive much faster with xTTS (cross platform transportable tablespaces) if you migrate from AIX to Linux. If you stay on AIX, then use AutoUpgrade.
Cheers,
Mike
Hi Mike
Maybe it has been already answer but I didn’t find the information .
This is his about the following :
“Local parameters take precedence over any global parameters set in the AutoUpgrade configuration file”
For example does it mean that I cannot have a global “add_during_upgrade_pfile” (containing parameters I want to add systematically to all DB) and a local “add_during_upgrade_pfile” (containing specific parameters to one db) that is merged with global one ?
If not that mean that I have to do the merge manually ?
Thanks for your attention
Francois
Francois,
it means that for instance, you set _cursor_obsolete_threshold=1024 in the global parameter file, and then you set _cursor_obsolete_threshold=1000 in the local parameter file. The value of the local would override the global value for this particular database only.
Cheers,
Mike
Hello Mike,
Where do we find the explanation for UPG errors. I am seeing UPG-1303 errors but not able to find any information. Is there anything like oerr UPG or something like that? Thanks
Hi Sunil,
fair point and good question:
https://mikedietrichde.com/2021/04/12/where-are-autoupgrade-error-codes-documented/
Cheers,
Mike
Mike,
For some reasons I am getting errors on error_code check:
$ORACLE_HOME/jdk/bin/java -jar autoupgrade.jar -error_code
usage: java -jar autoupgrade.jar [-version | -help | -create_sample_file ] [-settings ] [-config ] [-config_values
] [-clear_recovery_data ] [-mode ] [-console | -noconsole] [-restore_on_fail] [-debug] [-zip] [-sid ] [-d
]
-version Displays the current build of the jar
-help Displays the available options
-create_sample_file Creates a sample configuration file that be used as reference
config – Creates a sample configuration file
settings – Creates a sample internal configuration file to allow a deeper low level configuration
-settings Path to config file to overwrite internal settings
-config User config file with the databases to upgrade
-config_values User input comma-separated list where * separates database entries
“source_home=value,target_home=value,sid=value,*,source_home=value…”
-clear_recovery_data Remove the recovery checkpoint to start fresh the next time the tool is launched
-mode Mode on which the AutoUpgrade will start and behave
analyze – Executes the checks and generates a report of the database status
deploy – Performs the upgrade of the databases from start to end
fixups – Executes the checks and pre-upgrade fixups but do not start the upgrade
upgrade – Performs the database upgrade and post-upgrade actions. The database must already be up and
running with the target home
-console Start the AutoUpgrade with the console enabled (default)
-noconsole Start the AutoUpgrade with the console disabled
-restore_on_fail If present, when a job fails, the database is restored automatically. Errors in PDBs are not considered
fatal, only errors in CDB$ROOT or Non-CDBs
-debug Debug messages enabled
-zip Zips up log files required for filing an AutoUpgrade service request
-sid Zip file will contain all the SIDs that are given in a comma separated list
-d Path to save AutoUpgrade zip file
The config option with sample parameter creates a sample database configuration file with default values.
Or you can use it with a custom database configuration file with an execution mode (deploy, analyze, fixups or upgrade).
The settings parameter lets you use a file with Autoupgrade internal settings, can be default for base settings or you can specify a file path for custom
settings.
Unrecognized option: [-error_code]
$ORACLE_HOME/jdk/bin/java -jar autoupgrade.jar -version
build.hash e84c9c2
build.version 19.10.0
build.date 2020/10/23 10:36:46
build.max_target_version 19
build.supported_target_versions 12.2,18,19
build.type production
Not sure if I am missing something here
Thanks Sunil – I added the missing piece, the version of AU you need to have.
Sorry for the inconvenience.
Cheers,
Mike
Unrecognized option: [-error_code]
Looks like it works with Latest version
build.hash 59fbf3e
build.version 21.1.2
build.date 2021/02/24 17:11:08
build.max_target_version 21
build.supported_target_versions 12.2,18,19,21
build.type production
Hi Mike,
I tried the AutoUpgrade kit with analyze option and got this error. Can you tell what be wrong here?
2021-04-13 21:25:38.540 INFO
build.hash:59fbf3e
build.version:21.1.2
build.date:2021/02/24 17:11:08
build.max_target_version:21
build.supported_target_versions:12.2,18,19,21
build.type:production
build.label:HEAD
2021-04-13 21:25:41.351 ERROR Database initilization of [RVCD] failed due to [java.sql.SQLException]
2021-04-13 21:25:41.351 ERROR Errors executing [with segments as (select /*+ MATERIALIZE */ tablespace_name, inuse from( select ds.tablespace_name, row_number() over (partition by tablespace_name order by 1) rn, round(nvl(sum(ds.bytes) over (partition by ds.tablespace_name),0)/1024,2) as inuse from sys.dba_segments ds) where rn=1),tmp_ts_query as ( select null group_name, tablespace_name, bytes, maxbytes from sys.dba_temp_files where tablespace_name = ‘TEMP’ ),ts_qresult as ( SELECT /*+ MATERIALIZE */ nvl(dtf.group_name, dt.tablespace_name) as name, dt.contents as contents, decode(dt.contents,’TEMPORARY’,1,0) as temporary, decode(dt.extent_management,’LOCAL’,1,0) as localmanaged, nvl(ds.inuse,0) as inuse, nvl(round(sum(case dt.contents when ‘TEMPORARY’ then dtf.bytes else ddf.bytes end)/1024, 2),0) as alloc, nvl(round(sum(case dt.contents when ‘TEMPORARY’ then decode(dtf.maxbytes, 0, 0, dtf.maxbytes-dtf.bytes) else decode(ddf.maxbytes, 0, 0, ddf.maxbytes-ddf.bytes) end)/1024,2),0) as auto FROM sys.dba_tablespaces dt left join segments ds on (dt.tablespace_name = ds.tablespace_name) left join sys.dba_data_files ddf on (ddf.tablespace_name = dt.tablespace_name) left join tmp_ts_query dtf on (dtf.tablespace_name = dt.tablespace_name) WHERE (dt.tablespace_name in (‘UNDOTBS1’, ‘SYSTEM’,’SYSAUX’)) or (dt.tablespace_name in ( SELECT distinct T.tablespace_name FROM sys.dba_queues Q, sys.dba_tables T WHERE Q.queue_table=T.table_name AND Q.owner = T.owner)) or dtf.tablespace_name is not null group by nvl(dtf.group_name, dt.tablespace_name), dt.contents, dt.extent_management, ds.inuse )select tq.name || ‘#’ || tq.contents || ‘#’ || tq.temporary || ‘#’ || tq.localmanaged || ‘#’ || tq.inuse || ‘#’ || tq.alloc || ‘#’ || tq.auto || ‘#’ || (tq.alloc+tq.auto) avail from ts_qresult tq order by tq.name;
with segments as (select /*+ MATERIALIZE */ tablespace_name, inuse from( select ds.tablespace_name, row_number() over (partition by tablespace_name order by 1) rn, round(nvl(sum(ds.bytes) over (partition by ds.tablespace_name),0)/1024,2) as inuse from sys.dba_segments ds) where rn=1),tmp_ts_query as ( select null group_name, tablespace_name, bytes, maxbytes from sys.dba_temp_files where tablespace_name = ‘TEMP’ ),ts_qresult as ( SELECT /*+ MATERIALIZE */ nvl(dtf.group_name, dt.tablespace_name) as name, dt.contents as contents, decode(dt.contents,’TEMPORARY’,1,0) as temporary, decode(dt.extent_management,’LOCAL’,1,0) as localmanaged, nvl(ds.inuse,0) as inuse, nvl(round(sum(case dt.contents when ‘TEMPORARY’ then dtf.bytes else ddf.bytes end)/1024, 2),0) as alloc, nvl(round(sum(case dt.contents when ‘TEMPORARY’ then decode(dtf.maxbytes, 0, 0, dtf.maxbytes-dtf.bytes) else decode(ddf.maxbytes, 0, 0, ddf.maxbytes-ddf.bytes) end)/1024,2),0) as auto FROM sys.dba_tablespaces dt left join segments ds on (dt.tablespace_name = ds.tablespace_name) left join sys.dba_data_files ddf on (ddf.tablespace_name = dt.tablespace_name) left join tmp_ts_query dtf on (dtf.tablespace_name = dt.tablespace_name) WHERE (dt.tablespace_name in (‘UNDOTBS1’, ‘SYSTEM’,’SYSAUX’)) or (dt.tablespace_name in ( SELECT distinct T.tablespace_name FROM sys.dba_queues Q, sys.dba_tables T WHERE Q.queue_table=T.table_name AND Q.owner = T.owner)) or dtf.tablespace_name is not null group by nvl(dtf.group_name, dt.tablespace_name), dt.contents, dt.extent_management, ds.inuse )select tq.name || ‘#’ || tq.contents || ‘#’ || tq.temporary || ‘#’ || tq.localmanaged || ‘#’ || tq.inuse || ‘#’ || tq.alloc || ‘#’ || tq.auto || ‘#’ || (tq.alloc+tq.auto) avail from ts_qresult tq order by tq.name
ERROR at line 1:
ORA-04045: errors during recompilation/revalidation of SYS.DBMS_SPACE_ADMIN
ORA-16000: database or pluggable database open for read-only access
] [PDB$SEED]
2021-04-13 21:25:41.353 ERROR Database checks failed
2021-04-13 21:25:41.354 ERROR Dispatcher failed: AutoUpgException [UPG-1319]
2021-04-13 21:25:41.356 INFO Starting error management routine
2021-04-13 21:25:41.357 INFO Ended error management routine
2021-04-13 21:25:41.358 ERROR Error running dispatcher for job 103
Cause: Loading the current state of the database failed
2021-04-13 21:25:41.358 ERROR Dispatcher failed:
Error: UPG-1319
Thank you,
Hoa
Hi,
did you run it on a standby?
Or is this coming from a regular PDB$SEED??
It is a quite strange and unexpected error which I haven’t seen at other customers with the similar setup.
Can you please open an SR and upload all your logs:
java -jar autoupgrade.jar -config yourconfigfile -zip
Please add alert.log and “opatch lsiventory”
Cheers,
Mike
Hi Mike,
Before running autoupgrade, I remove all hidden parameters and restart the database.
One of these hidden parameters that seems to be on most of the databases is _use_single_log_writer=’TRUE’.
Now, I am seeing it was a recommendation for 12.1.0.2 on AIX.
Is this fixed on 19c? or should I put it back again after the upgrade?
Regards,
Stuart
Hi Stuart,
this was a clear 12.1.0.2 recommendation so regarding the issues, yes, they are fixed as far as I’m aware. And lately I haven’t seen such issues with the logwriter anymore. The other question is if you benefit from multiple logwriter slaves. But you certainly can live without this underscore for sure in 19c.
Cheers,
Mike
Excellent. Top Man (Y)
Hi Mike, can you please confirm that I can use autoupgrade tool on EXADATA databases ?
I think it isn’t plateforme dependant bu I want to be sure before satrtung my upgrade project of many databases on X7 EXADATA
Hi Ahmed,
yes, you can – and the MOS note will be updated finally very soon (final review phase this week).
Cheers,
Mike
after successfull PREFIXUPS, the AutoUpgrade restarted the instance and hitting this error
Thu May 06 23:53:03 2021
Instance shutdown complete
Thu May 06 23:53:11 2021
ERROR: Unable to get logical block size for spfile ‘+DATA/XXXX/spfilexxxxxx.ora’.
Adjusting the default value of parameter parallel_max_servers
from 320 to 226 due to the value of parameter processes (300)
Starting ORACLE instance (normal) (OS id: 25362596)
Thu May 06 23:53:11 2021
CLI notifier numLatches:13 maxDescs:519
Thu May 06 23:53:11 2021
**********************************************************************
after that most of the errors, related unable to start ASM instance, due to which unable to locate DATA and RECO disk..
Please help, this is Oracle RESTART (has) environment.urgent
upg> lsj
+—-+——-+———+———+——+————–+——–+——–+
|Job#|DB_NAME| STAGE|OPERATION|STATUS| START_TIME| UPDATED| MESSAGE|
+—-+——-+———+———+——+————–+——–+——–+
| 102| XXXXXX|DBUPGRADE| STOPPED| ERROR|21/05/06 23:16|23:54:42|UPG-1401|
+—-+——-+———+———+——+————–+——–+——–+
Total jobs 1
Please open an SR and upload the entire log directory gathered with
java -jar autoupgrade.jar -config yourconfig.cfg -zip
Add the alert.log including the alert.log from ASM, and ideally also an “opatch lsinventory”.
Log the SR with “Sev.1” but not 24×7 unless you’d like to travel the world for the next day.
AutoUpgrade does not touch the ASM instance as it is taken care of by the Grid Infrastructure installation.
Make also sure you’ve used the newest version of AutoUpgrade.
Cheers,
Mike
Hi Mike, , We are getting this error at analyze Phase, UPG-1316, om the userlogs “oracle.upgrade.commons.dbinspector.checks.disk_space_for_recovery_area.checkCode(disk_space_for_recovery_area.java:122)”, this database have been upgrating from 10G to 11g recently and we add db_recovery_file_dest_size and db_recovery_file_dest parametres. whiout thos parameters the alanaze phase ends but when we add tos parametres the analuze phase fails. thanks
Hi Hernando,
luckily this has been solved very quickly by Oracle Support. For those who are interested: An environment issue lead to the wrong “df” – which caused AutoUpgrade to fail on the check for available free space for archives and redo.
Cheers,
Mike
Hi Mike,
Shall AutoUpgrade Utility be used to upgrade from 19.7 SE2 to 10.10 EE (SE2->EE)..
could you also please share the link for manual steps..
thanks
K
No, as this is a patch, and not an upgrade.
In this case, the steps are very simple.
Stop the database in the old home.
Start it up in the new home.
Involve “datapatch -verbose”.
And of course, take a backup before all this.
Cheers,
Mike
Hi, I plan to upgrade a RAC from 12.1.0.2 to 19.3.0.0. I do not see where they indicate that autoupgrade.jar is compatible with 12.1.0.2, the notes I have read say from 12.2 onwards.
Is there a parameter in the autoupgrade.jar where it indicates that it is RAC?
Thanks
Did you see the sentence below the yellow box in the AutoUpgrade download note?
There is says:
For supported source Oracle Databases releases to be upgraded to above target releases, refer to Database Server Upgrade/Downgrade Compatibility Matrix (Doc ID 551141.1)
When you click on the note and select “19c” as your target, you will find:
11.2.0.4
12.1.0.2
12.2.0.1
18c
as upgrade-supported releases when you plan to upgrade to 19c.
Cheers,
Mike
Hi Mike,
As per Doc 2485457.1, auto upgrade is supported on DR environments for auto upgrade utility release 21c.
Is it possible to use 21c release while upgrade to 19c database or this only used with 21c RDBMS?
Thanks
Hi Mohamed,
yes, please do so – the 21c version just indicates that it will work for upgrades to 21c, too.
But when you call “java -jar autoupgrade.jar -version” it will tell you that you can easily upgrade to 19c as well.
Cheers,
Mike
i ran into below issue in my first upgrade using autoupgrade Autoupgrade “postfixups” Failed with Error -UPG-1604 SR opened SR 3-26823431891 and applied latest opatch and datapatch -verbose fixed the issue, second time same issue again happened with upg-1604, we tried the same latest opatch and databatch -verbose but this time resulting in another error – working via SR 3-26845468421
upg> lsj
+—-+——–+———-+———+——+————–+——–+——–+
|Job#| DB_NAME| STAGE|OPERATION|STATUS| START_TIME| UPDATED| MESSAGE|
+—-+——–+———-+———+——+————–+——–+——–+
| 101|CDBU01P1|POSTCHECKS| STOPPED| ERROR|21/08/27 11:30|22:22:33|UPG-1319|
+—-+——–+———-+———+——+————–+——–+——–+
Hi Raman,
sorry for the delay. I haven’t had checked the SR. But has the issue been solved by support?
If not, please give me a shout via the blog.
Cheers,
Mike
OK, Mike, you got me hooked into your blog :-). I just wish I have the hardware to run the lab, kinda visualizing/imagining stuff at the moment, wonder if Oracle can provide ‘free’ lab for users to test similar like liveSQL 🙂
Anyway, finally needing to test/use/run AutoUpgrade more than a year after I attended one of your demo sessions. Any chance the next version can include a config parameter for a pre- and post- script that the user may want to be executed in between run of AutoUpgrade. Yeah, I know, the alternative is we just script it ourselves to have those script as well as the AutoUpgrade script into one AutoUpgrade run script of our own 🙂 Or is such config available already? I don’t see any on the doc.
Thanks again for your blog. Will have to download your lab one of these days
P.S.
– BTW, Oracle Documentation link to Oracle 8.1.7 gives 404 🙂 Yeah, there are still Oracle9 DBs around 🙂
Hi Edwin,
we have the lab available – use the GREEN BUTTON option here:
https://apexapps.oracle.com/pls/apex/f?p=133:100:2240355569452::::SEARCH:hitchhiker
I will blog about it once it satifies our needs.
Cheers,
Mike
Hi Mike
I wanted to upgrade newly installed 19c to 21c with autoupgrade tool (latest july 2021 version).
While using deploy mode , I got following error.
Error: UPG-1401
Opening Database cdb1 in upgrade mode failed
Cause: Opening database for upgrade in the target home failed.
There is nothing in the logs.
Hi Zulal,
please in such cases, open an SR and upload all AU logs (java -jar autoupgrade.jar -config yourconfig.cfg -zip) plus alert plus opatch lsinventory to the SR.
I was away for a while, and you shouldn’t wait for this to be taken care on.
Cheers,
Mike
Please ignore my previous feedback. After reading further on the other section of your blog, looks like there is a before=pre and after=post script option already.
Any tips on how to get Oracle Support to really reply with something useful in regards to when your AutoUpgrade take 5+ hours to run, surely they can give just contact you, isn’t it? 🙂
See here for a guideline on how to escalate SRs when you are not satisfied:
https://blogs.oracle.com/support/request-support-management-attention-sr-attention
Cheers,
Mike
Hello, I’m running a “CREATE PLUGGABLE DATABASE” from a database link, from 12.2 to 19.14.
How do I run autoupgrade in this case. I tried, and it attempted to upgrade the whole database, not just the PDB. I’m running a catctl.pl instead which works fine, but wondered if there was an autoupgrade method.
I had “upg1.pdbs=” in the autoupgrade config file.
Thanks
Hi Ben,
got you – AutoUpgrade expects a different setup in this case – upgraded non-CDB. This setup plugs in a non-CDB and needs to do a conversion. The logic in 19c works differently. And honestly, I’m not a fan of this technique since I saw too many problems with it.
Thanks,
Mike
Hi Mike ,
Question:
Does the autoupgrade tool work in ExaCC as well?
Thanks
Hi Gerson,
technically it does work. But the dbaascli has been packed for old DBUA.
We are releasing a solution soon for those you want to use AutoUpgrade on ExaCS and ExaCC. Stay tuned please.
Cheers,
Mike
Hi Mike, got 2 questions on autoupgrade:
what stats need run prior either the day before or days before. Is it all 3 of these:
DBMS_STATS.GATHER_DICTIONARY_STATS;
DBMS_STATS.GATHER_SCHEMA_STATS(‘SYS’);
DBMS_STATS.GATHER_SCHEMA_STATS(‘SYSTEM’);
Other quick question, is there a way to get comments in the spfile if we use global.add_after_upgrade_pfile=/home/oracle/global_add_after.ora I have in the global_add_after.ora
_cursor_obsolete_threshold=1024#02-02-2022 Dennis Holder: MOS 2431353.1, evaluate after upgrade
when I look at the spfile output it doesn’t take my comments
*._cursor_obsolete_threshold=1024# added as per user request
Hi Dennis,
please see:
https://mikedietrichde.com/videos/
Episode: 9 from slide 58 onward.
Watch the video as well in addition and ignore the fact that it is saying “cloud” – the recommendations applied to all setups.
I think the comments option is possible with the newest option in Autoupgrade. It ignored it before, but it should be possible now.
Cheers,
Mike
Hello Mike
i have an upgrade project, where i need to upgrade 12.1.0.1 to 19c. Did an active duplicate to other server to test the process. Decided to first upgrade to 12.2 (ORACLE_HOME was patched to 12.2.0.1.220118 to use the autoupgrade tool) and then to 19c (patched to 19.15.0.0.220419 + some one-offs). Upgrade to 12.2 was fine, without issues but I did run into a problem, when upgradeing to 19c (-analyze ran fine), where the autoupgrade tool couldn’t finish the precheck [checkname] UNIAUD_RECORDS_IN_FILE but i got over it by deleting the *.aud files in adump folder + truncating aud$ table and also ran DBMS_AUDIT_MGMT.CLEAN_AUDIT_TRAIL- the last one took some time, but finished. Ran the autoupgrade again and no problems.
So started the same thing with the production database. Again, no problems with upgrading to 12.2 but same problem with upgrading to 19c. Only this time, even the -ANALYZE didn’t finish, i waited for 20+minutes for it to finish the same precheck [checkname] UNIAUD_RECORDS_IN_FILE and then just aborted the autoupgrade job. From the logs:
WARNING [3F35EE] This thread was interrupted while waiting for the following query to finish [/ 3F35EE /DECLARE
aud_recs NUMBER := 0;
db_version VARCHAR2(5);
cur_con_id NUMBER := 0;
BEGIN
— Get the current db version
SELECT SUBSTR(version,1,4) INTO db_version FROM sys.registry$
WHERE cid = ‘CATPROC’;
— lrg 22212908: Get current container ID
select sys_context(‘userenv’, ‘con_id’) into cur_con_id from dual;
— GV$UNIFIED_AUDIT_TRAIL gets us the number of unified audit records
— present in the OS spillover audit files starting RDBMS 12.2 release.
— Thus, this check should be restricted to db version >= 12.2.
— For other db versions = (‘12.2’) AND (cur_con_id != 1) AND (cur_con_id != 2) THEN
— Get the number of unified audit records present in OS Spillover
— audit files (.bin format)
SELECT count(*)
into aud_recs from gv$unified_audit_trail;
— Requiring at least 2 records is expected
IF (aud_recs > 1) THEN — records not yet loaded to DB audit table
DBMS_OUTPUT.PUT_LINE(‘FAILURE’);
return;
END IF;
END IF;
DBMS_OUTPUT.PUT_LINE(‘SUCCESS’);
EXCEPTION
WHEN NO_DATA_FOUND THEN DBMS_OUTPUT.PUT_LINE(‘SUCCESS’);
END;
/
] [asd] – ExecuteSql.quickSQL
2022-05-20 00:23:54.192 WARNING Error executing check UNIAUD_RECORDS_IN_FILE[asd] – Check.presentInDb
java.sql.SQLException: [3F35EE] Process abort was requested for query
After i aborted, tried to clean unified audit trail manually with the DBMS_AUDIT_MGMT.CLEAN_AUDIT_TRAIL but that query hangs as well (i only had limited time for downtime). After about an hour of no luck getting any further with this, i left the database to 12.2.0.1.220118. Next day started with troubleshooting and the outcome is that any procedure from the DBMS_AUDIT_MGMT package ahngs forever. Any query agaist unified_audit_trail table just hangs forever (do not finish in hours). exec DBMS_STATS.GATHER_FIXED_OBJECTS_STATS; hangs at SYS.X$UNIFIED_AUDIT_TRAIL (didn’t finish over the weekend, started friday, i canceled it on monday). I did create an SR for this but so far it seems that support does not understand what i’m saying.
At this point i’m considering planning a downtime with the client and just skipping those checks and everything else that might touch unified_audit_trail and deal with this under the 19c as 12.2 is out of support anyways. Could you give me recommendation what to do with this? Would it be a good decision to skip checks when there’s a problem in database?
Many thanks in advance
You please need to collect the logs with
java -jar autoupgrade.jar -config myconfig.cfg -zip
and upload it to an SR with Oracle Support. Even though I see what your issue is, please understand that I can’t debug this on the blog.
And I haven’t see it yet.
Cheers
Mike
Thanks for the reply, Mike. I actually did upload collected logs to SR, right away 😅 I hope the support will eventually come up with the solution.
Thanks Evelin – let me know if it doesn’t get progressed (and do not hesitate to raise the severity to Sev.1 (but NOT 24×7) if you think the SR got stuck).
Cheers
MIke
Hi Mike,
i am trying to apply an RU patch from 19.8 to 19.15 with the combo patch (33806152). it was going fine until it failed with the make example:
Make failed to invoke “/usr/ccs/bin/make -f ins_net_client.mk client_sharedlib ORACLE_HOME=/oracle/database/oracle/product/19.8.0.0 OPATCH_SESSION=apply”….’genclntsh: Failed to link libclntsh.so.19.1 make: Fatal error: Command failed for target `client_sharedlib’
The following make actions have failed :
Re-link fails on target “client_sharedlib”.
Re-link fails on target “ikgmgr”.
Re-link fails on target “igenezi”.
Re-link fails on target “iorion”.
Re-link fails on target “iexpdp”.
Re-link fails on target “itkprof”.
Re-link fails on target “isqlldr”.
Re-link fails on target “idg4pwd”.
Re-link fails on target “ioratop”.
Re-link fails on target “iimpdp”.
Re-link fails on target “iplshprof”.
Re-link fails on target “client_sharedlib”.
Re-link fails on target “iamdu”.
Re-link fails on target “ikfod”.
Re-link fails on target “ikfed”.
Re-link fails on target “idbv”.
Re-link fails on target “inid”.
Re-link fails on target “isbttest”.
Re-link fails on target “iorapwd”.
Re-link fails on target “irenamedg”.
Re-link fails on target “itrcldr”.
Re-link fails on target “itstshm”.
Re-link fails on target “idgmgrl”.
Re-link fails on target “iwrap”.
Re-link fails on target “ictxlc”.
Re-link fails on target “ictxkbtc”.
Re-link fails on target “ictxload”.
Re-link fails on target “itnsping”.
Re-link fails on target “genplusso”.
Re-link fails on target “gen_plusso”.
Re-link fails on target “irman”.
Re-link fails on target “ihsodbc”.
Re-link fails on target “itnslsnr”.
Re-link fails on target “ilsnrctl”.
Re-link fails on target “ldapsearch”.
Re-link fails on target “iwrc”.
Re-link fails on target “proc”.
Re-link fails on target “procob”.
Do you want to proceed? [y|n]
Y
trying to relink all manually but it did not work. this is RU patch of 19.15
Hi Jonathan,
please, this needs to be handled in an SR.
Thanks,
Mike
Hi Mike, Can you please assist and review SR 3-30892165271, AUT got to 97% handled a few fallouts and DB opened, yet recommendation is to revert to 11g GRP…. ?
Hi Demetrius,
sorry for my late reply but it looks as if the SR has been solved by you together with Oracle Support.
Cheers,
Mike
Hallo Mike,
Can we use the Auto Upgrade Tool also for Upgrade in an Majore Release like Oracle 19.3 to Oracle 19.15?
Regards
Lukas
Hi Lukas,
yes, the newer versions of AU support the usage for patching as well.
Cheers
Mike