What if … something goes wrong during your database upgrade with AutoUpgrade? Of course, this never happens. But just in case, no worries – one of the really big advantages of AutoUpgrade (and the command line upgrade with “dbupgrade” as well) is its resumability and restartablity. But in this blog post I would like to cover a bit more. I will show you more details on Troubleshooting, Restoring and Restarting AutoUpgrade.

Photo by Mark Boss on Unsplash
And as often, the cause for this blog post is an email from Hervé Schweitzer, the CTO of dbi-services. Hervé told me about a failed upgrade at a customer where they ran out of shared pool. He fixed the problem already, I’d say with a brutal slam with a hammer – but he asked my what would be the ideal and recommended way to solve it.
This gives me the opportunity to write about something I planned to write for a long time. And actually, the blog post became a bit longer than I initially planned for.
Please open an SR
No worries, this is my standard answer. We need the logs in order to be able to say anything useful. And my wonderful team mates built a very nice and useful mechanism into AutoUpgrade which collects the logs, and zips them together. This is very handy in case you want to open an SR.
AutoUpgrade has been built from second One in a way to make sure it logs everything. Having adequate and readable (more or less) log information allows not only our Support mates but also ourselves to figure out what has gone wrong (or right). If you don’t log, you’ll be lost.
Now, when you open an SR, please do the following:
- Double-check that you’ve use the newest version of AutoUpgrade:
$ java -jar $OH19/rdbms/admin/autoupgrade.jar -version build.hash 6010a62 build.version 19.8.1 build.date 2020/04/01 16:18:24 build.max_target_version 19 build.type production
If not, please download the newest version from MOS Note: 2485457.1 and repeat your test. At the moment, we release an updated and improved version every 4-6 weeks. Sometimes we deliver fix overnight if necessary. Hence, it’s really key that you check to have the newest version at first. At the end of MOS Note: 2485457.1 you will find a changelog which tells you exactly what we fixed in which version.
. - Then zip the logs together:
$ java -jar $OH19/rdbms/admin/autoupgrade.jar -config DB12.cfg -zip Path does not exist at '/home/oracle/scripts/alert_DB12.log' This path will not be zipped. Please review your configuration file. Processing: 344847 [bytes] in 20 Files | 0% Zipped successfully at /home/oracle/scripts/AUPG_200407_1334_642.zip
At the moment you will see a message regarding the alert.log not being included. We will fix this soon. But meanwhile, please upload at least 2 days of your alert.log to MOS as well in addition.
Or you do the following and change to the directory where the alert.log is located in:
$ cd $ORACLE_BASE/diag/rdbms/db12/DB12/trace [DB12] oracle@hol:/u01/app/oracle/diag/rdbms/db12/DB12/trace $ java -jar $OH19/rdbms/admin/autoupgrade.jar -config /home/oracle/scripts/DB12.cfg -zip Processing: 1224834 [bytes] in 21 Files | 0% Zipped successfully at /u01/app/oracle/diag/rdbms/db12/DB12/trace/AUPG_200407_1339_531.zip
And then it will be included, too.
. - Then open an SR:
Go to MyOracle Support and create an SR. Please don’t wonder about outdated choices – this is not in our control. And the correct routing will work anyways.
.
What if … you lost the job console?
This is a scenario but also a question I received quite often in the hands-on labs we’ve did at OOW and during workshop. So let me do a quick exercise. I will start the upgrade, and then terminate the job console.
$ java -jar $OH19/rdbms/admin/autoupgrade.jar -config DB12.cfg -mode deploy AutoUpgrade tool launched with default options Processing config file ... +--------------------------------+ | Starting AutoUpgrade execution | +--------------------------------+ 1 databases will be processed Type 'help' to list console commands upg>
The upgrade is now running. It starts with the prechecks and the fixups. I’m checking it a while later:
upg> lsj
+----+-------+---------+---------+-------+--------------+--------+--------+-----------+
|Job#|DB_NAME| STAGE|OPERATION| STATUS| START_TIME|END_TIME| UPDATED| MESSAGE|
+----+-------+---------+---------+-------+--------------+--------+--------+-----------+
| 101| DB12|DBUPGRADE|EXECUTING|RUNNING|20/04/07 14:34| N/A|14:41:57|5%Upgraded |
+----+-------+---------+---------+-------+--------------+--------+--------+-----------+
Total jobs 1
Killing the console
Now I “kill” the console – and I’m back at the command prompt.
How do you “kill” the console? The normal way would be to type the “exit” command and confirm with “y”. When you check the alert.log, you will recognize that the database gets shutdown at this point.
But I tried other ways, too, such as sending a “kill -9” to the process or trying CTRL+C and CTRL+V – and I received mixed results. In some cases the database was still up, in others it went down as well.
Upgrade stopped
In any case, my upgrade gets stopped the hard way. I’m checking the main worker’s log file in /home/oracle/logs/DB12/101/dbupgrade
:
**************** Catproc Tables **************** Parallel Phase #:9 [DB12] Files:67 Unexpected error encountered in catctlMain; Error Stack Below; exiting Died at /u01/app/oracle/product/19/rdbms/admin/catcon.pm line 18162. at /u01/app/oracle/product/19/rdbms/admin/catcon.pm line 18162. catcon::catcon_HandleSigINT("INT") called at /u01/app/oracle/product/19/rdbms/admin/catcon.pm line 9932 eval {...} called at /u01/app/oracle/product/19/rdbms/admin/catcon.pm line 9932 catcon::next_proc(4, 4, 3, undef, "/home/oracle/logs/DB12/101/dbupgrade/catupgrd20200407143441db12", ARRAY(0x2b983e8), 0, ARRAY(0x2b98298), ...) called at /u01/app/oracle/product/19/rdbms/admin/catcon.pm line 10146 catcon::pickNextProc(4, 4, 3, "/home/oracle/logs/DB12/101/dbupgrade/catupgrd20200407143441db12", ARRAY(0x2b983e8), 0, ARRAY(0x2b98298), "/home/oracle/logs/DB12/101/dbupgrade/catupgrd20200407143441db"..., ...) called at /u01/app/oracle/product/19/rdbms/admin/catcon.pm line 15287
Uh … this looks pretty ugly. My upgrade died.
But wasn’t command line upgrade always resumable? Of course it is. As is AutoUpgrade.
Let me call the command again:
$ java -jar $OH19/rdbms/admin/autoupgrade.jar -config DB12.cfg -mode deploy Previous execution found loading latest data Total jobs recovered: 1 +--------------------------------+ | Starting AutoUpgrade execution | +--------------------------------+ Type 'help' to list console commands
You may recognize the lines I marked in red. AutoUpgrade knows about the previous executions due to the log information kept under job number 101 in your log tree.
And here we go – AutoUpgrade is seamless resuming my upgrade.
upg> lsj
+----+-------+---------+---------+-------+--------------+--------+--------+-----------+
|Job#|DB_NAME| STAGE|OPERATION| STATUS| START_TIME|END_TIME| UPDATED| MESSAGE|
+----+-------+---------+---------+-------+--------------+--------+--------+-----------+
| 101| DB12|DBUPGRADE|EXECUTING|RUNNING|20/04/07 14:34| N/A|14:49:18|8%Upgraded |
+----+-------+---------+---------+-------+--------------+--------+--------+-----------+
Total jobs 1
This tool isn’t clearly dumb.
Now let me show you another scenario.
What if … your upgrade failed?
There may be various situations where you’ve got stuck or you simply want to have AutoUpgrade start from the beginning again. As you saw in my previous example AutoUpgrade will resume the upgrade from where it has been stopped. But if you’d like to have it start from the beginning again, you have to erase its memory.
The unofficial way would be to simply clean up the entire log tree.
But before doing such things, wait a bit. In which state is the database when the upgrade got stopped? It is still in UPGRADE
mode and started in the future environment? Set your environment correctly for the Oracle Home you are upgrading to. And then connect to the database. If the database is down, start it up in UPGRADE
mode.
SQL*Plus: Release 19.0.0.0.0 - Production on Tue Apr 7 21:42:35 2020 Version 19.6.0.0.0 Copyright (c) 1982, 2019, Oracle. All rights reserved. Connected to an idle instance. SQL> startup upgrade ORA-01078: failure in processing system parameters LRM-00109: could not open parameter file '/u01/app/oracle/product/19/dbs/initDB12.ora'
Oh. The SPFILE hasn’t been copied to the new home. But no worries, the AutoUpgrade tool has created the init.ora for after the upgrade already. You will find the files in this directory under your log tree:
[DB12] oracle@hol:~/logs/DB12/temp $ ls -lrt total 12 -rwx------. 1 oracle dba 1391 Apr 7 12:46 before_upgrade_pfile_DB12.ora -rwx------. 1 oracle dba 1391 Apr 7 12:46 after_upgrade_pfile_DB12.ora -rwx------. 1 oracle dba 1417 Apr 7 21:24 during_upgrade_pfile_DB12.ora -rwx------. 1 oracle dba 0 Apr 7 21:28 db12.restart
Use this file marked in red to start the database in STARTUP UPGRADE
mode:
SQL> startup upgrade pfile=/home/oracle/logs/DB12/temp/after_upgrade_pfile_DB12.ora
ORACLE instance started.
Total System Global Area 1258290752 bytes
Fixed Size 8896064 bytes
Variable Size 318767104 bytes
Database Buffers 922746880 bytes
Redo Buffers 7880704 bytes
Database mounted.
Database opened.
SQL>
So before doing anything, you’ll have to differ whether you ran the upgrade with upg1.restoration=yes
(the default when your database is in ARCHIVELOG
mode) or with upg1.restoration=no
. In the latter case, no guaranteed restore point has been written.
Let us check:
SQL> column name format a35
SQL> column guarantee_flashback_database format a30
SQL> select name, guarantee_flashback_database from v$restore_point;
NAME GUARANTEE_FLASHBACK_DATABASE
-------------------------------- ------------------------------
AUTOUPGRADE_221145114461854_DB12 YES
Ok, this means your database is in ARCHIVELOG
mode, AutoUpgrade has set a GRP – and you can flashback to it. You can do this from within SQL*Plus or – of course – in AutoUpgrade as well. I will do the latter to demonstrate it:
Flash me back, AutoUpgrade!
At first, I attach back to AutoUpgrade, and it will automatically resume.
$ java -jar $OH19/rdbms/admin/autoupgrade.jar -config DB12.cfg -mode deploy Previous execution found loading latest data Total jobs recovered: 1 +--------------------------------+ | Starting AutoUpgrade execution | +--------------------------------+ Type 'help' to list console commands
Then I will abort the job.
upg> abort -job 101 Are you sure you want to abort job [101] ? [y|N] y Abort job: [101][DB12] upg> ------------------------------------------------- Errors in database [DB12] Stage [DBUPGRADE] Operation [STOPPED] Status [ERROR] Info [ Error: UPG-1419 [Unexpected Exception Error] Cause: This indicates that the upgrade has been killed, by an abort or restore command. For further details, see the log file located at /home/oracle/logs/DB12/101/autoupgrade_20200407_user.log] ------------------------------------------------- Logs: [/home/oracle/logs/DB12/101/autoupgrade_20200407_user.log] ------------------------------------------------- upg> lsj +----+-------+---------+---------+------+--------------+--------+--------+--------+ |Job#|DB_NAME| STAGE|OPERATION|STATUS| START_TIME|END_TIME| UPDATED| MESSAGE| +----+-------+---------+---------+------+--------------+--------+--------+--------+ | 101| DB12|DBUPGRADE| STOPPED| ERROR|20/04/07 21:22| N/A|21:58:34|UPG-1419| +----+-------+---------+---------+------+--------------+--------+--------+--------+ Total jobs 1
And finally I will restore it, which will complete the flashback to the guaranteed restore point.
upg> restore -job 101 The job 101[DB12] on stage [DBUPGRADE] has the status [ABORTED] Are you sure you want to restore? all progress will be lost [y/N] y upg> lsj +----+-------+----------+---------+-------+--------------+--------+--------+----------------+ |Job#|DB_NAME| STAGE|OPERATION| STATUS| START_TIME|END_TIME| UPDATED| MESSAGE| +----+-------+----------+---------+-------+--------------+--------+--------+----------------+ | 101| DB12|GRPRESTORE|EXECUTING|RUNNING|20/04/07 21:22| N/A|22:00:52|Restore Phase[4]| +----+-------+----------+---------+-------+--------------+--------+--------+----------------+ Total jobs 1 upg> lsj +----+-------+-----+---------+--------+--------------+--------+--------+-----------+ |Job#|DB_NAME|STAGE|OPERATION| STATUS| START_TIME|END_TIME| UPDATED| MESSAGE| +----+-------+-----+---------+--------+--------------+--------+--------+-----------+ | 101| DB12|SETUP| STOPPED|FINISHED|20/04/07 21:22| N/A|22:01:17|DB Restored| +----+-------+-----+---------+--------+--------------+--------+--------+-----------+ Total jobs 1
This happens really fast.
No restore point, no flashback!
If you ran the upgrade with upg1.restoration=no
, then no guaranteed restore point has been written. You have two choices now. Either restore your backup and attempt the upgrade again, or try to resume it.
If you’d like to resume it, you follow the advice above. Start the database in the new environment in STARTUP UPGRADE
mode, then call AutoUpgrade with -mode deploy
again to resume the upgrade from where it has been stopped.
Start me up!
If you flashed the database back, you can restart the upgrade now again. If you’d like to pertain the previous logs, you may want the next run to have a new job number. For this purpose, the -clear_recovery_data
flag exists. It will erase AutoUpgrade’s memory about the previous runs:
$ java -jar $OH19/rdbms/admin/autoupgrade.jar -config DB12.cfg -mode deploy -clear_recovery_data The recovery data was removed, the AutoUpgrade will start from the beginning AutoUpgrade tool launched with default options Processing config file ... +--------------------------------+ | Starting AutoUpgrade execution | +--------------------------------+ 1 databases will be processed Type 'help' to list console commands upg>
The new run gets a new job number when you used the -clear_recovery_data
flag:
upg>lsj
+----+-------+---------+---------+-------+--------------+--------+--------+---------------+
|Job#|DB_NAME| STAGE|OPERATION| STATUS| START_TIME|END_TIME| UPDATED| MESSAGE|
+----+-------+---------+---------+-------+--------------+--------+--------+---------------+
| 102| DB12|PRECHECKS|PREPARING|RUNNING|20/04/07 22:15| N/A|22:15:29|Loading DB info|
+----+-------+---------+---------+-------+--------------+--------+--------+---------------+
Total jobs 1
upg>
Summary
In brief, the important question is: Did you run your upgrade with the default of having a guaranteed restore point, or not. You may need to start the database in the new environment with STARTUP UPGRADE
by using the init.ora
AutoUpgrade did create initially.
From there you can decide whether you’d like to flashback to the guaranteed restore point, either manually or within AutoUpgrade. Or if you’d like to resume the upgrade from where it has been stopped.
If you want to erase the memory of AutoUpgrade, use the -clear_recovery_data
flag when you call AutoUpgrade with -mode deploy
again.
Additional Information and Links
- MOS Note: 2485457.1 – AutoUpgrade Tool Download
- The new AutoUpgrade – Step by Step
- AutoUpgrade – New Version with RAC Support
–Mike
Hi Mike,
it would be good if the customer could choose the autoupgrade tool when creating a service request. This eliminates unnecessary questions from the support staff
Regards
Uwe
Uwe,
I agree by 100% – and on your behalf I will send another email to my support mates. But so far all attempts have not lead to any change 🙁
Cheers,
Mike
Hi Mike,
it looks like your colleagues have heard you When opening a service request it is now possible to select “autoupgrade”.
I have used it twice today 🙂
Regards
Uwe
hahaha!
Thanks for the feedback, Uwe!
Cheers,
Mike
I ran into a situation where upgrade was failing and workaround was setting a parameter in the spfile, I tried using the available method is adding in my config file the add during upgrade section but it didn’t took it, next I added manually in the pfile ” used during the upgrade” in the temp dir and it didn’t work either, so I had to abort job, change spfile and start over, so adding a init parameter mid upgrade (after starting and during errors) is not currently supported?
Hi Jason,
good idea – I think I will need a 2nd blog post explaining more tweaks and hacks 🙂
Give me a few days – will be there soon.
Cheers,
Mike
Hello Mike,
thanks for this (as usual) very informative post. Is there a possibility to ignore an upgrade error if Oracle Support tells you, that you can safely ignore it and resume?
On a couple of databases we maintain we ran into unpublished Bug 30084975 – 19C UPGRADE FAILS WITH ERROR “ORA-01403: NO DATA FOUND” caused by a18.sql in as we learned through our SR rare cases (auto partitioning with key value in CHAR Format).
Resume tries to rerun the Script (as it should) so it reproduces the error.
Thanks to Oracle Support we know the upgrade finishes ok and we can resume manual with creating the missing files (spfile, passwordfile) and execute the post upgrade tasks.
I am just curious if there is a more elegant way to solve this (e.g. to tell autoupgrade to ignore this error or to just resume after the failing script)?
Cheers,
Walter
Hi Walter,
no, you can’t skip such an issue in AutoUpgrade.
The bug will happen with a standard upgrade as well, and the a18.sql script issue has been tracked by: bug 31088341 (a18.sql needs changes in the repair tabpart$ portion added as part of fix #28757685.)
This one is fixed in 21c – and got logged for 19.3.0.
Interestingly, there are requests for inclusion for 12.2 and 18c, but none for 19c yet.
Did Oracle Support propose a fix?
I will discuss this with the team as well as we don’t want AutoUpgrade to fail here.
Cheers, and thanks for bringing this to our attention,
Mike
Hi Mike,
AutoUpgrade failed and errror in autoupgrade.err.log says below .
2020-05-10 02:04:26.287 ERROR The following checks have ERROR severity and no fixup is available or
the fixup failed to resolve the issue. Fix them manually before continuing:
P04PBDCD_2 PARAMETER_OBSOLETE – FixesToolBox.evaluateNonFixables
2020-05-10 02:04:26.459 ERROR Database fixups completed – ExecuteFixups.runFixups
2020-05-10 02:04:26.460 ERROR Upgrade Dispatcher failed. Cause: AutoUpgException [UPG-1312] – DispatcherExecuteContext.callExecuteContext
oracle.upgrade.autoupgrade.utils.errors.AutoUpgException: AutoUpgException [UPG-1312]
at oracle.upgrade.autoupgrade.dbchecks.FixesToolBox.evaluateNonFixables(FixesToolBox.java:195)
I restart the AutoUpgrade by resetting the value of obsolete parameter but it’s showing stopped even after killing/exiting/aborting the session and it start with the same job_id which was failed first time.
upg> resume -job 104
Resuming job: [104][P04PBDCD_2]
upg> lsj
+—-+———-+———+———+——-+————–+————–+——–+——-+
|Job#| DB_NAME| STAGE|OPERATION| STATUS| START_TIME| END_TIME| UPDATED|MESSAGE|
+—-+———-+———+———+——-+————–+————–+——–+——-+
| 104|P04PBDCD_2|PREFIXUPS| STOPPED|RUNNING|20/05/10 02:03|20/05/10 03:01|03:01:23| |
+—-+———-+———+———+——-+————–+————–+——–+——-+
I have raised “SR 3-23069838961 : AutoUpgrade failed ” and Oracle is saying that issue is with AutoUpgrade version but have upgraded another database with same version 2 days back.
Could you please help on this ?
Sandeep
Hi Sandeep,
when Support asks you to use a newer version, have you tried?
Or did you receive the information that the failure is known but not fixed yet?
Did you receive a bug or Jira reference number?
Cheers,
Mike
Hi Mike,
I used latest version only to start the upgrade (Doc ID 2485457.1).
No I haven’t received any number from them and got
from Oracle Engineer that they have raised predefect with ARE team on the (SR 3-23069838961 : AutoUpgrade failed 19C).
oracle@/u01/app/oracle/product/19.0.0/dbhome_1/jdk/bin/java -jar /u01/app/oracle/product/19.0.0/dbhome_1/rdbms/admiversionpgrade.jar -version
build.hash 255dd7d
build.version 19.9.0
build.date 2020/04/23 15:01:36
build.max_target_version 19
build.supported_target_versions 12.2,18,19
build.type production
Hi Sandeep,
can you please insist via the SR to get proper information?
I’m completely “under water” at the moment. I will check your SR later today or tomorrow. But please force Oracle Support to deliver proper information. If this is a known issue, they should be able to map this to a bug or Jira. If this is new, they need to file a new bug. If they have done neither thing, something is wrong here.
Thanks,
Mike
Hi Mike,
I received below response from Oracle support .
############################
The issue which you are facing looks like a new issue there is no information available right now.
3- What actions have been taken to date? What is your current diagnosis?
DETAILS:
========
Autoupgrade is continuous to fail with each attempt.
Tried to following workaround suggested. Customer confirmed after second attempt the recyclebin$ becomes empty.
Unable To Empty or delete rows from Sys.recyclebin$ which is causing dbua (upgrade) stopped and purge dba_recyclebin not helping ( Doc ID 1910945.1 )
As per the previous known SR # 3-21170553821 : (Closed) Issues with autoupgrade from 12.1.0.2 to 19c. Some what similar issue. Where customer try to reload the OJVM in the database and reattempt. But customer says their OJVM is valid and there is no issue what soever. So customer wants to know the way to cleanup the failed Autoupgrade and reattempt.
4) What have you discovered during your efforts to resolve the problem? Some examples would be : Debugged the execution of OUI, where you think the time is spent, particular entries in a registry like old Oracle homes and what it is doing, specific entries in a ORACLE_HOME etc
As per the Autoupgrade log, the initial issue reported that the PARAMETER_OBSOLETE. So, Customer has
020-05-10 02:04:26.287 ERROR The following checks have ERROR severity and no fixup is available or
the fixup failed to resolve the issue. Fix them manually before continuing:
P04PBDCD_2 PARAMETER_OBSOLETE – FixesToolBox.evaluateNonFixables
2020-05-10 02:04:26.459 ERROR Database fixups completed – ExecuteFixups.runFixups
2020-05-10 02:04:26.460 ERROR Upgrade Dispatcher failed. Cause: AutoUpgException [UPG-1312] – DispatcherExecuteContext.callExecuteContext
oracle.upgrade.autoupgrade.utils.errors.AutoUpgException: AutoUpgException [UPG-1312]
at oracle.upgrade.autoupgrade.dbchecks.FixesToolBox.evaluateNonFixables(FixesToolBox.java:195)
try to verify there were any dead lock at that period time in the database and none found.
AutoUpgrade.jar fails in preupgrade fixup phase with deadlock issues when DBA_PRIV_CAPTURES is enabled ( Doc ID 2593325.1 )
5) Is there a workaround ? (if so, give full details)
According to the Bug 30490988 [https://bug.oraclecorp.com/pls/bug/webbug_edit.edit_info_top?rptno=30490988] , the workaround is to delete the autoupgrade logs and redo the autoupgrade. but customer downtime is over and they will have to wait until next window.
6) Why do you believe this is a defect?
Autoupgrade is not cleaning up the failed attempts and continuous take last upgrade attempt job id: 104 and failing with the same message.
7) Why do you believe the problem is a bug with Oracle ?
The Autoupgrade is failing with the job id: 104 continuously and there is no information available in the MOS.
8) List any bugs you have researched and believe may be related to the problem.
Bug: 30490988 [https://bug.oraclecorp.com/pls/bug/webbug_edit.edit_info_top?rptno=30490988] : RHP: 20C AUTO UPGRADE RETRY FAILED AFTER FAILURE FOR PREUPGRADE CHECK
#################################################
Sandeep
Hi Mike ,
I got below Update from Oracle .
###############
The issue which you are facing looks like a new issue there is no information available right now.
3- What actions have been taken to date? What is your current diagnosis?
DETAILS:
========
Autoupgrade is continuous to fail with each attempt.
Tried to following workaround suggested. Customer confirmed after second attempt the recyclebin$ becomes empty.
Unable To Empty or delete rows from Sys.recyclebin$ which is causing dbua (upgrade) stopped and purge dba_recyclebin not helping ( Doc ID 1910945.1 )
As per the previous known SR # 3-21170553821 : (Closed) Issues with autoupgrade from 12.1.0.2 to 19c. Some what similar issue. Where customer try to reload the OJVM in the database and reattempt. But customer says their OJVM is valid and there is no issue what soever. So customer wants to know the way to cleanup the failed Autoupgrade and reattempt.
4) What have you discovered during your efforts to resolve the problem? Some examples would be : Debugged the execution of OUI, where you think the time is spent, particular entries in a registry like old Oracle homes and what it is doing, specific entries in a ORACLE_HOME etc
As per the Autoupgrade log, the initial issue reported that the PARAMETER_OBSOLETE. So, Customer has
020-05-10 02:04:26.287 ERROR The following checks have ERROR severity and no fixup is available or
the fixup failed to resolve the issue. Fix them manually before continuing:
P04PBDCD_2 PARAMETER_OBSOLETE – FixesToolBox.evaluateNonFixables
2020-05-10 02:04:26.459 ERROR Database fixups completed – ExecuteFixups.runFixups
2020-05-10 02:04:26.460 ERROR Upgrade Dispatcher failed. Cause: AutoUpgException [UPG-1312] – DispatcherExecuteContext.callExecuteContext
oracle.upgrade.autoupgrade.utils.errors.AutoUpgException: AutoUpgException [UPG-1312]
at oracle.upgrade.autoupgrade.dbchecks.FixesToolBox.evaluateNonFixables(FixesToolBox.java:195)
try to verify there were any dead lock at that period time in the database and none found.
AutoUpgrade.jar fails in preupgrade fixup phase with deadlock issues when DBA_PRIV_CAPTURES is enabled ( Doc ID 2593325.1 )
5) Is there a workaround ? (if so, give full details)
According to the Bug 30490988 [https://bug.oraclecorp.com/pls/bug/webbug_edit.edit_info_top?rptno=30490988] , the workaround is to delete the autoupgrade logs and redo the autoupgrade. but customer downtime is over and they will have to wait until next window.
6) Why do you believe this is a defect?
Autoupgrade is not cleaning up the failed attempts and continuous take last upgrade attempt job id: 104 and failing with the same message.
7) Why do you believe the problem is a bug with Oracle ?
The Autoupgrade is failing with the job id: 104 continuously and there is no information available in the MOS.
8) List any bugs you have researched and believe may be related to the problem.
Bug: 30490988 [https://bug.oraclecorp.com/pls/bug/webbug_edit.edit_info_top?rptno=30490988] : RHP: 20C AUTO UPGRADE RETRY FAILED AFTER FAILURE FOR PREUPGRADE CHECK
################
Actually bug 30490988 is not even with the AutoUpgrade team. And I tried to understand it – I don’t.
The bug has been placed in status 30 since December meaning, the bug owner wants to bug filer to deliver more information.
This hasn’t happened so nothing will happen here.
Please point your support engineer to either file a new issue or update the bug he or she gave you.
Thanks,
Mike
Hi Mike,
Wasn’t planning on the upgrade tool but seen you’re preso from 29 april. However when trying a 12.2 CDB database I got:
2020-06-11 17:11:02.690 ERROR It was not possible to determine the version value of database TRO7CBO from the returned value select version_full from sys.v$instance – UpgradeConfigDBValidator.findDBVersion
Seems it’s trying to use a column which is not yet availabe in 12.2
Created SR 3-23306993601 but maybe you or some other reader has a quick solution.
Hi Laurens,
please let me know whether you try to upgrade TO 12.2.0.1 or TO 19c?
Then, have you set your environment correctly when invoking autoupgrade?
If you upgrade to 19c, did you set the environment to your 19c home?
Does the CDB you’d like to plugin to exist already within the same environment?
Thanks,
Mike
The autoupgrade for 12.1.0.4 to 19.5.2 failed due to compilation error during upgrade
ORA-02290: check constraint (SYS.JAVA_DEV_DISABLED) violated, I had used -restore_on_fail option
I applied the workaround of enabling the dev java and recompled the objects manually and now registry shows all valid
In fact I had applied this workaround before the upgrade but somehow dev java got disabled during the upgrade
I resumed the job but it went ahead and tried to restore the database from GRP, instead of executing the post upgrade, how can I change this behavior so that I can resume from the point where it failed and continue with the next steps in the upgrade workflow that is post upgrade instead of executing the restore
Hi Haresh,
it rings a bell with me – I guess this wouldn’t have happened if you’d upgraded to a newer release. I filed bugs for the Mitigation Patch a while ago. And they all got fixed. But I would need to double check whether this got included into an RU already.
Have you checked this with Oracle Support?
Cheers,
Mike
Of course I use the atest version as of now and I followed the above advice, but now getting the error “Path does not exist at ‘/opt/oracle/diag/rdbms/SID/SID/trace/alert_SID.log’
Oracle uses $ORACLE_BASE/diag/rdbms/sid/SID/trace/… as diag pathname.
It doesn’t matter wether pwd is the trace path or not, it simply doesn’t use the same rule to build the diag pathname. Sic.
Hi Gerrit,
is the uppercase the problem?
Can you copy&paste or mail the complete error sequence?
Cheers,
Mike
Hi Mike ,
My two of the database failed today during postfixup phase and both failed due to same error .
I used latest AutoUpgrade version “build.date:2020/12/14 14:41:34″
The error was very strange and raised ” SR 3-24990637171 : AUTOUPGRADE Error running dispatcher”
Could you please help me to understand why it failed ?
upg> lsj
+—-+——-+———–+———+——+————–+——–+——–+
|Job#|DB_NAME| STAGE|OPERATION|STATUS| START_TIME| UPDATED| MESSAGE|
+—-+——-+———–+———+——+————–+——–+——–+
| 104|CPL01_2|POSTUPGRADE| STOPPED| ERROR|21/01/17 01:39|04:09:42|UPG-1699|
2021-01-17 04:09:26.460 INFO Copying/merging file sqlnet.ora ended
2021-01-17 04:09:42.323 INFO Context text file list /home/oracle/CPL01/upg_logs/CPL01_2/temp/ctx_move_text_file_list_CPL01_2 not found, skipping…
2021-01-17 04:09:42.333 ERROR Dispatcher failed: Unknown instance CPL01_2
2021-01-17 04:09:42.334 INFO Starting error management routine
2021-01-17 04:09:42.335 INFO Ended error management routine
2021-01-17 04:09:42.335 ERROR Error running dispatcher for job 104
Cause: Error finding error definition, contact Oracle Support
2021-01-17 04:09:42.335 ERROR Dispatcher failed:
Error: UPG-1699
[Unexpected exception error]
Cause: Error finding error definition, contact Oracle Support
Regards,
Sandeep
Hi Sandeep,
please see the SR you’ve raised. The developer needs your logs urgently as otherwise the bug Support filed on your behalf can’t be worked on:
java -jar autoupgrade.jar -config yourconfig.cfg -zip
Plus the alert.log as well.
Cheers,
Mike
Hi Mike,
Uploaded .
Hello all,
I can upgrade my database (12.1.0.2) with autoupgrade.jar version 19.10.0 (build date 2020/10/23 10:36:46) but i encouter an error if i try to upgrade the same database with autoupgrade.jar version 21.1.1 ( build.date 2020/12/14 14:41:34).
2021-01-22 09:39:07.071 ERROR Exception Error in Database Upgrade [CDB0BD1_lxpt240b]
2021-01-22 09:39:07.111 ERROR cdb0bd1_lxpt240b Return status is ERROR
2021-01-22 09:39:07.115 ERROR Dispatcher failed: AutoUpgException [UPG-1400#UPGRADE FAILED [CDB0BD1_lxpt240b]]
2021-01-22 09:39:07.118 INFO Starting error management routine
2021-01-22 09:39:07.120 INFO Ended error management routine
2021-01-22 09:39:07.121 ERROR Error running dispatcher for job 100
Cause: Database upgrade failed with errors
2021-01-22 09:39:07.122 ERROR Dispatcher failed:
Error: UPG-1400
UPGRADE FAILED [CDB0BD1_lxpt240b]
Cause: Database upgrade failed with errors
For further details, see the log file located at /home/logiciel/oracle/expl/log/upgrade_19c/CDB0BD1/DEPLOY/Log_20210122_0933/upgr/CDB0BD1/100/autoupgrade_20210122_user.log; aborting job 100 for database CDB0BD1_lxpt240b
Hi Kaddachi,
I can’t see what the actual issue is as UPG-1400 is a sort-of generic error.
You need to check the logs.
So what you do now is summarized here:
https://mikedietrichde.com/2020/04/08/troubleshooting-restoring-and-restarting-autoupgrade/
1. Open an SR
2. Collect the logs with the -zip option
3. Upload the logs and the alert.log and an “opatch lsinventory” output to the SR
Please forgive me but I can’t replace Oracle Support with this blog.
Thanks,
Mike
Ok Mike Thanks,
I have opended an SR and waiting for the answer.
Others questions please :
1. should call to DBMS_PRIVILEGE_CAPTURE.DISABLE_CAPTURE prior to upgrade on 12.1.0.2 Database ?
2. should i disbale all triggers “ON DATABASE” before upgrade ?
Thanks
Hi,
for the triggers, clearly NO as upgrade disables them.
I never tried to have the privilege capture ON before upgrade, but I guess that it will be stopped automatically.
Cheers,
Mike
Hi:
I could not find a solution to the problem I am facing when using autoupgrade tool to upgrade my database to 19c. It connects to the database and then fails with the following error:
ERROR Errors while connected to DB, edssorcl – JobRulesValidator.process
I have been working with Oracle support for a couple of weeks now but haven’t gotten any solution.
Details that may be useful:
Autoupgrade version:
build.hash e84c9c2
build.version 19.10.0
build.date 2020/10/23 10:36:46
build.max_target_version 19
build.supported_target_versions 12.2,18,19
build.type production
Current DB Version: 18.3.0.0.0
$ $ORACLE_HOME/bin/orabase -> /u01/app/oracle
$ $ORACLE_HOME/bin/orabasehome -> /u01/app/oracle/product/ora1800/db_1
If you need any additional details, please let me know.
Thanks,
Kishan.
Thank you.
– Kishan.
Hi Kishan,
what is the SR number? Without the logs, I can’t tell you anything.
Cheers,
Mike
Hi Mike and All,
Many weeks ago, i wrote :
I can upgrade my database (12.1.0.2) with autoupgrade.jar version 19.10.0 (build date 2020/10/23 10:36:46) but i encouter an error if i try to upgrade the same database with autoupgrade.jar version 21.1.1 ( build.date 2020/12/14 14:41:34).
=====
After analyze with Oracle Support, the bug is cauded by the ORACLE_HOME_19c/sqlplus/admin/glogin.sql.
So : Rename /home/logiciel/oracle/ora19600_2001/sqlplus/admin/glogin.sql to something else and then try the upgrade
this sloved the isue.
And theywilll fix this on AutoUpgrade, for time being you will need to use this workaround.
Thanks for the update.
Cheers,
Mike
hi Mike,
SR 3-25661297661
It is strange, autoupgrade can’t find the right RAC db spfile.
2021-04-12 12:22:34.141 INFO Updating parameter *.cluster_database=’true’ to *.cluster_database=’FALSE’ in /oracle/app/upgrade19c/autoupgrade/CRSDB06/CRSDB061/temp/during_upgrade_pfile_crsdb061.ora
2021-04-12 12:22:51.135 INFO Analyzing CRSDB061, 223 checks will run using 32 threads
2021-04-12 12:27:19.658 ERROR
DATABASE NAME: CRSDB061
CAUSE: ERROR at Line 4 in [Buffer]
REASON: LRM-00109: could not open parameter file ‘/oracle/app/oracle/product/18.3.0/dbhome_1/dbs/initCRSDB061.ora’
ACTION: [MANUAL]
DETAILS: 109, 0, “could not open parameter file ‘%.*s'”
// *Cause: The parameter file does not exist.
// *Action: Create an appropriate parameter file.
Thanks
Luke
Thanks
Hi Luke,
as you have an SR already, please go forward with Oracle Support.
Thanks,
Mike
Hi Mike,
Your blogs regarding AUTOUPGRADE saved plenty of time 🙂 Thanks for that!
I was trying to upgrade multiple databases using AUTOUPGRADE tool, initially fired so many errors in mid of upgrades each time, rectified everything and somehow upgraded the databases.
My only concern here is, post upgrade completion in windows server, DB and Lsitener services were not automatically created. I had to manually delete services pointing to OLD DB HOME and create new services in order for ‘STARTUP UPGRADE’ to work. Should the services to be created manually or its the usual behaviour of autoupgrade for Windows OS (version 2016 datacenter)
Hi Aarthi,
see here – and forgive me that I didn’t write a blog post especially for AU on Windows yet:
https://docs.oracle.com/en/database/oracle/oracle-database/19/upgrd/autoupgrade-microsoft-windows-access-cointrol-lists.html#GUID-A621851A-CAC8-4BB7-828B-CE5DA7DC62E7
AutoUpgrade uses the following procedure with services running on the database:
AutoUpgrade stops the services on the source database, and creates a temporary service on the target database Oracle home.
If a restore is required, then the service in the target is dropped, and the service in the source is restarted.
After Deploy Mode processing has completed successfully, the service in the target is dropped. At that point, it is the responsibility of the DBA for the upgraded Oracle Database to use ORADIM to create a service. Creating this service manually is required, because AutoUpgrade does not have the password to obtain permissions to create the ORADIM service.
The root cause for the need that You (or the DBA) needs to create the service is that AU can’t ask you for your admin password. We are not allowed to do this. So yes, on Win actually there is this manual task necessary.
Cheers,
Mike
hi,
i’m upgrading 2 RAC databases in AIX 7.2
from 18.4 to 19.11
using autoupgrade.jar version : build.hash e84c9c2 | build.version 19.10.0 | build.date 2020/10/23 10:36:46
1st env (dev – 2node RAC) went smoothly
2nd env (tst – 2node RAC) although console reported (empty lines stripped):
upg> Job 103 completed
——————- Final Summary ——————–
Number of databases [ 1 ]
Jobs finished successfully [1]
Jobs failed [0]
Jobs pending [0]
————- JOBS FINISHED SUCCESSFULLY ————-
Job 103 for FRGNS1
—- Drop GRP at your convenience once you consider it is no longer needed —-
Drop GRP from FRGNS1: drop restore point AUTOUPGRADE_9212_FRGNS1184000
xxxxxxtst:/home/oracle #
yet the following are missing
1) /xxx/cfgtoollogs/upgrade/auto/status/FRGNS1_sysupdates.success
2) /xxx/cfgtoollogs/upgrade/auto/status/status.log
3) /xxx/cfgtoollogs/upgrade/auto/status/status.html
4) /xxx/FRGNS/FRGNS1/103/sysupdates
i checked log from DEV where it went smoothly and if i get it right the only thing that sysupdates is attempting to do is
register db with new db home in crs and enable db
in TST env where the problem occured crs has the new home registered and db is enabled
srvctl and crsctl are working ok and db is restarted without issues many times,
still i’m not sure if the “process” failed or succeded after all.
can you pls give me any help ? can i find any info regarding “SYSUPDATES”
was it executed and failed to write log or failed completly and i should rollback and retry ?
tnx
nikos
i think i found my mistake
i accidentally copied autoupgrade.jar in rdbms/admin before running the runinstaller -applyru
thus jar was overwritten when applyru 32545008 (19.11).
TST env where no issue occurred was running with latest autoupgrade build.version 21.1.3
i assume that “sysupdates” might be part of 21 build but not 19.10.
yet the question actually stands
console reported no errors – i can’t find any errors in db alert log – srvctl/crsctl work fine
but i don’t have status.html and don’t know if i should rollback and start over or not
tnx
nikos
Ah, got you – thanks 🙂
Cheers,
Mike
The version of AU you are using is 9 months old – your upgrade was successful but this is why you miss some things.
This is the newest:
https://mikedietrichde.com/2021/07/22/new-version-of-autoupgrade-21-2-is-available-for-download/
Cheers,
Mike
Hello Mike,
Autoupgrade failed in Post Upgrade Step – OLD_TIME_ZONES_EXIST, it looks to be a problem updating/merging the tnsnames.ora file.
We have checked permissions on any related directories and they appear fine, Created a SR 3-26753776961,
We did not find any other issues, all the validation checks/views dba_registry, registry$history, and registry$sqlpatch looks right,
they are suggested to create manually the tnsnames.ora file and completed the upgrade.
We are planning for production upgrade now, the Oracle Support tech suggested to use DBUA or manual upgrade process to avoid this error,
Any suggestions or thoughts? Appreciate your help.
Hi Kishan,
thanks for bringing this to our attention. We will check back with the engineer.
And thanks for insisting in the SR to get a solution for the issue itself.
I brought the SR to my managament’s attention – we will check and monitor to make sure the issue gets diagnosed and fixed.
Thanks,
Mike
Hi, Autoupgrade failed in Post Upgrade Step – OLD_TIME_ZONES_EXIST, it looks to be a problem updating/merging the tnsnames.ora file. We did checked permissions on any related directories and they appear fine, Created a SR, and did not find any other issues, all the validation checks/views dba_registry, registry$history, and registry$sqlpatch looks right,
they are suggested to create manually the tnsnames.ora file and completed the upgrade. The upgrade seems to working fine so far. We are planning for production upgrade now, the Oracle Support tech suggested to use DBUA or manual upgrade process to avoid this error,
Any suggestions or thoughts? Appreciate your help.
regards,
Hi Kishan, see my other comment.
We will follow up with the support person.
Cheers,
Mike
Hi Mike,
Thanks to all the masterminds behind the auto upgrade tool. This saves a lot of time and effort.
We are doing POC to upgrade the databases from 12c to 19c running on windows server 2019 with Oracle 12c standard edition. The upgrade is failing with Error: UPG-1400 UPGRADE FAILED. Looking at the catupg logs we see REASON: SP2-0556: Invalid file name. REASON: SP2-0310: unable to open file “LOGIN.SQL”. I understand the search behaviour of login.sql has changed from 12c. We did use the SQLPATH environment variable pointing to login.sql but the issue persists. Also followed the instruction in the Doc ID 2593418.1 (auto Upgrade Fails with UPG-1400#UPGRADE FAILED). We have also opened an SR 3-27148783861 and no luck. Any pointers to this issue?
Appreciate your help.
Thx,Kumar
Hi Kumar,
I’m pretty certain that Support has helped you already – otherwise let me know.
Make sure you are using the most recent autoupgrade from the MOS Note.
Thanks,
Mike
If you run AutoUpgrade with a future start_time but now don’t want to run it, how do you sort of kill, stop/unsubmit it?
Hi Edwin, call AU and abort the job with -abort -job
Cheers,
Mike
Hi Mike
A bit stumped here, when I do a -help, I don’t actually see the -restore option. I can see a -restore_on_fail, but not -restore. AutoUpgrade that I have is buildversion 19.9.1 for Solaris and build.version 19.10.0 for Linux. Is this a top secret parameter so the user will be forced to read the full documentation 🙂
Please always use the most recent AU and download it from the MOS note.
19.9.1 and 19.10.0 are pretty old – please download the most recent version from MOS.
And regarding “restore”, see the doc:
https://docs.oracle.com/en/database/oracle/oracle-database/19/upgrd/autoupgrade-command-line-parameters.html#GUID-B969F325-EB44-42B3-AD93-43E47493E271
Cheers,
Mike
Thanks Mike, yeah. I downloaded the latest one and it shows that -restore when I do a help? It looks like someone may have used a different JAR file from the one that he said he used. Anyway, looks like, my AutoUpgrade is now all cooked up. Since I used an older JAR file when I did the deploy, the one without the -restore option, I am not able to do a -restore even though I downloaded the latest JAR file that has the -restore option? 🙁 Looks like I have to do a manual option instead 🙁
I am using Autoupgrade with the -mode -noconsole option and running it via the cron. It runs successfully fine. I am trying to do a rollback with -restore -job but it doesn’t appear to be an acceptable option to run. Is this by design? So the restore -job is only when you run it in console mode?
Hi Edwin,
-restore is not meant to be run in unattended mode.
Cheers,
Mike
Hello All,
Can we prevent autoupgrade to modify /etc/oratb and modify listener.ora ?
Thanks
No, unfortunately you can’t. It is supposed to take care on it.
Cheers,
Mike
Hello Mike
I’m refreshing a PDB from 12.2 into a CDB (19.14) using “CREATE PLUGGABLE DATABASE” from a clone database link. I can’t seem to find how to use autoupgrade correctly. It appears to want to upgrade the whole database.
How do I just upgrade one PDB? I can use catctl.pl, but that seems old fashioned now.
Thanks
Hi Ben,
the next version of AU will support it – the current doesn’t.
Cheers,
Mike
Hello Mike
Have run into a particular issue when the upgrade hangs for while almost indefinitely. Usually the upgrades will take no more than hour to complete and does not matter the size of the db instance.
I am upgrading db from 12.2.0.1 to 19.3 (19.11). This db upgrade takes close to 3 hours to complete.
I will well get message from these message from console for example:
upg> ————————————————-
job 101 has not shown progress in last 25 minutes
database [db10p]
Stage [PREFIXUPS]
Operation[EXECUTING]
Status [RUNNING]
Info [Remaining 3/4]
[Review log files for further information]
———————————————–
Logs: /u01/autoupgrade/db10p/101
Eventually the upgrade will hang at 95% until it completes within 2hrs and 45 minutes.
Any insight or recommendation.
Thanks,
Ignacio
Hi Ignacio,
I can’t diagnose this blindly. It could be the time zone change but there are many other potential reasons.
You need to gather the logs with
java -jar autoupgrade.jar -config yourconfig.cfg -zip
and upload it to support.
Thanks,
Mike
Hello Mike,
I am using the autoupgrade.jar for upgrading the DBs, so far it was very stable and consistent. But, in one of our DB environments, we are facing slowness like x10 slower upgrade time. DB version is from 12c to 19.14 and we are using the latest autoupgrade.jar file. During the tests, autoupgrade tool popped up a warning message like “25mins no activity” but the upgrade completed successfully in 3+ hours.
There are no errors on the log files.
I would be happy if you can your thoughts, or advise us how to investigate the issue.
Thank you
Adrian
Hi Adrian,
you must please open an SR and check with Support, There could be so many issues leading to this. I can’t blindly diagnose it without going through the logs.
You can find catupgrd0.log in the /dbupgrade directory of AutoUpgrade. Check for “elapsed” with a large time stamp, and then lower it until you find a hit. This may give you an indication what has taken how long.
Thanks
Mike
Hi Mike
Besides checking the log, is there a way to connect to the console from another session and check the status from there? So, let’s say I run the deploy on session1 and shift change 🙂 and another DBA come around connect to the same host as session2, can he connect to the same running deploy job and check the status from that session?
Hi Edwin,
you can always check the logs, the HTML state.html file from any other session to see what’s going on. But the tool is session-based.
Cheers,
Mike
Hi Mike,
The autoupgrade failed with an error of wallet not open , actually there was no wallet.
after which I thought of restoring the job to check the wallet status and that too failed.
now the original db is not mounting due to controlfile error.
alter database mount
*
ERROR at line 1:
ORA-00205: error in identifying control file, check alert log for more info
I copied one of old wallet. and checked the wallet
STATUS
——————
file
/sw01/oracle/admin/dbname/wallet
OPEN
Can you please suggest I am stuck here
Thanks
Smitah
Please make sure you use the absolute latest AU from MOS – and if it still fails, zip the logs together with:
java -jar autoupgrade.jar -config yourconfig.cfg -zip
and upload them to Support when you open an SR.
Thanks,
Mike
Hello Mike,
First of all, thank you so much for your posts, it helped me upgrade my DB very smoothly to 19c. The autoupgrade log shows as below:
Number of databases [ 1 ]
Jobs finished [1]
Jobs failed [0]
Jobs restored [0]
Jobs pending [0]
—- Drop GRP at your convenience once you consider it is no longer needed —-
Drop GRP from deva: drop restore point AUTOUPGRADE_9212_DEVA112040
Please check the summary report at:
/u01/app/oracle/cfgtoollogs/autoupgrade/cfgtoollogs/upgrade/auto/status/status.html
/u01/app/oracle/cfgtoollogs/autoupgrade/cfgtoollogs/upgrade/auto/status/status.log – Exit.summaryInfoSection
2022-09-18 21:26:00.977 INFO The console is closing now as per interruption request – ConsoleReader.run
2022-09-18 21:26:00.977 INFO Closing – ConsoleReader.run
2022-09-18 21:26:00.977 INFO Starting files protection – SetProtection.setGlobalProtections
2022-09-18 21:26:00.978 INFO Protecting global dir – SetProtection.setGlobalProtections
2022-09-18 21:26:00.978 INFO Starting – SetProtection.protectPath
2022-09-18 21:26:00.978 INFO Protecting path /u01/app/oracle/cfgtoollogs/autoupgrade/cfgtoollogs/upgrade/auto – SetProtection.protectPath
2022-09-18 21:26:00.980 INFO Protecting elapsed time 0 seconds – SetProtection.protectPath
2022-09-18 21:26:00.980 INFO End – SetProtection.protectPath
2022-09-18 21:26:00.980 INFO FilesProtector running for job 101 – Exit.lambda$coreExit$3
2022-09-18 21:26:00.980 INFO Starting files protection – SetProtection.setJobProtections
2022-09-18 21:26:00.980 INFO Protecting local dir – SetProtection.setJobProtections
2022-09-18 21:26:00.980 INFO Starting – SetProtection.protectPath
2022-09-18 21:26:00.980 INFO Protecting path /u01/app/oracle/cfgtoollogs/autoupgrade/deva/deva/101 – SetProtection.protectPath
2022-09-18 21:26:00.983 INFO Protecting elapsed time 0 seconds – SetProtection.protectPath
2022-09-18 21:26:00.983 INFO End – SetProtection.protectPath
2022-09-18 21:26:00.983 INFO Starting – SetProtection.protectPath
2022-09-18 21:26:00.983 INFO Protecting path /u01/app/oracle/cfgtoollogs/autoupgrade/deva/deva/temp – SetProtection.protectPath
2022-09-18 21:26:00.984 INFO Protecting elapsed time 0 seconds – SetProtection.protectPath
2022-09-18 21:26:00.984 INFO End – SetProtection.protectPath
2022-09-18 21:26:00.984 INFO End processing cmd [exit] – CommandController.processCmd
2022-09-18 21:26:00.985 INFO Executing global-postupgrade actions – Core.start
and the autoupgrade console seems stuck as below:
java -jar /u01/app/oracle/product/19.0.0/dbhome_1/rdbms/admin/autoupgrade.jar -config deva_config.cfg -mode d
eploy
AutoUpgrade is not fully tested on OpenJDK 64-Bit Server VM, Oracle recommends to use Java HotSpot(TM)
Previous execution found loading latest data
Total jobs recovered: 1
+——————————–+
| Starting AutoUpgrade execution |
+——————————–+
Type ‘help’ to list console commands
upg>
Could you please advise, if any further action needed to mark it as completed. for example, issue “Exit” command at this console manually. But I was expecting this autoupgrade to exit automatically after the upgrade completes.
Thanks in Advance..!!
Hi,
you shoulw at first check with “lsj” and “status -job n” where the upgrade is right now.
Then I personally check also the catupgr0.log and the alert.log if I have no sign of life anymore before I’d “exit”.
Cheers,
Mike
Hello Mike,
Thank you so much the this insight.
I used autoupgrade and my session got terminated in the postfixup steps(timezone upgrade) and when I try to resume, it fails as below:
/u01/app/oracle/product/19.0.0/dbhome_1/jdk/jre/bin/java -jar /u01/app/oracle/product/19.0.0/dbhome_1/rdbms/admin/autoupgrade.jar -config prodvm_upgrade_config.cfg -mode deploy
Previous execution found loading latest data
Total jobs recovered: 0
AutoUpgrade 22.4.220712 launched with default internal options
Processing config file …
———— ERROR ————
Error Cause: Database prodvm shutdown or open with incorrect binaries for DEPLOY. Ensure it is open with /u01/app/oracle/product/11.2.0.4
———— ERROR ————
Error Cause: Database prodvm currently has a status of CLOSED. For DEPLOY mode, open it with one of the following: [OPEN, MOUNTED].
Unable to connect to database prodvm for entry upg1
Could you please advise.
Sorry for the late reply – but you please need to open an SR and upload the AU logs.
I hope you have done so already.
Thanks,
Mike
Mike,
First thank you for all your documentation and helpful hints.
I am using Autoupgrade on a Windows Server (2019) for a non-CDB database. I have run into a problem and I have opened an SR#3-32337730041. The issue I have is the upgrade failed at about 90+% complete. (I failed to note exactly where between 90 and 97 ) I did abort the upgrade job and attempt a restore it went through most of the steps but failed also.
Currently I cannot startup the database in mode (startup, startup upgrade, startup mount) , if I use the pfile to open the db, can I still resume the failed job that I aborted?
Hi Angela,
I commented on one of your later posts. Sorry, I was traveling a lot the past weeks and just found now time to answer the open comments.
Cheers
Mike