Photo by Debby Hudson on Unsplash
In the previous blog posts I explained mainly how to create, alter and tweak the configuration file for the AutoUpgrade tool. By now you should have your config file set up and adjusted for your environment(s). Let me go a step further and explain the different processing modes of AutoUpgrade: ANALYZE, FIXUPS, UPGRADE and DEPLOY modes. Then you’ll have a precise idea what the tool is actually doing, and how it operates.
AutoUpgrade – Step-by-step
- The new AutoUpgrade Utility – Download, documentation and supported versions
- Create and adjust the config file for AutoUpgrade
- Config file for AutoUpgrade – Advanced options
- Config file for AutoUpgrade – Tweaking init parameters
- AutoUpgrade: ANALYZE, FIXUPS, UPGRADE and DEPLOY modes
- AutoUpgrade: Where do you find all the logfiles?
- UPG: The AutoUpgrade Command Line Interface
- Upgrading Multitenant databases with AutoUpgrade
- Moving to a new server with AutoUpgrade
- How to tweak the hidden settings in AutoUpgrade
- AutoUpgrade and Data Guard, RAC, Restart and non-CDB to PDB
- AutoUpgrade and Wallets
Overview – 4 different modes
First of all, you will find the different modes of AutoUpgrade described in the Oracle 19c Upgrade Guide. These modes are:
analyzefixupsupgradedeploy
Analyze Mode
The analyze mode does not change your source database. It just reads from it. And as its name says, it analyzes your database. But it does more than running just the preupgrade.jar. It checks your setup as well.
It produces two files with information about your database, one in html, another one in text format: SID.htmland SID_preupgrade.log.
java -jar autoupgrade.jar -config myconfig.cfg -mode analyze
We highly recommend that you run the analyze phase in all cases. Once it completes successfully without error, you can be sure that the database won’t get any unforeseen surprises during a subsequent unattended upgrade. You will use this mode also for cases where you move to a new server. You’ll execute it on the source server and set source and target homes to the same value (the source home) in your config file.
Fixups Mode
The fixups mode does more than the analyze. It changes your database in the source environment. It performs the same checks as it does in analyze of course before it fixes the findings.
It generates a status file (see the blog post about AutoUpgrade logs). In case you use it for a system move, you will execute the fixups phase on the source environment with source and target home set to the source home’s directory in the config file.
java -jar autoupgrade.jar -config myconfig.cfg -mode fixups
Upgrade Mode
The upgrade mode is only meant for cases where you don’t have access to the source home. You need this mode only for cases where you move to a new server, and the source home does not exist on this server. It runs the actual database upgrade and the post upgrade steps. But it does neither create a guaranteed restore point nor does it drain any load.
java -jar autoupgrade.jar -config myconfig.cfg -mode upgrade
In this mode, no postupgrade actions will be performed automatically.
Deploy Mode
The deploy mode is the most important – and the comprehensive mode as it does everything in one pass, analyze, fixups and upgrade including the postupgrade actions.
In addition to calling the modes directly, it creates a guaranteed restore point where possible and drains the load.
java -jar autoupgrade.jar -config myconfig.cfg -mode deploy
It writes all the necessary logs. And it upgrades your entire database from start till end with just a single call. Unattended.
Matrix and Overview
This graph shows you the operations done in each mode when you call the autoupgrade.jar:
AutoUpgrade Matrix: Which operations happen in which mode?
Best Practice and Recommendation
We recommend to always run the analyze mode at first. If it is successful, the AutoUpgrade can fix all things it found. Then you call autoupgrade.jar again in deploy mode with a start_time set to the desired time. And it will upgrade your database. The deploy will repeat the analyze part.
For instance, you call at 15:00h:
java -jar autoupgrade.jar -config myconfig.cfg -mode analyze
And then, with a start_time set in your config file at 23:00h the same day:
java -jar autoupgrade.jar -config myconfig.cfg -mode deploy
Once the analyze does not show any errors, all is fine for a unattended upgrade with deploy.
–Mike
i have Job in DEPLOY which failed and I try to do an anlyze job now . But autoupgrade always recovers the failed job. How can I prevent this and start anew ?
Abort -job did not kill that ting for good.
Hi Norbert,
which version are you using?
Removing the log file tree should prevent it from restarting it again.
Cheers,
Mike
And Frederick told me the other (more correct) way:
java -jar autoupgrade.jar -config config.cfg -mode analyze -clear_recovery_data
Cheers,
Mike
Hi Mike,
we have a real bad problem with the autoupgrade on Windows System when i tested.
I did an autoupgrade to 19c on an test database in version 12.2.0.1. Fresh set up, plain database, no features, no data, running on a VM where parallel also our OEM (18.5) and RMAN(18.5) Catalog Test Instances resides.
First of all (not sure if this is documented) i got some problems when i “deploy”ed because my flashback parameters had to be set. Without them the Deploy Phase will crash. The analyze was fine without warnings. This gets not checked by analyze.
After i cleared that, i did a new deploy which failed with following message (autoupgrade_user.log):
2019-08-22 12:03:33.518 WARNING 1 fixups ran with errors, review the logs for further input, summary report available at D:\Oracle\cfgtoollogs\upgrade\DBA1P\DBA1P\101\prefixups\failed_prefixups.log – DBUpgradeInspector.reportFixupsWithErrors
The problem here, the log file is missing. it seemed that he could not create that file and because of that he failed. (autoupgrade.log)
2019-08-22 12:03:33.519 ERROR Execution of the fixups resulted in an unexpected error. – FixesToolBox$1.run
java.lang.IllegalArgumentException: D:\Oracle\cfgtoollogs\upgrade\DBA1P\DBA1P\101\prefixups\failed_prefixups.log does not exists
At this time i think that the real mistake has taken place already.
Because, after that i wanted to restore back the upgrade but this failed because the restore process could not mount the database and failed at step 4/6.
The real problem i then found (because my other databases OEM and RMAN were not accessible anymore at this time) was:
That all permissions and ownership under D:\Oracle (my Oracle Base) had changed.
Our local User running the service was owner of that directory (and all sub structur) but got no permission on anything.
Otherwise Administrators Group, SYSTEM and the ora-dba Group got Special Permission (so i could read/write) but no Full Permission to re-grant any permission.
After knowing that this was the problem, our system administrators changed the ownership back to Administrator and granted full permission to our Oracle service user so that we could start our services. So now my OEM and RMAN are running fine again.
But i am not sure if i want to execute the autoupgrade again.
Do you know the code, so that you can check if there is a function which changes Permissions massively in the Oracle Base?
That Problem is really bad and i think we will do our upgrades manually to be safe, because this situation is not usable with our producion environments.
If you could check and give me response, that would be great.
Thanks Marcus
Hi Marcus,
sorry to hear about this.
But please open an SR – I’m currently not available.
Thanks,
Mike
Hi Mike,
for your information!
after a lot of checks and upgrades, where the problem did not occur, the problem occured again and this time I could find the problem.
To make it short, there is already a MOS Note: DBUA upgrade to 19.4 in Windows fails with Error:ORA-1722 (Doc ID 2579192.1)
I didn’t find the error in the logfiles last time because it got truncated (please dont ask me why) in the first time the error happened.
This time i could see the error message:
—————————————————————————————
10:53:44 5 ‘&C_ORACLE_HIGH_MAJ..&C_ORACLE_HIGH_RU..&C_ORACLE_HIGH_RUR’;
old 2: ‘MUST_BE_&C_ORACLE_HIGH_MAJ..&C_ORACLE_HIGH_RU..&C_ORACLE_HIGH_RUR’)
new 2: ‘MUST_BE_19.3.0’)
old 5: ‘&C_ORACLE_HIGH_MAJ..&C_ORACLE_HIGH_RU..&C_ORACLE_HIGH_RUR’
new 5: ‘19.3.0’
‘MUST_BE_19.3.0’)
*
ERROR at line 2:
ORA-01722: invalid number
—————————————————————————————–
The Problem is, that when you have Windows and Oracle 19 and the Patch 19.4 OR 19.5, then this error occurs.
It is realy disappointing that Oracle knows the error since 22.August and after 5 months the bug is not fixed.
Also it is not fixed in the October Patch!
You have to edit source files to upgrade your database to this version … are you (i mean Oracle, not you Mike) serious?
Result: Autougrade is working realy good in our tests and the only problem we got, is, when we are upgrading to a “patched 19 DB”, and that is not your fault!
When Oracle Support is working as good as your team … this would be heaven on earth!
Best regards,
Marcus
Hi Marcus,
I see what happens. You are right, this is bug 30155576.
The problem is that dbms_registry_basic.sql is not included into the Windows Bundle Patches 19.4.0 – and unfortunately not in 19.5.0 either. I can’t even see that this is fixed with the Jan 2020 BP. Hence, I suspect that this error will persist.
It is unfortunately out of my control. And it frustrates me as much as you as I’d assume this is a fix of 5 minutes just looking at the bug content which you can’t see.
I’m dropping the bug owner a quick message, although, no guarantee that this will trigger anything forward.
Sorry for the inconvenience – and thanks for the update!
Cheers,
Mike
Marcus,
I just received a message from the developer. He confirmed that the file will be included in the Jan 2020 BP.
Sorry again for the inconvenience!
Mike
Hello Mike,
I am currently testing the AutoUpgrade utility with a test database having a configuration which should throw some errors during the “ANALYZE” phase. One of them is a shared pool size of 200 MB which is actually too small for Oracle 19c. I executed the “ANALYZE” phase but the fact that the shared pool size is too small was unfortunately not mentioned as an issue.
When all other errors reported by the “ANALYZE” phase were resolved by executing the “FIXUPS” phase I tried to start the upgrade with “DEPLOY”, as you mentioned that at this point there won’t be any unforeseen surprises anymore during a subsequent unattended upgrade, but as expected I got an issue as the database could not be started in “UPGRADE” mode due to the fact that the size of the shared pool is too small.
So it appears that, although you had no errors reported anymore with “ANALYZE”, you still might be in for a surprise once you start the upgrade with “DEPLOY”.
For this issue I created a SR and got the following feedback :
**********************************************************************************************
Memory related sizing is not taken care by preupgrade/pre-check this is expected behavior.
**********************************************************************************************
The check for the size of the shared pool is indeed not present in Doc ID 2380601.1 – Database Preupgrade tool check list.
Is there any chance that the checking of the memory related sizings gets added in the “ANALYZE” phase ?
FYI, Here are the errors which were present in the generated logfile (logfile autoupgrade_20190829_user.log) after having executed AutoUpgrade in “DEPLOY” mode :
2019-08-29 14:56:36.917 ERROR
DATABASE NAME: upgr12t
CAUSE: ERROR at Line 4 in [Buffer]
REASON: ORA-00371: not enough shared pool memory, should be at least 255826304 bytes
ACTION: [MANUAL]
DETAILS: 00371, 00000, “not enough shared pool memory, should be at least %s bytes”
// *Cause: The SHARED_POOL_SIZE initialization parameter was too small.
// *Action: Increase the parameter value
Hi Chris,
please open an SR and upload the entire log directory.
Then send me the SR number and we’ll take care.
Cheers,
Mike
Hello Mike,
Thank you very much in advance!
I opened Service Request “3-20868565231 –
Value parameter shared_pool_size not checked during “Analyze” with AutoUpgrade” for this.
PS: This issue might be solved with the most recent version of “AutoUpgrade” which has been released on 03/09/2019 as I can find the following statement in the “List of bugs fixed on build version 20190823 (19.6)” :
AUPG-806 – shared_pool_size not flagged during analyze phase
I will test to see whether this issue still occurs with this most recent version of AutoUpgrade.
Greetings,
Chris Smids
Thanks Chris!
Let me know if the newest version fixed it.
CHeers,
Mike
Hello Mike,
The latest version of the AutoUpgrade tool (AutoUpgrade 19.6 build.version 20190823) has indeed fixed this issue as the “ANALYZE” phase now correctly mentions that the size of the shared pool is too small for my upgrade.
Thanks to the update last week on your blog, about the announcement that a new version of AutoUpgrade was released, I was able to identify that this issue might be fixed with this release as it was mentioned in the “List of bugs fixed on build version 20190823 (19.6)”.
This thus clearly indicates that one should always have a look whether a more recent version of AutoUpgrade is available for download but thanks to your blog we are always aware of this ;-).
Greetings,
Chris
Thanks a lot, Chris – glad that it works now!!!
And thanks for letting us know!
Cheers,
Mike
When starting deploy, the feeling of a process looping
2019-09-18 10: 49: 58.768 INFO Executing SQL [SELECT SPACE_LIMIT || ‘#’ || SPACE_USED FROM V $ RECOVERY_FILE_DEST;] in [XXXXXX, container: null] – ExecuteSql $ SQLClient.run
2019-09-18 10: 50: 59.220 INFO Reviewing job 101 [XXXXXX] – MonAssistant.run
2019-09-18 10: 50: 59.233 INFO Reviewing job 101 [XXXXXX], Current time: [2019-09-18TT: 50: 59.233] Latest job update: [2019-09-18TT: 47: 20.885]. 3 minutes have passed. – MonAssistant.checkForHungJob
2019-09-18 10: 50: 59.234 INFO Begin [/ app / oracle / product / db / 12102 / bin / sqlplus, -silent, /, as, sysdba] – ExecuteProcess.startSqlPlusProcess
2019-09-18 10: 50: 59.234 INFO Begin Setting Oracle Environment – ExecuteProcess.startSqlPlusProcess
2019-09-18 10: 50: 59.234 INFO [(SQLPATH = / tmp / upg_logs / XXXXXX / XXXXXX), (ORACLE_SID = XXXXXX), (ORACLE_UNQNAME = XXXXXX), (ORACLE_PATH = / tmp / upg_logs / XXXXXX / egaism) (ORACLE_BASE = / app / oracle), (TWO_TAS
2019-09-18 10: 50: 59.234 INFO Starting – ExecuteProcess.setLibsForSqlplus
2019-09-18 10: 50: 59.234 INFO Finished – ExecuteProcess.setLibsForSqlplus
2019-09-18 10: 50: 59.234 INFO End Setting Oracle Environment – ExecuteProcess.startSqlPlusProcess
2019-09-18 10: 50: 59.234 INFO Begin Creating Process – ExecuteProcess.startSqlPlusProcess
2019-09-18 10: 50: 59.235 INFO End Creating Process – ExecuteProcess.startSqlPlusProcess
2019-09-18 10: 50: 59.235 INFO Executing SQL [SELECT SPACE_LIMIT || ‘#’ || SPACE_USED FROM V $ RECOVERY_FILE_DEST;] in [XXXXXX, container: null] – ExecuteSql $ SQLClient.run
2019-09-18 10: 51: 59.785 INFO Reviewing job 101 [XXXXXX] – MonAssistant.run
2019-09-18 10: 51: 59.785 INFO Reviewing job 101 [XXXXXX], Current time: [2019-09-18TT: 51: 59.785] Latest job update: [2019-09-18TT: 47: 20.885]. 4 minutes have passed. – MonAssistant.checkForHungJob
2019-09-18 10: 51: 59.786 INFO Begin [/ app / oracle / product / db / 12102 / bin / sqlplus, -silent, /, as, sysdba] – ExecuteProcess.startSqlPlusProcess
2019-09-18 10: 51: 59.786 INFO Begin Setting Oracle Environment – ExecuteProcess.startSqlPlusProcess
2019-09-18 10: 51: 59.786 INFO [(SQLPATH = / tmp / upg_logs / XXXXXX / XXXXXX), (ORACLE_SID = XXXXXX), (ORACLE_UNQNAME = XXXXXX), (ORACLE_PATH = / tmp / upg_logs / XXXXXX / egaism) (ORACLE_BASE = / app / oracle), (TWO_TAS
2019-09-18 10: 51: 59.786 INFO Starting – ExecuteProcess.setLibsForSqlplus
2019-09-18 10: 51: 59.786 INFO Finished – ExecuteProcess.setLibsForSqlplus
2019-09-18 10: 51: 59.787 INFO End Setting Oracle Environment – ExecuteProcess.startSqlPlusProcess
2019-09-18 10: 51: 59.787 INFO Begin Creating Process – ExecuteProcess.startSqlPlusProcess
2019-09-18 10: 51: 59.788 INFO End Creating Process – ExecuteProcess.startSqlPlusProcess
2019-09-18 10: 51: 59.788 INFO Executing SQL [SELECT SPACE_LIMIT || ‘#’ || SPACE_USED FROM V $ RECOVERY_FILE_DEST;] in [XXXXXX, container: null] – ExecuteSql $ SQLClient.run
Alexey,
are you using the MOST recent tool (September version)?
If not, please try it again.
If yes, please open an SR, upload the entire log directory and share the SR number with me.
Cheers,
Mike
Hi Mike, hi Alexey,
I have seen it twice too on the following version:
build.hash 255dd7d
build.version 19.9.0
build.date 2020/04/23 15:01:36
build.max_target_version 19
build.supported_target_versions 12.2,18,19
build.type production
Has SR for that created? Have you found a root cause? It seems it occurs when you have MIN_RECOVERY_AREA_SIZE precheck reported. First attempt fails with:
Stage [PREFIXUPS]
Operation [STOPPED]
Status [ERROR]
Info [
Error: UPG-1312
[Unexpected exception error]
Cause: A failed check has an ERROR severity but the fixup is unavailable or failed to correct the problem. […]
The second run, even if you resize FRA manually, jumps into the loop. The workaround is quite easy – just fix FRA size between analyze and deploy runs (or just restart autoupgrade from scratch). I think I can easily replicate it, but I also have the logs to raise SR.
You see in the autoupgrade.log every minute:
2020-05-28 13:36:22.065 INFO Starting – ExecuteProcess.setLibsForSqlplus
2020-05-28 13:36:22.065 INFO Finished – ExecuteProcess.setLibsForSqlplus
2020-05-28 13:36:22.065 INFO End Setting Oracle Environment – ExecuteProcess.startSqlPlusProcess
2020-05-28 13:36:22.065 INFO Begin Creating process – ExecuteProcess.startSqlPlusProcess
2020-05-28 13:36:22.066 INFO End Creating process – ExecuteProcess.startSqlPlusProcess
2020-05-28 13:36:22.066 INFO Executing SQL [SELECT SPACE_LIMIT||’#’||SPACE_USED FROM SYS.V$RECOVERY_FILE_DEST;] in [SOMEDB, container:null] – ExecuteSql$SQLClient.run
2020-05-28 13:37:22.152 INFO Reviewing job 103[SOMEDB] – MonAssistant.run
2020-05-28 13:37:22.153 INFO Reviewing job 103[SOMEDB], Current time: [2020-05-28T13:37:22.153] Latest job update: [2020-05-28T13:32:15.523]. 5 minutes have passed. – MonAssistant.checkForHungJob
but sqlplus process is not running at all (there is no sqlplus process on a machine).
Best Regards,
Łukasz
Hi Mike,
I am able to run the analyze successfully.
But when changing to the deploy option, I get the message:
Invalid value for target_base: []
The deploy fails to execute…
Also I don’t see any lines about target_base in your examples.
Kind regards, Rene Einholz
Hi Rene,
can you please open an SR, upload the entire log directory – and send me the SR number please (via the blog is fine, too).
Cheers,
Mike
Hi Mike,
Thanks for the great articles!
I’m currently facing exactly the same problem and get the message : “Invalid value for target_base: []”.
Would you have information regarding the root cause of this issue?
Thanks for your help!
Hi Jack,
I can’t tell you as I didn’t receive the SR number.
Please open an SR and check with Support.
Cheers,
Mike
Hi, Mike!
Very cool utility, thanks!
I have few suggestions:
1)It can be better if Oracle makes an improvement – add autorefresh for status commands
I use the latest version 201912 but it does not update the status in automatic mode neither in -console nor in -noconsole mode. I would like to see something like sar 30 500 with an update interval of 30 seconds. I also noticed that the status.html file contains a function for automatic updating of the file, but as I run the utility on the database server remotely via ssh and download the file via sftp, this does not work 🙁
2) Is it possible to add information about current log files to -noconsole mode, similar to opatch or dbupgrade utilities. This information is present in -console mode, but i dislike it, in the current form of this mode.
Hi Sergey,
in the cfgtoollogs directory you’ll find a status file – this should be updated frequently. And as far as I know, you can tweak the settings how often it should be updated. You may please drop me an email with your suggestions – then I can share this with the team for a further discussion.
Thanks,
Mike
HI Mike,
can we login to the interactive upgrade tool prompt upg> seperatly as my session has closed and i just need to re initiate the job how can this be done?
or will it call same job again when i do a deploy command again.
@Alexey, I got same issue of looping (MonAssistant.checkForHungJob), even when I am doing the analyze. I made sure there is no other database sessions or locks. Did you resolve your issue?
No where in the doc you specified how to create myconfig.cfg. Without this this upgrade will not start at all.
Sample config file –
# build version 19.7.5
# build date 2020/02/11 15:28:49
#
#
# Global configurations
#
# This directory will include the following
# (1) AutoUpgrade’s global directory
# (2) Any logs, not directly tied to a job
# (3) Config files
# (4) progress.json and status.json
global.autoupg_log_dir=/home/oracle/upg_logs
#
# Database number 1
#
upg1.dbname=employee
upg1.start_time=NOW
upg1.source_home=/u01/app/oracle/product/11.2.0/dbhome_1
upg1.target_home=/u01/app/oracle/product/19.1.0/dbhome_1
upg1.sid=emp
upg1.log_dir=/home/oracle/upg_logs/employee
upg1.upgrade_node=servr1
upg1.target_version=19
#upg1.run_utlrp=yes
#upg1.timezone_upg=yes
Hi Rohit,
not really sure what you are complaining about.
See here:
https://mikedietrichde.com/2019/06/13/create-and-adjust-the-config-file-for-autoupgrade-19c/
and here:
https://docs.oracle.com/en/database/oracle/oracle-database/19/upgrd/autoupgrade-command-line-parameters.html#GUID-7DE1D595-26DC-49EA-B08E-A3A3A97359D6
Cheers,
Mike
analyze is clean n so as fixups but deploy failed with generic error –
It was not possible to find or validate the ORACLE_BASE for the source home of entry
## Doc ID 2573304.1 ##
CAUSE
Wrong entry in $ORACLE_HOME/install/orabasetab
SOLUTION
Modify $ORACLE_HOME/install/orabasetab to reflect the correct entries as given below :
ORACLE_HOME:ORACLE_BASE:OraInventory_OH_NAME:N:
I wish the documentation could cover this, when Oracle is advertising this autoupgrade utility then all such loop holes, knows errors must be listed .
Hi Rohit,
not sure how to react here. Documentation NEVER covers all the issues as far as I’m aware of. You can’t deploy documentation every night.
But we do our best to fix things as quickly as possible, deliver new versions with bug fixes faster than anybody else in Oracle and document as much as possible.
Nobody is perfect – but thanks for your input!
Cheers,
Mike
build.hash 255dd7d
build.version 19.9.0
build.date 2020/04/23 15:01:36
build.max_target_version 19
build.supported_target_versions 12.2,18,19
build.type production
cfg file file:
global.autoupg_log_dir=/oracle/
Database number 1
upg1.dbname=CORS_D
upg1.start_time=NOW
upg1.source_home=/oracle/product/12.2.0.1.200114p
upg1.target_home=/oracle/product/19.7.0.0
upg1.sid=CORS_D
upg1.log_dir=/oracle/admin/CORS_D/upgrade
upg1.upgrade_node=misc-dev-db1.internal.admin.cam.ac.uk
upg1.target_version=19
upg1.run_utlrp=yes
upg1.timezone_upg=yes
=============
autoupgrade errors:
1. the autoupgrade.jar changes the listener.ora entries of other databases which we don’t want to upgrade.
2. the scripts changes the permissions of of the global auto log directory /oracle
Hi Ansar,
please always with an SR number – I can’t replace Oracle Support via the blog. As much as we all would like to help, Oracle Support is the way to go.
Thanks,
Mike
Hi,
I just have a simple question regarding the use of AUTOUPGRADE.
Do the databases we are upgrading have to be started ? Just started in upgrade mode or … ? We are using INITsid.ORA files (not spfiles, because they are running on a windows cluster and this is a mandatory condition) and I was also wondering wether the post parameters that could be used in inisid.ora would be added to the current version of the initsid.ora file used to startup the database ? And when the process shutdown and startup the database again once migrated, does it use the good initsid.ora ? I have seen that this file was mentionned on the config.txt file used by autoupgrade unless I have eyes troubles 🙂
Thanks alot for your reply
Hi Joel,
the database needs to be up as otherwise AutoUpgrade can’t run an analyze on it.
With the init.ora, I’m not sure about whether we expect an SPFILE. I’d blindly guess so as this is the standard.
Thanks,
Mike
Hello:
I’m trying to migrate a test database – STAGING – from 11.2.0.4 to 19.7 (installed 19.3 software and patched later).
Review and prepared config file staging.cfg
#
# Database number 1
#
upg1.dbname=STAGING
upg1.start_time=NOW
upg1.source_home=/u01/app/11.2.0.4/oracle/database/
upg1.target_home=/u01/app/19.3/database/
upg1.sid=STAGING1
upg1.log_dir=/temp_ora1/LogsMigracion19/Staging_20200707/Staging.log
upg1.upgrade_node=flora1
upg1.target_version=19
and try to run analyze mode:
/u01/app/19.3/database/jdk/bin/java -jar /u01/app/19.3/database/rdbms/admin/autoupgrade.jar -config /u01/sw/staging.cfg -mode analyze
but two hours latter I see no progress:
upg> lsj
The jobs are still being prepared, try again in a few seconds
I wonder if I missed something. For example: Is mandatory to have in use a RECOVERY_AREA_USAGE ? That is the only difference I found between examples and my test.
Regards, Lorenzo
Hi Lorenzo,
it should not hang – can you check the logs please?
And use upg1.restoration=no if you don’t have your database in archivelog mode or want to avoid a GRP.
But check the logs at first.
Cheers,
Mike
Hi Mike,
I tried to perform analyze in one of my test database and it failed. I am new to this autoupgrade tool so instead of aborting it, I pressed a ctrl+z and then tried to anaylze. Now when I try to run the analyze again its giving me an error as below:
Another AutoUpgrade instance is already running using the same global.autoupg_log_dir=/u01/export/upgrade/orcl/upg_logs/cfgtoollogs/upgrade/auto value. Exiting
I am not able to run analyze again. I am not sure how can I get this analyze running again.
Thank you in advance.
Regards,
Prakash Thakur
Stop AU and remove the log tree.
Cheers,
Mike
Hi Alexey and Lukasz.
Hope you have solved the problem, but just for others who may face the same issue will try to show the mistake that I have made and how it solved the same issue.
I have run into the same loop of waiting for SELECT SPACE_LIMIT||’#’||SPACE_USED FROM SYS.V$RECOVERY_FILE_DEST; minute by minute passing and no resolution to that. Then realized that I have only extracted the ZIP file of 19c to Oracle home without running the ./runInstaller
After installing the software the problem resolved and we could upgrade multiple databases sharing the same home simultaneously.
Hope this will be helpful to others.
Thank you.
Regards,
Hekuran.
Hi Mike,
Receiving the following when trying to run the upgrade in deploy mode after running in analyze mode on my windows 2016 server. Upgrading from 18C to 19C.
2021-03-04 14:16:32.365 INFO Adding fixup PURGE_RECYCLEBIN PRECHECKS YES true ERROR to execution queue of xxxxxxxx
2021-03-04 14:16:32.912 INFO Begin Compiling Invalid Objects on Database [xxxxxxxx]
2021-03-04 14:16:39.990 INFO End Compiling Invalid Objects on Database [xxxxxxxx]
2021-03-04 14:16:39.990 INFO SUCCESSFULLY COMPILED [xxxxxxxx]
2021-03-04 14:16:39.990 INFO Deleting parameter *.local_listener=” in E:\App\orahome2\product\19.3.0\dbhome_1\rdbms\admin\autoupgrade\xxxxxxxx\xxxxxxxx\temp\during_upgrade_pfile_xxxxxxxx.ora
2021-03-04 14:16:40.005 INFO Adding parameter cluster_database=’FALSE’ to E:\App\orahome2\product\19.3.0\dbhome_1\rdbms\admin\autoupgrade\xxxxxxxx\xxxxxxxx\temp\during_upgrade_pfile_xxxxxxxx.ora
2021-03-04 14:16:40.021 INFO Deleting parameter *.core_dump_dest=’E:\App\orahome\product\18.3.0\ADMIN\xxxxxxxx\CDUMP’ in E:\App\orahome2\product\19.3.0\dbhome_1\rdbms\admin\autoupgrade\xxxxxxxx\xxxxxxxx\temp\during_upgrade_pfile_xxxxxxxx.ora
2021-03-04 14:16:40.021 INFO Deleting parameter *.core_dump_dest=’E:\App\orahome\product\18.3.0\ADMIN\xxxxxxxx\CDUMP’ in E:\App\orahome2\product\19.3.0\dbhome_1\rdbms\admin\autoupgrade\xxxxxxxx\xxxxxxxx\temp\after_upgrade_pfile_xxxxxxxx.ora
2021-03-04 14:16:50.365 INFO Analyzing xxxxxxxx, 85 checks will run using 8 threads
2021-03-04 14:16:55.428 INFO Generating Context text file list E:\App\orahome2\product\19.3.0\dbhome_1\rdbms\admin\autoupgrade\xxxxxxxx\xxxxxxxx\temp\ctx_move_text_file_list_xxxxxxxx
2021-03-04 14:16:55.803 INFO Defer redo log shipping to standby database
2021-03-04 14:18:10.951 INFO Copying password file from E:\App\orahome\product\18.3.0\dbhome_1\database\PWDxxxxxxxx.ora to E:\App\orahome2\product\19.3.0\dbhome_1\database\PWDxxxxxxxx.ora
2021-03-04 14:18:10.951 INFO Copying password file completed with success
2021-03-04 14:18:22.607 ERROR
DATABASE NAME: xxxxxxxx
CAUSE: ERROR at Line 2 in [Buffer]
REASON: ORA-01017: invalid username/password; logon denied
ACTION: [MANUAL]
DETAILS:
2021-03-04 14:18:22.607 ERROR Database Open Failed for xxxxxxxx ERROR:
ORA-01017: invalid username/password; logon denied
2021-03-04 14:18:22.607 ERROR Opening Database xxxxxxxx in upgrade mode failed
2021-03-04 14:18:22.607 ERROR Exception Error in Database Upgrade [xxxxxxxx]
2021-03-04 14:18:22.607 ERROR xxxxxxxx Return status is ERROR
2021-03-04 14:18:22.623 ERROR Dispatcher failed: AutoUpgException [UPG-1401#Opening Database xxxxxxxx in upgrade mode failed]
2021-03-04 14:18:22.623 INFO Starting error management routine
2021-03-04 14:18:22.623 INFO Ended error management routine
2021-03-04 14:18:22.623 ERROR Error running dispatcher for job 104
Cause: Opening database for upgrade in the target home failed
2021-03-04 14:18:22.623 ERROR Dispatcher failed:
Error: UPG-1401
Opening Database xxxxxxxx in upgrade mode failed
Cause: Opening database for upgrade in the target home failed
For further details, see the log file located at E:\App\orahome2\product\19.3.0\dbhome_1\rdbms\admin\autoupgrade\xxxxxxxx\xxxxxxxx\104\autoupgrade_20210303_user.log; aborting job 104 for database xxxxxxxx
Thanks!
Don
Hi Don,
please open an SR and upload all log (java -jar autoupgrade.jar -config yourconfig.cfg -zip) to the SR. Add alert.log and “opatch lsinventory”.
I can’t diagnose AU issues on the blog here unfortunately.
Cheers,
Mike
MIke,
I opened (SR 3-25362595811) on 3/4/2021 and have not yet received a response. I will upload the requested files.
Thanks! Don
Thanks Don!
If you don’t receive a response in a reasonable amount of time, raise the severity, maybe to 2 or 1 (but non 24×7) and make several updates into the SR.
At worst, escalate the SR.
Cheers,
Mike
Mike,
I finally received a response yesterday 3/9, and the person told me that Auto Upgrade is not the recommended approach to upgrade to 19C. He suggested DBUA as the first alternative. Also, it does not appear they have a documented workaround for dealing with the issue that I am encountering . I mentioned to him that I watched your 2 hour presentation on auto upgrade and I believe you clearly stated that auto upgrade is now Oracle’s the recommended approach to upgrade to 19C. So, as I mentioned in my response to him, not everyone appears to be on the same page regarding the recommended upgrade approach. Perhaps it is because I am on a Window platform?
I tried a couple of things but with no success. Perhaps you can take a look at the SR. Just trying to get my database upgraded to 19c the most efficient way.
Thanks for your follow-up!
Don
Hi Don,
we have a conversation internally 🙂
AU is the only recommended approach to upgrade databases – and this doesn’t seem to be a good statement. I read it as well.
I’ll check the SR.
And the ORA-12638 is coming from this change:
https://mikedietrichde.com/2021/03/10/ora-12638-on-windows-only-from-oracle-19-10-0-onwards/
Cheers,
Mike
Hi Mike,
Thanks for sharing all these articles on autoupgrade.
When upgrading a non-cdb 12cR2 DB to 19c PDB, due to network issue my session to the DB server disconnected. Earlier, I used to re-run autoupgrade in DEPLOY mode and it was able recover the upgrade job, but this time I got the error:
oracle> java -jar $ORACLE_HOME/rdbms/admin/autoupgrade.jar -config 12cr2upgrade.cfg -mode DEPLOY
Another AutoUpgrade instance is already running using the same global.autoupg_log_dir=/home/oracle/autoupgrade_global/cfgtoollogs/upgrade/auto value. Exiting
oracle>
Whereas from the logfiles, I can see that the upgrade is still in progress, in such cases how can I connect to the autoupgrade prompt so that I can monitor my upgrade using lsj, status & task commands.
oracle> cd upgrade11gR2/ORCL11/103/
oracle> ls -ltr
total 468
-rwx—— 1 oracle oinstall 0 Apr 13 17:30 autoupgrade_20210413.log.lck
-rwx—— 1 oracle oinstall 0 Apr 13 17:30 autoupgrade_err.log.lck
-rwx—— 1 oracle oinstall 0 Apr 13 17:30 autoupgrade_20210413_user.log.lck
-rwx—— 1 oracle oinstall 0 Apr 13 17:30 autoupgrade_err.log
drwx—— 2 oracle oinstall 4096 Apr 13 17:30 preupgrade
drwx—— 2 oracle oinstall 4096 Apr 13 17:35 prechecks
drwx—— 2 oracle oinstall 4096 Apr 13 17:35 prefixups
drwx—— 2 oracle oinstall 4096 Apr 13 17:39 drain
drwx—— 2 oracle oinstall 4096 Apr 13 18:21 dbupgrade
-rwx—— 1 oracle oinstall 7924 Apr 13 18:37 autoupgrade_20210413_user.log
-rwx—— 1 oracle oinstall 446987 Apr 13 18:37 autoupgrade_20210413.log
oracle>
oracle> tail -f autoupgrade_20210413_user.log
|CONTAINER| PERCENTAGE|
+———+————-+
| ORCL11|UPGRADE [91%]|
+———+————-+
2021-04-13 18:37:50.065 INFO [Upgrading] is [91%] completed for [orcl11]
+———+————-+
|CONTAINER| PERCENTAGE|
+———+————-+
| ORCL11|UPGRADE [91%]|
+———+————-+
2021-04-13 18:40:50.403 INFO [Upgrading] is [91%] completed for [orcl11]
+———+————-+
|CONTAINER| PERCENTAGE|
+———+————-+
| ORCL11|UPGRADE [91%]|
+———+————-+
Hi Nitish,
this is a very rare case. Normally, calling AU with the exact same command should bring you back into console mode.
If this is not the case, you either monitor it via the logs or via Python:
https://mikedietrichde.com/2020/07/16/autoupgrade-refresh-status-information-automatically/
or you access the status.log directly and refresh it. AU does write periodic updates into the file.
Or, this would be the hard way:
You’d kill the upgrade process, and restart then AU with -deploy.
But my fear would be that you may catch it during plugin which is irreversible – so you need to check with the logs beforehand please.
Cheers,
Mike
We are upgrading a Oracle 12 cdb, with 1 pluggable database, instance was started in upgrade mode, then ALTER PLUGGABLE DATABASE ALL OPEN UPGRADE;
When running autoupgrade in deploy mode, job does not start with (pluggable db name) currently has a status of OPEN_MIGRATE
Hi Matt,
when you start PDBs with ALTER PLUGGABLE DATABASE OPEN UPGRADE, and then you check the PDBs, they have OPEN_MIGRATE status. This is correct and expected.
Are you saying that AutoUpgrade does not upgrade your PDBs?
Thanks,
Mike
Hello mike, if -mode upgrade will not perform postupgrade automatically from another server, then how can I do it manually? Or what should I do to make postupgrade automatically? Should I use -mode deploy after the -mode upgrade are finished in the target server?
Hi Jerome,
see here please:
https://mikedietrichde.com/2020/08/03/oracle-autoupgrade-between-two-servers/
Cheers,
Mike