Sorry for the headline using TLAs (three letter acronyms). But I was tempted – and those of you being interested in Oracle Database patching my quickly realize that there are some Patching News: RURs are gone – long live MRPs.
RURs are gone?
Andy Mendelsohn announced the MRPs (Monthly Recommended Patches) in October at DOAG Conference in Nuernberg already. Due to Oracle Cloud World bussiness, I’ve had no time to write about it yet. And the introduction of the MRPs means that we won’t produce RURs anymore after a transition grace period.
Oracle Database Development EVP Andy Mendelsohn introducing MRPs (monthly recommended patches) instead of RURs at @DOAGeV in his keynote – will take off in October 2022 for Oracle 19 pic.twitter.com/PzFzGqgTBi
— Mike Dietrich (@MikeDietrichDE) September 21, 2022
RURs stood for Release Update Revision. The initial idea behind RURs was to deliver mostly security but also very important fixes on top of a previous Release Update (RU). For instance, if you took 19.14.0 from January 2022, then RUR 19.14.2 in July 2022 should deliver the security fixes equal to the RU 19.16.0 released at the same time. And in addition, it should correct some flaws in 19.14.0, 19.14.1 and 19.15.0. That was the idea.
As you can see in the graph above, RURs won’t be terminated right away. But read further please.
Why do RURs need to go?
Unfortunately, the RUR concept didn’t work so well since the time an issue got logged with Oracle Support, then handled and fixed in development was often too short to get the fix into this particular RUR. Code freeze dates and other reasons led to this situation. In addition, we produce proactively a high number of one-off fixes not included in an RU yet. But only on top of RUs, not on top of RURs.
So whenever a customer requested a one-off on top of an RUR (we call this a backport), that one-off usually had to be produced at first since it wasn’t available yet.
Finally, the reason why I never recommended RURs was mainly due to the fact that having a Support background made me feel very bad when you, as a customer, would open an SR, we find out that this issue is fixed already in, let’s say 19.16.0, but unfortunately you decided to be on RUR 19.14.2. Now you missed 997 (!!) fixes which are available already. That is the diff between 19.14.2 and 19.16.0 even though they both got released at the same time within the same cycle.
To me this was something I could not promote. And I clearly know that some very experienced people out there have different opinions on this. But overall, the download numbers of RURs were very low compared to RUs for a reason.
Therefore, we clearly recommended to apply RUs on a regular basis.
Please read more about this topic in MOS Note: 2898381.1 – Sunsetting of 19c RURs and FAQ.
Long live MRPs
Now with the October 2022 cycle there are only the final RURs released but instead from November 2022, one month after the release of the October RU, you will get access to the first MRP, the Monthly Recommended Patch. This patch bundle will contain only a small number of fixes. And it will be refreshed every month for 6 months for any given RU. So the relation to Release Updates (RUs) looks like this:
Please note two things:
- The change at first for the year on the x-axis, and the indicator for RUs on the y-axis which is different to the previous graphs.
- The lines for RUs are independent from each other. So even though MRP3 for RU 19.17.0 is supposed to be available at about the same time as RU 19.18.0 will be available, the RU 19.18.0 will have all fixes from 19.17.0 plus many news ones (we say “RUs are always cumulative) but the MRP3 on top of 19.17.0 may have potentially only 5-10 fixes. And those 5-10 fixes will be hopefully but not guaranteed in 19.18.0
The idea behind MRPs is that we can quickly deliver the most important fixes shown in MOS Note:555.1 – Oracle Database 19c Important Recommended One-Off Patches. So let’s assume you’d go live with 19.17.0, the October 2022 RU – but not in October but over the holiday period. At the end of the calendar year, 19.18.0 is not available yet. But at this time there is the 2nd MRP already there. With this you can fix the most knwon issues.
Of course, MRPs are cumulative as well. So with the MRP4 you will get also the fixes delivered with the previous MRPs on top of a given Release Update.
But let me clarify something very important: We are not increasing the frequency of patch bundles, and neither are you forced to apply MRPs. This is for your convenience to avoid the need to request a solid number of important fixes. If you take the most recent MRP when you go live on day X, then you can be sure that we put the most important fixes on top of your chosen RU in it.
That’s it – nothing more and nothing less. It’s purely meant to ease your DBA life a bit.
As I mentioned before, to allow a smooth transition, the RURs will be faded out as you can see in the RUR graph shown earlier in this blog post. So those of you who still use RURs will still get them for a short transition period.
Please find more information about MRPs in this MOS Note: 2898740.1 – Introducing Monthly Recommended Patches (MRPs) and FAQ
Why aren’t MRPs a perfect solution?
See the MRP as a intermediate solution on our way to Gold Images where you provide a list of patch numbers, and you’ll get an image to download. I think we’ll go in this direction the sooner or later. But until this is available, MRPs can help many customers.
Of course, those of you with a longer list of one-off and merge patches to apply on top of an RU won’t benefit from this solution since the MRPs may miss some of your fixes. And those who always wanted security fixes only will need to wait for the next RU generally.
But the MRPs allow us to deliver important fixes quickly, and much quicker than before.
Please share your experiences once the MRPs are out.
How do you apply an MRP?
As other patches as well, you will use opatch to apply the fixes to your Oracle Home. As usual, we recommend that you provision a new home with the right RU, and put the MRP on top of it before you switch from old to new home.
Does the Release Number change with MRPs?
The release number does not change when you apply an MRP on top of an RU. So when you take 19.17.0, and you’ll apply the MRP4 on top of it, the release will still be 19.17.0 whereas RURs did change the release number as you can see above. You will need to run opatch lsinventory to check whether an MRP has been applied. But since the MRP does not adjust the release number, and since you could deinstall one or more fixes of an MRP, you may need to check for the individual bug numbers to proof that an MRP had been installed.
Do you need to retest the application with MRPs?
Of course, with every patch you should do a little bit of testing in order to be sure it can be applied safely. But I’ve already had a discussion with a customer at Cloudworld who said: “No, we need to do full testing.” And I disagree. You won’t do full testing for several one-off patches you are going to apply. MRPs are nothing different then some important one-off patches put together. It does not change the release numbering nor does it introduce any new features or functionality. So please do the same amount of testing as you do when you apply a one-off fix or a merge patch.
Do you need to apply the MRPs now monthly?
This is a question we have heard from several customers already (thanks Daniel for the pointer). And clearly, you don’t have to. Our recommendation would be: Whenever you roll out a new home, make sure the most recent MRP is applied on top of your RU. But we would apply a future MRP to this RU only if an issue is coming up which seems very likely to appear in your environment. From a real world experience, I think it is more likely that you won’t take a monthly patch generally but only if you hit an issue.
Still, we highly recommend that – before you deploy a new home – that you apply the most recent MRP on top of it.
Does this affect Windows and other platforms such as AIX, SPARC or HP Itanium?
No, MRPs right now will be delivered on Linux only. Hence, nothing will change for Windows, and you will still receive quarterly Bundle Patches (BPs) as you did before. And on other non-Liunx ports you simply will go forward with quarterly Release Updates (RUs) and need to request the fixes you’d like to apply on top of the RU in accordance to MOS Note:555.1.
Will MRPs contain security fixes?
MRPs may include critical 3rd party security fixes. The quarterly database release update (RU) will continue to be primary mechanism for delivery of database Security fixes announced as part of the quarterly critical patch update (CPU) program.
Do I need to rollback an MRP when I patch to the next RU?
This is a tricky question since you will find two possible situations. At first, let’s assume you are on 19.17.0 with MRP4. This MRP has maybe 8 fixes included. Now you want to switch to 19.18.0 with the already available MRP1. What do you need to do now?
In case the 8 fixes are all included in 19.18.0 already, then you can just go forward and apply MRP1 for 19.18.0 on top. But in the likely case that not all of the 8 fixes are included in 19.18.0 you will have to rollback MRP4 for 19.17.0, then move to 19.18.0, then apply MRP1 for 19.18.0.
I know, this sounds obscure at first sight. But when you consider that MRPs are just a bundle of several one-off patches, it makes more sense since you would treat one-offs the same way. Now, as many of you patch only every other cycle, for example from 19.16.0 to 19.18.0, it is way more likely that the 8 fixes from my above example will be included in the RU released 6 months later.
Are MRPs cumulative?
Yes, you’ll see that MRPs are cumulative but only for each RU. This means, when you take MRP4 for 19.17.0 it will contain all fixes from the previous MRPs done for 19.17.0. But when you take MRP1 built for 19.18.0 there is no connection/relation between the 19.17.0 MRPs and the 19.18.0 MRPs. There may be some of the fixes included in both but ideally the fixes in (at least an early) 19.17.0 MRP may be in the 19.18.0 RU already.
Further Links and Information
- MOS Note: 2898381.1 – Sunsetting of 19c RURs and FAQ
- MOS Note: 2898740.1 – Introducing Monthly Recommended Patches (MRPs) and FAQ
- MOS Note:555.1 – Oracle Database 19c Important Recommended One-Off Patches
I think better ways for Future Patching can be implemented (offline & online).
Hope less Changes Happens on Patching Logic in Future 🙂
Important is to keep it simple as possible and so flexibel as possible (online & offline).
Maybe some ExaCC Logic (&scripts/APIs) for Patching (but more Automated for OneOff or current unfixed Problems with ask option or configure inside XML or Yaml file for offline Patching – will give further improvement for Customer).
we are pushing to move more towards this direction. Stay tuned please.
Our vision is that we can make patching as simple as patching an app on your cellphone.
Sounds good, thank you Mike for your great work to keep customers always up2date.
(Maybe ansible,docker,kubernetes patterns will play a role in future and extend Oracle Multitenant Features).
Do you know if MRPs will be available on Exadata Ccloud@Customer / ExaCS?
I hope so but I don’t know yet. The MAA team is involved in the whole MRP planning process. So I’d guess the teams will pick this up the sooner or later. Not sure about ExaCS.
Thank You Mike,
When will Datapump Bundle Patch available for 18.104.22.168.221018 ?
As soon as it is available – I can’t tell you more than “it’s in the making” since I have no influence on the process.
When should we update DST files? Oracle keeps releasing them and we are not sure when to apply a new DST update. These do not get applied with any RU. We are on v32 but I see that DST version 39 is out there.
I will have at first some good news with 19.18.0 if all goes well. We finally will deliver all available DST files with future RUs. This was a long journey but keep your fingers crossed together with me.
And then, your initial question:
At first, you are not forced to apply them and adjust DST. There are situations when you should do this, especially when your database is using TIMESTAMP with TIME ZONE datatypes and calculations. But for instance, a Swiss insurance who deals only with Swiss customers does not have much use for adjusting the time zone to tell the database that Syria and Jordan now rapidly both decided to skip the DST change next weekend.
So it depends on your use case and on the data types you are using. You can leverage the ?/rdbms/admin/utltz_countstats.sql script to check.
In MOS Note 2898740.1 is states:
No additional RURs will be delivered on any platform after the delivery of Oracle Database 19c RUR 19.16.2 in January, 2023. All customers should instead simply use the latest,
recommended 19c Release Update, such as 19.17.0, 19.18.0, or 19.19.0, etc.
So being a Windows (second class db platform) we do not have RURs, going by the slides. So for those of on Windows our path our quarterly patching now goes to RU each month?
Many questions come to mind with this information. First being, is Oracle moving away from running on Windows platforms? Or perhaps Oracle is eventually only support cloud platforms?
What do you think?
no, we are not moving away from Windows. I did not say anything about Windows since you’ve had the quarterly Bundle Patches – and they will stay fully unchanged. Of course, there won’t be MRPs on Windows as (for now) they’ll be available on Linux only.
Let me clarify and add this to the blog post. It will get longer this way but I’m very thankful for your questions.
Thank you, Mike. Love seeing your posts and love the labs.
Another problem with RURs–and I don’t know if you’d count this as a separate problem or a consequence of the other problems you noted–is that EBS and possibly other Oracle application products didn’t support the use of RURs. With EBS at least you could only take up RUs.
Also, a side (and snide) comment about Oracle’s TLA department… this is the second time now that Oracle has chosen to reuse an existing TLA for a patching-related release which will make for fun times when trying to use your favorite search engine to get more information! (The first winner of this prize of course was “CPU”…)
thanks for the addition – and regarding our TLA department, I totally agree. Even more for CPU which got renamed to SPU, and then back to CPU. I think after more than 25 years I could fill at least a 90 min presentation about TLAs easily …
no need to post this comment! on the second paragraph you wrote:
“if you took 19.14.0 from January 2022, then RUR 19.14.2 in July 2022 should deliver the security fixes equal to the RU 22.214.171.124 released at the same time.”
I believe it should be 126.96.36.199 not 188.8.131.52.
by the way, thank you informing about MRPs.
Thanks Mustafa – corrected it
I’m waiting for the ODA 19.17 which should be released on Oct/22.
I have already applied 19.17 on other platforms.
Why is the 19.17 taking so long to be released?
Would you know something about it?
I can’t tell you since I have no insights into the ODA certification process. I know from other platforms that often n-1 is recommended when n appears. So 19.16.0 when 19.17.0 gets released etc. (and this is not my personal opinion to be clear).
Hi Mike, will MRPs cause issues for subsequent RUs? In the past if you apply a one off patch you have to roll it back, apply the RU, and then apply a new version of the one off patch.
That’s an awesome question – let me clarify this and add it to the page then.
if MRPs are cumulative, then why are these bugfixes missing in December MRP, that were included in November MRP?
Bug 34333986 ORA-600 [KTUSCV1:CV Buf Too Big]/ [Kdbblkcheckerror] Block Corruption With Check Code 6127 After Migrating From Solaris to Linux
Bug 30691454 DBHOME patching completely hung with kpdbhashtable_find multiple instance hang
Bug 34538232 after gi patching and db upgrade to 19.16 sessions are consuming temptablespace.
Bug 34574048 DataGuard: “alter database switchover verify” fails with ORA-16467/ ORA-16724/ ORA-16629
Bug 34366627 DNFS IO Hang During Stress Test
thanks for the hint. I am on vacation already but you triggered my urge to find out what’s going on.
Sorry, I was irritated by Chapter 5 of Readme which lists only the Bugfixes per MRP that were added that month.