Why does your most important database run on Oracle 10.2.0.4?

Today is a sunny day. A wonderful beautiful spring day. Flowers are blooming. Trees are getting green. Birds are singing. All fine. And then I find this email in my inbox. A customer wants to migrate between two Big Endiann platforms with a important ERP system. Nothing special – until I read this: The ERP system is currently on Oracle 10.2.0.4 – and it should stay on Oracle 10.2.0.4. The first question I have: Why does your most important database run on Oracle 10.2.0.4? In 2018.

Why does your most important database run on Oracle 10.2.0.4?

This is of course an anonymous example. And it does not really matter who the customer is. There’s a reason why they want to keep their database release as is. Change implies risk. Change is expensive. And why should they move to a newer release as long as the old one is running fine? And of course the application and all its depended products are certified on it.

Well, quick flashback. Oracle Database 10.2.0.4 got released in Spring/Summer 2008. Ten years ago. It ran out of Extended Support on July 31, 2011. Almost seven years ago while I’m writing this blog post.

Do you remember 2008? This was the year when the global financial crisis became obvious. When Lehman went out of business and others such as Bear Stearns had to be bailed out. When Barack Obama got elected to become the next US president. Or speaking for techies: Apple launched the iPhone 2, the first iPhone with 3G support.

Oracle 10.2.0.4 was certified for instance on OEL/RHEL 4 and 5, on AIX 5 and 6.1 and Windows XP and Vista. Of course it was certified on other OS and versions as well.

Core Banking on Oracle 10g? Security?

Two years ago I visited the two largest banks of a European country on two consecutive days. Both showed me their database estate. And both had some 10g databases in production. I asked the IT Leads what systems they still run on Oracle 10g. Of course I received similar answers. Core banking, internet banking, ATM systems etc.

Then I did ask about the security impact. And everybody knew already what it means security-wise to operate your most important systems on such old databases. But the usual reasons were named:

• It does work so we don’t touch it
• The risk of changing it is too high
• We have no budgets at the moment
• Downtime requirements don’t allow us to change it
• The application is not certified on newer Oracle releases
• The application developers retired and we will need to change the code
• Application vendor is not in business anymore
• My database is not facing the internet so why should I care

Etc etc etc. I have heard them all. Many times. And I fully understand the situation.

But in all cases two things aren’t considered carefully:

• Such an old system means a lot of security risk
• The longer you wait, the more complicated it gets, the more risky it gets, the more it will cost

It really frustrates me a lot. It’s of course not only about the database. There are a lot of dependencies to be considered.

But does anybody really believe it will get easier or less risky or cheaper when you wait longer and longer?

Would you never service it?

This brings me back to the picture above. Young boys often dream about sports cars. At least in my generation we did. And as I’m German I dreamed of Porsche. Even though one of our neighbors drove an ugly 80’s Ferrari Mondial for a while. Now think of you’ve had the same dream. And one fine day you saved enough money to go to your local car dealer. And you buy the car you dreamed of since you were a child.

You drive it for the first time. What a feeling. And you store it safely in your garage at home. Lock the garage. You enjoy driving your dream car. Before you go to bed you’d go down to your garage. You’d look at the car, its lines, its fine color. You’d feel very happy.

Would you EVER think of NEVER bringing this car to service?

Of course not. You would know that the resell price would drop a lot if you never serviced it. Even worse, one day it will break down because you never exchanged the oil. Your tires will be old, flat and fall apart. Especially when you try to drive your Porsche 911 Turbo with 300 km/h on your way down South from Munich towards Garmisch-Partenkirchen. You would never take that risk, wouldn’t you?

Of course you would bring your car to the dealership for regular service checks and get your stamp into the service manual. I think there’s no question about it. Just because not only the value would drop but also because it would ruin the car and you’d increase the risk of having a break-down the longer you postpone the mandatory service checks.

What has this to do with databases and Oracle 10g?

People all over the world do exactly the opposite with their most important databases. Instead of servicing them (or as we say: patching) I receive multiple variations of excuses why a system can’t be patched. And I agree, it’s not always your fault. And it’s never easy. Not even fun. But the more important the system is, the less often it gets patched. Unfortunately this is my experience in many cases. Only if security audits get mandatory and force people to comply with strict rules, the pattern changes. And you safely can exchange the word “patching” with “upgrading”.

Shouldn’t it be exactly the other way round? The more important your system is, the more often you should take care on it, the more often you should make sure you have the right patches, the earlier you should plan future upgrades and check the application dependencies.

Of course you don’t plan to sell your databases. Therefore the sports car analogy is not perfect. But databases and the expensive sports car you dreamed of have something in common: Both need regular service intervals for patching (in RAC of course rolling) and upgrades. Don’t miss that. Don’t postpone it.

If you postpone patching and upgrades, you don’t only expose your database to security threats but you also lock in yourself. The longer you wait, the more complicated it gets, the more risky it will be, the more change you’ll see – and the more expensive your project will become.

Don’t miss your regular patching and upgrade intervals. Keep the risk as low as possible.

And don’t operate your most important databases in 2018 on Oracle 10.2.0.4. Don’t …

–Mike

P.S. Some final words if you don’t believe me but read till this point. Check out these links – the first hack happened because month-old Oracle Primavera patches didn’t get applied …:

• https://krebsonsecurity.com/2016/11/san-francisco-rail-system-hacker-hacked/
• https://krebsonsecurity.com/2018/03/checked-your-credit-since-the-equifax-hack/
• https://en.wikipedia.org/wiki/Yahoo!_data_breaches
• https://www.nytimes.com/2018/03/15/technology/saudi-arabia-hacks-cyberattacks.html