This question in similar occurences gets asked during every Upgrade Workshop at least once. People would like to know if they can run an Oracle Database or Oracle Real Application Clusters or Oracle Grid Control or Oracle Fusion Middleware or … in an VM environment with VMWare’s virtualisation products.
And the answer is: Yes, you can!!
But … there’s a fine print you should take care on before setting up virtual environments with a different solution than XEN based Oracle VM.
Please read Note:249212.1 – Support Position for Oracle Products Running on VMWare Virtualized Environments for further details:
Oracle customers with an active support contract and running supported versions of Oracle products will receive assistance from Oracle when running those products on VMware virtualized environments.
If Oracle identifies the underlying issue is not caused by Oracle’s products or is being run in a computing environment not supported by Oracle, Oracle will refer customers to VMware for further assistance and Oracle will provide assistance to VMware as applicable in resolving the issue.
If a problem is a known Oracle issue, Oracle support will recommend the appropriate solution on the native OS. If that solution does not work in the VMware virtualized environment, the customer will be referred to VMware for support. When the customer can demonstrate that the Oracle solution does not work when running on the native OS, Oracle will resume support, including logging a bug with Oracle Development for investigation if required.
If the problem is determined not to be a known Oracle issue, we will refer the customer to VMware for support. When the customer can demonstrate that the issue occurs when running on the native OS, Oracle will resume support, including logging a bug with Oracle Development for investigation if required.
NOTE: Oracle has not certified any of its products on VMware. For Oracle RAC, Oracle will only accept Service Requests as described in this note on Oracle RAC 11.2.0.2 and later releases.
Considering the fact that Oracle is probably the most expesive database available in the market these days, I would think that they would be a little more mature to try and support VMware.
Charl.
thanks for your comment – and I believe there’s a misunderstanding because of the wording sequence of the support note.
We DO SUPPORT Oracle on VMware environments. You just have to take into consideration in case of a failure that it could happen that you’ll have to be able to reproduce misbehaviour of an Oracle product outside a VMware environment.
For example, the optimizer will give you a wrong result – 16 instead of 18 rows (worst bugs, I know!). I would bet with you that Oracle Support won’t tell you now: “Hey man, this is a VMware env, you’ll have to reproduce it outside of VMware or we close the SR.” For sure, this won’t happen.
But if you have, let’s say a Win7 VMware environment, you run Oracle 11.2 in it and you’ll get an OSD error – or to make it less complicated – an ORA-7445 (core dump) and the system state dump shows a relation to the OS layer THEN Oracle Support might ask you to reproduce this behaviour outside a VMware environment. Especially in cases were we can’t reproduce this internally in our environments.
Actually there are hundreds of customers out there running Oracle in VMware environments. You can do this and we’ll support you.
I believe the misunderstanding results from “Oracle does not certify its products on VMware”. That’s right because we certify only vs. the base operating system. We neither certify OS patches (they are all certified by default). And in a VMware environment you’ll need a fully installed OS in there – and Oracle should be certified for this OS 🙂
Hope this helps – thanks and kind regards
Mike
Charl,
and just to add this information from a discussion with a colleague.
I won’t answer license questions here as I’m not 100% familiar with all topics and Oracle Sales is the better direction here to talk to. But still we get a lot of license questions regarding VMware.
” * Oracle states that in a VMware environment (2 cpu with 6 cores), ALL cores COULD be used for the database
o Even though only 2 virtual cores are assigned to the VMware running the DB
o The VMware will do its best to restrict the number of requests to the real cores simulating power for 2 cores
o But there is no guarantee that this is perfect and therefore in some situations the VMware could use more than 2 cores of real power
o On top of this, all requests from the 2 virtual cores will be handled by the 12 real cores and therefore the load is spread over the entire physical machine
* As a result of the above, you need to license ALL cores of the VMware host, even if you have only assigned 2 cores to the VMware virtual Windows box”
Hope this helps – regards
Mike
VMWare provides a CPU affinity mechanism in which you can designate on which cores a virtual machine can run. Would your statement in regards to licensing all physical cores on a host hold true if CPU affinity was set on the Oracle VM to restrict it to running on 2 specific physical cores?
Bill,
thanks for your comment. And the answer is YES as far as I know (so far).
Kind regards
Mike
Thanks for clearing that up. I would assume this would also mean that if you’re running it in a DRS enabled cluster, you would need to license all cores on all hosts in the cluster.
I certainly hope Oracle reconsiders their licensing strategy, as this is a definite hinderance for any new customers looking to implement new systems.
Hey Bill,
thanks again. And I can fully understand your point but I’m actually not aware of any changes to this licensening strategy. You might ask your Oracle sales contact – he or she might be able to find out more news about future plans. But honestly I would guess that Larry would more prefer people to use Oracle VM 😉
Thanks and kind regards
Mike
So If I am a new customer planning to run 100% virtual on vmware will oracle not sell me thier database software or recommend I use MSSQL?
Hi Korman,
thanks for your comment. And I’m not sure what your point is. You might read the support statement carefully again. It does neither say “not certified” nor “not supported”. It simply makes clear that:
a) we don’t certify versus VMware – that’s a matter of fact as we certify for OS only
b) in case of a failure it might happen that Oracle Support will ask you to reproduce the case outside of the VMware environment. Believe me this won’t happen if you hit a wrong query result bug with the optimizer. But if you get an ORA-7445 and the stack trace shows a dependency on OS-near modules then this might happen. So you should be prepared for such situations.
There are hundreds or thousands of customers out there using Oracle in VMware environments.
But – just a hint as VMware is not for free – you might use OracleVM which is based on open source XEN. We use this internally for thousands of environments.
Kind regards
Mike
Hi All,
I would like to inform you that i experienced few times oracle support asking me to install oracle aplication on physical machine. See following oracle response.
— quote —
=== ODM Answer ===
Thank you very much for your update.
Yes. you are correct. The FRM-93000 error message is not related with your incident because the time is not matched.
Unfortunately, in the application.log file, only this error message ( FRM-93000) was recorded in it and there is no other REP-XXX error message found in it and match with your incident time.
So, the internal server error may be caused by other factor (e.g. network, machine resource)
Since you are using VMWare with our product and it is not certified with Oracle, we strongly suggest you to install the Oracle Application server in the physical machine instead of VMWare.
We hope that we answered your question.
If you have any further question, please update the SR and we will be glad to assist you further.
Otherwise, please close the SR via My Oracle Support
Thank you very much again
— end —
i am running oracle application on redhat EL4 in VMware virtual machine. i want to make use of VMware feature for make application reliable.
But it seems application itself causing issue or wrong configuration. I don’t want to blame oracle. but i want to point out that oracle support can be easily request you to reproduce on physical machine.
Regards,
Anothai
Anothai,
thanks for your comment.
The Support Engineer did write:
“[..] So, the internal server error may be caused by other factor (e.g. network, machine resource) [..]”
That’s exactly the point I’d light to emphasize on. If the issue turns out to be dependent or close to OS/network/… layers then Oracle Support might ask you to reproduce the behaviour outside of a VMware environment. And pointing out that EBS did not get certified with VMware is right as well as this is true.
Me personally fully understand the need for virtualization solutions – and I did explain in another comment that we use OracleVM (which is XEN open source based) internally on hundreds or thousands of machines.
It’s impossible for a software vendor to test all configurations in a virtual environment as well. If you expect this from Oracle, MS, SAP and others believe me, the software won’t get cheaper in any way.
The support statement simply says:
You can do that but in case it turns out to be dependent in any way with the virtual layer you’ll have to be able to reproduce it outside your virtual environment.
Thanks and kind regards
Mike
Hi Mike,
Thank you for your response. It seems like chicken and egg for oracle customer and oracle support since oracle asking to reproduce on physical server. It would be better if oracle can prove that incident is not relate to oracle product. I believed that will be motivate users to use oracle product.
Regards,
Anothai
I got a Blue Screen issue in VMWARE Windows 2008R2 VM. I raised a case. The TS said that only 11.2.0.2 will be supported. The question is:
1) I cannot find the 11.2.0.2 for Windows 2008. Oracle is a strange company. the eEelivery web only provide 11.2.0.1 software download.
2) My most projects need 12+ VMs running on two Physical servers. I need Oracle RAC also. If it cannot provide it.
Oracle doesn’t support VMware will be a very bad thing to me and all my customers.
I can see more and more customer move to other DB.
Anothai,
sorry for the delay in my reply – the blog operators hit a bug in the MovableType software we are using which made it unable to administer anything in the past two weeks (not only Oracle sw has bugs *smile*).
Anyway, at the beginning of my career at Oracle I did 6 years RDBMS core and Mission Critical support. And I can tell you – believe me or not – how tough and strenouss it is to convince a customer that not Oracle but his HD controller or EMC or Windows RAID-5 disk array or … or … did produce the error. And back in these old days there was no VM layer involved. I could easily tell you tons of these stories (and you’ll fall asleep the sooner or later).
So from an Oracle Support perspective it is the only way to deal with VMs. Otherwise we’ll have to multiply or environments and increase the complexity a lot. And I bet you and others wouldn’t accomodate for that extra share of resources 😉
I’m still with you from your perspective – but from the Support viewpoint the whole picture looks a bit different 🙂
Thanks, Mike
Johnm,
thanks for your comment – and sorry for the delay in my reply (bug in the MovableType sw of this blog prevented any admin tasks for two weeks now).
You’ll find 11.2.0.2 for any release on support.oracle.com ONLY. Simple reason: it’s a patchset (even though it will be distributed per customer request as a full release install). This will happen for 11.2.0.3 as well etc. So Oracle isn’t THAT strange (and I agree with you, Oracle sometimes in strange in many different ways … believe me, I work now nearly 15 years for Oracle).
Again, Oracle does support VMware. But especially in case of a BLUE SCREEN on Win there might be a close relation to the OS layer. Typically OS drivers will cause a halt (aka Blue Screen). Actually I haven’t seen the database causing a blue screen as the Oracle stack as long as you don’t use clusterware/GI is not even close to the OS layer.
That’s why the Oracle support points you to a physical server.
Ask Microsoft to analyze your Blue Screen dump – I bet with you that it isn’t dependent to the Oracle threads as long as there are no ORA-7445 errors closely before in your alert.log.
I fully understand your frustration – but I do understand the support position as well 🙂
Mike
Mike,
I do not believe that VMWare affinity changes licensing requirements. I am basing this on the Partitioning Policy document, http://www.oracle.com/us/corporate/pricing/partitioning-070609.pdf.
Hi Michael,
thanks for your comment – hope you are doing well!!
I see your point. Currently VMware does simply sort of soft partitioning. It’s never clear on which of your underlying CPU resources the VM subsystem does operate.
And I can’t predict Oracle’s license strategy as this is done far above our heads. But still there might be possibilities to change things in one or the other direction.
Thanks again for this valuable input!
Mike
Hi all,
i’ve virtualized a Windows 2003 cluster with Oracle 9.2.0.5.
The cluster is working very well, but Oracle at least once a day is unresponsive.
The solution is to reboot the servers.
Is there a solution to my issue?
I’m running on VSphere 4.0 u1, and the OS had all the required patches.
On the physical nodes, we never had those issues.
Can You please help me understand if that version of Oracle is certified to be run on a VM?
Thank you.
Cico.
Hi Cico,
I have to smile a bit as 9.2.0.5 is PRETTY old, unsupported (evtn though 9.2.0.6 was certified on Win2003). VSphere is former VMware ESX Server, right?
Take my advice and first upgrade your database to 11.1.0.7 or 11.2.0.2 (for 11.2.0.2 you have to apply patch 9.2.0.8 first) – and if you still decide for any reason to use Oracle 9.2 then download 9.2.0.8 from support.oracle.com and apply it.
If the issue persists, check the alert.log. What is going on there? And open an SR with Oracle Support. Without checking the system nobody can tell what is causing the issue. Maybe your virus scanner on the OS host is running at exactly this time. I’ve seen so many possible issue causing such trouble – it’s not necessarily the VMware environment. Check Windows System/Event logs.
And if you don’t see the issue outside the VMware environment I would recommend to run without VMware – either no virtualization or use OracleVM or Oracle/Sun Virtual Box instead.
Thanks, Mike
Mike,
thank you for your response!
Yes VSphere is formerly ESX.
I’m asking myself: why on physical environment we rarely had those issues (once a month) and on virtualized env. we got those errors (more or less) once a day?
Thank you for your time!
Cico.
Hi Mike,
I am a system administrator in my company. We produce software that deal with high number of users (minimum 10 and maximum 10,000). Recently our customers insist on using VMs and our sales have suffered serious challenge. Especially one of the largest our sales also have been exposed to this risk.
I read contents and comments in this post and Thank you very much. the question that arises is that: Would you list the risks in using Oracle on Vm? This list will help us to give a good response to our customers.
Mike,
Your title is misleading, I think it is very clear that Oracle does not certify VMware in the first paragraph. With that caveat, the fact that Oracle would even attempt to support customer with uncertified software/hardware is pretty darn nice. Now, the big question is if Oracle is potentially going to ask the customer to reproduce the error on native o/s and bare metal, without the hypervisor layer, probably means that customers have to weigh the cost of spare hardware to provide debugging if needed, versus the cost of virtualizing their environment.
What paying Oracle customers should dislike about a policy such as this trick being pulled on VMware customers is the bit about reproducing a problem when "running on the native OS." Shouldn’t it also suffice to reproduce it in an Oracle VM (OVM environment) or even within Virtualbox for that matter?
One would think so. Is that too logical?
Kevin,
thanks for your comment – and an interesting statement from your side.
As you know both Oracle VM and Virtual Box is code Oracle has sort of control on – but not on VMware. As I’ve spent some time dealing with issues people did raise to me regarding this post (some simply sent emails and didn’t comment to the public) I can tell you that Oracle Support did always a pretty good job in not teasing customers. The 6 or 7 SRs I have looked at did extremly well justify if this is an independent issue or if it might be VM related.
Mike
I think that you recommend to use Oracle VM to get all layers of Oracle’s business. I installed oracle database on VMWare ESX in long time and I had no problem about administration. but one point is that VM solution just have one purpose … cost reduction
Now that Oracle RAC is supported in VMWare, would it be sufficient to reproduce an issue in a RAC/VMWare setup, rather than on physical hardware ?
All, this has been going on for some time and I have had both Oracle and VMware state that they fully support the VMware configuration. I know that some users have had performance problem and this is due to improper sizing of ether their physical host or guest OS “NOT THE UNDERLYING VMware”. I have ran both SUN VM and VMware and both perform great. See the below links.
http://www.vmware.com/support/policies/oracle-support.html
http://www.vmware.com/solutions/partners/alliances/oracle-vmware-support.html
Hi all,
I wonder if someone here could help me to resolve the problem I have.
I have an application composed of two software components and a data base. I have tested my application on a virtuel environment; that is : a virtual environement for each component and also for the oracle data base. With this virtual environement, I have no problem and my application works well.
Now, when I place each component of my application on a physiqual server and also the data base on a different physical server, I noticed problems. The major problem is that it seems that connections between one of teh components ans the data base, are lost and my application cannot continue.
can anyone give ma an idea, what can be the problem with the real environment.
Thank you in advance
Thanks for your post but sorry to say that I can’t help you with just this limited very general information.
But you might either:
(a) raise a support ticket at support.oracle.com or
(b) go to the OTN forums and search for the best matching topic and post your inquiry there.
Thanks and kind regards
Mike
Mike, when you say "in case of a failure that it could happen that you’ll have to be able to reproduce misbehaviour of an Oracle product outside a VMware environment" is it true to say that you always have the right to ask us to reproduce the bug on another hardware plateform ? Meening even if we run Oracle product on physical hardware, lets say an IBM Blade, and we find a new bug you never seen before, you could potentialy ask us to reproduce the bug on some other hardware to demonstrate that the bug is related to Oracle & not to a specific hardware ? My point is you do not certify on VMWAre nor on IBM nor on Dell, etc… you stop at the OS level, you cetify on AIX not on pSeries… what if a bug exist with the new Power7 processors but do not exist on Power6 ?
Mario,
thanks for your comment.
First of all you are 100% right – Oracle certifies on the OS level and just in very rare cases (RAC etc.) hardware gets special certifications. But we don’t certify usually on IBM, Dell, NEC or whatever hardware.
But in the VMware case you are involving an extra SOFTWARE layer between Oracle and the OS. And this will make things more complicated – by the way not only for Oracle but for everybody else as well.
As I’ve explained in many replies above already Support "may" ask you to reproduce misbehaviour out the VIRTUAL environment. It won’t matter on which hardware you’ll do that, it would simly matter that it’s the same OS platform (so logging a bug in Solaris won’t mean that you should verify that on Linux as well – we’ll do such a thing internally if required). Oracle Support won’t ask you for verification if you’ll get wrong results from a query – but the folks may ask you in case of an ORA-7445 core dump once the stack trace shows some relation to the OS layer. And this makes a 100% sense. How should we fix an issue which is not clearified to be dependent on our code but may depend on the virtualization code layer? We’ll do that actually if we (Oracle) have control on the virtualization code layer (Oracle VM, Virtual Box) – but not for 3rd party vendors.
Thanks, Mike
Mike,
You say "But in the VMware case you are involving an extra SOFTWARE layer between Oracle and the OS."
I do think this is not true… we certainly add extra software layer… between the hardware & the OS that you certify on… but not between the OS & Oracle… For you, it is a pure RedHat or OEL (we are a linux shop).
Hi Mike,
Thank you for a great article and for keeping the comments answered almost a year later!
Regards,
Anthony
"But in the VMware case you are involving an extra SOFTWARE layer between Oracle and the OS. And this will make things more complicated – by the way not only for Oracle but for everybody else as well."
Actually, the OS runs on VMware, so there is no software between Oracle and the OS as you stated.
We have over 400 VMs in our environment, and we have yet to see any VMware related issues.
@Mario:
You are right and I was wrong 😉 Thanks for the correction. But you are involving a piece of SW between Oracle, the OS and the guest OS – and that doesn’t make things easier. When you look for instance at the Windows architecture with the HAL (Hardware Abstraction Layer) and you put now a virtualization software plus a guest OS on top the virtualization software has to map all requests in the correct way to the host OS and all its layers. This is not trivial (but thanks to all the VM providers it usually works well).
But again the problem may arrise that Oracle gets errors related to the OS layer – and this may lead to the situation that you’ll have to be able to reproduce things without the virtualization layer.
😉
Mike
"Considering the fact that Oracle is probably the most expesive database available in the market these days" –> Well, just take a look at DB2 on z/OS and talk then about expensive 🙂
I see no issue with Oracle’s support/certification statement.
For the record, this is what Microsoft says about working with non-Microsoft, non-partnered software when you have premier support (article id 897615):
"…As part of the investigation, Microsoft may require the issue to be reproduced by the customer independently from the non-Microsoft hardware virtualization software. This may be done on Windows Server 2008 (with Hyper-V), the actual hardware platform with the Windows operating system installed directly upon it, or on both…".
I actually think Oracle’s policy is easier to read between the lines.
Maybe Oracle should include an additional clause that adds something like "…may require you to reproduce on Oracle VM…". That might increase the uptake of Oracle VM 😉
Great thread.
Does this position also translate to other non-Oracle virtualization mediums i.e. HP-UX 11i v3 Virtualization Server OE (VSE-OE) Integrity Virtual Machine? Will Oracle Support ask end users to replicate the issue directly to a native OS without a virtualiztion middle tier?
Thanks in advance.
Yes, as far as I know this will apply the HP’s virtualization software as well. But to be 100% sure you’ll have to double check with Oracle Support as I’m not in the position to give you any official statement on that topic.
Thanks for your input and kind regards
Mike
Hi Mike,
Nice info in this thread!
To add to Fernando’s question, what about LPAR’s?
Best regards,
Steven
Steven,
thanks for your question – and actually I don’t know how Support handles those – so you might please (I hate to say that but …) open a Service Request and ask Oracle Support.
Thanks, Mike
this is of course all x86 related.
hypervisors on Solaris/SPARC and AIX/POWER are supported.
Also worth mentioning if you are running Oracle on VMware there are two Oracle Enterprise Manager 12c (EM12c) plugins available from Blue Medora specifically focused on VMware that provide EM12c based performance / health / availability monitoring, management, as well as Oracle on VMware license compliance monitoring and alerting. Both of them can be found in the Oracle Extensibility Exchange:
http://apex.oracle.com/pls/apex/f?p=34841:11
It appears that the bottom line here is that Oracle does NOT support Oracle on VMware; Oracle does support Oracle on a particular OS and if an issue appears to be related to that OS being present on VMware, then the issue will need to be reproduced outside of VMware for Oracle to provide support for that issue.
I didn’t read all the comments, and I am not sure if someone already told that VMware has a special support policy for Oracle with expanded support and total ownership of the problems that occur in Oracle products running over VMware environment. Take a look at the following url:
https://www.vmware.com/br/support/policies/oracle-support
Thanks a lot for this information.
Cheers
Mike
Hi Mike,
When you say "Oracle will only provide support for issues that either are known to occur", do we have a repository/page where we keep all the known issues that may occur while running Oracle products on VMware?
Regards,
Sanjay
Hi all,
sorry but our great blogging software does not allow me to edit this 4 year old entry with a change – but this mos note:
Note:942852.1 – VMWare Certification for Oracle Products
mentioned still in the blog post disappeared without notice or pointer to a newer version from the support portal.
Sorry for the inconvenience
Mike
Sanjay,
no, unfortunately such a page does not exist – but it’s a good starting point at MOS Note 161818.1, then drill down via the links under the release number in the left-most column, drill further down to your specific patch set and check for the known issues and alerts. That’s the best you can do.
Thanks
Mike
Hi Mike,
there is some document or metalink note about the bellow text??
"But there is no guarantee that this is perfect and therefore in some situations the VMware could use more than 2 cores of real power o On top of this"
thanks,
regards
Mariano
Mariano,
I don’t think so. And I’m not even sure anymore if this is true these days. 4 years ago when I wrote the comment you are referring to it was true. You’ll have to check with Support for an official statement please.
Mike
I have run Oracle on VMWare and OVM and never had any issues that the underlying software Vendor did not resolve. What Oracle customers and potential customers need to understand is that Oracle Licensing does NOT allow for virtualiztion. They don’t even allow it on OVM. The only reason customers use OVM is so they can partition out CPUs to meet their licensing requirements and LMS will only recognize this if you go in to the hypervisor and PIN the CPUs to a given guest which of course defeats the point of virtualiztion.
Jay
I have run Oracle on VMWare and OVM and never had any issues that the underlying software Vendor did not resolve. What Oracle customers and potential customers need to understand is that Oracle Licensing does NOT allow for virtualiztion. They don’t even allow it on OVM. The only reason customers use OVM is so they can partition out CPUs to meet their licensing requirements and LMS will only recognize this if you go in to the hypervisor and PIN the CPUs to a given guest which of course defeats the point of virtualiztion.
Jay
I think it would be in everyone best interest if Oracle would adapt to a technology that has been around since the 80s. Provide licensing for the virtual layer and not just the physical. Get over yourself.
Soft Partitioning:
Examples of such partitioning type include: Solaris 9 Resource Containers, AIX Workload Manager, HP Process Resource Manager, Affinity Management, Oracle VM, and VMware.
Unless explicitly stated elsewhere in this document,
soft partitioning (includingfeatures/functionality of any technologies listed as examples above) is not permitted as a means to determine or limit the number of software licenses required for any given server or
cluster of servers.
I think that the ‘reproduce on outside of VMWare’ requirement is kinda bunk. If I called you and said I’m having this issue with my AMD Opteron HP server, would you tell me ‘I’m sorry, but you’ll need to reproduce it on an Intel based Dell server to prove that its Oracle that is really the problem.’? I don’t think so.
Talvez oracle esta forzando a usar sus productos… siento que esto tiene que ver mas con la parte comercial…
saludos,
Marcelo
Soft Partitioning, Threads, CPUs and Licensing…
According to new SE2 License there is 16 CPU threads and 2 CPU socket limit. This mean I can use maximum configuration with 2 CPU 4-core each with HT (Hyper Threading) enabled. If I have 6-core or 8-core CPUs in 2 socket configuration I have to disable HT to conform licensing requirement?
But on the same hardware with soft partitioning solution: lets say KVM (Oracle Linux Enterprise) I can’t launch two guests with OracleDB SE/SE1/SE2 on the same license?
"…soft partitioning … is not permitted as a means to determine or limit the number of software licenses required…" what that mean?, some example needed!
Sebastian, you don’t have to disable HT.
The only physical restriction you’ll have to fulfill with SE2 is the 2 CPU Socket requirement. Everything else is ensure via the resource manager by ourselves.
Please see the SE2 postings I’ve done in the past months for a more detailed explanation (e.g. "SE2 – Some Questions, Some Answer")
Cheers
Mike