A bit more than two months ago I did hear from several people having issues with our Hands-On Lab environment. And it became clear that only those who use Oracle Virtual Box 5 see such errors.
Then I read Danny Bryant‘s blog post (thanks to Deiby Gomez for pointing me to it) about similar issues and a potential solution yesterday:
And interestingly one of my colleagues, our PL/SQL product manager Bryn Llewellyn started an email thread and a test internally yesterday as well. The issue seem to occur only on newer versions of Apple’s MacBooks.
Potential Root Cause
The PERL issues seem to happen only on specific new Intel CPUs with a so called 4th level cache.
The current assumption is that Intel CPUs with Iris Pro graphics are affected. Iris Pro means eDRAM (embedded DRAM) which is reported as 4th level cache in CPUID. We have confirmed that Crystal Well and Broadwell CPUs with Iris Pro are affected. It is likely that the Xeon E3-1200 v4 family is also affected.
It seems to be that there’s a bug in the perl binary. It links against ancient code from the Intel compiler suite doing optimizations according to the CPU features. Very recent Intel CPUs have 4 cache descriptors.
People who encountered this used Virtual Box VBox 5.0.x – and it passes this information to the guest. This leads to a problem within the perl code. You won’t see it on VBox 4.3 as this version does not pass the information to the guest.
But actually it seems that this issue is independent of Virtual Box or any other virtualization software. It simply happens in this case as many people use VBox on Macs – and some Macs are equipped with this new CPU model. But people run Oracle in VBox environments and therefore see the issue as soon as they upgraded to VBox 5.0.x.
Potential Solutions
If you are using Oracle in VBox there are actually two solutions:
- Revert to VBox 4.3 as this won’t get you in trouble
This problem was not triggered on VBox 4.3.x because this version did not pass the full CPUID cache line information to the guest. - Run this sequence of commands in VBox 5.0 to tweak the CPUID bits passed to the guest:
VBoxManage setextradata VM_NAME "VBoxInternal/CPUM/HostCPUID/Cache/Leaf" "0x4" VBoxManage setextradata VM_NAME "VBoxInternal/CPUM/HostCPUID/Cache/SubLeaf" "0x4" VBoxManage setextradata VM_NAME "VBoxInternal/CPUM/HostCPUID/Cache/eax" "0" VBoxManage setextradata VM_NAME "VBoxInternal/CPUM/HostCPUID/Cache/ebx" "0" VBoxManage setextradata VM_NAME "VBoxInternal/CPUM/HostCPUID/Cache/ecx" "0" VBoxManage setextradata VM_NAME "VBoxInternal/CPUM/HostCPUID/Cache/edx" "0" VBoxManage setextradata VM_NAME "VBoxInternal/CPUM/HostCPUID/Cache/SubLeafMask" "0xffffffff"
- Of course you’ll need to replace VM_NAME by the name of your VM.
If the error happens on a bare metal machine meaning it happens not inside a virtual image but on a native environment then the only chance you’ll have (to my knowledge right now) is to exchange the PERL before doing really something such as running root.sh or rootupgrade.sh in your Grid Infrastructure installation or before using the DBCA or the catctl.pl tool to create or upgrade a database.
In this case please refer to the blog post of Laurent Leturgez:
Issues with Oracle PERL causing segmentation faults:
http://laurent-leturgez.com/2015/05/26/oracle-12c-vmware-fusion-and-the-perl-binarys-segmentation-fault
Further Information
This issues is currently tracked internally as bug 22539814: ERRORS INSTALLING GRID INFRASTRUCTURE 12.1.0.2 ON INTEL CPUS WITH 4 CACHE LEVEL.
So far we have not seen reports by people encountering this in a native environment but only by people using VBox 5.0.x or Parallels or VMware on a very modern version of Apple hardware.
–Mike
Hi.
I mentioned this in a post also.
https://oracle-base.com/blog/2015/10/05/virtualbox-oracle-12c-and-macs/
It’s a real pain in the backside. I think a lot of people thought it was a problem with the Oracle media at first, but I’ve only seen it happen on the Mac Spin of VirtualBox. It works fine using VirtualBox installed on Windows and Linux. I do all three, so I’ve tested it. 🙂
That workaround you pointed to is what I used also. It’s worked every time for me. Of course, it would be better if you didn’t need it. 🙂
Cheers
Tim…
Tim,
as in most cases I fully agree with you. And glad to hear that the workaround works for you as well.
My personal reading is that this will affect everybody the sooner or later with modern CPUs and this specific CPU configuration. So we may end up delivering a newer perl as a patch. But this is just my personal reading …
I’ll keep you posted.
Cheers
Mike
It’s not necessarily just the Perl binary… it’s anything that uses antiquated CPU instructions using the new Iris Pro chip in new MacBooks. For instance… the Oracle Database on Windows.
So Tim… it’s not specific to virtualization software… that’s just the only way that Oracle software is executed against the Iris Pro chipset.
Have a look at https://medium.com/red-pill-analytics/does-oracle-database-12c-just-not-work-on-windows-2f73e1485904?source=emailShare-4531552505c1-1453426272277
All homage is paid. Thanks guys.
Hi Mike,
after running the above commands i get now VM failing to start with:
Invalid HostCPUID entry: Invalid leaf number: ‘Cache’
(VERR_INVALID_NAME).
Fehlercode:
E_FAIL (0x80004005)
cheers
Daniel
Komponente:
ConsoleWrap
Interface:
IConsole {872da645-4a9b-1727-bee2-5585105b9eed}
Daniel,
sorry to see and read that. May I please ask you to log this into the Virtual Box forum?
https://forums.virtualbox.org/viewforum.php?f=5
https://www.virtualbox.org/wiki/Community
Thanks and sorry for the inconvenience!!!
Mike
For VMWare Fusion a potential fix would to add:
monitor_control.enable_fullcpuid = TRUE
cpuid.4.4.eax = "0000:0000:0000:0000:0000:0000:0000:0000"
in your vmx file