#ESXi #Intel #NUC #Enthusiast #Serpent #Canyon

The Intel NUC Enthusiast product line is typically geared towards content creators and gamers. These Intel NUCs, include Skull Canyon, Hades Canyon and Phantom Canyon, are all equipped with an onboard discrete GPU.


The Intel NUC 12 Enthusiast, codenamed Serpent Canyon, is the latest offering in this product line and it is also the first Intel NUC that includes both an Intel CPU and an Intel GPU, which is based on the latest Intel Arc graphics.

Within the VMware Community, both the Skull Canyon and Hades Canyon were extremely popular due to the additional graphics and storage capabilities at the time. With my recent updated findings on iGPU passthrough for recent Intel NUCs with ESXi and the combined compute and graphics solution from this latest offering from Intel, Intel NUC 12 Enthusiast can make for a pretty powerful and capable VMware setup!

Let’s take a closer look at the new Intel NUC 12 Enthusiast from a VMware point of view!

Compute

The Intel NUC 12 Enthusiast is a single SKU offering that includes the following CPU configuration:

  • Intel 12th Generation Intel Core i7-12700H
    • 14 Processor Cores (6P+8E), 20 threads, 24MB Intel Smart Cache, 35W TDP
    • P-Cores: 4.7GHz Turbo; E-Cores : 3.5GHz Turbo

You can also find the complete technical CPU specifications here.

Network


Unlike the previous Intel NUC Enthusiast kits which included two onboard network adapters, the Intel NUC 12 Enthusiast has only a single network adapter using an Intel i225 (2.5GbE) which is similiar to other Intel NUC 12th Gen models. The network adapter is automatically recognized when using the latest ESXi 8.0 release or with the Community Networking Driver for ESXi Fling, if you want to run earlier ESXi 7.0 versions.

If you need additional networking, you can take advantage of the two Thunderbolt 4 ports using these Thunderbolt 10GbE solutions for ESXi or look at USB-based networking by using the popular USB Network Native Driver for ESXi Fling, supporting over two dozen types of USB-based network adapters.

Storage


The storage options on the Intel NUC 12 Enthusiast is quite plentiful for the form factor with the ability to add up to three NVMe devices: 2 x M.2 PCIe x4 Gen 4 (2280) and 1 x M.2 PCIe x4 Gen 3 (2280). This means you can easily setup vSAN and also have ESXi and the ESX-OSData volume residing on the third and dedicated NVMe device, future proofing your investment for future vSphere releases. You can also forgo vSAN and setup multiple local VMFS datastores, giving you the flexibility based on your requirements.

If you need even more storage, you can also use the two Thunderbolt 4 ports and add these Thunderbolt M.2 NVMe solutions for ESXi providing you with even more storage capacity and configuration options.

Graphics


The discrete GPU in the Intel NUC 12 Enthusiast is an Intel Arc 770M, the mobile version of their flagship Intel Arc 770 GPU, which has up to 96 Execution Units and supports Direct X 12.1, OpenGL 4.6 and OpenCL 3.0 graphics APIs.

The question that I am sure many of you are wondering about is whether this new discrete Intel GPU will work with ESXi … and I am happy to share that it is indeed fully functional for GPU passthrough to a VM and specifically when using Ubuntu Linux. For further background, please take a look at my recent blog post which provides some updated findings to iGPU passthrough for recent Intel NUCs with ESXi.

Arc 770M

To passthrough and consume the discrete GPU, here are the high level instructions for setting up the VM:

Step 1 – Create and install Ubuntu Server 22.04 VM (recommend using 60GB storage or more, as additional packages will need to be installed). Once the OS has been installed, go ahead and shutdown the VM.

Step 2 – Enable passthrough of the iGPU under the ESXi Configure->Hardware->PCI Devices settings and then add a new PCI Device to the VM and select the iGPU. You can use either DirectPath IO or Dynamic DirectPath IO, it does not make a difference.

Step 3 – Optionally, if you wish to disable the default virtual graphics driver (svga), edit the VM and under VM Options->Advanced->Configuration Parameters change the following setting from true to false:

svga.present

Step 4 – Power on the VM and then follow these instructions for installing the Intel Graphic Drivers for Ubuntu 22.04 and once completed, you will now be able to successfully use the iGPU from within the Ubuntu VM.

Here is a screenshot of Ubunutu 22.04 VM which has the default virtual graphics disabled and is connected using a remote session utilizing the discrete GPU passthrough from an Intel NUC 12 Enthusiast running ESXi 8.0 (also works on latest ESXi 7.0 Update 3 release).

iGPU

The Intel NUC 12 Enthusiast also includes an integrated graphics (iGPU) device and I was also curious if I could passthrough this device to a different VM? Attempting to power on the VM with iGPU passthrough, I immediately received the following error:

Module DevicePowerOn power on failed. Failed to start the virtual machine.

I started to look at the VM logs (vmware.log) to see if there were any hints on why this was failing and saw the following:

PCIPassthru: Selected device 0000:03:00.0 is outside of the NUMA configuration
PCIPassthru: Device 0000:03:00.0 barIndex 0 type 3 realaddr 0x6b000000 size 16777216 flags 4
PCIPassthru: Device 0000:03:00.0 barIndex 2 type 3 realaddr 0x6000000000 size 17179869184 flags 12
PCIPassthru: Device has PCI Express Cap Version 2(size 60)
PCIPassthru: Registered a PCI device for 0000:03:00.0 vIRQ 0xff, physical MSI = Enabled (vmmInt = Enabled), IntrPin = 0
PCIPassthru: total number of pages needed (4198400) exceeds limit (917504), failing

The very last line got my attention and seems to indicate some memory limit was being reached and while searching online, I came across this VMware KB 2142307 outlining requirements for VMDirectPath I/O and Dynamic DirectPath I/O and decided to try one of the VM Advanced Settings:

pciPassthru.use64bitMMIO = “TRUE”

and that fixed the issue and allowed me to successfully power up the VM with the iGPU!

If we look at the VM logs again, we can see it has successfully created the IOMMU mapping for the iGPU:

PCIPassthru: Selected device 0000:03:00.0 is outside of the NUMA configuration
PCIPassthru: Device 0000:03:00.0 barIndex 0 type 3 realaddr 0x6b000000 size 16777216 flags 4
PCIPassthru: Device 0000:03:00.0 barIndex 2 type 3 realaddr 0x6000000000 size 17179869184 flags 12
PCIPassthru: Device has PCI Express Cap Version 2(size 60)
PCIPassthru: Registered a PCI device for 0000:03:00.0 vIRQ 0xff, physical MSI = Enabled (vmmInt = Enabled), IntrPin = 0
PCIPassthru: successfully created the IOMMU mappings
PCIPassthru: Attempted to program PCI cacheline size 32 not a power of 2 factor of original physical 64 for device 0000:03:00.0

To passthrough and consume the iGPU, here are the high level instructions for setting up the VM:

Step 1 – Create and install another Ubuntu Server 22.04 VM (recommend using 60GB storage or more, as additional packages will need to be installed). Once the OS has been installed, go ahead and shutdown the VM.

Step 2 – Enable passthrough of the iGPU under the ESXi Configure->Hardware->PCI Devices settings and then add a new PCI Device to the VM and select the iGPU. You can use either DirectPath IO or Dynamic DirectPath IO, it does not make a difference.

Step 3 – Optionally, if you wish to disable the default virtual graphics driver (svga), edit the VM and under VM Options->Advanced->Configuration Parameters change the following setting from true to false:

svga.present

Step 4 –Edit the VM and under VM Options->Advanced->Configuration Parameters add the following:

pciPassthru.use64bitMMIO = “TRUE”

Step 5 – Power on the VM and then follow these instructions for installing the Intel Graphic Drivers for Ubuntu 22.04 and once completed, you will now be able to successfully use the iGPU from within the Ubuntu VM.

Here is a screenshot of Ubunutu 22.04 VM which has the default virtual graphics disabled and is connected using a remote session utilizing the iGPU passthrough from an Intel NUC 12 Enthusiast running ESXi 8.0 (also works on latest ESXi 7.0 Update 3 release).


Now, how cool is that!? Passthrough of two GPUs to two completely different Ubuntu VMs running on ESXi! I can not wait to hear about the use cases folks have in mind with this additional graphics processing capability 😀

Here are some additional resources that I had used for setting up and accessing my two Ubuntu VMs that might be useful:

Customization

The classic Intel NUC skull or in the case of the Intel NUC 12 Enthusiast, the serpent can be customized using the Intel Software Studio Service if you have Windows installed on the physical Intel NUC 12 Enthusiast system.


The one nice thing about using the Intel Software Studio Service to customize the colors, which includes the mask but also the power button and status lights is that you immediately get real time feedback on what it actually looks like. If not, you can still customize the color settings by going into the BIOS but it does require a reboot to see the changes each time, which can be a bit time consuming.

Speaking of the mask, simliar to previous Intel NUC Enthusiast kits, it is also user replaceable with your own custom designs.


Included with the Intel NUC 12 Enthusiast is the skull and serpent, but you also create your own as you can see from the screenshot above 😀


Huge shoutout to the Simply NUC team for creating a custom WilliamLam.com logo which looks fantastic on the top (or side) of the Intel NUC 12 Enthusiast, as you can stand it up vertically as well.

Something to be aware of if you are in the market for an Intel NUC 12 Enthusiast is that if you purchase from Simply NUC, you can provide your own graphics and they will automatically print that for you as part of the order! To build and design your ideal Intel NUC 12 Enthusiast, please visit https://simplynuc.com/serpent-canyon/ for more details.

Form Factor


For those familiar with the classic 4×4 Intel NUC, the Intel NUC 12 Enthusiast is roughly about 1.5x of the classic Intel NUC, which is still pretty compact and but not as small as the classic Intel NUC and definitely much smaller than the recent NUC 9/11/12 Extreme. The Intel NUC 12 Enthusiast is ~2.5L volume with a measurement of 230 x 180 x 60 mm for those interested.

ESXi


Finally, no surprise here but the screenshot above shows ESXi 8.0 successfully running on the Intel NUC 12 Enthusiast, it has also been configured with vSAN using two of the NVMe devices with the third running ESXi and ESX-OSDATA. For those interested in running ESXi 8.0, no additional drivers are required as the Community Networking Driver for ESXi has been productized as part of the ESXi 8.0 release. If you wish to use the latest ESXi 7.x releases, it will require the use of the Community Networking Driver for ESXi Fling to recognize the onboard network device.

The following ESXi kernel option cpuUniformityHardCheckPanic=FALSE still needs to be appended to the existing kernel line by pressing SHIFT+O during the initial boot up. Alternatively, you can also add this entry to the boot.cfg when creating your ESXi bootable installer. Again, you need to append the entry and do not delete or modify the existing kernel options or you will boot ESXi into ramdisk only. If this entry is not added, then booting ESXi with processors that contain both P-Cores and E-Cores will result in a purple screen of death (PSOD) with following message “Fatal CPU mismatch on feature“.

Note: Once ESXi has been successfully installed, you can permanently set the kernel option by running the following ESXCLI command: localcli system settings kernel set -s cpuUniformityHardCheckPanic -v FALSE before rebooting OR you can reboot host and take out the USB and manually edit EFI\boot\boot.cfg and append the kernel option and this will ensure subsequent reboots will contain the required kernel option.

Share:

administrator

ahmedaljanahy Creative Designer @al.janahy Founder of @inkhost I hope to stay passionate in what I doing

Leave a Reply

Your email address will not be published. Required fields are marked *