
{"id":253687,"date":"2026-11-20T11:46:15","date_gmt":"2026-11-20T11:46:15","guid":{"rendered":"https:\/\/mycryptomania.com\/?p=253687"},"modified":"2026-11-20T11:46:15","modified_gmt":"2026-11-20T11:46:15","slug":"your-guide-to-the-prerequisites-for-running-nodes-on-oasis-part-4-tee-setup","status":"publish","type":"post","link":"https:\/\/mycryptomania.com\/?p=253687","title":{"rendered":"Your Guide to the Prerequisites for Running Nodes on Oasis (Part 4): TEE Setup"},"content":{"rendered":"<p>In a detailed 4-part series, I have been describing the prerequisites that you would need to run Oasis nodes. In the earlier parts, we have covered <a href=\"https:\/\/medium.com\/coinmonks\/your-guide-to-the-prerequisites-for-running-nodes-on-oasis-part-1-hardware-and-cloud-providers-1ad888ccb7c8\">hardware and cloud providers<\/a>, <a href=\"https:\/\/medium.com\/coinmonks\/your-guide-to-the-prerequisites-for-running-nodes-on-oasis-part-2-stake-requirements-and-node-ca6c25f46969\">stake requirements and node installation<\/a>, and <a href=\"https:\/\/medium.com\/coinmonks\/your-guide-to-the-prerequisites-for-running-nodes-on-oasis-part-3-system-configuration-bc7a2f023987\">system configuration<\/a> steps. In this final part, I will walk you through the process of setting up a Trusted Execution Environment (TEE).<\/p>\n<p>The preferred privacy-preserving technique used by Oasis in paratimes (parallel runtimes) and ROFL (runtime off-chain logic) is the hardware-based TEE.<\/p>\n<p><a href=\"https:\/\/www.intel.com\/content\/www\/us\/en\/architecture-and-technology\/software-guard-extensions.html\"><strong>Intel SGX<\/strong><\/a>\u200a\u2014\u200aused in most Oasis paratimes as well as ROFL SGX\u00a0apps<a href=\"https:\/\/www.intel.com\/content\/www\/us\/en\/products\/docs\/accelerator-engines\/trust-domain-extensions.html\"><strong>Intel TDX<\/strong><\/a>\u200a\u2014\u200aused by the ROFL TDX raw and container-based apps<\/p>\n<p><strong>Note<\/strong>: For the differences between these ROFL app types, see <a href=\"https:\/\/docs.oasis.io\/build\/rofl\/workflow\/init\">ROFL\u00a0flavors<\/a>.<\/p>\n<p>Before you can configure your nodes to run in a TEE, you need to ensure that your setup meets the 4 basic requirements to run SGX\/TDX enclaves.<\/p>\n<p>SGX\/TDX support for the\u00a0hardwareLatest BIOS updates installedSGX\/TDX enabled in the\u00a0BIOSLinux kernel, drivers, and software components properly installed and\u00a0running<\/p>\n<p><strong>Note<\/strong>: Step 1 here has been covered in part 1 of this prerequisites series, so we will proceed here with the subsequent steps.<\/p>\n<h3>Software Guard Extensions (SGX)<\/h3>\n<p><strong>BIOS configuration<\/strong><\/p>\n<p>Let\u2019s start with steps 2 and 3 from above. First, update your BIOS to the latest version with the latest microcode. You can then proceed with the following BIOS configuration. It is possible that some settings may not apply to all BIOS versions. In that case, configure only the relevant\u00a0ones.<\/p>\n<p><strong>SGX<\/strong>: ENABLE<strong>Hyper-Threading<\/strong>: DISABLE<strong>Intel SpeedStep<\/strong>: DISABLE<strong>Turbo Mode<\/strong>:\u00a0DISABLE<strong>CPU AES<\/strong>:\u00a0ENABLE<strong>SGX Auto MP Registration<\/strong>: ENABLE<\/p>\n<p><strong>Clock synchronization<\/strong><\/p>\n<p>This is a minor but important step. The node\u2019s local clock needs to be synchronized, for example using Network Time Protocol (NTP), because of additional sanity checks within runtime enclaves. If it is off by more than half a second, unexpected runtime aborts may\u00a0occur.<\/p>\n<p><strong>SGX device permissions<\/strong><\/p>\n<p>Ensuring proper device permissions is essential. The user running the Oasis Node binary must have access to the SGX device (\/dev\/sgx_enclave). This can usually be done by adding that user to the right group. For example, if the permissions of the device look like\u00a0this:<\/p>\n<p>crw-rw&#8212;- 1 root sgx 10, 125 Oct 28 09:28 \/dev\/sgx_enclave<\/p>\n<p>In this example, the group is sgx. If the user running the Oasis Node is oasis, add it to that\u00a0group:<\/p>\n<p>sudo adduser oasis sgx<\/p>\n<p>Missing this step may result in the runtime error showing \u201cpermission denied OS error 13\u201d during setup. Also, if you wish to run your node from an interactive session, then you will need to log out once for the permissions to take\u00a0effect.<\/p>\n<p><strong>AESM service<\/strong><\/p>\n<p>SGX enclave execution involves several Architectural Enclaves\u00a0(AE):<\/p>\n<p>Launch EnclaveProvisioning EnclaveProvisioning Certificate EnclaveQuoting EnclavePlatform Services\u00a0Enclaves<\/p>\n<p>Communication between SGX enclaves launched by applications and Intel-provided AE takes place through the Application Enclave Service Manager (AESM). AESM runs as a daemon and provides the socket through which applications can facilitate various SGX services such as launch approval, remote attestation quote signing,\u00a0etc.<\/p>\n<p>Running your Oasis node will also require DCAP attestation, that is, Data Center Attestation Primitives. A quick check to ensure that your system supports\u00a0this:<\/p>\n<p> cpuid -1  | grep &#8220;SGX&#8221;<\/p>\n<p>Look for the following line in the\u00a0output:<\/p>\n<p>      SGX_LC: SGX launch config supported      = true<\/p>\n<p><strong>DCAP attestation<\/strong><\/p>\n<p>The <a href=\"https:\/\/download.01.org\/intel-sgx\/sgx_repo\/\">official Intel SGX APT repository<\/a> is a convenient way to install the AESM service on Ubuntu 24.04 and newer\u00a0systems.<\/p>\n<p>Add the Intel SGX APT repository to your\u00a0system.<\/p>\n<p>curl -fsSL https:\/\/download.01.org\/intel-sgx\/sgx_repo\/ubuntu\/intel-sgx-deb.key | sudo gpg &#8211;dearmor -o \/usr\/share\/keyrings\/intel-sgx-deb.gpg<br \/>echo &#8220;deb [arch=amd64 signed-by=\/usr\/share\/keyrings\/intel-sgx-deb.gpg] https:\/\/download.01.org\/intel-sgx\/sgx_repo\/ubuntu $(lsb_release -cs) main&#8221; | sudo tee \/etc\/apt\/sources.list.d\/intel-sgx.list &gt; \/dev\/null<\/p>\n<p>Next, install the sgx-aesm-service, libsgx-aesm-ecdsa-plugin, libsgx-aesm-quote-ex-plugin, and libsgx-dcap-default-qpl packages.<\/p>\n<p>sudo apt update<br \/>sudo apt install sgx-aesm-service libsgx-aesm-ecdsa-plugin libsgx-aesm-quote-ex-plugin libsgx-dcap-default-qpl<\/p>\n<p>You can confirm the AESM service is running via a simple\u00a0command.<\/p>\n<p>sudo systemctl status aesmd.service<\/p>\n<p><strong>Quote provider configuration<\/strong><\/p>\n<p>Whether you use Intel Provisioning Certification Service (PCS), the Provisioning Certificate Caching Service (PCCS) of your cloud service provider, or your own PCCS, you will need to configure the Intel quote provider ( libsgx-dcap-default-qpl). You can find the configuration file at \/etc\/sgx_default_qcnl.conf. Also, you will need to always restart the aesmd.service after updating the configuration.<\/p>\n<p>sudo systemctl restart aesmd.service<\/p>\n<p>Let\u2019s now go through, one by one, how to proceed based on the quote provider.<\/p>\n<p><strong>Intel PCS<\/strong><\/p>\n<p>Using Intel PCS is the simplest and most generic way, but it may be less reliable than using your own PCCS. Some cloud providers require you to use their own PCCS (described later).<\/p>\n<p>If you are going with Intel PCS, you will need to update the pccs_url value in \/etc\/sgx_default_qcnl.conf to the Intel PCS API\u00a0URL.<\/p>\n<p>  \/\/PCCS server address<br \/>  &#8220;pccs_url&#8221;: &#8220;https:\/\/api.trustedservices.intel.com\/sgx\/certification\/v4\/&#8221;<\/p>\n<p>If you encounter an error in the Intel Quote Provider Library (QPL) configuration file, attestation will not work, and the AESM service may produce unhelpful errors,\u00a0e.g.<\/p>\n<p>Couldn&#8217;t find the platform library. (null)<\/p>\n<p>In that case, the only thing that needs to be changed is the pccs_url value mentioned above. <br \/><strong>Note<\/strong>: Do not add any comments and\/or change punctuation, as that might result in an invalid configuration file.<\/p>\n<p><strong>Cloud service provider\u2019s PCCS<\/strong><\/p>\n<p>As mentioned above, some cloud providers require you to use their own PCCS. Here is the configuration guidance for three of\u00a0them:<\/p>\n<p>Azure\u200a\u2014\u200aRefer to the <a href=\"https:\/\/learn.microsoft.com\/en-us\/azure\/security\/fundamentals\/trusted-hardware-identity-management#how-do-i-use-intel-qpl-with-trusted-hardware-identity-management\">Azure documentation<\/a> for details on how to configure this quote provider. Here, you will find an example of an Intel QPL configuration file that can be\u00a0used.Alibaba\u200a\u2014\u200aRefer to the <a href=\"https:\/\/www.alibabacloud.com\/help\/en\/ecs\/user-guide\/build-an-sgx-encrypted-computing-environment\">Alibaba Cloud documentation<\/a> for details on how to configure this quote provider. Here, you will find the required sgx_default_qcnl.conf changes.IBM\u200a\u2014\u200aRefer to the <a href=\"https:\/\/cloud.ibm.com\/docs\/vpc?topic=vpc-about-attestation-sgx-dcap-vpc\">IBM Cloud documentation<\/a> for details on how to configure this quote provider. Here, too, you will find the required sgx_default_qcnl.conf changes.<\/p>\n<p>If you go with a different cloud service provider, it is essential to consult their specific documentation for the appropriate configuration guidance.<\/p>\n<p><strong>Own PCCS<\/strong><\/p>\n<p>Some users prefer running their own PCCS to choosing from cloud service providers. The <a href=\"https:\/\/www.intel.com\/content\/www\/us\/en\/developer\/articles\/guide\/intel-software-guard-extensions-data-center-attestation-primitives-quick-install-guide.html\">official Intel instructions<\/a> will guide you through the\u00a0setup.<\/p>\n<p><strong>DCAP attestation Docker<\/strong><\/p>\n<p>Oasis also offers an easy alternative to install and run the AESM service on a <a href=\"https:\/\/docs.docker.com\/engine\/\">Docker<\/a>-enabled system. Start the process by using the <a href=\"https:\/\/github.com\/oasisprotocol\/oasis-core\/pkgs\/container\/aesmd\">AESM container image<\/a>.<\/p>\n<p>Next, execute the following command so that it pulls the latest version of the AESMD Docker container. This will also map the SGX devices and \/var\/run\/aesmd directory, while ensuring AESM is running in the background. All this will then start automatically on\u00a0boot.<\/p>\n<p>docker run <br \/>  &#8211;pull always <br \/>  &#8211;detach <br \/>  &#8211;restart always <br \/>  &#8211;device \/dev\/sgx_enclave <br \/>  &#8211;device \/dev\/sgx_provision <br \/>  &#8211;volume \/var\/run\/aesmd:\/var\/run\/aesmd <br \/>  &#8211;name aesmd <br \/>  ghcr.io\/oasisprotocol\/aesmd-dcap:master<\/p>\n<p>The Intel Quote Provider (QP) in the Docker container is configured to use the Intel PCS endpoint by default. To use an alternative to Intel PCS, you can override the Intel QP configuration here within the container. Use the volume flag to mount your customized configuration.<\/p>\n<p>docker run <br \/>  &#8211;pull always <br \/>  &#8211;detach <br \/>  &#8211;restart always <br \/>  &#8211;device \/dev\/sgx_enclave <br \/>  &#8211;device \/dev\/sgx_provision <br \/>  &#8211;volume \/var\/run\/aesmd:\/var\/run\/aesmd <br \/>  &#8211;volume \/etc\/sgx_default_qcnl.conf:\/etc\/sgx_default_qcnl.conf <br \/>  &#8211;name aesmd <br \/>  ghcr.io\/oasisprotocol\/aesmd-dcap:master<\/p>\n<p>You can find the default Intel QP config in the <a href=\"https:\/\/github.com\/intel\/SGXDataCenterAttestationPrimitives\/blob\/master\/QuoteGeneration\/qcnl\/linux\/sgx_default_qcnl.conf\">Intel SGX GitHub repository<\/a>.<\/p>\n<p><strong>Multi-socket systems<\/strong><\/p>\n<p>For a platform with multi-socket systems (e.g. systems with multiple CPUs), the whole process is more complex, especially if you use a hypervisor and run SGX workloads inside guest VMs. In such a case, additional provisioning will likely be required.<\/p>\n<p>Also, the system must be booted in UEFI mode for provisioning to work. This is an important step as the provisioning process uses UEFI variables to communicate with the BIOS. Moreover, you have to enable the SGX Auto Multi-Package (MP) Registration BIOS configuration setting.<\/p>\n<p><strong>Ubuntu 24.04+<\/strong><\/p>\n<p>After you have added Intel\u2019s SGX apt repository as discussed previously, you can now install the Intel SGX MP Registration Agent Service to provision and register your multi-socket system.<\/p>\n<p>sudo apt install sgx-ra-service<\/p>\n<p>After boot, the log in \/var\/log\/mpa_registration.log should indicate successful registration. In case of an error, the first step is to ensure that the SGX Auto MP Registration is enabled in the BIOS. You can also re-provision by rebooting and setting the SGX Factory Reset option to\u00a0enabled.<\/p>\n<p><strong>VMware vSphere\u00a08.0+<\/strong><\/p>\n<p>To enable SGX remote attestation on VMware vSphere-based systems, following the <a href=\"https:\/\/docs.vmware.com\/en\/VMware-vSphere\/8.0\/vsphere-vcenter-esxi-management\/GUID-F16476FD-3B66-462F-B7FB-A456BEDC3549.html\">vSphere guide<\/a> is recommended.<\/p>\n<p><strong>Setup check<\/strong><\/p>\n<p>The final step in SGX TEE setup is to test the settings. To test remote attestation against Intel SGX\u2019s development server, you can use the Oasis <a href=\"https:\/\/github.com\/oasisprotocol\/tools\/tree\/main\/attestation-tool#readme\">attestation tool<\/a> (\u00a0<a href=\"https:\/\/github.com\/oasisprotocol\/tools\/releases\">binary<\/a>).<\/p>\n<h3>Trust Domain Extensions (TDX)<\/h3>\n<p><strong>Note<\/strong>: TDX requires proper SGX installation to be in place first. So, before proceeding through the next steps, you must ensure there is a working SGX environment established in accordance with the SGX installation steps discussed above.<\/p>\n<p><strong>BIOS configuration<\/strong><\/p>\n<p><strong>SGX<\/strong>: ENABLE<strong>SGX memory<\/strong>: at least\u00a0256MB<strong>SGX Auto MP Registration<\/strong>: ENABLE<strong>Hyper-Threading<\/strong>: DISABLE<strong>Intel SpeedStep<\/strong>: DISABLE<strong>SecureBoot<\/strong>: DISABLE<strong>All Internal Graphics<\/strong>: DISABLE<strong>Turbo Mode<\/strong>:\u00a0DISABLE<strong>CPU AES<\/strong>:\u00a0ENABLE<strong>TDX<\/strong>: ENABLE<strong>Memory Encryption (TME)<\/strong>:\u00a0ENABLE<strong>Total Memory Encryption Bypass<\/strong>:\u00a0ENABLE<strong>Total Memory Encryption Multi-Tenant (TME-MT)<\/strong>: ENABLE<strong>TME-MT memory integrity<\/strong>: DISABLE<strong>TDX Secure Arbitration Mode Loader (SEAM Loader)<\/strong>:\u00a0ENABLE<strong>TME-MT\/TDX key split<\/strong>: non-zero\u00a0value<strong>ACPI S3 and deeper power states<\/strong>:\u00a0DISABLED<\/p>\n<p><strong>Host OS\u00a0setup<\/strong><\/p>\n<p>The instructions here are summarized to help you set up an environment for running the ROFL node and other TDX services on Ubuntu 26.04 or newer versions. For details, you can refer to the official <a href=\"https:\/\/ubuntu.com\/server\/docs\/how-to\/virtualisation\/intel-tdx\/\">Canonical TDX how-to<\/a> documentation.<\/p>\n<p>First, install the TDX quote generation service from Intel\u2019s SGX APT repository. Next, install the Quick Emulator (QEMU) for running guest virtual machines.<\/p>\n<p>sudo apt install tdx-qgs qemu-utils qemu-system-x86<\/p>\n<p>Then, disable ACPI S3 and enable TDX. Also, add kernel parameters nohibernate kvm_intel.tdx=1.<\/p>\n<p>sed -i -E &#8220;s\/GRUB_CMDLINE_LINUX=&#8221;(.*)&#8221;\/GRUB_CMDLINE_LINUX=&#8221;1 nohibernate kvm_intel.tdx=1&#8243;\/g&#8221; \/etc\/default\/grub<br \/>update-grub<\/p>\n<p>Now, ensure that the non-root user running Oasis Node is a member of sgx, sgx_prv, and kvm groups on the host. There should also be access to \/dev\/sgx*, \/dev\/kvm*, and \/dev\/*vsock* devices.<\/p>\n<p>Finally, reboot the system for all this to take\u00a0effect.<\/p>\n<p><strong>Setup check<\/strong><\/p>\n<p>The last step is to check that your system is ready to launch TDX VMs. An example\u00a0output:<\/p>\n<p>sudo dmesg | grep -i tdx[    1.368184] virt\/tdx: BIOS enabled: private KeyID range [32, 64)<br \/>[    1.368188] virt\/tdx: Disable ACPI S3. Turn off TDX in the BIOS to use ACPI S3.<br \/>[    9.483666] virt\/tdx: TDX module: attributes 0x0, vendor_id 0x8086, major_version 1, minor_version 5, build_date 20240725, build_num 784<br \/>[    9.483672] virt\/tdx: CMR: [0x100000, 0x77800000)<br \/>[    9.483674] virt\/tdx: CMR: [0x100000000, 0x3ffe000000)<br \/>[   10.587167] virt\/tdx: 1042424 KB allocated for PAMT<br \/>[   10.587173] virt\/tdx: module initializedls -al \/dev\/*kvm*crw-rw&#8212;- 1 root kvm 10, 232 Apr 10 09:56 \/dev\/kvm<\/p>\n<p><strong>Note<\/strong>: As this is an example, the dmesg timestamps and build numbers in your output can\u00a0vary.<\/p>\n<p>If any of the above is missing, you will need the following debug\u00a0tools.<\/p>\n<p>apt install msr-tools<br \/>git clone https:\/\/github.com\/canonical\/tdx &amp;&amp; .\/tdx\/system-report.sh### Model specific registers (MSRs)<br \/>MK_TME_ENABLED bit: 1 (expected value: 1)<br \/>SEAM_RR bit: 1 (expected value: 1)<br \/>NUM_TDX_PRIV_KEYS: 20<br \/>SGX_AND_MCHECK_STATUS: 0 (expected value: 0)<br \/>Production platform: Production (expected value: Production)sudo rdmsr 0x5030  # 0 if SGX in production mode<\/p>\n<p><strong>Troubleshooting<\/strong><\/p>\n<p>Troubleshooting while preparing for and running Oasis nodes is a subject on its own, and I will cover the details later. For now, you can check out the relevant steps in the <a href=\"https:\/\/docs.oasis.io\/node\/run-your-node\/troubleshooting#tee\">TEE troubleshooting section<\/a>. It should cover most of the common errors and solutions.<\/p>\n<p>For direct support on specific issues, the Oasis engineering team and other node operators are available in the <a href=\"https:\/\/discord.com\/invite\/BQCxwhT5wS\"><strong>node-operators channel<\/strong><\/a> on the official\u00a0Discord.<\/p>\n<p>This concludes the multi-part series on prerequisites for running Oasis nodes. In subsequent articles, I will go through in detail various types of Oasis nodes that were mentioned briefly in the <a href=\"https:\/\/medium.com\/coinmonks\/your-guide-to-the-nodes-you-can-run-on-oasis-25fcabc5d97e\">introduction article<\/a>.<\/p>\n<p><em>Originally published at <\/em><a href=\"https:\/\/dev.to\/dc600\/your-guide-to-the-prerequisites-for-running-nodes-on-oasis-part-4-tee-setup-5fdp\"><em>https:\/\/dev.to<\/em><\/a><em> on November 20,\u00a02026.<\/em><\/p>\n<p><a href=\"https:\/\/medium.com\/coinmonks\/your-guide-to-the-prerequisites-for-running-nodes-on-oasis-part-4-tee-setup-287f2bb4d405\">Your Guide to the Prerequisites for Running Nodes on Oasis (Part 4): TEE Setup<\/a> was originally published in <a href=\"https:\/\/medium.com\/coinmonks\">Coinmonks<\/a> on Medium, where people are continuing the conversation by highlighting and responding to this story.<\/p>","protected":false},"excerpt":{"rendered":"<p>In a detailed 4-part series, I have been describing the prerequisites that you would need to run Oasis nodes. In the earlier parts, we have covered hardware and cloud providers, stake requirements and node installation, and system configuration steps. In this final part, I will walk you through the process of setting up a Trusted [&hellip;]<\/p>\n","protected":false},"author":0,"featured_media":253688,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[2],"tags":[],"class_list":["post-253687","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-interesting"],"_links":{"self":[{"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/posts\/253687"}],"collection":[{"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"replies":[{"embeddable":true,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=253687"}],"version-history":[{"count":0,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/posts\/253687\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=\/wp\/v2\/media\/253688"}],"wp:attachment":[{"href":"https:\/\/mycryptomania.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=253687"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=253687"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/mycryptomania.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=253687"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}