Nutanix is a leading provider of hybrid multi-cloud solutions that help enterprises simplify their IT operations and reduce costs. One of the key advantages of the Nutanix platform is its ability to provide scalable and high-performance enterprise-grade storage capabilities. In recent years, Nutanix has partnered with Rancher to provide a comprehensive solution for managing Kubernetes clusters on Nutanix infrastructure.
The Rancher and Nutanix partnership provide organizations with a powerful platform for deploying and managing their Kubernetes clusters. Rancher’s Kubernetes platform can consume Nutanix AOS storage solutions, including Nutanix Volumes (block storage), Nutanix Files (distributed file storage), and Nutanix Objects (object storage), enabling organizations to store and manage their data efficiently and securely.
In addition to its storage capabilities, Nutanix also offers Nutanix Database Service (NDB), a database-as-a-service (DBaaS) solution that simplifies database management and deployment. With NDB, enterprises can easily deploy, scale, and manage their databases while taking advantage of the high-performance storage capabilities of the Nutanix platform.
Nutanix enhances Kubernetes security with Nutanix Flow, a software-defined networking solution that offers network visibility, micro-segmentation, and application-centric security policies. Integrating Nutanix Flow with Rancher enables fine-grained security controls and protects Kubernetes workloads from threats.
You can use one of two methods to deploy Rancher on Nutanix:
- Rancher running on Nutanix Kubernetes Engine (NKE)
- Rancher running on Rancher Kubernetes (RKE1, RKE2, or K3s)
Software Prerequisites
We used the following software versions for each component:
- Nutanix AOS 6.5.2.5 and later
- Nutanix Prism Central PC.2023.1.0.1
- Nutanix Kubernetes Engine 2.8
- Nutanix CSI Volume Driver 2.6.3
- Nutanix Files 4.3.0.1
- Nutanix Objects 4.1
- Nutanix Flow Network Security 3.1.0
- Nutanix or Rancher Kubernetes Clusters 1.23.16
- Rancher Multi-Cluster Manager 2.7.9
- Nutanix Rancher AHV Node Driver 3.5.0
- Rancher Monitoring 102.0.1+up40.1.2
- Rancher Backup Operator 102.0.1+up3.1.1
- Helm 3.0
Node Operating Systems
If you use NKE, Nutanix manages and provides the underlying node OS, which reduces the administrative effort required to provision Kubernetes clusters by upgrading the NKE node’s OS in a rolling fashion.
Determine which Rancher-supported container node OS and target Rancher Kubernetes platform (RKE1, RKE2, K3s, NKE) to use. We recommend using an OS that is equally certified with both Nutanix AHV and CSI Driver.
Rancher Sizing on Nutanix:
Rancher’s hardware footprint depends on a number of factors, including:
- Size of the managed infrastructure (e.g., node count, cluster count).
- Complexity of the desired access control rules (e.g., RoleBinding object count).
- Number of workloads (e.g., Kubernetes deployments, Fleet deployments).
- Usage patterns (e.g., subset of functionality actively used, frequency of use, number of concurrent users).
The below requirements are subject to typical use assumptions, which include:
- Under 60,000 total Kubernetes resources, per type.
- Up to 120 pods per node.
- Up to 200 CRDs in the upstream (local) cluster.
- Up to 100 CRDs in downstream clusters.
- Up to 50 Fleet deployments.
RKE
The following table lists minimum CPU and memory requirements for each node in the upstream cluster.
Please note that a highly available setup with at least three nodes is required for production.
| Managed Infrastructure Size | Maximum Number of Clusters | Maximum Number of Nodes | vCPUs | RAM |
|---|---|---|---|---|
| Small | 150 | 1500 | 4 | 16 GB |
| Medium | 300 | 3000 | 8 | 32 GB |
| Large (*) | 500 | 5000 | 16 | 64 GB |
(*): Large deployments require that you follow best practices for adequate performance.
RKE2 Kubernetes
The following table lists minimum CPU and memory requirements for each node in the upstream cluster.
Please note that a highly available setup with at least three nodes is required for production.
| Managed Infrastructure Size | Maximum Number of Clusters | Maximum Number of Nodes | vCPUs | RAM |
|---|---|---|---|---|
| Small | 150 | 1500 | 4 | 16 GB |
| Medium | 300 | 3000 | 8 | 32 GB |
| Large (*) | 500 | 5000 | 16 | 64 GB |
(*): Large deployments require that you follow best practices for adequate performance.
K3s Kubernetes
The following table lists minimum CPU and memory requirements for each node in the upstream cluster.
Please note that a highly available setup with at least three nodes is required for production.
| Managed Infrastructure Size | Maximum Number of Clusters | Maximum Number of Nodes | vCPUs | RAM | External Database Host (*) |
|---|---|---|---|---|---|
| Small | 150 | 1500 | 4 | 16 GB | 2 vCPUs, 8 GB + 1000 IOPS |
| Medium | 300 | 3000 | 8 | 32 GB | 4 vCPUs, 16 GB + 2000 IOPS |
| Large (**) | 500 | 5000 | 16 | 64 GB | 8 vCPUs, 32 GB + 4000 IOPS |
(*): External Database Host refers to hosting the K3s cluster data store on an dedicated external host. This is optional. Exact requirements depend on the external data store.
(**): Large deployments require that you follow best practices for adequate performance.
Tips for Running Rancher
A high-availability Kubernetes installation, defined as an installation of Rancher on a Kubernetes cluster with at least three nodes, should be used in any production installation of Rancher, as well as any installation deemed “important.” Multiple Rancher instances running on multiple nodes ensure high availability that cannot be accomplished with a single node environment.
Don’t run other workloads or microservices in the Kubernetes cluster that Rancher is installed on.
Disabling swap, double checking you have full network connectivity between all machines in the cluster, using unique hostnames, MAC addresses, and product_uuids for every node, checking that all correct ports are opened, and deploying with ssd backed etcd.
It’s strongly recommended to have a “staging” or “pre-production” environment of the Kubernetes cluster that Rancher runs on. This environment should mirror your production environment as closely as possible in terms of software and hardware configuration.
You must ensure that the hosts running the Rancher server are able to establish the following network connections:
- To the Nutanix Prism Central API (usually port 9440/TCP).
- To port 22/TCP and 2376/TCP on the created VMs
Setting up VM-VM Anti-Affinity Policies is recommended. These rules allow VMs assigned the etcd and control-plane roles to operate on separate AHV hosts when they are assigned to different node pools. This practice ensures that the failure of a single physical machine does not affect the availability of those planes.
Presales Consultant with more than 10 years of experience, supports our partners by understanding customer needs, presenting and demonstrating our vendors’ products and helping in deal closing.