Configuring VSAN networking on a new standard switch
Networking is the glue that holds the VSAN distributed storage nodes together. To permit redundancy, storage access, policy management, and so on, a robust and properly-configured network is key. From within vSphere, VSAN is enabled on a network interface as a service. If you are familiar with creating and enabling vMotion, management, and fault tolerance interfaces, you are already familiar with this process!
This recipe will cover the creation of a new VMkernel network interface in a new vSwitch with previously unused physical network adapters.
Getting ready
- You should be logged in to the vSphere Web Client as an administrator or user, authorized to alter cluster-level settings and networking.
- There should be physical 1GbE or 10GbE interfaces available for use by VSAN.
- VSAN network interfaces should be configured to use a unique IP subnet that is not otherwise in use in the infrastructure. This IP subnet does not need to be routable.
- A separate VLAN for VSAN traffic is optional.
- Your upstream physical switch should be able to handle multicast traffic on the interfaces used by VSAN.
How to do it…
- Navigate to Home | Hosts and Clusters | Datacenter | Cluster | Host | Manage | Networking:
- Click on Add host networking:
- Under Select connection type, choose vmkernel, and then click on Next:
- Under Select target device, choose New standard switch, and then click on Next:
- When prompted, select the applicable physical network adapters and click on OK, and then on Next:
- Under Port properties, enter a label for the new interface, a VLAN ID if appropriate, and check the Virtual SAN traffic box:
- Apply your network address. Ideally, the subnet should be unique (not used for management, vMotion, or fault tolerance):
- Complete the wizard.
- Finally, complete this recipe on all hosts in the VSAN cluster.
How it works…
As VSAN is a unique service at the hypervisor level, the service is bound to a specific VMkernel interface. This is similar behavior to vMotion and fault tolerance tagging. There is no best-effort networking with VSAN. If there are no tagged interfaces, the host will not be able to join the VSAN cluster and will be network isolated. Subsequently, HA configuration will also fail.
There's more…
The VSAN network should be in its own unique IP subnet to help avoid potential network irregularities related to default routes (for example, to prevent routed traffic from using VSAN VMkernel network interfaces to access the gateway). While VSAN interfaces can function in existing subnets and traffic will be restricted to the tagged interface, it is the best practice to give it its own subnet for maximum compatibility and reliability across the entire virtual and physical infrastructures.
VSAN traffic can be shared with other VMkernel interfaces, such as those used for vMotion. This is not ideal, however, as service separation across VMkernel interfaces is preferred. There is no harm with sharing VSAN traffic and non-VSAN traffic across physical 10GbE interfaces (for example, the VSAN VMkernel interface can exist in the same vSwitch/uplink set with non-VSAN interfaces). If you are using 1GbE physical interfaces, those interfaces should be dedicated to VSAN per VMware best practices.
See also
- For more information about physical networking requirements, multicast, LACP/link-aggregation, and so on, please see the Appendix B, Additional VSAN Information specifically, if desired.