Stackvora.tech

Migrate a VM with vMotion

Detailed guide on live migrating VMware VMs using vMotion, covering prerequisites, network configuration, migration types, and common issues for IT professionals.

Live migration of virtual machines without service interruption is a cornerstone of modern virtualization. VMware vMotion enables this capability, allowing workloads to move between physical hosts while maintaining continuous operation. This article provides an in-depth guide to performing vMotion migrations within a vSphere environment, covering prerequisites, configuration, and practical execution.

vMotion facilitates various scenarios, including hardware maintenance, load balancing, and disaster recovery preparation, by abstracting the VM from its underlying physical server. Understanding its mechanics and requirements is crucial for any administrator managing a vSphere infrastructure.

vMotion Prerequisites and Requirements

Successful vMotion operations depend on a well-configured vSphere environment. Several key components and settings must be in place before initiating a migration.

Shared Storage

For a standard vMotion (compute-only migration), the source and destination ESXi hosts must have access to the same shared storage. This storage can be Fibre Channel (FC), iSCSI, NFS, or vSAN. The VM's virtual disk files (.vmdk) must reside on this shared datastore, accessible to both hosts. If shared storage is not available, a Storage vMotion (or a combined vMotion and Storage vMotion) would be required, which moves the VM's disk files in addition to its compute state.

Networking Configuration

Dedicated network infrastructure for vMotion is highly recommended for performance and security. Each ESXi host participating in vMotion must have a VMkernel adapter configured for vMotion traffic.

  • VMkernel Adapter: Create or modify a VMkernel adapter on each host. Enable the 'vMotion' service on this adapter.
  • IP Addressing: These VMkernel adapters should be on the same IP subnet or a routed network segment with sufficient bandwidth. For best performance, dedicate a separate physical NIC (or a pair for redundancy) to this vMotion VMkernel port.
  • Bandwidth: VMware recommends at least a 10 GbE network for vMotion, especially for large VMs or concurrent migrations. While 1 GbE can work, it will be significantly slower and more impactful on host resources.
  • MTU: Consider enabling Jumbo Frames (MTU 9000) on the vMotion network path end-to-end (ESXi VMkernel adapter, physical switch ports) to improve performance, though this requires careful configuration to avoid fragmentation.

Example of configuring a vMotion VMkernel adapter using esxcli:

# Check existing VMkernel adapters
esxcli network ip interface list

# Create a new vMotion VMkernel adapter on vSwitch0 using vmnic1
esxcli network ip interface add -i vmk1 -p "Management Network" -M 9000
# Note: "Management Network" here refers to the port group. You might need to create a dedicated port group.
# For a dedicated vSwitch:
esxcli network vswitch standard add -v vSwitch1
esxcli network vswitch standard portgroup add -p "vMotion-PG" -v vSwitch1
esxcli network ip interface add -i vmk1 -p "vMotion-PG"

# Assign IP address to vmk1
esxcli network ip interface ipv4 set -i vmk1 -t static -I 172.16.1.10 -N 255.255.255.0 -g 172.16.1.1

# Enable vMotion on vmk1
esxcli network ip interface set -i vmk1 -e true -s vmotion

# Verify configuration
esxcli network ip interface list -i vmk1

Ensure that firewall rules on the ESXi hosts allow vMotion traffic (TCP port 8000 by default, though vMotion uses various ephemeral ports for data transfer). The vSphere client automatically configures firewall rules when enabling vMotion on a VMkernel adapter.

CPU Compatibility

Source and destination hosts must have compatible CPUs. They don't need to be identical, but they must belong to the same CPU vendor (Intel to Intel, AMD to AMD) and have instruction sets that are compatible enough for the VM to run on either host without issue. EVC (Enhanced vMotion Compatibility) clusters are designed to simplify this by presenting a common, baseline CPU feature set to all VMs within the cluster, enabling vMotion between hosts with different CPU generations.

Resource Availability

The destination host must have sufficient CPU, memory, and network resources available to accommodate the migrating VM.

Performing a vMotion Migration

The process for initiating a vMotion migration is straightforward using the vSphere Client.

  1. Log in to the vSphere Client.
  2. Navigate to the Virtual Machine you wish to migrate in the Inventory.
  3. Right-click the Virtual Machine and select Migrate... (or select Actions > Migrate).
  4. The Migrate wizard will appear. You will typically be presented with three options:
    • Change compute resource only: This is a standard vMotion. The VM's configuration and disk files remain on the same datastore, but its running state moves to a different host. Requires shared storage.
    • Change storage only: This is a Storage vMotion. The VM remains on the same host, but its disk files move to a different datastore.
    • Change both compute resource and storage: This is a combined vMotion and Storage vMotion. The VM's running state moves to a new host, and its disk files move to a new datastore. This option does not require shared storage between the hosts.
  5. Select Change compute resource only. Click Next.
  6. Select a compute resource: Choose the destination host or cluster. Ensure the compatibility checks pass. If there are warnings, address them (e.g., CPU incompatibility, network issues). Click Next.
  7. Select networks: If the destination host has different network labels or if you wish to change the VM's network adapter assignment, select the appropriate network for each virtual NIC. Click Next.
  8. Select vMotion priority:
    • Schedule high priority (recommended): Default, allocates more resources to the migration.
    • Schedule as regular priority: Less resource-intensive, but slower. Useful for non-critical VMs or during peak host utilization.
  9. Review your selections on the Ready to Complete page. Click Finish.

The migration process will begin. You can monitor its progress in the Recent Tasks panel of the vSphere Client. During a successful vMotion, the VM remains powered on and accessible throughout the entire process, typically experiencing only a momentary network latency spike of a few milliseconds.

vMotion Network Bandwidth and Performance

vMotion performance is heavily influenced by network bandwidth and latency. High-speed networks (10 GbE or higher) are essential for efficient migrations, especially with larger VMs or multiple concurrent migrations. VMware supports up to eight simultaneous vMotion operations per ESXi host (vSphere 6.0+) and up to four per datastore.

Key factors affecting vMotion performance:

  • Network Speed: 10 GbE vs. 1 GbE makes a significant difference.
  • Jumbo Frames: Can improve throughput on 10 GbE networks but requires end-to-end configuration.
  • Latency: Low latency between source and destination hosts is critical. VMware generally recommends <5 ms for standard vMotion and <10 ms for long-distance vMotion.
  • VM Memory Size: Larger VM memory footprint means more data to transfer.
  • Dirtying Rate: The rate at which the VM's memory pages change during migration. High dirtying rates prolong migration.

For long-distance vMotion (across different subnets or geographical locations), additional requirements apply, such as a high-bandwidth, low-latency link between sites and potentially a Layer 2 extension or routed vMotion network. The maximum round-trip time (RTT) for long-distance vMotion is 150 ms in vSphere 6.0+, though lower is always better.

Common Pitfalls and Troubleshooting

  • Compatibility Errors: The most common issue. Check CPU compatibility (EVC is your friend), network configuration (same subnet/reachable, vMotion enabled on VMkernel), and storage accessibility. The vSphere Client's compatibility checks are usually very informative.
  • Insufficient Resources: The destination host may lack sufficient CPU, memory, or network capacity. Verify resource reservations and available capacity.
  • Network Configuration Issues: Misconfigured VMkernel ports (e.g., vMotion service not enabled), incorrect IP addresses, firewall blocks (host or physical network), or incorrect VLAN tagging. Use vmkping to test connectivity between vMotion VMkernel adapters on source and destination hosts.
  • Shared Storage Not Accessible: Ensure the datastore containing the VM's disks is presented to and properly mounted by both ESXi hosts. Rescan HBAs and storage.
  • Host State: Ensure both ESXi hosts are not in maintenance mode, are connected to vCenter Server, and are not experiencing severe resource contention.
  • Too Many Concurrent Migrations: Exceeding the vMotion limits can lead to failures or severe performance degradation.
  • High Latency/Low Bandwidth: While often resulting in slow migrations rather than outright failures, very poor network conditions can cause timeouts. Monitor network performance.

Back to the knowledge base · Ask the AI assistant