AHV Networking

AHV NETWORKING

Nutanix Open vSwitch can function both as a software-based network(SDN) switch running within a virtual machine (VM) and AHV hypervisor. Open vSwitch is the default network switch in the Nutanix AHV and Citrix XenServer virtualization platform.

It also supports cloud computing software like OpenStack,oVirt.

Nutanix Open vSwitch(OVS) supports the following features :-

Network Commands

CVM and AHV host tagged VLAN configuration:-

nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0 tag=20"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl list port br0"
nutanix@CVM1$ change_cvm_vlan 20
nutanix@CVM1$ change_cvm_vlan 55
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0 tag=55"

Virtual Machine VLAN configuration with acli(Acropolis command line utility) 

nutanix@cvm1$ acli vm.nic_update <vm_name> <nic mac address> network=<network name>    -- to Manually update MAC to the VM.

nutanix@CVM1~$ acli vm.nic_update <vm name> <vm nic mac address> trunked_networks=<comma separated list of allowed VLAN IDs> update_vlan_trunk_info=true -- to make trunk virtual NIC

nutanix@CVM1~$ acli vm.nic_update <vm name> <vm nic mac address> vlan_mode=kAccess update_vlan_trunk_info=true

nutanix@CVM1$ manage_ovs --bridge_name br0 show_uplinks  -- to list uplinks
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-appctl bond/show br0-up"--
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl show"

nutanix@CVM1$ acli
<acropolis> net.list -----will show net related info
<acropolis> net.list_vms vlan.55

nutanix@CVM1$ manage_ovs show_interfaces   -- will show physical interfaces
nutanix@CVM1$ allssh "manage_ovs --bridge_name <bridge> show_uplinks"
nutanix@CVM1$ manage_ovs --bridge_name <bridge> --interfaces <interfaces> update_uplinks ---Add interfaces to the bridge

nutanix@CVM1$ manage_ovs --bridge_name <bridge> --interfaces <interfaces> --require_link=false update_uplinks -- if We want to update with single interface

Bond configuration for interface of 2x 10 Gb Ports:-

nutanix@CVM1$ manage_ovs --bridge_name br1 create_single_bridge
nutanix@CVM1$ manage_ovs --bridge_name br0 --bond_name br0-up --interfaces 10g update_uplinks
nutanix@CVM1$ manage_ovs --bridge_name br1 --bond_name br1-up --interfaces 1g update_uplinks
nutanix@cvm1$ acli net.create br1_vlan12 vswitch_name=br1 vlan=12

Bond configuration for interface of 4x 10 Gb Ports:-

nutanix@CVM1$ manage_ovs --bridge_name br1 create_single_bridge
nutanix@CVM1$ manage_ovs --bridge_name br2 create_single_bridge
nutanix@CVM1$ manage_ovs --bridge_name br0 --bond_name br0-up --interfaces eth4,eth5 update_uplinks
nutanix@CVM1$ manage_ovs --bridge_name br1 --bond_name br1-up --interfaces eth2,eth3 update_uplinks
nutanix@CVM1$ manage_ovs --bridge_name br2 --bond_name br2-up --interfaces eth0,eth1 update_uplinks
nutanix@cvm1$ acli net.create br1_vlan12 vswitch_name=br1 vlan=12
nutanix@cvm1$ acli net.create br2_vlan14 vswitch_name=br2 vlan=14

Load balance command

nutanix@CVM1$ ssh root@192.168.5.1 "ovs-appctl bond/show"

Set Load balance to active-backup configuration

nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up bond_mode=active-backup"

Set Load balance to balance-slb configuration

nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up bond_mode=balance-slb"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up other_config:bond-rebalance-interval=30000" -- interface rebalancing in respective mac 
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-appctl bond/show br0-up "

set Load balance to balance-tcp and LACP configuration

nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up other_config:lacp-fallback-ab=true"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up other_config:lacp-time=fast"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up lacp=active"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up bond_mode=balance-tcp"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-appctl bond/show br0-up"

Disable LACP

nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up bond_mode=active-backup"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up lacp=off"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up other_config:lacp-fallback-ab=true"

IPMI VLAN – Change IPMI VLAN tagging (ran from AHV host):

ipmitool lan set 1 vlan id <tag>

Examples:

Run from CVM, sets CVM VLAN ID to 96, on all hosts in cluster

nutanix@CVM1$ for g in `svmips`; do ssh $g "nohup sleep 90 && change_cvm_vlan 65” & ;done ;

Run from CVM, sets Hypervisor Host VLAN ID to 96, on all hosts in cluster

nutanix@CVM1$ hostssh ovs-vsctl set port br0 tag=96

Run from CVM, sets IPMI VLAN ID to 96 on all hosts in cluster

nutanix@CVM1$ hostssh ipmitool lan set 1 vlan id 96

Add ports to a bridge

nutanix@CVM1$ ovs-vsctl add-port <bridge> <port>

Remove ports from a bridge (example removes eth2 from br0):

nutanix@CVM1$ ovs-vsctl del-port br0 eth2

Add bond port to bridge

Network Commands 2

CVM and AHV host tagged VLAN configuration:-

nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0 tag=20"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl list port br0"
nutanix@CVM1$ change_cvm_vlan 20
nutanix@CVM1$ change_cvm_vlan 55
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0 tag=55"

VM VLAN configuration

nutanix@cvm1$ acli vm.nic_update <vm_name> <nic mac address> network=<network name>    -- to Manually update MAC to the VM.

nutanix@CVM1~$ acli vm.nic_update <vm name> <vm nic mac address> trunked_networks=<comma separated list of allowed VLAN IDs> update_vlan_trunk_info=true -- to make trunk virtual NIC

nutanix@CVM1~$ acli vm.nic_update <vm name> <vm nic mac address> vlan_mode=kAccess update_vlan_trunk_info=true

nutanix@CVM1$ manage_ovs --bridge_name br0 show_uplinks  -- to list uplinks
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-appctl bond/show br0-up"--
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl show"

nutanix@CVM1$ acli
<acropolis> net.list -----will show net related info
<acropolis> net.list_vms vlan.55

nutanix@CVM1$ manage_ovs show_interfaces   -- will show physical interfaces
nutanix@CVM1$ allssh "manage_ovs --bridge_name <bridge> show_uplinks"
nutanix@CVM1$ manage_ovs --bridge_name <bridge> --interfaces <interfaces> update_uplinks ---Add interfaces to the bridge

nutanix@CVM1$ manage_ovs --bridge_name <bridge> --interfaces <interfaces> --require_link=false update_uplinks -- if We want to update with single interface

Bond configuration for interface of 2x 10 Gb:-

nutanix@CVM1$ manage_ovs --bridge_name br1 create_single_bridge
nutanix@CVM1$ manage_ovs --bridge_name br0 --bond_name br0-up --interfaces 10g update_uplinks
nutanix@CVM1$ manage_ovs --bridge_name br1 --bond_name br1-up --interfaces 1g update_uplinks
nutanix@cvm1$ acli net.create br1_vlan12 vswitch_name=br1 vlan=12

Bond configuration for interface of 4x 10 Gb:-

nutanix@CVM1$ manage_ovs --bridge_name br1 create_single_bridge
nutanix@CVM1$ manage_ovs --bridge_name br2 create_single_bridge
nutanix@CVM1$ manage_ovs --bridge_name br0 --bond_name br0-up --interfaces eth4,eth5 update_uplinks
nutanix@CVM1$ manage_ovs --bridge_name br1 --bond_name br1-up --interfaces eth2,eth3 update_uplinks
nutanix@CVM1$ manage_ovs --bridge_name br2 --bond_name br2-up --interfaces eth0,eth1 update_uplinks
nutanix@cvm1$ acli net.create br1_vlan12 vswitch_name=br1 vlan=12
nutanix@cvm1$ acli net.create br2_vlan14 vswitch_name=br2 vlan=14

Load balance command

nutanix@CVM1$ ssh root@192.168.5.1 "ovs-appctl bond/show"

Set Load balance to active-backup configuration

nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up bond_mode=active-backup"

Set Load balance to balance-slb configuration

nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up bond_mode=balance-slb"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up other_config:bond-rebalance-interval=30000" -- interface rebalancing in respective mac 
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-appctl bond/show br0-up "

set Load balance to balance-tcp and LACP configuration

nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up other_config:lacp-fallback-ab=true"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up other_config:lacp-time=fast"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up lacp=active"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up bond_mode=balance-tcp"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-appctl bond/show br0-up"

Disable LACP

nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up bond_mode=active-backup"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up lacp=off"
nutanix@CVM1$ ssh root@192.168.5.1 "ovs-vsctl set port br0-up other_config:lacp-fallback-ab=true"


IPMI VLAN – Change IPMI VLAN tagging (ran from AHV host):

root@ahvhost1$ ipmitool lan set 1 vlan id <tag>

Examples:

Run from CVM, sets CVM VLAN ID to 96, on all hosts in cluster

nutanix@CVM1$for g in `svmips`; do ssh $g "nohup sleep 90 && change_cvm_vlan 65” & ;done ;

Run from CVM, sets Hypervisor Host VLAN ID to 96, on all hosts in cluster

hostssh ovs-vsctl set port br0 tag=96

Run from CVM, sets IPMI VLAN ID to 96 on all hosts in cluster

nutanix@CVM1$ hostssh ipmitool lan set 1 vlan id 96

Add ports to a bridge

nutanix@CVM1$ ovs-vsctl add-port <bridge> <port>

Remove ports from a bridge (example removes eth2 from br0):

nutanix@CVM1$ ovs-vsctl del-port br0 eth2

Add bond port to bridge

nutanix@CVM1$ ovs-vsctl add-bond <bridge> <port> <iface>

Show bond details

nutanix@CVM1$ ovs-appctl bond/show <bond>

nutanix@CVM1$Set ports to fall back to active-backup mode(Recommended)

nutanix@CVM1$ ovs-vsctl set port bond0 other_config:lacp-fallback-ab=true

nutanix@CVM1$ ovs-vsctl add-bond <bridge> <port> <iface>

Show bond details

nutanix@CVM1$ ovs-appctl bond/show <bond>

Set ports to fall back to active-backup mode(Recommended)

nutanix@CVM1$ ovs-vsctl set port bond0 other_config:lacp-fallback-ab=true

Reference :- OpenvSwitch

How to add or remove nics from OVS Switch