Implementing vCloud Director 1.5.1 and Cisco Nexus 1000v 1.5

This past week I spent a few days at the VCE Marlboro offices doing some lab validation. One of the topics that comes up on a regular basis is the integration of the Nexus 1000v in the Vblock Platform with the constraints of VMware's vCloud Director integration. VMware has released a patch known as vCloud Director 1.5.1 that doesn't allude to any new features of the 1000v product. My goal for the week was to see if VCD-NI and VLAN backed network pools could be automatically provisioned by vCloud Director.

The first test was to try and validate vCloud Director 1.5.1 with vShield Manager 5.0.1 and the current Vblock Certification Matrix 2.5.3 that uses Cisco Nexus 1000v 4.2(1)SV1(5.1), which is the supported version of Nexus 1000v that can do VXLAN among other things. The VCE 2.5.3 certification matrix also uses VMware vSphere Hypervisor ESXi 5.0 Builds 474610, 504890, 515841 and VMware vCenter Server 5.0 build 455964. The Certification Matrix is a very valuable piece to customers because it validates the software and firmware levels of all the components in a Vblock to guarantee integration and compatibility so the customer doesn't have to assume the risk of testing upgrades and releases, which can be a time consuming task for many environments.

Our test used ESXi build 515841 and vCenter 455964. These builds are normal vSphere 5.0 and NOT Update 1. After getting vCloud Director setup, we need to follow the directions lined out in Configuring Network Segmentation Manager. Of course, nothing is as spoon fed as I would like, so here is a step by step tutorial.


First thing you need to do is of course install the Nexus1000v 4.2(1)SV1(5.1). I built a tutorial called Standing Up The Cisco Nexus 1000v In Less Than 10 Minutes but that has quickly been outdated. I will create a new post soon detailing the steps of the new 1000v build because there is no longer a GUI part of the installation. Once the 1000v has been installed on the hosts used as vCloud provider vDC consumption

Step 0. VLAN Creation and trunking. During the setup of your Nexus 1000v, you should have added all the VLANs necessary for communication. You will need VLANs for External Portgroups, a VLAN(s) for VXLAN/VCDNI segmentation, or a few VLANs for VLAN Backed Network Pools.
n1000v# vlan 100
n1000v (vlan)# name External
n1000v (vlan)# vlan 150
n1000v (vlan)# name VXLAN-VCDNI
n1000v (vlan)# vlan 180-250 <- For VLAN backed Network Pools
n1000v# port-profile type vethernet External-100
n1000v# vmware port-group
n1000v# switchport mode access
n1000v# switchport access vlan 100
n1000v# port-binding ephemeral
n1000v# no shut
n1000v# state enabled
n1000v# port-profile type ethernet DATA-UPLINK
n1000v# switchport trunk allow vlan all (or the VLANs specified for trunking up to your switches)
n1000v# system vlan 100-250 (or whatever you specified for all your VLANs)

you can finish the rest for your ethernet uplinks...

Step 1. Turn on the features. The guide above only tell you about network-segmentation-manger, but you need the segmentation feature to allow vCloud to create bridged networks:
n1000v# conf t
n1000v# feature network-segmentation-manager
n1000v# feature segmentation

Step 2. Create a new organization and get the UUID from vCloud Director. Organization creation is simple, but getting the UUID can be tricky. You can get them through the vCloud API, PowerCLI, or through the vCloud Web Interface. The vCloud web interface is pretty simple for retrieving the UUID. From the admin portal, go to the Manage & Administrator tab, click on the Organizations button in the left pane, then click on the organization and a new tab is created within vCloud. Now if you look in the address bar, the UUID is the string of characters at the end. Copy that into your clipboard. (thanks to @jakerobinson for finding this). In this example my organization is IT and my UUID is c5e6d487-da66-42fd-b0f6-f885ea9ad13a.

Step 3: Create a port-profile and segmentation policy for your organization within the vCloud. You will see that steps 2-3 must be done for EVERY tenant/organization that is created for vCloud Director. Can anyone beg for automation? One thing to note here is that you can choose 2 types of Network Pools that need to be specified within the 1000v. Segmentation = VCD-NI Network Pools and VLAN = VLAN Backed Network Pools. I will show an example of each.

VXLAN/VCD-NI Example
n1000v# conf t
n1000v# port-profile type vethernet IT_Segmentation_Profile
n1000v# no shutdown
n1000v# state enabled
n1000v# conf t
n1000v# network-segment policy IT-Policy-VXLAN
n1000v# description "VCDNI and VXLAN segmentation for the IT Organization"
n1000v# type segmentation
n1000v# id c5e6d487-da66-42fd-b0f6-f885ea9ad13a
n1000v# import port-profile IT_Segmentation_Profile


VLAN Backed Example
n1000v# conf t
n1000v# port-profile type vethernet IT_VLANSegmented_Profile
n1000v# no shutdown
n1000v# state enabled
n1000v# conf t
n1000v# network-segment policy IT-Policy-VLANsegmented
n1000v# description "VLAN Backed segmentation for the IT Organization"
n1000v# type VLAN
n1000v# id c5e6d487-da66-42fd-b0f6-f885ea9ad13a
n1000v# import port-profile IT_VLANSegmented_Profile

Step 4: Configure vShield Manager. Go to the Web GUI for vShield Manager (https://ip_address) and enter your username and password. Go to the Settings tab and click on network. From there you will go to the Settings & Reports button on the left hand side which will bring you to the configuration tab. Within the configuration tab, go to the Networking page. From here we need to set 2 different things.

Create a Segment Pool ID Range. It can be a range between 4098-16000 I believe. No real rhyme or reason but I choose 8000-12000. I also chose a multicast address range of 224.10.0.1-224.10.0.150. Disclaimer: I'm going to be honest. I don't know the first thing about VXLAN so multicast is new to me. If you know how these values effect design criteria, I would be glad to hear about them. Knowing the amount of IPs that can given out via a multicast range and the amount of Segment Pool IDs would good knowledge to have. Please comment if you would suggest different values!

The second part is to add the Nexus 1000v VSM as an External Switch Provider. Click on the button that says "Add Switch Provider" and enter the following information. Add a notable name to it, then type in the API interface for it to talk to which is https://IP_of_VSM/n1k/services/NSM, then type your user credentials and click OK. A pop-up box will come up asking to accept the RSA key to establish communication. Click OK and you should have a green box.

Step 5. Create our External Networks and Network Pools. From here on out, it's business as usual with vCloud Director. This should be something you are familiar with if you have ever used vCloud Director. One thing to note is that using the Nexus 1000v doesn't populate good information within vCloud (hence -1 as a VLAN).

Step 6. Create your Organization Networks, add a VM to the Catalog, and start deploying vApps. As you can see,  vShield Edge devices are being deployed and new portgroups are being created on the 1000v.

If you go back into your Cisco Nexus 1000v VSM, you can see that there are bridged VXLAN networks being created by vCloud Director and port-profiles being created for the isolated organization networks.

This was a fun challenge to get working because there is little documentaton available, and the documentation that is available is a bit mis-leading.

I did not test anything related to scale or performance, just functionality.

As of 3/30/12, I have been told that there are still some remaining bugs when running 1000v and vCloud Director and there will be a 1000v Patch coming out at some point in April, please use this knowledge with caution.