Virtual Security Lab: Architecture

Last month I wrote about my aspirations to create virtual security lab for students on campus to use. Well, as of now the lab is up and running! It is comprised of four machines all running dual Xeon, dual-core processors with 12 GBs of ram per. One machine, acting as a file server, has 1 TB of storage on a Raid 10. The others have 500 GB for internal storage. Two machines run ESXi and act as hypervisors, one machine runs Windows 2008 as a management device, and the file server is running openfiler.

These four machines are connected via a 1 Gbps switch and are assigned static addresses. The management machine, using another NIC, DHCPs on the campus network to receive a campus-wide routable address for both academic buildings and student dorms. This machine runs vCenter Server which students (clients) connect to manage the VMs and hypervisors inside the private network. So far everything is running smoothly, with some minor issues.

The first issue we encountered when designing the architecture was the external network visibility. Since all the hardware described above is borrowed, we wanted enough agility to add and remove hypervisor hosts on a per-semester basis. All virtual machine configurations and image data are hosted on the file server. This can be moved from one host to another by importing the configuration (we did not opt to use vMotion, we're not that agile). We did opt to use vCenter Server instead of having vCenter clients connect directly to ESXi. The server middleware enables students to use one host to manage all hypervisors. Thus the addition of a management machine.

VMWare's documentation recommends installing the vCenter Server service on a virtual machine, but separating the ESXi hosts from the external network was favorable. This way we could minimize network configuration complexity by having vCenter Server (the contact point) running on the machine with a public facing network connection. Now we came to our second issue, centralized authentication. Openfiler and vCenter Server both support authenticating against an Active Directory. Since we already had a Windows 2008 management machine running vCenter Server, it seemed best to also use the machine as a domain controller. However VMWare does not support running vCenter Server on a domain controller. Actually their installer halts if it detects the Active Directory Services role enabled.

Luckily, you can use vCenter Server with Active Directory authentication by removing the Directory Services role, installing vCenter Server, then editing a few configuration files. The tutorial can be found here. Essentially:

1. Remove the Directory Services role.
2. Install vCenter Server with the default configuration options. In my case using the option for a bundled SQL server.
3. Reconfigure the Active Directory Application Mode (ADAM) instance.

Open a command prompt (cmd.exe), and stop the VCMSDS service.
net stop VMwareVCMSDS
Configure the Active Directory Domain Services (AD DS) store:
At the domain store configuration prompt type:
activate instance VMwareVCMSDS
LDAP port 3899
SSL port 6369 (though not used)

Then restart the VCMSDS service:
net start VMwareVCMSDS

4. Modify the ldapPort for vCenter.

Open "C:\Users\All Users\VMware\VMware VirtualCenter\instance.cfg"
Change ldapPort="389" to ldapPort="3899"

5. Install the Active Directory Services role
6. Change the VMwareVCMSDS service to log on using the Local System

Open services and find VMwareVCMSDS
Open the properties and change Log On to "Local System account"

The last step (6) is hidden in the original tutorial but in my opinion usually applies to those who are using vCenter in such an unsupported manner. Now that the vCenter Server was using Active Directory for authentication and user/group permissions. We went ahead and started configuring virtual machines. The last issue with constructing the architecture was trying to use display consoles on the virtual machines.

The display console, VMWare's MKS, establishes a connection from the vCenter client machine directly to the ESXi host over TCP port 903. Unfortunately, as myself as a few others have found, this is not intuitive. Many users try to implement similar architectures on their home network by utilizing a setup where the vCenter is in the DMZ. Since the ESXi hosts are not routable from the client, a display console is not possible. The error message is "Unable to connect to the MKS" followed by a no such host or port unreachable. Now we had to make a choice, whether to register the ESXi hosts on the campus network, or setup a proxy for clients to use.

After some reading I found that the vCenter client does not support SOCKS 5 proxies very well. So our solution was to implement a VPN. Well it's a good thing we chose a configuration using a Windows 2008 management host. By adding the Network Policy and Access Services role we had a VPN server up and running in minutes.

At the end of it all the virtual security lab should be up and running soon, running on an architecture including: 1 openfiler file server, 2 ESXi hypervisor hosts, and a Windows 2008 acting as a VPN server, vCenter Server, and domain controller.