Deploying ESXi Host Using Auto-Deploy Server

The idea of this method isn't installing ESXi hypervisor on physical host (either local drive, boot from SAN, or USB). Instead, you will have Auto-Deploy (AD) server which is hosting the hypervisor image. The host will request the image from the AD server which is then transferred over the network and loaded into host RAM. Accordingly the host will be running ESXi hypervisor without installing it on any of its disks. Every time the host reboots, the hypervisor will be loaded to its memory from AD.

In large deployments this is useful since you are having centralized location for hosting all ESXi images and distributing them among physical hosts based on set of rules. Let's examine it in more details.

Note: In case AD server isn't available, ESXi hosts won't boot.

Prerequisites:

1. DHCP server in order to assign IPs to hosts. For static IPs requirement, you may use DHCP reservations.

2. TFTP server which is hosting the PXE boot file. PXE Boot File contains a script which will run on the host to gather information about it and initiate HTTP request to AD server to get image profile based on its parameters

3. Hypervisor image loaded into AD server which is called Image Profile.
4. Rules defined in AD server to identify which image is assigned to which host which are called Deployment Rules.
5. PowerCLI which is used to create deployment rules as well as image profiles (can be default or customized ones).

Important: Once PowerCLI is installed, you will need to set the execution policy to “remotesigned” if you haven’t done so already. Start PowerCLI prompt and type Set-ExecutionPolicy RemoteSigned

Here is a summary of the process of how it works:

1. When the physical server boots, the server starts a PXE boot sequence. The DHCP server assigns an IP address to the host and provides the IP address of the TFTP server as well as PXE boot filename to download.

2. The host contacts the TFTP server and downloads the specified filename, which contains the gPXE boot file and a gPXE configuration file.

3. gPXE executes; this causes the host to make an HTTP boot request to the Auto Deploy server. This request includes information about the host, the host hardware, and host network information. This information is written to the physical server console when gPXE is executing similar to below snapshot.

Note: The URL of AD server where HTTP request is initiated can be found using vSphere Client by navigating to Inventory > Administration > Auto Deploy.

4. Based on the information passed to AD from gPXE, AD server matches the host against a deployment rule and assigns the correct image profile. The AD server then streams the assigned ESXi image across the network to the physical host.

Step-by-Step Configuration

1. Install AD server

This can run in separate windows server, vCenter windows based server, or vCSA (AD is already installed but you need to enable it only). The installation is achieved by inserting vCenter DVD. During the installation process here are the main parameters to be defined.

- Point AD to vCenter server to register with. It can be verified by navigating to Inventory > Administration > Auto Deploy.

- Define the max size of AD repository which is the place where image profiles and deployment rules are installed  

2. Configure TFTP and DHCP servers

I am not covering this section since there are many ways to get DHCP and TFTP servers. However, there are three main points after completion:

You need to load the PXE Boot File into TFTP server root directory to be provided to host. The PXE Boot File can be obtained as follow:

a. From vCenter console navigate to Inventory > Administration > Auto Deploy > Download TFTP Boot Zip.

b. Extract the contents of the zip file into tftp root directory

c. PXE Boot File name is undionly.kpxe.vmw-hardwired.

Note: All other contents of the zip file should be kept in tftp root directory

You need to configure option 66 in DHCP server to point to TFTP server. The standard name of this option is Next-Server. Also, you need to configure option 67 to point to PXE Boot File name. The standard name of this option is Boot-Filename.  

You need to make sure that you have connectivity between DHCP IP assigned to physical host and NIC IP used to identify AD server over the network

3. Create Image Profile

We mentioned that image profile is the hypervisor image that will be transferred from AD to host.

One point I would like to highlight is that ESXi Installable or Embedded ISO files aren't similar to image profiles. The iso files are bootable images which will run ESXi installer that will install ESXi image on disk. Image profile is similar to ESXi image installed on disk.

Therefore we need to manually create this profile using PowerCLI. There are three different terminologies which we need to understand.

• VIB: A VIB is an ESXi software package. It can be bug fixes, drivers, CIM providers, and applications that extend the ESXi platform.
• Image Profile: An image profile defines an ESXi image and consists of VIBs. An image profile always includes a base VIB, and might include more VIBs.
• Software Depot: A software depot is a collection of VIBs and image profiles. The software depot is a hierarchy of files and folders and can be available through an HTTP URL (online depot) or a ZIP file (offline depot). VMware online depot is https://hostupdate.vmware.com/software/VUM/PRODUCTION/main/vmw-depot-index.xml

Note: ESXi 5.0 U1 doesn't have offline software depot.

a. Start PowerCLI and connect to vCenter server using the command Connect-VIServer <srvIP>

b. Add a software depot using the command Add-EsxSoftwareDepot. E.g.

You may add more than one software depot. In fact you need to add all depots which you will be using to create your profile (including the custom VIBs which will be added to the profile)

In our example we have added two software depots online (VMware depot) and offline (QLogic Network driver).

Note: For offline depot, you need to download it in zip format. Next step will be unpack it and browse inside. You will see another zip file, vib file, and other folders. This inside zip file should be used as the software depot.

c. You may list the packages available in the depot using the command Get-EsxSoftwarePackage | select Name,Version,ReleaseDate | sort ReleaseDate

d. You may list the image profiles available in the depot using the command Get-EsxImageProfile | Sort-Object "ModifiedTime" -Descending | format-table -property Name,CreationTime

e. You need to copy one of the existing profiles in order to create your own profile to be used for your hosts.

new-esximageprofile -cloneprofile "existing_profile_name" -name "new_custom_name"

Note: The profile copied in our example is the one having ESXi 5.0.0 image and named as ESXi-5.0.0-20120404001-standard. You can verify the name using VMware download patches website.

f. The next step will be adding VIB packages to your profile. This will be your new custom profile. In our example we are adding QLogic Network driver to be part of our new profile. In case the package already exist in the profile, it won't be added and error will popup.

add-esxsoftwarepackage -imageprofile <profile_name> <package_name>

Note: Package name should include the initial prefix only. E.g for package net-qlcnic-5.0.736-1OEM.500.0.0.472560.x86_64.vib, the package name is net-qlcnic.

g. To make sure that all new packages are added to the profile, you can compare the new profile with the original one.

compare-esximageprofile -comparisonprofile "<modified_profile>" -referenceprofile "<original_profile>"

h. You can export your custom profile to be used later as software depot for further customization with future hosts.

export-esximageprofile -imageprofile "<Profile_Name>" -exporttobundle -filepath "<Location+Name>"

Note: Another reason to export is that once you exist your PowerCLI session, all custom profiles will be lost. By exporting them you can add them back to new PowerCLI sessions as software depots. 

4. Create Deployment Rules

As we mentioned, those rules will define the mapping between image profiles and ESXi hosts. Once the rules are applied to vCenter, the contents of image profile are copied to AD server in order to be used by ESXi host.

Note: You will notice that Cache Space In-Use increased if you navigate to Inventory > Administration > Auto Deploy since the contents of image profile are now uploaded.

a. Use New-DeployRule command to create new rule that map a host based on specific criteria to image profile.

New-DeployRule –Name “Img_Rule “ –Item “My_Custom_Profile” –Pattern “vendor=Cisco”, “ipv4=10.1.1.225,10.1.1.250”

In the above example the matching criteria's are Vendor string contains Cisco and IP address is either 10.1.1.255 or 10.1.1.250.

b. Create a deployment rule that assigns the ESXi host to a cluster within vCenter Server which has AD server registered.

New-DeployRule –Name “Default_Cluster” –Item “Cluster-1” -AllHosts

c. Apply the rules

Add-DeployRule Img_Rule

Add-DeployRule Default_Cluster

d. As soon as you add the deployment rules to the working rule set, vSphere Auto Deploy will, if necessary, start uploading VIBs to the Auto Deploy server in order to satisfy the rules you’ve defined.

e. Verify that these rules have been added to the working rule set with the Get-DeployRuleSet command.

Drawbacks

1. The image profile doesn’t contain any ESXi configuration state information, such as virtual switches, security settings, advanced parameters, and so forth. Host profiles are used to store this configuration state information in vCenter Server and pass that configuration information down to a host automatically. You can use a deployment rule to assign a host profile, or you can assign a host profile to a cluster and then use a deployment rule to join hosts to a cluster.
2. State information such as log files, generated private keys, and so forth is stored in host memory and is lost during a reboot. Therefore, you need to have extra configuration to save those details into external locations.