Musings of Rodos

VMware have released a new searchable HCL system that makes it much easier to check for compatibility. What does it look like and how do you use it? Read on.

Here is the URL
http://www.vmware.com/resources/compatibility/search.php
and the following image is an example search.



What do we have here? Lets say you have a BOM for a system, it includes a new card you are not familiar with, a NC360T. Enter that into the keyword search on the IOs tab. Great news, its supported in just about all versions, any they are all listed in front of you, could not be easier!

The page comes back with a number of components in the results. The first is a categorization based on partners. Lets say you type in something generic, this lets you quickly filter down to a particular vendor or subset, excellent feature.

For the results many of the details are hyperlinks. I have shown an exploded view of what it looks like when you click on a element. The opening page shows the specific details of that item.

This is a great new feature and is going to make our job so much easier. Do you self a favor and go and have a play with it, then create a bookmark!

Rodos

Many of the WAN acceleration vendors (Cisco, Riverbed, Expand, Packeteer, Cisco WANscaler, Exinda) are sprouting some amazing statistics for improvement of VDI performance over a WAN. However are these claims realistic? How does one navigate the VDI and WAN acceleration space?

Here is a claim from one vendor. 

Expand’s VDI solution can can provide acceleration by an average of 300% with peaks of up to 1000% for virtual desktop user traffic.

Is this realistic? Sounds like a lot of sales and marketing to me. I remember reading a white paper from one vendor sprouting their lead in the acceleration of VDI, which consisted of the argument that by reducing the bandwidth of the other protocols over the link VDI would magically get faster. Whilst true, if that’s all there is it’s not really an improvement on RDP is it? I am sure this was Riverbed but I can’t find the paper and its not on their website anymore. Instead on the page about virtualisation they talk about ACE in the desktop space, good grief, get with the program, no mention of VDI at all.

So how do we make some sense of this space?

Firstly there are two areas that need addressing, RDP and printing.

Lets start with RDP, it’s a protocol that could do with some assistance over the WAN.

RDP as a protocol is chatty, by which I mean there are lots of smaller size packets which go back and forth, rather than a large data steam that goes generally in one direction like a file transfer. This makes RDP more affected by latency and packet loss. It needs some bandwidth but not huge amounts for normal workloads. As a guide I always use 30k to 100k bps per user with 150ms or lower latency. The bandwidth is going to depend on screen size, colour depth and the type of work you are doing. Also remember that as you add users its not a simple multiplier of bandwidth required, as you are more concerned about concurrency than number of users, not everyone is banging on their keyboards and refreshing their screens constantly or at the same time.

The WAN acceleration providers have three basic techniques for improving RDP itself.

1. Improving the delivery and flow of TCP/IP. Riverbed call this Transport Streamlining and Cisco call it TCP Flow Optimization (TFO). Through lots of clever tricks at the TCP/IP layer to minimize effects of latency, fill packets, adjust window sizes and improve startup times these techniques can have a good impact on perceived and measured performance. The good thing here is that this occurs irrespective of the application protocol being transported, but different protocols will give different results.
2. Compression. All the vendors use compression to reduce the amount of traffic. However the important point here is that if the traffic is already compressed or if it’s encrypted the effectiveness of this compression may be negligible. 
3. Prioritization. If there are different protocols or even different sources or destinations, certain traffic can be prioritized over others. By jumping the queue time sensitive protocols such as RDP can show improved performance from the user perspective. However it’s important to remember that if the majority of traffic is all the same priority, improvements are not going to be achieved. If all the traffic over the WAN link is now just RDP traffic, it’s hard to make any improvements via prioritization.

There is another technique that the vendors sprout, which is application protocol specific acceleration, such as for CIFS or HTTP. By understanding the protocol and placing in optimizations to handle it specifically, dramatic gains can sometimes be achieved. However I have yet to see a vendor who when you dig deep, actually has techniques for improving RDP specific to its protocol. For RDP, it always falls back to the above categories of generic improvements around flow, compression and prioritization.

So how can this be put to advantage in VDI deployments? Here are my tips to give your WAN accelerator every chance at success.

• If you do have a mix of other traffic over the same link you may get some good results. With other traffic bandwidth reduced and prioritization applied you should see some improvement in your RDP traffic.
  
• Remove encryption from the RDP sessions. For your XP desktops you want to be installing the Microsoft patch http://support.microsoft.com/KB/956072 to allow the registry changes for turning the encryption level of RDP down or off. Of course you will need to take into account the security considerations and you may use some techniques in your WAN accelerator to add some encryption back in (after the fact). However removing the encryption allows for much better compression and caching of the data.
  
• Turn off compression of the RDP sessions. This is described on page 152 of the View Manager 3 manual, the setting is “Enable compression” and it’s enabled by default.
  

Now let’s turn our minds to printing.

Moving to server based computing and therefore moving print jobs from the LAN to the WAN often causes headaches. Print jobs have a tendency to become large, which is not usually a problem when there is a 10 Mb or 100Mb link between the users machine and the printer. However put a number of users behind a much smaller link and print jobs start queuing up. The printer can sit their slowly receiving a large job over the small link, and nothing else prints until its all finished. That one small job of a single page right at the end of the queue has to wait and wait, as does the user who printed it. This is nothing new, it has been an issue in the Citrix world for a long time and vendors like ThinPrint have come out with some fantastic technologies to compress and prioritize print jobs, with some amazing results.

Where does WAN acceleration come into the printing issue? Today many WAN acceleration vendors include specific printing acceleration services within their solutions. Yes, VMware View Client for Windows comes with a version of ThinPrint but that is only for printing to a local printer through the client. This can provide part of the solution but most Enterprise customers in my experience will have networked printers in the remote offices run off central or remote servers, and with VDI the print jobs will not go via the client. Therefore you may find that your WAN acceleration solution also solves much of your new printing problems too. It may not provide a Universal Print Driver or have all of the nice features like prioritization (or it may) but it may be good enough.

So don’t be fooled by all of the marketing by the WAN acceleration vendors, they are all attempting to jump on the VDI bandwagon and it is causing some confusion. Try to understand where the improvements are actually happing, are they a primary improvement to the actual protocols or are they simply secondary effects of fixing other data going over the link? How will you be able to integration printing? Armed with some knowledge you should be able to evaluate vendors products based on their presented information plus also be able to conduct some good testing.

I plan to get some tests done with Cisco WAAS and VDI over the Christmas period if I can, I will let you know how I get on.

On a side note, the VMTN Community Round Table this week has Jerry Chen, Senior Director of Enterprise Desktop Virtualization as a guest this week. If Jerry runs out of things to talk about lets see if we can get him to give a view on acceleration of RDP and what he knows or has seen in the market? Maybe even some futures stuff or who VMware is working with.

Rodos

Do you think that VMware have too many locations of important or relevant technical materials? Its starting to feel there are a lot of places where some good content has the potential to be isolated or fragmented.

Here are some of the places that I know of just off the top of my head which contains content.

1. VMTN. Here you will find community documents but some VMware teams put their content here too, such as the performance group.
2. VMware.com Resources - Technical Resources page. This page lists all of the technical papers, probably the most valuable resource.
3. VMworld Community. This is a new space and vendor and VMware information is starting to appear scattered throughout.
4. VMware Whitepapers page. More general white papers, and this does link to the Technical Resources list.
5. VI:OPS. the Virtual Infrastructure Operations site.
6. Documentation sets.
7. Knowledge base. Certainly a specific type of content thats going to be seperate. 

There are most likely others and I know there are some more on the way. I am sure there is an argument that each of these is targeted to a particular audience and its great to see VMware using the comminity to expand and enhance IP in multiple ways. However it does feel like things are starting to get fragmented and there is certainly a lot to keep your eye on if you want to be on top of the space. 

Can I turn 16TB of storage for 1000 VDI users into 619GB, let me show you how it’s actually done. The release today of VMware View Manager 3 brings to market the long anticipated thin provisioning of storage for virtual desktops. Previewed in 2007 as SVI (Scalable Virtual Images) what does this now released View Composer linked clone technology look like under the hood? How much storage will it actually use?

Here is the diagram presented on page 94 of the View Manager Administration Guide (http://www.vmware.com/pdf/viewmanager3_admin_guide.pdf).



This diagram as presented is a conceptual view of the storage. The important logical elements to note here are

• Parent VM. This is the standard virtual machine you use to create and maintain your various versions of your image. It can have various versions as different snapshots.


• Replica. The replica is a copy of a specific version of the Parent VM. That is its one of your snapshots states of the Parent. The key thing here is that the disk in the replica is a thin provisioned copy of the parent disk and snapshot.


• Clones. The diagram shows two clones. The clones are an instance of a replica for a particular VM. For its disk the clone uses the thin provisioned disk in the replica plus its own snapshot to provide the disk. Changes the clone makes to its disk are isolated in the clones snapshot and the replica disk remains untouched and shared by all the clones.

The diagram in the manual is not a great representation, here is my own one that adds some needed detail. Of course there is more complexity, but we can handle that.



What are we looking at here.

• Each box is a directory in your Datastore. You have one directory for your Parent VM (what I have labeled base image), one for each replica, a special one called a source and one for each clone (which I have labeled user).


• The Parent VM (base image) in blue is your standard VM. Notice that the C: disk is thick provisioned as is normal in ESX. A 15G disk will consume 15G in your base image. Then you have your snapshots. Notice that the C: and snapshot 0002 have been combined logically, as this is our view of the disk from the VM.


• Using the Add Desktop wizard in the View Administrator you can create a pool of desktops based on a snapshot from a ParentVM. As part of the process you have to choose a VM and one of its snapshots. When this is done a unique replica is created. This process is marked as (1) on the diagram. Here a copy of the machine is performed, into a new directory however the disk is thin provisioned. If our original disk was 15G yet only 2G was consumed, the disk in the replica will only by 2G. This process can take a short period of time as the data copies, but it is a once off process. This thin provisioned disk is the master disk that all of the clone VMs will use as their base. You can make changes to the parent VM, and the replica can not be harmed.


• What is not shown in the documentation is that a source directory is also created. This source directory is unique to the replica and contains all of the files required to make a clone. These files are essentially your standard VM files with an empty snapshot of the disk in the replica. It is my thinking that clones are created by making copies of the files in this directory. This is why the cloning process is very very fast, all of the work in the background is mostly done. My testing shows under 60 seconds to deploy a new clone. Again this creation of the source directory is a once off process.


• The clone (labeled user) directories are created once for each VM in the pool as required/directed by your pool configuration. The directory name is based on the naming convention given at pool creation. Here we have the files required to run the VM instance. The two important files are firstly the snapshot file which is based of the thin provisioned disk in the replica directory. This is where all of the writes for the VM will be stored, so this file will grow over time (until you Recomposition, Refresh or Rebalance the VM). The diagram tries to shows how the C: drive is made of the combination of the thin disk from the replica directory and the local snapshot file in the clone machine. A separate thin provisioned disk is also created for the user D: drive. This is where the quickprep and user data is stored. This user D drive will grow over time as data is put there, it can’t cant be shrunk.

There you have it, the storage layout of View Composer. What does it look like in reality? Here are some screen snippets.


Datastore Directories.

Here is the directories in the Datastore. You can see there is one Parent VM (XP_VMMAST), one replica directory with its matching source directory, and 3 clone directories XPROD 1 through 3.


Parent VM directory files

These are the files in the parent VM, all the usual suspects and the disk is 15G thick provisioned.


Replica directory files

These are the files in the replica directory. Notice the disk has shrunk to only 2G as its thin provisioned and there is our snaptshot which does not really get used.


Source directory files

These are the files in the source directory. They are pristine and clean, ready for use as a clone. Notice the vmdk file, its based on the replica name, a special kind of snapshot.



Clone VM directory files

These are the files in the actual VM clone directory, one directory for each provisioned desktop. Notice that the vmdk file has grown. This is the growth after booting windows the first time and letting it settle, 50M. Notice there are two more files here, one is the user D disk, which is persistent but thin provisioned, its grown to 23M in size. There is also a vswp file as the machine is booted, otherwise if suspended it would be the vmss.

There you have it. For this test of a 15G machine with just over 2.1G used, what would the storage look like for 1000 users. We will leave the user space aside, we need to cater for that either way. We just want to compare the old method with the new View Composer.

Parent VM is 16G.

Replica and source is 2.1G

1000 machines including swap is 600G

Grand storage space for 1000 users is 619GB.

Compare that to one week ago when it was 16TB, that’s some saving. Of course these figures a little extreme, we now have 1000 users running off a single 2.1G thin provisioned disk, its going to need a lot of spindles to deliver the IOPS required.

Exciting times. We are all going to see how View plays out over the next six months. There is some great architectural work to be done in designing for implementation.

Rodos (Rod Haywood)
Senior Consulting Architect - Virtualisation
Alphawest Services, Sydney Australia

VMware admins are always looking for quick reference to whitepapers and best practices. How do you do that?

Well there are some great sites which list links such as VMware-land. However the most popular document in VMTN (for non desktop products) is the "VMware Technical Resource Documents Listing" at http://communities.vmware.com/docs/DOC-2590

In a single page you will find the title, abstract and link to the PDF for every VMware published technical note. There are currently 195 documents listed.

The source for the listing is the VMware technical resources document list, however this has no usable search function, hence why this alternate listing was originally created. The excellent thing about the document list page is within seconds you can quick search with your browser for any keyword. Go on give it a try, jump over and search for a keyword that interests you. You may be surprised to find a paper that you never knew existed.

In the 11 months since I created this page it has had over 10,000 hits. This is dwarfed by some of the Fusion documents, one of which is over 33,000. VMware, its probably time to improved the search facility of the technical resource list. It would be great to be able to search within the documents, until then, at least we have a usable alternative.

Rodos