The impact of Silverlight on a virtual desktop

A customer has a (virtualized) Citrix XenApp farm scaled for 1500 concurrent users. The environment is based on Windows Server 2003 Standard with Citrix XenApp 5 (migration to XenApp 6.x is scheduled).

A business critital web-application is re-developped and requires Microsoft Silverlight. Before implementing Silverlight the impact should be determined. A big concern is a decrease in the capacity of the farm, number of concurrent users the farm can handle.

To determine the impact of Silverlight I’ve done some tests, including a LoadTest.

Versions used

Silverlight 4 is currently the most widespread version (73,07%, source) and most recent version of Silverlight. So the version installed on the test environment is Silverlight 4.

The customer uses Internet Explorer 8 as there main browser, this won’t be changed for the test.

Quick memory test

A quick test to determine the memory impact of Silverlight is just by simply looking at the memory consumed, which is visible in the task manager.

The base OS uses around 593MB (around 0,5GB). When Internet Explorer 8 is opened with an empy page (about : blank) the consumed memory increases to 646MB. So Internet Explorer 8 uses roughly 53MB. Next a common used news site is opened (https://www.nu.nl) which has some flash ads on it, the memory consumption increases to 672MB, so this site uses around 26MB. The same test is repeated (IE is closed and reopened with an empty page) with a Silverlight site the aggregates news pages (https://www.nuni.nl/), the memory consumption is now 734MB (!). The Silverlight site uses 88MB.

Impact on capacity

First of all I need to determine the baseline of the current situation. How many users can one physical server handle? Since this is a virtualized environment I’m testing four virtual Citrix XenApp servers, all on the same physical server. The maximum concurrent users is determined with a stresstest. The theoretical maximum of users (the point where errors occur) is used instead of the practical maximum (where response times are acceptable).

The stresstest adds users at a fairly quick pase, four sessions each 15 seconds, and goes on until 125 users are on the server. Since this is above the nominal load (roughly 100 is expected) sessions will fail and new sessions will arive, and fail, etc. This has an impact on the load and is therefor only used as a guideline.

The user actions that are simulated is based on an agreed (with the customer) scenario. It roughly contains the following applications : Adobe Reader, Microsoft Excel, Windows Explorer, Internet Explorer, Microsoft Outlook, Microsoft PowerPoint and Microsoft Word.

Both Internet Explorer and Microsoft Outlook remain opened, other applications are closed after use. This creates a ‘base load’ for each user. After each iteration the session sleeps (idle) for a set time.

In the user action scenario Internet Explorer is used. The webpage used is the corporate intranet webpage which is HTML only, which is perfect to create a baseline. In the second test the Intranet webpage is replaced with a Silverlight webpage (https://www.nuni.nl). The Silverlight webpage used is a site that aggregates news from multiple sites.

Based on the stresstest we can conclude that the maximum number of (theoretical) concurrent users is dropped from 100 to 50. Wow!

Note : This is a quick stresstest used as a guideline. Not all HTML only webpages will be replaced with Silverlight (more on that later)

Increase in footprint

Now I’ve established the theoretical maximum number of users I can use that number as the nominal load. I will use the nominal load of 50 users for a LoadTest which will show the impact of Silverlight in the performance footprint.

The same tests are repeated only a slower pase is used, two sessions each 30 seconds. A total of 50 sessions is added, the nominal load. After the 50 sessions are active the scripts iterate infinite. The performance footprint is measured after all users are active (roughly at 16 minutes). I’ve let the script run for another 10 minutes and looked at the performance metrics of both tests.

Since there are four logical servers (virtual) on one physical server, each server gets 12 or 13 sessions

The impact is mainly on memory consumption and processor activity, disk and network activity is not noticable. This (of course) depends on the webpage used, a Silverlight page can easily generate more network traffice.

Memory

The base memory used by the OS is around 0,6GB. The baseline (HTML only intranet) with 50 sessions result in 2GB committed memory, an additional 1,4GB. So each session uses 112MB. When the HTML only intranet page is replaced with the Silverlight page (https://www.nu.ni.nl/) the committed memory increases to 3GB, an additional 2,4GB. This means that each session now uses 192MB, an increase of 80MB.

Processor

The operating system, when idle, has nearly no activity. So the processor base usage is around 2%. The baseline with 50 users results in an average usage of 40% but varies from 20 to 60%, roughly 3% per session. When the HTML only intranet page is replaced for the Silverlight page the CPU usage increases to an average of 60% and varies between 40 and 80%, an average of around 4,6% per session..

Summary

So the increase per session for memory (from 112MB to 192MB) is +71% and the increase for processor (from 3% to 4,6%) is +53%. Of course these numbers are indicative, not all websites are equal and the same goes for the environment used.

IF (that’s a captital if) all HTML only websites are replaced with Silverlight, and webpage are used as frequent as in the scenario, the increase in footprint results in a decrease in the total theoretical capacity of 50% (from 100CCU to 50CCU).

Both the StressTest and the LoadTest are executed using the DeNamiK LoadGen.

User experience

The experience of users is very important on a virtual desktop environment since a remoting protocol is used to transfer the content of the screen. The remoting protocol used in ICA since this is a Citrix XenApp 5 environment. There is currently no optimization for Silverlight in Citrix ICA (unlike Adobe Flash).

Note: Citrix is currently redesigning multimedia redirection and server side rendering in HDX, Silverlight will be subject of optimization.

Measuring the user experience is difficult since there are so many factors to consider. Altough I do have a good method to get a good indication, there is no budget (and therefore time) to set this up.

A more pragmatic approach was needed so I checked the experience myself in a test session. Because the farm is configured with the ‘discard queued image that is replaced by another image’ option, images that are refreshed before they are displayed are discarded. This results in a choppy display with animations or movies but also in a very responsive session.

So the user experience (based on responsiveness) is good with Silverlight (or Adobe Flash or …) altough this depends on the configuration and requirements. When the quality of animations or videos are important this is hard to achieve with the current technologies.

Silverlight usage / adoption

To put this research in perspective I need to determine how likely it is that a Silverlight webpage is used instead of a HTML only webpage.

I’ve used the ‘BuiltWith Technology Usage Statistics’ website to determine how many Silverlight webpages are used. “BuiltWith Trends provides weekly updated free information about the most popular technology used on the web across all areas of web technology including but not limited to analytics, advertising, frameworks, ecommerce and website widgets”

The usage of Silverlight is not broad to say the least. Only 0,2% of the top million sites are using Silverlight, an increase of 0,05% since last year. From the top 10,000 sites only 0,06% use Silverlight which is a decrease of 0,04% since last year.

So the chance that a user will be using Silverlight, other than the business critical application, is pretty small.

Conclusion

The impact of Silverlight on a virtual desktop can be high compare to a HTML only webpage. When the quality of animations or videos is important it might be difficult to configure. But it is unlikely that Silverlight webpage are used since only 0,2% of the top million websites are using Silverlight.

In fact, even Microsoft is replacing Silveright with other techniques. Windows SkyDrive recently replaced Silverlight with HTML5 to reach a broader audience (like mobile platofrms).

Ingmar Verheij