{"id":2599,"date":"2011-05-23T15:00:00","date_gmt":"2011-05-23T14:00:00","guid":{"rendered":"https:\/\/ingmarverheij.com\/2011\/05\/loadtesting-best-practices-part-1\/"},"modified":"2011-05-23T16:40:35","modified_gmt":"2011-05-23T14:40:35","slug":"loadtesting-best-practices-part-1","status":"publish","type":"post","link":"https:\/\/ingmarverheij.com\/en\/loadtesting-best-practices-part-1\/","title":{"rendered":"Loadtesting best practices – Part 1"},"content":{"rendered":"

\"\"Before I start discussing best practices about loadtesting, let me first tell you what my definition of a loadtest<\/strong> is.<\/p>\n

\u201cTesting a system with representative load, to determine if the nominal load can be handled\u201d.<\/em><\/div>\n

This means that if you start loadtesting you need to know the nominal load and have an expectation of the outcome. You will find that most of the best practices has to do with preparation, not with fancy techniques. An different type of test that is frequently called a loadtest, is a stresstest. A stresstest has a different purpose, as can be read in my definition of a stresstest<\/strong>.<\/p>\n

\u201cTesting a system above its nominal load, with the purpose of determining if the system in the future can handle a bigger load and to find a bottleneck.\u201d<\/em><\/div>\n

LoadTest are commonly used to scale an environment. These environments can be a SBC environment like Citrix XenApp or Microsoft RDS, or a VDI environment. But other environments like file, print or webservers can be loadtested aswell. In fact, if you can create the load by simulating user actions, you probably can perform some sort of loadtest.<\/p>\n

Now let\u2019s start with the best practices. I\u2019ve written these down in the past years while <\/em>I was performing loadtests with the DeNamiK LoadGen<\/strong><\/a>, the best practices apply to (almost) all loadtesting applications. You can say I learned it the hard way. Although there\u2019s nothing wrong with that, you can learn from my mistakes.<\/p>\n

This is the first\u00a0 part, focussing on the \u201cbasics\u201d. Part two will focus on more advanced topics.<\/p>\n

 <\/p>\n

<\/p>\n

1 \u2013 Determine the purpose of the test<\/h5>\n

\"\"Before doing anything, you should determine why you\u2019re performing a test. What type of test<\/strong> are you planning, a load or a stresstest? Do you want to validate a certain result with a pre-defined load, or do you want to find the level where things start to stop functioning properly?<\/p>\n

For each test you should know how much <\/strong>sessions (or users) you\u2019re going to use to simulate the load. 10, 25, 100 or more? The more users, the bigger the impact on the environment and the preparations.<\/p>\n

The moment a session is launched, load is generated. Determine the when<\/strong> and how much <\/strong>sessions you want to launch simultaneously.<\/strong> The number of simultaneous users and the interval determines the load on the target environment, if you\u2019re purpose isn\u2019t to stress the environment you should take it slow. If you\u2019re testing a real-life scenario, determine a scenario that corresponds with real life.<\/p>\n

Create different load profiles<\/strong>. Users can generally be placed in different profiles like \u2018light users\u2019, \u2018medium users\u2019 and \u2018heavy users\u2019. Each profile has a different set of applications, workspeed and\u00a0 distribution.<\/p>\n

For all of the above the same rule applies: consult the customer and adapt the real life parameters as much as possible.<\/span><\/p>\n

<\/span><\/p>\n

2- Create a scenario<\/h5>\n

\"\"After a sessions is launched, load needs to be generated by simulating user actions. The user actions simulated should be determined before creating the script that simulates the actions.<\/p>\n

The user actions simulated should be written down in a scenario <\/strong>which describes a roadmap. The scenario described what applications are used and what actions are executed in that application. For each action the expected outcome, a result on the screen, is written down aswell. A good objective to pursue is that a regular user, with no experience with the application, can execute the same actions without asking what to do next.<\/p>\n

This way the guy that makes the script that simulates user actions won\u2019t have to know the application.<\/p>\n

The best scenario is a scenario that matches the workload of the customer. So consult the customer what applications are used and how the applications are used. Not all applications should be simulated, but it should correspond with the regular workload.<\/p>\n

 <\/p>\n

3 \u2013 Test users<\/h5>\n

\"\"Sessions require users to authenticate. It is recommended to use dedicated test users<\/strong> placed in a group. This way the users can be managed easily.<\/p>\n

Make sure the users exist, are configured and function before you start. If the users do not exist, and you\u2019re in charge of creating them, make sure you have an administrative account. And, know how and where to create the users, what properties needs to be set and where is the profile and home directory stored?<\/p>\n

Test users should all be equal<\/strong>. Since you\u2019re creating an automated test with simulated user actions, you want them to be as much equal as possible.<\/p>\n

 <\/p>\n

4 \u2013 SUT<\/h5>\n

\"\"Usually a test is conducted in an environment which has more than one system involved. Determine all systems <\/strong>that are under test (SUT)<\/strong> and therefore can influence the result or should be monitored.<\/p>\n

Each system that has a connection, and has influence on the end-result, is a system under test. For instance an Active Directory Domain Controller, Fileserver, SQL server, SAN or router. Altough your main purpose can be very simple, like testing a Citrix XenApp farm, each component in the chain can influence the test results.<\/p>\n

Windows machines can be monitored using performance counters, the following performance metrics give a good overview of the system. In case a bottleneck is reached, add more performance counters to gain more insight.<\/p>\n

\r\nMemory\r\n     Available MBytes\r\n     Committed Bytes\r\n     Free System Page\u00a0 Table Entries\r\n     Page Faults\/sec\r\n     Pool Nonpaged Bytes\r\n     Pool Paged Bytes\r\n\r\nNetwork Interface\r\n     Bytes Received\/Sec\r\n     Bytes Sent\/sec\r\n     Bytes Total\/sec\r\n\r\nPhysical Disk\r\n     _Total\\% Disk Time\r\n     _Total\\Current Disk Queue Lenght\r\n\r\nPaging File\r\n     _Total\\%Usage\r\n\r\nProcess\r\n     _Total\\Page File Bytes\r\n\r\nProcessor\r\n     _Total\\% Interrupt Time\r\n     _Total\\% Processor Time\r\n     _Total\\Interrups\/sec\r\n\r\nServer\r\n     Server Sessions\r\n\r\nSystem\r\n     Context Switches\/sec\r\n     Processor Queue Lenght\r\n\r\nTerminal Services\r\n     Active Sessions\r\n<\/pre>\n
 <\/p>\n
5 \u2013 Influences<\/h5>\n
\"\"During a test other processes <\/strong>might be active that can influence <\/strong>you\u2019re test results. That\u2019s fine if this is part of your plan, but not if you didn\u2019t knew. For instance when a back-up process starts during a test, that might influence the speed of the environment.<\/p>\n
Map the activities in the environment and communicate with the customer. If nobody knows they should influence the test, they might do.<\/p>\n
Except for a back-up process you might consider processes that influence the fileserver, databaseserver, SAN, WAN connection etc. etc.<\/p>\n
 <\/p>\n
6 \u2013 Computational vs perceived performance<\/h5>\n
\"\"Looking at the performance of a machine can be done in different ways. Generally these can be divided in two categories: Computational performance and perceived performance.<\/p>\n
Computational performance is based on CPU cycles, available memory, IOPS, network throughput etc. Although these are all valid performance indicators, they are nothing more than an indicator. They describe the performance they theoretically should provide. For example if a workload on a processor is 30%, is that good? Or is it inefficient?<\/p>\n
Tim Mangan described computational performance in his white paper \u201cPerceived Performance\u201d in September 2003:<\/p>\n
\u201cComputational capacity is the total amount of useful work that can be accomplished on a system in a given fixed period of time\u201d<\/em><\/div>\n
Perceived performance is from a different angle, the way users perceive the performance. This the end-result of the complete chain and should IMHO be on the top of your list. Perceived performance can be determined by measuring the responsetimes of user actions.<\/p>\n
Tim Mangan descbied perceived performance in his white paper \u201cPerceived Performance \u2013 Welcome to VDI\u201d in May 2011:<\/p>\n
\u201cA methodology where one analyzes the system with a goal of improving user productivity by focusing on issues that affect the performance as perceived by the users\u201d<\/em><\/div>\n
 <\/p>\n
7 \u2013 Creating bottlenecks<\/h5>\n
\"\"Make sure you\u2019re not creating a bottleneck<\/strong> with your test environment since this influences the outcome. For instance the the machines that hosts the sessions, the LoadBot \/ Loader. Make sure that the number of sessions hosted per machine does not exceed the capacity of the machine. Altough it might sound logical, i\u2019ve seen many cases where the number of sessions on one machine was much more than it can handle. This resulted in higher measurements, even when the target environment had enough capacity.<\/p>\n
The same applies for collecting performance metrics. Each performance metric that is collected requires bandwith, if the monitored system is on a slow WAN connection this might influence the test. I\u2019ve seen a test where the complete WAN connection was required for collecting performance metrics leaving no bandwith for the sessions.<\/p>\n
 <\/p>\n
Before you start a test, scale the LoadBots and determine the required bandwith for collection performance metrics.<\/p>\n
 <\/p>\n
 <\/p>\n
Ingmar Verheij<\/p>","protected":false},"excerpt":{"rendered":"
Before I start discussing best practices about loadtesting, let me first tell you what my definition of a loadtest is. \u201cTesting a system with representative load, to determine if the nominal load can be handled\u201d. This means that if you start loadtesting you need to know the nominal load and have an expectation of the […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"site-container-style":"default","site-container-layout":"default","site-sidebar-layout":"default","disable-article-header":"default","disable-site-header":"default","disable-site-footer":"default","disable-content-area-spacing":"default","footnotes":""},"categories":[302],"tags":[318,316,317],"class_list":["post-2599","post","type-post","status-publish","format-standard","hentry","category-performance-testing","tag-best-practice","tag-loadtest","tag-stresstest"],"_links":{"self":[{"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/posts\/2599","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/comments?post=2599"}],"version-history":[{"count":24,"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/posts\/2599\/revisions"}],"predecessor-version":[{"id":2601,"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/posts\/2599\/revisions\/2601"}],"wp:attachment":[{"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/media?parent=2599"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/categories?post=2599"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/ingmarverheij.com\/en\/wp-json\/wp\/v2\/tags?post=2599"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}