October 03, 2011 - by alexsalkever
This is a guest post written and contributed by Fred Beringer, VP Business Development EMEA at SOASTA. SOASTA provides load and performance testing products as well as solutions available as on-demand cloud services.
Common wisdom used to hold that you can’t load test live applications, and any testing during development wouldn’t accurately reflect real world conditions. So, it’s not surprising that more than 75% of companies today don’t adequately test and validate the scalability of their apps. Everyone knows that load testing is the best way to ensure production systems can handle live traffic, and to identify bottlenecks before they cause catastrophic failure, but the costs and risks often outweighted the benefits... until now.
In partnership with Joyent, Soasta is offering the Soasta CloudTest Lite suite to all Joyent customers for free. Now you can integrate load testing as part of the design process, from development through launch, and load test live apps in production to spot things that will break under heavy load long before your customers get to that point.
Let's take a look at the typical rationales for not load testing. Believe me, we hear these all the time.
Fair enough. Now let's test some of those assumptions with a look at real-world load test exercise experience of a leading eCommerce website in Europe selling the discontinued, but HOT, HP TouchPad. 1000 units at 99 Euros at a 7am sale. This is an Akamai snapshot during the peak. They topped 40k requests/sec (around 800k hits/sec) and had to handle 100 000 visitors in less than 10 minutes. Here are the results of the sales?
This is unfortunately a typical case of customer doing only internal testing, at low volume, and extrapolating results with a finger in the air. And they get sorry when they leave 99,000 screaming at their door. Bad business. Bad reputation. There are no valid reasons to be in that situation today. None. (If you want some additional details about this failure, you can read a more detailed article here.)
If you're scared at this stage, this is normal. After all, you thought load testing in a lab was sufficient. So, as application developers, what can you do to avoid such a nightmare?
Test the performance of your application early and as often as possible
I'm assuming you're doing unit testing at each of your code check-in? You should be able to run continuous performance testing as well (there are no valid reasons that this should not be the case). This is where you'll ensure a solid foundation for your application:
Test as often as possible on your production environment and don't be scared!
It's going to be ok, really. As you know, it is very easy to segment a portion of the live environment during a low-traffic period and allow for testing in this environment. Typically, a separate IP address is configured on a load balancer and servers are moved out of the live pool and placed into the test pool. Sometimes, configurations changes need to be made to the application servers in this cluster to point to new databases and be taken off of other shared components. This is a more costly approach because it requires extra hardware and the associated maintenance overhead. It's also less reliable because you start to deviate from the actual production configuration and you cannot test at true scale. It is still, however, a more realistic test than simply testing in the lab.
Code for testability
A lot of developers are scared to test on production system, even at a low volume, because they don't want to screw up some part of the system. A typical example would be the order placement of an eCommerce application. You definitely don't want to place an actual order! One benefit of being able to control the entire payload of a request with a product such as CloudTest Lite is that an engineer can add anything in the message needed to support testability. They have the flexibility to create messages that wouldn't normally show up through typical use of the application. One can programmatically set cookies values, modify headers, and change query strings or post data in the HTTP request. Marking a request is called "Transaction tagging". A tagged transaction will get as far as it needs to go to achieve adequate test coverage, possibly moving to alternate code paths. Tagging a transaction and handling it properly can ensure that payments don't get processed, orders don't get fulfilled, and that ultimately thousands of test orders don't end up getting shipped to an unfortunate engineer's doorstep.
Test your application at expected and unexpected volumes in your production environment, with traffic coming from outside your firewall.
SOASTA CloudTest Lite has been released to help as many developers and testers as possible. It's an absolutely free product (same codeline as our enterprise class Cloudtest Platform that leverages the cloud to generate load) and will help you optimize the performance of your application throughout its entire life-cycle. What do you get when you install the free download?
We've tried to make it as easy as possible to get you started. Follow these easy steps and you'll be up and running in less than 15 minutes!
If you need additional information about our product, don't hesitate to contact us at info[at]soasta.com. You can also interact with us via Twitter on our CloudTest account.