In the previous post I wrote about the test verification. The main idea was that before you launch a test you need to check that all your virtual user profiles can be executed correctly. In other words, the execution of a profile should produce a valid user session. It should not end with an error because the profile is incorrectly parameterized. Otherwise instead of the real testing you will simply load the web site with tons of incorrect requests.
One important thing that I also mentioned was that sometimes when a web site receives incorrect data in the request, it replies with the 200 status code contained in the response header, which means success. However at the same time it includes an error message in a human readable form in the body of the same response.
I demonstrated this with an example profile recorded on the following web forum: http://www.loadtestingtool.com/forum
Well, let’s imagine that we did our best on parameterization and reviewed all server responses after the test verification. They do not contain any errors, neither identified by HTTP codes, nor by such friendly messages inside the response bodies. Now what if we face the same problem during the test execution? Imagine that the web site starts to produce error messages when we execute a test with 1000 concurrent users. If it does not mark them with error codes, we will not see them in the report and we will not be able to review logs for all user sessions, because there can be many thousands.
That is why WAPT (and some other load testing tools) has a possibility to validate each server response by analyzing its content. Let’s see how this can be done using the same example.
I select the profile in the WAPT left view and switch to the “Properties” tab in the right view. This is what I see under the “Advanced” section.
Now I click the “Response Validation Rules…” button to display the dialog where the rules are specified. Our rule will be very simple: if the “error occurred” phrase is found inside the response body, it means that WAPT should treat such response as an error.
Note that this does not replace the status code validation mechanism, but works in addition to it.
By the way, you can also make WAPT treat slow responses as errors. This is another type of validation rule. It is useful if you have the exact response time requirement for your web site. In this case you can check the observance of your requirement by the error rate parameter.
For now we only use the rule specified in the above screen shot. When I verify the test after entering it, I get the following log.
Note the last line above. The server replies with the 200 status code, which means success, but WAPT also applied our validation rule and found the “error occurred” text inside the server response. That is why it marked the response with red indicating an error.
Sometimes we may want to apply different validation rules for different requests in the same profile. For some requests we may also want to disable validation while leaving it for all remaining ones. Such additional fine tuning of this feature can be done for each request. The corresponding options are available on the “Response processing” tab.
The purpose of the validation rules is to automatically distinguish between good and bad server responses received during the test run. So, any errors found this way will be reflected in the test report and graphs influencing the error rate parameters.