Subscribe to SoftLogica
Newsletter to get 10% discount on newly
released products!
Summary Graphs
Summary Graphs
Summary Graphs contain a number of graphs that display
consolidated information on all user profiles of current test scenario.
You can view Summary Graphs on test completion to know how tests were performed.
You can also view intermediate results at run-time while your test is being executed.
Click on Summary Graphs view in Navigation Pane to display the chart.
There are 4 tabs in this view:
You can manually choose what graphs will be shown on the chart. Check/uncheck options below the
chart to show/hide graphs. You can check the All option to display all graphs on this tab.
Tip:
Note that Y axis on the chart is different for various graphs. For example: : The number of
active users is displayed on the right side of the chart. : The number of
pages executed per timing unit (second, minute or hour) is displayed on the left side of the
chart.
X axis denotes time interval of test run. The whole time interval is divided
into a number of intervals, each of them equal to Refreshing time
value specified in Log and Report
Settings.
If you position a mouse pointer on graph nodes, you will see a hint with the value in
current node.
If you don't see the whole graph,
you can drag&drop the graph on the chart to move it and view its invisible parts.
You can increase/decrease the scale of any axis on the chart. To change the scale of axis,
position a mouse pointer on axis values and
drag&drop them to get desired scale.
Autofit chart
This option enables to fit graphs automatically to window size.
You can also double-click your mouse on the chart for this purpose.
Resample
It may occur that there are too many nodes displayed on the chart so that neighbour nodes
merge together. In this case, you can resample graphs down to represent them more visually.
For this, move the Resample arrow to the left.
Performance tab Avg response time (all profiles): Shows values of response
time averaged through all user profiles.
Average response time is one of the most important characteristics of load testing; it
measures web user experience.
Response time graph will tell you how long a user waits for server response to his request.
There are 3 important limits for response time values:
0.1 second.
It is an ideal response time. Users feel that the system is reacting instantaneously, and don't
sense any interruption.
1.0 second.
It is the highest acceptable response time. Users still don't feel an interruption, though they
will notice the delay. Response times above 1 second interrupt user experience.
10 seconds.
It is the limit after which response time becomes unacceptable. Moreover, recent studies show
that if response time exceeds 8 seconds, user experience is interrupted very much and most
users will leave the site or system.
Normally, response times should be as fast as possible. The interval of most comfortable
response times is 0.1 - 1 second. Although people can adapt to slower response times, they are
generally dissatisfied with times longer than 2 seconds.
Below are 3 possible variants of Response time graph behavior when the number of users is
increasing:
Flat (or very slight growth): It is an ideal result. The increase of load on the
server doesn't lead to increase of response time (or leads to very slight growth).
Gradual growth (essential growth): The increase of load on the server leads to
gradual growth of response time. It means that the server can handle the growing level of load
until the load exceeds some maximum value. Possible reasons of such situation are problems
with server hardware, for example, insufficient network bandwidth or low productivity of the
server.
Sharp growth: If response time graph exhibits a sharp growth beginning from some
level of user load while download time graph doesn't grow essentially, it means that the server
provides a poor performance when the load reaches this level, or even cannot cope with such
load. Users will see that the server responds very slowly, or doesn't respond at all.
Graphs described in first two cases - flat or gradually growing - can suddenly exhibit a sharp
growth. A point where the graph begins to grow sharply is called performance breakpoint. After
this breakpoint, server performance degrades noticeably.
Note that you may use such interpretation of response time graph only if download time graph
doesn't exhibit an essential growth. If percentage of download time from response time is
rather high, it indicates that there are problems with connection bandwidth, but not with
server performance.
Avg download time (all profiles): Shows
values of download time averaged
through all user profiles.
WAPT measures download time without images.
It is the time from the moment a user sees web page title (in browser's title bar) till the
moment he can start reading a page, i.e. when readable content of web page is displayed on
user's monitor.
Readable content of web page is page structure (frames,
tables) and HTML text.
In most cases, you will need to know this time rather than download
time of all page elements including images, flash movies, etc.
Download time is the measure of web server connection bandwidth. It indicates whether the
bandwidth is sufficient to provide required level of performance or not. If the growth of
virtual users number leads to rapid increase of download time, then most probably your server
connection bandwidth is inadequate.
On the whole, you should watch response and download times together. If download time is the
major part of response time value, it means that you have problems with connection bandwidth.
You should also analyze other results related to bandwidth: the speed of receiving the
information from the server and speed of sending the information to the server.
If Download time graph exhibits sharp fluctuations, you should provide more stable
test environment: make your test machine as close as possible to the production server,
and conduct tests at off-hours when network traffic is lower.
Pages per second: Shows the number of pages
executed per timing unit (second, minute or hour). Timing unit is specified
in Log and Report Settings
(Timing combo-box). Hits per second: Shows the number of hits executed per timing unit (second,
minute or hour).
Note that a hit is a single request for resource (page code, image, script
and so on) sent to the server, while each page may include many hits.
In particular, if "Show pictures" checkbox in advanced Internet Explorer
options is turned off, then the number of hits is equal to the number
of pages.
The number of pages/hits per second is
the most valuable result of testing a site/application capacity and overall performance.
If you perform stress testing (testing the site under abnormal or extreme conditions),
these graphs will help you find performance breakpoint.
Active users: Shows the number of virtual users participated in the test.
All: Displays all graphs on this tab.
Bandwidth tab Received: Shows how many kbits per second were received from the
server. Sent: Shows how many kbits per second were sent to the server.
Received per user: Shows receiving speed per virtual user (in kbits per second).
Sent per user: Shows sending speed per virtual user (in kbits per second).
These graphs will help you know whether the bandwidth of Internet connection to your server is
sufficient to provide acceptable level of performance or not.
Active users: Shows the number of virtual users participated in the test.
All: Displays all graphs on this tab.
Errors tab Total Errors %: Shows percentage of responses with errors (HTTP
errors, socket errors and timeouts) from the total number of responses.
HTTP Errors %: Shows percentage of responses with HTTP errors from
the total number of responses.
If you receive an HTTP error, it means that there are problems with the work of web server.
Socket Errors %: Shows percentage of responses with socket errors from the total number
of responses.
Socket error means bad connectivity; it indicates that there are problems with data transfer through
the socket.
Timeouts %: Shows percentage of responses with timeouts
(response timeouts and
socket timeouts)
from the total number of responses.
These graphs will help you know how error rate changes during a test when the number of virtual
users is increasing.
Error rate is the most valuable result of stress testing where you need to find the maximum
number of users that can be served correctly, without errors.
You will also need to watch error rate during
reliability/endurance tests to verify that it is in acceptable range even after a long run.
You can also view Errors report
to know which pages were executed with errors. Active users: Shows the number of virtual users participated in the test.
All: Displays all graphs on this tab.
Custom tab Custom tab represents all graphs from the previous tabs.
Saving the Summary Graphs You can save any chart as an image in png format
in required location. For this, select desired tab (Performance, Bandwidth,
Errors or Custom), right-click on the chart and select Save Current
Chart to Image... option on the context menu. In the displayed dialog,
enter file name, select a location and click Save. You will be
able to view the chart later, any time you wish just from Internet Explorer
or any other program that supports png format. It is also a convenient
way to transfer test results to any interested person.
: The number of
active users is displayed on the right side of the chart.
: The number of
pages executed per timing unit (second, minute or hour) is displayed on the left side of the
chart.
