Sample Performance Test Strategy Template
About ABC chat Application – Let’s assume that this is a chat workbench that is used in a company by their customer support agent, this chat application uses XMPP protocol i.e, Extensible Messaging and Presence Protocol and Open fire server for sending and receiving Instant messages.
Some enhancements have been made to this existing chat client like Remote PC control, PC diagnosis, Repair tools, Online chat, etc., so this performance Test strategy is a sample of such applications.
The purpose of this document is to define/explain how Performance Testing will be performed on the ABC chat application for the current and future state.
ABC chat application is an in-house remote support Agent workbench. This workbench will be used to fulfill customer requests. This Workbench has capabilities such as Online chat, Customer Identification, Remote PC control, PC diagnosis, and repair tools.
The key objectives of Performance Testing are as follows:
- To gain the confidence that the changes to the existing chat application are in line with the defined Service Level Agreement.
- To ensure that the application performance, service availability, and the stability of the application are not impacted as a result of the new enhancements.
- Transaction Response Times remain within the acceptable tolerance over the increasing Load profile.
- JVMs show stable memory usage over the increasing load profiles.
Below is the Performance Testing scope for ABC chat workbench:
- Knowledge acquisition of the key business transactions and build load distribution after a detailed study of the system.
- Identify the critical scenarios for performance testing with assistance from different project tracks.
- Use the previous release results as a baseline for future releases.
- Verify and validate the performance test environment and the Performance/Load test tool infrastructure for any additional Agent Machines.
- Preparation of performance test scripts using JMeter for the identified scenarios that mimic the identified peak load.
- Setup performance monitoring on the servers for monitoring the test in order to identify the bottlenecks during the test execution phase.
- Publish Performance test results.
- Coordinate with various stakeholders to resolve the identified performance issues.
- Baseline the performance level for future releases.
Out of Scope
- Functional Testing, UAT, System Testing & Security Testing.
- Performance testing/monitoring of any third-party interfaces.
- Performance Tuning. (Most of the times Tuning is done by a different team, if in case you have performance engineers to tune the system then you can add this in the Inscope).
- Code profiling / Hardware sizing / Capacity planning.
- Security / Vulnerability testing / UAT/ White box testing.
- Data generation for Performance Testing.
- Non-functional tests (For Example, failover, disaster recovery, back-up, usability) other than the performance tests.
- Testing of any mobile solution.
- Third-Party Application Performance Testing & Tuning.
- Realization of performance recommendations, Application code changes and the vendor-supported products/server configuration changes will be out of scope from the Performance Team perspective.
- Infrastructure Support / Build Deployment/ Environment Readiness/ Database Restore/ Network Support etc.
Performance testing for ABC chat will be conducted using Jmeter by writing custom XMPP plugins that use a smack library for XMPP connections. These libraries are used to set up connections, login and send chat messages to the XMPP server.
These libraries are bundled into a jar file that is deployed into the Jmeter and is designed based on the scenarios to be tested. The Jmeter Work Bench is installed in the local machine which connects to the JMeter server which has the Load Generators to generate the required load on the Chat server system to monitor the system behavior.
The test scenario will be scripted using the JMeter tool. The scripts would be customized as required. The schedule will be created with the required ramp-up to simulate the real-world scenarios.
The Test Scenario would be broken up and measured in the below aspects:
a) Baseline Test: To run each scenario with 1 Vuser and multiple iterations in order to identify whether the application performance meets the business Service Level Agreement or not.
b) Base Load Test: To meet the Business Benchmark under load test, the Performance Testing team will perform a baseload test which will help to identify any system performance issues with increasing load and creates the baseline for the next level of performance testing.
c) Peak Load / Scalability Test: Performance Testing team will perform multiple tests with increasing Vusers to meet the expected load and also to measure the application performance to establish the performance curve and identify whether the deployment can support the service level agreements under the peak user load.
It helps in tuning or capacity planning of the individual Java virtual machines (JVM), the total number of required JVMs, and the processors. This will be achieved by increasing the no of Vusers to 50%, 75%, 100% and 125% of peak capacity.
d) Endurance Test: Performance Testing team will run this test for a period of 8 Hours / 16 Hours /24 hours to identify memory leaks, performance issues over time, and overall system stability. During endurance tests, the Performance Testing team monitors the key performance indicators, such as transaction response times and the stability of memory usage.
System resources like CPU, Memory, and IO need to be monitored with the help of the project team.
The Performance test environment is assumed to be a replica of the production environment. The tests will be run with an incremental load to identify where the application fails.
Performance Test Scenarios
Include the excel with the set of scenarios.
Scenario 1: To validate the Agent and customer chat for X no. of concurren