AppSuite:Load Tests

From Open-Xchange

Software Requirements

  • Java SE JDK6 or JDK7
  • Git
  • SBT

Getting Performance Repository

  • Cloning Repository
 $ git clone https://git.open-xchange.com/git/wd/testing/performance
  • Checkout a branch, e.g. develop
 $ git checkout develop

Install SBT

Since we use the Gatling-SBT plugin, which allows us to use Gatling as a testing framework, we'll need to install SBT first.

Installing sbt on Mac

 $ port install sbt
 $ brew install sbt

Installing sbt on Windows

Installing sbt on Linux

Installing sbt manually

Build and execute the tests with the SBT-Plugin

After SBT has been successfully installed we can switch to the project's directory:

 $ cd ~/performance/com.openexchange.test.performance.gatling


SBT can be used both as a command line script and as a build console. Most of the commands can be run standalone by passing the command as an argument to SBT, e.g.

 $ sbt update

Note that if a command takes arguments, you need to quote the entire argument path, e.g.

 $ sbt 'gatling:testOnly com.openexchange.test.performance.gatling.tests.QAPerfOX6'


The following command line script will execute the Gatling OX6 Performance test:

 $ sbt 'set javaOptions ++=Seq("-Dgatling_host=10.0.0.1", "-Dgatling_protocol=http", "-Dgatling_port=80", "-Dgatling_user=100", "-Dgatling_duration_time=120", "-Dgatling_duration_unit=SECONDS")' clean "gatling:testOnly com.openexchange.test.performance.gatling.tests.QAPerfOX6"

Development Environment

If you want to execute/debug the tests via an IDE like Eclipse or IntelliJ IDEA then you will also need to install the following software:

IntelliJ IDEA

Download and install the IntelliJ IDEA Community Edition

Eclipse

Since there are a few problems with the Scala IDE for eclipse we recommend to use IntelliJ which works out of the box.

Importing Project into IntelliJ

  1. Open IntelliJ
  2. Click on Import Project...
  3. Select the gatling folder within the performance repository
  4. Import project from external model → SBT Project → Finish

Configurable Parameters

List of all available System-Properties and JVM-Arguments:

protocol
Which protocol to use. Possible values are: "http" or "https". Default value: "http".
host
Which host to use. Possible values are: "localhost", "10.0.0.1" or if you want to stress several groupware servers, to bypass a loadbalancer for example, then you can use comma-separated IPs or hostnames "10.0.0.1,10.0.0.2,localhost". Default value: "localhost".
users
How many concurrent users to simulate. Default value: "5".
durationTime
Duration time of the simulation. Default value: "30".
durationUnit
The Unit of the durationTime. Possible values are: "SECONDS", "MINUTES", "HOURS" or "DAYS". Default value: "SECONDS".
successpercent
Successful responses in percentage. Default value: "100".
reqpersec
Mean number of requests per second (lower bound). Default value: "0".
resptime
Mean response time (lower bound). Default value: "0".
csv_users
Name of the CSV-File for feeding the requests. Default value: "ox_user_credentials.csv".
syncmsgmax
How many objects to synchronize. Relevant only for QAPerfUSM. Default value: "50".
pimObjects
How many PIM Objects, e.g. contacts, should be created. Default value: "5".

Test Execution

Pre-Requisites for the Open-Xchange Groupware Server

Disabling rate limitation

  • Modify the parameter maxRateLenientClients @server.properties:
 $ vim /opt/etc/open-xchange/etc/server.properties
  • Specify a new User-Agent which is excluded from rate limit checks
 "GATLING*"

Resources/Data

In the data folder you can find an sample csv file which is used to perform a login. If you want to use your own CSV-File place it into the data directory.

  • Structure of the csv file:
 
 username,password
 user1,secret
 user2,secret
 user3,secret
 ...
 user998,secret
 user999,secret
 user1000,secret
 

Practical hints and tips

  • If you want to run tests for more than 24h, uncomment the "timeOut" → "simulation" property at gatling.conf and define a value of more than 24h (in seconds).
  • If you experience OutOfMemory issues when generating charts, adjust "charting" → "accuracy" to a higher value. This will lower the accuracy of the generated graphs. Especially for long-running tests, this could be set to 100ms.