AppSuite:Load Tests: Difference between revisions

From Open-Xchange
No edit summary
 
(14 intermediate revisions by 2 users not shown)
Line 1: Line 1:
== Software Requirements ==
= OX App Suite Load Tests =
* Java SE JDK6 or JDK7
* Git
* SBT


== Getting Performance Repository ==
We're providing load tests for various OX App Suite APIs and maintain simulations that correspond to specific client implementations. To use it, clone the following repository:
* Cloning Repository
<source lang="bash">$ git clone https://git.open-xchange.com/git/wd/testing/performance</source>


* Checkout a branch, e.g. ''develop''
<code>
<source lang="bash">$ git checkout develop</source>
$ git clone https://gitlab.open-xchange.com/qa/performance.git
</code>


== Install SBT ==
Documentation can be found at <code>com.openexchange.test.performance.gatling/project/doc</code>. Please checkout the branch that corresponds to the version of OX App Suite to be tested.
Since we use the Gatling-SBT plugin, which allows us to use Gatling as a testing framework, we'll need to install SBT first.


=== Installing sbt on Mac ===
[[Category: AppSuite]] [[Category: Testing]]
* [http://macports.org Macports]
<source lang="bash">$ port install sbt</source>
 
* [http://mxcl.github.com/homebrew Homebrew]
<source lang="bash">$ brew install sbt</source>
 
=== Installing sbt on Windows ===
* Download the [https://dl.bintray.com/sbt/native-packages/sbt/0.13.7/sbt-0.13.7.msi msi installer] and install it.
 
=== Installing sbt on Linux ===
* [https://dl.bintray.com/sbt/rpm/sbt-0.13.7.rpm RPM] package
* [https://dl.bintray.com/sbt/debian/sbt-0.13.7.deb DEB] package
 
=== Installing sbt manually ===
* [http://www.scala-sbt.org/0.13/tutorial/Manual-Installation.html Manual] instructions
 
== Build and execute the tests with the SBT-Plugin ==
After SBT has been successfully installed we can switch to the project's directory:
<source lang="bash">
$ cd ~/performance/com.openexchange.test.performance.gatling
</source>
 
SBT can be used both as a command line script and as a build console. Most of the commands can be run standalone by passing the command as an argument to SBT, e.g.
 
<source lang="bash">$ sbt update</source>
 
Display all available projects:
 
<source lang="bash">
$ sbt projects
[info] Loading project definition from ~/performance/com.openexchange.test.performance.gatling/project
[info] Set current project to root (in build file:~/performance/com.openexchange.test.performance.gatling/)
[info] In file:~/performance/com.openexchange.test.performance.gatling/
[info]   project-ajax
[info]   project-common
[info]   project-drive
[info]   project-guard
[info]   project-in8
[info]   project-jolokia
[info]   project-usm
[info] * root
</source>
 
Switch to a specific project:
 
<source lang="bash">
root> project project-ajax
[info] Set current project to project-ajax (in build file:~/performance/com.openexchange.test.performance.gatling/)
project-ajax>
</source>
 
To execute all tests in the project just enter:
 
<source lang="bash">
project-ajax> gatling:test
[info] Formatting 21 Scala sources {file:~/performance/com.openexchange.test.performance.gatling/}project-common(test) ...
[info] Formatting 66 Scala sources {file:~/performance/com.openexchange.test.performance.gatling/}project-ajax(test) ...
[info] Compiling 21 Scala sources to ~/performance/com.openexchange.test.performance.gatling/project-common/target/scala-2.10/test-classes...
[info] Compiling 66 Scala sources to ~/performance/com.openexchange.test.performance.gatling/project-ajax/target/scala-2.10/test-classes...
[info] Compiling 66 Scala sources to ~/performance/com.openexchange.test.performance.gatling/project-ajax/target/scala-2.10/gatling-classes...
Simulation com.openexchange.test.performance.gatling.simulations.QAPerfOX6 started...
</source>
 
Note that if a command takes arguments, you need to quote the entire argument path, e.g.
 
<source lang="bash">
$ sbt 'project-ajax/gatling:testOnly com.openexchange.test.performance.gatling.simulations.QAPerfOX6'
</source>
 
 
The following command line script will execute the Gatling OX6 Performance test:
<source lang="bash">$ sbt "project project-ajax" 'set javaOptions ++=Seq("-Dgatling_host=10.0.0.1", "-Dgatling_protocol=http", "-Dgatling_port=80", "-Dgatling_user=100", "-Dgatling_duration_time=120", "-Dgatling_duration_unit=SECONDS")' clean "gatling:testOnly com.openexchange.test.performance.gatling.simulations.QAPerfOX6"
</source>
 
== Development Environment ==
If you want to execute/debug the tests via an IDE like Eclipse or IntelliJ IDEA then you will also need to install the following software:
 
=== IntelliJ IDEA ===
Download and install the [http://www.jetbrains.com/idea/download/ IntelliJ IDEA Community Edition]
 
=== Eclipse ===
* [https://www.eclipse.org Eclipse 4.2] or later
* [http://scala-ide.org Scala IDE plugin]
 
Since there are a few problems with the Scala IDE for eclipse we recommend to use IntelliJ which works out of the box.
 
=== Importing Project into IntelliJ ===
 
==== Create project files ====
To create the Idea project files please use the sbt task '''gen-idea'''.
 
<source lang="bash">
$ cd ~/git/performance/com.openexchange.test.performance.gatling
 
$ sbt gen-idea
[info] Loading project definition from ~/git/performance/com.openexchange.test.performance.gatling/project
[info] Set current project to root (in build file:~/git/performance/com.openexchange.test.performance.gatling/)
...
</source>
This task will resolve all project dependencies and create the IDEA modules accordingly.
 
==== Import project ====
# Start IntelliJ
# Click on Import Project...
# Select the gatling folder within the performance repository
# Import project from external model → SBT Project → Finish
 
== Configurable Parameters ==
List of all available System-Properties and JVM-Arguments:
 
; gatling_protocol: Which protocol to use. ''Possible values are: '''"http"''' or '''"https"'''. Default value: '''"http"'''.''
 
; gatling_host: Which host to use. ''Possible values are: '''"localhost"''', '''"10.0.0.1"''' or if you want to stress several groupware servers, to bypass a loadbalancer for example, then you can use comma-separated IPs or hostnames '''"10.0.0.1,10.0.0.2,localhost"'''. Default value: '''"localhost"'''.''
 
; gatling_port: Which port to use. ''Possible values are: '''"80"''', '''"8009"'''
 
; gatling_user: How many concurrent users to simulate. ''Default value: '''"5"'''.''
 
; gatling_duration_time: Duration time of the simulation. ''Default value: '''"30"'''.''
 
; gatling_duration_unit: The Unit of the durationTime. ''Possible values are: '''"SECONDS"''', '''"MINUTES"''', '''"HOURS"''' or '''"DAYS"'''. Default value: '''"SECONDS"'''.''
 
; gatling_successpercent: Successful responses in percentage. ''Default value: '''"100"'''.''
 
; gatling_reqpersec: Mean number of requests per second (lower bound). ''Default value: '''"0"'''.''
 
; gatling_resptime: Mean response time (lower bound). ''Default value: '''"0"'''.''
 
; gatling_user_feeder: Name of the CSV-File for feeding the requests. ''Default value: '''"ox_user_credentials.csv"'''.''
 
; gatling_basic_auth_user: The user name of the basic access authentication.
 
; gatling_basic_auth_pass: The password of the basic access authentication.
 
; gatling_syncmsgmax: How many objects to synchronize. ''Relevant only for '''QAPerfUSM'''. Default value: '''"50"'''.''
 
; gatling_pimObjects: How many PIM Objects, e.g. contacts, should be created. ''Default value: '''"5"'''.''
 
== Test Execution ==
 
=== Pre-Requisites for the Open-Xchange Groupware Server ===
==== Disabling rate limitation ====
* Modify the parameter maxRateLenientClients @server.properties:
  $ vim /opt/etc/open-xchange/etc/server.properties
* Specify a new User-Agent which is excluded from rate limit checks
  "GATLING*"
 
==== Resources/Data ====
In the data folder you can find an sample csv file which is used to perform a login. If you want to use your own CSV-File place it into the data directory.
 
* Structure of the csv file:
  <code>
  username,password
  user1,secret
  user2,secret
  user3,secret
  ...
  user998,secret
  user999,secret
  user1000,secret
  </code>
 
== Practical hints and tips ==
* If you want to run tests for more than 24h, uncomment the "timeOut" → "simulation" property at gatling.conf and define a value of more than 24h (in seconds).
* If you experience OutOfMemory issues when generating charts, adjust "charting" → "accuracy" to a higher value. This will lower the accuracy of the generated graphs. Especially for long-running tests, this could be set to 100ms.

Latest revision as of 17:19, 11 July 2017

OX App Suite Load Tests

We're providing load tests for various OX App Suite APIs and maintain simulations that correspond to specific client implementations. To use it, clone the following repository:

$ git clone https://gitlab.open-xchange.com/qa/performance.git

Documentation can be found at com.openexchange.test.performance.gatling/project/doc. Please checkout the branch that corresponds to the version of OX App Suite to be tested.