|
|
(44 intermediate revisions by 3 users not shown) |
Line 1: |
Line 1: |
− | == Software Requirements == | + | = OX App Suite Load Tests = |
− | * Java SE JDK6 or JDK7
| |
− | * Git
| |
| | | |
− | == Getting Performance Repository ==
| + | We're providing load tests for various OX App Suite APIs and maintain simulations that correspond to specific client implementations. To use it, clone the following repository: |
− | * Cloning Repository
| |
− | <code>$ git clone https://git.open-xchange.com/git/wd/testing/performance</code>
| |
| | | |
− | * Checkout a branch, e.g. ''develop''
| |
− | <code>$ git checkout develop</code>
| |
− |
| |
− |
| |
− | == Development Environment ==
| |
− | If you want to execute the tests via an IDE like Eclipse then you will also need to install the following software:
| |
− | * [https://www.eclipse.org Eclipse 4.2] or later
| |
− | * [http://scala-ide.org Scala IDE plugin]
| |
− | * Maven integration from the default update site '''m2e'''
| |
− | * [http://alchim31.free.fr/m2e-scala/update-site m2e-scala]
| |
− | * [http://maven.apache.org/ Apache Maven]
| |
− |
| |
− | === Importing Project into Eclipse ===
| |
− | # Open Eclipse
| |
− | # Click on File → Import... → Maven → Existing Maven Project
| |
− | # Select the gatling folder within the performance repository
| |
− |
| |
− | == Configurable Parameters ==
| |
− | List of all available System-Properties and JVM-Arguments:
| |
− |
| |
− | ; protocol: Which protocol to use. ''Possible values are: '''"http"''' or '''"https"'''. Default value: '''"http"'''.''
| |
− |
| |
− | ; host: Which host to use. ''Possible values are: '''"localhost"''', '''"10.0.0.1"''' or if you want to stress several groupware servers, to bypass a loadbalancer for example, then you can use comma-separated IPs or hostnames '''"10.0.0.1,10.0.0.2,localhost"'''. Default value: '''"localhost"'''.''
| |
− |
| |
− | ; users: How many concurrent users to simulate. ''Default value: '''"5"'''.''
| |
− |
| |
− | ; durationTime: Duration time of the simulation. ''Default value: '''"30"'''.''
| |
− |
| |
− | ; durationUnit: The Unit of the durationTime. ''Possible values are: '''"SECONDS"''', '''"MINUTES"''', '''"HOURS"''' or '''"DAYS"'''. Default value: '''"SECONDS"'''.''
| |
− |
| |
− | ; successpercent: Successful responses in percentage. ''Default value: '''"100"'''.''
| |
− |
| |
− | ; reqpersec: Mean number of requests per second (lower bound). ''Default value: '''"0"'''.''
| |
− |
| |
− | ; resptime: Mean response time (lower bound). ''Default value: '''"0"'''.''
| |
− |
| |
− | ; csv_users: Name of the CSV-File for feeding the requests. ''Default value: '''"ox_user_credentials.csv"'''.''
| |
− |
| |
− | ; syncmsgmax: How many objects to synchronize. ''Relevant only for '''QAPerfUSM'''. Default value: '''"50"'''.''
| |
− |
| |
− | ; pimObjects: How many PIM Objects, e.g. contacts, should be created. ''Default value: '''"5"'''.''
| |
− |
| |
− | === Snippet from the <properties>-section in the pom.xml ===
| |
− | <properties>
| |
− | <protocol>http</protocol>
| |
− | <host>localhost</host>
| |
− | <port>80</port>
| |
− | <users>5</users>
| |
− | <durationTime>30</durationTime>
| |
− | <durationUnit>SECONDS</durationUnit>
| |
− | <successpercent>100</successpercent>
| |
− | <reqpersec>0</reqpersec>
| |
− | <resptime>0</resptime>
| |
− | <csv_users>ox_user_credentials.csv</csv_users>
| |
− | <syncmsgmax>50</syncmsgmax>
| |
− | <pimObjects>5</pimObjects>
| |
− | </properties>
| |
− |
| |
− | === Snippet from the <jvmArgs>-section in the pom.xml ===
| |
− | <jvmArgs>
| |
− | <jvmArg>-Dprotocol=${protocol}</jvmArg>
| |
− | <jvmArg>-Dhost=${host}</jvmArg>
| |
− | <jvmArg>-Dport=${port}</jvmArg>
| |
− | <jvmArg>-Dcsv_users=${csv_users}</jvmArg>
| |
− | <jvmArg>-Dusers=${users}</jvmArg>
| |
− | <jvmArg>-DdurationTime=${durationTime}</jvmArg>
| |
− | <jvmArg>-DdurationUnit=${durationUnit}</jvmArg>
| |
− | <jvmArg>-Dsuccesspercent=${successpercent}</jvmArg>
| |
− | <jvmArg>-Dreqpersec=${reqpersec}</jvmArg>
| |
− | <jvmArg>-Dresptime=${resptime}</jvmArg>
| |
− | <jvmArg>-Dsyncmsgmax=${syncmsgmax}</jvmArg>
| |
− | <jvmArg>-DpimObjects=${pimObjects}</jvmArg>
| |
− | </jvmArgs>
| |
− |
| |
− | == Test Execution ==
| |
− |
| |
− | === Pre-Requisites for the Open-Xchange Groupware Server ===
| |
− | ==== Disabling rate limitation ====
| |
− | * Modify the parameter maxRateLenientClients @server.properties:
| |
− | $ vim /opt/etc/open-xchange/etc/server.properties
| |
− | * Specify a new User-Agent which is excluded from rate limit checks
| |
− | "GATLING*"
| |
− |
| |
− | ==== Resources/Data ====
| |
− | In the data folder you can find an sample csv file which is used to perform a login. If you want to use your own CSV-File place it into the data directory.
| |
− |
| |
− | * Structure of the csv file:
| |
− | <code>
| |
− | username,password
| |
− | user1,secret
| |
− | user2,secret
| |
− | user3,secret
| |
− | ...
| |
− | user998,secret
| |
− | user999,secret
| |
− | user1000,secret
| |
− | </code>
| |
− |
| |
− | === Pre-compiled version of the gatling tests ===
| |
− | Execute the gatling tests via bash script (Linux) or batch file (Windows).
| |
− |
| |
− | * Unzip ox-gatling-*-bin.zip
| |
− | * Change working directory
| |
− | $ cd Downloads/ox-gatling-1.0.0-SNAPSHOT
| |
− | * Please '''adjust''' the ''JAVA_OPTS'' in the scripts ''bin/box-gatling.sh'' or ''bin/ox-gatling.bat'' to match your needs.
| |
− | ** The following example will stress the server with the ''IP 10.0.0.1'' for ''120 seconds'' and hold the load with permanently ''100 user'':
| |
− |
| |
− | <code>exec "$JAVACMD" $JAVA_OPTS -Dprotocol=http -Dhost=10.0.0.1 -Dcsv_users=ox_user_credentials.csv -Dusers=100 -DdurationTime=120 -DdurationUnit=SECONDS \
| |
− | -classpath "$CLASSPATH" \
| |
− | -Dapp.name="ox-gatling" \
| |
− | -Dapp.pid="$$" \
| |
− | -Dapp.repo="$REPO" \
| |
− | -Dapp.home="$BASEDIR" \
| |
− | -Dbasedir="$BASEDIR" \
| |
− | io.gatling.app.Gatling \
| |
− | "$@"</code>
| |
− |
| |
− | * Execute the script
| |
− | $ /bin/bash bin/ox-gatling.sh
| |
| <code> | | <code> |
− | Choose a simulation number:
| + | $ git clone https://gitlab.open-xchange.com/qa/performance.git |
− | [0] com.openexchange.test.performance.gatling.tests.QAPerfOX6
| |
− | [1] com.openexchange.test.performance.gatling.tests.QAPerfOXAppSuite
| |
− | [2] com.openexchange.test.performance.gatling.tests.QAPerfUSM
| |
− | [3] com.openexchange.test.performance.gatling.tests.QAPerfOXDrive
| |
− | [4] com.openexchange.test.performance.gatling.tests.QAPerfEAS
| |
| </code> | | </code> |
| | | |
− | Now you will see some output of the compilation process but finally you should be able to start the simulation by entering a number.
| + | Documentation can be found at <code>com.openexchange.test.performance.gatling/project/doc</code>. Please checkout the branch that corresponds to the version of OX App Suite to be tested. |
− | | |
− | === Execute the gatling tests via maven plugin ===
| |
− | * Switch to the root directory of the gatling project
| |
− | | |
− | <code>cd /GIT_REPO/performance/com.openexchange.test.performance.gatling/</code>
| |
− | | |
− | * Now you can start the performance tests either by using the default values from the pom.xml,
| |
− | | |
− | <code>mvn clean gatling:execute</code>
| |
− | | |
− | or by temporarily passing system properties:
| |
− | | |
− | <code>mvn clean gatling:execute -Dhost=localhost -Dcsv_users=ox_user_credentials.csv -Dusers=5 -DdurationTime=20 -DdurationUnit=MINUTES -Dgatling.simulationClass=com.openexchange.test.performance.gatling.tests.QAPerfOX6</code>
| |
| | | |
− | == Practical hints and tips ==
| + | [[Category: AppSuite]] [[Category: Testing]] |
− | * If you want to run tests for more than 24h, uncomment the "timeOut" → "simulation" property at gatling.conf and define a value of more than 24h (in seconds).
| |
− | * If you experience OutOfMemory issues when generating charts, adjust "charting" → "accuracy" to a higher value. This will lower the accuracy of the generated graphs. Especially for long-running tests, this could be set to 100ms.
| |