AppSuite:ImageConverter Install

Download & Installation Imageconverter

General Information

OX App Suite displays images (photos and graphics) in many different ways as

• Thumbnails
• Icons
• Images embedded in documents
• in Emails
• in an own pop-up window.

The image processing is done by the OX middleware and not by a dedicated service, so that the middleware might consume a lot of cpu time just to convert images to different target formats. Furthermore the OX middleware does not cache images for a longer time. Only thumbnails get cached, either by the database or file system based. These images get converted into the required formats (e.g. jpeg, png) when requested by an user action in the user Interface.

With OX App Suite 7.10 there is an alternative to process images by the OX middleware. The image conversion and delivery can be delegated to an extra service, the OX Image Converter server. It's deployment model is exactly corresponding to the documentconverter server via a client/server service.

The separation of the Image Converter service has the following advantages:

Configuration and usage of one or more separate storages Performance improvements via persistent cache Pre-rendering of pre-defined formats and sizes (defaults are auto:200x150, auto:480x320, auto:640x480, auto:800x600, auto:1280x720, auto:1920x1080") if resolution of the source image allows this. Best matching size will be delivered without new conversion, if an image is requested by the Middleware Browser is able to cache those images No limits for image sizes The default target format is specified by configured, predefined formats. By using the 'auto' (beside 'jpg' or 'png') target format, opaque source images are converted to JPEG target images and transparent images are converted to PNG target images. Reduce load of OX Middleware

Requirements

See the Open-Xchange software requirements page for details.

Mandatory Modules

Additional functional modules need to be installed separately on the user servicing OX App Suite middleware nodes:

• The Image Converter Client: See Image converter client installation instructions

Installation

The Image Converter deployment consists of the package open-xchange-imageconverter-server.

Redhat Enterprise Linux 6 or CentOS 6 (valid until 7.10.3)

Add the following repositories to your Open-Xchange yum configuration:

[open-xchange-imageconverter]
name=Open-Xchange-imageconverter
baseurl=https://software.open-xchange.com/products/appsuite/7.10.3/imageconverter/RHEL6/
gpgkey=https://software.open-xchange.com/oxbuildkey.pub
enabled=1
gpgcheck=1
metadata_expire=0m

if you have a valid maintenance subscription, please add also following repositories by replacing the credentials.

[open-xchange-imageconverter-updates]
name=Open-Xchange-imageconverter-updates
baseurl=https://LDBUSER:LDBPASSWORD@software.open-xchange.com/products/appsuite/7.10.3/imageconverter/updates/RHEL6/
gpgkey=https://software.open-xchange.com/oxbuildkey.pub
enabled=1
gpgcheck=1
metadata_expire=0m


$ yum install open-xchange-imageconverter-server

Redhat Enterprise Linux 7, CentOS 7 or Amazon Linux 2

Add the following repositories to your Open-Xchange yum configuration:

[open-xchange-imageconverter]
name=Open-Xchange-imageconverter
baseurl=https://software.open-xchange.com/products/appsuite/stable/imageconverter/RHEL7/
gpgkey=https://software.open-xchange.com/oxbuildkey.pub
enabled=1
gpgcheck=1
metadata_expire=0m

if you have a valid maintenance subscription, please add also following repositories by replacing the credentials.

[open-xchange-imageconverter-updates]
name=Open-Xchange-imageconverter-updates
baseurl=https://LDBUSER:LDBPASSWORD@software.open-xchange.com/products/appsuite/stable/imageconverter/updates/RHEL7/
gpgkey=https://software.open-xchange.com/oxbuildkey.pub
enabled=1
gpgcheck=1
metadata_expire=0m

$ yum install open-xchange-imageconverter-server

Debian GNU/Linux 9.0

Add the following repositories to your Open-Xchange apt configuration:

deb https://software.open-xchange.com/products/appsuite/stable/imageconverter/DebianStretch /

if you have a valid maintenance subscription, please add also following repositories by replacing the credentials.

deb https://LDBUSER:LDBPASSWORD@software.open-xchange.com/products/appsuite/stable/imageconverter/updates/DebianStretch /

$ apt-get update
$ apt-get install open-xchange-imageconverter-server

Debian GNU/Linux 10.0

Add the following repositories to your Open-Xchange apt configuration:

deb https://software.open-xchange.com/products/appsuite/stable/imageconverter/DebianBuster /

if you have a valid maintenance subscription, please add also following repositories by replacing the credentials.

deb https://LDBUSER:LDBPASSWORD@software.open-xchange.com/products/appsuite/stable/imageconverter/updates/DebianBuster /

$ apt-get update
$ apt-get install open-xchange-imageconverter-server

SUSE Linux Enterprise Server 12 (valid until 7.10.3)

$ zypper ar https://software.open-xchange.com/products/appsuite/7.10.3/imageconverter/SLE_12 imageconverter

if you have a valid maintenance subscription, please add also following repositories by replacing the credentials.

$ zypper ar https://LDBUSER:LDBPASSWORD@software.open-xchange.com/products/appsuite/7.10.3/imageconverter/updates/SLE_12 imageconverter-updates

$ zypper ref
$ zypper install open-xchange-imageconverter-server

Monitoring

Image Converter offers runtime information via JMX about the Image Converter. This article guides to the information how to access JMX data, configure and use Jolokia and integrate with munin.

Configuration

The following property files need to be adjusted on the Image Converter node to ensure correct runtime behavior of the ImageConverter implementation

• fileitem.properties
• imageconverter.properties

Filestores

Since the Image Converter service uses the FileItemService, which itself needs a dedicated filestore and a dedicated database schema, the following steps need to be performed by the admin prior to starting the Image Converter WebService.

Please note: Initially file-system based filestores (NFS) were supported only. Object storage support (S3, Scality sproxyd) has been added with 7.10.3.

Create a directory for the FileItemService filestore(s) with OS commands, that is writable for the user, running the OX backends:

$ mkdir -p /var/opt/fileitemstore
$ chown open-xchange:open-xchange /var/opt/fileitemstore

Register the FileItemService filestore(s) via the OX admin command registerfilestore after installation. Since the the RMI port for administrative calls to the Image Converter server is set to 1095 by default, this port has to be set within the environment, prior to making the registerfilestore call itself. If making the administration call on the system by using an already running middleware server, the port setting can be omitted.

Since the Image Converter filestore is a dedicated filestore for this service only, the number of allowed contexts for the filestore must be set to 0 (by using switch '-x 0') in order to prevent the middleware from using this filestore as an additional filestorage.

$ env RMI_HOSTNAME=rmi://localhost:1095 /opt/open-xchange/sbin/registerfilestore -A oxadminmaster -P ${AS_ADMINMASTER_PASS} -x 0 -t file:///var/opt/fileitemstore

Enter the id(s) of the FileItemService filestore(s) (put into the configdb database by using the registerfilestore command) into the fileitem.properties configuration file

Create a new, empty database schema to be used by the FileItemService:

$ /opt/open-xchange/imageconverter/sbin/initfileitemdb -i --fileitemdb-pass="${AS_DB_PASS}" --fileitemdb-user=openexchange --fileitemdb-host=localhost --fileitemdb-port=3306 --fileitemdb-dbname=fileitemdb

Enter the database connection data, used to initialize the FileItemService database in the previous 'initfileitemdb' call into the appropriate fields of the fileitem.properties configuration file. Please take care to enter the 'read' properties as well as the 'write' properties within the configuration file.

The following properties of the fileitem.properties need to be adjusted, at least (example values are provided):

$ com.openexchange.fileItem.fileStoreIds=3
$
$ com.openexchange.fileItem.readUrl=jdbc:mysql://localhost/fileitemdb
$ com.openexchange.fileItem.readProperty.1=user=openexchange
$ com.openexchange.fileItem.readProperty.2=password=${AS_DB_PASS}
$
$ com.openexchange.fileItem.writeUrl=jdbc:mysql://localhost/fileitemdb
$ com.openexchange.fileItem.writeProperty.1=user=openexchange
$ com.openexchange.fileItem.writeProperty.2=password=${AS_DB_PASS}

The Image Converter itself currently relies on the external ImageMagick framework, so that this package has to be installed on the node, the Image Converter is running on

ImageMagick

On Amazon Linux 2, ImageMagick is located under /bin. Therefore the property com.openexchange.imageconverter.imagemagick.searchPath must be adjusted accordingly.

Sizing and technical background

Summary

• One Image Converter node should have at least 4 cores
• One Image Converter node needs to have at least 4GB RAM
• Local storage space with a minimum of 10 GigaBytes needs to be available on an Image Converter node
• At least 2 Image Converter nodes should be available on a production system in parallel, each working on the same, shared database and shared file storage

Background

Whenever the OX App Suite UI is going to display image representations of user provided content like thumbnails for arbitrary source documents, images in Viewer application or images contained in documents that are currently edited in one of the Ox Documents applications, the frontend sends an appropriate request to the middleware. After pre processing the source content to create an image or after extracting a thumbnail image from a document to be returned, the middleware needs to process the resulting image according to different view constraints before the resulting image is returned to the OX App Suite UI to be displayed. Such image processing might include scaling, rotation and cropping operations, which can be expensive. In order to remove that image processing load from the OX App Suite middleware, an Image Converter (IC) can be deployed on a dedicated node in order to perform those operations. In addition, the IC is able to cache image results after processing to speed up access to images that have already been processed before.  To get a best fit for most use case, the IC creates a set of differently processed target files (e.g. different image resolutions) for each source image and stores those results within its configured cache. The following hardware recommendations for an IC deployement are based on collected inhouse metrics and should fit for most customer deployments. Nevertheless, there might be scenarios where these recommendations do not fit well, so that the customer should track provided IC metrics to ensure proper operation of the IC deployment. Detailed up to date documentation for all mentioned IC related configuration items can be found within the provided /opt/open-xchange/imageconverter/etc/imageconverter.properties file.

Storage

Temporary files

For proper operation, the IC needs to rely on a working directory. This working directory is used to temporarily store source image attachments from processing requests, made by the middleware to the IC, as well as to store intermediate target images after processing before those target result images are cached in a persistent way. Due to its nature as a spool and working directory, access times to the related working directory should be as fast as possible. As such this working directory should be located on a local, volume with fast access times that is not shared via NFS or similar. Given a maximum default upload size for files in OX App Suite of 10 MegaBytes and a maximum default IC job queue length of 1024 entries, the total available size of this working directory should be able to store 10 GigaBytes of image files in its own format (e.g. JPEG, PNG) in the worst use case. All files, created by the IC within this working directory are automatically removed at runtime or shutdown  of the IC. In case there was an unexpected, not intended shutdown of the IC or the whole system, a cleanup of all left over temporary files within the configured working directory will happen at IC startup as well in any case. The working directory, that needs to have write access for the IC  user can be configured via the following config item:

• com.openexchange.imageconverter.spoolPath

Persistent files

If an IC database and at least one FileStore is configured to be used by the IC, all processed result images are cached as persistent files, meaning that they will survive IC restarts and are preferably used before processing of the same source image happens again. The number of image keys (number of source images, whose processed list of target images is to be stored), the totally occupied space of all target images and the time to live for the image keys and corresponding files (based on last access time) within the cache are configured via the the following IC config properties:

• com.openexchange.imageconverter.cache.maxKeyCount    
• com.openexchange.imageconverter.cache.maxSizeGB
• com.openexchange.imageconverter.cache.keyTimeoutMinutes

If one of the configured cache constraints is reached, the cache automatically removes the oldest image keys and corresponding target image files for each image key from the cache until all cache metrics are below the configured cache constraints again. The value for each of the above cache constraints can be specified in detail or set to unlimited by the administrator. Please see the appropriate IC configuration documentation mentioned above for further details.

Memory usage

The IC deployment consists of the Java VM based IC itself and the corresponding image processing backends, which are ImageMagick processes at the moment, separately started by the IC for each single image transformation process. When the OX middleware performs an image based operation due to a request from the OX App Suite UI and is going to return an image response, it first checks, if the resulting image is already contained within the IC cache. If no appropriate cache entry can be retrieved, the OX middleware sends an image transformation request to the IC. The request is then put into an IC queue to be transformed by the image processing backends afterwards. The maximum length of this request queue can be specified by the following config property:

• com.openexchange.imageconverter.queueLength

If one image processing backend gets available, the IC takes the next job request from the FIFO queue and starts processing the list of target images via one started processing backend for each required target image. The number of processing backends that can  be started in parallel is specified by the following config property:

• com.openexchange.imageconverter.threadCount

Java VM memory usage

The maximum available runtime memory that can be used by the IC Java VM process is set to 1024 MegaBytes by default. This value has been chosen as a sensible and appropriate value based on other configuration default properties. Request image attachments are extracted and stored as temporary files to be processed later, so that this maximum available Java VM memory limit should be sufficient with the default queue length configured. If larger queue lengths are configured by the administrator, the maximum available Java VM memory limit should be increased as well.  Exact numbers for the increment of available Java VM memory cannot be given due to quite different source sizes of images in general.  A Java VM memory increment of 100 MegaBytes for an additional queue length of 100 entries should be sufficient.

Processing backend memory usage

Considering quite large high resolution images with alpha channel and an extent of e.g. 8000x6000 pixels (and sometimes even more), an image processing backend will import this image and use at least an input and output buffer for the unpacked image data of size 8000x6000x4 Bytes at maximum each when not scaling up the image. This sums up to about 366 MegaBytes for each image processing backend used in this example case. With a default of 8 processing backends, a pessimistic calculation would require about 3000 MegaBytes of runtime memory available for all image processing backends currently processing such kind of images.

Request throughput

When talking about request throughput, we need to take two different scenarios into account. The first one is the IC operation with no available cache or an available cache that has not been filled up much yet. In the (virtual) 'no cache' case we need to take the processing time for time consuming requests into account. Such requests might be #CacheAndGetImage requests as well as #GetImage requests in some cases.  According to inhouse metrics we collected on different IC test deployments so far, we achieved a median request time for those two request types of about 0.15s/request. Since those requests use one processing backend, we can calculate a median request throughput per second without using a cache the following way.

• 1.0 / 0.15s = 6.7/s (median requests per second for 8 processing backends using no cache)

When the cache is enabled and already filled up by preceding requests, a cache hit ratio between 80% and 90% can be reached in general. This means that 8-9 out of 10 requests can be responded in a fast way with a result taken from the cache without using the expensive processing backend path. For the request throughput, this means that we can multiply the pure processing request throughput with (1.0 / ( 1.0 - cacheHitRatio)) to get the resulting request throughput when having access to cached results as well. Assuming a conservative value of 75% cache hit ratio for a given system, meaning that 3 out of 4 requests can be responded in a fast way, the request throughput will be increased to:

• 6.7/s * (1.0 / (1.0 - 0.75)) = 26.8/s (median requests per second for 8 processing backends using a cache with a cache hit ratio of 75%)

Multiple nodes and number of users per node

The minimum number of IC nodes deployed on a customer system should be 2 to have at least one remaining IC node in case of the outage of the other IC node. This also reduces load for one IC node at all. More than 2 IC nodes can be deployed the same way in parallel in case the number of requests indicates this.

To avoid multiple conversions for the same source image on different IC nodes, all nodes should work on the same IC cache and filestore. This requires the admin to configure the same cache database on all IC nodes and the usage of a shared file storage across all IC nodes. In addition, the /opt/open-xchange/imageconverter/imageconverter.properties config file should have the same content on all IC nodes.

Metrics to Monitor

Due to the fact that each user has a quite different usage scenario when working in OX App Suite, the number of provisioned users per IC node, working in parallel with different kind of images or with no images at all on the customer deployment, the current recommendation for the number of IC nodes is to start with 2 IC nodes and to watch for the 'peak job count in queue' metric. In case this value is constantly at or near  the  maximum configured queue entry count (1024 by default), additional IC nodes should  be added to the deployment in parallel to be able to process all user requests within an appropriate time frame. Additional helpful metrics are CacheHitRatio (should be above 0.6) and CacheOldestEntrySeconds (>172800‬ = 2 days). If these values can't be met one may want to increase the size of the storage medium.