AppSuite:OX Abuse Shield

From Open-Xchange
Revision as of 07:38, 16 June 2016 by Khgras (talk | contribs)

API status: In Development

Dovecot Anti Abuse Shield (soon to be released)

Dovecot Anti-Abuse Shield is included along with Dovecot Pro and OX App Suite as a component to protect against login/authentication abuse.

Anti-Abuse Shield runs on a cluster of servers, and integrates with both OX App Suite and Dovecot to detect abuse, brute force attacks and also to enforce common authentication/authorization policies across the platform.

Key Features

Features of Dovecot Anti-Abuse Shield include:

  • Replicated/clustered architecture – Login reports are shared between all the servers in a cluster so there is a single view of abuse
  • Scriptable Policy Language – Using the Lua language, the functionality of the daemon can be extended to record and protect against a large variety of abusive behavior, as well as implement specific customer policies.
  • DNS Lookup Feature – For looking up IPs or domains in blacklists
  • GepIP Lookup Feature – GeoIP lookups can be made, and incorporated into policy decisions.
  • Ratelimiting and Tarpitting – Extremely flexible, these can be enabled and enforced based on IP address, login name, geoip location, time windows, etc.
  • Flexible In-Memory Statistics Database – A versatile and extensible in-memory database is used to store statistics information about abuse over time periods from a few minutes to many hours.
  • Integration with Customer Authentication/Authorization Systems – Customers can use the open HTTP REST API to benefit from the protection of the anti-abuse daemon in their own authentication/authorization systems.
  • Admin Console – To retrieve statistics and query server state

Pricing and availability

Please contact Open-Xchange Sales for further information and pricing details.

In General

The goal of Dovecot Anti-Abuse Shield is to detect brute forcing of passwords across many servers, services and instances, as well as enforce policy for authentication and authorization. In order to support the real world, brute force detection policy can be tailored to deal with "bulk, but legitimate" users of your service, as well as botnet-wide slowscans of passwords.

Here is how it works:

  • Report successful logins via JSON http-api
  • Report unsuccessful logins via JSON http-api
  • Query if a login should be allowed to proceed, should be delayed, or ignored via JSON http-api

Various other API functions are available, please see https://documentation.open-xchange.com/7.8.2/middleware/components/weakforced.html for full API documentation.

OX App Suite and Dovecot's POP/IMAP server are pre-integrated with Dovecot Anti-Abuse Shield. For more information on how to configure them to work with Anti-Abuse Shield, see \<ref for AppSuite doc\> and http://wiki2.dovecot.org/Authentication/Policy.

However it is also aimed to receive message from other services like:

  • Other IMAP/POP servers
  • Other Webmail logins
  • FTP logins
  • Authenticated SMTP
  • Self-service logins
  • Password recovery services

By gathering failed and successful login attempts from as many services as possible, brute forcing attacks can be detected and prevented more effectively.

The service runs as a daemon (called wforce), and can be clustered in a way that report information is shared between all members of the cluster.

Download and Installation

Redhat Enterprise Linux 7 or CentOS 7

If not already done, add the following repositories to your Open-Xchange yum configuration: 

 [open-xchange-backend]
name=Open-Xchange-backend
baseurl=https://software.open-xchange.com/products/dovecot_anti_abuse/stable/backend/RHEL7/
gpgkey=https://software.open-xchange.com/oxbuildkey.pub
enabled=1
gpgcheck=1
metadata_expire=0m

and run 

$ yum update
$ yum install weakforce.d

Debian GNU/Linux 8.0

If not already done, add the following repositories to your Open-Xchange apt configuration: 

deb https://software.open-xchange.com/products/dovecot_anti_abuse/stable/backend/DebianJessie /


$ apt-get update
$ apt-get install weakforce.d

Policies

Configuration and control of policy is almost entirely through a configuration file which is based on the Lua scripting language. There is a sensible default configuration in wforce.conf, and extensive support for crafting your own policies using the Lua scripting language.

Note that although there is a single Lua configuration file, the report and allow functions run in different lua states from the rest of the configuration, thus you cannot share state.

The following sample is from the default wforce.conf file: 

```
-- set up the things we want to track
field_map = {}
-- use hyperloglog to track cardinality of (failed) password attempts
field_map["diffFailedPasswords"] = "hll"
-- track those things over 6x10 minute windows
newStringStatsDB("OneHourDB", 600, 6, field_map)

-- this function counts interesting things when "report" is invoked
function twreport(lt)
	sdb = getStringStatsDB("OneHourDB")
	if (not lt.success)
	then
	   sdb:twAdd(lt.remote, "diffFailedPasswords", lt.pwhash)
	   addrlogin = lt.remote:tostring() .. lt.login
	   sdb:twAdd(addrlogin, "diffFailedPasswords", lt.pwhash)
	end
end

function allow(lt)
	sdb = getStringStatsDB("OneHourDB")
	if(sdb:twGet(lt.remote, "diffFailedPasswords") > 50)
	then
		return -1, "", "", {} -- BLOCK!
	end
	// concatenate the IP address and login string
	addrlogin = lt.remote:tostring() .. lt.login	
	if(sdb:twGet(addrlogin, "diffFailedPasswords") > 3)
	then
		return 3, "tarpitted", "diffFailedPasswords", {} -- must wait for 3 seconds
	end

	return 0, "", "", {} -- OK!
end
```

The main way to track statistics is by using a sliding-time-window based, in-memory stats database, that enables extremely efficient storage and retrieval of three types of statistics:

  • Integer - Counting the number of times relevant events happened
  • HyperLogLog - Counting the uniqueness or cardinality of a data set
  • CountMin - Distinct count of multiple items in a data set

Country-level IPv4 and v6 IP lookups are available, as are a variety of DNS lookup functions, including querying RBLs.

Many more metrics are available to base decisions on, and are documented in the wforce.conf man page, available on any system with Dovecot Anti-Abuse Shield installed. It also ships with wforce.conf.example, which gives many examples of how to configure the service and policy.

Simple Testing

To report (if you configured with 'webserver("127.0.0.1:8084", "secret")'): 

```
$ for a in {1..101}
  do 
    curl -X POST -H "Content-Type: application/json" --data '{"login":"ahu", "remote": "127.0.0.1",  "pwhash":"1234'$a'", "success":"false"}' \
    http://127.0.0.1:8084/?command=report -u wforce:secret
  done 
```

This reports 101 failed logins for one user, but with different password hashes.

Now to look up if we're still allowed in: 

```
$ curl -X POST -H "Content-Type: application/json" --data '{"login":"ahu", "remote": "127.0.0.1",  "pwhash":"1234"}' \
  http://127.0.0.1:8084/?command=allow -u wforce:super
{"status": -1, "msg": "diffFailedPasswords"}
```

It appears we are not!

You can also provide additional information for use by Anti-Abuse Shield using the optional "attrs" object. An example: 

```
$ curl -X POST -H "Content-Type: application/json" --data '{"login":"ahu", "remote": "127.0.0.1",
"pwhash":"1234", "attrs":{"attr1":"val1", "attr2":"val2"}}' \
  http://127.0.0.1:8084/?command=allow -u wforce:super
{"status": 0, "msg": ""}
```

An example using the optional attrs object using multi-valued attributes: 

```
$ curl -X POST -H "Content-Type: application/json" --data '{"login":"ahu", "remote": "127.0.0.1",
"pwhash":"1234", "attrs":{"attr1":"val1", "attr2":["val2","val3"]}}' \
  http://127.0.0.1:8084/?command=allow -u wforce:super
{"status": 0, "msg": ""}
```

There is also a command to reset the stats for a given login and/or IP Address, using the 'reset' command, the logic for which is also implemented in Lua. The default policy for reset is as follows: 

```
function reset(type, login, ip)
	 sdb = getStringStatsDB("OneHourDB")
	 if (string.find(type, "ip"))
	 then
		sdb:twReset(ip)
	 end
	 if (string.find(type, "login"))
	 then
		sdb:twReset(login)
	 end
	 if (string.find(type, "ip") and string.find(type, "login"))
	 then
		iplogin = ip:tostring() .. login
		sdb:twReset(iplogin)
	 end
	 return true
end
```

To test it out, try the following to reset the login 'ahu': 

```
$ curl -X POST -H "Content-Type: application/json" --data '{"login":"ahu"}'\
  http://127.0.0.1:8084/?command=reset -u wforce:super
{"status": "ok"}
```

You can reset IP addresses also: 

```
$ curl -X POST -H "Content-Type: application/json" --data '{"ip":"128.243.21.16"}'\
  http://127.0.0.1:8084/?command=reset -u wforce:super
{"status": "ok"}
```

Or both in the same command (this helps if you are tracking stats using compound keys combining both IP address and login): 

```
$ curl -X POST -H "Content-Type: application/json" --data '{"login":"ahu",  "ip":"FE80::0202:B3FF:FE1E:8329"}'\
  http://127.0.0.1:8084/?command=reset -u wforce:super
{"status": "ok"}
```

You can retrive all the know stats for a given IP or login with the 'getDBStats' command: 

```
$ curl -X POST -H "Content-Type: application/json" --data '{"ip":"127.0.0.1"}'\
  http://127.0.0.1:8084/?command=getDBStats -u wforce:super
{"blacklisted": false, "ip": "127.0.0.1", "stats": {"OneHourDB": {"diffFailedPasswords": 1}}}
```

There is also a "ping" command, to check the server is up and answering requests: 

```
$ curl -X GET http://127.0.0.1:8084/?command=ping -u wforce:super
{"status": "ok"}
```

Console

Available over TCP/IP, like this: 

```
setKey("Ay9KXgU3g4ygK+qWT0Ut4gH8PPz02gbtPeXWPdjD0HE=")
controlSocket("0.0.0.0:4004")
```

Launch wforce as a daemon (`wforce --daemon`), to connect, run `wforce -c`. Comes with autocomplete and command history. If you put an actual IP address in place of 0.0.0.0, you can use the same config to listen and connect remotely.

To get some stats, try: 

```
> stats()
40 reports, 8 allow-queries, 40 entries in database
```

Load balancing: siblings

For high-availability or performance reasons it may be desirable to run multiple instances of Anti Abuse Shield. To present a unified view of status however, these instances then need to share the data from reports. To do so, wforce implements a simple knowledge-sharing system.

Tuples received are broadcast (best effort, UDP) to all siblings. The sibling list is parsed such that we don't broadcast messages to ourselves accidentally, and can thus be identical across all servers.

To define siblings, use: 

```
setKey("Ay9KXgU3g4ygK+qWT0Ut4gH8PPz02gbtPeXWPdjD0HE=")
addSibling("192.168.1.79")
addSibling("192.168.1.30")
addSibling("192.168.1.54")
siblingListener("0.0.0.0")
```

The first line sets the authentication and encryption key for our sibling communications. To make your own key (recommended), run `makeKey()` on the console and paste the output in all your configuration files.

This last line configures that we also listen to our other siblings (which is nice). The default port is 4001, the protocol is UDP.

To view sibling stats: 

```
> siblings()
Address                             Sucesses  Failures     Note
192.168.1.79:4001                   18        7            
192.168.1.30:4001                   25        0            
192.168.1.54:4001                   0         0            Self
```

With this setup, several wforces are all kept in sync, and can be load balanced behind for example haproxy, which incidentally can also offer SSL.

Retrieved from "https://wiki.open-xchange.com/wiki/index.php?title=AppSuite:OX_Abuse_Shield&oldid=22023"