% WFORCE_POLICY(8) % Dovecot Oy % 2019
The default policy that comes with wforce is necessarily simple. This policy is designed to be more feature-rich and customizable, and deployable ideally straight out of the box.
The wforce policy is designed to solve the following problems:
The policy has two main feature areas; policies which act based on user-behaviour (i.e. detecting abuse behaviour), and policies which act on other information such as blacklists or whitelists.
The behavioural policies are based on recording statistics using the in-memory stats DB, using behaviour over the last hour as the main metric. Additionally, a 24-hour DB is also used to track the behaviour of abusers who consistently exceed the thresholds of the one-hour policies, and to whom more draconian actions can be applied.
The file config/wforce_config.lua contains the default values for all the policies. Changes to the policies are not made by changing this file, instead a table with the modified policies is passed to the init function of the config object. This is shown in the following example from the default wforce.conf file:
local config = require("config.wforce_config")
-- override the default values with the supplied table - see config/wforce_config.lua
config.initConfig({})
The above example shows an empty table being passed, which means to accept all the defaults. To change some of the defaults, only the keys which need to be changed are passed. For example:
local override_config = {
wforce_1hr_policies = {
maxDiffBadPasswordsPerIP = {
enabled = false
}
},
general_policies = {
countryBlacklist = {
policy_params = {"NG"},
}
},
diffCountryWL = { "NO", "SE", "DK", "FI" }
}
config.initConfig(override_config)
You can also set configuration on a per-domain basis (the login name
will be parsed, and if it is of the form
local override_config = {
wforce_1hr_policies = {
maxDiffBadPasswordsPerIP = {
enabled = false
}
}
}
config.initDomainConfig("foo.com", override_config)
The initDomainConfig() command will first copy the default config (including any changes made in initConfig()) before merging the changes supplied. So, only changes from the default configuration need to be supplied, as shown above.
Most policies are configured in the same way, as shown in the following example:
maxDiffBadPasswordsPerIPUser = {
enabled = true,
threshold = 3,
action = "tarpit",
action_value = 5,
policy_code = 503,
policy_text = "maxDiffBadPasswordsPerIPUser"
},
Not shown above:
The following one-hour policies can be enabled and configured. For each policy, the threshold at which the policy applies, and the action taken for that policy can be changed, as well as whether the policy itself is enabled or disabled.
Policy actions are as follows:
Policies (all thresholds are applied over a rolling one-hour period):
maxDiffBadPasswordsPerIP - This counts the number of different failed passwords used in the last hour by a single IP address. The default threshold is 30, and the default action is “reject”.
maxDiffBadPasswordsPerUser - This counts the number of different failed passwords used in the last hour by a single user/login. The default threshold is 9, and the default action is “reject”.
maxDiffBadPasswordsPerIPUser - This counts the number of different failed passwords used in the last hour by a single user/login and IP address (i.e. both must match). The default threshold is 3, and the default action is “tarpit” for 5 seconds.
maxDiffIPsPerUser - This counts the number of different IP addresses used in the last hour by a particular login. The default threshold is 10, and the default action is “reject”.
maxDiffLoginsPerIP - This counts the number of different usernames used in the last hour by a particular IP address. The default threshold is 20, and the default action is “reject”.
maxLoginsPerIP - This counts the total number of logins (successful or unsuccessful) in the last hour by an individual IP address. The default threshold is 600 (which equates to one login every 6 seconds), and the default action is “tarpit” for 5 seconds.
maxLoginsPerUser - This counts the total number of logins (successful or unsuccessful) in the last hour by a single user. The default threshold is 240 (which equates to one login every 15 seconds), and the default action is “tarpit” for 5 seconds.
maxFailedLoginsPerIP - This counts the total number of unsuccessful logins in the last hour by an individual IP address. The default threshold is 300 (which equates to one failed login every 12 seconds), and the default action is “tarpit” for 5 seconds.
maxFailedLoginsPerUser - This counts the total number of unsuccessful logins in the last hour by a single user. The default threshold is 120 (which equates to one failed login every 30 seconds), and the default action is “tarpit” for 5 seconds.
maxDiffCountriesPerUser - This counts the number of different countries that a user has logged in from (using the GeoIP Country DB) in the last hour. A whitelist of countries to ignore when tracking different countries can be configured. The default threshold is 3 (i.e. logging in from 4 different countries will trigger the policy), and the default action is “reject”.
A note on statistics - all IPv6 addresses are aggregated by the policy to a /64.
The following policies track abuse over a 24 hour period. Note that all the following policies are based on counting rejections caused by the previous policies. Any policies that do not cause a reject action (for the reject policies), or a tarpit action (for the tarpit policies) will not be tracked (note that the blacklist* actions do generate a reject action).
In general the default actions for these policies will blacklist the offending IP/login address; the more frequent the abuse, the longer the blacklist will be applied for.
Although these policies track rejects over a 24-hour period, there are in fact two different thresholds for each policy. One threshold is for the current hour, and another is for the entire 24-hour period. The default action of the current hour policies is to blacklist for an hour, wherease the default action of the 24-hour policies is to blacklist for 24 hours. These values can all be configured of course if the administrator wants to choose different values.
maxRejectsPerHourPerIP - This counts the number of rejects for a given IP address in the current hour. The default threshold is 60, and the default action is “blacklistip” for one hour.
maxRejectsPerHourPerUser - This counts the number of rejects for a given username in the current hour. The default threshold is 30 and the default action is “blacklistlogin” for one hour.
maxRejectsPerHourPerIPUser - This counts the number of rejects for a given IP address/username tuple in the current hour. The default threshold is 15 and the default action is “blacklistiplogin” for one hour.
maxRejectsPerDayPerIP - This counts the number of rejects for a given IP address in the last 24 hours. The default threshold is 200, and the default action is “blacklistip” for 24 hours.
maxRejectsPerDayPerUser - This counts the number of rejects for a given username in the last 24 hours. The default threshold is 100, and the default action is “blacklistlogin” for 24 hours.
maxRejectsPerDayPerIPUser - This counts the number of rejects for a given IP address/username tuple in the last 24 hours. The default threshold is 50, and the default action is “blacklistiplogin” for 24 hours.
maxTarpitsPerHourPerIP - This counts the number of tarpits for a given IP address in the current hour. The default threshold is 120, and the default action is “blacklistip” for 30 mins.
maxTarpitsPerHourPerUser - This counts the number of tarpits for a given username in the current hour. The default threshold is 60 and the default action is “blacklistlogin” for 30 mins.
maxTarpitsPerHourPerIPUser - This counts the number of tarpits for a given IP address/username tuple in the current hour. The default threshold is 30 and the default action is “blacklistiplogin” for 30 mins.
maxTarpitsPerDayPerIP - This counts the number of tarpits for a given IP address in the last 24 hours. The default threshold is 400, and the default action is “blacklistip” for 24 hours.
maxTarpitsPerDayPerUser - This counts the number of tarpits for a given username in the last 24 hours. The default threshold is 200, and the default action is “blacklistlogin” for 24 hours.
maxTarpitsPerDayPerIPUser - This counts the number of tarpits for a given IP address/username tuple in the last 24 hours. The default threshold is 100, and the default action is “blacklistiplogin” for 24 hours.
Note that rejects or tarpits caused by the 24-hour policies themselves are not counted when calculating the above statistics.
Non-behavioural policies are as follows:
The built-in blacklist check normally runs in wforce before the Lua policy engine is called. In order to disable this, and enable the blacklist to be checked from Lua instead, use the following configuration:
disableBuiltinBlacklists()
local override_config = {
checkBuiltInBlacklists = true
}
config.initConfig(override_config)
This can be useful if Lua-based whitelisting is used rather than the built-in whitelists (available only from wforce 2.1 onwards), since otherwise the built-in blacklist overrides the Lua-based whitelist.
The whitelist consists of a list of IP addresses (these can be in CIDR format), and/or usernames that will be whitelisted. By default “127.0.0.0/8”, “postmaster” and “abuse” will be whitelisted.
The IPs and usernames to be whitelisted can be configured in one of two ways.
Firstly by passing a table containing the required IPs and usernames to the “loadWhitelists()” function:
config.loadWhitelists({ip_wl = { "127.0.0.1/8" }, ipe_wl = { "127.0.2.1" }, login_wl = { "postmaster", "abuse" }})
The ip_wl table contains the IP whitelist. The ipe_wl table contains a set of exceptions to the IP whitelist. The login_wl table contains the login whitelist.
Secondly, they can be loaded from a file:
config.loadWhitelistsFromFile("whitelists.lua")
The file must have the following format:
ip_wl = { "127.0.0.0/8", "192.168.0.0/16" }
ipe_wl = { "127.0.2.1" }
login_wl = { "postmaster", "abuse" }
If the whitelist is changed while wforce is running, it can be reloaded simply by typing “config.loadWhitelistsFromFile(filename)” at the wforce console (or running the console as follows: wforce -c -e “config.loadWhitelistsFromFile(filename)”.
You can check if an IP or login are in the whitelist from your own functions by using:
if (config.checkIPWhitelist(ip)) then XXX end
if (config.checkIPWhitelistExceptions(ip)) then XXX end
if (config.checkLoginWhitelist(login)) then XXX end
Remember that wforce will change working directory to that of the specified configuration file, so filenames can be relative to that point.
The concept of delayed whitelisting is supported; when this is enabled, then for whitelisted IPs, rather than short-circuiting the policy, the full policy is run and the log message that would have been generated is logged , prepended with “Whitelisted IP or login:”. This allows examination of the logs to see what whitelisted IPs or users are doing, and what policies they would hit if they were not whitelisted.
To enable delayed whitelisting, set the “delayWhitelisting” config key to true.
The IP graylist consists of a list of IP addresses (these can be in CIDR format) that will be graylisted. Graylisted IPs will never be rejected or blacklisted, only tarpitted.
The number of seconds that graylisted IPs will be tarpitted for is configured using the “graylist_tarpit_secs” configuration field, e.g.
config.initConfig({graylist_tarpit_secs=5})
The IPs to be graylisted can be configured in one of two ways.
Firstly by passing a table containing the required IPs to the “loadGraylist()” function:
config.loadGraylist({ip_gl = { "127.0.0.1/8" }})
Secondly, they can be loaded from a file:
config.loadGraylistFromFile("graylist.lua")
The file must have the following format:
ip_gl = { "127.0.0.0/8", "192.168.0.0/16" }
If the graylist is changed while wforce is running, it can be reloaded simply by typing “config.loadGraylistFromFile(filename)” at the wforce console (or running the console as follows: wforce -c -e “config.loadGraylistFromFile(filename)”.
You can check if an IP is in the graylist from your own functions by using:
if (config.checkIPGraylist(ip)) then XXX end
Remember that wforce will change working directory to that of the specified configuration file, so filenames can be relative to that point.
The country blacklist consists of a list of countries that will be blacklisted by the policy. Any IPs matching those countries will be subject to the specified action. The default action is “reject”. The country list must be specified using the ISO Alpha-2 (i.e. 2-digit) country codes. The country codes are specified using the “policy_params” argument for the “countryBlacklist” policy, e.g.
countryBlacklist = {
enabled = true,
policy_params = { "NG", "JP", "US" }, -- A comma-separated list of 2-digit country codes that will be blacklisted
action = "reject",
policy_code = 551,
policy_text = "countryBlacklist"
}
Multiple RBLs can be configured, in order to reject connections from known-bad IPs. These are configured as follows:
wforce_enable_rbls = true,
wforce_rbls = {
spamhaus =
{ zone = "authbl.spamhaus.org",
ret = { "127.0.0.10" },
msg = "Your IP address is on the spamhaus auth blacklist, please visit https://www.spamhaus.org/lookup/"
},
mybl =
{ zone = "myisp.bl",
ret = { "127.0.0.2", "127.0.0.3" },
msg = "You have been blacklisted by myzone, please contact myisp support at https://myisp.net/support/blacklist"
}
}
It is possible to configure a custom fuction, which will be called whenever an IP matches an RBL zone. An example function follows:
local function processRBLBlock(lt, ret_msg, match_zone, match_ip, config)
return -1, ret_msg, "Remote IP matches RBL", {rbl_zone = match_zone, match_ip = match_ip}
end
Where the parameters are as follows:
It returns the following:
N.B. RBLs cannot be overridden on a per-domain basis - only the RBL configuration defined in initConfig() will be used.
The testMode config parameter can be set to true, in which the return value from the allow function will always be 0. Logging will be the same as if testMode was disabled, except that the key “test_mode=1” will be set in ret_attrs.
Wforce supports sending report data to external entities. Specifically now there is support for the following:
A typical configuration is for wforce to send all reports and allow commands to both Elasticsearch and trackalert, which is configured as follows (note that both are disabled by default):
elasticsearch = {
enabled = true,
commands = { "report", "allow"},
logstash_server = "http://127.0.0.1:8080",
webhook_secret = "secret",
webhook_basic_auth = "foo:bar"
},
trackalert = {
enabled = true,
trackalert_server = "http://127.0.0.1:8085",
webhook_secret = "secret",
webhook_basic_auth = "foo:bar"
}
The trackalert daemon will (optionally) cache information about previous successful logins in a Redis DB. This information can be queried by wforce in order to return a “suspiciousLogin: 1” field in the return data for allow requests, for example when a login is detected which uses a new device or IP address. The “suspiciousLogin” field can then be used by clients to perform extra checks, for example Second Factor Authentication.
To enable this behaviour in either daemon, configure the following, changing the values as appropriate (note that this is disabled by default):
redis = {
redis_enabled = true,
expire_secs = 5184000, -- 60 days expiry for redis cache entries
redis_server = "127.0.0.1", -- can also be a hostname
redis_username = "foo", -- optional if redis server is only configured with password
redis_password = "secret", -- optional is redis is configured with no authentication
redis_port = 6379,
}
If you want to use the standardized policy framework described in this document, but you have some local customization you’d like to make, this is achieveable using several mechanisms. Ideally you will use the existing policy framework, because this has a lot of benefits in terms of logging, white/graylisting and reporting. To do this there are several hooks that enable you to add new policies, new fields to the stats db, and even write your own lua functions.
The easiest way to add a new policy is to use the addOneHourPolicy() function, which is part of the config.wforce_config module.
This function looks like:
config.addOneHourPolicy("policyName", "fieldName", "ip")
This will register a new policy, which acts on the field name specified, using the key specified. The field name can be an existing field in the one hour statss db, or a custom field (see addOneHourCustomField(). The key can be “ip”, “login” or “iplogin”.
By itself this function will not achieve anything however because it will not be enabled. To enable the policy, you’ll need to add all the configuration for that policy using initConfig(). For example:
config.addOneHourPolicy("myNewPolicyByIP", "myNewField", "ip")
local override_config = {
wforce_1hr_policies = {
myNewPolicyByIP = {
enabled = true,
threshold = 70,
action = "reject",
policy_code = 9001,
policy_text = "myNewPolicyByIP",
}
}
}
The above will cause a reject action if the value of “myNewField” goes above 70 for any IP address.
When adding new policies it is often the case that a new field is required to be tracked in the stats db. The function addOneHourCustomField() can be used to do this, as follows:
addOneHourCustomField("fieldName", "int")
The second parameter can be “int”, “hll” or “countmin”.
The field will not be updated unless you write some Lua code to update it. This can be done using the “report prefunc” and “allow prefunc” hooks (see below).
If you require to perform some complex login in your policy, and not use a simple single field threshold, then you can create entirely custom policies using the addOneHourCustomPolicy() function. This is a hook that will run as part of the policy, and returns true if the policy matches and false if not. If returning true, a table is also returned with key-value pairs which will be added to the JSON returned to the client as well as being logged.
addOneHourCustomPolicy("policyName", custom_function)
local function badLoginPercent(hourdb, daydb, lt, policy, config)
local debugLog = config.debugLog
local statsdb = hourdb
local diff_logins = statsdb:twGet(lt.remote, "diffLogins")
local total_logins = statsdb:twGet(lt.remote, "numLogins")
local failed_logins = statsdb:twGet(lt.remote, "numFailedLogins")
local percent_failed = failed_logins/total_logins*100
local percent_success = 100 - percent_failed
local min_logins = policy.min_logins
local percent_failed_threshold = policy.percent_failed_threshold
local difflogins_threshold = policy.diff_logins_threshold
if tostring(percent_failed) == "-nan" or tostring(percent_failed) == "nan" then percent_failed=0 end;
if total_logins > min_logins and percent_failed > percent_failed_threshold and diff_logins > difflogins_threshold
then
if (debugLog) then infoLog("Combined threshold exceeded for policy badLoginPercent actual values are total logins ".. total_logins .. " percent failed " .. percent_failed .. " different logins ".. diff_logins, {}) end
return true, { percent_failed=percent_failed, diff_logins=diff_logins, total_logins=total_logins }
else
if (debugLog) then infoLog("Combined threshold NOT exceeded for policy badLoginPercent actual values are total logins ".. total_logins .. " percent failed " .. percent_failed .. " different logins ".. diff_logins, {}) end
return false
end
end
config.initConfig({
wforce_1hr_policies = {
badLoginPercentByIP = {
enabled = true,
percent_failed_threshold = 70,
diff_logins_threshold = 3,
action = "reject",
policy_code = 9010,
policy_text = "badLoginPercentByIP",
min_logins = 10
}
}
)
config.addOneHourCustomPolicy("policyName", custom_function)
The report prefunc is configured as follows:
report_prefunc = myReportFunction,
Where “myReportFunction” is a Lua function that is configured as follows:
function myReportFunction(lt)
-- Do the things a report function would normally do such as update statsdb fields
end
See the wforce documentation for more information on configuring report functions.
The allow prefunc is called before the rest of the policy, and is configured as follows:
allow_prefunc = myAllowFunction,
The allow postfunc is called after the rest of the policy, i.e. only if the return result would be 0, and is configured as follows:
allow_postfunc = myAllowFunction,
Where “myAllowFunction” is a Lua function that is configured as follows:
function myAllowFunction(lt)
-- do things
return 0, "", "", {}
end
For the allow prefunc, if the custom function returns any value other than 0 (i.e. “allow”), then the rest of the policy will not run.
See the wforce documentation for more information on configuring allow functions.
The file that contains the configuration defaults is called wforce_config.lua, which contains all the thresholds and actions for the various policies which can be enabled. These can be overridden on a global or per-domain basis with the initConfig() or initDomainConfig() functions.
Functions to load whitelists of IPs and logins are found in wforce_wl.lua.
The wforce.conf file may be modified, e.g. to add ACLs, Siblings, or modify the console crypto key, or the basic authentication password. This is also where the configuration can be overridden, and whitelists loaded.
The Lua “require” command is used to load all the policy files as Lua modules. To ensure this works correctly, wforce will change the working directory to the directory containing the wforce.conf file.
The policy consists of the following files. These will typically be installed in a manner which does not override the default configuration (e.g. in /etc/wforce/example-policy). In order to use the files as part of a standard install, they should be copied (not moved) to /etc/wforce (or wherever wforce is configured to read its configuration from by default). The directory layout must be preserved, otherwise the policy will fail (i.e. the wforce.lua file must be in a subdirectory relative to wforce.conf named “policy”).
File layout:
Files marked * are not designed to be edited by the administrator, so consider these read-only unless you really know what you are doing.