diff --git a/.gitignore b/.gitignore index b697c3dc..76a33e60 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,4 @@ htmlcov *.orig *.rej *.bak +__pycache__ diff --git a/.travis.yml b/.travis.yml index 1a0de9af..ea84432e 100644 --- a/.travis.yml +++ b/.travis.yml @@ -6,6 +6,7 @@ python: - "2.7" - "3.2" - "3.3" + - "pypy" before_install: - sudo apt-get update -qq install: diff --git a/ChangeLog b/ChangeLog index bbec55eb..f340b1dd 100644 --- a/ChangeLog +++ b/ChangeLog @@ -4,51 +4,66 @@ |_| \__,_|_|_/___|_.__/\__,_|_||_| ================================================================================ -Fail2Ban (version 0.9.0a1) 2013/??/?? +Fail2Ban (version 0.9.0a2) 2014/??/?? ================================================================================ -ver. 0.9.0 (2013/??/??) - alpha +ver. 0.9.0 (2014/??/??) - alpha ---------- -Carries all fixes in 0.8.9 and new features and enhancements. Nearly -all development is thanks to Steven Hiscocks (THANKS!) with only -code-review and minor additions from Yaroslav Halchenko. +Carries all fixes, features and enhancements from 0.8.12 with major changes. +Nearly all development is thanks to Steven Hiscocks (THANKS!), merging, +testcases and timezone support from Daniel Black, and code-review and minor +additions from Yaroslav Halchenko. + +The minimum supported python version is now 2.6. If you have python-2.4 or 2.5 +you can use the 0.8.12 version of fail2ban. + +Major changes have occured since version 0.8.12. Please test your +configuration before relying on it. - Refactoring (IMPORTANT -- Please review your setup and configuration): - Yaroslav Halchenko * [..bddbf1e] jail.conf was heavily refactored and now is similar to how it looked on Debian systems: - default action could be configured once for all jails - jails definitions only provide customizations (port, logpath) - no need to specify 'filter' if name matches jail name - Steven Hiscocks * [..5aef036] Core functionality moved into fail2ban/ module. Closes gh-26 + * Added fail2ban persistent database + - default location at /var/lib/fail2ban/fail2ban.sqlite3 + - allows active bans to be reinstated on restart + - log files read from last position after restart + * Added systemd journal backend + - Dependency on python-systemd + - New "journalmatch" option added to filter configs files + - New "systemd-journal" option added to fail2ban-regex + * Added python3 support + * Support %z (Timezone offset) and %f (sub-seconds) support for + datedetector. Enhanced existing date/time have been updated patterns to + support these. ISO8601 now defaults to localtime unless specified otherwise. + Some filters have been change as required to capture these elements in the + right timezone correctly. + - New features: - Steven Hiscocks * [..c7ae460] Multiline failregex. Close gh-54 * [8af32ed] Guacamole filter and support for Apache Tomcat date format - * [..4869186] Python3 support * [..b6059f4] 'timeout' option for actions Close gh-60 and Debian bug #410077. Also it would now capture and include stdout and stderr into logging messages in case of error or at DEBUG loglevel. - Daniel Black and TESTOVIK - * Multiline filter for sendmail-spam. Close gh-418 + * Added action xarf-login-attack to report formatted attack messages + according to the XARF standard (v0.2). Close gh-105 + * Support PyPy - Enhancements - Steven Hiscocks + * Multiline filter for sendmail-spam. Close gh-418 + * Multiline regex for Disconnecting: Too many authentication failures for + root [preauth]\nConnection closed by 6X.XXX.XXX.XXX [preauth] * Replacing use of deprecated API (.warning, .assertEqual, etc) * [..a648cc2] Filters can have options now too * [..e019ab7] Multiple instances of the same action are allowed in the same jail -- use actname option to disambiguate. - Daniel Black - * Support %z (Timezone offset) and %f (sub-seconds) support for - datedetector. Enhanced existing date/time have been updated patterns to - support these. ISO8601 now defaults to localtime unless specified otherwise. - Some filters have been change as required to capture these elements in the - right timezone correctly. ver. 0.8.12 (2013/12/XX) - things-can-only-get-better ----------- @@ -56,15 +71,46 @@ ver. 0.8.12 (2013/12/XX) - things-can-only-get-better - IMPORTANT incompatible changes: - Fixes: + - Rename firewall-cmd-direct-new to firewall-cmd-new to fit within jail name + name length. As per gh-395 - allow for ",milliseconds" in the custom date format of proftpd.log - allow for ", referer ..." in apache-* filter for apache error logs. + - allow for spaces at the beginning of kernel messages. Closes gh-448 + - recidive jail to block all protocols. Closes gh-440. Thanks Ioan Indreias + - smtps not a IANA standard and has been removed from Arch. Replaced with + 465. Thanks Stefan. Closes gh-447 + - mysqld-syslog-iptables rule was too long. Part of gh-447. + - add 'flushlogs' command to allow logrotation without clobbering logtarget + settings. Closes gh-458, Debian bug #697333, Redhat bug #891798. + - complain action - ensure where not matching other IPs in log sample. + Closes gh-467 + - Fix firewall-cmd actioncheck - patch from Adam Tkac. Redhat Bug #979622 + - Fix apache-common for apache-2.4 log file format. Thanks Mark White. + Closes gh-516 + - Asynchat changed to use push method which verifys whether all data was + send. This ensures that all data is sent before closing the connection. + +- Enhancements: + - added firewallcmd-ipset action + - long names on jails documented based on iptables limit of 30 less + len("fail2ban-"). + - remove indentation of name and loglevel while logging to SYSLOG to + resolve syslog(-ng) parsing problems. Closes Debian bug #730202. + - added squid filter. Thanks Roman Gelfand. + - updated check_fail2ban to return performance data for all jails. + - filter apache-noscript now includes php cgi scripts. + Thanks dani. Closes gh-503 + - added ufw action. Thanks Guilhem Lettron. lp-#701522 + - exim-spam filter to match spamassassin log entry for option SAdevnull. + Thanks Ivo Truxa. Closes gh-533 + - filter.d/nsd.conf -- also amended Unix date template to match nsd format + - loglines now also report "[PID]" after the name portion - New Features: - Daniel Black - * filter.d/solid-pop3d -- added thanks to Jacques Lav!gnotte on mailinglist. - -- Enhancements: + - Added filter for solid-pop3d -- thanks to Jacques Lav!gnotte on mailinglist. + - Added filter for apache-modsecurity + - Added filter for openwebmail thanks Ivo Truxa. Closes gh-543 ver. 0.8.11 (2013/11/13) - loves-unittests-and-tight-DoS-free-filter-regexes diff --git a/DEVELOP b/DEVELOP index b45c0524..dc34340b 100644 --- a/DEVELOP +++ b/DEVELOP @@ -34,465 +34,7 @@ When submitting pull requests on GitHub we ask you to: * Include a change to the relevant section of the ChangeLog; and * Include yourself in THANKS if not already there. -Filters -======= - -Filters are tricky. They need to: -* work with a variety of the versions of the software that generates the logs; -* work with the range of logging configuration options available in the - software; -* work with multiple operating systems; -* not make assumptions about the log format in excess of the software - (e.g. do not assume a username doesn't contain spaces and use \S+ unless - you've checked the source code); -* account for how future versions of the software will log messages - (e.g. guess what would happen to the log message if different authentication - types are added); -* not be susceptible to DoS vulnerabilities (see Filter Security below); and -* match intended log lines only. - -Please follow the steps from Filter Test Cases to Developing Filter Regular -Expressions and submit a GitHub pull request (PR) afterwards. If you get stuck, -you can push your unfinished changes and still submit a PR -- describe -what you have done, what is the hurdle, and we'll attempt to help (PR -will be automagically updated with future commits you would push to -complete it). - -Filter test cases ------------------ - -Purpose: - -Start by finding the log messages that the application generates related to -some form of authentication failure. If you are adding to an existing filter -think about whether the log messages are of a similar importance and purpose -to the existing filter. If you were a user of Fail2Ban, and did a package -update of Fail2Ban that started matching new log messages, would anything -unexpected happen? Would the bantime/findtime for the jail be appropriate for -the new log messages? If it doesn't, perhaps it needs to be in a separate -filter definition, for example like exim filter aims at authentication failures -and exim-spam at log messages related to spam. - -Even if it is a new filter you may consider separating the log messages into -different filters based on purpose. - -Cause: - -Are some of the log lines a result of the same action? For example, is a PAM -failure log message, followed by an application specific failure message the -result of the same user/script action? If you add regular expressions for -both you would end up with two failures for a single action. -Therefore, select the most appropriate log message and document the other log -message) with a test case not to match it and a description as to why you chose -one over another. - -With the selected log lines consider what action has caused those log -messages and whether they could have been generated by accident? Could -the log message be occurring due to the first step towards the application -asking for authentication? Could the log messages occur often? If some of -these are true make a note of this in the jail.conf example that you provide. - -Samples: - -It is important to include log file samples so any future change in the regular -expression will still work with the log lines you have identified. - -The sample log messages are provided in a file under testcases/files/logs/ -named identically as the corresponding filter (but without .conf extension). -Each log line should be preceded by a line with failJSON metadata (so the logs -lines are tested in the test suite) directly above the log line. If there is -any specific information about the log message, such as version or an -application configuration option that is needed for the message to occur, -include this in a comment (line beginning with #) above the failJSON metadata. - -Log samples should include only one, definitely not more than 3, examples of -log messages of the same form. If log messages are different in different -versions of the application log messages that show this are encouraged. - -Also attempt to inject an IP into the application (e.g. by specifying -it as a username) so that Fail2Ban possibly detects the IP -from user input rather than the true origin. See the Filter Security section -and the top example in testcases/files/logs/apache-auth as to how to do this. -One you have discovered that this is possible, correct the regex so it doesn't -match and provide this as a test case with "match": false (see failJSON below). - -If the mechanism to create the log message isn't obvious provide a -configuration and/or sample scripts testcases/files/config/{filtername} and -reference these in the comments above the log line. - -FailJSON metadata: - -A failJSON metadata is a comment immediately above the log message. It will -look like: - -# failJSON: { "time": "2013-06-10T10:10:59", "match": true , "host": "93.184.216.119" } - -Time should match the time of the log message. It is in a specific format of -Year-Month-Day'T'Hour:minute:Second. If your log message does not include a -year, like the example below, the year should be listed as 2005, if before Sun -Aug 14 10am UTC, and 2004 if afterwards. Here is an example failJSON -line preceding a sample log line: - -# failJSON: { "time": "2005-03-24T15:25:51", "match": true , "host": "198.51.100.87" } -Mar 24 15:25:51 buffalo1 dropbear[4092]: bad password attempt for 'root' from 198.51.100.87:5543 - -The "host" in failJSON should contain the IP or domain that should be blocked. - -For long lines that you do not want to be matched (e.g. from log injection -attacks) and any log lines to be excluded (see "Cause" section above), set -"match": false in the failJSON and describe the reason in the comment above. - -After developing regexes, the following command will test all failJSON metadata -against the log lines in all sample log files - -./fail2ban-testcases testSampleRegex - -Developing Filter Regular Expressions -------------------------------------- - -Date/Time: - -At the moment, Fail2Ban depends on log lines to have time stamps. That is why -before starting to develop failregex, check if your log line format known to -Fail2Ban. Copy the time component from the log line and append an IP address to -test with following command: - -./fail2ban-regex "2013-09-19 02:46:12 1.2.3.4" "" - -Output of such command should contain something like: - -Date template hits: -|- [# of hits] date format -| [1] Year-Month-Day Hour:Minute:Second - -Ensure that the template description matches time/date elements in your log line -time stamp. If there is no matched format then date template needs to be added -to server/datedetector.py. Ensure that a new template is added in the order -that more specific matches occur first and that there is no confusion between a -Day and a Month. - -Filter file: - -The filter is specified in a config/filter.d/{filtername}.conf file. Filter file -can have sections INCLUDES (optional) and Definition as follows: - -[INCLUDES] - -before = common.conf - -after = filtername.local - -[Definition] - -failregex = .... - -ignoreregex = .... - -This is also documented in the man page jail.conf (section 5). Other definitions -can be added to make failregex's more readable and maintainable to be used -through string Interpolations (see http://docs.python.org/2.7/library/configparser.html) - - -General rules: - -Use "before" if you need to include a common set of rules, like syslog or if -there is a common set of regexes for multiple filters. - -Use "after" if you wish to allow the user to overwrite a set of customisations -of the current filter. This file doesn't need to exist. - -Try to avoid using ignoreregex mainly for performance reasons. The case when you -would use it is if in trying to avoid using it, you end up with an unreadable -failregex. - -Syslog: - -If your application logs to syslog you can take advantage of log line prefix -definitions present in common.conf. So as a base use: - -[INCLUDES] - -before = common.conf - -[Definition] - -_daemon = app - -failregex = ^%(__prefix_line)s - -In this example common.conf defines __prefix_line which also contains the -_daemon name (in syslog terms the service) you have just specified. _daemon -can also be a regex. - -For example, to capture following line _daemon should be set to "dovecot" - -Dec 12 11:19:11 dunnart dovecot: pop3-login: Aborted login (tried to use disabled plaintext auth): rip=190.210.136.21, lip=113.212.99.193 - -and then ^%(__prefix_line)s would match "Dec 12 11:19:11 dunnart dovecot: -". Note it matches the trailing space(s) as well. - -Substitutions (AKA string interpolations): - -We have used string interpolations in above examples. They are useful for -making the regexes more readable, reuse generic patterns in multiple failregex -lines, and also to refer definition of regex parts to specific filters or even -to the user. General principle is that value of a _name variable replaces -occurrences of %(_name)s within the same section or anywhere in the config file -if defined in [DEFAULT] section. - -Regular Expressions: - -Regular expressions (failregex, ignoreregex) assume that the date/time has been -removed from the log line (this is just how fail2ban works internally ATM). - -If the format is like ' error 1.2.3.4 is evil' then you need to match -the < at the start so regex should be similar to '^<> is evil$' using - where the IP/domain name appears in the log line. - -The following general rules apply to regular expressions: - -* ensure regexes start with a ^ and are as restrictive as possible. E.g. do not - use .* if \d+ is sufficient; -* use functionality of Python regexes defined in the standard Python re library - http://docs.python.org/2/library/re.html; -* make regular expressions readable (as much as possible). E.g. - (?:...) represents a non-capturing regex but (...) is more readable, thus - preferred. - -If you have only a basic knowledge of regular repressions we advise to read -http://docs.python.org/2/library/re.html first. It doesn't take long and would -remind you e.g. which characters you need to escape and which you don't. - -Developing/testing a regex: - -You can develop a regex in a file or using command line depending on your -preference. You can also use samples you have already created in the test cases -or test them one at a time. - -The general tool for testing Fail2Ban regexes is fail2ban-regex. To see how to -use it run: - -./fail2ban-regex --help - -Take note of -l heavydebug / -l debug and -v as they might be very useful. - -TIP: Take a look at the source code of the application you are developing - failregex for. You may see optional or extra log messages, or parts there - of, that need to form part of your regex. It may also reveal how some - parts are constrained and different formats depending on configuration or - less common usages. - -TIP: For looking through source code - http://sourcecodebrowser.com/ . It has - call graphs and can browse different versions. - -TIP: Some applications log spaces at the end. If you are not sure add \s*$ as - the end part of the regex. - -If your regex is not matching, http://www.debuggex.com/?flavor=python can help -to tune it. fail2ban-regex -D ... will present Debuggex URLs for the regexs -and sample log files that you pass into it. - -In general use when using regex debuggers for generating fail2ban filters: -* use regex from the ./fail2ban-regex output (to ensure all substitutions are -done) -* replace with (?&.ipv4) -* make sure that regex type set to Python -* for the test data put your log output with the date/time removed - -When you have fixed the regex put it back into your filter file. - -Please spread the good word about Debuggex - Serge Toarca is kindly continuing -its free availability to Open Source developers. - -Finishing up: - -If you've added a new filter, add a new entry in config/jail.conf. The theory -here is that a user will create a jail.local with [filtername]\nenable=true to -enable your jail. - -So more specifically in the [filter] section in jail.conf: -* ensure that you have "enabled = false" (users will enable as needed); -* use "filter =" set to your filter name; -* use a typical action to disable ports associated with the application; -* set "logpath" to the usual location of application log file; -* if the default findtime or bantime isn't appropriate to the filter, specify - more appropriate choices (possibly with a brief comment line). - -Submit github pull request (See "Pull Requests" above) for -github.com/fail2ban/fail2ban containing your great work. - -Filter Security ---------------- - -Poor filter regular expressions are susceptible to DoS attacks. - -When a remote user has the ability to introduce text that would match filter's -failregex, while matching inserted text to the part, they have the -ability to deny any host they choose. - -So the part must be anchored on text generated by the application, and -not the user, to an extent sufficient to prevent user inserting the entire text -matching this or any other failregex. - -Ideally filter regex should anchor at the beginning and at the end of log line. -However as more applications log at the beginning than the end, anchoring the -beginning is more important. If the log file used by the application is shared -with other applications, like system logs, ensure the other application that use -that log file do not log user generated text at the beginning of the line, or, -if they do, ensure the regexes of the filter are sufficient to mitigate the risk -of insertion. - - -Examples of poor filters ------------------------- - -1. Too restrictive - -We find a log message: - - Apr-07-13 07:08:36 Invalid command fial2ban from 1.2.3.4 - -We make a failregex - - ^Invalid command \S+ from - -Now think evil. The user does the command 'blah from 1.2.3.44' - -The program diligently logs: - - Apr-07-13 07:08:36 Invalid command blah from 1.2.3.44 from 1.2.3.4 - -And fail2ban matches 1.2.3.44 as the IP that it ban. A DoS attack was successful. - -The fix here is that the command can be anything so .* is appropriate. - - ^Invalid command .* from - -Here the .* will match until the end of the string. Then realise it has more to -match, i.e. "from " and go back until it find this. Then it will ban -1.2.3.4 correctly. Since the is always at the end, end the regex with a $. - - ^Invalid command .* from $ - -Note if we'd just had the expression: - - ^Invalid command \S+ from $ - -Then provided the user put a space in their command they would have never been -banned. - -2. Unanchored regex can match other user injected data - -From the Apache vulnerability CVE-2013-2178 -( original ref: https://vndh.net/note:fail2ban-089-denial-service ). - -An example bad regex for Apache: - - failregex = [[]client []] user .* not found - -Since the user can do a get request on: - - GET /[client%20192.168.0.1]%20user%20root%20not%20found HTTP/1.0 -Host: remote.site - -Now the log line will be: - - [Sat Jun 01 02:17:42 2013] [error] [client 192.168.33.1] File does not exist: /srv/http/site/[client 192.168.0.1] user root not found - -As this log line doesn't match other expressions hence it matches the above -regex and blocks 192.168.33.1 as a denial of service from the HTTP requester. - -3. Over greedy pattern matching - -From: https://github.com/fail2ban/fail2ban/pull/426 - -An example ssh log (simplified) - - Sep 29 17:15:02 spaceman sshd[12946]: Failed password for user from 127.0.0.1 port 20000 ssh1: ruser remoteuser - -As we assume username can include anything including spaces its prudent to put -.* here. The remote user can also exist as anything so lets not make assumptions again. - - failregex = ^%(__prefix_line)sFailed \S+ for .* from ( port \d*)?( ssh\d+)?(: ruser .*)?$ - -So this works. The problem is if the .* after remote user is injected by the -user to be 'from 1.2.3.4'. The resultant log line is. - - Sep 29 17:15:02 spaceman sshd[12946]: Failed password for user from 127.0.0.1 port 20000 ssh1: ruser from 1.2.3.4 - -Testing with: - - fail2ban-regex -v 'Sep 29 17:15:02 Failed password for user from 127.0.0.1 port 20000 ssh1: ruser from 1.2.3.4' '^ Failed \S+ for .* from ( port \d*)?( ssh\d+)?(: ruser .*)?$' - -TIP: I've removed the bit that matches __prefix_line from the regex and log. - -Shows: - - 1) [1] ^ Failed \S+ for .* from ( port \d*)?( ssh\d+)?(: ruser .*)?$ - 1.2.3.4 Sun Sep 29 17:15:02 2013 - -It should of matched 127.0.0.1. So the first greedy part of the greedy regex -matched until the end of the string. The was no "from " so the regex -engine worked backwards from the end of the string until this was matched. - -The result was that 1.2.3.4 was matched, injected by the user, and the wrong IP -was banned. - -The solution here is to make the first .* non-greedy with .*?. Here it matches -as little as required and the fail2ban-regex tool shows the output: - - fail2ban-regex -v 'Sep 29 17:15:02 Failed password for user from 127.0.0.1 port 20000 ssh1: ruser from 1.2.3.4' '^ Failed \S+ for .*? from ( port \d*)?( ssh\d+)?(: ruser .*)?$' - - 1) [1] ^ Failed \S+ for .*? from ( port \d*)?( ssh\d+)?(: ruser .*)?$ - 127.0.0.1 Sun Sep 29 17:15:02 2013 - -So the general case here is a log line that contains: - - (fixed_data_1)(fixed_data_2)(user_injectable_data) - -Where the regex that matches fixed_data_1 is gready and matches the entire -string, before moving backwards and user_injectable_data can match the entire -string. - -Another case: - -ref: https://www.debuggex.com/r/CtAbeKMa2sDBEfA2/0 - -A webserver logs the following without URL escaping: - - [error] 2865#0: *66647 user "xyz" was not found in "/file", client: 1.2.3.1, server: www.host.com, request: "GET ", client: 3.2.1.1, server: fake.com, request: "GET exploited HTTP/3.3", host: "injected.host", host: "www.myhost.com" - -regex: - - failregex = ^ \[error\] \d+#\d+: \*\d+ user "\S+":? (?:password mismatch|was not found in ".*"), client: , server: \S+, request: "\S+ .+ HTTP/\d+\.\d+", host: "\S+" - -The .* matches to the end of the string. Finds that it can't continue to match -", client ... so it moves from the back and find that the user injected web URL: - - ", client: 3.2.1.1, server: fake.com, request: "GET exploited HTTP/3.3", host: "injected.host - -In this case there is a fixed host: "www.myhost.com" at the end so the solution -is to anchor the regex at the end with a $. - -If this wasn't the case then first .* needed to be made so it didn't capture -beyond . - -4. Application generates two identical log messages with different meanings - -If the application generates the following two messages under different -circumstances: - - client : authentication failed - client : authentication failed - - -Then it's obvious that a regex of "^client : authentication -failed$" will still cause problems if the user can trigger the second -log message with a of 123.1.1.1. - -Here there's nothing to do except request/change the application so it logs -messages differently. - +If you are developing filters see the FILTERS file for documentation. Code Testing ============ @@ -743,10 +285,14 @@ Releasing * https://github.com/fail2ban/fail2ban/issues?sort=updated&state=open * http://bugs.debian.org/cgi-bin/pkgreport.cgi?dist=unstable;package=fail2ban + * https://bugs.launchpad.net/ubuntu/+source/fail2ban * http://bugs.sabayon.org/buglist.cgi?quicksearch=net-analyzer%2Ffail2ban + * https://bugs.archlinux.org/?project=5&cat%5B%5D=33&string=fail2ban * https://bugs.gentoo.org/buglist.cgi?query_format=advanced&short_desc=fail2ban&bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=IN_PROGRESS&short_desc_type=allwords * https://bugzilla.redhat.com/buglist.cgi?query_format=advanced&bug_status=NEW&bug_status=ASSIGNED&component=fail2ban&classification=Red%20Hat&classification=Fedora * http://www.freebsd.org/cgi/query-pr-summary.cgi?text=fail2ban + * https://bugs.mageia.org/buglist.cgi?quicksearch=fail2ban + * https://build.opensuse.org/package/requests/openSUSE:Factory/fail2ban # Make sure the tests pass diff --git a/FILTERS b/FILTERS new file mode 100644 index 00000000..fd441e58 --- /dev/null +++ b/FILTERS @@ -0,0 +1,469 @@ + __ _ _ ___ _ + / _|__ _(_) |_ ) |__ __ _ _ _ + | _/ _` | | |/ /| '_ \/ _` | ' \ + |_| \__,_|_|_/___|_.__/\__,_|_||_| + +================================================================================ +Developing Filters +================================================================================ + +Filters +======= + +Filters are tricky. They need to: +* work with a variety of the versions of the software that generates the logs; +* work with the range of logging configuration options available in the + software; +* work with multiple operating systems; +* not make assumptions about the log format in excess of the software + (e.g. do not assume a username doesn't contain spaces and use \S+ unless + you've checked the source code); +* account for how future versions of the software will log messages + (e.g. guess what would happen to the log message if different authentication + types are added); +* not be susceptible to DoS vulnerabilities (see Filter Security below); and +* match intended log lines only. + +Please follow the steps from Filter Test Cases to Developing Filter Regular +Expressions and submit a GitHub pull request (PR) afterwards. If you get stuck, +you can push your unfinished changes and still submit a PR -- describe +what you have done, what is the hurdle, and we'll attempt to help (PR +will be automagically updated with future commits you would push to +complete it). + +Filter test cases +----------------- + +Purpose: + +Start by finding the log messages that the application generates related to +some form of authentication failure. If you are adding to an existing filter +think about whether the log messages are of a similar importance and purpose +to the existing filter. If you were a user of Fail2Ban, and did a package +update of Fail2Ban that started matching new log messages, would anything +unexpected happen? Would the bantime/findtime for the jail be appropriate for +the new log messages? If it doesn't, perhaps it needs to be in a separate +filter definition, for example like exim filter aims at authentication failures +and exim-spam at log messages related to spam. + +Even if it is a new filter you may consider separating the log messages into +different filters based on purpose. + +Cause: + +Are some of the log lines a result of the same action? For example, is a PAM +failure log message, followed by an application specific failure message the +result of the same user/script action? If you add regular expressions for +both you would end up with two failures for a single action. +Therefore, select the most appropriate log message and document the other log +message) with a test case not to match it and a description as to why you chose +one over another. + +With the selected log lines consider what action has caused those log +messages and whether they could have been generated by accident? Could +the log message be occurring due to the first step towards the application +asking for authentication? Could the log messages occur often? If some of +these are true make a note of this in the jail.conf example that you provide. + +Samples: + +It is important to include log file samples so any future change in the regular +expression will still work with the log lines you have identified. + +The sample log messages are provided in a file under testcases/files/logs/ +named identically as the corresponding filter (but without .conf extension). +Each log line should be preceded by a line with failJSON metadata (so the logs +lines are tested in the test suite) directly above the log line. If there is +any specific information about the log message, such as version or an +application configuration option that is needed for the message to occur, +include this in a comment (line beginning with #) above the failJSON metadata. + +Log samples should include only one, definitely not more than 3, examples of +log messages of the same form. If log messages are different in different +versions of the application log messages that show this are encouraged. + +Also attempt to inject an IP into the application (e.g. by specifying +it as a username) so that Fail2Ban possibly detects the IP +from user input rather than the true origin. See the Filter Security section +and the top example in testcases/files/logs/apache-auth as to how to do this. +One you have discovered that this is possible, correct the regex so it doesn't +match and provide this as a test case with "match": false (see failJSON below). + +If the mechanism to create the log message isn't obvious provide a +configuration and/or sample scripts testcases/files/config/{filtername} and +reference these in the comments above the log line. + +FailJSON metadata: + +A failJSON metadata is a comment immediately above the log message. It will +look like: + +# failJSON: { "time": "2013-06-10T10:10:59", "match": true , "host": "93.184.216.119" } + +Time should match the time of the log message. It is in a specific format of +Year-Month-Day'T'Hour:minute:Second. If your log message does not include a +year, like the example below, the year should be listed as 2005, if before Sun +Aug 14 10am UTC, and 2004 if afterwards. Here is an example failJSON +line preceding a sample log line: + +# failJSON: { "time": "2005-03-24T15:25:51", "match": true , "host": "198.51.100.87" } +Mar 24 15:25:51 buffalo1 dropbear[4092]: bad password attempt for 'root' from 198.51.100.87:5543 + +The "host" in failJSON should contain the IP or domain that should be blocked. + +For long lines that you do not want to be matched (e.g. from log injection +attacks) and any log lines to be excluded (see "Cause" section above), set +"match": false in the failJSON and describe the reason in the comment above. + +After developing regexes, the following command will test all failJSON metadata +against the log lines in all sample log files + +./fail2ban-testcases testSampleRegex + +Developing Filter Regular Expressions +------------------------------------- + +Date/Time: + +At the moment, Fail2Ban depends on log lines to have time stamps. That is why +before starting to develop failregex, check if your log line format known to +Fail2Ban. Copy the time component from the log line and append an IP address to +test with following command: + +./fail2ban-regex "2013-09-19 02:46:12 1.2.3.4" "" + +Output of such command should contain something like: + +Date template hits: +|- [# of hits] date format +| [1] Year-Month-Day Hour:Minute:Second + +Ensure that the template description matches time/date elements in your log line +time stamp. If there is no matched format then date template needs to be added +to server/datedetector.py. Ensure that a new template is added in the order +that more specific matches occur first and that there is no confusion between a +Day and a Month. + +Filter file: + +The filter is specified in a config/filter.d/{filtername}.conf file. Filter file +can have sections INCLUDES (optional) and Definition as follows: + +[INCLUDES] + +before = common.conf + +after = filtername.local + +[Definition] + +failregex = .... + +ignoreregex = .... + +This is also documented in the man page jail.conf (section 5). Other definitions +can be added to make failregex's more readable and maintainable to be used +through string Interpolations (see http://docs.python.org/2.7/library/configparser.html) + + +General rules: + +Use "before" if you need to include a common set of rules, like syslog or if +there is a common set of regexes for multiple filters. + +Use "after" if you wish to allow the user to overwrite a set of customisations +of the current filter. This file doesn't need to exist. + +Try to avoid using ignoreregex mainly for performance reasons. The case when you +would use it is if in trying to avoid using it, you end up with an unreadable +failregex. + +Syslog: + +If your application logs to syslog you can take advantage of log line prefix +definitions present in common.conf. So as a base use: + +[INCLUDES] + +before = common.conf + +[Definition] + +_daemon = app + +failregex = ^%(__prefix_line)s + +In this example common.conf defines __prefix_line which also contains the +_daemon name (in syslog terms the service) you have just specified. _daemon +can also be a regex. + +For example, to capture following line _daemon should be set to "dovecot" + +Dec 12 11:19:11 dunnart dovecot: pop3-login: Aborted login (tried to use disabled plaintext auth): rip=190.210.136.21, lip=113.212.99.193 + +and then ^%(__prefix_line)s would match "Dec 12 11:19:11 dunnart dovecot: +". Note it matches the trailing space(s) as well. + +Substitutions (AKA string interpolations): + +We have used string interpolations in above examples. They are useful for +making the regexes more readable, reuse generic patterns in multiple failregex +lines, and also to refer definition of regex parts to specific filters or even +to the user. General principle is that value of a _name variable replaces +occurrences of %(_name)s within the same section or anywhere in the config file +if defined in [DEFAULT] section. + +Regular Expressions: + +Regular expressions (failregex, ignoreregex) assume that the date/time has been +removed from the log line (this is just how fail2ban works internally ATM). + +If the format is like ' error 1.2.3.4 is evil' then you need to match +the < at the start so regex should be similar to '^<> is evil$' using + where the IP/domain name appears in the log line. + +The following general rules apply to regular expressions: + +* ensure regexes start with a ^ and are as restrictive as possible. E.g. do not + use .* if \d+ is sufficient; +* use functionality of Python regexes defined in the standard Python re library + http://docs.python.org/2/library/re.html; +* make regular expressions readable (as much as possible). E.g. + (?:...) represents a non-capturing regex but (...) is more readable, thus + preferred. + +If you have only a basic knowledge of regular repressions we advise to read +http://docs.python.org/2/library/re.html first. It doesn't take long and would +remind you e.g. which characters you need to escape and which you don't. + +Developing/testing a regex: + +You can develop a regex in a file or using command line depending on your +preference. You can also use samples you have already created in the test cases +or test them one at a time. + +The general tool for testing Fail2Ban regexes is fail2ban-regex. To see how to +use it run: + +./fail2ban-regex --help + +Take note of -l heavydebug / -l debug and -v as they might be very useful. + +TIP: Take a look at the source code of the application you are developing + failregex for. You may see optional or extra log messages, or parts there + of, that need to form part of your regex. It may also reveal how some + parts are constrained and different formats depending on configuration or + less common usages. + +TIP: For looking through source code - http://sourcecodebrowser.com/ . It has + call graphs and can browse different versions. + +TIP: Some applications log spaces at the end. If you are not sure add \s*$ as + the end part of the regex. + +If your regex is not matching, http://www.debuggex.com/?flavor=python can help +to tune it. fail2ban-regex -D ... will present Debuggex URLs for the regexs +and sample log files that you pass into it. + +In general use when using regex debuggers for generating fail2ban filters: +* use regex from the ./fail2ban-regex output (to ensure all substitutions are +done) +* replace with (?&.ipv4) +* make sure that regex type set to Python +* for the test data put your log output with the date/time removed + +When you have fixed the regex put it back into your filter file. + +Please spread the good word about Debuggex - Serge Toarca is kindly continuing +its free availability to Open Source developers. + +Finishing up: + +If you've added a new filter, add a new entry in config/jail.conf. The theory +here is that a user will create a jail.local with [filtername]\nenable=true to +enable your jail. + +So more specifically in the [filter] section in jail.conf: +* ensure that you have "enabled = false" (users will enable as needed); +* use "filter =" set to your filter name; +* use a typical action to disable ports associated with the application; +* set "logpath" to the usual location of application log file; +* if the default findtime or bantime isn't appropriate to the filter, specify + more appropriate choices (possibly with a brief comment line). + +Submit github pull request (See "Pull Requests" above) for +github.com/fail2ban/fail2ban containing your great work. + +Filter Security +--------------- + +Poor filter regular expressions are susceptible to DoS attacks. + +When a remote user has the ability to introduce text that would match filter's +failregex, while matching inserted text to the part, they have the +ability to deny any host they choose. + +So the part must be anchored on text generated by the application, and +not the user, to an extent sufficient to prevent user inserting the entire text +matching this or any other failregex. + +Ideally filter regex should anchor at the beginning and at the end of log line. +However as more applications log at the beginning than the end, anchoring the +beginning is more important. If the log file used by the application is shared +with other applications, like system logs, ensure the other application that use +that log file do not log user generated text at the beginning of the line, or, +if they do, ensure the regexes of the filter are sufficient to mitigate the risk +of insertion. + + +Examples of poor filters +------------------------ + +1. Too restrictive + +We find a log message: + + Apr-07-13 07:08:36 Invalid command fial2ban from 1.2.3.4 + +We make a failregex + + ^Invalid command \S+ from + +Now think evil. The user does the command 'blah from 1.2.3.44' + +The program diligently logs: + + Apr-07-13 07:08:36 Invalid command blah from 1.2.3.44 from 1.2.3.4 + +And fail2ban matches 1.2.3.44 as the IP that it ban. A DoS attack was successful. + +The fix here is that the command can be anything so .* is appropriate. + + ^Invalid command .* from + +Here the .* will match until the end of the string. Then realise it has more to +match, i.e. "from " and go back until it find this. Then it will ban +1.2.3.4 correctly. Since the is always at the end, end the regex with a $. + + ^Invalid command .* from $ + +Note if we'd just had the expression: + + ^Invalid command \S+ from $ + +Then provided the user put a space in their command they would have never been +banned. + +2. Unanchored regex can match other user injected data + +From the Apache vulnerability CVE-2013-2178 +( original ref: https://vndh.net/note:fail2ban-089-denial-service ). + +An example bad regex for Apache: + + failregex = [[]client []] user .* not found + +Since the user can do a get request on: + + GET /[client%20192.168.0.1]%20user%20root%20not%20found HTTP/1.0 +Host: remote.site + +Now the log line will be: + + [Sat Jun 01 02:17:42 2013] [error] [client 192.168.33.1] File does not exist: /srv/http/site/[client 192.168.0.1] user root not found + +As this log line doesn't match other expressions hence it matches the above +regex and blocks 192.168.33.1 as a denial of service from the HTTP requester. + +3. Over greedy pattern matching + +From: https://github.com/fail2ban/fail2ban/pull/426 + +An example ssh log (simplified) + + Sep 29 17:15:02 spaceman sshd[12946]: Failed password for user from 127.0.0.1 port 20000 ssh1: ruser remoteuser + +As we assume username can include anything including spaces its prudent to put +.* here. The remote user can also exist as anything so lets not make assumptions again. + + failregex = ^%(__prefix_line)sFailed \S+ for .* from ( port \d*)?( ssh\d+)?(: ruser .*)?$ + +So this works. The problem is if the .* after remote user is injected by the +user to be 'from 1.2.3.4'. The resultant log line is. + + Sep 29 17:15:02 spaceman sshd[12946]: Failed password for user from 127.0.0.1 port 20000 ssh1: ruser from 1.2.3.4 + +Testing with: + + fail2ban-regex -v 'Sep 29 17:15:02 Failed password for user from 127.0.0.1 port 20000 ssh1: ruser from 1.2.3.4' '^ Failed \S+ for .* from ( port \d*)?( ssh\d+)?(: ruser .*)?$' + +TIP: I've removed the bit that matches __prefix_line from the regex and log. + +Shows: + + 1) [1] ^ Failed \S+ for .* from ( port \d*)?( ssh\d+)?(: ruser .*)?$ + 1.2.3.4 Sun Sep 29 17:15:02 2013 + +It should of matched 127.0.0.1. So the first greedy part of the greedy regex +matched until the end of the string. The was no "from " so the regex +engine worked backwards from the end of the string until this was matched. + +The result was that 1.2.3.4 was matched, injected by the user, and the wrong IP +was banned. + +The solution here is to make the first .* non-greedy with .*?. Here it matches +as little as required and the fail2ban-regex tool shows the output: + + fail2ban-regex -v 'Sep 29 17:15:02 Failed password for user from 127.0.0.1 port 20000 ssh1: ruser from 1.2.3.4' '^ Failed \S+ for .*? from ( port \d*)?( ssh\d+)?(: ruser .*)?$' + + 1) [1] ^ Failed \S+ for .*? from ( port \d*)?( ssh\d+)?(: ruser .*)?$ + 127.0.0.1 Sun Sep 29 17:15:02 2013 + +So the general case here is a log line that contains: + + (fixed_data_1)(fixed_data_2)(user_injectable_data) + +Where the regex that matches fixed_data_1 is gready and matches the entire +string, before moving backwards and user_injectable_data can match the entire +string. + +Another case: + +ref: https://www.debuggex.com/r/CtAbeKMa2sDBEfA2/0 + +A webserver logs the following without URL escaping: + + [error] 2865#0: *66647 user "xyz" was not found in "/file", client: 1.2.3.1, server: www.host.com, request: "GET ", client: 3.2.1.1, server: fake.com, request: "GET exploited HTTP/3.3", host: "injected.host", host: "www.myhost.com" + +regex: + + failregex = ^ \[error\] \d+#\d+: \*\d+ user "\S+":? (?:password mismatch|was not found in ".*"), client: , server: \S+, request: "\S+ .+ HTTP/\d+\.\d+", host: "\S+" + +The .* matches to the end of the string. Finds that it can't continue to match +", client ... so it moves from the back and find that the user injected web URL: + + ", client: 3.2.1.1, server: fake.com, request: "GET exploited HTTP/3.3", host: "injected.host + +In this case there is a fixed host: "www.myhost.com" at the end so the solution +is to anchor the regex at the end with a $. + +If this wasn't the case then first .* needed to be made so it didn't capture +beyond . + +4. Application generates two identical log messages with different meanings + +If the application generates the following two messages under different +circumstances: + + client : authentication failed + client : authentication failed + + +Then it's obvious that a regex of "^client : authentication +failed$" will still cause problems if the user can trigger the second +log message with a of 123.1.1.1. + +Here there's nothing to do except request/change the application so it logs +messages differently. + + diff --git a/MANIFEST b/MANIFEST index c28137ca..b61604ed 100644 --- a/MANIFEST +++ b/MANIFEST @@ -5,8 +5,9 @@ TODO THANKS COPYING DEVELOP -doc/run-rootless.txt +FILTERS fail2ban-2to3 +fail2ban-testcases-all fail2ban-testcases-all-python3 bin/fail2ban-client bin/fail2ban-server @@ -24,6 +25,7 @@ fail2ban/client/__init__.py fail2ban/client/configurator.py fail2ban/client/csocket.py fail2ban/server/asyncserver.py +fail2ban/server/database.py fail2ban/server/filter.py fail2ban/server/filterpyinotify.py fail2ban/server/filtergamin.py @@ -59,22 +61,30 @@ fail2ban/tests/servertestcase.py fail2ban/tests/sockettestcase.py fail2ban/tests/utils.py fail2ban/tests/misctestcase.py -fail2ban/tests/config/apache-auth/digest/.htaccess -fail2ban/tests/config/apache-auth/digest/.htpasswd -fail2ban/tests/config/apache-auth/digest_time/.htaccess -fail2ban/tests/config/apache-auth/digest_time/.htpasswd -fail2ban/tests/config/apache-auth/basic/authz_owner/.htaccess -fail2ban/tests/config/apache-auth/basic/authz_owner/cant_get_me.html -fail2ban/tests/config/apache-auth/basic/authz_owner/.htpasswd -fail2ban/tests/config/apache-auth/basic/file/.htaccess -fail2ban/tests/config/apache-auth/basic/file/.htpasswd -fail2ban/tests/config/apache-auth/digest.py -fail2ban/tests/config/apache-auth/digest_wrongrelm/.htaccess -fail2ban/tests/config/apache-auth/digest_wrongrelm/.htpasswd -fail2ban/tests/config/apache-auth/digest_anon/.htaccess -fail2ban/tests/config/apache-auth/digest_anon/.htpasswd -fail2ban/tests/config/apache-auth/README -fail2ban/tests/config/apache-auth/noentry/.htaccess +fail2ban/tests/config/jail.conf +fail2ban/tests/config/fail2ban.conf +fail2ban/tests/config/filter.d/simple.conf +fail2ban/tests/config/action.d/brokenaction.conf +fail2ban/tests/files/config/apache-auth/digest/.htaccess +fail2ban/tests/files/config/apache-auth/digest/.htpasswd +fail2ban/tests/files/config/apache-auth/digest_time/.htaccess +fail2ban/tests/files/config/apache-auth/digest_time/.htpasswd +fail2ban/tests/files/config/apache-auth/basic/authz_owner/.htaccess +fail2ban/tests/files/config/apache-auth/basic/authz_owner/cant_get_me.html +fail2ban/tests/files/config/apache-auth/basic/authz_owner/.htpasswd +fail2ban/tests/files/config/apache-auth/basic/file/.htaccess +fail2ban/tests/files/config/apache-auth/basic/file/.htpasswd +fail2ban/tests/files/config/apache-auth/digest.py +fail2ban/tests/files/config/apache-auth/digest_wrongrelm/.htaccess +fail2ban/tests/files/config/apache-auth/digest_wrongrelm/.htpasswd +fail2ban/tests/files/config/apache-auth/digest_anon/.htaccess +fail2ban/tests/files/config/apache-auth/digest_anon/.htpasswd +fail2ban/tests/files/config/apache-auth/README +fail2ban/tests/files/config/apache-auth/noentry/.htaccess +fail2ban/tests/files/database_v1.db +fail2ban/tests/files/ignorecommand.py +fail2ban/tests/files/filter.d/testcase-common.conf +fail2ban/tests/files/filter.d/testcase01.conf fail2ban/tests/files/testcase01.log fail2ban/tests/files/testcase02.log fail2ban/tests/files/testcase03.log @@ -92,6 +102,7 @@ fail2ban/tests/files/logs/dovecot fail2ban/tests/files/logs/exim fail2ban/tests/files/logs/nginx-http-auth fail2ban/tests/files/logs/lighttpd-auth +fail2ban/tests/files/logs/openwebmail fail2ban/tests/files/logs/named-refused fail2ban/tests/files/logs/pam-generic fail2ban/tests/files/logs/postfix @@ -144,6 +155,7 @@ setup.py setup.cfg kill-server config/jail.conf +config/fail2ban.conf config/filter.d/common.conf config/filter.d/apache-auth.conf config/filter.d/apache-badbots.conf @@ -158,13 +170,17 @@ config/filter.d/exim.conf config/filter.d/gssftpd.conf config/filter.d/suhosin.conf config/filter.d/named-refused.conf +config/filter.d/openwebmail.conf +config/filter.d/pam-generic.conf +config/filter.d/php-url-fopen.conf +config/filter.d/postfix-sasl.conf +config/filter.d/pam-generic.conf +config/filter.d/php-url-fopen.conf +config/filter.d/postfix-sasl.conf config/filter.d/postfix.conf config/filter.d/proftpd.conf config/filter.d/pure-ftpd.conf config/filter.d/qmail.conf -config/filter.d/pam-generic.conf -config/filter.d/php-url-fopen.conf -config/filter.d/postfix-sasl.conf config/filter.d/sieve.conf config/filter.d/solid-pop3d.conf config/filter.d/sshd.conf @@ -201,7 +217,8 @@ config/action.d/osx-ipfw.conf config/action.d/sendmail-common.conf config/action.d/bsd-ipfw.conf config/action.d/dummy.conf -config/action.d/firewall-cmd-direct-new.conf +config/action.d/firewallcmd-new.conf +config/action.d/firewallcmd-ipset.conf config/action.d/iptables-ipset-proto6-allports.conf config/action.d/iptables-blocktype.conf config/action.d/iptables-ipset-proto4.conf @@ -229,7 +246,8 @@ config/action.d/sendmail-buffered.conf config/action.d/sendmail-whois.conf config/action.d/sendmail-whois-lines.conf config/action.d/shorewall.conf -config/fail2ban.conf +config/action.d/xarf-login-attack.conf +config/action.d/ufw.conf doc/run-rootless.txt man/fail2ban-client.1 man/fail2ban.1 diff --git a/README.md b/README.md index 129d24f7..2482856f 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ / _|__ _(_) |_ ) |__ __ _ _ _ | _/ _` | | |/ /| '_ \/ _` | ' \ |_| \__,_|_|_/___|_.__/\__,_|_||_| - v0.9.0a0 2013/??/?? + v0.9.0a2 2014/??/?? ## Fail2Ban: ban hosts that cause multiple authentication errors @@ -21,7 +21,7 @@ Installation: this case, you should use it instead.** Required: -- [Python2 >= 2.4 or Python3 >= 3.2](http://www.python.org) +- [Python2 >= 2.6 or Python3 >= 3.2](http://www.python.org) or [PyPy](http://pypy.org) Optional: - [pyinotify >= 0.8.3](https://github.com/seb-m/pyinotify) @@ -31,8 +31,8 @@ Optional: To install, just do: - tar xvfj fail2ban-0.8.11.tar.bz2 - cd fail2ban-0.8.11 + tar xvfj fail2ban-0.9.0.tar.bz2 + cd fail2ban-0.9.0 python setup.py install This will install Fail2Ban into /usr/share/fail2ban. The executable scripts are diff --git a/THANKS b/THANKS index 3d66ebc6..23870ca9 100644 --- a/THANKS +++ b/THANKS @@ -6,13 +6,17 @@ the project. If you have been left off, please let us know (preferably send a pull request on github with the "fix") and you will be added +Adam Tkac Adrien Clerc ache +ag4ve (Shawn) +Alasdair D. Campbell Amir Caspi Andrey G. Grozin Andy Fragen Arturo 'Buanzo' Busleiman Axel Thimm +Bas van den Dikkenberg Beau Raines Bill Heaton Carlos Alberto Lopez Perez @@ -22,6 +26,7 @@ Christoph Haas Christos Psonis Cyril Jaquier Daniel B. Cid +Daniel B. Daniel Black David Nutter Eric Gerbier @@ -30,10 +35,14 @@ ftoppi François Boulogne Frédéric Georgiy Mernov +Guilhem Lettron Guillaume Delvit Hanno 'Rince' Wagner Iain Lea +Ivo Truxa +John Thoe Jacques Lav!gnotte +Ioan Indreias Jonathan Kamens Jonathan Lanning Jonathan Underwood @@ -43,10 +52,12 @@ Justin Shore Kévin Drapel kjohnsonecl kojiro +Lee Clemens Manuel Arostegui Ramirez Marcel Dopita Mark Edgington Mark McKinstry +Mark White Markus Hoffmann Marvin Rouge mEDI @@ -60,10 +71,12 @@ RealRancor René Berber Robert Edeker Rolf Fokkens +Roman Gelfand Russell Odom Sebastian Arcus Sireyessire silviogarbes +Stefan Tatschner Stephen Gildea Steven Hiscocks TESTOVIK diff --git a/bin/fail2ban-regex b/bin/fail2ban-regex index 3661ba36..fc39c958 100755 --- a/bin/fail2ban-regex +++ b/bin/fail2ban-regex @@ -98,7 +98,7 @@ def get_opt_parser(): LOG: string a string representing a log line filename path to a log file (/var/log/auth.log) - "systemd-journal" search systemd journal (systemd-python required) + "systemd-journal" search systemd journal (systemd-python required) REGEX: string a string representing a 'failregex' @@ -239,7 +239,9 @@ class Fail2banRegex(object): if not self._datepattern_set: self._filter.setDatePattern(pattern) self._datepattern_set = True - print "Use datepattern : %s" % self._filter.getDatePattern()[1] + if pattern is not None: + print "Use datepattern : %s" % ( + self._filter.getDatePattern()[1], ) def setMaxLines(self, v): if not self._maxlines_set: @@ -309,7 +311,7 @@ class Fail2banRegex(object): def testIgnoreRegex(self, line): found = False try: - ret = self._filter.ignoreLine(line) + ret = self._filter.ignoreLine([(line, "", "")]) if ret is not None: found = True regex = self._ignoreregex[ret].inc() @@ -318,11 +320,11 @@ class Fail2banRegex(object): return False return found - def testRegex(self, line): + def testRegex(self, line, date=None): orgLineBuffer = self._filter._Filter__lineBuffer fullBuffer = len(orgLineBuffer) >= self._filter.getMaxLines() try: - line, ret = self._filter.processLine(line, checkAllRegex=True) + line, ret = self._filter.processLine(line, date, checkAllRegex=True) for match in ret: # Append True/False flag depending if line was matched by # more than one regex @@ -338,35 +340,32 @@ class Fail2banRegex(object): return False for bufLine in orgLineBuffer[int(fullBuffer):]: if bufLine not in self._filter._Filter__lineBuffer: - if self.removeMissedLine(bufLine): + try: + self._line_stats.missed_lines.pop( + self._line_stats.missed_lines.index("".join(bufLine))) + self._line_stats.missed_lines_timeextracted.pop( + self._line_stats.missed_lines_timeextracted.index( + "".join(bufLine[::2]))) + except ValueError: + pass + else: self._line_stats.matched += 1 return line, ret - def removeMissedLine(self, line): - """Remove `line` from missed lines, by comparing without time match""" - for n, missed_line in \ - enumerate(reversed(self._line_stats.missed_lines)): - timeMatch = self._filter.dateDetector.matchTime( - missed_line, incHits=False) - if timeMatch: - logLine = (missed_line[:timeMatch.start()] + - missed_line[timeMatch.end():]) - else: - logLine = missed_line - if logLine.rstrip("\r\n") == line: - self._line_stats.missed_lines.pop( - len(self._line_stats.missed_lines) - n - 1) - return True - return False - def process(self, test_lines): for line_no, line in enumerate(test_lines): - if line.startswith('#') or not line.strip(): - # skip comment and empty lines - continue - is_ignored = fail2banRegex.testIgnoreRegex(line) - line_datetimestripped, ret = fail2banRegex.testRegex(line) + if isinstance(line, tuple): + line_datetimestripped, ret = fail2banRegex.testRegex( + line[0], line[1]) + line = "".join(line[0]) + else: + line = line.rstrip('\r\n') + if line.startswith('#') or not line: + # skip comment and empty lines + continue + line_datetimestripped, ret = fail2banRegex.testRegex(line) + is_ignored = fail2banRegex.testIgnoreRegex(line_datetimestripped) if is_ignored: self._line_stats.ignored_lines.append(line) @@ -381,7 +380,7 @@ class Fail2banRegex(object): self._line_stats.missed_lines_timeextracted.append(line_datetimestripped) self._line_stats.tested += 1 - if line_no % 10 == 0: + if line_no % 10 == 0 and self._filter.dateDetector is not None: self._filter.dateDetector.sortTemplate() @@ -436,7 +435,7 @@ class Fail2banRegex(object): " %s %s%s" % ( ip[1], timeString, - ip[3] and " (multiple regex matched)" or "")) + ip[-1] and " (multiple regex matched)" or "")) print "\n%s: %d total" % (title, total) pprint_list(out, " #) [# of hits] regular expression") @@ -447,12 +446,14 @@ class Fail2banRegex(object): _ = print_failregexes("Ignoreregex", self._ignoreregex) - print "\nDate template hits:" - out = [] - for template in self._filter.dateDetector.getTemplates(): - if self._verbose or template.getHits(): - out.append("[%d] %s" % (template.getHits(), template.getName())) - pprint_list(out, "[# of hits] date format") + if self._filter.dateDetector is not None: + print "\nDate template hits:" + out = [] + for template in self._filter.dateDetector.getTemplates(): + if self._verbose or template.getHits(): + out.append("[%d] %s" % ( + template.getHits(), template.getName())) + pprint_list(out, "[# of hits] date format") print "\nLines: %s" % self._line_stats @@ -531,7 +532,7 @@ if __name__ == "__main__": sys.exit(-1) myjournal = journal.Reader(converters={'__CURSOR': lambda x: x}) journalmatch = fail2banRegex._journalmatch - fail2banRegex.setDatePattern("ISO8601") + fail2banRegex.setDatePattern(None) if journalmatch: try: for element in journalmatch: diff --git a/bin/fail2ban-testcases b/bin/fail2ban-testcases index bc1cb79a..578763d9 100755 --- a/bin/fail2ban-testcases +++ b/bin/fail2ban-testcases @@ -48,7 +48,7 @@ def get_opt_parser(): p.add_options([ Option('-l', "--log-level", type="choice", dest="log_level", - choices=('heavydebug', 'debug', 'info', 'warn', 'error', 'fatal'), + choices=('heavydebug', 'debug', 'info', 'warning', 'error', 'fatal'), default=None, help="Log level for the logger to use during running tests"), Option('-n', "--no-network", action="store_true", @@ -72,7 +72,7 @@ parser = get_opt_parser() logSys = logging.getLogger("fail2ban") # Numerical level of verbosity corresponding to a log "level" -verbosity = {'heavydebug': 3, +verbosity = {'heavydebug': 4, 'debug': 3, 'info': 2, 'warning': 1, diff --git a/config/action.d/apf.conf b/config/action.d/apf.conf index 9af3066d..f1d54dd2 100644 --- a/config/action.d/apf.conf +++ b/config/action.d/apf.conf @@ -41,3 +41,10 @@ actionban = apf --deny "banned by Fail2Ban " # Values: CMD # actionunban = apf --remove + +[Init] + +# Name used in APF configuration +# +name = default + diff --git a/config/action.d/blocklist_de.conf b/config/action.d/blocklist_de.conf new file mode 100644 index 00000000..d4170cab --- /dev/null +++ b/config/action.d/blocklist_de.conf @@ -0,0 +1,86 @@ +# Fail2Ban configuration file +# +# Author: Steven Hiscocks +# +# + +# Action to report IP address to blocklist.de +# Blocklist.de must be signed up to at www.blocklist.de +# Once registered, one or more servers can be added. +# This action requires the server 'email address' and the assoicate apikey. +# +# From blocklist.de: +# www.blocklist.de is a free and voluntary service provided by a +# Fraud/Abuse-specialist, whose servers are often attacked on SSH-, +# Mail-Login-, FTP-, Webserver- and other services. +# The mission is to report all attacks to the abuse deparments of the +# infected PCs/servers to ensure that the responsible provider can inform +# the customer about the infection and disable them +# +# IMPORTANT: +# +# Reporting an IP of abuse is a serious complaint. Make sure that it is +# serious. Fail2ban developers and network owners recommend you only use this +# action for: +# * The recidive where the IP has been banned multiple times +# * Where maxretry has been set quite high, beyond the normal user typing +# password incorrectly. +# * For filters that have a low likelyhood of receiving human errors +# + +[Definition] + +# Option: actionstart +# Notes.: command executed once at the start of Fail2Ban. +# Values: CMD +# +actionstart = + +# Option: actionstop +# Notes.: command executed once at the end of Fail2Ban +# Values: CMD +# +actionstop = + +# Option: actioncheck +# Notes.: command executed once before each actionban command +# Values: CMD +# +actioncheck = + +# Option: actionban +# Notes.: command executed when banning an IP. Take care that the +# command is executed with Fail2Ban user rights. +# Tags: See jail.conf(5) man page +# Values: CMD +# +actionban = curl --fail --data-urlencode 'server=' --data 'apikey=' --data 'service=' --data 'ip=' --data-urlencode 'logs=' --data 'format=text' --user-agent "fail2ban v0.8.12" "https://www.blocklist.de/en/httpreports.html" + +# Option: actionunban +# Notes.: command executed when unbanning an IP. Take care that the +# command is executed with Fail2Ban user rights. +# Tags: See jail.conf(5) man page +# Values: CMD +# +actionunban = + +[Init] + +# Option: email +# Notes server email address, as per blocklise.de account +# Values: STRING Default: None +# +#email = + +# Option: apikey +# Notes your user blocklist.de user account apikey +# Values: STRING Default: None +# +#apikey = + +# Option: service +# Notes service name you are reporting on, typically aligns with filter name +# see http://www.blocklist.de/en/httpreports.html for full list +# Values: STRING Default: None +# +#service = diff --git a/config/action.d/complain.conf b/config/action.d/complain.conf index ad14a87e..62331f19 100644 --- a/config/action.d/complain.conf +++ b/config/action.d/complain.conf @@ -58,7 +58,7 @@ actioncheck = actionban = ADDRESSES=`whois | perl -e 'while () { next if /^changed|@(ripe|apnic)\.net/io; $m += (/abuse|trouble:|report|spam|security/io?3:0); if (/([a-z0-9_\-\.+]+@[a-z0-9\-]+(\.[[a-z0-9\-]+)+)/io) { while (s/([a-z0-9_\-\.+]+@[a-z0-9\-]+(\.[[a-z0-9\-]+)+)//io) { if ($m) { $a{lc($1)}=$m } else { $b{lc($1)}=$m } } $m=0 } else { $m && --$m } } if (%%a) {print join(",",keys(%%a))} else {print join(",",keys(%%b))}'` IP= if [ ! -z "$ADDRESSES" ]; then - (printf %%b "\n"; date '+Note: Local timezone is %%z (%%Z)'; grep '' ) | "Abuse from " $ADDRESSES + (printf %%b "\n"; date '+Note: Local timezone is %%z (%%Z)'; grep -E '(^|[^0-9])([^0-9]|$)' ) | "Abuse from " $ADDRESSES fi # Option: actionunban diff --git a/config/action.d/firewallcmd-ipset.conf b/config/action.d/firewallcmd-ipset.conf new file mode 100644 index 00000000..2c4a36f1 --- /dev/null +++ b/config/action.d/firewallcmd-ipset.conf @@ -0,0 +1,69 @@ +# Fail2Ban action file for firewall-cmd/ipset +# +# This requires: +# ipset (package: ipset) +# firewall-cmd (package: firewalld) +# +# This is for ipset protocol 6 (and hopefully later) (ipset v6.14). +# Use ipset -V to see the protocol and version. +# +# IPset was a feature introduced in the linux kernel 2.6.39 and 3.0.0 kernels. +# +# If you are running on an older kernel you make need to patch in external +# modules. + +[INCLUDES] + +before = iptables-blocktype.conf + +[Definition] + +actionstart = ipset create fail2ban- hash:ip timeout + firewall-cmd --direct --add-rule ipv4 filter 0 -p -m multiport --dports -m set --match-set fail2ban- src -j + +actionstop = firewall-cmd --direct --remove-rule ipv4 filter 0 -p -m multiport --dports -m set --match-set fail2ban- src -j + ipset flush fail2ban- + ipset destroy fail2ban- + +actioncheck = firewall-cmd --direct --get-chains ipv4 filter | grep -q '^fail2ban-$' + +actionban = ipset add fail2ban- timeout -exist + +actionunban = ipset del fail2ban- -exist + +[Init] + +# Default name of the chain +# +name = default + +# Option: port +# Notes.: specifies port to monitor +# Values: [ NUM | STRING ] +# +port = ssh + +# Option: protocol +# Notes.: internally used by config reader for interpolations. +# Values: [ tcp | udp | icmp | all ] +# +protocol = tcp + +# Option: chain +# Notes specifies the iptables chain to which the fail2ban rules should be +# added +# Values: [ STRING ] +# +chain = INPUT_direct + +# Option: bantime +# Notes: specifies the bantime in seconds (handled internally rather than by fail2ban) +# Values: [ NUM ] Default: 600 + +bantime = 600 + + +# DEV NOTES: +# +# Author: Edgar Hoch and Daniel Black +# firewallcmd-new / iptables-ipset-proto6 combined for maximium goodness diff --git a/config/action.d/firewall-cmd-direct-new.conf b/config/action.d/firewallcmd-new.conf similarity index 71% rename from config/action.d/firewall-cmd-direct-new.conf rename to config/action.d/firewallcmd-new.conf index 55b6762d..bae72ca2 100644 --- a/config/action.d/firewall-cmd-direct-new.conf +++ b/config/action.d/firewallcmd-new.conf @@ -1,9 +1,5 @@ # Fail2Ban configuration file # -# Author: Edgar Hoch -# Copied from iptables-new.conf and modified for use with firewalld by Edgar Hoch. -# It uses "firewall-cmd" instead of "iptables". -# # Because of the --remove-rules in stop this action requires firewalld-0.3.8+ [INCLUDES] @@ -20,7 +16,7 @@ actionstop = firewall-cmd --direct --remove-rule ipv4 filter 0 -m state firewall-cmd --direct --remove-rules ipv4 filter fail2ban- firewall-cmd --direct --remove-chain ipv4 filter fail2ban- -actioncheck = firewall-cmd --direct --get-chains ipv4 filter | grep -q 'fail2ban-[ \t]' +actioncheck = firewall-cmd --direct --get-chains ipv4 filter | grep -q '^fail2ban-$' actionban = firewall-cmd --direct --add-rule ipv4 filter fail2ban- 0 -s -j @@ -50,3 +46,27 @@ protocol = tcp # Values: [ STRING ] # chain = INPUT_direct + +# DEV NOTES: +# +# Author: Edgar Hoch +# Copied from iptables-new.conf and modified for use with firewalld by Edgar Hoch. +# It uses "firewall-cmd" instead of "iptables". +# +# Output: +# +# $ firewall-cmd --direct --add-chain ipv4 filter fail2ban-name +# success +# $ firewall-cmd --direct --add-rule ipv4 filter fail2ban-name 1000 -j RETURN +# success +# $ sudo firewall-cmd --direct --add-rule ipv4 filter INPUT_direct 0 -m state --state NEW -p tcp --dport 22 -j fail2ban-name +# success +# $ firewall-cmd --direct --get-chains ipv4 filter +# fail2ban-name +# $ firewall-cmd --direct --get-chains ipv4 filter | od -h +# 0000000 6166 6c69 6232 6e61 6e2d 6d61 0a65 +# $ firewall-cmd --direct --get-chains ipv4 filter | grep -Eq 'fail2ban-name( |$)' ; echo $? +# 0 +# $ firewall-cmd -V +# 0.3.8 + diff --git a/config/action.d/ipfw.conf b/config/action.d/ipfw.conf index 09045815..37625209 100644 --- a/config/action.d/ipfw.conf +++ b/config/action.d/ipfw.conf @@ -43,7 +43,7 @@ actionban = ipfw add tcp from to # Tags: See jail.conf(5) man page # Values: CMD # -actionunban = ipfw delete `ipfw list | grep -i | awk '{print $1;}'` +actionunban = ipfw delete `ipfw list | grep -i "[^0-9][^0-9]" | awk '{print $1;}'` [Init] diff --git a/config/action.d/mail-whois-lines.conf b/config/action.d/mail-whois-lines.conf index 758c4eff..aa7d0950 100644 --- a/config/action.d/mail-whois-lines.conf +++ b/config/action.d/mail-whois-lines.conf @@ -39,10 +39,10 @@ actioncheck = actionban = printf %%b "Hi,\n The IP has just been banned by Fail2Ban after attempts against .\n\n - Here are more information about :\n - `whois `\n\n + Here is more information about :\n + `whois || echo missing whois program`\n\n Lines containing IP: in \n - `grep '\<\>' `\n\n + `grep '[^0-9][^0-9]' `\n\n Regards,\n Fail2Ban"|mail -s "[Fail2Ban] : banned from `uname -n`" diff --git a/config/action.d/mail-whois.conf b/config/action.d/mail-whois.conf index fa133ab3..e4c8450e 100644 --- a/config/action.d/mail-whois.conf +++ b/config/action.d/mail-whois.conf @@ -39,8 +39,8 @@ actioncheck = actionban = printf %%b "Hi,\n The IP has just been banned by Fail2Ban after attempts against .\n\n - Here are more information about :\n - `whois `\n + Here is more information about :\n + `whois || echo missing whois program`\n Regards,\n Fail2Ban"|mail -s "[Fail2Ban] : banned from `uname -n`" diff --git a/config/action.d/sendmail-common.conf b/config/action.d/sendmail-common.conf index e2820470..26dcb4c8 100644 --- a/config/action.d/sendmail-common.conf +++ b/config/action.d/sendmail-common.conf @@ -8,6 +8,56 @@ after = sendmail-common.local +[Definition] + +# Option: actionstart +# Notes.: command executed once at the start of Fail2Ban. +# Values: CMD +# +actionstart = printf %%b "Subject: [Fail2Ban] : started on `uname -n` + Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"` + From: <> + To: \n + Hi,\n + The jail has been started successfully.\n + Regards,\n + Fail2Ban" | /usr/sbin/sendmail -f + +# Option: actionstop +# Notes.: command executed once at the end of Fail2Ban +# Values: CMD +# +actionstop = printf %%b "Subject: [Fail2Ban] : stopped on `uname -n` + Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"` + From: <> + To: \n + Hi,\n + The jail has been stopped.\n + Regards,\n + Fail2Ban" | /usr/sbin/sendmail -f + +# Option: actioncheck +# Notes.: command executed once before each actionban command +# Values: CMD +# +actioncheck = + +# Option: actionban +# Notes.: command executed when banning an IP. Take care that the +# command is executed with Fail2Ban user rights. +# Tags: See jail.conf(5) man page +# Values: CMD +# +actionban = + +# Option: actionunban +# Notes.: command executed when unbanning an IP. Take care that the +# command is executed with Fail2Ban user rights. +# Tags: See jail.conf(5) man page +# Values: CMD +# +actionunban = + [Init] # Recipient mail address diff --git a/config/action.d/sendmail-whois-ipjailmatches.conf b/config/action.d/sendmail-whois-ipjailmatches.conf new file mode 100644 index 00000000..45b1f312 --- /dev/null +++ b/config/action.d/sendmail-whois-ipjailmatches.conf @@ -0,0 +1,37 @@ +# Fail2Ban configuration file +# +# Author: Cyril Jaquier +# +# + +[INCLUDES] + +before = sendmail-common.conf + +[Definition] + +# Option: actionban +# Notes.: command executed when banning an IP. Take care that the +# command is executed with Fail2Ban user rights. +# Tags: See jail.conf(5) man page +# Values: CMD +# +actionban = printf %%b "Subject: [Fail2Ban] : banned from `uname -n` + Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"` + From: <> + To: \n + Hi,\n + The IP has just been banned by Fail2Ban after + attempts against .\n\n + Here are more information about :\n + `/usr/bin/whois `\n\n + Matches for with failures IP:\n + \n\n + Regards,\n + Fail2Ban" | /usr/sbin/sendmail -f + +[Init] + +# Default name of the chain +# +name = default diff --git a/config/action.d/sendmail-whois-ipmatches.conf b/config/action.d/sendmail-whois-ipmatches.conf new file mode 100644 index 00000000..8193fb04 --- /dev/null +++ b/config/action.d/sendmail-whois-ipmatches.conf @@ -0,0 +1,37 @@ +# Fail2Ban configuration file +# +# Author: Cyril Jaquier +# +# + +[INCLUDES] + +before = sendmail-common.conf + +[Definition] + +# Option: actionban +# Notes.: command executed when banning an IP. Take care that the +# command is executed with Fail2Ban user rights. +# Tags: See jail.conf(5) man page +# Values: CMD +# +actionban = printf %%b "Subject: [Fail2Ban] : banned from `uname -n` + Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"` + From: <> + To: \n + Hi,\n + The IP has just been banned by Fail2Ban after + attempts against .\n\n + Here are more information about :\n + `/usr/bin/whois `\n\n + Matches with failures IP:\n + \n\n + Regards,\n + Fail2Ban" | /usr/sbin/sendmail -f + +[Init] + +# Default name of the chain +# +name = default diff --git a/config/action.d/sendmail-whois-lines.conf b/config/action.d/sendmail-whois-lines.conf index 5a331e24..270373e7 100644 --- a/config/action.d/sendmail-whois-lines.conf +++ b/config/action.d/sendmail-whois-lines.conf @@ -10,38 +10,6 @@ before = sendmail-common.conf [Definition] -# Option: actionstart -# Notes.: command executed once at the start of Fail2Ban. -# Values: CMD -# -actionstart = printf %%b "Subject: [Fail2Ban] : started on `uname -n` - Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"` - From: <> - To: \n - Hi,\n - The jail has been started successfully.\n - Regards,\n - Fail2Ban" | /usr/sbin/sendmail -f - -# Option: actionstop -# Notes.: command executed once at the end of Fail2Ban -# Values: CMD -# -actionstop = printf %%b "Subject: [Fail2Ban] : stopped on `uname -n` - Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"` - From: <> - To: \n - Hi,\n - The jail has been stopped.\n - Regards,\n - Fail2Ban" | /usr/sbin/sendmail -f - -# Option: actioncheck -# Notes.: command executed once before each actionban command -# Values: CMD -# -actioncheck = - # Option: actionban # Notes.: command executed when banning an IP. Take care that the # command is executed with Fail2Ban user rights. @@ -55,21 +23,13 @@ actionban = printf %%b "Subject: [Fail2Ban] : banned from `uname -n` Hi,\n The IP has just been banned by Fail2Ban after attempts against .\n\n - Here are more information about :\n - `/usr/bin/whois `\n\n + Here is more information about :\n + `/usr/bin/whois || echo missing whois program`\n\n Lines containing IP: in \n - `grep '\<\>' `\n\n + `grep '[^0-9][^0-9]' `\n\n Regards,\n Fail2Ban" | /usr/sbin/sendmail -f -# Option: actionunban -# Notes.: command executed when unbanning an IP. Take care that the -# command is executed with Fail2Ban user rights. -# Tags: See jail.conf(5) man page -# Values: CMD -# -actionunban = - [Init] # Default name of the chain diff --git a/config/action.d/sendmail-whois-matches.conf b/config/action.d/sendmail-whois-matches.conf new file mode 100644 index 00000000..ed664766 --- /dev/null +++ b/config/action.d/sendmail-whois-matches.conf @@ -0,0 +1,37 @@ +# Fail2Ban configuration file +# +# Author: Cyril Jaquier +# +# + +[INCLUDES] + +before = sendmail-common.conf + +[Definition] + +# Option: actionban +# Notes.: command executed when banning an IP. Take care that the +# command is executed with Fail2Ban user rights. +# Tags: See jail.conf(5) man page +# Values: CMD +# +actionban = printf %%b "Subject: [Fail2Ban] : banned from `uname -n` + Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"` + From: <> + To: \n + Hi,\n + The IP has just been banned by Fail2Ban after + attempts against .\n\n + Here are more information about :\n + `/usr/bin/whois `\n\n + Matches:\n + \n\n + Regards,\n + Fail2Ban" | /usr/sbin/sendmail -f + +[Init] + +# Default name of the chain +# +name = default diff --git a/config/action.d/sendmail-whois.conf b/config/action.d/sendmail-whois.conf index a65f9875..fc601277 100644 --- a/config/action.d/sendmail-whois.conf +++ b/config/action.d/sendmail-whois.conf @@ -10,38 +10,6 @@ before = sendmail-common.conf [Definition] -# Option: actionstart -# Notes.: command executed once at the start of Fail2Ban. -# Values: CMD -# -actionstart = printf %%b "Subject: [Fail2Ban] : started on `uname -n` - Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"` - From: <> - To: \n - Hi,\n - The jail has been started successfully.\n - Regards,\n - Fail2Ban" | /usr/sbin/sendmail -f - -# Option: actionstop -# Notes.: command executed once at the end of Fail2Ban -# Values: CMD -# -actionstop = printf %%b "Subject: [Fail2Ban] : stopped on `uname -n` - Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"` - From: <> - To: \n - Hi,\n - The jail has been stopped.\n - Regards,\n - Fail2Ban" | /usr/sbin/sendmail -f - -# Option: actioncheck -# Notes.: command executed once before each actionban command -# Values: CMD -# -actioncheck = - # Option: actionban # Notes.: command executed when banning an IP. Take care that the # command is executed with Fail2Ban user rights. @@ -55,19 +23,11 @@ actionban = printf %%b "Subject: [Fail2Ban] : banned from `uname -n` Hi,\n The IP has just been banned by Fail2Ban after attempts against .\n\n - Here are more information about :\n - `/usr/bin/whois `\n + Here is more information about :\n + `/usr/bin/whois || echo missing whois program`\n Regards,\n Fail2Ban" | /usr/sbin/sendmail -f -# Option: actionunban -# Notes.: command executed when unbanning an IP. Take care that the -# command is executed with Fail2Ban user rights. -# Tags: See jail.conf(5) man page -# Values: CMD -# -actionunban = - [Init] # Default name of the chain diff --git a/config/action.d/sendmail.conf b/config/action.d/sendmail.conf index 70f38329..46050e11 100644 --- a/config/action.d/sendmail.conf +++ b/config/action.d/sendmail.conf @@ -10,38 +10,6 @@ before = sendmail-common.conf [Definition] -# Option: actionstart -# Notes.: command executed once at the start of Fail2Ban. -# Values: CMD -# -actionstart = printf %%b "Subject: [Fail2Ban] : started on `uname -n` - Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"` - From: <> - To: \n - Hi,\n - The jail has been started successfully.\n - Regards,\n - Fail2Ban" | /usr/sbin/sendmail -f - -# Option: actionstop -# Notes.: command executed once at the end of Fail2Ban -# Values: CMD -# -actionstop = printf %%b "Subject: [Fail2Ban] : stopped on `uname -n` - Date: `LC_TIME=C date -u +"%%a, %%d %%h %%Y %%T +0000"` - From: <> - To: \n - Hi,\n - The jail has been stopped.\n - Regards,\n - Fail2Ban" | /usr/sbin/sendmail -f - -# Option: actioncheck -# Notes.: command executed once before each actionban command -# Values: CMD -# -actioncheck = - # Option: actionban # Notes.: command executed when banning an IP. Take care that the # command is executed with Fail2Ban user rights. @@ -58,14 +26,6 @@ actionban = printf %%b "Subject: [Fail2Ban] : banned from `uname -n` Regards,\n Fail2Ban" | /usr/sbin/sendmail -f -# Option: actionunban -# Notes.: command executed when unbanning an IP. Take care that the -# command is executed with Fail2Ban user rights. -# Tags: See jail.conf(5) man page -# Values: CMD -# -actionunban = - [Init] # Default name of the chain diff --git a/config/action.d/ufw.conf b/config/action.d/ufw.conf new file mode 100644 index 00000000..c826729d --- /dev/null +++ b/config/action.d/ufw.conf @@ -0,0 +1,40 @@ +# Fail2Ban action configuration file for ufw +# +# You are required to run "ufw enable" before this will have an effect. +# +# The insert position should be approprate to block the required traffic. +# A number after an allow rule to the application won't be much use. + +[Definition] + +actionstart = + +actionstop = + +actioncheck = + +actionban = [ -n "" ] && app="app " ; ufw insert from to $app + +actionunban = [ -n "" ] && app="app " ; ufw delete from to $app + +[Init] +# Option: insertpos +# Notes.: The postition number in the firewall list to insert the block rule +insertpos = 1 + +# Option: blocktype +# Notes.: reject or deny +blocktype = reject + +# Option: destination +# Notes.: The destination address to block in the ufw rule +destination = any + +# Option: application +# Notes.: application from sudo ufw app list +application = + +# DEV NOTES: +# +# Author: Guilhem Lettron +# Enhancements: Daniel Black diff --git a/config/action.d/xarf-login-attack.conf b/config/action.d/xarf-login-attack.conf new file mode 100644 index 00000000..c5ac5110 --- /dev/null +++ b/config/action.d/xarf-login-attack.conf @@ -0,0 +1,125 @@ +# Fail2Ban action for sending xarf Login-Attack messages to IP owner +# +# IMPORTANT: +# +# Emailing a IP owner of abuse is a serious complain. Make sure that it is +# serious. Fail2ban developers and network owners recommend you only use this +# action for: +# * The recidive where the IP has been banned multiple times +# * Where maxretry has been set quite high, beyond the normal user typing +# password incorrectly. +# * For filters that have a low likelyhood of receiving human errors +# +# DEPENDANCIES: +# +# This requires the dig command from bind-utils +# +# This uses the https://abusix.com/contactdb.html to lookup abuse contacts. +# +# XARF is a specification for sending a formatted response +# for non-messaging based abuse including: +# +# Login-Attack, Malware-Attack, Fraud (Phishing, etc.), Info DNSBL +# +# For details see: +# https://github.com/abusix/xarf-specification +# http://www.x-arf.org/schemata.html +# +# Author: Daniel Black +# Based on complain written by Russell Odom +# +# + +[Definition] + +actionstart = + +actionstop = + +actioncheck = + +actionban = oifs=${IFS}; IFS=.;SEP_IP=( ); set -- ${SEP_IP} ;ADDRESSES=$(dig +short -t txt -q $4.$3.$2.$1.abuse-contacts.abusix.org); IFS=${oifs} + IP= + FROM= + SERVICE= + FAILURES= + MATCHES='' + REPORTID=