Sep 182013

Indenting lines with leading tabs is an important technique for writing clear and readable code, and are essential in Python where they have syntactic meaning. Many IDEs insert tabs automatically, but if you’re using vi it can be tedious to insert tabs individually. Fortunately, there is an easy shortcut, and it’s a very handy command to remember.

In the vi session, enter command mode by pressing ESC. Then, set the size of the tabs with the following command:

  :set shiftwidth=2

To make this the default, put “set shiftwidth=2” in your ~/.exrc file.

Then, to tab shift the next 13 lines, you’d type a command like this:


Matt Parsons is a freelance Linux specialist who has designed, built and supported Unix and Linux systems in the finance, telecommunications and media industries.

He lives and works in London.

May 292013

There are times when debugging system behaviour when it can be very useful to see what the inputs and/or outputs of a process are: to read its STDIN and STDOUT filehandles. It can be handy to know what’s being printed to an unseen screen, particularly if there is no log file, or it may be handy to snoop on what a user is typing. This is a yet another way of using the Swiss Army knife that is strace.

In its most basic form, with few options, strace can print out every system call that a process makes, and if the “-f” or “-ff” switches are used, also print out all the system calls from the child processes that the process forks. Like this:

  # strace -f -p <PID>

This provides far too much information though, so it’s often desirable to run strace with filter expressions to narrow the search down, specified with the “-e” switch. The man page provides an exhaustive list of these filters.

To specify a filter for reading STDIN, STDOUT and STDERR, the following expression will do the trick:

  # strace -ff -e trace=write -e write=1,2 -p <PID>

Make use of this how you will.

Matt Parsons is a freelance Linux specialist who has designed, built and supported Unix and Linux systems in the finance, telecommunications and media industries.

He lives and works in London.

Jan 042013

Puppet configuration can quickly become very large, interrelated and unwieldy. Even the most restrained and discplined architect can easily let it get away from itself. As its spread out over numerous disparate files, often an overview of how everything’s interconnected can be useful. Fortunately, the Puppet agent has the ability to create .dot files, viewable with Graphwiz, which allows a node’s configuration to be visualized, and hopefully, to be understood.

To set this up, edit /etc/puppet/puppet.conf, [agent] section:

   graph = true
   graphdir = /var/lib/puppet/state/graphs

Run a puppet update:

   # puppetd -t --noop

This time, three “.dot” files will be written to the graphdir location:

This .dot format is a text file readable by the Graphviz program. It’s available from the standard Ubuntu and Fedora repositories. Install it (preferably on your workstation), copy across the generated Puppet agent .dot files and convert the dot files into pretty PNG image files like with the dot program which is installed as part of Graphviz:

   # dot -Tpng -o resources.png

And similarly for the other two files. The generated image contains a map of all the configuration resources and/or relationships of the Puppet client.

One can also view the .dot files directly, as they are text. In fact, one could also trawl the Puppet configuration root and generate .dot files manually, as described in this example on this blog.

After using Graphviz to create pretty configuration visualizations, these maps may bring about some clarity regarding the Puppet config, or possibly identify a circular dependency. Equally, the map may be convoluted and unreadable, which may be a reason to overhaul one’s Puppet configuration.

Matt Parsons is a freelance Linux specialist who has designed, built and supported Unix and Linux systems in the finance, telecommunications and media industries.

He lives and works in London.

Oct 292012
It goes without saying that security is becoming of increasing concern for anyone managing hosts connected to the Internet. You only have to open a new port on your firewall and you can just watch all the port-knocking traffic come rolling in from script kiddies all over the world. The attention is heart-warming at first, but quickly becomes tiresome. Fail2ban is what you need.

3D-printed-ban-hammerIntrusion Prevention software.

Fail2ban is a free, reliable, and easy to configure utility which performs the simple task of watching log files for evidence of suspicious connections, and then locking out traffic from the source IP address.

The default behaviour is to lock out connections for a certain period of time, which doesn’t need to be that long to disrupt and defuse a brute-force attack. When the ban time has elapsed, the ban configuration is reversed, so as to only temporarily inconvenience genuine access attempts which may have been incorrectly configured. Or if you’re feeling particularly zero-tolerant, ban them for an eternity.

Application Architecture

Fail2ban is available in all the standard repositories for the major distributions, and its installation is typical, according to the usual package procedures. For the purposes of this discussion, I’ve been using Fail2ban 0.8.6 installed on Centos 6.2.

The fail2ban configuration files are in the conventional place: /etc/fail2ban/. A discussion of these config files well illustrates how fail2ban works.


This contains mostly just information about logging – the debug level and the location of the log file. You can log either to the syslog or a file of your choice.


The main configuration file is /etc/fail2ban/jail.conf, and it contains blocks for each ban case. That is, one block of configuration settings for a given type of failure in a given log file, a specific action will be taken.

Below is the block pertaining to SSH failures, banned by an iptables rules:


enabled  = true
filter   = sshd
action   = iptables[name=SSH, port=ssh, protocol=tcp]
           sendmail-whois[name=SSH, dest=root,]
logpath  = /var/log/secure
maxretry = 5

So this states to monitor the file /var/log/secure using the sshd filter, and if it matches to perform the iptables and sendmail-whois actions. Filters and actions are defined in other configuration files.

Filters: /etc/fail2ban/filter.d/

The directory /etc/fail2ban/filter.d/ defines the regular expressions which are used to match on log messages indicating connection failures. This text will vary with different distributions and different versions of these services, so the regular expression sets are fairly extensive, although to be fair they realistically cannot be completely exhaustive. Check for new versions of Fail2ban periodically and keep it up to date.

As an example, the file /etc/fail2ban/filter.d/sshd.conf contains a list of regular expressions which define flagged text. Here is a truncated excerpt:

failregex = ^%(__prefix_line)s(?:error: PAM: )?Authentication failure for .* from s*$
            ^%(__prefix_line)s(?:error: PAM: )?User not known to the underlying authentication module for .* from s*$
            ^%(__prefix_line)sFailed (?:password|publickey) for .* from (?: port d*)?(?: sshd*)?$

You get the idea.

Actions: /etc/fail2ban/action.d/

The actions.d directory contains configuration files which define the verbs of fail2ban – that is, the tasks which can be performed should an alert be generated – “actionban”. Equally, each file also contains an “actionunban” which describes what is executed to undo the effects of the lockout after the ban period has expired.

The relevant lines of the iptables.conf file are displayed below. The first adds a drop rule to the iptables ruleset, and the second line removes the same rule.

actionban = iptables -I fail2ban- 1 -s  -j DROP
actionunban = iptables -D fail2ban- -s  -j DROP

The nice thing about fail2ban is that, as you an see here, the configuration is all fairly self-explanatory if you have a reasonable knowledge of shell scripting, and understand what it’s trying to do. The source IP is passed to the “actionban” command after being extracted from the filter, and this command inserts a DROP rule for this IP at the beginning of the firewall ruleset. Similar actions are used for banning source addresses with tcpwrappers.

Some of the actions are for sending notifications of events. The “sendmail-whois” action is rather useful in that it sends an alert mail, but also includes the results of a “whois” invocation so you can see the identity and location of the owner of the IP address from which the incursion is emanating.

How to Configure Fail2ban

The steps involved in configuring fail2ban are as follows:

  • Identify those files on your system that need to be watched – generally these would be the log files for an running services through which an outside party would be able to gain a connection to your host – /var/log/secure (SSH), /var/log/httpd/access_log (Apache HTTPD), or whatever.
  • Edit /etc/fail2ban/jail.conf:
    • select the config block that best suits your purpose, adapt an existing one, or write a new one;
    • set the variable enabled equal to “true”;
    • set your email address, if you want to be notified;
    • set the filter to use, according to what service or process is being monitored;
    • check and make sure that the regex in the action.d config file actually matches what you would expect for an alert.
  • Restart fail2ban: # /etc/init.d/fail2ban restart

Repeat the process for everything you feel needs to be monitored. Any problems, check the man page or the Fail2ban Wiki

Fail2ban is a wonderfully simple and straightforward tool that does exactly what it says, and whose operation and configuration is transparent to the systems administrator. It is by no means perfect, and in a site with a large number of hosts, you’d probably be better off using something centralised like OSSEC. However fail2ban is significantly easier to configure than OSSEC and can easily be rolled out to individual servers on an ad hoc basis.

So there you have your ban-hammer. Go on and wield it with extreme prejudice.
[flattr uid=’matthewparsons’ /]

Matt Parsons is a freelance Linux specialist who has designed, built and supported Unix and Linux systems in the finance, telecommunications and media industries.

He lives and works in London.