Five essential open source tools for the sysadmin

by Elena Blanco on 1 November 2004 , last updated

Archived This page has been archived. Its content will not be updated. Further details of our archive policy.

Introduction

Any systems administrator, regardless of the operating system(s) that they administer, knows that their job only gets harder; demand for existing services is ever increasing, there are always new services to provide, the upgrade cycle gets ever shorter and inevitably the job gets bigger. Therefore, it is vital to have a good set of tools that can help make the sysadmin more effective.

The most useful tools fall into two broad categories:

  • Tools to automate general housekeeping thus freeing the sysadmin’s time for other tasks.
  • Tools to assist with general maintenance and troubleshooting, both pre-emptive and during an incident.

For the sysadmin that has to work on multiple operating systems, a set of tools that are supported cross-platform can be a life saver. Clearly, being able to use the same tool whether you are on a Windows box or a Unix box has great benefit. Therefore, where possible, tools that implement generic functionality on multiple platforms have been chosen.

OpenSSH

All sysadmins need to connect across the network, be it the Internet or the LAN, at some point. In days long gone the network was considered to be a safe place but these days network level attacks are commonplace and therefore new precautions are required. Sysadmins by their very nature use privileged accounts, so it is vital that neither their authentication credentials nor data is exposed on the network. Consequently they need to be able to connect across the network securely in a manner that will protect their connection from eavesdropping or hijacking.

SSH, also known as the secure shell, is a protocol suite to build tools that provide secure network connections over insecure networks. There are currently two versions of the SSH protocol in use, version 1 and version 2, but each version uses public-key cryptography to provide an encrypted terminal session with strong authentication of both the server and client.

SSH features include:

  • A variety of user authentication methods.
  • Tunnelling arbitrary TCP connections through the SSH session, protecting normally insecure protocols such as IMAP and allowing secure passage through firewalls.
  • Automatic forwarding of X windows connections.
  • Support for external authentication methods, including Kerberos and SecurID.
  • Secure file transfers.

OpenSSH is an open source version of the SSH protocol suite that emerged from the OpenBSD community, making its first appearance in OpenBSD 2.6. Distribution of the software is divided between two teams: one is concerned with strictly OpenBSD development; the other takes the code from the first team and ports it to run on different operating systems. Unless you are an OpenBSD user you will be more interested in the code produced by the porting team, known as the ‘p’ releases, e.g. OpenSSH 4.2p1.

The OpenSSH suite includes the ssh program which replaces rlogin and telnet and the scp program which replaces rcp and ftp. The suite also contains a number of programs concerned with generating and managing the cryptographic keys used by OpenSSH. For day-to-day use it is hard to think of a more important tool for the sysadmin than the secure shell.

OpenSSH is licensed under a BSD style licence and is available for many Unix operating systems.

Windows users requiring an SSH client may be interested in PuTTY, a free implementation of Telnet and SSH for Win32 and Unix platforms including an xterm terminal emulator. PuTTY implements the client end of a network session running over the SSH protocol allowing the client machine to securely communicate with the server. PuTTY is distributed under the MIT licence. In addition the WinSCP Windows client supports the sftp and scp file copy protocols.

Both Windows and Unix sysadmins are likely to find the SSH FAQ very useful.

Perl

Perl has fast become the scripting language of choice for many, and it is now the most widely used web programming language. Perhaps some of its appeal comes from the fact that you can use Perl in either a procedural or object-oriented way so you can choose whichever you are happiest with, increasing the speed with which you can produce usable code from day one.

There are of course many factors that have influenced the wide take up of Perl but some of its strengths particularly pertinent to the sysadmin are:

  • Excellent text manipulation capabilities - Perl is a versatile tool for manipulating log files and this can be widely exploited when it comes to automating the general housekeeping associated with the management and extraction of data and trends from system and application logs.
  • A database integration interface that supports a huge number of third party databases - once a sysadmin is comfortable with Perl as a scripting language it can be used for any database programming vastly simplifying the management of disparate database platforms.
  • It is highly extensible - the Comprehensive Perl Archive Network (CPAN) holds thousands of third party modules so the chances are that whatever you may need to do, there will be a third party module that will help you do it.

It has been said the reason why Perl has been so widely adopted is that in the end, sysadmins like Perl because Perl makes them look good. Anything that makes the sysadmin’s job easier has got to be a good thing!

Perl is licensed under the Artistic License and runs on many computing platforms including Unix, Linux, Mac OS X and Windows. Extensive documentation can be found at perldoc.perl.org amongst other places.

Wireshark (formerly Ethereal)

A packet sniffing tool that allows you to see exactly what is going on across the network is crucial in determining the cause of any problems involving the access of remote servers or services. Wireshark is an extremely powerful network protocol analyser that allows you to capture packets from a network and analyse them live or to process captured packets at a later date by storing network traffic in a capture file. Wireshark (known as Ethereal until June 2006) can read data from Ethernet, Token-Ring, FDDI, serial (PPP and SLIP), 802.11 wireless LAN, and ATM connections, and many hundreds of network protocols can be dissected for analysis.

Wireshark allows you to view the network data that it collects in two ways. The first is via a GUI browser, and the second is via the TTY-mode TShark program. The latter can be extremely useful for a sysadmin dialling in to fix a problem; in these circumstances it is often more practical to have a text based tool rather than a full blown GUI. Troubleshooting connectivity problems can be a tedious task without a flexible and comprehensive tool that allows you to view the network at the packet level. Wireshark makes that process accessible and straightforward.

Wireshark is licensed under the GNU General Public License and runs on many computing platforms including Unix, Linux, Mac OS X and Windows.

Nagios

The ability to see the status of servers and services at a glance can be a useful tool for everyone from the sysadmin to the IT manager. For the sysadmin it is especially important to be aware of problems before the effects are felt elsewhere. Nagios is a widely used monitoring tool that can be configured to monitor all sorts of services on the network. Additionally, and very usefully, Nagios can monitor host-specific resources such as disk and memory usage, CPU load, processes and even databases via external plugins or agents that return status information about the resource. Nagios can then be configured to store thresholds for each resource that it measures and behave differently according to whether or not the resource is within threshold. This could include automatic notifications via email, instant message or SMS. Nagios includes a customisable web interface for viewing current status information, historical logs, and reports. Indeed most of what you would ever need to do is done via this web interface.

Nagios also provides a simple authorization scheme that allows the sysadmin to restrict what different users can both see and do via the web interface. This feature can be extremely useful as a single application can provide a customised interface appropriate to the user. For example there could be a general view only interface available to check service status for first line support staff, an expanded manager’s interface to view more detailed service and resource status with access to the escalation features, and the full blown sysadmin interface allowing access to all the features including scheduling and configuration.

Nagios can also be used as a rich source of historical information; status information, problem history and logging can be viewed and analysed to provide a rich picture of how hosts and services have behaved over time. This can be very useful in determining trends and patterns both for intermittent problem resolution and in determining expansion requirements.

Nagios is licensed under the GNU General Public License and, whilst the server software is available for Unix platforms only, there are client-side agents available for most operating systems that allow the Nagios server to monitor those hosts. However, as some of these are third-party products, it is possible that some may be released under other licences, or are not open source.

Nmap

Most services operate in an increasingly hostile environment. Any publicly exposed services, particularly those on sites known to have high bandwidth connections to the Internet, will be subject to some sort of port scanning or intrusion attempt on a fairly regular basis. Using the principle of set a thief to catch a thief, a sysadmin may find it enlightening to habitually use similar tools to the would-be intruder to determine whether there is any weakness in their armour that could potentially be exploited.

Nmap, Network Mapper, is a very flexible tool that will rapidly explore your entire network or indeed just a single host. Nmap’s flexibility means that you can start out using it very simply against a single host and then over time you can take advantage of the rich feature set that it offers to power users. When scanning networks Nmap uses raw IP packets in a variety of different ways to map out networks filled with IP filters, firewalls, routers, and other obstacles. On the network itself Nmap can determine what type of firewalls and packet filters are in use by using sophisticated port scanning techniques. As far as individual hosts are concerned Nmap can determine which hosts are present on the network, the operating system and version of these hosts along with details of the services that the hosts are running, including application versions.

Nmap can be used via a traditional command line interface or via a graphical interface so you are free to choose whichever mode you are most comfortable with. As a security scanning tool, Nmap’s strength lies in its ability to be as simple or complicated as you choose to make it whilst using very sophisticated techniques under the covers. This makes it an ideal tool for both the new and the experienced sysadmin.

Nmap should not be used on networks except with the explicit permission of those responsible for their maintenance and security as it is likely to trigger any intrusion detection systems which may be in place.

Nmap is licensed under the GNU General Public License and is available for many operating system platforms including Unix, Linux, MacOS X and Windows.

OSS Watch has produced the Top Tips for Selecting Open Source Software document which provides general advice on choosing open source programs.

Further reading

Links:

Related information from OSS Watch: