Today, I finally solved the issue. It turns out that something to do with the m4 package was causing the error, with the following output:

MozPlugger: Error: Failed to execute m4.
Mozplugger: M4 parsing of config generated error

So, I checked out the package in question, and found that it wasn't installed. To my great relief (and some frustration), a simple

sudo apt-get install m4

fixed the problem instantly.

I only hope that this post solves this extremely annoying issue for someone else.

Recently though, I've been looking in to Mercurial as a more modern alternative. Some things about Subversion are really starting to annoy me, like the inability to ignore files on a repository-wide level easily (I work with Python all the time, so .pyc files can really get on my nerves!). Mercurial seems slicker, and I'm interested in the concept of DVCS, as opposed to the classic client-server way of thinking.

However, Mercurial is not installed on the (rather ancient, now) Solaris servers at university. I really admire the job that the sysadmins do, so I am in no way condemning them for it not being installed. Instead, I set out to get it compiled and working myself.

Installing it on the servers, it seemed, was the easy part. I followed the installation instructions (making sure to include the LD_LIBRARY_PATH variable) and all was well.

Cloning from the server, however, wasn't so easy. The problem lies in the fact that Mercurial is now installed for me, and me alone. The hg binary lives in my home directory, not on the server's main path (i.e. the /usr/bin/ directory). Therefore, I needed to tell the client exactly what command to run on the server. If anyone else is having the same trouble, the command that I finally came up with looks like this:

hg --config ui.remotecmd="LD_LIBRARY_PATH=/usr/sfw/lib PYTHONPATH=~/lib/python ~/bin/hg"

I then aliased this command to hgtuck to save my sanity (the server is named tuck, after Friar Tuck in the Robin Hood legends), and I now have a perfectly working Mercurial install!

If anyone else is looking to do something similar, then I hope this little tip saves you some time!

Domain authentication can be achieved relatively simply, by using either Winbind (part of the Samba project) or the new kid on the block, Likewise. Likewise-Open offers a simple procedure for joining domains, and the new version comes packaged with it's own version of Kerberos. It also hashes the Windows SID into a UNIX UID in a consistent manner - so the UIDs are always the same across your entire environment. Sounds perfect, right? Well, no. Not quite. The problem with Likewise-Open, is that it's difficult to integrate with Samba. Though it does ship with a "compatibility module" called lwi_compat, which allows Samba to hook into Likewise's authentication module, I found this quite difficult to get working, and I only achieved partial success through guesswork - as the documentation didn't actually help much, given they only officially support Samba 3.0.x (while Ubuntu now uses 3.3.x). When I did get it going, however, it only recognised the Windows users' primary group, not any of the other groups they were members of. This meant that my (possibly overly) complex system of ACLs and user directories just didn't work at all. So, on to Plan B...

When I read through the short (but sweet) Ubuntu Wiki article entitled ActiveDirectoryWinbindHowto, I felt like somewhat of a fool after reading a small, illusive section called Adding more than one Linux machine to a Windows network. Bingo! This section described a problem whereby the traditional Winbind domain authentication method would lead to inconsistent UIDs across the network, and thus cause headaches when trying to achieve anything like what I was aiming for. It suggested using a method of mapping SIDs to UIDs called RID. I assume this stands for Relative ID, which is another kind of ID Active Directory uses to track users within a domain. These can possibly clash from domain to domain, so it is advised not to use this method when your network contains a trust between multiple Windows domains, but for the simpler setup (like my own) it's a godsend.

This meant that I could use RID mapping within Winbind, which is part of Samba itself (so no troubles integrating those two), and achieve a consistent SID-UID mapping scheme across the network, allowing me to finally enable access to the file shares via. NFS. Amazingly, NFS "Just Worked" straight away, and I've written some nice wrapper programs around chown, getfacl and setfacl to set the correct owner and permissions on entire directory trees, which saves a lot of time when your UIDs are changing as often as mine were! I'm also using autofs to automatically map user's home directories on the Linux machines, which has proven itself to be very useful. I just used static fstab entries to map the other "general" file shares, like software and media - as I couldn't seem to get autofs direct maps working (apparently they are only partially working in Ubuntu anyway, but it seems as though they are completely broken to me).

On a side note, I've also just finished developing a Python-based rsync backup program, which allows me to write a _very_ small script to backup remote servers using rsync over SSH, and tar up the contents of all the servers into one archive. This is really useful, as I have a lot of disparate locations on different servers that all need to be pulled onto the backup drive every night. Now though, I should really concentrate on some revision for the exams I have after Christmas!

Happy holidays, everyone! (That's Merry Christmas and a happy new year, but just between you and me).

To create the VSS snapshot, I used a script sourced from an MSDN blog called CreateShadow, which I modified slightly to suit my purpose. I had it keep the temporary variables script, so I could use it later on (once the backup has finished) to delete the snapshot.

Once the snapshot is created and exposed, I used Robocopy with the mirror (/MIR) switch, to copy the contents to the backup drive. It just so happens that the backup drive is connected to a Samba server running on Ubuntu. This meant that I ran into a problem with timestamps whereby files were always classified as "newer", even if they hadn't changed at all since the last run. I fixed this by using the Fat File Times (/FFT) switch which gives a 2-second granularity on the timestamp of files, which solved the issue straight away.

The backup having completed, the script calls the temporary variables script generated by the CreateShadow script, to reinstate the snapshot ID, which is then used to remove the shadow copy cleanly.

In theory, this is an extremely efficient and robust backup system - not to mention being completely free of any licence fees. I may improve it in the future by adding functionality with multiple backup sets - at the moment I only have one day to recover from any accidental deletions - barring the previous versions.

One thing I am stuggling with at present, however, is the fact that when the backup runs under scheduled task at 3am, a number of files throw access denied errors - namely any files or directories with special characters. This is a particularly strange issue as the process works flawlessly when launched manually. I am still trying to solve the issue, but I'll be sure to post an update if and when I find the solution.

I've recently upgraded the my new server, adding another 4GB of RAM to bring the total to 8GB - to give me some more room for playing around. Also, I've decided that with all that extra memory I can upgrade to VMware Server 2.0 safely, and all seems to have gone well. Also surprisingly, the load on the machine hasn't gone up, even with the addition of 2 new virtual machines.

The first new server is an anti-spam gateway for my Microsoft Exchange organisation, and is working flawlessly so far. Second is a Windows Server 2008 machine, which I plan to migrate the domain onto some time in the future. I plan to do some work with the brand new O/S, and see what's what. I'm sure there will be plenty of material to keep up-to-date with, so keep checking back!

This can be easily achieved using the hosts file, which is present on both linux and windows operating systems. Here I will discuss how to use the hosts file in Ubuntu linux.

First, you must gain access to the file. It can only be written to as root, so the sudo command must be used, in conjunction with your favourite editor. For example:

sudo gedit /etc/hosts

As you can see, the hosts file is located at /etc/hosts.

To add entries to the hosts file, simply add another line at the end, following the president set by the one or two lines alreay present. The syntax is basic, and consists of the IP to be aliased, then the name that you would like to alias it with. For example, the proxy in my last post could be entered as follows:

128.243.253.119    proxy

As it happens, this is actually a surprisingly effective method of blocking malicious websites, and even adverts. Quite simply, any domain that you would not like your browser to access, can be added to the hosts file with an IP of 127.0.0.1. This is the IP address for the local machine you are currently working on. For a huge list of such websites, check out www.someonewhocares.org/hosts. Simply copy and paste the file on that site into your hosts file, and you're set. It should be noted that this will work on windows, Mac or linux. On Windows XP/Vista the hosts file is located in:

C:\Windows\system32\drivers\etc\hosts

Obviously if your Windows installation is on a different drive, simply replace C: with the relevant letter.

Network Proxy

First and foremost, Ubuntu has a setting in gnome for the Network Proxy, which should set gnome's proxy - but I can't see as it affects anything at all - still, better to be safe than sorry.

At Nottingham University, the recommended configuration is a proxy auto-configuration script (proxy.pac) which is downloaded by the client and parsed to configure the appropriate proxy server. In this case, it is located at http://wwwcache.nottingham.ac.uk/proxy.pac. This URL is entered into the correct field of the gnome Network Proxy settings dialogue.

Terminal Proxy

Secondly, the terminal has a proxy configuration option, so that programs that run inside the terminal making HTTP requests can access the internet - namely wget and aptitude. This is slightly more difficult to configure than the previous, and is achieved like so:

The terminal proxy is set using a variable called http_proxy, which is set using the export command, i.e.

export http_proxy="http://proxy_server_ip:port"

In this case, the proxy server's IP and port for the SNS (Student Network Service) is 128.243.253.119:8080.

This change can be made permanent by editing /etc/bash.bashrc, and adding the above line to the end of the file. Otherwise, the change is only effective in the terminal window currently open by the user, and disappears when it is closed.

Synaptic

Lastly, Synaptic Package Manager must have the proxy set, in order to update your installation using the in-built Update Manager or Synaptic GUI. This is done by opening Synaptic, and choosing Settings, Preferences, and setting the above proxy information using the Network tab. Unfortunately Synaptic cannot read auto-configuration scripts, so the IP and port must be manually entered here.

Obviously once all this is done, Firefox must be configured to use the correct proxy - but I trust you know how to do that! This turns out to be quite a pain, so I'll be looking at ways to do this in one fell swoop. If anyone has any suggestions, then please let me know.

Note: These changes can be made much less painful by adding an entry to the hosts file for each of the proxy IPs you need to configure - so you only need to type the word proxy for example, instead of the entire IP. I will document this process shortly in a separate post.

sudo apt-get update

sudo do-release-upgrade

So, hopefully the improvements to gnome, and the new Dust theme will go down a treat when the upgrade's finished. Personally I'm looking forward to the new BBC iPlayer integration with Totem, Ubuntu's media player - and hoping for some improvements to Microsoft Exchange connectivity within Evolution, the mail client.

I shall keep the world up-to-date, as this journey continues!

This post

made extremely interesting and informative reading – if one can understand the material in that post, then a lot of load issues can be easily resolved– especially IOWait issues (my particular concern). My issue turned out to be a mixture of the settings above, and Postfix misbehaving.

See the word document here.

Update: Advice followed, PDF available here.

Cacti is a complete network graphing solution designed to harness the power of RRDTool's data storage and graphing functionality.

Brilliant. Network graphing is good, I want to see pretty charts and graphs about how my network is doing. So I gave it a go. Here's some of my ups-and-downs, and the end result.

First, I needed a linux machine to try this on. Cacti itself obviously wasn't enough of a challenge for me, I wanted to get it to work on an operating system with which I had very little experience. I chose Ubuntu Server 7.10 - I've worked with Ubuntu before, and I like the Aptitude package manager which would make this project somewhat easier for me.

So first of all, I installed the O/S. I'm using a Virtual Machine on my main VM host, which had some RAM to spare. I only have the machine 128MB, as I'm not going to be asking too much of it (hopefully). I didn't specify a LAMP install, even though that is exactly what would be required. I wanted to do all the fiddly stuff later on.

Once the O/S was on, I needed to install the required packages, and then Cacti itself. Cacti requires a web server, with PHP and GD (the image library), and a MySQL server. I followed this guide to get them all installed on this new machine, and then extracted and set up Cacti.

Worth noting here, is that when importing the cacti.sql file into the MySQL database, I first created the database called "cacti", then modified the cacti.sql file, adding "use cacti" to the beginning of the file - otherwise an error stating "no database selected" would appear.

Once the database was setup, and Cacti was extracted - I pointed Firefox to http://cacti/cacti (I had a creative moment and called the Cacti server cacti). The setup process was web-based from here-on, and Cacti was installed in a matter of seconds.

So, now I added my hosts (after enabling the SNMP service on my Windows Servers, and configuring the community), and created some graphs. Just network traffic graphs at first. After a few polls, I was amazed to see the graphs populating perfectly. After following these instructions I made them look so much better (maybe not sexy, though!), and the result was something like this:

Wonderful. Pretty graphs showing me how much the internet connection is being used. 100k eh? Somehow I think paying for 20Mb isn't worth it!

OK, so now I have lots of nice graphs, I wanted to get a Network Weathermap working - which is like a virtual network diagram, showing the traffic between each node on the map - as it reads the data from the same source as Cacti.

This was much easier than I thought - after adding the nodes and links into the config file, the values took on the colours of my scale as they should - and I had a lovely diagram of my network with automatically updating traffic information! Here's the end result.

And there we have it! Not at all as bad as I was expecting. I do hope this will be of help to anyone wanting to do something similar.