Rob Golding

Recently, I’ve been doing a lot of work trying to integrate Linux & Windows machines on a single network. The project begun with a need to allow windows users access to file shares using Samba, and Linux users access to the same data using NFS. As you may know, Samba (or more specifically the SMB protocol) uses a username/password combination to authenticate users, while NFS uses user IDs and group IDs (UIDs & GIDs) on the local machines to achieve the same end. Therefore, I needed a way to consistently authenticate Windows domain users on Linux machines, whilst maintaining a consistent conversion from SID -> UID, and vice-versa.

Domain authentication can be achieved relatively simply, by using either Winbind (part of the Samba project) or the new kid on the block, Likewise. Likewise-Open offers a simple procedure for joining domains, and the new version comes packaged with it’s own version of Kerberos. It also hashes the Windows SID into a UNIX UID in a consistent manner – so the UIDs are always the same across your entire environment. Sounds perfect, right? Well, no. Not quite. The problem with Likewise-Open, is that it’s difficult to integrate with Samba. Though it does ship with a “compatibility module” called lwi_compat, which allows Samba to hook into Likewise’s authentication module, I found this quite difficult to get working, and I only achieved partial success through guesswork – as the documentation didn’t actually help much, given they only officially support Samba 3.0.x (while Ubuntu now uses 3.3.x). When I did get it going, however, it only recognised the Windows users’ primary group, not any of the other groups they were members of. This meant that my (possibly overly) complex system of ACLs and user directories just didn’t work at all. So, on to Plan B…

When I read through the short (but sweet) Ubuntu Wiki article entitled ActiveDirectoryWinbindHowto, I felt like somewhat of a fool after reading a small, illusive section called Adding more than one Linux machine to a Windows network. Bingo! This section described a problem whereby the traditional Winbind domain authentication method would lead to inconsistent UIDs across the network, and thus cause headaches when trying to achieve anything like what I was aiming for. It suggested using a method of mapping SIDs to UIDs called RID. I assume this stands for Relative ID, which is another kind of ID Active Directory uses to track users within a domain. These can possibly clash from domain to domain, so it is advised not to use this method when your network contains a trust between multiple Windows domains, but for the simpler setup (like my own) it’s a godsend.

This meant that I could use RID mapping within Winbind, which is part of Samba itself (so no troubles integrating those two), and achieve a consistent SID-UID mapping scheme across the network, allowing me to finally enable access to the file shares via. NFS. Amazingly, NFS “Just Worked” straight away, and I’ve written some nice wrapper programs around chown, getfacl and setfacl to set the correct owner and permissions on entire directory trees, which saves a lot of time when your UIDs are changing as often as mine were! I’m also using autofs to automatically map user’s home directories on the Linux machines, which has proven itself to be very useful. I just used static fstab entries to map the other “general” file shares, like software and media – as I couldn’t seem to get autofs direct maps working (apparently they are only partially working in Ubuntu anyway, but it seems as though they are completely broken to me).

On a side note, I’ve also just finished developing a Python-based rsync backup program, which allows me to write a _very_ small script to backup remote servers using rsync over SSH, and tar up the contents of all the servers into one archive. This is really useful, as I have a lot of disparate locations on different servers that all need to be pulled onto the backup drive every night. Now though, I should really concentrate on some revision for the exams I have after Christmas!

Happy holidays, everyone! (That’s Merry Christmas and a happy new year, but just between you and me).

To go with the recent network upgrades and anti-spam system, I have been working on a new way to back all this information up. The solution I’ve come up with is surprisingly simple: VSS Snapshots with Robocopy to mirror the changes. The basic idea is that the backup script creates a Volume Shadow Copy Service Snapshot, and “exposes” (mounts) the snapshot with an unused drive letter. Robocopy then mirrors the contents of this snapshot to the backup drive, allowing even files that are locked to be backed up. Add in a bit of error-checking and status emails, and we have a pretty solid backup system. I’ll run through the details below.

To create the VSS snapshot, I used a script sourced from an MSDN blog called CreateShadow, which I modified slightly to suit my purpose. I had it keep the temporary variables script, so I could use it later on (once the backup has finished) to delete the snapshot.

Once the snapshot is created and exposed, I used Robocopy with the mirror (/MIR) switch, to copy the contents to the backup drive. It just so happens that the backup drive is connected to a Samba server running on Ubuntu. This meant that I ran into a problem with timestamps whereby files were always classified as “newer”, even if they hadn’t changed at all since the last run. I fixed this by using the Fat File Times (/FFT) switch which gives a 2-second granularity on the timestamp of files, which solved the issue straight away.

The backup having completed, the script calls the temporary variables script generated by the CreateShadow script, to reinstate the snapshot ID, which is then used to remove the shadow copy cleanly.

In theory, this is an extremely efficient and robust backup system – not to mention being completely free of any licence fees. I may improve it in the future by adding functionality with multiple backup sets – at the moment I only have one day to recover from any accidental deletions – barring the previous versions.

One thing I am stuggling with at present, however, is the fact that when the backup runs under scheduled task at 3am, a number of files throw access denied errors – namely any files or directories with special characters. This is a particularly strange issue as the process works flawlessly when launched manually. I am still trying to solve the issue, but I’ll be sure to post an update if and when I find the solution.

My home network has been growing and growing ever since the start. At first, just little linux firewall, and it’s not done yet.

I’ve recently upgraded the my new server, adding another 4GB of RAM to bring the total to 8GB – to give me some more room for playing around. Also, I’ve decided that with all that extra memory I can upgrade to VMware Server 2.0 safely, and all seems to have gone well. Also surprisingly, the load on the machine hasn’t gone up, even with the addition of 2 new virtual machines.

The first new server is an anti-spam gateway for my Microsoft Exchange organisation, and is working flawlessly so far. Second is a Windows Server 2008 machine, which I plan to migrate the domain onto some time in the future. I plan to do some work with the brand new O/S, and see what’s what. I’m sure there will be plenty of material to keep up-to-date with, so keep checking back!

In my last post, I talked about configuring Ubuntu for use with different proxy servers, and something became quite evident. It would have been nice to have a way of aliasing the IPs of those servers, so instead of typing the number out, one could simply type “proxy”, for example.

This can be easily achieved using the hosts file, which is present on both linux and windows operating systems. Here I will discuss how to use the hosts file in Ubuntu linux.

First, you must gain access to the file. It can only be written to as root, so the sudo command must be used, in conjunction with your favourite editor. For example:

sudo gedit /etc/hosts

As you can see, the hosts file is located at /etc/hosts.

To add entries to the hosts file, simply add another line at the end, following the president set by the one or two lines alreay present. The syntax is basic, and consists of the IP to be aliased, then the name that you would like to alias it with. For example, the proxy in my last post could be entered as follows:

128.243.253.119    proxy

As it happens, this is actually a surprisingly effective method of blocking malicious websites, and even adverts. Quite simply, any domain that you would not like your browser to access, can be added to the hosts file with an IP of 127.0.0.1. This is the IP address for the local machine you are currently working on. For a huge list of such websites, check out www.someonewhocares.org/hosts. Simply copy and paste the file on that site into your hosts file, and you’re set. It should be noted that this will work on windows, Mac or linux. On Windows XP/Vista the hosts file is located in:

C:\Windows\system32\drivers\etc\hosts

Obviously if your Windows installation is on a different drive, simply replace C: with the relevant letter.

I have recently enrolled on a Computer Science course at The University of Nottingham, and as such have had to make sure my machines correctly use their proxy server for web access. This post outlines the process of configuring Ubuntu for exactly that purpose – and could be applied to any network with a similar layout.

Network Proxy

First and foremost, Ubuntu has a setting in gnome for the Network Proxy, which should set gnome’s proxy – but I can’t see as it affects anything at all – still, better to be safe than sorry.

At Nottingham University, the recommended configuration is a proxy auto-configuration script (proxy.pac) which is downloaded by the client and parsed to configure the appropriate proxy server. In this case, it is located at http://wwwcache.nottingham.ac.uk/proxy.pac. This URL is entered into the correct field of the gnome Network Proxy settings dialogue.

Terminal Proxy

Secondly, the terminal has a proxy configuration option, so that programs that run inside the terminal making HTTP requests can access the internet – namely wget and aptitude. This is slightly more difficult to configure than the previous, and is achieved like so:

The terminal proxy is set using a variable called http_proxy, which is set using the export command, i.e.

export “http_proxy=http://proxy_server_ip:port”

In this case, the proxy server’s IP and port for the SNS (Student Network Service) is 128.243.253.119:8080.

This change can be made permanent by editing /etc/bash.bashrc, and adding the above line to the end of the file. Otherwise, the change is only effective in the terminal window currently open by the user, and disappears when it is closed.

Synaptic

Lastly, Synaptic Package Manager must have the proxy set, in order to update your installation using the in-built Update Manager or Synaptic GUI. This is done by opening Synaptic, and choosing Settings, Preferences, and setting the above proxy information using the Network tab. Unfortunately Synaptic cannot read auto-configuration scripts, so the IP and port must be manually entered here.

Obviously once all this is done, Firefox must be configured to use the correct proxy – but I trust you know how to do that! This turns out to be quite a pain, so I’ll be looking at ways to do this in one fell swoop. If anyone has any suggestions, then please let me know.

Note: These changes can be made much less painful by adding an entry to the hosts file for each of the proxy IPs you need to configure – so you only need to type the word proxy for example, instead of the entire IP. I will document this process shortly in a separate post.

The next incarnation of the Ubuntu linux distribution came out at the end of this week, and whilst typing this post, I am upgrading my laptop’s operating system. Unfortunately, I was unable to upgrade in the way that is described on the website here, instead my update manager seemed to think that the system was completely up-to-date, even after multiple refreshes. Therefore, I’m just upgrading the old-school way, by running the following commands:

sudo apt-get update

sudo do-release-upgrade

So, hopefully the improvements to gnome, and the new Dust theme will go down a treat when the upgrade’s finished. Personally I’m looking forward to the new BBC iPlayer integration with Totem, Ubuntu’s media player – and hoping for some improvements to Microsoft Exchange connectivity within Evolution, the mail client.

I shall keep the world up-to-date, as this journey continues!

I have recently upgraded my home server, shelling out on a new Core 2 Duo CPU and 4GB of RAM for the machine. Having this much RAM means that in order to use it, I had to install a 64-bit O/S. I chose Ubuntu Server 8.04 – and VMware Server to host my virtual machines. I have put together a Word Document with some notes on the issues and tips I came accros on the way, which could prove invaluable to anyone taking the same approach as me. One of the main sources for my research and tinkering ideas was a post on the VMware Community forums – http://communities.vmware.com/thread/146002 – linked to inside the Word Document.

This post

made extremely interesting and informative reading – if one can understand the material in that post, then a lot of load issues can be easily resolved– especially IOWait issues (my particular concern). My issue turned out to be a mixture of the settings above, and Postfix misbehaving.

See the word document here.

Update: Advice followed, PDF available here.

While improving the network at my house (an indeed, the network which supports this very web server), I started to explore the world of network monitoring and reporting. I had heard quite a bit about Cacti before, but never considered installing it. That was mostly due to the stories I had heard about how unholy difficult the damn thing is to get working properly. “Don’t even go there” was my mindset. Until now, that is.

Cacti is a complete network graphing solution designed to harness the power of RRDTool’s data storage and graphing functionality.

Brilliant. Network graphing is good, I want to see pretty charts and graphs about how my network is doing. So I gave it a go. Here’s some of my ups-and-downs, and the end result.

First, I needed a linux machine to try this on. Cacti itself obviously wasn’t enough of a challenge for me, I wanted to get it to work on an operating system with which I had very little experience. I chose Ubuntu Server 7.10 – I’ve worked with Ubuntu before, and I like the Aptitude package manager which would make this project somewhat easier for me.

So first of all, I installed the O/S. I’m using a Virtual Machine on my main VM host, which had some RAM to spare. I only have the machine 128MB, as I’m not going to be asking too much of it (hopefully). I didn’t specify a LAMP install, even though that is exactly what would be required. I wanted to do all the fiddly stuff later on.

Once the O/S was on, I needed to install the required packages, and then Cacti itself. Cacti requires a web server, with PHP and GD (the image library), and a MySQL server. I followed this guide to get them all installed on this new machine, and then extracted and set up Cacti.

Worth noting here, is that when importing the cacti.sql file into the MySQL database, I first created the database called “cacti”, then modified the cacti.sql file, adding “use cacti” to the beginning of the file – otherwise an error stating “no database selected” would appear.

Once the database was setup, and Cacti was extracted – I pointed Firefox to http://cacti/cacti (I had a creative moment and called the Cacti server cacti). The setup process was web-based from here-on, and Cacti was installed in a matter of seconds.

So, now I added my hosts (after enabling the SNMP service on my Windows Servers, and configuring the community), and created some graphs. Just network traffic graphs at first. After a few polls, I was amazed to see the graphs populating perfectly. After following these instructions I made them look so much better (maybe not sexy, though!), and the result was something like this:

Wonderful. Pretty graphs showing me how much the internet connection is being used. 100k eh? Somehow I think paying for 20Mb isn’t worth it!

OK, so now I have lots of nice graphs, I wanted to get a Network Weathermap working – which is like a virtual network diagram, showing the traffic between each node on the map – as it reads the data from the same source as Cacti.

This was much easier than I thought – after adding the nodes and links into the config file, the values took on the colours of my scale as they should – and I had a lovely diagram of my network with automatically updating traffic information! Here’s the end result.

And there we have it! Not at all as bad as I was expecting. I do hope this will be of help to anyone wanting to do something similar.

As an update to the previous post regarding the installation of a new IPCop as my network firewall, I have finally completed the configuration of its VPN service for use as a Roadwarrior. I can now connect to the IPCop machine from my laptop, using the OpenVPN client from anywhere in the world.

I was surprised with the ease of configuration once an addon called “Zerina” was installed. This made the process extremely simple to complete, even offering to package up an OpenVPN configuration file and certificate combination – so all that is needed to connect is one click!

With regards to the IPCop machine itself, it is one of the most stable servers I have ever put into operation. I literally installed the O/S (about 50mb) a couple of weeks ago – and since then there has been not one issue. Not even a restart – it’s just been chugging away on that old 400MHz Pentium II. I am in awe of the little thing – which is actually proving to be a damn sight faster than the overpowered and clunky ISA Server that I used to use.

Also, with the terrible OpenVPN logo, and the lack of suitable IPCop art, I hope the visio diagram to the left bears a resemblance to this post that could be appreciated by the reader. I definitely think it makes the post something special, would you not agree?

My latest project, to replace the bulky overpowered ISA firewall on my home network with a lean mean IPCop machine, was declared a great success a few days ago.

I am familiar with IPCop, as I used to use it a long time ago. Since then it has matured somewhat, but the feature set is pretty much the same as I remember. The new machine is a 400MHz PII, with 192mb RAM. It is sitting in the place of a Sempron 3000+ with 1GB RAM. Amazing, it’s doing the same job with a fraction of the power. And also, it uses a third of the electricity – 30W in total. Good news given the rise in energy prices!

The main challenge so far, which I still haven’t overcome, is how to get RoadWarrior VPN working, using the windows built-in VPN client, with L2TP/IPSec. This used to be trivial with ISA Server, but this isn’t quite the case with a linux firewall. I have been looking at other distributions such as monowall and pfSense, niether of which seem to spell out their ability to achieve this clearly. I am playing with a few of these on Virtual Machines, so hopefully I will come accross a way to do this before long – I’m starting to miss my RoadWarrior VPN server. How sad, eh?