- Articles

Django in Production: Part 3 - Automation & Monitoring

2012-01-14 13:25:00 +0000

This is the third part in a series about Django in Production. If you haven't already, read part 1 and part 2 first.

In the first two posts of this series, I described the core stack which powers a Django application in production, and the Celery task queue which can be used to execute code asynchronously. In this third post, I'll describe how a production Django application can be monitored, and how common tasks such as deployment can be automated.

Monitoring Django Applications

There are many ways to monitor a Django application, but one that is particularly useful is Graphite.

Graphite

Graphite provides real-time visualization and storage of numeric time-series
data on an enterprise level.

The key thing is not to be scared by the word enterprise. Graphite is a relatively simple 3-part system: Whisper is an efficient, pure-Python implementation of a round-robin database, and Carbon is a daemon which manages the Whisper database and provides caching. Finally, the Graphite "webapp" provides a Django frontend to the data stored in the Whisper database.

Graphite's web interface is, admittedly, hard to use. Its redeeming feature is a powerful URL-based API which allows you to compose graphs programatically - which is in some ways easier than navigating the difficult menu system. Once mastered, though, Graphite can produce some amazingly detailed stats about the very deepest internals of your application.

Clearly, the key to great graphs is lots (and lots) of data. In fact, 37signals report that their servers recorded an incredible 100,000,000 measurements in the first 10 days of 2012. Those measurements were made using a statsd, which is a Node.js daemon for collecting statistics over a simple UDP protocol, and sending them to Graphite.

Statsd

The great thing about statsd is that it uses UDP to collect statistics. This means that a client can be written in almost any language, and when the server isn't running the client is completely unaffected. After all, statistics are important, but not important enough to knock the entire application offline when they aren't being collected.

So far, there's been nothing Django-specific (save for the fact that Graphite is written in Django). In fact, these systems can be used to monitor just about anything.

django-statsd provides a very useful set of basic statistics for Django applications, though, and is highly recommended. Strangely, it's installed from PyPi with pip install django-statsd-mozilla, as the name was already taken by another app.

Once installed, just enable a couple of middleware classes which record every request/response and their execution time to statsd. For more specific stats, though, the client library is very easy to use:

stats.py

from django_statsd.clients import statsd
statsd.incr('response.200')

Interestingly, it also integrates with django-debug-toolbar, and can "monkeypatch" Django to enable model/cache statistics - but I'll leave that as as exercise for the reader.

Automation

Often, in their quest for the least effort possible, programmers automate too much. In the case of deploying applications, though, which tends to be a simple but repetitive task, I think we're justified.

Fabric

If you haven't already, you should spend some time reading about Fabric. Essentially, it's a Python library that allows SSH commands to be scripted, whether they are on one or many remote hosts.

Fabric is a Python library and command-line tool for streamlining the use of
SSH for application deployment or systems administration tasks.

Fabric uses a file at the top-level of a project, called fabfile.py, to define the functions which can be used on the command line.

There are some very complete examples of "fabfiles" which can be used for Django deployment, such as this one by Gareth Rushgrove.

For my projects, which involve only one server, I use a rather more simplified version:

import os from fabric.api import env, require, run, sudo, cd env.project_name = '' env.server_name = '' env.webapps_root = '/opt/webapps/' env.project_root = os.path.join(env.webapps_root, env.project_name) env.activate_script = os.path.join(env.project_root, 'env/bin/activate') env.wsgi_file = os.path.join(env.project_root, 'django.wsgi') env.repo_root = os.path.join(env.project_root, 'repository') env.search_index = os.path.join(env.project_root, 'search_index') env.requirements_file = os.path.join(env.repo_root, 'requirements.txt') env.manage_dir = os.path.join(env.repo_root, env.project_name) def production(): env.hosts = [env.server_name] prod = production def virtualenv(command, use_sudo=False): if use_sudo: func = sudo else: func = run func('source "%s" && %s' % (env.activate_script, command)) def update(): require('hosts', provided_by=[production]) with cd(env.repo_root): run('git pull origin master') def install_requirements(): require('hosts', provided_by=[production]) virtualenv('pip install -q -r %(requirements_file)s' % env) def manage_py(command, use_sudo=False): require('hosts', provided_by=[production]) with cd(env.manage_dir): virtualenv('python manage.py %s' % command, use_sudo) def syncdb(app=None): require('hosts', provided_by=[production]) manage_py('syncdb --noinput') def migrate(): require('hosts', provided_by=[production]) manage_py('migrate') def rebuild_index(): require('hosts', provided_by=[production]) manage_py('rebuild_index --noinput', use_sudo=True) sudo('chown -R www-data:www-data %(search_index)s' % env) def collectstatic(): require('hosts', provided_by=[production]) manage_py('collectstatic -l --noinput') def reload(): require('hosts', provided_by=[production]) sudo('supervisorctl status | grep %(project_name)s ' '| sed "s/.*[pid ]$[0-9]\+$\,.*/\\1/" ' '| xargs kill -HUP' % env) def deploy(): require('hosts', provided_by=[production]) update() install_requirements() syncdb() migrate() collectstatic() reload()

As you can probably tell from the code, this performs the following operations at the push of a button:

Updates the server's Git repository
Installs any new requirements with pip
Synchronises the database to create any new tables
Executes any South migrations that haven't yet been run
Links any static media
Reloads the webserver (see part 1 for the Gunicorn setup)

Once this file is in place, executing fab prod deploy from anywhere in the project's source tree will automatically deploy the latest code to the web server in seconds. With public key authentication, you're not even prompted for a password. As far as I'm concerned, that's well worth the effort!

Django in Production: Part 2 - Background Tasks

2011-11-27 20:58:00 +0000

This is the second part in a series about Django in Production. If you haven't already, read part 1 here.

In my last post, I described in detail the core stack which powers a Django application in production. In this second part, I'll talk about something which has become commonplace in modern web applications - background tasks.

Often websites perform complex tasks which take a long time to finish. These tasks don't always need to be done before the response is sent to the browser, so can be executed elsewhere. In cases like this, tasks can be performed asynchronously, by one or more "workers" which consume a task queue.

Celery

The most common task queue used with Django, at least in my experience, is called Celery.

Celery is an asynchronous task queue/job queue based on distributed message passing. It is focused on real-time operation, but supports scheduling as well.

Celery isn't dependant on Django in any way - it's a completely separate application. It does, however, integrate extremely well with Django projects. By installing the django-celery package and then adding djcelery to the list of INSTALLED_APPS, Celery can be run and managed through Django itself - thanks to a set of handy management commands.

This post isn't about how to get Celery working, so I'll skip over the details. For a more information on using Celery with Django, the project's documentation is excellent.

Celery In Production

Running Celery locally is easy: a simple python manage.py celeryd does the trick. In production, though, something a little more robust is needed.

Thankfully, it's not difficult at all. Just like Gunicorn, Celery can be overseen by Supervisor.

If you didn't read Part 1 of Django in Production (The Stack), now would be a good time to do that.

To add Celery to our Supervisor configuration, the following is all that's required:

[program:app_name_celery_worker]
command=/path/to/env/bin/python manage.py celeryd -E -l info
directory=/path/to/app/
user=www-data
autostart=true
autorestart=true
stdout_logfile=/path/to/app/django_logs/celeryd.log
redirect_stderr=true

In fact, Celery even comes with some example Supervisor configuration files, which can be adapted to suit the particular needs of a production setup.

This configuration can be added to the same file which contains the application server's program directives. This keeps all the daemons for an application neatly together in one file.

Worker Nodes

In development, a Celery worker would typically be spun up in another terminal window, alongside the development server. This same technique can't be assumed though, for a production setup.

At first, perhaps, the task queue worker(s) might be running on the same server as the application itself. As an application grows, there comes a point where it makes sense to have at least one server dedicated to running asynchronous tasks. This can then be scaled out to suit, by adding more and more worker servers as required.

RabbitMQ

Celery uses message passing to queue tasks for the workers to consume. This is provided by an altogether separate application - the message broker.

Celery's most commonly used message broker, and that with the most support, is RabbitMQ. The details of how to setup RabbitMQ are a little out of the scope of this post, but the server documentation are complete and provide a good starting point.

Virtual Hosts

In a large application with more than one worker node, the message broker would typically sit on a dedicated host (or even be clustered over multiple hosts). In this case, multiple applications might share the same message broker.

RabbitMQ supports this type of configuration through the use of virtual hosts (or VHosts). Much like virtual hosts used in Apache, these allow a single broker to be "split" into multiple, distinct parts. Access control can be configured on a per-vhost basis, so any single application is prevented from affecting another through the use of a shared broker.

Even if your message broker is only used by one application, that application should be configured with its own virtual host and credentials.

Monitoring

It's difficult to know exactly what a task queue like Celery is doing, as it doesn't present a pretty web interface for us to watch. It can, however, be monitored - and this is something I'll be talking about in detail in the next post of this series.

Part 3 of this series is now available: Automation and Monitoring.

Django in Production: Part 1 - The Stack

2011-11-12 22:35:00 +0000

Everyone has their preferred way of doing things, and this is more and more true when there are many options available. In the Django world, this translates to everyone having their favourite web server, database, proxy, and so on.

In spite of this, I'm going to spend some time over the next few posts describing how I deploy Django applications in production from a high-level perspective. In the first part of this series, I'll talk about the core stack which serves as the basis of the application.

Gunicorn

Python-based web applications have traditionally been run under Apache, using a module such as mod_python or mod_wsgi. Apache, however, can be somewhat of a resource-hog, particularly on virtual servers which often have limited memory. Also, as I'll discuss in detail later in this series, deploying code frequently to an Apache-hosted application can be troublesome. A new solution is in order.

There is quite a choice of Python-based web servers for us "cool kids" to use, many of which are tested in this detailed (if a little old) benchmark by Nicholas Piël.

Gunicorn offer super simple configuration, an extremely small footprint, and it's pure Python - so you can install it with a quick pip install gunicorn.

Serving Your App

Running the Gunicorn server couldn't be simpler. Similar in nature to the Django development server, you simply execute python manage.py run_gunicorn from within your project's directory. In production, however, there's an even better method.

Gunicorn installs an executable script to your environment, called gunicorn_django. To use it, you specify which settings module to use on the command line: gunicorn_django path/to/project/settings. We can use this to serve an application from a process control system such as Supervisor.

Supervisor

On a production server, the application needs to start and stop with the operating system. To achieve this, a process control system is usually required (particularly for daemons that don't do this job themselves, like Gunicorn's). Supervisor is a particularly good example, and is pretty simple to setup. On Ubuntu, install it with sudo apt-get install supervisor. Then place a configuration file in /etc/supervisor/conf.d/app_name.conf with the required options. An example configuration, shown below, shows how a Django application might be served by Gunicorn under the www-data user from its own virtualenv, and save all its output to a log file. The server also starts automatically, and is restarted if it crashes.

[program:app_name]
command=/path/to/env/bin/gunicorn_django app_name/settings
directory=/path/to/app/
user=www-data
autostart=true
autorestart=true
stdout_logfile=/path/to/app/django_logs/supervisord.log
redirect_stderr=true

nginx

Due to the simplicity of Gunicorn, it isn't designed to be exposed to the world as a "bare" server. Instead, it's recommend that a reverse proxy is placed in front - which does the job of communicating with the client (browser) directly.

It's down to the way that worker processes are used, so a (perhaps deliberately) slow client can easily perform a Denial of Service (DoS) attack, which ends up taking down the entire application. For a more detailed explanation, see the deploy page in Gunicorn's documentation.

As it happens, nginx does a fine job of acting as a reverse proxy. It's also extremely fast, stable, and easy to configure. The following configuration sets up nginx to serve an application running on port 8000 (the default for Gunicorn):

upstream app_server {
    server localhost:8000;
}

server {
    listen          80;
    client_max_body_size    4G;
    keepalive_timeout       5;

    access_log  /var/log/nginx/app_name.access.log;

    location /static/ {
        alias   /path/to/app/staticfiles/;
    }

    location / {
        proxy_pass          http://app_server;
        proxy_redirect      off;
        proxy_set_header    Host            $host;
        proxy_set_header    X-Real-IP       $remote_addr;
        proxy_set_header    X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

You might have noticed, too, that this configuration serves static media from a directory called staticfiles, within the application's folder. I would recommend that the Django contrib app (by the same name) is used to manage the static media for the project - and this is easily done by executing python manage.py collectstatic on the server.

MySQL

The last thing I'd like to mention in this first part of the series is the database server. The Django community in general, I've found, tends to prefer PostgreSQL - though my leaning is towards MySQL. The decision here isn't really too important, so I'm not going to go into the pros and cons of each. In the end, as is often the case, it simply comes down to personal preference.

Database Migrations

When deploying code to an application server, it will often be necessary to perform database migrations. In this case, South is the perfect tool for the job. Also, the relevant libraries must be present on the server for your particular database. For MySQL, this is installable from pip as a module named mysql-python.

To begin with, it's difficult to get used to the workflow of creating schema migrations, applying them to the database, and then committing the tested files to the database. Once the initial hurdles are cleared, though, it makes an awful lot of sense. In a future post I'll talk about how to automate the process of migrating a production database using this very technique.

In the next post in this series, I'll be talking about executing long-running background tasks with Celery.

Part 2 of this series is now available: Background Tasks.

django-radius

2011-09-19 16:48:00 +0100

I've been working on a little Django app for a while now, which solves a problem I often come across in Django projects. django-radius allows a developer to use a 3rd party RADIUS server to provide authentication in their Django application. Not only this, though, it also allows the use of multiple RADIUS servers, which can be specified in whichever way makes sense.

This might sound a little contrived, so I'll give an example. Imagine a web service, such as a helpdesk management system, whose users are organised into groups. Each user belongs to an organisation, which buys into the service on their behalf. Basecamp and Zendesk operate this model, to name just two. To login, a user must visit a particular URL - so that the system may determine which organisation that user belongs to (http://yourcompany.zendesk.com for example).

Now imagine that this service wishes to provide external authentication to its customer's systems, using RADIUS. This means that there's no need for users to have yet another password on yet another system, and is generally a Good Thing™.

In this scenario, the developers would have to find some way of validating the user's credentials using the correct RADIUS server, based on the URL they used to access the site with. For this type of problem, django-radius is the perfect solution.

This is my first real attempt at doing things properly in a Django app, and quite some effort has gone into documenting both the usage and the code. I'll leave it to those that are interested enough to read the README and find out how it works.

Hopefully this app will save a little time in someone else's project, and I'd love to hear from anyone with comments or suggestions on how it can be improved.

The End of an Era

2011-06-26 00:00:00 +0100

Well, here we are. Three years at The University of Nottingham, and what do I have to show for it? Well, this is what:

That's right! I have a recommendation for a first in Computer Science. It turns out both my supervisor and second marker were fond of my dissertation - Backtrac Backup System - and would like to see it used in a commercial setting. I also had my work sponsored by Servat Ltd., which went down very well with the powers that be.

I've enjoyed my time at university, but I'm ready to move on now. Or at least, I'd better be, after turning down a couple of very good offers for PhD studentships.

So what's next?

Why, I'm glad you asked! I'm currently working on a (secret) project with Marcus Whybrow, which will hopefully shape up to be something quite special. It is, of course, a Django project, and it's teaching us both about the finer details of developing and deploying a production system. There's a few posts in the pipeline about the things we've discovered this far, and we've hardly got going yet!

This time around, we've decided to go with a brand new web server all the cool kids are using: gunicorn. It's sitting behind an nginx reverse proxy, which also serves up the static media. Also, Fabric is shaping up to be a super time-saving tool, well worth writing about at some point in the near future.

So, well done to all those who have completed their degrees this year. I look forward to working with you in what people can't seem to stop calling the "real world".

Django on twistd.web.wsgi - Issue Workaround

2011-02-05 00:00:00 +0000

My last few posts have generally been about my final year project - Backtrac Backup System. One of the recent challenges I faced was getting the Django-based web interface to run under the Twisted WSGI server. In this post, I'll describe a major issue with this process, and how I worked around it.

TAC or TAP?

When Twisted is installed, it places an executable program called twistd on your path. The purpose of this program is to run a twisted application as a daemon, and manage all of the tricky daemonization code for you. This includes things like forking, PID files, and logging.

There are two ways to describe your application two the Twisted daemon: TAC files and TAP files. I'm pretty sure that TAC stands for Twisted Application Configuration, whereas TAP stands for Twisted Application Plugin.

The difference between the two, it seems, is that TAC files pass an Application object to twistd, while TAP files pass a ServiceMaker object (whose purpose is to return a Service object, when makeService is called).

Also, given that TAP files define aServiceMaker object which Twisted will call a method on, we can accept a parameter called options, which contains the parsed command-line arguments passed to the application. This excludes any arguments that were passed to twistd itself. As far as I can tell, this is the main different between TAC and TAP files. It also results in TAP files generally being longer and more verbose than their TAC counterparts.

For my application, I needed to accept an argument on the command-line, so I was tied to using a TAP file.

The Plugin

There are loads of examples of TAC files to run Django projects out there, but none for TAP files (at least that I could find, with my excellent Google-fu). So, I sat down with the Twisted API documentation, and started to write one!

server.py

class ServerServiceMaker(object):
    implements(IServiceMaker, IPlugin)
    tapname = 'backtracweb'
    description = 'Backtrac web server'
    options = Options

    def makeService(self, options):
        # make a new ThreadPool and start it
        pool = threadpool.ThreadPool()
        pool.start()

        # create WSGI resource using the thread pool + Django handler
        resource = wsgi.WSGIResource(reactor, pool, WSGIHandler())

        # create the site and a TCPServer service to serve it with
        site = server.Site(resource)
        web_service = internet.TCPServer(port, site)

        return web_service

serviceMaker = ServerServiceMaker()

Issue #4347

Now, this plugin works flawlessly when the -n flag is passed to twistd (don't daemonize). This means the server stays attached to the terminal, like the Django development server. When this flag is removed, however (as it would be when the app is used for real), the server doesn't respond to requests. Strangely, it is still running, and listening on the correct port - but it simply hangs when a request is made.

It turns out that this issue has already been reported on Twisted's Trac instance. 11 months ago. It's actually caused by the thread pool being started before the daemonization code is executed. This consists of forking, and killing off the parent twice - which in turn kills off the thread pool. Whoops!

The Workaround

One of the comments on the ticket suggests creating a custom service (implementing the IService interface) which starts a given thread pool when it is started. This means that the thread pool would be started after the program has daemonized itself, thus solving the issue. So, I got to work:

workaround.py

class ThreadPoolService(service.Service):
    def __init__(self, pool):
        self.pool = pool

    def startService(self):
        service.Service.startService(self)
        self.pool.start()

    def stopService(self):
        service.Service.stopService(self)
        self.pool.stop()

This is a dead simple class, and it should work once it's hooked into Twisted. Therein lies the problem though - how to combine these two services into just one - to return from the makeService method.

The answer came from a very helpful fellow in answer to my Stack Overflow question: use a MultiService. From here, everything fell right into place. The final TAP file includes a few more details that deal with command-line arguments and config file parsing, and it works flawlessly. I've also added a snippet from the linked example, which allows the static media to be served up by Twisted as well. Generally, I've been pleasantly surprised at the speed and stability of the Twisted web server.

For me, this was a frustrating issue, but I'm happy that I was able to work round it to achieve a successful result. I hope this post allows others to do the same.

Using Django Authentication with Twisted Perspective Broker

2010-11-10 00:00:00 +0000

For my third year project, I have hooked Twisted's Perspective Broker authentication into Django, so that a networked application can authenticate itself against Django's central user database (django.contrib.auth). The process is pretty quick and easy, thanks to Twisted's pluggable nature, and Django's pure simplicity.

The Checker

Firstly, we create a class which will actually do the "checking", to allow access (or not) given a username/password combination. That looks something like this:

auth.py

from zope.interface import implements
from twisted.python import failure, log
from twisted.cred import portal, checkers, error, credentials
from twisted.internet import defer

from django.contrib.auth.models import User, check_password

class DjangoAuthChecker:
    implements(checkers.ICredentialsChecker)
    credentialInterfaces = (credentials.IUsernamePassword,
                            credentials.IUsernameHashedPassword)

    def _passwordMatch(self, matched, user):
        if matched:
            return user
        else:
            return failure.Failure(error.UnauthorizedLogin())

    def requestAvatarId(self, credentials):
        try:
            user = User.objects.get(username=credentials.username)
            return defer.maybeDeferred(
                check_password,
                credentials.password,
                user.password).addCallback(self._passwordMatch, user)
        except User.DoesNotExist:
            return defer.fail(error.UnauthorizedLogin())

This code is relatively self-explanatory, once you understand Twisted's way of doing things. An authenticated "user" in Twisted (though this might not actually be a person) is called an Avatar. The requestAvatarId method therefore takes a set of credentials, and returns the Django user object that corresponds to those credentials (or a failure if necessary). This return value is wrapped up inside a Deferred, which is something that I won't go into detail about in this post. For more information on Deferred objects, this is one area where the Twisted documentation is quite detailed.

Edit: Fixed a bug in the above code on line 24, thanks to Tom Leys of GridSpy for pointing it out.

The Realm

The next thing we need to do is create a "realm", which will essentially convert the Django user object into an "avatar" - which is the object that the server will use to communicate with the client:

realm.py

class MyRealm(object):
    implements(portal.IRealm)

    def requestAvatar(self, user, mind, *interfaces):
        assert pb.IPerspective in interfaces
        avatar = MyAvatar(user)
        avatar.attached(mind)
        return pb.IPerspective, avatar, lambda a=avatar:a.detached(mind)

class MyAvatar(pb.Avatar):
    def __init__(self, user):
        self.user = user

    def attached(self, mind):
        self.remote = mind
        print 'User %s connected' % (self.user,)

    def detached(self, mind):
        self.remote = None
        print 'User %s disconnected' % (self.user,)

As you can probably tell, the realm has no responsibility for authentication. It simply accepts a user object (usually a username in normal Twisted apps), and returns an avatar. Here, I've written a very simple avatar that will print a message when each user connects and disconnects. The "mind" object allows the server to call remote methods on the client, and is documented (though not in great detail) in the Twisted Perspective Broker Authentication documentation.

Linking Things Together

That's pretty much it for the authentication code. All that's left now is to actually link this into a Perspective Broker:

pb.py

checker = DjangoAuthChecker()
realm = MyRealm()
p = portal.Portal(realm, [checker])

reactor.listenTCP(self.port, PBServerFactory(p))
reactor.run()

Client Login

To login to a Perspective Broker with authentication enabled is, you'll be glad to know, very simple:

factory = pb.PBClientFactory()
reactor.connectTCP('127.0.0.1', 8123, factory)
d = factory.login(
    cred.credentials.UsernamePassword('username', 'password'),
    client=self
)
d.addCallback(connected)
d.addErrback(error)
reactor.run()

Hopefully it's obvious what is going on here. Once we "login" to the factory providing the connection, a pair of callbacks are added to the deferred - the connected function will be called if we are successfully logged in, and the error function will be called if there is some kind of error (like an UnauthorizedLogin, for example). The success callback will be given the perspective as the only argument, so it can begin to call methods on the server in the usual way.

If you've solved this particular puzzle in some other way, or if it's just been of use - then I'd love to hear from you. In any case, this is an excellent example of how simple integration can be when you're using well documented open source systems. Cheers!

Fixing Google Chrome's 100% CPU Usage on Ubuntu

2010-06-07 00:00:00 +0100

I've been battling with this issue for a couple of months now, ever since I upgraded to Ubuntu 10.04 on my main desktop machine. Google Chrome would constantly max out one core, with no indication of what it was doing, or why.

Today, I finally solved the issue. It turns out that something to do with the m4 package was causing the error, with the following output:

MozPlugger: Error: Failed to execute m4.
Mozplugger: M4 parsing of config generated error

So, I checked out the package in question, and found that it wasn't installed. To my great relief (and some frustration), a simple sudo apt-get install m4 fixed the problem instantly.

I only hope that this post solves this extremely annoying issue for someone else.

Backtrac Implementation - The Major Decisions

2010-06-03 00:00:00 +0100

Thinking a lot about my 3rd year project, I've been considering how to implement a backup system based around a Django server. So far, I have reached the conclusion that the system will consist of three main parts:

The Django server (yet to be named)
The client daemon (backtracd)
The backup library itself (backuplib)

As I mentioned in the previous post, the Django server will be based around Celery. This will offer me the ability to schedule a task for a certain date and time in the future, which is a huge part of the implementation taken care of.

One thing I am yet to figure out, however, is how the server will communicate with the client daemons. At the moment I'm thinking XML-RPC, but it seems to me that it will end up a sort of mish-mash of different technologies - as the Django server will be exposing a JSON API (probably via Piston). I have also considered running Django on the clients too, so everything can use JSON/Piston, but this seems at the moment like an unnecessary burden.

Another thing to consider is the data that is to be transmitted between the nodes. Should I use a bunch of fields in a JSON object to represent the success/failure of a job, and how much data was backed up, etc.? Another option is to pickle a Result object of some sort - representing a result as a Python instance to be interpreted by the server. These are the things I love to think about when designing a system, and I'll certainly have plenty to write about in the inevitable report.

Something I can work on in the meantime, however, is backuplib - the client library that will actually copy the data. At the moment, it's just a wrapper around rsync/tar. That probably won't change, but it will need to get a bit smarter in the way it handles failures (and restores, of course) if it's going to make for the basis a decent backup system.

3rd Year Project: Back to Backtrac

2010-05-25 00:00:00 +0100

After contacting the university regarding my 3rd year project/dissertation, my ideal choice has been confirmed. I'll be making a distributed backup system, based around a Django web interface and AMQP.

Backtrac is a pretty old project, and one whose name I am likely to change once I'm working on it. It was started in my first semester of university last year, when I didn't have enough work to keep me busy. I didn't know enough about Django back then, though, so it was left unfinished and only half-working.

Now I think I know enough to pull this off, and it's a chance to do something I really enjoy - while getting marks for it in the process! If everything goes to plan I'll be supervised by Dr. Julie Greensmith, which I'm really looking forward to.

I'll be using an app called Celery to handle the background tasks, which will be simple "pings" to the client machines, telling them to start a backup. Celery uses AMQP, provided by a broker such as RabbitMQ (which is written in Erlang). It's usually used for processing intensive computational tasks, but can also be put to use for jobs that need executing on a periodic basis, like backups.

I can tell you're getting too excited, so I'll stop. Rest assured, though, I will be writing about the discoveries I make along the way to finishing this ambitious project.

Django Staging Server: Apache Configuration

2010-05-07 00:00:00 +0100

In this post, I'd like to talk a little about how my staging server (called Kaylee, after the character from Firefly), is configured to run the multiple Django projects that I have on the go. A lot of the other articles I have read on this subject use mod_python but my server is configured to use WSGI, so that's what I will be going over in this post.

Beginnings

Firstly, I use virtualenv to keep the Python environments for the different projects separate, but that's a topic in itself. This isn't a requirement, but it helps if you have dependencies that you don't want installed system-wide, or projects that use different versions of a particular package.

You can read more about virtualenv and Django here.

The directory structure for my /opt/webapps/ directory looks something like this:

webapps/
|-- apache
|   |-- coral.conf
|   |-- coral.wsgi
|   |-- myuni.conf
|   `-- myuni.wsgi
|-- coral
|   |-- env
|   `-- repository
|-- myuni
|   |-- env
|   `-- repository

The two projects shown here are Coral Issue Tracker and MyUni. The directory for each project contains the virtual env for that project, and the repository. Alongside the project folders is a directory called apache, which holds two files for each project: the WSGI file (<project-name>.wsgi) and the Apache configuration file (`.conf``).

Apache Configuration

Then, in the main Apache configuration, I use a directive that includes all these files automatically:

Liquid error: ClassNotFound: no lexer for alias 'conf' found

The individual project configuration files then contain a set of simple directives almost straight from the Django deployment documentation. For example, the MyUni configuration looks like this:

Liquid error: ClassNotFound: no lexer for alias 'conf' found

WSGI Files

The WSGI files are also pretty simple, though there are a couple of things worth explaining. Here is the WSGI file for the MyUni project:

Liquid error: ClassNotFound: no lexer for alias 'wsgi' found

The site.addsitedir() call in the WSGI file adds the library path in the virtual environment to the python path, by inspecting it to find all the packages that are installed. I then append the path to the project's repository to the path also, as is standard practice in this type of deployment.

The only other thing worth mentioning here, is the PYTHON_EGG_CACHE variable that I add to the environment. This is required if you are using Python modules that are packaged up as eggs, so that Apache can uncompress them in a directory to which it can write (/tmp/ on Ubuntu is 777 by default). If you are not using eggs, you can skip this line out altogether. Alternatively, you can uncompress the eggs first: $ unzip [package].egg. You can read more about this issue, and the workaround, here.

Summary

That's pretty much all there is to it. This setup also makes it pretty easy to deploy new versions of my projects, for which I have started using Fabric. I'll save that for another post though.

If you do things differently, then I'd love to hear about how - so please leave a comment!

Extending Settings Variables with local_settings.py in Django

2010-05-03 00:00:00 +0100

I discovered this hacky way to use the local_settings trick to extend and/or override values in the main Django settings file today. Some projects use a "reverse" version of the local settings trick (which is explained below), whereby the main settings file becomes settings_local.py or something similar, which first imports settings.py, and then extends or overrides the values as required.

I didn't want to change the name of the project settings file to settings_local.py, however, as it would mean changing the WSGI file on every server that the project runs on.

The Local Settings Trick

It's a well-known trick to use a file called local_settings.py or something similar, with a piece of code at the bottom of the main settings file:

settings.py

try:
    from local_settings import *
except:
    pass

This allows you to override the value of settings variables. It won't work, however, if you wish to extend a settings variable (for example, adding an app to INSTALLED_APPS). For this, I have found that the following ugly hack seems to do the job.

The Ugly Hack

For this hack, replace the snippet at the bottom of the main settings file with the following code:

settings.py

try:
    LOCAL_SETTINGS
except NameError:
    try:
        from local_settings import *
    except ImportError:
        pass

What this is doing is effectively checking for the presence of a variable called LOCAL_SETTINGS. The local settings file then contains this code:

local_settings.py

LOCAL_SETTINGS = True
from settings import *

This means that the local settings file will only be imported once, and it has all the variables in the main settings file available to extend at will. For example, to enable the Django debug toolbar:

local_settings.py

MIDDLEWARE_CLASSES +=
('debug_toolbar.middleware.DebugToolbarMiddleware',)
INTERNAL_IPS = ('127.0.0.1',)
INSTALLED_APPS += ('debug_toolbar',)

The code is ugly (and results in the main settings file being parsed twice) but it works!

Django Model Templates (sans-Denormalisation)

2010-04-27 00:00:00 +0100

I came across an interesting problem recently, while trying to model the structure of a university course in Django.

The model needed to represent the notion of a university module, which can be taught over a number of semesters and/or years, by different people, and with different students each time round. Some information remained common to each of these modules however, such as the code, name, and type of the module (single semester or full-year).

One solution that crossed my mind was to use a sort of template, and copy the common information over each time the module was taught. This would mean duplicating the common fields over a template model and the module's model, and copying the data every time a new "Module" object gets created. This just didn't sit well with me though, as I really don't like denomalising data when it's not absolutely necessary.

The solution I have settled on (for now, anyway) works something like this: I use the model template idea, but with a foreign key from the Module to it's template. Then, to access the fields on the template directly from the Module model, I use a combination of the Python property and a smart lambda function.

Here's a simplified version of the model to illustrate what I'm talking about:

models.py

class ModuleDefinition(models.Model):
    code = models.CharField(max_length=6, unique=True)
    name = models.CharField(max_length=200)
    credits = models.IntegerField(max_length=4)
    level = models.IntegerField(max_length=2)
    type = models.CharField(max_length=30, choices=MODULE_TYPE_CHOICES)

class Module(models.Model):
    code = property(lambda s: s._get_definition_property('code'))
    name = property(lambda s: s._get_definition_property('name'))
    credits = property(lambda s: s._get_definition_property('credits'))
    level = property(lambda s: s._get_definition_property('level'))

    definition = models.ForeignKey('ModuleDefinition')
    semester = models.CharField(max_length=20, choices=SEMESTER_CHOICES)
    year = models.CharField(max_length=20, choices=YEAR_CHOICES)
    convener = models.ForeignKey('auth.User', related_name='modules_convened')
    students = models.ManyToManyField('auth.User', blank=True, related_name='modules_taken')

    def _get_definition_property(self, property):
        return self.definition.__getattribute__(property)

In this model I've called the template a "definition", but the idea is the same. The _get_definition_property() method allows me to get a property from the parent definition programmatically, which is used in the lambda functions to add the fields from the definition to the model itself, as read-only properties. This allows me to access them as if they were a field stored on the model itself (i.e. module.code, module.name, etc.) which is handy in the templates where module.definition.code would get very old, very fast.

I'd be really interested to hear how others have tackled the same issue, as I'm sure it's not just restricted to university modules. Maybe soon I'll be making a post about how I model a timetable in Django...after all, how hard can it be!?

Oh, and if you're wondering, this code is for the MyUni project that is occupying me recently. I'm restricting myself to just thinking about how the models could work (and maybe writing little bit of code) until Rob and Ben come back to university.

MyUni and Django 1.2

2010-04-23 00:00:00 +0100

MyUni is hailed as my "latest venture" on the front page of my website. In fact, it was conceived early last year, when I was in my first year at Nottingham University, by Rob Miles and Ben Jenkinson. We were only just starting to get to grips with this new fad, called Django.

The project has gone through several iterations since then, never really reaching any form of completeness. This time round, we are using proper version control (thanks to Mercurial and BitBucket), and trying to adhere to Django best practices as much as possible.

Straight from the repository website:

MyUni is a portal-style web application for Universities, offering students
a central location to access all the relevant information about their course,
modules, assignments, and staff members.

The project came about because of a passionate hatred for the WebCT system that our university is using, and a general feeling that things could be (and should be) better.

Django 1.2 will bring some really exciting new features, which I am sure would make developing and using MyUni a better experience. One of the things I'm most looking forward to using is the messages framework. I plan to use it for a bunch of things, possibly including form error messages...maybe.

Now, if only the release date would stay in the same place for more than a week!

"Change is Coming"

2010-04-12 00:00:00 +0100

I'm sure that I am not the only one waiting for the new release of Ubuntu, 10.04 Lucid Lynx.

My netbook has been in need of a reinstall for quite a while now, and this will be the perfect opportunity - if only slightly inconvenient that it falls right in the middle of one of the busiest periods of the year for me!

I first expected that with 10.04 being a Long-Term Support (LTS) release, it would be lighter on new features, and heavier on bugfixes and the like. That doesn't seem to be the case, however, with a healthy list of new stuff on the website. I'm most looking forward to the faster boot times, as a slow boot often gets in my way when I'm starting up at the beginning of a lecture.

What's your most anticipated new feature of Ubuntu 10.04?

New Site Design

2010-03-13 00:00:00 +0000

It dawned upon me quite suddenly recently, that my site has been running this same design (Freshy 2 by Jide) since it was brought online in late 2007. That, people, is not acceptable!

So, I have been working on a totally new design, of my own creation this time. The main site has already been changed over, and I'm working on themeing the Wordpress install as we speak.

I'll be sure to update you when it's complete. In the meantime however, feedback, as always, is most welcome.

Update: The new design is now live! I'll still be working on it though, fixing up any little niggles that are bound to crop up.

Mercurial on University of Nottingham Computer Science Servers

2010-03-01 00:00:00 +0000

For a while now, I've been using Subversion to keep track of personal projects and coursework at university. SVN is installed on the UNIX servers as Computer Science, so it's a relatively trivial process to get up and running with a repository when I start a new assignment.

Recently though, I've been looking in to Mercurial as a more modern alternative. Some things about Subversion are really starting to annoy me, like the inability to ignore files on a repository-wide level easily (I work with Python all the time, so .pyc files can really get on my nerves!). Mercurial seems slicker, and I'm interested in the concept ofDVCS, as opposed to the classic client-server way of thinking.

However, Mercurial is not installed on the (rather ancient, now) Solaris servers at university. I really admire the job that the sysadmins do, so I am in no way condemning them for it not being installed. Instead, I set out to get it compiled and working myself.

Installing it on the servers, it seemed, was the easy part. I followed the installation instructions (making sure to include the LD_LIBRARY_PATH variable) and all was well.

Cloning from the server, however, wasn't so easy. The problem lies in the fact that Mercurial is now installed for me, and me alone. The hg binary lives in my home directory, not on the server's main path (i.e. the /usr/bin/ directory). Therefore, I needed to tell the client exactly what command to run on the server. If anyone else is having the same trouble, the command that I finally came up with looks like this:

hg.sh

hg --config ui.remotecmd="LD_LIBRARY_PATH=/usr/sfw/lib PYTHONPATH=~/lib/python ~/bin/hg"

I then aliased this command to hgtuck to save my sanity (the server is named tuck, after Friar Tuck in the Robin Hood legends), and I now have a perfectly working Mercurial install!

If anyone else is looking to do something similar, then I hope this little tip saves you some time!

Custom Section Numbering in LaTeX

2010-02-28 00:00:00 +0000

For our last coursework (which was for a really interesting compilers module), I chose to present my answers in LaTeX. It's been a very steep learning curve, but I'm extremely happy with the results. I don't even mind the slight drop in productivity that's caused by my uncontrollable urge to stop and admire my document every so often! One thing that had me stuck for a while, however, was the automatic section numbering system.

To answer the questions for the compilers coursework, I wanted my section_s to be numbered (1, 2, 3), my subsection_s to be numbered (a, b, c), and my subsubsection_s to be numbered (i, ii, iii). This numbering pattern matches that which is given in the question paper, whereby a number of nested enumerate_s were used. The default LaTeX section numbering however is (1, 1.1, 1.1.1). I finally found a way to alter this default behaviour by using the following code in the preamble:

renew.tex

\renewcommand{\thesubsection}{(\alph{subsection})}
\renewcommand{\thesubsubsection}{\roman{subsection}.}

This code modifies the way that the counters for subsections and subsubsections are printed (section counters are correct when left at the default).

I hope this saves someone even the few minutes it took me to work this out, once I'd applied some logic! It's very useful to know that LaTeX has a counter for everything. This particular post was very helpful in working out which ones to alter, and how.

Linux-Windows Integration (à la Likewise-Open & Winbind)

2009-12-24 00:00:00 +0000

Recently, I've been doing a lot of work trying to integrate Linux and Windows machines on a single network. The project begun with a need to allow windows users access to file shares using Samba, and Linux users access to the same data using NFS. As you may know, Samba (or more specifically the SMB protocol) uses a username/password combination to authenticate users, while NFS uses user IDs and group IDs (UIDs & GIDs) on the local machines to achieve the same end. Therefore, I needed a way to consistently authenticate Windows domain users on Linux machines, whilst maintaining a consistent conversion from SID to UID, and vice-versa.

Domain authentication can be achieved relatively simply, by using either Winbind (part of the Samba project) or the new kid on the block, Likewise. Likewise-Open offers a simple procedure for joining domains, and the new version comes packaged with it's own version of Kerberos.

It also hashes the Windows SID into a UNIX UID in a consistent manner - so the UIDs are always the same across your entire environment. Sounds perfect, right? Well, no. Not quite. The problem with Likewise-Open, is that it's difficult to integrate with Samba. Though it does ship with a "compatibility module" called lwi_compat, which allows Samba to hook into Likewise's authentication module, I found this quite difficult to get working, and I only achieved partial success through guesswork - as the documentation didn't actually help much, given they only officially support Samba 3.0.x (while Ubuntu now uses 3.3.x).

When I did get it going, however, it only recognised the Windows users' primary group, not any of the other groups they were members of. This meant that my (possibly overly) complex system of ACLs and user directories just didn't work at all. So, on to Plan B...

When I read through the short (but sweet) Ubuntu Wiki article entitled ActiveDirectoryWinbindHowto, I felt like somewhat of a fool after reading a small, illusive section called Adding more than one Linux machine to a Windows network. Bingo! This section described a problem whereby the traditional Winbind domain authentication method would lead to inconsistent UIDs across the network, and thus cause headaches when trying to achieve anything like what I was aiming for. It suggested using a method of mapping SIDs to UIDs called RID. I assume this stands for Relative ID, which is another kind of ID Active Directory uses to track users within a domain. These can possibly clash from domain to domain, so it is advised not to use this method when your network contains a trust between multiple Windows domains, but for the simpler setup (like my own) it's a godsend.

This meant that I could use RID mapping within Winbind, which is part of Samba itself (so no troubles integrating those two), and achieve a consistent SID-UID mapping scheme across the network, allowing me to finally enable access to the file shares via. NFS. Amazingly, NFS "Just Worked" straight away, and I've written some nice wrapper programs around chown, getfacl and setfacl to set the correct owner and permissions on entire directory trees, which saves a lot of time when your UIDs are changing as often as mine were! I'm also using autofs to automatically map user's home directories on the Linux machines, which has proven itself to be very useful. I just used static fstab entries to map the other "general" file shares, like software and media - as I couldn't seem to get autofs direct maps working (apparently they are only partially working in Ubuntu anyway, but it seems as though they are completely broken to me).

On a side note, I've also just finished developing a Python-based rsync backup program, which allows me to write a very small script to backup remote servers using rsync over SSH, and tar up the contents of all the servers into one archive. This is really useful, as I have a lot of disparate locations on different servers that all need to be pulled onto the backup drive every night. Now though, I should really concentrate on some revision for the exams I have after Christmas!

Happy holidays, everyone! (That's Merry Christmas and a happy new year, but just between you and me).

It Begins...

2009-09-22 00:00:00 +0100

The country has just undergone a mass-exodus of university students from their parents' houses back into halls. I played by own small part in blocking up the roads moving back into university accommodation this weekend, and it's all gearing up for the new school year.

I've been working with Django more and more lately, and I've written my first "commercial" application using my new favourite framework - a booking system for taught causes at the City Council. I'm really enjoying writing web applications with Django, and I'm sure this blog will start to resemble a web-developer's in the near future.

Also, I'm really looking forward to this year at university. As I understand it, there's a lot more work to be done, but the software engineering group project should be fun - as long as my "randomly chosen" team are happy with us using Python!

Exchange 2007 Autodiscover Issues

2009-07-07 00:00:00 +0100

Over the past week, I've been upgrading my e-mail system to Exchange 2007 (I was previously using Exchange '03). For me, it's very useful to have the systems that I write about, and consult for, installed at home in a "semi-production" manner. It means that I have a system to work on, and I am concerned with keeping it up and running as smoothly as possible - which introduces me to intricacies that I would not otherwise encounter, if I were just running a little test lab.

For example, when setting up Outlook Anywhere (the rebranded RPC/HTTP feature of Exchange, allowing Outlook users to connect from outside the organisation) I discovered a lot of "Sync Issues" appearing in my Inbox. The messages all had a common theme:

11:19:07 Synchronizer Version 12.0.6315
11:19:07 Synchronizing Mailbox 'User'
11:19:07 Synchronizing Hierarchy
11:19:07 Done
11:19:09 Microsoft Exchange offline address book
11:19:09  Not downloading Offline address book files. A server (URL) could not
be located.
11:19:09 0X8004010F

Clearly, something was wrong with the Offline Address Book. I was only getting these messages when using Outlook Anywhere, however, so this issue was obviously specific to RPC/HTTP.

Looking up the error code, I found that the problem I was experiencing was very common, but that nowhere seemed to have the ultimate repair. The information available was sparse, and I had to put together my own solution - which I will document below.

First, I registered an extra DNS (A) record for my email domain, called "autodiscover". I must be clear here, that this is for the mail domain, not for the domain used to access your OWA site. For example (and we'll go with the Microsoft classic here), if your users have addresses such as user1@contoso.com, user2@contoso.com and you access your OWA via https://mail.contoso.com/owa, then you need to register an A record for autodiscover.contoso.com.

Next, I prepared a new certificate request, that would hopefully end up with me obtaining a certificate that I could use to replace the current one, which would be valid for both mail.contoso.com and autodiscover.contoso.com (to continue with our example) - so that my Outlook clients could successfully access the autodiscover service, and download the OAB. To do this, I used the following EMS command:

Liquid error: ClassNotFound: no lexer for alias 'ps1' found

This command requires a little explanation. The -domainname switch is used to specify a list of addresses for which this server is valid. This is called a SAN (Subject Alternative Name). Not all CA's support SANs, but Windows Server 2008's CA Services does, which I will come back to later. Next, we give the certificate a "Friendly Name", which is just a reference for you, the administrator. Then we specify that we are looking to generate and save a request, and that we want to be able to export the private key. The Subject Name is important, but also slightly confusing. You must specify your country code (US, GB, ES), your organisation name, and Common Name (CN) - which is the most important one. This must be the URL used to access the SSL service using a web browser, so mine was mail.contoso.com.

Once this request is saved, I passed it on to my CA to get the certificate issued. If you're using a 3rd party CA (like VeriSign), then you'll have to check first whether they support SANs. I use self-signed certificates, and my CA is running Windows Server 2008, which does support SANs, so I issued the request internally. This is done by accessing the CertSrv website, at http://servername/certsrv, and clicking the "Request a Certificate" link. Then, I chose "Advanced Request", and pasted the request file's contents into the box, and picked the "Web Server" template.

This presented me with a downloadable certificate, which I saved locally in CER format on the exchange server. Then I used the following command to import the certificate:

Liquid error: ClassNotFound: no lexer for alias 'ps1' found

Once the certificate was imported, I enabled it for use with exchange. A similar command is used for this:

Liquid error: ClassNotFound: no lexer for alias 'ps1' found

This prompted for a list of services, where I entered IMAP, IIS, SMTP as these are the default installed services. Only IIS actually gets used here, so I shouldn't worry too much about this one. If you're not sure, then just enter the same as me. Lastly, it asked for a thumbprint, which I copied and pasted from the output of the import command. Finally, after accepting the confirmation, the certificate was enabled.

And that was it. Both OWA and Outlook Anywhere are now working perfectly, and hopefully this post will help at least one other lost soul with the same problem!

The Trials and Tribulations of Django + Git

2009-06-02 00:00:00 +0100

I just finished my last exam today - Web Programming and Scripting - which explains the distinct lack of activity around here in recent times. Thankfully I could end my exam season on a high, as web programming is, well, what I do - so it wasn't too much of a challenge!

Something strange happens to me every time exams come around. I seem to pick up new projects, and just run with them. This time, I've become involved with a small group of people at university, writing a portal-style information system for universities. I suppose most people call this behavior procrastination, but I'm quite deeply in denial about that.

Ever since my post about the Backtrac Backup System, I've been really enjoying using Django. Something about it just makes developing for the web, well, exciting. That can only be good, right? I am the designated server administrator for this latest project, mostly due to the fact that I am the only one with a server to administer, and some of the things I've learned so far seem worthy of a mention here. Firstly, we as developers were - how can I put it - stepping on each other toes somewhat. The project at this point had no version control, so we were just editing a bunch of files over SFTP. Obviously, some sort of Source Control Management was in order. I did some research, and decided that Git was a nice, modern alternative to the ever-popular SVN. It also meant that my server was constantly backed up by everyone on the team - but that's just a bonus!

So, I installed Git, and started a repository. A lot of effort went into learning how the system works, and more importantly, how to make it work for us. Directed Acyclic Graphs thankfully made some sense to me, so I could just about understand the documentation. I wrote some custom hooks, and a C Program to synchronise the web server. I was happy, and absolutely certain that this was the solution to all our woes. I was mistaken. Git just didn't work the way I had hoped. The custom hooks were throwing permission errors all over the place, and my development team (read: my friend Rob Miles) was locked out of the repository. We made the decision yesterday to scrap Git, and go back to the previous system of editing the files over SFTP. We are always in constant communication when developing for the project, so it's not too big a deal, but I feel that I failed as an administrator. You see, as an admin your job isn't just to play with cool toys and loud servers - your primary purpose is to give the users what they need to work, and that is most certainly not what I achieved.

I'm glad I took the time to learn Git - and I'm sure it will help me later in life, in some way or another. It's just unfortunate that it didn't work out the way I had hoped for our project.

Roaming Firefox Profiles

2009-04-13 00:00:00 +0100

As a sort of follow-on from my last post, I thought I'd write a little about the latest little addition to my system - roaming Firefox profiles. This is something I had always considered to be almost impossible to achieve, without complicated logon and logoff scripts that synchronise the correct folder(s) to give the same effect as a 'redirected' profile. Well I discovered a much easier way to achieve actual real profile redirection, when browsing around the features offered by Group Policy Client Side Extensions.

The basic idea is that Firefox has a file called profiles.ini, which takes care of all the configured profiles, and where they are stored. I used this file to change the default profile location to within the user's home directory on the file server. I had to use a home drive, mapped to the root of my users' folder redirection directory on the server, as I presumed UNC paths were unsupported in the .ini file. I did this with a GP Preference drive map using the %USERNAME% variable, and an amazing feature of GP Preferences - parsing and even altering ini files. You can specify which section of the ini file you are interested in, and which key you want changing. How useful! My policy looks like this:

You can probably work out what's going on here, but I'll give a quick overview. Basically, you specify the ini file to edit - in this case it's %APPDATA%\Mozilla\Firefox\profiles.ini. Using the %APPDATA% variable means that it will always resolve to the correct location in the user's local profile, whether they are on XP or Vista. Then we specify the section of the ini file - I'm interested in Profile0. This is the only profile present by default, but allows users to have multiple profiles if they wish without overwriting their settings when they log off. Finally, you specify the key to change, and what to change it to. I rename the default profile to Firefox, and change it's location to H:\Firefox. Also, this path is not relative, so I have to change IsRelative to 0.

OK, so now we have the Firefox profile location sorted, we need to make sure that folder exists - or Firefox will just overwrite our changes and make it's own folder in the default location. It's easy to use GP Preferences for this as well - as there's a Folders preference category. So I just made a new folders preference for \zeus\UserData\%USERNAME%\Firefox, with the action of create (zeus is my main DC and File Server). I used the UNC path to be sure that the folder is created, even if the drive map hadn't come into effect when this preference was applied. Also, a little trick I had to pull here was ticking the checkbox titled Run in logged-on user's security context on the common tab. This is because only the user has permissions on their home directory, so this preference needed to run in the context of that user for it to work successfully (without access denied errors).

Once this was finished, the system started to work flawlessly. I copied the contents of existing Firefox profiles to the newly created directories, and they were picked up by Firefox with no problems. New users get blank profiles as expected, but they are stored on the file server instead of the local machine. One little issue I have encountered is that a user can't logon at more than one machine, and start Firefox - as the program can't lock particular files in the profile. This just results in a message saying this Firefox is already running though, which is pretty much correct (and I can't see why this would ever cause problems for the user).

The last trick I employed, to make things a little speedier and to reduce unnecessary file server traffic, was to disable disk caching on the roaming Firefox profiles. To do this, I used a file policy in GP Preferences to copy a tiny user.js (Firefox's preference file override) which contained only one line:

firefox-pref.js

user_pref("browser.cache.disk.enable", false);

This turns off disk caching completely, which will not only save space on the file server, but should speed things up as well. I hosted this file elsewhere on the file server, and told the file policy to simply copy it into place, within the user's Firefox profile. Here's the preference:

So there you have it, my technique for enabling roaming Firefox profiles. If you've achieved the same through a different method, or have any ideas on this this could be improved, then I'd love to hear how - feel free to comment on this post.

Windows Server 2008 Migration

2009-04-10 00:00:00 +0100

Thanks to the MSDNAA program, I'm able to try out the latest version of Windows Server in the lab. I opted to migrate my domain across to a new machine, instead of performing an in-place upgrade. Personally I feel this is a much safer bet, and tend to migrate domain controllers whenever I'm doing something pretty major to a DC.

So far everything looks good, I've upped the forest and domain functional level to Server 2008, so I can now take advantage of some of the new features - though I'm yet to find out what they all are! The best thing so far (by a mile I'd like to add) is the addition of Group Policy Preferences. Although it's annoying having to install the Client-Side Extensions on every machine in the domain (that is if WSUS isn't in use), the gains far outweigh this bit of pain. I only wish an MSI could have been released, so that it could easily be pushed out using the existing Group Policy infrastructure. Never mind, eh?

Anyway, on with the good! The new GP Preferences allow an administrator to define, amongst others, drive maps for client machines, printer connections and power options. As you may be thinking, this just about does away with the need for logon scripts! Most, if not all of the common tasks that are performed with logon scripts can now be done from a group policy object.

There are also a lot of changes to the way Active Directory works. In Server 2008, Active Directory Domain Services can be installed on a machine, without actually making it a DC. What this means is that a standard server build can be 'sysprepped' with the files required for promoting the server to a DC, without actually doing the promotion. Also, Read-Only Domain Controllers (RODCs) have been introduced as a new feature. Essentially, an RODC just caches queries from a normal DC, usually located at another site - apparently allowing for faster logon times at remote sites with slow links. After discussion with a colleague, however, the benefits of such a system are maybe not quite as advertised. For example, only one RODC can be installed per site - so larger sites can't benefit from the redundancy and load balancing offered by multiple DCs, if RODCs are used. Also, the much-touted security advantages of using an RODC aren't as they seem either, as the database can be just as easily written to, just through another "normal" DC.

My Latest Project: BackTrac Backup System

2009-03-06 00:00:00 +0000

My life has been pretty busy as of late, mostly with a new project I am working on called Backtrac. I am developing a network backup solution, written entirely in Python - using the Django framework as a front-end web interface. This came at quite a good time, as I've been asked to give a presentation on any highly technical topic, on which I know my stuff. The plan is to use Backtrac as a base, and to explain the technical concepts behind the system that make it work. The things I am going on concentrate on are:

Pyhon in general
XML-RPC
Filesystem hardlinks
The MVC concept, and Django

This should be enough content to fill a 15-minute slot, I hope.

Now I'll explain a little bit about how Backtrac works, for the benefit of those not coming to the presentation.

What I wanted to achieve with Backtrac was a smart backup system, that doesn't necessarily have to be the fastest. I wanted a very detailed web interface, with scheduling capabilities and log view. I decided on Django for this, seeing as I had already decided on Python for the system itself.

The nodes in the system use XML-RPC to communicate, and SMB for the actual file-copying. The real bonus of this system is that it takes advantage of a little-understood feature, present in most modern file systems, called hardlinks. Hardlinks enable the user to essentially point to the same piece of data on a disk from more than one position. Essentially the concept is that if a file has been backed up before, why back it up again? Just create a link or "shortcut" (a hardlink) to the previously backed-up file. This is the basis for Backtrac.

Also, Django is turning out to be a real treat. This is the first time I have used an MVC like Django, and I'm really enjoying it. Web development is exciting again! I do agree with one point however, that Django makes the easy things easy, and the hard things impossible. Not quite impossible in my case, but I've had to do some pretty strange things to get Django to do what I want.

Finally, I've had the Backtrac project approved on Sourceforge.net, so watch out - I'll be doing the initial import soon and getting some web content online. First though, I have to decide on the best way to distribute the application. This is a bit more complicated than it might otherwise be, because there are three different aspects of the system - the client application, the server application, and the Django project. On this topic or any other, as usual, opinions are most welcome.

Leveraging VSS and Robocopy for Robust Backups

2009-01-14 00:00:00 +0000

To go with the recent network upgrades and anti-spam system, I have been working on a new way to back all this information up. The solution I've come up with is surprisingly simple: VSS Snapshots with Robocopy to mirror the changes. The basic idea is that the backup script creates a Volume Shadow Copy Service Snapshot, and "exposes" (mounts) the snapshot with an unused drive letter. Robocopy then mirrors the contents of this snapshot to the backup drive, allowing even files that are locked to be backed up. Add in a bit of error-checking and status emails, and we have a pretty solid backup system. I'll run through the details below.

To create the VSS snapshot, I used a script sourced from an MSDN blog called CreateShadow, which I modified slightly to suit my purpose. I had it keep the temporary variables script, so I could use it later on (once the backup has finished) to delete the snapshot.

Once the snapshot is created and exposed, I used Robocopy with the mirror (/MIR) switch, to copy the contents to the backup drive. It just so happens that the backup drive is connected to a Samba server running on Ubuntu. This meant that I ran into a problem with timestamps whereby files were always classified as "newer", even if they hadn't changed at all since the last run. I fixed this by using the Fat File Times (/FFT) switch which gives a 2-second granularity on the timestamp of files, which solved the issue straight away.

The backup having completed, the script calls the temporary variables script generated by the CreateShadow script, to reinstate the snapshot ID, which is then used to remove the shadow copy cleanly.

In theory, this is an extremely efficient and robust backup system - not to mention being completely free of any licence fees. I may improve it in the future by adding functionality with multiple backup sets - at the moment I only have one day to recover from any accidental deletions - barring the previous versions.

One thing I am struggling with at present, however, is the fact that when the backup runs under scheduled task at 3am, a number of files throw access denied errors - namely any files or directories with special characters. This is a particularly strange issue as the process works flawlessly when launched manually. I am still trying to solve the issue, but I'll be sure to post an update if and when I find the solution.

Office 2007: "There has been a network or file permission error."

2009-01-08 00:00:00 +0000

Recently I have been having a strange issue on my desktop PC, whereby saving an Office document (Word, Excel etc.) inside the My Documents folder - which is redirected to my file server - gives the following error:

There has been a network or file permission error. The network connection may
be lost.

I've been trying to find out the cause of this for some time now, while working round it by simply saving the file to my desktop and copying it over to My Documents.

Anyway, I have just found the problem. I had a little plugin installed that allowed the indexing service on Windows Vista to index a network location, meaning I could search the My Documents folder quickly. This was the cause of the issue, and removing the index solved the problem. I wanted to write the solution up here in the hope that this helps someone in the same situation.

Note: There have been other solutions to this problem cited, including anti-virus programs, and network congestion. The KB article for this issue is located here.

Bullet Bitten: VMware Server 2.0

2009-01-03 00:00:00 +0000

My home network has been growing and growing ever since the start. At first, just little Linux firewall, and it's not done yet.

I've recently upgraded the my new server, adding another 4GB of RAM to bring the total to 8GB - to give me some more room for playing around. Also, I've decided that with all that extra memory I can upgrade to VMware Server 2.0 safely, and all seems to have gone well. Also surprisingly, the load on the machine hasn't gone up, even with the addition of 2 new virtual machines.

The first new server is an anti-spam gateway for my Microsoft Exchange organisation, and is working flawlessly so far. Second is a Windows Server 2008 machine, which I plan to migrate the domain onto some time in the future. I plan to do some work with the brand new O/S, and see what's what. I'm sure there will be plenty of material to keep up-to-date with, so keep checking back!

Dell Studio Laptop Wakes Itself up from Hibernate

2008-11-28 00:00:00 +0000

My new laptop, a Dell Studio 1535, has started exhibiting a strange problem as of late; when I hibernate it over night, it resumes early in the morning, waking me up with the fan spinning.

I was preparing to send the thing back to Dell, when it happened one night and I checked out the event logs. It turns out that Windows Update was resuming the machine from hibernate - and then updating itself. After turning this off, I've had a good night's sleep! Also, Windows Defender was scheduled to scan the machine every morning at 2.00am, so that's been disabled too.

I hope this post can provide answers to anyone else experiencing a similar issue.

The Hosts File in Ubuntu

2008-11-26 00:00:00 +0000

In my last post, I talked about configuring Ubuntu for use with different proxy servers, and something became quite evident. It would have been nice to have a way of aliasing the IPs of those servers, so instead of typing the number out, one could simply type proxy, for example.

This can be easily achieved using the hosts file, which is present on both Linux and Windows operating systems. Here I will discuss how to use the hosts file in Ubuntu Linux.

First, you must gain access to the file. It can only be written to as root, so the sudo command must be used, in conjunction with your favourite editor. The hosts file on Ubuntu (and indeed other Linux distributions) is located at /etc/hosts.

To add entries to the hosts file, simply add another line at the end, following the precedent set by the one or two lines already present. The syntax is basic, and consists of the IP to be aliased, then the name that you would like to alias it with. For example, the proxy in my last post could be entered as follows:

128.243.253.119 proxy

As it happens, this is actually a surprisingly effective method of blocking malicious websites, and even adverts. Quite simply, any domain that you would not like your browser to access, can be added to the hosts file with an IP of 127.0.0.1. This is the IP address for the local machine you are currently working on. For a huge list of such websites, check out www.someonewhocares.org/hosts. Simply copy and paste the file on that site into your hosts file, and you're set. It should be noted that this will work on Windows, Mac or linux. On Windows XP/Vista the hosts file is located at C:\Windows\system32\drivers\etc\hosts.

Obviously if your Windows installation is on a different drive, simply replace C: with the relevant letter.

Ubuntu and The University of Nottingham's Proxy

2008-11-24 00:00:00 +0000

I have recently enrolled on a Computer Science course at The University of Nottingham, and as such have had to make sure my machines correctly use their proxy server for web access. This post outlines the process of configuring Ubuntu for exactly that purpose - and could be applied to any network with a similar layout.

Network Proxy

First and foremost, Ubuntu has a setting in gnome for the Network Proxy, which should set gnome's proxy - but I can't see as it affects anything at all - still, better to be safe than sorry.

At Nottingham University, the recommended configuration is a proxy auto-configuration script (proxy.pac) which is downloaded by the client and parsed to configure the appropriate proxy server. In this case, it is located at http://wwwcache.nottingham.ac.uk/proxy.pac. This URL is entered into the correct field of the gnome Network Proxy settings dialogue.

Terminal Proxy

Secondly, the terminal has a proxy configuration option, so that programs that run inside the terminal making HTTP requests can access the internet - namely wget and aptitude. This is slightly more difficult to configure than the previous, and is achieved like so:

The terminal proxy is set using a variable called http_proxy, which is set using the export command, i.e. export http_proxy="http://proxy_server_ip:port"

In this case, the proxy server's IP and port for the SNS (Student Network Service) is 128.243.253.119:8080.

This change can be made permanent by editing /etc/bash.bashrc, and adding the above line to the end of the file. Otherwise, the change is only effective in the terminal window currently open by the user, and disappears when it is closed.

Synaptic

Lastly, Synaptic Package Manager must have the proxy set, in order to update your installation using the in-built Update Manager or Synaptic GUI. This is done by opening Synaptic, and choosing Settings, Preferences, and setting the above proxy information using the Network tab. Unfortunately Synaptic cannot read auto-configuration scripts, so the IP and port must be manually entered here.

Obviously once all this is done, Firefox must be configured to use the correct proxy - but I trust you know how to do that! This turns out to be quite a pain, so I'll be looking at ways to do this in one fell swoop. If anyone has any suggestions, then please let me know.

Note: These changes can be made much less painful by adding an entry to the hosts file for each of the proxy IPs you need to configure - so you only need to type, for example, proxy, instead of the entire IP. I will document this process shortly in a separate post.

Upgrading to Ubuntu 8.10 (Intrepid Ibex)

2008-11-02 00:00:00 +0000

The next incarnation of the Ubuntu linux distribution came out at the end of this week, and whilst typing this post, I am upgrading my laptop's operating system. Unfortunately, I was unable to upgrade in the way that is described on the website, instead my update manager seemed to think that the system was completely up-to-date, even after multiple refreshes. Therefore, I'm just upgrading the old-school way, by running the following commands:

upgrade.sh

sudo apt-get update
sudo do-release-upgrade

So, hopefully the improvements to gnome, and the new Dust theme will go down a treat when the upgrade's finished. Personally I'm looking forward to the new BBC iPlayer integration with Totem, Ubuntu's media player - and hoping for some improvements to Microsoft Exchange connectivity within Evolution, the mail client.

I shall keep the world up-to-date, as this journey continues!

Foxmarks - The addon I could no longer live without

2008-10-21 00:00:00 +0100

As time goes on I seem to be accumulating more and more machines - whether they be physical or virtual - which causes somewhat of a problem with my bookmarks. I use my bookmarks as a sort of knowledgebase, with solutions to problems I have encountered in the past for example.

The solution to this dillema: Foxmarks. I've had this addon for a long time now - since it was just a simple little bookmarks backup application - so I'm really impressed with how it's turned out. Basically, Foxmarks syncronises your bookmarks with their server, so that you can access your bookmarks from any machine with firefox installed. It also functions as a backup if your profile goes bad.

Recently I've started at university, so I'm using my laptop as my primary machine nowadays, along with a firefox profile on the Computer Science machines. I've also got a few virtual machines on my laptop with firefox, so the amount of profiles is getting larger by the day. This is where foxmarks really comes into its own, keeping my bookmarks current between all my installations.

VMware Server 2.0 Released

2008-10-17 00:00:00 +0100

It's arrived! The final release of VMware Server 2.0 was announced last week. Will you be upgrading?

I'm not going to make the leap just yet. I've just upgraded my system (see previous post), and so far it's been up with not a single hickup since installation. I don't expect to be ripping up all my hard work for a while yet.

One thing that is interesting my at the moment - what are people's opinions on the new console? Personally, I think the new console is "sort of OK". They have the right idea - being able to manage the machine from a remote location without installing client software first - afterall it is called VMware Server. This makes sense in theory, but the console is so sluggish and, in my experience so far, buggy - that it just doesn't work yet. Starting and stopping virtual machines is slow, and the remote VM console takes a good number of seconds to launch. The interface is vastly improved from the beta that I covered in a post a long time ago, but still not perfect in my opinion.

Also, the console has its own Tomcat web server running on the host machine, which eats up a large amount of RAM. This is annoying at best, and disastrous at worst - when you need all the RAM you have for the Virtual Machines this software is supposed to be managing!

Anyway, I may install this on a test bed somewhere, but certainly not on my production system for some time. I'd love to know if I'm in the majority or the minority on this one.

Ideal Ubuntu Server Configuration for VMware Host

2008-09-13 00:00:00 +0100

I have recently upgraded my home server, shelling out on a new Core 2 Duo CPU and 4GB of RAM for the machine. Having this much RAM means that in order to use it, I had to install a 64-bit O/S. I chose Ubuntu Server 8.04 - and VMware Server to host my virtual machines. I have put together a Word Document with some notes on the issues and tips I came accros on the way, which could prove invaluable to anyone taking the same approach as me. One of the main sources for my research and tinkering ideas was a post on the VMware Community forums - http://communities.vmware.com/thread/146002 - linked to within the document.

This post made extremely interesting and informative reading - if one can understand the material in that post, then a lot of load issues can be easily resolved especially IOWait issues (my particular concern). My issue turned out to be a mixture of the settings above, and Postfix misbehaving.

See the PDF here.

Using Locally-Attached Network Printers with Terminal Services

2008-07-08 00:00:00 +0100

If you work with Microsoft's Terminal Services on a regular basis, you've probably come accross a printing issue at some point. The point of this post is to provide a potential solution to the problem whereby locally attached network printers cannot be used from a remote terminal services session. By locally attached, I mean directly connected via. IP address, and not by way of a print server. This is often the case with consumer grade network printers, or wireless equivalents.

The concept is simple - attach the network printer as normal, then share it. Then, connect to the share on the local machine - thus using the printer as it is connect via a print server - with the server being the local machine. Here are the steps required to achieve this:

First attach the printer as normal. For the type of connection I am describing this is via. a TCP/IP port. So choose New Printer in Printers and Faxes, and then Local printer attached to this computer.
When asked the type of port, choose TCP/IP, and enter the IP address of the network printer.
Once the printer is connected, share it. This can be done from the wizard, or by using the Sharing and Security panel afterwards.
Then, choose New Printer again, this time selecting a network printer. Then enter \<share_name> as the path to the printer. Obviously machine name is the name of your computer, and share name is the name with which you shared the printer in the previous step.
Note: This step will not introduce a new printer item to your list of printers, so don't be alarmed when nothing new appears. Just connect to the terminal server as in the last step.
Once this is done, connect to your terminal services session and wait for a minute or two for the printer to appear. Note that version 6 of the Microsoft Terminal Services Client may be required for this to work.

Although it seems hacky, and untidy, this solution seems to work well, so I hope this will help at least one person out there. Happy printing.

Cacti and Network Weathermap

2008-04-14 00:00:00 +0100

While improving the network at my house (an indeed, the network which supports this very web server), I started to explore the world of network monitoring and reporting. I had heard quite a bit about Cacti before, but never considered installing it. That was mostly due to the stories I had heard about how unholy difficult the damn thing is to get working properly. "Don't even go there" was my mindset. Until now, that is.

Cacti is a complete network graphing solution designed to harness
the power of RRDTool's data storage and graphing functionality.

Brilliant. Network graphing is good, I want to see pretty charts and graphs about how my network is doing. So I gave it a go. Here's some of my ups-and-downs, and the end result.

First, I needed a linux machine to try this on. Cacti itself obviously wasn't enough of a challenge for me, I wanted to get it to work on an operating system with which I had very little experience. I chose Ubuntu Server 7.10 - I've worked with Ubuntu before, and I like the Aptitude package manager which would make this project somewhat easier for me.

So first of all, I installed the O/S. I'm using a Virtual Machine on my main VM host, which had some RAM to spare. I only have the machine 128MB, as I'm not going to be asking too much of it (hopefully). I didn't specify a LAMP install, even though that is exactly what would be required. I wanted to do all the fiddly stuff later on.

Once the O/S was on, I needed to install the required packages, and then Cacti itself. Cacti requires a web server, with PHP and GD (the image library), and a MySQL server. I followed this guide to get them all installed on this new machine, and then extracted and set up Cacti.

Worth noting here, is that when importing the cacti.sql file into the MySQL database, I first created the database called cacti, then modified the SQL file, adding use cacti to the beginning of the file - otherwise an error stating "no database selected" would appear.

Once the database was setup, and Cacti was extracted - I pointed Firefox to the cacti server (I had a creative moment and called the Cacti server cacti). The setup process was web-based from here-on, and Cacti was installed in a matter of seconds.

So, now I added my hosts (after enabling the SNMP service on my Windows Servers, and configuring the community), and created some graphs. Just network traffic graphs at first. After a few polls, I was amazed to see the graphs populating perfectly. After following these instructions I made them look so much better (maybe not sexy, though!), and the result was something like this:

Wonderful. Pretty graphs showing me how much the internet connection is being used. 100k eh? Somehow I think paying for 20Mb isn't worth it!

OK, so now I have lots of nice graphs, I wanted to get a Network Weathermap working - which is like a virtual network diagram, showing the traffic between each node on the map - as it reads the data from the same source as Cacti.

This was much easier than I thought - after adding the nodes and links into the config file, the values took on the colours of my scale as they should - and I had a lovely diagram of my network with automatically updating traffic information! Here's the end result.

And there we have it! Not at all as bad as I was expecting. I do hope this will be of help to anyone wanting to do something similar.

Restoring the Separate _msdcs Zone

2008-02-10 00:00:00 +0000

Okay, so if you're anything like me - and like things to be done properly first time (and also to look neat), then you'll know what I mean when I talk about the separate _msdcs zone in DNS on a Windows AD DNS Server. Of course, you have to be a nerd, like me, to know what I'm talking about here also - but that's assumed seeing as you're reading this blog. I digress...

If you have ever reconfigured said DNS server, and recreated the DNS zones from scratch, you'll know that the neat zone that keeps all the SRV records separate from the oh-so-important A records, disappears - and gets put in a folder under the usual domain root.

Well, I have a solution to the ever so pressing issue. Obviously the only way anyone is going to risk breaking their whole Active Directory network will be if, like me, they are so totally OCD about this kind of thing.

So, if you're interested, I've written a short article on how to restore this behaviour, and published it as always on maxms.net. If you think it might help you out, then here's the link:

Edit: link no longer available

But remember, follow that article at your own risk!

Roadwarrior with IPCop & OpenVPN

2008-01-30 00:00:00 +0000

As an update to the previous post regarding the installation of a new IPCop as my network firewall, I have finally completed the configuration of its VPN service for use as a Roadwarrior. I can now connect to the IPCop machine from my laptop, using the OpenVPN client from anywhere in the world.

I was surprised with the ease of configuration once an addon called "Zerina" was installed. This made the process extremely simple to complete, even offering to package up an OpenVPN configuration file and certificate combination - so all that is needed to connect is one click!

With regards to the IPCop machine itself, it is one of the most stable servers I have ever put into operation. I literally installed the O/S (about 50mb) a couple of weeks ago - and since then there has been not one issue. Not even a restart - it's just been chugging away on that old 400MHz Pentium II. I am in awe of the little thing - which is actually proving to be a damn sight faster than the overpowered and clunky ISA Server that I used to use.

Also, with the terrible OpenVPN logo, and the lack of suitable IPCop art, I hope the visio diagram to the left bears a resemblance to this post that could be appreciated by the reader. I definitely think it makes the post something special, would you not agree?

New IPCop Firewall

2008-01-18 00:00:00 +0000

My latest project, to replace the bulky overpowered ISA firewall on my home network with a lean mean IPCop machine, was declared a great success a few days ago.

I am familiar with IPCop, as I used to use it a long time ago. Since then it has matured somewhat, but the feature set is pretty much the same as I remember. The new machine is a 400MHz PII, with 192mb RAM. It is sitting in the place of a Sempron 3000+ with 1GB RAM. Amazing, it's doing the same job with a fraction of the power. And also, it uses a third of the electricity - 30W in total. Good news given the rise in energy prices!

The main challenge so far, which I still haven't overcome, is how to get RoadWarrior VPN working, using the windows built-in VPN client, with L2TP/IPSec. This used to be trivial with ISA Server, but this isn't quite the case with a linux firewall. I have been looking at other distributions such as monowall and pfSense, niether of which seem to spell out their ability to achieve this clearly. I am playing with a few of these on Virtual Machines, so hopefully I will come accross a way to do this before long - I'm starting to miss my RoadWarrior VPN server. How sad, eh?

Active Directory Practices

2008-01-11 00:00:00 +0000

As a matter of curiosity more than anything, I often wonder whether other people's methods and practices for setting up AD are similar to my own. I will explain as best I can my own procedure, in an attempt to see how it compares to the rest of the IT community.

Firstly, I install DNS on the Domain Controller to-be. I don't do any configuration on the service, just install it. Then, running DCPromo, I allow the wizard to configure the DNS Service for me. This makes sure that the two separate zones will be present - _msdcs.domain.name, and domain.name. This seems much neater to me, and I like to see this result - so I allow the wizard to take care of it.

When the domain is installed, the first thing I usually do is open up the default domain policy and remove all the password complexity options. These are usually just an annoyance - and unless the network has any particular security needs, I disable them all. Maybe leaving the length value at 6 if it's inappropriate to turn it off completely. I like managing GPO's from the Group Policy Management Console (GPMC), so that usually gets installed straight away.

In regard to the structure of the domain, I make an OU with the domain's Netbios name in the root, and under that I create some OU's as follows:

Computers
Distribution Groups (If Exchange will be installed)
Security Groups
Servers
Exchange Servers (for a special shutdown script)
System Users
Users

As for an explanation for that Exchange Servers OU, I make a shutdown script to stop all the exchange servers when it shuts down, to make the process a hundred times faster. I am so impressed by this technique, it works flawlessly every time. This OU allows me to assign the shutdown script via GPO to all Exchange Servers in the domain. Note that the DC stays in its own Domain Controllers OU that is created by the system automatically.

I guess at this point I'm feeling like I should do a backup of the DC. DHCP servers need authorizing, and Remote Desktop needs configuring. When that's done, we're basically there. Get the clients joined to the domain and we're off!

I have no idea whether my procedure is similar to anyone elses, or in any way superior (or indeed inferior) to others. Give me some opinions anyway, it will be interesting to hear from the rest of the community.

VMware Server 2.0 Beta

2007-11-16 00:00:00 +0000

Seeing as Im getting into virtualization more and more recently, I decided to give the new beta of VMware Server 2.0 a go. I have a virtual machine for testing purposes on my desktop machine (a Dell Dimension 9200, E6600, 2GB RAM) so this should prove a perfect test bed for the software.

I have a particular personal interest in VMware. The server that so many of my posts have been about as of late runs VMware - it and a couple of virtual machines provide me with my email, directory, file servers and websites. Because of this, Im quite excited about the next release of VMware and hopefully the transition wont be too difficult.

Starting with the interface, which is the first noticeable difference one the installation is completed there is nothing that has been kept from version 1 a complete overhaul. Its now based totally in the browser on port 8222 (8333 for https) using its own installation of Tomcat to be precise. This brought back bad memories of managing Microsoft Virtual Server 2005 on a clients system but once I started using the new interface, I began to like it more and more. Its definitely an improvement over Microsofts attempt, and with the browser plug-in for Virtual Machine Remote Control (VMRC), it provides all the functionality of the previous console and more; even in firefox!

I can definitely say that the performance of VMware itself has improved in the new version, although I am running it on a Vista installation. The only real reason for this noticeable change is that the previous release would hang for a number of seconds when a virtual machine was started or stopped in Vista. It feels as though this is an issue that has been addressed in the beta, and so hopefully performance overall will have had the same attention.

Just a few shots of the new UI, first we have the login screen for the web-based management interface. My desktop is joined to the domain at my house, and the only account I could use to login was the builtin domain administrator account. Strange.

Next, the summary screen which shows some info about the host machine, which will be nice on a production server as it details the RAM usage. This will be something I look forward to using on my main server Zeus:

Finally, a shot of the only VM I have on this machine. In this case the VM is stopped, and it shows the hardware configured for the machine, and the 1x2.338GHz CPU looks promising - I could assign another!

That's all I have on this for now, but if I find any more interesting stuff I'll be sure to post an update. If you're interested you can grab a copy from here: http://www.vmware.com/beta/server/.

Server Cooling and Network Rewire

2007-11-08 00:00:00 +0000

OK, so seeing as the last post is about worrying errors reported by the RAID controller on my main server, Zeus, I've decided to do something about the heat that caused the errors in the first place.

The drive caddy in Zeus holds 4 drives, but they are far too close together, so I've spread them out somewhat, and added another fan, although it took some pursuation to get it in (by pursuation I mean using tinsnips to cut a hole in the front of the server). This is proving to work really well, now I just need to figure out how to get another exhaust fan in there somewhere.

Also, I have recabled the switch with colour-coded Cat5. We have red cables for the internet VLAN, yellow for the perimeter VLAN, and blue for the internal network. This has also been a good time to start wiring for gigabit, so I've ran the first cable to my desktop from the switch. All I need now is a few gigabic NICs and I'll be all set.

Update: I've got some pictures up now. We have zeus with its new ghetto fan mod at the front, blowing nice cool air over a terabyte worth of data in the first two. Then the new switch and the lovely neat wiring job in the last one. On a side note, I'm really happy with the switch, it's proving worth the cost after all. Looks pretty good as well sitting in the cabinet :). Now, if only I could get round to painting the inside of that server cupboard. It's shameful!

RAID Saves the Day

2007-11-03 00:00:00 +0000

Error occurred on Primary Master device on adapter 0. Primary Master - CDB 2a 00 01 58 aa 5d 00 00 01 00.

This is what I was greeted with in my inbox, as I started the day yesterday. It looks like at some time around 4am, the primary drive on my main server, Zeus, had been having a few problems to say the least. The drive gave about a hundred of the above errors, and then dropped off the RAID1 array. I opened up the case to see what was going on, and it would seem that heat was the killer, if anything. I shutdown the server, moved the drives around a bit to allow for better airflow, stuck an extra fan in, and booted it back up. The drive came back to life, and is rebuilding as I type this.

I am now, however, looking at a new SATA controller and 2x80GB Western Digital drives. Hopefully these will run cooler, faster, and allow for better airflow than their IDE counterparts.

Core Switch Installed

2007-10-12 00:00:00 +0100

The new core switch that was due to arrive has finally come, with no real delays which is surprising, since it was sent using Royal Mail. The new Netgear has really good management features, and the web interface is pretty detailed. Most of all though, I'm enjoying the ability to have a few VLANs, and a reasonable amount of traffic without the thing crashing on me every five minutes (thanks Planet).

I'm going to be doing some bandwidth testing soon as well, just to see what kind of speeds I'm getting, but I'm expecting it to do fine.

New Core Switch

2007-10-06 00:00:00 +0100

For a while now, I've been having problems with the network's core switch - it's a Planet FGSW-2620. Yes, I know, it's a Planet. My mistake in the first place, it wasn't even worth the cheap price I paid for it. It crashes almost every day, when the traffic level gets above approximately zero. To put it lightly, it's a nightmare. Even the management is terrible - it can only be managed by it's serial interface, has no web server, not even an IP - so there's no SNMP, no Telnet, nothing.

Anyway, lets get onto the new switch, that will be arriving any day now. It's a Netgear FSM726, and it looks a damn sight better than this Planet I've been running for a few months. I know it's no ProCurve, but it's got all the functionality of a high level enterprise switch - all types of SNMP and RMON, a web interface, Telnet, VLANS, the works. As long as I can monitor it somewhat, and it doesn't crash all the time, I'll be happy. Plus it will make a nice addition to the new server closet I've just moved into.

Network Redesign

2007-09-25 00:00:00 +0100

Okay, so the network that I have been managing for some time now has just undergone a pretty big redesign. Its actually a home network, but it spans 2 sites my house and my friends house. They are joined by a site-to-site VPN connection, which gives us a load of benefits like easily sharing photos, programs, and an AD/Exchange forest.

Up until recently, the network was running with just one physical server at each site, we shall call them Site A and Site B, each with VMware Server installed. Both servers were configured almost identically, with the host machine running AD and Exchange, and a VM running ISA Server 2006 for the firewall/VPN. Another VM was used for hosting some websites in the Perimeter network.

The redesign saw one new server in at Site A and two new servers at Site B although the main server at Site A has been upgraded significantly. The new servers were installed to take the firewall away from the Virtual Machine to a physical one as this is much more secure. Also, the second new server at Site B hosts Exchange, while this is now on a VM at Site A.

This network doesnt support many clients or users, but it used mostly for educational purposes. For that it is perfect. We have a multi-tree forest AD configuration, with one domain for each site (or each house!), and one Exchange organisation spans the entire forest, with one Exchange server at each site. This also helps if one server/network is down, as the other will pick up the email for both sites so we have a failsafe if one network is having problems.

I have published a public version of the network diagram, with external IP addresses/names removed, just in case anyone might find it interesting. Just click the thumbnail for a fullsize version.

As you may have noticed, I've used the names of gods from Greek and Roman mythology for the servers. The web servers are the oldest ones there so they haven't been renamed yet. Maybe an exiting project for the future!

Both networks now have a 20mb/784kb internet connection (up/down), so the VPN link is essentialy 784kb/sec both ways. That's pretty good for things like AD replication, but not brilliant for sharing files and photos.

The active directory is the aspect of the network I am most proud of. Since the rebuild it has been working flawlessly, although I am forever looking at ways to expand the directory. The DC at each site hosts a DNS zone for both domains, which provides redundancy for DNS if one DC is down, and both servers hold a copy of the Global Catalog. This allows for fast directory searches from both sites, and gives each Exchange server a GC to look to.

The forest is split logically, as well as physically, into sites. This allowed me to easily alter the replication schedule for the Domain Controllers, although I decided to leave this at hourly intervals, as I saw no reason to alter this value.

Hopefully the AD forest and network infrastructure will provide a solid base to expand on, and I will post about any major additions to the network. At present the clients consist of XP and Vista machines, but we are soon to aquire a new desktop, which will be running Vista, that will make a nice addition to AD.

Configuring Share Permissions

2007-09-19 00:00:00 +0100

When I was "starting out" in the IT field, I always used to setup shared folders in a certain way - I would set the Share Permissions to Everyone - Full Control, then use the NTFS Permissions to control access to the share - which always seemed like the most simple and secure way to do things.

However, when talking to the Network Administrator at my college, I was informed that setting the "Full Control" item was extremely bad from a security standpoint, as it allowed any user in the specified group (in this case, Everyone) - to change options in regard to the share configuration - like the permissions themselves.

So from this point on, I changed my habits to setting the Share Permissions to Authenticated Users - Change, and then using the NTFS permissions, as before, to control access to the data. Today, however, I decided to do my homework.

A quote from this article states the following:

"The recommended permissions have been tested, and work correctly; but there are alternative approaches. For example, some experienced administrators prefer always to set share permissions to Full Control for Everyone, and to rely entirely on NTFS permissions to restrict access."

To me, it sounds like Microsoft have no reservations about using this method, and certainly don't mention any security risks at all. So, hearing anyone else's opinion would be very useful, but seeing as it works well - for now I will carry on using the Authenticated Users - Change permission object.

phpBB3 WSOD (White Screen of Death)

2007-09-12 00:00:00 +0100

I don't know whether this is a popular or well-heard-of problem or not, but I have come across this twice now. The phpBB forums over at maxms.net have suffered the second "White Screen of Death" incident since I installed them. Firstly, I disabled Gzip compression to fix the issue. This time though, it wasn't so easy.

I ended up being forced to reinstall the same version of phpBB to another directory on the server, into a new database. Then I renamed the new DB to something different, and changed the old one to the new one's name. Phew! So in the end, I had a new install of phpBB, but with the same data as before - and it worked perfectly.

So if anyone else comes across this issue, firstly try disabling HTTP compression on any device in front of the web-server (which for me was MS ISA Server 2006). If that doesn't work, it looks like it has to be a reinstall - the longest process will be renaming the DB's, however, so be careful.

Anyhow, the forums are now back up and running - at forums.maxms.net. Hopefully this seemingly endless list of problems to fix is over now.

Out with Exchange, in with WSUS!

2007-09-09 00:00:00 +0100

Up until yesterday, my Exchange server was my DC - they were one and the same. As anyone will tell you, this isn't a particularly desirable configuration. For one thing, if you want to demote the DC, you have to uninstall Exchange first, and that means lots of migration and replication and...well it's not very nice!

So, I finally took the big step, and migrated all the Exchange data off the DC, and uninstalled it. So now I have a seperate Exchange server, which means tons more RAM free on the DC.

This also allowed me to install something I've been looking at for quite some time now - WSUS 3. I now realise how useful this piece of software can be. I am only managing about 5 computers, plus a few servers, but this makes keeping the machines up-to-date so much easier. Plus, you get lovely graphs like this:

Also worth mentioning, is that the Exchange server I've been telling you about actually runs as a virtual machine under VMware on the DC. It has 2GB of RAM, and seems to be coping fine, but with 1GB assigned to the Exchange VM, and 384mb assigned to another VM I have running on there (a web server), task manager seems to be having problems getting the memory details correct:

So we have 1GB + 384mb + whatever else is running on there (WSUS, DC, DHCP etc), and we end up with 1.0GB (or there abouts). Something's not right. Alas, the server seems to be handling the load fine, and with a gig of RAM apparently free, I have space to expand in the future. Brilliant.

Active Directory Replication Problems

2007-09-08 00:00:00 +0100

One word, or at least one acronym: GUID.

Background: I manage a multi-tree forest, with two trees, one in each of two sites. They are connected by a slow site-to-site VPN link, over which all AD replication takes place.

The domain controller at the forest-root-domain needed rebuilding, as the operating system was installed on a flaky single disk, and was due to be moved to a RAID1 array. So I thought it best to promote another DC, transfer all FSMO roles, rebuild the first, and transfer the roles back. This process went swimmingly, and the first DC was back online in no time.

However, when it came to the second site, it seemed that no replication whatsoever was taking place. After delving into AD with tools such as adsiedit and replmon, I discovered that the second DC had not "heard" about the rebuild of the first. This meant that the GUID had not been updated to hold the value of the newly installed server. The fact that I had used the same name as before didn't help the situation at all.

In the end, it was clear that I would have to either restore the original DC from a System State backup, or rebuild the second domain from scratch. I chose the latter, as it was a small domain, and wouldn't take very long. Now the process is complete, and we have a fully functioning forest again (after lots of metadata cleanup and /forceremoval's!).

I won't forget this one in a hurry - allow time for big changes to replicate before making more big changes!

Back to School!

2007-09-05 00:00:00 +0100

Well the first day of my last year at "school" is over - which is actually 6th form. Now I'll have much less time to spend on the IT side of things - which is disappointing. Just when lots of work comes my way!

It's not all bad though. The school seems to have gone all-out with it's IT spending this year, which is obviously a huge bonus to the new students (and us old ones as well!). We have new study areas, and new "flagship" IT labs, with brand-spanking Dells. Brilliant! I just hope the ancient AD infrastructure can support the hundred odd new machines. At least they have a real network manager now, so I won't be drafted in to sort things out when someone decides to seize the Schema Master role from a running server :(.