blog

A complaint to the EQC

Tue, 06 Sep 2011 18:51:00 +1200

Sorry, this isn't tech related. But I need it out there. I'm REALLY pissed off with the EQC at the moment. This is an organisation that up to last year we paid our levvies to, in the understanding they would be there for us when the worst happend. And then the worst happened.

At first they were fantastic. I had an inspection team out promptly surveying the damage, and critical repairs to my water supply were authorised. I received a settlement notice that looked fair and reasonable, and all was looking great.

But then the contactors who did the emergency repairs on our water complained they hadn't been paid. After a couple of months of being fobbed off by the EQC call centre they eventually got paid.

But as for the rest of my settlement, well, here's the content of the complaint I have just submitted.

Dear Sir/Madam

I was contacted in early June by one of your team saying the cheque they sent out after my claim in November had not been cashed and was now "stale". It turned out it had been sent to the wrong address (100a instead of 109a).

I advised that I am now residing at [address removed]. He said he would get another cheque sent out, and it would take 6-8 weeks to process the cancellation and get the new cheque sent.

However, I rang on 11 July and the person I spoke to believed the cheque had not yet been sent out, but they could not provide details as I had rung after 4pm, and everyone that had a clue had gone home.

I rang back on 12 July before 4pm and spoke to [name removed] in Brisbane. She advised that Accounts are waiting on confirmation from the banks that the old cheque is cancelled.

She also advised me that the bank account the payment goes in to has to be in the name of the Trust, so I sent in a statement from that account.

On August 30 I still had not received payment, so rang up again. This time I spoke to [name removed], and she advised that the cheque had not been cancelled and would have to be processed again from the start. When I queried as to how it could take so long she could not give me an answer. When I queried how much longer this would take, again she could not give me an answer. I asked if the payment could just be made straight away as the cheque was stale and unusable, and told that it couldn't, because that "wasn't correct procedure".

It has been a year now since the damage was done to my home. The payout value is less than $6k - not very much compared to the "millions of dollars per day" your organisation proudly advertises as paying out. My bank has advised me that a cheque is confirmed cancelled instantly upon request (it is, after all, 2011!) - it does not take '6-8 weeks'. I am totally at a loss to understand why this can take so long.

I have 6 main reasons for my complaint:

1) Your systems don't appear to do any error checking and manage to send letters to the wrong address (how many other letters from EQC am I missing?)

2) The person who rang me in June failed to get the old cheque cancelled and new one issued, as he promised

3) [name removed], who I spoke to in July, failed to get the old cheque cancelled and new one issued, as she promised

4) The excuse given for the delays in processing the cancellation of the cheque (that EQC was awaiting confirmation from the bank for 6-8 weeks) is not justified, as cancellations can be processed instantly. At most I'd expect a few business days.

5) Staff at the call center do not appear to have proper records of previous conversations I have had with them.

6) The cheque is long stale and cannot be cashed. Therefore there should be nothing stopping an electronic payment being made or a replacement cheque sent out.

At the end of the day, it is the EQC's fault the cheque was sent to the wrong address. I believe EQC staff should have been more pro-active in rectifying this situation. It should have been resolved with the first phone call in June.

I await your comment and proposed resolution.

Regards, Al Twohill

File permissions on Samba shares

Tue, 09 Nov 2010 15:08:00 +1300

Here's the situation. We have a server with a bunch of shares on it, that a bunch of different people access. Each share allows a different group access, and users can be in more than one group.

This works fine as-is when you are just editing a file, but if you are creating a new file it will get created with access set to the user and the user's default group. Often this means that it is the completely wrong group for the share! So we need to force our file server to a) set the correct group on file creation, and b) allow the group read-write access to new files by default.

To accomplish a), we need to set the gid sticky-bit. When the gid sticky-bit is set on a directory all new files will be created with the group ownership of the parent directory. We can set this on all directories fairly easily with something like this:

find /home/companydata/ -type d -exec chmod g+s {} \;

For b) we need to edit our smb.conf and add the following into each group share:

force create mode = 0660
force directory mode = 0770

Hopefully this is fairly straight forward. Don't forget to restart Samba!

Domain Aliases in Zimbra

Fri, 05 Nov 2010 09:29:00 +1300

I always forget how to set this up so I'll post here for my memory aid. If you need Zimbra to deliver all mail addressed to @foobar.com to the relevant account @foo.com -

As user zimbra:

zmprov cd foobar.com zimbraDomainType alias
zmprov md foobar.com zimbraMailCatchAllAddress @foobar.com zimbraMailCatchAllForwardingAddress @foo.com

This will forward user@foobar.com -> user@foo.com

Thanks to this post in the Zimbra forum.

Managing Silverstripe Upgrades In Your SVN Projects

Thu, 29 Jul 2010 11:30:00 +1200

Let's face it: FTP is for the web developers of the '90's; real developers use SVN (or some other source control software).

If you haven't heard of SVN and you do any type of file editing at all, you're probably spending far too much time keeping track of changes to files, and having a horrible time of it trying to keep everyone up to date with what you've done lately. I suggest you read the svn book.

If you do use SVN to help manage your web development, sooner or later you'll run into the same stumbling block I've just come across:

I want to update the site I'm working on to Silverstripe version X.X, but...

Don't worry, mon ami. I know you might want to:

Maintain site-specific hacks automagically without having to repatch every version
Track the changes of Silverstripe files, as well as your own
Pick and choose when to update your site
Be able to update folders across many computers in one fell swoop

We have it all covered below. I learnt most of this from a site on a similar topic for Drupal, and modified it to suit Silverstripe and based on my experience. And without further interruption, I bring you the main article:

Setting up our SVN environment

First up, I'm going to assume you have some basic knowledge of SVN, and you have it all set up in your environment. If you don't, read the svn book I linked earlier. Also, I'm assuming you're using a terminal in a UNIX-like environment. If you're not, you shall have to translate this yourself as required.

We're using a feature of SVN known as Vendor Branches. Well, I say feature but it's really just a clever use of the generic features of SVN. Anyways, here's how it's done.

Step One: Create the folders where we want to go

Create a folder structure similar to the following (I set up a temporary working folder to put all this in):

./vendors/
    silverstripe/
        installer/
        modules/
        upgrades/

This allows us to store the full installer (to make it super easy to set up new sites), any external modules we use (so we can upgrade them as well), and the files needed to do upgrades of existing sites.

Step Two: Start populating the folders with our current version

We'll set up both the installer and the upgrades folder with the version we're running. The coolest way to do it is with SVN (of course):

svn export http://svn.silverstripe.com/open/phpinstaller/tags/2.4.0 ./vendors/silverstripe/installer/v2.4.0
mkdir ./vendors/silverstripe/upgrades/v2.4.0
cp ./vendors/silverstripe/installer/v2.4.0/{cms,googlesitemaps,sapphire} ./vendors/silverstripe/upgrades/v2.4.0/

Change the 2.4.0 for whatever version you're currently running.

Step Three: Get that sucker up there!

Now, we're going to upload that folder into our SVN repository. That's going to create a new structure in the root (alongside tags, branches, and trunk) called vendors. We're also going to keep it nice and organised by having a folder for each version, as well as a folder that links to the current version. We can do that in a few easy commands:

svn import vendors http://svn.hol.net.nz/projects/vendors/ -m "Importing Silverstripe vendor release"
svn copy http://svn.hol.net.nz/projects/vendors/silverstripe/installer/v2.4.0 http://svn.hol.net.nz/projects/vendors/silverstripe/installer current -m "Tagging the 2.4.0 release of Silverstripe as current"
svn copy http://svn.hol.net.nz/projects/vendors/silverstripe/upgrades/v2.4.0 http://svn.hol.net.nz/projects/vendors/silverstripe/upgrades current -m "Tagging the 2.4.0 upgrade of Silverstripe as current"

Again, change the version and the url to match your setup. Don't try and use the exact link above or your computer might explode - you have been warned!

Step Four: Cleanup

Get rid of your working directory for now - we don't need the full structure anymore

rm -rf ./vendors

Working with our vendor branch

Now that we've set up this vendor branch, setting up a new website based on the current version of Silverstripe we have here is incredibly easy:

svn copy http://svn.hol.net.nz/projects/vendors/silverstripe/installer/v2.4.0 http://svn.hol.net.nz/projects/clientsite.co.nz/staging -m "Setting up new Silverstripe website"
svn co http://svn.hol.net.nz/projects/clientsite.co.nz/staging /my/local/server/files/clientsite.co.nz

In the example I gave I store all my client's websites under their domain name, and in either ./staging or ./live, based on the status of the code.

Upgrading Silverstripe

So, a new release of Silverstripe has come out, with a bunch of bug fixes and new features that we really want. However, in some of our projects we've changed some of the core files to twist the site to our unique application (I know it's not best practice, but it happens). Lets get our site up to date with the least hassle possible.

Step One: Download the core module files

We don't need the full installer in this case as we are upgrading our sites. So we'll download the three modules that are distributed separately.

svn export http://svn.silverstripe.com/open/modules/cms/tags/2.4.1/ ss-2.4.1/cms
svn export http://svn.silverstripe.com/open/modules/googlesitemaps/tags/0.1.5/ ss-2.4.1/googlesitemaps
svn export http://svn.silverstripe.com/open/modules/sapphire/tags/2.4.1/ ss-2.4.1/sapphire

Note you can see the individual versions of the modules by checking the svn:externals properties for the installer (easiest way is to browse to http://open.silverstripe.org/browser/phpinstaller/tags/2.4.1)

Step Two: Update the vendor branch

To do this next step we need a file called "svn_load_dirs.pl" which generally comes with SVN. If you can’t find it search for "svn_load_dirs.pl.in" and rename it "svn_load_dirs.pl". If you can’t find that either you can download svn_load_dirs.pl.in and rename it "svn_load_dirs.pl" You may need to edit the file to point it to the location where "svn" lives. To find out where svn lives type (usually '/usr/bin/svn'):

which svn

Then change the line that reads:

my $svn = '@SVN_BINDIR@/svn';

my $svn = '/usr/bin/svn';

Place that script somewhere handy, then run

/path/to/svn_load_dirs.pl http://svn.hol.net.nz/projects/vendors/silverstripe/upgrades/ current ss-2.4.1 -t v2.4.1 _no_auto_exe

This script will look at a (source) set of files in an svn repository, compare it to a (destination) set of files, and then automate whatever adds, changes, and deletes are necessary to make the source like the destination. It will check the changes in for you, and it will tag the changeset for you. We add the -no_auto_exe option to stop it from incorrectly thinking a lot of the files have svn:executable set. You will get a list of files deleted and added and are given the opertunity to specify them as simply being moved - this is optional. So now we will have the following folders in our repository:

http://svn.hol.net.nz/projects/vendors/silverstripe/upgrades/current
http://svn.hol.net.nz/projects/vendors/silverstripe/upgrades/v2.4.1
http://svn.hol.net.nz/projects/vendors/silverstripe/upgrades/v2.4.0

Note that current and v2.4.1 are essentially the same.

You probably also want to update the installer files. I like to keep this fresh and pristine, with no trace of previous versions, so I'd usually run through the following commands:

svn rm http://svn.hol.net.nz/projects/vendors/silverstripe/installer/current -m "Making way for new installer version"
svn export http://svn.silverstripe.com/open/phpinstaller/tags/2.4.1 ss-installer-2.4.1
svn import ss-installer-2.4.1/ http://svn.hol.net.nz/projects/vendors/silverstripe/installer/v2.4.1 -m "Uploading SS 2.4.1"
svn copy http://svn.hol.net.nz/projects/vendors/silverstripe/installer/v2.4.1 http://svn.hol.net.nz/projects/vendors/silverstripe/installer/current -m "Tagging SS 2.4.1 as current"

Step Thee: Update your sites

You can do this on each of your sites as and when you are ready. You'll need to check out your site's working directory if you haven't already done so.

svn co http://svn.hol.net.nz/projects/clientsite.co.nz/staging clientsite.co.nz

Now we will merge the differences between v2.4.0 and v2.4.1 into our client's site.

cd clientsite.co.nz
svn merge http://svn.oxynet.co.nz/projects/vendors/silverstripe/upgrades/v2.4.0 http://svn.oxynet.co.nz/projects/vendors/silverstripe/upgrades/current .

Don't forget the '.' at the end - it's kinda important.

Now, if you have modified any of the core files you may have to resolve conflicts before you can commit the client's code back into SVN. This is done as normal (again, check the svn book if you need help).

Make sure you test the site, do the /dev/build and everything else specified in the upgrade notes.

Finally, when we're happy we commit our changes back to SVN.

svn commit -m "Upgraded clientsite.co.nz from SS 2.4.0 to SS 2.4.1"

Reset all passwords in an Active Directory OU

Thu, 15 Jul 2010 16:00:00 +1200

Sometimes we need to reset everyone's password on the network - for instance if we are doing a server/network migration and we need to access everyones accounts. With this little cmd snippet you can easily tweak it to reset everyone in a particular OU's password:

dsquery user -startnode "ou=Inner Org Unit,ou=Outer Org Unit,dc=DomainName,dc=local" | dsmod user -pwd NewPassword -mustchpwd yes

Reset Administrator Password on SBS 2008

Sat, 24 Apr 2010 10:48:00 +1200

So you've managed to loose your admin password for your SBS Server 2008 eh? Well in my case, a faulty keyboard caused an incorrect password to be set. Whatever the reason you need to reset the password, I'll assume you're doing this for Good, and not Evil.

A quick google round came up with the usual password reset programs, but they only work on local passwords, not ones stored in Active Directory, so they were no good. Eventually I stumbled across the method. Credit goes here, but I like to keep useful tips on this site (more for myself than anything).

Things you'll need:

Physical access to the server
Your 2008 install DVD

Step One: Launch recovery console

Put your install DVD in the drive and reboot the server. Make sure you press the key to boot off the DVD. Once it loads, choose your language, then click Repair your computer. Click Command Prompt

Step Two: Do a bit of file jiggery-pokery

Run the following commands in the command prompt:

C:
cd windows\system32
move Utilman.exe Utilman.bak
copy Cmd.exe Utilman.exe

Basically we're moving Utilman out of the way temporarily and replacing it with cmd.exe (command prompt). Close the command prompt and click Reboot.

Step Three: Do the password reset

Start up your server as normal, and after you press CTRL-ALT-DELETE and have the login screen up, press Windows-U and the command prompt will come up. Type the following:

net user [username] [password]

Where [username] is the username you want to reset the password for, and [password] is the new password. Close the command prompt and log in with the new password!

Step Four: Close that back-door

We want to move the files back to where they were, otherwise anyone could go Windows-U to get an administrator's command prompt. Unfortunately Utilman.exe is a protected file so you can't move it back while in Windows. Reboot back into the recovery console as in Step One, and run the following commands:

C:
cd windows\system32
del Utilman.exe
move Utilman.bak Utilman.exe

All done! Reboot and pat yourself on your back.

HOL changes!

Mon, 09 Mar 2009 20:05:22 +1300

HOL has undergone a migration to the fantastic New Zealand-based open-source cms that is SilverStripe! It was a bit of an effort, but it could have been a lot harder.

I'm looking forward to adding some new bling that the Sapphire framework gives me.

Migrating Magic

A few people have asked how I managed to get all my blog pages from my ModX database into Silverstripe. After looking through the database I decided that it wasn't practical to import data directly into the database, as the ORM makes it too complex. Instead, I looked for a programmatical way to do this.

I'm not yet sure how to bootstrap the Sapphire API for temporary scripts so I don't know how to harness the Sapphire framework outside of building modules, so instead I looked at how blogs were posted in the first place. I firebugged the blog post form and found that it sent data to /blog/BlogEntryForm. It was easy enough to find the variables that I needed, then turn them into a URL that I could loop through the blog posts I'd exported into a csv file with phpMyAdmin and call.

The end URL format was /blog/BlogEntryForm?ID=&Author=Al&Title=My+Blog+Post&Content=This+is+my+blog+content

Hint: it's a lot easier if you use urlencode() to put all the '+'s in etc

HOL gets CAPTCHA'd

Thu, 12 Feb 2009 21:59:00 +1300

My god, I checked my moderation queue for blog comments and I had nearly 1000 spam comments waiting for me! Nothing that a good DELETE FROM in the database can't fix, but there might have been some real comments hidden in there! (I did find a few, and exluded them from the deletion. If I missed you I'm sorry and I don't hate you).

So now I've added a CAPTCHA code to the comment submission process. I'm sorry if that causes resentment, but you'll just have to tough it out.

Also, I'm very sorry about being so slow to reply to comments. For some reason I haven't been getting emails to let me know about new comments. Hopefully I've fixed that now.

Resolve .local domains with DNS

Thu, 05 Feb 2009 22:32:00 +1300

Many Linux distros come pre-configured with zeroconf, which lets machines resolve each other in a peer-to-peer manner. This is fine, except they chose the .local domain, which will conflict with any .local domain you may have set up in DNS.

Zeroconf: designed for no setup

...unless you're already set up, in which case it is a real PITA.

It can't have been really that hard to choose something less likely to cause issues (.zeroconf maybe?), but we've got to live with it.

A bit of background

Any security expert knows that you're a lot better off having a private IP range for your local computers. Not only does this save on IPv4 addresses by 'hiding' them behind a router, it also makes it much more difficult for attackers to access remotely. And if you're going to have local IPs it makes sense to have a private DNS server. And if you have a private DNS server you don't want the FQDN to conflict with anything on the net (ie it shouldn't end with .com, .co.nz etc). So you choose something friendly and close to home. .local even. Even Microsoft recommend your Active Directory domain ends with .local.

So, it is fairly reasonable to assume that when people create a local DNS server, they are going to give it a .local domain. And if you're already set up you don't want to have to change it. So here's how you don't have to.

Let me see the .local, dammit!

It's actually pretty simple. As root, edit

/etc/nsswitch.conf

Find the line that says

hosts:      files mdns4_minimal [NOTFOUND=return] dns

and change it to :

hosts:      files dns mdns4_minimal [NOTFOUND=return]

Done!

Installing and configuring Kubuntu Intrepid Ibex (8.10)

Fri, 07 Mar 2008 14:57:00 +1300

So, the latest and greatest version of the greatest Distro ever is out (haha flamebait :P), and I thought it would be good to document how my install went, so others might be able to get their system up and running nicely. But first:

The hardware:

The machine I'm installing this on is also my gaming machine, so it is dual booting Vista Ultimate x64. It also currently has Kubuntu 8.04, though I didn't use it much because it was too ugly and broken for my liking.

Specs are as follows:

CPU: Intel Dual Core E8400 @ 3.00 GHz
Motherboard: Asus P5E (Intel X38 Northbridge/ ICH9R Southbridge)
Memory: 2 GB DDR2-1200
Video: Gigabyte ATi Radeon HD 3870 + Sapphire ATi Radeon HD 3870
Storage: 2x Seagate 320GB SATA2 HDD w/ 16mb cache in striped RAID
Sound: Creative X-FI Extreme Music

It would be pretty straight-forward, apart from the X-Fi sound card, and the fakeraid requirement for dualbooting. I am wanting to re-format my linux partitions and keep Vista happy.

So, without further ado,

The install

I download the x86 live installer .iso from the nearest mirror. I'm still very suspicious of x64 on linux at this stage, and will continue to be until Java, Flash, and Skype (among others) are sorted out.

So, freshly burnt CD in drive, I give 8.04 one last wave good by and hit the reset button. The CD boot prompt is pretty much the same as the last version, and still miles behind Redhat-based loaders (such as OpenSUSE), but never mind. So I boot up and get the age old progress bar doing its thing. That disappears and I expect any minute to be be landed into the live CD.

Uh oh. My monitor light is blinking in a way I don't like. I try the old CTRL-ALT-BACKSPACE trick to no avail. Hmmm. I reboot, and choose 'Safe graphics' under the F4 menu. This time, we're off. I wish it would do that automatically though - how hard can it be to detect that a display mode isn't working and revert back to safe? Or, dare I say - the windows way of starting off in safe graphics mode then prompting to change once everything is loaded is a very good way to handle it.

Nevermind, I'm now in the live environment, and I have to say - KDE4.1 is a hell of a lot nicer and complete looking than KDE4. I'm quite impressed. There's no real Kubuntu styling though (save for a forum-signature-like Kubuntu image in the Kicker menu). After using distros such as OpenSUSE 11, which completely immerses you in a uniform look and feel, Kubuntu just seems bland. I only wish I had a decent design sense so I could help out in this area! But, at the end of the day you can't complain about the default KDE4.1 look.

So I fire up the Ubiquity installer and set off. All is well until I get to the hard-drives bit. It seems Ubiquity still doesn't support fakeraid. I was dreading a return to my Kubuntu With Fakeraid page, but then I found that the alternate cd fully supports fakeraid! Why isn't this noted somewhere on the download page?

Never mind, it only takes a few more minutes to download the alternate .iso. I also note that the New Zealand mirrors now show up as an option to download from, but they don't work... maybe this is just a temporary thing. No worries - I get it from across the ditch just fine.

The alternate cd install is fine for people who don't mind ugly terminal-based graphics, like me. Sure enough, when it comes to partitioning the installer prompts me that it found some fakeraids and would I like to enable them. Yes! This is waay easier than the last install - thanks guys!!

First boot, first trouble

OK, so the installer is all done, and I restart the computer. Hmmm... even though the grub installer recognised the Vista OS it didn't add it into the boot menu. Never mind - I can deal with that, but it might trip up new users (I'll detail how I fixed this in a wee bit).

The machine boots up the Kubuntu loading screen, then uh-oh - that blinking monitor light is back! However, we can fix that now we have a loaded system. Hit CTRL-ALT-F1 and you get the console login screen. Do your login, then install the graphics drivers. I have a couple of ATi cards, so I run

al@grif:~$ apt-cache search fglrx

This gives me a list of the packages that might help with my card.

fglrx-modaliases - Identifiers supported by the ATI graphics driver
jockey-common - user interface and desktop integration for driver management
jockey-gtk - GNOME user interface and desktop integration for driver management
jockey-kde - KDE user interface and desktop integration for driver management
xserver-xorg-video-radeon - X.Org X server -- ATI Radeon display driver
fglrx-amdcccle - Catalyst Control Center for the ATI graphics accelerators
fglrx-kernel-source - Kernel module source for the ATI graphics accelerators
xorg-driver-fglrx - Video driver for the ATI graphics accelerators
xorg-driver-fglrx-dev - Video driver for the ATI graphics accelerators (devel files)

Sweet! that looks promising. I install

al@grif:~$ sudo apt-get install xorg-driver-fglrx fglrx-amdcccle

Once that's done it's thing, I run

al@grif:~$ sudo  aticonfig --intitial

And now you can either restart, or just run

al@grif:~$ startx

Before you restart though, lets fix up the boot loader not showing Vista. Open up the menu.lst file in your favourite editor

al@grif:~$ sudo nano -w /boot/grub/menu.lst

Put the following text either before the bit that says "### BEGIN AUTOMAGIC KERNELS LIST" (if you want Vista at the top), or after "### END DEBIAN AUTOMAGIC KERNELS LIST". DO NOT PUT IT INBETWEEN THIS BLOCK! YOUR COMPUTER WILL EXPLODE!

title           Windows Vista Ultimate
root            (hd0,0)
makeactive
chainloader     +1

Save and exit by hitting CTRL-X . Now when you boot you should see Vista in there as well as Ubuntu.

Second boot - video working, now where is sound?

Now that I can boot into either OS, and have graphics I'm a bit more comfortable with how things are going. Next task is to get sound working. To make things difficult I have a Creative X-Fi, one of the worst supported sound cards in Linux (c'mon Creative dammit!). The only reason I keep it is because it is awesome when gaming.

I do have a sound card that came with my motherboard that is reasonable (and supported out of the box), but I don't want to have to switch my speakers back and forth all the time.

The solution is to use OSS. OSS is getting more and more support these days thanks to them re-releasing their code as opensource (and the fact that it is sooo much better designed and writen than ALSA). In saying that, I couldn't find an OSS package in the repos, so I had to compile it. It's not too hard though, and https://help.ubuntu.com/community/OpenSound details the steps to follow in a very clear way. I had to reboot for sound to work, as there still seems to be a bit of rough edges between phonon and OSS. The sound control panel applet doesn't really work any more either. But I don't care - I have sound!

Overall - the review

I'm going to finish off this bit soon - I promise!

The Little Browser That Wouldn't

Mon, 28 Jan 2008 17:42:00 +1300

When Firefox first came out it was such a breath of fresh air compared to Internet Explorer. Now its feeling like that same air has been kept in a box for the last year or so and has gone a little stale.

I use Firefox to do my work, which is creating and managing websites. I spend most of my time with the Firebug extension open, and also use the HTML validator extension extensively as well (notice that all these pages validate - check the links in the footer). For the most part, I love it. But, as the saying goes it only takes a little bit of urine to wreck the wine. Lately I've been irked by some issues that seem to have been around for a while.

Memory

If I open up my Task Manager on my Vista laptop, or the ProcessTable on my KDE System Guard and sort my processes by memory use, guess which app is usually up the top? Its our little wonderchild, who seems to have gotten a little fat of late.

In Vista, with only two tabs open (I usually have many more), Firefox is using about 107mb of my main memory, plus another 144mb of virtual memory. In KDE, it's using 133mb of main memor, and 235mb of virtual memory!! Contrast this with IE7 which with two tabs open is using 43mb of main memory and 108mb of virtual memory, and IE6 which is running on an XP virtual machine ticking along at only 32mb memory/23mb virtual memory.

Stability

It never used to be this way; Firefox has always been pretty reliable. Except lately, Gmail crashes on every second access attempt, as well as other pages crashing it. Now I know Google like to develop funky javascript code, but browsers should not let client side scripts do anything bad enough to crash!

My other stability irk is that every now and again, usually while running the Linux version of Firefox, new pages will appear to take the time to load, but Firefox won't update the screen to show that new page. You click on a link and the status bar shows the progress, then its all done - except you still see the old page. Closing down all the browser windows, waiting until the Firefox thread finishes (which is an annoying while after the last window closes), then reopening Fx and reloading all the tabs you had open. Annoying, huh? Worst part is it isn't an actual crash so no error report gets generated, and it's kinda hard to reproduce the problem, as it seems to happen fairly randomly.

Speed

I've nothing to conclusively test this against, but Firefox just feels slower these days. Of course, this is purely subjective, but then again, what really matters is the overal user experience isn't it? It doesn't really matter to me if any particular browser is any percent faster in loading time, javascript processing, page rendering, etc, until I notice a difference. And to be honest, right now I'm looking for alternatives.

In the end

At the end of the day, I'm not going to leave Fx until I find something as good as (or better than) Firebug. It has to be one of the most useful tools for a web developer, so I'm prepared to stay in the Fx camp and whinge until there is a better alternative or the Fx team sort out their little issues.

X-Fi nally on Kubuntu! Time to throw out the secondary sound card

Mon, 28 Jan 2008 17:42:00 +1300

Finally! I have my X-Fi Extreme Music working in Kubuntu. Now I don't have to have a second soundcard installed just to get me linux sound.

Warning

OSS (used here) only provides beta support for the X-Fi at the moment. All the usual yada-yad applies about installing this - it may blow up your hard drive, computer, house, or city. I take no responsibility.

How to install

I can't be bothered writing it all out this time - just follow the instructions here.

My First ModX Snippet - PrevNextLinks

Thu, 24 Jan 2008 19:06:00 +1300

I wanted something like what modx has as navigation in its help section, but couldn't find a snippet that did it. So I wrote my own.

PrevNextLinks.php.snippet

It's nothing majorly fancy, but its the first time I've actually written a snippet rather than just nicking other peoples and tinkering. Basically what it does is create the Previous and Next post links that you can probably see below the page header. I've tried to make it as adaptable as possible so it is useful to others, and not just me.

You can download it from here. Once you've downloaded it, create a new snippet called PrevNextLinks and copy and paste the text into it.

Usage

Just place the call

[!PrevNextLinks!]

on your page where you want the links to appear. If you look at the code in the snippet there are several config settings that let you change the sorting order, the link text, the classes that are wrapped around the links etc. You can do this from your snippet call, like

[!PrevNextLinks? &sortBy=`menuindex` &sortDir=`DESC`!]

Or just change the defaults in the snippet.

Let me know if you find this usefull!

Introducing ATi Graphics Cards and Kubuntu

Thu, 24 Jan 2008 12:12:00 +1300

I've had bad experiences with ATi cards and Linux before. Happily it was not the case this time. Here's how I set up my Sapphire ATi Radeon 3870 HD. I also throw in the desktop effects install at the end.

To change, or not to change

First up, you don't actually have to do this at all! The default open source driver works fine for most situations. That said, if you're like me and like bling and games you will really really want to use the 3D accelerator on that card of yours.

Now that we've decided we want to go to the dark side of proprietary drivers, the first step is to download them.

http://ati.amd.com/support/drivers/linux/linux-radeon.html

Put it somewhere the installer can create temporary files (your home directory is a good place). Now open up Konqueror and make it executable (change the file name to match your one).

sudo +x ati-driver-installer-7-11-x86.x86_64.run

And then execute it!

sudo ./ati-driver-installer-7-11-x86.x86_64.run

After it verifies the file contents a window will pop up and guide you through the installation. You can pretty much just click 'Next' through the entire installation.

Once thats done, we need to do our initial config. This will overwrite the display, screen and monitor sections of our xorg.conf file (not as bad as nVidia though - it rewrites everything, including my keyboard settings!!) so you may wish to back up the file first.

sudo aticonfig --initial -f

And we're done! Restart your X server by logging out then pressing Alt-E, or just press CTRL-ALT-BACKSPACE, then log back in again. You'll see in the K menu the ATI Catalyst Control Center, and you can now play with your desktop effects, 3D screensavers, and games. Enjoy!

Enable compiz

We have to whitelist our driver, as it's not confirmed to work in all cases. This translates to possible crashes. I'll update this after a while to say how stable it is. Open up the compiz file in kate

kdesu kate /usr/bin/compiz

Edit the line below the one that says 'WHITELIST=...' add fglrx to it. Mine looks like this

WHITELIST="fglrx nvidia intel ati radeon i810"

Then save the file.

In Ubuntu, now all you need to do is select Desktop Effects and you're done. Kubuntu however is holding out for KDE4 which brings its own bling. This means in Kubuntu you need to do it a little more manually.

sudo apt-get install compiz compiz-kde compizconfig-settings-manager librsvg2-common

Now press ALT-F2 and type

compiz --replace

The settings config is in K->Settings->Advanced Desktop Effect Settings. The screenshots below are a little more impressive.

My essential XP software installation list

Fri, 18 Jan 2008 12:30:00 +1300

This list is kind of the younger, skinnier, not-any-good-at-sports-or-maths brother of my essential Kubuntu software list. XP doesn't come with bugger all software, so the first thing to do is get some.

This list details a whole lot of great free software which makes windows a bit more bearable.

The list

Firefox

First and formost: Firefox. You can get it from here. This, and Windows update are the only sites I'll go to using IE - it's just too risky. Alternatives to Firefox on Windows are Opera and Safari.

Avg Free

Now we have a safer browser, lets get some anti-virus. Browse to http://free.grisoft.com. I actually think that the free version is better than the full one, as there is less stuff to slow down our computer.

Daemon Tools

We all get the need to run an ISO straight off the hard disk (or network) sometimes, and as Windows can't do this on it's own (Linux users can insert mock here), we need Daemon Tools. You can grab it from www.daemon-tools.cc

Tweak UI

From the bowels of Microsoft comes a PowerToy that is actually quite good. It lets you change settings that previously required mucking around in the registry. I also find the auto-login quite good as it means I'm protected from the network with passwords, but I don't have to type it in to boot. You can get the PowerToys from here.

7z

7z is Winzip on steroids, and its open source too. Grab it from www.7-zip.org.

Spybot - Search & Destroy

Yeah, its got a cool name. And it keeps your computer squeaky clean. Get it from www.safer-networking.org.

And we're good

Now you get to buy most of the rest of the software you want to use. As always, if anyone has any other useful programs they always use, let me know!

My essential Kubuntu software installation list

Fri, 18 Jan 2008 08:34:00 +1300

Unlike Windows, Kubuntu comes with a large amount of software already installed, like OpenOffice, Amarok, Kaffeine, etc. However I always find I need to install a few more, and whats more, its pretty much always the same. I figured then it would be easier to keep this list online so I can find it, and other people can use it and make suggestions on it too.

The list

All this will need to go on one line but I've presented it like this so it's possible to read.

sudo apt-get install nfs-common wine firefox gimp sun-java6-jre sun-java6-plugin sun-java6-fonts eclipse apache2 php5 mysql-server mysql-client phpmyadmin kde-devel kde-devel-extras kdesvn synaptic libxine1-ffmpg avifile-mjpeg-plugin avifile-player gstreamer0.10-pitfdll libwavpack1 libxine1-plugins libxvidcore4 kdeaddons msttcorefonts

There will undoubtably be others, and I'll add to this list as I remember them.

Don't forget after installing eclipse to edit the /etc/eclipse/java_home file and change it so java6 is at the top of the list, otherwise it will make you want to kill yourself.

The joys of rebuilding a PC

Thu, 17 Jan 2008 22:21:00 +1300

Installing update 1 of 86... Do not power off or unplug your computer...

Sigh. Actually, I'm not really that worried that there are lots of updates. For one thing, it shows that there are still some developers active at Microsoft (they didn't all get put off by Steve Ballmer). What does annoy me is the number of system restarts I need to do to get Windows up and running. So far, I restarted for the second stage of install, at the end of install, three times for my Asus motherboard driver installs, once for my ATi install, and twice for Windows Updates. Thats, er nine restarts just to get to the point I can start to install my apps and games (and many of those will 'need' restarts too!). It annoys me how so many software developers put arbitrary restart prompts at the end of their un/install processes. They just don't need to! Windows can restart individual services like Linux can too, and files that are still in use can bloody well stay there until I choose to restart thanks!

Lets contrast that with Kubuntu shall we? I put the CD in and the operating system loads. All by itself. No restarts so far. I choose to permanently install and go through the wizard. At the end, I get the choice of restarting into the installed system or staying on the Live version for now. And if I click restart later, I'm not prompted every ten minutes to restart (on a timed prompt that if you miss it, your computer restarts no less). Once I finish what I was doing (oh yes, I was surfing the net while I was installing the operating system - bugger watching glorified statements from Microsoft saying how good the operating system is while installing), I restart my computer. And there we are!! That was, ah, one restart if I counted right. True, I didn't count doing my updates afterwards (and there are quite a lot) but they don't need restarts. Either the update is applied automatically, or in the case of core system bits like the kernel, it paitently waits until the next time you restart.

And they say Linux isn't ready for the desktop....

Grif gets an upgrade

Wed, 16 Jan 2008 19:48:00 +1300

As I posted in 00000080 bytes of pain, my main PC (Grif) has been having issues, which turn out to be memory related. The nice folks at Unleash have given me some temporary memory while my dud memory is being returned under warranty. It turns out all I had was DDR1, but my motherboard supports 1 & 2, so I've got a 1GB stick of DDR2 at the moment. Meanwhile I've been busy ordering upgraded parts. I've got 2x 1GB sticks of Kingston goodness on its way, as well as a couple of 320GB SATA2 drives with a lovely 16mb of cache.

I also ordered a Asus P5E motherboard which arrived today. As Intel haven't changed their sockets for a while my venerable old P4 3.2GHz fits this new one, so I've decided to hold off on a new CPU till the Intel E8400 comes into NZ next month.

You can't see from this angle, but this motherboard has another full speed (16x) PCIe slot just above the bottom PCI slot. It's just begging for another HD3870 - I can hear it! Also, I have to say I'm impressed with this camera of Jess's - that CPU fan is doing 2200RPM and its not even blurred!

It wasn't all smooth sailing putting this bad boy in though. The SATA and ATA connectors on this motherboard aim off the side of the board instead of the usual vertical orientation. I later realised that this is to make sure there is room for big-assed graphics cards, but I had other problems. My case is a no-screw jobby and this includes the motherboard which is on a removable tray. The release grip nicely blocked my ATA connector so I couldn't attach my optical drives. Grim. Oh well, nothing a mini hacksaw can't fix. Now I have half a release grip but its all I need, and at least I have my optical drives again.

I've taken out my Sound Blaster X-Fi eXtreme Music in favour of the SoundMax ADI1988 PCIe card which was included with the motherboard, as it means I can have surround sound in Linux :). I'll trial the sound quality in Windows games and see what I think - I may end up putting the SB back in to take advantage of its 10,000 transistors.

Speaking of Linux, guess which operating system didn't bat an eye at the new motherboard? Hah, and guess which operating system BSOD's on boot at the moment? Not that hard really is it. I'm holding off re-installing windows till my new harddrives arrive, so no games for now (except MotoX).

Backup your LAMP with rsync

Mon, 14 Jan 2008 22:37:00 +1300

Continuing with the backup theme of my last post, I've needed to back up our local LAMP server because its starting to hold some important data. Maybe I'm getting paranoid because of my first hard drive failure in 10 years (the last one was a 2GB HDD that got progressively larger areas of dead sectors, so it was a nice gradual death). Maybe I've just been taught a lesson and am starting to do what I always should have done anyway. In any case, here is the little script I wrote:

The script

#!/bin/sh # System + MySQL backup script using rsync # # Copyright (c) 2008 Al Twohill # This script is licensed under GNU GPL version 2.0 or above # ###Adjust these variables as you need ### # #The directories to back up DIRS="/home /etc /var/www" # #The MySQL login details MUSER="root" MPASS="mypassword" MHOST="localhost" # #Temp location to store MySQL Dumps MYTEMP="/tmp/mysql.bak" # #Location to backup to BACKTO="/mnt/backup" # # ### The script (you shouldn't need to edit this) ### MYSQL="$(which mysql)" MYSQLDUMP="$(which mysqldump)" GZIP="$(which gzip)" ### Start MySQL Backup ### # Get all databases name DBS="$($MYSQL -u $MUSER -h $MHOST -p$MPASS -Bse 'show databases')" for db in $DBS do FILE=/tmp/mysql.bak/mysql-$db.gz $MYSQLDUMP -u $MUSER -h $MHOST -p$MPASS $db | $GZIP -9 > $FILE done ### Dump backup using rsync ### rsync -aHk $DIRS $MYTEMP $BACKTO ### All done ###

It's nice and simple and should be fairly straightforward. However, this is not a one-size-fits-all solution, so I'll step through what I did so you can adjust for your needs.

The lowdown

The first bit is really straightforward, we just create some variables and assign them values.

"DIRS" is a space-separated list of the directories we want to back up. On a typical LAMP server this is probably all you need, but its easy enough to amend.

Next up we have the details we need to login to the database. Obviously because we store our password in plaintext we must make sure that our backp script is readable only by root.

"MYTEMP" is where we temporarily store the dumps of our databases. Again, as the database dumps are (gzipped) plaintext, we should make sure this folder is only accessible by root.

Finally in our variable list is where to backup to ("BACKTO"). You can use any value in here that fits in with an rsync destination. For more information on this, see http://samba.anu.edu.au/rsync/, and also this post. I'm using /mnt/backup because I have a remote filesystem mounted there, but you could just as easily rsync directly to a remote host (if the host supports it).

The nitty gritty

Now we come to how the script really works. We now get the necessary commands to perform our operations, using a nifty bit of code I found here (and some other code also came from here - just rewritten for my needs).

MySQL is a little tricky to back up, as if you just take the db files you'll need to start and stop the entire database service to do backups/restores, which isn't ideal - especially if you have several databases on the server. To get around this we use the tool mysqldump, which does what it says - it dumps its databases into a text file. We could make this bit of the script a little less complicated and dump the entire set of databases into one file, but I like this method better as it makes it really easy to restore individual databases. So, we execute 'show databases' to get a list of all the dbs we need to back up, then we loop through that list dumping each one into it's own gzipped file. Neat!

Finally, we actually rsync everything away, the directories we wanted at the start and the mysqldump temp directory of to our destination. One crontab entry later and we're happy! Rsync takes care of seeing what has changed where and makes this whole process pretty efficient.

Happy backupping everyone!

Zimbra backups are go

Wed, 09 Jan 2008 18:46:00 +1300

My Zimbra server's hard drive burnt out the other week while I was on holiday in Nelson. Crap no backups! Luckily I had only recently downgraded from my Network Edition trial to the Open Source edition, which involved backing up the old install. Hence how my Zimbra on Ubuntu tutorial came about.

Now that my email is back up and running I don't want the same thing to happen to me again. I consider myself extremely lucky I only lost 2 weeks worth of emails (and it was holidays so not much was there during that time). While the NE has automatic live backups, the FOSS edition doesn't. So, time to get my A into G and build myself a backup script.

Stolen from the horses mouth

The inspiration for my script came from the Zimbra wiki, so kudoes to those guys who helped out with that. Mine is a bit different, because I didn't set up my LVM with free space available to do snapshots, and fixing that means resizing my root partition, which I'm not in the mood for at the moment. Besides, I really want a backup before I start mucking with my partitions...

How it works

The methodolgy is quite simple really:

Stop Zimbra
Rsync /opt/zimbra to somewhere on the local harddrive (this is faster and means we can restart Zimbra sooner and receive our emails again)
Start Zimbra (in the background, so we can keep going with the script)
Rsync from the local backup to my other server's backup area
Call it a day and go to bed (its after 3.30am when this runs)

Here is the script in full. You can copy and paste it into a file and make it executable. Only allow root access to this file though, as we don't want other users arbitrarily restarting our precious Zimbra server at all hours of the day do we?

#!/bin/bash
#
#    Script to backup Zimbra OSS by stopping the service and
#    rsyncing to a destination, then restarting the service
#    This script is partially based on the scripts at
#     http://wiki.zimbra.com/index.php?title=Open_Source_Edition_Backup_Procedure
#
#    Copyright (C) 2008 Al Twohill
#
#    This program is free software; you can redistribute it and/or modify
#    it under the terms of the GNU General Public License version 2 as
#    published by the Free Software Foundation.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program; if not, write to the Free Software
#    Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
#    Or download it from http://www.gnu.org/licenses/old-licenses/gpl-2.0.html
#### Modify these values as required for your installation
# local dir to backup to while zimbra is stopped
localPath=/opt/backup
# remote directory to backup to (as defined by rsync) (leave blank to skip)
remotePath=zmbackup@sarge::zmbackup/
# password file to use with rsync (leave blank if not required)
passwordFile=/opt/backup-password
# use rysnc verbosely (leave blank to keep it quiet)
V=
#### You shouldn't need to modify below here

say() {
        MESSAGE_PREFIX="zimbra backup:"
        MESSAGE="$1"
        TIMESTAMP=$(date +"%F %T")
        echo -e "$TIMESTAMP $MESSAGE_PREFIX $MESSAGE"
        logger -t $log_tag -p $log_facility.$log_level "$MESSAGE"
        logger -t $log_tag -p $log_facility_mail.$log_level "$MESSAGE"
        }
error () {
        MESSAGE_PREFIX="zimbra backup:"
        MESSAGE="$1"
        TIMESTAMP=$(date +"%F %T")
        echo -e $TIMESTAMP $MESSAGE >&2
        logger -t $log_tag -p $log_facility.$log_level_err "$MESSAGE"
        logger -t $log_tag -p $log_facility_mail.$log_level_err "$MESSAGE"
        exit
        }
say "backup started"
say "stopping the Zimbra services, this may take some time"
/etc/init.d/zimbra stop || error "error stopping Zimbra"
say "rsyncing the snapshot to the local directory $localPath"
rsync -aHk$V --delete /opt/zimbra/ $localPath || "error during local rsync but continuing script"
say "restarting Zimbra in the background"
(/etc/init.d/zimbra start && say "Zimbra: Services background startup completed") || error "services background startup FAILED" &
if [[ $remotePath != " ]]
        then say "begining rsync to remote directory"
        rsync -aHk$V --delete --password-file=$passwordFile $localPath $remotePath
fi
say "finished backup script"

The main thing you really need to watch out for is doing the remote rsync. If you don't need that, you can just set remotePath to blank and it will skip it, but otherwise you need to make sure that you have the rsync daemon running on the remote machine. Using the standard rsync over ssh means that you have to enter the password manually, which we don't want to do. We like sleep.

Configure that daemon

Its really easy to have your daemon going in Ubuntu. First off, create the user you're going to use for your backups. It's good to do this so that if the account gets compromised they can't do anything else, as we give the account bugger all privleges.

sudo useradd -d /files/backup/zimbra -s /bin/false zmbackup

The -s /bin/false means this user can't log into a shell in on the server, which is what we want. Now give that user a password.

sudo passwd zmbackup

Next, we can create our rsync config files. Create and edit the rsyncd.config file

sudo nano /etc/rsyncd.conf

Create your rsync module in there. My /etc/rsyncd.conf looks like this:

[zmbackup]
uid = zmbackup
gid = zmbackup
path = /files/backup/zimbra
comment = ZM Backup area
auth users = zmbackup
secrets file = /etc/rsyncd.secrets
readonly = false

Really easy actually. It looks a lot like the Samba config file (hmmm, now why could that b? (; ) Basically we just set theh uid and gid's to our backup user, the path to the backup user's home directory (make sure that this user can actually write to this directory), then there is the rsync daemon authentication information. We need to authenticate with rsync's own mechinism to get access to the module. The auth users is just a space delimited list of allowable usernames, then the secrets file holds the login information. Finally we need readonly to be false if we want to write to the thing.

Next up, create your secrets file.

sudo nano /etc/rsyncd.secrets

In this file, you put in the entries for the usernames and passwords you want to allow. Use the format

username:password

And put each username on a new line. Finally we want to lock down these files to stop pesky users from snooping at our authentication.

sudo chown root:root rsyncd.* && sudo chmod 660 rsyncd.*

Get ready to go!

We've done all our configuring, lets get the ball rolling! You'll need to enable the rsync daemon service in Ubuntu.

sudo nano /etc/default/rsync

Change "RSYNC_ENABLE=false" to "RSYNC_ENABLE=true" then save and close. Now we can start the service

sudo /etc/init.d/rsync start

Now we can do a test run from our Zimbra machine. If that works fine, then set up a crontab to have it go every night, at a time that you don't receive important email.