I kept having a horrible error building collectd 4.10.x on my Source Mage servers:

libiptc.c:85: error: redefinition of ‘struct xt_error_target’

The internets couldn’t really help me with the error, I think I was searching for the wrong words. There was also issues regarding warnings that gcc now treats as errors.

I ended up spending probably about an hour cloning their git repo, cherry-picking a few commits into the 4.10.3 tag, and then submitting the patches to the bug I created.

End result is that collectd-4.10.3 now builds on gcc 4.4.3 and 4.6.1. The fix is in the Grimoire on Source Mage, and should trickle out to stable eventually. I think I’m the only one who uses collectd, so it’s not all that important, heh.

Well, I was using an internal lighttpd to render the same thing internally, and I didn’t disable its directory listing at all. Whilst I was working on setting up collectd to monitor my Apache scoreboards, I noticed that it could also monitor the lighttpd scoreboard. So I was going about the config to set that up, and I noticed that the Apache wasn’t proxying it’s requests through to lighttpd, and was taking a lot of time and resources to do something simple.

Welp, having ADD like I do, I got sidetracked on doing that instead and started to rewrite my configs to not serve files directly, but proxy it to the lighttpd internally.

Originally, I had two aliases set up “/sourcemage” and “/sourcemage/fallback” which both pointed to the same spot on disk, This way you could reference things using “/sourcemage/file.tar.bz2″ or “/sourcemage/fallback/file.tar.bz2″. The reasoning behind this is simple: I set up my fallback mirror before Source Mage standardized on a fallback URL, and I wanted to keep both paths functional. Easy to do with aliases. Oh, and I also have an alias to “/sourcemage/codex” for all my local codex needs.

Switching to a proxied setup was a bit more complicated than I anticipated. The alias stuff doesn’t work the same way everything else does, in that what it finds first it goes with. I needed to have mod_rewrite rules wired in to properly redirect things and manipulate the URL sufficiently. But I have two special cases. I can’t simply redirect everything “/sourcemage/*” to “/sourcemage/fallback.” Also, I wanted to be able to continue to use “/sourcemage/file.tar.bz2″

My solution is relatively simple and follows:

#aliases
Alias /sourcemage/codex    "/srv/webMirrors/sourcemage.org/codex"
#rewrite /sourcemage/ to /sourcemage/fallback
#some very fancy rewrite rules to keep the old alias functionality
RewriteEngine On
RewriteRule ^/sourcemage/?$ /sourcemage/fallback/ [R]
RewriteRule ^/sourcemage/codex/?$ - [L]
RewriteRule ^/sourcemage/([a-zA-Z0-9.i_\-+]+)$ /sourcemage/fallback/$1 [R]
# do the actual passing through to the lighttpd (which handles huge
# directory listing much much better
ProxyPass /sourcemage/fallback http://fallback.shlrm.org/
ProxyPassReverse /sourcemage/fallback http://fallback.shlrm.org/

# old notes kept here for reference as to what the above rewrite
# rules are doing -- dkowis 2011-09-07
# going to proxy these to the lighttpd
#Alias /sourcemage/fallback "/var/spool/sorcery/"
#Alias /sourcemage          "/var/spool/sorcery/"
# directories!
#<Directory "/var/spool/sorcery/" >
#       Options none
#    AllowOverride None
#    Order allow,deny
#    Allow from all
#</Directory>

<Directory "/srv/webMirrors/sourcemage.org/codex/" >
        Options Indexes
        Order allow,deny
        Allow from all
</directory>

# vi: set ft=apache:

The rewriting rules are the most complicated part of this. The trick was getting it not to try to rewrite anything when the url matched “/sourcemage/codex” After that, the rest was really easy. Some matching logic to pick up on “/sourcemage/file.tar.bz2″ and redirect that to “/sourcemage/fallback/$1″ and everything worked as it did before, except way faster.

So I’m back to the original setup, with an old (1998) left-over 2GB hard-drive, and a modified Source Mage install. It’s in good shape, with some tmpfs stuff for things that write to the disk in order to hopefully prolong the life of the disk. I took a clonezilla snapshot of it as well, so when the drive dies, I can throw in a new (also old, I have several 2 to 4 GB drives that are in the closet that still work) one, and the router will be back up in a few moments.

Whilst it was unfortunate that the router wasn’t a good fit for my particular use, perhaps I’ll find another use for it. Since there are two Serial ports trivially accessible on the device, perhaps I’ll get a GPS module and set it up in the car as a mobile automatic war-driving toy. I’ve found instructions to hook up an SD card to it as well, that would give it plenty of easy to replace storage.

It was an interesting experiment, and I’m somewhat sad it didn’t work out.

Finding this information should’ve been obvious, but for some reason it wasn’t. Maybe I had some silly mental block or something. Anyways, get the kernel source from here. Then build it for your architecture. If you want to build a 32-bit kernel for some 32-bit domUs, and you’re running on 64-bit, it’s as easy as make ARCH=i386 <target>. This does work and I’ve verified it. Had to fool a Fedora domU into beleiving that it really was a 32-bit system.

I’ve found make tarbz2-pkg to be quite handy in packaging up the kernel to be deployed.

I have verified that this works in Xen 3.4.2 on a patched 2.6.29.6 kernel.

On my Xen 4.0 test box, it fails to suspend. I’m using the blktap2 stuff, and perhaps that doesn’t pause well. As far as I understand it, that blktap2 should not affect suspending, but I’m going to test it with some logical volume based vms. I usually use the lvm backed disks anyway, but the blktap2 stuff offers some interesting features, if it were to work. Not being able to pause stuff probably isn’t good, but again, I’m not sure what’s causing it.

Apparently, that is what’s causing it. The domU will pause and unpause just fine when using an lvm backed disk, rather than the blktap2 ones. I should test a couple of other combinations to see if things like tap:aio will behave, as that’s needed for nifty things like remus.

Finally!

From the official page:

The Xen 4.0 release contains a number of important new features and updates including:

• Blktap2 – High performance VHD implementation supporting snaphots and clonces including live snapshots
• Netchannel2 – Support for new Smart NICs with multi-queue and SR-IOV functionality
• Fault Tolerance – Live transactional synchronization of VM state between physical servers
• Libxenlight – New library providing higher-level control of Xen between various toolstacks
• PV-USB and VGA Pass-through

SourceMage has already updated packages for it.

There’s a repository and src RPMs available for Fedora here. Specifically the Xen 4.0 src RPM. I’ll try to remember to fire up a build on my xen DomU that I use to build RPMs for me :)

It is time for Source Mage’s annual server fundraiser again. We’re raising enough to pay for our dedicated server that hosts our git repositories, and provides the master for our mirrors. We own that server entirely (well actually our current Project Lead does, since Source Mage isn’t a “real” organization) and it’s hosted in a co-location facility with an SLA and all that good stuff. Our costs are only about $100 a month, which isn’t too bad. This fundable should cover it in its entirety and ensure that Source Mage remains up and operational on the internet.

Incidentally, it’s also time for the Project Lead to be reelected, or a new Project Lead voted in. So if you’re a Source Mage developer and, for some strange reason, you’re reading my blog, go make a nomination if you feel like it.

Now you can view the stats on who’s doing the most work in my Xen hosts. Yay.

It explodes if there’s a halted vm. It’s not built to handle the new xen 3.0 lifestyle of things. I guess I’ll be learning myself some python. Or I’ll translate it all to bash ;) I could probably do it in bash. Not that python isn’t used for everything in xen anyway. Okay, now I’m rambling.

It’s spam for spam! I didn’t understand Russian and so sent it to Enqlave on #sourcemage on irc.freenode.net. I suspected, however, that it was spam for spam. Stuff traveling from one computer through the “cloud” and to another computer. Had to be email spam. Well, mostly because the only other kinds of spam (pen15, pr0n, stock scams) it wasnt ;)

Go Sourcemagers! I’ve posted a scan of the check I received from fundable to show you that it really did happen and we really did raise the money we thought we did.

To those that donated:

I really appreciate the financial support you’ve chosen to show Source Mage. Without it, it’d be a lot more difficult to make things happen. We’d probably still be on perforce running under the desk of someone’s office. Or we’d have git running there instead. Now, however we’re able to maintain a solid presence on the web on a server that is running our distribution. That’s the true test of a Linux distro. Can it support it’s developers? On the flip side, “Can it’s developers support it?” You’ve all shown that we can.

To those that didn’t donate:

Start saving now! We’ve only got 364 days until the next fundable ;) (Note: I just made that number up, I was making the “364 days until Christmas” joke, so laugh)

I’m even on there! OMG! I do stuff for SourceMage! Mah name is on teh intarwebs!