Shlrm.org Blag » xen

OpenVSwitch

David Kowis — Thu, 20 May 2010 17:00:31 +0000

This is some interesting technology. Apparently it allows me to emulate a switch on my server such that I can get netflow information, and other such managed switch information, from my virtual machines on my xen box. I will have to build and implement this on my test xen box to see how difficult it’ll be to get it to behave.

I also like that it claims it can distribute across multiple hosts (‘claims’ because I haven’t verified it yet.) I wonder how difficult it will be to set up a switch that extends to my router, so I can just emulate my entire network’s switch in OpenVswitch. It’s certainly got some interesting potential.

There’s potential to get better information on the traffic happening in my network thanks to the netflows provided by this. Maybe it will be enough information to figure out why my traffic shaping doesn’t work well, and how I can improve it.

xm pause testing

David Kowis — Sun, 02 May 2010 00:05:31 +0000

When I got xm pause working, I was quite excited. However, it doesn’t work in all cases. I’ve done some testing to figure out what combination of things cause it to fail.

All of this is using xen 4.0 hypervisor, the kernel from here, and a pvops domU kernel built by hand.

Using blktap2 backed disks in the vhd format (vhd-util create -n name -s 1024) xm pause fails. The user-space disk devices are never released, and only a hard reboot will get it to function again.
Using a different driver for the blktap2, tap:tapdisk:aio, also fails to pause.
Using phy:/ backed disks, say through LVM works fine. The domU will pause, and resume nicely.

It would seem that there is something not quite ready to be used regarding the blktap2 backend, so I wouldn’t reccomend using it.

I also tried to use the original tap:aio drivers, but I cannot as they don’t seem to exist. I guess I’ll be using the plain old phy:/ drivers for now. That’s not a big deal as I don’t have any second xen machine to do anything fancy that the tap:aio drivers would provide. I guess I should poke on a mailing list or something.

xen pv_ops domU

David Kowis — Sat, 01 May 2010 23:30:19 +0000

…means I can finally pause/suspend a domU vm again!

Finding this information should’ve been obvious, but for some reason it wasn’t. Maybe I had some silly mental block or something. Anyways, get the kernel source from here. Then build it for your architecture. If you want to build a 32-bit kernel for some 32-bit domUs, and you’re running on 64-bit, it’s as easy as make ARCH=i386 . This does work and I’ve verified it. Had to fool a Fedora domU into beleiving that it really was a 32-bit system.

I’ve found make tarbz2-pkg to be quite handy in packaging up the kernel to be deployed.

I have verified that this works in Xen 3.4.2 on a patched 2.6.29.6 kernel.

On my Xen 4.0 test box, it fails to suspend. I’m using the blktap2 stuff, and perhaps that doesn’t pause well. As far as I understand it, that blktap2 should not affect suspending, but I’m going to test it with some logical volume based vms. I usually use the lvm backed disks anyway, but the blktap2 stuff offers some interesting features, if it were to work. Not being able to pause stuff probably isn’t good, but again, I’m not sure what’s causing it.

Apparently, that is what’s causing it. The domU will pause and unpause just fine when using an lvm backed disk, rather than the blktap2 ones. I should test a couple of other combinations to see if things like tap:aio will behave, as that’s needed for nifty things like remus.

Xen 4.0 released

David Kowis — Wed, 07 Apr 2010 12:37:45 +0000

Finally!

From the official page:

The Xen 4.0 release contains a number of important new features and updates including:

Blktap2 – High performance VHD implementation supporting snaphots and clonces including live snapshots

Netchannel2 – Support for new Smart NICs with multi-queue and SR-IOV functionality

Fault Tolerance – Live transactional synchronization of VM state between physical servers

Libxenlight – New library providing higher-level control of Xen between various toolstacks

PV-USB and VGA Pass-through

SourceMage has already updated packages for it.

There’s a repository and src RPMs available for Fedora here. Specifically the Xen 4.0 src RPM. I’ll try to remember to fire up a build on my xen DomU that I use to build RPMs for me :)

Xen Stats Revised

David Kowis — Mon, 08 Mar 2010 12:38:57 +0000

I had reimplemented some monitoring on my network and then I got to wondering if there was a revised version of the original xen-stats stuff I was using. Turns out there is. The site details instructions on how to set it up in cacti, and it works pretty well. I do wish there was a single graph to collect all for the hosts, instead of individual host graphs, but it’s not bad. Perhaps I’ll build a template that uses the data source and glues them all into one graph. But it provides reasonably accurate data, and it’s not difficult to set up.

Xen stats FTW!

(The UPS graphs I need to fix too, somethings broken in the PHP datasource it uses. So if you’ve got a reliable Network UPS Tools graphing source for cacti, other than this one, lemme know.)

Fedora 12 Xen RPMs

David Kowis — Tue, 17 Nov 2009 20:02:57 +0000

I built these on a recently updated Fedora 12 box. I haven’t yet had an opportunity to test them. If you do use them and they work great, or not, post in the comments please.

Xen 3.4.2 rpms for Fedora 11 x86_64

David Kowis — Thu, 12 Nov 2009 15:46:46 +0000

I was using xen 3.4.1 on my fedora 11 Domain0, but I couldn’t ever suspend anything. xm save always failed. Someone on IRC suggested that it was the guest kernel that was to blame. Turns out it was xen 3.4.1.

After many hours searching the intertubes for a srpm or a spec file for building xen 3.4.2 for fedora, I decided that I would hack an existing one to 3.4.2 and hope for the best.

It built after I removed a few patches. The patches mostly complained about being already applied, so I assume the bugs the patches were addressing were fixed in 3.4.2. I have generated rpms for x86_64 and put the spec file up on my site. Enjoy.

Fedora Domain 0 and Domain U I/O performance

David Kowis — Thu, 22 Oct 2009 16:07:07 +0000

I finally got a Fedora 11 domain0 up and running. Built my own kernel using the patches from gentoo-xen-kernel. Specifically, the 2.6.29.6 patches, as I had issues running the 2.6.30 kernel on this particular box. This box does not have the CPU virtualization extensions, so every VM has to be paravirtualized. Here, I discuss a bit on the awesomeness of pygrub, and then I publis a couple bonnie++ results from the dom0 and the domU.

First I’ll rave about pygrub. This allows me to give the owner of a paravirtualized box the ability to install their own kernel. There’s only one constraint: The kernel must support being a domU. So it doesn’t quite get you unmodified guest kernel support, but you can still install your own kernel if you want a specific kernel for some reason. Also, vanilla kernels starting from about 2.6.27 onward have support for running in a domU, so long as you turn it on. Fedora 11′s kernel has this ability, so you can do a bit of configuration, and install a Fedora 11 domU right off the CD. That’s pretty awesome.

I use LVM backed disks for domains rather than disk image files. I’ve found that the LVM backed disks perform much better than the file backed ones. The installs that I’ve done with Fedora 11 are actually using the disk I have given the domU as a physical disk, partitioning it, installing LVM and everything. I was concerned about the performance of two levels of LVM, so I ran bonnie++ on both the domU and the dom0. The dom0 is one level of LVM, and the domU will have two levels of LVM.

Domain 0 IO test result:

Version 1.03c       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
fedoraxen.shl 1512M 50362  83 41511  11 22227   1 46086  92 72866   0 207.4   0
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  9622  98 +++++ +++ +++++ +++  9834  99 +++++ +++ +++++ +++

Fedora 11 DomU (on the same box):

Version 1.03c       ------Sequential Output------ --Sequential Input- --Random-
                    -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
TestVM           2G 52894  96 65639  31 29314  14 36553  69 79138  11 189.6   0
                    ------Sequential Create------ --------Random Create--------
                    -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 16  7932  82 +++++ +++ 30738  99  9571  99 +++++ +++ 31084  99

So it seems that performance isn’t hit that badly, things are a bit slower, as to be expected with two levels of LVM, but not horribly so.

Migration proceeding

David Kowis — Tue, 01 Sep 2009 20:17:09 +0000

The new cpu coolers fit and I can highly recommend them. The loudest fan on the server is the 5-drive sata hot-swap bay. I’ll make a posting detailing the server and the pitfalls I encountered for others on the internet, if any others actually read this, heh.

Anyways, the migration is proceeding, I have 10VMs running on the xen host. I have a few more to build to get the rest of my services segregated enough. I’m using LDAP on a VM to have centralized authentication and authorization. It’s still evolving slowly, but I can get the services I need running on a vm as they were, and then build a new VM to start the LDAP integration. Xen rules.

Xen Stats

David Kowis — Mon, 07 Apr 2008 02:38:51 +0000

http://shlrm.org/xenstats/ (original scripts here)

Now you can view the stats on who’s doing the most work in my Xen hosts. Yay.

It explodes if there’s a halted vm. It’s not built to handle the new xen 3.0 lifestyle of things. I guess I’ll be learning myself some python. Or I’ll translate it all to bash ;) I could probably do it in bash. Not that python isn’t used for everything in xen anyway. Okay, now I’m rambling.