[9fans] Returning to Plan 9: Virtualization, Distributions

[Joel C. Salomon]

After a long hiatus, I'd like to get back to experimenting with Plan
9. I have an Ubuntu Linux laptop with AMD's virtualization extensions
supported by the CPU, so I figure my best bet is one of the umpteen
virtualization tools. Which is best supported by Plan 9 — virtualbox,
qemu, or something else?

Also, what distributions are best for amd64? Bell Labs'? 9front? 9atom?

Thanks,
—Joel

[ron minnich]

If you're serious about booting a 64-bit os you need NIX. But you're
not going to get graphics.

ron

[Gabriel Díaz López de la llave]

hello

i tried to boot it with simnow, but the network helper crashes on my
installation, and i can´t load nix via pxe. (not sure if that´s
related to the fact that i have an intel processor and in the manual
they say amd is requeried :-?)

I´ll try to do it with vmware one of these days


gabi

Post by ron minnich
If you're serious about booting a 64-bit os you need NIX. But you're
not going to get graphics.
ron

[erik quanstrom]

Post by Gabriel Díaz López de la llave
hello
i tried to boot it with simnow, but the network helper crashes on my
installation, and i can´t load nix via pxe. (not sure if that´s
related to the fact that i have an intel processor and in the manual
they say amd is requeried :-?)
I´ll try to do it with vmware one of these days

today's nix is quite raw. unless you're working on nix itself,
you'll be happier with plan 9.

- erik

[Joel C. Salomon]

Post by ron minnich
If you're serious about booting a 64-bit os you need NIX. But you're
not going to get graphics.
today's nix is quite raw. unless you're working on nix itself,
you'll be happier with plan 9.

Is NIX the only distribution for amd64, then? I just want to play
around in user space: learn Go, use Unicode in C, &c., &c. Would I be
better off using a 32-bit distro?

—Joel

[John Floren]

Post by ron minnich
If you're serious about booting a 64-bit os you need NIX. But you're
not going to get graphics.
today's nix is quite raw.  unless you're working on nix itself,
you'll be happier with plan 9.

Is NIX the only distribution for amd64, then?  I just want to play
around in user space: learn Go, use Unicode in C, &c., &c.  Would I be
better off using a 32-bit distro?
—Joel

You should be able to happily use Nix as a 32-bit Plan 9 system.
Unfortunately we don't distribute an install ISO, but I have been
playing around with a bootable USB stick with a full-blown fossil
environment on it. The current task is to get it booting the nix
kernel as well as the regular 32-bit ones--we're getting there!

I guess there's no reason we *couldn't* do an install ISO... without
modifications it would just give you a 32 bit environment with the Nix
source available, and the selection of changes we've made.


John

[erik quanstrom]

Post by Joel C. Salomon

Post by erik quanstrom
today's nix is quite raw. unless you're working on nix itself,
you'll be happier with plan 9.

Is NIX the only distribution for amd64, then? I just want to play
around in user space: learn Go, use Unicode in C, &c., &c. Would I be
better off using a 32-bit distro?

nix runs in x86_64 long mode, the others do not. everything should
run on amd64 hardware and accomplish what you're after, though.

- erik

[ron minnich]

There's confusion here, and I am partly to blame ...

if you get googlecode.com/p/nix-os

you'll get a file system image that will be usable on a 32-bit
machine. We use it with 9vx. That image includes all the bits you need
to build and boot a NIX kernel. The intent of this distro is to allow
32-bit users to create 64-bit kernels and test. So the nix-os repo is
designed to let you set up on a 32-bit machine and bootstrap a
separate 64-bit machine. We thus envision you having more than one
system available.

I use it as follows:
hg clone http://googlecode.com/p/nix-os nix-os
cd nix-os
./9vx.OSX10.6 -r . -u rminnich
(get on the machine)
objtype=386
cd /sys/src/ape/lib
mk install
cd /sys/src
objtype=amd64
mk install
cd /sys/src/nix/k10
mk install

You are then good to go, unless I missed a step. You need some 386
bits from ape to build the 64-bit code.

We've set up nix-os root file system to play nice with 9vx, which is
why we include a 9vx for osx in the file system.

so to repeat: nix-os is a mercurial image of a root file system for
32-bit nodes, and it is intended to make it easy for you to boot
64-bit nodes. We assumed that you had at least one 32-bit and one
64-bit system.

I hope this helps a little.

ron

[Francisco J Ballesteros]

But, if you want more than one core, be sure you
install the CL I sent (which has not yet been applied).

I'll commit it later today so you could get SMP without
applying any CL by hand.

Post by ron minnich
hg clone http://googlecode.com/p/nix-os nix-os
cd nix-os
./9vx.OSX10.6 -r . -u rminnich
(get on the machine)
objtype=386
cd /sys/src/ape/lib
mk install
cd /sys/src
objtype=amd64
mk install
cd /sys/src/nix/k10
mk install
You are then good to go, unless I missed a step. You need some 386
bits from ape to build the 64-bit code.
We've set up nix-os root file system to play nice with 9vx, which is
why we include a 9vx for osx in the file system.
so to repeat: nix-os is a mercurial image of a root file system for
32-bit nodes, and it is intended to make it easy for you to boot
64-bit nodes. We assumed that you had at least one 32-bit and one
64-bit system.
I hope this helps a little.
ron

[Jack Norton]

Post by Joel C. Salomon
After a long hiatus, I'd like to get back to experimenting with Plan
9. I have an Ubuntu Linux laptop with AMD's virtualization extensions
supported by the CPU, so I figure my best bet is one of the umpteen
virtualization tools. Which is best supported by Plan 9 — virtualbox,
qemu, or something else?
Also, what distributions are best for amd64? Bell Labs'? 9front? 9atom?
Thanks,
—Joel

I have had good luck with qemu-kvm. I've even got a VPS running with
all the management bells and wistles like libvirt and such. It has been
running solid since march (lab's plan9 with fossil only).
I ran a qemu/kvm instance at home for a while too. That was under
archlinux though (and an AMD cpu with the necessary extensions).
In the former case I had to really play with plan9.ini to get it to boot
all the way, but it required nothing out of the ordinary in the end.
Choose your virtualized NIC wisely I suppose.

-Jack

[Joel C. Salomon]

Post by Jack Norton

Post by Joel C. Salomon
After a long hiatus, I'd like to get back to experimenting with Plan
9. I have an Ubuntu Linux laptop with AMD's virtualization extensions
supported by the CPU, so I figure my best bet is one of the umpteen
virtualization tools. Which is best supported by Plan 9 — virtualbox,
qemu, or something else?

I have had good luck with qemu-kvm. I've even got a VPS running with
all the management bells and wistles like libvirt and such. It has been
running solid since march (lab's plan9 with fossil only).
I found that Virtualbox worked very well when I was fiddling with my
Macbook on the way back from IWP9. I haven't tried it on the thinkpad
yet.

Thanks; I'll try them in turn, see if I get one to work.

—Joel

[Giacomo Tesio]

Please, (b)log the path: I'd like to play again with plan9... but I
completely forgot how I had configured qemu-kvm (and I remember that I had
had some trouble with the network on my debian)... :-(


Giacomo

Post by Joel C. Salomon

Post by Jack Norton

Post by Joel C. Salomon
After a long hiatus, I'd like to get back to experimenting with Plan
9. I have an Ubuntu Linux laptop with AMD's virtualization extensions
supported by the CPU, so I figure my best bet is one of the umpteen
virtualization tools. Which is best supported by Plan 9 — virtualbox,
qemu, or something else?

I have had good luck with qemu-kvm. I've even got a VPS running with
all the management bells and wistles like libvirt and such. It has been
running solid since march (lab's plan9 with fossil only).
I found that Virtualbox worked very well when I was fiddling with my
Macbook on the way back from IWP9. I haven't tried it on the thinkpad
yet.

Thanks; I'll try them in turn, see if I get one to work.
—Joel

[John Floren]

Post by Joel C. Salomon
After a long hiatus, I'd like to get back to experimenting with Plan
9.  I have an Ubuntu Linux laptop with AMD's virtualization extensions
supported by the CPU, so I figure my best bet is one of the umpteen
virtualization tools.  Which is best supported by Plan 9 — virtualbox,
qemu, or something else?
Also, what distributions are best for amd64?  Bell Labs'? 9front? 9atom?
Thanks,
—Joel

I found that Virtualbox worked very well when I was fiddling with my
Macbook on the way back from IWP9. I haven't tried it on the thinkpad
yet.

John

[Lyndon Nerenberg]

Post by John Floren
I found that Virtualbox worked very well when I was fiddling with my
Macbook on the way back from IWP9. I haven't tried it on the thinkpad
yet.

What would be *really* helpful is if people who have actual real live
running this minute Plan 9 under some VM system would post their
*specific* VM and Plan9 configuration files to the Wiki.

Several people claim to be running Plan 9 under assorted VMs, but it's
very difficult for others to reproduce that success, and every time I ask
someone for specific configs the response is "well that was months ago and
I don't use it any more" or suchlike.

Not that I don't believe them, but basically I don't believe them ;-)

[erik quanstrom]

Post by Lyndon Nerenberg
What would be *really* helpful is if people who have actual real live
running this minute Plan 9 under some VM system would post their
*specific* VM and Plan9 configuration files to the Wiki.
Several people claim to be running Plan 9 under assorted VMs, but it's
very difficult for others to reproduce that success, and every time I ask
someone for specific configs the response is "well that was months ago and
I don't use it any more" or suchlike.
Not that I don't believe them, but basically I don't believe them ;-)

i think the problem is that there are so many configurations.
there are at least

vm versions * vm config * real hardware

many of them. hardware passthrough has got to be one of the
least appealing ideas that's come out of virtualization. you get
all the complications of a virtual environment, coupled with the
convenience and sheer joy of dealing with hardware.

- erik

[Lyndon Nerenberg]

Post by erik quanstrom
i think the problem is that there are so many configurations.
there are at least
vm versions * vm config * real hardware

I know. That's why it's important to get this info online. Without it,
there's no hope of figuring out what configurations will work reliably.

[andrey mirtchovski]

oh, one more thing: i used bridged network.

[Lyndon Nerenberg]

Post by andrey mirtchovski
oh, one more thing: i used bridged network.

One more more thing: 9fans is not the wiki :-P

[andrey mirtchovski]

when i set up virtualbox i had to do these two steps:

download this plan9 image: http://virtualboxes.org/images/plan-9/
download and install the extension pack:
https://www.virtualbox.org/wiki/Downloads

then the other step was to enable USB 2.0 in the Ports config setting.

that's all. boots fine as a single-cpu os.

[andrew zerger]

So many configs- is true, since the configuration of the host itself is
critical to the virtual platform, and your distroVersion/openbox vs.
distro2Version/openbox.

Especially the successful archlinux/gentoo/lfs qemu-kvm guys will be
like---- well first I (already) had a kernel compiled for my hardware
(lspci, kernel.config), then I enabled KVM support in the kernel, and
tun/tap support, then it just worked, because of course, all of that is in
the wikis/docs for those necessary steps.

If anyone has trouble I would recommend the docs say, compile your own
virtual host and glean issue/resolution wiki from what transpires there,
otherwise it will be a distro specific problem on the hardware support side
of distro->vm->(no-longer hardware phase)guest, and that distro/VM team
would be more interested to know how what is broken than anyone looking at
plan9 code.

Or- not being some kind of gentoo snob, if I had/'there were' some docs on
how to get host-side information on how many supported/unsupported
syscalls, etc, plan9 made to qemu, I think that would be useful for
improving the performance of plan9 on virtual hardware, but I'm not sure.
Just letting my mind wander at the end of the day. Those docs on debugging
qemu guests probably exist somewhere I won't see right away.

regards,
andrew



ps,
Here's a really bad startup script for qemu-kvm, haha
(not really, its just really bad .. okay 1 line)

kvm -net nic,macaddr=$DISTMAC \
-net tap,ifname=$DISTTAP,script=no,downscript=no \
$DVMOPT \
-hda $DISIMG -m $MVMRAM -daemonize

Post by erik quanstrom

Post by Lyndon Nerenberg
What would be *really* helpful is if people who have actual real live
running this minute Plan 9 under some VM system would post their
*specific* VM and Plan9 configuration files to the Wiki.
Several people claim to be running Plan 9 under assorted VMs, but it's
very difficult for others to reproduce that success, and every time I ask
someone for specific configs the response is "well that was months ago

and

Post by Lyndon Nerenberg
I don't use it any more" or suchlike.
Not that I don't believe them, but basically I don't believe them ;-)

i think the problem is that there are so many configurations.
there are at least
vm versions * vm config * real hardware
many of them. hardware passthrough has got to be one of the
least appealing ideas that's come out of virtualization. you get
all the complications of a virtual environment, coupled with the
convenience and sheer joy of dealing with hardware.
- erik

[Bakul Shah]

Post by Lyndon Nerenberg

Post by John Floren
I found that Virtualbox worked very well when I was fiddling with my
Macbook on the way back from IWP9. I haven't tried it on the thinkpad
yet.

What would be *really* helpful is if people who have actual real live
running this minute Plan 9 under some VM system would post their
*specific* VM and Plan9 configuration files to the Wiki.
Several people claim to be running Plan 9 under assorted VMs, but it's
very difficult for others to reproduce that success, and every time I ask
someone for specific configs the response is "well that was months ago and
I don't use it any more" or suchlike.
Not that I don't believe them, but basically I don't believe them ;-)

I have added my notes about vbox 4 on OS X to

http://plan9.bell-labs.com/wiki/plan9/virtual_machines/index.html

Feel free to update.

To do: add freebsd notes.

But I am not a fan of Wikis. Usually a wiki ends up being an
unstructured collection of useful facts that can go stale as
it takes a lot of effort to keep it organized.

[Lyndon Nerenberg]

Post by Bakul Shah
But I am not a fan of Wikis. Usually a wiki ends up being an
unstructured collection of useful facts that can go stale as
it takes a lot of effort to keep it organized.

I'm no fan either, but until someone comes up with an alternative there
is no way of knowing which one sucks more.

[ron minnich]

Post by Bakul Shah
But I am not a fan of Wikis. Usually a wiki ends up being an
unstructured collection of useful facts that can go stale as
it takes a lot of effort to keep it organized.

Would be interesting if every wiki entry came with an expiration date,
and entries just went away or no longer appeared. So much of what I
find on wikis is completely wrong any more.

ro

[Bakul Shah]

Post by ron minnich

Post by Bakul Shah
But I am not a fan of Wikis. Usually a wiki ends up being an
unstructured collection of useful facts that can go stale as
it takes a lot of effort to keep it organized.

Would be interesting if every wiki entry came with an expiration date,
and entries just went away or no longer appeared. So much of what I
find on wikis is completely wrong any more.

A wiki is fine for something like an encyclopedia which is
basically a collection of loosely coupled information, or as a
corkboard of research notes.

A handbook can start out organized. Everything can be put in
its proper place, easy to parallelize the effort and reorg is
easy (mostly just metadata reorg -- in wiki you will have to
do cut-n-paste between pages). See the online FreeBSD handbook
for an example.

[Ruben Schuller]

To throw my experiences in:

I've installed bell labs plan9 in qemu-kvm. What really sped up the
installation was having the disk image in a tmpfs, otherwise the
copying to the hard disk took hours. With tmpfs it was finished in some
minutes. After moving this image to the harddisk the plan9 system was
really slow, though. This also worked for 9front, which seemed to run
more smoothly with the image not in ram. Testing 9atom is to be done.

The environment for these tests is an archlinux system with an amd cpu
and 8G of ram.

Be aware that i've not tested networking, the installation was from the
iso.

Steps (also with 9front):
Installation:
$ mkdir tmp
$ sudo mount -t tmpfs -o size=6G tmpfs tmp
$ qemu-img create tmp/plan9.img 2g
$ qemu-kvm -hda tmp/plan9.img -cdrom plan9.iso

After installation:
$ mv tmp/plan9.img .
$ sudo umount tmp
$ qemu-kvm -hda plan9.img

I hope this is of any use :)

Ruben

[Jens Staal]

I just managed to install 9front in virtualbox on my Arch linux x86_64
laptop (intel (Intel(R) Pentium(R) Dual CPU T2370 @ 1.73GHz), 2GiB RAM).
I first tried with the Bell labs image which booted nicely from CD but
after install, the boot from the virtual disk image failed. The "copy to
disk" phase of the install took some time also here, but hardly hours.

I think the critical setting to make it work in virtualbox was to set
the IDE controller to PIIX3 and to uncheck "use host I/O cache". I was
planning to try the 9atom sometime later (but now I am mostly playing
with the system that does work).

[John Floren]

Post by Jens Staal
I just managed to install 9front in virtualbox on my Arch linux x86_64
first tried with the Bell labs image which booted nicely from CD but after
install, the boot from the virtual disk image failed. The "copy to disk"
phase of the install took some time also here, but hardly hours.
I think the critical setting to make it work in virtualbox was to set the
IDE controller to PIIX3 and to uncheck "use host I/O cache". I was planning
to try the 9atom sometime later (but now I am mostly playing with the system
that does work).

I seem to remember a problem with booting Plan 9 in Virtualbox after
the install; you have to disable the CDROM device to make it boot. I
think it was Virtualbox.


John

[Lucio De Re]

Post by John Floren
I seem to remember a problem with booting Plan 9 in Virtualbox after
the install; you have to disable the CDROM device to make it boot. I
think it was Virtualbox.

VMware has that problem. Caught me a few times.

++L