[9fans] (no subject)

[erik quanstrom]

It might be a bit
much for home use, but if I had a little bit of a budget I'd use Coraid's AoE
stuff as the basis for my storage.

Yeah, it's pretty overkill. I've previously worked at a storage
company as a file system guy and now I have at home a nice array with
ZFS on top. It works great, but I want to scale down. I want less
stuff, not more. And I want to use Plan9, not Solaris.

aoe doesn't require solaris, or any other operating system.
you can use it directly with a plan 9 file server, as i do.

- erik

[Jack Norton]

Post by erik quanstrom

It might be a bit
much for home use, but if I had a little bit of a budget I'd use Coraid's AoE
stuff as the basis for my storage.

Yeah, it's pretty overkill. I've previously worked at a storage
company as a file system guy and now I have at home a nice array with
ZFS on top. It works great, but I want to scale down. I want less
stuff, not more. And I want to use Plan9, not Solaris.

aoe doesn't require solaris, or any other operating system.
you can use it directly with a plan 9 file server, as i do.
- erik

I don't think he was implying that one needed the other. In any case I
figured I would ask -- are there any plans for a small scale AoE
appliance from coraid? Didn't there used to be a single drive AoE kit
long ago?
What is the list's personal/home usage of AoE like? Do you guys use
generic hardware and something like vblade (or those other ones people
have made for linux like ggblade or whatever it is called)?

-Jack

[erik quanstrom]

Post by Jack Norton
I don't think he was implying that one needed the other. In any case I
figured I would ask -- are there any plans for a small scale AoE
appliance from coraid? Didn't there used to be a single drive AoE kit
long ago?

there was once a single drive pata unit.

Post by Jack Norton
What is the list's personal/home usage of AoE like? Do you guys use
generic hardware and something like vblade (or those other ones people
have made for linux like ggblade or whatever it is called)?

http://www.quanstro.net/plan9/fs.html

i also use vblade(8) on plan 9 for testing.

- erik

[Aram Hăvărneanu]

Post by erik quanstrom
aoe doesn't require solaris, or any other operating system.
you can use it directly with a plan 9 file server, as i do.

Of course AoE doesn't require much, my comment was in the context of
Coraid's hardware appliance.

I don't have much use for AoE at home. At one point I used it to
network boot machines, but I only have laptops now, which have local
disks because I need to use them disconnected from the network
sometimes.

I need a higher level protocol like 9p or venti, and I'd rather have a
single Plan 9 machine with direct attached disks serving everything
than a Plan 9 front end serving 9p and another machine providing AoE
to it. I have way, way to many machines. Yesterday I've thrown away
5. I need less machines, not more :-).

Does the Coraid applience implement RAID in hardware or does it use
fs(3) or another software solution?

[Bakul Shah]

My fileserver is running freebsd zfs. Basically one machine for nfs, venti, cifs, timemachine + sundry other services. This has worked well since 2005. Initially I used h/w raid under zfs. This was a mistake, forcibly corrected when my machine died. Now I use raidz. Since then I have replaced disks with much bigger disks without taking the machine down, upgraded the os & zpool/zfs versions a couple of times (could've done without them but I prefer to run the latest stable release). Except for these events it has required hardly any maintenance (just checking vital signs like fan speed, any disk errors in weekly zpool scrub etc). One issue is FreeBSD install still doesn't install on zfs. But you can run a minimal root off a USB disk and manually setup zfs (which is pretty easy). I should probably run an AOE server on it as well! When I next replace this machine I will see if I can create a USB disk image.

Post by Aram Hăvărneanu

Post by erik quanstrom
aoe doesn't require solaris, or any other operating system.
you can use it directly with a plan 9 file server, as i do.

Of course AoE doesn't require much, my comment was in the context of
Coraid's hardware appliance.
I don't have much use for AoE at home. At one point I used it to
network boot machines, but I only have laptops now, which have local
disks because I need to use them disconnected from the network
sometimes.
I need a higher level protocol like 9p or venti, and I'd rather have a
single Plan 9 machine with direct attached disks serving everything
than a Plan 9 front end serving 9p and another machine providing AoE
to it. I have way, way to many machines. Yesterday I've thrown away
5. I need less machines, not more :-).
Does the Coraid applience implement RAID in hardware or does it use
fs(3) or another software solution?
--
Aram Hăvărneanu

[Aram Hăvărneanu]

Post by Bakul Shah
Since then I have replaced disks with much bigger disks without taking the
machine down, upgraded the os & zpool/zfs versions a couple of times

Could you do the latter without taking the machine down?

Upgrade zpool/zfs version yes, os, no.

[Charles Forsyth]

Could you do the latter without taking the machine down?

Post by Bakul Shah
Since then I have replaced disks with much bigger disks without taking the
machine down, upgraded the os & zpool/zfs versions a couple of times

[Aram Hăvărneanu]

zpool/zfs upgrade? Yes. Don't recall if I had to reboot
afterwards.

You don't.

[Charles Forsyth]

That's upgrading the stored formats, not the in-kernel(?) software support
for a particular version?

Post by Aram Hăvărneanu

zpool/zfs upgrade? Yes. Don't recall if I had to reboot
afterwards.

You don't.
--
Aram Hăvărneanu

[Lyndon Nerenberg]

Post by Charles Forsyth
That's upgrading the stored formats,

Yes.

[Bakul Shah]

Could you do the latter without taking the machine down?

Post by Bakul Shah
Since then I have replaced disks with much bigger disks without taking the
machine down, upgraded the os & zpool/zfs versions a couple of times

zpool/zfs upgrade? Yes. Don't recall if I had to reboot
afterwards. "Resilvering" to replace a disk takes a long time
so not having to take the machine for hours is nice. A quick
reboot is no big deal.

[erik quanstrom]

Post by Aram Hăvărneanu
I don't have much use for AoE at home. At one point I used it to
network boot machines, but I only have laptops now, which have local
disks because I need to use them disconnected from the network
sometimes.
I need a higher level protocol like 9p or venti, and I'd rather have a
single Plan 9 machine with direct attached disks serving everything
than a Plan 9 front end serving 9p and another machine providing AoE
to it. I have way, way to many machines. Yesterday I've thrown away
5. I need less machines, not more :-).

sure, but you haven't answered the question of how to do redundancy
and recovery. aoe is a good way to isolate these functions into an appliance.

Post by Aram Hăvărneanu
Does the Coraid applience implement RAID in hardware or does it use
fs(3) or another software solution?

if a coraid appliance were pcie-attached rather than ethernet attached,
would you still ask this question? do you think the block diagram of coraid
hardware looks fundamentally different than the block diagram of a raid
card?

- erik

[Jack Norton]

Post by erik quanstrom

Post by Aram Hăvărneanu
Does the Coraid applience implement RAID in hardware or does it use
fs(3) or another software solution?

if a coraid appliance were pcie-attached rather than ethernet attached,
would you still ask this question? do you think the block diagram of coraid
hardware looks fundamentally different than the block diagram of a raid
card?
- erik

I think he is trying to get you to divulge details on the software
running on the coraid appliance itself. We all know it is plan 9 based
(right?), so it would be interesting to know how this extra
functionality of raid was added. I know these are trade secrets though
so I've never asked.

I would still ask that question if it were pcie attached. Curiosity
will be my undoing though.

-Jack

[erik quanstrom]

Post by erik quanstrom
if a coraid appliance were pcie-attached rather than ethernet attached,
would you still ask this question?  do you think the block diagram of coraid
hardware looks fundamentally different than the block diagram of a raid
card?

It's just curiosity. I know the appliance is Plan 9 based. If it
uses an off-the-shelf RAID chip I might buy a card with that chip
since it works in Plan 9. If it's fs(3) I know fs(3) is good enough
for my needs. If it's something else at least I know fs(3) is not
good enough and I might be tempted to write something myself. So yes,
I'd ask even if it was a PCIe card instead of network appliance.

the motivation behind my question is that it's not clear to me that there is
such a thing as pure hardware raid. if someone knows of something that
implements the entire read/write path without a cpu, even with a degraded
or rebuilding raid, i'd be very interested in that. but as far as i know, there's
always a processor in there on the other side of the bus. in case of aoe, the
bus is ethernet and for a "hardware raid" card, it's usually some form
of pci.

(see wiki's raid article.)

- erik

[Aram Hăvărneanu]

Post by erik quanstrom
the motivation behind my question is that it's not clear to me that there is
such a thing as pure hardware raid.  if someone knows of something that
implements the entire read/write path without a cpu, even with a degraded
or rebuilding raid, i'd be very interested in that.

Ahh, I see what you mean. I agree that probably there is no such
thing as RAID purely implemented in hardware. I think for most people
the hardware epithet is used to describe a black box where the user
doesn't have access to it's internal gearing, be it hardware or
software. A microwave is a hardware device for its users because the
users don't need to care if the unit has firmware, discrete
electronics or mechanical gears.

Probably a better term would be to always use the term appliance,
instead of this hardware/softare false dichotomy.

Post by erik quanstrom
(see wiki's raid article.)

Plan 9 wiki? It mentions SiL 3112 SATA, 3114 SATA/RAID and VIA 82C686,
VT8237 SATA/RAID, strangely, under IDE section.

I haven't found yet relevant information about the SiL stuff, but the
VIA stuff is what Adaptec calls HostRAID and Linux fakeRAID.

A Happy New Year!

[erik quanstrom]

Post by Aram Hăvărneanu

Post by erik quanstrom
(see wiki's raid article.)

Plan 9 wiki? It mentions SiL 3112 SATA, 3114 SATA/RAID and VIA 82C686,
VT8237 SATA/RAID, strangely, under IDE section.
I haven't found yet relevant information about the SiL stuff, but the
VIA stuff is what Adaptec calls HostRAID and Linux fakeRAID.

wikipedia.

there's nothing strange about a sata device or even a raid of
various devices of any type being presented with an ide programming
interface. one could just as easily slap an ahci programming interface
on, but either requires translation software/hardware.

for me, the most important questions are
- how do i set up a raid/hot spares, and
- can i do this without rebooting.

- erik

[Aram Hăvărneanu]

Post by erik quanstrom
for me, the most important questions are
- how do i set up a raid/hot spares, and
- can i do this without rebooting.

Of course, and right now I'm doing exactly that using a different
operating system. Can I do that on Plan 9? I don't know, I'm trying
to find out without much success.

Post by erik quanstrom
wikipedia.

I really don't understand why are you sending me to read wikipedia.
Generally, I think of myself as a decent speaker, I know how to make
myself clear. It's obvious that in this case I failed. In my
previous job I have worked on a file system that among other things
also implements redundancy. If I implemented these things I guess I
know about them without having to read wikipedia.

The machine I use today for storage also runs bits of my own software.
It's very easy to administer, tells me when disks are broken, I can
just add disks for more storage without reboot and I can hot swap
disks. Can Plan 9 do this? I don't know, I guess not? It's fine by
me, I'm willing to sacrifice performance and ease of administration
for an operating system I like better. I'm willing to implement
myself what I need and doesn't exist yet, though I have a very hard
time understanding what's missing from these very, very vague
discussions.

Post by erik quanstrom
there's nothing strange about a sata device or even a raid of
various devices of any type being presented with an ide programming
interface.  one could just as easily slap an ahci programming interface
on, but either requires translation software/hardware.

I agree that's nothing inherently strange, but in practice it's
uncommon, at least in my experience. But then again, I'm not a
hardware guy, so my experience means nothing.