[9fans] growing/shrinking venti arena files -OR- arena file format

[s***@icebubble.org]

Hello,

I'm wondering if anyone can shed some light on growing and/or shrinking
arena files (i.e., disk partitions). With the growing popularity of
logical volume management, vitrualization, etc., resizing partitions is
becoming more and more common, and many file systems already have
"resize" tools or options to grow/shrink file system structures
according to changes in the size of the underlying device. I'm
wondering what capacities (if any) venti has for dealing with inceases
or decresases in the size of its arena files.

Yes, I know that the canonical way to add more storage to a venti server
is to format and add an ADDITIONAL arena file with venti/fmtindex -a.
But I'd like to know, specifically, if it's possible to grow/shrink an
arena file without breaking anything. If an arena file is extended,
will venti automatically make use of the extra space? Would the
additional arenas need to be formatted? And, if so, how? If empty
arenas at the end of an arena file are truncated (on an arena boundary,
ideally), would venti continue to function properly, fill up the
remaining space, and then continue on to the next arena file? Or, would
venti crash and burn in a pile of flaming bits and bytes?

Another way of stating this question might be to ask for information
about the arena file format. Knowing the file format, I could deduce
whether any inconsistency would be introduced by a change in file size.
Yes, I read the venti paper, including its descripion of arena files and
their contents. However, it is rather vague on such points, more like
an academic lecture or advertisement than a technical specification. Is
there a TECHNICAL SPECIFICATION for the arena file format? Getting my
eyeballs on one would probably allow me to figure this all out, for
myself. For that matter, any formal specifications for 9P2000 and the
venti protocol would be very helpful, too (for other purposes).

Any pointers (dereferenced or not :) would be greatly appeciated!

[l***@proxima.alt.za]

Post by s***@icebubble.org
Is
there a TECHNICAL SPECIFICATION for the arena file format?

The source code is eminently readable and the "words" directory (in
/sys/src/cmd/venti) contains quite a bit of details, at a glance. I'd
say there's enough there to make an educated guess or two.

In my experience, the most sensible way to resize disk capacity for
venti is to back up the arenas and then rewrite them to the new
format. I didn't expect it when I did it by accident, but it seems
practical, if not particularly fast. That, of course, may be a good
reason not to mess with arena sizes, it is hard to restore them
without spending a lot of time on them.

++L

[erik quanstrom]

Post by s***@icebubble.org
I'm wondering if anyone can shed some light on growing and/or shrinking
arena files (i.e., disk partitions). With the growing popularity of
logical volume management, vitrualization, etc., resizing partitions is
becoming more and more common, and many file systems already have
"resize" tools or options to grow/shrink file system structures
according to changes in the size of the underlying device. I'm
wondering what capacities (if any) venti has for dealing with inceases
or decresases in the size of its arena files.

i'll be honest, i don't understand this statement of the problem. the
venti abstraction has some answers for this problem. so i would be disinclined
to look "under the covers" to start.

venti arenas are very small (~650mb) and fill in order. so taking away empty
arenas should be no problem, while taking away partial arenas will lead
to data loss. 650mb seems like rounding error with today's storage, so
personally i would ignore the problem of resizing an arena. alternatively,
if one has truely tiny storage, i would be tempted to start with very small
arenas, and still add or remove full arenas. worst case, i think venti/copy
can be used to recover a deleted last arena to a smaller one. but the temporary
storage requirements are a bit greater.

- erik

[Brian L. Stuart]

Post by s***@icebubble.org
I'm wondering if anyone can shed some light on growing
and/or shrinking arena files (i.e., disk partitions). 
With the growing popularity of logical volume management,
vitrualization, etc., resizing partitions is becoming
more and more common, and many file systems already
have "resize" tools or options to grow/shrink file
system structures according to changes in the size of
the underlying device.  I'm wondering what capacities
(if any) venti has for dealing with inceases or
decresases in the size of its arena files.

To make sure I understand the question I'll attempt
to rephrase it.  If I have a logical volume containing
an arena partition and I resize that volume to a
larger size, is there a way to make venti aware of
the larger partition and format the additional space
as arenas in the same partition?  If I have understood
the question correctly, I don't think any of the existing
utilities will do that.  In principle, it should be
possible, assuming that you don't add so much that
the arena map won't fit into the fixed 512K map that
fmtarenas creates.  Fmtarenas just writes a partition
header and then loops through all the arenas calling
newarena.  You could probably update the partition
header and call newarena for each arena in the new
space.  There are probably some gotchas that I'm overlooking
at the moment, but that would be the place I'd start.

Post by s***@icebubble.org
Yes, I know that the canonical way to add more storage to a
venti server is to format and add an ADDITIONAL arena file
with venti/fmtindex -a.

I'd be inclined to think this is easier and safer
than munging around with an existing arena partition.

Post by s***@icebubble.org
Is there a TECHNICAL SPECIFICATION for the arena file
format?

The source code would the be most accurate spec on the
format.  I'd suggest starting with src/venti/srv/dat.h
and move on to look at fmtarenas.c.  Calls in it will
lead you to the other files that will be interesting.

Post by s***@icebubble.org
For that matter, any formal specifications for 9P2000

Section 5 of the man pages is the best reference for
this.

Post by s***@icebubble.org
venti protocol would be very helpful, too (for other
purposes).

The sources are the best place to look for this one
too.

BLS