[9fans] p9p vac/vacfs compatibility: uid/gid, ctime, 9P2000.L, 9pserve

[s***@icebubble.org]

Hello,

I'm trying to use vac/vacfs on p9p, using the Linux v9fs driver (-t 9p)
with unix domain sockets as transport (-o trans=unix). And I'm
confused. :(

When I mount the file system, all the files are owned by uid = gid = (-2
mod 2^32), unless I tell the Linux kernel to mount the fs as a specific
user/group. In the latter case, all files present with the specified
user/group (as expected). I can't find any way to get at the actual
uid/gids owning the files/directories vac'd. I have tried explicitly
specifying -o version=9p2000.u, but it exhibits the same symptoms: all
files and directories appear to be owned by uid = gid = (-2 mod 2^32).
All uids/gids in the vac'd tree existed in /etc/{passwd,group} at the
time of vac'ing, as well as at mount time.

So, I took a little looksie in /usr/lib/plan9/src/cmd/vac/...

----------------------------------------------------------------
/usr/lib/plan9/src/cmd/vac/vac.c:48
#ifdef PLAN9PORT
/*
* We're between a rock and a hard place here.
* The pw library (getpwnam, etc.) reads the
* password and group files into an on-stack buffer,
* so if you have some huge groups, you overflow
* the stack. Because of this, the thread library turns
* it off by default, so that dirstat returns "14571" instead of "rsc".
* But for vac we want names. So cautiously turn the pwlibrary
* back on (see threadmain) and make the main thread stack huge.
*/
extern int _p9usepwlibrary;
int mainstacksize = 4*1024*1024;

#endif

----------------------------------------------------------------
/usr/lib/plan9/src/cmd/vac/vac.c:399
vd->uid = dir->uid;
vd->gid = dir->gid;
vd->mid = dir->muid;

----------------------------------------------------------------
So, the source *appears* to stow uid/gid to venti as character strings.

BTW,

----------------------------------------------------------------
/usr/lib/plan9/src/cmd/vac/vac.c:406
vd->ctime = dir->mtime; /* ctime: not available on plan 9 */


----------------------------------------------------------------
Does this mean that vac doesn't store ctimes on p9p? Shouldn't this be
wrapped in something like:

----------------------------------------------------------------
#ifdef PLAN9PORT
vd->ctime = dir->ctime; /* ctime: available on p9p */
#else
vd->ctime = dir->mtime; /* ctime: not available on plan 9 */
#endif


----------------------------------------------------------------
Then, we have:
----------------------------------------------------------------
/usr/lib/plan9/src/cmd/vac/vacfs.c:696
dir.atime = vd->atime;
dir.mtime = vd->mtime;
dir.length = vd->size;

dir.name = vd->elem;
dir.uid = vd->uid;
dir.gid = vd->gid;
dir.muid = vd->mid;
#ifdef PLAN9PORT
dir.ext = ext;
dir.uidnum = atoi(vd->uid);
dir.gidnum = atoi(vd->gid);
#endif

----------------------------------------------------------------

Those dir.uidnum and dir.gidnum don't appear to be referenced anywhere
else in the code. But it makes me wonder... are the character string
uid/gids vac'd the textual names from /etc/{passwd,group}, or just ASCII
versions of the numeric ids, a la format "%d"?

And just what is "9P2000.L", anyway? I can't seem to find any
documentation on it.

Last, but not least, can anyone tell me why 9pserve is taking up so much
CPU time? From the docs, it looks like 9pserve should just be muxing 9p
connections and passes messages around. I had expected that disk
bandwith would be the bottleneck in my setup. But, as it turns out, the
bottleneck is actually this little fellow, 9pserve. I have no idea why
it's taking so much CPU to multiplex 9p, especially since there's only
one connection being "multiplexed"! Is there any way to bypass 9pserve
(since I only need one connection for now), i.e., doing 9p on sockets or
shared file descriptors?

Thanks!

[David du Colombier]

Post by s***@icebubble.org
When I mount the file system, all the files are owned by uid = gid =
(-2 mod 2^32), unless I tell the Linux kernel to mount the fs as a
specific user/group. In the latter case, all files present with the
specified user/group (as expected). I can't find any way to get at
the actual uid/gids owning the files/directories vac'd. I have tried
explicitly specifying -o version=9p2000.u, but it exhibits the same
symptoms: all files and directories appear to be owned by uid = gid =
(-2 mod 2^32). All uids/gids in the vac'd tree existed
in /etc/{passwd,group} at the time of vac'ing, as well as at mount
time.

The problem you are encountering is that v9fs doesn't translate
uid/gid strings back to their numeric equivalent in /etc/passwd.
Since Unix doesn't know how to handle these strings, it falls
back to -1.

Forget about 9p2000.u, it's deprecated.

Post by s***@icebubble.org
So, the source *appears* to stow uid/gid to venti as character
strings.

This is the case. You can show the actual uid/gid from the
Plan 9 vacfs and even on p9p, using vacfs -d.

Post by s***@icebubble.org
Does this mean that vac doesn't store ctimes on p9p?

Yes, vac doesn't store ctime. Maybe we could store it in vac,
as you suggest, but 9P doesn't handle it anyway.

Post by s***@icebubble.org
Those dir.uidnum and dir.gidnum don't appear to be referenced anywhere
else in the code.

I never paid much attention to it. It looks unused.

Post by s***@icebubble.org
But it makes me wonder... are the character string
uid/gids vac'd the textual names from /etc/{passwd,group}, or just
ASCII versions of the numeric ids, a la format "%d"?

They are the textual names.

The problem you are describing is not related to vac nor vacfs.
They both handle uid/gid properly, and, as you saw, p9p vac can
even translate numeric uid/gid to their string representation.

[s***@icebubble.org]

Post by David du Colombier
The problem you are encountering is that v9fs doesn't translate
uid/gid strings back to their numeric equivalent in /etc/passwd.
Since Unix doesn't know how to handle these strings, it falls
back to -1.

I had suspected that. However, the uid/gid value being displayed is not
-1. It's -2. Maybe that's what threw me. Perhaps, somewhere, there's:

#define EDRIVECRAZY -2

Post by David du Colombier
Forget about 9p2000.u, it's deprecated.

IIRC, the RFC for it didn't indicate deprecated status. (Perhaps IETF
could use an update.) Assuming it has been deprecated, with what has
9p2000.u been replaced?

[David du Colombier]

Post by s***@icebubble.org
IIRC, the RFC for it didn't indicate deprecated status. (Perhaps IETF
could use an update.) Assuming it has been deprecated, with what has
9p2000.u been replaced?

As far I know, these RFCs were early draft and were never completed
nor submitted.

I think the 9P2000.u authors moved to a new approach, which was
called 9P2010 and is now called 9P2000.l. I don't know anything
about it however.

Anyway, plan9port applications only speak 9P2000.

[Latchesar Ionkov]

Post by David du Colombier
Forget about 9p2000.u, it's deprecated.

Who decided that .u is deprecated? I definitely didn't and I am using
it for all my file servers that are supposed to work on Unix.

Thanks,
Lucho

[David du Colombier]

Post by Latchesar Ionkov
Who decided that .u is deprecated? I definitely didn't and I am using
it for all my file servers that are supposed to work on Unix.

Sorry, I mean the plan9port support for 9p2000.u is deprecated.
It was removed in december 2010.

[s***@icebubble.org]

OK, I'm stumped. Maybe I'm making this more complicated than it has to
be. Posing my question as simply as possible: If plan9port's vac is
used to archive a Linux file system to venti, is there any way to access
that file system in a way that exposes symbolic links, device nodes,
sockets, and FIFOs in the archive?

There are a number of possibilities I can think of (listed in
approximate order of increasing obscurity):

(1) plan9port's vacfs
(2) plan9port's unvac
(3) Netfiles (under acme)
(4) plan9port's 9p command
(5) Inferno

I have a small "test" Linux filesystem with a mixture of regular files,
directories, and character and block devices which I've archived
(successfully!) using plan9port's vac. I can see the special files in
the venti data. They're in there! But...


(1) When using Linux's v9fs to mount a file system served by plan9port's
vacfs, the symlinks and device nodes all stat as regular files. The
file size of symlinks reflects the length of the link target, and the
device files have a size of 3 bytes (for type, major, and minor). This
happens whether or not I include version=9p2000.u or 9p2000.L as mount
options.

When I use 9pfuse to mount the file system, I get a little further:

lrw-rw---- 1 root root 6 Mar 10 2005 video ->
-rw-rw---- 1 root root 3 Mar 10 2005 video10

The symbolic links show up as links, but with the null string ("") as
target. The Linux kernel interperets null symlinks to mean "./",
causing all the links to refer to the directory in which they reside. :(
The device files still present as ordinary files with a size of 3 bytes.
Running "readlink" on the symbolic links always returns the null string.


(2) plan9port's unvac explicitly skips extraction of files which are not
regular files or directories. So, unvac-ing the archive isn't possible,
either. :(


(3) According to the Plan 9 wiki, Netfiles will only work with versions
of OpenSSH older than the version I have.


(4) The 9p utility provided by plan9port gets me access to a little more
metadata. Using the 9p "ls" command:

-Lrwxrwxrwx M 0 root root 5 Jan 2 03:16 modem
D-rw-rw---- M 0 root disk 3 Mar 10 2005 hda

You can see the symlink and device bits set in the mode. (The "D"
represents a device node; "d" indicates a directory.)

Using the 9p "stat" command:

'modem' 'root' 'root' '' q (0000000000000003 0 ) m 0200000777 at 1325474216 mt 1325474215 l 5 t 0 d 0
'tty3' 'root' 'tty' '' q (0000000000001528 0 ) m 040000660 at 1110476900 mt 1110476900 l 3 t 0 d 0

The targets of symlinks are still showing up as the null string. (I'm
assuming that's what the '' represents.)


(5) Installing Inferno just to access vac'd Linux file systems would
really be tantamount to throwing up my hands and giving up on plan9port.
I'd like use plan9port, if possible, rather than have to layer a whole
new OS on top of Linux+p9p.


So, I'm fresh out of ideas. Is there any way, at all, to present the
full contents of a Linux filesystem that's been archived using the
plan9port vac command?

Thanks!

[David du Colombier]

Post by s***@icebubble.org
OK, I'm stumped. Maybe I'm making this more complicated than it has
to be. Posing my question as simply as possible: If plan9port's vac
is used to archive a Linux file system to venti, is there any way to
access that file system in a way that exposes symbolic links, device
nodes, sockets, and FIFOs in the archive?

The Vac file system does already handle some foreign file system
properties, but not everything.

If you don't want to change the Vac file system format, you'd
probably better use vbackup(8) to store an Unix file system
directly on Venti.

Post by s***@icebubble.org
(1) When using Linux's v9fs to mount a file system served by
plan9port's vacfs, the symlinks and device nodes all stat as regular
files. The file size of symlinks reflects the length of the link
target, and the device files have a size of 3 bytes (for type, major,
and minor). This happens whether or not I include version=9p2000.u
or 9p2000.L as mount options.

That's fine, because for both DMSYMLINK and DMDEVICE, the required
information (respectively target and type/major/minor) is written
in the data block.

Using 9p2000.u or 9p2000.l is not likely to change anything, since
vacfs and 9pserve only serve 9P2000 anyway.

Post by s***@icebubble.org
lrw-rw---- 1 root root 6 Mar 10 2005 video ->
-rw-rw---- 1 root root 3 Mar 10 2005 video10
The symbolic links show up as links, but with the null string ("") as
target. The Linux kernel interperets null symlinks to mean "./",
causing all the links to refer to the directory in which they
reside. :( The device files still present as ordinary files with a
size of 3 bytes. Running "readlink" on the symbolic links always
returns the null string.

I'm still not sure where the problem is.

V9fs only seems to display file and directory modes. 9pfuse seems
to display symlink too. But neither of two seems to correctly
interpret the special files from their mode and data.

The vac code looks correct. I just found a bug in the handling of
DMSYMLINK and DMDEVICE, but it's not directly related to your problem.

Post by s***@icebubble.org
(2) plan9port's unvac explicitly skips extraction of files which are
not regular files or directories. So, unvac-ing the archive isn't
possible, either. :(

Unvac is a special case because it extracts files from a file system to
another file system. But some properties may not be settable on
the target file system. Unvac is rather safe concerning the files it
extracts and the properties it sets. Fell free to change its behavior.

Post by s***@icebubble.org
(4) The 9p utility provided by plan9port gets me access to a little
-Lrwxrwxrwx M 0 root root 5 Jan 2 03:16 modem
D-rw-rw---- M 0 root disk 3 Mar 10 2005 hda
You can see the symlink and device bits set in the mode. (The "D"
represents a device node; "d" indicates a directory.)

The mode itself is just a 32-bit integer and is perfectly handled by
both Vac and 9P.

Post by s***@icebubble.org
'modem' 'root' 'root' '' q (0000000000000003 0 ) m 0200000777 at
1325474216 mt 1325474215 l 5 t 0 d 0
'tty3' 'root' 'tty' '' q (0000000000001528 0 ) m 040000660 at
1110476900 mt 1110476900 l 3 t 0 d 0
The targets of symlinks are still showing up as the null string. (I'm
assuming that's what the '' represents.)

No, this is the mid, which is not handled on Unix file systems.
The 9P stat format is described in stat(9p).

The symlink target is in the data blocks.

Post by s***@icebubble.org
(5) Installing Inferno just to access vac'd Linux file systems would
really be tantamount to throwing up my hands and giving up on
plan9port. I'd like use plan9port, if possible, rather than have to
layer a whole new OS on top of Linux+p9p.

I can't see how Inferno could help you. The Inferno vacfs doesn't
even handle any Unix file system property.

[David du Colombier]

Post by s***@icebubble.org
lrw-rw---- 1 root root 6 Mar 10 2005 video ->
-rw-rw---- 1 root root 3 Mar 10 2005 video10
The symbolic links show up as links, but with the null string ("") as
target. The Linux kernel interperets null symlinks to mean "./",
causing all the links to refer to the directory in which they
reside. :( The device files still present as ordinary files with a
size of 3 bytes. Running "readlink" on the symbolic links always
returns the null string.

OK, I found the problem. At least concerning 9pfuse and symlink.

9pfuse read the symlink target from Dir.ext (libc.h), which is
a 9P2000.u extension. But 9P2000.u is not supported anymore.
It should read the symlink target from the data instead.

I've something that work, but it's ugly.

As supposed, other Unix special files don't seem to be currently
supported by 9pfuse at all.

Honestly, I've been using Vac for years and I never
encountered the need to store Unix special files.

If you want to back up a full Unix file system to Venti,
you should simply use vbackup(8).

[s***@icebubble.org]

Post by David du Colombier
OK, I found the problem. At least concerning 9pfuse and symlink.

OK, great. So I'm not going crazy, after all. ;)

Post by David du Colombier
I've something that work, but it's ugly.

Would you care to share?

Post by David du Colombier
As supposed, other Unix special files don't seem to be currently
supported by 9pfuse at all.

I think that would require one or two new command line options to
9pfuse, to pass "-o dev,suid" to fusermount. (fusermount defaults to
"-o nodev,nosuid".) Come to think of it, 9pfuse should probably offer a
general way to pass mount options to fusermount, as a matter of
indirection.

Post by David du Colombier
If you want to back up a full Unix file system to Venti,
you should simply use vbackup(8).

That could work, if I also wanted to archive everything under /tmp (and
all the other temporary files peppered about Unix file systems).

[David du Colombier]

Post by s***@icebubble.org
Would you care to share?

Sure. But beware it's just a quick hack.

diff -r 92a579c6549c src/cmd/9pfuse/main.c
--- a/src/cmd/9pfuse/main.c Mon Dec 12 16:44:50 2011 -0500
+++ b/src/cmd/9pfuse/main.c Mon Jan 02 19:00:00 2012 +0100
@@ -837,6 +837,8 @@
void
fusereadlink(FuseMsg *m)
{
+ int n;
+ uchar *buf;
Dir *d;
CFid *fid;

@@ -852,9 +854,19 @@
replyfuseerrno(m, EINVAL);
return;
}
- replyfuse(m, d->ext, strlen(d->ext));
+ n = d->length;
free(d);
- return;
+ if(n > fusemaxwrite)
+ n = fusemaxwrite;
+ buf = emalloc(n);
+ n = fsread(fid, buf, n);
+ if(n < 0){
+ free(buf);
+ replyfuseerrstr(m);
+ return;
+ }
+ replyfuse(m, buf, n);
+ free(buf);
}

/*
diff -r 92a579c6549c src/cmd/vac/vacfs.c
--- a/src/cmd/vac/vacfs.c Mon Dec 12 16:44:50 2011 -0500
+++ b/src/cmd/vac/vacfs.c Mon Jan 02 19:00:00 2012 +0100
@@ -548,8 +548,6 @@
n = vacdirread(f, buf, off, cnt);
else if(vacfilegetmode(f->file)&ModeDevice)
return vtstrdup("device");
- else if(vacfilegetmode(f->file)&ModeLink)
- return vtstrdup("symbolic link");
else if(vacfilegetmode(f->file)&ModeNamedPipe)
return vtstrdup("named pipe");
else

Then symlinks are working. Now you need to fix the uid/gid
problem and handle devices, named pipes and sockets in 9pserve.

Also, you should apply this patch:

http://codereview.appspot.com/download/issue5500097_1.diff

It fixes two problems:

- fix segfault when the first file is DMSYMLINK or DMDEVICE,
- set the buffer to zero before calling readlink.

Symlinks will not work without this patch.