[9fans] upas locking

[Francisco J Ballesteros]

Hi,

we found recently that we had a problem with /mail/tmp/L.mbox
used by upasfs to lock.

Looking into, we found that in upas/common/libsys.c the
function generating the lock file name returns always "L.mbox"
but not .../L.<mbox file name> as it seems it should.

Also, the function doing the actual lock hides the error string.

I copied our changed file to /n/sources/contrib/nemo/libsys.c

Didn't create a patch yet because I'm not sure it's the right
change for this problem.

[erik quanstrom]

Post by Francisco J Ballesteros
Looking into, we found that in upas/common/libsys.c the
function generating the lock file name returns always "L.mbox"
but not .../L.<mbox file name> as it seems it should.

i think the comment above the function is important
"we use one lock per directory". it's been a long time since
i worked on nupas, so i don't recall all the details, but
iirc, imap4d relies on having one lock per directory.

of course, nupas takes pressure off the directory lock
by using mdir format, which doesn't require a lock. the
indicies are lock files themselves.

under what circumstaces did you find a problem?

- erik

[Francisco J Ballesteros]

Yes, I noticed the comment, but I thought it was wrong.

The problem was that we had quite a number of upas/fs's and scanmails
running and it was because one of them had a problem and kept a
/mail/tmp/L.mbox file locked. All other ones kept waiting for that
(although all of them were using different "mboxes" created by the
piped script at /mail/tmp/$pid.blah.blah

Making this change made problems go away, and I didn't notice any
problem in imap so far. I guess I'll have to reread the imap source to see
if it really requires one lock per dir instead of one per mbox, but even so,
I think imap should use a different lock for that, it's upas/fs the one I'm
talking about.

thanks for the hint

Post by erik quanstrom
i think the comment above the function is important
"we use one lock per directory". it's been a long time since
i worked on nupas, so i don't recall all the details, but
iirc, imap4d relies on having one lock per directory.

[erik quanstrom]

Post by Francisco J Ballesteros
The problem was that we had quite a number of upas/fs's and scanmails
running and it was because one of them had a problem and kept a
/mail/tmp/L.mbox file locked. All other ones kept waiting for that
(although all of them were using different "mboxes" created by the
piped script at /mail/tmp/$pid.blah.blah

i'm confused. doesn't this just make the problem smaller rather
than fixing it? i would think if scanmail has the same issue, then
mail will still go undelivered, right?

we don't run scanmail. since scanmail relies on building rules, it
needs constant maintence. and i didn't want to be on the hook
for that.

instead, i use mechanical defensive measures in smtpd and
validatesender. the most effective measure is enforcing the
rfc2821 rule that the ehlo string must look up in dns.
to appease imap4 clients, there's a flag to force auth
for clients ehlo'ing incorrectly.

spamhaus does a good job cleaning up what remains.

- erik

http://www.quanstro.net/magic/man2html/8/smtp

(yes, that's ugly!)

[Fco. J. Ballesteros]

Yes, perhaps we should stop using that.
Nevertheless, the problem I was trying to address is that
everyone is sharing /mail/tmp (see the scripts in /mail/lib,
although I'm sure you know them from memory :) ), and thus
if one has one problem, it can lock everyone else using /mail/tmp.

I'll take a look now to the countermeasures you suggest.
I think we are applying a few other measures, including
validatesender, but I'll take a look.

[Charles Forsyth]

Post by Fco. J. Ballesteros
if one has one problem, it can lock everyone else using /mail/tmp.

I use a private ramfs, which is also faster.

[l***@proxima.alt.za]

Post by Charles Forsyth

Post by Fco. J. Ballesteros
if one has one problem, it can lock everyone else using /mail/tmp.

I use a private ramfs, which is also faster.

Obvious, when one points it out!

++L

[Francisco J Ballesteros]

yes, except that the whole thing
got worse because one of our servers
had a lot of memory used due to
those and we put back the bind for
tmp.

Post by Charles Forsyth

Post by Fco. J. Ballesteros
if one has one problem, it can lock everyone else using /mail/tmp.

I use a private ramfs, which is also faster.

[erik quanstrom]

Post by Francisco J Ballesteros
yes, except that the whole thing
got worse because one of our servers
had a lot of memory used due to
those and we put back the bind for
tmp.

ramfs -l 10m

(the 9atom version of ramfs does proper accounting to
make this work.)

also, if you're using anything like mdir, you can deliver
the mail and delete it automaticly later.

- erik