[9fans] all you yacc experts

[ron minnich]

Go is pretty solid on 386 and I'm slowly puzzling my way through NIX support.

One thing that stands in the way of full native build is the bison issue.

If somebody wants to look at enhancing yacc so that the extra bison
bits can be supported, that would probably do the trick. I have no
idea of the level of effort, I have not looked.

We could run bison under linuxemu; I don't think this approach is as
good because it still leaves us a bit dependent on some outside force
for bison binaries. But that might be a good stopgap.

I'm looking forward to Go 1 because I'm pretty sure that most of what
we've had to do for this version of Go will go away :-)

I'm still amused that the best way to write portable C is to just ship
a reasonable C compiler with Go and use gcc to build that C compiler,
and then compile your portable C with it :-)

Thanks

ron

[Steve Simon]

I believe go can be tweeked a little build correctly without
bison, but some of the error messages gc generates will be less
helpful.

for me the best solution would be for yacc to be modified
to produce the error message tables directly, rather than
mimicing bison - producing a debug file withch can be post
processed with awk.

-Steve

[Bakul Shah]

Post by ron minnich
If somebody wants to look at enhancing yacc so that the extra bison
bits can be supported, that would probably do the trick. I have no
idea of the level of effort, I have not looked.

After some googling I see that src/cmd/gc/bisonerrors was
added by Russ in an evening to improve go error messages.

Clinton Jeffery in his "Generating LR syntax error messages
from examples" paper (http://www.cs.nmsu.edu/~jeffery/merr.pdf
-- not sure if this is the same paper as the one in TOPLAS)
says his Merr program works with Berkeley yacc, AT&T yacc &
bison. He also refers to a modified byacc called iyacc.

Might it be worth looking Merr or iyacc? Porting bison to
plan9 seems like a hugh punishment for a quick hack:-)
Implementing Jeffery's directly in yacc might benefit other
parsers as well.

[ron minnich]

Might it be worth looking Merr or iyacc?  Porting bison to
plan9 seems like a hugh punishment for a quick hack:-)
Implementing Jeffery's directly in yacc might benefit other
parsers as well.

If it's worth a look then I hope someone looks. I'm more trying to push that
than having this discussion :-)

I think porting bison is the wrong way to go, personally.

ron

[Bakul Shah]

Post by ron minnich

Might it be worth looking Merr or iyacc? Porting bison to
plan9 seems like a hugh punishment for a quick hack:-)
Implementing Jeffery's directly in yacc might benefit other
parsers as well.

If it's worth a look then I hope someone looks. I'm more trying to push that
than having this discussion :-)

I will take a look but don't hold your breath; my plate is
already full.

One further issue is whether go has any other bison
dependencies.

Post by ron minnich
I think porting bison is the wrong way to go, personally.

They lost me at configure. Why should a parser generator need
on anything more than string.h, stdio,h and may be stdlib.h is
beyond me. All it does is manipulate lists, sets and vectors!

[Steve Simon]

Post by Bakul Shah
One further issue is whether go has any other bison
dependencies.

as off march (when I last played with a port) it didn't,
there where a few bits of bison syntax which are different
in yacc but these could be papered over whith a few lines of
sed - ur yacc could even be taught this alternative syntax I suspose.

-Steve

[Lucio De Re]

Post by Bakul Shah
One further issue is whether go has any other bison
dependencies.

None as serious as the error message generation. But your research is
invaluable, let's hope it leads in the right direction.

++L

[Bruce Ellis]

I'm modifying yacc for other purposes, and yes I know how it works.

If bison is not too vomituos I will try and help.

brucee

Post by Bakul Shah
One further issue is whether go has any other bison
dependencies.

None as serious as the error message generation.  But your research is
invaluable, let's hope it leads in the right direction.
++L

[Lucio De Re]

Post by Bruce Ellis
I'm modifying yacc for other purposes, and yes I know how it works.

Nobody could doubt that :-)

Post by Bruce Ellis
If bison is not too vomituos I will try and help.

Don't overlook Bakul's suggestions, they may have much merit. A quick
glance at my previous efforts suggests having to fight with the wide
character functions in Unix. I seem to recall that fgb has done some
work in that direction?

++L

[Lucio De Re]

Post by ron minnich
I think porting bison is the wrong way to go, personally.

Having tried it, I can only agree. But I'm going to give it another
go, if only to remind myself why I didn't complete the task the first
time 'round.

++L

[Iruatã Souza]

Post by Bakul Shah

Post by ron minnich
If somebody wants to look at enhancing yacc so that the extra bison
bits can be supported, that would probably do the trick. I have no
idea of the level of effort, I have not looked.

After some googling I see that src/cmd/gc/bisonerrors was
added by Russ in an evening to improve go error messages.
Clinton Jeffery in his "Generating LR syntax error messages
from examples" paper (http://www.cs.nmsu.edu/~jeffery/merr.pdf
-- not sure if this is the same paper as the one in TOPLAS)
says his Merr program works with Berkeley yacc, AT&T yacc &
bison.  He also refers to a modified byacc called iyacc.
Might it be worth looking Merr or iyacc?  Porting bison to
plan9 seems like a hugh punishment for a quick hack:-)
Implementing Jeffery's directly in yacc might benefit other
parsers as well.

Sorry if it obvious, but Russ posted about the improvement on Go
errors: http://research.swtch.com/2010/01/generating-good-syntax-errors.html.

iru

[Bakul Shah]

Post by Bakul Shah

Post by ron minnich
If somebody wants to look at enhancing yacc so that the extra bison
bits can be supported, that would probably do the trick. I have no
idea of the level of effort, I have not looked.

After some googling I see that src/cmd/gc/bisonerrors was
added by Russ in an evening to improve go error messages.
Clinton Jeffery in his "Generating LR syntax error messages
from examples" paper (http://www.cs.nmsu.edu/~jeffery/merr.pdf
-- not sure if this is the same paper as the one in TOPLAS)
says his Merr program works with Berkeley yacc, AT&T yacc &
bison. He also refers to a modified byacc called iyacc.
Might it be worth looking Merr or iyacc? Porting bison to
plan9 seems like a hugh punishment for a quick hack:-)
Implementing Jeffery's directly in yacc might benefit other
parsers as well.

Reading Merr code was useful but it is written in Icon so....

After staring at go.y, bisonerrors, go.errors, bison's
y.output, and generated yerr.h for a while I think I
understand what is going on. I believe everything needed is
already in plan9's y.output -- no need to change yacc. It may
be easier to just make bisonerrors handle this than convert
plan9's y.output to bison compatible y.output. Of course,
things may look much bleaker in the morning but this is what I
think now!

Given that an array of <state, terminal, message> is generated
from y.output & go.errors, it is may be worth integrating this
logic in yacc. But that is for another day.

Ideally something like go.errors is interactively generated
from actual errors (sort of like how a spellchecker works --
you are presented with an error + its context and you indicate
the error message &/or fix). This is then fed back to yacc
along with the grammar.

[Bruce Ellis]

If all the information is available in y.output then surely modifying
yacc will cut out the middle man.

On my list.

brucee

Post by Bakul Shah

Post by Bakul Shah

Post by ron minnich
If somebody wants to look at enhancing yacc so that the extra bison
bits can be supported, that would probably do the trick. I have no
idea of the level of effort, I have not looked.

After some googling I see that src/cmd/gc/bisonerrors was
added by Russ in an evening to improve go error messages.
Clinton Jeffery in his "Generating LR syntax error messages
from examples" paper (http://www.cs.nmsu.edu/~jeffery/merr.pdf
-- not sure if this is the same paper as the one in TOPLAS)
says his Merr program works with Berkeley yacc, AT&T yacc &
bison.  He also refers to a modified byacc called iyacc.
Might it be worth looking Merr or iyacc?  Porting bison to
plan9 seems like a hugh punishment for a quick hack:-)
Implementing Jeffery's directly in yacc might benefit other
parsers as well.

Reading Merr code was useful but it is written in Icon so....
After staring at go.y, bisonerrors, go.errors, bison's
y.output, and generated yerr.h for a while I think I
understand what is going on.  I believe everything needed is
already in plan9's y.output -- no need to change yacc.  It may
be easier to just make bisonerrors handle this than convert
plan9's y.output to bison compatible y.output.  Of course,
things may look much bleaker in the morning but this is what I
think now!
Given that an array of <state, terminal, message> is generated
from y.output & go.errors, it is may be worth integrating this
logic in yacc. But that is for another day.
Ideally something like go.errors is interactively generated
from actual errors (sort of like how a spellchecker works --
you are presented with an error + its context and you indicate
the error message &/or fix). This is then fed back to yacc
along with the grammar.

[Lucio De Re]

Post by Bruce Ellis
If all the information is available in y.output then surely modifying
yacc will cut out the middle man.

That's not quite how I remember it, I'm sure that there was no trivial
way to alter the output from yacc to conform to the needs of Russ'
error messages. That said, even if y.output does include the
essentials, it would have been extremly insightful of Russ to design a
message format that would be worth including in yacc itself. It
certainly isn't the impression I got from looking at the available
documentation and sources,

I don't want to rain on anybody's parade, but I think a middle man
provides more flexibility and allows the developers on the yacc side
to focus on generating the type of output that makes the most sense,
rather than what the Go toolchain requires.

Post by Bruce Ellis
On my list.

Ron isn't the only developer that will benefit from your efforts, I'd
like to show my appreciation as well.

++L

[Bruce Ellis]

my motivation is that the limbo version of yacc is out of date
severely (doesn't cope with cc.y) and it seems like a good time to
have a 5c in limbo that copes with all and sundry arm variants - two
more were released this week - with loadable arch modules. and to
write shorter sentences.

brucee

Post by Lucio De Re

Post by Bruce Ellis
If all the information is available in y.output then surely modifying
yacc will cut out the middle man.

That's not quite how I remember it, I'm sure that there was no trivial
way to alter the output from yacc to conform to the needs of Russ'
error messages.  That said, even if y.output does include the
essentials, it would have been extremly insightful of Russ to design a
message format that would be worth including in yacc itself.  It
certainly isn't the impression I got from looking at the available
documentation and sources,
I don't want to rain on anybody's parade, but I think a middle man
provides more flexibility and allows the developers on the yacc side
to focus on generating the type of output that makes the most sense,
rather than what the Go toolchain requires.

Post by Bruce Ellis
On my list.

Ron isn't the only developer that will benefit from your efforts, I'd
like to show my appreciation as well.
++L

[Steve Simon]

Perhaps I'am way off base but surely the neatest solution would be to
modify yacc to produce a nice clean message format, modify the gc
to use that format.

The reason for not using just the same format blindly is it was designed
with the structure of gc in mind rather than a clean output format.

Who knows, by delving into yacc perhaps more or even better error
status can be extracted.

Then we just need to convince russ/the go authors to include plan9's yacc
as part of the go distribution, it wouldn't be unprecendented as go already
includes 8c, 8l and friends.

-Steve

[Bruce Ellis]

that's what i said ...

If all the information is available in y.output then surely modifying
yacc will cut out the middle man.

brucee

Post by Steve Simon
Perhaps I'am way off base but surely the neatest solution would be to
modify yacc to produce a nice clean message format, modify the gc
to use that format.
The reason for not using just the same format blindly is it was designed
with the structure of gc in mind rather than a clean output format.
Who knows, by delving into yacc perhaps more or even better error
status can be extracted.
Then we just need to convince russ/the go authors to include plan9's yacc
as part of the go distribution, it wouldn't be unprecendented as go already
includes 8c, 8l and friends.
-Steve

[ron minnich]

The bisonerrors script is 124 lines. This email thread is now 724
lines. I figure when the number of lines of this thread is 10x the
size of the bisonerrors script it would be nice to replace the endless
discussion with some code :-)

ron

[Scato Logic]

Let me get this straight. The goal is to get a successful full native build
of go under plan9. Currently, the build requires bison which no one wants
to port to plan9 but can be run via linuxemu. Brucee has offered to modify
his limbo version of yacc to make things work but that yacc will then need
to be run via the Inferno emu. Is this a comedy skit?

[Bruce Ellis]

Reread what I posted. I said that I will modify yacc for different
reasons and gave my reasons for doing so. I NEVER said RUN INFERNO.

Yes ron, what a lot of noise. And I'll SHOUT whenever I think it might
help, Mr aptly named Scatomancer.

brucee

Post by Scato Logic
Let me get this straight. The goal is to get a successful full native build
of go under plan9. Currently, the build requires bison which no one wants to
port to plan9 but can be run via linuxemu. Brucee has offered to modify his
limbo version of yacc to make things work but that yacc will then need to be
run via the Inferno emu. Is this a comedy skit?

[Anthony Martin]

Attached is a modified version of p9p yacc that
supports the Go grammar. I'll be sending a
version of Plan 9 yacc later today.

The following is a description of the changes.

1. The %error-verbose directive is ignored.

2. A description of the final grammar is
printed before the state descriptions
in y.output.

3. The 'x' format for character literals is
now used instead of prefixing with a space.

4. The YYEMPTY define is now used to clear
the lookahead token (instead of an explicit
negative one).

5. Make yychar and yystate globals so they
can be inspected by external code.

5. Support C++ style // comments in actions.

6. Add a usage message.

7. Fix a few uses of sprint and strcpy.


I've also sent out a changeset to the Go
development list which adds support for
using Plan 9 yacc to generate the special
errors.

One tiny nit is that Plan 9 uses the name
yytoknames for debugging where Bison uses
yytname. I've just used sed for this.

Any questions?
Anthony

[David Leimbach]

Pretty cool!

Post by Anthony Martin
Attached is a modified version of p9p yacc that
supports the Go grammar. I'll be sending a
version of Plan 9 yacc later today.
The following is a description of the changes.
1. The %error-verbose directive is ignored.
2. A description of the final grammar is
printed before the state descriptions
in y.output.
3. The 'x' format for character literals is
now used instead of prefixing with a space.
4. The YYEMPTY define is now used to clear
the lookahead token (instead of an explicit
negative one).
5. Make yychar and yystate globals so they
can be inspected by external code.
5. Support C++ style // comments in actions.
6. Add a usage message.
7. Fix a few uses of sprint and strcpy.
I've also sent out a changeset to the Go
development list which adds support for
using Plan 9 yacc to generate the special
errors.
One tiny nit is that Plan 9 uses the name
yytoknames for debugging where Bison uses
yytname. I've just used sed for this.
Any questions?
Anthony