unexpected EOF with fmt.Scanner - string

Question

If I want to scan through a string, I can do this:
package main
import (
"fmt"
"strings"
)
func main() {
r := strings.NewReader("west north east")
for {
var s string
_, e := fmt.Fscan(r, &s)
fmt.Printf("%q %v\n", s, e)
if e != nil { break }
}
}
Result:
"west" <nil>
"north" <nil>
"east" <nil>
"" EOF
I recently discovered fmt.Scanner [1], so I thought I would try to implement
it. I came up with this:
package main
import (
"fmt"
"strings"
)
type comma struct { tok string }
func (c *comma) Scan(state fmt.ScanState, verb rune) error {
tok, err := state.Token(false, func(r rune) bool {
return r != ','
})
if err != nil {
return err
}
if _, _, err := state.ReadRune(); err != nil {
if len(tok) == 0 {
return err
}
}
c.tok = string(tok)
return nil
}
func main() {
r := strings.NewReader("west,north,east")
for {
var c comma
_, e := fmt.Fscan(r, &c)
fmt.Printf("%q %v\n", c.tok, e)
if e != nil { break }
}
}
Result:
"west" <nil>
"north" <nil>
"east" <nil>
"" unexpected EOF
So the result is pretty close, but what bothers me is the unexpected EOF. Is
it possible to just get a regular EOF with a custom fmt.Scanner? Am I doing
something wrong here, or is this a bug?
https://golang.org/pkg/fmt#Scanner

Answer 1

Thanks to Ian Lance Taylor on the golang-nuts list, he suggested to panic
the error instead of return. In the Go code, Fscan calls a function
doScan, which in turn calls a function errorHandler [1]. This last function
uses recover to turn any panic into regular error. This program gives
idential output to my original example:
package main
import (
"fmt"
"strings"
)
type comma struct { tok string }
func (c *comma) Scan(state fmt.ScanState, verb rune) error {
tok, err := state.Token(false, func(r rune) bool {
return r != ','
})
if err != nil { return err }
if _, _, err := state.ReadRune(); err != nil {
if len(tok) == 0 {
panic(err)
}
}
c.tok = string(tok)
return nil
}
func main() {
r := strings.NewReader("west,north,east")
for {
var c comma
_, err := fmt.Fscan(r, &c)
fmt.Printf("%q %v\n", c.tok, err)
if err != nil { break }
}
}
https://github.com/golang/go/blob/go1.16.4/src/fmt/scan.go#L1056-L1067