First function
ReadF2C
takes a filename and channel, reads from file and inputs in channel.
Second function
WriteC2F
takes 2 channels and filename, takes value of each channel and saves the lower value in the output file. I'm sure there is a few syntax errors but i'm new to GO
package main
import (
"fmt"
"bufio"
"os"
"strconv"
)
func main() {
fmt.Println("Hello World!\n\n")
cs1 := make (chan int)
var nameinput string = "input.txt"
readF2C(nameinput,cs1)
cs2 := make (chan int)
cs3 := make (chan int)
cs2 <- 10
cs2 <- 16
cs2 <- 7
cs2 <- 2
cs2 <- 5
cs3 <- 8
cs3 <- 15
cs3 <- 14
cs3 <- 1
cs3 <- 6
var nameoutput string = "output.txt"
writeC2F (nameoutput,cs2,cs3)
}
func readF2C (fn string, ch chan int){
f,err := os.Open(fn)
r := bufio.NewReader(f)
for err != nil { // not end of file
fmt.Println(r.ReadString('\n'))
ch <- r.ReadString('\n')
}
if err != nil {
fmt.Println(r.ReadString('\n'))
ch <- -1
}
}
func writeC2F(fn string, // output text file
ch1 chan int, // first input channel
ch2 chan int){
var j int = 0
var channel1temp int
var channel2temp int
f,_ := os.Create(fn)
w := bufio.NewWriter(f)
channel1temp = <-ch1
channel2temp = <-ch2
for j := 1; j <= 5; j++ {
if (channel2temp < channel1temp){
n4, err := w.WriteString(strconv.Itoa(channel1temp))
} else{
n4, err := w.WriteString(strconv.Itoa(channel2temp))
}
w.flush()
}
}
This is the error messages I get:
prog.go:38: multiple-value r.ReadString() in single-value context
prog.go:65: w.flush undefined (cannot refer to unexported field or method bufio.(*Writer)."".flush)
There are multiple errors:
1)
Unlike C, Go enforces you to have your curly braces directly after your statements. So for an if case (and the same for func), instead of doing it like this:
if (channel2temp < channel1temp)
{
use this
if channel2temp < channel1temp {
2)
There is no while in Go. Use for
for {
...
}
or
for channel1temp != null || channel2temp != null {
...
}
3)
Usage of non-declared variables. Often easy to fix by making a short variable declaration the first time you initialize the variable. So instead of:
r = bufio.NewReader(file)
use
r := bufio.NewReader(file)
4)
Trying to a assign multi-value return into a single variable. If a function returns two values and you only need one, the variable you don't want can be discarded by assigning it to _. So instead of:
file := os.Open(fn)
use
file, _ := os.Open(fn)
but best practice would be to catch that error:
file, err := os.Open(fn)
if err != nil {
panic(err)
}
There are more errors on top of this, but maybe it will get you started.
I also suggest reading Effective Go since it will explain many of the things I've just mentioned.
Edit:
And there are help online for sure. It might be a new language, but the online material is really useful. Below is a few that I used when learning Go:
Effective Go: Good document on how to write idiomatic Go code
The Go programming language Tour: Online tour of Go with interactive examples.
Go By Example: Interactive examples of Go programs, starting with Hello World.
Go Specification: Surprisingly readable for being a specification. Maybe not a start point, but very useful.
Related
I have a string mixed with characters and numerals, but i want to increment the last character which happens to be a number, here is what i have, it works, but once i reach 10 rune goes to black since 10 decimal is zero, is there a better way to do this?
package main
import (
"fmt"
)
func main() {
str := "version-1.1.0-8"
rStr := []rune(str)
last := rStr[len(rStr)-1]
rStr[len(rStr)-1] = last + 1
}
So this works for str := "version-1.1.0-8" = version-1.1.0-9
str := version-1.1.0-9 = version-1.1.0-
I understand why it is happening, but I dont know how to fix it
Your intention is to increment the number represented by the last rune, so you should do that: parse out that number, increment it as a number, and "re-encode" it into string.
You can't operate on a single rune, as once the number reaches 10, it can only be represented using 2 runes. Another issue is if the last number is 19, incrementing it needs to alter the previous rune (and not adding a new rune).
Parsing the numbers and re-encoding though is much easier than one might think.
You can take advantage of the fmt package's fmt.Sscanf() and fmt.Sprintf() functions. Parsing and re-encoding is just a single function call.
Let's wrap this functionality into a function:
const format = "version-%d.%d.%d-%d"
func incLast(s string) (string, error) {
var a, b, c, d int
if _, err := fmt.Sscanf(s, format, &a, &b, &c, &d); err != nil {
return "", err
}
d++
return fmt.Sprintf(format, a, b, c, d), nil
}
Testing it:
s := "version-1.1.0-8"
for i := 0; i < 13; i++ {
var err error
if s, err = incLast(s); err != nil {
panic(err)
}
fmt.Println(s)
}
Output (try it on the Go Playground):
version-1.1.0-9
version-1.1.0-10
version-1.1.0-11
version-1.1.0-12
version-1.1.0-13
version-1.1.0-14
version-1.1.0-15
version-1.1.0-16
version-1.1.0-17
version-1.1.0-18
version-1.1.0-19
version-1.1.0-20
version-1.1.0-21
Another option would be to just parse and re-encode the last part, and not the complete version text. This is how it would look like:
func incLast2(s string) (string, error) {
i := strings.LastIndexByte(s, '-')
if i < 0 {
return "", fmt.Errorf("invalid input")
}
d, err := strconv.Atoi(s[i+1:])
if err != nil {
return "", err
}
d++
return s[:i+1] + strconv.Itoa(d), nil
}
Testing and output is the same. Try this one on the Go Playground.
I'm creating a simple channel that takes string values. But apparently I'm pushing each letter in the string instead of the whole string in each loop.
I'm probably missing something very fundamental. What am I doing wrong ?
https://play.golang.org/p/-6E-f7ALbD
Code:
func doStuff(s string, ch chan string) {
ch <- s
}
func main() {
c := make(chan string)
loops := [5]int{1, 2, 3, 4, 5}
for i := 0; i < len(loops); i++ {
go doStuff("helloooo", c)
}
results := <-c
fmt.Println("channel size = ", len(results))
// print the items in channel
for _, r := range results {
fmt.Println(string(r))
}
}
Your code sends strings on the channel properly:
func doStuff(s string, ch chan string){
ch <- s
}
The problem is at the receiver side:
results := <- c
fmt.Println("channel size = ", len(results))
// print the items in channel
for _,r := range results {
fmt.Println(string(r))
}
results will be a single value received from the channel (the first value sent on it). And you print the length of this string.
Then you loop over this string (results) using a for range which loops over its runes, and you print those.
What you want is loop over the values of the channel:
// print the items in channel
for s := range c {
fmt.Println(s)
}
This when run will result in a runtime panic:
fatal error: all goroutines are asleep - deadlock!
Because you never close the channel, and a for range on a channel runs until the channel is closed. So you have to close the channel sometime.
For example let's wait 1 second, then close it:
go func() {
time.Sleep(time.Second)
close(c)
}()
This way your app will run and quit after 1 second. Try it on the Go Playground.
Another, nicer solution is to use sync.WaitGroup: this waits until all goroutines are done doing their work (sending a value on the channel), then it closes the channel (so there is no unnecessary wait / delay).
var wg = sync.WaitGroup{}
func doStuff(s string, ch chan string) {
ch <- s
wg.Done()
}
// And in main():
for i := 0; i < len(loops); i++ {
wg.Add(1)
go doStuff("helloooo", c)
}
go func() {
wg.Wait()
close(c)
}()
Try this one on the Go Playground.
Notes:
To repeat something 5 times, you don't need that ugly loops array. Simply do:
for i := 0; i < 5; i++ {
// Do something
}
The reason you are getting back the letters instead of string is that you are assigning the channel result to a variable and iterating over the result of the channel assigned to this variable which in your case is a string, and in Go you can iterate over a string with a for range loop to get the runes.
You can simply print the channel without to iterate over the channel result.
package main
import (
"fmt"
)
func doStuff(s string, ch chan string){
ch <- s
}
func main() {
c := make(chan string)
loops := [5]int{1,2,3,4,5}
for i := 0; i < len(loops) ; i++ {
go doStuff("helloooo", c)
}
results := <- c
fmt.Println("channel size = ", len(results))
fmt.Println(results) // will print helloooo
}
I would need help to understand why the following code does not work. I am building a pipeline and trying to have a step that synchronize values from two source channels. My source/producer code looks something like below (in my real code i read the text from a file). The sources are sorted but are values are not guaranteed to be in both sources.
func Source() <-chan int{
out := make(chan int, 5)
go func() {
defer reader.Close()
out <- 1
out <- 2
out <- 3
out <- 4
out <- 5
out <- 7
close(out)
}()
return out
}
and the synchronization code looks like this:
func Sync(a, b <-chan int) <-chan int {
out := make(chan int)
go func() {
av, ak:= <-a
bv, bk:= <-b
for ak || bk {
if !ak || av < bv {
out <- bv
bv, bk = <-b
continue
}
if !bk|| bv > av {
out <- av
av, ak = <-a
continue
}
out <- av
av, ak = <-a
bv, bk = <-b
}
close(out)
}()
return out
}
and my program looks something like this:
func main() {
os := Source()
ns := Source()
for val := range Sync(ns, os) {
fmt.Printf("[SYNCED] %v \n", val)
}
}
The expected behaviour is that my both sources buffer values into the channel and my sync first reads value from the first source. Then from the second. Compare them and if they are equal continues to the next in the both channels. If the differ we will send out the value that is behind and replace it with a new one and make the same comparison again.
What happends is that it looks like the sync code is run several times for the values and I will get things like [SYNCED] 1 several times. Why?
Please help me get this fixed!
Regarding http://play.golang.org/p/uhd3EWrwEo and http://play.golang.org/p/Dqq7-cPaFq -
Actually, the code for ints will fail with the similar test case as well:
os := Source([]int{1, 2, 3})
ns := Source([]int{1, 3, 4})
puts the ints version to infinite loop.
This happens because when !aok || avalue > bvalue is checked, it does not take into account that if aok is true (some elements still are in a) and bok is false (no more elements in b), then avalue > "" is always true. So it tries to take another item from b (which is empty) and goes to infinite loop. Fixed code: http://play.golang.org/p/vYhuOZxRMl
I want to understand a bit more about how synchronisation of threads works in go. Below here I've have a functioning version of my program which uses a done channel for syncronization.
package main
import (
. "fmt"
"runtime"
)
func Goroutine1(i_chan chan int, done chan bool) {
for x := 0; x < 1000000; x++ {
i := <-i_chan
i++
i_chan <- i
}
done <- true
}
func Goroutine2(i_chan chan int, done chan bool) {
for x := 0; x < 1000000; x++ {
i := <-i_chan
i--
i_chan <- i
}
done <- true
}
func main() {
i_chan := make(chan int, 1)
done := make(chan bool, 2)
i_chan <- 0
runtime.GOMAXPROCS(runtime.NumCPU())
go Goroutine1(i_chan, done)
go Goroutine2(i_chan)
<-done
<-done
Printf("This is the value of i:%d\n", <-i_chan)
}
However when I try to run it with out any synchronisation. Using a wait statement and no channel to specify when it's done so no synchronisation.
const MAX = 1000000
func Goroutine1(i_chan chan int) {
for x := 0; x < MAX-23; x++ {
i := <-i_chan
i++
i_chan <- i
}
}
func main() {
i_chan := make(chan int, 1)
i_chan <- 0
runtime.GOMAXPROCS(runtime.NumCPU())
go Goroutine1(i_chan)
go Goroutine2(i_chan)
time.Sleep(100 * time.Millisecond)
Printf("This is the value of i:%d\n", <-i_chan)
}
It'll print out the wrong value of i. If you extend the wait for let say 1 sec it'll finish and print out the correct statement. I kind of understand that it has something with both thread not being finished before you print what's on the i_chan I'm just a bit curious about how this works.
Note that your first example would deadlock, since it never calls GoRoutine2 (the OP since edited the question).
If it calls GoRoutine2, then the expected i value is indeed 0.
Without synchronization, (as in this example), there is no guarantee that the main() doesn't exit before the completion of Goroutine1() and Goroutine2().
For a 1000000 loop, a 1 millisecond wait seems enough, but again, no guarantee.
func main() {
i_chan := make(chan int, 1)
i_chan <- 0
runtime.GOMAXPROCS(runtime.NumCPU())
go Goroutine2(i_chan)
go Goroutine1(i_chan)
time.Sleep(1 * time.Millisecond)
Printf("This is the value of i:%d\n", <-i_chan)
}
see more at "How to Wait for All Goroutines to Finish Executing Before Continuing", where the canonical way is to use the sync package’s WaitGroup structure, as in this runnable example.
Like [a-zA-Z0-9] string:
na1dopW129T0anN28udaZ
or hexadecimal string:
8c6f78ac23b4a7b8c0182d
By long I mean 2K and more characters.
This does about 200MBps on my box. There's obvious room for improvement.
type randomDataMaker struct {
src rand.Source
}
func (r *randomDataMaker) Read(p []byte) (n int, err error) {
for i := range p {
p[i] = byte(r.src.Int63() & 0xff)
}
return len(p), nil
}
You'd just use io.CopyN to produce the string you want. Obviously you could adjust the character set on the way in or whatever.
The nice thing about this model is that it's just an io.Reader so you can use it making anything.
Test is below:
func BenchmarkRandomDataMaker(b *testing.B) {
randomSrc := randomDataMaker{rand.NewSource(1028890720402726901)}
for i := 0; i < b.N; i++ {
b.SetBytes(int64(i))
_, err := io.CopyN(ioutil.Discard, &randomSrc, int64(i))
if err != nil {
b.Fatalf("Error copying at %v: %v", i, err)
}
}
}
On one core of my 2.2GHz i7:
BenchmarkRandomDataMaker 50000 246512 ns/op 202.83 MB/s
EDIT
Since I wrote the benchmark, I figured I'd do the obvious improvement thing (call out to the random less frequently). With 1/8 the calls to rand, it runs about 4x faster, though it's a big uglier:
New version:
func (r *randomDataMaker) Read(p []byte) (n int, err error) {
todo := len(p)
offset := 0
for {
val := int64(r.src.Int63())
for i := 0; i < 8; i++ {
p[offset] = byte(val & 0xff)
todo--
if todo == 0 {
return len(p), nil
}
offset++
val >>= 8
}
}
panic("unreachable")
}
New benchmark:
BenchmarkRandomDataMaker 200000 251148 ns/op 796.34 MB/s
EDIT 2
Took out the masking in the cast to byte since it was redundant. Got a good deal faster:
BenchmarkRandomDataMaker 200000 231843 ns/op 862.64 MB/s
(this is so much easier than real work sigh)
EDIT 3
This came up in irc today, so I released a library. Also, my actual benchmark tool, while useful for relative speed, isn't sufficiently accurate in its reporting.
I created randbo that you can reuse to produce random streams wherever you may need them.
You can use the Go package uniuri to generate random strings (or view the source code to see how they're doing it). You'll want to use:
func NewLen(length int) string
NewLen returns a new random string of the provided length, consisting of standard characters.
Or, to specify the set of characters used:
func NewLenChars(length int, chars []byte) string
This is actually a little biased towards the first 8 characters in the set (since 255 is not a multiple of len(alphanum)), but this will get you most of the way there.
import (
"crypto/rand"
)
func randString(n int) string {
const alphanum = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
var bytes = make([]byte, n)
rand.Read(bytes)
for i, b := range bytes {
bytes[i] = alphanum[b % byte(len(alphanum))]
}
return string(bytes)
}
If you want to generate cryptographically secure random string, I recommend you to take a look at this page. Here is a helper function that reads n random bytes from the source of randomness of your OS and then use these bytes to base64encode it. Note that the string length would be bigger than n because of base64.
package main
import(
"crypto/rand"
"encoding/base64"
"fmt"
)
func GenerateRandomBytes(n int) ([]byte, error) {
b := make([]byte, n)
_, err := rand.Read(b)
if err != nil {
return nil, err
}
return b, nil
}
func GenerateRandomString(s int) (string, error) {
b, err := GenerateRandomBytes(s)
return base64.URLEncoding.EncodeToString(b), err
}
func main() {
token, _ := GenerateRandomString(32)
fmt.Println(token)
}
Here Evan Shaw's answer re-worked without the bias towards the first 8 characters of the string. Note that it uses lots of expensive big.Int operations so probably isn't that quick! The answer is crypto strong though.
It uses rand.Int to make an integer of exactly the right size len(alphanum) ** n, then does what is effectively a base conversion into base len(alphanum).
There is almost certainly a better algorithm for this which would involve keeping a much smaller remainder and adding random bytes to it as necessary. This would get rid of the expensive long integer arithmetic.
import (
"crypto/rand"
"fmt"
"math/big"
)
func randString(n int) string {
const alphanum = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
symbols := big.NewInt(int64(len(alphanum)))
states := big.NewInt(0)
states.Exp(symbols, big.NewInt(int64(n)), nil)
r, err := rand.Int(rand.Reader, states)
if err != nil {
panic(err)
}
var bytes = make([]byte, n)
r2 := big.NewInt(0)
symbol := big.NewInt(0)
for i := range bytes {
r2.DivMod(r, symbols, symbol)
r, r2 = r2, r
bytes[i] = alphanum[symbol.Int64()]
}
return string(bytes)
}