I'm trying to reimplement a program it did in C a few years ago in Go
The program should read a "record"-like structured binary file and do something with the record (what is done with the records itself is not relevant for this question)
Such a datafile consists of many records where each record has the following definition:
REC_LEN U2 // length of record after header
REC_TYPE U1 //a type
REC_SUB U1 //a subtype
REC_LEN x U1 //"payload"
My problem now is how to specify that variable length byte[] in a struct in Go?
My plan was to use binary.Read to read the records
Here's what I've tried so far in Go:
type Record struct {
rec_len uint16
rec_type uint8
rec_sub uint8
data [rec_len]byte
}
Unfortunatelly it seems I can't reference a field of a struct within the same struct as I get the following error:
xxxx.go:15: undefined: rec_len
xxxx.go:15: invalid array bound rec_len
I'd appreciate any ideas pointing me in the right direction
Thanks
KR
You can read the record as follows:
var rec Record
// Slurp up the fixed sized header.
var buf [4]byte
_, err := io.ReadFull(r, buf[:])
if err != nil {
// handle error
}
rec.rec_len = binary.BigEndian.Uint16(buf[0:2])
rec.rec_type = buf[2]
rec.rec_sub = buf[3]
// Create the variable part and read it.
rec.data = make([]byte, rec.rec_len)
_, err = io.ReadFull(r, rec.data)
if err != nil {
// handle error
}
Related
how can I solve this error?
"excel range has bad size. it must be only 1 line or 1 row." this error for large data from Cplex with .csv format occurred.
Difficult to guess what happened but let me share a small csv OPL example:
.mod https://github.com/AlexFleischerParis/zooopl/blob/master/zoocsv.mod
int nbKids=...;
// a tuple is like a struct in C, a class in C++ or a record in Pascal
tuple bus
{
key int nbSeats;
float cost;
}
// This is a tuple set
{bus} buses=...;
// asserts help make sure data is fine
assert forall(b in buses) b.nbSeats>0;
assert forall(b in buses) b.cost>0;
// decision variable array
dvar int+ nbBus[buses];
// objective
minimize
sum(b in buses) b.cost*nbBus[b];
// constraints
subject to
{
sum(b in buses) b.nbSeats*nbBus[b]>=nbKids;
}
tuple t_result
{
key int nbSeats;
int nb;
}
{t_result} result={<b.nbSeats,nbBus[b]> | b in buses};
.dat
/*
the csv file is https://github.com/AlexFleischerParis/zooopl/blob/master/zoocsv.dat
nbseats;cost;
40;500;
30;400;
*/
nbKids=300;
// New feature in CPLEX 20.10
CSVConnection csvinput("zoobusesinput.csv?firstIsHeader=true", "");
buses from CSVRead(csvinput, "");
CSVConnection csvoutput("zoobusesoutput.csv?outputOnly=true&append=false&outputSeparator=;", "");
result to CSVPublish(csvoutput, "");
Here is sample code:
package main
import (
"fmt"
)
type A struct {
Name string
}
func (this *A) demo(tag string) {
fmt.Printf("%#v\n", this)
fmt.Println(tag)
}
func main() {
var ele A
ele.demo("ele are called")
ele2 := A{}
ele2.demo("ele2 are called")
}
Run results:
&main.A{Name:""}
ele are called
&main.A{Name:""}
ele2 are called
It looks like those are the same about var ele A and ele2 := A{}
So, the struct's Zero value is not nil, but a struct that all of the property are initialized Zero value. Is the guess right?
If the guess is right, then the nature of var ele A and ele2 := A{} are the same right?
Why guess (correctly) when there's some documentation ?
When storage is allocated for a variable, either through a declaration or a call of new, or when a new value is created, either through a composite literal or a call of make, and no explicit initialization is provided, the variable or value is given a default value.
Each element of such a variable or value is set to the zero value for its type:
false for booleans,
0 for integers,
0.0 for floats,
"" for strings,
and nil for pointers, functions, interfaces, slices, channels, and maps.
This initialization is done recursively, so for instance each element of an array of structs will have its fields zeroed if no value is specified.
Note that there's no way to set a struct value to nil (but you could set the value of a pointer to a struct to nil).
I don't believe the top-voted answer is clearly worded to answer the question, so here is a more clear explanation:
"The elements of an array or struct will have its fields zeroed if no value is specified. This initialization is done recursively:"
Source
Demonstrating #Denys Séguret's first-rate answer. Each element of such a variable or value is set to the zero value for its type (https://golang.org/ref/spec#The_zero_value):
package main
import "fmt"
func main() {
// false for booleans,
var bl bool // false
//0 for numeric types,
var in int // 0
// "" for strings,
var st string // ""
// and nil for pointers, functions, interfaces, channels,
var pi *int // <nil>
var ps *string // <nil>
var fu func() // <nil> Go vet error. https://stackoverflow.com/a/56663166/12817546
var ir interface{} // <nil>
var ch chan string // <nil>
fmt.Println(bl, in, st, pi, ps, fu, ir, ch)
// slices, and maps.
var sl []int // true
var mp map[int]string // true
var pm *map[int]string // <nil>
fmt.Printf("%v %v %v\n", sl == nil, mp == nil, pm)
}
I am having a problem figuring out how to reference elements of a sub structure.
See: http://play.golang.org/p/pamS_ZY01s
Given something like following.... How do you reference data in the room struct? I have tried fmt.Println(*n.Homes[0].Rooms[0].Size), but that does not work.
Begin Code example
package main
import (
"fmt"
)
type Neighborhood struct {
Name string
Homes *[]Home
}
type Home struct {
Color string
Rooms *[]Room
}
type Room struct {
Size string
}
func main() {
var n Neighborhood
var h1 Home
var r1 Room
n.Name = "Mountain Village"
h1.Color = "Blue"
r1.Size = "200 sq feet"
// Initiaize Array of Homes
homeslice := make([]Home, 0)
n.Homes = &homeslice
roomslice := make([]Room, 0)
h1.Rooms = &roomslice
*h1.Rooms = append(*h1.Rooms, r1)
*n.Homes = append(*n.Homes, h1)
fmt.Println(n)
fmt.Println(*n.Homes)
}
First, *[]Home is really wasteful. A slice is a three worded struct under the hood, one of them being itself a pointer to an array. You are introducing a double indirection there. This article on data structures in Go is very useful.
Now, because of this indirection, you need to put the dereference operator * in every pointer-to-slice expression. Like this:
fmt.Println((*(*n.Homes)[0].Rooms)[0].Size)
But, really, just take out the pointers.
I am trying to understand this code to convert from [16]byte to string:
// From: https://code.google.com/p/go/source/browse/src/pkg/net/interface_windows.go
func bytePtrToString(p *uint8) string {
a := (*[10000]uint8)(unsafe.Pointer(p))
i := 0
for a[i] != 0 {
i++
}
return string(a[:i])
}
// Type of ipl.IpAddress.String is [16]byte
str := bytePtrToString(&ipl.IpAddress.String[0])
I mean, what is the reason for the pointer magic? Couldn't it be written simply as follows?
func toString(p []byte) string {
for i, b := range p {
if b == 0 {
return string(p[:i])
}
}
return string(p)
}
// Type of ipl.IpAddress.String is [16]byte
str := toString(ipl.IpAddress.String[:])
No, it could not be written as you propose.
You forget what []byte is not array of bytes. Its slice of bytes.
In memory slice is just structure which has pointer to buffer (array of bytes, which will hold data) and some information about array size.
a := (*[10000]uint8)(unsafe.Pointer(p))
this code convert byte pointer to array of bytes.
In go array of bytes - its just bytes, like in C. Nothing more.
So if you will declare array of bytes:
var buffer [256]byte
Go will allocate exactly 256 bytes, array don't even have length. Length of array is part of its type, and only compiler have information about array length.
Whats why we can convert byte pointer to array. Its basically the same.
i := 0
for a[i] != 0 {
i++
}
Here we just find the end of null terminated string.
return string(a[:i])
Here we use slicing operation to create a string. In Go string is basically constant slice, so it will point to the same memory address as original byte pointer.
This two articles will help you understand this topic a bit better:
http://blog.golang.org/slices
http://blog.golang.org/go-slices-usage-and-internals
I got slightly confused this morning when the following code worked.
// s points to an empty string in memory
s := new(string)
// assign 1000 byte string to that address
b := make([]byte, 0, 1000)
for i := 0; i < 1000; i++ {
if i%100 == 0 {
b = append(b, '\n')
} else {
b = append(b, 'x')
}
}
*s = string(b)
// how is there room for it there?
print(*s)
http://play.golang.org/p/dAvKLChapd
I feel like I'm missing something obvious here. Some insight would be appreciated.
I hope I understood the question...
An entity of type string is implemented by a run time struct, roughly
type rt_string struct {
ptr *byte // first byte of the string
len int // number of bytes in the string
}
The line
*s = string(b)
sets a new value (of type rt_string) at *s. Its size is constant, so there's "room" for it.
More details in rsc's paper.