Go parse yaml file - struct

I'm trying to parse a yaml file with Go. Unfortunately I can't figure out how. The yaml file I have is this:
---
firewall_network_rules:
rule1:
src: blablabla-host
dst: blabla-hostname
...
I have this Go code, but it does not work:
package main
import (
"fmt"
"io/ioutil"
"path/filepath"
"gopkg.in/yaml.v2"
)
type Config struct {
Firewall_network_rules map[string][]string
}
func main() {
filename, _ := filepath.Abs("./fruits.yml")
yamlFile, err := ioutil.ReadFile(filename)
if err != nil {
panic(err)
}
var config Config
err = yaml.Unmarshal(yamlFile, &config)
if err != nil {
panic(err)
}
fmt.Printf("Value: %#v\n", config.Firewall_network_rules)
}
When I run this, I get an error. I think it's because I haven't created a struct for the src and dst key/values. FYI: when I change that to a list, it works.
So above code parses this:
---
firewall_network_rules:
rule1:
- value1
- value2
...

If you're working with google cloud or kubernetes more specifically and want to parse a service.yaml like this:
apiVersion: v1
kind: Service
metadata:
name: myName
namespace: default
labels:
router.deis.io/routable: "true"
annotations:
router.deis.io/domains: ""
spec:
type: NodePort
selector:
app: myName
ports:
- name: http
port: 80
targetPort: 80
- name: https
port: 443
targetPort: 443
Supplying a real world example so you get the hang of how nesting can be written.
type Service struct {
APIVersion string `yaml:"apiVersion"`
Kind string `yaml:"kind"`
Metadata struct {
Name string `yaml:"name"`
Namespace string `yaml:"namespace"`
Labels struct {
RouterDeisIoRoutable string `yaml:"router.deis.io/routable"`
} `yaml:"labels"`
Annotations struct {
RouterDeisIoDomains string `yaml:"router.deis.io/domains"`
} `yaml:"annotations"`
} `yaml:"metadata"`
Spec struct {
Type string `yaml:"type"`
Selector struct {
App string `yaml:"app"`
} `yaml:"selector"`
Ports []struct {
Name string `yaml:"name"`
Port int `yaml:"port"`
TargetPort int `yaml:"targetPort"`
NodePort int `yaml:"nodePort,omitempty"`
} `yaml:"ports"`
} `yaml:"spec"`
}
There's a convenient service called yaml-to-go https://zhwt.github.io/yaml-to-go/ which converts YAML to go structs, just input your YAML into that service and you get an autogenerated struct.
A JSON equivalent exists aswell: https://mholt.github.io/json-to-go/
And last unmarshal as a previous poster wrote:
var service Service
err = yaml.Unmarshal(yourFile, &service)
if err != nil {
panic(err)
}
fmt.Print(service.Metadata.Name)

Well, I think I've figured it out by myself. The following piece of code works fine. Any suggestions/improvements?
package main
import (
"fmt"
"io/ioutil"
"path/filepath"
"gopkg.in/yaml.v2"
)
type Config struct {
Firewall_network_rules map[string]Options
}
type Options struct {
Src string
Dst string
}
func main() {
filename, _ := filepath.Abs("./fruits.yml")
yamlFile, err := ioutil.ReadFile(filename)
if err != nil {
panic(err)
}
var config Config
err = yaml.Unmarshal(yamlFile, &config)
if err != nil {
panic(err)
}
fmt.Printf("Value: %#v\n", config.Firewall_network_rules)
}

Why not organize your yaml file like below if you don't care about the rule name?
---
firewall_network_rules:
-
name: rule1
src: blablabla-host
dst: blabla-hostname
-
name: rule2
src: bla-host
dst: bla-hostname
So the code will be like this, it is clean and extensible:
type Rule struct {
Name string `yaml:"name"`
Src string `yaml:"src"`
Dst string `yaml:"dst"`
}
type Config struct {
FirewallNetworkRules []Rule `yaml:"firewall_network_rules"`
}

If your YAML file is simple (single nesting) like following
mongo:
DB: database
COL: collection
log:
error: log/error/error.log
api:
key: jhgwewbcjwefwjfg
Here, you can use interface instead of declaring struct.
main(){
config := Config()
mongoConfig := config["mongo"]
mongo.MongoDial(
String(
Get(mongoConfig, "DB")
),
String(
Get(mongoConfig, "COL")
)
)
}
func Config() map[string]interface{} {
filename, _ := filepath.Abs("configs/config.yaml")
yamlFile, err := ioutil.ReadFile(filename)
if err != nil {
panic(err)
}
var config map[string]interface{}
err = yaml.Unmarshal(yamlFile, &config)
if err != nil {
panic(err)
}
return config
}
func Get(this interface{}, key string) interface{} {
return this.(map[interface{}]interface{})[key]
}
func String(payload interface{}) string {
var load string
if pay, oh := payload.(string); oh {
load = pay
}else{
load = ""
}
return load
}
This works fine for level 1 nesting, if you have complex nesting then it is recommended to use struct.

Related

How can i store in struct and use the value in different functions throughout the application?

I would like to store a mqtt client in a struct and use this client throughout the application.
My project structure looks like this:
-src
-payloads
-payload.go
-repositories
-repository.go
-main.go
payload.go looks like this:
package payload
import MQTT "github.com/eclipse/paho.mqtt.golang"
type MQTTClient struct {
Client MQTT.Client
}
in my repository.go I have a Connect() function like this:
func Connect() MQTT.Client {
deviceID := flag.String("device", "", "GCP Device-Id")
bridge := struct {
host *string
port *string
}{
flag.String("mqtt_host", "", "MQTT Bridge Host"),
flag.String("mqtt_port", "", "MQTT Bridge Port"),
}
projectID := flag.String("project", "", "GCP Project ID")
registryID := flag.String("registry", "", "Cloud IoT Registry ID (short form)")
region := flag.String("region", "", "GCP Region")
certsCA := flag.String("ca_certs", "", "Download https://pki.google.com/roots.pem")
privateKey := flag.String("private_key", "", "Path to private key file")
server := fmt.Sprintf("ssl://%v:%v", *bridge.host, *bridge.port)
topic := struct {
config string
telemetry string
}{
config: fmt.Sprintf("/devices/%v/config", *deviceID),
telemetry: fmt.Sprintf("/devices/%v/events/", *deviceID),
}
qos := flag.Int("qos", 0, "The QoS to subscribe to messages at")
clientid := fmt.Sprintf("projects/%v/locations/%v/registries/%v/devices/%v",
*projectID,
*region,
*registryID,
*deviceID,
)
log.Println("[main] Loading Google's roots")
certpool := x509.NewCertPool()
pemCerts, err := ioutil.ReadFile(*certsCA)
if err == nil {
certpool.AppendCertsFromPEM(pemCerts)
}
log.Println("[main] Creating TLS Config")
config := &tls.Config{
RootCAs: certpool,
ClientAuth: tls.NoClientCert,
ClientCAs: nil,
Certificates: []tls.Certificate{},
MinVersion: tls.VersionTLS12,
}
flag.Parse()
connOpts := MQTT.NewClientOptions().
AddBroker(server).
SetClientID(clientid).
SetAutoReconnect(true).
SetConnectRetry(true).
SetDefaultPublishHandler(onMessageReceived).
SetConnectionLostHandler(connLostHandler).
SetReconnectingHandler(reconnHandler).
SetTLSConfig(config)
connOpts.SetUsername("unused")
///JWT Generation Starts from Here
token := jwt.New(jwt.SigningMethodES256)
token.Claims = jwt.StandardClaims{
Audience: *projectID,
IssuedAt: time.Now().Unix(),
ExpiresAt: time.Now().Add(24 * time.Hour).Unix(),
}
//Reading key file
log.Println("[main] Load Private Key")
keyBytes, err := ioutil.ReadFile(*privateKey)
if err != nil {
log.Fatal(err)
}
//Parsing key from file
log.Println("[main] Parse Private Key")
key, err := jwt.ParseECPrivateKeyFromPEM(keyBytes)
if err != nil {
log.Fatal(err)
}
//Signing JWT with private key
log.Println("[main] Sign String")
tokenString, err := token.SignedString(key)
if err != nil {
log.Fatal(err)
}
//JWT Generation Ends here
connOpts.SetPassword(tokenString)
connOpts.OnConnect = func(c MQTT.Client) {
if token := c.Subscribe(topic.config, byte(*qos), nil); token.Wait() && token.Error() != nil {
log.Fatal(token.Error())
}
}
client := MQTT.NewClient(connOpts)
if token := client.Connect(); token.Wait() && token.Error() != nil {
fmt.Printf("Not Connected..Retrying... %s\n", server)
} else {
fmt.Printf("Connected to %s\n", server)
}
return client
}
No i am using this Connect() function in a go routine along with a grpc server like this..
func main() {
fmt.Println("Server started at port 5005")
lis, err := net.Listen("tcp", "0.0.0.0:5005")
if err != nil {
log.Fatalf("Failed to listen: %v", err)
}
//Creating keepAlive channel for mqttt subscribe
keepAlive := make(chan os.Signal)
defer close(keepAlive)
go func() {
//Connecting to mqtt client
client := repositories.Connect()
//passing the client to struct
res := &payload.MQTTClient{
Client: client,
}
fmt.Println(res)
//looking for interupt(Ctrl+C)
value := <-keepAlive
//If Ctrl+C is pressed then exit the application
if value == os.Interrupt {
fmt.Printf("Exiting the application")
os.Exit(3)
}
}()
s := grpc.NewServer()
MqttRepository := repositories.NewMqttRepository()
// It creates a new gRPC server instance
rpc.NewMqttServer(s, MqttRepository)
if err := s.Serve(lis); err != nil {
log.Fatalf("Failed to serve: %v", err)
}
}
Now I am storing the client from the Connect() function into the struct but I am not able to use it. How can I store client somewhere in main.go and then use it throughout the aplication?
If you want to have a struct available throughout your code, you might want to use a Singleton Pattern [1].
In Go you basically define an exported global variable in a package which will be available by all code that imports the package.
You can have client live in payload package (or whichever works for you, this is just an example):
package payload
import MQTT "github.com/eclipse/paho.mqtt.golang"
type MQTTClient struct {
Client MQTT.Client
}
var SingletonClient *MQTTClient
Then in main.go instead of
res := &payload.MQTTClient{
Client: client,
}
do
payload.SingletonClient = &payload.MQTTClient{
Client: client,
}
Voila, now you can use payload.SingletonClient everywhere you import payload package.
Be aware, that Singleton pattern is not considered a good practice, it would be much better if you could structure your code to pass the client around where it's needed. Sometimes it may be useful though.
Be aware, that any code that tries to use payload.SingletonClient before it's set in main.go will fail because the client will not be initialized yet, so you might want to use some kind of a lock to make sure that the global variable is set before using it.
[1] https://en.wikipedia.org/wiki/Singleton_pattern

How to execute govc/pyvmomi run command which lists output of server on local pc

I am able to list all the vm's using govmomi(not govc which is one line command in terminal) but I am trying to implement this:
govc guest.run -vm $vm_name cat sample.txt from scratch rather than running it via govc. The docs seems to be confusing. Does anyone know how it has been implemented underneath ? And is there something which is similar to this in pyvmomi ? Using Pyvmomi, I was able to execute the command on server from local pc and then store the output in a file on server and then transfer the file from server to local pc. But at later stage, I came across govc guest.run -vm $vm_name cat sample.txt command which gives the output on the console of local pc. Does it also behind the scene store the output temporarily on the server ? If not then how to implement this ?
This is what I have tried when trying to list all the vm's using govmomi, I want to extend this so that it can implement govc run command.
package main
import (
"flag"
"fmt"
"net/url"
"os"
"strings"
"text/tabwriter"
"sort"
"github.com/vmware/govmomi"
"github.com/vmware/govmomi/find"
"github.com/vmware/govmomi/property"
"github.com/vmware/govmomi/vim25/mo"
"github.com/vmware/govmomi/vim25/types"
"golang.org/x/net/context"
)
func GetEnvString(v string, def string) string {
r := os.Getenv(v)
if r == "" {
return def
}
return r
}
// GetEnvBool returns boolean from environment variable.
func GetEnvBool(v string, def bool) bool {
r := os.Getenv(v)
if r == "" {
return def
}
switch strings.ToLower(r[0:1]) {
case "t", "y", "1":
return true
}
return false
}
const (
envURL = "vcenter's_ip"
envUserName = "username#organisation.org"
envPassword = "pwd"
envInsecure = "true"
)
var urlDescription = fmt.Sprintf("ESX or vCenter URL [%s]", envURL)
var urlFlag = flag.String("url", GetEnvString(envURL, "https://username#organisation.org:pwd#ip_of_vcenter/sdk"), urlDescription)
var insecureDescription = fmt.Sprintf("Don't verify the server's certificate chain [%s]", envInsecure)
var insecureFlag = flag.Bool("insecure", GetEnvBool(envInsecure, true), insecureDescription)
func processOverride(u *url.URL) {
envUsername := os.Getenv(envUserName)
envPassword := os.Getenv(envPassword)
// Override username if provided
if envUsername != "" {
var password string
var ok bool
if u.User != nil {
password, ok = u.User.Password()
}
if ok {
u.User = url.UserPassword(envUsername, password)
} else {
u.User = url.User(envUsername)
}
}
// Override password if provided
if envPassword != "" {
var username string
if u.User != nil {
username = u.User.Username()
}
u.User = url.UserPassword(username, envPassword)
}
}
func exit(err error) {
fmt.Fprintf(os.Stderr, "Error: %s\n", err)
os.Exit(1)
}
type ByName []mo.VirtualMachine
func (n ByName) Len() int { return len(n) }
func (n ByName) Swap(i, j int) { n[i], n[j] = n[j], n[i] }
func (n ByName) Less(i, j int) bool { return n[i].Name < n[j].Name }
func main() {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
flag.Parse()
// Parse URL from string
u, err := url.Parse(*urlFlag)
if err != nil {
exit(err)
}
fmt.Println(u)
// Override username and/or password as required
processOverride(u)
// Connect and log in to ESX or vCenter
c, err := govmomi.NewClient(ctx, u, *insecureFlag)
if err != nil {
exit(err)
}
f := find.NewFinder(c.Client, true)
// Find one and only datacenter
dc, err := f.DefaultDatacenter(ctx)
if err != nil {
exit(err)
}
// Make future calls local to this datacenter
f.SetDatacenter(dc)
// Find virtual machines in datacenter
vms, err := f.VirtualMachineList(ctx, "in-temp-vm-2")// if instead of "in-temp-vm-2" *, then it lists down all the vm's
//fmt.Println(vms)
if err != nil {
exit(err)
}
//fmt.Println(vms);
pc := property.DefaultCollector(c.Client)
// Convert datastores into list of references
var refs []types.ManagedObjectReference
for _, vm := range vms {
refs = append(refs, vm.Reference())
}
// Retrieve name property for all vms
var vmt []mo.VirtualMachine
err = pc.Retrieve(ctx, refs, []string{"name"}, &vmt)
if err != nil {
exit(err)
}
tw := tabwriter.NewWriter(os.Stdout, 2, 0, 2, ' ', 0)
fmt.Println("Virtual machines found:", len(vmt))
sort.Sort(ByName(vmt))
for _, vm := range vmt {
fmt.Fprintf(tw, "%s\n", vm.Name)
}
tw.Flush()
}
This code is a foolish attempt to implement govc in golang basically using shell commands like bash.
package main
import (
"bytes"
"fmt"
"log"
//"os"
"os/exec"
)
const ShellToUse = "bash"
func Shellout(command string) (error, string, string) {
var stdout bytes.Buffer
var stderr bytes.Buffer
cmd := exec.Command(ShellToUse, "-c", command)
cmd.Stdout = &stdout
cmd.Stderr = &stderr
err := cmd.Run()
return err, stdout.String(), stderr.String()
}
func main() {
err, out, errout := Shellout("govc guest.run -vm $vm_name python3 sample.py")
if err != nil {
log.Printf("error: %v\n", err)
}
fmt.Println("--- stdout ---")
fmt.Println(out)
// fmt.Println("--- stderr ---")
fmt.Println(errout)
}
And this is my python implementation which executes command and stores it on server in a file and then I take that file from server to local pc, I don't want to store anything on server even temporarily:
from pyVim import connect
from config import *
from pyVmomi import vim, vmodl
import ssl
import os
import requests
service_instance = connect.SmartConnect(host="xxxx", port=aaa,user="yyy" , pwd=pwd,sslContext=ssl._create_unverified_context())
content = service_instance.RetrieveContent()
# # Find a VM
vm = searcher.FindByIp(ip="aaaa", vmSearch=True)
creds = vim.vm.guest.NamePasswordAuthentication(username='root', password=vmpwd)
pm = service_instance.content.guestOperationsManager.processManager
#executes and saves sample.txt into server
ps = vim.vm.guest.ProcessManager.ProgramSpec(programPath='/usr/bin/python', arguments='--version &> sample.txt')
res = pm.StartProgramInGuest(vm, creds, ps)
dest="/Users/username/Desktop/vcenter/sample.txt" #My local pc
src="/root/sample.txt" #Server's directory
fti = content.guestOperationsManager.fileManager.InitiateFileTransferFromGuest(vm, creds, src)
resp=requests.get(fti.url, verify=False)
#Writes into file
with open(dest, 'wb') as f:
f.write(resp.content)

How to convert a string value to the correct reflect.Kind in go?

Is there a generic helper method in Go to convert a string to the correct value based on reflect.Kind?
Or do I need to implement the switch over all kinds myself?
I have a value like "143" as a string and a reflect.Value with kind "UInt16" and like to convert that string value and set it into the UInt16 value of my struct.
My current code looks like:
func setValueFromString(v reflect.Value, strVal string) error {
switch v.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
val, err := strconv.ParseInt(strVal, 0, 64)
if err != nil {
return err
}
if v.OverflowInt(val) {
return errors.New("Int value too big: " + strVal)
}
v.SetInt(val)
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
val, err := strconv.ParseUint(strVal, 0, 64)
if err != nil {
return err
}
if v.OverflowUint(val) {
return errors.New("UInt value too big: " + strVal)
}
v.SetUint(val)
case reflect.Float32:
val, err := strconv.ParseFloat(strVal, 32)
if err != nil {
return err
}
v.SetFloat(val)
case reflect.Float64:
val, err := strconv.ParseFloat(strVal, 64)
if err != nil {
return err
}
v.SetFloat(val)
case reflect.String:
v.SetString(strVal)
case reflect.Bool:
val, err := strconv.ParseBool(strVal)
if err != nil {
return err
}
v.SetBool(val)
default:
return errors.New("Unsupported kind: " + v.Kind().String())
}
return nil
}
This works already, but I wonder if this is already implemented somewhere else.
Edit: Answer to the original question ("how to obtain a reflect.Kind from its string representation") is at the end. Answer to your edited question follows:
What you're doing is the fastest and "safest". If you don't want to hassle with that big switch, you may take advantage of e.g. the json package which already contains this switch to decode values from JSON string (in encoding/json/decode.go, unexported function literalStore()).
Your decoding function could look like this:
func Set(v interface{}, s string) error {
return json.Unmarshal([]byte(s), v)
}
A simple call to json.Unmarshal(). Using / testing it:
{
var v int
err := Set(&v, "1")
fmt.Println(v, err)
}
{
var v int
err := Set(&v, "d")
fmt.Println(v, err)
}
{
var v uint32
err := Set(&v, "3")
fmt.Println(v, err)
}
{
var v bool
err := Set(&v, "true")
fmt.Println(v, err)
}
{
var v float32
err := Set(&v, `5.1`)
fmt.Println(v, err)
}
{
var v string
err := Set(&v, strconv.Quote("abc"))
fmt.Println(v, err)
}
One thing to note: when you want to pass a string, that must be quoted, e.g. with strconv.Quote(). Output (try it on the Go Playground):
1 <nil>
0 invalid character 'd' looking for beginning of value
3 <nil>
true <nil>
5.1 <nil>
abc <nil>
If you don't want to require quoted strings (which just complicates things), you may build it into the Set() function:
func Set(v interface{}, s string) error {
if t := reflect.TypeOf(v); t.Kind() == reflect.Ptr &&
t.Elem().Kind() == reflect.String {
s = strconv.Quote(s)
}
return json.Unmarshal([]byte(s), v)
}
And then you may call it with the address of a string variable and a string value unquoted:
var v string
err := Set(&v, "abc")
fmt.Println(v, err)
Try this variant on the Go Playground.
Answer to the original question: how to obtain a reflect.Kind from its string representation:
Declaration of reflect.Kind:
type Kind uint
The different values of reflect.Kinds are constants:
const (
Invalid Kind = iota
Bool
Int
Int8
// ...
Struct
UnsafePointer
)
And the reflect package provides only a single method for the reflect.Kind() type:
func (k Kind) String() string
So as it stands, you cannot obtain a reflect.Kind from its string representation (only the reverse direction is possible by using the Kind.String() method). But it's not that hard to provide this functionality.
What we'll do is we build a map from all the kinds:
var strKindMap = map[string]reflect.Kind{}
We init it like this:
func init() {
for k := reflect.Invalid; k <= reflect.UnsafePointer; k++ {
strKindMap[k.String()] = k
}
}
This is possible and correct because constants are initialized using iota which evaluates to successive untyped integer constants, and the first value is reflect.Invalid and the last is reflect.UnsafePointer.
And now you can obtain reflect.Kind from its string representation by simply indexing this map. A helper function which does that:
func strToKind(s string) reflect.Kind {
k, ok := strKindMap[s]
if !ok {
return reflect.Invalid
}
return k
}
And we're done. Testing / using it:
fmt.Printf("All: %#v\n", strKindMap)
for _, v := range []string{"Hey", "uint8", "ptr", "func", "chan", "interface"} {
fmt.Printf("String: %q, Kind: %v (%#v)\n", v, strToKind(v), strToKind(v))
}
Output (try it on the Go Playground):
All: map[string]reflect.Kind{"int64":0x6, "uint8":0x8, "uint64":0xb, "slice":0x17, "uintptr":0xc, "int8":0x3, "array":0x11, "interface":0x14, "unsafe.Pointer":0x1a, "complex64":0xf, "complex128":0x10, "int":0x2, "uint":0x7, "int16":0x4, "uint16":0x9, "map":0x15, "bool":0x1, "int32":0x5, "ptr":0x16, "string":0x18, "func":0x13, "struct":0x19, "invalid":0x0, "uint32":0xa, "float32":0xd, "float64":0xe, "chan":0x12}
String: "Hey", Kind: invalid (0x0)
String: "uint8", Kind: uint8 (0x8)
String: "ptr", Kind: ptr (0x16)
String: "func", Kind: func (0x13)
String: "chan", Kind: chan (0x12)
String: "interface", Kind: interface (0x14)

Golang - Unmarshal/rebuild an object in 2 step passing by interface

I'm working with json and golang. I've made a TCP server and, I need to Unmarshal the message to know which type of service is asked, before Unmarshal the contained data. It's a bit hard to explain so this is my code:
package main
import (
"fmt"
"encoding/json"
)
type Container struct {
Type string
Object interface{}
}
type Selling struct {
Surname string
Firstname string
//......
Price int
}
type Buying struct {
ID int
Surname string
Firstname string
//..........
}
/*
type Editing struct {
ID int
...............
}
Informations, etc etc
*/
func main() {
tmp_message_json1 := Selling{Surname: "X", Firstname: "Mister", Price: 10}
//tmp_message_json1 := Buying{ID: 1, Surname: "X", Firstname: "Mister"}
tmp_container_json1 := Container{Type: "Selling", Object: tmp_message_json1}
json_tmp, _ := json.Marshal(tmp_container_json1)
/*...........
We init tcp etc etc and then a message comes up !
...........*/
c := Container{}
json.Unmarshal(json_tmp, &c)
//I unmarshal a first time to know the type of service asked
// first question: Does Unmarshal need to be used only one time?
// does I need to pass c.Object as a string to unmarshal it in the called functions?
if c.Type == "Buying" {
takeInterfaceBuying(c.Object)
} else if c.Type == "client" {
takeInterfaceSelling(c.Object)
} else {
fmt.Println("bad entry")
}
}
func takeInterfaceBuying(Object interface{}) {
bu := Object
fmt.Println(bu.Firstname, bu.Surname, "wants to buy the following product:", bu.ID)
}
func takeInterfaceSelling(Object interface{}) {
se := Object
fmt.Println(se.Firstname, se.Surname, "wants to sell something for:", se.Price)
}
I need to handle messages which comes up like it, but I don't know if it's possible? does it possible?
Thank for help !
There is json.RawMessage for this purpose.
package main
import (
"encoding/json"
"fmt"
)
type Container struct {
Type string
Object json.RawMessage
}
type Selling struct {
Surname string
Firstname string
Price int
}
type Buying struct {
ID int
Surname string
Firstname string
}
func main() {
rawJson1 := []byte(`{"Type":"Selling","Object":{"Surname":"X","Firstname":"Mister","Price":10}}`)
rawJson2 := []byte(`{"Type":"Buying","Object":{"ID":1,"Surname":"X","Firstname":"Mister"}}`)
processMessage(rawJson1)
processMessage(rawJson2)
}
func processMessage(data []byte) {
var c Container
json.Unmarshal(data, &c)
switch {
case c.Type == "Buying":
processBuying(c)
case c.Type == "Selling":
processSelling(c)
default:
fmt.Println("bad entry")
}
}
func processBuying(c Container) {
var bu Buying
json.Unmarshal(c.Object, &bu)
fmt.Println(bu.Firstname, bu.Surname, "wants to buy the following product:", bu.ID)
}
func processSelling(c Container) {
var se Selling
json.Unmarshal(c.Object, &se)
fmt.Println(se.Firstname, se.Surname, "wants to sell something for:", se.Price)
}
I might be wrong, but I don't think you can do it in one step.
First idea: Unmarshal in map[string]interface{}
Don't use type with unmarshal, instead use a map[string]interface{}, and then construct Selling/Buying from this map (or use directly the map)
type Container struct {
Type string
Object map[string]interface{}
}
Second idea: Two steps / clueless container
First: Unmarshall in a clueless Container that doesn't know the type
type CluelessContainer struct {
Type string
Object interface{} `json:"-"` // or just remove this line ?
}
Then unmarshall in the type aware container. You could use a factory pattern to come with the right struct.

How to print struct variables in console?

How can I print (to the console) the Id, Title, Name, etc. of this struct in Golang?
type Project struct {
Id int64 `json:"project_id"`
Title string `json:"title"`
Name string `json:"name"`
Data Data `json:"data"`
Commits Commits `json:"commits"`
}
To print the name of the fields in a struct:
fmt.Printf("%+v\n", yourProject)
From the fmt package:
when printing structs, the plus flag (%+v) adds field names
That supposes you have an instance of Project (in 'yourProject')
The article JSON and Go will give more details on how to retrieve the values from a JSON struct.
This Go by example page provides another technique:
type Response2 struct {
Page int `json:"page"`
Fruits []string `json:"fruits"`
}
res2D := &Response2{
Page: 1,
Fruits: []string{"apple", "peach", "pear"}}
res2B, _ := json.Marshal(res2D)
fmt.Println(string(res2B))
That would print:
{"page":1,"fruits":["apple","peach","pear"]}
If you don't have any instance, then you need to use reflection to display the name of the field of a given struct, as in this example.
type T struct {
A int
B string
}
t := T{23, "skidoo"}
s := reflect.ValueOf(&t).Elem()
typeOfT := s.Type()
for i := 0; i < s.NumField(); i++ {
f := s.Field(i)
fmt.Printf("%d: %s %s = %v\n", i,
typeOfT.Field(i).Name, f.Type(), f.Interface())
}
I want to recommend go-spew, which according to their github "Implements a deep pretty printer for Go data structures to aid in debugging"
go get -u github.com/davecgh/go-spew/spew
usage example:
package main
import (
"github.com/davecgh/go-spew/spew"
)
type Project struct {
Id int64 `json:"project_id"`
Title string `json:"title"`
Name string `json:"name"`
Data string `json:"data"`
Commits string `json:"commits"`
}
func main() {
o := Project{Name: "hello", Title: "world"}
spew.Dump(o)
}
output:
(main.Project) {
Id: (int64) 0,
Title: (string) (len=5) "world",
Name: (string) (len=5) "hello",
Data: (string) "",
Commits: (string) ""
}
my 2cents would be to use json.MarshalIndent -- surprised this isn't suggested, as it is the most straightforward. for example:
func prettyPrint(i interface{}) string {
s, _ := json.MarshalIndent(i, "", "\t")
return string(s)
}
no external deps and results in nicely formatted output.
I think it would be better to implement a custom stringer if you want some kind of formatted output of a struct
for example
package main
import "fmt"
type Project struct {
Id int64 `json:"project_id"`
Title string `json:"title"`
Name string `json:"name"`
}
func (p Project) String() string {
return fmt.Sprintf("{Id:%d, Title:%s, Name:%s}", p.Id, p.Title, p.Name)
}
func main() {
o := Project{Id: 4, Name: "hello", Title: "world"}
fmt.Printf("%+v\n", o)
}
p = Project{...}
fmt.Printf("%+v", p)
fmt.Printf("%#v", p) //with type
Alternatively, try using this function PrettyPrint()
// print the contents of the obj
func PrettyPrint(data interface{}) {
var p []byte
// var err := error
p, err := json.MarshalIndent(data, "", "\t")
if err != nil {
fmt.Println(err)
return
}
fmt.Printf("%s \n", p)
}
In order to use this you do not need any additional packages with the exception of fmt and encoding/json, just a reference, pointer to, or literal of the struct you have created.
To use just take your struct, initialize it in main or whatever package you are in and pass it into PrettyPrint().
type Prefix struct {
Network string
Mask int
}
func valueStruct() {
// struct as a value
var nw Prefix
nw.Network = "10.1.1.0"
nw.Mask = 24
fmt.Println("### struct as a pointer ###")
PrettyPrint(&nw)
}
It's output would be
### struct as a pointer ###
{
"Network": "10.1.1.0",
"Mask": 24
}
Play around with the code here.
It is very convenient to use package fmt to output:
fmt.Printf("%+v \n", yourProject)
if you want to see the full type of the sturct, you can use # replace + :
fmt.Printf("%#v \n", yourProject)
I recommend to use Pretty Printer Library. In that you can print any struct very easily.
Install Library
https://github.com/kr/pretty
or
go get github.com/kr/pretty
Now do like this in your code
package main
import (
fmt
github.com/kr/pretty
)
func main(){
type Project struct {
Id int64 `json:"project_id"`
Title string `json:"title"`
Name string `json:"name"`
Data Data `json:"data"`
Commits Commits `json:"commits"`
}
fmt.Printf("%# v", pretty.Formatter(Project)) //It will print all struct details
fmt.Printf("%# v", pretty.Formatter(Project.Id)) //It will print component one by one.
}
Also you can get difference between component through this library and so more. You can also have a look on library Docs here.
I like litter.
From their readme:
type Person struct {
Name string
Age int
Parent *Person
}
litter.Dump(Person{
Name: "Bob",
Age: 20,
Parent: &Person{
Name: "Jane",
Age: 50,
},
})
Sdump is pretty handy in tests:
func TestSearch(t *testing.T) {
result := DoSearch()
actual := litterOpts.Sdump(result)
expected, err := ioutil.ReadFile("testdata.txt")
if err != nil {
// First run, write test data since it doesn't exist
if !os.IsNotExist(err) {
t.Error(err)
}
ioutil.Write("testdata.txt", actual, 0644)
actual = expected
}
if expected != actual {
t.Errorf("Expected %s, got %s", expected, actual)
}
}
To print the struct as JSON:
fmt.Printf("%#v\n", yourProject)
Also possible with (as it was mentioned above):
fmt.Printf("%+v\n", yourProject)
But the second option prints string values without "" so it is harder to read.
I suggest u use fmt.Printf("%#v\n", s) , It will print golang type at the same time
package main
import (
"fmt"
"testing"
)
type student struct {
id int32
name string
}
type Project struct {
Id int64 `json:"project_id"`
Title string `json:"title"`
Name string `json:"name"`
}
func TestPrint(t *testing.T) {
s := Project{1, "title","jack"}
fmt.Printf("%+v\n", s)
fmt.Printf("%#v\n", s)
}
result:
{Id:1 Title:title Name:jack}
main.Project{Id:1, Title:"title", Name:"jack"}
You can do the json mashal first and print it as a string. There you can see it the whole struct value completely.
package main
import "fmt"
import "json"
type Project struct {
Id int64 `json:"project_id"`
Title string `json:"title"`
Name string `json:"name"`
}
func main() {
o := Project{Id: 4, Name: "hello", Title: "world"}
om, _ := json.marshal(o)
log.Printf("%s\n", string(om))
}
When you have more complex structures, you might need to convert to JSON before printing:
// Convert structs to JSON.
data, err := json.Marshal(myComplexStruct)
fmt.Printf("%s\n", data)
Source: https://gist.github.com/tetsuok/4942960
Sometimes, it might be handy to print the struct as valid Go code (the go/ast equivalent). For this purpose, https://github.com/hexops/valast does a great job:
package main
import (
"fmt"
"github.com/hexops/valast"
)
type ProjectData struct {
Title string `json:"title"`
Name string `json:"name"`
Data string `json:"data"`
Commits string `json:"commits"`
}
type Project struct {
Id int64 `json:"project_id"`
Data *ProjectData `json:"data"`
}
func main() {
p := Project{
Id: 1,
Data: &ProjectData{
Title: "Test",
Name: "Mihai",
Data: "Some data",
Commits: "Test Message",
},
}
fmt.Println(valast.String(p))
}
Output:
go run main.go
Project{Id: 1, Data: &ProjectData{
Title: "Test",
Name: "Mihai",
Data: "Some data",
Commits: "Test Message",
}}
Visit here to see the complete code. Here you will also find a link for an online terminal where the complete code can be run and the program represents how to extract structure's information(field's name their type & value). Below is the program snippet that only prints the field names.
package main
import "fmt"
import "reflect"
func main() {
type Book struct {
Id int
Name string
Title string
}
book := Book{1, "Let us C", "Enjoy programming with practice"}
e := reflect.ValueOf(&book).Elem()
for i := 0; i < e.NumField(); i++ {
fieldName := e.Type().Field(i).Name
fmt.Printf("%v\n", fieldName)
}
}
/*
Id
Name
Title
*/
Maybe this shouldn't be applied for production requests but if you are on debugging mode I suggest you follow the below approach.
marshalledText, _ := json.MarshalIndent(inputStruct, "", " ")
fmt.Println(string(marshalledText))
This results in formatting the data in json format with increased readability.
i suggest to use json.Unmarshal()
i try to print the id with this hope its helpfull:
var jsonString = `{"Id": 1, "Title": "the title", "Name": "the name","Data": "the data","Commits" : "the commits"}`
var jsonData = []byte(jsonString)
var data Project
var err = json.Unmarshal(jsonData, &data)
if err != nil {
fmt.Println(err.Error())
return
}
fmt.Println("Id :", data.Id)
There's also go-render, which handles pointer recursion and lots of key sorting for string and int maps.
Installation:
go get github.com/luci/go-render/render
Example:
type customType int
type testStruct struct {
S string
V *map[string]int
I interface{}
}
a := testStruct{
S: "hello",
V: &map[string]int{"foo": 0, "bar": 1},
I: customType(42),
}
fmt.Println("Render test:")
fmt.Printf("fmt.Printf: %#v\n", a)))
fmt.Printf("render.Render: %s\n", Render(a))
Which prints:
fmt.Printf: render.testStruct{S:"hello", V:(*map[string]int)(0x600dd065), I:42}
render.Render: render.testStruct{S:"hello", V:(*map[string]int){"bar":1, "foo":0}, I:render.customType(42)}
fmt.Printf("%+v\n", project)
This is the basic way of printing the details
You don't even need the verb. This dumps everything inside of data:
fmt.Println(data)
very simple
I don't have the structure of Data and Commits So I changed the
package main
import (
"fmt"
)
type Project struct {
Id int64 `json:"project_id"`
Title string `json:"title"`
Name string `json:"name"`
Data string `json:"data"`
Commits string `json:"commits"`
}
func main() {
p := Project{
1,
"First",
"Ankit",
"your data",
"Commit message",
}
fmt.Println(p)
}
For learning you can take help from here : https://gobyexample.com/structs
If you want to write in a log file, as I was searching previously. Then you should use:
log.Infof("Information %+v", structure)
Note:: This will not work with log.Info or log.Debug. In this case, "%v" will get printed, and all the values of the structure will be printed without printing the key/variable name.
A lot of answers for a simple question. I might as well throw my hat in the ring.
package main
import "fmt"
type Project struct {
Id int64 `json:"project_id"`
Title string `json:"title"`
Name string `json:"name"`
//Data Data `json:"data"`
//Commits Commits `json:"commits"`
}
var (
Testy Project
)
func dump_project(foo Project) {
fmt.Println("== Dump Project Struct ====")
fmt.Printf("Id: %d\n", foo.Id)
fmt.Println("Title: ", foo.Title)
fmt.Printf("Name: %v\n", foo.Name)
}
func main() {
fmt.Println("hello from go")
Testy.Id = 3
Testy.Title = "yo"
Testy.Name = "my name"
fmt.Println(Testy)
dump_project(Testy)
}
The output of the various print methods
hello from go
{3 yo my name}
== Dump Project Struct ====
Id: 3
Title: yo
Name: my name
Without using external libraries and with new line after each field:
log.Println(
strings.Replace(
fmt.Sprintf("%#v", post), ", ", "\n", -1))
type Response struct {
UserId int `json:"userId"`
Id int `json:"id"`
Title string `json:"title"`
Body string `json:"body"`
}
func PostsGet() gin.HandlerFunc {
return func(c *gin.Context) {
xs, err := http.Get("https://jsonplaceholder.typicode.com/posts")
if err != nil {
log.Println("The HTTP request failed with error: ", err)
}
data, _ := ioutil.ReadAll(xs`enter code here`.Body)
// this will print the struct in console
fmt.Println(string(data))
// this is to send as response for the API
bytes := []byte(string(data))
var res []Response
json.Unmarshal(bytes, &res)
c.JSON(http.StatusOK, res)
}
}
Another way is, create a func called toString that takes struct, format the
fields as you wish.
import (
"fmt"
)
type T struct {
x, y string
}
func (r T) toString() string {
return "Formate as u need :" + r.x + r.y
}
func main() {
r1 := T{"csa", "ac"}
fmt.Println("toStringed : ", r1.toString())
}
Most of these packages are relying on the reflect package to make such things possible.
fmt.Sprintf() is using -> func (p *pp) printArg(arg interface{}, verb rune) of standard lib
Go to line 638 -> https://golang.org/src/fmt/print.go
Reflection:
https://golang.org/pkg/reflect/
Example code:
https://github.com/donutloop/toolkit/blob/master/debugutil/prettysprint.go
fmt.Println("%+v", structure variable)
A better way to do this would be to create a global constant for the string "%+v" in a package called "commons"(maybe) and use it everywhere in your code
//In commons package
const STRUCTURE_DATA_FMT = "%+v"
//In your code everywhere
fmt.Println(commons.STRUCTURE_DATA_FMT, structure variable)

Resources