Go issue with strings - string

I'm having some trouble with strings in Golang. It seems that they don't get handed over to another function.
func Sendtext(ip string, port string, text string) (err int) {
targ := ip + ":" + port
raddr,e := net.ResolveTCPAddr("tcp",targ)
if e != nil {
os.Stdout.WriteString(e.String()+"\n")
return 1
}
conn,e := net.DialTCP("tcp",nil,raddr)
if e != nil {
os.Stdout.WriteString(e.String()+"\n")
return 1
}
conn.Write([]byte(text))
mess := make([]byte,1024)
conn.Read(mess)
message := string(mess)
conn.Close()
if message[0] == 'a' {
return 0
} else {
return 1
}
return 0
}
func main() {
os.Stdout.WriteString("Will send URL: ")
url := GetURL()
os.Stdout.WriteString(url + "\n\n")
_, port, pass, ip := browserbridge_config.ReadPropertiesFile()
os.Stdout.WriteString("sending this url to " + ip + ":" + port + "\n")
message := url + "\n" + pass + "\n"
os.Stdout.WriteString("\nsending... ")
e := Sendtext(ip, port, message)
if e != 0 {
os.Stdout.WriteString("ERROR\n")
os.Exit(e);
}
os.Stdout.WriteString("DONE\n")
}
and my config reader:
func ReadConfigFile(filename string) (browsercommand string, port string, pass string, ip string) {
// set defaults
browsercommand = "%u"
port = "7896"
pass = "hallo"
ip = "127.0.0.1"
// open file
file, err := os.Open(filename)
if err != nil {
os.Stdout.WriteString("Error opening config file. proceeding with standard config...")
return
}
// Get reader and buffer
reader := bufio.NewReader(file)
for {
part,_,err := reader.ReadLine()
if err != nil {
break
}
buffer := bytes.NewBuffer(make([]byte,2048))
buffer.Write(part)
s := strings.ToLower(buffer.String())
if strings.Contains(s,"browsercommand=") {
browsercommand = strings.Replace(s,"browsercommand=","",1)
} else {
if strings.Contains(s,"port=") {
port = strings.Replace(s,"port=","",1)
} else {
if strings.Contains(s,"password=") {
pass = strings.Replace(s,"password=","",1)
} else {
if strings.Contains(s,"ip=") {
ip = strings.Replace(s,"ip=","",1)
}
}
}
}
}
return
}
Output of this program:
Will send URL: test.de
sending this url to 192.168.2.100:7896
sending...
dial tcp 192.168.2.1:0: connection refused
ERROR
(192.168.2.1 is gateway)
I tried to os.Stdout.WriteString(targ) or os.Stdout.WriteString(ip) just at the top of Sendtext, and got no output.
The confusing thing about it: yesterday it worked xD (before I migrated ReadConfig into its own .go file)
I hope you can help me solving this...
sylar
Update:
As PeterSO said, the problem is not the handover of the strings
My first guess, that it must be the conversion of String to TCPAddr, is true, but it seems to be a problem with the strings, not with the net library.
I just added
ip = "192.168.2.100"
port = "7896"
right after the call of Sendtext, and that helped... (at least until a user needs to set a custom ip/port...)
I know that the problem firs occured when I decided to switch from goconf (http://code.google.com/p/goconf/) to my own. This is why I think the problem is in the ReadProperties() function.
I also realized that strconv.Atoi(port) returns 0 (parsing "7896": invalid argument)
When I used the server and client with implemented (not-changable) config, and then let the client read the password from the config file, the password comparison fails. When I also set the password right in the code (without reading a file), it works.
I really don't know what to do now... Any idea?

Go bytes package: func NewBuffer(buf []byte) *Buffer
NewBuffer creates and initializes a new Buffer using buf as its
initial contents. It is intended to prepare a Buffer to read
existing data. It can also be used to size the internal buffer for
writing. To do that, buf should have the desired capacity but a
length of zero.
In most cases, new(Buffer) (or just declaring a Buffer variable)
is preferable to NewBuffer. In particular, passing a non-empty buf
to NewBuffer and then writing to the Buffer will overwrite buf,
not append to it.
In your ReadConfigFile function, you write:
buffer := bytes.NewBuffer(make([]byte,2048))
buffer.Write(part)
The make([]byte,2048) function call creates an initial slice for buffer with a length and capacity of 2048 bytes. The buffer.Write(part) function call writes part by overwriting buffer. At the very least, you should have written make([]byte,0,2048) to initially give the buffer slice a length of zero and capacity of 2048 bytes.
Your ReadConfigFile function has other flaws. For example, the key=value format is very rigid, only keys hardcoded into the function are recognized, if a configuration file is not given it doesn't return the defaults, the configuration file is not closed, etc. Here's a basic implementation of a configuration file reader.
package main
import (
"bufio"
"fmt"
"os"
"strings"
)
type Config map[string]string
func ReadConfig(filename string) (Config, os.Error) {
config := Config{
"browsercommand": "%u",
"port": "7896",
"password": "hallo",
"ip": "127.0.0.1",
}
if len(filename) == 0 {
return config, nil
}
file, err := os.Open(filename)
if err != nil {
return nil, err
}
defer file.Close()
rdr := bufio.NewReader(file)
for {
line, err := rdr.ReadString('\n')
if eq := strings.Index(line, "="); eq >= 0 {
if key := strings.TrimSpace(line[:eq]); len(key) > 0 {
value := ""
if len(line) > eq {
value = strings.TrimSpace(line[eq+1:])
}
config[key] = value
}
}
if err == os.EOF {
break
}
if err != nil {
return nil, err
}
}
return config, nil
}
func main() {
config, err := ReadConfig(`netconfig.txt`)
if err != nil {
fmt.Println(err)
}
fmt.Println("config:", config)
ip := config["ip"]
pass := config["password"]
port := config["port"]
fmt.Println("values:", ip, port, pass)
}
Input:
[a section]
key=value
; a comment
port = 80
password = hello
ip= 217.110.104.156
# another comment
url =test.de
file =
Output:
config: map[browsercommand:%u key:value port:80 ip:217.110.104.156 url:test.de
file: password:hello]
values: 217.110.104.156 80 hello

Insert the following statement as the statement just before the call to the Sendtext function in the main function.
fmt.Println("\nmain:", "\nip = |", ip, "| \nport = |", port, "| \ntext = |", message, "|")
The output should look something like this:
main:
ip = | 192.168.2.100 |
port = | 7896 |
text = | test.de
hallo
|
Insert the following statement as the first statement in the Sendtext function.
fmt.Println("\nSendtext:", "\nip = |", ip, "| \nport = |", port, "| \ntext = |", text, "|")
The output should look something like this:
Sendtext:
ip = | 192.168.2.100 |
port = | 7896 |
text = | test.de
hallo
|
As expected, the arguments are passed to the parameters by value.

Solved it. The problem was the conversion of the 2048-long []byte to a string. This made the string to be equal long, but with a lot of NIL-chars after it.
So running a ip = strings.Replace(ip,string(0),"",-1) over all values at the end of ReadConfig() solved the problem.

Related

How to execute govc/pyvmomi run command which lists output of server on local pc

I am able to list all the vm's using govmomi(not govc which is one line command in terminal) but I am trying to implement this:
govc guest.run -vm $vm_name cat sample.txt from scratch rather than running it via govc. The docs seems to be confusing. Does anyone know how it has been implemented underneath ? And is there something which is similar to this in pyvmomi ? Using Pyvmomi, I was able to execute the command on server from local pc and then store the output in a file on server and then transfer the file from server to local pc. But at later stage, I came across govc guest.run -vm $vm_name cat sample.txt command which gives the output on the console of local pc. Does it also behind the scene store the output temporarily on the server ? If not then how to implement this ?
This is what I have tried when trying to list all the vm's using govmomi, I want to extend this so that it can implement govc run command.
package main
import (
"flag"
"fmt"
"net/url"
"os"
"strings"
"text/tabwriter"
"sort"
"github.com/vmware/govmomi"
"github.com/vmware/govmomi/find"
"github.com/vmware/govmomi/property"
"github.com/vmware/govmomi/vim25/mo"
"github.com/vmware/govmomi/vim25/types"
"golang.org/x/net/context"
)
func GetEnvString(v string, def string) string {
r := os.Getenv(v)
if r == "" {
return def
}
return r
}
// GetEnvBool returns boolean from environment variable.
func GetEnvBool(v string, def bool) bool {
r := os.Getenv(v)
if r == "" {
return def
}
switch strings.ToLower(r[0:1]) {
case "t", "y", "1":
return true
}
return false
}
const (
envURL = "vcenter's_ip"
envUserName = "username#organisation.org"
envPassword = "pwd"
envInsecure = "true"
)
var urlDescription = fmt.Sprintf("ESX or vCenter URL [%s]", envURL)
var urlFlag = flag.String("url", GetEnvString(envURL, "https://username#organisation.org:pwd#ip_of_vcenter/sdk"), urlDescription)
var insecureDescription = fmt.Sprintf("Don't verify the server's certificate chain [%s]", envInsecure)
var insecureFlag = flag.Bool("insecure", GetEnvBool(envInsecure, true), insecureDescription)
func processOverride(u *url.URL) {
envUsername := os.Getenv(envUserName)
envPassword := os.Getenv(envPassword)
// Override username if provided
if envUsername != "" {
var password string
var ok bool
if u.User != nil {
password, ok = u.User.Password()
}
if ok {
u.User = url.UserPassword(envUsername, password)
} else {
u.User = url.User(envUsername)
}
}
// Override password if provided
if envPassword != "" {
var username string
if u.User != nil {
username = u.User.Username()
}
u.User = url.UserPassword(username, envPassword)
}
}
func exit(err error) {
fmt.Fprintf(os.Stderr, "Error: %s\n", err)
os.Exit(1)
}
type ByName []mo.VirtualMachine
func (n ByName) Len() int { return len(n) }
func (n ByName) Swap(i, j int) { n[i], n[j] = n[j], n[i] }
func (n ByName) Less(i, j int) bool { return n[i].Name < n[j].Name }
func main() {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
flag.Parse()
// Parse URL from string
u, err := url.Parse(*urlFlag)
if err != nil {
exit(err)
}
fmt.Println(u)
// Override username and/or password as required
processOverride(u)
// Connect and log in to ESX or vCenter
c, err := govmomi.NewClient(ctx, u, *insecureFlag)
if err != nil {
exit(err)
}
f := find.NewFinder(c.Client, true)
// Find one and only datacenter
dc, err := f.DefaultDatacenter(ctx)
if err != nil {
exit(err)
}
// Make future calls local to this datacenter
f.SetDatacenter(dc)
// Find virtual machines in datacenter
vms, err := f.VirtualMachineList(ctx, "in-temp-vm-2")// if instead of "in-temp-vm-2" *, then it lists down all the vm's
//fmt.Println(vms)
if err != nil {
exit(err)
}
//fmt.Println(vms);
pc := property.DefaultCollector(c.Client)
// Convert datastores into list of references
var refs []types.ManagedObjectReference
for _, vm := range vms {
refs = append(refs, vm.Reference())
}
// Retrieve name property for all vms
var vmt []mo.VirtualMachine
err = pc.Retrieve(ctx, refs, []string{"name"}, &vmt)
if err != nil {
exit(err)
}
tw := tabwriter.NewWriter(os.Stdout, 2, 0, 2, ' ', 0)
fmt.Println("Virtual machines found:", len(vmt))
sort.Sort(ByName(vmt))
for _, vm := range vmt {
fmt.Fprintf(tw, "%s\n", vm.Name)
}
tw.Flush()
}
This code is a foolish attempt to implement govc in golang basically using shell commands like bash.
package main
import (
"bytes"
"fmt"
"log"
//"os"
"os/exec"
)
const ShellToUse = "bash"
func Shellout(command string) (error, string, string) {
var stdout bytes.Buffer
var stderr bytes.Buffer
cmd := exec.Command(ShellToUse, "-c", command)
cmd.Stdout = &stdout
cmd.Stderr = &stderr
err := cmd.Run()
return err, stdout.String(), stderr.String()
}
func main() {
err, out, errout := Shellout("govc guest.run -vm $vm_name python3 sample.py")
if err != nil {
log.Printf("error: %v\n", err)
}
fmt.Println("--- stdout ---")
fmt.Println(out)
// fmt.Println("--- stderr ---")
fmt.Println(errout)
}
And this is my python implementation which executes command and stores it on server in a file and then I take that file from server to local pc, I don't want to store anything on server even temporarily:
from pyVim import connect
from config import *
from pyVmomi import vim, vmodl
import ssl
import os
import requests
service_instance = connect.SmartConnect(host="xxxx", port=aaa,user="yyy" , pwd=pwd,sslContext=ssl._create_unverified_context())
content = service_instance.RetrieveContent()
# # Find a VM
vm = searcher.FindByIp(ip="aaaa", vmSearch=True)
creds = vim.vm.guest.NamePasswordAuthentication(username='root', password=vmpwd)
pm = service_instance.content.guestOperationsManager.processManager
#executes and saves sample.txt into server
ps = vim.vm.guest.ProcessManager.ProgramSpec(programPath='/usr/bin/python', arguments='--version &> sample.txt')
res = pm.StartProgramInGuest(vm, creds, ps)
dest="/Users/username/Desktop/vcenter/sample.txt" #My local pc
src="/root/sample.txt" #Server's directory
fti = content.guestOperationsManager.fileManager.InitiateFileTransferFromGuest(vm, creds, src)
resp=requests.get(fti.url, verify=False)
#Writes into file
with open(dest, 'wb') as f:
f.write(resp.content)

How to convert a string value to the correct reflect.Kind in go?

Is there a generic helper method in Go to convert a string to the correct value based on reflect.Kind?
Or do I need to implement the switch over all kinds myself?
I have a value like "143" as a string and a reflect.Value with kind "UInt16" and like to convert that string value and set it into the UInt16 value of my struct.
My current code looks like:
func setValueFromString(v reflect.Value, strVal string) error {
switch v.Kind() {
case reflect.Int, reflect.Int8, reflect.Int16, reflect.Int32, reflect.Int64:
val, err := strconv.ParseInt(strVal, 0, 64)
if err != nil {
return err
}
if v.OverflowInt(val) {
return errors.New("Int value too big: " + strVal)
}
v.SetInt(val)
case reflect.Uint, reflect.Uint8, reflect.Uint16, reflect.Uint32, reflect.Uint64:
val, err := strconv.ParseUint(strVal, 0, 64)
if err != nil {
return err
}
if v.OverflowUint(val) {
return errors.New("UInt value too big: " + strVal)
}
v.SetUint(val)
case reflect.Float32:
val, err := strconv.ParseFloat(strVal, 32)
if err != nil {
return err
}
v.SetFloat(val)
case reflect.Float64:
val, err := strconv.ParseFloat(strVal, 64)
if err != nil {
return err
}
v.SetFloat(val)
case reflect.String:
v.SetString(strVal)
case reflect.Bool:
val, err := strconv.ParseBool(strVal)
if err != nil {
return err
}
v.SetBool(val)
default:
return errors.New("Unsupported kind: " + v.Kind().String())
}
return nil
}
This works already, but I wonder if this is already implemented somewhere else.
Edit: Answer to the original question ("how to obtain a reflect.Kind from its string representation") is at the end. Answer to your edited question follows:
What you're doing is the fastest and "safest". If you don't want to hassle with that big switch, you may take advantage of e.g. the json package which already contains this switch to decode values from JSON string (in encoding/json/decode.go, unexported function literalStore()).
Your decoding function could look like this:
func Set(v interface{}, s string) error {
return json.Unmarshal([]byte(s), v)
}
A simple call to json.Unmarshal(). Using / testing it:
{
var v int
err := Set(&v, "1")
fmt.Println(v, err)
}
{
var v int
err := Set(&v, "d")
fmt.Println(v, err)
}
{
var v uint32
err := Set(&v, "3")
fmt.Println(v, err)
}
{
var v bool
err := Set(&v, "true")
fmt.Println(v, err)
}
{
var v float32
err := Set(&v, `5.1`)
fmt.Println(v, err)
}
{
var v string
err := Set(&v, strconv.Quote("abc"))
fmt.Println(v, err)
}
One thing to note: when you want to pass a string, that must be quoted, e.g. with strconv.Quote(). Output (try it on the Go Playground):
1 <nil>
0 invalid character 'd' looking for beginning of value
3 <nil>
true <nil>
5.1 <nil>
abc <nil>
If you don't want to require quoted strings (which just complicates things), you may build it into the Set() function:
func Set(v interface{}, s string) error {
if t := reflect.TypeOf(v); t.Kind() == reflect.Ptr &&
t.Elem().Kind() == reflect.String {
s = strconv.Quote(s)
}
return json.Unmarshal([]byte(s), v)
}
And then you may call it with the address of a string variable and a string value unquoted:
var v string
err := Set(&v, "abc")
fmt.Println(v, err)
Try this variant on the Go Playground.
Answer to the original question: how to obtain a reflect.Kind from its string representation:
Declaration of reflect.Kind:
type Kind uint
The different values of reflect.Kinds are constants:
const (
Invalid Kind = iota
Bool
Int
Int8
// ...
Struct
UnsafePointer
)
And the reflect package provides only a single method for the reflect.Kind() type:
func (k Kind) String() string
So as it stands, you cannot obtain a reflect.Kind from its string representation (only the reverse direction is possible by using the Kind.String() method). But it's not that hard to provide this functionality.
What we'll do is we build a map from all the kinds:
var strKindMap = map[string]reflect.Kind{}
We init it like this:
func init() {
for k := reflect.Invalid; k <= reflect.UnsafePointer; k++ {
strKindMap[k.String()] = k
}
}
This is possible and correct because constants are initialized using iota which evaluates to successive untyped integer constants, and the first value is reflect.Invalid and the last is reflect.UnsafePointer.
And now you can obtain reflect.Kind from its string representation by simply indexing this map. A helper function which does that:
func strToKind(s string) reflect.Kind {
k, ok := strKindMap[s]
if !ok {
return reflect.Invalid
}
return k
}
And we're done. Testing / using it:
fmt.Printf("All: %#v\n", strKindMap)
for _, v := range []string{"Hey", "uint8", "ptr", "func", "chan", "interface"} {
fmt.Printf("String: %q, Kind: %v (%#v)\n", v, strToKind(v), strToKind(v))
}
Output (try it on the Go Playground):
All: map[string]reflect.Kind{"int64":0x6, "uint8":0x8, "uint64":0xb, "slice":0x17, "uintptr":0xc, "int8":0x3, "array":0x11, "interface":0x14, "unsafe.Pointer":0x1a, "complex64":0xf, "complex128":0x10, "int":0x2, "uint":0x7, "int16":0x4, "uint16":0x9, "map":0x15, "bool":0x1, "int32":0x5, "ptr":0x16, "string":0x18, "func":0x13, "struct":0x19, "invalid":0x0, "uint32":0xa, "float32":0xd, "float64":0xe, "chan":0x12}
String: "Hey", Kind: invalid (0x0)
String: "uint8", Kind: uint8 (0x8)
String: "ptr", Kind: ptr (0x16)
String: "func", Kind: func (0x13)
String: "chan", Kind: chan (0x12)
String: "interface", Kind: interface (0x14)

go1.6 File method WriteStringfrequent calls led to a large system cache

go1.6 File method WriteStringfrequent calls led to a large system cache.
How to solve this problem.
go env: linux amd64.
Is this a problem of Linux system?
code:
package main
import (
"fmt"
"net/http"
"os"
"time"
)
var logCtxCh chan *http.Request
var accessLogFile *os.File
type HandlerHttp struct{}
func (this *HandlerHttp) ServeHTTP(w http.ResponseWriter, req *http.Request) {
sendAccessLog(req)
w.Write([]byte("Hello Word"))
}
func main() {
s := &http.Server{
Addr: ":8012",
Handler: &HandlerHttp{},
}
logCtxCh = make(chan *http.Request, 500)
go startAcessLog()
err:= s.ListenAndServe()
fmt.Println(err.Error())
}
func startAcessLog() {
for {
select {
case ctx := <-logCtxCh:
handleAccessLog(ctx)
}
}
}
func sendAccessLog(req *http.Request) {
logCtxCh <- req
}
func handleAccessLog(req *http.Request) {
uri := req.RequestURI
ip := req.RemoteAddr
agent := req.UserAgent()
refer := req.Referer()
method := req.Method
now := time.Now().Format("2006-01-02 15:04:05")
logText := fmt.Sprintf("%s %s %s %s %s %s\n",
now,
ip,
method,
uri,
agent,
refer,
)
fileName := fmt.Sprintf("/data/logs/zyapi/access_zyapi%s.log",
time.Now().Format("2006010215"),
)
writeLog(fileName, logText)
}
func writeLog(fileName, logText string) {
var err error
var exist = true
if _, err = os.Stat(fileName); os.IsNotExist(err) {
exist = false
}
if exist == false {
if accessLogFile != nil {
accessLogFile.Sync()
accessLogFile.Close()
}
accessLogFile, err = os.OpenFile(fileName, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, 0644)
if err == nil {
_, err = accessLogFile.WriteString(logText)
}
if err != nil {
fmt.Errorf(err.Error())
}
} else {
if accessLogFile == nil {
accessLogFile, err = os.OpenFile(fileName, os.O_WRONLY|os.O_APPEND, 0666)
if err != nil {
fmt.Errorf(err.Error())
return
}
}
_, err = accessLogFile.WriteString(logText)
if err != nil {
fmt.Errorf(err.Error())
}
}
}
test:
ab -n100000 -c10 -k "http://127.0.0.1:8012/"
ab -n100000 -c10 -k "http://127.0.0.1:8012/"
ab -n100000 -c10 -k "http://127.0.0.1:8012/"
ab -n100000 -c10 -k "http://127.0.0.1:8012/"
ab -n100000 -c10 -k "http://127.0.0.1:8012/"
After running several times the system file cache becomes very large
CONTAINER CPU % MEM USAGE/LIMIT MEM % NET I/O BLOCK I/O
api_8011 38.47% 6.442GB/6.442GB 100.00% 0B/0B 0B/115.4MB
api_8012 36.90% 6.442GB/6.442GB 99.99% 0B/0B 0B/115.6 MB
There's a bunch of things going on, I can't spot the bug right away but these things will help:
Try to use bufio.Writer as much as possible if you are calling file.WriteString, otherwise every single write will be a syscall, hurting performance.
You don't need to use select in you startAccessLog function:
func startAcessLog() {
for ctx := <-logCtxCh {
handleAccessLog(ctx)
}
}
Change your error checks from:
if err != nil {
fmt.Errorf(err.Error())
}
to:
if err != nil {
fmt.Println(err)
}
otherwise you are not printing errors. fmt.Errorf formats a string, like fmt.Sprintf does and returns it as an error. It doesn't print anything at all.
You should guard accessLog with a sync.Mutex or write to it via a channel. Why? Because there's more than one goroutine trying to work with accessLog and you don't want data races to happen.
Doing it via a channel would simplify your writeLog function a log. Currently it's hard to follow the logic. I initially thought you weren't properly closing the file.

Convert port to :port of type string in Golang

How do you convert a port inputted as a int by the user to a string of type ":port" (ie, it should have a ':' in front of it and should be converted to string). The output has to be feed to http.ListenAndServe().
You could (ought to?) use net.JoinHostPort(host, port string).
Convert port to a string with strconv.Itoa.
It'll also handle the case where the host part contains a colon: "host:port". Feed that directly to ListenAndServe.
Here is some sample code:
host := ""
port := 80
str := net.JoinHostPort(host, strconv.Itoa(port))
fmt.Printf("host:port = '%s'", str)
// Can be fed directly to:
// http.ListenAndServe(str, nil)
The above can be run on a playground: https://play.golang.org/p/AL3obKjwcjZ
if err := http.ListenAndServe(fmt.Sprintf(":%d", port), handler); err != nil {
log.Fatal(err)
}
Use strconv.Itoa()
Something like:
p := strconv.Itoa(port)
addr := ":" + p
// or for localhost only
// addr := "localhost:" + p
Then
if err := http.ListenAndServe(addr, nil); err != nil {
log.Fatal("ListenAndServe: ", err)
}

Read a text file, replace its words, output to another text file

So I am trying to make a program in GO to take a text file full of code and convert that into GO code and then save that file into a GO file or text file. I have been trying to figure out how to save the changes I made to the text file, but the only way I can see the changes is through a println statement because I am using strings.replace to search the string array that the text file is stored in and change each occurrence of a word that needs to be changed (ex. BEGIN -> { and END -> }). So is there any other way of searching and replacing in GO I don't know about or is there a way to edit a text file that I don't know about or is this impossible?
Thanks
Here is the code I have so far.
package main
import (
"os"
"bufio"
"bytes"
"io"
"fmt"
"strings"
)
func readLines(path string) (lines []string, errr error) {
var (
file *os.File
part []byte
prefix bool
)
if file, errr = os.Open(path); errr != nil {
return
}
defer file.Close()
reader := bufio.NewReader(file)
buffer := bytes.NewBuffer(make([]byte, 0))
for {
if part, prefix, errr = reader.ReadLine(); errr != nil {
break
}
buffer.Write(part)
if !prefix {
lines = append(lines, buffer.String())
buffer.Reset()
}
}
if errr == io.EOF {
errr = nil
}
return
}
func writeLines(lines []string, path string) (errr error) {
var (
file *os.File
)
if file, errr = os.Create(path); errr != nil {
return
}
defer file.Close()
for _,item := range lines {
_, errr := file.WriteString(strings.TrimSpace(item) + "\n");
if errr != nil {
fmt.Println(errr)
break
}
}
return
}
func FixBegin(lines []string) (errr error) {
var(
a string
)
for i := 0; ; i++ {
a = lines[i];
fmt.Println(strings.Replace(a, "BEGIN", "{", -1))
}
return
}
func FixEnd(lines []string) (errr error) {
var(
a string
)
for i := 0; ; i++ {
a = lines[i];
fmt.Println(strings.Replace(a, "END", "}", -1))
}
return
}
func main() {
lines, errr := readLines("foo.txt")
if errr != nil {
fmt.Println("Error: %s\n", errr)
return
}
for _, line := range lines {
fmt.Println(line)
}
errr = FixBegin(lines)
errr = writeLines(lines, "beer2.txt")
fmt.Println(errr)
errr = FixEnd(lines)
lines, errr = readLines("beer2.txt")
if errr != nil {
fmt.Println("Error: %s\n", errr)
return
}
errr = writeLines(lines, "beer2.txt")
fmt.Println(errr)
}
jnml#fsc-r630:~/src/tmp/SO/13789882$ ls
foo.txt main.go
jnml#fsc-r630:~/src/tmp/SO/13789882$ cat main.go
package main
import (
"bytes"
"io/ioutil"
"log"
)
func main() {
src, err := ioutil.ReadFile("foo.txt")
if err != nil {
log.Fatal(err)
}
src = bytes.Replace(src, []byte("BEGIN"), []byte("{"), -1)
src = bytes.Replace(src, []byte("END"), []byte("}"), -1)
if err = ioutil.WriteFile("beer2.txt", src, 0666); err != nil {
log.Fatal(err)
}
}
jnml#fsc-r630:~/src/tmp/SO/13789882$ cat foo.txt
BEGIN
FILE F(KIND=REMOTE);
EBCDIC ARRAY E[0:11];
REPLACE E BY "HELLO WORLD!";
WRITE(F, *, E);
END.
jnml#fsc-r630:~/src/tmp/SO/13789882$ go run main.go
jnml#fsc-r630:~/src/tmp/SO/13789882$ cat beer2.txt
{
FILE F(KIND=REMOTE);
EBCDIC ARRAY E[0:11];
REPLACE E BY "HELLO WORLD!";
WRITE(F, *, E);
}.
jnml#fsc-r630:~/src/tmp/SO/13789882$
I agree with #jnml wrt using ioutil to slurp the file and to write it back. But I think that the replacing shouldn't be done by multiple passes over []byte. Code and data are strings/text and should be treated as such (even if dealing with non ascii/utf8 encodings requires estra work); a one pass replacement (of all placeholders 'at once') avoids the risk of replacing results of previous changes (even if my regexp proposal must be improved to handle non-trivial tasks).
package main
import(
"fmt"
"io/ioutil"
"log"
"regexp"
"strings"
)
func main() {
// (1) slurp the file
data, err := ioutil.ReadFile("../tmpl/xpl.go")
if err != nil {
log.Fatal("ioutil.ReadFile: ", err)
}
s := string(data)
fmt.Printf("----\n%s----\n", s)
// => function that works for files of (known) other encodings that ascii or utf8
// (2) create a map that maps placeholder to be replaced to the replacements
x := map[string]string {
"BEGIN" : "{",
"END" : "}"}
ks := make([]string, 0, len(x))
for k := range x {
ks = append(ks, k)
}
// => function(s) that gets the keys from maps
// (3) create a regexp that finds the placeholder to be replaced
p := strings.Join(ks, "|")
fmt.Printf("/%s/\n", p)
r := regexp.MustCompile(p)
// => funny letters & order need more consideration
// (4) create a callback function for ..ReplaceAllStringFunc that knows
// about the map x
f := func(s string) string {
fmt.Printf("*** '%s'\n", s)
return x[s]
}
// => function (?) to do Step (2) .. (4) in a reusable way
// (5) do the replacing (s will be overwritten with the result)
s = r.ReplaceAllStringFunc(s, f)
fmt.Printf("----\n%s----\n", s)
// (6) write back
err = ioutil.WriteFile("result.go", []byte(s), 0644)
if err != nil {
log.Fatal("ioutil.WriteFile: ", err)
}
// => function that works for files of (known) other encodings that ascii or utf8
}
output:
go run 13789882.go
----
func main() BEGIN
END
----
/BEGIN|END/
*** 'BEGIN'
*** 'END'
----
func main() {
}
----
If your file size is huge, reading everything in memory might not be possible nor advised. Give BytesReplacingReader a try as it is done replacement in streaming fashion. And it's reasonably performant. If you want to replace two strings (such as BEGIN -> { and END -> }), just need to wrap two BytesReplacingReader over original reader, one for BEGIN and one for END:
r := NewBytesReplacingReader(
NewBytesReplacingReader(inputReader, []byte("BEGIN"), []byte("{"),
[]byte("END"), []byte("}")
// use r normally and all non-overlapping occurrences of
// "BEGIN" and "END" will be replaced with "{" and "}"

Resources