I've got this small code snippet to test 2 ways of converting byte slice to string object, one function to allocate a new string object, another uses unsafe pointer arithmetic to construct string*, which doesn't allocate new memory:
package main
import (
"fmt"
"reflect"
"unsafe"
)
func byteToString(b []byte) string {
return string(b)
}
func byteToStringNoAlloc(b []byte) string {
if len(b) == 0 {
return ""
}
sh := reflect.StringHeader{uintptr(unsafe.Pointer(&b[0])), len(b)}
return *(*string)(unsafe.Pointer(&sh))
}
func main() {
b := []byte("hello")
fmt.Printf("1st element of slice: %v\n", &b[0])
str := byteToString(b)
sh := (*reflect.StringHeader)(unsafe.Pointer(&str))
fmt.Printf("New alloc: %v\n", sh)
toStr := byteToStringNoAlloc(b)
shNoAlloc := (*reflect.StringHeader)(unsafe.Pointer(&toStr))
fmt.Printf("No alloc: %v\n", shNoAlloc) // why different from &b[0]
}
I run this program under go 1.13:
1st element of slice: 0xc000076068
New alloc: &{824634204304 5}
No alloc: &{824634204264 5}
I exptect that the "1st element of slice" should print out the same address like "No alloc", but acturally they're very different. Where did I get wrong?
First of all, type conversions are calling a internal functions, for this case it's slicebytetostring.
https://golang.org/src/runtime/string.go?h=slicebytetostring#L75
It does copy of slice's content into new allocated memory.
In the second case you're creating a new header of the slice and cast it into string header the new unofficial holder of slice's content.
The problem of this is that garbage collector doesn't handle such kind of cases and resulting string header will be marked as a single structure which has no relations with the actual slice which holds the actual content, so, your resulting string would be valid only while the actual content holders are alive (don't count this string header itself).
So once garbage collector sweep the actual content, your string will still point to the same address but already freed memory, and you'll get the panic error or undefined behavior if you touch it.
By the way, there's no need to use reflect package and its headers because direct cast already creates new header as a result:
*(*string)(unsafe.Pointer(&byte_slice))
I was browsing the docs, and I found StaticString. It states:
An simple string designed to represent text that is "knowable at compile-time".
I originally thought that String has the same behaviour as NSString, which is known at compile time, but it looks like that I was wrong. So my question is when should we use StaticString instead of a String, and is the only difference is that StaticString is known at compile-time?
One thing I found is
var a: String = "asdf" //"asdf"
var b: StaticString = "adsf" //{(Opaque Value), (Opaque Value), (Opaque Value)}
sizeofValue(a) //24
sizeofValue(b) //17
So it looks like StaticString has a little bit less memory footprint.
It appears that StaticString can hold string literals. You can't assign a variable of type String to it, and it can't be mutated (with +=, for example).
"Knowable at compile time" doesn't mean that the value held by the variable will be determined at compile time, just that any value assigned to it is known at compile time.
Consider this example which does work:
var str: StaticString
for _ in 1...10 {
switch arc4random_uniform(3) {
case 0: str = "zero"
case 1: str = "one"
case 2: str = "two"
default: str = "default"
}
print(str)
}
Any time you can give Swift more information about how a variable is to be used, it can optimize the code using it. By restricting a variable to StaticString, Swift knows the variable won't be mutated so it might be able to store it more efficiently, or access the individual characters more efficiently.
In fact, StaticString could be implemented with just an address pointer and a length. The address it points to is just the place in the static code where the string is defined. A StaticString doesn't need to be reference counted since it doesn't (need to) exist in the heap. It is neither allocated nor deallocated, so no reference count is needed.
"Knowable at compile time" is pretty strict. Even this doesn't work:
let str: StaticString = "hello " + "world"
which fails with error:
error: 'String' is not convertible to 'StaticString'
StaticString is knowable at compile time. This can lead to optimizations. Example:
EDIT: This part doesn't work, see edit below
Suppose you have a function that calculates an Int for some String values for some constants that you define at compile time.
let someString = "Test"
let otherString = "Hello there"
func numberForString(string: String) -> Int {
return string.stringValue.unicodeScalars.reduce(0) { $0 * 1 << 8 + Int($1.value) }
}
let some = numberForString(someString)
let other = numberForString(otherString)
Like this, the function would be executed with "Test" and "Hello there" when it really gets called in the program, when the app starts for example. Definitely at runtime. However if you change your function to take a StaticString
func numberForString(string: StaticString) -> Int {
return string.stringValue.unicodeScalars.reduce(0) { $0 * 1 << 8 + Int($1.value) }
}
the compiler knows that the passed in StaticString is knowable at compile time, so guess what it does? It runs the function right at compile time (How awesome is that!). I once read an article about that, the author inspected the generated assembly and he actually found the already computed numbers.
As you can see this can be useful in some cases like the one mentioned, to not decrease runtime performance for stuff that can be done at compile time.
EDIT: Dániel Nagy and me had a conversation. The above example of mine doesn't work because the function stringValue of StaticString can't be known at compile time (because it returns a String). Here is a better example:
func countStatic(string: StaticString) -> Int {
return string.byteSize // Breakpoint here
}
func count(string: String) -> Int {
return string.characters.count // Breakpoint here
}
let staticString : StaticString = "static string"
let string : String = "string"
print(countStatic(staticString))
print(count(string))
In a release build only the second breakpoint gets triggered whereas if you change the first function to
func countStatic(string: StaticString) -> Int {
return string.stringValue.characters.count // Breakpoint here
}
both breakpoints get triggered.
Apparently there are some methods which can be done at compile time while other can't. I wonder how the compiler figures this out actually.
I'm running into a disconnect between the online documentation and the behavior I see in my programs accessing C structs within GO code. go version says I am using:
go version go1.4.2 linux/amd64
According to the GO CGO documentation:
Within the Go file, C's struct field names that are keywords in Go can be
accessed by prefixing them with an underscore: if x points at a C struct with
a field named "type", x._type accesses the field. C struct fields that cannot
be expressed in Go, such as bit fields or misaligned data, are omitted in the
Go struct, replaced by appropriate padding to reach the next field or the end
of the struct.
I had troubles with this, so made a quick sample program to test it out:
package main
// struct rec
// {
// int i;
// double d;
// char* s;
// };
import "C"
import "fmt"
func main() {
s := "hello world"
r := C.struct_rec{}
r.i = 9
r.d = 9.876
r.s = C.CString(s)
fmt.Printf("\n\tr.i: %d\n\tr.d: %f\n\tr.s: %s\n",
r.i,
r.d,
C.GoString(r.s))
}
When I use underscores as the docs indicate (eg, substitute r._i for r.i above) I get the following compile error:
r._i undefined (type C.struct_rec has no field or method _i)
When I don't use underscores it works fine. I tried this with both pointers and non-pointers. The only other idea I can think of is that maybe it's because I allocated the instances in GO rather than C, is that the case??
Thanks for any help!
The answer is in the very quote you have in your question:
Within the Go file, C's struct field names that are keywords in Go can be accessed by prefixing them with an underscore(…)
i, d, and s are not keywords in Go.
Is there a simple way to format a string in Go without printing the string?
I can do:
bar := "bar"
fmt.Printf("foo: %s", bar)
But I want the formatted string returned rather than printed so I can manipulate it further.
I could also do something like:
s := "foo: " + bar
But this becomes difficult to read when the format string is complex, and cumbersome when one or many of the parts aren't strings and have to be converted first, like
i := 25
s := "foo: " + strconv.Itoa(i)
Is there a simpler way to do this?
Sprintf is what you are looking for.
Example
fmt.Sprintf("foo: %s", bar)
You can also see it in use in the Errors example as part of "A Tour of Go."
return fmt.Sprintf("at %v, %s", e.When, e.What)
1. Simple strings
For "simple" strings (typically what fits into a line) the simplest solution is using fmt.Sprintf() and friends (fmt.Sprint(), fmt.Sprintln()). These are analogous to the functions without the starter S letter, but these Sxxx() variants return the result as a string instead of printing them to the standard output.
For example:
s := fmt.Sprintf("Hi, my name is %s and I'm %d years old.", "Bob", 23)
The variable s will be initialized with the value:
Hi, my name is Bob and I'm 23 years old.
Tip: If you just want to concatenate values of different types, you may not automatically need to use Sprintf() (which requires a format string) as Sprint() does exactly this. See this example:
i := 23
s := fmt.Sprint("[age:", i, "]") // s will be "[age:23]"
For concatenating only strings, you may also use strings.Join() where you can specify a custom separator string (to be placed between the strings to join).
Try these on the Go Playground.
2. Complex strings (documents)
If the string you're trying to create is more complex (e.g. a multi-line email message), fmt.Sprintf() becomes less readable and less efficient (especially if you have to do this many times).
For this the standard library provides the packages text/template and html/template. These packages implement data-driven templates for generating textual output. html/template is for generating HTML output safe against code injection. It provides the same interface as package text/template and should be used instead of text/template whenever the output is HTML.
Using the template packages basically requires you to provide a static template in the form of a string value (which may be originating from a file in which case you only provide the file name) which may contain static text, and actions which are processed and executed when the engine processes the template and generates the output.
You may provide parameters which are included/substituted in the static template and which may control the output generation process. Typical form of such parameters are structs and map values which may be nested.
Example:
For example let's say you want to generate email messages that look like this:
Hi [name]!
Your account is ready, your user name is: [user-name]
You have the following roles assigned:
[role#1], [role#2], ... [role#n]
To generate email message bodies like this, you could use the following static template:
const emailTmpl = `Hi {{.Name}}!
Your account is ready, your user name is: {{.UserName}}
You have the following roles assigned:
{{range $i, $r := .Roles}}{{if $i}}, {{end}}{{.}}{{end}}
`
And provide data like this for executing it:
data := map[string]interface{}{
"Name": "Bob",
"UserName": "bob92",
"Roles": []string{"dbteam", "uiteam", "tester"},
}
Normally output of templates are written to an io.Writer, so if you want the result as a string, create and write to a bytes.Buffer (which implements io.Writer). Executing the template and getting the result as string:
t := template.Must(template.New("email").Parse(emailTmpl))
buf := &bytes.Buffer{}
if err := t.Execute(buf, data); err != nil {
panic(err)
}
s := buf.String()
This will result in the expected output:
Hi Bob!
Your account is ready, your user name is: bob92
You have the following roles assigned:
dbteam, uiteam, tester
Try it on the Go Playground.
Also note that since Go 1.10, a newer, faster, more specialized alternative is available to bytes.Buffer which is: strings.Builder. Usage is very similar:
builder := &strings.Builder{}
if err := t.Execute(builder, data); err != nil {
panic(err)
}
s := builder.String()
Try this one on the Go Playground.
Note: you may also display the result of a template execution if you provide os.Stdout as the target (which also implements io.Writer):
t := template.Must(template.New("email").Parse(emailTmpl))
if err := t.Execute(os.Stdout, data); err != nil {
panic(err)
}
This will write the result directly to os.Stdout. Try this on the Go Playground.
try using Sprintf(); it will not print the output but save it for future purpose.
check this out.
package main
import "fmt"
func main() {
address := "NYC"
fmt.Sprintf("I live in %v", address)
}
when you run this code, it will not output anything. But once you assigned the Sprintf() to a separate variable, it can be used for future purposes.
package main
import "fmt"
func main() {
address := "NYC"
fmt.Sprintf("I live in %v", address)
var city = fmt.Sprintf("lives in %v", address)
fmt.Println("Michael",city)
}
I've created go project for string formatting from template (it allow to format strings in C# or Python style) and by performance tests it is fater than fmt.Sprintf, you could find it here https://github.com/Wissance/stringFormatter
it works in following manner:
func TestStrFormat(t *testing.T) {
strFormatResult, err := Format("Hello i am {0}, my age is {1} and i am waiting for {2}, because i am {0}!",
"Michael Ushakov (Evillord666)", "34", "\"Great Success\"")
assert.Nil(t, err)
assert.Equal(t, "Hello i am Michael Ushakov (Evillord666), my age is 34 and i am waiting for \"Great Success\", because i am Michael Ushakov (Evillord666)!", strFormatResult)
strFormatResult, err = Format("We are wondering if these values would be replaced : {5}, {4}, {0}", "one", "two", "three")
assert.Nil(t, err)
assert.Equal(t, "We are wondering if these values would be replaced : {5}, {4}, one", strFormatResult)
strFormatResult, err = Format("No args ... : {0}, {1}, {2}")
assert.Nil(t, err)
assert.Equal(t, "No args ... : {0}, {1}, {2}", strFormatResult)
}
func TestStrFormatComplex(t *testing.T) {
strFormatResult, err := FormatComplex("Hello {user} what are you doing here {app} ?", map[string]string{"user":"vpupkin", "app":"mn_console"})
assert.Nil(t, err)
assert.Equal(t, "Hello vpupkin what are you doing here mn_console ?", strFormatResult)
}
In your case, you need to use Sprintf() for format string.
func Sprintf(format string, a ...interface{}) string
Sprintf formats according to a format specifier and returns the resulting string.
s := fmt.Sprintf("Good Morning, This is %s and I'm living here from last %d years ", "John", 20)
Your output will be :
Good Morning, This is John and I'm living here from last 20 years.
fmt.SprintF function returns a string and you can format the string the very same way you would have with fmt.PrintF
I came to this page specifically looking for a way to format an error string. So if someone needs help with the same, you want to use the fmt.Errorf() function.
The method signature is func Errorf(format string, a ...interface{}) error.
It returns the formatted string as a value that satisfies the error interface.
You can look up more details in the documentation - https://golang.org/pkg/fmt/#Errorf.
Instead of using template.New, you can just use the new builtin with
template.Template:
package main
import (
"strings"
"text/template"
)
func format(s string, v interface{}) string {
t, b := new(template.Template), new(strings.Builder)
template.Must(t.Parse(s)).Execute(b, v)
return b.String()
}
func main() {
bar := "bar"
println(format("foo: {{.}}", bar))
i := 25
println(format("foo: {{.}}", i))
}
In bytes_test.go I see:
a := Split([]byte(tt.s), []byte(tt.sep), tt.n)
where tt.s and tt.sep are strings. But when I try to do
a := bytes.Split([]byte("test"), []byte("e"), 0)
I get:
cannot convert "test" (type ideal string) to type []uint8 in conversion
cannot convert "e" (type ideal string) to type []uint8 in conversion
The following is valid code using the latest release -- release.2010-03-04 -- which includes, amongst other things, this change: "There is one language change: the ability to convert a string to []byte or []int. This deprecates the strings.Bytes and strings.Runes functions."
package main
import ("bytes"; "fmt")
func main() {
a := bytes.Split([]byte("test"), []byte("e"), 0)
fmt.Println(a)
}
Update to a current release of Go: Installing Go : Keeping up with releases.