I'm trying to initialize a byte array as a mix of numbers and a string. I've managed to do it using a pair of appends, or by splitting the string into individual chars, but for the sake of readability is there a way to do it as a single initializer containing a string?
// method 1, with two appends
a := []byte{1, 2}
a = append(a, []byte("foo")...);
a = append(a, 3, 4);
// method 2, splitting the string into chars
b := []byte{1, 2, 'f', 'o', 'o', 3, 4)
What you did in your first attempt in 3 lines is probably more readable that any one-liners:
(You can try all the examples on the Go Playground.)
// Doing one-by-one:
a := []byte{1, 2}
a = append(a, []byte("foo")...)
a = append(a, 3, 4)
fmt.Println(a)
// Using individual chars:
a = []byte{1, 2, 'f', 'o', 'o', 3, 4}
fmt.Println(a)
// Using a single string literal:
a = []byte("\x01\x02foo\x03\x04")
fmt.Println(a)
// Using several "nested" appends:
a = append(append([]byte{1, 2}, []byte("foo")...), 3, 4)
fmt.Println(a)
Except if you create a helper function:
func concat(s ...[]byte) []byte {
var res []byte
for _, v := range s {
res = append(res, v...)
}
return res
}
And then using it:
// With a utility function:
a = concat([]byte{1, 2}, []byte("foo"), []byte{3, 4})
fmt.Println(a)
// With a utility function, formatted differently:
a = concat(
[]byte{1, 2},
[]byte("foo"),
[]byte{3, 4},
)
fmt.Println(a)
You could also do it using a single keyed composite literal and a single copy() call to "insert" the string:
// With keyed literal and copy:
a = []byte{1, 2, 5: 3, 4}
copy(a[2:], "foo")
fmt.Println(a)
Still I don't think it's more readable or worth it.
Concat optimization
As per comments left below, #EliasVanOotegem benchmarked the solution above (using append on an empty slice) and compared it to summing the total capacity of the byte slice required and allocating that memory in one go. The latter turns out to be slightly more efficiend (~20%), so I'll include that version below:
func concat(s ...[]byte) []byte {
c := 0
for _, v := range s {
c += len(v)
}
res := make([]byte, 0, c) // allocate everything
for _, v := range s {
res = append(res, v...)
}
return res
}
I personally would use the following optimized version which does not require slice header assignments as it uses the builtin copy():
func concat(s ...[]byte) []byte {
size := 0
for _, v := range s {
size += len(v)
}
res, i := make([]byte, size), 0
for _, v := range s {
i += copy(res[i:], v)
}
return res
}
I think I misunderstood what you are trying initially. You could write it as a string with hex embedded:
c := []byte("\x01\x02foo\x03\x04")
FWIW : it is possible to remove the cast to []byte(...) in the append call :
a := []byte{65, 66}
a = append(a, "foo"...)
a = append(a, 67, 68)
fmt.Printf("%s", a)
// outputs : ABfooCD
see it on play.golang.org
Related
The task is to add zeros to string elements of 2d slice. So the stdin is [["7" "3" "1"]["2" "9"]] and I need to add zeros from the last element of each slice to the first one. For each step the counter of zeros is incremented by +1. Therefore, stdout is expected to be [["700", "30", "1"]["20", "9"]].
I have tried to do such an algorithm but can't get expected answer. Here is my code:
package main
import (
"fmt"
"strings"
)
func addZero(strs [][]string) [][]string {
zero := "0"
counter := 0
for i := range strs {
for j := range strs[i] {
strs[i][j] += strings.Repeat(zero, counter)
}
counter++
}
return strs
}
func main() {
fmt.Println(addZero([][]string{{"7", "3", "1"}, {"2", "9"}}))// here the result is [[7 3 1] [20 90]]
}
How to change my code to get an expected answer ?
Counting zeros has to reset in each line, so move that code inside the first loop.
Also range goes from index 0, and you want increasing zeroes from the end of lines, so counter has to start from len(strs[i])-1, and you have to decrement it:
func addZero(strs [][]string) [][]string {
for i := range strs {
zero := "0"
counter := len(strs[i]) - 1
for j := range strs[i] {
strs[i][j] += strings.Repeat(zero, counter)
counter--
}
}
return strs
}
With these changes output will be (try it on the Go Playground):
[[700 30 1] [20 9]]
Note that if you would process lines from the end, the suffix to append (the zeros) would increase. So you could ditch strings.Repeat() by keeping and "extending" the previous suffix:
func addZero(strs [][]string) [][]string {
for _, line := range strs {
zeros := ""
for i := len(line) - 1; i >= 0; i-- {
line[i] += zeros
zeros += "0"
}
}
return strs
}
This outputs the same. Try it on the Go Playground.
Also note that strings can be sliced, and the result shares the memory with the sliced string. So it's fast and does not create garbage! You could build a single, long zeros string holding just zeros, and you could slice this long string to have as many zeros as you need to append. This solution avoids any unnecessary string allocations and concatenations:
var zeros = strings.Repeat("0", 1000) // Use the maximum length here
func addZero(strs [][]string) [][]string {
for _, line := range strs {
count := len(line) - 1
for i := range line {
line[i] += zeros[:count-i]
}
}
return strs
}
This again outputs the same, try it on the Go Playground.
Let say I have input and output string where the output will be the frequency of each elements in the string and the char itself
input := "programming"
output := "p2ro2ga2min"
How can I print it based on index after I found the freq of distinct character
This is my code
func countFreq(s string) {
sMap := make(map[string]int)
for _, v := range s {
sMap[string(v)]++
}
for i, v := range sMap {
fmt.Printf("%v%v", i, v)
}
// Don't know what to do next
}
The output of code is
output: n1p1r2o1g2a1m2i1
#icza's answer is great. Here's an alternative that I thought of before but only just got around to writing.
It uses a string to keep track of the rune order but you could use a string builder if speed is important.
func countFreq(s string) {
sMap := make(map[rune]int)
sOut := ""
for _, c := range s {
sMap[c]++
if sMap[c] == 1 {
sOut += string(c)
}
}
for _, c := range sOut {
if sMap[c] > 1 {
fmt.Print(sMap[c])
}
fmt.Printf("%c", c)
}
}
You're counting runes, so use a map of map[rune]int, so you can omit the conversions back to string.
Maps are unordered, so if you want the output in the same order as the input, you can't (shouldn't) iterate over the map.
Once you counted the letters, range over the input letters again, and get the frequency from the map, and remove it. If the count is greater than 1, also print the number.
func countFreq(s string) {
sMap := make(map[rune]int)
for _, v := range s {
sMap[v]++
}
for _, v := range s {
count := sMap[v]
if count == 0 {
continue // Char already printed and removed
}
delete(sMap, v)
if count > 1 {
fmt.Print(count)
}
fmt.Print(string(v))
}
}
Testing it:
for _, s := range []string{"programming", "ab", "aba", "aabcdeac"} {
fmt.Println("In:", s)
fmt.Print("Out: ")
countFreq(s)
fmt.Println()
}
This will output (try it on the Go Playground):
In: programming
Out: p2ro2ga2min
In: ab
Out: ab
In: aba
Out: 2ab
In: aabcdeac
Out: 3ab2cde
Here is my code snippet:
var converter = map[rune]rune {//some data}
sample := "⌘こんにちは"
var tmp string
for _, runeValue := range sample {
fmt.Printf("%+q", runeValue)
tmp = fmt.Sprintf("%+q", runeValue)
}
The output of fmt.Printf("%+q", runeValue) is:
'\u2318'
'\u3053'
'\u3093'
'\u306b'
'\u3061'
'\u306f'
These value are literally rune but as the return type of Sprintf is string, I cannot use it in my map which is [rune]rune.
I was wondering how can I convert string to rune, or in other words how can I handle this problem?
A string is not a single rune, it may contain multiple runes. You may use a simple type conversion to convert a string to a []runes containing all its runes like []rune(sample).
The for range iterates over the runes of a string, so in your example runeValue is of type rune, you may use it in your converter map, e.g.:
var converter = map[rune]rune{}
sample := "⌘こんにちは"
for _, runeValue := range sample {
converter[runeValue] = runeValue
}
fmt.Println(converter)
But since rune is an alias for int32, printing the above converter map will print integer numbers, output will be:
map[8984:8984 12371:12371 12385:12385 12395:12395 12399:12399 12435:12435]
If you want to print characters, use the %c verb of fmt.Printf():
fmt.Printf("%c\n", converter)
Which will output:
map[⌘:⌘ こ:こ ち:ち に:に は:は ん:ん]
Try the examples on the Go Playground.
If you want to replace (switch) certain runes in a string, use the strings.Map() function, for example:
sample := "⌘こんにちは"
result := strings.Map(func(r rune) rune {
if r == '⌘' {
return 'a'
}
if r == 'こ' {
return 'b'
}
return r
}, sample)
fmt.Println(result)
Which outputs (try it on the Go Playground):
abんにちは
If you want the replacements defined by a converter map:
var converter = map[rune]rune{
'⌘': 'a',
'こ': 'b',
}
sample := "⌘こんにちは"
result := strings.Map(func(r rune) rune {
if c, ok := converter[r]; ok {
return c
}
return r
}, sample)
fmt.Println(result)
This outputs the same. Try this one on the Go Playground.
Convert string to rune array:
runeArray := []rune("пример")
Please read to the bottom before marking this as duplicate
I would like to be able to sort an array of strings (or a slice of structs based on one string value) alphabetically, but based on a custom alphabet or unicode letters.
Most times people advise using a collator that supports different pre-defined locales/alphabets. (See this answer for Java), but what can be done for rare languages/alphabets that are not available in these locale bundles?
The language I would like to use is not available in the list of languages supported and usable by Golangs's collate, so I need to be able to define a custom alphabet, or order of Unicode characters/runes for sorting.
Others suggest translate the strings into an english/ASCII sortable alphabet first, and then sort that. That's what's been suggested by a similar question in this solution done in Javascript or this solution in Ruby. But surely there must be a more efficient way to do this with Go.
Is it possible to create a Collator in Go that uses a custom alphabet/character set? Is that what func NewFromTable is for?
It seems that I should be able to use the Reorder function but it looks like this is not yet implemented in the language? The source code shows this:
func Reorder(s ...string) Option {
// TODO: need fractional weights to implement this.
panic("TODO: implement")
}
How can I define a custom alphabet order for comparing and sorting strings in go?
Note beforehand:
The following solution has been cleaned up and optimized, and published as a reusable library here: github.com/icza/abcsort.
Using abcsort, custom-sorting a string slice (using a custom alphabet) is as simple as:
sorter := abcsort.New("bac")
ss := []string{"abc", "bac", "cba", "CCC"}
sorter.Strings(ss)
fmt.Println(ss)
// Output: [CCC bac abc cba]
Custom-sorting a slice of structs by one of the struct field is like:
type Person struct {
Name string
Age int
}
ps := []Person{{Name: "alice", Age: 21}, {Name: "bob", Age: 12}}
sorter.Slice(ps, func(i int) string { return ps[i].Name })
fmt.Println(ps)
// Output: [{bob 12} {alice 21}]
Original answer follows:
We can implement custom sorting that uses a custom alphabet. We just need to create the appropriate less(i, j int) bool function, and the sort package will do the rest.
Question is how to create such a less() function?
Let's start by defining the custom alphabet. Convenient way is to create a string that contains the letters of the custom alphabet, enumerated (ordered) from smallest to highest. For example:
const alphabet = "bca"
Let's create a map from this alphabet, which will tell the weight or order of each letter of our custom alphabet:
var weights = map[rune]int{}
func init() {
for i, r := range alphabet {
weights[r] = i
}
}
(Note: i in the above loop is the byte index, not the rune index, but since both are monotone increasing, both will do just fine for rune weight.)
Now we can create our less() function. To have "acceptable" performance, we should avoid converting the input string values to byte or rune slices. To do that, we can call aid from the utf8.DecodeRuneInString() function which decodes the first rune of a string.
So we do the comparison rune-by-rune. If both runes are letters of the custom alphabet, we may use their weights to tell how they compare to each other. If at least one of the runes are not from our custom alphabet, we will fallback to simple numeric rune comparisons.
If 2 runes at the beginning of the 2 input strings are equal, we proceed to the next runes in each input string. We may do this my slicing the input strings: slicing them does not make a copy, it just returns a new string header that points to the data of the original strings.
All right, now let's see the implementation of this less() function:
func less(s1, s2 string) bool {
for {
switch e1, e2 := len(s1) == 0, len(s2) == 0; {
case e1 && e2:
return false // Both empty, they are equal (not less)
case !e1 && e2:
return false // s1 not empty but s2 is: s1 is greater (not less)
case e1 && !e2:
return true // s1 empty but s2 is not: s1 is less
}
r1, size1 := utf8.DecodeRuneInString(s1)
r2, size2 := utf8.DecodeRuneInString(s2)
// Check if both are custom, in which case we use custom order:
custom := false
if w1, ok1 := weights[r1]; ok1 {
if w2, ok2 := weights[r2]; ok2 {
custom = true
if w1 != w2 {
return w1 < w2
}
}
}
if !custom {
// Fallback to numeric rune comparison:
if r1 != r2 {
return r1 < r2
}
}
s1, s2 = s1[size1:], s2[size2:]
}
}
Let's see some trivial tests of this less() function:
pairs := [][2]string{
{"b", "c"},
{"c", "a"},
{"b", "a"},
{"a", "b"},
{"bca", "bac"},
}
for _, pair := range pairs {
fmt.Printf("\"%s\" < \"%s\" ? %t\n", pair[0], pair[1], less(pair[0], pair[1]))
}
Output (try it on the Go Playground):
"b" < "c" ? true
"c" < "a" ? true
"b" < "a" ? true
"a" < "b" ? false
"bca" < "bac" ? true
And now let's test this less() function in an actual sorting:
ss := []string{
"abc",
"abca",
"abcb",
"abcc",
"bca",
"cba",
"bac",
}
sort.Slice(ss, func(i int, j int) bool {
return less(ss[i], ss[j])
})
fmt.Println(ss)
Output (try it on the Go Playground):
[bca bac cba abc abcb abcc abca]
Again, if performance is important to you, you should not use sort.Slice() as that has to use reflection under the hood, but rather create your own slice type that implements sort.Interface, and in your implementation you can tell how to do it without using reflection.
This is how it could look like:
type CustStrSlice []string
func (c CustStrSlice) Len() int { return len(c) }
func (c CustStrSlice) Less(i, j int) bool { return less(c[i], c[j]) }
func (c CustStrSlice) Swap(i, j int) { c[i], c[j] = c[j], c[i] }
When you want to sort a string slice using the custom alphabet, simply convert your slice to CustStrSlice, so it can be passed directly to sort.Sort() (this type conversion does not make a copy of the slice or its elements, it just changes the type information):
ss := []string{
"abc",
"abca",
"abcb",
"abcc",
"bca",
"cba",
"bac",
}
sort.Sort(CustStrSlice(ss))
fmt.Println(ss)
Output of the above is again (try it on the Go Playground):
[bca bac cba abc abcb abcc abca]
Some things to note:
The default string comparison compares strings byte-wise. That is, if the input strings contain invalid UTF-8 sequences, the actual bytes will still be used.
Our solution is different in this regard, as we decode runes (we have to because we use a custom alphabet in which we allow runes that are not necessarily mapped to bytes 1-to-1 in UTF-8 encoding). This means if the input is not a valid UTF-8 sequence, the behavior might not be consistent with the default ordering. But if your inputs are valid UTF-8 sequences, this will do what you expect it to do.
One last note:
We've seen how a string slice could be custom-sorted. If we have a slice of structs (or a slice of pointers of structs), the sorting algorithm (the less() function) may be the same, but when comparing elements of the slice, we have to compare fields of the elements, not the struct elements themselves.
So let's say we have the following struct:
type Person struct {
Name string
Age int
}
func (p *Person) String() string { return fmt.Sprint(*p) }
(The String() method is added so we'll see the actual contents of the structs, not just their addresses...)
And let's say we want to apply our custom sorting on a slice of type []*Person, using the Name field of the Person elements. So we simply define this custom type:
type PersonSlice []*Person
func (p PersonSlice) Len() int { return len(p) }
func (p PersonSlice) Less(i, j int) bool { return less(p[i].Name, p[j].Name) }
func (p PersonSlice) Swap(i, j int) { p[i], p[j] = p[j], p[i] }
And that's all. The rest is the same, for example:
ps := []*Person{
{Name: "abc"},
{Name: "abca"},
{Name: "abcb"},
{Name: "abcc"},
{Name: "bca"},
{Name: "cba"},
{Name: "bac"},
}
sort.Sort(PersonSlice(ps))
fmt.Println(ps)
Output (try it on the Go Playground):
[{bca 0} {bac 0} {cba 0} {abc 0} {abcb 0} {abcc 0} {abca 0}]
Using table_test.go [1] as a starting point, I came up with the following. The
real work is being done by Builder.Add [2]:
package main
import (
"golang.org/x/text/collate"
"golang.org/x/text/collate/build"
)
type entry struct {
r rune
w int
}
func newCollator(ents []entry) (*collate.Collator, error) {
b := build.NewBuilder()
for _, ent := range ents {
err := b.Add([]rune{ent.r}, [][]int{{ent.w}}, nil)
if err != nil { return nil, err }
}
t, err := b.Build()
if err != nil { return nil, err }
return collate.NewFromTable(t), nil
}
Result:
package main
import "fmt"
func main() {
a := []entry{
{'a', 3}, {'b', 2}, {'c', 1},
}
c, err := newCollator(a)
if err != nil {
panic(err)
}
x := []string{"alfa", "bravo", "charlie"}
c.SortStrings(x)
fmt.Println(x) // [charlie bravo alfa]
}
https://github.com/golang/text/blob/3115f89c/collate/table_test.go
https://pkg.go.dev/golang.org/x/text/collate/build#Builder.Add
I have a mask that contains a binary counting of cpu_ids (0xA00000800000 for 3 CPUs) which I want to convert into a string of comma separated cpu_ids: "0,2,24".
I did the following Go implementation (I am a Go starter). Is it the best way to do it? Especially the handling of byte buffers seems to be inefficient!
package main
import (
"fmt"
"os"
"os/exec"
)
func main(){
cpuMap := "0xA00000800000"
cpuIds = getCpuIds(cpuMap)
fmt.Println(cpuIds)
}
func getCpuIds(cpuMap string) string {
// getting the cpu ids
cpu_ids_i, _ := strconv.ParseInt(cpuMap, 0, 64) // int from string
cpu_ids_b := strconv.FormatInt(cpu_ids_i, 2) // binary as string
var buff bytes.Buffer
for i, runeValue := range cpu_ids_b {
// take care! go returns code points and not the string
if runeValue == '1' {
//fmt.Println(bitString, i)
buff.WriteString(fmt.Sprintf("%d", i))
}
if (i+1 < len(cpu_ids_b)) && (runeValue == '1') {
//fmt.Println(bitString)
buff.WriteString(string(","))
}
}
cpuIds := buff.String()
// remove last comma
cpuIds = cpuIds[:len(cpuIds)-1]
//fmt.Println(cpuIds)
return cpuIds
}
Returns:
"0,2,24"
What you're doing is essentially outputting the indices of the "1"'s in the binary representation from left-to-right, and starting index counting from the left (unusal).
You can achieve the same using bitmasks and bitwise operators, without converting it to a binary string. And I would return a slice of indices instead of its formatted string, easier to work with.
To test if the lowest (rightmost) bit is 1, you can do it like x&0x01 == 1, and to shift a whole number bitwise to the right: x >>= 1. After a shift, the rightmost bit "disappears", and the previously 2nd bit becomes the 1st, so you can test again with the same logic. You may loop until the number is greater than 0 (which means it sill has 1-bits).
See this question for more examples of bitwise operations: Difference between some operators "|", "^", "&", "&^". Golang
Of course if we test the rightmost bit and shift right, we get the bits (indices) in reverse order (compared to what you want), and the indices are counted from right, so we have to correct this before returning the result.
So the solution looks like this:
func getCpuIds(cpuMap string) (r []int) {
ci, err := strconv.ParseInt(cpuMap, 0, 64)
if err != nil {
panic(err)
}
count := 0
for ; ci > 0; count, ci = count+1, ci>>1 {
if ci&0x01 == 1 {
r = append(r, count)
}
}
// Indices are from the right, correct it:
for i, v := range r {
r[i] = count - v - 1
}
// Result is in reverse order:
for i, j := 0, len(r)-1; i < j; i, j = i+1, j-1 {
r[i], r[j] = r[j], r[i]
}
return
}
Output (try it on the Go Playground):
[0 2 24]
If for some reason you need the result as a comma separated string, this is how you can obtain that:
buf := &bytes.Buffer{}
for i, v := range cpuIds {
if i > 0 {
buf.WriteString(",")
}
buf.WriteString(strconv.Itoa(v))
}
cpuIdsStr := buf.String()
fmt.Println(cpuIdsStr)
Output (try it on the Go Playground):
0,2,24