transforming strings in a golang struct - struct

I've got a json file of AES encrypted secrets. The structure is:
{
"username": "asdf123ASLdf3",
"password": "elisjdvo4etQW"
}
And a struct to hold these values
type Secrets struct {
Username string `json:"username"`
Password string `json:"password"`
}
It's easy to load the encrypted json values into the struct, but what I really want is a struct with the unencrypted values.
So, for each value, I'd like to run it though a function:
aesDecrypt(key string, value string) string
I'm happy to have this done on the first load, or to move everything over into a new struct.
I would like to avoid repeating the json keys or the field names.
What's the best way to do this?
(Also open to other ways to manage encrypted secrets in Go)

One option is to define a custom JSON Unmarshaler. Another is, as you mention, copy it to another struct.
Implementing the Unmarshaler interface
The key insight is knowing that you can override json.Unmarshal's
behaviour by implementing the Unmarshaler interface. In our
case, that means defining a function func (ss *Secrets)
UnmarshalJSON(bb []byte) error that will do the AES Decryption when
you try to unmarshal any JSON to a Secrets.
package main
import "fmt"
import "encoding/json"
type Secrets struct {
Username string `json:"username"`
Password string `json:"password"`
}
func main() {
jj := []byte(`{
"username": "asdf123ASLdf3",
"password": "elisjdvo4etQW"
}`)
var ss Secrets
json.Unmarshal(jj, &ss)
fmt.Println(ss)
}
func aesDecrypt(key, value string) string {
return fmt.Sprintf("'%s' decrypted with key '%s'", value, key)
}
func (ss *Secrets) UnmarshalJSON(bb []byte) error {
var objmap map[string]*string
err := json.Unmarshal(bb, &objmap)
ss.Username = aesDecrypt("my key", *objmap["password"])
ss.Password = aesDecrypt("my key", *objmap["username"])
return err
}
This outputs a Secrets struct:
{'elisjdvo4etQW' decrypted with key 'my key'
'asdf123ASLdf3' decrypted with key 'my key'}
See it in action at the Go Playground.
Copying to another struct
You could simply make a new Secrets struct every time you need to
decrypt the JSON. This could be tedious if you do it alot, or if you
have no need for the intermediate state.
package main
import "fmt"
import "encoding/json"
type Secrets struct {
Username string `json:"username"`
Password string `json:"password"`
}
func main() {
jj := []byte(`{
"username": "asdf123ASLdf3",
"password": "elisjdvo4etQW"
}`)
var ss Secrets
json.Unmarshal(jj, &ss)
decoded := Secrets{
aesDecrypt(ss.Username, "my key"),
aesDecrypt(ss.Password, "my key")}
fmt.Println(decoded)
}
func aesDecrypt(key, value string) string {
return fmt.Sprintf("'%s' decrypted with key '%s'", value, key)
}
Check it out at Go Playground.
This has the same output as above:
{'elisjdvo4etQW' decrypted with key 'my key'
'asdf123ASLdf3' decrypted with key 'my key'}
Obviously, you would use a different version of aesDecrypt, mine's
just a dummy. And, as always, you should actually be checking the
returned errors in your own code.

Related

TypeScript / JavaScript gRPC google.protobuf.Struct cannot be read

I have a TypeScript server trying to read a JSON object using a Struct but it seems to be partially working only for objects containing a "fields" key which then expects an object as value. Nonetheless, a Struct should work with any JSON object.
Using BloomRPC I am trying the following message:
{
"payload": {
"fields": {
"Hello": {
"whatever": 0
}
}
}
}
The server reads:
{ fields: { Hello: {} } }
If I send:
{
"payload": {
"anotherfield": {
"HelloWorld": {
"whatever": 0
}
}
}
}
I get an empty object on the server.
The simplified protobuf file looks like this:
syntax = "proto3";
import "google/protobuf/struct.proto";
// The service definition.
service TestTicketService {
rpc UpdateTicket (UpdateTicketRequest) returns (UpdateTicketResponse);
}
// The request message containing the required ticket information.
message UpdateTicketRequest {
string ticketId = 1;
google.protobuf.Struct payload = 2;
}
// The response message containing any potential error message
message UpdateTicketResponse {
string error = 1;
}
Any idea why google/protobuf/struct.proto doesn't work as expected?
What really confused me is that I was trying to pass normal JSON objects and expecting to read them. The whole point is that from the client side, the JSON object needs to be encoded in a very specific way.
For example:
"payload": {
"fields": {
"name": {
"stringValue": "joe"
},
"age": {
"numberValue": 28
}
}
}
You can figure out the format of the message by looking at the Struct proto file here: https://googleapis.dev/nodejs/asset/latest/v1_doc_google_protobuf_doc_struct.js.html
The idea of a struct is that you can store arbitrary data - but only simple types: null, number, string, bool, array and object.
This maps perfectly to JSON, and this is not by accident.
The google.protobuf.Struct message has a special JSON representation:
The JSON representation for Struct is JSON object.
So you can parse any JSON string into a protobuf Struct, and when serializing to JSON again, you also get the same JSON string again.
It is important to note that the in-memory representation of the parsed Struct is not equal to a JSON object. Protobuf does not have dynamic fields and has to represent JSON data in a more complicated manner. That is why struct.proto defines some other types.
When you want to create a Struct in JavaScript, it is probably the easiest way to just create the JSON object you want:
var jsonObject = {foo: "bar"};
var jsonString = JSON.stringify(jsonObject);
Now you can parse you Struct from this jsonObject or jsonString and put set resulting Struct as a field value in another protobuf message.
Since you are already using TypeScript, it might be worth checking out one of the alternative TypeScript implementations for protobuf.
I am the author of protobuf-ts. Creating a Struct is pretty straight-forward:
let struct = Struct.fromJson({foo: "bar"});
First, install #types/google-protobuf and:
let rqm = new UpdateTicketRequest();
rqm.setTicketId("1");
rqm.setPayload(Struct.fromJavaScript({
Hello:{
whatever: 0,
}
});
//and call the api....
UpdateTicket(rqm);

nodejs read .ini config file

I have a config file. It has variables stored in the following manner.
[general]
webapp=/var/www
data=/home/data
[env]
WEBAPP_DEPLOY=${general:webapp}/storage/deploy
SYSTEM_DEPLOY=${general:data}/deploy
As you can see it has 2 sections general and env. Section env uses the variables from section general.
So I want to read this file into a variable. Let's say config. Here's I want config object to look like:
{
general: {
webapp: '/var/www',
data: '/home/data'
},
env: {
WEBAPP_DEPLOY: '/var/www/storage/deploy',
SYSTEM_DEPLOY: '/home/data/deploy'
}
}
I general I am looking for a config parser for nodejs that supports string interpolation.
I would assume most ini libraries don't include the variable expansion functionality, but with lodash primitives a generic "deep object replacer" isn't too complex.
I've switched the : delimiter for . so has and get can lookup values directly.
const { get, has, isPlainObject, reduce } = require('lodash')
// Match all tokens like `${a.b}` and capture the variable path inside the parens
const re_token = /\${([\w$][\w\.$]*?)}/g
// If a string includes a token and the token exists in the object, replace it
function tokenReplace(value, key, object){
if (!value || !value.replace) return value
return value.replace(re_token, (match_string, token_path) => {
if (has(object, token_path)) return get(object, token_path)
return match_string
})
}
// Deep clone any plain objects and strings, replacing tokens
function plainObjectReplacer(node, object = node){
return reduce(node, (result, value, key) => {
result[key] = (isPlainObject(value))
? plainObjectReplacer(value, object)
: tokenReplace(value, key, object)
return result
}, {})
}
> plainObjectReplacer({ a: { b: { c: 1 }}, d: 'wat', e: '${d}${a.b.c}' })
{ a: { b: { c: 1 } }, d: 'wat', e: 'wat1' }
You'll find most config management tools (like ansible) can do this sort of variable expansion for you before app runtime, at deployment.

Cannot pass immutable value as inout argument: function call returns immutable value

I forked this project, so I am not as familiar with all of the details: https://github.com/nebs/hello-bluetooth/blob/master/HelloBluetooth/NSData%2BInt8.swift.
This is all part of an extension of NSData that the I am using to send 8-bit values to an Arduino.
func int8Value() -> Int8 {
var value: Int8 = 0
copyBytes(to: &UInt8(value), count: MemoryLayout<Int8>.size) //BUG
return value
}
However, it appears in Swift 3 that this now throws an error in the copyBytes section. Although I have seen some solutions such as passing an address in the parameter, I did not want to risk breaking the remaining parts of the code. Any suggestions on what to do for this?
The original code was incorrect. UInt8(value) generates a new, immutable value which you cannot write to. I assume the old compiler just let you get away with it, but it was never correct.
What they meant to do was to write to the expected type, and then convert the type at the end.
extension Data {
func int8Value() -> Int8 {
var value: UInt8 = 0
copyBytes(to: &value, count: MemoryLayout<UInt8>.size)
return Int8(value)
}
}
That said, I wouldn't do it that way today. Data will coerce its values to whatever type you want automatically, so this way is safer and simpler and very general:
extension Data {
func int8ValueOfFirstByte() -> Int8 {
return withUnsafeBytes{ return $0.pointee }
}
}
Or this way, which is specific to ints (and even simpler):
extension Data {
func int8Value() -> Int8 {
return Int8(bitPattern: self[0])
}
}

Get rid of these Optional values

Using Xcode 7 beta, Swift 2.0
I'm saving and loading credentials to keychain, somehow when loading I get "Optional(value)" back, it looks like this is really part of the string as it also displayed like so in a textbox or when sending to API
This is how I save and load credentials now, as you see I've done a lot of extra nil checking to make sure it is not nil or Optional, it is indeed a overuse of explanation marks...
func SaveCredentials(credentials : [String : String!]!) -> Bool
{
if(credentials.count == 2)
{
//only proceed when we have two keys: username and password
let username = credentials["username"]
let password = credentials["password"]
if let usernameStr = username
{//also tried username!=nil && password != nil
if let passwordStr = password
{ //usernameStr and passwordStr is of type String!
let NsDataUsername = usernameStr!.dataUsingEncoding(NSUTF8StringEncoding)
let NsDataPassword = passwordStr!.dataUsingEncoding(NSUTF8StringEncoding)
if(NsDataUsername != nil && NsDataPassword != nil)
{
LocalStorage.saveToKeyChain("username", data: NsDataUsername!)
LocalStorage.saveToKeyChain("password", data: NsDataPassword!)
return true
}
}
}
}
return false
}
func LoadCredentials() -> [String : String!]?
{
let NsDataUsername = LocalStorage.loadFromKeyChain("username")
let NsDataPassword = LocalStorage.loadFromKeyChain("password")
if(NsDataUsername != nil && NsDataPassword != nil)
{
let username : String! = String(NSString(data: NsDataUsername!, encoding: NSUTF8StringEncoding))
let password : String! = String(NSString(data: NsDataPassword!, encoding: NSUTF8StringEncoding))
if let usernameStr = username
{
if let passwordStr = password
{ // password is of type String!, passwordStr is of type String
var credentials : [String: String!] = [String : String]()
credentials["username"] = usernameStr
credentials["password"] = passwordStr
return credentials
}
}
}
return nil
}
And when I send to Api, this is my method that also requires a non-optional string. This method does work when logging in, getting strings from text fields, but does not filter out that Optional when coming from keychain.
func LoginUser(email : String!, password : String!)
{
print("LoginUser(email : \(email), password: \(password))")
var parameters = [String : AnyObject]()
parameters["UserName"] = email
parameters["Password"] = password
......
The strings that I send to the SaveCredentials method, are the same that the user logged in with:
func LoginLocalAccount(email : String!, password : String!)
{
databaseAPI.LoginUser(email!, password: password!) //login goes just fine
saveCredentials(email!, password: password!) //manages to get Optional in it..
}
I suspect it has something to do with saving and loading from keychain, for interests, this is what I use to save and load from keychain.
I want to get rid of them because when the app starts, it loads the credentials and tries to login at my API. Ofcourse I get an error back that the username is not a valid e-mail, because it is Optional(email#adress.com)
You're overusing !. You don't need them. Try to learn more about implicitly unwrapped optionals, optionals, ... Your code is a mess (no offense, everybody's learning).
Back to your optional problem, it's caused by this line:
let username : String! = String(NSString(data: NsDataUsername!, encoding: NSUTF8StringEncoding))
convenience init?(data: NSData, encoding: UInt) - inner part utilizes failable initializer, so, NSString? is the result. Then initialization of String with optional NSString? produces optional as well. But, it has no sense at all do it in this way.
First part - remove optional
Utilizing new guard:
guard let loadedPassword = NSString(data: passwordData, encoding: NSUTF8StringEncoding) else {
fatalError("Ooops")
}
loadedPassword contains NSString (not NSString?) now.
Second part - NSString -> String
You did probably read (if not, read) Strings and Characters about bridging, ... If you can freely exchange NSString with String, you can think that you're done:
var dict = [String:String]()
dict["password"] = loadedPassword
Nope. It produces following error:
NSString is not implicitly convertible to String; did you mean to
use 'as' to explicitly convert?
Slight change and now you're done:
var dict = [String:String]()
dict["password"] = loadedPassword as String
Complete example
let password = "Hallo"
guard let passwordData = password.dataUsingEncoding(NSUTF8StringEncoding) else {
fatalError("Ooops")
}
// save/load to/from keychain
guard let loadedPassword = NSString(data: passwordData, encoding: NSUTF8StringEncoding) else {
fatalError("Ooops")
}
var dict = [String:String]()
dict["password"] = loadedPassword as String
print(dict) // "[password: Hallo]\n"

GoLang put string in map

So, I'm trying to add a string to an existing map that is created from toml.
http://hastebin.com/vayolavose
When I try and build I get the error:
./web.go:56: arguments to copy have different element types: []proxy.Address and string
How would I go about converting it? I've been trying this for the past like 4 hours.
Thanks
while,the code below is your source code
func handleAddFunc(w http.ResponseWriter, r *http.Request) {
backend := r.FormValue("backend")
key := r.FormValue("key")
if !isAuthorized(key) {
respond(w, r, 403, "")
return
}
w.Header().Set("Content-Type", "text/plain")
if !readConfig() {
return
}
activeAddrs = make([]proxy.Address, len(config.Proxy.ServerAddrs))
backendAddr = make([]proxy.Address, len(backend))
copy(backendAddr, config.Proxy.ServerAddrs)
copy(backendAddr, backend)
loadBalancer.SetAddrs(backendAddr)
fmt.Fprintf(w, "Input value of ", backend, "and here is the byte", backendAddr)
}
your code's error, is copy(backendAddr, backend), variable backend is a string value from the request from, you may change this into []proxy.Address, such as (consider I donnot know the struct of proxy.Address ):
var backendAddr = []proxy.Address{}
for _,str := range strings.split(backend,","){
backendAddr = append(backendAddr, &proxy.Address(str))
}

Resources