I have a proto file like below:
syntax = "proto2";
package user;
message user_message {
required bytes username = 1;
optional string phonenum = 2;
optional string password = 3;
}
When I receive the message, that is serialised as a String from the sender.
This I try to convert to a JSON object:
base_msg = msg_periodic_pb2.m_apps_message()
base_msg.ParseFromString(message)
base_msg = json_format.MessageToJson(base_msg)
When I print the base_msg, after converting the message to JSON the username gets converted to a base64 string. How can I avoid this?
Check this comment.
"This behavior is actually to be expected, because bytes fields (unlike string fields) can contain non-UTF8 binary data, and since that cannot be directly represented in JSON, we have to base64-encode it."
So, if you really want to have strings. You cannot have the protobuf type as bytes.
Related
I want to send a pickled, encoded version of an object Account across to my server, and then decoding it at the server end and reinstating it as the object with corresponding data, however I am unsure how to convert it back from a string to the bytes (?) data type as an object.
On the clients end, this is essentially what happens:
command = 'insert account'
tempAccount = Account('Isabel', 'password')
pickledAcc = pickle.dumps(tempAccount)
clientCommand = f"{command},{pickledAcc}".encode('utf-8')
client.send(clientCommand)
However on the servers side, it receives an empty string as the pickledAcc part.
I have simplified my code a bit but the I think the essentials are there, but if you need more I can give it lol. Also should mention that I have used the proper length etiquette, i.e. sending a message before this to notify the server how long this message will be. And all of my server infrastructure works properly :)
Basically, I just need to know if it is possible to encode the pickled Account object to send it, or if doing so will not ever work and is stupid to do so.
The problem with the format line is that you insert the __repr__ of the pickledAcc instead of the real bytes. This will not get you the wanted result:
for example:
command = "test"
pickledAcc = pickle.dumps("test_data")
clientCommand = f"{command},{pickledAcc}".encode('utf-8')
Now client command will output:
b"test,b'\\x80\\x03X\\t\\x00\\x00\\x00test_dataq\\x00.'"
as you can see, the representation of the bytes array was encoded to utf-8 ("b\...")
To solve this problem I suggest you will convert the command to bytes array and then send clientCommand as a bytes array instead
hope that helped
Client side:
import base64
##--snip--##
pickledAcc = base64.b64encode(pickledAcc).decode()
clientCommand = f"{command},{pickledAcc}".encode('utf-8')
client.send(clientCommand)
Server Side:
import base64
##--snip--##
pickledAcc = base64.b64decode(pickledAcc)
pickledAcc = pickle.loads(pickledAcc)
I'm new to python3 and i need to get incoming bytes from a socket and store it as a string.
print(input_bytes)
b'data:\x00,\xff'
I don't need to decode the data
byte_input_to_string = input_bytes
print(byte_input_to_string)
b'data:\x00,\xff'
file = open('byte_data.txt','w')
file.write(byte_input_to_string)
file.close()
What i need is to store data as a string, i don't need to decode it. I just want the same data but as a string, so i can parse it, etc.
file.write(str(byte_input_to_string))
I'm using an API which returns a Base64 encoded file that I want to parse and harvest data from. I'm having trouble decoding the Base64, as it comes back with garbled characters. The code I have is below.
Base64 decoder = new Base64()
def jsonSlurper = new JsonSlurper()
def json = jsonSlurper.parseText(Requests.getInventory(app).toString())
String stockB64 = json.getAt("stock")
byte[] decoded = decoder.decode(stockB64)
println(new String(decoded, "US-ASCII"))
I've also tried println(new String(decoded, "UTF-8")) and this returns the same garbled output. I've pasted in an example snipped of the output for reference.
� ���v���
��W`�C�:�f��y�z��A��%J,S���}qF88D q )��'�C�c�X��������+n!��`nn���.��:�g����[��)��f^���c�VK��X�W_����������?4��L���D�������i�9|�X��������\���L�V���gY-K�^����
��b�����~s��;����g���\�ie�Ki}_������
What am I doing wrong here?
You don't need the Base64 class wherever you took it from. You can simply do stockB64.decodeBase64() to get the decoded byte array. Are you sure what you have there is actual text that is encoded. Usually base64 encoded means that this is some binary like an image. If it is text you could have put it as string in the json simply. Maybe save the resulting byte array to a file and then investigate the file type by content.
i have decoded a base64-encoded string using buffers, now i have noticed something funny:
this works fine, outputs the decoded string as utf8
decoded = new Buffer(data.content, 'base64')
console.log('Decoded:' + decoded);
// outputs content of a markdown file
however, this outputs hex characters:
decoded = new Buffer(data.content, 'base64')
console.log(decoded);
// outputs<Buffer 23 20 33 30 32 34 20 66 ...>
why is this or what am i doing wrong? shouldn't the output be the same?
The argument of console.log is formatted internally in node.js using util.format. (See line 52 of https://github.com/joyent/node/blob/v0.11.4/lib/console.js)
When you call console.log(obj);, obj is passed directly to util.format internally. When you call console.log('Decoded: '+obj), the string concatenation forces .toString() to be called on obj first, and then passes the resulting combined string to util.format internally.
So, in the first case, node.js is formatting a String object, in the second case, it's formatting a Buffer object directly.
From the Mozilla Javascript Docs:
Every object has a toString() method that is automatically called when
the object is to be represented as a text value or when an object is
referred to in a manner in which a string is expected.
Node.js buffer docs: Buffer#toString.
I have binary data stored in a file. I am doing this:
byte[] fileBytes = File.ReadAllBytes(#"c:\carlist.dat");
string ascii = Encoding.ASCII.GetString(fileBytes);
This is giving me following result with lot of invalid characters. What am i doing wrong?
?D{F ?x#??4????? NBR-OF-CARSNUMBER-OF-CARS!"#??? NBR-OF-CARS$%??1y0#123?G??#$ NBR-OF-CARS%45??1y# NUMBER-OF-CARSd?
hmm... seems like a save was made from a byte buffer where after NBR-OF-CARS was written some numeric data. If you have an access to the code that saves the file could you check if there are numbers over there and if there are - check does the code converts numbers to string before witing the value into the binary stream.