I'm attempting to convert a bytes string to a more "human view" one, but I don't know to which format convert to. Anyway I've already tried without success.
These are some examples:
b'\xe8t\xe6,\xf1:0\x10\xb3\x18\x8b\xfd\x08\x00E\x00\x004M[#\x00#\x06\x92\xae\xc0\xa8\x01h\x97e\x01E\xc4\xd6\x00PW-\xf8\x06h\x17\x9a\x86\x80\x10\x01\x06\\\xe5\x00\x00\x01\x01\x08\n\x00/\xf6\xe8\x81\xae.X'
or
b'\xe8t\xe6,\xf1:0\x10\xb3\x18\x8b\xfd\x08\x00E\x00\x004p\xd5#\x00#\x06o4\xc0\xa8\x01h\x97e\x01E\xc4\xee\x00P\xacw\xb7\xca\xa2\xe9\x96\xf8\x80\x10\x00\xed}E\x00\x00\x01\x01\x08\n\x00/\xf7 \x17$+\xf9'
or
b'\xe8t\xe6,\xf1:0\x10\xb3\x18\x8b\xfd\x08\x00E\x00\x004p\xdd#\x00#\x06o,\xc0\xa8\x01h\x97e\x01E\xc4\xee\x00P\xacw\xbc\x06\xa2\xe9\x98\xda\x80\x10\x00\xf5\x0f\x82\x00\x00\x01\x01\x08\n\x000*\xf0\x17$_\xc6'
or
b'\x01\x00^\x00\x00\x01\xe8t\xe6,\xf1:\x08\x00F\xa0\x00$O\x9d\x00\x00\x01\x021\xef\xc0\xa8\x01\xfe\xe0\x00\x00\x01\x94\x04\x00\x00\x11d\xec\x9b\x00\x00\x00\x00\x02\x00\x00\x00'
I've taken these from sniffing with scapy library, but now I really can't convert them.
Could you please show me the way?
Related
i am working with sockets (some DNS stuff) and i can't figure out how from this:
a = 'www'
make
b = b'\x77\x77\x77'
I know/think i need to:
1) convert each to hex value with hex(ord(char))
2) format it from '0x77' to '\x77'
3) convert it to bytes with bytes(a,'utf-8')
I tried many combinations, but i always failed at 2) and generally i think my steps are too complicated. Is there some simple solution to this?
I'll attempt an answer.
What you wish to do is work with the binary messages being exchanged with a DNS server.
You are wondering how to convert strings and integers into into the binary form.
Have a look at the struct module to pack and unpack binary messages.
You will also need to convert IP addresses from there binary form into strings.
Have a look at socket.inet_ntoa and socket.inet_aton.
Barry
I want to check if the given string to my function is plain text or base64 format. I am able to encode a plain text to base64 format and reverse it.
But I could not figure out any way to validate if the string is already base64 format. Can anyone please suggest correct way of doing this in node js? Is there any API for doing this already available in node js.
Valid base64 strings are a subset of all plain-text strings. Assuming we have a character string, the question is whether it belongs to that subset. One way is what Basit Anwer suggests. Those libraries require installing libicu though. A more portable way is to use the built-in Buffer:
Buffer.from(str, 'base64')
Unfortunately, this decoding function will not complain about non-Base64 characters. It will just ignore non-base64 characters. So, it alone will not help. But you can try encoding it back to base64 and compare the result with the original string:
Buffer.from(str, 'base64').toString('base64') === str
This check will tell whether str is pure base64 or not.
Encoding is byte level.
If you're dealing in strings then all you can do is to guess or keep meta data information with your string to identify
But you can check these libraries out:
https://www.npmjs.com/package/detect-encoding
https://github.com/mooz/node-icu-charset-detector
I am creating a raw MD5-Bytearray of some file and store it in a DB. Then, when I upload the file I validate the Checksum. So far everything works fine.
Now I am trying to display the Checksum in the standard form like, for example, this:
0709bfccaec24cbb5734b905dda8d616
but all I got were some cryptic things, e.g.
[B#1fd3b78a
How do I best get the String?
So the representation I was looking for is appearantly the Hexadezimal representation of the bytes. I converted and saved it as a String:
How to convert a byte array to a hex string in Java?
This is a Python 101 type question, but it had me baffled for a while when I tried to use a package that seemed to convert my string input into bytes.
As you will see below I found the answer for myself, but I felt it was worth recording here because of the time it took me to unearth what was going on. It seems to be generic to Python 3, so I have not referred to the original package I was playing with; it does not seem to be an error (just that the particular package had a .tostring() method that was clearly not producing what I understood as a string...)
My test program goes like this:
import mangler # spoof package
stringThing = """
<Doc>
<Greeting>Hello World</Greeting>
<Greeting>你好</Greeting>
</Doc>
"""
# print out the input
print('This is the string input:')
print(stringThing)
# now make the string into bytes
bytesThing = mangler.tostring(stringThing) # pseudo-code again
# now print it out
print('\nThis is the bytes output:')
print(bytesThing)
The output from this code gives this:
This is the string input:
<Doc>
<Greeting>Hello World</Greeting>
<Greeting>你好</Greeting>
</Doc>
This is the bytes output:
b'\n<Doc>\n <Greeting>Hello World</Greeting>\n <Greeting>\xe4\xbd\xa0\xe5\xa5\xbd</Greeting>\n</Doc>\n'
So, there is a need to be able to convert between bytes and strings, to avoid ending up with non-ascii characters being turned into gobbledegook.
The 'mangler' in the above code sample was doing the equivalent of this:
bytesThing = stringThing.encode(encoding='UTF-8')
There are other ways to write this (notably using bytes(stringThing, encoding='UTF-8'), but the above syntax makes it obvious what is going on, and also what to do to recover the string:
newStringThing = bytesThing.decode(encoding='UTF-8')
When we do this, the original string is recovered.
Note, using str(bytesThing) just transcribes all the gobbledegook without converting it back into Unicode, unless you specifically request UTF-8, viz., str(bytesThing, encoding='UTF-8'). No error is reported if the encoding is not specified.
In python3, there is a bytes() method that is in the same format as encode().
str1 = b'hello world'
str2 = bytes("hello world", encoding="UTF-8")
print(str1 == str2) # Returns True
I didn't read anything about this in the docs, but perhaps I wasn't looking in the right place. This way you can explicitly turn strings into byte streams and have it more readable than using encode and decode, and without having to prefex b in front of quotes.
This is a Python 101 type question,
It's a simple question but one where the answer is not so simple.
In python3, a "bytes" object represents a sequence of bytes, a "string" object represents a sequence of unicode code points.
To convert between from "bytes" to "string" and from "string" back to "bytes" you use the bytes.decode and string.encode functions. These functions take two parameters, an encoding and an error handling policy.
Sadly there are an awful lot of cases where sequences of bytes are used to represent text, but it is not necessarily well-defined what encoding is being used. Take for example filenames on unix-like systems, as far as the kernel is concerned they are a sequence of bytes with a handful of special values, on most modern distros most filenames will be UTF-8 but there is no gaurantee that all filenames will be.
If you want to write robust software then you need to think carefully about those parameters. You need to think carefully about what encoding the bytes are supposed to be in and how you will handle the case where they turn out not to be a valid sequence of bytes for the encoding you thought they should be in. Python defaults to UTF-8 and erroring out on any byte sequence that is not valid UTF-8.
print(bytesThing)
Python uses "repr" as a fallback conversion to string. repr attempts to produce python code that will recreate the object. In the case of a bytes object this means among other things escaping bytes outside the printable ascii range.
TRY THIS:
StringVariable=ByteVariable.decode('UTF-8','ignore')
TO TEST TYPE:
print(type(StringVariable))
Here 'StringVariable' represented as a string. 'ByteVariable' represent as Byte. Its not relevent to question Variables..
Hi Im fairly new to base64 code and could do with some info/help etc. I have recently used it while converting my photos into code as part of an experiment regarding my sculptures, I converted a photo into 152 pages of base 64 using an online converter, just to get started. I was wondering how I can reverse this, and also if I edited out some code would the image change or would it not convert at all!? Like I said I am interested in using this to help my experiments. thanks for any info .
Base64 decode functions will decode it. If you remove some of the base64 code it may not translate back into a valid image or translate into valid base64 code at all depending on how much you remove. Depending on how the image was originally compressed, it MAY give you something, but the results will be incredibly unpredictable... So you'll never know what the result of the change will be on the final product. Part of the base64 standard involves a padded width (which is why some strings end in up to 2 = signs).