I'm adding some capabilities to an api to allow third parties to store user data. I know some users may already base64 encode their user ids before submitting them through the api, others might not.
I've done some checking on double encoding (encoding base64 of an already base64 encoded string), and it doesn't SEEM to be causing any problems.
From my understanding, it isn't possible to check if a string is base64 encoded.
Is there something here I should be looking out for down the line? is there another way I should be doing this, or is it safe?
Also, i'm cleaning the data like this.
$eid=preg_replace("/[^a-zA-Z0-9\/=+]/", "",base64_encode(#$_GET['eid']));
that should be safe to store in the database as it strips out any suspect characters after the string is encoded. But I'll need to return the non-encoded string to through the API.
So at some point I'll need to do echo base64_decode($eid); And it seems to me that this could be an opportunity for a hacker to run malicious code through my server.
Is that right?
Related
I need to build a XMPP client. The server PLAIN mechanism to auth. (using zlib compression I think)
I captured trafic from other xmpp system thats use PLAIN mechanism and the text appear to be Base64 (id + token) ADc1Y2M2OWY0MzQwMTUwMjgyOWIwMWY2MDAyN2E0NDE2ADE1YTk0NzM3NTRiYjY2MGExMGYzYTA5MzA5NWQxMmY3 is what the client return. I put that into a Base64 decoder and its give me this : 75cc69f43401502829b01f60027a441615a9473754bb660a10f3a093095d12f7.
When I encode this using a Base64 encoder, Its give me something else than the first Base64 string (NzVjYzY5ZjQzNDAxNTAyODI5YjAxZjYwMDI3YTQ0MTYxNWE5NDczNzU0YmI2NjBhMTBmM2EwOTMwOTVkMTJmNw)
Can someone explain me? I couldn't find anything on google.
Edit:
https://xmpp.org/extensions/xep-0034.html#example-3
the result of your decoding is not correct, in fact the decoded value
contains two binary values that can't be displayed as a character
(here substituted by a �):
�75cc69f43401502829b01f60027a4416�15a9473754bb660a10f3a093095d12f7.
What you encoded then is based on a string in which the two binary
values are not present, so you encoded basically something different
and got of course a different result.
From jps
If I am using base64 to prevent enumeration attacks, is base64 good enough.
36667 order id = corresponds to the base code TCxJ1cxT5Is=
Is it possible to reverse engineer the base64 here back to the 36667?
Thanks
I want to check if the given string to my function is plain text or base64 format. I am able to encode a plain text to base64 format and reverse it.
But I could not figure out any way to validate if the string is already base64 format. Can anyone please suggest correct way of doing this in node js? Is there any API for doing this already available in node js.
Valid base64 strings are a subset of all plain-text strings. Assuming we have a character string, the question is whether it belongs to that subset. One way is what Basit Anwer suggests. Those libraries require installing libicu though. A more portable way is to use the built-in Buffer:
Buffer.from(str, 'base64')
Unfortunately, this decoding function will not complain about non-Base64 characters. It will just ignore non-base64 characters. So, it alone will not help. But you can try encoding it back to base64 and compare the result with the original string:
Buffer.from(str, 'base64').toString('base64') === str
This check will tell whether str is pure base64 or not.
Encoding is byte level.
If you're dealing in strings then all you can do is to guess or keep meta data information with your string to identify
But you can check these libraries out:
https://www.npmjs.com/package/detect-encoding
https://github.com/mooz/node-icu-charset-detector
Having searched around for a while now, I believe my problem may not be directly related to what others had. I am using unicode chars in forms (using angularjs for client-side) and noticed that the UTF8 strings didn't display on the server logs properly. Thus I decided to base64.encode all strings on the client side before submitting to the server (nodejs/express4). The JSON data arrives properly to the server, but when I try to convert it from base64 to UTF8 using a buffer I'm getting different symbols. I tested the strings on http://www.base64decode.org/ and they decode fine. Can anyone suggest what I might be doing wrong?
Example char: σ, base64="z4M=".
On the server this line decodes all JSON values to UTF8:
Object.keys(req.body).forEach(function(key) { req.body[key] = new Buffer(req.body[key], 'base64').toString('utf8'); });
And the "σ" char becomes "Ο" on the server. Anyone can assist?
Thus I decided to base64.encode all strings on the client side before submitting to the server (nodejs/express4).
No need to, really. Probably the thing you were doing wrong with utf-8 json is also wrong now.
Try to debug that.
noticed that the UTF8 strings didn't display on the server logs properly.
What do they display?
And on what OS are you?
Did you look at the logs with a hex viewer?
To me this looks like a typical "I have an a problem X, thought my solution half the way, but I am stuck with a sub-problem Y". Go back to X and attack it the right way (no base64).
User inserts a string in a html form input on browser. This string is saved in database. How this string is encoded and decoded at each stage based on character encoding?
Flow as per technology stack used: browser --> ajax post --> spring mvc -->hibernate -->mysql db
You can expect that the browser post is an URL encoded UTF-8. Within the Java JVM, the string uses UTF-16, therefore roughly doubling the size taken if it is English text. Hibernate is part of that and it does not really care about the encoding, although it does pass around with connection strings as described next (hibernate.connection.url property).
The UTF-16 string is then translated by the JDBC driver which, in case of MySQL, will use the characterEncoding property inside the connection string. It helps if this matches the encoding of the database declared in CREATE DATABASE statement, avoiding another re-encoding.
Finally, "latin" is not a name of a specific character set or encoding. You probably mean ISO 8859-1, also known as Latin-1. This is not a good choice for a web server as it will not be able to represent most non-English strings. You should use UTF-8 in the database and in the connection string, ending up with UTF-8 -> UTF-16 -> UTF-8 which is a safe and reasonably efficient sequence (not counting any encoding that might be happening in the browser itself).
If you decide to alter the database to use UTF-8, be careful about changing the encoding at table level, too. Each table may use its own encoding and it does not change automatically.