I have read a bit about the security of printf() in C++.
Examples can be found e.g. here.
It left me wondering if fmt.Printf() from golang is safe.
To be more specific if it is safe if the formatted string itself could be forged.
inputString := "String from user"
x := "test"
fmt.Printf(inputString, x, 15)
When trying to replicate the exploits from C++, golang does not seem to be vulnerable.
E.g. fmt.Printf("%s%s%s%s%s%s%s%s%s%s%s%s\n") does not crash the program in golang.
Such an analysis of course is no proof that this would be secure in golang. So i wanted to ask here: Have the developers of go foolproofed its printf function?
Edit: By foolproof I mean that it does not have any unexpected side effects.
I would expect the resulting string to be totally compromised of course.
I would not expect the user to be able to gain privileged information (like the content of variables not passed to printf), or the user to be able to write any memory (e.g. assign a new value to x).
Many of the memory issues in C/C++ are related to null termination and buffer overflows. Golang lacks both of those. Strings are managed resources. Baring a compiler bug, it's not possible to terminate a string in such a way as to escape into the stack.
Take your example, as the example. Due to the variadic nature of the function, having a lot of input handlers with no handlers does not impact the code. As far as printf knows, the format string needs nothing replaced. Since you can't pass in anything destructive, even if your example took a dynamic value for ' a ...interface{}', you're protected by the compiler's string protecting code.
tl;dr: Don't print untrusted data, escape it first. If in doubt or a hurry, use %q instead of %s.
Go's string formatting methods are, indeed, memory-safe, and the specific class of vulnerabilities you're worried about and which were so prevalent in C are not applicable.
However, it is not generally safe in any language to output untrusted, unsanitized input via any means without careful consideration. One well-known example of why is cross-site scripting. Less well-known than XSS attacks, however, are terminal escape sequence attacks.
Common terminals respond to a wide variety of escape sequences that can potentially do nasty things directly, or to help exploit another vulnerability. Attackers can include these sequences in messages to a targeted system - say, in the URL of a request to a webserver - and await an admin cating the logs or similar. This can also be used to hide information - including backspace sequences effectively hides whatever came before from view in a terminal.
The quickest option is to simply use %q instead of %s. This outputs the string as a Go string literal, suitable for use in Go source code. This is convenient and safe for printing, but if you're trying to match a specific format, may not be ideal. (strconv.Quote(s string) string will also perform the same operation directly).
This answer to another question has a more involved but easily customized option using strings.Map, however in the given form it removes all non-printable characters rather than escaping them.
One way or another, sanitize your output. You will save someone a lot of pain later, possibly yourself.
package main
import (
"fmt"
"strconv"
)
func main() {
s := "a string with some control\x08\x08\x08\x08\x08\x08\x08hidden characters"
// Prints with the control characters intact. Terminal output:
// a string with some hidden characters
fmt.Printf("%s\n", s)
// Prints as Go string literal. Terminal output:
// "a string with some control\b\b\b\b\b\b\bhidden characters"
fmt.Printf("%q\n", s)
// Prints as Go string literal, but without surrounding double-quotes.
// Terminal output:
// a string with some control\b\b\b\b\b\b\bhidden characters
x := strconv.Quote(s)
fmt.Printf("%s\n", x[1:len(x)-1])
}
Related
In my experience, we can use replace() filtering && and | to prevent command injection.
Our code need to send base64String to another process, but there is Stored Command Injection scaned by checkmarx, can Convert.ToBase64String(Encoding.UTF8.GetBytes(jsonFromDb)) deal with it? or we still need to do replacement
Good question. I think you should still make sure that you use prepared statements. Base64 only contains the letters of the ABC (upper and lowercase), digits, +, / and = as final padding character. That doesn't provide characters as quotes and such.
However, what would happen if somebody decides to use a different alphabet? What if somebody decides that base 64 is a waste of space? Because that's certainly what it is if you convert existing text to base 64. Then you'd suddenly be in problems.
All in all, using prepared statements is more secure, more performant and the best way of handling command injection. As an added bonus, you'd get rid of that pesky checkmarx.
I'd only use base 64 if there is absolutely no other way to change that other process, and if that process cannot be reached otherwise. In that case you might want to use base64url though, as it is likely you can do even less with _ and - instead of +, / and =.
But to be honest, that code needs to get cleaned up pronto.
I am taking user input in an application I am writing and would like to expand escape sequences that the user enters.
For example if the user enters \\n it will be interpreted into a str \\\\n. I want to, in a general way, interpret that string (into a newline) and similar ones.
I could of course use String::replace() on the most essential ones and live without the rest, but I would prefer a general solution that also handles hex escapes (\x61 is a).
Escapes are usually handled by the lexer / parser (basically they're part of the language grammar), I don't think there's an stdlib function which would manage them as it would be done as a much lower level.
Furthermore, escapes tend to be highly language-specific, in possibly unexpected ways. Rust has a singularly small list of escape sequences, which is probably desirable compared to the garbage available from C's, but I still do not know that you'd want to allow e.g. arbitrary hex or unicode escape sequences.
I would therefore recommend setting up your own explicitly supported list of escapes, though if you really do not want to there are probably third-party packages which can help you.
This question already has answers here:
'*' and '/' not recognized on input by a read statement
(2 answers)
Closed 4 years ago.
I am a scientist programming in Fortran, and I came up with a strange behaviour. In one of my programs I have a string containing several "words", and I want to read all words as substrings. The first word starts with an integer and a wildcard, like "2*something".
When I perform an internal read on that string, I expect to read all wods, but instead, the READ function repeatedly reads the first substring. I do not understand why, nor how to avoid this behaviour.
Below is a minimalist sample program that reproduces this behaviour. I would expect it to read the three substrings and to print "3*a b c" on the screen. Instead, I get "a a a".
What am I doing wrong? Can you please help me and explain what is going on?
I am compiling my programs under GNU/Linux x64 with Gfortran 7.3 (7.3.0-27ubuntu1~18.04).
PROGRAM testread
IMPLICIT NONE
CHARACTER(LEN=1024):: string
CHARACTER(LEN=16):: v1, v2, v3
string="3*a b c"
READ(string,*) v1, v2, v3
PRINT*, v1, v2, v3
END PROGRAM testread
You are using list-directed input (the * format specifier). In list-directed input, a number (n) followed by an asterisk means "repeat this item n times", so it is processed as if the input was a a a b c. You would need to have as input '3*a' b c to get what you want.
I will use this as another opportunity to point out that list-directed I/O is sometimes the wrong choice as its inherent flexibility may not be what you want. That it has rules for things like repeat counts, null values, and undelimited strings is often a surprise to programmers. I also often see programmers complaining that list-directed input did not give an error when expected, because the compiler had an extension or the programmer didn't understand just how liberal the feature can be.
I suggest you pick up a Fortran language reference and carefully read the section on list-directed I/O. You may find you need to use an explicit format or change your program's expectations.
Following the answer of #SteveLionel, here is the relevant part of the reference on list-directed sequential READ statements (in this case, for Intel Fortran, but you could find it for your specific compiler and it won't be much different).
A character string does not need delimiting apostrophes or quotation marks if the corresponding I/O list item is of type default character, and the following is true:
The character string does not contain a blank, comma (,), or slash ( / ).
The character string is not continued across a record boundary.
The first nonblank character in the string is not an apostrophe or a quotation mark.
The leading character is not a string of digits followed by an asterisk.
A nondelimited character string is terminated by the first blank, comma, slash, or end-of-record encountered. Apostrophes and quotation marks within nondelimited character strings are transferred as is.
In total, there are 4 forms of sequential read statements in Fortran, and you may choose the option that best fits your need:
Formatted Sequential Read:
To use this you change the * to an actual format specifier. If you know the length of the strings at advance, this would be as easy as '(a3,a2,a2)'. Or, you could come with a format specifier that matches your data, but this generally demands you knowing the length or format of stuff.
Formatted Sequential List-Directed:
You are currently using this option (the * format descriptor). As we already showed you, this kind of I/O comes with a lot of magic and surprising behavior. What is hitting you is the n*cte thing, that is interpreted as n repetitions of cte literal.
As said by Steve Lionel, you could put quotation marks around the problematic word, so it will be parsed as one-piece. Or, as proposed by #evets, you could split or break your string using the intrinsics index or scan. Another option could be changing your wildcard from asterisk to anything else.
Formatted Namelist:
Well, that could be an option if your data was (or could be) presented in the namelist format, but I really think it's not your case.
Unformatted:
This may not apply to your case because you are reading from a character variable, and an internal READ statement can only be formatted.
Otherwise, you could split your string by means of a function instead of a I/O operation. There is no intrinsic for this, but you could come with one without much trouble (see this thread for reference). As you may have noted already, manipulating strings in fortran is... awkward, at least. There are some libraries out there (like this) that may be useful if you are doing lots of string stuff in Fortran.
I have some ASCII-encoded files containing ascii representations of individual Unicode characters like ..., --, and so on that I'd like to convert to e.g. Unicode ellipsis and en-dash symbols for display purposes. This could be as simple as a simple replace filter over all such mappings (in the right order, to catch things like --- -> — and -- -> –, of course). (note: there are more than just those)
Does there exist a database of all such conversions somewhere? I assume the inverse must exist somehow to be able to gracefully convert unicode to plaintext whenever possible, e.g. … -> ....
It doesn't have to be extremely accurate or anything as long as the conversion is appropriate in most cases and makes sense. The output will be just be displayed to the user and won't be further processed. I could just compile a list myself as I go but it would be nice to save time and avoid duplicating effort if it has already been done.
Thanks!
A comprehensive list isn't a very good idea as there are a lot of Unicode characters that exist for compatibility, or are poorly supported (see my comment). Instead, you probably want to use a curated list/library like SmartyPants (ports/alternatives can be found for most other languages).
A few days ago, I asked why its not possible to store binary data, such as a jpg file into a string variable.
Most of the answers I got said that string is used for textual information such as what I'm writing now.
What is considered textual data though? Bytes of a certain nature represent a jpg file and those bytes could be represented by character byte values...I think. So when we say strings are for textual information, is there some sort of range or list of characters that aren't stored?
Sorry if the question sounds silly. Just trying to 'get it'
I see three major problems with storing binary data in strings:
Most systems assume a certain encoding within string variables - e.g. if it's a UTF-8, UTF-16 or ASCII string. New line characters may also be translated depending on your system.
You should watch out for restrictions on the size of strings.
If you use C style strings, every null character in your data will terminate the string and any string operations performed will only work on the bytes up to the first null.
Perhaps the most important: it's confusing - other developers don't expect to find random binary data in string variables. And a lot of code which works on strings might also get really confused when encountering binary data :)
I would prefer to store binary data as binary, you would only think of converting it to text when there's no other choice since when you convert it to a textual representation it does waste some bytes (not much, but it still counts), that's how they put attachments in email.
Base64 is a good textual representation of binary files.
I think you are referring to binary to text encoding issue. (translate a jpg into a string would require that sort of pre-processing)
Indeed, in that article, some characters are mentioned as not always supported, other can be confusing:
Some systems have a more limited character set they can handle; not only are they not 8-bit clean, some can't even handle every printable ASCII character.
Others have limits on the number of characters that may appear between line breaks.
Still others add headers or trailers to the text.
And a few poorly-regarded but still-used protocols use in-band signaling, causing confusion if specific patterns appear in the message. The best-known is the string "From " (including trailing space) at the beginning of a line used to separate mail messages in the mbox file format.
Whoever told you you can't put 'binary' data into a string was wrong. A string simply represents an array of bytes that you most likely plan on using for textual data... but there is nothing stopping you from putting any data in there you want.
I do have to be careful though, because I don't know what language you are using... and in some languages \0 ends the string.
In C#, you can put any data into a string... example:
byte[] myJpegByteArray = GetBytesFromSomeImage();
string myString = Encoding.ASCII.GetString(myJpegByteArray);
Before internationalization, it didn't make much difference. ASCII characters are all bytes, so strings, character arrays and byte arrays ended up having the same implementation.
These days, though, strings are a lot more complicated, in order to deal with thousands of foreign language characters and the linguistic rules that go with them.
Sure, if you look deep enough, everything is just bits and bytes, but there's a world of difference in how the computer interprets them. The rules for "text" make things look right when it's displayed to a human, but the computer is free to monkey with the internal representation. For example,
In Unicode, there are many encoding systems. Changing between them makes every byte different.
Some languages have multiple characters that are linguistically equivalent. These could switch back and forth when you least expect it.
There are different ways to end a line of text. Unintended translations between CRLF and LF will break a binary file.
Deep down everything is just bytes.
Things like strings and pictures are defined by rules about how to order bytes.
strings for example end in a byte with value 32 (or something else)
jpg's don't
Depends on the language. For example in Python string types (str) are really byte arrays, so they can indeed be used for binary data.
In C the NULL byte is used for string termination, so a sting cannot be used for arbitrary binary data, since binary data could contain null bytes.
In C# a string is an array of chars, and since a char is basically an alias for 16bit int, you can probably get away with storing arbitrary binary data in a string. You might get errors when you try to display the string (because some values might not actually correspond to a legal unicode character), and some operations like case conversions will probably fail in strange ways.
In short it might be possible in some langauges to store arbitrary binary data in strings, but they are not designed for this use, and you may run into all kinds of unforseen trouble. Most languages have a byte-array type for storing arbitrary binary data.
I agree with Jacobus' answer:
In the end all data structures are made up of bytes. (Well, if you go even deeper: of bits). With some abstraction, you could say that a string or a byte array are conventions for programmers, on how to access them.
In this regard, the string is an abstraction for data interpreted as a text. Text was invented for communication among humans, computers or programs do not communicate very well using text. SQL is textual, but is an interface for humans to tell a database what to do.
So in general, textual data, and therefore strings, are primarily for human to human, or human to machine interaction (say for the content of a message box). Using them for something else (e.g. reading or writing binary image data) is possible, but carries lots of risk bacause you are using the data type for something it was not designed to handle. This makes it much more error prone. You may be able to store binary data in strings, mbut just because you are able to shoot yourself in the foot, you should avoid doing so.
Summary: You can do it. But you better don't.
Your original question (c# - What is string really good for?) made very little sense. So the answers didn't make sense, either.
Your original question said "For some reason though, when I write this string out to a file, it doesn't open." Which doesn't really mean much.
Your original question was incomplete, and the answers were misleading and confusing. You CAN store anything in a String. Period. The "strings are for text" answers were there because you didn't provide enough information in your question to determine what's going wrong with your particular bit of C# code.
You didn't provide a code snippet or an error message. That's why it's hard to 'get it' -- you're not providing enough details for us to know what you don't get.