I'm trying to run a simple Postgres SQL insert:
insert into "Resources" values(1, 'How are you?');
But the result after insert is:
ID Data
--- ------
1 How are you$1
I know, to insert characters like single quote, I have to escape it with another single quote like: I can''t do it.
But what to do with ?
Knex interprets ? and ?? as positional bindings. You would normally use them inside a knex.raw() statement to safely inject some kind variable. For example:
knex.raw('UPDATE my_table SET my_column = ?', [someVariable])
Binding parameters like this is often necessary with raw statements to ensure whatever you're injecting is safely escaped.
So that's why you are seeing that behavior. The good news is, you can just escape question marks. From the Knex.js documentation:
To prevent replacement of ? one can use the escape sequence \\?.
Related
I have a table test(id,name).
I need to insert values like: user's log, 'my user', customer's.
insert into test values (1,'user's log');
insert into test values (2,''my users'');
insert into test values (3,'customer's');
I am getting an error if I run any of the above statements.
If there is any method to do this correctly please share. I don't want any prepared statements.
Is it possible using sql escaping mechanism?
String literals
Escaping single quotes ' by doubling them up → '' is the standard way and works of course:
'user's log' -- incorrect syntax (unbalanced quote)
'user''s log'
Plain single quotes (ASCII / UTF-8 code 39), mind you, not backticks `, which have no special purpose in Postgres (unlike certain other RDBMS) and not double-quotes ", used for identifiers.
In old versions or if you still run with standard_conforming_strings = off or, generally, if you prepend your string with E to declare Posix escape string syntax, you can also escape with the backslash \:
E'user\'s log'
Backslash itself is escaped with another backslash. But that's generally not preferable.
If you have to deal with many single quotes or multiple layers of escaping, you can avoid quoting hell in PostgreSQL with dollar-quoted strings:
'escape '' with '''''
$$escape ' with ''$$
To further avoid confusion among dollar-quotes, add a unique token to each pair:
$token$escape ' with ''$token$
Which can be nested any number of levels:
$token2$Inner string: $token1$escape ' with ''$token1$ is nested$token2$
Pay attention if the $ character should have special meaning in your client software. You may have to escape it in addition. This is not the case with standard PostgreSQL clients like psql or pgAdmin.
That is all very useful for writing PL/pgSQL functions or ad-hoc SQL commands. It cannot alleviate the need to use prepared statements or some other method to safeguard against SQL injection in your application when user input is possible, though. #Craig's answer has more on that. More details:
SQL injection in Postgres functions vs prepared queries
Values inside Postgres
When dealing with values inside the database, there are a couple of useful functions to quote strings properly:
quote_literal() or quote_nullable() - the latter outputs the unquoted string NULL for null input.
There is also quote_ident() to double-quote strings where needed to get valid SQL identifiers.
format() with the format specifier %L is equivalent to quote_nullable().
Like: format('%L', string_var)
concat() or concat_ws() are typically no good for this purpose as those do not escape nested single quotes and backslashes.
According to PostgreSQL documentation (4.1.2.1. String Constants):
To include a single-quote character within a string constant, write
two adjacent single quotes, e.g. 'Dianne''s horse'.
See also the standard_conforming_strings parameter, which controls whether escaping with backslashes works.
This is so many worlds of bad, because your question implies that you probably have gaping SQL injection holes in your application.
You should be using parameterized statements. For Java, use PreparedStatement with placeholders. You say you don't want to use parameterised statements, but you don't explain why, and frankly it has to be a very good reason not to use them because they're the simplest, safest way to fix the problem you are trying to solve.
See Preventing SQL Injection in Java. Don't be Bobby's next victim.
There is no public function in PgJDBC for string quoting and escaping. That's partly because it might make it seem like a good idea.
There are built-in quoting functions quote_literal and quote_ident in PostgreSQL, but they are for PL/PgSQL functions that use EXECUTE. These days quote_literal is mostly obsoleted by EXECUTE ... USING, which is the parameterised version, because it's safer and easier. You cannot use them for the purpose you explain here, because they're server-side functions.
Imagine what happens if you get the value ');DROP SCHEMA public;-- from a malicious user. You'd produce:
insert into test values (1,'');DROP SCHEMA public;--');
which breaks down to two statements and a comment that gets ignored:
insert into test values (1,'');
DROP SCHEMA public;
--');
Whoops, there goes your database.
In postgresql if you want to insert values with ' in it then for this you have to give extra '
insert into test values (1,'user''s log');
insert into test values (2,'''my users''');
insert into test values (3,'customer''s');
you can use the postrgesql chr(int) function:
insert into test values (2,'|| chr(39)||'my users'||chr(39)||');
When I used Python to insert values into PostgreSQL, I also met the question: column "xxx" does not exist.
The I find the reason in wiki.postgresql:
PostgreSQL uses only single quotes for this (i.e. WHERE name = 'John'). Double quotes are used to quote system identifiers; field names, table names, etc. (i.e. WHERE "last name" = 'Smith').
MySQL uses ` (accent mark or backtick) to quote system identifiers, which is decidedly non-standard.
It means PostgreSQL can use only single quote for field names, table names, etc. So you can not use single quote in value.
My situation is: I want to insert values "the difference of it’s adj for sb and it's adj of sb" into PostgreSQL.
How I figure out this problem:
I replace ' with ’, and I replace " with '. Because PostgreSQL value does not support double quote.
So I think you can use following codes to insert values:
insert into test values (1,'user’s log');
insert into test values (2,'my users');
insert into test values (3,'customer’s');
If you need to get the work done inside Pg:
to_json(value)
https://www.postgresql.org/docs/9.3/static/functions-json.html#FUNCTIONS-JSON-TABLE
You must have to add an extra single quotes -> ' and make doubling quote them up like below examples -> ' ' is the standard way and works of course:
Wrong way: 'user's log'
Right way: 'user''s log'
problem:
insert into test values (1,'user's log');
insert into test values (2,''my users'');
insert into test values (3,'customer's');
Solutions:
insert into test values (1,'user''s log');
insert into test values (2,'''my users''');
insert into test values (3,'customer''s');
I'm currently trying to add some dbeaver parameters to my script.
The simple SQL Looks like this:
SELECT * FROM CONTRACT WHERE CONTRACT_NUMER = :CONTRACTNUMBER;
Now, when I run the Statement, dbeaver prompts me for the contract number which I type in. (The column is a varchar) Unfortunately it only works, if I put my Input Parameter in single Apostrophe.
When I just type in the string the following error message appears:
LE123990123 IS NOT VALID IN THE CONTEXT WHERE IT IS USED. SQLCODE=-206, SQLSTATE=42703, DRIVER=4.19.49
How do I make this Parameter being treated as a String? DB2 11 is in use here.
You MUST put quote for CHAR and VARCHAR variable.
If you look bottom in BIND PARAMETERS WINDOWS, you can read:
"Use Tab to switch. String values must be quoted. You can use expression in values"
I'm trying to run a simple query with node-mysql like this:
client.query("UPDATE mytable SET emote='wtf?' WHERE id=5");
And I get the error
Error: too few parameters given
Because it found a question mark and expects parameters. How the heck do I tell it to ignore the question mark!? Escaping it with \ doesn't work. Passing ['?'] as a parameter is completely out of the question - what if I don't know the amount of question marks the query contains?
Currently I'm using a workaround that counts the amount of question marks in the query and passes that amount of question marks as parameters. But what if I were to actually need the parameters for what they were meant to do? I shiver at the thought.
So to summarize - How do I tell client.query() to ignore question marks?
Use this:
client.query('UPDATE mytable SET emote=? WHERE id=5', ['wtf?']);
Edit:
Passing ['?'] as a parameter is completely out of the question - what
if I don't know the amount of question marks the query contains?
I skipped over this. Why would you not know how many question marks the query contains? Why would you have an unknown query with question marks?
it's an unsolved bug: #118: Client.prototype.format shouldn't treat "?" as placeholder inside string literals
When I try to recreate your scenario, the repeat use of single quotes causes a problem:
client.query('UPDATE mytable SET emote='wtf?' WHERE id=5');
Specifically I see "SyntaxError: Unexpected identifier" when I try to launch the Node instance.
However, I'm able to run an equivalent query just fine, when I switch the outer quotes to double quotes, like so:
mysql.query("SELECT * FROM stories WHERE title='Test?'");
For comparison, this is the parameterized version of my query:
mysql.query('SELECT * FROM stories WHERE title=?',[req.params.title]);
I'm using replaceAll to replace single quotes with "\\\\'" per a colleague's suggestion, but I'm pretty sure that's not enough to prevent all SQL injections.
I did some googling and found this: http://wiki.postgresql.org/wiki/8.1.4_et._al._Security_Release_Technical_Info
This explains it for PostgreSQL, but does the replacing not work for all SQL managers? (Like, MySQL, for example?)
Also, I think I understand how the explanation I linked works for single backslash, but does it extend to my situation where I'm using four backslashes?
Please note that I'm not very familiar with databases and how they parse input, but this is my chance to learn more! Any insight would be appreciated.
Edit: I've gotten some really helpful, useful answers. My next question is, what kind of input would break my implementation? That is, if you give me input and I prepend all single quotes with four backslashes, what kind of input would you give me to inject SQL code? While I am convinced that my approach is naive and wrong, maybe some examples would better teach me how easy it is to inject SQL against my "prevention".
No, because what about backslashes? for instance if you turn ' into \' then the input \' will become \\' which is an unescaped single quote and a "character literal" backslash. For mysql there is mysql_real_escape_string() which should exist for every platform because its in the MySQL library bindings.
But there is another problem. And that is if you have no quote marks around the data segment. In php this looks like:
$query="select * from user where id=".$_GET[id];
The PoC exploit for this is very simple: http://localhost/vuln.php?id=sleep(10)
Even if you do a mysql_real_escape_string($_GET[id]) its still vulnerable to sqli because the attacker doesn't have to break out of quote marks in order to execute sql. The best solution is Parameterized Queries.
No.
This is not enough, and this is not the way to go. And I can say it without even knowing anything about your data, your SQL or even anything about your application. You should never, ever include any user data directly into your SQL. You should use parameterized statements instead.
Besides if you are asking this question you shouldn't write your own SQL by hand in the first place. Use a good ORM instead. Asking if your home-grown regular expression would make your application safe from SQL injection is like asking if your home-grown memory allocation routine that you have written in Assembly language is safe from buffer overruns - to which I would say: if you are asking this question then you should use a memory-safe language in the first place.
A simple case of SQL injection works like this (in pseudocode):
name = form_params["name"]
year = 2011
sql = "INSERT INTO Students (name, year) " +
"VALUES ('" + name + "', " + year + ");"
database_handle.query(sql)
year is supplied by you, the programmer, so it's not tainted, and can be embedded in the query in any way you find suitable; in this case — as an unquoted number.
But name is supplied by the user and so can be anything. Along comes Bobby Tables and inputs this value:
name = "Robert'); DROP TABLE Students; -- "
And the query becomes
INSERT INTO Students (name, year) VALUES ('Robert');
DROP TABLE Students; -- ', 2011);
That substitution turned your one query into two.
The first one gives an error because of the mismatched row count, but that doesn't matter, because the database is able to unambiguously find and run the second query. The attacker can work around the error by fiddling with the input anyway. The -- is a comment so that the rest of the input is ignored.
Note how data suddenly became code — a typical sign of a security problem.
What the suggested replacement does is this:
name = form_params["name"].regex_replace("'", "\\\\'")
How this works is confusing, hence my earlier comment. The string literal "\\\\'" represents the string \\'. The regex_replace function interprets that as the string \'. The database then sees
... VALUES ('Robert\'); DROP TABLE Students; -- ', 2011);
and interprets that correctly as a quite unusual name.
Among other problems this approach is very fragile. If the strings you use in your language don't substitute \\ as \, if your string substitution function doesn't interpret \\ as \ (if it's not a regex function or it uses $1 instead of \1 for backreferences) you could end up with an even number of slashes like
... VALUES ('Robert\\'); DROP TABLE Students; -- ', 2011);
and no SQL injection will be prevented.
The solution is not to check what the language and library does with all possible input you can think of, or to anticipate what it might do in a future version, but rather to use the facilities provided by the database. These usually come in two flavours:
database-aware escaping, which does exactly the right escaping of any data because the client library matches the server and it knows what the character encoding of the database you are querying is:
sql = "... '" + database_handle.escape(name) + "' ..."
out-of-band data submission (usually with prepared statments), so the data isn't even in the same string as the code:
sql = "... VALUES (:n, :y);"
database_handle.query(sql, n = name, y = year)
I am taking in a string from user input, and splitting it on whitespace (using \w) into an array of strings. I then loop through the array, and append a part of the where clause like this:
query += " AND ( "
+ "field1 LIKE '%" + searchStrings[i] +"%' "
+ " OR field2 LIKE '%" + searchStrings[i] +"%' "
+ " OR field3 LIKE '%" + searchStrings[i] +"%' "
+ ") ";
I feel like this is dangerous, since I am appending user input to my query. However, I know that there isn't any whitespace in any of the search strings, since I split the initial input on whitespace.
Is it possible to attack this via a SQL injection? Giving Robert');DROP TABLE students;-- wouldn't actually drop anything, since there needs to be whitespace in there. In that example, it would not behave properly, but no damage would be done.
Can anyone with more experience fighting SQL injections help me either fix this, or put my mind at ease?
Thanks!
EDIT:
Wow, that is a lot of great input. Thank you everyone who responded. I will investigate full-text search and, at a minimum, parameterize my query.
Just so I can better understand the problem, would it be possible to inject if all whitespace AND single quotes were escaped?
Any time you allow a user to enter data into a query string like this you are vulnerable to SQL injection and it should be avoided like the plague!
You should be very careful how you allow your searchStrings[] array to be populated. You should always append variable data to your query using parameter objects:
+ field1 like #PropertyVal Or field2 like #PropertyVal Or field3 like #PropertyVal etc...
And if you're using SQL Server for example
Query.Parameters.Add(new SqlParameter("PropertyVal", '%' + searchStrings[i] + '%'));
Be very wary how you build a query string that you're going to run against a production server, especially if it has any data of consequence in it!
In your example you mentioned Little Bobby Tables
Robert');DROP TABLE students;--
And cited that because it needs white space, you couldn't do it - but if the malicious user encoded it using something like this:
Robert');Exec(Replace('Drop_Table_students','_',Char(32)));--
I would say - better to be safe and do it the right way. There's no simple way to make sure you catch every scenario otherwise...
Yes, this script did not contain any whitespaces, just encoded characters that SQL decoded back and executed:
http://www.f-secure.com/weblog/archives/00001427.html
The injected script was something like this:
DECLARE%20#S%20NVARCHAR(4000);SET%20#S=CAST(0x440045004300
4C00410052004500200040005400200076006100720063006800610072
00280032003500350029002C0040004300200076006100720063006800
610072002800320035003500290020004400450043004C004100520045
0020005400610062006C0065005F0043007500720073006F0072002000
43005500520053004F005200200046004F0052002000730065006C0065
0063007400200061002E006E0061006D0065002C0062002E006E006100
6D0065002000660072006F006D0020007300790073006F0062006A0065
00630074007300200061002C0073007900730063006F006C0075006D00
6E00730020006200200077006800650072006500200061002E00690064
003D0062002E0069006400200061006E006400200061002E0078007400
7900700065003D00270075002700200061006E0064002000280062002E
00780074007900700065003D003900390020006F007200200062002E00
780074007900700065003D003300350020006…
Wich SQL decoded to :
DECLARE #T varchar(255)'#C
varchar(255) DECLARE Table_Cursor
CURSOR FOR select a.name'b.name from
sysobjects a'syscolumns b where
a.id=b.id and a.xtype='u' and
(b.xtype=99 or b.xtype=35 or b…
and so on, so its possible to do whatever you want with every table in the database without using any whitespaces.
There are just too many ways to get this wrong, that I wouldn't rely if anyone told me "no, this will be safe, because ..."
What about escaping the whitespace in some form (URL-Encode or somethign). What about using non-obvious Unicode whitespace characters that your simple tests don't check for. What if your DB supports some malicious operations that don't require a white space?
Do the correct way: Use a PreparedStatement (or whatever your platform uses for injection-safe parameterization), append and prepend the "%" to the user input and use that as a parameter.
The rule of thumb is: If the string you're appending isn't SQL, it must be escaped using prepared statements or the correct 'escape' function from your DB client library.
I totally agree that query parameters are the absolutely safest way to go. With them you have no risk of SQL injection whatsover (unless you do something stupid with the parameters) and there is no overhead of escaping.
If your DBMS does not support query parameters, then it MUST support string escaping. In the worst case you can try to escape single-quotes yourself, although there still is a Unicode exploit that can circumvent this. However, if your DBMS does not support query parameters, it probably doesn't support Unicode either. :)
Added: Also. queries like you wrote up there are killers for performance - no indexes can be used. I'd advise to look up your DBMS' full-text-indexing capabilities. They are meant exactly for cases such as this.
Yes, they could still inject items, without spaces it might not do much, but it is still a vulnerability.
In general blindly adding user input to a query is not a good idea.
Here's a trivial injection, if I set field1 to this I've listed all rows in your database. This may be bad for security...
'+field1+'
You should use parameters (these are valid in inline SQL too), e.g.
AND Field1 = #Field1