Insert values with single quotes in a Postgres table column [duplicate] - string

I have a table test(id,name).
I need to insert values like: user's log, 'my user', customer's.
insert into test values (1,'user's log');
insert into test values (2,''my users'');
insert into test values (3,'customer's');
I am getting an error if I run any of the above statements.
If there is any method to do this correctly please share. I don't want any prepared statements.
Is it possible using sql escaping mechanism?

String literals
Escaping single quotes ' by doubling them up → '' is the standard way and works of course:
'user's log' -- incorrect syntax (unbalanced quote)
'user''s log'
Plain single quotes (ASCII / UTF-8 code 39), mind you, not backticks `, which have no special purpose in Postgres (unlike certain other RDBMS) and not double-quotes ", used for identifiers.
In old versions or if you still run with standard_conforming_strings = off or, generally, if you prepend your string with E to declare Posix escape string syntax, you can also escape with the backslash \:
E'user\'s log'
Backslash itself is escaped with another backslash. But that's generally not preferable.
If you have to deal with many single quotes or multiple layers of escaping, you can avoid quoting hell in PostgreSQL with dollar-quoted strings:
'escape '' with '''''
$$escape ' with ''$$
To further avoid confusion among dollar-quotes, add a unique token to each pair:
$token$escape ' with ''$token$
Which can be nested any number of levels:
$token2$Inner string: $token1$escape ' with ''$token1$ is nested$token2$
Pay attention if the $ character should have special meaning in your client software. You may have to escape it in addition. This is not the case with standard PostgreSQL clients like psql or pgAdmin.
That is all very useful for writing PL/pgSQL functions or ad-hoc SQL commands. It cannot alleviate the need to use prepared statements or some other method to safeguard against SQL injection in your application when user input is possible, though. #Craig's answer has more on that. More details:
SQL injection in Postgres functions vs prepared queries
Values inside Postgres
When dealing with values inside the database, there are a couple of useful functions to quote strings properly:
quote_literal() or quote_nullable() - the latter outputs the unquoted string NULL for null input.
There is also quote_ident() to double-quote strings where needed to get valid SQL identifiers.
format() with the format specifier %L is equivalent to quote_nullable().
Like: format('%L', string_var)
concat() or concat_ws() are typically no good for this purpose as those do not escape nested single quotes and backslashes.

According to PostgreSQL documentation (4.1.2.1. String Constants):
To include a single-quote character within a string constant, write
two adjacent single quotes, e.g. 'Dianne''s horse'.
See also the standard_conforming_strings parameter, which controls whether escaping with backslashes works.

This is so many worlds of bad, because your question implies that you probably have gaping SQL injection holes in your application.
You should be using parameterized statements. For Java, use PreparedStatement with placeholders. You say you don't want to use parameterised statements, but you don't explain why, and frankly it has to be a very good reason not to use them because they're the simplest, safest way to fix the problem you are trying to solve.
See Preventing SQL Injection in Java. Don't be Bobby's next victim.
There is no public function in PgJDBC for string quoting and escaping. That's partly because it might make it seem like a good idea.
There are built-in quoting functions quote_literal and quote_ident in PostgreSQL, but they are for PL/PgSQL functions that use EXECUTE. These days quote_literal is mostly obsoleted by EXECUTE ... USING, which is the parameterised version, because it's safer and easier. You cannot use them for the purpose you explain here, because they're server-side functions.
Imagine what happens if you get the value ');DROP SCHEMA public;-- from a malicious user. You'd produce:
insert into test values (1,'');DROP SCHEMA public;--');
which breaks down to two statements and a comment that gets ignored:
insert into test values (1,'');
DROP SCHEMA public;
--');
Whoops, there goes your database.

In postgresql if you want to insert values with ' in it then for this you have to give extra '
insert into test values (1,'user''s log');
insert into test values (2,'''my users''');
insert into test values (3,'customer''s');

you can use the postrgesql chr(int) function:
insert into test values (2,'|| chr(39)||'my users'||chr(39)||');

When I used Python to insert values into PostgreSQL, I also met the question: column "xxx" does not exist.
The I find the reason in wiki.postgresql:
PostgreSQL uses only single quotes for this (i.e. WHERE name = 'John'). Double quotes are used to quote system identifiers; field names, table names, etc. (i.e. WHERE "last name" = 'Smith').
MySQL uses ` (accent mark or backtick) to quote system identifiers, which is decidedly non-standard.
It means PostgreSQL can use only single quote for field names, table names, etc. So you can not use single quote in value.
My situation is: I want to insert values "the difference of it’s adj for sb and it's adj of sb" into PostgreSQL.
How I figure out this problem:
I replace ' with ’, and I replace " with '. Because PostgreSQL value does not support double quote.
So I think you can use following codes to insert values:
insert into test values (1,'user’s log');
insert into test values (2,'my users');
insert into test values (3,'customer’s');

If you need to get the work done inside Pg:
to_json(value)
https://www.postgresql.org/docs/9.3/static/functions-json.html#FUNCTIONS-JSON-TABLE

You must have to add an extra single quotes -> ' and make doubling quote them up like below examples -> ' ' is the standard way and works of course:
Wrong way: 'user's log'
Right way: 'user''s log'
problem:
insert into test values (1,'user's log');
insert into test values (2,''my users'');
insert into test values (3,'customer's');
Solutions:
insert into test values (1,'user''s log');
insert into test values (2,'''my users''');
insert into test values (3,'customer''s');

Related

Cannot insert question mark in postgres insert

I'm trying to run a simple Postgres SQL insert:
insert into "Resources" values(1, 'How are you?');
But the result after insert is:
ID Data
--- ------
1 How are you$1
I know, to insert characters like single quote, I have to escape it with another single quote like: I can''t do it.
But what to do with ?
Knex interprets ? and ?? as positional bindings. You would normally use them inside a knex.raw() statement to safely inject some kind variable. For example:
knex.raw('UPDATE my_table SET my_column = ?', [someVariable])
Binding parameters like this is often necessary with raw statements to ensure whatever you're injecting is safely escaped.
So that's why you are seeing that behavior. The good news is, you can just escape question marks. From the Knex.js documentation:
To prevent replacement of ? one can use the escape sequence \\?.

How to put triple qoutes around existing string variable?

Let's say I have this variable:
st='MI'
and I want to convert it to:
st=''' 'MI' '''
to use it in a SQL command.
What's the best way to accomplish this?
Thanks in advance!
Tripleor single quoting are just ways of typing strings into source code files.
Once your program is running, your string is already a string, and there is no need to make any cnversion to use it as a parameter to a SQL driver function call.
What you may want i to have a string with an SQL statement that itself contains various (single or double) quote characters. If that is typed in your Python source code file, you can type the triple-quote straight. If you are getting these SQL statements from elsewhere, they are already strings, as I said above.
Now, there are a few instances in which you have a string in a running Python program, or a Python interactive session, that you would like printed, so that you can paste it directly in source code. For these cases you can try the "unicode_escape" codec (and recode it to text so that it does not double your backslashes:
In [56]: print("\n".encode("unicode_escape").decode("utf-8"))
\n

fetching compact version of JSONB in PostgreSQL

How to fetch compact JSONB from PostgreSQL?
All I got when fetching is with spaces:
SELECT data FROM a_table WHERE id = 1; -- data is JSONB column
{"unique": "bla bla", "foo": {"bar": {"in ...
^ ^ ^ ^ ^ --> spaces
What I want is:
{"unique":"bla bla","foo":{"bar":{"in ...
json_strip_nulls() does exactly what you're looking for:
SELECT json_build_object('a', 1);
returns
{"a" : 1}
But
SELECT json_strip_nulls(json_build_object('a', 1));
returns
{"a":1}
This function not only strips nulls as indicated by its function name and as documented, but incidentally also strips insignificant whitespace. The latter is not explicitly documented in PostgreSQL manual.
Tested in PostgreSQL 11.3, but probably works with earlier versions too.
jsonb is rendered in a standardized format on output. You would have to use json instead to preserve insignificant white space. Per documentation:
Because the json type stores an exact copy of the input text, it will
preserve semantically-insignificant white space between tokens, as
well as the order of keys within JSON objects. Also, if a JSON object
within the value contains the same key more than once, all the
key/value pairs are kept. (The processing functions consider the last
value as the operative one.) By contrast, jsonb does not preserve
white space, does not preserve the order of object keys, and does not
keep duplicate object keys.
The whitespace really shouldn't matter for JSON values.
There is a discussion, started in 2016, about a function jsonb_compact() that will solve the problem... But, it could take years (!).
Pretty solution
  (a real solution for this question and this other one)
We must to agree with the PostgreSQL's convention for CAST(var_jsonb AS text). When you need another cast convention, for example to debug or human-readable output, the built-in jsonb_pretty() function is a good choice.
Unfortunately PostgreSQL not offers other choices, like the compact one. So, you can overload jsonb_pretty() with a compact option:
CREATE or replace FUNCTION jsonb_pretty(
jsonb, -- input
compact boolean -- true for compact format
) RETURNS text AS $$
SELECT CASE
WHEN $2=true THEN json_strip_nulls($1::json)::text
ELSE jsonb_pretty($1)
END
$$ LANGUAGE SQL IMMUTABLE;
SELECT jsonb_pretty( jsonb_build_object('a',1, 'bla','bla bla'), true );
-- results {"a":1,"bla":"bla bla"}
Rationale
The JSON standard, RFC 8259 says "... Insignificant whitespace is allowed before or after any of the six structural characters". In other words, the cast from jsonb datatype to text has no canonical form. The PostgreSQL cast convention (using spaces) is arbitrary.
A lot of applications need to minimize a big JSONb output. Two typical ones: minimizing file size of a big JSONb saved by pg_file_write(); output online in a REST interface.
The PostgreSQL team must to appreciate a real CAST procedure, not a parser, but a direct text production from JSONb internal representation.
The workaround — to remove spaces from "JSON text" — is not a simple task, it need a good parser to avoid tampering content. The solution is a parser, it is not a regular expression workaround... And in nowadays the built-in parser is json_strip_nulls(), even as "incidential behavior" parser.

replaceAll quotes with backslashed quotes -- Is that enough?

I'm using replaceAll to replace single quotes with "\\\\'" per a colleague's suggestion, but I'm pretty sure that's not enough to prevent all SQL injections.
I did some googling and found this: http://wiki.postgresql.org/wiki/8.1.4_et._al._Security_Release_Technical_Info
This explains it for PostgreSQL, but does the replacing not work for all SQL managers? (Like, MySQL, for example?)
Also, I think I understand how the explanation I linked works for single backslash, but does it extend to my situation where I'm using four backslashes?
Please note that I'm not very familiar with databases and how they parse input, but this is my chance to learn more! Any insight would be appreciated.
Edit: I've gotten some really helpful, useful answers. My next question is, what kind of input would break my implementation? That is, if you give me input and I prepend all single quotes with four backslashes, what kind of input would you give me to inject SQL code? While I am convinced that my approach is naive and wrong, maybe some examples would better teach me how easy it is to inject SQL against my "prevention".
No, because what about backslashes? for instance if you turn ' into \' then the input \' will become \\' which is an unescaped single quote and a "character literal" backslash. For mysql there is mysql_real_escape_string() which should exist for every platform because its in the MySQL library bindings.
But there is another problem. And that is if you have no quote marks around the data segment. In php this looks like:
$query="select * from user where id=".$_GET[id];
The PoC exploit for this is very simple: http://localhost/vuln.php?id=sleep(10)
Even if you do a mysql_real_escape_string($_GET[id]) its still vulnerable to sqli because the attacker doesn't have to break out of quote marks in order to execute sql. The best solution is Parameterized Queries.
No.
This is not enough, and this is not the way to go. And I can say it without even knowing anything about your data, your SQL or even anything about your application. You should never, ever include any user data directly into your SQL. You should use parameterized statements instead.
Besides if you are asking this question you shouldn't write your own SQL by hand in the first place. Use a good ORM instead. Asking if your home-grown regular expression would make your application safe from SQL injection is like asking if your home-grown memory allocation routine that you have written in Assembly language is safe from buffer overruns - to which I would say: if you are asking this question then you should use a memory-safe language in the first place.
A simple case of SQL injection works like this (in pseudocode):
name = form_params["name"]
year = 2011
sql = "INSERT INTO Students (name, year) " +
"VALUES ('" + name + "', " + year + ");"
database_handle.query(sql)
year is supplied by you, the programmer, so it's not tainted, and can be embedded in the query in any way you find suitable; in this case — as an unquoted number.
But name is supplied by the user and so can be anything. Along comes Bobby Tables and inputs this value:
name = "Robert'); DROP TABLE Students; -- "
And the query becomes
INSERT INTO Students (name, year) VALUES ('Robert');
DROP TABLE Students; -- ', 2011);
That substitution turned your one query into two.
The first one gives an error because of the mismatched row count, but that doesn't matter, because the database is able to unambiguously find and run the second query. The attacker can work around the error by fiddling with the input anyway. The -- is a comment so that the rest of the input is ignored.
Note how data suddenly became code — a typical sign of a security problem.
What the suggested replacement does is this:
name = form_params["name"].regex_replace("'", "\\\\'")
How this works is confusing, hence my earlier comment. The string literal "\\\\'" represents the string \\'. The regex_replace function interprets that as the string \'. The database then sees
... VALUES ('Robert\'); DROP TABLE Students; -- ', 2011);
and interprets that correctly as a quite unusual name.
Among other problems this approach is very fragile. If the strings you use in your language don't substitute \\ as \, if your string substitution function doesn't interpret \\ as \ (if it's not a regex function or it uses $1 instead of \1 for backreferences) you could end up with an even number of slashes like
... VALUES ('Robert\\'); DROP TABLE Students; -- ', 2011);
and no SQL injection will be prevented.
The solution is not to check what the language and library does with all possible input you can think of, or to anticipate what it might do in a future version, but rather to use the facilities provided by the database. These usually come in two flavours:
database-aware escaping, which does exactly the right escaping of any data because the client library matches the server and it knows what the character encoding of the database you are querying is:
sql = "... '" + database_handle.escape(name) + "' ..."
out-of-band data submission (usually with prepared statments), so the data isn't even in the same string as the code:
sql = "... VALUES (:n, :y);"
database_handle.query(sql, n = name, y = year)

Can I be vulnerable to SQL injection by appending input with no whitespace to my query?

I am taking in a string from user input, and splitting it on whitespace (using \w) into an array of strings. I then loop through the array, and append a part of the where clause like this:
query += " AND ( "
+ "field1 LIKE '%" + searchStrings[i] +"%' "
+ " OR field2 LIKE '%" + searchStrings[i] +"%' "
+ " OR field3 LIKE '%" + searchStrings[i] +"%' "
+ ") ";
I feel like this is dangerous, since I am appending user input to my query. However, I know that there isn't any whitespace in any of the search strings, since I split the initial input on whitespace.
Is it possible to attack this via a SQL injection? Giving Robert');DROP TABLE students;-- wouldn't actually drop anything, since there needs to be whitespace in there. In that example, it would not behave properly, but no damage would be done.
Can anyone with more experience fighting SQL injections help me either fix this, or put my mind at ease?
Thanks!
EDIT:
Wow, that is a lot of great input. Thank you everyone who responded. I will investigate full-text search and, at a minimum, parameterize my query.
Just so I can better understand the problem, would it be possible to inject if all whitespace AND single quotes were escaped?
Any time you allow a user to enter data into a query string like this you are vulnerable to SQL injection and it should be avoided like the plague!
You should be very careful how you allow your searchStrings[] array to be populated. You should always append variable data to your query using parameter objects:
+ field1 like #PropertyVal Or field2 like #PropertyVal Or field3 like #PropertyVal etc...
And if you're using SQL Server for example
Query.Parameters.Add(new SqlParameter("PropertyVal", '%' + searchStrings[i] + '%'));
Be very wary how you build a query string that you're going to run against a production server, especially if it has any data of consequence in it!
In your example you mentioned Little Bobby Tables
Robert');DROP TABLE students;--
And cited that because it needs white space, you couldn't do it - but if the malicious user encoded it using something like this:
Robert');Exec(Replace('Drop_Table_students','_',Char(32)));--
I would say - better to be safe and do it the right way. There's no simple way to make sure you catch every scenario otherwise...
Yes, this script did not contain any whitespaces, just encoded characters that SQL decoded back and executed:
http://www.f-secure.com/weblog/archives/00001427.html
The injected script was something like this:
DECLARE%20#S%20NVARCHAR(4000);SET%20#S=CAST(0x440045004300
4C00410052004500200040005400200076006100720063006800610072
00280032003500350029002C0040004300200076006100720063006800
610072002800320035003500290020004400450043004C004100520045
0020005400610062006C0065005F0043007500720073006F0072002000
43005500520053004F005200200046004F0052002000730065006C0065
0063007400200061002E006E0061006D0065002C0062002E006E006100
6D0065002000660072006F006D0020007300790073006F0062006A0065
00630074007300200061002C0073007900730063006F006C0075006D00
6E00730020006200200077006800650072006500200061002E00690064
003D0062002E0069006400200061006E006400200061002E0078007400
7900700065003D00270075002700200061006E0064002000280062002E
00780074007900700065003D003900390020006F007200200062002E00
780074007900700065003D003300350020006…
Wich SQL decoded to :
DECLARE #T varchar(255)'#C
varchar(255) DECLARE Table_Cursor
CURSOR FOR select a.name'b.name from
sysobjects a'syscolumns b where
a.id=b.id and a.xtype='u' and
(b.xtype=99 or b.xtype=35 or b…
and so on, so its possible to do whatever you want with every table in the database without using any whitespaces.
There are just too many ways to get this wrong, that I wouldn't rely if anyone told me "no, this will be safe, because ..."
What about escaping the whitespace in some form (URL-Encode or somethign). What about using non-obvious Unicode whitespace characters that your simple tests don't check for. What if your DB supports some malicious operations that don't require a white space?
Do the correct way: Use a PreparedStatement (or whatever your platform uses for injection-safe parameterization), append and prepend the "%" to the user input and use that as a parameter.
The rule of thumb is: If the string you're appending isn't SQL, it must be escaped using prepared statements or the correct 'escape' function from your DB client library.
I totally agree that query parameters are the absolutely safest way to go. With them you have no risk of SQL injection whatsover (unless you do something stupid with the parameters) and there is no overhead of escaping.
If your DBMS does not support query parameters, then it MUST support string escaping. In the worst case you can try to escape single-quotes yourself, although there still is a Unicode exploit that can circumvent this. However, if your DBMS does not support query parameters, it probably doesn't support Unicode either. :)
Added: Also. queries like you wrote up there are killers for performance - no indexes can be used. I'd advise to look up your DBMS' full-text-indexing capabilities. They are meant exactly for cases such as this.
Yes, they could still inject items, without spaces it might not do much, but it is still a vulnerability.
In general blindly adding user input to a query is not a good idea.
Here's a trivial injection, if I set field1 to this I've listed all rows in your database. This may be bad for security...
'+field1+'
You should use parameters (these are valid in inline SQL too), e.g.
AND Field1 = #Field1

Resources