Using the following hashing method to store a database object - security

I would like to create a valid variable identifier that can be used across any database object. It seems Oracle has the strictest requirements and only allows 30 chars for a (non-DB) object name, so something like [_a-zA-Z]\W{,29}. My thinking was to base32-encode an md5, such as:
>>> base64.b32encode(hashlib.md5(b'asdf').digest()).rstrip(b'=')
# b'SEXMQA5SZZE6JJKBA2GUSWVVOA'
# len=26
Does that seem like an acceptable approach?

Related

Arrays of composite types in PostgreSQL via NodeJS

I'm using Node.JS ("pg" package) to connect to a PostgreSQL database hosted on Heroku. I need to create a column in my table that will contain an array of different data types. By looking at other questions previously asked on Stackoverflow, I understand i can create composite data types that I can use to declare the array with. Like:
create type my_item as (
field_1 text,
field_2 text,
field_3 text,
field_4 number
);
However, I don't understand how to implement this when using Node.JS. Where do I put it in my files and at what point do I run it?
I have an index.JS file containing my Pool instance and the database access info. My functions are stored in a models folder. Each function has its own SqlString variable which is then passed to the query. Like:
export async function getScores() {
const data = await query(`SELECT * FROM score`);
return data.rows;
}
Appreciate any help.
There is no such thing as array of different composite types in Postgresql. You might need to store the column as json/jsonb type instead and deal with them at the application level. Or create a superset type of all possible types in the array and deal with NULLs at the application level. That only works if the subset types don't overlap different types on the same key.
Also the main usecase for composites is related to INSERT/UPDATE/DELETE queries, aka anything that requires value interpolation from the application. Of course it's no use in your example code.

Node-Postgres/Knex returning CITEXT[] as a string in JS, instead of an array of strings

I'm using Knex, which itself uses package "pg" (aka "node-postgres").
If you SELECT some rows from a table with a TEXT[] column, all is well... in JS you get an array of strings.
But if you're using a CITEXT[] column, instead you just get back a string in JS like:
"{First-element,Second-element}"
Normally when you want to instruct the pg package on how to return specific postgres types, you can do something like this:
import {types} from 'pg';
types.setTypeParser(types.builtins.TIMESTAMPTZ, 'text');
types.setTypeParser(types.builtins.TIMESTAMP, 'text');
types.setTypeParser(types.builtins.DATE, 'text');
types.setTypeParser(types.builtins.TIME, 'text');
types.setTypeParser(types.builtins.TIMETZ, 'text');
The types.builtins.* constants have values that are hardcoded OID numbers for the known built-in types in postgres. Those OID numbers are the same across all postgres installations.
However due to the fact that CITEXT[] is an extension, the OID numbers for the CITEXT + CITEXT[] types will be different on every server, e.g. with the following SQL query:
SELECT typname, oid, typarray FROM pg_type WHERE typname like '%citext%';
On my development server I get:
typname|oid |typarray|
-------|-----|--------|
citext |17459|17464 |
_citext|17464|0 |
But on my production server I get:
typname|oid |typarray|
-------|-----|--------|
citext |18618|18623 |
_citext|18623|0 |
How can I solve this?
Some hacky options that I really don't want to do are:
Find out all the different OID values for all my servers and hard code them in - very hacky and really don't want to do this.
Write code specifically for every table/column that manually converts the strings to array - also hacky and repetitive
When the node process initializes, get the server's OID value for the server and then call the types.setTypeParser() function with that dynamic value - also not very good
How can I solve this for all tables/columns without these hacky solutions?
I don't believe there is any way how to do it without querying DB.
I would probably query the correct OID number before starting the node app and store it to an environment variable and then initialize pg types with the value from process.env.
That is also a bit hacky, but at least the hack is mostly encapsulated out of the application code.

Is there a way to pass a parameter to google bigquery to be used in their "IN" function

I'm currently writing an app that accesses google bigquery via their "#google-cloud/bigquery": "^2.0.6" library. In one of my queries I have a where clause where i need to pass a list of ids. If I use UNNEST like in their example and pass an array of strings, it works fine.
https://cloud.google.com/bigquery/docs/parameterized-queries
However, I have found that UNNEST can be really slow and just want to use IN on its own and pass in a string list of ids. No matter what format of string list I send, the query returns null results. I think this is because of the way they convert parameters in order to avoid sql injection. I have to use a parameter because I, myself want to avoid SQL injection attacks on my app. If i pass just one id it works fine, but if i pass a list it blows up so I figure it has something to do with formatting, but I know my format is correct in terms of what IN would normally expect i.e. IN ('', '')
Has anyone been able to just pass a param to IN and have it work? i.e. IN (#idParam)?
We declare params like this at the beginning of the script:
DECLARE var_country_ids ARRAY<INT64> DEFAULT [1,2,3];
and use like this:
WHERE if(var_country_ids is not null,p.country_id IN UNNEST(var_country_ids),true) AND ...
as you see we let NULL and array notation as well. We don't see issues with speed.

How to sanitize for SQL injection [duplicate]

This question's answers are a community effort. Edit existing answers to improve this post. It is not currently accepting new answers or interactions.
If user input is inserted without modification into an SQL query, then the application becomes vulnerable to SQL injection, like in the following example:
$unsafe_variable = $_POST['user_input'];
mysql_query("INSERT INTO `table` (`column`) VALUES ('$unsafe_variable')");
That's because the user can input something like value'); DROP TABLE table;--, and the query becomes:
INSERT INTO `table` (`column`) VALUES('value'); DROP TABLE table;--')
What can be done to prevent this from happening?
The correct way to avoid SQL injection attacks, no matter which database you use, is to separate the data from SQL, so that data stays data and will never be interpreted as commands by the SQL parser. It is possible to create an SQL statement with correctly formatted data parts, but if you don't fully understand the details, you should always use prepared statements and parameterized queries. These are SQL statements that are sent to and parsed by the database server separately from any parameters. This way it is impossible for an attacker to inject malicious SQL.
You basically have two options to achieve this:
Using PDO (for any supported database driver):
$stmt = $pdo->prepare('SELECT * FROM employees WHERE name = :name');
$stmt->execute([ 'name' => $name ]);
foreach ($stmt as $row) {
// Do something with $row
}
Using MySQLi (for MySQL):
Since PHP 8.2+ we can make use of execute_query() which prepares, binds parameters, and executes SQL statement in one method:
$result = $dbConnection->execute_query('SELECT * FROM employees WHERE name = ?', [$name]);
while ($row = $result->fetch_assoc()) {
// Do something with $row
}
Up to PHP8.1:
$stmt = $dbConnection->prepare('SELECT * FROM employees WHERE name = ?');
$stmt->bind_param('s', $name); // 's' specifies the variable type => 'string'
$stmt->execute();
$result = $stmt->get_result();
while ($row = $result->fetch_assoc()) {
// Do something with $row
}
If you're connecting to a database other than MySQL, there is a driver-specific second option that you can refer to (for example, pg_prepare() and pg_execute() for PostgreSQL). PDO is the universal option.
Correctly setting up the connection
PDO
Note that when using PDO to access a MySQL database real prepared statements are not used by default. To fix this you have to disable the emulation of prepared statements. An example of creating a connection using PDO is:
$dbConnection = new PDO('mysql:dbname=dbtest;host=127.0.0.1;charset=utf8mb4', 'user', 'password');
$dbConnection->setAttribute(PDO::ATTR_EMULATE_PREPARES, false);
$dbConnection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
In the above example, the error mode isn't strictly necessary, but it is advised to add it. This way PDO will inform you of all MySQL errors by means of throwing the PDOException.
What is mandatory, however, is the first setAttribute() line, which tells PDO to disable emulated prepared statements and use real prepared statements. This makes sure the statement and the values aren't parsed by PHP before sending it to the MySQL server (giving a possible attacker no chance to inject malicious SQL).
Although you can set the charset in the options of the constructor, it's important to note that 'older' versions of PHP (before 5.3.6) silently ignored the charset parameter in the DSN.
Mysqli
For mysqli we have to follow the same routine:
mysqli_report(MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT); // error reporting
$dbConnection = new mysqli('127.0.0.1', 'username', 'password', 'test');
$dbConnection->set_charset('utf8mb4'); // charset
Explanation
The SQL statement you pass to prepare is parsed and compiled by the database server. By specifying parameters (either a ? or a named parameter like :name in the example above) you tell the database engine where you want to filter on. Then when you call execute, the prepared statement is combined with the parameter values you specify.
The important thing here is that the parameter values are combined with the compiled statement, not an SQL string. SQL injection works by tricking the script into including malicious strings when it creates SQL to send to the database. So by sending the actual SQL separately from the parameters, you limit the risk of ending up with something you didn't intend.
Any parameters you send when using a prepared statement will just be treated as strings (although the database engine may do some optimization so parameters may end up as numbers too, of course). In the example above, if the $name variable contains 'Sarah'; DELETE FROM employees the result would simply be a search for the string "'Sarah'; DELETE FROM employees", and you will not end up with an empty table.
Another benefit of using prepared statements is that if you execute the same statement many times in the same session it will only be parsed and compiled once, giving you some speed gains.
Oh, and since you asked about how to do it for an insert, here's an example (using PDO):
$preparedStatement = $db->prepare('INSERT INTO table (column) VALUES (:column)');
$preparedStatement->execute([ 'column' => $unsafeValue ]);
Can prepared statements be used for dynamic queries?
While you can still use prepared statements for the query parameters, the structure of the dynamic query itself cannot be parametrized and certain query features cannot be parametrized.
For these specific scenarios, the best thing to do is use a whitelist filter that restricts the possible values.
// Value whitelist
// $dir can only be 'DESC', otherwise it will be 'ASC'
if (empty($dir) || $dir !== 'DESC') {
$dir = 'ASC';
}
To use the parameterized query, you need to use either Mysqli or PDO. To rewrite your example with mysqli, we would need something like the following.
<?php
mysqli_report(MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT);
$mysqli = new mysqli("server", "username", "password", "database_name");
$variable = $_POST["user-input"];
$stmt = $mysqli->prepare("INSERT INTO table (column) VALUES (?)");
// "s" means the database expects a string
$stmt->bind_param("s", $variable);
$stmt->execute();
The key function you'll want to read up on there would be mysqli::prepare.
Also, as others have suggested, you may find it useful/easier to step up a layer of abstraction with something like PDO.
Please note that the case you asked about is a fairly simple one and that more complex cases may require more complex approaches. In particular:
If you want to alter the structure of the SQL based on user input, parameterized queries are not going to help, and the escaping required is not covered by mysql_real_escape_string. In this kind of case, you would be better off passing the user's input through a whitelist to ensure only 'safe' values are allowed through.
Every answer here covers only part of the problem.
In fact, there are four different query parts which we can add to SQL dynamically: -
a string
a number
an identifier
a syntax keyword
And prepared statements cover only two of them.
But sometimes we have to make our query even more dynamic, adding operators or identifiers as well.
So, we will need different protection techniques.
In general, such a protection approach is based on whitelisting.
In this case, every dynamic parameter should be hardcoded in your script and chosen from that set.
For example, to do dynamic ordering:
$orders = array("name", "price", "qty"); // Field names
$key = array_search($_GET['sort'], $orders)); // if we have such a name
$orderby = $orders[$key]; // If not, first one will be set automatically.
$query = "SELECT * FROM `table` ORDER BY $orderby"; // Value is safe
To ease the process I wrote a whitelist helper function that does all the job in one line:
$orderby = white_list($_GET['orderby'], "name", ["name","price","qty"], "Invalid field name");
$query = "SELECT * FROM `table` ORDER BY `$orderby`"; // sound and safe
There is another way to secure identifiers - escaping but I rather stick to whitelisting as a more robust and explicit approach. Yet as long as you have an identifier quoted, you can escape the quote character to make it safe. For example, by default for mysql you have to double the quote character to escape it. For other other DBMS escaping rules would be different.
Still, there is an issue with SQL syntax keywords (such as AND, DESC and such), but white-listing seems the only approach in this case.
So, a general recommendation may be phrased as
Any variable that represents an SQL data literal, (or, to put it simply - an SQL string, or a number) must be added through a prepared statement. No Exceptions.
Any other query part, such as an SQL keyword, a table or a field name, or an operator - must be filtered through a white list.
Update
Although there is a general agreement on the best practices regarding SQL injection protection, there are still many bad practices as well. And some of them too deeply rooted in the minds of PHP users. For instance, on this very page there are (although invisible to most visitors) more than 80 deleted answers - all removed by the community due to bad quality or promoting bad and outdated practices. Worse yet, some of the bad answers aren't deleted, but rather prospering.
For example, there(1) are(2) still(3) many(4) answers(5), including the second most upvoted answer suggesting you manual string escaping - an outdated approach that is proven to be insecure.
Or there is a slightly better answer that suggests just another method of string formatting and even boasts it as the ultimate panacea. While of course, it is not. This method is no better than regular string formatting, yet it keeps all its drawbacks: it is applicable to strings only and, like any other manual formatting, it's essentially optional, non-obligatory measure, prone to human error of any sort.
I think that all this because of one very old superstition, supported by such authorities like OWASP or the PHP manual, which proclaims equality between whatever "escaping" and protection from SQL injections.
Regardless of what PHP manual said for ages, *_escape_string by no means makes data safe and never has been intended to. Besides being useless for any SQL part other than string, manual escaping is wrong, because it is manual as opposite to automated.
And OWASP makes it even worse, stressing on escaping user input which is an utter nonsense: there should be no such words in the context of injection protection. Every variable is potentially dangerous - no matter the source! Or, in other words - every variable has to be properly formatted to be put into a query - no matter the source again. It's the destination that matters. The moment a developer starts to separate the sheep from the goats (thinking whether some particular variable is "safe" or not) he/she takes his/her first step towards disaster. Not to mention that even the wording suggests bulk escaping at the entry point, resembling the very magic quotes feature - already despised, deprecated and removed.
So, unlike whatever "escaping", prepared statements is the measure that indeed protects from SQL injection (when applicable).
I'd recommend using PDO (PHP Data Objects) to run parameterized SQL queries.
Not only does this protect against SQL injection, but it also speeds up queries.
And by using PDO rather than mysql_, mysqli_, and pgsql_ functions, you make your application a little more abstracted from the database, in the rare occurrence that you have to switch database providers.
Use PDO and prepared queries.
($conn is a PDO object)
$stmt = $conn->prepare("INSERT INTO tbl VALUES(:id, :name)");
$stmt->bindValue(':id', $id);
$stmt->bindValue(':name', $name);
$stmt->execute();
As you can see, people suggest you use prepared statements at the most. It's not wrong, but when your query is executed just once per process, there would be a slight performance penalty.
I was facing this issue, but I think I solved it in very sophisticated way - the way hackers use to avoid using quotes. I used this in conjunction with emulated prepared statements. I use it to prevent all kinds of possible SQL injection attacks.
My approach:
If you expect input to be integer make sure it's really integer. In a variable-type language like PHP it is this very important. You can use for example this very simple but powerful solution: sprintf("SELECT 1,2,3 FROM table WHERE 4 = %u", $input);
If you expect anything else from integer hex it. If you hex it, you will perfectly escape all input. In C/C++ there's a function called mysql_hex_string(), in PHP you can use bin2hex().
Don't worry about that the escaped string will have a 2x size of its original length because even if you use mysql_real_escape_string, PHP has to allocate same capacity ((2*input_length)+1), which is the same.
This hex method is often used when you transfer binary data, but I see no reason why not use it on all data to prevent SQL injection attacks. Note that you have to prepend data with 0x or use the MySQL function UNHEX instead.
So, for example, the query:
SELECT password FROM users WHERE name = 'root';
Will become:
SELECT password FROM users WHERE name = 0x726f6f74;
or
SELECT password FROM users WHERE name = UNHEX('726f6f74');
Hex is the perfect escape. No way to inject.
Difference between UNHEX function and 0x prefix
There was some discussion in comments, so I finally want to make it clear. These two approaches are very similar, but they are a little different in some ways:
The 0x prefix can only be used for data columns such as char, varchar, text, block, binary, etc.
Also, its use is a little complicated if you are about to insert an empty string. You'll have to entirely replace it with '', or you'll get an error.
UNHEX() works on any column; you do not have to worry about the empty string.
Hex methods are often used as attacks
Note that this hex method is often used as an SQL injection attack where integers are just like strings and escaped just with mysql_real_escape_string. Then you can avoid the use of quotes.
For example, if you just do something like this:
"SELECT title FROM article WHERE id = " . mysql_real_escape_string($_GET["id"])
an attack can inject you very easily. Consider the following injected code returned from your script:
SELECT ... WHERE id = -1 UNION ALL SELECT table_name FROM information_schema.tables;
and now just extract table structure:
SELECT ... WHERE id = -1 UNION ALL SELECT column_name FROM information_schema.column WHERE table_name = __0x61727469636c65__;
And then just select whatever data ones want. Isn't it cool?
But if the coder of an injectable site would hex it, no injection would be possible because the query would look like this:
SELECT ... WHERE id = UNHEX('2d312075...3635');
Deprecated Warning:
This answer's sample code (like the question's sample code) uses PHP's MySQL extension, which was deprecated in PHP 5.5.0 and removed entirely in PHP 7.0.0.
Security Warning: This answer is not in line with security best practices. Escaping is inadequate to prevent SQL injection, use prepared statements instead. Use the strategy outlined below at your own risk. (Also, mysql_real_escape_string() was removed in PHP 7.)
IMPORTANT
The best way to prevent SQL Injection is to use Prepared Statements instead of escaping, as the accepted answer demonstrates.
There are libraries such as Aura.Sql and EasyDB that allow developers to use prepared statements easier. To learn more about why prepared statements are better at stopping SQL injection, refer to this mysql_real_escape_string() bypass and recently fixed Unicode SQL Injection vulnerabilities in WordPress.
Injection prevention - mysql_real_escape_string()
PHP has a specially-made function to prevent these attacks. All you need to do is use the mouthful of a function, mysql_real_escape_string.
mysql_real_escape_string takes a string that is going to be used in a MySQL query and return the same string with all SQL injection attempts safely escaped. Basically, it will replace those troublesome quotes(') a user might enter with a MySQL-safe substitute, an escaped quote \'.
NOTE: you must be connected to the database to use this function!
// Connect to MySQL
$name_bad = "' OR 1'";
$name_bad = mysql_real_escape_string($name_bad);
$query_bad = "SELECT * FROM customers WHERE username = '$name_bad'";
echo "Escaped Bad Injection: <br />" . $query_bad . "<br />";
$name_evil = "'; DELETE FROM customers WHERE 1 or username = '";
$name_evil = mysql_real_escape_string($name_evil);
$query_evil = "SELECT * FROM customers WHERE username = '$name_evil'";
echo "Escaped Evil Injection: <br />" . $query_evil;
You can find more details in MySQL - SQL Injection Prevention.
Security Warning: This answer is not in line with security best practices. Escaping is inadequate to prevent SQL injection, use prepared statements instead. Use the strategy outlined below at your own risk.
You could do something basic like this:
$safe_variable = mysqli_real_escape_string($dbConnection, $_POST["user-input"]);
mysqli_query($dbConnection, "INSERT INTO table (column) VALUES ('" . $safe_variable . "')");
This won't solve every problem, but it's a very good stepping stone. I left out obvious items such as checking the variable's existence, format (numbers, letters, etc.).
Whatever you do end up using, make sure that you check your input hasn't already been mangled by magic_quotes or some other well-meaning rubbish, and if necessary, run it through stripslashes or whatever to sanitize it.
Deprecated Warning:
This answer's sample code (like the question's sample code) uses PHP's MySQL extension, which was deprecated in PHP 5.5.0 and removed entirely in PHP 7.0.0.
Security Warning: This answer is not in line with security best practices. Escaping is inadequate to prevent SQL injection, use prepared statements instead. Use the strategy outlined below at your own risk. (Also, mysql_real_escape_string() was removed in PHP 7.)
Parameterized query AND input validation is the way to go. There are many scenarios under which SQL injection may occur, even though mysql_real_escape_string() has been used.
Those examples are vulnerable to SQL injection:
$offset = isset($_GET['o']) ? $_GET['o'] : 0;
$offset = mysql_real_escape_string($offset);
RunQuery("SELECT userid, username FROM sql_injection_test LIMIT $offset, 10");
or
$order = isset($_GET['o']) ? $_GET['o'] : 'userid';
$order = mysql_real_escape_string($order);
RunQuery("SELECT userid, username FROM sql_injection_test ORDER BY `$order`");
In both cases, you can't use ' to protect the encapsulation.
Source: The Unexpected SQL Injection (When Escaping Is Not Enough)
In my opinion, the best way to generally prevent SQL injection in your PHP application (or any web application, for that matter) is to think about your application's architecture. If the only way to protect against SQL injection is to remember to use a special method or function that does The Right Thing every time you talk to the database, you are doing it wrong. That way, it's just a matter of time until you forget to correctly format your query at some point in your code.
Adopting the MVC pattern and a framework like CakePHP or CodeIgniter is probably the right way to go: Common tasks like creating secure database queries have been solved and centrally implemented in such frameworks. They help you to organize your web application in a sensible way and make you think more about loading and saving objects than about securely constructing single SQL queries.
There are many ways of preventing SQL injections and other SQL hacks. You can easily find it on the Internet (Google Search). Of course PDO is one of the good solutions. But I would like to suggest you some good links prevention from SQL injection.
What is SQL injection and how to prevent
PHP manual for SQL injection
Microsoft explanation of SQL injection and prevention in PHP
And some other like Preventing SQL injection with MySQL and PHP.
Now, why you do you need to prevent your query from SQL injection?
I would like to let you know: Why do we try for preventing SQL injection with a short example below:
Query for login authentication match:
$query="select * from users where email='".$_POST['email']."' and password='".$_POST['password']."' ";
Now, if someone (a hacker) puts
$_POST['email']= admin#emali.com' OR '1=1
and password anything....
The query will be parsed into the system only up to:
$query="select * from users where email='admin#emali.com' OR '1=1';
The other part will be discarded. So, what will happen? A non-authorized user (hacker) will be able to log in as administrator without having his/her password. Now, he/she can do anything that the administrator/email person can do. See, it's very dangerous if SQL injection is not prevented.
I favor stored procedures (MySQL has had stored procedures support since 5.0) from a security point of view - the advantages are -
Most databases (including MySQL) enable user access to be restricted to executing stored procedures. The fine-grained security access control is useful to prevent escalation of privileges attacks. This prevents compromised applications from being able to run SQL directly against the database.
They abstract the raw SQL query from the application so less information of the database structure is available to the application. This makes it harder for people to understand the underlying structure of the database and design suitable attacks.
They accept only parameters, so the advantages of parameterized queries are there. Of course - IMO you still need to sanitize your input - especially if you are using dynamic SQL inside the stored procedure.
The disadvantages are -
They (stored procedures) are tough to maintain and tend to multiply very quickly. This makes managing them an issue.
They are not very suitable for dynamic queries - if they are built to accept dynamic code as parameters then a lot of the advantages are negated.
I think if someone wants to use PHP and MySQL or some other dataBase server:
Think about learning PDO (PHP Data Objects) – it is a database access layer providing a uniform method of access to multiple databases.
Think about learning MySQLi
Libraries examples:
---- PDO
----- No placeholders - ripe for SQL injection! It's bad
$request = $pdoConnection->("INSERT INTO parents (name, addr, city) values ($name, $addr, $city)");
----- Unnamed placeholders
$request = $pdoConnection->("INSERT INTO parents (name, addr, city) values (?, ?, ?);
----- Named placeholders
$request = $pdoConnection->("INSERT INTO parents (name, addr, city) value (:name, :addr, :city)");
--- MySQLi
$request = $mysqliConnection->prepare('
SELECT * FROM trainers
WHERE name = ?
AND email = ?
AND last_login > ?');
$query->bind_param('first_param', 'second_param', $mail, time() - 3600);
$query->execute();
P.S:
PDO wins this battle with ease. With support for twelve
different database drivers and named parameters, we can get used to its API. From a security standpoint, both of them are safe as long as the developer uses them the way they are supposed to be used
If possible, cast the types of your parameters. But it's only working on simple types like int, bool, and float.
$unsafe_variable = $_POST['user_id'];
$safe_variable = (int)$unsafe_variable ;
mysqli_query($conn, "INSERT INTO table (column) VALUES ('" . $safe_variable . "')");
For those unsure of how to use PDO (coming from the mysql_ functions), I made a very, very simple PDO wrapper that is a single file. It exists to show how easy it is to do all the common things applications need to be done. Works with PostgreSQL, MySQL, and SQLite.
Basically, read it while you read the manual to see how to put the PDO functions to use in real life to make it simple to store and retrieve values in the format you want.
I want a single column
$count = DB::column('SELECT COUNT(*) FROM `user`');
I want an array(key => value) results (i.e. for making a selectbox)
$pairs = DB::pairs('SELECT `id`, `username` FROM `user`');
I want a single row result
$user = DB::row('SELECT * FROM `user` WHERE `id` = ?', array($user_id));
I want an array of results
$banned_users = DB::fetch('SELECT * FROM `user` WHERE `banned` = ?', array('TRUE'));
Security Warning: This answer is not in line with security best practices. Escaping is inadequate to prevent SQL injection, use prepared statements instead.
A few guidelines for escaping special characters in SQL statements.
Don't use MySQL. This extension is deprecated. Use MySQLi or PDO instead.
MySQLi
For manually escaping special characters in a string you can use the mysqli_real_escape_string function. The function will not work properly unless the correct character set is set with mysqli_set_charset.
Example:
$mysqli = new mysqli('host', 'user', 'password', 'database');
$mysqli->set_charset('charset');
$string = $mysqli->real_escape_string($string);
$mysqli->query("INSERT INTO table (column) VALUES ('$string')");
For automatic escaping of values with prepared statements, use mysqli_prepare, and mysqli_stmt_bind_param where types for the corresponding bind variables must be provided for an appropriate conversion:
Example:
$stmt = $mysqli->prepare("INSERT INTO table (column1, column2) VALUES (?,?)");
$stmt->bind_param("is", $integer, $string);
$stmt->execute();
No matter if you use prepared statements or mysqli_real_escape_string, you always have to know the type of input data you're working with.
So if you use a prepared statement, you must specify the types of the variables for mysqli_stmt_bind_param function.
And the use of mysqli_real_escape_string is for, as the name says, escaping special characters in a string, so it will not make integers safe. The purpose of this function is to prevent breaking the strings in SQL statements, and the damage to the database that it could cause. mysqli_real_escape_string is a useful function when used properly, especially when combined with sprintf.
Example:
$string = "x' OR name LIKE '%John%";
$integer = '5 OR id != 0';
$query = sprintf( "SELECT id, email, pass, name FROM members WHERE email ='%s' AND id = %d", $mysqli->real_escape_string($string), $integer);
echo $query;
// SELECT id, email, pass, name FROM members WHERE email ='x\' OR name LIKE \'%John%' AND id = 5
$integer = '99999999999999999999';
$query = sprintf("SELECT id, email, pass, name FROM members WHERE email ='%s' AND id = %d", $mysqli->real_escape_string($string), $integer);
echo $query;
// SELECT id, email, pass, name FROM members WHERE email ='x\' OR name LIKE \'%John%' AND id = 2147483647
Security Warning: This answer is not in line with security best practices. Escaping is inadequate to prevent SQL injection, use prepared statements instead. Use the strategy outlined below at your own risk. (Also, mysql_real_escape_string() was removed in PHP 7.)
Warning: The mysql extension is removed at this time. we recommend using the PDO extension
Using this PHP function mysql_escape_string() you can get a good prevention in a fast way.
For example:
SELECT * FROM users WHERE name = '".mysql_escape_string($name_from_html_form)."'
mysql_escape_string — Escapes a string for use in a mysql_query
For more prevention, you can add at the end ...
wHERE 1=1 or LIMIT 1
Finally you get:
SELECT * FROM users WHERE name = '".mysql_escape_string($name_from_html_form)."' LIMIT 1
The simple alternative to this problem could be solved by granting appropriate permissions in the database itself.
For example: if you are using a MySQL database then enter into the database through terminal or the UI provided and just follow this command:
GRANT SELECT, INSERT, DELETE ON database TO username#'localhost' IDENTIFIED BY 'password';
This will restrict the user to only get confined with the specified query's only. Remove the delete permission and so the data would never get deleted from the query fired from the PHP page.
The second thing to do is to flush the privileges so that the MySQL refreshes the permissions and updates.
FLUSH PRIVILEGES;
more information about flush.
To see the current privileges for the user fire the following query.
select * from mysql.user where User='username';
Learn more about GRANT.
Regarding many useful answers, I hope to add some value to this thread.
SQL injection is an attack that can be done through user inputs (inputs that filled by a user and then used inside queries). The SQL injection patterns are correct query syntax while we can call it: bad queries for bad reasons, and we assume that there might be a bad person that try to get secret information (bypassing access control) that affect the three principles of security (confidentiality, integrity, and availability).
Now, our point is to prevent security threats such as SQL injection attacks, the question asking (how to prevent an SQL injection attack using PHP), be more realistic, data filtering or clearing input data is the case when using user-input data inside such query, using PHP or any other programming language is not the case, or as recommended by more people to use modern technology such as prepared statement or any other tools that currently supporting SQL injection prevention, consider that these tools not available anymore? How do you secure your application?
My approach against SQL injection is: clearing user-input data before sending it to the database (before using it inside any query).
Data filtering for (converting unsafe data to safe data)
Consider that PDO and MySQLi are not available. How can you secure your application? Do you force me to use them? What about other languages other than PHP? I prefer to provide general ideas as it can be used for wider border, not just for a specific language.
SQL user (limiting user privilege): most common SQL operations are (SELECT, UPDATE, INSERT), then, why give the UPDATE privilege to a user that does not require it? For example, login, and search pages are only using SELECT, then, why use DB users in these pages with high privileges?
RULE: do not create one database user for all privileges. For all SQL operations, you can create your scheme like (deluser, selectuser, updateuser) as usernames for easy usage.
See principle of least privilege.
Data filtering: before building any query user input, it should be validated and filtered. For programmers, it's important to define some properties for each user-input variables:
data type, data pattern, and data length. A field that is a number between (x and y) must be exactly validated using the exact rule, and for a field that is a string (text): pattern is the case, for example, a username must contain only some characters, let’s say [a-zA-Z0-9_-.]. The length varies between (x and n) where x and n (integers, x <=n).
Rule: creating exact filters and validation rules are best practices for me.
Use other tools: Here, I will also agree with you that a prepared statement (parametrized query) and stored procedures. The disadvantages here is these ways require advanced skills which do not exist for most users. The basic idea here is to distinguish between the SQL query and the data that is used inside. Both approaches can be used even with unsafe data, because the user-input data here does not add anything to the original query, such as (any or x=x).
For more information, please read OWASP SQL Injection Prevention Cheat Sheet.
Now, if you are an advanced user, start using this defense as you like, but, for beginners, if they can't quickly implement a stored procedure and prepared the statement, it's better to filter input data as much they can.
Finally, let's consider that a user sends this text below instead of entering his/her username:
[1] UNION SELECT IF(SUBSTRING(Password,1,1)='2',BENCHMARK(100000,SHA1(1)),0) User,Password FROM mysql.user WHERE User = 'root'
This input can be checked early without any prepared statement and stored procedures, but to be on the safe side, using them starts after user-data filtering and validation.
The last point is detecting unexpected behavior which requires more effort and complexity; it's not recommended for normal web applications.
Unexpected behavior in the above user input is SELECT, UNION, IF, SUBSTRING, BENCHMARK, SHA, and root. Once these words detected, you can avoid the input.
UPDATE 1:
A user commented that this post is useless, OK! Here is what OWASP.ORG provided:
Primary defenses:
Option #1: Use of Prepared Statements (Parameterized Queries)
Option #2: Use of Stored Procedures
Option #3: Escaping all User Supplied Input
Additional defenses:
Also Enforce: Least Privilege
Also Perform: White List Input Validation
As you may know, claiming an article should be supported by a valid argument, at least by one reference! Otherwise, it's considered as an attack and a bad claim!
Update 2:
From the PHP manual, PHP: Prepared Statements - Manual:
Escaping and SQL injection
Bound variables will be escaped automatically by the server. The
server inserts their escaped values at the appropriate places into the
statement template before execution. A hint must be provided to the
server for the type of bound variable, to create an appropriate
conversion. See the mysqli_stmt_bind_param() function for more
information.
The automatic escaping of values within the server is sometimes
considered a security feature to prevent SQL injection. The same
degree of security can be achieved with non-prepared statements if
input values are escaped correctly.
Update 3:
I created test cases for knowing how PDO and MySQLi send the query to the MySQL server when using a prepared statement:
PDO:
$user = "''1''"; // Malicious keyword
$sql = 'SELECT * FROM awa_user WHERE userame =:username';
$sth = $dbh->prepare($sql, array(PDO::ATTR_CURSOR => PDO::CURSOR_FWDONLY));
$sth->execute(array(':username' => $user));
Query Log:
189 Query SELECT * FROM awa_user WHERE userame ='\'\'1\'\''
189 Quit
MySQLi:
$stmt = $mysqli->prepare("SELECT * FROM awa_user WHERE username =?")) {
$stmt->bind_param("s", $user);
$user = "''1''";
$stmt->execute();
Query Log:
188 Prepare SELECT * FROM awa_user WHERE username =?
188 Execute SELECT * FROM awa_user WHERE username ='\'\'1\'\''
188 Quit
It's clear that a prepared statement is also escaping the data, nothing else.
As also mentioned in the above statement,
The automatic escaping of values within the server is sometimes considered a security feature to prevent SQL injection. The same degree of security can be achieved with non-prepared statements, if input values are escaped correctly
Therefore, this proves that data validation such as intval() is a good idea for integer values before sending any query. In addition, preventing malicious user data before sending the query is a correct and valid approach.
Please see this question for more detail: PDO sends raw query to MySQL while Mysqli sends prepared query, both produce the same result
References:
SQL Injection Cheat Sheet
SQL Injection
Information security
Security Principles
Data validation
Security Warning: This answer is not in line with security best practices. Escaping is inadequate to prevent SQL injection, use prepared statements instead. Use the strategy outlined below at your own risk. (Also, mysql_real_escape_string() was removed in PHP 7.)
Deprecated Warning: The mysql extension is deprecated at this time. we recommend using the PDO extension
I use three different ways to prevent my web application from being vulnerable to SQL injection.
Use of mysql_real_escape_string(), which is a pre-defined function in PHP, and this code add backslashes to the following characters: \x00, \n, \r, \, ', " and \x1a. Pass the input values as parameters to minimize the chance of SQL injection.
The most advanced way is to use PDOs.
I hope this will help you.
Consider the following query:
$iId = mysql_real_escape_string("1 OR 1=1");
$sSql = "SELECT * FROM table WHERE id = $iId";
mysql_real_escape_string() will not protect here. If you use single quotes (' ') around your variables inside your query is what protects you against this. Here is an solution below for this:
$iId = (int) mysql_real_escape_string("1 OR 1=1");
$sSql = "SELECT * FROM table WHERE id = $iId";
This question has some good answers about this.
I suggest, using PDO is the best option.
Edit:
mysql_real_escape_string() is deprecated as of PHP 5.5.0. Use either mysqli or PDO.
An alternative to mysql_real_escape_string() is
string mysqli_real_escape_string ( mysqli $link , string $escapestr )
Example:
$iId = $mysqli->real_escape_string("1 OR 1=1");
$mysqli->query("SELECT * FROM table WHERE id = $iId");
A simple way would be to use a PHP framework like CodeIgniter or Laravel which have inbuilt features like filtering and active-record so that you don't have to worry about these nuances.
Warning: the approach described in this answer only applies to very specific scenarios and isn't secure since SQL injection attacks do not only rely on being able to inject X=Y.
If the attackers are trying to hack into the form via PHP's $_GET variable or with the URL's query string, you would be able to catch them if they're not secure.
RewriteCond %{QUERY_STRING} ([0-9]+)=([0-9]+)
RewriteRule ^(.*) ^/track.php
Because 1=1, 2=2, 1=2, 2=1, 1+1=2, etc... are the common questions to an SQL database of an attacker. Maybe also it's used by many hacking applications.
But you must be careful, that you must not rewrite a safe query from your site. The code above is giving you a tip, to rewrite or redirect (it depends on you) that hacking-specific dynamic query string into a page that will store the attacker's IP address, or EVEN THEIR COOKIES, history, browser, or any other sensitive information, so you can deal with them later by banning their account or contacting authorities.
A good idea is to use an object-relational mapper like Idiorm:
$user = ORM::for_table('user')
->where_equal('username', 'j4mie')
->find_one();
$user->first_name = 'Jamie';
$user->save();
$tweets = ORM::for_table('tweet')
->select('tweet.*')
->join('user', array(
'user.id', '=', 'tweet.user_id'
))
->where_equal('user.username', 'j4mie')
->find_many();
foreach ($tweets as $tweet) {
echo $tweet->text;
}
It not only saves you from SQL injections, but from syntax errors too! It also supports collections of models with method chaining to filter or apply actions to multiple results at once and multiple connections.
There are so many answers for PHP and MySQL, but here is code for PHP and Oracle for preventing SQL injection as well as regular use of oci8 drivers:
$conn = oci_connect($username, $password, $connection_string);
$stmt = oci_parse($conn, 'UPDATE table SET field = :xx WHERE ID = 123');
oci_bind_by_name($stmt, ':xx', $fieldval);
oci_execute($stmt);
This post is marked obsolete because the content is out of date. It is not currently accepting new interactions.
Deprecated Warning:
This answer's sample code (like the question's sample code) uses PHP's MySQL extension, which was deprecated in PHP 5.5.0 and removed entirely in PHP 7.0.0.
Security Warning: This answer is not in line with security best practices. Escaping is inadequate to prevent SQL injection, use prepared statements instead. Use the strategy outlined below at your own risk. (Also, mysql_real_escape_string() was removed in PHP 7.)
Using PDO and MYSQLi is a good practice to prevent SQL injections, but if you really want to work with MySQL functions and queries, it would be better to use
mysql_real_escape_string
$unsafe_variable = mysql_real_escape_string($_POST['user_input']);
There are more abilities to prevent this: like identify - if the input is a string, number, char or array, there are so many inbuilt functions to detect this. Also, it would be better to use these functions to check input data.
is_string
$unsafe_variable = (is_string($_POST['user_input']) ? $_POST['user_input'] : '');
is_numeric
$unsafe_variable = (is_numeric($_POST['user_input']) ? $_POST['user_input'] : '');
And it is so much better to use those functions to check input data with mysql_real_escape_string.
I've written this little function several years ago:
function sqlvprintf($query, $args)
{
global $DB_LINK;
$ctr = 0;
ensureConnection(); // Connect to database if not connected already.
$values = array();
foreach ($args as $value)
{
if (is_string($value))
{
$value = "'" . mysqli_real_escape_string($DB_LINK, $value) . "'";
}
else if (is_null($value))
{
$value = 'NULL';
}
else if (!is_int($value) && !is_float($value))
{
die('Only numeric, string, array and NULL arguments allowed in a query. Argument '.($ctr+1).' is not a basic type, it\'s type is '. gettype($value). '.');
}
$values[] = $value;
$ctr++;
}
$query = preg_replace_callback(
'/{(\\d+)}/',
function($match) use ($values)
{
if (isset($values[$match[1]]))
{
return $values[$match[1]];
}
else
{
return $match[0];
}
},
$query
);
return $query;
}
function runEscapedQuery($preparedQuery /*, ...*/)
{
$params = array_slice(func_get_args(), 1);
$results = runQuery(sqlvprintf($preparedQuery, $params)); // Run query and fetch results.
return $results;
}
This allows running statements in an one-liner C#-ish String.Format like:
runEscapedQuery("INSERT INTO Whatever (id, foo, bar) VALUES ({0}, {1}, {2})", $numericVar, $stringVar1, $stringVar2);
It escapes considering the variable type. If you try to parameterize table, column names, it would fail as it puts every string in quotes which is an invalid syntax.
SECURITY UPDATE: The previous str_replace version allowed injections by adding {#} tokens into user data. This preg_replace_callback version doesn't cause problems if the replacement contains these tokens.

Best practice to pass query conditions in ajax request

I'm writing a REST api in node js that will execute a sql query and send the results;
in the request I need to send the WHERE conditions; ex:
GET 127.0.0.1:5007/users //gets the list of users
GET 127.0.0.1:5007/users
id = 1 //gets the user with id 1
Right now the conditions are passed from the client to the rest api in the request's headers.
In the API I'm using sequelize, an ORM that needs to receive WHERE conditions in a particular form (an object); ex: having the condition:
(x=1 AND (y=2 OR z=3)) OR (x=3 AND y=1)
this needs to be formatted as a nested object:
-- x=1
-- AND -| -- y=2
| -- OR ----|
| -- z=3
-- OR -|
|
| -- x=3
-- AND -|
-- y=1
so the object would be:
Sequelize.or (
Sequelize.and (
{x=1},
Sequelize.or(
{y=2},
{z=3}
)
),
Sequelize.and (
{x=3},
{y=1}
)
)
Now I'm trying to pass a simple string (like "(x=1 AND (y=2 OR z=3)) OR (x=3 AND y=1)"), but then I will need a function on the server that can convert the string in the needed object (this method in my opinion has the advantage that the developer writing the client, can pass the where conditions in a simple way, like using sql, and this method is also indipendent from the used ORM, with no need to change the client if we need to change the server or use a different ORM);
The function to read and convert the conditions' string into an object is giving me headache (I'm trying to write one without success, so if you have some examples about how to do something like this...)
What I would like to get is a route capable of executing almost any kind of sql query and give the results:
now I have a different route for everything:
127.0.0.1:5007/users //to get all users
127.0.0.1:5007/users/1 //to get a single user
127.0.0.1:5007/lastusers //to get user registered in the last month
and so on for the other tables i need to query (one route for every kind of request I need in the client);
instead I would like to have only one route, something like:
127.0.0.1:5007/request
(when calling this route I will pass the table name and the conditions' string)
Do you think this solution would be a good solution or you generally use other ways to handle this kind of things?
Do you have any idea on how to write a function to convert the conditions' string into the desired object?
Any suggestion would be appreciated ;)
I would strongly advise you not to expose any part of your database model to your clients. Doing so means you can't change anything you expose without the risk of breaking the clients. One suggestion as far as what you've supplied is that you can and should use query parameters to cut down on the number of endpoints you've got.
GET /users //to get all users
GET /users?registeredInPastDays=30 //to get user registered in the last month
GET /users/1 //to get a single user
Obviously "registeredInPastDays" should be renamed to something less clumsy .. it's just an example.
As far as the conditions string, there ought to be plenty of parsers available online. The grammar looks very straightforward.
IMHO the main disadvantage of your solution is that you are creating just another API for quering data. Why create sthm from scratch if it is already created? You should use existing mature query API and focus on your business logic rather then inventing sthm new.
For example, you can take query syntax from Odata. Many people have been developing that standard for a long time. They have already considered different use cases and obstacles for query API.
Resources are located with a URI. You can use or mix three ways to address them:
Hierarchically with a sequence of path segments:
/users/john/posts/4711
Non hierarchically with query parameters:
/users/john/posts?minVotes=10&minViews=1000&tags=java
With matrix parameters which affect only one path segment:
/users;country=ukraine/posts
This is normally sufficient enough but it has limitations like the maximum length. In your case a problem is that you can't easily describe and and or conjunctions with query parameters. But you can use a custom or standard query syntax. For instance if you want to find all cars or vehicles from Ford except the Capri with a price between $10000 and $20000 Google uses the search parameter
q=cars+OR+vehicles+%22ford%22+-capri+%2410000..%2420000
(the %22 is a escaped ", the %24 a escaped $).
If this does not work for your case and you want to pass data outside of the URI the format is just a matter of your taste. Adding a custom header like X-Filter may be a valid approach. I would tend to use a POST. Although you just want to query data this is still RESTful if you treat your request as the creation of a search result resource:
POST /search HTTP/1.1
your query-data
Your server should return the newly created resource in the Location header:
HTTP/1.1 201 Created
Location: /search/3
The result can still be cached and you can bookmark it or send the link. The downside is that you need an additional POST.

Resources