Case insensitive active record query in rails-4 - search

Currently I am working on a project in rails 4 in which I have a user-page at example.com/username but it only finds the record if i use username in proper case, how can I perform case insensitive search in active record rails.
Check out my code below
#user = User.find_by_username(params[:username].downcase)

you will have to use something similar to where("username ILIKE ?"). the ILIKE or similar syntax is dependant on your database though.

Another option could be:
def self.search(query)
where("lower(name) LIKE lower(?)", "%#{query}%")
end
This would work in sqlite or in PostgreSQL.

You can use Arel Tables.
# Suppose the target username in DB is 'FooBAR'.
t = User.arel_table
#user = User.find_by(t[:username].matches 'foobar')
This will find users with usernames like 'FooBar', 'foobar', 'FOOBAR' and so on.
More details here: https://github.com/rails/arel.

Related

Checking if a item exists withot being case sensitive

Is possible to check if an element exists without being case sensitive without using queries with PonyORM?
I have a table with the definition:
Client(id: int PRIMARY KEY, name: Text, location: Text)
So, it would be something like:
Client.exists(name='stackoverflow'.upper())
This does not work as it only converts into case a part of the comparison, but not the contents from the Database.
I mean the equivalent to the query:
SELECT * From Client WHERE UPPER(Client.name) = UPPER('stackoverflow')
But using the already existing functionality in ponyORM with the exists() method.
I have found the solution using Lambda functions.
Writing this:
Client.exists(lambda: str(c.name).upper() == 'stackoverflow'.upper())

How to write a LIKE query in Azure CosmosDB?

I want to retrieve data from Cosmos DB with the following query:
SELECT * FROM c WHERE c.pi like '09%001'
(This is a SQL query, which I can use in MySQL)
Here, pi is a string value, which can be 09001001 or 09025001.
Is there a way to use a LIKE command in Cosmos DB?
I know that cosmos DB uses CONTAINS, but this cannot be used when you want to match specifically the beginning or end of the string.
UPDATE :
You can now use the LIKE keyword to do text searches in Azure Cosmos DB SQL (core) API!
EXAMPLE:
SELECT *
FROM c
WHERE c.description LIKE "%cereal%"
OLD Answer:
This can be achieved in 2 ways
(i) Currently Azure Cosmosdb supports the CONTAINS, STARTSWITH, and ENDSWITH built-in functions which are equivalent to LIKE.
The keyword for LIKE in Cosmosdb is Contains .
SELECT * FROM c WHERE CONTAINS(c.pi, '09')
So, in your case if you want to match the pattern 09%001, you need to use:
SELECT * FROM c WHERE STARTSWITH(c.pi, '09') AND ENDSWITH(c.pi, '001')
(ii) As 404 mentioned, Use SQL API User Defined Functions which supports regex :
function executeRegex(str, pattern) {
let regex=RegExp(pattern);
return regex.test(str);
}
SELECT udf.EXECUTE_REGEX("foobar", ".*bar")
Another possibility is creating your own User Defined Function. As example here's a regex check:
function matchRegex(str, pattern) {
let regex=RegExp(pattern);
return regex.test(str);
}
Created under the name MATCH_REGEX it can then be used like:
SELECT udf.MATCH_REGEX("09001001", "^09.*001$")
As note: it'll kill any index optimization that for instance STARTSWITH would have. Although allows for much more complex patterns. It can therefor be beneficial to use additional filters that are capable of using the index to narrow down the search. E.g. using StartsWith(c.property1, '09') as addition to the above example in a WHERE clause.
UPDATE:
Cosmos now has a RegexMatch function that can do the same. While the documentation for the MongoApi mentions the $regex can use the index to optimize your query if it adheres to certain rules this does not seem to be the case for the SqlApi (at this moment).

How to sanitize for SQL injection [duplicate]

This question's answers are a community effort. Edit existing answers to improve this post. It is not currently accepting new answers or interactions.
If user input is inserted without modification into an SQL query, then the application becomes vulnerable to SQL injection, like in the following example:
$unsafe_variable = $_POST['user_input'];
mysql_query("INSERT INTO `table` (`column`) VALUES ('$unsafe_variable')");
That's because the user can input something like value'); DROP TABLE table;--, and the query becomes:
INSERT INTO `table` (`column`) VALUES('value'); DROP TABLE table;--')
What can be done to prevent this from happening?
The correct way to avoid SQL injection attacks, no matter which database you use, is to separate the data from SQL, so that data stays data and will never be interpreted as commands by the SQL parser. It is possible to create an SQL statement with correctly formatted data parts, but if you don't fully understand the details, you should always use prepared statements and parameterized queries. These are SQL statements that are sent to and parsed by the database server separately from any parameters. This way it is impossible for an attacker to inject malicious SQL.
You basically have two options to achieve this:
Using PDO (for any supported database driver):
$stmt = $pdo->prepare('SELECT * FROM employees WHERE name = :name');
$stmt->execute([ 'name' => $name ]);
foreach ($stmt as $row) {
// Do something with $row
}
Using MySQLi (for MySQL):
Since PHP 8.2+ we can make use of execute_query() which prepares, binds parameters, and executes SQL statement in one method:
$result = $dbConnection->execute_query('SELECT * FROM employees WHERE name = ?', [$name]);
while ($row = $result->fetch_assoc()) {
// Do something with $row
}
Up to PHP8.1:
$stmt = $dbConnection->prepare('SELECT * FROM employees WHERE name = ?');
$stmt->bind_param('s', $name); // 's' specifies the variable type => 'string'
$stmt->execute();
$result = $stmt->get_result();
while ($row = $result->fetch_assoc()) {
// Do something with $row
}
If you're connecting to a database other than MySQL, there is a driver-specific second option that you can refer to (for example, pg_prepare() and pg_execute() for PostgreSQL). PDO is the universal option.
Correctly setting up the connection
PDO
Note that when using PDO to access a MySQL database real prepared statements are not used by default. To fix this you have to disable the emulation of prepared statements. An example of creating a connection using PDO is:
$dbConnection = new PDO('mysql:dbname=dbtest;host=127.0.0.1;charset=utf8mb4', 'user', 'password');
$dbConnection->setAttribute(PDO::ATTR_EMULATE_PREPARES, false);
$dbConnection->setAttribute(PDO::ATTR_ERRMODE, PDO::ERRMODE_EXCEPTION);
In the above example, the error mode isn't strictly necessary, but it is advised to add it. This way PDO will inform you of all MySQL errors by means of throwing the PDOException.
What is mandatory, however, is the first setAttribute() line, which tells PDO to disable emulated prepared statements and use real prepared statements. This makes sure the statement and the values aren't parsed by PHP before sending it to the MySQL server (giving a possible attacker no chance to inject malicious SQL).
Although you can set the charset in the options of the constructor, it's important to note that 'older' versions of PHP (before 5.3.6) silently ignored the charset parameter in the DSN.
Mysqli
For mysqli we have to follow the same routine:
mysqli_report(MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT); // error reporting
$dbConnection = new mysqli('127.0.0.1', 'username', 'password', 'test');
$dbConnection->set_charset('utf8mb4'); // charset
Explanation
The SQL statement you pass to prepare is parsed and compiled by the database server. By specifying parameters (either a ? or a named parameter like :name in the example above) you tell the database engine where you want to filter on. Then when you call execute, the prepared statement is combined with the parameter values you specify.
The important thing here is that the parameter values are combined with the compiled statement, not an SQL string. SQL injection works by tricking the script into including malicious strings when it creates SQL to send to the database. So by sending the actual SQL separately from the parameters, you limit the risk of ending up with something you didn't intend.
Any parameters you send when using a prepared statement will just be treated as strings (although the database engine may do some optimization so parameters may end up as numbers too, of course). In the example above, if the $name variable contains 'Sarah'; DELETE FROM employees the result would simply be a search for the string "'Sarah'; DELETE FROM employees", and you will not end up with an empty table.
Another benefit of using prepared statements is that if you execute the same statement many times in the same session it will only be parsed and compiled once, giving you some speed gains.
Oh, and since you asked about how to do it for an insert, here's an example (using PDO):
$preparedStatement = $db->prepare('INSERT INTO table (column) VALUES (:column)');
$preparedStatement->execute([ 'column' => $unsafeValue ]);
Can prepared statements be used for dynamic queries?
While you can still use prepared statements for the query parameters, the structure of the dynamic query itself cannot be parametrized and certain query features cannot be parametrized.
For these specific scenarios, the best thing to do is use a whitelist filter that restricts the possible values.
// Value whitelist
// $dir can only be 'DESC', otherwise it will be 'ASC'
if (empty($dir) || $dir !== 'DESC') {
$dir = 'ASC';
}
To use the parameterized query, you need to use either Mysqli or PDO. To rewrite your example with mysqli, we would need something like the following.
<?php
mysqli_report(MYSQLI_REPORT_ERROR | MYSQLI_REPORT_STRICT);
$mysqli = new mysqli("server", "username", "password", "database_name");
$variable = $_POST["user-input"];
$stmt = $mysqli->prepare("INSERT INTO table (column) VALUES (?)");
// "s" means the database expects a string
$stmt->bind_param("s", $variable);
$stmt->execute();
The key function you'll want to read up on there would be mysqli::prepare.
Also, as others have suggested, you may find it useful/easier to step up a layer of abstraction with something like PDO.
Please note that the case you asked about is a fairly simple one and that more complex cases may require more complex approaches. In particular:
If you want to alter the structure of the SQL based on user input, parameterized queries are not going to help, and the escaping required is not covered by mysql_real_escape_string. In this kind of case, you would be better off passing the user's input through a whitelist to ensure only 'safe' values are allowed through.
Every answer here covers only part of the problem.
In fact, there are four different query parts which we can add to SQL dynamically: -
a string
a number
an identifier
a syntax keyword
And prepared statements cover only two of them.
But sometimes we have to make our query even more dynamic, adding operators or identifiers as well.
So, we will need different protection techniques.
In general, such a protection approach is based on whitelisting.
In this case, every dynamic parameter should be hardcoded in your script and chosen from that set.
For example, to do dynamic ordering:
$orders = array("name", "price", "qty"); // Field names
$key = array_search($_GET['sort'], $orders)); // if we have such a name
$orderby = $orders[$key]; // If not, first one will be set automatically.
$query = "SELECT * FROM `table` ORDER BY $orderby"; // Value is safe
To ease the process I wrote a whitelist helper function that does all the job in one line:
$orderby = white_list($_GET['orderby'], "name", ["name","price","qty"], "Invalid field name");
$query = "SELECT * FROM `table` ORDER BY `$orderby`"; // sound and safe
There is another way to secure identifiers - escaping but I rather stick to whitelisting as a more robust and explicit approach. Yet as long as you have an identifier quoted, you can escape the quote character to make it safe. For example, by default for mysql you have to double the quote character to escape it. For other other DBMS escaping rules would be different.
Still, there is an issue with SQL syntax keywords (such as AND, DESC and such), but white-listing seems the only approach in this case.
So, a general recommendation may be phrased as
Any variable that represents an SQL data literal, (or, to put it simply - an SQL string, or a number) must be added through a prepared statement. No Exceptions.
Any other query part, such as an SQL keyword, a table or a field name, or an operator - must be filtered through a white list.
Update
Although there is a general agreement on the best practices regarding SQL injection protection, there are still many bad practices as well. And some of them too deeply rooted in the minds of PHP users. For instance, on this very page there are (although invisible to most visitors) more than 80 deleted answers - all removed by the community due to bad quality or promoting bad and outdated practices. Worse yet, some of the bad answers aren't deleted, but rather prospering.
For example, there(1) are(2) still(3) many(4) answers(5), including the second most upvoted answer suggesting you manual string escaping - an outdated approach that is proven to be insecure.
Or there is a slightly better answer that suggests just another method of string formatting and even boasts it as the ultimate panacea. While of course, it is not. This method is no better than regular string formatting, yet it keeps all its drawbacks: it is applicable to strings only and, like any other manual formatting, it's essentially optional, non-obligatory measure, prone to human error of any sort.
I think that all this because of one very old superstition, supported by such authorities like OWASP or the PHP manual, which proclaims equality between whatever "escaping" and protection from SQL injections.
Regardless of what PHP manual said for ages, *_escape_string by no means makes data safe and never has been intended to. Besides being useless for any SQL part other than string, manual escaping is wrong, because it is manual as opposite to automated.
And OWASP makes it even worse, stressing on escaping user input which is an utter nonsense: there should be no such words in the context of injection protection. Every variable is potentially dangerous - no matter the source! Or, in other words - every variable has to be properly formatted to be put into a query - no matter the source again. It's the destination that matters. The moment a developer starts to separate the sheep from the goats (thinking whether some particular variable is "safe" or not) he/she takes his/her first step towards disaster. Not to mention that even the wording suggests bulk escaping at the entry point, resembling the very magic quotes feature - already despised, deprecated and removed.
So, unlike whatever "escaping", prepared statements is the measure that indeed protects from SQL injection (when applicable).
I'd recommend using PDO (PHP Data Objects) to run parameterized SQL queries.
Not only does this protect against SQL injection, but it also speeds up queries.
And by using PDO rather than mysql_, mysqli_, and pgsql_ functions, you make your application a little more abstracted from the database, in the rare occurrence that you have to switch database providers.
Use PDO and prepared queries.
($conn is a PDO object)
$stmt = $conn->prepare("INSERT INTO tbl VALUES(:id, :name)");
$stmt->bindValue(':id', $id);
$stmt->bindValue(':name', $name);
$stmt->execute();
As you can see, people suggest you use prepared statements at the most. It's not wrong, but when your query is executed just once per process, there would be a slight performance penalty.
I was facing this issue, but I think I solved it in very sophisticated way - the way hackers use to avoid using quotes. I used this in conjunction with emulated prepared statements. I use it to prevent all kinds of possible SQL injection attacks.
My approach:
If you expect input to be integer make sure it's really integer. In a variable-type language like PHP it is this very important. You can use for example this very simple but powerful solution: sprintf("SELECT 1,2,3 FROM table WHERE 4 = %u", $input);
If you expect anything else from integer hex it. If you hex it, you will perfectly escape all input. In C/C++ there's a function called mysql_hex_string(), in PHP you can use bin2hex().
Don't worry about that the escaped string will have a 2x size of its original length because even if you use mysql_real_escape_string, PHP has to allocate same capacity ((2*input_length)+1), which is the same.
This hex method is often used when you transfer binary data, but I see no reason why not use it on all data to prevent SQL injection attacks. Note that you have to prepend data with 0x or use the MySQL function UNHEX instead.
So, for example, the query:
SELECT password FROM users WHERE name = 'root';
Will become:
SELECT password FROM users WHERE name = 0x726f6f74;
or
SELECT password FROM users WHERE name = UNHEX('726f6f74');
Hex is the perfect escape. No way to inject.
Difference between UNHEX function and 0x prefix
There was some discussion in comments, so I finally want to make it clear. These two approaches are very similar, but they are a little different in some ways:
The 0x prefix can only be used for data columns such as char, varchar, text, block, binary, etc.
Also, its use is a little complicated if you are about to insert an empty string. You'll have to entirely replace it with '', or you'll get an error.
UNHEX() works on any column; you do not have to worry about the empty string.
Hex methods are often used as attacks
Note that this hex method is often used as an SQL injection attack where integers are just like strings and escaped just with mysql_real_escape_string. Then you can avoid the use of quotes.
For example, if you just do something like this:
"SELECT title FROM article WHERE id = " . mysql_real_escape_string($_GET["id"])
an attack can inject you very easily. Consider the following injected code returned from your script:
SELECT ... WHERE id = -1 UNION ALL SELECT table_name FROM information_schema.tables;
and now just extract table structure:
SELECT ... WHERE id = -1 UNION ALL SELECT column_name FROM information_schema.column WHERE table_name = __0x61727469636c65__;
And then just select whatever data ones want. Isn't it cool?
But if the coder of an injectable site would hex it, no injection would be possible because the query would look like this:
SELECT ... WHERE id = UNHEX('2d312075...3635');
Deprecated Warning:
This answer's sample code (like the question's sample code) uses PHP's MySQL extension, which was deprecated in PHP 5.5.0 and removed entirely in PHP 7.0.0.
Security Warning: This answer is not in line with security best practices. Escaping is inadequate to prevent SQL injection, use prepared statements instead. Use the strategy outlined below at your own risk. (Also, mysql_real_escape_string() was removed in PHP 7.)
IMPORTANT
The best way to prevent SQL Injection is to use Prepared Statements instead of escaping, as the accepted answer demonstrates.
There are libraries such as Aura.Sql and EasyDB that allow developers to use prepared statements easier. To learn more about why prepared statements are better at stopping SQL injection, refer to this mysql_real_escape_string() bypass and recently fixed Unicode SQL Injection vulnerabilities in WordPress.
Injection prevention - mysql_real_escape_string()
PHP has a specially-made function to prevent these attacks. All you need to do is use the mouthful of a function, mysql_real_escape_string.
mysql_real_escape_string takes a string that is going to be used in a MySQL query and return the same string with all SQL injection attempts safely escaped. Basically, it will replace those troublesome quotes(') a user might enter with a MySQL-safe substitute, an escaped quote \'.
NOTE: you must be connected to the database to use this function!
// Connect to MySQL
$name_bad = "' OR 1'";
$name_bad = mysql_real_escape_string($name_bad);
$query_bad = "SELECT * FROM customers WHERE username = '$name_bad'";
echo "Escaped Bad Injection: <br />" . $query_bad . "<br />";
$name_evil = "'; DELETE FROM customers WHERE 1 or username = '";
$name_evil = mysql_real_escape_string($name_evil);
$query_evil = "SELECT * FROM customers WHERE username = '$name_evil'";
echo "Escaped Evil Injection: <br />" . $query_evil;
You can find more details in MySQL - SQL Injection Prevention.
Security Warning: This answer is not in line with security best practices. Escaping is inadequate to prevent SQL injection, use prepared statements instead. Use the strategy outlined below at your own risk.
You could do something basic like this:
$safe_variable = mysqli_real_escape_string($dbConnection, $_POST["user-input"]);
mysqli_query($dbConnection, "INSERT INTO table (column) VALUES ('" . $safe_variable . "')");
This won't solve every problem, but it's a very good stepping stone. I left out obvious items such as checking the variable's existence, format (numbers, letters, etc.).
Whatever you do end up using, make sure that you check your input hasn't already been mangled by magic_quotes or some other well-meaning rubbish, and if necessary, run it through stripslashes or whatever to sanitize it.
Deprecated Warning:
This answer's sample code (like the question's sample code) uses PHP's MySQL extension, which was deprecated in PHP 5.5.0 and removed entirely in PHP 7.0.0.
Security Warning: This answer is not in line with security best practices. Escaping is inadequate to prevent SQL injection, use prepared statements instead. Use the strategy outlined below at your own risk. (Also, mysql_real_escape_string() was removed in PHP 7.)
Parameterized query AND input validation is the way to go. There are many scenarios under which SQL injection may occur, even though mysql_real_escape_string() has been used.
Those examples are vulnerable to SQL injection:
$offset = isset($_GET['o']) ? $_GET['o'] : 0;
$offset = mysql_real_escape_string($offset);
RunQuery("SELECT userid, username FROM sql_injection_test LIMIT $offset, 10");
or
$order = isset($_GET['o']) ? $_GET['o'] : 'userid';
$order = mysql_real_escape_string($order);
RunQuery("SELECT userid, username FROM sql_injection_test ORDER BY `$order`");
In both cases, you can't use ' to protect the encapsulation.
Source: The Unexpected SQL Injection (When Escaping Is Not Enough)
In my opinion, the best way to generally prevent SQL injection in your PHP application (or any web application, for that matter) is to think about your application's architecture. If the only way to protect against SQL injection is to remember to use a special method or function that does The Right Thing every time you talk to the database, you are doing it wrong. That way, it's just a matter of time until you forget to correctly format your query at some point in your code.
Adopting the MVC pattern and a framework like CakePHP or CodeIgniter is probably the right way to go: Common tasks like creating secure database queries have been solved and centrally implemented in such frameworks. They help you to organize your web application in a sensible way and make you think more about loading and saving objects than about securely constructing single SQL queries.
There are many ways of preventing SQL injections and other SQL hacks. You can easily find it on the Internet (Google Search). Of course PDO is one of the good solutions. But I would like to suggest you some good links prevention from SQL injection.
What is SQL injection and how to prevent
PHP manual for SQL injection
Microsoft explanation of SQL injection and prevention in PHP
And some other like Preventing SQL injection with MySQL and PHP.
Now, why you do you need to prevent your query from SQL injection?
I would like to let you know: Why do we try for preventing SQL injection with a short example below:
Query for login authentication match:
$query="select * from users where email='".$_POST['email']."' and password='".$_POST['password']."' ";
Now, if someone (a hacker) puts
$_POST['email']= admin#emali.com' OR '1=1
and password anything....
The query will be parsed into the system only up to:
$query="select * from users where email='admin#emali.com' OR '1=1';
The other part will be discarded. So, what will happen? A non-authorized user (hacker) will be able to log in as administrator without having his/her password. Now, he/she can do anything that the administrator/email person can do. See, it's very dangerous if SQL injection is not prevented.
I favor stored procedures (MySQL has had stored procedures support since 5.0) from a security point of view - the advantages are -
Most databases (including MySQL) enable user access to be restricted to executing stored procedures. The fine-grained security access control is useful to prevent escalation of privileges attacks. This prevents compromised applications from being able to run SQL directly against the database.
They abstract the raw SQL query from the application so less information of the database structure is available to the application. This makes it harder for people to understand the underlying structure of the database and design suitable attacks.
They accept only parameters, so the advantages of parameterized queries are there. Of course - IMO you still need to sanitize your input - especially if you are using dynamic SQL inside the stored procedure.
The disadvantages are -
They (stored procedures) are tough to maintain and tend to multiply very quickly. This makes managing them an issue.
They are not very suitable for dynamic queries - if they are built to accept dynamic code as parameters then a lot of the advantages are negated.
I think if someone wants to use PHP and MySQL or some other dataBase server:
Think about learning PDO (PHP Data Objects) – it is a database access layer providing a uniform method of access to multiple databases.
Think about learning MySQLi
Libraries examples:
---- PDO
----- No placeholders - ripe for SQL injection! It's bad
$request = $pdoConnection->("INSERT INTO parents (name, addr, city) values ($name, $addr, $city)");
----- Unnamed placeholders
$request = $pdoConnection->("INSERT INTO parents (name, addr, city) values (?, ?, ?);
----- Named placeholders
$request = $pdoConnection->("INSERT INTO parents (name, addr, city) value (:name, :addr, :city)");
--- MySQLi
$request = $mysqliConnection->prepare('
SELECT * FROM trainers
WHERE name = ?
AND email = ?
AND last_login > ?');
$query->bind_param('first_param', 'second_param', $mail, time() - 3600);
$query->execute();
P.S:
PDO wins this battle with ease. With support for twelve
different database drivers and named parameters, we can get used to its API. From a security standpoint, both of them are safe as long as the developer uses them the way they are supposed to be used
If possible, cast the types of your parameters. But it's only working on simple types like int, bool, and float.
$unsafe_variable = $_POST['user_id'];
$safe_variable = (int)$unsafe_variable ;
mysqli_query($conn, "INSERT INTO table (column) VALUES ('" . $safe_variable . "')");
For those unsure of how to use PDO (coming from the mysql_ functions), I made a very, very simple PDO wrapper that is a single file. It exists to show how easy it is to do all the common things applications need to be done. Works with PostgreSQL, MySQL, and SQLite.
Basically, read it while you read the manual to see how to put the PDO functions to use in real life to make it simple to store and retrieve values in the format you want.
I want a single column
$count = DB::column('SELECT COUNT(*) FROM `user`');
I want an array(key => value) results (i.e. for making a selectbox)
$pairs = DB::pairs('SELECT `id`, `username` FROM `user`');
I want a single row result
$user = DB::row('SELECT * FROM `user` WHERE `id` = ?', array($user_id));
I want an array of results
$banned_users = DB::fetch('SELECT * FROM `user` WHERE `banned` = ?', array('TRUE'));
Security Warning: This answer is not in line with security best practices. Escaping is inadequate to prevent SQL injection, use prepared statements instead.
A few guidelines for escaping special characters in SQL statements.
Don't use MySQL. This extension is deprecated. Use MySQLi or PDO instead.
MySQLi
For manually escaping special characters in a string you can use the mysqli_real_escape_string function. The function will not work properly unless the correct character set is set with mysqli_set_charset.
Example:
$mysqli = new mysqli('host', 'user', 'password', 'database');
$mysqli->set_charset('charset');
$string = $mysqli->real_escape_string($string);
$mysqli->query("INSERT INTO table (column) VALUES ('$string')");
For automatic escaping of values with prepared statements, use mysqli_prepare, and mysqli_stmt_bind_param where types for the corresponding bind variables must be provided for an appropriate conversion:
Example:
$stmt = $mysqli->prepare("INSERT INTO table (column1, column2) VALUES (?,?)");
$stmt->bind_param("is", $integer, $string);
$stmt->execute();
No matter if you use prepared statements or mysqli_real_escape_string, you always have to know the type of input data you're working with.
So if you use a prepared statement, you must specify the types of the variables for mysqli_stmt_bind_param function.
And the use of mysqli_real_escape_string is for, as the name says, escaping special characters in a string, so it will not make integers safe. The purpose of this function is to prevent breaking the strings in SQL statements, and the damage to the database that it could cause. mysqli_real_escape_string is a useful function when used properly, especially when combined with sprintf.
Example:
$string = "x' OR name LIKE '%John%";
$integer = '5 OR id != 0';
$query = sprintf( "SELECT id, email, pass, name FROM members WHERE email ='%s' AND id = %d", $mysqli->real_escape_string($string), $integer);
echo $query;
// SELECT id, email, pass, name FROM members WHERE email ='x\' OR name LIKE \'%John%' AND id = 5
$integer = '99999999999999999999';
$query = sprintf("SELECT id, email, pass, name FROM members WHERE email ='%s' AND id = %d", $mysqli->real_escape_string($string), $integer);
echo $query;
// SELECT id, email, pass, name FROM members WHERE email ='x\' OR name LIKE \'%John%' AND id = 2147483647
Security Warning: This answer is not in line with security best practices. Escaping is inadequate to prevent SQL injection, use prepared statements instead. Use the strategy outlined below at your own risk. (Also, mysql_real_escape_string() was removed in PHP 7.)
Warning: The mysql extension is removed at this time. we recommend using the PDO extension
Using this PHP function mysql_escape_string() you can get a good prevention in a fast way.
For example:
SELECT * FROM users WHERE name = '".mysql_escape_string($name_from_html_form)."'
mysql_escape_string — Escapes a string for use in a mysql_query
For more prevention, you can add at the end ...
wHERE 1=1 or LIMIT 1
Finally you get:
SELECT * FROM users WHERE name = '".mysql_escape_string($name_from_html_form)."' LIMIT 1
The simple alternative to this problem could be solved by granting appropriate permissions in the database itself.
For example: if you are using a MySQL database then enter into the database through terminal or the UI provided and just follow this command:
GRANT SELECT, INSERT, DELETE ON database TO username#'localhost' IDENTIFIED BY 'password';
This will restrict the user to only get confined with the specified query's only. Remove the delete permission and so the data would never get deleted from the query fired from the PHP page.
The second thing to do is to flush the privileges so that the MySQL refreshes the permissions and updates.
FLUSH PRIVILEGES;
more information about flush.
To see the current privileges for the user fire the following query.
select * from mysql.user where User='username';
Learn more about GRANT.
Regarding many useful answers, I hope to add some value to this thread.
SQL injection is an attack that can be done through user inputs (inputs that filled by a user and then used inside queries). The SQL injection patterns are correct query syntax while we can call it: bad queries for bad reasons, and we assume that there might be a bad person that try to get secret information (bypassing access control) that affect the three principles of security (confidentiality, integrity, and availability).
Now, our point is to prevent security threats such as SQL injection attacks, the question asking (how to prevent an SQL injection attack using PHP), be more realistic, data filtering or clearing input data is the case when using user-input data inside such query, using PHP or any other programming language is not the case, or as recommended by more people to use modern technology such as prepared statement or any other tools that currently supporting SQL injection prevention, consider that these tools not available anymore? How do you secure your application?
My approach against SQL injection is: clearing user-input data before sending it to the database (before using it inside any query).
Data filtering for (converting unsafe data to safe data)
Consider that PDO and MySQLi are not available. How can you secure your application? Do you force me to use them? What about other languages other than PHP? I prefer to provide general ideas as it can be used for wider border, not just for a specific language.
SQL user (limiting user privilege): most common SQL operations are (SELECT, UPDATE, INSERT), then, why give the UPDATE privilege to a user that does not require it? For example, login, and search pages are only using SELECT, then, why use DB users in these pages with high privileges?
RULE: do not create one database user for all privileges. For all SQL operations, you can create your scheme like (deluser, selectuser, updateuser) as usernames for easy usage.
See principle of least privilege.
Data filtering: before building any query user input, it should be validated and filtered. For programmers, it's important to define some properties for each user-input variables:
data type, data pattern, and data length. A field that is a number between (x and y) must be exactly validated using the exact rule, and for a field that is a string (text): pattern is the case, for example, a username must contain only some characters, let’s say [a-zA-Z0-9_-.]. The length varies between (x and n) where x and n (integers, x <=n).
Rule: creating exact filters and validation rules are best practices for me.
Use other tools: Here, I will also agree with you that a prepared statement (parametrized query) and stored procedures. The disadvantages here is these ways require advanced skills which do not exist for most users. The basic idea here is to distinguish between the SQL query and the data that is used inside. Both approaches can be used even with unsafe data, because the user-input data here does not add anything to the original query, such as (any or x=x).
For more information, please read OWASP SQL Injection Prevention Cheat Sheet.
Now, if you are an advanced user, start using this defense as you like, but, for beginners, if they can't quickly implement a stored procedure and prepared the statement, it's better to filter input data as much they can.
Finally, let's consider that a user sends this text below instead of entering his/her username:
[1] UNION SELECT IF(SUBSTRING(Password,1,1)='2',BENCHMARK(100000,SHA1(1)),0) User,Password FROM mysql.user WHERE User = 'root'
This input can be checked early without any prepared statement and stored procedures, but to be on the safe side, using them starts after user-data filtering and validation.
The last point is detecting unexpected behavior which requires more effort and complexity; it's not recommended for normal web applications.
Unexpected behavior in the above user input is SELECT, UNION, IF, SUBSTRING, BENCHMARK, SHA, and root. Once these words detected, you can avoid the input.
UPDATE 1:
A user commented that this post is useless, OK! Here is what OWASP.ORG provided:
Primary defenses:
Option #1: Use of Prepared Statements (Parameterized Queries)
Option #2: Use of Stored Procedures
Option #3: Escaping all User Supplied Input
Additional defenses:
Also Enforce: Least Privilege
Also Perform: White List Input Validation
As you may know, claiming an article should be supported by a valid argument, at least by one reference! Otherwise, it's considered as an attack and a bad claim!
Update 2:
From the PHP manual, PHP: Prepared Statements - Manual:
Escaping and SQL injection
Bound variables will be escaped automatically by the server. The
server inserts their escaped values at the appropriate places into the
statement template before execution. A hint must be provided to the
server for the type of bound variable, to create an appropriate
conversion. See the mysqli_stmt_bind_param() function for more
information.
The automatic escaping of values within the server is sometimes
considered a security feature to prevent SQL injection. The same
degree of security can be achieved with non-prepared statements if
input values are escaped correctly.
Update 3:
I created test cases for knowing how PDO and MySQLi send the query to the MySQL server when using a prepared statement:
PDO:
$user = "''1''"; // Malicious keyword
$sql = 'SELECT * FROM awa_user WHERE userame =:username';
$sth = $dbh->prepare($sql, array(PDO::ATTR_CURSOR => PDO::CURSOR_FWDONLY));
$sth->execute(array(':username' => $user));
Query Log:
189 Query SELECT * FROM awa_user WHERE userame ='\'\'1\'\''
189 Quit
MySQLi:
$stmt = $mysqli->prepare("SELECT * FROM awa_user WHERE username =?")) {
$stmt->bind_param("s", $user);
$user = "''1''";
$stmt->execute();
Query Log:
188 Prepare SELECT * FROM awa_user WHERE username =?
188 Execute SELECT * FROM awa_user WHERE username ='\'\'1\'\''
188 Quit
It's clear that a prepared statement is also escaping the data, nothing else.
As also mentioned in the above statement,
The automatic escaping of values within the server is sometimes considered a security feature to prevent SQL injection. The same degree of security can be achieved with non-prepared statements, if input values are escaped correctly
Therefore, this proves that data validation such as intval() is a good idea for integer values before sending any query. In addition, preventing malicious user data before sending the query is a correct and valid approach.
Please see this question for more detail: PDO sends raw query to MySQL while Mysqli sends prepared query, both produce the same result
References:
SQL Injection Cheat Sheet
SQL Injection
Information security
Security Principles
Data validation
Security Warning: This answer is not in line with security best practices. Escaping is inadequate to prevent SQL injection, use prepared statements instead. Use the strategy outlined below at your own risk. (Also, mysql_real_escape_string() was removed in PHP 7.)
Deprecated Warning: The mysql extension is deprecated at this time. we recommend using the PDO extension
I use three different ways to prevent my web application from being vulnerable to SQL injection.
Use of mysql_real_escape_string(), which is a pre-defined function in PHP, and this code add backslashes to the following characters: \x00, \n, \r, \, ', " and \x1a. Pass the input values as parameters to minimize the chance of SQL injection.
The most advanced way is to use PDOs.
I hope this will help you.
Consider the following query:
$iId = mysql_real_escape_string("1 OR 1=1");
$sSql = "SELECT * FROM table WHERE id = $iId";
mysql_real_escape_string() will not protect here. If you use single quotes (' ') around your variables inside your query is what protects you against this. Here is an solution below for this:
$iId = (int) mysql_real_escape_string("1 OR 1=1");
$sSql = "SELECT * FROM table WHERE id = $iId";
This question has some good answers about this.
I suggest, using PDO is the best option.
Edit:
mysql_real_escape_string() is deprecated as of PHP 5.5.0. Use either mysqli or PDO.
An alternative to mysql_real_escape_string() is
string mysqli_real_escape_string ( mysqli $link , string $escapestr )
Example:
$iId = $mysqli->real_escape_string("1 OR 1=1");
$mysqli->query("SELECT * FROM table WHERE id = $iId");
A simple way would be to use a PHP framework like CodeIgniter or Laravel which have inbuilt features like filtering and active-record so that you don't have to worry about these nuances.
Warning: the approach described in this answer only applies to very specific scenarios and isn't secure since SQL injection attacks do not only rely on being able to inject X=Y.
If the attackers are trying to hack into the form via PHP's $_GET variable or with the URL's query string, you would be able to catch them if they're not secure.
RewriteCond %{QUERY_STRING} ([0-9]+)=([0-9]+)
RewriteRule ^(.*) ^/track.php
Because 1=1, 2=2, 1=2, 2=1, 1+1=2, etc... are the common questions to an SQL database of an attacker. Maybe also it's used by many hacking applications.
But you must be careful, that you must not rewrite a safe query from your site. The code above is giving you a tip, to rewrite or redirect (it depends on you) that hacking-specific dynamic query string into a page that will store the attacker's IP address, or EVEN THEIR COOKIES, history, browser, or any other sensitive information, so you can deal with them later by banning their account or contacting authorities.
A good idea is to use an object-relational mapper like Idiorm:
$user = ORM::for_table('user')
->where_equal('username', 'j4mie')
->find_one();
$user->first_name = 'Jamie';
$user->save();
$tweets = ORM::for_table('tweet')
->select('tweet.*')
->join('user', array(
'user.id', '=', 'tweet.user_id'
))
->where_equal('user.username', 'j4mie')
->find_many();
foreach ($tweets as $tweet) {
echo $tweet->text;
}
It not only saves you from SQL injections, but from syntax errors too! It also supports collections of models with method chaining to filter or apply actions to multiple results at once and multiple connections.
There are so many answers for PHP and MySQL, but here is code for PHP and Oracle for preventing SQL injection as well as regular use of oci8 drivers:
$conn = oci_connect($username, $password, $connection_string);
$stmt = oci_parse($conn, 'UPDATE table SET field = :xx WHERE ID = 123');
oci_bind_by_name($stmt, ':xx', $fieldval);
oci_execute($stmt);
This post is marked obsolete because the content is out of date. It is not currently accepting new interactions.
Deprecated Warning:
This answer's sample code (like the question's sample code) uses PHP's MySQL extension, which was deprecated in PHP 5.5.0 and removed entirely in PHP 7.0.0.
Security Warning: This answer is not in line with security best practices. Escaping is inadequate to prevent SQL injection, use prepared statements instead. Use the strategy outlined below at your own risk. (Also, mysql_real_escape_string() was removed in PHP 7.)
Using PDO and MYSQLi is a good practice to prevent SQL injections, but if you really want to work with MySQL functions and queries, it would be better to use
mysql_real_escape_string
$unsafe_variable = mysql_real_escape_string($_POST['user_input']);
There are more abilities to prevent this: like identify - if the input is a string, number, char or array, there are so many inbuilt functions to detect this. Also, it would be better to use these functions to check input data.
is_string
$unsafe_variable = (is_string($_POST['user_input']) ? $_POST['user_input'] : '');
is_numeric
$unsafe_variable = (is_numeric($_POST['user_input']) ? $_POST['user_input'] : '');
And it is so much better to use those functions to check input data with mysql_real_escape_string.
I've written this little function several years ago:
function sqlvprintf($query, $args)
{
global $DB_LINK;
$ctr = 0;
ensureConnection(); // Connect to database if not connected already.
$values = array();
foreach ($args as $value)
{
if (is_string($value))
{
$value = "'" . mysqli_real_escape_string($DB_LINK, $value) . "'";
}
else if (is_null($value))
{
$value = 'NULL';
}
else if (!is_int($value) && !is_float($value))
{
die('Only numeric, string, array and NULL arguments allowed in a query. Argument '.($ctr+1).' is not a basic type, it\'s type is '. gettype($value). '.');
}
$values[] = $value;
$ctr++;
}
$query = preg_replace_callback(
'/{(\\d+)}/',
function($match) use ($values)
{
if (isset($values[$match[1]]))
{
return $values[$match[1]];
}
else
{
return $match[0];
}
},
$query
);
return $query;
}
function runEscapedQuery($preparedQuery /*, ...*/)
{
$params = array_slice(func_get_args(), 1);
$results = runQuery(sqlvprintf($preparedQuery, $params)); // Run query and fetch results.
return $results;
}
This allows running statements in an one-liner C#-ish String.Format like:
runEscapedQuery("INSERT INTO Whatever (id, foo, bar) VALUES ({0}, {1}, {2})", $numericVar, $stringVar1, $stringVar2);
It escapes considering the variable type. If you try to parameterize table, column names, it would fail as it puts every string in quotes which is an invalid syntax.
SECURITY UPDATE: The previous str_replace version allowed injections by adding {#} tokens into user data. This preg_replace_callback version doesn't cause problems if the replacement contains these tokens.

Filter/query using OR?

I don't understand whats wrong with this CQL - It just checks if a new users email or username would conflict with an already existing person:
select email,username from person where email=? or username=?
Am I supposed to break it up into two separate queries? It says:
line 1:33 missing EOF at 'or')
You have to use separate queries and merge: there is no OR in CQL (yet).

how SQL injection is done? [duplicate]

This question already has answers here:
Closed 12 years ago.
Possible Duplicate:
XKCD SQL injection - please explain
What is the general concept behind sql injection ?
Being a rails developer
This is unsafe
Booking.find(:all, :conditions => [ 'bookings.user_id = #{params[user_id]]}'] )
and this is safe:--
Booking.find(:all, :conditions => [ 'bookings.user_id = ?', params[user_id]] )
am i right?
So my question is how the sql injection is done?
How those guys do some stuff like that. Any live example/ tutorial where somebody is showing this kind of stuff. Anything basic for knowing the logic.
SQL Injection happens when a programmer gets lazy. A vulnerable query would look like this:
DECLARE #cmd varchar(256)
SET cmd='SELECT #col FROM Table'
EXEC #cmd
With #col being a variable passed into a stored procedure.
Usually, the user would enter a column in that already exists for that variable. But a more devious user could enter something like this:
* FROM Table; DROP DATABASE data;--
The * FROM Table; finishes off the previous statement. Then, DROP DATABASE data; is the payload that does bad things, in this case, dropping the database. Finally, the -- comments out the rest of the query so it doesn't get any errors from the injection.
So, instead of executing this:
SELECT column
FROM Table
You get this:
SELECT *
FROM Table;
DROP DATABASE data;
--
Which is not good.
And this:
All the user has to do is enter:
1234; DROP TABLE BOOKINGS
...
I don't know about rails, but by doing this Booking.find(:all, :conditions => [ 'bookings.user_id = #{params[user_id]]}'] ), you risk that the user give to user_id the value 1 OR 1=1 and as you can see, it will modify your request.
With more injection you could do something like 1; DROP TABLE BOOKINGS etc.
Basically injection is just "hijacking" a basic request to add yours.
Bobby tables
If you have a simple query like
SELECT * FROM bookings WHERE user_id = ORDER BY user_id ASC;
if you don't check user id, it can close your query, then start a new (harmful one) and discard the rest. To achieve this, generally, you would enter something like
1; DELETE FROM bookings; --
initial ; closes the good query, the bad query comes next, then it is closed with ; and -- makes sure that anything that would come next in the good query is commented out. You then end up with
SELECT * FROM bookings WHERE user_id = 1; DELETE FROM bookings; -- ORDER BY user_id ASC;
If your data in properly cleaned and sanatized, a user can try to get their own SQL code to run on the server. for example, let's say you have a query like this:
"SELECT * FROM products WHERE product_type = $type"
where type is unchanged user input from a text field. now, if I were to search for this type:
(DELETE FROM products)
You are gonna be in a world of hurt. This is why it's important to make sure all user input in sanatized before running it in the DB.
Plenty of excellent papers on the theory of SQL injection here:
sql injection filetype:pdf
Should be easy enough to hunt one down that is specific to your language/DB combination.

Resources