I'm making use of QueryProvider in msticpy.data.data_provider to run a Kusto query statement in Jupyter notebook.
The purpose of the query is to extract a specific part of a string that is typically session (other text), where I want to extract the (other text) - hence the extract function in Line 5.
As the content of the (other text) varies, I used the \w+ in the regex.
I can't execute the query successfully as it keeps complaining of syntax error. I have tried to escape the characters but it seems to have no effect because the same error appear. Would anyone have an idea what is the issue? Or point me to any resources?
Screenshot of current code and error returned
you need to escape the backslash (see: https://learn.microsoft.com/en-us/azure/kusto/query/scalar-data-types/string#string-literals)
regardless, you'd be better off using the parse operator
print s = "session abc"
| extend session = extract(#"session (\w+)", 1, s)
print s = "session abc"
| parse s with "session " session
Related
I am using janusgraph with cassandra as storage backend. I am using a node package called as bcrypt to encrypt a password before saving it. The data type for that property in janusgraph is String. This is the password hash string which got generated - $2a$10$JSR6FClewTOHGxwpt/F0AePRzGnKvV2L9gj4TL1dA9fQERLWrig7u
This is the error I am getting while trying to save it in the db:
"message": "startup failed:\nScript88.groovy: 1: illegal string body character after dollar sign;\n solution: either escape a literal dollar sign \"\\$5\" or bracket the value expression \"${5}\" # line 1, column 228.\n elf_reg_ind\",\"2\",\"self_reg_pw\",\"$2a$10$J\n ^\n\n1 error\n",
"Exception-Class": "org.codehaus.groovy.control.MultipleCompilationErrorsException"
Please let me know if you need any other info.
The query you are passing to the server gets compiled with Groovy, and Groovy is attempting to resolve the $ as an identifier. You have a literal $ in your hash, so you need to put a \ in front of each $ to escape it. For example:
{ "gremlin":
"g.V(1234).property('hash', '\$2a\$10\$JSR6FClewTOHGxwpt/F0AePRzGnKvV2L9gj4TL1dA9fQERLWrig7u')"
}
Not a groovy guru myself, but I realised this evaluation is only attempted when using double quotes so I managed to solve this issue by ensuring that I use single quotes.
I'm using PHP so the process is to first json_encode then interchange double quotes for single quotes taking care of any already escaped quotes (if any) in between.
Objective
All I am trying to do is retrieve a single record from a specific table where the primary key matches. I have a feeling I'm greatly over complicating this as it seems to be a simple enough task. I have a theory that it may not know the variable value because it isn't actually pulling it from the Python code but instead trying to find a variable by the same name in the database.
EDIT: Is it possible that I need to wrap my where clause in an expression statement?
Attempted
My Python code is
def get_single_record(name_to_search):
my_engine = super_secret_inhouse_engine_constructor("sample_data.csv")
print("Searching for " + name_to_search)
statement = my_engine.tables["Users"].select().where(my_engine.tables["Users"].c.Name == name_to_search)
# Print out the raw SQL so we can see what exactly it's checking for
print("You are about to run: " + str(statement))
# Print out each result (should only be one)
print("Results:")
for item in my_engine.execute(statement):
print(item)
I tried hard coding a string in its place.
I tried using like instead of where.
All to the same end result.
Expected
I expect it to generate something along the lines of SELECT * FROM MyTable WHERE Name='Todd'.
Actual Result
Searching for Todd
STATEMENT: SELECT "Users"."Name", ...
FROM "Users"
WHERE "Users"."Name" = ?
That is an actual question mark appearing my statement, not simply my own confusion. This is then followed by it printing out a collection of all the records from the table, as though it successfully matched everything.
EDIT 2: Running either my own hard coded SQL string or the generated query by Alchemy returns every record from the table. I'm beginning to think the issue may be with the engine I've set up not accepting the query.
Why I'm Confused
According to the official documentation and third party sources, I should be able to compare to hardcoded strings and then, by proxy, be able to compare to a variable.
I managed to save file path into DocumentDB.
When i try to search document using C# with particular path, it throw exception from DocumentDB. The path i try to search is "\\Dev4\ete\" as show in Error message.
When i tried to run below query in portal. It successfully return me result.
Anyone faced this issue before? What might be the root cause?
The reason this is failing is because '\' is an escape character.
So, when the backend gets your query of "SELECT * FROM c WHERE c.name = "\\Dev4\ete\" it interprets this as \Dev4\ete\ and then that last \ causes the problem because it is seen as a literal escape character with nothing after it.
You should escape each use of '\' with another '\' i.e.
The example query shown above would be
SELECT * FROM c WHERE c.Name = "\\\\\\\\Dev4\\\\ete\\\\"
Same as if you had the literal \n in your node name you would need to escape out the escape characters of \n
I am trying to execute a sql statement in groovy by passing the statement as a string variable rather than a string literal. The reason I am using a string variable is because I am reading the sql statement from a file.
For example (assuming sql is a valid Instance of groovy.sql.Sql - which I have verified):
sql.execute("insert into table(id) values(1)")
Works just fine.
However, the following does not:
def str = "insert into table(id) values(1)"
sql.execute(str)
The last example just hangs when I run it. No SQL errors, it just stalls. I tried putting a println after the execute and it never reaches the println statement.
So, I tried the following variations:
sql.execute("$str")
and
sql.execute("${str}")
and even
sql.execute("?", [str])
for curiosity's sake, but all give the following error:
Mar 21, 2013 6:28:16 PM groovy.sql.Sql execute
WARNING: Failed to execute: ? because: Invalid SQL type: sqlKind = 0
Caught: java.sql.SQLException: Invalid SQL type: sqlKind = 0
java.sql.SQLException: Invalid SQL type: sqlKind = 0
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:77)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1010)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1315)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3576)
at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:3677)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1374)
at runSqlFile.run(runSqlFile.groovy:40)
I am admittedly new to groovy, but I can't figure out why a string literal works but the variable will not. Any help here would be greatly appreciated. Let me know if more information is needed.
Also, the reason I am grabbing individual sql statements from a file is because I could not find a way to execute an entire .sql file using groovy. If there is a simple way to do that which I haven't found, that would also solve my problem. Thanks in advance.
Figured out the reason: I had SQL Developer open connected with the same credentials that I was using in my groovy script. I had executed a few commands in SQL Developer and neglected to commit them, which caused a conflict when running the groovy script. This was the cause of the stalling. As soon as I committed the changes made in SQL Developer, the script ran fine.
Well, this works fine with mySql, so my guess is that it's an issue with the Oracle JDBC driver. I've no experience with it myself, but I understand it's not the most reliable piece of software you can find.
This is a bit of a shot in the dark, but:
sql.execute("$str")
sql.execute("${str}")
In both these calls, the value passed to sql.execute isn't actually a String but a GStringImpl, which might have something to do with your issue.
About your last example:
sql.execute("?", [str])
This just isn't valid. ? will be interpreted as a value, not an SQL statement. What you're doing is passing a string, which happens to contain valid sql. The result if it contained "foobar" would be the same.
The only example for which I can't come up with anything is:
def str = "insert into table(id) values(1)"
sql.execute(str)
Are you sure you're not doing anything to str before passing it to sql.execute? Can you make sure it doesn't somehow become wrapped in a specialised groovy class such as GStringImpl ?
I'm using replaceAll to replace single quotes with "\\\\'" per a colleague's suggestion, but I'm pretty sure that's not enough to prevent all SQL injections.
I did some googling and found this: http://wiki.postgresql.org/wiki/8.1.4_et._al._Security_Release_Technical_Info
This explains it for PostgreSQL, but does the replacing not work for all SQL managers? (Like, MySQL, for example?)
Also, I think I understand how the explanation I linked works for single backslash, but does it extend to my situation where I'm using four backslashes?
Please note that I'm not very familiar with databases and how they parse input, but this is my chance to learn more! Any insight would be appreciated.
Edit: I've gotten some really helpful, useful answers. My next question is, what kind of input would break my implementation? That is, if you give me input and I prepend all single quotes with four backslashes, what kind of input would you give me to inject SQL code? While I am convinced that my approach is naive and wrong, maybe some examples would better teach me how easy it is to inject SQL against my "prevention".
No, because what about backslashes? for instance if you turn ' into \' then the input \' will become \\' which is an unescaped single quote and a "character literal" backslash. For mysql there is mysql_real_escape_string() which should exist for every platform because its in the MySQL library bindings.
But there is another problem. And that is if you have no quote marks around the data segment. In php this looks like:
$query="select * from user where id=".$_GET[id];
The PoC exploit for this is very simple: http://localhost/vuln.php?id=sleep(10)
Even if you do a mysql_real_escape_string($_GET[id]) its still vulnerable to sqli because the attacker doesn't have to break out of quote marks in order to execute sql. The best solution is Parameterized Queries.
No.
This is not enough, and this is not the way to go. And I can say it without even knowing anything about your data, your SQL or even anything about your application. You should never, ever include any user data directly into your SQL. You should use parameterized statements instead.
Besides if you are asking this question you shouldn't write your own SQL by hand in the first place. Use a good ORM instead. Asking if your home-grown regular expression would make your application safe from SQL injection is like asking if your home-grown memory allocation routine that you have written in Assembly language is safe from buffer overruns - to which I would say: if you are asking this question then you should use a memory-safe language in the first place.
A simple case of SQL injection works like this (in pseudocode):
name = form_params["name"]
year = 2011
sql = "INSERT INTO Students (name, year) " +
"VALUES ('" + name + "', " + year + ");"
database_handle.query(sql)
year is supplied by you, the programmer, so it's not tainted, and can be embedded in the query in any way you find suitable; in this case — as an unquoted number.
But name is supplied by the user and so can be anything. Along comes Bobby Tables and inputs this value:
name = "Robert'); DROP TABLE Students; -- "
And the query becomes
INSERT INTO Students (name, year) VALUES ('Robert');
DROP TABLE Students; -- ', 2011);
That substitution turned your one query into two.
The first one gives an error because of the mismatched row count, but that doesn't matter, because the database is able to unambiguously find and run the second query. The attacker can work around the error by fiddling with the input anyway. The -- is a comment so that the rest of the input is ignored.
Note how data suddenly became code — a typical sign of a security problem.
What the suggested replacement does is this:
name = form_params["name"].regex_replace("'", "\\\\'")
How this works is confusing, hence my earlier comment. The string literal "\\\\'" represents the string \\'. The regex_replace function interprets that as the string \'. The database then sees
... VALUES ('Robert\'); DROP TABLE Students; -- ', 2011);
and interprets that correctly as a quite unusual name.
Among other problems this approach is very fragile. If the strings you use in your language don't substitute \\ as \, if your string substitution function doesn't interpret \\ as \ (if it's not a regex function or it uses $1 instead of \1 for backreferences) you could end up with an even number of slashes like
... VALUES ('Robert\\'); DROP TABLE Students; -- ', 2011);
and no SQL injection will be prevented.
The solution is not to check what the language and library does with all possible input you can think of, or to anticipate what it might do in a future version, but rather to use the facilities provided by the database. These usually come in two flavours:
database-aware escaping, which does exactly the right escaping of any data because the client library matches the server and it knows what the character encoding of the database you are querying is:
sql = "... '" + database_handle.escape(name) + "' ..."
out-of-band data submission (usually with prepared statments), so the data isn't even in the same string as the code:
sql = "... VALUES (:n, :y);"
database_handle.query(sql, n = name, y = year)