The following is confusing me a lot. I have been spending quite a bit of time trying to understand why collection.find() doesn't work with regex passed as an object. The regex match is coming over HTTP wrapped in the body of a POST request. Then I try to gather the query (in string format) and perform the query. The problem seems to be that unless the regex is written inside Node without quotes, it won't work. That is, it must be a literal without quotes.
For example, the following works fine:
var query1 = {
company: {
'$regex': /goog/
}
};
collection.find(query1, {}).toArray(function (err, docs) {
// Got results back. Awesome.
});
However, if the data comes wrapped in an object, it doesn't return anything. I suspect it's because the value gets quoted behind the scenes (i.e. "/goog/"):
// Assume
var query2 = {
company: {
'$regex': query.company
}
};
collection.find(query2, {}).toArray(function (err, docs) {
// Got nothing back.
});
I have tested it with the mongo shell and I can confirm the following:
// Returns 5 results
db.getCollection("contacts").find( { "company": /goog/ } )
// Doesn't match anything
db.getCollection("contacts").find( { "company": "/goog/" } )
Furthermore, I just discovered the following: if I write the value with quotes
// Works fine
var companyRegex = {'$regex': /goog/};
var query3 = {
company: companyRegex
};
So technically, a "literal" regex without quotes wrapped in an object works fine. But if it's a string, it won't work. Even after trying to replace the double-quotes and single-quotes with nothing (i.e. essentially removing them.)
Any idea how can I get the regex match be passed verbatim to find()? I've researched it, finding lots of potential solutions, alas it's not working for me.
Thanks in advance!
Let me focus on one line of your post. This is where the problem might be:
The regex match is coming over HTTP wrapped in the body of a POST request.
This seems problematic because:
The only structures that survive serialization between client/server are:
boolean
number
string
null *
objects and arrays containing these basic types
objects and arrays containing object and arrays [of more obj/array] of these basic types
Regexp, Date, Function, and a host of others require reconstruction, which means
passing a string or pair of strings for the match and option components of the Regexp and running Regexp() on the receiving end to reconstruct.
Regexp gets a bit messy because Regexp.toString() and Regexp() do not appear to be inverses of each others: /someMatch/.toString() is "/someMatch/" but RegExp("/someMatch/") is //someMatch// and what was needed instead to rebuild the regexp was just RegExp("someMatch"), which is /someMatch/. I hope this helps.
JSON.stringify(/someMatch/) is {} (at least on Chrome).
So instead of trying to build a general transport, I recommend re instantiating a particular field as a regexp.
* Irrelevant note: (null is fine but undefined is peculiar. JSON won't stringify undefineds in objects and turns undefined into null in Arrays. I recognize this isn't part of your problem, just trying to be complete in describing what can be serialized.)
Related
I am trying to execute a simple query with pattern matching using LIKE in the WHERE clause using node-oracledb NodeJS library, however I can't seem to figure out how to do it. This doesn't seem to work
const baseQuery =
`select item, item_desc
from item_master
where item_level=tran_level
and item like '%:item%'`;
const binds = { item: '550' };
const result = await conn.execute(baseQuery, binds, {});
It keeps throwing the error below-
[Error: ORA-01036: illegal variable name/number] {
errorNum: 1036,
offset: 0
}
I've read the node-oracledb documentation but this seemingly simple use case of pattern match queries doesn't seem to be documented anywhere. Is pattern matching supported? if yes, then what am I doing wrong?
This was answered in your cross post at https://github.com/oracle/node-oracledb/issues/1195
Use a normal bind variable in the SQL statement (which is important for scalability and security, so that data is never treated as part of the SQL statement). Then concatenate whatever pattern matching syntax you want to the data.
To quote sla100's answer in the GitHub issue:
SQL:
and item like :item
JS:
const binds = { item: '%550%' };
You can do this with LIKE comparisons and also REGEXP_LIKE. The node-oracledb documentation also has examples of doing this in WHERE IN.
I have written a piece of software that parses and formats the fourth parameter of a graphql resolver function (the info object) to be used elsewhere. I would like to write unit tests for this software. Specifically, I do not want to build the GraphQLResolveInfo object myself, because doing that would be very cumbersome, error-prone and hard to maintain. Instead, I want to write human-readable query strings and convert them to GraphQLResolveInfo objects so I can pass those to my software.
After extensive googling and reading of the graphql-js source code, I have not found a simple way to do what they are doing internally. I'm really hoping that I am missing something.
What I am not trying to do is use the graphql-tag library, because that just generates an AST which has a very different format from the GraphQLResolveInfo type.
Has anyone done this before? Help would be much appreciated!
I will keep monitoring this question to see if a better answer comes along, but I've finally managed to solve my particular issue by creating as close an approximation of the GraphQLResolveInfo object as I need for my particular use case.
The GraphQLResolveInfo object is composed of several attributes, two of which are called fieldNodes and fragments. Both are in fact parts of the same AST that graphql-tag generates from a query string. These are the only parts of the GraphQLResolveInfo object that concern the software I wrote, the rest of it is ignored.
So here is what I did:
import gql from 'graphql-tag';
// The converter function
const convertQueryToResolveInfo = (query) => {
const operation = query.definitions
.find(({ kind }) => kind === 'OperationDefinition');
const fragments = query.definitions
.filter(({ kind }) => kind === 'FragmentDefinition')
.reduce((result, current) => ({
...result,
[current.name.value]: current,
}), {});
return {
fieldNodes: operation.selectionSet.selections,
fragments,
};
};
// An example call
const query = gql`
query {
foo {
bar
}
}
`;
const info = convertQueryToResolveInfo(query);
From the AST generated by graphql-tag, I extract and modify the operation and fragment definitions so that they look the way they do within the GraphQLResolveInfo object. This is by no means perfect and may be subject to change in the future depending on how my software evolves, but it is a relatively brief solution for my particular problem.
So in my js-code I have this line:
var _script = {
_script: {
script: {
lang: 'painless',
source: `
"""
if(1>2){
params._source.id;
}
else{
params._source.id;
}
"""
`
},
type: 'string',
order: params._source.id
}
}
This will fail. I see in the log this error message:
,\"reason\":\"unexpected token ['\\\"\\\\n if(1>2){\\\\n params._source.id;\\\\n }\\\\n else{\\\\n params._source.id;\\\\n }\\\\n \\\"'] was expecting one of [{<EOF>, ';'}].\"}}}]},
I have tried first to have without tilde-character. And then it also fails.
I then tried to have tilde at the beginning, something like:
var _script = `{
Thing is that the final json that will be sent to elastic is not shown in the code above. So "_script" is only a little part of all the json.
I was wondering if I added the tilde at the very beginning and end of the whole json. Maybe it could work? I need to work it out where it is.
But just in theory: do you think the problem is there? Putting the tilde around all the json? Or is it something else?
The triple " is not valid JSON, it only works internally to the Elastic stack (i.e. from Kibana Dev Tools to ES).
The way I usually do it from Node.js is to add each line to an array and then I join that array, like this:
const code = [];
code.push("if(1>2){");
code.push("params._source.id;");
code.push("} else {");
code.push("params._source.id;");
code.push("}");
source = code.join(" ");
It's not super legible, I admit. Another way is to use stored scripts so you can simple reference your script by ID in Node.js.
Does the BotBuilder Node SDK actively strip out anything that is stored the dialogData object?
For example, I have created a simple loop and I am storing a regex in session.dialogData.questions. When I console log this after storing it, I can see that my regex is stored as expected:
{
validation: /^[0-9]{19}$/,
}
However, when I try and log the same session.dialogData.questions object in the next step of my waterfall, then the regex seems to have been converted into an empty object:
{
validation: {}
}
I presume this a deliberate attempt to prevent XSS and other types of exploitation?
The code for this example can be found below:
const builder = require('botbuilder')
const lib = new builder.Library('FormBuilder')
lib.dialog('/', [
(session, args) => {
session.dialogData.questions = {
validation: /^[0-9]{19}$/
}
console.log(session.dialogData.questions)
builder.Prompts.confirm(session, 'Would you like to proceed?')
},
(session, results) => {
console.log(session.dialogData.questions)
}
])
module.exports.createLibrary = () => {
return lib.clone()
}
Regarding your initial question, no the SDK doesn't actively strip anything out of the dialogData object. Anything that is, except for regexp...
I'm not sure why this is, but for the time being I recommend storing your pattern as a string, '^[0-9]{19}$', and then constructing a new regexp via new RegExp(session.dialogData.questions.validation) when needed.
I tried storing a method to construct a new RegExp using this.questions.validation, but likewise this was also stripped out.
Edit:
Per Ezequiel's comment, this isn't a Bot Framework issue in the end. It is not possible to store non-serializable data inside JSON.
This is very odd... I'm using populate() with a ref to fill in an array within my schema, but then the properties are inaccessible. In other words, the schema is like this:
new Model('User',{
'name': String,
'installations': [ {type: String, ref: 'Installations'} ],
'count': Number,
}
Of course, Insallations is another model.
Then I find & populate a set of users...
model.find({count: 0}).populate('installations').exec( function(e, d){
for(var k in d)
{
var user = d[k];
for(var i in user.installations)
{
console.log(user.installations[i]);
}
}
} );
So far so good! I see nice data printed out, like this:
{ runs: 49,
hardware: 'macbookpro10,1/x86_64',
mode: 'debug',
version: '0.1' }
However, if I try to actually ACCESS any of those properties, they're all undefined! For example, if I add another console log:
console.log(user.installations[i].mode);
Then I see "undefined" printed for this log.
If I try to operate on the object, like this:
Object.keys(user.installations[i]).forEach(function(key) { } );
Then I get a typical "[TypeError: Object.keys called on non-object]" error, indicating that user.installations[i] is not an object (even though it is outputted to the console as if it were). So, I even tried something ugly like...
var install = JSON.parse(JSON.stringify(user.installations[i]));
console.log(install, install.mode);
And, again, the first output (install) is a nice object containing the property 'mode'... but the 2nd output is undefined.
What gives?
Finally, I solved this...
I tried doing a console.log(typeof user.installations[i]); and got "string" as the output. This seemed odd, given that printing the object directly created console output (above) that looked like a normal object, not a string. So, I tried doing a JSON.parse(); on the object, but received the error "SyntaxError: Unexpected token r"
Finally, I realized what was going on. The "pretty console output" I described above was the result of a string formatted with \n (newlines). I had not expected that, for whatever reason. The JSON.parse() error is due to the fact that there is a known necessity with the node.js parser when attempting to parse object keys without quotations; see the SO question here:
Why does JSON.parse('{"key" : "value"}') do just fine but JSON.parse('{key : "value"}') doesn't? .
Specifically, note that the JSON parser in my case is failing on the character 'r', the fist character of "runs," which is the first key in my JSON string (above). At first I was afraid I needed to get a custom JSON parser, but then the problem hit me.
Look back to my original schema. I used a String-type to define the installation ref, because the array field was storing the installations's _id property as a String. I assume the .populate() field is converting the object to a String on output.
Finally, I looked at the Mongoose docs a little closer and realized I'm supposed to be referencing the objects based upon Schema.ObjectID. This explains everything, but certainly gives me some fixing to do in my schemas and code elsewhere...