Usecase that I am trying to solve is:
Find all page references of all components under /apps.
i.e. First find all pages where a component is being used, and
then do this for all components under /apps.
By using the report builder tool for Adobe AEM: https://adobe-consulting-services.github.io/acs-aem-commons/features/report-builder/configuring.html
Query I am trying:
SELECT * FROM [nt:base] AS s
WHERE [sling:resourceType] IN (SELECT path FROM [cq:Component] AS s WHERE [componentGroup] IS NOT NULL AND ISDESCENDANTNODE([/apps]))
AND ISDESCENDANTNODE([/content])
Background:
I only need to sanitize the resultset from inner query.
Without sanitization, it would spit path of the form /apps/acs-commons/components/utilities/report-builder/columns/text
while sling:resourceType from outer query can only accept acs-commons/components/utilities/report-builder/columns/text.
So I need to strip out /apps/ from the inner query resultset path.
Here is the error message:
Caused by: java.text.ParseException: Query: SELECT * FROM [nt:base] AS s
WHERE [sling:resourceType] IN (SELECT(*)CAST(path, AS STRING) FROM [cq:Component] AS s WHERE [componentGroup] IS NOT NULL AND ISDESCENDANTNODE([/apps]))
AND ISDESCENDANTNODE([/content]); expected: static operand
I don't think you can manipulate result set using jcr sql2 syntax, stored procs are usually used to manipulate result sets akin to PL/SQL and I did not find any reference to this in jcr docs. In fact, to my knowledge jcr does not even support aggregate functions like MAX(), COUNT(), etc
Hacky way to do this -> you would probably have to execute the inner query first to retrieve all the components in /apps, modify the result set manually(stripping out /apps) and feed it to the outer query.
SELECT * FROM [nt:unstructured] AS comp
WHERE ISDESCENDANTNODE(comp, "/content/prj")
AND [sling:resourceType] IN ("prj/components/content/accordion","prj/components/content/breadcrumb")
To fasten the process, you can use text editors like notepad++ which helps you with block selection (ctrl + alt + shift and then left click mouse button and drag to select) to remove /apps, add start/end double quotes, comma and replace newline char to get it all in one line and construct the overall query.
Would be interested to know what others think and if this can be accomplished with just jcr sql2syntax.
Related
As community member called Saideep help with an coding issue I had here
I have attempted to modify the code to search for values with without www (after removing either http or https) and prepending it to a field value.
So, at the moment, my code does the following:
You can see from the image, that code successfully removes https:// and http, but it fails to prepend the values with www in the field newwebsite_url when doesn't exists in the homepage_url field
For example movingahead.com should appear as www.movingahead.com in newwebsite_url
My code is as follows:
SELECT tt.homepage_url
,concat(iff(left(v1.RightString, 4)='www.', null, 'www.')) as addwww , LEFT(v1.RightString,COALESCE(NULLIF(CHARINDEX('/',v1.RightString)-1,-1),150)) as newwebsite_url
FROM basecrmcbreport.organizations tt
inner join (select (SUBSTRING(homepage_url,CHARINDEX('//',homepage_url)+2,150)) from basecrmcbreport.organizations)v1(RightString) on tt.homepage_url like concat('%',v1.RightString,'%') escape '|';
I know its a concatenation issue, but not where to fix it.
Any thoughts?
As you suspected, the issue here is with concatenation. When using concat in the query, we have just given the part where taking www. if it is absent and null if it is present.
But we are not concatenating this with the extracted part from URL (instead using it to create new column). To fix this, we have to place the above function value inside concat.
The following is the demonstration of the same using my sample data.
Using the following updated query, I was able to achieve your requirement.
%sql
SELECT tt.homepage_url
,concat(iff(left(v1.RightString, 4)='www.', '', 'www.'),LEFT(v1.RightString,COALESCE(NULLIF(CHARINDEX('/',replace(v1.RightString,'\\','/'))-1,-1),150))) as newwebsite_url
FROM demo tt
inner join (select (SUBSTRING(homepage_url,CHARINDEX('//',homepage_url)+2,150)) from demo)v1(RightString) on tt.homepage_url like concat('%',v1.RightString,'%') escape '|';
NOTE: I have also replaced \ with /. Also, I have used '' (empty string) instead of using null in concat.
At the end of the pipeline I wanted to write the below query
INSERT INTO [TestTable] (Job_name, status) VALUES (Job_name, current_timestamp()).
Jobname will be passed as a parameter.
Can this be written in Lookup, please let me know.
Definitely possible, but should you write massive inserts/scripts within a lookup... probably not a great idea, but see below (Truncate Example, but will work the same with an insert)
I use this method for small things like truncating a table, but never with big code that should be stored as source in the DB.
EDIT:
If you need to pass parameters or Variables into the lookup you should use string interpolation like so:
INSERT INTO dbo.MyTable
SELECT '#{variables('YourVariable')}' as Variable1,
'#{pipeline().Pipeline} as PipelineName
Can you tell me, please?
Why does the mysql2 value substitution not work inside the IN operator?
I do this, but nothing works.
Only the first character of the array is being substituted (number 6)
"select * from products_categories WHERE category_id IN (?)", [6,3]);
You can do it like this, of course:
IN(?,?,?,?,?,?,?,?,?,?) [6,3,1,1,1,1,1,1,1,1,1]
But that's not right, I thought that the IN should be automatically substituted from an array =(
I haven't used this, but my gut feeling tells that array items map to question marks based on indexes, so in your case 6 binds to first ? and 3 looks for another one, but doesn't find.
If I were you, I'd try to make sure that my first array item is then actually array, so I'd rewrite it:
"select * from products_categories WHERE category_id IN (?)", [[6,3]]);
I suspect you are using this with .execute(), which is short for prepared statements "prepare first if never executed before"+execute. While api is very similar to .query() one biggest difference is that in case of prepared statement only parameters are sent at execution time, unlike .query() where whole query text is interpolated with all parameters on the client. As a result, you need to send exactly the number of parameters as you have number of placeholders in original query text ( in you example - one ?). The whole [6,3,1,1,1,1,1,1,1,1,1] is treated as one parameter and sent to server as "6,3,1,1,1,1,1,1,1,1,1" string ( because during prepare step that parameter was likely reported by server as VAR_CHAR )
The solution is 1) use .query() and interpolate on the client or 2) build enough ?s dynamically and prepare different PS for different number of IN parameters
I have a table in Databricks that has a column (called "properties") which contains JSON data. I've successfully used get_json_object() in a SparkSql notebook to retrieve properties from it like so:
%sql
select distinct_id, get_json_object(properties, "$.time")
from my_table
This works well. However, there are sub-properties in the properties column that have asterisks in their names, e.g. *Plan. Accessing these properties in the standard way, e.g. $.*Plan doesn't work, since * has special meaning for get_json_object(). I've tried accessing these properties using escape chars, like so:
%sql
select distinct_id, get_json_object(properties, "$.\*Plan")
from my_table
... along with alternative escapes, but to no avail. Is there a way to extract JSON sub-properties that can escape the asterisk?
Thanks!
You can use LATERAL VIEW and json_tuple as a workaround. It's not so fussy about special characters eg:
SELECT x.*
FROM my_table
LATERAL VIEW json_tuple( properties, '*Plan' ) x
Or if you are wedded to using get_json_object you can clean up the string beforehand (although you have somewhat defeated the point of using JSON):
%sql
select
distinct_id,
get_json_object(replace(properties, '*', ''), '$.Plan' ) z
from my_table
I couldn't personally get any escaping methods to work ( eg \u0042 or \ ) but happy to be corrected.
Ideally, don't put such strange characters in your JSON in the first place.
I have an SSIS package that obtains a list of new GUIDs from a SQL table. I then shred the GUIDs into a string variable so that I have them separated out by comma. An example of how they appear in the variable is:
'5f661168-aed2-4659-86ba-fd864ca341bc','f5ba6d28-7283-4bed-9f11-e8f6bef225c5'
The problem is in the data flow task. I use the variable as a parameter in a SQL query to get my source data and I cannot get my results. When the WHERE clause looks like:
WHERE [GUID] IN (?)
I get an invalid character error so I found out the implicit conversion doesn't work with the GUIDs like I thought they would. I could resolve this by putting {} around the GUID if this were a single GUID but there are a potential 4 or 5 different GUIDs this will need to retrieve at runtime.
Figuring I could get around it with this:
WHERE CAST([GUID] AS VARCHAR(50)) IN (?)
But this simply produces no results and there should be two in my current test.
I figure there must be a way to accomplish this... What am I missing?
You can't, at least not using the mechanics you have provided.
You cannot concatenate values and make that work with a parameter.
I'm open to being proven wrong on this point but I'll be damned if I can make it work.
How can I make it work?
The trick is to just go old school and make your query via string building/concatenation.
In my package, I defined two variables, filter and query. filter will be the concatenation you are already performing.
query will be an expression (right click, properties: set EvaluateAsExpression to True, Expression would be something like "SELECT * FROM dbo.RefData R WHERE R.refkey IN (" + #[User::filter] + ")"
In your data flow, then change your source to SQL Command from variable. No mapping required there.
Basic look and feel would be like
OLE Source query