I have a textInput on UI.R that I want use as a
filter to my query in sqldf on GLOBAL.R, but I know how to do this.
Can anyone help me ?
Here´s my UI.R
fluidPage(Application title titlePanel("Robsonb - CRAI"),
sidebarLayout(
# Sidebar with a slider and selection inputs
sidebarPanel(
selectInput("selection", "Escolha uma lista:",
choices = books),
actionButton("update", "Alterar"),
hr(),
textInput(inputId="descritor",value="cárie", label = ""),
sliderInput("freq",
"Minimum Frequency:",
min = 1, max = 50, value = 15),
sliderInput("max",
"Maximum Number of Words:",
min = 1, max = 300, value = 100)
),
# Show Word Cloud
mainPanel(
plotOutput("plot"),
textOutput("descritor")
) ) )
Here´s my Global.R, where I do a sql query. I want to do something
like sqldf("select Assuntos, sysno from outcome O3 where
[Resumo.em.português] LIKE . I´m using a shiny package.
library(tm)
library(wordcloud)
library(memoise)
library(sqldf)
# The list of valid books
books <<- list("Resumo de Teses FOUSP" = "tesesFO"
)
# Using "memoise" to automatically cache the results
getTermMatrix <- memoise(function(book) {
# Careful not to let just any name slip in here; a
# malicious user could manipulate this value.
if (!(book %in% books))
stop("Base não encontrada")
outcome <- read.csv2("TesesFOResumo.csv", colClasses = "character")
#input<- input$descritor
sqldf("select Assuntos, sysno from outcome O3 where [Resumo.em.português] LIKE '%cárie%' UNION ALL
select Assuntos, sysno from outcome O1 where sysno in (select sysno from outcome O2 where assuntos LIKE '%cárie%')")
text <- sqldf("select Assuntos, sysno from outcome O3 where [Resumo.em.português] LIKE '%cárie%' UNION ALL
select Assuntos, sysno from outcome O1 where sysno in (select sysno from outcome O2 where assuntos LIKE '%cárie%')")
myCorpus = Corpus(VectorSource(text))
myCorpus = tm_map(myCorpus, content_transformer(tolower))
myCorpus = tm_map(myCorpus, removePunctuation)
myCorpus = tm_map(myCorpus, removeNumbers)
myCorpus = tm_map(myCorpus, removeWords,
c(stopwords("SMART"), "de", "da", "dos", "the", "and", "but"))
myDTM = TermDocumentMatrix(myCorpus,
control = list(minWordLength = 1))
m = as.matrix(myDTM)
sort(rowSums(m), decreasing = TRUE)
})
Related
Fist row in Data File:
1,Maria,Anders,Berlin,Germany,0300174321
f = open("Customer.csv", "r")
for row in f.readlines(i):
a = row
x = a.split(",")
ID1 = print(x[0].replace("",""))
FIRST_NM1 = print(x[1])
LAST_NM1 = print(x[2])
CITY1 = print(x[3])
COUNTRY1 = print(x[4])
PHONE1 = print(x[5])
cursor = cs.cursor()
cursor.execute("INSERT INTO sales.dbo.Customer_temp (ID,FIRST_NM,LAST_NM,CITY,COUNTRY,PHONE) VALUES ('%s','%s','%s','%s','%s','%s')" %(ID1,FIRST_NM1,LAST_NM1,CITY1,COUNTRY1,PHONE1))
cs.commit();
But it is inserting None in all rows so could you please suggest me.
Instead of printing the values you need to assign them;
FIRST_NM1 = x[1]
LAST_NM1 = x[2]
CITY1 = x[3]
etc..
I have a sql like:
String sql = """SELECT id, name, sex, age, bron_year, address, phone, state, comment, is_hbp, is_dm, is_cva, is_copd, is_chd, is_cancer, is_floating, is_poor, is_disability, is_mental
FROM statistics_stin WHERE 1=1
${p.team_num == 0 ? "" : "AND team_num = ${p.team_num}"}
${p.zone == 0 ? "" : "AND team_id = ${p.zone}"}
${p.is_hbp == 2 ? "" : "AND is_hbp = ${p.is_hbp}"}
${p.is_dm == 2 ? "" : "AND is_dm = ${p.is_dm}"}
${p.is_chd == 2 ? "" : "AND is_chd = ${p.is_chd}"}
${p.is_cva == 2 ? "" : "AND is_cva = ${p.is_cva}"}
${p.is_copd == 2 ? "" : "AND is_copd = ${p.is_copd}"}
${p.is_cancer == 2 ? "" : "AND is_cancer = ${p.is_cancer}"}
${p.is_floating == 2 ? "" : "AND is_floating = ${p.is_floating}"}
${p.is_poor == 2 ? "" : "AND is_poor = ${p.is_poor}"}
${p.is_disability == 2 ? "" : "AND is_disability = ${p.is_disability}"}
${p.is_mental == 2 ? "" : "AND is_mental = ${p.is_mental}"}
${p.is_aged == 2 ? "" : (p.is_aged == 1 ? " AND age >= 65" : " AND age < 65")}
${p.is_prep_aged == 2 ? "" : (p.is_prep_aged == 1 ? "AND (age BETWEEN 60 AND 64)" : "AND (age < 60 OR age > 64)")}
${p.is_young == 2 ? "" : (p.is_young == 1 ? " AND age < 60" : " AND age >= 60")}
ORDER BY team_id ASC, id ASC
LIMIT ${start}, ${page_size}
""";
Then use :
def datasource = ctx.lookup("jdbc/mysql");
def executer = Sql.newInstance(datasource);
def rows = executer.rows(sql);
Here the p is a json object like:
p = {is_aged=2, is_cancer=2, is_chd=1, is_copd=2, is_cva=2, is_disability=2, is_dm=2, is_floating=2, is_hbp=1, is_mental=2, is_poor=2, pn=1, team_num=0, zone=0}
This way has sql injection. I know I can use params type like:
executer.rows('SELECT * from statistics_stin WHERE is_chd=:is_chd', [is_chd: 1]);
But this case has to many AND conditions, use or not will be decided by the json p.
How to do this please?
You have a problem of dynamic SQL binding, i.e. the number of bind parameters is not constant, but depend on the input.
There is an elegant solution from Tom Kyte
which has even more elegant implementation in Groovy
The basic idea is simple bind all variables, the variables that have an input value and should be used are processed normally, e.g
col1 = :col1
the variables that have no input (and shall be ignored) are binded with a dummy construct:
(1=1 or :col2 is NULL)
i.e. they are effectively ignored by shortcut evaluation.
Here two examples for three columns
def p = ["col1" : 1, "col2" : 2, "col3" : 3]
This input leads to a full query
SELECT col1, col2, col3
FROM tab WHERE
col1 = :col1 AND
col2 = :col2 AND
col3 = :col3
ORDER by col1,col2,col3
For limited input
p = [ "col3" : 3]
you get this query
SELECT col1, col2, col3
FROM tab WHERE
(1=1 or :col1 is NULL) AND
(1=1 or :col2 is NULL) AND
col3 = :col3
ORDER by col1,col2,col3
Here the Groovy creation of the SQL Statement
String sql = """SELECT col1, col2, col3
FROM tab WHERE
${(!p.col1) ? "(1=1 or :col1 is NULL)" : "col1 = :col1"} AND
${(!p.col2) ? "(1=1 or :col2 is NULL)" : "col2 = :col2"} AND
${(!p.col3) ? "(1=1 or :col3 is NULL)" : "col3 = :col3"}
ORDER by col1,col2,col3
"""
You can even get rid of the ugly 1=1 predicate;)
Another option is to build your bindings as you build the query then execute the appropriate implementation of rows
def query = new StringBuilder( "SELECT id, name, sex, age, bron_year, address, phone, state, comment, is_hbp, is_dm, is_cva, is_copd, is_chd, is_cancer, is_floating, is_poor, is_disability, is_mental FROM statistics_stin WHERE 1=1" )
def binds = []
if ( p.team_num == 0 ) {
query.append( ' AND team_num = ? ' )
binds << p.team_num
}
if ( p.zone == 0 ) {
query.append( ' AND team_id = ? ' )
binds << p.zone == 0
}
...
executer.rows(query.toString(), binds);
Using the DocumentDB query playground, I am working on a filter type of query. I have a set of attributes in my data that are set up to allow the user to search by the specific attribute. Each attribute type becomes and OR statement if multiple items are selected from the name in the name/value collection. If attributes are selected that differ (i.e. color and size) this becomes an AND statement.
SELECT food.id,
food.description,
food.tags,
food.foodGroup
FROM food
JOIN tag1 IN food.tags
JOIN tag2 IN food.tags
WHERE (tag1.name = "snacks" OR tag1.name = "granola bars")
AND (tag2.name = "microwave")
This query works beautifully in the playground.
The main issue is that I have up to 12 attributes, and maybe more. Once I hit 5 joins, that is my maximum allowed number of joins, so the query below doesn't work. (note that this isn't playground data, but a sample of my own)
SELECT s.StyleID FROM StyleSearch s
JOIN a0 in s.Attributes
JOIN a1 in s.Attributes
JOIN a2 in s.Attributes
JOIN a3 in s.Attributes
JOIN a4 in s.Attributes
JOIN a5 in s.Attributes
WHERE (a0 = "color-finish|Grey" OR a0 = "color-finish|Brown" OR a0 = "color-finish|Beige")
AND (a1 = "fabric-type|Polyester" OR a1 = "fabric-type|Faux Leather")
AND (a2 = "design-features|Standard" OR a2 = "design-features|Reclining")
AND (a3 = "style_parent|Contemporary" OR a3 = "style_parent|Modern" OR a3 = "style_parent|Transitional")
AND (a4 = "price_buckets|$1500 - $2000" OR a4 = "price_buckets|$2000 and Up")
AND (a5 = "dimension_width|84 in +")
I am not 100% sure I am using the proper query to perform this, but a simple where clause per below which works in SQL brings back anything matching in the or statements so I end up with items from each "AND statement.
SELECT s.StyleID FROM StyleSearch s
JOIN a in s.Attributes
WHERE (a = "color-finish|Grey" OR a = "color-finish|Brown" OR a = "color-finish|Beige")
AND (a = "fabric-type|Polyester" OR a = "fabric-type|Faux Leather")
AND (a = "design-features|Standard" OR a = "design-features|Reclining")
AND (a = "style_parent|Contemporary" OR a = "style_parent|Modern" OR a = "style_parent|Transitional")
AND (a = "price_buckets|$1500 - $2000" OR a = "price_buckets|$2000 and Up")
AND (a = "dimension_width|84 in +")
Here is an example of the data:
{
"StyleID": "chf_12345-bmc",
"Attributes": [
"brand|chf",
"color|red",
"color|yellow",
"dimension_depth|30 in +",
"dimension_height|counter height",
"materials_parent|wood",
"price_buckets|$500 - $1000",
"style_parent|rustic",
"dimension_width|55 in +"
]
}
I am looking for the proper way to handle this. Thanks in advance.
Is it possible for you to change the structure of your document to add filter attributes specifically for your query on e.g.
{
"StyleID": "chf_12345-bmc",
"Attributes": [
"brand|chf",
"color|red",
"color|yellow",
"dimension_depth|30 in +",
"dimension_height|counter height",
"materials_parent|wood",
"price_buckets|$500 - $1000",
"style_parent|rustic",
"dimension_width|55 in +"
],
"filter_color": "red,yellow",
"filter_fabric_type":"Polyester,leather"
}
This would eliminate the join restriction because now your query looks something like this:
SELECT s.StyleID FROM StyleSearch s
WHERE (CONTAINS(s.filter_color, "Grey") OR CONTAINS(s.filter_color, "Red"))
AND (CONTAINS(s.filter_fabric_type, "Polyester") OR CONTAINS(s.filter_fabric_type, "Leather"))
Of course this does mean that you have additional fields to maintain.
You might also consider writing a stored proc for this and using javascript to loop through your collection and filtering that way: DocumentDB stored procedure tutorial
I'm working on row level security in Spotfire (6.5) report.
It should be implemented on 3 levels, lets call it L1, L2 and L3. There is additional mapping table that contains Userlogins and specified values on all levels where user has access. Additionaly if user is not in mapping table he is some kind of Root user so he has access to everything.
On DB side it looks like that:
CREATE TABLE SECURITY
(
USER_ID VARCHAR2(100 BYTE)
, L1 VARCHAR2(100 BYTE)
, L2 VARCHAR2(100 BYTE)
, L3 VARCHAR2(100 BYTE)
--, L1L2L3 VARCHAR2(100 BYTE) -- option there could be one column that contains lowest possible level
);
INSERT INTO SECURITY (USER_ID, L1) VALUES ('UNAME1','A');
INSERT INTO SECURITY (USER_ID, L2) VALUES ('UNAME2','BB');
INSERT INTO SECURITY (USER_ID, L3) VALUES ('UNAME3','CCC');
CREATE TABLE SECURED_DATA
(
L1 VARCHAR2(100 BYTE)
, L2 VARCHAR2(100 BYTE)
, L3 VARCHAR2(100 BYTE)
, V1 NUMBER
);
INSERT INTO SECURED_DATA (L1, V1) VALUES ('A',1);
INSERT INTO SECURED_DATA (L1, L2, V1) VALUES ('B','BB',2);
INSERT INTO SECURED_DATA (L1, L2, L3, V1) VALUES ('C','CC','CCC',3);
Finally I've made Information Link and then I've changed its' sql code to something like that:
SELECT
M.*
FROM
SECURITY S
INNER JOIN SECURED_DATA M
ON
(
M.L1 = S.L1
AND S.USER_ID = (%CURRENT_USER%)
)
UNION ALL
SELECT
M.*
FROM
SECURITY S
INNER JOIN SECURED_DATA M
ON
(
M.L2 = S.L2
AND S.USER_ID = (%CURRENT_USER%)
)
UNION ALL
SELECT
M.*
FROM
SECURITY S
INNER JOIN SECURED_DATA M
ON
(
M.L3 = S.L3
AND S.USER_ID = (%CURRENT_USER%)
)
UNION ALL
SELECT
M.*
FROM
SECURED_DATA M
WHERE
(
SELECT
COUNT(1)
FROM
SECURITY S
WHERE S.USER_ID = (%CURRENT_USER%)
)
=0
It works fine, but I'm worndering if there is more smart and more Spotfire way to get it?
Many thanks and regards,
Maciej
My guess on "more smart and more Spotfire way" is that you want to be able to cache a single data set and use it for multiple users, limiting it in the analytic rather than in the data pull. There is some danger to this, if we're doing it for security's sake, because the data will technically be in the analytic, and if they have permission to edit and add visualizations, you no longer control what they can and cannot see. If there's any authoring allowed in Web Player for the specific analytic, I recommend all securities be done DataBase-side.
If you want to do it in Spotfire anyways, here is my recommendation:
Have an Information Link (for example case, named IL_SecurityCheck) which is Select * from SECURITY WHERE S.USER_ID = (%CURRENT_USER%).
If they move from a cover page to the page with the data in it, you can put the code in the script to change pages; if not, you can use a method I explained here: Spotfire Current Date in input field with calendar popup to fire off a script on open.
Button Script required:
from Spotfire.Dxp.Data import *
crossSource = Document.Data.Tables["IL_SecurityCheck"]
rowCount = crossSource.RowCount
rowIndexSet = IndexSet(rowCount, True)
print rowCount
#rowCount = Document.Data.Tables["Managed Care UpDownStream"].RowCount
colCurs = DataValueCursor.CreateFormatted(crossSource.Columns["L1"])
colCurs2 = DataValueCursor.CreateFormatted(crossSource.Columns["L2"])
colCurs3 = DataValueCursor.CreateFormatted(crossSource.Columns["L3"])
x = ""
if rowIndexSet.IsEmpty != True:
for row in crossSource.GetRows(rowIndexSet, colCurs):
if colCurs.CurrentValue is not None:
x += "[L1] = '" + colCurs.CurrentValue + "' and "
for row in crossSource.GetRows(rowIndexSet, colCurs2):
if colCurs2.CurrentValue is not None:
x += "[L2] = '" + colCurs2.CurrentValue + "' and "
for row in crossSource.GetRows(rowIndexSet, colCurs3):
if colCurs3.CurrentValue is not None:
x += "[L3] = '" + colCurs3.CurrentValue + "' and "
x = x[:len(x) - 4]
else:
x = "1=1"
Document.Properties["SecurityLimits"] = x
Visualization Data Limited by Expression: ${SecurityLimits}
I am trying to build an easy search based on 2 MySQL tables. One called keywords (words) and another called keyword2data (map which binds words to datasource).
Keywords holds id and keyword whilst keywords2data holds keyword_id and data_id.
data_id it self is a reference to a 3rd but in this case unimportant table.
What i want is to be able to search for example "dog sled" and get all data_id's which has those keywords bound to it.
SELECT k2d.`data_id` , k2d.`keyword_id`
FROM keywords2data as k2d, keywords as k
WHERE k2d.`keyword_id` = k.`id`
&& (k.`keyword` = 'dog' || k.`keyword` = 'sled')
LIMIT 10
Gives me all data_id which has either dog or sled bound to it, not necessary both, which is what i want.
SELECT k2d.`data_id` , k2d.`keyword_id`
FROM keywords2data as k2d, keywords as k
WHERE k2d.`keyword_id` = k.`id`
&& (k.`keyword` = 'dog' && k.`keyword` = 'sled')
LIMIT 10
Gives me nothing since no single row in keywords2data holds 2 keywords.
What is the right way to do this?
How about something like
SELECT k2d.`data_id` ,
k2d.`keyword_id`
FROM keywords2data as k2d INNER JOIN
keywords as k ON k2d.`keyword_id` = k.`id` INNER JOIN
keywords as k2 ON k2d.`keyword_id` = k2.`id`
WHERE k.`keyword` = 'dog'
AND k2.`keyword` = 'sled'
LIMIT 10
How about this?
SELECT k2d.`data_id`,
k2d.`keyword_id`
FROM keywords2data AS k2d
INNER JOIN keywords AS k
ON k2d.`keyword_id` = k.`id`
WHERE k.`keyword` IN ( 'dog', 'sled', 'rex' )
GROUP BY k.keyword
HAVING COUNT(*) = 3
Possibly, this?
To extend to match more keywords you'd just add more words to the OR statement in the subquery and change the =2 afterwards.
This assumes that each data item is linked to a keyword using kerywords2data once and only once.
SELECT k2d.data_id
, k2d.keyword_id
FROM keywords2data AS k2d
, keywords AS k
WHERE k2d.keyword_id = k.id
AND (
SELECT COUNT(*)
FROM keywords2data AS sqk2d
, keywords AS sqk
WHERE sqk2d.data_id = k2d.data_id
AND sqk2d.keyword_id = sqk.id
AND (sqk.keyword = 'dog' || sqk.keyword = 'sled')
) = 2
LIMIT 10
Here's a version that doesn't return the data_id repeated (as per comments), but also doesn't return any keywords at all:
SELECT k2d.data_id
FROM keywords2data AS k2d
WHERE (
SELECT COUNT(*)
FROM keywords2data AS sqk2d
, keywords AS sqk
WHERE sqk2d.data_id = k2d.data_id
AND sqk2d.keyword_id = sqk.id
AND (sqk.keyword = 'dog' || sqk.keyword = 'sled')
) = 2
LIMIT 10