in order to access a nuxeo installation i am using nuxeo shell and run a query
query -uid "SELECT * FROM Document where dc:created > DATE '2022-03-10' and ecm:path STARTSWITH '/default-domain/workspaces/Patricia/Documents' and ecm:primaryType <> 'Folder' AND ecm:currentLifeCycleState != 'deleted'"
i do get results, but they are limited to 1000 rows.
How do i raise or eliminate this limit ?
Or alternatively, how can i run a query in Nuxeo's UI so i can export a list of uid ?
Related
I have an hql file. I want to run it using pyspark with Hive warehouse connector. There is an executeQuery method to run queries. I want to know whether hql files can be run like that. Can we run complex queries like that.
Please suggest.
Thanks
I have following solution where i have assumed that there will be multiple queries in hql file.
HQL File : sample_query.hql
select * from schema.table;
select * from schema.table2;
Code : Iterate over each query. You can do as you wish(in terms of HWC operation) in each iteration.
with open('sample_query.hql', 'r') as file:
hql_file = file.read().rstrip()
for query in [x.lstrip().rstrip() for x in hql_file.split(";") if len(x) != 0] :
hive.executeQuery("{0}".format(query))
I am newbie to Hadoop and Hive. My current requirement is to collect the stats of number of records loaded in 15 tables on each run day. Instead of executing each select Count(*) query and copy output manually to XL. Could anyone suggest what is the best method to automate this task please?
Note: we are not having any GUI to run Hive Queries, submitting Hive queries in normal Unix terminal.
Export to the CSV or TSV file, then open file in Excel. Normally it generates TSV file (tab-separated). This is how to transform it to comma-separated if you prefer CSV;
hive -e "SELECT 'table1' as source, count(*) cnt FROM db.table1
UNION ALL
SELECT 'table2' as source, count(*) cnt FROM db.table2" | tr "\t" "," > mydata.csv
Add more tables to the query.
You can mount directory in which you are writing output file in Windows using SAMBA/NFS. Schedule the command using crontab and voila, every day you have updated file.
Also you can connect directly using ODBC drivers:
https://mapr.com/blog/connecting-apache-hive-to-odbc/
https://learn.microsoft.com/en-us/azure/hdinsight/hadoop/apache-hadoop-connect-excel-hive-odbc-driver
Error connecting Hortonworks Hive ODBC in Excel 2013
I would like to remove some items from a table using Impex. The following example throws no errors, but nothing is removed.
REMOVE ProductReference;pk[unique=true]
"#% impex.exportItemsFlexibleSearch(""select {pk} from {ProductReference as pr} where {pr.referenceType}=( {{select {pk} from {ProductReferenceTypeEnum as prte} where {prte.code} = 'CROSSELLING'}})"");"
The query produces results as expected. Is REMOVE not compatible with flexible search, or am I missing something?
The problem is, that I am running an import over hotfolder and I want to remove all existing items beforehand. Alternative solutions are welcome.
Importing the query-
REMOVE ProductReference;pk[unique=true]
"#% impex.exportItemsFlexibleSearch(""select {pk} from {ProductReference as pr} where {pr.referenceType}=( {{select {pk} from {ProductReferenceTypeEnum as prte} where {prte.code} = 'CROSSELLING'}})"");"
is not working because you have not selected the Enable code execution checkbox.Also, as suggested by #B.M replacing the script with impex.includeSQLData() and #% impex.initDatabase() would not have any effect if the checkbox is not selected.However, selecting the checkbox and running the above script will give error, because there is no method by the name, exportItemsFlexibleSearch in the class MyImpExImportReader(which is called on running import).The method exportItemsFlexibleSearch is available in DeprecatedExporter (which is called on running export not import).Now, running this impex script in export will execute successfully without any error, but it won't remove anything. Instead, it will create a zip file with an impex and a script file. This script file will have the impex header for removing the items returned by the query. Using this zip file we can delete the items, conditionally.
Go to HMC -> Cronjobs -> Create a new cronjob of type Impex import job -> Upload the zip file in media attribute -> Create -> Run the impex.
This would delete the items returned by the query.There is another way of deleting the items selected by the query.
We need to export the script to generate zip file of import script and media file.Resultant zip file need to be imported with enablecodeexecution checked
Alternatively groovy script can be executed, an example:
import de.hybris.platform.servicelayer.search.FlexibleSearchQuery;
flexibleSearchService = spring.getBean("flexibleSearchService")
modelService = spring.getBean("modelService")
query = "select {pk} from {trigger}";
flexibleSearchService.search(query).result.each
{
modelService.remove(it)
}
use HAC -> SQL Query console to delete using direct SQL command.
Enable Commit code
After running the update go to Monitoring tab and clear the cache
I just started to automate our test process. I made the following case:
Example:
* Settings *
Library Selenium2Library
Library Remote http://localhost:0000/ WITH NAME JDBCDB2 #DB2
Library ExcelLibrary
* Test Cases *
Connect to database connection parm login password
Store Query Result To File select * from table where x=y :\Testresults.txt
what keyword to use (instead of Store query results to file) so i can read the query from a file, instead of writing the full select statement.
How to write the sql results to an excel. Results seperated in columns/records
I have dataImportScheduler configured which posts and HTTP request to import the increments or changes into the index. what I want to be able to do is each time when delta import runs it should run a delete query as well which has some criteria e.g. documenttype:deleted to delete the unwanted data in the index.
the delta import query i am using is
http://address:8080/solr-multicore/dataimport?command=delta-import&clean=false&commit=true
You can use deletedPkQuery to clean up the records which have been deleted.
deletedPkQuery : Only used in delta-import
Example -
<entity name="album" query="SELECT * from albums" deletedPkQuery="SELECT deleted_id as id FROM deletes WHERE deleted_at > '${dataimporter.last_index_time}'">
This would help you to delete the records as well without the timestamp.
Yes, It is possible.
If you want to do delete only, you can remove both deltaQuery and deltaImportQuery and use the only deletedPkQuery as:
SELECT id FROM db WHERE deletion = 1 AND solrsync_date > '${dataimporter.db.last_index_time}'"
Note:- Condition for delete can be anything.
And then run :
http://host:8983/solr/core/dataimport?command=delta-import