Modeshape full-text-search works only on binary files - text

I am trying to perform a full-text-search on my Modeshape 5.3.0.Final repository. The query is as simple as:
Query query = queryManager.createQuery("SELECT * FROM [nt:resource] as data WHERE ISDESCENDANTNODE('/somenode') AND CONTAINS(data.*,'*" + text + "*')
Looks like it works well for binary stored files (i.e. pdf,doc,docx, etc...) but it does not match txt files, or any text format file.
This is my repository configuration
{
"name": "Persisted-Repository",
"textExtraction": {
"extractors": {
"tikaExtractor": {
"name": "General content-based extractor",
"classname": "tika"
}
}
},
"workspaces": {
"predefined": [
"otherWorkspace"
],
"default": "default",
"allowCreation": true
},
"security": {
"anonymous": {
"roles": [
"readonly",
"readwrite",
"admin"
],
"useOnFailedLogin": false
}
},
"storage": {
"persistence": {
"type": "file",
"path": "/var/content/storage"
},
"binaryStorage": {
"type": "file",
"directory": "/var/content/binaries",
"minimumBinarySizeInBytes": 999,
"mimeTypeDetection": "content"
}
},
"indexProviders": {
"lucene": {
"classname": "lucene",
"directory": "/var/content/indexes"
}
},
"indexes": {
"textFromFiles": {
"kind": "text",
"provider": "lucene",
"nodeType": "nt:resource",
"columns": "jcr:data(BINARY)"
}
}
}
Currently I'm performing a hack to get around this issue by executing another search for configured text file extensions and manually using Tika (maybe since it's text already Tika is not required here...) to extract the text and search for occurrences.
Does anybody know if this is expected behavior or maybe I am doing something wrong?
Cheers!

Related

Swagger Flask Python3, JSON $ref not including external file

References to internal sections in swagger.json work correctly, but attempting to reference a JSON file in the same directory seems to be ignored. I have some large example data files so would like to reference them instead of including them in the swagger.json file.
How do I get swagger to dereference the $ref tags for external files in the ./static/ directory (this is the same location as the swagger.json file)? Is there a swagger log file that shows what/how it loads files?
Things that work:
http://localhost:5002/swagger
http://localhost:5002/static/swagger.json
http://localhost:5002/static/components.json
So this proves the web server can find the static/components.json file.
Part of swagger.json:
"components": {
"schemas": {
"id": {
"properties": {
"id": {
"type": "number",
"format": "int64"
}
}
},
"componentsPost": {
"type": "object",
"properties": {
"id": {
"$ref": "#/components/schemas/id" <-- THIS WORKS
},
"name": {
"type": "string"
},
"description": {
"type": "string"
},
"component_type_id": {
"$ref": "#/components/schemas/id" <-- THIS WORKS
},
"system_type_id": {
"$ref": "#/components/schemas/id" <-- THIS WORKS
},
"parent_id": {
"$ref": "#/components/schemas/id"
}
},
"example": {
"$ref": "components.json" <-- THIS DOES *NOT* WORK
}
},
Note: components.json is different than the #components/schema/id
reference, which is in the same file as shown above.
Beginning of components.json data file:
{
"components": [
{
"id": 99999,
"name": "root",
"description": "Root Component",
"component_type_id": 1,
"system_type_id": 1
},
{
"id": 1,
"name": "lan1",
"description": "LAN",
"component_type_id": 1,
"system_type_id": 4,
"parent_id": 99999
},
The swagger.json file gets loaded but components.json does not:
[06/Jan/2021 12:32:53] "GET /static/swagger.json HTTP/1.1" 200 -
The swagger web site shows the $ref, not the expected components.json:
Alternate code I have tried:
"example": {
"components": {
"$ref": "components.json"
}
}
"example": {
"$ref": "/static/components.json"
}
"example": {
"$ref": "./static/components.json"
}
"example": {
"$ref": "static/components.json"
}

Commercetools - Use query predicates to filter on a collection attribute

I'd like to get all the Category items that have between their ancestors an ancestor with a certain "id".
Here is the JSON of one of the categories returned by GETting from /categories:
{
"id": "4627f3b0-fe52-4cc6-b03e-3fd72e701342",
"version": 1,
"lastMessageSequenceNumber": 1,
"createdAt": "2019-02-18T13:48:51.677Z",
"lastModifiedAt": "2019-02-18T13:48:51.677Z",
"lastModifiedBy": {
"clientId": "_anonymous"
},
"createdBy": {
"clientId": "_anonymous"
},
"key": "snowboard-gloves",
"name": {
"en": "Snowboard Gloves"
},
"slug": {
"en": "snowboard-gloves"
},
"description": {
"en": "Gloves specifically designed for snowboarding"
},
"ancestors": [
{
"typeId": "category",
"id": "b27086d2-33f2-43c3-aad1-4c01b2b9a886"
}
],
"parent": {
"typeId": "category",
"id": "b27086d2-33f2-43c3-aad1-4c01b2b9a886"
},
"orderHint": "0.000016",
"metaTitle": {
"en": "Snowboard Gloves"
},
"metaDescription": {
"en": "Gloves specifically designed for snowboarding"
},
"assets": []
}
I'd like to call the /categories API with a where clause on ancestors[x].id = "b27086d2-33f2-43c3-aad1-4c01b2b9a886" but from the documentation I don't understand how I should write the query predicate.
Can anyone help me?
The query predicate follows the structure of the json response. Nested fields are accessed with () brackets.
Try this out
ancestors(id = "idb27086d2-33f2-43c3-aad1-4c01b2b9a886")

Acumatica API insert data "Bad Request"

I'm trying to insert a stock item into Acumatica using the API, but I'm getting a 400 error - Bad Request. I'm using HttpClient to login, retrieve a stock item, and send the insert request. All is working except the insert request. I have tried the following url (including expand parameter):
http://localhost/AcumaticaERP/entity/Default/6.00.001/StockItem?$expand=Attributes,CrossReferences,UOMConversions,VendorDetails,WarehouseDetails
... and also the following url (without expand parameter)
http://localhost/AcumaticaERP/entity/Default/6.00.001/StockItem
I'm calling via HttpClient PutAsync, passing in the URLs mentioned above, and the data is JSON from a stock item retrieved with the API, and which doesn't exist in the current db.
client.PutAsync(insertUrl, new StringContent(data, Encoding.UTF8, "application/json")).Result;
Any ideas what I'm missing?
NEW DETAILS:
After further debugging and testing with Postman, the error from the PUT seems to be "Error reading JObject from JsonReader. Current JsonReader item is not an object: StartArray. Path '', line 1, position 1." The JSON was taken directly from a GET though, so I'm not sure what the problem could be. Below is the JSON returning an error from the PUT.
[
{
"id": "cc27ce56-6302-4f1b-97aa-49cca3ed32ea",
"rowNumber": 1,
"note": "",
"Attributes": [],
"BaseUOM": {
"value": "EA"
},
"CrossReferences": [],
"DefaultIssueLocationID": {
"value": "R1S1"
},
"DefaultReceiptLocationID": {
"value": "RECEIVING"
},
"DefaultWarehouseID": {
"value": "WHOLESALE"
},
"Description": {
"value": "tonyitem2"
},
"ImageUrl": {},
"InventoryID": {
"value": "TONYITEM2"
},
"IsAKit": {
"value": false
},
"ItemClass": {
"value": "CONSUMER 300TOYS"
},
"ItemStatus": {
"value": "Active"
},
"ItemType": {
"value": "Finished Good"
},
"LastModified": {
"value": "2018-08-03T12:09:19.907-04:00"
},
"LotSerialClass": {
"value": "NOTTRACKED"
},
"PurchaseUOM": {
"value": "EA"
},
"SalesUOM": {
"value": "EA"
},
"UOMConversions": [],
"VendorDetails": [],
"Volume": {
"value": 0
},
"WarehouseDetails": [
{
"id": "3ca5ea4c-c651-498e-8e6c-49119481982c",
"rowNumber": 1,
"note": "",
"DefaultIssueLocationID": {
"value": "R1S1"
},
"DefaultReceiptLocationID": {
"value": "RECEIVING"
},
"IsDefault": {
"value": true
},
"QtyOnHand": {
"value": 0
},
"WarehouseID": {
"value": "WHOLESALE"
},
"custom": {},
"files": []
}
],
"Weight": {
"value": 0
},
"custom": {},
"files": []
}
]
I also tried removing the brackets surrounding the JSON, and then the error is:
"No entity satisfies the condition.". Could the issue be that the ids have GUIDs, but I'm trying to do an insert? How do you indicate that the PUT is supposed to be inserting?
I finally got the PUT to insert. What finally worked was removing the wrapping braces "[" and "]" around the entire JSON; plus removing all "id", "rowNumber", "custom", and "files" fields, and all empty collections (e.g. Attributes, CrossReferences) in my JSON. I'm not sure which of these being removed resolved it and allowed me to insert, but it finally worked.
It's real unfortunate that the JSON you retrieve via GET can't be inserted via PUT without stripping all of this out first though. What a pain.

Speeding up Cloudant query for type text index

We have a table with this type of structure:
{_id:15_0, createdAt: 1/1/1, task_id:[16_0, 17_0, 18_0], table:”details”, a:b, c: d, more}
We created indexes using
{
"index": {},
"name": "paginationQueryIndex",
"type": "text"
}
It auto created
{
"ddoc": "_design/28e8db44a5a0862xxx",
"name": "paginationQueryIndex",
"type": "text",
"def": {
"default_analyzer": "keyword",
"default_field": {
},
"selector": {
},
"fields": [
],
"index_array_lengths": true
}
}
We are using the following query
{
"selector": {
"createdAt": { "$gt": 0 },
"task_id": { "$in": [ "18_0" ] },
"table": "details"
},
"sort": [ { "createdAt": "desc" } ],
"limit”: 20
}
It takes 700-800 ms for first time, after that it decreases to 500-600 ms
Why does it take longer the first time?
Any way to speed up the query?
Any way to add indexes to specific fields if type is “text”? (instead of indexing all the fields in these records)
You could try creating the index more explicitly, defining the type of each field you wish to index e.g.:
{
"index": {
"fields": [
{
"name": "createdAt",
"type": "string"
},
{
"name": "task_id",
"type": "string"
},
{
"name": "table",
"type": "string"
}
]
},
"name": "myindex",
"type": "text"
}
Then your query becomes:
{
"selector": {
"createdAt": { "$gt": "1970/01/01" },
"task_id": { "$in": [ "18_0" ] },
"table": "details"
},
"sort": [ { "createdAt": "desc" } ],
"limit": 20
}
Notice that I used strings where the data type is a string.
If you're interested in performance, try removing clauses from your query one at-a-time to see if one is causing the performance problem. You can also look at the explanation of your query to see if it using your index correctly.
Documentation on creating an explicit text query index is here

Error creating a customContent on a confluence addon

Today I was trying to create a confluence addon for my company and I've try following atlassian documents.
My problem comes trying to run the express app when adding a new customContent to the atlassian-connect.json, after running npm start I get the following error.
Failed to register with host https‍://admin:xxx#xxx.atlassian.net/wiki (200)
{"type":"INSTALL","pingAfter":300,"status":{"done":true,"statusCode":200,"con
tentType":"application/vnd.atl.plugins.task.install.err+json","subCode":"upm.
pluginInstall.error.descriptor.not.from.marketplace","source":"https‍://1a0adc
8f.ngrok.io/atlassian-connect.json","name":"https‍://1a0adc8f.ngrok.io/atlassi
an-connect.json"},"links":{"self":"/wiki/rest/plugins/1.0/pending/b88594d3-c3
c2-4760-b687-c8d860c0a377","alternate":"/wiki/rest/plugins/1.0/tasks/b88594d3
-c3c2-4760-b687-c8d860c0a377"},"timestamp":1502272147602,"userKey":"xxx","id":"xxx"}
Add-on not registered; no compatible hosts detected
This is my atlassian-connect.json file:
{
"key": "my-add-on",
"name": "Ping Pong",
"description": "My very first add-on",
"vendor": {
"name": "Angry Nerds",
"url": "https://www.atlassian.com/angrynerds"
},
"baseUrl": "{{localBaseUrl}}",
"links": {
"self": "{{localBaseUrl}}/atlassian-connect.json",
"homepage": "{{localBaseUrl}}/atlassian-connect.json"
},
"authentication": {
"type": "jwt"
},
"lifecycle": {
"installed": "/installed"
},
"scopes": [
"READ"
],
"modules": {
"generalPages": [
{
"key": "hello-world-page-jira",
"location": "system.top.navigation.bar",
"name": {
"value": "Hello World"
},
"url": "/hello-world",
"conditions": [{
"condition": "user_is_logged_in"
}]
},
{
"key": "customersViewer",
"location": "system.header/left",
"name": {
"value": "Hello World"
},
"url": "/hello-world",
"conditions": [{
"condition": "user_is_logged_in"
}]
}
],
"customContent": [
{
"key": "customer",
"name": {
"value": "Customers"
},
"uiSupport": {
"contentViewComponent": {
"moduleKey": "customersViewer"
},
"listViewComponent": {
"moduleKey": "customerList"
},
"icons": {
"item": {
"url": "/images/customers.png"
}
}
},
"apiSupport": {
"supportedContainerTypes": ["space"]
}
}
]
}
}
Does anybody has an idea on whats going on?
The contentViewComponent can't find the generalPage it is referencing in moduleKey.
From the docs:
In the snippet above, the moduleKey “customersViewer” maps to a
generalPage module we have defined in our add-on. This generalPage is
passed the context parameters we specify, and visualizes our content
accordingly.
If you change the generalPage with the key hello-world-page-confluence to customersVieweryou be able to install and get up and running.

Resources