I am newbie to Lucene. Just started. I have a few basic questions:
How to view all the indexes that are created using Stratio Lucene ?
How to delete indexes created using Stratio Lucene ?
What is the difference between
fields: {
fld_1: {type: "string"},
fld_2: {type: "text"}
}
type: "string" and type: "text"
The reason I ask for the difference is because I ran in to an error when trying to create my very first lucene index. My column in Cassandra is something like this: 'fld_1 text', but when I tried to create and index on fld_1 like above it threw an exception
ConfigurationException: 'schema' is invalid : Unparseable JSON schema: Unexpected character ('}' (code 125)): was expecting either valid name character (for unquoted name) or double-quote (for quoted) to start field name
at [Source: {
fields: {
The Lucene index script:
CREATE CUSTOM INDEX lucene_index ON testTable ()
USING 'com.stratio.cassandra.lucene.Index'
WITH OPTIONS = {
'refresh_seconds': '1',
'schema': '{
fields: {
fld_1: {type: "string"},
fld_2: {type: "string"},
id: {type: "integer"},
test_timestamp: {type: "date", pattern: "yyyy/MM/dd HH:mm:ss"}
}
}'
};
Thanks!
First : You can't view only the Stratio Lucene Index, below query will show you all the index
SELECT * FROM system."IndexInfo";
Second : You can delete index with DROP INDEX index_name command. i.e
DROP INDEX test;
Third : In Stratio Lucene Index, string is a not-analyzed text value and text is a language-aware text value analyzed according to the specified analyzer.
Which means that if you specify a field as string it will directly index and queried. But if you use text then it will first analyzed by your specified analyzer, default is default_analyzer(org.apache.lucene.analysis.standard.StandardAnalyzer) then index and queried.
Edited :
You have to first create a text field in cassandra then specified it when creating index.
Example :
ALTER TABLE testtable ADD lucene text;
CREATE CUSTOM INDEX lucene_index ON testTable (lucene) USING 'com.stratio.cassandra.lucene.Index'
WITH OPTIONS = {
'refresh_seconds': '1',
'schema': '{
fields: {
fld_1: {type: "string"},
fld_2: {type: "string"},
id: {type: "integer"},
test_timestamp: {type: "date", pattern: "yyyy/MM/dd HH:mm:ss"}
}
}'
};
For more : https://github.com/Stratio/cassandra-lucene-index/blob/branch-3.0.13/doc/documentation.rst#text-mapper
Related
In order to make validation over a api, i'm send companyId as UUID like: 71158c1a-56fd-4dd4-8e7f-fb95711a41de
To have this validation I used jsonschema with the following patterns (test all 3 of them):
/^[0-9a-fA-F]{8}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{12}$
/^[0-9a-fA-F]{8}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{12}$/gi
[\w]{8}-[\w]{4}-[\w]{4}-[\w]{4}-[\w]{12}
jsonschema:
companyId: {
type: "string",
default: "",
title: "The companyId Schema",
pattern: "/^[0-9a-fA-F]{8}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{4}\b-[0-9a-fA-F]{12}$",
examples: ["71158c1a-56fd-4dd4-8e7f-fb95711a41de"],
},
For some reason the validation returned me errors:
path: Ä 'companyId' Å,
property: 'instance.companyId',
message: 'does not match pattern "/ÜÄ0-9a-fA-FÅä8åÖÖb-Ä0-9a-fA-FÅä4åÖÖb-Ä0-9a-fA-FÅä4åÖÖb-Ä0-9a-fA-FÅä4åÖÖb-Ä0-9a-fA-FÅä12å$"',
schema: ä
type: 'string',
default: '',
title: 'The companyId Schema',
pattern: '/ÜÄ0-9a-fA-FÅä8åÖb-Ä0-9a-fA-FÅä4åÖb-Ä0-9a-fA-FÅä4åÖb-Ä0-9a-fA-FÅä4åÖb-Ä0-9a-fA-FÅä12å$',
examples: ÄArrayÅ
å,
instance: 'aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee',
name: 'pattern',
argument: '/ÜÄ0-9a-fA-FÅä8åÖb-Ä0-9a-fA-FÅä4åÖb-Ä0-9a-fA-FÅä4åÖb-Ä0-9a-fA-FÅä4åÖb-Ä0-9a-fA-FÅä12å$',
stack: 'instance.companyId does not match pattern "/ÜÄ0-9a-fA-FÅä8åÖÖb-Ä0-9a-fA-FÅä4åÖÖb-Ä0-9a-fA-FÅä4åÖÖb-Ä0-9a-fA-FÅä4åÖÖb-Ä0-9a-fA-FÅä12å$"'
å,
Also im getting these cyrillic letters, maybe this is the reason?
The latest version of JSON Schema supports the uuid format (you may need to explicitly turn on format validation in the implementation, as by default it is supposed to be annotation-only):
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "string",
"format": "uuid",
}
Also, you have a stray / in your pattern before the ^ anchor, so your pattern will never match.
I have events collection inserted with a below record in ARANGODB. ( I am new to Arango )
INSERT {
"source": "ABC",
"target": "ZYX",
"tranno": "ABCDEF",
"type": "REST",
"attributes" : { "myID" : "12345"}
} INTO events
But trying to create full text index on attributes, resulting in error as given below. It would be great if you could help with this.
events.createIndex ({ type: "fulltext", fields: [ "attributes" ], minLength: 3 })
Query: AQL: syntax error, unexpected identifier near 'events.createIndex ({ type: "ful...' at position 1:1 (while parsing)
Unlike SQL, AQL is a language used for data selection and data manipulation.
It is not a data definition language, so you can't use AQL to create indexes.
In order to create an index, please use ArangoDB's web interface (Collections => target collection => Indexes => "+" icon) or the ArangoShell. The ArangoShell is a separate executable that is shipped with all ArangoDB packages.
In the ArangoShell you can use the command
db.events.createIndex ({ type: "fulltext", fields: [ "attributes" ], minLength: 3 })
to create the index.
I can able to search the case by company name
var mySearch = search.create({
type: search.Type.SUPPORT_CASE,
columns: [{
name: 'title'
}, {
name: 'company'
}],
filters: [{
name: 'company',
operator: 'is',
values: 'Test'
}]
});
return mySearch.run({
ld: mySearch.id
}).getRange({
start: 0,
end: 1000
});
But I am not able to search case by company id.
companyId is 115
Below are not working
i)
filters: [{
name: 'company',
operator: 'is',
values: 115
}]
ii)
filters: [{
name: 'companyid',
operator: 'is',
values: 115
}]
According to the Case schema company is a Text filter, meaning you would have to provide it with the precise Name of the company, not the internal ID.
Instead you may want to use the customer.internalid joined filter to provide the internal ID. Also, Internal ID fields are nearly always Select fields, meaning they do not accept the is operator, but instead require the anyof or noneof operator.
You can find the valid operators by field type on the Help page titled Search Operators
First, you can try this :
var supportcaseSearchObj = search.create({
type: "supportcase",
filters:
[
["company.internalid","anyof","100"]
],
columns:
[
search.createColumn({
name: "casenumber",
sort: search.Sort.ASC
}),
"title",
"company",
"contact",
"stage",
"status",
"profile",
"startdate",
"createddate",
"category",
"assigned",
"priority"
]
});
Second : how did I get this ? The answer is hint that will make your life easier :
Install the "NetSuite Saved Search Code Export" chrome plugin.
In Netsuite UI, create your saved search (it is always easier that doing it in code).
After saving the search, open it again for edition.
At the top right corner (near list, search menu in the netsuite page), you will see a link "Export as script" : click on it and you will get your code ;)
If you can not install the chrome plugin :
In Netsuite UI, create your saved search (it is always easier that doing it in code).
In your code, load your saved search
Add a log.debug to show the [loadedesearchVar].filters
You can then copy what you will see in the log to use it as your search filters.
Good luck!
When I query from collection in MongoDB and it has results:
[ { details:
[ { owner: '57f52829bcc705bb1c37d611',
nameprd: 'fsfsdaf',
price: 15000000,
descrice: 'sfsdfsdaf',
number: 4,
dateOff: '2016-06-12T17:00:00.000Z',
_csrf: 'CPlxeLpq-vYfTTWTgSpR6bsyapbDVgDCKzTc',
image: 'samsung-galaxy-note-7.png',
createdAt: '2016-10-06T16:43:11.109Z',
updatedAt: '2016-10-06T16:43:13.061Z',
id: '57f67f1f7ab99e5824652208' } ],
name: 'Máy tính bảng',
_csrf: 'Ze6OhtgL-2hZvG7TuP9NO4fjY90rA7x46bWA',
createdAt: '2016-10-05T16:19:53.331Z',
updatedAt: '2016-10-06T16:43:13.021Z',
id: '57f52829bcc705bb1c37d611' },
]
Now, how to get value which called details in this result.
Thanks.
You should add the query the following syntax: ,{'details':1}
For example:
If that is the original query:
db.person.find({'name':'joe'})
Than the following query returns only the details value of the query:
db.person.find({'name':'joe'},{'details':1})
The addition of the ,{'details':1} means that you want to get only the data for the details. It is uses as a filter to the extensive query.
When I try to find all members within 50km of Salt Lake City, Utah from the Mongo shell I get the error:
error: {
"$err" : "point not in interval of [ -180, 180 ] :: caused by :: { 0: 0.0, 1: 50000.0 }",
"code" : 16433
}
Here is the query I am running:
db.members.find(
{ 'geo.point' :
{ $near :
{
$geometry : {
type : "Point" ,
coordinates : [ 111.000 , 40.000 ]
},
$maxDistance : 50000
}
}
}
)
Member schema is like this:
var memberSchema = mongoose.Schema({
name: {
first: {type:String, default:''},
last: {type:String, default:''},
},
geo: {
latitude: {type:String, default:''},
longitude: {type:String, default:''},
country: {type:String, default:''},
state: {type:String, default:''},
place: {type:String, default:''},
zip: {type:String, default:''},
point: {type: [Number], index: '2d'}
}
});
Member object in DB looks like this:
{
"_id" : ObjectId("xxxxxxxxxxxxxxxxxxx"),
"name": {
"first": "Thom",
"last": "Allen"
},
"geo" : {
"point" : [ -111.8833, 40.7500 ],
"zip" : "84115",
"state" : "UT",
"country" : "US",
"longitude" : "-111.8833",
"latitude" : "40.7500"
}
}
Is it possible that my fields are not stored in the correct format? If I change 50000 to anything below 180 it will work, but that is not how it should function according to the docs here:
http://docs.mongodb.org/manual/reference/operator/query/near/
** Just a heads up, the proper mongo location array IS in fact [longitude, latitude].
A few things. First, I think your query is off - you are querying for coordinates : [ 111.000 , 40.000 ] and it should be coordinates : [ -111.000 , 40.000 ]
Second, the example data point your provide [ -111.8833, 40.7500 ] is more than 50 km from your corrected query point, it's actually about 122 km (test it here: http://andrew.hedges.name/experiments/haversine/ )
So, correcting for those two issues if I store the data in mongodb as you have stored it I can do the following:
1) create the correct index:
db.members.ensureIndex({ "geo.point": "2dsphere" })
2) run this query:
db.members.find({ 'geo.point':
{$geoWithin:
{$centerSphere: [[ -111.000 , 40.000 ], 113/6371]}
}
} )
Note that I've divided 113 km/ 6371 which gives you radians which is what is required for this specific query.
Try it yourself. In general you will be better off if you can store things in the future using GeoJSON but with your existing schema and the above index and query I'm able to get the correct results.
What you have in your data is the format for legacy co-ordinate pairs but you are trying to query using the GeoJSON syntax.
The only valid index form for legacy co-ordinate pairs is a "2d" index, so if you have created a "2d sphere" index that will not work. So you need to remove any "2d sphere" index and create a "2d" index as follows:
db.members.ensureIndex({ "geo.point": "2d" })
If you actually intend to use the GeoJSON form and "2dsphere" index type, then you need the data to support it, for example:
{
"loc" : {
"type" : "Point",
"coordinates" : [ 3, 6 ]
}
}
So it needs that underlying structure of "type" and "coordinates" in order to use this index type and query form.