How to search through multiple fields with elasticsearch? I've tried many queries but none of them worked out. I want the search to be case insensitive and one field is more important than the other. My query looks like this:
const eQuery = {
query: {
query_string: {
query: `*SOME_CONTENT_HERE*`,
fields: ['title^3', 'description'],
default_operator: 'OR',
},
},
}
esClient.search(
{
index: 'movies',
body: eQuery,
},
function(error, response) {
},
)
Mapping looks like this:
{
mappings: {
my_index_type: {
dynamic_templates: [{ string: { mapping: { type: 'keyword' }, match_mapping_type: 'string' } }],
properties: {
created_at: { type: 'long' },
description: { type: 'keyword' },
title: { type: 'keyword' },
url: { type: 'keyword' },
},
},
_default_: {
dynamic_templates: [{ string: { mapping: { type: 'keyword' }, match_mapping_type: 'string' } }],
},
},
}
The problem is the type: keyword in your mapping for fields description and title. Keyword type fields are not analyzed i.e they store the indexed data exactly like it was sent to elastic. It comes into use when you want to match things like unique IDs etc. Read: https://www.elastic.co/guide/en/elasticsearch/reference/current/keyword.html
You should read about analyzers for elasticsearch. You can create your custom analyzers very easily which can change the data you send them in different ways, like lowercasing everything before they index or search.
Luckily, there are pre-configured analyzers for basic operations such as lowercasing. If you change the type of your description and title fields to type: text, your query would work.
Read: https://www.elastic.co/guide/en/elasticsearch/reference/current/text.html
Also, i see you have dynamic templates configured for your index. So, if you do not specify the mappings for your index explicitly, all your string fields (like description and title) will be treated as type: keyword.
If you build your index like this:
PUT index_name
{
"mappings": {
index_type: {
"properties": {
"description": {"type": "text"},
"title": {"type": "text"}, ...
}
}
}
}
your problem should be solved. This is because type: text fields are analyzed by the standard analyzer by default which lowercases the input, among other things. Read: https://www.elastic.co/guide/en/elasticsearch/reference/current/analysis-standard-analyzer.html
Related
I am trying to search by different fields such as title and description. When i type keywords, elasticseach must found something if description or title includes that i typed keywords. This is my goal. How can i reach my goal?
You can see the sample code that i used for one field.
query: {
regexp: {
title: `.*${q}.*`,
},
},
I also tried below one but it gave syntax error.
query: {
regexp: {
title: `.*${q}.*`,
},
regexp: {
description: `.*${q}.*`,
},
},
To do so, you need to use a bool query.
GET /<you index>/_search
{
"query": {
"bool": {
"should": [
{
"regexp": {
"title": ".*${q}.*"
}
},
{
"regexp": {
"description": ".*${q}.*"
}
}
]
}
}
}
You can find the documentation => [doc]
Its my first time trying prisma and am stuck. So I have "products" and "filters" model.
I want the following query to work. The idea is, I want to fetch the products with dynamic matching query params (name and value). The product query parameters come dynamically from the frontend.
const products = await prisma.product.findMany({
where: {
categoryName,
subCategoryName,
filters: {
some: {
AND: [
{
name: "RAM",
value: "32GB",
},
{
name: "Storage",
value: "1TB",
},
],
},
},
},
include: {
images: true,
},
});
If there's only one parameter, like
{
name:"RAM",
value:"32GB"
}
the query returns appropriate products, but if there are more that one query params (like in the original code above), it returns empty array.
my product schema looks like this, simplified,
name String
filters Filter[]
my filter schema looks like this, simplified
name String
value String?
product Product? #relation(fields: [productId], references:[id])
productId Int?
Thank you very much
I've found the solution here
https://github.com/prisma/prisma/discussions/8216#discussioncomment-992302
It should be like this instead apparently.
await prisma.product.findMany({
where: {
AND: [
{ price: 21.99 },
{ filters: { some: { name: 'ram', value: '8GB' } } },
{ filters: { some: { name: 'storage', value: '256GB' } } },
],
},
})
I'm building a search engine to find places from a huge Database stored in elasticsearch and want my results to be based on the nearest places from the user's position so I used the completion suggester method with its context option, but I'm facing a problem in the implementation.
I followed the documentation step by step but it always returns an empty array.
Here's how I'm creating my index:
`location: {
type: "geo_point"
},
context2: {
type: "completion",
analyzer: "my_analyzer",
contexts: {
name: "location",
type: "geo",
precision: 4
}
},
and how I'm performing my search
contextSuggester: {
prefix: req.body['q'],
completion: {
field: "context2",
size : 7,
skip_duplicates:true,
contexts: {
location: {
lat: 43.662,
lon: -79.380
}
},
fuzzy: {
fuzziness: "auto"
}
}
}
So basically i have model with a bunch of string fields like so:
const Schema: Schema = new Schema(
{
title: {
type: String,
trim: true
},
description: {
type: String,
trim: true
},
...
}
);
Schema.index({ '$**': 'text' });
export default mongoose.model('Watch', Schema);
where I index all of them.
Now when I search being that this schema is used as a ref for another model I do a search like this where user is an instance of the other model
const { search, limit = 5 } = req.query;
const query = search && { match: { $text: { $search: new RegExp(search, 'i') } } };
const { schemaRes } = await user
.populate({
path: 'schema',
...query,
options: {
limit
}
})
.execPopulate();
and the searching itself seems to work ok, the problem is when search fields starts to be more specific it seems to me the it does not regard it well.
Example
db
{ title: 'Rolex', name: 'Submariner', description: 'Nice' }
{ title: 'Rolex', name: 'Air-King', description: 'Nice' }
When the search param is Rolex I get both items which is ok but when the search param becomes Rolex Air-King i keep on getting both items which to me is not ok because I would rather get only one.
Is there something I could do to achieve this?
Returning both items is correct, since both items match your search params, but with different similarity score.
You can output the similarity score to help sorting the result.
user.aggregate([
{ $match: { $text: { $search: "Rolex Air-King" } } },
{ $set: { score: { $meta: "textScore" } } }
])
// new RegExp("Rolex Air-King", 'i') is not necessary and even invalid,
// as $search accepts string and is already case-insensitive by default
The query will return
[{
"_id": "...",
"title": "Rolex",
"name": "Air-King",
"description": "Nice",
"score": 2.6
},
{
"_id": "....",
"title": "Rolex",
"name": "Submariner",
"description": "Nice",
"score": 1.1
}]
Since the second result item matches your search query (even partially), MongoDB returns it.
You could use the score to help sort the items. But determining the right threshold to filter the result is complex, as the score depends on the word count as well.
On a side note: You can assign different weights to the fields if they are not equally important
https://docs.mongodb.com/manual/tutorial/control-results-of-text-search/
I have tons of articles in various stores. Some of these articles are own brand articles and should be ranked higher than other articles in my elasticsearch search results (both ownbrand and non ownbrand should be shown however.)
I already tried different approached with field_value_factor but that doesn't seem to go well with a boolean field.
I also tried the approached solution in Boosting an elasticsearch result based on a boolean field value but that didn't worked well for me. The results with the ownBrand approach were still way lower ranked then a lot of non ownBrand articles.
Index:
schema: {
articleId: { type: 'text' },
brandId: { type: 'text' },
brandName: { type: 'text' },
countryId: { type: 'text' },
description: { type: 'text' },
isOwnBrand: { type: 'boolean' },
stores: { type: 'keyword' },
},
};
Query:
query: {
function_score: {
query: {
bool: {
must: {
multi_match: {
query: searchterm,
fields: ['name^5', 'name.ngram'],
fuzziness: 'auto',
prefix_length: 2,
},
},
filter: [{ term: { stores: storeId } }],
},
},
},
},
};
The result should prioritize fields with isOwnBrand = true at the top while still showing relevant articles with isOwnBrand = false below.
I am a bit lost on how to handle this.
You can use Field Value factor. Below should work fine even on a boolean field as well. try it
{
"query": {
"function_score": {
"query" {...}, #your normal query as in question
#add below after query
"field_value_factor": {
"field": "isOwnBrand",
"factor": 1.2,
"modifier": "sqrt",
"missing": 1
}
}
}
}
One caveat i can think of but haven't tested - since false is 0, above script will score down all documents with false to 0 score, which messes up scoring. You could either make the isOwnBrand a number field and set priority starting 1
OR you could also use script_score