So I'm trying to implement postgresql full text search through Sequelize 3 but there're a few problems including that, I couldn't add text search functions in current where clause:
where: {
published: true,
and: where(
fn("tsmatch",
col("tokens"),
fn("plainto_tsquery", query),
),
true,
)
},
I know we can use raw sql statement but the problem is that existing implementation is based on Sequelize operators and that's what Sequelize recommends for security.
Found a good resource for adding and reusing a function for it here but am stuck in combining it with other query params.
Ok, I've ended up using where clause again:
where: {
published: true,
where: where(
fn("tsmatch",
col("textTokens"),
fn("plainto_tsquery", text),
),
true,
)
},
Hope this would help others facing similar issue.
Sequelize version 6.5.0+ supports the TSVECTOR datatype and Op.match:
Model.findAll({
where: {
textTokens: { [Op.match]: sequelize.fn('to_tsquery', query) }
}
})
Related
the problem I am facing is as follows:
Search value: 'cooking'
JSON object::
data: {
skills: {
items: [ { name: 'cooking' }, ... ]
}
}
Expected result: Should find all the "skill items" that contain 'cooking' inside their name, using TypeORM and Nest.js.
The current code does not support search on the backend, and I should implement this. I want to use TypeORM features, rather than handling it with JavaScript.
Current code: (returns data based on the userId)
const allItems = this.dataRepository.find({ where: [{ user: { id: userId } }] })
I investigated the PostgreSQL documentation regarding the PostgreSQL functions and even though I understand how to create a raw SQL query, I am struggling to convert this to the TypeORM equivalent.
Note: I researched many StackOverflow issues before creating this question, but do inform me If I missed the right one. I will be glad to investigate.
Can you help me figure out the way to query this with TypeORM?
UPDATE
Let's consider the simple raw query:
SELECT *
FROM table1 t
WHERE t.data->'skills' #> '{"items":[{ "name": "cooking"}]}';
This query will provide the result for any item within the items array that will match exact name - in this case, "cooking".
That's totally fine, and it can be executed as a raw request but it is certainly not easy to maintain in the future, nor to use pattern matching and wildcards (I couldn't find a solution to do that, If you know how to do it please share!). But, this solution is good enough when you have to work on the exact matches. I'll keep this question updated with the new findings.
use Like in Where clause:
servicePoint = await this.servicePointAddressRepository.find({
where: [{ ...isActive, name: Like("%"+key+"%"), serviceExecutive:{id: userId} },
{ ...isActive, servicePointId: Like("%"+key+"%")},
{ ...isActive, branchCode: Like("%"+key+"%")},
],
skip: (page - 1) * limit,
take: limit,
order: { updatedAt: "DESC" },
relations:["serviceExecutive","address"]
});
This may help you! I'm matching with key here.
I've been trying to get full search text to work for a while now without any success. The current documentation has this example:
[Op.match]: Sequelize.fn('to_tsquery', 'fat & rat') // match text search for strings 'fat' and 'rat' (PG only)
So I've built the following query:
Title.findAll({
where: {
keywords: {
[Op.match]: Sequelize.fn('to_tsquery', 'test')
}
}
})
And keywords is defined as a TSVECTOR field.
keywords: {
type: DataTypes.TSVECTOR,
},
It seems like it's generating the query properly, but I'm not getting the expected results. This is the query that it's being generated by Sequelize:
Executing (default): SELECT "id" FROM "Tests" AS "Test" WHERE "Test"."keywords" ## to_tsquery('test');
And I know that there are multiple records in the database that have 'test' in their vector, such as the following one:
{
"id": 3,
"keywords": "'keyword' 'this' 'test' 'is' 'a'",
}
so I'm unsure as to what's going on. What would be the proper way to search for matches based on a TSVECTOR field?
It's funny, but these days I am also working on the same thing and getting the same problem.
I think part of the solution is here (How to implement PostgresQL tsvector for full-text search using Sequelize?), but I haven't been able to get it to work yet.
If you find examples, I'm interested. Otherwise as soon as I find the solution that works 100% I will update this answer.
What I also notice is when I add data (seeds) from sequelize, it doesn't add the lexemes number after the data of the field in question. Do you have the same behavior ?
last thing, did you create the index ?
CREATE INDEX tsv_idx ON data USING gin(column);
To create a unique index in sequelize migrations we can do something like below,
await queryInterface.addIndex(SCHOOL_TABLE, {
fields: ['name', 'school_id'],
unique: true,
name: SCHOOL_NAME_ID_UNIQUE_INDEX,
where: {
is_deleted: false
},
transaction,
})
The problem is It allows duplicates due to case senstivity.
In the doc here, It is mentioned that fields should be an array of attributes.
How can I apply lower() to name field so that It can become case insensitive?
I am using a workaround for now, using raw query. I don't think addIndex() supports using functions on fields.
await queryInterface.sequelize.query(`CREATE UNIQUE INDEX IF NOT EXISTS ${SCHOOL_NAME_ID_UNIQUE_INDEX}
ON ${SCHEMA}.schools USING btree (lower(name), school_id) where is_deleted = false;
`, { transaction});
We need to have sequelize return dates in a particular format, not the default one. As far as I can tell, there is no way to set that up in options, or any other way. Short of manually updating the dates every time after they are retrieved, anyone been able to solve this easily? Or am I missing something?
You can, use the Sequelize fn method. From the API Reference, the fn function will help create an object representing a SQL function in your query.
For example:
model.findAll({
attributes: [
'id',
[sequelize.fn('date_format', sequelize.col('date_col'), '%Y-%m-%d'), 'date_col_formed']
]})
.then(function(result) {
console.log(result);
});
Will return data values:
[
{"id": 1, "date_col_formed": "2014-01-01"},
{"id": 2, "date_col_formed": "2014-01-02"}
// and so on...
]
21/06/2019
A little bit late but providing an update.
Sequelize is a powerful ORM (I am not saying is the best solution out there) but has a very bad documentation.
Anyway if you want to have this configured in your models one way apart from having to repeat this across your queries as other responses state you could be doing:
const Test = sequelize.define('test', {
// attributes
name: {
type: DataType.STRING,
allowNull: false
},
createdAt: {
type: DataType.DATE,
//note here this is the guy that you are looking for
get() {
return moment(this.getDataValue('createdAt')).format('DD/MM/YYYY h:mm:ss');
}
},
updatedAt: {
type: DataType.DATE,
get() {
return moment(this.getDataValue('updatedAt')).format('DD/MM/YYYY h:mm:ss');
}
}
If you are using dates to provide info as: last updated, first login, last login. This is the way to go.
The function get returns the new formatted element so it should not be restricted to dates either! Just bear in mind that this will slow your query so use cautiously. ;)
more info (i know i know but docs is the name of the game): https://sequelize.org/docs/v6/core-concepts/getters-setters-virtuals/
you can define custom instance methods getDate/setDate which would translate date between sequelize internal representation and desired format like so https://sequelize.org/master/manual/model-basics.html#taking-advantage-of-models-being-classes
If you are wondering to cast it as VARCHAR :
attributes:{include:[[sequelize.cast(sequelize.col('dob'), 'VARCHAR') , 'dob']],exclude:['dob']}
In case of Model that we create using Sequelize CLI
try something like this
var sequelize= require('../models');
model.findAll({
attributes: [
'id',
'title'
[sequelize.Sequelize.fn('date_format', sequelize.Sequelize.col('col_name'), '%d %b %y'), 'col_name']
]}.then(function(result))
{ // dateformate=04 Nov 2017
console.log(result)
}
visit this link for formate
To format Date, you can use Sequelize.fn method. I tried all mentioned methods but it won't work with date_formate. Try to create with bellow one.
model.findAll({
attributes: [
'id',
[sequelize.fn('FORMAT', sequelize.col('col_name'), 'yyyy-mm-dd'), 'col_name']
]})
.then(function(result) {
console.log(result);
});
here, i used 'FORMAT' instead of 'DATE_FORMAT', because it through error :
'date_format' is not a recognized built-in function name in api call.
For me, date_format and Format functions in above answers do not work.
I solved as below:
attributes: [
[sequelize.literal('date("dateTime")'), 'dateWithoutTime'],
],
I'm new to sequelize and Node.js.
I coded for test sequelize, but error occured "ER_NO_SUCH_TABLE : Table 'db.node_tests' doesn't exist"
Error is very simple.
However, I want to get data from "node_test" table.
I think sequelize appends 's' character.
There is my source code.
var Sequelize = require('sequelize');
var sequelize = new Sequelize('db', 'user', 'pass');
var nodeTest = sequelize.define('node_test',
{ uid: Sequelize.INTEGER
, val: Sequelize.STRING} );
nodeTest.find({where:{uid:'1'}})
.success(function(tbl){
console.log(tbl);
});
I already create table "node_test", and inserted data using mysql client.
Does I misunderstood usage?
I found the answer my own question.
I appended Sequelize method option following. {define:{freezeTableName:true}}
Then sequelize not appends 's' character after table name.
Though the answer works nicely, I nowadays recommend the use of the tableName option when declaring the model:
sequelize.define('node_test', {
uid: Sequelize.INTEGER,
val: Sequelize.STRING
}, {
tableName: 'node_test'
});
http://docs.sequelizejs.com/manual/tutorial/models-definition.html
Sequelize is using by default the plural of the passed model name. So it will look for the table "node_tests" or "NodeTests". Also it can create the table for you if you want that.
nodeTest.sync().success(function() {
// here comes your find command.
})
Sync will try to create the table if it does not already exist. You can also drop the existing table and create a new one from scratch by using sync({ force: true }). Check the SQL commands on your command line for more details about what is going on.
When you define a model to an existing table, you need to set two options for sequelize to:
find your table name as-is and
not fret about sequelize's default columns updatedAt and createdAt that it expects.
Simply add both options like so:
var nodeTest = sequelize.define('node_test',
{ uid: Sequelize.INTEGER , val: Sequelize.STRING},
{ freezeTableName: true , timestamps: false} //add both options here
);
Note the options parameter:
sequelize.define('name_of_your_table',
{attributes_of_your_table_columns},
{options}
);
Missing either options triggers respective errors when using sequelize methods such as nodeTest.findAll().
> ER_NO_SUCH_TABLE //freezeTableName
> ER_BAD_FIELD_ERROR //timestamps
Alternatively, you can:
create a fresh table through sequelize. It will append "s" to the table name and create two timestamp columns as defaults or
use sequelize-auto, an awesome npm package to generate sequelize models from your existing database programmatically.
Here's the sequelize documentation for option configurations.
In my case, it was due to case. I was having:
sequelize.define('User', {
The correct way is to use lowercase:
sequelize.define('user', {