How to use Lowercase function in Sequelize Postgres - node.js

I am trying to use the lowercase function to do string searching in Sequelize.
I manage to do it using the ilike.
My question is how to use the lowercase function in this scenario?
The findAll using ilike is as following:
Db.models.Person.findAll(where: {firstName: {$ilike: `somename`}});
How do I change it to lower(firstname) = lower('somename');

PostgreSQL generally uses case-sensitive collations. This means data that appears in each column is compared literally against your query.
You have several options:
1. Follow Ben's answer, and wrap your wrap columns and database in a sequelize.fn('lower') call.
Pros: No database changes.
Cons: You need to remember to use it for every query. Foregoes indexes (unless you've already created a functional index) and scans tables sequentially, resulting in slower look-ups with large tables. Quite verbose code.
2. Use ILIKE, to case-insensitively match a pattern
To find the name exactly:
Db.models.Person.findAll(where: {firstName: {$iLike: 'name'}});
To find a fragment, which may be contained within arbitrary characters:
Db.models.Person.findAll(where: {firstName: {$iLike: '%name%'}});
Pros: Easy to remember. No Sequelize function wrappers - it's a built-in operator, so syntax is neater. No special indexes or database changes required.
Cons: Slow, unless you start messing with extensions like pg_trgm
3. Define your text columns with the citext type, which implicitly compares lowercase
Defining your column types as 'citext' (instead of text or character varying) has the same practical effect of turning this:
select * from people where name = 'DAVID'
to this...
select * from people where LOWER(name) = LOWER('DAVID')
The PostgreSQL documentation shows this as an example of how to create your table with the citext type:
CREATE TABLE users (
nick CITEXT PRIMARY KEY,
pass TEXT NOT NULL
);
INSERT INTO users VALUES ( 'larry', md5(random()::text) );
INSERT INTO users VALUES ( 'Tom', md5(random()::text) );
INSERT INTO users VALUES ( 'Damian', md5(random()::text) );
INSERT INTO users VALUES ( 'NEAL', md5(random()::text) );
INSERT INTO users VALUES ( 'Bjørn', md5(random()::text) );
SELECT * FROM users WHERE nick = 'Larry';
TL;DR basically swap out your "text" columns for "citext".
The citext module comes bundled with PostgreSQL 8.4, so there's no need to install any extensions. But you do need to enable it on each database you use it with the following SQL:
CREATE EXTENSION IF NOT EXISTS citext WITH SCHEMA public;
And then in your Sequelize definitions, define a type attribute:
// Assuming `Conn` is a new Sequelize instance
const Person = Conn.define('person', {
firstName: {
allowNull: false,
type: 'citext' // <-- this is the only change
}
});
Then your searches against that column will always be case-insensitive with regular where = queries
Pros: No need to wrap your queries in ugly sequelize.fn calls. No need to remember to explicitly lowercase. Locale aware, so works across all character sets.
Cons: You need to remember to use it in your Sequelize definitions when first defining your table. Always activated - you need to know that you'll want to do case insensitive searching when defining your tables.

You can use native functions in the where clause:
Db.models.Person.findAll({
where: sequelize.where(
sequelize.fn('lower', sequelize.col('firstname')),
sequelize.fn('lower', 'somename')
)
});
which would translate to
select * from person where lower(firstname) = lower('somename');

Related

How to query fields with multiple values in Azure Cognitive Search

Working on Azure Cognitive Search with backend as MS SQL table, have some scenarios where need help to define a query.
Sample table structure and data :
Scenarios 1 : Need to define a query which will return data based on category.
I have tied query using search.ismatch but its uses prefix search and matches other categories as well with similar kind of values i.e. "Embedded" and "Embedded Vision"
$filter=Region eq 'AA' and search.ismatch('Embedded*','Category')
https://{AZ_RESOURCE_NAME}.search.windows.net/indexes/{INDEX_NAME}/docs?api-version=2020-06-30-Preview&$count=true&$filter=Region eq 'AA' and search.ismatch('Embedded*','Category')
And it will response with below result, where it include "Embedded" and "Embedded Vision" both categories.
But my expectation is to fetch data only if it match "Embedded" category, as highlighted below
Scenario 2: For the above Scenario 1, Need little enhancement to find records with multiple category
For example if I pass multiple categories (i.e. "Embedded" , "Automation") need below highlighted output
you'll need to use a different analyzer which will break the tokens on every ';' just for the category field rather than 'whitespaces'.
You should first ensure your Category data is populated as a Collection(Edm.String) in the index. See Supported Data Types in the official documentation. Each of your semicolon-separated values should be separate values in the collection, in a property called Category (or similar).
You can then filter by string values in the collection. See rules for filtering string collections. Assuming that your index contains a string collection field called Category, you can filter by categories containing Embedded like this:
Category/any(c: c eq 'Embedded')
You can filter by multiple values like this:
Category/any(c: search.in(c, 'Embedded, Automation'))
Start with clean data in your index using proper types for the data you have. This allows you to implement proper facets and you can utilize the syntax made specifically for this. Trying to work around this with wildcards is a hack that should be avoided.
To solve above mention problem used a below SQL function which will convert category to a json string array supported by Collection(Edm.String) data type in Azure Search.
Sql Function
CREATE FUNCTION dbo.GetCategoryAsArray
(
#ID VARCHAR(20)
)
RETURNS NVARCHAR(MAX)
AS
BEGIN
DECLARE #result NVARCHAR(MAX) = ''
SET #result = REPLACE(
STUFF(
(SELECT
','''+ TRIM(Value) + ''''
FROM dbo.TABLEA p
CROSS APPLY STRING_SPLIT (Category, ';')
WHERE p.ID = #ID
FOR XML PATH('')
),1,1,''),'&','&')
RETURN '[' + #result + ']'
END
GO
View to use function and return desired data
CREATE View dbo.TABLEA_VIEW AS
select
id
,dbo. GetCategoryAsArray(id) as CategoryArr
,type
,region
,Category
from dbo.TABLEA
Defined a new Azure Search Index using above SQL View as data source and during Index column mapping defined CategoryArr column as Collection(Edm.String) data type
Query to use to achieve expected output from Azure Search
$filter=Region eq 'AA' and CategoryArr/any(c: search.in(c, 'Embedded, Automation'))

Does a dynamic prepared statement makes sense?

I want to create dynamic prepared statements, that every part is dynamic, the values, the table and the WHERE part.
I use nodejs + PostgreSQL and the pg module to talk to the PostgreSQL. The pg module offers a different syntax to go along with node.js , but I guess the principles are the same. This is based to the official example here
//dynamic that can change
let select = 'name , email, age';
let table = 'user';
let where = 'id=$1 AND gender=$2';
let values = [1,'female'];
//prepare
const query = {
// give the query a unique name
name: 'fetch-user',
text: 'SELECT' + select + 'FROM' + table + 'WHERE' + where,
values: values
}
//execute
client.query(query)
.then(res => console.log(res.rows[0]))
.catch(e => console.error(e.stack))
I was wondering if this will make sense , performance-wise.
I red the documentation and , what I understand is that by having all the parts of a prepared statement dynamic, then the planning may be not so effective , or not effective at all.
What should I do? Should I keep this dynamic syntax? Or it doesn't make any sense, so I have to create multiple prepared statements and use them for different tables?
Thanks
There should be no performance issues here. The "dynamic" part of your SQL is just a string you're passing into the query object, so the only performance to consider is resolving the text property. You're passing your database a fully prepared statement; it's nodejs that is resolving the different variables to come up with the query object's text property.

Sequelize get random entry with association and limit, "missing FROM-clause entry

Im trying to get a random entry from a table and its associations.
Recipe.findAll({
order: [
[sequelize.fn('RANDOM')]
],
where: {
'$RecipeCategory.name$': category
},
include:[
{
model: models.Category,
as: 'RecipeCategory',
},{
model: models.Product,
}],
subQuery:false,
limit:1})
With the above code I'm getting a random entry of Recipe and its associations with limit 1. For example , it returns only 1 product and I need all products. I need to get one recipe with all products.
Any suggestions?
If I remove the subQuery option , I receive this:
SequelizeDatabaseError: missing FROM-clause entry for table "RecipeCategory"
Im searching 5 days for the solution and I think that I have check all related answers in this.'
edit: generated query
SELECT "Recipe"."id", "Recipe"."name", "Recipe"."description", "Recipe"."pic", "Recipe"."createdAt", "Recipe"."updatedAt", "RecipeCategory"."id" AS "RecipeCategory.id", "RecipeCategory"."name" AS "RecipeCategory.name", "RecipeCategory"."description" AS "RecipeCategory.description", "RecipeCategory"."createdAt" AS "RecipeCategory.createdAt", "RecipeCategory"."updatedAt" AS "RecipeCategory.updatedAt", "RecipeCategory.Recipes_Categories"."createdAt" AS "RecipeCategory.Recipes_Categories.createdAt", "RecipeCategory.Recipes_Categories"."updatedAt" AS "RecipeCategory.Recipes_Categories.updatedAt", "RecipeCategory.Recipes_Categories"."CategoryId" AS "RecipeCategory.Recipes_Categories.CategoryId", "RecipeCategory.Recipes_Categories"."RecipeId" AS "RecipeCategory.Recipes_Categories.RecipeId", "Products"."id" AS "Products.id", "Products"."name" AS "Products.name", "Products"."protein" AS "Products.protein", "Products"."fat" AS "Products.fat", "Products"."carbohydrates" AS "Products.carbohydrates", "Products"."salt" AS "Products.salt", "Products"."min" AS "Products.min", "Products"."max" AS "Products.max", "Products"."vegan" AS "Products.vegan", "Products"."piece" AS "Products.piece", "Products"."createdAt" AS "Products.createdAt", "Products"."updatedAt" AS "Products.updatedAt", "Products.Product_Recipe"."position" AS "Products.Product_Recipe.position", "Products.Product_Recipe"."createdAt" AS "Products.Product_Recipe.createdAt", "Products.Product_Recipe"."updatedAt" AS "Products.Product_Recipe.updatedAt", "Products.Product_Recipe"."ProductId" AS "Products.Product_Recipe.ProductId", "Products.Product_Recipe"."RecipeId" AS "Products.Product_Recipe.RecipeId"
FROM
"Recipes" AS "Recipe"
LEFT OUTER JOIN (
"Recipes_Categories" AS "RecipeCategory.Recipes_Categories"
INNER JOIN
"Categories" AS "RecipeCategory" ON "RecipeCategory"."id" = "RecipeCategory.Recipes_Categories"."CategoryId"
) ON "Recipe"."id" = "RecipeCategory.Recipes_Categories"."RecipeId"
LEFT OUTER JOIN (
"Product_Recipes" AS "Products.Product_Recipe"
INNER JOIN "Products" AS "Products" ON "Products"."id" = "Products.Product_Recipe"."ProductId"
) ON "Recipe"."id" = "Products.Product_Recipe"."RecipeId"
WHERE "RecipeCategory"."name" = 'meal-3'
ORDER BY RANDOM()
LIMIT 1;
It looks like using an include on the same level as where is not supported as of this thread in 2015. It's not clear what the solution would be in your case.
A relavant quote from the above thread by a sequelize contributor:
include.where won't since you need a negative query.
You could probably try a raw where. .and({filter_level: ...}, ['RAW QUERY']})
On a side note.. On the off chance that you aren't fully commited to using sequelize, I personally recommend using knex.js. I use it in production code. It's a good balanced library. You get a lot of the flexibility/ease of use of an ORM like sequelize, without losing too much sight/control of the query itself. It's very easy to add/run raw queries if the library doesn't support what you do.
This is pretty rough because I'm pretty sure that either sequelize
is overcomplicating this or your database structure is over complicated. This
shouldn't require 4 joins! Here's the general idea:
assume your db auto incrememts the recipes id
get number of recipes from recipes table, using COUNT()
generate a random number between 0 and COUNT(), this is the id of your random recipe
query the recipe table for for the recipe, left join on the products table to get, join on recipe id. This is a simple query that can be written easily in knex.
Profit! you now have a random recipe and all products required for the recipe.

How to convert Rep[T] to T in slick 3.0?

I used a code, generated from slick code generator.
My table has more than 22 columns, hence it uses HList
It generates 1 type and 1 function:
type AccountRow
def AccountRow(uuid: java.util.UUID, providerid: String, email: Option[String], ...):AccountRow
How do I write compiled insert code from generated code?
I tried this:
val insertAccountQueryCompiled = {
def q(uuid:Rep[UUID], providerId:Rep[String], email:Rep[Option[String]], ...) = Account += AccountRow(uuid, providerId, email, ...)
Compiled(q _)
}
I need to convert Rep[T] to T for AccountRow function to work. How do I do that?
Thank you
;TLDR; Not possible
Explanation
There are two levels of abstraction in Slick: Querys and DBIOActions.
When you're dealing with Querys, you have to access your schema definitions, and rows, Reps and, basically, it's very constrained as it's the closest level of abstraction to the actual DB you're using. A Rep refers to an hypothetical value in the database, not in your program.
Then you have DBIOActions, which are the next level... not just some definition of a query, but the execution of it. You usually get DBIOActions when getting information out of a query, like with the result method or (TADAN!) when inserting rows.
Inserts and Updates are not queries and so what you're trying to do is not possible. You're dealing with DBIOAction (the += method), and Query stuff (the Rep types). The only way to get a Rep inside a DBIOAction is by executing a Query and obtaining a DBIOAction and then composing both Actions using flatMap or for comprehensions (which is the same).

How to reference one foreign key column with multiple primary key column

I am creating BOOK_Issue table which will contain id of person to whom the book is issued.
i have a column name user_id witch will contain ids from tbl_student as well as tbl_faculty. so how to set user_id field of book_issue table with reference to two primary key columns.
Your database schema is not correct.
If you expect unique IDs then they should be in one table.
You can create a table with all the users, and have a column to set their type (student, faculty). Then create 2 different tables for each type that has the proper information for each user based on their type.
Create a "person" superclass that can be either of type "student" or type "faculty". Reference this from the BOOK_Issue table instead.
Basically to create this relationship, you'll need one unique ID that spans both "student" and "faculty". Put this in a table (tbl_person?) and have each row in tbl_student and tbl_faculty reference this new table. It's probably also best to then pull out the fields present in both tbl_student and tbl_faculty and put them in this new supertable instead.
You can solve the problem by either having an extra column in BOOK_Issue table, next to user_id, which indicates if this is a Student ID or a Faculty ID.
Alternatively, the IDs themselves may readily include some pattern which indicate their nature (for example no all faculty Ids may start with say "UC", and none of the student Id are so).
The two solutions above then allow using queries similar to the following
SELECT B.*,
CASE B.BorrowerType -- could be LEFT(user_id, 2) etc...
WHEN 'S' THEN S.Name
WHEN 'F' Then F.Name
END As Name,
CASE B.BorrowerType
WHEN 'S' THEN S.PhoneNumber
WHEN 'F' Then F.Phone -- Note that these constructs allow
-- mapping distinct columns names etc.
END As PhoneNr
FROM BOOK_Issue B
LEFT JOIN tbl_student S ON B.BorrowerType = 'S' AND B.user_id = S.id
LEFT JOIN tbl_faculty F ON B.BorrowerType = 'F' AND B.user_id = F.id
WHERE B.DueDate < '11/23/2009' -- or some other condition
This can get a bit heavy when we need to get multiple columns from the student/faculty tables. A possible alternative is a UNION, but this would then cause the repeating of the search clause.
Finally, the best solution but not avaible on all DBMS is a sub-query driven by an "IF B.BorrowerType = 'S' " condition.
This should be your table design:
FacultyTable (FacultyID, FacultyName)
StudentsTable (StudentID, StudentName, FacultlyID, ...)
BookTable (BookID, BookName, ...)
UsersTable(UserID, UserName, UserPassword, StudentID, LastLogin, ...)
Now this is the main thing:
BookIssedTable(BookIssedID, BookID, UserID)
//This table tells me that a book of "BookID was issued to a user of "UserID"
//this can be better for this is certainly a great improvement from the initial design.

Resources