I'm trying to store comments for a blog in Cassandra and have come up with this schema (got the idea from here):
create table comments ( slug varchar, ts timestamp, value text, primary key (slug,ts));
Using CQL (I'm using node.js with Helenus driver) I'm trying to add some data to it, here's what I've got so far:
var helenus = require('helenus'),
pool = new helenus.ConnectionPool({
hosts: ['localhost:9160'],
keyspace: 'blogks'
});
pool.on('error', function(err) {
console.error(err.name, err.message);
});
module.exports.addComment = function(slug, comment,callback){
pool.connect(function(connErr,keyspace){
if(connErr){
callback(connErr);
return;
}
var cql = "INSERT INTO comments (slug,ts,value) VALUES (?, ?, ?);";
pool.cql(cql,[slug,serializeDate(new Date()),serializeJSON(comment)],function(err,results){
callback(err,results);
});
});
}
function serializeDate(date){
var dateSerializer = new helenus.Marshal('DateType');
return dateSerializer.serialize(date).toString('hex');
}
function serializeJSON(data){
var utf8Serializer = new helenus.Marshal('UTF8Type');
return utf8Serializer.serialize(JSON.stringify(data)).toString("hex");
}
The idea being you can pass in a comment json object to this method, and push it to cassandra. I'm calling it like this to test:
var comments = require('./comments.js');
comments.addComment("myslug",{id:2,name:"Alex",comment:"Hello!"},function(e,r){
console.log(e);
console.log(r);
})
but whenever I do (I tried various CQL drivers), I get this cryptic error message:
[HelenusInvalidRequestException: unable to coerce 'value' to a formatted date (long)] name: 'HelenusInvalidRequestException'
I tried changing the timestamp to be all different data types, but no luck. The very strange thing is that at first, my primary key was just the slug itself, and then it did work. However, I want to store all comments in one row ordered by timestamp, therefore I had to go with this approach of using two columns for the primary key. Any ideas?
EDIT So based on rs_atl's suggestion, here's what I've tried:
This line:
dateSerializer.serialize(date).toString('hex')
did in fact return garbage (0000013ccfacf5c4), I must have misunderstood how to use the API. So here's what I tried instead:
I tried just new Date().getTime() which in JS does in fact return Unix-style epoch, but that didn't work I got the same error. I then tried using moment.js to try and format the string:
moment().format("YYYY-MM-DD HH:mm:ss")
which seems to be returning the correct format, but again. Same error. Then I tried this:
moment().format("YYYY-MM-DD HH:mm") + "Z"
no luck. I then tried this:
moment().format("YYYY-MM-DD HH:mmZ")
which does in fact add the timezone info at the end, but still, no luck. Any ideas?
it seems you are using the wrong CQL version. You expect to use CQL 3.0.0 but the driver defaults to CQL2. You need to specify the correct CQL version in the initialization.
pool = new helenus.ConnectionPool({
hosts: ['localhost:9160'],
keyspace: 'blogks',
cqlVersion: '3.0.0'
});
And also please ensure you are using the latest version (0.6.2 as of now) of Helenus.
I'm not exactly sure what the output of this line is:
return dateSerializer.serialize(date).toString('hex');
but this is where your issue is. It appears that you are not outputting a valid date that Cassandra understands. A valid date is either:
A Unix-style epoch as a long value.
A string in one of the following forms:
yyyy-mm-dd HH:mm
yyyy-mm-dd HH:mm:ss
yyyy-mm-dd HH:mmZ
yyyy-mm-dd HH:mm:ssZ
yyyy-mm-dd'T'HH:mm
yyyy-mm-dd'T'HH:mmZ
yyyy-mm-dd'T'HH:mm:ss
yyyy-mm-dd'T'HH:mm:ssZ
yyyy-mm-dd
yyyy-mm-ddZ
Check to make sure you are writing one of these valid timestamp types.
For timestamps, I've always use int in milliseconds to represent time.
Try:
Date.now()
instead of:
serializeDate(new Date())
Related
I am using Supabase as my DB.
I have users table with id field being populated with uuid_generate_v4().
I also have table bank_ao with foreign key bankId pointed to users.id.
When I do following:
const uuid = req.params.id //this 100% works correctly and returns UUID every time!
await supabase
.from("bank_ao")
.select()
.eq("bankId", uuid);
I get this error
{"code":"22P02","details":null,"hint":null,"message":"invalid input syntax for type bigint: \"914eda70-2ecf-49b0-9ea6-87640944ed16\""}
I don't have BIGINT field in my entire DB. I have checked 100x.
I also tried to add ' around my uuid variable, but it still doesn't work.
Any ideas?
Could this be some error specific to Supabase?
You need to cast the UUID into text to compare it there:
const uuid = 'e1bee290-45d1-4233-ad96-b9924288d64c'
const supabase = createClient(SUPABASE_URL, SUPABASE_KEY);
const { data: ret, error } = await supabase
.from('bank_ao')
.select()
.eq('bankId::text', uuid);
console.log(ret);
Assuming table format as:
Output:
[
{
bankId: 'e1bee290-45d1-4233-ad96-b9924288d64c',
created_at: '2023-02-08T16:03:21.064501+00:00',
foo: 'bar',
bar: { bar: 'foo' }
}
]
Please note that for using BigInt in Javascript safely, then you can check it here (similar approach).
The problem was the following.
Before I used UUID as my id column value, it used to be BIGINT. When I changed it, it didn't change in cache, or some other type of memory. So, it threw an error.
I fixed it by re-creating entire database (I tried with table first, but it didn't work).
I have read online that if this is local version of Supabase you can just restart Postgres and it works fine (https://github.com/supabase/supabase/discussions/11891).
When I console log the response body right before I save it to the database, my response body shape looks correct. See below
//console.log response body
CreateOpenHourDto {
day: 'WEDNESDAY',
startTime: 1663858800000,
endTime: 1663878600000,
calendarId: 1
}
However, whenever I go into prisma studio and check the new db entry, the startTime, endTime is differnt.
There is nothing I have done to transform the data. Any tips are appreciated.
I am using
nestjs, prisma, postgres sql
My prisma model listed start and end times as "int" types and it should have been BigInt. For anyone that plans on using bigint. Be aware
Prisma returns records as plain JavaScript objects. If you attempt to use JSON.stringify on an object that includes a BigInt field, you will see the following error:
Do not know how to serialize a BigInt
To work around this issue, use a customized implementation of JSON.stringify:
JSON.stringify(
this,
(key, value) => (typeof value === 'bigint' ? value.toString() : value) // return everything else unchanged
)
This sounds hacky but its coming straight form the documentation as of the time of this comment.
https://www.prisma.io/docs/concepts/components/prisma-client/working-with-fields#working-with-bigint
I want to get all data between two dates from sql server with using mssql and query of sequelize in nodejs.
Start and End dates come from react frontend with axios.post.
For example start date is 2021-01-04T08:53:00.000Z and end date is 2021-01-05T21:00:00.000Z in express.
app.post('/getData', function (req, res) {
let startTime=req.body.start
let endTime=req.body.end
sequelize.query(` select Date_Time from S001T01 where Date_Time between ${startTime} and ${endTime}`, {
type: sequelize.QueryTypes.SELECT
}).then(result => {
console.log(result)
})
});
It returns many error, one of them is UnhandledPromiseRejectionWarning: SequelizeDatabaseError: Invalid column name '2021-01-05T21:00:00.000Z 23:59:00'..
In table viewer Date_Time column shown like that
I don't know what should I do.
Thanks for help.
You might have a wrong datetime format. 2021-01-05T21:00:00.000Z 23:59:00 doen't correspond to the traditional ISO 8601.
If you want all entries between 2021-01-04T08:53:00.000Z and 2021-01-05T21:00:00.000Z, your SQL query should look like this :
select Date_Time from S001T01 where Date_Time between 2021-01-04T08:53:00.000Z and 2021-01-05T21:00:00.000Z
You should also consider using modified datetime instead on concatenation.
Maybe something like thing should do the job :
var dt = new Date();
dt.setHours( dt.getHours() + 2 );
I hope it helped you.
I am using node-mssql and my function goes like this:
const getPoliciesDue = async (userId, toDate) => {
const db = await sql.connect(config) // const sql = require('mssql'), config = {config for my DB}
const request = db.request()
.input('userId', sql.VarChar, userId) // userId is a string
.input('toDate', sql.Date, toDate) // toDate is a javascript Date() object
const result = await request.query(
'SELECT policyNumber, holderId, ... FROM Policies WHERE Policies.userId = #userId AND Policies.toDate <= #toDate'
)
return result.recordset
}
I want to get all the policies which are expiring before a certain date belonging to a certain user.
But when I run this exact query, I get the error
Must declare the scalar variable #userId
I removed #userId from the WHERE clause and put '#toDate' (quotes around the variable). Now I get an error
Conversion failed when converting date and/or time from character string
The documentation says input() accepts the Date() object. I have tried passing date object as well as a string in YYYY-MM-DD format, but with no avail. Passing a string in YYYY-MM-DD format works for an INSERT query, but not for a SELECT query. How am I supposed to run my query?
Note I can't run the query in a single line, like
db.request().input().query()
because there is another input which will later go in conditionally.
Okay, a huge oversight on my part and sincere apologies to those who tried to find an answer. I found out that I was running the query() method on the wrong object. I was running the query method on db object whereas I should have run it on the request object. The code works perfectly after calling the method on the right object.
i would like to have a function in my controller to get data based on time range. First, i have all the data in mongodb, there is a attribute ModifiedTime as string looks like 2015-02-25T17:17:33Z. Second, i define the model in sails with ModifiedTime:
{ type: 'datetime', columnName: 'ModifiedTime' }
In the model.js, I set schema: true. Then in my controller, i try to use
User.find({ModifiedTime : {'<=' : new Date('2015-03-18T00:00:00Z')}}).exec(function(err,st){
if (err) return res(err);
if (!st) return res(new Error('Invalid ModifiedTime.'));
return res.json(st);
} );
But i get nothing, see always [] in the browse. I used waterline http://localhost:1337/User to check the data in browse. i can see all the data from mongodb. The strange thing is, i see something like ModifiedTime": "2015-02-18T17:36:53Z. so, for me, it looks like the ModifiedTime in sails is still a string, am i right? but i set the type as datetime in the model. I hope, it could transfer the string of mongodb to datetime in background, won't it? Please give some advise. i spend already too much time for that :(
thank you very much!
WJS
You're right / wrong.
In your DB you state the datetime is a string. If it is already a string then you can't set it to be a date in your sails model. You should simply compare two strings instead of transforming 2015-03-18T00:00:00Z into a date.
User.find({ModifiedTime : {'<=' : '2015-03-18T00:00:00Z'}}).exec(function(err,st){
if (err) return res(err);
if (!st) return res(new Error('Invalid ModifiedTime.'));
return res.json(st);
} );
If you truley want to use Date/Time then you must go through your original data and change modified time to a date/time object.