I have made a JS logger application in Sails.js and everything looks great.
Now, I'm on my way to scale the application: I need to use several tables for the same model (e.g. a table for session1 sesson2 etc based on id).
Let's say that for model "Pageview", I'll be using different tables, like "Pageview1", "Pageview2", and so on.
How could I define dynamically the tableName of that model, so I can change it on every request according to a param or attribute.
So far, I have tried this way
var tableId = 2;
Pageview.tableName = "pageview" + tableId;
Pageview.create(values, function...);
That code does not break the application, or throw any errors, but the record was saved on the default table, instead of the one I wanted.
I dont think it is possible to do so , but when i read this
I think you can use the custom format to dynamically define table names.
If you are inserting a row into a table with an auto increment primary key, you can retrieve the insert id like this:
connection.query('INSERT INTO posts SET ?', {title: 'test'}, function(err, result) {
if (err) throw err;
console.log(result.insertId);
});
if this is valid for rows , then it might be valid for tables.
Related
I am using Sequelize in my node js server. I am ending up with validation errors because my code tries to write the record twice instead of creating it once and then updating it since it's already in DB (Postgresql).
This is the flow I use when the request runs:
const latitude = req.body.latitude;
var metrics = await models.user_car_metrics.findOne({ where: { user_id: userId, car_id: carId } })
if (metrics) {
metrics.latitude = latitude;
.....
} else {
metrics = models.user_car_metrics.build({
user_id: userId,
car_id: carId,
latitude: latitude
....
});
}
var savedMetrics = await metrics();
return res.status(201).json(savedMetrics);
At times, if the client calls the endpoint very fast twice or more the endpoint above tries to save two new rows in user_car_metrics, with the same user_id and car_id, both FK on tables user and car.
I have a constraint:
ALTER TABLE user_car_metrics DROP CONSTRAINT IF EXISTS user_id_car_id_unique, ADD CONSTRAINT user_id_car_id_unique UNIQUE (car_id, user_id);
Point is, there can only be one entry for a given user_id and car_id pair.
Because of that, I started seeing validation issues and after looking into it and adding logs I realize the code above adds duplicates in the table (without the constraint). If the constraint is there, I get validation errors when the code above tries to insert the duplicate record.
Question is, how do I avoid this problem? How do I structure the code so that it won't try to create duplicate records. Is there a way to serialize this?
If you have a unique constraint then you can use upsert to either insert or update the record depending on whether you have a record with the same primary key value or column values that are in the unique constraint.
await models.user_car_metrics.upsert({
user_id: userId,
car_id: carId,
latitude: latitude
....
})
See upsert
PostgreSQL - Implemented with ON CONFLICT DO UPDATE. If update data contains PK field, then PK is selected as the default conflict key. Otherwise, first unique constraint/index will be selected, which can satisfy conflict key requirements.
Was hoping to get some help with this since I've tried everything and the docs are basically completely useless when trying to figure out what to do here.
Essentially, I have a model called Address, which contains all the fields you'd expect.
This auto creates a table called Addresses and by itself works fine. However, I also want to make an "Address_History" table that uses the same model.
Basically, I want this so that in my route if a user enters a new address, it stores the old address in the history table and then puts the new one inside the addresses table.
I can do the actual code for this, but I just can't get this damn association to work in a way like I said above in regards to having 2 separate tables using the same model.
I've tried everything including the as keyword and all that seems to do is add an extra column, not a table.
Basically I want something like this (But working);
User.hasOne(Address, {
as: 'Address',
});
Address.belongsTo(User, {
as: 'Address',
});
User.hasMany(Address, {
as: 'addressHistory',
});
Address.belongsTo(User, {
as: 'addressHistory',
});
Any help is greatly appreciated.
The as option in hasMany() function does not create a new table in your DB. To create a new table, you need to call init() or define(), which are static functions. However, you can keep your codes pretty much DRY by modularizing the ctor and options. For example,
const AddressCommonCtor = {
// your common column fields here
};
const AddressCommonOptions = {
// options here
// Make sure `modelName` field is not included here
// because you want to create two different tables with different names in your DB
};
class Address extends Model {};
class AddressHistory extends Model {};
Address.init(AddressCommonCtor, AddressCommonOptions);
AddressHistory.init(AddressCommonCtor, AddressCommonOptions);
Then you can associate Address and AddressHistory with User like this:
User.hasOne(Address, {
as: 'address',
});
User.hasMany(AddressHistory, {
as: 'addressHistory',
});
I have a base in airtable and now I need to up columns(fields) programmatically.
For example sql has a query for this operation
ALTER TABLE Employees
ADD EmployeeID numeric NOT NULL IDENTITY (1, 1)
Is there a API for Airtable to do such operations?
Airtable offers an API to use, but not for adding columns. There is a Node.js package and several other programing libraries available on Github.
They have official Doc's as well https://airtable.com/api. With out getting creative and hacking something together, it is not possible to modify a table’s schema via the API. So you cannot do actions like add columns or modify the values in an existing table.
var Airtable = require('airtable');
var base = new Airtable({apiKey: 'YOUR_API_KEY'}).base('app$$%&#*(#');
base('Employees').update('EmployeeID', {
"EmployeeID": "Number"
}, function(err, record) {
if (err) { console.error(err); return; }
console.log(record.get('EmployeeID'));
});
Here is a quick snippet of an update function to update a record.
you can't update dynamically an Airtable table. You can update only the existent field values. The response from Nick C is the syntax to PATCH a record.
I am trying to query dynamodb using the following code:
const AWS = require('aws-sdk');
let dynamo = new AWS.DynamoDB.DocumentClient({
service: new AWS.DynamoDB(
{
apiVersion: "2012-08-10",
region: "us-east-1"
}),
convertEmptyValues: true
});
dynamo.query({
TableName: "Jobs",
KeyConditionExpression: 'sstatus = :st',
ExpressionAttributeValues: {
':st': 'processing'
}
}, (err, resp) => {
console.log(err, resp);
});
When I run this, I get an error saying:
ValidationException: Query condition missed key schema element: id
I do not understand this. I have defined id as the partition key for the jobs table and need to find all the jobs that are in processing status.
You're trying to run a query using a condition that does not include the primary key. This is how queries work in DynamoDB. You would need to do a scan for the info in your case, however, I don't think that is the best option.
I think you want to set up a global secondary index and use that to query for the processing status.
In another answer #smcstewart responded to this question. But he provides a link instead of commenting why this error occurs. I want to add a brief comment hoping it will save your time.
AWS docs on Querying a Table states that you can do WHERE condition queries (e.g. SQL query SELECT * FROM Music WHERE Artist='No One You Know') in the DynamoDB way, but with one important caveat:
You MUST specify an EQUALITY condition for the PARTITION key, and you can optionally provide another condition for the SORT key.
Meaning you can only use key attributes with Query. Doing it in any other way would mean that DynamoDB would run a full scan for you which is NOT efficient - less efficient than using Global secondary indexes.
So if you need to query on non-key attributes using Query is usually NOT an option - best option is using Global Secondary Indexes as suggested by #smcstewart.
I found this guide to be useful to create a Global secondary index manually.
If you need to add it using CloudFormation here is a relevant page.
I was getting this error for a different scenario. Here is my scenario.
(It's very unlikely that anyone else ends up with this case, but incase)
I had a query working on a Table (say table A). Table A had a partition key m_id and sort key u_id.
I had a query to fetch data using m_id. The query was working.
'''
var queryParams = {
ExpressionAttributeValues: {
':m_id': mId
},
KeyConditionExpression: 'm_id = :m_id',
TableName: "A"
};
let connections = await docClient.query(queryParams).promise();
'''
I created another Table say Table B. I made some errors in naming keys so I simply deleted and created a table with the same name again, Table B. Table B had partition key m_id, and sort key s_id.
I copied pasted the same query which I was using for Table A, I changed Table name only because partition key had the same name.
To my shock, I get this expectation.
"ValidationException: Query condition missed key schema element"
I rechecked all the names, I compared the query with the working query. Everything was fine.
I thought maybe because, I was deleting recreating Table B, it could be something with that. So I create a fresh Table with a new Name Table B2 with the same key names as Table B.
In my query that was throwing exceptions, I changed only the Table name from B to B2.
And the Exception was gone.
If you are getting this on a fresh table, where no query has worked earlier, creating a new Table with a new name is an option.
If you delete a Table only to change partition key names, it may be safer to use a new name for Table as well (Dynamo could be referring metadata by table names and not by internal identifiers, it is possible that old metadata stays even if you delete a table. Just a guess given I faced this case).
EDIT:2022-July-12
This error does not leave me. My own answer was helpful but one more case, there was a trailing space in name of Key in the table. And Dynamo does not even check for spaces in key names.
You have to create an global secondary index for the status field.
Then, you code could look like smth like this:
dynamo.query({
TableName: "Jobs",
IndexName: 'status',
KeyConditionExpression: '#s = :st',
ExpressionAttributeValues: {
':st': 'processing'
},
ExpressionAttributeNames: {
'#s': 'status',
},
}, (err, resp) => {
console.log(err, resp);
});
Note: scan operation is indeed very costly, especially if you table is huge in size
i solved the problem using AWS.DynamoDB.DocumentClient() with scan, for sample (nodejs):
var docClient = new AWS.DynamoDB.DocumentClient();
var params = {
TableName: "product",
FilterExpression: "#cg = :data",
ExpressionAttributeNames: {
"#cg": "categoria",
},
ExpressionAttributeValues: {
":data": category,
}
};
docClient.scan(params, onScan);
function onScan(err, data) {
if (err) {
// for the log in server
console.error("Unable to scan the table. Error JSON:", JSON.stringify(err, null, 2));
res.json(err);
} else {
console.log("Scan succeeded.");
res.json(data);
}
}
I read all the documentation and this seemingly simple operation seems completely ignored throughout the entire README.
Currently, I am trying to run a SELECT query and console.log the results, but it is simply returning a database object. How do I view the results from my query in Node console?
exports.runDB = function() {
db.serialize(function() {
console.log(db.run('SELECT * FROM archive'));
});
db.close();
}
run does not have retrieval capabilities. You need to use all, each, or get
According to the documentation for all:
Note that it first retrieves all result rows and stores them in
memory. For queries that have potentially large result sets, use the
Database#each function to retrieve all rows or Database#prepare
followed by multiple Statement#get calls to retrieve a previously
unknown amount of rows.
As an illistration:
db.all('SELECT url, rowid FROM archive', function(err, table) {
console.log(table);
});
That will return all entries in the archive table as an array of objects.