The problems is the excel file I create in a node/express app, downloads the file with zero bytes via http. What should I be doing to download the file with data?
I have a node/express code that generates an excel file and saves it to a temporary folder. Ultimately I intend to move this code to google cloud functions once it works. The problems is, even though, the excel file does get created with data (as I can see it in the temp folder), the res.download(filename) via http downloads a file with zero bytes. What should I be doing to download the file with data?
Sample code follows below. Please ignore significance of sample data, commented code and console.logs
// import * as functions from 'firebase-functions';
import * as Excel from 'exceljs';
import * as express from 'express';
import * as os from 'os';
import * as path from 'path';
const app = express();
const tempFilePath = path.join(os.tmpdir(), "excel.xlsx");
interface Country {
id: number;
country: string;
capital: string;
population: number;
}
interface Heading {
header: string;
key: string;
width: number;
}
interface Align { col: string; align: string; }
function addBottomBorder(worksheet: any, cols: number, rowNo: number) {
for (let i = 0; i < cols; i++) {
const col = String.fromCharCode(65 + i) + rowNo;
worksheet.getCell(col).border = {
bottom: {style:'thin'},
};
}
}
function hiliteRow(worksheet: any, row: number, color: string, cols: number) {
for (let i = 0; i < cols; i++) {
const col = String.fromCharCode(65 + i) + row;
worksheet.getCell(col).fill = {
type: 'pattern',
pattern: 'solid',
fgColor: { argb: color }
};
}
}
function createHeadings(worksheet: any, headings: Heading[]) {
worksheet.columns = headings;
hiliteRow(worksheet, 1, '808B96', headings.length);
addBottomBorder(worksheet, headings.length, 1);
// add filters
const lastColumn = String.fromCharCode(64 + headings.length);
worksheet.autoFilter = { from: 'A1', to: `${lastColumn}1`, };
}
async function generate(data: Country[], headings: Heading[], sheetname: string, alignments: Align[]) {
const workbook = new Excel.Workbook();
const worksheet: any = workbook.addWorksheet(sheetname);
createHeadings(worksheet, headings);
for (const alignObj of alignments) {
worksheet.getColumn(alignObj.col).alignment = { vertical: 'middle', horizontal: alignObj.align };
}
data.forEach((r: any , i: number) => {
worksheet.addRow(r);
if (i % 2 !== 0) {
hiliteRow(worksheet, i + 2, 'D6DBDF', headings.length)
}
});
addBottomBorder(worksheet, headings.length, data.length + 1);
console.log(tempFilePath);
workbook.xlsx.writeFile(tempFilePath).then( result => {
console.log('...ready', result)
return tempFilePath;
})
.catch( err => {
return err;
})
}
app.get('/', async (req, res) => {
const alignments = [
{ col: 'A', align: 'left'},
{ col: 'D', align: 'right'},
];
const columns = [
{ header: 'ID', key: 'id', width: 4 },
{ header: 'Country', key: 'country', width: 10 },
{ header: 'Capital', key: 'capital', width: 22 },
{ header: 'Population', key: 'population', width: 9 }
];
const data = [{
id: 1,
country: 'USA',
capital: 'Washington DC',
population: 325
}, {
id: 2,
country: 'UK',
capital: 'London',
population: 66
}, {
id: 3,
country: 'Italy',
capital: 'Rome',
population: 60.59
}, {
id: 4,
country: 'China',
capital: 'Beijing',
population: 1386
}, {
id: 5,
country: 'Canada',
capital: 'Ottawa',
population: 36.7
}, {
id: 6,
country: 'UK',
capital: 'London',
population: 66
}, {
id: 7,
country: 'Italy',
capital: 'Rome',
population: 60.59
}, {
id: 8,
country: 'China',
capital: 'Beijing',
population: 1386
}, {
id: 9,
country: 'Canada',
capital: 'Ottawa',
population: 36.7
}
];
const sheetname = 'countries';
try {
generate(data, columns, sheetname, alignments).then( notImportant => {
console.log(notImportant);
// !!!!!!!!! - a file with zero bytes is downloaded
// !!!!!!!!! - same result with res.sendFile(...)
res.status(200).download(tempFilePath);
})
} catch (error) {
res.status(500).send(error);
}
})
app.listen(3001, () => {
console.log('...listening')
})
// exports.getExcel = functions.https.onRequest(app);
The solution for me to this issue was to make 2 endpoints - 1s to generate the file, and a second to download the file
i need to get the id for the inserted/updated record when using .upsert() in sequelize.
right now .upsert() returns a boolean indicating whether the row was created or updated.
return db.VenueAddress.upsert({
addressId:address.addressId,
venueId: venue.venueId,
street: address.street,
zipCode: address.zipCode,
venueAddressDeletedAt: null
}).then(function(test){
//test returned here as true or false how can i get the inserted id here so i can insert data in other tables using this new id?
});
I don't think that returning the upserted record was available when the OP asked this question, but it has since been implemented with this PR. As of Sequelize v4.32.1, you can pass a boolean returning as a query param to select between returning an array with the record and a boolean, or just a boolean indicating whether or not a new record was created.
You do still need to provide the id of the record you want to upsert or a new record will be created.
For example:
const [record, created] = await Model.upsert(
{ id: 1, name: 'foo' }, // Record to upsert
{ returning: true } // Return upserted record
);
I wanted upsert to return the created or updated object. It doesn't because only PGSQL supports it directly, apparently.
So I created a naive implementation that will - probably in a non-performant way, and possibly with all sorts of race conditions, do that:
Sequelize.Model.prototype.findCreateUpdate = function(findWhereMap, newValuesMap) {
return this.findOrCreate({
where: findWhereMap,
defaults: findWhereMap
})
.spread(function(newObj, created) {
// set:
for(var key in newValuesMap) {
newObj[key] = newValuesMap[key];
}
return newObj.save();
});
};
Usage when trying to create/update a move in a game (contrived example alert!):
models.Game
.findOne({where: {gameId: gameId}})
.then(function(game) {
return db.Move.findCreateUpdate(
{gameId: gameId, moveNum: game.moveNum+1},
{startPos: 'kr4', endPos: 'Kp2'}
);
});
This is what worked for me:
Model.upsert({
title:your title,
desc:your description,
location:your locations
}).then(function (test) {
if(test){
res.status(200);
res.send("Successfully stored");
}else{
res.status(200);
res.send("Successfully inserted");
}
})
It will check db to find based on your primary key. If it finds then, it will update the data otherwise it will create a new row/insert into a new row.
i know this is an old post, but in case this helps anyone
const upsert = async (model: any, values: any, condition: any): Promise<any> => {
const obj = await model.findOne({ where: condition })
if (obj) {
// only do update is value is different from queried object from db
for (var key in values) {
const val = values[key]
if (parseFloat(obj[key]) !== val) {
obj.isUpdatedRecord = true
return obj.update(values)
}
}
obj.isUpdatedRecord = false
return obj
} else {
// insert
const merged = { ...values, ...condition }
return model.create(merged)
}
}
It isn't using upsert, but .bulkCreate has an updateOnDuplicate parameter, which allows you to update certain fields (instead of creating a new row) in the event that the primary key already exists.
MyModel.bulkCreate(
newRows,
{
updateOnDuplicate: ["venueId", ...]
}
)
I believe this returns the resulting objects, and so I think this might enable the functionality you're looking for?
janmeier said:
This is only supported by postgres, so to keep the API consistent across dialects this is not possible.
please see : https://github.com/sequelize/sequelize/issues/3354
I believe my solution is the most up to date with most minimal coding.
const SequelizeModel = require('sequelize/lib/model')
SequelizeModel.upsert = function() {
return this.findOne({
where: arguments[0].where
}).then(obj => {
if(obj) {
obj.update(arguments[0].defaults)
return
}
return this.create(arguments[0].defaults)
})
}
I know this is an old post, but in case this helps anyone...you can get the returned id or any other value in this way based on OP data.
var data = {
addressId:address.addressId,
venueId: venue.venueId,
street: address.street,
zipCode: address.zipCode,
venueAddressDeletedAt: null
}
const result = await db.VenueAddress.upsert(data, { returning: true });
console.log('resulttttttttttttttttt =>', result)
res.status(200).json({ message: 'Your success message', data: result[0].id});
Noticed how I passed { returning: true } and get the value from the result data.
Super old, but if it helps someone:
const [city, created] = await City.upsert({
id: 5,
cityName: "Glasgow",
population: 99999,
});
created is the boolean saying whether the item was created, and in city you have the whole item, where you can get your id.
No need of returning, and this is db agnostic :)
The only solution for SQLite in Sequelize 6.14.0 is to query the inserted row again
I haven't found a solution that works besides a new SELECT query.
It does work in PostgreSQL however.
Presumably, this is because RETURNING was only implemented relatively recently in SQLite 3.35.0 from 2021: https://www.sqlite.org/lang_returning.html and Sequelize doesn't use that version yet.
I've tried both:
Model.upsert with returning: true: did not work on SQLite. BTW, as mentioned at: https://sequelize.org/api/v6/class/src/model.js~model#static-method-upsert returning already defaults to true now, so you don't need to pass it explicitly
Model.bulkCreate with updatOnDuplicate
In both of those cases, some dummy value is returned when the object is present, not the one that is actually modified.
Minimal runnable examples from https://cirosantilli.com/sequelize
update_on_duplicate.js
#!/usr/bin/env node
const assert = require('assert')
const path = require('path')
const { DataTypes, Sequelize } = require('sequelize')
let sequelize
if (process.argv[2] === 'p') {
sequelize = new Sequelize('tmp', undefined, undefined, {
dialect: 'postgres',
host: '/var/run/postgresql',
})
} else {
sequelize = new Sequelize({
dialect: 'sqlite',
storage: 'tmp.sqlite',
})
}
function assertEqual(rows, rowsExpect) {
assert.strictEqual(rows.length, rowsExpect.length)
for (let i = 0; i < rows.length; i++) {
let row = rows[i]
let rowExpect = rowsExpect[i]
for (let key in rowExpect) {
assert.strictEqual(row[key], rowExpect[key])
}
}
}
;(async () => {
const Integer = sequelize.define('Integer',
{
value: {
type: DataTypes.INTEGER,
unique: true, // mandatory
},
name: {
type: DataTypes.STRING,
},
inverse: {
type: DataTypes.INTEGER,
},
},
{
timestamps: false,
}
);
await Integer.sync({ force: true })
await Integer.create({ value: 2, inverse: -2, name: 'two' });
await Integer.create({ value: 3, inverse: -3, name: 'three' });
await Integer.create({ value: 5, inverse: -5, name: 'five' });
let rows
// Initial state.
rows = await Integer.findAll({ order: [['id', 'ASC']]})
assertEqual(rows, [
{ id: 1, value: 2, name: 'two', inverse: -2 },
{ id: 2, value: 3, name: 'three', inverse: -3 },
{ id: 3, value: 5, name: 'five', inverse: -5 },
])
// Update.
rows = await Integer.bulkCreate(
[
{ value: 2, name: 'TWO' },
{ value: 3, name: 'THREE' },
{ value: 7, name: 'SEVEN' },
],
{ updateOnDuplicate: ["name"] }
)
// PostgreSQL runs the desired:
//
// INSERT INTO "Integers" ("id","value","name") VALUES (DEFAULT,2,'TWO'),(DEFAULT,3,'THREE'),(DEFAULT,7,'SEVEN') ON CONFLICT ("value") DO UPDATE SET "name"=EXCLUDED."name" RETURNING "id","value","name","inverse";
//
// but "sequelize": "6.14.0" "sqlite3": "5.0.2" does not use the desired RETURNING which was only added in 3.35.0 2021: https://www.sqlite.org/lang_returning.html
//
// INSERT INTO `Integers` (`id`,`value`,`name`) VALUES (NULL,2,'TWO'),(NULL,3,'THREE'),(NULL,7,'SEVEN') ON CONFLICT (`value`) DO UPDATE SET `name`=EXCLUDED.`name`;
//
// so not sure how it returns any IDs at all, is it just incrementing them manually? In any case, those IDs are
// all wrong as they don't match the final database state, Likely RETURNING will be added at some point.
//
// * https://stackoverflow.com/questions/29063232/sequelize-upsert
// * https://github.com/sequelize/sequelize/issues/7478
// * https://github.com/sequelize/sequelize/issues/12426
// * https://github.com/sequelize/sequelize/issues/3354
if (sequelize.options.dialect === 'postgres') {
assertEqual(rows, [
{ id: 1, value: 2, name: 'TWO', inverse: -2 },
{ id: 2, value: 3, name: 'THREE', inverse: -3 },
// The 6 here seems to be because the new TWO and THREE initially take up dummy rows,
// but are finally restored to final values.
{ id: 6, value: 7, name: 'SEVEN', inverse: null },
])
} else {
assertEqual(rows, [
// These IDs are just completely wrong as mentioned at: https://github.com/sequelize/sequelize/issues/12426
// Will be fixed when one day they use RETURNING.
{ id: 4, value: 2, name: 'TWO', inverse: undefined },
{ id: 5, value: 3, name: 'THREE', inverse: undefined },
{ id: 6, value: 7, name: 'SEVEN', inverse: undefined },
])
}
// Final state.
rows = await Integer.findAll({ order: [['id', 'ASC']]})
assertEqual(rows, [
{ id: 1, value: 2, name: 'TWO', inverse: -2 },
{ id: 2, value: 3, name: 'THREE', inverse: -3 },
{ id: 3, value: 5, name: 'five', inverse: -5 },
{ id: 6, value: 7, name: 'SEVEN', inverse: null },
])
})().finally(() => { return sequelize.close() });
upsert.js
#!/usr/bin/env node
const assert = require('assert')
const path = require('path')
const { DataTypes, Sequelize } = require('sequelize')
let sequelize
if (process.argv[2] === 'p') {
sequelize = new Sequelize('tmp', undefined, undefined, {
dialect: 'postgres',
host: '/var/run/postgresql',
})
} else {
sequelize = new Sequelize({
dialect: 'sqlite',
storage: 'tmp.sqlite',
})
}
function assertEqual(rows, rowsExpect) {
assert.strictEqual(rows.length, rowsExpect.length)
for (let i = 0; i < rows.length; i++) {
let row = rows[i]
let rowExpect = rowsExpect[i]
for (let key in rowExpect) {
assert.strictEqual(row[key], rowExpect[key])
}
}
}
;(async () => {
const Integer = sequelize.define('Integer',
{
value: {
type: DataTypes.INTEGER,
unique: true,
},
name: {
type: DataTypes.STRING,
},
inverse: {
type: DataTypes.INTEGER,
},
},
{
timestamps: false,
}
);
await Integer.sync({ force: true })
await Integer.create({ value: 2, inverse: -2, name: 'two' });
await Integer.create({ value: 3, inverse: -3, name: 'three' });
await Integer.create({ value: 5, inverse: -5, name: 'five' });
let rows
// Initial state.
rows = await Integer.findAll({ order: [['id', 'ASC']]})
assertEqual(rows, [
{ id: 1, value: 2, name: 'two', inverse: -2 },
{ id: 2, value: 3, name: 'three', inverse: -3 },
{ id: 3, value: 5, name: 'five', inverse: -5 },
])
// Update.
rows = [(await Integer.upsert({ value: 2, name: 'TWO' }))[0]]
if (sequelize.options.dialect === 'postgres') {
assertEqual(rows, [
{ id: 1, value: 2, name: 'TWO', inverse: -2 },
])
} else {
// Unexpected ID returned due to the lack of RETURNING, we wanted it to be 1.
assertEqual(rows, [
{ id: 3, value: 2, name: 'TWO', inverse: undefined },
])
}
rows = [(await Integer.upsert({ value: 3, name: 'THREE' }))[0]]
if (sequelize.options.dialect === 'postgres') {
assertEqual(rows, [
{ id: 2, value: 3, name: 'THREE', inverse: -3 },
])
} else {
assertEqual(rows, [
{ id: 3, value: 3, name: 'THREE', inverse: undefined },
])
}
rows = [(await Integer.upsert({ value: 7, name: 'SEVEN' }))[0]]
if (sequelize.options.dialect === 'postgres') {
assertEqual(rows, [
{ id: 6, value: 7, name: 'SEVEN', inverse: null },
])
} else {
assertEqual(rows, [
{ id: 6, value: 7, name: 'SEVEN', inverse: undefined },
])
}
// Final state.
rows = await Integer.findAll({ order: [['value', 'ASC']]})
assertEqual(rows, [
{ id: 1, value: 2, name: 'TWO', inverse: -2 },
{ id: 2, value: 3, name: 'THREE', inverse: -3 },
{ id: 3, value: 5, name: 'five', inverse: -5 },
{ id: 6, value: 7, name: 'SEVEN', inverse: null },
])
})().finally(() => { return sequelize.close() });
package.json
{
"name": "tmp",
"private": true,
"version": "1.0.0",
"dependencies": {
"pg": "8.5.1",
"pg-hstore": "2.3.3",
"sequelize": "6.14.0",
"sql-formatter": "4.0.2",
"sqlite3": "5.0.2"
}
}
In both of those examples, we see that PostgreSQL runs the desired:
INSERT INTO "Integers" ("id","value","name") VALUES (DEFAULT,2,'TWO'),(DEFAULT,3,'THREE'),(DEFAULT,7,'SEVEN') ON CONFLICT ("value") DO UPDATE SET "name"=EXCLUDED."name" RETURNING "id","value","name","inverse";
which works due to RETURNING, but sequelize does not use the desired RETURNING
INSERT INTO `Integers` (`id`,`value`,`name`) VALUES (NULL,2,'TWO'),(NULL,3,'THREE'),(NULL,7,'SEVEN') ON CONFLICT (`value`) DO UPDATE SET `name`=EXCLUDED.`name`;
Tested on Ubuntu 21.10, PostgreSQL 13.5.
Which I myself resolved as follows:
return db.VenueAddress.upsert({
addressId:address.addressId,
venueId: venue.venueId,
street: address.street,
zipCode: address.zipCode,
venueAddressDeletedAt: null
},{individualHooks: true}).then(function(test){
// note individualHooks
});