Related
I am building a Node JS application using Express JS. I need to implement the file upload logic for my application. I am using Formidable, https://www.npmjs.com/package/formidable for uploading file(s). I could upload the files using that library without any issue when the request body format is simple. Now, I am trying to upload the file which is a property/ field of an object in the request body. The following is the dummy request body structure.
{
users: [
{
name: `Test User 1`,
photoFile: {{ here I want to upload file for this user }}
},
{
name: `Test User 2`,
photoFile: {{ here I want to upload file for this user }}
},
// the list goes on
]
}
I am trying to send the test payload/ data in the Postman as in the screenshot below.
This is my code to parse the form
private getForm = async (
req: Request,
options: FormOptions = {
multiples: false
}
) => {
const tempUploadDir = this.getTempUploadDirectory(req);
if (!(await this.storage.exits(tempUploadDir))) {
await this.storage.mkdir(tempUploadDir);
}
return new IncomingForm({
multiples: options.multiples,
uploadDir: tempUploadDir,
maxFileSize: config.fileSystem.maxUploadLimit
});
};
public parseFormData = async <T>(
request: Request,
options: FormOptions = {
multiples: false
}
): Promise<T> => {
const form = await this.getForm(request, options);
return new Promise<T>((resolve) => {
form.parse(request, (err, fields, files) => {
if (err) {
if (err.code === FILE_TOO_LARGE_ERROR_CODE) {
// file too large
throw new UploadMaxFileSizeException(
`Upload file size limit exceeded.`
);
}
throw err;
}
let filesData: {
[key: string]: IUploadedFile[];
} = {};
for (let fileField in files) {
if (`length` in files[fileField]) {
filesData[fileField] = files[fileField] as IUploadedFile[];
} else {
filesData[fileField] = [files[fileField] as IUploadedFile];
}
}
return resolve({
...fields,
...files
} as T);
});
});
};
When I print out the result of parseFormData, I am getting the following result.
As you can see, the field field, 'users[0][photoFile]' is not parsed putting into the corresponding field of the request body object. Instead, the entire field name is string, 'users[0][photoFile]'. What is wrong with my code and how can I fix it?
I my project, file will upload to server when user upload files and get the link back
And when submit form, user avatar just is a link, not a file
I am trying to list files and trash some file from it but the update API throwing error file not find. Though list API is returning me the result and I am passing the same file id to the update API.
drive.files.list(
{
q:
"mimeType='application/zip' and name = '" +
prevFileName +
"' and '" +
parent +
"' in parents",
fields: "files(kind,id,name,mimeType,teamDriveId,parents,createdTime)",
corpora: "drive",
includeTeamDriveItems: true,
supportsTeamDrives: true,
teamDriveId: teamDriveId
},
(err, res) => {
if (err) {
console.log("Error listing files ", err.message);
} else {
const prevFiles = res && res.data && res.data.files;
console.log("prevFiles : ", prevFiles);
if (prevFiles && prevFiles[0] && prevFiles[0].name === prevFileName) {
drive.files.update(
{
fileId: prevFiles[0].id
},
{ resource: { trashed: true } },
(err, result) => {
if (!err) {
console.log("Prev file successfully deleted : ", result);
resolve("File upload SuccessFully: ", file.data);
} else {
console.error("Error removing prev file :: ", err);
}
}
);
}
}
}
);
Here I am getting a file with some json response
[ { kind: 'drive#file',
id: 'ksnadoasjndajn12n1n212',
name: 'xxx_x_x.zip',
mimeType: 'application/zip',
parents: [ 'xxxx-sandadaadakmke' ],
createdTime: '2021-07-14T06:12:11.169Z',
teamDriveId: 'xxxonaijsojxxx' } ]
But when I am passing this id in update API it is throwing error.
My Auth scopes are
var SCOPES = [
"https://www.googleapis.com/auth/drive",
"https://www.googleapis.com/auth/drive.appdata",
"https://www.googleapis.com/auth/drive.file",
"https://www.googleapis.com/auth/drive.readonly",
"https://www.googleapis.com/auth/drive.metadata.readonly",
"https://www.googleapis.com/auth/drive.metadata",
"https://www.googleapis.com/auth/drive.photos.readonly"
];
Please help me out in this.
Thanks.
Since your drive.files.list request contains includeTeamDriveItems: true and supportsTeamDrives: true, you should specify the respective option for drive.files.update
By default supportsAllDrives is set to false unless you explicitly set it to true.
Thuss, files located on a team drive will not be found.
Note: supportsTeamDrives is deprecated, setting supportsAllDrives to true is enough.
I have the Lambda function below that is supposed to take changes to an existing Dynamodb table and make them to another Dynamodb table.
I purposely left the code I've tried commented out I am down to two paths both of which throw errors in different places.
If I include SharedIniFileCredentials in the code I get the following error in the Cloudwatch logs:
{
"message": "Missing credentials in config, if using AWS_CONFIG_FILE, set AWS_SDK_LOAD_CONFIG=1",
"errno": -2,
"syscall": "open",
"code": "CredentialsError",
"path": "/home/sbx_user1051/.aws/credentials",
"time": "2020-11-16T02:57:31.177Z",
"originalError": {
"message": "Could not load credentials from SharedIniFileCredentials",
"errno": -2,
"syscall": "open",
"code": "CredentialsError",
"path": "/home/sbx_user1051/.aws/credentials",
"time": "2020-11-16T02:57:31.177Z",
"originalError": {
"errno": -2,
"syscall": "open",
"code": "ENOENT",
"path": "/home/sbx_user1051/.aws/credentials",
"message": "ENOENT: no such file or directory, open '/home/sbx_user1051/.aws/credentials'"
}
}
}
If I remove that an just let the role that I created and attached (which the AWS documentation says is all I need to do) do the work for permissions I get no error in Cloudwatch logs but I see in my console statements that the putItem (or put if I use the other class) never gets executed. The code returns before it executes those functions. So if I go to the AWS console and look at the Trigger tab on my Dynamodb table I see "PROBLEM: Function call failed".
I have add environment variable. I have hard coded the key/secret into the code. I have tried running this on the command line with AWS CLI. I added environment variables. I have made sure my credentials and config files are populated correctly. Same two issues no matter what I do.
I even tried AWS CLI. If I run this on the command line with the following command
aws lambda invoke --function-name myfunction --cli-binary-format raw-in-base64-out --payload file://mynewitem2.json output.txt
I get this after a few seconds go by
Read timeout on endpoint URL: "https://lambda.us-east-2.amazonaws.com/2015-03-31/functions/myfunction/invocations"
If I add the config file statements and then re-run it on the command line I get this, but the Cloudwatch logs do not show that I reached the putItem or put commands:
{
"StatusCode": 200,
"ExecutedVersion": "$LATEST"
}
Does anyone have any suggestions here? I have looked through all the similar requests for help with no luck. I've looked at the AWS documentation over and over.
I'm completely stuck here. I must be missing something. I just don't know what it is. Do any of you??
Here is the Lambda function:
'use strict';
console.log("starting myplugin insertion . . . ");
var AWS = require("aws-sdk");
AWS.config.logger = console;
//AWS.config = new AWS.Config();
//AWS.config.accessKeyId = "...";
//AWS.config.secretAccessKey = "...";
//AWS.config.region = 'us-east-2';
//AWS.config.update({
// aws_access_key_id : '...',
// aws_secret_access_key : '...',
// region: 'us-east-2'
// });
var creds = new AWS.SharedIniFileCredentials({profile: 'myprofile'});
console.log('Creds problem: ', JSON.stringify(creds, null, 2));
AWS.config.credentials = creds;
AWS.config.getCredentials(function(err) {
if (err)
console.log('Cred problem: ', JSON.stringify(err, null, 2));
else {
console.log("Cred Access:", JSON.stringify(AWS.config.credentials, null, 2));
}
});
var ddb = new AWS.DynamoDB({
'apiVersion': '2012-08-10'
});
exports.handler = (event, context, callback) => {
console.log(JSON.stringify(event, null, 2));
event.Records.forEach((record) => {
console.log('my Stream record: ', JSON.stringify(record, null, 2));
if (record.eventName == 'INSERT') {
console.log('my INSERTING RECORD');
var params = {
TableName: 'myplugin_temp',
Item: {
"client" : record.dynamodb.NewImage.client,
"expiration" : record.dynamodb.NewImage.expiration,
"notificationurl" : record.dynamodb.NewImage.notificationurl,
"clientid" : record.dynamodb.NewImage.clientid,
"s3path" : record.dynamodb.NewImage.s3path,
"language" : record.dynamodb.NewImage.language,
"filename" : record.dynamodb.NewImage.filename,
"timecreated" : record.dynamodb.NewImage.timecreated,
"appid" : record.dynamodb.NewImage.appid,
"subtitle" : record.dynamodb.NewImage.subtitle,
"host" : record.dynamodb.NewImage.host,
"mediatype" : record.dynamodb.NewImage.mediatype,
"sourcemimtype" : record.dynamodb.NewImage.sourcemimetype,
}
};
console.log("my UP HERE");
ddb.putItem(params, function(err,data) {
console.log("my HERE");
if (err) {
console.log("my INSERTING Error", JSON.stringify(err, null, 2));
} else {
console.log("my INSERTING Success",JSON.stringify(data, null, 2));
console.log("my COMPLETED INSERTION MODULE");
}
});
}
if (record.eventName == 'REMOVE') {
console.log('my DELETING RECORD');
var params = {
TableName: 'myplugin_temp',
Item: {
"client" : record.dynamodb.NewImage.client,
"expiration" : record.dynamodb.NewImage.expiration,
"notificationurl" : record.dynamodb.NewImage.notificationurl,
"clientid" : record.dynamodb.NewImage.clientid,
"s3path" : record.dynamodb.NewImage.s3path,
"language" : record.dynamodb.NewImage.language,
"filename" : record.dynamodb.NewImage.filename,
"timecreated" : record.dynamodb.NewImage.timecreated,
"appid" : record.dynamodb.NewImage.appid,
"subtitle" : record.dynamodb.NewImage.subtitle,
"host" : record.dynamodb.NewImage.host,
"mediatype" : record.dynamodb.NewImage.mediatype,
"sourcemimtype" : record.dynamodb.NewImage.sourcemimetype,
}
};
ddb.deleteItem(params, function(err, data) {
if (err) {
console.log("my DELETING Error", JSON.stringify(err, null, 2));
} else {
console.log("my DEL Success", JSON.stringify(data, null, 2));
}
});
}
if (record.eventName == 'MODIFY') {
console.log('my MODIFYING RECORD');
var params = {
TableName: 'myplugin_temp',
Item: {
"client" : record.dynamodb.NewImage.client,
"expiration" : record.dynamodb.NewImage.expiration,
"notificationurl" : record.dynamodb.NewImage.notificationurl,
"clientid" : record.dynamodb.NewImage.clientid,
"s3path" : record.dynamodb.NewImage.s3path,
"language" : record.dynamodb.NewImage.language,
"filename" : record.dynamodb.NewImage.filename,
"timecreated" : record.dynamodb.NewImage.timecreated,
"appid" : record.dynamodb.NewImage.appid,
"subtitle" : record.dynamodb.NewImage.subtitle,
"host" : record.dynamodb.NewImage.host,
"mediatype" : record.dynamodb.NewImage.mediatype,
"sourcemimtype" : record.dynamodb.NewImage.sourcemimetype,
}
};
ddb.updateItem(params, function(err,data) {
if (err) {
console.log("my UPDATING Error", JSON.stringify(err, null, 2));
} else {
console.log("my UPDATING Success",JSON.stringify(data, null, 2));
}
});
}
});
//callback(null, `my Successfully processed records.`);
console.log(JSON.stringify(callback, null, 2));
};
Well, some notes to be highlighted :
AWS.SharedIniFileCredentials should load a particular path of the user home. On linux it will be on /home/[user]/.aws/credentials. While the lambda function has no access to that system path. In other words, AWS.SharedIniFileCredentials is possible to be implemented on the system which is you have access to the home directory.
Loading Credentials in Node.js from the Shared Credentials File
There are two ways to setup the permission for lambda to have access into aws services i.e dynamodb :
Creating an IAM role for lambda and add dynamodb access to the policy. Actually it should ask you to create new role policy when creating new lambda function from aws console.
How to Create an AWS IAM Policy to Grant AWS Lambda Access to an Amazon DynamoDB Table
Using IAM user keys that you have done above
Remember to have knowledge about async and sync process on javascript. On your lambda function, you miss this part there for example :
// it is a promise function that running async process
ddb.putItem(params, function(err,data) {
...
})
I would prefer to suggest change the code using the latest approach and aws promise best practice like so :
exports.handler = async (event) => {
try {
let allTasks = [];
event.Records.forEach((record) => {
// set params here
allTasks.push(ddb.putItem(params).promise());
...
})
if (allTasks.length > 0) {
return Promise.all(allTasks).then(results => {
console.log(results)
})
}
return
} catch(e) {
console.log(e)
throw e
}
}
I am exploring Cloud Functions with FCM to do a push notification to my iOS app device. I have the following structure and I am listening for people who registered to an event. I want to extract the eventHost so that I can go to my users node to find the userUID and eventually his deviceID, and send a push notification to him.
{
"events" : {
"XXX" : {
"eventHost" : "YYY", //<-- How do I get this node?
"eventID" : "XXX",
"registered" : { //<-- Listening for this node
"ASKDJHAIUCHA" : {
"name" : "Emma Watson",
"userUID" : "ASKDJHAIUCHA"
}
},
},
},
"users" : {
"YYY" : {
"deviceID" : "1234456",
"name" : "Andrew Garfield",
"userUID" : "YYY"
},
}
}
My code for the Cloud Functions as such so far:
exports.sendNotification = functions.database.ref('/events/{eventId}/registered')
.onWrite(event => {
const register = event.data.val();
const eventHost = functions.database.ref('/events/' + event.params.eventId + '/eventHost')
console.log('sendNotifications', eventHost);
const payload = {
notification: {
title: "Event Registration",
body: "Someone registered to your event!"
}
};
const options = {
priority: "high"
};
return admin.messaging().sendToDevice("theDeviceID", payload, options)
});
And my Swift portion when adding value into the database is as such:
#IBAction func registerButtonDidTap(_ sender: Any) {
let personDict = [FIRConstants.UserInfo.Name: user.userName,
FIRConstants.UserInfo.UserUID: user.userUID]
let registerPerson = [user.userUID!: personDict as AnyObject] as [String: AnyObject]
let values = ["registered": registerPerson]
FIRHelperClient.sharedInstance.checkIfEventHasRegistrants(ref, event.eventID!) { (has, error) in
if let error = error {
print(error.localizedDescription)
} else {
if let has = has {
if has {
self.ref.child("events").child(self.event.eventID!).child("registered").updateChildValues(registerPerson)
} else {
self.ref.child("events").child(self.event.eventID!).updateChildValues(values)
}
}
}
}
}
My cloud functions is definitely not complete as currently I am hardcoding theDeviceID. As I am pretty inexperience with Node.js and am trying to write both the iOS code in Swift and the server side code, so pls pardon me if this question is elementary. Some advice here would be helpful, thanks.
You'll need to read the host, which your code currently doesn't do.
exports.sendNotification = functions.database.ref('/events/{eventId}/registered')
.onWrite(event => {
const register = event.data.val();
const eventHostRef = functions.database.ref('/events/' + event.params.eventId + '/eventHost')
return eventHostRef.once('value', (eventHostSnapshot) => {
const eventHost = eventHostSnapshot.val();
console.log('sendNotifications', eventHost);
const payload = {
notification: {
title: "Event Registration",
body: "Someone registered to your event!"
}
};
const options = {
priority: "high"
};
return admin.messaging().sendToDevice("theDeviceID", payload, options)
});
});
I highly recommend that you spend some time learning how to interact with the Firebase Database through JavaScript before continuing though. This doesn't have to be through Cloud Functions. You could also use the Firebase Database Admin SDK from Node.js on your client or take the Firebase codelab for web. Whichever one you take, it will ensure you're much better prepared for interacting with the database through Cloud Functions.
As a final warning: you're nesting multiple data types under a single list. This is not recommended, since it leads to all kinds of problems down the line. Instead, I would pull the registered users into their own top-level node, so that you get:
{
"events" : {
"XXX" : {
"eventHost" : "YYY",
"eventID" : "XXX"
}
},
"registered" : {
"XXX" : {
"ASKDJHAIUCHA" : {
"name" : "Emma Watson",
"userUID" : "ASKDJHAIUCHA"
}
}
},
"users" : {
"YYY" : {
"deviceID" : "1234456",
"name" : "Andrew Garfield",
"userUID" : "YYY"
}
}
}
Has anyone created an elastic beanstalk application with the AWS javascript sdk? I've been able to update existing applications using grunt, that works really well. But as part of a continuous integration/continuous deployment project, we want to also create the app when it's not there. I find the documentation confusing, and in AWS's usual fashion, lacking in any kind of cohesive examples, that say, "do this, then this." If anyone has done this and can point me in the right direction, that would be a great help. At this point in time, I'm not sure whether it's a single step or multi step process.
So, here's a basic node package to build an app. I have uploaded a basic API app as a zip file, it doesn't much of anything. The idea is that once it's created, I can then update it using a grunt script - there are a couple of very good grunt modules that will do that, once it's created. But the initial creation was missing. Easy enough to add on more parameters to this now, too.
var applicationName = process.argv[2];
var environmentName = process.argv[3];
var regionName = process.argv[4];
var AWS = require('aws-sdk');
AWS.config.update({region: regionName});
var applicationParams = {
ApplicationName: applicationName
};
var environmentParams =
{
ApplicationName: applicationName, /* required */
EnvironmentName: environmentName, /* required */
VersionLabel: 'initial',
SolutionStackName: "64bit Amazon Linux 2015.03 v1.4.4 running Node.js",
CNAMEPrefix: applicationName,
Tier:
{
Version: " ",
Type: "Standard",
Name: "WebServer"
},
OptionSettings:
[
{
Namespace: 'aws:elasticbeanstalk:environment',
OptionName: 'EnvironmentType',
Value: 'SingleInstance'
},
{
Namespace: 'aws:autoscaling:launchconfiguration',
OptionName: 'EC2KeyName',
Value: 'MyPemFile'
},
{
Namespace: 'aws:autoscaling:launchconfiguration',
OptionName: 'IamInstanceProfile',
Value: 'aws-elasticbeanstalk-ec2-role'
},
{
Namespace: 'aws:autoscaling:launchconfiguration',
OptionName: 'InstanceType',
Value: 't1.micro'
}
],
};
var versionParams =
{
ApplicationName: applicationName, /* required */
VersionLabel: 'initial', /* required */
AutoCreateApplication: true,
SourceBundle:
{
S3Bucket: 'beanstalk-test-ff',
S3Key: 'test-app.zip'
}
};
var elasticbeanstalk = new AWS.ElasticBeanstalk();
elasticbeanstalk.createApplication(applicationParams, function(err, data)
{
console.log('Creating application');
console.log(data);
if (err)
{
if (err.message.indexOf("already exists") > -1)
{
console.log('Application already exists, continuing on');
}
else
{
console.log(err,err.stack); // an error occurred
}
}
else
{
elasticbeanstalk.createApplicationVersion(versionParams, function(err, data)
{
console.log('Creating application version....');
console.log(data);
if (err) console.log(err, err.stack); // an error occurred
else
{
elasticbeanstalk.createEnvironment(environmentParams, function(err, data)
{
console.log('Creating application environment....');
console.log(data);
if (err) console.log(err, err.stack); // an error occurred
});
}
});
}
});