I am trying to upload an image to my s3 bucket using the npm s3 module (https://www.npmjs.org/package/s3)
I used the following params
var params = {
localFile: 'image.png',
s3Params: {
Bucket: 'newstie.com',
Key: '/newsite/image'
}
};
And I am getting the following logs and I wasn't able to understand what I did wrong. Can you please help me? Many Thanks
progress 12181 0 12181
progress 12181 12181 12181
unable to upload Error
at Request.extractError (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/services/s3.js:257:35)
at Request.callListeners (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/sequential_executor.js:114:20)
at Request.callListeners (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/sequential_executor.js:115:16)
at Request.emit (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/sequential_executor.js:81:10)
at Request.emit (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:578:14)
at Request.transition (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:12:12)
at AcceptorStateMachine.runTo (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/state_machine.js:28:10
at Request. (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:28:9)
at Request. (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:580:12)
progress 12181 0 12181
progress 12181 12181 12181
unable to upload Error
at Request.extractError (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/services/s3.js:257:35)
at Request.callListeners (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/sequential_executor.js:114:20)
at Request.callListeners (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/sequential_executor.js:115:16)
at Request.emit (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/sequential_executor.js:81:10)
at Request.emit (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:578:14)
at Request.transition (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:12:12)
at AcceptorStateMachine.runTo (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/state_machine.js:28:10
at Request. (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:28:9)
at Request. (/Users/Desktop/newsite/node_modules/s3/node_modules/aws-sdk/lib/request.js:580:12)
I guess we need more code. I'm recommending using the aws-sdk. Works perfectly, easy to upload, sync, trigger jobs and so on.
But to answer your question and assuming the file you're trying to upload is located in the same directory as your script your code would look something like this:
var params = {
localFile: __dirname + '/image.png',
s3Params: {
Bucket: 'newstie.com',
Key: '/newsite/image'
}
};
Related
I get the error message when I want to upload a file size > 1MB to S3 Bucket in my company's network:
(If file size < 1MB, upload file working fine)
500: null
at Request.extractError (...\node_modules\aws-sdk\lib\services\s3.js:711:35)
at Request.callListeners (...\node_modules\aws-sdk\lib\sequential_executor.js:106:20)
at Request.emit (...\node_modules\aws-sdk\lib\sequential_executor.js:78:10)
at Request.emit (...\node_modules\aws-sdk\lib\request.js:686:14)
at Request.transition (...\node_modules\aws-sdk\lib\request.js:22:10)
at AcceptorStateMachine.runTo (...\node_modules\aws-sdk\lib\state_machine.js:14:12)
at ...\node_modules\aws-sdk\lib\state_machine.js:26:10
at Request.<anonymous> (...\node_modules\aws-sdk\lib\request.js:38:9)
at Request.<anonymous> (...\node_modules\aws-sdk\lib\request.js:688:12)
at Request.callListeners (...\node_modules\aws-sdk\lib\sequential_executor.js:116:18) {
code: 500,
region: null,
time: 2022-11-22T09:07:55.279Z,
requestId: null,
extendedRequestId: undefined,
cfId: undefined,
statusCode: 500,
retryable: true
}
And I found this error is simlar to this issue (https://github.com/localstack/localstack/issues/1410),
but I still get the same error after I try to set s3ForcePathStyle to true while creating s3 object.
Here is my code:
const s3 = new AWS.S3({apiVersion: '2006-03-01', s3ForcePathStyle: true});
let result = await s3.upload( {Bucket: bucket_name, Key: file_path, Body: request.files.buffer}).promise();
// using upload and putObject api return same error
console.log(result);
But if I run the code in my home, there is no error, and file upload succeed.
In my code, I set the NODE_TLS_REJECT_UNAUTHORIZED diabled to avoid ssl certificate problem, but still not working
process.env['NODE_TLS_REJECT_UNAUTHORIZED'] = 0;
I also try to use AWS CLI CMD (aws s3api put-object ...) in my company, upload the file size > 1MB is working fine!!
I don't understand why I cannot using node.js code to upload the file size > 1MB in my company, doesn't AWS CLI and npm aws-sdk library using same protocol?
I guess this error maybe related to my company's environment, but AWS CLI is working fine. So, I think it probably could be fixed the problem in my code, hopping someone can help, thank you so much.
Finally, I try to set enpoint and remove s3ForcePathStyle while careting s3 object.
It's working now!
const s3 = new AWS.S3({apiVersion: '2006-03-01', endpoint: 's3.amazonaws.com'});
But I still don't know why?
I have permissions, and can create Databases/Tables in the console just fine. but when i make code using the aws-sdk it fails with an InternalServerException: Internal Server Error. Is this only supported in the v3 api?
const params = {
DatabaseName,
Tags: getDefaultTags(customer,username)
};
console.log(params);
const promise = writeClient.createDatabase(params).promise();
to repeat, this is not a permissions issue, I have the region specified, I just get an internal error if I try to use the SDK:
InternalServerException: Internal Server Error
at Request.extractError (<<coderoot>>\node_modules\aws-sdk\lib\protocol\json.js:52:27)
at Request.callListeners (<<coderoot>>\node_modules\aws-sdk\lib\sequential_executor.js:106:20)
at Request.emit (<<coderoot>>\node_modules\aws-sdk\lib\sequential_executor.js:78:10)
at Request.emit (<<coderoot>>\node_modules\aws-sdk\lib\request.js:686:14)
at Request.transition (<<coderoot>>\node_modules\aws-sdk\lib\request.js:22:10)
at AcceptorStateMachine.runTo (<<coderoot>>\node_modules\aws-sdk\lib\state_machine.js:14:12)
at <<coderoot>>\node_modules\aws-sdk\lib\state_machine.js:26:10
at Request.<anonymous> (<<coderoot>>\node_modules\aws-sdk\lib\request.js:38:9)
at Request.<anonymous> (<<coderoot>>\node_modules\aws-sdk\lib\request.js:688:12)
at Request.callListeners (<<coderoot>>\node_modules\aws-sdk\lib\sequential_executor.js:116:18)
if I use the v3, I get the same error eventually, this example is creating a table, but it's the same idea:
InternalServerException: Internal Server Error
at deserializeAws_json1_0InternalServerExceptionResponse (<<coderoot>>\node_modules\#aws-sdk\client-timestream-write\dist-cjs\protocols\Aws_json1_0.js:899:23)
at deserializeAws_json1_0CreateTableCommandError (<<coderoot>>\node_modules\#aws-sdk\client-timestream-write\dist-cjs\protocols\Aws_json1_0.js:239:25)
at processTicksAndRejections (node:internal/process/task_queues:96:5)
at async <<coderoot>>\node_modules\#aws-sdk\middleware-serde\dist-cjs\deserializerMiddleware.js:7:24
at async <<coderoot>>\node_modules\#aws-sdk\middleware-signing\dist-cjs\middleware.js:13:20
at async StandardRetryStrategy.retry (<<coderoot>>\node_modules\#aws-sdk\middleware-retry\dist-cjs\StandardRetryStrategy.js:51:46)
at async <<coderoot>>\node_modules\#aws-sdk\middleware-logger\dist-cjs\loggerMiddleware.js:6:22
at async Object.createTable (<<coderoot>>\timestream.db.js:228:22)
at async handleCustomerCreation (<<coderoot>>\router.js:60:21)
at async router (<<coderoot>>\router.js:266:13)
I have a lambda function within AWS which is triggered by an S3 upload event. When uploading a file containing whitespace to the S3 bucket I see
"AccessDenied: Access Denied",
" at Request.extractError (/var/task/node_modules/aws-sdk/lib/services/s3.js:700:35)",
" at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:106:20)",
" at Request.emit (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:78:10)",
" at Request.emit (/var/task/node_modules/aws-sdk/lib/request.js:688:14)",
" at Request.transition (/var/task/node_modules/aws-sdk/lib/request.js:22:10)",
" at AcceptorStateMachine.runTo (/var/task/node_modules/aws-sdk/lib/state_machine.js:14:12)",
" at /var/task/node_modules/aws-sdk/lib/state_machine.js:26:10",
" at Request.<anonymous> (/var/task/node_modules/aws-sdk/lib/request.js:38:9)",
" at Request.<anonymous> (/var/task/node_modules/aws-sdk/lib/request.js:690:12)",
" at Request.callListeners (/var/task/node_modules/aws-sdk/lib/sequential_executor.js:116:18)"
I am uploading a file hello world.csv
console.log('File path: ' + submission.s3.object.key);
gives the below:
File path: hello+world.csv
My lambda is using const AWS = require('aws-sdk'); to read the content of the event from the S3 bucket.
I have seen similar issues raised previously but they seem to have been resolved.
How can I reference the correct path to my file within my lambda?
you need to escape them spaces
var path = '/path/with spaces/in/them/';
var escapedPath = path.replace(/(\s)/, "\\ ");
on yow case it might look like this
var path = submission.s3.object.key;
var escapedPath = path.replace(/(\s)/, "\\ ");
console.log('File path: ' + espacedPath);
I am trying to get twit (https://www.npmjs.com/package/twit) to stream live tweets from specific twitter users.
The code I have is
var stream = T.stream('statuses/filter', { follow: ['nodejs'] })
stream.on('tweet', function (tweet) {
console.log(tweet.text)
})
but it gives the following error
Error: Bad Twitter streaming request: 406
at Object.exports.makeTwitError (/Users/simoncarr/dev/node/nodeletetweet/scripts/get_mp_data/node_modules/twit/lib/helpers.js:74:13)
at Request.<anonymous> (/Users/simoncarr/dev/node/nodeletetweet/scripts/get_mp_data/node_modules/twit/lib/streaming-api-connection.js:96:29)
at Request.emit (events.js:187:15)
at Gunzip.<anonymous> (/Users/simoncarr/dev/node/nodeletetweet/scripts/get_mp_data/node_modules/request/request.js:1083:12)
at Object.onceWrapper (events.js:273:13)
at Gunzip.emit (events.js:187:15)
at endReadableNT (_stream_readable.js:1094:12)
at process._tickCallback (internal/process/next_tick.js:63:19)
Emitted 'error' event at:
at Request.<anonymous> (/Users/simoncarr/dev/node/nodeletetweet/scripts/get_mp_data/node_modules/twit/lib/streaming-api-connection.js:99:14)
at Request.emit (events.js:187:15)
[... lines matching original stack trace ...]
at process._tickCallback (internal/process/next_tick.js:63:19)
simply tracking words in a tweet works just fine, for instance the following code works OK.
var stream = T.stream('statuses/filter', { track: ['apples'] })
stream.on('tweet', function (tweet) {
console.log(tweet.text)
})
The twitter documentation suggests that I should be able to simply replace track with follow.
https://developer.twitter.com/en/docs/tweets/filter-realtime/api-reference/post-statuses-filter.html
You need to use the numeric user ID, not the screen name, in the follow parameter. You can use the users/show endpoint to find the user ID for a given screen name.
On BotFramework (NodeJS), I was trying to replicate the demo available at https://learn.microsoft.com/en-us/bot-framework/nodejs/bot-builder-nodejs-send-receive-attachments . It actually works well.
Code in case ms article change:
// Create your bot with a function to receive messages from the user
var bot = new builder.UniversalBot(connector, function (session) {
var msg = session.message;
if (msg.attachments && msg.attachments.length > 0) {
// Echo back attachment
var attachment = msg.attachments[0];
session.send({
text: "You sent:",
attachments: [
{
contentType: attachment.contentType,
contentUrl: attachment.contentUrl,
name: attachment.name
}
]
});
} else {
// Echo back users text
session.send("You said: %s", session.message.text);
}
});
However, the problem I am facing is that when I do a call from Skype (Normal), I receive an error message:
2017-12-07T02:16:15.815Z Error: POST to 'https://smba.trafficmanager.net/apis/v3/conversations/<My Conversation>/activities' failed: [400] Bad Request
at Request._callback (/app/node_modules/botbuilder/lib/bots/ChatConnector.js:545:46)
at Request.self.callback (/app/node_modules/request/request.js:186:22)
at emitTwo (events.js:126:13)
at Request.emit (events.js:214:7)
at Request.<anonymous> (/app/node_modules/request/request.js:1163:10)
at emitOne (events.js:116:13)
at Request.emit (events.js:211:7)
at IncomingMessage.<anonymous> (/app/node_modules/request/request.js:1085:12)
at Object.onceWrapper (events.js:313:30)
at emitNone (events.js:111:20)
Any ideas?
[Update: It happens only when I create the attachment response. So I expect that's where I have an issue]
Actually the code on the MS Website is not up-to-date (in a way).
If I follow the code visible at : https://github.com/Microsoft/BotBuilder-Samples/tree/master/Node/core-ReceiveAttachment
By example, I can receive the attachment and save it somewhere on a public folder. Once completed, then I can send back the "public" URL as attachment and then it works.