I have Lambda setup locally using Docker and Sam. I can hit an endpoint and run the Lambda method locally, but if i want to test the below code, I have to actually deploy the Lambda since I'm not sure how to get the aws-sdk to work in a local environment.
const payload = JSON.stringify({
"bucket": process.env.AWS_S3_ENV_BUCKET,
"region": process.env.AWS_REGION,
"folder": 'somePath/',
"files": ['somefile.jpg', 'anotherfile.jpg'],
"zipFileName": 'zipZippedFile.zip'
})
const params = {
FunctionName: 'zippidyDoDah',
Payload: payload
}
global.Lambda.invoke(params, function (error, data) {
console.log('error: ', error)
console.log('data: ', data)
})
Does anyone have any insight on this?
If you install the AWS Command Line Interface and run an aws configure you can enter the access key and secret key of the user that you want this code to be executed as. These credentials are stored in ~/.aws/credentials. You should be able to inject the AWS CLI and these credentials into your docker container and (assuming they are your [default]) they should get picked up automatically by your process. You should read about AWS CLI Profiles too.
Related
I'm trying to connect to my dynamoDB table from inside a React js app. I have AWS credentials set up locally. When I run my app, I get the following error on Chrome Devtools: "Error: Credential is missing".
Oddly, if I run the AWS example found below using pretty much the same code via node on terminal, it works fine. https://github.com/awsdocs/aws-doc-sdk-examples/blob/main/javascriptv3/example_code/dynamodb/src/partiQL_examples/src/partiql_getItem.js
To run the AWS example, I created a new mjs file inside my react SRC folder, so it should have all the same access as the React app, right? No credentials are explicitly added in the mjs file or the react app.
Why doesn't the React environment have access to the credentials? I've tried both ~/.aws/credentials and environment variables. The AWS SDK seems to say that it should just work for Node. Any thoughts?
import { DynamoDBClient, ExecuteStatementCommand} from '#aws-sdk/client-dynamodb';
function App() {
const dynamoDB = new DynamoDBClient({ region : "us-west-2"});
async function loadFromCloud () {
const command = new ExecuteStatementCommand({
Statement: `select * from TableX`
});
try {
const data = await dynamoDB.send(command);
console.log(data);
} catch (error) {
console.log(error);
}
}
Make sure that you have AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY specified as environment variables and that you also specify AWS_SDK_LOAD_CONFIG=1
However, your React app is run in the browser. It should only work in a node.js app.
I am trying to download a file from an EC2 instance and store it temporarily in the tmp folder inside AWS Lambda. This is what I have tried:
let Client = require('ssh2-sftp-client');
let sftp = new Client();
sftp.connect({
host: host,
username: user,
privateKey : fs.readFileSync(pemfile)
}).then(() => {
return sftp.get('/config/test.txt' , fs.createWriteStream('/tmp/test.txt'))
}).then(() => {
sftp.end();
}).catch(err => {
console.error(err.message);
});
The function runs without generating an error but nothing is written to the destination file. What am I doing wrong here and how could I debug this? Also is there a better way of doing this altogether?
This is not the cloud way to do it IMO. Create a S3 bucket, and create proper Lambda execution role for the lambda function to be able to read from the bucket. Also, create a role for the EC2 instance to be able to write to the same S3 bucket. Using S3 API from both sides, the lambda function and the EC2, should be enough to share the file.
Think about this approach: you decouple your solution from a VPC and region perspective. Also, since the lambda only needs to access S3, you save a ENI (elastic network interface) resources, so you are not using your VPC private ips. These are just advantages that may not care in your case, but it is good to be aware of them.
I'm building Lambda function for CloudFront which checks if request has cookies, if not then forwards to login page. I need to customize response header location based on environment - for each env that will be different.
Initially I tried with environment variables but I got an error during deployment:
InvalidLambdaFunctionAssociation: The function cannot have environment variables
So I switched to use aws-sdk with SSM ssm.getParameter but after zipping lambda archive with aws-sdk and one more depedency it's around 13 MB. The limit for Lambda#Edge functions is 1 MB.
I'm wondering would be the best way to approach that. Generate file with environment variables on each Lambda build and require it in index.js?
Use SSM but don't include AWS SDK in your Lambda function. Lambda documentation says that the AWS SDK is included in the Lambda runtime.
To test this, I created a new Node.js 12 Lambda function from scratch in the Lambda console & replaced its existing code with this:
const AWS = require('aws-sdk');
const SSM = new AWS.SSM();
exports.handler = async() => {
return {
statusCode: 200,
body: await SSM.getParameter({ Name: 'my-param' }).promise(),
};
};
This works! Downloading the deployment package of this function from the Lambda console showed that it's just 276 bytes in size. I then deployed this to Lambda#Edge & that worked too!
I am trying to publish message from AWS Lamba using Nodejs to AWS IoT .
I have zipped the project and uploaded on to the AWS IoT
below is the code snippet
var awsIot = require('aws-iot-device-sdk');
var device = awsIot.device({
keyPath: 'keyfilepath',
certPath: 'pem file path',
caPath: 'root-CA.crt',
clientId: 'iotTest7526532135',
host: 'host id'
});
device
.on('connect', function() {
console.log('connect');
device.subscribe('topic_3');
device.publish('topic_1', JSON.stringify({ message_id:23,Message:'HelloWorld'}));
});
device
.on('message', function(topic, payload) {
console.log('message', topic, payload.toString());
});
I am getting below error
"errorMessage": "Cannot find module 'aws-iot-device-sdk'",
I know that iot sdk is missing, I am not sure how to install it on AWS Lambda.
Any suggestions will be really helpful
I would highly recommend not using the aws-iot-device-sdk to interact with AWS Iot from a Lambda function.
You need to understand there are 2 javascript APIs that you can use to access AWS IoT
The AWS IOT Device SDKs for javascript, using MQTT as a protocol and x509 certificates for authentication. These are typically used in devices running outside of your AWS cloud.
The AWS SDK for javascript, using HTTP as a protocol, and IAM roles (among other things) for authentication. These SDKs are typically run inside your AWS cloud (such as a Lambda)
There are multiple reasons why you should opt for the HTTP based SDK :
The aws-iot-device-sdk is specifically targeted toward devices "living" outside of Aws (IoT gateways / devices in the field) that need to remotely connect.
The Device SDK uses MQTT and x509 certificates to interact with AWS IoT. There is no need to configure x509 securities in your lambda. A Lambda running on your AWS account can easily access AWS IoT through an IAM role, so if your lambda function is configured with the correct role, you can use the standard aws sdks.
A protocol like MQTT (or any pub/sub protocol) doesn't match well with a serverless lambda architecture. In your lambda function you are subscribing to a topic, something that you don't typically do in a short-lived lambda function.
The AWS SDK for NodeJS is provided to your lambda out of the box.There's no need to require or package additional node modules)
Your code can become as simple as this (notice that there are no credentials or extra node modules required) :
var AWS = require('aws-sdk');
var iotdata = new AWS.IotData({endpoint:"yourendpoint.iot.eu-central-1.amazonaws.com"});
exports.handler = function(event, context, callback) {
console.log("found iotdata",iotdata);
var params = {
topic: 'topic/test',
payload: 'blah',
qos: 0
};
iotdata.publish(params, function(err, data){
if(err){
console.log("Error occured : ",err);
}
else{
console.log("success.....");
}
});
callback();
};
When you zip your project, you also zip ./node_modules folder. So as long as aws-iot-device-sdk is there (along with all the dependencies), your Lambda will be fine.
So all you need is:
npm install aws-iot-device-sdk
zip ...
You'll need to make sure you're uploading your package.json file as well, which should have a dependency requirement for aws-iot-device-sdk
you can add the package to your package.json by running
npm -i --save aws-iot-device-sdk
from your project directory.
If you are just going to publish the data to IoT topic, you are better of using http protocol instead of MQTT connection.
And using HTTP connection doesnt require aws-iot-device-sdk package. AWS default SDK has iotdata. And iotdata will provide the http connection to the device.
iotHTTPConnection= AWS.IotData({endpoint: Your_END_POINT});
iotHTTPConnection.publish(params) will publish the data using http without iot specific sdk.
https://docs.aws.amazon.com/iot/latest/apireference/API_iotdata_Publish.html
There is no http subscribe though.
Add something like the below in your package.json file. then run npm install. This will create a node_modules folder. Now zip it and upload it once again.
"aws-iot-device-sdk": "^2.1.0"
There is not a good practice adding iot-sdk library into your Lambda function. Thats also the reason it doesn't come pre-installed into the Lambda services. It is clearly a library to connect devices to AWS cloud.
In your case as they mentioned above, the pre-installed aws-sdk is all you need.
Though you can't subscribe to a topic for the reason that the protocol to publish a message is HTTP.
But usually isn't a reason for that too.
I need to publish data from aws lambda through mqtt protocol using aws iot. i have created a lambda function with node.js code. like this
exports.handler = (event, context, callback) => {
var awsIot = require('aws-iot-device-sdk');
var device = awsIot.device({
keyPath: 'samplepath/test.pem.key',
certPath: 'samplepath/test.crt',
caPath: 'samplepath',
clientId: 'sampleId',
region: 'us-east-1'
});
device
.on('connect', function () {
console.log('connected');
device.publish('test_topic', JSON.stringify({ "test_name": "hello", "test_value": 1001 }));
console.log('published successfully');
callback(null, 'item added');
});
}
I got mqtt message on subscriber. but lambda produce error message like this
Task timed out after 10.00 seconds
I have used context.succeed() instead of callback, lambda is exited properly. i cant get any messages on subscriber.
In both cases console prints published successfully message properly.
What is the issue related with my publishing code?
I understand my lambda function is timing out when connecting to AWS
IoT. About the sdk we are using, the aws-iot-device-sdk is designed to
use inside of an embedded device. When we are using a Lambda function
or trying to publish in a computer, the best practice is use the
aws-sdk. Using the aws-sdk we don't need to use the certificates to
publish in the AWS IoT, we just use the AWS credentials to do this.
Also, with aws-sdk we can do administrative tasks in the IoT, we can
create a thing, create a certificate, etc.
Coming to my code, the reason the function does not end and times out
is because the callback must be waiting for an asynchronous call to
finish execution, which I assume is being help up by the connection
being maintained from the function to IoT. The reason
context.succeed() exited properly but we did not get any messages must
be because context.succeed does not wait for our async calls to finish
execution.
Make sure you disconnect from the device after you published the message, otherwise Lambda will wait while the connection stays alive (see http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html, look for callbackWaitsForEmptyEventLoop).
To disconnect when done, simply change callback(null, 'item added'); to
device.end((err) => {
callback(err, "item added");
});