I am trying to publish message from AWS Lamba using Nodejs to AWS IoT .
I have zipped the project and uploaded on to the AWS IoT
below is the code snippet
var awsIot = require('aws-iot-device-sdk');
var device = awsIot.device({
keyPath: 'keyfilepath',
certPath: 'pem file path',
caPath: 'root-CA.crt',
clientId: 'iotTest7526532135',
host: 'host id'
});
device
.on('connect', function() {
console.log('connect');
device.subscribe('topic_3');
device.publish('topic_1', JSON.stringify({ message_id:23,Message:'HelloWorld'}));
});
device
.on('message', function(topic, payload) {
console.log('message', topic, payload.toString());
});
I am getting below error
"errorMessage": "Cannot find module 'aws-iot-device-sdk'",
I know that iot sdk is missing, I am not sure how to install it on AWS Lambda.
Any suggestions will be really helpful
I would highly recommend not using the aws-iot-device-sdk to interact with AWS Iot from a Lambda function.
You need to understand there are 2 javascript APIs that you can use to access AWS IoT
The AWS IOT Device SDKs for javascript, using MQTT as a protocol and x509 certificates for authentication. These are typically used in devices running outside of your AWS cloud.
The AWS SDK for javascript, using HTTP as a protocol, and IAM roles (among other things) for authentication. These SDKs are typically run inside your AWS cloud (such as a Lambda)
There are multiple reasons why you should opt for the HTTP based SDK :
The aws-iot-device-sdk is specifically targeted toward devices "living" outside of Aws (IoT gateways / devices in the field) that need to remotely connect.
The Device SDK uses MQTT and x509 certificates to interact with AWS IoT. There is no need to configure x509 securities in your lambda. A Lambda running on your AWS account can easily access AWS IoT through an IAM role, so if your lambda function is configured with the correct role, you can use the standard aws sdks.
A protocol like MQTT (or any pub/sub protocol) doesn't match well with a serverless lambda architecture. In your lambda function you are subscribing to a topic, something that you don't typically do in a short-lived lambda function.
The AWS SDK for NodeJS is provided to your lambda out of the box.There's no need to require or package additional node modules)
Your code can become as simple as this (notice that there are no credentials or extra node modules required) :
var AWS = require('aws-sdk');
var iotdata = new AWS.IotData({endpoint:"yourendpoint.iot.eu-central-1.amazonaws.com"});
exports.handler = function(event, context, callback) {
console.log("found iotdata",iotdata);
var params = {
topic: 'topic/test',
payload: 'blah',
qos: 0
};
iotdata.publish(params, function(err, data){
if(err){
console.log("Error occured : ",err);
}
else{
console.log("success.....");
}
});
callback();
};
When you zip your project, you also zip ./node_modules folder. So as long as aws-iot-device-sdk is there (along with all the dependencies), your Lambda will be fine.
So all you need is:
npm install aws-iot-device-sdk
zip ...
You'll need to make sure you're uploading your package.json file as well, which should have a dependency requirement for aws-iot-device-sdk
you can add the package to your package.json by running
npm -i --save aws-iot-device-sdk
from your project directory.
If you are just going to publish the data to IoT topic, you are better of using http protocol instead of MQTT connection.
And using HTTP connection doesnt require aws-iot-device-sdk package. AWS default SDK has iotdata. And iotdata will provide the http connection to the device.
iotHTTPConnection= AWS.IotData({endpoint: Your_END_POINT});
iotHTTPConnection.publish(params) will publish the data using http without iot specific sdk.
https://docs.aws.amazon.com/iot/latest/apireference/API_iotdata_Publish.html
There is no http subscribe though.
Add something like the below in your package.json file. then run npm install. This will create a node_modules folder. Now zip it and upload it once again.
"aws-iot-device-sdk": "^2.1.0"
There is not a good practice adding iot-sdk library into your Lambda function. Thats also the reason it doesn't come pre-installed into the Lambda services. It is clearly a library to connect devices to AWS cloud.
In your case as they mentioned above, the pre-installed aws-sdk is all you need.
Though you can't subscribe to a topic for the reason that the protocol to publish a message is HTTP.
But usually isn't a reason for that too.
Related
I am using serveless + aws + node.js.
I have a lambda calling another lambda. I can't get to run the lot locally.
I can invoke both lambdas locally with 'serverless invoke local -f ...' BUT
the caller one comes back with:
{"message":"Function not found: arn:aws:lambda:eu-west-1:5701xxxxxxxxxx:function:the-right-function-name"}
as if the caller function invoked the callee on AWS and not locally.
Is there anyway to do stay local and if yes, what may I be missing?
You can achieve that with this plugin. There is a feature of AWS SDK for Lambda that allows you to override the API endpoint of Lambda service. Therefore you can set it to localhost.
const AWS = require('aws-sdk');
const endpoint = process.env.SERVERLESS_SIMULATE ?
process.env.SERVERLESS_SIMULATE_LAMBDA_ENDPOINT :
undefined
const lambda = new AWS.Lambda({ endpoint })
For more details, refer to the plugin's readme. Also there is a nice article about that.
We are trying to migrate our zip microservice from regular application in nodejs Express to AWS API Gateway integrated with AWS Lambda.
Our current application sends request to our API, gets list of attachments and then visits those attachments and pipes their content back to user in form of zip archive. It looks something like this:
module.exports = function requestHandler(req, res) {
//...
//irrelevant code
//...
return getFileList(params, token).then(function(fileList) {
const filename = `attachments_${params.id}`;
res.set('Content-Disposition', `attachment; filename=${filename}.zip`);
streamFiles(fileList, filename).pipe(res); <-- here magic happens
}, function(error) {
errors[error](req, res);
});
};
I have managed to do everything except the part where I have to stream content out of Lambda function.
I think one of possible solutions is to use aws-serverless-express, but I'd like a more elegant solution.
Anyone has any ideas? Is it even possible to stream out of Lambda?
Unfortunately lambda does not support streams as events or return values. (It's hard to find it mentioned explicitly in the documentation, except by noting how invocation and contexts/callbacks are described in the working documentation).
In the case of your example, you will have to await streamFiles and then return the completed result.
(aws-serverless-express would not help here, if you check the code they wait for your pipe to finish before returning: https://github.com/awslabs/aws-serverless-express/blob/master/src/index.js#L68)
n.b. There's a nuance here that a lot of the language SDK's support streaming for requests/responses, however this means connecting to the stream transport, e.g. the stream downloading the complete response from the lambda, not listening to a stream emitted from the lambda.
Had the same issue, now sure how you can do stream/pipe via the native lambda + API Gateway directly... but it's technically possible.
We used Serverless Framework and were able to use XX.pipe(res) using this starter kit (https://github.com/serverless/examples/tree/v3/aws-node-express-dynamodb-api)
What's interesting is that this just wraps over native lambda + API Gateway so, technically it is possible as they have done it.
Good luck
I have a self-hosted parse-server in AWS EC2. I want to update my database when I receive POSTnotifications from Apple. For that, I created a cloud function, but since Apple asks for a urlto send notifications, I'm not sure how to make my cloud function directly accessible via url or if I need to create an endpoint somewhere (AWS) to receive the notification from Apple and then make a new httpRequest or curl to my cloud function.
I'm looking for any directions or services (AWS) on how to perform this.
I don't think you need to do anything in AWS. You just need to add the cloudFunction to your main.js
As an example, here is an endPoint called add3NumbersTogether . I put this in my main.js file and then I can call this code from iOs (or another client). In iOs I use the Parse iOs SDK to make calls
Parse.Cloud.define("add3NumbersTogether", function(request, response) {
response.success( {request.params.num1 + request.params.num2 + request.params.num3});
});
I am working on a server for an API that I am developing that is being built using node.js and requires the use of AWS Cognito. Before this, we developed a working version of this application on client side that used the AWS SDK, and I am currently trying to translate that functionality over to the server side. I am struggling to find a good way of doing this and have a few specific questions that, if answered, would probably allow me to get the implementation I am looking for. Basically, I want to know what the AWSCognito object is and how to access it like I see in the following line of code.
var poolData = {
UserPoolId : 'us-east-###########',
ClientId : '########################'
};
var userPool = new AWSCognito.CognitoIdentityServiceProvider.CognitoUserPool(poolData);
Currently my code is set up using the complete AWS SDK, installed using [a] and accessed in my code using [b].
[a] npm install --save aws-sdk
[b] var AWS = require('aws-sdk');
Is there a way to access this AWSCognito object from my AWS object? If so, how do I do that? If not, how do I get access to it/is it even possible to access it?
Edit: In broad summary, I just want access to the following functions and have no clue how to access them from a node.js server-
userPool.signUp(username, password, attributes, callback)
cognitoUser.confirmPassword(verification, newPassword, {})
cognitoUser.forgotPassword({})
cognitoUser.authenticateUser(authenticationDetails, {})
userPool.getCurrentUser()
cognitoUser.getSession(callback)
cognitoUser.confirmRegistration(verification, bool, callback)
cognitoUser.updateAttributes(attributeList, callback)
cognitoUser.getUserAttributes(callback)
Edit: Update 1
The AWSCognito object is something being set to a global variable in the window by the AWS Cognito SDK. This is a process that only works client-side (yes I know there are hacky solutions to emulate a window on my server, I would prefer not to use these). Is there an equivalent object on server side that I can access and call functions from?
You can use Cognito in a Node.JS environment, but doing so with the AWS SDK for JavaScript is a bit different from doing so with the AWS Cognito SDK. Based on the names of the functions you want to access, you should take a look at the Cognito Identity Provider Service. Operations that start with admin are meant to be called from a server using AWS credentials.
I need to publish data from aws lambda through mqtt protocol using aws iot. i have created a lambda function with node.js code. like this
exports.handler = (event, context, callback) => {
var awsIot = require('aws-iot-device-sdk');
var device = awsIot.device({
keyPath: 'samplepath/test.pem.key',
certPath: 'samplepath/test.crt',
caPath: 'samplepath',
clientId: 'sampleId',
region: 'us-east-1'
});
device
.on('connect', function () {
console.log('connected');
device.publish('test_topic', JSON.stringify({ "test_name": "hello", "test_value": 1001 }));
console.log('published successfully');
callback(null, 'item added');
});
}
I got mqtt message on subscriber. but lambda produce error message like this
Task timed out after 10.00 seconds
I have used context.succeed() instead of callback, lambda is exited properly. i cant get any messages on subscriber.
In both cases console prints published successfully message properly.
What is the issue related with my publishing code?
I understand my lambda function is timing out when connecting to AWS
IoT. About the sdk we are using, the aws-iot-device-sdk is designed to
use inside of an embedded device. When we are using a Lambda function
or trying to publish in a computer, the best practice is use the
aws-sdk. Using the aws-sdk we don't need to use the certificates to
publish in the AWS IoT, we just use the AWS credentials to do this.
Also, with aws-sdk we can do administrative tasks in the IoT, we can
create a thing, create a certificate, etc.
Coming to my code, the reason the function does not end and times out
is because the callback must be waiting for an asynchronous call to
finish execution, which I assume is being help up by the connection
being maintained from the function to IoT. The reason
context.succeed() exited properly but we did not get any messages must
be because context.succeed does not wait for our async calls to finish
execution.
Make sure you disconnect from the device after you published the message, otherwise Lambda will wait while the connection stays alive (see http://docs.aws.amazon.com/lambda/latest/dg/nodejs-prog-model-context.html, look for callbackWaitsForEmptyEventLoop).
To disconnect when done, simply change callback(null, 'item added'); to
device.end((err) => {
callback(err, "item added");
});