In AWS Lambda people can create a node.js function and trigger it through events, for example a message, etc..
I wonder how this can work 'under the hood' and how to put something like this together in Haskell. The uploaded functions are basically single function libraries without any main function.
Means on the CLI or via an API you can call any of your functions by name (and with the required input) and you get the output defined by the function signature--or, alternatively an error of course.
Would it be possible to do this in Haskell?
To clarify: what I want to do is for example loading a number of different single function Haskell libraries on a Haskell platform or any other execution context that is in my data center and execute / call them by name via the CLI or an API just in the same way AWS Lambda works with node.js functions.
If you want reproduce the same functionality (functions as services) exists a lot of technologies you can use (soap, rpc, rest, ...). If you unknown it I suggest to you read about.
My ever favorite is SOAP but is so unpopular and Haskell support is limited soap (see related question). SOAP (as others) provide exactly you want.
But you must to concrete your real problem to select the best technology.
If you are looking for distribute your own Haskell code Cloud Haskell may be a good starting point.
If you need more like "web server" then take a look to sodium, elm, ... in a Javascript style or servant (generate client code to some languages).
Anyway, even AWS Lambda require support for each language and you should not expect to find one ubiquitous technology (like HTTP) for RPC.
You may want to check out the "serverless for haskell" framework: http://qmu.li
Not only you can run individual haskell functions as Lambda functions with it but you can also describe your whole AWS infrastructure in haskell. (vs. doing it in CloudFormation json/yaml template), build it all locally and easily deploy to AWS.
Related
I have a problem facing to invoke an AWS.Lambda function from within another lambda function in local machine by using sls offline
Could anyone can help me on this
I aleady try with child_process(spawn), but i failed
Thanks for your reponse
The best way to manage workflows that involve mutiple Lambda functions is to use Step Functions. Then you can use the serverless-step-functions module combined with serverless-step-functions-offline for local development.
Easiest way to invoke another lambda function is to use the AWS SDK.
Note that lambda functions are not meant to be used just like normal functions in code. Normally functions can be used to factor your code to smaller parts, and to make it reusable in other cases. While you should probably still write the actual code this way, the you might want to deploy only larger entities as lambda functions. If you have a lot of code shared between the lambda functions you can consider using Layers to share the common code.
If having multiple lambda functions can't be avoided, step functions are probably the way to build that structure.
I'm looking to build a REST API using Node and Express and I'd like to provide documentation with it. I don't want to craft this by hand and it appears that there are solutions available in the forms of Swagger, RAML and Api Blueprint/Apiary.
What I'd really like is to have the documentation auto-generate from the API code as is possible in .NET land with Swashbuckle or the Microsoft provided solution but they're made possible by strong typing and reflection.
For the JS world it seems like the correct option is to use the Swagger/RAML/Api Blueprint markup to define the API and then generate the documentation and scaffold the server from that. The former seems straightforward but I'm less sure about the latter. What I've seen of the server code generation for all of these options seem very limited. There needs to be some way to separate the auto-generated code from the manual code so that the definition can be updated easily and I've seen no sign or discussion on that. It doesn't seem like an insurmountable problem (I'm much more familiar with .NET than JS so I could easily be missing something) and there is mention of this issue and solutions being worked on in a previous Stack Overflow question from over a year ago.
Can anyone tell me if I'm missing/misunderstanding anything and if any solution for the above problem exists?
the initial version of swagger-node-express did just this--you would define some metadata from the routes, models, etc., and the documentation would auto-generate from it. Given how dynamic javascript is, this became a bit cumbersome for many to use, as it required you to keep the metadata up-to-date against the models in a somewhat decoupled manner.
Fast forward and the latest swagger-node project takes an alternative approach which can be considered in-line with "generating documentation from code" in a sense. In this project (and swagger-inflector for java, and connexion for python) take the approach that the swagger specification is the DSL for the api, and the routing logic is handled by what is defined in the swagger document. From there, you simply implement the controllers.
If you treat the swagger specification "like code" then this is a very efficient way to go--the documentation can literally never be out of date, since it is used to construct all routes, validate all input variables, and connect the API to your business layer.
While true code generation, such as what is available from the swagger-codegen project can be extremely effective, it does require some clever integration with your code after you initially construct the server. That consideration is completely removed from the workflow with the three projects above.
I hope this is helpful!
My experience with APIs and dynamic languages is that the accent is on verification instead of code generation.
For example, if using a compiled language I generate artifacts from the API spec and use that to enforce correctness. Round tripping is supported via the generation of interfaces instead of concrete classes.
With a dynamic language, the spec is used at test time to guarantee that both all the defined API is test covered and that the responses are conform to the spec (I tend to not validate requests because of Postel's law, but it is possible too).
I am very new to node.js and I can not seem to find a definition anywhere as to what node.js bindings are. I have seen this term used in slides and nodejs talks but it was never clearly explained. Can anyone help clarify this concept for me? I have attached a picture of what I am referring to.
Rather than understanding what node.js bindings are, it is more useful to understand what "bindings" are in the first place.
Let's say you are writing a web application where a node.js (JavaScript) backend:
receives requests from clients,
conducts queries to databases,
sorts the query results and finally
returns the results to the client.
Now normally you would write all the code yourself. However, you know that there is an excellent sorting library that can take care of step 3 (i.e. sorting query results). The only problem is that the library is written in a system programming language such as C/C++ whereas your code is written in JavaScript. Normally you can't use that library in your code because they are in different programming languages, but with bindings, you can.
Bindings basically are libraries that "bind" two different programming languages so that code written in one language can be used in code written in another library. With the presence of bindings, you don't have to write all the code again just because they are in different languages. Another motivation for bindings is that you can benefit from the advantages of different programming languages. For example, C/C++ are much faster than JavaScript. It might be beneficial to write some code in C/C++ for performance purposes.
Now let's take a look at the picture you attached. V8 engine, according to Google Official website, is "written in C++". libuv adds a layer of abstraction that provides asynchronous I/O operations, written in C. However, the core functionalities of Node.js, such as networking, Database queries, file system I/O, are provided in libraries (or modules if you prefer) that are written in JavaScript. Plus, your code is written in JavaScript as well. Now in order for these pieces of technology written in different programming languages to communicate with each other, you have to "bind" them together, using bindings. These bindings are node.js bindings.
I've written an article lately that explains the architecture of Node.js' internal codebase where I explained how binds fit into Node.js!
Node.js bindings are series of methods that can be used in Node.js code which are in reality just running C++ code behind the scenes.
fs.readFile()
This method is not part of javascript. It's provided to v8 as part of the node.js runtime. So javascript does not know how to read a file from disk but C++ does. So when we use javascript code and node.js to read a file from disk it just defers all of that to the C++ function that can actually read the file from disk and get the results back.
Javascript also has bindings in the browser too. for example;
document.querySelector()
is not a javascript code. It is implemented by chrome V8 engine.
Upon further research i've come across this article. I hope this helps anyone out:
http://pravinchavan.wordpress.com/2013/11/08/c-binding-with-node-js/
I learned Java programming before learning any other programming languages. As I learn Node.js, I'm getting all the terminology confused. I have always thought API as a library of methods, classes, etc. that someone has built to make our lives easier. Then I learned about module, which I basically assume to be the same thing as an API(list of methods already built by someone). THEN, I learned about the Express Framework, which again, is a list of methods just like a module and an API. Moreover, the way we incorporate these features into our program is all by doing something like
Var http = require('http');
Therefore, can someone who understands the distinctions between these terms put these terms in context(examples) that could address my question.
Thanks a lot for the help.
A library is just a collection of numerous modules, classes, functions, etc. that are related to each other.
A framework is either a type of or a part of a library that is setup for you to build on top of rather than just call upon. And the distinction between Library and Framework can sometimes be a bit blurred.
With Express, you build upon the Application and its Router which handles incoming requests and determines when to call your code.
app.get('/', function (req, res) {
// ...
});
Though, frameworks can also span beyond code into tools. compound.js' executable is a good example of this.
A module is an individual piece of a library or framework. With Node, it's a single script file and the Object that is exported from the script.
An API is the summary/description of how you interact with the library, framework, or module.
It's usually what you'll find in documentation and is the accessible members, their name, their type, what arguments they accept, etc.
Express' API is:
express
Type: function
Named Arguments: (none)
Returns: Application
Members
listen
Type: function
Named Arguments:
name: port
Type: Number
etc.
This is largely an opinionated question. But I will attempt to provide the some terms commonly used by the Node community, and roughly the factual differences between them.
Module as it pertains to Node is very similar to what you would associate with a Java Library. It provides a wrapper around things that Node users find they do a lot. Frequently providing wrappers around node library functions for doing things everyone wants to do. A simple example would be a recursive file system reader, like wrench. Modules also extend to files you use to modularize your code. For example, modules aren't only installed via NPM, but separate javascript files you write as part of your code base to separate code functionality, under standard OOP practices.
require('someNPMINStalledModule')
require('./someFileInYourCodeBase.js')
both are modules. One is installed via NPM and located in node_modules directory, in the directory you launched node from. The latter example is a javascript file located in the directory you launched node from.
Then there are frameworks. At the core these do the same thing as modules, however, they are meant to be more wide spread, and really change the way you use node. In the java world frameworks like Express would be similar to things like Grails. You can still include and do everything you can do in Java, but grails wraps some things for you, and provides convenient powerful method calls for doing batches of work in a less verbose way. In the end you end up with functionally equivalent code, but Grails has allowed you to accomplish more in fewer lines of code, by generalizing the language a little more. But it still, as I said, allows you to use native code, when Grails doesn't provide the functionality you need. At the cost of this 'few lines of code' gain, you have added a layer of abstraction, additional function calls, etc. This distinction is unimportant, unless you are one who cares deeply about style. A hardcore ExpressJS developer likely wouldn't like it if you included a plain node http server in your code. Not so much because it is invalid Node, or from a perforamnce view any different, it wrecks the style of your code. If your code uses a framework, you should stick to using the coding conventions as used in this framework. But if you use a module like wrench to recursively search a directory, it is still perfectly stylistically acceptable to use fs.readFile, to read a single file.
Then there are mini applications which is a module that allow you to quickly launch simple things like serving a file. For example: http-server will server a directory of files to any port you wish, with a simple command line. You wouldn't use them in your own code with 'require' but this type of module can honestly be some of the most useful thing node provides, I highly recommend using some. (Nodemon, http-server, and grunt are a few highly useful examples of modules that can help make your development life easier)
Finally there are Native Extensions. The concurrency that Node provides comes from the V8 backend. Replicating this in pure Javscript is impossible, and the only way to write truly asyncrhonous code is to take advantage of asynchronous operations provided by the Node API, do some really wonky logic with process.nextTick, fork child processes, or write native extensions. Native Extensions provide truly concurrent operations that Node does not provide. Database communication is the most obvious example, but anyone can develope a C++ extension that will spawn threads to do work. There is also a very handy module that launches threads to handle bits of Javascript called "threads a gogo". It simplifies the launching of truly concurrent work, though if you're in a position where such things are necessary, you may find that you're using the wrong language for your use case. Ultimately these are no different from Modules in the way that you use them, but being aware of the fact that they can provide additional concurrent method for I/O type of operations not provided by NodeJS APIs is a unique and very important distinction.
I am looking at using Amazon cloud services (EC2, S3 etc) for hosting. I've been looking at the JSON metadata that can be specified to configure various instances and the complexity has me concerned. Is there a dsl that will generate valid JSON metadata and more importantly validate the entries?
Unfortunately, I drew a blank after searching for this recently. I'm using Amazon Web Services CloudFormation (is that the JSON metadata you're talking about?).
There are a couple of issues with CloudFormation JSON files:
I'm at well over 1,500 lines and it's impossible to read,
You can't express everything the API gives you, notably in the area of Virtual Private Clouds,
There are lots of bugs that are taking a long time to fix, Elastic Load Balancers losing HTTPS information, for example.
So I've been using straight-up API calls in Scala using the Java API. It's actually really nice.
The Java API has a flavor of "setters" starting with with that return this so they can be chained. In Scala, you can use these to act like a poor-man's DSL. So you can do things like
val updateRequest = new UpdateAutoScalingGroupRequest()
.withAutoScalingGroupName(group.getAutoScalingGroupName)
.withAvailabilityZones(subnetAZsOfOurVPC)
.withVPCZoneIdentifier(subnetNamesOfOurVPC)
as.updateAutoScalingGroup(updateRequest)
Other things are easy to do in Scala with the Java API. For example, group all your Subnets by VPC in a Map, simply do
val subnetsByVPC = ec2.describeSubnets(new DescribeSubnetsRequest).getSubnets.groupBy(_.getVpcId)
In case anyone is still looking for the AWS CloudFormation DSL - we've been using the Ruby DSL for CloudFormation:
https://github.com/bazaarvoice/cloudformation-ruby-dsl
This neat project provides a tool to convert your existing CloudFormation template(s) to the Ruby DSL
It will generate valid JSON output
Validating Ruby template entries is similar to validating a regular CloudFormation template (see cfn-validate-template)
Your template becomes Ruby code, so it's easy to have reusable modules (DRY)
You can define local variables
You can have comments in your DSL template
Greatly improved readability
Greatly reduced DSL template size
The CloudFormation request template body size limits are annoying - we have to upload our large CloudFormation templates to S3, then create/update stacks using their S3 URLs.
There is now, though I haven't used it yet: Coffin a CoffeeScript DSL for CloudFormation.
If you're not talking about CloudFormation, but instead more general API talking, then the nicest interface I've found is AWS' own aws-sdk ruby gem. Unlike the other SDKs that they publish which are quite nicely-done-but-crude make-client/make-request/get-response/look-at-result affairs, the ruby SDK wraps a nicer domain-model over the top, so you interact via collections at a higher abstract level.
It also has quite-nice performance features in that you can cache responses to save on round-trips, if you know you don't need fresh responses.
I have CloudFormation templates upwards of 3000 lines. I have found that putting comments in the JSON helps tremendously!! You just have to strip it out before using it. There is a validator that would validate the template and strip out the comments: http://cloudformation-validation.com/
No, as of Feb 2022, AWS still does not provide any domain-specific language for infrastructure as code. They have nothing similar to Azure's Bicep or Terraform's HCL.
This really surprises me, as I generally think of AWS as being more expensive, but ahead of the curve, compared to other major competitors (Azure and GCP).
However, Cloud Formation now allows both JSON and YAML formats. Slight improvement?? IMHO, not really when you have an entire repo that represents your entire cloud stack. If you're using AWS, use Terraform to manage IaC.