We have multiple aws modules in git and when we use a module in other project we specify the path of the module in git as a source like this:
module "module_name" {
source = "git::https://gitlab_domain_name.com/terraform/modules/aws/module_name.git?ref=v1.0.0"
...
}
I want to know if there is a benefit to use a terraform private registry to store our modules like for instance when developing in Java we use a repository to store JAR packaged or also when working with docker images.
Yes, there are benefits to a private registry. Namely you can put some description, documentation, examples there and you get a better visual representation of what the module does, its inputs, outputs and resources.
But apart from that in terms of functionality of the module it behaves the same way. A java registry for example (e.g. nexus) makes sense because you do not want to force everyone to build the libs themselves (maybe they can't at all) and therefore having a place where pre-built libraries are stored does make sense. That reasoning does not apply to terraform since there is nothing being compiled.
Whole different story for custom providers, in that case you need a private registry to provide the compiled golang binaries, but you can write one yourself without too much effort (since terraform 0.13), it is just a http rest api.
Related
I've been trying to find the answer of this question regarding the equivalent of Terraform modules in Pulumi, the closest answer is this link to this blog. Bear in mind that I'm a beginner for using Pulumi too.
With Terraform, you can create a git repository containing all of your modules, version it and pull it into various other git repositories by using source = "git#github.com:xyz". Terraform allows you also to turn On and Off resources based on conditionals such as regions, account number or environments for instance (with count method for modules and resources).
Apparently Pulumi does not have this concept, looks like you need to duplicate your code in each repository or create a giant monolith repository containing all of your code. I'm also wondering what's the best practice for feature flags, turning On and Off resources for each of your specific stacks, what sort of conditionals you'll be using for this.
Thanks again for your highlights!
Broadly, you should create libraries in your language of choice and put reusable functions, classes, and components in there. For example, if you use TypeScript, you could create an NPM module (public or private) with any code that you want to reuse across projects and teams.
More specifically, if you are looking for a way to combine multiple resources into a reusable higher-level abstraction, you can implement it as a Component Resource. A component would accept inputs, instantiate a few resources in its constructor, and return output values. You can still package a component (or multiple components) as a reusable library.
Pulumi also allows you to create multi-language components, where you would implement it in one language but then publish it in all the supported languages for everyone to use. You can ship a multi-language component as a package in Pulumi registry to simplify discovery and installation. Read more in Pulumi Packages and multi-language Components and see other component in the Registry.
I wrote an article about implementing feature flags with pulumi here. In summary you should use the Pulumi configuration system. You can store configuration values in the stack settings file, which are automatically named Pulumi..yaml. You should split resources into their own python modules which you can then optionally import based on the settings in the config files.
I have a concept of a nodejs server service, that consists of
a rest API and
some asynchronous workers.
Both should live in different subnets and git repositories. But both access database and should use the same entities and types.
So I would like to have something like shared module (don't know the proper term in npm ecosystem) with just entities / types, that would use typescript and use that shared module in both rest API and async workers. The shared module needs to be versioned in git separate (private) repository.
How can I do this in nodejs?
So far I read something about treating it like a library (here and here). I especially don't know, how and where should I put the js files, compiled from typescript.
If all you need is versioning, maybe consider treating it like a regular npm module?
npm-install supports git repositories and git supports release tags release tags
From those two all you need to do is compile typescript to an index.js file at the root of the module (library), and you can access it by require inside where it's needed.
(just be mindful that you will need each shared repository to have a package.json)
I'm a beginner of Haskell, and I got a question here, how to release or build a module which can be used in another program ?
For example, when I use Java, I can write some classes, some functions and make it to be a jar that can be use in another program. So , how does this go with Haskell? Is there necessary to use the same functions, modules with the source code ?
And thanks.
If you're fine with sharing your library and its source code with everyone, it is common to upload it to Hackage. Other projects built with any common Haskell build tool know how to interface with Hackage packages
If you want to selectively share your library and its source code, you can upload it to a private or public github repository, and it could be used as a stack dependency
If you want to share the library locally with yourself, stack supports depending on local paths
If you wish to share your library without sharing source code, you can expose a C-api and make a DLL
I've got a VS2013 solution with a mix of NodeJS (using TypeScript) and C# class library projects (they're bound together by EdgeJS). Where the NodeJS projects are concerned, one can be considered a library (for a RabbitMQ bus implementation), two are applications which are meant to be hosted as part of a fourth project with both using the bus.
So, one project (host) which will depend on three projects (bus, app1 and app2) (it starts the bus, and passes it to app1 and app2).
Of course, I could just lump all these projects together and be done with it - but that's a horrible idea.
How do I package these projects up for proper reuse and referencing (like assemblies in traditional .NET)?
Is that best done with NPM? If so, does VS provide anything in this area? If not, then what?
Note that, aside from the Bus project, I'm not looking to release these publicly - I'm not sure if that changes anything.
In general, if something can be bundled together as an independent library, then it's best to consider this a Node package and thus, refactor that logic out to it's own project. It sounds like you've already done this to some extent, separating out your bus, app1, and app2 projects. I would recommend they each have their own Git repositories if they are separate packages.
Here's some documentation to get you started with Node packages:
https://www.npmjs.org/doc/misc/npm-developers.html
The host project, if it's not something you would package but instead deploy, probably does not need to be bundled as a Node package. I would instead just consider this something that would be pulled down from Git and started on some server machine.
With all that said, your last line is important:
I'm not looking to release these publicly
GitHub does have private repositories, but as of now npmjs.org does not have private repositories. There are options to create your own private repository (Sinopia and Kappa offer different ways of accomplishing this), but if you don't want this code available for everyone do not deploy it do npmjs.org. You can still package it up in the way I've outlined it above, just not deploy it as of yet.
I'm writing a number of pieces of code (for internal use) using node.js and want to store the modules (packaged up for npm) in a package repository for each distribution to the various machines they will be installed on.
Ideally, I'd like a solution similar to Debian's apt repositories in which I can run a private repository server and configure npm to use a list of repositories to install from (When installing "foo", if "foo" is known by my private server install it from there, otherwise install it from the public server).
However, it looks like the npm registry configuration key only accepts a single URL.
Is there a way to achieve what I want?
The closest I've been able to find have been:
Mirroring the public repository locally and adding my packages on top of it… but I don't want to keep that amount of data (2.5G and still downloading) replicated on AWS.
Hosting all my packages in git repositories and installing from there (which is more of a hassle).
Hosting static packages on HTTP (as far as I can tell, this would prevent me from automatically getting "the latest version". I suppose I could do something with symlinks, but that is still less flexible than git, requires full URLs (which need to be kept up to date), and doesn't give a searchable repository.
I just set this up for my work. Here's what I did:
Setup empty NPM registry: I followed the instructions from this fork of npmjs.org, which adds much improved documentation.
Setup Kappa: I used Kappa, a great npm proxy from Paypal. (I'm guessing they have a very similar use case to most people who want a private repository; this was exactly what I wanted).
Setup npm_lazy (optional): I wanted a nice cache of frequently used packages in case npmjs.org went down, so I added npm_lazy in front of the whole thing, as a caching layer.
Whole thing took two days(ish) to get up and running. As a side note, if you're worried about people pushing to the public registry by accident, I recommend adding this to your package.json:
"publishConfig": { "registry": "http://my-registry.example.com" },
This really is just a bit of paranoia; once you setup your npm to point to your Kappa/npm_lazy instance, Kappa handles publishing to your private repository for you.
Note: Kappa will only every publish to the first repository in it's config. If you need to publish to both your private registry, and the public, you will need to work out your own solution.
In your package.json, you can use any url that points to a valid npm packed module. I use an s3 with a bucket name that is hard to guess.
npm pack
s3cmd put *.tgz s3://path-to-your bucket
S3 is just an example, you could use any mean that can place a file on a web server, it can even be protected via basic auth.
I believe Paypal's Kappa project would suit your need.
Here is an article describing Paypal's Kraken project and how Kappa fits in.
I understand it wasn't available at the time of Quentin's question, but perhaps this will be useful for others that come along here.
npm-registry-client GitHub issue #42 lists several ways how to create your own repository mirror. namely:
https://github.com/cnpm/cnpmjs.org
https://github.com/rlidwka/sinopia
https://github.com/rlidwka/sinopia#similar-existing-things
https://github.com/davglass/registry-static
Overall it seems to me that you can get best answers by searching through issues in repositories owned by https://github.com/npm or by asking your question there
Based only on listening to a recent episode of NodeUp (#37?), I think you may want to have a look at irisnpm. From what I remember it's a service which gives you a merged set of the public modules and your own private modules.
As Dominic Barnes suggested, we can replicate only _design documents (CouchDB table schemes)
How to replicate design documents only?
However, it needs to check if some data needed.
You could replicate the modules you need and then write a proxy server which looks for a module in your replication. If a module doesn't exist, it could pipe the request to NPM and return the result from there.