I have a folder that contains all my cloned GitHub repositories. Now I would like to get a list of all the repositories that reference a given dependency.
I think of something such as:
$ whouses async
And then I'd like to get a list of all repositories, where async is either referenced as a dependency or a devDependency. Basically, all whouses would need to do is to enter each sub-folder of the current folder and check the package.json file.
Is there a tool available that does this, or am I better off writing one for myself?
Well, there's a way to browse the npm registry for all published modules that depend on a library: https://npmjs.org/browse/depended/async. That may help you a bit. For a local set of modules I don't know off the top of my head if anything exists already or not.
Related
I alter some code in a package at
C:\Users\usr\AppData\Local\.meteor\packages\accounts-ui-unstyled\1.3.0\web.browser\login_buttons.js
The thing is , after I alter the code and run “meteor” in the command line the changes are not implemented, I even deleted the whole package mentioned before and run the app and it was like … nothing happened, it’s like the application have some sort of a cache of the packages that he doesn’t have to go to that path to get them , instead it uses what it had from it before.
Can anyone please explain this to me ? What’s happening here ?
The correct way of "changing" a package is to git clone the package from git (or otherwise retrieve it's source) into either a project internal /packages folder or a project external folder (requires environment variable METOER_PACKAGE_DIRS).
If the package is, as in your case, a Meteor internal package, you can also copy only the package into your project and even add it to your versioning.
In this package you then apply your changes. It will be used in favor of the atmosphere package.
A good practice is to also increment the package version, so it is known for everyone that a custom version is in use.
Why you should not change packages inside the users \Users\...\.meteor installation packages folder?
This is the path to packages, that will be used as defaults for every new meteor project you create. Deep changes can create deep damage to your projects since changing a package will apply to all dependent projects.
Think also about project specific customization. The above described method will allow this, too.
Is there any way to search among scoped packages? I have access to a private repository so I would like to see what packages are available there and try to figure out the package names. I totally understand that, if somebody doesn't have access to a repository then they mustn't see the content of it. But what about those who have access and would like to use it effectively?
Documentation doesn't help, however it is a basic scenario for me.
Depends on what branch you will be checking out. Anyway you could simply check out the main branch and look into the package.json file. Under dependencies there will be listed all the npm packages that the project is using. One other way would be to manually search through the node-modules folder.
Project A contains a few functions and data models I use in diff't repos, all tied to the same product. I'd like to turn them into an npm module, but without extracting the code from project A.
When I see other modules on npm, they generally tie to a github repo that contains all the source code, as well as a full stack to run/modify the module.
Does this mean I have to extract the code from project A into its own repository, build/configure a stack to allow it to run in isolation from project A, and then import it back into project A & other projects?
Or is it possible to just export the functions w/o a full stack, and without moving the code from my main project?
an attempt to pre-empt 'duplicate' comments:
this Q talks about working with an existing module, which doesn't answer my concern, as it has to do w/ worrying about pull requests being merged on time
npm link, discussed here, looks like it'd do the trick if I'd already extracted the code from the project, but I'd like to avoid that.
If you really want to share a snippet through npm but still use the code at the same place in your project, you could extract the code into its own repo, but still use it inside your project as a git sub-module.
Create a submodule repository from a folder and keep its git commit history
Do you know if it's standard for npm modules in their own repos to include the full stack for running them?
Ideally, it's to test them and ease the development, but it's totally optional. You could only put a JavaScript file and the package.json and it would work.
I have a repo which consists of several "micro-services" which I upload to AWS's Lambda. In addition I have a few shared libraries that I'd like to package up when sending to AWS.
Therefore my directory structure looks like:
/micro-service-1
/dist
package.json
index.js
/micro-service-2
/dist
package.json
index.js
/shared-component-1
/dist
package.json
component-name-1.js
/shared-component-2
/dist
package.json
component-name-2.js
The basic deployment leverages the handy node-lambda npm module but when I reference a local shared component with a statement like:
var sharedService = require('../../shared-component-1/dist/index');
This works just fine with the node-lambda run command but node-lambda deploy drops this local dependency. Probably makes sense because I'm going below the "root" directory in my dependency so I thought maybe I'd leverage gulp to make this work but I'm pretty darn new to it so I may be doing something dumb. My strategy was to:
Have gulp deploy depend on a local-deps task
the local-deps task would:
npm build --production to a directory
then pipe this directory over to the micro-service under the /local directory
clean up the install in the shared
I would then refer to all shared components like so:
var sharedService = require('local/component-name-1');
Hopefully this makes what I'm trying to achieve. Does this strategy make sense? Is there a simpler way I should be considering? Does anyone have any examples of anything like this in "gulp speak"?
I have an answer to this! :D
TL;DR - Use npm link to link create a symbolic link between your common component and the dependent component.
So, I have a a project with only two modules:
- main-module
- referenced-module
Each of these is a node module. If I cd into referenced-module and run npm link, then cd into main-module and npm link referenced-module, npm will 'install' my referenced-module into my main-module and store it in my node_modules folder. NOTE: When running the second npm link, the name of the project is the one you find in your package.json, not the name of the directory (see npm link documentation, previously linked).
Now, in my main-module all I need to do is var test = require('referenced-module') and I can use that to my hearts content. Be sure to module.exports your code from your referenced-module!
Now, when you zip up main-module to deploy it to AWS Lambda, the links are resolved and the real modules are put in their place! I've tested this and it works, though not with node-lambda yet, though I don't see why this should be a problem (unless it does something different with the package restores).
What's nice about this approach as well is that any changes I make to my referenced-module are automatically picked up by my main-module during development, so I don't have to run any gulp tasks or anything to sync them.
I find this is quite a nice, clean solution and I was able to get it working within a few minutes. If anything I've described above doesn't make any sense (as I've only just discovered this solution myself!), please leave a comment and I'll try and clarify for you.
UPDATE FEB 2016
Depending on your requirements and how large your application is, there may be an interesting alternative that solves this problem even more elegantly than using symlinking. Take a look at Serverless. It's quite a neat way of structuring serverless applications and includes useful features like being able to assign API Gateway endpoints that trigger the Lambda function you are writing. It even allows you to script CloudFormation configurations, so if you have other resources to deploy then you could do so here. Need a 'beta' or 'prod' stage? This can do it for you too. I've been using it for just over a week and while there is a bit of setup to do and things aren't always as clear as you'd like, it is quite flexible and the support community is good!
While using serverless we faced a similar issue, when having the need to share code between AWS Lambdas. Initially we used to duplication the code, across each microservice, but later as always it became difficult to manage.
Since the development done in Windows Environment, using symbolic links was not an option for us.
Then we came up with a solution to use a shared folder to keep the local dependencies and use a custom written gulp task to copy these dependencies across each of the microservice endpoints so that the dependency can be required similar to npm package.
One of the decisions we made is not to keep two places to define the dependencies for microservices, so we used the same package.json to define the local shared dependencies, where gulp task passes this file and copy the shared dependencies accordingly also installing the npm dependencies with a single command.
Later we made the code open source as npm modules serverless-dependency-install and gulp-dependency-install.
To be completely specific:
I am writing a Node.js app that is intended to be a websocket bot for Slack.
A Node project exists that abstracts the majority of the Slack API. (It is NOT an npm module.)
I'm not overly familiar with grunt, etc. but I can get the dependencies to install and utilize all this code by placing my own mybot.js in the root folder of this git clone and running node mybot.js with mybot.js being based on the files in the example folder.
Committing to my own repository, I don't want to commit any of the aforementioned project code -- it's not mine! I do, however, want it as a dependency. Unfortunately, this code by Slack is not an npm module that makes it easy to do. The project has a /bin folder and a /src folder full of coffee script, etc. that grunt builds to .js files.
The Slack project code has its own dependencies. In my way of thinking, those are sub-dependencies for me, or cascading dependencies. My project only depends on whatever the Slack project depends on.
I would like to be able to update my project with updates (manually, or via build) from the git repo of the Slack project as needed.
It seems there must be a way for me to include this project as a dependency, and once built, properly reference it's bin and src folder objects (bin/slack, src/message, client, channel, user, etc.) without committing it to my own repository. Especially great if it could be in a subfolder separate from my own model definitions. In a way, this seems no different to me than including jQuery in my website layout via a CDN. I'm only asking for the jQuery project and depending on my link flavor, I can get a specific version or the latest version, etc.
So, it turns out the comment by Ben pointing me to the npmjs.com slack-client npm module was the help I really needed. I just didn't really know how to ask the right question, I think.
And while I hate to look a gift horse in the mouth, a little more than a link, Ben, would've saved me another three hours, probably. Perhaps: "It is an npm module, not just a project from github." But thank you, even if it took me a while to decipher what you were saying.