I'm a web developer and I work for a secret electronics facility. All our computers are connected to the secret internal network, with no internet access. I'm allowed to transfer-in files using an 18 terabyte hard drive from an "internet computer".
Creating a Python PiPy server was no problem, I just used "bandersnatch", all versions of all pip packages ever are only 13.9 terabytes.
I want to do the same with NPM, but I read the the CouchDB database that used to be clonable, is deprecated, and verdaccio doesn't actually provide a mechanism to clone the real NPM registry, so you end up with an empty NPM registry.
Just a simple React project has more than 800 dependencies, and I have no way to name them all, or manually input the required versions. I need the whole entire thing.
Related
I have a Node.js API on a Windows computer without internet access and I need to access data from a Microsoft SQL Server database. I have the module "mssql" however I need to use Windows authentication. I found online that this method requires the module "msnodesqlv8," however when I install the module on an internet-connected device and then move the files onto the non-internet device, the module won't run. The error I receive when I run the API indicates that the module needs to be rebuilt. Whenever I run "npm rebuild," the build attempts to install the dependencies - which of course is not possible without internet.
To try to circumvent this issue, I removed the dependency list prior to rebuild, but the rebuild still snags trying to install "safe-buffer." I don't know where this install is getting triggered. When I search for "safe-buffer" in the project, no references to "safe-buffer" are found. My assumption is that a dependency is trying to use it and I have not skimmed through every dependency yet.
I also tried packing the "msnodesqlv8" module, then installing it from the .tgz file, but this produced the same errors.
One avenue I have not yet tried is packing the module on the internet device, then transerfing the .tgz file.
Any recommendations would be awesome - I am blocked until I can solve this issue.
I was able to solve this problem by downloading a prebuild from the Github releases. I noticed in the stack trace of the rebuild that the process was accessing a specific build from this link, so I moved it into the "prebuilds" folder where it was expecting it, and the module powered up on the next rebuild.
This process has to be replicated on every device, from workstation to servers, but it works. Just make sure you download the build that matches your system.
I'm trying to develop an Azure function locally using WebStorm/PHPStorm, was wondering if Azure functions have any pre-installed/built-in npm packages ready in the cloud, since deploying my final product becomes a process if I want to include all of them. If so, how may I check which ones are available?
You can go to https://yourfunctionappname.scm.azurewebsites.net/DebugConsole to check.
If you run npm list -g --depth=0, you will get nothing.
We do not have access rights to all folders, so we cannot view the specific software packages used after configuration on the system drive, but can only see a series of packages that came with the creation of the function app.
Azure Function is based on web app sand box, so you can find pre-installed package in this folder: %ProgramFiles(x86)%\SiteExtensions(There will be many version.)
https://github.com/projectkudu/kudu/wiki/Azure-Site-Extensions#pre-installed-site-extensions-package
My Node application needs to be deployed on Windows and Linux. The main deployment package is built on a Linux CI server.
When this package is deployed to Windows, it crashes immediately due to missing native bindings, such as those for sqlite. Only the bindings for the build platform (Linux) are restored.
With a deadline approaching, we just set up a Windows build configuration which outputs a Windows specific package that contains the appropriate bindings, and we choose the appropriate artifact to bundle in the installer.
This works but feels fragile, as we would need to keep the Node versions in sync between the two otherwise unrelated environments. I would like to be able to do this with a single build configuration.
I couldn't find any guidance on how this is done. I'm imagining a command-line option like --platform=windows to npm ci, or a modification to package.json but I couldn't find any information about this. Presumably this is a reasonably rare requirement, and perhaps there is no tooling around this, which would be a shame.
Another requirement is that the application must be installed without an internet connection. We cannot run npm ci or npm install when we install it as some of our clients do not permit their servers to access the public internet.
Based on your requirements it sounds like building a package on each required platform would be the safest bet, with the least number moving parts to go wrong.
As the comments have suggested most projects rely on an npm install on the required platform so you are stepping into not that common territory.
This works but feels fragile, as we would need to keep the Node versions in sync between the two otherwise unrelated environments. I would like to be able to do this with a single build configuration.
Node uses NODE_MODULE_VERSION (displayed on the releases page) to track ABI compatibility for native modules. This only changes with a new major Node release number.
The CI builds would need to create app packages for each major version of Node you run on each platform. Keeping the Node.js major versions in sync for the application a good thing in any case. Running Node N and N-1 builds until that can be achieved is good cover and probably the best option with the air gap requirements.
NPM Cache
If the air gapped clients are largely on common networks, an NPM cache/proxy (nexus/verdaccio) may be of use. The NPM cache will need a process to snapshot the repo after a production npm install on all required platforms, to be pushed out to your endpoints. Unfortunately binary modules are often distributed out of band from NPM so won't be stored in regular NPM caches. Each client instance will need a complete build environment to build any native modules from source which can sometime present it's own difficulties on Windows platforms.
Alternatives
Node.js is not a great platform for distributing packaged applications to many diverse clients, especially if you need to distribute Node itself. Any language with an external VM requirement presents difficulties. Nodes package management choices and reliance on native modules exacerbate this.
I've given up in the past and converted clients (albeit thin) to Go, as it lends itself to cross platform distribution a lot better by removing the external runtime requirement and having less variables.
Okay, so I get it that I might be looking for a totally unexpected behavior.
My current company has nothing for me to do for a few weeks, so I would like to start a side project. The problem is, the firewall is really strong here, so I cannot download anything with Git or Npm. I also am not allowed to do any request to the IT support, since I am not in my company's office but some offices owned by a client (that applies said strict policy). In short, I am stuck with firewall and proxy policies that I cannot modify.
I may download a module's zipped archive through the browser, and install it from there. However, it has multiple dependencies, that themselves have dependencies, and so forth. And since I cannot run npm install to retrieve the dependencies, I'm stuck.
I saw multiple possible solutions in order to solve those dependencies issues:
make NPM run all requests and downloads through the browser, since browsers are allowed to access to the network. I don't see any options for that so far.
Download all required dependencies as tarball and step-by-step install each of them. Because of the potential number of dependencies being huge, I am looking for a huge bulk of modules commonly used to download once.
Most solutions I find make the assumption that I may use npm install properly, while my proxy doesn't allow it.
I wouldn't like to spend days on Chrome's built in game. Any idea?
maybe you can create the project somewhere else, and then "import" it on your office local machine:
npm init
edit package.json with your dependecies
npm install
put everything on usb stick and put it on your local computer at work.
or
send an archive of the code via email and download the tarball from email at office.
Is there a way to install meteor packages globally?
So, having the once globally installed packages installable without internet connection for projects created later, avoid repetitive downloading, and other benefits one may imagine.
Like in Node.js, using npm command (of Node Package Manager) with -g flag, npm install -g, doing so npm installs node packages into a global directory and when wanted to be loaded from javascript programs, loading from there if available, as well as looking in and loading packages from project's node modules folder.
Meteor already downloads packages into a global repository that all your local apps benefit off of.
So if you meteor add iron:router#1.0.7 it is downloaded and added to your project. Next time another project requires the same version, it is used off that same spot.
Also, there is a PACKAGES_DIR environment variable, when set, allows you to keep your own local packages centrally, so that you can share them among projects. In fact, you can keep that on a network drive (NFS) which your whole team can mount and consume centrally.
Yet, there is an inherent problem. Meteor's version resolver looks up for updates unless you pin down your package dependency versions so that is exactly why meteor seems to be so desperate to be connected.
Even if you pin your dependencies, the packages you depend on may not have (which apparently is the case for most packages) so Meteor keeps looking for updates to the whole package tree and downloads those that it deems satisfying the version constraint resolver.
The good news is, they are constantly improving their tooling, requiring lower number of lookups, faster builds, better search etc.
All in all, in essence, there is not much you can do unless Meteor provides some way of hosting an entire mirror of its package repository for you to consume offline. And I guess it is very unlikely to happen.
Meteor is a tool for the connected world and it does assume your connectivity. Heck, the whole journey begins with a curl https://install.meteor.com/ | sh
And yes, it would be great if we could hack away on a remote beach, or the 12 hour flight to that beach.
Until then, happy coding online ;)