Is there an option to pick specific files to be only included in a gitlab release instead of having to have the whole repository packaged as zip/tar files.
.
├── docker-compose.yml
├── Dockerfile
├── flow-definitions/ -- folder of flow definition
│ ├── main_flow_definition.json -- root flow definition for nifi instance
│ ├── bin/ -- folder for flow definitions impacted by dev cycle
│ │ ├── impacted_flow_definition.json
│ │ ├── impacted_flow_definition_2.json
│ │ └── ...
│ └── scripts/ -- scripts relevant to data pipelines
│ ├── databases/ -- DDL for target databases used in ETL
│ │ ├── db_name/
│ │ │ ├── <DDL scripts>
│ │ │ └── ...
│ │ └── ...
│ └── groovy/ -- Groovy scripts used in NiFi processors
│ ├── df_folder/
│ │ ├── <groovy scripts used in processors>
│ │ └── ...
│ └── ...
├── README.md
└── .gitignore
Currently I do the releases in gitlab through the UI and it packages the whole repository when a release is done. As you can see from the above folder structure majority of the time the changes are only within the bin/ folder. As the docker file and other components don't usually change frequently.
So instead of packaging all the files, it would be good to release versions that just have the specific changed files in this case within the bin/ folder.
I couldn't find a way to actually achieve this through gitlab if that's an option available
Related
Cargo has a number of special filenames that it recognizes during the build process. For example, build.rs provides build instructions, lib.rs/main.rs are entry points, and mod.rs is a module. I’d like to avoid accidentally naming my modules something that might be treated specially, or that might be misinterpreted as being treated specially.
Is there a complete listing of all the “special” filenames that Cargo recognizes?
There is a good overview in the cargo book.
.
├── Cargo.lock
├── Cargo.toml
├── src/
│ ├── lib.rs
│ ├── main.rs
│ └── bin/
│ ├── named-executable.rs
│ ├── another-executable.rs
│ └── multi-file-executable/
│ ├── main.rs
│ └── some_module.rs
├── benches/
│ ├── large-input.rs
│ └── multi-file-bench/
│ ├── main.rs
│ └── bench_module.rs
├── examples/
│ ├── simple.rs
│ └── multi-file-example/
│ ├── main.rs
│ └── ex_module.rs
└── tests/
├── some-integration-tests.rs
└── multi-file-test/
├── main.rs
└── test_module.rs
But all of those are just defaults and you can change them via settings in the Cargo.toml file.
build.rs shouldn't ever clash with modules since it lives in the root rather than under src/
I don't think mod.rs is cargo specific but rather how rust modules are defined.
I have monorepo setup and upgraded from yarn 2 to npm 7.
Was working perfectly fine till I tried to deploy my application to the standard app engine environment.
gcloud app deploy kicks off a build inside Cloud Build, which builds my project via buildpacks (nodejs14 buildpack).
My problem is, that this buildpack includes nodejs14 and npm#6 and without using the flex environment it doesn't seem that there is a way to use npm#7 in this standard app engine cloud build.
What I've tried by now:
Use same OS and install node_modules, tar them, upload them, untar them. (could not really do it)
Link packages locally before uploading (don't know if I did that right. App was not finding local packages afterwards)
Trying to use a cloudbuild.yaml, but it seems that this is not possible with standard environment.
in the custom build step "gcp-build": "npm i -g npm#7" (does not work, permission error, see full build log here)
I would really appreciate every pointer in any direction that could help me out.
This is my project setup:
[project]/
├── packages/
│ ├── app
│ │ ├── node_modules/
│ │ ├── src/
│ │ ├── app.yaml
│ │ ├── ...
│ │ └── package.json
│ ├── shared
│ │ ├── node_modules/
│ │ ├── src/
│ │ ├── ...
│ │ └── package.json
│ ├── shared-admin
│ │ ├── node_modules/
│ │ ├── src/
│ │ ├── ...
│ │ └── package.json
│ ├── base-server
│ │ ├── node_modules/
│ │ ├── src/
│ │ ├── ...
│ │ └── package.json
├── package.json
│
└── node_modules/
Many people are using lerna and/or yarn workspace.
I guess either migrating from them to Bazel, or just using them with Bazel together is good to be guided with an example project.
For example, currently, I have a directory structure like this, where foo is an express server and bar is a library consumed by foo, both based on typescript.
<project root>
├── jest.config.js
├── lerna.json
├── package.json
├── packages
│ ├── bar
│ │ ├── jest.config.js
│ │ ├── package.json
│ │ ├── src
│ │ │ └── index.ts
│ │ ├── test
│ │ │ └── unit
│ │ │ └── index.test.ts
│ │ ├── tsconfig.build.json
│ │ └── tsconfig.json
│ └── foo
│ ├── jest.config.js
│ ├── package.json
│ ├── src
│ │ ├── hello.ts
│ │ └── index.ts
│ ├── test
│ │ ├── integration
│ │ │ └── index.test.ts
│ │ └── unit
│ │ └── index.test.ts
│ ├── tsconfig.build.json
│ └── tsconfig.json
├── tsconfig.build.json
├── tsconfig.json
└── yarn.lock
How should I align it with Bazel, like you know, WORKSPACE, BUILD, and their contents?
Any tips or examples?
Thanks!
There are some examples of repo structures somewhat similar to this in the rules_nodejs examples directory. This shows (in this case an Angular app) having shared libs and consuming them, but the principle is the same here.
Generally, you'd only have one WORKSPACE file in the root of your project. While it's possible to have multiple package.json files for different apps and libs, it adds some extra complexity to the ts_library rules which, for getting started, may be best avoided. This example repo shows multiple package.json files, but without Typescript.
For BUILD (or BUILD.bazel) files, the minimum you'll need here is one in foo and one in bar (and one at the root). The more BUILD files you have, the more you split up the compilation units for your source, therefore increasing incrementality.
Then add ts_library rules to those BUILD files, docs for which can be found here, they also show the differences between using tsc directly and ts_library. You can then define source dependencies between foo and bar, a quick example shown below:
packages/foo/BUILD:
ts_libaray(
name = "foo",
srcs = glob(["src/**/*.ts"]),
deps = [
"//packages/bar", <-- this is the source dep for bar
"#npm//some-package",
],
)
packages/bar/BUILD:
ts_libaray(
name = "bar",
srcs = glob(["src/**/*.ts"]),
deps = [
"#npm//some-other-package",
],
)
I want to use the global node_modules (C:\Users\<user_name>\AppData\Roaming\npm\node_modules) or common node_modules (D:\node_modules) instead of local node_modules (D:\angular\projects\project1\node_modules).
I want to use the node_modules for various projects.
This is my project structure.
D:\
├── node_modules (common for all projects)
│ ├── Module-N
├── projects
│ ├── project1
│ │ ├── src
│ │ ├── app
│ │ | ├── Module-N (importing from common)
|
│ ├── project2
│ │ ├── src
│ │ ├── app
│ │ | ├── Module-N (importing from common)
|
├── ....
try working with relative path like when you require you own modules for the global models
for example
assuming we want to get a module from the global node modules and the file we are working on is seats on projects/project1/src
we will do require('../../../node_modules/my_global_model')
try looking into tsconfigs paths: https://www.typescriptlang.org/docs/handbook/module-resolution.html
I installed express, mongodb, and mongoose.
This is the result of my npm ls:
/home/merc/Bookings
├─┬ connect-mongo#0.1.9
│ └─┬ mongodb#0.9.9-8
│ └── bson#0.0.4
├─┬ express#3.0.0rc2
│ ├── commander#0.6.1
│ ├─┬ connect#2.4.2
│ │ ├── bytes#0.1.0
│ │ ├── formidable#1.0.11
│ │ ├── pause#0.0.1
│ │ └── qs#0.4.2
│ ├── cookie#0.0.4
│ ├── crc#0.2.0
│ ├── debug#0.7.0
│ ├── fresh#0.1.0
│ ├── methods#0.0.1
│ ├── mkdirp#0.3.3
│ ├── range-parser#0.0.4
│ └─┬ send#0.0.3
│ └── mime#1.2.6
├─┬ jade#0.27.2
│ ├── commander#0.6.1
│ └── mkdirp#0.3.0
├─┬ mongodb#1.1.2
│ └── bson#0.1.1
└─┬ mongoose#3.0.0
├── hooks#0.2.1
├─┬ mongodb#1.1.2
│ └── bson#0.1.1
└── ms#0.1.0
You can clearly see that for some reason Jade is on the root directory (I assume this happened when I run "express". But then again, "mongodb" is available in different versions (0.9.9 and 1.1.2) and so is bson (0.1.1 and a worrying 0.0.4).
Hence my questions: how are dependencies managed with npm? Does every package simple install whatever they like, whichever version they pick?
I guess the question is: is this kind of duplication "normal", and "by design" so to speak?
Merc.
The short answer is, yes, this is by design. When you require a module from the node_modules directory, it uses the top-level directory--e.g., whichever one you specify in your package.json.
Other packages have their own package.json files, and are free to use whatever versions they want, and when they require them down in their own code, they will use their own node_modules folder.
Ideally, the modules you use have tests, etc. that ensure that versions (or even specify a range of versions, such as 0.9.x) of dependencies they specify work well, and seeing older versions of sub-dependencies in there doesn't necessarily mean danger, although new versions of these modules could of course potentially fix bugs and so forth. It may be worth finding a module you're concerned about on GitHub, downloading the repository, updating the package.json and dependencies yourself and running the tests to see if a new version works. If so, perhaps the author would be willing to accept a pull request with your update.