I am trying to get a project going with gRPC that has the servers written in golang and the clients written in node
I have a protobuf file structure like this
- $GOPATH/src/company/api
|- server
|- svc1
|- svc1.proto
|- svc2
|- svc2.proto
|- hardware
|- hardware1
|- hardware1.proto
|- hardware2
|- hardware2.proto
|- lib
|- used-by-both
|- used-by-both.proto
When defining a service that uses something in the lib directory, I use -I$GOPATH/src then import using import "company/api/lib/used-by-both"; in the proto file and that seems to work for golang when I generate everything.
However, when I generate the node files, the import paths in the packages that depend on files from lib to be incorrect when the while api directory is installed as a node module
If I try to make the -I option for the compiler more specific and shorten the import statement, the golang package import gets messed up
Any advice on how best to compile these proto files in order to make use of them easliy as go packages and node modules?
Related
I have a project with this structure,
src
|- main.ts
libs
|- httpClient.ts
For now, when I run typescript build all the files are bundled like this.
dist
|-src
|- main.ts
|-libs
|- httpClient.ts
What I wanted to do is like this.
dist
|- main.ts
|-libs
|- httpClient.ts
I wonder if this is possible.
The situation:
client and server both share a folder shared
when we change shared in our development flow, we want the corresponding references to change in customer and server
server works because somehow with npm it seems to work, shared and server use npm
customer doesn't work and uses yarn
mixed typescript and js project
Code Structure:
root/
|- client/
|- package.json
|- src/
|- ...
|- server/
|- package.json
|- src/
|- ...
|- shared/
|- package.json // we don't want to change version every change
|- src/
|- ...
What's been tried
3 solutions proposed here
create a folder common under root and just require the files you need from your files. But you could end up with "long" require such as require("../../../../../common/file")
long require doesn't work with resolution with webpack - and isn't a nice solution
use module-alias to avoid that problem: https://github.com/ilearnio/module-alias
module-alias seems to be the same solution as the next one, in how it behaves.
you could make common a local module (using file:) and install it in package.json https://docs.npmjs.com/files/package.json#local-paths
we currently do this, but we still have to reinstall the shared folder on a change, we currently use yarn upgrade shared which takes a long time and still requires us to know when we need to run it.
in addition we've attempted to get yarn link working but doesn't seem to link properly
we've attempted to use a post-pull hook with husky to run yarn upgrade shared but it's too slow to run every pull especially since it's not needed often
we've considered a true mono-repo 'package' like Lerna but still don't think that it's necessary for the cost to migrate
link-module-alias to sym link folders on post install script, but this fails with typescript files
Goal
either
find a way to automatically sync these in dev environement
find a solution that installs/updates manually - but is fast and can be run on every pull
find a way to automate running yarn upgrade shared that runs (roughly) only when needed not all the time
I guess we could find a way to automate the version increment on any change of shared's version key, and then it's tracked, we could run yarn install and that work work.
The find solution was that we were using React-Native and therefore the normal steps for syncing would work for the IDE but not the React-Native app.
Here is a great article describing how to get metro bundler working with it - however for Typescript we added it to the TSConfig and for the IDE we still needed to add it to our package using the file:../shared directive.
I would like to use a parent NPM package from inside a subfolder, as part of the development process. Let's call my library "logary".
Structure:
.
|- package.json
|- tsconfig.json
|- src
|- ... all the source code
|- dist
|- ... compiled TS as .js, .js.map and .d.ts
|- examples
|- with-nextjs
|- package.json
|- tsconfig.json
The library is written in TypeScript and so is the example. The parent compiles with outDir: ./dist in tsconfig.json. I have multiple entry points (one for the main lib, one for each peer dependency as for example: import withLogary from 'logary/plugins/nextjs'.
I would like to be able to write import Logary from 'logary'; inside ./examples/with-nextjs and have Logary, the constructor, have types attached to it be fully "resolvable" by the TypeScript compiler. I want changes in . to be reflected when ./examples/with-nextjs is reloaded.
It's not a monorepo (in the common sense). I've tried yarn link in . and then yarn link logary inside ./examples/with-nextjs, which doesn't seem to solve it (cannot import module). Furthermore, as I have bumped the version in . to 6.0.0, as I yarn install inside ./examples/with-nextjs, yarn complains that version is not published to npmjs.com and asks me to select a previous version, which it then downloads.
I know there's Lerna ("Splitting up large codebases into separate independently versioned packages is extremely useful for code sharing." — this is a tiny code-base and only has a single package, I just want to reference it in a single direction from an example) and Yarn Workspaces ("It allows you to setup multiple packages in such a way that you only need to run yarn install once to install all of them in a single pass." — which is not my aim)
I'm not interested in adding any special config (like paths) to tsconfig.json in ./examples/with-nextjs because then it's not just to copy the example to as your own app.
➡ How do I best structure a TypeScript library project with multiple peer dependencies and with an examples folder?
Maybe related?
Use yarn workspaces and typescript's project references to reference another package subdirectory — use exports in package.json to solve the logary/plugins/nextjs problem (I'm happy doing that, but I can't get the parent project properly linked with peers correctly installed, so just import Logary from 'logary' in the examples uses the old published version, despite me having run yarn link)
Peer Dependencies In A Monorepo — this is probably a good solution to the peer dependencies problem (e.g. with https://www.npmjs.com/package/rollup-plugin-peer-deps-external)
Using Lerna with unpublished packages — has a similar question, but doesn't ask about consume-only app packages like my with-nextjs. Also, my root is at . not at packages/logary; but maybe I must conform?
Using composite: true in tsconfig.json in . and also references in the child project's tsconfig.json changes how the outDir works; it adds the src/ folder as prefix to all output paths, which is not what I want
I am developing an AWS SAM application in nodejs and have two lambda functions (each are distinct node modules) which both share a third node module which is shared utility functions.
I am using the install-local npm module to install the shared local utilities node module into the lambda function node modules post-npm-install because SAM local-start doesn't appear to work with symlinked local modules.
This works fine when I start the SAM API locally and hit the end point and I can successfully debug the Lambda functions, however, when I step from the Lambda function code into a function within the shared utilities node module, it opens the file from the lambda function's node_modules directory (local-install doesn't symlink from node_modules to the shared code on my local file system).
I would very much like to somehow map the file in the lambda function's node_modules to the shared utility code on my local file system.
Here is my directory structure:
root
|
|- LambdaFunction1
|- app.js
|- node_modules
|- SharedUtilityFunctions
|- app.js
|- node_modules
|- LambdaFunction2
|- app.js
|- node_modules
|- SharedUtilityFunctions
|- app.js
|- node_modules
|- SharedUtilityFunctions
|- app.js
|- node_modules
In this example, I want to (only when debugging in VSCode) map root/LambdaFunction1/node_modules/SharedUtilityFunctions to root/SharedUtilityFunctions
Does anyone know how this can be achieved?
My higher goal is to create a .zip file with the server bundle and all needed assets.
That file can be then moved to another device and executed there (the other device has the same NodeJS version installed). Building this on the device is not an option, because it's offline.
build/
|- node_modules //this is where I would like to have my externals
|- server.bundle.js
|- public/
My current solution is to pick and copy the external modules by hand, which is neither scalable nor convenient.
Would there be a possiblity, to let webpack handle this for me? Or even bundling externals like express into the bundle itself.
You could have just express in the dependencyarray on package.json, then run npm install --productionto just install that, so your node_modules is going to be small.
In the webpack config, you have to be sure to add externals (webpack_externals). Due to some binaries in some node dependencies, webpack is not able to bundle them together. You are going to have a require call for just those dependencies.