I notice that Node.js projects often include folders like these:
/libs, /vendor, /support, /spec, /tests
What exactly do these mean? What's the different between them, and where should I include referenced code?
Concerning the folders you mentioned:
/libs is usually used for custom classes/functions/modules
/vendor or /support contains 3rd party libraries (added as git
sub-module when using git as source control)
/spec contains specifications for BDD tests.
/tests contains the unit-tests for an application (using a testing
framework, see
here)
NOTE: both /vendor and /support are deprecated since NPM introduced a clean package management. It's recommended to handle all 3rd-party dependencies using NPM and a package.json file
When building a rather large application, I recommend the following additional folders (especially if you are using some kind of MVC- / ORM-Framework like express or mongoose):
/models contains all your ORM models (called Schemas in mongoose)
/views contains your view-templates (using any templating language supported in express)
/public contains all static content (images, style-sheets, client-side JavaScript)
/assets/images contains image files
/assets/pdf contains static pdf files
/css contains style sheets (or compiled output by a css engine)
/js contains client side JavaScript
/controllers contain all your express routes, separated by module/area of your application (note: when using the bootstrapping functionality of express, this folder is called /routes)
I got used to organize my projects this way and i think it works out pretty well.
Update for CoffeeScript-based Express applications (using connect-assets):
/app contains your compiled JavaScript
/assets/ contains all client-side assets that require compilation
/assets/js contains your client-side CoffeeScript files
/assets/css contains all your LESS/Stylus style-sheets
/public/(js|css|img) contains your static files that are not handled by any compilers
/src contains all your server-side specific CoffeeScript files
/test contains all unit testing scripts (implemented using a testing-framework of your choice)
/views contains all your express views (be it jade, ejs or any other templating engine)
There is a discussion on GitHub because of a question similar to this one:
https://gist.github.com/1398757
You can use other projects for guidance, search in GitHub for:
ThreeNodes.js - in my opinion, seems to have a specific structure not suitable for every project;
lighter - an more simple structure, but lacks a bit of organization;
And finally, in a book (http://shop.oreilly.com/product/0636920025344.do) suggests this structure:
├── index.html
├── js/
│ ├── main.js
│ ├── models/
│ ├── views/
│ ├── collections/
│ ├── templates/
│ └── libs/
│ ├── backbone/
│ ├── underscore/
│ └── ...
├── css/
└── ...
More example from my project architecture you can see here:
├── Dockerfile
├── README.md
├── config
│ └── production.json
├── package.json
├── schema
│ ├── create-db.sh
│ ├── db.sql
├── scripts
│ └── deploy-production.sh
├── src
│ ├── app -> Containes API routes
│ ├── db -> DB Models (ORM)
│ └── server.js -> the Server initlializer.
└── test
Basically, the logical app separated to DB and APP folders inside the SRC dir.
Assuming we are talking about web applications and building APIs:
One approach is to categorize files by feature. To illustrate:
We are developing a library application. In the first version of the application, a user can:
Search for books and see metadata of books
Search for authors and see their books
In a second version, users can also:
Create an account and log in
Loan/borrow books
In a third version, users can also:
Save a list of books they want to read/mark favorites
First we have the following structure:
books
└─ entities
│ └─ book.js
│ └─ author.js
│
└─ services
│ └─ booksService.js
│ └─ authorsService.js
│
└─ repositories
│ └─ booksRepository.js
│ └─ authorsRepository.js
│
└─ controllers
│ └─ booksController.js
│ └─ authorsController.js
│
└─ tests
└─ ...
We then add on the user and loan features:
user
└─ controllers
└─ entities
└─ services
└─ ...
loan
└─ controllers
└─ ...
And then the favorites functionality:
favorites
└─ controllers
└─ entities
└─ ...
For any new developer that gets handed the task to add on that the books search should also return information if any book have been marked as favorite, it's really easy to see where in the code he/she should look.
Then when the product owner sweeps in and exclaims that the favorites feature should be removed completely, it's easy to remove it.
It's important to note that there's no consensus on what's the best approach and related frameworks in general do not enforce nor reward certain structures.
I find this to be a frustrating and huge overhead but equally important. It is sort of a downplayed version (but IMO more important) of the style guide issue. I like to point this out because the answer is the same: it doesn't matter what structure you use as long as it's well defined and coherent.
So I'd propose to look for a comprehensive guide that you like and make it clear that the project is based on this.
It's not easy, especially if you're new to this! Expect to spend hours researching. You'll find most guides recommending an MVC-like structure. While several years ago that might have been a solid choice, nowadays that's not necessarily the case. For example here's another approach.
This is indirect answer, on the folder structure itself, very related.
A few years ago I had same question, took a folder structure but had to do a lot directory moving later on, because the folder was meant for a different purpose than that I have read on internet, that is, what a particular folder does has different meanings for different people on some folders.
Now, having done multiple projects, in addition to explanation in all other answers, on the folder structure itself, I would strongly suggest to follow the structure of Node.js itself, which can be seen at: https://github.com/nodejs/node. It has great detail on all, say linters and others, what file and folder structure they have and where. Some folders have a README that explains what is in that folder.
Starting in above structure is good because some day a new requirement comes in and but you will have a scope to improve as it is already followed by Node.js itself which is maintained over many years now.
Just clone the repo from GitHub
https://github.com/abhinavkallungal/Express-Folder-Structure
This is a basic structure of a node.js express.js project
with already setup MongoDB as database, hbs as view engine, nodemon also,
so you can easily set up node js express project
Step 1: download or clone the repo
Step 2: Open in any code editor
Step 3: Open the terminal on the folder path
Step 4: run the comment in terminal npm start
Step 5: start coding
Related
My understanding is that Bazel expects projects to be under a monorepo with a WORKSPACE file at the top-level and BUILD files in every project:
Repo
├── ProjectA
│ └── BUILD
├── ProjectB
│ └── BUILD
└── WORKSPACE
However, going through the Bazel NodeJS rules documentation, it seems to suggest that every project should have it's own WORKSPACE file where it defines its dependencies. i.e. ...
Repo
├── ProjectA
│ ├── BUILD
│ └── WORKSPACE
└── ProjectB
├── BUILD
└── WORKSPACE
This looks similar to a multi-repo with every project referencing other projects as an external dependency, which seemed okay to me, until I realized that for external dependencies, Bazel requires all transitive dependencies to be specified in the WORKSPACE file for every package, which is definitely not ideal.
What's the easiest way to use Bazel with NodeJS projects, with some projects possibly written in other languages? Also, is there an example somewhere for Bazel being used in a multi-repo setting?
Thanks!
I think the 2 possible options are in fact
Repo
├── MyProject
│ └── BUILD
├── third_party
│ └── ProjectB
│ └─ BUILD
└── WORKSPACE
or
Repo
├── MyProject
│ └── BUILD
└── WORKSPACE
where in the second case WORKSPACE references ProjectB with npm_install rule as defined in https://github.com/bazelbuild/rules_nodejs#using-bazel-managed-dependencies
I'm still trying to figure this out myself, but what I've gathered so far is that there is only one WORKSPACE file at the root of the repo. You need to have a package.json file (probably at the root) containing all the dependencies used in the whole repo then call npm_install or yarn_install in the WORKSPACE file to download them all.
Then your package BUILD file can reference a dependency with #npm//some_package as in:
filegroup(
name = 'sources',
srcs = ['index.js'],
)
npm_package(
name = 'pkg',
srcs = [ 'package.json'],
deps = [
':sources'
'#npm//lodash'
],
)
There are a few different dependency edge cases I haven't figured out yet so this may not be perfectly correct. Good luck.
How to start the pyscaffold project?
I use this command to create the project putup sampleProject
but i don't know how to start this project?
You don't start a pyscaffold project per say -- Its goal is simply to create the files and folder that you will commonly need for your project. See my structure below from "putup MyTestProject". Look at all the nice stuff already created that you now don't have to do by hand.
To get started, you need to start adding packages/code to "..src/mytestproject" and run that code like you normally would.
Might I recommend for you the use of a good IDEA, such as pycharm? I think you will find it makes starting your journey much easier.
A second recommendation -- if you are just getting started, you might skip pyscaffold for now. While a great tool, it might add confusion that you don't need right now.
MyTestProject/
├── AUTHORS.rst
├── CHANGELOG.rst
├── docs
│ ├── authors.rst
│ ├── changelog.rst
│ ├── conf.py
│ ├── index.rst
│ ├── license.rst
│ ├── Makefile
│ └── _static
├── LICENSE.txt
├── README.rst
├── requirements.txt
├── setup.cfg
├── setup.py
├── src
│ └── mytestproject
│ ├── __init__.py
│ └── skeleton.py
└── tests
├── conftest.py
└── test_skeleton.py
[Edit]
With respect to why "python skeleton.py" gives an output, the library is simply providing an example to show the user where to start adding code, and how the code relates to the tests (test_skeleton.py). The intent is that skeleton.py will be erased and replaced with your code structure. This may be some python.py files or packages and sub packages with python.py files. Read it this way; "Your Code goes here ... and here is an arbitrary example to get you started."
But you have to ask yourself what you are trying to accomplish? If you are just creating a few scripts for yourself -- for nobody else in the world to see, do you need the additional stuff (docs, setup, licensing, etc?) If the answer is no - don't use pyscaffold, just create your scripts in a venv and be on your way. This scaffolding is meant to give you most of what you need to create a full, github worthy, project to potentially share with the world. Based on what I gather your python experience to be, I don't think you want to use pyscaffold.
But specific to your question. Were I starting with pyscaffold, I would erase skeleton.py, replace it with "mytester.py", use the begins library to parse my incoming command arguments, then write individual methods to respond to my command line calls.
I want to create a library in Typescript that I can share via npm. Specifically, I want to use webpack to generate a js bundle along with a definition file to share the types with the js. So I'd have a tree of files like:
├── lib
│ ├── lib.d.ts
│ └── lib.min.js
├── test
...
├── ts
│ ├── errors
│ │ ├── CannotModifyAlteredObject.ts
│ ├── Lib.ts
│ ├── PostProcessors.ts
│ ├── Serializers.ts
├── tsconfig.json
├── typings.json
├── LICENSE
├── package.json
├── README.md
└── webpack.lib.config.js
And all the types exported by ts/Lib.ts would be exported to a single .d.ts in the lib directory to sit next to the js bundle.
I've looked at the following questions/sources:
Writing npm modules in typescript
How to create a typescript library (and the question it duplicates)
This unanswered question
The offical typescript guide to creating packages
This example typescript library project
And another SO question
However, none of these provide an example using webpack. Being able to bundle everything you need to use the library (apart from the nodejs runtime) into a single file is pretty important for my use case, so webpack fits this role well. I'd like to be able to generate a .d.ts file that maps to what webpack creates. However, I want to avoid creating the .d.ts file manually - it should be possible to automatically extract the types without having manually created .d.ts files get out of sync with my source code. Is there a way of doing this?
Typescript seems to be a better choice for writing code where the code base is huge and requires more consistency.
But the examples and experiences I find around the internet are more on the client side, may be coz of the Angular2.0 decision to use AtScript/Typescript.
Though Typescript adds a lot of power to Javascript, there is something that keeps on pissing me off. How do I manage codebase with duplicate files. My understanding is I have two ways to do it.
1st Way
Use a build tool like gulp/grunt, watch for changes, and compile Typescript to Javascript in the same folder. This may look something like this:
├── models/
│ ├── User.ts
│ ├── User.js
│ ├── Likes.ts
│ ├── Likes.js
2nd Way
Another way will be the same but of instead of outputting Javascript into the same folder, I can clone the entire workspace:
├── typescript
│ ├── models/
│ │ ├── User.ts
│ │ ├── Likes.ts
├── javascript
│ ├── models/
│ │ ├── User.js
│ │ ├── Likes.js
Both are not the best solution for me. Is there any other way to manage codebase for nodejs. For client it is simple as we have only one output file.
Is there any other way to manage codebase for nodejs. For client it is simple as we have only one output file
No. We personally try to go with the second.
With the JS excluded from the REPO and rebuilt pre npm publish.
I'm using package.json file to define nodejs requirements, along with npm update, and of course is working fine.
How can I manage (update the easy way) other third party libraries with node? For example:
https://github.com/documentcloud/backbone.git
https://github.com/twitter/bootstrap.git
In vendor folder.
Summary: I think you want to use http://twitter.github.com/bower/
Details:
There are two ways to understand your question:
how to manage/update non-npm code?
how to manage/update client-side javascript assets?
The question is worded as the former, but from your included examples I think what you want to ask the latter.
In case of server-side code, just insist all code gets shipped with npm-style package.json manifest. If the author of the code is unresponsive, fork it and add the manifest. There is no excuse.
For client-side code, the situation is different. There is no established standard for package management, however it's a widely recognized problem and very active field of development. Several challengers have risen recently, trying to grab the dominant position: BPM, Jam or Ender. It's your call which to pick, they are well summarized here: A package manager for web assets
However, all of the above address a slightly too ambitious problem - they try to sort out the transport of those modules to the browser (via lazy loading, require-js style, dependency resolution, concatenation/minification etc.) This also makes them more difficult to use.
A new entrant to the field is Bower from Twitter. It focuses just on download/update lifecycle in your vendor folder and ignores the browser delivery. I like that approach. You should check it out.
You could go for git submodules:
http://git-scm.com/book/en/Git-Tools-Submodules
Using someone else's repo as a Git Submodule on GitHub
[UPDATE 1]
Do this at the root of your repository:
git submodule add git://github.com/documentcloud/backbone.git vendors/backbone
git submodule add git://github.com/twitter/bootstrap.git vendors/bootstrap
Check this for more: http://skyl.org/log/post/skyl/2009/11/nested-git-repositories-with-github-using-submodule-in-three-minutes/
Although this may not be the nodejs way, and some purists may complain, Composer will do what you want. Even though Composer is used with PHP projects there is no reason why it cannot be used to manage 3rd party non-npm repos for nodejs projects too. Obviously it is preferable for the 3rd party library to include a package.json but thats not aways going to happen. I tried this on my current nodejs app and it worked perfectly for me.
Pros:
One json file specifies custom external dependancies that do not have package.json files
Customize where packages are stored in the project folder
Works with private repos and repos not originally intended for use with nodejs
Cons:
Requires php cli
An extra step for updating dependancies
An extra json file for dependancies
Here how to do it (You need to be able to run php from the cli):
1. Download Composer (directly into your root nodejs project folder)
curl -s https://getcomposer.org/composer.phar > composer.phar
2. Create a composer.json file (in the root of the project)
{
"repositories": [
{
"type": "package",
"package": {
"name": "twitter/bootstrap",
"version": "2.0.0",
"dist": {
"url": "https://github.com/twitter/bootstrap/zipball/master",
"type": "zip"
},
"source": {
"url": "https://github.com/twitter/bootstrap.git",
"type": "git",
"reference": "master"
}
}
}
],
"require": {
"twitter/bootstrap": "2.0.0"
}
}
3. Run the Composer update
php composer.phar update
This will download the package into the vendor folder as you requested:
├── vendor
│ ├── ...
│ ├── composer
│ │ └── installed.json
│ └── twitter
│ └── bootstrap
│ ├── LICENSE
│ ├── Makefile
│ ├── README.md
│ ├── docs
│ │ ├── assets
│ │ └── ...
│ ├── img
│ │ ├── glyphicons-halflings-white.png
│ │ └── glyphicons-halflings.png
│ ├── js
│ │ ├── bootstrap-affix.js
│ │ └── ...
│ ├── less
│ │ ├── accordion.less
│ │ └── ...
│ └── ...