Where to place robots.txt - Nodejs express project - node.js

Can you please tell me where to place robots.txt and sitemap.xl in nodejs express project folder structure so that it will be available to search engines.
I placed it along with app.js and inside public folder,but i got "Cannot GET /sitemap.xml" error in both the case

Have you set the route for your public folder, the one that is shared or served to the public?
like for example:
app.use(express.static('public'))
All the content inside the folder: "public" like html,css or robots.txt will be served.

Related

Can someone please explain how static files in express js work?

I am new with nodejs and much more new with express module. I have a app setup like this;
The chart.js is my nodejs file. I am trying to make my js files and css file static by using app.use(express.static(-I didn't understand what i need to write here-)) in order to render my index.html properly but I don't know how to use and I did not understand the documentation. In the documentation they say coder able to use static like app.use(express.static('public')) but they don't mention about what is public, where it is in the project, what does it contain. Can someone please help mi about this situation? How does this express.static works and how can I make my files static?
NOTE: DO NOT PUT PRIVATE FILES INSIDE YOUR STATIC FOLDER.
app.use(express.static(__dirname + '/public'));
here you see inside of express.static() function is the path of your static folder that will be going to access directly from the browser and you don't need to write their routes because that folder will give all the access to the public. like css,js files. and those files you will be able to access as its directory.
in the above picture, you have html, css and js files in public folder which is located on root folder of the application. you need to access those public static files which are not related to nodejs so it should be defined as static on your server node js code as : app.use(express.static(__dirname + '/public'));. and it will get all the routes like:
http://localhost:3000/css/style.css
http://localhost:3000/javascript/script.js
http://localhost:3000/favicon.ico
http://localhost:3000/index.html
http://localhost:3000/robots.txt
you also can set prefix for those static routes. for that, you need to give prefix as: app.use('static_folder', express.static(__dirname + '/public')); then it will be looks like :
http://localhost:3000/static_folder/css/style.css
http://localhost:3000/static_folder/javascript/script.js
http://localhost:3000/static_folder/favicon.ico
http://localhost:3000/static_folder/index.html
http://localhost:3000/static_folder/robots.txt

It doesn't seem to be getting my index.html file in my views directory. I get an error on the localhost 3000, cannot get

You can view my files in github at https://github.com/elmoreka/myTaskLIst.git.
It doesn't seem to be getting my index.html file in my views directory. I get an error on the localhost 3000, cannot get .
The index.html is not served because the path used for res.render() is incorrect.
In your file routes/index.js, the rendered path is:
res.render('views/index.html');
However, as the view root is defined as myTaskList/views, the above code would try to find file myTaskList/views/views/index.html, which does not exist. To fix this issue, the rendered path need to be as:
res.render('index.html');
As per Expressjs, index.html is a normal static file in your case, that needs to be sent from the server and rendered at the client(browser). Since you haven't used any template engine (ejs) for your index page then the correct expressjs API to be used is
res.sendFile('index.html');
and not
res.render('index.html'); // this is not ejs template file.
In server.js you are defining "client" as your static folder:
// Set Static folder
app.use(express.static(path.join(__dirname, 'client')));
If you want to serve a static file like your index.html you need to move it to your static folder "client". Then you can send it to the client with res.sendFile().

Serve Static content to subdomain with express?

I have an express app, which is just the default blank app. I have then added the line:
app.use(serveStatic('docs/public', {'index': ['index.html', 'index.htm']}))
to serve the contents of my docs/public directory. This works great, but it is being served to the root of my application, so I no longer see the default express index page.
I would like to see the static html (which I am currently seeing as my index) as a subdomain, e.g. blog.mydomain.com. Or at least as mydomain.com/blog. How do I serve the static content to a subdomain?
NB: the static file names and folders can't change, as my ./docs directory is a hexo project and any changes would break the generation of the static content in the docs/public folder.
I have tried to use express-subdomain so I added code like this to my app.js:
var router = express.Router();
router.use(serveStatic('docs/public', {'index': ['index.html', 'index.htm']}))
app.use(subdomain('docs', router));
However, If I run that, I get the express index page at http://localhost:3000/ (not the static one) which is what I want, but if I go to http://docs.localhost:3000/ then I don't get my static content, still the original express index. In fact, if I remove the subdomain code nd run the app, navigate tot he docs subdomain I still get the same result.
I have also tried to use the subdomain module:
app.use(subdomain('docs', serveStatic('docs/public', {'index': ['index.html', 'index.htm']})));
However, that has the same result as above (not serving any static content).
So how can I serve the static content I am currently serving on a subdomain?
#GeorgeEdwards In your example code you've called the static middleware before the subdomain middleware. The order is important... it should be something like this:
app.use(subdomain('docs', express.static('docs/public')));
This means any requests with the docs subdomain will be handled by the express static middleware.

How to include static html pages into my NodeJS project

I have a NodeJS project setup. I want to integrate my static html pages into it. Where can I do it ?
Thank you !
To expose static html, create a directory named "public" that lives next to your primary application file (app.js).
Assuming you are using the express framework, you can expose this directory by using the following code:
app.use(express.static(__dirname + "/public"));
where
var app = express();
Subsequently, any request to "/{something}" will attempt to resolve a route to a static file in your public directory, so if you had a file named {something} in that directory it would get served right up.

How do I prevent Node.js / Express serving up my application's source code?

I never thought this would be a problem with Node.js and Express, but on a crazy whim I decided to type into a browser the location of one of the source files in my Node.js Express project - something like:
http://www.mywebsite.com/mynodejsapp/app.js
To my extreme horror, my application's source code popped right up, publicly available for all to see.
So, that aside: how do I stop it in Node.js / Express?
My setup code is pretty straightforward:
var app = express();
app.configure(function() {
app.use(express.static('/home/prod/server/app/public'));
});
app.listen(8888);
To clarify, this is what my folder structure looks like:
/home/prod/server/
/home/prod/server/app.js
/home/prod/server/public/
All sorts of various files that are intended for public access live under /public. All of my server source code lives under /server/, and my understanding of Express's static folder configuration is that the static folder is the only place that Express happily serves up files from the filesystem from.
Any ideas?
From what you posted it really smells like the URL you entered is served by e.g. Apache/nginx/... and you did put your node app within the document root. The answer is simple in this (and any similar) case:
You never put any of your sourcecode files within the document root or another HTTP-accessible folder. In your case, /home/prod/server/app/public should contain only client-side stuff (HTML, CSS, Graphics, (minified) client-side JS) and nginx should not have anything above this folder as its document root.

Resources