I was implementing a simple web server in Go. As I have no experience in web development, this striked as a serious question for me.
Let's say I'm serving web pages with a modified loadPage function from here
func loadPage(title string) []byte {
filename := title
body, _ := ioutil.ReadFile(filename)
return body
}
func handler(w http.ResponseWriter, req *http.Request) {
content := loadPage(req.URL.Path[1:])
fmt.Fprintf(w, "%s", content)
}
Technically this allows me to write a request in a form of
http://example.com/../../etc/passwd
and the code would happily serve the /etc/passwd file, but it does not. Does this mean that there is some sort of protection against ../ in the Go http package or http protocol itself, or am I just doing something wrong and it is a security hole?
net/http does this in its HTTP request multiplexer, ServeMux:
ServeMux also takes care of sanitizing the URL request path, redirecting any request containing . or .. elements to an equivalent .- and ..-free URL.
The relevant function is the private func cleanPath(p string) string, which calls path.Clean:
1415 np := path.Clean(p)
path.Clean does the appropriate removals:
97 case path[r] == '.' && path[r+1] == '.' && (r+2 == n || path[r+2] == '/'):
98 // .. element: remove to last /
99 r += 2
100 switch {
101 case out.w > dotdot:
102 // can backtrack
103 out.w--
104 for out.w > dotdot && out.index(out.w) != '/' {
105 out.w--
106 }
There's an additional case if the path isn't rooted, but cleanPath above ensures it is so, by prepending a forward-slash to the path to be cleaned if there isn't one already.
Related
I'm using Go with command to execute curl which works as expected
curl := exec.Command("curl", "https://services.odata.org/V3/northwind/northwind.svc/")
out, err := curl.Output()
if err != nil {
fmt.Println("erorr" , err)
return
}
fmt.Println(out)
Now I want to use some placeholders like
curl -O http://quiet-waters-1228.herokuapp.com/assets/image.jpg
but now I need to get the url for command
e.g. if I run in bash mytool url I got the url value
`curl -O $(mytool url)`
The problem is that we need to execute the command in the code and I'm not sure how to pass it
curl := exec.Command("curl", "curl -O $(url)")
out, err := curl.Output()
if err != nil {
fmt.Println("erorr" , err)
return
}
fmt.Println(out)
In os package you have slice of strings which contains all arguments passed by shell to your program.
os.Args 0th value, i.e., first element is going to be name of the command itself.
If your tool command is mytool, os.Args[0] contains mytool.
Rest are going to be the arguments, which are passed by shell.
package main
import (
"log"
"os"
"os/exec"
)
func main() {
if len(os.Args) < 2 {
// If argument is not provided quit
log.Fatalln("url not provided")
}
url := os.Args[1] // URL
cmd := exec.Command("curl", "-O", url)
cmd.Run()
}
You can also download multiple URLs concurrently,
var wg *sync.WaitGroup
func main() {
urls := os.Args[1:]
wg = new(sync.WaitGroup)
wg.Add(len(urls))
for _, url := range urls {
go download(url)
}
wg.Wait()
}
func download(url string) {
defer wg.Done()
cmd := exec.Command("curl", "-O", url)
cmd.Run()
}
I just started to get into golang and as I plan to host at least two websites, I chose to use Mux to display different routes by "filtering" domains. Whenever I try to access my main route, it just gives me an 404 error. (Also, the fact that the "www" part is absent is perfectly normal. I don't type that to access the site).
But if I launch the server as a file server, I can access my files, so the server in itself is working I guess
func redirect(w http.ResponseWriter, req *http.Request) {
target := "https://" + req.Host + req.URL.Path
http.Redirect(w, req, target,
http.StatusTemporaryRedirect)
}
func main() {
go http.ListenAndServe(":80", http.HandlerFunc(redirect)) // Redirection
// Serveur sécurisé
r := mux.NewRouter()
r.PathPrefix("/static/").Handler(http.StripPrefix("/static/", http.FileServer(http.Dir("/root/go/src/web/static/"))))
s := r.Host("echecderi.me").Subrouter()
s.HandleFunc("/", indexEchec)
http.ListenAndServeTLS(":443", "domain-crt.pem", "domain-key.pem", nil)
}
func indexEchec(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "<h1>Echec de rime</h1> </br> <img src=\"/static/echecderime/echec.gif\">")
}
I think you need to give r as the last parameter to http.ListenAndServeTLS.
you can also use a http.server instance
//create server instance
server := http.Server{
Addr: ":443",
TLSConfig: tlsConfig(cert),
}
rtr := mux.NewRouter()
rtr.HandleFunc("/profile", HandlerProfile).Methods("GET")
//rtr.HandleFunc( other routes...
//pass mux handler to server
server.Handler = rtr
server.ListenAndServeTLS("", "")
I have a node server running, and I am trying to figure out how to convey the paths to files on my servers (and how to respond to get requests for these resources) so that I basically have a static file server, but one that I can control based on request parameters (POST or GET etc.). Right now my file structure is set up as follows (dir_ means directory):
Main Folder:
server.js
dir_content:
home.html
style.css
dir_uploads:
dir_finished:
file1.txt
file2.txt
To respond to my requests, my code looks like the following:
http.createServer(function(request, response) {
if(request.method.toLowerCase() == 'get') {
var filePath = './dir_content' + request.url;
if (filePath == './dir_content/') {
filePath = './dir_content/home.html';
}
fs.exists(filePath, function (exists) {
if (exists) {
fs.readFile(filePath, function (error, content) {
if (error) {
response.writeHead(500);
response.end();
}
else {
response.writeHead(200, {'Content-Type': contentType});
response.end(content, 'utf-8');
}
})
This allows me to respond to any GET requests for web pages with the correct page (if it exists) but it eventually warps the path to a resource on my server.
Example: Someone trying to retrieve file1.txt would navigate to localhost:8080/dir_content/dir_uploads/dir_finished/file1.txt
but my code would add the additional ./dir_content to their request, making it look like they were trying to visit:
localhost:8080/dir_content/dir_content/dir_uploads/dir_finished/file1.txt
Is there a simpler way to provide accurate absolute paths to resources in folders WITHOUT external node modules (by setting a sort of base directory somehow)?
At first glance it appears you aren't accounting for the request.url possibly including the dir_content prefix. If that is indeed true, then I would recommend a simple regular expression test to see if it is there:
if (!/^[\/]?dir_content\//i.test(request.url))
filePath = '/dir_content' + request.url;
filePath = '.' + filePath;
Simple fiddle to demonstrate the regular expression:
http://jsfiddle.net/97Q43/
In my node.js app I have functions which can be passed either
OS-style paths e.g. c:\my\docs\mydoc.doc (or /usr/docs/mydoc.doc or whatever is local)
File URLS e.g. file://c:/my/docs/mydoc.doc (which I'm not sure about the validity of '\'s in??)
Either way, I need to check to see if they refer to a specific location which will always exist as a local OS-style path e.g. c:\mydata\directory\ or /usr/mydata/directory
Obviously for OS-style paths I can just compare them as strings - they SHOULD always be the same (they're created with path) but FILE:// URLS don't necessarily use path.sep and so won't "string match"?
Any suggestions as to the best way to handle this (I'm personally tempted to break EVERYTHING by one-or-more slashes of either sort and then check each piece??
I'm going to post my own take on this - as it came from a suggestion I got from someone on Facebook (no - really!) and it uses path in a way it probably wasn't intended for - e.g. I'm not sure it's the right 'solution' - Im not sure I'm not exploiting path a bit.
The Facebook tip was that path is really just a utility for handing strings with "/" and "\" separators - it ignores everything else - doesn't care what's in there at all.
On that basis, we can use
path.normalize(ourpath)
which will convert all separators to the local OS preferred ones (path.sep)
That means they will match my OS-style directory (which is also made with path) and so I can compare those - without resorting to manually gutting-out slashes...
e.g.
Before
file://awkward/use/of\\slashes\in/this/path
After
file:\awkward\use\of\slashes\in\this\path (Windows)
or
file:/awkward/use/of/slashes/in/this/path (everywhere else)
Removing file:// before (or file: + path.sep after) = local OS-style path!?
Just do some manipulation of the string and check to make sure they are the same after correcting for the differences:
var path = require('path');
var pathname = "\\usr\\home\\newbeb01\\Desktop\\testinput.txt";
var pathname2 = "file://usr/home/newbeb01/Desktop/testinput.txt"
if(PreparePathNameForComparing(pathname) == PreparePathNameForComparing(pathname2))
{ console.log("Same path"); }
else
{ console.log("Not the same path"); }
function PreparePathNameForComparing(pathname)
{
var returnString = pathname;
//try to find the file uri prefix, if there strip it off
if(pathname.search("file://") != -1 || pathname.search("FILE://") != -1)
{ returnString = pathname.substring(6, pathname.length); }
//now make all slashes the same
if(path.sep === '\\') //replace all '/' with '\\'
{ returnString = returnString.replace(/\//g, '\\'); }
else //replace all '\\' with '/'
{ returnString = returnString.replace(/\\/g, '/'); }
return returnString;
}
I checked to see if the URI path name indicator "file://" was there, if so, I deleted it off my compare string. Then I normalized based on the path separator node path module will give me. This way it should work in either Linux or Windows environment.
I want to use Jade templates at the client side with Backbone. How can I do that?
For now, I have successfully configured Backbone (Marionette) to compile Jade templates for use in its Views:
Marionette.TemplateCache.prototype.compileTemplate = (tmplStr) ->
console.log "jade stuff: ", jade.compile(tmplStr)
return jade.compile(tmplStr)
The "problem" is: I am currently writing templates like:
script(type="text/template", id="tmplMainView")
| h1= title
| p= content
Notice the pipes (|) those are to prevent Jade from trying to interpret/parse them on server side. How can I eliminate those?
UPDATE
Perhaps I can use the jade --client flag ... but it gives a single compiled function: for example
h1= title
Becomes
function anonymous(locals, attrs, escape, rethrow, merge) {
attrs = attrs || jade.attrs; escape = escape || jade.escape; rethrow = rethrow || jade.rethrow; merge = merge || jade.merge;
var buf = [];
with (locals || {}) {
var interp;
buf.push('<h1>');
var __val__ = title
buf.push(escape(null == __val__ ? "" : __val__));
buf.push('</h1>');
}
return buf.join("");
}
That means I have to have 1 Jade/compiled JS for each template? How might I use it? Also I think many JS files is a slow way to work? But since template functions are all named anonymous, how can I then concat or somehow work with them effectively?
Check the ClientJade project.
From their site:
clientjade is a command line tool to compile your jade templates into client side templates for use in the browser. It will automatically include everything you need to render the templates, no need to include jade.js or runtime.js.
$ clientjade test1.jade test2.jade > templates.js
And then include template.js file in your html. To render the templates, just make a call like this:
//jade.render(domNode, templateName, data);
jade.render(document.getElementById('test1'), 'test1', { name: 'Bob' });
After looking at Jadify and ClientJade, and encountering a few problems along the way ... (perhaps its just somethings I missed out), I decided to explore simply compiling the templates on server side.
I defined a Node module (used by ExpressJS) which does the compilation and returns the compiled JS source (which I served with /js/templates.js).
fs = require "fs"
jade = require "jade"
async = require "async"
# done callback will be passed (source, err)
exports.compile = (done, templatesDir) ->
js = "var Templates = {}; \n\n"
# get all files in templates directory
fs.readdir templatesDir, (err, files) ->
# keep only ".jade" files
jadeFiles = files.filter (file) ->
file.substr(-5) == ".jade"
# function to compile jade templates (appending to js source)
compileTmpl = (file, doneCompile) ->
# "test.jade" becomes "test"
key = file.substr(0, file.indexOf("."))
filePath = templatesDir + file
fs.readFile filePath, (err, src) ->
# store js function source into Templates.{key}
js += "Templates." + key + " = " + jade.compile(src, { debug: false, client: true }).toString() + "; \n\n"
doneCompile(err)
# foreach jadeFile, compile template, then write templates.js file
async.forEach jadeFiles, compileTmpl, (err) ->
done(js, err)
And I use the precompiled templates on client side by including the templates.js and use templates like:
Templates.tmpl1()
Templates.tmpl2({ something: "Hello world", ... })
More on https://coderwall.com/p/hlojkw