Logstash has an open port where everyone can send in data.
Anonymous data messes everything up
All data from all customers in one pool is messed up too.
So I read and tried https://www.elastic.co/guide/en/x-pack/5.6/logstash.html (also for Version 6 and 7) but this does not seem to be it. It rather authenticates against elasticsearch than in front of logstash. What I like to have is some sort of this:
input {
# One port to rule them all - possible?
tcp {
port => 5000
}
}
output {
elasticsearch {
hosts => "elasticsearch:9200"
user => elastic
password => nope
document_id => "%{[#metadata][fingerprint]}"
# Here comes the user prefix again.
index => "%{[user]}-%{[host]}-%{+YYYY.MM.dd}"
}
if [user] == "foo" {
# Also put things in IRC
}
if [user] == "qux" {
# Forward somewhere else
}
}
In the end we like to have the data separated available when using Elastic or Kibana which may be no big deal when I read the documentation. But I also think that there should be some auth in front of logstash. Correct me if I am wrong.
If not:
How to prevent anonymous data?
How to distinguish them well?
Since you can't change the original message, you will need to filter based on the host that sent that message, you can use the host field for that, as it seems you are already doing on your output.
To prevent anonymous data you can use SSL, so each machine sending to your logstash will need to have the certificate files, or you can configure a firewall on the logstash machine and configure it to allow connections from your costumers only.
The x-pack security feature that you mentioned is a paid feature, but its focus is on putting a security layer on elasticsearch/kibana, there is no auth in front of logstash, to do that you need firewall rules and/or ssl certificates.
Related
TL;DR: How to detect local website users?
I have a self-hosted website running in the student-building I live in. In this website I would like a page for and links to certain local applications, like the webremote of the RPi running Kodi, an FTP, a page of instructions etc.
I don't want those to be visible to random internet users, so is there any way for a website to detect whether the user is accessing the website from inside the local network? Preferably in JavaScript, but PHP would also be fine.
in PHP it can be done in several ways. The simplest way is to do a simple IP check.
if ($_SERVER['REMOTE_ADDR'] = "10.1.0.25") { // Internet IP or IP rangehere
// Show links for internal users
} else {
// Show stuff for all other users
}
or for range of IPs assuming 192.168.1.x addresses
if ($_SERVER['REMOTE_ADDR'] >= "192.168.1.1" && $_SERVER['REMOTE_ADDR'] <= "192.168.1.1") {
// Internal Info
} else {
// External Info
}
I am trying to develop a chrome extension that would set the "host" header on certain requests. But the documentation is contradicting as to if the "host" header can be modified or not.
Both of these issues indicate that a) it should not be possible and b) it is impossible
https://code.google.com/p/chromium/issues/detail?id=154900
https://code.google.com/p/chromium/issues/detail?id=158073
Yet multiple extensions in the gallery state they do modify the "host" header.
e.g.
https://chrome.google.com/webstore/detail/header-hacker/phnffahgegfkcobeaapbenpmdnkifigc?hl=en
https://chrome.google.com/webstore/detail/change-http-request-heade/ppmibgfeefcglejjlpeihfdimbkfbbnm
Is it possible to modify the "host" header in the windows version of chrome, and if so how?
Background: I want to be able to test load balanced web instances hitting each host directly via ip address. The "hosts" file is to cumbersome for a large number of hosts. At the moment I use curl to pass the modified "host" header, but I really need the solution in the browser and available for others.
#kzahel was right, and the note about redirection was spot on, here is my basic working code.
chrome.webRequest.onBeforeSendHeaders.addListener(function (details) {
if (details.url.indexOf('toast.txt') <= -1)
return;
details.requestHeaders.push({
name: 'Host',
value: 'testhost:80'
});
return { requestHeaders: details.requestHeaders };
}, {
urls: ['http://*/*']
}, ['requestHeaders', 'blocking']);
chrome.webRequest.onBeforeRequest.addListener(function (details) {
if (details.url.indexOf('sauce') <= -1)
return;
var url = 'http://127.0.0.1/toast.txt';
return { redirectUrl: url };
}, {
urls: ['http://*/*']
}, ['blocking']);
Admittedly a slightly odd example but it goes like this.
My local IIS has a site created that points to a folder that has a file "toast.txt", which is bound to "testhost".
Windows can no way of knowing about "testhost" e.g. cannot ping it.
With the extension running.
Enter the address http://testhost/sauce
The extension notes the "sauce" in the "onBeforeRequest" method and redirects to "http://127.0.0.1/toast.txt" which in turn is picked up on by the "onBeforeSendHeaders" method where the "Host" header is added containing "testhost:80". All this occurs before the request leaves Chrome.
Now IIS receives the request "/toast.txt" that by itself would result in a 404, but because the "Host" header is set it pushes the request to the new website that is bound to "testhost" which then returns the "toast.txt" file.
Phew!
It looks like you shouldn't have difficulty doing this. Use onBeforeRequest
onBeforeRequest: Fires when a request is about to occur. This event is sent before any TCP connection is made and can be used to cancel or redirect requests.
Since this is triggered before any connection to the server is made, you should be able to modify the host header then [edit: if host header is not available, then use a redirect]. Make sure you have the "requestHeaders" permission in the manifest or else you won't see the request headers at all.
I have an interesting issue I've been trying to resolve for a few days.
I'm currently working with an Windows Server 2003 machine that is running a standard instance of Active Directory.
The directory contains two domain components (DCs) that both house users that are going to be authorizing against the directory, via my application.
I'm using :
The IP address of the server as the host name
An SSL connection via port 3269
The GSS Negotiate Auth Mechanism
A BaseDN that is a parentDN of both DC's
The sAMAccountName as the login name
The problem is, I cannot successfully authorize any users from DC1, yet all of the ones who belong to DC2 are completely fine and work great. I get this error on DC1 :
8009030C: LdapErr: DSID-0C09043E, comment: AcceptSecurityContext error, data 0, vece
System.DirectoryServices.Protocols.LdapException: The supplied credential is invalid.
However, using Softerra's LDAP Broswer, I can connect in and authorize the same exact user without any issue, so I know the credentials are correct.
From what I can tell, both of these DC's are configured the same... I've browsed both of them for something, anything that is different... but have found nothing that really stands out.
I posted something months ago about this particular setup, and the code I'm using is in that thread as well.
Set callback for System.DirectoryServices.DirectoryEntry to handle self-signed SSL certificate?
Any help here would be much appreciated.
Thanks!
I was able to get this working, but for the life of me I cannot figure out why this was the case. Basically, this error...
8009030C: LdapErr: DSID-0C09043E, comment: AcceptSecurityContext error, data 0, vece System.DirectoryServices.Protocols.LdapException: The supplied credential is invalid.
...was dead on. The issue was that users logging in under what I called DC2 needed to issue the bind with the domain AND sAMAccountName (Ex. LIB\JSmith), as opposed to DC1, which allowed just the sAMAccountName to be entered.
I figured the best way to make this programmatic was to use the principal binding account to query for the DN of the user. From that DN, using some crafty RegEx, I'm able to capture the domain they inherit from, and issue two separate binds.
SearchResultEntry ResultEntry = userResponse.Entries[0];
//Let's get the root domain of the user now using our DN RegEx and that search result
Regex RegexForBaseDN = new Regex(config.LdapAuth.LdapDnRegex);
Match match = RegexForBaseDN.Match(ResultEntry.DistinguishedName);
string domain = match.Groups[1].Value;
//Try binding the user with their domain\username
try
{
var thisUser = new NetworkCredential{
Domain = domain,
UserName = username,
Password = Pin
};
//If this goes well, we'll continue forward
ldapconn.Bind(thisUser);
}
//If that doesn't work, try biding them with the highest level domain
catch (LdapException ex)
{
if (ex.ErrorCode.Equals(LdapErrorCodes.LDAP_INVALID_CREDENTIALS))
{
var thisUserOnce = new NetworkCredential{
Domain = config.LdapAuth.LdapDomain,
UserName = username,
Password = Pin
};
//If this goes well, we'll continue forward
ldapconn.Bind(thisUserOnce);
}
}
It's not nearly as elegant as I would have wanted, but it does work for this particular scenario.
However, I'm still really interested in why the naming conventions are different depending on which DC the user inherit's from.
I'm having trouble finding documentation on the request object argument used in replication filters ('req' in the sample below):
function(doc, req) {
// what is inside req???
return false;
}
This old CouchBase blog post has a little code snippet that shows the userCtx variable being a part of the request object:
What is this userCtx? When you make an authenticated request against
CouchDB, either using HTTP basic auth, secure cookie auth or OAuth,
CouchDB will verify the user’s credentials. If they match a CouchDB
user, it populates the req.userCtx object with information about the
user.
This userCtx object is extremely useful for restricting replication of documents to the owner of the document. Check out this example:
function(doc, req) {
// require a valid request user that owns the current doc
if (!req.userCtx.name) {
throw("Unauthorized!");
}
if(req.userCtx.name == doc.owner) {
return true;
}
return false;
}
But the problem now is that CouchDB requires the filter method to be explicitly chosen by the initiator of the replication (in this case, the initiator is a mobile user of my web app):
curl -X POST http://127.0.0.1:5984/_replicate \
-d '{"source":"database", \
"target":"http://example.com:5984/database", \
"filter":"example/filtername"
}'
The Question
Is there a way to enforce a specific filter by default so that users are restricted to replicating only their own data? I'm thinking the best way to do this is to use a front end to CouchDB, like Nginx, and restrict all replication requests to ones that include that filter. Thoughts? Would love a way to do this without another layer in front of CouchDB.
Data replication stands right with user ability to read data. Since if your users shares data within single database all of them has right to replicate all of them to their local couches. So you couldn't apply any documents read restriction unless you've split single shared database into several personal ones - this is common use case for such situations.
There is no any way to enforce apply changes feed filter or other parameters like views has. However, you can use rewrites to wraps requests to some resources with predefined query parameters or even with dynamic ones. This is a little not solution that you'd expected, but still better that nginx and some logic at his side: probably, you'd to allow users to specify custom filters with custom query parameters and enforce you're own only if nothing specified, right?
P.S. Inside req object is very useful about current request. Partially it was described at wiki, but it's a little out of date. However, it's easily to view it with simple show function:
function(doc, req){
return {json: req}
}
But please bear with me. I do not need help with ndns or JavaScript. I need help with DNS Resource Records.
I can already send resource records. I just need to know how to send the right ones for an Authoritative DNS Server.
I am writing the DNS server using ndns. Ndns is supposed to do the low level communications for me, but I still have to know the basics of DNS. Ndns is not documented except for this example. It is in JavaScript, but it should be pretty easy to read anyway. When a request is received, it adds a resource record to the response and sends the response
function handleDnsRequest(request, response) {
response.addRR(
ndns.ns_s.ar, // Section AR
'node.js', // Name
ndns.ns_t.txt, // Type TXT
ndns.ns_c.in, // Class IN
1991, // TTL
'http://nodejs.org/' // Value
);
response.send();
}
So, no matter what the request, this handler adds a response record as follows
Section AR (Additional Records)
Name "node.js"
Type TXT (Text String)
Class IN (Internet)
TTL 1991 (~33 minutes)
Value (Text String)
Which gives this output on Windows nslookup
C:\>nslookup - 127.0.0.1
node.js text =
"http://nodejs.org/"
Default Server: UnKnown
Address: 127.0.0.1
> google.com
Server: UnKnown
Address: 127.0.0.1
Name: google.com
>
How can I send correct responses? I want to start off by sending a fixed IP address for all A records no matter what and to deny most everything else as unsupported or whatnot.
In a typical log in to nslookup, ask for an a record What would be the typical list of Resource Records that would come out of the DNS server?
I want to start off by sending a fixed
IP address for all A records no matter
what and to deny most everything else
as unsupported or whatnot.
Aha, now we're getting somewhere.
You need to return an RR in the answer section that has the same "owner name" as that in the (first) question, with the appropriate fields.
Try this:
function listener (req, res)
{
res.addRR(
ndns.ns_s.an, // answer section
req.question[0].name, // name
ndns.ns_t.a, // type
ndns.ns_c.in, // class
3600, // TTL
'127.0.0.1' // RDATA
);
res.header.aa = 1; // authoritative answer
res.header.ra = 0; // recursion not available
res.send ();
}
This only handles the default response, and doesn't check whether the inbound query was for an A record or not.
To refuse other queries you'll want to check for:
req.question.length == 1
req.question[0].type == ndns.ns_t.a
req.question[0].class == ndns.ns_c.in
and then set res.header.rcode to something non-zero.
A real authoritative server would also send DNS server names in the authority section, but you should be able to get away without doing so here.