We want to add firewall block rules automatically after detecting malicious IPs on pfsense. The time spent manually on this issue is going to waste in most cases. We're looking for ways to automate this and looking for custom packages on pfsense. There are tools like suricata that can act as detection systems, some python scripts on github and easyrule on pfsense(last 2 are for shell).
What we actually want to do is create a database to store malicious IPs, and a package or script pulling IPs from this database to create firewall rules automatically. How should we approach this? Or is there any real world examples that people use with firewalls(especially pfsense)?
You don't need to make a IP list by yourself. There are many of them available on the internet.
You can use the following lists:
https://feodotracker.abuse.ch/downloads/ipblocklist.txt
https://sslbl.abuse.ch/blacklist/sslipblacklist.txt
https://cinsarmy.com/list/ci-badguys.txt
http://www.talosintelligence.com/documents/ip-blacklist
Once you have the IP lists then use pfBlockerNG to block them.
Related
Guys I really want to know how to hide my IP Address. And know why is it necessary to hide it?
I tried using anonymox, but I want to hide it without using any third party software.
You could try the methods listed here : https://pc4u.org/windows-10-how-to-connect-to-a-free-vpn-without-going-through-third-party-software/ if you don't want to use a third party software to hide your IP Address. You need to setup a VPN in your computer to achieve that. This will tunnel your network connection.
Source: pc4u.org
The only way to accomplish this without 3rd party software would be to use an online proxy and configure your browser to use said proxy. This will only change your IP as it appears to sites you visit through the browser and not other services you may be using on your computer.
The "best" way to "hide" or change your IP is to use a VPN (which you'll need software for). You can purchase a VPN service from one of Many providers. Some of the providers have their own apps that you can use, or, you can manually configure your own OpenVPN client. There are also many great scripts out there, AWS one-click servers, and cheap VPS providers that make it easier than ever to create your own VPN server. This might be over the "average" users head though...
The benefit of using a VPN is that it not only changes your IP, also encrypt your traffic.
It should be noted that VPN's are not 100% fool-proof. If not configured properly, you will expose your real IP. In addition, many VPN providers are not reputable.
The best recommendation I can make on this and every other topic on online privacy is this site here:
https://privacytools.io
This is (IMO) the best, most comprehensive source of information about protecting your privacy online. They will guide you in the right direction regarding VPN's, proxies, securing your browser, and much, much more. Check it out. Seriously...
OpenVPN has builds for all operating systems. https://openvpn.net
Like i said though, you'll have to purchase access (or if you're brave, find a free 'solution') from a provider and then configure OpenVPN to use your purchased credentials. This is usually about 5$ a month (for the solid / no logs / unlimited bandwidth ones ). There are many, many posts about setting up OpenVPN here on stackoverflow.
Finally, as far as the proxies go, again, you can purchase access to some of the reputable ones or search for a free one - though, in my experience the free proxies are very touch and go.
I've been reading on SpamAssassin for some time now and learned a lot but I cannot seem to figure out a way I cannot find a way to create a rule where a 3rd party script can be executed to for a custom rule.
THis would have to be something user based not globally.
I want to run additional verifications on domains and email addresses.
I wish to build a reputation system in which if a domain or email address are checked against the contacts list and other things.
I have have considered modifying the profile to add regex rules but that seems like a way to complicated way of doing it. A more preferred scenario would be to simply run a 3rd party script that returns the score for each domain and email address.
Out of the box, SpamAssassin has no such facility, but since you ask on a programming site, I assume you are not alien to writing some code on your own.
The plugin facility in SpamAssassin was designed for this sort of thing. You can create a piece of Perl code which gets called for each message which SpamAssassin analyzes, and you have access to everything Perl has access to.
In particular, look at the pyzor plugin which calls an external program and returns its analysis results to SpamAssassin. There's a fair amount of boilerplate there, but the part you need to start with is getting the right arguments to the helper_app_pipe_open call (on line 282 as of version 3.4.0, which is what I link to above). These things are configurable so you could perhaps even just reconfigure the path to pyzor to your own program as a proof of concept. Note that it needs to accept a check argument and some other parameters, and a message from a temporary file on its standard input.
Mail::SpamAssassin::Plugin.pm contains POD documentation for the plugin API. Other files in the module tree contain useful documentation too; in particular, you might want to refer to the general documentation in Mail::SpamAssassin.pm and Mail::SpamAssassin::Conf.pm to understand the configuration parameters you can pass to your plugin.
Out of the box, there is a new TxRep plugin that automatically recognizes senders you've seen recently. There is also a collection of whitelist and blacklist options.
If you wanted to implement something yourself, I think you'll quickly find that an exec mechanism won't scale well. Perhaps try crafting your own DNSBL instead. This can be done with custom code and any DNS server (e.g. bind, dnsmasq, etc) or with a DNS server designed for this purpose, such as RBLDNSD. The SA wiki on DnsBlocklists has directions for how to hook it into SA.
Usually, people seeking this kind of solution don't have DNSBLs configured properly. I'd take a look into that before trying to build your own project.
I have a few domains hosted on the same server. These domains contain very similar types of data(similar to stackoverflow.com, stackexchange.com, serverfault.com). They all have the same idea, except minor things are different like images, titles, etc.
How can I use the same base code(like a php/javascript include, or even database) on each of these domains without copy/paste and having many copies to update when i make a change?
any advice would be great!
edit:
for a novice like myself in this department, where would I look to set variables in the virtual host which would tell the code which template/database my code should use for my specific host?
What kind of server setup or access would i need to have for this to be configured properly?
I am using PHP and MySQL for my sites.
If a thorough answer cannot be supplied, direct me as to how I can contact my hosting provider in such a way that they can understand and help me configure what I desire!
Set a variable in the virtual host that tells the code which template, database, etc. the code should use for that specific virtual host.
We run a reverse proxy in front of our application tier and I'm wondering where the "best practice" place for handling the IP restriction is.
Currently, we use the application security to restrict access to specific resources by IP address but this has caused some issues when we moved to running behind a reverse proxy. It's quite easy to configure the allow/deny rules at the proxy instead of the application but since we run multiple applications behind the proxy, making modifications to the config there has the potential to affect other application (not a huge danger, but still present).
Is it better to do the filter further up the chain or closer to the application?
Are there any gotchas, like what we've encountered by doing application restriction and adding a reverse proxy where all the requests "come from" the proxy, forcing us to use a header to find the "real" IP address.
We filter as early as possible and keep it away from the application; these sort of things are better managed by network operations. The reason being is that app developers or maintainers are not always in on the loop when changing ip addresses and the network ops people are usually the first to know. Also network type tools are usually better at providing / restricting access that software level tools.
I would never restrict by IP address. Restrictions like that are the job of a security layer, not of the Network layer, which is where IP addresses live. I rarely find value in having an Application restrict the implementation of the Network.
This depends on the type of resources that need to be restricted by IP. If parts of the application need to be restricted via IP then the application should be handling it. If the entire application needs to be blocked then you should be further up the chain.
The general rule is to restrict as early as possible without compromising any audit systems you have in place (it is almost always a good idea to know when people try to break your security system).
I restrict by IP addresses as early as possible - this eliminates unnecessary traffic in the following layers or subnetworks. So my advice is similar to u07ch's do it as early as possible.
Folks, we all know that IP blacklisting doesn't work - spammers can come in through a proxy, plus, legitimate users might get affected... That said, blacklisting seems to me to be an efficient mechanism to stop a persistent attacker, given that the actual list of IP's is determined dynamically, based on application's feedback and user behavior.
For example:
- someone trying to brute-force your login screen
- a poorly written bot issues very strange HTTP requests to your site
- a script-kiddie uses a scanner to look for vulnerabilities in your app
I'm wondering if the following mechanism would work, and if so, do you know if there are any tools that do it:
In a web application, developer has a hook to report an "offense". An offense can be minor (invalid password) and it would take dozens of such offenses to get blacklisted; or it can be major, and a couple of such offenses in a 24-hour period kicks you out.
Some form of a web-server-level block kicks in on before every page is loaded, and determines if the user comes from a "bad" IP.
There's a "forgiveness" mechanism built-in: offenses no longer count against an IP after a while.
Thanks!
Extra note: it'd be awesome if the solution worked in PHP, but I'd love to hear your thoughts about the approach in general, for any language/platform
Take a look at fail2ban. A python framework that allows you to raise IP tables blocks from tailing log files for patterns of errant behaviour.
are you on a *nix machine? this sort of thing is probably better left to the OS level, using something like iptables
edit:
in response to the comment, yes (sort of). however, the idea is that iptables can work independently. you can set a certain threshold to throttle (for example, block requests on port 80 TCP that exceed x requests/minute), and that is all handled transparently (ie, your application really doesn't need to know anything about it, to have dynamic blocking take place).
i would suggest the iptables method if you have full control of the box, and would prefer to let your firewall handle throttling (advantages are, you don't need to build this logic into your web app, and it can save resources as requests are dropped before they hit your webserver)
otherwise, if you expect blocking won't be a huge component, (or your app is portable and can't guarantee access to iptables), then it would make more sense to build that logic into your app.
I think it should be a combination of user-name plus IP block. Not just IP.
you're looking at custom lockout code. There are applications in the open source world that contain various flavors of such code. Perhaps you should look at some of those, although your requirements are pretty trivial, so mark an IP/username combo, and utilize that for blocking an IP for x amount of time. (Note I said block the IP, not the user. The user may try to get online via a valid IP/username/pw combo.)
Matter of fact, you could even keep traces of user logins, and when logging in from an unknown IP with a 3 strikes bad username/pw combo, lock that IP out for however long you like for that username. (Do note that a lot of ISPs share IPs, thus....)
You might also want to place a delay in authentication, so that an IP cannot attempt a login more than once every 'y' seconds or so.
I have developed a system for a client which kept track of hits against the web server and dynamically banned IP addresses at the operating system/firewall level for variable periods of time for certain offenses, so, yes, this is definitely possible. As Owen said, firewall rules are a much better place to do this sort of thing than in the web server. (Unfortunately, the client chose to hold a tight copyright on this code, so I am not at liberty to share it.)
I generally work in Perl rather than PHP, but, so long as you have a command-line interface to your firewall rules engine (like, say, /sbin/iptables), you should be able to do this fairly easily from any language which has the ability to execute system commands.
err this sort of system is easy and common, i can give you mine easily enough
its simply and briefly explained here http://www.alandoherty.net/info/webservers/
the scripts as written arn't downloadable {as no commentry currently added} but drop me an e-mail, from the site above, and i'll fling the code at you and gladly help with debugging/taloring it to your server