I want to be able to convert hostnames to IP addresses in an Apache access log file (i.e. the opposite of what logresolve does).
I have an accesslog file that has been converted with logresolve but I want to revert it.
Each line starts, as an example:
hostname.com - - [01/Jan/2016:00:00:00 +0000] "GET /stuff HTTP/1.1" 200 1046 "http://mywebsite.com" "Mozilla/5.0 (Windows NT 6.1)"
How do I convert hostname.com to an IP address for every line?
Related
Today I check the access log of my website, and find a very strange thing.
When user accessing the file "/downloads/files/ddbfr.exe" with user agent "LWP::Simple/6.13 libwww-perl/6.13", he will get 404 error:
108.162.215.191 - - [22/Dec/2021:12:30:35 -0800] "GET /dbf-repair/ddbfr.exe HTTP/1.1" 301 - "-" "LWP::Simple/6.13 libwww-perl/6.13"
172.70.98.107 - - [22/Dec/2021:12:30:36 -0800] "HEAD /downloads/files/ddbfr.exe HTTP/1.1" 404 - "-" "LWP::Simple/6.13 libwww-perl/6.13"
108.162.215.191 - - [22/Dec/2021:12:30:36 -0800] "GET /dbf-repair/ddbfr.exe HTTP/1.1" 301 - "-" "LWP::Simple/6.13 libwww-perl/6.13"
172.70.98.33 - - [22/Dec/2021:12:30:37 -0800] "HEAD /downloads/files/ddbfr.exe HTTP/1.1" 404 - "-" "LWP::Simple/6.13 libwww-perl/6.13"
However, when accessing the same file "/downloads/files/ddbfr.exe" with other user agent, such as FireFox, then everything will be OK:
172.70.210.253 - - [22/Dec/2021:13:10:28 -0800] "GET /downloads/files/ddbfr.exe HTTP/1.1" 200 8035760 "-" "Mozilla/5.0 (X11; Linux x86_64; en-US; rv:1.9.2.12) Gecko/20101026 Firefox/3.6.12"
It seems that my server is returned 404 for specific agent. However, I check my .htaccess file but cannot find anything like the one in 301 redirect based on user-agent. Why?
Note: There is a redirect from /dbf-repair/ddfr.exe to /downloads/files/ddbfr.exe in .htaccess.
I have a Magento 1.9.2 system that is currently undergoing a brute force attack against its xml-rpc endpoint from an EC2 host.
I can simply firewall the source address but that is a short term solution, since it will likely face another attack from a different address. I would like to be able to detect these attacks automatically to lock them down.
Fail2ban is commonly used under such circumstances but in order for it to work, I understand that it must be able to find login failure messages in a log file somewhere, however Magento does not seem to be logging the failed attempts.
How can I prevent the xml-rpc endpoint being brute forced?
54.246.87.74 - - [20/Jul/2015:13:10:24 +0000] "POST /index.php/api/xmlrpc/ HTTP/1.1" 200 777 "-" "XML-RPC.NET"
54.246.87.74 - - [20/Jul/2015:13:10:24 +0000] "POST /index.php/api/xmlrpc/ HTTP/1.1" 200 777 "-" "XML-RPC.NET"
54.246.87.74 - - [20/Jul/2015:13:10:25 +0000] "POST /index.php/api/xmlrpc/ HTTP/1.1" 200 777 "-" "XML-RPC.NET"
54.246.87.74 - - [20/Jul/2015:13:10:26 +0000] "POST /index.php/api/xmlrpc/ HTTP/1.1" 200 777 "-" "XML-RPC.NET"
54.246.87.74 - - [20/Jul/2015:13:10:27 +0000] "POST /index.php/api/xmlrpc/ HTTP/1.1" 200 777 "-" "XML-RPC.NET"
Action taken so far
I've configured fail2ban with a new filter and jail to lock it down but I still don't know if this is the best solution.
filter.d/magento-xmlrpc.conf
[Definition]
failregex = ^<HOST> .*POST .*api\/xmlrpc\/
ignoreregex =
jail.local
[magento-xmlrpc]
enabled = true
port = http,https
filter = magento-xmlrpc
logpath = /home/user/logs/access.log
maxretry = 20
findtime = 30
bantime = 600
I have noticed numerous entries in Tomcat's local_access_log for various resources coming from IP address 127.0.0.1. These are clearly attempts to hack in. For example, here is a request to get access to the "manager" app:
127.0.0.1 - - [30/Apr/2015:13:35:13 +0000] "GET /manager/html HTTP/1.1" 401 2474
here is another one:
127.0.0.1 - - [30/Apr/2015:21:23:37 +0000] "POST /cgi-bin/php?%2D%64+%61%6C%6C%6F%77%5F%75%72%6C%5F%69%6E%63%6C%75%64%65%3D%6F%6E+%2D%64+%73%61%66%65%5F%6D%6F%64%65%3D%6F%66%66+%2D%64+%73%75%68%6F%73%69%6E%2E%73%69%6D%75%6C%61%74%69%6F%6E%3D%6F%6E+%2D%64+%64%69%73%61%62%6C%65%5F%66%75%6E%63%74%69%6F%6E%73%3D%22%22+%2D%64+%6F%70%65%6E%5F%62%61%73%65%64%69%72%3D%6E%6F%6E%65+%2D%64+%61%75%74%6F%5F%70%72%65%70%65%6E%64%5F%66%69%6C%65%3D%70%68%70%3A%2F%2F%69%6E%70%75%74+%2D%64+%63%67%69%2E%66%6F%72%63%65%5F%72%65%64%69%72%65%63%74%3D%30+%2D%64+%63%67%69%2E%72%65%64%69%72%65%63%74%5F%73%74%61%74%75%73%5F%65%6E%76%3D%22%79%65%73%22+%2D%64+%63%67%69%2E%66%69%78%5F%70%61%74%68%69%6E%66%6F%3D%31+%2D%64+%61%75%74%6F%5F%70%72%65%70%65%6E%64%5F%66%69%6C%65%3D%70%68%70%3A%2F%2F%69%6E%70%75%74+%2D%6E HTTP/1.1" 404 1016
When decoded, the URL is this:
127.0.0.1 - - [30/Apr/2015:21:23:37 0000] "POST /cgi-bin/php?-d allow_url_include=on -d safe_mode=off -d suhosin.simulation=on -d disable_functions="" -d open_basedir=none -d auto_prepend_file=php://input -d cgi.force_redirect=0 -d cgi.redirect_status_env="yes" -d cgi.fix_pathinfo=1 -d auto_prepend_file=php://input -n HTTP/1.1" 404 1016
There are lots of such entries, all from IP address 127.0.0.1. Obviously, since this is the address of localhost, I can't block it. More over, I am not sure if there is something that I can do about it. Is there possibly an exploit that should be patched up? For instance, is there a version of Tomcat that has a related vulnerability? I am running Tomcat 8.
Much thanks for any advice!
UPDATE: thanks for the suggestion about a proxy. Turned out that httpd was indeed installed and not surprisingly, there are suspicious request. For example:
[Sat Mar 30 17:26:49 2013] [error] [client 5.34.247.59] Invalid URI in request GET /_mem_bin/../../../../winnt/system32/cmd.exe?/c+dir HTTP/1.0
[Sat Mar 30 17:26:49 2013] [error] [client 5.34.247.59] Invalid URI in request GET /_mem_bin/../../../../winnt/system32/cmd.exe?/c+dir%20c:\\ HTTP/1.0
[Sat Mar 30 17:26:49 2013] [error] [client 5.34.247.59] Invalid URI in request GET /_mem_bin/../../../../winnt/system32/cmd.exe?/c+dir%20c:\\ HTTP/1.0
This is not a windows system so cmd.exe has not place for it...
If you have a proxy server running on your computer, that will often receive requests and then call the primary server using the localhost (127.0.0.1) interface.
This could explain why you're logging these requests.
I have the following logs sent to logstash via syslog input (the logs come from docker containers sent to logstash via logspout)
So, in the following log sample, the first line is a php-fpm error and the second line is a nginx access log
docker-nginx-php-composer|2015/03/17 16:31:28 [error] 9#0: *41 FastCGI sent in stderr: "PHP message: PHP Fatal error: Call to undefined function teetstst() in /var/www/index.php on line 3" while reading response header from upstream, client: 10.10.37.110, server: _, request: "GET /favicon.ico HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "turing:49213"
docker-nginx-php-composer|10.10.37.110 - - [17/Mar/2015:16:31:28 +0100] "GET /favicon.ico HTTP/1.1" 200 134 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/41.0.2272.89 Safari/537.36"
```
OK so now I want to filter all this mess so I can "tag" PHP-FPM logs for example or filter only the warning and errors of PHP-FPM ...
I suppose I have to use the grok filter from logstash but I don't know exactly how ?
let say I want to filter only PHP errors, put a "php" tag on it and also filter 404 and 500 errors from nginx, how can I achieve this easily with logstash/grok ?
Any clues ? examples ?
Thanks :)
I would define two grok patterns:
php-fpm that starts with a date
nginx that starts with an ip
You can add_tag in each grok{} as desired.
There is a shared printer at my workplace. We send jobs and then go to the printer and authenticate, so printer prints your documents only when you present at it. Periodically, we change domain passwords, so I also have to change it in /etc/cups/printers.conf (windows users just change domain password). So, that's how it works.
But, suddenly, it stop receive my jobs. When I send job I have no errors and have this:
sudo tail /var/log/cups/access_log
localhost - - [14/Apr/2015:12:15:14 +0300] "POST /printers/Generic-PCL-6-PCL-XL HTTP/1.1" 200 499 Create-Job successful-ok
localhost - - [14/Apr/2015:12:15:14 +0300] "POST /printers/Generic-PCL-6-PCL-XL HTTP/1.1" 200 1273674 Send-Document successful-ok
localhost - - [14/Apr/2015:12:17:59 +0300] "POST / HTTP/1.1" 200 183 Renew-Subscription successful-ok
On cups page in browser it shows state for job - "Pending since (date/time)".
It seems like job was sent successfully, but when I came to printer I've got nothing and no job in my queue. Our IT support fix problems only for Windows users and who on Linux - on their own. So, I don't know what to do and what logs I should inspect. Please, help.
Probably, some updates broke it down. But I have found another solution - I add printer not via samba, but via lp and it doesn't ask username/password:
cat /etc/cups/printers.conf
# Printer configuration file for CUPS v1.5.3
# Written by cupsd
# DO NOT EDIT THIS FILE WHEN CUPSD IS RUNNING
<DefaultPrinter KonicaMinolta>
UUID urn:uuid:0f60c08a-ecfb-326a-421c-86aa3519147b
Info MyCompany Office printer
Location WestCorridor
MakeModel Generic PostScript Printer Foomatic/Postscript (recommended)
DeviceURI lpd://Company_printer_server_address/lp
State Idle
StateTime 1429265417
Type 8433692
Accepting Yes
Shared Yes
JobSheets none none
QuotaPeriod 0
PageLimit 0
KLimit 0
OpPolicy default
ErrorPolicy stop-printer
</Printer>
If somebody can provide another solution or some explanation why it is so, I will be glad to see.
As far as debugging you can view more data in your CUPS logs if you edit your /etc/cups/cupsd.conf file, find the section "loglevel" change "info" to "debug"
Then you should restart CUPS with:
/etc/init.d/cups restart
Then your log will be in
/var/log/cups/error_log