Logstash output to https - logstash

i have logstash running on windows 8.1
the output is set to https address. wich is hosted on windows server 2012 r2.
when the request gets to the server it gets rejected. this is what i get from logstash:
←[33mUnhandled exception {:request=><FTW::Request(#13468) #headers=FTW::HTTP::He
aders <{"host"=>"taasukaesb.ewavetest.co.il", "connection"=>"keep-alive", "conte
nt-type"=>"application/json", "content-length"=>517}> #method="POST" #body="{\"A
ctivityTime\":\"2015-08-25T08:48:11.691Z\",\"LogType\":\"Info\",\"ServiceName\":
\"Portal.Nlog2Esbtest\",\"Context\":\"Portal\",\"UserId\":\"8664e362-f63d-4d10-8
a23-3b86b9f22cc7\",\"LogSubType\":\"Info\",\"TransactionCode\":\"5d14a403-a566-4
9ab-b8bc-a40c999eebe5\",\"Servers\":\"SlavaNili\",\"ServiceId\":\"8664e362-f63d-
4d10-8a23-3b86b9f22cc7\",\"RequestIPs\":\"172.19.5.8\",\"EntityClass\":\" \",\"M
ethods\":\"Button2_Click\",\"SourceFilePath\":\" \",\"SourceLineNum\":34,\"Title
\":\"TEST INFO\",\"Details\":\"this is a test to tell that something happend, bu
t its cool..\"}" #logger=#<Cabin::Channel:0xd0816b0 #metrics=#<Cabin::Metrics:0x
192e6ba1 #metrics_lock=#<Mutex:0x306a119a>, #metrics={}, #channel=#<Cabin::Chann
el:0xd0816b0 ...>>, #subscriber_lock=#<Mutex:0x7fd74c77>, #level=:info, #subscri
bers={}, #data={}> #request_uri="/TaasukaService.svc/TransactionLog/CreateJson?C
ontext=Portal&UserToken=a441b37f-3403-43fd-8f58-d1da3024133a" #version=1.1 #port
=443 #protocol="https" >, :response=>nil, :exception=>#<OpenSSL::SSL::SSLError:
An existing connection was forcibly closed by the remote host>, :stacktrace=>["o
rg/jruby/ext/openssl/SSLSocket.java:195:in `connect_nonblock'", "C:/logstash/ins
tall/vendor/bundle/jruby/1.9/gems/ftw-0.0.44/lib/ftw/connection.rb:413:in `do_se
cure'", "C:/logstash/install/vendor/bundle/jruby/1.9/gems/ftw-0.0.44/lib/ftw/con
nection.rb:393:in `secure'", "C:/logstash/install/vendor/bundle/jruby/1.9/gems/f
tw-0.0.44/lib/ftw/agent.rb:449:in `connect'", "C:/logstash/install/vendor/bundle
/jruby/1.9/gems/ftw-0.0.44/lib/ftw/agent.rb:283:in `execute'", "C:/logstash/inst
all/vendor/bundle/jruby/1.9/gems/logstash-output-http-1.0.0/lib/logstash/outputs
/http.rb:126:in `receive'", "C:/logstash/install/vendor/bundle/jruby/1.9/gems/lo
gstash-core-1.5.3-java/lib/logstash/outputs/base.rb:88:in `handle'", "(eval):83:
in `output_func'", "C:/logstash/install/vendor/bundle/jruby/1.9/gems/logstash-co
re-1.5.3-java/lib/logstash/pipeline.rb:244:in `outputworker'", "C:/logstash/inst
all/vendor/bundle/jruby/1.9/gems/logstash-core-1.5.3-java/lib/logstash/pipeline.
rb:166:in `start_outputs'"], :level=>:warn}←[0m
notice the ssl error.
now this serivce is fine and accessible. works with no problem from this computer. but only when setting the output in logstash to HTTPS this happens.
this is a follow up from this post:
https://discuss.elastic.co/t/logstash-output-to-https/27946/3
that no one in elastic answerd.
thanks for any help.

Related

dnsmasq - forwarding local dns queries

I have two Openwrt APs with dnsmasq on each ap. Lets call them DNS1 (main AP on 192.168.10.1) and DNS2 (dumb AP on 192.168.10.2). DNS1 is also the only DHCP server on my local network. I have stubby running on each instance to resolve external DNS requests on ports 5453. I have a couple of static dhcp hosts on DNS1 which I sync to DNS2 and they resolve fine. My problem is, I cannot get DNS2 to query DNS1 if it cannot resolve a local (lan) query. To clarify further: Since DNS1 is also handles DHCP, a new client (client1) will only get resolved by DNS1. Any client using DNS2 as its dns server/resolver will not be able to resolve client1 or client1.lan. I thought adding a 'server=/lan/192.168.10.1' would do the trick, but no luck. Here is my /etc/config/dhcp and autogenerated DNSMASQ.conf from DNS2:
config dnsmasq
option leasefile '/tmp/dhcp.leases'
option localservice '1'
option quietdhcp '1'
option cachesize '4096'
option readethers '1'
option localise_queries '1'
option expandhosts '1'
option noresolv '1'
option rebind_protection '1'
option rebind_localhost '1'
option filterwin2k '1'
option domain 'lan'
option domainneeded '1'
list addnhosts '/adblock/custom'
list addnhosts '/adblock/dlhosts'
list addnhosts '/adblock/dlhosts-ipv6'
option local_ttl '300'
list server '/lan/192.168.10.1'
list server '127.0.0.1#5453'
# auto-generated config file from /etc/config/dhcp
conf-file=/etc/dnsmasq.conf
domain-needed
filterwin2k
no-resolv
localise-queries
read-ethers
enable-ubus=dnsmasq
expand-hosts
bind-dynamic
local-service
quiet-dhcp
cache-size=4096
domain=lan
server=/lan/192.168.10.1
server=127.0.0.1#5453
addn-hosts=/tmp/hosts
addn-hosts=/adblock/custom
addn-hosts=/adblock/dlhosts
addn-hosts=/adblock/dlhosts-ipv6
dhcp-leasefile=/tmp/dhcp.leases
local-ttl=300
stop-dns-rebind
rebind-localhost-ok
dhcp-broadcast=tag:needs-broadcast
conf-dir=/tmp/dnsmasq.d
user=dnsmasq
group=dnsmasq
dhcp-ignore-names=tag:dhcp_bogus_hostname
bogus-priv
conf-file=/usr/share/dnsmasq/rfc6761.conf
This is likely dnsmasq's rebind protection kicking in from stop-dns-rebind. Check your logs, if you see lines like this then that is your issue.
dnsmasq[3835]: possible DNS-rebind attack detected: hostname.lan
You want to add rebind-domain-ok=lan to your dnsmasq.conf. Your OpenWRT config should look like this:
config dnsmasq
list rebind_domain 'lan'

ASP.NET Core fails to run under IIS: HTTP Error 500.0 - ANCM In-Process Handler Load Failure

I just upgraded my web application from ASP.NET (Framework) MVC to ASP.NET Core 2.2. The application runs fine under IIS Express, however as soon as I try to run it under IIS it gives the generic error:
HTTP Error 500.0 - ANCM In-Process Handler Load Failure Common causes
of this issue: The specified version of Microsoft.NetCore.App or
Microsoft.AspNetCore.App was not found. The in process request
handler, Microsoft.AspNetCore.Server.IIS, was not referenced in the
application. ANCM could not find dotnet.
I've searched the web and see that many developers are running into the same issue, it seems that Microsoft really missed the mark on this one and have made it as painful as possible to develop ASP.NET Core apps under IIS. Regardless, I'd like to break through this brick wall that I've been hitting my head against for the last 24 hours, so could use some help.
Things I've tried so far:
Installed the latest version of .NET Core Runtime and Hosting Bundle, version 2.2.4.
Installed the latest version of .NET Core SDK, version 2.2.203.
Ensured that "dotnet" is runnable from command line, and that the x64 version is before the x86 version in my environment variables.
C:\WINDOWS\system32>where dotnet C:\Program Files\dotnet\dotnet.exe
C:\Program Files (x86)\dotnet\dotnet.exe
Tried changing my auto-generated Web.config from aspNetCore processPath="%LAUNCHER_PATH%" arguments="%LAUNCHER_ARGS%" to aspNetCore processPath="dotnet" arguments="./Housters.Web.dll" however the changes don't stick, every time I build my web application the Web.config gets reverted back tot he original %LAUNCHER_PATH% values.
Tried changing "AspNetCoreModuleV2" to "AspNetCoreModule" in the Web.config.
Added IsTransformWebConfigDisabled=true to my Web.csproj file in order to keep my custom Web.config from being overwritten. It didn't work, the Web.config is still overwritten.
Tried switching AspNetCoreHostingModel=InProcess to RuntimeFrameworkName=Microsoft.AspNetCore.App in my Web.csproj file.
Cleaning the solution, as well as manually deleting the bin, obj, and even .vs folders.
None of these methods helped, I still get the same ANCM In-Process Handler Load Failure error. The Event Viewer does show a more useful error:
Application 'C:\Housters\Web\' wasn't able to start. Executable was
not found at 'C:\Housters\Web\%LAUNCHER_PATH%.exe'
However, I can't do anything about that because my Web.config changes don't persist. But really, the default settings should work out of the box, we shouldn't have to jump through hoops to get IIS development working. Why is it looking for an .exe when an ASP.NET Core project by default only generates a .dll? And why isn't it replacing %LAUNCHER_PATH% with the correct path, e.g. "bin/Debug"?
EDIT: I should also mention that at times I've been able to get my custom Web.config to persist, and when I do then every time I try to run the web application I get a JIT debugger prompt:
Choosing to debug, or viewing the event log just gives me a really unfriendly error message:
Faulting application name: w3wp.exe, version: 10.0.17134.1, time
stamp: 0xed729d4e Faulting module name: KERNELBASE.dll, version:
10.0.17134.556, time stamp: 0xb9f4a0f1 Exception code: 0xc0020001 Fault offset: 0x000000000003a388 Faulting process id: 0x2650 Faulting
application start time: 0x01d502b1f21b112e Faulting application path:
c:\windows\system32\inetsrv\w3wp.exe Faulting module path:
C:\WINDOWS\System32\KERNELBASE.dll Report Id:
5b5c14cb-8ffa-47b4-8cc1-42f951bc1256 Faulting package full name:
Faulting package-relative application ID:
You're getting this error because you're not specifying if you want the website hosted in process or out of process... in process can be specified by using UseIIS().
Try changing your CreateWebHostBuilder method in Program.cs to:
public static IWebHostBuilder CreateWebHostBuilder(string[] args)
{
var env = Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT");
var builder = WebHost.CreateDefaultBuilder(args);
if (env == EnvironmentName.Staging || env == EnvironmentName.Production)
builder.UseIIS();
builder.UseStartup<Startup>();
return builder;
}
I set ASPNETCORE_ENVIRONMENT in my .pubxml deployment profile by adding:
<PropertyGroup>
<EnvironmentName>Staging</EnvironmentName>
</PropertyGroup>
The cause of this error may also be because the version of the .NET SDK which is in the global.json file of your application is not installed on your local machine. The solution is to check which version is configured in the global.json file and install it on your machine. I managed to solve this error this way (01/06/2022).
The following error will appear in the Windows Event View:
I solve this error just by changing in the web.config AspNetCoreModuleV2 to AspNetCoreModule, but this is not perfect solution.
Another solution is deploying application as Self-Contained when you will publish it.

LetsEncrypt-ACMESharp http-01 challenge on IIS invalid

On server A (non-IIS) I executed:
Import-Module ACMESharp
Initialize-ACMEVault
New-ACMERegistration -Contacts mailto:somebody#derryloran.com -AcceptTos
New-ACMEIdentifier -Dns www.derryloran.com -Alias dns1
Complete-ACMEChallenge dns1 -ChallengeType http-01 -Handler manual
Response back asked:
* Handle Time: [08/05/2017 22:46:27]
* Challenge Token: [BkqO-eYZ5sjgl9Uf3XpM5_s6e5OEgCj9FimuyPACOhI]
To complete this Challenge please create a new file
under the server that is responding to the hostname
and path given with the following characteristics:
* HTTP URL: [http://www.derryloran.com/.well-known/acme-challenge/BkqO-eYZ5sjgl9Uf3XpM5_s6e5OEgCj9FimuyPACOhI]
* File Path: [.well-known/acme-challenge/BkqO-eYZ5sjgl9Uf3XpM5_s6e5OEgCj9FimuyPACOhI]
* File Content: [BkqO-eYZ5sjgl9Uf3XpM5_s6e5OEgCj9FimuyPACOhI.X-01XUeWTE-LgpxWF4D-W_ZvEfu6ue2fAd7DJNhomQM]
* MIME Type: [text/plain]
Server B is serving www.derryloran.com a page at http://www.derryloran.com/.well-known/acme-challenge/BkqO-eYZ5sjgl9Uf3XpM5_s6e5OEgCj9FimuyPACOhI correctly I believe but when I then, back on Server A execute:
Submit-ACMEChallenge dns1 -ChallengeType http-01
(Update-ACMEIdentifier dns1 -ChallengeType http-01).Challenges | Where-Object {$_.Type -eq "http-01"}
...but the status goes invalid after a few seconds. FWIW I've tried this several times always with same result. Why? What am I doing wrong?
I appreciate there's a lot more to go once I've got the certificate but the site is being served in a docker container hence the Server A/B complexities...
Omg, how many times?!? The file had a BOM when created in VS. Recreating using Notepad++ and saving as UTF-8 (without BOM) and I'm getting a valid response now.

Gitlab misdetects binary file with text file and raises Internal Error (500 Whoops)

What is Issue?
When I push a link of the commit which invlolves a binary file from Commits view of a project on Gitlab, I recieve an Internal error ,"500 Whoops, something went wrong on our end."
This issue also appears when creating Merge Request whose origin is the same commit above.
Production.log says,
Started GET "/TempTest/bsp/commit/3098a49f2fd1c77be0c383994aa6655f5d15ebf8" for 127.0.0.1 at 2016-05-30 16:17:15 +0900
Processing by Projects::CommitController#show as HTML
Parameters:{"namespace_id"=>"TempTest", "project_id"=>"bsp", "id"=>"3098a49f2fd1c77be0c383994aa6655f5d15ebf8"}
Encoding::CompatibilityError (incompatible character encodings: UTF-8 and ASCII-8BIT):
app/views/projects/diffs/_file.html.haml:54:in `_app_views_projects_diffs__file_html_haml__1070266479743635718_49404820'
app/views/projects/diffs/_diffs.html.haml:22:in `block in _app_views_projects_diffs__diffs_html_haml__2984561770205002953_48487320'
app/views/projects/diffs/_diffs.html.haml:17:in `each_with_index'
app/views/projects/diffs/_diffs.html.haml:17:in `_app_views_projects_diffs__diffs_html_haml__2984561770205002953_48487320'
app/views/projects/commit/show.html.haml:12:in `_app_views_projects_commit_show_html_haml__3333221152053087461_45612480'
app/controllers/projects/commit_controller.rb:30:in `show'
lib/gitlab/middleware/go.rb:16:in `call'
Completed 500 Internal Server Error in 210ms (Views: 8.7ms | ActiveRecord: 10.5ms)
Gtilab seems to misdetect binary file with text file.
So HTML formatting engine seems to meet an error.("Encoding::CompatibilityError")
It's ok for me that Gitlab sometimes misdetects binary file with text file, but problem is that Gitlab server stops the transaction by Internal Error when such a misdetects occurs.
Could anyone tell me how to continue server transaction even if such a misjudge occurs?
For example, I assume the following answer.
e.g.1) Force to recognize a file to be a binary.
e.g.2) Bypass a HTML transforming when such a error occurs.
What I tried to resolve.
I added the description '*.XXX binary' to .gitattribute to confirm whether I can let a certain file recognize that it was binary file for Gitlab forcibly.
The Git client recognized the file to be binary file, and the diff did not output a text. However, there was no effect in Gitlab even if I did push it.
versions info
I faced this issue at first on Gitlab 8.6.2, but same issue occurs on 8.8.3.
I use git-2.7.2
Thank you.

Unable to run python cgi scripts using CGIHTTPRequestHandler in Python 3.3

I am a noob; trying to create and use a simple webserver in Python that executes CGI scripts written in Python. I am using Windows XP and Python v3.3.0. I have a "myserver" directory which contains "myserver.py","sample.html" and the directory "cgi-bin" which in turn contains "cgi_demo.py"
myserver.py
from http.server import HTTPServer
from http.server import CGIHTTPRequestHandler
port = 8080
host = '127.0.0.1'
server_address = (host,port)
httpd = HTTPServer(server_address,CGIHTTPRequestHandler)
print("Starting my web server on port "+str(port))
httpd.serve_forever()
cgi_demo.py
import cgi
import cgitb; cgitb.enable()
print("Content-type: text/html")
print
print("<html><body>")
for i in range(0,100):
print(i,"<br>")
print("</body></html>")
Now the directory listing works fine for "myserver" but not for "cgi-bin"; maybe that is how it is coded - I don't have a problem here. "sample.html" is retrieved fine too. However, the execution of "cgi_demo.py" is not proper. I get a blank page in the browser; and the console window (which is blank too) appears and disappears. Moreover, on the server's console I get the message
127.0.0.1 - - [29/Nov/2012 12:00:31] "GET /cgi-bin/cgi_demo.py HTTP/1.1" 200 -
127.0.0.1 - - [29/Nov/2012 12:00:31] command: C:\Python33\python.exe -u "D:\python apps\my web server\cgi-bin\cgi_demo.py" ""
127.0.0.1 - - [29/Nov/2012 12:00:32] CGI script exited OK
Please tell me what is wrong! I get the feeling that the output stream of my script is not connected to the server. What am I doing wrong? Don't say that I have to extend CGIHTTPRequestHandler!!
SORRY for the trouble!
Well, it is my fault. 2 things to note:
[1]The console window that appeared and disappeared; it only happens when I use IDLE to execute the server. If the script is already running in a normal windows console then this does not happen. My Feeling was WRONG.
[2]There is an bug/error in my cgi script. After printing the HTTP header; the print statement that I wrote was just "print" instead of actually being "print()".This is so embarrassing! But, even then why didn't the interpreter catch this error?

Resources