ngx-socket-io connected false but flask_socketio logs show 200 OK - python-3.x

Any tips to debug would help, this is just the basic implementation.
I am using the exact tutorial as here to setup the ngx socket client:
https://www.npmjs.com/package/ngx-socket-io
flask-socketio for the server
https://flask-socketio.readthedocs.io/en/latest/
Server-side shows no error, but the registered handler methods to emit/receive seem blind.
Server Logs:
127.0.0.1 - - [28/Jan/2021 10:58:51] "GET /socket.io/?EIO=3&transport=polling&t=NT8Dtbg HTTP/1.1" 200 418 0.000640
(12285) accepted ('127.0.0.1', 37400)
fa1a1d40bbc349c384a121302ef567c7: Received request to upgrade to websocket
127.0.0.1 - - [28/Jan/2021 10:58:51] "GET /socket.io/?EIO=3&transport=polling&t=NT8Dtcd&sid=fa1a1d40bbc349c384a121302ef567c7 HTTP/1.1" 200 235 0.000408
127.0.0.1 - - [28/Jan/2021 10:58:51] "GET /socket.io/?EIO=3&transport=polling&t=NT8DtdE&sid=fa1a1d40bbc349c384a121302ef567c7 HTTP/1.1" 200 235 0.000268
fa1a1d40bbc349c384a121302ef567c7: Upgrade to websocket successful
5002732d42184ba6b453e7d4f35e864e: Received packet PING data None
5002732d42184ba6b453e7d4f35e864e: Sending packet PONG data None
Client logs:
config: {url: "http://127.0.0.1:5000/", options: {…}}
emptyConfig: {url: "", options: {…}}
eventObservables$: {}
ioSocket: Socket
acks: {}
connected: false
disconnected: true
flags: {}
ids: 0
io: Manager
autoConnect: true
backoff: Backoff {ms: 1000, max: 5000, factor: 2, jitter: 0.5, attempts: 0}
connecting: [Socket]
decoder: Decoder {reconstructor: null, _callbacks: {…}}
encoder: Encoder {}
encoding: false
engine: Socket {secure: false, agent: false, hostname: "127.0.0.1", port: "5000", query: {…}, …}
lastPing: Thu Jan 28 2021 11:00:56
Server code (no print)
#socketio.on('connect')
def test_connect():
print('\n\nClient connected')
Why can't I get the connection working? Thanks

Double check npm/python environment of current versions of socketio and engineio.
https://github.com/miguelgrinberg/python-socketio
d7e95928d73c42fab431e94ce2df40fc: Sending packet OPEN data {'sid': 'd7e95928d73c42fab431e94ce2df40fc', 'upgrades': ['websocket'], 'pingTimeout': 60000, 'pingInterval': 25000}
Client connected
d7e95928d73c42fab431e94ce2df40fc: Sending packet MESSAGE data 0

Related

Errors when publishing my nuxtjs website on SSR mode

I have some issues while trying to publish a nuxtjs site. Usually, I was using the generate command, but for this one I need to go full SSR, so I'm going for nuxt start.
But after building and starting the app, it's a mess. The build goes perfectly in the console, and the application start. The problem is when I try to access the site, it loads partially, but I got all these errors in the browser:
manifest.3a7efd91c5f63f114507.js
Failed to load resource: the server responded with a status of 404 ()
vendor.7519259bf7bdf608079e.js
Failed to load resource: the server responded with a status of 404 ()
app.a5cb9356f53306f973dc.js
Failed to load resource: the server responded with a status of 404 ()
default.1f3ad14df16ee86595af.js
Failed to load resource: the server responded with a status of 404 ()
index.260dc65b69022a31ad58.js
Failed to load resource: the server responded with a status of 404 ()
/_nuxt/pages/spot/_slug.e57cc2e78d8e0b160fe7.js
Failed to load resource: the server responded with a status of 404 ()
manifest.3a7efd91c5f63f114507.js
Failed to load resource: the server responded with a status of 404 ()
default.1f3ad14df16ee86595af.js
Failed to load resource: the server responded with a status of 404 ()
index.260dc65b69022a31ad58.js
Failed to load resource: the server responded with a status of 404 ()
vendor.7519259bf7bdf608079e.js
Failed to load resource: the server responded with a status of 404 ()
app.a5cb9356f53306f973dc.js
Failed to load resource: the server responded with a status of 404 ()
Nothing seems wrong during the build. When I use nuxt start, I get this:
$ nuxt start
nuxt:axios BaseURL: http://localhost:3042/api (Browser: /api) +0ms
OPEN http://localhost:3042
Here's my server conf file:
# Site global
server {
listen 443 ssl http2;
server_name www.mywebsite.com;
access_log off;
location = /robots.txt { access_log off; log_not_found off; }
location = /favicon.ico { access_log off; log_not_found off; }
location / {
proxy_pass http://127.0.0.1:3042/;
include /etc/nginx/conf.d/proxy.conf;
root /var/www/mywebsite/site;
add_header Access-Control-Allow-Origin *;
}
location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|txt|srt|swf)$ {
root /var/www/mywebsite/site/;
expires 30d;
}
ssl_certificate /etc/letsencrypt/live/www.mywebsite.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/www.mywebsite.com/privkey.pem;
}
# Redirection
server {
listen 80;
server_name mywebsite.com www.mywebsite.com;
location / {
return 301 https://www.mywebsite.com$request_uri;
}
}
And my nuxt config file:
const pkg = require('./package')
module.exports = {
mode: 'universal',
loading: { color: '#bb2b4d' },
router: {
linkActiveClass: '-active',
base: '/'
},
css: ['#/assets/icons/css/icons.css', '#/assets/snickles/snickles.css'],
plugins: ['~plugins/vue-filters.js', '~plugins/vue-modal.js'],
minify: {
removeEmptyAttributes: false,
collapseWhitespace: true,
conservativeCollapse: true,
collapseBooleanAttributes: true,
removeTagWhitespace: false,
removeStyleLinkTypeAttributes: true
},
modules: [
'#nuxtjs/axios'
],
axios: {
},
env: {
api: {
spots: `https://rest.mywebsite.com/spots`
}
},
proxy: {
},
build: {
extend(config, ctx) {
// Run ESLint on save
if (ctx.isDev && ctx.isClient) {
config.module.rules.push({
enforce: 'pre',
test: /\.(js|vue)$/,
loader: 'eslint-loader',
exclude: /(node_modules)/
})
}
}
},
postcss: [require('autoprefixer')],
vendor: ['moment', 'vue-js-modal']
}
Did I forget anything?
The most strange part is that it works perfectly well when I do the same on my own pc and not on my server. I checked the npm and node versions, they are the same (latest to date). Also, if testing with a demo template from NuxtJS, it works perfectly with the exact same server configuration.
By the way, the server is a debian 8, with all packages up to date.
Thanks in advance for any hint.
Edit: If of any use, the error log:
2018/02/14 19:12:54 [error] 12981#12981: *239930 open() "/var/www/mywebsite/site/_nuxt/pages/spot/_slug.e57cc2e78d8e0b160fe7.js" failed (2: No such file or directory), client: xxx.xxx.xxx.xxx., server: www.mywebsite.com, request: "GET /_nuxt/pages/spot/_slug.e57cc2e78d8e0b160fe7.js HTTP/2.0", host: "www.mywebsite.com", referrer: "https://www.mywebsite.com/"
2018/02/14 19:12:57 [error] 12981#12981: *239930 open() "/var/www/mywebsite/site/_nuxt/manifest.3a7efd91c5f63f114507.js" failed (2: No such file or directory), client: xxx.xxx.xxx.xxx, server: www.mywebsite.com, request: "GET /_nuxt/manifest.3a7efd91c5f63f114507.js HTTP/2.0", host: "www.mywebsite.com", referrer: "https://www.mywebsite.com/"
2018/02/14 19:12:57 [error] 12981#12981: *239930 open() "/var/www/mywebsite/site/_nuxt/vendor.7519259bf7bdf608079e.js" failed (2: No such file or directory), client: xxx.xxx.xxx.xxx, server: www.mywebsite.com, request: "GET /_nuxt/vendor.7519259bf7bdf608079e.js HTTP/2.0", host: "www.mywebsite.com", referrer: "https://www.mywebsite.com/"
2018/02/14 19:12:57 [error] 12981#12981: *239930 open() "/var/www/mywebsite/site/_nuxt/app.a5cb9356f53306f973dc.js" failed (2: No such file or directory), client: xxx.xxx.xxx.xxx, server: www.mywebsite.com, request: "GET /_nuxt/app.a5cb9356f53306f973dc.js HTTP/2.0", host: "www.mywebsite.com", referrer: "https://www.mywebsite.com/"
Again, it’s working perfectly fine with other nuxt projects, with a similar configuration. Indeed it can’t find these files in this folder, as they’re not in it — which is perfectly normal. It’s up to the app to get the routes to these files, which it usually does pretty well, with the same directory output (as I said, it’s supposed not to be in a _nuxt folder).
Thanks.
This was tagged with nginx, so, here's the nginx way of solving the problem.
After the troubleshooting through the comments, you report receiving the following in your error_log:
2018/02/14 19:12:57 [error] 12981#12981: *239930 open() "/var/www/mywebsite/site/_nuxt/manifest.3a7efd91c5f63f114507.js" failed (2: No such file or directory), client: xxx.xxx.xxx.xxx, server: www.mywebsite.com, request: "GET /_nuxt/manifest.3a7efd91c5f63f114507.js HTTP/2.0", host: "www.mywebsite.com", referrer: "https://www.mywebsite.com/"
Subsequently, running find / -type f -name manifest.3a7efd91c5f63f114507.js, or similar, results in the corresponding file being located in /var/www/mywebsite/site/.nuxt/dist.
As such, your nginx configuration is wrong, because you make it look for these things in the incorrect folder — your config has root /var/www/mywebsite/site/; instead.
The proper way would may be to use a prefix-based location together with the alias directive:
location /_nuxt/ {
alias /var/www/mywebsite/site/.nuxt/dist/;
}
However, if /_nuxt/ has stuff that may have to be proxy_pass'ed to the upstream, and you want to continue using the pcre-based location you already have had in your original config, then an alternative solution like below is also an option (otherwise, you obviously would have to remove it as redundant, to make sure that the prior prefix-based location works, see http://nginx.org/r/location):
location ~* ^.+.(jpg|jpeg|gif|css|png|js|ico|txt|srt|swf)$ {
rewrite ^/_nuxt(/.*) $1 break;
root /var/www/mywebsite/site/.nuxt/dist/;
}

Node Socket.io on HA Proxy with multiple end points

I am tying to deploy my node websocket service on two boxes and masking it using haproxy but its not working.
frontend http-in
mode http
bind *:80
acl is_websocket path_beg /prodSocket
acl is_websocket hdr(Upgrade) -i WebSocket
acl is_websocket hdr_beg(Host) -i ws
use_backend prodSocket if is_websocket
acl is_websocket path_beg /demoSocket
acl is_websocket hdr(Upgrade) -i WebSocket
acl is_websocket hdr_beg(Host) -i ws
use_backend demoSocket if is_websocket
backend demoSocket
timeout server 180s
server 148.251.76.84 148.251.76.84:9000 weight 1 maxconn 1024 check
backend prodSocket
timeout server 180s
server 148.251.76.85 148.251.76.85:9000 weight 1 maxconn 1024 check
Client code -
var socket = io('http://localhost/prodSocket', {
'force new connection': false,
'reconnection delay': 500,
'max reconnection attempts': 10,
});
socket.emit('client', { my: 'data' });
socket.on('news', function (data) {
console.log(data);
});
The above code does not work but if I make following changes it works -
frontend http-in
mode http
bind *:80
acl is_websocket path_beg /socket.io
acl is_websocket hdr(Upgrade) -i WebSocket
acl is_websocket hdr_beg(Host) -i ws
use_backend socket if is_websocket
backend socket
timeout server 180s
server 148.251.76.84 148.251.76.84:9000 weight 1 maxconn 1024 check
Client code -
var socket = io('http://localhost:9090', {
'force new connection': false,
'reconnection delay': 500,
'max reconnection attempts': 10,
});
socket.emit('client', { my: 'data' });
socket.on('news', function (data) {
console.log(data);
});
I understood that socket io is calling /socket.io endpoint for creating a socket connection but how can then deploy my service over two different end points ?
Versions -
Socket.io - 1.4.5
Node - v5.6.0
HAproxy - 1.4.24
Ubuntu - 14.04
HAProxy Log using /socket.io endpoint -
config -
frontend http-in
mode http
bind *:9090
acl is_websocket path_beg /socket.io
acl is_websocket hdr(Upgrade) -i WebSocket
acl is_websocket hdr_beg(Host) -i ws
use_backend socket if is_websocket
CLient script -
var socket = io('http://localhost:9090', {
'force new connection': false,
'reconnection delay': 500,
'max reconnection attempts': 10,
});
Jul 21 10:50:04 localhost haproxy[11981]: 127.0.0.1:48571 [21/Jul/2016:10:49:51.830] http-in socket/148.251.76.84 0/0/171/676/12725 101 187 - - ---- 2/2/2/2/0 0/0 "GET /socket.io/?EIO=3&transport=websocket&sid=PvK2vnQO1_IepDHOAAAJ HTTP/1.1"
Jul 21 10:50:55 localhost haproxy[11981]: 127.0.0.1:48573 [21/Jul/2016:10:49:51.832] http-in socket/148.251.76.84 0/0/123/126/63531 200 1551 - - cD-- 2/2/2/2/0 0/0 "GET /socket.io/?EIO=3&transport=polling&t=LOBvKmM&sid=PvK2vnQO1_IepDHOAAAJ HTTP/1.1"
Jul 21 10:50:55 localhost haproxy[11981]: 127.0.0.1:48569 [21/Jul/2016:10:49:51.505] http-in socket/148.251.76.84 0/0/152/159/64144 200 1199 - - cD-- 1/1/1/1/0 0/0 "GET /socket.io/?EIO=3&transport=polling&t=LOBvKhF HTTP/1.1"
HAProxy Log using /prodSocket endpoint -
Config -
frontend http-in
mode http
bind *:9090
acl is_websocket path_beg /prodSocket
acl is_websocket hdr(Upgrade) -i WebSocket
acl is_websocket hdr_beg(Host) -i ws
use_backend socket if is_websocket
CLient script -
var socket = io('http://localhost:9090/prodSocket', {
'force new connection': false,
'reconnection delay': 500,
'max reconnection attempts': 10,
});
Jul 21 10:55:11 localhost haproxy[12361]: Proxy socket started.
Jul 21 10:55:11 localhost haproxy[12361]: 127.0.0.1:48856 [21/Jul/2016:10:55:11.767] http-in http-in/<NOSRV> -1/-1/-1/-1/0 503 213 - - SC-- 0/0/0/0/0 0/0 "GET /socket.io/?EIO=3&transport=polling&t=LOBwYtK HTTP/1.1"
Jul 21 10:55:16 localhost haproxy[12362]: 127.0.0.1:48859 [21/Jul/2016:10:55:16.229] http-in http-in/<NOSRV> -1/-1/-1/-1/0 503 213 - - SC-- 0/0/0/0/0 0/0 "GET /socket.io/?EIO=3&transport=polling&t=LOBwZz3 HTTP/1.1"
Jul 21 10:55:17 localhost haproxy[12362]: 127.0.0.1:48860 [21/Jul/2016:10:55:17.364] http-in http-in/<NOSRV> -1/-1/-1/-1/0 503 213 - - SC-- 0/0/0/0/0 0/0 "GET /socket.io/?EIO=3&transport=polling&t=LOBwaEn HTTP/1.1"
Jul 21 10:55:19 localhost haproxy[12362]: 127.0.0.1:48862 [21/Jul/2016:10:55:19.075] http-in http-in/<NOSRV> -1/-1/-1/-1/0 503 213 - - SC-- 0/0/0/0/0 0/0 "GET /socket.io/?EIO=3&transport=polling&t=LOBwafV HTTP/1.1"
Jul 21 10:55:22 localhost haproxy[12362]: 127.0.0.1:48865 [21/Jul/2016:10:55:22.262] http-in http-in/<NOSRV> -1/-1/-1/-1/0 503 213 - - SC-- 0/0/0/0/0 0/0 "GET /socket.io/?EIO=3&transport=polling&t=LOBwbRI HTTP/1.1"
Jul 21 10:55:27 localhost haproxy[12362]: 127.0.0.1:48869 [21/Jul/2016:10:55:27.271] http-in http-in/<NOSRV> -1/-1/-1/-1/0 503 213 - - SC-- 0/0/0/0/0 0/0 "GET /socket.io/?EIO=3&transport=polling&t=LOBwcfa HTTP/1.1"

Impossible to use grok date match correctly

I have this message
2016/02/22 08:40:10 [error] 2127#0: *193 open()
"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg"
failed (2: No such file or directory), client: 192.168.144.95, server:
api.magritte.arte.tv, request: "GET
/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg
HTTP/1.1", host: "api.magritte.arte.tv", referrer:
"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos"
And I parse it this way
grok {
match => { "message" => "(?<timestamp>%{YEAR}/%{MONTHNUM2}/%{MONTHDAY} %{TIME}) \[%{LOGLEVEL:severity}\] %{POSINT:pid}#%{NUMBER:tid}:( \*%{NUMBER:cid})? %{GREEDYDATA:errormessage}(?:, client: (?<client>%{IP}|%{HOSTNAME}))(?:, server: %{IPORHOST:server})(?:, request: %{QS:request})?(?:, upstream: \"%{URI:upstream}\")?(?:, host: %{QS:host})?(?:, referrer: \"%{URI:referrer}\")?"}
}
date {
match => [ "timestamp" , "YYYY/MM/dd HH:mm:ss" ]
}
When a new message arrives the following behaviour occurs
message sent to rabbit : OK
message read from rabbit : OK
problem when logstash read message
"reason"=>"failed to parse [timestamp]",
"caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid
format: \"2016/02/22 08:40:10\" is malformed at \"/02/22
08:40:10\""}}}}, :level=>:warn}
But I have no idea where my error is. using http://grokconstructor.appspot.com/do/match#result all seems OK
The full log in the logstash is
{:timestamp=>"2016-02-22T08:43:29.968000+0100", :message=>"Failed action. ", :status=>400, :action=>["index", {:_id=>nil, :_index=>"logstash-2016.02.22", :_type=>"nginx_error", :_routing=>nil}, #<LogStash::Event:0x75f8f9a0 #metadata_accessors=#<LogStash::Util::Accessors:0x402f1514 #store={}, #lut={}>, #cancelled=false, #data={"message"=>"2016/02/22 08:40:10 [error] 2127#0: *193 open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory), client: 192.168.144.95, server: api.magritte.arte.tv, request: \"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\", host: \"api.magritte.arte.tv\", referrer: \"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos\"", "#version"=>"1", "#timestamp"=>"2016-02-22T07:40:10.000Z", "path"=>"/var/log/nginx/api.magritte.arte.tv_error.log", "host"=>["magritte.arte.tv", "\"api.magritte.arte.tv\""], "type"=>"nginx_error", "application"=>"api", "timestamp"=>"2016/02/22 08:40:10", "severity"=>"error", "pid"=>2127, "tid"=>0, "cid"=>193, "errormessage"=>"open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory)", "client"=>"192.168.144.95", "server"=>"api.magritte.arte.tv", "request"=>"\"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\"", "referrer"=>"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos"}, #metadata={}, #accessors=#<LogStash::Util::Accessors:0x27ca0e3f #store={"message"=>"2016/02/22 08:40:10 [error] 2127#0: *193 open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory), client: 192.168.144.95, server: api.magritte.arte.tv, request: \"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\", host: \"api.magritte.arte.tv\", referrer: \"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos\"", "#version"=>"1", "#timestamp"=>"2016-02-22T07:40:10.000Z", "path"=>"/var/log/nginx/api.magritte.arte.tv_error.log", "host"=>["magritte.arte.tv", "\"api.magritte.arte.tv\""], "type"=>"nginx_error", "application"=>"api", "timestamp"=>"2016/02/22 08:40:10", "severity"=>"error", "pid"=>2127, "tid"=>0, "cid"=>193, "errormessage"=>"open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory)", "client"=>"192.168.144.95", "server"=>"api.magritte.arte.tv", "request"=>"\"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\"", "referrer"=>"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos"}, #lut={"type"=>[{"message"=>"2016/02/22 08:40:10 [error] 2127#0: *193 open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory), client: 192.168.144.95, server: api.magritte.arte.tv, request: \"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\", host: \"api.magritte.arte.tv\", referrer: \"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos\"", "#version"=>"1", "#timestamp"=>"2016-02-22T07:40:10.000Z", "path"=>"/var/log/nginx/api.magritte.arte.tv_error.log", "host"=>["magritte.arte.tv", "\"api.magritte.arte.tv\""], "type"=>"nginx_error", "application"=>"api", "timestamp"=>"2016/02/22 08:40:10", "severity"=>"error", "pid"=>2127, "tid"=>0, "cid"=>193, "errormessage"=>"open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory)", "client"=>"192.168.144.95", "server"=>"api.magritte.arte.tv", "request"=>"\"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\"", "referrer"=>"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos"}, "type"], "[type]"=>[{"message"=>"2016/02/22 08:40:10 [error] 2127#0: *193 open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory), client: 192.168.144.95, server: api.magritte.arte.tv, request: \"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\", host: \"api.magritte.arte.tv\", referrer: \"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos\"", "#version"=>"1", "#timestamp"=>"2016-02-22T07:40:10.000Z", "path"=>"/var/log/nginx/api.magritte.arte.tv_error.log", "host"=>["magritte.arte.tv", "\"api.magritte.arte.tv\""], "type"=>"nginx_error", "application"=>"api", "timestamp"=>"2016/02/22 08:40:10", "severity"=>"error", "pid"=>2127, "tid"=>0, "cid"=>193, "errormessage"=>"open() \"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\" failed (2: No such file or directory)", "client"=>"192.168.144.95", "server"=>"api.magritte.arte.tv", "request"=>"\"GET /static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg HTTP/1.1\"", "referrer"=>"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos"}, "type"]}>>], :response=>{"create"=>{"_index"=>"logstash-2016.02.22", "_type"=>"nginx_error", "_id"=>"AVMH7uSoo1ZDC2Pzezhl", "status"=>400, "error"=>{"type"=>"mapper_parsing_exception", "reason"=>"failed to parse [timestamp]", "caused_by"=>{"type"=>"illegal_argument_exception", "reason"=>"Invalid format: \"2016/02/22 08:40:10\" is malformed at \"/02/22 08:40:10\""}}}}, :level=>:warn}
I think it's a quote issue ...
1
Message in nginx logfile 2016/02/22 08:40:10 [error] 2127#0: *193
open()
"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg"
failed (2: No such file or directory), client: 192.168.144.95, server:
api.magritte.arte.tv, request: "GET
/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg
HTTP/1.1", host: "api.magritte.arte.tv", referrer:
"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos"
It's the one I use to do my grok parsing (logstash sourceside)
rabbitmq message payload
{"message":"2016/02/22 08:40:10 [error] 2127#0: *193 open()
\"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\"
failed (2: No such file or directory), client: 192.168.144.95, server:
api.magritte.arte.tv, request: \"GET
/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg
HTTP/1.1\", host: \"api.magritte.arte.tv\", referrer:
\"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos\"","#version":"1","#timestamp":"2016-02-22T07:40:10.000Z","path":"/var/log/nginx/api.magritte.arte.tv_error.log","host":["magritte.arte.tv","\"api.magritte.arte.tv\""],"type":"nginx_error","application":"api","timestamp":"2016/02/22
08:40:10","severity":"error","pid":2127,"tid":0,"cid":193,"errormessage":"open()
\"/etc/nginx/nginx/html/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg\"
failed (2: No such file or
directory)","client":"192.168.144.95","server":"api.magritte.arte.tv","request":"\"GET
/static-cdn.arte.tv/resize-preprod/nQa5oWnNDknADSxe0mPEMd5McUA=/940x530/smart/default/prog_img/IMG_APIOS/051000/051700/051757-001_1137283_32_202.jpg
HTTP/1.1\"","referrer":"https://api.magritte.arte.tv/api/oauth/user/documentation/opa/endpoint/27/-api-opa-v2-videos"}
Some backslashes are added
This backslash prevent logstash (target side) to correctly handle the messages.
With this, it works :)
grok {
match => { "message" => "(?<timestamp>%{YEAR}/%{MONTHNUM2}/%{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND}) \[%{LOGLEVEL:severity}\] %{POSINT:p_id}#%{NUMBER:t_id}:( \*%{NUMBER:c_id})? %{GREEDYDATA:errormessage}(?:, client: (?<client>%{IP}|%{HOSTNAME}))(?:, server: %{IPORHOST:server})(?:, request: %{QS:request})?(?:, upstream: %{QS:upstream})?(?:, host: %{QS:vhost})?(?:, referrer: \"%{URI:referrer}\")?"}
}
date {
match => [ "timestamp" , "yyyy/MM/dd HH:mm:ss" ]
}
A grok pattern is available, that can solve your problem.
grok {
match => ["message","%{DATESTAMP:timestamp}" ]
}
Try y instead of Y in date filter.
joda.time.format.DateTimeFormat
date {
match => [ "timestamp" , "yyyy/MM/dd HH:mm:ss" ]
}

Generating signed XPI via jpm failed

There was a problem signing an Add-On via jpm: The command
jpm -v sign --api-key 'user:xxxxxxxx:xxx' --api-secret xxxxxxxxxxxxxxxxxxxxxxxxx
failed with the error message
Error: Received bad response from the server while requesting https://addons.mozilla.org/api/v3/addons/%40addonname/versions/0.x.y/
Signing via the web interface worked.
How can this be fixed?
The full verbose output is
JPM [info] binary set to /usr/bin/firefox
JPM [info] verbose set
JPM [info] Checking compatability bootstrap.js and install.rdf for xpi
Validating the manifest
JPM [info] Creating fallbacks if they are necessary..
Creating XPI
JPM [info] Creating XPI...
JPM [info] .jpmignore found
JPM [info] Adding: /path/to/file
[...]
JPM [info] Adding: /path/to/another_file
JPM [info] XPI created at /tmp/tmp-unsigned-xpi-25600yuqzFN3Alxa0/#addonname-0.x.y.xpi (121ms)
JPM [info] Removing fallbacks if they were necessary..
JPM [info] Creating updateRDF...
Created XPI at /tmp/tmp-unsigned-xpi-25600yuqzFN3Alxa0/#addonname-0.x.y.xpi
JPM [info] Created XPI for signing: /tmp/tmp-unsigned-xpi-25600yuqzFN3Alxa0/#addonname-0.x.y.xpi
JPM [debug] [API] -{ url: 'https://addons.mozilla.org/api/v3/addons/%40addonname/versions/0.x.y/',
formData:
{ upload:
{ _readableState: [Object],
readable: true,
domain: null,
_events: [Object],
_eventsCount: 1,
_maxListeners: undefined,
path: '/tmp/tmp-unsigned-xpi-25600yuqzFN3Alxa0/#addonname-0.x.y.xpi',
fd: null,
flags: 'r',
mode: 438,
start: undefined,
end: undefined,
autoClose: true,
pos: undefined } },
headers: { Authorization: '<REDACTED>', Accept: 'application/json' } }
JPM [debug] [API] <- { headers:
{ allow: 'GET, PUT, HEAD, OPTIONS',
'content-type': 'application/json',
date: 'Thu, 17 Dec 2015 10:17:23 GMT',
server: 'nginx',
'set-cookie': '<REDACTED>',
'strict-transport-security': 'max-age=31536000',
vary: 'Accept, X-Mobile, User-Agent',
'www-authenticate': 'JWT realm="api"',
'x-frame-options': 'DENY',
'content-length': '51',
connection: 'Close' },
response: { detail: 'Incorrect authentication credentials.' } }
JPM [error] FAIL
Error: Received bad response from the server while requesting https://addons.mozilla.org/api/v3/addons/%40addonname/versions/0.x.y/
status: 401
response: {"detail":"Incorrect authentication credentials."}
headers: {"allow":"GET, PUT, HEAD, OPTIONS","content-type":"application/json","date":"Thu, 17 Dec 2015 10:17:23 GMT","server":"nginx","set-cookie":["multidb_pin_writes=y; expires=Thu, 17-Dec-2015 10:17:38 GMT; Max-Age=15; Path=/"],"strict-transport-security":"max-age=31536000","vary":"Accept, X-Mobile, User-Agent","www-authenticate":"JWT realm=\"api\"","x-frame-options":"DENY","content-length":"51","connection":"Close"}
at /usr/local/lib/node_modules/jpm/lib/amo-client.js:85:13
at tryCatchReject (/usr/local/lib/node_modules/jpm/node_modules/when/lib/makePromise.js:845:30)
at runContinuation1 (/usr/local/lib/node_modules/jpm/node_modules/when/lib/makePromise.js:804:4)
at Fulfilled.when (/usr/local/lib/node_modules/jpm/node_modules/when/lib/makePromise.js:592:4)
at Pending.run (/usr/local/lib/node_modules/jpm/node_modules/when/lib/makePromise.js:483:13)
at Scheduler._drain (/usr/local/lib/node_modules/jpm/node_modules/when/lib/Scheduler.js:62:19)
at Scheduler.drain (/usr/local/lib/node_modules/jpm/node_modules/when/lib/Scheduler.js:27:9)
at doNTCallback0 (node.js:417:9)
at process._tickCallback (node.js:346:13)
These posts (1), (2) show that this error can occur if your local clock is off by more than 60 seconds with the Mozilla server.
This is also documented in the Olympia 3.0 Authentication API (see exp parameter):
iat
This is a standard JWT claim indicating the issued at time. It should be a Unix epoch timestamp and must be in UTC time.
exp
This is a standard JWT claim indicating the expiration time. It should be a Unix epoch timestamp in UTC time and must be no longer
than 60 seconds past the issued at time.
Note:
If you’re having trouble authenticating, make sure your system clock
is correct and consider synchronizing it with something like NTP
(Network Time Protocol).

How to setup custom DNS with Azure Websites Preview?

I created a new Azure Website, using Umbraco as the CMS. I got a page up and going, and I already have a .co.nz domain with www.domains4less.com. There's a whole lot of stuff on the internet about pointing URLs to Azure, but that seems to be more of a redirection service than anything (i.e. my URLs still use azurewebsites.net once I land on my site).
Has anybody had any luck getting it to go?
Here's the error I get when I try adding the DNS entry to Azure (I'm in reserved mode, reemdairy is the name of the website):
There was an error processing your request. Please try again in a few
moments.
Browser: 5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.5 (KHTML, like
Gecko) Chrome/19.0.1084.56 Safari/536.5 User language: undefined
Portal Version: 6.0.6002.18488 (rd_auxportal_stable.120609-0259)
Subscriptions: 3aabe358-d178-4790-a97b-ffba902b2851 User email
address: touchmaniacs#live.com
Last 10 Requests
message: Failure: Ajax call to: Websites/UpdateConfig. failed with
status: error (500) in 2.57 seconds. x-ms-client-request-id was:
38834edf-c9f3-46bb-a1f7-b2839c692bcf-2012-06-12 22:25:14Z dateTime:
Wed Jun 13 2012 10:25:17 GMT+1200 (New Zealand Standard Time)
durationSeconds: 2.57 url: Websites/UpdateConfig status: 500
textStatus: error clientMsRequestId:
38834edf-c9f3-46bb-a1f7-b2839c692bcf-2012-06-12 22:25:14Z sessionId:
09c72263-6ce7-422b-84d7-4c21acded759 referrer:
https://manage.windowsazure.com/#Workspaces/WebsiteExtension/Website/reemdairy/configure
host: manage.windowsazure.com response: {"message":"Try again. Contact
support if the problem persists.","ErrorMessage":"Try again. Contact
support if the problem
persists.","httpStatusCode":"InternalServerError","operationTrackingId":"","stackTrace":null}
message: Complete: Ajax call to: Websites/GetConfig. completed with
status: success (200) in 1.021 seconds. x-ms-client-request-id was:
a0cdcced-13d0-44e2-866d-e0b061b9461b-2012-06-12 22:24:43Z dateTime:
Wed Jun 13 2012 10:24:44 GMT+1200 (New Zealand Standard Time)
durationSeconds: 1.021 url: Websites/GetConfig status: 200 textStatus:
success clientMsRequestId:
a0cdcced-13d0-44e2-866d-e0b061b9461b-2012-06-12 22:24:43Z sessionId:
09c72263-6ce7-422b-84d7-4c21acded759 referrer:
https://manage.windowsazure.com/#Workspaces/WebsiteExtension/Website/reemdairy/configure
host: manage.windowsazure.com
message: Complete: Ajax call to:
https://manage.windowsazure.com/Service/OperationTracking?subscriptionId=3aabe358-d178-4790-a97b-ffba902b2851.
completed with status: success (200) in 1.887 seconds.
x-ms-client-request-id was:
a7689fe9-b9f9-4d6c-8926-734ec9a0b515-2012-06-12 22:24:40Z dateTime:
Wed Jun 13 2012 10:24:42 GMT+1200 (New Zealand Standard Time)
durationSeconds: 1.887 url:
https://manage.windowsazure.com/Service/OperationTracking?subscriptionId=3aabe358-d178-4790-a97b-ffba902b2851
status: 200 textStatus: success clientMsRequestId:
a7689fe9-b9f9-4d6c-8926-734ec9a0b515-2012-06-12 22:24:40Z sessionId:
09c72263-6ce7-422b-84d7-4c21acded759 referrer:
https://manage.windowsazure.com/#Workspaces/WebsiteExtension/Website/reemdairy/configure
host: manage.windowsazure.com
message: Complete: Ajax call to: /Service/GetUserSettings. completed
with status: success (200) in 0.941 seconds. x-ms-client-request-id
was: 805e554d-1e2e-4214-afd5-be87c0f255d1-2012-06-12 22:24:40Z
dateTime: Wed Jun 13 2012 10:24:40 GMT+1200 (New Zealand Standard
Time) durationSeconds: 0.941 url: /Service/GetUserSettings status: 200
textStatus: success clientMsRequestId:
805e554d-1e2e-4214-afd5-be87c0f255d1-2012-06-12 22:24:40Z sessionId:
09c72263-6ce7-422b-84d7-4c21acded759 referrer:
https://manage.windowsazure.com/#Workspaces/WebsiteExtension/Website/reemdairy/configure
host: manage.windowsazure.com
message: Complete: Ajax call to:
Extensions/ApplicationsExtension/SqlAzure/ClusterSuffix. completed
with status: success (200) in 0.483 seconds. x-ms-client-request-id
was: 85157ceb-c538-40ca-8c1e-5cc07c57240f-2012-06-12 22:24:39Z
dateTime: Wed Jun 13 2012 10:24:40 GMT+1200 (New Zealand Standard
Time) durationSeconds: 0.483 url:
Extensions/ApplicationsExtension/SqlAzure/ClusterSuffix status: 200
textStatus: success clientMsRequestId:
85157ceb-c538-40ca-8c1e-5cc07c57240f-2012-06-12 22:24:39Z sessionId:
09c72263-6ce7-422b-84d7-4c21acded759 referrer:
https://manage.windowsazure.com/#Workspaces/WebsiteExtension/Website/reemdairy/configure
host: manage.windowsazure.com
message: Complete: Ajax call to:
Extensions/ApplicationsExtension/SqlAzure/GetClientIp. completed with
status: success (200) in 0.309 seconds. x-ms-client-request-id was:
2eb194b6-66ca-49e2-9016-e0f89164314c-2012-06-12 22:24:39Z dateTime:
Wed Jun 13 2012 10:24:40 GMT+1200 (New Zealand Standard Time)
durationSeconds: 0.309 url:
Extensions/ApplicationsExtension/SqlAzure/GetClientIp status: 200
textStatus: success clientMsRequestId:
2eb194b6-66ca-49e2-9016-e0f89164314c-2012-06-12 22:24:39Z sessionId:
09c72263-6ce7-422b-84d7-4c21acded759 referrer:
https://manage.windowsazure.com/#Workspaces/WebsiteExtension/Website/reemdairy/configure
host: manage.windowsazure.com
message: Complete: Ajax call to:
Extensions/ApplicationsExtension/SqlAzure/DefaultServerLocation.
completed with status: success (200) in 0.309 seconds.
x-ms-client-request-id was:
1bc165ef-2081-48f2-baed-16c6edf8ea67-2012-06-12 22:24:39Z dateTime:
Wed Jun 13 2012 10:24:40 GMT+1200 (New Zealand Standard Time)
durationSeconds: 0.309 url:
Extensions/ApplicationsExtension/SqlAzure/DefaultServerLocation
status: 200 textStatus: success clientMsRequestId:
1bc165ef-2081-48f2-baed-16c6edf8ea67-2012-06-12 22:24:39Z sessionId:
09c72263-6ce7-422b-84d7-4c21acded759 referrer:
https://manage.windowsazure.com/#Workspaces/WebsiteExtension/Website/reemdairy/configure
host: manage.windowsazure.com
message: Complete: Ajax call to:
Extensions/ApplicationsExtension/SqlAzure/ServerLocations. completed
with status: success (200) in 0.309 seconds. x-ms-client-request-id
was: e1fba7df-6a12-47f8-9434-bf17ca7d93f4-2012-06-12 22:24:39Z
dateTime: Wed Jun 13 2012 10:24:40 GMT+1200 (New Zealand Standard
Time) durationSeconds: 0.309 url:
Extensions/ApplicationsExtension/SqlAzure/ServerLocations status: 200
textStatus: success clientMsRequestId:
e1fba7df-6a12-47f8-9434-bf17ca7d93f4-2012-06-12 22:24:39Z sessionId:
09c72263-6ce7-422b-84d7-4c21acded759 referrer:
https://manage.windowsazure.com/#Workspaces/WebsiteExtension/Website/reemdairy/configure
host: manage.windowsazure.com
Its good that you have Windows Azure Website (Reserve) because only Reserve websites can have CNAME or A record point to your_site.azurewebsites.net.
May I ask what are you setting, CNAME or A Record?
Here is the step by step direction on setting either CNAME (which is suggested for websites and webroles) so please follow and suggest at what step you have an issue:
Configuring a Custom Domain Name in Windows Azure
I've successfully done this with the exact same setup (Umbraco, running on Azure with a domains4less.co.nz domain). DNS setup with Domains4less can be obtuse - but here's how to setup a CNAME to your app:
From 'Manage Domains', hit 'Setup' on the domain you want. Next page, 'Advanced Control Panel' and login. Expand 'Services', open 'Domain Templates' and click 'Add a domain template'
Give your template a name (e.g 'yourdomain.co.nz'), and scroll down to the CNAME Records section. Click 'Add a CNAME record', Prefix field: www, Destination Address: YOURAPPNAME.azurewebsites.net.. Save.
Open 'Domains' in the left-side menu and click '[edit]' on the domain you want to setup, open the 'Advanced Mode' tab and choose your template from the Template drop-down list. Save, done.
Phew.

Resources