How to create subscription using pubsub emulator http API? - emulation

After starting the pubsub emulator, I'm trying to create a topic and a subscription using the HTTP API. Creating a topic succeeds, but I cannot figure out why creating a subscription doesn't. Am I doing something wrong or is this a bug in the tool? You can see the logs below:
$ curl -s -X PUT http://localhost:8085/v1/projects/myproject/topics/mytopic
{
"name": "projects/myproject/topics/mytopic"
}
$ curl -s -X PUT http://localhost:8085/v1/projects/myproject/subscriptions/mysub \
--data '{"topic":"projects/myproject/topics/mytopic"}'
Not Found
On the emulator side, I see the following:
# create topic logs
[pubsub] Apr 29, 2020 10:37:19 AM io.gapi.emulators.grpc.GrpcServer$3 operationComplete
[pubsub] INFO: Adding handler(s) to newly registered Channel.
[pubsub] Apr 29, 2020 10:37:19 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
[pubsub] INFO: Detected non-HTTP/2 connection.
[pubsub] Apr 29, 2020 10:37:19 AM io.gapi.emulators.grpc.GrpcServer$3 operationComplete
[pubsub] INFO: Adding handler(s) to newly registered Channel.
[pubsub] Apr 29, 2020 10:37:19 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
[pubsub] INFO: Detected HTTP/2 connection.
# create subscription logs
[pubsub] Apr 29, 2020 10:37:27 AM io.gapi.emulators.grpc.GrpcServer$3 operationComplete
[pubsub] INFO: Adding handler(s) to newly registered Channel.
[pubsub] Apr 29, 2020 10:37:27 AM io.gapi.emulators.netty.HttpVersionRoutingHandler channelRead
[pubsub] INFO: Detected non-HTTP/2 connection.
[pubsub] Apr 29, 2020 10:37:27 AM io.gapi.emulators.netty.NotFoundHandler handleRequest
[pubsub] INFO: Unknown request URI: /v1/projects/myproject/subscriptions/mysub

Even though creating a topic works with the above command (no content type necessary), in order to send data with the --data '...' option, you also need to send the content type header. So the following command does work:
$ curl -s -X PUT http://localhost:8085/v1/projects/myproject/subscriptions/mysub \
-H 'content-type: application/json' \
--data '{"topic":"projects/myproject/topics/mytopic"}'

Related

Google Tsunami Security Scanner not working as expected

I have an Ubuntu VM 20.04LTS where I installed tsunami-security-scanner and also run the docker example with a vulnerable image.
The example works as expected founding a vulnerability: JUPYTER_NOTEBOOK_EXPOSED_UI
When I tried with a live target I have, I got the following JSON otuput:
{
"scanStatus": "PARTIALLY_SUCCEEDED",
"scanStartTimestamp": "2020-07-09T03:35:32.611Z",
"scanDuration": "45.315s",
"fullDetectionReports": {
},
"statusMessage": "Failed plugins:\n/Tsunami Team (tsunami-dev#google.com)/VULN_DETECTION/NcrackWeakCredentialDetectorPlugin/0.1\n/Tsunami Team (tsunami-dev#google.com)/VULN_DETECTION/WordPressInstallPageDetector/0.1"
}
NcrackWeakCredentialDetectorPlugin and WordPressInstallPageDetector have failed but don't know why.
This is the console output:
(...)
Jul 10, 2020 1:00:33 AM com.google.tsunami.workflow.DefaultScanningWorkflow fingerprintNetworkServices
INFO: Port scanning phase done, moving to service fingerprinting phase with '0' fingerprinter(s) selected.
Jul 10, 2020 1:00:34 AM com.google.tsunami.workflow.DefaultScanningWorkflow detectVulnerabilities
INFO: Service fingerprinting phase done, moving to vuln detection phase.
Jul 10, 2020 1:00:34 AM com.google.tsunami.plugins.detectors.credentials.ncrack.NcrackWeakCredentialDetector detect
INFO: Starting weak credential detection using ncrack.
Jul 10, 2020 1:00:34 AM com.google.tsunami.plugins.detectors.exposedui.hadoop.yarn.YarnExposedManagerApiDetector detect
INFO: Starting unauthenticated Apache Yarn ResourceManager API detection
Jul 10, 2020 1:00:34 AM com.google.tsunami.plugins.detectors.exposedui.jenkins.JenkinsExposedUiDetector detect
INFO: Starting exposed ui detection for Jenkins
Jul 10, 2020 1:00:34 AM com.google.tsunami.plugins.detectors.exposedui.jupyter.JupyterExposedUiDetector detect
INFO: Starting exposed ui detection for Jupyter Notebook
Jul 10, 2020 1:00:34 AM com.google.tsunami.plugins.detectors.exposedui.wordpress.WordPressInstallPageDetector detect
INFO: Starting unfinished install page detection for WordPress.
Jul 10, 2020 1:00:34 AM com.google.tsunami.common.net.http.HttpClient send
INFO: Sending HTTP 'GET' request to 'http://xxx.xxx.xxx.xxx/terminals/1'.
Jul 10, 2020 1:00:34 AM com.google.tsunami.common.net.http.HttpClient send
INFO: Sending HTTP 'GET' request to 'http://xxx.xxx.xxx.xxx/cluster/cluster'.
Jul 10, 2020 1:00:34 AM com.google.tsunami.common.net.http.HttpClient send
INFO: Sending HTTP 'GET' request to 'http://xxx.xxx.xxx.xxx/view/all/newJob'.
Jul 10, 2020 1:00:34 AM com.google.tsunami.common.net.http.HttpClient send
INFO: Sending HTTP 'GET' request to 'http://xxx.xxx.xxx.xxx/wp-login.php'.
Jul 10, 2020 1:00:34 AM com.google.tsunami.common.net.http.HttpClient send
INFO: Sending HTTP 'GET' request to 'http://xxx.xxx.xxx.xxx/wp-admin/install.php?step=1'.
Jul 10, 2020 1:00:35 AM com.google.tsunami.common.net.http.HttpClient parseResponse
INFO: Received HTTP response with code '404' for request to 'http://xxx.xxx.xxx.xxx/view/all/newJob'.
Jul 10, 2020 1:00:35 AM com.google.tsunami.common.net.http.HttpClient parseResponse
INFO: Received HTTP response with code '404' for request to 'http://xxx.xxx.xxx.xxx/cluster/cluster'.
Jul 10, 2020 1:00:35 AM com.google.tsunami.common.net.http.HttpClient parseResponse
INFO: Received HTTP response with code '404' for request to 'http://xxx.xxx.xxx.xxx/terminals/1'.
Jul 10, 2020 1:00:35 AM com.google.tsunami.common.net.http.HttpClient parseResponse
INFO: Received HTTP response with code '200' for request to 'http://xxx.xxx.xxx.xxx/wp-login.php'.
Jul 10, 2020 1:00:35 AM com.google.tsunami.plugin.PluginExecutorImpl buildFailedResult
WARNING: Plugin '/Tsunami Team (tsunami-dev#google.com)/VULN_DETECTION/NcrackWeakCredentialDetectorPlugin/0.1' failed.
Jul 10, 2020 1:00:35 AM com.google.tsunami.plugins.detectors.exposedui.hadoop.yarn.YarnExposedManagerApiDetector detect
INFO: YarnExposedManagerApiDetector finished, detected '0' vulns.
Jul 10, 2020 1:00:35 AM com.google.tsunami.common.net.http.HttpClient parseResponse
INFO: Received HTTP response with code '200' for request to 'http://xxx.xxx.xxx.xxx/wp-admin/install.php?step=1'.
Jul 10, 2020 1:00:35 AM com.google.tsunami.plugin.PluginExecutorImpl buildFailedResult
WARNING: Plugin '/Tsunami Team (tsunami-dev#google.com)/VULN_DETECTION/WordPressInstallPageDetector/0.1' failed.
Jul 10, 2020 1:00:35 AM com.google.tsunami.workflow.DefaultScanningWorkflow generateScanResults
INFO: Tsunami scanning workflow done. Generating scan results.
Jul 10, 2020 1:00:35 AM com.google.tsunami.workflow.DefaultScanningWorkflow lambda$runAsync$0
INFO: Tsunami scanning workflow traces:
Port scanning phase (28.90 s) with 1 plugin(s):
/Tsunami Team (tsunami-dev#google.com)/PORT_SCAN/NmapPortScanner/0.1
Service fingerprinting phase (759.9 ms) with 0 plugin(s):
Vuln detection phase (694.7 ms) with 5 plugin(s):
/Tsunami Team (tsunami-dev#google.com)/VULN_DETECTION/NcrackWeakCredentialDetectorPlugin/0.1 was selected for the following services: http (TCP, port 80), ssh (TCP, port 8080)
/Tsunami Team (tsunami-dev#google.com)/VULN_DETECTION/YarnExposedManagerApiDetector/0.1 was selected for the following services: http (TCP, port 80), ssh (TCP, port 8080)
/Tsunami Team (tsunami-dev#google.com)/VULN_DETECTION/JenkinsExposedUiDetector/0.1 was selected for the following services: http (TCP, port 80), ssh (TCP, port 8080)
/Tsunami Team (tsunami-dev#google.com)/VULN_DETECTION/JupyterExposedUiDetector/0.1 was selected for the following services: http (TCP, port 80), ssh (TCP, port 8080)
/Tsunami Team (tsunami-dev#google.com)/VULN_DETECTION/WordPressInstallPageDetector/0.1 was selected for the following services: http (TCP, port 80), ssh (TCP, port 8080)
# of detected vulnerability: 0.
Jul 10, 2020 1:00:35 AM com.google.tsunami.main.cli.TsunamiCli run
INFO: Tsunami scan finished, saving results.
Jul 10, 2020 1:00:35 AM com.google.tsunami.common.io.archiving.RawFileArchiver archive
INFO: Archiving data to file system with filename '/tmp/tsunami-output.json'.
Jul 10, 2020 1:00:35 AM com.google.tsunami.main.cli.TsunamiCli run
INFO: TsunamiCli finished...
Jul 10, 2020 1:00:35 AM com.google.tsunami.main.cli.TsunamiCli main
INFO: Full Tsunami scan took 38.52 s.
kali#kali:~/tsunami/plugins$ ls -l
total 340
-rw-r--r-- 1 kali kali 7145 Jul 9 15:13 exposed_hadoop_yarn_api-0.0.1-SNAPSHOT.jar
-rw-r--r-- 1 kali kali 6179 Jul 9 15:13 exposed_jenkins_ui-0.0.1-SNAPSHOT.jar
-rw-r--r-- 1 kali kali 5888 Jul 9 15:13 exposed_jupyter_notebook-0.0.1-SNAPSHOT.jar
-rw-r--r-- 1 kali kali 41535 Jul 9 15:13 ncrack_weak_credential_detector-0.0.1-SNAPSHOT.jar
-rw-r--r-- 1 kali kali 267309 Jul 9 15:13 nmap_port_scanner-0.0.1-SNAPSHOT.jar
-rw-r--r-- 1 kali kali 6248 Jul 9 15:13 wordpress_exposed_installation_page-0.0.1-SNAPSHOT.jar
kali#kali:~/tsunami$ nmap -v
Starting Nmap 7.80 ( https://nmap.org ) at 2020-07-10 01:02 EDT
Read data files from: /usr/bin/../share/nmap
WARNING: No targets were specified, so 0 hosts scanned.
Nmap done: 0 IP addresses (0 hosts up) scanned in 0.07 seconds
kali#kali:~/tsunami$ ncrack -v
Starting Ncrack 0.7 ( http://ncrack.org ) at 2020-07-10 01:03 EDT
No services specified!
QUITTING!
I made a focus on these two lines:
WARNING: Plugin '/Tsunami Team (tsunami-dev#google.com)/VULN_DETECTION/NcrackWeakCredentialDetectorPlugin/0.1' failed.
WARNING: Plugin '/Tsunami Team (tsunami-dev#google.com)/VULN_DETECTION/WordPressInstallPageDetector/0.1' failed.
The same plugins worked before against the docker example.
I also set up a Kali 2020.2 VM with tsunami-security-scanner, just to check what happens to have the docker image in a VM and running the Tsunami in another VM.
Again worked as expected.
I can't figure out what is failing.
Thanks in advance!

pubsub-emulator throw error and publisher throw "Retry total timeout exceeded before any response was received" when publish 50k messages

Environment details
OS: uname -a => Darwin US_C02WG0GXHV2V 17.7.0 Darwin Kernel Version 17.7.0: Thu Jan 23 07:05:23 PST 2020; root:xnu-4570.71.69~1/RELEASE_X86_64 x86_64
Node.js version: node -v => v10.16.2
npm version: npm -v => 6.14.4
#google-cloud/pubsub version: "#google-cloud/pubsub": "^1.7.3"
Description
When I tried to publish 50k messages, the pubsub-emulator throw below error infinitely:
Error stack
[pubsub] May 06, 2020 2:47:20 PM io.grpc.netty.NettyServerHandler onStreamError
[pubsub] 警告: Stream Error
[pubsub] io.netty.handler.codec.http2.Http2Exception$StreamException: Stream closed before write could take place
[pubsub] at io.netty.handler.codec.http2.Http2Exception.streamError(Http2Exception.java:149)
[pubsub] at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$FlowState.cancel(DefaultHttp2RemoteFlowController.java:481)
[pubsub] at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$1.onStreamClosed(DefaultHttp2RemoteFlowController.java:105)
[pubsub] at io.netty.handler.codec.http2.DefaultHttp2Connection.notifyClosed(DefaultHttp2Connection.java:356)
[pubsub] at io.netty.handler.codec.http2.DefaultHttp2Connection$ActiveStreams.removeFromActiveStreams(DefaultHttp2Connection.java:1000)
[pubsub] at io.netty.handler.codec.http2.DefaultHttp2Connection$ActiveStreams.deactivate(DefaultHttp2Connection.java:956)
[pubsub] at io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultStream.close(DefaultHttp2Connection.java:512)
[pubsub] at io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultStream.close(DefaultHttp2Connection.java:518)
[pubsub] at io.netty.handler.codec.http2.Http2ConnectionHandler.closeStream(Http2ConnectionHandler.java:599)
[pubsub] at io.netty.handler.codec.http2.Http2ConnectionHandler.processRstStreamWriteResult(Http2ConnectionHandler.java:872)
[pubsub] at io.netty.handler.codec.http2.Http2ConnectionHandler.access$1000(Http2ConnectionHandler.java:66)
[pubsub] at io.netty.handler.codec.http2.Http2ConnectionHandler$3.operationComplete(Http2ConnectionHandler.java:796)
[pubsub] at io.netty.handler.codec.http2.Http2ConnectionHandler$3.operationComplete(Http2ConnectionHandler.java:793)
[pubsub] at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:502)
[pubsub] at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:476)
[pubsub] at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:415)
[pubsub] at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:540)
[pubsub] at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:529)
[pubsub] at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:101)
[pubsub] at io.netty.util.internal.PromiseNotificationUtil.trySuccess(PromiseNotificationUtil.java:48)
[pubsub] at io.netty.channel.ChannelOutboundBuffer.safeSuccess(ChannelOutboundBuffer.java:703)
[pubsub] at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:258)
[pubsub] at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:338)
[pubsub] at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:428)
[pubsub] at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:939)
[pubsub] at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:360)
[pubsub] at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:906)
[pubsub] at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1370)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:739)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:731)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:717)
[pubsub] at io.netty.handler.logging.LoggingHandler.flush(LoggingHandler.java:265)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:739)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:731)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:717)
[pubsub] at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:739)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:731)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:717)
[pubsub] at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:201)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:739)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:731)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:717)
[pubsub] at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:978)
[pubsub] at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:253)
[pubsub] at io.grpc.netty.WriteQueue.flush(WriteQueue.java:118)
[pubsub] at io.grpc.netty.WriteQueue.access$000(WriteQueue.java:32)
[pubsub] at io.grpc.netty.WriteQueue$1.run(WriteQueue.java:44)
[pubsub] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
[pubsub] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
[pubsub] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:495)
[pubsub] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905)
[pubsub] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[pubsub] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[pubsub] at java.lang.Thread.run(Thread.java:748)
[pubsub]
[pubsub] May 06, 2020 2:47:20 PM io.grpc.netty.NettyServerHandler onStreamError
[pubsub] 警告: Stream Error
[pubsub] io.netty.handler.codec.http2.Http2Exception$StreamException: Stream closed before write could take place
[pubsub] at io.netty.handler.codec.http2.Http2Exception.streamError(Http2Exception.java:149)
[pubsub] at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$FlowState.cancel(DefaultHttp2RemoteFlowController.java:481)
[pubsub] at io.netty.handler.codec.http2.DefaultHttp2RemoteFlowController$1.onStreamClosed(DefaultHttp2RemoteFlowController.java:105)
[pubsub] at io.netty.handler.codec.http2.DefaultHttp2Connection.notifyClosed(DefaultHttp2Connection.java:356)
[pubsub] at io.netty.handler.codec.http2.DefaultHttp2Connection$ActiveStreams.removeFromActiveStreams(DefaultHttp2Connection.java:1000)
[pubsub] at io.netty.handler.codec.http2.DefaultHttp2Connection$ActiveStreams.deactivate(DefaultHttp2Connection.java:956)
[pubsub] at io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultStream.close(DefaultHttp2Connection.java:512)
[pubsub] at io.netty.handler.codec.http2.DefaultHttp2Connection$DefaultStream.close(DefaultHttp2Connection.java:518)
[pubsub] at io.netty.handler.codec.http2.Http2ConnectionHandler.closeStream(Http2ConnectionHandler.java:599)
[pubsub] at io.netty.handler.codec.http2.Http2ConnectionHandler.processRstStreamWriteResult(Http2ConnectionHandler.java:872)
[pubsub] at io.netty.handler.codec.http2.Http2ConnectionHandler.access$1000(Http2ConnectionHandler.java:66)
[pubsub] at io.netty.handler.codec.http2.Http2ConnectionHandler$3.operationComplete(Http2ConnectionHandler.java:796)
[pubsub] at io.netty.handler.codec.http2.Http2ConnectionHandler$3.operationComplete(Http2ConnectionHandler.java:793)
[pubsub] at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:502)
[pubsub] at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:476)
[pubsub] at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:415)
[pubsub] at io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:540)
[pubsub] at io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:529)
[pubsub] at io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:101)
[pubsub] at io.netty.util.internal.PromiseNotificationUtil.trySuccess(PromiseNotificationUtil.java:48)
[pubsub] at io.netty.channel.ChannelOutboundBuffer.safeSuccess(ChannelOutboundBuffer.java:703)
[pubsub] at io.netty.channel.ChannelOutboundBuffer.remove(ChannelOutboundBuffer.java:258)
[pubsub] at io.netty.channel.ChannelOutboundBuffer.removeBytes(ChannelOutboundBuffer.java:338)
[pubsub] at io.netty.channel.socket.nio.NioSocketChannel.doWrite(NioSocketChannel.java:428)
[pubsub] at io.netty.channel.AbstractChannel$AbstractUnsafe.flush0(AbstractChannel.java:939)
[pubsub] at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.flush0(AbstractNioChannel.java:360)
[pubsub] at io.netty.channel.AbstractChannel$AbstractUnsafe.flush(AbstractChannel.java:906)
[pubsub] at io.netty.channel.DefaultChannelPipeline$HeadContext.flush(DefaultChannelPipeline.java:1370)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:739)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:731)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:717)
[pubsub] at io.netty.handler.logging.LoggingHandler.flush(LoggingHandler.java:265)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:739)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:731)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:717)
[pubsub] at io.netty.channel.ChannelDuplexHandler.flush(ChannelDuplexHandler.java:117)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:739)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:731)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:717)
[pubsub] at io.netty.handler.codec.http2.Http2ConnectionHandler.flush(Http2ConnectionHandler.java:201)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush0(AbstractChannelHandlerContext.java:739)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.invokeFlush(AbstractChannelHandlerContext.java:731)
[pubsub] at io.netty.channel.AbstractChannelHandlerContext.flush(AbstractChannelHandlerContext.java:717)
[pubsub] at io.netty.channel.DefaultChannelPipeline.flush(DefaultChannelPipeline.java:978)
[pubsub] at io.netty.channel.AbstractChannel.flush(AbstractChannel.java:253)
[pubsub] at io.grpc.netty.WriteQueue.flush(WriteQueue.java:118)
[pubsub] at io.grpc.netty.WriteQueue.access$000(WriteQueue.java:32)
[pubsub] at io.grpc.netty.WriteQueue$1.run(WriteQueue.java:44)
[pubsub] at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:163)
[pubsub] at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:404)
[pubsub] at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:495)
[pubsub] at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:905)
[pubsub] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
[pubsub] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
[pubsub] at java.lang.Thread.run(Thread.java:748)
[pubsub]
My publisher also throws many same errors:
{ Error: Retry total timeout exceeded before any response was received
at repeat (/Users/ldu020/workspace/xxx/xxx-master/workflow/node_modules/google-gax/src/normalCalls/retries.ts:83:23)
at Timeout.setTimeout (/Users/ldu020/workspace/xxx/xxx-master/workflow/node_modules/google-gax/src/normalCalls/retries.ts:124:13)
at ontimeout (timers.js:436:11)
at tryOnTimeout (timers.js:300:5)
at listOnTimeout (timers.js:263:5)
at Timer.processTimers (timers.js:223:10) code: 4 }
{ Error: Retry total timeout exceeded before any response was received
at repeat (/Users/ldu020/workspace/xxx/xxx-master/workflow/node_modules/google-gax/src/normalCalls/retries.ts:83:23)
at Timeout.setTimeout (/Users/ldu020/workspace/xxx/xxx-master/workflow/node_modules/google-gax/src/normalCalls/retries.ts:124:13)
at ontimeout (timers.js:436:11)
at tryOnTimeout (timers.js:300:5)
at listOnTimeout (timers.js:263:5)
at Timer.processTimers (timers.js:223:10) code: 4 }
My subscriber throw below errors:
[2020-05-06T06:29:08.932Z]Received message 46689:
Data: message payload 3998
Attributes: {}
[2020-05-06T06:29:08.932Z]Received message 46690:
Data: message payload 3999
Attributes: {}
ERROR: Error: Failed to "acknowledge" for 500 message(s). Reason: 4 DEADLINE_EXCEEDED: Deadline exceeded
ERROR: Error: Failed to "acknowledge" for 100 message(s). Reason: 4 DEADLINE_EXCEEDED: Deadline exceeded
ERROR: Error: Failed to "acknowledge" for 200 message(s). Reason: 4 DEADLINE_EXCEEDED: Deadline exceeded
ERROR: Error: Failed to "modifyAckDeadline" for 400 message(s). Reason: 4 DEADLINE_EXCEEDED: Deadline exceeded
ERROR: Error: Failed to "modifyAckDeadline" for 500 message(s). Reason: 4 DEADLINE_EXCEEDED: Deadline exceeded
ERROR: Error: Failed to "modifyAckDeadline" for 100 message(s). Reason: 4 DEADLINE_EXCEEDED: Deadline exceeded
ERROR: Error: Failed to "modifyAckDeadline" for 200 message(s). Reason: 4 DEADLINE_EXCEEDED: Deadline exceeded
ERROR: Error: Failed to connect to channel. Reason: Failed to connect before the deadline
Actually, I don't know it's a bug or I am using pubsub incorrectly. I found there are some questions on StackOverflow and issues on GitHub:
Retry total timeout exceeded before any response was received in GCP PubSub
https://github.com/googleapis/nodejs-pubsub/issues/770
https://github.com/googleapis/gax-nodejs/issues/741
But didn't find a solution.
Steps to reproduce
I made a minimal code example for reproducing it. Please tell what's going on? Thanks!
Repo: https://github.com/mrdulin/nodejs-gcp/tree/master/src/pubsub/pubsub-emulator
Start pubsub-emulator
gcloud beta emulators pubsub start --project=$PROJECT_ID
Create topic:
npx ts-node ./publisher.ts create pubsub-emulator-t1
Create a subscription for the topic:
npx ts-node ./subscriber.ts create pubsub-emulator-t1 pubsub-emulator-t1-sub
Listen for the messages:
npx ts-node ./subscriber.ts receive pubsub-emulator-t1-sub
Publish 50k messages:
npx ts-node ./publisher.ts publish pubsub-emulator-t1
Additional information
I also got the error: Retry total timeout exceeded before any response was received in production environments. So it may not be a problem of pubsub-emulator
#google/pubsub version: "#google-cloud/pubsub": "^1.6.0"
That's why I am trying to make an example to reproduce it. But can't figure out what's going on.
The only way I can reproduce this error now is to post 50k messages.
Test cases: https://gist.github.com/mrdulin/79f1689a9baaafaef90fcad42646bf6d

Jenkins GitHub Webhook not working

I am running Jenkins 2.89.4 on the official Docker LTS image in an Amazon Linux EC2 instance. I am using the GitHub webhook and when I test it I get this on the docker logs;
Feb 20, 2018 4:48:23 PM io.relution.jenkins.awssqs.logging.Log write
INFO: 000380 Create new SQSTriggerQueue(8d894fcf-becd-449c-a65b-2a404c57b7bc, , )
Feb 20, 2018 4:48:23 PM hudson.plugins.scm_sync_configuration.ScmSyncConfigurationBusiness queueChangeSet
INFO: Queue of changeset A hudson.scm.SubversionSCM.xml
A jenkins.model.ArtifactManagerConfiguration.xml
A hudson.tasks.Shell.xml
A scm-sync-configuration.xml
A hudson.tasks.Mailer.xml
A config.xml
A jenkins.plugins.awslogspublisher.AWSLogsConfig.xml
A hudson.plugins.jira.JiraProjectProperty.xml
A hudson.plugins.git.GitSCM.xml
A jenkins.metrics.api.MetricsAccessKey.xml
A hudson.plugins.timestamper.TimestamperConfig.xml
A jenkins.model.JenkinsLocationConfiguration.xml
A hudson.plugins.build_timeout.operations.BuildStepOperation.xml
A hudson.triggers.SCMTrigger.xml
A hudson.plugins.emailext.ExtendedEmailPublisher.xml
A hudson.maven.MavenModuleSet.xml
aborted (scm manipulator not settled !)
Feb 20, 2018 4:49:03 PM com.squareup.okhttp.internal.Platform$JdkWithJettyBootPlatform getSelectedProtocol
INFO: ALPN callback dropped: SPDY and HTTP/2 are disabled. Is alpn-boot on the boot class path?

Unable to connect to artifactory after service restart on CentOS7

I'm a total linux newbie and I'll drop the story here so hopefully someone knows what's going on.
I have a linux box running CentOS and I have artifactory running as a service. After I restarted the server everything started back up but I'm no longer able to connect to artifactory. I checked the artifactory log and everything says it started fine
artifactory.log
2017-02-09 22:15:12,149 [art-init] [INFO ] (o.a.s.ArtifactoryApplicationContext:360) - Artifactory application context is ready.
2017-02-09 22:15:12,152 [art-init] [INFO ] (o.a.w.s.ArtifactoryContextConfigListener:225) -
###########################################################
### Artifactory successfully started (7.122 seconds) ###
###########################################################
Then I checked the tomcat log to make sure that that started fine
catalina.log
INFO: Starting service Catalina
Feb 09, 2017 10:25:42 PM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/7.0.56
Feb 09, 2017 10:25:42 PM org.apache.catalina.startup.HostConfig deployDescriptor
INFO: Deploying configuration descriptor /opt/jfrog/artifactory/tomcat/conf/Catalina/localhost/artifactory.xml
Feb 09, 2017 10:25:47 PM org.apache.catalina.startup.HostConfig deployDescriptor
INFO: Deployment of configuration descriptor /opt/jfrog/artifactory/tomcat/conf/Catalina/localhost/artifactory.xml has finished in 4,865 ms
Feb 09, 2017 10:25:47 PM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deploying web application directory /opt/jfrog/artifactory/tomcat/webapps/ROOT
Feb 09, 2017 10:25:47 PM org.apache.catalina.startup.HostConfig deployDirectory
INFO: Deployment of web application directory /opt/jfrog/artifactory/tomcat/webapps/ROOT has finished in 246 ms
Feb 09, 2017 10:25:47 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["http-bio-8081"]
Feb 09, 2017 10:25:47 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler ["ajp-bio-8019"]
I'm at a point where I'm not even sure what to check. When I try to connect to the page I just get the "The Connection has time out".
If anyone knows anything I can check I'd be extremely grateful.
edit:
Running a netstat -tulnp it shows the ports as open
tcp6 0 0 127.0.0.1:8015 :::* LISTEN 3538/java
tcp6 0 0 :::8081 :::* LISTEN 3538/java
tcp6 0 0 :::8019 :::* LISTEN 3538/java
Turns out I had a firewall on.
Since this is an internal site I disabled it by
systemctl stop firewalld
Would it be worth my time to just allow those ports access?

JADE Hangs when executed on 64Bit Linux at Agent Invocation

I've got a pesky issue I hope i can request some assistance from the community on....
I have a Ubuntu 12.04LTS 64 Bit linux server on which I run an instance of JADE (no gui) that creates a container and connects to a Main RMA on my local machine just fine. I see the container pop up and life is good. The problem is when I re-invoke that same instance using an Agent - it hangs. Note: I've recompiled the PingAgent Example on the 64Bit machine with the same vintage of Java (8) as on my workstation (windows). Is it a 32 vs 64 bit thing or are there issues with Jade on Linux?
Any assistance would be greatly appreciated ....
Below are the particulars:
Good instance (no agent, no hanging ):
RUN:
versaggi#drools:/usr/local/jade$ java -classpath lib/jade.jar:classes jade.Boot -container -local-host 50.116.61.17 -host versaggi.game-host.org -port 1099
Result:
Aug 12, 2014 7:35:42 PM jade.core.Runtime beginContainer
INFO: ----------------------------------
This is JADE 4.3.2 - revision 6708 of 2014/03/28 15:19:44
downloaded in Open Source, under LGPL restrictions,
at http://jade.tilab.com/
----------------------------------------
Aug 12, 2014 7:35:42 PM jade.imtp.leap.LEAPIMTPManager initialize
INFO: Listening for intra-platform commands on address:
- jicp://50.116.61.17:1099
Aug 12, 2014 7:35:43 PM jade.core.BaseService init
INFO: Service jade.core.management.AgentManagement initialized
Aug 12, 2014 7:35:43 PM jade.core.BaseService init
INFO: Service jade.core.messaging.Messaging initialized
Aug 12, 2014 7:35:43 PM jade.core.BaseService init
INFO: Service jade.core.resource.ResourceManagement initialized
Aug 12, 2014 7:35:43 PM jade.core.BaseService init
INFO: Service jade.core.mobility.AgentMobility initialized
Aug 12, 2014 7:35:43 PM jade.core.BaseService init
INFO: Service jade.core.event.Notification initialized
Aug 12, 2014 7:35:43 PM jade.core.AgentContainerImpl joinPlatform
INFO: --------------------------------------
Agent container Container-7#50.116.61.17 is ready.
--------------------------------------------
Bad Instance (Agent Included in invocation, hangs ....)
versaggi#drools:/usr/local/jade$ java -classpath lib/jade.jar:classes jade.Boot -container -local-host 50.116.61.17 -host versaggi.game-host.org -port 1099 -**agents** ping1:examples.PingAgent.PingAgent
Aug 12, 2014 7:34:25 PM jade.core.Runtime beginContainer
INFO: ----------------------------------
This is JADE 4.3.2 - revision 6708 of 2014/03/28 15:19:44
downloaded in Open Source, under LGPL restrictions,
at http://jade.tilab.com/
----------------------------------------
Aug 12, 2014 7:34:25 PM jade.imtp.leap.LEAPIMTPManager initialize
INFO: Listening for intra-platform commands on address:
- jicp://50.116.61.17:1099
Aug 12, 2014 7:34:26 PM jade.core.BaseService init
INFO: Service jade.core.management.AgentManagement initialized
Aug 12, 2014 7:34:26 PM jade.core.BaseService init
INFO: Service jade.core.messaging.Messaging initialized
Aug 12, 2014 7:34:26 PM jade.core.BaseService init
INFO: Service jade.core.resource.ResourceManagement initialized
Aug 12, 2014 7:34:26 PM jade.core.BaseService init
INFO: Service jade.core.mobility.AgentMobility initialized
Aug 12, 2014 7:34:26 PM jade.core.BaseService init
INFO: Service jade.core.event.Notification initialized
**< *** HANGS HERE ***>**
**< *** KILLED FROM RAM HERE ***>**
Aug 12, 2014 7:34:36 PM jade.imtp.leap.NodeSkel executeCommand
WARNING: Error serving H-Command jade.core.management.AgentManagement/8: java.lang.NullPointerException
Aug 12, 2014 7:34:36 PM jade.core.messaging.MessageManager shutdown
INFO: MessageManager shutting down ...
Aug 12, 2014 7:34:36 PM jade.core.Runtime$1 run
INFO: JADE is closing down now.
I have determined the source of the problem but not a solution, most likely because a solutions the this problem may not exist.
The JADE container will eventually complete in time (after a timeout period) but the Agent will NOT be instantiated (See bold error msg below).
The problem is that I am instantiating the Main container on a local PC behind a firewall on a private address 10.0.0.12 port 1099. The remote container is instantiated on a public Linux Server that actually correctly connects to my local PC via a port forward of 1099 rule in my local router to the IP of 10.0.0.12.
The real problem arises because the Main Container which controls the remote containers and thusly the agents dictates that the agent communication must go through 10.0.0.12 port 1099 and conveys that to the remote container on the public linux server, which promptly tries to instantiate the agent communication through 10.0.0.12 port 1099....and consequently times out.
In the invoking JAVA call the option "-host 24.228.171.118 -port 1099" actually points to my router which does a port forward - it is NOT the IP of the PC initiating the Main container.
It appears that this particular set up does not work well inherently ....
You see the issue here ....
If there is a configuration option I can employ to change this (that anyone know of) I'd be grateful....
.
versaggi#drools:/usr/local/jade$ java -classpath lib/jade.jar:classes jade.Boot -container -local-host 50.116.61.17 -host 24.228.171.118 -port 1099 -agents ping1:examples.PingAgent.PingAgent
.
Aug 13, 2014 1:08:20 PM jade.core.Runtime beginContainer
INFO: ----------------------------------
This is JADE 4.3.2 - revision 6708 of 2014/03/28 15:19:44
downloaded in Open Source, under LGPL restrictions,
at http://jade.tilab.com/
----------------------------------------
Aug 13, 2014 1:08:21 PM jade.imtp.leap.LEAPIMTPManager initialize
INFO: Listening for intra-platform commands on address:
- jicp://50.116.61.17:1099
Aug 13, 2014 1:08:21 PM jade.core.BaseService init
INFO: Service jade.core.management.AgentManagement initialized
Aug 13, 2014 1:08:21 PM jade.core.BaseService init
INFO: Service jade.core.messaging.Messaging initialized
Aug 13, 2014 1:08:21 PM jade.core.BaseService init
INFO: Service jade.core.resource.ResourceManagement initialized
Aug 13, 2014 1:08:21 PM jade.core.BaseService init
INFO: Service jade.core.mobility.AgentMobility initialized
Aug 13, 2014 1:08:21 PM jade.core.BaseService init
INFO: Service jade.core.event.Notification initialized
.
Aug 13, 2014 1:12:36 PM jade.core.AgentContainerImpl
startBootstrapAgents
SEVERE: Cannot create agent ping1: Remote object unreachable - Caused
by: ICPException delivering command to address
jicp://10.0.0.12:1099. - Caused by: Error creating connection. -
Caused by: Connection timed out
.
Aug 13, 2014 1:12:36 PM jade.core.AgentContainerImpl joinPlatform
INFO: --------------------------------------
Agent container Container-3#50.116.61.17 is ready.
--------------------------------------------
.

Resources