Apache Storm Issues with bolts on the same worker - multithreading

I have a question around Apache Storm and how multi-threading works.
We have 2 bolts that are transferring data to each others: Prediction Bolt to FragmentFilter bolt
Both bolts are on the same worker using port 6705 as you can see in the picture here:
worker_description
When I open the log, I notice that the prediction bolt stops working to let the FragmentFilter bolt do his job, they are not running in parralel based on the logs and information below:
2021-10-01 15:26:12.492 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions start process
2021-10-01 15:26:12.492 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions end process
2021-10-01 15:26:12.492 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions start process
2021-10-01 15:26:12.492 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions end process
2021-10-01 15:26:12.492 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions start process
2021-10-01 15:26:12.493 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions end process
2021-10-01 15:26:12.545 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [227aea40-2e7e-421c-91b3-da8259b8bd15, ]
2021-10-01 15:26:12.635 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [7255c225-a8c5-41bc-8b4e-7f640063a0ee, ]
2021-10-01 15:26:12.643 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [66665829-f708-4f2e-8e5d-ee8866d91662, ]
2021-10-01 15:26:12.857 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [e2b8bde1-747a-4546-8e19-30c7623ab0a1, ]
2021-10-01 15:26:12.974 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [17278f27-ae05-4bb0-a20c-9ca6a0ccdab1, ]
2021-10-01 15:26:13.156 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [24715630-93c4-40bd-9086-0671c486e692, ]
2021-10-01 15:26:13.180 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [501ca1e5-c2ed-45b0-ae90-313f4103548c, ]
2021-10-01 15:26:13.611 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [c575e479-53c5-47cd-b3bb-521e75c83028, ]
2021-10-01 15:26:13.844 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [e5d6ee4d-6202-43f9-a9ef-17d9be5b02f7, ]
2021-10-01 15:26:13.916 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [4b04c06c-b92d-439c-b5ed-64bb52b1d7fc, ]
2021-10-01 15:26:14.091 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [28c9a824-fedf-4478-8d2d-fade0289438e, ]
2021-10-01 15:26:14.435 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [675a05f0-7f88-4f65-a26e-6f25723db0ae, ]
2021-10-01 15:26:14.781 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [1547c67f-6e37-4f24-aabc-cedd11a050ff, ]
2021-10-01 15:26:14.816 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [d037c364-35c8-4364-b305-403d48fb0f77, ]
2021-10-01 15:26:14.816 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [369b8eee-f8ac-43bf-af81-f087f6de62f6, ]
2021-10-01 15:26:15.095 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [9a85a64f-0261-41bc-9672-27d039cbab5c, ]
2021-10-01 15:26:15.234 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [4d306b23-06a6-42b0-88e2-28ba9deaaad9, ]
2021-10-01 15:26:15.260 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [9b6aaa36-2223-4f1b-9653-1e60ede94f8b, ]
2021-10-01 15:26:15.285 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [76d3dbb4-1546-45ad-993b-5fa3efad8087, ]
2021-10-01 15:26:15.399 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [d9644f1a-85e0-46e1-bbfd-faa85cd145b7, ]
2021-10-01 15:26:15.479 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [894a12fb-7b06-4c1c-9976-386b0100fd71, ]
2021-10-01 15:26:15.693 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [6bad7162-d811-4712-8049-3ed2c5b69f29, ]
2021-10-01 15:26:15.697 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [8dfb1d95-c869-4b1f-b660-9d4cc7320a30, ]
2021-10-01 15:26:15.711 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [1c102b77-da2b-4af9-bb11-bfc04dedad24, ]
2021-10-01 15:26:15.728 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [f66c2bc0-ba67-45a7-9c41-cd64b7ae70d5, ]
2021-10-01 15:26:15.855 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [ca2679b1-13d0-405b-ab76-9ff89cfc3729, ]
2021-10-01 15:26:15.975 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [75b20990-12b7-4d88-a2e1-23541e5b1b78, ]
2021-10-01 15:26:15.991 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [bf7c2551-3a63-46bd-906e-e403a4a6a26b, ]
2021-10-01 15:26:16.095 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [b360b50a-1a31-4e05-ac23-a798740d8d55, ]
2021-10-01 15:26:16.168 STDIO Thread-11-FragmentFilter-executor[4 4] [INFO] Emitted values: [033862c7-ac16-49ef-bb82-eb1c87739508, ]
2021-10-01 15:26:16.257 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions start process
2021-10-01 15:26:16.257 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions start add_row
2021-10-01 15:26:16.333 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions end add_row
2021-10-01 15:26:16.333 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions end process
2021-10-01 15:26:16.334 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions start process
2021-10-01 15:26:16.334 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions start update
2021-10-01 15:26:16.337 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions end update
2021-10-01 15:26:16.337 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions end process
2021-10-01 15:26:16.337 o.a.s.t.ShellBolt Thread-20 [INFO] ShellLog pid:28655, name:predictions start process
Thread-20 for the prediction bolt seems to be stopping at 2021-10-01 15:26:12.493 to let the thread-11 (FragmentFilter Bolt) process information then Bolt prediction restarts working at 2021-10-01 15:26:16.257 loosing 4 seconds in the process. This is creating a huge latency problem for us.
Is that normal ? how can we solve it ?

Related

Time_Series_mean of every 10min data

I have the data like follows:
df.index value
2019-02-28 00:00:00 101
2019-02-28 00:10:00 97
2019-02-28 00:20:00 97
2019-02-28 00:30:00 96
2019-02-28 00:40:00 110
2019-02-28 00:50:00 117
2019-02-28 01:00:00 121
2019-02-28 01:10:00 114
2019-02-28 01:20:00 112
2019-02-28 01:30:00 103
2019-02-28 01:40:00 104
2019-02-28 01:50:00 105
2019-02-28 02:00:00 106
2019-02-28 02:10:00 104
2019-02-28 02:20:00 103
2019-02-28 02:30:00 97
2019-02-28 02:40:00 101
2019-02-28 02:50:00 103
2019-02-28 03:00:00 102
2019-02-28 03:10:00 101
Is there to method helping me to make a new dataframe resampled by 15min, where for full hours and 30min takes value from above data and for 15min (the average of 10 and 20) and similarly for 45 (the average of 40 and 50)? Additionally how to check if the dataframe starts with full hour?
The part of the code I tried to use is
df_15=pd.date_range(start=df.index[0], end=df.index[-1], freq='15T')
df_15=df_15.to_frame(index=False)
> for row in range(0,len(df_15+1),6):
> mean=df.iloc[row]
> df_mean=pd.concat([df_mean,mean])
> mean=(df.iloc[row+1]+df.iloc[row+2])/2
> df_mean=pd.concat([df_mean,mean])
> mean=df.iloc[row+3]
> df_mean=pd.concat([df_mean,mean])
> mean=(df.iloc[row+4]+df.iloc[row+5])/2
> df_mean=pd.concat([df_mean,mean])
but I get an error
TypeError: Addition/subtraction of integers and integer-arrays with DatetimeArray is no longer supported. Instead of adding/subtracting n, use n * obj.freq

React failed create react app module not found [duplicate]

This question already has answers here:
Cannot find module coa/compile.js
(6 answers)
Closed 1 year ago.
I bought a new pc today.
I installed the latest reccomanded version of NodeJS, installed npm.
I checked if they are installed correctly using the node -v and npm -v and i can see
Node:
v16.13.0
npm:
8.1.0
Then tried to build a new react app with the command npx create-react-app test-app. I tried this, then tried to install it globally but i had the same feedback from the error.
So the problem is:
When i lunch the command create-react-app, it crashes giving me this error back. Anyone can explain me what is happening?
The error starts at row 111
Thanks everyone!
0 verbose cli [
0 verbose cli 'C:\\Program Files\\nodejs\\node.exe',
0 verbose cli 'C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js',
0 verbose cli 'install',
0 verbose cli '-g',
0 verbose cli 'coa'
0 verbose cli ]
1 info using npm#8.1.0
2 info using node#v16.13.0
3 timing npm:load:whichnode Completed in 1ms
4 timing config:load:defaults Completed in 4ms
5 timing config:load:file:C:\Program Files\nodejs\node_modules\npm\npmrc Completed in 4ms
6 timing config:load:builtin Completed in 5ms
7 timing config:load:cli Completed in 3ms
8 timing config:load:env Completed in 1ms
9 timing config:load:file:C:\Users\Luca\Desktop\Progetti\.npmrc Completed in 0ms
10 timing config:load:project Completed in 1ms
11 timing config:load:file:C:\Users\Amministratore\.npmrc Completed in 1ms
12 timing config:load:user Completed in 1ms
13 timing config:load:file:C:\Users\Amministratore\AppData\Roaming\npm\etc\npmrc Completed in 0ms
14 timing config:load:global Completed in 0ms
15 timing config:load:validate Completed in 1ms
16 timing config:load:credentials Completed in 1ms
17 timing config:load:setEnvs Completed in 1ms
18 timing config:load Completed in 18ms
19 timing npm:load:configload Completed in 18ms
20 timing npm:load:setTitle Completed in 1ms
21 timing npm:load:setupLog Completed in 0ms
22 timing config:load:flatten Completed in 5ms
23 timing npm:load:cleanupLog Completed in 3ms
24 timing npm:load:configScope Completed in 0ms
25 timing npm:load:projectScope Completed in 0ms
26 timing npm:load Completed in 30ms
27 timing arborist:ctor Completed in 2ms
28 timing idealTree:init Completed in 9ms
29 timing idealTree:userRequests Completed in 4ms
30 silly idealTree buildDeps
31 silly fetch manifest coa#*
32 http fetch GET 200 https://registry.npmjs.org/coa 76ms (cache hit)
33 silly placeDep ROOT coa#3.1.3 OK for: want: *
34 silly fetch manifest #types/q#^1.5.1
35 silly fetch manifest chalk#^2.4.1
36 silly fetch manifest q#^1.1.2
37 http fetch GET 200 https://registry.npmjs.org/#types%2fq 24ms (cache hit)
38 http fetch GET 200 https://registry.npmjs.org/chalk 24ms (cache hit)
39 http fetch GET 200 https://registry.npmjs.org/q 28ms (cache hit)
40 timing idealTree:#root Completed in 130ms
41 silly placeDep node_modules/coa #types/q#1.5.5 OK for: coa#3.1.3 want: ^1.5.1
42 silly placeDep node_modules/coa chalk#2.4.2 OK for: coa#3.1.3 want: ^2.4.1
43 silly placeDep node_modules/coa q#1.5.1 OK for: coa#3.1.3 want: ^1.1.2
44 silly fetch manifest ansi-styles#^3.2.1
45 silly fetch manifest escape-string-regexp#^1.0.5
46 silly fetch manifest supports-color#^5.3.0
47 http fetch GET 200 https://registry.npmjs.org/ansi-styles 24ms (cache hit)
48 http fetch GET 200 https://registry.npmjs.org/supports-color 24ms (cache hit)
49 http fetch GET 200 https://registry.npmjs.org/escape-string-regexp 28ms (cache hit)
50 timing idealTree:node_modules/coa Completed in 44ms
51 timing idealTree:node_modules/coa/node_modules/#types/q Completed in 0ms
52 silly placeDep node_modules/coa ansi-styles#3.2.1 OK for: chalk#2.4.2 want: ^3.2.1
53 silly placeDep node_modules/coa escape-string-regexp#1.0.5 OK for: chalk#2.4.2 want: ^1.0.5
54 silly placeDep node_modules/coa supports-color#5.5.0 OK for: chalk#2.4.2 want: ^5.3.0
55 silly fetch manifest color-convert#^1.9.0
56 silly fetch manifest has-flag#^3.0.0
57 http fetch GET 200 https://registry.npmjs.org/color-convert 21ms (cache hit)
58 http fetch GET 200 https://registry.npmjs.org/has-flag 23ms (cache hit)
59 timing idealTree:node_modules/coa/node_modules/chalk Completed in 44ms
60 silly placeDep node_modules/coa color-convert#1.9.3 OK for: ansi-styles#3.2.1 want: ^1.9.0
61 silly fetch manifest color-name#1.1.3
62 http fetch GET 200 https://registry.npmjs.org/color-name 12ms (cache hit)
63 timing idealTree:node_modules/coa/node_modules/ansi-styles Completed in 18ms
64 silly placeDep node_modules/coa color-name#1.1.3 OK for: color-convert#1.9.3 want: 1.1.3
65 timing idealTree:node_modules/coa/node_modules/color-convert Completed in 3ms
66 timing idealTree:node_modules/coa/node_modules/color-name Completed in 1ms
67 timing idealTree:node_modules/coa/node_modules/escape-string-regexp Completed in 0ms
68 timing idealTree:node_modules/coa/node_modules/q Completed in 0ms
69 silly placeDep node_modules/coa has-flag#3.0.0 OK for: supports-color#5.5.0 want: ^3.0.0
70 timing idealTree:node_modules/coa/node_modules/supports-color Completed in 2ms
71 timing idealTree:node_modules/coa/node_modules/has-flag Completed in 0ms
72 timing idealTree:buildDeps Completed in 245ms
73 timing idealTree:fixDepFlags Completed in 2ms
74 timing idealTree Completed in 262ms
75 timing reify:loadTrees Completed in 269ms
76 timing reify:diffTrees Completed in 3ms
77 silly reify moves {}
78 timing reify:retireShallow Completed in 1ms
79 timing reify:createSparse Completed in 9ms
80 timing reify:loadBundles Completed in 0ms
81 silly audit bulk request {
81 silly audit coa: [ '3.1.3' ],
81 silly audit '#types/q': [ '1.5.5' ],
81 silly audit chalk: [ '2.4.2' ],
81 silly audit q: [ '1.5.1' ],
81 silly audit 'ansi-styles': [ '3.2.1' ],
81 silly audit 'escape-string-regexp': [ '1.0.5' ],
81 silly audit 'supports-color': [ '5.5.0' ],
81 silly audit 'color-convert': [ '1.9.3' ],
81 silly audit 'color-name': [ '1.1.3' ],
81 silly audit 'has-flag': [ '3.0.0' ]
81 silly audit }
82 silly tarball no local data for coa#https://registry.npmjs.org/coa/-/coa-3.1.3.tgz. Extracting by manifest.
83 timing reifyNode:node_modules/coa/node_modules/escape-string-regexp Completed in 188ms
84 timing reifyNode:node_modules/coa/node_modules/ansi-styles Completed in 189ms
85 timing reifyNode:node_modules/coa/node_modules/has-flag Completed in 191ms
86 timing reifyNode:node_modules/coa/node_modules/supports-color Completed in 192ms
87 timing reifyNode:node_modules/coa/node_modules/#types/q Completed in 194ms
88 timing reifyNode:node_modules/coa/node_modules/color-name Completed in 198ms
89 timing reifyNode:node_modules/coa/node_modules/q Completed in 199ms
90 timing reifyNode:node_modules/coa/node_modules/chalk Completed in 199ms
91 timing reifyNode:node_modules/coa/node_modules/color-convert Completed in 200ms
92 http fetch GET 200 https://registry.npmjs.org/coa/-/coa-3.1.3.tgz 223ms (cache miss)
93 timing reifyNode:node_modules/coa Completed in 287ms
94 timing reify:unpack Completed in 287ms
95 timing reify:unretire Completed in 0ms
96 timing build:queue Completed in 6ms
97 info run coa#3.1.3 preinstall node_modules/coa start /B node compile.js & node compile.js
98 info run coa#3.1.3 preinstall { code: 1, signal: null }
99 timing reify:rollback:createSparse Completed in 27ms
100 timing reify:rollback:retireShallow Completed in 0ms
101 timing command:install Completed in 762ms
102 verbose stack Error: command failed
102 verbose stack at ChildProcess.<anonymous> (C:\Program Files\nodejs\node_modules\npm\node_modules\#npmcli\promise-spawn\index.js:64:27)
102 verbose stack at ChildProcess.emit (node:events:390:28)
102 verbose stack at maybeClose (node:internal/child_process:1064:16)
102 verbose stack at Process.ChildProcess._handle.onexit (node:internal/child_process:301:5)
103 verbose pkgid coa#3.1.3
104 verbose cwd C:\Users\Luca\Desktop\Progetti
105 verbose Windows_NT 10.0.19042
106 verbose argv "C:\\Program Files\\nodejs\\node.exe" "C:\\Program Files\\nodejs\\node_modules\\npm\\bin\\npm-cli.js" "install" "-g" "coa"
107 verbose node v16.13.0
108 verbose npm v8.1.0
109 error code 1
110 error path C:\Users\Amministratore\AppData\Roaming\npm\node_modules\coa
111 error command failed
112 error command C:\Windows\system32\cmd.exe /d /s /c start /B node compile.js & node compile.js
113 error node:internal/modules/cjs/loader:936
113 error throw err;
113 error ^
113 error
113 error Error: Cannot find module 'C:\Users\Amministratore\AppData\Roaming\npm\node_modules\coa\compile.js'
113 error at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
113 error at Function.Module._load (node:internal/modules/cjs/loader:778:27)
113 error at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
113 error at node:internal/main/run_main_module:17:47 {
113 error code: 'MODULE_NOT_FOUND',
113 error requireStack: []
113 error }
113 error node:internal/modules/cjs/loader:936
113 error throw err;
113 error ^
113 error
113 error Error: Cannot find module 'C:\Users\Amministratore\AppData\Roaming\npm\node_modules\coa\compile.js'
113 error at Function.Module._resolveFilename (node:internal/modules/cjs/loader:933:15)
113 error at Function.Module._load (node:internal/modules/cjs/loader:778:27)
113 error at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
113 error at node:internal/main/run_main_module:17:47 {
113 error code: 'MODULE_NOT_FOUND',
113 error requireStack: []
113 error }
114 verbose exit 1
It's working now!
COA got compromised but is now back up again without the malicious code inside.
Check https://github.com/veged/coa/issues/99 for more info.

Why Cassandra TableWriter writing 0 records and how to fix it?

I am trying to write an RDD into a Cassandra table.
As shown below TableWriter wrote 0 rows several times and finally writes to Cassandra.
18/10/22 07:15:50 INFO TableWriter: Wrote 0 rows to log_by_date in 0.171 s.
18/10/22 07:15:50 INFO Executor: Finished task 4.0 in stage 0.0 (TID 4). 622 bytes result sent to driver
18/10/22 07:15:50 INFO TableWriter: Wrote 0 rows to log_by_date in 0.220 s.
18/10/22 07:15:50 INFO Executor: Finished task 1.0 in stage 0.0 (TID 1). 665 bytes result sent to driver
18/10/22 07:15:50 INFO TableWriter: Wrote 0 rows to log_by_date in 0.194 s.
18/10/22 07:15:50 INFO TableWriter: Wrote 0 rows to log_by_date in 0.224 s.
18/10/22 07:15:50 INFO Executor: Finished task 6.0 in stage 0.0 (TID 6). 708 bytes result sent to driver
18/10/22 07:15:50 INFO TableWriter: Wrote 0 rows to log_by_date in 0.231 s.
18/10/22 07:15:50 INFO Executor: Finished task 5.0 in stage 0.0 (TID 5). 622 bytes result sent to driver
18/10/22 07:15:50 INFO Executor: Finished task 3.0 in stage 0.0 (TID 3). 622 bytes result sent to driver
18/10/22 07:15:50 INFO TableWriter: Wrote 0 rows to log_by_date in 0.246 s.
18/10/22 07:15:50 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 708 bytes result sent to driver
18/10/22 07:15:50 INFO TaskSetManager: Finished task 4.0 in stage 0.0 (TID 4) in 418 ms on localhost (executor driver) (1/8)
18/10/22 07:15:50 INFO TaskSetManager: Finished task 2.0 in stage 0.0 (TID 2) in 433 ms on localhost (executor driver) (2/8)
18/10/22 07:15:50 INFO TaskSetManager: Finished task 5.0 in stage 0.0 (TID 5) in 426 ms on localhost (executor driver) (3/8)
18/10/22 07:15:50 INFO TaskSetManager: Finished task 3.0 in stage 0.0 (TID 3) in 433 ms on localhost (executor driver) (4/8)
18/10/22 07:15:50 INFO TaskSetManager: Finished task 0.0 in stage 0.0 (TID 0) in 456 ms on localhost (executor driver) (5/8)
18/10/22 07:15:50 INFO TaskSetManager: Finished task 1.0 in stage 0.0 (TID 1) in 436 ms on localhost (executor driver) (6/8)
18/10/22 07:15:50 INFO TaskSetManager: Finished task 6.0 in stage 0.0 (TID 6) in 424 ms on localhost (executor driver) (7/8)
18/10/22 07:15:50 INFO **TableWriter: Wrote 1 rows to log_by_date in 0.342 s.**
Why it is failing to save it sevaral times prior, how to tune it for production?
This is not a failure as noted by user10465355. When Spark breaks a job into Tasks it is possible that the work is not evenly distributed or that there isn't enough work for every task to have work to do. This results in some tasks being empty, so when they are processed by the Spark Cassandra Connector they write 0 rows.
For example say;
You read 100 records into 10 Spark Partitions/Tasks
You do a filter which eliminate values with a filter so now only 30 records remain in 5 tasks. The other 5 are empty.
When you write you will now only see records written for 5 tasks, and 5 tasks will report they had no rows written.

jmeter runs for scheduler duration but hangs at end

I'm running jmeter from the command line with a 300 second duration.
However it rarely finishes the whole job and returns to the command line - I mostly have to cancel it.
This is what I see:
C:\dev\tools\apache-jmeter-3.1\bin>jmeter.bat -n -t c:/dev/workspace/docs/JMeter-stress2.jmx -j c:/dev/log/jmeter.log -l c:/dev/log/jmeter-results.csv
Writing log file to: c:\dev\log\jmeter.log
Creating summariser <summary>
Created the tree successfully using c:/dev/workspace/docs/JMeter-stress2.jmx
Starting the test # Tue Mar 07 15:43:07 GMT 2017 (1488901387136)
Waiting for possible Shutdown/StopTestNow/Heapdump message on port 4445
summary + 1573 in 00:00:23 = 69.0/s Avg: 166 Min: 47 Max: 2175 Err: 0 (0.00%) Active: 12 Started: 12 Finished: 0
summary + 2135 in 00:00:30 = 71.3/s Avg: 150 Min: 44 Max: 4022 Err: 0 (0.00%) Active: 12 Started: 12 Finished: 0
summary = 3708 in 00:00:53 = 70.3/s Avg: 157 Min: 44 Max: 4022 Err: 0 (0.00%)
summary + 2039 in 00:00:30 = 68.0/s Avg: 187 Min: 44 Max: 31024 Err: 0 (0.00%) Active: 12 Started: 12 Finished: 0
summary = 5747 in 00:01:23 = 69.4/s Avg: 168 Min: 44 Max: 31024 Err: 0 (0.00%)
summary + 2051 in 00:00:30 = 68.3/s Avg: 168 Min: 41 Max: 30813 Err: 0 (0.00%) Active: 12 Started: 12 Finished: 0
summary = 7798 in 00:01:53 = 69.2/s Avg: 168 Min: 41 Max: 31024 Err: 0 (0.00%)
summary + 2296 in 00:00:30 = 76.5/s Avg: 168 Min: 41 Max: 32443 Err: 0 (0.00%) Active: 12 Started: 12 Finished: 0
summary = 10094 in 00:02:23 = 70.7/s Avg: 168 Min: 41 Max: 32443 Err: 0 (0.00%)
summary + 1015 in 00:00:30 = 33.8/s Avg: 348 Min: 42 Max: 30255 Err: 5 (0.49%) Active: 12 Started: 12 Finished: 0
summary = 11109 in 00:02:53 = 64.3/s Avg: 184 Min: 41 Max: 32443 Err: 5 (0.05%)
summary + 1880 in 00:00:30 = 62.6/s Avg: 177 Min: 41 Max: 30265 Err: 0 (0.00%) Active: 12 Started: 12 Finished: 0
summary = 12989 in 00:03:23 = 64.1/s Avg: 183 Min: 41 Max: 32443 Err: 5 (0.04%)
summary + 1499 in 00:00:30 = 50.0/s Avg: 262 Min: 41 Max: 30417 Err: 5 (0.33%) Active: 12 Started: 12 Finished: 0
summary = 14488 in 00:03:53 = 62.2/s Avg: 191 Min: 41 Max: 32443 Err: 10 (0.07%)
summary + 2383 in 00:00:30 = 79.4/s Avg: 148 Min: 42 Max: 3687 Err: 0 (0.00%) Active: 12 Started: 12 Finished: 0
summary = 16871 in 00:04:23 = 64.2/s Avg: 185 Min: 41 Max: 32443 Err: 10 (0.06%)
summary + 1870 in 00:00:30 = 62.3/s Avg: 172 Min: 41 Max: 30890 Err: 0 (0.00%) Active: 12 Started: 12 Finished: 0
summary = 18741 in 00:04:53 = 64.0/s Avg: 184 Min: 41 Max: 32443 Err: 10 (0.05%)
summary + 483 in 00:00:35 = 14.0/s Avg: 344 Min: 43 Max: 31082 Err: 3 (0.62%) Active: 1 Started: 12 Finished: 11
summary = 19224 in 00:05:27 = 58.7/s Avg: 188 Min: 41 Max: 32443 Err: 13 (0.07%)
Terminate batch job (Y/N)? y
The last line of output before I cancel it hangs there indefinitely until I kill it.
The errors are from kerberos, which doesn't have a good reputation in this organisation :( It puts the error logging into the *.csv output file which makes it unusable, but I guess that's a different question. I only mention it because it might be the cause of the hanging.
This is what I see in the end of the log file. Notice the timestamp of the shutdown message - the log statement before that is the last before it hangs. The errors in the logging stem from connection problems with the kerberos server.
2017/03/07 15:48:00 INFO - jmeter.reporters.Summariser: summary + 1870 in 00:00:30 = 62.3/s Avg: 172 Min: 41 Max: 30890 Err: 0 (0.00%) Active: 12 Started: 12 Finished: 0
2017/03/07 15:48:00 INFO - jmeter.reporters.Summariser: summary = 18741 in 00:04:53 = 64.0/s Avg: 184 Min: 41 Max: 32443 Err: 10 (0.05%)
2017/03/07 15:48:04 ERROR - jmeter.protocol.http.sampler.HTTPHC4Impl: Can't execute httpRequest with subject:Subject:
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Stopping because end time detected by thread: GET get_forecast 5-2
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Thread finished: GET get_forecast 5-2
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Stopping because end time detected by thread: GET get_forecast 5-1
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Thread finished: GET get_forecast 5-1
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Stopping because end time detected by thread: GET forecast with history 4-1
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Thread finished: GET forecast with history 4-1
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Stopping because end time detected by thread: POST data/save 2-2
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Thread finished: POST data/save 2-2
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Stopping because end time detected by thread: POST forecast/save 3-1
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Thread finished: POST forecast/save 3-1
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Stopping because end time detected by thread: GET forecast with history 4-3
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Thread finished: GET forecast with history 4-3
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Stopping because end time detected by thread: POST data/save 2-1
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Thread finished: POST data/save 2-1
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Stopping because end time detected by thread: GET forecast with history 4-2
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Thread finished: GET forecast with history 4-2
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Stopping because end time detected by thread: GET get_forecast 5-3
2017/03/07 15:48:07 INFO - jmeter.threads.JMeterThread: Thread finished: GET get_forecast 5-3
2017/03/07 15:48:08 INFO - jmeter.threads.JMeterThread: Stopping because end time detected by thread: POST data/save 2-3
2017/03/07 15:48:08 INFO - jmeter.threads.JMeterThread: Thread finished: POST data/save 2-3
2017/03/07 15:48:13 ERROR - jmeter.protocol.http.sampler.HTTPHC4Impl: Can't execute httpRequest with subject:Subject:
2017/03/07 15:48:13 INFO - jmeter.threads.JMeterThread: Stopping because end time detected by thread: POST forecast/save 3-3
2017/03/07 15:48:13 INFO - jmeter.threads.JMeterThread: Thread finished: POST forecast/save 3-3
2017/03/07 15:48:34 INFO - jmeter.reporters.Summariser: summary + 483 in 00:00:35 = 14.0/s Avg: 344 Min: 43 Max: 31082 Err: 3 (0.62%) Active: 1 Started: 12 Finished: 11
2017/03/07 15:48:34 INFO - jmeter.reporters.Summariser: summary = 19224 in 00:05:27 = 58.7/s Avg: 188 Min: 41 Max: 32443 Err: 13 (0.07%)
2017/03/07 15:48:34 INFO - jmeter.threads.JMeterThread: Stopping because end time detected by thread: POST forecast/save 3-2
2017/03/07 15:48:34 INFO - jmeter.threads.JMeterThread: Thread finished: POST forecast/save 3-2
2017/03/07 15:51:21 INFO - jmeter.reporters.ResultCollector: Shutdown hook started
2017/03/07 15:51:21 INFO - jmeter.reporters.ResultCollector: Shutdown hook ended
Update 2017-03-10
Only progress in defining the problem better :(
Why isn't JMeter dumping the connections when I set the connection time-out to 5 secs (connect) + 5 secs (response) on the HTTP request defaults dialog.
Why do I see a max time on the JMeter output of > 30000ms despite the JMeter connection time-out?
Why do I see no stack traces on the server-side? Possibly exceptions getting swallowed maybe.
Have you tried to lower the simulated load? I see that max response time is around 30 sec., JMeter may fail to stop after the desired because some threads may be blocked waiting for several server responses.
You should also use jvisualvm to monitor JMeter's VM while running the load test to ensure there is enough memory available, as well as look at waiting threads. It may help you to find out the issue.

Get timestamps by line with iperf3 in bash script

I'm currently getting this output from iperf3
2016-03-03 21:33:50 [ 4] 0.00-1.00 sec 113 MBytes 950 Mbits/sec
2016-03-03 21:33:50 [ 4] 1.00-2.00 sec 112 MBytes 941 Mbits/sec 0
2016-03-03 21:33:50 [ 4] 2.00-3.00 sec 113 MBytes 944 Mbits/sec 0
I want to create Graphics from this data, and as iperf3 can't update timestamps by line (as far as I know..) I'm looking for a way to increment the output file line by line.
result should be like:
2016-03-03 21:33:50 [ 4] 0.00-1.00 sec 113 MBytes 950 Mbits/sec
2016-03-03 21:33:51 [ 4] 1.00-2.00 sec 112 MBytes 941 Mbits/sec 0
2016-03-03 21:33:52 [ 4] 2.00-3.00 sec 113 MBytes 944 Mbits/sec 0
so an action (+1) has to be done on each line containing Mbits/sec until the end of the file.
I guess that sed and/or date command may be helpful and a loop may be useful but can't see how to build it with time values..
awk '$10=="Mbits/sec"\
{command="date -d "$2" +%s";command |getline $2;close(command)};1' 1txt \
| awk -vi=1 '$10=="Mbits/sec"{$2=$2+i};i=i+1'\
| awk '$10=="Mbits/sec"{command="date -d #"$2" +%T";command|getline $2;close(command)};1'
tested it on a file 1txt having values:
2016-03-03 21:33:50 [ 4] 0.00-1.00 sec 113 MBytes 950 Mbits/sec
2016-03-03 21:33:50 [ 4] 1.00-2.00 sec 112 MBytes 941 Mbits/sec 0
2016-03-03 21:33:50 [ 4] 2.00-3.00 sec 113 MBytes 944 Mbits/sec 0
2016-03-03 21:33:50 [ 4] 2.00-3.00 sec 113 MBytes 944 bits/sec 0
the output as expected after execution was:
2016-03-03 21:33:51 [ 4] 0.00-1.00 sec 113 MBytes 950 Mbits/sec
2016-03-03 21:33:52 [ 4] 1.00-2.00 sec 112 MBytes 941 Mbits/sec 0
2016-03-03 21:33:53 [ 4] 2.00-3.00 sec 113 MBytes 944 Mbits/sec 0
2016-03-03 21:33:50 [ 4] 2.00-3.00 sec 113 MBytes 944 bits/sec 0
P.S: you can ofcourse make it more compact and efficient by combining the awk's in a single command. But this helps in better understanding of whats going on.
You can do this using sed, but this is not trivial... It is much easier to do it using perl:
perl -lne 'print $1.($2 + ($.) - 1).$3 if /(.+)(50)(.+)/' file.txt
-l enable line ending processing, specifies line terminator
-n assume loop around program
-e one line of program
print print command
. string concatenation
$number variables contain the parts of the string that matched the capture groups ()
$. the current record number
($2 + ($.) - 1) means: 50 + 'current record number' - 1
if /(.+)(50)(.+)/' statement with regular expression referred to by print
file.txt file with your datas

Resources