OpenProject Docker Error: Could not spawn process for application /app - openproject

Upgraded the community edition of OpenProject from a working 9.0.3 version to "latest" (10.0.2?) in Docker and now getting the following error when trying to navigate to OpenProject (seeing the "We're sorry, but something went wrong." page)
Log is below, but looks like error may be:
[ E 2019-10-10 01:02:17.7066 370/Tc age/Cor/App/Implementation.cpp:221 ]: Could not spawn process for application /app: A timeout occurred while starting a preloader process.
Not sure where to start on troubleshooting this...
[ N 2019-10-10 01:00:38.4197 370/T5 age/Cor/SecurityUpdateChecker.h:519 ]: Security update check: no update found (next check in 24 hours)
Forcefully loading the application. Use :environment to avoid eager loading.
hook registered
App 402 output: hook registered
Creating scope :order_by_name. Overwriting existing method Sprint.order_by_name.
App 402 output: Creating scope :order_by_name. Overwriting existing method Sprint.order_by_name.
2019-10-10 01:02:16.042 UTC [440] openproject#openproject LOG: could not receive data from client: Connection reset by peer
[ E 2019-10-10 01:02:17.7066 370/Tc age/Cor/App/Implementation.cpp:221 ]: Could not spawn process for application /app: A timeout occurred while starting a preloader process.
Error ID: 89e9b405
Error details saved to: /tmp/passenger-error-1pJJ4k.html
[ E 2019-10-10 01:02:17.7223 370/T7 age/Cor/Con/CheckoutSession.cpp:276 ]: [Client 1-1] Cannot checkout session because a spawning error occurred. The identifier of the error is 89e9b405. Please see earlier logs for details about the error.
[ E 2019-10-10 01:02:17.8692 370/T7 age/Cor/Con/CheckoutSession.cpp:276 ]: [Client 1-2] Cannot checkout session because a spawning error occurred. The identifier of the error is 89e9b405. Please see earlier logs for details about the error.
[Worker(host:8416036a7682 pid:135)] Starting job worker
2019-10-10T01:03:17+0000: [Worker(host:8416036a7682 pid:135)] Starting job worker
App 459 output: hook registered
App 459 output: Creating scope :order_by_name. Overwriting existing method Sprint.order_by_name.
[ E 2019-10-10 01:05:00.6208 370/Th age/Cor/App/Implementation.cpp:221 ]: Could not spawn process for application /app: A timeout occurred while starting a preloader process.
Error ID: c22d62f4
Error details saved to: /tmp/passenger-error-8jCQPq.html
[ E 2019-10-10 01:05:00.6559 370/T7 age/Cor/Con/CheckoutSession.cpp:276 ]: [Client 1-3] Cannot checkout session because a spawning error occurred. The identifier of the error is c22d62f4. Please see earlier logs for details about the error.

Probably the problem can be solved by increasing the PassengerStartTimeout from 90 (Default) to 200.
I done it by adding the line
PassengerStartTimeout 200
to the /opt/openproject-10.3.1-1/apache2/conf/bitnami/passenger.conf
and restarted apache server
Hope this helps

In openproject Docker image, add PASSENGER_START_TIMEOUT=200 to environment variable.
docker run -d -e PASSENGER_START_TIMEOUT=200 ...add any options... openproject/community:10
This works for me.

Related

`initialize': Insecure operation - initialize (SecurityError) - passenger in rails 1.8.7

I am deploying a ruby ​​on rails application in version 1.8.7 - nginx and it generates the following error.
some help
2022-06-29 10:28:46.0419 36057/T7 age/Cor/Con/CheckoutSession.cpp:281 ]: [Client 1-1] Cannot checkout session because a spawning error occurred. The identifier of the error is cd89e238. Please see earlier logs for details about the error.
2022/06/29 10:40:34 [info] 36618#36618: Using 32768KiB of shared memory for nchan in /etc/nginx/nginx.conf:69
[ N 2022-06-29 10:40:34.7154 36633/T1 age/Wat/WatchdogMain.cpp:1373 ]: Starting Passenger watchdog...
[ N 2022-06-29 10:40:34.7412 36647/T1 age/Cor/CoreMain.cpp:1340 ]: Starting Passenger core...
[ N 2022-06-29 10:40:34.7413 36647/T1 age/Cor/CoreMain.cpp:256 ]: Passenger core running in multi-application mode.
[ N 2022-06-29 10:40:34.7516 36647/T1 age/Cor/CoreMain.cpp:1015 ]: Passenger core online, PID 36647
[ N 2022-06-29 10:40:37.4505 36647/T5 age/Cor/SecurityUpdateChecker.h:519 ]: Security update check: no update found (next check in 24 hours)
App 36678 output: /usr/share/passenger/helper-scripts/rack-preloader.rb:64:in `initialize': Insecure operation - initialize (SecurityError)
App 36678 output: from /usr/share/passenger/helper-scripts/rack-preloader.rb:64:in `open'
App 36678 output: from /usr/share/passenger/helper-scripts/rack-preloader.rb:64:in `try_write_file'
App 36678 output: from /usr/share/passenger/helper-scripts/rack-preloader.rb:82:in `record_journey_step_end'
App 36678 output: from /usr/share/passenger/helper-scripts/rack-preloader.rb:48:in `init_passenger'
App 36678 output: from /usr/share/passenger/helper-scripts/rack-preloader.rb:183
[ E 2022-06-29 10:41:17.3505 36647/Tj age/Cor/App/Implementation.cpp:221 ]: Could not spawn process for application /home/Vivienda: The application process exited prematurely.
Error ID: 03b3e0b8
Error details saved to: /tmp/passenger-error-jTSu7e.html
[ E 2022-06-29 10:41:17.3543 36647/T8 age/Cor/Con/CheckoutSession.cpp:281 ]: [Client 1-1] Cannot checkout session because a spawning error occurred. The identifier of the error is 03b3e0b8. Please see earlier logs for details about the error.

How to get a basic direvent watcher working?

I have read through the direvent documentation and am trying to get a simple watch working. Since I am having so much trouble with it, I am wondering if the issue has to do with the fact that the system I am using is nixos.
Here is the simple watcher file, watcher, I've created:
watcher {
path ./dir;
command "echo $file";
}
I run it in the foreground, so I can see the output, with direvent --foreground watcher. Once it's running, I create a file in dir, thus creating an event for it to respond to. However, it fails with the following output:
$ direvent --foreground watcher
direvent: [INFO] direvent 5.2 started
direvent: [ERROR] process 8552 failed with status 127
direvent: [ERROR] process 8555 failed with status 127
direvent: [ERROR] process 8557 failed with status 127
Since 127 usually means 'command not found', I tried specifying the path to echo, i.e. running this watcher instead:
watcher {
path ./dir;
command "/run/current-system/sw/bin/echo $file";
}
Then the output still gives an error, albeit a different one:
$ direvent --foreground watcher
direvent: [INFO] direvent 5.2 started
direvent: [ERROR] process 8645 failed with status 1
direvent: [ERROR] process 8651 failed with status 1
direvent: [ERROR] process 8652 failed with status 1
So the failure is now with status 1. I am not sure what to try next. I'm wondering if this issue is due to the fact that I am running nixos. Anyone know what I might try next to get direvent working?
direvent has two other flag that may be useful for you.
--debug(-d) to give extra information.
There's also --lint(t) that check the configuration file for errors, but I suspect this isn't your issue if direvent is running.
Source: https://www.gnu.org.ua/software/direvent/manual/direvent.html

Anchore Engine - Jenkins CI plugin

We are trying to scan our docker images using Anchore Engine Jenkins plugin.
Currently we create our application docker images, push it in our own private local registry and then deploy it in our test environments.
Now, we want to setup docker image scanning in our CI/CD process to check for any vulnerabilities.
We have installed Anchore Engine using the recommended Docker-Compose yaml method given in the Documentation link:
https://anchore.freshdesk.com/support/solutions/articles/36000020729-install-on-docker-swarm
Post installation, we installed the
Anchore Container Image Scanner Plugin in Jenkins.
We configured the plugin as mentioned in the document link:
https://wiki.jenkins.io/display/JENKINS/Anchore+Container+Image+Scanner+Plugin
However, the scanning fails. Error Message as follows:
2018-10-11T07:01:44.647 INFO AnchoreWorker Analysis request accepted, received image digest sha256:7d6fb7e5e7a74a4309cc436f6d11c29a96cbf27a4a8cb45a50cb0a326dc32fe8
2018-10-11T07:01:44.647 INFO AnchoreWorker Waiting for analysis of 10.180.25.2:5000/hello-world:latest, polling status periodically
2018-10-11T07:01:44.647 DEBUG AnchoreWorker anchore-engine get policy evaluation URL: http://10.180.25.2:8228/v1/images/sha256:7d6fb7e5e7a74a4309cc436f6d11c29a96cbf27a4a8cb45a50cb0a326dc32fe8/check?tag=10.180.25.2:5000/hello-world:latest&detail=true
2018-10-11T07:01:44.648 DEBUG AnchoreWorker Attempting anchore-engine get policy evaluation (1/300)
2018-10-11T07:01:44.675 DEBUG AnchoreWorker anchore-engine get policy evaluation failed. URL: http://10.180.25.2:8228/v1/images/sha256:7d6fb7e5e7a74a4309cc436f6d11c29a96cbf27a4a8cb45a50cb0a326dc32fe8/check?tag=10.180.25.2:5000/hello-world:latest&detail=true, status: HTTP/1.1 404 NOT FOUND, error: {
"detail": {},
"httpcode": 404,
"message": "image is not analyzed - analysis_status: not_analyzed"
}
NOTE:
In Image TAG 10.180.25.2:5000/hello-world:latest, 10.180.25.2:5000 is our local private registry and hello-world:latest is latest hello-world image available in docker hub which we pulled and pushed in our registry to try out image scanning using Anchore-Engine.
Unfortunately we are not able to find much resource online to try and resolve the above mentioned issue.
Anyone who might have worked on Anchore-Engine, please may I request to have a look and help us resolve this issue.
Also, any suggestions or alternatives to anchore-engine or detailed steps in case we might have missed anything would be really appreciated.
End of the output is as follows:
2018-10-15T00:48:43.880 WARN AnchoreWorker anchore-engine get policy evaluation failed. HTTP method: GET, URL: http://10.180.25.2:8228/v1/images/sha256:7d6fb7e5e7a74a4309cc436f6d11c29a96cbf27a4a8cb45a50cb0a326dc32fe8/check?tag=10.180.25.2:5000/hello-world:latest&detail=true, status: 404, error: {
"detail": {},
"httpcode": 404,
"message": "image is not analyzed - analysis_status: not_analyzed"
}
2018-10-15T00:48:43.880 WARN AnchoreWorker Exhausted all attempts polling anchore-engine. Analysis is incomplete for sha256:7d6fb7e5e7a74a4309cc436f6d11c29a96cbf27a4a8cb45a50cb0a326dc32fe8
2018-10-15T00:48:43.880 ERROR AnchorePlugin Failing Anchore Container Image Scanner Plugin step due to errors in plugin execution
hudson.AbortException: Timed out waiting for anchore-engine analysis to complete (increasing engineRetries might help). Check above logs for errors from anchore-engine
at com.anchore.jenkins.plugins.anchore.BuildWorker.runGatesEngine(BuildWorker.java:480)
at com.anchore.jenkins.plugins.anchore.BuildWorker.runGates(BuildWorker.java:343)
at com.anchore.jenkins.plugins.anchore.AnchoreBuilder.perform(AnchoreBuilder.java:338)
at hudson.tasks.BuildStepCompatibilityLayer.perform(BuildStepCompatibilityLayer.java:81)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:744)
at hudson.model.Build$BuildExecution.build(Build.java:206)
at hudson.model.Build$BuildExecution.doRun(Build.java:163)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:504)
at hudson.model.Run.execute(Run.java:1724)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:97)
at hudson.model.Executor.run(Executor.java:421)
I also checked status and found below:
docker run anchore/engine-cli:latest anchore-cli --u admin --p admin123 --url http://172.18.0.1:8228/v1 system status
Service analyzer (dockerhostid-anchore-engine, http://anchore-engine:8084): up
Service catalog (dockerhostid-anchore-engine, http://anchore-engine:8082): up
Service policy_engine (dockerhostid-anchore-engine, http://anchore-engine:8087): down (unavailable)
Service simplequeue (dockerhostid-anchore-engine, http://anchore-engine:8083): up
Service apiext (dockerhostid-anchore-engine, http://anchore-engine:8228): up
Service kubernetes_webhook (dockerhostid-anchore-engine, http://anchore-engine:8338): up
Engine DB Version: 0.0.7
Engine Code Version: 0.2.4
It seems service policy engine is down
Service policy_engine (dockerhostid-anchore-engine, http://anchore-engine:8087): down (unavailable)
I also checked the docker logs . I found below error:
[service:policy_engine] 2018-10-15 09:37:46+0000 [-] [bootstrap] [DEBUG] service (policy_engine) starting in: 4
[service:policy_engine] 2018-10-15 09:37:46+0000 [-] [bootstrap] [INFO] Registration complete.
[service:policy_engine] 2018-10-15 09:37:46+0000 [-] [bootstrap] [INFO] Checking feeds client credentials
[service:policy_engine] 2018-10-15 09:37:46+0000 [-] [bootstrap] [DEBUG] Initializing a feeds client
[service:policy_engine] 2018-10-15 09:37:47+0000 [-] [bootstrap] [DEBUG] init values: [None, None, None, (), None, None]
[service:policy_engine] 2018-10-15 09:37:47+0000 [-] [bootstrap] [DEBUG] using values: ['https://ancho.re/v1/service/feeds', 'https://ancho.re/oauth/token', 'https://ancho.re/v1/account/users', 'anon#ancho.re', 3, 60]
[service:policy_engine] 2018-10-15 09:37:47+0000 [-] [urllib3.connectionpool] [DEBUG] Starting new HTTPS connection (1): ancho.re
[service:policy_engine] 2018-10-15 09:37:50+0000 [-] [bootstrap] [ERROR] Preflight checks failed with error: HTTPSConnectionPool(host='ancho.re', port=443): Max retries exceeded with url: /v1/account/users/anon#ancho.re (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ffa905f0b90>: Failed to establish a new connection: [Errno 113] No route to host',)). Aborting service startup
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/anchore_manager/cli/service.py", line 158, in startup_service
raise Exception("process exited: " + str(rc))
Exception: process exited: 1
[anchore-policy-engine] [anchore_manager.cli.service/startup_service()] [INFO] service process exited at (Mon Oct 15 09:37:50 2018): process exited: 1
[anchore-policy-engine] [anchore_manager.cli.service/startup_service()] [INFO] exiting service thread
Thanks and Regards,
Rohan Shetty
When images are added to anchore-engine, they are queued for analysis which moves them through a simple state machine that starts with ‘not_analyzed’, goes to ‘analyzing’ and finally ends in either ‘analyzed’ or ‘analysis_failed’. Only when an image has reached ‘analyzed’ will a policy evaluation be possible.
The anchore Jenkins plugin will add an image, then poll the engine for image status/evaluation for the configured number of tries (default 300). Once the image goes to ‘analyzed’ (where policy evaluation is possible), the plugin will then receive a policy evaluation result from the engine.
The plugin will fail the build (by default) if the max retries has been performed and the image has not reached ‘analyzed’, if the image does reach ‘analyzed’ but the policy evaluation is producing a ‘fail’ result (meaning the image didn’t pass your configured policy checks). Note that all build failure behavior can be controlled in the plugin (I.e. there are options to allow the plugin to succeed even if the analysis or image eval fails).
You’ll need to look at the end of the output from your build run (instead of just the beginning from your post), and combined with the information above, it should be clear which scenario is causing the plugin to fail the build.
We have resolved the issue.
Root Cause:
We were not able to establish a successful https connection to URL : https://ancho.re from within the anchore-engine docker container.
As a result the service:policy_engine was not able to start.
https://ancho.re is required to download policy feeds and sync-up periodically. Without these policy anchore-engine won't be able to analyse the docker images.
Solution:
1) We passed a HTTPS_PROXY URL as an environment variable in the docker-compose.yaml of anchore-engine.
We used this proxy URL to bypass restrictions in our environment and establish a connection with https://ancho.re url.
2) Restarted the docker containers.
Finally we got all services up and running including Anchore policy-engine.
FYI:
It takes a while to download all the required Feeds depending on your internet speed.
Lastly, Thanks to the Anchore community for quick responses and support over slack.
Hope this helps.
Warm Regards,
Rohan Shetty

Azure-IoT - Raspberry pi3 Forever script exited with code: 7

I'm running a forever script sending data from raspberry pi 3 to azure-iot-hub, using the following,
root#raspberrypi3:~# forever start /home/pi/azure/iam/ble_azure.js
After working for about 1.5 days, I stopped getting messages at auzre-iot-hub, when I checked forever list I got the following:
root#raspberrypi3:~# forever list
info: Forever processes running
data: uid command script forever pid id logfile uptime
data: [0] NWgI /usr/bin/nodejs /home/pi/azure/iam/ble_azure.js 8990 3784 /root/.forever/NWgI.log 0:21:17:38.742
When I checked the log file I get this error message:
/home/pi/azure/iam/node_modules/applicationinsights/AutoCollection/Exceptions.js:27
throw error;
^
NotConnectedError: mqtt.js returned client disconnecting error
at translateError (/home/pi/azure/iam/node_modules/azure-iot-device-mqtt/lib/mqtt-translate-error.js:25:11)
at MqttTwinReceiver._handleError (/home/pi/azure/iam/node_modules/azure-iot-device-mqtt/lib/mqtt-twin-receiver.js:201:42)
at /home/pi/azure/iam/node_modules/azure-iot-device-mqtt/lib/mqtt-twin-receiver.js:64:18
at MqttClient._checkDisconnecting (/home/pi/azure/iam/node_modules/mqtt/lib/client.js:314:7)
at MqttClient.subscribe (/home/pi/azure/iam/node_modules/mqtt/lib/client.js:423:12)
at /home/pi/azure/iam/node_modules/azure-iot-device-mqtt/lib/mqtt-twin-receiver.js:62:22
at _combinedTickCallback (internal/process/next_tick.js:73:7)
at process._tickCallback (internal/process/next_tick.js:104:9)
error: Forever detected script exited with code: 7
error: Script restart attempt #34
state has changed poweredOn
started scanning
[IoT hub Client] Connect error: mqtt.js returned premature close error
Through the log file, mqtt error keeps happening multiple times and forever handled it successfully, what I can't understand why after 1.5 days, I get this error:
**error: Forever detected script exited with code: 7
error: Script restart attempt #34**
Also why I keep getting such mqtt error, why it keeps disconnecting?
**NotConnectedError: mqtt.js returned client disconnecting error**
Forever --version
v0.15.3
root#raspberrypi3:~# uname -a
Linux raspberrypi3 4.9.35-v7+ #1014 SMP Fri Jun 30 14:47:43 BST 2017 armv7l GNU/Linux
Thanks
If you're using Client.fromConnectionString to instantiate the client object the SDK disconnects and reconnects every 45 minutes to renew the shared access signature token. (it doesn't happen with AMQP that uses a different authentication mechanism). It might be that when re-establishing the connection the client hits this "premature close" error that we've been tracking in this issue.
There are 2 things that could help limit potential errors linked to disconnecting/reconnecting:
Use X509 certificates instead of connection strings to authenticate.
Create a client object using Client.fromSharedAccessSignature and build a long-lived signature that does not require disconnecting and reconnecting as often.
Last but not least, the next release of the SDK (1.2.0) will include a retry/reconnect logic that is way more robust than what was there before. I'll update the issue to point to it when it is released.

NEO4J local server does not start

I am running Linux in VirtualBox and am having an issue that I did not encounter on my machine with Linux as the primary OS.
When launching the neo4j service through sudo ./neo4j start in /opt/neo4j-community-2.3.1/bin I get a timeout with the message Failed to start within 120 seconds. Neo4j Server may have failed to start, please check the logs
my log from /opt/neo4j-community-2.3.1/data/graph.db/messages.log says:
http://pastebin.com/wUA715QQ
and data/log/console.log says:
2016-01-06 02:07:03.404+0100 INFO Successfully started database
2016-01-06 02:07:03.603+0100 INFO Successfully stopped database
2016-01-06 02:07:03.604+0100 INFO Successfully shutdown Neo4j Server
2016-01-06 02:07:03.608+0100 ERROR Failed to start Neo4j: Starting Neo4j failed: Component 'org.neo4j.server.security.auth.FileUserRepository#9ab182' was successfully initialized, but failed to start. Please see attached cause exception. Starting Neo4j failed: Component 'org.neo4j.server.security.auth.FileUserRepository#9ab182' was successfully initialized, but failed to start. Please see attached cause exception.
org.neo4j.server.ServerStartupException: Starting Neo4j failed: Component 'org.neo4j.server.security.auth.FileUserRepository#9ab182' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.server.exception.ServerStartupErrors.translateToServerStartupError(ServerStartupErrors.java:67)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:234)
at org.neo4j.server.Bootstrapper.start(Bootstrapper.java:97)
at org.neo4j.server.CommunityBootstrapper.start(CommunityBootstrapper.java:48)
at org.neo4j.server.CommunityBootstrapper.main(CommunityBootstrapper.java:35)
Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'org.neo4j.server.security.auth.FileUserRepository#9ab182' was successfully initialized, but failed to start. Please see attached cause exception.
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:462)
at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:111)
at org.neo4j.server.AbstractNeoServer.start(AbstractNeoServer.java:194)
... 3 more
Caused by: java.nio.file.AccessDeniedException: /opt/neo4j-community-2.3.1/data/dbms/auth
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:84)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:102)
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:107)
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:214)
at java.nio.file.Files.newByteChannel(Files.java:361)
at java.nio.file.Files.newByteChannel(Files.java:407)
at java.nio.file.Files.readAllBytes(Files.java:3152)
at org.neo4j.server.security.auth.FileUserRepository.loadUsersFromFile(FileUserRepository.java:208)
at org.neo4j.server.security.auth.FileUserRepository.start(FileUserRepository.java:73)
at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.start(LifeSupport.java:452)
... 5 more
Any idea why the server won't start?
Check the permissions on /opt/neo4j-community-2.3.1/data/dbms/auth
See the line that says:
Caused by: java.nio.file.AccessDeniedException: /opt/neo4j-community-2.3.1/data/dbms/auth

Resources