Since the logstash process on windows gets closed immediately after the connection to the machine is lost or logged off from the machine, How to keep the logstash process running continuously and in back-end on windows?
Use Non-Sucking Service Manager to install logstash as a windows service. Services can be started at boot and run without an active user login.
You can use the Windows Service Wrapper (https://github.com/kohsuke/winsw).
Simple put the wsw.exe into the logstash bin directory, rename it to logstash.exe and do the same for the configuration xml file. So you have a logstash.exe and logstash.xml file in the bin directory of logstash.
Now you need to adjust the xml file a bit. Mine looks like this (tweak it to your needs):
<configuration>
<id>logstash</id>
<name>Logstash</name>
<description>Logstash from elastic.co</description>
<executable>D:\logstash\bin\logstash.bat</executable>
<arguments></arguments>
<serviceaccount>
<domain>local</domain>
<user>waffel</user>
<password>XXX</password>-->
</serviceaccount>
<onfailure action="restart" delay="10 sec"/>
<onfailure action="restart" delay="20 sec"/>
<onfailure action="none" />
<resetfailure>1 hour</resetfailure>
<priority>Normal</priority>
<stoptimeout>15 sec</stoptimeout>
<stopparentprocessfirst>false</stopparentprocessfirst>
<startmode>Automatic</startmode>
<waithint>15 sec</waithint>
<sleeptime>1 sec</sleeptime>
<logpath>D:\logstash\logs</logpath>
<log mode="append"/>
<env name="JAVA_HOME" value="D:\jdk8x64" />
</configuration>
Then you can simple hit from the cmd (as admin)
logstash.exe test
Or to install the service
logstash.exe install
and then you can run from cmd (or service management)
logstash.exe start
logstash.exe stop
You may whatch the logsfiles for potential errors. For me it works fine with logstash 5.5.1
Related
Can any node.js experts tell me how I might configure node JS to autostart a server when my machine boots?
I'm on Windows
This isn't something to configure in node.js at all, this is purely OS responsibility (Windows in your case). The most reliable way to achieve this is through a Windows Service.
There's this super easy module that installs a node script as a windows service, it's called node-windows (npm, github, documentation). I've used before and worked like a charm.
var Service = require('node-windows').Service;
// Create a new service object
var svc = new Service({
name:'Hello World',
description: 'The nodejs.org example web server.',
script: 'C:\\path\\to\\helloworld.js'
});
// Listen for the "install" event, which indicates the
// process is available as a service.
svc.on('install',function(){
svc.start();
});
svc.install();
p.s.
I found the thing so useful that I built an even easier to use wrapper around it (npm, github).
Installing it:
npm install -g qckwinsvc
Installing your service:
> qckwinsvc
prompt: Service name: [name for your service]
prompt: Service description: [description for it]
prompt: Node script path: [path of your node script]
Service installed
Uninstalling your service:
> qckwinsvc --uninstall
prompt: Service name: [name of your service]
prompt: Node script path: [path of your node script]
Service stopped
Service uninstalled
If you are using Linux, macOS or Windows pm2 is your friend. It's a process manager that handle clusters very well.
You install it:
npm install -g pm2
Start a cluster of, for example, 3 processes:
pm2 start app.js -i 3
And make pm2 starts them at boot:
pm2 startup
It has an API, an even a monitor interface:
Go to github and read the instructions. It's easy to use and very handy. Best thing ever since forever.
If I'm not wrong, you can start your application using command line and thus also using a batch file. In that case it is not a very hard task to start it with Windows login.
You just create a batch file with the following content:
node C:\myapp.js
and save it with .bat extention. Here myapp.js is your app, which in this example is located in C: drive (spcify the path).
Now you can just throw the batch file in your startup folder which is located at C:\Users\%username%\AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup
Just open it using %appdata% in run dailog box and locate to >Roaming>Microsoft>Windows>Start Menu>Programs>Startup
The batch file will be executed at login time and start your node application from cmd.
This can easily be done manually with the Windows Task Scheduler.
First, install forever.
Then, create a batch file that contains the following:
cd C:\path\to\project\root
call C:\Users\Username\AppData\Roaming\npm\forever.cmd start server.js
exit 0
Lastly, create a scheduled task that runs when you log on. This task should call the batch file.
I would recommend installing your node.js app as a Windows service, and then set the service to run at startup. That should make it a bit easier to control the startup action by using the Windows Services snapin rather than having to add or remove batch files in the Startup folder.
Another service-related question in Stackoverflow provided a couple of (apprently) really good options. Check out How to install node.js as a Windows Service. node-windows looks really promising to me. As an aside, I used similar tools for Java apps that needed to run as services. It made my life a whole lot easier. Hope this helps.
you should try this
npm forever
https://www.npmjs.com/package/forever
Use pm2 to start and run your nodejs processes on windows.
Be sure to read this github discussion of how to set up task scheduler to start pm2: https://github.com/Unitech/pm2/issues/1079
Here is another solution I wrote in C# to auto startup native node server or pm2 server on Windows.
I know there are multiple ways to achieve this as per solutions shared above. I haven't tried all of them but some third party services lack clarity around what are all tasks being run in the background. I have achieved this through a powershell script similar to the one mentioned as windows batch file. I have scheduled it using Windows Tasks Scheduler to run every minute. This has been quite efficient and transparent so far. The advantage I have here is that I am checking the process explicitly before starting it again. This wouldn't cause much overhead to the CPU on the server. Also you don't have to explicitly place the file into the startup folders.
function CheckNodeService ()
{
$node = Get-Process node -ErrorAction SilentlyContinue
if($node)
{
echo 'Node Running'
}
else
{
echo 'Node not Running'
Start-Process "C:\Program Files\nodejs\node.exe" -ArgumentList "app.js" -WorkingDirectory "E:\MyApplication"
echo 'Node started'
}
}
CheckNodeService
Simply use this, install, run and save current process list
https://www.npmjs.com/package/pm2-windows-startup
By my exp., after restart server, need to logon, in order to trigger the auto startup.
Need to create a batch file inside project folder.
Write this code in batch file
#echo off
start npm start
save batch file with myprojectname.bat
Go to run command and press window + R
Enter this command :- shell:common startup
Press ok then folder will be open.
Folder path like as C:\ProgramData\Microsoft\Windows\Start Menu\Programs\StartUp
You will be paste your myprojectname.bat file.
You can check also. Need to system restart.
Copied directly from this answer:
You could write a script in any language you want to automate this (even using nodejs) and then just install a shortcut to that script in the user's %appdata%\Microsoft\Windows\Start Menu\Programs\Startup folder
I have downloaded node.js executable. How can I run that executable as windows service?
I cannot use standard node.js installer, since I need to run multiple version of node.js concurrently.
Late to the party, but node-windows will do the trick too.
It also has system logging built in.
There is an API to create scripts from code, i.e.
var Service = require('node-windows').Service;
// Create a new service object
var svc = new Service({
name:'Hello World',
description: 'The nodejs.org example web server.',
script: 'C:\\path\\to\\helloworld.js'
});
// Listen for the "install" event, which indicates the
// process is available as a service.
svc.on('install',function(){
svc.start();
});
svc.install();
FD: I'm the author of this module.
I found the thing so useful that I built an even easier to use wrapper around it (npm, github).
Installing it:
npm install -g qckwinsvc
Installing your service:
qckwinsvc
prompt: Service name: [name for your service]
prompt: Service description: [description for it]
prompt: Node script path: [path of your node script]
Service installed
Uninstalling your service:
qckwinsvc --uninstall
prompt: Service name: [name of your service]
prompt: Node script path: [path of your node script]
Service stopped
Service uninstalled
WinSer is a node.js friendly wrapper around the popular NSSM (Non-Sucking Service Manager)
From this blog
Next up, I wanted to host node as a service, just like IIS. This way
it’d start up with my machine, run in the background, restart
automatically if it crashes and so forth.
This is where nssm, the non-sucking service manager, enters the
picture. This tool lets you host a normal .exe as a Windows service.
Here are the commands I used to setup an instance of the your node
application as a service, open your cmd like administrator and type
following commands:
nssm.exe install service_name c:\your_nodejs_directory\node.exe c:\your_application_directory\server.js
net start service_name
I'm not addressing the question directly, but providing an alternative that might also meet your requirement in a more node.js fashion way.
Functionally the requirements are:
Have the logic (app) running in the background
Be able to start/stop the logic
Automatically start the logic when system boots up
These requirements can be satisfied by using a process manager (PM) and making the process manager start on system startup. Two good PMs that are Windows-friendly are:
PM2
forever
To make the PM start automatically, the most simple way is to create a scheduled task with a "At Startup" trigger:
Since qckwinsvc has not been updated for a while there's a new version called qckwinsvc2 (npm, github)
It now supports args passed to the service. It also keeps a local cache so you don't have to provide a path every time you want to perform an action
Use the now arg to start the service as soon as it's installed
qckwinsvc2 install name="Hello" description="Hello World" path="C:\index.js" args="--y" now
qckwinsvc2 uninstall name="Hello"
qckwinsvc2 list
The process manager + task scheduler approach I posted a year ago works well with some one-off service installations. But recently I started to design system in a micro-service fashion, with many small services talking to each other via IPC. So manually configuring each service has become unbearable.
Towards the goal of installing services without manual configuration, I created serman, a command line tool (install with npm i -g serman) to install an executable as a service. All you need to write (and only write once) is a simple service configuration file along with your executable. Run
serman install <path_to_config_file>
will install the service. stdout and stderr are all logged. For more info, take a look at the project website.
A working configuration file is very simple, as demonstrated below. But it also has many useful features such as <env> and <persistent_env> below.
<service>
<id>hello</id>
<name>hello</name>
<description>This service runs the hello application</description>
<executable>node.exe</executable>
<!--
{{dir}} will be expanded to the containing directory of your
config file, which is normally where your executable locates
-->
<arguments>"{{dir}}\hello.js"</arguments>
<logmode>rotate</logmode>
<!-- OPTIONAL FEATURE:
NODE_ENV=production will be an environment variable
available to your application, but not visible outside
of your application
-->
<env name="NODE_ENV" value="production"/>
<!-- OPTIONAL FEATURE:
FOO_SERVICE_PORT=8989 will be persisted as an environment
variable machine-wide.
-->
<persistent_env name="FOO_SERVICE_PORT" value="8989" />
</service>
https://nssm.cc/ service helper good for create windows service by batch file
i use from nssm & good working for any app & any file
I'm trying to deploy Kibana 4 to Azure Websites. I can't use bin/kibanta.bat file since Azure Websites uses start script in package.json to bootstrap application. I tried to update package.json start script to run bin\kibana.js file and environment variables in it. After that Azure starts running Kibana server but I'm getting this error: Uncaught ReferenceError: ZeroClipboard is not defined (http://kibana-site.azurewebsites.net/index.js?_b=5827:89458). Does anyone tried to make Kibana 4 run not using bin\kibana* files? Maybe I have to specify additional environment variables?
The solution is explained here: https://github.com/elastic/kibana/issues/2617
It's a bug that was fixed later.
We have started migrating our one of several projects to team city as part of CI. Below is how we have setup teamcity build. We are trying to deploy WebSite.
1) Build Step 1 (Package installation)
Using "command line " runner type install required package.
2) Build Step 2 (Build)
Using Runner type "Visual Studio (sln)" (Visual Studio 2010) build website.
3) Build Step 3 (Deploy Web Site)
Using ".Net Process Runner", deployer.exe (x86 built with .Net Framework 4) deploy site.
Deployer.exe reads config file. Config file contains "BuildId", "Environment" and "Servers" where we want build to be pushed.
<buildType id="bt52">
<env name="Debug">
<server path="SERVER1" />
</env>
<env name="QA">
<server path="SERVER2" />
<server path="SERVER3" />
</env>
<env name="UAT">
<server path="SERVER4" />
<server path="SERVER5" />
</env>
</buildType>
Deployer.exe is called with required parameters as below. Which reads config and deploys site to Server2 and Server3.
Deployer.exe "bt52" "QA" "siteQA" "E:\BuildAgent\work\2483052e33e5e1e8\src\diy\" msdeploy.exe
Problem area is step #3.
When we run deployer.exe using .Net process runner as part of team city we see its hanging and not responsind sometime even for 45 minutes. When we try to execute same deployer.exe from build server using command line script executes within couple of seconds.
E:\TeamCity_custom_applications\deployer>Deployer.exe farm1-1 QA siteQA E:\BuildAgent\work\2483052e33e5e1e8\src\diy\ msdeploy.exe
Info
: Processing batch run ... Info : Processing command ...msdeploy.exe
-verb:sync -source:contentPath="E:\BuildAgent\work\2483052e33e5e1e8\src\diy\" -dest:contentPath="siteQA",wmsvc="SERVER2",userName="*****",password="******",authType="Basic"-skip:objectName=filePath,absolutePath=web.config -skip:objectName=dirPath,absolutePath="bin" -enableRule:DoNotDeleteRule -allowUntrusted Info : output >>Total changes: 0 (0 added, 0 deleted, 0 updated, 0 parameters changed, 0
bytes copied) Info : error >>(none) Info : ExitCode >> 0 Info :
Processing command ...msdeploy.exe -verb:sync
-source:contentPath="E:\BuildAgent\work\2483052e33e5e1e8\src\diy\" -dest:contentPath="siteQA",wmsvc="SERVER3",userName="******",password="******",authType="Basic"
-skip:objectName=filePath,absolutePath=web.config -skip:objectName=dirPath,absolutePath="bin" -enableRule:DoNotDeleteRule -allowUntrusted Info : output >>Total changes: 0 (0 added, 0 deleted, 0 updated, 0 parameters ch anged, 0
bytes copied) Info : error >>(none) Info : ExitCode >> 0
Info: Deploy Script Complete.
One more thing we observed is running deployer.exe through teamcity I see that site content gets copied but only for 1 server and teamcity build status stays in "Running" mode. I am wondering if someone can please put little bit of insight on how can I look into this issue.
Update 1:
Thanks for your time looking into it !! What we ended up doing is, Instead of running command "msdeploy.exe" from "cmd.exe" we added "msdeploy.exe" location as Environment variable and executed "msdeploy.exe" in loop for # of servers. This has resolved issue of hanging. Now I am just curious to know why would it behave in such manner where if you execute "msdeploy.exe" from "cmd.exe" it would hang while running directly "msdeploy.exe" it would execute successfully. Any insight into same would be greatly appreciated.
Update 2:
I have added image which explains behavior using process explorer. If we kill msdeploy.exe from process explorer than for next all deployments to that server will not have the issue of build hanging. Please see below image
To be honest, it sounds like you're running into issues with redirecting input/output streams. TeamCity is running your application in a totally headless environment and then you, in turn, are attempting to redirect and parse the output of msdeploy.exe
If that's the case, I'd recommend looking into using the MSDeploy API instead of msdeploy.exe. The latter is just a command line wrapper for the former, so all the functionality is available to you. There's a sample deployment application available on the IIS blog if you need help getting started.
It seems you have NUnit build step configured in TeamCity and invoke cmd.exe from your test. This looks like an issue with the test code then. Most probably it will reproduce without TeamCity if you run the test in question with NUnit directly.
As Richard noted, most probably the issue root cause is related to stdin/stdout processing.
If you want to fix it in your code, you can try to experiment by explicitly closing stdin or the other way around, try writing something into it, etc.
Work around we did is, we observed msdeploy doesn't take more than 3-5 seconds to execute and deploy (Even for our biggest project which is almost 300mb website). So we set timeout of 20 seconds. So far since last 1 weeks we have not seen any issue with it and hopefully it will not cause more trouble but still we are not sure why such behavior.
I have downloaded node.js executable. How can I run that executable as windows service?
I cannot use standard node.js installer, since I need to run multiple version of node.js concurrently.
Late to the party, but node-windows will do the trick too.
It also has system logging built in.
There is an API to create scripts from code, i.e.
var Service = require('node-windows').Service;
// Create a new service object
var svc = new Service({
name:'Hello World',
description: 'The nodejs.org example web server.',
script: 'C:\\path\\to\\helloworld.js'
});
// Listen for the "install" event, which indicates the
// process is available as a service.
svc.on('install',function(){
svc.start();
});
svc.install();
FD: I'm the author of this module.
I found the thing so useful that I built an even easier to use wrapper around it (npm, github).
Installing it:
npm install -g qckwinsvc
Installing your service:
qckwinsvc
prompt: Service name: [name for your service]
prompt: Service description: [description for it]
prompt: Node script path: [path of your node script]
Service installed
Uninstalling your service:
qckwinsvc --uninstall
prompt: Service name: [name of your service]
prompt: Node script path: [path of your node script]
Service stopped
Service uninstalled
WinSer is a node.js friendly wrapper around the popular NSSM (Non-Sucking Service Manager)
From this blog
Next up, I wanted to host node as a service, just like IIS. This way
it’d start up with my machine, run in the background, restart
automatically if it crashes and so forth.
This is where nssm, the non-sucking service manager, enters the
picture. This tool lets you host a normal .exe as a Windows service.
Here are the commands I used to setup an instance of the your node
application as a service, open your cmd like administrator and type
following commands:
nssm.exe install service_name c:\your_nodejs_directory\node.exe c:\your_application_directory\server.js
net start service_name
I'm not addressing the question directly, but providing an alternative that might also meet your requirement in a more node.js fashion way.
Functionally the requirements are:
Have the logic (app) running in the background
Be able to start/stop the logic
Automatically start the logic when system boots up
These requirements can be satisfied by using a process manager (PM) and making the process manager start on system startup. Two good PMs that are Windows-friendly are:
PM2
forever
To make the PM start automatically, the most simple way is to create a scheduled task with a "At Startup" trigger:
Since qckwinsvc has not been updated for a while there's a new version called qckwinsvc2 (npm, github)
It now supports args passed to the service. It also keeps a local cache so you don't have to provide a path every time you want to perform an action
Use the now arg to start the service as soon as it's installed
qckwinsvc2 install name="Hello" description="Hello World" path="C:\index.js" args="--y" now
qckwinsvc2 uninstall name="Hello"
qckwinsvc2 list
The process manager + task scheduler approach I posted a year ago works well with some one-off service installations. But recently I started to design system in a micro-service fashion, with many small services talking to each other via IPC. So manually configuring each service has become unbearable.
Towards the goal of installing services without manual configuration, I created serman, a command line tool (install with npm i -g serman) to install an executable as a service. All you need to write (and only write once) is a simple service configuration file along with your executable. Run
serman install <path_to_config_file>
will install the service. stdout and stderr are all logged. For more info, take a look at the project website.
A working configuration file is very simple, as demonstrated below. But it also has many useful features such as <env> and <persistent_env> below.
<service>
<id>hello</id>
<name>hello</name>
<description>This service runs the hello application</description>
<executable>node.exe</executable>
<!--
{{dir}} will be expanded to the containing directory of your
config file, which is normally where your executable locates
-->
<arguments>"{{dir}}\hello.js"</arguments>
<logmode>rotate</logmode>
<!-- OPTIONAL FEATURE:
NODE_ENV=production will be an environment variable
available to your application, but not visible outside
of your application
-->
<env name="NODE_ENV" value="production"/>
<!-- OPTIONAL FEATURE:
FOO_SERVICE_PORT=8989 will be persisted as an environment
variable machine-wide.
-->
<persistent_env name="FOO_SERVICE_PORT" value="8989" />
</service>
https://nssm.cc/ service helper good for create windows service by batch file
i use from nssm & good working for any app & any file