I have Spring Boot 2 executable jar (runs by command java -jar myapp.jar) which I need to deploy to remote linux server as systemd service. Jenkins CI will manage building this jar from source and copy to proper location and restart java service associated with myapp.jar. Linux server has just one user 'jonas' with sudo privilege, in addition to jenkins user. Where do I need to put myapp.jar on linux server from security best practice point of view in production environment ? Possible location:
/var/myapplications/myapp.jar
/home/jona/myapplications/myapp.jar
/opt//myapplications/myapp.jar
Also where the log files generated by myapp.jar are going to be located?
Based on my experience, I would recommend to keep all third parties and applications under /opt/{{ app_name }}
You can follow if you like following structure which makes deployment of new artefacts much easier with option to quickly rollback if needed.
/opt/{{ app_name }}/releases/{{ app_version }}
/opt/{{ app_name }}/current <- Symlink to above
/opt/{{ app_name }}/current/logs <- directory for logs which would be part of the releases/{{ app_version }}
Your deployment script can create respective folders, manage permissions and symlinks for this purpose.
What i personally like is to make sure it runs under a non-privileged user with proper permissions to ensure only that particular user can read/execute/write on the directory recursively which would include logs which might have some sensitive information that might be included as part of the exception, however this purely depends on how you log.
Related
I've created a jenkins server, and I am trying to build a .net core 2.0.0 project on the server. I've been able to successfully pull from source control and store source files in the workspace. However, I'm running into an issue with running the dotnet build command. This is what I'm getting.
/usr/share/dotnet/sdk/2.0.0/Microsoft.Common.CurrentVersion.targets(4116,5):
error MSB3021: Unable to copy file
"obj/Debug/netcoreapp2.0/ubuntu.16.04-x64/Musify.pdb" to
"bin/Debug/netcoreapp2.0/ubuntu.16.04-x64/Musify.pdb". Access to the
path is denied. [/var/lib/jenkins/workspace/Musify/Musify.csproj]
now, I've given read write and execute permissions to every file and directory in /usr/share/dotnet/sdk/2.0.0/, and I've given read write and execute to every file and directory in my workspace (/var/lib/jenkins/workspace/Musify). I also believe my jenkins user is part of the sudo group.
The weird thing I am experiencing, is that I am able to, as root, run dotnet build in my workspace directory (/var/lib/jenkins/workspace/Musify), and the project builds. I cannot however, get the same results under the jenkins user (who should be part of the sudo group). My question is, how can I verify that Jenkins is using the jenkins system user, and that this user has the correct permissions to run this command. I am hosting jenkins on an ubuntu 16.04 x64 server.
UPDATE:
At the command line on your jenkins host run
ps -ef | grep jenkins
the first column will give you the USERID and it should be, as you say, jenkins
Then if you can login as jenkins to the host where the jenkins server is running run the following ....
groups
this will list out the groups that jenkins is a part of
If you want to fix the dotnet build issue take following actions:
Set DOTNET_CLI_HOME environment variable on the docker to a common
path like /tmp on the container. This path is used by the dotnet
to create necessary files to build the project. Check
Dotnet build permission denied in Docker container running Jenkins
Use -o or another accessible path to create the artifacts in the desired directory. e.g. dotnet build -o /tmp/dotnet/build/
microsoftisnotthatbad.sln
Re the jenkins user problem, run whoami in the container. If you get whoami: cannot find name for user ID blahblah it means the user is not found in the passwd file. There are 2 answers under Docker Plugin for Jenkins Pipeline - No user exists for uid 1005, if item 1 did not work, try the second:
Mount the host passwd to the container.
If the jenkins user is logged using an identity provider like LDAP on the Jenkins server or the slave server your job is using, the passwd file of the host will not have the jenkins user. Check the other answer on that post.
I am able to get logback logging to work correctly on my development environment (Windows) using Tomee 1.6.0.1; however, whenever I try to deploy the webapp on my Tomee server on Linux, it never creates the "test.app.log" file specified in the logback.xml configuration file inside of the WEB-INF/classes directory of the webapp.
Using maven, I have included logback-classic-1.1.2 and logback-core-1.1.2 in the WEB-INF/lib directory of the webapp.
I double check permissions for the directory to which I wish to write (CATALINA_BASE/logs), but it is set to rw for all users so I doubt that is the problem.
No matter what I try, all of my logs keep getting pushed to catalina.out. Can anybody solve the mystery as to why logging with logback works correctly on Windows (development environment) but not Linux?
It turns out that we were separating our EJB's into another directory and adding that directory as a "Deployment dir" in tomee.xml. This unhooks the webapp independent logging.
Romain, from Tomee fame, suggested I use either VirtualWebappLoader or jars.txt implementations.
I am new to Jenkins, starting now.
I see the following errors here.
Building in workspace /var/lib/jenkins/workspace/GIF_DECODER
[GIF_DECODER] $ /bin/bash /tmp/hudson3370393570271110742.sh
/tmp/hudson3370393570271110742.sh: line 2: cd: /home/joshis1/Data/E-Drive/Blog/Ready_to_post/Ubuntu/GIF_Decoder_LINUX: Permission denied
/var/lib/jenkins/workspace/GIF_DECODER
I have given the build commands as the following. This is under execute shell.
#!/bin/bash
cd /home/joshis1/Data/E-Drive/Blog/Ready_to_post/Ubuntu/GIF_Decoder_LINUX
pwd
make clean
make everything
I am not understanding what is the issue here? Looks to be a permission year. I have jenkins in sudoers list. I don't why it looks for /var/lib/jenkins/workspace/GIF_Decoder.
I see the jenkins page under - http://:8080/job/GIF_DECODER/configure
How can I setup the hostname so that instead of localhost I have some meaningful name? Please help. Also, what is the best document/video to start with Jenkins for linux users? How to debug Jenkins issue seen? How can I instantiate a build by sending an email?
I am not understanding what is the issue here?
It's a permission error. It looks like your jenkins user does not have permissions for /home/joshis1/Data/E-Drive/Blog/Ready_to_post/Ubuntu/GIF_Decoder_LINUX
How can I setup the hostname so that instead of localhost I have some meaningful name?
Try the hostname of your server in the URL. e.g. http://myserver:8080/job/GIF_DECODER/configure
Also, what is the best document/video to start with Jenkins for linux users?
I expect that you have a basic understanding of how linux works. This is important on how to configure your jenkins and write the necessary build steps. The most basic information you will find on http://jenkins-ci.org and basic installation https://wiki.jenkins-ci.org/display/JENKINS/Installing+Jenkins
How to debug Jenkins issue seen?
That depends ...
... on the issue. If you have build problems check the console output of the job first. In addition, Jenkins writes its logfiles into the jenkins home directory.
How can I instantiate a build by sending an email?
Check out the mail commander plugin
I have a very simple NServiceBus.Host.exe application that is using the default logging and the Production profile. According to the documentation, this should result in an appending file log that should appear in the same folder as the EXE. However, when I run the application as a service, the log file doesn't appear in the same folder as the EXE, and thus far I've been unable to locate it at all. The service is running as Local System. Do I need to run it as a user account and look for the file in the AppData folder somewhere? Is it under c:\windows somewhere? Where is it and is there a way for me to have it actually log to a file in the same folder as the EXE as advertised?
Update:
Using ProcMon and ProcExp from SysInternals, I can see that there is no attempt to create any log file in the folder where my EXE exists, nor are there any file permission errors while trying to create a log file anywhere, at least not from the PID of the service (if for some reason log4net spins up another process to do this work then I might have missed it).
It turns out that the service wasn't actually running in the Production profile. I had for some reason gotten it into my head that services would run in the production profile by default, while running it in interactive mode would use Lite by default. Not so - the service will use the Lite profile unless you specify otherwise. I changed my command to install the service from:
NServiceBus.Host.exe /install /displayName:MyService
to
NServiceBus.Host.exe /install /displayName:MyService NServiceBus.Production
and this fixed the issue.
I'm asking in both contexts: technically and stylistically.
Can my application/daemon keep a pidfile in /opt/my_app/run/?
Is it very bad to do so?
My need is this: my daemon runs under a specific user, and the implementor must mkdir a new directory in /var/run, chown, and chgrp it to make my daemon run. Seems easier to just keep the pidfile local (to the daemon).
I wouldn't put a pidfile under an application installation directory such as /opt/my_app/whatever. This directory could be mounted read-only, could be shared between machines, could be watched by a daemon that treats any change there as a possible break-in attempt…
The normal location for pidfiles is /var/run. Most unices will clean this directory on boot; under Ubuntu this is achieved by /var/run an in-memory filesystem (tmpfs).
If you start your daemon from a script that's running as root, have it create a subdirectory /var/run/gmooredaemon and chown it to the daemon-running user before suing to the user and starting the daemon.
On many modern Linux systems, if you start the daemon from a script or launcher that isn't running as root, you can put the pidfile in /run/user/$UID, which is a per-user equivalent of the traditional /var/run. Note that the root part of the launcher, or a boot script running as root, needs to create the directory (for a human user, the directory is created when the user logs in).
Otherwise, pick a location under /tmp or /var/tmp, but this introduces additional complexity because the pidfile's name can't be uniquely determined if it's in a world-writable directory.
In any case, make it easy (command-line option, plus perhaps a compile-time option) for the distributor or administrator to change the pidfile location.
The location of the pid file should be configurable. /var/run is standard for pid files, the same as /var/log is standard for logs. But your daemon should allow you to overwrite this setting in some config file.
/opt is used to install 'self-contained' applications, so nothing wrong here. Using /opt/my_app/etc/ for config files, /opt/my_app/log/ for logs and so on - common practice for this kind of application.
This away you can distribute your applications as a TGZ file instead of maintaining a package for every package manager (at least DEB since you tagged ubuntu). I would recommend this for in-house applications or situations where you have great control over the environment. The reasoning is that it makes no sense if the safe costs more than what you are putting inside (the work required to pack the application should not eclipse the effort required to write the application).
Another convention, if you're not running the script as root, is to put the pidfile in ~/.my_app/my_app.pid. It's simpler this way while still being secure as the home directory is not world-writeable.