Tired of creating /run/postgresql and setting read and execute writes after every reboot - linux

I'm running Arch Linux, I installed PostgreSQL as any other arch package. I'm running postgres with a local database located in my user directory. (postgres -D /home/user/data/) When I do so, I get the error FATAL: could not create lock file "/run/postgresql/.s.PGSQL.5432.lock": No such file or directory. Creating the directory /run/postgresql and giving the postgres user access solves this problem
$ sudo mkdir /run/postgresql
$ sudo chmod a+w /run/postgresql
however I'm tired of writing these commands every time I reboot, as /run gets cleared when rebooting. I could write a script to execute these commands, but I feel like I'm doing this the wrong way to begin with. Is there any way I could let postgres create its directory itself, or maybe have it not use /run/postgres for it's lock files in the first place?

Postgres creates the lock file in /run/postgresql by default.
From the manpage:
-k directory
Specifies the directory of the Unix-domain socket on which postgres is
to listen for connections from client applications. The default is
normally /run/postgresql, but can be changed at build time.
Use -k directory to tell postgres to use a different directory.
Run your command as postgres -k /tmp -D /home/user/data/.

Solution 1 (By managing temporary directory /run/postgresql, /var/run/postgresql)
Directory /run/postgresql is a temporary directory. Path
/var/run/postgresql is usually a symbolic link to /run/postgresql.
systemd-tmpfiles is mechanism to manage such temporary files and directories. systemd-tmpfiles creates temporary directories during
boot and sets their owner, group and permissions. It may read configuration files in three different locations. Files in
/etc/tmpfiles.d override files with the same name in
/usr/lib/tmpfiles.d and /run/tmpfiles.d.
We can create directory /run/postgresql on the fly at boot time using systemd-tmpfiles mechanism by creating postgresql configuration file as below
echo "d /run/postgresql 0755 postgres postgres -" > /usr/lib/tmpfiles.d/postgresql.conf
Solution 2 (By relocating PostgreSQL lock file location)
Another way to fix the issue is to relocate the PostgreSQL lock file
location. We can do so by using below query
ALTER SYSTEM SET unix_socket_directories='<any-existing-path-with-valid-permissions>, /tmp';
Here we can provide any path for PostgreSQL lock file which is already present on the system and have required permissions to manage lock files by postgres user.

Related

Unable to create / edit files as non-root through Samba mount

I'm trying to setup a code-server (vscode in browser) instance and read/write from a mounted samba share. Unfortunately when I try to add a file it gives me an error that I do not have permissions to read/write to that folder. When I try to add files with the same credentials on Windows it does work though. This is the error that VSCode gives me:
Unable to write file
'vscode-remote://localhost:8080/home/user/repository/test'
(NoPermissions (FileSystemError): Error: EACCES: permission denied,
open '/home/gmetitieri/user/test')
If I sudo touch file.txt then the file will be created and added. I already used chmod and added full access to the folder but it still won't work. Is this a credentials thing or am I missing something?
I already tried this answer but it still doesn't let me write as non-root
Edit: This is the command I used to mount the drive (just with different folder names and IP address):
sudo mount -t cifs -o rw,vers=3.0,credentials=/root/.examplecredentials //192.168.18.112/sharedDir /media/share
Considering "non-root through Samba", especially in new releases of OpenSuse (...15.3 -- 15.4), I do few movements into normal configuration panels (no sudo commands or anything technical).
Using Yast Firewall section -- For now (immediate solution):
I turn off the firewall, then see what you can turn on (after this) to keep the samba working with Microsoft Windows.
More details on how to do this with images on my website.
This happens when the directory on the Samba share does not have permission for non-root users.
In your smb4.conf file:
[test]
comment = Test share
path = /path/to/directory
force user = unixuser
valid users = sambauser
In this example, unixuser should be the owner of the files in /path/to/directory. The user logged into Samba in this example is a user called sambauser.

Create PostgreSQL Database without root privilege

Currently, I use
$ sudo service postgresql start
to start the PostgreSQL server and
$ sudo -u postgres createdb testdb --owner ownername
to create a database. However, these commands need root privilege. How can I do these without root privilege/sudo on Linux (Ubuntu)?
You can run PostgreSQL without root privs by creating a new instance (which PostgreSQL calls a "cluster") and starting it.
You can't use the Ubuntu init scripts, wrapper tools like pg_ctlcluster, etc if you do this. You must use only PostgreSQL's own tools.
To create the new PostgreSQL instance with the superuser equal to your username, data directory in your home directory, and md5 auth enabled by default, use:
initdb -D $HOME/my_postgres -A md5 -U $USER
Adjust as desired; see initdb --help.
You'll then need to edit postgresql.conf to change the port to a non-default one, since your system probably runs its own postgres on the default port 5432. (If you want to limit access strictly to you, you can instead set listen_addresses = '' and unix_socket_directories = /home/myuser/postgres_socket or whatever. But it's simpler to just use a different port.)
To start it:
pg_ctl -D $HOME/my_postgres -w start
To connect to it, specify the port you chose:
psql -p 5434 ...
(If you changed unix_socket_directories you'll also want to specify the path you gave, like -h /home/myuser/postgres_socket.)
To make psql etc connect to your postgres by default, edit your ~/.bashrc to add something like
export PGPORT=5434
but note that'll also affect the default port for connections to other hosts.
To stop it:
pg_ctl -D $HOME/my_postgres -w stop
but you can also just shut down without stopping it, it doesn't care and will recover safely when you start it next.
To autostart it when you log in when it's set up in your home directory you'd have to use your desktop environment's run-at-startup features. They vary depending on environment and version so I can't give details here; it's different for GNOME 3, Unity (ubuntu), KDE, XFCE, etc.
Note that this approach still uses the system packages for PostgreSQL. This is important because if you uninstall (say) PostgreSQL 9.4 and install 9.6, your copy in your home dir will stop working. If you want it entirely independent of system packages, as you probably do if you don't control the system, you should compile PostgreSQL from source or use the binary installer to install inside your home directory.
Postgres can run without root permission.
Just download from
https://www.enterprisedb.com/download-postgresql-binaries
and run
Init database
./initdb -D /data
Run postgres
./bin/postgres -D /data
Create database
./bin/createdb mydb
Connect with psql
./bin/psql mydb
(https://www.golery.com/pencil/vU)

pg_upgrade oldbindir check fails

I'm trying to upgrade postgresql database from 9.1 to 9.4. I ran the 9.4 version of pg_upgrade as follows:
sudo -u postgres pg_upgrade -b <path>/postgres/9.1/bin -B <path>/postgres/9.4/bin -d <path>/9.1DBs/ -D <path>/<empty folder>
I get:
check failed for: <path>/postgres/9.1/bin Permission denied
The folder exists, I set the permissions to 777, even gave the postgres user ownership of the directory structure, but I'm still getting the same error.
Running in verbose mode does not provide any extra info.
The partition with that contains the old binaries is mounted under /media/ and I have read/write access.
Any ideas what could be causing this?
This error
check failed for: /postgres/9.1/bin Permission denied
happens because access permission on directories (x bit) need to be set in every directory in the hierarchy, from the root to the directory containing these executables files.
This is detailed at Unix.SE in:
Do the parent directory's permissions matter when accessing a subdirectory?

Cassandra moving data_file_firectories

Regarding the location of cassandra created data files and system files, I need to move the "commitlog_directory", "data_file_directories" and "saved_caches_directory" which have settings in the "cassandra.yaml" config file. It is currently at the default location "/var/lib/cassandra". The data is only some test data and of course the system generated keyspaces which are
dse_perf
dse_system
OpsCenter
system
system_traces
There are also the commitlog and saved_caches.db to move.
I am thinking of moving the keyspace directories with linux shell commands but I'm very unsure if they will become corrupt somehow. There is simply no space in the default drive and we need to move everything to the secondary and tertiary mounted drives.
Right now I'm in the process of moving all the files and resetting the yaml settings.
I have two questions -
Regarding the cassandra.yaml file, are there any other files besides this that are depended upon to have the location of the commitlog_directory and data_file_directories and saved_caches_directory, and their 'wrong location' will cause failure once I move all these files? I am also concerned the files (like the db files) inside the tables themselves have references to their own location and cause failure once they are moved.
If I just move the three settings commitlog_directory and data_file_directories and saved_caches_directory, will dse/cassandra actually create all the system keyspaces (system_traces, dse_perf, system, OpsCenter, dse_system), and the commitlof and the saved_caches.db, and will any other upstream config files be out of sync with that (same as first part of question 1)?
It is a very new installation so re installing would not be the end of the world but I realllly don't want to because we have kerberos and all kinds of other stuff on top of this cluster now.
This OS is ubuntu 14.0.4 and the DSE version is 4.7.
I just finished doing this. My instances are in AWS EC2 so your process may vary, but in essence:
create a new volume and attach it to the instance. my new device was
/dev/xvdg.
create new mount point sudo mkdir /new_data
format the new volume sudo mkfs -t ext4 /dev/xvdg
edit /etc/fstab so that your mount will survive reboots and add this
line /dev/xvdg /new_data ext4 defaults,nofail,nobootwait 0 2
mount the new volume sudo mount -a
make the new directories sudo mkdir -p
/new_data/lib/cassandra/commitlog
chown the ownership sudo chown -R cassandra:cassandra
/new_data/lib/cassandra
change cassandra.yaml to point to the new dirs
drain the node. if you're moving the data dir, copy over the data
from the old location to the new location. if you're moving
commitlog only, just restart cassandra.
I was able to move all the files and the commitlog as well. I changed the yaml and pointed it to where I wanted it to go. Remember to run the following command afterward -
chown -R cassandra:cassandra
And voila! Everything is reading/writing as it should. Cassandra is neato.

Shared volume/file permissions/ownership (Docker)

I'm having a slightly annoying issue while using a Docker container (I'm on Ubuntu, so no virtualization like VMWare or b2d). I've built my image, and have a running container that has one shared (mounted) directory from my host, and one shared (mounted) file from my host. Here's the docker run command in full:
docker run -dit \
-p 80:80 \
--name my-container \
-v $(pwd)/components:/var/www/components \
-v $(pwd)/index.php:/var/www/index.php \
my-image
This works great, and both /components (and its contents) and the file are shared appropriately. However, when I want to make changes to either the directory (e.g. adding a new file or folder), or edit the mounted file (or any file in the directory), I'm unable to do so due to incorrect permissions. Running ls- lFh shows that the owner and group for the mounted items have been changed to libuuid:libuuid. Modifying either the file or parent directory requires root permissions, which impedes my workflow (as I'm working from Sublime Text, not Terminal, I'm presented with a popup for admin privs).
Why does this occur? How can I work around this / handle this properly? From Managing Data Volumes: Mount a Host File as a Data Volume:
Note: Many tools used to edit files including vi and sed --in-place may result in an inode change. Since Docker v1.1.0, this will produce an error such as “sed: cannot rename ./sedKdJ9Dy: Device or resource busy”. In the case where you want to edit the mounted file, it is often easiest to instead mount the parent directory.
This would seem to suggest that instead of mounting /components and /index.php, I should instead mount the parent directory of both. Sounds great in theory, but based on the behavior of the -v option and how it interacts with /directory, it would seem that every file in my parent directory would be altered to be owned by libuuid:libuuid. Additionally, I have lots of things inside the parent directory that are not needed in the container - things like build tools, various files, some compressed folders, etc. Mounting the whole parent directory would seem to be wasteful.
Running chown user:group on /components and /index.php on my host machine allow me to work around this and seem to continue to sync with the container. Is this something I'll need to do every time I run a container with mounted host volumes? I'm guessing that there is a more efficient way to do this, and I'm just not finding an explanation for my particular use-case anywhere.
I am using this container for development of a module for another program, and have no desire to manage a data-only container - the only files that matter are from my host; persistence isn't needed elsewhere (like a database, etc).
Dockerfile
/setup
Created on pastebin to avoid an even longer post. Never expires.
After creating the image, this is the run command I'm using:
docker run -dit \
-p 80:80 \
--name my-container \
-v $(pwd)/components:/var/www/wp-content/plugins/my-plugin-directory/components \
-v $(pwd)/index.php:/var/www/wp-content/plugins/my-plugin-directory/index.php \
my-image
It looks like your chown -R nginx:nginx ... commands inside your container are changing the ownership bits on your files to be owned by libuuid on your host machine.
See Understanding user file ownership in docker: how to avoid changing permissions of linked volumes for a basic explanation on how file ownership bits work between your host and your docker containers.

Resources