We are facing a peculiar problem on one of our 2 environments. A PutFile processor throws the following error
PutFile[id=xxx] Penalizing StandardFlowFileRecord[uuid=xxx,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=xxx, container=default, section=1012], offset=94495, length=9778],offset=0,name=xxxxxxxxxxxxxxxx_PROD_20200513020001.json.gz,size=9778] and routing to 'failure' because the output directory /data/home/datadelivery/OUT/Test does not exist and Processor is configured not to create missing directories
After enabling the creation of missing directories, the error changes to:
Could not set create directory with permissions 664 because /data/home/datadelivery/OUT/Test: java.nio.file.AccessDeniedException: /data/home/datadelivery/OUT/TestPutFile[id=xxx...
Based on the error message one would think that it is an issue with file and folder permissions, however, the path /data/home/datadelivery/OUT/Test exists, and the nifi user can access and create files and folders in there as well (verified from the command line). The same folder permissions and ownership rights are configured on our DEV environment, where the PutFile processor works as expected. We could change the configuration to use a different location, but I'd rather find the root cause instead.
Where should I start debugging?
Thank you for your help in advance!
Kind regards, Julius
Strange issue, I would try to set full permission on the folder/file you want to write (ie chmod 777 + chown nifi:nifi + recursively), and see if the error is still there. If not it's kind of a start ...
Restarting the NiFi service solved the problem. The issue was that the Unix user (nifi) was modified months after starting the NiFi service. Most probably this was the reason the PutFile processor wasn't able to access a folder which the nifi unix user could.
Related
I am running a default t2.nano ec2 linux ami. Nothing is changed on it. I am trying to rsync my local changes to the server. There is a permissions issue that I don't know enough about to fix.
My structure is as follows. I'm trying to push my work to the technology directory. The technology directory is mapped to a staging domain. i.e. technology.staging.com
:/var/www/html/technology
this is from the root, and it does work fine, it's the rsync that is failing.
when I push locally to that directory I get a "failed: Permission denied (13)" error.
I'm running an nginx server and assigned permissions to the www directory as follows:
sudo chown -R nginx:nginx /var/www
My user is ec2-user which is the normal default. Here is where I am tripped up. You can see the var directory is given root access.
You can see that the www directory then has permissions set to nginx so our server can access the files. I believe I need to add the ec2-user to this directory as well as the nginx user so that I can rsync my files there and the server will still have access I'm just unsure of how to do that.
As a test, I created a test directory at this location and it worked successfully.
:/home/ec2-user/test
you can see the permission here are set for the ec2-user which is why it works i'm sure.
Here's the command I'm running on my local machine to rsync my files which fails.
rsync -azP -e "ssh -i /Users/username/devwork/company/comp.pem" company_technology/ ec2-user#1.2.3.4:/var/www/html/technology
Here's the command that was working.
rsync -azP -e "ssh -i /Users/username/devwork/company/comp.pem" company_technology/ ec2-user#1.2.3.4:/home/ec2-user/test
I have done enough research and testing to know that it's a permissions error, I just can't figure out the right way to solve it. Do I need to create a group and assign both the nginx and ec2-user to the group and then give that group the same permissions level on the :/var directory.
Side note, what permissions level do I set for the chown to make these permissions that are currently set?
I have server config files in the :/etc/nginx/conf.d/ directory that map to the directories I create inside of :/var/www/html directory so I can have multiple sites hosted on the server.
So in this example, I have a config file at :/etc/nginx/conf.d/technology.conf which maps to the directory at :/var/www/html/technology
Thank you in advance, again, I do feel like I have put forth the research and effort to show that I've gone as far as I know how to do.
The answer made sense after I spent roughly a day playing around. You have to give access to both the ec2-user and the nginx group. I believe you never want to put a user in a group that involves the server itself, I think things would go south.
After changing the owner to both the ec2-user and nginx group, it still didn't work exactly the way I wanted it to. The reason was, I needed the nginx permissions to be updated to what they had when they were assigned the user role.
Basically, theec2-user had write permissions and the server did not. we wanted the user to have write permissions so they could rsync my local files to the directory on the server, and the nginx group needed the same level of permissions to display the pages. Now that I think about it, the nginx group may have only needed read permissions to display things, but this at least solved the problem for now.
Here is the command I ran on the server to update the ownership and the permissions, as well as the output.
modify ownership
sudo chown -R ec2-user:nginx :/var/www/html/technology
modify permissions
sudo chmod -R o=rwx,g+rwx,o-w technology
The end result looks like this
You can see the permissions match, and the ownership is as we expected. The only thing I have to figure out is after I rsync new files to the server, I need to run the previous code to update the permissions again. I'm sure that will come to me later, but I hope this helps anyone in the same situation.
I'm using Apache 2.4 on Ubuntu 16.04 and everyday I have the same problem on my Laravel 5.5 application:
"The stream or file "myapp/storage/logs/laravel-date.log" could not be opened: failed to open stream: Permission denied"
And I see a lot of people have had the same issue, and the fix is usually changing permissions and ownership.
My problem is that actually two different users need access to create and write logs: ubuntu (I'm using AWS) and www-data (Apache user).
So if I change ownership to www-data whenever I try to run an artisan command I get the error, and if I change it to ubuntu I get the same issue whenever apache wants to log an error or something.
I have tried to make ubuntu part of the www-data group but that doesn't seem to fix the issue, since whenever a new log file is created with the following permissions
-rw-rw-r-
Which I think it's what gives me the issue.
So, any help? Thanks in advance
I have a freeradius server setup with google authenticator to provide a basic working multi-factor setup.
Everything works when I run radiusd in debug mode as root. If I start it as a service, logons fail and this messages is recorded when processing messages:
radiusd(pam_google_authenticator)[1115]: Failed to read "/home/user#domain.com/.google_authenticator" for "user#domain.com"
I think this must be a permissions issue since it works fine when run as root.
I don't really want to edit the permissions on each secret file for every user.
I have tried specifying root in
/etc/raddb/radiusd.conf
user = root group = root
but still the service fails unless run from the command line as root. Does anyone have a nice elegant solution to this conundrum?
I think you should check out your systemd service file for radiusd. It might look something like:
https://github.com/ipfire/ipfire-3.x/blob/master/freeradius/systemd/freeradius.service
You can add User= and Group= in the [Service] section of the .service file if needed. See
https://unix.stackexchange.com/questions/347358/how-to-change-service-user-in-centos-7
and
https://serverfault.com/questions/806617/configuring-systemd-service-to-run-with-root-access
It would be a good idea to put the contents of the .service file for radiusd in your post.
I have a dedicated server running Linux/Apache with cPanel. On one of my website's users are able to register and upload images.
However, over the last few days, i have received complaints that users cannot upload any media. When I check the /images/ directory I see that the /temp/ folder is missing.
So I re-created it and set the permissions to 0777.
Once that is completed all works fine. But I cannot understand why the directory is being removed. No one else has access to the site and I do not see anything in my error logs that show anything unusual.
Any ideas why this may happen?
Thank you
You need to check your PHP Script has code to unlink(dir) or Something
and Check cron list whether you have added any CRON to remove the TMP folder.
for Write permission 755 is enough If you are using Fcgi php handler, cgi, dso. Have a try 755 permission with one of the listed Handlers.
I have a cloud hosting linux solution. I had vsftpd working on it, but after having issues and tinkering with a lot of settings, I now have an issue where users can login using FTP and connect to the correct home directory, navigate within it, download files but they cannot upload files to the server. They get a time out error, which appears to be a permissions error, but I can't narrow it down any more than that. /var/logs/syslog gives nothing away.
The folders belong to the users. The parent www folder is set to 555. Can anyone help with this issue at all?
Cheers,
T
Try to set the permissions to 755, 555 doesn't allow writing for anyone. Are your user and group different?
You also may need to enable logging for FTP server. The time out error may include some other errors, not only permission denied.
To have extended logging change the variables in your ftp config file:
dual_log_enable=YES
log_ftp_protocol=YES
xferlog_enable=YES
syslog_enable=NO
and check the log file name there.
you must create a folder into user folder (Example : /var/www/user1/upload).
and set permission 777 (Example : chmod 777 /var/www/user1/upload).
then upload file into this folder.