I have existing Hadoop setup with Hive, a new user on Linux node is unable to create the table giving below exception:
(message:Got exception: org.apache.hadoop.security.AccessControlException Permission denied: user=
access=WRITE, inode="/bigdata/hivetmp/user/hive/warehouse/gpeh.db":hadoop1:supergroup:drwxr-xr-x
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.checkFsPermission(FSPermissionChecker.java:271)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:257)
at org.apache.hadoop.hdfs.server.namenode.FSPermissionChecker.check(FSPermissionChecker.java:238)
However, with same user I am able to create the database.
I have provided necessary access to the user on required folders.
Its just a permission issue.
If you see at the HDFS level
inode="/bigdata/hivetmp/user/hive/warehouse/gpeh.db" :hadoop1:supergroup:drwxr-xr-x at
That means the new user doesnt have write permission at the database.
You can just change it to give the new user write permission and try again.
Related
hope you can help me with a concern about an error with awswrangler.
this is the case: i have 2 aws accounts, AccountA and AccountB, both with lakeformation enabled, i have a set of databases in AccA and another set in AccB, so we share AccountB databases to AccountA through lakeformation so we can query their Db/tables with Athena in AccountA.
i am trying to automate a sql query with python, so i'm using awswrangler to achieve this, but i'm getting a not very specific error when in run the query in python.
when i run "select * from DatabaseAccB.Table" get this error "HIVE_METASTORE_ERROR: Table is missing storage descriptor" what could be the cause? i tried with boto3.Athena session and same result.
this may should help, when i query select * from DatabaseAccB.Table with my user, this runs fine. but when i try to do it with lambda or glue job, fails with error mentioned before.
PD: AccountA has only select/describe permission on tables in AccountB. Can show some code if you need.
PD2: if run "select * from DatabaseAccA.Table" query runs fine
tried with Boto 3, same result.
Tried using lambda, same result.
Tried giving admin access to glue role in AccountA, same result.
I think that there something happening with Lakeformation.
Thanks!
Make sure your Lambda/Glue Job execution Roles have the following Lake Formation permissions, all granted from AccountA's Console/CLI:
DESCRIBE on Resource Links (AccountA's Glue Catalog);
SELECT, DROP, etc, on the Shared DB/Table (AccountB's Glue Catalog);
Resource Link permissions must be granted in pairs: even though your queries point to a Resource Link the Principal executing the query in Athena/Redshift Spectrum still needs to have "normal" (SELECT, INSERT, etc) permissions on the underlying shared Database/Table granted by AccountA's Lake Formation Administrator.
For the AWS Wrangler part, if the problem still persists, maybe you'll need to be explicit on which Glue Catalog ID it'll execute the query upon (at the moment I'm not sure if this parameter exists in AWS Wrangler though).
I am trying to run paragraphs using zeppelin spark object method
z.run("noteId","paragraphId")
z.run("paragraphId")
I have tried both the methods, but nothing seems to be triggering the next paragraph. I do not get any errors also. Please let me know if I am missing something here.
On checking the zeppelin server logs I am getting the below error :
ERROR [2020-08-26 15:46:37,742] ({Thread-35} RemoteInterpreterEventPoller.java[run]:250) - Can't handle event RemoteInterpreterEvent(type:RUN_INTERPRETER_CONTEXT_RUNNER, data:{"logger":{"traceCapable":true,"name":"org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$ParagraphRunner"},"noteId":"2FHDFMR16","paragraphId":"20200819-131115_701985359"})
org.apache.zeppelin.rest.exception.ForbiddenException: HTTP 403 Forbidden
at org.apache.zeppelin.socket.NotebookServer.onRemoteRunParagraph(NotebookServer.java:2153)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterEventPoller.run(RemoteInterpreterEventPoller.java:141)
I have created my notebook using the admin user only. Do we need to provide any special access? I am just using admin user.
Update 1:
I changed notebook settings in which we removed user and owner permissions. It's triggering, but it's with an anonymous user. It will be a problem if we are working in multi-user environment.
I think instead of that
Try
z.z.run
I'm using VBA and sending a query to an ADODB, the Conn.Execute method.
How do I retrieve the messages? As an example:
ORA-00942: table or view does not exist
or
1,670 rows inserted.
or
Error starting at line : 19 in command -
TRUNCATE TABLE SYSBO.GA_OPS_TMP_ORDERS
Error report -
ORA-01031: insufficient privileges
01031. 00000 - "insufficient privileges"
*Cause: An attempt was made to perform a database operation without
the necessary privileges.
*Action: Ask your database administrator or designated security
administrator to grant you the necessary privileges
How can I resolve this error:
Msg 5123, Level 16, State 1, Line 1 CREATE FILE encountered operating system error 5(Access is denied.)
while attempting to open or create the physical file 'C:\Banking.mdf'.
You need to ensure that you have the appropriate account permissions on the account in order to access the folder the mdf file is located in.
More detail can be found in the following link.
https://dba.stackexchange.com/questions/22250/sql-server-create-file-encountered-operating-system-error-5-access-is-denied
I encountered the same error when I called 'update-database' the first time I tried to run Entity Framework Core / EF Core.
I solved it!
The full error message was:
CREATE FILE encountered operating system error 5(Access is denied.) while attempting to open or create the physical file 'C:\Users\peterObjectData.mdf'.
Creating files in C:\Users\ directory requires admin rights.
To fix this, one can add a file location and name for the mdf file in the connection string:
optionsBuilder.UseSqlServer("Server = (localdb)\mssqllocaldb; Database = ObjectData; Trusted_Connection = True; AttachDbFileName=D:\Databases\ObjectDB.mdf");
Here, "D:\Databases\ObjectDB.mdf" is the location and filename of the .mdf file created. This fixed my issue.
If you want to read more on ConnectionStrings and AttachDbFileName, here's the documentation:
https://learn.microsoft.com/en-us/dotnet/api/system.data.sqlclient.sqlconnection.connectionstring?view=netframework-4.7.2#examples
That must be your C drive denied the SQL server to create file. You can either change the direction to other drive(D:;E:) or if you dont want to, then you have to change the security properties for your C:\ drive. Right click C:, choose properties, security, change authenticate user and user to full control then you good
I would like to create a view in SQL Server 2008 that displays the data contained in an Excel file.
I do not want to use the import data as these data are updated.
I found this: http://support.microsoft.com/kb/321686/en
I made these commands to enable some options :
sp_configure 'show advanced options', 1
RECONFIGURE
GO
sp_configure 'ad hoc distributed queries', 1
RECONFIGURE
GO
But when I try to read the Excel file with this query:
SELECT * FROM INTO XLImport3 OPENDATASOURCE ('Microsoft.Jet.OLEDB.4.0', 'Data Source=C:\Documents and Settings\jconnor\Desktop\Test.xlsx') ... [Sheet1$]
I back (it's a translation from french) :
The OLE DB provider " Microsoft.Jet.OLEDB.4.0 " for linked server " (null) " returned >message " Unspecified error " .
Msg 7303 , Level 16 , State 1, Line 1
Unable to initialize the object data source OLE DB provider " Microsoft.Jet.OLEDB.4.0 " for >linked server " (null) " .
Does someone have any clue ?
Thank you in advance
Kal,
So your main error is likely this;
OLE DB provider "MICROSOFT.JET.OLEDB.4.0" for linked server "(null)" returned message "Unspecified error".
Msg 7303, Level 16, State 1, Line 1
Cannot initialize the data source object of OLE DB provider "MICROSOFT.JET.OLEDB.4.0" for linked server "(null)".
I would check some permissions.
Check the permissions on the Temp folder
This is needed because the provider uses the temp folder while retrieving the data. The folder can be one of the below based on whether you use a local system account or network domain account.
For network accounts, folder is
\Windows\ServiceProfiles\NetworkService\AppData\Local\Temp
and for local system account its \Windows\ServiceProfiles\LocalService\AppData\Local\Temp
Right click on this folder and give it read write access to the account (or group) executing the code. That solved the error for me.
Also
http://blogs.msdn.com/b/spike/archive/2008/07/23/ole-db-provider-microsoft-jet-oledb-4-0-for-linked-server-null-returned-message-unspecified-error.aspx
This is because the SQL Server Service is trying to write the temp DSN to the temp folder for the login that started the service, in this case the Admin/Admin login.
The temp folder is something like: C:\Documents and Settings\Admin\Local Settings\Temp
.15 As mentioned, the OleDbProvider will always execute in the context of the user who initialized it, in this case User/User.
.16 User/User has no rights on this folder (C:\Documents and Settings\Admin\Local Settings\Temp).
If running FileMon when the SQL is executed, we can see the following:
(Actually, try using Process Monitor - http://technet.microsoft.com/en-us/sysinternals/bb896645)
sqlservr.exe:000 QUERY INFORMATION C:\DOCUME~1\Admini~1\LOCALS~1\Temp ACCESS DENIED Attributes: Error
So to summarize so far:
The SQL Server service is started as Admin/Admin, when the select is made, the OleDb provider is invoked by User/User.
Now the OleDb provider attempts to create a temporary DSN in the temp directory. This will be the temp directory for the SQL Server service (Admin/Admin)
but the user (in this case User/User) does not have write permissions on this folder. And the error occurs.
There are two ways to resolve this.
Option 1:
Log out of the machine and log in as the account that starts the SQL Server Service (in this case Admin/Admin) then start a command prompt
and type “set t” (no quotes), this will show something like:
TEMP=C:\DOCUME~1\Admin\LOCALS~1\Temp
TMP=C:\DOCUME~1\Admin\LOCALS~1\Temp
these are the environment variables set for %TEMP% and %TMP%, so go to that folder and right click and select Properties -> Security,
then add the user, in this case User/User, note that the default for the user is Read&Execute/List Folder Content/Read, this not enough, you have to select Write as well.
Log out, and log in again as User/User and rerun the command from SSMS. This time it should work.
Option 2:
Log on as Admin and change the TEMP and TMP variable to, for example, C:\Temp, basically this moves the Temp directory out of the Documents and Settings folder.
However, you must restart the SQL server for this to take effect.
So basically, what happens is that when SQL Server starts, it uses the Temp folder of the startup account (Admin/Admin) but the MICROSOFT.JET.OLEDB.4.0 will always execute
as the user who calls the SQL command (User/User) and this will fail unless this user does not have Write access to that temp folder.
Without knowing all setups out there, perhaps option 2 is the preferred solution since with option 1, you will have to add ALL the users that will invoke the provider which may not be practical.
Also, when changing the startup account for the SQL Server service, then the TEMP directory for that account will be used, and you will see the error again until you, again, give write permissions for all the users on this TEMP folder...or a user group (preferred).