Hi im developping an ssis package that imports excel files (.xlsx) from an ftp server to a local folder then they are imported to a sql server table . I'm using a foreach mapping to the name of files. The import from the ftp server to local work fine, but the import from the local folder to the sql table failed.
It seems that I have a problem in excel source. These are the errors:
Start SSIS package "Package.dtsx."
Information: 0x1 at Script Task, C # My Message: System.Collections.ArrayList
Information: 0x4004300A at Data Flow Task, SSIS.Pipeline: Validation phase begins.
Error: 0xC0202009 at Data Flow Task, Excel Source [1]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E37.
Error: 0xC02020E8 to Flow Task data, Excel Source [1]: Failed to open a rowset for "Sheet1 $". Verify that the object exists in the database.
Error: 0xC004706B to Flow Task data SSIS.Pipeline: validation failed "component" Excel Source "(1)". Returned validation status "VS_ISBROKEN."
Error: 0xC004700C to Flow Task data SSIS.Pipeline: Failed to validate one or more components.
Error: 0xC0024107 at Data Flow Task: There were errors during task validation.
Warning: 0x80019002 at Foreach Loop Container: SSIS Warning Code DTS_W_MAXIMUMERRORCOUNTREACHED. The Execution method succeeded, but the number of errors (6) reached the maximum allowed (1); leading to a failure. This occurs when the number of errors reaches the number specified in MaximumErrorCount. Change the value of MaximumErrorCount or fix the errors.
SSIS package "Package.dtsx" finished: Success.
The program '[5504] Package.dtsx: DTS' has exited with code 0 (0x0).
As configuration I have:
For the excel manager connexion, I made an expression for connectionString = #[User::variable1] + #[User::DOWNLOAD_DIRECTORY_LOCAL] + #[User::FTP_FILE_URL] + #[User::variable2]
variable 1 =Provider=Microsoft.ACE.OLEDB.12.0;Data Source=
variable 2 = ;Extended Properties="EXCEL 12.0;HDR=YES";
I made also the delay validation property to true for data flow task, ftp task, foreach task and excel connection.
I just wrote a package to do the very same thing myself. Things to check in this order:
in your Excel Data Connection have you browsed to the excel files in your local folder (once they are there) and selected one (you need to copy one in there while developing)? so when you go to your excel source object inside your Data Flow Task (inside the For Each) you can select the Excel Data Connection and then see Sheet$1 under "name of the excel sheet"?
Once you are sure you have done above have you then right-clicked on the Excel Data Connection and in the Expressions property added ExcelFilePath = #[User::FTP_FILE_URL]? (note you need to select 'Fully Qualified' under Retrieve File Name on the Collection tab of the For Each container)
in your Excel Data Connection have you selected the right version (Excel 2007) for the .xlsx files or Excel 2003 for .xls? I noticed a small bug where when I changed the filename it defaulted back to 2007, I had to manually change it back (again) to 2003.
Check at least one workbook exists in the folder before the step runs. There is some code around here about how to add a script task to validate at least one file being in User::DOWNLOAD_DIRECTORY_LOCAL.
I got a load of errors about the driver for Microsoft.ACE.OLEDB.12.0, plus had issues with a 64-bit server and had to wrap the package in a job and check the 'use 32-bit runtime' option under execution options in the job properties. Check the driver is working OK (although it usually gives a specific driver error if you haven't got it set up right).
Um that's it offhand just quickly before I head home. Let me know if it works or is still a fail..
The question you should ask yourself when having this issue is:
Where do I run my dstx file from ?
Is it from Microsoft Visual Studio ?
Is it from a SQL Agent ?
Is it from the Integration Services Package Execution Utiliy ?
Then refine your question to find the answer on the forums.
Related
Background:
I have an Excel file that is a workflow tool for one of our teams. One of the key processes is importing data from csv files and then uploading into MS Access. This part of the process works fine every time, no errors.
I added a new process to the tool, similar to the one mentioned above, this process scans data from one sheet and if a row meets a criteria it is copied into another sheet. Once the analysis is complete it gets uploaded into MS Access using the following code:
objAccess.DoCmd.TransferSpreadsheet acImport, acSpreadsheetTypeExcel9, "tb_age_extract", Application.ActiveWorkbook.FullName, True, "AGEUPLOADDATA"
The strange thing is this will run ok for me but when the users try and run it they get a runtime error 3051, the full error message is:
The Microsoft Access database engine cannot open or write to the file '\Folder1\Folder 2\FileName.xlsm'. It is already opened exclusively by another user, or you need permission to view and write its data.
I ran a test using a 50 count loop, all 50 runs ran ok with no errors, then the user tries to run it and they get the error. I have also experienced the error as well on my machine.
What is puzzling is the error is sporadic.
Has anyone experienced something similar with a 3051 error?
We are currently facing an issue with Excel Source component. We have to read mutipples Excel files with same structure and insert their data into one SQL table.
We use ForEach container that read all the files in the specified folder (with variables) and transfer data to the SQL table. On the first file (whiwh was used for the Excel Source configuration), there is a column with more than 4000 chars (5036 exacly). Excel driver automatically identified it as DT_NTEXT (and it's ok) and the auto-created SQL table has NVARCHAR(MAX) as type.
On the fifth file, this column doesn't contains data at all. And when trying to transfer data, the package fails with the error below
[Excel Source [14]] Error: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E21.
An OLE DB record is available. Source: "Microsoft Access Database Engine" Hresult: 0x80040E21 Description: "Multiple-step OLE DB operation generated errors. Check each OLE DB status value, if available. No work was done.".
[Excel Source [14]] Error: Failed to retrieve long data for column "Comment".
[Excel Source [14]] Error: There was an error with Excel Source.Outputs[Excel Source Output] on Excel Source. The column status returned was: "DBSTATUS_UNAVAILABLE".
[Excel Source [14]] Error: SSIS Error Code DTS_E_INDUCEDTRANSFORMFAILUREONERROR. The "Excel Source.Outputs[Excel Source Output]" failed because error code 0xC0209071 occurred, and the error row disposition on "Excel Source" specifies failure on error. An error occurred on the specified object of the specified component. There may be error messages posted before this with more information about the failure.
[SSIS.Pipeline] Error: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on Excel Source returned error code 0xC0209029. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure.
I have made some search on the internet, no solution worked for me.
I tried:
Add IMEX=1 at the end of the Excel ConnectionString
Update the TypeGuessRows value on the registry key for the JET driver
Changed output type on Advanced Editor
and others solutions maybe, but I do not remember them all
Any idea on how to solve this?
Thanks
I'm trying to use the Power Query source component in a generic way from SSIS (VS2019).
The idea would be to use a for each loop to load and transform Excel files. At run time, I need to set the connection manager properties for each file as well as the PQY script to be executed on the file.
What I did so far is trying to create a JSON connection string inside a script component and assign the connection string to the connection manager. It keeps on saying that the file requires credentials.
Would someone already experienced that kind of dev? All the files do have the same structure so far, do meta-data need to be refreshed too?
[Edit]
1. In the control flow, I'm retrieving the PQY script I want to apply from a DB.
Before transormations, script starts like this:
let Source = Excel.Workbook(File.Contents("path_to_a_file.xlsx"),null,true),RawData_Sheet = Source{[Item="Table1",Kind="Table"]}[Data]..."
In the C# script task, I'm replacing the path to excel file by the current file variable. M Script is stored in a variable used in the PQY component.
C# Script is then updating the PQY connection manager to target the appropriate file:
ConnectionManager _conn = Dts.Connections["Power Query Connection Manager"];
String _ConnectString = "[{kind:File,path:path_to_a_file.xlss,AuthenticationKind:Windows,Username:myusername,Password:mypassword}]";
_conn.ConnectionString = _ConnectString;
The PQY component is left has it is, connected to ["Power Query Connection Manager"] and getting its script from the variable I set.
PQY configuration screen
Thanks for any tip on this,
Olivier
I can't address the specifics of the PQ but generic anything in a Data Flow will not work.
The Data Flow task works because it makes a strict contract between the source(s) and the destination(s). These columns with these data types will be in play during the run. It's a design-time contract because that allows the run-time engine to allocate resources based on how many buffers of data the system can support. Each row is X bytes, we have Y bytes of memory available, so Z buffers worth of data plus parallelism stuff.
Wish I had a better story to tell you.
I have an Excel Macro Workbook (binary extension) that suddenly has developed 2 issues occasionally (sometimes it runs fine).
#1 Issue - Occasionally there is an error on Line 5
It appears the connection does not open. We get the error:
Microsoft][ODBC Microsoft Access Driver]General error Unable to open registry key Temporary (volatile) Ace DSN for process 0x7f8 Thread 0x1174 DBC
#2 Issue - Occasionally there is an error on Line 7
It appears the SQL can not run. It indicates the Worksheet tab can not be found even though it does exist. Note for this error we have no issue with Line 5 (Open Connection). The error that displays is:
[Microsoft][ODBC Excel Driver] The microsoft jet database engine could not find the object 'Worksheet Name'. Make sure the object exists...
For Issue #2 once we close the error dialog box and THE MACRO GETS DELETED !!!
Notes
The code module has not been changed in years Some days we get no errors/issues
This issue occurs for multiple Excel workbooks run by multiple people that run different VBA code blocks. However 1 of the code blocks is the same across all these macros which is the code snippet above.
The source data has not changed in years in terms of formatting or volume.
All these workbooks are on a Network server.
We are on Windows7/Excel 2010/ODBC Excel Driver (xls xlsx xlsm xlsb)
14.00.7180.5000
Code Snippet
1-Set adoConn = Nothing
2-Set adoRS = Nothing
3-Application.EnableCancelKey = xldisabled
4-strConn = "Driver={Microsoft Excel Driver (*.xls,*.xlsx,*.xlsm,*.xlsb)};DriverId=1046;Dbq=" & strFileName & ";IMEX=1;"
5-Set adoConn = New ADODB.Connection: adoConn.Open strConn
6-Application.EnableCancelKey = xlInterrupt
7-Set adoRS = New ADODB.Recordset:adoRS.Open Source:=strSQL,ActiveConnection:=adoConn
I'm curious as to why you're using ODBC to connect Excel to another Excel file, as opposed as to linking the data directly.
Intermittent connection problems point to a network issue. I would suggest removing and re-creating the ODBC connection, but I'd first want to confirm the value of strFileName as referred to in the connection string. Is it an actual server on your internal network, or an external source? (ie, is it a UNC path?)
An intermittent Unable to open registry key error likely sounds worse than it is; when having connection problems, Windows & Excel will easily get confused as to what the real problem is.
It sounds like your setup has been in place, possibly un-maintained, for a long time. When is the last time the client and server were rebooted? (Should be nightly.) Defrag, disk cleanup, etc?
As for the issue of the macro "getting deleted" when you close an error dialog, I'd need to see proof of that. I think you're misunderstanding something, unless you're re-writing the macro from scratch each time this happens..
When configuring an Excel Data Flow Component (Source) using the AccessMode of OpenRowset from Variable and setting the correct variable, I get the following error when trying to configure the shape/preview the data:
TITLE: Microsoft Visual Studio
Exception from HRESULT: 0xC02020E8
Error at Merge Files [Excel Source [2]]: SSIS Error Code DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code: 0x80040E37.
Error at Merge Files [Excel Source [2]]: Opening a rowset for "[WORKSHEET]" failed. Check that the object exists in the database.
I have checked that variable name is correctly set to the name of a valid worksheet.
How do I fix this error?
You need to append a dollar sign on the end of the worksheet variable, e.g. WORKSHEET$