Related
I would like to know how I can count the number of columns of an excel sheet using Talend
If you want to count no. of columns in the Excel file then please follow the below instructions.
Use tfileinputExcel component.
Add some basic instruction in the component with limit 1 but important is "Put one name in Schema 'excel_header' (You can use any name) and change its type to 'Dynamic' ". It will fetch the full row from the excel file.
Now Use second component 'tjavaRow'. Link your excel component through the 'main' row to tjavarow component.
Syn your schema in tjavarow component and put below mentioned code.
System.out.println("Counter ::: "+input_row.excel_heade.getColumnCount());
Now your job will count columns for you.
Note: You can store this counter value in any context variable and can use it.
Thank you
tFileInputExcel (just read 1 row i.e. column headers record) ----> tFileOutputDelimited
tFileInputDelimited (read the file created above) --> tNormalize --> tFilterRow_1 (filter null & blank rows) --> tJava_1
In tJava_1 component, you can get the number of rows by using (String)globalMap.get("tFilterRow_1_NB_LINE_OK").
This number of rows actually represent your number of columns in excel.
I need to create a SSIS package that would extract data from an Excel source and load it into a SQL Server Destination.
The Excel file name would have a date, typically the file name would look like emp_20110909.xls where 11 is the Month, 09 is the Day and 09 is the Year. Now I want to capture this date and in the destination table add another column named "Extracted_Date" and populate the captured date for all the records extracted from this excel.
Can anyone tell me how to do that process?
Excel as a data source offers no explicit functionality for this whereas the Flat File Source does. I blogged about this under What is the name of a file
What you're looking to do is have a Foreach File Enumerator look in a folder for your Excel file(s). Assign the value of the currently found file to a variable like #[User::CurrentFileName]. That would look something like C:\ssisdata\mySource\Input\emp_110909.xls
You would update the Excel Connection Manager to have an expression on the ExcelFilePath property so now as the value of #[User::CurrentFileName] changes, so does the actual referenced file. You can find plenty of references to using the foreach enumerator on the web or search my answers
The last bit you need is to parse the value of CurrentFileName to find the year
(11), month (09) and day (09) elements - or maybe you want it as one big value (110909). For this, I would create 4 variables: FileDate, FileYear, FileMonth, FileDay all as string. Yes, they're numbers but for our usage, treating them as string is going to be easier.
FileDate will correspond to everything between the underscore following emp up until the period of xls. We're going to use the Expression language of SSIS to do this and the particular elements will be SUBSTRING, FINDSTRING and LEN
SUBSTRING(#[User::CurrentFileName], FINDSTRING(#[User::CurrentFileName], "emp_", 1) + LEN("emp_"), 6)
Here, I was lazy and just "knew" the length was 6 and hardcoded as such. In the event that someone gives us a emp_20110909.xls this will fail. The preceding expression would be modified by finding the position of the period and then calculating the length from the emp_ position.
Now that we know FileDate, we can use SUBSTRING to slice out the first 2 elements for year, next 2 for month and final two for day.
You can then inject those values into your Data Flow via a Derived Task or push into an audit table via Execute SQL Task.
I have an Excel file with multiple sheets, but only the first sheet has the date listed on it. What I was trying to do was read the excel file, put it into a variable to be able to be used later on in a data flow task.
Normally it would be your run of the mill read and write data flow task, but since this information lies in the first page of the excel sheet with just mostly information about the report, it makes reading the information a bit more difficult.
Here's what the sheet looks like, and the only information that I was wanting from this whole sheet was on the Data Period line more specifically, Dec 2016
Any direction would be greatly appreciated, thank you.
Excel sheets can be queried like tables. You can use an Execute SQL Task to read a range of cells iterate over the results, or you can read a single cell as if it were a range and store its value in a variable.
The process is described in Read Excel Value in SSIS and contains quite a few gotchas :
Add an Excel conneciton manager that points to your Excel file
Set its result type to Single Row.
Set the query to SELECT * FROM [Sheet1$A6:A6]. That's the first gotcha. You can't specify column names. In a dataflow query you can write SELECT RIGHT(F1,8) FROM
[Sheet1$A6:A6] to extract only the date part. This doesn't work in
the Execute SQL Task.
In the Result Set section, map the 0 result set to a new string variable, eg PeriodCell. The name has to be 0. That's the second gotcha
You can create another variable based on an expression that returns only the 8 rightmost characters of PeriodCell, eg RIGHT( #[User::PeriodCell],8)
You can parse the string directly into a date if your system uses an English locale. In this case, you could create a DateTime variable with an expression (DT_DATE)RIGHT( #[User::PeriodCell],8). For example, (DT_DATE)"Dec 2016" returns 1/1/2016
Unfortunately, this won't work if your locale is not English, even if you change the package's Locale property.
If you have load the content of the cells of the "Data Period" column into a SQL table with SSIS, you can easilty convert them from excel date format to SQL date format usign one of the following:
Date and time
select dateadd(second, (#time_xls - ROUND(#time_xls,0))*86400, dateadd(d, ROUND(#time_xls,0),'1899-12-30'))
For example the value 42853.4673611111 is converted into "2017-04-28 11:13:00.000"
Only date
select dateadd(d,#time_xls,'1899-12-30')
For example the value 36464 is converted into "1999-10-31 00:00:00.000".
I have been successfully using a linked server with SQL Server Management Studio importing a file from Excel which has four columns.
The Excel document looks like (no TOOL means blank cell, rows 6-199)
TDS HOLDER TOOL
1 3 1187
2 4 09812
3 5 9082
4 2 ----
5 76 ----
6 9
7 1
. .
. .
. .
200 18 CT-2989
201 98 CT-9871
When I import it as is, it will grab the cells with the numbers at the top, cells that contain ------ and then when it gets to the cells which are blank it will then print NULL for the rest of the data, which is incorrect.
When I alter my Excel document so that the 'CT' values are at the top, it will grab all of the proper CT and TL values in column 3.
The problem is with the SQl Server Import and Export wizard. It uses the data in the top few rows of the spreadsheet to decide on the data types in each column. When your Tools column has numbers at the top the wizard decides the data type of the column is float. When the column had "CT-2989" at the top it chooses a char type. Once it has chosen the float type it will ignore CT-2989 because it isn't convertable to a floating point number. The simplest solution to the problem it to arrange your Excel spreadsheet with a dummy row at the very top which gives the wizard the proper types for each column. For example, make the first data cell in the Tools column "abcdefg", assuming the rest of the data in that column consists of up to 7 alphanumeric characters. Once your data has imported into SSMS, delete the dummy row.
If you go to the "Review DataType Mapping" page of the wizard it will show that the Tools column has been detected as containing float data when the numeric data is at the top of the spreadsheet. Note that the even if the destination type for the Tools column is nvarchar, the wizard makes it's own decisions regarding source type.
There are other solutions using openrowset() and SSIS, but this one is quick and simple.
Here the problem is with OLEDB which is unable to handel mixed data(numbers + text) so there is no solution only a few hacks some of them already mentioned above I just want to add a few more:-
1) In excel sheet Keep the data consistent and maintain distinct coloumns for each depending upon its data type e.g. text,numeric or whole numbers, fractional numbers etc.
2) Ok before importing it break down the sheet in multiple sheets based on its datatype so that OLEDB won't get confiused.
3) Import the excel sheet in MS Access so that it all the data would get a data type then import it into SQL this would handle NULL too very wisely.
Save the worksheet as .CSV and import as a flat file from task, when reviewing the data, unchecked the Datatypes indicator.
I am using pear package with PHP writing data into Excel sheet from mysql.
In mysql table I am having a column with named EAN. Its value is a 13 digit number.
When I try to run the program remaining all columns data is displaying correctly,
but this 14 digit number is displaying like a scientific notation 4.04623E+12.
Original number is 4046228004518. I am using the below line of code.
$worksheet->write($i, 2, $sel_row['EAN']);
Try writing an apostrophe (') before value. It will force Excel to display value as text and thus not convert it to scientific notation.
I had the same issue with spring & apache poi.
This solution worked for me -
XSSFCellStyle numeric = (XSSFCellStyle) wb.createCellStyle();
numeric.setDataFormat(workbook.createDataFormat().getFormat("0"));
cell.setCellValue([large number]); //4046228004518
cell.setCellStyle(numeric);