I am using the Toad Automation Designer to export data of table to Excel.
Unfortunately my table contains more then 65000 rows (100k) and every single Excel file
can only contain 65k entries.
My workaround was to write two SQL statements to create two Excel files
select * from my_table offset 0 rows FETCH NEXT 65000 ROWS ONLY
select * from my_table offset 64999 rows
That is ok for the moment but I am looking for a more dynamic way to export the whole
table into several Excel files without writing multiple SQL statement because in future
I will have maybe 300k entries in my database table.
So I am looking for a possibility to write something like a little script in the
Automation Designer or something similar.
Any Ideas?
Had similar issue: Toad .xls download populates one tab (64K rows). Only difference is that my Toad (11) would then populate another tab with residual data (the next 64K rows), then another tab and so on until all data was downloaded... it sometimes took many tabs.
I changed the download file type to .xlsx and then all the data downloaded to one tab.
safety tip: After download, Excel will have no problem opening the downloaded xlsx - but I have to open/save/close the downloaded .xlsx in Excel prior to using an ms access import spec. (else access' import spec complains the file's unreadable).
Related
How to Export the Data/ Table containing more than 100k records) From an Access Database to an Excel format. I tried by unchecking export data with formatting and layout option in .xlsx file format. But i can see that some data is missing in some records in the excel file.
Is there any other way to export the entire 100k records to excel without missing any data.?
Option 1: Open the Access data table hit CTRL-C open Excel and hit CTRL-V. Your computer may be 'not responding' for a few minutes but worth a shot
Option 2: Open Excel > Data > (import)From Access (top left of window). Navigate to your database/table/access file and follow the prompts. Unfortunately i cant help you beyond this as I don't know your file structure
Option 3 Export from Access into Excel. I found this article particularly useful, especially the part highlighting 'troubleshooting missing or incorrect values'
I have an Excel 2016 with 30 graphs based on PowerPivot. PowerPivot fetches the data from another Excel sheet, but I want it to get the data from a SQL server table instead.
How can I change the data source type in PowerPivot? I've tried looking in the Excel xml without any luck. Would be a lot of work re-creating all graphs over again just to switch data source
Thanks
Dennis
One suggestion I would make for the future, if all the users are using 2016 is to use Power Query which comes standard with that version of excel. In the Power Query loading data into Power Pivot scenario, all Power Pivot cares about is the column names. This means that the query can be changed between data source types without causing issues, as long as the same column names are changed.
As an example, I have one file that based on a parameter flag rips data out of a series of excel files on a shared network drive or Share Point. Both of which would be different data sources. The first opening a folder as the data source, then excel files listed within the folder. The other opening a share point list as its data source, then navigating though excel files.
I am trying to open a file that i exported from TOAD which has 1.4 million records. It has 4 columns.
I tried opening the file in notepad++, deleting half records there and copying them into other file and saving them. But when i open the 2 files in excel, all the columns are combined as one column.
Could someone give me a solution on how to divide the 1.4million records excel file into 2 files without messing up the columns or data.
If you are married to Excel, skip Toad completely and bring the data in directly with MS Query.
If this isn't SQL Server, you will need to set up an ODBC connection, and from there you can bind the query to the spreadsheet as follows:
From the "Data" tab, select "From Other Sources"
Pick SQL Server if it's SQL Server or MS Query if it's anything else
Skip all of the menus and paste in your SQL once you get to MS Query
Close MS Query, and your live query (and its results) will be dynamically linked to an Excel table (aka ListObject)
The great thing about this is when you want to refresh the query, right-click and refresh. Done.
Better still, you can set up the ODBC connection for your boss and he can do it himself.
Oh, and if you want this split into two datasets, change your SQL to pull the first half and the second half and have them each in a different worksheet. How you do this depends on your DBMS, which I'd encourage you to tag in your question.
If you make a data connection to another Excel file or an Access table it will import that data into excel as a Table. This is great! But when I do the same with "From Text" and choose a CSV, it loads no problem, but it loads as a RANGE not a TABLE. This is highly frustrating as I NEED it to be in the table format so that I can take advantage of the dynamic column names.
Am I missing a tickbox somewhere? I'm not opposed to using VBA but it really seems odd that Excel can't do CSV to Table so I'm hoping for a native solution.
I should mention that if it's a VBA solution, it CANT break the workbook. So if I'm doing something like:
=SUMIFS(CSVDATA[SalesDollars], CSVDATA[RepName], "BOB")
It will still work after refreshing.
So you can get around the 'data connection' not creating a table by downloading the MS Excel 2010 plugin (from Microsoft) called Power Query. It's free and is a default feature in MS Excel 2013.
It will allow you to choose to create a Power Query Table from External Data >> From File >> From CSV
Upon doing this, it will create a named table for you and a Power Query object attached to the table. You can use the table itself the same way you normally would - with the Columns as references for formulas.
For instance, the default table that I just created using the steps above was auto-named: Table_ExternalData_1
I can then select it using the normal method in formulas:
=Table_ExternalData_1[Column2]
etc...
Hope that helps.
I have an formatted excel destination with one of the columns being a percentage and another being currency. I'm loading this excel with data from a sql table using SSIS. However the excel is not formatted after the load. What is happening?
I was able export two columns of information (percent and dollar value) to an Excel file (97-2003 format) from an SSIS package created in Visual Studio 2010. The data was sourced from a table on a server using SQL Server 2012.
Steps:
Created an Excel file containing a column called Percent and another called Cost. Both were formatted accordingly and are located in the first row.
Saved the file in a 97-2003 format and then closed it.
Created a table in a database and populated it with a couple of records.
create table test_export (
mypercent numeric(18,2),
mydollar numeric(18,2))
insert into test_export values (2.1, 50.00)
insert into test_export values (4.5, 120.00)
Opened Visual Studio 2010 and created a new SSIS package. Created an OLEDB connection to the database.
Under the Control Flow tab added a Data Flow Task and double-clicked on it after it was added. This should now highlight the Data Flow tab.
Add an OLE DB Source and point to the newly added database connection, under Data access mode select Table or View and then select the table created in step #3. Select the OK button.
Add an Excel Destination making certain to create a new Excel connection manager for it and point to the Sheet1$. Make certain to indicate that the first row contains headers. Select the OK button and connect the OLE DB Source task to the Excel Destination task below. Since only numbers are involved, there is no need to apply a Data Conversion task between the two. But in the Excel Destination task remember to select Mappings and connect from left to right which source columns match up with which destination columns.
In the Solution Explorer pane, right-click on the solution and select Properties from the drop-down menu. Open Configuration Properties on the left and the select Debugging under it. On the right under Debug Options, look for Run64BitRuntime and change it from True to False if you have not already done so. Select the OK button.
Run the SSIS package.
Stop the SSIS package when all of the tasks show green for completed.
Open the Excel file and it will now contain the values imported from the database table with the Excel percent and dollar formats preserved from step #1.
I ran these steps in a test and everything worked perfectly. If your formats are not being preserved, then you are likely leaving out a step.
Hope this helps and please indicate if it answered your question.
I had the same problem. What I did was:
apply conditional formatting on the column.
see attached picture
It is also important that you have in the first row, a model row, filled with numeric/text values, in order for the ssis to export the numbers as numbers and not as strings model row
If your template contains several rows before the place where ssis start to export the data, you might need to have more model rows. (I needed 4). This rows have to be the first rows in the document, regardless where you start the export.
You can hide the model rows