I have a workbook that contains a pivot table which is updated by a macro. Before the data is refreshed, though, the connection string gets changed:
With ThisWorkbook.Connections("Data").ODBCConnection
.Connection = [Redacted]
.CommandText = "EXEC ExtractCases " & Client
.BackgroundQuery = False
.Refresh
End With
This seems to cause the pivot table to create a new connection (called either Connection or Data1, and I can't seem to figure out what it does to choose between them) and point itself to that. So I then have to add lines like these:
Sheets("Pivot").PivotTables("Pivot").ChangeConnection ThisWorkbook.Connections("Data")
Sheets("Pivot").PivotTables("Pivot").PivotCache.Refresh
Which seems to work (except when it doesn't), but leaves a lot of dead connections knocking around the workbook causing confusion.
I've tried manually deleting the Connection connection, but then it suddenly names itself Data1 itself for no apparent reason and the system gets upset because a non-existent Connection can't be deleted.
Is there something obvious I'm doing wrong? Is there some magic way to fix this so it doesn't create the second one in the first place to cause these kinds of headaches?
Note: I am running this code in Excel 2010, but the workbook has to be openable by 2003; however, I remove the VB module before distribution, so 2010 macro stuff is fine, it's just things in the workbook proper that might get tripped up by this...
I have experienced the same problem in Excel 2010 (might be the same for earlier versions, I dunno).
I have tried the same approach as you i.e. changing the connection of the Pivot Table in the VBA-code AFTER I have edited the commandText of the connection string. As you, I noted sometimes success and other times failure.
I haven't been able to find out why the problem arises and in which cases the above mentioned approach results in success or failure.
I have, however, found a working solution:
In your VBA code, you need to perform the following steps in the said order:
Change the commandText (which as you know results in the creation of a new
connection now in use by the Pivot Table).
Delete the old connection string.
Rename the connection string from step 1 to the name of the connection string deleted in step 2.
Refresh the Pivot Table.
NB: This only works if there is only one pivot table using the connection. If you have created extra Pivot Tables by copying the first one (i.e. they share the same Pivot Cache), the above mentioned procedure won't work (and I don't know why).
However, if you use only one Pivot Table with the connection string the approach will work.
I do not believe that it is the update of the connection string that is causing your problem. There is a bug when updating the CommandText property of an ODBC connection that causes an extra connection to be created. If you temporarily switch to an OLEDB connection, update your CommandText property and then switch back to ODBC it does not create the new connection. Don't ask me why... this just works for me.
I created a module that allows you to update the CommandText and/or Connection string. Insert this code into a new module:
Option Explicit
Sub UpdateWorkbookConnection(WorkbookConnectionObject As WorkbookConnection, Optional ByVal CommandText As String = "", Optional ByVal ConnectionString As String = "")
With WorkbookConnectionObject
If .Type = xlConnectionTypeODBC Then
If CommandText = "" Then CommandText = .ODBCConnection.CommandText
If ConnectionString = "" Then ConnectionString = .ODBCConnection.Connection
.ODBCConnection.Connection = Replace(.ODBCConnection.Connection, "ODBC;", "OLEDB;", 1, 1, vbTextCompare)
ElseIf .Type = xlConnectionTypeOLEDB Then
If CommandText = "" Then CommandText = .OLEDBConnection.CommandText
If ConnectionString = "" Then ConnectionString = .OLEDBConnection.Connection
Else
MsgBox "Invalid connection object sent to UpdateWorkbookConnection function!", vbCritical, "Update Error"
Exit Sub
End If
If StrComp(.OLEDBConnection.CommandText, CommandText, vbTextCompare) <> 0 Then
.OLEDBConnection.CommandText = CommandText
End If
If StrComp(.OLEDBConnection.Connection, ConnectionString, vbTextCompare) <> 0 Then
.OLEDBConnection.Connection = ConnectionString
End If
.Refresh
End With
End Sub
This UpdateWorkbookConnection subroutine only works on updating OLEDB or ODBC connections. The connection does not necessarily have to be linked to a pivot table. It also fixes another problem and allows you to update the connection even if there are multiple pivot tables based on the same connection.
To initiate the update just call the function with the connection object and command text parameters like this:
UpdateWorkbookConnection ActiveWorkbook.Connections("Connection"), "exec sp_MyAwesomeProcedure", "ODBC;..."
you may add this code, after you refresh connection.
With ThisWorkbook
.RefreshAll
End With
Had the same problem. Have a start date and end date field on the worksheet that is used to modify the period for the data in a pivot table. Added the following code for the worksheet:
Private Sub Worksheet_Change(ByVal Target As Range)
'Update the query when the date range has been changed.
If (Target.Row = Worksheets("Revenue").Range("StartDate").Row Or _
Target.Row = Worksheets("Revenue").Range("EndDate").Row) And _
Target.Column = Worksheets("Revenue").Range("StartDate").Column Then
FilterTableData
End If
End Sub
Sub FilterTableData()
'Declare variables
Dim noOfConnections As Integer
Dim loopCount As Integer
Dim conn As WorkbookConnection
Dim connectionName As String
Dim startDate As Date
Dim endDate As Date
Dim strMonth As String
Dim strDay As String
Dim startDateString As String
Dim endDateString As String
'Remove current connections
'Note: Excel creates a new connection with a new name as soon as you change the query for the connection. To avoid
' ending up with multiple connections delete all connections and start afresh.
'First delete all fake connections
noOfConnections = ActiveWorkbook.Connections.Count
For loopCount = noOfConnections To 1 Step -1
Set conn = ActiveWorkbook.Connections.Item(loopCount)
If conn Is Nothing Then
conn.Delete
End If
Next loopCount
'Then delete all extra connections
noOfConnections = ActiveWorkbook.Connections.Count
For loopCount = noOfConnections To 1 Step -1
If loopCount = 1 Then
Set conn = ActiveWorkbook.Connections.Item(loopCount)
conn.Name = "Connection1"
Else
Set conn = ActiveWorkbook.Connections.Item(loopCount)
conn.Delete
End If
Next loopCount
'Create date strings for use in query.
startDate = Worksheets("Revenue").Range("B1")
strDay = Day(startDate)
If Len(strDay) = 1 Then
strDay = "0" & strDay
End If
strMonth = Month(startDate)
If Len(strMonth) = 1 Then
strMonth = "0" & strMonth
End If
startDateString = Year(startDate) & "-" & strMonth & "-" & strDay & " 00:00:00"
endDate = Worksheets("Revenue").Range("B2")
strDay = Day(endDate)
If Len(strDay) = 1 Then
strDay = "0" & strDay
End If
strMonth = Month(endDate)
If Len(strMonth) = 1 Then
strMonth = "0" & strMonth
End If
endDateString = Year(endDate) & "-" & strMonth & "-" & strDay & " 00:00:00"
'Modify the query in accordance with the new date range
With conn.ODBCConnection
.CommandText = Array( _
"SELECT INVOICE.ACCOUNT_PERIOD, INVOICE.INVOICE_NUMBER, INVOICE_ITEM.LAB, INVOICE_ITEM.TOTAL_PRICE, ", _
"INVOICE.INVOICED_ON" & Chr(13) & "" & Chr(10) & _
"FROM Lab.dbo.INVOICE INVOICE, Lab.dbo.INVOICE_ITEM INVOICE_ITEM" & Chr(13) & "" & Chr(10) & _
"WHERE INVOICE.INVOICE_NUMBER = INVOICE_ITEM.INVOICE_NUMBER AND ", _
"INVOICE.INVOICED_ON > {ts '" & startDateString & "'} AND INVOICE.INVOICED_ON < {ts '" & endDateString & "'} ")
End With
'Refresh the data and delete any surplus connections
noOfConnections = ActiveWorkbook.Connections.Count
If noOfConnections = 1 Then
'Rename connection
ActiveWorkbook.Connections.Item(1).Name = "Connection"
'Refresh the data
ActiveWorkbook.Connections("Connection").Refresh
Else
'Refresh the data
ActiveWorkbook.Connections("Connection").Refresh
'Delete the old connection
ActiveWorkbook.Connections("Connection1").Delete
End If
'Refresh the table
ActiveSheet.PivotTables("Revenue").Update
End Sub
Related
First let me apologize for the spaghetti code as I am not sure of the best way to handle errors like this.
The situation I run into is that our data gets moved between two different data centers throughout the year, so I need to test the connection based on the error number -2147467259, which happens on cfRS.Open CIFstr, adoConn in the top section of the code if there is no connection to that server.
I have tried using IF statements to look at the above error number, but those were not succesful, so below is the wonderful spaghetti code I have written.
Error Number 3021 happens on cfRS.MoveFirst within the Branson: section of this code when I delete the value from Sheet1.Range("B103") and I am not sure why the error is not being handled with the On Error GoTo Err3021. The ErrHandler: in the top section of the code works just fine. Any help is greatly appreciated.
CODE:
Option Explicit
Sub CIFIncoming()
Dim adoConn As New ADODB.Connection
Dim cfRS As New ADODB.Recordset
Dim Name As String, Address1 As String, Address2 As String
Dim City As String, State As String, Zip As String
Dim HomePhone As String, CellPhone As String
Dim BSA As String
Dim strConn As String
Dim CIFstr As String, CIF As String
On Error GoTo ErrHandler
'\\\\BEGIN DATABASE INFORMATION GRAB////
' 1. Sets the Connection String to the Data Base
' 2. Opens the connection to the database
' 3. Sets the SQL String to get the fields from the Data Base
' 4. Defines the CIF Number to use in the SQL String
' 5. Opens the Recordset
' 6. Moves the cursor in the DataBase to first position
strConn = [REDACTED]
adoConn.Open strConn
CIF = UCase(Sheet1.Range("B103").Text)
CIFstr = "SELECT " & _
"cfna1, cfna2, cfna3, cfcity, cfstat, LEFT(cfzip, 5), cfhpho, cfcel1, cfudsc6 " & _
"FROM cncttp08.jhadat842.cfmast cfmast " & _
"WHERE cfcif# = '" & CIF & "'"
cfRS.Open CIFstr, adoConn
cfRS.MoveFirst
'\\\\END DATABASE INFORMATION GRAB////
'\\\\BEGIN WORKSHEET INFORMATION PLACEMENT////
' 1. Assigns each field from the Database to a variable
' 2. Moves data from Database to specific cells
Name = cfRS.Fields(0) 'cfna1
Address1 = cfRS(1) 'cfna2
Address2 = cfRS(2) 'cfna3
City = Trim(cfRS.Fields(3)) 'cfcity
State = Trim(cfRS.Fields(4)) 'cfstat
Zip = cfRS.Fields(5) 'cfzip
HomePhone = cfRS.Fields(6) 'cfhpho
CellPhone = cfRS.Fields(7) 'cfcel1
BSA = cfRS.Fields(8) 'cfudsc6
With Sheet1
.Range("B104") = Name
.Range("B105") = Address1
.Range("B106") = Address2
.Range("B107") = City & ", " & State & " " & Zip
End With
'\\\\END WORKSHEET INFORMATION PLACEMENT////
'\\\\BEGIN FINAL DATABASE OPERATIONS////
' 1. Closes connection to Database
' 2. Sets the Recordset from the Database to Nothing
' 3. Exits sub when there are no errors
cfRS.Close
Set cfRS = Nothing
Exit Sub
'\\\\END FINAL DATABASE OPERATIONS
ErrHandler:
'THIS HANDLES ERROR 3021
If Err.Number = 3021 Then
With Sheet1
.Range("B104") = vbNullString
.Range("B105") = vbNullString
.Range("B106") = vbNullString
.Range("B107") = ""
End With
End If
If Err.Number = -2147467259 Then GoTo Branson
Branson:
On Error GoTo Err3021
CIF = UCase(Sheet1.Range("B103").Text)
CIFstr = "SELECT " & _
"cfna1, cfna2, cfna3, cfcity, cfstat, LEFT(cfzip, 5), cfhpho, cfcel1, cfudsc6 " & _
"FROM bhschlp8.jhadat842.cfmast cfmast " & _
"WHERE cfcif# = '" & CIF & "'"
cfRS.Open CIFstr, adoConn
cfRS.MoveFirst
'\\\\END DATABASE INFORMATION GRAB////
'\\\\BEGIN WORKSHEET INFORMATION PLACEMENT////
' 1. Assigns each field from the Database to a variable
' 2. Moves data from Database to specific cells
Name = cfRS.Fields(0) 'cfna1
Address1 = cfRS(1) 'cfna2
Address2 = cfRS(2) 'cfna3
City = Trim(cfRS.Fields(3)) 'cfcity
State = Trim(cfRS.Fields(4)) 'cfstat
Zip = cfRS.Fields(5) 'cfzip
HomePhone = cfRS.Fields(6) 'cfhpho
CellPhone = cfRS.Fields(7) 'cfcel1
BSA = cfRS.Fields(8) 'cfudsc6
With Sheet1
.Range("B104") = Name
.Range("B105") = Address1
.Range("B106") = Address2
.Range("B107") = City & ", " & State & " " & Zip
End With
'\\\\END WORKSHEET INFORMATION PLACEMENT////
'\\\\BEGIN FINAL DATABASE OPERATIONS////
' 1. Closes connection to Database
' 2. Sets the Recordset from the Database to Nothing
' 3. Exits sub when there are no errors
cfRS.Close
Set cfRS = Nothing
Exit Sub
'\\\\END FINAL DATABASE OPERATIONS
Err3021:
'THIS HANDLES ERROR 3021
If Err.Number = 3021 Then
With Sheet1
.Range("B104") = vbNullString
.Range("B105") = vbNullString
.Range("B106") = vbNullString
.Range("B107") = ""
End With
End If
End Sub
Before you do cfRS.MoveFirst, do If not(cfRS.bof and cfRS.eof) then.
When your query's Where clause is WHERE cfcif# = '' then you have no rows. When you try to execute a cfRS.MoveFirst when you have no records, you get your error 3012. so you will want to ensure that you HAVE records before you try to navigate the recordset and then manipulate it.
I am having trouble getting data from an Access Database. I found this code online, and it seems to work (to an extent), but for some reason it will only pull the column headers, and none of the data from the query. I am not too familiar with Access, that is why I pulled one from offline.
Someone had a similar post a while back, where the code they used was the same, and our queries were exactly the same, but we had different issues.
Importing Data From Access Using Excel VBA
Would anyone happen to know why the data won't pull?
Sub getDataFromAccess()
Dim DBFullName As String
Dim Connect As String, Source As String
Dim Connection As ADODB.Connection
Dim Recordset As ADODB.Recordset
Dim Col As Integer
Dim startdt As String
Dim stopdt As String
Dim refresh
refresh = MsgBox("Start New Query?", vbYesNo)
If refresh = vbYes Then
Sheet1.Cells.Clear
startdt = Application.InputBox("Please Input Start Date for Query (MM/DD/YYYY): ", "Start Date")
stopdt = Application.InputBox("Please Input Stop Date for Query (MM/DD/YYYY): ", "Stop Date")
DBFullName = "X:\MyDocuments\CMS\CMS Database.mdb"
' Open the connection
Set Connection = New ADODB.Connection
Connect = "Provider=Microsoft.ACE.OLEDB.12.0;"
Connect = Connect & "Data Source=" & DBFullName & ";"
Connection.Open ConnectionString:=Connect
Set Recordset = New ADODB.Recordset
With Recordset
Source = "SELECT * FROM Tracking WHERE [Date_Logged] BETWEEN " & startdt & " AND " & stopdt & " ORDER BY [Date_Logged]"
.Open Source:=Source, ActiveConnection:=Connection
For Col = 0 To Recordset.Fields.Count - 1
Range(“A1”).Offset(0, Col).Value = Recordset.Fields(Col).Name
Next
Range(“A1”).Offset(1, 0).CopyFromRecordset Recordset
End With
ActiveSheet.Columns.AutoFit
Set Recordset = Nothing
Connection.Close
Set Connection = Nothing
End Sub
An easy way to get data in Excel, especially from Access, is to use the menu "Data > Access". This creates a connection to a table, that you can freely edit.
At the very least, that is a convenient way to limit your investigations to:
the query you typed (the connection string will always be OK, so if you're getting no values, it comes from the query)
or the VBA itself (if the table is returning values but not the corresponding VBA Sub, then you know it comes from the VBA itself, not the SQL).
I'm skipping the creation of connection becuse it's really straightforward; it's better to focus on what you can do once the table has been created.
Edit the connection
When you select the table and go to menu "Data > Properties", then in the window you click on the top right button "Connection properties", you get to the definition of the connection, i.e. some properties in the first tab and the actual definition in the second tab.
If you move the .mdb file, you'll have to change the connection string accordingly. There should be no other events forcing you to alter it.
If you want to type an actual complex query, you'll need to:
Change the command type from "Table" to "SQL"
Type the query in the bottom edit box.
Note if you want to define dynamic parameters in the WHERE clause, you can put question marks (?) instead of hardcoded values. Question marks can be linked to either constants (with a prompt to change their values) or cell.
Use in VBA
Once you checked with the connection that everything works, you have 2 solutions to put that in VBA.
Either use exactly the code you have above; in that case, you can make things easy by simply copying the connection string and the query.
Alternatively and this is what I would recommend, the table we have built previously can be updated very easily in VBA.
Use this piece of code:
WorksheetWithTable.ListObjects(1).QueryTable.Refresh
You really don't need more than this 1 line of code to do the refresh.
If you set your query to automatically refresh when a cell's value is being modified, then you do not even need it at all.
Note #1: Instead of an index in .ListObjects(1), you can use the table name.
Node #2: Refresh has an optional parameters to drive if the query is to be refresh in the background. True means the VBA code will not wait for the execution to end before moving to the next instruction. False, obviously, is the opposite.
The posted code is missing End If line. Perhaps this is just a posting typo because code should not compile and run.
The query SQL needs # delimiters for the date parameters:
Source = "SELECT * FROM Tracking WHERE [Date_Logged] BETWEEN #" & startdt & "# AND #" & stopdt & "# ORDER BY [Date_Logged]"
Text field would need apostrophe delimiters. Number field does not need delimiters.
I solved the answer to my own question after hours, i found a different set of code that worked fine. Thank you all for your help!
Sub getdatamdb()
Dim cn As Object, rs As Object
Dim intColIndex As Integer
Dim DBFullName As String
Dim TargetRange As Range
10 DBFullName = "X:\MyDocuments\CMS\CMS Database.mdb"
20 On Error GoTo Whoa
30 Application.ScreenUpdating = False
40 Set TargetRange = Sheets("Sheet1").Range("A1")
50 Set cn = CreateObject("ADODB.Connection")
60 cn.Open "Provider=Microsoft.Jet.OLEDB.4.0; Data Source=" & DBFullName & ";"
70 Set rs = CreateObject("ADODB.Recordset")
80 rs.Open "SELECT * FROM Tracking WHERE [Date_Logged] BETWEEN #" & startdt & "# AND #" & stopdt & "# ORDER BY [Date_Logged]", cn, , , adCmdText
' Write the field names
90 For intColIndex = 0 To rs.Fields.Count - 1
100 TargetRange.Offset(1, intColIndex).Value = rs.Fields(intColIndex).Name
110 Next
' Write recordset
120 TargetRange.Offset(1, 0).CopyFromRecordset rs
LetsContinue:
130 Application.ScreenUpdating = True
140 On Error Resume Next
150 rs.Close
160 Set rs = Nothing
170 cn.Close
180 Set cn = Nothing
190 On Error GoTo 0
200 Exit Sub
Whoa:
210 MsgBox "Error Description :" & Err.Description & vbCrLf & _
"Error at line :" & Erl & vbCrLf & _
"Error Number :" & Err.Number
220 Resume LetsContinue
End If
End Sub
I am trying to export six recordsets generated by a Do-Loop to six specific tabs in a single MS Excel workbook using VBA. Instead of updating the single tabs, however, the code creates six open iterations of the workbook with only the first being editable, the remainder read-only. The recordsets are successfully exported into the correct tab in the desired format.
Function ExportRecordset2XLS2(ByVal rs As DAO.Recordset, strSheetName)
Dim xls As Object
Dim xlwb As Object
Dim xlws As Object
Dim fld As DAO.Field
Dim strPath As String07
Dim strTitleRange,strHeaderRange, strBodyRange as String
On Error GoTo err_handler
strPath = "C:\Database\Roster.xlsx"
Set xls = CreateObject("Excel.Application")
Set xlwb = xls.Workbooks.Open(strPath)
xls.Visible = False
xls.ScreenUpdating = False
Set xlws = xlwb.Worksheets(strSheetName)
xlws.Activate
'Define ranges for formatting
intFields = rs.Fields.Count
intRows = rs.RecordCount
strTitleRange = "A1:" & Chr(64 + intFields) & "1"
strHeaderRange = "A2:" & Chr(64 + intFields) & "2"
strBodyRange = "A3:" & Chr(64 + intFields) & (intRows + 2)
'Build TITLE Row
xlws.Range("A1").Select
xls.ActiveCell = Format(Now(), "YYYY") & " Roster (" & strSheetName & ")"
'Build HEADER Row
xlws.Range("A2").Select
For Each fld In rs.Fields
xls.ActiveCell = fld.Name
xls.ActiveCell.Offset(0, 1).Select
Next
rs.MoveFirst
'Paste Recordset into Worksheet(strSheetName) starting in A3
xlws.Range("A3").CopyFromRecordset rs
On Error Resume Next
xls.Visible = True 'Make excel visible to the user
Set rs = Nothing
Set xlws = Nothing
Set xlwb = Nothing
xls.ScreenUpdating = True
Set xls = Nothing
xls.Quit
Exit Function
err_handler:
DoCmd.SetWarnings True
MsgBox Err.Description, vbExclamation, Err.Number
Exit Function
End Function
I suspect the problem revolves around how the function opens the .xlsx file for editing; I have tried programmatically closing the active worksheet and/or workbook in various ways and sequences to no effect. I could presumably insert a break into the code that generates the recordset to allow MS Excel to open then close, before repeating the process with the next tab, but there must be a more elegant way.
Image of multiple iterations in Excel
** As a side note, I did post this question also to answers.microsoft.com before finding this forum. Sorry. **
Thanks in advance, Erik
For each workbook opened you can check the security and reset it so it can be edited:
If Application.ProtectedViewWindows.Count > 0 Then
Application.ActiveProtectedViewWindow.Edit
End If
As expected, this turned out to be series of small issues that resulted in MS Excel holding the workbook file in read-only status after the function would error out. SOlved after scrutinizing each line of code to find individual lines that were failing.
Try this methodology and feedback.
Dim qdf As DAO.QueryDef
Dim dbs As DAO.Database
Dim rstMgr As DAO.Recordset
Dim strSQL As String, strTemp As String, strMgr As String
' Replace PutEXCELFileNameHereWithoutdotxls with actual EXCEL
' filename without the .xls extension
' (for example, MyEXCELFileName, BUT NOT MyEXCELFileName.xls)
Const strFileName As String = "PutEXCELFileNameHereWithoutdotxls"
Const strQName As String = "zExportQuery"
Set dbs = CurrentDb
' Create temporary query that will be used for exporting data;
' we give it a dummy SQL statement initially (this name will
' be changed by the code to conform to each manager's identification)
strTemp = dbs.TableDefs(0).Name
strSQL = "SELECT * FROM [" & strTemp & "] WHERE 1=0;"
Set qdf = dbs.CreateQueryDef(strQName, strSQL)
qdf.Close
strTemp = strQName
' *** code to set strSQL needs to be changed to conform to your
' *** database design -- ManagerID and EmployeesTable need to
' *** be changed to your table and field names
' Get list of ManagerID values -- note: replace my generic table and field names
' with the real names of the EmployeesTable table and the ManagerID field
strSQL = "SELECT DISTINCT ManagerID FROM EmployeesTable;"
Set rstMgr = dbs.OpenRecordset(strSQL, dbOpenDynaset, dbReadOnly)
' Now loop through list of ManagerID values and create a query for each ManagerID
' so that the data can be exported -- the code assumes that the actual names
' of the managers are in a lookup table -- again, replace generic names with
' real names of tables and fields
If rstMgr.EOF = False And rstMgr.BOF = False Then
rstMgr.MoveFirst
Do While rstMgr.EOF = False
' *** code to set strMgr needs to be changed to conform to your
' *** database design -- ManagerNameField, ManagersTable, and
' *** ManagerID need to be changed to your table and field names
' *** be changed to your table and field names
strMgr = DLookup("ManagerNameField", "ManagersTable", _
"ManagerID = " & rstMgr!ManagerID.Value)
' *** code to set strSQL needs to be changed to conform to your
' *** database design -- ManagerID, EmployeesTable need to
' *** be changed to your table and field names
strSQL = "SELECT * FROM EmployeesTable WHERE " & _
"ManagerID = " & rstMgr!ManagerID.Value & ";"
Set qdf = dbs.QueryDefs(strTemp)
qdf.Name = "q_" & strMgr
strTemp = qdf.Name
qdf.SQL = strSQL
qdf.Close
Set qdf = Nothing
' Replace C:\FolderName\ with actual path
DoCmd.TransferSpreadsheet acExport, acSpreadsheetTypeExcel9, _
strTemp, "C:\FolderName\" & strFileName & ".xls"
rstMgr.MoveNext
Loop
End If
rstMgr.Close
Set rstMgr = Nothing
dbs.QueryDefs.Delete strTemp
dbs.Close
Set dbs = Nothing
I have a csv file that has approx 600 fields and approx 100k of rows, i would like to import only select fields and only certian rows where a select set of fields match a certain set of criteria into an existing excel worksheet tab
I attempted to use ms query within excel but it stops at 255 columns, i can import the whole file in excel 2010 (250m) but it is a memory hog and by the time i remove the unneeded fields and rows it locks up my computer.
I would like to kick the import process off with an excel vba macro. I have all the front end code of file selection, etc.... But need some assistance in the text read query convert to excel area of vba
Any assitance would be greatly appreciated
Thanks
Tom
For that many records you would be better off importing the .csv into Microsoft Access, indexing some fields, writing a query that contains only what you want, and then exporting to Excel from the query.
If you really need an Excel-only solution, do the following:
Open up the VBA editor. Navigate to Tools -> References. Select the most recent ActiveX Data Objects Library. (ADO for short). On my XP machine running Excel 2003, it's version 2.8.
Create a module if you don't have one already. Or create one anyway to contain the code at the bottom of this post.
In any blank worksheet paste the following values starting at cell A1:
SELECT Field1, Field2
FROM C:\Path\To\file.csv
WHERE Field1 = 'foo'
ORDER BY Field2
(Formatting issues here. select from, etc should each be in their own row in col A for reference. The other stuff are the important bits and should go in column B.)
Amend the input fields as appropriate for your filename and query requirements, then run thegetCsv() subroutine. It will put the results in a QueryTable object starting at cell C6.
I personally hate QueryTables but the .CopyFromRecordset method I prefer to use with ADO doesn't give you field names. I left the code for that method in, commented out, so you can investigate that way. If you use it, you can get rid of the call to deleteQueryTables() because it's a really ugly hack, it deletes whole columns which you may not like, etc.
Happy coding.
Option Explicit
Function ExtractFileName(filespec) As String
' Returns a filename from a filespec
Dim x As Variant
x = Split(filespec, Application.PathSeparator)
ExtractFileName = x(UBound(x))
End Function
Function ExtractPathName(filespec) As String
' Returns the path from a filespec
Dim x As Variant
x = Split(filespec, Application.PathSeparator)
ReDim Preserve x(0 To UBound(x) - 1)
ExtractPathName = Join(x, Application.PathSeparator) & Application.PathSeparator
End Function
Sub getCsv()
Dim cnCsv As New ADODB.Connection
Dim rsCsv As New ADODB.Recordset
Dim strFileName As String
Dim strSelect As String
Dim strWhere As String
Dim strOrderBy As String
Dim strSql As String
Dim qtData As QueryTable
strSelect = ActiveSheet.Range("B1").Value
strFileName = ActiveSheet.Range("B2").Value
strWhere = ActiveSheet.Range("B3").Value
strOrderBy = ActiveSheet.Range("B4").Value
strSql = "SELECT " & strSelect
strSql = strSql & vbCrLf & "FROM " & ExtractFileName(strFileName)
If strWhere <> "" Then strSql = strSql & vbCrLf & "WHERE " & strWhere
If strOrderBy <> "" Then strSql = strSql & vbCrLf & "ORDER BY " & strOrderBy
With cnCsv
.Provider = "Microsoft.Jet.OLEDB.4.0"
.ConnectionString = "Data Source=" & ExtractPathName(strFileName) & ";" & _
"Extended Properties=""text;HDR=yes;FMT=Delimited(,)"";Persist Security Info=False"
.Open
End With
rsCsv.Open strSql, cnCsv, adOpenForwardOnly, adLockReadOnly, adCmdText
'ActiveSheet.Range("C6").CopyFromRecordset rsCsv
Call deleteQueryTables
Set qtData = ActiveSheet.QueryTables.Add(rsCsv, ActiveSheet.Range("C6"))
qtData.Refresh
rsCsv.Close
Set rsCsv = Nothing
cnCsv.Close
Set cnCsv = Nothing
End Sub
Sub deleteQueryTables()
On Error Resume Next
With Application
.ScreenUpdating = False
.Calculation = xlCalculationManual
End With
Dim qt As QueryTable
Dim qtName As String
Dim nName As Name
For Each qt In ActiveSheet.QueryTables
qtName = qt.Name
qt.Delete
For Each nName In Names
If InStr(1, nName.Name, qtName) > 0 Then
Range(nName.Name).EntireColumn.Delete
nName.Delete
End If
Next nName
Next qt
With Application
.ScreenUpdating = True
.Calculation = xlCalculationAutomatic
End With
End Sub
You can parse your input file extracting the lines that conform to your criteria. The following code uses the split function on each line of the CSV file to separate the fields and then checks to see if it matches the required criteria. If all the criteria match then selected fields are saved in a new CSV file then you can just open the smaller file. You will need to set the microsoft scripting runtime reference in the VBA editor for this to work.
This method should use little memory as it processes 1 line at a time, I tested it on data of 600 fields and 100000 lines and it took about 45 seconds to process the file with no noticable increase in RAM usage in windows task manager. It is CPU intensive and the time taken would increase as the complexity data, conditions and the number of fields copied increases.
If you prefer to write directly to an existing sheet this can be easily acheived, but you would have to rememove any old data there first.
Sub Extract()
Dim fileHandleInput As Scripting.TextStream
Dim fileHandleExtract As Scripting.TextStream
Dim fsoObject As Scripting.FileSystemObject
Dim sPath As String
Dim sFilenameExtract As String
Dim sFilenameInput As String
Dim myVariant As Variant
Dim bParse As Boolean 'To check if the line should be written
sFilenameExtract = "Exctract1.CSV"
sFilenameInput = "Input.CSV"
Set fsoObject = New FileSystemObject
sPath = ThisWorkbook.Path & "\"
'Check if this works ie overwrites existing file
If fsoObject.FileExists(sPath & sFilenameExtract) Then
Set fileHandleExtract = fsoObject.OpenTextFile(sPath & sFilenameExtract, ForWriting)
Else
Set fileHandleExtract = fsoObject.CreateTextFile((sPath & sFilenameExtract), True)
End If
Set fileHandleInput = fsoObject.OpenTextFile(sPath & sFilenameInput, ForReading)
'extracting headers for selected fields in this case the 1st, 2nd and 124th fields
myVariant = Split(fileHandleInput.ReadLine, ",")
fileHandleExtract.WriteLine (myVariant(0) & "," & _
myVariant(1) & "," & _
myVariant(123))
'Parse each line (row) of the inputfile
Do While Not fileHandleInput.AtEndOfStream
myVariant = Split(fileHandleInput.ReadLine, ",")
'Set bParse initially to true
bParse = True
'Check if the first element is greater than 123
If Not myVariant(0) > 123 Then bParse = False
'Check if second element is one of allowed values
'Trim used to remove pesky leading or lagging values when checking
Select Case Trim(myVariant(1))
Case "Red", "Yellow", "Green", "Blue", "Black"
'Do nothing as value found
Case Else
bParse = False 'As wasn't a value in the condition
End Select
'If the conditions were met by the line then write specific fields to extract file
If bParse Then
fileHandleExtract.WriteLine (myVariant(0) & "," & _
myVariant(1) & "," & _
myVariant(123))
End If
Loop
'close files and cleanup
fileHandleExtract.Close
fileHandleInput.Close
Set fileHandleExtract = Nothing
Set fileHandleInput = Nothing
Set fsoObject = Nothing
End Sub
I have a table in an Excel file with some data, and I want to export these data to my database on Access (in a concrete table on my database called Water Quality) with a VBA code to avoid to open my Database every time that I want to introduce more data on it.
At the moment I have this code but it's not working...
Sub Button14_Click()
' Macro purpose: To add record to Access database using ADO and SQL
' NOTE: Reference to Microsoft ActiveX Data Objects Libary required
' Exports data from the active worksheet to a table in an Access database
'Dim cnt As New ADODB.Connection
'Dim rst As New ADODB.Recordset
Dim cnt As DAO.Database
Dim rst As Recordset
Dim dbPath As String
Dim tblName As String
Dim rngColHeads As Range
Dim rngTblRcds As Range
Dim colHead As String
Dim rcdDetail As String
Dim ch As Integer
Dim cl As Integer
Dim notNull As Boolean
Dim sConnect As String
'Set the string to the path of your database as defined on the worksheet
dbPath = "C:\Documents and Settings\Administrador\Mis documentos\MonEAU\modelEAU Database V.2.accdb"
tblName = "Water Quality"
Set rngColHeads = ActiveSheet.Range("tblHeadings")
Set rngTblRcds = ActiveSheet.Range("tblRecords")
'Set the database connection string here
sConnect = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source='" & dbPath & "';" 'For use with *.accdb files
' sConnect = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" & dbPath & ";" 'For use with *.mdb files
'Concatenate a string with the names of the column headings
colHead = " ("
For ch = 1 To rngColHeads.Count
colHead = colHead & rngColHeads.Columns(ch).Value
Select Case ch
Case Is = rngColHeads.Count
colHead = colHead & ")"
Case Else
colHead = colHead & ","
End Select
Next ch
'Open connection to the database
cnt.Open sConnect
'Begin transaction processing
On Error GoTo EndUpdate
cnt.BeginTrans
'Insert records into database from worksheet table
For cl = 1 To rngTblRcds.Rows.Count
'Assume record is completely Null, and open record string for concatenation
notNull = False
rcdDetail = "('"
'Evaluate field in the record
For ch = 1 To rngColHeads.Count
Select Case rngTblRcds.Rows(cl).Columns(ch).Value
'if empty, append value of null to string
Case Is = Empty
Select Case ch
Case Is = rngColHeads.Count
rcdDetail = Left(rcdDetail, Len(rcdDetail) - 1) & "NULL)"
Case Else
rcdDetail = Left(rcdDetail, Len(rcdDetail) - 1) & "NULL,'"
End Select
'if not empty, set notNull to true, and append value to string
Case Else
notNull = True
Select Case ch
Case Is = rngColHeads.Count
rcdDetail = rcdDetail & rngTblRcds.Rows(cl).Columns(ch).Value & "')"
Case Else
rcdDetail = rcdDetail & rngTblRcds.Rows(cl).Columns(ch).Value & "','"
End Select
End Select
Next ch
'If record consists of only Null values, do not insert it to table, otherwise
'insert the record
Select Case notNull
Case Is = True
rst.Open "INSERT INTO " & tblName & colHead & " VALUES " & rcdDetail, cnt
Case Is = False
'do not insert record
End Select
Next cl
EndUpdate:
'Check if error was encounted
If Err.Number <> 0 Then
'Error encountered. Rollback transaction and inform user
On Error Resume Next
cnt.RollbackTrans
MsgBox "There was an error. Update was not succesful!", vbCritical, "Error!"
Else
On Error Resume Next
cnt.CommitTrans
End If
'Close the ADO objects
cnt.Close
Set rst = Nothing
Set cnt = Nothing
On Error GoTo 0
End Sub
At the moment, the problem is when I debug the code, there appears the compiling error: "Method or data member not found" on the function "cnt.Open sConnect".
If this is possible, any help would be greatly appreciated.
Note: I'm using Office 2010.
Your compile error is due to these 2 lines:
Dim cnt As DAO.Database
cnt.Open sConnect
A DAO.Database object does not have an .Open method, which explains "Method or data member not found". Too often error messages can be somewhat vague and just not very helpful. However, in this case, I can't think how the error message could be any more clear.
There is something more which I don't understand. sConnect looks like an ADO connection string. But cnt is a DAO (database) object. You can't mashup the two object models like that in one statement.
You have this just before your active variable declarations:
'Dim cnt As New ADODB.Connection
Then later in your procedure, you have:
'Close the ADO objects
cnt.Close
So perhaps you originally intended cnt to be an ADO.Connection object and didn't adapt the rest of your code after you switched it to a DAO.Database object.
I suggest you revise your code to sort out the DAO vs. ADO confusion, then show us the new code if you have any remaining problems. And please show us only the minimum tested code required to reproduce the problem you're hoping to solve. TIA for your consideration.
I only have Access databases that open the excel file (instead of the other way around) but from looking through my code I think you should go straight to this:
`Set cnt = OpenDatabase_
(dbPath, False, True, "Access 8.0;")
Found this on http://support.microsoft.com/kb/190195 too.
Does this help?