I built a data entry UserForm to populate a worksheet that will serve as the raw database. The raw data requires further manipulation and analysis in order to be reported, so I set up a database connection using Get External Data>From Microsoft Query>Excel Files, pointed it to the file I was already working in, selected the fields I wanted and performed basic functions on those I wanted aggregated. This creates an Excel table where I then use formulas that to complete the analysis. It works great for me; I can add entries to the database, Refresh the summary table, the new entries are added and the formulas populate automatically.
The problem is that no one else can refresh the table because it's looking locally for the file. The connection string is:
DSN=Excel Files;DBQ=C:\Users\MyName\Desktop\Folder 1\Results.xlsm;DefaultDir=C:\Users\MyName\Desktop\Folder 1;DriverId=1046;MaxBufferSize=2048;PageTimeout=5;
I have a very basic understanding of the database connections, but I need this file to be as automated as possible by request of my colleague. Can I fix the connection string so that the file is "flexible" and can be refreshed on any computer? Is this the best solution? If not, what else can I do that does not involve downloading additional plugins or 3rd party add-ins?
If what you need is a file containing the raw data (a Database) AND one or more excel files connected to it that pick up the data from the database and work with this data, you need to split the two things. You can do the database with an access file located on a shared directory with an appropriate table and you can reproduce the user form in this file so the insertion of the data will be made in this file. Then you connect one or more excel files (using connection Mode = Share Deny None, so you can update the data and at the same time work with them from the excel files), the data will be imported in the files in tables and here you do all the proessing you need.
If one file is enough for you (you don't need to have a database with the row data separated and you don't need to use the file from different location simultaneously) and all the problem is that if the file is opened from a different location from the one specifyed in the connection string it does not work...well in this case (that seems the case) i don't know why to use a connection to the same file.
If what you need is a table for work with, just create it selecting the range with the data you already have inserted (Create a table - quick start guide) and then when you add data through the form instead of adding them in a "normal" row, add them to a new row of the table with something like WorkSheets("name").ListObjects("table_name").ListRows.Add and add the data in the new table row.
Related
I recently modernized all my excel files, and started using the magic of PowerQueries and PowerPivot.
Background: I have 2 files:
- First one is a "master" with all sales and production logs, and everything works inside that excel file with Power queries to tables stored in that same file.
- Second one is mostly a different set of data about continuous improvement data, but i'd like to start linking them with the master file by having charts that compare efficiency to production, etc.
As it is now, I am using links by entering a direct reference to the cells/ranges in the master file (i.e: [Master.xlsm]!$A1:B2) However, every new version of the Master file, I have to update the links and it's not scalable if I have more documents in the future.
Options:
- Is it possible to store all the queries or data from the Master files in a separate file in the same folder and "call" for it when needed either in my Sales/Production master file or the Manufacturing file? That could be a database or connection file that has the queries to the data stored in the master file.
- If not, what is the best way to connect my Manufacturing file to my Master file without entering specifically the filename?
My fear is that as soon as the Master file name will change (date, version), I will have to navigate inside the queries and fix all the links again. Additionally, I wanna make this futureproof early one as I plan to gather large amounts of data and start more measurements.
Thanks for your help!
Once you have a data model built, you can create a connection to it from other Excel files. If you are looking for a visible way to control the source path of the connected file, you can add a named range to the Excel file that is connecting to the data model, and in the named range, enter the file path. In Power Query, add a new query that returns your named range (the file path), and swap out the static file path in your queries with the new named range query.
Here is a sample M code that gets the contents of a named range. This query is named "folderPath_filesToBeAudited".
let
Source = Excel.CurrentWorkbook(){[Name="folderPath_filesToBeAudited"]}[Content]{0}[Column1]
in
Source
Here is an example of M code showing how to use the new query to reference the file path.
Folder.Files(folderPath_filesToBeAudited)
Here is a step-by-step article.
https://accessanalytic.com.au/powerquery_namedcells_parameters/
I have 4 Sheets sheets with similar fields. I intend to merge these sheets together to create a master file that has all information in one sheet. However, i need Tableau to connect to the final merged file so i can create dashboards off. This works locally as i have an access program that appends the tables together and creates a new table which Tableau connects to.
The main issue is i am trying to take this process offline (to run online to locally), meaning i need a database that can;
1- Drop content of the tables, pick up the sheets from a specified folder, import them into specified tables.
2- Append new tables into master tables.
All of this should be done automatically at a scheduled time.
I tried using SQL server (SQL Agent for scheduling job import/append etc) for this requirement but i need to know if something else is out there that can serve this purpose efficiently.
Thank you
As long as the sheets have the exact fields, you should be able to use Tableau's Union feature. This feature will allow you to do a wildcard search for sheets within a folder structure. Anytime the data is refreshed in Tableau it will reach back out to the folder and update/union what is currently there.
Hello and thanks for any help. I'm a noob with Access but my company has asked a question about possibly importing some information for certain records in Access. Basically, I work for an insurance company and they have a claims system which was built in Access about a decade ago by someone who has left the company recently. The system works and is fine for current needs. However, we were recently asked to amend a field for certain records (claims). Because there are about 200-300 records, we are looking at a possible import solution.
The problem is, I have never done this before and am worried that it might affect other records or other fields in the records. The only thing that needs to change is one field and the rest must remain unchanged.
I know the Access table name & record numbers (RecordNo - textbox in Access) and the name of the field I want to change (Reference - textbox in Access) but am unsure how this can be imported, how the excel file needs to be prepared and how to make sure no new records are added but instead existing records are amended.
For instance, can I just have 2 columns, one called "RecordNo" and the other "Reference"? Or do I need to add blank columns to account for the extra columns in the Access table? Do I need to create a named range or an excel table or simply put the columns in Excel? Is there any specific formatting that I should be using (Text or General or something else - the Reference will be a text value as it has both numbers and letters)? When importing in Access using the import wizard, do I need to choose "Append a copy of the records to the table"? How will it know which record to amend as the Access table will contain thousands of records that I don't want changed in any way?
I also have access to the "Navigation Pane" in Access where I can find tables and queries etc and not sure if the records in question can be bulk amended on the table instead?
To make matters more complicated, the Access database is on the server and needs to be accessed by multiple users at the same time so I would ideally like to test this out on a separate copy. But copying it to my own computer does not sever the connection with the copy on the server and any changes are reflected in the original copy immediately.
I tried looking online but I can't seem to find anything that will quash my worries. I can find a few articles talking about importing issues, though not what I would be interested in, but they are all for previous versions of Access and really I can barely understand the current version. We are using Excel and Access 2013.
If possible, I would rather not use VBA as the current database has a lot of it anyway and its's difficult to manage and navigate. I also have no idea about Access VBA, just excel.
Thank you
To make matters more complicated, the Access database is on the server
and needs to be accessed by multiple users at the same time so I would
ideally like to test this out on a separate copy. But copying it to my
own computer does not sever the connection with the copy on the server
and any changes are reflected in the original copy immediately
That is because you are only copying the front end. There is somewhere a separate access file that is the back end. Or it could be a different database system like MS SQL Server or MySQL. So your first task it to find where the actual data is.
Beyond that, under no circumstances import an excel file directly into an existing table. Create an excel file with the necessary fields (record identifier and new value) and import it as a new table, then create an update query to effect the changes you need.
In my Access 2013 Desktop database, I want to validate an Excel form through VBA before importing its rows to a table.
The problem is that the imported Excel table may contain in the same column values that are numeric, string, null, etc. depending on the row due to mistakes from the user, so these values have to be initially of VBA type Variant before validation.
If the row is invalid (e.g. one value is not of numeric type while it should be, one mandatory field is null, etc.), I want to be able to store its values and let the user correct it later on. I am therefore looking for a way to build a table where I could store these rows, but where the fields are kept intact, so with the possible mixed data types.
This seems not to be possible, as Access tables need apparently to have fields of a defined type, so nothing equivalent to Variant.
How could I achieve this? What would be the simplest and most efficient way to store rows of Variant values while keeping the structure of the columns? The only way I could think of so far is to concatenate these values for each row in a string using some defined separator, and store this in a String column in an Access table, but maybe there is an easier way to do?
Thanks a lot for your help.
EDIT: Reading your answers I realized that I forgot to specify that the people filling and sending the Excel forms (who typically do not have Access on their computers) are not the same users than the ones who have the Access database and need to import the data. In this configuration it is impossible to use directly Access forms to input data, which would of course simplify tremendously the task. While we try to enforce rules for people who fill the Excel form at the first place there will be mistakes and the Access application needs to cope with them. And the importing process for the users manipulating Access should be as simple as possible (these users are not experts in IT).
I though about your problem for a while and made a few notes as I thought below.
I would strongly advise that unless the excel workbook is providing other functionality that you do not use excel, but create an simple Access database with a form they can use to interact with "their" data. They do not necessarily need an Access license to do this. Access 2013 even has new web features that can be used if you have sharepoint to collect data via a Access created web frontend.
If you are to collect data in Excel, then you must use excel validation and VBA code to validate the data as much as possible before transferring to access.
There are complexities that may or may not be an issue for you, things like:
Can users edit/created/delete the same rows concurrently - what happens if they do! You may need to "lock" rows when they are downloaded from Access. But what happens if tow users add the same record, or ones deletes a row before another user commits it back again.
Can a user open multiple excel files and edit the data they store without committing the data to Access?
Can multiple users login to the same Workbook with uncommitted changes and edit the data.
Using access will simplify your code as it will be able to prevent erroneous data being entered and remove the need to deal with the above issues and others
In summary you are using excel as a front end "form" to data stored in Access.
Each excel file can have data that has yet to be up
I would suggest that the primary key of a row is the "path and name" of the excel file that was used to create it & a unique numeric identifier. The unique identifier is a counter that is maintained for each excel file. The "path and filename" could be replaced by a unique identifier created for each file.
Many users enter data using multiple instances of Excel into a form which results in one or more rows being updated/inserted/deleted in an Excel table stored in each spreadsheet.
I would expect that whenever Excel is opened the user enters a username and excel will grab "their data" from Access. Alternatively a workbook might be set up for each department or "case type" and only interact with data that matches this "custom criteria set up in the workbook". Excel would not necessarily need to store data itself longterm. The workbook might always save data back to the database when it is saved.
You say Excel VBA performs limited validation on the data (but no complete validation). It should be used as much as possible and arguably should be able to do exactly the same validation that access can do. At the very least it should enforce datatypes etc (eg using the standard data validation rules on the excel data menu) or perhaps use VBA/controls to get any access data it needs to validate the data entered.
After updating the data the user can "commit" (ie save) the data to the access database. Before closing the workbook you might want to commit the data.
(An issue is whether a user might open many workbooks perhaps on many machines without committing the data.)
An Access "staging" table can be created with all columns having the datatype "shorttext" and not-required.
The process that loads data from an excel table into Access, will uploaded all excel rows into this staging table. It will then validate each row in the staging table and process all valid data into the main table(s) that have data types, relationships constraints etc.. Any valid rows in the staging table are flagged as "VALIDATED & TRANSFERRED", Invalid rows are flagged as INVALID. The "VALIDATED AND TRANSFERRED ROWS" are subsequently ignored, but kept so you can check what processing has happened, perhaps only whilst testing.
The data in excel is then updated with the Valid/Invalid status from Access and suitable messages given to the excel user. The user may correct and then re-commit the data.
Each excel file has a status of "changed/Unchanged" to indicate whether a user has changed data in the file.
When a user opens an excel workbook and status is unchanged it will refresh it's data. If it's changed a refresh probably can't be done.
In order for the data in Access to be updated/deleted/inserted with the changes made in Excel, there will need to be a unique identifier for each row that cannot be changed by the users in excel. This is likely to be the Path&Filename or the UserName logged into Excel and a numeric counter (which is maintained for each file or user). (This assumes that the user will have to commit changes in one excel file before they work on another.)
Anyway, without knowing more it's difficult to fully design what you will need, but I hope these thoughts help you
Harvey
I have an xls with a pivot table using a proc to populate it.
I deleted the data connection in 'admin tools - odbc - file DSN' and it still works, how is that possible?
Does that mean that the recipent doesn't have to have the file DSN present in order to see data in the pivot table?
That is possible because when you use the file DSN, excel reads the content of the file, which essentially has all the entries to construct the connection string to outside data provider. Then Excel will construct the connection string and use that to populate the QueryTable. So once you successfully refresh the QueryTable for the first time, you no longer need that file DSN. So it will work event after you delete it or in some else’s machine without that DSN.
On a side note when you use file DSN in a QueryTable and later if you update the FileDSN (say to a new database or any thing like that) your QueryTable refresh will not reflect that change, you have to reconstruct the QueryTable.
when you create pivot table once, afterward that .xls file will work independently in your Hard Drive. So, it doesn't require any DSN furthermore.
The Pivot Table keeps a Pivot Table data cache behind it. You can manipulate this data any way your like, but it won't refresh, obviously, until you reconnect.
Stan Scott