I'm in a VERY difficult problem.
I have a Microsoft Access Data Base, but it was made in the most chaotic way possible. The DB has like 150+ tables, Only uses like 50% of the tables. The relations are almost random. But, somehow, it delivers some information.
I need to get a particular component of the DB, but is so tangled that I can not manage to get into the table that creates that value. I revised every table, one by one, and found nothing.
I used mdbtools for Linux to try to inspect with more details the DB. But unfortunately has not been developed in years, and it closes every time. Maybe because the DB is "big" ? -700 mg-
I'm wondering: is there a way to see all the relations the arrives to the particular value I'm looking? Or to decompile the DB? I have no idea in which language it was made. I'm suspecting that it was made in Visual, just because is rather crappy.
Well, waiting for some help.
I would suggest using (still) MS Access for this. But, if relationships look messy on the diagram, you can query one of the system tables (MSysRelationships) directly to get ALL the relationships you need (e.g. for particular table etc.):
To unhide system tables in early versions of Access (97-2003), follow the instructions here:
For Access 2007, do the following:
Related
I am trying to backup a database through sqlachemy and save it as a file. I tried using the extension, Flask-AlchemyDumps, but it appears to no longer be supported.
I musted be missing something obvious as this is surly an action a lot of developers want to do. Does anyone know how I should be backing up the database?
Thanks in advance
J Kirkman
SQLAlchemy is an ORM which sits between your code and the database. It's useful if you want to interact with specific rows and relationships without having to keep track of lots of ids and joins.
What you're looking for is a way to dump the entire contents of your DB to disk, presumably so you can restore it later/elsewhere. This is a bulk action, which is your first clue that an ORM may not be a suitable tool. (ORMs tend to be fast enough for small to medium operations, but slow and not ideal for actions which affect 10s of 1000s of rows at once.) And indeed, this isn't usually something you'd use an ORM for, it's a feature of your DB, presumably Postgres or MySQL. If you happen to be using Heroku, you can use their command line tool to do this.
I’m just starting out with CouchDB (2.1), and I’m planning to use it to replicate confidential per-user data from a mobile app up to my server. I’ve read that per-user databases are the best way to do this, and I’ve set that up. Each database has a mix of user-created documents of types Foo and Bar.
Now, I’d also like to be able to collect multi-user slices of that data together into one database and build views on it for admin reporting. Say I want a database which contains all the Foos from all users. So far so good, an entry in _replicator with a filter from each user database to one target does the job.
But looking at the combined database, I can’t tell which user a given Foo came from. I could write the user id into each document within the per-user database but that seems redundant and adds the complexity of validation. Is there any other way?
CouchDB's replicator simply tries to match up the exact state of a given document in the target database — and if it can't, it stores ± the exact source contents anyway (as a conflicting version).
Furthermore the _rev field of a document, which the replication system uses to check if a document needs to be updated, is actually based on (a hash over) the other document fields.
So unfortunately you can't add metadata during replication. This would indeed be handy for this and other per-user vs. shared replication situations, but it's not something CouchDB currently supports, and it would break some optimizations to add support for it.
I could write the user id into each document within the per-user database but that seems redundant and adds the complexity of validation. Is there any other way?
Including something like a .user field in each document is the right solution.
As far as being redundant, I wouldn't think of it that way — or at least, not as a bad thing. You'll find with CouchDB (and like other NoSQL stores) there's a trend to "denormalize" data to begin with. Especially given the things replication lets me do operationally and architecturally, I'd much rather have a self-contained document than one that relies on metadata derived from a database name.
I'm not sure exactly how in your case an extra field will make validation more complex, so I can't fully speak to that. You do want to make sure the user writing the document has set it "honestly", and so yes there is a bit more complication, but usually not too burdensome in most cases.
I am building a tool that searches people based on a number of attributes. The values for these attributes are scattered across several systems.
As an example, dateOfBirth is stored in a SQL Server database as part of system ABC. That person's sales region assignment is stored in some horrible legacy database. Other attributes are stored in a system only accessible over an XML web service.
To make matters worse, the the legacy database and the web service can be really slow.
What strategies and tips should I consider for implementing a search across all these systems?
Note: Although I posted an answer, I'm not confident its a great answer. I don't intend to accept my own answer unless no one else gives better insight.
You could consider using an indexing mechanism to retrieve and locally index the data across all the systems, and then perform your searches against the index. Searches would be an awful lot faster and more reliable.
Of course, this just shifts the problem from one part of your system to another - now your indexing mechanism has to handle failures and heterogeneous systems, but that may be an easier problem to solve.
Another factor is how often the data changes. If you have to query data in real-time that goes stale very quickly, then indexing may not be practical.
If you can get away with a restrictive search, start by returning a list based on the search criteria corresponding to the fastest data source. Then join up those records with the other systems and remove records which don't match the search criteria.
If you have to implement OR logic, this approach is not going to work.
While not an actual answer, this might at least get you partway to a workable solution. We had a similar situation at a previous employer - lots of data sources, different ways of accessing those data sources, different access permissions, military/government/civilian sources, etc. We used Mule, which is built around the Enterprise Service Bus concept, to connect these data sources to our application. My details are a bit sketchy, as I wasn't the actual implementor, just an integrator, but what we did was define a channel in Mule. Then you write a simple integration piece to go between the channel and the data source, and the application and the channel. The integration piece does the work of making the actual query, and formatting the results, so we had a generic SQL integration piece for accessing a database, and for things like web services, we had some base classes that implemented common functionality, so the actual customization of the integration piecess was a lot less work than it sounds like. The application could then query the channel, which would handle accessing the various data sources, transforming them into a normalized bit of XML, and return the results to the application.
This had a lot of advantages for our situation. We could include new data sources for existing queries by simply connecting them to the channel - the application didn't have to know or care what data sources where there, as it only looked at the data from the channel. Since data can be pushed or pulled from the channel, we could have a data source update the application when, for example, it was updated.
It took a while to get it configured and working, but once we got it going, we were pretty successful with it. In our demo setup, we ended up with 4 or 5 applications acting as both producers and consumers of data, and connecting to maybe 10 data sources.
Have you thought of moving the data into a separate structure?
For example, Lucene stores data to be searched in a schema-less inverted indexed. You could have a separate program that retrieves data from all your different sources and puts them in a Lucene index. Your search could work against this index and the search results could contain a unique identifier and the system it came from.
http://lucene.apache.org/java/docs/
(There are implementations in other languages as well)
Have you taken a look at YQL? It may not be the perfect solution but I might give you starting point to work from.
Well, for starters I'd parallelize the queries to the different systems. That way we can minimize the query time.
You might also want to think about caching and aggregating the search attributes for subsequent queries in order to speed things up.
You have the option of creating an aggregation service or middleware that aggregates all the different systems so that you can provide a single interface for querying. If you do that, this is where I'd do the previously mentioned cache and parallize optimizations.
However, with all of that it you will need weighing up the development time/deployment time /long term benefits of the effort against migrating the old legacy database to a faster more modern one. You haven't said how tied into other systems those databases are so it may not be a very viable option in the short term.
EDIT: in response to data going out of date. You can consider caching if your data if you don't need the data to always match the database in real time. Also, if some data doesn't change very often (e.g. dates of birth) then you should cache them. If you employ caching then you could make your system configurable as to what tables/columns to include or exclude from the cache and you could give each table/column a personalizable cache timeout with an overall default.
Use Pentaho/Kettle to copy all of the data fields that you can search on and display into a local MySQL database
http://www.pentaho.com/products/data_integration/
Create a batch script to run nightly and update your local copy. Maybe even every hour. Then, write your query against your local MySQL database and display the results.
I run a sports program where i have a master roll of who is in which class in excel. I want to link this to a database in access that stores the other information about each athlete, e.g. address, parents name, school, medical details. I want to be able to add names to class in the excel speadsheet and have this automatically generate a record for that person in access. There also needs to be some failsafe for athletes that are in multiple classes. I was also doing class roles as pivot tables out of the access database so i need to code for classes and also have this allow for athletes in multiple classes/disciplines.
It is easy enough to update an Access table from Excel via ADO, after that it is very much about your tables and indexes. If you are not familiar with relational databases, you might like to read http://r937.com/relational.html. That being said, it would be a lot easier to work in Access and output to Excel when necessary.
I agree I think this is a classic case of trying to get excel to do something its not best for. If you try to create some kind of hybrid system with excel pushing data into access then it will end in tears at some point.
The best thing in this case would be to port the whole thing to some kind of database. If the number of uses and the usage falls into the range for access/jet then that would be a great choice. If more users/higher usage is going to be needed then maybe look to SQL express to hold the data and access as a front end.
There was a thread a few days ago about someone being sick when maintaining an access DB, he wanted to rewrite it in .net. The point of that thread boiled down to using the correct tool for the correct job. No one can blankly say “Access sucks, everything should be in SQL server/.net” because if used in the correct way and for the correct projects access is a great tool.
So to bring it back to this thread it looks like you have “outgrown” excel and should be looking at some kind of database with access being a strong candidate
If you want to display the data in Excel (so you can do sorts, filters, etc.) then you could store the data in Access as has been suggested, then instead of exporting a report every time you want to use it, link your Excel file to Access using a Database Query.
In Excel 2003 go to Data->Import External Data->New Database Query and create a new data source to your Access mdb.
That way your data is stored in a much better way, whilst still having the Excel viewability that everyone(?) loves.
I have a few large forms that I need to provide visual cues about the before/after state, so the person approving the form can see what has been modified (not the previous answer, tho that would be a plus). This is currently being done with an extra column for each column of data (Name, Name_IsModified, Phone, Phone_IsModified, etc...). I'm curious if there was a better way to getting around this, leveraging SubSonic?
The initial load is done by grabbing data from 6 source tables on 3 different servers. This data is saved in the form tables, where it resides until it is approved by various people who will manually update it into the live systems that then update the 6 source tables. The visual cues are primarily used during the approval process, but are occassionaly used to research when a change has been made in the past.
Since I have to make a bunch of updates, I thought this might be a good time to break away from the legacy 2000+ lines of code, making my job a bit easier!!!
Thanks,
Zach
All of the properties on SubSonic objects are actually collections and you can pull this out and review changes - all without reflection.
We have a "DirtyColumns" collection (not sure if it's public or not) that we use to run updates - this would be the thing you'd want to have a look at.