I am using sqlserver 2008, How can I calculate the total memory occupied by the Database with tables (>30) and also data in it.
I mean if I have DB (DB_name) and with few tables(tblabc, tbldef......) with data in it, how to calculate the total memory occupied by the database in the server.
Kindly Help me.
Thanks
Ramm
See the sizes of mdf and log files
EDIT: Sql Server stores its db in mdf files(one or multiple). You need the lof file too. See where your db is stored and these files are files you need.
Be aware that if you are using FILESTREAM, the actual files are not in the db (mdf)
EDIT2:Books Online
When you create a database, you must either specify an initial size for the data and log files or accept the default size. As data is added to the database, these files become full.
So, there is a file with some size even if you have no data..
By default, the data files grow as much as required until no disk space remains.
...
Alternatively, SQL Server lets you create data files that can grow automatically when they fill with data, but only to a predefined maximum size. This can prevent the disk drives from running out of disk space completely.
If data is added (and there is no more space in the file)the file grows, but when it is deleted, it keeps its size, you need to shrink it...
I suppose that you refer to disk space and not memory. That would be very hard to get correct since you would have to know exactly how SQL Server stores the data, indexes and so on. Fortunately you do not have to calculate it, just fire up Microsoft SQL Server Management Studio. Right click on your database->Reports->Disk usage.
Related
The following is a screen shot of what I see on the dashboard of my Azure SQL database. I did check the documentation on this so I do understand what each data point means. What I'd like to understand is whether it's problematic for the used space to be that close to allocated space.
The only data point I set is the max storage space and the rest are managed by Azure SQL Database so it should be OK but I don't want to make assumptions.
Your allocated space will grow automatically, you don't have to worry about it, that is normal. You will always see used space close to allocated space. The key is when allocated space and used space are getting close to the maximum storage size.
If you see the database size reaching the maximum size you may need to run the following statement to increase the maximum size or adjust the maximum size using Azure portal.
ALTER DATABASE AzureDB2 MODIFY (EDITION='STANDARD', MAXSIZE= 50 GB)
I know from on-premise Data warehouse setups, you better enlarge the allocated space in larger chunks of the database than every time small chunks.. IMO you would like to tune the autogrowth when you want to insert large chunks of data.
I have a linux server on AWS that is hosting a postgres database server and a nodejs api server. It has 240GB RAM, and major portion is dedicated to the database. Approx 32GB is the memory is left open to the OS to deal among the
app server, various OS-level caches, and whatever need arises. There is no explicit fsync. The database + indexes add up to about 500-600 GB.
Within the database some operations may spill over to disk for want of enough work memory. E.g., a sort operation or a temp file. Thus, within a span
of say 1-2 seconds, a temp file may be created, written to, read back and then deleted. The temp file size I am guessing should seldom exceed 100MB.
Question - will the temp file in this scenario necessarily cause disk io? If not, how does one assess the likelihood? What parameters majorly influence this?
I want to know the max Excel File size which we can load into db using a Simple ETL SSIS package. If file size depends upon system configs or resources, Then how can we calculate it? In my case I am trying to load an excel file of 500+Mbs.
My Package gets hanged even while trying to map columns.
Thanks.
The only real limitation is the size of the machine's memory (RAM) where the package is running on, as SSIS loads the data into memory.
Thus, if you only have 2GB of RAM, I wouldn't try to load files bigger than 1 GB. (you must have RAM left for SQL Server to operate, and don't forget about all your other applications)
Also remember if you're not pipelining your data flows properly, and you have blocking parts like Aggregate or SQL Command objects, then you are going to be loading way more into memory than you should be.
The file size is not as important if you have no blocking parts. SSIS won't load the entire object into memory, and you can specify how much it uses. But if there are blocking parts, then it will need the entire object in memory.
Note that another big memory hog could be Lookup tasks with Full Caching - these can take large amounts of memory up if you are loading big tables.
Hope this helps.
I recently started using Couch for a large app I'm working on.
I database with 7907 documents, and wanted to rename the database. I poked around for a bit, but couldn't figure out how to rename it, so I figured I would just replicate it to a local database of the name I wanted.
The first time I tried, the replication failed, I believe the error was a timeout. I tried again, and it worked very quickly, which was a little disconcerting.
After the replication, I'm showing that the new database has the correct amount of records, but the database size is about 1/3 of the original.
Also a little odd is that if I refresh futon, the size of the original fluctuates between 94.6 and 95.5 mb
This leaves me with a few questions:
Is the 2nd database storing references to the first? If so, can I delete the first without causing harm?
Why would the size be so different? Had the original built indexes that the new one eventually will?
Why is the size fluctuating?
edit:
A few things that might be helpful:
This is on a cloudant couchdb install
I checked the first and last record of the new db, and they match, so I don't believe futon is underreporting.
Replicating to a new database is similar to compaction. Both involve certain side-effects (incidentally, and intentionally, respectively) which reduce the size of the new .couch file.
The b-tree indexes get balanced
Data from old document revisions is discarded.
Metadata from previous updates to the DB is discarded.
Replications store to/from checkpoints, so if you re-replicate from the same source, to the same location (i.e. re-run a replication that timed out), it will pick up where it left off.
Answers:
Replication does not create a reference to another database. You can delete the first without causing harm.
Replicating (and compacting) generally reduces disk usage. If you have any views in any design documents, those will re-build when you first query them. View indexes use their own .view file which also consumes space.
I am not sure why the size is fluctuating. Browser and proxy caches are the bane of CouchDB (and web) development. But perhaps it is also a result of internal Cloudant behavior (for example, different nodes in the cluster reporting slightly different sizes).
I recently encountered a situation where my CouchDB instance used all available disk space on a 20GB VM instance.
Upon investigation I discovered that a directory in /usr/local/var/lib/couchdb/ contained a bunch of .view files, the largest of which was 16GB. I was able to remove the *.view files to restore normal operation. I'm not sure why the .view files grew so large and how CouchDB manages .view files.
A bit more information. I have a VM running Ubuntu 9.10 (karmic) with 512MB and CouchDB 0.10. The VM has a cron job which invokes a Python script which queries a view. The cron job runs once every five minutes. Every time the view is queried the size of a .view file increases. I've written a job to monitor this on an hourly basis and after a few days I don't see the file rolling over or otherwise decreasing in size.
Does anyone have any insights into this issue? Is there a piece of documentation I've missed? I haven't been able to find anything on the subject but that may be due to looking in the wrong places or my search terms.
CouchDB is very disk hungry, trading disk space for performance. Views will increase in size as items are added to them. You can recover disk space that is no longer needed with cleanup and compaction.
Every time you create update or delete a document then the view indexes will be updated with the relevant changes to the documents. The update to the view will happen when it is queried. So if you are making lots of document changes then you should expect your index to grow and will need to be managed with compaction and cleanup.
If your views are very large for a given set of documents then you may have poorly designed views. Alternatively your design may just require large views and you will need to manage that as you would any other resource.
It would be easier to tell what is happening if you could describe what document updates (inc create and delete) are happening and what your view functions are emitting, especially for the large view.
That your .view files grow, each time you access a view is because CouchDB updates views on access. CouchDB views need compaction like databases too. If you have frequent changes to your documents, resulting in changes in your view, you should run view compaction from time to time. See http://wiki.apache.org/couchdb/HTTP_view_API#View_Compaction
To reduce the size of your views, have a look at the data, you are emitting. When you emit(foo, doc) the entire document is copied to the view to it is very instantly available when you query the view. the function(doc) { emit(doc.title, doc); } will result in a view as big as the database itself. You could also emit(doc.title, nil); and use the include_docs option to let CouchDB fetch the document from the database when you access the view (which will result in a slightly performance penalty). See http://wiki.apache.org/couchdb/HTTP_view_API#Querying_Options
Use sequential or monotonic id's for documents instead of random
Yes, couchdb is very disk hungry, and it needs regular compactions. But there is another thing that can help reducing this disk usage, specially sometimes when it's unnecessary.
Couchdb uses B+ trees for storing data/documents which is very good data structure for performance of data retrieval. However use of B-tree trades in performance for disk space usage. With completely random Id, B+-tree fans out quickly. As the minimum fill rate is 1/2 for every internal node, the nodes are mostly filled up to the 1/2 (as the data spreads evenly due to its randomness) generating more internal nodes. Also new insertions can cause a rewrite of full tree. That's what randomness can cause ;)
Instead, use of sequential or monotonic ids can avoid all.
I've had this problem too, trying out CouchDB for a browsed-based game.
We had about 100.000 unexpected visitors on the first day of a site launch, and within 2 days the CouchDB database was taking about 40GB in space. This made the server crash because the HD was completely full.
Compaction brought that back to about 50MB. I also set the _revs_limit (which defaults to 1000) to 10 since we didn't care about revision history, and it's running perfectly since. After almost 1M users, the database size is usually about 2-3GB. When i run compaction it's about 500MB.
Setting document revision limit to 10:
curl -X PUT -d "10" http://dbuser:dbpassword#127.0.0.1:5984/yourdb/_revs_limit
Or without user:password (not recommended):
curl -X PUT -d "10" http://127.0.0.1:5984/yourdb/_revs_limit