Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 2 years ago.
Improve this question
I'm working on a Rust project that collects daily statistics for a web-site (number of requests, number of unique users, average latency etc.). I'd like to store this data in a compact key-value store where the key is a date (or a date string) and the value is an object that contain the statistics. I also need this data to be persisted to a file.
I don't have any special performance or storage requirements. That's why I don't want to use major DBs like Redis, MongoDB or Cassandra that require a separate installation and significant resources to run. I'd like something much simpler and lightweight.
The ideal solution for me would be a library that can read and write key-value data and persist it into a file. The data size I'm aiming for is around 1000-2000 records.
Can you recommend a library I can use?
I can recommend PickleDB-rs. I think it answers most of your requirements. PickleDB-rs is a Rust version of Python's PickleDB. It's intended for small DBs (I think 1000-2000 records should be ok) and the performance isn't guaranteed to be as great as large scale DBs, but for the purpose of dumping daily web-site stats into a file it should be sufficient.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 4 years ago.
Improve this question
Assume a site like https://www.wood-database.com/wood-finder/ (our working example). Each page of it has data on a wood species. Assuming we need to sort the woods by a ratio of its data, for example hardness/weight, the site's tools aren't very useful.
What would be useful, though, is passing that data into an excel, which could trivially calculate the ratio and sort.
What ways are there to automatically fill that sheet out? What other tools besides excel could do it?
You should have a look at python, it's perfectly fit for the job. You could use the request library together with beatifulsoup to begin with, then load all data into a Pandas Dataframe and simply export it to excel (standard funtionality of Pandas).
If you really want to scrape the site thoroughly, you could consider using Scrapy (https://scrapy.org/)
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 3 years ago.
Improve this question
I am making a web app using node and dynamodb. For sql database there is Sequelize and for mongodb there is mongoose. But for dynamodb, what should I use? Should I use dynamodb data mapper? Are there any other alternative for dyanmodb?
Why not using the one AWS has made ? DynamoDB Data mapper. Its a typescript project so it comes with types as well. I dont know the difference with this and the one you picked. Speaking for mine, it's a good choice for many applications. If your attributes are of types string,int,datetime... then it would help you. But I couldn't use it with nested types. If you a 'map' attribute and it has some mutable nested attributes the mapper cannot handle that. There is only just update item functionality.
In short,
If you are going with a schemeless model unlike SQL where you attributes vary for each entry
If you are doing an attribute specific operations like only updating a field of map instead of the entire map attribute (because UpdateItem updates all the nested fields)
Then I wouldn't suggest using mappers. Instead, like me, for such projects you can create small dynamodb modules (like clients) that handles all the DocumentClient operations. Document client becomes so natural and easy to use a while after you get used to it.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I am using Spark SQL to do some analysis.
I`m wondering is there any Front end projects can be used to view the result? I mean the analysis result not the job successful / faile status
For example, granafa， kibana, etc..
Regards
Mingwei
If you mean visualization of your results (like the ones you've mentioned) you might be interested in Apache Zeppellin. It's more like IPython Notebook so you can write your code there and visualize results.
Otherwise you'd have to tell us what is your storage format and where are you storing your results - maybe there are some visualization tools for it.
Actually if you store your results of Spark jobs in ElasticSearch you can use Kibana with it.
Otherwise, I don't think there is anything. The difference between what you are referring to (openTSDB and Elasticsearch) and Spark is that the latter is not a datastore.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 9 years ago.
Improve this question
I'm trying to implement a system using node.js in which a number of sites would contain js loaded from a common host, and trigger an action when some user visits n+ sites.
I suppose a nosql solution storing a mapping of ip address => array of sites visited would be preferable to a RDBM both in terms of performance and simplicity. The actions I need are "add to array if not there already" and getting the length of the array. Also, I wouldn't like it all to sit in memory all the time, since the db might get large some day.
What's a system that fits these requirements the best? MongoDB seems like a nice option given $addToSet exists, but maybe there's something better in terms of RAM usage?
When I hear about working with lists or sets, the first choice is Redis
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 7 years ago.
Improve this question
The scenario is this:
My company has 2000 customers, and we need to send the all 2000 customers a credit application via PDF. What complicates this is that certain parts of the PDF need to have customer data filled in before the form is sent to the customer. The data that needs to be filled in is currently inside of an Excel array.
As you probably guessed, what I want to do is write a script that takes the Excel array and for each row of data the script will fill it into the PDF form, save a copy of it, print the document, and repeat until all rows have been filled into their respective PDF forms.
My questions are as follows:
Is there one particular programming language (or framework) that is particularly well suited for this, and does it have a low learning curve? (I only know enough to write basic JavaScript at the moment)
In the recommended language/framework, what specifically will I need to learn? (aside from the basics like print, for each, if statements)
Are there any particular or general GOTCHAs I should watch out for in writing the script? Keep in mind, this will be the first computer script I will have ever created, so even basic/elementary GOTCHAs can come into play due to my total lack of experience.
EDIT
I should probably specify that I would prefer to write this script in Python if it is at all possible, simply because of all the good things I have heard about it so far.
There are applications that might be able to do this without programming. Here's one that costs $249
http://www.pureforms.com/Products/PFPrintMerge/pfprintmerge.htm
I have never used it -- but there are plenty of products if you search google. If you decide to go the programming route -- you need a PDF SDK for whatever language/framework you choose. There are many to choose from for .NET and Java.
EDIT: You asked for python. See this other SO question.