For fun and maybe for profit, i want to implement the following:
scheduled or manually triggered process that logs into all my bank accounts
process knows bank site structure, and goes through "cashback/partner deals" pages
all deal information is collected in one place and in one format
when i'm going to buy something, i can quickly see if any of my cards has a special offer for that place. so that i can pay with the card that offers the best deal. ideally, this should happen on my android phone. that doesn't imply a standalone app though, e.g. you can search inside an email or googledoc or anything.
any ideas on implementation? don't limit yourself. suggest crazy things, as long as they work.
PS i did look for an existing website that offers something similar, but they all seem to focus on cards themselves rather than on specific deals for stores.
Depending on the bank website, your code may need to be able to execute JavaScript to interact with it. Take a look at CasperJS for the web scraping part.
Related
I've been doing a lot of research, and perhaps just need a few dots connected.
I have an idea for a mobile app/website that contains lists of local eating/drinking establishments along with the deals/specials they offer each day. The idea is to create an app that people can refer to in order to save money on a night out.
I'm familiar enough with HTML/CSS/JS to create a functioning website, but when it comes to backend I'm a little confused. Editing the markup in order to reflect changes (e.g. a new deal starts or new establishment opens up) is a bit cumbersome. Now I know I want a database with my information ready to be displayed on my page. Does this mean that I need to develop my own API for everything, and then make sure it integrates with the hosting website that I end up choosing?
I feel like I'm missing something that should make it obvious what the next step is. Can anyone offer any advice?
The short answer is yes, you are exactly right.
The long answer is that is definetly one way to do it. But, for large projext just using JS can get quite cumbersomoe on your client end. Usually the first level would be using something like ajax. It's a great way to start and you can go a long way with just ajax. This is acutually where most people "start" when using just javascript to make api calls. The next level would be to use a framework like Angular. This will of course do more for you than just help handle api calls and it requires a larger investment in learning.
So that is all client side...
Now for the server side part... When you publish a website you are now dealing with "server-side" content. You have taken your static content and it is served up from the server but it's always the same static content from the server then it becomes dynamic on the client when all the javascript starts getting parsed.
The API is another server side component. But instead of being static like your pages, a bunch of files just sitting there, it is an actual application on the server. It takes a command via an api request and then does its thinking and then spits out a response object dynamically to the requester, which in this case will be the JS on your site.
Now, if you don't like the idea of learning to make your own API there are resources out there that will host an api for you and give you a gui to build your own API. I can't recommend one because I have never used one, but I do work with businesses that do and they love the fact they don't have to hire a dev to make thier apis. The downside is they are tied to that service and limited to the functionality that the service offers. It's not a big limitation as the services are quire powerful but if you are going to be managing complex data sets then it would probably be better to learn to make your own api.
Hope that clears things up a bit for you!
I have developed a program using curses, everything is cool so far but I was wondering myself if there is a good pattern to split different views/panels of my program into smaller chunks callable by my main loop?
Further informations:
This program is a rather small automation tool/wizard aiming to ease our application for customers requiring the on-premise installation.
This wizard is a 3 steps one and it’s used to grab informations from our customer installation needs depending of it’s chosen architecture.
The first step is requesting the customer to give us its identification informations such has contract number, company name, licence number and preferred contact.
The second step is requesting the customer to give us informations about either he want a standalone installation (All-In-One install) or a N-Tiers installation plus the required informations like the requested custom SSL VHostName or Tiers IP/Credentials.
The third and final step is showing the customer a progress bar and informations of the required services (MySQL/HTTPd/HAProxy/PHP-FPM) and our application.
I know that I do not especially need to use curses library for such a program but our UX Team requested it as it is part of our customer experience with the solution.
You can look at the Forms library. It's a nice extension to ncurses that allows you to better manage input forms like yours. It offers a simple function interface to read the fields, change their properties, etc., as well as many different field types (including regexp-validated fields). In your case, you can simply create three forms, and post/unpost them in succession.
as such way to do things is not really usual, do not expect any framework to be available (like those available for WebUI for instance).
I so decided to create my own "Framework/factory" and so to be able to split every aspect of my app in a logic that would be similar of those used by web applications.
The source-code is dirty and really not pythonic, but it is well working so far and quite easy to maintain.
Thanks everyone for your answers and ideas.
I will soon be developing an application to log and priorities worker shifts. It only needs to be small, and simple, but I am wondering what framework to use. All that needs to happen is boxes with names are in 3 lists, and the user can switch these around at will. It must log the times, which I will need access to in real time.
Im new to application development of this sort, and would like any suggestions. I have time to learn new technologies / languages.
Portability / device independence would also be useful. Should I consider a Web Application in Javascript? or something more like Python.
Thanks.
Even if your application is going to be simple that does not mean that whole system will be simple too. I can imagine that your app will serve only as a front end to something much bigger. If that is a case and you really have freedom to choose what language you will develop your app with consider choosing something that you will feel comfortable to work with. But before you will make your decision I would go to whoever gave you that task and try to get as much information about it as you can because expected features can help in choosing technology.
First of all, it seems that it is up to you to decide if it should be web or desktop app. In my opinion it is completely wrong situation. You should get clear requirements on what kind of application customer expects and in what environment it should work. And I would not move a finger towards a code before somebody gave me that information. You write that portability and device independence would be useful but is it actual requirement or you just think it would be nice feature to deliver?
EDITED (to answer a question in comment)
Probably there is as much possible solutions as people that would answer you so in the end you will have to make your own choices.
One way of doing it (considering that you want to learn something new and have a challenge :) would be implement WCF service that would act as a data provider from your database (so every GetUsers(), GetVacationDays() methods would be in it) and it would take care of some business logic (for example CalculateMaxValidWorkingTime() or whatever). That service would be a real power horse of your system. Since you don't have clear requirement whether it should be desktop or web app you could satisfy both possibilities by creating thin clients that would communicate with your service and they would be just a pretty front ends. And if you keep in mind that you can consume webservices practically with everything from C++ to .NET (C#/VB) to Javascript to Python to PHP after you done with service you would not be constrained with one particular technology/language.
Regarding databases I won't advice because personally I hate dealing with them and it always was somebody else's task to provide me with pretty API :)
Like a lot of businesses my employer is dealing with the new world of PCI compliance by avoiding the hard stuff and redirecting our customers to a third-party payment service. The process will entail the customer entering order details into our system but then being redirected to the merchant bank's payment service for the entry of those all important card details.
We wish to retain the services of some business that periodically fills in stages 1 and 2 of our order form with some dummy data, presses place order and sees that the URL it ends up at is in fact the one we're expecting, a bit like a bot or a web spider.
If it finds we've been clickjacked it would alert us by text message or twitter feed or whatever the cool kids are using these days.
Does anyone know of a service that performs this function?
No, I don't believe that there is a service like this. Usually companies with specific testing needs like this will use QuickTest Pro.
I'm still in the process of going through some suggestions and hammering out what exactly we're going to do but almost all the info I've gained has come from:
http://www.softwareqatest.com/index.html
A devastatingly useful site which provides more than answers to this functional testing scenario. There are a couple of Web-Based services which execute QA Functional Testing scripts against your site and send alerts and reports if the tests fail.
The two I had a quick look at were http://www.dotcom-monitor.com/ and http://www.watchmouse.com/en/
The latter service uses Badboy scripts in its tests so you can home brew them and then upload to their server for regular execution.
I would like to create an app for a myBB forum. So the site on the forum will look nicer and much more cleaner on an iPhone or Android.
Is it possible without an API? It isn't my site ether.
everything is possible, it's just a matter of resources...
technically, you can write an app for everything on the web, but:
an API will tell you how you can do things with the site, without having to reverse engineer all pages/posts/..., and the format of every output resulting from post/get operations. reverse engineering may take a long time, and you will surely not come accross all possible results (error pages, bad authentication...);
an API is quite stable and is always updated with great care from the developpers so as not to break existing applications. without an API, there is no guarantees that your app will not break with the next release of the forum when it is upgraded;
a web API generally defines an output format which is easily parseable: many API outputs XML or JSON, which can be processed with standard libraries. without an API, the output format is plain HTML, which may be difficult to reorganize in order to show the results in a different format.
so, yes, you can definitely write an app for a myBB forum, but it may require a fair amount of work.
You can do, it's called screen scraping and is what was done before XML, the semantic web, SOAP, web services and then JSON apis tried to solve the problem better.
In screen scraping, you grab the site's HTML, parse it, get the data you want out of it, then do what you need with that data. It's more work, and breaks each time the site's layout changes, hence the history of improvements to it.
You mention the site in question is not yours. Many sites do not regard screen scraping as fair use, so check with the site's terms and conditions that you can legally create an app from the data posted there.
you can consider useing HTML5 ... do you think it doable for use app ?