Screenshot of Website
I am trying to write a Bot Program that purpose is to simulate Human accessing the website to make booking.
So basic things is that whenever 7AM, all people start accessing to the website and it makes the booking very difficult.
So I decided to write an automatic bot programming to keep accessing the website until successful (possible with a Timing Parameter on Initial Access like 7AM -0.5MS).
One difficulty is that when accessing, we need to input 3-5 digits verification code from a picture (usually different each time).
So I m thinking whether I should start doing this project using JavaScript or Node.JS.
I programmed C# before, so web programming is completely new.
That's why I wanna ask you guys what kind of language I should use, and is there any reference material or article you can share?
Thanks and Warm regards,
Wilson
Website link: https://w1.leisurelink.lcsd.gov.hk/leisurelink/application/checkCode.do?flowId=2&lang=EN
well i would recommend using puppeteer its fairly simple to use, basically get your main function going with whatever you want to do and put a timeout on it, you can get Local time to start code execution at 7:00 and keep executing either until u get what u want out of it or say until the loop reaches 150-200 attempts
Related
I've been doing a lot of research, and perhaps just need a few dots connected.
I have an idea for a mobile app/website that contains lists of local eating/drinking establishments along with the deals/specials they offer each day. The idea is to create an app that people can refer to in order to save money on a night out.
I'm familiar enough with HTML/CSS/JS to create a functioning website, but when it comes to backend I'm a little confused. Editing the markup in order to reflect changes (e.g. a new deal starts or new establishment opens up) is a bit cumbersome. Now I know I want a database with my information ready to be displayed on my page. Does this mean that I need to develop my own API for everything, and then make sure it integrates with the hosting website that I end up choosing?
I feel like I'm missing something that should make it obvious what the next step is. Can anyone offer any advice?
The short answer is yes, you are exactly right.
The long answer is that is definetly one way to do it. But, for large projext just using JS can get quite cumbersomoe on your client end. Usually the first level would be using something like ajax. It's a great way to start and you can go a long way with just ajax. This is acutually where most people "start" when using just javascript to make api calls. The next level would be to use a framework like Angular. This will of course do more for you than just help handle api calls and it requires a larger investment in learning.
So that is all client side...
Now for the server side part... When you publish a website you are now dealing with "server-side" content. You have taken your static content and it is served up from the server but it's always the same static content from the server then it becomes dynamic on the client when all the javascript starts getting parsed.
The API is another server side component. But instead of being static like your pages, a bunch of files just sitting there, it is an actual application on the server. It takes a command via an api request and then does its thinking and then spits out a response object dynamically to the requester, which in this case will be the JS on your site.
Now, if you don't like the idea of learning to make your own API there are resources out there that will host an api for you and give you a gui to build your own API. I can't recommend one because I have never used one, but I do work with businesses that do and they love the fact they don't have to hire a dev to make thier apis. The downside is they are tied to that service and limited to the functionality that the service offers. It's not a big limitation as the services are quire powerful but if you are going to be managing complex data sets then it would probably be better to learn to make your own api.
Hope that clears things up a bit for you!
For fun and maybe for profit, i want to implement the following:
scheduled or manually triggered process that logs into all my bank accounts
process knows bank site structure, and goes through "cashback/partner deals" pages
all deal information is collected in one place and in one format
when i'm going to buy something, i can quickly see if any of my cards has a special offer for that place. so that i can pay with the card that offers the best deal. ideally, this should happen on my android phone. that doesn't imply a standalone app though, e.g. you can search inside an email or googledoc or anything.
any ideas on implementation? don't limit yourself. suggest crazy things, as long as they work.
PS i did look for an existing website that offers something similar, but they all seem to focus on cards themselves rather than on specific deals for stores.
Depending on the bank website, your code may need to be able to execute JavaScript to interact with it. Take a look at CasperJS for the web scraping part.
I'll be the first to admit that my programming experience and skill in web services is practically non-existent. I usually program things that run completely isolated or locally, with either C or assembly. I'm proficient enough to get a website going, with some basic authentication and directory read access on the system. That's about it.
I'm trying to do a project that's well outside of my comfort zone and get some experience in controlling stuff remotely/via web. Using a Raspberry Pi running Debian, I'm running a program on it in C that takes in information such as video and UART data, does some crunching and triggers some outputs and writes events to a file/folder. This component is fairly straightforward to get running automatically. Getting a webserver up so a remote user can look at the files and pictures the driver program creates is also extremely easy.
The problem for me comes in trying to make a GUI on a webpage that can be used to manually control these outputs. I'm going to need some scripting to handle the button presses on the web page, clearly, but is there a scripting language in particular that stands out for using kernel objects/system calls so I can actually talk to that process? I figure the best way is to use message queues, but I don't know if Python or PHP (or another scripting language) are capable of doing this, and if there are any that are better at this than others. What is the preferred way of doing this?
I know it's possible since we've all seen those kitten-cams with the flash container where you can move the camera or trigger things. I just have no idea where to start.
Thanks for any help
Java can call native commands via JNI (http://en.wikipedia.org/wiki/Java_Native_Interface) from within a JVM. So if you already have C code that can handle the controls, it's just a matter of getting Java code to call them.
As for the scripts to handle button presses, there are several options. One way is to do it asynchronously via AJAX (which requires some JavaScript knowledge) or the other is by doing the traditional page refresh on each press. Sorry to be a bit vague on the answer, but this requires a lengthy explanation of how the whole JSP (Java Server Pages)/Servlets eco system works.
Here's a good place to start:
http://www.apl.jhu.edu/~hall/java/Servlet-Tutorial/
I will soon be developing an application to log and priorities worker shifts. It only needs to be small, and simple, but I am wondering what framework to use. All that needs to happen is boxes with names are in 3 lists, and the user can switch these around at will. It must log the times, which I will need access to in real time.
Im new to application development of this sort, and would like any suggestions. I have time to learn new technologies / languages.
Portability / device independence would also be useful. Should I consider a Web Application in Javascript? or something more like Python.
Thanks.
Even if your application is going to be simple that does not mean that whole system will be simple too. I can imagine that your app will serve only as a front end to something much bigger. If that is a case and you really have freedom to choose what language you will develop your app with consider choosing something that you will feel comfortable to work with. But before you will make your decision I would go to whoever gave you that task and try to get as much information about it as you can because expected features can help in choosing technology.
First of all, it seems that it is up to you to decide if it should be web or desktop app. In my opinion it is completely wrong situation. You should get clear requirements on what kind of application customer expects and in what environment it should work. And I would not move a finger towards a code before somebody gave me that information. You write that portability and device independence would be useful but is it actual requirement or you just think it would be nice feature to deliver?
EDITED (to answer a question in comment)
Probably there is as much possible solutions as people that would answer you so in the end you will have to make your own choices.
One way of doing it (considering that you want to learn something new and have a challenge :) would be implement WCF service that would act as a data provider from your database (so every GetUsers(), GetVacationDays() methods would be in it) and it would take care of some business logic (for example CalculateMaxValidWorkingTime() or whatever). That service would be a real power horse of your system. Since you don't have clear requirement whether it should be desktop or web app you could satisfy both possibilities by creating thin clients that would communicate with your service and they would be just a pretty front ends. And if you keep in mind that you can consume webservices practically with everything from C++ to .NET (C#/VB) to Javascript to Python to PHP after you done with service you would not be constrained with one particular technology/language.
Regarding databases I won't advice because personally I hate dealing with them and it always was somebody else's task to provide me with pretty API :)
I want build a search service for one particular thing. The data is freely available out there, via free classified services, and a host of other sites.
Are there any building blocks, e.g. open-source crawlers that I would customize - rather than build from scratch, that I can use?
Any advice on building such a product? Not just technical, but any privacy/legal things that I might need to take into consideration.
E.g. do I need to 'give credit' where the results are from and put a link to the original - if I get them from many places?
Edit: By the way, I am using GWT with JS for the front-end, haven't decided on the language for the back-end. Either PHP or Python. Thoughts?
There are few blocks in python you can use.
beautifulsoup [http://www.crummy.com/software/BeautifulSoup/] for parsing HTML. It can handle bad code too, and its API is veeery easy... way better than any DOM-like tool for me. My friend used it to scrape his old phpbb forum with success. It has pretty good docs.
mechanize [http://wwwsearch.sourceforge.net/mechanize/] is a webbrowser-simulating http client library. It handles cookies, filling forms and so on. Also easy to use, but it helps if you understand how does http work.
http://dev.scrapy.org/ -- this is a relatively new thing: a whole scraping framework based on twisted. I haven't played with it much.
I use first two for my needs; f.e. it needs 20 lines of code to get an automatic testing tool for a 3-stage poll, with simulation of waiting for user entering data and so on.
I made a screen-scraper in Ruby that took like five minutes. Apparently this dude has it down to 60 seconds! I'm not sure if Ruby is as scalable or fast as what you're looking for, but I've never seen a faster route to a proof-of-concept or a prototype.
The secret is a library called "hpricot", which was built for exactly this purpose.
I don't know anything about PHP or Python or what's available for those development systems/languages.
Good luck!