What language should I choose to make web focused bots? (to parse, cross-check...) - web

wonderful community here :)
I'm getting into programming and my main interest right now is making a bot to automate cross checking of web products and prices across different web platforms like ebay, amazon, etc.
What language do you suggest is the best for that purpose? Thank you.

Related

What programming languages are required for the ServiceM8 API?

I am currently on the look out for a backend web developer but before I search for one I need to know the programming languages to find the right person.
ServiceM8's customer service team told me Python for web and Swift for app. Can I confirm this? You can see their API doc here: https://developer.servicem8.com/docs/getting-started
I am not sure what the difference is here (I know they are both programming languages, but not sure when they'd be used).
Thanks!
Dmitri.

Constantly running web application

I want to build a constantly running forex trading application (even if the web page is not opened),
But I don't know what is the best way to do it, it's a software that should run constantly with constant internet connection, should I write it in Python as a PC application or web application ?, or is there anything better ?
Thank you for your support,
I don't see any point of making a web application in your case. Architecture of algorithmic trading systems is a broad subject but most of the time it's an application which is only connected to a market data provider and a broker. A trading system can be built as a web application for example for browsing portfolio, historical trades or when some external events trigger trades but you should focus on the strategy itself rather than on a time consuming UI.
You should learn some basics from books/online course depending on your programming language but I think that Python is a great choice for rapid prototyping of strategies. You can also try some existing platforms like Quantopian which can give a quick start for strategies testing.
Focus your efforts on thorough testing of the strategy because most of the time they look good on a paper but turn out to be an utter garbage in practice and without a strategy there is no need for an infrastructure.
There are lots of technologies for developing "always on" scanners; I won't scour the internet for you; however, to give you an example, you can use something like Azure Functions -- it can run your scanner on a schedule. It can be built with Python, if that's your language of choice.
Of course, there are a lot of third-party tools to choose from as well.

How to track user interest and display products based on their interest

I have an e-commerce website and I need to display products in homepage based on user interest like advertisement showing on facebook and google based on the search we did on the internet.
Is there any API from facebook or google or any website for fetching user interests?
I have wondered how facebook, google, and booking.com collaboratively tracking us even if they are different companies.
Is there any common gateway or common interface where these big companies share user info by making a system with cookies to track among all as a centralized big data model for behavioral advertising?
I am looking for answers from experts for this. I really need to build a system for tracking user interest based on their search on google and other websites.
So you need to use something called recommender systems, using a machine learning algorithm, you'll recommend products to people based on ratings and interest. by using previous data of the same user or other users (just like when you get recommended videos on youtube).
this topic is too big for me to explain it step by step in here, and you need to first have a good understanding of primitive machine learning such as classification, regression ... etc
so if you're interested make sure to check a coursera course called Machine Learning (Stanford University) it's taught by machine learning rockstar Andrew NG, and it doesn't only teach you machine learning, but it takes you from somebody with no idea on the subject to an expert (technically) the course used MATLAB/OCTAVE and it has an entire section on Recommender Systems which is what you need, after you've finished just implement what you learned in the language of your choice !
PS:
you can always look up tutorial online for implementing Recommender Systems, but you will waste so much time because you would have no idea about what you're doing without understanding the theory which you will master in the course I've recommended, the course can also be found easily on youtube. but taking it for FREE in coursera will help you more because you'll have hands-on programming experience, on the different subjects.
For that google and facebook has their algorithm to track user movement and they use it for showing ads on their website.
i don't think that is available for common use.
I think you won't be able to Track there interest live Facebook, Google, Amazon, twitter. They are collecting your Interest form there own platform.
Also they manage large Add provider. SO once a Add has been clicked by you, it has been tracked. Also Google Play, itunes has access to your Phone.

Resource for developing a website

Can anyone recommend resources to learn how to develop websites, as opposed to web applications?
I am looking to develop a website for a consulting company to be precise. I would be more interested in best practices for creating the layout of a website (user appeal, eye candy, not an eye sore)
Thanks
-M
It really depends upon the language you want to use, your current skill sets, who's going to maintain the site, what OS the site will be hosted on etc etc.
I suspect you need to narrow down your question.
What do you mean by web site rather than web application? Are you talking about the dynamic nature of the content or somethign else?
update
If you're looking for discussions on design of websites (visual design, UX etc) then I'm a great fan of Smashing Magazine.
http://www.smashingmagazine.com/
It doesn't often speak about MS technologies (ASP.NET etc) but it's a great place to see discussions and papers on "what makes a great website". Some recent examples:
http://www.smashingmagazine.com/2009/05/15/optimizing-conversion-rates-its-all-about-usability/
http://www.smashingmagazine.com/2009/05/14/non-profit-website-design-examples-and-best-practices/
Subscribe to their RSS feed and see what those colouring-in people get up to.
Here's your first port of call.
Unless you're artistically inclined, I recommend purchasing or contracting the template design to someone who is skilled in this area.
For $60 a year, you can have unlimited downloads and unlimited use of all the templates at the following site:
http://www.dreamtemplate.com/
There are many more here:
http://www.templatemonster.com/website-templates.php
http://www.w3schools.com/
for purely informational sites, html, and css will probably be plenty, though I think I would reccomend using wordpress if you're just trying to put content on the internet
If you speak German or French, http://www.selfhtml.org is quite a good resource.
Otherwise, I would recommend http://www.w3schools.com/ or http://htmldog.com/. Both are very good as they really go deeply into the matter and tell about standards from the beginning.
sitepoint.com
Their best content is packaged in their books, but their articles are good, too. Covers design best-practices and web standards, but also has good tips on the business of web design and managing clients.
You may want to look at the alistapart website.
simply the best I have seen for this.
I would also - since I have just been reminded of it use
http://www.webmonkey.com/
http://w3schools.com/
http://www.w3schools.com/ is a good start.

Web programming language

I want to go learn web programming,but besides names and a little of html I don't know anything.So I ask you what programming languages you recommend,why,what can be done with it,tools to learn ,etc.I don't know if it matters but I program in a Linux enviroment(Ubuntu).
I don't want to do hardcore web programming ,I only want to be able to develop complete websites and understand network concepts.
Well, most cheap/free web hosts support PHP, MySQL, and most browsers support Javascript.
Target those and you'll be on a reasonably good path.
Some support Ruby and Python, and you might choose that path if you want to learn those languages.
Good luck!
-Adam
Definitely start with HTML, and basic CSS. These are the core of web programming, and you need to understand them well to do anything of consequence.
Once you've got those down, you'll want to move on to a server-side language. The easiest is PHP, but be careful of picking up bad habits, since it's a loose environment; if you stick with PHP, you'll eventually want to use it with an MVC framework such as CodeIgniter, which encourage separating presentation and logic. To get a little more advanced, you can also try Python or Ruby. Get yourself some server space to mess around with; most shared hosting plans are $5-10/month.
For Javascript/AJAX, you'll probably want to start by using Firebug or Firefox's DOM inspector to learn the relationship between the HTML of a page and the DOM object which Javascript uses to interact with it. Once you understand how the DOM works, toy around with jQuery to start doing useful (and fun!) stuff.
You'll also eventually want to learn MySQL (or a similar SQL variant), but that can probably wait, since you can do lots of interesting things without tackling database stuff.
Above all, be patient and persistent, and make use of every resource at your disposal: books, Google, Stack Overflow, and cheat sheets.
Ruby all the way. It's exactly what you need if you're interested in web dev and completely starting from scratch programming-wise. From the basics of Ruby it's pretty easy to get into Rails, which is a very beginner-friendly web framework.
Many great books on Amazon (look for the highest rated of course) on both Ruby and Rails.
Great starting point for links:
http://www.ruby-lang.org/en/documentation/
As well check out Learn to Program
Now get learnin'.
Google App Engine offers free hosting for applications that do not exceed the specified limits. The server-side language is Python, the client-side language is JavaScript inside Django templates. Everything has worked nicely for me in Ubuntu 8.10.
GREAT Question,
a couple of years ago I was in the same place -
What HONESTLY Helped me was using Dreamweaver, I set it in split mode and started watching the code
I used this and started delving into the PHP Application world and could see what was going on (sort of)
I know you can use dreamweaver with wine on the linux, and it can help you do a TON
granted if you continue it has the potential to limit you to the dreamweaver world... but can help you learn and create at the same time
(I build full php apps from scratch now with a notepad... but I started with DW)
The question I'd have is what kind of scale on web programming are you wanting to do? If it is small stuff then the LAMP stack would be my suggestion while if you want to get more into 3-tier architecture then Java or ASP.Net may be worth getting into for middleware or business logic code.
With the exception of the reference to the LAMP stack above, there has been no Perl recommendations. I like Perl as it is easy enough to build a fairly full featured web application (using CGI, or mod_perl). Of course, you are going to have to learn HTML/CSS if you are going to do anything on the web. I feel Perl is a good choice for web development as it is fairly robust and full featured with all of the modules available on CPAN. Combine with an application framework like CGI::Application or Catalyst and you can build sophisticated web apps in a short amount of time. Also, using a tool like XAMPP can help as you won't have to worry about web server or database cofigurations to get started.

Resources