What is the programming model (language) behind the web pages that dynamically add contents based on the user interaction. For example, if you visit this site, there is a text box which user has to put some input. Then it will open some other components, e.g. a button and config page. As you can see the URL remains intact. The source pages shows that it uses JavaScript. So, what is the first step to follow that?
There is no specific language for the web, there's a lot. It really isn't very different from developing a regular desktop application. It just exists within the browser sandbox.
This question is for too general to be answered with any specifics. If you really want to make an interactive website - you just need a bit of html, css and some javascript.
Here's an example of some code that submits a form with an input:
https://www.w3schools.com/js/tryit.asp?filename=tryjs_form_submit
Related
I'm using js-crawler to crawl websites, and encountered now with CNN, that part of its landing page links are inside (urls that are generated dynamically, for some reason).
Thing is crawler don't really touch scripts - how should I address it? should I write my own code in addition to my node.js crawler? is there advanced crawler that knows how to handle this dynamic behavior?
Making my comment into an answer:
Crawling content that is generated by client-side Javascript is a complicated problem that not even Google has fully solved.
The only way to truly do it is to use some sort of head-less browser that is safely sandboxed on your server where the page is loaded into a browser-like environment where it can run its own scripts and generate its own content and then you can examine the resulting DOM.
Even then, it won't necessarily generate content that requires user interaction (like clicking on a tab to show some content).
I want to build a task-specific web browser. For example, when the user uses that browser, the program should give options like these:
What is your interest today? Please select from the following:
computer science
data science
web development
psychology
biology
social media
etc.
After the user selects their interest, the browser should allow the user to search or study only those selected content types, so the user cannot get diverted from their task. The purpose of this browser is to avoid time-wasting. Because whenever someone tries to study or do some professional work, they get diverted by opening social media tabs and movies.
Which programming language will be suitable for making this browser?
You could do this by making a custom version of an existing browser, as guest271314 suggested, but that would require far more effort than necessary. All you need to make is a browser extension, such as a Firefox Add-on or a Chrome Extension. Browser extensions are usually written with JavaScript, HTML, and CSS. Each browser needs to have an extension made individually, but you can usually share a lot of the code between them. Read the linked documentation for help creating an extension for each browser.
There are already existing browser extensions like you describe that prevent you from visiting sites that you put on a list ahead of time, such as StayFocusd and WasteNoTime. Maybe you just want to use one of those extensions instead of writing a new one.
Is it possible to show chrome extension icon inside the web page?I am able to create an extension using page action ,icon is shown inside the address bar ,but i need to show it inside the web page ...
Well, there is no special API you can simply use for that.
You can inject your own UI elements into the page, but it's going to be difficult. I will only outline this, don't ask for specific code.
You will need a Content Script (also, read the Architecture Overview page if you haven't yet) to access the contents of the page.
Then, you need to create and append your elements to the DOM of the page. In your case, you could possibly add an <img> tag with your icon and a click listener that does something.
Note that to avoid clashes with CSS of the page itself it is recommended to use Shadow DOM techniques.
Of course, this may or may not work well depending on the page you're injecting into.
Say someone else has a website generated by JavaScript, so I can't go look at the source and read what should be on the screen. How can I grab the text on the screen so I can feed it into another program? Also, how can I write a program that automatically clicks on radio buttons, links, etc. that satisfy certain criteria?
You can write a web scraping tool in Perl or Python. Or, you can use existing tools and frameworks to achieve that.
Check out Scrapy, an open-source tool written in Python.
Take a look at Selenium too.
To parse dynamic content you could see the javascript source and get that same content the same way the webpage is getting it. (ie. replicating ajax calls and such)
If you want to submit data (not actually click on the elements) as if it were clicked/edited/selected you could also send a request containing the same data that the server is expecting by using some HTTP library, like CURL. See an example here.
If you need to handle content generated by script, then your first problem is to cause the script to execute. Further, the script will want to generate the content into a DOM. That means you need to have a DOM, and a script engine, and probably HTTP access to the Internet, and XML handling, etc.
If that sounds a lot like a web browser, then you're listening.
What you basically need is a web browser that you can control from a program. You'll need to be able to tell it to browse to a page, click buttons and links, etc., then you'll need to read back the resulting DOM.
Only then will you need to parse the page.
If you're in the Microsoft world, then you can use the WebBrowser control. There are several forms of this, and they all amount to the same thing: you can have Internet Explorer run inside of your program, and your program can control it.
I understand there are other browsers that can be controlled from a program, but since I don't know their details, I'll wait for someone else to tell us both.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I am experienced in HTML and JavaScript but have never made a (real) Chrome Extension (other than the "Hello World' tutorial).
What I want to do is make an extension that adds functionality to viewing images. Basically I want a pieice of JavaScript to load when you go to a .jpg, .png, ect. page.
I couldn't get much out of the "Hello World" tutorial or the Docs and I've looked at code from XML Tree, but I'm having trouble understanding it.
I don't know where to go from here.
Since you're experienced in JavaScript and HTML, tell you what the best source is?
http://developer.chrome.com/extensions/getstarted.html
Documentation
The above link (Chrome Extension Documentation) is so simple to read once you get a hang of it. For example, I didn't know JavaScript when I created my first Chrome Extension (Reload all tabs). I recommend you to read the docs (will only take you an hour), and follow the steps that you need.
Make sure you understand the different UI's you can tap to
Browser Actions: You use browser actions to put icons in the main Chrome toolbar.
Context Menus: You use context menus to add items to Chromes context menu.
Desktop Notifications: Use desktop notifications to notify users that something is important.
Option Pages: If you want an options page, this is your guy!
Page Actions: If you want to override certain pages (New Tab, History, Bookmark Manager)
Make sure you realize there are many browser interactions that you can tap into:
Bookmarks: Access to your bookmarks
Cookies: Access to Cookies
History: Access to History
Management: Access to Extension Management
Tabs: Access to your Tabs
Windows: Access to your Windows (which has Tabs).
Make sure you understand the difference between Background Pages and Content Scripts, and their limitations.
Make sure you realize there are some neat functions in the Chrome.* API.
Make sure you understand permissions.
Many more
145 Chrome Questions I answered on Stackoverflow
If you want more examples, I have answered many questions regarding Chrome Extensions that might help you get started (145+ Questions Answered), for example, here are some of the more recent ones:
Content Script Skeleton
Send data from Background Page to Content Script
Communication with the embedding page, Simple fetching of the pages JS variable
Taking Screenshots in Chrome
Walkthrough building an extension
Executing JavaScript when a user clicks on a browser action.
How to capture selected text and send it to a web service
Walkthrough how to create an extension to access Disqus Comment Box
Walkthrough how to highlight the DIV that the mouse if hovering over
Walkthrough how to move to the top of the page extension
Walkthrough how to create a bubble overlay when you select text.
etc and 120 more answers for Chrome Extensions, the above is what I copied from the results on the first page. So a search in Google will get you many results.
Real open source extension I created
If you want to see real source code of the extensions I have written (some are super small, some are super big):
Reload all tabs extension
Open link in a foreground tab extension
HTML5 Haptics Chrome extension
Set Wallpaper extension
Prayer times extension
Facebook friend exporter extension
Proxy Anywhere extension
Resources
If you still want more resources, you can:
Read the documentation.
Read the Official Chromium Extension Mailing list
The unofficial Internet Relay Chatroom #chromium-extensions on irc.freenode.net
Google (Will bring you to Stackoverflow, or Quora)
I hope this helped!
well it’s not that tough to create a chrome extension ,anyone who has a basic understanding of HTML,CSS,Javascript can actually make one!
If you want to make a extension I have wrote an article on Medium it might be useful here's the link
Also I have attached my code on github here’s the link
Keep Learning!