There's a very powerful CLI tool - wget. But it seems like it is bad for downloading wikis - where it download the whole database - instead of just downloading current versions of all articles... Perphaps there's a script (python or perl) to load each page of a given web-site in a browser - and download it.
Use HTTrack.
It can be called via command-line as well as via GUI. It's a feature-rich product. They have user guide for both command-line and GUI usage.
Related
Not quite sure if it possible at all, so decided to ask here.
I need to automate some things and search the way to drag'n'drop local file to browser window (with specific URL) via Terminal command for uploading.
I use Mac, but I think Linux will fit here as well.
If there is any solution or module on Bash / Python / Node.js I will gladly give it a try.
take a look at requests package in python language.
you can make a POST request and send information you want to the web server.
I want to edit GS (google app script) files in a decent editor (Emacs or VIM) and keep it under git, so I need to be able to install it to run it.
Right now I copy the whole file and paste it into the code window in the browser, which is clearly suboptimal.
I wonder if there is a way to do it.
For some reason, Edit With Emacs does not work with the script editor.
The google apps script
Importing and Exporting Projects guide details using the Drive REST API
Note: Import/Export of scripts is limited to Standalone scripts
Only standalone scripts can be imported or exported. Container-bound scripts cannot be accessed through the Google Drive REST API.
This Google repo details how to work with scripts in a local dev environment
It uses node-google-apps-script which is really all you need to import/export your script project to a local machine and back to Google
It is a little bit of work setting it up; there are quite a few steps to work through
Despite this question is a bit old, I consider it could be useful for someone still wanting to edit App Scripts outside the browser.
There is a tool named clasp which aims to provide a way to edit an app script project in a local environment: https://github.com/google/clasp.
You just need to install via:
npm install -g #google/clasp
And enable the Google Apps Script API: https://script.google.com/home/usersettings
To use it, you should include a package.json via npm init (tested with version 2.4.0) and then make:
clasp login
clasp clone "projectID"
Afterwards, you could edit the files with your favorite editor. Finally, you could push the changes via:
clasp push
For more information, please check the repository of the project, pointed out above.
There are so many plugins available which will help push to a webserver, but are there any that will download?
A Bit of Background
I'd like to automate the process of publishing my CMS-based website. The only issue is that our marketing people regularly blog and make content changes, so I'd like to first download the content files which have changed to my local development environment (.md files, which are not accessible from the web) before I push everything up to the staging server.
Does anything like this exist? I've searched NPM quite thoroughly, as well as this question which unfortunately didn't yield any results.
I did see a pretty robust cURL based plugin, however it doesn't support FTP authentication and since these files are not web-accessible directly, I'll need to leverage FTP.
A quick google search gave me this: https://www.npmjs.com/package/grunt-ftp
certain pages require confirmation(eg. check a checkbox to accept agreement) prior to download
http://www.oracle.com/technetwork/java/javase/downloads/jdk7-downloads-1880260.html
is it possible to download this from linux command line(no x server available)?
I use wget to directly download.
If you want a console only text web-browser, use lynx or links
If you want to script interaction, consider curl (and also libcurl, which is interfaced to many scripting languages, e.g. python etc).
I have a windows application which installs a Chrome extension via the windows registry. I wish for this application to generate some one-time information for Chrome to read based on information typed in by the user during the installation process.
Assuming I am not using NPAPI in the Chrome extension, is there anywhere the installer can place information such that the extension will see it?
Edit: I also wish to launching chrome at the end of the installation.
Another way you can pass information to an installed extension from outside of Chrome is to have a page with your extension that you then open Chrome too and pass the info in the hash...such as....
chrome.exe "chrome-extension://emcggffhhapbbkcodabdliakappfibcf/showHash.html#info"
Problem with this method is your installing the extension using the simple registry method (Im guessing) and not using the Policy method. With the Policy method you can force an install and it will happen even if Chrome is allready open (where as according to the docs the simple method happens the next time Chrome is opened). Downside to this is you will have to make an uninstaller yourself as you cant uninstall an extension from Chrome that is installed with this method. Im also not sure how quick/often it will be before the extension is installed (couldnt find it in the docs and too lazy to try it ;)) and youd need to make your installer wait a bit for it to be installed....
http://www.chromium.org/developers/how-tos/adding-new-policies
http://dev.chromium.org/administrators/policy-list-3#ExtensionInstallForcelist
http://dev.chromium.org/administrators/policy-templates (says where in the registry to add them)
Another possible method could be to pack the extension at install time and add a file with the info that the extension could read. Problem with this method is that the extensions ID would change (might not be a problem for you?) or youll have to include the PEM in your installer which you probably dont want to do....
chrome.exe --pack-extension="C:\simple-example" --no-message-box
Many people wish there were an event firing on extension installing.
There's a workaround, not elegant way to send info to the browser from outside: launch chrome asking to open an url.
I use it with a local html file. My application execute a command line like:
"pathToChrome\Chrome.exe" "file://pathToHtmlFile/myFile.html?param1=value1¶m2=value2"
The info I pass are the page's parameters.
The catch is that this page is read by the extensions in one of many ways:
You can write a content script this page will fire
You can put some javascript on this page to write down the parameters as cookies, for the extension to read in the future (without calling the extension at this time)
It hasn't to be a local page. If your page is on a server, it can save the parameters in the server, ir it worthy.
It hasn't to be even your page. You can call any page on Internet, but beeing sure it will fire your content script extension, and it will read your "customized" parameters.
Instead of communicating through the windows registry, you can create a WebSQL from the installer and from the extension read the data from there.
You will need to a bit of research about how to this, but this is possible. the steps should be:
The installer will create the database and register to chrome (maybe with the Databases.db)
The extension will use openDatabase to create a connection to the database
The extension will do a transaction and read the needed file.
Another option is to add file to the crx for example "installer_info.json" and do an AJAX request from the extension to the "installer_info.json" file.
There is no formal way for doing this things, little research and you will have a way.