I have created a Chrome Extension that reroutes downloads from a given website and places them into a specific folder with an extracted number from the URL. It does this
by listening for the "onCreated" event for downloads in Chrome, and it modifies the download URL and file name if the URL starts with the given website.
My issue is, when this Extension is active and I start up Chrome the Download History automatically starts downloading. The only way I was able to fix this was to clear the Download History, which is not ideal if I want to pass this out to people and I am not sure if this will crop up again. I have tried adding a "chrome.downloads.erase" into my code but this did not help unfortunately.
I'm really confused as to why this happens, my code doesn't touch the Download History. Does anyone with a better understanding of Chrome and it's API have any idea's to why this is happening?
Thanks,
Jordan
My code:
service-worker.js
chrome.downloads.onCreated.addListener(function(downloadItem) {
console.log("Download has been triggered");
//Checks for downloads from Example.com
if (downloadItem.url.startsWith("https.www.example.com")){
console.log("The download is from Example.com");
//need to cancel original download
chrome.downloads.cancel(downloadItem.id)
//Gets ID of Current Tab
var currentTab
chrome.tabs.query({currentWindow: true, active: true }, function(tabs1){
currentTab = tabs1[0];
//Splits URL into an Array so it can pull Case Number
const myArray = currentTab.url.split("/");
let CaseNumber = myArray[myArray.length - 1];
console.log("Case Number:", CaseNumber);
//Splits URL into an Array so it can pull File Name
const myArray1 = downloadItem.url.split("/")
let NewFileName = myArray1[myArray1.length - 1]
//Sets Directory and adds Case Number to it
const dir = "Cases\\"
let FullPath = dir.concat(CaseNumber);
//console.log(FullPath);
//Adds the Filename to the Path
let FileNameAndPath = FullPath.concat("\\");
FileNameAndPath = FileNameAndPath.concat(NewFileName)
//remove File from Downloads of Chrome
chrome.downloads.erase({id: downloadItem.id}, function() {
});
console.log("Path:", FileNameAndPath);
//Sets download location and filename
chrome.downloads.download({
url: downloadItem.finalUrl,
filename: FileNameAndPath,
});
});
}
});
Related
Goal: Try to download a pdf file from Amazon S3 to my local machine via a NodeJS/VueJS application without creating a file on the server's filesystem.
Server: NodeJs(v 18.9.0) Express (4.17.1)
Middleware function that retrieves the file from S3 and converts the stream into a base64 string and sends that string to the client:
const filename = 'lets_go_to_the_snackbar.pdf';
const s3 = new AWS.S3(some access parameters);
const params = {
Bucket: do_not_kick_this_bucket,
Key: `yellowbrickroad/${filename}`
}
try {
const data = await s3
.getObject(params)
.promise();
const byte_string = Buffer.from(data.Body).toString('base64');
res.send(byte_string);
} catch (err) {
console.log(err);
}
Client: VueJS( v 3.2.33)
Function in component receives byte string via an axios (v 0.26.1) GET call to the server. The code to download is as follows:
getPdfContent: async function (filename) {
const resp = await AxiosService.getPdf(filename) // Get request to server made here.
const uriContent = `data:application/pdf;base64,${resp.data}`
const link = document.createElement('a')
link.href = uriContent
link.download = filename
document.body.appendChild(link) // Also tried omitting this line along with...
link.click()
link.remove() // ...omitting this line
}
Expected Result(s):
Browser opens a window to allow a directory to be selected as the file's destination.
Directory Selected.
File is downloaded.
Ice cream and mooncakes are served.
Actual Results(s):
Browser opens a window to allow a directory to be selected as the file's destination
Directory Selected.
Receive Failed - Network Error message.
Lots of crying...
Browser: Chrome (Version 105.0.5195.125 (Official Build) (x86_64))
Read somewhere that Chrome will balk at files larger than 4MB, so I checked the S3 bucket and according to Amazon S3 the file size is a svelte 41.7KB.
After doing some reading, a possible solution was presented that I tried to implement. It involved making a change to the VueJs getPdfContent function as follows:
getPdfContent: async function (filename) {
const resp = await AxiosService.getPdf(filename) // Get request to server made here.
/**** This is the line that was changed ****/
const uriContent = window.URL.createObjectURL(new Blob([resp.data], { type: 'application/pdf' } ))
const link = document.createElement('a')
link.href = uriContent
link.download = filename
document.body.appendChild(link) // Also tried omitting this line along with...
link.click()
link.remove() // ...omitting this line
}
Actual Results(s) for updated code:
Browser opens a window to allow a directory to be selected as the file's destination
Directory Selected.
PDF file downloaded.
Trying to open the file produces the message:
The file “lets_go_to_the_snackbar.pdf” could not be opened.
It may be damaged or use a file format that Preview doesn’t recognize.
I am able to download the file directly from S3 using the AWS S3 console with no problems opening the file.
I've read through similar postings and tried implementing their solutions, but found no joy. I would be highly appreciative if someone can
Give me an idea of where I am going off the path towards reaching the goal
Point me towards the correct path.
Thank you in advance for your help.
After doing some more research I found the problem was how I was returning the data from the server back to the client. I did not need to modify the data received from the S3 service.
Server Code:
let filename = req.params.filename;
const params = {
Bucket: do_not_kick_this_bucket,
Key: `yellowbrickroad/${filename}`
}
try {
const data = await s3
.getObject(params)
.promise();
/* Here I did not modify the information returned */
res.send(data.Body);
res.end();
} catch (err) {
console.log(err);
}
On the client side my VueJS component receives a Blob object as the response
Client Code:
async getFile (filename) {
let response = await AuthenticationService.downloadFile(filename)
const uriContent = window.URL.createObjectURL(new Blob([response.data]))
const link = document.createElement('a')
link.setAttribute('href', uriContent)
link.setAttribute('download', filename)
document.body.appendChild(link)
link.click()
link.remove()
}
In the end the goal was achieved; a file on S3 can be downloaded directly to a user's local machine without the application storing a file on the server.
I would like to mention Sunpun Sandaruwan's answer which gave me the final clue I needed to reach my goal.
I'm quite confused on how to use the Amplify library to actually download an mp3 file stored in my s3 bucket. I am able to list the bucket contents and parse it all out into a tree viewer for users to browse the various files, but once I select a file I can't get it to trigger a download.
I'm confident my amplify configuration is correct since I can see all my expected directories and when I select the file I want to download, I see the response size being correct:
You can see it takes 2+ seconds and appears to be downloading the data/mp3 file, but the user is never prompted to save the file and it's not in my Downloads folder.
Here is a capture of my file metadata setup from my bucket:
And the method I'm calling:
getFile (fileKey) {
Storage.get(fileKey, {download: true})
}
Without the "download : true" configuration, I get the verified URL back in the response. I'd like to avoid making a 2nd request using that URL download the file if possible. Anything else I may have missed? Is it better for s3 operations to go back to the standard aws-sdk? Thanks in advance!
I ended up using a combination of this answer:
https://stackoverflow.com/a/36894564
and this snippet:
https://gist.github.com/javilobo8/097c30a233786be52070986d8cdb1743
So the file gets downloaded in the response data(result), I added more meta data tags to the files to get the file name and title. Finally adding the link to the DOM and executing a click() on it saves the file named correctly. Full solution below:
getFile (fileKey) {
Storage.get(fileKey, {download: true}).then(result => {
console.log(result)
let mimeType = result.ContentType
let fileName = result.Metadata.filename
if (mimeType !== 'audio/mp3') {
throw new TypeError("Unexpected MIME Type")
}
try {
let blob = new Blob([result.Body], {type: mimeType})
//downloading the file depends on the browser
//IE handles it differently than chrome/webkit
if (window.navigator && window.navigator.msSaveOrOpenBlob) {
window.navigator.msSaveOrOpenBlob(blob, fileName)
} else {
let objectUrl = URL.createObjectURL(blob);
let link = document.createElement('a')
link.href = objectUrl
link.setAttribute('download', fileName)
document.body.appendChild(link)
link.click()
document.body.removeChild(link)
}
} catch (exc) {
console.log("Save Blob method failed with the following exception.");
console.log(exc);
}
})
}
}
I am downloading a file through puppeteer into my directory. I need to upload this file to an s3 bucket so I need to pick up the file name. But the problem is, this file name has a time stamp that changes every time so I can't keep a hard coded name. So is there a way around this to get a constant name every time (even if the old file is replaced), or how to rename the file being downloaded?
I thought of using node's fs.rename() function but that would again require the current file name.
I want a constant file name to hard code and then upload into the s3 bucket.
await page._client.send('Page.setDownloadBehavior', {behavior: 'allow', downloadPath: './xml'}); // This sets the directory
await page.keyboard.press('Tab');
await page.keyboard.press('Enter'); // This downloads an XML file.
You have two options:
Monitor the requests/responses to log the name of the file and rename it via Node.js
Use the Chrome DevTools Protocol to modify the response header
Option 1: Monitor the requests / response
This is the most straight-forward way to do it. Monitor all responses and in case you notice the response that is being downloaded, use the name to rename it locally via fs.rename.
Code Sample
const path = require('path');
// ...
page.on('response', response => {
const url = response.request().url();
const contentType = response.headers()['content-type'];
if (/* URL and/or contentType matches pattern */) {
const fileName = path.basename(response.request().url());
// handle and rename file name (after making sure it's downloaded)
}
});
The code listens to all responses and wait for a specific pattern (e.g. contentType === 'application/pdf'). Then it takes the file name from the request. Depending on your use case, you might want to check the Content-Disposition header in addition. After that, you have to wait until the file is downloaded (e.g. file is present and file size does not change) and then you can rename it.
Option 2: Use the Chrome DevTools Protocol to modify the response header
I'm 99% sure, that this is possible. You need to intercept the response which is currently not supported by puppeteer itself. But as the Chrome DevTools Protocol is supporting this functionality, you can use it using the low-level protocol.
The idea is to intercept the response and change the Content-Disposition header to your desired file name.
Here is the idea:
Use chrome-remote-interface or a CDP Session to activate Network.requestIntercepted
Listen for Network.requestIntercepted events
Send Network.getResponseBodyForInterception to receive the body of the response
Modify the body and add (or change) the Content-Disposition header to include your filename
Call Network.continueInterceptedRequest with your modified response
Your file should then be save with your modified file name. Check out this comment on github for a code sample. As I already explained it is a rather sophisticated approach as long as puppeteer does not support modifying responses.
You can save the file using GUID as the filename and rename it when the download is completed.
const puppeteer = require('puppeteer');
const fs = require('fs');
const path = require('path');
const downloadFolder = path.resolve('./DOWNLOAD-FOLDER-HERE');
// Act like a dictionary storing the filename for each file with guid
let guids = {};
const browser = await puppeteer.launch({
headless: false
});
let client = await browser.target().createCDPSession();
await client.send('Browser.setDownloadBehavior', {
behavior: 'allowAndName', //allow downloading file and save the file using guid as the filename
downloadPath: downloadFolder, // specify the download folder
eventsEnabled: true //set true to emit download events (e.g. Browser.downloadWillBegin and Browser.downloadProgress)
});
client.on('Browser.downloadWillBegin', async (event) => {
//some logic here to determine the filename
//the event provides event.suggestedFilename and event.url
guids[event.guid] = 'FILENAME.pdf';
});
client.on('Browser.downloadProgress', async (event) => {
// when the file has been downloaded, locate the file by guid and rename it
if(event.state === 'completed') {
fs.renameSync(path.resolve(downloadFolder, event.guid), path.resolve(downloadFolder, guids[event.guid]));
}
});
when reading BOOKS on scribd.com the download functionality is not enabled. even browsing through the html source code I was unable to download the actual book. Great stuff ... but HOW did they do this ?
I am looking to implement something similar, to display a pdf (or converted from pdf) in such a way that the visitor cannot download the file
Most solutions I have seen are based on obfusticating the url.. but with a little effort people can find the url and download the file. ScribD seems to have covered this quite well..
Any suggestions , ideas how to implement such a download protection ?
It actually works dinamically building the HTML based on AJAX requests made while you're flipping pages. It is not image based. That's why you're finding it difficult to download the content.
However, it is not that safe for now. I present a solution below to download books that is working today (27th Jan 2020) not for teaching you how to do that (it is not legal), but to show you how you should prevent (or, at least, making it harder) users from downloading content if you're building something similar.
If you have a paid account and open the book page (the one that opens when you click 'Start Reading'), you can download an image of each book page by loading a library such as dom-to-image.
For instance, you could load the library using the developer tools (all code shown below must be typed in the page console):
if (injectDomToImage == undefined) {
var injectDomToImage = document.createElement('script');
injectDomToImage.src = "https://cdnjs.cloudflare.com/ajax/libs/dom-to-image/2.6.0/dom-to-image.min.js";
document.getElementsByTagName('head')[0].appendChild(injectDomToImage);
}
And then, you could define functions such as these:
function downloadPage(page, prefix) {
domtoimage.toJpeg(document.getElementsByClassName('reader_and_banner_container')[0], {
quality: 1,
})
.then(function(dataUrl) {
var link = document.createElement('a');
link.download = `${prefix}_page_${page}.jpg`;
link.href = dataUrl;
link.click();
nextPage(page, prefix);
});
}
function checkPageChanged(page, oldPageCounter, prefix) {
let newPageCounter = $('.page_counter').html();
if (oldPageCounter === newPageCounter) {
setTimeout(function() {
checkPageChanged(page, oldPageCounter, prefix);
}, 500);
} else {
setTimeout(function() {
downloadPage(page + 1, prefix);
}, 500);
}
}
function nextPage(page, prefix) {
let oldPageCounter = $('.page_counter').html();
$('.next_btn').trigger('click');
// Wait until page counter has changed (page loading has finished).
checkPageChanged(page + 1, oldPageCounter, prefix);
}
function download(prefix) {
downloadPage(1, prefix);
}
Finally, you could download each book page as a JPG image using:
download('test_');
It will download each page as test_page_.jpg
In order to prevent such type of 'robot', they could, for example, have used Re-CAPTCHA v3 that works in background seeking for 'robot'-like behaviour.
I have a script to test that - on click - generates an iFrame which downloads a file. How can I intercept the response with CasperJS?
I already tried the sequence:
casper.click('element');
casper.withFrame('frame', function(){
console.log(this.getCurrentUrl()); // only return about:blank, but should have URL
console.log("content: " + this.getHTML()); // Yep, empty HMTL
this.on('resource.received', function(resource){
console.log(resource.url); // never executed
});
});
I need the content of the file but can not really produce the URL without clicking the element or changing the script I'm testing.
Ideas?
I tried other events, but none got fired when downloading via the iframe. I found another solution that works - but if you have something better, I'd like to try it.
Here it comes:
// Check downloaded File
.then(function(){
// Fetch URL via internals
var url = this.evaluate(function(){
return $('__saveReportFrame').src; // JavaScript function in the page
});
var file = fs.absolute('plaintext.txt');
this.download(url, file);
var fileString = fs.read(file);
// Whatever test you want to make
test.assert(fileString.match(/STRING/g) !== null, 'Downloaded File is good');
})