Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 1 year ago.
Improve this question
I am making an application using Node.js and Express. I am able to save files to the server using multer, however while sending responses I have two options,
Send the URI of the image in JSON, and let the front-end call this to display the image.
Send the image data using some form of encoding like Base64 as part of JSON.
Since I am new to web development, I am confused as to which option to use. Some people have said that the first option requires two API calls, so can be slow. While I have also heard that the second option will take up more memory resources.
What other things should I consider while choosing, and is there any other way of sending images to the client side?
Option 1
Is less complex since no conversion is needed. These 2 API calls won't slow you down. The image size is way more important!!.. The file can be stored/accessed directly on filesystem and served from there. Also a filedownload is implemented in a short period of time. Also the base64 encoding makes the file roughly ~33% (!!) bigger what has a huge impact on large files regarding performance.
Option 2
Base 64 is more secure as nobody can link to your website as described here .
You only need to use base64 for security reasons OR if you have to transfer the image data as string if you cannot transfer it as binary.
Use Case
If this is your private non-productive project just try out both and use the one you like. In the end you are learning something.. It's only important to stay consistent !
If one option fits better to you, just implement it the way you like. You can always refactor a given part of the application later when you may have more experience or when the core parts of your application are finished. Sometimes, after working a while with one of the techniques it gets more clear which approach to use.
For learning it's sometimes better to go ahead, and implement something what works
and start to refactor as problems occur. Rather than overengineering small
Related
Is anything preventing a server from dynamically changing the data of an image as it's being loaded by the browser?
To better explain what I am asking let's slow time a little bit or imagine a really slow connection.
Browser lands on the page where it's just one in so after being done with parsing, layout calculation, painting or whatever browser needs to do before being able to start downloading the image data. Eventually, it asks for the image and the server begins streaming it back. The image starts being displayed from top to bottom as its getting more data from the server. Does the server have complete freedom to provide whatever data it wants as long as it adheres to the image format in question?
Let's say every pixel is random color or whatever resolution/size is being used for data chunks being exchanged? Or is there something preventing changing it while the download is in flight perhaps some kind of hash or any other mechanism?
I guess today's image formats are not being loaded in this top-down fashion so feel free to assume old formats, browsers, protocols or travel back in time where such a thing would be possible.
Disclaimer:
This question is out of pure curiosity I am not solving any problem. I don't have code to share demonstrating what I am trying to do as I am not trying to do anything, nor I am using any specific language or framework just me not knowing enough about how the internet and its series of tubes work to answer this weird question I have.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
Say I have a program that does the following:
wait for key press
Once key pressed, do complex query (takes 10 seconds) and print the result
repeat
Now if I my key presses are 10 seconds apart, this would not be a problem. How do I handle key presses really close together. Even worse, keys pressed at the exact same time.
Is this information bound to be lost?
I know threading might be an option but what is general way of doing this?
Is it possible to just store every key pressed even though other code is running and be able to tend to it later?
Interrupts. Universally, computers provide a mechanism for peripherals to request the attention of a CPU by asserting an Interrupt Request Signal. The CPU, when it is ready to accept interrupts, responds by saving minimal state somewhere, then executing code which can query the peripheral, possibly accepting data (keypress) from it.
If you are using an OS this is all hidden by the kernel, which typically exposes some mechanisms for you to choose how you want to deal with it:
Queue up the keypresses and process them later. Thus if I want to have query 1,3,5 in that order, I can press those keys in succession and go for a smoke while your long processing occurs.
Discard the lookahead keypresses; thus demand the user interact with a lousy UI. Search for "homer simpson work from home" to see how to work around this.
If you are using an OS, you might need to look up various ioctl's to enable this behaviour, use a UI packages similar to curses, or other.
If you aren't using an OS, your job is both tougher and easier: you have to write the code to talk to the keyboard, but implementing the policy is 1/10 th work of figuring out some baroque UI library.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I have a bunch of MIDI files to render with the same Kontakt patch.
I would like to automatic render a these MIDI files with command line, without needing to load any program manually or assign manually the kontakt patch to the midi track.
I want to write my own program "MyProgram" and use it as follows :
For example
MyProgram.exe -MIDI myMidiFile.mid -kontakt myPatch.nki -out myWav.wav,
which will render the MIDI file as Wav file with the specific Kontakt Patch I assigned.
I don't have any plaform constraint, however, Windows would be the best I guess because of the VST context.
I have no idea how to achieve that, if someone has an idea?
Cheers
Not possible as specified unless NI has released something that I'm unaware of.
What is instead possible is generic VSTi plugin state persistence. (See http://vstdev.richackard.com/doc/vstsdk/faq.html). What you need is a command line VST host that will export/import the state as a file, so you can load it back again. This is useful because Kontakt is a VST plugin, so you'll be able to automate anything you want by controlling Kontakt over VST. Once set up, it would be conceptually possible to load and apply this state via a fully automated command line tool. This could definitely be implemented using the SDK, http://www.steinberg.net/en/company/developers.html
More help on VST host development: http://teragonaudio.com/article/How-to-make-your-own-VST-host.html
I can only find one similar tool that already exists; I don't know for sure if it will work for you, but the forum posts I'm reading suggest it supports VST save states.
http://teragonaudio.com/MrsWatson.html
Mrs Watson is open source, so you can extend it to better suit your needs if necessary.
I don't think you will be able to do it directly from the .nki file because this is Kontakt's own format and there isn't a function in the VST spec that will let you load it in that form.
What you may be able to do though is load Kontakt into a sequencer/DAW, load the nki file and then save the plugin state to an fxp file and then load that in your application. Note that Kontakt isn't currently VST3 so you'll need to use the VST2.x SDK.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
I've been a web developer for quite some time, and I am used to transforming all my designs into the png file format in order to build my webpages. Despite the fact that png, contrary to jpg, allows transparency in the images, is it a better solution?
The question regards page loading time and best webdesign practices, as well as file size versus quality of the images.
What do you think is the best solution to use?
It depends.
PNG is better for crisp images with a low number of colours,
JPG is better for a low-bandwidth image - however it is not as crisp and therefore not very good for GUI.
Generally, JPG is for photos and pictures, whereas PNG (or GIF) is for layout.
You may find this page interesting, as it goes over the basics of PNG vs GIF.
Google has written about it very well. From Selecting the right image format, you can find a flow-chart to make the decision:
Given the ever-rising speed of the average net connection I don't tend to think that page loading time is much of a concern any more [ducks!]; It's really far more useful to think about what you are trying to achieve with the resources you have at your end: For example is bandwidth limited? Then tending towards heavier compression is a no-brainer. Is the graphic content of the site going to expand, ensuring that the cost of server space will increase over time? Then tending towards heavier compression will delay that cost. Is it an art portfolio site? Then -- aha! -- compression artefacts in the sampler work may actually be desirable! Are you trying to flog a game? Then the screenshots should probably be ultra-crisp.
Generally, then, I would repeat what has been said, although perhaps in slightly different language: For site furnishings, which tend to be computer-generated and will be cached for re-use between pages, tend towards png; For site content, which will often be page-specific and likely large and complex enough to mask lossy compression, tend towards jpg.
With specific reference to switching to png where you decide it is appropriate, run everything through PNGCrush as a matter of course -- otherwise they won't get displayed with the colours you expect in every browser and the overall quality of your design will be diminished.
"Given the ever-rising speed of the average net connection I don't tend to think that page loading time is much of a concern any more "
According to the website SEO optimisation, Google rank your website worse if page load time is above 2 seconds, so compression is necessary, especially on new website designs heavy on graphics.
jpg is usually preferred for photographic images that have a lot of subtly different colors. png works well with computer generated graphics.
That's my rule of thumb.
I am a starter and mostly what I do is if the image is too big I make small unless it's a background image. And if it is for layout I choose PNG and JPG for pictures.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
I want to protect only certain numbers that are displayed after each request. There are about 30 such numbers. I was planning to have images generated in the place of those numerbers, but if the image is not warped as with captcha, wont scripts be able to decipher the number anyway? Also, how much of a performance hit would loading images be vs text?
The only way to make sure bad-guys don't get your data is not to share it with anyone. Any other solution is essentially entering an arms race with the screen-scrapers. At one point or another, one of you will find the arms-race too costly to continue. If the data you are sharing has any perceptible value, then probably the screen-scrapers will be very determined.
It's not possible.
You use javascript and encrypt the page, using document.write() calls after decrypting. I either scrape from the browser's display or feed the page through a JS engine to get the output.
You use Flash. I can poke into the flash file and get the values. You encrypt them in the flash and I can just run it then grab the output from the interpreter's display as a sequence of images.
You use images and I can just feed them through an OCR.
You're in an arms race. What you need to do is make your information so useful and your pages so easy to use that you become the authority source. It's also handy to change your output formats regularly to keep up, but screen scrapers can handle that unless you make fairly radical changes. Radical changes drive users away because the page is continually unfamiliar to them.
Your image solution wont' help much, and images are far less efficient. A number is usually only a few bytes long in HTML encoding. Images start at a few hundred bytes and expand to a 1k or more depending on how large you want. Images also will not render in the font the user has selected for their browser window, and are useless to people who use assisted computing devices (visually impaired people).
Apart from the images, you could display the numbers using JavaScript or flash.
You could also use CSS to position individual digits using various combinations of absolute or relative positions.
You could also use JavaScript to help you create these DIV.
The point is just to obfuscate enough that it becomes really hard.
One more solution is to use images of segments or single dots and re-construct the images of the digits using CSS, a bit like a dot-matrix display.
You could litter the source of the page with these absolutely positioned DIVs and again make it more difficult to reconstruct by creating them dynamically.
At any rate, you can't stop a determined scraper from getting to the data: it doesn't take a lot to automate a web browser and take screenshots that can be fed to an OCR.
There is nothing anyone from paying someone pennies to get the data manually anyway.
The point is: how determined are your opponents (user?).
It's a bit like the software protection business: making things hard enough that you would deter casual 'pirates' is not too hard, and it's a fairly good approach in general.
However, if there is much value in the data you present, there is nothing you can really do to protect it.
All you can do it make it hard enough so that casual 'thieves' will prefer to continue paying for your services rather than circumvent it.
Javascript would probably be the easiest to implement, but you could get really creative and have large blocks of numbers with certain ones being viewable by placing layers on top of the invalid numbers, blending the wrong numbers into the background, or making them invisible via css and semi-randomly generated class names.
I can't believe I'm promoting a common malware scripting tactic, but...
You could encode the numbers as encoded Javascript that gets rendered at runtime.
Generate an image containing those numbers and display the image. :-)
I think you guys are being too reactive with these solutions. Javascript, Capcha, even litigation and the DMCA process don't address the complex adaptive nature of web scraping and data theft. Don't you think the "ideal" solution to prevent malicious bots and website scraping would be something working in a real-time proactive mitigation strategy? Very similar to a Content Protection Network. Just say'n.
Examples:
IBM - IBM ISS Data Security Services
DISTIL - www.distil.it
Can you provide a little more detail on what it is you're doing? Certainly there's a performance hit to create an image instead of dumping out the text of a number, but how often would you be doing this per day?
Using JavaScript is the same as using text. It's trivial to reverse engineer.
Use animated numbers using flash. It may not be fool proof but it would make it harder to crack.
What about posting a lot of dummy numbers and showing the right ones with external CSS? Just as long the scraper doesn't start to parse the external CSS.
Don't output the numbers, i.e. prefix
echo $secretNumber;
with //.
For all those that recommend using Javascript, or CSS to obfuscate the numbers, well there's probably a way around it. Firefox has a plugin called abduction. Basically what it does is saves the page to a file as an image. You could probably modify this plugin to save the image, and then analyze the image to find out the secret number that is trying to be hidden.
Basically, if there's enough incentive behind scraping these numbers from the page, then it will be done. Otherwise, just post a regular number, and make it easier on your users so they won't have to worry so much about not being able to copy and paste the number, or other such problems the result from this trickery.
just do something unexpected and weird (different every time) w/ CSS box model. Force them to actually use a browser backed screenscraper.
I don't think this is possible, you can make their job harder (use images as some suggested here) but this is all you can do, you can't stop a determined person from getting the data, if you don't want them to scrape your data, don't publish it, as simple as that ...
Assuming these numbers are updated often (if they aren't then protecting them is completely moot as a human can just transcribe them by hand) you can limit automated scraping via throttling. An automated script would have to hit your site often to check for updates, if you can limit these checks you win, without resorting to obfuscation.
For pointers on throttling see this question.