Reasoning
I'm bundling small minified SVGs (icons) with my CSS via LESS's data-uri method, to reduce HTTP requests similar to the purpose of icon fonts such as Octicons or Ye Olde CSS Sprites.
However, LESS encodes them in Base64.
This is sub-optimal in the case of SVG, which can be Data URI'd in UTF-8 (example).
There's three reasons why this is sub-optimal:
1: Base64 is silly for text
The purpose of Base64 is to encode binary data using only 6 bits per byte, making it safe to embed in text files. This is great for PNGs and JPEGs, but it makes any text file 33% larger for no reason. If you're now thinking "well gzip takes care of that", then keep in mind that...
2: Encoding text in Base64 makes gzip much less effective
To understand why this is the case, consider this:
btoa('width') === 'd2lkdGg='
btoa(' width') === 'IHdpZHRo'
btoa(' width') === 'ICB3aWR0aA=='
As a practical example, let's take an actual SVG and experiment with it:
$ wc -c *
68630 tiger.svg
25699 tiger.svg.gz
91534 tiger.txt
34633 tiger.txt.gz
Even after gzipping, it's still ~35% larger.
3: It disregards some free sources of redundancy
Think about the width example above. Every SVG will have this substring, and if you embed SVGs in a CSS, you'll probably have this keyword somewhere else (or others), which gzip could benefit from (because this is how Huffman Coding works), but not if it's hidden by Base64.
The Question
How can I embed SVGs in LESS as data: URIs using UTF-8 instead of Base64?
I can imagine a thousand ways to do this involving build tools like Grunt, but it breaks my workflow because I won't be able to do things like style: include:less all.less from a Jade view (I do this in development), or even just #import 'images.less'; from a less file.
I'm an idiot. This is simple:
data-uri('image/svg+xml;charset=UTF-8', 'path/to.svg')
I had to read LESS's source to figure this one out.
All the benefits I mention above are gained here, in particular that if you have tons of small SVGs, they will benefit from the redundancy between each other. And it works in all browsers.
Related
I need to transfer a generated 1920x1080 image out of a Node pipeline that has no connectivity to either the internet or me, with the only controllable output being plaintext logs
So in order to retrieve that image, I need some way to transfer that data through a log file, which is tricky given the image is around 45KB with .jpeg compression
Converting the image to a base64 data URI works perfectly, as I can copy it from the logs and see the image in my browser, but the size of the URI is unwieldly at 40k+ characters
Is there a more compact or efficient way of doing this?
This link gives insight: Binary To text Encoding
Base85 Base85/ASCII85 is probably most promising but offers no more than 5% improvement. Is ~42KB less unwieldy than 45KB? How long will the solution live, how "shareable" must the output be and must image quality be preserved?
If image quality for your purpose is not all that important, you may get better mileage out of fiddling with encoding options (JPEG is a lossy method of encoding so you could save significant space at the cost of letting the images get more grainy although at 45KB they probably already are (?)).
You may want to get rid of mention of URIs in your question. You're interested in Base64 as an encoding method (for which 64 common text characters are used as digits... like decimal uses 10, hexadecimal 16 and Base85 85 etc). Base64 is just a ubiquitous "standard" (actually a family of several standards - see Base64 Variants).
It just seems to me that when writing code for dynamic data visualization, I end up doing the same things over and over in different languages/platforms. Now if I had a cross platform language(which I do) and something like a binary version of SVG, I could make my code target that and use/create interpreters for whatever platform I currently need to use it on.
The reason I don't want SVG is because the plaintext part makes it too slow for my purposes. I could of course just create my own intermediary format but if there is something already out there that's implemented by various things then the less work for me!
Depending on what you mean by “too slow”, the answer varies:
Filesize too large
Officially, the closest thing SVG has to a binary format is SVGZ, which is a gzipped SVG file with the .svgz extension. All conforming SVG viewers should be able to open it. Making one is simple on *nix systems:
gzip yourfile.svg && mv yourfile.svg.gz yourfile.svgz
You could also try Brotli compression, which tends to have smaller filesize at the cost of more compression time.
Including other assets is inefficient
SVG can only bundle bitmaps and other binary data through base64 encoding, which has a fair amount of overhead.
PDF can include “streams” of raw binary data, and is surprisingly efficient when programmatically generated.
Parsing the text data takes too long
This is tricky. PDF and its brother, Encapsulated PostScript, are also old, well-supported vector graphic formats. Unfortunately, they too are also text at their core, with optional compression.
You could try Computer Graphics Metafiles, which can be compiled ahead of time. But I’m unsure how well-supported they are across consumer devices.
From a comment:
Almost nothing about the performance of SVG other than the transmission cost of sending it over a network is down to it being plaintext
No, that's completely wrong. I worked at CSIRO using XML for massive 3D models. GeoScience Australia did a formal study into the parsing speed - parsing floating point numbers from text is relatively expensive for big data sets, compared to reading a 4 or 8 byte binary representation.
I've spent a lot of time optimising my internal binary formats for Touchgram and am now looking at vector art.
One of the techniques you can use is a combination of
variable-length integer coding and
normalising your points to a scale represented by integers, then storing paths as sequences of deltas
That can yield paths where often only 1 or 2 bytes are used per step, as opposed to the typical 12.
Consider a basic line
<polyline class="Connect" points="100,200 100,100" />
I could represent that with 4 bytes instead of 53.
So far, all I've been able to find in binary SVG is this post about a Go project linking to the project description and repo
Adobe Flash SWF files may work. Due to its previous ubiquity, 'players' and libraries were written for many platforms. The specifications were open and license permitting. For simple 2D graphics, earlier, more compatible versions would do fine.
The files are binary and extraordinarily small.
I am developing a game in j2me. How to handle images in .dat format.
I downloaded some games and extracted jar , found some dat format images and not able to open that images and images size also very less.. what tools I need to use?
Ref link
enter link description here
Not able to find solution?
A dat file could be anything. Depends what the developer felt like doing.
Some developers chose to strip PNG files of their header, and added the header back in the code. This was partly done in order to save a few bytes (because they mattered back then), and partly because of the challenge in doing it like that, and partly because it ensured all images used the exact same palette.
So that's one possibility, but it really could be anything.
As stated by mr_lou, there really isn't anything special about a .dat extension.
The steps to re-compile a file usually start with opening the file up in a hex editor and then looking at the first bits of information in the file. You then basically work from there to re-compile the data necessary for a 'normal' program to interpret the file. In particular, the first 8-16 bytes are often very helpful for determining what type of file it is "supposed" to be.
If you are looking at a png file (that's what I usually prefer to use for art assets) then you can reference http://en.wikipedia.org/wiki/Portable_Network_Graphics to see how a 'normal' png might look. When you're tweaking to save bytes you often strip unnecessary fields from png headers (things like the ancillary chunks) and using a common palette.
However, remember that it's not necessarily image data. It could be things like level data, sound, default stats or any particular amount of stuff.
I'm starting to venture out from using jpeg and gif files to png, I was wondering if there were any standards for using png beside IE's lack of support for it. I also want to know if there was any current articles about setting I should be using when optimizing for web? Right now I'm using photoshop to do this, should I be using firework instead?
Which optimizations you use depends on the type of image. If your image contains only few colors, you might use png-8, otherwise you may need png-24. Same goes for the use of transparency/alpha blending.
The Photoshop save for web-feature does a fine job, but when your website has a lot of visitors, you may benefit from using PNGCrush for further compressing your images. You can use the YSlow plugin for FireFox to test how much bandwidth you can save by crushing your images.
Also, you can make use of CSS-sprites if your design allows it. This can result in less (but larger) images and therefore less requests and sometimes less bandwidth. But this doen't depend on the type of images you use.
Png is supported by IE, by the way. Only the alpha-transparency is not supported by IE 6, but there are CSS/Javascript trics to work around that, although they do not work for background images.
I wouldn't quit using jpg. Jpg is very useful when it comes to pictures. Png files are convenient for small images like buttons, graphical elements, and for images with large plain areas, like screenshots.
So, to simplify my life I want to be able to append from 1 to 7 additional characters on the end of some jpg images my program is processing*. These are dummy padding (fillers, etc - probably all 0x00) just to make the file size a multiple of 8 bytes for block encryption.
Having tried this out with a few programs, it appears they are fine with the additional characters, which occur after the FF D9 that specifies the end of the image - so it appears that the file format is well defined enough that the 'corruption' I'm adding at the end shouldn't matter.
I can always post process the files later if needed, but my preference is to do the simplest thing possible - which is to let them remain (I'm decrypting other file types and they won't mind, so having a special case is annoying).
I figure with all the talk of Steganography hullaballo years ago, someone has some input here...
(encryption processing by 8 byte blocks, I don't want to save pre-encrypted file size, so append 0x00 to input data, and leave them there after decoding)
No, you can add bits to the end of a jpg file, without making it unusable. The heading of the jpg file tells how to read it, so the program reading it will stop at the end of the jpg data.
In fact, people have hidden zip files inside jpg files by appending the zip data to the end of the jpg data. Because of the way these formats are structured, the resulting file is valid in either format.
You can .. but the results may be unpredictable.
Even though there is enough information in the format to tell the client to ignore the extra data it is likely not a case the programmer tested for.
A paranoid program might look at the size, notice the discrepancy and decide it won't process your file because clearly it doesn't fully understand it. This is particularly likely when reading data from the web when random bytes in a file could be considered a security risk.
You can embed your data in the XMP tag within a JPEG (or EXIF or IPTC fields for that matter).
XMP is XML so you have a fair bit of flexibility there to do you own custom stuff.
It's probably not the simplest thing possible but putting your data here will maintain the integrity of the JPEG and require no "post processing".
You data will then show up in other imaging software such as PhotoShop, which may not be ideal.
As others have stated, you have no control how programs process image files and therefore some programs may find the images valid others may not.
However, there is a bigger issue here. Judging by your question, I'm deducing you're practicing "security through obscurity." It's widely considered a very bad practice. Use Google to find a plethora of articles about the topic.