It just seems to me that when writing code for dynamic data visualization, I end up doing the same things over and over in different languages/platforms. Now if I had a cross platform language(which I do) and something like a binary version of SVG, I could make my code target that and use/create interpreters for whatever platform I currently need to use it on.
The reason I don't want SVG is because the plaintext part makes it too slow for my purposes. I could of course just create my own intermediary format but if there is something already out there that's implemented by various things then the less work for me!
Depending on what you mean by “too slow”, the answer varies:
Filesize too large
Officially, the closest thing SVG has to a binary format is SVGZ, which is a gzipped SVG file with the .svgz extension. All conforming SVG viewers should be able to open it. Making one is simple on *nix systems:
gzip yourfile.svg && mv yourfile.svg.gz yourfile.svgz
You could also try Brotli compression, which tends to have smaller filesize at the cost of more compression time.
Including other assets is inefficient
SVG can only bundle bitmaps and other binary data through base64 encoding, which has a fair amount of overhead.
PDF can include “streams” of raw binary data, and is surprisingly efficient when programmatically generated.
Parsing the text data takes too long
This is tricky. PDF and its brother, Encapsulated PostScript, are also old, well-supported vector graphic formats. Unfortunately, they too are also text at their core, with optional compression.
You could try Computer Graphics Metafiles, which can be compiled ahead of time. But I’m unsure how well-supported they are across consumer devices.
From a comment:
Almost nothing about the performance of SVG other than the transmission cost of sending it over a network is down to it being plaintext
No, that's completely wrong. I worked at CSIRO using XML for massive 3D models. GeoScience Australia did a formal study into the parsing speed - parsing floating point numbers from text is relatively expensive for big data sets, compared to reading a 4 or 8 byte binary representation.
I've spent a lot of time optimising my internal binary formats for Touchgram and am now looking at vector art.
One of the techniques you can use is a combination of
variable-length integer coding and
normalising your points to a scale represented by integers, then storing paths as sequences of deltas
That can yield paths where often only 1 or 2 bytes are used per step, as opposed to the typical 12.
Consider a basic line
<polyline class="Connect" points="100,200 100,100" />
I could represent that with 4 bytes instead of 53.
So far, all I've been able to find in binary SVG is this post about a Go project linking to the project description and repo
Adobe Flash SWF files may work. Due to its previous ubiquity, 'players' and libraries were written for many platforms. The specifications were open and license permitting. For simple 2D graphics, earlier, more compatible versions would do fine.
The files are binary and extraordinarily small.
I have read that some image viewers were hacked by appropriately doctored images (in a format particularly suited for that? not sure about the details).
So, how could this threat be completely eliminated? For instance, suppose we make a viewer for the affected format written in managed code and have it convert images to a pure BMP (or something else so simple that its viewers cannot be hacked); would the problem go away? How about first convert to BMP and then introduce some pervasive minor random pixel alterations to better disrupt the possible hack?
Let's say this image sanitization converter would be incorporated into the firewall so that only "safe" sanitized images would, by default, end up loaded during regular browsing. Would this solve the problem? Or is my reasoning incorrect due to flawed understanding of the nature of image based exploits?
Your question, i think, boils down to whether the attack lives in the data or in the image. If it's in the data, then merely decoding the image and re-encoding with a known-good encoder it will ensure safety. If it's in the image, then you may need to alter the pixels.
The famous example of this was mishandling of JPEG comment fields with bad lengths, originally a bug in Netscape, but later independently introduced into Windows. This was very much a data rather than an image problem; the part of the data in question isn't even image data, it's metadata. If you decoded the image with a sandboxed decoder, perhaps detecting and recovering from the corrupt comment field, then re-encoded it with a friendly encoder, the result would be safe even for vulnerable decoders. Doing this naively would lead to a loss of quality; it is possible to losslessly transcode JPEGs, but it requires code to specifically do so.
My gut feeling is that image-level attacks are not possible, only data-level attacks. Image formats are well-enough specified, and fundamentally simple enough, that the content of the image really shouldn't affect the decoding process. I certainly can't prove that, though, nor really even argue for it.
Information about JPEG-LS is easily found on Google and in a lot of DICOM chapters.
However, there are links/pages/readings that mention JPEG-LL, too. However, i have taken a deeper look at the DICOM standard, not a chapter has ever mentioned anything about JPEG-LL, but in a lot of other conformance statements/forums/articles, JPEG-LL has been mentioned...
So, what is the difference between JPEG-LS and JPEG-LL?
I suppose JPEG-LL is the lossless version of the JPEG (usually called JPEG Lossless), with transfer syntax 1.2.840.10008.1.2.4.70, and it's the most common lossless compression. Anyway, it's not the best possible (it does not support signed pixel values, and its compression rates are about 2-2.5).
JPEG-LS is a completely different standard with transfer syntax 1.2.840.10008.1.2.4.80, developped by HP, which is faster and have better ratios than standard JPEG. However, it's not widely used.
I supposed JPEG-LL refers to ITU-T T.81, ISO/IEC IS 10918-1 aka JPEG lossless. While JPEG-LS refers to ITU-T T.87, ISO/IEC IS 14495-1.
Technically speaking JPEG-LL is exactly the same standard as the usual 8bits lossy jpeg as found on internet website. JPEG-LS is much much less common.
I have some ideas I would like to experiment with relating to data compression, but am finding it difficult to decipher some parts of how the standard are applied "in real life". I would like to look at some sample compressed files to observe how the the blocks are arranged and the huffman tree(s) are structured.
Are there any tools in existence which can help visualize this for a given compressed file (zip/gzip/deflate etc)? I'm picturing something like a tree view or some form of graph visualizer.
You might be interested in this (if you are still interested that is :-P)
http://jvns.ca/blog/2013/10/24/day-16-gzip-plus-poetry-equals-awesome/
I made a "entropy image" tool.
The entropy_image tool replaces each pixel
with the (estimated) number of bits
necessary to encode that pixel using range coding or Huffman compression.
I hope this isn't the only compression visualization tool in the world.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
When doing small icons, header graphics and the like for websites, is it better to use GIFs or PNGs?
Obviously if transparency effects are required, then PNGs are definitely the way to go, and for larger, more photographic images I'd use JPEGs - but for normal web "furniture", which would you recommend and why? It may just be the tools I'm using, but GIF files usually seem to be a bit smaller than a comparible PNG, but using them just seems so 1987.
As a general rule, PNG is never worse, and often better than GIF because of superior compression. There might be some edge cases where GIF is slightly better (because the PNG format may have a slightly larger overhead from metadata) but it's really not worth the worry.
It may just be the tools I'm using, but GIF files usually seem to be a bit smaller than a comparible PNG
That may indeed be due to the encoding tool you use.
/EDIT: Wow, there seem to be a lot of misconceptions about PNG file size. To quote Matt:
There's nothing wrong with GIFs for images with few colours, and as you have noticed they tend to be smaller.
This is a typical encoding mistake and not inherent in the format. You can control the colour depth and make the PNG file as small. Please refer to the relevant section in the Wikipedia article.
Also, lacking support in MSIE6 is blown out of proportion by Chrono:
If you need transparency and can get by with GIFs, then I'd recommend them because IE6 supports them. IE6 doesn't do well with transparent PNGs.
That's wrong. MSIE6 does support PNG transparency. It doesn't support the alpha channel (without a few hacks), though but this is a different matter since GIFs don't have it at all.
The only technical reason to use GIFs instead of PNGs is when use need animation and don't want to rely on other formats.
The W3C mention 3 advantages of PNG over GIF.
• Alpha channels (variable
transparency),
• Cross-platform gamma correction
(control of image brightness) and
color correction
• Two-dimensional interlacing (a
method of progressive display).
Also, have a look at these resources for guidance:
PNG v's GIF (W3C Guidance)
PNG FAQ
Wow, I'm really suprised with all the wrong answers here. PNG-8 will always be smaller than GIF when properly optimized. Just run your PNG-8 files through PngCrush or any of the other PNG optimization routines.
The key things to understand:
PNG8 and GIF are lossless <= 256 colors
PNG8 can always be smaller than GIF
GIF should never be used unless you need animation
and of course,
Use JPG for black&white or full color photographic images
Use PNG for low color, line art, screenshot type images
The main reason to use PNG over GIF from a legal standpoint is covered here:
http://www.cloanto.com/users/mcb/19950127giflzw.html
The patents have apparently expired as of 2004, but the idea that you can use PNG as open-source over GIF is appealing to many people.
(png open source reference: http://www.linuxtoday.com/news_story.php3?ltsn=1999-09-09-021-04-PS)
Be careful of color shifts when using PNG. This link gives an example, and contains many more links with further explanation:
http://www.hanselman.com/blog/GammaCorrectionAndColorCorrectionPNGIsStillTooHard.aspx
GIF images are not subject to this problem.
I don't think it makes a lot of difference (customers don't care). Personally I would choose PNGs because they are a W3C standard.
Be cautious with the PNG transparency effects: they don't work with IE6.
For images on the web, each format has its pros and cons. For photograph-type images (ie lots and lots of colours, no hard edges) use a JPEG.
For icons and the like, you have a choice between PNG and GIF. GIFs are limited to 256 colours. PNGs can be formatted like GIFs (ie 256 colours, with 1-bit transparency that will work in IE6), but for small images they're slightly larger than GIFs. 24-bit PNGs support both a large gamut, and alpha transparency (although it's troublesome in IE6).
PNGS are your only really sensible choice for things like screenshots (ie, both lots of colours and hard edges), and personally, that's what I stick with most of the time, unless I have something for which JPEG is more suitable (like a photo).
Indexed PNG (less than 256 colors) is actually always smaller than gif, so I use that most of the time.
For computer generated graphics (i.e. drawn by yourself in Photoshop, Gimp, etc.) JPG is out of the question, because it is lossy - i.e. you get random gray pixels. For static images, PNG is better in every way: more colors, scalable transparency (say, 10% transparent, .gif only supports 0% and 100%), but there is a problem that some versions of Internet Explorer don't do PNG transparency correctly, so you get flat non-transparent background that looks ugly. If you don't care about those IE users, go for PNG.
BTW, if you want animations, go for GIF.
PNG is a 100% replacement for GIF files and is supported by all web browsers you are likely to encounter.
There are very, very few situations where GIF would be preferable. The most important one is animation--the GIF89a standard supports animation, and virtually every browser supports it, but the plain old PNG format does not--you would need to use MNG for that, which has limited browser support.
Virtually all browsers support single-bit transparency in PNG files (the type of transparency offered by the GIF format). There is a lack of support in IE6 for PNG's full 8-bit transparency, but that can be rectified for most situations by a little CSS magic.
If your PNG files are coming out larger than equivalent GIF files, it is almost certainly because your source image has more than 256 colors. GIF files are indexed to a maximum palette of 256 colors, while PNG files in most graphics programs are saved by default in a 24-bit lossless format. If file size is more important than accurate colors, save the file as an 8-bit indexed PNG and it should be equivalent to GIF or better.
It is possible to "hack" a GIF file to have more than 256 colors using a combination of animation frames with do-not-replace flags and multiple palettes, but this approach has been virtually forgotten about since the advent of PNG.
A major problem with GIFs are that it is a patent-encumbered format (EDIT: This is apparently no longer true). If you don't care about that, feel free to use GIFs. PNGs have a lot more flexibility over GIFs, particularly in the area of colorspace, but that flexibility often means you'll want to "optimize" the PNGs before publishing them. A web search should uncover tools for your platform for this.
Of course, if you want animation, GIF is the only way to go, since MNG was basically a non-starter for some reason.
"It may just be the tools I'm using, but GIF files usually seem to be a bit smaller than a comparible PNG, but using them just seems so 1987."
It probably is your tools. From the PNG FAQ:
"There are two main reasons behind this phenomenon: comparing apples and oranges (that is, not comparing the same image types), and using bad tools." continued...
But you could always try saving as both (using the same colour depth) and see which comes out smaller.
Of course, if you want to standardise on one graphic format for your site, PNG is likely to be the best one to use.
Personally I use gif's quite a bit for my images, as they work everywhere, obviously your transparency limitation is one key element that would direct someone towards a specific format.
I don't see any downfalls to using gif's.
If they get smaller and you have nothing to gain from using the features PNG offers (which is alpha channel transparency and more than 256 colors) then I see no reason why you should use PNG.
gif files will tend to be a little smaller since they don't support a transparency alpha channel (and maybe for some other reasons). Personally, I don't feel the size difference is really worth worrying about nearly as much as it used to. Most people are using the web with some sort of broadband now, so I doubt they will notice a difference.
It's probably more important to use the type of images that your manipulation tools work best with.
Plus, I like the ability to put an image on any background and have a drop shadow work, which points me more towards the png format.
I usually use gif's because of the size, but there is also png-8 which is 256 colours as well.
If you need fancy semi-transparent stuff then use png-24.
I usually use the 'save for web' feature in photoshop, which lets you fiddle with filetype, number of colours etc and see the result before you save. Of course I would use the smallest possible which still looks good in my eyes.
I use jpg for all non-transparent images. You can control the compression, which I like. I found this web site that compares the two. jpg is smaller and looks better.