I'm a JPEG novice, and I'm trying to decode a (possibly corrupted) JPEG (or rather, JFIF) file.
My image viewer program complains about an illegal 0xb9 marker.
The file does not have an SOF (Start Of Frame) marker, instead it has this APP1 segment
ff e1 00 0b 50 49 43 00 02 28 3c 01 00
followed by this segment with a 0xb9 marker:
ff b9 00 11 08 06 4c 04 d3 03 01 22 00 02 11 01 03 11 01
What am I looking at here?
EDIT
I have been asked about where the file came from. Here's the story:
About 20 years ago I bought a collection of magazines on CD-ROMs. The magazines can, however, only be read on old Windows computers, so I'm trying to find another way to read them - preferably one that works on my Linux computer. As far as I can tell, the magazine pages are stored as a lot of JFIF files that are simply concatenated together.
So to start with, I've extracted one of the JFIF files, and now I'm trying to find a way to view it.
EDIT 2
I've been asked to share a picture file. I'm not sure if I'm committing a copyright violation by doing so, so I'll remove the file again after a couple of days. Anyway, one of the problematic picture files may be downloaded from here:
https://www.dropbox.com/s/9da72gdri8c9xwp/f1000.jpg
I have no idea what the file contains except that it is probably a page from MAD Magazine.
The ff b9 segment looks very much like a SOF0 segement, and changing it to ff c0 (SOF0) makes the picture visible, but only as a collection of random pixels.
The APP1 segment mentioned above contains the string PIC, which is not part of any APP1 segement type that I'm aware of.
EDIT 3
Since this question has now been answered, I will remove the file from Dropbox in order to avoid any copyright issues. Thank you to all who contributed to this.
According to ITU T.86 Amendment 1, the APP1 segment with identifier "PIC" contains "Accusoft Pegasus custom fields." This suggests that the file may have been created using Accusoft tools, and that they might be able to decode it as well.
(Indeed, in your comment above you say that you managed to successfully open the file using the demo version of Accusoft's "Apollo" picture viewer.)
As for the FFB9 segment marker, I'm still not sure what it means. I did manage to find some code in the JPEG XT reference implementation (called "libjpeg"; not to be confused with the widely used library of the same name by the Independent JPEG Group) that appears to recognize it, and includes a comment saying that it denotes "residual scan, ac coded". Whatever that means, however, I'm not 100% sure about.
Apparently it has something to do with supporting images with more than 8 bits of dynamic range per pixel, with the "residual scan" storing additional high-precision pixel data which can be combined with the basic 8 bit JPEG image data to reconstruct the full HDR image. But if so, it seems strange to have a file with only such a residual scan marker without any normal SOI marker. Given that, according to jakub_d, the JPEG XT library cannot actually parse your file, it might be that it's using the marker in some nonstandard way.
Related
I've been trying to write a script for a buffer overflow attack on SLmail service (Within a legal setting, in a VPN. I am following a penetration testing course.).
The issue I'm having came up while trying to define which characters were bad. As it turns out, anything above 7F is getting mangled. I have followed the exact example that my textbook gave me, I tried googling for similar examples, not a single one I found ever mentioned that issue.
In theory, the only bad characters for this particular buffer should be x00, x0a and x0d.
Here everything above 7F is a mess. I get C2s and C3s appearing every other byte, while the rest of the bytes are somehow translated. (FF turns into BF, for example.). This is rendering me completely unable to have my shellcode sent through. I've tried removing some or changing the order. No matter the order I put them in, anything above 7F will come out translated with C2/C3s every other byte.
Link to both my script code and the memory dump resulting from it.
(The for is weird, I know.)
I figured it out.
I was using py3, which required strings to be encoded.
By translating the script into py2.7, I no longer needed to encode them and they went through without any mangling.
https://imgur.com/a/OOct5Z9
I'm working in linux on some homework and I cant seem to find anything online of what exactly the encoding actually means on MIPS instructions... I have used objdump -d to obtain a file that reads out, for example, lines like this:
68 90 8b 04 08 push $0x8048b90
what do each of the bytes in the first part mean? I assume the las four have to to with the actual register being pushed, but any help would be appreciated, or even a link to somewhere I could look up this kind of stuff because I cannot find it
I don't want sound-to-text software. What I need is the following:
I'll record multiple (say 50+) audio streams (recordings of radio stations)
from that recordings, I'll mark interesting audio clips - their length ranges from 2 to 60 seconds - there will be few thousands of such audio clips
library should be able to find other instances of same audio clips from recorded sound streams
confidence factor should be reported to used and additional input provided so the recognition could perform better next time
Do you know of such software library? LGPL would be most valuable to me, but I can go for commercial license as well.
Audio clips will contain both music, text, effects, or any combination thereof. So, TEXT recognition is out of the question.
Architecture: c++, C# for glue, CUDA if possible.
I have not found any libraries (yet), but two interesting papers, which may give you terminology and background to refine your searches:
Audio Fingerprinting for Broadcast Streams
Audio Segment Retrieval using HMM
EDIT: Searching for "Audio fingerprinting" came to a page of implementations, both open source and commercial.
http://wiki.musicbrainz.org/AudioFingerprint
Picard seems to be well established, and could be useful if your clips contain music.
Here is an introduction to Audio fingerprinting
What you are describing is a matched filter and all you need is a cross-correlation function which should be part of any reasonable DSP library. Depending upon your choice of processor architecture and language you may even be able to find a vectorized library that can perform this operation more efficiently.
If you don't really care about performance you could use Python...
$ python
Python 2.6.4 (r264:75706, Dec 7 2009, 18:45:15)
[GCC 4.4.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import scipy
>>> interesting_clip = [ 5, 7, 2, 1]
>>> full_stream = [ 1, 5, 7, 2, 1, 4, 3, 2, 4, 7, 1, 2, 2, 5, 1]
>>> correlation = scipy.correlate (full_stream, interesting_clip)
>>> print correlation
[56 79 55 28 41 49 44 53 73 48 28 35]
>>> for offset, value in enumerate(correlation) :
... if (value > 60) :
... print "match at position", offset, "with value of", value
...
match at position 1 with value of 79
match at position 8 with value of 73
My threshold above is arbitrarily. You should experimentally determine what is appropriate for you.
Keep in mind that the longer your "interesting clip", the longer it will take to compute the correlation. While longer clips will help actual matches stand out better from non-matches, you probably won't need more than a few seconds.
AudioDB is an open source c++ project that searches for similar sections of audio, and handles noisy streams, and can give you a measure of similarity. It can be run as client/server, but I believe you can do a standalone program.
The other answers about dsp correlation are kind of correct, but in general these dsp algorithms want to compare two streams of the same length, which have the similar parts overlapping.
What you need requires it to work on arbitrary segments of the stream; this is what AudioDB was built for. (One application is to find hidden references/sampling or blatant copyright misuse.) I've used it for finding sounds that were played backwards, and it also finds the case where some noise or speech changes are introduced.
Note that it is still under development even though the dates on the home page seem to off. I would subscribe to the mailing list and ask what the current state is and how you might go about incorporating it.
You might want to look at this paper by Li-Chun Wang regarding www.shazam.com.
It is not an API but it does give details of how their algorithm was developed.
Take a look at the Microsoft Speech API (SAPI):
http://msdn.microsoft.com/en-us/library/ee125077%28VS.85%29.aspx
All the other requirements you listed are basically implementation details that you'll have to implement on your own. For example, as the software interprets the audio streams, it can store them in SQL server with full text indexing ... from that you do the searches to find similar/same audio clips.
There are of course other ways to implement that, and this is but one idea :-)
I would go somewhere in line with Tim Kryger's answer and use simple statistical correlation functions, as you want to stay content-agnostic.
As for the features I would definately try MFCC as it's used both in speech processing and music recognition (genres, songs). You can find MFCC and a wealth of other audio features available in the excellent open source Vamp plugins (or its more high-level bundle, a program called Sonic Annotator) or alternatively in the Marsyas framework.
I need to build a simple app that takes (click only) input from 4 USB mice connected in addition to the built-in touchpad on a notebook. My preferred operating system for this setup would be Linux.
Any idea how I might be able to discern in an application which mouse a click came from? I'm open to C programming or whatever it takes. It's a simple, one-off project, so nothing too elaborate though.
For what it's worth, I think I found an answer to my question.
bobince's mention of xorg led me to look in /etc/X11/xorg.conf. That turns out to be full of comments like
# commented out by update-manager, HAL is now used
I had heard of HAL before, and not just in 2001. I tried man -k hal and found lshal, which lists 133 (!) HAL devices in my PC. In the entry for one of my mice, I found
linux.sysfs_path = '/sys/devices/pci0000:00/0000:00:0b.0/usb2/2-7/2-7:1.0/input/input6/event6'
which turns out to be a directory in the file system. Exploring from there, I discovered a reference back to /dev/input/mouse3. In fact, all my mice were sitting there in /dev/input!
Wearing my superuser cape, I was able to read /dev/input/mouse3:
root#carl-ubuntu:/dev/input# od -t x1 -w6 mouse3
0000000 09 00 00 08 00 00
*
so it turns out a left mouse click is 09 00 00 08 00 00, consistently and repeatably.
Conclusion: I can read mouse clicks from /dev/input/mouseX. Having done chmod a+r on those files, I can even read them from a normal user account. I need to figure out a way to stop 4 mice running wild in the hands of probably drunk people from actually interacting directly with the GUI, but that's a problem for another day.
MPX is where it's at for multiple-mouse and multitouch under Linux, but you'll need to be using xorg xserver 1.7 to get the ‘proper’ version of it; this is generally taken as part of X11R7.5, which has only just come out as ‘stable’ and has not been integrated by the distros yet. (Even xorg-edgers doesn't have it, though that's where you'd keep an eye on if you're an Ubuntu-er.)
GTK+ seems to have had some work put into allowing you to detect which mouse clicked (GdkEvent.gdk_event_get_device), but I don't know what the timetable is for getting this into a full stable release. Nor do I know how far along Qt4 is with it. So in summary the situation is: it works if you're willing to put the time into grabbing, compiling and fixing stuff, but it's not smooth with mainstream Linux, yet.
I'm not too sure where to start for this, but it sounds a lot to me like it'd be similar to getting multi-touch to work. Maybe start looking for multi-touch drivers for linux?
Also, luvieere's first link might be helpful.
I would like to be able to detect the presence of a color profile in JPEG. To be clear: I'm not interested in actually carrying out the color corrections or doing anything else with them.
I did some investigation, but I could not find any definite source of information. Here is what I know so far:
There is a color profile information in the XMP block. The XMP block is in a 0xFFE1 JPEG section (the same as EXIF). The marked of this section is followed by the http://ns.adobe.com/xap/1.0/ string. However, when I stripped this section, Photoshop was still able to detect the color profile.
There is a color space tag in EXIF itself as well (tag = 0xA001). It has only one value defined in the EXIF specification: 1 = sRGB. I noticed that in my image it takes the value 0xFFFF which means uncalibrated. So it’s not this tag.
I did an experiment with one image saving it in two different color profiles, and besides the changes in XMP and EXIF, I noticed that there is probably a whole new JPEG section. It starts by 0xFFE2 and its marker is followed by the string ICC_PROFILE. This seems to be the most likely case.
Does anybody have experience with managing color profiles in JPEG? I would say that the information in EXIF and XMP are just metadata, and that the important thing is the presence of the ICC_PROFILE section.
ICC profiles are stored in a JPEG file's Application Segments. These blocks are for letting applications store whatever data they need, so they are not mandated by the JPEG spec. In other words, the JPEG file format does not define a way to embed color profiles, you are just seeing the de facto ways of doing it.
Since the application segments can be used by anyone for anything, you need to identify and validate the contents, before you can say for certain that it contains an ICC profile.
So there are three ways common ways ICC profiles could be supplied:
By embedding an ICC profile in Application Segment 02, prepended with ICC_PROFILE\0x00\0x01\0x01 for identifying it. I'm currently unsure if the two \0x01 mean anything, or if they are always that. After those 14 characters, the actual ICC profile is stored.
By embedding an ICC profile in an embedded EXIF metadata file in Application Segment 01.
By embedding an ICC profile in an embedded XAM metadata file in Application Segment 01. This is prepended with http://ns.adobe.com/xap/1.0/\0x00.
I have not been able to find any concrete information either. It would be nice if ICC said, "this is how to do it", but I have not yet been able to find any of the sort. Thus as far as I know, any of these could potentially be used as an ICC profile and it would depend on the application on whether it would load the profile or not.
Application Segment 02 seems to be the common way of doing it, though what you want to check might depend on your use case. As the other comment showed, some HELIOS EtherShare software has/had its own way of doing it. (I have not yet confirmed the presence of an ICC profile in the metadata, however I haven't really looked that hard and it wouldn't surprise me if you could find files with multiple conflicting ICC profiles in the wild.)