all.
I'm using a 3rd party lib to archive/decompress some files.
Input/output format is .zip.
But I've discovered that the lib could read well archives created with WinRAR, but cannot read archives created with 7-zip.
Could anyone tell me what the problem is and how it could be fixed.
hex view of starting of archive, created with winRar:
50 4b 03 04 14 00 00 00 08 00 EC 88 25 45 6e 1a ....
hex view of starting of archive, created with 7-zip:
50 4b 03 04 14 00 00 00 08 00 ED 88 25 45 6e 1a ....
the difference is in byte EC and ED.
This is the header structure of a ZIP file:
+---------------------------------------------------------------------+
| BYTE |
|0x0 0x1 0x2 0x3|0x4 0x5|0x6 0x7| 0x8 0x9 |0xA 0xB|0xC 0xD| |
+---------------+-------+-------+-----------+--------+--------+-------+
| signature |version| flags |compression|mod time|mod date| CRC32 |
+---------------+-------+-------+-----------+--------+--------+-------+
| 50 4B 03 04| 14 00| 00 00| 08 00 |EC/ED 88| 25 45| |
+---------------------------------------------------------------------+
signature is always 0x50 0x4B 0x03 0x04 (as in your case). 0x50 = 'P', 0x4B = 'K' (Phillip Katz)
version needed to extract (0x14 0x00 = 20 -> ZIP 2.0)
general purpose bit flags (no flags, your file is not crypted / default language encoding)
compression method is deflate (08)
So the only things that change is the modification time (just some minutes).
Even the first two bytes of the CRC32 are the same. If the only difference is the EC / ED value, the problem shouldn't be the archive content.
Further references: https://users.cs.jmu.edu/buchhofp/forensics/formats/pkzip.html
7-Zip archives can be created using several different compression formats which are not compatible with standard ZIP compression/decompression libraries based for example on Info-ZIP source code.
Applications like WinRAR and Total Commander supporting creation/extraction of ZIP files natively use therefore additionally a DLL for extracting files from non standard 7-Zip archives.
See question Can I use the EXE or DLL files from 7-Zip in a Commercial Application? on FAQ page of 7-Zip.
Related
I have a zip file over 15gb with an error. I need to modify a bash code to handle the error (not to correct it just to "catch it"), the thing is that the size of the file makes me go very slow when I want to test something, I need to recreate the same error but with a file of like 1gb so I can do testing faster, I have tried everything that came to my mind but without success.
This is the error:
$ zip -T AUDIOS.zip
Could not find:AUDIOS.z01
Hit c (change path to where this split file is)
q (abort archive - quit)
or ENTER(Try reading this split again)
The error is obvious but I think is the result of a corrupted file because the zip file is not really a multipart zip so I assume there are some errors in the headers of the file or something like that that makes the zip command think that it is a multipart file.
Note:If I use 7z to unzip the file everything goes well, it says that there are error headers but the file uncompress the right way anyways.
I wrote the last two paragraphs just to give some context, what I really need is to recreate the error in a smaller zip file so when I execute zip -T zipfile it returns the same as the big file.
The link posted by #tink does match the error you are seeing. The unknown is whether the corruption in your zip file matches the one in the link. There may be multiple reason that can trigger that error message. Not sure if that is important to you.
If you want to verify that the details in the link match with your zip file get a copy of zipdetails from here and run this
perl zipdetails -v AUDIOS.zip
It will provide a detailed output of the zip file. For this error you need to post the data at the end of the zip file that looks like this. I've highlighted the part in the link that is the cause of the problem.
00BF 0004 50 4B 06 06 ZIP64 END CENTRAL DIR 06064B50
RECORD
00C3 0008 2C 00 00 00 Size of record 000000000000002C
00 00 00 00
00CB 0001 1E Created Zip Spec 1E '3.0'
00CC 0001 03 Created OS 03 'Unix'
00CD 0001 2D Extract Zip Spec 2D '4.5'
00CE 0001 00 Extract OS 00 'MS-DOS'
00CF 0004 00 00 00 00 Number of this disk 00000000
00D3 0004 00 00 00 00 Central Dir Disk no 00000000
00D7 0008 01 00 00 00 Entries in this disk 0000000000000001
00 00 00 00
00DF 0008 01 00 00 00 Total Entries 0000000000000001
00 00 00 00
00E7 0008 5A 00 00 00 Size of Central Dir 000000000000005A
00 00 00 00
00EF 0008 65 00 00 00 Offset to Central dir 0000000000000065
00 00 00 00
00F7 0004 50 4B 06 07 ZIP64 END CENTRAL DIR 07064B50
LOCATOR
00FB 0004 00 00 00 00 Central Dir Disk no 00000000
00FF 0008 BF 00 00 00 Offset to Central dir 00000000000000BF
00 00 00 00
0107 0004 00 00 00 00 Total no of Disks 00000000 <==== This value should be 1
010B 0004 50 4B 05 06 END CENTRAL HEADER 06054B50
010F 0002 00 00 Number of this disk 0000
0111 0002 00 00 Central Dir Disk no 0000
0113 0002 01 00 Entries in this disk 0001
0115 0002 01 00 Total Entries 0001
0117 0004 5A 00 00 00 Size of Central Dir 0000005A
011B 0004 FF FF FF FF Offset to Central Dir FFFFFFFF
011F 0002 00 00 Comment Length 0000
There is a test file zero.zip that I use in the test harness for zipdetails to check this particular error. If this is indeed the same as you corruption you can just use zero.zip for your testing.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about a specific programming problem, a software algorithm, or software tools primarily used by programmers. If you believe the question would be on-topic on another Stack Exchange site, you can leave a comment to explain where the question may be able to be answered.
Closed last year.
Improve this question
I have generated an AES-256 ciphertext in hex (cipher.hex) that I am trying to decrypt:
53 9B 33 3B 39 70 6D 14 90 28 CF E1 D9 D4 A4 07
with a corresponding 256-bit key in hex (key.hex):
80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01
I started by attempting to decode both into base64 using the following commands:
base64 cipher.hex > input;
base64 key.hex > key;
and lastly, passed them into openssl as seen below:
openssl aes-256-cbc -d -a -pass file:key -in input -out out
at which point I get back "bad magic number" from openssl with no guidance on what went wrong.
Is there something wrong with the procedure I'm following? I have also tried converting the hex values into binary... really not sure what input to give openssl from hex, let alone if base64 decoding a newline-wrapped hexidecimal is valid.
You're trying to use CBC and password-based encryption on something that is encrypted with ECB and a raw key. You're also base64 encoding the hex characters, not the actual bytes. You're encoding <ASCII for 5><ASCII for 3><ASCII for space>... You want to encode 0x53, 0x9B, 0x33, etc. Base64 isn't needed here in any case.
First, convert your hex input to raw data (not base64). I like xxd for this:
echo "53 9B 33 3B 39 70 6D 14 90 28 CF E1 D9 D4 A4 07" | xxd -r -p > input
Next, you need your key in the hex format openssl likes, no spaces or newlines:
KEY=$(echo "80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01" | tr -d '[ \n]'
echo $KEY
# prints 8000000000000000000000000000000000000000000000000000000000000001
And now, you can decrypt the data (aes-256-ecb with a hex key):
openssl aes-256-ecb -d -in input -K $KEY | xxd
This prints:
00000000: 8070 6050 4030 2010 0807 0605 0403 02 .p`P#0 ........
Which is too orderly to be wrong. (0x80, 0x70, 0x60, 0x50, ...)
If this were CBC, there would be an IV, and in most cases a padding block. That said, for a single block with no padding and a NULL IV (which you should never use), CBC is equivalent to ECB, so it is possible to decrypt this as CBC and get the same result:
openssl aes-256-cbc -iv 00000000000000000000000000000000 -d -in input -K $KEY | xxd
The "bad magic number" error is because you used the -pass option, which employs OpenSSL's password key derivation algorithm. Files that have been encrypted that way start with the ASCII sequence Salted__. That's missing, so the input is rejected. (It wouldn't work anyway, since this was encrypted with a raw key.)
This is a great solution as far as it goes:
How to use bluetoothctl like hcitool lescan to report repeated proximity beacons
I want to take it one step further though: instead of printing the beacon detections to the terminal, I want to pipe each detected beacon's MAC address to a command that publishes it to a Mosquitto broker. This would be useful for adding the beacons to HomeAssistant, for example, for presence detection of a key fob attached to a person's house keys. I managed to get this idea to work by modifying the last two lines of the script above but it only publishes the first detection ... then no more. I guess somehow the scan is being turned off or aborted by trying to run the mosquitto_pub command. (something to do with interrupting the thread or subshells????)
Here are the last lines of the script with the MAC addresses of my TWO TEST BEACONS obfuscated.
You can see the full script that I modified at the link above.
) | sed --unbuffered --quiet --expression 's/^.*Device //p' \
| grep --line-buffered -E 'FF:FF::::|FF:GG::::' \
| stdbuf -oL cut -c 1-17 \
| { read topic; mosquitto_pub -d -u username -P password -m ON -t monitor/$topic; }
Note:
| stdbuf -oL cut -c 1-17 \ trims the output down to just the MAC address - e.g. FF:GG::::
The last line publishes the ON command to an unique state topic ending in the beacon's MAC address e.g. 'monitor/FF:GG::::'.
If I remove that last line the detections are added as they occur to the terminal console in an ever growing list as time goes by. But once I add the last line it only works once, then goes dead.
How can the script be made to respond to each detection, not just the first one?
And why exactly does it only respond to the first detection then hang?
The command line tools of BlueZ are not designed for this purpose, so it makes them very difficult to pipe scan results reliably. In my experience, processes die or hang, and you end up with broken pipes.
The C APIs of BlueZ are much more stable. I wrote a simple command line program in C that you could use instead of bluetoothctl for this purpose:
https://gist.github.com/davidgyoung/0a18028b4338ff6cb201fba274502662
That program must be compiled with cc scanner.c -lbluetooth -o scanner, after which you can start the scanner with just scanner. The output of the program will be something like this:
B8:27:EB:1F:93:4D -68 02 01 06 11 06 82 75 25 D9 37 9D D7 8F 5F 4A F4 20 00 00 75 30
71:5C:23:9D:BC:7F -68 02 01 1A 02 0A 0C 0B FF 4C 00 10 06 03 1A 3B D4 B2 EB
B8:27:EB:1F:93:4D -68 02 01 06 11 06 82 75 25 D9 37 9D D7 8F 5F 4A F4 20 00 00 75 30
4A:53:7F:64:71:EC -91 03 03 9F FE 17 16 9F FE 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
69:0D:FF:7B:75:65 -73 02 01 1A 02 0A 0C 0A FF 4C 00 10 05 03 1C 27 BB 63
61:39:71:E9:1D:C9 -93 02 01 1A 02 0A 18 0A FF 4C 00 10 05 01 18 3B 24 12
B8:27:EB:1F:93:4D -68 02 01 06 11 06 82 75 25 D9 37 9D D7 8F 5F 4A F4 20 00 00 75 30
Each line above shows the MAC address of the detected bluetooth device, followed by the signal strength RSSI, then the hex bytes of the advertisement.
I use this program to pipe into other shell scripts, and have found it to be reliable over many days. Feel free to use it if helpful.
There is a well known blog post going around on how to set a usb bluetooth 4 dongle to be an iBeacon. It boils down to this magical command:
sudo hcitool -i hci0 cmd 0x08 0x0008 1e 02 01 1a 1a ff 4c 00 02 15 e2 c5 6d b5 df fb 48 d2 b0 60 d0 f5 a7 10 96 e0 00 00 00 00 c5 00 00 00 00 00 00 00 00 00 00 00 00 00
The issue with this example is that it is so opaque it's hard to use it in any more general format. I've been able to break it apart a bit:
sudo hcitool -i hci0 cmd
sends an hci command to the hci0 device
0x08 0x0008
is just magic to set the ad package, other stackoverflow commands have said "just use it, don't ask
1e
is the length of the ENTIRE following data packet in bytes
02 01 1a 1a
Are flags to set up the ad packet (details on request)
ff 4c 00 ...
is the 'company specific data' that encodes the iBeacon info
What I've tried to do is replace the "FF ..." bytes with the opcodes for setting the NAME parameter "04 09 41 42 43" (which should set it to ABC) but that doesn't work.
I'm surprised the hcitool doesn't give us some examples on how to set the ad packet as this would be very useful in setting all sorts of other params (like TEMP or POWER). Has anyone else had any experience in using hcitool to set things like NAME?
Late reply, but somebody might find this useful. I found it as I was looking around for solutions myself when using hcitool.
If you use hcitool cmd --help it will tell you something like this cmd <ogf> <ocf> .... It helps to look at the Bluetooth Core Specification to find out what 0x08 and 0x0008 would be for OGF and OCF. Specifically Vol. 2, Part E, 7.8
For the LE Controller Commands, the OGF code is defined as 0x08
and for the OCF of 0x0008
Advertising_Data_Length, Advertising_Data
So basically, with 0x08 0x0008 you say you are setting (in the LE Controller) the length of the data that is sent. As for the name, since the length of the BLE advertisement packet is 31 bytes (1E), you need to send the whole 31 bytes. So if you only have ABC as the name, setting 04 09 41 42 43 is correct, but that's only five bytes. For 31 you need to add 00 26 times. Just be careful you don't add too much or too little.
Also, I wasn't under the impression that BLE ad. packets are of fixed 31 byte size, but they are at least for hcitool. It doesn't work when you specifically set the outgoing size to something smaller than 1E.
No. None of this answers is correct and clean.
1) BLE have a separate command set. "LE Set Advertising Data" command must be used (see 7.8.7 vol 2 part E).
2) LE Set Advertising Data is much complex than what explained above. There are at least 2 1-octet length fields, packet must be 32 bytes length, zero padded, but the first length byte must not be 0x1e (31) but the length of the significant used part, containing one or more headers. Each header still contains a length, one AS Type byte (0x09 for set local name) and the name.
SET LE LOCAL NAME:
hciconfig hci0 down
hciconfig hci0 up
hcitool -i hci0 cmd 0x08 0x0008 0c 0b 09 6c 69 6e 6b 6d 6f 74 69 6f 6e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
hciconfig hci0 leadv 0
0x0c : length of the following group of headers
0x0b : length of this header
0x09 : AD Type for complete name
rest 0x0a length is the name
Check out this answer to a similar question. It basically describes how you can download the giant Bluetooth Core Spec document, and read through all the commands that it offers you. You can use the hcitool command to execute any of these commands if you can just figure out the right format (and figure out what the commands actually do!)
Major caveat: I have not tried setting the name myself, but glancing at the spec, it looks like this is described on page 482 of the spec in section "7.3.11 Write Local Name Command". According to this the command consists of:
OCF: 0x0013
Name (up to 248 bytes)
So I would try a command like this:
hcitool -i hci0 cmd 0x08 0x0013 41 42 43
One other tip: When you issue commands like this, try running hcidump & so the command executes in the background. Then, you can enter experimental hcitool commands (or even hciconfig commands) and see annotated details about what (human readable) commands executed and what errors occurred, if any.
Using the above tip, you can also try executing hciconfig name abc to set the local name using that command line tool, while you are executing a hcidump & in the background. This should show you the proper hcitool command values to use.
It would appear you need to use two commands rather than one. The iBeacon data is contained in the "LE Set Advertising Data" data and has been mentioned elsewhere in this post. To see a BLE friendly name you can use an additional command "LE Set Scan Response Data", this requires the receiver to scan your device (rather than passively read the advertising packet). So you can combine Angelo's example with this one to set the device as an iBeacon and set the "friendly name" which is the Scan Response data
sudo hcitool -i hci0 cmd 0x08 0x0008 1E 02 01 1A 1A FF 4C 00 02 15 E2 0A 39 F4 73 F5 4B C4 A1 2F 17 D1 AD 07 A9 61 00 00 00 00 C8 00
sudo hcitool -i hci0 cmd 0x08 0x0009 0c 0b 09 6c 69 6e 6b 6d 6f 74 69 6f 6e 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
This worked for me using an Ubuntu box with a BLE dongle and then scanning for the beacon using this android BLE scanning app
When building static libraries with VS2005 I keep getting linker warnings that VC80.pdb cant be found with my library.lib. Apparently, as a result, the edit and continue feature of the IDE fails to work any project that incorporates library.lib
What magic is needed to tell VS2005 to produce a static lib with edit and continue debug info that does NOT reference or require vs80.pdb when linked into a project?
--Upon Further Understanding--
So, In order to get edit-and-continue to function with a pre-compiled static lib, we need to place the vs80.pdb and vs80.pdb file into SVN along with the .lib, AND rename the pdb/idb to prevent conflicts when doing this with multiple pre-compiled libs.
vc80.pdb is the file that contains the debug information for your lib. In the ide Property pages:configuration properties:c\c++:output files allows you to rename this to something more appropriate, such as the name of your lib. When the linker links your lib into the target exe it looks for this pdb (there is a pointer to it in the lib) and extracts the info from that pdb and drops it in the exe's pdb.
/Fd[name] is the option for renaming the pdb
/ZI is the option for compiling with a pdb that includes Edit and Continue information.
All linked libs and the final taget exe or dll need /ZI, to enable edit and continue.
I made a tiny testlib.lib and used "dumpbin /all" to get the following showing the pointer to the debug info (this is a tiny excerpt):
SECTION HEADER #7
.debug$T name
0 physical address
0 virtual address
48 size of raw data
838 file pointer to raw data (00000838 to 0000087F)
0 file pointer to relocation table
0 file pointer to line numbers
0 number of relocations
0 number of line numbers
42100040 flags
Initialized Data
Discardable
1 byte align
Read Only
RAW DATA #7
00000000: 04 00 00 00 42 00 15 15 D5 EA 1E C9 7C 10 3A 40 ....B...Õê.É|.:#
00000010: 93 63 CE 95 77 15 49 4A 03 00 00 00 64 3A 5C 64 .cÎ.w.IJ....d:\d
00000020: 65 76 5C 74 65 73 74 5C 74 65 73 74 6C 69 62 5C ev\test\testlib\
00000030: 74 65 73 74 6C 69 62 5C 64 65 62 75 67 5C 76 63 testlib\debug\vc
00000040: 38 30 2E 70 64 62 00 F1 80.pdb.ñ
If you can live without 'edit and continue', try using /Z7.
I use it for all the .lib files that are stored in source control. No .pdb file is created - the debug info is stored inside the .lib file.