Encoding the x64 VEX prefix - 64-bit

I'm using this guide to the VEX prefix: http://wiki.osdev.org/X86-64_Instruction_Encoding#REX_prefix
I'm lost on how to encode actual VEX instructions though. The ADDPD instruction (Intel x64 page 459) Opcode looks like this: VEX.NDS.128.66.0F.WIG 58 /r
Ummm what?? The VEX guide doesn't say anything about 'NDS' or 'WIG'. What am I missing?

You're missing section 3.1.1.2 "Opcode Column in the Instruction Summary Table (Instructions with VEX prefix)" (page 64 in the manual I have on my PC, probably slightly different in other versions)
For this instance, the important parts are:
VEX (obviously) means there is a VEX prefix.
NDS, the VEX.vvvv field specifies a register (it doesn't always, sometimes it is unused and must be set to all ones) and that it is a read-only first source operand.
128 fairly obvious, it's the 128bit wide version
55, it doesn't say that, it says 66 or at least it should. Other choices are F2, F3, and "nothing". Indicates what the values of the VEX.pp field should be, corresponds to mandatory prefix in the old encoding.
0F, opcode map specifier, corresponds to 0F prefix in the old encoding (there are also 0F3A and 0F38)
WIG (other options are W1 and W0), means the VEX.W field is ignored (W0 and W1 mean the W bit must be respectively 0 or 1)

Related

Extend section in Mach-O file

I am trying to extract libraries from the Dyld_shared_cache, and need to fix in external references.
For example, the pointers in the __DATA.__objc_selrefs section usually point to data outside the mach-o file, to fix that I would have to copy the corresponding c-string from the dyld and append it to the __TEXT.__objc_methname section.
Though from my understanding of the Mach-O file format, this extension of the __TEXT.__objc_methname would shift all the sections after it and would force me to fix all the offsets and pointers that reference them. Is there a way to add data to a section without breaking a lot of things?
Thanks!
Thanks to #Kamil.S for the idea about adding a new load command and section.
One way to achieve adding more data to a section is to create a duplicate segment and section and insert it before the __LINKEDIT segment.
Slide the __LINKEDIT segment so we have space to add the new section.
define the slide amount, this must be page-aligned, so I choose 0x4000.
add the slide amount to the relevant load commands, this includes but is not limited to:
__LINKEDIT segment (duh)
dyld_info_command
symtab_command
dysymtab_command
linkedit_data_commands
physically move the __LINKEDIT in the file.
duplicate the section and change the following1
size, should be the length of your new data.
addr, should be in the free space.
offset, should be in the free space.
duplicate the segment and change the following1
fileoff, should be the start of the free space.
vmaddr, should be the start of the free space.
filesize, anything as long as it is bigger than your data.
vmsize, must be identical to filesize.
nsects, change to reflect how many sections your adding.
cmdsize, change to reflect the size of the segment command and its section commands.
insert the duplicated segment and sections before the __LINKEDIT segment
update the mach_header
ncmds
sizeofcmds
physically write the extra data in the file.
you can optionally change the segname and sectname fields, though it isn't necessary. thanks Kamil.S!
UPDATE
After clarifing with OP that extension of __TEXT.__objc_methname would happen during Mach-O post processing of an existing executable I had a fresh look on the problem.
Another take would be to create a new load command LC_SEGMENT_64 with a new __TEXT_EXEC.__objc_methname segment / section entry (normally __TEXT_EXEC is used for some kernel stuff but essentially it's the same thing as __TEXT). Here's a quick POC to ilustrate the concept:
#import <Foundation/Foundation.h>
int main(int argc, const char * argv[]) {
#autoreleasepool {
printf("%lx",[NSObject new]);
}
return 0;
}
Compile like this:
gcc main.m -c -o main.o
ld main.o -rename_section __TEXT __objc_methname __TEXT_EXEC __objc_methname -lobjc -lc
Interestingly only ld up to High Sierra 10.14.6 generates __TEXT.__objc_methname, no trace of it on Catalina, it's done differently.
UPDATE2.
Playing around with it, I noticed execution rights for __TEXT segment (and __TEXT_EXEC for that matter) are not required for __objc_methname to work.
Even better specific segment & section names are not required:
I could pull off:
__DATA.__objc_methname
__DATA_CONST.__objc_methname
__ARBITRARY.__arbitrary
or in my case last __DATA section
__DATA.__objc_classrefs where the original the data got concatenated by the selector name.
It's all fine as long as a proper null terminated C-string with the selector name is there. If I intentionally break the "new\0" in hex editor or MachOView I'll get
"+[NSObject ne]: unrecognized selector sent to instance ..."
upon launching my POC executable so the value is used for sure.
So to sum __TEXT.__objc_methname section itself is likely some debugger hint made by the linker. The app runtime seems to only need selector names as char* anywhere in memory.

Security Access Condition for smart card file structure

We are working with SCOSTA smart card with ISO7816-4 APDU Commands.
I had create file structure but didn't understand how to set security Access Condition in Compact and Extended Format
For Example.
I am using TLV structure for Creating File MF,DF,EF,SE etc
Deticated File will have following FCP:
Tag 8C
Len 08
Security Attribute AM 7F
VALUE FF Delete File(Self):Never
VALUE FF Terminate DF:Never
VALUE 22 Activate File:(SE#2)
VALUE 22 Deactivate File:(SE#2)
VALUE 22 Create File EF:(SE#2)
VALUE 22 Create File DF:(SE#2)
VALUE FF Delete File(child):Never
So My Question is :
Where from VALUE 22 Set and what is the meaning of SE#2 ?
How to calculate AM Byte?
I refer ISO/IEC 7816-4:2005(E) Document Page No:24-25
Tag 8C indicates an access rule in compact format. There an Access Mode Byte (7F) is followed by as many Security Condition Bytes as bits are set in the access mode byte. The example looks reasonable, even if in my opinion, it would be more intuitive, to leave out the bits (and corresponding SCBs) for which only a NEVER security condition byte follows. (An operation not explicitly allowed, should be rejected anyway).
The coding of the Access mode byte for a DF is found in table 16 and the encoding of the security condition byte is in table 20 of ISO 7816-4.
SE#2 means, that the key IDs necessary for User Authentication, External Authentication and Secure Messaging are determined by Security Environment 2, the value 22 means, that in your case only an external authentication is necessary.

Include binary file in as86/bin86

I have written a bit of code in i8086 assembler that is supposed to put a 80x25 image into the VRAM and show it on screen.
entry start
start:
mov di,#0xb800 ; Point ES:DI at VRAM
mov es,di
mov di,#0x0000
mov si,#image ; And DS:SI at Image
mov cx,#0x03e8 ; Image is 1000 bytes
mov bl,#0x20 ; Print spaces
; How BX is used:
; |XXXX XXXX XXXXXXXX|
; ^^^^^^^^^ BL contains ascii whitespace
; ^^^^ BH higher 4 bits contain background color
; ^^^^ BH lower 4 bits contain unused foreground color
img_loop:
seg ds ; Load color
mov bh,[si]
seg es ; Write a whitespace and color to VRAM
mov [di],bx
add di,#2 ; Advance one 'pixel'
sal bh,#4 ; Shift the unused lower 4-bits so that they become background color for the 2nd pixel
seg es
mov [di],bx
add di,#2
add si,#1
sub cx,#1 ; Repeat until 1 KiB is read
jnz img_loop
endless:
jmp endless
image:
GET splash.bin
The problem is that I cannot get the as86 assembler to include the binary data from the image file. I have looked at the the man page but I could not find anything that works.
If I try to build above code it gives me no error, but the output file produced by the linker is only 44 bytes in size, so clearly it did not bother to put in the 1000 byte image.
Can anybody help me with that? What am I doing wrong?
I am not certain that this will help you, as I have never tried it for 8086 code. But you might be able to make it work.
The objcopy program can convert binary objects to various different formats. Like this example from the man objcopy page:
objcopy -I binary -O <output_format> -B <architecture> \
--rename-section .data=.rodata,alloc,load,readonly,data,contents \
<input_binary_file> <output_object_file>
So from that you'd have an object file with your <input_binary_file> in a section named .rodata. But you could name it whatever you wanted. Then use a linker to link your machine code to the image data.
The symbol names are created for you too. Also from the man page:
-B
--binary-architecture=bfdarch
Useful when transforming a architecture-less input file into an object file. In this case the output architecture can be set to
bfdarch. This option will be ignored if the input file has a known
bfdarch. You can access this binary data inside a program by
referencing the special symbols that are created by the conversion
process. These symbols are called _binary_objfile_start,
_binary_objfile_end and _binary_objfile_size. e.g. you can transform a picture file into an object file and then access it in your code using
these symbols.
If your whole code is pure code (no executable headers, no relocation...) you can just manually concatenate the image at the end of the code (and of course remove GET splash.bin). In Linux for example you can do cat code-binary image-binary > final-binary.
Thank you everybody else trying to help me. Unfortunately I did not get the objcopy to work (maybe I am just too stupid, who knows) and while I actually used cat at first, I had to include multiple binary files soon, which should still be accessible via labels in my assembler code, so that was not a solution either.
What I ended up doing was the following: You reserve the exact amount of bytes in your assembler source code directly after the label you wanna put in your binary file, i.e.:
splash_img:
.SPACE 1000
snake_pit:
.SPACE 2000
Then you assemble your source code creating a symbol table by adding the -s option, i.e. -s snake.symbol to your call to as86. The linker call does not change. Now you have a binary file that has a bunch of zeroes at the position you wanna have your binary data, and you have a symbol table that should look similar to this:
0 00000762 ---R- snake_pit
0 0000037A ---R- splash_img
All you gotta do now is get a program to override the binary file created by the linker with your binary include file starting at the addresses found in the symbol table. It is up to you how you wanna do it, there are a lot of ways, I ended up writing a small C program that does this.
Then I just call ./as86_binin snake snake.symbols splash_img splash.bin and it copies the binary include into my linked assembler program.
I am sorry for answering my own question now, but I felt like this is the best way to do it. It is quite unfortunate bin86 doesn't have a simple binary include macro on its own. If anybody else runs into this problem in the future, I hope this will help you.

J2ME - PNG created in PhotoShop not displaying in emulator

My midlet is showing some images fine, but not others.
They are all 8-bit PNGs, but the ones that aren't displaying are the ones I have created myself in PhotoShop.
So I am thinking maybe my PhotoShop (CS6) settings are wrong...
PNG-8, Selective, Diffusion, Colors: 256, Dither: 100%, Matte: None, Web
Snap: 0%, Convert to sRGB: ticked, Width: 48, Height: 48, Percent: 100%,
Quality: Bicubic.
I've experimented with a few of these settings, but to no avail.
Any ideas?
There is a similar problem here but this is opposite to mine in that PhotoShop mends things in that case, rather than breaks things...
My code is...
image = Image.createImage("/img/loading1.png");
...and here is my stack trace:
java.io.EOFException
at javax.imageio.stream.ImageInputStreamImpl.readFully(
ImageInputStreamImpl.java:353)
at java.io.DataInputStream.readUTF(DataInputStream.java:609)
at javax.imageio.stream.ImageInputStreamImpl.readUTF(ImageInputStreamImpl.java:332)
at com.sun.kvem.png.PNGImageReader.parse_iTXt_chunk(PNGImageReader.java:447)
at com.sun.kvem.png.PNGImageReader.readMetadata(PNGImageReader.java:650)
at com.sun.kvem.png.PNGImageReader.readImage(PNGImageReader.java:1312)
at com.sun.kvem.png.PNGImageReader.read(PNGImageReader.java:1582)
at com.sun.kvem.midp.GraphicsBridge.loadImage(GraphicsBridge.java:2602)
at com.sun.kvem.midp.GraphicsBridge.createImageFromData(GraphicsBridge.java:2511)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.sun.kvem.sublime.MethodExecution.process(MethodExecution.java:42)
at com.sun.kvem.sublime.SublimeExecutor.processRequest(SublimeExecutor.java:63)
at javax.microedition.lcdui.Image.createImage(Image.java:315)
The image in question does exist - both in the project and in the jar that is built.
Here is the image in question:
According to the crash log, the PNG decoder in J2ME fails inside the non-critical chunk iTXt:1
> com.sun.kvem.png.PNGImageReader.readMetadata
> com.sun.kvem.png.PNGImageReader.parse_iTXt_chunk
> javax.imageio.stream.ImageInputStreamImpl.readUTF
> java.io.DataInputStream.readUTF
According to libpng documentation, the text part of an iTXt chunk must be valid UTF8:
... The remaining chunk data is the main UTF-8 text, either zlib-compressed or not, according to the compression flag. Since its length can be determined from the chunk length, it is not null-terminated. As with the other two text chunks, newlines should be represented by single line-feed characters (decimal 10), and all other control characters (1-9, 11-31, and 127-159) are discouraged.
and so normally this would indicate that the stream read is not valid UTF8 text - it contains 'raw' bytes higher than the plain ASCII range 0..127 that do not conform to UTF8 rules.
I found that not to be the case in the sample image. There is only one set of consecutive bytes that form a UTF8 code sequence, and it is a valid one:
<?xpacket begin="EFBBBF" id=" ..
(the bolded section represents 3 data bytes in hexadecimal notation). I first suspected this was the error:
If the BOM character appears in the middle of a data stream, Unicode says it should be interpreted as a "zero-width non-breaking space" (inhibits line-breaking between word-glyphs). In Unicode 3.2, this usage is deprecated in favour of the "Word Joiner" character, U+2060.[1] This allows U+FEFF to be only used as a BOM.
(http://en.wikipedia.org/wiki/Byte_order_mark)
.. and so a fully conforming UTF8 reader should inspect its bytes and throw an UTFDataFormatException when it encounters a BOM anywhere else than as the very first value. Surprisingly, this does not seem to be the problem! First of all, there is no indication any of the readUTF sources do anything else than only verify if the UTF8 code is valid on its own, irrespective of its value. There are lots of 'invalid' Unicode code points (values that do not represent a valid Unicode character or instruction), but it appears to me they are all silently ignored. But I noticed the common readUTF functions only implement a small subset of UTF8/Unicode (see, e.g., Modified UTF-8 in Oracle's documentation).
So the problem lies elsewhere. Another clue to this is that the error thrown is not UTFDataFormatException but rather EOFException, indicating the read buffer ran out of the number of bytes it was promised to contain.
(warning: pure conjecture follows)
Looking at a source of DataInputStream, I find this snippet of code:
588 public final static String readUTF(DataInput in) throws IOException {
589 int utflen = in.readUnsignedShort();
followed by a loop to read utflen bytes (not "Unicode characters"). This is wrong for an iTXt chunk, as it does not have a 'first word' to indicate its length. The number of bytes in the plain text can be derived from the chunk length (which is, per PNG convention, the total data length excluding the length long word, the iTXt signature itself, and the final CRC32 code) minus the length of the zero-terminated keyword name, language, and "translated keyword" strings, and the two bytes which indicate compression of the full plain text.
As a work-around, remove the iTXt chunks from your PNG images. The data itself -- XMP Metadata -- is most likely not interesting at all for your purposes (but feel free to read what benefits Adobe thinks it has). And if your workflow does not use it, it's just a useless hunk of uncompressed text, taking up 814 bytes of the total of 981 bytes in your sample image -- a whopping 83%!
You can use an external utility to remove extraneous data chunks; the command line for the popular pngcrush, for example, is
pngcrush -rem alla -rem text InputFile.png OutputFile.png
(from en.wikipedia.org/wiki/Pngcrush).
Or directly from Photoshop: if you save a PNG 'the usual way' with the "Save As" menu option, the metadata goes in and there is no checkbox to get rid of it. If you use "Save for Web & Devices" instead, you get a large dialog with a lot of handy options, such as a drop down list labelled "Metadata".
Choosing "All" I got an even larger file; my version of Photoshop creates a massive 3K chunk of XMP Metadata, including a 2K totally empty 'filler' block...
Selecting "Copyright" or "None" finally got rid of all the crud (presumably because I did not fill in any copyright information), and then you get a nice 169 bytes long PNG, in which the only metadata is that software used is called "Adobe ImageReady".
1 Which is kind of ironic. Per PNG specifications,
.. A decoder encountering an unknown chunk in which the ancillary bit is 1 can safely ignore the chunk and proceed to display the image.
(source)
This "ancillary bit" is the 5th bit of the first byte of the chunk ID: 0 (uppercase) = critical, 1 (lowercase) = ancillary, i.e., if the first character of the chunk ID is a capital, a PNG reader must read and interpret its data correctly, and if it's not, it can be skipped silently.
So technically, the writers of J2ME could safely have ignored this entire chunk. But they messed it up, attempt to read it, and now the code crashes on all programs that merely try to read the image data in PNGs which happen to contain iTXt chunks.

problems with CICS Request Node CCSID

i got this problem:
i got a message flow developed in WMB7 fix 6, for integrated with CICS. My CICS CCSID is 037. The broker is running in a z/Linux with locale = en_US.UTF-8 and locale charmap = UTF-8. The MQSeries is in 1208. I got problems with special characters like (ñ,Ñ, á etc etc)
In my message flow i got this code:
DECLARE CICSRespMsg BLOB;
DECLARE CICSRespChar CHARACTER;
DECLARE MsgOut BLOB;
DECLARE MsgOutChar CHARACTER;
--EBCDIC TO ASCII
SET CICSRespMsg = InputRoot.BLOB.BLOB;
SET CICSRespChar = CAST(CICSRespMsg AS CHARACTER CCSID 037);
SET MsgOut = CAST(CICSRespChar AS BLOB CCSID 850);
SET MsgOutChar = CAST(MsgOut AS CHARACTER CCSID 850);
I tried changing from 850 to 819 and i got the same issue. Hope you can help me. Thanks so much!. ;(
So I'm not allowed to ask for clarification in my "answer", so I'll show you how to debug your problem as I can't provide you with an exact solution with the information provided.
You've shown a snippet of ESQL which is converting from ibm-037 to ibm-850 via Unicode. As ibm-850 doesn't support ñ I would expect the conversion to fail. However ibm-819, a.k.a latin-1, a.k.a iso-8859-1 does support the character and the conversion of ñ should succeed.
I don't know what you're doing after the compute node, so look at your input and output nodes, and look at the CCSID in the Properties folder. You say the MQSeries is in 1208 which I assume you mean the queue managers default CCSID is set to 1208. If this is being used on the output node then you'll have a problem as utf-8 (ibm-1208) is incompatible with latin-1 for these characters.
Place a trace node after your input node and trace to a file with ${Root} as the trace expression, place another trace node before your output node tracing the same to a different file. Look at the bytes:
ñ in 037 is 0x49
ñ in 819 is 0xf1
ñ in 1208 is 0xc3b1
if you see 0x1a it's been replaced with a substitution character.
If you want the output to be UTF-8 ensure that you use 1208 instead of 850/819 above and make sure that OutputRoot.Properties.CodedCharSetId is set to 1208.
If you want the output to be in latin-1, use 819 above and ensure that OutputRoot.Properties.CodedCharSetId is set to 819.
Hope this helps,
Andreas

Resources