How to enable performance counters in Rocket Core - riscv

I am trying to enable performance counters in the Rocket core.
I don't see the implementation of mhpmevent3, mhpmevent4, mhpmcounter3, mhpmcounter4 in the RTL of the defaultConfig.
I see that following parameter is set to true by default but none of the mentioned counters does not get generated in the defaultConfig.
haveBasicCounters: Boolean = true
https://github.com/chipsalliance/rocket-chip/blob/b503f8ac28a497b2463ffbac84bfe66533ace0bb/src/main/scala/rocket/RocketCore.scala#L37
Can someone please explain how to enable these basic performance counters in Rocket?
I tried increasing nPerfCounters: Int = 0 to 8 and using following code to configure and read hpmcounter3. But counter never changed, it had a random value in each run.
addi x14, zero, 0x04
slli x14, x14, 8
addi x14, x14, 2
csrrw zero, mhpmevent3, x14
...
csrr x28, mhpmcounter3 #Read counters in two different places
...
csrr x28, mhpmcounter3
I am using the configuration mentioned in https://static.dev.sifive.com/U54-MC-RVCoreIP.pdf#page=16 3.11 as suggested in rocket-chip github issues.

Related

How to calculate time for an asm delay loop on x86 linux?

I was going through this link delay in assembly to add delay in assembly. I want to perform some experiment by adding different delay value.
The useful code to generate delay
; start delay
mov bp, 43690
mov si, 43690
delay2:
dec bp
nop
jnz delay2
dec si
cmp si,0
jnz delay2
; end delay
What I understood from the code, the delay is proportion to the time it spends to execute nop instructions (43690x43690 ). So in different system and different version of OS, delay will be different. Am I right?
Can anyone explain to me how to calculate the amount of delay in nsec, the following assembly code is generating so that I can conclude my experiment with respect to delay I added in my experimental setup?
This is the code I am using to generate delay without understanding the logic behind use of 43690 value ( I used only one loop against two loops in original source code). To generate different delay (without knowing its value), I just varied number 43690 to 403690 or other value.
Code in 32bit OS
movl $43690, %esi ; ---> if I vary this 4003690 then delay value ??
.delay2:
dec %esi
nop
jnz .delay2
How much delay is generated by this assembly code ?
If I want to generate 100nsec or 1000nsec or any other delay in microsec, what will be initial value I need to load in register?
I am using ubuntu 16.04 (both 32bit as well as 64bit ), in Intel(R) Core(TM) i5-7200U CPU # 2.50GHz and Core-i3 CPU 3470 # 3.20GHz processor.
Thank you in advance.
There is no very good way to get accurate and predictable timing from fixed counts for delay loops on a modern x86 PC, especially in user-space under a non-realtime OS like Linux. (But you could spin on rdtsc for very short delays; see below). You can use a simple delay-loop if you need to sleep at least long enough and it's ok to sleep longer when things go wrong.
Normally you want to sleep and let the OS wake your process, but this doesn't work for delays of only a couple microseconds on Linux. nanosleep can express it, but the kernel doesn't schedule with such precise timing. See How to make a thread sleep/block for nanoseconds (or at least milliseconds)?. On a kernel with Meltdown + Spectre mitigation enabled, a round-trip to the kernel takes longer than a microsecond anyway.
(Or are you doing this inside the kernel? I think Linux already has a calibrated delay loop. In any case, it has a standard API for delays: https://www.kernel.org/doc/Documentation/timers/timers-howto.txt, including ndelay(unsigned long nsecs) which uses the "jiffies" clock-speed estimate to sleep for at least long enough. IDK how accurate that is, or if it sometimes sleeps much longer than needed when clock speed is low, or if it updates the calibration as the CPU freq changes.)
Your (inner) loop is totally predictable at 1 iteration per core clock cycle on recent Intel/AMD CPUs, whether or not there's a nop in it. It's under 4 fused-domain uops, so you bottleneck on the 1-per-clock loop throughput of your CPUs. (See Agner Fog's x86 microarch guide, or time it yourself for large iteration counts with perf stat ./a.out.) Unless there's competition from another hyperthread on the same physical core...
Or unless the inner loop spans a 32-byte boundary, on Skylake or Kaby Lake (loop buffer disabled by microcode updates to work around a design bug). Then your dec / jnz loop could run at 1 per 2 cycles because it would require fetching from 2 different uop-cache lines.
I'd recommend leaving out the nop to have a better chance of it being 1 per clock on more CPUs, too. You need to calibrate it anyway, so a larger code footprint isn't helpful (so leave out extra alignment, too). (Make sure calibration happens while CPU is at max turbo, if you need to ensure a minimum delay time.)
If your inner loop wasn't quite so small (e.g. more nops), see Is performance reduced when executing loops whose uop count is not a multiple of processor width? for details on front-end throughput when the uop count isn't a multiple of 8. SKL / KBL with disabled loop buffers run from the uop cache even for tiny loops.
But x86 doesn't have a fixed clock frequency (and transitions between frequency states stop the clock for ~20k clock cycles (8.5us), on a Skylake CPU).
If running this with interrupts enabled, then interrupts are another unpredictable source of delays. (Even in kernel mode, Linux usually has interrupts enabled. An interrupts-disabled delay loop for tens of thousands of clock cycles seems like a bad idea.)
If running in user-space, then I hope you're using a kernel compiled with realtime support. But even then, Linux isn't fully designed for hard-realtime operation, so I'm not sure how good you can get.
System management mode interrupts are another source of delay that even the kernel doesn't know about. PERFORMANCE IMPLICATIONS OF
SYSTEM MANAGEMENT MODE from 2013 says that 150 microseconds is considered an "acceptable" latency for an SMI, according to Intel's test suite for PC BIOSes. Modern PCs are full of voodoo. I think/hope that the firmware on most motherboards doesn't have much SMM overhead, and that SMIs are very rare in normal operation, but I'm not sure. See also Evaluating SMI (System Management Interrupt) latency on Linux-CentOS/Intel machine
Extremely low-power Skylake CPUs stop their clock with some duty-cycle, instead of clocking lower and running continuously. See this, and also Intel's IDF2015 presentation about Skylake power management.
Spin on RDTSC until the right wall-clock time
If you really need to busy-wait, spin on rdtsc waiting for the current time to reach a deadline. You need to know the reference frequency, which is not tied to the core clock, so it's fixed and nonstop (on modern CPUs; there are CPUID feature bits for invariant and nonstop TSC. Linux checks this, so you could look in /proc/cpuinfo for constant_tsc and nonstop_tsc, but really you should just check CPUID yourself on program startup and work out the RDTSC frequency (somehow...)).
I wrote such a loop as part of a silly-computer-tricks exercise: a stopwatch in the fewest bytes of x86 machine code. Most of the code size is for the string manipulation to increment a 00:00:00 display and print it. I hard-coded the 4GHz RDTSC frequency for my CPU.
For sleeps of less than 2^32 reference clocks, you only need to look at the low 32 bits of the counter. If you do your compare correctly, wrap-around takes care of itself. For the 1-second stopwatch, a 4.3GHz CPU would have a problem, but for nsec / usec sleeps there's no issue.
;;; Untested, NASM syntax
default rel
section .data
; RDTSC frequency in counts per 2^16 nanoseconds
; 3200000000 would be for a 3.2GHz CPU like your i3-3470
ref_freq_fixedpoint: dd 3200000000 * (1<<16) / 1000000000
; The actual integer value is 0x033333
; which represents a fixed-point value of 3.1999969482421875 GHz
; use a different shift count if you like to get more fractional bits.
; I don't think you need 64-bit operand-size
; nanodelay(unsigned nanos /*edi*/)
; x86-64 System-V calling convention
; clobbers EAX, ECX, EDX, and EDI
global nanodelay
nanodelay:
; take the initial clock sample as early as possible.
; ideally even inline rdtsc into the caller so we don't wait for I$ miss.
rdtsc ; edx:eax = current timestamp
mov ecx, eax ; ecx = start
; lea ecx, [rax-30] ; optionally bias the start time to account for overhead. Maybe make this a variable stored with the frequency.
; then calculate edi = ref counts = nsec * ref_freq
imul edi, [ref_freq_fixedpoint] ; counts * 2^16
shr edi, 16 ; actual counts, rounding down
.spinwait: ; do{
pause ; optional but recommended.
rdtsc ; edx:eax = reference cycles since boot
sub eax, ecx ; delta = now - start. This may wrap, but the result is always a correct unsigned 0..n
cmp eax, edi ; } while(delta < sleep_counts)
jb .spinwait
ret
To avoid floating-point for the frequency calculation, I used fixed-point like uint32_t ref_freq_fixedpoint = 3.2 * (1<<16);. This means we just use an integer multiply and shift inside the delay loop. Use C code to set ref_freq_fixedpoint during startup with the right value for the CPU.
If you recompile this for each target CPU, the multiply constant can be an immediate operand for imul instead of loading from memory.
pause sleeps for ~100 clock on Skylake, but only for ~5 clocks on previous Intel uarches. So it hurts timing precision a bit, maybe sleeping up to 100 ns past a deadline when the CPU frequency is clocked down to ~1GHz. Or at a normal ~3GHz speed, more like up to +33ns.
Running continously, this loop heated up one core of my Skylake i7-6700k at ~3.9GHz by ~15 degrees C without pause, but only by ~9 C with pause. (From a baseline of ~30C with a big CoolerMaster Gemini II heatpipe cooler, but low airflow in the case to keep fan noise low.)
Adjusting the start-time measurement to be earlier than it really is will let you compensate for some of the extra overhead, like branch-misprediction when leaving the loop, as well as the fact that the first rdtsc doesn't sample the clock until probably near the end of its execution. Out-of-order execution can let rdtsc run early; you might use lfence, or consider rdtscp, to stop the first clock sample from happening out-of-order ahead of instructions before the delay function is called.
Keeping the offset in a variable will let you calibrate the constant offset, too. If you can do this automatically at startup, that could be good to handle variations between CPUs. But you need some high-accuracy timer for that to work, and this is already based on rdtsc.
Inlining the first RDTSC into the caller and passing the low 32 bits as another function arg would make sure the "timer" starts right away even if there's an instruction-cache miss or other pipeline stall when calling the delay function. So the I$ miss time would be part of the delay interval, not extra overhead.
The advantage of spinning on rdtsc:
If anything happens that delays execution, the loop still exits at the deadline, unless execution is currently blocked when the deadline passes (in which case you're screwed with any method).
So instead of using exactly n cycles of CPU time, you use CPU time until the current time is n * freq nanoseconds later than when you first checked.
With a simple counter delay loop, a delay that's long enough at 4GHz would make you sleep more than 4x too long at 0.8GHz (typical minimum frequency on recent Intel CPUs).
This does run rdtsc twice, so it's not appropriate for delays of only a couple nanoseconds. (rdtsc itself is ~20 uops, and has a throughput of one per 25 clocks on Skylake/Kaby Lake.) I think this is probably the least bad solution for a busy-wait of hundreds or thousands of nanoseconds, though.
Downside: a migration to another core with unsynced TSC could result in sleeping for the wrong time. But unless your delays are very long, the migration time will be longer than the intended delay. The worst case is sleeping for the delay-time again after the migration. The way I do the compare: (now - start) < count, instead of looking for a certain target target count, means that unsigned wraparound will make the compare true when now-start is a large number. You can't get stuck sleeping for nearly a whole second while the counter wraps around.
Downside: maybe you want to sleep for a certain number of core cycles, or to pause the count when the CPU is asleep.
Downside: old CPUs may not have a non-stop / invariant TSC. Check these CPUID feature bits at startup, and maybe use an alternate delay loop, or at least take it into account when calibrating. See also Get CPU cycle count? for my attempt at a canonical answer about RDTSC behaviour.
Future CPUs: use tpause on CPUs with the WAITPKG CPUID feature.
(I don't know which future CPUs are expected to have this.)
It's like pause, but puts the logical core to sleep until the TSC = the value you supply in EDX:EAX. So you could rdtsc to find out the current time, add / adc the sleep time scaled to TSC ticks to EDX:EAX, then run tpause.
Interestingly, it takes another input register where you can put a 0 for a deeper sleep (more friendly to the other hyperthread, probably drops back to single-thread mode), or 1 for faster wakeup and less power-saving.
You wouldn't want to use this to sleep for seconds; you'd want to hand control back to the OS. But you could do an OS sleep to get close to your target wakeup if it's far away, then mov ecx,1 or xor ecx,ecx / tpause ecx for whatever time is left.
Semi-related (also part of the WAITPKG extension) are the even more fun umonitor / umwait, which (like privileged monitor/mwait) can have a core wake up when it sees a change to memory in an address range. For a timeout, it has the same wakeup on TSC = EDX:EAX as tpause.

Can the same data structure share more then 1 cache line

I'm a newbie in low-level programming and my current work is about Scala development. But I just questioned myself with something (probably very stupid question) I could not find an answer just by myself using Google or commonly available profiling tools.
Can the Integers (or any other data type 4 bytes size) share more than one L1-cache line.? I'm using Haswell.
From what I can tell... Yes, it can. Consider the following code:
;allocate 100 bytes with mmap
mov [rax + 62], dword -1 ; we misaligned the Integers which now has to share 2 cache lines.
mov ebx, [rax + 62] ; should result in 2 cache hit
Since cache line is 64B aligned we put int at 64b-aligned-address + 62 position so 2 bytes of the int are in 1 cache line and the other 2 in another.
Is my understanding correct? If no, can you please help me to understand how it works in a nutshell. I'm not asking about cache coherence. My program is run on a single core in a single thread.

Linux perf reporting cache misses for unexpected instruction

I'm trying to apply some performance engineering techniques to an implementation of Dijkstra's algorithm. In an attempt to find bottlenecks in the (naive and unoptimised) program, I'm using the perf command to record the number of cache misses. The snippet of code that is relevant is the following, which finds the unvisited node with the smallest distance:
for (int i = 0; i < count; i++) {
if (!visited[i]) {
if (tmp == -1 || dist[i] < dist[tmp]) {
tmp = i;
}
}
}
For the LLC-load-misses metric, perf report shows the following annotation of the assembly:
│ for (int i = 0; i < count; i++) { ▒
1.19 │ ff: add $0x1,%eax ▒
0.03 │102: cmp 0x20(%rsp),%eax ▒
│ ↓ jge 135 ▒
│ if (!visited[i]) { ▒
0.07 │ movslq %eax,%rdx ▒
│ mov 0x18(%rsp),%rdi ◆
0.70 │ cmpb $0x0,(%rdi,%rdx,1) ▒
0.53 │ ↑ jne ff ▒
│ if (tmp == -1 || dist[i] < dist[tmp]) { ▒
0.07 │ cmp $0xffffffff,%r13d ▒
│ ↑ je fc ▒
0.96 │ mov 0x40(%rsp),%rcx ▒
0.08 │ movslq %r13d,%rsi ▒
│ movsd (%rcx,%rsi,8),%xmm0 ▒
0.13 │ ucomis (%rcx,%rdx,8),%xmm0 ▒
57.99 │ ↑ jbe ff ▒
│ tmp = i; ▒
│ mov %eax,%r13d ▒
│ ↑ jmp ff ▒
│ } ▒
│ } ▒
│ }
My question then is the following: why does the jbe instruction produce so many cache misses? This instruction should not have to retrieve anything from memory at all if I am not mistaken. I figured it might have something to do with instruction cache misses, but even measuring only L1 data cache misses using L1-dcache-load-misses shows that there are a lot of cache misses in that instruction.
This stumps me somewhat. Could anyone explain this (in my eyes) odd result? Thank you in advance.
About your example:
There are several instructions before and at the high counter:
│ movsd (%rcx,%rsi,8),%xmm0
0.13 │ ucomis (%rcx,%rdx,8),%xmm0
57.99 │ ↑ jbe ff
"movsd" loads word from (%rcx,%rsi,8) (some array access) into xmm0 register, and "ucomis" loads another word from (%rcx,%rdx,8) and compares it with just loaded value in xmm0 register. "jbe" is conditional jump which depends on compare outcome.
Many modern Intel CPUs (and AMD probably too) can and will fuse (combine) some combinations of operations (realworldtech.com/nehalem/5 "into a single uop, CMP+JCC") together, and cmp + conditional jump very common instruction combination to be fused (you can check it with Intel IACA simulating tool, use ver 2.1 for your CPU). Fused pair may be reported in perf/PMUs/PEBS incorrectly with skew of most events towards one of two instructions.
This code probably means that expression "dist[i] < dist[tmp]" generates two memory accesses, and both of values are used in ucomis instruction which is (partially?) fused with jbe conditional jump. Either dist[i] or dist[tmp] or both expressions generates high number of misses. Any of such miss will block ucomis to generate result and block jbe to give next instruction to execute (or to retire predicted instructions). So, jbe may get all fame of high counters instead of real memory-access instructions (and for "far" event like cache response there is some skew towards last blocked instruction).
You may try to merge visited[N] and dist[N] arrays into array[N] of struct { int visited; float dist} to force prefetching of array[i].dist when you access array[i].visited or you may try to change order of vertex access, or renumber graph vertex, or do some software prefetch for next one or more elements (?)
About generic perf event by name problems and possible uncore skew.
perf (perf_events) tool in Linux uses predefined set of events when called as perf list, and some listed hardware events can be not implemented; others are mapped to current CPU capabilities (and some mappings are not fully correct). Some basic info about real PMU is in your https://software.intel.com/sites/products/collateral/hpc/vtune/performance_analysis_guide.pdf (but it has more details for related Nehalem-EP variant).
For your Nehalem (Intel Core i5 750 with L3 cache of 8MB and without multi-CPU/multi-socket/NUMA support) perf will map standard ("Generic cache events") LLC-load-misses event as .. "OFFCORE_RESPONSE.ANY_DATA.ANY_LLC_MISS" as written in the best documentation of perf event mappings (the only one) - kernel source code
http://elixir.free-electrons.com/linux/v4.8/source/arch/x86/events/intel/core.c#L1103
u64 nehalem_hw_cache_event_ids ...
[ C(LL ) ] = {
[ C(OP_READ) ] = {
/* OFFCORE_RESPONSE.ANY_DATA.LOCAL_CACHE */
[ C(RESULT_ACCESS) ] = 0x01b7,
/* OFFCORE_RESPONSE.ANY_DATA.ANY_LLC_MISS */
[ C(RESULT_MISS) ] = 0x01b7,
...
/*
* Nehalem/Westmere MSR_OFFCORE_RESPONSE bits;
* See IA32 SDM Vol 3B 30.6.1.3
*/
#define NHM_DMND_DATA_RD (1 << 0)
#define NHM_DMND_READ (NHM_DMND_DATA_RD)
#define NHM_L3_MISS (NHM_NON_DRAM|NHM_LOCAL_DRAM|NHM_REMOTE_DRAM|NHM_REMOTE_CACHE_FWD)
...
u64 nehalem_hw_cache_extra_regs
..
[ C(LL ) ] = {
[ C(OP_READ) ] = {
[ C(RESULT_ACCESS) ] = NHM_DMND_READ|NHM_L3_ACCESS,
[ C(RESULT_MISS) ] = NHM_DMND_READ|NHM_L3_MISS,
I think this event is not precise: cpu pipeline will post (with out-of-order) load request to the cache hierarchy and will execute other instructions. After some time (around 10 cycles to reach and get response from L2 and 40 cycles to reach L3) there will be response with miss flag in the corresponding (offcore?) PMU to increment counter. On this counter overflow, profiling interrupt will be generated from this PMU. In several cpu clock cycles it will reach pipeline to interrupt it, perf_events subsystem's handler will handle this with registering current (interrupted) EIP/RIP Instruction pointer and reset PMU counter back to some negative value (for example, -100000 to get interrupt for every 100000 L3 misses counted; use perf record -e LLC-load-misses -c 100000 to set exact count or perf will autotune limit to get some default frequency). The registered EIP/RIP is not the IP of load command and it may be also not the EIP/RIP of command which wants to use the loaded data.
But if your CPU is the only socket in the system and you access normal memory (not some mapped PCI-express space), L3 miss in fact will be implemented as local memory access and there are some counters for this... (https://software.intel.com/en-us/node/596851 - "Any memory requests missing here must be serviced by local or remote DRAM").
There are some listings of PMU events for your CPU:
Official Intel's "Intel® 64 and IA-32 Architectures Software Developer Manuals" (SDM): https://software.intel.com/en-us/articles/intel-sdm, Volume 3, Appendix A
3B: https://software.intel.com/sites/default/files/managed/7c/f1/253669-sdm-vol-3b.pdf "18.8 PERFORMANCE MONITORING FOR PROCESSORS BASED ON INTEL® MICROARCHITECTURE CODE NAME NEHALEM" from page 213 "Vol 3B 18-35"
3B: https://software.intel.com/sites/default/files/managed/7c/f1/253669-sdm-vol-3b.pdf "19.8 - Processors based on Intel® microarchitecture code name Nehalem" from page 365 and "Vol. 3B 19-61")
Some other volume for Offcore response encoding? Vol. 3A 18-26?
from oprofile http://oprofile.sourceforge.net/docs/intel-corei7-events.php
from libpfm4's showevtinfo http://www.bnikolic.co.uk/blog/hpc-prof-events.html (note, this page with Sandy Bridge list, get libpfm4 ant run on your PC to get your list). There is also check_events tool in libpfm4 to help your encode event as raw for perf.
from VTune documentation: http://www.hpc.ut.ee/dokumendid/ips_xe_2015/vtune_amplifier_xe/documentation/en/help/reference/pmw_sp/events/offcore_response.html
from Nehalem PMU guide: https://software.intel.com/sites/default/files/m/5/2/c/f/1/30320-Nehalem-PMU-Programming-Guide-Core.pdf
ocperf tool from Intel's perf developer Andi Kleen, part of his pmu-tools https://github.com/andikleen/pmu-tools. ocperf is just wrapper for perf and this package will download event description and any supported event name will be converted into correct raw encoding ofperf`.
There should be some information about ANY_LLC_MISS offcore PMU event implementation and list of PEBS events for Nhm, but I can't find it now.
I can recommend you to use ocperf from https://github.com/andikleen/pmu-tools with any PMU events of your CPU without need to manually encode them. There are some PEBS events in your CPU, and there is Latency profiling / perf mem for some kind of memory access profiling (some random perf mem pdfs: 2012 post "perf: add memory access sampling support",RH 2013 - pg26-30, still not documented in 2015 - sowa pg19, ls /sys/devices/cpu/events). For newer CPUs there are newer tools like ucevent.
I also can recommend you to try cachegrind profiler/cache simulator tool of valgrind program with kcachegrind GUI to view profiles. Valgrind-based profilers may help you to get basic idea about how the code works: they collect exact instruction execution counts for every instruction, and cachegrind also simulates some abstract multi-level cache. But real CPU will execute several instruction per cycle (so, callgrind/cachegrind cost model of 1 instruction = 1 cpu clock cycle gives some error; cachegrind cache model have not the same logic as real cache). And all valgrind tools are dynamic binary instrumentation tools which will slow down your program 20-30 times compared to native run.
When you read a memory location, the processor will try to prefetch the adjacent memory locations and cache them.
That works well if you are reading an array of objects which are all allocated in memory in contiguous blocks.
However, if for example you have an array of pointers which live in the heap, it is less likely that you will be iterating over contiguous portions of memory unless you are using some sort of custom allocator specifically designed for this.
Because of this, dereferencing should be seen as some sort of cost. An array of structs can be more efficient to an array of pointers to structs.
Herb Sutter (member of C++ commitee) speaks about this in this talk https://youtu.be/TJHgp1ugKGM?t=21m31s

Game Boy: What constitutes a "half-carry"?

The Game Boy Z80 CPU has a half-carry flag, and I can't seem to find much information about when to set/clear it.
What I understand so far is that any 8-bit add, subtract, shift, or rotate operation (and maybe others?) set it to bit 4 of the result(?), and the DAA instruction sets/uses this somehow. What I'm not sure is how 16-bit instructions affect it and whether it's affected or not by the use of certain registers.
It's the carry from bit 3 to bit 4, just like the normal carry flag records carry from bit 7. So, e.g. to get the half carry bit in an add:
((a&0xf) + (value&0xf))&0x10
Which gives 0x10 if half carry should be set, 0 otherwise. Getting half carry from the other relevant ops follows naturally - the questions is whether there was carry from the low nibble to the high.
To put things in perspective, the z80 has a 4bit ALU and performs 8bit ops by doing two 4bit ops. So it gets half carry very naturally, as an intermediate result.
DAA is interested in the flag because if half carry is set then two digits that add up to more than 16 were added in the low nibble; that will have correctly produced carry into the upper nibble but will have left the low nibble 6 lower than it should be, since there were six more values between 10, when it should have generated carry, and 16, when it did.
For 16-bit operations, the carry from bit 3 to bit 4 in the register's high byte sets the flag. In other words, bit 11 to bit 12.
(Note the above bits are labeled 0-15, from least to most significant)
See here: http://www.z80.info/z80code.htm
16 bit arithmetic
If you want to add numbers that are more than the 0-255 that can
be stored in the A register, then the HL, IX or IY registers can
be used. Thus LD HL,1000H:LD BC,2000H:ADD HL,BC will give
A CZPSNH BC DE HL IX IY A' CZPSNH' BC' DE' HL' SP
00 000000 2000 0000 3000 0000 0000 00 000000 0000 0000 0000 0000
The flags are set as follows.
C or carry flag 1 if answer >65535 else 0
Z or zero flag not changed
P flag not changed
S or sign flag not changed
N flag 0
H or half carry flag 1 if carry from bit 11 to bit 12 else 0
Since the half-carry flag is one of the most common stumbling blocks for Game Boy emulator makers, I'll take the liberty to post a link to a recent question of mine regarding the subject as an answer:
Game Boy: Half-carry flag and 16-bit instructions (especially opcode 0xE8)
A summary of the above thread (answer by #gekkio):
It depends on the instruction, but the flags are always updated based on the same bit positions if you think in terms of 8-bit values...it just varies whether we're talking about the high or low byte of the 16-bit value. Bit 11 is just bit 3 of the high byte.
ADD SP, e: H from bit 3, C from bit 7 (flags from low byte op)
LD HL, SP+e: H from bit 3, C from bit 7 (flags from low byte op)
ADD HL, rr: H from bit 11, C from bit 15 (flags from high byte op)
INC rr: no flag updates (executed by the 16-bit inc/dec unit)
DEC rr: no flag updates (executed by the 16-bit inc/dec unit)

Assembly: what are semantic NOPs?

I was wondering what are "semantic NOPs" in assembly?
Code that isn't an actual nop but doesn't affect the behavior of the program.
In C, the following sequence could be thought of as a semantic NOP:
{
// Since none of these have side affects, they are effectively no-ops
int x = 5;
int y = x * x;
int z = y / x;
}
They are instructions that have no effect, like a NOP, but take more bytes. Useful to get code aligned to a cache line boundary. An instruction like lea edi,[edi+0] is an example, it would take 7 NOPs to fill the same number of bytes but takes only 1 cycle instead of 7.
A semantic NOP is a collection of machine language instructions that have no effect at all or almost no effect (most instructions change condition codes) whose only purpose is obfuscation of what the program is actually doing.
Code that executes but doesn't do anything meaningful. These are also called "opaque predicates," and are used most often by obfuscators.
A true "semantic nop" is an instruction which has no effect other than taking some time and advancing the program counter. Many machines where register-to-register moves do not affect flags, for example, have numerous instructions that will move a register to itself. On the 8088, for example, any of the following would be semantic NOPs:
mov al,al
mov bl,bl
mov cl,cl
...
mov ax,ax
mob bx,bx
mov cx,cx
...
xchg ax,ax
xchg bx,bx
xchg cx,cx
...
Note that all of the above except for "xchg ax,ax" are two-byte instructions. Intel has therefore declared that "xchg ax,ax" should be used when a one-byte NOP is required. Indeed, if one assembles "mov ax,ax" and disassembles it, it will disassemble as "NOP".
Note that in some cases an instruction or instruction sequence may have potential side-effects, but nonetheless be more desirable than the usual "nop". On the 6502, for example, if one needs a 7-cycle delay and the stack pointer is valid but the top-of-stack value is irrelevant, a PHP followed by a PLP will kill seven cycles using only two bytes of code. If the top-of-stack value isn't a spare byte of RAM, though, the sequence would fail.

Resources