How to implement GetTargetLUNs in vss Hardware provider? - volume-shadow-service

I am implementing VSS Hardware provider for ZFS based iSCSI Target. We have implemented AreLunSupported, precommitsnapshot and commitsnapshot etc functions and till this point it is working fine. But after this it is failing with "VSS_E_NO_SNAPSHOTS_IMPORTED" error in LocateLun method. and I think we are not filling Target LUN information properly.
My questions are:
How to find serial number of target LUN ? Do I need to mount newly created snapshot and then get the serial number ?
Do we need to fill interconnect, storage identifier information also or can I just pass NULL for these.

Q: How to find serial number of target LUN ? Do I need to mount newly created snapshot and then get the serial number ?
No, you should not mount the snapshot at this point. You should use an out-of-band mechanism to directly communicate with your storage (I'm assuming your 'ZFS based iSCSI target' is coming from a NAS box), probably a REST API call, to figure out the serial number of the snapshot.
Let me elaborate some more on serial number of the snapshot:
VSS expects the 'shadow copy' to be a concrete, real volume, similar to the primary volume (in your case an iSCSI target)
Since you are using ZFS snapshots, without dwelling much into your exact implementation, you have 2 options to obtain the serial number for a concrete LUN:
a. If your storage allows exposing a ZFS snapshot directory as a iSCSI target, the create that iSCSI target and use its Page83 identifier
b. If not, create a ZFS clone using the ZFS snapshot and expose that as an iSCSI target and use its Page83 identifier
Q: Do we need to fill interconnect, storage identifier information also
or can I just pass NULL for these.
For all practical purposes, it usually suffices to simply copy the VDS_LUN_INFORMATION for the original source LUN and only edit the m_szSerialNumber field with that of the target LUN (assuming that the product ID, vendor ID etc. all will remain the same)
This link explains in detail what is expected out of a VSS Hardware Provider implementation: https://msdn.microsoft.com/en-us/library/windows/desktop/aa384600(v=vs.85).aspx
Unique Page 83 Information
Both the original LUN and the newly created shadow copy LUN must have
at least one unique storage identifier in the page 83 data. At least
one STORAGE_IDENTIFIER with a type of 1, 2, 3, or 8, and an
association of 0 must be unique on the original LUN and the newly
created shadow copy LUN.
Bonus chatter (Answer ends at this point):
Now, #2(b) above might raise eyebrows since you are creating a clone ahead-of-time and it is not yet being used. The reason for this is, the above steps need to be performed in IVssHardwareSnapshotProvider::FillInLunInfo and this same VDS_LUN_INFORMATION contents are passed later to IVssHardwareSnapshotProvider::LocateLuns (VSS is trying to tell you to locate the LUNs that you earlier told it were the shadow copy LUNs). Hence, regardless of whether you will be using the clone or not in future, you must have the concrete LUN (iSCSI target) created upfront.
A silver lining to this is: if you are sure that the workflow of the VSS Requestor will never mount the shadow copy, then you can get away with this by faking some (valid) info in VDS_LUN_INFORMATION during IVssHardwareSnapshotProvider::FillInLunInfo. For this to work, you will have to create a 'transportable' shadow copy (the VSS requestor uses the VSS_CTX_FILE_SHARE_BACKUP | VSS_VOLSNAP_ATTR_TRANSPORTABLE flags). The only use-case for such a shadow copy would be to perform a hardware-resync on it, in which the VSS Hardware Provider implements the IVssHardwareSnapshotProvider::ResyncLuns method and performs a ZFS snapshot rollback in it.

Related

When and which function is used for modifying file on sysfs of Linux?

i'm analizing block layer's sysfs functions.
i added(attached) a file(diagram) which i made to explain function sequence flow of
/usr/src/linux-source-4.8.0/linux-source/4.8.0/block/blk-mq-sysfs.c.
I understanded these functions' relationship. but i couldn't find how kernel change values of attribute file.
i heard that these files are created in the /sysfs/ hierarchy by calling sysfs_create_group() function.
When i do some I/O requests, system make some files like below.
(i use nvme ssd 750 series)
root#leedoosol:/sys/devices/pci0000:00/0000:00:03.0/0000:03:00.0/nvme/nvme0/nvme0n1/mq/0/cpu0# ls
completed dispatched merged rq_list
kernel would have made these files to give us information about completed request numbers, dispatched number, merged number, pending request_list.
And kernel should have changed value of these file while dealing with I/O request. but i don't know when and how kernel change these value.
i want to know when and how kernel change these values of attribute file because i have to find out what these values of attribute file means exactly.
here my environments.
1.) 2 socket per 10cores
2.) Kernel version : 4.8.17
3.) intel SSD 750 series
maybe I found answer. show store functions are called when i read my attribute file.
kernel don't fix their value of attribute file. kernel don't need to.
when i use 'cat' to the attribute file(in my example 'dispatched'), the file will be opened and then, several structs concerned with that file will be created in ram(of coures, in case of sysfs, backing store will not exist).
read() function will be called and then, show() function will called.

Missing VolumeGUIDs in Windows 2016 MountedDevices registry key

Our software runs on Linux and we need to create a mapping between Linux device name(something like /dev/sda1) and VolumeGUID as it appears in Windows, since we are examining Windows disks/partitions.
We get this information from MountedDevices Windows Registry subkey.
Problem occurs on Windows 2016, where Volume{GUID}s are no longer listed in MountedDevices subkey.
I managed to figure out, that Volume{GUID} is not a random GUID anymore(which is probably why they do not have to be stored it registry anymore), but it gets composed from data in the partition table.
In case of GPT, the VolumeGUID is actually a GPT partition GUID, which is great, because I can easily reconstruct those VolumeGUIDs.
In case of MBR, it is something like:
\?\Volume{46e21ed5-0000-0000-0000-100000000000}\
\?\Volume{46e21ed5-0000-0000-0000-104000000000}\
\?\Volume{46e21ed5-0000-0000-0000-108000000000}\
\?\Volume{46e21ed5-0000-0000-0000-20c000000000}\
...
Where 46e21ed5 is actually a disk signature, but I'm not sure what other fields mean. It looks like there's a partition offset (0x400 = 1024, and each partition is 1024MB in the provided example), but something does not add up for the last partition which has 20C00.
Does anyone have more information on this subject and how these volume GUIDs get composed? (Google does not find any information on this subject)
Regards

DMIDecode product_uuid and product_serial.what is the difference?

There are product_uuid and product_serial files in dir /sys/class/dmi/id/.
How it are generated? What is the difference?
Can I change this files?
Is it save a value after reinstall operation system?
How it are generated?
Those values are generated in kernel code. You can find them pretty easily using git grep command (with keywords you are interested in) in your kernel source directory:
$ git grep --all-match -n -e '\bdmi\b' -e product_uuid -e product_serial
So, product_uuid and product_serial sysfs nodes are created in drivers/firmware/dmi-id.c:
DEFINE_DMI_ATTR_WITH_SHOW(product_serial, 0400, DMI_PRODUCT_SERIAL);
DEFINE_DMI_ATTR_WITH_SHOW(product_uuid, 0400, DMI_PRODUCT_UUID);
From DEFINE_DMI_ATTR_WITH_SHOW definition you can see that both attributes are accessed via sys_dmi_field_show() function, which in turn calls dmi_get_system_info(), which just returns corresponding element from dmi_ident array. This table is populated in dmi_decode() routine:
dmi_save_ident(dm, DMI_PRODUCT_SERIAL, 7);
dmi_save_uuid(dm, DMI_PRODUCT_UUID, 8);
So product_uuid is generated in dmi_save_uuid() function. Just read its code to understand how it's done.
product_serial is generated in dmi_save_ident() function. It boils down to code like this:
(struct dmi_header *)(dmi_base)[7];
where dmi_base is address (remapped to virtual memory obviously) of DMI table, and 7 corresponds to DMI_PRODUCT_SERIAL constant.
To better understand this please see SMBIOS specification, specifically Table 9 – System Information (Type 1) Structure, which corresponds to this command:
# dmidecode --type 1
What is the difference?
As for product_uuid -- look at SMBIOS specification, section 7.2.1 System - UUID. It has description and also table with explanation for each part of this number. Using that table you can decode your UUID and extract some information from it, like timestamp, etc.
As for product_serial -- I believe it's self-explanatory, it's just a serial number for your device. You can usually find it printed on some sticker on your computer. For example, for my laptop it's on the bottom. It's the same string that I see in /sys/class/dmi/id/product_serial.
Can I change this files?
Those files are actually not real files but just an interface to kernel functions. Read about sysfs for details. So in order to "change" those files you need to edit mentioned kernel files accordingly, then rebuild the whole kernel and boot it (instead of one provided by your distribution).
Also, as #ChristopheVu-Brugier mentioned in comment, you can change those values in DMI table (in some tricky way though). But I wouldn't recommend it. Those values definitely have some meaning and may be useful in some cases (if not for you, then for some software in your PC).
Is it save a value after reinstall operation system?
Those values are actually obtained from DMI table, which is hardcoded along with BIOS to permanent memory (flash chip with BIOS on your motherboard) and you just read those values from this DMI table using kernel functions by reading those files.

AHCI specification

I have a question regarding the AHCI spec:
Is the variable pDmaXferCnt in the port used when the transfer is a DMA write or read?
The description in the spec seems to indicate that it isn't, but the PRDs are used instead.
But how does the HBA know how much data is to be sent or received to/from a SATA device?
This information will be available in the sector count of a H2D FIS, but unless I have overlooked it there doesn't seem to be a register of variable that holds this value.
The DX:transmit state also seems to indicate that pDmaXferCnt will have a set value, yet I can’t see where it would be set for a DMA read/write operation.
Thanks
From the spec:
"Implementation Note:
HBA state variables are used to describe the required externally visible behavior. Implementations are not required to have internal state values that directly correspond to these variables." - meaning you (maybe) won't find the pDmaXferCnt externally in a register.
There is another way to track the count though.
Under the HBA Memory Space Usage part of the AHCI spec, there are the data structures of the command list (list of command headers) and the command table (pointed to by the command header, each command table is a command to be sent). These are both accessible to the HBA.
In the command header in DW0 is the PRDTL - which is the count of how many PRD's to be used in the transfer.
Now in the actual command table that the command header points to, contains the actual PRD's, in each PRD is their own DBC or data byte count (amount of data in bytes to be DMA'd at the location specified in DBA). So if you take the each of the PRD's * there own DBC's and add them up you'll get the amount of data to be transfered.
Alternately in the in the command header DW1 is the PRDBC which is the count of bytes transfered, so you could check that after the command.
HBA - Host Bus Adapter
PRDTL - Physical Region Descriptor Table Length
PRD - Physical Region Descriptor (tracks where in physical memory and the count of bytes is to be transfered)
DBC - data byte count (inside a PRD)
DBA - data base address (physical address inside a PRD)
PRDBC - Physical Region Descriptor Byte Count
DMA - direct memory access
For more reading: http://www.intel.com/content/www/us/en/io/serial-ata/serial-ata-ahci-spec-rev1-3-1.html

How do you find the major and minor numbers for devices in linux

Trying to get access to a partially rooted Galaxy S2 external sd card.
The problem is that /dev/block/mmcblk1p1 does not exist on the phone. This is the device name that should allow me to put the "recovery" image onto the sdcard so that the unit will be a phone again.
Problem is, I don't know where to find the magic Major and Minor numbers for this device and I'm trying to figure out where in the kernel source I should be looking for them.
Could someone point me at the right kernel files to find this information?
Standard devices use predefined major numbers and minor numbers starting from 0 for the first instance and upward depending on how many instances there are going to be.
Look at the Linux Documentation file(devices.txt) to see the full list but the section of interest to you is:
179 block MMC block devices
0 = /dev/mmcblk0 First SD/MMC card
1 = /dev/mmcblk0p1 First partition on first MMC card
8 = /dev/mmcblk1 Second SD/MMC card
...
The start of next SD/MMC card can be configured with
CONFIG_MMC_BLOCK_MINORS, or overridden at boot/modprobe
time using the mmcblk.perdev_minors option. That would
bump the offset between each card to be the configured
value instead of the default 8.
So /dev/block/mmcblk1p1 would be major 179, minor 9.
According to hotplug.txt
Entries for block devices are found at the following locations:
/sys/block/*/dev
/sys/block/*/*/dev
So try looking in /sys/block/mmcblk1p1/dev.
EDIT:
Looking at it again I actually think that it will be in /sys/block/mmcblk1/mmcblk1p1/dev

Resources