I've written a script to backup my server's HD every night. At the end of the script, I sync, wait a couple of minutes, sync and then I issue sg_start --stop to stop the device. The idea is to extend the lifetime of the device by switching the HD off after ten minutes of incremental backup (desktop disks will survive several thousand on/off cycles but only a few hundred hours of continuous running).
This doesn't always work; I often find the drive still spinning the next morning. Is there a shell command which I can use to check that the drive has stopped (so I can issue the stop command again [EDIT2]or write a script to create a process list when the drive is running so I can debug this[/EDIT2])?
[EDIT] I've tried sg_inq (as suggested by the sg_start man page) but this command always returns 0.
I've tried hdparm but it always returns "drive state is: unknown" for USB drives (attached via /dev/sdX) and when trying to spin down the drive, I get "HDIO_DRIVE_CMD(setidle1) failed: Input/output error".
sdparm seems to support to set the idle timer on the drive (see "Power mode condition page") but the IDLE option has "Changeable: n" and I haven't found an option which tells me the drive power state.
[EDIT2] Note: I can stop the drive with sg_start --stop from the console. That always works; it just doesn't always stay down until midnight. The sever is in the basement (where it's nice and cool) and I'd rather have a way to check whether the drive is up or not from the warm living room :) If I had a command which told me the status of the drive, I could write a script to alert me when it spins up (check every minute) and then I could try to figure out what might be causing this.
If that matters: I'm using openSUSE 11.1.
When you say you've tried hdparm, you haven't said what you have tried. I have some usb hard drives in an enclosure, and some of the commands work for it and others don't, but I guess it all depends on all facets of the transport mechanism.
hdparm -S 120 /dev/sda
Should tell the drive to sleep itself after ~10 minutes of inactivity.
I'm guessing you have already tried this, but its not obvious, and writing this as an answer may help a future reader.
Nothing accesses the drive but the backup script.
This is nice in theory, but I have found on odd occasions this is not enough. There are lots of processes and tasks that if they even look at the drive a spin up may occur if the lookup is for some reason out of the disk-cache.
Common culprits are tools like updatedb scanning all mounts for files, and fam or gamin doing funky stuff to monitor disks for changes.
If in doubt, add a layer of certainty by mounting the device before executing the script, and unmounting it when you are done.
Seeing things that could cause a wakeup
lsof +D /mountpoint
You should probably parse the output of this as well before attempting to unmount to be sure that nothing is still trying to use it.
You should probably also be doing a lazy umount,
umount -l /mountpoint
so if anything accesses it between you doing lsof| grep and calling umount, it will still unmount the drive and stop things being able to read it.
HAL and friends
Its also possible that HAL and friends are doing wakeups to the drive probing for connect/removal state. I really hope it isn't, but it does that on some device types. It seems an unlikely cause, but I'll consider anything possible.
Try fun stuff like
lsof /dev/devicehere
and
lsof /dev/devicehere1
Which appears to get a comprehensive list of all things that would be accessing the handle either directly or indirectly.
You also need to check the mount options and use "noatime" as a mount option, otherwise the kernel still updates the access times periodically. So this may be the cause of your problem too.
If hdparm returns this:
drive state is: unknown
And smartctl returns this:
CHECK POWER MODE: incomplete response, ATA output registers missing
CHECK POWER MODE not implemented, ignoring -n option
Then it is not possible to obtain the usb drive sleep status. I tried multiple creative things like the USB power consumption or monitoring file access, but I found nothing to verify the drive's state.
The only solution is to replace the USB SATA adapter (or enclosure) against one with a different USB bridge chip. The one that did not work for me, used the JMicron JMS567 (152d:0578 / 0x152d 0x0578) which could be perhaps updated, but I was not able to test it as the update tool needs an ARM CPU. Now I'm using two different adapters which both use the Asmedia ASM1051E / ASM1053E / ASM1153 / ASM1153E and they passthrough all commands.
You can check the bridge chip of your USB SATA adapter / enclosure as follows:
lsusb -v | grep -E ': ID |idVendor|idProduct'
Which returns something like:
Bus 002 Device 003: ID 174c:55aa ASMedia Technology Inc. ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge
idVendor 0x174c ASMedia Technology Inc.
idProduct 0x55aa ASM1051E SATA 6Gb/s bridge, ASM1053E SATA 6Gb/s bridge, ASM1153 SATA 3Gb/s bridge
I often find the drive still spinning the next morning.
Couldn't this just be because it was spun up again when the server eg. wrote to a log file or something during the night? You might try sdparm, which can return status information on a drive. But I think it's better to just set the option in your BIOS that lets your HD automatically spin down after an amount of inactivity, it's easier.
Have you tried stopping the drive with the following command:
eject -t /dev/yourHD
This works quite good for my USB hard drives.
If hdparm -C /dev/sda said drive state is: unknown than it's not supported for your drive and there is no way to tell
Your additional questions (answered by others already)
what's using a drive?
lsof
fuser
triggers
how to force a drive to stay sleeping?
hdparm
umount
your device can still be woken when unmounted but likely only by something you do (smartctl, blkid, etc)
Related: you can also automatically pause backup or whatever for an hour if your drives get to hot.
Related
I've got a quick question, but I can't find an answer.
Is it possible in linux (or in python) to see if an external usb pen drive is idling?
I need to know this for a python script I'm writing.
I need to rename a folder on an external usb pen drive as soon as nothing is writing to it.
edit: I know there is lsof command to list open files. 'lsof /theDir' only works half. It works OK when the process copying to the USB is still running. But when the process stops, lsof shows nothing. But the OS is still writing to the USB from its buffer.
You can check if all I/O has been processed by having a look at /sys/block/<dev>/stat.
The ninth column contains the number of I/Os currently in flight. Check https://www.kernel.org/doc/Documentation/block/stat.txt
Once this numner is zero the device should be idle.
To force all buffers to be written immediately you could execute sync and wait until it returns.
Nevertheless be aware that you have a race condition here if you are not controlling the writing - after you decided that the device is idle some other process could start writing to it.
I have a rooted Sony prs900, running a linux 2.6.23 #2 PREEMPT kernel, for ARMv6. (Montavista linux kernel). I'm having problems with figuring out how power management works, both for running the system and for powering up and down the audio port.
I can neither figure out how to read the battery/powerline status information, nor get the audio chip to play sound, etc ... although I have been studying the kernel modules for a while...
It's worth a little money for help, say $100 paypal donation to an email account, (or more if this takes a long time...) for the first person able to explain to me how to do them in a way that works.
Eg: read battery status, and change some power modes like getting the audio amplifiers to power up/down so that the audio played to /dev/dsp (oss emulation) actually comes out as sound rather than just being consumed by the chip and ignored...
The actual sony kernel, and binary packages of cross compiler tools are located on the main page. Actual kernel sourcecode is also available.
What I have learned so far myself :
The sony is using a wolfson micro WM8350 audio driver and battery charger/power management chip for all the system's power; eg: it can power down/up the SD memory cards, send more power to the cpu, power up audio amplifiers, etc. See: WM8350 Datasheet.
Pretty much, the whole problem revolves around getting the WM8350 kernel drivers to work...
Although the company brags quite a bit about it's support under linux, they don't have any application notes or examples that are actually helpful that I can find, other than the datasheet. I suspect the kernel drivers I have are beta code, because they don't seem to be behaving well (several error messages in the kernel log about wm8350 registers not being readable happen at every boot even when running only the sony's native software...).
The kernel driver's source-code of most interest are in: linux-2.6.23_091126/drivers/mxc/pmic/{core,wm8350}
Notice, the wm8350 is a competitor to the MC14783, but the linux kernel drivers use the same {core} driver source code for both chips; The sony ONLY has the wm8350 on it -- there is no MC14783 present.
The code that I most want most desperately to understand how to make operate is found in the subdirectory {wm8350}, eg: wm8350/wm8350pm/power_supply_sysfs.c.
I want the audio to fire up too, but 'm not quite sure where the pertinent audio amplifier code is yet...
Very clearly the wm8350pm code is designed to export a /sys directory interface; right now /sys is mounted and operational on the system; but I'm not very familiar with the semantics of these newer style interfaces... they aren't quite like the old APM power interfaces for Linux laptops...
First I checked the obvious:
If I do a "cat /sys/power/state" it returns the word "mem" and nothing else.
The file has permissions -rw-r--r--, so potentially it could be written -- but I don't know with what. The string "mem" does not exist anywhere in the source code for the wm8350pm drivers, so I don't even know if /sys/power/state is part of the source code.
Doing a find /sys -iname "wm8350" reveals a handful of directories with the patterns:
wm8350-rtc , wm8350-pmic , wm8350-bl , wm8350-power , wm8350-led
wm8350-hifi-dai , wm8350-codec
wm8350-imx32ads.0
So, I do an ls-l on each directory, and look for actual files rather than symbolic links or subdirectories, and what I find are stock useless writable files: bind, unbind, uevent,
and a very few read only files: pmic_reg, dapm_widget, modalias, codec_reg which aren't very helpful.
It's no surprise that:
Doing: cat /sys/devices/platform/wm8350-ebx5016-audi/modalias gives "wm8350-ebx5016-audio"
Doing: cat /sys/devices/platform/wm8350-imx32ads.0/modalias gives "wm8350-imx32ads"
and since audio is off... Doing: cat /sys/devices/platform/wm8350-ebx5016-audi/dapm_widget reveals the audio state:
Headphone Jack: Off
Line In Jack: On
Mic Bias: Off
Left DAC: Off
Right DAC: Off
... (all else off and omitted except )...
EBX5016-hifi: PM State: D3hot
The last two files, I expect should do wm8350 chip register dumps... and one did.
Doing: cat /sys/devices/wm8350-pmic/pmic-reg causes a long pause, then nothing is printed.
but:
Doing: cat /sys/devices/wm8350/platform/wm8350-ebx5016-audi/wm8350-codec/codec_reg does prints a list of registers up to e8 which is just a few bytes larger than the datasheet says the chip should be (0x00 to 0xe6).
I tried using a python program to play wav files, (works on my desktop computer), and I noticed that /dev/dsp does open, the mixers DO set volume levels, and nothing comes out. So -- the audio driver is not able to enable the sound amplifiers on it's own automatically.
There are no alsa sound files in /dev, nor are any alsa tools found on the embedded machine... so I assume Sony is strictly using OSS /dev/dsp and /dev/mixer.
There is only one other access point I can find to the ws8350:
There IS a device driver /dev/wm8350.
That driver created by the source code in subdirectory wm8350/wm8350_reg.c ; in theory it should be able to read and write to all registers using ioctls() calls from a user space. However, something appears grossly wrong with it, for I wrote a test program to read the wm8350 registers... and most of the registers return error messages rather than allowing to be read, including the most pulic ID registers (0x00, 0x01) etc.
So, I'm quite stuck. Pointers, thoughts, hints, are quite desired.
I would like to change your question a little bit.
How does Linux ASOC (alsa system on chip) power management work?
I will answer this and then give some hints on using this specific chip.
.. If I do a cat /sys/power/state it returns the word "mem" and nothing else. The file has permissions -rw-r--r--, so potentially it could be written -- but I don't know with what. The string "mem" does not exist anywhere in the source code for the wm8350pm drivers, so I don't even know if /sys/power/state is part of the source code.
You need to get an understanding of the Linux driver model. Hardware in Linux is structured like a tree. The rational is that things must be powered up/down in specific sequences. For instance, you should not power down the PCI bus controller before powering down the PCI peripherals. Linux builds a tree of hardware and each driver (code) and device (data/actual hardware) has specific call backs/function pointers which handle some specific tasks.
probe - Are you there? Determines actual hardware/device is present.
remove - Shuts down device. Module removal, power off, etc.
suspend - going to sleep.
resume - waking up.
Three and four may look interesting to you. Now, to read about what /sys/power/state is about. The text mem, means that suspend to memory is supported by your system. In this mode, Linux does these steps,
Find first lowest level active bus.
suspend devices on that bus.
suspend bus and de-activate.
If a bus is active go to step 1.
Set CPU to low power state (suspend to RAM).
This is not quite the full story. A few devices may support a wake-up. They will have extra call-backs to enable waking the system from sleep modes. Read the documentation to find out about this.
That is general power management and driver/device structure. Now, how is the ASOC (alsa system on chip) structured?
There are typically three drivers/devices that get stitched together.
Codec - The wm8350 in your case. This includes audio amplifier drive circuitry and can include sound mixing and source controls. Supports digital to analog and analog to digital, typically through an i2s interface. The i2s is not the only interface. Usually a register bank is controlled through a secondary interface; i2c in the wm8350 case.
DAI - Refer to chapter 1.2.18.1 of the iMx31 reference manual; the hardware is called the SSI by Freescale. The next chapter on the AUDMUX is also useful to understand audio support on the iMx31/32.
Machine file - this is the board specific routing. It hooks the DAI to the codec and is the parent of both. It provides board clocking information and other specific configuration. For instance, it may use the AUDMUX to route the physical pins to the SSI block.
An i2c (or SPI) interface from the codec driver to send control commands to the coded chip. Some chips might uses a wacky i2s interface or something else for control (but not in your case).
Now if you understood this, you will see that some features of the wm8350 seem to break the Linux model. The DAI interface can be stopped (digital audio), but the i2c interface must remain alive to program the registers related to the power functionality in the codec/PMIC (power management IC).
The latest WM8350 calls the IC a multi-function device and support was introduced in 2.6.35. The initial support may not have included the WM8350 features. Unfortunately, without some details on the layout of the Sony prs900 board, it would be difficult to know how to use the WM8350 PMIC functionality. The code will involve the iMx31 CPU, the WM8350, the i2c connection, and possibly some power supply circuitry.
For certain, you can just try echo mem > /sys/power/state and see what happens. If it works, you are lucky. The power/current consumption in sleep might not be optimal, but it will probably be hard to fix with the 2.6.23 kernel. You will want to look through the /sys directories for wake-up sources and possibly register these before issuing the suspend to memory command.
I can neither figure out how to read the battery/powerline status information, nor get the audio chip to play sound, etc ... although I have been studying the kernel modules for a while...
From the above discussions, the battery and powerline status will probably be found through another device. However, the pmic_reg file may actually give the status if things are connected properly on the board.
The audio chip will use ALSA. You need to use either alsamixer or the command line amixer to set up audio routes through the codec, so the DAI channel (PCM from iMx32) is routed and sent to the speaker. To minimize power consumption, things are usually turned off by default. The /dev/dsp files are just OSS compatibility. This configuration will support ALSA natively. You are better off to use ALSA if possible.
Donate to the OSF and get a tax receipt, if this was helpful enough.
I need to write a program in linux (debian, to be exact) that disables a USB device if it doesn't pass specific filters. For example, the program might be set to disallow webcams, usb sticks and keyboards, enable mice and printer (through usb). The filters may change in runtime. For example, the program might receive a message to enable usb sticks and it should respond without rebooting the system. The program is written in python but embedding c code (or others) is also acceptable.
What I have tried
I have tried many methods but some of them aren't about programming.
First, I tried to mess with udev. I can monitor the activities when a device is plugged and write filters. There used to be an option "ignore_device" that ignored the filtered devices. For example, to ignore all the devices that are a member of usb subsystem, I would write this as a udev rule:
ACTION=="add", SUBSYSTEM=="usb", OPTIONS+="ignore_device"
But this option is released in this release of udev. What I get so far is that udev can currently be used mainly for monitoring. Sure, I can write additional rules for the rule above that runs a script, but I have to do the disabling elsewhere.
Second, I tried ioctl to send a DISCONNECT signal to device handle. I'm testing this with a USB mouse. This is the python code for that: (I have also tried this in C, nothing changed)
import fcntl
import sys
USBDEVFS_RESET = ord('U') << (4*2) | 20
USBDEVFS_DISCONNECT = ord('U') << (4*2) | 22
raw_name = "/dev/bus/usb/{:03d}/{:03d}"
filename = raw_name.format(1,2)
fd = open(filename, "wb")
fcntl.ioctl(fd, USBDEVFS_DISCONNECT, 0)
Here, if I would send USBDEVFS_RESET, it works, the mouse input is ignored for a second or two. But disconnect signal raises an error:
IOError: [Errno 25] Inappropriate ioctl for device
What I get from here is, I cannot send disconnect signal to a mouse. Maybe a usb stick or printer or some other devices would work, I haven't tried. I want to develop the program as generic as possible so as to prevent writing additional device-specific code, so this approach seems useless for me. And another point here is that when I manually disconnect/connect my mouse, I see events in udev monitor. But when I send reset signal, no event is sent.
The udev monitor says that the mouse was mounted to this path: /sys/bus/usb/devices/1-3 (which is a symlink for /sys/devices/pci0000:00/0000:00:14.0/usb1/1-3). Some documents told me that this folder contains the device's settings and setting /power/level to "off" or "suspended" would turn the device power off. But I cannot manipulate any files in /power. Come to think about it, it might not be a good idea after all.
The Question
So, the question is, what is the best way to achieve such task? I have an idea but I'm not sure whether it will work and even if it does, it might be overkill. My idea is writing a "wrapper driver" that identifies itself to linux kernel as driver for all usb devices. The "wrapper driver" reads the device information and if the device is good to go, it acts as a wrapper for real drivers in the kernel, calling their functions. If not, the "wrapper driver" just ignores the device.
I'm not sure it can even be done, I'm not experienced in kernel or driver programming.
Another way is, -somehow- getting the handle of the device programatically and telling it to power off (or making it busy forever, whatever works). I have also done some little research but couldn't find a proper-easy way. They say all devices are considered as "files" but I cannot reach those files at all.
Note:
The question is mostly about linux internals but it also involves kernel programming. I read a lot about usb manipulation/monitoring programs, I read manual pages of udevadm. But these approaches do not help me at all. I think I need to alter either kernel or device internals programatically.
I have also tried manipulating authorized file that resides in /sys/devices/pci0000:00/0000:00:14.0/usb1/1-4 (for a keyboard). It's default value is 1. Changing it to 0 successfully disabled the bus (NOT device, but the physical usb port. The same device can still be used when plugged in another port). But making it 0 also stopped udev events from this usb port. So, I can disable the port if the user plugs in a forbidden device but I cannot decide when to enable it since I cannot listen to remove events in udev. Would it make sense if I delve deeper to lower levels of code (possibly kernel) and listen to usb events in some other way?
I think the simplest way to solve your problem is balcklisting all usb device drivers excepting mouse, keyboard etc.
The cleanest way is whitlisting mouse etc. with udev using usb device id's
Writing a wrapper driver is an absolute overkill never chose that way.
One possible solution could be tying usb access to user permissions. here is a related link you may find handy
http://robots.mobilerobots.com/wiki/Linux_udev_USB_Device_Permissions_Configuration
Assume the following situation under Linux:
A process is continuously reading from an USB-serial converter device (/dev/ttyUSB0). That device is suddenly unplugged and plugged in again (or is resetting itself for some reason). The process continues to have a valid file handle for /dev/ttyUSB0 but won't receive any data from the device unless the process re-opens the device (because udev has deleted and re-created the device node).
Is there a direct way to detect such a situation (ie. not indirectly by detecting a timeout in data flow) so that the process knows it has to re-open the device? Would it be reliable to monitor the modification time of /dev/ttyUSB0 using stat()?
Additional details:
The process opens the device file by using the standard open() function.
/dev is a tmpfs controlled by udev.
Note: I do not want to use any udev rules for this and prefer a solution implemented directly in the process.
If a USB device is hot-unplugged, operations on the device will begin failing with -EIO; you can detect this and take appropriate action.
If the device node is actually being removed and re-created (which I believe it is if you have udev), then you should be able to use the inode number to tell when this happens.
Just call fstat() on your open file descriptor, stat() on /dev/ttyUSB0, and compare the st_ino fields of the two struct stats.
And let me know if it actually works. :-)
I think FAM or gamin will detect these events.
if I plugin a USB memory stick, I see a new folder in /sys/bus/usb/devices ... thus a new USB-device.
Also I see a new folder in /sys/block ... thus a new block-device.
My question is: How can I get a waterproof mapping between those two devices? Means:
If I get a new device in /sys/bus/usb/devices, how can I programatically (f.i. by checking /sys/...) find out which block device is mapped/related to this usb-device and vice-versa?!
The information in /sys is organized in multiple ways (by driver, by bus, etc.), and there are many symbolic links to go from one hierarchy to another.
Case in point (example seen on kernel 2.6.26): starting from the block device in /sys/block/sdc, the symbolic link /sys/block/sdc/device points inside the per-device-type hierarchy. You can see that it's an USB device because the target of the link is something like
../../devices/pci0000:00/0000:00:1d.7/usb8/8-2/8-2:1.0/host9/target9:0:0/9:0:0:0
Conversely, USB devices are listed in /sys/bus/usb/devices, and we can see that 8-2:1.0 is a disk-like device because /sys/bus/usb/devices/8-2:1.0/driver links to usb-storage. To find out what the associated block device is, it seems we need to go down to the directory /sys/bus/usb/devices/8-2:1.0/host9/target9:0:0/9:0:0:0 which contains a symbolic link block:sdc whose target is /sys/block/sdc.
ADDED: Caution: the exact structure of /sys changes from kernel version to kernel version. For example, with kernel 2.6.32, /sys/block/sdc/device points directly into the /dev/bus/scsi without going through the USB hop.
A different approach is to call the udevadm info command. udevadm info -p /sys/block/sdc --query=… gives information on a device based on its /sys entry, while udevadm info -n sdc --query=… gives information on the device /dev/sdc.
The information includes bus information, for example udevadm info -p /sys/block/sdc --query=env shows
ID_BUS=usb
ID_PATH=pci-0000:00:1d.7-usb-0:2:1.0-scsi-0:0:0:0
The udev documentation may have more information of interest to you.
A final word of caution: there are all kinds of complex cases that may make whatever you do not so waterproof. How will your program deal with a single USB device that is an array of disks that are assigned multiple block devices? Conversely, how will your program deal with a RAID array assembled from multiple devices (perhaps some of them USB and some of them not)? Do you care about other removable media types such as Firewire and e-SATA? etc. You won't be able to predict all corner cases, so make sure to fail gracefully.
As far as I found out, it's possible to access udev information via "libudev" library. There's also a good sample on the net available: http://www.signal11.us/oss/udev/
I was able to modify it to read out all "/dev/sd*" devices and get their Vendor-ID, Product-ID as well as Serial number. I think this solution is kernel/linux distribution independant enough. But I still have to verify this.