I am looking at some pointers for understanding how the Linux kernel implements the setting up of various hardware clocks. This basically relates to working with setting up the various clocks that hardware features like the LCD, UART etc will use. For example when Linux boots how does it handle setting up the clocks for UART or USB. Maybe something like a Clock manager or something.
I am basically trying to implement something similar for a different OS on a new hardware that i am working on. Any help would be really appreciated.
[Edit]
Thanks for the replies and the links. So here is what i have implemented up until now. This should give you an idea of where I'm headed.
I looked up the Hardware Reference Manual for the particular system I'm targeting and wrote some code to monitor/modify the signals/pins of the peripherals I am interested in i.e. turning them ON/OFF from the command line.Now a collection of these clocks/signals together control a peripheral.The HRM would say that if you want to turn on the UART or something then turn on such and such signals/pins. And #BjoernD yes I am using something like a mmap() function to talk to the peripherals.
The meat of my question is that I want to understand the design and implementation of a Clock/Peripheral Manager which uses the utility that I have already written. This Clock/Peripheral Manager would give me the control of enabling/disabling the peripherals I want.Basically this Manager would enable me to make changes in the init code that is right now running. Also during run time processes can call this Manager to turn ON/OFF the devices so that power consumption is optimized. It might not have made perfect sense but I'm myself trying to wrap my head around this.
Now I'm sure something like this would have been implemented in Linux or for that matter any OS for performance issues (nobody would want to waste power by turning on all peripherals at boot time). I want to understand the Software Architecture of it. Reference from any OS would do as of now to atleast get a headstart. Also I am not writing my own OS, there is an OS in place but Im looking more at a board level software aka BSP to work on. But thanks for the OS link anyways, they are really good. Appreciate it.
Thanks!
What you want to achieve is highly specific to a) the platform you are using and b) the device you want to use. For instance, on x86 there are 3 ways to communicate with a device:
Interrupts allow the device to signal the CPU. The OS usually provides mechanisms to register interrupt handlers - functions that are called upon occurrence of an interrupt. In Linux see request_irq() and friends in linux/include/interrupt.h
Memory-mapped I/O is physical memory of the device that the platform's BIOS makes available in the same way you also access plain physical memory - simply by writing to a memory address. What exactly is behind such memory (e.g., network interface config registers or an LCD frame buffer) depends on the device and is usually specified in the device's data sheet.
I/O ports are accessed through a special address space and special instructions (INB/OUTB & co.). Other than that they work similar to I/O memory.
There's a multitude of ways to find out what resources a device provies and where the BIOS mapped them. Some platforms use ACPI tables (google yourself for the 1,000k page spec), PCI provides info on devices in a standardized way through the PCI config space, USB has similar ways of discovering devices attached to the bus, and some devices, e.g., UARTS, are simply specified to be available at a pre-configured I/O range that is fixed for your platform.
As a start for understanding Linux, I'd recommend "Understanding the Linux kernel". For specifics on how Linux handles devices and what is there to write drivers, have a look at Linux Device Drivers. Furthermore, you will need to have a look at the peculiarities of your platform and the device you want to drive.
If you want to start an own OS, a UART is certainly something that will be veeery helpful to print debug output, so you might want to go for this first.
Now that I wrote down all this, it seems that your actual question is: How to get started with Operating System design. This question should be highly valuable for you: What are some resources for getting started in operating system development?
The two big power users in most computers are the CPU and the disks. Both of these have capabilities for power saving in Linux. The CPU clock can be slowed down when the system is not busy, and the disk motors can be stopped when no I/O is happening. For a UART, even if you save all of the power that it uses by turning off its clock, it is still tiny compared to the others because a UART doesn't have much logic in it.
Best ways to save power are
1) more efficient power supply
2) replace rotating disk with SSD
3) Slow down the CPU and memory bus
Related
Environment:
I have an embedded linux system running with an ARM based iMX7 processor. It runs on a build from yocto linux which is very much based on Fedora.
Scenario:
My system uses Suspend To RAM feature which is linux system power saving mode that is explained quite well in this link. This is done to save power at a certain stage.
Objective:
Now, I need to keep the wifi link open during this stage. And as I read from some discussions like this, it seems to be possible to do so.
How can I do this?
Read up on similar discussions:
Reading through this discussion, it explains how to do this on a intel based desktop linux computer. But I don't have the /etc/NetworkManager on my embedded linux device. Probably there is a different way to do it on a Fedora based embdded linux system.
Can I get some suggestions on how to do this or even how to approach this?
None of the articles you quoted even suggest that it is possible to leave WiFi on - in fact one of them says it can't be done. All they provide is various tricks to make the wake-up faster.
Depends on the hardware but very likely, leaving it on is really impossible. Suspend-to-ram includes a hardware command that switches the CPU clock off, places its interconnect buses into idle state, and disables main power to all the peripherals (leaving only standby power to those peripherals that support standby mode).
I don't know if your WiFi device has support for running on standby power nor whether the embedded hardware you have has the ability to provide that power to it while the CPU is off. If that ability exists, it will likely be accessible as a kernel driver parameter.
You may be able to save some startup time when waking up from standby by providing a static configuration for your WiFi device rather than using the default automatic connection (which involves searching for a router to connect to, obtaining an IP address, etc.).
You can't just keep wifi active during sleep/resume. You can optimise reconnection speed but I believe NM and connman both do that already.
Context
Debian 64 bit. kernel 3.18.x
Litterally struggling to understand how a network driver is initialized.
I mean how to choose which flag to set. I dig in the kernel for days now to train myself. The card setup is the only point I miss.
I take the intel 82574 as an example. I downloaded the card's datasheet, saw many information but not a clue on how to setup the hardware.
Question
Where to start to know what flags to set ? The datasheet didn't helped me (i am not very experienced but willing to learn).
Please give me a starting point, a tip or anything to help me understand what is going on in the already written open sourced driver.
How can a developer knows how to initialize his nic ? (yes reinventing the wheel the time to understand)
You'll need to read the source code of the kernel module that handles your specific NIC.
EDIT: Of course, to develop such a module, you'd usually just use a register map as specified in a data sheet or application node; often, manufacturers develop their linux drivers themselves, so the driver developers might even be the same people that developed the chipset (because it's really handy to have a platform to test against -- it's impossible to test hardware without having something like a driver, so you might as well write a proper driver).
Furthermore, devices often come with code examples -- no one is going to build a device based on an IC that he has not seen in action.
If you've got access to neither proper documentation nor source, you can only reverse engineer - and that's an incredibly large field.
Using your example with the Intel 82574 Network Adapter, Intel provides a zip file of the source code used to build the Linux driver. The driver is like all drivers in that it hooks into the OS API for Networking.
The Linux networking API is document on both the linux.org site and discussed on popular Linux sites like lwn.org. Below is the link to lwn's chapter on Network drivers using the networking API called NAPI.
https://static.lwn.net/images/pdf/LDD3/ch17.pdf
You'll notice in the Intel igb driver source code that the NAPI net_device data structure is one of the first things that is setup. It registers the driver with the OS. This way the OS knows which igb functions to call when loading/unloading the driver, or when needing to send/receive data.
The igb functions read/modify/write the necessary bits in the 82574's memory-mapped registers that control and monitor the device. The device registers are all documented in the 82574 datasheet available on Intel's site. And this is usually the case for almost any networking company like Broadcom/Chelsio/Mellanox/Marvell.
Hope that helps a little more.
hi I'm really interested in Linux kernel development but I'm having trouble understanding a few big concepts.
First, is a device driver programmer and embedded programmer two completely differnet jobs??
I mean I know embedded programmers get in to firmwares and circuits and stuff like that
but do device driver programmers also do the same thing?
Second, I heard that it's good to start linux kernel development by writing device drivers.
Does this mean you have to master device drivers?
Frankly, i want to know exactly what each of these programmers do
what I really want is to understand ARM and x86 based linux kernel and get in to development
and I'm just curious do I really have to know all the circuits and stuff like that.
P.S, is a system programmer also a differnet job??
Some concepts within today's Linux kernel are really complex: scheduling, memory management (MM), locking, stuff specific to each architecture, security, etc.
However, it is generally true that device drivers are somewhat simple (that is, compared to the rest), because their job is usually to act as bridges between userspace interfaces and the actual drived device. Consequently, they seldom play with the internal mechanics of the kernel, except for the drivers API, of course. Also, the kernel community is much more inclined to accept device drivers contributions since they affect only specific use cases (whereas contributions to the core, like MM or security, affect everyone).
Now, before trying anything on the kernel side, make sure to understand the userspace properly. Because, as mentioned above, device drivers register functions that get called when a user calls specific system calls (syscalls) on the special file representing the device. You must then understand very well those system calls.
Before actually writing code, go read a few books mentioned here. They are technical books, but you will need to understand the subject properly.
Also, go read actual code. That's probably the best way to learn: looking at what others did (which is, after all, the very essence of free software). You can start by looking at simple drivers, like the ones in drivers/leds.
About job titles: an "embedded programmer" (or embedded computer engineer) is of course a good candidate for writing device drivers. Embedded computer engineers may also write firmwares, microcontroller programs and actual userspace applications that act close to hardware components (device control, serial protocols, etc.).
You don't need to master circuits to write device driver code, but you certainly need to understand core hardware concepts like interrupts, memory-mapped I/O, timing, buses, locking, power management and possibly some assembly language.
Linux kernel 2.6
I've got an fpga that is loaded over GPIO connected to a development board running linux.
The fpga will transmit and receive data over the pci-express bus. However, this is enumerated
at boot and as such, no link is discovered (because the fpga is not loaded at boot).
How can I force re-enumeration of the pci-e bus in linux?
Is there a simple command or will I have to make kernel changes?
I need the capability to hotplug pcie devices.
As root, try the following command:
echo "1" > /sys/bus/pci/rescan
See this link for more information: http://www.kernel.org/doc/Documentation/ABI/testing/sysfs-bus-pci
I wonder what platform you are on: A work around (aka hack) for this that works on x86 systems is to have the BIOS basically statically configure a PCI device at whatever bus, device, function the FPGA normally lands on, then the OS will enumerate the device and reserve the PCI space for it (even though the device isn't really there). Then in your device driver you will have to do some extra things like setup the BARs and int lines manually after the fpga has been programmed. Of course this requires modifying the BIOS, which if you are working with a BIOS vendor you can contract them to make this change for you, if you are not working with a BIOS vendor then it will be much harder... Also keep in mind that I was working on VxWorks on x86, and we had a AMI make a custom BIOS for our boards...
If you don't have a BIOS, then consider programming it in the bootloader, there you already have the ability to read from disk, and adding GPIO capabilities probably isn't too difficult (assuming you are using jtag and GPIOs?), in fact depending on what bootloader you use it might already be able to do GPIO?
The issues with modifying the kernel to do this is that you have to find the sweet spot where you can read the bitfile, before the PCI enumeration... If for example the disk device drivers are initialized after PCI, then obviously you must do some radical changes to the kernel just to read the bitfile prior to PCI enumeration, which might cause other annoying problems...
One other option which you may have already discovered, and which is really only ok for development time: Power up the system, program the fpga board, then do a reset (without power cycle, for example: sudo reboot now), the FPGA should keep its configuration, and linux should enumerate it...
After turning on your computer, the BIOS enumerates the PCI bus and attempts to fulfill all IO space and memory mapped IO (MMIO) requests. It sets up these BAR's initially, and when the operating system loads these BAR's can be changed by the OS as it sees fit while the PCI bus driver enumerates the bus yet again. It is even possible for the superuser of the system to run the command setpci to change these BAR's after the BIOS has already attempted to configure them and the OS has loaded (may cause drivers to fail and several other bad things if done improperly).
I have had to do this in cases where the card in question was not assigned any resources by the BIOS since the region requested required a 64-bit address and the BIOS only operated with 32-bit address assignments. I was able to go in after-the-fact and change these addresses (originally assigned by the BIOS) to whatever addresses I saw fit, insert the kernel module, and my driver would map and use these newly-assigned addresses for the card without knowing the difference.
The problem that exists with hotplugging PCI-Express cards is that the power to the slot, itself, cannot be turned on/off without specific hotplug controllers that need to exist on the motherboard/backplane. Not having these hotplug controllers to turn the slot's power off may lead to shorts between the tiny pins when the card is physically inserted and/or removed if power is still present. Hotplug events, however, can be initiated by either end (the host or the endpoint device). This does not seem to be the case, however if your FPGA already has a link established with the root complex, a possible solution to your problem would be to generate hotplug interrupts to cause a bus rescan in the OS.
There is a major problem, though -- if your card does not actually obtain a link to the root complex, it won't be able to generate any hotplug events; which seems to be the case. After booting, the FPGA should toggle the PRESENT line on the PCIe bus to tell the OS there is a card ready to be enumerated. Once detected, the OS should attempt to establish a link to the card and assign memory regions to the device. After the OS enumerates the card you'll be able to load drivers against it and see it in lspci. You stated you're using kernel 2.6, which does have support for hotplugging and dynamic resource allocation so this method should work as long as your FPGA supports the ability to toggle the PRESENT PCIe line, too.
I plan to develop a nice little application that will run on an arm-based embedded Linux platform; however, since that platform will be battery-powered, I'm searching for relevant information on how to handle power save.
It is kind of important to get decent battery time.
I think the Linux kernel implemented some support for this, but I can't find any documentation on this subject.
Any input on how to design my program and the system is welcome.
Any input on how the Linux kernel tries to solves this type of problem is also welcome.
Other questions:
How much does the program in user space need to do?
And do you need to modify the kernel?
What kernel system calls or APIs are good to know about?
Update:
It seems like the folks involved with the "Free Electrons" site have produced some nice presentations on this subject.
http://free-electrons.com/services/power-management/
http://free-electrons.com/docs/power
http://free-electrons.com/docs/optimizations
But maybe someone else has even more information on this subject?
Update:
It seems like Adam Shiemke's idea to go look at the MeeGo project may be the best tip so far.
It may be the best battery powered Embedded Linux project out there at this moment.
And Nokia is usually kind of good at this type of thing.
Update:
One has to be careful about Android since it has a "modified" Linux kernel in the bottom, and some of the things the folks at Google have done do not use baseline/normal Linux kernels. I think that some of their power management ideas could be troublesome to reuse for other projects.
I haven't actually done this, but I have experience with the two apart (Linux and embedded power management). There are two main Linux distributions that come to mind when thinking about power management, Android and MeeGo. MeeGo uses (as far as I can tell) an unmodified 2.6 kernel with some extras hanging on. I wasn't able to find a lot on exactly what their power management strategy is, although I suspect more will be coming out about it in the near future as the product approaches maturity.
There is much more information available on Android, however. They run a fairly heavily modified 2.6 kernel. You can see a good bit on the different strategies implemented in http://elinux.org/Android_Power_Management (as well as kernel drama). Some other links:
https://groups.google.com/group/android-kernel/browse_thread/thread/ee356c298276ad00/472613d15af746ea?lnk=raot&pli=1
http://www.ok-labs.com/blog/entry/context-switching-in-context/
I'm sure that you can find more links of this nature. Since both projects are open source, you can grab the kernel code, and probably get further information from people who actually know what they are talking about in forms and groups.
At the driver level, you need to make sure that your drivers can properly handle suspend and shut devices off that are not in use. Most devices aimed at the mobile market offer very fine-grained support to turn individual components off, and to tweak clock settings (remember, power is proportional to clock^2).
Hope this helps.
You can do quite a bit of power-saving without requiring any special support from the OS, assuming you are writing (or at least have the source code for) your application and drivers.
Your drivers need to be able to disable their associated devices and bring them back up without requiring a restart or introducing system instability. If your devices are connected to a PCI/PCIe bus, research which power states they support (D0 - D3) and what your driver needs to do to transition between these low-power modes. If you are selecting hardware devices to use, look for devices that adhere to the PCI Power Management Specification or have similar functionality (such as a sleep mode and a "wake up" interrupt signal).
When your device boots up, every device that has the ability to detect whether it is connected to anything needs to do so. If any ports or buses detect that they are not being used, power them down or put them to sleep. A port running at full power but sitting unused can waste more power than you might think it would. Depending on your particular hardware and use case, it might also be useful to have a background app that monitors device usage, identifies unused/idle resources, and acts appropriately (like a "screen saver" for your hardware).
Your application software should make sure to detect whether hardware devices are powered up before attempting to use them. If you need to access a device that might be placed in a low-power mode, your application needs to be able to handle a potentially lengthy delay in waiting for the device to wake up and respond. Your applications should also be considerate of a device's need to sleep. If you need to send a series of commands to a hardware device, try to buffer them up and send them out all at once instead of spacing them out and requiring multiple wakeup->send->sleep cycles.
Don't be afraid to under-clock your system components slightly. Besides saving power, this can help them run cooler (which requires less power for cooling). I have seen some designs that use a CPU that is more powerful than necessary by a decent margin, which is then under-clocked by as much as 40% (bringing the performance down to the original level but at a fraction of the power cost). Also, don't be afraid to spend power to save power. That is, don't be afraid to use CPU time monitoring hardware devices for opportunities to disable/hibernate them (even if it will cause your CPU to use a bit more power). Most of the time, this tradeoff results in a net power savings.
One of the most important things to think of as a power aware application developer is to avoid unnecessary timers. If possible use interrupt driven solutions instead of polled solutions. If a timer must be used then use as long poll interval as is possible.
For example if something special should be done at a certain room temperature it is unnecessary to check the temperature every 100 ms since temperature in a room changes slowly. A more reasonable polling interval is could be 60 s.
This affects the power consumption in several ways. In Linux the CPUIDLE subsystem takes the CPU (SOC) to as deep power saving state as possible depending on when it predicts the next wakeup to occur. Having a lot of timers in a system will fragment the sleep making it impossible to go to the deeper sleep states for longer periods. A typical deep sleep state for CPUIDLE turns the CPU off but keeps the RAM in self refresh. When a timer triggers the CPU will boot and serve the timer of the application.
It's not actually your topic, but it might come in handy to log your progress: i was looking for testing / measuring my embedded linux system. chris desjardins from this forum recommended me this:
I have successfully used bootchart in the past:
http://elinux.org/Bootchart
Here is a list of other things that may also help:
http://elinux.org/Boot_Time