What is the simplest way to export data from a bare-bones OS?
I’m developing some assignments for my Computer Architecture course that require students to time different segments of code as accurately as possible. My idea is to insert the code to be timed right into the "Bare Bones"/"Hello World" tutorial from the OSDev wiki (http://wiki.osdev.org/Bare_Bones), which will effectively run the code under test right inside a minimal OS kernel.
This technique works rather well; but, at the moment, my only output option is the VGA text mode. I would like to be able to save the experiment results so the students can analyze/graph the data.
I’m currently installing the “mini-OS” onto a USB flash drive and booting from the flash drive. My original idea was to use BIOS to dump the experiment data back onto the USB drive; but, it looks like calling BIOS routines from protected mode is non-trivial (i.e., requires switching to real or V86 mode).
All I need to do is dump raw binary data somewhere that another machine can read it. I don't need a file system or anything fancy like that. Is there a relatively simple way to access the USB flash drive (or some other external device), or will I need to find/write a complete USB driver or network driver stack? Or, is there a simpler solution?
This post (Real mode BIOS routine and Protected Mode) mentions PwnOS; but the link to the code is broken.
USB drivers are very hard. However, as you mentioned network drivers, you might want to look into serial ports. They are much easier to program than USB.
The osdev wiki should have enough information about this topic.
Related
Environment:
I have an embedded linux system running with an ARM based iMX7 processor. It runs on a build from yocto linux which is very much based on Fedora.
Scenario:
My system uses Suspend To RAM feature which is linux system power saving mode that is explained quite well in this link. This is done to save power at a certain stage.
Objective:
Now, I need to keep the wifi link open during this stage. And as I read from some discussions like this, it seems to be possible to do so.
How can I do this?
Read up on similar discussions:
Reading through this discussion, it explains how to do this on a intel based desktop linux computer. But I don't have the /etc/NetworkManager on my embedded linux device. Probably there is a different way to do it on a Fedora based embdded linux system.
Can I get some suggestions on how to do this or even how to approach this?
None of the articles you quoted even suggest that it is possible to leave WiFi on - in fact one of them says it can't be done. All they provide is various tricks to make the wake-up faster.
Depends on the hardware but very likely, leaving it on is really impossible. Suspend-to-ram includes a hardware command that switches the CPU clock off, places its interconnect buses into idle state, and disables main power to all the peripherals (leaving only standby power to those peripherals that support standby mode).
I don't know if your WiFi device has support for running on standby power nor whether the embedded hardware you have has the ability to provide that power to it while the CPU is off. If that ability exists, it will likely be accessible as a kernel driver parameter.
You may be able to save some startup time when waking up from standby by providing a static configuration for your WiFi device rather than using the default automatic connection (which involves searching for a router to connect to, obtaining an IP address, etc.).
You can't just keep wifi active during sleep/resume. You can optimise reconnection speed but I believe NM and connman both do that already.
Context
Debian 64 bit. kernel 3.18.x
Litterally struggling to understand how a network driver is initialized.
I mean how to choose which flag to set. I dig in the kernel for days now to train myself. The card setup is the only point I miss.
I take the intel 82574 as an example. I downloaded the card's datasheet, saw many information but not a clue on how to setup the hardware.
Question
Where to start to know what flags to set ? The datasheet didn't helped me (i am not very experienced but willing to learn).
Please give me a starting point, a tip or anything to help me understand what is going on in the already written open sourced driver.
How can a developer knows how to initialize his nic ? (yes reinventing the wheel the time to understand)
You'll need to read the source code of the kernel module that handles your specific NIC.
EDIT: Of course, to develop such a module, you'd usually just use a register map as specified in a data sheet or application node; often, manufacturers develop their linux drivers themselves, so the driver developers might even be the same people that developed the chipset (because it's really handy to have a platform to test against -- it's impossible to test hardware without having something like a driver, so you might as well write a proper driver).
Furthermore, devices often come with code examples -- no one is going to build a device based on an IC that he has not seen in action.
If you've got access to neither proper documentation nor source, you can only reverse engineer - and that's an incredibly large field.
Using your example with the Intel 82574 Network Adapter, Intel provides a zip file of the source code used to build the Linux driver. The driver is like all drivers in that it hooks into the OS API for Networking.
The Linux networking API is document on both the linux.org site and discussed on popular Linux sites like lwn.org. Below is the link to lwn's chapter on Network drivers using the networking API called NAPI.
https://static.lwn.net/images/pdf/LDD3/ch17.pdf
You'll notice in the Intel igb driver source code that the NAPI net_device data structure is one of the first things that is setup. It registers the driver with the OS. This way the OS knows which igb functions to call when loading/unloading the driver, or when needing to send/receive data.
The igb functions read/modify/write the necessary bits in the 82574's memory-mapped registers that control and monitor the device. The device registers are all documented in the 82574 datasheet available on Intel's site. And this is usually the case for almost any networking company like Broadcom/Chelsio/Mellanox/Marvell.
Hope that helps a little more.
So, I know Linux kernel is quite "heavy" when considering lower scale embedded systems, but currently but we're a 2 man team trying to see how to create our own embedded system.
I'm the one in charge of all software (the other guy is a HW guy), and thus I would like to re-use existing libraries and frameworks as much as possible, and I would like to bounce off some ideas with gurus around here.
I am fairly comfortable in Linux, but the booting and initialization process is new to me, and I need to dive in to that soon enough. Any book recommendations are welcome as well!
I haven't designed any embedded systems before.. Only own some ARM dev boards (beagleboard and raspberry pi).
Current I have prototype of the software running on beagleboard already, and now we're thinking how to minimize the cost, and to create something our own..
It's a system connected to the internet, and I need to run a tiny web server with some scripting support. Performance wise I don't think it needs to be too powerful.
I would like to minimize all bootloader etc work, since I'm a one man SW team, and just concentrate on the application itself.
Of course I understand that I need to configure our kernel for this, but this is indeed why I thought selecting some SoC would be good, since they usually have some linux and bootloaders ready..
First I thought that Cirrus EP9301 would be perfect, since it seems to be a good package, and not very expensive.. But it seems that it's already in end-of-life, and also support for this is very bad (people on the cirrus forums constantly complain about it).
Are there some good choices for this kind of project, which would enable us "easily" to get linux kernel up and running, with still maintaining some kind of decent BOM (hopefully 20USD or so) ?
Your hardware guy should already know this, but go with an existing reference design. Take the raspberry pi, the beagleboard/bone, open-rd, or any number of other existing systems and clone the part you need. As a result the linux porting will be a matter of removing what you are not using from the reference design instead of adding new stuff and hoping it works. If you go with flat pack parts you can do the work in your garage, if you go with bgas you need the equipment for that or pay someone to do it. (can you tell yet that I hate bgas?).
Is linux a requirement, if not that opens the door to a lot more devices using freertos or chibios or a number of other solutions. the stm32f4 discovery board for example is $20, uses what can barely be called a microcontroller for all the features it has (cortex-m4). Supposedly possible to run uclinux on a cortex-m, but definitely possible to run any number of rtoses and have an ip stack, etc. stellaris (ti.com) has a number of eval boards, one/some with ethernet already (use as a reference design). You can also take the wiznet approach (or a spi ethernet) and use any microcontroller (puts you into the avr/msp430 level and price range). Bang for buck the cortex-m's are good, arm based so comfortable to work with, etc.
Using linux if you are already not an experienced at porting to an embedded platform, and dont want to learn that on this go around, I would definitely go with a clone of an existing design, leverage as much as you can from a project with folks that are experienced at porting linux to a platform. If need be take an existing board (beagle/raspi/openrd) and go through the motions of porting to the platform with the cheat sheet of having access to an existing port, see if you cant get uboot ported and linux booting, etc. (dont really need uboot at all, that is possibly an unnecessary complication, just get dram up and pass the atags, etc to linux and just branch to it, pretty easy to launch linux from bare metal).
You could probably do worse than taking the Broadcom BCM2835 - used on the Raspberry Pi - as your starting point - especially if you want to avoid kernel and boot-loader work and a source of reference schematics. If this proves too expensive, check out other devices in the Broadcom range.
A few bits of advice
You probably want some flash rather than a MMC card interface for production use. eMMC is an option. NAND flash is a nightmare due to rapid component obsolescence and the need to get own and dirty with the MTD drivers.
USB Ethernet will be easier to integrate than a controller hanging off a general purpose bus, but won't perform as well. SmSC seems to be popular source for either
You could also have a look at the work that Olimex is doing with their linux boards. Perhaps even order a som and then combine it with other external components.
I've spent a lot of time developing an operating system and working on my low level boot loader. But now I want to take some time off my operating system while not leaving the low-level environment and doing something involving security.
So I chose to build my own standard password utility following the pre-boot authentication scheme. Since I want the software to be at least a little portable I want it to use as little external support as is possible. I figured that I'd be best if I somehow managed to 'hook' into the bios somewhere between the self checks and the int 19 bootstrap from within a running real mode OS.
However finding information on how to modify the bios code proved to be impossible. I've found nothing on how to achieve the before mentioned. I have only found pages describing how to flash your bios.
Does anyone know how I can read/write bios code? Or can someone provide links to pages that describe this?
I know that it's not only possible to brick my device but it is also likely, I'm aware of the risk and willing to take it.
Pinczakko's articles on BIOS reverse engineering are a great place to start looking at this. There was also a book published by the same author but it is now out of print.
I'm not sure if this approach is the best approach towards a secure boot, but the articles on this site are very detailed and should point you towards a method for modifying your BIOS firmware.
I'm not really sure what you are trying to achieve, but:
The BIOS is completely hardware specific - each manufacturer will have their own mechanism for updating / flashing BIOS and so trying to come up with a portable mechanism for updating a BIOS is destined for failure. For example when using Bochs you "update" the BIOS by specifying a different BIOS ROM image.
If you want to modifty / write your own BIOS then its going to be completely specific to that hardware. Your best bet would be to start with something like Bochs as its open source - as you can take a look at the source code for the BIOS (and easily test / debug it) you stand a reasonable chance of understanding the BIOS code and modifying it into something that works, however I suspect this isn't what you are trying to do.
Why not just perform this authentication as your OS boots? If you want to protect the data then you should encrypt it and require that the user supply log in / supply the decryption key on startup.
If you were thinking of working with "legacy" PC BIOS, I would dissuade you from trying for many of the reasons Justin mentioned: 1) legacy BIOS is PC vendor-specific; 2) it is closed source and proprietary; 3) there are no industry standards defining legacy BIOS interfaces for extending the system as you are trying to do.
On the other hand, if you have access to a UEFI-based BIOS PC, you may be able to write your own PEI/DXE driver(s) to implement such a feature. This will at least point you in the right direction:
http://sourceforge.net/apps/mediawiki/tianocore/index.php?title=Welcome
Intel Press book on the topic: Beyond BIOS
Regarding the practicality of read/writing the BIOS, you'll need to identify the SPI part containg the BIOS and get a ROM burner. The SPI part may or may not be socketed; if it is not socketed, you'll need a soldering iron and be able to create a socket/header for the part. You obviously do not want to embark on this project with your primary computer system. Perhaps you could find an older system or a reference board.
I am looking at some pointers for understanding how the Linux kernel implements the setting up of various hardware clocks. This basically relates to working with setting up the various clocks that hardware features like the LCD, UART etc will use. For example when Linux boots how does it handle setting up the clocks for UART or USB. Maybe something like a Clock manager or something.
I am basically trying to implement something similar for a different OS on a new hardware that i am working on. Any help would be really appreciated.
[Edit]
Thanks for the replies and the links. So here is what i have implemented up until now. This should give you an idea of where I'm headed.
I looked up the Hardware Reference Manual for the particular system I'm targeting and wrote some code to monitor/modify the signals/pins of the peripherals I am interested in i.e. turning them ON/OFF from the command line.Now a collection of these clocks/signals together control a peripheral.The HRM would say that if you want to turn on the UART or something then turn on such and such signals/pins. And #BjoernD yes I am using something like a mmap() function to talk to the peripherals.
The meat of my question is that I want to understand the design and implementation of a Clock/Peripheral Manager which uses the utility that I have already written. This Clock/Peripheral Manager would give me the control of enabling/disabling the peripherals I want.Basically this Manager would enable me to make changes in the init code that is right now running. Also during run time processes can call this Manager to turn ON/OFF the devices so that power consumption is optimized. It might not have made perfect sense but I'm myself trying to wrap my head around this.
Now I'm sure something like this would have been implemented in Linux or for that matter any OS for performance issues (nobody would want to waste power by turning on all peripherals at boot time). I want to understand the Software Architecture of it. Reference from any OS would do as of now to atleast get a headstart. Also I am not writing my own OS, there is an OS in place but Im looking more at a board level software aka BSP to work on. But thanks for the OS link anyways, they are really good. Appreciate it.
Thanks!
What you want to achieve is highly specific to a) the platform you are using and b) the device you want to use. For instance, on x86 there are 3 ways to communicate with a device:
Interrupts allow the device to signal the CPU. The OS usually provides mechanisms to register interrupt handlers - functions that are called upon occurrence of an interrupt. In Linux see request_irq() and friends in linux/include/interrupt.h
Memory-mapped I/O is physical memory of the device that the platform's BIOS makes available in the same way you also access plain physical memory - simply by writing to a memory address. What exactly is behind such memory (e.g., network interface config registers or an LCD frame buffer) depends on the device and is usually specified in the device's data sheet.
I/O ports are accessed through a special address space and special instructions (INB/OUTB & co.). Other than that they work similar to I/O memory.
There's a multitude of ways to find out what resources a device provies and where the BIOS mapped them. Some platforms use ACPI tables (google yourself for the 1,000k page spec), PCI provides info on devices in a standardized way through the PCI config space, USB has similar ways of discovering devices attached to the bus, and some devices, e.g., UARTS, are simply specified to be available at a pre-configured I/O range that is fixed for your platform.
As a start for understanding Linux, I'd recommend "Understanding the Linux kernel". For specifics on how Linux handles devices and what is there to write drivers, have a look at Linux Device Drivers. Furthermore, you will need to have a look at the peculiarities of your platform and the device you want to drive.
If you want to start an own OS, a UART is certainly something that will be veeery helpful to print debug output, so you might want to go for this first.
Now that I wrote down all this, it seems that your actual question is: How to get started with Operating System design. This question should be highly valuable for you: What are some resources for getting started in operating system development?
The two big power users in most computers are the CPU and the disks. Both of these have capabilities for power saving in Linux. The CPU clock can be slowed down when the system is not busy, and the disk motors can be stopped when no I/O is happening. For a UART, even if you save all of the power that it uses by turning off its clock, it is still tiny compared to the others because a UART doesn't have much logic in it.
Best ways to save power are
1) more efficient power supply
2) replace rotating disk with SSD
3) Slow down the CPU and memory bus