Au0586 half title page 11/17/05 2:05 PM Page 1
E
MBEDDED
L
INUX
S
YSTEM
D
ESIGN AND
D
EVELOPMENT

TEAM FLY
Au0586 title page 11/17/05 2:04 PM Page 1
Boca Raton New York
E
MBEDDED
L
INUX
S
YSTEM
D
ESIGN AND
D
EVELOPMENT
P. Raghavan • Amol Lad • Sriram Neelakandan
Published in 2006 by
Auerbach Publications
Taylor & Francis Group
6000 Broken Sound Parkway NW, Suite 300

QA76.76.O63R335 2005
005.4'32--dc22 2005048179
Visit the Taylor & Francis Web site at

and the Auerbach Publications Web site at

Taylor & Francis Group
is the Academic Division of T&F Informa plc.

All source code in the book is released under GNU GPL v2. It can be used as desired under terms and
conditions of GNU GPL v2.

Trademarks

Ⅲ

MIPS is a registered trademark and YAMON is a trademark of MIPS Technologies.

Ⅲ

IBM and ClearCase are registered trademarks and PowerPC is a trademark of International Business
Machines Corporation.

Ⅲ

UNIX is a registered trademark in the United States and other countries, licensed exclusively through
X/Open Company Limited.

Ⅲ


Ⅲ

GoAhead is a registered trademark of GoAhead Software, Inc

Ⅲ

RTLinux is a registered trademark and FSMLabs, RTLinuxPro and RTCore are trademarks of Finite State
Machine Labs, Inc

Ⅲ

Debian is a registered trademark of Software in the Public Interest, Inc

Ⅲ

LMBench is a trademark of BitMover, Inc

Ⅲ

VRTX is a trademark of Microtech Research Inc

Ⅲ

VxWorks and pSOS are registered trademarks of Wind River Systems, Inc

Ⅲ

Trolltech is registered trademark and Qt is a trademark of Trolltech in Norway, the United States and
other countries


QNX and Neutrino are registered trademarks of QNX Software Systems Ltd

Ⅲ

Nucleus is a trademark of Accelerated Technology, Inc

Ⅲ

Accelerated Technology is a registered trademark of Mentor Graphics Corporation

Ⅲ

ARM and StrongARM are registered trademarks and ARM7 and ARM9 are trademarks of Advanced RISC
Machines, Ltd.

Ⅲ

AMD is a registered trademark of Advanced Micro Devices, Inc.

Ⅲ

Intel and Pentium are registered trademarks and i386 and XScale are trademarks of Intel Corporation.

Ⅲ

Sharp is a registered trademark of Sharp Electronics Corp.

Ⅲ

SPARC is a registered trademark of SPARC International, Inc., and is used under license by Sun


®

, TimeStorm®, TimeStorm IDE™, TimeStorm LVS™, TimeStorm LDS™, TimeStorm LHD™,
TimeSys Reservations™, TimeTrace

®

, Linux/RT

TM

and TimeWiz

®

are registered or unregistered trademarks
of TimeSys Corporation in the United States and other countries.

Ⅲ

NeoMagic is registered trademark of NeoMagic Corporation.

Ⅲ

Transmeta is a trademark of Transmeta Corporation.

Ⅲ

Broadcom is a registered trademark of Broadcom Corporation and/or its subsidiaries.

Ⅲ

Philips® is a registered trademark of Philips Consumer Electronics Corporation.

Ⅲ

Cadenux is a trademark of Cadenux, LLC.

Ⅲ

ELinOS is a registered trademark of SYSGO AG.

Ⅲ

Metrowerks and CodeWarrior are trademarks of Metrowerks Corp. in the U.S. or other countries.

Ⅲ

FreeBSD is a registered trademark of the FreeBSD Foundation.

Ⅲ

IEEE and POSIX are registered trademarks of Institute of Electrical and Electronics Engineers, Inc. in
the United States.

Ⅲ

Xtensa is a trademark belonging to Tensilica Inc.

Ⅲ


In memory of my late father

Amol

To Lord Krsna,

...

my parents, my wife Parul, and my brother Amit

Sriram

To my family and all Linux enthusiasts

ix
Contents
1
Intr oduction ............................................................................................. 1
1.1 History of Embedded Linux .................................................... 2
1.1.1 Year 1999 ..................................................................... 3
1.1.2 Year 2000 ..................................................................... 4
1.1.3 Year 2001 ..................................................................... 4
1.1.4 Year 2002 ..................................................................... 5
1.1.5 Year 2003 ..................................................................... 6
1.1.6 Year 2004 ..................................................................... 6
1.2 Why Embedded Linux? ............................................................ 7
1.2.1 Vendor Independence................................................. 7
1.2.2 Time to Market............................................................ 8
1.2.3 Varied Hardware Support ........................................... 8

2.1.3 Microkernel ................................................................ 31
2.2 Linux Kernel Architecture...................................................... 32
2.2.1 Hardware Abstraction Layer (HAL).......................... 33
2.2.2 Memory Manager....................................................... 33
2.2.3 Scheduler.................................................................... 34
2.2.4 File System................................................................. 35
2.2.5 IO Subsystem............................................................. 36
2.2.6 Networking Subsystems ............................................ 36
2.2.7 IPC .............................................................................. 36
2.3 User Space .............................................................................. 36
2.4 Linux Start-Up Sequence........................................................ 41
2.4.1 Boot Loader Phase .................................................... 42
2.4.2 Kernel Start-Up .......................................................... 43
2.4.3 User Space Initialization ........................................... 47
2.5 GNU Cross-Platform Toolchain ............................................. 48
2.5.1 Building Toolchain .................................................... 50
2.5.2 Building Toolchain for MIPS .................................... 55
3
Boar d Support Package ......................................................................... 59
3.1 Inserting BSP in Kernel Build Procedure............................. 60
3.2 The Boot Loader Interface..................................................... 62
3.3 Memory Map........................................................................... 66
3.3.1 The Processor Memory Map — MIPS Memory
Model.......................................................................... 67
3.3.2 Board Memory Map .................................................. 68
3.3.3 Software Memory Map.............................................. 68
3.4 Interrupt Management............................................................ 72
3.5 The PCI Subsystem................................................................. 77
3.5.1 Uniqueness of PCI Architecture............................... 77
3.5.2 PCI Software Architecture......................................... 79

4.5.2 Filling up
mtd_info
for NAND Flash Chip ......... 108
4.5.3 Registering
mtd_info
.............................................. 109
4.5.4 Sample Mapping Driver for NOR Flash ................ 111
4.6 MTD Block and Character Devices..................................... 114
4.7 Mtdutils Package................................................................... 116
4.8 Embedded File Systems ....................................................... 116
4.8.1 Ramdisk.................................................................... 117
4.8.2 RAMFS ...................................................................... 117
4.8.3 CRAMFS (Compressed RAM File System) ............. 117
4.8.4 Journaling Flash File Systems — JFFS and
JFFS2......................................................................... 117
4.8.5 NFS — Network File System.................................. 119
4.8.6 PROC File System.................................................... 119
4.9 Optimizing Storage Space.................................................... 120
4.9.1 Kernel Space Optimization..................................... 120
4.9.2 Application Space Optimization............................. 121
4.9.3 Applications for Embedded Linux ......................... 122
4.10 Tuning Kernel Memory........................................................ 124
xii Embedded Linux System Design and Development
5
Embedded Drivers ............................................................................... 127
5.1 Linux Serial Driver................................................................ 128
5.1.1 Driver Initialization and Start-Up ........................... 130
5.1.2 Data Transmission ................................................... 134
5.1.3 Data Reception ........................................................ 134
5.1.4 Interrupt Handler..................................................... 134

6.4.2 RTOS Task APIs Emulation .................................... 185
6.4.3 IPC and Timer APIs Emulation .............................. 191
6.5 Kernel API Driver................................................................. 191
6.5.1 Writing User-Space Stubs........................................ 194
6.5.2 Kapi Driver Implementation................................... 195
6.5.3 Using the Kapi Driver............................................. 199
Note................................................................................................. 200
Contents xiii
7
Real-Time Linux ................................................................................... 201
7.1 Real-Time Operating System ............................................... 202
7.2 Linux and Real-Time ............................................................ 202
7.2.1 Interrupt Latency ..................................................... 203
7.2.2 ISR Duration ............................................................ 204
7.2.3 Scheduler Latency.................................................... 205
7.2.4 Scheduler Duration.................................................. 207
7.2.5 User-Space Real-Time ............................................. 209
7.3 Real-Time Programming in Linux ....................................... 209
7.3.1 Process Scheduling.................................................. 210
7.3.2 Memory Locking...................................................... 213
7.3.3 POSIX Shared Memory ........................................... 223
7.3.4 POSIX Message Queues.......................................... 225
7.3.5 POSIX Semaphores.................................................. 232
7.3.6 Real-Time Signals .................................................... 233
7.3.7 POSIX.1b Clock and Timers................................... 241
7.3.8 Asynchronous I/O ................................................... 246
7.4 Hard Real-Time Linux .......................................................... 252
7.4.1 Real-Time Application Interface (RTAI)................. 253
7.4.2 ADEOS...................................................................... 258
8

9.3 Introduction to Display Hardware ...................................... 313
9.3.1 Display System ........................................................ 313
9.3.2 Input Interface ......................................................... 316
9.4 Embedded Linux Graphics .................................................. 316
9.5 Embedded Linux Graphics Driver ...................................... 316
9.5.1 Linux Frame Buffer Interface ................................. 317
9.5.2 Frame Buffer Internals ............................................ 326
9.6 Windowing Environments, Toolkits, and Applications ..... 328
9.6.1 Nano-X ..................................................................... 335
9.7 Conclusion............................................................................. 340
Notes ............................................................................................... 340
10
uClinux ................................................................................................. 341
10.1 Linux on MMU-Less Systems............................................... 341
10.1.1 Linux Versus uClinux .............................................. 342
10.2 Program Load and Execution.............................................. 343
10.2.1 Fully Relocatable Binaries (FRB)............................ 345
10.2.2 Position Independent Code (PIC).......................... 345
10.2.3 bFLT File Format ..................................................... 346
10.2.4 Loading a bFLT File ................................................ 347
10.3 Memory Management........................................................... 358
10.3.1 Heap ......................................................................... 358
10.3.2 Stack ......................................................................... 363
10.4 File / Memory Mapping—The Intricacies of mmap( )
in uClinux ............................................................................. 364
10.5 Process Creation ................................................................... 365
10.6 Shared Libraries .................................................................... 367
10.6.1 uClinux Shared Library Implementation
(libN.so).................................................................... 367
10.7 Porting Applications to uClinux.......................................... 370

The industrial revolution appears as a knife-edge change from a rural self-
employed lifestyle to a clock-punching, whistle-blowing corporate urban way
of life. Being in the middle of the current revolution makes it hard to realize
that in fifty years most people will consider the messy, dynamic, no-rules
embedded product development environment of today as an obvious clean
transition caused by technological changes.
The first embedded software project I worked on didn’t use an off-the-
shelf operating system—there was none. It wasn’t until several years later that
WindRiver introduced VxWorks
®
. In the mid-1990s it appeared that nothing
could unseat VxWorks; yet, recently WindRiver announced a Linux-based
product. Why the change? Today the most common embedded operating
system used in new products is Linux.
For fourteen years I was part of a small army of firmware engineers working
on the development of HP LaserJet™ printers. The printer used a homegrown
operating system that as I recall was called LaserJet O.S. Usually the very best
engineers worked on supporting and extending the operating system. Any
LaserJet O.S. documentation that existed, engineers had created. Any test suite
was similarly a burden placed on the engineer’s shoulders. The effort and
expense of these highly talented engineers seldom led to any features that
differentiated the product from the competitors. The most important lesson I
learned from the experience was to always put your most talented engineers
on the features that make your product unique and outsource the infrastruc-
ture. Embedded Linux is often the best choice for the operating system
infrastructure for products needing nontrivial connectivity.
Whether you support Linux in-house or purchase a Linux board support
package for your processor, you will still need to understand the overall system
and at times the details of a particular subsystem. In this book the authors
have done a good job fitting all the pieces together that are necessary for

Finally, the best reason for reading this book on embedded Linux is because
the technology is so fascinating. Anyone who had developed embedded
products the old way, with one single executable image, will be amazed at
the flexibility and power of using embedded Linux. Anyone new to embedded
development will find most of the power and flexibility available on their
desktop PC works the same in their embedded development environment.
Todd Fischer
President and Founder
Cadenux
xix
Preface
When we were in college in the mid-1990s we heard of an exciting new
technology called the Internet that was to have a profound impact on our
lives. Along with the Internet we also heard of an open source operating
system, Linux, which was being developed by hundreds of programmers
around the world. Linux gave us an opportunity to understand the internals
of the operating system and we quickly became Linux enthusiasts. We realized
that Linux was more than an operating system; here was a movement with
few parallels in human history as it was based on the concepts of human
dignity, choice, and freedom. Linux gave young programmers like us the reach
to the latest technology.
When we became embedded professionals Linux had yet to make a strong
presence in the embedded market. However, we were hearing of some exciting
improvements such as running a hard real-time kernel along with the Linux
kernel and running Linux on MMU-less microcontrollers. Our happiness grew
unbounded when we were asked by a customer to move our software on a
MIPS-based SoC from a commercial RTOS to embedded Linux. Our experience
revealed that the road to embedded Linux is not a very smooth ride. Some
of the main reasons were:
1. There is undoubtedly lots of information about embedded Linux on the

Benefits to the Reader
The book offers solutions to problems that a developer faces when program-
ming in an embedded Linux environment. Some of the common problems are:
Ⅲ Understand the embedded Linux development model.
Ⅲ Write, debug, and profile applications and drivers in embedded Linux.
Ⅲ Understand embedded Linux BSP architecture.
The book offers practical solutions to the above problems.
After reading this book the reader will
Ⅲ Understand the embedded Linux development environment.
Ⅲ Understand and create Linux BSP for a hardware platform.
Ⅲ Understand the Linux model for embedded storage and write drivers and
applications for the same.
Ⅲ Understand various embedded Linux drivers such as serial, I2C, and so on.
Ⅲ Port applications to embedded Linux from a traditional RTOS.
Ⅲ Write real-time applications in embedded Linux.
Ⅲ Learn methods to find memory leaks and memory corruption in applica-
tions and drivers.
Ⅲ Learn methods to profile applications and the kernel.
Ⅲ Understand uCLinux architecture and its programming model.
Ⅲ Understand the embedded Linux graphics subsystem.
Preface xxi
The book is also an aid to managers in choosing an embedded Linux
distribution, creating a roadmap for the transition to embedded Linux, and
applying the Linux licensing model in a commercial product.
Audience
Primary Audience
Ⅲ Architects: They are more concerned with real-time issues, performance,
and porting plans.
Ⅲ Software programmers: They need to get into the minute details of the
technology.

roaming here and there, like any mother telling her kid to study. I also express
thanks to my father who kept on asking me about the status of the manuscript,
like a project manager. I thank my wife, Parul, for her patience during
manuscript preparation. I remember when Raghav told me about this project
and asked me to join. It was just two months after my marriage. I thank Parul
for her encouragement and also thank her for helping me out in formatting
the manuscript.
Amol
Thanks to Raghav who had the idea of writing this book. I still remember
the first meeting when he instilled the confidence in me to take up this work.
I thank my dad, mom, and sister for their support. Thanks to the entire “boys”
gang at Bangalore who have been kind enough to share the powerful
“Athlon/Audigy/ATI Radeon 9500” game PC for mean activities such as running
Linux and typing sample code. They consider running a word processor on
such a PC as a gross waste of computing power.
Sriram
We take this opportunity to thank Todd Fischer, president and founder,
Cadenux, for giving us time from his busy schedule to write the foreword for
the book. We thank David McCullogh, one of the uClinux core maintainers,
xxiv Embedded Linux System Design and Development
and Dr. Paul Dale for reviewing the chapter on uClinux and for providing
their valuable comments. We also thank Greg Haerr, CEO of Century Software
and founder of the Nano-X windowing system, for his valuable review com-
ments on the embedded graphics chapter. We thank Satish MM, director of
Verismo Networks, for his valuable comments on GPL. We thank our close
friend and guide, Deepak Shenoy, for coming up with the idea to write a
book based on our development experience. Finally we thank all Linux kernel
developers and user-space programmers for taking Linux to new heights.