docs/build.tex

   1 One of the biggest challenges to getting started with embedded devices is that you
   2 cannot just install a copy of Linux and expect to be able to compile a firmware.
   3 Even if you did remember to install a compiler and every development tool offered,
   4 you still would not have the basic set of tools needed to produce a firmware image.
   5 The embedded device represents an entirely new hardware platform, which is
   6 most of the time incompatible with the hardware on your development machine, so in a process called
   7 cross compiling you need to produce a new compiler capable of generating code for
   8 your embedded platform, and then use it to compile a basic Linux distribution to
   9 run on your device.
  10
  11 The process of creating a cross compiler can be tricky, it is not something that is
  12 regularly attempted and so there is a certain amount of mystery and black magic
  13 associated with it. In many cases when you are dealing with embedded devices you will
  14 be provided with a binary copy of a compiler and basic libraries rather than
  15 instructions for creating your own -- it is a time saving step but at the same time
  16 often means you will be using a rather dated set of tools. Likewise, it is also common
  17 to be provided with a patched copy of the Linux kernel from the board or chip vendor,
  18 but this is also dated and it can be difficult to spot exactly what has been
  19 modified to make the kernel run on the embedded platform.
  20
  21 \subsection{Building an image}
  22
  23 OpenWrt takes a different approach to building a firmware; downloading, patching
  24 and compiling everything from scratch, including the cross compiler. To put it
  25 in simpler terms, OpenWrt does not contain any executables or even sources, it is an
  26 automated system for downloading the sources, patching them to work with the given
  27 platform and compiling them correctly for that platform. What this means is that
  28 just by changing the template, you can change any step in the process.
  29
  30 As an example, if a new kernel is released, a simple change to one of the Makefiles
  31 will download the latest kernel, patch it to run on the embedded platform and produce
  32 a new firmware image -- there is no work to be done trying to track down an unmodified
  33 copy of the existing kernel to see what changes had been made, the patches are
  34 already provided and the process ends up almost completely transparent. This does not
  35 just apply to the kernel, but to anything included with OpenWrt -- It is this one
  36 simple understated concept which is what allows OpenWrt to stay on the bleeding edge
  37 with the latest compilers, latest kernels and latest applications.
  38
  39 So let's take a look at OpenWrt and see how this all works.
  40
  41
  42 \subsubsection{Download OpenWrt}
  43
  44 This article refers to the "Kamikaze" branch of OpenWrt, which can be downloaded via
  45 subversion using the following command:
  46
  47 \begin{Verbatim}
  48 $ svn checkout https://svn.openwrt.org/openwrt/trunk kamikaze
  49 \end{Verbatim}
  50
  51 Additionally, there is a trac interface on \href{https://dev.openwrt.org/}{https://dev.openwrt.org/}
  52 which can be used to monitor svn commits and browse the source repository.
  53
  54
  55 \subsubsection{The directory structure}
  56
  57 There are four key directories in the base:
  58
  59 \begin{itemize}
  60     \item \texttt{tools}
  61     \item \texttt{toolchain}
  62     \item \texttt{package}
  63     \item \texttt{target}
  64 \end{itemize}
  65
  66 \texttt{tools} and \texttt{toolchain} refer to common tools which will be
  67 used to build the firmware image, the compiler, and the C library.
  68 The result of this is three new directories, \texttt{build\_dir/host}, which is a temporary
  69 directory for building the target independent tools, \texttt{build\_dir/toolchain-\textit{<arch>}*}
  70 which is used for building the toolchain for a specific architecture, and
  71 \texttt{staging\_dir/toolchain-\textit{<arch>}*} where the resulting toolchain is installed.
  72 You will not need to do anything with the toolchain directory unless you intend to
  73 add a new version of one of the components above.
  74
  75 \begin{itemize}
  76     \item \texttt{build\_dir/host}
  77     \item \texttt{build\_dir/toolchain-\textit{<arch>}*}
  78 \end{itemize}
  79
  80 \texttt{package} is for exactly that -- packages. In an OpenWrt firmware, almost everything
  81 is an \texttt{.ipk}, a software package which can be added to the firmware to provide new
  82 features or removed to save space. Note that packages are also maintained outside of the main
  83 trunk and can be obtained from subversion using the package feeds system:
  84
  85 \begin{Verbatim}
  86 $ ./scripts/feeds update
  87 \end{Verbatim}
  88
  89 Those packages can be used to extend the functionality of the build system and need to be
  90 symlinked into the main trunk. Once you do that, the packages will show up in the menu for
  91 configuration. From kamikaze you would do something like this:
  92
  93 \begin{Verbatim}
  94 $ ./scripts/feeds search nmap
  95 Search results in feed 'packages':
  96 nmap       Network exploration and/or security auditing utility
  97
  98 $ ./scripts/feeds install nmap
  99 \end{Verbatim}
 100
 101 To include all packages, issue the following command:
 102
 103 \begin{Verbatim}
 104 $ make package/symlinks
 105 \end{Verbatim}
 106
 107 \texttt{target} refers to the embedded platform, this contains items which are specific to
 108 a specific embedded platform. Of particular interest here is the "\texttt{target/linux}"
 109 directory which is broken down by platform \textit{<arch>} and contains the patches to the
 110 kernel, profile config, for a particular platform. There's also the "\texttt{target/image}" directory
 111 which describes how to package a firmware for a specific platform.
 112
 113 Both the target and package steps will use the directory "\texttt{build\_dir/\textit{<arch>}}"
 114 as a temporary directory for compiling. Additionally, anything downloaded by the toolchain,
 115 target or package steps will be placed in the "\texttt{dl}" directory.
 116
 117 \begin{itemize}
 118     \item \texttt{build\_dir/\textit{<arch>}}
 119     \item \texttt{dl}
 120 \end{itemize}
 121
 122 \subsubsection{Building OpenWrt}
 123
 124 While the OpenWrt build environment was intended mostly for developers, it also has to be
 125 simple enough that an inexperienced end user can easily build his or her own customized firmware.
 126
 127 Running the command "\texttt{make menuconfig}" will bring up OpenWrt's configuration menu
 128 screen, through this menu you can select which platform you're targeting, which versions of
 129 the toolchain you want to use to build and what packages you want to install into the
 130 firmware image. Note that it will also check to make sure you have the basic dependencies for it
 131 to run correctly.  If that fails, you will need to install some more tools in your local environment
 132 before you can begin.
 133
 134 Similar to the linux kernel config, almost every option has three choices,
 135 \texttt{y/m/n} which are represented as follows:
 136
 137 \begin{itemize}
 138     \item{\texttt{<*>} (pressing y)} \\
 139         This will be included in the firmware image
 140     \item{\texttt{<M>} (pressing m)} \\
 141         This will be compiled but not included (for later install)
 142     \item{\texttt{< >} (pressing n)} \\
 143         This will not be compiled
 144 \end{itemize}
 145
 146 After you've finished with the menu configuration, exit and when prompted, save your
 147 configuration changes.
 148
 149 If you want, you can also modify the kernel config for the selected target system.
 150 simply run "\texttt{make kernel\_menuconfig}" and the build system will unpack the kernel sources
 151 (if necessary), run menuconfig inside of the kernel tree, and then copy the kernel config
 152 to \texttt{target/linux/\textit{<platform>}/config} so that it is preserved over
 153 "\texttt{make clean}" calls.
 154
 155 To begin compiling the firmware, type "\texttt{make}". By default
 156 OpenWrt will only display a high level overview of the compile process and not each individual
 157 command.
 158
 159 \subsubsection{Example:}
 160
 161 \begin{Verbatim}
 162 make[2] toolchain/install
 163 make[3] -C toolchain install
 164 make[2] target/compile
 165 make[3] -C target compile
 166 make[4] -C target/utils prepare
 167
 168 [...]
 169 \end{Verbatim}
 170
 171 This makes it easier to monitor which step it's actually compiling and reduces the amount
 172 of noise caused by the compile output. To see the full output, run the command
 173 "\texttt{make V=99}".
 174
 175 During the build process, buildroot will download all sources to the "\texttt{dl}"
 176 directory and will start patching and compiling them in the "\texttt{build\_dir/\textit{<arch>}}"
 177 directory. When finished, the resulting firmware will be in the "\texttt{bin}" directory
 178 and packages will be in the "\texttt{bin/packages}" directory.
 179
 180
 181 \subsection{Creating packages}
 182
 183 One of the things that we've attempted to do with OpenWrt's template system is make it
 184 incredibly easy to port software to OpenWrt. If you look at a typical package directory
 185 in OpenWrt you'll find two things:
 186
 187 \begin{itemize}
 188     \item \texttt{package/\textit{<name>}/Makefile}
 189     \item \texttt{package/\textit{<name>}/patches}
 190     \item \texttt{package/\textit{<name>}/files}
 191 \end{itemize}
 192
 193 The patches directory is optional and typically contains bug fixes or optimizations to
 194 reduce the size of the executable. The package makefile is the important item, provides
 195 the steps actually needed to download and compile the package.
 196
 197 The files directory is also optional and typicall contains package specific startup scripts or default configuration files that can be used out of the box with OpenWrt.
 198
 199 Looking at one of the package makefiles, you'd hardly recognize it as a makefile.
 200 Through what can only be described as blatant disregard and abuse of the traditional
 201 make format, the makefile has been transformed into an object oriented template which
 202 simplifies the entire ordeal.
 203
 204 Here for example, is \texttt{package/bridge/Makefile}:
 205
 206 \begin{Verbatim}[frame=single,numbers=left]
 207 # $Id: Makefile 5624 2006-11-23 00:29:07Z nbd $
 208
 209 include $(TOPDIR)/rules.mk
 210
 211 PKG_NAME:=bridge
 212 PKG_VERSION:=1.0.6
 213 PKG_RELEASE:=1
 214
 215 PKG_SOURCE:=bridge-utils-$(PKG_VERSION).tar.gz
 216 PKG_SOURCE_URL:=@SF/bridge
 217 PKG_MD5SUM:=9b7dc52656f5cbec846a7ba3299f73bd
 218 PKG_CAT:=zcat
 219
 220 PKG_BUILD_DIR:=$(BUILD_DIR)/bridge-utils-$(PKG_VERSION)
 221
 222 include $(INCLUDE_DIR)/package.mk
 223
 224 define Package/bridge
 225   SECTION:=net
 226   CATEGORY:=Base system
 227   TITLE:=Ethernet bridging configuration utility
 228   URL:=http://bridge.sourceforge.net/
 229 endef
 230
 231 define Package/bridge/description
 232   Manage ethernet bridging:
 233   a way to connect networks together to form a larger network.
 234 endef
 235
 236 define Build/Configure
 237     $(call Build/Configure/Default, \
 238         --with-linux-headers="$(LINUX_DIR)" \
 239     )
 240 endef
 241
 242 define Package/bridge/install
 243     $(INSTALL_DIR) $(1)/usr/sbin
 244     $(INSTALL_BIN) $(PKG_BUILD_DIR)/brctl/brctl $(1)/usr/sbin/
 245 endef
 246
 247 $(eval $(call BuildPackage,bridge))
 248 \end{Verbatim}
 249
 250 As you can see, there's not much work to be done; everything is hidden in other makefiles
 251 and abstracted to the point where you only need to specify a few variables.
 252
 253 \begin{itemize}
 254     \item \texttt{PKG\_NAME} \\
 255         The name of the package, as seen via menuconfig and ipkg
 256     \item \texttt{PKG\_VERSION} \\
 257         The upstream version number that we are downloading
 258     \item \texttt{PKG\_RELEASE} \\
 259         The version of this package Makefile
 260     \item \texttt{PKG\_SOURCE} \\
 261         The filename of the original sources
 262     \item \texttt{PKG\_SOURCE\_URL} \\
 263         Where to download the sources from (no trailing slash), you can add multiple download sources by separating them with a \\ and a carriage return.
 264     \item \texttt{PKG\_MD5SUM} \\
 265         A checksum to validate the download
 266     \item \texttt{PKG\_CAT} \\
 267         How to decompress the sources (zcat, bzcat, unzip)
 268     \item \texttt{PKG\_BUILD\_DIR} \\
 269         Where to compile the package
 270 \end{itemize}
 271
 272 The \texttt{PKG\_*} variables define where to download the package from;
 273 \texttt{@SF} is a special keyword for downloading packages from sourceforge. There is also
 274 another keyword of \texttt{@GNU} for grabbing GNU source releases. If any of the above mentionned download source fails, the OpenWrt mirrors will be used as source.
 275
 276 The md5sum (if present) is used to verify the package was downloaded correctly and
 277 \texttt{PKG\_BUILD\_DIR} defines where to find the package after the sources are
 278 uncompressed into \texttt{\$(BUILD\_DIR)}.
 279
 280 At the bottom of the file is where the real magic happens, "BuildPackage" is a macro
 281 set up by the earlier include statements. BuildPackage only takes one argument directly --
 282 the name of the package to be built, in this case "\texttt{bridge}". All other information
 283 is taken from the define blocks. This is a way of providing a level of verbosity, it's
 284 inherently clear what the contents of the \texttt{description} template in
 285 \texttt{Package/bridge} is, which wouldn't be the case if we passed this information
 286 directly as the Nth argument to \texttt{BuildPackage}.
 287
 288 \texttt{BuildPackage} uses the following defines:
 289
 290 \textbf{\texttt{Package/\textit{<name>}}:} \\
 291     \texttt{\textit{<name>}} matches the argument passed to buildroot, this describes
 292     the package the menuconfig and ipkg entries. Within \texttt{Package/\textit{<name>}}
 293     you can define the following variables:
 294
 295     \begin{itemize}
 296         \item \texttt{SECTION} \\
 297             The type of package (currently unused)
 298         \item \texttt{CATEGORY} \\
 299             Which menu it appears in menuconfig: Network, Sound, Utilities, Multimedia ...
 300         \item \texttt{TITLE} \\
 301             A short description of the package
 302         \item \texttt{URL} \\
 303             Where to find the original software
 304         \item \texttt{MAINTAINER} (optional) \\
 305             Who to contact concerning the package
 306         \item \texttt{DEPENDS} (optional) \\
 307             Which packages must be built/installed before this package. To reference a dependency defined in the
 308                         same Makefile, use \textit{<dependency name>}. If defined as an external package, use
 309                         \textit{+<dependency name>}. For a kernel version dependency use: \textit{@LINUX\_2\_<minor version>}
 310     \end{itemize}
 311
 312 \textbf{\texttt{Package/\textit{<name>}/conffiles} (optional):} \\
 313    A list of config files installed by this package, one file per line.
 314
 315 \textbf{\texttt{Build/Prepare} (optional):} \\
 316    A set of commands to unpack and patch the sources. You may safely leave this
 317    undefined.
 318
 319 \textbf{\texttt{Build/Configure} (optional):} \\
 320    You can leave this undefined if the source doesn't use configure or has a
 321    normal config script, otherwise you can put your own commands here or use
 322    "\texttt{\$(call Build/Configure/Default,\textit{<first list of arguments, second list>})}" as above to
 323    pass in additional arguments for a standard configure script. The first list of arguments will be passed
 324    to the configure script like that: \texttt{--arg 1} \texttt{--arg 2}. The second list contains arguments that should be
 325    defined before running the configure script such as autoconf or compiler specific variables.
 326
 327    To make it easier to modify the configure command line, you can either extend or completely override the following variables:
 328    \begin{itemize}
 329      \item \texttt{CONFIGURE\_ARGS} \\
 330              Contains all command line arguments (format: \texttt{--arg 1} \texttt{--arg 2})
 331      \item \texttt{CONFIGURE\_VARS} \\
 332              Contains all environment variables that are passed to ./configure (format: \texttt{NAME="value"})
 333    \end{itemize}
 334
 335 \textbf{\texttt{Build/Compile} (optional):} \\
 336    How to compile the source; in most cases you should leave this undefined.
 337
 338    As with \texttt{Build/Configure} there are two variables that allow you to override
 339    the make command line environment variables and flags:
 340    \begin{itemize}
 341      \item \texttt{MAKE\_FLAGS} \\
 342            Contains all command line arguments (typically variable overrides like \texttt{NAME="value"}
 343          \item \texttt{MAKE\_VARS} \\
 344            Contains all environment variables that are passed to the make command
 345    \end{itemize}
 346
 347 \textbf{\texttt{Build/InstallDev} (optional):} \\
 348         If your package provides a library that needs to be made available to other packages,
 349         you can use the \texttt{Build/InstallDev} template to copy it into the staging directory
 350         which is used to collect all files that other packages might depend on at build time.
 351         When it is called by the build system, two parameters are passed to it. \texttt{\$(1)} points to
 352         the regular staging dir, typically \texttt{staging\_dir/\textit{ARCH}}, while \texttt{\$(2)} points
 353         to \texttt{staging\_dir/host}. The host staging dir is only used for binaries, which are
 354         to be executed or linked against on the host and its \texttt{bin/} subdirectory is included
 355         in the \texttt{PATH} which is passed down to the build system processes.
 356         Please use \texttt{\$(1)} and \texttt{\$(2)} here instead of the build system variables
 357         \texttt{\$(STAGING\_DIR)} and \texttt{\$(STAGING\_DIR\_HOST)}, because the build system behavior
 358         when staging libraries might change in the future to include automatic uninstallation.
 359
 360 \textbf{\texttt{Package/\textit{<name>}/install}:} \\
 361    A set of commands to copy files out of the compiled source and into the ipkg
 362    which is represented by the \texttt{\$(1)} directory. Note that there are currently
 363    4 defined install macros:
 364    \begin{itemize}
 365        \item \texttt{INSTALL\_DIR} \\
 366            install -d -m0755
 367        \item \texttt{INSTALL\_BIN} \\
 368            install -m0755
 369        \item \texttt{INSTALL\_DATA} \\
 370            install -m0644
 371        \item \texttt{INSTALL\_CONF} \\
 372            install -m0600
 373    \end{itemize}
 374
 375 The reason that some of the defines are prefixed by "\texttt{Package/\textit{<name>}}"
 376 and others are simply "\texttt{Build}" is because of the possibility of generating
 377 multiple packages from a single source. OpenWrt works under the assumption of one
 378 source per package Makefile, but you can split that source into as many packages as
 379 desired. Since you only need to compile the sources once, there's one global set of
 380 "\texttt{Build}" defines, but you can add as many "Package/<name>" defines as you want
 381 by adding extra calls to \texttt{BuildPackage} -- see the dropbear package for an example.
 382
 383 After you have created your \texttt{package/\textit{<name>}/Makefile}, the new package
 384 will automatically show in the menu the next time you run "make menuconfig" and if selected
 385 will be built automatically the next time "\texttt{make}" is run.
 386
 387 \subsection{Creating kernel modules packages}
 388
 389 The OpenWrt distribution makes the distinction between two kind of kernel modules, those coming along with the mainline kernel, and the others available as a separate project. We will see later that a common template is used for both of them.
 390
 391 For kernel modules that are part of the mainline kernel source, the makefiles are located in \textit{package/kernel/modules/*.mk} and they appear under the section "Kernel modules"
 392
 393 For external kernel modules, you can add them to the build system just like if they were software packages by defining a KernelPackage section in the package makefile.
 394
 395 Here for instance the Makefile for the I2C subsytem kernel modules :
 396
 397 \begin{Verbatim}[frame=single,numbers=left]
 398 # $Id $
 399
 400 I2CMENU:=I2C Bus
 401
 402 define KernelPackage/i2c-core
 403   TITLE:=I2C support
 404   DESCRIPTION:=Kernel modules for i2c support
 405   SUBMENU:=$(I2CMENU)
 406   KCONFIG:=CONFIG_I2C_CORE CONFIG_I2C_DEV
 407   FILES:=$(MODULES_DIR)/kernel/drivers/i2c/*.$(LINUX_KMOD_SUFFIX)
 408   AUTOLOAD:=$(call AutoLoad,50,i2c-core i2c-dev)
 409 endef
 410 $(eval $(call KernelPackage,i2c-core))
 411 \end{Verbatim}
 412
 413 To group kernel modules under a common description in menuconfig, you might want to define a \textit{<description>MENU} variable on top of the kernel modules makefile.
 414
 415 \begin{itemize}
 416     \item \texttt{TITLE} \\
 417         The name of the module as seen via menuconfig
 418     \item \texttt{DESCRIPTION} \\
 419         The description as seen via help in menuconfig
 420     \item \texttt{SUBMENU} \\
 421         The sub menu under which this package will be seen
 422     \item \texttt{KCONFIG} \\
 423         Kernel configuration option dependency. For external modules, remove it.
 424     \item \texttt{FILES} \\
 425         Files you want to inlude to this kernel module package, separate with spaces.
 426     \item \texttt{AUTOLOAD} \\
 427         Modules that will be loaded automatically on boot, the order you write them is the order they would be loaded.
 428 \end{itemize}
 429
 430 After you have created your \texttt{package/kernel/modules/\textit{<name>}.mk}, the new kernel modules package
 431 will automatically show in the menu under "Kernel modules" next time you run "make menuconfig" and if selected
 432 will be built automatically the next time "\texttt{make}" is run.
 433
 434 \subsection{Conventions}
 435
 436 There are a couple conventions to follow regarding packages:
 437
 438 \begin{itemize}
 439     \item \texttt{files}
 440     \begin{enumerate}
 441         \item configuration files follow the convention \\
 442         \texttt{\textit{<name>}.conf}
 443         \item init files follow the convention \\
 444         \texttt{\textit{<name>}.init}
 445     \end{enumerate}
 446     \item \texttt{patches}
 447     \begin{enumerate}
 448         \item patches are numerically prefixed and named related to what they do
 449     \end{enumerate}
 450 \end{itemize}
 451
 452 \subsection{Troubleshooting}
 453
 454 If you find your package doesn't show up in menuconfig, try the following command to
 455 see if you get the correct description:
 456
 457 \begin{Verbatim}
 458   TOPDIR=$PWD make -C package/<name> DUMP=1 V=99
 459 \end{Verbatim}
 460
 461 If you're just having trouble getting your package to compile, there's a few
 462 shortcuts you can take. Instead of waiting for make to get to your package, you can
 463 run one of the following:
 464
 465 \begin{itemize}
 466     \item \texttt{make package/\textit{<name>}/clean V=99}
 467     \item \texttt{make package/\textit{<name>}/install V=99}
 468 \end{itemize}
 469
 470 Another nice trick is that if the source directory under \texttt{build\_dir/\textit{<arch>}}
 471 is newer than the package directory, it won't clobber it by unpacking the sources again.
 472 If you were working on a patch you could simply edit the sources under the
 473 \texttt{build\_dir/\textit{<arch>}/\textit{<source>}} directory and run the install command above,
 474 when satisfied, copy the patched sources elsewhere and diff them with the unpatched
 475 sources. A warning though - if you go modify anything under \texttt{package/\textit{<name>}}
 476 it will remove the old sources and unpack a fresh copy.
 477
 478 Other useful targets include:
 479
 480 \begin{itemize}
 481     \item \texttt{make package/\textit{<name>}/prepare V=99}
 482     \item \texttt{make package/\textit{<name>}/compile V=99}
 483     \item \texttt{make package/\textit{<name>}/configure V=99}
 484 \end{itemize}
 485