[generic]: 3.0 is out

[openwrt.git] / docs / adding.tex

diff --git a/docs/adding.tex b/docs/adding.tex
index 3b58084..97547ac 100644 (file)
--- a/docs/adding.tex
+++ b/docs/adding.tex
@@ -15,7 +15,7 @@ when they do, you will most likely not be able to complete the firmware creation
 
 This is one of the reasons why OpenWrt and other firmware exists: providing a 
 version independent, and tools independent firmware, that can be run on various 
 
 This is one of the reasons why OpenWrt and other firmware exists: providing a 
 version independent, and tools independent firmware, that can be run on various 

-platforms, known to be running Linux originaly.
+platforms, known to be running Linux originally.

 
 \subsection{Which Operating System does this device run?}
 
 
 \subsection{Which Operating System does this device run?}
 


@@ -30,14 +30,19 @@ fingerprinting, we will show here an example using \textbf{nmap}:
 
 \begin{Verbatim}
 nmap -P0 -O <IP address>
 
 \begin{Verbatim}
 nmap -P0 -O <IP address>

-Not shown: 1694 closed ports
-PORT     STATE SERVICE
-631/tcp  open  ipp
-1033/tcp open  netinfo
-6000/tcp open  X11
-Device type: general purpose
-Running: Apple Mac OS X 10.4.X
-OS details: Apple Mac OS X 10.4.8 (Tiger)
+Starting Nmap 4.20 ( http://insecure.org ) at 2007-01-08 11:05 CET
+Interesting ports on 192.168.2.1:
+Not shown: 1693 closed ports
+PORT   STATE SERVICE
+22/tcp open  ssh
+23/tcp open  telnet
+53/tcp open  domain
+80/tcp open  http
+MAC Address: 00:13:xx:xx:xx:xx (Cisco-Linksys)
+Device type: broadband router
+Running: Linksys embedded
+OS details: Linksys WRT54GS v4 running OpenWrt w/Linux kernel 2.4.30
+Network Distance: 1 hop

 \end{Verbatim}
 
 nmap is able to report whether your device uses a Linux TCP/IP stack, and if so,
 \end{Verbatim}
 
 nmap is able to report whether your device uses a Linux TCP/IP stack, and if so,


@@ -50,7 +55,16 @@ on the device, and which version of the service is being used:
 
 \begin{verbatim}
 nmap -P0 -sV <IP address>
 
 \begin{verbatim}
 nmap -P0 -sV <IP address>

-
+Starting Nmap 4.20 ( http://insecure.org ) at 2007-01-08 11:06 CET
+Interesting ports on 192.168.2.1:
+Not shown: 1693 closed ports
+PORT   STATE SERVICE VERSION
+22/tcp open  ssh     Dropbear sshd 0.48 (protocol 2.0)
+23/tcp open  telnet  Busybox telnetd
+53/tcp open  domain  ISC Bind dnsmasq-2.35
+80/tcp open  http    OpenWrt BusyBox httpd
+MAC Address: 00:13:xx:xx:xx:xx (Cisco-Linksys)
+Service Info: Device: WAP

 \end{verbatim}
 
 The web server version, if identified, can be determining in knowing the Operating
 \end{verbatim}
 
 The web server version, if identified, can be determining in knowing the Operating


@@ -107,27 +121,27 @@ Scroll over the firmware to find printable words that can be significant.
 
 \subsubsection{Amount of flash memory}
 
 
 \subsubsection{Amount of flash memory}
 

-Linux can hardly fit in a 2MB flash device, once you have open the device and 
-located the flash chip, try to find other the Internet its characteristics. If
+Linux can hardly fit in a 2MB flash device, once you have opened the device and 
+located the flash chip, try to find its characteristics on the Internet. If

 your flash chip is a 2MB or less device, your device is most likely to run a 
 proprietary OS such as WindRiver VxWorks, or a custom manufacturer OS like Zyxel ZynOS.
 
 your flash chip is a 2MB or less device, your device is most likely to run a 
 proprietary OS such as WindRiver VxWorks, or a custom manufacturer OS like Zyxel ZynOS.
 

-OpenWrt does not currently run on devices which have equal or less than 2MB of 
-flash memory. This limitation will probably not be worked around since those 
-devices are most of the time micro routers, or Wireless Access Points, which are
-not the main OpenWrt target.
+OpenWrt does not currently run on devices which have 2MB or less of flash memory.
+This limitation will probably not be worked around since those devices are most 
+of the time micro-routers, or Wireless Access Points, which are not the main 
+OpenWrt target.

 
 \subsubsection{Pluging a serial port}
 
 
 \subsubsection{Pluging a serial port}
 

-By using a serial port, you may reach the console that is being shown by the device
+By using a serial port and a level shifter, you may reach the console that is being shown by the device

 for debugging or flashing purposes. By analysing the output of this device, you can
 for debugging or flashing purposes. By analysing the output of this device, you can

-easily notice if the device uses a Linux kenrel or something different.
+easily notice if the device uses a Linux kernel or something different.

 
 \subsection{Finding and using the manufacturer SDK}
 
 Once you are sure your device run a Linux based firmware, you will be able to start
 
 \subsection{Finding and using the manufacturer SDK}
 
 Once you are sure your device run a Linux based firmware, you will be able to start

-hacking on it. If the manufacturer respect the GPL, it will have release with the 
-device, a Sample Development Kit.
+hacking on it. If the manufacturer respected the GPL, it will have released a Sample
+Development Kit with the device.

 
 \subsubsection{GPL violations}
 
 
 \subsubsection{GPL violations}
 


@@ -147,9 +161,11 @@ CD-ROM the open source software used to build or modify the firmware.
 
 In conformance to the GPL license, you have to release the following sources:
 
 
 In conformance to the GPL license, you have to release the following sources:
 

-- complete toolchain that made the kernel and applications be compiled (gcc, binutils, libc)
-- tools to build a custom firmware (mksquashfs, mkcramfs ...)
-- kernel sources with patches to make it run on this specific hardware, this does not include binary drivers
+\begin{itemize}
+\item complete toolchain that made the kernel and applications be compiled (gcc, binutils, libc)
+\item tools to build a custom firmware (mksquashfs, mkcramfs ...)
+\item kernel sources with patches to make it run on this specific hardware, this does not include binary drivers
+\end{itemize}

 
 Thank you very much in advance for your answer.
 
 
 Thank you very much in advance for your answer.
 


@@ -172,11 +188,11 @@ You should anyway be able to use the following components:
 \item binary tools to create a valid firmware image
 \end{itemize}
 
 \item binary tools to create a valid firmware image
 \end{itemize}
 

-Your work is now divided into the following tasks:
+Your work can be divided into the following tasks:

 
 \begin{itemize}
 \item create a clean patch of the hardware specific part of the linux kernel
 
 \begin{itemize}
 \item create a clean patch of the hardware specific part of the linux kernel

-\item spot potential kernel GPL violations especially on firewall and USB stack stuff
+\item spot potential kernel GPL violations especially on netfilter and USB stack stuff

 \item make the binary drivers work, until there are open source drivers
 \item use standard a GNU toolchain to make working executables
 \item understand and write open source tools to generate a valid firmware image
 \item make the binary drivers work, until there are open source drivers
 \item use standard a GNU toolchain to make working executables
 \item understand and write open source tools to generate a valid firmware image


@@ -201,7 +217,7 @@ VERSION = 2
 PATCHLEVEL = x
 SUBLEVEL = y
 EXTRAVERSION = z
 PATCHLEVEL = x
 SUBLEVEL = y
 EXTRAVERSION = z

-NAME=Avast! A bilge rat!
+NAME=A fancy name

 \end{verbatim}
 
 So now, you know that you have to download a standard kernel tarball at 
 \end{verbatim}
 
 So now, you know that you have to download a standard kernel tarball at 


@@ -222,19 +238,55 @@ your specific device. Of course, the content produced by the \textbf{diff -urN}
 may not always be relevant, so that you have to clean up those patches to only 
 let the "must have" code into them.
 
 may not always be relevant, so that you have to clean up those patches to only 
 let the "must have" code into them.
 

-The fist patch will contain all the code that is needed by the board to be 
+The first patch will contain all the code that is needed by the board to be 

 initialized at startup, as well as processor detection and other boot time 
 specific fixes.
 
 initialized at startup, as well as processor detection and other boot time 
 specific fixes.
 

-The second patch will contain all useful definitions for that board: adresses, 
-kernel granularity, redifinitions, processor family and features ...
+The second patch will contain all useful definitions for that board: addresses, 
+kernel granularity, redefinitions, processor family and features ...

 
 The third patch may contain drivers for: serial console, ethernet NIC, wireless 
 NIC, USB NIC ... Most of the time this patch contains nothing else than "glue"
 code that has been added to make the binary driver work with the Linux kernel. 
 
 The third patch may contain drivers for: serial console, ethernet NIC, wireless 
 NIC, USB NIC ... Most of the time this patch contains nothing else than "glue"
 code that has been added to make the binary driver work with the Linux kernel. 

-This code might not be useful if you plan on writing from scratch drivers for 
+This code might not be useful if you plan on writing drivers from scratch for 

 this hardware.
 
 this hardware.
 

+\subsubsection{Using the device bootloader}
+
+The bootloader is the first program that is started right after your device has 
+been powered on. This program, can be more or less sophisticated, some do let you 
+do network booting, USB mass storage booting ... The bootloader is device and 
+architecture specific, some bootloaders were designed to be universal such as 
+RedBoot or U-Boot so that you can meet those loaders on totally different 
+platforms and expect them to behave the same way.
+
+If your device runs a proprietary operating system, you are very likely to deal 
+with a proprietary boot loader as well. This may not always be a limitation, 
+some proprietary bootloaders can even have source code available (i.e : Broadcom CFE).
+
+According to the bootloader features, hacking on the device will be more or less 
+easier. It is very probable that the bootloader, even exotic and rare, has a 
+documentation somewhere over the Internet. In order to know what will be possible 
+with your bootloader and the way you are going to hack the device, look over the 
+following features :
+
+\begin{itemize}
+\item does the bootloader allow net booting via bootp/DHCP/NFS or tftp
+\item does the bootloader accept loading ELF binaries ?
+\item does the bootloader have a kernel/firmware size limitation ?
+\item does the bootloader expect a firmware format to be loaded with ?
+\item are the loaded files executed from RAM or flash ?
+\end{itemize}
+
+Net booting is something very convenient, because you will only have to set up network
+booting servers on your development station, and keep the original firmware on the device
+till you are sure you can replace it. This also prevents your device from being flashed,
+and potentially bricked every time you want to test a modification on the kernel/filesystem.
+
+If your device needs to be flashed every time you load a firmware, the bootlader might
+only accept a specific firmware format to be loaded, so that you will have to 
+understand the firmware format as well.
+

 \subsubsection{Making binary drivers work}
 
 As we have explained before, manufacturers do release binary drivers in their GPL
 \subsubsection{Making binary drivers work}
 
 As we have explained before, manufacturers do release binary drivers in their GPL


@@ -259,7 +311,6 @@ order to make binary drivers work with your custom kernel:
 \item CONFIG\_DEBUG\_KERNEL
 \item CONFIG\_DETECT\_SOFTLOCKUP
 \item CONFIG\_DEBUG\_KOBJECT
 \item CONFIG\_DEBUG\_KERNEL
 \item CONFIG\_DETECT\_SOFTLOCKUP
 \item CONFIG\_DEBUG\_KOBJECT

-\item CONFIG\_EMBEDDED

 \item CONFIG\_KALLSYMS
 \item CONFIG\_KALLSYMS\_ALL
 \end{itemize}
 \item CONFIG\_KALLSYMS
 \item CONFIG\_KALLSYMS\_ALL
 \end{itemize}


@@ -280,18 +331,18 @@ You might want to understand the firmware format, even if you are not yet capabl
 of running a custom firmware on your device, because this is sometimes a blocking
 part of the flashing process.
 
 of running a custom firmware on your device, because this is sometimes a blocking
 part of the flashing process.
 

-A firmare format is most of the time composed of the following fields:
+A firmware format is most of the time composed of the following fields:

 
 \begin{itemize}
 
 \begin{itemize}

-\item header, containing a firmare version and additional fields: Vendor, Hardware version ...
+\item header, containing a firmware version and additional fields: Vendor, Hardware version ...

 \item CRC32 checksum on either the whole file or just part of it
 \item CRC32 checksum on either the whole file or just part of it

-\item Binary or compressed kernel image
-\item Binary or compressed root filesystem image
+\item Binary and/or compressed kernel image
+\item Binary and/or compressed root filesystem image

 \item potential garbage
 \end{itemize}
 
 Once you have figured out how the firmware format is partitioned, you will have 
 \item potential garbage
 \end{itemize}
 
 Once you have figured out how the firmware format is partitioned, you will have 

-to write your own tool that produces valid firmare binaries. One thing to be very
+to write your own tool that produces valid firmware binaries. One thing to be very

 careful here is the endianness of either the machine that produces the binary 
 firmware and the device that will be flashed using this binary firmware.
 
 careful here is the endianness of either the machine that produces the binary 
 firmware and the device that will be flashed using this binary firmware.
 


@@ -306,7 +357,7 @@ firmware image and flash is structured. You will find below a commented example
 that covers the case of the device where the bootloader can pass to the kernel its partition plan.
 
 First of all, you need to make your flash map driver be visible in the kernel 
 that covers the case of the device where the bootloader can pass to the kernel its partition plan.
 
 First of all, you need to make your flash map driver be visible in the kernel 

-configuration options, this can be done by editing the file 
+configuration options, this can be done by editing the file \

 \textbf{linux/drivers/mtd/maps/Kconfig}:
 
 \begin{verbatim}
 \textbf{linux/drivers/mtd/maps/Kconfig}:
 
 \begin{verbatim}


@@ -367,7 +418,7 @@ static int __init device_mtd_init(void)
                return -EIO;
        }
 
                return -EIO;
        }
 

-          // Initlialise the device map
+          // Initialize the device map

        simple_map_init(&device_map);
 
           /* MTD informations are closely linked to the flash map device
        simple_map_init(&device_map);
 
           /* MTD informations are closely linked to the flash map device


@@ -423,7 +474,117 @@ module_init(device_mtd_init);
 module_exit(device_mtd_cleanup);
 
 
 module_exit(device_mtd_cleanup);
 
 

-// Macros defining licence and author, parameters can be defined here too.
+// Macros defining license and author, parameters can be defined here too.

 MODULE_LICENSE("GPL");
 MODULE_AUTHOR("Me, myself and I <memyselfandi@domain.tld");
 \end{verbatim}
 MODULE_LICENSE("GPL");
 MODULE_AUTHOR("Me, myself and I <memyselfandi@domain.tld");
 \end{verbatim}

+
+\subsection{Adding your target in OpenWrt}
+
+Once you spotted the key changes that were made to the Linux kernel
+to support your target, you will want to create a target in OpenWrt
+for your hardware. This can be useful to benefit from the toolchain
+that OpenWrt builds as well as the resulting user-space and kernel
+configuration options.
+
+Provided that your target is already known to OpenWrt, it will be
+as simple as creating a \texttt{target/linux/board} directory
+where you will be creating the following directories and files.
+
+Here for example, is a \texttt{target/linux/board/Makefile}:
+
+\begin{Verbatim}[frame=single,numbers=left]
+#
+# Copyright (C) 2009 OpenWrt.org
+#
+# This is free software, licensed under the GNU General Public License v2.
+# See /LICENSE for more information.
+#
+include $(TOPDIR)/rules.mk
+
+ARCH:=mips
+BOARD:=board
+BOARDNAME:=Eval board
+FEATURES:=squashfs jffs2 pci usb
+
+LINUX_VERSION:=2.6.27.10
+
+include $(INCLUDE_DIR)/target.mk
+
+DEFAULT_PACKAGES += hostapd-mini
+
+define Target/Description
+        Build firmware images for Evaluation board
+endef
+
+$(eval $(call BuildTarget))
+\end{Verbatim}
+
+\begin{itemize}
+    \item \texttt{ARCH} \\
+        The name of the architecture known by Linux and uClibc
+    \item \texttt{BOARD} \\
+        The name of your board that will be used as a package and build directory identifier
+    \item \texttt{BOARDNAME} \\
+        Expanded name that will appear in menuconfig
+    \item \texttt{FEATURES} \\
+        Set of features to build filesystem images, USB, PCI, VIDEO kernel support
+    \item \texttt{LINUX\_VERSION} \\
+        Linux kernel version to use for this target
+    \item \texttt{DEFAULT\_PACKAGES} \\
+        Set of packages to be built by default
+\end{itemize}
+
+A partial kernel configuration which is either named \texttt{config-default} or which matches the kernel version \texttt{config-2.6.x} should be present in \texttt{target/linux/board/}.
+This kernel configuration will only contain the relevant symbols to support your target and can be changed using \texttt{make kernel\_menuconfig}.
+
+To patch the kernel sources with the patches required to support your hardware, you will have to drop them in \texttt{patches} or in \texttt{patches-2.6.x} if there are specific
+changes between kernel versions. Additionnaly, if you want to avoid creating a patch that will create files, you can put those files into \texttt{files} or \texttt{files-2.6.x}
+with the same directory structure that the kernel uses (e.g: drivers/mtd/maps, arch/mips ..).
+
+The build system will require you to create a \texttt{target/linux/board/image/Makefile}:
+
+\begin{Verbatim}[frame=single,numbers=left]
+#
+# Copyright (C) 2009 OpenWrt.org
+#
+# This is free software, licensed under the GNU General Public License v2.
+# See /LICENSE for more information.
+#
+include $(TOPDIR)/rules.mk
+include $(INCLUDE_DIR)/image.mk
+
+define Image/BuildKernel
+        cp $(KDIR)/vmlinux.elf $(BIN_DIR)/openwrt-$(BOARD)-vmlinux.elf
+        gzip -9 -c $(KDIR)/vmlinux > $(KDIR)/vmlinux.bin.gz
+        $(STAGING_DIR_HOST)/bin/lzma e $(KDIR)/vmlinux $(KDIR)/vmlinux.bin.l7
+        dd if=$(KDIR)/vmlinux.bin.l7 of=$(BIN_DIR)/openwrt-$(BOARD)-vmlinux.lzma bs=65536 conv=sync
+        dd if=$(KDIR)/vmlinux.bin.gz of=$(BIN_DIR)/openwrt-$(BOARD)-vmlinux.gz bs=65536 conv=sync
+endef
+
+define Image/Build/squashfs
+    $(call prepare_generic_squashfs,$(KDIR)/root.squashfs)
+endef
+
+define Image/Build
+        $(call Image/Build/$(1))
+        dd if=$(KDIR)/root.$(1) of=$(BIN_DIR)/openwrt-$(BOARD)-root.$(1) bs=128k conv=sync
+
+        -$(STAGING_DIR_HOST)/bin/mkfwimage \
+                -B XS2 -v XS2.ar2316.OpenWrt \
+                -k $(BIN_DIR)/openwrt-$(BOARD)-vmlinux.lzma \
+                -r $(BIN_DIR)/openwrt-$(BOARD)-root.$(1) \
+                -o $(BIN_DIR)/openwrt-$(BOARD)-ubnt2-$(1).bin
+endef
+
+$(eval $(call BuildImage))
+
+\end{Verbatim}
+
+\begin{itemize}
+    \item \texttt{Image/BuildKernel} \\
+        This template defines changes to be made to the ELF kernel file
+    \item \texttt{Image/Build} \\
+       This template defines the final changes to apply to the rootfs and kernel, either combined or separated
+       firmware creation tools can be called here as well.
+\end{itemize}