User Tools

Site Tools


cucumber_linux_from_scratch

Cucumber Linux from Scratch

Chapter 1 - Introduction

Chapter 2 - Preparing the Build Location

Introduction

In this chapter, the build environment is prepared. We will create two subdirectories: one for building the temporary system (Chapter 5) and another for building the final system (Chapters 6 & 7).

Creating the Directories

The Cucumber Linux from scratch build will reside in /opt/culfs. Start by creating that directory and changing the working directory to it:

mkdir -pv /opt/culfs
cd /opt/culfs

The two subdirectories (lfscript and chroot) will be created when in subsequent chapters as they become necessary.

Chapter 3 - Downloading the Package Sources

Introduction

There are two source trees that must be downloaded: the LFScript source tree (which will be used to build chapter 5) and the Cucumber Linux source tree (which will be used to build chapters 6 & 7).

Downloading LFScript

Clone the Git repository for the Cucumber Linux version of LFScript:

git clone https://github.com/cucumberlinux/lfscript.git

If you are building the current development version of Cucumber Linux, then no further action is necessary for this step and you may move on to the Downloading the Cucumber Linux Source Tree section; however, if you are building a different version of Cucumber Linux, it will be necessary to switch to the Git branch for that version. For example, if you are building any version of Cucumber Linux 2.x, run:

cd lfscript
git checkout 2.x
cd ..

Downloading the Cucumber Linux Source Tree

Clone the Git repository for the official Cucumber Linux source tree:

install -m 755 -d chroot/usr
cd chroot/usr
git clone https://github.com/cucumberlinux/ports.git

If you are building a version of Cucumber Linux other than the current development version, switch to the appropriate Git branch for it:

cd ports
git checkout 2.0
cd ..

Temporarily change the portmake configuration so it expects to find the ports tree in /opt:

if [ -e /etc/portmake.conf ]; then
  mv /etc/portmake.conf{,.orig}
fi
echo PORTS_TREE=/opt/culfs/chroot/usr/ports > /etc/portmake.conf

Download the source tarballs for the phase 2 packages. This must be done now, as the chroot environment will not have internet access until after phase 2 is built. This step will also verify the integrity of all the downloaded tarballs via checksums and PGP signatures.

ports/utilities/tools/portmake download-recursive phase2

Restore the original portmake.conf.

if [ -e /etc/portmake.conf.orig ]; then
  mv /etc/portmake.conf{.orig,}
else
  rm /etc/portmake.conf
fi

Downloading a Couple of Additional Packages

There are two additional packages that must be downloaded now so they are available immediately inside the chroot environment: pkgtools and tar 1.13. Create a directory to store them in:

cd ..
mkdir sources
cd sources

Download the source archives for them:

wget https://github.com/cucumberlinux/pkgtools/archive/c2.0.0.tar.gz
wget https://ftp.gnu.org/gnu/tar/tar-1.13.tar.gz

Note that you *must* download tar version 1.13 specifically. The version of pkgtools that is downloaded should be the latest patch release available for the version of Cucumber Linux you are building. For example, if you are building Cucumber Linux 3.0, then you should use the latest version of pkgtools 3.0.x.

Finally cd back to /opt/culfs.

cd ../..

Chapter 4 - Final Perparations

Introduction

In this chapter, we will perform a few additional tasks to prepare for building the temporary system.

Adding the lfs user

Create a directory to download the phase 1 source tarballs to:

mkdir /​opt/​culfs/​lfscript/​sources

Ensure that the lfs user has full access to /opt/culfs/lfscript:

chown -Rv lfs /opt/culfs/lfscript

Chapter 5 - Building Phase 1

Introduction

This phase is akin to Chapter 5 of Linux from Scratch: Constructing the Temporary System. This step will be automated using a modified version of LFScript.

This chapter has two parts: first, we will build a tarball for the /tools directory. Then, we will extract it to /opt/culfs/chroot/tools and prepare the chroot environment.

Building the Tools Tarball

CD back into the lfscript directory:

cd lfscript

Run the following command to build the tools tarball:

./lfscript -B

Note that lfscript will exit with a reported failure. This is because lfscript will attempt to continue on after chapter 5 and begin building chapter 6. It will fail at building chapter 6 (because we don't use it for that part on Cucumber Linux), but will still build the tools tarball successfully. Therefore, this error can be safely ignored. When you reach the error message, press ^C.

Installing /tools to the Chroot

CD to the chroot environment and extract the tools tarball:

cd ../chroot
tar -xvJf ../lfscript/packages-*/lfs-*/toolchain.bak.txz

Preparing the Chroot

There are a few additional steps that must be performed before entering the chroot environment.

Set the LFS Variable

Set the LFS variable to the chroot directory:

export LFS=/opt/culfs/chroot

Set up the Kernel File Systems

For details about this step see http://www.linuxfromscratch.org/lfs/view/8.0/chapter06/kernfs.html.

Various file systems exported by the kernel are used to communicate to and from the kernel itself. Create the mount points for them:

mkdir -pv $LFS/{dev,proc,sys,run}

Perform a bind mount for /dev:

mount --bind /dev $LFS/dev

Mount the remaining virtual file systems:

mount -vt devpts devpts $LFS/dev/pts -o gid=5,mode=620
mount -vt proc proc $LFS/proc
mount -vt sysfs sysfs $LFS/sys
mount -vt tmpfs tmpfs $LFS/run

Important Note about the Chroot Environment

If at any point you reboot your build system or close out of the shell you are performing the build in, it will be necessary to repeat the steps in the “Preparing the Chroot” section before reentering the chroot environment.

Chapter 6 - Building Phase 2

Introduction

This phase is akin to Chapter 6 of Linux from Scratch: Installing Basic System Software. The majority of this chapter will be automated using the official Cucumber Linux ports tree and the portmake utility; however, a few of the early packages will need to be built by hand.

Enter the Chroot

After completing the “Preparing the Chroot” section of Chapter 5, enter the chroot environment:

chroot "$LFS" /tools/bin/env -i \
  HOME=/root                  \
  TERM="$TERM"                \
  PS1='\u:\w\$ '              \
  PATH=/bin:/usr/bin:/sbin:/usr/sbin:/tools/bin/:/tools/sbin \
  /tools/bin/bash --login +h
  

Final Preparations

Create Essential Directories

It is time to create some structure in the LFS file system. Create a standard directory tree by issuing the following commands:

mkdir -pv /{bin,boot,etc/{opt,sysconfig},home,lib/firmware,mnt,opt}
mkdir -pv /{media/{floppy,cdrom},sbin,srv,var}
install -dv -m 0750 /root
install -dv -m 1777 /tmp /var/tmp
mkdir -pv /usr/{,local/}{bin,include,lib,sbin,src}
mkdir -pv /usr/{,local/}share/{color,dict,doc,info,locale,man}
mkdir -v  /usr/{,local/}share/{misc,terminfo,zoneinfo}
mkdir -v  /usr/libexec
mkdir -pv /usr/{,local/}share/man/man{1..8}

case $(uname -m) in
 x86_64) mkdir -v /lib64 ;;
esac

mkdir -v /var/{log,mail,spool}
ln -sv /run /var/run
ln -sv /run/lock /var/lock
mkdir -pv /var/{opt,cache,lib/{color,misc,locate},local}

Some programs use hardwired paths to other programs that do not exist yet. Create some temporary symlinks to satisfy them:

install -m 755 -d /bin /usr/{bin,lib} /etc /sbin /var/log
ln -sv /tools/bin/{bash,cat,echo,pwd,stty} /bin
ln -sv /tools/bin/perl /usr/bin
ln -sv /tools/lib/libgcc_s.so{,.1} /usr/lib
ln -sv /tools/lib/libstdc++.so{,.6} /usr/lib
sed 's/tools/usr/' /tools/lib/libstdc++.la > /usr/lib/libstdc++.la
ln -sv bash /bin/sh
ln -sv /proc/self/mounts /etc/mtab
ln -sv /tools/sbin/installpkg /sbin/
ln -sv /tools/sbin/makepkg /sbin/
ln -sv /tools/sbin/upgradepkg /sbin/
ln -sv /tools/bin/m4 /usr/bin/

Create Essential Files

Create a basic /etc/passwd:

cat > /etc/passwd << "EOF"
root::0:0:root:/root:/bin/bash
bin:x:1:1:bin:/dev/null:/bin/false
daemon:x:6:6:Daemon User:/dev/null:/bin/false
messagebus:x:18:18:D-Bus Message Daemon User:/var/run/dbus:/bin/false
nobody:x:99:99:Unprivileged User:/dev/null:/bin/false
EOF

Create a basic /etc/group:

cat > /etc/group << "EOF"
root:x:0:
bin:x:1:daemon
sys:x:2:
kmem:x:3:
tape:x:4:
tty:x:5:
daemon:x:6:
floppy:x:7:
disk:x:8:
lp:x:9:
dialout:x:10:
audio:x:11:
video:x:12:
utmp:x:13:
usb:x:14:
cdrom:x:15:
adm:x:16:
messagebus:x:18:
systemd-journal:x:23:
input:x:24:
mail:x:34:
nogroup:x:99:
users:x:999:
EOF

Initialize Log Files

The login, agetty, and init programs (and others) use a number of log files to record information such as who was logged into the system and when. However, these programs will not write to the log files if they do not already exist. Initialize the log files and give them proper permissions:

touch /var/log/{btmp,lastlog,wtmp}
chgrp utmp /var/log/lastlog
chmod 664  /var/log/lastlog
chmod 600  /var/log/btmp

Build a few Packages Manually

The following packages must be built manually: pkgtools, which and tar 1.13.

First, cd to the directory the sources are in:

cd /sources

Install pkgtools

Install pkgtools to /tools. Note: substitute the version of pkgtools you downloaded in chapter 3 for “c2.0.0”

tar -xzf c2.0.0.tar.gz
cd pkgtools-*
make DESTDIR=/tools install

Install Tar 1.13

Install Tar 1.13 to /tools:

cd ..
tar -xzf tar-1.13.tar.gz
cd tar-1.13
./configure --prefix=/tools --program-suffix=-1.13
make
make install

"Install" Which

The full fledged version of Which is not necessary at this point, so we will install a minimal script to emulate the essential functionality:

cat > /tools/bin/which << "EOF"
#!/bin/bash
type -pa "$@" | head -n 1 ; exit ${PIPESTATUS[0]}
EOF
chmod -v 755 /tools/bin/which

Building the Phase 2 Bootstrap

It is necessary to build a small set of bootstrap packages manually before building the rest of phase 2.

First, create a dummy gpg to use as a workaround. We will not install gpg until later, but certain packages will not build without it installed. This is safe to do since we already verified the integrity of the downloaded sources when we initially downloaded them in chapter 3. It will be placed in /tools to ensure that it is deleted at the end of the build and will not leak into the final system.

cat > /tools/bin/gpg << EOF
#!/bin/bash

exit 0
EOF
chmod 755 /tools/bin/gpg

Build and install them:

/usr/ports/utilities/tools/portmake install linux-headers
/usr/ports/utilities/tools/portmake install man-pages
/usr/ports/utilities/tools/portmake install glibc

Run the test suites for glibc. This step is considered critical and should not be skipped. A few test failures is nothing to be alarmed about; it is usually cause for alarm only if a large percentage of the test fail.

cd /tmp/glibc-*/src/glibc-*/build
make -j $(nproc) check

Adjusting the Toolchain

Perform the Adjustment

First, backup the /tools linker, and replace it with the adjusted linker we made in chapter 5. We'll also create a link to its counterpart in /tools/$(uname -m)-pc-linux-gnu/bin:

mv -v /tools/bin/{ld,ld-old}
mv -v /tools/$(uname -m)-pc-linux-gnu/bin/{ld,ld-old}
mv -v /tools/bin/{ld-new,ld}
ln -sv /tools/bin/ld /tools/$(uname -m)-pc-linux-gnu/bin/ld

Next, amend the GCC specs file so that it points to the new dynamic linker. Simply deleting all instances of “/tools” should leave us with the correct path to the dynamic linker. Also adjust the specs file so that GCC knows where to find the correct headers and Glibc start files. A sed command accomplishes this:

case $(uname -m) in
  x86_64) export LIBDIRSUFFIX=64 ;;
esac

gcc -dumpspecs | sed -e 's@/tools@@g'                   \
  -e '/\*startfile_prefix_spec:/{n;s@.*@/usr/lib${LIBDIRSUFFIX}/ @}' \
  -e '/\*cpp:/{n;s@$@ -isystem /usr/include@}' >      \
  `dirname $(gcc --print-libgcc-file-name)`/specs

It is a good idea to visually inspect the specs file to verify the intended change was actually made.

Sanity Check the Toolchain

It is imperative at this point to ensure that the basic functions (compiling and linking) of the adjusted toolchain are working as expected. To do this, perform the following sanity checks:

cd /tmp
echo 'int main(){}' > dummy.c
cc dummy.c -v -Wl,--verbose &> dummy.log
readelf -l a.out | grep ': /lib'

There should be no errors, and the output of the last command will be (allowing for platform-specific differences in dynamic linker name):

[Requesting program interpreter: /lib64/ld-linux-x86-64.so.2]

Now make sure that we're setup to use the correct start files:

grep -o '/usr/lib.*/crt[1in].*succeeded' dummy.log

The output of the last command should be (allowing for platform specific variations):

/usr/lib/../lib/crt1.o succeeded
/usr/lib/../lib/crti.o succeeded
/usr/lib/../lib/crtn.o succeeded

Verify that the compiler is searching for the correct header files:

grep -B1 '^ /usr/include' dummy.log

This command should return the following output:

#include <...> search starts here:
 /usr/include

Next, verify that the new linker is being used with the correct search paths:

grep 'SEARCH.*/usr/lib' dummy.log |sed 's|; |\n|g'

References to paths that have components with '-linux-gnu' should be ignored, but otherwise the output of the last command should be:

SEARCH_DIR("/usr/lib")
SEARCH_DIR("/lib")

Next make sure that we're using the correct libc:

grep "/lib.*/libc.so.6 " dummy.log

The output of the last command should be:

attempt to open /lib/libc.so.6 succeeded

Lastly, make sure GCC is using the correct dynamic linker:

grep found dummy.log

The output of the last command should be (allowing for platform-specific differences in dynamic linker name):

found ld-linux-x86-64.so.2 at /lib/ld-linux-x86-64.so.2

Building the Remainder of Phase 2

Now it is time to build the rest of phase 2. Fortunately, the remainder of this process can be automated with portmake. Run the following (note that this command will take a long time):

/usr/ports/utilities/tools/portmake install phase2

Chapter 7 - Building Phase 3

This phase is akin to building select packages from the Beyond Linux from Scratch Book.

cucumber_linux_from_scratch.txt · Last modified: 2018/07/07 12:26 by z5t1