Typographical Conventions

To make things easy to follow, a number of conventions are used throughout the book. Here are some examples:

./configure --prefix=/usr

install-info: unknown option
`--dir-file=/mnt/lfs/usr/info/dir'

Emphasis

https://www.linuxfromscratch.org/

SeaMonkey-2.53.16

cat > $LFS/etc/group << "EOF"
root:x:0:
bin:x:1:
......
EOF

<REPLACED TEXT>

root

Conventions Used for Package Dependencies

When new packages are created, the software's authors depend on prior work. In order to build a package in BTFS, these dependencies must be built before the desired package can be compiled. For each package, prerequisites are listed in one or more separate sections: Required, Recommended, and Optional.

Required Dependencies

These dependencies are the bare minimum needed to build the package. Packages in TFS, and the required dependencies of these required packages, are omitted from this list. Always remember to check for nested dependencies. If a dependency is said to be “runtime”, then it is not needed for building the package, but only to use it after installation.

Recommended Dependencies

These are dependencies the BTFS editors have determined are important to give the package reasonable capabilities. If a recommended dependency is not said to be “runtime”, package installation instructions assume it is installed. If it is not installed, the instructions may require modification, to accommodate the missing package. A recommended “runtime” dependency does not need to be installed before building the package, but must be built afterwards for running the package with reasonable capabilities.

Optional Dependencies

These are dependencies the package may use. Integration of optional dependencies may be automatic by the package, or additional steps not presented by BTFS may be necessary. Optional dependencies are sometimes listed without explicit BTFS instructions. In this case you must determine how to perform the installation yourself.

Conventions Used for Kernel Configuration Options

Some packages require specific kernel configuration options. The general layout for these looks like this:

Master section --->
  Subsection --->
    [*]     Required parameter                     [CONFIG_REQU_PAR]
    <*>     Required parameter (not as module)     [CONFIG_REQU_PAR_NMOD]
    <*/M>   Required parameter (could be a module) [CONFIG_REQU_PAR_MOD]
    <*/M/ > Optional parameter                     [CONFIG_OPT_PAR]
    [ ] Incompatible parameter                     [CONFIG_INCOMP_PAR]
    < > Incompatible parameter (even as module)    [CONFIG_INCOMP_PAR_MOD]

[CONFIG_...] on the right gives the name of the option, so you can easily check whether it is set in your .config file. The meaning of the various entries is:

Note that, depending on other selections, the angle brackets (<>) in the configuration menu may appear as braces ({}) if the option cannot be unselected, or even as dashes (-*- or -M-), when the choice is imposed. The help text describing the option specifies the other selections on which this option relies, and how those other selections are set.

SBU values in BTFS

As in TFS, each package in BTFS has a build time listed in Standard Build Units (SBUs). These times are relative to the time it took to build binutils in TFS, and are intended to provide some insight into how long it will take to build a package. Most times listed are for a single processor or core to build the package. In some cases, large, long running builds tested on multi-core systems have SBU times listed with comments such as '(parallelism=4)'. These values indicate testing was done using multiple cores. Note that while this speeds up the build on systems with the appropriate hardware, the speedup is not linear and to some extent depends on the individual package and the specific hardware used.

For packages which use ninja (i.e., anything using meson) or rust, by default all cores are used; similar comments will be seen on such packages even when the build time is minimal.

Where even a parallel build takes more than 15 SBU, on certain machines the time may be considerably greater even when the build does not use swap. In particular, different micro-architectures will build some files at different relative speeds, and this can introduce delays when certain make targets wait for another file to be created. Where a large build uses a lot of C++ files, processors with Simultaneous Multi Threading will share the Floating Point Unit and can take 45% longer than when using four 'prime' cores (measured on an intel i7 using taskset and keeping the other cores idle).

Some packages do not support parallel builds; for these, the make command must specify -j1. Packages that are known to impose such limits are so marked in the text.

Things to Check Prior to Asking

Before asking for help, you should review the following items:

Things to Mention

Apart from a brief explanation of the problem you're having, the essential things to include in your request are:

(Note that saying that you've deviated from the book doesn't mean that we won't help you. It'll just help us to see other possible causes of your problem.)

Expect guidance instead of specific instructions. If you are instructed to read something, please do so. It generally implies that the answer was way too obvious and that the question would not have been asked if a little research was done prior to asking. The volunteers in the mailing list prefer not to be used as an alternative to doing reasonable research on your end. In addition, the quality of your experience with BLFS is also greatly enhanced by this research, and the quality of volunteers is enhanced because they don't feel that their time has been abused, so they are far more likely to participate.

An excellent article on asking for help on the Internet in general has been written by Eric S. Raymond. It is available online at http://www.catb.org/~esr/faqs/smart-questions.html. Read and follow the hints in that document and you are much more likely to get a response to start with and also to get the help you actually need.

Current Editors

Contributors and Past Editors

The list of contributors is far too large to provide detailed information about the contributions for each contributor. Over the years, the following individuals have provided significant inputs to the book:

General Acknowledgments

Building Software as an Unprivileged (non-root) User

The golden rule of Unix System Administration is to use your superpowers only when necessary. Hence, BTFS recommends that you build software as an unprivileged user and only become the root user when installing the software. This philosophy is followed in all the packages in this book. Unless otherwise specified, all instructions should be executed as an unprivileged user. The book will advise you on instructions that need root privileges.

Unpacking the Software

If a file is in .tar format and compressed, it is unpacked by running one of the following commands:

tar -xvf filename.tar.gz
tar -xvf filename.tgz
tar -xvf filename.tar.Z
tar -xvf filename.tar.bz2

You can also use a slightly different method:

bzcat filename.tar.bz2 | tar -xv

Finally, you sometimes need to be able to unpack patches which are generally not in .tar format. The best way to do this is to copy the patch file to the parent of the 'build' directory and then run one of the following commands depending on whether the file is a .gz or .bz2 file:

gunzip -v patchname.gz
bunzip2 -v patchname.bz2

Verifying File Integrity

Generally, to verify that the downloaded file is complete, many package maintainers also distribute md5sums of the files. To verify the md5sum of the downloaded files, download both the file and the corresponding md5sum file to the same directory (preferably from different on-line locations), and (assuming file.md5sum is the md5sum file downloaded) run the following command:

md5sum -c file.md5sum

If there are any errors, they will be reported. Note that the BTFS book includes md5sums for all the source files also. To use the BTFS supplied md5sums, you can create a file.md5sum (place the md5sum data and the exact name of the downloaded file on the same line of a file, separated by white space) and run the command shown above. Alternately, simply run the command shown below and compare the output to the md5sum data shown in the BTFS book.

md5sum <name_of_downloaded_file>

MD5 is not cryptographically secure, so the md5sums are only provided for detecting unmalicious changes to the file content. For example, an error or truncation introduced during network transfer, or a “stealth” update to the package from the upstream (updating the content of a released tarball instead of making a new release properly).

There is no “100%” secure way to make sure the genuity of the source files. Assuming the upstream is managing their website correctly (the private key is not leaked and the domain is not hijacked), and the trust anchors have been set up correctly using make-ca-1.13 on the BTFS system, we can reasonably trust download URLs to the upstream official website with https protocol. Note that BLFS book itself is published on a website with https, so you should already have some confidence in https protocol or you wouldn't trust the book content.

If the package is downloaded from an unofficial location (for example a local mirror), checksums generated by cryptographically secure digest algorithms (for example SHA256) can be used to verify the genuity of the package. Download the checksum file from the upstream official website (or somewhere you can trust) and compare the checksum of the package from unofficial location with it. For example, SHA256 checksum can be checked with the command:

sha256sum -c file.sha256sum

If GnuPG-2.4.0 is installed, you can also verify the genuity of the package with a GPG signature. Import the upstream GPG public key with:

gpg --recv-key keyID

keyID should be replaced with the key ID from somewhere you can trust (for example, copy it from the upstream official website using https). Now you can verify the signature with:

gpg --recv-key file.sig file

The advantage of GnuPG signature is, once you imported a public key which can be trusted, you can download both the package and its signature from the same unofficial location and verify them with the public key. So you won't need to connect to the official upstream website to retrieve a checksum for each new release. You only need to update the public key if it's expired or revoked.

Creating Log Files During Installation

For larger packages, it is convenient to create log files instead of staring at the screen hoping to catch a particular error or warning. Log files are also useful for debugging and keeping records. The following command allows you to create an installation log. Replace <command> with the command you intend to execute.

( <command> 2>&1 | tee compile.log && exit $PIPESTATUS )

2>&1 redirects error messages to the same location as standard output. The tee command allows viewing of the output while logging the results to a file. The parentheses around the command run the entire command in a subshell and finally the exit $PIPESTATUS command ensures the result of the <command> is returned as the result and not the result of the tee command.

Using Multiple Processors

For many modern systems with multiple processors (or cores) the compilation time for a package can be reduced by performing a "parallel make" by either setting an environment variable or telling the make program how many processors are available. For instance, a Core2Duo can support two simultaneous processes with:

export MAKEFLAGS='-j2'

or just building with:

make -j2

If you have applied the optional sed when building ninja in TFS, you can use:

export NINJAJOBS=2

when a package uses ninja, or just:

ninja -j2

but for ninja, the default number of jobs is <N>+2, where <N> is the number of processors available, so that using the above commands is rather for limiting the number of jobs (see below for why this could be necessary).

Generally the number of processes should not exceed the number of cores supported by the CPU. To list the processors on your system, issue: grep processor /proc/cpuinfo.

In some cases, using multiple processes may result in a 'race' condition where the success of the build depends on the order of the commands run by the make program. For instance, if an executable needs File A and File B, attempting to link the program before one of the dependent components is available will result in a failure. This condition usually arises because the upstream developer has not properly designated all the prerequisites needed to accomplish a step in the Makefile.

If this occurs, the best way to proceed is to drop back to a single processor build. Adding '-j1' to a make command will override the similar setting in the MAKEFLAGS environment variable.

Automated Building Procedures

There are times when automating the building of a package can come in handy. Everyone has their own reasons for wanting to automate building, and everyone goes about it in their own way. Creating Makefiles, Bash scripts, Perl scripts or simply a list of commands used to cut and paste are just some of the methods you can use to automate building BTFS packages. Detailing how and providing examples of the many ways you can automate the building of packages is beyond the scope of this section. This section will expose you to using file redirection and the yes command to help provide ideas on how to automate your builds.

File Redirection to Automate Input

You will find times throughout your BTFS journey when you will come across a package that has a command prompting you for information. This information might be configuration details, a directory path, or a response to a license agreement. This can present a challenge to automate the building of that package. Occasionally, you will be prompted for different information in a series of questions. One method to automate this type of scenario requires putting the desired responses in a file and using redirection so that the program uses the data in the file as the answers to the questions.

Building the CUPS package is a good example of how redirecting a file as input to prompts can help you automate the build. If you run the test suite, you are asked to respond to a series of questions regarding the type of test to run and if you have any auxiliary programs the test can use. You can create a file with your responses, one response per line, and use a command similar to the one shown below to automate running the test suite:

make check < ../cups-1.1.23-testsuite_parms

This effectively makes the test suite use the responses in the file as the input to the questions. Occasionally you may end up doing a bit of trial and error determining the exact format of your input file for some things, but once figured out and documented you can use this to automate building the package.

Using yes to Automate Input

Sometimes you will only need to provide one response, or provide the same response to many prompts. For these instances, the yes command works really well. The yes command can be used to provide a response (the same one) to one or more instances of questions. It can be used to simulate pressing just the Enter key, entering the Y key or entering a string of text. Perhaps the easiest way to show its use is in an example.

First, create a short Bash script by entering the following commands:

cat > blfs-yes-test1 << "EOF"
#!/bin/bash

echo -n -e "\n\nPlease type something (or nothing) and press Enter ---> "

read A_STRING

if test "$A_STRING" = ""; then A_STRING="Just the Enter key was pressed"
else A_STRING="You entered '$A_STRING'"
fi

echo -e "\n\n$A_STRING\n\n"
EOF
chmod 755 blfs-yes-test1

Now run the script by issuing ./blfs-yes-test1 from the command line. It will wait for a response, which can be anything (or nothing) followed by the Enter key. After entering something, the result will be echoed to the screen. Now use the yes command to automate the entering of a response:

yes | ./blfs-yes-test1

Notice that piping yes by itself to the script results in y being passed to the script. Now try it with a string of text:

yes 'This is some text' | ./blfs-yes-test1

The exact string was used as the response to the script. Finally, try it using an empty (null) string:

yes '' | ./blfs-yes-test1

Notice this results in passing just the press of the Enter key to the script. This is useful for times when the default answer to the prompt is sufficient. This syntax is used in the Net-tools instructions to accept all the defaults to the many prompts during the configuration step. You may now remove the test script, if desired.

File Redirection to Automate Output

In order to automate the building of some packages, especially those that require you to read a license agreement one page at a time, requires using a method that avoids having to press a key to display each page. Redirecting the output to a file can be used in these instances to assist with the automation. The previous section on this page touched on creating log files of the build output. The redirection method shown there used the tee command to redirect output to a file while also displaying the output to the screen. Here, the output will only be sent to a file.

Again, the easiest way to demonstrate the technique is to show an example. First, issue the command:

ls -l /usr/bin | more

Of course, you'll be required to view the output one page at a time because the more filter was used. Now try the same command, but this time redirect the output to a file. The special file /dev/null can be used instead of the filename shown, but you will have no log file to examine:

ls -l /usr/bin | more > redirect_test.log 2>&1

Notice that this time the command immediately returned to the shell prompt without having to page through the output. You may now remove the log file.

The last example will use the yes command in combination with output redirection to bypass having to page through the output and then provide a y to a prompt. This technique could be used in instances when otherwise you would have to page through the output of a file (such as a license agreement) and then answer the question of “do you accept the above?”. For this example, another short Bash script is required:

cat > blfs-yes-test2 << "EOF"
#!/bin/bash

ls -l /usr/bin | more

echo -n -e "\n\nDid you enjoy reading this? (y,n) "

read A_STRING

if test "$A_STRING" = "y"; then A_STRING="You entered the 'y' key"
else A_STRING="You did NOT enter the 'y' key"
fi

echo -e "\n\n$A_STRING\n\n"
EOF
chmod 755 blfs-yes-test2

This script can be used to simulate a program that requires you to read a license agreement, then respond appropriately to accept the agreement before the program will install anything. First, run the script without any automation techniques by issuing ./blfs-yes-test2.

Now issue the following command which uses two automation techniques, making it suitable for use in an automated build script:

yes | ./blfs-yes-test2 > blfs-yes-test2.log 2>&1

If desired, issue tail blfs-yes-test2.log to see the end of the paged output, and confirmation that y was passed through to the script. Once satisfied that it works as it should, you may remove the script and log file.

Finally, keep in mind that there are many ways to automate and/or script the build commands. There is not a single “correct” way to do it. Your imagination is the only limit.

Dependencies

For each package described, BTFS lists the known dependencies. These are listed under several headings, whose meaning is as follows:

Using the Most Current Package Sources

On occasion you may run into a situation in the book when a package will not build or work properly. Though the Editors attempt to ensure that every package in the book builds and works properly, sometimes a package has been overlooked or was not tested with this particular version of BTFS.

If you discover that a package will not build or work properly, you should see if there is a more current version of the package. Typically this means you go to the maintainer's web site and download the most current tarball and attempt to build the package. If you cannot determine the maintainer's web site by looking at the download URLs, use Google and query the package's name. For example, in the Google search bar type: 'package_name download' (omit the quotes) or something similar. Sometimes typing: 'package_name home page' will result in you finding the maintainer's web site.

Stripping One More Time

In TFS, stripping of debugging symbols and unneeded symbol table entries was discussed a couple of times. When building BTFS packages, there are generally no special instructions that discuss stripping again. Stripping can be done while installing a package, or afterwards.

Stripping while Installing a Package

There are several ways to strip executables installed by a package. They depend on the build system used (see below the section about build systems), so only some generalities can be listed here:

Stripping Installed Executables

The strip utility changes files in place, which may break anything using it if it is loaded in memory. Note that if a file is in use but just removed from the disk (i.e. not overwritten nor modified), this is not a problem since the kernel can use “deleted” files. Look at /proc/*/maps and it is likely that you'll see some (deleted) entries. The mv just removes the destination file from the directory but does not touch its content, so that it satisfies the condition for the kernel to use the old (deleted) file. But this approach can detach hard links into duplicated copies, causing a bloat which is obviously unwanted as we are stripping to reduce system size. If two files in a same file system share the same inode number, they are hard links to each other and we should reconstruct the link. The script below is just an example. It should be run as the root user:

cat > /usr/sbin/strip-all.sh << "EOF"
#!/usr/bin/bash

if [ $EUID -ne 0 ]; then
  echo "Need to be root"
  exit 1
fi

last_fs_inode=
last_file=

{ find /usr/lib -type f -name '*.so*' ! -name '*dbg'
  find /usr/lib -type f -name '*.a'
  find /usr/{bin,sbin,libexec} -type f
} | xargs stat -c '%m %i %n' | sort | while read fs inode file; do
       if ! readelf -h $file >/dev/null 2>&1; then continue; fi
       if file $file | grep --quiet --invert-match 'not stripped'; then continue; fi

       if [ "$fs $inode" = "$last_fs_inode" ]; then
         ln -f $last_file $file;
         continue;
       fi

       cp --preserve $file    ${file}.tmp
       strip --strip-unneeded ${file}.tmp
       mv ${file}.tmp $file

       last_fs_inode="$fs $inode"
       last_file=$file
done
EOF
chmod 744 /usr/sbin/strip-all.sh

If you install programs in other directories such as /opt or /usr/local, you may want to strip the files there too. Just add other directories to scan in the compound list of find commands between the braces.

For more information on stripping, see https://www.technovelty.org/linux/stripping-shared-libraries.html.

Working with different build systems

There are now three different build systems in common use for converting C or C++ source code into compiled programs or libraries and their details (particularly, finding out about available options and their default values) differ. It may be easiest to understand the issues caused by some choices (typically slow execution or unexpected use of, or omission of, optimizatons) by starting with the CFLAGS and CXXFLAGS environment variables. There are also some programs which use rust.

Most TFS and BTFS builders are probably aware of the basics of CFLAGS and CXXFLAGS for altering how a program is compiled. Typically, some form of optimization is used by upstream developers (-O2 or -O3), sometimes with the creation of debug symbols (-g), as defaults.

If there are contradictory flags (e.g. multiple different -O values), the last value will be used. Sometimes this means that flags specified in environment variables will be picked up before values hardcoded in the Makefile, and therefore ignored. For example, where a user specifies '-O2' and that is followed by '-O3' the build will use '-O3'.

There are various other things which can be passed in CFLAGS or CXXFLAGS, such as forcing compilation for a specific microarchitecture (e.g. -march=amdfam10, -march=native) or specifying a specific standard for C or C++ (-std=c++17 for example). But one thing which has now come to light is that programmers might include debug assertions in their code, expecting them to be disabled in releases by using -DNDEBUG. Specifically, if Mesa-23.1.8 is built with these assertions enabled, some activities such as loading levels of games can take extremely long times, even on high-class video cards.

Autotools with Make

This combination is often described as 'CMMI' (configure, make, make install) and is used here to also cover the few packages which have a configure script that is not generated by autotools.

Sometimes running ./configure --help will produce useful options about switches which might be used. At other times, after looking at the output from configure you may need to look at the details of the script to find out what it was actually searching for.

Many configure scripts will pick up any CFLAGS or CXXFLAGS from the environment, but CMMI packages vary about how these will be mixed with any flags which would otherwise be used (variously: ignored, used to replace the programmer's suggestion, used before the programmer's suggestion, or used after the programmer's suggestion).

In most CMMI packages, running 'make' will list each command and run it, interspersed with any warnings. But some packages try to be 'silent' and only show which file they are compiling or linking instead of showing the command line. If you need to inspect the command, either because of an error, or just to see what options and flags are being used, adding 'V=1' to the make invocation may help.

CMake

CMake works in a very different way, and it has two backends which can be used on BTFS: 'make' and 'ninja'. The default backend is make, but ninja can be faster on large packages with multiple processors. To use ninja, specify '-G Ninja' in the cmake command. However, there are some packages which create fatal errors in their ninja files but build successfully using the default of Unix Makefiles.

The hardest part of using CMake is knowing what options you might wish to specify. The only way to get a list of what the package knows about is to run cmake -LAH and look at the output for that default configuration.

Perhaps the most-important thing about CMake is that it has a variety of CMAKE_BUILD_TYPE values, and these affect the flags. The default is that this is not set and no flags are generated. Any CFLAGS or CXXFLAGS in the environment will be used. If the programmer has coded any debug assertions, those will be enabled unless -DNDEBUG is used. The following CMAKE_BUILD_TYPE values will generate the flags shown, and these will come after any flags in the environment and therefore take precedence.

CMake tries to produce quiet builds. To see the details of the commands which are being run, use make VERBOSE=1 or ninja -v.

By default, CMake treats file installation differently from the other build systems: if a file already exists and is not newer than a file that would overwrite it, then the file is not installed. This may be a problem if a user wants to record which file belongs to a package, either using LD_PRELOAD, or by listing files newer than a timestamp. The default can be changed by setting the variable CMAKE_INSTALL_ALWAYS to 1 in the environment, for example by export'ing it.

Meson

Meson has some similarities to CMake, but many differences. To get details of the defines that you may wish to change you can look at meson_options.txt which is usually in the top-level directory.

If you have already configured the package by running meson and now wish to change one or more settings, you can either remove the build directory, recreate it, and use the altered options, or within the build directory run meson configure, e.g. to set an option:

meson configure -D<some_option>=true

If you do that, the file meson-private/cmd_line.txt will show the last commands which were used.

Meson provides the following buildtype values, and the flags they enable come after any flags supplied in the environment and therefore take precedence.

Although the 'release' buildtype is described as enabling -DNDEBUG, and all CMake Release builds pass that, it has so far only been observed (in verbose builds) for Mesa-23.1.8. That suggests that it might only be used when there are debug assertions present.

The -DNDEBUG flag can also be provided by passing -Db_ndebug=true.

To see the details of the commands which are being run in a package using meson, use 'ninja -v'.

Rustc and Cargo

Most released rustc programs are provided as crates (source tarballs) which will query a server to check current versions of dependencies and then download them as necessary. These packages are built using cargo --release. In theory, you can manipulate the RUSTFLAGS to change the optimize-level (default is 3, like -O3, e.g. -Copt-level=3) or to force it to build for the machine it is being compiled on, using -Ctarget-cpu=native but in practice this seems to make no significant difference.

If you find an interesting rustc program which is only provided as unpackaged source, you should at least specify RUSTFLAGS=-Copt-level=2 otherwise it will do an unoptimized compile with debug info and run much slower.

The rust developers seem to assume that everyone will compile on a machine dedicated to producing builds, so by default all CPUs are used. This can often be worked around, either by exporting CARGO_BUILD_JOBS=<N> or passing --jobs <N> to cargo. For compiling rustc itself, specifying --jobs <N> on invocations of x.py (together with the CARGO_BUILD_JOBS environment variable, which looks like a "belt and braces" approach but seems to be necessary) mostly works. The exception is running the tests when building rustc, some of them will nevertheless use all online CPUs, at least as of rustc-1.42.0.

Optimizing the build

Many people will prefer to optimize compiles as they see fit, by providing CFLAGS or CXXFLAGS. For an introduction to the options available with gcc and g++ see https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html and https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html and info gcc.

Some packages default to '-O2 -g', others to '-O3 -g', and if CFLAGS or CXXFLAGS are supplied they might be added to the package's defaults, replace the package's defaults, or even be ignored. There are details on some desktop packages which were mostly current in April 2019 at https://www.linuxfromscratch.org/~ken/tuning/ - in particular, README.txt, tuning-1-packages-and-notes.txt, and tuning-notes-2B.txt. The particular thing to remember is that if you want to try some of the more interesting flags you may need to force verbose builds to confirm what is being used.

Clearly, if you are optimizing your own program you can spend time to profile it and perhaps recode some of it if it is too slow. But for building a whole system that approach is impractical. In general, -O3 usually produces faster programs than -O2. Specifying -march=native is also beneficial, but means that you cannot move the binaries to an incompatible machine - this can also apply to newer machines, not just to older machines. For example programs compiled for 'amdfam10' run on old Phenoms, Kaveris, and Ryzens : but programs compiled for a Kaveri will not run on a Ryzen because certain op-codes are not present. Similarly, if you build for a Haswell not everything will run on a SandyBridge.

There are also various other options which some people claim are beneficial. At worst, you get to recompile and test, and then discover that in your usage the options do not provide a benefit.

If building Perl or Python modules, or Qt packages which use qmake, in general the CFLAGS and CXXFLAGS used are those which were used by those 'parent' packages.

Options for hardening the build

Even on desktop systems, there are still a lot of exploitable vulnerabilities. For many of these, the attack comes via javascript in a browser. Often, a series of vulnerabilities are used to gain access to data (or sometimes to pwn, i.e. own, the machine and install rootkits). Most commercial distros will apply various hardening measures.

In the past, there was Hardened LFS where gcc (a much older version) was forced to use hardening (with options to turn some of it off on a per-package basis). The current TFS and BTFS books are carrying forward a part of its spirit by enabling PIE (-fPIE -pie) and SSP (-fstack-protector-strong) as the defaults for GCC and clang. What is being covered here is different - first you have to make sure that the package is indeed using your added flags and not over-riding them.

For hardening options which are reasonably cheap, there is some discussion in the 'tuning' link above (occasionally, one or more of these options might be inappropriate for a package). These options are -D_FORTIFY_SOURCE=2 and (for C++) -D_GLIBCXX_ASSERTIONS. On modern machines these should only have a little impact on how fast things run, and often they will not be noticeable.

The main distros use much more, such as RELRO (Relocation Read Only) and perhaps -fstack-clash-protection. You may also encounter the so-called “userspace retpoline” (-mindirect-branch=thunk etc.) which is the equivalent of the spectre mitigations applied to the linux kernel in late 2018. The kernel mitigations caused a lot of complaints about lost performance, if you have a production server you might wish to consider testing that, along with the other available options, to see if performance is still sufficient.

Whilst gcc has many hardening options, clang/LLVM's strengths lie elsewhere. Some options which gcc provides are said to be less effective in clang/LLVM.

The Needed Encoding is Not a Valid Option in the Program

Severity: Critical

Some programs require the user to specify the character encoding for their input or output data and present only a limited choice of encodings. This is the case for the -X option in Enscript-1.6.6, the -input-charset option in unpatched Cdrtools-3.02a09, and the character sets offered for display in the menu of Links-2.29. If the required encoding is not in the list, the program usually becomes completely unusable. For non-interactive programs, it may be possible to work around this by converting the document to a supported input character set before submitting to the program.

A solution to this type of problem is to implement the necessary support for the missing encoding as a patch to the original program or to find a replacement.

The Program Assumes the Locale-Based Encoding of External Documents

Severity: High for non-text documents, low for text documents

Some programs, nano-7.2 or JOE-4.6 for example, assume that documents are always in the encoding implied by the current locale. While this assumption may be valid for the user-created documents, it is not safe for external ones. When this assumption fails, non-ASCII characters are displayed incorrectly, and the document may become unreadable.

If the external document is entirely text based, it can be converted to the current locale encoding using the iconv program.

For documents that are not text-based, this is not possible. In fact, the assumption made in the program may be completely invalid for documents where the Microsoft Windows operating system has set de facto standards. An example of this problem is ID3v1 tags in MP3 files (see the BLFS Wiki ID3v1Coding page for more details). For these cases, the only solution is to find a replacement program that doesn't have the issue (e.g., one that will allow you to specify the assumed document encoding).

Among BTFS packages, this problem applies to nano-7.2, JOE-4.6, and all media players except Audacious-4.3.

Another problem in this category is when someone cannot read the documents you've sent them because their operating system is set up to handle character encodings differently. This can happen often when the other person is using Microsoft Windows, which only provides one character encoding for a given country. For example, this causes problems with UTF-8 encoded TeX documents created in Linux. On Windows, most applications will assume that these documents have been created using the default Windows 8-bit encoding.

In extreme cases, Windows encoding compatibility issues may be solved only by running Windows programs under Wine.

The Program Uses or Creates Filenames in the Wrong Encoding

Severity: Critical

The POSIX standard mandates that the filename encoding is the encoding implied by the current LC_CTYPE locale category. This information is well-hidden on the page which specifies the behavior of Tar and Cpio programs. Some programs get it wrong by default (or simply don't have enough information to get it right). The result is that they create filenames which are not subsequently shown correctly by ls, or they refuse to accept filenames that ls shows properly. For the GLib-2.76.2 library, the problem can be corrected by setting the G_FILENAME_ENCODING environment variable to the special "@locale" value. Glib2 based programs that don't respect that environment variable are buggy.

The Zip-3.0 and UnZip-6.0 have this problem because they hard-code the expected filename encoding. UnZip contains a hard-coded conversion table between the CP850 (DOS) and ISO-8859-1 (UNIX) encodings and uses this table when extracting archives created under DOS or Microsoft Windows. However, this assumption only works for those in the US and not for anyone using a UTF-8 locale. Non-ASCII characters will be mangled in the extracted filenames.

The general rule for avoiding this class of problems is to avoid installing broken programs. If this is impossible, the convmv command-line tool can be used to fix filenames created by these broken programs, or intentionally mangle the existing filenames to meet the broken expectations of such programs.

In other cases, a similar problem is caused by importing filenames from a system using a different locale with a tool that is not locale-aware (e.g. OpenSSH). In order to avoid mangling non-ASCII characters when transferring files to a system with a different locale, any of the following methods can be used:

The last four methods work because the filenames are automatically converted from the sender's locale to UNICODE and stored or sent in this form. They are then transparently converted from UNICODE to the recipient's locale encoding.

The Program Breaks Multibyte Characters or Doesn't Count Character Cells Correctly

Severity: High or critical

Many programs were written in an older era where multibyte locales were not common. Such programs assume that C "char" data type, which is one byte, can be used to store single characters. Further, they assume that any sequence of characters is a valid string and that every character occupies a single character cell. Such assumptions completely break in UTF-8 locales. The visible manifestation is that the program truncates strings prematurely (i.e., at 80 bytes instead of 80 characters). Terminal-based programs don't place the cursor correctly on the screen, don't react to the "Backspace" key by erasing one character, and leave junk characters around when updating the screen, usually turning the screen into a complete mess.

Fixing this kind of problems is a tedious task from a programmer's point of view, like all other cases of retrofitting new concepts into the old flawed design. In this case, one has to redesign all data structures in order to accommodate to the fact that a complete character may span a variable number of "char"s (or switch to wchar_t and convert as needed). Also, for every call to the "strlen" and similar functions, find out whether a number of bytes, a number of characters, or the width of the string was really meant. Sometimes it is faster to write a program with the same functionality from scratch.

Among BTFS packages, this problem applies to xine-ui-0.99.14 and all the shells.

The Package Installs Manual Pages in Incorrect or Non-Displayable Encoding

Severity: Low

TFS expects that manual pages are in the language-specific (usually 8-bit) encoding, as specified on the TFS Man DB page. However, some packages install translated manual pages in UTF-8 encoding (e.g., Shadow, already dealt with), or manual pages in languages not in the table. Not all BTFS packages have been audited for conformance with the requirements put in TFS (the large majority have been checked, and fixes placed in the book for packages known to install non-conforming manual pages). If you find a manual page installed by any of BTFS packages that is obviously in the wrong encoding, please remove or convert it as needed, and report this to BTFS team as a bug.

You can easily check your system for any non-conforming manual pages by copying the following short shell script to some accessible location,

#!/bin/sh
# Begin checkman.sh
# Usage: find /usr/share/man -type f | xargs checkman.sh
for a in "$@"
do
    # echo "Checking $a..."
    # Pure-ASCII manual page (possibly except comments) is OK
    grep -v '.\\"' "$a" | iconv -f US-ASCII -t US-ASCII >/dev/null 2>&1 \
        && continue
    # Non-UTF-8 manual page is OK
    iconv -f UTF-8 -t UTF-8 "$a" >/dev/null 2>&1 || continue
    # Found a UTF-8 manual page, bad.
    echo "UTF-8 manual page: $a" >&2
done
# End checkman.sh

and then issuing the following command (modify the command below if the checkman.sh script is not in your PATH environment variable):

find /usr/share/man -type f | xargs checkman.sh

Note that if you have manual pages installed in any location other than /usr/share/man (e.g., /usr/local/share/man), you must modify the above command to include this additional location.

Decent Rescue Boot Device Needs

This section is really about creating a rescue device. As the name rescue implies, the host system has a problem, often lost partition information or corrupted file systems, that prevents it from booting and/or operating normally. For this reason, you must not depend on resources from the host being "rescued". To presume that any given partition or hard drive will be available is a risky presumption.

In a modern system, there are many devices that can be used as a rescue device: floppy, cdrom, usb drive, or even a network card. Which one you use depends on your hardware and your BIOS. In the past, a rescue device was thought to be a floppy disk. Today, many systems do not even have a floppy drive.

Building a complete rescue device is a challenging task. In many ways, it is equivalent to building an entire LFS system. In addition, it would be a repetition of information already available. For these reasons, the procedures for a rescue device image are not presented here.

Creating a Rescue Floppy

The software of today's systems has grown large. Linux 2.6 no longer supports booting directly from a floppy. In spite of this, there are solutions available using older versions of Linux. One of the best is Tom's Root/Boot Disk available at http://www.toms.net/rb/. This will provide a minimal Linux system on a single floppy disk and provides the ability to customize the contents of your disk if necessary.

Creating a Bootable CD-ROM

There are several sources that can be used for a rescue CD-ROM. Just about any commercial distribution's installation CD-ROMs or DVDs will work. These include RedHat, Ubuntu, and SuSE. One very popular option is Knoppix.

Also, the LFS Community has developed its own LiveCD available at https://www.linuxfromscratch.org/livecd/. This LiveCD, is no longer capable of building an entire LFS/BLFS system, but is still a good rescue CD-ROM. If you download the ISO image, use xorriso to copy the image to a CD-ROM.

The instructions for using GRUB2 to make a custom rescue CD-ROM are also available in LFS Chapter 10.

Creating a Bootable USB Drive

A USB Pen drive, sometimes called a Thumb drive, is recognized by Linux as a SCSI device. Using one of these devices as a rescue device has the advantage that it is usually large enough to hold more than a minimal boot image. You can save critical data to the drive as well as use it to diagnose and recover a damaged system. Booting such a drive requires BIOS support, but building the system consists of formatting the drive, adding GRUB as well as the Linux kernel and supporting files.

User Notes: https://wiki.linuxfromscratch.org/blfs/wiki/CreatingaCustomBootDevice

Setting a smaller screen size in grub

Modern screens often have a lot more pixels then the screens used in the past. If your screen is 1600 pixels wide, an 8x16 font will give you 200 columns of text - unless your monitor is enormous, the text will be tiny. One of the ways to work around this is to tell grub to use a smaller size, such as 1024x768 or 800x600 or even 640x480. Even if your screen does not have a 4:3 aspect ratio, this should work.

To try this, you can reboot and edit grub's command-line to insert a 'video=' parameter between the 'root=/dev/sdXn' and 'ro', for example root=/dev/sda2 video=1024x768 ro based on the example in TFS section 10.4.4 : ../../../../lfs/view/development/chapter10/grub.html.

If you decide that you wish to do this, you can then (as the root user) edit /boot/grub/grub.cfg.

Using the standard psf fonts

In TFS the kbd package is used. The fonts it provides are PC Screen Fonts, usually called PSF, and they were installed into /usr/share/consolefonts. Where these include a unicode mapping table, the file suffix is often changed to .psfu although packages such as terminus-font (see below) do not add the 'u'. These fonts are usually compressed with gzip to save space, but that is not essential.

The initial PC text screens had 8 colours, or 16 colours if the bright versions of the original 8 colours were used. A PSF font can include up to 256 characters (technically, glyphs) while allowing 16 colours, or up to 512 characters (in which case, the bright colours will not be available). Clearly, these console fonts cannot be used to display CJK text - that would need thousands of available glyphs.

Some fonts in kbd can cover more than 512 codepoints ('characters'), with varying degrees of fidelity: unicode contains several whitespace codepoints which can all be mapped to a space, varieties of dashes can be mapped to a minus sign, smart quotes can map to the regular ASCII quotes rather than to whatever is used for "codepoint not present or invalid", and those cyrillic or greek letters which look like latin letters can be mapped onto them, so 'A' can also do duty for cyrillic A and greek Alpha, and 'P' can also do duty for cyrillic ER and greek RHO. Unfortunately, where a font has been created from a BDF file (the method in terminus and debian's console-setup ) such mapping of additional codepoints onto an existing glyph is not always done, although the terminus ter-vXXn fonts do this well.

There are over 120 combinations of font and size in kbd: often a font is provided at several character sizes, and sometimes varieties cover different subsets of unicode. Most are 8 pixels wide, in heights from 8 to 16 pixels, but there are a few which are 9 pixels wide, some others which are 12x22, and even one (latarcyrheb-sun32.psfu) which has been scaled up to 16x32. Using a bigger font is another way of making text on a large screen easier to read.

Testing different fonts

You can test fonts as a normal user. If you have a font which has not been installed, you can load it with :

setfont /path/to/yourfont.ext

For the fonts already installed you only need the name, so using gr737a-9x16.psfu.gz as an example:

setfont gr737a-9x16

To see the glyphs in the font, use:

showconsolefont

If the font looks as if it might be useful, you can then go on to test it more thoroughly.

When you find a font which you wish to use, as the root user) edit /etc/sysconfig/console as described in TFS section 9.6.5 ../../../../lfs/view/development/chapter09/usage.html. .

For fonts not supplied with the kbd package you will need to optionally compress it / them with gzip and then install it / them as the root user.

Editing fonts using psf-tools

Although some console fonts are created from BDF files, which is a text format with hex values for the pixels in each row of the character, there are more-modern tools available for editing psf fonts. The psftools package allows you to dump a font to a text representation with a dash for a pixel which is off (black) and a hash for a pixel which is on (white). You can then edit the text file to add more characters, or reshape them, or map extra codepoints onto them, and then create a new psf font with your changes.

Using fonts from Terminus-font

The Terminus Font package provides fixed-width bitmap fonts designed for long (8 hours and more per day) work with computers. Under 'Character variants' on that page is a list of patches (in the alt/ directory). If you are using a graphical browser to look at that page, you can see what the patches do, e.g. 'll2' makes 'l' more visibly different from 'i' and '1'.

By default terminus-fonts will try to create several types of font, and it will fail if bdftopcf from Xorg Applications has not been installed. The configure script is only really useful if you go on to install all the fonts (console and X11 bitmap) to the correct directories, as in a distro. To build only the PSF fonts and their dependencies, run:

make psf

This will create more than 240 ter-*.psf fonts. The 'b' suffix indicates bright, 'n' indicates normal. You can then test them to see if any fit your requirements. Unless you are creating a distro, there seems little point in installing them all.

As an example, to install the last of these fonts, you can gzip it and then as the root user:

install -v -m644 ter-v32n.psf.gz /usr/share/consolefonts

Microcode updates for CPUs

In general, microcode can be loaded by the BIOS or UEFI, and it might be updated by upgrading to a newer version of those. On linux, you can also load the microcode from the kernel if you are using an AMD family 10h or later processor (first introduced late 2007), or an Intel processor from 1998 and later (Pentium4, Core, etc), if updated microcode has been released. These updates only last until the machine is powered off, so they need to be applied on every boot.

Intel provide updates of their microcode for Skylake and later processors as new vulnerabilities come to light, and have in the past provided updates for processors from SandyBridge onwards, although those are no-longer supported for new fixes. New versions of AMD firmware are rare and usually only apply to a few models, although motherboard manufacturers get AGESA (AMD Generic Encapsulated Software Architecture) updates to change BIOS values, e.g. to support more memory variants, new vulnerability fixes or newer CPUs.

There were two ways of loading the microcode, described as 'early' and 'late'. Early loading happens before userspace has been started, late loading happens after userspace has started. However, late loading is known to be problematic and not supported anymore (see the kernel commit x86/microcode: Taint and warn on late loading.) Indeed, early loading is needed to work around one particular erratum in early Intel Haswell processors which had TSX enabled. (See Intel Disables TSX Instructions: Erratum Found in Haswell, Haswell-E/EP, Broadwell-Y.) Without this update glibc can do the wrong thing in uncommon situations.

In previous versions of this book, late loading of microcode to see if it gets applied was recommended, followed by using an initrd to force early loading. But now that the contents of the Intel microcode tarball is documented, and AMD microcode can be read by a Python script to determine which machines it covers, there is no real reason to use late loading.

It might be still possible to manually force late loading of microcode. But it may cause kernel malfunction and you should take the risk yourself. You will need to reconfigure your kernel for either method. The instructions here will show you how to create an initrd for early loading. It is also possible to build the same microcode bin file into the kernel, which allows early loading but requires the kernel to be recompiled to update the microcode.

To confirm what processor(s) you have (if more than one, they will be identical) look in /proc/cpuinfo. Determine the decimal values of the cpu family, model and stepping by running the following command (it will also report the current microcode version):

head -n7 /proc/cpuinfo

Convert the cpu family, model and stepping to pairs of hexadecimal digits, and remember the value of the “microcode” field. You can now check if there is any microcode available.

If you are creating an initrd to update firmware for different machines, as a distro would do, go down to 'Early loading of microcode' and cat all the Intel blobs to GenuineIntel.bin or cat all the AMD blobs to AuthenticAMD.bin. This creates a larger initrd - for all Intel machines in the 20200609 update the size was 3.0 MB compared to typically 24 KB for one machine.

General Setup --->
  [*] Initial RAM filesystem and RAM disk (initramfs/initrd) support [CONFIG_BLK_DEV_INITRD]
Processor type and features  --->
  [*] CPU microcode loading support  [CONFIG_MICROCODE]
  [*]      Intel microcode loading support [CONFIG_MICROCODE_INTEL]

Family=0x10 Model=0x05 Stepping=0x03: Patch=0x010000c8 Length=960 bytes

General Setup --->
  [*] Initial RAM filesystem and RAM disk (initramfs/initrd) support [CONFIG_BLK_DEV_INITRD]
Processor type and features  --->
  [*] CPU microcode loading support  [CONFIG_MICROCODE]
  [*]      AMD microcode loading support [CONFIG_MICROCODE_AMD]

mkdir -p initrd/kernel/x86/microcode
cd initrd

cp -v ../<MYCONTAINER> kernel/x86/microcode/AuthenticAMD.bin

cp -v ../intel-ucode/<XX-YY-ZZ> kernel/x86/microcode/GenuineIntel.bin

find . | cpio -o -H newc > /boot/microcode.img

initrd /microcode.img

initrd /boot/microcode.img

dmesg | grep -e 'microcode' -e 'Linux version' -e 'Command line'

[    0.000000] microcode: microcode updated early to revision 0xb8, date = 2022-08-31
[    0.000000] Linux version 6.1.11 (xry111@stargazer) (gcc (GCC) 12.2.0, GNU ld (GNU Binutils) 2.40) #2 SMP PREEMPT_DYNAMIC Tue Feb 14 23:23:31 CST 2023
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-6.1.11-lfs-11.3-rc1 root=PARTUUID=<CLASSIFIED> ro
[    0.452924] microcode: sig=0x706e5, pf=0x80, revision=0xb8
[    0.453197] microcode: Microcode Update Driver: v2.2.

[    0.000000] Linux version 4.15.3 (ken@testserver) (gcc version 7.3.0 (GCC))
               #2 SMP Sun Feb 18 02:32:03 GMT 2018
[    0.000000] Command line: BOOT_IMAGE=/vmlinuz-4.15.3-sda5 root=/dev/sda5 ro
[    0.307619] microcode: microcode updated early to new patch_level=0x010000c8
[    0.307678] microcode: CPU0: patch_level=0x010000c8
[    0.307723] microcode: CPU1: patch_level=0x010000c8
[    0.307795] microcode: Microcode Update Driver: v2.2.

Firmware for Video Cards

mkdir -pv /lib/firmware/radeon
cp -v <YOUR_BLOBS> /lib/firmware/radeon

Device Drivers --->
  Graphics support --->
      Direct Rendering Manager --->
        [*] Direct Rendering Manager (XFree86 ... support)  [CONFIG_DRM]
      [M] ATI Radeon                                        [CONFIG_DRM_RADEON]

mkdir -pv /lib/firmware/amdgpu
cp -v <YOUR_BLOBS> /lib/firmware/amdgpu

Device Drivers --->
  Graphics support --->
      Direct Rendering Manager --->
        [*] Direct Rendering Manager (XFree86 ... support)  [CONFIG_DRM]
        [M] AMD GPU                                         [CONFIG_DRM_AMDGPU]
        Display Engine Configuration --->
          [*] AMD DC - Enable new display engine (NEW)      [CONFIG_DRM_AMD_DC]

Device Drivers --->
  Graphics support --->
      Direct Rendering Manager --->
        <*> Direct Rendering Manager (XFree86 ... support)  [CONFIG_DRM]
      <*/M> Nouveau (NVIDIA) cards                          [CONFIG_DRM_NOUVEAU]

wget https://raw.github.com/imirkin/re-vp2/master/extract_firmware.py
wget https://us.download.nvidia.com/XFree86/Linux-x86/325.15/NVIDIA-Linux-x86-325.15.run
sh NVIDIA-Linux-x86-325.15.run --extract-only
python2 extract_firmware.py
mkdir -p /lib/firmware/nouveau
cp -d nv* vuc-* /lib/firmware/nouveau/

Firmware for Network Interfaces

The kernel likes to load firmware for some network drivers, particularly those from Realtek (the /lib/linux-firmware/rtl_nic/) directory, but they generally appear to work without it. Therefore, you can boot the kernel, check dmesg for messages about this missing firmware, and if necessary download the firmware and put it in the specified directory in /lib/firmware so that it will be found on subsequent boots. Note that with current kernels this works whether or not the driver is compiled in or built as a module, there is no need to build this firmware into the kernel. Here is an example where the R8169 driver has been compiled in but the firmware was not made available. Once the firmware had been provided, there was no mention of it on later boots.

dmesg | grep firmware | grep r8169
[    7.018028] r8169 0000:01:00.0: Direct firmware load for rtl_nic/rtl8168g-2.fw failed with error -2
[    7.018036] r8169 0000:01:00.0 eth0: unable to load firmware patch rtl_nic/rtl8168g-2.fw (-2)

Firmware for Regulatory Database of Wireless Devices

Different countries have different regulations on the radio spectrum usage of wireless devices. You can install a firmware to make the wireless devices obey local spectrum regulations, so you won't be inquired by local authority or find your wireless NIC jamming the frequencies of other devices (for example, remote controllers). The regulatory database firmware can be downloaded from https://kernel.org/pub/software/network/wireless-regdb/. To install it, simply extract regulatory.db and regulatory.db.p7s from the tarball into /lib/firmware. Note that either the cfg80211 driver needs to be selected as a module for the regulatory.* files to be loaded, or those files need to be included as firmware into the kernel, as explained above in the section called “Firmware for Video Cards”.

The access point (AP) would send a country code to your wireless NIC, and wpa_supplicant-2.10 would tell the kernel to load the regulation of this country from regulatory.db, and enforce it. Note that several AP don't send this country code, so you may be locked to a rather restricted usage (specially if you want to use your interface as an AP).

Firmware for Other Devices

Identifying the correct firmware will typically require you to install pciutils-3.9.0, and then use lspci to identify the device. You should then search online to check which module it uses, which firmware, and where to obtain the firmware — not all of it is in linux-firmware.

If possible, you should begin by using a wired connection when you first boot your LFS system. To use a wireless connection you will need to use a network tools such as iw-5.19, Wireless Tools-29, or wpa_supplicant-2.10.

Firmware may also be needed for other devices such as some SCSI controllers, bluetooth adaptors, or TV recorders. The same principles apply.

Multiple Sound Cards

If there are multiple sound cards in a system, the "default" sound card becomes random. The method to establish sound card order depends on whether the drivers are modules or not. If the sound card drivers are compiled into the kernel, control is via kernel command line parameters in /boot/grub/grub.cfg. For example, if a system has both an FM801 card and a SoundBlaster PCI card, the following can be appended to the command line:

snd-fm801.index=0 snd-ens1371.index=1

If the sound card drivers are built as modules, the order can be established in the /etc/modprobe.conf file with:

options snd-fm801 index=0
options snd-ens1371 index=1

USB Device Issues

USB devices usually have two kinds of device nodes associated with them.

The first kind is created by device-specific drivers (e.g., usb_storage/sd_mod or usblp) in the kernel. For example, a USB mass storage device would be /dev/sdb, and a USB printer would be /dev/usb/lp0. These device nodes exist only when the device-specific driver is loaded.

The second kind of device nodes (/dev/bus/usb/BBB/DDD, where BBB is the bus number and DDD is the device number) are created even if the device doesn't have a kernel driver. By using these "raw" USB device nodes, an application can exchange arbitrary USB packets with the device, i.e., bypass the possibly-existing kernel driver.

Access to raw USB device nodes is needed when a userspace program is acting as a device driver. However, for the program to open the device successfully, the permissions have to be set correctly. By default, due to security concerns, all raw USB devices are owned by user root and group usb, and have 0664 permissions (the read access is needed, e.g., for lsusb to work and for programs to access USB hubs). Packages (such as SANE and libgphoto2) containing userspace USB device drivers also ship udev rules that change the permissions of the controlled raw USB devices. That is, rules installed by SANE change permissions for known scanners, but not printers. If a package maintainer forgot to write a rule for your device, report a bug to both BLFS (if the package is there) and upstream, and you will need to write your own rule.

There is one situation when such fine-grained access control with pre-generated udev rules doesn't work. Namely, PC emulators such as KVM, QEMU and VirtualBox use raw USB device nodes to present arbitrary USB devices to the guest operating system (note: patches are needed in order to get this to work without the obsolete /proc/bus/usb mount point described below). Obviously, maintainers of these packages cannot know which USB devices are going to be connected to the guest operating system. You can either write separate udev rules for all needed USB devices yourself, or use the default catch-all "usb" group, members of which can send arbitrary commands to all USB devices.

Before Linux-2.6.15, raw USB device access was performed not with /dev/bus/usb/BBB/DDD device nodes, but with /proc/bus/usb/BBB/DDD pseudofiles. Some applications (e.g., VMware Workstation) still use only this deprecated technique and can't use the new device nodes. For them to work, use the "usb" group, but remember that members will have unrestricted access to all USB devices. To create the fstab entry for the obsolete usbfs filesystem:

usbfs  /proc/bus/usb  usbfs  devgid=14,devmode=0660  0  0

Udev Device Attributes

Fine-tuning of device attributes such as group name and permissions is possible by creating extra udev rules, matching on something like this. The vendor and product can be found by searching the /sys/devices directory entries or using udevadm info after the device has been attached. See the documentation in the current udev directory of /usr/share/doc for details.

SUBSYSTEM=="usb_device", SYSFS{idVendor}=="05d8", SYSFS{idProduct}=="4002", \
  GROUP:="scanner", MODE:="0660"

Devices for Servers

In some cases, it makes sense to disable udev completely and create static devices. Servers are one example of this situation. Does a server need the capability of handling dynamic devices? Only the system administrator can answer that question, but in many cases the answer will be no.

If dynamic devices are not desired, then static devices must be created on the system. In the default configuration, the /etc/rc.d/rcS.d/S10udev boot script mounts a tmpfs partition over the /dev directory. This problem can be overcome by mounting the root partition temporarily:

mount --bind / /mnt
cp -a /dev/* /mnt/dev
rm /etc/rc.d/rcS.d/{S10udev,S50udev_retry}
umount /mnt

At this point, the system will use static devices upon the next reboot. Create any desired additional devices using mknod.

If you want to restore the dynamic devices, recreate the /etc/rc.d/rcS.d/{S10udev,S50udev_retry} symbolic links and reboot again. Static devices do not need to be removed (console and null are always needed) because they are covered by the tmpfs partition. Disk usage for devices is negligible (about 20–30 bytes per entry.)

Devices for DVD Drives

If the initial boot process does not set up the /dev/dvd device properly, it can be installed using the following modification to the default udev rules. As the root user, run:

sed '1d;/SYMLINK.*cdrom/ a\
KERNEL=="sr0", ENV{ID_CDROM_DVD}=="1", SYMLINK+="dvd", OPTIONS+="link_priority=-100"' \
/lib/udev/rules.d/60-cdrom_id.rules > /etc/udev/rules.d/60-cdrom_id.rules

/etc/profile

Here is a base /etc/profile. This file starts by setting up some helper functions and some basic parameters. It specifies some bash history parameters and, for security purposes, disables keeping a permanent history file for the root user. It also sets a default user prompt. It then calls small, single purpose scripts in the /etc/profile.d directory to provide most of the initialization.

For more information on the escape sequences you can use for your prompt (i.e., the PS1 environment variable) see info bash -- Node: Printing a Prompt.

cat > /etc/profile << "EOF"
# Begin /etc/profile
# Written for Beyond Linux From Scratch
# by James Robertson <jameswrobertson@earthlink.net>
# modifications by Dagmar d'Surreal <rivyqntzne@pbzpnfg.arg>

# System wide environment variables and startup programs.

# System wide aliases and functions should go in /etc/bashrc.  Personal
# environment variables and startup programs should go into
# ~/.bash_profile.  Personal aliases and functions should go into
# ~/.bashrc.

# Functions to help us manage paths.  Second argument is the name of the
# path variable to be modified (default: PATH)
pathremove () {
        local IFS=':'
        local NEWPATH
        local DIR
        local PATHVARIABLE=${2:-PATH}
        for DIR in ${!PATHVARIABLE} ; do
                if [ "$DIR" != "$1" ] ; then
                  NEWPATH=${NEWPATH:+$NEWPATH:}$DIR
                fi
        done
        export $PATHVARIABLE="$NEWPATH"
}

pathprepend () {
        pathremove $1 $2
        local PATHVARIABLE=${2:-PATH}
        export $PATHVARIABLE="$1${!PATHVARIABLE:+:${!PATHVARIABLE}}"
}

pathappend () {
        pathremove $1 $2
        local PATHVARIABLE=${2:-PATH}
        export $PATHVARIABLE="${!PATHVARIABLE:+${!PATHVARIABLE}:}$1"
}

export -f pathremove pathprepend pathappend

# Set the initial path
export PATH=/usr/bin

# Attempt to provide backward compatibility with LFS earlier than 11
if [ ! -L /bin ]; then
        pathappend /bin
fi

if [ $EUID -eq 0 ] ; then
        pathappend /usr/sbin
        if [ ! -L /sbin ]; then
                pathappend /sbin
        fi
        unset HISTFILE
fi

# Set up some environment variables.
export HISTSIZE=1000
export HISTIGNORE="&:[bf]g:exit"

# Set some defaults for graphical systems
export XDG_DATA_DIRS=${XDG_DATA_DIRS:-/usr/share/}
export XDG_CONFIG_DIRS=${XDG_CONFIG_DIRS:-/etc/xdg/}
export XDG_RUNTIME_DIR=${XDG_RUNTIME_DIR:-/tmp/xdg-$USER}

# Set up a red prompt for root and a green one for users.
NORMAL="\[\e[0m\]"
RED="\[\e[1;31m\]"
GREEN="\[\e[1;32m\]"
if [[ $EUID == 0 ]] ; then
  PS1="$RED\u [ $NORMAL\w$RED ]# $NORMAL"
else
  PS1="$GREEN\u [ $NORMAL\w$GREEN ]\$ $NORMAL"
fi

for script in /etc/profile.d/*.sh ; do
        if [ -r $script ] ; then
                . $script
        fi
done

unset script RED GREEN NORMAL

# End /etc/profile
EOF

install --directory --mode=0755 --owner=root --group=root /etc/profile.d

cat > /etc/profile.d/bash_completion.sh << "EOF"
# Begin /etc/profile.d/bash_completion.sh
# Import bash completion scripts

# If the bash-completion package is installed, use its configuration instead
if [ -f /usr/share/bash-completion/bash_completion ]; then

  # Check for interactive bash and that we haven't already been sourced.
  if [ -n "${BASH_VERSION-}" -a -n "${PS1-}" -a -z "${BASH_COMPLETION_VERSINFO-}" ]; then

    # Check for recent enough version of bash.
    if [ ${BASH_VERSINFO[0]} -gt 4 ] || \
       [ ${BASH_VERSINFO[0]} -eq 4 -a ${BASH_VERSINFO[1]} -ge 1 ]; then
       [ -r "${XDG_CONFIG_HOME:-$HOME/.config}/bash_completion" ] && \
            . "${XDG_CONFIG_HOME:-$HOME/.config}/bash_completion"
       if shopt -q progcomp && [ -r /usr/share/bash-completion/bash_completion ]; then
          # Source completion code.
          . /usr/share/bash-completion/bash_completion
       fi
    fi
  fi

else

  # bash-completions are not installed, use only bash completion directory
  if shopt -q progcomp; then
    for script in /etc/bash_completion.d/* ; do
      if [ -r $script ] ; then
        . $script
      fi
    done
  fi
fi

# End /etc/profile.d/bash_completion.sh
EOF

install --directory --mode=0755 --owner=root --group=root /etc/bash_completion.d

cat > /etc/profile.d/dircolors.sh << "EOF"
# Setup for /bin/ls and /bin/grep to support color, the alias is in /etc/bashrc.
if [ -f "/etc/dircolors" ] ; then
        eval $(dircolors -b /etc/dircolors)
fi

if [ -f "$HOME/.dircolors" ] ; then
        eval $(dircolors -b $HOME/.dircolors)
fi

alias ls='ls --color=auto'
alias grep='grep --color=auto'
EOF

cat > /etc/profile.d/extrapaths.sh << "EOF"
if [ -d /usr/local/lib/pkgconfig ] ; then
        pathappend /usr/local/lib/pkgconfig PKG_CONFIG_PATH
fi
if [ -d /usr/local/bin ]; then
        pathprepend /usr/local/bin
fi
if [ -d /usr/local/sbin -a $EUID -eq 0 ]; then
        pathprepend /usr/local/sbin
fi

if [ -d /usr/local/share ]; then
        pathprepend /usr/local/share XDG_DATA_DIRS
fi

# Set some defaults before other applications add to these paths.
pathappend /usr/share/man  MANPATH
pathappend /usr/share/info INFOPATH
EOF

cat > /etc/profile.d/readline.sh << "EOF"
# Set up the INPUTRC environment variable.
if [ -z "$INPUTRC" -a ! -f "$HOME/.inputrc" ] ; then
        INPUTRC=/etc/inputrc
fi
export INPUTRC
EOF

cat > /etc/profile.d/umask.sh << "EOF"
# By default, the umask should be set.
if [ "$(id -gn)" = "$(id -un)" -a $EUID -gt 99 ] ; then
  umask 002
else
  umask 022
fi
EOF

cat > /etc/profile.d/i18n.sh << "EOF"
# Set up i18n variables
export LANG=<ll>_<CC>.<charmap><@modifiers>
EOF

/etc/bashrc

Here is a base /etc/bashrc. Comments in the file should explain everything you need.

cat > /etc/bashrc << "EOF"
# Begin /etc/bashrc
# Written for Beyond Linux From Scratch
# by James Robertson <jameswrobertson@earthlink.net>
# updated by Bruce Dubbs <bdubbs@linuxfromscratch.org>

# System wide aliases and functions.

# System wide environment variables and startup programs should go into
# /etc/profile.  Personal environment variables and startup programs
# should go into ~/.bash_profile.  Personal aliases and functions should
# go into ~/.bashrc

# Provides colored /bin/ls and /bin/grep commands.  Used in conjunction
# with code in /etc/profile.

alias ls='ls --color=auto'
alias grep='grep --color=auto'

# Provides prompt for non-login shells, specifically shells started
# in the X environment. [Review the LFS archive thread titled
# PS1 Environment Variable for a great case study behind this script
# addendum.]

NORMAL="\[\e[0m\]"
RED="\[\e[1;31m\]"
GREEN="\[\e[1;32m\]"
if [[ $EUID == 0 ]] ; then
  PS1="$RED\u [ $NORMAL\w$RED ]# $NORMAL"
else
  PS1="$GREEN\u [ $NORMAL\w$GREEN ]\$ $NORMAL"
fi

unset RED GREEN NORMAL

# End /etc/bashrc
EOF

~/.bash_profile

Here is a base ~/.bash_profile. If you want each new user to have this file automatically, just change the output of the command to /etc/skel/.bash_profile and check the permissions after the command is run. You can then copy /etc/skel/.bash_profile to the home directories of already existing users, including root, and set the owner and group appropriately.

cat > ~/.bash_profile << "EOF"
# Begin ~/.bash_profile
# Written for Beyond Linux From Scratch
# by James Robertson <jameswrobertson@earthlink.net>
# updated by Bruce Dubbs <bdubbs@linuxfromscratch.org>

# Personal environment variables and startup programs.

# Personal aliases and functions should go in ~/.bashrc.  System wide
# environment variables and startup programs are in /etc/profile.
# System wide aliases and functions are in /etc/bashrc.

if [ -f "$HOME/.bashrc" ] ; then
  source $HOME/.bashrc
fi

if [ -d "$HOME/bin" ] ; then
  pathprepend $HOME/bin
fi

# Having . in the PATH is dangerous
#if [ $EUID -gt 99 ]; then
#  pathappend .
#fi

# End ~/.bash_profile
EOF

~/.profile

Here is a base ~/.profile. The comments and instructions for using /etc/skel for .bash_profile above also apply here. Only the target file names are different.

cat > ~/.profile << "EOF"
# Begin ~/.profile
# Personal environment variables and startup programs.

if [ -d "$HOME/bin" ] ; then
  pathprepend $HOME/bin
fi

# Set up user specific i18n variables
#export LANG=<ll>_<CC>.<charmap><@modifiers>

# End ~/.profile
EOF

~/.bashrc

Here is a base ~/.bashrc.

cat > ~/.bashrc << "EOF"
# Begin ~/.bashrc
# Written for Beyond Linux From Scratch
# by James Robertson <jameswrobertson@earthlink.net>

# Personal aliases and functions.

# Personal environment variables and startup programs should go in
# ~/.bash_profile.  System wide environment variables and startup
# programs are in /etc/profile.  System wide aliases and functions are
# in /etc/bashrc.

if [ -f "/etc/bashrc" ] ; then
  source /etc/bashrc
fi

# Set up user specific i18n variables
#export LANG=<ll>_<CC>.<charmap><@modifiers>

# End ~/.bashrc
EOF

~/.bash_logout

This is an empty ~/.bash_logout that can be used as a template. You will notice that the base ~/.bash_logout does not include a clear command. This is because the clear is handled in the /etc/issue file.

cat > ~/.bash_logout << "EOF"
# Begin ~/.bash_logout
# Written for Beyond Linux From Scratch
# by James Robertson <jameswrobertson@earthlink.net>

# Personal items to perform on logout.

# End ~/.bash_logout
EOF

/etc/dircolors

If you want to use the dircolors capability, then run the following command. The /etc/skel setup steps shown above also can be used here to provide a ~/.dircolors file when a new user is set up. As before, just change the output file name on the following command and assure the permissions, owner, and group are correct on the files created and/or copied.

dircolors -p > /etc/dircolors

If you wish to customize the colors used for different file types, you can edit the /etc/dircolors file. The instructions for setting the colors are embedded in the file.

Finally, Ian Macdonald has written an excellent collection of tips and tricks to enhance your shell environment. You can read it online at https://www.caliban.org/bash/index.shtml.

Introduction to Firewall Creation

The purpose of a firewall is to protect a computer or a network against malicious access. In a perfect world every daemon or service, on every machine, is perfectly configured and immune to security flaws, and all users are trusted implicitly to use the equipment as intended. However, this is rarely, if ever, the case. Daemons may be misconfigured, or updates may not have been applied for known exploits against essential services. Additionally, you may wish to choose which services are accessible by certain machines or users, or you may wish to limit which machines or applications are allowed external access. Alternatively, you simply may not trust some of your applications or users. For these reasons, a carefully designed firewall should be an essential part of system security.

While a firewall can greatly limit the scope of the above issues, do not assume that having a firewall makes careful configuration redundant, or that any negligent misconfiguration is harmless. A firewall does not prevent the exploitation of any service you offer outside of it. Despite having a firewall, you need to keep applications and daemons properly configured and up to date.

Meaning of the Word "Firewall"

The word firewall can have several different meanings.

Conclusion

BLFS provides an utility to manage the kernel Netfilter interface, iptables-1.8.9. It has been around since early 2.4 kernels, and has been the standard since. This is likely the set of tools that will be most familiar to existing admins. Other tools have been developed more recently, see the list of further readings below for more details. Here you will find a list of URLs that contain comprehensive information about building firewalls and further securing your system.

Extra Information