FreeBSD getting into development and make your own release to test changes

Posted on Mon 05 January 2015 in FreeBSD

When I installed FreeBSD, I stumbled over a bug with my T420. I now want to help fix it or to implement a workaround for my T420 to use it with gpt and ZFS-Root, Therefor  I needed to setup a development environment for FreeBSD on my Thinkpad. To make this process easier for others I will write down all the points that helped me and collect all in this post. Please see this post as work in progress as I may change or add some points in future.

Getting familiar with the names and versioning of FreeBSD

The latest release is 10.1 as I am writing this. The ongoing new development is always done in the so called HEAD or CURRENT tree. This is the latest stuff which will lead to the next major version 11.0 some day. Since FreeBSD uses subversion, all the source code of the base system is organized within this source code versioning system. I will explain how to build your CURRENT image from the source code here, this should enable you to build every other version or self patched version by your own. Please consider that a CURRENT system is for development or very experienced people who want to follow the latest changes. You will not get much help in official places if it comes to problems which are only connected to CURRENT as some times there are uncompleted features or bugs introduced. It should not happen put some times it can.

If you want to have a more stable system with some fixes that are back ported to the latest release version you can use the STABLE branch. I personally run STABLE on the machines I am working on. STABLE as it name indicates is quite stable. The patches and new features included are tested and ported to stable after they where some time in CURRENT.

Release is what you want if you need a super stable system. This is bullet proven stuff only touched by security fixes. If you have no need to run something newer, this is what makes you happy for production. To be even more paranoid you can skip the .0 releases as well of a new major version. New major versions bring bigger changes and may have a different behavior or feeling. Keep in mind that your staff need some time to adapt to new changes, too. A lot of companies skip .0 releases of any software they can avoid it for production. That is not a bad idea. You will have the most fun in production if everything is just working ;-)

Setup a development environment

You need FreeBSD to develop FreeBSD. That's not quit surprising, but if you want to build a CURRENT system you need to use a CURRENT system as base. So today I use a 11 snapshot image to build my latest current images from source code.

I will come over the points I found hard to find in the documentation. The people on the FreeBSD IRC channel helped me a lot to find the locations where the documentation is. In many cases you need to know what you're searching. It is easy to find the correct documentation if you are more familiar with the wording, but still first use a search engine to try to find the information yourself. It helps a lot if you searched the docs by yourself. You will learn more by doing it and people are more friendly if they recognize that you already have spend some time to help yourself.

Some more details and examples are collected by TJ from BSDNow can be found here: http://www.bsdnow.tv/tutorials/stable-current

From now on I assume you have a working FreeBSD - CURRENT system installed and running.

Install necessary packages or ports

To check out the source code you will need to use subversion. Install it using pkg or ports.

As mentioned by FAndrey you don't need to install subversion client anymore. FreeBSD comes with svnlite in base system since FreeBSD 10. So if you don't want svn you can use svnlite without installing an additional package or port. Thanks for that hint.

Using pkg:

pkg install subversion

Using ports

cd /usr/src/contrib/subversion
make install clean

Start populating /usr/src

The source code is managed in a subversion repository. There are some mirrors around so you can use one of them near to your location. As I am from Europe I will use one of the mirror located here. Find a mirror next to your location in the mirrors list:

https://www.freebsd.org/doc/handbook/svn.html#svn-mirrors

Very important is that you verify the fingerprint of your mirror and only use the encrypted  connection!

svn co https://svn0.eu.FreeBSD.org/base/head/ /usr/src/

Depending on the speed of your internet connection this will take a while. Grab a cup of coffee or tee.

If everything worked fine, your /usr/src should look like this:

-rw-r--r-- 1 root wheel 6.0K Nov 15 18:04 COPYRIGHT
-rw-r--r-- 1 root wheel 734B Nov 15 18:04 LOCKS
-rw-r--r-- 1 root wheel 6.3K Nov 15 18:04 MAINTAINERS
-rw-r--r-- 1 root wheel 18K Nov 15 18:04 Makefile
-rw-r--r-- 1 root wheel 62K Nov 15 18:26 Makefile.inc1
-rw-r--r-- 1 root wheel 287K Nov 15 18:26 ObsoleteFiles.inc
-rw-r--r-- 1 root wheel 3.1K Nov 15 18:13 README
-rw-r--r-- 1 root wheel 42K Nov 15 18:04 UPDATING
drwxr-xr-x 42 root wheel 44B Nov 15 18:04 bin
drwxr-xr-x 9 root wheel 11B Nov 15 18:26 cddl
drwxr-xr-x 87 root wheel 87B Nov 15 18:23 contrib
drwxr-xr-x 5 root wheel 6B Nov 15 18:12 crypto
drwxr-xr-x 26 root wheel 95B Nov 15 18:04 etc
drwxr-xr-x 14 root wheel 16B Nov 15 18:13 games
drwxr-xr-x 5 root wheel 9B Nov 15 18:13 gnu
drwxr-xr-x 8 root wheel 105B Nov 15 18:13 include
drwxr-xr-x 9 root wheel 12B Nov 15 18:24 kerberos5
drwxr-xr-x 106 root wheel 108B Nov 15 18:26 lib
drwxr-xr-x 37 root wheel 42B Nov 15 18:04 libexec
drwxr-xr-x 13 root wheel 17B Nov 15 18:12 release
drwxr-xr-x 4 root wheel 6B Nov 15 18:13 rescue
drwxr-xr-x 91 root wheel 99B Nov 15 18:13 sbin
drwxr-xr-x 7 root wheel 9B Nov 15 18:13 secure
drwxr-xr-x 29 root wheel 31B Nov 15 18:25 share
drwxr-xr-x 53 root wheel 54B Nov 15 18:11 sys
drwxr-xr-x 3 root wheel 6B Nov 15 18:24 tests
drwxr-xr-x 15 root wheel 19B Nov 15 18:14 tools
drwxr-xr-x 258 root wheel 265B Nov 15 18:13 usr.bin
drwxr-xr-x 212 root wheel 220B Nov 15 18:25 usr.sbin

This is how you did the initial check out of the source code. You will need to update the tree from time to time to get the latest version of the source code. How the FreeBSD project is using subversion to control all the changes is explained very well in this video from Stefan Sperling, Subversion for FreeBSD developers which was recorded on EuroBSDcon 2014 in Sofia.

Configure your make.conf

The make.conf file contains the default make options. This files is important to be adjusted to your system and needs.

In my case it looks like this for my workstation:

MAKE_JOBS_NUMBER?=8

OPTIONS_UNSET = NLS DOCS

#For my NVIDIA CARD
WITH_NVIDIA_GL=YES
WITH_NVIDIA=YES
WITHOUT_NOUVEAU=YES

Make your changes or add a patch

You have the complete source code in your /usr/src directory. Browse it or add a patch you made. All base components are in this tree. There are some additional tools to build and manage the source code as well. Check the MAINTAINERS file to see who is the contact for a patch you created and publish it on the mailing list to get it reviewed and maybe added to the upstream version of FreeBSD after discussion.

Build world and build your custom kernel

To have a system build with this options we will follow the documentation to rebuild world and kernel. This page is very detailed and should help you to do all necessary tasks: https://www.freebsd.org/doc/handbook/makeworld.html

Make your own FreeBSD-CURRENT install media

We are on a BSD system. The best source for information is the system itself. Try to search the man pages. The most BSD developers care a lot about documentation of their stuff and that the necessary information to run there programs are delivered in the man pages.

man release

In that man page the release scripts a very well documented. You can create your own install media images to test you changes or have your customized installation media for you environment.

Did I missed something? Let me know!

This is work in progress and I will update it with pleasure if you're missing something. Ping me and I will try to add some more sections or if you miss some links or hints.