FreeBSD getting into development and make your own release to test changes
Posted on Mon 05 January 2015 in FreeBSD
When I installed FreeBSD, I stumbled over a bug with my T420. I now want to help fix it or to implement a workaround for my T420 to use it with gpt and ZFS-Root, Therefor I needed to setup a development environment for FreeBSD on my Thinkpad. To make this process easier for others I will write down all the points that helped me and collect all in this post. Please see this post as work in progress as I may change or add some points in future.
Getting familiar with the names and versioning of FreeBSD
The latest release is 10.1 as I am writing this. The ongoing new development is always done in the so called HEAD or CURRENT tree. This is the latest stuff which will lead to the next major version 11.0 some day. Since FreeBSD uses subversion, all the source code of the base system is organized within this source code versioning system. I will explain how to build your CURRENT image from the source code here, this should enable you to build every other version or self patched version by your own. Please consider that a CURRENT system is for development or very experienced people who want to follow the latest changes. You will not get much help in official places if it comes to problems which are only connected to CURRENT as some times there are uncompleted features or bugs introduced. It should not happen put some times it can.
If you want to have a more stable system with some fixes that are back ported to the latest release version you can use the STABLE branch. I personally run STABLE on the machines I am working on. STABLE as it name indicates is quite stable. The patches and new features included are tested and ported to stable after they where some time in CURRENT.
Release is what you want if you need a super stable system. This is bullet proven stuff only touched by security fixes. If you have no need to run something newer, this is what makes you happy for production. To be even more paranoid you can skip the .0 releases as well of a new major version. New major versions bring bigger changes and may have a different behavior or feeling. Keep in mind that your staff need some time to adapt to new changes, too. A lot of companies skip .0 releases of any software they can avoid it for production. That is not a bad idea. You will have the most fun in production if everything is just working ;-)
Setup a development environment
You need FreeBSD to develop FreeBSD. That's not quit surprising, but if you want to build a CURRENT system you need to use a CURRENT system as base. So today I use a 11 snapshot image to build my latest current images from source code.
I will come over the points I found hard to find in the documentation. The people on the FreeBSD IRC channel helped me a lot to find the locations where the documentation is. In many cases you need to know what you're searching. It is easy to find the correct documentation if you are more familiar with the wording, but still first use a search engine to try to find the information yourself. It helps a lot if you searched the docs by yourself. You will learn more by doing it and people are more friendly if they recognize that you already have spend some time to help yourself.
Some more details and examples are collected by TJ from BSDNow can be found here: http://www.bsdnow.tv/tutorials/stable-current
From now on I assume you have a working FreeBSD - CURRENT system installed and running.
Install necessary packages or ports
To check out the source code you will need to use subversion. Install it using pkg or ports.
As mentioned by FAndrey you don't need to install subversion client anymore. FreeBSD comes with svnlite in base system since FreeBSD 10. So if you don't want svn you can use svnlite without installing an additional package or port. Thanks for that hint.
Using pkg:
pkg install subversion
Using ports
cd /usr/src/contrib/subversion
make install clean
Start populating /usr/src
The source code is managed in a subversion repository. There are some mirrors around so you can use one of them near to your location. As I am from Europe I will use one of the mirror located here. Find a mirror next to your location in the mirrors list:
https://www.freebsd.org/doc/handbook/svn.html#svn-mirrors
Very important is that you verify the fingerprint of your mirror and only use the encrypted connection!
svn co https://svn0.eu.FreeBSD.org/base/head/ /usr/src/
Depending on the speed of your internet connection this will take a while. Grab a cup of coffee or tee.
If everything worked fine, your /usr/src should look like this:
-rw-r--r-- 1 root wheel 6.0K Nov 15 18:04 COPYRIGHT -rw-r--r-- 1 root wheel 734B Nov 15 18:04 LOCKS -rw-r--r-- 1 root wheel 6.3K Nov 15 18:04 MAINTAINERS -rw-r--r-- 1 root wheel 18K Nov 15 18:04 Makefile -rw-r--r-- 1 root wheel 62K Nov 15 18:26 Makefile.inc1 -rw-r--r-- 1 root wheel 287K Nov 15 18:26 ObsoleteFiles.inc -rw-r--r-- 1 root wheel 3.1K Nov 15 18:13 README -rw-r--r-- 1 root wheel 42K Nov 15 18:04 UPDATING drwxr-xr-x 42 root wheel 44B Nov 15 18:04 bin drwxr-xr-x 9 root wheel 11B Nov 15 18:26 cddl drwxr-xr-x 87 root wheel 87B Nov 15 18:23 contrib drwxr-xr-x 5 root wheel 6B Nov 15 18:12 crypto drwxr-xr-x 26 root wheel 95B Nov 15 18:04 etc drwxr-xr-x 14 root wheel 16B Nov 15 18:13 games drwxr-xr-x 5 root wheel 9B Nov 15 18:13 gnu drwxr-xr-x 8 root wheel 105B Nov 15 18:13 include drwxr-xr-x 9 root wheel 12B Nov 15 18:24 kerberos5 drwxr-xr-x 106 root wheel 108B Nov 15 18:26 lib drwxr-xr-x 37 root wheel 42B Nov 15 18:04 libexec drwxr-xr-x 13 root wheel 17B Nov 15 18:12 release drwxr-xr-x 4 root wheel 6B Nov 15 18:13 rescue drwxr-xr-x 91 root wheel 99B Nov 15 18:13 sbin drwxr-xr-x 7 root wheel 9B Nov 15 18:13 secure drwxr-xr-x 29 root wheel 31B Nov 15 18:25 share drwxr-xr-x 53 root wheel 54B Nov 15 18:11 sys drwxr-xr-x 3 root wheel 6B Nov 15 18:24 tests drwxr-xr-x 15 root wheel 19B Nov 15 18:14 tools drwxr-xr-x 258 root wheel 265B Nov 15 18:13 usr.bin drwxr-xr-x 212 root wheel 220B Nov 15 18:25 usr.sbin
This is how you did the initial check out of the source code. You will need to update the tree from time to time to get the latest version of the source code. How the FreeBSD project is using subversion to control all the changes is explained very well in this video from Stefan Sperling, Subversion for FreeBSD developers which was recorded on EuroBSDcon 2014 in Sofia.
Configure your make.conf
The make.conf file contains the default make options. This files is important to be adjusted to your system and needs.
In my case it looks like this for my workstation:
MAKE_JOBS_NUMBER?=8 OPTIONS_UNSET = NLS DOCS #For my NVIDIA CARD WITH_NVIDIA_GL=YES WITH_NVIDIA=YES WITHOUT_NOUVEAU=YES
Make your changes or add a patch
You have the complete source code in your /usr/src directory. Browse it or add a patch you made. All base components are in this tree. There are some additional tools to build and manage the source code as well. Check the MAINTAINERS file to see who is the contact for a patch you created and publish it on the mailing list to get it reviewed and maybe added to the upstream version of FreeBSD after discussion.
Build world and build your custom kernel
To have a system build with this options we will follow the documentation to rebuild world and kernel. This page is very detailed and should help you to do all necessary tasks: https://www.freebsd.org/doc/handbook/makeworld.html
Make your own FreeBSD-CURRENT install media
We are on a BSD system. The best source for information is the system itself. Try to search the man pages. The most BSD developers care a lot about documentation of their stuff and that the necessary information to run there programs are delivered in the man pages.
man release
In that man page the release scripts a very well documented. You can create your own install media images to test you changes or have your customized installation media for you environment.
Did I missed something? Let me know!
This is work in progress and I will update it with pleasure if you're missing something. Ping me and I will try to add some more sections or if you miss some links or hints.