JP van Oosten

Notes from the Debian packaging symposium

Dec 22, 2012

A few weeks ago, Frank Brokken and Jurjen Bokma organised a Debian / Ubuntu packaging symposium, with special guest Tony Mancill. We learned a lot about creating Debian packages and creating repositories. I wrote this blog-post to make sure I don't forget the most important steps and maybe give others a chance to learn how to create packages as well.

These notes are a combination of my own experiences and notes that can be found on the symposiums website. Please don't hesitate to contact me if you have any questions or remarks (jp at this domain).

Quick overview (or: TL;DR)

Required packages

Install the following packages:

Chroots and cowbuilder

It is a good idea to build your packages in a chroot. Besides giving you a clean and homogeneous environment, it is a convenient way to test whether your package builds and installs in a clean environment (i.e., all your dependencies are correctly specified).

Cowbuilder is a tool to create "copy-on-write" chroot environments. This means that when you work in the chroot environment (building or testing your package), files that are changed are cleaned-up after you're finished. This means that you can set-up your clean environment once, and be confident it stays clean. The advantage over copying the entire environment everytime is that files are only copied when they are being written to.

In order to create your cowbuilder environment, you can either use the cowbuilder-create-base script (mirror) Jurjen created, or use the following:

sudo cowbuilder --create --distribution=sid \
    --basepath=/var/cache/pbuilder/base-sid.cow

Make sure that, if you use Jurjens script, you create a symlink from /var/cache/pbuilder/$DIST-base.cow to /var/cache/pbuilder/base-$DIST.cow because some build-tools expect the COWs to be at the latter location.

Tony suggested a couple of shell-functions for easy entering and saving changes in chroots:

cb-shell () {
    chr=$1 ; shift
    sudo cowbuilder \
        --bindmount $HOME \
        --login \
        --basepath=/var/cache/pbuilder/base-${chr}.cow $@
}

cb-shell-save () {
    cb-shell $@ --save-after-login
}

The cb-shell-save function can be used to adjust your chroot a bit. Good practice is to put the name of your chroot in /etc/debian_chroot, so that it will appear in your prompt (which is convenient to quickly see whether you are working in your chroot or not). You can also create a local user, which persists if you use cb-shell-save.

Configuring your environment

In this step, we will create the configuration files and environment variables necessary for the debhelper-scripts, pbuilder and git-buildpackage.

First, make sure you have a valid gpg key. If you don't have one, create one with gpg --gen-key. It's good practice to actually use a (long) phrase as your passphrase. Since you must type it a couple of times, you can use an agent for convenience. Install gnupg-agent and make sure use-agent is specified in ~/.gnupg/gpg.conf if you want to use the agent. Make sure you also put the following in your shell start up scripts (.bashrc or so):

if test -f $HOME/.gpg-agent-info && kill -0 $(cut -d: -f 2 $HOME/.gpg-agent-info) 2>/dev/null; then
    eval $(cat $HOME/.gpg-agent-info)
else
    eval $(gpg-agent --daemon --write-env-file $HOME/.gpg-agent-info)
fi
GPG_TTY=$(tty)
# GPG_AGENT_INFO is set from within $HOME/.gpg-agent-info, but still needs to be exported
export GPG_TTY GPG_AGENT_INFO

For dh_make and the build-scripts, you need to specify the following environment variables:

export DEBEMAIL=john@sudoe.com
export DEBFULLNAME="John Sudoe"
export DEBSIGN_KEYID=D1D9258A

DEBSIGN_KEYID is the id of the gpg key with which you want to sign your packages. You can find it using gpg --list-secret-keys.

Note that if you use git, it is a good idea to make sure you use the same name and email address to commit your debian-changes. Put the following in your ~/.gitconfig or in <project>/.git/config:

[user]
        name = John Sudoe
        email = john@sudoe.com

Next, we will configure pbuilder. Its configuration file, ~/.pbuilderrc, is a shell-script. Jurjen suggests the following (I changed ${DIST}-base.cow to base-${DIST}.cow to conform to pbuilder defaults):

PDEBUILD_PBUILDER=cowbuilder

if [ -n "${DIST}" ]; then
    export DISTRIBUTION=$DIST

    AUTO_DEBSIGN=yes

    BASETGZ="$(dirname $BASETGZ)/$DIST-base.tgz"

    export BASEPATH=/var/cache/pbuilder/base-${DIST}.cow
    export BUILDPLACE=/var/cache/pbuilder/${DIST}.cow

    install -d ../lastbuild/${DIST}
    export BUILDRESULT=../lastbuild/${DIST}

    if echo $@|grep -qe --create ; then
        BUILDPLACE=/var/cache/pbuilder/base-${DIST}.cow
        install -d ${BUILDPLACE}
    fi

    APTCACHE="/var/cache/pbuilder/$DIST/aptcache/"

    case ${DIST} in
        'lenny'|'squeeze'|'wheezy'|'sid' )
            MIRRORSITE=http://ftp.nl.debian.org/debian
            ;;
        'lucid'|'maverick'|'natty'|'oneiric'|'precise' )
            MIRRORSITE=http://nl.archive.ubuntu.com/ubuntu
            ;;
    esac

fi

And finally, we configure git-buildpackage by putting the following in ~/.gbp.conf:

[DEFAULT]
builder = /usr/bin/git-pbuilder
cleaner = fakeroot debian/rules clean
postbuild = lintian $GBP_CHANGES_FILE
pristine-tar = True
upstream-branch = upstream

[git-buildpackage]
export-dir = ../build-area/
tarball-dir = ../tarballs/

[git-import-orig]
dch = false

Building a package

There are a number of ways to build a package. If you only have a source tar-ball, you can use dh_make and pdebuild. The first tool creates the debian/ directory, with debian/rules -- specifying how to build the package -- and debian/control -- specifying the name, license, section, dependencies and description, among others. pdebuild is a wrapper around a couple of other wrappers: pdebuild uses debuild, which in turn uses dpkg-buildpackage, which use the dh_* scripts to run ./configure, make and make install. pdebuild also can use cowbuilder to run everything in the copy-on-write chroot to keep your environment clean.

Example

The following example is from the exercises at the symposium.

wget http://gnu.xl-mirror.nl/hello/hello-2.7.tar.gz
tar zxf hello-2.7.tar.gz
cd hello-2.7/
dh_make -f ../hello-2.7.tar.gz -s -c gpl3 && rm debian/{*.ex,*.EX,README.*}
DIST=sid pdebuild

The dh_make line uses -f ../hello-2.7.tar.gz to specify the location of the original source archive, -s to indicate we build a single binary, and -c gpl3 to specify the gpl3 license. After that, we remove the example and readme files that we don't need. Finally, we build the package for the sid distribution. This is done inside a clean copy of the base-sid.cow chroot.

Building packages from git repositories

When creating debian packages from upstream sources, you can use git to keep track of debian specific changes. A typical layout of your repository would be to have an upstream branch, containing the actual upstream sources; a pristine-tar branch, used to (re)create tar balls exactly (see below) and a master branch, combining upstream with your debian/ directory. With this layout, building the package can be done quickly with DIST=sid git-buildpackage. If you have configured git-buildpackage as above, then your package will be located in ../build-area.

git-buildpackage expects to have <package>_<version>.orig.tar.gz in ../tarballs/. A good way to keep the tarballs in your repository is using pristine-tar, which commits a tarball and subsequent delta's so that you don't need to store complete tarballs for each release. Putting a tarball in version control can be done with pristine-tar commit <package>_<version>.orig.tar.gz. Creating it for use with, for instance, git-buildpackage, can be done with pristine-tar checkout ../tarballs/<package>_<version>.orig.tar.gz. The advantage is that you can keep copies of the tarballs, but don't have to waste space for files that don't change between versions.

Final remarks

These are the basic steps to take when creating debian packages. The COWbuilder set-up is really convenient, especially if you create and test packages in a number of different settings (for example, you can have a default chroot for sid and wheezy and any Ubuntu release). Using git and git-buildpackage, you can keep track of your changes to the debian-specific files and automatically have it generate changelogs for you (using git-dch).

If you want these packages to appear in the debian repositories, you will probably have to contact someone willing to sponsor your upload. To get a sponsor, you usually have to make a request on the debian-mentors mailinglist (see the debian mentors FAQ for more information).

If you would rather host your own repository, we briefly touched on how to manage repositories with reprepro, but that is a topic for another post.

If you read this far, and didn't have your question answered, or have remarks about the post, you are welcome to e-mail me at jp at this domain, or contact me through twitter @jpvoosten.