Package maintainer guide

This article describes the steps needed to start working on our repos. If you are going to make a new -libre replacement for some package check our guide for creating libre replacements.

Note:

As a package maintainer, you are part of the extended community team (the 'Community' role on the issue tracker). Please register yourself on the issue tracker, so that you can be assigned to tasks related to your packages, and manage their progress.

As a full team member (the 'Hacker' role on the issue tracker), please subscribe to and read the 'dev' mailing list regularly, check for email sent to the address(es) as listed in hackers.git and PKGBUILDs regularly, and keep your GPG signing key from expiring. Whenever you have the time, try to hang around the #parabola, check the 'assist' mailing list, issue tracker, and web forum, and give user support for the software that you are familiar with.

1 Installation

Install libretools. It will also install ssh, rsync, git, and other tools needed to package.

# pacman -S libretools

Note: If you are upgrading libretools, you will need to merge your /etc/libretools.conf.pacnew.

And in the case you are not using Systemd (e.g. OpenRC) see #Compiling packages without Systemd

2 Configuring everything

2.1 Set GPGKEY

Note: Only full team members must setup a GPG key. It is required for librerelease. This section is otherwise optional.

Create your GPGKEY.

If you use a fancy-pants, bloated, key manager (like the one for GNOME), then the steps here might be different; maybe that tool will do this for you. But, the GPGKEY variable will need to be set. It can be set as an environment variable, or in /etc/makepkg.conf, or in ~/.config/pacman/makepkg.conf.

To avoid having to configure it in multiple places, lukeshu likes to parse the default-key setting in his gpg.conf, and set an environment variable to that; by sticking this you can set it as an environmental variable, getting the value formI prefer to set it as an environmental variable by putting the following in my ~/.profile:

File: ~/.profile

if [ -z "$GPGKEY" -a -f "${GNUPGHOME:-${HOME}/.gnupg}/gpg.conf" ]; then
	export GPGKEY=`sed -nr 's/^\s*default-key\s+//p' "${GNUPGHOME:-${HOME}/.gnupg}/gpg.conf"`
fi

2.2 Configure SSH

Note: Only full team members must setup an SSH key. It is required for librerelease. This section is otherwise optional.

Create your SSH Keys.

You may need to configure your SSH IdentityFile, if you use a different key than ~/.ssh/id_rsa for Parabola:

File: ~/.ssh/config

Host parabola.nu *.parabola.nu
        IdentityFile ~/.ssh/YOUR-PARABOLA-KEY # if you use a different key than ~/.ssh/id_rsa for Parabola

Whether you use id_rsa or a different name for your key; send the .pub file (so ~/.ssh/id_rsa.pub by default) to the Parabola developer mailing list, so it can be placed in hackers.git, giving you access.

Note: The key and the email should be signed with your GnuPG key. If you use the default key name, you can use gpg --armor --sign ~/.ssh/id_rsa.pub to sign your SSH key.

2.3 Configure git

Note: Only full team members must setup git in this way. It is required for committing to abslibre. This section is otherwise optional.

You may configure your ~/.gitconfig as follows:

File: ~/.gitconfig

[color]
	ui = auto
[push]
	default = simple
[user]
	name = Your Name
	email = your@email.com
	signingkey = [Your GPG key fingerprint here]
[commit]
	gpgsign = true

3 Packaging Environment

Packaging can be messy and become a burden (like any repetitive task). Libretools exist to make it lighter, and to organize your build recipes and build chroots.

3.1 Working directory

Before you can use the packaging-related parts of libretools, you'll need to create a working directory. The working directory includes the abslibre build recipes, upstream source-balls, and a release staging area. The default location of this directory is ~/packages; but the location is configurable in /etc/libretools.conf: WORKDIR.

There's a handy createworkdir script which will populate the $WORKDIR target directory, ready to use. The directory will initially contain a complete, checked-out git clone of the abslibre.git tree, with the appropriate remote URLs pre-configured.

$ createworkdir

You can find more info about the structure of this directory tree here.

3.1.1 Updating ABSLibre

When building a package, you need an up-to-date copy of the recipe for the package. To update abslibre in your $WORKDIR (~/packages by default) just:

$ cd $WORKDIR/abslibre && git pull

3.2 Build Chroots

You'll need a clean packaging environment to make sure the packages you make work in every parabola system despite what settings and packages you have on yours. That's what a chroot is for.

You need to create a chroot dir for building packages inside. This will also help you identify missing dependencies and keep your every day system clean too.

When creating a build chroot, you will need to ensure that it is created with the appropriate default pacman.conf and makepkg.conf configuration files for the target architecture. The -A option ensures that is done sanely, for the general use-case.

If you are using libretools only for yourself, you can create a build chroot with the command below:

 $ sudo librechroot -A x86_64 make

If the chroot already exists and is up to date,librechroot make will print nothing:

Every build chroot actually has a unique name though. In the example above, that will be 'default'. Most people can ignore that caveat; but those who are publishing to the Parabola repos will need to mind it, because we require all packages to be built for all supported arches, whenever applicable and possible. If you are publishing to the Parabola repos, you will need multiple per-arch chroots with distinct names; so use the following commands instead:

 $ sudo librechroot -A armv7h -n armv7h make
 $ sudo librechroot -A i686   -n i686   make
 $ sudo librechroot -A x86_64 -n x86_64 make

The value of the -n name is arbitrary. Use anything you like; but the names must be distinct for each chroot.

If the specified (or unspecified default) chroot was not created previously, it will use the default configuration files for the specified target architecture. If the specified (or unspecified default) chroot was created previously, it will re-use the existing configuration files, already installed in the chroot. So, neglecting to pass explicit names to librechroot make may yield an unexpected result in the latter case. Also, any command which requires a chroot will automatically create one if it doesn't exist. That can be equivalently problematic, as the default is x86_64. So, it is best to prepare (and name) all of your chroots before building any packages.

If you want to rename a chroot, you should delete the existing one, and create a new one.

$ sudo rm -rf /path/to/chroot/

The default location for chroots is /var/lib/archbuild/<CHROOT_NAME>.

Note: Chroots can often become corrupted or insane. It is often necessary to destroy the chroot in that way, and to re-create it fresh.

Also, note that we explicitly used sudo, instead of giving the usual #, to signify running it as root. It is essential to run librechroot, libremakepkg, librestage, and librerelease as a login user via `sudo`, in order to setup the important environment variables such as $LIBREHOME. This is because part of the path to the chroot is based on the username, as is the default path to $WORKDIR. See librechroot help for more details.

3.2.1 Updating the chroot

You of course, want to keep the packages in the chroot up-to-date.

 $ sudo librechroot update

If you have multiple chroots, use:

 $ sudo librechroot -n <CHROOT_NAME> update

3.2.2 Reset chroot to a clean state

In order to reset the chroot to a clean state, use:

$ sudo librechroot clean-pkgs
$ sudo librechroot clean-repo

If you have multiple chroots, use:

$ sudo librechroot -n <CHROOT_NAME> clean-pkgs
$ sudo librechroot -n <CHROOT_NAME> clean-repo

clean-pkgs is normally not necessary. libremakepkg will do that automatically. However, it is a best to clean the in-chroot package repo, whenever it is not needed. The in-chroot repo (at <CHROOT_ROOT>/repo) retains recently built packages, for use as dependencies. This is often necessary for packages with strongly-coupled dependencies. These relationships and build order are often expressed as comments in the PKGBUILD. Once all of the strongly-coupled packages are built though, it is best to publish them to the main repos, and to clean them from the chroots, before moving onto another package set. They will accumulate otherwise; but worse, that they will be preferred by any dependent, which may lead to unsatisfied dependencies in the published package. For example, if some /repo package was destined for a *-testing repo, it would be incorrect to build a package against it, which is destined for a non *-testing repo.

4 Building packages

Before making a package, you should update ABSLibre and your chroot, as explained in the previous section.

Make sure that you have configured the PACKAGER and GPGKEY variables in /etc/makepkg.conf. If you are building packages on a remote build server, it is not strictly necessary to set GPGKEY. However you will need to setup libretools in this way on your local machine, download the build packages, and run librestage/librerelease locally. That is best practice; because you should test out the builds before publishing them.

Note: Only full team members must set the PACKAGER and GPGKEY variables. They are required for publishing to the package repos.

Change directory (cd) into the abslibre tree where the interesting PKGBUILD is $WORKDIR/abslibre/<REPO>/<PKGBASE>, and run libremakepkg (just as you would for the standard makepkg).

$ sudo libremakepkg

or, if you have multiple chroots, pass the -n option, just as with librechroot.

$ sudo libremakepkg -n <CHROOT_NAME>

Warning: Do not build packages in any directory under $WORKDIR/staging! In fact, do not ever `cd` into staging. Keep it pristine. If it becomes littered, rm -rf $WORKDIR/staging is always perfectly safe (erm ... unless librerelease is running).

Sometimes libremakepkg will fail with an error of the form:

 foo-1.2.3.tar.gz ... FAILED (unknown public key ABCDEFGHIJKLMNOP)

That it normal. You will need to retrieve that key into your local user's GPG keyring:

$ gpg --recv-key ABCDEFGHIJKLMNOP

and to try again to build the package. You may need to repeat these steps, if libremakepkg complains again about another missing key.

After the package is built, it will automatically be added to a local pacman repository inside the chroot, so it can be used as a dependency for other packages which you may build next.

Check the libremakepkg page for a description on how libremakepkg works (or run libremakepkg help to see the available options).

5 Creating packages

Please follow this guide: Creating -libre packages.

Also, take a look at Common packaging mistakes.

6 Uploading packages

6.1 mksource() and intermediate source-balls

Some packages will require an intermediate (un-published) source-balls. The actual procedure may vary in each case; and some utilize the mksource() mechanism to automate this.

The libremakepkg program will generate the intermediate libre source-ball automatically, for any PKGBUILD with a mksource() function; and librerelease will later upload it to the expected location on the server.

Some other packages may require the intermediate source-ball to be created manually, before the package can be built. If the libre changes are extensive, and would benefit from VCS beyond abslibre, it may be reasonable to create a new project on git.parabola.nu for the problematic sources, rather than using the mksource mechanism; and to release the libre source-ball via that project. Examples of that:

• linux-libre
• u-boot-libre (done through the libreboot project)

Most Parabola native packages (including the libretools package itself) are managed in that way. The code is stored in Parabola's VCS; and release source-balls are not published until needed by a PKGBUILD. For example, to generate the libretools source-ball expected by the libretools PKGBUILD, ensure that a versioned git tag exists of the form: vYYYYMMDD (e.g. v20190907) on the current tip of the master branch of the libretools git repo; then build the source tarball with a command of this form:

  make dist LIBRETOOLS_VERSION=YYYYMMDD

It is sufficient to copy the artifact (the intermediate source-ball) to your local abslibre build directory, before building the package. Afterward, librerelease will upload it to the other/ directory on the repo server, along with the built package.

6.2 Uploading PKGBUILD

All of your packages will need to include a PKGBUILD uploaded to the abslibre.git.

You can simply push any updates as follows:

$ git add [editedfile(s)]
$ git commit -m "I changed something, just letting everyone know"
$ git push

6.3 Uploading binary

6.3.1 Everything on the same machine

If you have the corresponding gpg and ssh keys on the machine you used to build the packages, to upload packages to our repo, first tell librestage to add the package to the list of packages that will be uploaded:

$ librestage repo-to-put-the-package-in # stage the package to be uploaded

Then verify if the packages you are about to upload matches what you would expect. Here I added the debootstrap package to the list of packages that will be uploaded:

$ librerelease -l
  -> libre
     debootstrap-1.0.110-1.par1-any.pkg.tar.xz
  -> sources
     parabola/debootstrap-1.0.110-1.par1-any.src.tar.gz

Then if everything looks good, you can then upload the package:

$ librerelease # upload all staged packages

By default, librerelease will also clean the local pacman repository in your chroot (sudo librechroot clean-repo). If you don't want it to do this, change the value for HOOKPOSTRELEASE in /etc/libretools.conf.

6.3.2 separate machines for building and ssh/gpg

If you have the corresponding SSH and gpg keys on a different machine, you will need to copy the directory you ran libremakepkg on the machine that has the gpg and ssh keys.

For instance if you worked on the libre/debootstrap package you will need copy the debootstrap directory in the computer that has your keys.

When you do that, It's a good practice to put it in a directory that has the name of the repository (here 'libre') that your pakcage should go in.

This is to avoid some mistakes: the librestage command (without arguments) will autodetect the repository the package should go in based on the parent directory name (here 'libre'). So if you forget to add the repository as an argument, and that your parent directory has a name that differs from the repository (like 'rsync' for instance), it may go in the wrong repository or produce an error.

You also need libretool installed and configured on the computer that has your corresponding SSH and gpg key.

If it's not already done you can install it with this command:

 # pacman -S --needed libretools

You will also need to make sure that:

• GPGKEY is setup correctly as described in the corresponding section on the GPGKEY configuration
• SSH is configured as described in the corresponding section on SSH configuration

When that is done, uploading the package is not very different from the situation where you have everything on the same computer:

$ librestage repo-to-put-the-package-in # stage the package to be uploaded
$ librerelease -l # check if the packages to be uploaded are the ones you expect
$ librerelease # upload all staged packages

7 Tips and Tricks

7.1 Building i686 packages from an x86_64 machine

See Building 32-bit packages on a 64-bit system

7.2 Building armv7h packages from an x86 (i686 and/or x86_64) machine

Libretools added support for building armv7h packages on an x86 system. See Building armv7h packages on a x86 system#Libretools.

7.3 Releasing several packages at the same time

Originally, librerelease would decide where to put a package and upload it to the repo server right away. Now, librestage does the first part and librerelease does the second, so in practice, you can package several packages and stage them for uploading later. Before running librerelease, make sure you run gpg-agent, so you'll only have to enter your password once.

7.4 Staging one package for several repositories

Take note librestage accepts several repo names as arguments. If you do so, it will stage the package(s) on all of them, allowing you to upload packages in any number of repos you want. It's not a feature to abuse on, but it's useful while we migrate to [libre] and [libre-testing] repos... you'll have to release into [core] and [libre], or [testing] and [libre-testing] at the same time, for instance.

7.5 MD5sum checking

If you change the sources array on a PKGBUILD, you'll have to change the md5sums array to reflect this. Remove it from the PKGBUILD and use this command to generate md5sum for your packages:

$ makepkg -g >> PKGBUILD

7.6 Building packages from source control

The librefetch tool exists to help with building packages from source control. If you are familiar with the old mksource or SRCBUILD systems, those work; just add https://repo.parabola.nu/other/$pkgname/$pkgname-$pkgver.tar.gz to your source array, and librefetch will automatically build it for you. Of mksource and SRCBUILD, I recommend mksource, it is much simpler. For example, here is a portion of my PKGBUILD to build a bleeding-edge version of wmii:

File: PKGBUILD

source=("https://repo.parabola.nu/other/~lukeshu/wmii/wmii-hg-${pkgver}.tar.gz")
md5sums=('1ab85d01467e5e53c9b16ca19cca9824')

mkdepends=("mercurial")
mksource=("wmii-hg${pkgver}::hg+https://code.google.com/p/wmii/#revision=${pkgver}")
mkmd5sums=('SKIP')
mksource() {
	# don't include this precompiled binary (probably included by accident)
	rm "${srcdir}/wmii-hg${pkgver}/lib/libutf/libutf.a"
}

Use librefetch -g to generate the mkmd5sums array, and makepkg -g to generate the plain md5sums array.

librestage/librerelease uploads the generated tarball for you.

7.7 SSH connection to Parabola

Make sure the permissions in your ~/.ssh directory are 700:

$ chmod 700 ~/.ssh

Also make sure the permissions of your SSH key is 600:

$ chmod 600 ~/.ssh/*

If you added the repo host for your ssh config, you can just:

$ ssh repo@repo

and to copy files using scp:

$ scp local-file repo@repo:remote-path

7.8 Compiling packages without Systemd

librechroot (part of nonsystemd/libretools) uses chroot-nspawn for compiling in chroots.

Chroot-nspawn is a systemd-nspawn wrapper for chroot. It combines features from chroot and unshared. To use it, simply install chroot-nspawn package.

# pacman -S chroot-nspawn

7.9 Maintaining Package on time to stay up to date!

THIS IS WITH AUR3 ... WE HAVE TO FIND THE WAY WITH AUR4!!!

At the time you have installed libretools and start to build packages the next step is to maintain them up to date.

To make it easy, there is a script that tells you when your package is outdated.

Firstly, what you have to do is to clone the aur-git:

git clone git://pkgbuild.com/aur-mirror.git

Then install expac if not already done

pacman -S expac

Then copy that script in a filename you prefer, for example, "out-of-date":

pattern="$1"
cd aur-mirror
expac -S '%r/%n %v %b %p' -t '%F %T %z' | grep "^pcr" | cut -b 5- | awk -v pattern="$pattern" "\$1 ~ pattern" | while IFS= read -r line
do
  set -- ${line}
  if [[ -e ${1}/PKGBUILD ]] ; then
    # Skip plowshare because its pkgver is too long
    [[ ${1} =~ plowshare ]] && continue
    source ${1}/PKGBUILD && [[ "${2}" < "${pkgver}-${pkgrel}" ]] && {
      printf "%-40s%15s   %-20s   %-50s\n" "${1}" "${2}" "${3} ${4} ${5}" "${*: 6}"
      date="$(git log -1 --format='%ci' -- "${1}/PKGBUILD")"
      printf "%-40s%15s   %-50s\n" "${pkgname}" "${pkgver}-${pkgrel}" "${date}"
      echo --
    }
  else
    echo "${1} is not in AUR"
    echo --
  fi
done

Save it and chmod +x it:

chmod +x out-of-date

Now you have just to execute it using "^a" to list all in alphabetical order the package:

./out-of-date ^a

Now you are an "up-dated" maintainer!