Pacman2pacman

From ParabolaWiki
Jump to: navigation, search

1 What Pacman2pacman is

Pacman2pacman is a plugin for pacman which allows it to download from torrent and HTTP mirrors simultaneously, and to constantly seed downloaded packages back up. This will reduce load on our servers and make Parabola more autonomous.

2 How it works

transmission-daemon runs at all times to seed packages and download them when needed.

A script `pacman2pacman-get' is put in pacman.conf's `XferCommand' setting. This script downloads a .torrent file for that package, modifies it to use your selected mirror as a webseed, and tells transmission-daemon to download the package by bittorrent.

pacman2pacman-get is written in bash and uses transmission-remote from the transmission-cli package to communicate with transmission-daemon. A progress bar is displayed as the file downloads by polling transmission-daemon to know how far downloaded the file is.

3 Installing

Make sure you have the pcr repo enabled in pacman.conf, then: pacman -S pacman2pacman

Then activate transmission (if it is not already) and enable it on start.

sudo systemctl start transmission
sudo systemctl enable transmission

Put the following into /etc/pacman.conf, under the [options] section:

XferCommand = /usr/bin/pacman2pacman-get %u %o

Once you have installed or updated some stuff, you can see the packages being seeded with transmission-remote -l

You can provide feedback by speaking to Xylon on IRC or opening bugs on: https://labs.parabola.nu/projects/pacman2pacman/issues

4 How the pacman mirrorlist is used

The pacman mirrorlist is still used with pacman2pacman, for two things:

  1. the download URL that pacman gives to the pacman2pacman.-get script (through XferCommand) is used as a webseed for the torrent.
  2. pacman2pacman needs the .torrent file to give to transmission. this is about 400 bytes. pacman2pacman-get chooses 3 random mirrors from your mirrorlist and tries to load the torrent file from all three simultaneously. The first one to respond is used, if none respond in 6 seconds then the download is done with HTTP instead.

5 Use cases

5.1 Local peer discovery

If you activate local peer discovery in transmission, the computers that are on the local network will then be able to participate, which will result in way bigger download speed because the local network is usually way faster.

6 Notes

Xylon is running opentracker on taskenizer.crabdance.com to track the torrents. Regard the stats page: taskenizer.crabdance.com:6969/stats.

fauno has suggested a transmission-daemon settings file with some security options enabled: https://github.com/fauno/duraskel/blob/develop/src/.config/transmission/settings.json

fauno has also provided a list of links for transmission optimization: http://www.pps.univ-paris-diderot.fr/~jch/software/bittorrent/tcp-congestion-control.html http://falkhusemann.de/blog/2012/07/transmission-utp-and-udp-buffer-optimizations/ http://blog.lxgr.net/posts/2013/01/28/my-openwrt-setup/ https://github.com/dtaht/deBloat/ http://www.bufferbloat.net/projects/bloat/wiki/Wiki

We plan to make a system for VPS owners to seed the most popular packages: PDCS

7 Possible massive performance enhancement

ATM pacman downloads all packages sequentially. It would be much faster if we could make it download all packages in parallel.

The plan could be: when pacman asks pacman2pacman-get to download a file, it adds the torrent to transmission and returns instantly. When it gets to the last download, pacman2pacman-get blocks and shows a progress meter for all files. In order to do this we would have to modify pacman to give some indication to pacman2pacman-get when it gets to the last download.

Implementation idea: patch pacman, modify pacman_upgrade in src/pacman/upgrade.c so before downloading all packages it calls a configuration-specified optional program with all URLs in arguments; write such script for pacman2pacman that adds all torrents to transmission.