Home | News | Download | Packages | Forum | Wiki | Github

Why so few mirrors of Void packages?

(The_Fred) #1

Is it due to bandwidth constraints for the build machines? Or is it a matter of just querying more mirror server admins and asking if they will host? Other constraints? See below.

I don’t know much about how mirroring actually works.

Here are my guesses:

Scenario A:

  • when a build completes the new packages are uploaded to master mirror
  • from the master mirror other mirrors then pull any new packages
    (probably using rsync but maybe just curl/wget/etc)

Scenario B:

  • when a build completes the new packages are uploaded to all mirrors
    one after the other until all mirrors get updated

Scenario C:

  • none of the above
  • mirrors pull from the build machines using rsync (or curl/wget)

(The_Fred) #2

I read on distroowatch that the packages are uploaded as they are built so that answers one of the questions here.

I also found a mirror (think it is at princeton) that is only partial. Maybe it’s abandoned? A mirror that says it’s unofficial at mirror.frgl.pl

The gatech mirror is often slow or unresponsive. Clarkson no longer has rsync for void even though it says it does.

rsync mirror.clarkson.edu::

parrot          Parrot
gentoo          Gentoo
gentoo-distfiles        Gentoo
gentoo-portage  Gentoo Portage
centos          CentOS
gnu             GNU Software
tdf             The Document Foundation
dag             DAG/RPMForge/RepoForge
repoforge       DAG/RPMForge/RepoForge
rpmforge        DAG/RPMForge/RepoForge

This site has an interesting software mirror interface that shows the status of the different mirrored repos/distros.

(The_Fred) #3

FYI there is no reliable rsync mirror as the gatech one fails too often.

(Jared) #4

Hey there!

I’m the Clarkson University Mirror co-operator. Looking at the rsyncd config file, you are right, void is listed in the website, but is not reflected in the configs. Oops!

I added it to our current configuration (and hopefully didn’t do anything bad like expose private files I may not know of - but it looks similar to the “gatech” mirror’s directory structure so it’s probably fine).

Have a nice day,

Jared D.

(Michael Aldridge) #5

I’m going to dig up a dead thread here, not often, but its worth putting the right information out there.

I manage the build infrastructure across the Void fleet, here’s how it works:

  1. A PR is accepted on GitHub
  2. A webhook fires to the scheduler at build.voidlinux.eu
  3. All build machines are asked to begin building.
  4. Packages are synced back to build.voidlinux.eu
  5. Packages are signed and repodata- is updated
  6. Mirrors around the world obtain the packages via rsync.

Please note that steps 3-5 are being reworked and so this may be out of date soon.

As for why there are few mirrors, well we don’t really need that many. We’re not saturating the resources we have yet so it doesn’t make sense to spend money on more of them. If you’re in the states, I think the fastest mirror is the UT Dallas one which is iirc on a 5Gbit pipe. We have another mirror in the US which is coming online now (its up but does not necessarily have DNS pointing to it yet). As far as mirrors in other locales, if you are at a site which has IPv4 and IPv6 connectivity, unlimited transit, and the capacity to host a VM for us with at least 1.5T of storage, feel free to shoot me an email.

@jrddunbr please shoot me an email at maldridge[at]voidlinux.eu with a reliable email I can use as a site contact for your mirror.


How do you find time for all that while working at Google? Does the rework involve Google servers or services? Thanks for all you do.

(Michael Aldridge) #7

I enjoy what I do, both during the day at work and with Void.

As far as putting Void on anything to do with Google, I’d love to put a mirror in the cloud but that turns out to be quite expensive in terms of bandwidth. The rework for Void involves primarily making the builders smarter using software developed by other members of the project, and trying to get the infrastructure to “call for help” rather than someone having to notice what is wrong.

Void’s servers are a mix of colo’d boxes at Hetzner and a few other places, as well as a few VMs generously donated from a few small VPS providers. We’re not really the type of project that is easily sponsor-able by the large providers right now.

The only other major refactor going on right now is to do with DNS, which will be on GCP DNS, but that’s more because we want to be able to have multiple people with access and our current DNS provider can’t really do that (how this isn’t a standard feature yet I’m not really sure). More information on this is in issue #10 on void-infrastructure.