Advice on maintaining a partial package mirror for personal use


(Jeffrey) #1

I’ve been doing a lot of experiments with void lately, which has mostly meant using xbps-install to do chroot installs over and over, often installing the same packages. Which normally wouldn’t be a big deal—I’m looking at about 200MB for a base install, except that my primary internet connection is a shared WiFi, which gets speeds up to about 1MB/s, but sometimes (often around midnight, I’m thinking someone has something scheduled to run at night) it degrades to about 10KB/s, and downloading all the packages I just downloaded is annoying.

I know I can use the -R switch to specify a local repository pointing the the package cache of the host, which is what I’ve been doing, but it meant that I had to install a bunch of packages onto the host that I didn’t want (like nvidia drivers on a computer that will never have nvidea graphics). It’s also an issue if I want to install conflicting packages (which I do—host has i3 and I want to install i3-gaps in the chroot.

I’d like one of two things, and I was wondering if there’s a solution I completely missed (I looked through all the xbps-tools), or if someone else has worked around a similar problem.

Either,

The software acts as a proxy, gives local files if they’re up to date, otherwise fetches newest packages and downloads them I thought about using a caching http proxy, but that means https is no longer an option. (http proxies proxy https using HTTP/1.1 CONNECT so they lose all ability to cache)

Or; I specify ahead of time all the packages I want to download, and the software gets those packages as well as the packages they depend on into a local repository. If I try to use xbps to install a package not in this directory, it fails (or if another repository is specified, it looks there—as you would expect). Later I’d like to issue a single command to look for package updates (like xbps-install -Su, except that packages are downloaded but not yet installed). This would be dead simple to do using wget, xbps-rindex and xbps-query, except for dependency and update management.


#2

You could use nginx as a caching proxy


(Jeffrey) #3

I’m not familiar with nginx—can you let me know if I’m understanding what this does/how to use it? I would use -R http://localhost/current with xbps, and nginx would rewrite requests like

GET /current/x86_64-repo-data
Host: localhost

to (over https)

GET /current/x86_64-repo-data
Host: repo.voidlinux.org

and the client (xbps) would have no idea localhost is acting as a proxy?

Thank you


#4

You could setup nginx on a server in your local network and it will save each requested package in the cache and following requests for the same package will be served from the cache.