Home | News | Download | Packages | Forum | Wiki | Github

Idea for XBPS Package Template Auto Updater, for simple packages


#1

To start off, if this has been proposed before or already exists please pardon my ignorance.

So, after looking at void-updates.txt it seems to me that we can improve the speed at which some SIMPLE packages update in our repo by making them update automatically, so here is the what/why/how I am proposing:

What:
A script that checks void-updates.txt once a week, and attempts to update the packages just by updating the version number and checksum and then running xbps-src on it. If xbps-src is successful, the script would then either email the maintainer, submit an issue on the github page, directly submit a PR for the updated package or some combination of the three. This would be done for all packages in void-updates.txt, excluding duplicates of the same package (ie it will only try to update packageXYZQ.n to packageXYZQ.n+newest; not to n+1, then n+2… all the way to n+newest).

Why:
Many packages ca be updated by simply incrementing the version number in the current template file and re-calculating the checksum. Yet they do not get updated because of various reasons with that package’s maintainer (busy, sick, no longer involved, etc).

How
A dedicated computer with some python and/or shell scripts and xbps-src. The script should only target simple packages that just need a version bump.

Some Thought on implementation and potential pitfalls.

  • Packages with patches should not be attempted as they are not simple, and the goal of this script is to reduce mindless work and auto-update simple packages.
  • Packages that have a daily release should be restricted to being run through the script once or twice a month.
  • Many packages will build on some architectures and fail on others. So we should build for a problematic arch that has a high failure rate, once again to ensure the script only auto-updates simple packages.
  • Even if the script succeeds and submits a PR, it should email the normal maintainer.
  • The script should keep a history of failed builds (package and exact version).
  • For builds that fail, the output of xbps-src should be available, and emailed to the maintainer.
  • Packages that fail to build due to missing or out of date dependencies should be logged into to history and not re-tried until thoes dependencies are met.
  • Packages that fail to build for issues unrelated to dependencies should be logged (package and exact version) and avoid running them again for at least a week. Packages that fail twice should be delayed a whole month before being tried again, and packages that fail 3 times (package and exact version) should be blacklisted from the script. Failing because of a build configuration problem or something similar is not a sign of a simple package.

Thus I propose we work on writing a script that does these things to help out Void.

What do you guys think? I think it would be worthwhile to have something like this helping out the maintainers (if they don’t already have something like it).


#2

Testing and making sure its a stable release and it doesn’t break anything else are the main issues while updating packages.
A human reading/processing the template and the build output has its benefits too, its easier to spot issues and improvements this way.


#3

Pinging the maintainer could be useful though.


#4

It’s a good idea, should be how Void works, and is how I first thought it did, because it’s so logical.

You ask what we think. I think it’s the only way forward with Void’s workhorse founder gone. He did the packaging work of ten men.

The management feedback thread missed this topic. It’s the only issue. The rest is fluff. I don’t care the distro brand name, the homepage URL, who controls IRC, or how donations happen, if at all.

I care about stale packages. We have atm fresh GIMP, Cinnamon, and RetroShare. While “too big” for a script, said script could free up time from small packages to reach big ones promptly. A script could likely handle point-point-point-plus-one version bumps.


(maxice8 alter) #5

more stuff to be added.

  • strict do_check() (if no tests, fail)
  • never send PR automatically, ask for maintainer confirmation
  • opt-in for maintainer
  • always fail if filepaths are added/removed (except for files that always change because their path includes their version, like python-* and *-dkms pkgs).
  • make it fail if anything is written to stderr

#6

Settings pertain to a package, not its maintainer. The way a package maintains shouldn’t change with personnel. New template fields could define maintenance appropriate for a package. An automatic PR, or even buildbot input, is the entire point if you ask me. You want as many packages as possible to be as automatic as possible, for as long as possible.


(maxice8 alter) #7

The maintainer is the final authority on the package besides a void linux member so it is per-maintainer in whatever way you slice it.


#8

Changing the version number, running xgensum -i foo and then ./xbps-src pkg foo is what the bot would do. That doesn’t help at all, the most time consuming part is to verify if everything works fine.

As you can see on github by looking at the number of open PRs updates are really really really not the problem.


#9

Changing the version number, running xgensum -i foo and then ./xbps-src pkg foo is what the bot would do. That doesn’t help at all, the most time consuming part is to verify if everything works fine.

As you can see on github by looking at the number of open PRs updates are really really really not the problem.

@Duncaen, OK understood. That makes sense.

Is there anything we can whip up to help you guys quickly check if packages work correctly? Maybe unit tests or something? Or is simply a matter of “write better PRs?”


#10

I don’t think its feasible to write unit tests for external packages.
We have now a do_check step in templates to run unit tests provided by the package but many packages don’t have any tests.

You can review PRs and test the packages locally, but that’s really hard for packages that you don’t use.

The best way to solve this is to get more maintainers that use the packages they maintain, this way merging packages would be easier because we know the maintainer wouldn’t just update the template and create a PR without any testing. With the current state where people update many packages I don’t know if they tested them or how thoroughly they tested them and that makes me hesitate to merge many PRs at once.


#11

I totally agree with you, and of course I understand your hesitation. :+1:


#12

Aren’t half of those PRs just exactly what you described a bot doing? Could Void skip the whole PR in these cases, and just let pkg users report problems? Did the mighty Void Linux founder himself test all the packages he packaged (hundreds, thousands)?