buildbot/phase1, branch v1

phase1,phase2: improve round robin builds

2021-07-21T19:56:13Z

There seems to be some issue with database updates, where database updates are asynchronous and thus the status update of the previous build (complete_at DB column) might take little bit longer, then preparation of the next/current build. This is issue during job prioritization as this might result in the build of the same builder twice in a row, for example: 2021-06-22 02:42:54+0000 [-] : build finished 2021-06-22 02:42:55+0000 [-] prioritizeBuilders: mipsel_24kc complete_at: 2021-06-21 20:17:14+00:00 2021-06-22 03:14:30+0000 [-] prioritizeBuilders: mipsel_24kc complete_at: 2021-06-22 02:42:54+00:00 Build finishes at 02:42:54, scheduler then asks for next build at 02:42:55, but the build still has the old complete_at timestamp 2021-06-21 20:17:14 instead of the correct one 2021-06-22 02:42:54, thus scheduling the build of oldest mipsel_24kc builder one more time. This is so far very promising workaround attempt which checks latest builder complete_at in builds table which seems to be updated faster, thus using greater complete_at value seems to work for now. References: https://github.com/buildbot/buildbot/issues/4592#issuecomment-801163587 Signed-off-by: Petr Štetiar

phase1,phase2: add file space usage reporting

2021-03-25T08:54:49Z

Add `du` step so it's clear how much space was used by the build. Signed-off-by: Petr Štetiar

phase1, phase2: exclude temporary source files from sourceupload step

2021-03-25T08:51:21Z

When the dl/ directory is shared with several workers, there can be temporary "*.hash" and "*.dl" files being created by other workers. These files should be excluded from the sourceupload step, otherwise it can cause a failure when rsync tries to read them. See for example: https://buildbot.openwrt.org/master/packages/#/builders/14/builds/18/steps/39/logs/stdio Signed-off-by: Baptiste Jonglez

phase1, phase2: don't mark the build failed if a non-critical step fails

2021-03-25T08:50:42Z

Some steps are informative or optional (haltOnFailure=False), so it makes no sense to mark the whole build as failed if such a step fails. Depending on the case, we either mark the build as "warning" (warnOnFailure=True) or as successful (warnOnFailure=False). Signed-off-by: Baptiste Jonglez

phase1,phase2: fix reconfig command by creating twistd.pid

2021-03-19T20:54:30Z

Currently it's not possible to reconfig buildmaster as twistd.pid is missing so fix it by creating one manually. Signed-off-by: Petr Štetiar

phase1,phase2: implement round robin builds

2021-03-17T08:33:29Z

Gather newest complete and not skipped build timestamps, reverse sort them and use that as builder priority, so the newest built targets are at the bottom. Signed-off-by: Petr Štetiar

phase1,phase2: shuffle order of builds

2021-03-15T21:47:41Z

It seems that once there are new commits, buildbot restarts building all targets in alphabetical order. The current logic pretty much disables the building of the alphabetically last targets, so until there is proper solution just workaround it by randomly shuffling the builders list. References: https://github.com/buildbot/buildbot/issues/5193 Signed-off-by: Petr Štetiar

treewide: use worker term

2021-03-12T13:36:13Z

[Placeholder for your politically correct commit description] Suggested-by: Paul Spooren Signed-off-by: Petr Štetiar

builtmaster,phase1: add missing services dict

2021-03-05T15:11:31Z

Fixes following error: 2021-03-05 11:00:54+0000 [-] error while parsing config file: ... builtins.KeyError: 'services' Signed-off-by: Petr Štetiar

buildmaster: replace horizons with JanitorConfigurator

2021-03-05T15:11:31Z

The old c['logHorizon'] way of configuring is not supported anymore, we should use JanitorConfigurator. A new __Janitor builder will be created to help keep an eye on the cleanup activities and for that we need buildbot-worker package as well. Signed-off-by: Petr Štetiar