buildbot/phase2, branch v5

ci: add basic config checking with ruff

2023-05-16T14:52:33Z

Signed-off-by: Petr Štetiar

phase2: fix just another byte/string inconsistencies

2022-03-26T09:18:10Z

File "/phase2/master.cfg", line 203, in parse_feed_entry if parts[0].startswith("src-git"): builtins.TypeError: startswith first arg must be bytes or a tuple of bytes, not str Fixes: f3fcdbec877e ("phase2: fix obsolete assumption about feed methods") Signed-off-by: Petr Štetiar

phase2: handle missing change sources

2022-02-23T17:46:41Z

In commit f3fcdbec877e ("phase2: fix obsolete assumption about feed methods") we've fixed stuck buildmaster for master/phase2 builds, but the issue wasn't obvious at first sight and some digging was necessary. So lets make it obvious if/when it happens next time. Signed-off-by: Petr Štetiar

phase2: fix obsolete assumption about feed methods

2022-02-23T17:23:43Z

In commit 7fae1e5677e9 ("feeds: use git-src-full to allow Git versioning") there was `src-git` method of shallow clone replaced with `src-git-full` and this has caused issue with GitPoller configuration as it only assumed, that only `src-git` method is going to be used for feed updates. Fix currently stuck buildbot master by allowing usage of both Git update methods. Signed-off-by: Petr Štetiar

phase1,phase2: improve round robin builds

2021-07-21T19:56:13Z

There seems to be some issue with database updates, where database updates are asynchronous and thus the status update of the previous build (complete_at DB column) might take little bit longer, then preparation of the next/current build. This is issue during job prioritization as this might result in the build of the same builder twice in a row, for example: 2021-06-22 02:42:54+0000 [-] : build finished 2021-06-22 02:42:55+0000 [-] prioritizeBuilders: mipsel_24kc complete_at: 2021-06-21 20:17:14+00:00 2021-06-22 03:14:30+0000 [-] prioritizeBuilders: mipsel_24kc complete_at: 2021-06-22 02:42:54+00:00 Build finishes at 02:42:54, scheduler then asks for next build at 02:42:55, but the build still has the old complete_at timestamp 2021-06-21 20:17:14 instead of the correct one 2021-06-22 02:42:54, thus scheduling the build of oldest mipsel_24kc builder one more time. This is so far very promising workaround attempt which checks latest builder complete_at in builds table which seems to be updated faster, thus using greater complete_at value seems to work for now. References: https://github.com/buildbot/buildbot/issues/4592#issuecomment-801163587 Signed-off-by: Petr Štetiar

phase1,phase2: add file space usage reporting

2021-03-25T08:54:49Z

Add `du` step so it's clear how much space was used by the build. Signed-off-by: Petr Štetiar

phase1, phase2: exclude temporary source files from sourceupload step

2021-03-25T08:51:21Z

When the dl/ directory is shared with several workers, there can be temporary "*.hash" and "*.dl" files being created by other workers. These files should be excluded from the sourceupload step, otherwise it can cause a failure when rsync tries to read them. See for example: https://buildbot.openwrt.org/master/packages/#/builders/14/builds/18/steps/39/logs/stdio Signed-off-by: Baptiste Jonglez

phase1, phase2: don't mark the build failed if a non-critical step fails

2021-03-25T08:50:42Z

Some steps are informative or optional (haltOnFailure=False), so it makes no sense to mark the whole build as failed if such a step fails. Depending on the case, we either mark the build as "warning" (warnOnFailure=True) or as successful (warnOnFailure=False). Signed-off-by: Baptiste Jonglez

phase1,phase2: fix reconfig command by creating twistd.pid

2021-03-19T20:54:30Z

Currently it's not possible to reconfig buildmaster as twistd.pid is missing so fix it by creating one manually. Signed-off-by: Petr Štetiar

phase1,phase2: implement round robin builds

2021-03-17T08:33:29Z

Gather newest complete and not skipped build timestamps, reverse sort them and use that as builder priority, so the newest built targets are at the bottom. Signed-off-by: Petr Štetiar