build: fix pkg-config detection when inside of a nix-shell The output of command_all when inside a nix-shell looks like the below where /usr does not match: ➜ scripts/command_all.sh pkg-config /nix/store/ifr6srqgpvygd5vp14748d109ri31isv-pkg-config-wrapper-0.29.2/bin/pkg-config Signed-off-by: Sandro Jäckel <sandro.jaeckel@gmail.com>
treewide: add ORIG_PATH variable Add a variable that stores the original value of $PATH in the host system's shell, before Make alters it. This can be useful for when it is necessary to ignore symlinks and programs made by the build system. Define this new variable before all instances of 'export PATH:=' or similar. Signed-off-by: Michael Pratt <mcpratt@pm.me>
treewide: derive host and hostpkg path from STAGING_DIR STAGING_DIR may be provided from command line. We currently hardcoded STAGING_DIR_HOST and STAGING_DIR_HOSTPKG to the default location but we currently have some relocatable patch that derive the path from STAGING_DIR. Fix this and correctly derive STAGING_DIR_HOST and STAGING_DIR_HOSTPKG from STAGING_DIR. The intention is to fix inconsistency from the relocatable patch and the use of STAGING_DIR_HOST that is always hardcoded. This with a wrong configuration may end up in broken state with some host tools expecing a PATH from STAGING_DIR and others using library from the default staging_dir/host path. To save downstream project the original implementation is saved while fixing the inconsistency between patch and .mk. Signed-off-by: Christian Marangi <ansuelsmth@gmail.com>
Makefile: fix stray \ warnings with grep-3.8 We simply grep for "/usr". So no need for "-E" or "\/". Furthermore, in the new grep versions this creates warnings. As written in the grep-3.8 announcement: Regular expressions with stray backslashes now cause warnings, as their unspecified behavior can lead to unexpected results. For example, '\a' and 'a' are not always equivalent <https://bugs.gnu.org/39678>. Fixes warnings in the form of: grep: warning: stray \ before / Signed-off-by: Nick Hainke <vincent@systemli.org>
Makefile: replace head call with grep's -m head is not necessary here. Signed-off-by: Rosen Penev <rosenp@gmail.com>
build: don't remove BUILD_LOG_DIR in _clean targetclean should not remove BUILD_LOG Fixes: db34b93331e9 (add a version that can be bumped to force toolchain/target rebuild) Signed-off-by: Huangbin Zhan <zhanhb88@gmail.com>
treewide: drop use of which Ubuntu started to flag which as deprecated and it seems which is not really standard and may vary across Distro. Drop the use of which and use the standard 'command -v' for this simple task. Which is still present in the prereq if some package/script still use which. A utility script called command_all.sh is implemented that will just mimic the output of which -a. Signed-off-by: Ansuel Smith <ansuelsmth@gmail.com>
toolchain: add a version that can be bumped to force toolchain/target rebuild This can be used to ensure that the compiled code is up to date, when something important changes in the toolchain. A recent example of this is the gcc 11 fix for a code miscompilation issue Signed-off-by: Felix Fietkau <nbd@nbd.name>
Revert "build: replace which with Bash command built-in" This reverts commit c7aec47e5e3a3ff7b5fdaa11cd1e62cae6746acb. The original commit replaces 'which' with 'command'. Sadly most of them are not equivalent and for 'which -a', there is no easy replacements that would not reimplement PATH parsing logic. Hence revert. Keeping a dependency on which is absolutely fine. Signed-off-by: Clemens Fruhwirth <clemens@endorphin.org>
Revert "build: use config.site generated by autoconf-lean, drop hardcoded sitefiles" This reverts commit f439e291304a93b982e912dc91b80ca950a594f3. Signed-off-by: Daniel Golle <daniel@makrotopia.org>
build: use config.site generated by autoconf-lean, drop hardcoded sitefiles Signed-off-by: Felix Fietkau <nbd@nbd.name>
build: use SPDX license tags The license folder is a core part of OpenWrt and all GPL-2.0 licensed. Use SPDX license tags to allow machines to check licenses. Signed-off-by: Paul Spooren <mail@aparcar.org> [rebase, keep some Copyright lines, sharpen commit message] Signed-off-by: Adrian Schmutzler <freifunk@adrianschmutzler.de>
build: use ccache -C for cleaning the cache This keeps the configuration, like the size of the cache, and the statistics intact. Move the removal of the cache directory to the distclean target, but only delete the .ccache directory inside of our build tree, as we should not mess with a user-configured external ccache directory this destructively. Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
build: replace which with Bash command built-in `which` utility is not shipped by default for example on recent Arch Linux and then any steps relying on its presence fails, like for example following Python3 prereq build check: $ python3 --version Python 3.9.1 $ make /bin/sh: line 1: which: command not found /bin/sh: line 1: which: command not found /bin/sh: line 1: which: command not found ... Checking 'python3'... failed. ... Fix this by switching to Bash builtin `command` which should provide same functionality. Fixes: FS#3525 Signed-off-by: Petr Štetiar <ynezz@true.cz>
build: use CCACHE_DIR for ccache directory It is being set by rules.mk with the same expression. Signed-off-by: Sven Wegener <sven.wegener@stealer.net>
build: improve ccache support Set CCACHE_DIR to $(TOPDIR)/.ccache and CCACHE_BASEDIR to $(TOPDIR). This allows to do clean and dirclean. Cache hit rate for test build after dirclean is ~65%. If CCACHE is enabled stats are printed out at the end of building process. CCACHE_DIR config variable allows to override default, which could be useful when sharing cache with many builds. cacheclean make target allows to clean the cache. Changes from v1: - remove ccache directory using CCACHE_DIR variable - remove ccache leftovers from sdk and toolchain make files - introduce CONFIG_CCACHE_DIR variable - introduce cacheclean make target Signed-off-by: Roman Yeryomin <roman@advem.lv>
build: refactor JSON info files to `profiles.json` JSON info files contain machine readable information of built profiles and resulting images. These files were added in commit 881ed09ee6e2 ("build: create JSON files containing image info"). They are useful for firmware wizards and script checking for reproducibility. Currently all JSON files are stored next to the built images, resulting in up to 168 individual files for the ath79/generic target. This patch refactors the JSON creation to store individual per image (not per profile) files in $(BUILD_DIR)/json_info_files and create an single overview file called `profiles.json` in the target directory. Storing per image files and not per profile solves the problem of parallel file writes. If a profiles sysupgrade and factory image are finished at the same time both processes would write to the same JSON file, resulting in randomly broken outputs. Some target like x86/64 do not use the image code yet, resulting in missing JSON files. If no JSON info files were created, no `profiles.json` files is created as it would be empty anyway. As before, this creation is enabled by default only if `BUILDBOT` is set. Tested via buildroot & ImageBuilder on ath79/generic, imx6 and x86/64. Signed-off-by: Paul Spooren <mail@aparcar.org> [json_info_files dir handling in Make, if case refactoring] Signed-off-by: Petr Štetiar <ynezz@true.cz>
build: clean menuconfig utility as part of dirclean When sharing a common build directory between different build platforms eg. macos v docker based linux v virtual machine, a 'make dirclean' isn't quite enough to clean all the platform related binaries. The 'conf' and 'mconf' aka 'make menuconfig/defconfig & friends' utilities remain. Clean those as part of 'dirclean' so they get rebuilt for the current platform on the next 'make' Signed-off-by: Kevin Darbyshire-Bryant <ldir@darbyshire-bryant.me.uk>
build: add buildinfo as single Makefile target The prepare target was added some 11 years ago to build tools and toolchain and was recently extended to create buildinfo files for reproducibility, meaning {feeds,version,config}.buildinfo. As the buildbot workflow is more complex than the single prepare (kmod feed insertion), prepare is only used to create those buildinfo files. Running prepare however runs `target/compile` as well, taking time even everything is already compiled. Splitting this allows the buildbot to run only the `buildinfo` target while others can still use the convenience feature `prepare`. Signed-off-by: Paul Spooren <mail@aparcar.org>
build: fix host menu config targets using ncurses On a recent Gentoo Linux installation, invoking `make menuconfig`, `make kernel_menuconfig` or `make kernel_nconfig` in the build system fails, whereas for example `make menuconfig` in the kernel tree alone works as expected. This is happening because STAGING_PREFIX is not defined when kernel's {menu,n}config target calls pkg-config from the toolchain/host and thus pkg-config returns an empty value, and the fallback values in the kernel config script are applied but those are off and the linking fails. Solution is to use system's pkg-config for all ncurses based menu config targets in order to provide proper compiler/linker flags. Ref: FS#2423 Cc: Thomas Albers <thomas.gameiro@gmail.com> Tested-by: Stijn Tintel <stijn@linux-ipv6.be> Tested-by: Eneas U de Queiroz <cotequeiroz@gmail.com> Signed-off-by: Petr Štetiar <ynezz@true.cz>