build: add CycloneDX SBOM JSON support CycloneDX is an open source standard developed by the OWASP foundation. It supports a wide range of development ecosystems, a comprehensive set of use cases, and focuses on automation, ease of adoption, and progressive enhancement of SBOMs (Software Bill Of Materials) throughout build pipelines. So lets add support for CycloneDX SBOM for packages and images manifests. Signed-off-by: Petr Štetiar <ynezz@true.cz>
build: generate index.json The index.json file lies next to Packages index files and contains a json dict with the package architecture and a dict of package names and versions. This can be used for downstream project to know what packages in which versions are available. Signed-off-by: Paul Spooren <mail@aparcar.org>
build: fix opkg install step for large package selection When the list of packages to be installed in a built image exceeds a certain number, then 'opkg install' executed for target '$(curdir)/install' in package/Makefile fails with: /usr/bin/env: Argument list too long. On Linux, the length of a command-line parameter is limited by MAX_ARG_STRLEN to max 128 kB. * https://elixir.bootlin.com/linux/latest/source/include/uapi/linux/binfmts.h#L15 * https://www.in-ulm.de/~mascheck/various/argmax/ To solve the problem, store the package list being passed to 'opkg install' in a temporary file and use the shell command substitution to pass the content of the file to 'opkg install'. This guarantees that the length of the command-line parameters passed to the bash shell is short. The following bash script demonstrates the problem: ---------------------------------------------------------------------------- count=${1:-1000} FILES="" a_file="/home/egorenar/Repositories/openwrt-rel/bin/targets/alpine/generic/packages/base-files_1414-r16464+19-e887049fbb_arm_cortex-a15_neon-vfpv4.ipk" for i in $(seq 1 $count); do FILES="$FILES $a_file" done env bash -c "echo $FILES >/dev/null" echo "$FILES" | wc -c ---------------------------------------------------------------------------- Test run: ---------------------------------------------------------------------------- $ ./test.sh 916 130989 $ ./test.sh 917 ./test.sh: line 14: /bin/env: Argument list too long 131132 ---------------------------------------------------------------------------- Signed-off-by: Alexander Egorenkov <egorenar-dev@posteo.net> [reword commit subject] Signed-off-by: Paul Spooren <mail@aparcar.org>
build: always build package/kernel/linux build: always build package/kernel/linux If no in-tree module packages are selected, the build system does not process package/kernel/linux. This package is required for building the virtual 'kernel' package, which is specified as a dependency for all kernel packages. Signed-off-by: Felix Fietkau <nbd@nbd.name>
build: store SourceDateEpoch in manifest The usage of granular `SOURCE_DATE_EPOCH` for packages is an incrementing integer which could be useful for downstream tooling, therefore add it to the packages manifest. Signed-off-by: Paul Spooren <mail@aparcar.org>
packages: apply usign padding workarounds to package indexes if needed Since usign miscalculates SHA-512 digests for input sizes of exactly 64 + N * 128 + 110 or 64 + N * 128 + 111 bytes, we need to apply some white space padding to avoid triggering the hashing edge case. While usign itself has been fixed already, there is still many firmwares in the wild which use broken usign versions to verify current package indexes so we'll need to carry this workaround in the forseeable future. Ref: https://forum.openwrt.org/t/signature-check-failed/41945 Ref: https://git.openwrt.org/5a52b379902471cef495687547c7b568142f66d2 Signed-off-by: Jo-Philipp Wich <jo@mein.io>
build: add ABI_VERSION to binary package names Add the ABI_VERSION source makefile variable to the binary package basename and resolve source dependencies on packages with ABI_VERSION set to such expanded names. If for example a package specifies DEPENDS:=libopenssl while the OpenSSL Makefile specifies ABI_VERSION:=1.0.0, the resulting ipk control data dependency will be "Depends: libopenssl1.0.0" and the libopenssl ipk file will be called "libopenssl1.0.0_<version>_<arch>.ipk". The next time a library such as OpenSSL is updated to an incompatible version, the ABI_VERSION shall be changed accordingly to prevent opkg from simply upgrading to an incompatible library without considering the dependencies of already installed packages. Also introduce another "SourceName" control field which is required by the newly introduced "scritps/ipkg-remove" to determine the proper related .ipk files to delete upon buildroot package clean operations. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
imagebuilder: reuse rootfs preparation from rootfs.mk In addition to removing redundant code, this fixes various issues in IB-generated images that have been fixed in prepare_rootfs before, including better handling of CONFIG_CLEAN_IPKG and enabling of initscripts from FILES. We also reuse the opkg macro and remove --force-... flags that have been removed from rootfs.mk as well. Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
include/rootfs.mk: pass additional files dir to prepare_rootfs as an argument Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
build: remove package preconfig feature This feature has been unused for years, and its scope is too limited to be actually useful. Signed-off-by: Matthias Schiffer <mschiffer@universe-factory.net>
build: cleanup tmp/ dir of target rootfs Signed-off-by: Yousong Zhou <yszhou4tech@gmail.com>
build: skip opkg host dependency within the SDK Signed-off-by: Felix Fietkau <nbd@nbd.name>
build: move opkg host dependency from package/install to package/compile Improves parallel build behavior, since it allows opkg to be built at the same time as other packages Signed-off-by: Felix Fietkau <nbd@nbd.name>
build: make <subdir>/install opt-in, use it for target/ only Fixes buildbot errors on running make target/install or toolchain/install Signed-off-by: Felix Fietkau <nbd@nbd.name>
Revert "build: always run package/cleanup before package/compile" This reverts commit 2990a21058243b067bb56fed36ee69205595993e. This introduces a race condition, let's fix this in buildbot instead. Signed-off-by: Felix Fietkau <nbd@nbd.name>
build: always run package/cleanup before package/compile Remove unnecessary stampfile indirection Fixes an issue with the command sequence used by buildbot Signed-off-by: Felix Fietkau <nbd@nbd.name>
build: fix triggering opkg/host compilation Commit 131db36 "build: remove separate /install step for host builds" dropped the package/*/host/install targets in favor to performing the install steps within the compile target instead. Adjust package/Makefile accordingly in order to prevent a missing staging_dir/host/bin/opkg when staging package archives into the rootfs. Signed-off-by: Jo-Philipp Wich <jo@mein.io>
build: add support for automatically removing build dir contents during build This is used to save space on buildbot instances. If any part of a package needs to be rebuild, the whole package is rebuilt from scratch. Stamp files are preserved to allow dependency checks to work Signed-off-by: Felix Fietkau <nbd@nbd.name>
opkg: drop S/MIME support It has never been used by default (due to being too bloated), and it is properly replaced by usign (which has been the default for a long time now). Remove this feature to simplify the build system Signed-off-by: Felix Fietkau <nbd@nbd.name>
package/Makefile & ipkg-make-index.sh: add full package data list The external script used to generate the package lists for the LEDE wiki's table of packages [1] and package indexes [2] requires a "Source:" field in the package lists to find package makefiles. The package makefiles are used to read the package's Category and Submenu. The "Source:" field was removed in commit b4aa3c899cbf1f81cd8a1ea197481ac84a9d646e to reduce package list sizes and lessen opkg issues in low ram devices. Add a separate package list file with full data to be used by the wiki's script. It's called Packages.manifest and isn't compressed as it's not necessary. 1. https://lede-project.org/packages/start 2. https://lede-project.org/packages/index/start Signed-off-by: Alberto Bursi <alberto.bursi@outlook.it>