Commit Graph

154 Commits

Author SHA1 Message Date
Matthias Schiffer 071cf7b20f
Switch to Lua for target definitions
The old bash-based parsing code was way too complex. Replace it with Lua.
2019-06-15 23:34:07 +02:00
Matthias Schiffer 16a78b4e9e
scripts/check_site.lua: use new format() function to simplify var_error() 2019-04-28 19:17:47 +02:00
Matthias Schiffer 5abddd2976
scripts/check_site.lua: fix array_to_string with non-string/number array elements
Fixes need_one_of() with boolean elements.
2019-04-28 19:16:28 +02:00
Martin Weinelt 423aafbd29 gluon-core: improve channel and add chanlist validation 2019-04-28 18:17:40 +02:00
bobcanthelpyou fe521db441 build: add site check for obsolete settings (#1702) 2019-04-28 14:38:08 +02:00
Petr Štetiar 9dc1ac7f58
build: replace hardcoded paths with overridable variables
Signed-off-by: Petr Štetiar <ynezz@true.cz>
2019-04-22 22:00:45 +02:00
bobcanthelpyou 2c4f022994 build: rename DEVICES to GLUON_DEVICES (#1686) 2019-04-12 01:05:04 +02:00
David Bauer c1b9ea2d9c gluon-scheduled-domain-switch: add package (#1555)
This package allows to automatically switch to another domain, either
at a given point in time or after the node was offline long enough.
2019-02-12 11:00:29 +01:00
David Bauer 6e48dedeb8 scripts: include binaries in patch
This commit allows for binary content in patches created via 'make
update-patches'.

Binary patches are needed for BDFs required for al boards on the new
ipq40xx target. Before this commit, binaries were omitted from the
generated patch which resulted in the patch not able to apply.

Patches without binary content do not differ in case they are created
with the '--binary' flag.
2018-07-16 01:39:36 +02:00
Matthias Schiffer 7460431faa
scripts/copy_output.sh: add support for "extra" images
The new extra_image command allows to copy additional images that don't
fall into the sysupgrade or factory categories to the output directory.
These images will be placed in the "other" subdirectory.

The extra_image command takes 3 arguments: in_suffix (suffix in OpenWrt
bin directory), out_suffix (suffix in Gluon output directory) and ext (file
extension). Example:

  extra_image '-tftp' '-tftp' '.bin'

The ext argument can be omitted to control the whole filename suffix
including the extension from in_suffix and out_suffix. Multiple extra_image
commands can be used for a single device; aliases work as usual.

Based-on-patch-by: David Bauer <mail@david-bauer.net>
2018-07-14 18:33:37 +02:00
Matthias Schiffer e8cb34bae2
scripts/copy_output.sh: refactor 2018-07-14 18:17:36 +02:00
Matthias Schiffer 63c16c01a0
scripts: move DEFAULT_FEEDS definition to a separate file
modules.sh can't depend on openwrt/feeds.conf.default, otherwise the
initial update will fail.
2018-07-11 21:47:50 +02:00
Matthias Schiffer 210d97c53e
Switch to OpenWrt 18.06 branch 2018-07-10 22:57:40 +02:00
Matthias Schiffer a49cf395bd
cripts/check_site.lua: print paths separated by '.' rather than '/'
Using slashs instead of dots is a bit unexpected, and as long as we're not
going for full-blown JSONPath, neither is more correct than the other.
2018-06-29 20:47:59 +02:00
Matthias Schiffer aeae9df001
check_site: improve error message for unset values 2018-04-27 17:48:36 +02:00
Matthias Schiffer ad7d154c5a
scripts/check_site.lua: show type of incorrect values in error messages 2018-04-01 23:21:18 +02:00
Matthias Schiffer b7be32d9f4
scripts/check_site.lua: print proper error message for non-string keys in need_alphanumeric_key() 2018-04-01 23:18:53 +02:00
Martin Weinelt 85c0bd6f57
scripts/check_site.lua: add table_keys() helper 2018-03-20 17:23:48 +01:00
Matthias Schiffer 1a7d93a2b9
scripts/check_site.lua: add support for alternative checks 2018-03-20 17:15:55 +01:00
Matthias Schiffer 6ae067cb37
gluon-core: add domain aliases and pretty name
Based-on-patch-by: lemoer <git@irrelefant.net>
2018-01-26 12:32:47 +01:00
Matthias Schiffer 8cb0388416
scripts/check_site.lua: enable in_site()/in_domain() checks 2018-01-26 12:32:47 +01:00
Matthias Schiffer 51c0ceeb55
scripts/check_site.lua: merge site and domains for validation
Each domain is validated separately, preferring domain values to site
values.

Based-on-patch-by: lemoer <git@irrelefant.net>
2018-01-26 12:32:46 +01:00
Matthias Schiffer 24a085a542
gluon-site: add default domain and check for it
This does not do anything yet, as this_domain() is not implemented yet.

Based-on-patch-by: lemoer <git@irrelefant.net>
2018-01-26 12:32:46 +01:00
Matthias Schiffer 8b44d09963
scripts/generate_manifest.sh: fix filesize for symlinks
scripts/filesize.sh doesn't follow symlinks.

Fixes #1316
2018-01-24 09:49:21 +01:00
Matthias Schiffer 775028475b
check_site: move site loading logic to check_site_lib (which is renamed to check_site.lua) 2018-01-19 12:33:52 +01:00
Matthias Schiffer 43628c1679
scripts/check_site_lib.lua: improve display of invalid values 2018-01-19 11:02:33 +01:00
Matthias Schiffer 7ccdacd294
treewide: rework check_site_lib.lua
In addition to significant internal differences in check_site_lib.lua (in
particular unifying error handling to a single place for the upcoming
multi-domain support), this changes the way fields are addressed in site
check scripts: rather than providing a string like 'next_node.ip6', the
path is passed as an array {'next_node', 'ip6'}.

Other changes in site check scripts:
* need_array and need_table now pass the full path to the sub fields to the
subcheck instead of the key and value
* Any check referring to a field inside a table implies that all higher
levels must be tables if they exist: a check for {'next_node', 'ip6'} adds
an implicit (optional) check for {'next_node'}, which allows to remove many
explicit checks for such tables
2018-01-19 10:12:43 +01:00
Matthias Schiffer 020afc856f
gluon-site: install domain configs
The domain configs are not checked yet, and not used for anything.

Based-on-patch-by: lemoer <git@irrelefant.net>
2018-01-19 05:44:25 +01:00
Matthias Schiffer 8d8d4fa0c3
scripts/check_site_lib.lua: introduce in_site() and in_domain() helpers
These helpers are no-ops for now, until the actual multi-domain support is
added.
2018-01-19 03:30:50 +01:00
Matthias Schiffer 21b3dd3259
build: add file size field to manifest lines
We would like to check the file size instead of downloading indefinitely.
The file size is added in another copy of the manifest lines, which is
ignored by older autoupdater implementations.
2017-12-28 22:57:37 +01:00
Matthias Schiffer 87c741b45e
scripts: run feeds.sh with bash like the other scripts handling modules
Fixes #1262
2017-11-23 17:44:46 +01:00
Matthias Schiffer becc90d818
build: delete lede/tmp on module updates
LEDE doesn't always notice that the metadata needs to be refreshed.
2017-10-29 18:44:30 +01:00
Matthias Schiffer 08cbbf9925
build: move all generic config generation to targets/generic
As the default package list is now handled using the 'packages' directive,
explicit wpad-mini removals in target definitions can be dropped.
2017-09-07 03:21:42 +02:00
Matthias Schiffer 587c5016cb
build: handle generic and target-specific config in the same run of target_config{,_check}.sh
This, together with a new try_config directive, will allow checking the
generic config and make it more flexible.
2017-09-07 00:17:57 +02:00
Matthias Schiffer 6ca841bad5
build: introduce GLUON_FEATURES
To reduce the number of packages that need to be listed in
GLUON_SITE_PACKAGES, this adds a new variable GLUON_FEATURES. Sets of
packages are enabled automatically based on the combination of listed
feature flags.

Site-specified package feeds can provide their own feature flag
definitions.
2017-07-09 02:31:38 +02:00
Matthias Schiffer 62132a590f
scripts: add support for manifest-only model aliases 2017-06-13 12:21:03 +02:00
Matthias Schiffer ad91ab1be7
build: don't copy packages to output directory for targets without opkg
Fixes #1126
2017-05-15 23:23:02 +02:00
Matthias Schiffer afc2e0c03d
build: scripts/target_config_check.sh: output all relevant errors
Don't stop after the first error, as later error messages may make the
cause of the error much clearer in some cases.
2017-05-02 17:05:55 +02:00
Matthias Schiffer 9e6cfaee0d
scripts: generate SHA256 manifest lines in addition to SHA512
The current autoupdater will only respect the last line for a given model
name, so we can add SHA256 checksums as long as they occur before the
corresponding SHA512 line.
2017-02-25 00:38:50 +01:00
kb-light c9563cdebd scripts: check_site_lib: introduce need_string_array_match() (#1016) 2017-02-21 09:45:33 +01:00
Matthias Schiffer 2e6d457ec1
scripts/update.sh: don't hide errors because of invalid commit IDs 2017-02-12 08:03:46 +01:00
kb-light f606e0ee8a scripts: fix manifest generation 2017-01-28 10:48:42 +01:00
Matthias Schiffer 3d1860e092
build: allow using DEVICES variable for sysupgrade_image/factory_image images
Setting DEVICES will also suppress cleaning the package directory,
improving incremental build speed.
2017-01-27 01:30:47 +01:00
Matthias Schiffer 70b116fd61
build: apply model-specific site packages at the end of the package list 2017-01-19 18:00:46 +01:00
Matthias Schiffer 78b2775eec
Use LEDE as base for Gluon 2017-01-18 17:21:43 +01:00
Matthias Schiffer 62ac802d4a
scripts/update-patches: include unabbreviated object IDs in patch files
The default abbreviation length depends on factors like user configuration,
git version and number of repository objects. Use unabbreviates IDs to make
patch generation more reproducible.
2016-12-20 01:08:11 +01:00
Matthias Schiffer f1c23b3b3c
scripts/sha512sum.sh: output error message to stderr 2016-10-10 06:18:18 +02:00
Jan-Tarek Butt 0f3e660842
scripts: improve variable escaping 2016-10-10 05:57:35 +02:00
Matthias Schiffer b2473f05cb
scripts/update.sh: don't disable commit signing for modules 2016-08-27 21:31:54 +02:00
kb-light 4d98fbb1bb scripts: update need_string_array() to work as expected 2016-08-26 23:28:26 +02:00
kb-light 21e033213c scripts: add functions need_one_of and need_array_of to check_site_lib.lua
need_one_of(varname, array, required) checks weather the value of the specified variable is part of given array.
need_array_of(varname, array, required) is similar to need_one_of() but assume that varname points to an array.
2016-08-26 23:28:26 +02:00
Matthias Schiffer 0eac102971
scripts/update.sh: fix checkout of correct base commit after fetching 2016-08-12 09:23:07 +02:00
Matthias Schiffer bb3db1795b
update/patch: avoid applying patches directly in build repos
Switching branches and applying patches in the build repos will
unnecessarily touch many files, causing rebuilds of packages that didn't
really change; furthermore, it is filling the reflog with many entries.

Don't ever switch to base branch in the build repos and apply patches in
a temporary clone to avoid these issues.

In addition, GPG signing is generally disabled in the build repos to
override potential global configuration (as signing doesn't make sense and
will slow down rebases).
2016-08-09 03:41:52 +02:00
Matthias Schiffer e678ab140e
scripts/update-patches: disable rename detection
git 2.9 enables rename detection by default, changing the content of some
of our patch files. Explicitly disable it.
2016-07-01 13:22:09 +02:00
Matthias Schiffer efc004abf2
scripts/patch: never GPG-sign automated commits, regardless of the default configuration 2016-05-28 16:04:40 +02:00
Matthias Schiffer d5f5ce74e8 build: add module separators/headings to patch/update/update-patches scripts
Suggested-by: Steffen Möller <steffen_moeller@gmx.de>
2016-03-26 00:03:04 +01:00
Matthias Schiffer 33d028783f build: remove rather useless "patch" and "unpatch" make targets 2016-03-25 23:57:15 +01:00
Johannes Rudolph 13658879e5 git submodules
Possibility to work with git submodules
2016-03-06 03:09:09 +01:00
Matthias Schiffer ec1c82566a scripts/patch.sh: add --committer-date-is-author-date to `git am`
This option will make the generated commit IDs deterministic, greatly
reducing the number of repository objects created when calling `make
update` repeatedly.
2016-02-28 03:30:14 +01:00
Matthias Schiffer f23e024787 Convert site.conf to JSON during build
This will allow us to use its content from other languages than Lua as
well.
2016-01-02 23:49:25 +01:00
Matthias Schiffer 05f146f817 Validate section names from site.conf in various packages 2015-10-11 19:34:14 +02:00
Matthias Schiffer 0f91fa6fa8 scripts/patch.sh: apply patches in a loop instead of a single `git am` command
Using a loop makes the script a bit nicer, and it works around a git bug
corrupting patch dates when multiple patches are applied at once.
2015-10-08 23:11:44 +02:00
Nils Schneider 56e7753035 check_site: allow subcheck to be nil in need_table() 2015-07-25 21:02:24 +02:00
Matthias Schiffer 63059c7e8f Switch to new reduced gluon package module 2015-04-22 21:07:31 +02:00
Matthias Schiffer 267831b04d Remove now unused md5sum.sh script 2015-03-19 19:08:41 +01:00
Matthias Schiffer 1c6e52c9b6 Allow running `make update` without git identity configuration 2015-01-21 17:37:41 +01:00
Matthias Schiffer b67f094138 Fix site modules 2015-01-13 21:56:54 +01:00
Matthias Schiffer 8cfd64d2db Make module scripts use GLUONDIR and GLUON_SITEDIR
This fixes `make update` not working correctly when GLUON_SITEDIR is set
manually.
2015-01-11 09:48:31 +01:00
Nils Schneider 3a8af0ce38 scripts: fix GLUON_SITE_CONFIG
check_site.sh and site.sh were setting GLUON_SITE_CONFIG on their own
even though the environment variable was already exported.
2014-07-29 10:05:10 +02:00
Matthias Schiffer cba86b0211 scripts/rfc3339date.lua: fix pattern validity 2014-07-21 03:44:53 +02:00
Matthias Schiffer c4109b349c Add DATE and PRIORITY fields to the manifest 2014-07-20 16:35:20 +02:00
Matthias Schiffer 442ef0bd36 check_site_lib: add need_boolean check 2014-07-16 16:39:41 +02:00
Julian Kornberger 3e0d439165 Allow specifying a branch for a module
If a given commit is not included in the master branch, the update script fails.
2014-07-15 15:31:52 +02:00
Matthias Schiffer 17bf228884 Add helper scripts used by packages to validate site.conf 2014-07-05 02:48:09 +02:00
Matthias Schiffer 705595574d Remove things not needed anymore with the new site config 2014-05-15 22:59:13 +02:00
Matthias Schiffer 1b520c6856 Use site_config loader from gluon packages 2014-05-15 22:59:12 +02:00
Matthias Schiffer 62feaab756 Add "portable" md5sum and sha512sum scripts 2014-05-15 22:59:12 +02:00
Matthias Schiffer 3a6561a40c New Lua-based site configuration 2014-05-15 22:59:12 +02:00
Daniel Ehlers bd28aa99ae Return exit code on missing value
Let the site configuration value script return 1 when
the requested value is not available.This enables us
to react on failures, so we can for example use a default
value.

With this patch the script will return with exitcode 1 if
the value is not found and no output on stdout. Otherwise
exitcode is 0 and the value is returned on stdout.
2014-05-09 13:38:26 +02:00
Matthias Schiffer 5fb924193d Allow whitespace and other special characters in generate expressions from arrays 2014-02-23 06:13:02 +01:00
Matthias Schiffer 58584018de Add array support to the simple generate script (doesn't work with spaces) 2014-01-21 02:55:02 +01:00
Matthias Schiffer 6942303486 Add site.sh to get site configuration 2014-01-20 02:30:25 +01:00
Matthias Schiffer 9f3a09a765 scripts/configure.pl: add function to generate a nightly version string 2014-01-20 01:48:03 +01:00
Jan-Philipp Litza ca95374ea1 Enable the site config to include own package feeds 2014-01-19 17:37:46 +01:00
Matthias Schiffer cacf010f4a Don't warn about whitespace when patching 2014-01-11 13:42:09 +01:00
Matthias Schiffer cfe3b79ca8 Fix `make patch' for more than one patch file 2014-01-09 20:20:34 +01:00
Matthias Schiffer 7537464262 Nicer feed specification 2014-01-02 15:16:32 +01:00
Matthias Schiffer 59adcccab3 Fix update.sh 2014-01-01 18:30:55 +01:00
Matthias Schiffer 9274724486 Add support for old git version without -C 2013-12-31 17:03:39 +01:00
Matthias Schiffer b7676e5818 Fix update-patches patch names 2013-12-28 23:07:01 +01:00
Matthias Schiffer 76167579cd Fix branch name for modules without patches 2013-12-28 22:43:16 +01:00
Matthias Schiffer 37f1248d3d Add some more fixes and some checks for the new module system 2013-12-28 22:13:18 +01:00
Matthias Schiffer 1b33134353 Make patch and update scripts a bit nicer 2013-12-28 21:35:16 +01:00
Matthias Schiffer a6ae2786f3 Make patch generation deterministic 2013-12-28 17:17:02 +01:00
Matthias Schiffer 1cc0948597 Add update-patches target 2013-12-28 15:31:11 +01:00
Matthias Schiffer 1117cc39de Add patch and unpatch targets 2013-12-28 15:11:59 +01:00
Matthias Schiffer 3ddc077738 Add make target to check out modules 2013-12-28 14:54:49 +01:00
Matthias Schiffer 2f919846c2 Split up configure script 2013-09-30 19:04:35 +02:00
Matthias Schiffer b72ffb1bdb configure: use tabs instead of spaces 2013-09-30 17:14:40 +02:00
Matthias Schiffer 31da712536 Ignore lists in site config 2013-09-30 17:13:13 +02:00