| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This means that from now on, there is no global setting to switch off
this redundancy check. Individual SUBST classes can still set their own
SUBST_NOOP_OK.<id> in order to ignore no-op filename patterns.
The current bulk builds do not show any build failures that are caused
by this, which means that really almost all packages have been migrated.
|
|
It had been switched off to not affect packages in the stable branch
2020Q2. Now starts the last round where it is possible to disable this
check. After 2020Q3, all SUBST blocks must either find their patterns or
be explicitly marked as potential no-ops. This will help to find
outdated SUBST blocks.
|
|
With our current version of pdksh, a "false && something" construct under
"set -e" conditions will continue as it does with other shells, but if the
construct is within a for loop then it exits, causing failures in the
substitution code. An explicit "|| true" is necessary to avoid this.
Approved during the freeze by wiz.
|
|
There are still some packages that fail the strict SUBST check. These
packages should nevertheless be built using the default pkgsrc
configuration, at least in the stable 2020Q2 branch. After 2020Q2 has
been switched, the strict SUBST checks will be activated again in the
default configuration.
|
|
This is a package-settable variable, and if a package leaves it
undefined, "bmake show-all-subst" should show exactly this.
|
|
To avoid bmake warnings because of duplicate class names, the :O:u
modifier had been added in r1.66 on 2020-03-21. This had the side effect
that the subst classes are now applied in alphabetical order instead of
declaration order.
For this to actually matter, there must be a file that is affected by two
different subst classes and in which the substitutions depend on each
other or prevent each other. Chances for that are pretty low.
The order is intentionally documented as being unspecified, to allow for
future modifications, just in case that a bmake variable modifier is
invented that filters for duplicates without requiring the duplicates to
be adjacent to each other. In that situation, it would be nicer to
switch back to declaration order instead of alphabetical.
|
|
These variables don't record whether a file is changed but instead
whether a pattern was found.
|
|
Fixes PR pkg/55364.
|
|
|
|
Note that a typical reason to need this is using find to generate a
list of files.
|
|
This makes the SUBST blocks stricter than before, to detect outdated or
unnecessary definitions.
Filename patterns that are not affected by any of the SUBST_SED
expressions make the build fail. It is still ok if only some of the
files from a pattern are affected and some aren't.
The latest bulk build shows that most of the build failures are fixed.
The packages that fail in that build are mostly due to other failures,
like missing C headers, wrong PLIST files, unresolved references at link
time. There may be a few packages that still fail because of this, but
these are near the leaves of the dependency tree.
https://mail-index.netbsd.org/pkgsrc-bulk/2020/05/14/msg018919.html
|
|
The escaping inside the backticks had been wrong. Because of this,
parentheses and semicolons were interpreted as shell syntax.
Switching to $(...) command substitution removes the need for quoting
some of the characters and makes the whole command simpler to understand.
Doing the escaping for the backticks command properly would have involved
lots of special cases.
The $(...) command substitution was used sparingly in pkgsrc up to now
because some older or broken shells do not support it. Since these
shells do not end up as the shell that runs the commands from Makefiles,
that's not a problem.
|
|
|
|
To work properly, the $(...) should have been $$(...).
In pkgsrc the command substitution is usually done via `backticks`, for
compatibility with /bin/sh from Solaris. To fix the shell parse errors,
the special characters are properly escaped inside the command
substitution.
|
|
|
|
devel/ruby-redmine.
|
|
|
|
Since SUBST_FILTER_CMD is a shell command, it may contain arbitrary
characters. The condition in mk/subst.mk that tested whether
SUBST_FILTER_CMD was the default filter command was evaluated at run
time. In such an evaluation, the variables (lhs and rhs) are fully
expanded before parsing the condition. This means that these variables
must not contain quotes or unquoted condition operators.
Exactly this situation came up in one of the regression tests. The
quoted "0-9" was copied verbatimly into the condition, including the
quotes. The resulting condition was:
"tr -d "0-9"" == "LC_ALL=C /usr/bin/sed "
This produced a syntax error because of the left-hand side. Adding a :Q
modifier would have helped for the left-hand side, but this would have
been necessary for the right-hand side as well. Since an empty SUBST_SED
is defined not to "contain only identity substitutions", the first
condition can simply be removed.
The whole condition in the shell program had not worked anyway since it
expanded to either "[ true ]" or to "[ false ]", and both of these
commands exited successfully.
|
|
There are several cases where patterns like s|man|${PKGMANDIR}| appear in
SUBST_SED. Up to now, these had been categorized as no-ops and required
extra code to make the package build when SUBST_NOOP_OK was set to "no".
This was against the original intention of SUBST_NOOP_OK, which was to
find outdated substitution patterns that do not occur in SUBST_FILES
anymore, most often because the packages have been updated since.
The identity substitutions do appear in the files, they just don't change
them. Typical cases are for PKGMANDIR, DEVOSSAUDIO, PREFIX, and these
variables may well be different in another pkgsrc setup. These patterns
are therefore excluded from the SUBST_NOOP_OK check.
|
|
These often lead to broken patches, unless the patches are generated very
cautiously. Because of this, pkglint already warns about this.
|
|
|
|
|
|
The indentation of the inner loop has been fixed.
The chmod is only run if the file has actually changed. In the other
case, the file would have been removed right after the chmod, which made
the chmod unnecessary.
For compatibility with ancient operating systems whose /bin/sh still does
not understand negated conditions (SunOS), these conditions have been
avoided and were written using && and || instead.
The inner loop has been flattened a bit, to compensate for the
indentation of the outer loop.
|
|
This avoids creating a temporary directory.
The "set -f" option is not used anywhere else in pkgsrc, even though it
has been available since 1985 in the 8th Edition of Research Unix. Even
AIX and IRIX have that option, so it seems a safe bet.
|
|
|
|
|
|
https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/V1_chap09.html
says in section 9.3.2 BRE Ordinary Characters that only very few
characters may be preceded with a backslash.
As a side effect, this change allows parentheses in the variable names
listed in SUBST_VARS (even if that will never happen in practice).
The reason that the regression test had not replaced VAR.[] before was
simply that this variable had not been listed in SUBST_VARS.
|
|
|
|
This makes the code a bit more readable.
|
|
This variable allows to make SUBST stricter than before. This will break
several packages that have redundant filename patterns. Most of these are
typos or outdated and should be updated or removed.
|
|
|
|
At least some implementations of rmdir(1) do not allow you to remove the
current working directory. Fixes bootstrap on SunOS.
|
|
When fixing the SUBST definitions in a package, it can hapen that the
substitution aborts in the middle. In such a case the cookie should not
be written and the substitution should be retried. Otherwise the build
may continue with half the substitutions done.
|
|
Seen in multimedia/libmp4v2, where a pattern also matches the CVS
directory from the distfiles.
|
|
The severity now depends only on the setting of SUBST_NOOP_OK. Right now
this means that some former warnings will be reported as info only, but
that will change after switching the default of SUBST_NOOP_OK after
2020Q1. Then they will all be reported as warnings, followed by the final
error saying that the pattern has no effect.
This change makes it easier to detect inconsistencies and outdated
definitions, for example by setting the global SUBST_NOOP_OK=no and
redefining WARNING_MSG to actuall fail.
|
|
The diff program is only used to produce informative output in the build
logs, nevertheless its output might be translated if there are lines that
do not end with a newline.
|
|
This is useful in bulk builds or when trying to understand what happens
under the hood, since the SUBST code leaves no .orig files around.
|
|
|
|
The default value of SUBST_MESSAGE is based on SUBST_FILES, and that
variable may use the :sh modifier to list files from WRKSRC, which may
not exist at load time.
|
|
In the case of pkglocaledir, the SUBST_FILES are generated by a shell
command. That command assumes that the WRKDIR already exists. Therefore
SUBST_FILES must be evaluated as late as possible.
See mk/configure/replace-localedir.mk; an example package that fails is
devel/gettext-tools.
|
|
In a bulk build with very strict settings (WARNING_MSG fails, as well as
no-op substitutions), it became clear that nearly all of the cases where
SUBST didn't replace anything were bugs in the package definition.
Most of them were just outdated, which is no surprise given that some
packages are already over 20 years old.
For backwards compatibility, SUBST_NOOP_OK defaults to "yes" right now.
After correcting the affected packages, the default will change to "no".
|
|
|
|
All variables that are used or defined in the file are now listed in the
_VARGROUPS section.
The "is text file" command variable has been renamed since pkglint
thought the former variable name would specify a filename, not a shell
command.
The "is text file" command has been rewritten to only rely on tr(1)
instead of both tr(1) and wc(1). This makes it both simpler and maybe
also a little faster, since the file only has to be read once.
The SUBST_TARGETS variable has been removed since it is used nowhere
else. To get the list of all subst targets (should that ever be
necessary), use the expression ${SUBST_CLASSES:S,^,subst-,}.
|
|
|
|
|
|
Up to now, there was a central list of variable name patterns that
defined whether a variable was printed as a sorted list, as a list or as
a single value.
Now each variable group decides on its own which of the variables are
printed in which way, using the usual glob patterns. This is more
flexible since different files sometimes differ in their naming
conventions.
Two variable groups are added: license (for everything related to
LICENSE) and go (for lang/go).
|
|
|
|
|
|
Up to now, using subst.mk may have led to file corruption during active
package development. This happened when a sed(1) command had a syntax
error, in which case the whole sed(1) command was terminated, leaving an
empty original file behind.
This commit changes that behavior by applying the sed(1) commands to
the original file and saving the result in a temporary file. Only
after that succeeded is the original file overwritten.
During this rewrite, SUBST_POSTCMD has been removed, since it was
only used in one place (mk/wrapper), and since it relied on the exact
sequence of the internal commands. No package in either main pkgsrc
or pkgsrc-wip uses this variable right now.
|
|
From Leonardo Taccari in followup to PR 48254.
|
