| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Previously, auditing the mtime or ctime properties of a file
on ext4 (or any nanosecond resolution filesystem) when running
under ruby 1.9+ would spuriously report a change on every catalog
run.
This was because the yaml library (or libraries, more on that
shortly) correctly serialize the nanosecond resolution of the Time
object to yaml, *but* on deserialization they produce a Time object
truncated to microseconds. This led to a false inequality operation
leading to the spurious audit messages.
This patch is an unfortunate hack consisting of special-casing
the comparison of Time objects in the audit equality operation.
For context, I also considered alternatives:
* Monkey-patching the Time object == method to ignore nanoseconds.
But this would affect all Time objects and could have undesired
consequences.
* Monkey-patching the yaml library to correctly deserialize. But
this got messy because there are several different yaml libraries
that may be involved.
* Intercepting the serialization in Puppet::Util::Yaml, but this
would mean discarding nanoseconds for all serialization of Time
objects to yaml.
In the end, this spot fix seemed like the least evil solution.
|
|
This is because these methods are also used for deserialization from
other formats than PSON. The method for serializing a object is called
to_data_hash, so makes sense to call this from_data_hash.
|
|
* stable:
(RE-902) Remove sid from build_defaults
(pup-1524) Correct resource_present? to account for purged
|
|
Checking that catalogs have particular resources is useful. Let's make
it easier. This also adds parameter matching to the resource matcher.
|
|
The fix for http://projects.puppetlabs.com/issues/23081 included
e376b9d8, which refactored some of the resource_harness code and
introduced ResourceApplicationContext.resource_present?
At the time that method only checked current_values[:ensure] against
:absent, which meant that if current_values[:ensure] was :purged,
then resource_present? was true and the code thus attempted to
sync additional properties. This resulted in the double events
noted in this ticket (the first one in manage_via_ensure_if_possible,
the second (unnecessary) one in sync_if_needed).
This change adds a present? method to Puppet::Type, the default
implementation of which just checks for :ensure != :absent, but
which must be overridden by other types that have additional values
for :ensure that also imply that the resource is not present. It also
overrides that method in the package, file, and interface types,
each of which offer such additional values for :ensure.
In addition, there was another code path for duplicate events
because the previous code didn't distinguish between not using
ensure and the ensure check failing (e.g. due to a provider failure).
To address this, the code was refactored slightly to introduce
the NO_ACTION return from sync_if_needed.
|
|
Previously, special handling was added to synthesize a report
event even when the property application failed early. However,
when this change was made the report schema wasn't updated,
so the schema did not allow for several fields in the event
to be null (as they are for one of these synthesized events).
This change updates the report schema to reflect that certain
fields can be null. It also adds a spec test including an error
event, which requires the schema change in order to pass.
|
|
|
|
This adds a new matcher to the JSONMatchers to validate json against a
given json-schema. All of the json schema validation now goes through
this one matcher, which also handles skipping the checks on windows.
|
|
|
|
- All previous File and FileTest calls to exist? or exists? go through
the new FileSystem::File abstraction so that the implementation can
later be swapped for a Windows specific one to support symlinks
|
|
This changes removes a large block of unit tests, comprising
1296 individual tests built up via 8 nested 'each' loops. Why
remove so many tests?
First, in the course of refactoring for #23081 we triggered
test failures in this block only when we also triggered a test
failure elsewhere, and conversely, it didn't trigger failures
where other individual tests did register a useful failure.
Second, several people have stared at this unit test block
and not been able to figure out what it is uniquely testing.
So in short the value of this test block is not clear, so we're
removing it. Of course, if we hit future gaps in unit test coverage,
we'll add tests back in to cover gaps.
|
|
|
|
This makes non-public methods explicitly private. It also reorders the
methods to read in a manner where the top is the public and highest
level, and it becomes more low level as we move down the file.
|
|
|
|
Tags were being stored in an array, which is inefficient as the number of tags
grows. This commit changes Puppet::Util::Tagging to store tags in a set. A new
class is added, Puppet::Util::TagSet, which subclasses the built in set while
allowing it to be compatible with the old serialization format.
|
|
This renames validate_as_json methods to be specific to the schema they are
validating (since that's embedded in each definition) and inlines the
implementations that didn't need to be abstracted.
|
|
|
|
* upstream/stable:
(#22471) Finish removing duplication of yaml exceptions
(#21975) Don't monkey patch Object#instance_variables
(#19994) Ensure that failed ParsedFile operations are marked flushed
(maint) Don't use subject in parsedfile spec
(#22384) Log 404 and 406 responses at info
(#22471) Remove duplication by using Puppet::Util::Yaml
(#22471) Extract YAML interaction into shared code
(Maint) Stop asserting nothingness
(Maint) Simplify code
(#22471) Rescue errors specific to the YAML parser
(Maint) Convert to new block expects
|
|
We were monkey patching Object#instance_variables to return symbols instead of
strings on ruby 1.8 which broke the use of SOAP. The patch was only used to
make it easier to do set operations when determining attributes to serialize
in zaml. This narrows the behavior of converting to symbols into the zaml code
so we don't have to override Object#instance_variables.
|
|
(#22356) Add api documentation for reports
|
|
1.8.7 does not have File.absolute_path which broke the test. Method not
needed since jsonschema for report is no "all in one file".
The schema gem does not work on windows; tests cannot run there.
HTTP PUT is the method used for the report API, not POST.
|
|
This adds documentation and a complete json-schema (draft 4) for the
report API endpoint. The documentation includes an example.
A report test validates the schema itself as well as a generated report
to ensure that the schema describes what is generated.
|
|
|
|
The problem with the previous version of this code was that it only
allowed the report to support exactly what the user had selected. This
is alright (for the most part) for the agent, but is incorrect for the
master since the master needs to understand both formats at once. The
remedy this the default_format is now selected based on the
report_serialization_format setting whereas the supported_formats are
always both PSON and YAML.
|
|
Prior to this commit, supported_formats was returning an array of
strings, which violated the contract for FormatHandler's
most_suitable_format_for method. This commit fixes supported_formats
to return an array of symbols.
|
|
Previously, when round-tripping a report using YAML, the
Puppet::Resource::Status#time object could lose precision (depending on
the ruby version and platform), so it did not match the expected value. For
example, the expected value was:
"time"=>"2013-09-04T17:01:38.510604415+00:00"
But the round-tripped value was:
"time"=>"2013-09-04T17:01:38.510604000+00:00"
This commit changes the test to explicitly compare status fields, and
handles the `time` field specially to avoid loss of precision during
YAML serialization.
|
|
Without making agents able to select the format to serialize reports, a
newer agent would be unable to submit reports to an older master.
Although we ask users to upgrade masters first, that is not often done
in practice. In order to allow those kinds of setups to work, this makes
the report format selectable.
Puppet 3.2.2 and later support reports as pson, so only older, insecure
masters need to have the agent's change this new setting.
|
|
zaphod42/issue/master/22229-ensure-generated-resources-contained
(#22229) Ensure generated resources contained
|
|
The catalog tracks the relationship between the container and the
contained resource but did not provide any clear way to query for the
containment information. This adds a new method #container_of(resource)
that will return the container of the given resource, when one exists.
This removes some duplication around understanding what the container
is.
|
|
The generated resources were left out in the cold with respect to
containers: they had none at all. This changes that to make sure that
the generated resources (via #generate and #eval_generate) have the same
container as the resource that generated them.
|
|
|
|
The Puppet::Catalog#relationship_graph method used to be a factory for
the Puppet::Graph::RelationshipGraph. That was removed when the
prioritizer was added in an attempt to remove some of the
responsibilities of catalog. Inside puppet, all of the consumers were
updated. Unfortunately many tests of types rely on the old behavior in
order to test eval_generate, which may be implemented using
introspection of the relationship graph. This reverts back to the old
behavior of the relationship_graph being a factory method and instead
memoizes the prioritizer so that the same instance can be used for the
relationship_graph as well as the transaction.
|
|
This ties the selection of the Puppet::Graph::Prioritizer to the
Puppet[:ordering] setting. In order to make it easier to ensure that all
of the uses of the prioritizer use the same one (because it needs to
track state) this also moves around the relationship of the catalog and
the transaction. Previously the catalog was responsible for converting
itself into a relationship graph. This now puts that responsibility on
the transaction, which much now be constructed with the desired
Prioritizer. The selection is still done in the catalog (because that is
where #apply lives, which is the public starting point of applying a
catalog).
Additionally this adds a random prioritizer.
|
|
The underlying red-black tree that is used as the priority queue for
tracking what resources to evaluate will only allow a single value per
key. This means that when generated resources are added they need to get
unique priority keys. However there could be an arbitrary number of
these resources added in between existing resources. This commit
provides a Puppet::Graph::Key that behaves like the numbering scheme of
an outline so that priorities can be arbitrarily nested and the nested
priorities are between the priorities at the outer level.
The integration tests for the file type caught this error. It only
showed up when 2 unique, new resources are generated.
|
|
When a generated resource duplicates a resource already in the catalog,
we want to avoid adding it a second time, but we still want to create
dependencies between the resource and the generating resource. For
example, we may have a recursive file resource, but explicitly model one
of the files that will be generated. We still want the explicit file
resource to depend on its generated parent directory.
This also refactors the resource generation methods, splitting them up
into smaller methods.
|
|
Resources created with eval_generate need to apply with the same
priority as their parent resource.
|
|
The logic for determining how to add generated resources into the
catalog was tightly bound with the rest of the transaction code. This
makes it hard to test the exact logic around that. By extracting a class
it exposes that logic in a much more testable form.
|
|
When querying whether a resource was scheduled, we were unnecessarily creating
a Status for the resource even though it was unused.
|
|
Previously the catalog had to track if a report had been given for
logging. This encapsulates that relationship so that only the report
knows how to control the logging destinations.
|
|
In Puppet::Configurer, generate a 'transaction_uuid' using SecureRandom.uuid.
For the catalog retrieval, pass that uuid into the 'fact_options' hash which
generates the query string for the catalog request. For the report, set the
report's variable, also named 'transaction_uuid,' and serialize that to_pson.
Also:
- In configurer.rb, rename fact_options to query_options since the hash ends
up including more than just the facts.
- Add spec tests for transaction uuid modelled on those for configuration version
|
|
This changes the agent to submit reports in PSON. An effect of this
change is that reports will not be able to submit to masters that do not
have the changes for CVE-2013-3567 which allow the master to parse
PSON formatted reports.
|
|
It's possible to run the "ensure" block without having executed either the
main or "rescue" blocks, if an Exception not derived from StandardError is
raised or when testing with Mocha expectations.
If this happens during the property.sync, this causes an immediate exit from
the main block, but the "ensure" block is still run. The event message won't
have been initialized yet, so this then causes the logger to report:
Puppet::Util::Log requires a message
This ensures a message is logged from the context of the resource before
letting the exception propagate up to the application.
|
|
In the case where a resource triggers an event with invalidate_refreshes
enabled (i.e. a service starts), the dequeue process would attempt to
invalidate other events registered for this resource. If no other
notifications had been received on the resource already, then the hash element
for the resource would be nil. The resource would then fail:
Error: /Stage[main]//Service[service]: Could not evaluate: undefined method `[]=' for nil:NilClass
|
|
The existing spec example for dequeuing events used a single resource and
self-refresh, which didn't accurately reflect the two resource layout in the
original issue #7165 where one resource notifies another.
|
|
Documentation to --detailed-exitcodes states that mask 0x4 is applied in case
"there were failures during the transaction." Currently only failures to change
resource state are reported, failures to restart the resource are still ignored.
Only change (code=2) to Exec['/bin/true'] is considered here:
$ puppet apply --detailed-exitcodes -e 'service { "duckfaced": restart => "/bin/false", ensure => running, status => "/bin/true" } exec { "/bin/true": notify => Service["duckfaced"] }'
$ echo $?
2
Needless to say, information about failed restarts is usually as important to
the user as the status changes, since they usually mean that desired
configuration is not in effect.
This change makes Puppet consider information about failed restarts when
setting the failure bit.
Another possibility to avoid loosing the information would be to introduce
another bit in the exit code mask, but that would change semantics with respect
to existing documentation and thus probably be more confusing.
|
|
For Ruby 1.9, commit 252c884 added code in the Puppet::Transaction::Event
class to handle strings rather than a resource object that responded to []
with symbols, for the purposes of testing.
In Ruby 2.0, the TypeError message changed a bit, so the rescue didn't work as
expected. This is now reworked so the test passes in an object that responds
appropriately.
|
|
An invalidate_refreshes option can now be specified on newvalue calls inside
type definitions. When a property is changed to a value with this option set,
it causes all currently scheduled refreshes (from notify or subscribe) to be
discarded after evaluation of the resource.
This is designed for instances where a file resource is changed and refreshes
a service resource. The service itself would then be evaluated, the service is
started and then it would be refreshed, causing an additional restart. This
change allows a type definition to specify that a property change to a
particular value invalidates certain needless refreshes.
|
|
Without this patch Ruby 1.9 is still complaining loudly about trying to
parse the spec files. The previous attempt to clean up this problem in
edc3ddf works for Ruby 1.8 but not 1.9.
I'd prefer to remove the shebang lines entirely, but doing so will cause
encoding errors in Ruby 1.9. This patch strives for a happy middle
ground of convincing Ruby it is actually working with Ruby while not
confusing it to think it should exec() to rspec.
This patch is the result of the following command run against the source
tree:
find spec -type f -print0 | \
xargs -0 perl -pl -i -e 's,^\#\!\s?/(.*)rspec,\#! /usr/bin/env ruby,'
|
|
Sorting YAML attributes makes for marginally nicer human-friendly YAML output,
but ends up costing a non-trivial amount of computation to determine the
ordering of objects.
We can skip that with pretty much no adverse effects.
Signed-off-by: Daniel Pittman <daniel@puppetlabs.com>
|
|
Without this patch some spec files are using `ruby -S rspec` and others
are using `rspec`.
We should standardize on a single form of the interpreter used for spec
files.
`ruby -S rspec` is the best choice because it correctly informs editors
such as Vim with Syntastic that the file is a Ruby file rather than an
Rspec file.
|
