diff options
| author | Stig Sandbeck Mathisen <ssm@debian.org> | 2013-09-15 12:56:04 +0200 |
|---|---|---|
| committer | Stig Sandbeck Mathisen <ssm@debian.org> | 2013-09-15 12:56:04 +0200 |
| commit | 6a7cea9da38e05c36cfccc04de363e092b87dffb (patch) | |
| tree | 98045b4e83c801f828a13eca18796f7ea28e8fd5 | |
| parent | 821c15fb73e2b02f01ea88588e228222d411d79d (diff) | |
| parent | b9836647d08336bddaae7ecdbd5407121c0af10f (diff) | |
| download | puppet-upstream/3.3.0.tar.gz | |
Imported Upstream version 3.3.0upstream/3.3.0
574 files changed, 20376 insertions, 8062 deletions
diff --git a/COMMITTERS.md b/COMMITTERS.md index 1c61d0761..a6288c4c0 100644 --- a/COMMITTERS.md +++ b/COMMITTERS.md @@ -13,6 +13,17 @@ on?" This is already called out in the [CONTRIBUTING.md](http://goo.gl/XRH2J). Therefore, it is the responsibility of the committer to re-base the change set on the appropriate branch which should receive the contribution. +It is also the responsibility of the committer to review the change set in an +effort to make sure the end users must opt-in to new behavior that is +incompatible with previous behavior. We employ the use of [feature +flags](http://stackoverflow.com/questions/7707383/what-is-a-feature-flag) as +the primary way to achieve this user opt-in behavior. Finally, it is the +responsibility of the committer to make sure the `master` and `stable` branches +are both clean and working at all times. Clean means that dead code is not +allowed, everything needs to be usable in some manner at all points in time. +Stable is not an indication of the build status, but rather an expression of +our intent that the `stable` branch does not receive new functionality. + The rest of this document addresses the concerns of the committer. This document will help guide the committer decide which branch to base, or re-base a contribution on top of. This document also describes our branch management @@ -37,8 +48,21 @@ making the decision what base branch to merge the change set into. **base branch** - A branch in Git that contains an active history of changes and will eventually be released using semantic version guidelines. The branch -named master will always exist as a base branch. All other base branches will -be associated with a specific released version of Puppet, e.g. 2.7.x and 3.0.x. +named `master` will always exist as a base branch. The other base branches are +`stable`, and `security` described below. + +**master branch** - The branch where new functionality that are not bug fixes +is merged. + +**stable branch** - The branch where bug fixes against the latest release or +release candidate are merged. + +**security** - Where critical security fixes are merged. These change sets +will then be merged into release branches independently from one another. (i.e. +no merging up). Please do not submit pull requests against the security branch +and instead report all security related issues to security@puppetlabs.com as +per our security policy published at +[https://puppetlabs.com/security/](https://puppetlabs.com/security/). Committer Guide ==== @@ -53,32 +77,58 @@ This section provides a guide to help a committer decide the specific base branch that a change set should be merged into. The latest minor release of a major release is the only base branch that should -be patched. Older minor releases in a major release do not get patched. Before -the switch to [semantic versions](http://semver.org/) committers did not have -to think about the difference between minor and major releases. Committing to -the latest minor release of a major release is a policy intended to limit the -number of active base branches that must be managed. +be patched. These patches will be merged into `master` if they contain new +functionality. They will be merged into `stable` and `master` if they fix a +critical bug. Older minor releases in a major release do not get patched. + +Before the switch to [semantic versions](http://semver.org/) committers did not +have to think about the difference between minor and major releases. +Committing to the latest minor release of a major release is a policy intended +to limit the number of active base branches that must be managed. Security patches are handled as a special case. Security patches may be -applied to earlier minor releases of a major release. +applied to earlier minor releases of a major release, but the patches should +first be merged into the `security` branch. Security patches should be merged +by Puppet Labs staff members. Pull requests should not be submitted with the +security branch as the base branch. Please send all security related +information or patches to security@puppetlabs.com as per our [Security +Policy](https://puppetlabs.com/security/). + +The CI systems are configured to run against `master` and `stable`. Over time, +these branches will refer to different versions, but their name will remain +fixed to avoid having to update CI jobs and tasks as new versions are released. How to commit a change set to multiple base branches --- -A change set may apply to multiple releases. In this situation the change set -needs to be committed to multiple base branches. This section provides a guide -for how to merge patches across releases, e.g. 2.7 is patched, how should the -changes be applied to 3.0? - -First, merge the change set into the lowest numbered base branch, e.g. 2.7. -Next, merge the changed base branch up through all later base branches by using -the `--no-ff --log` git merge options. We commonly refer to this as our "merge -up process" because we merge in once, then merge up multiple times. - -When a new minor release branch is created (e.g. 3.1.x) then the previous one -is deleted (e.g. 3.0.x). Any security or urgent fixes that might have to be -applied to the older code line is done by creating an ad-hoc branch from the -tag of the last patch release of the old minor line. +A change set may apply to multiple branches, for example a bug fix should be +applied to the stable release and the development branch. In this situation +the change set needs to be committed to multiple base branches. This section +provides a guide for how to merge patches into these branches, e.g. +`stable` is patched, how should the changes be applied to `master`? + +First, rebase the change set onto the `stable` branch. Next, merge the change +set into the `stable` branch using a merge commit. Once merged into `stable`, +merge the same change set into `master` without doing a rebase as to preserve +the commit identifiers. This merge strategy follows the [git +flow](http://nvie.com/posts/a-successful-git-branching-model/) model. Both of +these change set merges should have a merge commit which makes it much easier +to track a set of commits as a logical change set through the history of a +branch. Merge commits should be created using the `--no-ff --log` git merge +options. + +Any merge conflicts should be resolved using the merge commit in order to +preserve the commit identifiers for each individual change. This ensures `git +branch --contains` will accurately report all of the base branches which +contain a specific patch. + +Using this strategy, the stable branch need not be reset. Both `master` and +`stable` have infinite lifetimes. Patch versions, also known as bug fix +releases, will be tagged and released directly from the `stable` branch. Major +and minor versions, also known as feature releases, will be tagged and released +directly from the `master` branch. Upon release of a new major or minor +version all of the changes in the `master` branch will be merged into the +`stable` branch. Code review checklist --- @@ -100,7 +150,6 @@ branch: or modify it.) HINT: `git diff master --check` * Does the change set conform to the contributing guide? - Commit citizen guidelines: --- @@ -117,7 +166,8 @@ paying attention to our automated build tools. backwards compatible change set into master, then the target version should be 3.2.0 in the issue tracker.) * Ensure the pull request is closed (Hint: amend your merge commit to contain - the string `closes: #123` where 123 is the pull request number. + the string `closes #123` where 123 is the pull request number and github + will automatically close the pull request when the branch is pushed.) Example Procedure ==== @@ -130,7 +180,7 @@ Suppose a contributor submits a pull request based on master. The change set fixes a bug reported against Puppet 3.1.1 which is the most recently released version of Puppet. -In this example the committer should rebase the change set onto the 3.1.x +In this example the committer should rebase the change set onto the `stable` branch since this is a bug rather than new functionality. First, the committer pulls down the branch using the `hub` gem. This tool @@ -142,44 +192,53 @@ branch to track the remote branch. Switched to a new branch 'jeffmccune-fix_foo_error' At this point the topic branch is a descendant of master, but we want it to -descend from 3.1.x. The committer creates a new branch then re-bases the -change set: +descend from `stable`. The committer rebases the change set onto `stable`. - $ git branch bug/3.1.x/fix_foo_error - $ git rebase --onto 3.1.x master bug/3.1.x/fix_foo_error + $ git branch bug/stable/fix_foo_error + $ git rebase --onto stable master bug/stable/fix_foo_error First, rewinding head to replay your work on top of it... Applying: (#23456) Fix FooError that always bites users in 3.1.1 The `git rebase` command may be interpreted as, "First, check out the branch -named `bug/3.1.x/fix_foo_error`, then take the changes that were previously -based on `master` and re-base them onto `3.1.x`. +named `bug/stable/fix_foo_error`, then take the changes that were previously +based on `master` and re-base them onto `stable`. -Now that we have a topic branch containing the change set based on the correct +Now that we have a topic branch containing the change set based on the `stable` release branch, the committer merges in: - $ git checkout 3.1.x - Switched to branch '3.1.x' - $ git merge --no-ff --log bug/3.1.x/fix_foo_error + $ git checkout stable + Switched to branch 'stable' + $ git merge --no-ff --log bug/stable/fix_foo_error Merge made by the 'recursive' strategy. foo | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 foo -Once merged into the first base branch, the committer merges up: +Once merged into the first base branch, the committer merges the `stable` +branch into `master`, being careful to preserve the same commit identifiers. $ git checkout master Switched to branch 'master' - $ git merge --no-ff --log 3.1.x + $ git merge --no-ff --log stable Merge made by the 'recursive' strategy. foo | 0 1 file changed, 0 insertions(+), 0 deletions(-) create mode 100644 foo -Once the change set has been merged "in and up." the committer pushes. (Note, -the checklist should be complete at this point.) Note that both the 3.1.x and -master branches are being pushed at the same time. +Once the change set has been merged into one base branch, the change set should +not be modified in order to keep the history clean, avoid "double" commits, and +preserve the usefulness of `git branch --contains`. If there are any merge +conflicts, they are to be resolved in the merge commit itself and not by +re-writing (rebasing) the patches for one base branch, but not another. + +Once the change set has been merged into `stable` and into `master`, the +committer pushes. Please note, the checklist should be complete at this point. +It's helpful to make sure your local branches are up to date to avoid one of +the branches failing to fast forward while the other succeeds. Both the +`stable` and `master` branches are being pushed at the same time. - $ git push puppetlabs master:master 3.1.x:3.1.x + $ git push puppetlabs master:master stable:stable That's it! The committer then updates the pull request, updates the issue in -our issue tracker, and keeps an eye on the build status. +our issue tracker, and keeps an eye on the [build +status](http://jenkins.puppetlabs.com). @@ -14,7 +14,7 @@ end platforms :ruby do gem 'pry', :group => :development gem 'yard', :group => :development - gem 'redcarpet', :group => :development + gem 'redcarpet', '~> 2.0', :group => :development gem "racc", "~> 1.4", :group => :development end @@ -22,11 +22,22 @@ gem "puppet", :path => File.dirname(__FILE__), :require => false gem "facter", *location_for(ENV['FACTER_LOCATION'] || '~> 1.6') gem "hiera", *location_for(ENV['HIERA_LOCATION'] || '~> 1.0') gem "rake", :require => false -gem "rspec", "~> 2.11.0", :require => false -gem "mocha", "~> 0.10.5", :require => false gem "rgen", "0.6.5", :require => false -gem "yarjuf", "~> 1.0" + +group(:development, :test) do + + # Jenkins workers may be using RSpec 2.9, so RSpec 2.11 syntax + # (like `expect(value).to eq matcher`) should be avoided. + gem "rspec", "~> 2.11.0", :require => false + + # Mocha is not compatible across minor version changes; because of this only + # versions matching ~> 0.10.5 are supported. All other versions are unsupported + # and can be expected to fail. + gem "mocha", "~> 0.10.5", :require => false + + gem "yarjuf", "~> 1.0" +end group(:extra) do gem "rack", "~> 1.4", :require => false @@ -33,7 +33,7 @@ Generally, you need the following things installed: Contributions ------ -Please see our [Contibution +Please see our [Contribution Documents](https://github.com/puppetlabs/puppet/blob/master/CONTRIBUTING.md) and our [Developer Documentation](https://github.com/puppetlabs/puppet/blob/master/README_DEVELOPER.md). @@ -51,3 +51,7 @@ site](http://projects.puppetlabs.com). A [mailing list](https://groups.google.com/forum/?fromgroups#!forum/puppet-users) is available for asking questions and getting help from others. In addition there is an active #puppet channel on Freenode. + +HTTP API +-------- +{file:api_docs/http_api_index.md HTTP API Index} diff --git a/README_DEVELOPER.md b/README_DEVELOPER.md index 95c632f20..7d1990899 100644 --- a/README_DEVELOPER.md +++ b/README_DEVELOPER.md @@ -13,7 +13,7 @@ this difference while working on the static compiler and types and providers. The two different types of catalog becomes relevant when writing spec tests because we frequently need to wire up a fake catalog so that we can exercise -types, providers, or terminii that filter the catalog. +types, providers, or termini that filter the catalog. The two different types of catalogs are so-called "resource" catalogs and "RAL" (resource abstraction layer) catalogs. At a high level, the resource catalog @@ -130,67 +130,35 @@ method. # Ruby Dependencies # -Puppet is considered an Application as it relates to the recommendation of -adding a Gemfile.lock file to the repository and the information published at -[Clarifying the Roles of the .gemspec and -Gemfile](http://yehudakatz.com/2010/12/16/clarifying-the-roles-of-the-gemspec-and-gemfile/) +To install the dependencies run: -To install the dependencies run: `bundle install` to install the dependencies. + $ bundle install --path .bundle/gems/ -A checkout of the source repository should be used in a way that provides -puppet as a gem rather than a simple Ruby library. The parent directory should -be set along the `GEM_PATH`, preferably before other tools such as RVM that -manage gemsets using `GEM_PATH`. +Once this is done, you can interact with puppet through bundler using `bundle +exec <command>` which will ensure that `<command>` is executed in the context +of puppet's dependencies. -For example, Puppet checked out into `/workspace/src/puppet` using `git -checkout https://github.com/puppetlabs/puppet` in `/workspace/src` can be used -with the following actions. The trick is to symlink `gems` to `src`. +For example to run the specs: - $ cd /workspace - $ ln -s src gems - $ mkdir specifications - $ pushd specifications; ln -s ../gems/puppet/puppet.gemspec; ln -s ../gems/puppet/lib; popd - $ export GEM_PATH="/workspace:${GEM_PATH}" - $ gem list puppet + $ bundle exec rake spec -This should list out +To run puppet itself (for a resource lookup say): - puppet (2.7.19) + $ bundle exec puppet resource host localhost -The final directory structure should look like this: +which should return something like: - /workspace/src --- git working directory - /gems -> src - /specifications/puppet.gemspec -> ../gems/puppet/puppet.gemspec - /lib -> ../gems/puppet/lib - -## Bundler ## - -With a source checkout of Puppet properly setup as a gem, dependencies can be -installed using [Bundler](http://gembundler.com/) - - $ bundle install - Fetching gem metadata from http://rubygems.org/........ - Using diff-lcs (1.1.3) - Installing facter (1.6.11) - Using metaclass (0.0.1) - Using mocha (0.10.5) - Using puppet (2.7.19) from source at /workspace/puppet-2.7.x/src/puppet - Using rack (1.4.1) - Using rspec-core (2.10.1) - Using rspec-expectations (2.10.0) - Using rspec-mocks (2.10.1) - Using rspec (2.10.0) - Using bundler (1.1.5) - Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed. + host { 'localhost': + ensure => 'present', + ip => '127.0.0.1', + target => '/etc/hosts', + } # Running Tests # Puppet Labs projects use a common convention of using Rake to run unit tests. The tests can be run with the following rake task: - rake spec - # Or if using Bundler bundle exec rake spec This allows the Rakefile to set up the environment beforehand if needed. This @@ -199,14 +167,20 @@ method is how the unit tests are run in [Jenkins](https://jenkins.puppetlabs.com Under the hood Puppet's tests use `rspec`. To run all of them, you can directly use 'rspec': - rspec - # Or if using Bundler bundle exec rspec To run a single file's worth of tests (much faster!), give the filename, and use the nested format to see the descriptions: - rspec spec/unit/ssl/host_spec.rb --format nested + bundle exec rspec spec/unit/ssl/host_spec.rb --format nested + +## Testing dependency version requirements + +Puppet is only compatible with certain versions of RSpec and Mocha. If you are +not using Bundler to install the required test libraries you must ensure that +you are using the right library versions. Using unsupported versions of Mocha +and RSpec will probably display many spurious failures. The supported versions +of RSpec and Mocha can be found in the project Gemfile. # A brief introduction to testing in Puppet @@ -468,6 +442,95 @@ describe "mocking an object" do end end ``` +### Writing tests without side effects + +When properly written each test should be able to run in isolation, and tests +should be able to be run in any order. This makes tests more reliable and allows +a single test to be run if only that test is failing, instead of running all +17000+ tests each time something is changed. However, there are a number of ways +that can make tests fail when run in isolation or out of order. + +#### Using instance variables + +Puppet has a number of older tests that use `before` blocks and instance +variables to set up fixture data, instead of `let` blocks. These can retain +state between tests, which can lead to test failures when tests are run out of +order. + +```ruby +# test.rb +RSpec.configure do |c| + c.mock_framework = :mocha +end + +describe "fixture data" do + describe "using instance variables" do + + # BAD + before :all do + # This fixture will be created only once and will retain the `foo` stub + # between tests. + @fixture = stub 'test data' + end + + it "can be stubbed" do + @fixture.stubs(:foo).returns :bar + @fixture.foo.should == :bar + end + + it "should not keep state between tests" do + # The foo stub was added in the previous test and shouldn't be present + # in this test. + expect { @fixture.foo }.to raise_error + end + end + + describe "using `let` blocks" do + + # GOOD + # This will be recreated between tests so that state isn't retained. + let(:fixture) { stub 'test data' } + + it "can be stubbed" do + fixture.stubs(:foo).returns :bar + fixture.foo.should == :bar + end + + it "should not keep state between tests" do + # since let blocks are regenerated between tests, the foo stub added in + # the previous test will not be present here. + expect { fixture.foo }.to raise_error + end + end +end +``` + +``` +bundle exec rspec test.rb -fd + +fixture data + using instance variables + can be stubbed + should not keep state between tests (FAILED - 1) + using `let` blocks + can be stubbed + should not keep state between tests + +Failures: + + 1) fixture data using instance variables should not keep state between tests + Failure/Error: expect { @fixture.foo }.to raise_error + expected Exception but nothing was raised + # ./test.rb:17:in `block (3 levels) in <top (required)>' + +Finished in 0.00248 seconds +4 examples, 1 failure + +Failed examples: + +rspec ./test.rb:16 # fixture data using instance variables should not keep state between tests +``` + ### RSpec references @@ -703,11 +766,11 @@ default clientbucket. Create a module that recursively downloads something. The jeffmccune-filetest module will recursively copy the rubygems source tree. - $ puppet module install jeffmccune-filetest + $ bundle exec puppet module install jeffmccune-filetest Start the master with the StaticCompiler turned on: - $ puppet master \ + $ bundle exec puppet master \ --catalog_terminus=static_compiler \ --verbose \ --no-daemonize @@ -722,7 +785,7 @@ Add the special Filebucket[puppet] resource: Get the static catalog: - $ puppet agent --test + $ bundle exec puppet agent --test You should expect all file metadata to be contained in the catalog, including a checksum representing the content. When managing an out of sync file resource, @@ -72,4 +72,8 @@ namespace "ci" do ENV["LOG_SPEC_ORDER"] = "true" sh %{rspec -r yarjuf -f JUnit -o result.xml -fd spec} end + + task :el6tests do + sh "cd acceptance/config/el6; rm -f el6.tar.gz; tar -czvf el6.tar.gz *" + end end diff --git a/ext/build_defaults.yaml b/ext/build_defaults.yaml index 40c38344d..71636e6a0 100644 --- a/ext/build_defaults.yaml +++ b/ext/build_defaults.yaml @@ -9,14 +9,15 @@ gpg_name: 'info@puppetlabs.com' gpg_key: '4BD6EC30' sign_tar: FALSE # a space separated list of mock configs -final_mocks: 'pl-el-5-i386 pl-el-6-i386 pl-fedora-17-i386 pl-fedora-18-i386 pl-fedora-19-i386' -yum_host: 'burji.puppetlabs.com' +final_mocks: 'pl-el-5-i386 pl-el-6-i386 pl-fedora-18-i386 pl-fedora-19-i386' +yum_host: 'yum.puppetlabs.com' yum_repo_path: '/opt/repository/yum/' build_gem: TRUE build_dmg: TRUE -apt_host: 'burji.puppetlabs.com' +apt_host: 'apt.puppetlabs.com' apt_repo_url: 'http://apt.puppetlabs.com' apt_repo_path: '/opt/repository/incoming' ips_repo: '/var/pkgrepo' ips_store: '/opt/repository' ips_host: 'solaris-11-ips-repo.acctest.dc1.puppetlabs.net' +tar_host: 'downloads.puppetlabs.com' diff --git a/ext/debian/changelog b/ext/debian/changelog index f4425a33f..13adb524f 100644 --- a/ext/debian/changelog +++ b/ext/debian/changelog @@ -1,8 +1,8 @@ -puppet (3.2.4-1puppetlabs1) hardy lucid natty oneiric unstable sid squeeze wheezy precise; urgency=low +puppet (3.3.0-1puppetlabs1) hardy lucid natty oneiric unstable sid squeeze wheezy precise; urgency=low - * Update to version 3.2.4-1puppetlabs1 + * Update to version 3.3.0-1puppetlabs1 - -- Puppet Labs Release <info@puppetlabs.com> Wed, 14 Aug 2013 15:00:25 -0700 + -- Puppet Labs Release <info@puppetlabs.com> Thu, 12 Sep 2013 13:57:55 -0700 puppet (3.2.3-0.1rc0puppetlabs1) lucid unstable sid squeeze wheezy precise quantal raring; urgency=low diff --git a/ext/debian/puppet-common.manpages b/ext/debian/puppet-common.manpages index 233d716cd..d034f863a 100644 --- a/ext/debian/puppet-common.manpages +++ b/ext/debian/puppet-common.manpages @@ -1,3 +1,35 @@ man/man5/puppet.conf.5 -man/man8/puppet.8 man/man8/extlookup2hiera.8 +man/man8/puppet.8 +man/man8/puppet-agent.8 +man/man8/puppet-apply.8 +man/man8/puppet-catalog.8 +man/man8/puppet-cert.8 +man/man8/puppet-certificate.8 +man/man8/puppet-certificate_request.8 +man/man8/puppet-certificate_revocation_list.8 +man/man8/puppet-config.8 +man/man8/puppet-describe.8 +man/man8/puppet-device.8 +man/man8/puppet-doc.8 +man/man8/puppet-facts.8 +man/man8/puppet-file.8 +man/man8/puppet-filebucket.8 +man/man8/puppet-help.8 +man/man8/puppet-inspect.8 +man/man8/puppet-instrumentation_data.8 +man/man8/puppet-instrumentation_listener.8 +man/man8/puppet-instrumentation_probe.8 +man/man8/puppet-key.8 +man/man8/puppet-kick.8 +man/man8/puppet-man.8 +man/man8/puppet-module.8 +man/man8/puppet-node.8 +man/man8/puppet-parser.8 +man/man8/puppet-plugin.8 +man/man8/puppet-queue.8 +man/man8/puppet-report.8 +man/man8/puppet-resource.8 +man/man8/puppet-resource_type.8 +man/man8/puppet-secret_agent.8 +man/man8/puppet-status.8 diff --git a/ext/debian/puppet.manpages b/ext/debian/puppet.manpages deleted file mode 100644 index f8464f76f..000000000 --- a/ext/debian/puppet.manpages +++ /dev/null @@ -1,32 +0,0 @@ -man/man8/puppet-agent.8 -man/man8/puppet-apply.8 -man/man8/puppet-catalog.8 -man/man8/puppet-describe.8 -man/man8/puppet-cert.8 -man/man8/puppet-certificate.8 -man/man8/puppet-certificate_request.8 -man/man8/puppet-certificate_revocation_list.8 -man/man8/puppet-config.8 -man/man8/puppet-device.8 -man/man8/puppet-doc.8 -man/man8/puppet-facts.8 -man/man8/puppet-file.8 -man/man8/puppet-filebucket.8 -man/man8/puppet-help.8 -man/man8/puppet-inspect.8 -man/man8/puppet-instrumentation_data.8 -man/man8/puppet-instrumentation_listener.8 -man/man8/puppet-instrumentation_probe.8 -man/man8/puppet-key.8 -man/man8/puppet-kick.8 -man/man8/puppet-man.8 -man/man8/puppet-module.8 -man/man8/puppet-node.8 -man/man8/puppet-parser.8 -man/man8/puppet-plugin.8 -man/man8/puppet-queue.8 -man/man8/puppet-report.8 -man/man8/puppet-resource.8 -man/man8/puppet-resource_type.8 -man/man8/puppet-secret_agent.8 -man/man8/puppet-status.8 diff --git a/ext/gentoo/init.d/puppet b/ext/gentoo/init.d/puppet index 83d30b024..bfa4b1947 100644 --- a/ext/gentoo/init.d/puppet +++ b/ext/gentoo/init.d/puppet @@ -10,7 +10,7 @@ depend() { checkconfig() { if [[ ! -d "${PUPPET_PID_DIR}" ]] ; then - eerror "Please make sure PUPPET_PID_DIR is defined and points to a existing directory" + eerror "Please make sure PUPPET_PID_DIR is defined and points to an existing directory" return 1 fi diff --git a/ext/gentoo/init.d/puppetmaster b/ext/gentoo/init.d/puppetmaster index 9a48754a2..9a0184432 100755 --- a/ext/gentoo/init.d/puppetmaster +++ b/ext/gentoo/init.d/puppetmaster @@ -10,7 +10,7 @@ depend() { checkconfig() { if [[ ! -d "${PUPPETMASTER_PID_DIR}" ]] ; then - eerror "Please make sure PUPPETMASTER_PID_DIR is defined and points to a existing directory" + eerror "Please make sure PUPPETMASTER_PID_DIR is defined and points to an existing directory" return 1 fi diff --git a/ext/ips/puppet.p5m b/ext/ips/puppet.p5m index df09627c5..64cacf8f4 100644 --- a/ext/ips/puppet.p5m +++ b/ext/ips/puppet.p5m @@ -1,6 +1,6 @@ -set name=pkg.fmri value=pkg://puppetlabs.com/system/management/puppet@3.2.4,11.4.2-0 +set name=pkg.fmri value=pkg://puppetlabs.com/system/management/puppet@3.3.0,12.4.1-0 set name=pkg.summary value="Puppet, an automated configuration management tool" -set name=pkg.human-version value="3.2.4" +set name=pkg.human-version value="3.3.0" set name=pkg.description value="Puppet, an automated configuration management tool" set name=info.classification value="org.opensolaris.category.2008:System/Administration and Configuration" set name=org.opensolaris.consolidation value="puppet" diff --git a/ext/osx/PackageInfo.plist b/ext/osx/PackageInfo.plist deleted file mode 100644 index 08930e1ab..000000000 --- a/ext/osx/PackageInfo.plist +++ /dev/null @@ -1,36 +0,0 @@ -<?xml version="1.0" encoding="UTF-8"?> -<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd"> -<plist version="1.0"> -<dict> - <key>CFBundleIdentifier</key> - <string>com.reductivelabs.puppet</string> - <key>CFBundleShortVersionString</key> - <string>{SHORTVERSION}</string> - <key>IFMajorVersion</key> - <integer>{MAJORVERSION}</integer> - <key>IFMinorVersion</key> - <integer>{MINORVERSION}</integer> - <key>IFPkgFlagAllowBackRev</key> - <false/> - <key>IFPkgFlagAuthorizationAction</key> - <string>RootAuthorization</string> - <key>IFPkgFlagDefaultLocation</key> - <string>/</string> - <key>IFPkgFlagFollowLinks</key> - <true/> - <key>IFPkgFlagInstallFat</key> - <false/> - <key>IFPkgFlagIsRequired</key> - <false/> - <key>IFPkgFlagOverwritePermissions</key> - <false/> - <key>IFPkgFlagRelocatable</key> - <false/> - <key>IFPkgFlagRestartAction</key> - <string>None</string> - <key>IFPkgFlagRootVolumeOnly</key> - <true/> - <key>IFPkgFlagUpdateInstalledLanguages</key> - <false/> -</dict> -</plist> diff --git a/ext/osx/createpackage.sh b/ext/osx/createpackage.sh deleted file mode 100755 index 411aaf5fb..000000000 --- a/ext/osx/createpackage.sh +++ /dev/null @@ -1,187 +0,0 @@ -#!/bin/bash -# -# Script to build an "old style" not flat pkg out of the puppet repository. -# -# Author: Nigel Kersten (nigelk@google.com) -# -# Last Updated: 2008-07-31 -# -# Copyright 2008 Google Inc. -# -# Licensed under the Apache License, Version 2.0 (the "License"); -# you may not use this file except in compliance with the License. -# You may obtain a copy of the License at -# -# http://www.apache.org/licenses/LICENSE-2.0 -# -# Unless required by applicable law or agreed to in writing, software -# distributed under the License is distributed on an "AS IS" BASIS, -# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -# See the License for the specific language governing permissions and -# limitations under the License - - -INSTALLRB="install.rb" -BINDIR="/usr/bin" -SBINDIR="/usr/sbin" -SITELIBDIR="/usr/lib/ruby/site_ruby/1.8" -PACKAGEMAKER="/Developer/usr/bin/packagemaker" -PROTO_PLIST="PackageInfo.plist" -PREFLIGHT="preflight" - - -function find_installer() { - # we walk up three directories to make this executable from the root, - # root/conf or root/conf/osx - if [ -f "./${INSTALLRB}" ]; then - installer="$(pwd)/${INSTALLRB}" - elif [ -f "../${INSTALLRB}" ]; then - installer="$(pwd)/../${INSTALLRB}" - elif [ -f "../../${INSTALLRB}" ]; then - installer="$(pwd)/../../${INSTALLRB}" - else - installer="" - fi -} - -function find_puppet_root() { - puppet_root=$(dirname "${installer}") -} - -function install_puppet() { - echo "Installing Puppet to ${pkgroot}" - "${installer}" --destdir="${pkgroot}" --bindir="${BINDIR}" --sbindir="${SBINDIR}" --sitelibdir="${SITELIBDIR}" - mkdir -p ${pkgroot}/var/lib/puppet - chown -R root:admin "${pkgroot}" - chmod -R go-w "${pkgroot}" -} - -function install_docs() { - echo "Installing docs to ${pkgroot}" - docdir="${pkgroot}/usr/share/doc/puppet" - mkdir -p "${docdir}" - for docfile in COPYING LICENSE README README.queueing README.rst; do - install -m 0644 "${puppet_root}/${docfile}" "${docdir}" - done - chown -R root:wheel "${docdir}" - chmod 0755 "${docdir}" -} - -function get_puppet_version() { - puppet_version=$(RUBYLIB="${pkgroot}/${SITELIBDIR}:${RUBYLIB}" ruby -e "require 'puppet'; puts Puppet.version") -} - -function prepare_package() { - # As we can't specify to follow symlinks from the command line, we have - # to go through the hassle of creating an Info.plist file for packagemaker - # to look at for package creation and substitue the version strings out. - # Major/Minor versions can only be integers, so we have "0" and "245" for - # puppet version 0.24.5 - # Note too that for 10.5 compatibility this Info.plist *must* be set to - # follow symlinks. - VER1=$(echo ${puppet_version} | awk -F "." '{print $1}') - VER2=$(echo ${puppet_version} | awk -F "." '{print $2}') - VER3=$(echo ${puppet_version} | awk -F "." '{print $3}') - major_version="${VER1}" - minor_version="${VER2}${VER3}" - cp "${puppet_root}/conf/osx/${PROTO_PLIST}" "${pkgtemp}" - sed -i '' "s/{SHORTVERSION}/${puppet_version}/g" "${pkgtemp}/${PROTO_PLIST}" - sed -i '' "s/{MAJORVERSION}/${major_version}/g" "${pkgtemp}/${PROTO_PLIST}" - sed -i '' "s/{MINORVERSION}/${minor_version}/g" "${pkgtemp}/${PROTO_PLIST}" - - # We need to create a preflight script to remove traces of previous - # puppet installs due to limitations in Apple's pkg format. - mkdir "${pkgtemp}/scripts" - cp "${puppet_root}/conf/osx/${PREFLIGHT}" "${pkgtemp}/scripts" - - # substitute in the sitelibdir specified above on the assumption that this - # is where any previous puppet install exists that should be cleaned out. - sed -i '' "s|{SITELIBDIR}|${SITELIBDIR}|g" "${pkgtemp}/scripts/${PREFLIGHT}" - # substitute in the bindir sepcified on the assumption that this is where - # any old executables that have moved from bindir->sbindir should be - # cleaned out from. - sed -i '' "s|{BINDIR}|${BINDIR}|g" "${pkgtemp}/scripts/${PREFLIGHT}" - chmod 0755 "${pkgtemp}/scripts/${PREFLIGHT}" -} - -function create_package() { - rm -fr "$(pwd)/puppet-${puppet_version}.pkg" - echo "Building package" - echo "Note that packagemaker is reknowned for spurious errors. Don't panic." - "${PACKAGEMAKER}" --verbose --no-recommend --no-relocate \ - --root "${pkgroot}" \ - --info "${pkgtemp}/${PROTO_PLIST}" \ - --scripts ${pkgtemp}/scripts \ - --out "$(pwd)/puppet-${puppet_version}.pkg" - if [ $? -ne 0 ]; then - echo "There was a problem building the package." - cleanup_and_exit 1 - exit 1 - else - echo "The package has been built at:" - echo "$(pwd)/puppet-${puppet_version}.pkg" - fi -} - -function cleanup_and_exit() { - if [ -d "${pkgroot}" ]; then - rm -fr "${pkgroot}" - fi - if [ -d "${pkgtemp}" ]; then - rm -fr "${pkgtemp}" - fi - exit $1 -} - -# Program entry point -function main() { - - if [ $(whoami) != "root" ]; then - echo "This script needs to be run as root via su or sudo." - cleanup_and_exit 1 - fi - - find_installer - - if [ ! "${installer}" ]; then - echo "Unable to find ${INSTALLRB}" - cleanup_and_exit 1 - fi - - find_puppet_root - - if [ ! "${puppet_root}" ]; then - echo "Unable to find puppet repository root." - cleanup_and_exit 1 - fi - - pkgroot=$(mktemp -d -t puppetpkg) - - if [ ! "${pkgroot}" ]; then - echo "Unable to create temporary package root." - cleanup_and_exit 1 - fi - - pkgtemp=$(mktemp -d -t puppettmp) - - if [ ! "${pkgtemp}" ]; then - echo "Unable to create temporary package root." - cleanup_and_exit 1 - fi - - install_puppet - install_docs - get_puppet_version - - if [ ! "${puppet_version}" ]; then - echo "Unable to retrieve puppet version" - cleanup_and_exit 1 - fi - - prepare_package - create_package - - cleanup_and_exit 0 -} - -main "$@" diff --git a/ext/osx/postflight b/ext/osx/postflight.erb index 2e36b25ef..2e36b25ef 100644..100755 --- a/ext/osx/postflight +++ b/ext/osx/postflight.erb diff --git a/ext/osx/preflight b/ext/osx/preflight deleted file mode 100644 index 027e4d9cf..000000000 --- a/ext/osx/preflight +++ /dev/null @@ -1,1784 +0,0 @@ -#!/bin/bash -# -# Make sure that old puppet cruft is removed -# This also allows us to downgrade puppet as -# it's more likely that installing old versions -# over new will cause issues. -# -# ${3} is the destination volume so that this works correctly -# when being installed to volumes other than the current OS. - - - - - - - - -/bin/rm -Rf "${3}/hiera/backend/puppet_backend.rb" - -/bin/rm -Rf "${3}/hiera/scope.rb" - -/bin/rm -Rf "${3}/hiera_puppet.rb" - -/bin/rm -Rf "${3}/puppet.rb" - -/bin/rm -Rf "${3}/puppet/agent.rb" - -/bin/rm -Rf "${3}/puppet/agent/disabler.rb" - -/bin/rm -Rf "${3}/puppet/agent/locker.rb" - -/bin/rm -Rf "${3}/puppet/application.rb" - -/bin/rm -Rf "${3}/puppet/application/agent.rb" - -/bin/rm -Rf "${3}/puppet/application/apply.rb" - -/bin/rm -Rf "${3}/puppet/application/ca.rb" - -/bin/rm -Rf "${3}/puppet/application/catalog.rb" - -/bin/rm -Rf "${3}/puppet/application/cert.rb" - -/bin/rm -Rf "${3}/puppet/application/certificate.rb" - -/bin/rm -Rf "${3}/puppet/application/certificate_request.rb" - -/bin/rm -Rf "${3}/puppet/application/certificate_revocation_list.rb" - -/bin/rm -Rf "${3}/puppet/application/config.rb" - -/bin/rm -Rf "${3}/puppet/application/describe.rb" - -/bin/rm -Rf "${3}/puppet/application/device.rb" - -/bin/rm -Rf "${3}/puppet/application/doc.rb" - -/bin/rm -Rf "${3}/puppet/application/face_base.rb" - -/bin/rm -Rf "${3}/puppet/application/facts.rb" - -/bin/rm -Rf "${3}/puppet/application/file.rb" - -/bin/rm -Rf "${3}/puppet/application/filebucket.rb" - -/bin/rm -Rf "${3}/puppet/application/help.rb" - -/bin/rm -Rf "${3}/puppet/application/indirection_base.rb" - -/bin/rm -Rf "${3}/puppet/application/inspect.rb" - -/bin/rm -Rf "${3}/puppet/application/instrumentation_data.rb" - -/bin/rm -Rf "${3}/puppet/application/instrumentation_listener.rb" - -/bin/rm -Rf "${3}/puppet/application/instrumentation_probe.rb" - -/bin/rm -Rf "${3}/puppet/application/key.rb" - -/bin/rm -Rf "${3}/puppet/application/kick.rb" - -/bin/rm -Rf "${3}/puppet/application/man.rb" - -/bin/rm -Rf "${3}/puppet/application/master.rb" - -/bin/rm -Rf "${3}/puppet/application/module.rb" - -/bin/rm -Rf "${3}/puppet/application/node.rb" - -/bin/rm -Rf "${3}/puppet/application/parser.rb" - -/bin/rm -Rf "${3}/puppet/application/plugin.rb" - -/bin/rm -Rf "${3}/puppet/application/queue.rb" - -/bin/rm -Rf "${3}/puppet/application/report.rb" - -/bin/rm -Rf "${3}/puppet/application/resource.rb" - -/bin/rm -Rf "${3}/puppet/application/resource_type.rb" - -/bin/rm -Rf "${3}/puppet/application/secret_agent.rb" - -/bin/rm -Rf "${3}/puppet/application/status.rb" - -/bin/rm -Rf "${3}/puppet/configurer.rb" - -/bin/rm -Rf "${3}/puppet/configurer/downloader.rb" - -/bin/rm -Rf "${3}/puppet/configurer/fact_handler.rb" - -/bin/rm -Rf "${3}/puppet/configurer/plugin_handler.rb" - -/bin/rm -Rf "${3}/puppet/daemon.rb" - -/bin/rm -Rf "${3}/puppet/data_binding.rb" - -/bin/rm -Rf "${3}/puppet/defaults.rb" - -/bin/rm -Rf "${3}/puppet/dsl.rb" - -/bin/rm -Rf "${3}/puppet/dsl/resource_api.rb" - -/bin/rm -Rf "${3}/puppet/dsl/resource_type_api.rb" - -/bin/rm -Rf "${3}/puppet/error.rb" - -/bin/rm -Rf "${3}/puppet/external/base64.rb" - -/bin/rm -Rf "${3}/puppet/external/dot.rb" - -/bin/rm -Rf "${3}/puppet/external/lock.rb" - -/bin/rm -Rf "${3}/puppet/external/nagios.rb" - -/bin/rm -Rf "${3}/puppet/external/nagios/base.rb" - -/bin/rm -Rf "${3}/puppet/external/nagios/grammar.ry" - -/bin/rm -Rf "${3}/puppet/external/nagios/makefile" - -/bin/rm -Rf "${3}/puppet/external/nagios/parser.rb" - -/bin/rm -Rf "${3}/puppet/external/pson/common.rb" - -/bin/rm -Rf "${3}/puppet/external/pson/pure.rb" - -/bin/rm -Rf "${3}/puppet/external/pson/pure/generator.rb" - -/bin/rm -Rf "${3}/puppet/external/pson/pure/parser.rb" - -/bin/rm -Rf "${3}/puppet/external/pson/version.rb" - -/bin/rm -Rf "${3}/puppet/face.rb" - -/bin/rm -Rf "${3}/puppet/face/ca.rb" - -/bin/rm -Rf "${3}/puppet/face/catalog.rb" - -/bin/rm -Rf "${3}/puppet/face/catalog/select.rb" - -/bin/rm -Rf "${3}/puppet/face/certificate.rb" - -/bin/rm -Rf "${3}/puppet/face/certificate_request.rb" - -/bin/rm -Rf "${3}/puppet/face/certificate_revocation_list.rb" - -/bin/rm -Rf "${3}/puppet/face/config.rb" - -/bin/rm -Rf "${3}/puppet/face/facts.rb" - -/bin/rm -Rf "${3}/puppet/face/file.rb" - -/bin/rm -Rf "${3}/puppet/face/file/download.rb" - -/bin/rm -Rf "${3}/puppet/face/file/store.rb" - -/bin/rm -Rf "${3}/puppet/face/help.rb" - -/bin/rm -Rf "${3}/puppet/face/help/action.erb" - -/bin/rm -Rf "${3}/puppet/face/help/face.erb" - -/bin/rm -Rf "${3}/puppet/face/help/global.erb" - -/bin/rm -Rf "${3}/puppet/face/help/man.erb" - -/bin/rm -Rf "${3}/puppet/face/instrumentation_data.rb" - -/bin/rm -Rf "${3}/puppet/face/instrumentation_listener.rb" - -/bin/rm -Rf "${3}/puppet/face/instrumentation_probe.rb" - -/bin/rm -Rf "${3}/puppet/face/key.rb" - -/bin/rm -Rf "${3}/puppet/face/man.rb" - -/bin/rm -Rf "${3}/puppet/face/module.rb" - -/bin/rm -Rf "${3}/puppet/face/module/build.rb" - -/bin/rm -Rf "${3}/puppet/face/module/changes.rb" - -/bin/rm -Rf "${3}/puppet/face/module/generate.rb" - -/bin/rm -Rf "${3}/puppet/face/module/install.rb" - -/bin/rm -Rf "${3}/puppet/face/module/list.rb" - -/bin/rm -Rf "${3}/puppet/face/module/search.rb" - -/bin/rm -Rf "${3}/puppet/face/module/uninstall.rb" - -/bin/rm -Rf "${3}/puppet/face/module/upgrade.rb" - -/bin/rm -Rf "${3}/puppet/face/node.rb" - -/bin/rm -Rf "${3}/puppet/face/node/clean.rb" - -/bin/rm -Rf "${3}/puppet/face/parser.rb" - -/bin/rm -Rf "${3}/puppet/face/plugin.rb" - -/bin/rm -Rf "${3}/puppet/face/report.rb" - -/bin/rm -Rf "${3}/puppet/face/resource.rb" - -/bin/rm -Rf "${3}/puppet/face/resource_type.rb" - -/bin/rm -Rf "${3}/puppet/face/secret_agent.rb" - -/bin/rm -Rf "${3}/puppet/face/status.rb" - -/bin/rm -Rf "${3}/puppet/feature/base.rb" - -/bin/rm -Rf "${3}/puppet/feature/eventlog.rb" - -/bin/rm -Rf "${3}/puppet/feature/libuser.rb" - -/bin/rm -Rf "${3}/puppet/feature/pson.rb" - -/bin/rm -Rf "${3}/puppet/feature/rack.rb" - -/bin/rm -Rf "${3}/puppet/feature/rails.rb" - -/bin/rm -Rf "${3}/puppet/feature/rdoc1.rb" - -/bin/rm -Rf "${3}/puppet/feature/rubygems.rb" - -/bin/rm -Rf "${3}/puppet/feature/selinux.rb" - -/bin/rm -Rf "${3}/puppet/feature/ssh.rb" - -/bin/rm -Rf "${3}/puppet/feature/stomp.rb" - -/bin/rm -Rf "${3}/puppet/feature/zlib.rb" - -/bin/rm -Rf "${3}/puppet/file_bucket.rb" - -/bin/rm -Rf "${3}/puppet/file_bucket/dipper.rb" - -/bin/rm -Rf "${3}/puppet/file_bucket/file.rb" - -/bin/rm -Rf "${3}/puppet/file_collection.rb" - -/bin/rm -Rf "${3}/puppet/file_collection/lookup.rb" - -/bin/rm -Rf "${3}/puppet/file_serving.rb" - -/bin/rm -Rf "${3}/puppet/file_serving/base.rb" - -/bin/rm -Rf "${3}/puppet/file_serving/configuration.rb" - -/bin/rm -Rf "${3}/puppet/file_serving/configuration/parser.rb" - -/bin/rm -Rf "${3}/puppet/file_serving/content.rb" - -/bin/rm -Rf "${3}/puppet/file_serving/fileset.rb" - -/bin/rm -Rf "${3}/puppet/file_serving/metadata.rb" - -/bin/rm -Rf "${3}/puppet/file_serving/mount.rb" - -/bin/rm -Rf "${3}/puppet/file_serving/mount/file.rb" - -/bin/rm -Rf "${3}/puppet/file_serving/mount/modules.rb" - -/bin/rm -Rf "${3}/puppet/file_serving/mount/plugins.rb" - -/bin/rm -Rf "${3}/puppet/file_serving/terminus_helper.rb" - -/bin/rm -Rf "${3}/puppet/file_serving/terminus_selector.rb" - -/bin/rm -Rf "${3}/puppet/file_system.rb" - -/bin/rm -Rf "${3}/puppet/file_system/path_pattern.rb" - -/bin/rm -Rf "${3}/puppet/forge.rb" - -/bin/rm -Rf "${3}/puppet/forge/cache.rb" - -/bin/rm -Rf "${3}/puppet/forge/errors.rb" - -/bin/rm -Rf "${3}/puppet/forge/repository.rb" - -/bin/rm -Rf "${3}/puppet/indirector.rb" - -/bin/rm -Rf "${3}/puppet/indirector/active_record.rb" - -/bin/rm -Rf "${3}/puppet/indirector/catalog/active_record.rb" - -/bin/rm -Rf "${3}/puppet/indirector/catalog/compiler.rb" - -/bin/rm -Rf "${3}/puppet/indirector/catalog/json.rb" - -/bin/rm -Rf "${3}/puppet/indirector/catalog/queue.rb" - -/bin/rm -Rf "${3}/puppet/indirector/catalog/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/catalog/static_compiler.rb" - -/bin/rm -Rf "${3}/puppet/indirector/catalog/store_configs.rb" - -/bin/rm -Rf "${3}/puppet/indirector/catalog/yaml.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate/ca.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate/disabled_ca.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate/file.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate_request/ca.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate_request/disabled_ca.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate_request/file.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate_request/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate_revocation_list/ca.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate_revocation_list/disabled_ca.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate_revocation_list/file.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate_revocation_list/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate_status.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate_status/file.rb" - -/bin/rm -Rf "${3}/puppet/indirector/certificate_status/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/code.rb" - -/bin/rm -Rf "${3}/puppet/indirector/couch.rb" - -/bin/rm -Rf "${3}/puppet/indirector/data_binding/hiera.rb" - -/bin/rm -Rf "${3}/puppet/indirector/data_binding/none.rb" - -/bin/rm -Rf "${3}/puppet/indirector/direct_file_server.rb" - -/bin/rm -Rf "${3}/puppet/indirector/envelope.rb" - -/bin/rm -Rf "${3}/puppet/indirector/errors.rb" - -/bin/rm -Rf "${3}/puppet/indirector/exec.rb" - -/bin/rm -Rf "${3}/puppet/indirector/face.rb" - -/bin/rm -Rf "${3}/puppet/indirector/facts/active_record.rb" - -/bin/rm -Rf "${3}/puppet/indirector/facts/couch.rb" - -/bin/rm -Rf "${3}/puppet/indirector/facts/facter.rb" - -/bin/rm -Rf "${3}/puppet/indirector/facts/inventory_active_record.rb" - -/bin/rm -Rf "${3}/puppet/indirector/facts/inventory_service.rb" - -/bin/rm -Rf "${3}/puppet/indirector/facts/memory.rb" - -/bin/rm -Rf "${3}/puppet/indirector/facts/network_device.rb" - -/bin/rm -Rf "${3}/puppet/indirector/facts/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/facts/store_configs.rb" - -/bin/rm -Rf "${3}/puppet/indirector/facts/yaml.rb" - -/bin/rm -Rf "${3}/puppet/indirector/file_bucket_file/file.rb" - -/bin/rm -Rf "${3}/puppet/indirector/file_bucket_file/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/file_bucket_file/selector.rb" - -/bin/rm -Rf "${3}/puppet/indirector/file_content.rb" - -/bin/rm -Rf "${3}/puppet/indirector/file_content/file.rb" - -/bin/rm -Rf "${3}/puppet/indirector/file_content/file_server.rb" - -/bin/rm -Rf "${3}/puppet/indirector/file_content/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/file_content/selector.rb" - -/bin/rm -Rf "${3}/puppet/indirector/file_metadata.rb" - -/bin/rm -Rf "${3}/puppet/indirector/file_metadata/file.rb" - -/bin/rm -Rf "${3}/puppet/indirector/file_metadata/file_server.rb" - -/bin/rm -Rf "${3}/puppet/indirector/file_metadata/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/file_metadata/selector.rb" - -/bin/rm -Rf "${3}/puppet/indirector/file_server.rb" - -/bin/rm -Rf "${3}/puppet/indirector/hiera.rb" - -/bin/rm -Rf "${3}/puppet/indirector/indirection.rb" - -/bin/rm -Rf "${3}/puppet/indirector/instrumentation_data.rb" - -/bin/rm -Rf "${3}/puppet/indirector/instrumentation_data/local.rb" - -/bin/rm -Rf "${3}/puppet/indirector/instrumentation_data/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/instrumentation_listener.rb" - -/bin/rm -Rf "${3}/puppet/indirector/instrumentation_listener/local.rb" - -/bin/rm -Rf "${3}/puppet/indirector/instrumentation_listener/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/instrumentation_probe.rb" - -/bin/rm -Rf "${3}/puppet/indirector/instrumentation_probe/local.rb" - -/bin/rm -Rf "${3}/puppet/indirector/instrumentation_probe/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/json.rb" - -/bin/rm -Rf "${3}/puppet/indirector/key/ca.rb" - -/bin/rm -Rf "${3}/puppet/indirector/key/disabled_ca.rb" - -/bin/rm -Rf "${3}/puppet/indirector/key/file.rb" - -/bin/rm -Rf "${3}/puppet/indirector/ldap.rb" - -/bin/rm -Rf "${3}/puppet/indirector/memory.rb" - -/bin/rm -Rf "${3}/puppet/indirector/node/active_record.rb" - -/bin/rm -Rf "${3}/puppet/indirector/node/exec.rb" - -/bin/rm -Rf "${3}/puppet/indirector/node/ldap.rb" - -/bin/rm -Rf "${3}/puppet/indirector/node/memory.rb" - -/bin/rm -Rf "${3}/puppet/indirector/node/plain.rb" - -/bin/rm -Rf "${3}/puppet/indirector/node/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/node/store_configs.rb" - -/bin/rm -Rf "${3}/puppet/indirector/node/write_only_yaml.rb" - -/bin/rm -Rf "${3}/puppet/indirector/node/yaml.rb" - -/bin/rm -Rf "${3}/puppet/indirector/none.rb" - -/bin/rm -Rf "${3}/puppet/indirector/plain.rb" - -/bin/rm -Rf "${3}/puppet/indirector/queue.rb" - -/bin/rm -Rf "${3}/puppet/indirector/report/processor.rb" - -/bin/rm -Rf "${3}/puppet/indirector/report/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/report/yaml.rb" - -/bin/rm -Rf "${3}/puppet/indirector/request.rb" - -/bin/rm -Rf "${3}/puppet/indirector/resource/active_record.rb" - -/bin/rm -Rf "${3}/puppet/indirector/resource/ral.rb" - -/bin/rm -Rf "${3}/puppet/indirector/resource/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/resource/store_configs.rb" - -/bin/rm -Rf "${3}/puppet/indirector/resource/validator.rb" - -/bin/rm -Rf "${3}/puppet/indirector/resource_type.rb" - -/bin/rm -Rf "${3}/puppet/indirector/resource_type/parser.rb" - -/bin/rm -Rf "${3}/puppet/indirector/resource_type/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/run/local.rb" - -/bin/rm -Rf "${3}/puppet/indirector/run/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/ssl_file.rb" - -/bin/rm -Rf "${3}/puppet/indirector/status.rb" - -/bin/rm -Rf "${3}/puppet/indirector/status/local.rb" - -/bin/rm -Rf "${3}/puppet/indirector/status/rest.rb" - -/bin/rm -Rf "${3}/puppet/indirector/store_configs.rb" - -/bin/rm -Rf "${3}/puppet/indirector/terminus.rb" - -/bin/rm -Rf "${3}/puppet/indirector/yaml.rb" - -/bin/rm -Rf "${3}/puppet/interface.rb" - -/bin/rm -Rf "${3}/puppet/interface/action.rb" - -/bin/rm -Rf "${3}/puppet/interface/action_builder.rb" - -/bin/rm -Rf "${3}/puppet/interface/action_manager.rb" - -/bin/rm -Rf "${3}/puppet/interface/documentation.rb" - -/bin/rm -Rf "${3}/puppet/interface/face_collection.rb" - -/bin/rm -Rf "${3}/puppet/interface/option.rb" - -/bin/rm -Rf "${3}/puppet/interface/option_builder.rb" - -/bin/rm -Rf "${3}/puppet/interface/option_manager.rb" - -/bin/rm -Rf "${3}/puppet/metatype/manager.rb" - -/bin/rm -Rf "${3}/puppet/module.rb" - -/bin/rm -Rf "${3}/puppet/module_tool.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/applications.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/applications/application.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/applications/builder.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/applications/checksummer.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/applications/generator.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/applications/installer.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/applications/searcher.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/applications/uninstaller.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/applications/unpacker.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/applications/upgrader.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/checksums.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/contents_description.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/dependency.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/errors.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/errors/base.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/errors/installer.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/errors/shared.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/errors/uninstaller.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/errors/upgrader.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/install_directory.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/metadata.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/modulefile.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/shared_behaviors.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/skeleton.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/skeleton/templates/generator/Modulefile.erb" - -/bin/rm -Rf "${3}/puppet/module_tool/skeleton/templates/generator/README.erb" - -/bin/rm -Rf "${3}/puppet/module_tool/skeleton/templates/generator/manifests/init.pp.erb" - -/bin/rm -Rf "${3}/puppet/module_tool/skeleton/templates/generator/spec/spec_helper.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/skeleton/templates/generator/tests/init.pp.erb" - -/bin/rm -Rf "${3}/puppet/module_tool/tar.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/tar/gnu.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/tar/mini.rb" - -/bin/rm -Rf "${3}/puppet/module_tool/tar/solaris.rb" - -/bin/rm -Rf "${3}/puppet/network.rb" - -/bin/rm -Rf "${3}/puppet/network/auth_config_parser.rb" - -/bin/rm -Rf "${3}/puppet/network/authconfig.rb" - -/bin/rm -Rf "${3}/puppet/network/authentication.rb" - -/bin/rm -Rf "${3}/puppet/network/authorization.rb" - -/bin/rm -Rf "${3}/puppet/network/authstore.rb" - -/bin/rm -Rf "${3}/puppet/network/client_request.rb" - -/bin/rm -Rf "${3}/puppet/network/format.rb" - -/bin/rm -Rf "${3}/puppet/network/format_handler.rb" - -/bin/rm -Rf "${3}/puppet/network/formats.rb" - -/bin/rm -Rf "${3}/puppet/network/http.rb" - -/bin/rm -Rf "${3}/puppet/network/http/api.rb" - -/bin/rm -Rf "${3}/puppet/network/http/api/v1.rb" - -/bin/rm -Rf "${3}/puppet/network/http/compression.rb" - -/bin/rm -Rf "${3}/puppet/network/http/connection.rb" - -/bin/rm -Rf "${3}/puppet/network/http/handler.rb" - -/bin/rm -Rf "${3}/puppet/network/http/rack.rb" - -/bin/rm -Rf "${3}/puppet/network/http/rack/httphandler.rb" - -/bin/rm -Rf "${3}/puppet/network/http/rack/rest.rb" - -/bin/rm -Rf "${3}/puppet/network/http/webrick.rb" - -/bin/rm -Rf "${3}/puppet/network/http/webrick/rest.rb" - -/bin/rm -Rf "${3}/puppet/network/http_pool.rb" - -/bin/rm -Rf "${3}/puppet/network/resolver.rb" - -/bin/rm -Rf "${3}/puppet/network/rest_controller.rb" - -/bin/rm -Rf "${3}/puppet/network/rights.rb" - -/bin/rm -Rf "${3}/puppet/network/server.rb" - -/bin/rm -Rf "${3}/puppet/node.rb" - -/bin/rm -Rf "${3}/puppet/node/environment.rb" - -/bin/rm -Rf "${3}/puppet/node/facts.rb" - -/bin/rm -Rf "${3}/puppet/parameter.rb" - -/bin/rm -Rf "${3}/puppet/parameter/package_options.rb" - -/bin/rm -Rf "${3}/puppet/parameter/path.rb" - -/bin/rm -Rf "${3}/puppet/parameter/value.rb" - -/bin/rm -Rf "${3}/puppet/parameter/value_collection.rb" - -/bin/rm -Rf "${3}/puppet/parser.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/arithmetic_operator.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/astarray.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/asthash.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/block_expression.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/boolean_operator.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/branch.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/caseopt.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/casestatement.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/collection.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/collexpr.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/comparison_operator.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/definition.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/else.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/function.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/hostclass.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/ifstatement.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/in_operator.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/lambda.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/leaf.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/match_operator.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/method_call.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/minus.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/node.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/nop.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/not.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/relationship.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/resource.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/resource_defaults.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/resource_instance.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/resource_override.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/resource_reference.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/resourceparam.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/selector.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/tag.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/top_level_construct.rb" - -/bin/rm -Rf "${3}/puppet/parser/ast/vardef.rb" - -/bin/rm -Rf "${3}/puppet/parser/collector.rb" - -/bin/rm -Rf "${3}/puppet/parser/compiler.rb" - -/bin/rm -Rf "${3}/puppet/parser/e_parser_adapter.rb" - -/bin/rm -Rf "${3}/puppet/parser/files.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/collect.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/create_resources.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/defined.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/each.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/extlookup.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/fail.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/file.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/foreach.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/fqdn_rand.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/generate.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/hiera.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/hiera_array.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/hiera_hash.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/hiera_include.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/include.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/inline_template.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/md5.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/realize.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/reduce.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/regsubst.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/reject.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/require.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/search.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/select.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/sha1.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/shellquote.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/slice.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/split.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/sprintf.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/tag.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/tagged.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/template.rb" - -/bin/rm -Rf "${3}/puppet/parser/functions/versioncmp.rb" - -/bin/rm -Rf "${3}/puppet/parser/grammar.ra" - -/bin/rm -Rf "${3}/puppet/parser/lexer.rb" - -/bin/rm -Rf "${3}/puppet/parser/makefile" - -/bin/rm -Rf "${3}/puppet/parser/methods.rb" - -/bin/rm -Rf "${3}/puppet/parser/parser.rb" - -/bin/rm -Rf "${3}/puppet/parser/parser_factory.rb" - -/bin/rm -Rf "${3}/puppet/parser/parser_support.rb" - -/bin/rm -Rf "${3}/puppet/parser/relationship.rb" - -/bin/rm -Rf "${3}/puppet/parser/resource.rb" - -/bin/rm -Rf "${3}/puppet/parser/resource/param.rb" - -/bin/rm -Rf "${3}/puppet/parser/scope.rb" - -/bin/rm -Rf "${3}/puppet/parser/templatewrapper.rb" - -/bin/rm -Rf "${3}/puppet/parser/type_loader.rb" - -/bin/rm -Rf "${3}/puppet/parser/yaml_trimmer.rb" - -/bin/rm -Rf "${3}/puppet/pops.rb" - -/bin/rm -Rf "${3}/puppet/pops/adaptable.rb" - -/bin/rm -Rf "${3}/puppet/pops/adapters.rb" - -/bin/rm -Rf "${3}/puppet/pops/containment.rb" - -/bin/rm -Rf "${3}/puppet/pops/issues.rb" - -/bin/rm -Rf "${3}/puppet/pops/label_provider.rb" - -/bin/rm -Rf "${3}/puppet/pops/model/ast_transformer.rb" - -/bin/rm -Rf "${3}/puppet/pops/model/ast_tree_dumper.rb" - -/bin/rm -Rf "${3}/puppet/pops/model/factory.rb" - -/bin/rm -Rf "${3}/puppet/pops/model/model.rb" - -/bin/rm -Rf "${3}/puppet/pops/model/model_label_provider.rb" - -/bin/rm -Rf "${3}/puppet/pops/model/model_tree_dumper.rb" - -/bin/rm -Rf "${3}/puppet/pops/model/tree_dumper.rb" - -/bin/rm -Rf "${3}/puppet/pops/parser/egrammar.ra" - -/bin/rm -Rf "${3}/puppet/pops/parser/eparser.rb" - -/bin/rm -Rf "${3}/puppet/pops/parser/grammar.ra" - -/bin/rm -Rf "${3}/puppet/pops/parser/lexer.rb" - -/bin/rm -Rf "${3}/puppet/pops/parser/makefile" - -/bin/rm -Rf "${3}/puppet/pops/parser/parser_support.rb" - -/bin/rm -Rf "${3}/puppet/pops/patterns.rb" - -/bin/rm -Rf "${3}/puppet/pops/utils.rb" - -/bin/rm -Rf "${3}/puppet/pops/validation.rb" - -/bin/rm -Rf "${3}/puppet/pops/validation/checker3_1.rb" - -/bin/rm -Rf "${3}/puppet/pops/validation/validator_factory_3_1.rb" - -/bin/rm -Rf "${3}/puppet/pops/visitable.rb" - -/bin/rm -Rf "${3}/puppet/pops/visitor.rb" - -/bin/rm -Rf "${3}/puppet/property.rb" - -/bin/rm -Rf "${3}/puppet/property/ensure.rb" - -/bin/rm -Rf "${3}/puppet/property/keyvalue.rb" - -/bin/rm -Rf "${3}/puppet/property/list.rb" - -/bin/rm -Rf "${3}/puppet/property/ordered_list.rb" - -/bin/rm -Rf "${3}/puppet/provider.rb" - -/bin/rm -Rf "${3}/puppet/provider/aixobject.rb" - -/bin/rm -Rf "${3}/puppet/provider/augeas/augeas.rb" - -/bin/rm -Rf "${3}/puppet/provider/cisco.rb" - -/bin/rm -Rf "${3}/puppet/provider/command.rb" - -/bin/rm -Rf "${3}/puppet/provider/computer/computer.rb" - -/bin/rm -Rf "${3}/puppet/provider/confine.rb" - -/bin/rm -Rf "${3}/puppet/provider/confine/exists.rb" - -/bin/rm -Rf "${3}/puppet/provider/confine/false.rb" - -/bin/rm -Rf "${3}/puppet/provider/confine/feature.rb" - -/bin/rm -Rf "${3}/puppet/provider/confine/true.rb" - -/bin/rm -Rf "${3}/puppet/provider/confine/variable.rb" - -/bin/rm -Rf "${3}/puppet/provider/confine_collection.rb" - -/bin/rm -Rf "${3}/puppet/provider/confiner.rb" - -/bin/rm -Rf "${3}/puppet/provider/cron/crontab.rb" - -/bin/rm -Rf "${3}/puppet/provider/exec.rb" - -/bin/rm -Rf "${3}/puppet/provider/exec/posix.rb" - -/bin/rm -Rf "${3}/puppet/provider/exec/shell.rb" - -/bin/rm -Rf "${3}/puppet/provider/exec/windows.rb" - -/bin/rm -Rf "${3}/puppet/provider/file/posix.rb" - -/bin/rm -Rf "${3}/puppet/provider/file/windows.rb" - -/bin/rm -Rf "${3}/puppet/provider/group/aix.rb" - -/bin/rm -Rf "${3}/puppet/provider/group/directoryservice.rb" - -/bin/rm -Rf "${3}/puppet/provider/group/groupadd.rb" - -/bin/rm -Rf "${3}/puppet/provider/group/ldap.rb" - -/bin/rm -Rf "${3}/puppet/provider/group/pw.rb" - -/bin/rm -Rf "${3}/puppet/provider/group/windows_adsi.rb" - -/bin/rm -Rf "${3}/puppet/provider/host/parsed.rb" - -/bin/rm -Rf "${3}/puppet/provider/interface/cisco.rb" - -/bin/rm -Rf "${3}/puppet/provider/ldap.rb" - -/bin/rm -Rf "${3}/puppet/provider/macauthorization/macauthorization.rb" - -/bin/rm -Rf "${3}/puppet/provider/mailalias/aliases.rb" - -/bin/rm -Rf "${3}/puppet/provider/maillist/mailman.rb" - -/bin/rm -Rf "${3}/puppet/provider/mcx/mcxcontent.rb" - -/bin/rm -Rf "${3}/puppet/provider/mount.rb" - -/bin/rm -Rf "${3}/puppet/provider/mount/parsed.rb" - -/bin/rm -Rf "${3}/puppet/provider/naginator.rb" - -/bin/rm -Rf "${3}/puppet/provider/nameservice.rb" - -/bin/rm -Rf "${3}/puppet/provider/nameservice/directoryservice.rb" - -/bin/rm -Rf "${3}/puppet/provider/nameservice/objectadd.rb" - -/bin/rm -Rf "${3}/puppet/provider/nameservice/pw.rb" - -/bin/rm -Rf "${3}/puppet/provider/network_device.rb" - -/bin/rm -Rf "${3}/puppet/provider/package.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/aix.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/appdmg.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/apple.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/apt.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/aptitude.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/aptrpm.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/blastwave.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/dpkg.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/fink.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/freebsd.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/gem.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/hpux.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/macports.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/msi.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/nim.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/openbsd.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/opkg.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/pacman.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/pip.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/pkg.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/pkgdmg.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/pkgin.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/pkgutil.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/portage.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/ports.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/portupgrade.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/rpm.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/rug.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/sun.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/sunfreeware.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/up2date.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/urpmi.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/windows.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/windows/exe_package.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/windows/msi_package.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/windows/package.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/yum.rb" - -/bin/rm -Rf "${3}/puppet/provider/package/yumhelper.py" - -/bin/rm -Rf "${3}/puppet/provider/package/zypper.rb" - -/bin/rm -Rf "${3}/puppet/provider/parsedfile.rb" - -/bin/rm -Rf "${3}/puppet/provider/port/parsed.rb" - -/bin/rm -Rf "${3}/puppet/provider/scheduled_task/win32_taskscheduler.rb" - -/bin/rm -Rf "${3}/puppet/provider/selboolean/getsetsebool.rb" - -/bin/rm -Rf "${3}/puppet/provider/selmodule/semodule.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/base.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/bsd.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/daemontools.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/debian.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/freebsd.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/gentoo.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/init.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/launchd.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/openrc.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/openwrt.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/redhat.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/runit.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/service.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/smf.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/src.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/systemd.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/upstart.rb" - -/bin/rm -Rf "${3}/puppet/provider/service/windows.rb" - -/bin/rm -Rf "${3}/puppet/provider/ssh_authorized_key/parsed.rb" - -/bin/rm -Rf "${3}/puppet/provider/sshkey/parsed.rb" - -/bin/rm -Rf "${3}/puppet/provider/user/aix.rb" - -/bin/rm -Rf "${3}/puppet/provider/user/directoryservice.rb" - -/bin/rm -Rf "${3}/puppet/provider/user/hpux.rb" - -/bin/rm -Rf "${3}/puppet/provider/user/ldap.rb" - -/bin/rm -Rf "${3}/puppet/provider/user/pw.rb" - -/bin/rm -Rf "${3}/puppet/provider/user/user_role_add.rb" - -/bin/rm -Rf "${3}/puppet/provider/user/useradd.rb" - -/bin/rm -Rf "${3}/puppet/provider/user/windows_adsi.rb" - -/bin/rm -Rf "${3}/puppet/provider/vlan/cisco.rb" - -/bin/rm -Rf "${3}/puppet/provider/zfs/zfs.rb" - -/bin/rm -Rf "${3}/puppet/provider/zone/solaris.rb" - -/bin/rm -Rf "${3}/puppet/provider/zpool/zpool.rb" - -/bin/rm -Rf "${3}/puppet/rails.rb" - -/bin/rm -Rf "${3}/puppet/rails/benchmark.rb" - -/bin/rm -Rf "${3}/puppet/rails/database/001_add_created_at_to_all_tables.rb" - -/bin/rm -Rf "${3}/puppet/rails/database/002_remove_duplicated_index_on_all_tables.rb" - -/bin/rm -Rf "${3}/puppet/rails/database/003_add_environment_to_host.rb" - -/bin/rm -Rf "${3}/puppet/rails/database/004_add_inventory_service_tables.rb" - -/bin/rm -Rf "${3}/puppet/rails/database/schema.rb" - -/bin/rm -Rf "${3}/puppet/rails/fact_name.rb" - -/bin/rm -Rf "${3}/puppet/rails/fact_value.rb" - -/bin/rm -Rf "${3}/puppet/rails/host.rb" - -/bin/rm -Rf "${3}/puppet/rails/inventory_fact.rb" - -/bin/rm -Rf "${3}/puppet/rails/inventory_node.rb" - -/bin/rm -Rf "${3}/puppet/rails/param_name.rb" - -/bin/rm -Rf "${3}/puppet/rails/param_value.rb" - -/bin/rm -Rf "${3}/puppet/rails/puppet_tag.rb" - -/bin/rm -Rf "${3}/puppet/rails/resource.rb" - -/bin/rm -Rf "${3}/puppet/rails/resource_tag.rb" - -/bin/rm -Rf "${3}/puppet/rails/source_file.rb" - -/bin/rm -Rf "${3}/puppet/rb_tree_map.rb" - -/bin/rm -Rf "${3}/puppet/reference/configuration.rb" - -/bin/rm -Rf "${3}/puppet/reference/function.rb" - -/bin/rm -Rf "${3}/puppet/reference/indirection.rb" - -/bin/rm -Rf "${3}/puppet/reference/metaparameter.rb" - -/bin/rm -Rf "${3}/puppet/reference/providers.rb" - -/bin/rm -Rf "${3}/puppet/reference/report.rb" - -/bin/rm -Rf "${3}/puppet/reference/type.rb" - -/bin/rm -Rf "${3}/puppet/relationship.rb" - -/bin/rm -Rf "${3}/puppet/reports.rb" - -/bin/rm -Rf "${3}/puppet/reports/http.rb" - -/bin/rm -Rf "${3}/puppet/reports/log.rb" - -/bin/rm -Rf "${3}/puppet/reports/rrdgraph.rb" - -/bin/rm -Rf "${3}/puppet/reports/store.rb" - -/bin/rm -Rf "${3}/puppet/reports/tagmail.rb" - -/bin/rm -Rf "${3}/puppet/resource.rb" - -/bin/rm -Rf "${3}/puppet/resource/catalog.rb" - -/bin/rm -Rf "${3}/puppet/resource/status.rb" - -/bin/rm -Rf "${3}/puppet/resource/type.rb" - -/bin/rm -Rf "${3}/puppet/resource/type_collection.rb" - -/bin/rm -Rf "${3}/puppet/resource/type_collection_helper.rb" - -/bin/rm -Rf "${3}/puppet/run.rb" - -/bin/rm -Rf "${3}/puppet/scheduler.rb" - -/bin/rm -Rf "${3}/puppet/scheduler/job.rb" - -/bin/rm -Rf "${3}/puppet/scheduler/scheduler.rb" - -/bin/rm -Rf "${3}/puppet/scheduler/splay_job.rb" - -/bin/rm -Rf "${3}/puppet/scheduler/timer.rb" - -/bin/rm -Rf "${3}/puppet/settings.rb" - -/bin/rm -Rf "${3}/puppet/settings/base_setting.rb" - -/bin/rm -Rf "${3}/puppet/settings/boolean_setting.rb" - -/bin/rm -Rf "${3}/puppet/settings/config_file.rb" - -/bin/rm -Rf "${3}/puppet/settings/directory_setting.rb" - -/bin/rm -Rf "${3}/puppet/settings/duration_setting.rb" - -/bin/rm -Rf "${3}/puppet/settings/errors.rb" - -/bin/rm -Rf "${3}/puppet/settings/file_setting.rb" - -/bin/rm -Rf "${3}/puppet/settings/path_setting.rb" - -/bin/rm -Rf "${3}/puppet/settings/string_setting.rb" - -/bin/rm -Rf "${3}/puppet/settings/terminus_setting.rb" - -/bin/rm -Rf "${3}/puppet/settings/value_translator.rb" - -/bin/rm -Rf "${3}/puppet/simple_graph.rb" - -/bin/rm -Rf "${3}/puppet/ssl.rb" - -/bin/rm -Rf "${3}/puppet/ssl/base.rb" - -/bin/rm -Rf "${3}/puppet/ssl/certificate.rb" - -/bin/rm -Rf "${3}/puppet/ssl/certificate_authority.rb" - -/bin/rm -Rf "${3}/puppet/ssl/certificate_authority/interface.rb" - -/bin/rm -Rf "${3}/puppet/ssl/certificate_factory.rb" - -/bin/rm -Rf "${3}/puppet/ssl/certificate_request.rb" - -/bin/rm -Rf "${3}/puppet/ssl/certificate_revocation_list.rb" - -/bin/rm -Rf "${3}/puppet/ssl/certificate_signer.rb" - -/bin/rm -Rf "${3}/puppet/ssl/configuration.rb" - -/bin/rm -Rf "${3}/puppet/ssl/digest.rb" - -/bin/rm -Rf "${3}/puppet/ssl/host.rb" - -/bin/rm -Rf "${3}/puppet/ssl/inventory.rb" - -/bin/rm -Rf "${3}/puppet/ssl/key.rb" - -/bin/rm -Rf "${3}/puppet/ssl/validator.rb" - -/bin/rm -Rf "${3}/puppet/status.rb" - -/bin/rm -Rf "${3}/puppet/test/test_helper.rb" - -/bin/rm -Rf "${3}/puppet/transaction.rb" - -/bin/rm -Rf "${3}/puppet/transaction/event.rb" - -/bin/rm -Rf "${3}/puppet/transaction/event_manager.rb" - -/bin/rm -Rf "${3}/puppet/transaction/report.rb" - -/bin/rm -Rf "${3}/puppet/transaction/resource_harness.rb" - -/bin/rm -Rf "${3}/puppet/type.rb" - -/bin/rm -Rf "${3}/puppet/type/augeas.rb" - -/bin/rm -Rf "${3}/puppet/type/component.rb" - -/bin/rm -Rf "${3}/puppet/type/computer.rb" - -/bin/rm -Rf "${3}/puppet/type/cron.rb" - -/bin/rm -Rf "${3}/puppet/type/exec.rb" - -/bin/rm -Rf "${3}/puppet/type/file.rb" - -/bin/rm -Rf "${3}/puppet/type/file/checksum.rb" - -/bin/rm -Rf "${3}/puppet/type/file/content.rb" - -/bin/rm -Rf "${3}/puppet/type/file/ctime.rb" - -/bin/rm -Rf "${3}/puppet/type/file/ensure.rb" - -/bin/rm -Rf "${3}/puppet/type/file/group.rb" - -/bin/rm -Rf "${3}/puppet/type/file/mode.rb" - -/bin/rm -Rf "${3}/puppet/type/file/mtime.rb" - -/bin/rm -Rf "${3}/puppet/type/file/owner.rb" - -/bin/rm -Rf "${3}/puppet/type/file/selcontext.rb" - -/bin/rm -Rf "${3}/puppet/type/file/source.rb" - -/bin/rm -Rf "${3}/puppet/type/file/target.rb" - -/bin/rm -Rf "${3}/puppet/type/file/type.rb" - -/bin/rm -Rf "${3}/puppet/type/filebucket.rb" - -/bin/rm -Rf "${3}/puppet/type/group.rb" - -/bin/rm -Rf "${3}/puppet/type/host.rb" - -/bin/rm -Rf "${3}/puppet/type/interface.rb" - -/bin/rm -Rf "${3}/puppet/type/k5login.rb" - -/bin/rm -Rf "${3}/puppet/type/macauthorization.rb" - -/bin/rm -Rf "${3}/puppet/type/mailalias.rb" - -/bin/rm -Rf "${3}/puppet/type/maillist.rb" - -/bin/rm -Rf "${3}/puppet/type/mcx.rb" - -/bin/rm -Rf "${3}/puppet/type/mount.rb" - -/bin/rm -Rf "${3}/puppet/type/nagios_command.rb" - -/bin/rm -Rf "${3}/puppet/type/nagios_contact.rb" - -/bin/rm -Rf "${3}/puppet/type/nagios_contactgroup.rb" - -/bin/rm -Rf "${3}/puppet/type/nagios_host.rb" - -/bin/rm -Rf "${3}/puppet/type/nagios_hostdependency.rb" - -/bin/rm -Rf "${3}/puppet/type/nagios_hostescalation.rb" - -/bin/rm -Rf "${3}/puppet/type/nagios_hostextinfo.rb" - -/bin/rm -Rf "${3}/puppet/type/nagios_hostgroup.rb" - -/bin/rm -Rf "${3}/puppet/type/nagios_service.rb" - -/bin/rm -Rf "${3}/puppet/type/nagios_servicedependency.rb" - -/bin/rm -Rf "${3}/puppet/type/nagios_serviceescalation.rb" - -/bin/rm -Rf "${3}/puppet/type/nagios_serviceextinfo.rb" - -/bin/rm -Rf "${3}/puppet/type/nagios_servicegroup.rb" - -/bin/rm -Rf "${3}/puppet/type/nagios_timeperiod.rb" - -/bin/rm -Rf "${3}/puppet/type/notify.rb" - -/bin/rm -Rf "${3}/puppet/type/package.rb" - -/bin/rm -Rf "${3}/puppet/type/port.rb" - -/bin/rm -Rf "${3}/puppet/type/resources.rb" - -/bin/rm -Rf "${3}/puppet/type/router.rb" - -/bin/rm -Rf "${3}/puppet/type/schedule.rb" - -/bin/rm -Rf "${3}/puppet/type/scheduled_task.rb" - -/bin/rm -Rf "${3}/puppet/type/selboolean.rb" - -/bin/rm -Rf "${3}/puppet/type/selmodule.rb" - -/bin/rm -Rf "${3}/puppet/type/service.rb" - -/bin/rm -Rf "${3}/puppet/type/ssh_authorized_key.rb" - -/bin/rm -Rf "${3}/puppet/type/sshkey.rb" - -/bin/rm -Rf "${3}/puppet/type/stage.rb" - -/bin/rm -Rf "${3}/puppet/type/tidy.rb" - -/bin/rm -Rf "${3}/puppet/type/user.rb" - -/bin/rm -Rf "${3}/puppet/type/vlan.rb" - -/bin/rm -Rf "${3}/puppet/type/whit.rb" - -/bin/rm -Rf "${3}/puppet/type/yumrepo.rb" - -/bin/rm -Rf "${3}/puppet/type/zfs.rb" - -/bin/rm -Rf "${3}/puppet/type/zone.rb" - -/bin/rm -Rf "${3}/puppet/type/zpool.rb" - -/bin/rm -Rf "${3}/puppet/util.rb" - -/bin/rm -Rf "${3}/puppet/util/adsi.rb" - -/bin/rm -Rf "${3}/puppet/util/autoload.rb" - -/bin/rm -Rf "${3}/puppet/util/backups.rb" - -/bin/rm -Rf "${3}/puppet/util/cacher.rb" - -/bin/rm -Rf "${3}/puppet/util/checksums.rb" - -/bin/rm -Rf "${3}/puppet/util/classgen.rb" - -/bin/rm -Rf "${3}/puppet/util/colors.rb" - -/bin/rm -Rf "${3}/puppet/util/command_line.rb" - -/bin/rm -Rf "${3}/puppet/util/command_line/puppet_option_parser.rb" - -/bin/rm -Rf "${3}/puppet/util/command_line/trollop.rb" - -/bin/rm -Rf "${3}/puppet/util/constant_inflector.rb" - -/bin/rm -Rf "${3}/puppet/util/diff.rb" - -/bin/rm -Rf "${3}/puppet/util/docs.rb" - -/bin/rm -Rf "${3}/puppet/util/errors.rb" - -/bin/rm -Rf "${3}/puppet/util/execution.rb" - -/bin/rm -Rf "${3}/puppet/util/execution_stub.rb" - -/bin/rm -Rf "${3}/puppet/util/feature.rb" - -/bin/rm -Rf "${3}/puppet/util/fileparsing.rb" - -/bin/rm -Rf "${3}/puppet/util/filetype.rb" - -/bin/rm -Rf "${3}/puppet/util/graph.rb" - -/bin/rm -Rf "${3}/puppet/util/inifile.rb" - -/bin/rm -Rf "${3}/puppet/util/inline_docs.rb" - -/bin/rm -Rf "${3}/puppet/util/instance_loader.rb" - -/bin/rm -Rf "${3}/puppet/util/instrumentation.rb" - -/bin/rm -Rf "${3}/puppet/util/instrumentation/data.rb" - -/bin/rm -Rf "${3}/puppet/util/instrumentation/indirection_probe.rb" - -/bin/rm -Rf "${3}/puppet/util/instrumentation/instrumentable.rb" - -/bin/rm -Rf "${3}/puppet/util/instrumentation/listener.rb" - -/bin/rm -Rf "${3}/puppet/util/instrumentation/listeners/log.rb" - -/bin/rm -Rf "${3}/puppet/util/instrumentation/listeners/performance.rb" - -/bin/rm -Rf "${3}/puppet/util/json_lockfile.rb" - -/bin/rm -Rf "${3}/puppet/util/ldap.rb" - -/bin/rm -Rf "${3}/puppet/util/ldap/connection.rb" - -/bin/rm -Rf "${3}/puppet/util/ldap/generator.rb" - -/bin/rm -Rf "${3}/puppet/util/ldap/manager.rb" - -/bin/rm -Rf "${3}/puppet/util/libuser.conf" - -/bin/rm -Rf "${3}/puppet/util/libuser.rb" - -/bin/rm -Rf "${3}/puppet/util/loadedfile.rb" - -/bin/rm -Rf "${3}/puppet/util/lockfile.rb" - -/bin/rm -Rf "${3}/puppet/util/log.rb" - -/bin/rm -Rf "${3}/puppet/util/log/destination.rb" - -/bin/rm -Rf "${3}/puppet/util/log/destinations.rb" - -/bin/rm -Rf "${3}/puppet/util/log/rate_limited_logger.rb" - -/bin/rm -Rf "${3}/puppet/util/log_paths.rb" - -/bin/rm -Rf "${3}/puppet/util/logging.rb" - -/bin/rm -Rf "${3}/puppet/util/metaid.rb" - -/bin/rm -Rf "${3}/puppet/util/methodhelper.rb" - -/bin/rm -Rf "${3}/puppet/util/metric.rb" - -/bin/rm -Rf "${3}/puppet/util/monkey_patches.rb" - -/bin/rm -Rf "${3}/puppet/util/monkey_patches/lines.rb" - -/bin/rm -Rf "${3}/puppet/util/nagios_maker.rb" - -/bin/rm -Rf "${3}/puppet/util/network_device.rb" - -/bin/rm -Rf "${3}/puppet/util/network_device/base.rb" - -/bin/rm -Rf "${3}/puppet/util/network_device/cisco.rb" - -/bin/rm -Rf "${3}/puppet/util/network_device/cisco/device.rb" - -/bin/rm -Rf "${3}/puppet/util/network_device/cisco/facts.rb" - -/bin/rm -Rf "${3}/puppet/util/network_device/cisco/interface.rb" - -/bin/rm -Rf "${3}/puppet/util/network_device/config.rb" - -/bin/rm -Rf "${3}/puppet/util/network_device/ipcalc.rb" - -/bin/rm -Rf "${3}/puppet/util/network_device/transport.rb" - -/bin/rm -Rf "${3}/puppet/util/network_device/transport/base.rb" - -/bin/rm -Rf "${3}/puppet/util/network_device/transport/ssh.rb" - -/bin/rm -Rf "${3}/puppet/util/network_device/transport/telnet.rb" - -/bin/rm -Rf "${3}/puppet/util/package.rb" - -/bin/rm -Rf "${3}/puppet/util/pidlock.rb" - -/bin/rm -Rf "${3}/puppet/util/platform.rb" - -/bin/rm -Rf "${3}/puppet/util/plugins.rb" - -/bin/rm -Rf "${3}/puppet/util/posix.rb" - -/bin/rm -Rf "${3}/puppet/util/profiler.rb" - -/bin/rm -Rf "${3}/puppet/util/profiler/logging.rb" - -/bin/rm -Rf "${3}/puppet/util/profiler/none.rb" - -/bin/rm -Rf "${3}/puppet/util/profiler/object_counts.rb" - -/bin/rm -Rf "${3}/puppet/util/profiler/wall_clock.rb" - -/bin/rm -Rf "${3}/puppet/util/provider_features.rb" - -/bin/rm -Rf "${3}/puppet/util/pson.rb" - -/bin/rm -Rf "${3}/puppet/util/queue.rb" - -/bin/rm -Rf "${3}/puppet/util/queue/stomp.rb" - -/bin/rm -Rf "${3}/puppet/util/rails/cache_accumulator.rb" - -/bin/rm -Rf "${3}/puppet/util/rails/collection_merger.rb" - -/bin/rm -Rf "${3}/puppet/util/rails/reference_serializer.rb" - -/bin/rm -Rf "${3}/puppet/util/rdoc.rb" - -/bin/rm -Rf "${3}/puppet/util/rdoc/code_objects.rb" - -/bin/rm -Rf "${3}/puppet/util/rdoc/generators/puppet_generator.rb" - -/bin/rm -Rf "${3}/puppet/util/rdoc/generators/template/puppet/puppet.rb" - -/bin/rm -Rf "${3}/puppet/util/rdoc/parser.rb" - -/bin/rm -Rf "${3}/puppet/util/reference.rb" - -/bin/rm -Rf "${3}/puppet/util/resource_template.rb" - -/bin/rm -Rf "${3}/puppet/util/retryaction.rb" - -/bin/rm -Rf "${3}/puppet/util/rubygems.rb" - -/bin/rm -Rf "${3}/puppet/util/run_mode.rb" - -/bin/rm -Rf "${3}/puppet/util/selinux.rb" - -/bin/rm -Rf "${3}/puppet/util/ssl.rb" - -/bin/rm -Rf "${3}/puppet/util/storage.rb" - -/bin/rm -Rf "${3}/puppet/util/subclass_loader.rb" - -/bin/rm -Rf "${3}/puppet/util/suidmanager.rb" - -/bin/rm -Rf "${3}/puppet/util/symbolic_file_mode.rb" - -/bin/rm -Rf "${3}/puppet/util/tagging.rb" - -/bin/rm -Rf "${3}/puppet/util/terminal.rb" - -/bin/rm -Rf "${3}/puppet/util/user_attr.rb" - -/bin/rm -Rf "${3}/puppet/util/warnings.rb" - -/bin/rm -Rf "${3}/puppet/util/windows.rb" - -/bin/rm -Rf "${3}/puppet/util/windows/error.rb" - -/bin/rm -Rf "${3}/puppet/util/windows/file.rb" - -/bin/rm -Rf "${3}/puppet/util/windows/process.rb" - -/bin/rm -Rf "${3}/puppet/util/windows/registry.rb" - -/bin/rm -Rf "${3}/puppet/util/windows/root_certs.rb" - -/bin/rm -Rf "${3}/puppet/util/windows/security.rb" - -/bin/rm -Rf "${3}/puppet/util/windows/sid.rb" - -/bin/rm -Rf "${3}/puppet/util/windows/user.rb" - -/bin/rm -Rf "${3}/puppet/util/zaml.rb" - -/bin/rm -Rf "${3}/puppet/vendor.rb" - -/bin/rm -Rf "${3}/puppet/vendor/load_safe_yaml.rb" - -/bin/rm -Rf "${3}/puppet/vendor/require_vendored.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/CHANGES.md" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/Gemfile" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/LICENSE.txt" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/README.md" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/Rakefile" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/deep.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/parse/date.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/parse/hexadecimal.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/parse/sexagesimal.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/psych_handler.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/psych_resolver.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/resolver.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/safe_to_ruby_visitor.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/syck_hack.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/syck_node_monkeypatch.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/syck_resolver.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/transform.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/transform/to_boolean.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/transform/to_date.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/transform/to_float.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/transform/to_integer.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/transform/to_nil.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/transform/to_symbol.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/transform/transformation_map.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/lib/safe_yaml/version.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/run_specs_all_ruby_versions.sh" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/safe_yaml.gemspec" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/spec/exploit.1.9.2.yaml" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/spec/exploit.1.9.3.yaml" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/spec/psych_resolver_spec.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/spec/resolver_specs.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/spec/safe_yaml_spec.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/spec/spec_helper.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/spec/support/exploitable_back_door.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/spec/syck_resolver_spec.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/spec/transform/base64_spec.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/spec/transform/to_date_spec.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/spec/transform/to_float_spec.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/spec/transform/to_integer_spec.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml/spec/transform/to_symbol_spec.rb" - -/bin/rm -Rf "${3}/puppet/vendor/safe_yaml_patches.rb" - -/bin/rm -Rf "${3}/puppet/version.rb" - -/bin/rm -Rf "${3}/semver.rb" - - - - -/bin/rm -Rf "${3}/extlookup2hiera" - -/bin/rm -Rf "${3}/puppet" - - - -# remove old doc files -/bin/rm -Rf "${3}/" - -# These files used to live in the sbindir but were -# removed in Pupppet >= 3.0. Remove them -/bin/rm -Rf "${3}/puppetca" -/bin/rm -Rf "${3}/puppetd" -/bin/rm -Rf "${3}/puppetmasterd" -/bin/rm -Rf "${3}/puppetqd" -/bin/rm -Rf "${3}/puppetrun" diff --git a/ext/osx/preflight.erb b/ext/osx/preflight.erb new file mode 100755 index 000000000..d04f30560 --- /dev/null +++ b/ext/osx/preflight.erb @@ -0,0 +1,37 @@ +#!/bin/bash +# +# Make sure that old puppet cruft is removed +# This also allows us to downgrade puppet as +# it's more likely that installing old versions +# over new will cause issues. +# +# ${3} is the destination volume so that this works correctly +# when being installed to volumes other than the current OS. + +<% begin %> +<% require 'rubygems' %> +<% rescue LoadError %> +<% end %> +<% require 'rake' %> + +<% Dir.chdir("lib") %> +<% FileList["**/*"].select {|i| File.file?(i)}.each do |file| %> +/bin/rm -Rf "${3}<%= @apple_libdir %>/<%=file%>" +<% end %> + +<% Dir.chdir("../bin") %> +<% FileList["**/*"].select {|i| File.file?(i)}.each do |file| %> +/bin/rm -Rf "${3}<%= @apple_bindir %>/<%=file%>" +<% end %> +<% Dir.chdir("..") %> + +# remove old doc files +/bin/rm -Rf "${3}<%= @apple_docdir %>/<%= @package_name %>" + +# These files used to live in the sbindir but were +# removed in Pupppet >= 3.0. Remove them +/bin/rm -Rf "${3}<%= @apple_sbindir %>/puppetca" +/bin/rm -Rf "${3}<%= @apple_sbindir %>/puppetd" +/bin/rm -Rf "${3}<%= @apple_sbindir %>/puppetmasterd" +/bin/rm -Rf "${3}<%= @apple_sbindir %>/puppetqd" +/bin/rm -Rf "${3}<%= @apple_sbindir %>/puppetrun" diff --git a/ext/osx/prototype.plist b/ext/osx/prototype.plist.erb index 4848868b0..e56659a6b 100644 --- a/ext/osx/prototype.plist +++ b/ext/osx/prototype.plist.erb @@ -3,15 +3,15 @@ <plist version="1.0"> <dict> <key>CFBundleIdentifier</key> - <string></string> + <string><%= @title %></string> <key>CFBundleShortVersionString</key> - <string>3.2.4</string> + <string><%= @version %></string> <key>IFMajorVersion</key> - <integer></integer> + <integer><%= @package_major_version %></integer> <key>IFMinorVersion</key> - <integer></integer> + <integer><%= @package_minor_version %></integer> <key>IFPkgBuildDate</key> - <date>2013-08-14-14-59-40</date> + <date><%= @build_date %></date> <key>IFPkgFlagAllowBackRev</key> <false/> <key>IFPkgFlagAuthorizationAction</key> @@ -29,7 +29,7 @@ <key>IFPkgFlagRelocatable</key> <false/> <key>IFPkgFlagRestartAction</key> - <string></string> + <string><%= @pm_restart %></string> <key>IFPkgFlagRootVolumeOnly</key> <true/> <key>IFPkgFlagUpdateInstalledLanguages</key> diff --git a/ext/redhat/puppet.spec b/ext/redhat/puppet.spec index dac007a75..2d9bccada 100644 --- a/ext/redhat/puppet.spec +++ b/ext/redhat/puppet.spec @@ -16,8 +16,8 @@ %endif # VERSION is subbed out during rake srpm process -%global realversion 3.2.4 -%global rpmversion 3.2.4 +%global realversion 3.3.0 +%global rpmversion 3.3.0 %global confdir ext/redhat @@ -98,7 +98,6 @@ The server can also function as a certificate authority and file server. %prep %setup -q -n %{name}-%{realversion} -patch -s -p1 < ext/redhat/rundir-perms.patch %build @@ -392,8 +391,8 @@ fi rm -rf %{buildroot} %changelog -* Wed Aug 14 2013 Puppet Labs Release <info@puppetlabs.com> - 3.2.4-1 -- Build for 3.2.4 +* Thu Sep 12 2013 Puppet Labs Release <info@puppetlabs.com> - 3.3.0-1 +- Build for 3.3.0 * Thu Jun 27 2013 Matthaus Owens <matthaus@puppetlabs.com> - 3.2.3-0.1rc0 - Bump requires on ruby-rgen to 0.6.5 diff --git a/ext/redhat/rundir-perms.patch b/ext/redhat/rundir-perms.patch deleted file mode 100644 index 25d1fcfbe..000000000 --- a/ext/redhat/rundir-perms.patch +++ /dev/null @@ -1,28 +0,0 @@ -From c181799a30427966cbe028fde7b390cac9cf44e9 Mon Sep 17 00:00:00 2001 -From: Matthaus Litteken <matthaus@puppetlabs.com> -Date: Fri, 4 May 2012 10:59:26 -0700 -Subject: [PATCH] Tighten rundir perms (rhbz #495096) - -The loose default permissions are not required for Red Hat systems -installed via rpm packages because the packages create the required -service user/group. ---- - lib/puppet/defaults.rb | 2 +- - 1 files changed, 1 insertions(+), 1 deletions(-) - -diff --git a/lib/puppet/defaults.rb b/lib/puppet/defaults.rb -index dc498e7..eef36d2 100644 ---- a/lib/puppet/defaults.rb -+++ b/lib/puppet/defaults.rb -@@ -84,7 +84,7 @@ module Puppet - :rundir => { - :default => nil, - :type => :directory, -- :mode => 01777, -+ :mode => 0755, - :desc => "Where Puppet PID files are kept." - }, - :genconfig => { --- -1.7.7.6 - diff --git a/ext/windows/service/daemon.rb b/ext/windows/service/daemon.rb index 666b11d2f..93701c2a6 100755 --- a/ext/windows/service/daemon.rb +++ b/ext/windows/service/daemon.rb @@ -55,7 +55,7 @@ class WindowsDaemon < Win32::Daemon runinterval = 1800 end - pid = Process.create(:command_line => "\"#{puppet}\" agent --onetime #{args}").process_id + pid = Process.create(:command_line => "\"#{puppet}\" agent --onetime #{args}", :creation_flags => Process::CREATE_NEW_CONSOLE).process_id log_debug("Process created: #{pid}") log_debug("Service waiting for #{runinterval} seconds") diff --git a/install.rb b/install.rb index 8cae44e17..bb450f94f 100755 --- a/install.rb +++ b/install.rb @@ -107,6 +107,7 @@ end def do_libs(libs, strip = 'lib/') libs.each do |lf| + next if File.directory? lf olf = File.join(InstallOptions.site_dir, lf.sub(/^#{strip}/, '')) op = File.dirname(olf) if $haveftools @@ -414,7 +415,7 @@ FileUtils.cd File.dirname(__FILE__) do rdoc = glob(%w{bin/* lib/**/*.rb README* }).reject { |e| e=~ /\.(bat|cmd)$/ } ri = glob(%w{bin/*.rb lib/**/*.rb}).reject { |e| e=~ /\.(bat|cmd)$/ } man = glob(%w{man/man[0-9]/*}) - libs = glob(%w{lib/**/*.rb lib/**/*.erb lib/**/*.py lib/puppet/util/command_line/*}) + libs = glob(%w{lib/**/*}) check_prereqs prepare_installation diff --git a/lib/hiera/backend/puppet_backend.rb b/lib/hiera/backend/puppet_backend.rb index 6f0a99188..a227ce240 100644 --- a/lib/hiera/backend/puppet_backend.rb +++ b/lib/hiera/backend/puppet_backend.rb @@ -48,7 +48,7 @@ class Hiera Hiera.debug("Looking up #{key} in Puppet backend") - include_class = Puppet::Parser::Functions.function(:include) + Puppet::Parser::Functions.function(:include) loaded_classes = scope.catalog.classes hierarchy(scope, order_override).each do |klass| diff --git a/lib/puppet/application.rb b/lib/puppet/application.rb index 110ababae..463c77b32 100644 --- a/lib/puppet/application.rb +++ b/lib/puppet/application.rb @@ -87,7 +87,7 @@ module Puppet # # === Setup # Applications can use the setup block to perform any initialization. -# The defaul +setup+ behaviour is to: read Puppet configuration and manage log level and destination +# The default +setup+ behaviour is to: read Puppet configuration and manage log level and destination # # === What and how to run # If the +dispatch+ block is defined it is called. This block should return the name of the registered command @@ -97,7 +97,7 @@ module Puppet # === Execution state # The class attributes/methods of Puppet::Application serve as a global place to set and query the execution # status of the application: stopping, restarting, etc. The setting of the application status does not directly -# aftect its running status; it's assumed that the various components within the application will consult these +# affect its running status; it's assumed that the various components within the application will consult these # settings appropriately and affect their own processing accordingly. Control operations (signal handlers and # the like) should set the status appropriately to indicate to the overall system that it's the process of # stopping or restarting (or just running as usual). @@ -377,13 +377,14 @@ class Application end def setup_logs - if options[:debug] or options[:verbose] + if options[:debug] || options[:verbose] Puppet::Util::Log.newdestination(:console) - if options[:debug] - Puppet::Util::Log.level = :debug - else - Puppet::Util::Log.level = :info - end + end + + if options[:debug] + Puppet::Util::Log.level = :debug + elsif options[:verbose] + Puppet::Util::Log.level = :info end Puppet::Util::Log.setup_default unless options[:setdest] @@ -402,7 +403,7 @@ class Application # Create an option parser option_parser = OptionParser.new(self.class.banner) - # He're we're building up all of the options that the application may need to handle. The main + # Here we're building up all of the options that the application may need to handle. The main # puppet settings defined in "defaults.rb" have already been parsed once (in command_line.rb) by # the time we get here; however, our app may wish to handle some of them specially, so we need to # make the parser aware of them again. We might be able to make this a bit more efficient by diff --git a/lib/puppet/application/agent.rb b/lib/puppet/application/agent.rb index b450caff6..3a42dd83e 100644 --- a/lib/puppet/application/agent.rb +++ b/lib/puppet/application/agent.rb @@ -1,12 +1,12 @@ require 'puppet/application' require 'puppet/run' +require 'puppet/daemon' +require 'puppet/util/pidlock' class Puppet::Application::Agent < Puppet::Application run_mode :agent - attr_accessor :args, :agent, :daemon, :host - def app_defaults super.merge({ :catalog_terminus => :rest, @@ -28,7 +28,6 @@ class Puppet::Application::Agent < Puppet::Application :detailed_exitcodes => false, :verbose => false, :debug => false, - :centrallogs => false, :setdest => false, :enable => false, :disable => false, @@ -42,14 +41,9 @@ class Puppet::Application::Agent < Puppet::Application options[opt] = val end - @args = {} - require 'puppet/daemon' - @daemon = Puppet::Daemon.new - @daemon.argv = ARGV.dup + @argv = ARGV.dup end - option("--centrallogging") - option("--disable [MESSAGE]") do |message| options[:disable] = true options[:disable_message] = message @@ -85,10 +79,6 @@ class Puppet::Application::Agent < Puppet::Application options[:waitforcert] = arg.to_i end - option("--port PORT","-p") do |arg| - @args[:Port] = arg - end - def help <<-'HELP' @@ -323,12 +313,25 @@ Copyright (c) 2011 Puppet Labs, LLC Licensed under the Apache 2.0 License end def run_command - return fingerprint if options[:fingerprint] - return onetime if Puppet[:onetime] - main + if options[:fingerprint] + fingerprint + else + # It'd be nice to daemonize later, but we have to daemonize before + # waiting for certificates so that we don't block + daemon = daemonize_process_when(Puppet[:daemonize]) + + wait_for_certificates + + if Puppet[:onetime] + onetime(daemon) + else + main(daemon) + end + end end def fingerprint + host = Puppet::SSL::Host.new unless cert = host.certificate || host.certificate_request $stderr.puts "Fingerprint asked but no certificate nor certificate request have yet been issued" exit(1) @@ -340,22 +343,26 @@ Copyright (c) 2011 Puppet Labs, LLC Licensed under the Apache 2.0 License puts digest.to_s end - def onetime + def onetime(daemon) + if Puppet[:listen] + Puppet.notice "Ignoring --listen on onetime run" + end + unless options[:client] - $stderr.puts "onetime is specified but there is no client" + Puppet.err "onetime is specified but there is no client" exit(43) return end - @daemon.set_signal_traps + daemon.set_signal_traps begin - exitstatus = @agent.run + exitstatus = daemon.agent.run rescue => detail Puppet.log_exception(detail) end - @daemon.stop(:exit => false) + daemon.stop(:exit => false) if not exitstatus exit(1) @@ -366,10 +373,13 @@ Copyright (c) 2011 Puppet Labs, LLC Licensed under the Apache 2.0 License end end - def main + def main(daemon) + if Puppet[:listen] + setup_listen(daemon) + end Puppet.notice "Starting Puppet client version #{Puppet.version}" - @daemon.start + daemon.start end # Enable all of the most common test options. @@ -384,6 +394,43 @@ Copyright (c) 2011 Puppet Labs, LLC Licensed under the Apache 2.0 License options[:detailed_exitcodes] = true end + def setup + setup_test if options[:test] + + setup_logs + + exit(Puppet.settings.print_configs ? 0 : 1) if Puppet.settings.print_configs? + + if options[:fqdn] + Puppet[:certname] = options[:fqdn] + end + + Puppet.settings.use :main, :agent, :ssl + + # Always ignoreimport for agent. It really shouldn't even try to import, + # but this is just a temporary band-aid. + Puppet[:ignoreimport] = true + + Puppet::Transaction::Report.indirection.terminus_class = :rest + # we want the last report to be persisted locally + Puppet::Transaction::Report.indirection.cache_class = :yaml + + if Puppet[:catalog_cache_terminus] + Puppet::Resource::Catalog.indirection.cache_class = Puppet[:catalog_cache_terminus] + end + + if options[:fingerprint] + # in fingerprint mode we just need + # access to the local files and we don't need a ca + Puppet::SSL::Host.ca_location = :none + else + Puppet::SSL::Host.ca_location = :remote + setup_agent + end + end + + private + def enable_disable_client(agent) if options[:enable] agent.enable @@ -393,7 +440,7 @@ Copyright (c) 2011 Puppet Labs, LLC Licensed under the Apache 2.0 License exit(0) end - def setup_listen + def setup_listen(daemon) Puppet.warning "Puppet --listen / kick is deprecated. See http://links.puppetlabs.com/puppet-kick-deprecation" unless FileTest.exists?(Puppet[:rest_authconfig]) Puppet.err "Will not start without authorization file #{Puppet[:rest_authconfig]}" @@ -404,87 +451,34 @@ Copyright (c) 2011 Puppet Labs, LLC Licensed under the Apache 2.0 License # No REST handlers yet. server = Puppet::Network::Server.new(Puppet[:bindaddress], Puppet[:puppetport]) - @daemon.server = server - end - - def setup_host - @host = Puppet::SSL::Host.new - waitforcert = options[:waitforcert] || (Puppet[:onetime] ? 0 : Puppet[:waitforcert]) - cert = @host.wait_for_cert(waitforcert) unless options[:fingerprint] + daemon.server = server end def setup_agent - # We need tomake the client either way, we just don't start it + # We need to make the client either way, we just don't start it # if --no-client is set. require 'puppet/agent' require 'puppet/configurer' - @agent = Puppet::Agent.new(Puppet::Configurer, (not(Puppet[:onetime]))) - - enable_disable_client(@agent) if options[:enable] or options[:disable] + agent = Puppet::Agent.new(Puppet::Configurer, (not(Puppet[:onetime]))) - @daemon.agent = agent if options[:client] + enable_disable_client(agent) if options[:enable] or options[:disable] - # It'd be nice to daemonize later, but we have to daemonize before the - # waitforcert happens. - @daemon.daemonize if Puppet[:daemonize] - - setup_host - - @objects = [] - - # This has to go after the certs are dealt with. - if Puppet[:listen] - unless Puppet[:onetime] - setup_listen - else - Puppet.notice "Ignoring --listen on onetime run" - end - end + @agent = agent if options[:client] end - def setup - setup_test if options[:test] - - setup_logs - - exit(Puppet.settings.print_configs ? 0 : 1) if Puppet.settings.print_configs? - - args[:Server] = Puppet[:server] - if options[:fqdn] - args[:FQDN] = options[:fqdn] - Puppet[:certname] = options[:fqdn] - end - - if options[:centrallogs] - logdest = args[:Server] - - logdest += ":" + args[:Port] if args.include?(:Port) - Puppet::Util::Log.newdestination(logdest) - end - - Puppet.settings.use :main, :agent, :ssl - - # Always ignoreimport for agent. It really shouldn't even try to import, - # but this is just a temporary band-aid. - Puppet[:ignoreimport] = true - - # We need to specify a ca location for all of the SSL-related i - # indirected classes to work; in fingerprint mode we just need - # access to the local files and we don't need a ca. - Puppet::SSL::Host.ca_location = options[:fingerprint] ? :none : :remote + def daemonize_process_when(should_daemonize) + daemon = Puppet::Daemon.new(Puppet::Util::Pidlock.new(Puppet[:pidfile])) + daemon.argv = @argv + daemon.agent = @agent - Puppet::Transaction::Report.indirection.terminus_class = :rest - # we want the last report to be persisted locally - Puppet::Transaction::Report.indirection.cache_class = :yaml + daemon.daemonize if should_daemonize - if Puppet[:catalog_cache_terminus] - Puppet::Resource::Catalog.indirection.cache_class = Puppet[:catalog_cache_terminus] - end + daemon + end - unless options[:fingerprint] - setup_agent - else - setup_host - end + def wait_for_certificates + host = Puppet::SSL::Host.new + waitforcert = options[:waitforcert] || (Puppet[:onetime] ? 0 : Puppet[:waitforcert]) + host.wait_for_cert(waitforcert) end end diff --git a/lib/puppet/application/apply.rb b/lib/puppet/application/apply.rb index 8fa01b179..3d7370d2c 100644 --- a/lib/puppet/application/apply.rb +++ b/lib/puppet/application/apply.rb @@ -239,8 +239,6 @@ Copyright (c) 2011 Puppet Labs, LLC Licensed under the Apache 2.0 License exit(Puppet.settings.print_configs ? 0 : 1) if Puppet.settings.print_configs? Puppet::Util::Log.newdestination(:console) unless options[:logset] - client = nil - server = nil Signal.trap(:INT) do $stderr.puts "Exiting" diff --git a/lib/puppet/application/device.rb b/lib/puppet/application/device.rb index 48a1d593b..b80691f3c 100644 --- a/lib/puppet/application/device.rb +++ b/lib/puppet/application/device.rb @@ -195,7 +195,7 @@ Licensed under the Apache 2.0 License require 'puppet/configurer' configurer = Puppet::Configurer.new - report = configurer.run(:network_device => true, :pluginsync => Puppet[:pluginsync]) + configurer.run(:network_device => true, :pluginsync => Puppet[:pluginsync]) rescue => detail Puppet.log_exception(detail) ensure @@ -210,7 +210,7 @@ Licensed under the Apache 2.0 License def setup_host @host = Puppet::SSL::Host.new waitforcert = options[:waitforcert] || (Puppet[:onetime] ? 0 : Puppet[:waitforcert]) - cert = @host.wait_for_cert(waitforcert) + @host.wait_for_cert(waitforcert) end def setup @@ -230,7 +230,7 @@ Licensed under the Apache 2.0 License # but this is just a temporary band-aid. Puppet[:ignoreimport] = true - # We need to specify a ca location for all of the SSL-related i + # We need to specify a ca location for all of the SSL-related # indirected classes to work; in fingerprint mode we just need # access to the local files and we don't need a ca. Puppet::SSL::Host.ca_location = :remote diff --git a/lib/puppet/application/kick.rb b/lib/puppet/application/kick.rb index 6720aecf0..960cdc867 100644 --- a/lib/puppet/application/kick.rb +++ b/lib/puppet/application/kick.rb @@ -157,7 +157,7 @@ with '--genconfig'. option requires LDAP support at this point. * --ping: - Do a ICMP echo against the target host. Skip hosts that don't respond + Do an ICMP echo against the target host. Skip hosts that don't respond to ping. @@ -240,7 +240,7 @@ Copyright (c) 2011 Puppet Labs, LLC Licensed under the Apache 2.0 License def run_for_host(host) if options[:ping] - out = %x{ping -c 1 #{host}} + %x{ping -c 1 #{host}} unless $CHILD_STATUS == 0 $stderr.print "Could not contact #{host}\n" exit($CHILD_STATUS) diff --git a/lib/puppet/application/master.rb b/lib/puppet/application/master.rb index 80d044c19..fafbcf3b4 100644 --- a/lib/puppet/application/master.rb +++ b/lib/puppet/application/master.rb @@ -1,4 +1,6 @@ require 'puppet/application' +require 'puppet/daemon' +require 'puppet/util/pidlock' class Puppet::Application::Master < Puppet::Application @@ -152,10 +154,8 @@ Copyright (c) 2012 Puppet Labs, LLC Licensed under the Apache 2.0 License exit(0) end - # Create this first-off, so we have ARGV - require 'puppet/daemon' - @daemon = Puppet::Daemon.new - @daemon.argv = ARGV.dup + # save ARGV to protect us from it being smashed later by something + @argv = ARGV.dup end def run_command @@ -183,7 +183,6 @@ Copyright (c) 2012 Puppet Labs, LLC Licensed under the Apache 2.0 License def main require 'etc' - # Make sure we've got a localhost ssl cert Puppet::SSL::Host.localhost @@ -200,21 +199,10 @@ Copyright (c) 2012 Puppet Labs, LLC Licensed under the Apache 2.0 License end end - unless options[:rack] - require 'puppet/network/server' - @daemon.server = Puppet::Network::Server.new(Puppet[:bindaddress], Puppet[:masterport]) - @daemon.daemonize if Puppet[:daemonize] - else - require 'puppet/network/http/rack' - @app = Puppet::Network::HTTP::Rack.new() - end - - Puppet.notice "Starting Puppet master version #{Puppet.version}" - - unless options[:rack] - @daemon.start + if options[:rack] + start_rack_master else - return @app + start_webrick_master end end @@ -282,4 +270,38 @@ Copyright (c) 2012 Puppet Labs, LLC Licensed under the Apache 2.0 License setup_ssl end + + private + + # Start a master that will be using WeBrick. + # + # This method will block until the master exits. + def start_webrick_master + require 'puppet/network/server' + daemon = Puppet::Daemon.new(Puppet::Util::Pidlock.new(Puppet[:pidfile])) + + daemon.argv = @argv + daemon.server = Puppet::Network::Server.new(Puppet[:bindaddress], Puppet[:masterport]) + daemon.daemonize if Puppet[:daemonize] + + announce_start_of_master + + daemon.start + end + + # Start a master that will be used for a Rack container. + # + # This method immediately returns the Rack handler that must be returned to + # the calling Rack container + def start_rack_master + require 'puppet/network/http/rack' + + announce_start_of_master + + return Puppet::Network::HTTP::Rack.new() + end + + def announce_start_of_master + Puppet.notice "Starting Puppet master version #{Puppet.version}" + end end diff --git a/lib/puppet/application/queue.rb b/lib/puppet/application/queue.rb index 086b7a5f0..87e269e33 100644 --- a/lib/puppet/application/queue.rb +++ b/lib/puppet/application/queue.rb @@ -1,5 +1,7 @@ require 'puppet/application' require 'puppet/util' +require 'puppet/daemon' +require 'puppet/util/pidlock' class Puppet::Application::Queue < Puppet::Application @@ -10,9 +12,7 @@ class Puppet::Application::Queue < Puppet::Application end def preinit - require 'puppet/daemon' - @daemon = Puppet::Daemon.new - @daemon.argv = ARGV.dup + @argv = ARGV.dup # Do an initial trap, so that cancels don't get a stack trace. @@ -153,6 +153,8 @@ Copyright (c) 2011 Puppet Labs, LLC Licensed under the Apache 2.0 License require 'puppet/resource/catalog' Puppet::Resource::Catalog.indirection.terminus_class = :store_configs + daemon = Puppet::Daemon.new(Puppet::Util::Pidlock.new(Puppet[:pidfile])) + daemon.argv = @argv daemon.daemonize if Puppet[:daemonize] # We want to make sure that we don't have a cache diff --git a/lib/puppet/bindings.rb b/lib/puppet/bindings.rb new file mode 100644 index 000000000..b6a8a3afa --- /dev/null +++ b/lib/puppet/bindings.rb @@ -0,0 +1,147 @@ +# This class allows registration of named bindings that are later contributed to a layer via +# a binding scheme. +# +# The intended use is for a .rb file to be placed in confdir's or module's `lib/bindings` directory structure, with a +# name corresponding to the symbolic bindings name. +# +# Here are two equivalent examples, the first using chained methods (which is compact for simple cases), and the +# second which uses a block. +# +# @example MyModule's lib/bindings/mymodule/default.rb +# Puppet::Bindings.newbindings('mymodule::default') do +# bind.integer.named('meaning of life').to(42) +# end +# +# @example Using blocks +# Puppet::Bindings.newbindings('mymodule::default') do +# bind do +# integer +# name 'meaning of life' +# to 42 +# end +# end +# +# If access is needed to the scope, this can be declared as a block parameter. +# @example MyModule's lib/bindings/mymodule/default.rb with scope +# Puppet::Bindings.newbindings('mymodule::default') do |scope| +# bind.integer.named('meaning of life').to("#{scope['::fqdn']} also think it is 42") +# end +# +# If late evaluation is wanted, this can be achieved by binding a puppet expression. +# @example binding a puppet expression +# Puppet::Bindings.newbindings('mymodule::default') do |scope| +# bind.integer.named('meaning of life').to(puppet_string("${::fqdn} also think it is 42") +# end +# +# It is allowed to define methods in the block given to `newbindings`, these can be used when +# producing bindings. (Care should naturally be taken to not override any of the already defined methods). +# @example defining method to be used while creating bindings +# Puppet::Bindings.newbindings('mymodule::default') do +# def square(x) +# x * x +# end +# bind.integer.named('meaning of life squared').to(square(42)) +# end +# +# For all details see {Puppet::Pops::Binder::BindingsFactory}, which is used behind the scenes. +# @api public +# +class Puppet::Bindings + extend Enumerable + + Environment = Puppet::Node::Environment + + # Constructs and registers a {Puppet::Pops::Binder::Bindings::NamedBindings NamedBindings} that later can be contributed + # to a bindings layer in a bindings configuration via a URI. The name is symbolic, fully qualified with module name, and at least one + # more qualifying name (where the name `default` is used in the default bindings configuration. + # + # The given block is called with a `self` bound to an instance of {Puppet::Pops::Binder::BindingsFactory::BindingsContainerBuilder} + # which most notably has a `#bind` method which it turn calls a block bound to an instance of + # {Puppet::Pops::Binder::BindingsFactory::BindingsBuilder}. + # Depending on the use-case a direct chaining method calls or nested blocks may be used. + # + # @example simple bindings + # Puppet::Bindings.newbindings('mymodule::default') do + # bind.name('meaning of life').to(42) + # bind.integer.named('port').to(8080) + # bind.integer.named('apache::port').to(8080) + # end + # + # The block form is more suitable for longer, more complex forms of bindings. + # + def self.newbindings(name, &block) + register_proc(name, block) + end + + def self.register_proc(name, block) + adapter = NamedBindingsAdapter.adapt(Environment.current) + adapter[name] = block + end + + # Registers a named_binding under its name + # @param named_bindings [Puppet::Pops::Binder::Bindings::NamedBindings] The named bindings to register. + # @api public + # + def self.register(named_bindings) + adapter = NamedBindingsAdapter.adapt(Environment.current) + adapter[named_bindings.name] = named_bindings + end + + def self.resolve(scope, name) + entry = get(name) + return entry unless entry.is_a?(Proc) + named_bindings = Puppet::Pops::Binder::BindingsFactory.safe_named_bindings(name, scope, &entry).model + adapter = NamedBindingsAdapter.adapt(Environment.current) + adapter[named_bindings.name] = named_bindings + named_bindings + end + + # Returns the named bindings with the given name, or nil if no such bindings have been registered. + # @param name [String] The fully qualified name of a binding to get + # @return [Proc, Puppet::Pops::Binder::Bindings::NamedBindings] a Proc producing named bindings, or a named bindings directly + # @api public + # + def self.get(name) + adapter = NamedBindingsAdapter.adapt(Environment.current) + adapter[name] + end + + def self.[](name) + get(name) + end + + # Supports Enumerable iteration (k,v) over the named bindings hash. + def self.each + adapter = NamedBindingsAdapter.adapt(Environment.current) + adapter.each_pair {|k,v| yield k,v } + end + + # A NamedBindingsAdapter holds a map of name to Puppet::Pops::Binder::Bindings::NamedBindings. + # It is intended to be used as an association between an Environment and named bindings. + # + class NamedBindingsAdapter < Puppet::Pops::Adaptable::Adapter + def initialize() + @named_bindings = {} + end + + def [](name) + @named_bindings[name] + end + + def has_name?(name) + @named_bindings.has_key? + end + + def []=(name, value) + unless value.is_a?(Puppet::Pops::Binder::Bindings::NamedBindings) || value.is_a?(Proc) + raise ArgumentError, "Given value must be a NamedBindings, or a Proc producing one, got: #{value.class}." + end + @named_bindings[name] = value + end + + def each_pair(&block) + @named_bindings.each_pair(&block) + end + end + +end
\ No newline at end of file diff --git a/lib/puppet/coercion.rb b/lib/puppet/coercion.rb new file mode 100644 index 000000000..f3db8f62b --- /dev/null +++ b/lib/puppet/coercion.rb @@ -0,0 +1,29 @@ +# Various methods used to coerce values into a canonical form. +# +# @api private +module Puppet::Coercion + # Try to coerce various input values into boolean true/false + # + # Only a very limited subset of values are allowed. This method does not try + # to provide a generic "truthiness" system. + # + # @param value [Boolean, Symbol, String] + # @return [Boolean] + # @raise + # @api private + def self.boolean(value) + # downcase strings + if value.respond_to? :downcase + value = value.downcase + end + + case value + when true, :true, 'true', :yes, 'yes' + true + when false, :false, 'false', :no, 'no' + false + else + fail('expected a boolean value') + end + end +end diff --git a/lib/puppet/configurer.rb b/lib/puppet/configurer.rb index 4d9cb111b..921015750 100644 --- a/lib/puppet/configurer.rb +++ b/lib/puppet/configurer.rb @@ -3,6 +3,7 @@ require 'sync' require 'timeout' require 'puppet/network/http_pool' require 'puppet/util' +require 'securerandom' class Puppet::Configurer require 'puppet/configurer/fact_handler' @@ -58,18 +59,19 @@ class Puppet::Configurer @running = false @splayed = false @environment = Puppet[:environment] + @transaction_uuid = SecureRandom.uuid end # Get the remote catalog, yo. Returns nil if no catalog can be found. - def retrieve_catalog(fact_options) - fact_options ||= {} + def retrieve_catalog(query_options) + query_options ||= {} # First try it with no cache, then with the cache. - unless (Puppet[:use_cached_catalog] and result = retrieve_catalog_from_cache(fact_options)) or result = retrieve_new_catalog(fact_options) + unless (Puppet[:use_cached_catalog] and result = retrieve_catalog_from_cache(query_options)) or result = retrieve_new_catalog(query_options) if ! Puppet[:usecacheonfailure] Puppet.warning "Not using cache on failed catalog" return nil end - result = retrieve_catalog_from_cache(fact_options) + result = retrieve_catalog_from_cache(query_options) end return nil unless result @@ -100,11 +102,11 @@ class Puppet::Configurer end end - def prepare_and_retrieve_catalog(options, fact_options) + def prepare_and_retrieve_catalog(options, query_options) # set report host name now that we have the fact options[:report].host = Puppet[:node_name_value] - unless catalog = (options.delete(:catalog) || retrieve_catalog(fact_options)) + unless catalog = (options.delete(:catalog) || retrieve_catalog(query_options)) Puppet.err "Could not retrieve catalog; skipping run" return end @@ -116,6 +118,7 @@ class Puppet::Configurer def apply_catalog(catalog, options) report = options[:report] report.configuration_version = catalog.version + report.transaction_uuid = @transaction_uuid report.environment = @environment benchmark(:notice, "Finished catalog run") do @@ -126,6 +129,10 @@ class Puppet::Configurer report end + def get_transaction_uuid + { :transaction_uuid => @transaction_uuid } + end + # The code that actually runs the catalog. # This just passes any options on to the catalog, # which accepts :tags and :ignoreschedules. @@ -137,7 +144,7 @@ class Puppet::Configurer Puppet::Util::Log.newdestination(report) begin unless Puppet[:node_name_fact].empty? - fact_options = get_facts(options) + query_options = get_facts(options) end begin @@ -146,7 +153,7 @@ class Puppet::Configurer if node.environment.to_s != @environment Puppet.warning "Local environment: \"#{@environment}\" doesn't match server specified node environment \"#{node.environment}\", switching agent to \"#{node.environment}\"." @environment = node.environment.to_s - fact_options = nil + query_options = nil end end rescue Puppet::Error, Net::HTTPError => detail @@ -154,9 +161,12 @@ class Puppet::Configurer Puppet.warning(detail) end - fact_options = get_facts(options) unless fact_options + query_options = get_facts(options) unless query_options - unless catalog = prepare_and_retrieve_catalog(options, fact_options) + # add the transaction uuid to the catalog query options hash + query_options.merge! get_transaction_uuid if query_options + + unless catalog = prepare_and_retrieve_catalog(options, query_options) return nil end @@ -171,7 +181,7 @@ class Puppet::Configurer end Puppet.warning "Local environment: \"#{@environment}\" doesn't match server specified environment \"#{catalog.environment}\", restarting agent run with environment \"#{catalog.environment}\"" @environment = catalog.environment - return nil unless catalog = prepare_and_retrieve_catalog(options, fact_options) + return nil unless catalog = prepare_and_retrieve_catalog(options, query_options) tries += 1 end @@ -200,6 +210,13 @@ class Puppet::Configurer Puppet::Transaction::Report.indirection.save(report, nil, :environment => @environment) if Puppet[:report] rescue => detail Puppet.log_exception(detail, "Could not send report: #{detail}") + if detail.message =~ /Could not intern from pson.*Puppet::Transaction::Report/ + Puppet.notice("There was an error sending the report.") + Puppet.notice("This error is possibly caused by sending the report in a format the master can not handle.") + Puppet.notice("A puppet master older than 3.2.2 can not handle pson reports.") + Puppet.notice("Set report_serialization_format=yaml on the agent to send reports to older masters.") + Puppet.notice("See http://links.puppetlabs.com/deprecate_yaml_on_network for more information.") + end end def save_last_run_summary(report) @@ -225,10 +242,10 @@ class Puppet::Configurer end end - def retrieve_catalog_from_cache(fact_options) + def retrieve_catalog_from_cache(query_options) result = nil @duration = thinmark do - result = Puppet::Resource::Catalog.indirection.find(Puppet[:node_name_value], fact_options.merge(:ignore_terminus => true, :environment => @environment)) + result = Puppet::Resource::Catalog.indirection.find(Puppet[:node_name_value], query_options.merge(:ignore_terminus => true, :environment => @environment)) end Puppet.notice "Using cached catalog" result @@ -237,10 +254,10 @@ class Puppet::Configurer return nil end - def retrieve_new_catalog(fact_options) + def retrieve_new_catalog(query_options) result = nil @duration = thinmark do - result = Puppet::Resource::Catalog.indirection.find(Puppet[:node_name_value], fact_options.merge(:ignore_cache => true, :environment => @environment)) + result = Puppet::Resource::Catalog.indirection.find(Puppet[:node_name_value], query_options.merge(:ignore_cache => true, :environment => @environment)) end result rescue SystemExit,NoMemoryError diff --git a/lib/puppet/configurer/fact_handler.rb b/lib/puppet/configurer/fact_handler.rb index f8aee5a11..bb76146cc 100644 --- a/lib/puppet/configurer/fact_handler.rb +++ b/lib/puppet/configurer/fact_handler.rb @@ -29,16 +29,9 @@ module Puppet::Configurer::FactHandler def facts_for_uploading facts = find_facts - #format = facts.class.default_format - if facts.support_format?(:b64_zlib_yaml) - format = :b64_zlib_yaml - else - format = :yaml - end - - text = facts.render(format) + text = facts.render(:pson) - {:facts_format => format, :facts => CGI.escape(text)} + {:facts_format => :pson, :facts => CGI.escape(text)} end end diff --git a/lib/puppet/daemon.rb b/lib/puppet/daemon.rb index c67bf53b1..562602418 100755 --- a/lib/puppet/daemon.rb +++ b/lib/puppet/daemon.rb @@ -1,13 +1,33 @@ -require 'puppet' -require 'puppet/util/pidlock' require 'puppet/application' require 'puppet/scheduler' -# A module that handles operations common to all daemons. This is included -# into the Server and Client base classes. +# Run periodic actions and a network server in a daemonized process. +# +# A Daemon has 3 parts: +# * config reparse +# * (optional) an agent that responds to #run +# * (optional) a server that response to #stop, #start, and #wait_for_shutdown +# +# The config reparse will occur periodically based on Settings. The server will +# be started and is expected to manage its own run loop (and so not block the +# start call). The server will, however, still be waited for by using the +# #wait_for_shutdown method. The agent is run periodically and a time interval +# based on Settings. The config reparse will update this time interval when +# needed. +# +# The Daemon is also responsible for signal handling, starting, stopping, +# running the agent on demand, and reloading the entire process. It ensures +# that only one Daemon is running by using a lockfile. +# +# @api private class Puppet::Daemon attr_accessor :agent, :server, :argv + def initialize(pidfile, scheduler = Puppet::Scheduler::Scheduler.new()) + @scheduler = scheduler + @pidfile = pidfile + end + def daemonname Puppet.run_mode.name end @@ -26,6 +46,8 @@ class Puppet::Daemon Process.setsid Dir.chdir("/") + + close_streams end # Close stdin/stdout/stderr so that we can finish our transition into 'daemon' mode. @@ -52,19 +74,6 @@ class Puppet::Daemon Puppet::Daemon.close_streams end - # Create a pidfile for our daemon, so we can be stopped and others - # don't try to start. - def create_pidfile - Puppet::Util.synchronize_on(Puppet.run_mode.name,Sync::EX) do - raise "Could not create PID file: #{pidfile}" unless Puppet::Util::Pidlock.new(pidfile).lock - end - end - - # Provide the path to our pidfile. - def pidfile - Puppet[:pidfile] - end - def reexec raise Puppet::DevError, "Cannot reexec unless ARGV arguments are set" unless argv command = $0 + " " + argv.join(" ") @@ -83,13 +92,6 @@ class Puppet::Daemon agent.run({:splay => false}) end - # Remove the pid file for our daemon. - def remove_pidfile - Puppet::Util.synchronize_on(Puppet.run_mode.name,Sync::EX) do - Puppet::Util::Pidlock.new(pidfile).unlock - end - end - def restart Puppet::Application.restart! reexec unless agent and agent.running? @@ -136,18 +138,29 @@ class Puppet::Daemon # Start the listening server, if required. server.start if server - # now that the server has started, we've waited just about as long as possible to close - # our streams and become a "real" daemon process. This is in hopes of allowing - # errors to have the console available as a fallback for logging for as long as - # possible. - close_streams if Puppet[:daemonize] - # Finally, loop forever running events - or, at least, until we exit. run_event_loop server.wait_for_shutdown if server end + private + + # Create a pidfile for our daemon, so we can be stopped and others + # don't try to start. + def create_pidfile + Puppet::Util.synchronize_on(Puppet.run_mode.name,Sync::EX) do + raise "Could not create PID file: #{@pidfile.file_path}" unless @pidfile.lock + end + end + + # Remove the pid file for our daemon. + def remove_pidfile + Puppet::Util.synchronize_on(Puppet.run_mode.name,Sync::EX) do + @pidfile.unlock + end + end + def run_event_loop agent_run = Puppet::Scheduler.create_job(Puppet[:runinterval], Puppet[:splay], Puppet[:splaylimit]) do # Splay for the daemon is handled in the scheduler @@ -167,9 +180,7 @@ class Puppet::Daemon reparse_run.disable if Puppet[:filetimeout] == 0 agent_run.disable unless agent - scheduler = Puppet::Scheduler::Scheduler.new([reparse_run, agent_run]) - - scheduler.run_loop + @scheduler.run_loop([reparse_run, agent_run]) end end diff --git a/lib/puppet/defaults.rb b/lib/puppet/defaults.rb index 154cdbe18..793e88c29 100644 --- a/lib/puppet/defaults.rb +++ b/lib/puppet/defaults.rb @@ -1,8 +1,13 @@ -# The majority of Puppet's configuration settings are set in this file. - - module Puppet + def self.default_diffargs + if (Facter.value(:kernel) == "AIX" && Facter.value(:kernelmajversion) == "5300") + "" + else + "-u" + end + end + ############################################################################################ # NOTE: For information about the available values for the ":type" property of settings, # see the docs for Settings.define_settings @@ -80,7 +85,9 @@ module Puppet :rundir => { :default => nil, :type => :directory, - :mode => 01777, + :mode => 0755, + :owner => "service", + :group => "service", :desc => "Where Puppet PID files are kept." }, :genconfig => { @@ -173,7 +180,7 @@ module Puppet "this provides the default environment for nodes we know nothing about." }, :diff_args => { - :default => "-u", + :default => default_diffargs, :desc => "Which arguments to pass to the diff command when printing differences between\n" + "files. The command to use can be chosen with the `diff` setting.", }, @@ -236,6 +243,19 @@ module Puppet :desc => "The hiera configuration file. Puppet only reads this file on startup, so you must restart the puppet master every time you edit it.", :type => :file, }, + :binder => { + :default => false, + :desc => "Turns the binding system on or off. This includes hiera-2 and data in modules. The binding system aggregates data from + modules and other locations and makes them available for lookup. The binding system is experimental and any or all of it may change.", + :type => :boolean, + }, + :binder_config => { + :default => nil, + :desc => "The binder configuration file. Puppet reads this file on each request to configure the bindings system. + If set to nil (the default), a $confdir/binder_config.yaml is optionally loaded. If it does not exists, a default configuration + is used. If the setting :binding_config is specified, it must reference a valid and existing yaml file.", + :type => :file, + }, :catalog_terminus => { :type => :terminus, :default => "compiler", @@ -316,7 +336,7 @@ module Puppet Requires that `puppet queue` be running.", :hook => proc do |value| if value - # This reconfigures the terminii for Node, Facts, and Catalog + # This reconfigures the termini for Node, Facts, and Catalog Puppet.settings[:storeconfigs] = true # But then we modify the configuration @@ -368,6 +388,11 @@ module Puppet :desc => "Freezes the 'main' class, disallowing any code to be added to it. This\n" + "essentially means that you can't have any code outside of a node, class, or definition other\n" + "than in the site manifest.", + }, + :stringify_facts => { + :default => true, + :type => :boolean, + :desc => "Flatten fact values to strings using #to_s. Means you can't have arrays or hashes as fact values.", } ) Puppet.define_settings(:module_tool, @@ -378,6 +403,10 @@ module Puppet :module_working_dir => { :default => '$vardir/puppet-module', :desc => "The directory into which module tool data is stored", + }, + :module_skeleton_dir => { + :default => '$module_working_dir/skeleton', + :desc => "The directory which the skeleton for module tool generate is stored.", } ) @@ -525,16 +554,6 @@ EOT :owner => "service", :desc => "Where each client stores the CA certificate." }, - ## JJM - The ssl_client_ca_chain setting is commented out because it is - # intended for (#3143) and is not expected to be used until CA chaining is - # supported. - # :ssl_client_ca_chain => { - # :type => :file, - # :mode => 0644, - # :owner => "service", - # :desc => "The list of CA certificate to complete the chain of trust to CA certificates \n" << - # "listed in the ssl_client_ca_auth file." - # }, :ssl_client_ca_auth => { :type => :file, :mode => 0644, @@ -544,16 +563,6 @@ EOT "listed in this file. If this setting has no value then the Puppet master's CA \n" << "certificate (localcacert) will be used." }, - ## JJM - The ssl_server_ca_chain setting is commented out because it is - # intended for (#3143) and is not expected to be used until CA chaining is - # supported. - # :ssl_server_ca_chain => { - # :type => :file, - # :mode => 0644, - # :owner => "service", - # :desc => "The list of CA certificate to complete the chain of trust to CA certificates \n" << - # "listed in the ssl_server_ca_auth file." - # }, :ssl_server_ca_auth => { :type => :file, :mode => 0644, @@ -1072,6 +1081,24 @@ EOT can be guaranteed to support this format, but it will be used for all classes that support it.", }, + :report_serialization_format => { + :default => "pson", + :type => :enum, + :values => ["pson", "yaml"], + :desc => "The serialization format to use when sending reports to the + `report_server`. Possible values are `pson` and `yaml`. This setting + affects puppet agent, but not puppet apply (which processes its own + reports). + + This should almost always be set to `pson`. It can be temporarily set to + `yaml` to let agents using this Puppet version connect to a puppet master + running Puppet 3.0.0 through 3.2.4.", + :hook => proc do |value| + if value == "yaml" + Puppet.deprecation_warning("Sending reports in 'yaml' is deprecated; use 'pson' instead.") + end + end + }, :agent_catalog_run_lockfile => { :default => "$statedir/agent_catalog_run.lock", :type => :string, # (#2888) Ensure this file is not added to the settings catalog. @@ -1220,6 +1247,26 @@ EOT turn off waiting for certificates by specifying a time of 0, in which case puppet agent will exit if it cannot get a cert. #{AS_DURATION}", + }, + :ordering => { + :type => :enum, + :values => ["manifest", "title-hash", "random"], + :default => "title-hash", + :desc => "How unrelated resources should be ordered when applying a catalog. + Allowed values are `title-hash`, `manifest`, and `random`. This + setting affects puppet agent and puppet apply, but not puppet master. + + * `title-hash` (the default) will order resources randomly, but will use + the same order across runs and across nodes. + * `manifest` will use the order in which the resources were declared in + their manifest files. + * `random` will order resources randomly and change their order with each + run. This can work like a fuzzer for shaking out undeclared dependencies. + + Regardless of this setting's value, Puppet will always obey explicit + dependencies set with the before/require/notify/subscribe metaparameters + and the `->`/`~>` chaining arrows; this setting only affects the relative + ordering of _unrelated_ resources." } ) @@ -1297,6 +1344,16 @@ EOT :smtpserver => { :default => "none", :desc => "The server through which to send email reports.", + }, + :smtpport => { + :default => 25, + :desc => "The TCP port through which to send email reports.", + }, + :smtphelo => { + :default => Facter["fqdn"].value, + :desc => "The name by which we identify ourselves in SMTP HELO for reports. + If you send to a smtpserver which does strict HELO checking (as with Postfix's + `smtpd_helo_restrictions` access controls), you may need to ensure this resolves.", } ) diff --git a/lib/puppet/error.rb b/lib/puppet/error.rb index 2e498636c..15952662e 100644 --- a/lib/puppet/error.rb +++ b/lib/puppet/error.rb @@ -1,7 +1,7 @@ module Puppet # The base class for all Puppet errors. It can wrap another exception class Error < RuntimeError - attr_reader :original + attr_accessor :original def initialize(message, original=nil) super(message) @original = original diff --git a/lib/puppet/external/base64.rb b/lib/puppet/external/base64.rb deleted file mode 100755 index 57359dc18..000000000 --- a/lib/puppet/external/base64.rb +++ /dev/null @@ -1,19 +0,0 @@ -# a stupid hack class to get rid of all of the warnings but -# still make the encode/decode methods available - -# 1.8.2 has a Base64 class, but 1.8.1 just imports the methods directly -# into Object - -require 'base64' - -unless defined?(Base64) - class Base64 - def Base64.encode64(*args) - Object.method(:encode64).call(*args) - end - - def Base64.decode64(*args) - Object.method(:decode64).call(*args) - end - end -end diff --git a/lib/puppet/external/dot.rb b/lib/puppet/external/dot.rb index 81515f157..51da1c592 100644 --- a/lib/puppet/external/dot.rb +++ b/lib/puppet/external/dot.rb @@ -39,9 +39,9 @@ module DOT 'height', # default: .5; height in inches 'label', # default: node name; any string 'layer', # default: overlay range; all, id or id:id - 'orientation', # dafault: 0.0; node rotation angle + 'orientation', # default: 0.0; node rotation angle 'peripheries', # shape-dependent number of node boundaries - 'regular', # default: false; force polygon to be regular + 'regular', # default: false; force polygon to be regular 'shape', # default: ellipse; node shape; see Section 2.1 and Appendix E 'shapefile', # external EPSF or SVG custom shape file 'sides', # default: 4; number of sides for shape=polygon diff --git a/lib/puppet/external/nagios/base.rb b/lib/puppet/external/nagios/base.rb index e4a6feaea..ce75f66ea 100755 --- a/lib/puppet/external/nagios/base.rb +++ b/lib/puppet/external/nagios/base.rb @@ -215,7 +215,7 @@ class Nagios::Base # Now access the parameters directly, to make it at least less # likely we'll end up in an infinite recursion. if mname.to_s =~ /=$/ - @parameters[pname] = *args + @parameters[pname] = args.first else return @parameters[mname] end @@ -271,7 +271,6 @@ class Nagios::Base # okay, this sucks # how do i get my list of ocs? def to_ldif - base = self.class.ldapbase str = self.dn + "\n" ocs = Array.new if self.class.ocs @@ -410,7 +409,6 @@ class Nagios::Base end newtype :servicedependency do - auxiliary = true setparameters :dependent_host_name, :dependent_hostgroup_name, :dependent_service_description, :host_name, :hostgroup_name, :service_description, :inherits_parent, :execution_failure_criteria, @@ -433,7 +431,6 @@ class Nagios::Base end newtype :hostdependency do - auxiliary = true setparameters :dependent_host_name, :dependent_hostgroup_name, :host_name, :hostgroup_name, :inherits_parent, :execution_failure_criteria, :notification_failure_criteria, :dependency_period, @@ -454,7 +451,6 @@ class Nagios::Base end newtype :hostextinfo do - auxiliary = true setparameters :host_name, :notes, :notes_url, :icon_image, :icon_image_alt, :vrml_image, :statusmap_image, "2d_coords".intern, "3d_coords".intern, :register, :use @@ -463,7 +459,6 @@ class Nagios::Base end newtype :serviceextinfo do - auxiliary = true setparameters :host_name, :service_description, :notes, :notes_url, :action_url, :icon_image, :icon_image_alt, diff --git a/lib/puppet/external/pson/common.rb b/lib/puppet/external/pson/common.rb index 289913ac1..2ea2b0e49 100644 --- a/lib/puppet/external/pson/common.rb +++ b/lib/puppet/external/pson/common.rb @@ -127,7 +127,7 @@ module PSON # defiance of RFC 4627 to be parsed by the Parser. This option defaults # to false. # * *create_additions*: If set to false, the Parser doesn't create - # additions even if a matchin class and create_id was found. This option + # additions even if a matching class and create_id was found. This option # defaults to true. def parse(source, opts = {}) PSON.parser.new(source, opts).parse @@ -146,7 +146,7 @@ module PSON # defiance of RFC 4627 to be parsed by the Parser. This option defaults # to true. # * *create_additions*: If set to false, the Parser doesn't create - # additions even if a matchin class and create_id was found. This option + # additions even if a matching class and create_id was found. This option # defaults to true. def parse!(source, opts = {}) opts = { diff --git a/lib/puppet/external/pson/pure/generator.rb b/lib/puppet/external/pson/pure/generator.rb index 95372941d..e0eabffc3 100644 --- a/lib/puppet/external/pson/pure/generator.rb +++ b/lib/puppet/external/pson/pure/generator.rb @@ -58,7 +58,7 @@ module PSON module Pure module Generator # This class is used to create State instances, that are use to hold data - # while generating a PSON text from a a Ruby data structure. + # while generating a PSON text from a Ruby data structure. class State # Creates a State object from _opts_, which ought to be Hash to create # a new State instance configured by _opts_, something else to create @@ -366,7 +366,7 @@ module PSON # This method creates a raw object hash, that can be nested into # other data structures and will be unparsed as a raw string. This # method should be used, if you want to convert raw strings to PSON - # instead of UTF-8 strings, e. g. binary data. + # instead of UTF-8 strings, e.g. binary data. def to_pson_raw_object { PSON.create_id => self.class.name, diff --git a/lib/puppet/external/pson/pure/parser.rb b/lib/puppet/external/pson/pure/parser.rb index e8fbc98f4..f14e6161c 100644 --- a/lib/puppet/external/pson/pure/parser.rb +++ b/lib/puppet/external/pson/pure/parser.rb @@ -61,7 +61,7 @@ module PSON # defiance of RFC 4627 to be parsed by the Parser. This option defaults # to false. # * *create_additions*: If set to false, the Parser doesn't create - # additions even if a matchin class and create_id was found. This option + # additions even if a matching class and create_id was found. This option # defaults to true. # * *object_class*: Defaults to Hash # * *array_class*: Defaults to Array diff --git a/lib/puppet/face/ca.rb b/lib/puppet/face/ca.rb index 2cdf9186a..55475de87 100644 --- a/lib/puppet/face/ca.rb +++ b/lib/puppet/face/ca.rb @@ -204,7 +204,7 @@ Puppet::Face.define(:ca, '0.1.0') do when_invoked do |host, options| raise "Not a CA" unless Puppet::SSL::CertificateAuthority.ca? - unless ca = Puppet::SSL::CertificateAuthority.instance + unless Puppet::SSL::CertificateAuthority.instance raise "Unable to fetch the CA" end Puppet::SSL::Host.ca_location = :only diff --git a/lib/puppet/face/config.rb b/lib/puppet/face/config.rb index e1d3fa26a..b30c5baf8 100644 --- a/lib/puppet/face/config.rb +++ b/lib/puppet/face/config.rb @@ -36,7 +36,7 @@ Puppet::Face.define(:config, '0.0.1') do EOT when_invoked do |*args| - options = args.pop + args.pop args = [ "all" ] if args.empty? diff --git a/lib/puppet/face/help.rb b/lib/puppet/face/help.rb index ef445f862..44380ab30 100644 --- a/lib/puppet/face/help.rb +++ b/lib/puppet/face/help.rb @@ -129,7 +129,7 @@ Detail: "#{detail.message}" # Return a list of all applications (both legacy and Face applications), along with a summary # of their functionality. - # @returns [Array] An Array of Arrays. The outer array contains one entry per application; each + # @return [Array] An Array of Arrays. The outer array contains one entry per application; each # element in the outer array is a pair whose first element is a String containing the application # name, and whose second element is a String containing the summary for that application. def all_application_summaries() @@ -140,7 +140,7 @@ Detail: "#{detail.message}" begin face = Puppet::Face[appname, :current] result << [appname, face.summary] - rescue Puppet::Error => detail + rescue Puppet::Error result << [ "! #{appname}", "! Subcommand unavailable due to error. Check error logs." ] end else diff --git a/lib/puppet/face/module/list.rb b/lib/puppet/face/module/list.rb index d95fa7706..83523c78b 100644 --- a/lib/puppet/face/module/list.rb +++ b/lib/puppet/face/module/list.rb @@ -237,8 +237,8 @@ Puppet::Face.define(:module, '1.0.0') do # { :text => "puppetlabs-mysql (v1.0.0)" } # # The value of a module's :text is affected by three (3) factors: the format - # of the tree, it's dependency status, and the location in the modulepath - # relative to it's parent. + # of the tree, its dependency status, and the location in the modulepath + # relative to its parent. # # Returns a Hash # diff --git a/lib/puppet/feature/rails.rb b/lib/puppet/feature/rails.rb index 50c68cff4..9e2ce4fe6 100644 --- a/lib/puppet/feature/rails.rb +++ b/lib/puppet/feature/rails.rb @@ -23,7 +23,7 @@ Puppet.features.add(:rails) do require 'active_record' require 'active_record/version' - rescue LoadError => detail + rescue LoadError if FileTest.exists?("/usr/share/rails") count = 0 Dir.entries("/usr/share/rails").each do |dir| diff --git a/lib/puppet/file_bucket/dipper.rb b/lib/puppet/file_bucket/dipper.rb index 2433a0ea0..c675f09a4 100644 --- a/lib/puppet/file_bucket/dipper.rb +++ b/lib/puppet/file_bucket/dipper.rb @@ -77,7 +77,6 @@ class Puppet::FileBucket::Dipper if restore if newcontents = getfile(sum) - tmp = "" newsum = Digest::MD5.hexdigest(newcontents) changed = nil if FileTest.exists?(file) and ! FileTest.writable?(file) diff --git a/lib/puppet/file_bucket/file.rb b/lib/puppet/file_bucket/file.rb index be38a7b46..cd9f5ab5a 100644 --- a/lib/puppet/file_bucket/file.rb +++ b/lib/puppet/file_bucket/file.rb @@ -13,6 +13,15 @@ class Puppet::FileBucket::File attr :contents attr :bucket_path + def self.supported_formats + # This should really be :raw, like is done for Puppet::FileServing::Content + # but this class hasn't historically supported `from_raw`, so switching + # would break compatibility between newer 3.x agents talking to older 3.x + # masters. However, to/from_s has been supported and achieves the desired + # result without breaking compatibility. + [:s] + end + def initialize(contents, options = {}) raise ArgumentError.new("contents must be a String, got a #{contents.class}") unless contents.is_a?(String) @contents = contents @@ -45,11 +54,10 @@ class Puppet::FileBucket::File self.new(contents) end - def to_pson - { "contents" => contents }.to_pson - end - + # This method is deprecated, but cannot be removed for awhile, otherwise + # older agents sending pson couldn't backup to filebuckets on newer masters def self.from_pson(pson) + Puppet.deprecation_warning("Deserializing Puppet::FileBucket::File objects from pson is deprecated. Upgrade to a newer version.") self.new(pson["contents"]) end end diff --git a/lib/puppet/file_serving/base.rb b/lib/puppet/file_serving/base.rb index 2403f0a9f..93b5abee2 100644 --- a/lib/puppet/file_serving/base.rb +++ b/lib/puppet/file_serving/base.rb @@ -15,7 +15,7 @@ class Puppet::FileServing::Base def exist? stat return true - rescue => detail + rescue return false end diff --git a/lib/puppet/file_serving/configuration/parser.rb b/lib/puppet/file_serving/configuration/parser.rb index 91ad6ab36..3bd9c9ebd 100644 --- a/lib/puppet/file_serving/configuration/parser.rb +++ b/lib/puppet/file_serving/configuration/parser.rb @@ -1,19 +1,19 @@ require 'puppet/file_serving/configuration' -require 'puppet/util/loadedfile' +require 'puppet/util/watched_file' -class Puppet::FileServing::Configuration::Parser < Puppet::Util::LoadedFile +class Puppet::FileServing::Configuration::Parser Mount = Puppet::FileServing::Mount MODULES = 'modules' # Parse our configuration file. def parse - raise("File server configuration #{self.file} does not exist") unless FileTest.exists?(self.file) - raise("Cannot read file server configuration #{self.file}") unless FileTest.readable?(self.file) + raise("File server configuration #{@file} does not exist") unless FileTest.exists?(@file) + raise("Cannot read file server configuration #{@file}") unless FileTest.readable?(@file) @mounts = {} @count = 0 - File.open(self.file) { |f| + File.open(@file) { |f| mount = nil f.each_line { |line| # Have the count increment at the top, in case we throw exceptions. @@ -37,10 +37,10 @@ class Puppet::FileServing::Configuration::Parser < Puppet::Util::LoadedFile when "deny" deny(mount, value) else - raise ArgumentError.new("Invalid argument '#{var}'", @count, file) + raise ArgumentError.new("Invalid argument '#{var}'", @count, @file) end else - raise ArgumentError.new("Invalid line '#{line.chomp}'", @count, file) + raise ArgumentError.new("Invalid line '#{line.chomp}'", @count, @file) end } } @@ -50,37 +50,43 @@ class Puppet::FileServing::Configuration::Parser < Puppet::Util::LoadedFile @mounts end + def initialize(filename) + @file = Puppet::Util::WatchedFile.new(filename) + end + + def changed? + @file.changed? + end + private # Allow a given pattern access to a mount. def allow(mount, value) - # LAK:NOTE See http://snurl.com/21zf8 [groups_google_com] - x = value.split(/\s*,\s*/).each { |val| + value.split(/\s*,\s*/).each { |val| begin mount.info "allowing #{val} access" mount.allow(val) rescue Puppet::AuthStoreError => detail - raise ArgumentError.new(detail.to_s, @count, file) + raise ArgumentError.new(detail.to_s, @count, @file) end } end # Deny a given pattern access to a mount. def deny(mount, value) - # LAK:NOTE See http://snurl.com/21zf8 [groups_google_com] - x = value.split(/\s*,\s*/).each { |val| + value.split(/\s*,\s*/).each { |val| begin mount.info "denying #{val} access" mount.deny(val) rescue Puppet::AuthStoreError => detail - raise ArgumentError.new(detail.to_s, @count, file) + raise ArgumentError.new(detail.to_s, @count, @file) end } end # Create a new mount. def newmount(name) - raise ArgumentError, "#{@mounts[name]} is already mounted at #{name}", @count, file if @mounts.include?(name) + raise ArgumentError, "#{@mounts[name]} is already mounted at #{name}", @count, @file if @mounts.include?(name) case name when "modules" mount = Mount::Modules.new(name) diff --git a/lib/puppet/forge.rb b/lib/puppet/forge.rb index 94281d008..3c6626328 100644 --- a/lib/puppet/forge.rb +++ b/lib/puppet/forge.rb @@ -89,38 +89,6 @@ class Puppet::Forge end end - def get_release_packages_from_repository(install_list) - install_list.map do |release| - modname, version, file = release - cache_path = nil - if file - begin - cache_path = repository.retrieve(file) - rescue OpenURI::HTTPError => e - raise HttpResponseError.new(:uri => uri.to_s, :input => modname, :message => e.message) - end - else - raise RuntimeError, "Malformed response from module repository." - end - cache_path - end - end - - # Locate a module release package on the local filesystem and move it - # into the `Puppet.settings[:module_working_dir]`. Do not unpack it, just - # return the location of the package on disk. - def get_release_package_from_filesystem(filename) - if File.exist?(File.expand_path(filename)) - repository = Repository.new('file:///') - uri = URI.parse("file://#{URI.escape(File.expand_path(filename))}") - cache_path = repository.retrieve(uri) - else - raise ArgumentError, "File does not exists: #{filename}" - end - - cache_path - end - def retrieve(release) repository.retrieve(release) end diff --git a/lib/puppet/forge/cache.rb b/lib/puppet/forge/cache.rb index c9aeb55a6..0a315cc2a 100644 --- a/lib/puppet/forge/cache.rb +++ b/lib/puppet/forge/cache.rb @@ -6,7 +6,7 @@ class Puppet::Forge # Provides methods for reading files from local cache, filesystem or network. class Cache - # Instantiate new cahe for the +repositry+ instance. + # Instantiate new cache for the +repository+ instance. def initialize(repository, options = {}) @repository = repository @options = options diff --git a/lib/puppet/forge/errors.rb b/lib/puppet/forge/errors.rb index 2c7f40f13..3bd615ea3 100644 --- a/lib/puppet/forge/errors.rb +++ b/lib/puppet/forge/errors.rb @@ -30,7 +30,7 @@ module Puppet::Forge::Errors # # @return [String] the multiline version of the error message def multiline - message = <<-EOS.chomp + <<-EOS.chomp Could not connect via HTTPS to #{@uri} Unable to verify the SSL certificate The certificate may not be signed by a valid CA @@ -57,7 +57,7 @@ Could not connect via HTTPS to #{@uri} # # @return [String] the multiline version of the error message def multiline - message = <<-EOS.chomp + <<-EOS.chomp Could not connect to #{@uri} There was a network communications problem The error we caught said '#{@detail}' @@ -78,7 +78,7 @@ Could not connect to #{@uri} @input = options[:input] @message = options[:message] response = options[:response] - @response = "#{response.code} #{response.message}" + @response = "#{response.code} #{response.message.strip}" message = "Could not execute operation for '#{@input}'. Detail: " message << @message << " / " if @message diff --git a/lib/puppet/forge/repository.rb b/lib/puppet/forge/repository.rb index 7badc4c28..c0c128bcc 100644 --- a/lib/puppet/forge/repository.rb +++ b/lib/puppet/forge/repository.rb @@ -2,6 +2,7 @@ require 'net/https' require 'zlib' require 'digest/sha1' require 'uri' +require 'puppet/util/http_proxy' require 'puppet/forge/errors' class Puppet::Forge @@ -37,45 +38,9 @@ class Puppet::Forge @consumer_version = consumer_version end - # Read HTTP proxy configurationm from Puppet's config file, or the - # http_proxy environment variable. - def http_proxy_env - proxy_env = ENV["http_proxy"] || ENV["HTTP_PROXY"] || nil - begin - return URI.parse(proxy_env) if proxy_env - rescue URI::InvalidURIError - return nil - end - return nil - end - - def http_proxy_host - env = http_proxy_env - - if env and env.host then - return env.host - end - - if Puppet.settings[:http_proxy_host] == 'none' - return nil - end - - return Puppet.settings[:http_proxy_host] - end - - def http_proxy_port - env = http_proxy_env - - if env and env.port then - return env.port - end - - return Puppet.settings[:http_proxy_port] - end - # Return a Net::HTTPResponse read for this +request_path+. def make_http_request(request_path) - request = Net::HTTP::Get.new(URI.escape(request_path), { "User-Agent" => user_agent }) + request = Net::HTTP::Get.new(URI.escape(@uri.path + request_path), { "User-Agent" => user_agent }) if ! @uri.user.nil? && ! @uri.password.nil? request.basic_auth(@uri.user, @uri.password) end @@ -115,7 +80,7 @@ class Puppet::Forge # # @return [Net::HTTP::Proxy] object constructed from repo settings def get_http_object - proxy_class = Net::HTTP::Proxy(http_proxy_host, http_proxy_port) + proxy_class = Net::HTTP::Proxy(Puppet::Util::HttpProxy.http_proxy_host, Puppet::Util::HttpProxy.http_proxy_port) proxy = proxy_class.new(@uri.host, @uri.port) if @uri.scheme == 'https' @@ -133,7 +98,9 @@ class Puppet::Forge # Return the local file name containing the data downloaded from the # repository at +release+ (e.g. "myuser-mymodule"). def retrieve(release) - return cache.retrieve(@uri + release) + uri = @uri.dup + uri.path = uri.path.chomp('/') + release + return cache.retrieve(uri) end # Return the URI string for this repository. @@ -156,9 +123,7 @@ class Puppet::Forge private :user_agent def ruby_version - # the patchlevel is not available in ruby 1.8.5 - patch = defined?(RUBY_PATCHLEVEL) ? "-p#{RUBY_PATCHLEVEL}" : "" - "Ruby/#{RUBY_VERSION}#{patch} (#{RUBY_RELEASE_DATE}; #{RUBY_PLATFORM})" + "Ruby/#{RUBY_VERSION}-p#{RUBY_PATCHLEVEL} (#{RUBY_RELEASE_DATE}; #{RUBY_PLATFORM})" end private :ruby_version end diff --git a/lib/puppet/graph.rb b/lib/puppet/graph.rb new file mode 100644 index 000000000..a0aed09d5 --- /dev/null +++ b/lib/puppet/graph.rb @@ -0,0 +1,11 @@ +module Puppet::Graph + require 'puppet/graph/prioritizer' + require 'puppet/graph/sequential_prioritizer' + require 'puppet/graph/title_hash_prioritizer' + require 'puppet/graph/random_prioritizer' + + require 'puppet/graph/simple_graph' + require 'puppet/graph/rb_tree_map' + require 'puppet/graph/key' + require 'puppet/graph/relationship_graph' +end diff --git a/lib/puppet/graph/key.rb b/lib/puppet/graph/key.rb new file mode 100644 index 000000000..0a06069cc --- /dev/null +++ b/lib/puppet/graph/key.rb @@ -0,0 +1,26 @@ +# Sequential, nestable keys for tracking order of insertion in "the graph" +# @api private +class Puppet::Graph::Key + include Comparable + + attr_reader :value + protected :value + + def initialize(value = [0]) + @value = value + end + + def next + next_values = @value.clone + next_values[-1] += 1 + Puppet::Graph::Key.new(next_values) + end + + def down + Puppet::Graph::Key.new(@value + [0]) + end + + def <=>(other) + @value <=> other.value + end +end diff --git a/lib/puppet/graph/prioritizer.rb b/lib/puppet/graph/prioritizer.rb new file mode 100644 index 000000000..f734a30dd --- /dev/null +++ b/lib/puppet/graph/prioritizer.rb @@ -0,0 +1,29 @@ +# Base, template method, class for Prioritizers. This provides the basic +# tracking facilities used. +# +# @api private +class Puppet::Graph::Prioritizer + def initialize + @priority = {} + end + + def forget(key) + @priority.delete(key) + end + + def record_priority_for(key, priority) + @priority[key] = priority + end + + def generate_priority_for(key) + raise NotImplementedError + end + + def generate_priority_contained_in(container, key) + raise NotImplementedError + end + + def priority_of(key) + @priority[key] + end +end diff --git a/lib/puppet/graph/random_prioritizer.rb b/lib/puppet/graph/random_prioritizer.rb new file mode 100644 index 000000000..c1af21dc7 --- /dev/null +++ b/lib/puppet/graph/random_prioritizer.rb @@ -0,0 +1,16 @@ +# Assign a random priority to items. +# +# @api private +class Puppet::Graph::RandomPrioritizer < Puppet::Graph::Prioritizer + def generate_priority_for(key) + if priority_of(key).nil? + record_priority_for(key, SecureRandom.uuid) + else + priority_of(key) + end + end + + def generate_priority_contained_in(container, key) + generate_priority_for(key) + end +end diff --git a/lib/puppet/rb_tree_map.rb b/lib/puppet/graph/rb_tree_map.rb index 7ec6349c9..bf5fe707f 100644 --- a/lib/puppet/rb_tree_map.rb +++ b/lib/puppet/graph/rb_tree_map.rb @@ -30,7 +30,7 @@ # # Most methods have O(log n) complexity. -class Puppet::RbTreeMap +class Puppet::Graph::RbTreeMap include Enumerable attr_reader :size @@ -230,7 +230,7 @@ class Puppet::RbTreeMap def rotate_left r = @right - r_key, r_value, r_color = r.key, r.value, r.color + r_key, r_value = r.key, r.value b = r.left r.left = @left @left = r @@ -243,7 +243,7 @@ class Puppet::RbTreeMap def rotate_right l = @left - l_key, l_value, l_color = l.key, l.value, l.color + l_key, l_value = l.key, l.value b = l.right l.right = @right @right = l diff --git a/lib/puppet/graph/relationship_graph.rb b/lib/puppet/graph/relationship_graph.rb new file mode 100644 index 000000000..54a38ad54 --- /dev/null +++ b/lib/puppet/graph/relationship_graph.rb @@ -0,0 +1,246 @@ +# The relationship graph is the final form of a puppet catalog in +# which all dependency edges are explicitly in the graph. This form of the +# catalog is used to traverse the graph in the order in which resources are +# managed. +# +# @api private +class Puppet::Graph::RelationshipGraph < Puppet::Graph::SimpleGraph + attr_reader :blockers + + def initialize(prioritizer) + super() + + @prioritizer = prioritizer + + @ready = Puppet::Graph::RbTreeMap.new + @generated = {} + @done = {} + @blockers = {} + @providerless_types = [] + end + + def populate_from(catalog) + add_all_resources_as_vertices(catalog) + build_manual_dependencies + build_autorequire_dependencies(catalog) + + write_graph(:relationships) if catalog.host_config? + + replace_containers_with_anchors(catalog) + + write_graph(:expanded_relationships) if catalog.host_config? + end + + def add_vertex(vertex, priority = nil) + super(vertex) + + if priority + @prioritizer.record_priority_for(vertex, priority) + else + @prioritizer.generate_priority_for(vertex) + end + end + + def add_relationship(f, t, label=nil) + super(f, t, label) + @ready.delete(@prioritizer.priority_of(t)) + end + + def remove_vertex!(vertex) + super + @prioritizer.forget(vertex) + end + + def resource_priority(resource) + @prioritizer.priority_of(resource) + end + + # Enqueue the initial set of resources, those with no dependencies. + def enqueue_roots + vertices.each do |v| + @blockers[v] = direct_dependencies_of(v).length + enqueue(v) if @blockers[v] == 0 + end + end + + # Decrement the blocker count for the resource by 1. If the number of + # blockers is unknown, count them and THEN decrement by 1. + def unblock(resource) + @blockers[resource] ||= direct_dependencies_of(resource).select { |r2| !@done[r2] }.length + if @blockers[resource] > 0 + @blockers[resource] -= 1 + else + resource.warning "appears to have a negative number of dependencies" + end + @blockers[resource] <= 0 + end + + def clear_blockers + @blockers.clear + end + + def enqueue(*resources) + resources.each do |resource| + @ready[@prioritizer.priority_of(resource)] = resource + end + end + + def finish(resource) + direct_dependents_of(resource).each do |v| + enqueue(v) if unblock(v) + end + @done[resource] = true + end + + def next_resource + @ready.delete_min + end + + def traverse(options = {}, &block) + continue_while = options[:while] || lambda { true } + pre_process = options[:pre_process] || lambda { |resource| } + overly_deferred_resource_handler = options[:overly_deferred_resource_handler] || lambda { |resource| } + canceled_resource_handler = options[:canceled_resource_handler] || lambda { |resource| } + teardown = options[:teardown] || lambda {} + + report_cycles_in_graph + + enqueue_roots + + deferred_resources = [] + + while continue_while.call() && (resource = next_resource) + if resource.suitable? + made_progress = true + + pre_process.call(resource) + + yield resource + + finish(resource) + else + deferred_resources << resource + end + + if @ready.empty? and deferred_resources.any? + if made_progress + enqueue(*deferred_resources) + else + deferred_resources.each do |resource| + overly_deferred_resource_handler.call(resource) + finish(resource) + end + end + + made_progress = false + deferred_resources = [] + end + end + + if !continue_while.call() + while (resource = next_resource) + canceled_resource_handler.call(resource) + finish(resource) + end + end + + teardown.call() + end + + private + + def add_all_resources_as_vertices(catalog) + catalog.resources.each do |vertex| + add_vertex(vertex) + end + end + + def build_manual_dependencies + vertices.each do |vertex| + vertex.builddepends.each do |edge| + add_edge(edge) + end + end + end + + def build_autorequire_dependencies(catalog) + vertices.each do |vertex| + vertex.autorequire(catalog).each do |edge| + # don't let automatic relationships conflict with manual ones. + next if edge?(edge.source, edge.target) + + if edge?(edge.target, edge.source) + vertex.debug "Skipping automatic relationship with #{edge.source}" + else + vertex.debug "Autorequiring #{edge.source}" + add_edge(edge) + end + end + end + end + + # Impose our container information on another graph by using it + # to replace any container vertices X with a pair of verticies + # { admissible_X and completed_X } such that that + # + # 0) completed_X depends on admissible_X + # 1) contents of X each depend on admissible_X + # 2) completed_X depends on each on the contents of X + # 3) everything which depended on X depens on completed_X + # 4) admissible_X depends on everything X depended on + # 5) the containers and their edges must be removed + # + # Note that this requires attention to the possible case of containers + # which contain or depend on other containers, but has the advantage + # that the number of new edges created scales linearly with the number + # of contained verticies regardless of how containers are related; + # alternatives such as replacing container-edges with content-edges + # scale as the product of the number of external dependences, which is + # to say geometrically in the case of nested / chained containers. + # + Default_label = { :callback => :refresh, :event => :ALL_EVENTS } + def replace_containers_with_anchors(catalog) + stage_class = Puppet::Type.type(:stage) + whit_class = Puppet::Type.type(:whit) + component_class = Puppet::Type.type(:component) + containers = catalog.resources.find_all { |v| (v.is_a?(component_class) or v.is_a?(stage_class)) and vertex?(v) } + # + # These two hashes comprise the aforementioned attention to the possible + # case of containers that contain / depend on other containers; they map + # containers to their sentinels but pass other verticies through. Thus we + # can "do the right thing" for references to other verticies that may or + # may not be containers. + # + admissible = Hash.new { |h,k| k } + completed = Hash.new { |h,k| k } + containers.each { |x| + admissible[x] = whit_class.new(:name => "admissible_#{x.ref}", :catalog => catalog) + completed[x] = whit_class.new(:name => "completed_#{x.ref}", :catalog => catalog) + priority = @prioritizer.priority_of(x) + add_vertex(admissible[x], priority) + add_vertex(completed[x], priority) + } + # + # Implement the six requirements listed above + # + containers.each { |x| + contents = catalog.adjacent(x, :direction => :out) + add_edge(admissible[x],completed[x]) if contents.empty? # (0) + contents.each { |v| + add_edge(admissible[x],admissible[v],Default_label) # (1) + add_edge(completed[v], completed[x], Default_label) # (2) + } + # (3) & (5) + adjacent(x,:direction => :in,:type => :edges).each { |e| + add_edge(completed[e.source],admissible[x],e.label) + remove_edge! e + } + # (4) & (5) + adjacent(x,:direction => :out,:type => :edges).each { |e| + add_edge(completed[x],admissible[e.target],e.label) + remove_edge! e + } + } + containers.each { |x| remove_vertex! x } # (5) + end +end diff --git a/lib/puppet/graph/sequential_prioritizer.rb b/lib/puppet/graph/sequential_prioritizer.rb new file mode 100644 index 000000000..485e7ecef --- /dev/null +++ b/lib/puppet/graph/sequential_prioritizer.rb @@ -0,0 +1,31 @@ +# This implements a priority in which keys are given values that will keep them +# in the same priority in which they priorities are requested. Nested +# structures (those in which a key is contained within another key) are +# preserved in such a way that child keys are after the parent and before the +# key after the parent. +# +# @api private +class Puppet::Graph::SequentialPrioritizer < Puppet::Graph::Prioritizer + def initialize + super + @container = {} + @count = Puppet::Graph::Key.new + end + + def generate_priority_for(key) + if priority_of(key).nil? + @count = @count.next + record_priority_for(key, @count) + else + priority_of(key) + end + end + + def generate_priority_contained_in(container, key) + @container[container] ||= priority_of(container).down + priority = @container[container].next + record_priority_for(key, priority) + @container[container] = priority + priority + end +end diff --git a/lib/puppet/simple_graph.rb b/lib/puppet/graph/simple_graph.rb index 2556cbfce..6c68ec617 100644 --- a/lib/puppet/simple_graph.rb +++ b/lib/puppet/graph/simple_graph.rb @@ -3,7 +3,7 @@ require 'puppet/relationship' require 'set' # A hopefully-faster graph class to replace the use of GRATR. -class Puppet::SimpleGraph +class Puppet::Graph::SimpleGraph # # All public methods of this class must maintain (assume ^ ensure) the following invariants, where "=~=" means # equiv. up to order: @@ -178,7 +178,11 @@ class Puppet::SimpleGraph # Given we are in a failure state here, any extra cost is more or less # irrelevant compared to the cost of a fix - which is on a human # time-scale. - state[:scc].select { |c| c.length > 1 }.map {|x| x.sort }.sort + state[:scc].select do |component| + multi_vertex_component?(component) || single_vertex_referring_to_self?(component) + end.map do |component| + component.sort + end.sort end # Perform a BFS on the sub graph representing the cycle, with a view to @@ -516,7 +520,7 @@ class Puppet::SimpleGraph (adjacencies[direction][other_vertex] ||= Set.new).add(edge) end end - result[vertex] = Puppet::SimpleGraph::VertexWrapper.new(vertex, adjacencies) + result[vertex] = Puppet::Graph::SimpleGraph::VertexWrapper.new(vertex, adjacencies) end result end @@ -544,4 +548,19 @@ class Puppet::SimpleGraph instance_variable_set("@#{varname}", value) end end + + def multi_vertex_component?(component) + component.length > 1 + end + private :multi_vertex_component? + + def single_vertex_referring_to_self?(component) + if component.length == 1 + vertex = component[0] + adjacent(vertex).include?(vertex) + else + false + end + end + private :single_vertex_referring_to_self? end diff --git a/lib/puppet/graph/title_hash_prioritizer.rb b/lib/puppet/graph/title_hash_prioritizer.rb new file mode 100644 index 000000000..d980b7aaf --- /dev/null +++ b/lib/puppet/graph/title_hash_prioritizer.rb @@ -0,0 +1,16 @@ +# Prioritize keys, which must be Puppet::Resources, based on a static hash of +# the key's ref. This prioritizer does not take containment into account. +# +# @api private +require 'digest/sha1' + +class Puppet::Graph::TitleHashPrioritizer < Puppet::Graph::Prioritizer + def generate_priority_for(resource) + record_priority_for(resource, + Digest::SHA1.hexdigest("NaCl, MgSO4 (salts) and then #{resource.ref}")) + end + + def generate_priority_contained_in(container, resource) + generate_priority_for(resource) + end +end diff --git a/lib/puppet/indirector.rb b/lib/puppet/indirector.rb index b20550a43..65f852ba8 100644 --- a/lib/puppet/indirector.rb +++ b/lib/puppet/indirector.rb @@ -10,7 +10,7 @@ module Puppet::Indirector require 'puppet/indirector/indirection' require 'puppet/indirector/terminus' require 'puppet/indirector/envelope' - require 'puppet/network/format_handler' + require 'puppet/network/format_support' def self.configure_routes(application_routes) application_routes.each do |indirection_name, termini| @@ -38,7 +38,7 @@ module Puppet::Indirector # populate this class with the various new methods extend ClassMethods include Puppet::Indirector::Envelope - extend Puppet::Network::FormatHandler + include Puppet::Network::FormatSupport # record the indirected class name for documentation purposes options[:indirected_class] = name diff --git a/lib/puppet/indirector/catalog/compiler.rb b/lib/puppet/indirector/catalog/compiler.rb index 95d96d642..9b1cc0305 100644 --- a/lib/puppet/indirector/catalog/compiler.rb +++ b/lib/puppet/indirector/catalog/compiler.rb @@ -23,7 +23,8 @@ class Puppet::Resource::Catalog::Compiler < Puppet::Indirector::Code if text_facts.is_a?(Puppet::Node::Facts) facts = text_facts else - facts = Puppet::Node::Facts.convert_from(format, text_facts) + # We unescape here because the corrosponding code in Puppet::Configurer::FactHandler escapes + facts = Puppet::Node::Facts.convert_from(format, CGI.unescape(text_facts)) end unless facts.name == request.key @@ -81,12 +82,14 @@ class Puppet::Resource::Catalog::Compiler < Puppet::Indirector::Code str += " in environment #{node.environment}" if node.environment config = nil - Puppet::Util::Profiler.profile(str) do - begin - config = Puppet::Parser::Compiler.compile(node) - rescue Puppet::Error => detail - Puppet.err(detail.to_s) if networked? - raise + benchmark(:notice, str) do + Puppet::Util::Profiler.profile(str) do + begin + config = Puppet::Parser::Compiler.compile(node) + rescue Puppet::Error => detail + Puppet.err(detail.to_s) if networked? + raise + end end end diff --git a/lib/puppet/indirector/catalog/static_compiler.rb b/lib/puppet/indirector/catalog/static_compiler.rb index 6e688bc4b..fe3bfb5d7 100644 --- a/lib/puppet/indirector/catalog/static_compiler.rb +++ b/lib/puppet/indirector/catalog/static_compiler.rb @@ -49,6 +49,7 @@ class Puppet::Resource::Catalog::StaticCompiler < Puppet::Indirector::Code next unless source =~ /^puppet:/ file = resource.to_ral + if file.recurse? add_children(request.key, catalog, resource, file) else @@ -59,6 +60,18 @@ class Puppet::Resource::Catalog::StaticCompiler < Puppet::Indirector::Code catalog end + # Take a resource with a fileserver based file source remove the source + # parameter, and insert the file metadata into the resource. + # + # This method acts to do the fileserver metadata retrieval in advance, while + # the file source is local and doesn't require an HTTP request. It retrieves + # the file metadata for a given file resource, removes the source parameter + # from the resource, inserts the metadata into the file resource, and uploads + # the file contents of the source to the file bucket. + # + # @param host [String] The host name of the node requesting this catalog + # @param resource [Puppet::Resource] The resource to replace the metadata in + # @param file [Puppet::Type::File] The file RAL associated with the resource def find_and_replace_metadata(host, resource, file) # We remove URL info from it, so it forces a local copy # rather than routing through the network. @@ -71,6 +84,14 @@ class Puppet::Resource::Catalog::StaticCompiler < Puppet::Indirector::Code replace_metadata(host, resource, metadata) end + # Rewrite a given file resource with the metadata from a fileserver based file + # + # This performs the actual metadata rewrite for the given file resource and + # uploads the content of the source file to the filebucket. + # + # @param host [String] The host name of the node requesting this catalog + # @param resource [Puppet::Resource] The resource to add the metadata to + # @param metadata [Puppet::FileServing::Metadata] The metadata of the given fileserver based file def replace_metadata(host, resource, metadata) [:mode, :owner, :group].each do |param| resource[param] ||= metadata.send(param) @@ -89,6 +110,12 @@ class Puppet::Resource::Catalog::StaticCompiler < Puppet::Indirector::Code Puppet.info "Metadata for #{resource} in catalog for '#{host}' added from '#{old_source}'" end + # Generate children resources for a recursive file and add them to the catalog. + # + # @param host [String] The host name of the node requesting this catalog + # @param catalog [Puppet::Resource::Catalog] + # @param resource [Puppet::Resource] + # @param file [Puppet::Type::File] The file RAL associated with the resource def add_children(host, catalog, resource, file) file = resource.to_ral @@ -102,6 +129,14 @@ class Puppet::Resource::Catalog::StaticCompiler < Puppet::Indirector::Code end end + # Given a recursive file resource, recursively generate its children resources + # + # @param host [String] The host name of the node requesting this catalog + # @param catalog [Puppet::Resource::Catalog] + # @param resource [Puppet::Resource] + # @param file [Puppet::Type::File] The file RAL associated with the resource + # + # @return [Array<Puppet::Resource>] The recursively generated File resources for the given resource def get_child_resources(host, catalog, resource, file) sourceselect = file[:sourceselect] children = {} @@ -137,18 +172,33 @@ class Puppet::Resource::Catalog::StaticCompiler < Puppet::Indirector::Code # I think this is safe since it's a URL, not an actual file children[meta.relative_path][:source] = source + "/" + meta.relative_path + resource.each do |param, value| + # These should never be passed to our children. + unless [:parent, :ensure, :recurse, :recurselimit, :target, :alias, :source].include? param + children[meta.relative_path][param] = value + end + end replace_metadata(host, children[meta.relative_path], meta) end children end + # Remove any file resources in the catalog that will be duplicated by the + # given file resources. + # + # @param children [Array<Puppet::Resource>] + # @param catalog [Puppet::Resource::Catalog] def remove_existing_resources(children, catalog) existing_names = catalog.resources.collect { |r| r.to_s } both = (existing_names & children.keys).inject({}) { |hash, name| hash[name] = true; hash } both.each { |name| children.delete(name) } end + # Retrieve the source of a file resource using a fileserver based source and + # upload it to the filebucket. + # + # @param resource [Puppet::Resource] def store_content(resource) @summer ||= Object.new @summer.extend(Puppet::Util::Checksums) diff --git a/lib/puppet/indirector/certificate/rest.rb b/lib/puppet/indirector/certificate/rest.rb index 1cbf06afc..ab18b3542 100644 --- a/lib/puppet/indirector/certificate/rest.rb +++ b/lib/puppet/indirector/certificate/rest.rb @@ -2,7 +2,7 @@ require 'puppet/ssl/certificate' require 'puppet/indirector/rest' class Puppet::SSL::Certificate::Rest < Puppet::Indirector::REST - desc "Find and save certificates over HTTP via REST." + desc "Find certificates over HTTP via REST." use_server_setting(:ca_server) use_port_setting(:ca_port) diff --git a/lib/puppet/indirector/exec.rb b/lib/puppet/indirector/exec.rb index 6acdbb5e9..19f028340 100644 --- a/lib/puppet/indirector/exec.rb +++ b/lib/puppet/indirector/exec.rb @@ -7,7 +7,7 @@ class Puppet::Indirector::Exec < Puppet::Indirector::Terminus name = request.key external_command = command - # Make sure it's an arry + # Make sure it's an array raise Puppet::DevError, "Exec commands must be an array" unless external_command.is_a?(Array) # Make sure it's fully qualified. diff --git a/lib/puppet/indirector/facts/facter.rb b/lib/puppet/indirector/facts/facter.rb index a66270757..a4ee18ad3 100644 --- a/lib/puppet/indirector/facts/facter.rb +++ b/lib/puppet/indirector/facts/facter.rb @@ -19,7 +19,7 @@ class Puppet::Node::Facts::Facter < Puppet::Indirector::Code end end.flatten dirs = module_fact_dirs + Puppet[:factpath].split(File::PATH_SEPARATOR) - x = dirs.uniq.each do |dir| + dirs.uniq.each do |dir| load_facts_in_dir(dir) end end @@ -55,7 +55,7 @@ class Puppet::Node::Facts::Facter < Puppet::Indirector::Code result = Puppet::Node::Facts.new(request.key, Facter.to_hash) result.add_local_facts - result.stringify + Puppet[:stringify_facts] ? result.stringify : result.sanitize result end diff --git a/lib/puppet/indirector/facts/inventory_active_record.rb b/lib/puppet/indirector/facts/inventory_active_record.rb index b6c703bb5..e0e04baaf 100644 --- a/lib/puppet/indirector/facts/inventory_active_record.rb +++ b/lib/puppet/indirector/facts/inventory_active_record.rb @@ -46,7 +46,6 @@ class Puppet::Node::Facts::InventoryActiveRecord < Puppet::Indirector::ActiveRec def search(request) return [] unless request.options matching_nodes = [] - fact_names = [] fact_filters = Hash.new {|h,k| h[k] = []} meta_filters = Hash.new {|h,k| h[k] = []} request.options.each do |key,value| diff --git a/lib/puppet/indirector/facts/network_device.rb b/lib/puppet/indirector/facts/network_device.rb index 10030678f..2c8da947f 100644 --- a/lib/puppet/indirector/facts/network_device.rb +++ b/lib/puppet/indirector/facts/network_device.rb @@ -9,7 +9,7 @@ class Puppet::Node::Facts::NetworkDevice < Puppet::Indirector::Code result = Puppet::Node::Facts.new(request.key, Puppet::Util::NetworkDevice.current.facts) result.add_local_facts - result.stringify + Puppet[:stringify_facts] ? result.stringify : result.sanitize result end diff --git a/lib/puppet/indirector/file_bucket_file/file.rb b/lib/puppet/indirector/file_bucket_file/file.rb index ca79e783e..a8fd79b7e 100644 --- a/lib/puppet/indirector/file_bucket_file/file.rb +++ b/lib/puppet/indirector/file_bucket_file/file.rb @@ -22,7 +22,6 @@ module Puppet::FileBucketFile return nil unless path_match(dir_path, files_original_path) if request.options[:diff_with] - hash_protocol = sumtype(checksum) file2_path = path_for(request.options[:bucket_path], request.options[:diff_with], 'contents') raise "could not find diff_with #{request.options[:diff_with]}" unless ::File.exists?(file2_path) return `diff #{file_path.inspect} #{file2_path.inspect}` @@ -45,7 +44,9 @@ module Puppet::FileBucketFile checksum, files_original_path = request_to_checksum_and_path(request) save_to_disk(instance, files_original_path) - instance.to_s + + # don't echo the request content back to the agent + model.new('') end def validate_key(request) diff --git a/lib/puppet/indirector/indirection.rb b/lib/puppet/indirector/indirection.rb index 6e128eaac..a07bddcce 100644 --- a/lib/puppet/indirector/indirection.rb +++ b/lib/puppet/indirector/indirection.rb @@ -88,7 +88,7 @@ class Puppet::Indirector::Indirection text += scrub(@doc) + "\n\n" if @doc text << "* **Indirected Class**: `#{@indirected_class}`\n"; - if s = terminus_setting + if terminus_setting text << "* **Terminus Setting**: #{terminus_setting}\n" end @@ -247,7 +247,7 @@ class Puppet::Indirector::Indirection result = terminus.destroy(request) - if cache? and cached = cache.find(request(:find, key, nil, options)) + if cache? and cache.find(request(:find, key, nil, options)) # Reuse the existing request, since it's equivalent. cache.destroy(request) end diff --git a/lib/puppet/indirector/memory.rb b/lib/puppet/indirector/memory.rb index c44d62de2..0e3afaef7 100644 --- a/lib/puppet/indirector/memory.rb +++ b/lib/puppet/indirector/memory.rb @@ -15,6 +15,15 @@ class Puppet::Indirector::Memory < Puppet::Indirector::Terminus @instances[request.key] end + def search(request) + found_keys = @instances.keys.find_all { |key| key.include?(request.key) } + found_keys.collect { |key| @instances[key] } + end + + def head(request) + not find(request).nil? + end + def save(request) @instances[request.key] = request.instance end diff --git a/lib/puppet/indirector/node/ldap.rb b/lib/puppet/indirector/node/ldap.rb index 20045c8b5..f1a79ef6e 100644 --- a/lib/puppet/indirector/node/ldap.rb +++ b/lib/puppet/indirector/node/ldap.rb @@ -9,15 +9,14 @@ class Puppet::Node::Ldap < Puppet::Indirector::Ldap # The attributes that Puppet class information is stored in. def class_attributes - # LAK:NOTE See http://snurl.com/21zf8 [groups_google_com] - x = Puppet[:ldapclassattrs].split(/\s*,\s*/) + Puppet[:ldapclassattrs].split(/\s*,\s*/) end # Separate this out so it's relatively atomic. It's tempting to call # process instead of name2hash() here, but it ends up being # difficult to test because all exceptions get caught by ldapsearch. # LAK:NOTE Unfortunately, the ldap support is too stupid to throw anything - # but LDAP::ResultError, even on bad connections, so we are rough handed + # but LDAP::ResultError, even on bad connections, so we are rough-handed # with our error handling. def name2hash(name) info = nil @@ -198,7 +197,6 @@ class Puppet::Node::Ldap < Puppet::Indirector::Ldap end def merge_parent(info) - parent_info = nil parent = info[:parent] # Preload the parent array with the node name. diff --git a/lib/puppet/indirector/report/processor.rb b/lib/puppet/indirector/report/processor.rb index f8673bbf9..ef2232ef2 100644 --- a/lib/puppet/indirector/report/processor.rb +++ b/lib/puppet/indirector/report/processor.rb @@ -43,8 +43,7 @@ class Puppet::Transaction::Report::Processor < Puppet::Indirector::Code # Handle the parsing of the reports attribute. def reports - # LAK:NOTE See http://snurl.com/21zf8 [groups_google_com] - x = Puppet[:reports].gsub(/(^\s+)|(\s+$)/, '').split(/\s*,\s*/) + Puppet[:reports].gsub(/(^\s+)|(\s+$)/, '').split(/\s*,\s*/) end def processors(&blk) diff --git a/lib/puppet/indirector/report/rest.rb b/lib/puppet/indirector/report/rest.rb index a0b27ffb2..26a80f3bb 100644 --- a/lib/puppet/indirector/report/rest.rb +++ b/lib/puppet/indirector/report/rest.rb @@ -9,7 +9,7 @@ class Puppet::Transaction::Report::Rest < Puppet::Indirector::REST private def deserialize_save(content_type, body) - format = Puppet::Network::FormatHandler.protected_format(content_type) + format = Puppet::Network::FormatHandler.format_for(content_type) format.intern(Array, body) end end diff --git a/lib/puppet/indirector/request.rb b/lib/puppet/indirector/request.rb index 06db55f61..fd20d56e6 100644 --- a/lib/puppet/indirector/request.rb +++ b/lib/puppet/indirector/request.rb @@ -5,7 +5,7 @@ require 'puppet/util/pson' require 'puppet/network/resolver' # This class encapsulates all of the information you need to make an -# Indirection call, and as a a result also handles REST calls. It's somewhat +# Indirection call, and as a result also handles REST calls. It's somewhat # analogous to an HTTP Request object, except tuned for our Indirector. class Puppet::Indirector::Request attr_accessor :key, :method, :options, :instance, :node, :ip, :authenticated, :ignore_cache, :ignore_terminus @@ -158,20 +158,42 @@ class Puppet::Indirector::Request # Create the query string, if options are present. def query_string - return "" unless options and ! options.empty? - "?" + options.collect do |key, value| + return "" if options.nil? || options.empty? + "?" + encode_params(expand_into_parameters(options.to_a)) + end + + def expand_into_parameters(data) + data.inject([]) do |params, key_value| + key, value = key_value + + expanded_value = case value + when Array + value.collect { |val| [key, val] } + else + [key_value] + end + + params.concat(expand_primitive_types_into_parameters(expanded_value)) + end + end + + def expand_primitive_types_into_parameters(data) + data.inject([]) do |params, key_value| + key, value = key_value case value - when nil; next - when true, false; value = value.to_s - when Fixnum, Bignum, Float; value = value # nothing - when String; value = CGI.escape(value) - when Symbol; value = CGI.escape(value.to_s) - when Array; value = CGI.escape(YAML.dump(value)) + when nil + params + when true, false, String, Symbol, Fixnum, Bignum, Float + params << [key, value] else raise ArgumentError, "HTTP REST queries cannot handle values of type '#{value.class}'" end + end + end - "#{key}=#{value}" + def encode_params(params) + params.collect do |key, value| + "#{key}=#{CGI.escape(value.to_s)}" end.join("&") end diff --git a/lib/puppet/indirector/resource/rest.rb b/lib/puppet/indirector/resource/rest.rb index 031fc416c..824af41d1 100644 --- a/lib/puppet/indirector/resource/rest.rb +++ b/lib/puppet/indirector/resource/rest.rb @@ -9,7 +9,7 @@ class Puppet::Resource::Rest < Puppet::Indirector::REST def deserialize_save(content_type, body) # Body is [ral_res.to_resource, transaction.report] - format = Puppet::Network::FormatHandler.protected_format(content_type) + format = Puppet::Network::FormatHandler.format_for(content_type) ary = format.intern(Array, body) [Puppet::Resource.from_pson(ary[0]), Puppet::Transaction::Report.from_pson(ary[1])] end diff --git a/lib/puppet/indirector/resource_type/parser.rb b/lib/puppet/indirector/resource_type/parser.rb index a167a7dca..4af03d0f8 100644 --- a/lib/puppet/indirector/resource_type/parser.rb +++ b/lib/puppet/indirector/resource_type/parser.rb @@ -2,9 +2,26 @@ require 'puppet/resource/type' require 'puppet/indirector/code' require 'puppet/indirector/resource_type' +# The main terminus for Puppet::Resource::Type +# +# This exposes the known resource types from within Puppet. Only find +# and search are supported. When a request is received, Puppet will +# attempt to load all resource types (by parsing manifests and modules) and +# returns a description of the resource types found. The format of these +# objects is documented at {Puppet::Resource::Type}. +# +# @api public class Puppet::Indirector::ResourceType::Parser < Puppet::Indirector::Code desc "Return the data-form of a resource type." + # Find will return the first resource_type with the given name. It is + # not possible to specify the kind of the resource type. + # + # @param request [Puppet::Indirector::Request] The request object. + # The only parameters used from the request are `environment` and + # `key`, which corresponds to the resource type's `name` field. + # @return [Puppet::Resource::Type, nil] + # @api public def find(request) krt = request.environment.known_resource_types @@ -20,19 +37,21 @@ class Puppet::Indirector::ResourceType::Parser < Puppet::Indirector::Code nil end - # This is the "search" indirection method for resource types. It searches - # through a specified environment for all custom declared classes - # (a.k.a 'hostclasses'), defined types (a.k.a. 'definitions'), and nodes. + # Search for resource types using a regular expression. Unlike `find`, this + # allows you to filter the results by the "kind" of the resource type + # ("class", "defined_type", or "node"). All three are searched if no + # `kind` filter is given. This also accepts the special string "`*`" + # to return all resource type objects. + # + # @param request [Puppet::Indirector::Request] The request object. The + # `key` field holds the regular expression used to search, and + # `options[:kind]` holds the kind query parameter to filter the + # result as described above. The `environment` field specifies the + # environment used to load resources. + # + # @return [Array<Puppet::Resource::Type>, nil] # - # @param [Puppet::Indirector::Request] request - # Important properties of the request parameter: - # 1. request.environment : The environment in which to look for types. - # 2. request.key : A String that will be treated as a regular expression to - # be matched against the names of the available types. You may also - # pass a "*", which will match all available types. - # 3. request.options[:kind] : a String that can be used to filter the output - # to only return the desired kinds. The current supported values are - # 'class', 'defined_type', and 'node'. + # @api public def search(request) krt = request.environment.known_resource_types # Make sure we've got all of the types loaded. diff --git a/lib/puppet/interface.rb b/lib/puppet/interface.rb index 9ee1c800b..d40ef8c61 100644 --- a/lib/puppet/interface.rb +++ b/lib/puppet/interface.rb @@ -101,7 +101,7 @@ class Puppet::Interface def [](name, version) unless face = Puppet::Interface::FaceCollection[name, version] # REVISIT (#18042) no sense in rechecking if version == :current -- josh - if current = Puppet::Interface::FaceCollection[name, :current] + if Puppet::Interface::FaceCollection[name, :current] raise Puppet::Error, "Could not find version #{version} of #{name}" else raise Puppet::Error, "Could not find Puppet Face #{name.to_s}" diff --git a/lib/puppet/interface/documentation.rb b/lib/puppet/interface/documentation.rb index ef5c5a2b1..ec312512e 100644 --- a/lib/puppet/interface/documentation.rb +++ b/lib/puppet/interface/documentation.rb @@ -7,9 +7,6 @@ class Puppet::Interface # We need to identify an indent: the minimum number of whitespace # characters at the start of any line in the text. - # - # Using split rather than each_line is because the later only takes a - # block on Ruby 1.8.5 / Centos, and we support that. --daniel 2011-05-03 indent = text.split(/\n/).map {|x| x.index(/[^\s]/) }.compact.min if indent > 0 then @@ -79,7 +76,7 @@ class Puppet::Interface # @api private def build_synopsis(face, action = nil, arguments = nil) - output = PrettyPrint.format do |s| + PrettyPrint.format do |s| s.text("puppet #{face}") s.text(" #{action}") unless action.nil? s.text(" ") @@ -105,7 +102,6 @@ class Puppet::Interface display_global_options.sort.each do |option| wrap = %w{ [ ] } s.group(0, *wrap) do - desc = Puppet.settings.setting(option).desc type = Puppet.settings.setting(option).default type ||= Puppet.settings.setting(option).type.to_s.upcase s.text "--#{option} #{type}" @@ -134,11 +130,11 @@ class Puppet::Interface # Sets examples. # @param text [String] Example text # @api public - # @returns [void] + # @return [void] # @dsl Faces # @overload examples # Returns documentation of examples - # @returns [String] The examples + # @return [String] The examples # @api private attr_doc :examples @@ -147,11 +143,11 @@ class Puppet::Interface # Sets optional notes. # @param text [String] The notes # @api public - # @returns [void] + # @return [void] # @dsl Faces # @overload notes # Returns any optional notes - # @returns [String] The notes + # @return [String] The notes # @api private attr_doc :notes @@ -160,11 +156,11 @@ class Puppet::Interface # Sets the license text # @param text [String] the license text # @api public - # @returns [void] + # @return [void] # @dsl Faces # @overload license # Returns the license - # @returns [String] The license + # @return [String] The license # @api private attr_doc :license diff --git a/lib/puppet/interface/option.rb b/lib/puppet/interface/option.rb index 288b016f9..199039cb1 100644 --- a/lib/puppet/interface/option.rb +++ b/lib/puppet/interface/option.rb @@ -84,7 +84,7 @@ class Puppet::Interface::Option unless found = declaration.match(/^-+(?:\[no-\])?([^ =]+)/) then raise ArgumentError, "Can't find a name in the declaration #{declaration.inspect}" end - name = found.captures.first + found.captures.first end # @api private diff --git a/lib/puppet/interface/option_builder.rb b/lib/puppet/interface/option_builder.rb index f4d7c6cf0..72e09cacc 100644 --- a/lib/puppet/interface/option_builder.rb +++ b/lib/puppet/interface/option_builder.rb @@ -57,7 +57,7 @@ class Puppet::Interface::OptionBuilder def after_action(&block) block or raise ArgumentError, "#{@option} after_action requires a block" if @option.after_action - raise ArgumentError, "#{@option} already has a after_action set" + raise ArgumentError, "#{@option} already has an after_action set" end unless block.arity == 3 then raise ArgumentError, "after_action takes three arguments, action, args, and options" diff --git a/lib/puppet/metatype/manager.rb b/lib/puppet/metatype/manager.rb index af0187368..3d640879f 100644 --- a/lib/puppet/metatype/manager.rb +++ b/lib/puppet/metatype/manager.rb @@ -131,7 +131,7 @@ module Manager def rmtype(name) # Then create the class. - klass = rmclass(name, :hash => @types) + rmclass(name, :hash => @types) singleton_class.send(:remove_method, "new#{name}") if respond_to?("new#{name}") end @@ -145,7 +145,7 @@ module Manager @types ||= {} # We are overwhelmingly symbols here, which usually match, so it is worth - # having this special-case to return quickly. Like, 25K to 300 symbols to + # having this special-case to return quickly. Like, 25K symbols vs. 300 # strings in this method. --daniel 2012-07-17 return @types[name] if @types[name] diff --git a/lib/puppet/module.rb b/lib/puppet/module.rb index abc7b2fd8..8d5434edd 100644 --- a/lib/puppet/module.rb +++ b/lib/puppet/module.rb @@ -147,6 +147,12 @@ class Puppet::Module init_manifests + searched_manifests end + def all_manifests + return [] unless File.exists?(manifests) + + Dir.glob(File.join(manifests, '**', '*.{rb,pp}')) + end + def metadata_file return @metadata_file if defined?(@metadata_file) @@ -234,7 +240,7 @@ class Puppet::Module dep_mod = begin environment.module_by_forge_name(forge_name) - rescue => e + rescue nil end diff --git a/lib/puppet/module_tool.rb b/lib/puppet/module_tool.rb index f19b4395b..98876b6dd 100644 --- a/lib/puppet/module_tool.rb +++ b/lib/puppet/module_tool.rb @@ -17,7 +17,7 @@ module Puppet # Is this a directory that shouldn't be checksummed? # # TODO: Should this be part of Checksums? - # TODO: Rename this method to reflect it's purpose? + # TODO: Rename this method to reflect its purpose? # TODO: Shouldn't this be used when building packages too? def self.artifact?(path) case File.basename(path) diff --git a/lib/puppet/module_tool/applications/application.rb b/lib/puppet/module_tool/applications/application.rb index 88ee99aa3..93338be53 100644 --- a/lib/puppet/module_tool/applications/application.rb +++ b/lib/puppet/module_tool/applications/application.rb @@ -47,6 +47,16 @@ module Puppet::ModuleTool elsif require_modulefile raise ArgumentError, "No Modulefile found." end + extra_metadata_path = File.join(@path, 'metadata.json') + if File.file?(extra_metadata_path) + File.open(extra_metadata_path) do |f| + begin + @metadata.extra_metadata = PSON.load(f) + rescue PSON::ParserError + raise ArgumentError, "Could not parse JSON #{extra_metadata_path}" + end + end + end end @metadata end diff --git a/lib/puppet/module_tool/applications/installer.rb b/lib/puppet/module_tool/applications/installer.rb index 08f736a0a..e53f6308b 100644 --- a/lib/puppet/module_tool/applications/installer.rb +++ b/lib/puppet/module_tool/applications/installer.rb @@ -122,7 +122,7 @@ module Puppet::ModuleTool # # Resolve installation conflicts by checking if the requested module - # or one of it's dependencies conflicts with an installed module. + # or one of its dependencies conflicts with an installed module. # # Conflicts occur under the following conditions: # @@ -153,7 +153,10 @@ module Puppet::ModuleTool :version => release[:version][:vstring] } dependency = is_dependency ? dependency_info : nil - latest_version = @versions["#{@module_name}"].sort_by { |h| h[:semver] }.last[:vstring] + all_versions = @versions["#{@module_name}"].sort_by { |h| h[:semver] } + versions = all_versions.select { |x| x[:semver].special == '' } + versions = all_versions if versions.empty? + latest_version = versions.last[:vstring] raise InstallConflictError, :requested_module => @module_name, @@ -172,7 +175,7 @@ module Puppet::ModuleTool # Check if a file is a vaild module package. # --- # FIXME: Checking for a valid module package should be more robust and - # use the acutal metadata contained in the package. 03132012 - Hightower + # use the actual metadata contained in the package. 03132012 - Hightower # +++ # def is_module_package?(name) diff --git a/lib/puppet/module_tool/dependency.rb b/lib/puppet/module_tool/dependency.rb index 9ebffab2b..847a2e3c1 100644 --- a/lib/puppet/module_tool/dependency.rb +++ b/lib/puppet/module_tool/dependency.rb @@ -2,6 +2,8 @@ module Puppet::ModuleTool class Dependency + attr_reader :full_module_name, :username, :name, :version_requirement, :repository + # Instantiates a new module dependency with a +full_module_name+ (e.g. # "myuser-mymodule"), and optional +version_requirement+ (e.g. "0.0.1") and # optional repository (a URL string). diff --git a/lib/puppet/module_tool/errors/upgrader.rb b/lib/puppet/module_tool/errors/upgrader.rb index 43893856a..deee1bfef 100644 --- a/lib/puppet/module_tool/errors/upgrader.rb +++ b/lib/puppet/module_tool/errors/upgrader.rb @@ -14,7 +14,7 @@ module Puppet::ModuleTool::Errors @installed_version = options[:installed_version] @dependency_name = options[:dependency_name] @conditions = options[:conditions] - super "Could not upgrade '#{@module_name}'; module is not installed" + super "Could not upgrade '#{@module_name}';" end def multiline diff --git a/lib/puppet/module_tool/metadata.rb b/lib/puppet/module_tool/metadata.rb index 8f94316bd..37a753626 100644 --- a/lib/puppet/module_tool/metadata.rb +++ b/lib/puppet/module_tool/metadata.rb @@ -79,6 +79,14 @@ module Puppet::ModuleTool @description = description end + def extra_metadata + @extra_metadata || {} + end + + def extra_metadata=(extra_metadata) + @extra_metadata = extra_metadata + end + def project_page @project_page || 'UNKNOWN' end @@ -121,21 +129,25 @@ module Puppet::ModuleTool end end + def to_hash() + return extra_metadata.merge({ + 'name' => @full_module_name, + 'version' => @version, + 'source' => source, + 'author' => author, + 'license' => license, + 'summary' => summary, + 'description' => description, + 'project_page' => project_page, + 'dependencies' => dependencies, + 'types' => types, + 'checksums' => checksums + }) + end + # Return the PSON record representing this instance. def to_pson(*args) - return { - :name => @full_module_name, - :version => @version, - :source => source, - :author => author, - :license => license, - :summary => summary, - :description => description, - :project_page => project_page, - :dependencies => dependencies, - :types => types, - :checksums => checksums - }.to_pson(*args) + return to_hash.to_pson(*args) end end end diff --git a/lib/puppet/module_tool/modulefile.rb b/lib/puppet/module_tool/modulefile.rb index df68f3246..321fcac37 100644 --- a/lib/puppet/module_tool/modulefile.rb +++ b/lib/puppet/module_tool/modulefile.rb @@ -57,18 +57,18 @@ module Puppet::ModuleTool @metadata.license = license end - # Set the summary - def summary(summary) + # Set the summary + def summary(summary) @metadata.summary = summary end - # Set the description - def description(description) + # Set the description + def description(description) @metadata.description = description - end + end - # Set the project page - def project_page(project_page) + # Set the project page + def project_page(project_page) @metadata.project_page = project_page end end diff --git a/lib/puppet/module_tool/shared_behaviors.rb b/lib/puppet/module_tool/shared_behaviors.rb index b43b16688..2b79b4ed4 100644 --- a/lib/puppet/module_tool/shared_behaviors.rb +++ b/lib/puppet/module_tool/shared_behaviors.rb @@ -36,7 +36,7 @@ module Puppet::ModuleTool::Shared mod_name, releases = pair mod_name = mod_name.gsub('/', '-') releases.each do |rel| - semver = SemVer.new(rel['version'] || '0.0.0') rescue SemVer.MIN + semver = SemVer.new(rel['version'] || '0.0.0') rescue SemVer::MIN @versions[mod_name] << { :vstring => rel['version'], :semver => semver } @versions[mod_name].sort! { |a, b| a[:semver] <=> b[:semver] } @urls["#{mod_name}@#{rel['version']}"] = rel['file'] @@ -107,7 +107,9 @@ module Puppet::ModuleTool::Shared @conditions.each { |_, conds| conds.delete_if { |c| c[:module] == mod } } end - valid_versions = @versions["#{mod}"].select { |h| range === h[:semver] } + versions = @versions["#{mod}"].select { |h| range === h[:semver] } + valid_versions = versions.select { |x| x[:semver].special == '' } + valid_versions = versions if valid_versions.empty? unless version = valid_versions.last req_module = @module_name diff --git a/lib/puppet/module_tool/skeleton.rb b/lib/puppet/module_tool/skeleton.rb index 33f92636e..c5e26e35f 100644 --- a/lib/puppet/module_tool/skeleton.rb +++ b/lib/puppet/module_tool/skeleton.rb @@ -23,7 +23,7 @@ module Puppet::ModuleTool # Return Pathname of custom templates directory. def custom_path - Pathname(Puppet.settings[:module_working_dir]) + 'skeleton' + Pathname(Puppet.settings[:module_skeleton_dir]) end # Return Pathname of default template directory. diff --git a/lib/puppet/module_tool/skeleton/templates/generator/manifests/init.pp.erb b/lib/puppet/module_tool/skeleton/templates/generator/manifests/init.pp.erb index 82f75fa1f..d9d0df0b5 100644 --- a/lib/puppet/module_tool/skeleton/templates/generator/manifests/init.pp.erb +++ b/lib/puppet/module_tool/skeleton/templates/generator/manifests/init.pp.erb @@ -15,16 +15,16 @@ # Here you should define a list of variables that this module would require. # # [*sample_variable*] -# Explanation of how this variable affects the funtion of this class and if it -# has a default. e.g. "The parameter enc_ntp_servers must be set by the +# Explanation of how this variable affects the funtion of this class and if +# it has a default. e.g. "The parameter enc_ntp_servers must be set by the # External Node Classifier as a comma separated list of hostnames." (Note, -# global variables should not be used in preference to class parameters as of -# Puppet 2.6.) +# global variables should be avoided in favor of class parameters as +# of Puppet 2.6.) # # === Examples # # class { <%= metadata.name %>: -# servers => [ 'pool.ntp.org', 'ntp.local.company.com' ] +# servers => [ 'pool.ntp.org', 'ntp.local.company.com' ], # } # # === Authors diff --git a/lib/puppet/module_tool/skeleton/templates/generator/tests/init.pp.erb b/lib/puppet/module_tool/skeleton/templates/generator/tests/init.pp.erb index ba4456396..740a095d2 100644 --- a/lib/puppet/module_tool/skeleton/templates/generator/tests/init.pp.erb +++ b/lib/puppet/module_tool/skeleton/templates/generator/tests/init.pp.erb @@ -2,10 +2,11 @@ # should have a corresponding test manifest that declares that class or defined # type. # -# Tests are then run by using puppet apply --noop (to check for compilation errors -# and view a log of events) or by fully applying the test in a virtual environment -# (to compare the resulting system state to the desired state). +# Tests are then run by using puppet apply --noop (to check for compilation +# errors and view a log of events) or by fully applying the test in a virtual +# environment (to compare the resulting system state to the desired state). # -# Learn more about module testing here: http://docs.puppetlabs.com/guides/tests_smoke.html +# Learn more about module testing here: +# http://docs.puppetlabs.com/guides/tests_smoke.html # include <%= metadata.name %> diff --git a/lib/puppet/network/auth_config_parser.rb b/lib/puppet/network/auth_config_parser.rb index 070a058fe..2af06051c 100644 --- a/lib/puppet/network/auth_config_parser.rb +++ b/lib/puppet/network/auth_config_parser.rb @@ -27,6 +27,9 @@ class AuthConfigParser name = $1.chomp right = rights.newright(name, count, @file) when /^\s*(allow(?:_ip)?|deny(?:_ip)?|method|environment|auth(?:enticated)?)\s+(.+?)(\s*#.*)?$/ + if right.nil? + raise Puppet::ConfigurationError, "Missing or invalid 'path' before right directive at line #{count} of #{@file}" + end parse_right_directive(right, $1, $2, count) else raise Puppet::ConfigurationError, "Invalid line #{count}: #{line}" diff --git a/lib/puppet/network/authconfig.rb b/lib/puppet/network/authconfig.rb index 6cd86ae20..1c3eaede6 100644 --- a/lib/puppet/network/authconfig.rb +++ b/lib/puppet/network/authconfig.rb @@ -1,4 +1,3 @@ -require 'puppet/util/loadedfile' require 'puppet/network/rights' module Puppet diff --git a/lib/puppet/network/authorization.rb b/lib/puppet/network/authorization.rb index d7b88a5ec..43b8d0633 100644 --- a/lib/puppet/network/authorization.rb +++ b/lib/puppet/network/authorization.rb @@ -7,7 +7,7 @@ module Puppet::Network # Create our config object if necessary. If there's no configuration file # we install our defaults def self.authconfig - @auth_config_file ||= Puppet::Util::LoadedFile.new(Puppet[:rest_authconfig]) + @auth_config_file ||= Puppet::Util::WatchedFile.new(Puppet[:rest_authconfig]) if (not @auth_config) or @auth_config_file.changed? begin @auth_config = Puppet::Network::AuthConfigParser.new_from_file(Puppet[:rest_authconfig]).parse diff --git a/lib/puppet/network/authstore.rb b/lib/puppet/network/authstore.rb index 14e71847a..90498002d 100755 --- a/lib/puppet/network/authstore.rb +++ b/lib/puppet/network/authstore.rb @@ -232,7 +232,7 @@ module Puppet name.downcase.split(".").reverse end - # Parse our input pattern and figure out what kind of allowal + # Parse our input pattern and figure out what kind of allowable # statement it is. The output of this is used for later matching. Octet = '(\d|[1-9]\d|1\d\d|2[0-4]\d|25[0-5])' IPv4 = "#{Octet}\.#{Octet}\.#{Octet}\.#{Octet}" @@ -246,7 +246,7 @@ module Puppet def parse_ip(value) @name = :ip @exact, @length, @pattern = *case value - when /^(?:#{IP})\/(\d+)$/ # 12.34.56.78/24, a001:b002::efff/120, c444:1000:2000::9:192.168.0.1/112 + when /^(?:#{IP})\/(\d+)$/ # 12.34.56.78/24, a001:b002::efff/120, c444:1000:2000::9:192.168.0.1/112 [:inexact, $1.to_i, IPAddr.new(value)] when /^(#{IP})$/ # 10.20.30.40, [:exact, nil, IPAddr.new(value)] diff --git a/lib/puppet/network/format_handler.rb b/lib/puppet/network/format_handler.rb index b94a4f902..880b58b3c 100644 --- a/lib/puppet/network/format_handler.rb +++ b/lib/puppet/network/format_handler.rb @@ -5,33 +5,10 @@ require 'puppet/network/format' module Puppet::Network::FormatHandler class FormatError < Puppet::Error; end - class FormatProtector - attr_reader :format - - def protect(method, args) - Puppet::Network::FormatHandler.format(format).send(method, *args) - rescue => details - direction = method.to_s.include?("intern") ? "from" : "to" - error = FormatError.new("Could not #{method} #{direction} #{format}: #{details}") - error.set_backtrace(details.backtrace) - raise error - end - - def initialize(format) - @format = format - end - - [:intern, :intern_multiple, :render, :render_multiple, :mime].each do |method| - define_method(method) do |*args| - protect(method, args) - end - end - end - @formats = {} + def self.create(*args, &block) - instance = Puppet::Network::Format.new(*args) - instance.instance_eval(&block) if block_given? + instance = Puppet::Network::Format.new(*args, &block) @formats[instance.name] = instance instance @@ -43,18 +20,15 @@ module Puppet::Network::FormatHandler } end - def self.extended(klass) - klass.extend(ClassMethods) - - # LAK:NOTE This won't work in 1.9 ('send' won't be able to send - # private methods, but I don't know how else to do it. - klass.send(:include, InstanceMethods) - end - def self.format(name) @formats[name.to_s.downcase.intern] end + def self.format_for(name) + name = format_to_canonical_name(name) + format(name) + end + def self.format_by_extension(ext) @formats.each do |name, format| return format if format.extension == ext @@ -73,15 +47,6 @@ module Puppet::Network::FormatHandler @formats.values.find { |format| format.mime == mimetype } end - # Use a delegator to make sure any exceptions generated by our formats are - # handled intelligently. - def self.protected_format(name) - name = format_to_canonical_name(name) - @format_protectors ||= {} - @format_protectors[name] ||= FormatProtector.new(name) - @format_protectors[name] - end - # Return a format name given: # * a format name # * a mime-type @@ -99,81 +64,35 @@ module Puppet::Network::FormatHandler out.name end - module ClassMethods - def format_handler - Puppet::Network::FormatHandler - end - - def convert_from(format, data) - format_handler.protected_format(format).intern(self, data) - end - - def convert_from_multiple(format, data) - format_handler.protected_format(format).intern_multiple(self, data) - end - - def render_multiple(format, instances) - format_handler.protected_format(format).render_multiple(instances) - end - - def default_format - supported_formats[0] - end - - def support_format?(name) - Puppet::Network::FormatHandler.format(name).supported?(self) - end - - def supported_formats - result = format_handler.formats.collect { |f| format_handler.format(f) }.find_all { |f| f.supported?(self) }.collect { |f| f.name }.sort do |a, b| - # It's an inverse sort -- higher weight formats go first. - format_handler.format(b).weight <=> format_handler.format(a).weight - end - - result = put_preferred_format_first(result) - - Puppet.debug "#{friendly_name} supports formats: #{result.map{ |f| f.to_s }.sort.join(' ')}; using #{result.first}" - - result - end - - private - - def friendly_name - if self.respond_to? :indirection - indirection.name - else - self - end - end - - def put_preferred_format_first(list) - preferred_format = Puppet.settings[:preferred_serialization_format].to_sym - if list.include?(preferred_format) - list.delete(preferred_format) - list.unshift(preferred_format) - else - Puppet.debug "Value of 'preferred_serialization_format' (#{preferred_format}) is invalid for #{friendly_name}, using default (#{list.first})" + # Determine which of the accepted formats should be used given what is supported. + # + # @param accepted [Array<String, Symbol>] the accepted formats in a form a + # that generally conforms to an HTTP Accept header. Any quality specifiers + # are ignored and instead the formats are simply in strict preference order + # (most preferred is first) + # @param supported [Array<Symbol>] the names of the supported formats (order + # does not matter) + # @return [Puppet::Network::Format, nil] the most suitable format + # @api private + def self.most_suitable_format_for(accepted, supported) + format_name = accepted.collect do |accepted| + accepted.to_s.sub(/;q=.*$/, '') + end.collect do |accepted| + begin + if accepted == '*/*' + formats + else + format_to_canonical_name(accepted) + end + rescue ArgumentError + nil end - list - end - end - - module InstanceMethods - def render(format = nil) - format ||= self.class.default_format - - Puppet::Network::FormatHandler.protected_format(format).render(self) - end - - def mime(format = nil) - format ||= self.class.default_format - - Puppet::Network::FormatHandler.protected_format(format).mime + end.flatten.find do |accepted| + supported.include?(accepted) end - def support_format?(name) - self.class.support_format?(name) + if format_name + format_for(format_name) end end end diff --git a/lib/puppet/network/format_support.rb b/lib/puppet/network/format_support.rb new file mode 100644 index 000000000..7cc6cc001 --- /dev/null +++ b/lib/puppet/network/format_support.rb @@ -0,0 +1,106 @@ +require 'puppet/network/format_handler' + +# Provides network serialization support when included +module Puppet::Network::FormatSupport + def self.included(klass) + klass.extend(ClassMethods) + end + + module ClassMethods + def convert_from(format, data) + get_format(format).intern(self, data) + rescue => err + raise Puppet::Network::FormatHandler::FormatError, "Could not intern from #{format}: #{err}", err.backtrace + end + + def convert_from_multiple(format, data) + get_format(format).intern_multiple(self, data) + rescue => err + raise Puppet::Network::FormatHandler::FormatError, "Could not intern_multiple from #{format}: #{err}", err.backtrace + end + + def render_multiple(format, instances) + get_format(format).render_multiple(instances) + rescue => err + raise Puppet::Network::FormatHandler::FormatError, "Could not render_multiple to #{format}: #{err}", err.backtrace + end + + def default_format + supported_formats[0] + end + + def support_format?(name) + Puppet::Network::FormatHandler.format(name).supported?(self) + end + + def supported_formats + result = format_handler.formats.collect do |f| + format_handler.format(f) + end.find_all do |f| + f.supported?(self) + end.sort do |a, b| + # It's an inverse sort -- higher weight formats go first. + b.weight <=> a.weight + end.collect do |f| + f.name + end + + result = put_preferred_format_first(result) + + Puppet.debug "#{friendly_name} supports formats: #{result.join(' ')}" + + result + end + + # @api private + def get_format(format_name) + format_handler.format_for(format_name) + end + + private + + def format_handler + Puppet::Network::FormatHandler + end + + def friendly_name + if self.respond_to? :indirection + indirection.name + else + self + end + end + + def put_preferred_format_first(list) + preferred_format = Puppet.settings[:preferred_serialization_format].to_sym + if list.include?(preferred_format) + list.delete(preferred_format) + list.unshift(preferred_format) + else + Puppet.debug "Value of 'preferred_serialization_format' (#{preferred_format}) is invalid for #{friendly_name}, using default (#{list.first})" + end + list + end + end + + def render(format = nil) + format ||= self.class.default_format + + self.class.get_format(format).render(self) + rescue => err + raise Puppet::Network::FormatHandler::FormatError, "Could not render to #{format}: #{err}", err.backtrace + end + + def mime(format = nil) + format ||= self.class.default_format + + self.class.get_format(format).mime + rescue => err + raise Puppet::Network::FormatHandler::FormatError, "Could not mime to #{format}: #{err}", err.backtrace + end + + def support_format?(name) + self.class.support_format?(name) + end +end + diff --git a/lib/puppet/network/formats.rb b/lib/puppet/network/formats.rb index c5f799a9a..45d75dc6c 100644 --- a/lib/puppet/network/formats.rb +++ b/lib/puppet/network/formats.rb @@ -1,14 +1,12 @@ require 'puppet/network/format_handler' Puppet::Network::FormatHandler.create_serialized_formats(:yaml) do - # Yaml doesn't need the class name; it's serialized. def intern(klass, text) data = YAML.load(text, :safe => true, :deserialize_symbols => true) return data if data.is_a?(klass) klass.from_pson(data) end - # Yaml doesn't need the class name; it's serialized. def intern_multiple(klass, text) YAML.load(text, :safe => true, :deserialize_symbols => true).collect do |data| if data.is_a?(klass) @@ -28,7 +26,6 @@ Puppet::Network::FormatHandler.create_serialized_formats(:yaml) do instances.to_yaml end - # Unlike core's yaml, ZAML should support 1.8.1 just fine def supported?(klass) true end @@ -159,7 +156,6 @@ Puppet::Network::FormatHandler.create(:console, if datum.is_a? Hash and datum.keys.all? { |x| x.is_a? String or x.is_a? Numeric } output = '' column_a = datum.empty? ? 2 : datum.map{ |k,v| k.to_s.length }.max + 2 - column_b = 79 - column_a datum.sort_by { |k,v| k.to_s } .each do |key, value| output << key.to_s.ljust(column_a) output << json.render(value). @@ -169,6 +165,16 @@ Puppet::Network::FormatHandler.create(:console, return output end + # Print one item per line for arrays + if datum.is_a? Array + output = '' + datum.each do |item| + output << item.to_s + output << "\n" + end + return output + end + # ...or pretty-print the inspect outcome. return json.render(datum) end diff --git a/lib/puppet/network/http/compression.rb b/lib/puppet/network/http/compression.rb index c8d001169..6e8314b72 100644 --- a/lib/puppet/network/http/compression.rb +++ b/lib/puppet/network/http/compression.rb @@ -69,7 +69,7 @@ module Puppet::Network::HTTP::Compression out = @uncompressor.inflate(chunk) @first = false return out - rescue Zlib::DataError => z + rescue Zlib::DataError # it can happen that we receive a raw deflate stream # which might make our inflate throw a data error. # in this case, we try with a verbatim (no header) diff --git a/lib/puppet/network/http/connection.rb b/lib/puppet/network/http/connection.rb index aab13b05c..c75bab97c 100644 --- a/lib/puppet/network/http/connection.rb +++ b/lib/puppet/network/http/connection.rb @@ -3,9 +3,13 @@ require 'puppet/ssl/host' require 'puppet/ssl/configuration' require 'puppet/ssl/validator' require 'puppet/network/authentication' +require 'uri' module Puppet::Network::HTTP + # This will be raised if too many redirects happen for a given HTTP request + class RedirectionLimitExceededException < Puppet::Error ; end + # This class provides simple methods for issuing various types of HTTP # requests. It's interface is intended to mirror Ruby's Net::HTTP # object, but it provides a few important bits of additional @@ -18,10 +22,34 @@ module Puppet::Network::HTTP class Connection include Puppet::Network::Authentication - def initialize(host, port, use_ssl = true) + OPTION_DEFAULTS = { + :use_ssl => true, + :verify_peer => true, + :redirect_limit => 10 + } + + # Creates a new HTTP client connection to `host`:`port`. + # @param host [String] the host to which this client will connect to + # @param port [Fixnum] the port to which this client will connect to + # @param options [Hash] options influencing the properties of the created connection, + # the following options are recognized: + # :use_ssl [Boolean] true to connect with SSL, false otherwise, defaults to true + # :verify_peer [Boolean] true to verify the peer's certificate, false otherwise, defaults to true + # :redirect_limit [Fixnum] the number of allowed redirections, defaults to 10 + # passing any other option in the options hash results in a Puppet::Error exception + # @note the HTTP connection itself happens lazily only when {#request}, or one of the {#get}, {#post}, {#delete}, {#head} or {#put} is called + # @api private + def initialize(host, port, options = {}) @host = host @port = port - @use_ssl = use_ssl + + unknown_options = options.keys - OPTION_DEFAULTS.keys + raise Puppet::Error, "Unrecognized option(s): #{unknown_options.map(&:inspect).sort.join(', ')}" unless unknown_options.empty? + + options = OPTION_DEFAULTS.merge(options) + @use_ssl = options[:use_ssl] + @verify_peer = options[:verify_peer] + @redirect_limit = options[:redirect_limit] end def get(*args) @@ -45,32 +73,22 @@ module Puppet::Network::HTTP end def request(method, *args) - ssl_validator = Puppet::SSL::Validator.new(:ssl_configuration => ssl_configuration) - # Perform our own validation of the SSL connection in addition to OpenSSL - ssl_validator.register_verify_callback(connection) - response = connection.send(method, *args) - # Check the peer certs and warn if they're nearing expiration. - warn_if_near_expiration(*ssl_validator.peer_certs) - - response - rescue OpenSSL::SSL::SSLError => error - if error.message.include? "certificate verify failed" - msg = error.message - msg << ": [" + ssl_validator.verify_errors.join('; ') + "]" - raise Puppet::Error, msg - elsif error.message =~ /hostname (\w+ )?not match/ - leaf_ssl_cert = ssl_validator.peer_certs.last - - valid_certnames = [leaf_ssl_cert.name, *leaf_ssl_cert.subject_alt_names].uniq - msg = valid_certnames.length > 1 ? "one of #{valid_certnames.join(', ')}" : valid_certnames.first - - raise Puppet::Error, "Server hostname '#{connection.address}' did not match server certificate; expected #{msg}" - else - raise + current_args = args.dup + @redirect_limit.times do |redirection| + response = execute_request(method, *args) + return response unless [301, 302, 307].include?(response.code.to_i) + + # handle the redirection + location = URI.parse(response['location']) + @connection = initialize_connection(location.host, location.port, location.scheme == 'https') + + # update to the current request path + current_args = [location.path] + current_args.drop(1) + # and try again... end + raise RedirectionLimitExceededException, "Too many HTTP redirections for #{@host}:#{@port}" end - # TODO: These are proxies for the Net::HTTP#request_* methods, which are # almost the same as the "get", "post", etc. methods that we've ported above, # but they are able to accept a code block and will yield to it. For now @@ -91,7 +109,6 @@ module Puppet::Network::HTTP end # end of Net::HTTP#request_* proxies - def address connection.address end @@ -104,15 +121,42 @@ module Puppet::Network::HTTP connection.use_ssl? end - private def connection - @connection || initialize_connection + @connection || initialize_connection(@host, @port, @use_ssl) + end + + def execute_request(method, *args) + ssl_validator = Puppet::SSL::Validator.new(:ssl_configuration => ssl_configuration) + # Perform our own validation of the SSL connection in addition to OpenSSL + ssl_validator.register_verify_callback(connection) + + response = connection.send(method, *args) + + # Check the peer certs and warn if they're nearing expiration. + warn_if_near_expiration(*ssl_validator.peer_certs) + + response + rescue OpenSSL::SSL::SSLError => error + if error.message.include? "certificate verify failed" + msg = error.message + msg << ": [" + ssl_validator.verify_errors.join('; ') + "]" + raise Puppet::Error, msg + elsif error.message =~ /hostname (\w+ )?not match/ + leaf_ssl_cert = ssl_validator.peer_certs.last + + valid_certnames = [leaf_ssl_cert.name, *leaf_ssl_cert.subject_alt_names].uniq + msg = valid_certnames.length > 1 ? "one of #{valid_certnames.join(', ')}" : valid_certnames.first + + raise Puppet::Error, "Server hostname '#{connection.address}' did not match server certificate; expected #{msg}" + else + raise + end end - def initialize_connection - args = [@host, @port] + def initialize_connection(host, port, use_ssl) + args = [host, port] if Puppet[:http_proxy_host] == "none" args << nil << nil else @@ -125,7 +169,7 @@ module Puppet::Network::HTTP # give us a reader for ca_file... Grr... class << @connection; attr_accessor :ca_file; end - @connection.use_ssl = @use_ssl + @connection.use_ssl = use_ssl # Use configured timeout (#1176) @connection.read_timeout = Puppet[:configtimeout] @connection.open_timeout = Puppet[:configtimeout] @@ -137,7 +181,7 @@ module Puppet::Network::HTTP # Use cert information from a Puppet client to set up the http object. def cert_setup - if FileTest.exist?(Puppet[:hostcert]) and FileTest.exist?(ssl_configuration.ca_auth_file) + if @verify_peer and FileTest.exist?(Puppet[:hostcert]) and FileTest.exist?(ssl_configuration.ca_auth_file) @connection.cert_store = ssl_host.ssl_store @connection.ca_file = ssl_configuration.ca_auth_file @connection.cert = ssl_host.certificate.content diff --git a/lib/puppet/network/http/handler.rb b/lib/puppet/network/http/handler.rb index b7abd44d4..c9f7c991d 100644 --- a/lib/puppet/network/http/handler.rb +++ b/lib/puppet/network/http/handler.rb @@ -14,8 +14,32 @@ module Puppet::Network::HTTP::Handler include Puppet::Network::Authorization include Puppet::Network::Authentication - attr_reader :server, :handler + # These shouldn't be allowed to be set by clients + # in the query string, for security reasons. + DISALLOWED_KEYS = ["node", "ip"] + + class HTTPError < Exception + attr_reader :status + + def initialize(message, status) + super(message) + @status = status + end + end + + class HTTPNotAcceptableError < HTTPError + def initialize(message) + super("Not Acceptable: " + message, 406) + end + end + class HTTPNotFoundError < HTTPError + def initialize(message) + super("Not Found: " + message, 404) + end + end + + attr_reader :server, :handler # Retrieve all headers from the http request, as a hash with the header names # (lower-cased) as the keys @@ -33,29 +57,12 @@ module Puppet::Network::HTTP::Handler raise NotImplementedError end - # Which format to use when serializing our response or interpreting the request. - # IF the client provided a Content-Type use this, otherwise use the Accept header - # and just pick the first value. - def format_to_use(request) - unless header = accept_header(request) - raise ArgumentError, "An Accept header must be provided to pick the right format" - end - - format = nil - header.split(/,\s*/).each do |name| - next unless format = Puppet::Network::FormatHandler.format(name) - next unless format.suitable? - return format - end - - raise "No specified acceptable formats (#{header}) are functional on this machine" - end - def request_format(request) if header = content_type_header(request) header.gsub!(/\s*;.*$/,'') # strip any charset format = Puppet::Network::FormatHandler.mime(header) raise "Client sent a mime-type (#{header}) that doesn't correspond to a format we support" if format.nil? + report_if_deprecated(format) return format.name.to_s if format.suitable? end @@ -89,6 +96,8 @@ module Puppet::Network::HTTP::Handler end rescue SystemExit,NoMemoryError raise + rescue HTTPError => e + return do_exception(response, e.message, e.status) rescue Exception => e return do_exception(response, e) ensure @@ -111,9 +120,13 @@ module Puppet::Network::HTTP::Handler # for authorization issues status = 403 if status == 400 end + if exception.is_a?(Exception) Puppet.log_exception(exception) + else + Puppet.notice(exception.to_s) end + set_content_type(response, "text/plain") set_response(response, exception.to_s, status) end @@ -125,21 +138,18 @@ module Puppet::Network::HTTP::Handler # Execute our find. def do_find(indirection_name, key, params, request, response) - unless result = model(indirection_name).indirection.find(key, params) - Puppet.info("Could not find #{indirection_name} for '#{key}'") - return do_exception(response, "Could not find #{indirection_name} #{key}", 404) + model_class = model(indirection_name) + unless result = model_class.indirection.find(key, params) + raise HTTPNotFoundError, "Could not find #{indirection_name} #{key}" end - # The encoding of the result must include the format to use, - # and it needs to be used for both the rendering and as - # the content type. - format = format_to_use(request) + format = accepted_response_formatter_for(model_class, request) set_content_type(response, format) rendered_result = result if result.respond_to?(:render) Puppet::Util::Profiler.profile("Rendered result in #{format}") do - rendered_result = result.render(format) + rendered_result = result.render(format) end end @@ -151,8 +161,7 @@ module Puppet::Network::HTTP::Handler # Execute our head. def do_head(indirection_name, key, params, request, response) unless self.model(indirection_name).indirection.head(key, params) - Puppet.info("Could not find #{indirection_name} for '#{key}'") - return do_exception(response, "Could not find #{indirection_name} #{key}", 404) + raise HTTPNotFoundError, "Could not find #{indirection_name} #{key}" end # No need to set a response because no response is expected from a @@ -165,10 +174,10 @@ module Puppet::Network::HTTP::Handler result = model.indirection.search(key, params) if result.nil? - return do_exception(response, "Could not find instances in #{indirection_name} with '#{key}'", 404) + raise HTTPNotFoundError, "Could not find instances in #{indirection_name} with '#{key}'" end - format = format_to_use(request) + format = accepted_response_formatter_for(model, request) set_content_type(response, format) set_response(response, model.render_multiple(format, result)) @@ -176,20 +185,25 @@ module Puppet::Network::HTTP::Handler # Execute our destroy. def do_destroy(indirection_name, key, params, request, response) - result = model(indirection_name).indirection.destroy(key, params) + model_class = model(indirection_name) + formatter = accepted_response_formatter_or_yaml_for(model_class, request) - return_yaml_response(response, result) + result = model_class.indirection.destroy(key, params) + + set_content_type(response, formatter) + set_response(response, formatter.render(result)) end # Execute our save. def do_save(indirection_name, key, params, request, response) - data = body(request).to_s - raise ArgumentError, "No data to save" if !data or data.empty? + model_class = model(indirection_name) + formatter = accepted_response_formatter_or_yaml_for(model_class, request) + sent_object = read_body_into_model(model_class, request) - format = request_format(request) - obj = model(indirection_name).convert_from(format, data) - result = model(indirection_name).indirection.save(obj, key) - return_yaml_response(response, result) + result = model_class.indirection.save(sent_object, key) + + set_content_type(response, formatter) + set_response(response, formatter.render(result)) end # resolve node name from peer's ip address @@ -205,9 +219,41 @@ module Puppet::Network::HTTP::Handler private - def return_yaml_response(response, body) - set_content_type(response, Puppet::Network::FormatHandler.format("yaml")) - set_response(response, body.to_yaml) + def report_if_deprecated(format) + if format.name == :yaml || format.name == :b64_zlib_yaml + Puppet.deprecation_warning("YAML in network requests is deprecated and will be removed in a future version. See http://links.puppetlabs.com/deprecate_yaml_on_network") + end + end + + def accepted_response_formatter_for(model_class, request) + accepted_formats = accept_header(request) or raise HTTPNotAcceptableError, "Missing required Accept header" + response_formatter_for(model_class, request, accepted_formats) + end + + def accepted_response_formatter_or_yaml_for(model_class, request) + accepted_formats = accept_header(request) || "yaml" + response_formatter_for(model_class, request, accepted_formats) + end + + def response_formatter_for(model_class, request, accepted_formats) + formatter = Puppet::Network::FormatHandler.most_suitable_format_for( + accepted_formats.split(/\s*,\s*/), + model_class.supported_formats) + + if formatter.nil? + raise HTTPNotAcceptableError, "No supported formats are acceptable (Accept: #{accepted_formats})" + end + + report_if_deprecated(formatter) + formatter + end + + def read_body_into_model(model_class, request) + data = body(request).to_s + raise ArgumentError, "No data to save" if !data or data.empty? + + format = request_format(request) + model_class.convert_from(format, data) end def get?(request) @@ -253,30 +299,45 @@ module Puppet::Network::HTTP::Handler end def decode_params(params) - params.inject({}) do |result, ary| + params.select { |key, _| allowed_parameter?(key) }.inject({}) do |result, ary| param, value = ary - next result if param.nil? || param.empty? - - param = param.to_sym - - # These shouldn't be allowed to be set by clients - # in the query string, for security reasons. - next result if param == :node - next result if param == :ip - value = CGI.unescape(value) - if value =~ /^---/ - value = YAML.load(value, :safe => true, :deserialize_symbols => true) - else - value = true if value == "true" - value = false if value == "false" - value = Integer(value) if value =~ /^\d+$/ - value = value.to_f if value =~ /^\d+\.\d+$/ - end - result[param] = value + result[param.to_sym] = parse_parameter_value(param, value) result end end + def allowed_parameter?(name) + not (name.nil? || name.empty? || DISALLOWED_KEYS.include?(name)) + end + + def parse_parameter_value(param, value) + case value + when /^---/ + Puppet.debug("Found YAML while processing request parameter #{param} (value: <#{value}>)") + Puppet.deprecation_warning("YAML in network requests is deprecated and will be removed in a future version. See http://links.puppetlabs.com/deprecate_yaml_on_network") + YAML.load(value, :safe => true, :deserialize_symbols => true) + when Array + value.collect { |v| parse_primitive_parameter_value(v) } + else + parse_primitive_parameter_value(value) + end + end + + def parse_primitive_parameter_value(value) + case value + when "true" + true + when "false" + false + when /^\d+$/ + Integer(value) + when /^\d+\.\d+$/ + value.to_f + else + value + end + end + def configure_profiler(request_headers, request_params) if (request_headers.has_key?(Puppet::Network::HTTP::HEADER_ENABLE_PROFILING.downcase) or Puppet[:profile]) Puppet::Util::Profiler.current = Puppet::Util::Profiler::WallClock.new(Puppet.method(:debug), request_params.object_id) diff --git a/lib/puppet/network/http/rack/rest.rb b/lib/puppet/network/http/rack/rest.rb index 5a830674c..b2cfa078d 100644 --- a/lib/puppet/network/http/rack/rest.rb +++ b/lib/puppet/network/http/rack/rest.rb @@ -51,7 +51,7 @@ class Puppet::Network::HTTP::RackREST < Puppet::Network::HTTP::RackHttpHandler # Retrieve all headers from the http request, as a map. def headers(request) request.env.select {|k,v| k.start_with? 'HTTP_'}.inject({}) do |m, (k,v)| - m[k.sub(/^HTTP_/, '').downcase] = v + m[k.sub(/^HTTP_/, '').gsub('_','-').downcase] = v m end end diff --git a/lib/puppet/network/http/webrick/rest.rb b/lib/puppet/network/http/webrick/rest.rb index 72d3905be..0ed3db36c 100644 --- a/lib/puppet/network/http/webrick/rest.rb +++ b/lib/puppet/network/http/webrick/rest.rb @@ -15,9 +15,15 @@ class Puppet::Network::HTTP::WEBrickREST < WEBrick::HTTPServlet::AbstractServlet # Retrieve the request parameters, including authentication information. def params(request) - result = request.query - result = decode_params(result) - result.merge(client_information(request)) + params = request.query || {} + + params = Hash[params.collect do |key, value| + all_values = value.list + [key, all_values.length == 1 ? value : all_values] + end] + + params = decode_params(params) + params.merge(client_information(request)) end # WEBrick uses a service method to respond to requests. Simply delegate to the handler response method. diff --git a/lib/puppet/network/http_pool.rb b/lib/puppet/network/http_pool.rb index b96c561eb..f037f318e 100644 --- a/lib/puppet/network/http_pool.rb +++ b/lib/puppet/network/http_pool.rb @@ -12,8 +12,8 @@ module Puppet::Network::HttpPool # Retrieve a cached http instance if caching is enabled, else return # a new one. - def self.http_instance(host, port, use_ssl = true) - Puppet::Network::HTTP::Connection.new(host, port, use_ssl) + def self.http_instance(host, port, use_ssl = true, verify_peer = true) + Puppet::Network::HTTP::Connection.new(host, port, :use_ssl => use_ssl, :verify_peer => verify_peer) end end diff --git a/lib/puppet/network/resolver.rb b/lib/puppet/network/resolver.rb index 0e0460eb1..98554eb8b 100644 --- a/lib/puppet/network/resolver.rb +++ b/lib/puppet/network/resolver.rb @@ -18,6 +18,7 @@ module Puppet::Network::Resolver when :ca then service = '_x-puppet-ca' when :report then service = '_x-puppet-report' when :file then service = '_x-puppet-fileserver' + else service = "_x-puppet-#{service_name.to_s}" end srv_record = "#{service}._tcp.#{domain}" diff --git a/lib/puppet/network/server.rb b/lib/puppet/network/server.rb index c58e3608a..a5cd1c7e5 100644 --- a/lib/puppet/network/server.rb +++ b/lib/puppet/network/server.rb @@ -1,114 +1,38 @@ require 'puppet/network/http' -require 'puppet/util/pidlock' require 'puppet/network/http/webrick' +# +# @api private class Puppet::Network::Server attr_reader :address, :port - # TODO: does anything actually call this? It seems like it's a duplicate of - # the code in Puppet::Daemon, but that it's not actually called anywhere. - - # Put the daemon into the background. - def daemonize - if pid = fork - Process.detach(pid) - exit(0) - end - - # Get rid of console logging - Puppet::Util::Log.close(:console) - - Process.setsid - Dir.chdir("/") - end - - def close_streams() - Puppet::Daemon.close_streams() - end - - # Create a pidfile for our daemon, so we can be stopped and others - # don't try to start. - def create_pidfile - Puppet::Util.synchronize_on(Puppet.run_mode.name,Sync::EX) do - raise "Could not create PID file: #{pidfile}" unless Puppet::Util::Pidlock.new(pidfile).lock - end - end - - # Remove the pid file for our daemon. - def remove_pidfile - Puppet::Util.synchronize_on(Puppet.run_mode.name,Sync::EX) do - Puppet::Util::Pidlock.new(pidfile).unlock - end - end - - # Provide the path to our pidfile. - def pidfile - Puppet[:pidfile] - end - - def initialize(address, port, handlers = nil) + def initialize(address, port) @port = port @address = address @http_server = Puppet::Network::HTTP::WEBrick.new @listening = false - @routes = {} - self.register(handlers) if handlers # Make sure we have all of the directories we need to function. Puppet.settings.use(:main, :ssl, :application) end - # Register handlers for REST networking, based on the Indirector. - def register(*indirections) - raise ArgumentError, "Indirection names are required." if indirections.empty? - indirections.flatten.each do |name| - Puppet::Indirector::Indirection.model(name) || raise(ArgumentError, "Cannot locate indirection '#{name}'.") - @routes[name.to_sym] = true - end - end - - # Unregister Indirector handlers. - def unregister(*indirections) - raise "Cannot unregister indirections while server is listening." if listening? - indirections = @routes.keys if indirections.empty? - - indirections.flatten.each do |i| - raise(ArgumentError, "Indirection [#{i}] is unknown.") unless @routes[i.to_sym] - end - - indirections.flatten.each do |i| - @routes.delete(i.to_sym) - end - end - def listening? @listening end - def listen + def start raise "Cannot listen -- already listening." if listening? @listening = true @http_server.listen(address, port) end - def unlisten + def stop raise "Cannot unlisten -- not currently listening." unless listening? @http_server.unlisten @listening = false end - def start - create_pidfile - close_streams if Puppet[:daemonize] - listen - end - - def stop - unlisten - remove_pidfile - end - def wait_for_shutdown @http_server.wait_for_shutdown end diff --git a/lib/puppet/node/environment.rb b/lib/puppet/node/environment.rb index 087cd5ccc..25b454638 100644 --- a/lib/puppet/node/environment.rb +++ b/lib/puppet/node/environment.rb @@ -7,10 +7,61 @@ require 'puppet/parser/parser_factory' class Puppet::Node end -# Model the environment that a node can operate in. This class just -# provides a simple wrapper for the functionality around environments. +# Puppet::Node::Environment acts as a container for all configuration +# that is expected to vary between environments. +# +# ## Thread local variables +# +# The Puppet::Node::Environment uses a number of `Thread.current` variables. +# Since all web servers that Puppet runs on are single threaded these +# variables are effectively global. +# +# ### `Thread.current[:environment]` +# +# The 'environment' thread variable represents the current environment that's +# being used in the compiler. +# +# ### `Thread.current[:known_resource_types]` +# +# The 'known_resource_types' thread variable represents a singleton instance +# of the Puppet::Resource::TypeCollection class. The variable is discarded +# and regenerated if it is accessed by an environment that doesn't match the +# environment of the 'known_resource_types' +# +# This behavior of discarding the known_resource_types every time the +# environment changes is not ideal. In the best case this can cause valid data +# to be discarded and reloaded. If Puppet is being used with numerous +# environments then this penalty will be repeatedly incurred. +# +# In the worst case (#15106) demonstrates that if a different environment is +# accessed during catalog compilation, for whatever reason, the +# known_resource_types can be discarded which loses information that cannot +# be recovered and can cause a catalog compilation to completely fail. +# +# ## The root environment +# +# In addition to normal environments that are defined by the user,there is a +# special 'root' environment. It is defined as an instance variable on the +# Puppet::Node::Environment metaclass. The environment name is `*root*` and can +# be accessed by calling {Puppet::Node::Environment.root}. +# +# The primary purpose of the root environment is to contain parser functions +# that are not bound to a specific environment. The main case for this is for +# logging functions. Logging functions are attached to the 'root' environment +# when {Puppet::Parser::Functions.reset} is called. +# +# The root environment is also used as a fallback environment when the +# current environment has been requested by {Puppet::Node::Environment.current} +# requested and no environment was set by {Puppet::Node::Environment.current=} class Puppet::Node::Environment + + # This defines a mixin for classes that have an environment. It implements + # `environment` and `environment=` that respects the semantics of the + # Puppet::Node::Environment class + # + # @api public module Helper + def environment Puppet::Node::Environment.new(@environment) end @@ -26,9 +77,34 @@ class Puppet::Node::Environment include Puppet::Util::Cacher + # @!attribute seen + # @scope class + # @api private + # @return [Hash<Symbol, Puppet::Node::Environment>] All memoized environments @seen = {} - # Return an existing environment instance, or create a new one. + # Create a new environment with the given name, or return an existing one + # + # The environment class memoizes instances so that attempts to instantiate an + # environment with the same name with an existing environment will return the + # existing environment. + # + # @overload self.new(environment) + # @param environment [Puppet::Node::Environment] + # @return [Puppet::Node::Environment] the environment passed as the param, + # this is implemented so that a calling class can use strings or + # environments interchangeably. + # + # @overload self.new(string) + # @param string [String, Symbol] + # @return [Puppet::Node::Environment] An existing environment if it exists, + # else a new environment with that name + # + # @overload self.new() + # @return [Puppet::Node::Environment] The environment as set by + # Puppet.settings[:environment] + # + # @api public def self.new(name = nil) return name if name.is_a?(self) name ||= Puppet.settings.value(:environment) @@ -44,35 +120,92 @@ class Puppet::Node::Environment @seen[symbol] = obj end + # Retrieve the environment for the current thread + # + # @note This should only used when a catalog is being compiled. + # + # @api private + # + # @return [Puppet::Node::Environment] the currently set environment if one + # has been explicitly set, else it will return the '*root*' environment def self.current Thread.current[:environment] || root end + # Set the environment for the current thread + # + # @note This should only set when a catalog is being compiled. Under normal + # This value is initially set in {Puppet::Parser::Compiler#environment} + # + # @note Setting this affects global state during catalog compilation, and + # changing the current environment during compilation can cause unexpected + # and generally very bad behaviors. + # + # @api private + # + # @param env [Puppet::Node::Environment] def self.current=(env) Thread.current[:environment] = new(env) end + + # @return [Puppet::Node::Environment] The `*root*` environment. + # + # This is only used for handling functions that are not attached to a + # specific environment. + # + # @api private def self.root @root end + # Clear all memoized environments and the 'current' environment + # + # @api private def self.clear @seen.clear Thread.current[:environment] = nil end + # @!attribute [r] name + # @api public + # @return [Symbol] the human readable environment name that serves as the + # environment identifier attr_reader :name - # Return an environment-specific setting. + # Return an environment-specific Puppet setting. + # + # @api public + # + # @param param [String, Symbol] The environment setting to look up + # @return [Object] The resolved setting value def [](param) Puppet.settings.value(param, self.name) end + # Instantiate a new environment + # + # @note {Puppet::Node::Environment.new} is overridden to return memoized + # objects, so this will not be invoked with the normal Ruby initialization + # semantics. + # + # @param name [Symbol] The environment name def initialize(name) @name = name extend MonitorMixin end + # The current global TypeCollection + # + # @note The environment is loosely coupled with the {Puppet::Resource::TypeCollection} + # class. While there is a 1:1 relationship between an environment and a + # TypeCollection instance, there is only one TypeCollection instance available + # at any given time. It is stored in the Thread.current collection as + # 'known_resource_types'. 'known_resource_types' is accessed as an instance + # method, but is global to all environment variables. + # + # @api public + # @return [Puppet::Resource::TypeCollection] The current global TypeCollection def known_resource_types # This makes use of short circuit evaluation to get the right thread-safe # per environment semantics with an efficient most common cases; we almost @@ -89,7 +222,8 @@ class Puppet::Node::Environment } end - # Yields each modules' plugin directory. + # Yields each modules' plugin directory if the plugin directory (modulename/lib) + # is present on the filesystem. # # @yield [String] Yields the plugin directory from each module to the block. # @api public @@ -100,10 +234,22 @@ class Puppet::Node::Environment end end + # Locate a module instance by the module name alone. + # + # @api public + # + # @param name [String] The module name + # @return [Puppet::Module, nil] The module if found, else nil def module(name) modules.find {|mod| mod.name == name} end + # Locate a module instance by the full forge name (EG authorname/module) + # + # @api public + # + # @param forge_name [String] The module name + # @return [Puppet::Module, nil] The module if found, else nil def module_by_forge_name(forge_name) author, modname = forge_name.split('/') found_mod = self.module(modname) @@ -112,17 +258,34 @@ class Puppet::Node::Environment nil end - # Cache the modulepath, so that we aren't searching through - # all known directories all the time. + # @!attribute [r] modulepath + # Return all existent directories in the modulepath for this environment + # @note This value is cached so that the filesystem doesn't have to be + # re-enumerated every time this method is invoked, since that + # enumeration could be a costly operation and this method is called + # frequently. The cache expiry is determined by `Puppet[:filetimeout]`. + # @see Puppet::Util::Cacher.cached_attr + # @api public + # @return [Array<String>] All directories present in the modulepath cached_attr(:modulepath, Puppet[:filetimeout]) do dirs = self[:modulepath].split(File::PATH_SEPARATOR) dirs = ENV["PUPPETLIB"].split(File::PATH_SEPARATOR) + dirs if ENV["PUPPETLIB"] validate_dirs(dirs) end - # Return all modules from this environment, in the order they appear - # in the modulepath - # Cache the list, because it can be expensive to create. + # @!attribute [r] modules + # Return all modules for this environment in the order they appear in the + # modulepath. + # @note If multiple modules with the same name are present they will + # both be added, but methods like {#module} and {#module_by_forge_name} + # will return the first matching entry in this list. + # @note This value is cached so that the filesystem doesn't have to be + # re-enumerated every time this method is invoked, since that + # enumeration could be a costly operation and this method is called + # frequently. The cache expiry is determined by `Puppet[:filetimeout]`. + # @see Puppet::Util::Cacher.cached_attr + # @api public + # @return [Array<Puppet::Module>] All modules for this environment cached_attr(:modules, Puppet[:filetimeout]) do module_references = [] seen_modules = {} @@ -140,12 +303,18 @@ class Puppet::Node::Environment module_references.collect do |reference| begin Puppet::Module.new(reference[:name], reference[:path], self) - rescue Puppet::Module::Error => e + rescue Puppet::Module::Error nil end end.compact end + # Generate a warning if the given directory in a module path entry is named `lib`. + # + # @api private + # + # @param path [String] The module directory containing the given directory + # @param name [String] The directory name def warn_about_mistaken_path(path, name) if name == "lib" Puppet.debug("Warning: Found directory named 'lib' in module path ('#{path}/lib'); unless " + @@ -155,6 +324,14 @@ class Puppet::Node::Environment end # Modules broken out by directory in the modulepath + # + # @note This method _changes_ the current working directory while enumerating + # the modules. This seems rather dangerous. + # + # @api public + # + # @return [Hash<String, Array<Puppet::Module>>] A hash whose keys are file + # paths, and whose values is an array of Puppet Modules for that path def modules_by_path modules_by_path = {} modulepath.each do |path| @@ -170,6 +347,33 @@ class Puppet::Node::Environment modules_by_path end + # All module requirements for all modules in the environment modulepath + # + # @api public + # + # @comment This has nothing to do with an environment. It seems like it was + # stuffed into the first convenient class that vaguely involved modules. + # + # @example + # environment.module_requirements + # # => { + # # 'username/amodule' => [ + # # { + # # 'name' => 'username/moduledep', + # # 'version' => '1.2.3', + # # 'version_requirement' => '>= 1.0.0', + # # }, + # # { + # # 'name' => 'username/anotherdep', + # # 'version' => '4.5.6', + # # 'version_requirement' => '>= 3.0.0', + # # } + # # ] + # # } + # # + # + # @return [Hash<String, Array<Hash<String, String>>>] See the method example + # for an explanation of the return value. def module_requirements deps = {} modules.each do |mod| @@ -191,21 +395,40 @@ class Puppet::Node::Environment deps end + # @return [String] The stringified value of the `name` instance variable + # @api public def to_s name.to_s end + # @return [Symbol] The `name` value, cast to a string, then cast to a symbol. + # + # @api public + # + # @note the `name` instance variable is a Symbol, but this casts the value + # to a String and then converts it back into a Symbol which will needlessly + # create an object that needs to be garbage collected def to_sym to_s.to_sym end + # Return only the environment name when serializing. + # # The only thing we care about when serializing an environment is its # identity; everything else is ephemeral and should not be stored or # transmitted. + # + # @api public def to_zaml(z) self.to_s.to_zaml(z) end + # Validate a list of file paths and return the paths that are directories on the filesystem + # + # @api private + # + # @param dirs [Array<String>] The file paths to validate + # @return [Array<String>] All file paths that exist and are directories def validate_dirs(dirs) dirs.collect do |dir| File.expand_path(dir) @@ -216,6 +439,22 @@ class Puppet::Node::Environment private + # Reparse the manifests for the given environment + # + # There are two sources that can be used for the initial parse: + # + # 1. The value of `Puppet.settings[:code]`: Puppet can take a string from + # its settings and parse that as a manifest. This is used by various + # Puppet applications to read in a manifest and pass it to the + # environment as a side effect. This is attempted first. + # 2. The contents of `Puppet.settings[:manifest]`: Puppet will try to load + # the environment manifest. By default this is `$manifestdir/site.pp` + # + # @note This method will return an empty hostclass if + # `Puppet.settings[:ignoreimport]` is set to true. + # + # @return [Puppet::Parser::AST::Hostclass] The AST hostclass object + # representing the 'main' hostclass def perform_initial_import return empty_parse_result if Puppet.settings[:ignoreimport] # parser = Puppet::Parser::Parser.new(self) @@ -236,9 +475,13 @@ class Puppet::Node::Environment raise error end + # Return an empty toplevel hostclass to indicate that no file was loaded + # + # This is used as the return value of {#perform_initial_import} when + # `Puppet.settings[:ignoreimport]` is true. + # + # @return [Puppet::Parser::AST::Hostclass] def empty_parse_result - # Return an empty toplevel hostclass to use as the result of - # perform_initial_import when no file was actually loaded. return Puppet::Parser::AST::Hostclass.new('') end diff --git a/lib/puppet/node/facts.rb b/lib/puppet/node/facts.rb index fdfbd045d..b18a1e88d 100755 --- a/lib/puppet/node/facts.rb +++ b/lib/puppet/node/facts.rb @@ -28,6 +28,7 @@ class Puppet::Node::Facts def add_local_facts values["clientcert"] = Puppet.settings[:certname] values["clientversion"] = Puppet.version.to_s + values["clientnoop"] = Puppet.settings[:noop] end def initialize(name, values = {}) @@ -64,6 +65,14 @@ class Puppet::Node::Facts end end + # Sanitize fact values by converting everything not a string, boolean + # numeric, array or hash into strings. + def sanitize + values.each do |fact, value| + values[fact] = sanitize_fact value + end + end + def ==(other) return false unless self.name == other.name strip_internal == other.send(:strip_internal) @@ -93,11 +102,11 @@ class Puppet::Node::Facts end def timestamp=(time) - self.values[:_timestamp] = time + self.values['_timestamp'] = time end def timestamp - self.values[:_timestamp] + self.values['_timestamp'] end private @@ -108,4 +117,21 @@ class Puppet::Node::Facts newvals.find_all { |name, value| name.to_s =~ /^_/ }.each { |name, value| newvals.delete(name) } newvals end + + def sanitize_fact(fact) + if fact.is_a? Hash then + ret = {} + fact.each_pair { |k,v| ret[sanitize_fact k]=sanitize_fact v } + ret + elsif fact.is_a? Array then + fact.collect { |i| sanitize_fact i } + elsif fact.is_a? Numeric \ + or fact.is_a? TrueClass \ + or fact.is_a? FalseClass \ + or fact.is_a? String + fact + else + fact.to_s + end + end end diff --git a/lib/puppet/parameter.rb b/lib/puppet/parameter.rb index fe26c17c0..cce9fb57e 100644 --- a/lib/puppet/parameter.rb +++ b/lib/puppet/parameter.rb @@ -1,5 +1,4 @@ require 'puppet/util/methodhelper' -require 'puppet/util/log_paths' require 'puppet/util/logging' require 'puppet/util/docs' @@ -22,7 +21,6 @@ require 'puppet/util/docs' class Puppet::Parameter include Puppet::Util include Puppet::Util::Errors - include Puppet::Util::LogPaths include Puppet::Util::Logging include Puppet::Util::MethodHelper @@ -241,10 +239,11 @@ class Puppet::Parameter end # @overload validate {|| ... } - # Defines an optional method that is used to validate the parameter's value. + # Defines an optional method that is used to validate the parameter's DSL/string value. # Validation should raise appropriate exceptions, the return value of the given block is ignored. # The easiest way to raise an appropriate exception is to call the method {Puppet::Util::Errors.fail} with # the message as an argument. + # To validate the munged value instead, just munge the value (`munge(value)`). # # @return [void] # @dsl type @@ -306,19 +305,29 @@ class Puppet::Parameter # attr_accessor :parent - # @!method line() - # @return [Integer] Returns the result of calling the same method on the associated resource. - # @!method file - # @return [Integer] Returns the result of calling the same method on the associated resource. - # @!method version - # @return [Integer] Returns the result of calling the same method on the associated resource. - # - [:line, :file, :version].each do |param| - define_method(param) do - resource.send(param) - end + # Returns a string representation of the resource's containment path in + # the catalog. + # @return [String] + def path + @path ||= '/' + pathbuilder.join('/') end + # @return [Integer] Returns the result of calling the same method on the associated resource. + def line + resource.line + end + + # @return [Integer] Returns the result of calling the same method on the associated resource. + def file + resource.file + end + + # @return [Integer] Returns the result of calling the same method on the associated resource. + def version + resource.version + end + + # Initializes the parameter with a required resource reference and optional attribute settings. # The option `:resource` must be specified or an exception is raised. Any additional options passed # are used to initialize the attributes of this parameter by treating each key in the `options` hash as @@ -377,9 +386,9 @@ class Puppet::Parameter tmp end - # @todo Original comment = _return the full path to us, for logging and rollback; not currently - # used_ This is difficult to figure out (if it is used or not as calls are certainly made to "pathbuilder" - # method is several places, not just sure if it is this implementation or not. + # Returns an array of strings representing the containment heirarchy + # (types/classes) that make up the path to the resource from the root + # of the catalog. This is mostly used for logging purposes. # # @api private def pathbuilder @@ -477,7 +486,7 @@ class Puppet::Parameter # @todo This original comment _"All of the checking should possibly be # late-binding (e.g., users might not exist when the value is assigned # but might when it is asked for)."_ does not seem to be correct, the implementation - # calls both validate an munge on the given value, so no late binding. + # calls both validate and munge on the given value, so no late binding. # # The given value is validated and then munged (if munging has been specified). The result is store # as the value of this arameter. diff --git a/lib/puppet/parameter/boolean.rb b/lib/puppet/parameter/boolean.rb new file mode 100644 index 000000000..86f0c05dd --- /dev/null +++ b/lib/puppet/parameter/boolean.rb @@ -0,0 +1,10 @@ +require 'puppet/coercion' + +# This specialized {Puppet::Parameter} handles boolean options, accepting lots +# of strings and symbols for both truthiness and falsehood. +# +class Puppet::Parameter::Boolean < Puppet::Parameter + def unsafe_munge(value) + Puppet::Coercion.boolean(value) + end +end diff --git a/lib/puppet/parameter/path.rb b/lib/puppet/parameter/path.rb index e82a16701..d5fb29e6a 100644 --- a/lib/puppet/parameter/path.rb +++ b/lib/puppet/parameter/path.rb @@ -45,7 +45,7 @@ class Puppet::Parameter::Path < Puppet::Parameter # This default implementation does not perform any munging, it just checks the one/many paths # constraints. A derived implementation can perform this check as: # `paths.is_a?(Array) and ! self.class.arrays?` and raise a {Puppet::Error}. - # @param [String, Array<String>] one of multiple paths + # @param paths [String, Array<String>] one of multiple paths # @return [String, Array<String>] the given paths # @raise [Puppet::Error] if the given paths does not comply with the on/many paths rule. def unsafe_munge(paths) diff --git a/lib/puppet/parameter/value.rb b/lib/puppet/parameter/value.rb index 35921fc31..dc4de2ae5 100644 --- a/lib/puppet/parameter/value.rb +++ b/lib/puppet/parameter/value.rb @@ -34,7 +34,7 @@ class Puppet::Parameter::Value # Initializes the instance with a literal accepted value, or a regular expression. # If anything else is passed, it is turned into a String, and then made into a Symbol. - # @param [Symbol, Regexp, Object] the value to accept, Symbol, a regular expression, or object to convert. + # @param name [Symbol, Regexp, Object] the value to accept, Symbol, a regular expression, or object to convert. # @api private # def initialize(name) diff --git a/lib/puppet/parameter/value_collection.rb b/lib/puppet/parameter/value_collection.rb index 144f8ba74..7e2bee331 100644 --- a/lib/puppet/parameter/value_collection.rb +++ b/lib/puppet/parameter/value_collection.rb @@ -189,7 +189,7 @@ class Puppet::Parameter::ValueCollection end # Returns a valid value matcher (a literal or regular expression) - # @todo This looks odd, asking for an instance that matches a symbol, or a instance that has + # @todo This looks odd, asking for an instance that matches a symbol, or an instance that has # a regexp. What is the intention here? Marking as api private... # # @return [Puppet::Parameter::Value] a valid valud matcher diff --git a/lib/puppet/parser/ast/arithmetic_operator.rb b/lib/puppet/parser/ast/arithmetic_operator.rb index 9b4fe769d..162d263a7 100644 --- a/lib/puppet/parser/ast/arithmetic_operator.rb +++ b/lib/puppet/parser/ast/arithmetic_operator.rb @@ -80,4 +80,12 @@ class Puppet::Parser::AST raise ArgumentError, "Invalid arithmetic operator #{@operator}" unless %w{+ - * / % << >>}.include?(@operator) end end + + # Used by future parser instead of ArithmeticOperator to enable concatenation + class ArithmeticOperator2 < ArithmeticOperator + # Overrides the arithmetic operator to allow concatenation + def assert_concatenation_supported + end + end + end diff --git a/lib/puppet/parser/ast/casestatement.rb b/lib/puppet/parser/ast/casestatement.rb index 8370d11f3..67510ca72 100644 --- a/lib/puppet/parser/ast/casestatement.rb +++ b/lib/puppet/parser/ast/casestatement.rb @@ -15,9 +15,6 @@ class Puppet::Parser::AST value = @test.safeevaluate(scope) - retvalue = nil - found = false - # Iterate across the options looking for a match. default = nil @options.each do |option| diff --git a/lib/puppet/parser/ast/lambda.rb b/lib/puppet/parser/ast/lambda.rb index 00c0e860a..976f817cd 100644 --- a/lib/puppet/parser/ast/lambda.rb +++ b/lib/puppet/parser/ast/lambda.rb @@ -36,31 +36,50 @@ class Puppet::Parser::AST # def call(scope, *args) raise Puppet::ParseError, "Too many arguments: #{args.size} for #{parameters.size}" unless args.size <= parameters.size + + # associate values with parameters merged = parameters.zip(args) - missing = merged.select { |e| !e[1] && e[0].size == 1 } + # calculate missing arguments + missing = parameters.slice(args.size, parameters.size - args.size).select {|e| e.size == 1} unless missing.empty? optional = parameters.count { |p| p.size == 2 } raise Puppet::ParseError, "Too few arguments; #{args.size} for #{optional > 0 ? ' min ' : ''}#{parameters.size - optional}" end evaluated = merged.collect do |m| - # Ruby 1.8.7 zip seems to produce a different result than Ruby 1.9.3 in some situations - n = m[0].is_a?(Array) ? m[0][0] : m[0] - v = m[1] || (m[0][1]).safeevaluate(scope) # given value or default expression value - [n, v] + # m can be one of + # m = [["name"], "given"] + # | [["name", default_expr], "given"] + # + # "given" is always an optional entry. If a parameter was provided then + # the entry will be in the array, otherwise the m array will be a + # single element. + given_argument = m[1] + argument_name = m[0][0] + default_expression = m[0][1] + + value = if m.size == 1 + default_expression.safeevaluate(scope) + else + given_argument + end + [argument_name, value] end # Store the evaluated name => value associations in a new inner/local/ephemeral scope # (This is made complicated due to the fact that the implementation of scope is overloaded with # functionality and an inner ephemeral scope must be used (as opposed to just pushing a local scope # on a scope "stack"). + + # Ensure variable exists with nil value if error occurs. + # Some ruby implementations does not like creating variable on return + result = nil begin elevel = scope.ephemeral_level scope.ephemeral_from(Hash[evaluated], file, line) result = safeevaluate(scope) ensure scope.unset_ephemeral_var(elevel) - result ||= nil end result end diff --git a/lib/puppet/parser/ast/leaf.rb b/lib/puppet/parser/ast/leaf.rb index ac4d2c26a..af9f792cc 100644 --- a/lib/puppet/parser/ast/leaf.rb +++ b/lib/puppet/parser/ast/leaf.rb @@ -131,7 +131,13 @@ class Puppet::Parser::AST def evaluate_container(scope) container = variable.respond_to?(:evaluate) ? variable.safeevaluate(scope) : variable - (container.is_a?(Hash) or container.is_a?(Array)) ? container : scope[container, {:file => file, :line => line}] + if container.is_a?(Hash) || container.is_a?(Array) + container + elsif container.is_a?(::String) + scope[container, {:file => file, :line => line}] + else + raise Puppet::ParseError, "#{variable} is #{container.inspect}, not a hash or array" + end end def evaluate_key(scope) @@ -148,9 +154,10 @@ class Puppet::Parser::AST def evaluate(scope) object = evaluate_container(scope) accesskey = evaluate_key(scope) - raise Puppet::ParseError, "#{variable} is not an hash or array when accessing it with #{accesskey}" unless object.is_a?(Hash) or object.is_a?(Array) + raise Puppet::ParseError, "#{variable} is not a hash or array when accessing it with #{accesskey}" unless object.is_a?(Hash) or object.is_a?(Array) - object[array_index_or_key(object, accesskey)] || :undef + result = object[array_index_or_key(object, accesskey)] + result.nil? ? :undef : result end # Assign value to this hashkey or array index diff --git a/lib/puppet/parser/ast/nop.rb b/lib/puppet/parser/ast/nop.rb index bf35c6a5c..15383aa1f 100644 --- a/lib/puppet/parser/ast/nop.rb +++ b/lib/puppet/parser/ast/nop.rb @@ -2,7 +2,7 @@ require 'puppet/parser/ast/branch' class Puppet::Parser::AST # This class is a no-op, it doesn't produce anything - # when evaluated, hence it's name :-) + # when evaluated, hence its name :-) class Nop < AST::Leaf def evaluate(scope) # nothing to do diff --git a/lib/puppet/parser/ast/resource_override.rb b/lib/puppet/parser/ast/resource_override.rb index 0e922eafa..78fe74171 100644 --- a/lib/puppet/parser/ast/resource_override.rb +++ b/lib/puppet/parser/ast/resource_override.rb @@ -23,8 +23,6 @@ class Puppet::Parser::AST # Get our object reference. resource = @object.safeevaluate(scope) - hash = {} - # Evaluate all of the specified params. params = @parameters.collect { |param| param.safeevaluate(scope) diff --git a/lib/puppet/parser/compiler.rb b/lib/puppet/parser/compiler.rb index cd80ac663..20f668504 100644 --- a/lib/puppet/parser/compiler.rb +++ b/lib/puppet/parser/compiler.rb @@ -35,6 +35,22 @@ class Puppet::Parser::Compiler attr_reader :node, :facts, :collections, :catalog, :resources, :relationships, :topscope + # The injector that provides lookup services, or nil if accessed before the compiler has started compiling and + # bootstrapped. The injector is initialized and available before any manifests are evaluated. + # + # @return [Puppet::Pops::Binder::Injector, nil] The injector that provides lookup services for this compiler/environment + # @api public + # + attr_accessor :injector + + # The injector that provides lookup services during the creation of the {#injector}. + # @return [Puppet::Pops::Binder::Injector, nil] The injector that provides lookup services during injector creation + # for this compiler/environment + # + # @api private + # + attr_accessor :boot_injector + # Add a collection to the global list. def_delegator :@collections, :<<, :add_collection def_delegator :@relationships, :<<, :add_relationship @@ -51,7 +67,6 @@ class Puppet::Parser::Compiler end end - # Store a resource in our resource table. def add_resource(scope, resource) @resources << resource @@ -94,6 +109,10 @@ class Puppet::Parser::Compiler Puppet::Util::Profiler.profile("Compile: Created settings scope") { create_settings_scope } + if is_binder_active? + Puppet::Util::Profiler.profile("Compile: Created injector") { create_injector } + end + Puppet::Util::Profiler.profile("Compile: Evaluated main") { evaluate_main } Puppet::Util::Profiler.profile("Compile: Evaluated AST node") { evaluate_ast_node } @@ -135,7 +154,7 @@ class Puppet::Parser::Compiler # # Sometimes we evaluate classes with a fully qualified name already, in which # case, we tell scope.find_hostclass we've pre-qualified the name so it - # doesn't need to search it's namespaces again. This gets around a weird + # doesn't need to search its namespaces again. This gets around a weird # edge case of duplicate class names, one at top scope and one nested in our # namespace and the wrong one (or both!) getting selected. See ticket #13349 # for more detail. --jeffweiss 26 apr 2012 @@ -197,6 +216,47 @@ class Puppet::Parser::Compiler @resource_overrides[resource.ref] end + def injector + create_injector if @injector.nil? + @injector + end + + def boot_injector + create_boot_injector(nil) if @boot_injector.nil? + @boot_injector + end + + # Creates the boot injector from registered system, default, and injector config. + # @return [Puppet::Pops::Binder::Injector] the created boot injector + # @api private Cannot be 'private' since it is called from the BindingsComposer. + # + def create_boot_injector(env_boot_bindings) + assert_binder_active() + boot_contribution = Puppet::Pops::Binder::SystemBindings.injector_boot_contribution(env_boot_bindings) + final_contribution = Puppet::Pops::Binder::SystemBindings.final_contribution + binder = Puppet::Pops::Binder::Binder.new() + binder.define_categories(boot_contribution.effective_categories) + binder.define_layers(Puppet::Pops::Binder::BindingsFactory.layered_bindings(final_contribution, boot_contribution)) + @boot_injector = Puppet::Pops::Binder::Injector.new(binder) + end + + # Answers if Puppet Binder should be active or not, and if it should and is not active, then it is activated. + # @return [Boolean] true if the Puppet Binder should be activated + def is_binder_active? + should_be_active = Puppet[:binder] || Puppet[:parser] == 'future' + if should_be_active + # TODO: this should be in a central place, not just for ParserFactory anymore... + Puppet::Parser::ParserFactory.assert_rgen_installed() + @@binder_loaded ||= false + unless @@binder_loaded + require 'puppet/pops' + require 'puppetx' + @@binder_loaded = true + end + end + should_be_active + end + private # If ast nodes are enabled, then see if we can find and evaluate one. @@ -303,16 +363,11 @@ class Puppet::Parser::Compiler # not find the resource they were supposed to override, so we # want to throw an exception. def fail_on_unevaluated_overrides - remaining = [] - @resource_overrides.each do |name, overrides| - remaining.concat overrides - end + remaining = @resource_overrides.values.flatten.collect(&:ref) - unless remaining.empty? + if !remaining.empty? fail Puppet::ParseError, - "Could not find resource(s) %s for overriding" % remaining.collect { |o| - o.ref - }.join(", ") + "Could not find resource(s) #{remaining.join(', ')} for overriding" end end @@ -320,22 +375,11 @@ class Puppet::Parser::Compiler # look for resources, because we want to consider those to be # parse errors. def fail_on_unevaluated_resource_collections - remaining = [] - @collections.each do |coll| - # We're only interested in the 'resource' collections, - # which result from direct calls of 'realize'. Anything - # else is allowed not to return resources. - # Collect all of them, so we have a useful error. - if r = coll.resources - if r.is_a?(Array) - remaining += r - else - remaining << r - end - end - end + remaining = @collections.collect(&:resources).flatten.compact - raise Puppet::ParseError, "Failed to realize virtual resources #{remaining.join(', ')}" unless remaining.empty? + if !remaining.empty? + raise Puppet::ParseError, "Failed to realize virtual resources #{remaining.join(', ')}" + end end # Make sure all of our resources and such have done any last work @@ -366,10 +410,8 @@ class Puppet::Parser::Compiler raise "Couldn't find main" end - names = [] - Puppet::Type.eachmetaparam do |name| - next if Puppet::Parser::Resource.relationship_parameter?(name) - names << name + names = Puppet::Type.metaparams.select do |name| + !Puppet::Parser::Resource.relationship_parameter?(name) end data = {} @@ -405,9 +447,6 @@ class Puppet::Parser::Compiler # Set up all of our internal variables. def initvars - # The list of objects that will available for export. - @exported_resources = {} - # The list of overrides. This is used to cache overrides on objects # that don't exist yet. We store an array of each override. @resource_overrides = Hash.new do |overs, ref| @@ -430,8 +469,7 @@ class Puppet::Parser::Compiler # Create our initial scope and a resource that will evaluate main. @topscope = Puppet::Parser::Scope.new(self) - @main_stage_resource = Puppet::Parser::Resource.new("stage", :main, :scope => @topscope) - @catalog.add_resource(@main_stage_resource) + @catalog.add_resource(Puppet::Parser::Resource.new("stage", :main, :scope => @topscope)) # local resource array to maintain resource ordering @resources = [] @@ -481,4 +519,24 @@ class Puppet::Parser::Compiler # The order of these is significant for speed due to short-circuting resources.reject { |resource| resource.evaluated? or resource.virtual? or resource.builtin_type? } end + + # Creates the injector from bindings found in the current environment. + # @return [void] + # @api private + # + def create_injector + assert_binder_active() + composer = Puppet::Pops::Binder::BindingsComposer.new() + layered_bindings = composer.compose(topscope) + binder = Puppet::Pops::Binder::Binder.new() + binder.define_categories(composer.effective_categories(topscope)) + binder.define_layers(layered_bindings) + @injector = Puppet::Pops::Binder::Injector.new(binder) + end + + def assert_binder_active + unless is_binder_active? + raise ArgumentError, "The Puppet Binder is only available when either '--binder true' or '--parser future' is used" + end + end end diff --git a/lib/puppet/parser/files.rb b/lib/puppet/parser/files.rb index 04333906e..1d5b64966 100644 --- a/lib/puppet/parser/files.rb +++ b/lib/puppet/parser/files.rb @@ -1,10 +1,5 @@ require 'puppet/module' -#require 'puppet/parser/parser' -# This is a silly central module for finding -# different kinds of files while parsing. This code -# doesn't really belong in the Puppet::Module class, -# but it doesn't really belong anywhere else, either. module Puppet; module Parser; module Files module_function diff --git a/lib/puppet/parser/functions/create_resources.rb b/lib/puppet/parser/functions/create_resources.rb index 6c0f696a7..054b1f70c 100644 --- a/lib/puppet/parser/functions/create_resources.rb +++ b/lib/puppet/parser/functions/create_resources.rb @@ -45,52 +45,29 @@ Puppet::Parser::Functions::newfunction(:create_resources, :arity => -3, :doc => ENDHEREDOC raise ArgumentError, ("create_resources(): wrong number of arguments (#{args.length}; must be 2 or 3)") if args.length > 3 - # figure out what kind of resource we are - type_of_resource = nil - type_name = args[0].downcase - type_exported, type_virtual = false - if type_name.start_with? '@@' - type_name = type_name[2..-1] - type_exported = true - elsif type_name.start_with? '@' - type_name = type_name[1..-1] - type_virtual = true - end - if type_name == 'class' - type_of_resource = :class - else - if resource = Puppet::Type.type(type_name.to_sym) - type_of_resource = :type - elsif resource = find_definition(type_name.downcase) - type_of_resource = :define - else - raise ArgumentError, "could not create resource of unknown type #{type_name}" - end + type, instances, defaults = args + defaults ||= {} + + resource = Puppet::Parser::AST::Resource.new(:type => type.sub(/^@{1,2}/, '').downcase, :instances => + instances.collect do |title, params| + Puppet::Parser::AST::ResourceInstance.new( + :title => Puppet::Parser::AST::Leaf.new(:value => title), + :parameters => defaults.merge(params).collect do |name, value| + Puppet::Parser::AST::ResourceParam.new( + :param => name, + :value => Puppet::Parser::AST::Leaf.new(:value => value)) + end) + end) + + if type.start_with? '@@' + resource.exported = true + elsif type.start_with? '@' + resource.virtual = true end - # iterate through the resources to create - defaults = args[2] || {} - args[1].each do |title, params| - params = Puppet::Util.symbolizehash(defaults.merge(params)) - raise ArgumentError, 'params should not contain title' if(params[:title]) - case type_of_resource - # JJM The only difference between a type and a define is the call to instantiate_resource - # for a defined type. - when :type, :define - p_resource = Puppet::Parser::Resource.new(type_name, title, :scope => self, :source => resource) - p_resource.virtual = type_virtual - p_resource.exported = type_exported - {:name => title}.merge(params).each do |k,v| - p_resource.set_parameter(k,v) - end - if type_of_resource == :define then - resource.instantiate_resource(self, p_resource) - end - compiler.add_resource(self, p_resource) - when :class - klass = find_hostclass(title) - raise ArgumentError, "could not find hostclass #{title}" unless klass - klass.ensure_in_catalog(self, params) - compiler.catalog.add_class(title) - end + + begin + resource.safeevaluate(self) + rescue Puppet::ParseError => internal_error + raise internal_error.original end end diff --git a/lib/puppet/parser/functions/each.rb b/lib/puppet/parser/functions/each.rb index 5973de374..3a522098d 100644 --- a/lib/puppet/parser/functions/each.rb +++ b/lib/puppet/parser/functions/each.rb @@ -42,7 +42,6 @@ Puppet::Parser::Functions::newfunction( end enumerator = o.each index = 0 - result = nil if serving_size == 1 (o.size).times do pblock.call(scope, enumerator.next) @@ -68,7 +67,6 @@ Puppet::Parser::Functions::newfunction( raise ArgumentError, "Block must define at most two parameters (for hash entry key and value)." end enumerator = o.each_pair - result = nil if serving_size == 1 (o.size).times do pblock.call(scope, enumerator.next) diff --git a/lib/puppet/parser/functions/extlookup.rb b/lib/puppet/parser/functions/extlookup.rb index d012bcd72..2a81ccc4e 100644 --- a/lib/puppet/parser/functions/extlookup.rb +++ b/lib/puppet/parser/functions/extlookup.rb @@ -9,7 +9,7 @@ uses CSV files but the concept can easily be adjust for databases, yaml or any other queryable data source. The object of this is to make it obvious when it's being used, rather than -magically loading data in when an module is loaded I prefer to look at the code +magically loading data in when a module is loaded I prefer to look at the code and see statements like: $snmp_contact = extlookup(\"snmp_contact\") @@ -26,7 +26,7 @@ Over time there will be a lot of this kind of thing spread all over your manifes and adding an additional client involves grepping through manifests to find all the places where you have constructs like this. -This is a data problem and shouldn't be handled in code, a using this function you +This is a data problem and shouldn't be handled in code, and using this function you can do just that. First you configure it in site.pp: diff --git a/lib/puppet/parser/functions/foreach.rb b/lib/puppet/parser/functions/foreach.rb index f6020e34d..113e96a58 100644 --- a/lib/puppet/parser/functions/foreach.rb +++ b/lib/puppet/parser/functions/foreach.rb @@ -42,7 +42,6 @@ Puppet::Parser::Functions::newfunction( end enumerator = o.each index = 0 - result = nil if serving_size == 1 (o.size).times do pblock.call(scope, enumerator.next) @@ -68,7 +67,6 @@ Puppet::Parser::Functions::newfunction( raise ArgumentError, "Block must define at most two parameters (for hash entry key and value)." end enumerator = o.each_pair - result = nil if serving_size == 1 (o.size).times do pblock.call(scope, enumerator.next) diff --git a/lib/puppet/parser/functions/hiera_include.rb b/lib/puppet/parser/functions/hiera_include.rb index 5fa7994fb..5107ce9ec 100644 --- a/lib/puppet/parser/functions/hiera_include.rb +++ b/lib/puppet/parser/functions/hiera_include.rb @@ -35,7 +35,7 @@ module Puppet::Parser::Functions key, default, override = HieraPuppet.parse_args(args) if answer = HieraPuppet.lookup(key, default, self, override, :array) method = Puppet::Parser::Functions.function(:include) - send(method, answer) + send(method, [answer]) else raise Puppet::ParseError, "Could not find data item #{key}" end diff --git a/lib/puppet/parser/functions/lookup.rb b/lib/puppet/parser/functions/lookup.rb new file mode 100644 index 000000000..47ab719a0 --- /dev/null +++ b/lib/puppet/parser/functions/lookup.rb @@ -0,0 +1,44 @@ +Puppet::Parser::Functions.newfunction(:lookup, :type => :rvalue, :arity => -2, :doc => <<-'ENDHEREDOC') do |args| +Looks up data defined using Puppet Bindings. +The function is callable with one or two arguments and optionally with a lambda to process the result. +The second argument can be a type specification; a String that describes the type of the produced result. +If a value is found, an assert is made that the value is compliant with the specified type. + +When called with one argument; the name: + + lookup('the_name') + +When called with two arguments; the name, and the expected type: + + lookup('the_name', 'String') + +Using a lambda to process the looked up result. + + lookup('the_name') |$result| { if $result == undef { 'Jane Doe' } else { $result }} + +The type specification is one of: + +* the basic types; 'Integer', 'String', 'Float', 'Boolean', or 'Pattern' (regular expression) +* an Array with an optional element type given in '[]', that when not given defaults to '[Data]' +* a Hash with optional key and value types given in '[]', where key type defaults to 'Literal' and value to 'Data', if + only one type is given, the key defaults to 'Literal' +* the abstract type 'Literal' which is one of the basic types +* the abstract type 'Data' which is 'Literal', or type compatible with Array[Data], or Hash[Literal, Data] +* the abstract type 'Collection' which is Array or Hash of any element type. +* the abstract type 'Object' which is any kind of type + +ENDHEREDOC + + unless Puppet[:binder] || Puppet[:parser] == 'future' + raise Puppet::ParseError, "The lookup function is only available with settings --binder true, or --parser future" + end + type_parser = Puppet::Pops::Types::TypeParser.new + pblock = args[-1] if args[-1].is_a?(Puppet::Parser::AST::Lambda) + type_name = args[1] unless args[1].is_a?(Puppet::Parser::AST::Lambda) + type = type_parser.parse( type_name || "Data") + result = compiler.injector.lookup(self, type, args[0]) + if pblock + result = pblock.call(self, result.nil? ? :undef : result) + end + result.nil? ? :undef : result +end diff --git a/lib/puppet/parser/functions/slice.rb b/lib/puppet/parser/functions/slice.rb index 44f451bf2..505ab7261 100644 --- a/lib/puppet/parser/functions/slice.rb +++ b/lib/puppet/parser/functions/slice.rb @@ -79,7 +79,7 @@ Puppet::Parser::Functions::newfunction( end end raise ArgumentError, ("slice(): wrong argument type (#{args[1]}; must be number.") unless slice_size - raise ArgumentError, ("slice(): wrong argument value: #{slice_size}; is not an positive integer number > 0") unless slice_size.is_a?(Fixnum) && slice_size > 0 + raise ArgumentError, ("slice(): wrong argument value: #{slice_size}; is not a positive integer number > 0") unless slice_size.is_a?(Fixnum) && slice_size > 0 receiver = args[0] # the block is optional, ok if nil, function then produces an array diff --git a/lib/puppet/parser/grammar.ra b/lib/puppet/parser/grammar.ra index 509f36fb3..6c1228d71 100644 --- a/lib/puppet/parser/grammar.ra +++ b/lib/puppet/parser/grammar.ra @@ -783,7 +783,6 @@ hasharrayaccesses: hasharrayaccess end ---- header ---- require 'puppet' -require 'puppet/util/loadedfile' require 'puppet/parser/lexer' require 'puppet/parser/ast' diff --git a/lib/puppet/parser/lexer.rb b/lib/puppet/parser/lexer.rb index cfaa34d7d..fee38d946 100644 --- a/lib/puppet/parser/lexer.rb +++ b/lib/puppet/parser/lexer.rb @@ -460,7 +460,6 @@ class Puppet::Parser::Lexer skip until token_queue.empty? and @scanner.eos? do - yielded = false matched_token, value = find_token # error out if we didn't match anything at all diff --git a/lib/puppet/parser/parser.rb b/lib/puppet/parser/parser.rb index 4a472c991..cb6790ef1 100644 --- a/lib/puppet/parser/parser.rb +++ b/lib/puppet/parser/parser.rb @@ -7,7 +7,6 @@ require 'racc/parser.rb' require 'puppet' -require 'puppet/util/loadedfile' require 'puppet/parser/lexer' require 'puppet/parser/ast' diff --git a/lib/puppet/parser/parser_factory.rb b/lib/puppet/parser/parser_factory.rb index bf80c12d2..06c612f39 100644 --- a/lib/puppet/parser/parser_factory.rb +++ b/lib/puppet/parser/parser_factory.rb @@ -29,7 +29,8 @@ module Puppet::Parser def self.eparser(environment) # Since RGen is optional, test that it is installed @@asserted ||= false - assert_rgen_installed() unless @asserted + assert_rgen_installed() unless @@asserted + @@asserted = true require 'puppet/parser' require 'puppet/parser/e_parser_adapter' EParserAdapter.new(Puppet::Parser::Parser.new(environment)) @@ -53,7 +54,7 @@ module Puppet::Parser container.left_expr = litstring raise "no eContainer" if litstring.eContainer() != container raise "no eContainingFeature" if litstring.eContainingFeature() != :left_expr - rescue =>e + rescue raise Puppet::DevError.new("The gem 'rgen' version >= 0.6.1 is required when using '--parser future'. An older version is installed, please update.") end end diff --git a/lib/puppet/parser/parser_support.rb b/lib/puppet/parser/parser_support.rb index 1458230b8..2fb231ff4 100644 --- a/lib/puppet/parser/parser_support.rb +++ b/lib/puppet/parser/parser_support.rb @@ -69,7 +69,7 @@ class Puppet::Parser::Parser # Raise a Parse error. def error(message, options = {}) - if brace = @lexer.expected + if @lexer.expected message += "; expected '%s'" end except = Puppet::ParseError.new(message) diff --git a/lib/puppet/parser/relationship.rb b/lib/puppet/parser/relationship.rb index ac525fcbe..a9d592cbf 100644 --- a/lib/puppet/parser/relationship.rb +++ b/lib/puppet/parser/relationship.rb @@ -50,7 +50,7 @@ class Puppet::Parser::Relationship unless source_resource = catalog.resource(source) raise ArgumentError, "Could not find resource '#{source}' for relationship on '#{target}'" end - unless target_resource = catalog.resource(target) + unless catalog.resource(target) raise ArgumentError, "Could not find resource '#{target}' for relationship from '#{source}'" end Puppet.debug "Adding relationship from #{source} to #{target} with '#{param_name}'" diff --git a/lib/puppet/parser/scope.rb b/lib/puppet/parser/scope.rb index e80680e71..c249e7566 100644 --- a/lib/puppet/parser/scope.rb +++ b/lib/puppet/parser/scope.rb @@ -74,6 +74,22 @@ class Puppet::Parser::Scope def parent @parent end + + def add_entries_to(target = {}) + @parent.add_entries_to(target) unless @parent.nil? + # do not return pure ephemeral ($0-$n) + if is_local_scope? + @symbols.each do |k, v| + if v == :undef + target.delete(k) + else + target[ k ] = v + end + end + end + target + end + end # Initialize a new scope suitable for parser function testing. This method @@ -182,7 +198,7 @@ class Puppet::Parser::Scope @tags = [] # The symbol table for this scope. This is where we store variables. - @symtable = Ephemeral.new + @symtable = Ephemeral.new(nil, true) @ephemeral = [ Ephemeral.new(@symtable) ] @@ -267,7 +283,7 @@ class Puppet::Parser::Scope # @api public def lookupvar(name, options = {}) unless name.is_a? String - raise Puppet::DevError, "Scope variable name is a #{name.class}, not a string" + raise Puppet::ParseError, "Scope variable name #{name.inspect} is a #{name.class}, not a string" end table = @ephemeral.last @@ -305,7 +321,7 @@ class Puppet::Parser::Scope # The scope of the inherited thing of this scope's resource. This could # either be a node that was inherited or the class. # - # @returns [Puppet::Parser::Scope] The scope or nil if there is not an inherited scope + # @return [Puppet::Parser::Scope] The scope or nil if there is not an inherited scope def inherited_scope if has_inherited_class? qualified_scope(resource.resource_type.parent) @@ -320,7 +336,7 @@ class Puppet::Parser::Scope # scope in which it was included. The chain of parent scopes is followed # until a node scope or the topscope is found # - # @returns [Puppet::Parser::Scope] The scope or nil if there is no enclosing scope + # @return [Puppet::Parser::Scope] The scope or nil if there is no enclosing scope def enclosing_scope if has_enclosing_scope? if parent.is_topscope? or parent.is_nodescope? @@ -347,7 +363,11 @@ class Puppet::Parser::Scope def lookup_qualified_variable(class_name, variable_name, position) begin - qualified_scope(class_name).lookupvar(variable_name, position) + if lookup_as_local_name?(class_name, variable_name) + self[variable_name] + else + qualified_scope(class_name).lookupvar(variable_name, position) + end rescue RuntimeError => e location = if position[:lineproc] " at #{position[:lineproc].call}" @@ -361,6 +381,21 @@ class Puppet::Parser::Scope end end + # Handles the special case of looking up fully qualified variable in not yet evaluated top scope + # This is ok if the lookup request originated in topscope (this happens when evaluating + # bindings; using the top scope to provide the values for facts. + # @param class_name [String] the classname part of a variable name, may be special "" + # @param variable_name [String] the variable name without the absolute leading '::' + # @return [Boolean] true if the given variable name should be looked up directly in this scope + # + def lookup_as_local_name?(class_name, variable_name) + # not a local if name has more than one segment + return nil if variable_name =~ /::/ + # partial only if the class for "" cannot be found + return nil unless class_name == "" && klass = find_hostclass(class_name) && class_scope(klass).nil? + is_topscope? + end + def has_inherited_class? is_classscope? and resource.resource_type.parent end @@ -378,9 +413,10 @@ class Puppet::Parser::Scope end private :qualified_scope - # Return a hash containing our variables and their values, optionally (and - # by default) including the values defined in our parent. Local values - # shadow parent values. + # Returns a Hash containing all variables and their values, optionally (and + # by default) including the values defined in parent. Local values + # shadow parent values. Ephemeral scopes for match results ($0 - $n) are not included. + # def to_hash(recursive = true) if recursive and parent target = parent.to_hash(recursive) @@ -388,14 +424,8 @@ class Puppet::Parser::Scope target = Hash.new end - @symtable.each do |name, value| - if value == :undef - target.delete(name) - else - target[name] = value - end - end - + # add all local scopes + @ephemeral.last.add_entries_to(target) target end @@ -441,7 +471,7 @@ class Puppet::Parser::Scope raise Puppet::ParseError.new("Cannot assign to a numeric match result variable '$#{name}'") unless options[:ephemeral] end unless name.is_a? String - raise Puppet::DevError, "Scope variable name is a #{name.class}, not a string" + raise Puppet::ParseError, "Scope variable name #{name.inspect} is a #{name.class}, not a string" end table = effective_symtable options[:ephemeral] @@ -469,7 +499,7 @@ class Puppet::Parser::Scope # scope's symtable. If the parameter `use_ephemeral` is true, the "top most" ephemeral "table" # will be returned (irrespective of it being a match scope or a local scope). # - # @param [Boolean] whether the top most ephemeral (of any kind) should be used or not + # @param use_ephemeral [Boolean] whether the top most ephemeral (of any kind) should be used or not def effective_symtable use_ephemeral s = @ephemeral.last return s if use_ephemeral @@ -523,12 +553,7 @@ class Puppet::Parser::Scope if level == :all @ephemeral = [ Ephemeral.new(@symtable)] else - # If we ever drop 1.8.6 and lower, this should be replaced by a single - # pop-with-a-count - or if someone more ambitious wants to monkey-patch - # that feature into older rubies. --daniel 2012-07-16 - (@ephemeral.size - level).times do - @ephemeral.pop - end + @ephemeral.pop(@ephemeral.size - level) end end diff --git a/lib/puppet/parser/type_loader.rb b/lib/puppet/parser/type_loader.rb index f94ee2ba5..132242f88 100644 --- a/lib/puppet/parser/type_loader.rb +++ b/lib/puppet/parser/type_loader.rb @@ -95,15 +95,7 @@ class Puppet::Parser::TypeLoader # behavior) only load files from the first module of a given name. E.g., # given first/foo and second/foo, only files from first/foo will be loaded. environment.modules.each do |mod| - manifest_files = [] - if File.exists?(mod.manifests) - Find.find(mod.manifests) do |path| - if path.end_with?(".pp") || path.end_with?(".rb") - manifest_files << path - end - end - end - load_files(mod.name, manifest_files) + load_files(mod.name, mod.all_manifests) end end @@ -117,7 +109,7 @@ class Puppet::Parser::TypeLoader # Try to load the object with the given fully qualified name. def try_load_fqname(type, fqname) return nil if fqname == "" # special-case main. - name2files(fqname).each do |filename| + files_to_try_for(fqname).each do |filename| begin imported_types = import_from_modules(filename) if result = imported_types.find { |t| t.type == type and t.name == fqname } @@ -125,7 +117,6 @@ class Puppet::Parser::TypeLoader return result end rescue Puppet::ImportError => detail - # We couldn't load the item # I'm not convienced we should just drop these errors, but this # preserves existing behaviours. end @@ -171,13 +162,17 @@ class Puppet::Parser::TypeLoader # Return a list of all file basenames that should be tried in order # to load the object with the given fully qualified name. - def name2files(fqname) - result = [] - ary = fqname.split("::") - while ary.length > 0 - result << ary.join(File::SEPARATOR) - ary.pop + def files_to_try_for(qualified_name) + qualified_name.split('::').inject([]) do |paths, name| + add_path_for_name(paths, name) + end + end + + def add_path_for_name(paths, name) + if paths.empty? + [name] + else + paths.unshift(File.join(paths.first, name)) end - return result end end diff --git a/lib/puppet/pops.rb b/lib/puppet/pops.rb index b0a36aa2c..be30a59ee 100644 --- a/lib/puppet/pops.rb +++ b/lib/puppet/pops.rb @@ -1,4 +1,5 @@ module Puppet + module Pops require 'puppet/pops/patterns' require 'puppet/pops/utils' @@ -14,9 +15,18 @@ module Puppet require 'puppet/pops/issues' require 'puppet/pops/label_provider' require 'puppet/pops/validation' + require 'puppet/pops/issue_reporter' require 'puppet/pops/model/model' + module Types + require 'puppet/pops/types/types' + require 'puppet/pops/types/type_calculator' + require 'puppet/pops/types/type_factory' + require 'puppet/pops/types/type_parser' + require 'puppet/pops/types/class_loader' + end + module Model require 'puppet/pops/model/tree_dumper' require 'puppet/pops/model/ast_transformer' @@ -26,10 +36,43 @@ module Puppet require 'puppet/pops/model/model_label_provider' end + module Binder + module SchemeHandler + # the handlers are auto loaded via bindings + end + module Producers + require 'puppet/pops/binder/producers' + end + + require 'puppet/pops/binder/binder' + require 'puppet/pops/binder/bindings_model' + require 'puppet/pops/binder/binder_issues' + require 'puppet/pops/binder/bindings_checker' + require 'puppet/pops/binder/bindings_factory' + require 'puppet/pops/binder/bindings_label_provider' + require 'puppet/pops/binder/bindings_validator_factory' + require 'puppet/pops/binder/injector_entry' + require 'puppet/pops/binder/key_factory' + require 'puppet/pops/binder/injector' + require 'puppet/pops/binder/hiera2' + require 'puppet/pops/binder/bindings_composer' + require 'puppet/pops/binder/bindings_model_dumper' + require 'puppet/pops/binder/system_bindings' + require 'puppet/pops/binder/bindings_loader' + + module Config + require 'puppet/pops/binder/config/binder_config' + require 'puppet/pops/binder/config/binder_config_checker' + require 'puppet/pops/binder/config/issues' + require 'puppet/pops/binder/config/diagnostic_producer' + end + end + module Parser require 'puppet/pops/parser/eparser' require 'puppet/pops/parser/parser_support' require 'puppet/pops/parser/lexer' + require 'puppet/pops/parser/evaluating_parser' end module Validation @@ -37,4 +80,6 @@ module Puppet require 'puppet/pops/validation/validator_factory_3_1' end end + + require 'puppet/bindings' end diff --git a/lib/puppet/pops/adaptable.rb b/lib/puppet/pops/adaptable.rb index 86cc97a27..a7df1e4b4 100644 --- a/lib/puppet/pops/adaptable.rb +++ b/lib/puppet/pops/adaptable.rb @@ -68,7 +68,7 @@ module Puppet::Pops::Adaptable # def self.get(o) attr_name = :"@#{instance_var_name(self.name)}" - if existing = o.instance_variable_defined?(attr_name) + if o.instance_variable_defined?(attr_name) o.instance_variable_get(attr_name) else nil @@ -93,7 +93,7 @@ module Puppet::Pops::Adaptable # def self.adapt(o, &block) attr_name = :"@#{instance_var_name(self.name)}" - adapter = if existing = o.instance_variable_defined?(attr_name) && value = o.instance_variable_get(attr_name) + adapter = if o.instance_variable_defined?(attr_name) && value = o.instance_variable_get(attr_name) value else associate_adapter(create_adapter(o), o) diff --git a/lib/puppet/pops/adapters.rb b/lib/puppet/pops/adapters.rb index 07b3a1caa..13e53845c 100644 --- a/lib/puppet/pops/adapters.rb +++ b/lib/puppet/pops/adapters.rb @@ -47,6 +47,10 @@ module Puppet::Pops::Adapters # representing the adapted object from the origin. Not including any # trailing whitespace. attr_accessor :length + + def extract_text_from_string(string) + string.slice(offset, length) + end end # A LoaderAdapter adapts an object with a {Puppet::Pops::Loader}. This is used to make further loading from the diff --git a/lib/puppet/pops/binder/binder.rb b/lib/puppet/pops/binder/binder.rb new file mode 100644 index 000000000..a21098ebc --- /dev/null +++ b/lib/puppet/pops/binder/binder.rb @@ -0,0 +1,421 @@ +# The Binder is responsible for processing layered bindings that can be used to setup an Injector. +# +# An instance should be created, and calls should then be made to {#define_categories} to define the available categories, and +# their precedence. This should be followed by a call to {#define_layers} which will match the layered bindings against the +# effective categories (filtering out everything that does not apply, handle overrides, abstract entries etc.). +# The constructed hash with `key => InjectorEntry` mappings is obtained as {#injector_entries}, and is used to initialize an +# {Puppet::Pops::Binder::Injector Injector}. +# +# @api public +# +class Puppet::Pops::Binder::Binder + # This limits the number of available categorizations, including "common". + # @api private + PRECEDENCE_MAX = 1000 + + # @api private + attr_reader :category_precedences + + # @api private + attr_reader :category_values + + # @api private + attr_reader :injector_entries + + # @api private + attr_reader :key_factory + + # Whether the binder is fully configured or not + # @api public + # + attr_reader :configured + + # @api public + def initialize + @category_precedences = {} + @category_values = {} + @key_factory = Puppet::Pops::Binder::KeyFactory.new() + + # Resulting hash of all key -> binding + @injector_entries = {} + + # Not configured until the fat lady sings + @configured = false + + @next_anonymous_key = 0 + end + + # Answers the question 'is this binder configured?' to the point it can be used to instantiate an Injector + # @api public + def configured?() + configured() + end + + # Defines the effective categories in precedence order (highest precedence first). + # The 'common' (lowest precedence) category should not be included in the list. + # A sanity check is made that there are no more than 1000 categorizations (which is pretty wild). + # + # The term 'effective categories' refers to the evaluated list of tuples (categorization, category-value) represented with + # an instance of Puppet::Pops::Binder::Bindings::EffectiveCategories. + # + # @param effective_categories [Puppet::Pops::Binder::Bindings::EffectiveCategories] effective categories (i.e. with evaluated values) + # @raise ArgumentError if this binder is already configured + # @raise ArgumentError if the argument is not an EffectiveCategories + # @raise ArgumentError if there is an attempt to redefine a category (non unique, or 'common'). + # @return [Puppet::Pops::Binder::Binder] self + # @api public + # + def define_categories(effective_categories) + raise ArgumentError, "This categories are already defined. Cannot redefine." unless @category_precedences.empty? + + # Note: a model instance is used since a Hash does not have a defined order in all Rubies. + unless effective_categories.is_a?(Puppet::Pops::Binder::Bindings::EffectiveCategories) + raise ArgumentError, "Expected Puppet::Pops::Binder::Bindings::EffectiveCategories, but got a: #{effective_categories.class}" + end + categories = effective_categories.categories + raise ArgumentError, "Category limit (#{PRECEDENCE_MAX}) exceeded" unless categories.size <= PRECEDENCE_MAX + + # Automatically add the 'common' category with lowest precedence + @category_precedences['common'] = 0 + + # if categories contains "common", it should be last - simply drop it if present + if last = categories[-1] + if last.categorization == 'common' + categories.delete_at(-1) + end + end + # Process the given categories (highest precedence is first in the list) + categories.each_with_index do |c, index| + cname = c.categorization + raise ArgumentError, "Attempt to redefine categorization: #{cname}" if @category_precedences[cname] + @category_precedences[cname] = PRECEDENCE_MAX - index + @category_values[cname] = c.value + end + self + end + + # Binds layers from highest to lowest as defined by the given LayeredBindings. + # @note + # Categories must be set with #define_categories before calling this method. The model should have been + # validated to get better error messages if the model is invalid. This implementation expects the model + # to be valid, and any errors raised will be more technical runtime errors. + # + # @param layered_bindings [Puppet::Pops::Binder::Bindings::LayeredBindings] the named and ordered layers + # @raise ArgumentError if categories have not been defined + # @raise ArgumentError if this binder is already configured + # @raise ArgumentError if bindings with unresolved 'override' surfaces as an effective binding + # @raise ArgumentError if the given argument has the wrong type, or if model is invalid in some way + # @return [Puppet::Pops::Binder::Binder] self + # @api public + # + def define_layers(layered_bindings) + raise ArgumentError, "This binder is already configured. Cannot redefine its content." if configured?() + + raise ArgumentError, "Categories must be defined first" if @category_precedences.empty? + LayerProcessor.new(self, key_factory).bind(layered_bindings) + injector_entries.each do |k,v| + unless key_factory.is_contributions_key?(k) || v.is_resolved?() + raise ArgumentError, "Binding with unresolved 'override' detected: #{self.class.format_binding(v.binding)}}" + end + end + # and the fat lady has sung + @configured = true + self + end + + # @api private + def next_anonymous_key + tmp = @next_anonymous_key + @next_anonymous_key += 1 + tmp + end + + # @api private + def self.format_binding(b) + type_name = Puppet::Pops::Types::TypeCalculator.new().string(b.type) + layer_name, bindings_name = get_named_binding_layer_and_name(b) + "binding: '#{type_name}/#{b.name}' in: '#{bindings_name}' in layer: '#{layer_name}'" + end + + # @api private + def self.format_contribution_source(b) + layer_name, bindings_name = get_named_binding_layer_and_name(b) + "(layer: #{layer_name}, bindings: #{bindings_name})" + end + + # @api private + def self.get_named_binding_layer_and_name(b) + return ['<unknown>', '<unknown>'] if b.nil? + return [get_named_layer(b), b.name] if b.is_a?(Puppet::Pops::Binder::Bindings::NamedBindings) + get_named_binding_layer_and_name(b.eContainer) + end + + # @api private + def self.get_named_layer(b) + return '<unknown>' if b.nil? + return b.name if b.is_a?(Puppet::Pops::Binder::Bindings::NamedLayer) + get_named_layer(b.eContainer) + end + + # Processes the information in a layer, aggregating it to the injector_entries hash in its parent binder. + # A LayerProcessor holds the intermediate state required while processing one layer. + # + # @api private + # + class LayerProcessor + attr :effective_prec + attr :prec_stack + attr :bindings + attr :binder + attr :key_factory + attr :contributions + + def initialize(binder, key_factory) + @binder = binder + @key_factory = key_factory + @prec_stack = [] + @effective_prec = nil + @bindings = [] + @contributions = [] + @@bind_visitor ||= Puppet::Pops::Visitor.new(nil,"bind",0,0) + end + + # Add the binding to the list of potentially effective bindings from this layer + # @api private + # + def add(b) + bindings << Puppet::Pops::Binder::InjectorEntry.new(effective_prec, b) + end + + # Add a multibind contribution + # @api private + # + def add_contribution(b) + contributions << Puppet::Pops::Binder::InjectorEntry.new(effective_prec, b) + end + + # Bind given abstract binding + # @api private + # + def bind(binding) + @@bind_visitor.visit_this(self, binding) + end + + # @return [Puppet::Pops::Binder::InjectorEntry] the entry with the highest (category) precedence + # @api private + def highest(b1, b2) + case b1.precedence <=> b2.precedence + when 1 + b1 + when -1 + b2 + when 0 + raise_conflicting_binding(b1, b2) + end + end + + # Raises a conflicting bindings error given two InjectorEntry's with same precedence in the same layer + # (if they are in different layers, something is seriously wrong) + def raise_conflicting_binding(b1, b2) + b1_layer_name, b1_bindings_name = Puppet::Pops::Binder::Binder.get_named_binding_layer_and_name(b1.binding) + b2_layer_name, b2_bindings_name = Puppet::Pops::Binder::Binder.get_named_binding_layer_and_name(b2.binding) + + # The resolution is per layer, and if they differ something is serious wrong as a higher layer + # overrides a lower; so no such conflict should be possible: + unless b1_layer_name == b2_layer_name + raise ArgumentError, [ + 'Internal Error: Conflicting binding for', + "'#{b1.binding.name}'", + 'being resolved across layers', + "'#{b1_layer_name}' and", + "'#{b2_layer_name}'" + ].join(' ') + end + + # Conflicting bindings made from the same source + if b1_bindings_name == b2_bindings_name + raise ArgumentError, [ + 'Conflicting binding for name:', + "'#{b1.binding.name}'", + 'in layer:', + "'#{b1_layer_name}', ", + 'both from:', + "'#{b1_bindings_name}'" + ].join(' ') + end + + # Conflicting bindings from different sources + raise ArgumentError, [ + 'Conflicting binding for name:', + "'#{b1.binding.name}'", + 'in layer:', + "'#{b1_layer_name}',", + 'from:', + "'#{b1_bindings_name}', and", + "'#{b2_bindings_name}'" + ].join(' ') + end + + + # Produces the key for the given Binding. + # @param binding [Puppet::Pops::Binder::Bindings::Binding] the binding to get a key for + # @return [Object] an opaque key + # @api private + # + def key(binding) + k = if is_contribution?(binding) + # contributions get a unique (sequential) key + binder.next_anonymous_key() + else + key_factory.binding_key(binding) + end + end + + # @api private + def is_contribution?(binding) + ! binding.multibind_id.nil? + end + + # @api private + def push_precedences(precedences) + prec_stack.push(precedences) + @effective_prec = nil # clear cache + end + + # @api private + def pop_precedences() + prec_stack.pop() + @effective_prec = nil # clear cache + end + + # Returns the effective precedence as an array with highest precedence first. + # Internally the precedence is an array with the highest precedence first. + # + # @api private + # + def effective_prec() + unless @effective_prec + @effective_prec = prec_stack.flatten.uniq.sort.reverse + if @effective_prec.size == 0 + @effective_prec = [ 0 ] # i.e. "common" + end + end + @effective_prec + end + + # @api private + def bind_Binding(o) + if is_contribution?(o) + add_contribution(o) + else + add(o) + end + end + + # @api private + def bind_Bindings(o) + o.bindings.each {|b| bind(b) } + end + + # @api private + def bind_NamedBindings(o) + # Name is ignored here, it should be introspected when needed (in case of errors) + o.bindings.each {|b| bind(b) } + end + + # Process CategorizedBindings by calculating precedence, and then if satisfying the predicates, process the contained + # bindings. + # @api private + # + def bind_CategorizedBindings(o) + precedences = o.predicates.collect do |p| + prec = binder.category_precedences[p.categorization] + + # Skip bindings if the categorization is not present, or + # if the category value is not the effective value for the categorization + # Ignore the value for the common category (it is not possible to state common 'false' etc.) + # + return unless prec + return unless binder.category_values[p.categorization] == p.value.downcase || p.categorization == 'common' + prec + end + push_precedences(precedences) + o.bindings.each {|b| bind(b) } + pop_precedences() + end + + # Process layered bindings from highest to lowest layer + # @api private + # + def bind_LayeredBindings(o) + o.layers.each do |layer| + processor = LayerProcessor.new(binder, key_factory) + # All except abstract (==error) are transfered to injector_entries + + processor.bind(layer).each do |k, v| + entry = binder.injector_entries[k] + unless key_factory.is_contributions_key?(k) + if v.is_abstract?() + layer_name, bindings_name = Puppet::Pops::Binder::Binder.get_named_binding_layer_and_name(v.binding) + type_name = key_factory.type_calculator.string(v.binding.type) + raise ArgumentError, "The abstract binding '#{type_name}/#{v.binding.name}' in '#{bindings_name}' in layer '#{layer_name}' was not overridden" + end + raise ArgumentError, "Internal Error - redefinition of key: #{k}, (should never happen)" if entry + binder.injector_entries[k] = v + else + entry ? entry << v : binder.injector_entries[k] = v + end + end + end + end + + # Processes one named ("top level") layer consisting of a list of NamedBindings + # @api private + # + def bind_NamedLayer(o) + o.bindings.each {|b| bind(b) } + this_layer = {} + + # process regular bindings + bindings.each do |b| + bkey = key(b.binding) + + # ignore if a higher layer defined it, but ensure override gets resolved + if x = binder.injector_entries[bkey] + x.mark_override_resolved() + next + end + + # if already found in this layer, one wins (and resolves override), or it is an error + existing = this_layer[bkey] + winner = existing ? highest(existing, b) : b + this_layer[bkey] = winner + if existing + winner.mark_override_resolved() + end + end + + # Process contributions + # - organize map multibind_id to bindings with this id + # - for each id, create an array with the unique anonymous keys to the contributed bindings + # - bind the index to a special multibind contributions key (these are aggregated) + # + c_hash = Hash.new {|hash, key| hash[ key ] = [] } + contributions.each {|b| c_hash[ b.binding.multibind_id ] << b } + # - for each id + c_hash.each do |k, v| + index = v.collect do |b| + bkey = key(b.binding) + this_layer[bkey] = b + bkey + end + contributions_key = key_factory.multibind_contributions(k) + unless this_layer[contributions_key] + this_layer[contributions_key] = [] + end + this_layer[contributions_key] += index + end + this_layer + end + end +end diff --git a/lib/puppet/pops/binder/binder_issues.rb b/lib/puppet/pops/binder/binder_issues.rb new file mode 100644 index 000000000..1258b1e78 --- /dev/null +++ b/lib/puppet/pops/binder/binder_issues.rb @@ -0,0 +1,142 @@ +# @api public +module Puppet::Pops::Binder::BinderIssues + + # NOTE: The methods #issue and #hard_issue are done in a somewhat funny way + # since the Puppet::Pops::Issues is a module with these methods defined on the module-class + # This makes it hard to inherit them in this module. (Likewise if Issues was a class, and they + # need to be defined for the class, and such methods are also not inherited, it becomes more + # difficult to reuse these. It did not seem as a good idea to refactor Issues at this point + # in time - they should both probably be refactored once bindings support is finished. + # Meanwhile, they delegate to Issues. + + + # (see Puppet::Pops::Issues#issue) + def self.issue (issue_code, *args, &block) + Puppet::Pops::Issues.issue(issue_code, *args, &block) + end + + # (see Puppet::Pops::Issues#hard_issue) + def self.hard_issue(issue_code, *args, &block) + Puppet::Pops::Issues.hard_issue(issue_code, *args, &block) + end + + # Producer issues (binding identified using :binding argument) + + # @api public + MISSING_NAME = issue :MISSING_NAME, :binding do + "#{label.a_an_uc(binding)} with #{label.a_an(semantic)} has no name" + end + + # @api public + MISSING_KEY = issue :MISSING_KEY, :binding do + "#{label.a_an_uc(binding)} with #{label.a_an(semantic)} has no key" + end + + # @api public + MISSING_VALUE = issue :MISSING_VALUE, :binding do + "#{label.a_an_uc(binding)} with #{label.a_an(semantic)} has no value" + end + + # @api public + MISSING_EXPRESSION = issue :MISSING_EXPRESSION, :binding do + "#{label.a_an_uc(binding)} with #{label.a_an(semantic)} has no expression" + end + + # @api public + MISSING_CLASS_NAME = issue :MISSING_CLASS_NAME, :binding do + "#{label.a_an_uc(binding)} with #{label.a_an(semantic)} has no class name" + end + + # @api public + CACHED_PRODUCER_MISSING_PRODUCER = issue :PRODUCER_MISSING_PRODUCER, :binding do + "#{label.a_an_uc(binding)} with #{label.a_an(semantic)} has no producer" + end + + # @api public + INCOMPATIBLE_TYPE = issue :INCOMPATIBLE_TYPE, :binding, :expected_type, :actual_type do + "#{label.a_an_uc(binding)} with #{label.a_an(semantic)} has an incompatible type: expected #{label.a_an(expected_type)}, but got #{label.a_an(actual_type)}." + end + + # @api public + MULTIBIND_INCOMPATIBLE_TYPE = issue :MULTIBIND_INCOMPATIBLE_TYPE, :binding, :actual_type do + "#{label.a_an_uc(binding)} with #{label.a_an(semantic)} cannot bind #{label.a_an(actual_type)} value" + end + + # @api public + MODEL_OBJECT_IS_UNBOUND = issue :MODEL_OBJECT_IS_UNBOUND do + "#{label.a_an_uc(semantic)} is not contained in a binding" + end + + # Binding issues (binding identified using semantic) + + # @api public + MISSING_PRODUCER = issue :MISSING_PRODUCER do + "#{label.a_an_uc(semantic)} has no producer" + end + + # @api public + MISSING_TYPE = issue :MISSING_TYPE do + "#{label.a_an_uc(semantic)} has no type" + end + + # @api public + MULTIBIND_NOT_COLLECTION_PRODUCER = issue :MULTIBIND_NOT_COLLECTION_PRODUCER, :actual_producer do + "#{label.a_an_uc(semantic)} must have a MultibindProducerDescriptor, but got: #{label.a_an(actual_producer)}" + end + + # @api public + MULTIBIND_TYPE_ERROR = issue :MULTIBIND_TYPE_ERROR, :actual_type do + "#{label.a_an_uc(semantic)} is expected to bind a collection type, but got: #{label.a_an(actual_type)}." + end + + # @api public + MISSING_BINDINGS = issue :MISSING_BINDINGS do + "#{label.a_an_uc(semantic)} has zero bindings" + end + + # @api public + MISSING_BINDINGS_NAME = issue :MISSING_BINDINGS_NAME do + "#{label.a_an_uc(semantic)} has no name" + end + + # @api public + MISSING_PREDICATES = issue :MISSING_PREDICATES do + "#{label.a_an_uc(semantic)} has zero predicates" + end + + # @api public + MISSING_CATEGORIZATION = issue :MISSING_CATEGORIZATION do + "#{label.a_an_uc(semantic)} has a category without categorization" + end + + # @api public + MISSING_CATEGORY_VALUE = issue :MISSING_CATEGORY_VALUE do + "#{label.a_an_uc(semantic)} has a category without value" + end + + # @api public + MISSING_LAYERS = issue :MISSING_LAYERS do + "#{label.a_an_uc(semantic)} has zero layers" + end + + # @api public + MISSING_LAYER_NAME = issue :MISSING_LAYER_NAME do + "#{label.a_an_uc(semantic)} has a layer without name" + end + + # @api public + MISSING_BINDINGS_IN_LAYER = issue :MISSING_BINDINGS_IN_LAYER, :layer do + "#{label.a_an_uc(semantic)} has zero bindings in #{label.label(layer)}" + end + + # @api public + PRECEDENCE_MISMATCH_IN_CONTRIBUTION = issue :PRECEDENCE_MISMATCH_IN_CONTRIBUTION, :categorization do + "Precedence mismatch: binding contribution '#{semantic.name}', category: '#{categorization}' is not in correct order" + end + + # @api public + MISSING_CATEGORY_PRECEDENCE = issue :MISSING_CATEGORY_PRECEDENCE, :categorization do + "Missing category precedence: binding contribution '#{semantic.name}', category: '#{categorization}' not found in overall config" + end + +end
\ No newline at end of file diff --git a/lib/puppet/pops/binder/bindings_checker.rb b/lib/puppet/pops/binder/bindings_checker.rb new file mode 100644 index 000000000..dad4bd7c2 --- /dev/null +++ b/lib/puppet/pops/binder/bindings_checker.rb @@ -0,0 +1,217 @@ +# A validator/checker of a bindings model +# @api public +# +class Puppet::Pops::Binder::BindingsChecker + Bindings = Puppet::Pops::Binder::Bindings + Issues = Puppet::Pops::Binder::BinderIssues + Types = Puppet::Pops::Types + + attr_reader :type_calculator + attr_reader :acceptor + + # @api public + def initialize(diagnostics_producer) + @@check_visitor ||= Puppet::Pops::Visitor.new(nil, "check", 0, 0) + @type_calculator = Puppet::Pops::Types::TypeCalculator.new() + @expression_validator = Puppet::Pops::Validation::ValidatorFactory_3_1.new().checker(diagnostics_producer) + @acceptor = diagnostics_producer + end + + # Validates the entire model by visiting each model element and calling `check`. + # The result is collected (or acted on immediately) by the configured diagnostic provider/acceptor + # given when creating this Checker. + # + # @api public + # + def validate(b) + check(b) + b.eAllContents.each {|c| check(c) } + end + + # Performs binding validity check + # @api private + def check(b) + @@check_visitor.visit_this(self, b) + end + + # Checks that a binding has a producer and a type + # @api private + def check_Binding(b) + # Must have a type + acceptor.accept(Issues::MISSING_TYPE, b) unless b.type.is_a?(Types::PObjectType) + + # Must have a producer + acceptor.accept(Issues::MISSING_PRODUCER, b) unless b.producer.is_a?(Bindings::ProducerDescriptor) + end + + # Checks that the producer is a Multibind producer and that the type is a PCollectionType + # @api private + def check_Multibinding(b) + # id is optional (empty id blocks contributions) + + # A multibinding must have PCollectionType + acceptor.accept(Issues::MULTIBIND_TYPE_ERROR, b, {:actual_type => b.type}) unless b.type.is_a?(Types::PCollectionType) + + # if the producer is nil, a suitable producer will be picked automatically + unless b.producer.nil? || b.producer.is_a?(Bindings::MultibindProducerDescriptor) + acceptor.accept(Issues::MULTIBIND_NOT_COLLECTION_PRODUCER, b, {:actual_producer => b.producer}) + end + end + + # Checks that the bindings object contains at least one binding. Then checks each binding in turn + # @api private + def check_Bindings(b) + acceptor.accept(Issues::MISSING_BINDINGS, b) unless has_entries?(b.bindings) + end + + # Checks that a name has been associated with the bindings + # @api private + def check_NamedBindings(b) + acceptor.accept(Issues::MISSING_BINDINGS_NAME, b) unless has_chars?(b.name) + check_Bindings(b) + end + + # Check that the category has a categorization and a value + # @api private + def check_Category(c) + acceptor.accept(Issues::MISSING_CATEGORIZATION, binding_parent(c)) unless has_chars?(c.categorization) + acceptor.accept(Issues::MISSING_CATEGORY_VALUE, binding_parent(c)) unless has_chars?(c.value) + end + + # Check that the binding contains at least one predicate and that all predicates are categorized and has a value + # @api private + def check_CategorizedBindings(b) + acceptor.accept(Issues::MISSING_PREDICATES, b) unless has_entries?(b.predicates) + check_Bindings(b) + end + + # @api private + def check_EffectiveCategories(ec) + end + + # Check layer has a name + # @api private + def check_NamedLayer(l) + acceptor.accept(Issues::MISSING_LAYER_NAME, binding_parent(l)) unless has_chars?(l.name) +# It is ok to have an empty layer +# acceptor.accept(Issues::MISSING_BINDINGS_IN_LAYER, binding_parent(l), { :layer => l.name }) unless has_entries?(l.bindings) + end + + # Checks that the binding has layers and that each layer has a name and at least one binding + # @api private + def check_LayeredBindings(b) + acceptor.accept(Issues::MISSING_LAYERS, b) unless has_entries?(b.layers) + end + + # Checks that the non caching producer has a producer to delegate to + # @api private + def check_NonCachingProducerDescriptor(p) + acceptor.accept(Issues::PRODUCER_MISSING_PRODUCER, p) unless p.producer.is_a?(Bindings::ProducerDescriptor) + end + + # Checks that a constant value has been declared in the producer and that the type + # of the value is compatible with the type declared in the binding + # @api private + def check_ConstantProducerDescriptor(p) + # the product must be of compatible type + # TODO: Likely to change when value becomes a typed Puppet Object + b = binding_parent(p) + if p.value.nil? + acceptor.accept(Issues::MISSING_VALUE, p, {:binding => b}) + else + infered = type_calculator.infer(p.value) + unless type_calculator.assignable?(b.type, infered) + acceptor.accept(Issues::INCOMPATIBLE_TYPE, p, {:binding => b, :expected_type => b.type, :actual_type => infered}) + end + end + end + + # Checks that an expression has been declared in the producer + # @api private + def check_EvaluatingProducerDescriptor(p) + unless p.expression.is_a?(Puppet::Pops::Model::Expression) + acceptor.accept(Issues::MISSING_EXPRESSION, p, {:binding => binding_parent(p)}) + end + end + + # Checks that a class name has been declared in the producer + # @api private + def check_InstanceProducerDescriptor(p) + acceptor.accept(Issues::MISSING_CLASS_NAME, p, {:binding => binding_parent(p)}) unless has_chars?(p.class_name) + end + + # Checks that a type and a name has been declared. The type must be assignable to the type + # declared in the binding. The name can be an empty string to denote 'no name' + # @api private + def check_LookupProducerDescriptor(p) + b = binding_parent(p) + unless type_calculator.assignable(b.type, p.type) + acceptor.accept(Issues::INCOMPATIBLE_TYPE, p, {:binding => b, :expected_type => b.type, :actual_type => p.type }) + end + acceptor.accept(Issues::MISSING_NAME, p, {:binding => b}) if p.name.nil? # empty string is OK + end + + # Checks that a key has been declared, then calls producer_LookupProducerDescriptor to perform + # checks associated with the super class + # @api private + def check_HashLookupProducerDescriptor(p) + acceptor.accept(Issues::MISSING_KEY, p, {:binding => binding_parent(p)}) unless has_chars?(p.key) + check_LookupProducerDescriptor(p) + end + + # Checks that the type declared in the binder is a PArrayType + # @api private + def check_ArrayMultibindProducerDescriptor(p) + b = binding_parent(p) + acceptor.accept(Issues::MULTIBIND_INCOMPATIBLE_TYPE, p, {:binding => b, :actual_type => b.type}) unless b.type.is_a?(Types::PArrayType) + end + + # Checks that the type declared in the binder is a PHashType + # @api private + def check_HashMultibindProducerDescriptor(p) + b = binding_parent(p) + acceptor.accept(Issues::MULTIBIND_INCOMPATIBLE_TYPE, p, {:binding => b, :actual_type => b.type}) unless b.type.is_a?(Types::PHashType) + end + + # Checks that the producer that this producer delegates to is declared + # @api private + def check_ProducerProducerDescriptor(p) + unless p.producer.is_a?(Bindings::ProducerDescriptor) + acceptor.accept(Issues::PRODUCER_MISSING_PRODUCER, p, {:binding => binding_parent(p)}) + end + end + + # @api private + def check_Expression(t) + @expression_validator.validate(t) + end + + # @api private + def check_PObjectType(t) + # Do nothing + end + + # Returns true if the argument is a non empty string + # @api private + def has_chars?(s) + s.is_a?(String) && !s.empty? + end + + # @api private + def has_entries?(s) + !(s.nil? || s.empty?) + end + + # @api private + def binding_parent(p) + begin + x = p.eContainer + if x.nil? + acceptor.accept(Issues::MODEL_OBJECT_IS_UNBOUND, p) + return nil + end + p = x + end while !p.is_a?(Bindings::AbstractBinding) + p + end +end diff --git a/lib/puppet/pops/binder/bindings_composer.rb b/lib/puppet/pops/binder/bindings_composer.rb new file mode 100644 index 000000000..7284f90c8 --- /dev/null +++ b/lib/puppet/pops/binder/bindings_composer.rb @@ -0,0 +1,241 @@ +# The BindingsComposer handles composition of multiple bindings sources +# It is directed by a {Puppet::Pops::Binder::Config::BinderConfig BinderConfig} that indicates how +# the final composition should be layered, and what should be included/excluded in each layer +# +# The bindings composer is intended to be used once per environment as the compiler starts its work. +# +# TODO: Possibly support envdir: scheme / relative to environment root (== same as confdir if there is only one environment). +# This is probably easier to do after ENC changes described in ARM-8 have been implemented. +# TODO: If same config is loaded in a higher layer, skip it in the lower (since it is meaningless to load it again with lower +# precedence. (Optimization, or possibly an error, should produce a warning). +# +class Puppet::Pops::Binder::BindingsComposer + + # The BindingsConfig instance holding the read and parsed, but not evaluated configuration + # @api public + # + attr_reader :config + + # map of scheme name to handler + # @api private + attr_reader :scheme_handlers + + # @return Hash<String, Puppet::Module> map of module name to module instance + # @api private + attr_reader :name_to_module + + # @api private + attr_reader :confdir + + # @api private + attr_reader :diagnostics + + # Container of all warnings and errors produced while initializing and loading bindings + # + # @api public + attr_reader :acceptor + + # @api public + def initialize() + @acceptor = Puppet::Pops::Validation::Acceptor.new() + @diagnostics = Puppet::Pops::Binder::Config::DiagnosticProducer.new(acceptor) + @config = Puppet::Pops::Binder::Config::BinderConfig.new(@diagnostics) + if acceptor.errors? + Puppet::Pops::IssueReporter.assert_and_report(acceptor, :message => 'Binding Composer: error while reading config.') + raise Puppet::DevError.new("Internal Error: IssueReporter did not raise exception for errors in bindings config.") + end + end + + # Configures and creates the boot injector. + # The read config may optionally contain mapping of bindings scheme handler name to handler class, and + # mapping of biera2 backend symbolic name to backend class. + # If present, these are turned into bindings in the category 'extension' (which is only used in the boot injector) which + # has higher precedence than 'default'. This is done to allow users to override the default bindings for + # schemes and backends. + # @param scope [Puppet::Parser:Scope] the scope (used to find compiler and injector for the environment) + # @api private + # + def configure_and_create_injector(scope) + # create the injector (which will pick up the bindings registered above) + @scheme_handlers = SchemeHandlerHelper.new(scope) + + # get extensions from the config + # ------------------------------ + scheme_extensions = @config.scheme_extensions + hiera_backends = @config.hiera_backends + + # Define a named bindings that are known by the SystemBindings + boot_bindings = Puppet::Pops::Binder::BindingsFactory.named_bindings(Puppet::Pops::Binder::SystemBindings::ENVIRONMENT_BOOT_BINDINGS_NAME) do + scheme_extensions.each_pair do |scheme, class_name| + # turn each scheme => class_name into a binding (contribute to the buildings-schemes multibind). + # do this in category 'extensions' to allow them to override the 'default' + when_in_category('extension', 'true').bind do + name(scheme) + instance_of(Puppetx::BINDINGS_SCHEMES_TYPE) + in_multibind(Puppetx::BINDINGS_SCHEMES) + to_instance(class_name) + end + end + hiera_backends.each_pair do |symbolic, class_name| + # turn each symbolic => class_name into a binding (contribute to the hiera backends multibind). + # do this in category 'extensions' to allow them to override the 'default' + when_in_category('extension', 'true').bind do + name(symbolic) + instance_of(Puppetx::HIERA2_BACKENDS_TYPE) + in_multibind(Puppetx::HIERA2_BACKENDS) + to_instance(class_name) + end + end + end + + @injector = scope.compiler.create_boot_injector(boot_bindings.model) + end + + # @return [Puppet::Pops::Binder::Bindings::LayeredBindings] + def compose(scope) + # The boot injector is used to lookup scheme-handlers + configure_and_create_injector(scope) + + # get all existing modules and their root path + @name_to_module = {} + scope.environment.modules.each {|mod| name_to_module[mod.name] = mod } + + # setup the confdir + @confdir = Puppet.settings[:confdir] + + factory = Puppet::Pops::Binder::BindingsFactory + contributions = [] + configured_layers = @config.layering_config.collect do | layer_config | + # get contributions with effective categories + contribs = configure_layer(layer_config, scope, diagnostics) + # collect the contributions separately for later checking of category precedence + contributions.concat(contribs) + # create a named layer with all the bindings for this layer + factory.named_layer(layer_config['name'], *contribs.collect {|c| c.bindings }.flatten) + end + + # must check all contributions are based on compatible category precedence + # (Note that contributions no longer contains the bindings as a side effect of setting them in the collected + # layer. The effective categories and the name remains in the contributed model; this is enough for checking + # and error reporting). + check_contribution_precedence(contributions) + + # Add the two system layers; the final - highest ("can not be overridden" layer), and the lowest + # Everything here can be overridden 'default' layer. + # + configured_layers.insert(0, Puppet::Pops::Binder::SystemBindings.final_contribution) + configured_layers.insert(-1, Puppet::Pops::Binder::SystemBindings.default_contribution) + + # and finally... create the resulting structure + factory.layered_bindings(*configured_layers) + end + + # Evaluates configured categorization and returns the result. + # The result is not cached. + # @api public + # + def effective_categories(scope) + unevaluated_categories = @config.categorization + parser = Puppet::Pops::Parser::EvaluatingParser.new() + file_source = @config.config_file or "defaults in: #{__FILE__}" + evaluated_categories = unevaluated_categories.collect do |category_tuple| + evaluated_categories = [ category_tuple[0], parser.evaluate_string( scope, parser.quote( category_tuple[1] ), file_source ) ] + if evaluated_categories[1].is_a?(String) + # category values are always in lower case + evaluated_categories[1] = evaluated_categories[1].downcase + else + raise ArgumentError, "Categorization value must be a string, category #{evaluated_categories[0]} evaluation resulted in a: '#{result[1].class}'" + end + evaluated_categories + end + Puppet::Pops::Binder::BindingsFactory::categories(evaluated_categories) + end + + private + + # Checks that contribution's effective categorization is in the same relative order as in the overall + # categorization precedence. + # + def check_contribution_precedence(contributions) + cat_prec = { } + @config.categorization.each_with_index {|c, i| cat_prec[ c[0] ] = i } + contributions.each() do |contrib| + # Contributions that do not specify their opinion about categorization silently accepts the precedence + # set in the root configuration - and may thus produce an unexpected result + # + next unless ec = contrib.effective_categories + next unless categories = ec.categories + prev_prec = -1 + categories.each do |c| + prec = cat_prec[c.categorization] + issues = Puppet::Pops::Binder::BinderIssues + unless prec + diagnostics.accept(issues::MISSING_CATEGORY_PRECEDENCE, c, :categorization => c.categorization) + next + end + unless prec > prev_prec + diagnostics.accept(issues::PRECEDENCE_MISMATCH_IN_CONTRIBUTION, c, :categorization => c.categorization) + end + prev_prec = prec + end + end + end + + def configure_layer(layer_description, scope, diagnostics) + name = layer_description['name'] + + # compute effective set of uris to load (and get rid of any duplicates in the process + included_uris = array_of_uris(layer_description['include']) + excluded_uris = array_of_uris(layer_description['exclude']) + effective_uris = Set.new(expand_included_uris(included_uris)).subtract(Set.new(expand_excluded_uris(excluded_uris))) + + # Each URI should result in a ContributedBindings + effective_uris.collect { |uri| scheme_handlers[uri.scheme].contributed_bindings(uri, scope, self) } + end + + def array_of_uris(descriptions) + return [] unless descriptions + descriptions = [descriptions] unless descriptions.is_a?(Array) + descriptions.collect {|d| URI.parse(d) } + end + + def expand_included_uris(uris) + result = [] + uris.each do |uri| + unless handler = scheme_handlers[uri.scheme] + raise ArgumentError, "Unknown bindings provider scheme: '#{uri.scheme}'" + end + result.concat(handler.expand_included(uri, self)) + end + result + end + + def expand_excluded_uris(uris) + result = [] + uris.each do |uri| + unless handler = scheme_handlers[uri.scheme] + raise ArgumentError, "Unknown bindings provider scheme: '#{uri.scheme}'" + end + result.concat(handler.expand_excluded(uri, self)) + end + result + end + + class SchemeHandlerHelper + T = Puppet::Pops::Types::TypeFactory + HASH_OF_HANDLER = T.hash_of(T.type_of('Puppetx::Puppet::BindingsSchemeHandler')) + def initialize(scope) + @scope = scope + @cache = nil + end + def [] (scheme) + load_schemes unless @cache + @cache[scheme] + end + + def load_schemes + @cache = @scope.compiler.boot_injector.lookup(@scope, HASH_OF_HANDLER, Puppetx::BINDINGS_SCHEMES) || {} + end + end + +end diff --git a/lib/puppet/pops/binder/bindings_factory.rb b/lib/puppet/pops/binder/bindings_factory.rb new file mode 100644 index 000000000..a5d3ca46a --- /dev/null +++ b/lib/puppet/pops/binder/bindings_factory.rb @@ -0,0 +1,847 @@ +# A helper class that makes it easier to construct a Bindings model. +# +# The Bindings Model +# ------------------ +# The BindingsModel (defined in {Puppet::Pops::Binder::Bindings} is a model that is intended to be generally free from Ruby concerns. +# This means that it is possible for system integrators to create and serialize such models using other technologies than +# Ruby. This manifests itself in the model in that producers are described using instances of a `ProducerDescriptor` rather than +# describing Ruby classes directly. This is also true of the type system where type is expressed using the {Puppet::Pops::Types} model +# to describe all types. +# +# This class, the `BindingsFactory` is a concrete Ruby API for constructing instances of classes in the model. +# +# Named Bindings +# -------------- +# The typical usage of the factory is to call {named_bindings} which creates a container of bindings wrapped in a *build object* +# equipped with convenience methods to define the details of the just created named bindings. +# The returned builder is an instance of {Puppet::Pops::Binder::BindingsFactory::BindingsContainerBuilder BindingsContainerBuilder}. +# +# Binding +# ------- +# A Binding binds a type/name key to a producer of a value. A binding is conveniently created by calling `bind` on a +# `BindingsContainerBuilder`. The call to bind, produces a binding wrapped in a build object equipped with convenience methods +# to define the details of the just created binding. The returned builder is an instance of +# {Puppet::Pops::Binder::BindingsFactory::BindingsBuilder BindingsBuilder}. +# +# Multibinding +# ------------ +# A multibinding works like a binding, but it requires an additional ID. It also places constraints on the type of the binding; +# it must be a collection type (Hash or Array). +# +# Constructing and Contributing Bindings from Ruby +# ------------------------------------------------ +# The bindings system is used by referencing bindings symbolically; these are then specified in a Ruby file which is autoloaded +# by Puppet. The entry point for user code that creates bindings is described in {Puppet::Bindings Bindings}. +# That class makes use of a BindingsFactory, and the builder objects to make it easy to construct bindings. +# +# It is intended that a user defining bindings in Ruby should be able to use the builder object methods for the majority of tasks. +# If something advanced is wanted, use of one of the helper class methods on the BuildingsFactory, and/or the +# {Puppet::Pops::Types::TypeCalculator TypeCalculator} will be required to create and configure objects that are not handled by +# the methods in the builder objects. +# +# Chaining of calls +# ------------------ +# Since all the build methods return the build object it is easy to stack on additional calls. The intention is to +# do this in an order that is readable from left to right: `bind.string.name('thename').to(42)`, but there is nothing preventing +# making the calls in some other order e.g. `bind.to(42).name('thename').string`, the second is quite unreadable but produces +# the same result. +# +# For sake of human readability, the method `name` is alsp available as `named`, with the intention that it is used after a type, +# e.g. `bind.integer.named('the meaning of life').to(42)` +# +# Methods taking blocks +# ---------------------- +# Several methods take an optional block. The block evaluates with the builder object as `self`. This means that there is no +# need to chain the methods calls, they can instead be made in sequence - e.g. +# +# bind do +# integer +# named 'the meaning of life' +# to 42 +# end +# +# or mix the two styles +# +# bind do +# integer.named 'the meaning of life' +# to 42 +# end +# +# Unwrapping the result +# --------------------- +# The result from all methods is a builder object. Call the method `model` to unwrap the constructed bindings model object. +# +# bindings = BindingsFactory.named_bindings('my named bindings') do +# # bind things +# end.model +# +# @example Create a NamedBinding with content +# result = Puppet::Pops::Binder::BindingsFactory.named_bindings("mymodule::mybindings") do +# bind.name("foo").to(42) +# when_in_category("node", "kermit.example.com").bind.name("foo").to(43) +# bind.string.name("site url").to("http://www.example.com") +# end +# result.model() +# +# @api public +# +module Puppet::Pops::Binder::BindingsFactory + + # Alias for the {Puppet::Pops::Types::TypeFactory TypeFactory}. This is also available as the method + # `type_factory`. + # + T = Puppet::Pops::Types::TypeFactory + + # Abstract base class for bindings object builders. + # Supports delegation of method calls to the BindingsFactory class methods for all methods not implemented + # by a concrete builder. + # + # @abstract + # + class AbstractBuilder + # The built model object. + attr_reader :model + + # @param binding [Puppet::Pops::Binder::Bindings::AbstractBinding] The binding to build. + # @api public + def initialize(binding) + @model = binding + end + + # Provides convenient access to the Bindings Factory class methods. The intent is to provide access to the + # methods that return producers for the purpose of composing more elaborate things than the builder convenience + # methods support directly. + # @api private + # + def method_missing(meth, *args, &block) + factory = Puppet::Pops::Binder::BindingsFactory + if factory.respond_to?(meth) + factory.send(meth, *args, &block) + else + super + end + end + end + + # A bindings builder for an AbstractBinding containing other AbstractBinding instances. + # @api public + class BindingsContainerBuilder < AbstractBuilder + + # Adds an empty binding to the container, and returns a builder for it for further detailing. + # An optional block may be given which is evaluated using `instance_eval`. + # @return [BindingsBuilder] the builder for the created binding + # @api public + # + def bind(&block) + binding = Puppet::Pops::Binder::Bindings::Binding.new() + model.addBindings(binding) + builder = BindingsBuilder.new(binding) + builder.instance_eval(&block) if block_given? + builder + end + + # Binds a multibind with the given identity where later, the looked up result contains all + # contributions to this key. An optional block may be given which is evaluated using `instance_eval`. + # @param id [String] the multibind's id used when adding contributions + # @return [MultibindingsBuilder] the builder for the created multibinding + # @api public + # + def multibind(id, &block) + binding = Puppet::Pops::Binder::Bindings::Multibinding.new() + binding.id = id + model.addBindings(binding) + builder = MultibindingsBuilder.new(binding) + builder.instance_eval(&block) if block_given? + builder + end + + # Adds a categorized bindings to this container. Returns a BindingsContainerBuilder to allow adding + # bindings in the newly created container. An optional block may be given which is evaluated using `instance_eval`. + # @param categorization [String] the name of the categorization e.g. 'node' + # @param category_value [String] the value in that category e.g. 'kermit.example.com' + # @return [BindingsContainerBuilder] the builder for the created categorized bindings container + # @api public + # + def when_in_category(categorization, category_value, &block) + when_in_categories({categorization => category_value}, &block) + end + + # Adds a categorized bindings to this container. Returns a BindingsContainerBuilder to allow adding + # bindings in the newly created container. + # The result is that a processed request must match all the given categorizations + # with the given values. An optional block may be given which is evaluated using `instance_eval`. + # @param categories_hash Hash[String, String] a hash with categorization and categorization value entries + # @return [BindingsContainerBuilder] the builder for the created categorized bindings container + # @api public + # + def when_in_categories(categories_hash, &block) + binding = Puppet::Pops::Binder::Bindings::CategorizedBindings.new() + categories_hash.each do |k,v| + pred = Puppet::Pops::Binder::Bindings::Category.new() + pred.categorization = k + pred.value = v + binding.addPredicates(pred) + end + model.addBindings(binding) + builder = BindingsContainerBuilder.new(binding) + builder.instance_eval(&block) if block_given? + builder + end + end + + # Builds a Binding via convenience methods. + # + # @api public + # + class BindingsBuilder < AbstractBuilder + + # @param binding [Puppet::Pops::Binder::Bindings::AbstractBinding] the binding to build. + # @api public + def initialize(binding) + super binding + data() + end + + # Sets the name of the binding. + # @param name [String] the name to bind. + # @api public + def name(name) + model.name = name + self + end + + # Same as {#name}, but reads better in certain combinations. + # @api public + alias_method :named, :name + + # Sets the binding to be abstract (it must be overridden) + # @api public + def abstract + model.abstract = true + self + end + + # Sets the binding to be override (it must override something) + # @api public + def override + model.override = true + self + end + + # Makes the binding a multibind contribution to the given multibind id + # @param id [String] the multibind id to contribute this binding to + # @api public + def in_multibind(id) + model.multibind_id = id + self + end + + # Sets the type of the binding to the given type. + # @note + # This is only needed if something other than the default type `Data` is wanted, or if the wanted type is + # not provided by one of the convenience methods {#array_of_data}, {#boolean}, {#float}, {#hash_of_data}, + # {#integer}, {#literal}, {#pattern}, {#string}, or one of the collection methods {#array_of}, or {#hash_of}. + # + # To create a type, use the method {#type_factory}, to obtain the type. + # @example creating a Hash with Integer key and Array[Integer] element type + # tc = type_factory + # type(tc.hash(tc.array_of(tc.integer), tc.integer) + # @param type [Puppet::Pops::Types::PObjectType] the type to set for the binding + # @api public + # + def type(type) + model.type = type + self + end + + # Sets the type of the binding to Integer. + # @return [Puppet::Pops::Types::PIntegerType] the type + # @api public + def integer() + type(T.integer()) + end + + # Sets the type of the binding to Float. + # @return [Puppet::Pops::Types::PFloatType] the type + # @api public + def float() + type(T.float()) + end + + # Sets the type of the binding to Boolean. + # @return [Puppet::Pops::Types::PBooleanType] the type + # @api public + def boolean() + type(T.boolean()) + end + + # Sets the type of the binding to String. + # @return [Puppet::Pops::Types::PStringType] the type + # @api public + def string() + type(T.string()) + end + + # Sets the type of the binding to Pattern. + # @return [Puppet::Pops::Types::PPatternType] the type + # @api public + def pattern() + type(T.pattern()) + end + + # Sets the type of the binding to the abstract type Literal. + # @return [Puppet::Pops::Types::PLiteralType] the type + # @api public + def literal() + type(T.literal()) + end + + # Sets the type of the binding to the abstract type Data. + # @return [Puppet::Pops::Types::PDataType] the type + # @api public + def data() + type(T.data()) + end + + # Sets the type of the binding to Array[Data]. + # @return [Puppet::Pops::Types::PArrayType] the type + # @api public + def array_of_data() + type(T.array_of_data()) + end + + # Sets the type of the binding to Array[T], where T is given. + # @param t [Puppet::Pops::Types::PObjectType] the type of the elements of the array + # @return [Puppet::Pops::Types::PArrayType] the type + # @api public + def array_of(t) + type(T.array_of(t)) + end + + # Sets the type of the binding to Hash[Literal, Data]. + # @return [Puppet::Pops::Types::PHashType] the type + # @api public + def hash_of_data() + type(T.hash_of_data()) + end + + # Sets type of the binding to `Hash[Literal, t]`. + # To also limit the key type, use {#type} and give it a fully specified + # hash using {#type_factory} and then `hash_of(value_type, key_type)`. + # @return [Puppet::Pops::Types::PHashType] the type + # @api public + def hash_of(t) + type(T.hash_of(t)) + end + + # Sets the type of the binding based on the given argument. + # @overload instance_of(t) + # The same as calling {#type} with `t`. + # @param t [Puppet::Pops::Types::PObjectType] the type + # @overload instance_of(o) + # Infers the type from the given Ruby object and sets that as the type - i.e. "set the type + # of the binding to be that of the given data object". + # @param o [Object] the object to infer the type from + # @overload instance_of(c) + # @param c [Class] the Class to base the type on. + # Sets the type based on the given ruby class. The result is one of the specific puppet types + # if the class can be represented by a specific type, or the open ended PRubyType otherwise. + # @overload instance_of(s) + # The same as using a class, but instead of giving a class instance, the class is expressed using its fully + # qualified name. This method of specifying the type allows late binding (the class does not have to be loaded + # before it can be used in a binding). + # @param s [String] the fully qualified classname to base the type on. + # @return the resulting type + # @api public + # + def instance_of(t) + type(T.type_of(t)) + end + + # Provides convenient access to the type factory. + # This is intended to be used when methods taking a type as argument i.e. {#type}, {#array_of}, {#hash_of}, and {#instance_of}. + # @note + # The type factory is also available via the constant {T}. + # @api public + def type_factory + Puppet::Pops::Types::TypeFactory + end + + # Sets the binding's producer to a singleton producer, if given argument is a value, a literal producer is created for it. + # To create a producer producing an instance of a class with lazy loading of the class, use {#to_instance}. + # + # @overload to(a_literal) + # Sets a constant producer in the binding. + # @overload to(a_class, *args) + # Sets an Instantiating producer (producing an instance of the given class) + # @overload to(a_producer_descriptor) + # Sets the producer from the given producer descriptor + # @return [BindingsBuilder] self + # @api public + # + def to(producer, *args) + case producer + when Class + producer = Puppet::Pops::Binder::BindingsFactory.instance_producer(producer.name, *args) + when Puppet::Pops::Model::Expression + producer = Puppet::Pops::Binder::BindingsFactory.evaluating_producer(producer) + when Puppet::Pops::Binder::Bindings::ProducerDescriptor + else + # If given producer is not a producer, create a literal producer + producer = Puppet::Pops::Binder::BindingsFactory.literal_producer(producer) + end + model.producer = producer + self + end + + # Sets the binding's producer to a producer of an instance of given class (a String class name, or a Class instance). + # Use a string class name when lazy loading of the class is wanted. + # + # @overload to_instance(class_name, *args) + # @param class_name [String] the name of the class to instantiate + # @param args [Object] optional arguments to the constructor + # @overload to_instance(a_class) + # @param a_class [Class] the class to instantiate + # @param args [Object] optional arguments to the constructor + # + def to_instance(type, *args) + class_name = case type + when Class + type.name + when String + type + else + raise ArgumentError, "to_instance accepts String (a class name), or a Class.*args got: #{type.class}." + end + model.producer = Puppet::Pops::Binder::BindingsFactory.instance_producer(class_name, *args) + end + + # Sets the binding's producer to a singleton producer + # @overload to_producer(a_producer) + # Sets the producer to an instantiated producer. The resulting model can not be serialized as a consequence as there + # is no meta-model describing the specialized producer. Use this only in exceptional cases, or where there is never the + # need to serialize the model. + # @param a_producer [Puppet::Pops::Binder::Producers::Producer] an instantiated producer, not serializeable ! + # + # @overload to_producer(a_class, *args) + # @param a_class [Class] the class to create an instance of + # @param args [Object] the arguments to the given class' new + # + # @overload to_producer(a_producer_descriptor) + # @param a_producer_descriptor [Puppet::Pops::Binder::Bindings::ProducerDescriptor] a descriptor + # producing Puppet::Pops::Binder::Producers::Producer + # + # @api public + # + def to_producer(producer, *args) + case producer + when Class + producer = Puppet::Pops::Binder::BindingsFactory.instance_producer(producer.name, *args) + when Puppet::Pops::Binder::Bindings::ProducerDescriptor + when Puppet::Pops::Binder::Producers::Producer + # a custom producer instance + producer = Puppet::Pops::Binder::BindingsFactory.literal_producer(producer) + else + raise ArgumentError, "Given producer argument is none of a producer descriptor, a class, or a producer" + end + metaproducer = Puppet::Pops::Binder::BindingsFactory.producer_producer(producer) + model.producer = metaproducer + self + end + + # Sets the binding's producer to a series of producers. + # Use this when you want to produce a different producer on each request for a producer + # + # @overload to_producer(a_producer) + # Sets the producer to an instantiated producer. The resulting model can not be serialized as a consequence as there + # is no meta-model describing the specialized producer. Use this only in exceptional cases, or where there is never the + # need to serialize the model. + # @param a_producer [Puppet::Pops::Binder::Producers::Producer] an instantiated producer, not serializeable ! + # + # @overload to_producer(a_class, *args) + # @param a_class [Class] the class to create an instance of + # @param args [Object] the arguments to the given class' new + # + # @overload to_producer(a_producer_descriptor) + # @param a_producer_descriptor [Puppet::Pops::Binder::Bindings::ProducerDescriptor] a descriptor + # producing Puppet::Pops::Binder::Producers::Producer + # + # @api public + # + def to_producer_series(producer, *args) + case producer + when Class + producer = Puppet::Pops::Binder::BindingsFactory.instance_producer(producer.name, *args) + when Puppet::Pops::Binder::Bindings::ProducerDescriptor + when Puppet::Pops::Binder::Producers::Producer + # a custom producer instance + producer = Puppet::Pops::Binder::BindingsFactory.literal_producer(producer) + else + raise ArgumentError, "Given producer argument is none of a producer descriptor, a class, or a producer" + end + non_caching = Puppet::Pops::Binder::Bindings::NonCachingProducerDescriptor.new() + non_caching.producer = producer + metaproducer = Puppet::Pops::Binder::BindingsFactory.producer_producer(non_caching) + + non_caching = Puppet::Pops::Binder::Bindings::NonCachingProducerDescriptor.new() + non_caching.producer = metaproducer + + model.producer = non_caching + self + end + + # Sets the binding's producer to a "non singleton" producer (each call to produce produces a new instance/copy). + # @overload to_series_of(a_literal) + # a constant producer + # @overload to_series_of(a_class, *args) + # Instantiating producer + # @overload to_series_of(a_producer_descriptor) + # a given producer + # + # @api public + # + def to_series_of(producer, *args) + case producer + when Class + producer = Puppet::Pops::Binder::BindingsFactory.instance_producer(producer.name, *args) + when Puppet::Pops::Binder::Bindings::ProducerDescriptor + else + # If given producer is not a producer, create a literal producer + producer = Puppet::Pops::Binder::BindingsFactory.literal_producer(producer) + end + non_caching = Puppet::Pops::Binder::Bindings::NonCachingProducerDescriptor.new() + non_caching.producer = producer + model.producer = non_caching + self + end + + # Sets the binding's producer to one that performs a lookup of another key + # @overload to_lookup_of(type, name) + # @overload to_lookup_of(name) + # @api public + # + def to_lookup_of(type, name=nil) + unless name + name = type + type = Puppet::Pops::Types::TypeFactory.data() + end + model.producer = Puppet::Pops::Binder::BindingsFactory.lookup_producer(type, name) + self + end + + # Sets the binding's producer to a one that performs a lookup of another key and they applies hash lookup on + # the result. + # + # @overload to_lookup_of(type, name) + # @overload to_lookup_of(name) + # @api public + # + def to_hash_lookup_of(type, name, key) + model.producer = Puppet::Pops::Binder::BindingsFactory.hash_lookup_producer(type, name, key) + self + end + + # Sets the binding's producer to one that produces the first found lookup of another key + # @param list_of_lookups [Array] array of arrays [type name], or just name (implies data) + # @example + # binder.bind().name('foo').to_first_found('fee', 'fum', 'extended-bar') + # binder.bind().name('foo').to_first_found( + # [T.ruby(ThisClass), 'fee'], + # [T.ruby(ThatClass), 'fum'], + # 'extended-bar') + # @api public + # + def to_first_found(*list_of_lookups) + producers = list_of_lookups.collect do |entry| + if entry.is_a?(Array) + case entry.size + when 2 + Puppet::Pops::Binder::BindingsFactory.lookup_producer(entry[0], entry[1]) + when 1 + Puppet::Pops::Binder::BindingsFactory.lookup_producer(Puppet::Pops::Types::TypeFactory.data(), entry[0]) + else + raise ArgumentError, "Not an array of [type, name], name, or [name]" + end + else + Puppet::Pops::Binder::BindingsFactory.lookup_producer(T.data(), entry) + end + end + model.producer = Puppet::Pops::Binder::BindingsFactory.first_found_producer(*producers) + self + end + + # Sets options to the producer. + # See the respective producer for the options it supports. All producers supports the option `:transformer`, a + # puppet or ruby lambda that is evaluated with the produced result as an argument. The ruby lambda gets scope and + # value as arguments. + # @note + # A Ruby lambda is not cross platform safe. Use a puppet lambda if you want a bindings model that is. + # + # @api public + def producer_options(options) + options.each do |k, v| + arg = Puppet::Pops::Binder::Bindings::NamedArgument.new() + arg.name = k.to_s + arg.value = v + model.addProducer_args(arg) + end + self + end + end + + # A builder specialized for multibind - checks that type is Array or Hash based. A new builder sets the + # multibinding to be of type Hash[Data]. + # + # @api public + class MultibindingsBuilder < BindingsBuilder + # Constraints type to be one of {Puppet::Pops::Types::PArrayType PArrayType}, or {Puppet::Pops::Types::PHashType PHashType}. + # @raise [ArgumentError] if type constraint is not met. + # @api public + def type(type) + unless type.class == Puppet::Pops::Types::PArrayType || type.class == Puppet::Pops::Types::PHashType + raise ArgumentError, "Wrong type; only PArrayType, or PHashType allowed, got '#{type.to_s}'" + end + model.type = type + self + end + + # Overrides the default implementation that will raise an exception as a multibind requires a hash type. + # Thus, if nothing else is requested, a multibind will be configured as Hash[Data]. + # + def data() + hash_of_data() + end + end + + # Produces a ContributedBindings. + # A ContributedBindings is used by bindings providers to return a set of named bindings. + # + # @param name [String] the name of the contributed bindings (for human use in messages/logs only) + # @param named_bindings [Puppet::Pops::Binder::Bindings::NamedBindings, Array<Puppet::Pops::Binder::Bindings::NamedBindings>] the + # named bindings to include + # @param effective_categories [Puppet::Pops::Binder::Bindings::EffectiveCategories] the contributors opinion about categorization + # this is used to ensure consistent use of categories. + # + def self.contributed_bindings(name, named_bindings, effective_categories) + cb = Puppet::Pops::Binder::Bindings::ContributedBindings.new() + cb.name = name + named_bindings = [named_bindings] unless named_bindings.is_a?(Array) + named_bindings.each {|b| cb.addBindings(b) } + cb.effective_categories = effective_categories + cb + end + + # Creates a named binding container, the top bindings model object. + # A NamedBindings is typically produced by a bindings provider. + # + # The created container is wrapped in a BindingsContainerBuilder for further detailing. + # Unwrap the built result when done. + # @api public + # + def self.named_bindings(name, &block) + binding = Puppet::Pops::Binder::Bindings::NamedBindings.new() + binding.name = name + builder = BindingsContainerBuilder.new(binding) + builder.instance_eval(&block) if block_given? + builder + end + + # This variant of {named_bindings} evaluates the given block as a method on an anonymous class, + # thus, if the block defines methods or do something with the class itself, this does not pollute + # the base class (BindingsContainerBuilder). + # @api private + # + def self.safe_named_bindings(name, scope, &block) + binding = Puppet::Pops::Binder::Bindings::NamedBindings.new() + binding.name = name + anon = Class.new(BindingsContainerBuilder) do + def initialize(b) + super b + end + end + anon.send(:define_method, :_produce, block) + builder = anon.new(binding) + case block.arity + when 0 + builder._produce() + when 1 + builder._produce(scope) + end + builder + end + + # Creates a literal/constant producer + # @param value [Object] the value to produce + # @return [Puppet::Pops::Binder::Bindings::ProducerDescriptor] a producer description + # @api public + # + def self.literal_producer(value) + producer = Puppet::Pops::Binder::Bindings::ConstantProducerDescriptor.new() + producer.value = value + producer + end + + # Creates a non caching producer + # @param producer [Puppet::Pops::Binder::Bindings::Producer] the producer to make non caching + # @return [Puppet::Pops::Binder::Bindings::ProducerDescriptor] a producer description + # @api public + # + def self.non_caching_producer(producer) + p = Puppet::Pops::Binder::Bindings::NonCachingProducerDescriptor.new() + p.producer = producer + p + end + + # Creates a producer producer + # @param producer [Puppet::Pops::Binder::Bindings::Producer] a producer producing a Producer. + # @return [Puppet::Pops::Binder::Bindings::ProducerDescriptor] a producer description + # @api public + # + def self.producer_producer(producer) + p = Puppet::Pops::Binder::Bindings::ProducerProducerDescriptor.new() + p.producer = producer + p + end + + # Creates an instance producer + # An instance producer creates a new instance of a class. + # If the class implements the class method `inject` this method is called instead of `new` to allow further lookups + # to take place. This is referred to as *assisted inject*. If the class method `inject` is missing, the regular `new` method + # is called. + # + # @param class_name [String] the name of the class + # @param args[Object] arguments to the class' `new` method. + # @return [Puppet::Pops::Binder::Bindings::ProducerDescriptor] a producer description + # @api public + # + def self.instance_producer(class_name, *args) + p = Puppet::Pops::Binder::Bindings::InstanceProducerDescriptor.new() + p.class_name = class_name + args.each {|a| p.addArguments(a) } + p + end + + # Creates a Producer that looks up a value. + # @param type [Puppet::Pops::Types::PObjectType] the type to lookup + # @param name [String] the name to lookup + # @return [Puppet::Pops::Binder::Bindings::ProducerDescriptor] a producer description + # @api public + def self.lookup_producer(type, name) + p = Puppet::Pops::Binder::Bindings::LookupProducerDescriptor.new() + p.type = type + p.name = name + p + end + + # Creates a Hash lookup producer that looks up a hash value, and then a key in the hash. + # + # @return [Puppet::Pops::Binder::Bindings::ProducerDescriptor] a producer description + # @param type [Puppet::Pops::Types::PObjectType] the type to lookup (i.e. a Hash of some key/value type). + # @param name [String] the name to lookup + # @param key [Object] the key to lookup in the looked up hash (type should comply with given key type). + # @api public + # + def self.hash_lookup_producer(type, name, key) + p = Puppet::Pops::Binder::Bindings::HashLookupProducerDescriptor.new() + p.type = type + p.name = name + p.key = key + p + end + + # Creates a first-found producer that looks up from a given series of keys. The first found looked up + # value will be produced. + # @param producers [Array<Puppet::Pops::Binder::Bindings::ProducerDescriptor>] the producers to consult in given order + # @return [Puppet::Pops::Binder::Bindings::ProducerDescriptor] a producer descriptor + # @api public + def self.first_found_producer(*producers) + p = Puppet::Pops::Binder::Bindings::FirstFoundProducerDescriptor.new() + producers.each {|p2| p.addProducers(p2) } + p + end + + # Creates an evaluating producer that evaluates a puppet expression. + # A puppet expression is most conveniently created by using the {Puppet::Pops::Parser::EvaluatingParser EvaluatingParser} as it performs + # all set up and validation of the parsed source. Two convenience methods are used to parse an expression, or parse a ruby string + # as a puppet string. See methods {puppet_expression}, {puppet_string} and {parser} for more information. + # + # @example producing a puppet expression + # expr = puppet_string("Interpolated $fqdn", __FILE__) + # + # @param expression [Puppet::Pops::Model::Expression] a puppet DSL expression as producer by the eparser. + # @return [Puppet::Pops::Binder::Bindings::ProducerDescriptor] a producer descriptor + # @api public + # + def self.evaluating_producer(expression) + p = Puppet::Pops::Binder::Bindings::EvaluatingProducerDescriptor.new() + p.expression = expression + p + end + + # Creates an EffectiveCategories from a list of tuples `[categorization category ...]`, or `[[categorization category] ...]` + # This method is used by backends to create a model of the effective categories. + # @api public + # + def self.categories(tuple_array) + result = Puppet::Pops::Binder::Bindings::EffectiveCategories.new() + tuple_array.flatten.each_slice(2) do |c| + cat = Puppet::Pops::Binder::Bindings::Category.new() + cat.categorization = c[0] + cat.value = c[1] + result.addCategories(cat) + end + result + end + + # Creates a NamedLayer. This is used by the bindings system to create a model of the layers. + # + # @api public + # + def self.named_layer(name, *bindings) + result = Puppet::Pops::Binder::Bindings::NamedLayer.new() + result.name = name + bindings.each { |b| result.addBindings(b) } + result + end + + # Create a LayeredBindings. This is used by the bindings system to create a model of all given layers. + # @param named_layers [Puppet::Pops::Binder::Bindings::NamedLayer] one or more named layers + # @return [Puppet::Pops::Binder::Bindings::LayeredBindings] the constructed layered bindings. + # @api public + # + def self.layered_bindings(*named_layers) + result = Puppet::Pops::Binder::Bindings::LayeredBindings.new() + named_layers.each {|b| result.addLayers(b) } + result + end + + # @return [Puppet::Pops::Parser::EvaluatingParser] a parser for puppet expressions + def self.parser + @parser ||= Puppet::Pops::Parser::EvaluatingParser.new() + end + + # Parses and produces a puppet expression from the given string. + # @param string [String] puppet source e.g. "1 + 2" + # @param source_file [String] the source location, typically `__File__` + # @return [Puppet::Pops::Model::Expression] an expression (that can be bound) + # @api public + # + def self.puppet_expression(string, source_file) + parser.parse_string(string, source_file).current + end + + # Parses and produces a puppet string expression from the given string. + # The string will automatically be quoted and special characters escaped. + # As an example if given the (ruby) string "Hi\nMary" it is transformed to + # the puppet string (illustrated with a ruby string) "\"Hi\\nMary\”" before being + # parsed. + # + # @param string [String] puppet source e.g. "On node $!{fqdn}" + # @param source_file [String] the source location, typically `__File__` + # @return [Puppet::Pops::Model::Expression] an expression (that can be bound) + # @api public + # + def self.puppet_string(string, source_file) + parser.parse_string(parser.quote(string), source_file).current + end +end diff --git a/lib/puppet/pops/binder/bindings_label_provider.rb b/lib/puppet/pops/binder/bindings_label_provider.rb new file mode 100644 index 000000000..f7555468a --- /dev/null +++ b/lib/puppet/pops/binder/bindings_label_provider.rb @@ -0,0 +1,46 @@ +# A provider of labels for bindings model object, producing a human name for the model object. +# @api private +# +class Puppet::Pops::Binder::BindingsLabelProvider < Puppet::Pops::LabelProvider + def initialize + @@label_visitor ||= Puppet::Pops::Visitor.new(self,"label",0,0) + end + + # Produces a label for the given object without article. + # @return [String] a human readable label + # + def label o + @@label_visitor.visit(o) + end + + def label_PObjectType o ; "#{Puppet::Pops::Types::TypeFactory.label(o)}" end + def label_ProducerDescriptor o ; "Producer" end + def label_NonCachingProducerDescriptor o ; "Non Caching Producer" end + def label_ConstantProducerDescriptor o ; "Producer['#{o.value}']" end + def label_EvaluatingProducerDescriptor o ; "Evaluating Producer" end + def label_InstanceProducerDescriptor o ; "Producer[#{o.class_name}]" end + def label_LookupProducerDescriptor o ; "Lookup Producer[#{o.name}]" end + def label_HashLookupProducerDescriptor o ; "Hash Lookup Producer[#{o.name}][#{o.key}]" end + def label_FirstFoundProducerDescriptor o ; "First Found Producer" end + def label_ProducerProducerDescriptor o ; "Producer[Producer]" end + def label_MultibindProducerDescriptor o ; "Multibind Producer" end + def label_ArrayMultibindProducerDescriptor o ; "Array Multibind Producer" end + def label_HashMultibindProducerDescriptor o ; "Hash Multibind Producer" end + def label_Bindings o ; "Bindings" end + def label_NamedBindings o ; "Named Bindings" end + def label_Category o ; "Category '#{o.categorization}/#{o.value}'" end + def label_CategorizedBindings o ; "Categorized Bindings" end + def label_LayeredBindings o ; "Layered Bindings" end + def label_NamedLayer o ; "Layer '#{o.name}'" end + def label_EffectiveCategories o ; "Effective Categories" end + def label_ContributedBindings o ; "Contributed Bindings" end + def label_NamedArgument o ; "Named Argument" end + + def label_Binding(o) + 'Binding' + (o.multibind_id.nil? ? '' : ' In Multibind') + end + def label_Multibinding(o) + 'Multibinding' + (o.multibind_id.nil? ? '' : ' In Multibind') + end + +end diff --git a/lib/puppet/pops/binder/bindings_loader.rb b/lib/puppet/pops/binder/bindings_loader.rb new file mode 100644 index 000000000..44c14adec --- /dev/null +++ b/lib/puppet/pops/binder/bindings_loader.rb @@ -0,0 +1,79 @@ +require 'rgen/metamodel_builder' + +# The ClassLoader provides a Class instance given a class name or a meta-type. +# If the class is not already loaded, it is loaded using the Puppet Autoloader. +# This means it can load a class from a gem, or from puppet modules. +# +class Puppet::Pops::Binder::BindingsLoader + @autoloader = Puppet::Util::Autoload.new("BindingsLoader", "puppet/bindings", :wrap => false) + + # Returns a XXXXX given a fully qualified class name. + # Lookup of class is never relative to the calling namespace. + # @param name [String, Array<String>, Array<Symbol>, Puppet::Pops::Types::PObjectType] A fully qualified + # class name String (e.g. '::Foo::Bar', 'Foo::Bar'), a PObjectType, or a fully qualified name in Array form where each part + # is either a String or a Symbol, e.g. `%w{Puppetx Puppetlabs SomeExtension}`. + # @return [Class, nil] the looked up class or nil if no such class is loaded + # @raise ArgumentError If the given argument has the wrong type + # @api public + # + def self.provide(scope, name) + case name + when String + provide_from_string(scope, name) + + when Array + provide_from_name_path(scope, name.join('::'), name) + + else + raise ArgumentError, "Cannot provide a bindings from a '#{name.class.name}'" + end + end + + # If loadable name exists relative to a a basedir or not. Returns the loadable path as a side effect. + # @return [String, nil] a loadable path for the given name, or nil + # + def self.loadable?(basedir, name) + # note, "lib" is added by the autoloader + # + paths_for_name(name).find {|p| File.exists?(File.join(basedir, "lib/puppet/bindings", p)+'.rb') } + end + + private + + def self.provide_from_string(scope, name) + name_path = name.split('::') + # always from the root, so remove an empty first segment + if name_path[0].empty? + name_path = name_path[1..-1] + end + provide_from_name_path(scope, name, name_path) + end + + def self.provide_from_name_path(scope, name, name_path) + # If bindings is already loaded, try this first + result = Puppet::Bindings.resolve(scope, name) + + unless result + # Attempt to load it using the auto loader + paths_for_name(name).find {|path| @autoloader.load(path) } + result = Puppet::Bindings.resolve(scope, name) + end + result + end + + def self.paths_for_name(fq_name) + [de_camel(fq_name), downcased_path(fq_name)] + end + + def self.downcased_path(fq_name) + fq_name.to_s.gsub(/::/, '/').downcase + end + + def self.de_camel(fq_name) + fq_name.to_s.gsub(/::/, '/'). + gsub(/([A-Z]+)([A-Z][a-z])/,'\1_\2'). + gsub(/([a-z\d])([A-Z])/,'\1_\2'). + tr("-", "_"). + downcase + end +end
\ No newline at end of file diff --git a/lib/puppet/pops/binder/bindings_model.rb b/lib/puppet/pops/binder/bindings_model.rb new file mode 100644 index 000000000..aa01e1366 --- /dev/null +++ b/lib/puppet/pops/binder/bindings_model.rb @@ -0,0 +1,215 @@ +require 'rgen/metamodel_builder' + +# The Bindings model is a model of Key to Producer mappings (bindings). +# The central concept is that a Bindings is a nested structure of bindings. +# A top level Bindings should be a NamedBindings (the name is used primarily +# in error messages). A Key is a Type/Name combination. +# +# TODO: In this version, references to "any object" uses the class Object, +# but this is only temporary. The intent is to use specific Puppet Objects +# that are typed using the Puppet Type System (to enable serialization). +# +# @see Puppet::Pops::Binder::BindingsFactory The BindingsFactory for more details on how to create model instances. +# @api public +module Puppet::Pops::Binder::Bindings + + # @abstract + # @api public + # + class AbstractBinding < Puppet::Pops::Model::PopsObject + abstract + end + + # An abstract producer + # @abstract + # @api public + # + class ProducerDescriptor < Puppet::Pops::Model::PopsObject + abstract + contains_one_uni 'transformer', Puppet::Pops::Model::LambdaExpression + end + + # All producers are singleton producers unless wrapped in a non caching producer + # where each lookup produces a new instance. It is an error to have a nesting level > 1 + # and to nest a NonCachingProducerDescriptor. + # + # @api public + # + class NonCachingProducerDescriptor < ProducerDescriptor + contains_one_uni 'producer', ProducerDescriptor + end + + # Produces a constant value (i.e. something of {Puppet::Pops::Types::PDataType PDataType}) + # @api public + # + class ConstantProducerDescriptor < ProducerDescriptor + # TODO: This should be a typed Puppet Object + has_attr 'value', Object + end + + # Produces a value by evaluating a Puppet DSL expression + # @api public + # + class EvaluatingProducerDescriptor < ProducerDescriptor + contains_one_uni 'expression', Puppet::Pops::Model::Expression + end + + # An InstanceProducer creates an instance of the given class + # Arguments are passed to the class' `new` operator in the order they are given. + # @api public + # + class InstanceProducerDescriptor < ProducerDescriptor + # TODO: This should be a typed Puppet Object ?? + has_many_attr 'arguments', Object, :upperBound => -1 + has_attr 'class_name', String + end + + # A ProducerProducerDescriptor, describes that the produced instance is itself a Producer + # that should be used to produce the value. + # @api public + # + class ProducerProducerDescriptor < ProducerDescriptor + contains_one_uni 'producer', ProducerDescriptor, :lowerBound => 1 + end + + # Produces a value by looking up another key (type/name) + # @api public + # + class LookupProducerDescriptor < ProducerDescriptor + contains_one_uni 'type', Puppet::Pops::Types::PObjectType + has_attr 'name', String + end + + # Produces a value by looking up another multibound key, and then looking up + # the detail using a detail_key. + # This is used to produce a specific service of a given type (such as a SyntaxChecker for the syntax "json"). + # @api public + # + class HashLookupProducerDescriptor < LookupProducerDescriptor + has_attr 'key', String + end + + # Produces a value by looking up each producer in turn. The first existing producer wins. + # @api public + # + class FirstFoundProducerDescriptor < ProducerDescriptor + contains_many_uni 'producers', LookupProducerDescriptor + end + + # @api public + # @abstract + class MultibindProducerDescriptor < ProducerDescriptor + abstract + end + + # Used in a Multibind of Array type unless it has a producer. May explicitly be used as well. + # @api public + # + class ArrayMultibindProducerDescriptor < MultibindProducerDescriptor + end + + # Used in a Multibind of Hash type unless it has a producer. May explicitly be used as well. + # @api public + # + class HashMultibindProducerDescriptor < MultibindProducerDescriptor + end + + # Plays the role of "Hash[String, Object] entry" but with keys in defined order. + # + # @api public + # + class NamedArgument < Puppet::Pops::Model::PopsObject + has_attr 'name', String, :lowerBound => 1 + has_attr 'value', Object, :lowerBound => 1 + end + + # Binds a type/name combination to a producer. Optionally marking the bindidng as being abstract, or being an + # override of another binding. Optionally, the binding defines producer arguments passed to the producer when + # it is created. + # + # @api public + class Binding < AbstractBinding + contains_one_uni 'type', Puppet::Pops::Types::PObjectType + has_attr 'name', String + has_attr 'override', Boolean + has_attr 'abstract', Boolean + # If set is a contribution in a multibind + has_attr 'multibind_id', String, :lowerBound => 0 + # Invariant: Only multibinds may have lowerBound 0, all regular Binding must have a producer. + contains_one_uni 'producer', ProducerDescriptor, :lowerBound => 0 + contains_many_uni 'producer_args', NamedArgument, :lowerBound => 0 + end + + + # A multibinding is a binding other bindings can contribute to. + # + # @api public + class Multibinding < Binding + has_attr 'id', String + end + + # A container of Binding instances + # @api public + # + class Bindings < AbstractBinding + contains_many_uni 'bindings', AbstractBinding + end + + # The top level container of bindings can have a name (for error messages, logging, tracing). + # May be nested. + # @api public + # + class NamedBindings < Bindings + has_attr 'name', String + end + + # A category predicate (the request has to be in this category). + # @api public + # + class Category < Puppet::Pops::Model::PopsObject + has_attr 'categorization', String, :lowerBound => 1 + has_attr 'value', String, :lowerBound => 1 + end + + # A container of Binding instances that are in effect when the + # predicates (min one) evaluates to true. Multiple predicates are handles as an 'and'. + # Note that 'or' semantics are handled by repeating the same rules. + # @api public + # + class CategorizedBindings < Bindings + contains_many_uni 'predicates', Category, :lowerBound => 1 + end + + # A named layer of bindings having the same priority. + # @api public + class NamedLayer < Puppet::Pops::Model::PopsObject + has_attr 'name', String, :lowerBound => 1 + contains_many_uni 'bindings', NamedBindings + end + + # A list of layers with bindings in descending priority order. + # @api public + # + class LayeredBindings < Puppet::Pops::Model::PopsObject + contains_many_uni 'layers', NamedLayer + end + + # A list of categories consisting of categroization name and category value (i.e. the *state of the request*) + # @api public + # + class EffectiveCategories < Puppet::Pops::Model::PopsObject + # The order is from highest precedence to lowest + contains_many_uni 'categories', Category + end + + # ContributedBindings is a named container of one or more NamedBindings. + # The intent is that a bindings producer returns a ContributedBindings which in addition to the bindings + # may optionally contain provider's opinion about the precedence of categories, and their category values. + # This enables merging of bindings, and validation of consistency. + # + # @api public + # + class ContributedBindings < NamedLayer + contains_one_uni 'effective_categories', EffectiveCategories + end +end diff --git a/lib/puppet/pops/binder/bindings_model_dumper.rb b/lib/puppet/pops/binder/bindings_model_dumper.rb new file mode 100644 index 000000000..de62e525f --- /dev/null +++ b/lib/puppet/pops/binder/bindings_model_dumper.rb @@ -0,0 +1,205 @@ + +# Dumps a Pops::Binder::Bindings model in reverse polish notation; i.e. LISP style +# The intention is to use this for debugging output +# TODO: BAD NAME - A DUMP is a Ruby Serialization +# NOTE: use :break, :indent, :dedent in lists to do just that +# +class Puppet::Pops::Binder::BindingsModelDumper < Puppet::Pops::Model::TreeDumper + Bindings = Puppet::Pops::Binder::Bindings + + attr_reader :type_calculator + attr_reader :expression_dumper + + def initialize + super + @type_calculator = Puppet::Pops::Types::TypeCalculator.new() + @expression_dumper = Puppet::Pops::Model::ModelTreeDumper.new() + end + + def dump_BindingsFactory o + do_dump(o.model) + end + + def dump_BindingsBuilder o + do_dump(o.model) + end + + def dump_BindingsContainerBuilder o + do_dump(o.model) + end + + def dump_NamedLayer o + result = ['named-layer', (o.name.nil? ? '<no-name>': o.name), :indent] + if o.bindings + o.bindings.each do |b| + result << :break + result << do_dump(b) + end + end + result << :dedent + result + end + + + def dump_Array o + o.collect {|e| do_dump(e) } + end + + def dump_ASTArray o + ["[]"] + o.children.collect {|x| do_dump(x)} + end + + def dump_ASTHash o + ["{}"] + o.value.sort_by{|k,v| k.to_s}.collect {|x| [do_dump(x[0]), do_dump(x[1])]} + end + + def dump_Integer o + o.to_s + end + + # Dump a Ruby String in single quotes unless it is a number. + def dump_String o + "'#{o}'" + end + + def dump_NilClass o + "()" + end + + def dump_Object o + ['dev-error-no-polymorph-dump-for:', o.class.to_s, o.to_s] + end + + def is_nop? o + o.nil? || o.is_a?(Model::Nop) || o.is_a?(AST::Nop) + end + + def dump_ProducerDescriptor o + result = [o.class.name] + result << expression_dumper.dump(o.transformer) if o.transformer + result + end + + def dump_NonCachingProducerDescriptor o + dump_ProducerDescriptor(o) + do_dump(o.producer) + end + + def dump_ConstantProducerDescriptor o + ['constant', do_dump(o.value)] + end + + def dump_EvaluatingProducerDescriptor o + result = dump_ProducerDescriptor(o) + result << expression_dumper.dump(o.expression) + end + + def dump_InstanceProducerDescriptor + # TODO: o.arguments, o. transformer + ['instance', o.class_name] + end + + def dump_ProducerProducerDescriptor o + # skip the transformer lambda... + result = ['producer-producer', do_dump(o.producer)] + result << expression_dumper.dump(o.transformer) if o.transformer + result + end + + def dump_LookupProducerDescriptor o + ['lookup', do_dump(o.type), o.name] + end + + def dump_PObjectType o + type_calculator.string(o) + end + + def dump_HashLookupProducerDescriptor o + # TODO: transformer lambda + result = ['hash-lookup', do_dump(o.type), o.name, "[#{do_dump(o.key)}]"] + result << expression_dumper.dump(o.transformer) if o.transformer + result + end + + def dump_FirstFoundProducerDescriptor o + # TODO: transformer lambda + ['first-found', do_dump(o.producers)] + end + + def dump_ArrayMultibindProducerDescriptor o + ['multibind-array'] + end + + def dump_HashMultibindProducerDescriptor o + ['multibind-hash'] + end + + def dump_NamedArgument o + "#{o.name} => #{do_dump(o.value)}" + end + + def dump_Binding o + result = ['bind'] + result << 'override' if o.override + result << 'abstract' if o.abstract + result.concat([do_dump(o.type), o.name]) + result << "(in #{o.multibind_id})" if o.multibind_id + result << ['to', do_dump(o.producer)] + do_dump(o.producer_args) + result + end + + def dump_Multibinding o + result = ['multibind', o.id] + result << 'override' if o.override + result << 'abstract' if o.abstract + result.concat([do_dump(o.type), o.name]) + result << "(in #{o.multibind_id})" if o.multibind_id + result << ['to', do_dump(o.producer)] + do_dump(o.producer_args) + result + end + + def dump_Bindings o + do_dump(o.bindings) + end + + def dump_NamedBindings o + result = ['named-bindings', o.name, :indent] + o.bindings.each do |b| + result << :break + result << do_dump(b) + end + result << :dedent + result + end + + def dump_Category o + ['category', o.categorization, do_dump(o.value)] + end + + def dump_CategorizedBindings o + result = ['when', do_dump(o.predicates), :indent] + o.bindings.each do |b| + result << :break + result << do_dump(b) + end + result << :dedent + result + end + + def dump_LayeredBindings o + result = ['layers', :indent] + o.layers.each do |layer| + result << :break + result << do_dump(layer) + end + result << :dedent + result + end + + def dump_EffectiveCategories o + ['categories', do_dump(o.categories)] + end + + def dump_ContributedBindings o + ['contributed', o.name, do_dump(o.effective_categories), do_dump(o.bindings)] + end +end diff --git a/lib/puppet/pops/binder/bindings_validator_factory.rb b/lib/puppet/pops/binder/bindings_validator_factory.rb new file mode 100644 index 000000000..d22675eaa --- /dev/null +++ b/lib/puppet/pops/binder/bindings_validator_factory.rb @@ -0,0 +1,28 @@ +# Configures validation suitable for the bindings model +# @api public +# +class Puppet::Pops::Binder::BindingsValidatorFactory < Puppet::Pops::Validation::Factory + Issues = Puppet::Pops::Binder::BinderIssues + + # Produces the checker to use + def checker diagnostic_producer + Puppet::Pops::Binder::BindingsChecker.new(diagnostic_producer) + end + + # Produces the label provider to use + def label_provider + Puppet::Pops::Binder::BindingsLabelProvider.new() + end + + # Produces the severity producer to use + def severity_producer + p = super + + # Configure each issue that should **not** be an error + # + p[Issues::MISSING_BINDINGS] = :warning + p[Issues::MISSING_LAYERS] = :warning + + p + end +end diff --git a/lib/puppet/pops/binder/config/binder_config.rb b/lib/puppet/pops/binder/config/binder_config.rb new file mode 100644 index 000000000..aa5c45e54 --- /dev/null +++ b/lib/puppet/pops/binder/config/binder_config.rb @@ -0,0 +1,139 @@ +module Puppet::Pops::Binder::Config + # Class holding the Binder Configuration + # The configuration is obtained from the file 'binder_config.yaml' + # that must reside in the root directory of the site + # @api public + # + class BinderConfig + + # The bindings hierarchy is an array of categorizations where the + # array for each category has exactly three elements - the categorization name, + # category value, and the path that is later used by the backend to read + # the bindings for that category + # + # @return [Array<Hash<String, String>, Hash<String, Array<String>>] + # @api public + # + attr_reader :layering_config + + # @return [Array<Array(String, String)>] Array of Category tuples where Strings are not evaluated. + # @api public + # + attr_reader :categorization + + # @return <Hash<String, String>] ({}) optional mapping of bindings-scheme to handler class name + attr_reader :scheme_extensions + + # @return <Hash<String, String>] ({}) optional mapping of hiera backend name to backend class name + attr_reader :hiera_backends + + # @return [String] the loaded config file + attr_accessor :config_file + + DEFAULT_LAYERS = [ + { 'name' => 'site', 'include' => ['confdir-hiera:/', 'confdir:/default?optional'] }, + { 'name' => 'modules', 'include' => ['module-hiera:/*/', 'module:/*::default'] }, + ] + + DEFAULT_CATEGORIES = [ + ['node', "${fqdn}"], + ['osfamily', "${osfamily}"], + ['environment', "${environment}"], + ['common', "true"] + ] + + DEFAULT_SCHEME_EXTENSIONS = {} + + DEFAULT_HIERA_BACKENDS_EXTENSIONS = {} + + def default_config() + # This is hardcoded now, but may be a user supplied default configuration later + {'version' => 1, 'layers' => default_layers, 'categories' => default_categories} + end + + def confdir() + Puppet.settings[:confdir] + end + + # Creates a new Config. The configuration is loaded from the file 'binder_config.yaml' which + # is expected to be found in confdir. + # + # @param diagnostics [DiagnosticProducer] collector of diagnostics + # @api public + # + def initialize(diagnostics) + @config_file = Puppet.settings[:binder_config] + # if file is stated, it must exist + # otherwise it is optional $confdir/binder_conf.yaml + # and if that fails, the default + case @config_file + when NilClass + # use the config file if it exists + rootdir = confdir + if rootdir.is_a?(String) + expanded_config_file = File.expand_path(File.join(rootdir, '/binder_config.yaml')) + if File.exist?(expanded_config_file) + @config_file = expanded_config_file + end + else + raise ArgumentError, "No Puppet settings 'confdir', or it is not a String" + end + when String + unless File.exist?(@config_file) + raise ArgumentError, "Cannot find the given binder configuration file '#{@config_file}'" + end + else + raise ArgumentError, "The setting binder_config is expected to be a String, got: #{@config_file.class.name}." + end + unless @config_file.is_a?(String) && File.exist?(@config_file) + @config_file = nil # use defaults + end + + validator = BinderConfigChecker.new(diagnostics) + begin + data = @config_file ? YAML.load_file(@config_file) : default_config() + validator.validate(data, @config_file) + rescue Errno::ENOENT + diagnostics.accept(Issues::CONFIG_FILE_NOT_FOUND, @config_file) + rescue Errno::ENOTDIR + diagnostics.accept(Issues::CONFIG_FILE_NOT_FOUND, @config_file) + rescue ::SyntaxError => e + diagnostics.accept(Issues::CONFIG_FILE_SYNTAX_ERROR, @config_file, :detail => e.message) + end + + unless diagnostics.errors? + @layering_config = data['layers'] or default_layers + @categorization = data['categories'] or default_categories + @scheme_extensions = (data['extensions'] and data['extensions']['scheme_handlers'] or default_scheme_extensions) + @hiera_backends = (data['extensions'] and data['extensions']['hiera_backends'] or default_hiera_backends_extensions) + else + @layering_config = [] + @categorization = {} + @scheme_extensions = {} + @hiera_backends = {} + end + end + + # The default_xxx methods exists to make it easier to do mocking in tests. + + # @api private + def default_layers + DEFAULT_LAYERS + end + + # @api private + def default_categories + DEFAULT_CATEGORIES + end + + # @api private + def default_scheme_extensions + DEFAULT_SCHEME_EXTENSIONS + end + + # @api private + def default_hiera_backends_extensions + DEFAULT_HIERA_BACKENDS_EXTENSIONS + end + end +end diff --git a/lib/puppet/pops/binder/config/binder_config_checker.rb b/lib/puppet/pops/binder/config/binder_config_checker.rb new file mode 100644 index 000000000..24270ae14 --- /dev/null +++ b/lib/puppet/pops/binder/config/binder_config_checker.rb @@ -0,0 +1,183 @@ +module Puppet::Pops::Binder::Config + # Validates the consistency of a Binder::BinderConfig + class BinderConfigChecker + # Create an instance with a diagnostic producer that will receive the result during validation + # @param diangostics [DiagnosticProducer] The producer that will receive the diagnostic + # @api public + # + def initialize(diagnostics) + @diagnostics = diagnostics + t = Puppet::Pops::Types + @type_calculator = t::TypeCalculator.new() + @array_of_string_type = t::TypeFactory.array_of(t::TypeFactory.string()) + end + + # Validate the consistency of the given data. Diagnostics will be emitted to the DiagnosticProducer + # that was set when this checker was created + # + # @param data [Object] The data read from the config file + # @param config_file [String] The full path of the file. Used in error messages + # @api public + # + def validate(data, config_file) + @unique_layer_names = Set.new() + + if data.is_a?(Hash) + check_top_level(data, config_file) + else + accept(Issues::CONFIG_IS_NOT_HASH, config_file) + end + end + + private + + def accept(issue, semantic, options = {}) + @diagnostics.accept(issue, semantic, options) + end + + def check_top_level(data, config_file) + if layers = (data['layers'] || data[:layers]) + check_layers(layers, config_file) + else + accept(Issues::CONFIG_LAYERS_MISSING, config_file) + end + + if categories = (data['categories'] || data[:categories]) + check_categories(categories, config_file) + else + accept(Issues::CONFIG_CATEGORIES_MISSING, config_file) + end + + if version = (data['version'] or data[:version]) + accept(Issues::CONFIG_WRONG_VERSION, config_file, {:expected => 1, :actual => version}) unless version == 1 + else + accept(Issues::CONFIG_VERSION_MISSING, config_file) + end + if extensions = data['extensions'] + check_extensions(extensions, config_file) + end + end + + def check_layers(layers, config_file) + unless layers.is_a?(Array) + accept(Issues::LAYERS_IS_NOT_ARRAY, config_file, :klass => data.class) + else + layers.each {|layer| check_layer(layer, config_file) } + end + end + + def check_layer(layer, config_file) + unless layer.is_a?(Hash) + accept(Issues::LAYER_IS_NOT_HASH, config_file, :klass => layer.class) + return + end + layer.each_pair do |k, v| + case k + when 'name' + unless v.is_a?(String) + accept(Issues::LAYER_NAME_NOT_STRING, config_file, :class_name => v.class.name) + end + + unless @unique_layer_names.add?(v) + accept(Issues::DUPLICATE_LAYER_NAME, config_file, :name => v.to_s ) + end + + when 'include' + check_bindings_references('include', v, config_file) + + when 'exclude' + check_bindings_references('exclude', v, config_file) + + when Symbol + accept(Issues::LAYER_ATTRIBUTE_IS_SYMBOL, config_file, :name => k.to_s) + + else + accept(Issues::UNKNOWN_LAYER_ATTRIBUTE, config_file, :name => k.to_s ) + end + end + end + + def check_categories(categories, config_file) + unless categories.is_a?(Array) + accept(Issues::CATEGORIES_IS_NOT_ARRAY, config_file, :klass => categories.class) + else + categories.each { |entry| check_category(entry, config_file) } + end + end + + def check_category(category, config_file) + type = @type_calculator.infer(category) + unless @type_calculator.assignable?(@array_of_string_type, type) + accept(Issues::CATEGORY_IS_NOT_ARRAY, config_file, :type => @type_calculator.string(type)) + return + end + unless category.size == 2 + accept(Issues::CATEGORY_NOT_TWO_STRINGS, config_file, :count => category.size) + return + end + unless category[0] =~ /[a-z][a-zA-Z0-9_]*/ + accept(Issues::INVALID_CATEGORY_NAME, config_file, :name => category[0]) + end + end + + # references to bindings is a single String URI, or an array of String URI + # @param kind [String] 'include' or 'exclude' (used in issue messages) + # @param value [String, Array<String>] one or more String URI binding references + # @param config_file [String] reference to the loaded config file + # + def check_bindings_references(kind, value, config_file) + return check_reference(value, kind, config_file) if value.is_a?(String) + accept(Issues::BINDINGS_REF_NOT_STRING_OR_ARRAY, config_file, :kind => kind ) unless value.is_a?(Array) + value.each {|ref| check_reference(ref, kind, config_file) } + end + + # A reference is a URI in string form having one of the schemes: + # - module-hiera + # - confdir-hiera + # - enc + # + # and with a path (at least '/') + # + def check_reference(value, kind, config_file) + begin + uri = URI.parse(value) + + unless uri.scheme + accept(Issues::MISSING_SCHEME, config_file, :uri => uri) + end + unless uri.path + accept(Issues::REF_WITHOUT_PATH, config_file, :uri => uri, :kind => kind) + end + + rescue InvalidURIError => e + accept(Issues::BINDINGS_REF_INVALID_URI, config_file, :msg => e.message) + end + end + + def check_extensions(extensions, config_file) + unless extensions.is_a?(Hash) + accept(Issues::EXTENSIONS_NOT_HASH, config_file, :actual => extensions.class.name) + return + end + # check known extensions + extensions.each_key do |key| + unless ['scheme_handlers', 'hiera_backends'].include? key + accept(Issues::UNKNOWN_EXTENSION, config_file, :extension => key) + end + end + + if binding_schemes = extensions['scheme_handlers'] + unless binding_schemes.is_a?(Hash) + accept(Issues::EXTENSION_BINDING_NOT_HASH, config_file, :extension => 'scheme_handlers', :actual => binding_schemes.class.name) + end + end + + if hiera_backends = extensions['hiera_backends'] + unless hiera_backends.is_a?(Hash) + accept(Issues::EXTENSION_BINDING_NOT_HASH, config_file, :extension => 'hiera_backends', :actual => hiera_backends.class.name) + end + end + + end + end +end diff --git a/lib/puppet/pops/binder/config/diagnostic_producer.rb b/lib/puppet/pops/binder/config/diagnostic_producer.rb new file mode 100644 index 000000000..312dc2b24 --- /dev/null +++ b/lib/puppet/pops/binder/config/diagnostic_producer.rb @@ -0,0 +1,32 @@ +module Puppet::Pops::Binder::Config + # Generates validation diagnostics + class Puppet::Pops::Binder::Config::DiagnosticProducer + def initialize(acceptor) + @acceptor = acceptor + @severity_producer = Puppet::Pops::Validation::SeverityProducer.new + end + + def accept(issue, semantic, arguments={}) + arguments[:semantic] ||= semantic + severity = severity_producer.severity(issue) + @acceptor.accept(Puppet::Pops::Validation::Diagnostic.new(severity, issue, nil, nil, arguments)) + end + + def errors?() + @acceptor.errors? + end + + def severity_producer + p = @severity_producer + # All are errors, if there is need to mark some as warnings... + # p[Issues::XXX] = :warning + + # ignored because there is a default + p[Puppet::Pops::Binder::Config::Issues::CONFIG_LAYERS_MISSING] = :ignore + + # ignored because there is a default + p[Puppet::Pops::Binder::Config::Issues::CONFIG_CATEGORIES_MISSING] = :ignore + p + end + end +end diff --git a/lib/puppet/pops/binder/config/issues.rb b/lib/puppet/pops/binder/config/issues.rb new file mode 100644 index 000000000..d67df45ee --- /dev/null +++ b/lib/puppet/pops/binder/config/issues.rb @@ -0,0 +1,106 @@ +module Puppet::Pops::Binder::Config::Issues + # (see Puppet::Pops::Issues#issue) + def self.issue (issue_code, *args, &block) + Puppet::Pops::Issues.issue(issue_code, *args, &block) + end + + CONFIG_FILE_NOT_FOUND = issue :CONFIG_FILE_NOT_FOUND do + "The binder configuration file: #{semantic} can not be found." + end + + CONFIG_FILE_SYNTAX_ERROR = issue :CONFIG_FILE_SYNTAX_ERROR, :detail do + "Syntax error in configuration file: #{detail}" + end + + CONFIG_IS_NOT_HASH = issue :CONFIG_IS_NOT_HASH do + "The configuration file '#{semantic}' has no hash at the top level" + end + + CONFIG_LAYERS_MISSING = issue :CONFIG_LAYERS_MISSING do + "The configuration file '#{semantic}' has no 'layers' entry in the top level hash" + end + + CONFIG_VERSION_MISSING = issue :CONFIG_VERSION_MISSING do + "The configuration file '#{semantic}' has no 'version' entry in the top level hash" + end + + CONFIG_CATEGORIES_MISSING = issue :CONFIG_CATEGORIES_MISSING do + "The configuration file '#{semantic}' has no 'categories' entry in the top level hash" + end + + LAYERS_IS_NOT_ARRAY = issue :LAYERS_IS_NOT_ARRAY, :klass do + "The configuration file '#{semantic}' should contain a 'layers' key with an Array value, got: #{klass.name}" + end + + CATEGORIES_IS_NOT_ARRAY = issue :CATEGORIES_IS_NOT_ARRAY, :klass do + "The configuration file '#{semantic}' should contain a 'categories' key with an Array value, got: #{klass.name}" + end + + CATEGORY_IS_NOT_ARRAY = issue :CATEGORY_IS_NOT_ARRAY, :klass do + "The configuration file '#{semantic}' each entry in 'categories' should be an Array(String, String), got: #{klass.name}" + end + + CATEGORY_NOT_TWO_STRINGS = issue :CATEGORY_NOT_TWO_STRINGS, :count do + "The configuration file '#{semantic}' each entry in 'categories' should be an array with 2 strings, got: #{count}" + end + + INVALID_CATEGORY_NAME = issue :INVALID_CATEGORY_NAME, :name do + "The configuration file '#{semantic}' contains the invalid category: '#{name}', must match /[a-z][a-zA-Z0-9_]*/" + end + + LAYER_IS_NOT_HASH = issue :LAYER_IS_NOT_HASH, :klass do + "The configuration file '#{semantic}' should contain one hash per layer, got #{klass.name} instead of Hash" + end + + DUPLICATE_LAYER_NAME = issue :DUPLICATE_LAYER_NAME, :name do + "Duplicate layer '#{name}' in configuration file #{semantic}" + end + + UNKNOWN_LAYER_ATTRIBUTE = issue :UNKNOWN_LAYER_ATTRIBUTE, :name do + "Unknown layer attribute '#{name}' in configuration file #{semantic}" + end + + BINDINGS_REF_NOT_STRING_OR_ARRAY = issue :BINDINGS_REF_NOT_STRING_OR_ARRAY, :kind do + "Configuration file #{semantic} has bindings reference in '#{kind}' that is neither a String nor an Array." + end + + MISSING_SCHEME = issue :MISSING_SCHEME, :uri do + "Configuration file #{semantic} contains a bindings reference: '#{uri}' without scheme." + end + + UNKNOWN_REF_SCHEME = issue :UNKNOWN_REF_SCHEME, :uri, :kind do + "Configuration file #{semantic} contains a bindings reference: '#{kind}' => '#{uri}' with unknown scheme" + end + + REF_WITHOUT_PATH = issue :REF_WITHOUT_PATH, :uri, :kind do + "Configuration file #{semantic} contains a bindings reference: '#{kind}' => '#{uri}' without path" + end + + BINDINGS_REF_INVALID_URI = issue :BINDINGS_REF_INVALID_URI, :msg do + "Configuration file #{semantic} contains a bindings reference: '#{kind}' => invalid uri, msg: '#{msg}'" + end + + LAYER_ATTRIBUTE_IS_SYMBOL = issue :LAYER_ATTRIBUTE_IS_SYMBOL, :name do + "Configuration file #{semantic} contains a layer attribute '#{name}' that is a Symbol (should be String)" + end + + LAYER_NAME_NOT_STRING = issue :LAYER_NAME_NOT_STRING, :class_name do + "Configuration file #{semantic} contains a layer name that is not a String, got a: '#{class_name}'" + end + + CONFIG_WRONG_VERSION = issue :CONFIG_WRONG_VERSION, :expected, :actual do + "The configuration file '#{semantic}' has unsupported 'version', expected: #{expected}, but got: #{actual}." + end + + EXTENSIONS_NOT_HASH = issue :EXTENSIONS_NOT_HASH, :actual do + "The configuration file '#{semantic}' contains 'extensions', expected: Hash, but got: #{actual}." + end + + EXTENSION_BINDING_NOT_HASH = issue :EXTENSION_BINDING_NOT_HASH, :extension, :actual do + "The configuration file '#{semantic}' contains '#{extension}', expected: Hash, but got: #{actual}." + end + + UNKNOWN_EXTENSION = issue :UNKNOWN_EXTENSION, :actual do + "The configuration file '#{semantic}' contains the unknown extension: #{extension}." + end +end diff --git a/lib/puppet/pops/binder/hiera2.rb b/lib/puppet/pops/binder/hiera2.rb new file mode 100644 index 000000000..e86299120 --- /dev/null +++ b/lib/puppet/pops/binder/hiera2.rb @@ -0,0 +1,10 @@ +# The Hiera2 Module contains the classes needed to configure a bindings producer +# to read module specific data. The configuration is expected to be found in +# a hiera.yaml file in the root of each module +module Puppet::Pops::Binder::Hiera2 + require 'puppet/pops/binder/hiera2/config_checker' + require 'puppet/pops/binder/hiera2/config' + require 'puppet/pops/binder/hiera2/diagnostic_producer' + require 'puppet/pops/binder/hiera2/bindings_provider' + require 'puppet/pops/binder/hiera2/issues' +end diff --git a/lib/puppet/pops/binder/hiera2/bindings_provider.rb b/lib/puppet/pops/binder/hiera2/bindings_provider.rb new file mode 100644 index 000000000..3cfc41aeb --- /dev/null +++ b/lib/puppet/pops/binder/hiera2/bindings_provider.rb @@ -0,0 +1,148 @@ +module Puppet::Pops::Binder::Hiera2 + Model = Puppet::Pops::Model + + # A BindingsProvider instance is used for creating a bindings model from a module directory + # @api public + # + class BindingsProvider + # The resulting name of loaded bindings (given when initializing) + attr_reader :name + + # Creates a new BindingsProvider by reading the hiera_conf.yaml configuration file. Problems + # with the configuration are reported propagated to the acceptor + # + # @param name [String] the name to assign to the result (and in error messages if there is no result) + # @param hiera_config_dir [String] Path to the directory containing a hiera_config + # @param acceptor [Puppet::Pops::Validation::Acceptor] Acceptor that will receive diagnostics + def initialize(name, hiera_config_dir, acceptor) + @name = name + @parser = Puppet::Pops::Parser::EvaluatingParser.new() + @diagnostics = DiagnosticProducer.new(acceptor) + @type_calculator = Puppet::Pops::Types::TypeCalculator.new() + @config = Config.new(hiera_config_dir, @diagnostics) + end + + # Loads a bindings model using the hierarchy and backends configured for this instance. + # + # @param scope [Puppet::Parser::Scope] The hash used when expanding + # @return [Puppet::Pops::Binder::Bindings::ContributedBindings] A bindings model with effective categories + def load_bindings(scope) + backends = BackendHelper.new(scope) + factory = Puppet::Pops::Binder::BindingsFactory + result = factory.named_bindings(name) + + hierarchy = {} + precedence = [] + + @config.hierarchy.each do |key, value, path| + source_file = File.join(@config.module_dir, 'hiera.config.yaml') + category_value = @parser.evaluate_string(scope, @parser.quote(value), source_file) + + hierarchy[key] = { + :bindings => result.when_in_category(key, category_value), + :path => @parser.evaluate_string(scope, @parser.quote(path)), + :unique_keys =>Set.new()} + + precedence << [key, category_value] + end + + @config.backends.each do |backend_key| + backend = backends[backend_key] + + hierarchy.each_pair do |hier_key, hier_val| + bindings = hier_val[:bindings] + unique_keys = hier_val[:unique_keys] + + hiera_data_file_path = hier_val[:path] + backend.read_data(@config.module_dir, hiera_data_file_path).each_pair do |key, value| + if unique_keys.add?(key) + b = bindings.bind().name(key) + # Transform value into a Model::Expression + expr = build_expr(value, hiera_data_file_path) + if is_constant?(expr) + # The value is constant so toss the expression + b.type(@type_calculator.infer(value)).to(value) + else + # Use an evaluating producer for the binding + b.to(expr) + end + end + end + end + end + + factory.contributed_bindings(name, result.model, factory.categories(precedence)) + end + + private + + # @return true unless the expression is a Model::ConcatenatedString or + # somehow contains one + def is_constant?(expr) + if expr.is_a?(Model::ConcatenatedString) + false + else + !expr.eAllContents.any? { |v| v.is_a?(Model::ConcatenatedString) } + end + end + + # Transform the value into a Model::Expression. Strings are parsed using + # the Pops::Parser::Parser to produce either Model::LiteralString or Model::ConcatenatedString + # + # @param value [Object] May be an String, Number, TrueClass, FalseClass, or NilClass nested to any depth using Hash or Array. + # @param hiera_data_file_path [String] The source_file used when reporting errors + # @return [Model::Expression] The expression that corresponds to the value + def build_expr(value, hiera_data_file_path) + case value + when Symbol + value.to_s + when String + @parser.parse_string(@parser.quote(value)).current + when Hash + value.inject(Model::LiteralHash.new) do |h,(k,v)| + e = Model::KeyedEntry.new + e.key = build_expr(k, hiera_data_file_path) + e.value = build_expr(v, hiera_data_file_path) + h.addEntries(e) + h + end + when Enumerable + value.inject(Model::LiteralList.new) {|a,v| a.addValues(build_expr(v, hiera_data_file_path)); a } + when Numeric + expr = Model::LiteralNumber.new + expr.value = value; + expr + when TrueClass, FalseClass + expr = Model::LiteralBoolean.new + expr.value = value; + expr + when NilClass + Model::Nop.new + else + @diagnostics.accept(Issues::UNABLE_TO_PARSE_INSTANCE, value.class.name) + nil + end + end + end + + # @api private + class BackendHelper + T = Puppet::Pops::Types::TypeFactory + HASH_OF_BACKENDS = T.hash_of(T.type_of('Puppetx::Puppet::Hiera2Backend')) + def initialize(scope) + @scope = scope + @cache = nil + end + + def [] (backend_key) + load_backends unless @cache + @cache[backend_key] + end + + def load_backends + @cache = @scope.compiler.boot_injector.lookup(@scope, HASH_OF_BACKENDS, Puppetx::HIERA2_BACKENDS) || {} + end + end + +end + diff --git a/lib/puppet/pops/binder/hiera2/config.rb b/lib/puppet/pops/binder/hiera2/config.rb new file mode 100644 index 000000000..104cb5531 --- /dev/null +++ b/lib/puppet/pops/binder/hiera2/config.rb @@ -0,0 +1,69 @@ +module Puppet::Pops::Binder::Hiera2 + + # Class holding the Hiera2 Configuration + # The configuration is obtained from the file 'hiera.yaml' + # that must reside in the root directory of the module + # @api public + # + class Puppet::Pops::Binder::Hiera2::Config + DEFAULT_HIERARCHY = [ ['osfamily', '${osfamily}', 'data/osfamily/${osfamily}'], ['common', 'true', 'data/common']] + DEFAULT_BACKENDS = ['yaml', 'json'] + + if defined?(::Psych::SyntaxError) + YamlLoadExceptions = [::StandardError, ::ArgumentError, ::Psych::SyntaxError] + else + YamlLoadExceptions = [::StandardError, ::ArgumentError] + end + + # Returns a list of configured backends. + # + # @return [Array<String>] backend names + attr_reader :backends + + # Root directory of the module holding the configuration + # + # @return [String] An absolute path + attr_reader :module_dir + + # The bindings hierarchy is an array of categorizations where the + # array for each category has exactly three elements - the categorization name, + # category value, and the path that is later used by the backend to read + # the bindings for that category + # + # @return [Array<Array(String, String, String)>] + # @api public + attr_reader :hierarchy + + # Creates a new Config. The configuration is loaded from the file 'hiera.yaml' which + # is expected to be found in the given module_dir. + # + # @param module_dir [String] The module directory + # @param diagnostics [DiagnosticProducer] collector of diagnostics + # @api public + # + def initialize(module_dir, diagnostics) + @module_dir = module_dir + config_file = File.join(module_dir, 'hiera.yaml') + validator = ConfigChecker.new(diagnostics) + begin + data = YAML.load_file(config_file) + validator.validate(data, config_file) + unless diagnostics.errors? + # if these are missing the result is nil, and they get default values later + @hierarchy = data['hierarchy'] + @backends = data['backends'] + end + rescue Errno::ENOENT + diagnostics.accept(Issues::CONFIG_FILE_NOT_FOUND, config_file) + rescue Errno::ENOTDIR + diagnostics.accept(Issues::CONFIG_FILE_NOT_FOUND, config_file) + rescue ::SyntaxError => e + diagnostics.accept(Issues::CONFIG_FILE_SYNTAX_ERROR, e) + rescue *YamlLoadExceptions => e + diagnostics.accept(Issues::CONFIG_FILE_SYNTAX_ERROR, e) + end + @hierarchy ||= DEFAULT_HIERARCHY + @backends ||= DEFAULT_BACKENDS + end + end +end diff --git a/lib/puppet/pops/binder/hiera2/config_checker.rb b/lib/puppet/pops/binder/hiera2/config_checker.rb new file mode 100644 index 000000000..53b32ed81 --- /dev/null +++ b/lib/puppet/pops/binder/hiera2/config_checker.rb @@ -0,0 +1,68 @@ +module Puppet::Pops::Binder::Hiera2 + + # Validates the consistency of a Hiera2::Config + class ConfigChecker + + # Create an instance with a diagnostic producer that will receive the result during validation + # @param diangostics [DiagnosticProducer] The producer that will receive the diagnostic + def initialize(diagnostics) + @diagnostics = diagnostics + end + + # Validate the consistency of the given data. Diagnostics will be emitted to the DiagnosticProducer + # that was set when this checker was created + # + # @param data [Object] The data read from the config file + # @param config_file [String] The full path of the file. Used in error messages + def validate(data, config_file) + if data.is_a?(Hash) + # If the version is missing, it is not meaningful to continue + return unless check_version(data['version'], config_file) + check_hierarchy(data['hierarchy'], config_file) + check_backends(data['backends'], config_file) + else + @diagnostics.accept(Issues::CONFIG_IS_NOT_HASH, config_file) + end + end + + private + + # Version is required and must be >= 2. A warning is issued if version > 2 as this checker is + # for version 2 only. + # @return [Boolean] false if it is meaningless to continue checking + def check_version(version, config_file) + if version.nil? + # This is not hiera2 compatible + @diagnostics.accept(Issues::MISSING_VERSION, config_file) + return false + end + unless version >= 2 + @diagnostics.accept(Issues::WRONG_VERSION, config_file, :expected => 2, :actual => version) + return false + end + unless version == 2 + # it may have a sane subset, hence a different error (configured as warning) + @diagnostics.accept(Issues::LATER_VERSION, config_file, :expected => 2, :actual => version) + end + return true + end + + def check_hierarchy(hierarchy, config_file) + if !hierarchy.is_a?(Array) || hierarchy.empty? + @diagnostics.accept(Issues::MISSING_HIERARCHY, config_file) + else + hierarchy.each do |value| + unless value.is_a?(Array) && value.length() == 3 + @diagnostics.accept(Issues::CATEGORY_MUST_BE_THREE_ELEMENT_ARRAY, config_file) + end + end + end + end + + def check_backends(backends, config_file) + if !backends.is_a?(Array) || backends.empty? + @diagnostics.accept(Issues::MISSING_BACKENDS, config_file) + end + end + end +end diff --git a/lib/puppet/pops/binder/hiera2/diagnostic_producer.rb b/lib/puppet/pops/binder/hiera2/diagnostic_producer.rb new file mode 100644 index 000000000..0ae2d4611 --- /dev/null +++ b/lib/puppet/pops/binder/hiera2/diagnostic_producer.rb @@ -0,0 +1,36 @@ +module Puppet::Pops::Binder::Hiera2 + # Generates validation diagnostics + class Puppet::Pops::Binder::Hiera2::DiagnosticProducer + attr_reader :acceptor + def initialize(an_acceptor) + raise ArgumentError, "Not an acceptor" unless an_acceptor.is_a?(Puppet::Pops::Validation::Acceptor) + @acceptor = an_acceptor + @severity_producer = Puppet::Pops::Validation::SeverityProducer.new + end + + def accept(issue, semantic, arguments={}) + arguments[:semantic] ||= semantic + severity = severity_producer.severity(issue) + acceptor.accept(Puppet::Pops::Validation::Diagnostic.new(severity, issue, nil, nil, arguments)) + end + + def errors?() + acceptor.errors? + end + + def severity_producer + p = @severity_producer + p[Issues::UNRESOLVED_STRING_VARIABLE] = :warning + + # Warning since if it does not blow up on anything else, a sane subset of later version was used + p[Issues::LATER_VERSION] = :warning + + # Ignore MISSING_BACKENDS because a default will be provided + p[Issues::MISSING_BACKENDS] = :ignore + + # Ignore MISSING_HIERARCHY because a default will be provided + p[Issues::MISSING_HIERARCHY] = :ignore + p + end + end +end diff --git a/lib/puppet/pops/binder/hiera2/issues.rb b/lib/puppet/pops/binder/hiera2/issues.rb new file mode 100644 index 000000000..b3b0a10d0 --- /dev/null +++ b/lib/puppet/pops/binder/hiera2/issues.rb @@ -0,0 +1,67 @@ +module Puppet::Pops::Binder::Hiera2::Issues + # (see Puppet::Pops::Issues#issue) + def self.issue (issue_code, *args, &block) + Puppet::Pops::Issues.issue(issue_code, *args, &block) + end + + CONFIG_IS_NOT_HASH = issue :CONFIG_IS_NOT_HASH do + "The configuration file '#{semantic}' has no hash at the top level" + end + + MISSING_HIERARCHY = issue :MISSING_HIERARCHY do + "The configuration file '#{semantic}' contains no hierarchy" + end + + MISSING_BACKENDS = issue :MISSING_BACKENDS do + "The configuration file '#{semantic}' contains no backends" + end + + CATEGORY_MUST_BE_THREE_ELEMENT_ARRAY = issue :CATEGORY_MUST_BE_THREE_ELEMENT_ARRAY do + "The configuration file '#{semantic}' has a malformed hierarchy (should consist of arrays with three string entries)" + end + + CONFIG_FILE_NOT_FOUND = issue :CONFIG_FILE_NOT_FOUND do + "The configuration file '#{semantic}' does not exist" + end + + CONFIG_FILE_SYNTAX_ERROR = issue :CONFIG_FILE_SYNTAX_ERROR do + "Unable to parse: #{semantic}" + end + + CANNOT_LOAD_BACKEND = issue :CANNOT_LOAD_BACKEND, :key, :error do + "Backend '#{key}' in configuration file '#{semantic}' cannot be loaded: #{error}" + end + + BACKEND_FILE_DOES_NOT_DEFINE_CLASS = issue :BACKEND_FILE_DOES_NOT_DEFINE_CLASS, :class_name do + "The file '#{semantic}' does not define class #{class_name}" + end + + NOT_A_BACKEND_CLASS = issue :NOT_A_BACKEND_CLASS, :key, :class_name do + "Class #{class_name}, loaded using key #{key} in file '#{semantic}' is not a subclass of Backend" + end + + METADATA_JSON_NOT_FOUND = issue :METADATA_JSON_NOT_FOUND do + "The metadata file '#{semantic}' does not exist" + end + + UNSUPPORTED_STRING_EXPRESSION = issue :UNSUPPORTED_STRING_EXPRESSION, :expr do + "String '#{semantic}' contains an unsupported expression (type was #{expr.class.name})" + end + + UNRESOLVED_STRING_VARIABLE = issue :UNRESOLVED_STRING_VARIABLE, :key do + "Variable '#{key}' found in string '#{semantic}' cannot be resolved" + end + + MISSING_VERSION = issue :MISSING_VERSION do + "The configuration file '#{semantic}' does not have a version." + end + + WRONG_VERSION = issue :WRONG_VERSION do + "The configuration file '#{semantic}' has the wrong version, expected: #{expected}, actual: #{actual}" + end + + LATER_VERSION = issue :LATER_VERSION do + "The configuration file '#{semantic}' has a version that is newer (features may not work), expected: #{expected}, actual: #{actual}" + end + +end diff --git a/lib/puppet/pops/binder/hiera2/json_backend.rb b/lib/puppet/pops/binder/hiera2/json_backend.rb new file mode 100644 index 000000000..dd4c0f220 --- /dev/null +++ b/lib/puppet/pops/binder/hiera2/json_backend.rb @@ -0,0 +1,18 @@ +require 'json' + +# A Backend implementation capable of reading JSON syntax +class Puppet::Pops::Binder::Hiera2::JsonBackend < Puppetx::Puppet::Hiera2Backend + def read_data(module_dir, source) + begin + source_file = File.join(module_dir, "#{source}.json") + JSON.parse(File.read(source_file)) + rescue Errno::ENOTDIR + # This is OK, the file doesn't need to be present. Return an empty hash + {} + rescue Errno::ENOENT + # This is OK, the file doesn't need to be present. Return an empty hash + {} + end + end +end + diff --git a/lib/puppet/pops/binder/hiera2/yaml_backend.rb b/lib/puppet/pops/binder/hiera2/yaml_backend.rb new file mode 100644 index 000000000..d24a0dc5b --- /dev/null +++ b/lib/puppet/pops/binder/hiera2/yaml_backend.rb @@ -0,0 +1,21 @@ +# A Backend implementation capable of reading YAML syntax +class Puppet::Pops::Binder::Hiera2::YamlBackend < Puppetx::Puppet::Hiera2Backend + def read_data(module_dir, source) + begin + source_file = File.join(module_dir, "#{source}.yaml") + # if file is present but empty or has only "---", YAML.load_file returns false, + # in which case fall back to returning an empty hash + YAML.load_file(source_file) || {} + rescue TypeError => e + # SafeYaml chokes when trying to load using utf-8 and the file is empty + raise e if File.size?(source_file) + {} + rescue Errno::ENOTDIR + # This is OK, the file doesn't need to be present. Return an empty hash + {} + rescue Errno::ENOENT + # This is OK, the file doesn't need to be present. Return an empty hash + {} + end + end +end diff --git a/lib/puppet/pops/binder/injector.rb b/lib/puppet/pops/binder/injector.rb new file mode 100644 index 000000000..722e8abe5 --- /dev/null +++ b/lib/puppet/pops/binder/injector.rb @@ -0,0 +1,688 @@ +# The injector is the "lookup service" class +# +# Initialization +# -------------- +# The injector is initialized with a configured {Puppet::Pops::Binder::Binder Binder}. The Binder instance contains a resolved set of +# `key => "binding information"` that is used to setup the injector. +# +# Lookup +# ------ +# It is possible to lookup either the value, or a producer of the value. The {#lookup} method looks up a value, and the +# {#lookup_producer} looks up a producer. +# Both of these methods can be called with three different signatures; `lookup(key)`, `lookup(type, name)`, and `lookup(name)`, +# with the corresponding calls to obtain a producer; `lookup_producer(key)`, `lookup_producer(type, name)`, and `lookup_producer(name)`. +# +# It is possible to pass a block to {#lookup} and {#lookup_producer}, the block is passed the result of the lookup +# and the result of the block is returned as the value of the lookup. This is useful in order to provide a default value. +# +# @example Lookup with default value +# injector.lookup('favourite_food') {|x| x.nil? ? 'bacon' : x } +# +# Singleton or Not +# ---------------- +# The lookup of a value is always based on the lookup of a producer. For *singleton producers* this means that the value is +# determined by the first value lookup. Subsequent lookups via `lookup` or `lookup_producer` will produce the same instance. +# +# *Non singleton producers* will produce a new instance on each request for a value. For constant value producers this +# means that a new deep-clone is produced for mutable objects (but not for immutable objects as this is not needed). +# Custom producers should have non singleton behavior, or if this is not possible ensure that the produced result is +# immutable. (The behavior if a custom producer hands out a mutable value and this is mutated is undefined). +# +# Custom bound producers capable of producing a series of objects when bound as a singleton means that the producer +# is a singleton, not the value it produces. If such a producer is bound as non singleton, each `lookup` will get a new +# producer (hence, typically, restarting the series). However, the producer returned from `lookup_producer` will not +# recreate the producer on each call to `produce`; i.e. each `lookup_producer` returns a producer capable of returning +# a series of objects. +# +# @see Puppet::Pops::Binder::Binder Binder, for details about how to bind keys to producers +# @see Puppet::Pops::Binder::BindingsFactory BindingsFactory, for a convenient way to create a Binder and bindings +# +# Assisted Inject +# --------------- +# The injector supports lookup of instances of classes *even if the requested class is not explicitly bound*. +# This is possible for classes that have a zero argument `initialize` method, or that has a class method called +# `inject` that takes two arguments; `injector`, and `scope`. +# This is useful in ruby logic as a class can then use the given injector to inject details. +# An `inject` class method wins over a zero argument `initialize` in all cases. +# +# @example Using assisted inject +# # Class with assisted inject support +# class Duck +# attr_reader :name, :year_of_birth +# +# def self.inject(injector, scope, binding, *args) +# # lookup default name and year of birth, and use defaults if not present +# name = injector.lookup(scope,'default-duck-name') {|x| x ? x : 'Donald Duck' } +# year_of_birth = injector.lookup(scope,'default-duck-year_of_birth') {|x| x ? x : 1934 } +# self.new(name, year_of_birth) +# end +# +# def initialize(name, year_of_birth) +# @name = name +# @year_of_birth = year_of_birth +# end +# end +# +# injector.lookup(scope, Duck) +# # Produces a Duck named 'Donald Duck' or named after the binding 'default-duck-name' (and with similar treatment of +# # year_of_birth). +# @see Puppet::Pops::Binder::Producers::AssistedInjectProducer AssistedInjectProducer, for more details on assisted injection +# +# Access to key factory and type calculator +# ----------------------------------------- +# It is important to use the same key factory, and type calculator as the binder. It is therefor possible to obtaint +# these with the methods {#key_factory}, and {#type_calculator}. +# +# Special support for producers +# ----------------------------- +# There is one method specially designed for producers. The {#get_contributions} method returns an array of all contributions +# to a given *contributions key*. This key is obtained from the {#key_factory} for a given multibinding. The returned set of +# contributed bindings is sorted in descending precedence order. Any conflicts, merges, etc. is performed by the multibinding +# producer configured for a multibinding. +# +# @api public +# +class Puppet::Pops::Binder::Injector + + Producers = Puppet::Pops::Binder::Producers + + # An Injector is initialized with a configured {Puppet::Pops::Binder::Binder Binder}. + # + # @param configured_binder [Puppet::Pops::Binder::Binder,nil] The configured binder containing effective bindings. A given value + # of nil creates an injector that returns or yields nil on all lookup requests. + # @raise ArgumentError if the given binder is not fully configured + # + # @api public + # + def initialize(configured_binder) + if configured_binder.nil? + @impl = Private::NullInjectorImpl.new() + else + @impl = Private::InjectorImpl.new(configured_binder) + end + end + + # The KeyFactory used to produce keys in this injector. + # The factory is shared with the Binder to ensure consistent translation to keys. + # A compatible type calculator can also be obtained from the key factory. + # @return [Puppet::Pops::Binder::KeyFactory] the key factory in use + # + # @api public + # + def key_factory() + @impl.key_factory + end + + # Returns the TypeCalculator in use for keys. The same calculator (as used for keys) should be used if there is a need + # to check type conformance, or infer the type of Ruby objects. + # + # @return [Puppet::Pops::Types::TypeCalculator] the type calculator that is in use for keys + # @api public + # + def type_calculator() + @impl.type_calculator() + end + + # Lookup (a.k.a "inject") of a value given a key. + # The lookup may be called with different parameters. This method is a convenience method that + # dispatches to one of #lookup_key or #lookup_type depending on the arguments. It also provides + # the ability to use an optional block that is called with the looked up value, or scope and value if the + # block takes two parameters. This is useful to provide a default value or other transformations, calculations + # based on the result of the lookup. + # + # @overload lookup(scope, key) + # (see #lookup_key) + # @param scope [Puppet::Parser::Scope] the scope to use for evaluation + # @param key [Object] an opaque object being the full key + # + # @overload lookup(scope, type, name = '') + # (see #lookup_type) + # @param scope [Puppet::Parser::Scope] the scope to use for evaluation + # @param type [Puppet::Pops::Types::PObjectType] the type of what to lookup + # @param name [String] the name to use, defaults to empty string (for unnamed) + # + # @overload lookup(scope, name) + # Lookup of Data type with given name. + # @see #lookup_type + # @param scope [Puppet::Parser::Scope] the scope to use for evaluation + # @param name [String] the Data/name to lookup + # + # @yield [value] passes the looked up value to an optional block and returns what this block returns + # @yield [scope, value] passes scope and value to the block and returns what this block returns + # @yieldparam scope [Puppet::Parser::Scope] the scope given to lookup + # @yieldparam value [Object, nil] the looked up value or nil if nothing was found + # + # @raise [ArgumentError] if the block has an arity that is not 1 or 2 + # + # @api public + # + def lookup(scope, *args, &block) + @impl.lookup(scope, *args, &block) + end + + # Looks up a (typesafe) value based on a type/name combination. + # Creates a key for the type/name combination using a KeyFactory. Specialization of the Data type are transformed + # to a Data key, and the result is type checked to conform with the given key. + # + # @param type [Puppet::Pops::Types::PObjectType] the type to lookup as defined by Puppet::Pops::Types::TypeFactory + # @param name [String] the (optional for non `Data` types) name of the entry to lookup. + # The name may be an empty String (the default), but not nil. The name is required for lookup for subtypes of + # `Data`. + # @return [Object, nil] the looked up bound object, or nil if not found (type conformance with given type is guaranteed) + # @raise [ArgumentError] if the produced value does not conform with the given type + # + # @api public + # + def lookup_type(scope, type, name='') + @impl.lookup_type(scope, type, name) + end + + # Looks up the key and returns the entry, or nil if no entry is found. + # Produced type is checked for type conformance with its binding, but not with the lookup key. + # (This since all subtypes of PDataType are looked up using a key based on PDataType). + # Use the Puppet::Pops::Types::TypeCalculator#instance? method to check for conformance of the result + # if this is wanted, or use #lookup_type. + # + # @param key [Object] lookup of key as produced by the key factory + # @return [Object, nil] produced value of type that conforms with bound type (type conformance with key not guaranteed). + # @raise [ArgumentError] if the produced value does not conform with the bound type + # + # @api public + # + def lookup_key(scope, key) + @impl.lookup_key(scope, key) + end + + # Lookup (a.k.a "inject") producer of a value given a key. + # The producer lookup may be called with different parameters. This method is a convenience method that + # dispatches to one of #lookup_producer_key or #lookup_producer_type depending on the arguments. It also provides + # the ability to use an optional block that is called with the looked up producer, or scope and producer if the + # block takes two parameters. This is useful to provide a default value, call a custom producer method, + # or other transformations, calculations based on the result of the lookup. + # + # @overload lookup_producer(scope, key) + # (see #lookup_proudcer_key) + # @param scope [Puppet::Parser::Scope] the scope to use for evaluation + # @param key [Object] an opaque object being the full key + # + # @overload lookup_producer(scope, type, name = '') + # (see #lookup_type) + # @param scope [Puppet::Parser::Scope] the scope to use for evaluation + # @param type [Puppet::Pops::Types::PObjectType], the type of what to lookup + # @param name [String], the name to use, defaults to empty string (for unnamed) + # + # @overload lookup_producer(scope, name) + # Lookup of Data type with given name. + # @see #lookup_type + # @param scope [Puppet::Parser::Scope] the scope to use for evaluation + # @param name [String], the Data/name to lookup + # + # @return [Puppet::Pops::Binder::Producers::Producer, Object, nil] a producer, or what the optional block returns + # + # @yield [producer] passes the looked up producer to an optional block and returns what this block returns + # @yield [scope, producer] passes scope and producer to the block and returns what this block returns + # @yieldparam producer [Puppet::Pops::Binder::Producers::Producer, nil] the looked up producer or nil if nothing was bound + # @yieldparam scope [Puppet::Parser::Scope] the scope given to lookup + # + # @raise [ArgumentError] if the block has an arity that is not 1 or 2 + # + # @api public + # + def lookup_producer(scope, *args, &block) + @impl.lookup_producer(scope, *args, &block) + end + + # Looks up a Producer given an opaque binder key. + # @return [Puppet::Pops::Binder::Producers::Producer, nil] the bound producer, or nil if no such producer was found. + # + # @api public + # + def lookup_producer_key(scope, key) + @impl.lookup_producer_key(scope, key) + end + + # Looks up a Producer given a type/name key. + # @note The result is not type checked (it cannot be until the producer has produced an instance). + # @return [Puppet::Pops::Binder::Producers::Producer, nil] the bound producer, or nil if no such producer was found + # + # @api public + # + def lookup_producer_type(scope, type, name='') + @impl.lookup_producer_type(scope, type, name) + end + + # Returns the contributions to a multibind given its contribution key (as produced by the KeyFactory). + # This method is typically used by multibind value producers, but may be used for introspection of the injector's state. + # + # @param scope [Puppet::Parser::Scope] the scope to use + # @param contributions_key [Object] Opaque key as produced by KeyFactory as the contributions key for a multibinding + # @return [Array<Puppet::Pops::Binder::InjectorEntry>] the contributions sorted in deecending order of precedence + # + # @api public + # + def get_contributions(scope, contributions_key) + @impl.get_contributions(scope, contributions_key) + end + + # Returns an Injector that returns (or yields) nil on all lookups, and produces an empty structure for contributions + # This method is intended for testing purposes. + # + def self.null_injector + self.new(nil) + end + +# The implementation of the Injector is private. +# @see Puppet::Pops::Binder::Injector The public API this module implements. +# @api private +# +module Private + + # This is a mocking "Null" implementation of Injector. It never finds anything + # @api private + class NullInjectorImpl + attr_reader :entries + attr_reader :key_factory + attr_reader :type_calculator + + def initialize + @entries = [] + @key_factory = Puppet::Pops::Binder::KeyFactory.new() + @type_calculator = @key_factory.type_calculator + end + + def lookup(scope, *args, &block) + raise ArgumentError, "lookup should be called with two or three arguments, got: #{args.size()+1}" unless args.size.between?(1,2) + # call block with result if given + if block + case block.arity + when 1 + block.call(:undef) + when 2 + block.call(scope, :undef) + else + raise ArgumentError, "The block should have arity 1 or 2" + end + else + val + end + + end + + # @api private + def lookup_key(scope, key) + nil + end + + # @api private + def lookup_producer(scope, *args, &block) + lookup(scope, *args, &block) + end + + # @api private + def lookup_producer_key(scope, key) + nil + end + + # @api private + def lookup_producer_type(scope, type, name='') + nil + end + + def get_contributions() + [] + end + end + + # @api private + # + class InjectorImpl + # Hash of key => InjectorEntry + # @api private + # + attr_reader :entries + + attr_reader :key_factory + + attr_reader :type_calculator + + def initialize(configured_binder) + raise ArgumentError, "Given Binder is not configured" unless configured_binder && configured_binder.configured?() + @entries = configured_binder.injector_entries() + + # It is essential that the injector uses the same key factory as the binder since keys must be + # represented the same (but still opaque) way. + # + @key_factory = configured_binder.key_factory() + @type_calculator = key_factory.type_calculator() + @@transform_visitor ||= Puppet::Pops::Visitor.new(nil,"transform", 2, 2) + @recursion_lock = [ ] + end + + # @api private + def lookup(scope, *args, &block) + raise ArgumentError, "lookup should be called with two or three arguments, got: #{args.size()+1}" unless args.size.between?(1,2) + + val = case args[ 0 ] + + when Puppet::Pops::Types::PObjectType + lookup_type(scope, *args) + + when String + raise ArgumentError, "lookup of name should only pass the name" unless args.size == 1 + lookup_key(scope, key_factory.data_key(args[ 0 ])) + + else + raise ArgumentError, 'lookup using a key should only pass a single key' unless args.size == 1 + lookup_key(scope, args[ 0 ]) + end + + # call block with result if given + if block + case block.arity + when 1 + block.call(val) + when 2 + block.call(scope, val) + else + raise ArgumentError, "The block should have arity 1 or 2" + end + else + val + end + end + + # Produces a key for a type/name combination. + # @api private + def named_key(type, name) + key_factory.named_key(type, name) + end + + # Produces a key for a PDataType/name combination + # @api private + def data_key(name) + key_factory.data_key(name) + end + + # @api private + def lookup_type(scope, type, name='') + val = lookup_key(scope, named_key(type, name)) + unless key_factory.type_calculator.instance?(type, val) + raise ArgumentError, "Type error: incompatible type, #{type_error_detail(type, val)}" + end + val + end + + # @api private + def type_error_detail(expected, actual) + actual_t = type_calculator.infer(actual) + "expected: #{type_calculator.string(expected)}, got: #{type_calculator.string(actual_t)}" + end + + # @api private + def lookup_key(scope, key) + if @recursion_lock.include?(key) + raise ArgumentError, "Lookup loop detected for key: #{key}" + end + begin + @recursion_lock.push(key) + case entry = get_entry(key) + when NilClass + nil + when Puppet::Pops::Binder::InjectorEntry + val = produce(scope, entry) + return nil if val.nil? + unless key_factory.type_calculator.instance?(entry.binding.type, val) + raise "Type error: incompatible type returned by producer, #{type_error_detail(entry.binding.type, val)}" + end + val + when Producers::AssistedInjectProducer + entry.produce(scope) + else + # internal, direct entries + entry + end + ensure + @recursion_lock.pop() + end + end + + # Should be used to get entries as it converts missing entries to NotFound entries or AssistedInject entries + # + # @api private + def get_entry(key) + case entry = entries[ key ] + when NilClass + # not found, is this an assisted inject? + if clazz = assistable_injected_class(key) + entry = Producers::AssistedInjectProducer.new(self, clazz) + entries[ key ] = entry + else + entries[ key ] = NotFound.new() + entry = nil + end + when NotFound + entry = nil + end + entry + end + + # Returns contributions to a multibind in precedence order; highest first. + # Returns an Array on the form [ [key, entry], [key, entry]] where the key is intended to be used to lookup the value + # (or a producer) for that entry. + # @api private + def get_contributions(scope, contributions_key) + result = {} + return [] unless contributions = lookup_key(scope, contributions_key) + contributions.each { |k| result[k] = get_entry(k) } + result.sort {|a, b| a[0] <=> b[0] } + #result.sort_by {|key, entry| entry } + end + + # Produces an injectable class given a key, or nil if key does not represent an injectable class + # @api private + # + def assistable_injected_class(key) + kt = key_factory.get_type(key) + return nil unless kt.is_a?(Puppet::Pops::Types::PRubyType) && !key_factory.is_named?(key) + type_calculator.injectable_class(kt) + end + + def lookup_producer(scope, *args, &block) + raise ArgumentError, "lookup_producer should be called with two or three arguments, got: #{args.size()+1}" unless args.size <= 2 + + p = case args[ 0 ] + when Puppet::Pops::Types::PObjectType + lookup_producer_type(scope, *args) + + when String + raise ArgumentError, "lookup_producer of name should only pass the name" unless args.size == 1 + lookup_producer_key(scope, key_factory.data_key(args[ 0 ])) + + else + raise ArgumentError, "lookup_producer using a key should only pass a single key" unless args.size == 1 + lookup_producer_key(scope, args[ 0 ]) + end + + # call block with result if given + if block + case block.arity + when 1 + block.call(p) + when 2 + block.call(scope, p) + else + raise ArgumentError, "The block should have arity 1 or 2" + end + else + p + end + end + + # @api private + def lookup_producer_key(scope, key) + if @recursion_lock.include?(key) + raise ArgumentError, "Lookup loop detected for key: #{key}" + end + begin + @recursion_lock.push(key) + producer(scope, get_entry(key), :multiple_use) + ensure + @recursion_lock.pop() + end + end + + # @api private + def lookup_producer_type(scope, type, name='') + lookup_producer_key(scope, named_key(type, name)) + end + + # Returns the producer for the entry + # @return [Puppet::Pops::Binder::Producers::Producer] the entry's producer. + # + # @api private + # + def producer(scope, entry, use) + return nil unless entry # not found + return entry.producer(scope) if entry.is_a?(Producers::AssistedInjectProducer) + unless entry.cached_producer + entry.cached_producer = transform(entry.binding.producer, scope, entry) + end + unless entry.cached_producer + raise ArgumentError, "Injector entry without a producer #{format_binding(entry.binding)}" + end + entry.cached_producer.producer(scope) + end + + # @api private + def transform(producer_descriptor, scope, entry) + @@transform_visitor.visit_this(self, producer_descriptor, scope, entry) + end + + # Returns the produced instance + # @return [Object] the produced instance + # @api private + # + def produce(scope, entry) + return nil unless entry # not found + producer(scope, entry, :single_use).produce(scope) + end + + # @api private + def named_arguments_to_hash(named_args) + nb = named_args.nil? ? [] : named_args + result = {} + nb.each {|arg| result[ :"#{arg.name}" ] = arg.value } + result + end + + # @api private + def merge_producer_options(binding, options) + named_arguments_to_hash(binding.producer_args).merge(options) + end + + # @api private + def format_binding(b) + Puppet::Pops::Binder::Binder.format_binding(b) + end + + # Handles a missing producer (which is valid for a Multibinding where one is selected automatically) + # @api private + # + def transform_NilClass(descriptor, scope, entry) + unless entry.binding.is_a?(Puppet::Pops::Binder::Bindings::Multibinding) + raise ArgumentError, "Binding without producer detected, #{format_binding(entry.binding)}" + end + case entry.binding.type + when Puppet::Pops::Types::PArrayType + transform(Puppet::Pops::Binder::Bindings::ArrayMultibindProducerDescriptor.new(), scope, entry) + when Puppet::Pops::Types::PHashType + transform(Puppet::Pops::Binder::Bindings::HashMultibindProducerDescriptor.new(), scope, entry) + else + raise ArgumentError, "Unsupported multibind type, must be an array or hash type, #{format_binding(entry.binding)}" + end + end + + # @api private + def transform_ArrayMultibindProducerDescriptor(descriptor, scope, entry) + make_producer(Producers::ArrayMultibindProducer, descriptor, scope, entry, named_arguments_to_hash(entry.binding.producer_args)) + end + + # @api private + def transform_HashMultibindProducerDescriptor(descriptor, scope, entry) + make_producer(Producers::HashMultibindProducer, descriptor, scope, entry, named_arguments_to_hash(entry.binding.producer_args)) + end + + # @api private + def transform_ConstantProducerDescriptor(descriptor, scope, entry) + producer_class = singleton?(descriptor) ? Producers::SingletonProducer : Producers::DeepCloningProducer + producer_class.new(self, entry.binding, scope, merge_producer_options(entry.binding, {:value => descriptor.value})) + end + + # @api private + def transform_InstanceProducerDescriptor(descriptor, scope, entry) + make_producer(Producers::InstantiatingProducer, descriptor, scope, entry, + merge_producer_options(entry.binding, {:class_name => descriptor.class_name, :init_args => descriptor.arguments})) + end + + # @api private + def transform_EvaluatingProducerDescriptor(descriptor, scope, entry) + make_producer(Producers::EvaluatingProducer, descriptor, scope, entry, + merge_producer_options(entry.binding, {:expression => descriptor.expression})) + end + + # @api private + def make_producer(clazz, descriptor, scope, entry, options) + singleton_wrapped(descriptor, scope, entry, clazz.new(self, entry.binding, scope, options)) + end + + # @api private + def singleton_wrapped(descriptor, scope, entry, producer) + return producer unless singleton?(descriptor) + Producers::SingletonProducer.new(self, entry.binding, scope, + merge_producer_options(entry.binding, {:value => producer.produce(scope)})) + end + + # @api private + def transform_ProducerProducerDescriptor(descriptor, scope, entry) + p = transform(descriptor.producer, scope, entry) + clazz = singleton?(descriptor) ? Producers::SingletonProducerProducer : Producers::ProducerProducer + clazz.new(self, entry.binding, scope, merge_producer_options(entry.binding, + merge_producer_options(entry.binding, { :producer_producer => p }))) + end + + # @api private + def transform_LookupProducerDescriptor(descriptor, scope, entry) + make_producer(Producers::LookupProducer, descriptor, scope, entry, + merge_producer_options(entry.binding, {:type => descriptor.type, :name => descriptor.name})) + end + + # @api private + def transform_HashLookupProducerDescriptor(descriptor, scope, entry) + make_producer(Producers::LookupKeyProducer, descriptor, scope, entry, + merge_producer_options(entry.binding, {:type => descriptor.type, :name => descriptor.name, :key => descriptor.key})) + end + + # @api private + def transform_NonCachingProducerDescriptor(descriptor, scope, entry) + # simply delegates to the wrapped producer + transform(descriptor.producer, scope, entry) + end + + # @api private + def transform_FirstFoundProducerDescriptor(descriptor, scope, entry) + make_producer(Producers::FirstFoundProducer, descriptor, scope, entry, + merge_producer_options(entry.binding, {:producers => descriptor.producers.collect {|p| transform(p, scope, entry) }})) + end + + # @api private + def singleton?(descriptor) + ! descriptor.eContainer().is_a?(Puppet::Pops::Binder::Bindings::NonCachingProducerDescriptor) + end + + # Special marker class used in entries + # @api private + class NotFound + end + end +end +end
\ No newline at end of file diff --git a/lib/puppet/pops/binder/injector_entry.rb b/lib/puppet/pops/binder/injector_entry.rb new file mode 100644 index 000000000..f61773706 --- /dev/null +++ b/lib/puppet/pops/binder/injector_entry.rb @@ -0,0 +1,53 @@ +# Represents an entry in the injectors internal data. +# +# @api public +# +class Puppet::Pops::Binder::InjectorEntry + # @return [Object] An opaque object representing the precedence + # @api public + attr_reader :precedence + + # @return [Puppet::Pops::Binder::Bindings::Binding] The binding for this entry + # @api public + attr_reader :binding + + # @api private + attr_accessor :resolved + + # @api private + attr_accessor :cached_producer + + # @api private + def initialize(precedence, binding) + @precedence = precedence.freeze + @binding = binding + @cached_producer = nil + end + + # Marks an overriding entry as resolved (if not an overriding entry, the marking has no effect). + # @api private + # + def mark_override_resolved() + @resolved = true + end + + # The binding is resolved if it is non-override, or if the override has been resolved + # @api private + # + def is_resolved?() + !binding.override || resolved + end + + def is_abstract? + binding.abstract + end + + # Compares against another InjectorEntry by comparing precedence. + # @param injector_entry [InjectorEntry] entry to compare against. + # @return [Integer] 1, if this entry has higher precedence, 0 if equal, and -1 if given entry has higher precedence. + # @api public + # + def <=> (injector_entry) + precedence <=> injector_entry.precedence + end +end diff --git a/lib/puppet/pops/binder/key_factory.rb b/lib/puppet/pops/binder/key_factory.rb new file mode 100644 index 000000000..e5d890c63 --- /dev/null +++ b/lib/puppet/pops/binder/key_factory.rb @@ -0,0 +1,61 @@ +# The KeyFactory is responsible for creating keys used for lookup of bindings. +# @api public +# +class Puppet::Pops::Binder::KeyFactory + + attr_reader :type_calculator + # @api public + def initialize(type_calculator = Puppet::Pops::Types::TypeCalculator.new()) + @type_calculator = type_calculator + end + + # @api public + def binding_key(binding) + named_key(binding.type, binding.name) + end + + # @api public + def named_key(type, name) + [(@type_calculator.assignable?(@type_calculator.data, type) ? @type_calculator.data : type), name] + end + + # @api public + def data_key(name) + [@type_calculator.data, name] + end + + # @api public + def is_contributions_key?(s) + return false unless s.is_a?(String) + s.start_with?('mc_') + end + + # @api public + def multibind_contributions(multibind_id) + "mc_#{multibind_id}" + end + + # @api public + def is_named?(key) + key.is_a?(Array) && key[1] && !key[1].empty? + end + + # @api public + def is_data?(key) + return false unless key.is_a?(Array) && key[0].is_a?(Puppet::Pops::Types::PObjectType) + type_calculator.assignable?(type_calculator.data(), key[0]) + end + + # @api public + def is_ruby?(key) + return key.is_a?(Array) && key[0].is_a?(Puppet::Pops::Types::PRubyType) + end + + # Returns the type of the key + # @api public + # + def get_type(key) + return nil unless key.is_a?(Array) + key[0] + end +end
\ No newline at end of file diff --git a/lib/puppet/pops/binder/producers.rb b/lib/puppet/pops/binder/producers.rb new file mode 100644 index 000000000..2cd8fb6db --- /dev/null +++ b/lib/puppet/pops/binder/producers.rb @@ -0,0 +1,829 @@ +# This module contains the various producers used by Puppet Bindings. +# The main (abstract) class is {Puppet::Pops::Binder::Producers::Producer} which documents the +# Producer API and serves as a base class for all other producers. +# It is required that custom producers inherit from this producer (directly or indirectly). +# +# The selection of a Producer is typically performed by the Innjector when it configures itself +# from a Bindings model where a {Puppet::Pops::Binder::Bindings::ProducerDescriptor} describes +# which producer to use. The configuration uses this to create the concrete producer. +# It is possible to describe that a particular producer class is to be used, and also to describe that +# a custom producer (derived from Producer) should be used. This is available for both regular +# bindings as well as multi-bindings. +# +# +# @api public +# +module Puppet::Pops::Binder::Producers + # Producer is an abstract base class representing the base contract for a bound producer. + # Typically, when a lookup is performed it is the value that is returned (via a producer), but + # it is also possible to lookup the producer, and ask it to produce the value (the producer may + # return a series of values, which makes this especially useful). + # + # When looking up a producer, it is of importance to only use the API of the Producer class + # unless it is known that a particular custom producer class has been bound. + # + # Custom Producers + # ---------------- + # The intent is that this class is derived for custom producers that require additional + # options/arguments when producing an instance. Such a custom producer may raise an error if called + # with too few arguments, or may implement specific `produce` methods and always raise an + # error on #produce indicating that this producer requires custom calls and that it can not + # be used as an implicit producer. + # + # Features of Producer + # -------------------- + # The Producer class is abstract, but offers the ability to transform the produced result + # by passing the option `:transformer` which should be a Puppet Lambda Expression taking one argument + # and producing the transformed (wanted) result. + # + # @abstract + # @api public + # + class Producer + # A Puppet 3 AST Lambda Expression + # @api public + # + attr_reader :transformer + + # Creates a Producer. + # Derived classes should call this constructor to get support for transformer lambda. + # + # @param injector [Puppet::Pops::Binder::Injector] The injector where the lookup originates + # @param binding [Puppet::Pops::Binder::Bindings::Binding, nil] The binding using this producer + # @param scope [Puppet::Parser::Scope] The scope to use for evaluation + # @option options [Puppet::Pops::Model::LambdaExpression] :transformer (nil) a transformer of produced value + # @api public + # + def initialize(injector, binding, scope, options) + if transformer_lambda = options[:transformer] + if transformer_lambda.is_a?(Proc) + raise ArgumentError, "Transformer Proc must take two arguments; scope, value." unless transformer_lambda.arity == 2 + @transformer = transformer_lambda + else + raise ArgumentError, "Transformer must be a LambdaExpression" unless transformer_lambda.is_a?(Puppet::Pops::Model::LambdaExpression) + raise ArgumentError, "Transformer lambda must take one argument; value." unless transformer_lambda.parameters.size() == 1 + # NOTE: This depends on Puppet 3 AST Lambda + @transformer = Puppet::Pops::Model::AstTransformer.new().transform(transformer_lambda) + end + end + end + + # Produces an instance. + # @param scope [Puppet::Parser:Scope] the scope to use for evaluation + # @param args [Object] arguments to custom producers, always empty for implicit productions + # @return [Object] the produced instance (should never be nil). + # @api public + # + def produce(scope, *args) + do_transformation(scope, internal_produce(scope)) + end + + # Returns the producer after possibly having recreated an internal/wrapped producer. + # This implementation returns `self`. A derived class may want to override this method + # to perform initialization/refresh of its internal state. This method is called when + # a producer is requested. + # @see Puppet::Pops::Binder::ProducerProducer for an example of implementation. + # @param scope [Puppet::Parser:Scope] the scope to use for evaluation + # @return [Puppet::Pops::Binder::Producer] the producer to use + # @api public + # + def producer(scope) + self + end + + protected + + # Derived classes should implement this method to do the production of a value + # @param scope [Puppet::Parser::Scope] the scope to use when performing lookup and evaluation + # @raise [NotImplementedError] this implementation always raises an error + # @abstract + # @api private + # + def internal_produce(scope) + raise NotImplementedError, "Producer-class '#{self.class.name}' should implement #internal_produce(scope)" + end + + # Transforms the produced value if a transformer has been defined. + # @param scope [Puppet::Parser::Scope] the scope used for evaluation + # @param produced_value [Object, nil] the produced value (possibly nil) + # @return [Object] the transformed value if a transformer is defined, else the given `produced_value` + # @api private + # + def do_transformation(scope, produced_value) + return produced_value unless transformer + produced_value = :undef if produced_value.nil? + transformer.call(scope, produced_value) + end + end + + # Abstract Producer holding a value + # @abstract + # @api public + # + class AbstractValueProducer < Producer + + # @api public + attr_reader :value + + # @param injector [Puppet::Pops::Binder::Injector] The injector where the lookup originates + # @param binding [Puppet::Pops::Binder::Bindings::Binding, nil] The binding using this producer + # @param scope [Puppet::Parser::Scope] The scope to use for evaluation + # @option options [Puppet::Pops::Model::LambdaExpression] :transformer (nil) a transformer of produced value + # @option options [Puppet::Pops::Model::LambdaExpression, nil] :value (nil) the value to produce + # @api public + # + def initialize(injector, binding, scope, options) + super + # nil is ok here, as an abstract value producer may be used to signal "not found" + @value = options[:value] + end + end + + # Produces the same/singleton value on each production + # @api public + # + class SingletonProducer < AbstractValueProducer + protected + + # @api private + def internal_produce(scope) + value() + end + end + + # Produces a deep clone of its value on each production. + # @api public + # + class DeepCloningProducer < AbstractValueProducer + protected + + # @api private + def internal_produce(scope) + case value + when Integer, Float, TrueClass, FalseClass, Symbol + # These are immutable + return value + when String + # ok if frozen, else fall through to default + return value() if value.frozen? + end + # The default: serialize/deserialize to get a deep copy + Marshal.load(Marshal.dump(value())) + end + end + + # This abstract producer class remembers the injector and binding. + # @abstract + # @api public + # + class AbstractArgumentedProducer < Producer + + # @api public + attr_reader :injector + + # @api public + attr_reader :binding + + # @param injector [Puppet::Pops::Binder::Injector] The injector where the lookup originates + # @param binding [Puppet::Pops::Binder::Bindings::Binding, nil] The binding using this producer + # @param scope [Puppet::Parser::Scope] The scope to use for evaluation + # @option options [Puppet::Pops::Model::LambdaExpression] :transformer (nil) a transformer of produced value + # @api public + # + def initialize(injector, binding, scope, options) + super + @injector = injector + @binding = binding + end + end + + # @api public + class InstantiatingProducer < AbstractArgumentedProducer + + # @api public + attr_reader :the_class + + # @api public + attr_reader :init_args + + # @param injector [Puppet::Pops::Binder::Injector] The injector where the lookup originates + # @param binding [Puppet::Pops::Binder::Bindings::Binding, nil] The binding using this producer + # @param scope [Puppet::Parser::Scope] The scope to use for evaluation + # @option options [Puppet::Pops::Model::LambdaExpression] :transformer (nil) a transformer of produced value + # @option options [String] :class_name The name of the class to create instance of + # @option options [Array<Object>] :init_args ([]) Optional arguments to class constructor + # @api public + # + def initialize(injector, binding, scope, options) + # Better do this, even if a transformation of a created instance is kind of an odd thing to do, one can imagine + # sending it to a function for further detailing. + # + super + class_name = options[:class_name] + raise ArgumentError, "Option 'class_name' must be given for an InstantiatingProducer" unless class_name + # get class by name + @the_class = Puppet::Pops::Types::ClassLoader.provide(class_name) + @init_args = options[:init_args] || [] + raise ArgumentError, "Can not load the class #{class_name} specified in binding named: '#{binding.name}'" unless @the_class + end + + protected + + # Performs initialization the same way as Assisted Inject does (but handle arguments to + # constructor) + # @api private + # + def internal_produce(scope) + result = nil + # A class :inject method wins over an instance :initialize if it is present, unless a more specific + # constructor exists. (i.e do not pick :inject from superclass if class has a constructor). + # + if the_class.respond_to?(:inject) + inject_method = the_class.method(:inject) + initialize_method = the_class.instance_method(:initialize) + if inject_method.owner <= initialize_method.owner + result = the_class.inject(injector, scope, binding, *init_args) + end + end + if result.nil? + result = the_class.new(*init_args) + end + result + end + end + + # @api public + class FirstFoundProducer < Producer + # @api public + attr_reader :producers + + # @param injector [Puppet::Pops::Binder::Injector] The injector where the lookup originates + # @param binding [Puppet::Pops::Binder::Bindings::Binding, nil] The binding using this producer + # @param scope [Puppet::Parser::Scope] The scope to use for evaluation + # @option options [Puppet::Pops::Model::LambdaExpression] :transformer (nil) a transformer of produced value + # @option options [Array<Puppet::Pops::Binder::Producers::Producer>] :producers list of producers to consult. Required. + # @api public + # + def initialize(injector, binding, scope, options) + super + @producers = options[:producers] + raise ArgumentError, "Option :producers' must be set to a list of producers." if @producers.nil? + raise ArgumentError, "Given 'producers' option is not an Array" unless @producers.is_a?(Array) + end + + protected + + # @api private + def internal_produce(scope) + # return the first produced value that is non-nil (unfortunately there is no such enumerable method) + producers.reduce(nil) {|memo, p| break memo unless memo.nil?; p.produce(scope)} + end + end + + # Evaluates a Puppet Expression and returns the result. + # This is typically used for strings with interpolated expressions. + # @api public + # + class EvaluatingProducer < Producer + # A Puppet 3 AST Expression + # @api public + # + attr_reader :expression + + # @param injector [Puppet::Pops::Binder::Injector] The injector where the lookup originates + # @param binding [Puppet::Pops::Binder::Bindings::Binding, nil] The binding using this producer + # @param scope [Puppet::Parser::Scope] The scope to use for evaluation + # @option options [Puppet::Pops::Model::LambdaExpression] :transformer (nil) a transformer of produced value + # @option options [Array<Puppet::Pops::Model::Expression>] :expression The expression to evaluate + # @api public + # + def initialize(injector, binding, scope, options) + super + expr = options[:expression] + raise ArgumentError, "Option 'expression' must be given to an EvaluatingProducer." unless expr + @expression = Puppet::Pops::Model::AstTransformer.new().transform(expr) + end + + # @api private + def internal_produce(scope) + expression.evaluate(scope) + end + end + + # @api public + class LookupProducer < AbstractArgumentedProducer + + # @api public + attr_reader :type + + # @api public + attr_reader :name + + # @param injector [Puppet::Pops::Binder::Injector] The injector where the lookup originates + # @param binding [Puppet::Pops::Binder::Bindings::Binding, nil] The binding using this producer + # @param scope [Puppet::Parser::Scope] The scope to use for evaluation + # @option options [Puppet::Pops::Model::LambdaExpression] :transformer (nil) a transformer of produced value + # @option options [Puppet::Pops::Types::PObjectType] :type The type to lookup + # @option options [String] :name ('') The name to lookup + # @api public + # + def initialize(injector, binder, scope, options) + super + @type = options[:type] + @name = options[:name] || '' + raise ArgumentError, "Option 'type' must be given in a LookupProducer." unless @type + end + + protected + + # @api private + def internal_produce(scope) + injector.lookup_type(scope, type, name) + end + end + + # @api public + class LookupKeyProducer < LookupProducer + + # @api public + attr_reader :key + + # @param injector [Puppet::Pops::Binder::Injector] The injector where the lookup originates + # @param binding [Puppet::Pops::Binder::Bindings::Binding, nil] The binding using this producer + # @param scope [Puppet::Parser::Scope] The scope to use for evaluation + # @option options [Puppet::Pops::Model::LambdaExpression] :transformer (nil) a transformer of produced value + # @option options [Puppet::Pops::Types::PObjectType] :type The type to lookup + # @option options [String] :name ('') The name to lookup + # @option options [Puppet::Pops::Types::PObjectType] :key The key to lookup in the hash + # @api public + # + def initialize(injector, binder, scope, options) + super + @key = options[:key] + raise ArgumentError, "Option 'key' must be given in a LookupKeyProducer." if key.nil? + end + + protected + + # @api private + def internal_produce(scope) + + result = super + result.is_a?(Hash) ? result[key] : nil + end + end + + # Produces the given producer, then uses that producer. + # @see ProducerProducer for the non singleton version + # @api public + # + class SingletonProducerProducer < Producer + + # @api public + attr_reader :value_producer + + # @param injector [Puppet::Pops::Binder::Injector] The injector where the lookup originates + # @param binding [Puppet::Pops::Binder::Bindings::Binding, nil] The binding using this producer + # @param scope [Puppet::Parser::Scope] The scope to use for evaluation + # @option options [Puppet::Pops::Model::LambdaExpression] :transformer (nil) a transformer of produced value + # @option options [Puppet::Pops::Model::LambdaExpression] :producer_producer a producer of a value producer (required) + # @api public + # + def initialize(injector, binding, scope, options) + super + p = options[:producer_producer] + raise ArgumentError, "Option :producer_producer must be given in a SingletonProducerProducer" unless p + @value_producer = p.produce(scope) + end + + protected + + # @api private + def internal_produce(scope) + value_producer.produce(scope) + end + end + + # A ProducerProducer creates a producer via another producer, and then uses this created producer + # to produce values. This is useful for custom production of series of values. + # On each request for a producer, this producer will reset its internal producer (i.e. restarting + # the series). + # + # @param producer_producer [#produce(scope)] the producer of the producer + # + # @api public + # + class ProducerProducer < Producer + + # @api public + attr_reader :producer_producer + + # @api public + attr_reader :value_producer + + # Creates new ProducerProducer given a producer. + # + # @param injector [Puppet::Pops::Binder::Injector] The injector where the lookup originates + # @param binding [Puppet::Pops::Binder::Bindings::Binding, nil] The binding using this producer + # @param scope [Puppet::Parser::Scope] The scope to use for evaluation + # @option options [Puppet::Pops::Model::LambdaExpression] :transformer (nil) a transformer of produced value + # @option options [Puppet::Pops::Binder::Producer] :producer_producer a producer of a value producer (required) + # + # @api public + # + def initialize(injector, binding, scope, options) + super + unless producer_producer = options[:producer_producer] + raise ArgumentError, "The option :producer_producer must be set in a ProducerProducer" + end + raise ArgumentError, "Argument must be a Producer" unless producer_producer.is_a?(Producer) + + @producer_producer = producer_producer + @value_producer = nil + end + + # Updates the internal state to use a new instance of the wrapped producer. + # @api public + # + def producer(scope) + @value_producer = @producer_producer.produce(scope) + self + end + + protected + + # Produces a value after having created an instance of the wrapped producer (if not already created). + # @api private + # + def internal_produce(scope, *args) + producer() unless value_producer + value_producer.produce(scope) + end + end + + # This type of producer should only be created by the Injector. + # + # @api private + # + class AssistedInjectProducer < Producer + # An Assisted Inject Producer is created when a lookup is made of a type that is + # not bound. It does not support a transformer lambda. + # @note This initializer has a different signature than all others. Do not use in regular logic. + # @api private + # + def initialize(injector, clazz) + raise ArgumentError, "class must be given" unless clazz.is_a?(Class) + + @injector = injector + @clazz = clazz + @inst = nil + end + + def produce(scope, *args) + producer(scope, *args) unless @inst + @inst + end + + # @api private + def producer(scope, *args) + @inst = nil + # A class :inject method wins over an instance :initialize if it is present, unless a more specific zero args + # constructor exists. (i.e do not pick :inject from superclass if class has a zero args constructor). + # + if @clazz.respond_to?(:inject) + inject_method = @clazz.method(:inject) + initialize_method = @clazz.instance_method(:initialize) + if inject_method.owner <= initialize_method.owner || initialize_method.arity != 0 + @inst = @clazz.inject(@injector, scope, nil, *args) + end + end + if @inst.nil? + unless args.empty? + raise ArgumentError, "Assisted Inject can not pass arguments to no-args constructor when there is no class inject method." + end + @inst = @clazz.new() + end + self + end + end + + # Abstract base class for multibind producers. + # Is suitable as base class for custom implementations of multibind producers. + # @abstract + # @api public + # + class MultibindProducer < AbstractArgumentedProducer + attr_reader :contributions_key + + # @param injector [Puppet::Pops::Binder::Injector] The injector where the lookup originates + # @param binding [Puppet::Pops::Binder::Bindings::Binding, nil] The binding using this producer + # @param scope [Puppet::Parser::Scope] The scope to use for evaluation + # @option options [Puppet::Pops::Model::LambdaExpression] :transformer (nil) a transformer of produced value + # + # @api public + # + def initialize(injector, binding, scope, options) + super + @contributions_key = injector.key_factory.multibind_contributions(binding.id) + end + + # @param expected [Array<Puppet::Pops::Types::PObjectType>, Puppet::Pops::Types::PObjectType] expected type or types + # @param actual [Object, Puppet::Pops::Types::PObjectType> the actual value (or its type) + # @return [String] a formatted string for inclusion as detail in an error message + # @api private + # + def type_error_detail(expected, actual) + tc = injector.type_calculator + expected = [expected] unless expected.is_a?(Array) + actual_t = tc.is_ptype?(actual) ? actual : tc.infer(actual) + expstrs = expected.collect {|t| tc.string(t) } + "expected: #{expstrs.join(', or ')}, got: #{tc.string(actual_t)}" + end + end + + # A configurable multibind producer for Array type multibindings. + # + # This implementation collects all contributions to the multibind and then combines them using the following rules: + # + # - all *unnamed* entries are added unless the option `:priority_on_unnamed` is set to true, in which case the unnamed + # contribution with the highest priority is added, and the rest are ignored (unless they have the same priority in which + # case an error is raised). + # - all *named* entries are handled the same way as *unnamed* but the option `:priority_on_named` controls their handling. + # - the option `:uniq` post processes the result to only contain unique entries + # - the option `:flatten` post processes the result by flattening all nested arrays. + # - If both `:flatten` and `:uniq` are true, flattening is done first. + # + # @note + # Collection accepts elements that comply with the array's element type, or the entire type (i.e. Array[element_type]). + # If the type is restrictive - e.g. Array[String] and an Array[String] is contributed, the result will not be type + # compliant without also using the `:flatten` option, and a type error will be raised. For an array with relaxed typing + # i.e. Array[Data], it it valid to produce a result such as `['a', ['b', 'c'], 'd']` and no flattening is required + # and no error is raised (but using the array needs to be aware of potential array, non-array entries. + # The use of the option `:flatten` controls how the result is flattened. + # + # @api public + # + class ArrayMultibindProducer < MultibindProducer + + # @return [Boolean] whether the result should be made contain unique (non-equal) entries or not + # @api public + attr_reader :uniq + + # @return [Boolean, Integer] If result should be flattened (true), or not (false), or flattened to given level (0 = none, -1 = all) + # @api public + attr_reader :flatten + + # @return [Boolean] whether priority should be considered for named contributions + # @api public + attr_reader :priority_on_named + + # @return [Boolean] whether priority should be considered for unnamed contributions + # @api public + attr_reader :priority_on_unnamed + + # @param injector [Puppet::Pops::Binder::Injector] The injector where the lookup originates + # @param binding [Puppet::Pops::Binder::Bindings::Binding, nil] The binding using this producer + # @param scope [Puppet::Parser::Scope] The scope to use for evaluation + # @option options [Puppet::Pops::Model::LambdaExpression] :transformer (nil) a transformer of produced value + # @option options [Boolean] :uniq (false) if collected result should be post-processed to contain only unique entries + # @option options [Boolean, Integer] :flatten (false) if collected result should be post-processed so all contained arrays + # are flattened. May be set to an Integer value to indicate the level of recursion (-1 is endless, 0 is none). + # @option options [Boolean] :priority_on_named (true) if highest precedented named element should win or if all should be included + # @option options [Boolean] :priority_on_unnamed (false) if highest precedented unnamed element should win or if all should be included + # @api public + # + def initialize(injector, binding, scope, options) + super + @uniq = !!options[:uniq] + @flatten = options[:flatten] + @priority_on_named = options[:priority_on_named].nil? ? true : options[:priority_on_name] + @priority_on_unnamed = !!options[:priority_on_unnamed] + + case @flatten + when Integer + when true + @flatten = -1 + when false + @flatten = nil + when NilClass + @flatten = nil + else + raise ArgumentError, "Option :flatten must be nil, Boolean, or an integer value" unless @flatten.is_a?(Integer) + end + end + + protected + + # @api private + def internal_produce(scope) + seen = {} + included_keys = [] + + injector.get_contributions(scope, contributions_key).each do |element| + key = element[0] + entry = element[1] + + name = entry.binding.name + existing = seen[name] + empty_name = name.nil? || name.empty? + if existing + if empty_name && priority_on_unnamed + if (seen[name] <=> entry) >= 0 + raise ArgumentError, "Duplicate key (same priority) contributed to Array Multibinding '#{binding.name}' with unnamed entry." + end + next + elsif !empty_name && priority_on_named + if (seen[name] <=> entry) >= 0 + raise ArgumentError, "Duplicate key (same priority) contributed to Array Multibinding '#{binding.name}', key: '#{name}'." + end + next + end + else + seen[name] = entry + end + included_keys << key + end + result = included_keys.collect do |k| + x = injector.lookup_key(scope, k) + assert_type(binding(), injector.type_calculator(), x) + x + end + + result.flatten!(flatten) if flatten + result.uniq! if uniq + result + end + + # @api private + def assert_type(binding, tc, value) + infered = tc.infer(value) + unless tc.assignable?(binding.type.element_type, infered) || tc.assignable?(binding.type, infered) + raise ArgumentError, ["Type Error: contribution to '#{binding.name}' does not match type of multibind, ", + "#{type_error_detail([binding.type.element_type, binding.type], value)}"].join() + end + end + end + + # @api public + class HashMultibindProducer < MultibindProducer + + # @return [Symbol] One of `:error`, `:merge`, `:append`, `:priority`, `:ignore` + # @api public + attr_reader :conflict_resolution + + # @return [Boolean] + # @api public + attr_reader :uniq + + # @return [Boolean, Integer] Flatten all if true, or none if false, or to given level (0 = none, -1 = all) + # @api public + attr_reader :flatten + + # The hash multibind producer provides options to control conflict resolution. + # By default, the hash is produced using `:priority` resolution - the highest entry is selected, the rest are + # ignored unless they have the same priority which is an error. + # + # @param injector [Puppet::Pops::Binder::Injector] The injector where the lookup originates + # @param binding [Puppet::Pops::Binder::Bindings::Binding, nil] The binding using this producer + # @param scope [Puppet::Parser::Scope] The scope to use for evaluation + # @option options [Puppet::Pops::Model::LambdaExpression] :transformer (nil) a transformer of produced value + # @option options [Symbol, String] :conflict_resolution (:priority) One of `:error`, `:merge`, `:append`, `:priority`, `:ignore` + # <ul><li> `ignore` the first found highest priority contribution is used, the rest are ignored</li> + # <li>`error` any duplicate key is an error</li> + # <li>`append` element type must be compatible with Array, makes elements be arrays and appends all found</li> + # <li>`merge` element type must be compatible with hash, merges hashes with retention of highest priority hash content</li> + # <li>`priority` the first found highest priority contribution is used, duplicates with same priority raises and error, the rest are + # ignored.</li></ul> + # @option options [Boolean, Integer] :flatten (false) If appended should be flattened. Also see {#flatten}. + # @option options [Boolean] :uniq (false) If appended result should be made unique. + # + # @api public + # + def initialize(injector, binding, scope, options) + super + @conflict_resolution = options[:conflict_resolution].nil? ? :priority : options[:conflict_resolution] + @uniq = !!options[:uniq] + @flatten = options[:flatten] + + unless [:error, :merge, :append, :priority, :ignore].include?(@conflict_resolution) + raise ArgumentError, "Unknown conflict_resolution for Multibind Hash: '#{@conflict_resolution}." + end + + case @flatten + when Integer + when true + @flatten = -1 + when false + @flatten = nil + when NilClass + @flatten = nil + else + raise ArgumentError, "Option :flatten must be nil, Boolean, or an integer value" unless @flatten.is_a?(Integer) + end + + if uniq || flatten || conflict_resolution.to_s == 'append' + etype = binding.type.element_type + unless etype.class == Puppet::Pops::Types::PDataType || etype.is_a?(Puppet::Pops::Types::PArrayType) + detail = [] + detail << ":uniq" if uniq + detail << ":flatten" if flatten + detail << ":conflict_resolution => :append" if conflict_resolution.to_s == 'append' + raise ArgumentError, ["Options #{detail.join(', and ')} cannot be used with a Multibind ", + "of type #{injector.type_calculator.string(binding.type)}"].join() + end + end + end + + protected + + # @api private + def internal_produce(scope) + seen = {} + included_entries = [] + + injector.get_contributions(scope, contributions_key).each do |element| + key = element[0] + entry = element[1] + + name = entry.binding.name + raise ArgumentError, "A Hash Multibind contribution to '#{binding.name}' must have a name." if name.nil? || name.empty? + + existing = seen[name] + if existing + case conflict_resolution.to_s + when 'priority' + # skip if duplicate has lower prio + if (comparison = (seen[name] <=> entry)) <= 0 + raise ArgumentError, "Internal Error: contributions not given in decreasing precedence order" unless comparison == 0 + raise ArgumentError, "Duplicate key (same priority) contributed to Hash Multibinding '#{binding.name}', key: '#{name}'." + end + next + + when 'ignore' + # skip, ignore conflict if prio is the same + next + + when 'error' + raise ArgumentError, "Duplicate key contributed to Hash Multibinding '#{binding.name}', key: '#{name}'." + + end + else + seen[name] = entry + end + included_entries << [key, entry] + end + result = {} + included_entries.each do |element| + k = element[ 0 ] + entry = element[ 1 ] + x = injector.lookup_key(scope, k) + name = entry.binding.name + assert_type(binding(), injector.type_calculator(), name, x) + if result[ name ] + merge(result, name, result[ name ], x) + else + result[ name ] = conflict_resolution().to_s == 'append' ? [x] : x + end + end + result + end + + # @api private + def merge(result, name, higher, lower) + case conflict_resolution.to_s + when 'append' + unless higher.is_a?(Array) + higher = [higher] + end + tmp = higher + [lower] + tmp.flatten!(flatten) if flatten + tmp.uniq! if uniq + result[name] = tmp + + when 'merge' + result[name] = lower.merge(higher) + + end + end + + # @api private + def assert_type(binding, tc, key, value) + unless tc.instance?(binding.type.key_type, key) + raise ArgumentError, ["Type Error: key contribution to #{binding.name}['#{key}'] ", + "is incompatible with key type: #{tc.label(binding.type)}, ", + type_error_detail(binding.type.key_type, key)].join() + end + + if key.nil? || !key.is_a?(String) || key.empty? + raise ArgumentError, "Entry contributing to multibind hash with id '#{binding.id}' must have a name." + end + + unless tc.instance?(binding.type.element_type, value) + raise ArgumentError, ["Type Error: value contribution to #{binding.name}['#{key}'] ", + "is incompatible, ", + type_error_detail(binding.type.element_type, value)].join() + end + end + end + +end
\ No newline at end of file diff --git a/lib/puppet/pops/binder/scheme_handler/confdir_hiera_scheme.rb b/lib/puppet/pops/binder/scheme_handler/confdir_hiera_scheme.rb new file mode 100644 index 000000000..8be9f9018 --- /dev/null +++ b/lib/puppet/pops/binder/scheme_handler/confdir_hiera_scheme.rb @@ -0,0 +1,67 @@ +# Similar to {Puppet::Pops::Binder::SchemeHandler::ModuleHieraScheme ModuleHieraScheme} but path is +# relative to the `$confdir` instead of relative to a module root. +# +# Does not handle wild-cards. +# @api public +class Puppet::Pops::Binder::SchemeHandler::ConfdirHieraScheme < Puppetx::Puppet::BindingsSchemeHandler + + # (Puppetx::Puppet::BindingsSchemeHandler.contributed_bindings) + # + def contributed_bindings(uri, scope, composer) + split_path = uri.path.split('/') + name = split_path[1] + confdir = composer.confdir + provider = Puppet::Pops::Binder::Hiera2::BindingsProvider.new(uri.to_s, File.join(confdir, uri.path), composer.acceptor) + provider.load_bindings(scope) + end + + # This handler does not support wildcards. + # The given uri is simply returned in an array. + # @param uri [URI] the uri to expand + # @return [Array<URI>] the uri wrapped in an array + # @todo Handle optional and possibly hiera-1 hiera.yaml config file in the expected location (the same as missing) + # @api public + # + def expand_included(uri, composer) + result = [] + if config_exist?(uri, composer) + result << uri unless is_ignored_hiera_version?(uri, composer) + else + result << uri unless is_optional?(uri) + end + result + end + + # This handler does not support wildcards. + # The given uri is simply returned in an array. + # @param uri [URI] the uri to expand + # @return [Array<URI>] the uri wrapped in an array + # @api public + # + def expand_excluded(uri, composer) + [uri] + end + + def config_exist?(uri, composer) + File.exist?(File.join(composer.confdir, uri.path, 'hiera.yaml')) + end + + # A hiera.yaml that exists, is readable, can be loaded, and does not have version >= 2 set is ignored. + # All other conditions are reported as 'not ignored' even if there are errors; these will be handled later + # as if the hiera.yaml is a hiera-2 file. + # @api private + def is_ignored_hiera_version?(uri, composer) + config_file = File.join(composer.confdir, uri.path, 'hiera.yaml') + begin + data = YAML.load_file(config_file) + if data.is_a?(Hash) + ver = data[:version] || data['version'] + return ver.nil? || ver < 2 + end + rescue Errno::ENOENT + rescue Errno::ENOTDIR + rescue ::SyntaxError => e + end + return false + end +end diff --git a/lib/puppet/pops/binder/scheme_handler/confdir_scheme.rb b/lib/puppet/pops/binder/scheme_handler/confdir_scheme.rb new file mode 100644 index 000000000..aafbf7172 --- /dev/null +++ b/lib/puppet/pops/binder/scheme_handler/confdir_scheme.rb @@ -0,0 +1,34 @@ +require 'puppet/pops/binder/scheme_handler/symbolic_scheme' + +# Similar to {Puppet::Pops::Binder::SchemeHandler::ModuleScheme ModuleScheme}, but relative to the config root. +# Does not support wildcard expansion. +# +# URI +# --- +# The URI scheme is `confdir:/[<FQN>]['?' | [?optional]` where FQN is the fully qualified name of the bindings to load. +# The referecence is made optional by using a URI query of `?` or `?optional`. +# +# @todo +# If the file to load is outside of the file system rooted at $confdir (in a gem, or just on the Ruby path), it can not +# be marked as optional as it will always be ignored. +# +class Puppet::Pops::Binder::SchemeHandler::ConfdirScheme < Puppet::Pops::Binder::SchemeHandler::SymbolicScheme + + def expand_included(uri, composer) + fqn = fqn_from_path(uri)[1] + if is_optional?(uri) + if Puppet::Pops::Binder::BindingsLoader.loadable?(composer.confdir, fqn) + [URI.parse('confdir:/' + fqn)] + else + [] + end + else + # assume it exists (do not give error if not, since it may be excluded later) + [URI.parse('confdir:/' + fqn)] + end + end + + def expand_excluded(uri, composer) + [URI.parse("confdir:/#{fqn_from_path(uri)[1]}")] + end +end diff --git a/lib/puppet/pops/binder/scheme_handler/module_hiera_scheme.rb b/lib/puppet/pops/binder/scheme_handler/module_hiera_scheme.rb new file mode 100644 index 000000000..38f57d033 --- /dev/null +++ b/lib/puppet/pops/binder/scheme_handler/module_hiera_scheme.rb @@ -0,0 +1,92 @@ +# The `module-hiera:` scheme uses the path to denote a directory relative to a module root +# The path starts with the name of the module, or '*' to denote *any module*. +# +# @example All root hiera.yaml from all modules. +# module-hiera:/* +# +# @example The hiera.yaml from the module `foo`'s relative path `<foo root>/bar`. +# module-hiera:/foo/bar +# +class Puppet::Pops::Binder::SchemeHandler::ModuleHieraScheme < Puppetx::Puppet::BindingsSchemeHandler + # (Puppetx::Puppet::BindingsSchemeHandler.contributed_bindings) + # @api public + def contributed_bindings(uri, scope, composer) + split_path = uri.path.split('/') + name = split_path[1] + mod = composer.name_to_module[name] + provider = Puppet::Pops::Binder::Hiera2::BindingsProvider.new(uri.to_s, File.join(mod.path, split_path[ 2..-1 ]), composer.acceptor) + provider.load_bindings(scope) + end + + # Expands URIs with wildcards and checks optionality. + # @param uri [URI] the uri to possibly expand + # @return [Array<URI>] the URIs to include + # @api public + # + def expand_included(uri, composer) + result = [] + split_path = uri.path.split('/') + if split_path.size > 1 && split_path[-1].empty? + split_path.delete_at(-1) + end + + # 0 = "", since a URI with a path must start with '/' + # 1 = '*' or the module name + case split_path[ 1 ] + when '*' + # create new URIs, one per module name that has a hiera.yaml file relative to its root + composer.name_to_module.each_pair do | name, mod | + if File.exist?(File.join(mod.path, split_path[ 2..-1 ], 'hiera.yaml' )) + path_parts =["", name] + split_path[2..-1] + result << URI.parse('module-hiera:'+File.join(path_parts)) + end + end + when nil + raise ArgumentError, "Bad bindings uri, the #{uri} has neither module name or wildcard '*' in its first path position" + else + # If uri has query that is empty, or the text 'optional' skip this uri if it does not exist + if query = uri.query() + if query == '' || query == 'optional' + if File.exist?(File.join(mod.path, split_path[ 2..-1 ], 'hiera.yaml' )) + result << URI.parse('module-hiera:' + uri.path) + end + end + else + # assume it exists (do not give error since it may be excluded later) + result << URI.parse('module-hiera:' + File.join(split_path)) + end + end + result + end + + # Expands URIs with wildcards and checks optionality. + # @param uri [URI] the uri to possibly expand + # @return [Array<URI>] the URIs to exclude + # @api public + # + def expand_excluded(uri, composer) + result = [] + split_path = uri.path.split('/') + if split_path.size > 1 && split_path[-1].empty? + split_path.delete_at(-1) + end + + # 0 = "", since a URI with a path must start with '/' + # 1 = '*' or the module name + case split_path[ 1 ] + when '*' + # create new URIs, one per module name that has a hiera.yaml file relative to its root + composer.name_to_module.each_pair do | name, mod | + path_parts =["", mod.name] + split_path[2..-1] + result << URI.parse('module-hiera:'+File.join(path_parts)) + end + + when nil + raise ArgumentError, "Bad bindings uri, the #{uri} has neither module name or wildcard '*' in its first path position" + else + # create a clean copy (get rid of optional, fragments etc. and a trailing "/") + result << URI.parse('module-hiera:' + File.join(split_path)) + end + result + end +end diff --git a/lib/puppet/pops/binder/scheme_handler/module_scheme.rb b/lib/puppet/pops/binder/scheme_handler/module_scheme.rb new file mode 100644 index 000000000..b409f8a67 --- /dev/null +++ b/lib/puppet/pops/binder/scheme_handler/module_scheme.rb @@ -0,0 +1,84 @@ +require 'puppet/pops/binder/scheme_handler/symbolic_scheme' + +# The module scheme allows loading bindings using the Puppet autoloader. +# Optional uris are handled by checking if the symbolic name can be resolved to a loadable file +# from modules. +# +# URI +# --- +# The uri is on the format: `module:/[fqn][? | ?optional]` where `fqn` is a fully qualified bindings name +# starting with the module name or '*' to denote 'any module'. A URI query of `?` or `?optional` makes the +# request optional; if no loadable file is found, it is simply skipped. +# +# +# @todo +# Does currently only support checking of optionality against files under a module. If the result should be loaded +# from any other location it can not be marked as optional as it will be ignored. +# +class Puppet::Pops::Binder::SchemeHandler::ModuleScheme < Puppet::Pops::Binder::SchemeHandler::SymbolicScheme + + # Expands URIs with wildcards and checks optionality. + # @param uri [URI] the uri to possibly expand + # @return [Array<URI>] the URIs to include + # @api public + # + def expand_included(uri, composer) + result = [] + split_name, fqn = fqn_from_path(uri) + + # supports wild card in the module name + case split_name[0] + when '*' + # create new URIs, one per module name that has a corresponding .rb file relative to its + # '<root>/lib/puppet/bindings/' + # + composer.name_to_module.each_pair do | mod_name, mod | + expanded_name_parts = [mod_name] + split_name[1..-1] + expanded_name = expanded_name_parts.join('::') + if Puppet::Pops::Binder::BindingsLoader.loadable?(mod.path, expanded_name) + result << URI.parse('module:/' + expanded_name) + end + end + when nil + raise ArgumentError, "Bad bindings uri, the #{uri} has neither module name or wildcard '*' in its first path position" + else + joined_name = split_name.join('::') + # skip optional uri if it does not exist + if is_optional?(uri) + mod = composer.name_to_module[split_name[0]] + if mod && Puppet::Binder::BindingsLoader.loadable?(mod.path, joined_name) + result << URI.parse('module:/' + joined_name) + end + else + # assume it exists (do not give error if not, since it may be excluded later) + result << URI.parse('module:/' + joined_name) + end + end + result + end + + # Expands URIs with wildcards + # @param uri [URI] the uri to possibly expand + # @return [Array<URI>] the URIs to exclude + # @api public + # + def expand_excluded(uri, composer) + result = [] + split_name, fqn = fqn_from_path(uri) + + case split_name[ 0 ] + when '*' + # create new URIs, one per module name + composer.name_to_module.each_pair do | name, mod | + result << URI.parse('module:/' + ([name] + split_name).join('::')) + end + + when nil + raise ArgumentError, "Bad bindings uri, the #{uri} has neither module name or wildcard '*' in its first path position" + else + # create a clean copy (get rid of optional, fragments etc. and any trailing stuff + result << URI.parse('module:/' + split_name.join('::')) + end + result + end +end diff --git a/lib/puppet/pops/binder/scheme_handler/symbolic_scheme.rb b/lib/puppet/pops/binder/scheme_handler/symbolic_scheme.rb new file mode 100644 index 000000000..af8c9c786 --- /dev/null +++ b/lib/puppet/pops/binder/scheme_handler/symbolic_scheme.rb @@ -0,0 +1,54 @@ +# Abstract base class for schemes based on symbolic names of bindings. +# This class helps resolve symbolic names by computing a path from a fully qualified name (fqn). +# There are also helper methods do determine if the symbolic name contains a wild-card ('*') in the first +# portion of the fqn (with the convention that '*' means 'any module'). +# +# @abstract +# @api public +# +class Puppet::Pops::Binder::SchemeHandler::SymbolicScheme < Puppetx::Puppet::BindingsSchemeHandler + + # Shared implementation for module: and confdir: since the distinction is only in checks if a symbolic name + # exists as a loadable file or not. Once this method is called it is assumed that the name is relative + # and that it should exist relative to some loadable ruby location. + # + # TODO: this needs to be changed once ARM-8 Puppet DSL concrete syntax is also supported. + # @api public + # + def contributed_bindings(uri, scope, composer) + fqn = fqn_from_path(uri)[1] + bindings = Puppet::Pops::Binder::BindingsLoader.provide(scope, fqn) + raise ArgumentError, "Cannot load bindings '#{uri}' - no bindings found." unless bindings + # Must clone as the the rest mutates the model + cloned_bindings = Marshal.load(Marshal.dump(bindings)) + # Give no effective categories (i.e. ok with whatever categories there is) + Puppet::Pops::Binder::BindingsFactory.contributed_bindings(fqn, cloned_bindings, nil) + end + + # @api private + def fqn_from_path(uri) + split_path = uri.path.split('/') + if split_path.size > 1 && split_path[-1].empty? + split_path.delete_at(-1) + end + + fqn = split_path[ 1 ] + raise ArgumentError, "Module scheme binding reference has no name." unless fqn + split_name = fqn.split('::') + # drop leading '::' + split_name.shift if split_name[0] && split_name[0].empty? + [split_name, split_name.join('::')] + end + + # True if super thinks it is optional or if it contains a wildcard. + # @return [Boolean] true if the uri represents an optional set of bindings. + # @api public + def is_optional?(uri) + super(uri) || has_wildcard?(uri) + end + + # @api private + def has_wildcard?(uri) + (path = uri.path) && path.split('/')[1].start_with?('*::') + end +end diff --git a/lib/puppet/pops/binder/system_bindings.rb b/lib/puppet/pops/binder/system_bindings.rb new file mode 100644 index 000000000..557074825 --- /dev/null +++ b/lib/puppet/pops/binder/system_bindings.rb @@ -0,0 +1,72 @@ +class Puppet::Pops::Binder::SystemBindings + # Constant with name used for bindings used during initialization of injector + ENVIRONMENT_BOOT_BINDINGS_NAME = 'puppet::env::injector::boot' + + Factory = Puppet::Pops::Binder::BindingsFactory + @extension_bindings = Factory.named_bindings("puppet::extensions") + @default_bindings = Factory.named_bindings("puppet::default") + # Bindings in effect when real injector is created + @injector_boot_bindings = Factory.named_bindings("puppet::injector_boot") + + def self.extensions() + @extension_bindings + end + + def self.default_bindings() + @default_bindings + end + + def self.injector_boot_bindings() + @injector_boot_bindings + end + +# def self.env_boot_bindings() +# Puppet::Bindings[Puppet::Pops::Binder::SystemBindings::ENVIRONMENT_BOOT_BINDINGS_NAME] +# end + + def self.final_contribution + effective_categories = Factory.categories([['common', 'true']]) + Factory.contributed_bindings("puppet-final", [deep_clone(@extension_bindings.model)], effective_categories) + end + + def self.default_contribution + effective_categories = Factory.categories([['common', 'true']]) + Factory.contributed_bindings("puppet-default", [deep_clone(@default_bindings.model)], effective_categories) + end + + def self.injector_boot_contribution(env_boot_bindings) + # Compose the injector_boot_bindings contributed from the puppet runtime book (i.e. defaults for + # extensions that should be active in the boot injector - see Puppetx initialization. + # + bindings = [deep_clone(@injector_boot_bindings.model), deep_clone(@injector_default_bindings)] + + # Add the bindings that come from the bindings_composer as it may define custom extensions added in the bindings + # configuration. (i.e. bindings required to be able to lookup using bindings schemes and backends when + # configuring the real injector). + # + bindings << env_boot_bindings unless env_boot_bindings.nil? + + # Use an 'extension' category for extension bindings to allow them to override the default + # bindings since they are placed in the same layer (this is done to avoid having a separate layer). + # The purpose for allowing overrides is that someone may want to replace say 'yaml' with a different version, + # (say one that uses a YAML implementation that actually works ok in ruby 1.8.7 ;-)), an encrypted parser, etc. + effective_categories = Factory.categories([['extension', 'true'],['common', 'true']]) + + # return the composition and the cateogires. + Factory.contributed_bindings("puppet-injector-boot", bindings, effective_categories) + end + + def self.factory() + Puppet::Pops::Binder::BindingsFactory + end + + def self.type_factory() + Puppet::Pops::Types::TypeFactory + end + + private + + def self.deep_clone(o) + Marshal.load(Marshal.dump(o)) + end +end diff --git a/lib/puppet/pops/issue_reporter.rb b/lib/puppet/pops/issue_reporter.rb new file mode 100644 index 000000000..675b03992 --- /dev/null +++ b/lib/puppet/pops/issue_reporter.rb @@ -0,0 +1,75 @@ +class Puppet::Pops::IssueReporter + + # @param acceptor [Puppet::Pops::Validation::Acceptor] the acceptor containing reported issues + # @option options [String] :message (nil) A message text to use as prefix in a single Error message + # @option options [Boolean] :emit_warnings (false) A message text to use as prefix in a single Error message + # @option options [Boolean] :emit_errors (true) whether errors should be emitted or only given message + # @option options [Exception] :exception_class (Puppet::ParseError) The exception to raise + # + def self.assert_and_report(acceptor, options) + return unless acceptor + + max_errors = Puppet[:max_errors] + max_warnings = Puppet[:max_warnings] + 1 + max_deprecations = Puppet[:max_deprecations] + 1 + emit_warnings = options[:emit_warnings] || false + emit_errors = options[:emit_errors] || true + emit_message = options[:message] + emit_exception = options[:exception_class] || Puppet::ParseError + + # If there are warnings output them + warnings = acceptor.warnings + if emit_warnings && warnings.size > 0 + formatter = Puppet::Pops::Validation::DiagnosticFormatterPuppetStyle.new + emitted_w = 0 + emitted_dw = 0 + acceptor.warnings.each do |w| + if w.severity == :deprecation + # Do *not* call Puppet.deprecation_warning it is for internal deprecation, not + # deprecation of constructs in manifests! (It is not designed for that purpose even if + # used throughout the code base). + # + Puppet.warning(formatter.format(w)) if emitted_dw < max_deprecations + emitted_dw += 1 + else + Puppet.warning(formatter.format(w)) if emitted_w < max_warnings + emitted_w += 1 + end + break if emitted_w > max_warnings && emitted_dw > max_deprecations # but only then + end + end + + # If there were errors, report the first found. Use a puppet style formatter. + errors = acceptor.errors + if errors.size > 0 + unless emit_errors + raise emit_exception.new(emit_message) + end + formatter = Puppet::Pops::Validation::DiagnosticFormatterPuppetStyle.new + if errors.size == 1 || max_errors <= 1 + # raise immediately + raise emit_exception.new(format_with_prefix(emit_message, formatter.format(errors[0]))) + end + emitted = 0 + if emit_message + Puppet.err(emit_message) + end + errors.each do |e| + Puppet.err(formatter.format(e)) + emitted += 1 + break if emitted >= max_errors + end + warnings_message = (emit_warnings && warnings.size > 0) ? ", and #{warnings.size} warnings" : "" + giving_up_message = "Found #{errors.size} errors#{warnings_message}. Giving up" + exception = emit_exception.new(giving_up_message) + exception.file = errors[0].file + raise exception + end + parse_result + end + + def self.format_with_prefix(prefix, message) + return message unless prefix + [prefix, message].join(' ') + end +end
\ No newline at end of file diff --git a/lib/puppet/pops/issues.rb b/lib/puppet/pops/issues.rb index 1440c9030..0276f4022 100644 --- a/lib/puppet/pops/issues.rb +++ b/lib/puppet/pops/issues.rb @@ -44,8 +44,12 @@ module Puppet::Pops::Issues # Create a Message Data where all hash keys become methods for convenient interpolation # in issue text. msgdata = MessageData.new(*arg_names) - # Evaluate the message block in the msg data's binding - msgdata.format(hash, &message_block) + begin + # Evaluate the message block in the msg data's binding + msgdata.format(hash, &message_block) + rescue StandardError => e + raise RuntimeError, "Error while reporting issue: #{issue_code}. #{e.message}", caller + end end end @@ -91,8 +95,8 @@ module Puppet::Pops::Issues # # @param issue_code [Symbol] the issue code for the issue used as an identifier, should be the same as the constant # the issue is bound to. - # @param *args [Symbol] required arguments that must be passed when formatting the message, may be empty - # @param &block [Proc] a block producing the message string, evaluated in a MessageData scope. The produced string + # @param args [Symbol] required arguments that must be passed when formatting the message, may be empty + # @param block [Proc] a block producing the message string, evaluated in a MessageData scope. The produced string # should not end with a period as additional information may be appended. # # @see MessageData @@ -123,7 +127,7 @@ module Puppet::Pops::Issues # @todo configuration # NAME_WITH_HYPHEN = issue :NAME_WITH_HYPHEN, :name do - "#{label.a_an_uc(semantic)} may not have a name contain a hyphen. The name '#{name}' is not legal" + "#{label.a_an_uc(semantic)} may not have a name containing a hyphen. The name '#{name}' is not legal" end # When a variable name contains a hyphen and these are illegal. diff --git a/lib/puppet/pops/model/ast_transformer.rb b/lib/puppet/pops/model/ast_transformer.rb index a9ce9946d..b16d951bc 100644 --- a/lib/puppet/pops/model/ast_transformer.rb +++ b/lib/puppet/pops/model/ast_transformer.rb @@ -93,7 +93,7 @@ class Puppet::Pops::Model::AstTransformer end def transform_ArithmeticExpression(o) - ast o, AST::ArithmeticOperator, :lval => transform(o.left_expr), :rval=>transform(o.right_expr), + ast o, AST::ArithmeticOperator2, :lval => transform(o.left_expr), :rval=>transform(o.right_expr), :operator => o.operator.to_s end @@ -533,7 +533,7 @@ class Puppet::Pops::Model::AstTransformer def transform_IfExpression(o) args = { :test => transform(o.test), :statements => transform(o.then_expr) } args[:else] = transform(o.else_expr) # Tests say Nop should be there (unless is_nop? o.else_expr), probably not needed - result = ast o, AST::IfStatement, args + ast o, AST::IfStatement, args end # Unless is not an AST object, instead an AST::IfStatement is used with an AST::Not around the test @@ -541,9 +541,9 @@ class Puppet::Pops::Model::AstTransformer def transform_UnlessExpression(o) args = { :test => ast(o, AST::Not, :value => transform(o.test)), :statements => transform(o.then_expr) } - # AST 3.1 does not allow else on unless in the grammar, but it is ok since unless is encoded as a if !x + # AST 3.1 does not allow else on unless in the grammar, but it is ok since unless is encoded as an if !x args.merge!({:else => transform(o.else_expr)}) unless is_nop?(o.else_expr) - result = ast o, AST::IfStatement, args + ast o, AST::IfStatement, args end # Puppet 3.1 AST only supports calling a function by name (it is not possible to produce a function diff --git a/lib/puppet/pops/model/ast_tree_dumper.rb b/lib/puppet/pops/model/ast_tree_dumper.rb index ffe8fd848..624b12267 100644 --- a/lib/puppet/pops/model/ast_tree_dumper.rb +++ b/lib/puppet/pops/model/ast_tree_dumper.rb @@ -131,7 +131,7 @@ class Puppet::Pops::Model::AstTreeDumper < Puppet::Pops::Model::TreeDumper if o.is_a? String o # A Ruby String, not quoted - elsif n = Puppet::Pops::Utils.to_n(o.value) + elsif Puppet::Pops::Utils.to_n(o.value) o.value # AST::String that is a number without quotes else "'#{o.value}'" # AST::String that is not a number diff --git a/lib/puppet/pops/model/factory.rb b/lib/puppet/pops/model/factory.rb index 835a3415c..3cfa15fca 100644 --- a/lib/puppet/pops/model/factory.rb +++ b/lib/puppet/pops/model/factory.rb @@ -11,6 +11,8 @@ class Puppet::Pops::Model::Factory attr_accessor :current + alias_method :model, :current + # Shared build_visitor, since there are many instances of Factory being used @@build_visitor = Puppet::Pops::Visitor.new(self, "build") # Initialize a factory with a single object, or a class with arguments applied to build of @@ -180,10 +182,14 @@ class Puppet::Pops::Model::Factory end # Builds body :) from different kinds of input - # @param body [nil] unchanged, produces nil - # @param body [Array<Expression>] turns into a BlockExpression - # @param body [Expression] produces the given expression - # @param body [Object] produces the result of calling #build with body as argument + # @overload f_build_body(nothing) + # @param nothing [nil] unchanged, produces nil + # @overload f_build_body(array) + # @param array [Array<Expression>] turns into a BlockExpression + # @overload f_build_body(expr) + # @param expr [Expression] produces the given expression + # @overload f_build_body(obj) + # @param obj [Object] produces the result of calling #build with body as argument def f_build_body(body) case body when NilClass @@ -429,7 +435,7 @@ class Puppet::Pops::Model::Factory # Does nothing if file is nil. # # @param file [String,nil] the file/path to the origin, may contain URI scheme of file: or some other URI scheme - # @returns [Factory] returns self + # @return [Factory] returns self # def record_origin(file) return self unless file @@ -457,11 +463,11 @@ class Puppet::Pops::Model::Factory a.documentation = doc_string end - # Returns symbolic information about a expected share of a resource expression given the LHS of a resource expr. + # Returns symbolic information about an expected share of a resource expression given the LHS of a resource expr. # # * `name { }` => `:resource`, create a resource of the given type - # * `Name { }` => ':defaults`, set defauls for the referenced type - # * `Name[] { }` => `:override`, ioverrides nstances referenced by LHS + # * `Name { }` => ':defaults`, set defaults for the referenced type + # * `Name[] { }` => `:override`, overrides instances referenced by LHS # * _any other_ => ':error', all other are considered illegal # def self.resource_shape(expr) @@ -657,14 +663,20 @@ class Puppet::Pops::Model::Factory memo[-1] = Puppet::Pops::Model::Factory.IMPORT(expr.is_a?(Array) ? expr : [expr]) else memo[-1] = Puppet::Pops::Model::Factory.CALL_NAMED(name, false, expr.is_a?(Array) ? expr : [expr]) + if expr.is_a?(Model::CallNamedFunctionExpression) + # Patch statement function call to expression style + # This is needed because it is first parsed as a "statement" and the requirement changes as it becomes + # an argument to the name to call transform above. + expr.rval_required = true + end end else memo << expr - end - if expr.is_a?(Model::CallNamedFunctionExpression) - # patch expression function call to statement style - # TODO: This is kind of meaningless, but to make it compatible... - expr.rval_required = false + if expr.is_a?(Model::CallNamedFunctionExpression) + # Patch rvalue expression function call to statement style. + # This is not really required but done to be AST model compliant + expr.rval_required = false + end end memo end diff --git a/lib/puppet/pops/model/model.rb b/lib/puppet/pops/model/model.rb index 82a9490e7..5f5302be3 100644 --- a/lib/puppet/pops/model/model.rb +++ b/lib/puppet/pops/model/model.rb @@ -5,7 +5,7 @@ # It describes a Metamodel containing DSL instructions, a description of PuppetType and related # classes needed to evaluate puppet logic. # The metamodel resembles the existing AST model, but it is a semantic model of instructions and -# the types that they operate on rather than a Abstract Syntax Tree, although closely related. +# the types that they operate on rather than an Abstract Syntax Tree, although closely related. # # The metamodel is anemic (has no behavior) except basic datatype and type # assertions and reference/containment assertions. diff --git a/lib/puppet/pops/model/tree_dumper.rb b/lib/puppet/pops/model/tree_dumper.rb index f0a7abaa7..518b06dc4 100644 --- a/lib/puppet/pops/model/tree_dumper.rb +++ b/lib/puppet/pops/model/tree_dumper.rb @@ -25,9 +25,9 @@ class Puppet::Pops::Model::TreeDumper parts.each_index do |i| if i > 0 # separate with space unless previous ends with whitepsace or ( - result << ' ' if parts[i] != ")" && parts[i-1] !~ /.*(?:\s+|\()$/ && parts[i] !~ /\s+/ + result << ' ' if parts[i] != ")" && parts[i-1] !~ /.*(?:\s+|\()$/ && parts[i] !~ /^\s+/ end - result << parts[i] + result << parts[i].to_s end result end diff --git a/lib/puppet/pops/parser/egrammar.ra b/lib/puppet/pops/parser/egrammar.ra index bb6a5db32..eb8357d58 100644 --- a/lib/puppet/pops/parser/egrammar.ra +++ b/lib/puppet/pops/parser/egrammar.ra @@ -706,7 +706,6 @@ end ---- header ---- require 'puppet' -require 'puppet/util/loadedfile' require 'puppet/pops' module Puppet diff --git a/lib/puppet/pops/parser/eparser.rb b/lib/puppet/pops/parser/eparser.rb index 26fea92cf..e476e0358 100644 --- a/lib/puppet/pops/parser/eparser.rb +++ b/lib/puppet/pops/parser/eparser.rb @@ -7,7 +7,6 @@ require 'racc/parser.rb' require 'puppet' -require 'puppet/util/loadedfile' require 'puppet/pops' module Puppet @@ -21,7 +20,7 @@ module Puppet module Parser class Parser < Racc::Parser -module_eval(<<'...end egrammar.ra/module_eval...', 'egrammar.ra', 719) +module_eval(<<'...end egrammar.ra/module_eval...', 'egrammar.ra', 718) # Make emacs happy # Local Variables: diff --git a/lib/puppet/pops/parser/evaluating_parser.rb b/lib/puppet/pops/parser/evaluating_parser.rb new file mode 100644 index 000000000..3a6cdbb66 --- /dev/null +++ b/lib/puppet/pops/parser/evaluating_parser.rb @@ -0,0 +1,162 @@ + +# Does not support "import" and parsing ruby files +# +class Puppet::Pops::Parser::EvaluatingParser + + def initialize() + @parser = Puppet::Pops::Parser::Parser.new() + end + + def parse_string(s, file_source = 'unknown') + @file_source = file_source + clear() + # Handling of syntax error can be much improved (in general), now it bails out of the parser + # and does not have as rich information (when parsing a string), need to update it with the file source + # (ideally, a syntax error should be entered as an issue, and not just thrown - but that is a general problem + # and an improvement that can be made in the eparser (rather than here). + # Also a possible improvement (if the YAML parser returns positions) is to provide correct output of position. + # + begin + assert_and_report(@parser.parse_string(s)) + rescue Puppet::ParseError => e + e.file = @file_source unless e.file + raise e + end + end + + def parse_file(file) + @file_source = file + clear() + assert_and_report(@parser.parse_file(file)) + end + + def evaluate_string(scope, s, file_source='unknown') + evaluate(scope, parse_string(s, file_source)) + end + + def evaluate_file(file) + evaluate(parse_file(file)) + end + + def clear() + @acceptor = nil + end + + def evaluate(scope, model) + return nil unless model + ast = Puppet::Pops::Model::AstTransformer.new(@file_source, nil).transform(model) + return nil unless ast + ast.safeevaluate(scope) + end + + def acceptor() + @acceptor ||= Puppet::Pops::Validation::Acceptor.new + @acceptor + end + + def validator() + @validator ||= Puppet::Pops::Validation::ValidatorFactory_3_1.new().validator(acceptor) + end + + def assert_and_report(parse_result) + return nil unless parse_result + # make sure the result has an origin (if parsed from a string) + unless Puppet::Pops::Adapters::OriginAdapter.get(parse_result.model) + Puppet::Pops::Adapters::OriginAdapter.adapt(parse_result.model).origin = @file_source + end + validator.validate(parse_result) + + max_errors = Puppet[:max_errors] + max_warnings = Puppet[:max_warnings] + 1 + max_deprecations = Puppet[:max_deprecations] + 1 + + # If there are warnings output them + warnings = acceptor.warnings + if warnings.size > 0 + formatter = Puppet::Pops::Validation::DiagnosticFormatterPuppetStyle.new + emitted_w = 0 + emitted_dw = 0 + acceptor.warnings.each {|w| + if w.severity == :deprecation + # Do *not* call Puppet.deprecation_warning it is for internal deprecation, not + # deprecation of constructs in manifests! (It is not designed for that purpose even if + # used throughout the code base). + # + Puppet.warning(formatter.format(w)) if emitted_dw < max_deprecations + emitted_dw += 1 + else + Puppet.warning(formatter.format(w)) if emitted_w < max_warnings + emitted_w += 1 + end + break if emitted_w > max_warnings && emitted_dw > max_deprecations # but only then + } + end + + # If there were errors, report the first found. Use a puppet style formatter. + errors = acceptor.errors + if errors.size > 0 + formatter = Puppet::Pops::Validation::DiagnosticFormatterPuppetStyle.new + if errors.size == 1 || max_errors <= 1 + # raise immediately + require 'debugger'; debugger + raise Puppet::ParseError.new(formatter.format(errors[0])) + end + emitted = 0 + errors.each do |e| + Puppet.err(formatter.format(e)) + emitted += 1 + break if emitted >= max_errors + end + warnings_message = warnings.size > 0 ? ", and #{warnings.size} warnings" : "" + giving_up_message = "Found #{errors.size} errors#{warnings_message}. Giving up" + exception = Puppet::ParseError.new(giving_up_message) + exception.file = errors[0].file + raise exception + end + parse_result + end + + def quote(x) + self.class.quote(x) + end + + # Translates an already parsed string that contains control characters, quotes + # and backslashes into a quoted string where all such constructs have been escaped. + # Parsing the return value of this method using the puppet parser should yield + # exactly the same string as the argument passed to this method + # + # The method makes an exception for the two character sequences \$ and \s. They + # will not be escaped since they have a special meaning in puppet syntax. + # + # @param x [String] The string to quote and "unparse" + # @return [String] The quoted string + # + def self.quote(x) + escaped = '"' + p = nil + x.each_char do |c| + case p + when nil + # do nothing + when "\t" + escaped << '\\t' + when "\n" + escaped << '\\n' + when "\f" + escaped << '\\f' + # TODO: \cx is a range of characters - skip for now + # when "\c" + # escaped << '\\c' + when '"' + escaped << '\\"' + when '\\' + escaped << if c == '$' || c == 's'; p; else '\\\\'; end # don't escape \ when followed by s or $ + else + escaped << p + end + p = c + end + escaped << p unless p.nil? + escaped << '"' + end +end diff --git a/lib/puppet/pops/parser/lexer.rb b/lib/puppet/pops/parser/lexer.rb index 106f407ba..1e090f7bc 100644 --- a/lib/puppet/pops/parser/lexer.rb +++ b/lib/puppet/pops/parser/lexer.rb @@ -31,10 +31,14 @@ class Puppet::Pops::Parser::Lexer attr_accessor :regex, :name, :string, :skip, :skip_text alias skip? skip - # @param string_or_regex[String] a literal string token matcher - # @param string_or_regex[Regexp] a regular expression token text matcher - # @param name [String] the token name (what it is known as in the grammar) - # @param options [Hash] see {#set_options} + # @overload initialize(string) + # @param string [String] a literal string token matcher + # @param name [String] the token name (what it is known as in the grammar) + # @param options [Hash] see {#set_options} + # @overload initialize(regex) + # @param regex [Regexp] a regular expression token text matcher + # @param name [String] the token name (what it is known as in the grammar) + # @param options [Hash] see {#set_options} # def initialize(string_or_regex, name, options = {}) if string_or_regex.is_a?(String) @@ -565,7 +569,6 @@ class Puppet::Pops::Parser::Lexer skip until token_queue.empty? and @scanner.eos? do - yielded = false offset = @scanner.pos matched_token, value = find_token end_offset = @scanner.pos @@ -646,7 +649,6 @@ class Puppet::Pops::Parser::Lexer # we search for the next quote that isn't preceded by a # backslash; the caret is there to match empty strings last = @scanner.matched - tmp_offset = @scanner.pos str = @scanner.scan_until(/([^\\]|^|[^\\])([\\]{2})*[#{terminators}]/) || lex_error(positioned_message("Unclosed quote after #{format_quote(last)} followed by '#{followed_by}'")) str.gsub!(/\\(.)/m) { ch = $1 diff --git a/lib/puppet/pops/types/class_loader.rb b/lib/puppet/pops/types/class_loader.rb new file mode 100644 index 000000000..abacd704e --- /dev/null +++ b/lib/puppet/pops/types/class_loader.rb @@ -0,0 +1,118 @@ +require 'rgen/metamodel_builder' + +# The ClassLoader provides a Class instance given a class name or a meta-type. +# If the class is not already loaded, it is loaded using the Puppet Autoloader. +# This means it can load a class from a gem, or from puppet modules. +# +class Puppet::Pops::Types::ClassLoader + @autoloader = Puppet::Util::Autoload.new("ClassLoader", "", :wrap => false) + + # Returns a Class given a fully qualified class name. + # Lookup of class is never relative to the calling namespace. + # @param name [String, Array<String>, Array<Symbol>, Puppet::Pops::Types::PObjectType] A fully qualified + # class name String (e.g. '::Foo::Bar', 'Foo::Bar'), a PObjectType, or a fully qualified name in Array form where each part + # is either a String or a Symbol, e.g. `%w{Puppetx Puppetlabs SomeExtension}`. + # @return [Class, nil] the looked up class or nil if no such class is loaded + # @raise ArgumentError If the given argument has the wrong type + # @api public + # + def self.provide(name) + case name + when String + provide_from_string(name) + + when Array + provide_from_name_path(name.join('::'), name) + + when Puppet::Pops::Types::PObjectType, Puppet::Pops::Types::PType + provide_from_type(name) + + else + raise ArgumentError, "Cannot provide a class from a '#{name.class.name}'" + end + end + + private + + def self.provide_from_type(type) + case type + when Puppet::Pops::Types::PRubyType + provide_from_string(type.ruby_class) + + when Puppet::Pops::Types::PBooleanType + # There is no other thing to load except this Enum meta type + RGen::MetamodelBuilder::MMBase::Boolean + + when Puppet::Pops::Types::PType + # TODO: PType should have a type argument (a PObjectType) + Class + + # Although not expected to be the first choice for getting a concrete class for these + # types, these are of value if the calling logic just has a reference to type. + # + when Puppet::Pops::Types::PArrayType ; Array + when Puppet::Pops::Types::PHashType ; Hash + when Puppet::Pops::Types::PPatternType ; Regexp + when Puppet::Pops::Types::PIntegerType ; Integer + when Puppet::Pops::Types::PStringType ; String + when Puppet::Pops::Types::PFloatType ; Float + when Puppet::Pops::Types::PNilType ; NilClass + else + nil + end + end + + def self.provide_from_string(name) + name_path = name.split('::') + # always from the root, so remove an empty first segment + if name_path[0].empty? + name_path = name_path[1..-1] + end + provide_from_name_path(name, name_path) + end + + def self.provide_from_name_path(name, name_path) + # If class is already loaded, try this first + result = find_class(name_path) + + unless result.is_a?(Class) + # Attempt to load it using the auto loader + loaded_path = nil + if paths_for_name(name).find {|path| loaded_path = path; @autoloader.load(path) } + result = find_class(name_path) + unless result.is_a?(Class) + raise RuntimeError, "Loading of #{name} using relative path: '#{loaded_path}' did not create expected class" + end + end + end + return nil unless result.is_a?(Class) + result + end + + def self.find_class(name_path) + name_path.reduce(Object) do |ns, name| + begin + ns.const_get(name) + rescue NameError + return nil + end + end + end + + def self.paths_for_name(fq_name) + [de_camel(fq_name), downcased_path(fq_name)] + end + + def self.downcased_path(fq_name) + fq_name.to_s.gsub(/::/, '/').downcase + end + + def self.de_camel(fq_name) + fq_name.to_s.gsub(/::/, '/'). + gsub(/([A-Z]+)([A-Z][a-z])/,'\1_\2'). + gsub(/([a-z\d])([A-Z])/,'\1_\2'). + tr("-", "_"). + downcase + end + +end
\ No newline at end of file diff --git a/lib/puppet/pops/types/type_calculator.rb b/lib/puppet/pops/types/type_calculator.rb new file mode 100644 index 000000000..3158a2cc5 --- /dev/null +++ b/lib/puppet/pops/types/type_calculator.rb @@ -0,0 +1,557 @@ +# The TypeCalculator can answer questions about puppet types. +# +# The Puppet type system is primarily based on sub-classing. When asking the type calculator to infer types from Ruby in general, it +# may not provide the wanted answer; it does not for instance take module inclusions and extensions into account. In general the type +# system should be unsurprising for anyone being exposed to the notion of type. The type `Data` may require a bit more explanation; this +# is an abstract type that includes all literal types, as well as Array with an element type compatible with Data, and Hash with key +# compatible with Literal and elements compatible with Data. Expressed differently; Data is what you typically express using JSON (with +# the exception that the Puppet type system also includes Pattern (regular expression) as a literal. +# +# Inference +# --------- +# The `infer(o)` method infers a Puppet type for literal Ruby objects, and for Arrays and Hashes. +# +# Assignability +# ------------- +# The `assignable?(t1, t2)` method answers if t2 conforms to t1. The type t2 may be an instance, in which case +# its type is inferred, or a type. +# +# Instance? +# --------- +# The `instance?(t, o)` method answers if the given object (instance) is an instance that is assignable to the given type. +# +# String +# ------ +# Creates a string representation of a type. +# +# Creation of Type instances +# -------------------------- +# Instance of the classes in the {Puppet::Pops::Types type model} are used to denote a specific type. It is most convenient +# to use the {Puppet::Pops::Types::TypeFactory TypeFactory} when creating instances. +# +# @note +# In general, new instances of the wanted type should be created as they are assigned to models using containment, and a +# contained object can only be in one container at a time. Also, the type system may include more details in each type +# instance, such as if it may be nil, be empty, contain a certain count etc. Or put differently, the puppet types are not +# singletons. +# +# Equality and Hash +# ----------------- +# Type instances are equal in terms of Ruby eql? and `==` if they describe the same type, but they are not `equal?` if they are not +# the same type instance. Two types that describe the same type have identical hash - this makes them usable as hash keys. +# +# Types and Subclasses +# -------------------- +# In general, the type calculator should be used to answer questions if a type is a subtype of another (using {#assignable?}, or +# {#instance?} if the question is if a given object is an instance of a given type (or is a subtype thereof). +# Many of the types also have a Ruby subtype relationship; e.g. PHashType and PArrayType are both subtypes of PCollectionType, and +# PIntegerType, PFloatType, PStringType,... are subtypes of PLiteralType. Even if it is possible to answer certain questions about +# type by looking at the Ruby class of the types this is considered an implementation detail, and such checks should in general +# be performed by the type_calculator which implements the type system semantics. +# +# The PRubyType +# ------------- +# The PRubyType corresponds to a Ruby Class, except for the puppet types that are specialized (i.e. PRubyType should not be +# used for Integer, String, etc. since there are specialized types for those). +# When the type calculator deals with PRubyTypes and checks for assignability, it determines the "common ancestor class" of two classes. +# This check is made based on the superclasses of the two classes being compared. In order to perform this, the classes must be present +# (i.e. they are resolved from the string form in the PRubyType to a loaded, instantiated Ruby Class). In general this is not a problem, +# since the question to produce the common super type for two objects means that the classes must be present or there would have been +# no instances present in the first place. If however the classes are not present, the type calculator will fall back and state that +# the two types at least have Object in common. +# +# @see Puppet::Pops::Types::TypeFactory TypeFactory for how to create instances of types +# @see Puppet::Pops::Types::TypeParser TypeParser how to construct a type instance from a String +# @see Puppet::Pops::Types Types for details about the type model +# +# @api public +# +class Puppet::Pops::Types::TypeCalculator + + Types = Puppet::Pops::Types + + # @api public + # + def initialize + @@assignable_visitor ||= Puppet::Pops::Visitor.new(nil,"assignable",1,1) + @@infer_visitor ||= Puppet::Pops::Visitor.new(nil,"infer",0,0) + @@string_visitor ||= Puppet::Pops::Visitor.new(nil,"string",0,0) + + da = Types::PArrayType.new() + da.element_type = Types::PDataType.new() + @data_array = da + + h = Types::PHashType.new() + h.element_type = Types::PDataType.new() + h.key_type = Types::PLiteralType.new() + @data_hash = h + + @data_t = Types::PDataType.new() + @literal_t = Types::PLiteralType.new() + @numeric_t = Types::PNumericType.new() + @t = Types::PObjectType.new() + end + + # Convenience method to get a data type for comparisons + # @api private the returned value may not be contained in another element + # + def data + @data_t + end + + # Answers the question 'is it possible to inject an instance of the given class' + # A class is injectable if it has a special *assisted inject* class method called `inject` taking + # an injector and a scope as argument, or if it has a zero args `initialize` method. + # + # @param klazz [Class, PRubyType] the class/type to check if it is injectable + # @return [Class, nil] the injectable Class, or nil if not injectable + # @api public + # + def injectable_class(klazz) + # Handle case when we get a PType instead of a class + if klazz.is_a?(Types::PRubyType) + klazz = Puppet::Pops::Types::ClassLoader.provide(klazz) + end + + # data types can not be injected (check again, it is not safe to assume that given RubyType klazz arg was ok) + return false unless type(klazz).is_a?(Types::PRubyType) + if (klazz.respond_to?(:inject) && klazz.method(:inject).arity() == -4) || klazz.instance_method(:initialize).arity() == 0 + klazz + else + nil + end + end + + # Answers 'can an instance of type t2 be assigned to a variable of type t' + # @api public + # + def assignable?(t, t2) + # nil is assignable to anything + if is_pnil?(t2) + return true + end + + if t.is_a?(Class) + t = type(t) + end + + if t2.is_a?(Class) + t2 = type(t2) + end + + @@assignable_visitor.visit_this(self, t, t2) + end + + # Answers 'what is the Puppet Type corresponding to the given Ruby class' + # @param c [Class] the class for which a puppet type is wanted + # @api public + # + def type(c) + raise ArgumentError, "Argument must be a Class" unless c.is_a? Class + + # Can't use a visitor here since we don't have an instance of the class + case + when c <= Integer + type = Types::PIntegerType.new() + when c == Float + type = Types::PFloatType.new() + when c == Numeric + type = Types::PNumericType.new() + when c == String + type = Types::PStringType.new() + when c == Regexp + type = Types::PPatternType.new() + when c == NilClass + type = Types::PNilType.new() + when c == FalseClass, c == TrueClass + type = Types::PBooleanType.new() + when c == Class + type = Types::PType.new() + when c == Array + # Assume array of data values + type = Types::PArrayType.new() + type.element_type = Types::PDataType.new() + when c == Hash + # Assume hash with literal keys and data values + type = Types::PHashType.new() + type.key_type = Types::PLiteralType.new() + type.element_type = Types::PDataType.new() + else + type = Types::PRubyType.new() + type.ruby_class = c.name + end + type + end + + # Answers 'what is the Puppet Type of o' + # @api public + # + def infer(o) + @@infer_visitor.visit_this(self, o) + end + + # Answers 'is o an instance of type t' + # @api public + # + def instance?(t, o) + assignable?(t, infer(o)) + end + + # Answers if t is a puppet type + # @api public + # + def is_ptype?(t) + return t.is_a?(Types::PObjectType) + end + + # Answers if t represents the puppet type PNilType + # @api public + # + def is_pnil?(t) + return t.nil? || t.is_a?(Types::PNilType) + end + + # Answers, 'What is the common type of t1 and t2?' + # @api public + # + def common_type(t1, t2) + raise ArgumentError, 'two types expected' unless (is_ptype?(t1) || is_pnil?(t1)) && (is_ptype?(t2) || is_pnil?(t2)) + + # if either is nil, the common type is the other + if is_pnil?(t1) + return t2 + elsif is_pnil?(t2) + return t1 + end + + # Simple case, one is assignable to the other + if assignable?(t1, t2) + return t1 + elsif assignable?(t2, t1) + return t2 + end + + # when both are arrays, return an array with common element type + if t1.is_a?(Types::PArrayType) && t2.is_a?(Types::PArrayType) + type = Types::PArrayType.new() + type.element_type = common_type(t1.element_type, t2.element_type) + return type + end + + # when both are hashes, return a hash with common key- and element type + if t1.is_a?(Types::PHashType) && t2.is_a?(Types::PHashType) + type = Types::PHashType.new() + type.key_type = common_type(t1.key_type, t2.key_type) + type.element_type = common_type(t1.element_type, t2.element_type) + return type + end + + # Common abstract types, from most specific to most general + if common_numeric?(t1, t2) + return Types::PNumericType.new() + end + + if common_literal?(t1, t2) + return Types::PLiteralType.new() + end + + if common_data?(t1,t2) + return Types::PDataType.new() + end + + if t1.is_a?(Types::PRubyType) && t2.is_a?(Types::PRubyType) + if t1.ruby_class == t2.ruby_class + return t1 + end + # finding the common super class requires that names are resolved to class + c1 = Types::ClassLoader.provide_from_type(t1) + c2 = Types::ClassLoader.provide_from_type(t2) + if c1 && c2 + c2_superclasses = superclasses(c2) + superclasses(c1).each do|c1_super| + c2_superclasses.each do |c2_super| + if c1_super == c2_super + result = Types::PRubyType.new() + result.ruby_class = c1_super.name + return result + end + end + end + end + end + # If both are RubyObjects + + if common_pobject?(t1, t2) + return Types::PObjectType.new() + end + end + + # Produces the superclasses of the given class, including the class + def superclasses(c) + result = [c] + while s = c.superclass + result << s + c = s + end + result + end + # Produces a string representing the type + # @api public + # + def string(t) + @@string_visitor.visit_this(self, t) + end + + + # Reduces an enumerable of types to a single common type. + # @api public + # + def reduce_type(enumerable) + enumerable.reduce(nil) {|memo, t| common_type(memo, t) } + end + + # Reduce an enumerable of objects to a single common type + # @api public + # + def infer_and_reduce_type(enumerable) + reduce_type(enumerable.collect() {|o| infer(o) }) + end + + # The type of all classes is PType + # @api private + # + def infer_Class(o) + Types::PType.new() + end + + # @api private + def infer_Object(o) + type = Types::PRubyType.new() + type.ruby_class = o.class.name + type + end + + # The type of all types is PType + # @api private + # + def infer_PObjectType(o) + Types::PType.new() + end + + # The type of all types is PType + # This is the metatype short circuit. + # @api private + # + def infer_PType(o) + Types::PType.new() + end + + # @api private + def infer_String(o) + Types::PStringType.new() + end + + # @api private + def infer_Float(o) + Types::PFloatType.new() + end + + # @api private + def infer_Integer(o) + Types::PIntegerType.new() + end + + # @api private + def infer_Regexp(o) + Types::PPatternType.new() + end + + # @api private + def infer_NilClass(o) + Types::PNilType.new() + end + + # @api private + def infer_TrueClass(o) + Types::PBooleanType.new() + end + + # @api private + def infer_FalseClass(o) + Types::PBooleanType.new() + end + + # @api private + def infer_Array(o) + type = Types::PArrayType.new() + type.element_type = if o.empty? + Types::PNilType.new() + else + infer_and_reduce_type(o) + end + type + end + + # @api private + def infer_Hash(o) + type = Types::PHashType.new() + if o.empty? + ktype = Types::PNilType.new() + etype = Types::PNilType.new() + else + ktype = infer_and_reduce_type(o.keys()) + etype = infer_and_reduce_type(o.values()) + end + type.key_type = ktype + type.element_type = etype + type + end + + # False in general type calculator + # @api private + def assignable_Object(t, t2) + false + end + + # @api private + def assignable_PObjectType(t, t2) + t2.is_a?(Types::PObjectType) + end + + # @api private + def assignable_PLiteralType(t, t2) + t2.is_a?(Types::PLiteralType) + end + + # @api private + def assignable_PNumericType(t, t2) + t2.is_a?(Types::PNumericType) + end + + # @api private + def assignable_PIntegerType(t, t2) + t2.is_a?(Types::PIntegerType) + end + + # @api private + def assignable_PStringType(t, t2) + t2.is_a?(Types::PStringType) + end + + # @api private + def assignable_PFloatType(t, t2) + t2.is_a?(Types::PFloatType) + end + + # @api private + def assignable_PBooleanType(t, t2) + t2.is_a?(Types::PBooleanType) + end + + # @api private + def assignable_PPatternType(t, t2) + t2.is_a?(Types::PPatternType) + end + + # @api private + def assignable_PCollectionType(t, t2) + t2.is_a?(Types::PCollectionType) + end + + # Array is assignable if t2 is an Array and t2's element type is assignable + # @api private + def assignable_PArrayType(t, t2) + return false unless t2.is_a?(Types::PArrayType) + assignable?(t.element_type, t2.element_type) + end + + # Hash is assignable if t2 is a Hash and t2's key and element types are assignable + # @api private + def assignable_PHashType(t, t2) + return false unless t2.is_a?(Types::PHashType) + assignable?(t.key_type, t2.key_type) && assignable?(t.element_type, t2.element_type) + end + + # Data is assignable by other Data and by Array[Data] and Hash[Literal, Data] + # @api private + def assignable_PDataType(t, t2) + t2.is_a?(Types::PDataType) || assignable?(@data_array, t2) || assignable?(@data_hash, t2) + end + + # Assignable if t2's ruby class is same or subclass of t1's ruby class + # @api private + def assignable_PRubyType(t1, t2) + return false unless t2.is_a?(Types::PRubyType) + c1 = class_from_string(t1.ruby_class) + c2 = class_from_string(t2.ruby_class) + return false unless c1.is_a?(Class) && c2.is_a?(Class) + !!(c2 <= c1) + end + + # @api private + def string_PType(t) ; "Type" ; end + + # @api private + def string_PObjectType(t) ; "Object" ; end + + # @api private + def string_PBooleanType(t) ; "Boolean" ; end + + # @api private + def string_PLiteralType(t) ; "Literal" ; end + + # @api private + def string_PDataType(t) ; "Data" ; end + + # @api private + def string_PNumericType(t) ; "Numeric" ; end + + # @api private + def string_PIntegerType(t) ; "Integer" ; end + + # @api private + def string_PFloatType(t) ; "Float" ; end + + # @api private + def string_PPatternType(t) ; "Pattern" ; end + + # @api private + def string_PStringType(t) ; "String" ; end + + # @api private + def string_PRubyType(t) ; "Ruby[#{t.ruby_class}]" ; end + + # @api private + def string_PArrayType(t) + "Array[#{string(t.element_type)}]" + end + + # @api private + def string_PHashType(t) + "Hash[#{string(t.key_type)}, #{string(t.element_type)}]" + end + + private + + def class_from_string(str) + str.split('::').inject(Object) do |memo, name_segment| + memo.const_get(name_segment) + end + end + + def common_data?(t1, t2) + assignable?(@data_t, t1) && assignable?(@data_t, t2) + end + + def common_literal?(t1, t2) + assignable?(@literal_t, t1) && assignable?(@literal_t, t2) + end + + def common_numeric?(t1, t2) + assignable?(@numeric_t, t1) && assignable?(@numeric_t, t2) + end + + def common_pobject?(t1, t2) + assignable?(@t, t1) && assignable?(@t, t2) + end +end diff --git a/lib/puppet/pops/types/type_factory.rb b/lib/puppet/pops/types/type_factory.rb new file mode 100644 index 000000000..2be11be11 --- /dev/null +++ b/lib/puppet/pops/types/type_factory.rb @@ -0,0 +1,147 @@ +# Helper module that makes creation of type objects simpler. +# @api public +# +module Puppet::Pops::Types::TypeFactory + @type_calculator = Puppet::Pops::Types::TypeCalculator.new() + + Types = Puppet::Pops::Types + + # Produces the Integer type + # @api public + # + def self.integer() + Types::PIntegerType.new() + end + + # Produces the Float type + # @api public + # + def self.float() + Types::PFloatType.new() + end + + # Produces a string representation of the type + # @api public + # + def self.label(t) + @type_calculator.string(t) + end + + # Produces the String type + # @api public + # + def self.string() + Types::PStringType.new() + end + + # Produces the Boolean type + # @api public + # + def self.boolean() + Types::PBooleanType.new() + end + + # Produces the Pattern type + # @api public + # + def self.pattern() + Types::PPatternType.new() + end + + # Produces the Literal type + # @api public + # + def self.literal() + Types::PLiteralType.new() + end + + # Produces the abstract type Collection + # @api public + # + def self.collection() + Types::PCollectionType.new() + end + + # Produces the Data type + # @api public + # + def self.data() + Types::PDataType.new() + end + + # Produces a type for Array[o] where o is either a type, or an instance for which a type is inferred. + # @api public + # + def self.array_of(o) + type = Types::PArrayType.new() + type.element_type = type_of(o) + type + end + + # Produces a type for Hash[Literal, o] where o is either a type, or an instance for which a type is inferred. + # @api public + # + def self.hash_of(value, key = literal()) + type = Types::PHashType.new() + type.key_type = type_of(key) + type.element_type = type_of(value) + type + end + + # Produces a type for Array[Data] + # @api public + # + def self.array_of_data() + type = Types::PArrayType.new() + type.element_type = data() + type + end + + # Produces a type for Hash[Literal, Data] + # @api public + # + def self.hash_of_data() + type = Types::PHashType.new() + type.key_type = literal() + type.element_type = data() + type + end + + # Produce a type corresponding to the class of given unless given is a String, Class or a PObjectType. + # When a String is given this is taken as a classname. + # + def self.type_of(o) + if o.is_a?(Class) + @type_calculator.type(o) + elsif o.is_a?(Types::PObjectType) + o + elsif o.is_a?(String) + type = Types::PRubyType.new() + type.ruby_class = o + type + else + @type_calculator.infer(o) + end + end + + # Produces a type for a class or infers a type for something that is not a class + # @note + # To get the type for the class' class use `TypeCalculator.infer(c)` + # + # @overload ruby(o) + # @param o [Class] produces the type corresponding to the class (e.g. Integer becomes PIntegerType) + # @overload ruby(o) + # @param o [Object] produces the type corresponding to the instance class (e.g. 3 becomes PIntegerType) + # + # @api public + # + def self.ruby(o) + if o.is_a?(Class) + @type_calculator.type(o) + else + type = Types::PRubyType.new() + type.ruby_class = o.class.name + type + end + end +end diff --git a/lib/puppet/pops/types/type_parser.rb b/lib/puppet/pops/types/type_parser.rb new file mode 100644 index 000000000..f62178a4d --- /dev/null +++ b/lib/puppet/pops/types/type_parser.rb @@ -0,0 +1,117 @@ +# This class provides parsing of Type Specification from a string into the Type +# Model that is produced by the Puppet::Pops::Types::TypeFactory. +# +# The Type Specifications that are parsed are the same as the stringified forms +# of types produced by the {Puppet::Pops::Types::TypeCalculator TypeCalculator}. +# +# @api public +class Puppet::Pops::Types::TypeParser + # @api private + TYPES = Puppet::Pops::Types::TypeFactory + + # @api public + def initialize + @parser = Puppet::Pops::Parser::Parser.new() + @type_transformer = Puppet::Pops::Visitor.new(nil, "interpret", 0, 0) + end + + # Produces a *puppet type* based on the given string. + # + # @example + # parser.parse('Integer') + # parser.parse('Array[String]') + # parser.parse('Hash[Integer, Array[String]]') + # + # @param string [String] a string with the type expressed in stringified form as produced by the + # {Puppet::Pops::Types::TypeCalculator#string TypeCalculator#string} method. + # @return [Puppet::Pops::Types::PObjectType] a specialization of the PObjectType representing the type. + # + # @api public + # + def parse(string) + @string = string + model = @parser.parse_string(@string) + if model + interpret(model.current) + else + raise_invalid_type_specification_error + end + end + + # @api private + def interpret(ast) + @type_transformer.visit_this(self, ast) + end + + # @api private + def interpret_Object(anything) + raise_invalid_type_specification_error + end + + # @api private + def interpret_QualifiedReference(name_ast) + case name_ast.value + when "integer" + TYPES.integer + when "float" + TYPES.float + when "string" + TYPES.string + when "boolean" + TYPES.boolean + when "pattern" + TYPES.pattern + when "data" + TYPES.data + when "array" + TYPES.array_of_data + when "hash" + TYPES.hash_of_data + else + raise_unknown_type_error(name_ast) + end + end + + # @api private + def interpret_AccessExpression(parameterized_ast) + parameters = parameterized_ast.keys.collect { |param| interpret(param) } + case parameterized_ast.left_expr.value + when "array" + if parameters.size != 1 + raise_invalid_parameters_error("Array", 1, parameters.size) + end + TYPES.array_of(parameters[0]) + when "hash" + if parameters.size == 1 + TYPES.hash_of(parameters[0]) + elsif parameters.size != 2 + raise_invalid_parameters_error("Hash", "1 or 2", parameters.size) + else + TYPES.hash_of(parameters[1], parameters[0]) + end + else + raise_unknown_type_error(parameterized_ast.left_expr) + end + end + + private + + def raise_invalid_type_specification_error + raise Puppet::ParseError, + "The expression <#{@string}> is not a valid type specification." + end + + def raise_invalid_parameters_error(type, required, given) + raise Puppet::ParseError, + "Invalid number of type parameters specified: #{type} requires #{required}, #{given} provided" + end + + def raise_unknown_type_error(ast) + raise Puppet::ParseError, "Unknown type <#{original_text_of(ast)}>" + end + + def original_text_of(ast) + position = Puppet::Pops::Adapters::SourcePosAdapter.adapt(ast) + position.extract_text_from_string(@string) + end +end diff --git a/lib/puppet/pops/types/types.rb b/lib/puppet/pops/types/types.rb new file mode 100644 index 000000000..f524308b4 --- /dev/null +++ b/lib/puppet/pops/types/types.rb @@ -0,0 +1,132 @@ +require 'rgen/metamodel_builder' + +# The Types model is a model of Puppet Language types. +# +# The exact relationship between types is not visible in this model wrt. the PDataType which is an abstraction +# of Literal, Array[Data], and Hash[Literal, Data] nested to any depth. This means it is not possible to +# infer the type by simply looking at the inheritance hierarchy. The {Puppet::Pops::Types::TypeCalculator} should +# be used to answer questions about types. The {Puppet::Pops::Types::TypeFactory} should be used to create an instance +# of a type whenever one is needed. +# +# @api public +# +module Puppet::Pops::Types + + # The type of types. + # @api public + class PType < Puppet::Pops::Model::PopsObject + end + + # Base type for all types except {Puppet::Pops::Types::PType PType}, the type of types. + # @api public + class PObjectType < Puppet::Pops::Model::PopsObject + + module ClassModule + def hash + self.class.hash + end + + def ==(o) + self.class == o.class + end + + alias eql? == + end + + end + + # @api public + class PNilType < PObjectType + end + + # A flexible data type, being assignable to its subtypes as well as PArrayType and PHashType with element type assignable to PDataType. + # + # @api public + class PDataType < PObjectType + end + + # Type that is PDataType compatible, but is not a PCollectionType. + # @api public + class PLiteralType < PDataType + end + + # @api public + class PStringType < PLiteralType + end + + # @api public + class PNumericType < PLiteralType + end + + # @api public + class PIntegerType < PNumericType + end + + # @api public + class PFloatType < PNumericType + end + + # @api public + class PPatternType < PLiteralType + end + + # @api public + class PBooleanType < PLiteralType + end + + # @api public + class PCollectionType < PObjectType + contains_one_uni 'element_type', PObjectType + module ClassModule + def hash + [self.class, element_type].hash + end + + def ==(o) + self.class == o.class && element_type == o.element_type + end + end + end + + # @api public + class PArrayType < PCollectionType + module ClassModule + def hash + [self.class, element_type].hash + end + + def ==(o) + self.class == o.class && element_type == o.element_type + end + end + end + + # @api public + class PHashType < PCollectionType + contains_one_uni 'key_type', PObjectType + module ClassModule + def hash + [self.class, key_type, element_type].hash + end + + def ==(o) + self.class == o.class && key_type == o.key_type && element_type == o.element_type + end + end + end + + # @api public + class PRubyType < PObjectType + has_attr 'ruby_class', String + module ClassModule + def hash + [self.class, ruby_class].hash + end + + def ==(o) + self.class == o.class && ruby_class == o.ruby_class + end + end + + end +end diff --git a/lib/puppet/pops/validation.rb b/lib/puppet/pops/validation.rb index 5e9aa2b19..f003c1d42 100644 --- a/lib/puppet/pops/validation.rb +++ b/lib/puppet/pops/validation.rb @@ -1,18 +1,98 @@ # A module with base functionality for validation of a model. # -# * SeverityProducer - produces a severity (:error, :warning, :ignore) for a given Issue -# * DiagnosticProducer - produces a Diagnostic which binds an Issue to an occurrence of that issue -# * Acceptor - the receiver/sink/collector of computed diagnostics -# * DiagnosticFormatter - produces human readable output for a Diagnostic +# * **Factory** - an abstract factory implementation that makes it easier to create a new validation factory. +# * **SeverityProducer** - produces a severity (:error, :warning, :ignore) for a given Issue +# * **DiagnosticProducer** - produces a Diagnostic which binds an Issue to an occurrence of that issue +# * **Acceptor** - the receiver/sink/collector of computed diagnostics +# * **DiagnosticFormatter** - produces human readable output for a Diagnostic # module Puppet::Pops::Validation + + # This class is an abstract base implementation of a _model validation factory_ that creates a validator instance + # and associates it with a fully configured DiagnosticProducer. + # + # A _validator_ is responsible for validating a model. There may be different versions of validation available + # for one and the same model; e.g. different semantics for different puppet versions, or different types of + # validation configuration depending on the context/type of validation that should be performed (static, vs. runtime, etc.). + # + # This class is abstract and must be subclassed. The subclass must implement the methods + # {#label_provider} and {#checker}. It is also expected that the sublcass will override + # the severity_producer and configure the issues that should be reported as errors (i.e. if they should be ignored, produce + # a warning, or a deprecation warning). + # + # @abstract Subclass must implement {#checker}, and {#label_provider} + # @api public + # + class Factory + + # Produces a validator with the given acceptor as the recipient of produced diagnostics. + # The acceptor is where detected issues are received (and typically collected). + # + # @param acceptor [Acceptor] the acceptor is the receiver of all detected issues + # @return [#validate] a validator responding to `validate(model)` + # + # @api public + # + def validator(acceptor) + checker(diagnostic_producer(acceptor)) + end + + # Produces the diagnostics producer to use given an acceptor of issues. + # + # @param acceptor [Acceptor] the acceptor is the receiver of all detected issues + # @return [DiagnosticProducer] a detector of issues + # + # @api public + # + def diagnostic_producer(acceptor) + Puppet::Pops::Validation::DiagnosticProducer.new(acceptor, severity_producer(), label_provider()) + end + + # Produces the SeverityProducer to use + # Subclasses should implement and add specific overrides + # + # @return [SeverityProducer] a severity producer producing error, warning or ignore per issue + # + # @api public + # + def severity_producer + Puppet::Pops::Validation::SeverityProducer.new + end + + # Produces the checker to use. + # + # @abstract + # + # @api public + # + def checker(diagnostic_producer) + raise NoMethodError, "checker" + end + + # Produces the label provider to use. + # + # @abstract + # + # @api public + # + def label_provider + raise NoMethodError, "label_provider" + end + end + # Decides on the severity of a given issue. # The produced severity is one of `:error`, `:warning`, or `:ignore`. # By default, a severity of `:error` is produced for all issues. To configure the severity # of an issue call `#severity=(issue, level)`. # + # @return [Symbol] a symbol representing the severity `:error`, `:warning`, or `:ignore` + # + # @api public + # class SeverityProducer + # Creates a new instance where all issues are diagnosed as :error unless overridden. + # @api public # def initialize # If diagnose is not set, the default is returned by the block @@ -20,13 +100,17 @@ module Puppet::Pops::Validation end # Returns the severity of the given issue. - # @returns [Symbol] severity level :error, :warning, or :ignore + # @return [Symbol] severity level :error, :warning, or :ignore + # @api public # - def severity issue + def severity(issue) assert_issue(issue) @severities[issue] end + # @see {#severity} + # @api public + # def [] issue severity issue end @@ -35,26 +119,35 @@ module Puppet::Pops::Validation # # @param issue [Puppet::Pops::Issues::Issue] the issue for which to set severity # @param level [Symbol] the severity level (:error, :warning, or :ignore). + # @api public # - def []= issue, level + def []=(issue, level) assert_issue(issue) assert_severity(level) raise Puppet::DevError.new("Attempt to demote the hard issue '#{issue.issue_code}' to #{level}") unless issue.demotable? || level == :error @severities[issue] = level end - # Returns true if the issue should be reported or not. - # @returns [Boolean] this implementation returns true for errors and warnings + # Returns `true` if the issue should be reported or not. + # @return [Boolean] this implementation returns true for errors and warnings + # + # @api public # def should_report? issue diagnose = self[issue] diagnose == :error || diagnose == :warning || diagnose == :deprecation end + # Checks if the given issue is valid. + # @api private + # def assert_issue issue raise Puppet::DevError.new("Attempt to get validation severity for something that is not an Issue. (Got #{issue.class})") unless issue.is_a? Puppet::Pops::Issues::Issue end + # Checks if the given severity level is valid. + # @api private + # def assert_severity level raise Puppet::DevError.new("Illegal severity level: #{option}") unless [:ignore, :warning, :error, :deprecation].include? level end @@ -157,8 +250,11 @@ module Puppet::Pops::Validation def format_location diagnostic file = diagnostic.file - line = diagnostic.source_pos.line - pos = diagnostic.source_pos.pos + line = pos = nil + if diagnostic.source_pos + line = diagnostic.source_pos.line + pos = diagnostic.source_pos.pos + end if file && line && pos "#{file}:#{line}:#{pos}:" elsif file && line @@ -187,8 +283,12 @@ module Puppet::Pops::Validation # have to be used here for backwards compatibility. def format_location diagnostic file = diagnostic.file - line = diagnostic.source_pos.line - pos = diagnostic.source_pos.pos + line = pos = nil + if diagnostic.source_pos + line = diagnostic.source_pos.line + pos = diagnostic.source_pos.pos + end + if file && line && pos " at #{file}:#{line}:#{pos}" elsif file and line @@ -214,7 +314,7 @@ module Puppet::Pops::Validation # class Acceptor - # All diagnstic in the order they were issued + # All diagnostic in the order they were issued attr_reader :diagnostics # The number of :warning severity issues + number of :deprecation severity issues @@ -261,7 +361,7 @@ module Puppet::Pops::Validation end def errors_and_warnings - @diagnostics.select {|d| d.severity != :ignored} + @diagnostics.select {|d| d.severity != :ignore } end # Returns the ignored diagnostics in the order thwy were reported (if reported at all) @@ -269,9 +369,38 @@ module Puppet::Pops::Validation @diagnostics.select {|d| d.severity == :ignore } end - # Add a diagnostic to the set of diagnostics + # Add a diagnostic, or all diagnostics from another acceptor to the set of diagnostics + # @param diagnostic [Puppet::Pops::Validation::Diagnostic, Puppet::Pops::Validation::Acceptor] diagnostic(s) that should be accepted def accept(diagnostic) - self.send(diagnostic.severity, diagnostic) + if diagnostic.is_a?(Acceptor) + diagnostic.diagnostics.each {|d| self.send(d.severity, d)} + else + self.send(diagnostic.severity, diagnostic) + end + end + + # Prunes the contain diagnostics by removing those for which the given block returns true. + # The internal statistics is updated as a consequence of removing. + # @return [Array<Puppet::Pops::Validation::Diagnostic, nil] the removed set of diagnostics or nil if nothing was removed + # + def prune(&block) + removed = [] + @diagnostics.delete_if do |d| + if should_remove = yield(d) + removed << d + end + should_remove + end + removed.each do |d| + case d.severity + when :error + @error_count -= 1 + when :warning + @warning_count -= 1 + # there is not ignore_count + end + end + removed.empty? ? nil : removed end private diff --git a/lib/puppet/pops/validation/checker3_1.rb b/lib/puppet/pops/validation/checker3_1.rb index 69df726e1..65cbbbeaf 100644 --- a/lib/puppet/pops/validation/checker3_1.rb +++ b/lib/puppet/pops/validation/checker3_1.rb @@ -281,7 +281,7 @@ class Puppet::Pops::Validation::Checker3_1 end def check_QueryExpression(o) - rvalue(o.expr) if o.expr # is optional + query(o.expr) if o.expr # is optional end def relation_Object(o, rel_expr) diff --git a/lib/puppet/pops/validation/validator_factory_3_1.rb b/lib/puppet/pops/validation/validator_factory_3_1.rb index ee5ca3e8d..738eac404 100644 --- a/lib/puppet/pops/validation/validator_factory_3_1.rb +++ b/lib/puppet/pops/validation/validator_factory_3_1.rb @@ -1,20 +1,8 @@ # Configures validation suitable for 3.1 + iteration # -class Puppet::Pops::Validation::ValidatorFactory_3_1 +class Puppet::Pops::Validation::ValidatorFactory_3_1 < Puppet::Pops::Validation::Factory Issues = Puppet::Pops::Issues - # Produces a validator with the given acceptor as the recipient of produced diagnostics. - # - def validator acceptor - checker(diagnostic_producer(acceptor)) - end - - # Produces the diagnostics producer to use given an acceptor as the recipient of produced diagnostics - # - def diagnostic_producer acceptor - Puppet::Pops::Validation::DiagnosticProducer.new(acceptor, severity_producer(), label_provider()) - end - # Produces the checker to use def checker diagnostic_producer Puppet::Pops::Validation::Checker3_1.new(diagnostic_producer) @@ -27,12 +15,14 @@ class Puppet::Pops::Validation::ValidatorFactory_3_1 # Produces the severity producer to use def severity_producer - p = Puppet::Pops::Validation::SeverityProducer.new + p = super # Configure each issue that should **not** be an error # - p[Issues::RT_NO_STORECONFIGS_EXPORT] = :warning - p[Issues::RT_NO_STORECONFIGS] = :warning + # Validate as per the current runtime configuration + p[Issues::RT_NO_STORECONFIGS_EXPORT] = Puppet[:storeconfigs] ? :ignore : :warning + p[Issues::RT_NO_STORECONFIGS] = Puppet[:storeconfigs] ? :ignore : :warning + p[Issues::NAME_WITH_HYPHEN] = :deprecation p[Issues::DEPRECATED_NAME_AS_TYPE] = :deprecation diff --git a/lib/puppet/property.rb b/lib/puppet/property.rb index 44a0b611d..74c6e0292 100644 --- a/lib/puppet/property.rb +++ b/lib/puppet/property.rb @@ -194,7 +194,7 @@ class Puppet::Property < Puppet::Parameter def call_valuemethod(name, value) if method = self.class.value_option(name, :method) and self.respond_to?(method) begin - event = self.send(method) + self.send(method) rescue Puppet::Error raise rescue => detail @@ -611,7 +611,7 @@ class Puppet::Property < Puppet::Parameter end # (see #should=) - def value=(value) - self.should = value + def value=(values) + self.should = values end end diff --git a/lib/puppet/property/boolean.rb b/lib/puppet/property/boolean.rb new file mode 100644 index 000000000..bd21a2e79 --- /dev/null +++ b/lib/puppet/property/boolean.rb @@ -0,0 +1,7 @@ +require 'puppet/coercion' + +class Puppet::Property::Boolean < Puppet::Property + def unsafe_munge(value) + Puppet::Coercion.boolean(value) + end +end diff --git a/lib/puppet/property/keyvalue.rb b/lib/puppet/property/keyvalue.rb index 8184b77b3..3f604d891 100644 --- a/lib/puppet/property/keyvalue.rb +++ b/lib/puppet/property/keyvalue.rb @@ -70,7 +70,7 @@ module Puppet ";" end - # Retrieves the key-hash from the provider by invoking it's method named the same as this property. + # Retrieves the key-hash from the provider by invoking its method named the same as this property. # @return [Hash] the hash from the provider, or `:absent` # def retrieve diff --git a/lib/puppet/provider.rb b/lib/puppet/provider.rb index 43e14eeec..f875e3df0 100644 --- a/lib/puppet/provider.rb +++ b/lib/puppet/provider.rb @@ -185,7 +185,7 @@ class Puppet::Provider # is lazy (when a resource is evaluated) and the absence of commands # that will be present after other resources have been applied no longer needs to be specified as # optional. - # @param [Hash{String => String}] command_specs Named commands that the provider will + # @param [Hash{String => String}] hash Named commands that the provider will # be executing on the system. Each command is specified with a name and the path of the executable. # (@see #has_command) # @see commands @@ -564,7 +564,7 @@ class Puppet::Provider # Sets the given parameters values as the current values for those parameters. # Other parameters are unchanged. - # @param [Array<Puppet::Parameter] the parameters with values that should be set + # @param [Array<Puppet::Parameter>] params the parameters with values that should be set # @return [void] # def set(params) diff --git a/lib/puppet/provider/aixobject.rb b/lib/puppet/provider/aixobject.rb index dce452a0e..ed27b4e52 100755 --- a/lib/puppet/provider/aixobject.rb +++ b/lib/puppet/provider/aixobject.rb @@ -7,24 +7,20 @@ class Puppet::Provider::AixObject < Puppet::Provider desc "Generic AIX resource provider" # The real provider must implement these functions. - def lscmd(value=@resource[:name]) - raise Puppet::Error, "Method not defined #{@resource.class.name} #{@resource.name}: #{detail}" + def lscmd( _value = @resource[:name] ) + raise Puppet::Error, "Method not defined #{@resource.class.name} #{@resource.name}: Base AixObject provider doesn't implement lscmd" end - def lscmd(value=@resource[:name]) - raise Puppet::Error, "Method not defined #{@resource.class.name} #{@resource.name}: #{detail}" + def addcmd( _extra_attrs = [] ) + raise Puppet::Error, "Method not defined #{@resource.class.name} #{@resource.name}: Base AixObject provider doesn't implement addcmd" end - def addcmd(extra_attrs = []) - raise Puppet::Error, "Method not defined #{@resource.class.name} #{@resource.name}: #{detail}" - end - - def modifycmd(attributes_hash) - raise Puppet::Error, "Method not defined #{@resource.class.name} #{@resource.name}: #{detail}" + def modifycmd( _attributes_hash = {} ) + raise Puppet::Error, "Method not defined #{@resource.class.name} #{@resource.name}: Base AixObject provider doesn't implement modifycmd" end def deletecmd - raise Puppet::Error, "Method not defined #{@resource.class.name} #{@resource.name}: #{detail}" + raise Puppet::Error, "Method not defined #{@resource.class.name} #{@resource.name}: Base AixObject provider doesn't implement deletecmd" end # Valid attributes to be managed by this provider. @@ -152,7 +148,7 @@ class Puppet::Provider::AixObject < Puppet::Provider # Parse AIX command attributes from the output of an AIX command, that # which format is a list of space separated of key=value pairs: # "uid=100 groups=a,b,c". - # It returns an hash. + # It returns a hash. # # If a mapping is provided, the keys are translated as defined in the # mapping hash. And only values included in mapping will be added @@ -161,7 +157,7 @@ class Puppet::Provider::AixObject < Puppet::Provider def parse_attr_list(str, mapping=self.class.attribute_mapping_from) properties = {} attrs = [] - if !str or (attrs = str.split()).empty? + if str.nil? or (attrs = str.split()).empty? return nil end @@ -169,11 +165,13 @@ class Puppet::Provider::AixObject < Puppet::Provider if i.include? "=" # Ignore if it does not include '=' (key_str, val) = i.split('=') # Check the key - if !key_str or key_str.empty? + if key_str.nil? or key_str.empty? info "Empty key in string 'i'?" continue end + key_str.strip! key = key_str.to_sym + val.strip! if val properties = self.load_attribute(key, val, mapping, properties) end @@ -193,7 +191,7 @@ class Puppet::Provider::AixObject < Puppet::Provider def parse_colon_list(str, key_list, mapping=self.class.attribute_mapping_from) properties = {} attrs = [] - if !str or (attrs = str.split(':')).empty? + if str.nil? or (attrs = str.split(':')).empty? return nil end @@ -227,9 +225,9 @@ class Puppet::Provider::AixObject < Puppet::Provider # Execute lsuser, split all attributes and add them to a dict. begin output = execute(self.lscmd) - @objectinfo = self.parse_command_output(execute(self.lscmd)) + @objectinfo = self.parse_command_output(output) # All attributtes without translation - @objectosinfo = self.parse_command_output(execute(self.lscmd), nil) + @objectosinfo = self.parse_command_output(output, nil) rescue Puppet::ExecutionFailure => detail # Print error if needed. FIXME: Do not check the user here. Puppet.debug "aix.getinfo(): Could not find #{@resource.class.name} #{@resource.name}: #{detail}" @@ -241,10 +239,10 @@ class Puppet::Provider::AixObject < Puppet::Provider # Like getinfo, but it will not use the mapping to translate the keys and values. # It might be usefult to retrieve some raw information. def getosinfo(refresh = false) - if @objectosinfo .nil? or refresh == true + if @objectosinfo.nil? or refresh == true getinfo(refresh) end - @objectosinfo + @objectosinfo || Hash.new end @@ -290,7 +288,7 @@ class Puppet::Provider::AixObject < Puppet::Provider # providers, preferably with values already filled in, not resources. def self.instances objects=[] - self.list_all.each { |entry| + list_all.each { |entry| objects << new(:name => entry, :ensure => :present) } objects @@ -383,7 +381,7 @@ class Puppet::Provider::AixObject < Puppet::Provider end # Refresh de info. - hash = getinfo(true) + getinfo(true) end def initialize(resource) diff --git a/lib/puppet/provider/augeas/augeas.rb b/lib/puppet/provider/augeas/augeas.rb index f8cd5aabc..2246636da 100644 --- a/lib/puppet/provider/augeas/augeas.rb +++ b/lib/puppet/provider/augeas/augeas.rb @@ -327,6 +327,8 @@ Puppet::Type.type(:augeas).provide(:augeas) do def print_errors(errors) errors.each do |errnode| + error = @aug.get(errnode) + debug("#{errnode} = #{error}") unless error.nil? @aug.match("#{errnode}/*").each do |subnode| subvalue = @aug.get(subnode) debug("#{subnode} = #{subvalue}") @@ -334,7 +336,7 @@ Puppet::Type.type(:augeas).provide(:augeas) do end end - # Determines if augeas acutally needs to run. + # Determines if augeas actually needs to run. def need_to_run? force = resource[:force] return_value = true diff --git a/lib/puppet/provider/command.rb b/lib/puppet/provider/command.rb index b3b377674..d737c272b 100644 --- a/lib/puppet/provider/command.rb +++ b/lib/puppet/provider/command.rb @@ -16,8 +16,8 @@ class Puppet::Provider::Command @options = options end - # @param [Array<String>] Any command line arguments to pass to the executable - # @returns The output from the command + # @param args [Array<String>] Any command line arguments to pass to the executable + # @return The output from the command def execute(*args) resolved_executable = @resolver.which(@executable) or raise Puppet::Error, "Command #{@name} is missing" @executor.execute([resolved_executable] + args, @options) diff --git a/lib/puppet/provider/group/aix.rb b/lib/puppet/provider/group/aix.rb index a7ae72cf0..666748378 100755 --- a/lib/puppet/provider/group/aix.rb +++ b/lib/puppet/provider/group/aix.rb @@ -116,7 +116,7 @@ Puppet::Type.type(:group).provide :aix, :parent => Puppet::Provider::AixObject d end def attributes - filter_attributes(getosinfo(refresh = false)) + filter_attributes(getosinfo(false)) end def attributes=(attr_hash) diff --git a/lib/puppet/provider/group/ldap.rb b/lib/puppet/provider/group/ldap.rb index 19e9e1b14..01ba43d3b 100644 --- a/lib/puppet/provider/group/ldap.rb +++ b/lib/puppet/provider/group/ldap.rb @@ -40,6 +40,6 @@ Puppet::Type.type(:group).provide :ldap, :parent => Puppet::Provider::Ldap do # Only use the first result. group = result[0] - gid = group[:gid][0] + group[:gid][0] end end diff --git a/lib/puppet/provider/macauthorization/macauthorization.rb b/lib/puppet/provider/macauthorization/macauthorization.rb index 1ec796de4..fe9c56985 100644 --- a/lib/puppet/provider/macauthorization/macauthorization.rb +++ b/lib/puppet/provider/macauthorization/macauthorization.rb @@ -186,7 +186,7 @@ Puppet::Type.type(:macauthorization).provide :macauthorization, :parent => Puppe Plist::Emit.save_plist(values, tmp.path) cmds = [] cmds << :security << "authorizationdb" << "write" << name - output = execute(cmds, :failonfail => false, :combine => false, :stdinfile => tmp.path.to_s) + execute(cmds, :failonfail => false, :combine => false, :stdinfile => tmp.path.to_s) rescue Errno::EACCES => e raise Puppet::Error.new("Cannot save right to #{tmp.path}: #{e}") ensure diff --git a/lib/puppet/provider/mailalias/aliases.rb b/lib/puppet/provider/mailalias/aliases.rb index 35c2f98fe..87ee5b465 100755 --- a/lib/puppet/provider/mailalias/aliases.rb +++ b/lib/puppet/provider/mailalias/aliases.rb @@ -1,11 +1,9 @@ require 'puppet/provider/parsedfile' - - Puppet::Type.type(:mailalias).provide( - :aliases, +Puppet::Type.type(:mailalias).provide( + :aliases, :parent => Puppet::Provider::ParsedFile, :default_target => "/etc/aliases", - :filetype => :flat ) do text_line :comment, :match => /^#/ @@ -13,10 +11,7 @@ require 'puppet/provider/parsedfile' record_line :aliases, :fields => %w{name recipient}, :separator => /\s*:\s*/, :block_eval => :instance do def post_parse(record) - # LAK:NOTE See http://snurl.com/21zf8 [groups_google_com] - # It's not sufficient to assign to an existing hash. - recipient = record[:recipient].split(/\s*,\s*/).collect { |d| d.gsub(/^['"]|['"]$/, '') } - record[:recipient] = recipient + record[:recipient] = record[:recipient].split(/\s*,\s*/).collect { |d| d.gsub(/^['"]|['"]$/, '') } record end diff --git a/lib/puppet/provider/mcx/mcxcontent.rb b/lib/puppet/provider/mcx/mcxcontent.rb index 0c0061278..d6c871434 100644 --- a/lib/puppet/provider/mcx/mcxcontent.rb +++ b/lib/puppet/provider/mcx/mcxcontent.rb @@ -83,7 +83,7 @@ Puppet::Type.type(:mcx).provide :mcxcontent, :parent => Puppet::Provider do def exists? begin has_mcx? - rescue Puppet::ExecutionFailure => e + rescue Puppet::ExecutionFailure return false end end @@ -111,10 +111,16 @@ Puppet::Type.type(:mcx).provide :mcxcontent, :parent => Puppet::Provider do ds_t = TypeMap[ds_type] ds_path = "/Local/Default/#{ds_t}/#{ds_name}" + if has_mcx? + Puppet.debug "Removing MCX from #{ds_path}" + dscl 'localhost', '-mcxdelete', ds_path + end + tmp = Tempfile.new('puppet_mcx') begin tmp << val tmp.flush + Puppet.debug "Importing MCX into #{ds_path}" dscl 'localhost', '-mcximport', ds_path, tmp.path ensure tmp.close diff --git a/lib/puppet/provider/mount.rb b/lib/puppet/provider/mount.rb index e2aba8076..0839d99ce 100644 --- a/lib/puppet/provider/mount.rb +++ b/lib/puppet/provider/mount.rb @@ -5,10 +5,15 @@ require 'puppet' module Puppet::Provider::Mount # This only works when the mount point is synced to the fstab. def mount - # Manually pass the mount options in, since some OSes *cough*OS X*cough* don't - # read from /etc/fstab but still want to use this type. args = [] - args << "-o" << self.options if self.options and self.options != :absent + + # In general we do not have to pass mountoptions because we always + # flush /etc/fstab before attempting to mount. But old code suggests + # that MacOS always needs the mount options to be explicitly passed to + # the mount command + if Facter.value(:kernel) == 'Darwin' + args << "-o" << self.options if self.options and self.options != :absent + end args << resource[:name] mountcmd(*args) diff --git a/lib/puppet/provider/nameservice.rb b/lib/puppet/provider/nameservice.rb index edb098012..8e76b67af 100644 --- a/lib/puppet/provider/nameservice.rb +++ b/lib/puppet/provider/nameservice.rb @@ -227,7 +227,7 @@ class Puppet::Provider::NameService < Puppet::Provider @etcmethod ||= ("get" + self.class.section.to_s + "nam").intern begin @objectinfo = Etc.send(@etcmethod, @resource[:name]) - rescue ArgumentError => detail + rescue ArgumentError @objectinfo = nil end end diff --git a/lib/puppet/provider/nameservice/directoryservice.rb b/lib/puppet/provider/nameservice/directoryservice.rb index eb332dfae..f0cbecea4 100644 --- a/lib/puppet/provider/nameservice/directoryservice.rb +++ b/lib/puppet/provider/nameservice/directoryservice.rb @@ -108,7 +108,7 @@ class Puppet::Provider::NameService::DirectoryService < Puppet::Provider::NameSe # JJM: List all objects of this Puppet::Type already present on the system. begin dscl_output = execute(get_exec_preamble("-list")) - rescue Puppet::ExecutionFailure => detail + rescue Puppet::ExecutionFailure fail("Could not get #{@resource_type.name} list from DirectoryService") end dscl_output.split("\n") @@ -166,7 +166,7 @@ class Puppet::Provider::NameService::DirectoryService < Puppet::Provider::NameSe dscl_vector = get_exec_preamble("-read", resource_name) begin dscl_output = execute(dscl_vector) - rescue Puppet::ExecutionFailure => detail + rescue Puppet::ExecutionFailure fail("Could not get report. command execution failed.") end @@ -237,7 +237,7 @@ class Puppet::Provider::NameService::DirectoryService < Puppet::Provider::NameSe dscl_vector = self.get_exec_preamble("-merge", resource_name) dscl_vector << "AuthenticationAuthority" << ";ShadowHash;" begin - dscl_output = execute(dscl_vector) + execute(dscl_vector) rescue Puppet::ExecutionFailure => detail fail("Could not set AuthenticationAuthority.") end @@ -332,7 +332,7 @@ class Puppet::Provider::NameService::DirectoryService < Puppet::Provider::NameSe def self.convert_xml_to_binary(plist_data) Puppet.debug('Converting XML plist to binary') Puppet.debug('Executing: \'plutil -convert binary1 -o - -\'') - IO.popen('plutil -convert binary1 -o - -', mode='r+') do |io| + IO.popen('plutil -convert binary1 -o - -', 'r+') do |io| io.write plist_data.to_plist io.close_write @converted_plist = io.read @@ -345,7 +345,7 @@ class Puppet::Provider::NameService::DirectoryService < Puppet::Provider::NameSe def self.convert_binary_to_xml(plist_data) Puppet.debug('Converting binary plist to XML') Puppet.debug('Executing: \'plutil -convert xml1 -o - -\'') - IO.popen('plutil -convert xml1 -o - -', mode='r+') do |io| + IO.popen('plutil -convert xml1 -o - -', 'r+') do |io| io.write plist_data io.close_write @converted_plist = io.read diff --git a/lib/puppet/provider/package/appdmg.rb b/lib/puppet/provider/package/appdmg.rb index 1e292e351..2910c8599 100644 --- a/lib/puppet/provider/package/appdmg.rb +++ b/lib/puppet/provider/package/appdmg.rb @@ -88,7 +88,7 @@ Puppet::Type.type(:package).provide(:appdmg, :parent => Puppet::Provider::Packag end end ensure - FileUtils.remove_entry_secure(tmpdir, force=true) + FileUtils.remove_entry_secure(tmpdir, true) end end diff --git a/lib/puppet/provider/package/apt.rb b/lib/puppet/provider/package/apt.rb index b7bc031b8..16618d359 100755 --- a/lib/puppet/provider/package/apt.rb +++ b/lib/puppet/provider/package/apt.rb @@ -45,7 +45,6 @@ Puppet::Type.type(:package).provide :apt, :parent => :dpkg, :source => :dpkg do checkforcdrom cmd = %w{-q -y} - keep = "" if config = @resource[:configfiles] if config == :keep cmd << "-o" << 'DPkg::Options::=--force-confold' diff --git a/lib/puppet/provider/package/dpkg.rb b/lib/puppet/provider/package/dpkg.rb index e55dbeb45..062091bec 100755 --- a/lib/puppet/provider/package/dpkg.rb +++ b/lib/puppet/provider/package/dpkg.rb @@ -11,39 +11,75 @@ Puppet::Type.type(:package).provide :dpkg, :parent => Puppet::Provider::Package commands :dpkg_deb => "/usr/bin/dpkg-deb" commands :dpkgquery => "/usr/bin/dpkg-query" + # Performs a dpkgquery call with a pipe so that output can be processed + # inline in a passed block. + # @param args [Array<String>] any command line arguments to be appended to the command + # @param block expected to be passed on to execpipe + # @return whatever the block returns + # @see Puppet::Util::Execution.execpipe + # @api private + def self.dpkgquery_piped(*args, &block) + cmd = args.unshift(command(:dpkgquery)) + Puppet::Util::Execution.execpipe(cmd, &block) + end + def self.instances packages = [] # list out all of the packages - cmd = "#{command(:dpkgquery)} -W --showformat '${Status} ${Package} ${Version}\\n'" - Puppet.debug "Executing '#{cmd}'" - Puppet::Util::Execution.execpipe(cmd) do |process| - # our regex for matching dpkg output - regex = %r{^(\S+) +(\S+) +(\S+) (\S+) (\S*)$} - fields = [:desired, :error, :status, :name, :ensure] - hash = {} - - # now turn each returned line into a package object - process.each_line { |line| - if hash = parse_line(line) - packages << new(hash) - end - } + dpkgquery_piped('-W', '--showformat', self::DPKG_QUERY_FORMAT_STRING) do |pipe| + until pipe.eof? + hash = parse_multi_line(pipe) + packages << new(hash) if hash + end end packages end - self::REGEX = %r{^(\S+) +(\S+) +(\S+) (\S+) (\S*)$} - self::FIELDS = [:desired, :error, :status, :name, :ensure] + private + + # Note: self:: is required here to keep these constants in the context of what will + # eventually become this Puppet:Type::Package::ProviderDpkg class. + self::DPKG_DESCRIPTION_DELIMITER = ':DESC:' + self::DPKG_QUERY_FORMAT_STRING = %Q{'${Status} ${Package} ${Version} #{self::DPKG_DESCRIPTION_DELIMITER} ${Description}\\n#{self::DPKG_DESCRIPTION_DELIMITER}\\n'} + self::FIELDS_REGEX = %r{^(\S+) +(\S+) +(\S+) (\S+) (\S*) #{self::DPKG_DESCRIPTION_DELIMITER} (.*)$} + self::FIELDS= [:desired, :error, :status, :name, :ensure, :description] + self::END_REGEX = %r{^#{self::DPKG_DESCRIPTION_DELIMITER}$} + + # Handles parsing one package's worth of multi-line dpkg-query output. Will + # emit warnings if it encounters an initial line that does not match + # DPKG_QUERY_FORMAT_STRING. Swallows extra description lines silently. + # + # @param pipe [IO] the pipe yielded while processing dpkg output + # @return [Hash,nil] parsed dpkg-query entry as a hash of FIELDS strings or + # nil if we failed to parse + # @api private + def self.parse_multi_line(pipe) + + line = pipe.gets + unless hash = parse_line(line) + Puppet.warning "Failed to match dpkg-query line #{line.inspect}" + return nil + end + + consume_excess_description(pipe) + + return hash + end + # @param line [String] one line of dpkg-query output + # @return [Hash,nil] a hash of FIELDS or nil if we failed to match + # @api private def self.parse_line(line) - if match = self::REGEX.match(line) + hash = nil + + if match = self::FIELDS_REGEX.match(line) hash = {} - self::FIELDS.zip(match.captures) { |field,value| + self::FIELDS.zip(match.captures) do |field,value| hash[field] = value - } + end hash[:provider] = self.name @@ -53,14 +89,33 @@ Puppet::Type.type(:package).provide :dpkg, :parent => Puppet::Provider::Package hash[:ensure] = :absent end hash[:ensure] = :held if hash[:desired] == 'hold' - else - Puppet.warning "Failed to match dpkg-query line #{line.inspect}" - return nil end - hash + return hash end + # Silently consumes the extra description lines from dpkg-query and brings + # us to the next package entry start. + # + # @note dpkg-query Description field has a one line summary and a multi-line + # description. dpkg-query binary:Summary is what we want to use but was + # introduced in 2012 dpkg 1.16.2 + # (https://launchpad.net/debian/+source/dpkg/1.16.2) and is not not available + # in older Debian versions. So we're placing a delimiter marker at the end + # of the description so we can consume and ignore the multiline description + # without issuing warnings + # + # @param pipe [IO] the pipe yielded while processing dpkg output + # @return nil + def self.consume_excess_description(pipe) + until pipe.eof? + break if self::END_REGEX.match(pipe.gets) + end + return nil + end + + public + def install unless file = @resource[:source] raise ArgumentError, "You cannot install dpkg packages without a source" @@ -94,26 +149,24 @@ Puppet::Type.type(:package).provide :dpkg, :parent => Puppet::Provider::Package end def query - packages = [] - - fields = [:desired, :error, :status, :name, :ensure] - - hash = {} + hash = nil # list out our specific package begin - output = dpkgquery( + self.class.dpkgquery_piped( "-W", "--showformat", - '${Status} ${Package} ${Version}\\n', + self.class::DPKG_QUERY_FORMAT_STRING, @resource[:name] - ) + ) do |pipe| + hash = self.class.parse_multi_line(pipe) + end rescue Puppet::ExecutionFailure # dpkg-query exits 1 if the package is not found. return {:ensure => :purged, :status => 'missing', :name => @resource[:name], :error => 'ok'} end - hash = self.class.parse_line(output) || {:ensure => :absent, :status => 'missing', :name => @resource[:name], :error => 'ok'} + hash ||= {:ensure => :absent, :status => 'missing', :name => @resource[:name], :error => 'ok'} if hash[:error] != "ok" raise Puppet::Error.new( @@ -148,4 +201,5 @@ Puppet::Type.type(:package).provide :dpkg, :parent => Puppet::Provider::Package execute([:dpkg, "--set-selections"], :failonfail => false, :combine => false, :stdinfile => tmpfile.path.to_s) end end + end diff --git a/lib/puppet/provider/package/fink.rb b/lib/puppet/provider/package/fink.rb index 663c79866..7efa131b9 100755 --- a/lib/puppet/provider/package/fink.rb +++ b/lib/puppet/provider/package/fink.rb @@ -35,8 +35,6 @@ Puppet::Type.type(:package).provide :fink, :parent => :dpkg, :source => :dpkg do end cmd = %w{-b -q -y} - keep = "" - cmd << :install << str finkcmd(cmd) diff --git a/lib/puppet/provider/package/freebsd.rb b/lib/puppet/provider/package/freebsd.rb index 5008c9143..adb72c607 100755 --- a/lib/puppet/provider/package/freebsd.rb +++ b/lib/puppet/provider/package/freebsd.rb @@ -16,8 +16,6 @@ Puppet::Type.type(:package).provide :freebsd, :parent => :openbsd do end def install - should = @resource.should(:ensure) - if @resource[:source] =~ /\/$/ if @resource[:source] =~ /^(ftp|https?):/ Puppet::Util.withenv :PACKAGESITE => @resource[:source] do diff --git a/lib/puppet/provider/package/openbsd.rb b/lib/puppet/provider/package/openbsd.rb index ab2b610a2..75c01ec83 100755 --- a/lib/puppet/provider/package/openbsd.rb +++ b/lib/puppet/provider/package/openbsd.rb @@ -10,6 +10,8 @@ Puppet::Type.type(:package).provide :openbsd, :parent => Puppet::Provider::Packa confine :operatingsystem => :openbsd has_feature :versionable + has_feature :install_options + has_feature :uninstall_options def self.instances packages = [] @@ -17,7 +19,7 @@ Puppet::Type.type(:package).provide :openbsd, :parent => Puppet::Provider::Packa begin execpipe(listcmd) do |process| # our regex for matching pkg_info output - regex = /^(.*)-(\d[^-]*)[-]?(\D*)(.*)$/ + regex = /^(.*)-(\d[^-]*)[-]?(\w*)(.*)$/ fields = [:name, :ensure, :flavor ] hash = {} @@ -27,8 +29,6 @@ Puppet::Type.type(:package).provide :openbsd, :parent => Puppet::Provider::Packa fields.zip(match.captures) { |field,value| hash[field] = value } - yup = nil - name = hash[:name] hash[:provider] = self.name @@ -52,15 +52,18 @@ Puppet::Type.type(:package).provide :openbsd, :parent => Puppet::Provider::Packa [command(:pkginfo), "-a"] end - def install - should = @resource.should(:ensure) - + def parse_pkgconf unless @resource[:source] if File.exist?("/etc/pkg.conf") File.open("/etc/pkg.conf", "rb").readlines.each do |line| if matchdata = line.match(/^installpath\s*=\s*(.+)\s*$/i) @resource[:source] = matchdata[1] - break + elsif matchdata = line.match(/^installpath\s*\+=\s*(.+)\s*$/i) + if @resource[:source].nil? + @resource[:source] = matchdata[1] + else + @resource[:source] += ":" + matchdata[1] + end end end @@ -73,6 +76,12 @@ Puppet::Type.type(:package).provide :openbsd, :parent => Puppet::Provider::Packa "You must specify a package source or configure an installpath in /etc/pkg.conf" end end + end + + def install + cmd = [] + + parse_pkgconf if @resource[:source][-1,1] == ::File::SEPARATOR e_vars = { 'PKG_PATH' => @resource[:source] } @@ -82,14 +91,16 @@ Puppet::Type.type(:package).provide :openbsd, :parent => Puppet::Provider::Packa full_name = @resource[:source] end - Puppet::Util.withenv(e_vars) { pkgadd full_name } + cmd << install_options + cmd << full_name + + Puppet::Util.withenv(e_vars) { pkgadd cmd.flatten.compact.join(' ') } end def get_version execpipe([command(:pkginfo), "-I", @resource[:name]]) do |process| # our regex for matching pkg_info output regex = /^(.*)-(\d[^-]*)[-]?(\D*)(.*)$/ - fields = [ :name, :version, :flavor ] master_version = 0 version = -1 @@ -120,7 +131,43 @@ Puppet::Type.type(:package).provide :openbsd, :parent => Puppet::Provider::Packa end end + def install_options + join_options(resource[:install_options]) + end + + def uninstall_options + join_options(resource[:uninstall_options]) + end + + # Turns a array of options into flags to be passed to pkg_add(8) and + # pkg_delete(8). The options can be passed as a string or hash. Note + # that passing a hash should only be used in case -Dfoo=bar must be passed, + # which can be accomplished with: + # install_options => [ { '-Dfoo' => 'bar' } ] + # Regular flags like '-L' must be passed as a string. + # @param options [Array] + # @return Concatenated list of options + # @api private + def join_options(options) + return unless options + + options.collect do |val| + case val + when Hash + val.keys.sort.collect do |k| + "#{k}=#{val[k]}" + end.join(' ') + else + val + end + end + end + def uninstall - pkgdelete @resource[:name] + pkgdelete uninstall_options.flatten.compact.join(' '), @resource[:name] + end + + def purge + pkgdelete "-c", "-q", @resource[:name] end end diff --git a/lib/puppet/provider/package/opkg.rb b/lib/puppet/provider/package/opkg.rb index 619ba1d18..439afeb01 100755 --- a/lib/puppet/provider/package/opkg.rb +++ b/lib/puppet/provider/package/opkg.rb @@ -18,7 +18,6 @@ Puppet::Type.type(:package).provide :opkg, :source => :opkg, :parent => Puppet:: process.each_line { |line| if match = regex.match(line) fields.zip(match.captures) { |field,value| hash[field] = value } - name = hash[:name] hash[:provider] = self.name packages << new(hash) hash = {} diff --git a/lib/puppet/provider/package/pacman.rb b/lib/puppet/provider/package/pacman.rb index 33c7eb42f..0b47a4e8b 100644 --- a/lib/puppet/provider/package/pacman.rb +++ b/lib/puppet/provider/package/pacman.rb @@ -84,7 +84,6 @@ Puppet::Type.type(:package).provide :pacman, :parent => Puppet::Provider::Packag hash[field] = value } - name = hash[:name] hash[:provider] = self.name packages << new(hash) diff --git a/lib/puppet/provider/package/pip.rb b/lib/puppet/provider/package/pip.rb index 22762c293..d91753fa5 100644 --- a/lib/puppet/provider/package/pip.rb +++ b/lib/puppet/provider/package/pip.rb @@ -48,7 +48,7 @@ Puppet::Type.type(:package).provide :pip, # it is not installed or `pip` itself is not available. def query self.class.instances.each do |provider_pip| - return provider_pip.properties if @resource[:name] == provider_pip.name + return provider_pip.properties if @resource[:name].downcase == provider_pip.name.downcase end return nil end diff --git a/lib/puppet/provider/package/pkgdmg.rb b/lib/puppet/provider/package/pkgdmg.rb index be9d3a70f..15c3639de 100644 --- a/lib/puppet/provider/package/pkgdmg.rb +++ b/lib/puppet/provider/package/pkgdmg.rb @@ -10,6 +10,7 @@ require 'puppet/provider/package' require 'facter/util/plist' +require 'puppet/util/http_proxy' Puppet::Type.type(:package).provide :pkgdmg, :parent => Puppet::Provider::Package do desc "Package management based on Apple's Installer.app and @@ -53,20 +54,30 @@ Puppet::Type.type(:package).provide :pkgdmg, :parent => Puppet::Provider::Packag end def self.installpkgdmg(source, name) + http_proxy_host = Puppet::Util::HttpProxy.http_proxy_host + http_proxy_port = Puppet::Util::HttpProxy.http_proxy_port + unless source =~ /\.dmg$/i || source =~ /\.pkg$/i raise Puppet::Error.new("Mac OS X PKG DMG's must specify a source string ending in .dmg or flat .pkg file") end require 'open-uri' cached_source = source tmpdir = Dir.mktmpdir + ext = /(\.dmg|\.pkg)$/i.match(source)[0] begin if %r{\A[A-Za-z][A-Za-z0-9+\-\.]*://} =~ cached_source - cached_source = File.join(tmpdir, name) - begin - curl "-o", cached_source, "-C", "-", "-k", "-L", "-s", "--url", source - Puppet.debug "Success: curl transfered [#{name}]" + cached_source = File.join(tmpdir, "#{name}#{ext}") + args = [ "-o", cached_source, "-C", "-", "-k", "-L", "-s", "--fail", "--url", source ] + if http_proxy_host and http_proxy_port + args << "--proxy" << "#{http_proxy_host}:#{http_proxy_port}" + elsif http_proxy_host and not http_proxy_port + args << "--proxy" << http_proxy_host + end + begin + curl *args + Puppet.debug "Success: curl transfered [#{name}] (via: curl #{args.join(" ")})" rescue Puppet::ExecutionFailure - Puppet.debug "curl did not transfer [#{name}]. Falling back to slower open-uri transfer methods." + Puppet.debug "curl #{args.join(" ")} did not transfer [#{name}]. Falling back to slower open-uri transfer methods." cached_source = source end end @@ -97,7 +108,7 @@ Puppet::Type.type(:package).provide :pkgdmg, :parent => Puppet::Provider::Packag installpkg(cached_source, name, source) end ensure - FileUtils.remove_entry_secure(tmpdir, force=true) + FileUtils.remove_entry_secure(tmpdir, true) end end diff --git a/lib/puppet/provider/package/pkgutil.rb b/lib/puppet/provider/package/pkgutil.rb index a36574ff0..157066415 100755 --- a/lib/puppet/provider/package/pkgutil.rb +++ b/lib/puppet/provider/package/pkgutil.rb @@ -63,7 +63,7 @@ Puppet::Type.type(:package).provide :pkgutil, :parent => :sun, :source => :sun d def self.availlist output = pkguti ["-a"] - list = output.split("\n").collect do |line| + output.split("\n").collect do |line| next if line =~ /^common\s+package/ # header of package list next if noise?(line) diff --git a/lib/puppet/provider/package/portage.rb b/lib/puppet/provider/package/portage.rb index b1fb5df84..0daa68a40 100644 --- a/lib/puppet/provider/package/portage.rb +++ b/lib/puppet/provider/package/portage.rb @@ -6,7 +6,15 @@ Puppet::Type.type(:package).provide :portage, :parent => Puppet::Provider::Packa has_feature :versionable - commands :emerge => "/usr/bin/emerge", :eix => "/usr/bin/eix", :update_eix => "/usr/bin/eix-update" + { + :emerge => "/usr/bin/emerge", + :eix => "/usr/bin/eix", + :update_eix => "/usr/bin/eix-update", + }.each_pair do |name, path| + has_command(name, path) do + environment :HOME => '/' + end + end confine :operatingsystem => :gentoo diff --git a/lib/puppet/provider/package/ports.rb b/lib/puppet/provider/package/ports.rb index 62c65c8b8..9141e30f5 100755 --- a/lib/puppet/provider/package/ports.rb +++ b/lib/puppet/provider/package/ports.rb @@ -44,12 +44,12 @@ Puppet::Type.type(:package).provide :ports, :parent => :freebsd, :source => :fre match = $2 info = $3 - unless pkgstuff =~ /^(\S+)-([^-\s]+)$/ + unless pkgstuff =~ /^\S+-([^-\s]+)$/ raise Puppet::Error, "Could not match package info '#{pkgstuff}'" end - name, version = $1, $2 + version = $1 if match == "=" or match == ">" # we're up to date or more recent diff --git a/lib/puppet/provider/package/rpm.rb b/lib/puppet/provider/package/rpm.rb index 63df5cc49..910669b46 100755 --- a/lib/puppet/provider/package/rpm.rb +++ b/lib/puppet/provider/package/rpm.rb @@ -7,9 +7,13 @@ Puppet::Type.type(:package).provide :rpm, :source => :rpm, :parent => Puppet::Pr has_feature :versionable + # Note: self:: is required here to keep these constants in the context of what will + # eventually become this Puppet:Type::Package::ProviderRpm class. + self::RPM_DESCRIPTION_DELIMITER = ':DESC:' # The query format by which we identify installed packages - NEVRAFORMAT = "%{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}" - NEVRA_FIELDS = [:name, :epoch, :version, :release, :arch] + self::NEVRA_FORMAT = %Q{'%{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH} #{self::RPM_DESCRIPTION_DELIMITER} %{SUMMARY}\\n'} + self::NEVRA_REGEX = %r{^(\S+) (\S+) (\S+) (\S+) (\S+)(?: #{self::RPM_DESCRIPTION_DELIMITER} ?(.*))?$} + self::NEVRA_FIELDS = [:name, :epoch, :version, :release, :arch, :description] commands :rpm => "rpm" @@ -44,11 +48,11 @@ Puppet::Type.type(:package).provide :rpm, :source => :rpm, :parent => Puppet::Pr # list out all of the packages begin - execpipe("#{command(:rpm)} -qa #{nosignature} #{nodigest} --qf '#{NEVRAFORMAT}\n'") { |process| + execpipe("#{command(:rpm)} -qa #{nosignature} #{nodigest} --qf #{self::NEVRA_FORMAT}") { |process| # now turn each returned line into a package object process.each_line { |line| hash = nevra_to_hash(line) - packages << new(hash) + packages << new(hash) unless hash.empty? } } rescue Puppet::ExecutionFailure @@ -65,7 +69,7 @@ Puppet::Type.type(:package).provide :rpm, :source => :rpm, :parent => Puppet::Pr #NOTE: Prior to a fix for issue 1243, this method potentially returned a cached value #IF YOU CALL THIS METHOD, IT WILL CALL RPM #Use get(:property) to check if cached values are available - cmd = ["-q", @resource[:name], "#{self.class.nosignature}", "#{self.class.nodigest}", "--qf", "#{NEVRAFORMAT}\n"] + cmd = ["-q", @resource[:name], "#{self.class.nosignature}", "#{self.class.nodigest}", "--qf", self.class::NEVRA_FORMAT] begin output = rpm(*cmd) @@ -74,7 +78,7 @@ Puppet::Type.type(:package).provide :rpm, :source => :rpm, :parent => Puppet::Pr end # FIXME: We could actually be getting back multiple packages - # for multilib + # for multilib and this will only return the first such package @property_hash.update(self.class.nevra_to_hash(output)) @property_hash.dup @@ -86,7 +90,7 @@ Puppet::Type.type(:package).provide :rpm, :source => :rpm, :parent => Puppet::Pr @resource.fail "RPMs must specify a package source" end - cmd = [command(:rpm), "-q", "--qf", "#{NEVRAFORMAT}\n", "-p", "#{@resource[:source]}"] + cmd = [command(:rpm), "-q", "--qf", self.class::NEVRA_FORMAT, "-p", source] h = self.class.nevra_to_hash(execfail(cmd, Puppet::Error)) h[:ensure] end @@ -135,12 +139,25 @@ Puppet::Type.type(:package).provide :rpm, :source => :rpm, :parent => Puppet::Pr self.install end + private + + # @param line [String] one line of rpm package query information + # @return [Hash] of NEVRA_FIELDS strings parsed from package info + # if we failed to parse + # @note warns if failed to match a line, and returns an empty Hash. + # @api private def self.nevra_to_hash(line) - line.chomp! + line.strip! hash = {} - NEVRA_FIELDS.zip(line.split) { |f, v| hash[f] = v } - hash[:provider] = self.name - hash[:ensure] = "#{hash[:version]}-#{hash[:release]}" - hash + + if match = self::NEVRA_REGEX.match(line) + self::NEVRA_FIELDS.zip(match.captures) { |f, v| hash[f] = v } + hash[:provider] = self.name + hash[:ensure] = "#{hash[:version]}-#{hash[:release]}" + else + Puppet.warning "Failed to match rpm line #{line}" + end + + return hash end end diff --git a/lib/puppet/provider/package/rug.rb b/lib/puppet/provider/package/rug.rb index 28729952d..f004b945e 100644 --- a/lib/puppet/provider/package/rug.rb +++ b/lib/puppet/provider/package/rug.rb @@ -22,7 +22,7 @@ Puppet::Type.type(:package).provide :rug, :parent => :rpm do # Add the package version wanted += "-#{should}" end - output = rug "--quiet", :install, "-y", wanted + rug "--quiet", :install, "-y", wanted unless self.query raise Puppet::ExecutionFailure.new( diff --git a/lib/puppet/provider/package/urpmi.rb b/lib/puppet/provider/package/urpmi.rb index 35dfc7a41..4954d17ba 100644 --- a/lib/puppet/provider/package/urpmi.rb +++ b/lib/puppet/provider/package/urpmi.rb @@ -1,16 +1,6 @@ Puppet::Type.type(:package).provide :urpmi, :parent => :rpm, :source => :rpm do desc "Support via `urpmi`." - commands :urpmi => "urpmi", :urpmq => "urpmq", :rpm => "rpm" - - if command('rpm') - confine :true => begin - rpm('-ql', 'rpm') - rescue Puppet::ExecutionFailure - false - else - true - end - end + commands :urpmi => "urpmi", :urpmq => "urpmq", :rpm => "rpm", :urpme => "urpme" defaultfor :operatingsystem => [:mandriva, :mandrake] @@ -30,12 +20,10 @@ Puppet::Type.type(:package).provide :urpmi, :parent => :rpm, :source => :rpm do wanted += "-#{should}" end - output = urpmi "--auto", wanted + urpmi "--auto", wanted unless self.query - raise Puppet::Error.new( - "Could not find package #{self.name}" - ) + raise Puppet::Error, "Package #{self.name} was not present after trying to install it" end end @@ -56,4 +44,12 @@ Puppet::Type.type(:package).provide :urpmi, :parent => :rpm, :source => :rpm do # Install in urpmi can be used for update, too self.install end + + # For normal package removal the urpmi provider will delegate to the RPM + # provider. If the package to remove has dependencies then uninstalling via + # rpm will fail, but `urpme` can be used to remove a package and its + # dependencies. + def purge + urpme '--auto', @resource[:name] + end end diff --git a/lib/puppet/provider/package/windows/exe_package.rb b/lib/puppet/provider/package/windows/exe_package.rb index 9b16c48ad..5525ca0c6 100644 --- a/lib/puppet/provider/package/windows/exe_package.rb +++ b/lib/puppet/provider/package/windows/exe_package.rb @@ -41,7 +41,7 @@ class Puppet::Provider::Package::Windows end def self.install_command(resource) - ['cmd.exe', '/c', 'start', '/w', quote(resource[:source])] + ['cmd.exe', '/c', 'start', '"puppet-install"', '/w', quote(resource[:source])] end def uninstall_command diff --git a/lib/puppet/provider/package/windows/package.rb b/lib/puppet/provider/package/windows/package.rb index 04f0b30b8..fd007de6c 100644 --- a/lib/puppet/provider/package/windows/package.rb +++ b/lib/puppet/provider/package/windows/package.rb @@ -1,29 +1,4 @@ -# 'puppet/type/package' is being required here to avoid a load order issue that -# manifests as 'uninitialized constant Puppet::Util::Windows::MsiPackage' or -# 'uninitialized constant Puppet::Util::Windows::Package' (or similar case -# where Puppet::Provider::Package::Windows somehow ends up pointing to -# Puppet:Util::Windows) if puppet/provider/package/windows/package is loaded -# before the puppet/type/package. -# -# Example: -# -# jpartlow@percival:~/work/puppet$ bundle exec rspec spec/unit/provider/package/windows/package_spec.rb spec/unit/provider/package/rpm_spec.rb -# Run options: exclude {:broken=>true} -# ..F..FFF........................ -# -# Failures: -# -# 1) Puppet::Util::Package::Windows::Package::each should yield each package it finds -# Failure/Error: Puppet::Provider::Package::Windows::MsiPackage.expects(:from_registry).with('Google', {}).returns(package) -# NameError: -# uninitialized constant Puppet::Util::Windows::MsiPackage -# # ./spec/unit/provider/package/windows/package_spec.rb:24:in `block (3 levels) in <top (required)>' -# -# --- -# -# Needs more investigation to pinpoint what's going on. -# -require 'puppet/type/package' +require 'puppet/provider/package' require 'puppet/util/windows' class Puppet::Provider::Package::Windows diff --git a/lib/puppet/provider/package/yum.rb b/lib/puppet/provider/package/yum.rb index f2f2c1013..dd41aa470 100644 --- a/lib/puppet/provider/package/yum.rb +++ b/lib/puppet/provider/package/yum.rb @@ -74,7 +74,7 @@ Puppet::Type.type(:package).provide :yum, :parent => :rpm, :source => :rpm do end end - output = yum "-d", "0", "-e", "0", "-y", operation, wanted + yum "-d", "0", "-e", "0", "-y", operation, wanted is = self.query raise Puppet::Error, "Could not find package #{self.name}" unless is diff --git a/lib/puppet/provider/package/zypper.rb b/lib/puppet/provider/package/zypper.rb index 8fb50d2f5..56afd8318 100644 --- a/lib/puppet/provider/package/zypper.rb +++ b/lib/puppet/provider/package/zypper.rb @@ -1,7 +1,7 @@ Puppet::Type.type(:package).provide :zypper, :parent => :rpm do desc "Support for SuSE `zypper` package manager. Found in SLES10sp2+ and SLES11" - has_feature :versionable + has_feature :versionable, :install_options commands :zypper => "/usr/bin/zypper" @@ -51,9 +51,9 @@ Puppet::Type.type(:package).provide :zypper, :parent => :rpm do #zypper 0.6.13 (OpenSuSE 10.2) does not support auto agree with licenses if major < 1 and minor <= 6 and patch <= 13 - zypper quiet, :install, noconfirm, wanted + zypper quiet, :install, noconfirm, install_options, wanted else - zypper quiet, :install, license, noconfirm, wanted + zypper quiet, :install, license, noconfirm, install_options, wanted end unless self.query @@ -81,4 +81,23 @@ Puppet::Type.type(:package).provide :zypper, :parent => :rpm do # zypper install can be used for update, too self.install end + + def install_options + join_options(resource[:install_options]) + end + + def join_options(options) + return unless options + + options.collect do |val| + case val + when Hash + val.keys.sort.collect do |k| + "#{k} '#{val[k]}'" + end.join(' ') + else + val + end + end + end end diff --git a/lib/puppet/provider/parsedfile.rb b/lib/puppet/provider/parsedfile.rb index b9e8ccfce..e922d9381 100755 --- a/lib/puppet/provider/parsedfile.rb +++ b/lib/puppet/provider/parsedfile.rb @@ -160,15 +160,6 @@ class Puppet::Provider::ParsedFile < Puppet::Provider [resource_type.validproperties, resource_type.parameters].flatten.each do |attr| attr = attr.intern define_method(attr) do -# if @property_hash.empty? -# # Note that this swaps the provider out from under us. -# prefetch -# if @resource.provider == self -# return @property_hash[attr] -# else -# return @resource.provider.send(attr) -# end -# end # If it's not a valid field for this record type (which can happen # when different platforms support different fields), then just # return the should value, so the resource shuts up. @@ -416,8 +407,6 @@ class Puppet::Provider::ParsedFile < Puppet::Provider end self.class.flush(@property_hash) - - #@property_hash = {} end def initialize(record) @@ -425,7 +414,7 @@ class Puppet::Provider::ParsedFile < Puppet::Provider # The 'record' could be a resource or a record, depending on how the provider # is initialized. If we got an empty property hash (probably because the resource - # is just being initialized), then we want to set up some defualts. + # is just being initialized), then we want to set up some defaults. @property_hash = self.class.record?(resource[:name]) || {:record_type => self.class.name, :ensure => :absent} if @property_hash.empty? end diff --git a/lib/puppet/provider/scheduled_task/win32_taskscheduler.rb b/lib/puppet/provider/scheduled_task/win32_taskscheduler.rb index 140ab1cd9..5a8741122 100644 --- a/lib/puppet/provider/scheduled_task/win32_taskscheduler.rb +++ b/lib/puppet/provider/scheduled_task/win32_taskscheduler.rb @@ -74,7 +74,7 @@ Puppet::Type.type(:scheduled_task).provide(:win32_taskscheduler) do task.trigger_count.times do |i| trigger = begin task.trigger(i) - rescue Win32::TaskScheduler::Error => e + rescue Win32::TaskScheduler::Error # Win32::TaskScheduler can't handle all of the # trigger types Windows uses, so we need to skip the # unhandled types to prevent "puppet resource" from diff --git a/lib/puppet/provider/service/base.rb b/lib/puppet/provider/service/base.rb index ad6e956cd..0d960854f 100755 --- a/lib/puppet/provider/service/base.rb +++ b/lib/puppet/provider/service/base.rb @@ -92,7 +92,7 @@ Puppet::Type.type(:service).provide :base, :parent => :service do end begin output = kill pid - rescue Puppet::ExecutionFailure => detail + rescue Puppet::ExecutionFailure @resource.fail "Could not kill #{self.name}, PID #{pid}: #{output}" end return true diff --git a/lib/puppet/provider/service/daemontools.rb b/lib/puppet/provider/service/daemontools.rb index 6f751f43c..f7784cc61 100644 --- a/lib/puppet/provider/service/daemontools.rb +++ b/lib/puppet/provider/service/daemontools.rb @@ -129,7 +129,7 @@ Puppet::Type.type(:service).provide :daemontools, :parent => :base do Puppet.notice "Configuring #{resource[:name]}" command = [ resource[:manifest], resource[:name] ] #texecute("setupservice", command) - rv = system("#{command}") + system("#{command}") end rescue Puppet::ExecutionFailure => detail raise Puppet::Error.new( "Cannot config #{self.service} to enable it: #{detail}" ) @@ -157,7 +157,7 @@ Puppet::Type.type(:service).provide :daemontools, :parent => :base do File.symlink(self.daemon, self.service) end end - rescue Puppet::ExecutionFailure => detail + rescue Puppet::ExecutionFailure raise Puppet::Error.new( "No daemon directory found for #{self.service}") end @@ -173,7 +173,7 @@ Puppet::Type.type(:service).provide :daemontools, :parent => :base do File.unlink(self.service) end end - rescue Puppet::ExecutionFailure => detail + rescue Puppet::ExecutionFailure raise Puppet::Error.new( "No daemon directory found for #{self.service}") end self.stop diff --git a/lib/puppet/provider/service/debian.rb b/lib/puppet/provider/service/debian.rb index a0707725b..7d14eaa3c 100755 --- a/lib/puppet/provider/service/debian.rb +++ b/lib/puppet/provider/service/debian.rb @@ -54,7 +54,7 @@ Puppet::Type.type(:service).provide :debian, :parent => :init do end def get_start_link_count - Dir.glob("/etc/rc*.d/S*#{@resource[:name]}").length + Dir.glob("/etc/rc*.d/S??#{@resource[:name]}").length end def enable diff --git a/lib/puppet/provider/service/init.rb b/lib/puppet/provider/service/init.rb index 19bc31568..23ad824b2 100755 --- a/lib/puppet/provider/service/init.rb +++ b/lib/puppet/provider/service/init.rb @@ -43,6 +43,10 @@ Puppet::Type.type(:service).provide :init, :parent => :base do excludes += %w{wait-for-state portmap-wait} # these excludes were found with grep -r -L start /etc/init.d excludes += %w{rcS module-init-tools} + # Prevent puppet failing to get status of the new service introduced + # by the fix for this (bug https://bugs.launchpad.net/ubuntu/+source/lightdm/+bug/982889) + # due to puppet's inability to deal with upstart services with instances. + excludes += %w{plymouth-ready} end # List all services of this type. @@ -106,33 +110,23 @@ Puppet::Type.type(:service).provide :init, :parent => :base do end def search(name) - paths.each { |path| + paths.each do |path| fqname = File.join(path,name) - begin - stat = File.stat(fqname) - rescue - # should probably rescue specific errors... + if File.exist? fqname + return fqname + else self.debug("Could not find #{name} in #{path}") - next end + end - # if we've gotten this far, we found a valid script - return fqname - } - - paths.each { |path| + paths.each do |path| fqname_sh = File.join(path,"#{name}.sh") - begin - stat = File.stat(fqname_sh) - rescue - # should probably rescue specific errors... + if File.exist? fqname_sh + return fqname_sh + else self.debug("Could not find #{name}.sh in #{path}") - next end - - # if we've gotten this far, we found a valid script - return fqname_sh - } + end raise Puppet::Error, "Could not find init script for '#{name}'" end diff --git a/lib/puppet/provider/service/openrc.rb b/lib/puppet/provider/service/openrc.rb index a59f1d304..8243f651e 100644 --- a/lib/puppet/provider/service/openrc.rb +++ b/lib/puppet/provider/service/openrc.rb @@ -10,8 +10,10 @@ Puppet::Type.type(:service).provide :openrc, :parent => :base do defaultfor :operatingsystem => :gentoo defaultfor :operatingsystem => :funtoo + has_command(:rcstatus, '/bin/rc-status') do + environment :RC_SVCNAME => nil + end commands :rcservice => '/sbin/rc-service' - commands :rcstatus => '/bin/rc-status' commands :rcupdate => '/sbin/rc-update' self::STATUSLINE = /^\s+(.*?)\s*\[\s*(.*)\s*\]$/ diff --git a/lib/puppet/provider/service/redhat.rb b/lib/puppet/provider/service/redhat.rb index 0dd5f41a8..c1c6401c1 100755 --- a/lib/puppet/provider/service/redhat.rb +++ b/lib/puppet/provider/service/redhat.rb @@ -21,25 +21,22 @@ Puppet::Type.type(:service).provide :redhat, :parent => :init, :source => :init end def enabled? + # Checkconfig always returns 0 on SuSE unless the --check flag is used. + args = (Facter.value(:osfamily) == 'Suse' ? ['--check'] : []) + begin - output = chkconfig(@resource[:name]) + chkconfig(@resource[:name], *args) rescue Puppet::ExecutionFailure return :false end - # If it's disabled on SuSE, then it will print output showing "off" - # at the end - if output =~ /.* off$/ - return :false - end - :true end # Don't support them specifying runlevels; always use the runlevels # in the init scripts. def enable - output = chkconfig(@resource[:name], :on) + chkconfig(@resource[:name], :on) rescue Puppet::ExecutionFailure => detail raise Puppet::Error, "Could not enable #{self.name}: #{detail}" end diff --git a/lib/puppet/provider/service/runit.rb b/lib/puppet/provider/service/runit.rb index 08db7ed5e..d326b1ebf 100644 --- a/lib/puppet/provider/service/runit.rb +++ b/lib/puppet/provider/service/runit.rb @@ -9,12 +9,13 @@ Puppet::Type.type(:service).provide :runit, :parent => :daemontools do When detecting the service directory it will check, in order of preference: * `/service` - * `/var/service` * `/etc/service` + * `/var/service` The daemon directory should be in one of the following locations: * `/etc/sv` + * `/var/lib/service` or this can be overriden in the service resource parameters:: @@ -46,7 +47,7 @@ Puppet::Type.type(:service).provide :runit, :parent => :daemontools do break end end - raise "Could not find the daemon directory (tested [/var/lib/service,/etc])" unless @defpath + raise "Could not find the daemon directory (tested [/etc/sv,/var/lib/service])" unless @defpath end @defpath end diff --git a/lib/puppet/provider/service/systemd.rb b/lib/puppet/provider/service/systemd.rb index 3996cfe70..1ff070b86 100755 --- a/lib/puppet/provider/service/systemd.rb +++ b/lib/puppet/provider/service/systemd.rb @@ -37,7 +37,7 @@ Puppet::Type.type(:service).provide :systemd, :parent => :base do def status begin - output = systemctl("is-active", @resource[:name]) + systemctl("is-active", @resource[:name]) rescue Puppet::ExecutionFailure return :stopped end diff --git a/lib/puppet/provider/ssh_authorized_key/parsed.rb b/lib/puppet/provider/ssh_authorized_key/parsed.rb index 6654f34b6..44fef458e 100644 --- a/lib/puppet/provider/ssh_authorized_key/parsed.rb +++ b/lib/puppet/provider/ssh_authorized_key/parsed.rb @@ -47,7 +47,7 @@ Puppet::Type.type(:ssh_authorized_key).provide( def flush raise Puppet::Error, "Cannot write SSH authorized keys without user" unless @resource.should(:user) - raise Puppet::Error, "User '#{@resource.should(:user)}' does not exist" unless uid = Puppet::Util.uid(@resource.should(:user)) + raise Puppet::Error, "User '#{@resource.should(:user)}' does not exist" unless Puppet::Util.uid(@resource.should(:user)) # ParsedFile usually calls backup_target much later in the flush process, # but our SUID makes that fail to open filebucket files for writing. # Fortunately, there's already logic to make sure it only ever happens once, diff --git a/lib/puppet/provider/sshkey/parsed.rb b/lib/puppet/provider/sshkey/parsed.rb index e105bbb4c..f874683b7 100755 --- a/lib/puppet/provider/sshkey/parsed.rb +++ b/lib/puppet/provider/sshkey/parsed.rb @@ -27,8 +27,6 @@ Puppet::Type.type(:sshkey).provide( }, :pre_gen => proc { |hash| if hash[:host_aliases] - names = [hash[:name], hash[:host_aliases]].flatten - hash[:name] = [hash[:name], hash[:host_aliases]].flatten.join(",") hash.delete(:host_aliases) end diff --git a/lib/puppet/provider/user/aix.rb b/lib/puppet/provider/user/aix.rb index d46b24778..0831f2e26 100755 --- a/lib/puppet/provider/user/aix.rb +++ b/lib/puppet/provider/user/aix.rb @@ -38,7 +38,7 @@ Puppet::Type.type(:user).provide :aix, :parent => Puppet::Provider::AixObject do has_features :manages_expiry, :manages_password_age # Attribute verification (TODO) - #verify :gid, "GID must be an string or int of a valid group" do |value| + #verify :gid, "GID must be a string or int of a valid group" do |value| # value.is_a? String || value.is_a? Integer #end # @@ -48,7 +48,7 @@ Puppet::Type.type(:user).provide :aix, :parent => Puppet::Provider::AixObject do # User attributes to ignore from AIX output. def self.attribute_ignore - [] + ["name"] end # AIX attributes to properties mapping. @@ -60,19 +60,20 @@ Puppet::Type.type(:user).provide :aix, :parent => Puppet::Provider::AixObject do # :to Method to adapt puppet property to aix command value. Optional. # :from Method to adapt aix command value to puppet property. Optional self.attribute_mapping = [ - #:name => :name, {:aix_attr => :pgrp, :puppet_prop => :gid, - :to => :gid_to_attr, :from => :gid_from_attr}, + :to => :gid_to_attr, + :from => :gid_from_attr }, {:aix_attr => :id, :puppet_prop => :uid}, {:aix_attr => :groups, :puppet_prop => :groups}, {:aix_attr => :home, :puppet_prop => :home}, {:aix_attr => :shell, :puppet_prop => :shell}, {:aix_attr => :expires, :puppet_prop => :expiry, - :to => :expiry_to_attr, :from => :expiry_from_attr}, + :to => :expiry_to_attr, + :from => :expiry_from_attr }, {:aix_attr => :maxage, :puppet_prop => :password_max_age}, {:aix_attr => :minage, :puppet_prop => :password_min_age}, {:aix_attr => :attributes, :puppet_prop => :attributes}, - {:aix_attr => :gecos, :puppet_prop => :comment}, + { :aix_attr => :gecos, :puppet_prop => :comment }, ] #-------------- @@ -140,7 +141,7 @@ Puppet::Type.type(:user).provide :aix, :parent => Puppet::Provider::AixObject do if key == :attributes raise Puppet::Error, "Attributes must be a list of pairs key=value on #{@resource.class.name}[#{@resource.name}]" \ unless value and value.is_a? Hash - return value.select { |k,v| true }.map { |pair| pair.join("=") } + return value.map { |k,v| k.to_s.strip + "=" + v.to_s.strip} end super(key, value, mapping, objectinfo) @@ -213,6 +214,11 @@ Puppet::Type.type(:user).provide :aix, :parent => Puppet::Provider::AixObject do expiry_date end + def open_security_passwd + # helper method for tests + File.open("/etc/security/passwd", 'r') + end + #-------------------------------- # Getter and Setter # When the provider is initialized, create getter/setter methods for each @@ -228,15 +234,15 @@ Puppet::Type.type(:user).provide :aix, :parent => Puppet::Provider::AixObject do def password password = :absent user = @resource[:name] - f = File.open("/etc/security/passwd", 'r') + f = open_security_passwd # Skip to the user f.each_line { |l| break if l =~ /^#{user}:\s*$/ } if ! f.eof? f.each_line { |l| # If there is a new user stanza, stop break if l =~ /^\S*:\s*$/ - # If the password= entry is found, return it - if l =~ /^\s*password\s*=\s*(.*)$/ + # If the password= entry is found, return it, stripping trailing space + if l =~ /^\s*password\s*=\s*(\S*)\s*$/ password = $1; break; end } @@ -271,18 +277,30 @@ Puppet::Type.type(:user).provide :aix, :parent => Puppet::Provider::AixObject do end end + def managed_attribute_keys(hash) + managed_attributes ||= @resource.original_parameters[:attributes] || hash.keys.map{|k| k.to_s} + managed_attributes.map {|attr| key, value = attr.split("="); key.strip.to_sym} + end + + def should_include?(key, managed_keys) + !self.class.attribute_mapping_from.include?(key) and + !self.class.attribute_ignore.include?(key) and + managed_keys.include?(key) + end + def filter_attributes(hash) - # Return only not managed attributtes. - hash.select { - |k,v| !self.class.attribute_mapping_from.include?(k) and - !self.class.attribute_ignore.include?(k) + # Return only managed attributtes. + managed_keys = managed_attribute_keys(hash) + results = hash.select { + |k,v| should_include?(k, managed_keys) }.inject({}) { |hash, array| hash[array[0]] = array[1]; hash } + results end def attributes - filter_attributes(getosinfo(refresh = false)) + filter_attributes(getosinfo(false)) end def attributes=(attr_hash) diff --git a/lib/puppet/provider/user/directoryservice.rb b/lib/puppet/provider/user/directoryservice.rb index ea3719d29..c9110808d 100644 --- a/lib/puppet/provider/user/directoryservice.rb +++ b/lib/puppet/provider/user/directoryservice.rb @@ -188,7 +188,7 @@ Puppet::Type.type(:user).provide :directoryservice do def self.convert_xml_to_binary(plist_data) Puppet.debug('Converting XML plist to binary') Puppet.debug('Executing: \'plutil -convert binary1 -o - -\'') - IO.popen('plutil -convert binary1 -o - -', mode='r+') do |io| + IO.popen('plutil -convert binary1 -o - -', 'r+') do |io| io.write Plist::Emit.dump(plist_data) io.close_write @converted_plist = io.read @@ -201,7 +201,7 @@ Puppet::Type.type(:user).provide :directoryservice do def self.convert_binary_to_xml(plist_data) Puppet.debug('Converting binary plist to XML') Puppet.debug('Executing: \'plutil -convert xml1 -o - -\'') - IO.popen('plutil -convert xml1 -o - -', mode='r+') do |io| + IO.popen('plutil -convert xml1 -o - -', 'r+') do |io| io.write plist_data io.close_write @converted_plist = io.read @@ -297,11 +297,8 @@ Puppet::Type.type(:user).provide :directoryservice do end end - # If a non-numerical gid value is passed, assume it is a group name and - # lookup that group's GID value to use when setting the GID - if (attribute == :gid) and value.class == 'Fixnum' - value = self.class.get_attribute_from_dscl('Groups', value, 'PrimaryGroupID')['dsAttrTypeStandard:PrimaryGroupID'][0] - end + # Ensure group names are converted to integers. + value = Puppet::Util.gid(value) if attribute == :gid ## Set values ## # For the :password and :groups properties, call the setter methods diff --git a/lib/puppet/provider/user/ldap.rb b/lib/puppet/provider/user/ldap.rb index 2601fdb20..4de89666c 100644 --- a/lib/puppet/provider/user/ldap.rb +++ b/lib/puppet/provider/user/ldap.rb @@ -27,7 +27,6 @@ Puppet::Type.type(:user).provide :ldap, :parent => Puppet::Provider::Ldap do # Use the last field of a space-separated array as # the sn. LDAP requires a surname, for some stupid reason. manager.generates(:sn).from(:cn).with do |cn| - x = 1 cn[0].split(/\s+/)[-1] end diff --git a/lib/puppet/provider/user/user_role_add.rb b/lib/puppet/provider/user/user_role_add.rb index 72cdea4c2..3ec49507d 100644 --- a/lib/puppet/provider/user/user_role_add.rb +++ b/lib/puppet/provider/user/user_role_add.rb @@ -1,5 +1,6 @@ require 'puppet/util' require 'puppet/util/user_attr' +require 'date' Puppet::Type.type(:user).provide :user_role_add, :parent => :useradd, :source => :useradd do @@ -195,6 +196,7 @@ Puppet::Type.type(:user).provide :user_role_add, :parent => :useradd, :source => line_arr = line.split(':') if line_arr[0] == @resource[:name] line_arr[1] = cryptopw + line_arr[2] = (Date.today - Date.new(1970,1,1)).to_i.to_s line = line_arr.join(':') end fh.print line diff --git a/lib/puppet/provider/user/useradd.rb b/lib/puppet/provider/user/useradd.rb index 400788e13..741d0c1d7 100644 --- a/lib/puppet/provider/user/useradd.rb +++ b/lib/puppet/provider/user/useradd.rb @@ -71,7 +71,7 @@ Puppet::Type.type(:user).provide :useradd, :parent => Puppet::Provider::NameServ end def local_username - user = finduser('uid', @resource.uid) + finduser('uid', @resource.uid) end def localuid diff --git a/lib/puppet/provider/zone/solaris.rb b/lib/puppet/provider/zone/solaris.rb index 6e9f813f6..6e0d14c31 100644 --- a/lib/puppet/provider/zone/solaris.rb +++ b/lib/puppet/provider/zone/solaris.rb @@ -23,8 +23,7 @@ Puppet::Type.type(:zone).provide(:solaris) do end def self.instances - # LAK:NOTE See http://snurl.com/21zf8 [groups_google_com] - x = adm(:list, "-cp").split("\n").collect do |line| + adm(:list, "-cp").split("\n").collect do |line| new(line2hash(line)) end end diff --git a/lib/puppet/reference/metaparameter.rb b/lib/puppet/reference/metaparameter.rb index f65dc2ba9..7c701a110 100644 --- a/lib/puppet/reference/metaparameter.rb +++ b/lib/puppet/reference/metaparameter.rb @@ -1,4 +1,4 @@ -metaparameter = Puppet::Util::Reference.newreference :metaparameter, :doc => "All Puppet metaparameters and all their details" do +Puppet::Util::Reference.newreference :metaparameter, :doc => "All Puppet metaparameters and all their details" do types = {} Puppet::Type.loadall diff --git a/lib/puppet/reference/type.rb b/lib/puppet/reference/type.rb index 87aed435c..56487ab23 100644 --- a/lib/puppet/reference/type.rb +++ b/lib/puppet/reference/type.rb @@ -1,4 +1,4 @@ -type = Puppet::Util::Reference.newreference :type, :doc => "All Puppet resource types and all their details" do +Puppet::Util::Reference.newreference :type, :doc => "All Puppet resource types and all their details" do types = {} Puppet::Type.loadall diff --git a/lib/puppet/reports/rrdgraph.rb b/lib/puppet/reports/rrdgraph.rb index 3ac9edb57..f283f7c27 100644 --- a/lib/puppet/reports/rrdgraph.rb +++ b/lib/puppet/reports/rrdgraph.rb @@ -3,7 +3,7 @@ Puppet::Reports.register_report(:rrdgraph) do must have the Ruby RRDtool library installed to use this report, which you can get from [the RubyRRDTool RubyForge page](http://rubyforge.org/projects/rubyrrdtool/). - This package may also be available as `ruby-rrd` or `rrdtool-ruby` in your + This package may also be available as `librrd-ruby`, `ruby-rrd`, or `rrdtool-ruby` in your distribution's package management system. The library and/or package will both require the binary `rrdtool` package from your distribution to be installed. diff --git a/lib/puppet/reports/tagmail.rb b/lib/puppet/reports/tagmail.rb index f3485432c..528b6fa38 100644 --- a/lib/puppet/reports/tagmail.rb +++ b/lib/puppet/reports/tagmail.rb @@ -133,7 +133,7 @@ Puppet::Reports.register_report(:tagmail) do pid = Puppet::Util.safe_posix_fork do if Puppet[:smtpserver] != "none" begin - Net::SMTP.start(Puppet[:smtpserver]) do |smtp| + Net::SMTP.start(Puppet[:smtpserver], Puppet[:smtpport], Puppet[:smtphelo]) do |smtp| reports.each do |emails, messages| smtp.open_message_stream(Puppet[:reportfrom], *emails) do |p| p.puts "From: #{Puppet[:reportfrom]}" diff --git a/lib/puppet/resource.rb b/lib/puppet/resource.rb index b783b8be6..eae59f389 100644 --- a/lib/puppet/resource.rb +++ b/lib/puppet/resource.rb @@ -5,6 +5,8 @@ require 'puppet/parameter' # The simplest resource class. Eventually it will function as the # base class for all resource-like behaviour. +# +# @api public class Puppet::Resource # This stub class is only needed for serialization compatibility with 0.25.x. # Specifically, it exists to provide a compatibility API when using YAML @@ -316,15 +318,25 @@ class Puppet::Resource # We make a request to the backend for the key 'foo::port' not 'foo' # def lookup_external_default_for(param, scope) - if resource_type.type == :hostclass + # Only lookup parameters for host classes + return nil unless resource_type.type == :hostclass + + name = "#{resource_type.name}::#{param}" + # Lookup with injector (optionally), and if no value bound, lookup with "classic hiera" + result = nil + if scope.compiler.is_binder_active? + result = scope.compiler.injector.lookup(scope, name) + end + if result.nil? Puppet::DataBinding.indirection.find( - "#{resource_type.name}::#{param}", + name, :environment => scope.environment.to_s, :variables => Puppet::DataBinding::Variables.new(scope)) else - nil + result end end + private :lookup_external_default_for def set_default_parameters(scope) @@ -396,7 +408,7 @@ class Puppet::Resource # Produce a canonical method name. def parameter_name(param) param = param.to_s.downcase.to_sym - if param == :name and n = namevar + if param == :name and namevar param = namevar end param diff --git a/lib/puppet/resource/catalog.rb b/lib/puppet/resource/catalog.rb index dbc0f176b..3120d3ff7 100644 --- a/lib/puppet/resource/catalog.rb +++ b/lib/puppet/resource/catalog.rb @@ -1,17 +1,17 @@ require 'puppet/node' require 'puppet/indirector' -require 'puppet/simple_graph' require 'puppet/transaction' - require 'puppet/util/pson' - require 'puppet/util/tagging' +require 'puppet/graph' -# This class models a node catalog. It is the thing -# meant to be passed from server to client, and it contains all -# of the information in the catalog, including the resources -# and the relationships between them. -class Puppet::Resource::Catalog < Puppet::SimpleGraph +# This class models a node catalog. It is the thing meant to be passed +# from server to client, and it contains all of the information in the +# catalog, including the resources and the relationships between them. +# +# @api public + +class Puppet::Resource::Catalog < Puppet::Graph::SimpleGraph class DuplicateResourceError < Puppet::Error include Puppet::ExternalFileError end @@ -64,33 +64,49 @@ class Puppet::Resource::Catalog < Puppet::SimpleGraph [$1, $2] end - # Add a resource to our graph and to our resource table. - # This is actually a relatively complicated method, because it handles multiple - # aspects of Catalog behaviour: - # * Add the resource to the resource table - # * Add the resource to the resource graph - # * Add the resource to the relationship graph - # * Add any aliases that make sense for the resource (e.g., name != title) - def add_resource(*resource) - add_resource(*resource[0..-2]) if resource.length > 1 - resource = resource.pop - raise ArgumentError, "Can only add objects that respond to :ref, not instances of #{resource.class}" unless resource.respond_to?(:ref) - fail_on_duplicate_type_and_title(resource) - title_key = title_key_for_ref(resource.ref) + def add_resource(*resources) + resources.each do |resource| + add_one_resource(resource) + end + end - @transient_resources << resource if applying? - @resource_table[title_key] = resource + # @param resource [A Resource] a resource in the catalog + # @return [A Resource, nil] the resource that contains the given resource + # @api public + def container_of(resource) + adjacent(resource, :direction => :in)[0] + end - # If the name and title differ, set up an alias + def add_one_resource(resource) + fail_on_duplicate_type_and_title(resource) - if resource.respond_to?(:name) and resource.respond_to?(:title) and resource.respond_to?(:isomorphic?) and resource.name != resource.title - self.alias(resource, resource.uniqueness_key) if resource.isomorphic? - end + add_resource_to_table(resource) + create_resource_aliases(resource) resource.catalog = self if resource.respond_to?(:catalog=) + add_resource_to_graph(resource) + end + private :add_one_resource + + def add_resource_to_table(resource) + title_key = title_key_for_ref(resource.ref) + @resource_table[title_key] = resource + @resources << title_key + end + private :add_resource_to_table + + def add_resource_to_graph(resource) add_vertex(resource) @relationship_graph.add_vertex(resource) if @relationship_graph end + private :add_resource_to_graph + + def create_resource_aliases(resource) + if resource.respond_to?(:isomorphic?) and resource.isomorphic? and resource.name != resource.title + self.alias(resource, resource.uniqueness_key) + end + end + private :create_resource_aliases # Create an alias for a resource. def alias(resource, key) @@ -120,32 +136,32 @@ class Puppet::Resource::Catalog < Puppet::SimpleGraph @aliases[resource.ref] << newref end - # Apply our catalog to the local host. Valid options - # are: - # :tags - set the tags that restrict what resources run - # during the transaction - # :ignoreschedules - tell the transaction to ignore schedules - # when determining the resources to run + # Apply our catalog to the local host. + # @param options [Hash{Symbol => Object}] a hash of options + # @option options [Puppet::Transaction::Report] :report + # The report object to log this transaction to. This is optional, + # and the resulting transaction will create a report if not + # supplied. + # @option options [Array[String]] :tags + # Tags used to filter the transaction. If supplied then only + # resources tagged with any of these tags will be evaluated. + # @option options [Boolean] :ignoreschedules + # Ignore schedules when evaluating resources + # @option options [Boolean] :for_network_device + # Whether this catalog is for a network device + # + # @return [Puppet::Transaction] the transaction created for this + # application + # + # @api public def apply(options = {}) - @applying = true - Puppet::Util::Storage.load if host_config? - transaction = Puppet::Transaction.new(self, options[:report]) - register_report = options[:report].nil? - - transaction.tags = options[:tags] if options[:tags] - transaction.ignoreschedules = true if options[:ignoreschedules] - transaction.for_network_device = options[:network_device] - - transaction.add_times :config_retrieval => self.retrieval_duration || 0 + transaction = create_transaction(options) begin - Puppet::Util::Log.newdestination(transaction.report) if register_report - begin + transaction.report.as_logging_destination do transaction.evaluate - ensure - Puppet::Util::Log.close(transaction.report) if register_report end rescue Puppet::Error => detail Puppet.log_exception(detail, "Could not apply complete catalog: #{detail}") @@ -159,14 +175,20 @@ class Puppet::Resource::Catalog < Puppet::SimpleGraph yield transaction if block_given? - return transaction - ensure - @applying = false + transaction end - # Are we in the middle of applying the catalog? - def applying? - @applying + # The relationship_graph form of the catalog. This contains all of the + # dependency edges that are used for determining order. + # + # @return [Puppet::Graph::RelationshipGraph] + # @api public + def relationship_graph + if @relationship_graph.nil? + @relationship_graph = Puppet::Graph::RelationshipGraph.new(prioritizer) + @relationship_graph.populate_from(self) + end + @relationship_graph end def clear(remove_resources = true) @@ -174,6 +196,7 @@ class Puppet::Resource::Catalog < Puppet::SimpleGraph # We have to do this so that the resources clean themselves up. @resource_table.values.each { |resource| resource.remove } if remove_resources @resource_table.clear + @resources = [] if @relationship_graph @relationship_graph.clear @@ -214,8 +237,7 @@ class Puppet::Resource::Catalog < Puppet::SimpleGraph @name = name if name @classes = [] @resource_table = {} - @transient_resources = [] - @applying = false + @resources = [] @relationship_graph = nil @host_config = true @@ -242,109 +264,6 @@ class Puppet::Resource::Catalog < Puppet::SimpleGraph end end - # Create a graph of all of the relationships in our catalog. - def relationship_graph - unless @relationship_graph - # It's important that we assign the graph immediately, because - # the debug messages below use the relationships in the - # relationship graph to determine the path to the resources - # spitting out the messages. If this is not set, - # then we get into an infinite loop. - @relationship_graph = Puppet::SimpleGraph.new - - # First create the dependency graph - self.vertices.each do |vertex| - @relationship_graph.add_vertex vertex - vertex.builddepends.each do |edge| - @relationship_graph.add_edge(edge) - end - end - - # Lastly, add in any autorequires - @relationship_graph.vertices.each do |vertex| - vertex.autorequire(self).each do |edge| - unless @relationship_graph.edge?(edge.source, edge.target) # don't let automatic relationships conflict with manual ones. - unless @relationship_graph.edge?(edge.target, edge.source) - vertex.debug "Autorequiring #{edge.source}" - @relationship_graph.add_edge(edge) - else - vertex.debug "Skipping automatic relationship with #{(edge.source == vertex ? edge.target : edge.source)}" - end - end - end - end - @relationship_graph.write_graph(:relationships) if host_config? - - # Then splice in the container information - splice!(@relationship_graph) - - @relationship_graph.write_graph(:expanded_relationships) if host_config? - end - @relationship_graph - end - - # Impose our container information on another graph by using it - # to replace any container vertices X with a pair of verticies - # { admissible_X and completed_X } such that that - # - # 0) completed_X depends on admissible_X - # 1) contents of X each depend on admissible_X - # 2) completed_X depends on each on the contents of X - # 3) everything which depended on X depens on completed_X - # 4) admissible_X depends on everything X depended on - # 5) the containers and their edges must be removed - # - # Note that this requires attention to the possible case of containers - # which contain or depend on other containers, but has the advantage - # that the number of new edges created scales linearly with the number - # of contained verticies regardless of how containers are related; - # alternatives such as replacing container-edges with content-edges - # scale as the product of the number of external dependences, which is - # to say geometrically in the case of nested / chained containers. - # - Default_label = { :callback => :refresh, :event => :ALL_EVENTS } - def splice!(other) - stage_class = Puppet::Type.type(:stage) - whit_class = Puppet::Type.type(:whit) - component_class = Puppet::Type.type(:component) - containers = vertices.find_all { |v| (v.is_a?(component_class) or v.is_a?(stage_class)) and vertex?(v) } - # - # These two hashes comprise the aforementioned attention to the possible - # case of containers that contain / depend on other containers; they map - # containers to their sentinels but pass other verticies through. Thus we - # can "do the right thing" for references to other verticies that may or - # may not be containers. - # - admissible = Hash.new { |h,k| k } - completed = Hash.new { |h,k| k } - containers.each { |x| - admissible[x] = whit_class.new(:name => "admissible_#{x.ref}", :catalog => self) - completed[x] = whit_class.new(:name => "completed_#{x.ref}", :catalog => self) - } - # - # Implement the six requierments listed above - # - containers.each { |x| - contents = adjacent(x, :direction => :out) - other.add_edge(admissible[x],completed[x]) if contents.empty? # (0) - contents.each { |v| - other.add_edge(admissible[x],admissible[v],Default_label) # (1) - other.add_edge(completed[v], completed[x], Default_label) # (2) - } - # (3) & (5) - other.adjacent(x,:direction => :in,:type => :edges).each { |e| - other.add_edge(completed[e.source],admissible[x],e.label) - other.remove_edge! e - } - # (4) & (5) - other.adjacent(x,:direction => :out,:type => :edges).each { |e| - other.add_edge(completed[x],admissible[e.target],e.label) - other.remove_edge! e - } - } - containers.each { |x| other.remove_vertex! x } # (5) - end - # Remove the resource from our catalog. Notice that we also call # 'remove' on the resource, at least until resource classes no longer maintain # references to the resource instances. @@ -358,20 +277,21 @@ class Puppet::Resource::Catalog < Puppet::SimpleGraph end remove_vertex!(resource) if vertex?(resource) @relationship_graph.remove_vertex!(resource) if @relationship_graph and @relationship_graph.vertex?(resource) + @resources.delete(title_key) resource.remove end end # Look a resource up by its reference (e.g., File[/etc/passwd]). def resource(type, title = nil) - # Always create a resource reference, so that it always canonizes how we - # are referring to them. + # Always create a resource reference, so that it always + # canonicalizes how we are referring to them. if title res = Puppet::Resource.new(type, title) else # If they didn't provide a title, then we expect the first # argument to be of the form 'Class[name]', which our - # Reference class canonizes for us. + # Reference class canonicalizes for us. res = Puppet::Resource.new(nil, type) end title_key = [res.type, res.title.to_s] @@ -388,7 +308,9 @@ class Puppet::Resource::Catalog < Puppet::SimpleGraph end def resources - @resource_table.values.uniq + @resources.collect do |key| + @resource_table[key] + end end def self.from_pson(data) @@ -407,10 +329,9 @@ class Puppet::Resource::Catalog < Puppet::SimpleGraph end if resources = data['resources'] - resources = PSON.parse(resources) if resources.is_a?(String) - resources.each do |res| - resource_from_pson(result, res) - end + result.add_resource(*resources.collect do |res| + Puppet::Resource.from_pson(res) + end) end if edges = data['edges'] @@ -444,11 +365,6 @@ class Puppet::Resource::Catalog < Puppet::SimpleGraph result.add_edge(edge) end - def self.resource_from_pson(result, res) - res = Puppet::Resource.from_pson(res) if res.is_a? Hash - result.add_resource(res) - end - PSON.register_document_type('Catalog',self) def to_pson_data_hash { @@ -458,7 +374,7 @@ class Puppet::Resource::Catalog < Puppet::SimpleGraph 'name' => name, 'version' => version, 'environment' => environment.to_s, - 'resources' => vertices.collect { |v| v.to_pson_data_hash }, + 'resources' => @resources.collect { |v| @resource_table[v].to_pson_data_hash }, 'edges' => edges. collect { |e| e.to_pson_data_hash }, 'classes' => classes }, @@ -525,6 +441,28 @@ class Puppet::Resource::Catalog < Puppet::SimpleGraph private + def prioritizer + @prioritizer ||= case Puppet[:ordering] + when "title-hash" + Puppet::Graph::TitleHashPrioritizer.new + when "manifest" + Puppet::Graph::SequentialPrioritizer.new + when "random" + Puppet::Graph::RandomPrioritizer.new + else + raise Puppet::DevError, "Unknown ordering type #{Puppet[:ordering]}" + end + end + + def create_transaction(options) + transaction = Puppet::Transaction.new(self, options[:report], prioritizer) + transaction.tags = options[:tags] if options[:tags] + transaction.ignoreschedules = true if options[:ignoreschedules] + transaction.for_network_device = options[:network_device] + + transaction + end + # Verify that the given resource isn't declared elsewhere. def fail_on_duplicate_type_and_title(resource) # Short-curcuit the common case, @@ -550,7 +488,7 @@ class Puppet::Resource::Catalog < Puppet::SimpleGraph result.environment = self.environment map = {} - vertices.each do |resource| + resources.each do |resource| next if virtual_not_exported?(resource) next if block_given? and yield resource diff --git a/lib/puppet/resource/status.rb b/lib/puppet/resource/status.rb index 72a5cb193..24332a599 100644 --- a/lib/puppet/resource/status.rb +++ b/lib/puppet/resource/status.rb @@ -1,3 +1,5 @@ +require 'time' + module Puppet class Resource class Status @@ -9,12 +11,14 @@ module Puppet STATES = [:skipped, :failed, :failed_to_restart, :restarted, :changed, :out_of_sync, :scheduled] attr_accessor *STATES - attr_reader :source_description, :default_log_level, :time, :resource - attr_reader :change_count, :out_of_sync_count, :resource_type, :title + attr_reader :source_description, :containment_path, + :default_log_level, :time, :resource, :change_count, + :out_of_sync_count, :resource_type, :title YAML_ATTRIBUTES = %w{@resource @file @line @evaluation_time @change_count @out_of_sync_count @tags @time @events @out_of_sync - @changed @resource_type @title @skipped @failed}. + @changed @resource_type @title @skipped @failed + @containment_path}. map(&:to_sym) @@ -54,8 +58,20 @@ module Puppet @events end + def failed_because(detail) + @real_resource.log_exception(detail, "Could not evaluate: #{detail}") + failed = true + # There's a contract (implicit unfortunately) that a status of failed + # will always be accompanied by an event with some explanatory power. This + # is useful for reporting/diagnostics/etc. So synthesize an event here + # with the exception detail as the message. + add_event(@real_resource.event(:status => "failure", :message => detail.to_s)) + end + def initialize(resource) + @real_resource = resource @source_description = resource.path + @containment_path = resource.pathbuilder @resource = resource.to_s @change_count = 0 @out_of_sync_count = 0 @@ -78,6 +94,7 @@ module Puppet @resource_type = data['resource_type'] @title = data['title'] @resource = data['resource'] + @containment_path = data['containment_path'] @file = data['file'] @line = data['line'] @evaluation_time = data['evaluation_time'] @@ -85,6 +102,7 @@ module Puppet @out_of_sync_count = data['out_of_sync_count'] @tags = data['tags'] @time = data['time'] + @time = Time.parse(@time) if @time.is_a? String @out_of_sync = data['out_of_sync'] @changed = data['changed'] @skipped = data['skipped'] @@ -95,6 +113,27 @@ module Puppet end end + def to_pson + { + 'title' => @title, + 'file' => @file, + 'line' => @line, + 'resource' => @resource, + 'resource_type' => @resource_type, + 'containment_path' => @containment_path, + 'evaluation_time' => @evaluation_time, + 'tags' => @tags, + 'time' => @time.iso8601(9), + 'failed' => @failed, + 'changed' => @changed, + 'out_of_sync' => @out_of_sync, + 'skipped' => @skipped, + 'change_count' => @change_count, + 'out_of_sync_count' => @out_of_sync_count, + 'events' => @events, + }.to_pson + end + def to_yaml_properties YAML_ATTRIBUTES & instance_variables end diff --git a/lib/puppet/resource/type.rb b/lib/puppet/resource/type.rb index 97581bcb4..4cf841475 100644 --- a/lib/puppet/resource/type.rb +++ b/lib/puppet/resource/type.rb @@ -6,6 +6,14 @@ require 'puppet/parser/ast/leaf' require 'puppet/parser/ast/block_expression' require 'puppet/dsl' +# Puppet::Resource::Type represents nodes, classes and defined types. +# +# It has a standard format for external consumption, usable from the +# resource_type indirection via rest and the resource_type face. See the +# {file:api_docs/http_resource_type.md#Schema resource type schema +# description}. +# +# @api public class Puppet::Resource::Type Puppet::ResourceType = self include Puppet::Util::InlineDocs @@ -14,13 +22,7 @@ class Puppet::Resource::Type RESOURCE_KINDS = [:hostclass, :node, :definition] - # We have reached a point where we've established some naming conventions - # in our documentation that don't entirely match up with our internal names - # for things. Ideally we'd change the internal representation to match the - # conventions expressed in our docs, but that would be a fairly far-reaching - # and risky change. For the time being, we're settling for mapping the - # internal names to the external ones (and vice-versa) during serialization - # and deserialization. These two hashes is here to help with that mapping. + # Map the names used in our documentation to the names used internally RESOURCE_KINDS_TO_EXTERNAL_NAMES = { :hostclass => "class", :node => "node", @@ -54,41 +56,26 @@ class Puppet::Resource::Type data = data.inject({}) { |result, ary| result[ary[0].intern] = ary[1]; result } - # This is a bit of a hack; when we serialize, we use the term "parameters" because that - # is the terminology that we use in our documentation. However, internally to this - # class we use the term "arguments". Ideally we'd change the implementation to be consistent - # with the documentation, but that would be challenging right now because it could potentially - # touch a lot of places in the code, not to mention that we already have another meaning for - # "parameters" internally. So, for now, we will simply transform the internal "arguments" - # value to "parameters" when serializing, and the opposite when deserializing. - # --cprice 2012-04-23 + # External documentation uses "parameters" but the internal name + # is "arguments" data[:arguments] = data.delete(:parameters) new(type, name, data) end - # This method doesn't seem like it has anything to do with PSON in particular, and it shouldn't. - # It's just transforming to a simple object that can be serialized and de-serialized via - # any transport format. Should probably be renamed if we get a chance to clean up our - # serialization / deserialization, and there are probably many other similar methods in - # other classes. - # --cprice 2012-04-23 + def to_pson(*args) + to_data_hash.to_pson(*args) + end - def to_pson_data_hash + def to_data_hash data = [:doc, :line, :file, :parent].inject({}) do |hash, param| next hash unless (value = self.send(param)) and (value != "") hash[param.to_s] = value hash end - # This is a bit of a hack; when we serialize, we use the term "parameters" because that - # is the terminology that we use in our documentation. However, internally to this - # class we use the term "arguments". Ideally we'd change the implementation to be consistent - # with the documentation, but that would be challenging right now because it could potentially - # touch a lot of places in the code, not to mention that we already have another meaning for - # "parameters" internally. So, for now, we will simply transform the internal "arguments" - # value to "parameters" when serializing, and the opposite when deserializing. - # --cprice 2012-04-23 + # External documentation uses "parameters" but the internal name + # is "arguments" data['parameters'] = arguments.dup unless arguments.empty? data['name'] = name @@ -100,19 +87,6 @@ class Puppet::Resource::Type data end - # It seems wrong that we have a 'to_pson' method on this class, but not a 'to_yaml'. - # As a result, if you use the REST API to retrieve one or more objects of this type, - # you will receive different data if you use 'Accept: yaml' vs 'Accept: pson'. That - # seems really, really wrong. The "Accept" header should never affect what data is - # being returned--only the format of the data. If the data itself is going to differ, - # then there should be a different request URL. Documenting the REST API becomes - # a much more complex problem when the "Accept" header can change the semantics - # of the response. --cprice 2012-04-23 - - def to_pson(*args) - to_pson_data_hash.to_pson(*args) - end - # Are we a child of the passed class? Do a recursive search up our # parentage tree to figure it out. def child_of?(klass) @@ -134,7 +108,19 @@ class Puppet::Resource::Type resource.add_edge_to_stage - code.safeevaluate(scope) if code + if code + if @match # Only bother setting up the ephemeral scope if there are match variables to add into it + begin + elevel = scope.ephemeral_level + scope.ephemeral_from(@match, file, line) + code.safeevaluate(scope) + ensure + scope.unset_ephemeral_var(elevel) + end + else + code.safeevaluate(scope) + end + end evaluate_ruby_code(resource, scope) if ruby_code end @@ -154,6 +140,8 @@ class Puppet::Resource::Type set_arguments(options[:arguments]) + @match = nil + @module_name = options[:module_name] end @@ -162,7 +150,7 @@ class Puppet::Resource::Type def match(string) return string.to_s.downcase == name unless name_is_regex? - @name =~ string + @match = @name.match(string) end # Add code from a new instance to our code. @@ -242,7 +230,6 @@ class Puppet::Resource::Type def assign_parameter_values(parameters, resource) return unless parameters - scope = resource.scope || {} # It'd be nice to assign default parameter values here, # but we can't because they often rely on local variables diff --git a/lib/puppet/resource/type_collection.rb b/lib/puppet/resource/type_collection.rb index 39a054700..4ca8a9689 100644 --- a/lib/puppet/resource/type_collection.rb +++ b/lib/puppet/resource/type_collection.rb @@ -1,9 +1,13 @@ require 'puppet/parser/type_loader' +require 'puppet/util/file_watcher' +require 'puppet/util/warnings' class Puppet::Resource::TypeCollection attr_reader :environment attr_accessor :parse_failed + include Puppet::Util::Warnings + def clear @hostclasses.clear @definitions.clear @@ -22,7 +26,7 @@ class Puppet::Resource::TypeCollection # So we can keep a list and match the first-defined regex @node_list = [] - @watched_files = {} + @watched_files = Puppet::Util::FileWatcher.new end def import_ast(ast, modname) @@ -130,29 +134,29 @@ class Puppet::Resource::TypeCollection end def stale? - @watched_files.values.detect { |file| file.changed? } + @watched_files.changed? end def version - return @version if defined?(@version) - - if environment[:config_version] == "" - @version = Time.now.to_i - return @version + if !defined?(@version) + if environment[:config_version] == "" + @version = Time.now.to_i + else + @version = Puppet::Util::Execution.execute([environment[:config_version]]).strip + end end - @version = Puppet::Util::Execution.execute([environment[:config_version]]).strip - + @version rescue Puppet::ExecutionFailure => e - raise Puppet::ParseError, "Unable to set config_version: #{e.message}" + raise Puppet::ParseError, "Execution of config_version command `#{environment[:config_version]}` failed: #{e.message}" end - def watch_file(file) - @watched_files[file] = Puppet::Util::LoadedFile.new(file) + def watch_file(filename) + @watched_files.watch(filename) end - def watching_file?(file) - @watched_files.include?(file) + def watching_file?(filename) + @watched_files.watching?(filename) end private @@ -202,7 +206,7 @@ class Puppet::Resource::TypeCollection if @notfound[fqname] and Puppet[:ignoremissingtypes] # do not try to autoload if we already tried and it wasn't conclusive # as this is a time consuming operation. Warn the user. - Puppet.warning "Not attempting to load #{type} #{fqname} as this object was missing during a prior compilation" + debug_once "Not attempting to load #{type} #{fqname} as this object was missing during a prior compilation" else result = loader.try_load_fqname(type, fqname) @notfound[fqname] = result.nil? diff --git a/lib/puppet/run.rb b/lib/puppet/run.rb index 1353420d0..762244119 100644 --- a/lib/puppet/run.rb +++ b/lib/puppet/run.rb @@ -95,6 +95,10 @@ class Puppet::Run end def to_pson - @options.merge(:background => @background).to_pson + { + :options => @options, + :background => @background, + :status => @status + }.to_pson end end diff --git a/lib/puppet/scheduler/scheduler.rb b/lib/puppet/scheduler/scheduler.rb index 908fbdef1..2057669c8 100644 --- a/lib/puppet/scheduler/scheduler.rb +++ b/lib/puppet/scheduler/scheduler.rb @@ -1,38 +1,37 @@ module Puppet::Scheduler class Scheduler - def initialize(jobs, timer=Puppet::Scheduler::Timer.new) + def initialize(timer=Puppet::Scheduler::Timer.new) @timer = timer - @jobs = jobs end - def run_loop - mark_start_times(@timer.now) - while not enabled_jobs.empty? - @timer.wait_for(min_interval_to_next_run_from(@timer.now)) - run_ready(@timer.now) + def run_loop(jobs) + mark_start_times(jobs, @timer.now) + while not enabled(jobs).empty? + @timer.wait_for(min_interval_to_next_run_from(jobs, @timer.now)) + run_ready(jobs, @timer.now) end end private - def enabled_jobs - @jobs.select(&:enabled?) + def enabled(jobs) + jobs.select(&:enabled?) end - def mark_start_times(start_time) - @jobs.each do |job| + def mark_start_times(jobs, start_time) + jobs.each do |job| job.start_time = start_time end end - def min_interval_to_next_run_from(from_time) - enabled_jobs.map do |j| + def min_interval_to_next_run_from(jobs, from_time) + enabled(jobs).map do |j| j.interval_to_next_from(from_time) end.min end - def run_ready(at_time) - enabled_jobs.each do |j| + def run_ready(jobs, at_time) + enabled(jobs).each do |j| # This check intentionally happens right before each run, # instead of filtering on ready schedulers, since one may adjust # the readiness of a later one diff --git a/lib/puppet/settings.rb b/lib/puppet/settings.rb index 291e26ee1..b47039b66 100644 --- a/lib/puppet/settings.rb +++ b/lib/puppet/settings.rb @@ -1,23 +1,26 @@ require 'puppet' require 'sync' require 'getoptlong' -require 'puppet/util/loadedfile' +require 'puppet/util/watched_file' require 'puppet/util/command_line/puppet_option_parser' -require 'puppet/settings/errors' -require 'puppet/settings/string_setting' -require 'puppet/settings/file_setting' -require 'puppet/settings/directory_setting' -require 'puppet/settings/path_setting' -require 'puppet/settings/boolean_setting' -require 'puppet/settings/terminus_setting' -require 'puppet/settings/duration_setting' -require 'puppet/settings/config_file' -require 'puppet/settings/value_translator' # The class for handling configuration files. class Puppet::Settings include Enumerable + require 'puppet/settings/errors' + require 'puppet/settings/base_setting' + require 'puppet/settings/string_setting' + require 'puppet/settings/enum_setting' + require 'puppet/settings/file_setting' + require 'puppet/settings/directory_setting' + require 'puppet/settings/path_setting' + require 'puppet/settings/boolean_setting' + require 'puppet/settings/terminus_setting' + require 'puppet/settings/duration_setting' + require 'puppet/settings/config_file' + require 'puppet/settings/value_translator' + # local reference for convenience PuppetOptionParser = Puppet::Util::CommandLine::PuppetOptionParser @@ -156,7 +159,34 @@ class Puppet::Settings end private :unsafe_clear - # This is mostly just used for testing. + # Clear @cache, @used and the Environment. + # + # Whenever an object is returned by Settings, a copy is stored in @cache. + # As long as Setting attributes that determine the content of returned + # objects remain unchanged, Settings can keep returning objects from @cache + # without re-fetching or re-generating them. + # + # Whenever a Settings attribute changes, such as @values or @preferred_run_mode, + # this method must be called to clear out the caches so that updated + # objects will be returned. + def flush_cache + @sync.synchronize do + unsafe_flush_cache + end + end + + def unsafe_flush_cache + clearused + + # Clear the list of environments, because they cache, at least, the module path. + # We *could* preferentially just clear them if the modulepath is changed, + # but we don't really know if, say, the vardir is changed and the modulepath + # is defined relative to it. We need the defined?(stuff) because of loading + # order issues. + Puppet::Node::Environment.clear if defined?(Puppet::Node) and defined?(Puppet::Node::Environment) + end + private :unsafe_flush_cache + def clearused @cache.clear @used = [] @@ -222,7 +252,7 @@ class Puppet::Settings # "no-" prefix on flag/boolean options). # # @param [String] opt the command line option that we are munging - # @param [String, TrueClass, FalseClass] the value for the setting (as determined by the OptionParser) + # @param [String, TrueClass, FalseClass] val the value for the setting (as determined by the OptionParser) def self.clean_opt(opt, val) # rewrite --[no-]option to --no-option if that's what was given if opt =~ /\[no-\]/ and !val @@ -455,6 +485,10 @@ class Puppet::Settings mode = mode.to_s.downcase.intern raise ValidationError, "Invalid run mode '#{mode}'" unless [:master, :agent, :user].include?(mode) @preferred_run_mode_name = mode + # Changing the run mode has far-reaching consequences. Flush any cached + # settings so they will be re-generated. + flush_cache + mode end # Return all of the parameters associated with a given section. @@ -506,7 +540,7 @@ class Puppet::Settings def config_file_name begin return self[:config_file_name] if self[:config_file_name] - rescue SettingsError => err + rescue SettingsError # This just means that the setting wasn't explicitly set on the command line, so we will ignore it and # fall through to the default name. end @@ -562,7 +596,7 @@ class Puppet::Settings # Call any hooks we should be calling. settings_with_hooks.each do |setting| each_source(env) do |source| - if value = @values[source][setting.name] + if @values[source][setting.name] # We still have to use value to retrieve the value, since # we want the fully interpolated value, not $vardir/lib or whatever. # This results in extra work, but so few of the settings @@ -593,6 +627,17 @@ class Puppet::Settings end private :apply_metadata + SETTING_TYPES = { + :string => StringSetting, + :file => FileSetting, + :directory => DirectorySetting, + :path => PathSetting, + :boolean => BooleanSetting, + :terminus => TerminusSetting, + :duration => DurationSetting, + :enum => EnumSetting, + } + # Create a new setting. The value is passed in because it's used to determine # what kind of setting we're creating, but the value itself might be either # a default or a value, so we can't actually assign it. @@ -603,15 +648,7 @@ class Puppet::Settings hash[:section] = hash[:section].to_sym if hash[:section] if type = hash[:type] - unless klass = { - :string => StringSetting, - :file => FileSetting, - :directory => DirectorySetting, - :path => PathSetting, - :boolean => BooleanSetting, - :terminus => TerminusSetting, - :duration => DurationSetting, - } [type] + unless klass = SETTING_TYPES[type] raise ArgumentError, "Invalid setting type '#{type}'" end hash.delete(:type) @@ -656,7 +693,7 @@ class Puppet::Settings @files = [] [main_config_file, user_config_file].each do |path| if FileTest.exist?(path) - @files << Puppet::Util::LoadedFile.new(path) + @files << Puppet::Util::WatchedFile.new(path) end end @files @@ -664,10 +701,11 @@ class Puppet::Settings private :files # Checks to see if any of the config files have been modified - # @return the filename of the first file that is found to have changed, or nil if no files have changed + # @return the filename of the first file that is found to have changed, or + # nil if no files have changed def any_files_changed? files.each do |file| - return file.file if file.changed? + return file.to_str if file.changed? end nil end @@ -753,16 +791,8 @@ class Puppet::Settings @sync.synchronize do # yay, thread-safe @values[type][param] = value - @cache.clear - - clearused + unsafe_flush_cache - # Clear the list of environments, because they cache, at least, the module path. - # We *could* preferentially just clear them if the modulepath is changed, - # but we don't really know if, say, the vardir is changed and the modulepath - # is defined relative to it. We need the defined?(stuff) because of loading - # order issues. - Puppet::Node::Environment.clear if defined?(Puppet::Node) and defined?(Puppet::Node::Environment) end value @@ -952,8 +982,15 @@ Generated on #{Time.now}. end private :find_value - # Find the correct value using our search path. Optionally accept an environment - # in which to search before the other configuration sections. + # Find the correct value using our search path. + # + # @param param [String, Symbol] The value to look up + # @param environment [String, Symbol] The environment to check for the value + # @param bypass_interpolation [true, false] Whether to skip interpolation + # + # @return [Object] The looked up value + # + # @raise [InterpolationError] def value(param, environment = nil, bypass_interpolation = false) param = param.to_sym environment &&= environment.to_sym @@ -969,8 +1006,8 @@ Generated on #{Time.now}. # Check the cache first. It needs to be a per-environment # cache so that we don't spread values from one env # to another. - if cached = @cache[environment||"none"][param] - return cached + if @cache[environment||"none"].has_key?(param) + return @cache[environment||"none"][param] end val = uninterpolated_value(param, environment) @@ -1183,7 +1220,7 @@ Generated on #{Time.now}. # begin return true if self[:config] - rescue InterpolationError => err + rescue InterpolationError # This means we failed to interpolate, which means that they didn't # explicitly specify either :config or :confdir... so we'll fall out to # the default value. diff --git a/lib/puppet/settings/boolean_setting.rb b/lib/puppet/settings/boolean_setting.rb index 13125d0e6..b933116ca 100644 --- a/lib/puppet/settings/boolean_setting.rb +++ b/lib/puppet/settings/boolean_setting.rb @@ -1,5 +1,3 @@ -require 'puppet/settings/base_setting' - # A simple boolean. class Puppet::Settings::BooleanSetting < Puppet::Settings::BaseSetting # get the arguments in getopt format diff --git a/lib/puppet/settings/config_file.rb b/lib/puppet/settings/config_file.rb index 88f45f4f5..b1120105b 100644 --- a/lib/puppet/settings/config_file.rb +++ b/lib/puppet/settings/config_file.rb @@ -1,5 +1,3 @@ -require 'puppet/settings/errors' - ## # @api private # diff --git a/lib/puppet/settings/directory_setting.rb b/lib/puppet/settings/directory_setting.rb index 732d5e875..2a9472f92 100644 --- a/lib/puppet/settings/directory_setting.rb +++ b/lib/puppet/settings/directory_setting.rb @@ -1,5 +1,3 @@ -require 'puppet/settings/file_setting' - class Puppet::Settings::DirectorySetting < Puppet::Settings::FileSetting def type :directory diff --git a/lib/puppet/settings/duration_setting.rb b/lib/puppet/settings/duration_setting.rb index 776c4dd88..3bbb3aab6 100644 --- a/lib/puppet/settings/duration_setting.rb +++ b/lib/puppet/settings/duration_setting.rb @@ -1,5 +1,3 @@ -require 'puppet/settings/base_setting' - # A setting that represents a span of time, and evaluates to an integer # number of seconds after being parsed class Puppet::Settings::DurationSetting < Puppet::Settings::BaseSetting diff --git a/lib/puppet/settings/enum_setting.rb b/lib/puppet/settings/enum_setting.rb new file mode 100644 index 000000000..d1f210810 --- /dev/null +++ b/lib/puppet/settings/enum_setting.rb @@ -0,0 +1,16 @@ +class Puppet::Settings::EnumSetting < Puppet::Settings::BaseSetting + attr_accessor :values + + def type + :enum + end + + def munge(value) + if values.include?(value) + value + else + raise Puppet::Settings::ValidationError, + "Invalid value '#{value}' for parameter #{@name}. Allowed values are '#{values.join("', '")}'" + end + end +end diff --git a/lib/puppet/settings/file_setting.rb b/lib/puppet/settings/file_setting.rb index 8227c5a4b..265bc0613 100644 --- a/lib/puppet/settings/file_setting.rb +++ b/lib/puppet/settings/file_setting.rb @@ -1,5 +1,3 @@ -require 'puppet/settings/string_setting' - # A file. class Puppet::Settings::FileSetting < Puppet::Settings::StringSetting class SettingError < StandardError; end diff --git a/lib/puppet/settings/path_setting.rb b/lib/puppet/settings/path_setting.rb index ab7205dd9..167965706 100644 --- a/lib/puppet/settings/path_setting.rb +++ b/lib/puppet/settings/path_setting.rb @@ -1,5 +1,3 @@ -require 'puppet/settings/string_setting' - class Puppet::Settings::PathSetting < Puppet::Settings::StringSetting def munge(value) if value.is_a?(String) diff --git a/lib/puppet/settings/string_setting.rb b/lib/puppet/settings/string_setting.rb index 6a62bdb6a..30be173a4 100644 --- a/lib/puppet/settings/string_setting.rb +++ b/lib/puppet/settings/string_setting.rb @@ -1,6 +1,3 @@ -# The base element type. -require 'puppet/settings/base_setting' - class Puppet::Settings::StringSetting < Puppet::Settings::BaseSetting def type :string diff --git a/lib/puppet/settings/terminus_setting.rb b/lib/puppet/settings/terminus_setting.rb index 8e8abbbde..376565bd0 100644 --- a/lib/puppet/settings/terminus_setting.rb +++ b/lib/puppet/settings/terminus_setting.rb @@ -1,5 +1,3 @@ -require 'puppet/settings/base_setting' - class Puppet::Settings::TerminusSetting < Puppet::Settings::BaseSetting def munge(value) case value diff --git a/lib/puppet/ssl/certificate_authority.rb b/lib/puppet/ssl/certificate_authority.rb index 6229a2d16..83ff6a114 100644 --- a/lib/puppet/ssl/certificate_authority.rb +++ b/lib/puppet/ssl/certificate_authority.rb @@ -84,7 +84,10 @@ class Puppet::SSL::CertificateAuthority store = autosign_store(auto) if auto != true Puppet::SSL::CertificateRequest.indirection.search("*").each do |csr| - sign(csr.name) if auto == true or store.allowed?(csr.name, "127.1.1.1") + if auto == true or store.allowed?(csr.name, "127.1.1.1") + Puppet.info "Autosigning #{csr.name}" + sign(csr.name) + end end end @@ -128,16 +131,18 @@ class Puppet::SSL::CertificateAuthority end # Generate a new certificate. + # @return Puppet::SSL::Certificate def generate(name, options = {}) raise ArgumentError, "A Certificate already exists for #{name}" if Puppet::SSL::Certificate.indirection.find(name) - host = Puppet::SSL::Host.new(name) # Pass on any requested subjectAltName field. san = options[:dns_alt_names] host = Puppet::SSL::Host.new(name) host.generate_certificate_request(:dns_alt_names => san) - sign(name, !!san) + # CSR may have been implicitly autosigned, generating a certificate + # Or sign explicitly + host.certificate || sign(name, !!san) end # Generate our CA certificate. @@ -192,9 +197,26 @@ class Puppet::SSL::CertificateAuthority pass end - # List all signed certificates. + # Lists the names of all signed certificates. + # + # @return [Array<String>] def list - Puppet::SSL::Certificate.indirection.search("*").collect { |c| c.name } + list_certificates.collect { |c| c.name } + end + + # Return all the certificate objects as found by the indirector + # API for PE license checking. + # + # Created to prevent the case of reading all certs from disk, getting + # just their names and verifying the cert for each name, which then + # causes the cert to again be read from disk. + # + # @author Jeff Weiss <jeff.weiss@puppetlabs.com> + # @api Puppet Enterprise Licensing + # + # @return [Array<Puppet::SSL::Certificate>] + def list_certificates + Puppet::SSL::Certificate.indirection.search("*") end # Read the next serial from the serial file, and increment the @@ -354,16 +376,87 @@ class Puppet::SSL::CertificateAuthority return true # good enough for us! end - # Verify a given host's certificate. - def verify(name) - unless cert = Puppet::SSL::Certificate.indirection.find(name) - raise ArgumentError, "Could not find a certificate for #{name}" + # Utility method for optionally caching the X509 Store for verifying a + # large number of certificates in a short amount of time--exactly the + # case we have during PE license checking. + # + # @example Use the cached X509 store + # x509store(:cache => true) + # + # @example Use a freshly create X509 store + # x509store + # x509store(:cache => false) + # + # @param [Hash] options the options used for retrieving the X509 Store + # @option options [Boolean] :cache whether or not to use a cached version + # of the X509 Store + # + # @return [OpenSSL::X509::Store] + def x509_store(options = {}) + if (options[:cache]) + return @x509store unless @x509store.nil? + @x509store = create_x509_store + else + create_x509_store end + end + private :x509_store + + # Creates a brand new OpenSSL::X509::Store with the appropriate + # Certificate Revocation List and flags + # + # @return [OpenSSL::X509::Store] + def create_x509_store store = OpenSSL::X509::Store.new store.add_file Puppet[:cacert] store.add_crl crl.content if self.crl store.purpose = OpenSSL::X509::PURPOSE_SSL_CLIENT store.flags = OpenSSL::X509::V_FLAG_CRL_CHECK_ALL|OpenSSL::X509::V_FLAG_CRL_CHECK if Puppet.settings[:certificate_revocation] + store + end + private :create_x509_store + + # Utility method which is API for PE license checking. + # This is used rather than `verify` because + # 1) We have already read the certificate from disk into memory. + # To read the certificate from disk again is just wasteful. + # 2) Because we're checking a large number of certificates against + # a transient CertificateAuthority, we can relatively safely cache + # the X509 Store that actually does the verification. + # + # Long running instances of CertificateAuthority will certainly + # want to use `verify` because it will recreate the X509 Store with + # the absolutely latest CRL. + # + # Additionally, this method explicitly returns a boolean whereas + # `verify` will raise an error if the certificate has been revoked. + # + # @author Jeff Weiss <jeff.weiss@puppetlabs.com> + # @api Puppet Enterprise Licensing + # + # @param cert [Puppet::SSL::Certificate] the certificate to check validity of + # + # @return [Boolean] true if signed, false if unsigned or revoked + def certificate_is_alive?(cert) + x509_store(:cache => true).verify(cert.content) + end + + # Verify a given host's certificate. The certname is passed in, and + # the indirector will be used to locate the actual contents of the + # certificate with that name. + # + # @param name [String] certificate name to verify + # + # @raise [ArgumentError] if the certificate name cannot be found + # (i.e. doesn't exist or is unsigned) + # @raise [CertificateVerficationError] if the certificate has been revoked + # + # @return [Boolean] true if signed, there are no cases where false is returned + def verify(name) + unless cert = Puppet::SSL::Certificate.indirection.find(name) + raise ArgumentError, "Could not find a certificate for #{name}" + end + store = x509_store raise CertificateVerificationError.new(store.error), store.error_string unless store.verify(cert.content) end diff --git a/lib/puppet/test/test_helper.rb b/lib/puppet/test/test_helper.rb index cc57bd359..c09a1ade2 100644 --- a/lib/puppet/test/test_helper.rb +++ b/lib/puppet/test/test_helper.rb @@ -83,6 +83,7 @@ module Puppet::Test Puppet::Node::Environment.clear Puppet::Parser::Functions.reset + Puppet::Application.clear! Puppet.clear_deprecation_warnings end diff --git a/lib/puppet/transaction.rb b/lib/puppet/transaction.rb index 68671aee5..a37ec59ad 100644 --- a/lib/puppet/transaction.rb +++ b/lib/puppet/transaction.rb @@ -1,12 +1,14 @@ -# the class that actually walks our resource/property tree, collects the changes, -# and performs them - require 'puppet' require 'puppet/util/tagging' require 'puppet/application' require 'digest/sha1' +# the class that actually walks our resource/property tree, collects the changes, +# and performs them +# +# @api private class Puppet::Transaction + require 'puppet/transaction/additional_resource_generator' require 'puppet/transaction/event' require 'puppet/transaction/event_manager' require 'puppet/transaction/resource_harness' @@ -23,19 +25,93 @@ class Puppet::Transaction # Handles most of the actual interacting with resources attr_reader :resource_harness + attr_reader :prefetched_providers + include Puppet::Util include Puppet::Util::Tagging - # Wraps application run state check to flag need to interrupt processing - def stop_processing? - Puppet::Application.stop_requested? + def initialize(catalog, report, prioritizer) + @catalog = catalog + + @report = report || Puppet::Transaction::Report.new("apply", catalog.version, catalog.environment) + + @prioritizer = prioritizer + + @report.add_times(:config_retrieval, @catalog.retrieval_duration || 0) + + @event_manager = Puppet::Transaction::EventManager.new(self) + + @resource_harness = Puppet::Transaction::ResourceHarness.new(self) + + @prefetched_providers = Hash.new { |h,k| h[k] = {} } end - # Add some additional times for reporting - def add_times(hash) - hash.each do |name, num| - report.add_times(name, num) + # This method does all the actual work of running a transaction. It + # collects all of the changes, executes them, and responds to any + # necessary events. + def evaluate(&block) + block ||= method(:eval_resource) + generator = AdditionalResourceGenerator.new(@catalog, relationship_graph, @prioritizer) + @catalog.vertices.each { |resource| generator.generate_additional_resources(resource) } + + Puppet.info "Applying configuration version '#{catalog.version}'" if catalog.version + + continue_while = lambda { !stop_processing? } + + pre_process = lambda do |resource| + prefetch_if_necessary(resource) + + # If we generated resources, we don't know what they are now + # blocking, so we opt to recompute it, rather than try to track every + # change that would affect the number. + relationship_graph.clear_blockers if generator.eval_generate(resource) end + + providerless_types = [] + overly_deferred_resource_handler = lambda do |resource| + # We don't automatically assign unsuitable providers, so if there + # is one, it must have been selected by the user. + if resource.provider + resource.err "Provider #{resource.provider.class.name} is not functional on this host" + else + providerless_types << resource.type + end + + resource_status(resource).failed = true + end + + canceled_resource_handler = lambda do |resource| + resource_status(resource).skipped = true + resource.debug "Transaction canceled, skipping" + end + + teardown = lambda do + # Just once per type. No need to punish the user. + providerless_types.uniq.each do |type| + Puppet.err "Could not find a suitable provider for #{type}" + end + end + + relationship_graph.traverse(:while => continue_while, + :pre_process => pre_process, + :overly_deferred_resource_handler => overly_deferred_resource_handler, + :canceled_resource_handler => canceled_resource_handler, + :teardown => teardown) do |resource| + if resource.is_a?(Puppet::Type::Component) + Puppet.warning "Somehow left a component in the relationship graph" + else + resource.info "Starting to evaluate the resource" if Puppet[:evaltrace] and @catalog.host_config? + seconds = thinmark { block.call(resource) } + resource.info "Evaluated in %0.2f seconds" % seconds if Puppet[:evaltrace] and @catalog.host_config? + end + end + + Puppet.debug "Finishing transaction #{object_id}" + end + + # Wraps application run state check to flag need to interrupt processing + def stop_processing? + Puppet::Application.stop_requested? && catalog.host_config? end # Are there any failed resources in this transaction? @@ -43,35 +119,50 @@ class Puppet::Transaction report.resource_statuses.values.detect { |status| status.failed? } end - # Apply all changes for a resource - def apply(resource, ancestor = nil) - status = resource_harness.evaluate(resource) - add_resource_status(status) - event_manager.queue_events(ancestor || resource, status.events) unless status.failed? - rescue => detail - resource.err "Could not evaluate: #{detail}" - end - # Find all of the changed resources. def changed? report.resource_statuses.values.find_all { |status| status.changed }.collect { |status| catalog.resource(status.resource) } end - # Find all of the applied resources (including failed attempts). - def applied_resources - report.resource_statuses.values.collect { |status| catalog.resource(status.resource) } + def relationship_graph + catalog.relationship_graph end - # Copy an important relationships from the parent to the newly-generated - # child resource. - def add_conditional_directed_dependency(parent, child, label=nil) - relationship_graph.add_vertex(child) - edge = parent.depthfirst? ? [child, parent] : [parent, child] - if relationship_graph.edge?(*edge.reverse) - parent.debug "Skipping automatic relationship to #{child}" - else - relationship_graph.add_edge(edge[0],edge[1],label) + def resource_status(resource) + report.resource_statuses[resource.to_s] || add_resource_status(Puppet::Resource::Status.new(resource)) + end + + # The tags we should be checking. + def tags + self.tags = Puppet[:tags] unless defined?(@tags) + + super + end + + def prefetch_if_necessary(resource) + provider_class = resource.provider.class + return unless provider_class.respond_to?(:prefetch) and !prefetched_providers[resource.type][provider_class.name] + + resources = resources_by_provider(resource.type, provider_class.name) + + if provider_class == resource.class.defaultprovider + providerless_resources = resources_by_provider(resource.type, nil) + providerless_resources.values.each {|res| res.provider = provider_class.name} + resources.merge! providerless_resources end + + prefetch(provider_class, resources) + end + + private + + # Apply all changes for a resource + def apply(resource, ancestor = nil) + status = resource_harness.evaluate(resource) + add_resource_status(status) + event_manager.queue_events(ancestor || resource, status.events) unless status.failed? + rescue => detail + resource.err "Could not evaluate: #{detail}" end # Evaluate a single resource. @@ -87,31 +178,6 @@ class Puppet::Transaction event_manager.process_events(resource) end - # This method does all the actual work of running a transaction. It - # collects all of the changes, executes them, and responds to any - # necessary events. - def evaluate - add_dynamically_generated_resources - - Puppet.info "Applying configuration version '#{catalog.version}'" if catalog.version - - relationship_graph.traverse do |resource| - if resource.is_a?(Puppet::Type::Component) - Puppet.warning "Somehow left a component in the relationship graph" - else - resource.info "Starting to evaluate the resource" if Puppet[:evaltrace] and @catalog.host_config? - seconds = thinmark { eval_resource(resource) } - resource.info "Evaluated in %0.2f seconds" % seconds if Puppet[:evaltrace] and @catalog.host_config? - end - end - - Puppet.debug "Finishing transaction #{object_id}" - end - - def events - event_manager.events - end - def failed?(resource) s = resource_status(resource) and s.failed? end @@ -144,56 +210,6 @@ class Puppet::Transaction found_failed end - def eval_generate(resource) - return false unless resource.respond_to?(:eval_generate) - raise Puppet::DevError,"Depthfirst resources are not supported by eval_generate" if resource.depthfirst? - begin - made = resource.eval_generate.uniq - return false if made.empty? - made = Hash[made.map(&:name).zip(made)] - rescue => detail - resource.log_exception(detail, "Failed to generate additional resources using 'eval_generate: #{detail}") - return false - end - made.values.each do |res| - begin - res.tag(*resource.tags) - @catalog.add_resource(res) - res.finish - rescue Puppet::Resource::Catalog::DuplicateResourceError - res.info "Duplicate generated resource; skipping" - end - end - sentinel = Puppet::Type.type(:whit).new(:name => "completed_#{resource.title}", :catalog => resource.catalog) - - # The completed whit is now the thing that represents the resource is done - relationship_graph.adjacent(resource,:direction => :out,:type => :edges).each { |e| - # But children run as part of the resource, not after it - next if made[e.target.name] - - add_conditional_directed_dependency(sentinel, e.target, e.label) - relationship_graph.remove_edge! e - } - - default_label = Puppet::Resource::Catalog::Default_label - made.values.each do |res| - # Depend on the nearest ancestor we generated, falling back to the - # resource if we have none - parent_name = res.ancestors.find { |a| made[a] and made[a] != res } - parent = made[parent_name] || resource - - add_conditional_directed_dependency(parent, res) - - # This resource isn't 'completed' until each child has run - add_conditional_directed_dependency(res, sentinel, default_label) - end - - # This edge allows the resource's events to propagate, though it isn't - # strictly necessary for ordering purposes - add_conditional_directed_dependency(resource, sentinel, default_label) - true - end - # A general method for recursively generating new resources from a # resource. def generate_additional_resources(resource) @@ -218,29 +234,11 @@ class Puppet::Transaction end end - def add_dynamically_generated_resources - @catalog.vertices.each { |resource| generate_additional_resources(resource) } - end - # Should we ignore tags? def ignore_tags? ! @catalog.host_config? end - # this should only be called by a Puppet::Type::Component resource now - # and it should only receive an array - def initialize(catalog, report = nil) - @catalog = catalog - - @report = report || Puppet::Transaction::Report.new("apply", catalog.version, catalog.environment) - - @event_manager = Puppet::Transaction::EventManager.new(self) - - @resource_harness = Puppet::Transaction::ResourceHarness.new(self) - - @prefetched_providers = Hash.new { |h,k| h[k] = {} } - end - def resources_by_provider(type_name, provider_name) unless @resources_by_provider @resources_by_provider = Hash.new { |h, k| h[k] = Hash.new { |h, k| h[k] = {} } } @@ -256,23 +254,6 @@ class Puppet::Transaction @resources_by_provider[type_name][provider_name] || {} end - def prefetch_if_necessary(resource) - provider_class = resource.provider.class - return unless provider_class.respond_to?(:prefetch) and !prefetched_providers[resource.type][provider_class.name] - - resources = resources_by_provider(resource.type, provider_class.name) - - if provider_class == resource.class.defaultprovider - providerless_resources = resources_by_provider(resource.type, nil) - providerless_resources.values.each {|res| res.provider = provider_class.name} - resources.merge! providerless_resources - end - - prefetch(provider_class, resources) - end - - attr_reader :prefetched_providers - # Prefetch any providers that support it, yo. We don't support prefetching # types, just providers. def prefetch(provider_class, resources) @@ -287,155 +268,13 @@ class Puppet::Transaction @prefetched_providers[type_name][provider_class.name] = true end - # We want to monitor changes in the relationship graph of our - # catalog but this is complicated by the fact that the catalog - # both is_a graph and has_a graph, by the fact that changes to - # the structure of the object can have adverse serialization - # effects, by threading issues, by order-of-initialization issues, - # etc. - # - # Since the proper lifetime/scope of the monitoring is a transaction - # and the transaction is already commiting a mild law-of-demeter - # transgression, we cut the Gordian knot here by simply wrapping the - # transaction's view of the resource graph to capture and maintain - # the information we need. Nothing outside the transaction needs - # this information, and nothing outside the transaction can see it - # except via the Transaction#relationship_graph - - class Relationship_graph_wrapper - require 'puppet/rb_tree_map' - attr_reader :real_graph,:transaction,:ready,:generated,:done,:blockers,:unguessable_deterministic_key - def initialize(real_graph,transaction) - @real_graph = real_graph - @transaction = transaction - @ready = Puppet::RbTreeMap.new - @generated = {} - @done = {} - @blockers = {} - @unguessable_deterministic_key = Hash.new { |h,k| h[k] = Digest::SHA1.hexdigest("NaCl, MgSO4 (salts) and then #{k.ref}") } - @providerless_types = [] - end - def method_missing(*args,&block) - real_graph.send(*args,&block) - end - def add_vertex(v) - real_graph.add_vertex(v) - end - def add_edge(f,t,label=nil) - key = unguessable_deterministic_key[t] - - ready.delete(key) - - real_graph.add_edge(f,t,label) - end - # Enqueue the initial set of resources, those with no dependencies. - def enqueue_roots - vertices.each do |v| - blockers[v] = direct_dependencies_of(v).length - enqueue(v) if blockers[v] == 0 - end - end - # Decrement the blocker count for the resource by 1. If the number of - # blockers is unknown, count them and THEN decrement by 1. - def unblock(resource) - blockers[resource] ||= direct_dependencies_of(resource).select { |r2| !done[r2] }.length - if blockers[resource] > 0 - blockers[resource] -= 1 - else - resource.warning "appears to have a negative number of dependencies" - end - blockers[resource] <= 0 - end - def enqueue(*resources) - resources.each do |resource| - key = unguessable_deterministic_key[resource] - ready[key] = resource - end - end - def finish(resource) - direct_dependents_of(resource).each do |v| - enqueue(v) if unblock(v) - end - done[resource] = true - end - def next_resource - ready.delete_min - end - def traverse(&block) - real_graph.report_cycles_in_graph - - enqueue_roots - - deferred_resources = [] - - while (resource = next_resource) && !transaction.stop_processing? - if resource.suitable? - made_progress = true - - transaction.prefetch_if_necessary(resource) - - # If we generated resources, we don't know what they are now - # blocking, so we opt to recompute it, rather than try to track every - # change that would affect the number. - blockers.clear if transaction.eval_generate(resource) - - yield resource - - finish(resource) - else - deferred_resources << resource - end - - if ready.empty? and deferred_resources.any? - if made_progress - enqueue(*deferred_resources) - else - fail_unsuitable_resources(deferred_resources) - end - - made_progress = false - deferred_resources = [] - end - end - - # Just once per type. No need to punish the user. - @providerless_types.uniq.each do |type| - Puppet.err "Could not find a suitable provider for #{type}" - end - end - - def fail_unsuitable_resources(resources) - resources.each do |resource| - # We don't automatically assign unsuitable providers, so if there - # is one, it must have been selected by the user. - if resource.provider - resource.err "Provider #{resource.provider.class.name} is not functional on this host" - else - @providerless_types << resource.type - end - - transaction.resource_status(resource).failed = true - - finish(resource) - end - end - end - - def relationship_graph - @relationship_graph ||= Relationship_graph_wrapper.new(catalog.relationship_graph,self) - end - def add_resource_status(status) - report.add_resource_status status - end - - def resource_status(resource) - report.resource_statuses[resource.to_s] || add_resource_status(Puppet::Resource::Status.new(resource)) + report.add_resource_status(status) end # Is the resource currently scheduled? def scheduled?(resource) - self.ignoreschedules or resource_harness.scheduled?(resource_status(resource), resource) + self.ignoreschedules or resource_harness.scheduled?(resource) end # Should this resource be skipped? @@ -454,19 +293,18 @@ class Puppet::Transaction end elsif resource.virtual? resource.debug "Skipping because virtual" - elsif resource.appliable_to_device? ^ for_network_device - resource.debug "Skipping #{resource.appliable_to_device? ? 'device' : 'host'} resources because running on a #{for_network_device ? 'device' : 'host'}" + elsif !host_and_device_resource?(resource) && resource.appliable_to_host? && for_network_device + resource.debug "Skipping host resources because running on a device" + elsif !host_and_device_resource?(resource) && resource.appliable_to_device? && !for_network_device + resource.debug "Skipping device resources because running on a posix host" else return false end true end - # The tags we should be checking. - def tags - self.tags = Puppet[:tags] unless defined?(@tags) - - super + def host_and_device_resource?(resource) + resource.appliable_to_host? && resource.appliable_to_device? end def handle_qualified_tags( qualified ) diff --git a/lib/puppet/transaction/additional_resource_generator.rb b/lib/puppet/transaction/additional_resource_generator.rb new file mode 100644 index 000000000..60a2af2eb --- /dev/null +++ b/lib/puppet/transaction/additional_resource_generator.rb @@ -0,0 +1,126 @@ +# Adds additional resources to the catalog and relationship graph that are +# generated by existing resources. There are two ways that a resource can +# generate additional resources, either through the #generate method or the +# #eval_generate method. +# +# @api private +class Puppet::Transaction::AdditionalResourceGenerator + def initialize(catalog, relationship_graph, prioritizer) + @catalog = catalog + @relationship_graph = relationship_graph + @prioritizer = prioritizer + end + + def generate_additional_resources(resource) + return unless resource.respond_to?(:generate) + begin + generated = resource.generate + rescue => detail + resource.log_exception(detail, "Failed to generate additional resources using 'generate': #{detail}") + end + return unless generated + generated = [generated] unless generated.is_a?(Array) + generated.collect do |res| + @catalog.resource(res.ref) || res + end.each do |res| + priority = @prioritizer.generate_priority_contained_in(resource, res) + add_resource(res, resource, priority) + + add_conditional_directed_dependency(resource, res) + generate_additional_resources(res) + end + end + + def eval_generate(resource) + return false unless resource.respond_to?(:eval_generate) + raise Puppet::DevError,"Depthfirst resources are not supported by eval_generate" if resource.depthfirst? + begin + generated = replace_duplicates_with_catalog_resources(resource.eval_generate) + return false if generated.empty? + rescue => detail + resource.log_exception(detail, "Failed to generate additional resources using 'eval_generate: #{detail}") + return false + end + add_resources(generated, resource) + + made = Hash[generated.map(&:name).zip(generated)] + contain_generated_resources_in(resource, made) + connect_resources_to_ancestors(resource, made) + + true + end + + private + + def replace_duplicates_with_catalog_resources(generated) + generated.collect do |generated_resource| + @catalog.resource(generated_resource.ref) || generated_resource + end + end + + def contain_generated_resources_in(resource, made) + sentinel = Puppet::Type.type(:whit).new(:name => "completed_#{resource.title}", :catalog => resource.catalog) + priority = @prioritizer.generate_priority_contained_in(resource, sentinel) + @relationship_graph.add_vertex(sentinel, priority) + + redirect_edges_to_sentinel(resource, sentinel, made) + + made.values.each do |res| + # This resource isn't 'completed' until each child has run + add_conditional_directed_dependency(res, sentinel, Puppet::Graph::RelationshipGraph::Default_label) + end + + # This edge allows the resource's events to propagate, though it isn't + # strictly necessary for ordering purposes + add_conditional_directed_dependency(resource, sentinel, Puppet::Graph::RelationshipGraph::Default_label) + end + + def redirect_edges_to_sentinel(resource, sentinel, made) + @relationship_graph.adjacent(resource, :direction => :out, :type => :edges).each do |e| + next if made[e.target.name] + + @relationship_graph.add_relationship(sentinel, e.target, e.label) + @relationship_graph.remove_edge! e + end + end + + def connect_resources_to_ancestors(resource, made) + made.values.each do |res| + # Depend on the nearest ancestor we generated, falling back to the + # resource if we have none + parent_name = res.ancestors.find { |a| made[a] and made[a] != res } + parent = made[parent_name] || resource + + add_conditional_directed_dependency(parent, res) + end + end + + def add_resources(generated, resource) + generated.each do |res| + priority = @prioritizer.generate_priority_contained_in(resource, res) + add_resource(res, resource, priority) + end + end + + def add_resource(res, parent_resource, priority) + if @catalog.resource(res.ref).nil? + res.tag(*parent_resource.tags) + @catalog.add_resource(res) + @relationship_graph.add_vertex(res, priority) + @catalog.add_edge(@catalog.container_of(parent_resource), res) + res.finish + end + end + + # Copy an important relationships from the parent to the newly-generated + # child resource. + def add_conditional_directed_dependency(parent, child, label=nil) + @relationship_graph.add_vertex(child) + edge = parent.depthfirst? ? [child, parent] : [parent, child] + if @relationship_graph.edge?(*edge.reverse) + parent.debug "Skipping automatic relationship to #{child}" + else + @relationship_graph.add_relationship(edge[0],edge[1],label) + end + end +end diff --git a/lib/puppet/transaction/event.rb b/lib/puppet/transaction/event.rb index ac8937f8e..0af7a8f63 100644 --- a/lib/puppet/transaction/event.rb +++ b/lib/puppet/transaction/event.rb @@ -26,6 +26,7 @@ class Puppet::Transaction::Event def initialize(options = {}) @audited = false + set_options(options) @time = Time.now end @@ -37,12 +38,26 @@ class Puppet::Transaction::Event @desired_value = data['desired_value'] @historical_value = data['historical_value'] @message = data['message'] - @name = data['name'].intern + @name = data['name'].intern if data['name'] @status = data['status'] @time = data['time'] @time = Time.parse(@time) if @time.is_a? String end + def to_pson + { + 'audited' => @audited, + 'property' => @property, + 'previous_value' => @previous_value, + 'desired_value' => @desired_value, + 'historical_value' => @historical_value, + 'message' => @message, + 'name' => @name, + 'status' => @status, + 'time' => @time.iso8601(9), + }.to_pson + end + def property=(prop) @property = prop.to_s end diff --git a/lib/puppet/transaction/report.rb b/lib/puppet/transaction/report.rb index 4ae09bc70..7223a5fba 100644 --- a/lib/puppet/transaction/report.rb +++ b/lib/puppet/transaction/report.rb @@ -44,6 +44,10 @@ class Puppet::Transaction::Report # @return [???] the configuration version attr_accessor :configuration_version + # An agent generated transaction uuid, useful for connecting catalog and report + # @return [String] uuid + attr_accessor :transaction_uuid + # The host name for which the report is generated # @return [String] the host name attr_accessor :host @@ -58,7 +62,7 @@ class Puppet::Transaction::Report attr_reader :resource_statuses # A list of log messages. - # @return [Array<String>] logged messages + # @return [Array<Puppet::Util::Log>] logged messages attr_reader :logs # A hash of metric name to metric value. @@ -88,26 +92,22 @@ class Puppet::Transaction::Report # attr_reader :puppet_version - # @return [Integer] (3) a report format version number - # @todo Unclear what this is - a version? + # @return [Integer] report format version number. This value is constant for + # a given version of Puppet; it is incremented when a new release of Puppet + # changes the API for the various objects that make up a report. # attr_reader :report_format - # This is necessary since Marshal doesn't know how to - # dump hash with default proc (see below "@records") ? - # @todo there is no "@records" to see below, uncertain what this is for. - # @api private - # - def self.default_format - :yaml - end - def self.from_pson(data) obj = self.allocate obj.initialize_from_hash(data) obj end + def as_logging_destination(&block) + Puppet::Util::Log.with_destination(self, &block) + end + # @api private def <<(msg) @logs << msg @@ -165,7 +165,7 @@ class Puppet::Transaction::Report end # @api private - def initialize(kind, configuration_version=nil, environment=nil) + def initialize(kind, configuration_version=nil, environment=nil, transaction_uuid=nil) @metrics = {} @logs = [] @resource_statuses = {} @@ -173,9 +173,10 @@ class Puppet::Transaction::Report @host = Puppet[:node_name_value] @time = Time.now @kind = kind - @report_format = 3 + @report_format = 4 @puppet_version = Puppet.version @configuration_version = configuration_version + @transaction_uuid = transaction_uuid @environment = environment @status = 'failed' # assume failed until the report is finalized end @@ -185,6 +186,7 @@ class Puppet::Transaction::Report @puppet_version = data['puppet_version'] @report_format = data['report_format'] @configuration_version = data['configuration_version'] + @transaction_uuid = data['transaction_uuid'] @environment = data['environment'] @status = data['status'] @host = data['host'] @@ -214,6 +216,24 @@ class Puppet::Transaction::Report end end + def to_pson + { + 'host' => @host, + 'time' => @time.iso8601(9), + 'configuration_version' => @configuration_version, + 'transaction_uuid' => @transaction_uuid, + 'report_format' => @report_format, + 'puppet_version' => @puppet_version, + 'kind' => @kind, + 'status' => @status, + 'environment' => @environment, + + 'logs' => @logs, + 'metrics' => @metrics, + 'resource_statuses' => @resource_statuses, + }.to_pson + end + # @return [String] the host name # @api public # @@ -295,6 +315,10 @@ class Puppet::Transaction::Report instance_variables - [:@external_times] end + def self.supported_formats + [Puppet[:report_serialization_format].intern] + end + private def calculate_change_metric diff --git a/lib/puppet/transaction/resource_harness.rb b/lib/puppet/transaction/resource_harness.rb index fedccfe99..99ac751b5 100644 --- a/lib/puppet/transaction/resource_harness.rb +++ b/lib/puppet/transaction/resource_harness.rb @@ -29,13 +29,12 @@ class Puppet::Transaction::ResourceHarness end def perform_changes(resource) - current = resource.retrieve_resource + current_values = resource.retrieve_resource.to_hash - cache resource, :checked, Time.now + cache(resource, :checked, Time.now) return [] if ! allow_changes?(resource) - current_values = current.to_hash historical_values = Puppet::Util::Storage.cache(resource).dup desired_values = {} resource.properties.each do |property| @@ -133,33 +132,31 @@ class Puppet::Transaction::ResourceHarness end def evaluate(resource) - start = Time.now status = Puppet::Resource::Status.new(resource) - perform_changes(resource).each do |event| - status << event - end + begin + perform_changes(resource).each do |event| + status << event + end - if status.changed? && ! resource.noop? - cache(resource, :synced, Time.now) - resource.flush if resource.respond_to?(:flush) + if status.changed? && ! resource.noop? + cache(resource, :synced, Time.now) + resource.flush if resource.respond_to?(:flush) + end + rescue => detail + status.failed_because(detail) + ensure + status.evaluation_time = Time.now - status.time end - return status - rescue => detail - resource.fail "Could not create resource status: #{detail}" unless status - resource.log_exception(detail, "Could not evaluate: #{detail}") - status.failed = true - return status - ensure - (status.evaluation_time = Time.now - start) if status + status end def initialize(transaction) @transaction = transaction end - def scheduled?(status, resource) + def scheduled?(resource) return true if Puppet[:ignoreschedules] return true unless schedule = schedule(resource) diff --git a/lib/puppet/type.rb b/lib/puppet/type.rb index eefd52140..55b4a5fad 100644 --- a/lib/puppet/type.rb +++ b/lib/puppet/type.rb @@ -7,7 +7,6 @@ require 'puppet/util' require 'puppet/util/autoload' require 'puppet/metatype/manager' require 'puppet/util/errors' -require 'puppet/util/log_paths' require 'puppet/util/logging' require 'puppet/util/tagging' @@ -77,7 +76,6 @@ module Puppet class Type include Puppet::Util include Puppet::Util::Errors - include Puppet::Util::LogPaths include Puppet::Util::Logging include Puppet::Util::Tagging @@ -303,7 +301,7 @@ class Type # @param metaparam [??? Puppet::Parameter] the meta-parameter to get documentation for. # @return [String] the documentation associated with the given meta-parameter, or nil of not such documentation # exists. - # @raises [?] if the given metaparam is not a meta-parameter in this type + # @raise if the given metaparam is not a meta-parameter in this type # def self.metaparamdoc(metaparam) @@metaparamhash[metaparam].doc @@ -354,11 +352,10 @@ class Type # Returns parameters that act as a key. # All parameters that return true from #isnamevar? or is named `:name` are included in the returned result. # @todo would like a better explanation - # @return Array<??? Puppet::Parameter> - # + # @return [Array<Puppet::Parameter>] WARNING: this return type is uncertain def self.key_attribute_parameters @key_attribute_parameters ||= ( - params = @parameters.find_all { |param| + @parameters.find_all { |param| param.isnamevar? or param.name == :name } ) @@ -748,6 +745,13 @@ class Type @parameters[name] = klass.new(:resource => self) end + # Returns a string representation of the resource's containment path in + # the catalog. + # @return [String] + def path + @path ||= '/' + pathbuilder.join('/') + end + # Returns the value of this object's parameter given by name # @param name [String] the name of the parameter # @return [Object] the value @@ -1003,7 +1007,7 @@ class Type # Parameters and meta-parameters are not included in the result. # @todo As oposed to all non contained properties? How is this different than any of the other # methods that also "gets" properties/parameters/etc. ? - # @return [Array<Object>] array of all property values (mix of types) + # @return [Puppet::Resource] array of all property values (mix of types) # @raise [fail???] if there is a provider and it is not suitable for the host this is evaluated for. def retrieve fail "Provider #{provider.class.name} is not functional on this host" if self.provider.is_a?(Puppet::Provider) and ! provider.class.suitable? @@ -1031,12 +1035,16 @@ class Type result end - # ??? - # @todo what does this do? It seems to create a new Resource based on the result of calling #retrieve - # and if that is a Hash, else this method produces nil. - # @return [Puppet::Resource, nil] a new Resource, or nil, if this object did not produce a Hash as the - # result from #retrieve + # Retrieve the current state of the system as a Puppet::Resource. For + # the base Puppet::Type this does the same thing as #retrieve, but + # specific types are free to implement #retrieve as returning a hash, + # and this will call #retrieve and convert the hash to a resource. + # This is used when determining when syncing a resource. + # + # @return [Puppet::Resource] A resource representing the current state + # of the system. # + # @api private def retrieve_resource resource = retrieve resource = Resource.new(type, title, :parameters => resource) if resource.is_a? Hash @@ -1044,7 +1052,7 @@ class Type end # Returns a hash of the current properties and their values. - # If a resource is absent, it's value is the symbol `:absent` + # If a resource is absent, its value is the symbol `:absent` # @return [Hash{Puppet::Property => Object}] mapping of property instance to its value # def currentpropvalues @@ -1097,7 +1105,7 @@ class Type # Put the default provider first, then the rest of the suitable providers. provider_instances = {} providers_by_source.collect do |provider| - all_properties = self.properties.find_all do |property| + self.properties.find_all do |property| provider.supports_parameter?(property) end.collect do |property| property.name @@ -1138,7 +1146,7 @@ class Type # Converts a simple hash into a Resource instance. # @todo as opposed to a complex hash? Other raised exceptions? - # @param [Hash{Symbol, String => Object}] resource attribute to value map to initialize the created resource from + # @param [Hash{Symbol, String => Object}] hash resource attribute to value map to initialize the created resource from # @return [Puppet::Resource] the resource created from the hash # @raise [Puppet::Error] if a title is missing in the given hash def self.hash2resource(hash) @@ -1152,12 +1160,7 @@ class Type # Now create our resource. resource = Puppet::Resource.new(self.name, title) - [:catalog].each do |attribute| - if value = hash[attribute] - hash.delete(attribute) - resource.send(attribute.to_s + "=", value) - end - end + resource.catalog = hash.delete(:catalog) hash.each do |param, value| resource[param] = value @@ -1165,10 +1168,12 @@ class Type resource end - # Creates the path for logging and such. - # @todo "and such?", what? - # @api private + + # Returns an array of strings representing the containment heirarchy + # (types/classes) that make up the path to the resource from the root + # of the catalog. This is mostly used for logging purposes. # + # @api private def pathbuilder if p = parent [p.pathbuilder, self.ref].flatten @@ -1284,43 +1289,43 @@ class Type end newmetaparam(:alias) do - desc "Creates an alias for the object. Puppet uses this internally when you - provide a symbolic title: + desc %q{Creates an alias for the resource. Puppet uses this internally when you + provide a symbolic title and an explicit namevar value: file { 'sshdconfig': path => $operatingsystem ? { - solaris => \"/usr/local/etc/ssh/sshd_config\", - default => \"/etc/ssh/sshd_config\" + solaris => '/usr/local/etc/ssh/sshd_config', + default => '/etc/ssh/sshd_config', }, - source => \"...\" + source => '...' } service { 'sshd': - subscribe => File['sshdconfig'] + subscribe => File['sshdconfig'], } When you use this feature, the parser sets `sshdconfig` as the title, and the library sets that as an alias for the file so the dependency lookup in `Service['sshd']` works. You can use this metaparameter yourself, - but note that only the library can use these aliases; for instance, - the following code will not work: + but note that aliases generally only work for creating relationships; anything + else that refers to an existing resource (such as amending or overriding + resource attributes in an inherited class) must use the resource's exact + title. For example, the following code will not work: - file { \"/etc/ssh/sshd_config\": + file { '/etc/ssh/sshd_config': owner => root, group => root, - alias => 'sshdconfig' + alias => 'sshdconfig', } - file { 'sshdconfig': - mode => 644 + File['sshdconfig'] { + mode => 644, } There's no way here for the Puppet parser to know that these two stanzas should be affecting the same file. - See the [Language Guide](http://docs.puppetlabs.com/guides/language_guide.html) for more information. - - " + } munge do |aliases| aliases = [aliases] unless aliases.is_a?(Array) @@ -1408,7 +1413,7 @@ class Type @value.each do |ref| unless @resource.catalog.resource(ref.to_s) description = self.class.direction == :in ? "dependency" : "dependent" - fail "Could not find #{description} #{ref} for #{resource.ref}" + fail ResourceError, "Could not find #{description} #{ref} for #{resource.ref}" end end end @@ -1459,7 +1464,7 @@ class Type self.debug("requires #{related_resource.ref}") end - rel = Puppet::Relationship.new(source, target, subargs) + Puppet::Relationship.new(source, target, subargs) end end end @@ -1809,7 +1814,7 @@ class Type end # @todo this does what? where and how? - # @returns [String] the name of the provider + # @return [String] the name of the provider defaultto { prov = @resource.class.defaultprovider prov.name if prov @@ -1819,7 +1824,7 @@ class Type provider_class = provider_class[0] if provider_class.is_a? Array provider_class = provider_class.class.name if provider_class.is_a?(Puppet::Provider) - unless provider = @resource.class.provider(provider_class) + unless @resource.class.provider(provider_class) raise ArgumentError, "Invalid #{@resource.class.name} provider '#{provider_class}'" end end @@ -1942,8 +1947,9 @@ class Type # Adds dependencies to the catalog from added autorequirements. # See {autorequire} for how to add an auto-requirement. # @todo needs details - see the param rel_catalog, and type of this param - # @param rel_catalog [Puppet::Catalog, nil] the catalog to add dependencies to. Defaults to the - # catalog (TODO: what is the type of the catalog). + # @param rel_catalog [Puppet::Resource::Catalog, nil] the catalog to + # add dependencies to. Defaults to the current catalog (set when the + # type instance was added to a catalog) # @raise [Puppet::DevError] if there is no catalog # def autorequire(rel_catalog = nil) @@ -1953,7 +1959,7 @@ class Type reqs = [] self.class.eachautorequire { |type, block| # Ignore any types we can't find, although that would be a bit odd. - next unless typeobj = Puppet::Type.type(type) + next unless Puppet::Type.type(type) # Retrieve the list of names from the block. next unless list = self.instance_eval(&block) @@ -2104,7 +2110,6 @@ class Type # def self.validate(&block) define_method(:validate, &block) - #@validate = block end # @return [String] The file from which this type originates from @@ -2157,8 +2162,8 @@ class Type # resources; one that causes the title to be set to resource.title, and one that # causes the title to be resource.ref ("for components") - what is a component? # - # @overaload initialize(hsh) - # @param hsh [Hash] + # @overload initialize(hash) + # @param [Hash] hash # @raise [Puppet::ResourceError] when the type validation raises # Puppet::Error or ArgumentError # @overload initialize(resource) @@ -2351,14 +2356,14 @@ class Type end end - # Returns the title of this object, or it's name if title was not explicetly set. + # Returns the title of this object, or its name if title was not explicetly set. # If the title is not already set, it will be computed by looking up the {#name_var} and using # that value as the title. # @todo it is somewhat confusing that if the name_var is a valid parameter, it is assumed to # be the name_var called :name, but if it is a property, it uses the name_var. # It is further confusing as Type in some respects supports multiple namevars. # - # @return [String] Returns the title of this object, or it's name if title was not explicetly set. + # @return [String] Returns the title of this object, or its name if title was not explicetly set. # @raise [??? devfail] if title is not set, and name_var can not be found. def title unless @title @@ -2406,14 +2411,15 @@ class Type def exported?; !!@exported; end # @return [Boolean] Returns whether the resource is applicable to `:device` - # @todo Explain what this means + # Returns true if a resource of this type can be evaluated on a 'network device' kind + # of hosts. # @api private def appliable_to_device? self.class.can_apply_to(:device) end # @return [Boolean] Returns whether the resource is applicable to `:host` - # @todo Explain what this means + # Returns true if a resource of this type can be evaluated on a regular generalized computer (ie not an appliance like a network device) # @api private def appliable_to_host? self.class.can_apply_to(:host) diff --git a/lib/puppet/type/component.rb b/lib/puppet/type/component.rb index c8c09409b..4783ef023 100644 --- a/lib/puppet/type/component.rb +++ b/lib/puppet/type/component.rb @@ -29,8 +29,6 @@ Puppet::Type.newtype(:component) do def initialize(*args) @extra_parameters = {} super - - catalog.alias(self, ref) if catalog and ! catalog.resource(ref) end # Component paths are special because they function as containers. diff --git a/lib/puppet/type/cron.rb b/lib/puppet/type/cron.rb index ed1b9d53d..198d6c171 100755 --- a/lib/puppet/type/cron.rb +++ b/lib/puppet/type/cron.rb @@ -243,15 +243,26 @@ Puppet::Type.newtype(:cron) do newproperty(:special) do desc "A special value such as 'reboot' or 'annually'. Only available on supported systems such as Vixie Cron. - Overrides more specific time of day/week settings." + Overrides more specific time of day/week settings. + Set to 'absent' to make puppet revert to a plain numeric schedule." def specials - %w{reboot yearly annually monthly weekly daily midnight hourly} + %w{reboot yearly annually monthly weekly daily midnight hourly absent} + + [ :absent ] end validate do |value| raise ArgumentError, "Invalid special schedule #{value.inspect}" unless specials.include?(value) end + + def munge(value) + # Support value absent so that a schedule can be + # forced to change to numeric. + if value == "absent" or value == :absent + return :absent + end + value + end end newproperty(:minute, :parent => CronParam) do diff --git a/lib/puppet/type/exec.rb b/lib/puppet/type/exec.rb index 35e2c062c..85724cc30 100755 --- a/lib/puppet/type/exec.rb +++ b/lib/puppet/type/exec.rb @@ -79,11 +79,6 @@ module Puppet # Actually execute the command. def sync - olddir = nil - - # We need a dir to change to, even if it's just the cwd - dir = self.resource[:cwd] || Dir.pwd - event = :executed_command tries = self.resource[:tries] try_sleep = self.resource[:try_sleep] @@ -161,7 +156,10 @@ module Puppet use this then any error output is not currently captured. This is because of a bug within Ruby. If you are using Puppet to create this user, the exec will automatically require the user, - as long as it is specified by name." + as long as it is specified by name. + + Please note that the $HOME environment variable is not automatically set + when using this attribute." # Most validation is handled by the SUIDManager class. validate do |user| @@ -232,7 +230,7 @@ module Puppet value = value.shift if value.is_a?(Array) begin value = Float(value) - rescue ArgumentError => e + rescue ArgumentError raise ArgumentError, "The timeout must be a number." end [value, 0.0].max diff --git a/lib/puppet/type/file.rb b/lib/puppet/type/file.rb index 9a40b0c4d..862ab41ba 100644 --- a/lib/puppet/type/file.rb +++ b/lib/puppet/type/file.rb @@ -5,6 +5,7 @@ require 'uri' require 'fileutils' require 'enumerator' require 'pathname' +require 'puppet/parameter/boolean' require 'puppet/util/diff' require 'puppet/util/checksums' require 'puppet/util/backups' @@ -53,7 +54,7 @@ Puppet::Type.newtype(:file) do end munge do |value| - ::File.expand_path(value) + ::File.join(::File.split(::File.expand_path(value))) end end @@ -160,26 +161,22 @@ Puppet::Type.newtype(:file) do end end - newparam(:replace, :boolean => true) do + newparam(:replace, :boolean => true, :parent => Puppet::Parameter::Boolean) do desc "Whether to replace a file or symlink that already exists on the local system but whose content doesn't match what the `source` or `content` attribute specifies. Setting this to false allows file resources to initialize files without overwriting future changes. Note that this only affects content; Puppet will still manage ownership and permissions. Defaults to `true`." - newvalues(:true, :false) - aliasvalue(:yes, :true) - aliasvalue(:no, :false) defaultto :true end - newparam(:force, :boolean => true) do + newparam(:force, :boolean => true, :parent => Puppet::Parameter::Boolean) do desc "Perform the file operation even if it will destroy one or more directories. You must use `force` in order to: * `purge` subdirectories * Replace directories with files or links * Remove a directory when `ensure => absent`" - newvalues(:true, :false) defaultto false end @@ -210,7 +207,7 @@ Puppet::Type.newtype(:file) do defaultto :manage end - newparam(:purge, :boolean => true) do + newparam(:purge, :boolean => true, :parent => Puppet::Parameter::Boolean) do desc "Whether unmanaged files should be purged. This option only makes sense when managing directories with `recurse => true`. @@ -225,8 +222,6 @@ Puppet::Type.newtype(:file) do but if you do not, this will destroy data." defaultto :false - - newvalues(:true, :false) end newparam(:sourceselect) do @@ -242,7 +237,7 @@ Puppet::Type.newtype(:file) do newvalues(:first, :all) end - newparam(:show_diff, :boolean => true) do + newparam(:show_diff, :boolean => true, :parent => Puppet::Parameter::Boolean) do desc "Whether to display differences when the file changes, defaulting to true. This parameter is useful for files that may contain passwords or other secret data, which might otherwise be included in Puppet reports or @@ -250,8 +245,6 @@ Puppet::Type.newtype(:file) do is false, then no diffs will be shown even if this parameter is true." defaultto :true - - newvalues(:true, :false) end # Autorequire the nearest ancestor directory found in the catalog. @@ -380,12 +373,6 @@ Puppet::Type.newtype(:file) do return [] unless self.recurse? recurse - #recurse.reject do |resource| - # catalog.resource(:file, resource[:path]) - #end.each do |child| - # catalog.add_resource child - # catalog.relationship_graph.add_edge self, child - #end end def ancestors @@ -475,16 +462,6 @@ Puppet::Type.newtype(:file) do end end - # Should we be purging? - def purge? - @parameters.include?(:purge) and (self[:purge] == :true or self[:purge] == "true") - end - - # Should we be showing diffs? - def show_diff? - @parameters.include?(:show_diff) and (self[:show_diff] == :true or self[:show_diff] == "true") - end - # Recursively generate a list of file resources, which will # be used to copy remote files, manage local files, and/or make links # to map to another directory. @@ -773,7 +750,7 @@ Puppet::Type.newtype(:file) do # @return [Boolean] If the current file can be backed up and needs to be backed up. def can_backup?(type) - if type == "directory" and self[:force] == :false + if type == "directory" and not force? # (#18110) Directories cannot be removed without :force, so it doesn't # make sense to back them up. false @@ -785,7 +762,7 @@ Puppet::Type.newtype(:file) do # @return [Boolean] True if the directory was removed # @api private def remove_directory(wanted_type) - if self[:force] == :true + if force? debug "Removing existing directory for replacement with #{wanted_type}" FileUtils.rmtree(self[:path]) stat_needed @@ -842,7 +819,7 @@ Puppet::Type.newtype(:file) do def write_temporary_file? # unfortunately we don't know the source file size before fetching it # so let's assume the file won't be empty - (c = property(:content) and c.length) || (s = @parameters[:source] and 1) + (c = property(:content) and c.length) || @parameters[:source] end # There are some cases where all of the work does not get done on diff --git a/lib/puppet/type/file/content.rb b/lib/puppet/type/file/content.rb index a590e03a6..5807d1885 100755 --- a/lib/puppet/type/file/content.rb +++ b/lib/puppet/type/file/content.rb @@ -75,7 +75,7 @@ module Puppet def checksum_type if source = resource.parameter(:source) result = source.checksum - else checksum = resource.parameter(:checksum) + else result = resource[:checksum] end if result =~ /^\{(\w+)\}.+/ @@ -100,6 +100,9 @@ module Puppet if resource.should_be_file? return false if is == :absent else + if resource[:ensure] == :present and resource[:content] and s = resource.stat + resource.warning "Ensure set to :present but file type is #{s.ftype} so no content will be synced" + end return true end diff --git a/lib/puppet/type/file/ctime.rb b/lib/puppet/type/file/ctime.rb index 5f94863b4..e9fbdaf98 100644 --- a/lib/puppet/type/file/ctime.rb +++ b/lib/puppet/type/file/ctime.rb @@ -1,6 +1,8 @@ module Puppet Puppet::Type.type(:file).newproperty(:ctime) do - desc "A read-only state to check the file ctime." + desc %q{A read-only state to check the file ctime. On most modern \*nix-like + systems, this is the time of the most recent change to the owner, group, + permissions, or content of the file.} def retrieve current_value = :absent diff --git a/lib/puppet/type/file/ensure.rb b/lib/puppet/type/file/ensure.rb index 1f5abdb5c..de19e10eb 100755 --- a/lib/puppet/type/file/ensure.rb +++ b/lib/puppet/type/file/ensure.rb @@ -47,7 +47,7 @@ module Puppet property.sync else @resource.write(:ensure) - mode = @resource.should(:mode) + @resource.should(:mode) end end diff --git a/lib/puppet/type/file/mode.rb b/lib/puppet/type/file/mode.rb index 1b3a1e8ab..b6b4becf2 100755 --- a/lib/puppet/type/file/mode.rb +++ b/lib/puppet/type/file/mode.rb @@ -80,7 +80,6 @@ module Puppet # If we're a directory, we need to be executable for all cases # that are readable. This should probably be selectable, but eh. def dirmask(value) - orig = value if FileTest.directory?(resource[:path]) and value =~ /^\d+$/ then value = value.to_i(8) value |= 0100 if value & 0400 != 0 diff --git a/lib/puppet/type/file/mtime.rb b/lib/puppet/type/file/mtime.rb index 10867ddf4..3d57a4998 100644 --- a/lib/puppet/type/file/mtime.rb +++ b/lib/puppet/type/file/mtime.rb @@ -1,6 +1,7 @@ module Puppet Puppet::Type.type(:file).newproperty(:mtime) do - desc "A read-only state to check the file mtime." + desc %q{A read-only state to check the file mtime. On \*nix-like systems, this + is the time of the most recent change to the content of the file.} def retrieve current_value = :absent diff --git a/lib/puppet/type/group.rb b/lib/puppet/type/group.rb index e7d77b8d7..a66a29452 100755 --- a/lib/puppet/type/group.rb +++ b/lib/puppet/type/group.rb @@ -1,6 +1,7 @@ require 'etc' require 'facter' require 'puppet/property/keyvalue' +require 'puppet/parameter/boolean' module Puppet newtype(:group) do @@ -104,11 +105,9 @@ module Puppet isnamevar end - newparam(:allowdupe, :boolean => true) do + newparam(:allowdupe, :boolean => true, :parent => Puppet::Parameter::Boolean) do desc "Whether to allow duplicate GIDs. Defaults to `false`." - newvalues(:true, :false) - defaultto false end @@ -142,18 +141,17 @@ module Puppet defaultto :minimum end - newparam(:system, :boolean => true) do + newparam(:system, :boolean => true, :parent => Puppet::Parameter::Boolean) do desc "Whether the group is a system group with lower GID." - newvalues(:true, :false) - defaultto false end - newparam(:forcelocal, :boolean => true, :required_features => :libuser ) do + newparam(:forcelocal, :boolean => true, + :required_features => :libuser, + :parent => Puppet::Parameter::Boolean) do desc "Forces the mangement of local accounts when accounts are also being managed by some other NSS" - newvalues(:true, :false) defaultto false end @@ -163,7 +161,7 @@ module Puppet # # (see Puppet::Settings#service_group_available?) # - # @returns [Boolean] if the group exists on the system + # @return [Boolean] if the group exists on the system # @api private def exists? provider.exists? diff --git a/lib/puppet/type/host.rb b/lib/puppet/type/host.rb index f4ced3170..d63b37d1a 100755 --- a/lib/puppet/type/host.rb +++ b/lib/puppet/type/host.rb @@ -71,8 +71,7 @@ module Puppet isnamevar validate do |value| - # LAK:NOTE See http://snurl.com/21zf8 [groups_google_com] - x = value.split('.').each do |hostpart| + value.split('.').each do |hostpart| unless hostpart =~ /^([\d\w]+|[\d\w][\d\w\-]+[\d\w])$/ raise Puppet::Error, "Invalid host name" end diff --git a/lib/puppet/type/mcx.rb b/lib/puppet/type/mcx.rb index f9213c115..86bc953a8 100644 --- a/lib/puppet/type/mcx.rb +++ b/lib/puppet/type/mcx.rb @@ -73,7 +73,6 @@ MCX settings refer to, the MCX resource will autorequire that user, group, or co def setup_autorequire(type) # value returns a Symbol - name = value(:name) ds_type = value(:ds_type) ds_name = value(:ds_name) if ds_type == type diff --git a/lib/puppet/type/mount.rb b/lib/puppet/type/mount.rb index b3598965d..27624e31b 100755 --- a/lib/puppet/type/mount.rb +++ b/lib/puppet/type/mount.rb @@ -1,3 +1,5 @@ +require 'puppet/property/boolean' + module Puppet # We want the mount to refresh when it changes. newtype(:mount, :self_refresh => true) do @@ -118,6 +120,10 @@ module Puppet device is supporting by the mount, including network devices or devices specified by UUID rather than device path, depending on the operating system." + + validate do |value| + raise Puppet::Error, "device must not contain whitespace: #{value}" if value =~ /\s/ + end end # Solaris specifies two devices, not just one. @@ -128,10 +134,11 @@ module Puppet # Default to the device but with "dsk" replaced with "rdsk". defaultto do - if Facter["osfamily"].value == "Solaris" - device = @resource.value(:device) - if device =~ %r{/dsk/} + if Facter.value(:osfamily) == "Solaris" + if device = resource[:device] and device =~ %r{/dsk/} device.sub(%r{/dsk/}, "/rdsk/") + elsif fstype = resource[:fstype] and fstype == 'nfs' + '-' else nil end @@ -139,36 +146,63 @@ module Puppet nil end end + + validate do |value| + raise Puppet::Error, "blockdevice must not contain whitespace: #{value}" if value =~ /\s/ + end end newproperty(:fstype) do desc "The mount type. Valid values depend on the operating system. This is a required option." + + validate do |value| + raise Puppet::Error, "fstype must not contain whitespace: #{value}" if value =~ /\s/ + end end newproperty(:options) do desc "Mount options for the mounts, as they would appear in the fstab." + + validate do |value| + raise Puppet::Error, "option must not contain whitespace: #{value}" if value =~ /\s/ + end end newproperty(:pass) do desc "The pass in which the mount is checked." defaultto { - 0 if @resource.managed? + if @resource.managed? + if Facter.value(:osfamily) == 'Solaris' + '-' + else + 0 + end + end } end - newproperty(:atboot) do + newproperty(:atboot, :parent => Puppet::Property::Boolean) do desc "Whether to mount the mount at boot. Not all platforms support this." + + def munge(value) + munged = super + if munged + :yes + else + :no + end + end end newproperty(:dump) do desc "Whether to dump the mount. Not all platform support this. Valid values are `1` or `0`. or `2` on FreeBSD, Default is `0`." - if Facter["operatingsystem"].value == "FreeBSD" + if Facter.value(:operatingsystem) == "FreeBSD" newvalue(%r{(0|1|2)}) else newvalue(%r{(0|1)}) @@ -197,6 +231,14 @@ module Puppet desc "The mount path for the mount." isnamevar + + validate do |value| + raise Puppet::Error, "name must not contain whitespace: #{value}" if value =~ /\s/ + end + + munge do |value| + value.gsub(/^(.+?)\/*$/, '\1') + end end newparam(:remounts) do @@ -222,7 +264,6 @@ module Puppet def value(name) name = name.intern - ret = nil if property = @parameters[name] return property.value end diff --git a/lib/puppet/type/package.rb b/lib/puppet/type/package.rb index cdbec7f7b..5500f4499 100644 --- a/lib/puppet/type/package.rb +++ b/lib/puppet/type/package.rb @@ -294,8 +294,8 @@ module Puppet end newparam(:flavor) do - desc "Newer versions of OpenBSD support 'flavors', which are - further specifications for which type of package you want." + desc "OpenBSD supports 'flavors', which are further specifications for + which type of package you want." end newparam(:install_options, :parent => Puppet::Parameter::PackageOptions, :required_features => :install_options) do diff --git a/lib/puppet/type/resources.rb b/lib/puppet/type/resources.rb index 3e25ea5aa..12bd3cac5 100644 --- a/lib/puppet/type/resources.rb +++ b/lib/puppet/type/resources.rb @@ -1,4 +1,5 @@ require 'puppet' +require 'puppet/parameter/boolean' Puppet::Type.newtype(:resources) do @doc = "This is a metatype that can manage other resource types. Any @@ -17,15 +18,15 @@ Puppet::Type.newtype(:resources) do munge { |v| v.to_s } end - newparam(:purge, :boolean => true) do + newparam(:purge, :boolean => true, :parent => Puppet::Parameter::Boolean) do desc "Purge unmanaged resources. This will delete any resource that is not specified in your configuration and is not required by any specified resources." - newvalues(:true, :false) + defaultto :false validate do |value| - if [:true, true, "true"].include?(value) + if munge(value) unless @resource.resource_type.respond_to?(:instances) raise ArgumentError, "Purging resources of type #{@resource[:name]} is not supported, since they cannot be queried from the system" end @@ -76,7 +77,7 @@ Puppet::Type.newtype(:resources) do def able_to_ensure_absent?(resource) resource[:ensure] = :absent - rescue ArgumentError, Puppet::Error => detail + rescue ArgumentError, Puppet::Error err "The 'ensure' attribute on #{self[:name]} resources does not accept 'absent' as a value" false end diff --git a/lib/puppet/type/schedule.rb b/lib/puppet/type/schedule.rb index 6820501cc..e1586cbdb 100755 --- a/lib/puppet/type/schedule.rb +++ b/lib/puppet/type/schedule.rb @@ -130,7 +130,7 @@ module Puppet def match?(previous, now) # The lowest-level array is of the hour, minute, second triad # then it's an array of two of those, to present the limits - # then it's array of those ranges + # then it's an array of those ranges @value = [@value] unless @value[0][0].is_a?(Array) @value.each do |value| @@ -304,9 +304,6 @@ module Puppet # than the unit of time, we match. We divide the scale # by the repeat, so that we'll repeat that often within # the scale. - diff = (now.to_i - previous.to_i) - comparison = (scale / @resource[:repeat]) - return (now.to_i - previous.to_i) >= (scale / @resource[:repeat]) end end diff --git a/lib/puppet/type/selmodule.rb b/lib/puppet/type/selmodule.rb index a2c467736..70ef60581 100644 --- a/lib/puppet/type/selmodule.rb +++ b/lib/puppet/type/selmodule.rb @@ -1,5 +1,5 @@ # -# Simple module for manageing SELinux policy modules +# Simple module for managing SELinux policy modules # Puppet::Type.newtype(:selmodule) do diff --git a/lib/puppet/type/service.rb b/lib/puppet/type/service.rb index a6a95892f..0a74649cf 100644 --- a/lib/puppet/type/service.rb +++ b/lib/puppet/type/service.rb @@ -142,9 +142,7 @@ module Puppet munge do |value| value = [value] unless value.is_a?(Array) - # LAK:NOTE See http://snurl.com/21zf8 [groups_google_com] - # It affects stand-alone blocks, too. - paths = value.flatten.collect { |p| x = p.split(File::PATH_SEPARATOR) }.flatten + value.flatten.collect { |p| p.split(File::PATH_SEPARATOR) }.flatten end defaultto { provider.class.defpath if provider.class.respond_to?(:defpath) } diff --git a/lib/puppet/type/tidy.rb b/lib/puppet/type/tidy.rb index 63f3e772a..8297cf938 100755 --- a/lib/puppet/type/tidy.rb +++ b/lib/puppet/type/tidy.rb @@ -1,3 +1,5 @@ +require 'puppet/parameter/boolean' + Puppet::Type.newtype(:tidy) do require 'puppet/file_serving/fileset' require 'puppet/file_bucket/dipper' @@ -185,13 +187,11 @@ Puppet::Type.newtype(:tidy) do defaultto :atime end - newparam(:rmdirs, :boolean => true) do + newparam(:rmdirs, :boolean => true, :parent => Puppet::Parameter::Boolean) do desc "Tidy directories in addition to files; that is, remove directories whose age is older than the specified criteria. This will only remove empty directories, so all contained files must also be tidied before a directory gets removed." - - newvalues :true, :false end # Erase PFile's validate method diff --git a/lib/puppet/type/user.rb b/lib/puppet/type/user.rb index d118ee07f..9ec6e922f 100755 --- a/lib/puppet/type/user.rb +++ b/lib/puppet/type/user.rb @@ -1,5 +1,6 @@ require 'etc' require 'facter' +require 'puppet/parameter/boolean' require 'puppet/property/list' require 'puppet/property/ordered_list' require 'puppet/property/keyvalue' @@ -283,35 +284,29 @@ module Puppet defaultto :minimum end - newparam(:system, :boolean => true) do + newparam(:system, :boolean => true, :parent => Puppet::Parameter::Boolean) do desc "Whether the user is a system user, according to the OS's criteria; on most platforms, a UID less than or equal to 500 indicates a system user. Defaults to `false`." - newvalues(:true, :false) - defaultto false end - newparam(:allowdupe, :boolean => true) do + newparam(:allowdupe, :boolean => true, :parent => Puppet::Parameter::Boolean) do desc "Whether to allow duplicate UIDs. Defaults to `false`." - newvalues(:true, :false) - defaultto false end - newparam(:managehome, :boolean => true) do + newparam(:managehome, :boolean => true, :parent => Puppet::Parameter::Boolean) do desc "Whether to manage the home directory when managing the user. This will create the home directory when `ensure => present`, and delete the home directory when `ensure => absent`. Defaults to `false`." - newvalues(:true, :false) - defaultto false validate do |val| - if val.to_s == "true" + if munge(val) raise ArgumentError, "User provider #{provider.class.name} can not manage home directories" if provider and not provider.class.manages_homedir? end end @@ -370,7 +365,7 @@ module Puppet # # (see Puppet::Settings#service_user_available?) # - # @returns [Boolean] if the user exists on the system + # @return [Boolean] if the user exists on the system # @api private def exists? provider.exists? @@ -559,10 +554,11 @@ module Puppet end end - newparam(:forcelocal, :boolean => true, :required_features => :libuser ) do + newparam(:forcelocal, :boolean => true, + :required_features => :libuser, + :parent => Puppet::Parameter::Boolean) do desc "Forces the mangement of local accounts when accounts are also being managed by some other NSS" - newvalues(:true, :false) defaultto false end end diff --git a/lib/puppet/type/yumrepo.rb b/lib/puppet/type/yumrepo.rb index 2b3c03497..bd52121c9 100644 --- a/lib/puppet/type/yumrepo.rb +++ b/lib/puppet/type/yumrepo.rb @@ -1,5 +1,3 @@ -# Description of yum repositories - require 'puppet/util/inifile' module Puppet @@ -232,7 +230,7 @@ module Puppet desc "Whether this repository is enabled, as represented by a `0` or `1`. #{ABSENT_DOC}" newvalue(:absent) { self.should = :absent } - newvalue(%r{(0|1)}) { } + newvalue(/^(0|1)$/) { } end newproperty(:gpgcheck, :parent => Puppet::IniProperty) do @@ -240,7 +238,7 @@ module Puppet from this repository, as represented by a `0` or `1`. #{ABSENT_DOC}" newvalue(:absent) { self.should = :absent } - newvalue(%r{(0|1)}) { } + newvalue(/^(0|1)$/) { } end newproperty(:gpgkey, :parent => Puppet::IniProperty) do @@ -280,7 +278,7 @@ module Puppet desc "Whether yum will allow the use of package groups for this repository, as represented by a `0` or `1`. #{ABSENT_DOC}" newvalue(:absent) { self.should = :absent } - newvalue(%r{(0|1)}) { } + newvalue(/^(0|1)$/) { } end newproperty(:failovermethod, :parent => Puppet::IniProperty) do @@ -294,7 +292,7 @@ module Puppet desc "Whether HTTP/1.1 keepalive should be used with this repository, as represented by a `0` or `1`. #{ABSENT_DOC}" newvalue(:absent) { self.should = :absent } - newvalue(%r{(0|1)}) { } + newvalue(/^(0|1)$/) { } end newproperty(:http_caching, :parent => Puppet::IniProperty) do @@ -322,7 +320,7 @@ module Puppet that the `protectbase` plugin is installed and enabled. #{ABSENT_DOC}" newvalue(:absent) { self.should = :absent } - newvalue(%r{(0|1)}) { } + newvalue(/^(0|1)$/) { } end newproperty(:priority, :parent => Puppet::IniProperty) do @@ -358,6 +356,12 @@ module Puppet newvalue(/.*/) { } end + newproperty(:s3_enabled, :parent => Puppet::IniProperty) do + desc "Access the repo via S3. #{ABSENT_DOC}" + newvalue(:absent) { self.should = :absent } + newvalue(/^(0|1)$/) { } + end + newproperty(:sslcacert, :parent => Puppet::IniProperty) do desc "Path to the directory containing the databases of the certificate authorities yum should use to verify SSL certificates.\n#{ABSENT_DOC}" diff --git a/lib/puppet/util.rb b/lib/puppet/util.rb index 2e2251fff..c1a82340e 100644 --- a/lib/puppet/util.rb +++ b/lib/puppet/util.rb @@ -11,6 +11,8 @@ require 'tempfile' require 'pathname' require 'ostruct' require 'puppet/util/platform' +require 'puppet/util/symbolic_file_mode' +require 'securerandom' module Puppet module Util @@ -22,6 +24,8 @@ module Util require 'puppet/util/posix' extend Puppet::Util::POSIX + extend Puppet::Util::SymbolicFileMode + @@sync_objects = {}.extend MonitorMixin @@ -174,7 +178,6 @@ module Util # Only benchmark if our log level is high enough if level != :none and Puppet::Util::Log.sendlevel?(level) - result = nil seconds = Benchmark.realtime { yield } @@ -396,15 +399,19 @@ module Util # # The default_mode is the mode to use when the target file doesn't already # exist; if the file is present we copy the existing mode/owner/group values - # across. + # across. The default_mode can be expressed as an octal integer, a numeric string (ie '0664') + # or a symbolic file mode. def replace_file(file, default_mode, &block) raise Puppet::DevError, "replace_file requires a block" unless block_given? + unless valid_symbolic_mode?(default_mode) + raise Puppet::DevError, "replace_file default_mode: #{default_mode} is invalid" + end + mode = symbolic_mode_to_int(normalize_symbolic_mode(default_mode)) + file = Pathname(file) tempfile = Tempfile.new(file.basename.to_s, file.dirname.to_s) - file_exists = file.exist? - # Set properties of the temporary file before we write the content, because # Tempfile doesn't promise to be safe from reading by other people, just # that it avoids races around creating the file. @@ -415,7 +422,7 @@ module Util # secure" tempfile permissions instead. Magic happens later. unless Puppet.features.microsoft_windows? # Grab the current file mode, and fall back to the defaults. - stat = file.lstat rescue OpenStruct.new(:mode => default_mode, + stat = file.lstat rescue OpenStruct.new(:mode => mode, :uid => Process.euid, :gid => Process.egid) @@ -456,7 +463,7 @@ module Util # Yes, the arguments are reversed compared to the rename in the rest # of the world. Puppet::Util::Windows::File.replace_file(file, tempfile.path) - rescue Puppet::Util::Windows::Error => e + rescue Puppet::Util::Windows::Error # This might race, but there are enough possible cases that there # isn't a good, solid "better" way to do this, and the next call # should fail in the same way anyhow. @@ -475,7 +482,7 @@ module Util end # Set the permissions to what we want. - Puppet::Util::Windows::Security.set_mode(default_mode, file.to_s) + Puppet::Util::Windows::Security.set_mode(mode, file.to_s) # ...and finally retry the operation. retry diff --git a/lib/puppet/util/autoload.rb b/lib/puppet/util/autoload.rb index 059202190..fc407a971 100644 --- a/lib/puppet/util/autoload.rb +++ b/lib/puppet/util/autoload.rb @@ -85,7 +85,6 @@ class Puppet::Util::Autoload # returns nil if no file is found def get_file(name, env=nil) name = name + '.rb' unless name =~ /\.rb$/ - dirname, base = File.split(name) path = search_directories(env).find { |dir| File.exist?(File.join(dir, name)) } path and File.join(path, name) end diff --git a/lib/puppet/util/backups.rb b/lib/puppet/util/backups.rb index cc7a997b8..b1daf78fa 100644 --- a/lib/puppet/util/backups.rb +++ b/lib/puppet/util/backups.rb @@ -19,7 +19,7 @@ module Puppet::Util::Backups def perform_backup_with_bucket(fileobj) file = (fileobj.class == String) ? fileobj : fileobj.name - case File.stat(file).ftype + case File.lstat(file).ftype when "directory" # we don't need to backup directories when recurse is on return true if self[:recurse] @@ -40,8 +40,6 @@ module Puppet::Util::Backups begin bfile = file + backup - # Ruby 1.8.1 requires the 'preserve' addition, but - # later versions do not appear to require it. # N.B. cp_r works on both files and directories FileUtils.cp_r(file, bfile, :preserve => true) return true diff --git a/lib/puppet/util/classgen.rb b/lib/puppet/util/classgen.rb index e25599b53..8785f87b3 100644 --- a/lib/puppet/util/classgen.rb +++ b/lib/puppet/util/classgen.rb @@ -36,7 +36,7 @@ module Puppet::Util::ClassGen # Creates a new module. # @param name [String] the name of the generated module - # @param optinos [Hash] hash with options + # @param options [Hash] hash with options # @option options [Array<Class>] :array if specified, the generated class is appended to this array # @option options [Hash<{String => Object}>] :attributes a hash that is applied to the generated class # by calling setter methods corresponding to this hash's keys/value pairs. This is done before the given diff --git a/lib/puppet/util/command_line/puppet_option_parser.rb b/lib/puppet/util/command_line/puppet_option_parser.rb index b80f7f679..60cf5c440 100644 --- a/lib/puppet/util/command_line/puppet_option_parser.rb +++ b/lib/puppet/util/command_line/puppet_option_parser.rb @@ -11,7 +11,7 @@ module Puppet # This is a command line option parser. It is intended to have an API that is very similar to # the ruby stdlib 'OptionParser' API, for ease of integration into our existing code... however, # However, we've removed the OptionParser-based implementation and are only maintaining the - # it's impilemented based on the third-party "trollop" library. This was done because there + # it's implemented based on the third-party "trollop" library. This was done because there # are places where the stdlib OptionParser is not flexible enough to meet our needs. class PuppetOptionParser @@ -22,8 +22,6 @@ module Puppet @parser = Trollop::Parser.new do banner usage_msg - create_default_short_options = false - handle_help_and_version = false end end diff --git a/lib/puppet/util/command_line/trollop.rb b/lib/puppet/util/command_line/trollop.rb index 57ff89320..ed1246c0e 100644 --- a/lib/puppet/util/command_line/trollop.rb +++ b/lib/puppet/util/command_line/trollop.rb @@ -454,7 +454,7 @@ class Parser # chronic is not available end time ? Date.new(time.year, time.month, time.day) : Date.parse(param) - rescue ArgumentError => e + rescue ArgumentError raise CommandlineError, "option '#{arg}' needs a date" end end diff --git a/lib/puppet/util/constant_inflector.rb b/lib/puppet/util/constant_inflector.rb index ff440ba34..960f7c796 100644 --- a/lib/puppet/util/constant_inflector.rb +++ b/lib/puppet/util/constant_inflector.rb @@ -11,8 +11,7 @@ module Puppet module Util module ConstantInflector def file2constant(file) - # LAK:NOTE See http://snurl.com/21zf8 [groups_google_com] - x = file.split("/").collect { |name| name.capitalize }.join("::").gsub(/_+(.)/) { |term| $1.capitalize } + file.split("/").collect { |name| name.capitalize }.join("::").gsub(/_+(.)/) { |term| $1.capitalize } end module_function :file2constant diff --git a/lib/puppet/util/errors.rb b/lib/puppet/util/errors.rb index 3a0500dd5..a03e3a8bb 100644 --- a/lib/puppet/util/errors.rb +++ b/lib/puppet/util/errors.rb @@ -22,6 +22,7 @@ module Puppet::Util::Errors def adderrorcontext(error, other = nil) error.line ||= self.line if error.respond_to?(:line=) and self.respond_to?(:line) and self.line error.file ||= self.file if error.respond_to?(:file=) and self.respond_to?(:file) and self.file + error.original ||= other if error.respond_to?(:original=) error.set_backtrace(other.backtrace) if other and other.respond_to?(:backtrace) diff --git a/lib/puppet/util/file_watcher.rb b/lib/puppet/util/file_watcher.rb new file mode 100644 index 000000000..fda994d2d --- /dev/null +++ b/lib/puppet/util/file_watcher.rb @@ -0,0 +1,28 @@ +class Puppet::Util::FileWatcher + include Enumerable + + def each(&blk) + @files.keys.each(&blk) + end + + def initialize + @files = {} + end + + def changed? + @files.values.any?(&:changed?) + end + + def watch(filename) + return if watching?(filename) + @files[filename] = Puppet::Util::WatchedFile.new(filename) + end + + def watching?(filename) + @files.has_key?(filename) + end + + def clear + @files.clear + end +end diff --git a/lib/puppet/util/fileparsing.rb b/lib/puppet/util/fileparsing.rb index 84344c96b..e19bb365b 100644 --- a/lib/puppet/util/fileparsing.rb +++ b/lib/puppet/util/fileparsing.rb @@ -161,7 +161,6 @@ module Puppet::Util::FileParsing # In this case, we try to match the whole line and then use the # match captures to get our fields. if match = regex.match(line) - fields = [] ret = {} record.fields.zip(match.captures).each do |field, value| if value == record.absent @@ -216,8 +215,7 @@ module Puppet::Util::FileParsing # Split text into separate lines using the record separator. def lines(text) # Remove any trailing separators, and then split based on them - # LAK:NOTE See http://snurl.com/21zf8 [groups_google_com] - x = text.sub(/#{self.line_separator}\Q/,'').split(self.line_separator) + text.sub(/#{self.line_separator}\Q/,'').split(self.line_separator) end # Split a bunch of text into lines and then parse them individually. diff --git a/lib/puppet/util/filetype.rb b/lib/puppet/util/filetype.rb index 40278f1df..c65f8026e 100755 --- a/lib/puppet/util/filetype.rb +++ b/lib/puppet/util/filetype.rb @@ -203,7 +203,6 @@ class Puppet::Util::FileType # Only add the -u flag when the @path is different. Fedora apparently # does not think I should be allowed to set the @path to my own user name def cmdbase - cmd = nil if @uid == Puppet::Util::SUIDManager.uid || Facter.value(:operatingsystem) == "HP-UX" return "crontab" else diff --git a/lib/puppet/util/graph.rb b/lib/puppet/util/graph.rb deleted file mode 100644 index 58ca1ab4d..000000000 --- a/lib/puppet/util/graph.rb +++ /dev/null @@ -1,27 +0,0 @@ -require 'puppet' -require 'puppet/simple_graph' - -# A module that handles the small amount of graph stuff in Puppet. -module Puppet::Util::Graph - # Make a graph where each of our children gets converted to - # the receiving end of an edge. Call the same thing on all - # of our children, optionally using a block - def to_graph(graph = nil, &block) - # Allow our calling function to send in a graph, so that we - # can call this recursively with one graph. - graph ||= Puppet::SimpleGraph.new - - self.each do |child| - unless block_given? and ! yield(child) - graph.add_edge(self, child) - - child.to_graph(graph, &block) if child.respond_to?(:to_graph) - end - end - - # Do a topsort, which will throw an exception if the graph is cyclic. - - graph - end -end - diff --git a/lib/puppet/util/http_proxy.rb b/lib/puppet/util/http_proxy.rb new file mode 100644 index 000000000..8c979c400 --- /dev/null +++ b/lib/puppet/util/http_proxy.rb @@ -0,0 +1,38 @@ +module Puppet::Util::HttpProxy + + def self.http_proxy_env + # Returns a URI object if proxy is set, or nil + proxy_env = ENV["http_proxy"] || ENV["HTTP_PROXY"] + begin + return URI.parse(proxy_env) if proxy_env + rescue URI::InvalidURIError + return nil + end + return nil + end + + def self.http_proxy_host + env = self.http_proxy_env + + if env and env.host then + return env.host + end + + if Puppet.settings[:http_proxy_host] == 'none' + return nil + end + + return Puppet.settings[:http_proxy_host] + end + + def self.http_proxy_port + env = self.http_proxy_env + + if env and env.port then + return env.port + end + + return Puppet.settings[:http_proxy_port] + end + +end diff --git a/lib/puppet/util/ldap/manager.rb b/lib/puppet/util/ldap/manager.rb index 2ccd102bc..9df184968 100644 --- a/lib/puppet/util/ldap/manager.rb +++ b/lib/puppet/util/ldap/manager.rb @@ -109,14 +109,13 @@ class Puppet::Util::Ldap::Manager # Find the associated entry for a resource. Returns a hash, minus # 'dn', or nil if the entry cannot be found. def find(name) - result = nil connect do |conn| begin conn.search2(dn(name), 0, "objectclass=*") do |result| # Convert to puppet-appropriate attributes return entry2provider(result) end - rescue => detail + rescue return nil end end diff --git a/lib/puppet/util/loadedfile.rb b/lib/puppet/util/loadedfile.rb deleted file mode 100755 index d2f5d0923..000000000 --- a/lib/puppet/util/loadedfile.rb +++ /dev/null @@ -1,61 +0,0 @@ -# A simple class that tells us when a file has changed and thus whether we -# should reload it - -require 'puppet' - -module Puppet - class NoSuchFile < Puppet::Error; end - class Util::LoadedFile - attr_reader :file, :statted - - # Provide a hook for setting the timestamp during testing, so we don't - # have to depend on the granularity of the filesystem. - attr_writer :tstamp - - # Determine whether the file has changed and thus whether it should - # be reparsed. - def changed? - # Allow the timeout to be disabled entirely. - return true if Puppet[:filetimeout] < 0 - tmp = stamp - - # We use a different internal variable than the stamp method - # because it doesn't keep historical state and we do -- that is, - # we will always be comparing two timestamps, whereas - # stamp just always wants the latest one. - if tmp == @tstamp - return false - else - @tstamp = tmp - return @tstamp - end - end - - # Create the file. Must be passed the file path. - def initialize(file) - @file = file - @statted = 0 - @stamp = nil - @tstamp = stamp - end - - # Retrieve the filestamp, but only refresh it if we're beyond our - # filetimeout - def stamp - if @stamp.nil? or (Time.now.to_i - @statted >= Puppet[:filetimeout]) - @statted = Time.now.to_i - begin - @stamp = File.stat(@file).ctime - rescue Errno::ENOENT, Errno::ENOTDIR - @stamp = Time.now - end - end - @stamp - end - - def to_s - @file - end - end -end - diff --git a/lib/puppet/util/log.rb b/lib/puppet/util/log.rb index 20b839322..3d34bba2c 100644 --- a/lib/puppet/util/log.rb +++ b/lib/puppet/util/log.rb @@ -142,6 +142,19 @@ class Puppet::Util::Log end end + def Log.with_destination(destination, &block) + if @destinations.include?(destination) + yield + else + newdestination(destination) + begin + yield + ensure + close(destination) + end + end + end + # Route the actual message. FIXME There are lots of things this method # should do, like caching and a bit more. It's worth noting that there's # a potential for a loop here, if the machine somehow gets the destination set as @@ -261,6 +274,18 @@ class Puppet::Util::Log @line = data['line'] if data['line'] end + def to_pson + { + 'level' => @level, + 'message' => @message, + 'source' => @source, + 'tags' => @tags, + 'time' => @time.iso8601(9), + 'file' => @file, + 'line' => @line, + }.to_pson + end + def message=(msg) raise ArgumentError, "Puppet::Util::Log requires a message" unless msg @message = msg.to_s @@ -279,16 +304,11 @@ class Puppet::Util::Log # If they pass a source in to us, we make sure it is a string, and # we retrieve any tags we can. def source=(source) - if source.respond_to?(:source_descriptors) - descriptors = source.source_descriptors - @source = descriptors[:path] - - descriptors[:tags].each { |t| tag(t) } - - [:file, :line].each do |param| - next unless descriptors[param] - send(param.to_s + "=", descriptors[param]) - end + if source.respond_to?(:path) + @source = source.path + source.tags.each { |t| tag(t) } + self.file = source.file + self.line = source.line else @source = source.to_s end @@ -301,6 +321,7 @@ class Puppet::Util::Log def to_s message end + end # This is for backward compatibility from when we changed the constant to Puppet::Util::Log diff --git a/lib/puppet/util/log/destinations.rb b/lib/puppet/util/log/destinations.rb index c7fee11b9..e33cfd9a8 100644 --- a/lib/puppet/util/log/destinations.rb +++ b/lib/puppet/util/log/destinations.rb @@ -126,56 +126,6 @@ Puppet::Util::Log.newdesttype :console do end end -Puppet::Util::Log.newdesttype :host do - def initialize(host) - Puppet.info "Treating #{host} as a hostname" - args = {} - if host =~ /:(\d+)/ - args[:Port] = $1 - args[:Server] = host.sub(/:\d+/, '') - else - args[:Server] = host - end - - @name = host - - @driver = Puppet::Network::Client::LogClient.new(args) - end - - def handle(msg) - unless msg.is_a?(String) or msg.remote - @hostname ||= Facter["hostname"].value - unless defined?(@domain) - @domain = Facter["domain"].value - @hostname += ".#{@domain}" if @domain - end - if Puppet::Util.absolute_path?(msg.source) - msg.source = @hostname + ":#{msg.source}" - elsif msg.source == "Puppet" - msg.source = @hostname + " #{msg.source}" - else - msg.source = @hostname + " #{msg.source}" - end - begin - #puts "would have sent #{msg}" - #puts "would have sent %s" % - # CGI.escape(YAML.dump(msg)) - begin - tmp = CGI.escape(YAML.dump(msg)) - rescue => detail - puts "Could not dump: #{detail}" - return - end - # Add the hostname to the source - @driver.addlog(tmp) - rescue => detail - Puppet.log_exception(detail) - Puppet::Util::Log.close(self) - end - end - end -end - # Log to a transaction report. Puppet::Util::Log.newdesttype :report do attr_reader :report diff --git a/lib/puppet/util/log_paths.rb b/lib/puppet/util/log_paths.rb deleted file mode 100644 index ebca95355..000000000 --- a/lib/puppet/util/log_paths.rb +++ /dev/null @@ -1,22 +0,0 @@ -module Puppet::Util::LogPaths - # return the full path to us, for logging and rollback - # some classes (e.g., FileTypeRecords) will have to override this - def path - @path ||= '/' + pathbuilder.join('/') - end - - def source_descriptors - descriptors = {} - - descriptors[:tags] = tags - - [:path, :file, :line].each do |param| - next unless value = send(param) - descriptors[param] = value - end - - descriptors - end - -end - diff --git a/lib/puppet/util/metric.rb b/lib/puppet/util/metric.rb index 8e0bc0347..49d4edfc3 100644 --- a/lib/puppet/util/metric.rb +++ b/lib/puppet/util/metric.rb @@ -15,6 +15,14 @@ class Puppet::Util::Metric metric end + def to_pson + { + 'name' => @name, + 'label' => @label, + 'values' => @values + }.to_pson + end + # Return a specific value def [](name) if value = @values.find { |v| v[0] == name } @@ -85,7 +93,6 @@ class Puppet::Util::Metric args.push("--title",self.label) args.push("--imgformat","PNG") args.push("--interlace") - i = 0 defs = [] lines = [] #p @values.collect { |s,l| s } diff --git a/lib/puppet/util/monkey_patches.rb b/lib/puppet/util/monkey_patches.rb index 7b1bac58f..194331c01 100644 --- a/lib/puppet/util/monkey_patches.rb +++ b/lib/puppet/util/monkey_patches.rb @@ -1,4 +1,3 @@ -require 'puppet/util' module Puppet::Util::MonkeyPatches end @@ -67,53 +66,6 @@ class Object end end -# Workaround for yaml_initialize, which isn't supported before Ruby -# 1.8.3. -if RUBY_VERSION == '1.8.1' || RUBY_VERSION == '1.8.2' - YAML.add_ruby_type( /^object/ ) { |tag, val| - type, obj_class = YAML.read_type_class( tag, Object ) - r = YAML.object_maker( obj_class, val ) - if r.respond_to? :yaml_initialize - r.instance_eval { instance_variables.each { |name| remove_instance_variable name } } - r.yaml_initialize(tag, val) - end - r - } -end - -class Fixnum - # Returns the int itself. This method is intended for compatibility to - # character constant in Ruby 1.9. 1.8.5 is missing it; add it. - def ord - self - end unless method_defined? 'ord' -end - -class Array - # Ruby < 1.8.7 doesn't have this method but we use it in tests - def combination(num) - return [] if num < 0 || num > size - return [[]] if num == 0 - return map{|e| [e] } if num == 1 - tmp = self.dup - self[0, size - (num - 1)].inject([]) do |ret, e| - tmp.shift - ret += tmp.combination(num - 1).map{|a| a.unshift(e) } - end - end unless method_defined? :combination - - alias :count :length unless method_defined? :count - - # Ruby 1.8.5 lacks `drop`, which we don't want to lose. - def drop(n) - n = n.to_int - raise ArgumentError, "attempt to drop negative size" if n < 0 - - slice(n, length - n) or [] - end unless method_defined? :drop -end - - class Symbol # So, it turns out that one of the biggest memory allocation hot-spots in # our code was using symbol-to-proc - because it allocated a new instance @@ -187,6 +139,10 @@ class IO end unless singleton_methods.include?(:binwrite) end +class Float + INFINITY = (1.0/0.0) if defined?(Float::INFINITY).nil? +end + class Range def intersection(other) raise ArgumentError, 'value must be a Range' unless other.kind_of?(Range) @@ -205,15 +161,6 @@ class Range alias_method :&, :intersection unless method_defined? :& end -# Ruby 1.8.5 doesn't have tap -module Kernel - def tap - yield(self) - self - end unless method_defined?(:tap) -end - - ######################################################################## # The return type of `instance_variables` changes between Ruby 1.8 and 1.9 # releases; it used to return an array of strings in the form "@foo", but @@ -266,97 +213,6 @@ if RUBY_VERSION[0,3] == '1.8' end end -# The mv method in Ruby 1.8.5 can't mv directories across devices -# File.rename causes "Invalid cross-device link", which is rescued, but in Ruby -# 1.8.5 it tries to recover with a copy and unlink, but the unlink causes the -# error "Is a directory". In newer Rubies remove_entry is used -# The implementation below is what's used in Ruby 1.8.7 and Ruby 1.9 -if RUBY_VERSION == '1.8.5' - require 'fileutils' - - module FileUtils - def mv(src, dest, options = {}) - fu_check_options options, OPT_TABLE['mv'] - fu_output_message "mv#{options[:force] ? ' -f' : ''} #{[src,dest].flatten.join ' '}" if options[:verbose] - return if options[:noop] - fu_each_src_dest(src, dest) do |s, d| - destent = Entry_.new(d, nil, true) - begin - if destent.exist? - if destent.directory? - raise Errno::EEXIST, dest - else - destent.remove_file if rename_cannot_overwrite_file? - end - end - begin - File.rename s, d - rescue Errno::EXDEV - copy_entry s, d, true - if options[:secure] - remove_entry_secure s, options[:force] - else - remove_entry s, options[:force] - end - end - rescue SystemCallError - raise unless options[:force] - end - end - end - module_function :mv - - alias move mv - module_function :move - end -end - -# Ruby 1.8.6 doesn't have it either -# From https://github.com/puppetlabs/hiera/pull/47/files: -# In ruby 1.8.5 Dir does not have mktmpdir defined, so this monkey patches -# Dir to include the 1.8.7 definition of that method if it isn't already defined. -# Method definition borrowed from ruby-1.8.7-p357/lib/ruby/1.8/tmpdir.rb -unless Dir.respond_to?(:mktmpdir) - def Dir.mktmpdir(prefix_suffix=nil, tmpdir=nil) - case prefix_suffix - when nil - prefix = "d" - suffix = "" - when String - prefix = prefix_suffix - suffix = "" - when Array - prefix = prefix_suffix[0] - suffix = prefix_suffix[1] - else - raise ArgumentError, "unexpected prefix_suffix: #{prefix_suffix.inspect}" - end - tmpdir ||= Dir.tmpdir - t = Time.now.strftime("%Y%m%d") - n = nil - begin - path = "#{tmpdir}/#{prefix}#{t}-#{$$}-#{rand(0x100000000).to_s(36)}" - path << "-#{n}" if n - path << suffix - Dir.mkdir(path, 0700) - rescue Errno::EEXIST - n ||= 0 - n += 1 - retry - end - - if block_given? - begin - yield path - ensure - FileUtils.remove_entry_secure path - end - else - path - end - end -end - # (#19151) Reject all SSLv2 ciphers and handshakes require 'openssl' class OpenSSL::SSL::SSLContext @@ -399,3 +255,13 @@ if Puppet::Util::Platform.windows? end end +# Older versions of SecureRandom (e.g. in 1.8.7) don't have the uuid method +module SecureRandom + def self.uuid + # Copied from the 1.9.1 stdlib implementation of uuid + ary = self.random_bytes(16).unpack("NnnnnN") + ary[2] = (ary[2] & 0x0fff) | 0x4000 + ary[3] = (ary[3] & 0x3fff) | 0x8000 + "%08x-%04x-%04x-%04x-%04x%08x" % ary + end unless singleton_methods.include?(:uuid) +end diff --git a/lib/puppet/util/network_device/cisco/facts.rb b/lib/puppet/util/network_device/cisco/facts.rb index 658c0abd0..08943faa1 100644 --- a/lib/puppet/util/network_device/cisco/facts.rb +++ b/lib/puppet/util/network_device/cisco/facts.rb @@ -64,7 +64,7 @@ class Puppet::Util::NetworkDevice::Cisco::Facts def uptime_to_seconds(uptime) captures = (uptime.match /^(?:(\d+) years?,)?\s*(?:(\d+) weeks?,)?\s*(?:(\d+) days?,)?\s*(?:(\d+) hours?,)?\s*(\d+) minutes?$/).captures - seconds = captures.zip([31536000, 604800, 86400, 3600, 60]).inject(0) do |total, (x,y)| + captures.zip([31536000, 604800, 86400, 3600, 60]).inject(0) do |total, (x,y)| total + (x.nil? ? 0 : x.to_i * y) end end diff --git a/lib/puppet/util/network_device/config.rb b/lib/puppet/util/network_device/config.rb index cf28761ba..ef49dd393 100644 --- a/lib/puppet/util/network_device/config.rb +++ b/lib/puppet/util/network_device/config.rb @@ -1,8 +1,8 @@ require 'ostruct' -require 'puppet/util/loadedfile' +require 'puppet/util/watched_file' require 'puppet/util/network_device' -class Puppet::Util::NetworkDevice::Config < Puppet::Util::LoadedFile +class Puppet::Util::NetworkDevice::Config def self.main @main ||= self.new @@ -18,12 +18,9 @@ class Puppet::Util::NetworkDevice::Config < Puppet::Util::LoadedFile FileTest.exists?(@file) end - def initialize() - @file = Puppet[:deviceconfig] + def initialize + @file = Puppet::Util::WatchedFile.new(Puppet[:deviceconfig]) - raise Puppet::DevError, "No device config file defined" unless @file - return unless self.exists? - super(@file) @devices = {} read(true) # force reading at start @@ -31,9 +28,9 @@ class Puppet::Util::NetworkDevice::Config < Puppet::Util::LoadedFile # Read the configuration file. def read(force = false) - return unless FileTest.exists?(@file) + return unless exists? - parse if force or changed? + parse if force or @file.changed? end private diff --git a/lib/puppet/util/network_device/transport/ssh.rb b/lib/puppet/util/network_device/transport/ssh.rb index daee1d3a3..cc613bda0 100644 --- a/lib/puppet/util/network_device/transport/ssh.rb +++ b/lib/puppet/util/network_device/transport/ssh.rb @@ -36,7 +36,7 @@ class Puppet::Util::NetworkDevice::Transport::Ssh < Puppet::Util::NetworkDevice: raise TimeoutError, "timed out while opening an ssh connection to the host" rescue Net::SSH::AuthenticationFailed raise Puppet::Error, "SSH authentication failure connecting to #{host} as #{user}" - rescue Net::SSH::Exception => detail + rescue Net::SSH::Exception raise Puppet::Error, "SSH connection failure to #{host}" end diff --git a/lib/puppet/util/pidlock.rb b/lib/puppet/util/pidlock.rb index 33c6bf14f..35e4ad431 100644 --- a/lib/puppet/util/pidlock.rb +++ b/lib/puppet/util/pidlock.rb @@ -34,6 +34,9 @@ class Puppet::Util::Pidlock @lockfile.lock_data.to_i end + def file_path + @lockfile.file_path + end def clear_if_stale return if lock_pid.nil? diff --git a/lib/puppet/util/posix.rb b/lib/puppet/util/posix.rb index cf4c24e48..2af0e4b91 100755 --- a/lib/puppet/util/posix.rb +++ b/lib/puppet/util/posix.rb @@ -31,7 +31,7 @@ module Puppet::Util::POSIX begin return Etc.send(method, id).send(field) - rescue NoMethodError, ArgumentError => detail + rescue NoMethodError, ArgumentError # ignore it; we couldn't find the object return nil end diff --git a/lib/puppet/util/profiler.rb b/lib/puppet/util/profiler.rb index 52f8745b7..0c3a3768f 100644 --- a/lib/puppet/util/profiler.rb +++ b/lib/puppet/util/profiler.rb @@ -10,7 +10,7 @@ module Puppet::Util::Profiler NONE = Puppet::Util::Profiler::None.new - # @returns This thread's configured profiler + # @return This thread's configured profiler def self.current Thread.current[:profiler] || NONE end diff --git a/lib/puppet/util/rdoc.rb b/lib/puppet/util/rdoc.rb index 0f683b74d..becfa4ba3 100644 --- a/lib/puppet/util/rdoc.rb +++ b/lib/puppet/util/rdoc.rb @@ -45,7 +45,7 @@ module Puppet::Util::RDoc end end - # launch a output to console manifest doc + # launch an output to console manifest doc def manifestdoc(files) Puppet[:ignoreimport] = true files.select { |f| FileTest.file?(f) }.each do |f| diff --git a/lib/puppet/util/rdoc/generators/puppet_generator.rb b/lib/puppet/util/rdoc/generators/puppet_generator.rb index 249c9a8ba..5c6aca28e 100644 --- a/lib/puppet/util/rdoc/generators/puppet_generator.rb +++ b/lib/puppet/util/rdoc/generators/puppet_generator.rb @@ -795,7 +795,6 @@ module Generators end @values["title"] = "#{@values['classmod']}: #{h_name}" - c = @context @values["full_name"] = h_name files = [] diff --git a/lib/puppet/util/rdoc/generators/template/puppet/puppet.rb b/lib/puppet/util/rdoc/generators/template/puppet/puppet.rb index e03381f22..6a1329ae6 100644 --- a/lib/puppet/util/rdoc/generators/template/puppet/puppet.rb +++ b/lib/puppet/util/rdoc/generators/template/puppet/puppet.rb @@ -25,21 +25,37 @@ module RDoc FONTS = "Verdana,Arial,Helvetica,sans-serif" STYLE = %{ +/* Reset */ +html,body,div,span,applet,object,iframe,h1,h2,h3,h4,h5,h6,p,blockquote,pre,a,abbr,acronym,address,big,cite,code,del,dfn,em,font,img,ins,kbd,q,s,samp,small,strike,strong,sub,sup,tt,var,dl,dt,dd,ol,ul,li,fieldset,form,label,legend,table,caption,tbody,tfoot,thead,tr,th,td{margin:0;padding:0;border:0;outline:0;font-weight:inherit;font-style:inherit;font-size:100%;font-family:inherit;vertical-align:baseline;} +:focus{outline:0;} +body{line-height:1;color:#282828;background:#fff;} +ol,ul{list-style:none;} +table{border-collapse:separate;border-spacing:0;} +caption,th,td{text-align:left;font-weight:normal;} +blockquote:before,blockquote:after,q:before,q:after{content:"";} +blockquote,q{quotes:"""";} + body { font-family: Verdana,Arial,Helvetica,sans-serif; - font-size: 90%; - margin: 0; - margin-left: 40px; - padding: 0; - background: white; + font-size: 0.9em; } +pre { + background: none repeat scroll 0 0 #F7F7F7; + border: 1px dashed #DDDDDD; + color: #555555; + font-family: courier; + margin: 10px 19px; + padding: 10px; + } + h1,h2,h3,h4 { margin: 0; color: #efefef; background: transparent; } -h1 { font-size: 150%; } -h2,h3,h4 { margin-top: 1em; } +h1 { font-size: 1.2em; } +h2,h3,h4 { margin-top: 1em; color:#558; } +h2,h3 { font-size: 1.1em; } -a { background: #eef; color: #039; text-decoration: none; } -a:hover { background: #039; color: #eef; } +a { color: #037; text-decoration: none; } +a:hover { color: #04d; } /* Override the base stylesheet's Anchor inside a table cell */ td > a { @@ -58,32 +74,26 @@ td > a { /* === Structural elements =================================== */ div#index { - margin: 0; - margin-left: -40px; padding: 0; - font-size: 90%; } div#index a { - margin-left: 0.7em; + display:inline-block; + padding:2px 10px; } + div#index .section-bar { - margin-left: 0px; - padding-left: 0.7em; - background: #ccc; - font-size: small; + background: #ffe; + padding:10px; } div#classHeader, div#fileHeader { - width: auto; - color: white; - padding: 0.5em 1.5em 0.5em 1.5em; - margin: 0; - margin-left: -40px; - border-bottom: 3px solid #006; + border-bottom: 1px solid #ddd; + padding:10px; + font-size:0.9em; } div#classHeader a, div#fileHeader a { @@ -92,8 +102,9 @@ div#classHeader a, div#fileHeader a { } div#classHeader td, div#fileHeader td { - background: inherit; color: white; + padding:3px; + font-size:0.9em; } @@ -110,19 +121,19 @@ div#nodeHeader { } .class-name-in-header { - font-size: 180%; font-weight: bold; } div#bodyContent { - padding: 0 1.5em 0 1.5em; + padding: 10px; } div#description { - padding: 0.5em 1.5em; - background: #efefef; - border: 1px dotted #999; + padding: 10px; + background: #f5f5f5; + border: 1px dotted #ddd; + line-height:1.2em; } div#description h1,h2,h3,h4,h5,h6 { @@ -165,17 +176,18 @@ table.header-table { .section-bar { color: #333; - border-bottom: 1px solid #999; - margin-left: -20px; + border-bottom: 1px solid #ddd; + padding:10px 0; + margin:5px 0 10px 0; } +div#class-list, div#methods, div#includes, div#resources, div#requires, div#realizes, div#attribute-list { padding:10px; } .section-title { background: #79a; color: #eee; padding: 3px; margin-top: 2em; - margin-left: -30px; border: 1px solid #999; } @@ -191,22 +203,18 @@ table.header-table { /* --- Method classes -------------------------- */ .method-detail { - background: #efefef; - padding: 0; - margin-top: 0.5em; - margin-bottom: 1em; - border: 1px dotted #ccc; + background: #f5f5f5; } .method-heading { - color: black; - background: #ccc; - border-bottom: 1px solid #666; - padding: 0.2em 0.5em 0 0.5em; + color: #333; + font-style:italic; + background: #ddd; + padding:5px 10px; } .method-signature { color: black; background: inherit; } .method-name { font-weight: bold; } .method-args { font-style: italic; } -.method-description { padding: 0 0.5em 0 0.5em; } +.method-description { padding: 10px 10px 20px 10px; } /* --- Source code sections -------------------- */ diff --git a/lib/puppet/util/retryaction.rb b/lib/puppet/util/retryaction.rb index ba318ec1a..bd578c147 100644 --- a/lib/puppet/util/retryaction.rb +++ b/lib/puppet/util/retryaction.rb @@ -14,7 +14,6 @@ module Puppet::Util::RetryAction raise RetryException::NoRetriesGiven if parameters[:retries].nil? parameters[:retry_exceptions] ||= Hash.new - start = Time.now failures = 0 begin diff --git a/lib/puppet/util/subclass_loader.rb b/lib/puppet/util/subclass_loader.rb deleted file mode 100644 index 7d943745e..000000000 --- a/lib/puppet/util/subclass_loader.rb +++ /dev/null @@ -1,78 +0,0 @@ -# A module for loading subclasses into an array and retrieving -# them by name. Also sets up a method for each class so -# that you can just do Klass.subclass, rather than Klass.subclass(:subclass). -# -# This module is currently used by network handlers and clients. -module Puppet::Util::SubclassLoader - attr_accessor :loader, :classloader - - # Iterate over each of the subclasses. - def each - @subclasses ||= [] - @subclasses.each { |c| yield c } - end - - # The hook method that sets up subclass loading. We need the name - # of the method to create and the path in which to look for them. - def handle_subclasses(name, path) - raise ArgumentError, "Must be a class to use SubclassLoader" unless self.is_a?(Class) - @subclasses = [] - - @loader = Puppet::Util::Autoload.new(self, path, :wrap => false) - - @subclassname = name - - @classloader = self - - # Now create a method for retrieving these subclasses by name. Note - # that we're defining a class method here, not an instance. - meta_def(name) do |subname| - subname = subname.to_s.downcase - - unless c = @subclasses.find { |c| c.name.to_s.downcase == subname } - loader.load(subname) - c = @subclasses.find { |c| c.name.to_s.downcase == subname } - - # Now make the method that returns this subclass. This way we - # normally avoid the method_missing method. - define_method(subname) { c } if c and ! respond_to?(subname) - end - return c - end - end - - # Add a new class to our list. Note that this has to handle subclasses of - # subclasses, thus the reason we're keeping track of the @@classloader. - def inherited(sub) - @subclasses ||= [] - sub.classloader = self.classloader - if self.classloader == self - @subclasses << sub - else - @classloader.inherited(sub) - end - end - - # See if we can load a class. - def method_missing(method, *args) - unless self == self.classloader - super - end - return nil unless defined?(@subclassname) - self.send(@subclassname, method) || nil - end - - # Retrieve or calculate a name. - def name(dummy_argument=:work_arround_for_ruby_GC_bug) - @name ||= self.to_s.sub(/.+::/, '').intern - - @name - end - - # Provide a list of all subclasses. - def subclasses - @loader.loadall - @subclasses.collect { |klass| klass.name } - end -end - diff --git a/lib/puppet/util/symbolic_file_mode.rb b/lib/puppet/util/symbolic_file_mode.rb index de07b061a..970dededc 100644 --- a/lib/puppet/util/symbolic_file_mode.rb +++ b/lib/puppet/util/symbolic_file_mode.rb @@ -1,6 +1,8 @@ require 'puppet/util' -module Puppet::Util::SymbolicFileMode +module Puppet +module Util +module SymbolicFileMode SetUIDBit = ReadBit = 4 SetGIDBit = WriteBit = 2 StickyBit = ExecBit = 1 @@ -138,3 +140,5 @@ module Puppet::Util::SymbolicFileMode return result end end +end +end diff --git a/lib/puppet/util/tagging.rb b/lib/puppet/util/tagging.rb index 80e45b305..4161c0291 100644 --- a/lib/puppet/util/tagging.rb +++ b/lib/puppet/util/tagging.rb @@ -62,8 +62,6 @@ module Puppet::Util::Tagging private def handle_qualified_tags(qualified) - # LAK:NOTE See http://snurl.com/21zf8 [groups_google_com] - x = 1 qualified.each do |name| name.split("::").each do |tag| @tags << tag unless @tags.include?(tag) diff --git a/lib/puppet/util/warnings.rb b/lib/puppet/util/warnings.rb index 7e26feaa0..82e80e4be 100644 --- a/lib/puppet/util/warnings.rb +++ b/lib/puppet/util/warnings.rb @@ -6,6 +6,9 @@ module Puppet::Util::Warnings Puppet::Util::Warnings.maybe_log(msg, self.class) { Puppet.notice msg } end + def debug_once(msg) + Puppet::Util::Warnings.maybe_log(msg, self.class) { Puppet.debug msg } + end def warnonce(msg) Puppet::Util::Warnings.maybe_log(msg, self.class) { Puppet.warning msg } diff --git a/lib/puppet/util/watched_file.rb b/lib/puppet/util/watched_file.rb new file mode 100755 index 000000000..6b28ab402 --- /dev/null +++ b/lib/puppet/util/watched_file.rb @@ -0,0 +1,37 @@ +require 'puppet/util/watcher' + +# Monitor a given file for changes on a periodic interval. Changes are detected +# by looking for a change in the file ctime. +class Puppet::Util::WatchedFile + # @!attribute [r] filename + # @return [String] The fully qualified path to the file. + attr_reader :filename + + # @param filename [String] The fully qualified path to the file. + # @param file_timeout [Integer] The polling interval for checking for file + # changes. Setting the timeout to a negative value will treat the file as + # always changed. Defaults to `Puppet[:filetimeout]` + def initialize(filename, timer = Puppet::Util::Watcher::Timer.new(Puppet[:filetimeout])) + @filename = filename + @timer = timer + + @info = Puppet::Util::Watcher::PeriodicWatcher.new( + Puppet::Util::Watcher::Common.file_ctime_change_watcher(@filename), + timer) + end + + # @return [true, false] If the file has changed since it was last checked. + def changed? + @info.changed? + end + + # Allow this to be used as the name of the file being watched in various + # other methods (such as File.exist?) + def to_str + @filename + end + + def to_s + "<WatchedFile: filename = #{@filename}, timeout = #{@timer.timeout}>" + end +end diff --git a/lib/puppet/util/watcher.rb b/lib/puppet/util/watcher.rb new file mode 100644 index 000000000..78b21f8af --- /dev/null +++ b/lib/puppet/util/watcher.rb @@ -0,0 +1,17 @@ +module Puppet::Util::Watcher + require 'puppet/util/watcher/timer' + require 'puppet/util/watcher/change_watcher' + require 'puppet/util/watcher/periodic_watcher' + + module Common + def self.file_ctime_change_watcher(filename) + Puppet::Util::Watcher::ChangeWatcher.watch(lambda do + begin + File.stat(filename).ctime + rescue Errno::ENOENT, Errno::ENOTDIR + :absent + end + end) + end + end +end diff --git a/lib/puppet/util/watcher/change_watcher.rb b/lib/puppet/util/watcher/change_watcher.rb new file mode 100644 index 000000000..1ce300261 --- /dev/null +++ b/lib/puppet/util/watcher/change_watcher.rb @@ -0,0 +1,33 @@ +# Watches for changes over time. It only re-examines the values when it is requested to update readings. +# @api private +class Puppet::Util::Watcher::ChangeWatcher + def self.watch(reader) + Puppet::Util::Watcher::ChangeWatcher.new(nil, nil, reader).next_reading + end + + def initialize(previous, current, value_reader) + @previous = previous + @current = current + @value_reader = value_reader + end + + def changed? + if uncertain? + false + else + @previous != @current + end + end + + def uncertain? + @previous.nil? || @current.nil? + end + + def change_current_reading_to(new_value) + Puppet::Util::Watcher::ChangeWatcher.new(@current, new_value, @value_reader) + end + + def next_reading + change_current_reading_to(@value_reader.call) + end +end diff --git a/lib/puppet/util/watcher/periodic_watcher.rb b/lib/puppet/util/watcher/periodic_watcher.rb new file mode 100644 index 000000000..0fe7ed556 --- /dev/null +++ b/lib/puppet/util/watcher/periodic_watcher.rb @@ -0,0 +1,37 @@ +# Monitor a given watcher for changes on a periodic interval. +class Puppet::Util::Watcher::PeriodicWatcher + # @param watcher [Puppet::Util::Watcher::ChangeWatcher] a watcher for the value to watch + # @param timer [Puppet::Util::Watcher::Timer] A timer to determin when to + # recheck the watcher. If the timout of the timer is negative, then the + # watched value is always considered to be changed + def initialize(watcher, timer) + @watcher = watcher + @timer = timer + + @timer.start + end + + # @return [true, false] If the file has changed since it was last checked. + def changed? + return true if always_consider_changed? + + @watcher = examine_watched_info(@watcher) + @watcher.changed? + end + + private + + def always_consider_changed? + @timer.timeout < 0 + end + + def examine_watched_info(known) + if @timer.expired? + @timer.start + known.next_reading + else + known + end + end +end + diff --git a/lib/puppet/util/watcher/timer.rb b/lib/puppet/util/watcher/timer.rb new file mode 100644 index 000000000..329b3ab10 --- /dev/null +++ b/lib/puppet/util/watcher/timer.rb @@ -0,0 +1,19 @@ +class Puppet::Util::Watcher::Timer + attr_reader :timeout + + def initialize(timeout) + @timeout = timeout + end + + def start + @start_time = now + end + + def expired? + (now - @start_time) >= @timeout + end + + def now + Time.now.to_i + end +end diff --git a/lib/puppet/util/windows/user.rb b/lib/puppet/util/windows/user.rb index 185c38f03..51eef779d 100644 --- a/lib/puppet/util/windows/user.rb +++ b/lib/puppet/util/windows/user.rb @@ -44,7 +44,7 @@ module Puppet::Util::Windows::User def password_is?(name, password) logon_user(name, password) true - rescue Puppet::Util::Windows::Error => e + rescue Puppet::Util::Windows::Error false end module_function :password_is? diff --git a/lib/puppet/version.rb b/lib/puppet/version.rb index 8abce41c7..b0ace7de6 100644 --- a/lib/puppet/version.rb +++ b/lib/puppet/version.rb @@ -7,7 +7,7 @@ module Puppet - PUPPETVERSION = '3.2.4' + PUPPETVERSION = '3.3.0' ## # version is a public API method intended to always provide a fast and diff --git a/lib/puppetx.rb b/lib/puppetx.rb new file mode 100644 index 000000000..b5f30fc69 --- /dev/null +++ b/lib/puppetx.rb @@ -0,0 +1,109 @@ +# The Puppet Extensions Module. +# +# Submodules of this module should be named after the publisher (e.g. 'user' part of a Puppet Module name). +# The submodule {Puppetx::Puppet} contains the puppet extension points. +# +# This module also contains constants that are used when defining extensions. +# +# @api public +# +module Puppetx + + # The lookup **key** for the multibind containing syntax checkers used to syntax check embedded string in non + # puppet DSL syntax. + # @api public + SYNTAX_CHECKERS = 'puppetx::puppet::syntaxcheckers' + + # The lookup **type** for the multibind containing syntax checkers used to syntax check embedded string in non + # puppet DSL syntax. + # @api public + SYNTAX_CHECKERS_TYPE = 'Puppetx::Puppet::SyntaxChecker' + + # The lookup **key** for the multibind containing a map from scheme name to scheme handler class for bindings schemes. + # @api public + BINDINGS_SCHEMES = 'puppetx::puppet::bindings::schemes' + + # The lookup **type** for the multibind containing a map from scheme name to scheme handler class for bindings schemes. + # @api public + BINDINGS_SCHEMES_TYPE = 'Puppetx::Puppet::BindingsSchemeHandler' + + # The lookup **key** for the multibind containing a map from hiera-2 backend name to class implementing the backend. + # @api public + HIERA2_BACKENDS = 'puppetx::puppet::hiera2::backends' + + # The lookup **type** for the multibind containing a map from hiera-2 backend name to class implementing the backend. + # @api public + HIERA2_BACKENDS_TYPE = 'Puppetx::Puppet::Hiera2Backend' + + # This module is the name space for extension points + # @api public + module Puppet + + if ::Puppet[:binder] || ::Puppet[:parser] == 'future' + # Extension-points are registered here: + # - If in a Ruby submodule it is best to create it here + # - The class does not have to be required; it will be auto required when the binder + # needs it. + # - If the extension is a multibind, it can be registered here; either with a required + # class or a class reference in string form. + + # Register extension points + # ------------------------- + system_bindings = ::Puppet::Pops::Binder::SystemBindings + extensions = system_bindings.extensions() + extensions.multibind(SYNTAX_CHECKERS).name(SYNTAX_CHECKERS).hash_of(SYNTAX_CHECKERS_TYPE) + extensions.multibind(BINDINGS_SCHEMES).name(BINDINGS_SCHEMES).hash_of(BINDINGS_SCHEMES_TYPE) + extensions.multibind(HIERA2_BACKENDS).name(HIERA2_BACKENDS).hash_of(HIERA2_BACKENDS_TYPE) + + # Register injector boot bindings + # ------------------------------- + boot_bindings = system_bindings.injector_boot_bindings() + + # Register the default bindings scheme handlers + require 'puppetx/puppet/bindings_scheme_handler' + { 'module' => 'ModuleScheme', + 'confdir' => 'ConfdirScheme', + 'module-hiera' => 'ModuleHieraScheme', + 'confdir-hiera' => 'ConfdirHieraScheme' + }.each do |scheme, class_name| + boot_bindings.bind.name(scheme).instance_of(BINDINGS_SCHEMES_TYPE).in_multibind(BINDINGS_SCHEMES). + to_instance("Puppet::Pops::Binder::SchemeHandler::#{class_name}") + end + + # Register the default hiera2 backends + require 'puppetx/puppet/hiera2_backend' + { 'json' => 'JsonBackend', + 'yaml' => 'YamlBackend' + }.each do |symbolic, class_name| + boot_bindings.bind.name(symbolic).instance_of(HIERA2_BACKENDS_TYPE).in_multibind(HIERA2_BACKENDS). + to_instance("Puppet::Pops::Binder::Hiera2::#{class_name}") + end + end + end + + # Module with implementations of various extensions + # @api public + module Puppetlabs + # Default extensions delivered in Puppet Core are included here + + # @api public + module SyntaxCheckers + if ::Puppet[:binder] || ::Puppet[:parser] == 'future' + + # Classes in this name-space are lazily loaded as they may be overridden and/or never used + # (Lazy loading is done by binding to the name of a class instead of a Class instance). + + # Register extensions + # ------------------- + system_bindings = ::Puppet::Pops::Binder::SystemBindings + bindings = system_bindings.default_bindings() + bindings.bind do + name('json') + instance_of(SYNTAX_CHECKERS_TYPE) + in_multibind(SYNTAX_CHECKERS) + to_instance('Puppetx::Puppetlabs::SyntaxCheckers::Json') + end + end + end + end +end
\ No newline at end of file diff --git a/lib/puppetx/puppet/bindings_scheme_handler.rb b/lib/puppetx/puppet/bindings_scheme_handler.rb new file mode 100644 index 000000000..8ae66d388 --- /dev/null +++ b/lib/puppetx/puppet/bindings_scheme_handler.rb @@ -0,0 +1,130 @@ +module Puppetx::Puppet + # BindingsSchemeHandler is a Puppet Extension Point for the purpose of extending Puppet with a + # handler of a URI scheme used in the Puppet Bindings / Injector system. + # The intended use is to create a class derived from this class and then register it with the + # Puppet Binder. + # + # Creating the Extension Class + # ---------------------------- + # As an example, a class for getting LDAP data and transforming into bindings based on an LDAP URI scheme (such as RFC 2255, 4516) + # may be authored in say a puppet module called 'exampleorg/ldap'. The name of the class should start with `Puppetx::<user>::<module>`, + # e.g. 'Puppetx::Exampleorg::Ldap::LdapBindingsSchemeHandler" and + # be located in `lib/puppetx/exampleorg/Ldap/LdapBindingsSchemeHandler.rb`. (These rules are not enforced, but it make the class + # both auto-loadable, and guaranteed to not have a name that clashes with some other LdapBindingsSchemeHandler from some other + # author/organization. + # + # The Puppet Binder will auto-load the file when it + # has a binding to the class `Puppetx::Exampleorg::Ldap::LdapBindingsSchemeHandler' + # The Ruby Module `Puppetx` is created by Puppet, the remaining modules should be created by the loaded logic - e.g.: + # + # @example Defining an LdapBindingsSchemeHandler + # module Puppetx::Exampleorg + # module Ldap + # class LdapBindingsSchemeHandler < Puppetx::Puppetlabs::BindingsSchemeHandler + # # implement the methods + # end + # end + # end + # + # + # The expand_included method + # -------------------------- + # This method is given a URI (as entred by a user in a bindings configuration) and the handler's first task is to + # perform checking, transformation, and possible expansion into multiple URIs for loading. The result is always an array + # of URIs. This method allows users to enter wild-cards, or to represent something symbolic that is transformed into one or + # more "real URIs" to load. (It is allowed to change scheme!). + # If the "optional" feature is supported, the handler should not include the URI in the result unless it will be able to produce + # bindings for the given URI (as an option it may produce an empty set of bindings). + # + # The expand_excluded method + # --------------------------- + # This method is given an URI (as entered by the user in a bindings configuration), and it is the handler's second task + # to perform checking, transformation, and possible expansion into multiple URIs that should not be loaded. The result is always + # an array of URIs. The user may be allowed to enter wild-cards etc. The URIs produced by this method should have the same syntax + # as those produced by {#expand_included} since they are excluded by comparison. + # + # The contributed_bindings method + # ------------------------------- + # As the last step, the handler is being called once per URI that was included, and not later excluded to produce the + # contributed bindings. It is given three arguments, uri (the uri to load), scope (to provide access to the rest of the + # environment), and an acceptor (of issues), on which issues can be recorded. + # + # Reporting Errors/Issues + # ----------------------- + # Issues are reported by calling the given composer's acceptor, which takes a severity (e.g. `:error`, + # `:warning`, or `:ignore`), an {Puppet::Pops::Issues::Issue Issue} instance, and a {Puppet::Pops::Adapters::SourcePosAdapter + # SourcePosAdapter} (which describes details about linenumber, position, and length of the problem area). If the scheme is + # not based on file, line, pos - nil can be passed. The URI itself can be passed as file. + # + # @example Reporting an issue + # # create an issue with a symbolic name (that can serve as a reference to more details about the problem), + # # make the name unique + # issue = Puppet::Pops::Issues::issue(:EXAMPLEORG_LDAP_ILLEGAL_URI) { "The URI is not a valid Ldap URI" } + # source_pos = nil + # + # # report it + # composer.acceptor.accept(Puppet::Pops::Validation::Diagnostic.new(:error, issue, uri.to_s, source_pos, {})) + # + # Instead of reporting issues, an exception can be raised. + # + # @abstract + # @api public + # + class BindingsSchemeHandler + + # Produces the bindings contributed to the binding system based on the given URI. + # @param uri [URI] the URI to load bindings from + # @param scope [Puppet::Pops::Parser::Scope] access to scope and the rest of the environment + # @param composer [Puppet::Pops::Binder::BindingsComposer] a composer giving access to modules by name, and a diagnostics acceptor + # @return [Puppet::Pops::Binder::Bindings::ContributedBindings] the bindings to contribute, most conveniently + # created by calling {Puppet::Pops::Binder::BindingsFactory.contributed_bindings}. + # @api public + # + def contributed_bindings(uri, scope, composer) + raise NotImplementedError, "The BindingsProviderScheme for uri: '#{uri}' must implement 'contributed_bindings'" + end + + # Expands the given URI for the purpose of including the bindings it refers to. The input may contain + # wild-cards (if supported by this handler), and it is this methods responsibility to transform such into + # real loadable URIs. + # + # A scheme handler that does not support optionality, or wildcards should simply return the given URI + # in an Array. + # + # @param uri [URI] the uri for which bindings are to be produced. + # @param composer [Puppet::Pops::Binder::BindingsComposer] a composer giving access to modules by name, and a diagnostics acceptor + # @return [Array<URI>] the transformed, and possibly expanded set of URIs to include. + # @api public + # + def expand_included(uri, composer) + [uri] + end + + # Expands the given URI for the purpose of excluding the bindings it refers to. The input may contain + # wild-cards (if supported by this handler), and it is this methods responsibility to transform such into + # real loadable URIs (that match those produced by {#expand_included} that should be excluded from the result. + # + # A scheme handler that does not support optionality, or wildcards should simply return the given URI + # in an Array. + # + # @param uri [URI] the uri for which bindings are to be produced. + # @param composer [Puppet::Pops::Binder::BindingsComposer] a composer giving access to modules by name, and a diagnostics acceptor + # @return [Array<URI>] the transformed, and possibly expanded set of URIs to include- + # @api public + # + def expand_excluded(uri, composer) + [uri] + end + + # Returns whether the uri is optional or not. A scheme handler does not have to use this method + # to determine optionality, but if it supports such a feature, and there is no technical problem in supporting + # it this way, it should be done the same (or at least similar) way across all scheme handlers. + # + # This method interprets a URI ending with `?` or has query that is '?optional' as optional. + # + # @return [Boolean] whether the uri is an optional reference or not. + def is_optional?(uri) + (query = uri.query) && query == '' || query == 'optional' + end + end +end diff --git a/lib/puppetx/puppet/hiera2_backend.rb b/lib/puppetx/puppet/hiera2_backend.rb new file mode 100644 index 000000000..d4985d602 --- /dev/null +++ b/lib/puppetx/puppet/hiera2_backend.rb @@ -0,0 +1,31 @@ +module Puppetx::Puppet + + # Hiera2Backend is a Puppet Extension Point for the purpose of extending Puppet with a hiera data compatible + # backend. The intended use is to create a class derived from this class and then register it with the + # Puppet Binder under a backend name in the `binder_config.yaml` file to map symbolic name to class name. + # + # The responsibility of a Hiera2 backend is minimal. It should read the given file (with some extesion(s) determined by + # the backend, and return a hash of the content. If the directory does not exist, or the file does not exist an empty + # hash should be produced. + # + # @abstract + # @api public + # + class Hiera2Backend + # Produces a hash with data read from the file in the given + # directory having the given file_name (with extensions appended under the discretion of this + # backend). + # + # Should return an empty hash if the directory or the file does not exist. May raise exception on other types of errors, but + # not return nil. + # + # @param directory [String] the path to the directory containing the file to read + # @param file_name [String] the file name (without extension) that should be read + # @return [Hash<String, Object>, Hash<Symbol, Object>] the produced hash with data, may be empty if there was no file + # @api public + # + def read_data(directory, file_name) + raise NotImplementedError, "The class #{self.class.name} should have implemented the method 'read_data(directory, file_name)'" + end + end +end diff --git a/lib/puppetx/puppet/syntax_checker.rb b/lib/puppetx/puppet/syntax_checker.rb new file mode 100644 index 000000000..6baa1479d --- /dev/null +++ b/lib/puppetx/puppet/syntax_checker.rb @@ -0,0 +1,91 @@ +module Puppetx::Puppet + # SyntaxChecker is a Puppet Extension Point for the purpose of extending Puppet with syntax checkers. + # The intended use is to create a class derived from this class and then register it with the + # Puppet Binder. + # + # Creating the Extension Class + # ---------------------------- + # As an example, a class for checking custom xml (aware of some custom schemes) may be authored in + # say a puppet module called 'exampleorg/xmldata'. The name of the class should start with `Puppetx::<user>::<module>`, + # e.g. 'Puppetx::Exampleorg::XmlData::XmlChecker" and + # be located in `lib/puppetx/exampleorg/xml_data/xml_checker.rb`. The Puppet Binder will auto-load this file when it + # has a binding to the class `Puppetx::Exampleorg::XmlData::XmlChecker' + # The Ruby Module `Puppetx` is created by Puppet, the remaining modules should be created by the loaded logic - e.g.: + # + # @example Defining an XmlChecker + # module Puppetx::Exampleorg + # module XmlData + # class XmlChecker < Puppetx::Puppetlabs::SyntaxCheckers::SyntaxChecker + # def check(text, syntax_identifier, acceptor, location_hash) + # # do the checking + # end + # end + # end + # end + # + # Implementing the check method + # ----------------------------- + # The implementation of the {#check} method should naturally perform syntax checking of the given text/string and + # produce found issues on the given `acceptor`. These can be warnings or errors. The method should return `false` if + # any warnings or errors were produced (it is up to the caller to check for error/warning conditions and report them + # to the user). + # + # Issues are reported by calling the given `acceptor`, which takes a severity (e.g. `:error`, + # or `:warning), an {Puppet::Pops::Issues::Issue} instance, and a {Puppet::Pops::Adapters::SourcePosAdapter} + # (which describes details about linenumber, position, and length of the problem area). Note that the + # `location_info` given to the check method holds information about the location of the string in its *container* + # (e.g. the source position of a Heredoc); this information can be used if more detailed information is not + # available, or combined if there are more details (relative to the start of the checked string). + # + # @example Reporting an issue + # # create an issue with a symbolic name (that can serve as a reference to more details about the problem), + # # make the name unique + # issue = Puppet::Pops::Issues::issue(:EXAMPLEORG_XMLDATA_ILLEGAL_XML) { "syntax error found in xml text" } + # source_pos = Puppet::Pops::Adapters::SourcePosAdapter.new() + # source_pos.line = info[:line] # use this if there is no detail from the used parser + # source_pos.pos = info[:pos] # use this pos if there is no detail from used parser + # + # # report it + # acceptor.accept(Puppet::Pops::Validation::Diagnostic.new(:error, issue, info[:file], source_pos, {})) + # + # There is usually a cap on the number of errors/warnings that are presented to the user, this is handled by the + # reporting logic, but care should be taken to not generate too many as the issues are kept in memory until + # the checker returns. The acceptor may set a limit and simply ignore issues past a certain (high) number of reported + # issues (this number is typically higher than the cap on issues reported to the user). + # + # The `syntax_identifier` + # ----------------------- + # The extension makes use of a syntax identifier written in mime-style. This identifier can be something simple + # as 'xml', or 'json', but can also consist of several segments joined with '+' where the most specific syntax variant + # is placed first. When searching for a syntax checker; say for JSON having some special traits, say 'userdata', the + # author of the text may indicate this as the text having the syntax "userdata+json" - when a checker is looked up it is + # first checked if there is a checker for "userdata+json", if none is found, a lookup is made for "json" (since the text + # must at least be valid json). The given identifier is passed to the checker (to allow the same checker to check for + # several dialects/specializations). + # + # Use in Puppet DSL + # ----------------- + # The Puppet DSL Heredoc support and Puppet Templates makes use of the syntax checker extension. A user of a + # heredoc can specify the syntax in the heredoc tag, e.g.`@(END:userdata+json)`. + # + # + # @abstract + # + class SyntaxChecker + # Checks the text for syntax issues and reports them to the given acceptor. + # This implementation is abstract, it raises {NotImplementedError} since a subclass should have implemented the + # method. + # + # @param text [String] The text to check + # @param syntax_identifier [String] The syntax identifier in mime style (e.g. 'json', 'json-patch+json', 'xml', 'myapp+xml' + # @option location_info [String] :file The filename where the string originates + # @option location_info [Integer] :line The line number identifying the location where the string is being used/checked + # @option location_info [Integer] :position The position on the line identifying the location where the string is being used/checked + # @return [Boolean] Whether the checked string had issues (warnings and/or errors) or not. + # @api public + # + def check(text, syntax_identifier, acceptor, location_info) + raise NotImplementedError, "The class #{self.class.name} should have implemented the method check()" + end + end +end
\ No newline at end of file diff --git a/lib/puppetx/puppetlabs/syntax_checkers/json.rb b/lib/puppetx/puppetlabs/syntax_checkers/json.rb new file mode 100644 index 000000000..e31f52688 --- /dev/null +++ b/lib/puppetx/puppetlabs/syntax_checkers/json.rb @@ -0,0 +1,39 @@ +# A syntax checker for JSON. +# @api public +class Puppetx::Puppetlabs::SyntaxCheckers::Json < Puppetx::Puppet::SyntaxChecker + + # Checks the text for JSON syntax issues and reports them to the given acceptor. + # This implementation is abstract, it raises {NotImplementedError} since a subclass should have implemented the + # method. + # + # @param text [String] The text to check + # @param syntax [String] The syntax identifier in mime style (e.g. 'json', 'json-patch+json', 'xml', 'myapp+xml' + # @option location_info [String] :file The filename where the string originates + # @option location_info [Integer] :line The line number identifying the location where the string is being used/checked + # @option location_info [Integer] :position The position on the line identifying the location where the string is being used/checked + # @return [Boolean] Whether the checked string had issues (warnings and/or errors) or not. + # @api public + # + def check(text, syntax, acceptor, location_info={}) + raise ArgumentError.new("Json syntax checker: the text to check must be a String.") unless text.is_a?(String) + raise ArgumentError.new("Json syntax checker: the syntax identifier must be a String, e.g. json, data+json") unless syntax.is_a?(String) + raise ArgumentError.new("Json syntax checker: invalid Acceptor, got: '#{acceptor.class.name}'.") unless acceptor.is_a?(Puppet::Pops::Validation::Acceptor) + raise ArgumentError.new("Json syntax checker: location_info must be a Hash") unless info.is_a?(Hash) + + begin + JSON.parse(text) + rescue => e + # Cap the message to 100 chars and replace newlines + msg = "Json syntax checker:: Cannot parse invalid JSON string. \"#{e.message().slice(0,100).gsub(/\r?\n/, "\\n")}\"" + + # TODO: improve the pops API to allow simpler diagnostic creation while still maintaining capabilities + # and the issue code. (In this case especially, where there is only a single error message being issued). + # + issue = Puppet::Pops::Issues::issue(:ILLEGAL_JSON) { msg } + source_pos = Puppet::Pops::Adapters::SourcePosAdapter.new() + source_pos.line = location_info[:line] + source_pos.pos = location_info[:pos] + acceptor.accept(Puppet::Pops::Validation::Diagnostic.new(:error, issue, location_info[:file], source_pos, {})) + end + end +end diff --git a/lib/semver.rb b/lib/semver.rb index bf6691972..01ff1eb16 100644 --- a/lib/semver.rb +++ b/lib/semver.rb @@ -119,6 +119,6 @@ class SemVer < Numeric MIN.instance_variable_set(:@vstring, 'vMIN') MAX = SemVer.new('8.0.0') - MAX.instance_variable_set(:@major, (1.0/0)) # => Infinity + MAX.instance_variable_set(:@major, Float::INFINITY) # => Infinity MAX.instance_variable_set(:@vstring, 'vMAX') end diff --git a/man/man8/puppet-kick.8 b/man/man8/puppet-kick.8 index 3e587fcbe..fbd3959e4 100644 --- a/man/man8/puppet-kick.8 +++ b/man/man8/puppet-kick.8 @@ -114,7 +114,7 @@ Print the hosts you would connect to but do not actually connect\. This option r . .TP \-\-ping -Do a ICMP echo against the target host\. Skip hosts that don\'t respond to ping\. +Do an ICMP echo against the target host\. Skip hosts that don\'t respond to ping\. . .SH "EXAMPLE" . diff --git a/spec/fixtures/integration/provider/cron/crontab/unspecialized b/spec/fixtures/integration/provider/cron/crontab/unspecialized new file mode 100644 index 000000000..e6a40821f --- /dev/null +++ b/spec/fixtures/integration/provider/cron/crontab/unspecialized @@ -0,0 +1,15 @@ +# HEADER: some simple +# HEADER: header +@daily /bin/unnamed_special_command >> /dev/null 2>&1 + +# commend with blankline above and below + +17-19,22 0-23/2 * * 2 /bin/unnamed_regular_command + +# Puppet Name: My daily failure +MAILTO="" +* * * * * /bin/false +# Puppet Name: Monthly job +SHELL=/bin/sh +MAILTO=mail@company.com +15 14 1 * * $HOME/bin/monthly diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/hiera1config/binder_config.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/hiera1config/binder_config.yaml new file mode 100644 index 000000000..9088193a4 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/hiera1config/binder_config.yaml @@ -0,0 +1,18 @@ +--- +version: 1 +layers: + [{name: site, include: 'confdir-hiera:/'}, + {name: modules, include: ['module-hiera:/*/', 'module:/*::default'] } + ] +categories: + [['node', '$fqdn'], + ['environment', '${environment}'], + ['osfamily', '${osfamily}'], + ['common', 'true'] + ] +#extensions: +# scheme_handlers: +# echo: 'Puppetx::Awesome::EchoSchemeHandler' +# +# hiera_backends: +# echo: 'Puppetx::Awesome::EchoBackend' diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/hiera1config/hiera.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/hiera1config/hiera.yaml new file mode 100644 index 000000000..717eacc4b --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/hiera1config/hiera.yaml @@ -0,0 +1,8 @@ +--- +hierarchy: + - '%fqdn' + - 'common' + +backends: + - yaml + - json diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/hiera1config/modules/good/common.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/hiera1config/modules/good/common.yaml new file mode 100644 index 000000000..38494c2d5 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/hiera1config/modules/good/common.yaml @@ -0,0 +1 @@ +the_meaning_of_life: 300
\ No newline at end of file diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/hiera1config/modules/good/hiera.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/hiera1config/modules/good/hiera.yaml new file mode 100644 index 000000000..5d3ea65a0 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/hiera1config/modules/good/hiera.yaml @@ -0,0 +1,10 @@ +--- +version: 2 +hierarchy: + [ ['node', '${fqdn}', '${fqdn}' ], + ['common', 'true', 'common' ] + ] + +backends: + - yaml + - json diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/ok/binder_config.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/ok/binder_config.yaml new file mode 100644 index 000000000..46148bca6 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/ok/binder_config.yaml @@ -0,0 +1,19 @@ +--- +version: 1 +layers: + [{name: site, include: 'confdir-hiera:/'}, + {name: test, include: 'echo:/quick/brown/fox'}, + {name: modules, include: ['module-hiera:/*/', 'module:/*::default'], exclude: 'module-hiera:/bad/' } + ] +categories: + [['node', '$fqdn'], + ['environment', '${environment}'], + ['osfamily', '${osfamily}'], + ['common', 'true'] + ] +extensions: + scheme_handlers: + echo: 'Puppetx::Awesome::EchoSchemeHandler' + + hiera_backends: + echo: 'Puppetx::Awesome::EchoBackend' diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/ok/common.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/ok/common.yaml new file mode 100644 index 000000000..e412898ba --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/ok/common.yaml @@ -0,0 +1 @@ +has_funny_hat: 'the pope'
\ No newline at end of file diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/ok/hiera.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/ok/hiera.yaml new file mode 100644 index 000000000..a56580ac6 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/ok/hiera.yaml @@ -0,0 +1,11 @@ +--- +version: 2 + +hierarchy: + [ ['node', '${fqdn}', '${fqdn}' ], + ['common', 'true', 'common' ] + ] + +backends: + - yaml + - json diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/ok/localhost.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/ok/localhost.yaml new file mode 100644 index 000000000..1976ccec8 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/ok/localhost.yaml @@ -0,0 +1 @@ +the_meaning_of_life: 42
\ No newline at end of file diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/common.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/common.yaml new file mode 100644 index 000000000..4bdf31981 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/common.yaml @@ -0,0 +1,3 @@ +awesome_x: 'golden' +the_meaning_of_life: 100 +has_funny_hat: 'kkk'
\ No newline at end of file diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/hiera.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/hiera.yaml new file mode 100644 index 000000000..f83cb1194 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/hiera.yaml @@ -0,0 +1,13 @@ +--- +version: 2 + +hierarchy: + [ ['node', '${fqdn}', '${fqdn}' ], +# ['osfamily', '${osfamily}', 'osfamily' ] + ['common', 'true', 'common' ] + ] + +backends: + - yaml + - json + - echo diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/lib/puppet/bindings/awesome/default.rb b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/lib/puppet/bindings/awesome/default.rb new file mode 100644 index 000000000..2517202aa --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/lib/puppet/bindings/awesome/default.rb @@ -0,0 +1,4 @@ +Puppet::Bindings.newbindings('awesome::default') do |scope| + bind.name('all your base').to('are belong to us') + bind.name('env_meaning_of_life').to(puppet_string("$environment thinks it is 42", __FILE__)) +end
\ No newline at end of file diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/lib/puppetx/awesome/echo_backend.rb b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/lib/puppetx/awesome/echo_backend.rb new file mode 100644 index 000000000..7de067a9e --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/lib/puppetx/awesome/echo_backend.rb @@ -0,0 +1,11 @@ +require 'puppetx/puppet/hiera2_backend' + +module Puppetx + module Awesome + class EchoBackend < Puppetx::Puppet::Hiera2Backend + def read_data(directory, file_name) + {"echo::#{file_name}" => "echo... #{File.basename(directory)}/#{file_name}"} + end + end + end +end
\ No newline at end of file diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/lib/puppetx/awesome/echo_scheme_handler.rb b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/lib/puppetx/awesome/echo_scheme_handler.rb new file mode 100644 index 000000000..3f97a89ab --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/lib/puppetx/awesome/echo_scheme_handler.rb @@ -0,0 +1,18 @@ +require 'puppetx/puppet/bindings_scheme_handler' + +module Puppetx + module Awesome + # A binding scheme that echos its path + # 'echo:/quick/brown/fox' becomes key '::quick::brown::fox' => 'echo: quick brown fox'. + # (silly class for testing loading of extension) + # + class EchoSchemeHandler < Puppetx::Puppet::BindingsSchemeHandler + def contributed_bindings(uri, scope, composer) + factory = ::Puppet::Pops::Binder::BindingsFactory + bindings = factory.named_bindings("echo") + bindings.bind.name(uri.path.gsub(/\//, '::')).to("echo: #{uri.path.gsub(/\//, ' ').strip!}") + result = factory.contributed_bindings("echo", bindings.model, nil) + end + end + end +end
\ No newline at end of file diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/localhost.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/localhost.yaml new file mode 100644 index 000000000..01aba4d93 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/awesome/localhost.yaml @@ -0,0 +1 @@ +good_x: 'golden'
\ No newline at end of file diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/bad/common.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/bad/common.yaml new file mode 100644 index 000000000..bcb9f61ce --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/bad/common.yaml @@ -0,0 +1,3 @@ +bad_x: 'rotten' +the_meaning_of_life: 200 +has_funny_hat: 'the syldavians'
\ No newline at end of file diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/bad/hiera_config.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/bad/hiera_config.yaml new file mode 100644 index 000000000..bb00d74f9 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/bad/hiera_config.yaml @@ -0,0 +1,9 @@ +# BROKEN ON PURPOSE +--- +hierarchyyyyyy: + - ['node', '${fqdn}', '${fqdn}' + - ['common', 'true', 'common' + +backendsssssss: + - yaml + - json diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/good/common.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/good/common.yaml new file mode 100644 index 000000000..ad5a5d9aa --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/good/common.yaml @@ -0,0 +1,2 @@ +good_x: 'decent' +the_meaning_of_life: 300
\ No newline at end of file diff --git a/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/good/hiera.yaml b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/good/hiera.yaml new file mode 100644 index 000000000..a56580ac6 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/bindings_composer/ok/modules/good/hiera.yaml @@ -0,0 +1,11 @@ +--- +version: 2 + +hierarchy: + [ ['node', '${fqdn}', '${fqdn}' ], + ['common', 'true', 'common' ] + ] + +backends: + - yaml + - json diff --git a/spec/fixtures/unit/pops/binder/config/binder_config/ok/binder_config.yaml b/spec/fixtures/unit/pops/binder/config/binder_config/ok/binder_config.yaml new file mode 100644 index 000000000..c51ee4c01 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/config/binder_config/ok/binder_config.yaml @@ -0,0 +1,9 @@ +--- +version: 1 +layers: + - {name: site, include: 'confdir-hiera:/'} + - {name: modules, include: 'module-hiera:/*/', exclude: 'module-hiera:/bad/' } +categories: + - ['node', '$fqn'] + - ['environment', '$environment'] + - ['common', 'true']
\ No newline at end of file diff --git a/spec/fixtures/unit/pops/binder/hiera2/bindings_provider/ok/hiera.yaml b/spec/fixtures/unit/pops/binder/hiera2/bindings_provider/ok/hiera.yaml new file mode 100644 index 000000000..4ea7358a4 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/hiera2/bindings_provider/ok/hiera.yaml @@ -0,0 +1,9 @@ +--- +version: 2 + +hierarchy: + - ['node', '${node}', '${node}' ] + +backends: + - yaml + - json diff --git a/spec/fixtures/unit/pops/binder/hiera2/bindings_provider/ok/node.example.com.json b/spec/fixtures/unit/pops/binder/hiera2/bindings_provider/ok/node.example.com.json new file mode 100644 index 000000000..5b4a104d5 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/hiera2/bindings_provider/ok/node.example.com.json @@ -0,0 +1,9 @@ +{ + "a_json_number": 142, + "a_string": "don't want this to override", + "a_json_string": "one hundred and forty two", + "a_json_eval": "the answer from \"json\" is ${a} and \\${a}.", + "a_json_eval2": "the answer\nfrom \\\"json\\\" is $a and \\$a", + "a_json_array": ["a", "b", 100], + "a_json_hash": { "a": 1, "b": 2 } +} diff --git a/spec/fixtures/unit/pops/binder/hiera2/bindings_provider/ok/node.example.com.yaml b/spec/fixtures/unit/pops/binder/hiera2/bindings_provider/ok/node.example.com.yaml new file mode 100644 index 000000000..01340d6c7 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/hiera2/bindings_provider/ok/node.example.com.yaml @@ -0,0 +1,5 @@ +a_number: 42 +a_string: forty two +an_eval: "the answer from \"yaml\" is ${a}." +an_eval2: "the answer\nfrom \\\"yaml\\\" is $a and \\$a" + diff --git a/spec/fixtures/unit/pops/binder/hiera2/config/bad_syntax/hiera.yaml b/spec/fixtures/unit/pops/binder/hiera2/config/bad_syntax/hiera.yaml new file mode 100644 index 000000000..3c1c2d811 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/hiera2/config/bad_syntax/hiera.yaml @@ -0,0 +1,10 @@ +--- +version: 2 + +hierarchy + os: + - ${osfamily} + - for_${osfamily} + +backends: + - yaml diff --git a/spec/fixtures/unit/pops/binder/hiera2/config/malformed_hierarchy/hiera.yaml b/spec/fixtures/unit/pops/binder/hiera2/config/malformed_hierarchy/hiera.yaml new file mode 100644 index 000000000..13869783f --- /dev/null +++ b/spec/fixtures/unit/pops/binder/hiera2/config/malformed_hierarchy/hiera.yaml @@ -0,0 +1,8 @@ +--- +version: 2 + +hierarchy: + ['os', '${osfamily}' ] + +backends: + - yaml diff --git a/spec/fixtures/unit/pops/binder/hiera2/config/missing/foo.txt b/spec/fixtures/unit/pops/binder/hiera2/config/missing/foo.txt new file mode 100644 index 000000000..e5ce6c87a --- /dev/null +++ b/spec/fixtures/unit/pops/binder/hiera2/config/missing/foo.txt @@ -0,0 +1 @@ +# Do not delete diff --git a/spec/fixtures/unit/pops/binder/hiera2/config/no_backends/hiera.yaml b/spec/fixtures/unit/pops/binder/hiera2/config/no_backends/hiera.yaml new file mode 100644 index 000000000..f1e9f4251 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/hiera2/config/no_backends/hiera.yaml @@ -0,0 +1,7 @@ +--- +version: 2 +hierarchy: + os: + - ${osfamily} + - for_${osfamily} + diff --git a/spec/fixtures/unit/pops/binder/hiera2/config/no_hierarchy/hiera.yaml b/spec/fixtures/unit/pops/binder/hiera2/config/no_hierarchy/hiera.yaml new file mode 100644 index 000000000..238752dd0 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/hiera2/config/no_hierarchy/hiera.yaml @@ -0,0 +1,4 @@ +--- +version: 2 +backends: + - yaml diff --git a/spec/fixtures/unit/pops/binder/hiera2/config/not_a_hash/hiera.yaml b/spec/fixtures/unit/pops/binder/hiera2/config/not_a_hash/hiera.yaml new file mode 100644 index 000000000..a9e37cff5 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/hiera2/config/not_a_hash/hiera.yaml @@ -0,0 +1,2 @@ +- first +- second diff --git a/spec/fixtures/unit/pops/binder/hiera2/config/ok/hiera.yaml b/spec/fixtures/unit/pops/binder/hiera2/config/ok/hiera.yaml new file mode 100644 index 000000000..4e2f493d1 --- /dev/null +++ b/spec/fixtures/unit/pops/binder/hiera2/config/ok/hiera.yaml @@ -0,0 +1,8 @@ +--- +version: 2 + +hierarchy: + - ['os', '${osfamily}', 'for_${osfamily}' ] + +backends: + - yaml diff --git a/spec/fixtures/unit/pops/binder/hiera2/yaml_backend/empty/common.yaml b/spec/fixtures/unit/pops/binder/hiera2/yaml_backend/empty/common.yaml new file mode 100644 index 000000000..e69de29bb --- /dev/null +++ b/spec/fixtures/unit/pops/binder/hiera2/yaml_backend/empty/common.yaml diff --git a/spec/fixtures/unit/pops/binder/hiera2/yaml_backend/invalid/common.yaml b/spec/fixtures/unit/pops/binder/hiera2/yaml_backend/invalid/common.yaml new file mode 100644 index 000000000..ed97d539c --- /dev/null +++ b/spec/fixtures/unit/pops/binder/hiera2/yaml_backend/invalid/common.yaml @@ -0,0 +1 @@ +--- diff --git a/spec/fixtures/unit/pops/binder/hiera2/yaml_backend/ok/common.yaml b/spec/fixtures/unit/pops/binder/hiera2/yaml_backend/ok/common.yaml new file mode 100644 index 000000000..0aa0b6bcb --- /dev/null +++ b/spec/fixtures/unit/pops/binder/hiera2/yaml_backend/ok/common.yaml @@ -0,0 +1,2 @@ +--- +brillig: slithy diff --git a/spec/fixtures/unit/provider/package/openbsd/pkginfo_flavors.list b/spec/fixtures/unit/provider/package/openbsd/pkginfo_flavors.list new file mode 100644 index 000000000..1ed49a5eb --- /dev/null +++ b/spec/fixtures/unit/provider/package/openbsd/pkginfo_flavors.list @@ -0,0 +1,2 @@ +bash-3.1.17-static GNU Bourne Again Shell +vim-7.0.42-no_x11 vi clone, many additional features diff --git a/spec/integration/agent/logging_spec.rb b/spec/integration/agent/logging_spec.rb new file mode 100755 index 000000000..284928ed7 --- /dev/null +++ b/spec/integration/agent/logging_spec.rb @@ -0,0 +1,178 @@ +#! /usr/bin/env ruby +require 'spec_helper' + +require 'puppet' +require 'puppet/daemon' +require 'puppet/application/agent' + +# The command line flags affecting #20900 and #20919: +# +# --onetime +# --daemonize +# --no-daemonize +# --logdest +# --verbose +# --debug +# (no flags) (-) +# +# d and nd are mutally exclusive +# +# Combinations without logdest, verbose or debug: +# +# --onetime --daemonize +# --onetime --no-daemonize +# --onetime +# --daemonize +# --no-daemonize +# - +# +# 6 cases X [--logdest=console, --logdest=syslog, --logdest=/some/file, <nothing added>] +# = 24 cases to test +# +# X [--verbose, --debug, <nothing added>] +# = 72 cases to test +# +# Expectations of behavior are defined in the expected_loggers, expected_level methods, +# so adapting to a change in logging behavior should hopefully be mostly a matter of +# adjusting the logic in those methods to define new behavior. +# +# Note that this test does not have anything to say about what happens to logging after +# daemonizing. +describe 'agent logging' do + ONETIME = '--onetime' + DAEMONIZE = '--daemonize' + NO_DAEMONIZE = '--no-daemonize' + LOGDEST_FILE = '--logdest=/dev/null/foo' + LOGDEST_SYSLOG = '--logdest=syslog' + LOGDEST_CONSOLE = '--logdest=console' + VERBOSE = '--verbose' + DEBUG = '--debug' + + DEFAULT_LOG_LEVEL = :notice + INFO_LEVEL = :info + DEBUG_LEVEL = :debug + CONSOLE = :console + SYSLOG = :syslog + EVENTLOG = :eventlog + FILE = :file + + ONETIME_DAEMONIZE_ARGS = [ + [ONETIME], + [ONETIME, DAEMONIZE], + [ONETIME, NO_DAEMONIZE], + [DAEMONIZE], + [NO_DAEMONIZE], + [], + ] + LOG_DEST_ARGS = [LOGDEST_FILE, LOGDEST_SYSLOG, LOGDEST_CONSOLE, nil] + LOG_LEVEL_ARGS = [VERBOSE, DEBUG, nil] + + shared_examples "an agent" do |argv, expected| + before(:each) do + # Don't actually run the agent, bypassing cert checks, forking and the puppet run itself + Puppet::Application::Agent.any_instance.stubs(:run_command) + end + + def double_of_bin_puppet_agent_call(argv) + argv.unshift('agent') + command_line = Puppet::Util::CommandLine.new('puppet', argv) + command_line.execute + end + + if Puppet.features.microsoft_windows? && argv.include?(DAEMONIZE) + + it "should exit on a platform which cannot daemonize if the --daemonize flag is set" do + expect { double_of_bin_puppet_agent_call(argv) }.to raise_error(SystemExit) + end + + else + + it "when evoked with #{argv}, logs to #{expected[:loggers].inspect} at level #{expected[:level]}" do + # This logger is created by the Puppet::Settings object which creates and + # applies a catalog to ensure that configuration files and users are in + # place. + # + # It's not something we are specifically testing here since it occurs + # regardless of user flags. + Puppet::Util::Log.expects(:newdestination).with(instance_of(Puppet::Transaction::Report)).once + expected[:loggers].each do |logclass| + Puppet::Util::Log.expects(:newdestination).with(logclass).at_least_once + end + double_of_bin_puppet_agent_call(argv) + + Puppet::Util::Log.level.should == expected[:level] + end + + end + end + + def self.no_log_dest_set_in(argv) + ([LOGDEST_SYSLOG, LOGDEST_CONSOLE, LOGDEST_FILE] & argv).empty? + end + + def self.verbose_or_debug_set_in_argv(argv) + !([VERBOSE, DEBUG] & argv).empty? + end + + def self.log_dest_is_set_to(argv, log_dest) + argv.include?(log_dest) + end + + # @param argv Array of commandline flags + # @return Set<Symbol> of expected loggers + def self.expected_loggers(argv) + loggers = Set.new + loggers << CONSOLE if verbose_or_debug_set_in_argv(argv) + loggers << 'console' if log_dest_is_set_to(argv, LOGDEST_CONSOLE) + loggers << '/dev/null/foo' if log_dest_is_set_to(argv, LOGDEST_FILE) + if Puppet.features.microsoft_windows? + # an explicit call to --logdest syslog on windows is swallowed silently with no + # logger created (see #suitable() of the syslog Puppet::Util::Log::Destination subclass) + # however Puppet::Util::Log.newdestination('syslog') does get called...so we have + # to set an expectation + loggers << 'syslog' if log_dest_is_set_to(argv, LOGDEST_SYSLOG) + + loggers << EVENTLOG if no_log_dest_set_in(argv) + else + # posix + loggers << 'syslog' if log_dest_is_set_to(argv, LOGDEST_SYSLOG) + loggers << SYSLOG if no_log_dest_set_in(argv) + end + return loggers + end + + # @param argv Array of commandline flags + # @return Symbol of the expected log level + def self.expected_level(argv) + case + when argv.include?(VERBOSE) then INFO_LEVEL + when argv.include?(DEBUG) then DEBUG_LEVEL + else DEFAULT_LOG_LEVEL + end + end + + # @param argv Array of commandline flags + # @return Hash of expected loggers and the expected log level + def self.with_expectations_based_on(argv) + { + :loggers => expected_loggers(argv), + :level => expected_level(argv), + } + end + +# For running a single spec (by line number): rspec -l150 spec/integration/agent/logging_spec.rb +# debug_argv = [] +# it_should_behave_like( "an agent", [debug_argv], with_expectations_based_on([debug_argv])) + + ONETIME_DAEMONIZE_ARGS.each do |onetime_daemonize_args| + LOG_LEVEL_ARGS.each do |log_level_args| + LOG_DEST_ARGS.each do |log_dest_args| + argv = (onetime_daemonize_args + [log_level_args, log_dest_args]).flatten.compact + + describe "for #{argv}" do + it_should_behave_like( "an agent", argv, with_expectations_based_on(argv)) + end + end + end + end +end diff --git a/spec/integration/configurer_spec.rb b/spec/integration/configurer_spec.rb index c91dfa8aa..355458f02 100755 --- a/spec/integration/configurer_spec.rb +++ b/spec/integration/configurer_spec.rb @@ -23,7 +23,7 @@ describe Puppet::Configurer do @catalog.add_resource(Puppet::Type.type(:notify).new(:title => "testing")) # Make sure we don't try to persist the local state after the transaction ran, - # because it will fail during test (the state file is in an not existing directory) + # because it will fail during test (the state file is in a not-existing directory) # and we need the transaction to be successful to be able to produce a summary report @catalog.host_config = false diff --git a/spec/integration/defaults_spec.rb b/spec/integration/defaults_spec.rb index 2764be1bb..8c8432b6f 100755 --- a/spec/integration/defaults_spec.rb +++ b/spec/integration/defaults_spec.rb @@ -92,12 +92,6 @@ describe "Puppet defaults" do Puppet.settings.setting(:yamldir).group.should == Puppet.settings[:group] end - # See #1232 - it "should not specify a user or group for the rundir" do - Puppet.settings.setting(:rundir).owner.should be_nil - Puppet.settings.setting(:rundir).group.should be_nil - end - it "should specify that the host private key should be owned by the service user" do Puppet.settings.stubs(:service_user_available?).returns true Puppet.settings.setting(:hostprivkey).owner.should == Puppet.settings[:user] diff --git a/spec/integration/network/authconfig_spec.rb b/spec/integration/network/authconfig_spec.rb index fd7f3065e..9db0c1099 100644 --- a/spec/integration/network/authconfig_spec.rb +++ b/spec/integration/network/authconfig_spec.rb @@ -40,6 +40,13 @@ describe Puppet::Network::AuthConfig do @auth = parser.parse end + def add_raw_stanza(stanza) + parser = Puppet::Network::AuthConfigParser.new( + stanza + ) + @auth = parser.parse + end + def request(args = {}) args = { :key => 'key', @@ -87,6 +94,18 @@ describe Puppet::Network::AuthConfig do @auth.should allow(request) end + it 'should warn about missing path before allow_ip in stanza' do + expect { + add_raw_stanza("allow_ip 10.0.0.1\n") + }.to raise_error Puppet::ConfigurationError, /Missing or invalid 'path' before right directive at line.*/ + end + + it 'should warn about missing path before allow in stanza' do + expect { + add_raw_stanza("allow host.domain.com\n") + }.to raise_error Puppet::ConfigurationError, /Missing or invalid 'path' before right directive at line.*/ + end + it "should support hostname backreferences" do add_regex_rule('^/test/([^/]+)$', "allow $1.domain.com") diff --git a/spec/integration/network/server/webrick_spec.rb b/spec/integration/network/server/webrick_spec.rb index 747c84209..5ae490410 100755 --- a/spec/integration/network/server/webrick_spec.rb +++ b/spec/integration/network/server/webrick_spec.rb @@ -11,7 +11,6 @@ describe Puppet::Network::Server, :unless => Puppet.features.microsoft_windows? # This reduces the odds of conflicting port numbers between concurrent runs # of the suite on the same machine dramatically. let(:port) { 20000 + ($$ % 40000) } - let(:handlers) { [:node] } let(:address) { '127.0.0.1' } before :each do @@ -30,7 +29,7 @@ describe Puppet::Network::Server, :unless => Puppet.features.microsoft_windows? ca = Puppet::SSL::CertificateAuthority.new ca.generate(Puppet[:certname]) unless Puppet::SSL::Certificate.indirection.find(Puppet[:certname]) - @server = Puppet::Network::Server.new(address, port, handlers) + @server = Puppet::Network::Server.new(address, port) end after do @@ -45,31 +44,31 @@ describe Puppet::Network::Server, :unless => Puppet.features.microsoft_windows? describe "when listening" do it "should be reachable on the specified address and port" do - @server.listen + @server.start expect { TCPSocket.new('127.0.0.1', port) }.to_not raise_error end it "should use any specified bind address" do - @server.stubs(:unlisten) # we're breaking listening internally, so we have to keep it from unlistening + @server.stubs(:stop) # we're breaking listening internally, so we have to keep it from unlistening Puppet::Network::HTTP::WEBrick.any_instance.expects(:listen).with(address, port) - @server.listen + @server.start end it "should not allow multiple servers to listen on the same address and port" do - @server.listen - server2 = Puppet::Network::Server.new(address, port, handlers) - expect { server2.listen }.to raise_error + @server.start + server2 = Puppet::Network::Server.new(address, port) + expect { server2.start }.to raise_error end after :each do - @server.unlisten if @server && @server.listening? + @server.stop if @server && @server.listening? end end describe "after unlistening" do it "should not be reachable on the port and address assigned" do - @server.listen - @server.unlisten + @server.start + @server.stop expect { TCPSocket.new('127.0.0.1', port) }.to raise_error(Errno::ECONNREFUSED) end end diff --git a/spec/integration/parser/catalog_spec.rb b/spec/integration/parser/catalog_spec.rb new file mode 100644 index 000000000..c11e251dd --- /dev/null +++ b/spec/integration/parser/catalog_spec.rb @@ -0,0 +1,85 @@ +require 'spec_helper' +require 'matchers/include_in_order' +require 'puppet_spec/compiler' +require 'puppet/indirector/catalog/compiler' + +describe "Transmission of the catalog to the agent" do + include PuppetSpec::Compiler + + it "preserves the order in which the resources are added to the catalog" do + resources_in_declaration_order = ["Class[First]", + "Second[position]", + "Class[Third]", + "Fourth[position]"] + + master_catalog, agent_catalog = master_and_agent_catalogs_for(<<-EOM) + define fourth() { } + class third { } + + define second() { + fourth { "position": } + } + + class first { + second { "position": } + class { "third": } + } + + include first + EOM + + expect(resources_in(master_catalog)). + to include_in_order(*resources_in_declaration_order) + expect(resources_in(agent_catalog)). + to include_in_order(*resources_in_declaration_order) + end + + it "does not contain unrealized, virtual resources" do + virtual_resources = ["Unrealized[unreal]", "Class[Unreal]"] + + master_catalog, agent_catalog = master_and_agent_catalogs_for(<<-EOM) + class unreal { } + define unrealized() { } + + class real { + @unrealized { "unreal": } + @class { "unreal": } + } + + include real + EOM + + expect(resources_in(master_catalog)).to_not include(*virtual_resources) + expect(resources_in(agent_catalog)).to_not include(*virtual_resources) + end + + it "does not contain unrealized, exported resources" do + exported_resources = ["Unrealized[unreal]", "Class[Unreal]"] + + master_catalog, agent_catalog = master_and_agent_catalogs_for(<<-EOM) + class unreal { } + define unrealized() { } + + class real { + @@unrealized { "unreal": } + @@class { "unreal": } + } + + include real + EOM + + expect(resources_in(master_catalog)).to_not include(*exported_resources) + expect(resources_in(agent_catalog)).to_not include(*exported_resources) + end + + def master_and_agent_catalogs_for(manifest) + master_catalog = Puppet::Resource::Catalog::Compiler.new.filter(compile_to_catalog(manifest)) + agent_catalog = Puppet::Resource::Catalog.convert_from(:pson, master_catalog.render(:pson)) + + [master_catalog, agent_catalog] + end + + def resources_in(catalog) + catalog.resources.map(&:ref) + end +end diff --git a/spec/integration/provider/cron/crontab_spec.rb b/spec/integration/provider/cron/crontab_spec.rb index b061abb0d..4681ff109 100644 --- a/spec/integration/provider/cron/crontab_spec.rb +++ b/spec/integration/provider/cron/crontab_spec.rb @@ -160,6 +160,17 @@ describe Puppet::Type.type(:cron).provider(:crontab), '(integration)', :unless = run_in_catalog(resource) expect_output('modify_entry') end + it "should change a special schedule to numeric if requested" do + resource = Puppet::Type.type(:cron).new( + :name => 'My daily failure', + :special => 'absent', + :command => '/bin/false', + :target => crontab_user1, + :user => crontab_user1 + ) + run_in_catalog(resource) + expect_output('unspecialized') + end it "should not try to move an entry from one file to another" do # force the parsedfile provider to also parse user1's crontab random_resource = Puppet::Type.type(:cron).new( diff --git a/spec/integration/provider/mount_spec.rb b/spec/integration/provider/mount_spec.rb index d97902c3b..310b58d1f 100755 --- a/spec/integration/provider/mount_spec.rb +++ b/spec/integration/provider/mount_spec.rb @@ -20,6 +20,7 @@ describe "mount provider (integration)", :unless => Puppet.features.microsoft_wi @current_options = "local" @current_device = "/dev/disk1s1" Puppet::Type.type(:mount).defaultprovider.stubs(:default_target).returns(@fake_fstab) + Facter.stubs(:value).with(:kernel).returns('Darwin') Facter.stubs(:value).with(:operatingsystem).returns('Darwin') Facter.stubs(:value).with(:osfamily).returns('Darwin') Puppet::Util::ExecutionStub.set do |command, options| diff --git a/spec/integration/transaction_spec.rb b/spec/integration/transaction_spec.rb index 3deab0f68..4fdd69315 100755 --- a/spec/integration/transaction_spec.rb +++ b/spec/integration/transaction_spec.rb @@ -33,7 +33,7 @@ describe Puppet::Transaction do resource.expects(:eval_generate).returns([child_resource]) - transaction = Puppet::Transaction.new(catalog) + transaction = Puppet::Transaction.new(catalog, nil, Puppet::Graph::RandomPrioritizer.new) resource.expects(:retrieve).raises "this is a failure" resource.stubs(:err) @@ -49,7 +49,7 @@ describe Puppet::Transaction do resource.virtual = true catalog.add_resource resource - transaction = Puppet::Transaction.new(catalog) + transaction = Puppet::Transaction.new(catalog, nil, Puppet::Graph::RandomPrioritizer.new) resource.expects(:evaluate).never @@ -74,7 +74,7 @@ describe Puppet::Transaction do resource.virtual = true catalog.add_resource resource - transaction = Puppet::Transaction.new(catalog) + transaction = Puppet::Transaction.new(catalog, nil, Puppet::Graph::RandomPrioritizer.new) resource.expects(:evaluate).never @@ -86,7 +86,7 @@ describe Puppet::Transaction do resource = Puppet::Type.type(:interface).new :name => "FastEthernet 0/1" catalog.add_resource resource - transaction = Puppet::Transaction.new(catalog) + transaction = Puppet::Transaction.new(catalog, nil, Puppet::Graph::RandomPrioritizer.new) transaction.for_network_device = false transaction.expects(:apply).never.with(resource, nil) @@ -100,7 +100,7 @@ describe Puppet::Transaction do resource = Puppet::Type.type(:file).new :path => make_absolute("/foo/bar"), :backup => false catalog.add_resource resource - transaction = Puppet::Transaction.new(catalog) + transaction = Puppet::Transaction.new(catalog, nil, Puppet::Graph::RandomPrioritizer.new) transaction.for_network_device = true transaction.expects(:apply).never.with(resource, nil) @@ -114,7 +114,7 @@ describe Puppet::Transaction do resource = Puppet::Type.type(:interface).new :name => "FastEthernet 0/1" catalog.add_resource resource - transaction = Puppet::Transaction.new(catalog) + transaction = Puppet::Transaction.new(catalog, nil, Puppet::Graph::RandomPrioritizer.new) transaction.for_network_device = true transaction.expects(:apply).with(resource, nil) @@ -128,7 +128,7 @@ describe Puppet::Transaction do resource = Puppet::Type.type(:schedule).new :name => "test" catalog.add_resource resource - transaction = Puppet::Transaction.new(catalog) + transaction = Puppet::Transaction.new(catalog, nil, Puppet::Graph::RandomPrioritizer.new) transaction.for_network_device = true transaction.expects(:apply).with(resource, nil) @@ -341,6 +341,6 @@ describe Puppet::Transaction do trans = catalog.apply - trans.resource_harness.should be_scheduled(trans.resource_status(resource), resource) + trans.resource_harness.should be_scheduled(resource) end end diff --git a/spec/integration/type/file_spec.rb b/spec/integration/type/file_spec.rb index 65206b377..6fa71a11d 100755 --- a/spec/integration/type/file_spec.rb +++ b/spec/integration/type/file_spec.rb @@ -454,7 +454,7 @@ describe Puppet::Type.type(:file) do catalog.apply File.readlink(link).should == dest2 - Find.find(bucket[:path]) { |f| File.file?(f) }.should be_nil + File.exist?(bucket[:path]).should be_false end it "should backup directories to the local filesystem by copying the whole directory" do diff --git a/spec/integration/util/settings_spec.rb b/spec/integration/util/settings_spec.rb index ef06d21d9..360b78310 100755 --- a/spec/integration/util/settings_spec.rb +++ b/spec/integration/util/settings_spec.rb @@ -10,23 +10,27 @@ describe Puppet::Settings do { :noop => {:default => false, :desc => "noop"} } end + def define_settings(section, settings_hash) + settings.define_settings(section, minimal_default_settings.update(settings_hash)) + end + + let(:settings) { Puppet::Settings.new } + it "should be able to make needed directories" do - settings = Puppet::Settings.new - settings.define_settings :main, minimal_default_settings.update( - :maindir => { - :default => tmpfile("main"), - :type => :directory, - :desc => "a", - } + define_settings(:main, + :maindir => { + :default => tmpfile("main"), + :type => :directory, + :desc => "a", + } ) settings.use(:main) - File.should be_directory(settings[:maindir]) + expect(File.directory?(settings[:maindir])).to be_true end it "should make its directories with the correct modes" do - settings = Puppet::Settings.new - settings.define_settings :main, minimal_default_settings.update( + define_settings(:main, :maindir => { :default => tmpfile("main"), :type => :directory, @@ -37,6 +41,49 @@ describe Puppet::Settings do settings.use(:main) - (File.stat(settings[:maindir]).mode & 007777).should == (Puppet.features.microsoft_windows? ? 0755 : 0750) + expect(File.stat(settings[:maindir]).mode & 007777).to eq(Puppet.features.microsoft_windows? ? 0755 : 0750) + end + + it "reparses configuration if configuration file is touched", :if => !Puppet.features.microsoft_windows? do + config = tmpfile("config") + define_settings(:main, + :config => { + :type => :file, + :default => config, + :desc => "a" + }, + :environment => { + :default => 'dingos', + :desc => 'test', + } + ) + + Puppet[:filetimeout] = '1s' + + File.open(config, 'w') do |file| + file.puts <<-EOF +[main] +environment=toast + EOF + end + + settings.initialize_global_settings + expect(settings[:environment]).to eq('toast') + + # First reparse establishes WatchedFiles + settings.reparse_config_files + + sleep 1 + + File.open(config, 'w') do |file| + file.puts <<-EOF +[main] +environment=bacon + EOF + end + + # Second reparse if later than filetimeout, reparses if changed + settings.reparse_config_files + expect(settings[:environment]).to eq('bacon') end end diff --git a/spec/lib/matchers/include_in_order.rb b/spec/lib/matchers/include_in_order.rb new file mode 100644 index 000000000..56e4e41c7 --- /dev/null +++ b/spec/lib/matchers/include_in_order.rb @@ -0,0 +1,21 @@ +RSpec::Matchers.define :include_in_order do |*expected| + include RSpec::Matchers::Pretty + + match do |actual| + elements = expected.dup + actual.each do |elt| + if elt == elements.first + elements.shift + end + end + elements.empty? + end + + def failure_message_for_should + "expected #{@actual.inspect} to include#{expected_to_sentence} in order" + end + + def failure_message_for_should_not + "expected #{@actual.inspect} not to include#{expected_to_sentence} in order" + end +end diff --git a/spec/lib/matchers/include_in_order_spec.rb b/spec/lib/matchers/include_in_order_spec.rb new file mode 100644 index 000000000..cd7d9a76d --- /dev/null +++ b/spec/lib/matchers/include_in_order_spec.rb @@ -0,0 +1,30 @@ +require 'spec_helper' +require 'matchers/include_in_order' + +describe "Matching whether elements are included in order" do + context "an empty array" do + it "is included in an empty array" do + expect([]).to include_in_order() + end + + it "is included in a non-empty array" do + expect([1]).to include_in_order() + end + end + + it "[1,2,3] is included in [0,1,2,3,4]" do + expect([0,1,2,3,4]).to include_in_order(1,2,3) + end + + it "[2,1] is not included in order in [1,2]" do + expect([1,2]).not_to include_in_order(2,1) + end + + it "[2,4,6] is included in order in [1,2,3,4,5,6]" do + expect([1,2,3,4,5,6]).to include_in_order(2,4,6) + end + + it "overlapping ordered array is not included" do + expect([1,2,3]).not_to include_in_order(2,3,4) + end +end diff --git a/spec/lib/matchers/relationship_graph_matchers.rb b/spec/lib/matchers/relationship_graph_matchers.rb new file mode 100644 index 000000000..3d08cdd87 --- /dev/null +++ b/spec/lib/matchers/relationship_graph_matchers.rb @@ -0,0 +1,48 @@ +module RelationshipGraphMatchers + class EnforceOrderWithEdge + def initialize(before, after) + @before = before + @after = after + end + + def matches?(actual_graph) + @actual_graph = actual_graph + + @reverse_edge = actual_graph.edge?( + vertex_called(actual_graph, @after), + vertex_called(actual_graph, @before)) + + @forward_edge = actual_graph.edge?( + vertex_called(actual_graph, @before), + vertex_called(actual_graph, @after)) + + @forward_edge && !@reverse_edge + end + + def failure_message_for_should + "expect #{@actual_graph.to_dot_graph} to only contain an edge from #{@before} to #{@after} but #{[forward_failure_message, reverse_failure_message].compact.join(' and ')}" + end + + def forward_failure_message + if !@forward_edge + "did not contain an edge from #{@before} to #{@after}" + end + end + + def reverse_failure_message + if @reverse_edge + "contained an edge from #{@after} to #{@before}" + end + end + + private + + def vertex_called(graph, name) + graph.vertices.find { |v| v.ref =~ /#{Regexp.escape(name)}/ } + end + end + + def enforce_order_with_edge(before, after) + EnforceOrderWithEdge.new(before, after) + end +end diff --git a/spec/lib/puppet_spec/compiler.rb b/spec/lib/puppet_spec/compiler.rb index 949c826e6..0286fbc8e 100644 --- a/spec/lib/puppet_spec/compiler.rb +++ b/spec/lib/puppet_spec/compiler.rb @@ -3,4 +3,28 @@ module PuppetSpec::Compiler Puppet[:code] = string Puppet::Parser::Compiler.compile(node) end + + def compile_to_ral(manifest) + catalog = compile_to_catalog(manifest) + ral = catalog.to_ral + ral.finalize + ral + end + + def compile_to_relationship_graph(manifest, prioritizer = Puppet::Graph::SequentialPrioritizer.new) + ral = compile_to_ral(manifest) + graph = Puppet::Graph::RelationshipGraph.new(prioritizer) + graph.populate_from(ral) + graph + end + + def apply_compiled_manifest(manifest, prioritizer = Puppet::Graph::SequentialPrioritizer.new) + transaction = Puppet::Transaction.new(compile_to_ral(manifest), + Puppet::Transaction::Report.new("apply"), + prioritizer) + transaction.evaluate + transaction.report.finalize_report + + transaction + end end diff --git a/spec/lib/puppet_spec/pops.rb b/spec/lib/puppet_spec/pops.rb new file mode 100644 index 000000000..442c85ba6 --- /dev/null +++ b/spec/lib/puppet_spec/pops.rb @@ -0,0 +1,16 @@ +module PuppetSpec::Pops + extend RSpec::Matchers::DSL + + # Checks if an Acceptor has a specific issue in its list of diagnostics + matcher :have_issue do |expected| + match do |actual| + actual.diagnostics.index { |i| i.issue == expected } != nil + end + failure_message_for_should do |actual| + "expected Acceptor[#{actual.diagnostics.collect { |i| i.issue.issue_code }.join(',')}] to contain issue #{expected.issue_code}" + end + failure_message_for_should_not do |actual| + "expected Acceptor[#{actual.diagnostics.collect { |i| i.issue.issue_code }.join(',')}] to not contain issue #{expected.issue_code}" + end + end +end diff --git a/spec/monkey_patches/publicize_methods.rb b/spec/monkey_patches/publicize_methods.rb deleted file mode 100755 index b39e9c002..000000000 --- a/spec/monkey_patches/publicize_methods.rb +++ /dev/null @@ -1,11 +0,0 @@ -# Some monkey-patching to allow us to test private methods. -class Class - def publicize_methods(*methods) - saved_private_instance_methods = methods.empty? ? self.private_instance_methods : methods - - self.class_eval { public(*saved_private_instance_methods) } - yield - self.class_eval { private(*saved_private_instance_methods) } - end -end - diff --git a/spec/spec_helper.rb b/spec/spec_helper.rb index 14d2b0465..7b4bd7219 100755 --- a/spec/spec_helper.rb +++ b/spec/spec_helper.rb @@ -33,7 +33,6 @@ require 'puppet_spec/fixtures' require 'puppet_spec/matchers' require 'puppet_spec/database' require 'monkey_patches/alias_should_to_must' -require 'monkey_patches/publicize_methods' require 'puppet/test/test_helper' Pathname.glob("#{dir}/shared_contexts/*.rb") do |file| diff --git a/spec/unit/application/agent_spec.rb b/spec/unit/application/agent_spec.rb index 9bcbad6ee..2ec32dbfb 100755 --- a/spec/unit/application/agent_spec.rb +++ b/spec/unit/application/agent_spec.rb @@ -11,18 +11,28 @@ describe Puppet::Application::Agent do before :each do @puppetd = Puppet::Application[:agent] - @puppetd.stubs(:puts) - @daemon = stub_everything 'daemon' + + @daemon = Puppet::Daemon.new(nil) + @daemon.stubs(:daemonize) + @daemon.stubs(:start) + @daemon.stubs(:stop) Puppet::Daemon.stubs(:new).returns(@daemon) Puppet[:daemonize] = false + @agent = stub_everything 'agent' Puppet::Agent.stubs(:new).returns(@agent) + @puppetd.preinit Puppet::Util::Log.stubs(:newdestination) + @ssl_host = stub_everything 'ssl host' + Puppet::SSL::Host.stubs(:new).returns(@ssl_host) + Puppet::Node.indirection.stubs(:terminus_class=) Puppet::Node.indirection.stubs(:cache_class=) Puppet::Node::Facts.indirection.stubs(:terminus_class=) + + $stderr.expects(:puts).never end it "should operate in agent run_mode" do @@ -94,75 +104,80 @@ describe Puppet::Application::Agent do @puppetd.command_line.stubs(:args).returns([]) end - [:centrallogging, :enable, :debug, :fqdn, :test, :verbose, :digest].each do |option| + [:enable, :debug, :fqdn, :test, :verbose, :digest].each do |option| it "should declare handle_#{option} method" do @puppetd.should respond_to("handle_#{option}".to_sym) end it "should store argument value when calling handle_#{option}" do - @puppetd.options.expects(:[]=).with(option, 'arg') @puppetd.send("handle_#{option}".to_sym, 'arg') + + @puppetd.options[option].should == 'arg' end end describe "when handling --disable" do - it "should declare handle_disable method" do - @puppetd.should respond_to(:handle_disable) - end - it "should set disable to true" do - @puppetd.options.stubs(:[]=) - @puppetd.options.expects(:[]=).with(:disable, true) @puppetd.handle_disable('') + + @puppetd.options[:disable].should == true end it "should store disable message" do - @puppetd.options.stubs(:[]=) - @puppetd.options.expects(:[]=).with(:disable_message, "message") @puppetd.handle_disable('message') + + @puppetd.options[:disable_message].should == 'message' end end it "should set client to false with --no-client" do @puppetd.handle_no_client(nil) + @puppetd.options[:client].should be_false end it "should set waitforcert to 0 with --onetime and if --waitforcert wasn't given" do + @agent.stubs(:run).returns(2) Puppet[:onetime] = true - Puppet::SSL::Host.any_instance.expects(:wait_for_cert).with(0) - @puppetd.setup_host + + @ssl_host.expects(:wait_for_cert).with(0) + + expect { execute_agent }.to exit_with 0 end it "should use supplied waitforcert when --onetime is specified" do + @agent.stubs(:run).returns(2) Puppet[:onetime] = true @puppetd.handle_waitforcert(60) - Puppet::SSL::Host.any_instance.expects(:wait_for_cert).with(60) - @puppetd.setup_host + + @ssl_host.expects(:wait_for_cert).with(60) + + expect { execute_agent }.to exit_with 0 end it "should use a default value for waitforcert when --onetime and --waitforcert are not specified" do - Puppet::SSL::Host.any_instance.expects(:wait_for_cert).with(120) - @puppetd.setup_host + @ssl_host.expects(:wait_for_cert).with(120) + + execute_agent end it "should use the waitforcert setting when checking for a signed certificate" do Puppet[:waitforcert] = 10 - Puppet::SSL::Host.any_instance.expects(:wait_for_cert).with(10) - @puppetd.setup_host + @ssl_host.expects(:wait_for_cert).with(10) + + execute_agent end it "should set the log destination with --logdest" do - @puppetd.options.stubs(:[]=).with { |opt,val| opt == :setdest } Puppet::Log.expects(:newdestination).with("console") @puppetd.handle_logdest("console") end it "should put the setdest options to true" do - @puppetd.options.expects(:[]=).with(:setdest,true) - @puppetd.handle_logdest("console") + + @puppetd.options[:setdest].should == true end it "should parse the log destination from the command line" do @@ -174,50 +189,37 @@ describe Puppet::Application::Agent do end it "should store the waitforcert options with --waitforcert" do - @puppetd.options.expects(:[]=).with(:waitforcert,42) - @puppetd.handle_waitforcert("42") - end - it "should set args[:Port] with --port" do - @puppetd.handle_port("42") - @puppetd.args[:Port].should == "42" + @puppetd.options[:waitforcert].should == 42 end - end describe "during setup" do before :each do - @puppetd.options.stubs(:[]) Puppet.stubs(:info) FileTest.stubs(:exists?).returns(true) Puppet[:libdir] = "/dev/null/lib" - Puppet::SSL::Host.stubs(:ca_location=) Puppet::Transaction::Report.indirection.stubs(:terminus_class=) Puppet::Transaction::Report.indirection.stubs(:cache_class=) Puppet::Resource::Catalog.indirection.stubs(:terminus_class=) Puppet::Resource::Catalog.indirection.stubs(:cache_class=) Puppet::Node::Facts.indirection.stubs(:terminus_class=) - @host = stub_everything 'host' - Puppet::SSL::Host.stubs(:new).returns(@host) Puppet.stubs(:settraps) end describe "with --test" do - before :each do - #Puppet.settings.stubs(:handlearg) - @puppetd.options.stubs(:[]=) - end - it "should call setup_test" do - @puppetd.options.stubs(:[]).with(:test).returns(true) + @puppetd.options[:test] = true @puppetd.expects(:setup_test) + @puppetd.setup end it "should set options[:verbose] to true" do - @puppetd.options.expects(:[]=).with(:verbose,true) @puppetd.setup_test + + @puppetd.options[:verbose].should == true end it "should set options[:onetime] to true" do Puppet[:onetime] = false @@ -225,8 +227,9 @@ describe Puppet::Application::Agent do Puppet[:onetime].should == true end it "should set options[:detailed_exitcodes] to true" do - @puppetd.options.expects(:[]=).with(:detailed_exitcodes,true) @puppetd.setup_test + + @puppetd.options[:detailed_exitcodes].should == true end end @@ -241,29 +244,30 @@ describe Puppet::Application::Agent do end it "should set log level to debug if --debug was passed" do - @puppetd.options.stubs(:[]).with(:debug).returns(true) + @puppetd.options[:debug] = true @puppetd.setup_logs Puppet::Util::Log.level.should == :debug end it "should set log level to info if --verbose was passed" do - @puppetd.options.stubs(:[]).with(:verbose).returns(true) + @puppetd.options[:verbose] = true @puppetd.setup_logs Puppet::Util::Log.level.should == :info end [:verbose, :debug].each do |level| it "should set console as the log destination with level #{level}" do - @puppetd.options.stubs(:[]).with(level).returns(true) + @puppetd.options[level] = true - Puppet::Util::Log.expects(:newdestination).with(:console) + Puppet::Util::Log.expects(:newdestination).at_least_once + Puppet::Util::Log.expects(:newdestination).with(:console).once @puppetd.setup_logs end end it "should set a default log destination if no --logdest" do - @puppetd.options.stubs(:[]).with(:setdest).returns(false) + @puppetd.options[:setdest] = false Puppet::Util::Log.expects(:setup_default) @@ -275,7 +279,7 @@ describe Puppet::Application::Agent do it "should print puppet config if asked to in Puppet config" do Puppet[:configprint] = "pluginsync" Puppet.settings.expects(:print_configs).returns true - expect { @puppetd.setup }.to exit_with 0 + expect { execute_agent }.to exit_with 0 end it "should exit after printing puppet config if asked to in Puppet config" do @@ -283,17 +287,7 @@ describe Puppet::Application::Agent do Puppet[:modulepath] = path Puppet[:configprint] = "modulepath" Puppet::Settings.any_instance.expects(:puts).with(path) - expect { @puppetd.setup }.to exit_with 0 - end - - it "should set a central log destination with --centrallogs" do - @puppetd.options.stubs(:[]).with(:centrallogs).returns(true) - Puppet[:server] = "puppet.reductivelabs.com" - Puppet::Util::Log.stubs(:setup_default) - - Puppet::Util::Log.expects(:newdestination).with("puppet.reductivelabs.com") - - @puppetd.setup + expect { execute_agent }.to exit_with 0 end it "should use :main, :puppetd, and :ssl" do @@ -309,7 +303,7 @@ describe Puppet::Application::Agent do end it "should install a none ca location in fingerprint mode" do - @puppetd.options.stubs(:[]).with(:fingerprint).returns(true) + @puppetd.options[:fingerprint] = true Puppet::SSL::Host.expects(:ca_location=).with(:none) @puppetd.setup @@ -373,7 +367,7 @@ describe Puppet::Application::Agent do [:enable, :disable].each do |action| it "should delegate to enable_disable_client if we #{action} the agent" do - @puppetd.options.stubs(:[]).with(action).returns(true) + @puppetd.options[action] = true @puppetd.expects(:enable_disable_client).with(@agent) @puppetd.setup @@ -383,41 +377,43 @@ describe Puppet::Application::Agent do describe "when enabling or disabling agent" do [:enable, :disable].each do |action| it "should call client.#{action}" do - @puppetd.options.stubs(:[]).with(action).returns(true) + @puppetd.options[action] = true @agent.expects(action) - expect { @puppetd.enable_disable_client(@agent) }.to exit_with 0 + expect { execute_agent }.to exit_with 0 end end it "should pass the disable message when disabling" do - @puppetd.options.stubs(:[]).with(:disable).returns(true) - @puppetd.options.stubs(:[]).with(:disable_message).returns("message") + @puppetd.options[:disable] = true + @puppetd.options[:disable_message] = "message" @agent.expects(:disable).with("message") - expect { @puppetd.enable_disable_client(@agent) }.to exit_with 0 + + expect { execute_agent }.to exit_with 0 end it "should pass the default disable message when disabling without a message" do - @puppetd.options.stubs(:[]).with(:disable).returns(true) - @puppetd.options.stubs(:[]).with(:disable_message).returns(nil) + @puppetd.options[:disable] = true + @puppetd.options[:disable_message] = nil @agent.expects(:disable).with("reason not specified") - expect { @puppetd.enable_disable_client(@agent) }.to exit_with 0 - end - it "should finally exit" do - expect { @puppetd.enable_disable_client(@agent) }.to exit_with 0 + expect { execute_agent }.to exit_with 0 end end it "should inform the daemon about our agent if :client is set to 'true'" do - @puppetd.options.expects(:[]).with(:client).returns true - @daemon.expects(:agent=).with(@agent) - @puppetd.setup + @puppetd.options[:client] = true + + execute_agent + + @daemon.agent.should == @agent end it "should not inform the daemon about our agent if :client is set to 'false'" do @puppetd.options[:client] = false - @daemon.expects(:agent=).never - @puppetd.setup + + execute_agent + + @daemon.agent.should be_nil end it "should daemonize if needed" do @@ -426,66 +422,72 @@ describe Puppet::Application::Agent do @daemon.expects(:daemonize) - @puppetd.setup + execute_agent end it "should wait for a certificate" do - @puppetd.options.stubs(:[]).with(:waitforcert).returns(123) - @host.expects(:wait_for_cert).with(123) + @puppetd.options[:waitforcert] = 123 + @ssl_host.expects(:wait_for_cert).with(123) - @puppetd.setup + execute_agent end it "should not wait for a certificate in fingerprint mode" do - @puppetd.options.stubs(:[]).with(:fingerprint).returns(true) - @puppetd.options.stubs(:[]).with(:waitforcert).returns(123) - @host.expects(:wait_for_cert).never - - @puppetd.setup - end + @puppetd.options[:fingerprint] = true + @puppetd.options[:waitforcert] = 123 + @puppetd.options[:digest] = 'MD5' - it "should setup listen if told to and not onetime" do - Puppet[:listen] = true - @puppetd.options.stubs(:[]).with(:onetime).returns(false) + certificate = mock 'certificate' + certificate.stubs(:digest).with('MD5').returns('ABCDE') + @ssl_host.stubs(:certificate).returns(certificate) - @puppetd.expects(:setup_listen) + @ssl_host.expects(:wait_for_cert).never + @puppetd.expects(:puts).with('ABCDE') - @puppetd.setup + execute_agent end describe "when setting up listen" do before :each do FileTest.stubs(:exists?).with('auth').returns(true) File.stubs(:exist?).returns(true) - @puppetd.options.stubs(:[]).with(:serve).returns([]) + @puppetd.options[:serve] = [] @server = stub_everything 'server' Puppet::Network::Server.stubs(:new).returns(@server) end it "should exit if no authorization file" do + Puppet[:listen] = true Puppet.stubs(:err) FileTest.stubs(:exists?).with(Puppet[:rest_authconfig]).returns(false) - expect { @puppetd.setup_listen }.to exit_with 14 + + expect do + execute_agent + end.to exit_with 14 end it "should use puppet default port" do Puppet[:puppetport] = 32768 + Puppet[:listen] = true Puppet::Network::Server.expects(:new).with(anything, 32768) - @puppetd.setup_listen + execute_agent end - + it "should issue a warning that listen is deprecated" do + Puppet[:listen] = true + Puppet.expects(:warning).with() { |msg| msg =~ /kick is deprecated/ } - @puppetd.setup_listen + + execute_agent end end describe "when setting up for fingerprint" do before(:each) do - @puppetd.options.stubs(:[]).with(:fingerprint).returns(true) + @puppetd.options[:fingerprint] = true end it "should not setup as an agent" do @@ -502,11 +504,6 @@ describe Puppet::Application::Agent do @daemon.expects(:daemonize).never @puppetd.setup end - - it "should setup our certificate host" do - @puppetd.expects(:setup_host) - @puppetd.setup - end end describe "when configuring agent for catalog run" do @@ -526,114 +523,112 @@ describe Puppet::Application::Agent do describe "when running" do before :each do - @puppetd.agent = @agent - @puppetd.daemon = @daemon - @puppetd.options.stubs(:[]).with(:fingerprint).returns(false) + @puppetd.options[:fingerprint] = false end it "should dispatch to fingerprint if --fingerprint is used" do - @puppetd.options.stubs(:[]).with(:fingerprint).returns(true) + @puppetd.options[:fingerprint] = true @puppetd.stubs(:fingerprint) - @puppetd.run_command + + execute_agent end it "should dispatch to onetime if --onetime is used" do - @puppetd.options.stubs(:[]).with(:onetime).returns(true) + @puppetd.options[:onetime] = true @puppetd.stubs(:onetime) - @puppetd.run_command + + execute_agent end it "should dispatch to main if --onetime and --fingerprint are not used" do - @puppetd.options.stubs(:[]).with(:onetime).returns(false) + @puppetd.options[:onetime] = false @puppetd.stubs(:main) - @puppetd.run_command + + execute_agent end describe "with --onetime" do before :each do @agent.stubs(:run).returns(:report) - @puppetd.options.stubs(:[]).with(:client).returns(:client) - @puppetd.options.stubs(:[]).with(:detailed_exitcodes).returns(false) + Puppet[:onetime] = true + @puppetd.options[:client] = :client + @puppetd.options[:detailed_exitcodes] = false Puppet.stubs(:newservice) end it "should exit if no defined --client" do - $stderr.stubs(:puts) - @puppetd.options.stubs(:[]).with(:client).returns(nil) - expect { @puppetd.onetime }.to exit_with 43 + @puppetd.options[:client] = nil + + Puppet.expects(:err).with('onetime is specified but there is no client') + + expect { execute_agent }.to exit_with 43 end it "should setup traps" do @daemon.expects(:set_signal_traps) - expect { @puppetd.onetime }.to exit_with 0 + + expect { execute_agent }.to exit_with 0 end it "should let the agent run" do @agent.expects(:run).returns(:report) - expect { @puppetd.onetime }.to exit_with 0 - end - it "should finish by exiting with 0 error code" do - expect { @puppetd.onetime }.to exit_with 0 + expect { execute_agent }.to exit_with 0 end it "should stop the daemon" do @daemon.expects(:stop).with(:exit => false) - expect { @puppetd.onetime }.to exit_with 0 + + expect { execute_agent }.to exit_with 0 end describe "and --detailed-exitcodes" do before :each do - @puppetd.options.stubs(:[]).with(:detailed_exitcodes).returns(true) + @puppetd.options[:detailed_exitcodes] = true end it "should exit with agent computed exit status" do Puppet[:noop] = false @agent.stubs(:run).returns(666) - expect { @puppetd.onetime }.to exit_with 666 + expect { execute_agent }.to exit_with 666 end it "should exit with the agent's exit status, even if --noop is set." do Puppet[:noop] = true @agent.stubs(:run).returns(666) - expect { @puppetd.onetime }.to exit_with 666 + expect { execute_agent }.to exit_with 666 end end end describe "with --fingerprint" do before :each do - @cert = stub_everything 'cert' - @puppetd.options.stubs(:[]).with(:fingerprint).returns(true) - @puppetd.options.stubs(:[]).with(:digest).returns(:MD5) - @host = stub_everything 'host' - @puppetd.stubs(:host).returns(@host) + @cert = mock 'cert' + @puppetd.options[:fingerprint] = true + @puppetd.options[:digest] = :MD5 end it "should fingerprint the certificate if it exists" do - @host.expects(:certificate).returns(@cert) - @cert.expects(:digest).with('MD5').returns "fingerprint" - @puppetd.fingerprint - end + @ssl_host.stubs(:certificate).returns(@cert) + @cert.stubs(:digest).with('MD5').returns "fingerprint" + + @puppetd.expects(:puts).with "fingerprint" - it "should fingerprint the certificate request if no certificate have been signed" do - @host.expects(:certificate).returns(nil) - @host.expects(:certificate_request).returns(@cert) - @cert.expects(:digest).with('MD5').returns "fingerprint" @puppetd.fingerprint end - it "should display the fingerprint" do - @host.stubs(:certificate).returns(@cert) - @cert.stubs(:digest).with('MD5').returns("DIGEST") + it "should fingerprint the certificate request if no certificate have been signed" do + @ssl_host.stubs(:certificate).returns(nil) + @ssl_host.stubs(:certificate_request).returns(@cert) + @cert.stubs(:digest).with('MD5').returns "fingerprint" - @puppetd.expects(:puts).with "DIGEST" + @puppetd.expects(:puts).with "fingerprint" @puppetd.fingerprint end @@ -642,14 +637,19 @@ describe Puppet::Application::Agent do describe "without --onetime and --fingerprint" do before :each do Puppet.stubs(:notice) - @puppetd.options.stubs(:[]).with(:client) + @puppetd.options[:client] = nil end it "should start our daemon" do @daemon.expects(:start) - @puppetd.main + execute_agent end end end + + def execute_agent + @puppetd.setup + @puppetd.run_command + end end diff --git a/spec/unit/application/apply_spec.rb b/spec/unit/application/apply_spec.rb index eed6d41de..97728c06e 100755 --- a/spec/unit/application/apply_spec.rb +++ b/spec/unit/application/apply_spec.rb @@ -148,7 +148,7 @@ describe Puppet::Application::Apply do STDIN.stubs(:read) - @transaction = Puppet::Transaction.new(@catalog) + @transaction = stub('transaction') @catalog.stubs(:apply).returns(@transaction) Puppet::Util::Storage.stubs(:load) diff --git a/spec/unit/application/doc_spec.rb b/spec/unit/application/doc_spec.rb index 8528c3a49..e2dc0fe69 100755 --- a/spec/unit/application/doc_spec.rb +++ b/spec/unit/application/doc_spec.rb @@ -13,7 +13,7 @@ describe Puppet::Application::Doc do Puppet::Util::Log.stubs(:newdestination) end - it "should declare a other command" do + it "should declare an other command" do @doc.should respond_to(:other) end diff --git a/spec/unit/application/face_base_spec.rb b/spec/unit/application/face_base_spec.rb index 88b9aeb07..9fdd8e843 100755 --- a/spec/unit/application/face_base_spec.rb +++ b/spec/unit/application/face_base_spec.rb @@ -33,7 +33,7 @@ describe Puppet::Application::FaceBase do end describe "with just an action" do - before :all do + before(:each) do # We have to stub Signal.trap to avoid a crazy mess where we take # over signal handling and make it impossible to cancel the test # suite run. @@ -304,8 +304,8 @@ describe Puppet::Application::FaceBase do end [[1, 2], ["one"], [{ 1 => 1 }]].each do |input| - it "should render #{input.class} using JSON" do - app.render(input, {}).should == input.to_pson.chomp + it "should render Array as one item per line" do + app.render(input, {}).should == input.collect { |item| item.to_s + "\n" }.join('') end end diff --git a/spec/unit/application/facts_spec.rb b/spec/unit/application/facts_spec.rb index 1e60e1399..c648c5891 100755 --- a/spec/unit/application/facts_spec.rb +++ b/spec/unit/application/facts_spec.rb @@ -16,6 +16,7 @@ describe Puppet::Application::Facts do end it "should return facts if a key is given to find" do + Puppet[:stringify_facts] = false Puppet::Node::Facts.indirection.reset_terminus_class Puppet::Node::Facts.indirection.expects(:find).returns(Puppet::Node::Facts.new('whatever', {})) subject.command_line.stubs(:args).returns %w{find whatever --render-as yaml} diff --git a/spec/unit/application/master_spec.rb b/spec/unit/application/master_spec.rb index 579303907..d7f26b74a 100755 --- a/spec/unit/application/master_spec.rb +++ b/spec/unit/application/master_spec.rb @@ -47,21 +47,6 @@ describe Puppet::Application::Master, :unless => Puppet.features.microsoft_windo @master.preinit end - - it "should create a Puppet Daemon" do - Puppet::Daemon.expects(:new).returns(@daemon) - - @master.preinit - end - - it "should give ARGV to the Daemon" do - argv = stub 'argv' - ARGV.stubs(:dup).returns(argv) - @daemon.expects(:argv=).with(argv) - - @master.preinit - end - end [:debug,:verbose].each do |option| diff --git a/spec/unit/application/queue_spec.rb b/spec/unit/application/queue_spec.rb index 64fd44650..64c9a9708 100755 --- a/spec/unit/application/queue_spec.rb +++ b/spec/unit/application/queue_spec.rb @@ -8,7 +8,6 @@ describe Puppet::Application::Queue, :unless => Puppet.features.microsoft_window before :each do @queue = Puppet::Application[:queue] @queue.stubs(:puts) - @daemon = stub_everything 'daemon' Puppet::Util::Log.stubs(:newdestination) Puppet::Resource::Catalog.indirection.stubs(:terminus_class=) @@ -40,14 +39,6 @@ describe Puppet::Application::Queue, :unless => Puppet.features.microsoft_window @queue.options[:debug].should be_false end - - it "should create a Daemon instance and copy ARGV to it" do - ARGV.expects(:dup).returns "eh" - daemon = mock("daemon") - daemon.expects(:argv=).with("eh") - Puppet::Daemon.expects(:new).returns daemon - @queue.preinit - end end describe "when handling options" do @@ -65,10 +56,13 @@ describe Puppet::Application::Queue, :unless => Puppet.features.microsoft_window end describe "during setup" do + let(:daemon) { stub("Daemon", :daemonize => nil, :argv= => []) } + before :each do + Puppet::Daemon.stubs(:new).returns(daemon) + @queue.preinit @queue.options.stubs(:[]) - @queue.daemon.stubs(:daemonize) Puppet.stubs(:info) Puppet.features.stubs(:stomp?).returns true Puppet::Resource::Catalog.indirection.stubs(:terminus_class=) @@ -79,7 +73,7 @@ describe Puppet::Application::Queue, :unless => Puppet.features.microsoft_window it "should fail if the stomp feature is missing" do Puppet.features.expects(:stomp?).returns false - lambda { @queue.setup }.should raise_error(ArgumentError) + expect { @queue.setup }.to raise_error(ArgumentError) end it "should issue a warning that queue is deprecated" do @@ -139,7 +133,7 @@ describe Puppet::Application::Queue, :unless => Puppet.features.microsoft_window it "should daemonize if needed" do Puppet[:daemonize] = true - @queue.daemon.expects(:daemonize) + daemon.expects(:daemonize) @queue.setup end diff --git a/spec/unit/application/resource_spec.rb b/spec/unit/application/resource_spec.rb index 748bb5f0d..f2e2596ca 100755 --- a/spec/unit/application/resource_spec.rb +++ b/spec/unit/application/resource_spec.rb @@ -36,7 +36,7 @@ describe Puppet::Application::Resource do @resource_app.host.should == :whatever end - it "should load an display all types with types option" do + it "should load a display all types with types option" do type1 = stub_everything 'type1', :name => :type1 type2 = stub_everything 'type2', :name => :type2 Puppet::Type.stubs(:loadall) diff --git a/spec/unit/configurer/fact_handler_spec.rb b/spec/unit/configurer/fact_handler_spec.rb index f8ba94247..a74a91dfa 100755 --- a/spec/unit/configurer/fact_handler_spec.rb +++ b/spec/unit/configurer/fact_handler_spec.rb @@ -5,19 +5,19 @@ require 'puppet/configurer/fact_handler' class FactHandlerTester include Puppet::Configurer::FactHandler + + def reload_facter + # don't want to do this in tests + end end describe Puppet::Configurer::FactHandler do - before do + before :each do @facthandler = FactHandlerTester.new + Puppet::Node::Facts.indirection.terminus_class = :memory end describe "when finding facts" do - before :each do - @facthandler.stubs(:reload_facter) - Puppet::Node::Facts.indirection.terminus_class = :memory - end - it "should use the node name value to retrieve the facts" do foo_facts = Puppet::Node::Facts.new('foo') bar_facts = Puppet::Node::Facts.new('bar') @@ -48,61 +48,30 @@ describe Puppet::Configurer::FactHandler do end it "should fail if finding facts fails" do - Puppet[:trace] = false - Puppet[:certname] = "myhost" Puppet::Node::Facts.indirection.expects(:find).raises RuntimeError - lambda { @facthandler.find_facts }.should raise_error(Puppet::Error) + expect { @facthandler.find_facts }.to raise_error(Puppet::Error, /Could not retrieve local facts/) end - end - it "should only load fact plugins once" do - Puppet::Node::Facts.indirection.expects(:find).once - @facthandler.find_facts + it "should only load fact plugins once" do + Puppet::Node::Facts.indirection.expects(:find).once + @facthandler.find_facts + end end - # I couldn't get marshal to work for this, only yaml, so we hard-code yaml. it "should serialize and CGI escape the fact values for uploading" do - facts = stub 'facts' - facts.expects(:support_format?).with(:b64_zlib_yaml).returns true - facts.expects(:render).returns "my text" - text = CGI.escape("my text") + facts = Puppet::Node::Facts.new(Puppet[:node_name_value], 'my_name_fact' => 'other_node_name') + Puppet::Node::Facts.indirection.save(facts) + text = CGI.escape(@facthandler.find_facts.render(:pson)) - @facthandler.expects(:find_facts).returns facts - - @facthandler.facts_for_uploading.should == {:facts_format => :b64_zlib_yaml, :facts => text} + @facthandler.facts_for_uploading.should == {:facts_format => :pson, :facts => text} end it "should properly accept facts containing a '+'" do - facts = stub 'facts' - facts.expects(:support_format?).with(:b64_zlib_yaml).returns true - facts.expects(:render).returns "my+text" - text = "my%2Btext" - - @facthandler.expects(:find_facts).returns facts - - @facthandler.facts_for_uploading.should == {:facts_format => :b64_zlib_yaml, :facts => text} - end - - it "use compressed yaml as the serialization if zlib is supported" do - facts = stub 'facts' - facts.expects(:support_format?).with(:b64_zlib_yaml).returns true - facts.expects(:render).with(:b64_zlib_yaml).returns "my text" - text = CGI.escape("my text") - - @facthandler.expects(:find_facts).returns facts - - @facthandler.facts_for_uploading - end - - it "should use yaml as the serialization if zlib is not supported" do - facts = stub 'facts' - facts.expects(:support_format?).with(:b64_zlib_yaml).returns false - facts.expects(:render).with(:yaml).returns "my text" - text = CGI.escape("my text") - - @facthandler.expects(:find_facts).returns facts + facts = Puppet::Node::Facts.new('foo', 'afact' => 'a+b') + Puppet::Node::Facts.indirection.save(facts) + text = CGI.escape(@facthandler.find_facts.render(:pson)) - @facthandler.facts_for_uploading + @facthandler.facts_for_uploading.should == {:facts_format => :pson, :facts => text} end end diff --git a/spec/unit/configurer_spec.rb b/spec/unit/configurer_spec.rb index d4377ccf5..cba1df584 100755 --- a/spec/unit/configurer_spec.rb +++ b/spec/unit/configurer_spec.rb @@ -104,7 +104,7 @@ describe Puppet::Configurer do @agent.run(:pluginsync => false) end - it "should carry on when it can't fetch it's node definition" do + it "should carry on when it can't fetch its node definition" do error = Net::HTTPError.new(400, 'dummy server communication error') Puppet::Node.indirection.expects(:find).raises(error) @agent.run.should == 0 @@ -137,20 +137,17 @@ describe Puppet::Configurer do it "should use the provided report if it was passed one" do report = Puppet::Transaction::Report.new("apply") - Puppet::Transaction::Report.expects(:new).never - @catalog.expects(:apply).with{|options| options[:report] == report} + @catalog.expects(:apply).with {|options| options[:report] == report} @agent.run(:report => report) end it "should set the report as a log destination" do report = Puppet::Transaction::Report.new("apply") - Puppet::Transaction::Report.expects(:new).returns report - Puppet::Util::Log.expects(:newdestination).with(report) - Puppet::Util::Log.expects(:close).with(report) + report.expects(:<<).with(instance_of(Puppet::Util::Log)).at_least_once - @agent.run + @agent.run(:report => report) end it "should retrieve the catalog" do @@ -480,6 +477,25 @@ describe Puppet::Configurer do Puppet.expects(:err) expect { @configurer.save_last_run_summary(@report) }.to_not raise_error end + + it "should create the last run file with the correct mode" do + Puppet.settings.setting(:lastrunfile).expects(:mode).returns('664') + @configurer.save_last_run_summary(@report) + + if Puppet::Util::Platform.windows? + require 'puppet/util/windows/security' + mode = Puppet::Util::Windows::Security.get_mode(Puppet[:lastrunfile]) + else + mode = File.stat(Puppet[:lastrunfile]).mode + end + (mode & 0777).should == 0664 + end + + it "should report invalid last run file permissions" do + Puppet.settings.setting(:lastrunfile).expects(:mode).returns('892') + Puppet.expects(:err).with(regexp_matches(/Could not save last run local report.*892 is invalid/)) + @configurer.save_last_run_summary(@report) + end end describe "when retrieving a catalog" do diff --git a/spec/unit/daemon_spec.rb b/spec/unit/daemon_spec.rb index dfb428467..b727d669c 100755 --- a/spec/unit/daemon_spec.rb +++ b/spec/unit/daemon_spec.rb @@ -19,24 +19,29 @@ end describe Puppet::Daemon, :unless => Puppet.features.microsoft_windows? do include PuppetSpec::Files - before do - # Forking agent not needed here - @agent = Puppet::Agent.new(TestClient.new, false) - @daemon = Puppet::Daemon.new - @daemon.stubs(:close_streams).returns nil - end + class RecordingScheduler + attr_reader :jobs - it "should be able to manage an agent" do - @daemon.should respond_to(:agent) + def run_loop(jobs) + @jobs = jobs + end end - it "should be able to manage a network server" do - @daemon.should respond_to(:server) + let(:server) { stub("Server", :start => nil, :wait_for_shutdown => nil) } + let(:agent) { Puppet::Agent.new(TestClient.new, false) } + + let(:pidfile) { stub("PidFile", :lock => true, :unlock => true, :file_path => 'fake.pid') } + let(:scheduler) { RecordingScheduler.new } + + let(:daemon) { Puppet::Daemon.new(pidfile, scheduler) } + + before do + daemon.stubs(:close_streams).returns nil end it "should reopen the Log logs when told to reopen logs" do Puppet::Util::Log.expects(:reopen) - @daemon.reopen_logs + daemon.reopen_logs end let(:server) { stub("Server", :start => nil, :wait_for_shutdown => nil) } @@ -50,52 +55,81 @@ describe Puppet::Daemon, :unless => Puppet.features.microsoft_windows? do Puppet.expects(:notice) - @daemon.expects(method) + daemon.expects(method) - @daemon.set_signal_traps + daemon.set_signal_traps end end end describe "when starting" do before do - @daemon.stubs(:create_pidfile) - @daemon.stubs(:set_signal_traps) - @daemon.stubs(:run_event_loop) + daemon.stubs(:set_signal_traps) end it "should fail if it has neither agent nor server" do - lambda { @daemon.start }.should raise_error(Puppet::DevError) + expect { daemon.start }.to raise_error(Puppet::DevError) end it "should create its pidfile" do - @daemon.agent = @agent + pidfile.expects(:lock).returns(true) - @daemon.expects(:create_pidfile) - @daemon.start + daemon.agent = agent + daemon.start + end + + it "should fail if it cannot lock" do + pidfile.expects(:lock).returns(false) + daemon.agent = agent + + expect { daemon.start }.to raise_error(RuntimeError, "Could not create PID file: #{pidfile.file_path}") end it "should start its server if one is configured" do - @daemon.server = server + daemon.server = server server.expects(:start) - @daemon.stubs(:server).returns server - @daemon.start + daemon.start + end + + it "disables the reparse of configs if the filetimeout is 0" do + Puppet[:filetimeout] = 0 + daemon.agent = agent + + daemon.start + + scheduler.jobs[0].should_not be_enabled + end + + it "disables the agent run when there is no agent" do + Puppet[:filetimeout] = 0 + daemon.server = server + + daemon.start + + scheduler.jobs[1].should_not be_enabled + end + + it "waits for the server to shutdown when there is one" do + daemon.server = server + + server.expects(:wait_for_shutdown) + + daemon.start end it "waits for the server to shutdown when there is one" do - @daemon.server = server + daemon.server = server server.expects(:wait_for_shutdown) - @daemon.start + daemon.start end end describe "when stopping" do before do - @daemon.stubs(:remove_pidfile) Puppet::Util::Log.stubs(:close_all) # to make the global safe to mock, set it to a subclass of itself, # then restore it in an after pass @@ -109,115 +143,57 @@ describe Puppet::Daemon, :unless => Puppet.features.microsoft_windows? do it "should stop its server if one is configured" do server.expects(:stop) - @daemon.stubs(:server).returns server - expect { @daemon.stop }.to exit_with 0 + + daemon.server = server + + expect { daemon.stop }.to exit_with 0 end it 'should request a stop from Puppet::Application' do Puppet::Application.expects(:stop!) - expect { @daemon.stop }.to exit_with 0 + expect { daemon.stop }.to exit_with 0 end it "should remove its pidfile" do - @daemon.expects(:remove_pidfile) - expect { @daemon.stop }.to exit_with 0 + pidfile.expects(:unlock) + + expect { daemon.stop }.to exit_with 0 end it "should close all logs" do Puppet::Util::Log.expects(:close_all) - expect { @daemon.stop }.to exit_with 0 + expect { daemon.stop }.to exit_with 0 end it "should exit unless called with ':exit => false'" do - expect { @daemon.stop }.to exit_with 0 + expect { daemon.stop }.to exit_with 0 end it "should not exit if called with ':exit => false'" do - @daemon.stop :exit => false - end - end - - describe "when creating its pidfile" do - it "should use an exclusive mutex" do - Puppet.run_mode.expects(:name).returns "me" - Puppet::Util.expects(:synchronize_on).with("me",Sync::EX) - @daemon.create_pidfile - end - - it "should lock the pidfile using the Pidlock class" do - pidfile = mock 'pidfile' - - Puppet.run_mode.expects(:name).returns "eh" - Puppet[:pidfile] = make_absolute("/my/file") - - Puppet::Util::Pidlock.expects(:new).with(make_absolute("/my/file")).returns pidfile - - pidfile.expects(:lock).returns true - @daemon.create_pidfile - end - - it "should fail if it cannot lock" do - pidfile = mock 'pidfile' - - Puppet.run_mode.expects(:name).returns "eh" - Puppet[:pidfile] = make_absolute("/my/file") - - Puppet::Util::Pidlock.expects(:new).with(make_absolute("/my/file")).returns pidfile - - pidfile.expects(:lock).returns false - - lambda { @daemon.create_pidfile }.should raise_error - end - end - - describe "when removing its pidfile" do - it "should use an exclusive mutex" do - Puppet.run_mode.expects(:name).returns "me" - - Puppet::Util.expects(:synchronize_on).with("me",Sync::EX) - - @daemon.remove_pidfile - end - - it "should do nothing if the pidfile is not present" do - pidfile = mock 'pidfile', :unlock => false - - Puppet[:pidfile] = make_absolute("/my/file") - Puppet::Util::Pidlock.expects(:new).with(make_absolute("/my/file")).returns pidfile - - @daemon.remove_pidfile - end - - it "should unlock the pidfile using the Pidlock class" do - pidfile = mock 'pidfile', :unlock => true - - Puppet[:pidfile] = make_absolute("/my/file") - Puppet::Util::Pidlock.expects(:new).with(make_absolute("/my/file")).returns pidfile - - @daemon.remove_pidfile + daemon.stop :exit => false end end describe "when reloading" do it "should do nothing if no agent is configured" do - @daemon.reload + daemon.reload end it "should do nothing if the agent is running" do - @agent.expects(:running?).returns true + agent.expects(:running?).returns true - @daemon.agent = @agent + daemon.agent = agent - @daemon.reload + daemon.reload end it "should run the agent if one is available and it is not running" do - @agent.expects(:running?).returns false - @agent.expects(:run).with({:splay => false}) + agent.expects(:running?).returns false + agent.expects(:run).with({:splay => false}) - @daemon.agent = @agent + daemon.agent = agent - @daemon.reload + daemon.reload end end @@ -232,48 +208,48 @@ describe Puppet::Daemon, :unless => Puppet.features.microsoft_windows? do it 'should set Puppet::Application.restart!' do Puppet::Application.expects(:restart!) - @daemon.stubs(:reexec) - @daemon.restart + daemon.stubs(:reexec) + daemon.restart end it "should reexec itself if no agent is available" do - @daemon.expects(:reexec) + daemon.expects(:reexec) - @daemon.restart + daemon.restart end it "should reexec itself if the agent is not running" do - @agent.expects(:running?).returns false - @daemon.agent = @agent - @daemon.expects(:reexec) + agent.expects(:running?).returns false + daemon.agent = agent + daemon.expects(:reexec) - @daemon.restart + daemon.restart end end describe "when reexecing it self" do before do - @daemon.stubs(:exec) - @daemon.stubs(:stop) + daemon.stubs(:exec) + daemon.stubs(:stop) end it "should fail if no argv values are available" do - @daemon.expects(:argv).returns nil - lambda { @daemon.reexec }.should raise_error(Puppet::DevError) + daemon.expects(:argv).returns nil + lambda { daemon.reexec }.should raise_error(Puppet::DevError) end it "should shut down without exiting" do - @daemon.argv = %w{foo} - @daemon.expects(:stop).with(:exit => false) + daemon.argv = %w{foo} + daemon.expects(:stop).with(:exit => false) - @daemon.reexec + daemon.reexec end it "should call 'exec' with the original executable and arguments" do - @daemon.argv = %w{foo} - @daemon.expects(:exec).with($0 + " foo") + daemon.argv = %w{foo} + daemon.expects(:exec).with($0 + " foo") - @daemon.reexec + daemon.reexec end end end diff --git a/spec/unit/defaults_spec.rb b/spec/unit/defaults_spec.rb new file mode 100644 index 000000000..f86283a64 --- /dev/null +++ b/spec/unit/defaults_spec.rb @@ -0,0 +1,44 @@ +require 'spec_helper' +require 'puppet/settings' + +describe "Defaults" do + describe ".default_diffargs" do + describe "on AIX" do + before(:each) do + Facter.stubs(:value).with(:kernel).returns("AIX") + end + describe "on 5.3" do + before(:each) do + Facter.stubs(:value).with(:kernelmajversion).returns("5300") + end + it "should be empty" do + Puppet.default_diffargs.should == "" + end + end + [ "", + nil, + "6300", + "7300", + ].each do |kernel_version| + describe "on kernel version #{kernel_version.inspect}" do + before(:each) do + Facter.stubs(:value).with(:kernelmajversion).returns(kernel_version) + end + + it "should be '-u'" do + Puppet.default_diffargs.should == "-u" + end + end + end + end + describe "on everything else" do + before(:each) do + Facter.stubs(:value).with(:kernel).returns("NOT_AIX") + end + + it "should be '-u'" do + Puppet.default_diffargs.should == "-u" + end + end + end +end diff --git a/spec/unit/face/node_spec.rb b/spec/unit/face/node_spec.rb index 3da1595cb..d7e06b7b6 100755 --- a/spec/unit/face/node_spec.rb +++ b/spec/unit/face/node_spec.rb @@ -67,8 +67,8 @@ describe Puppet::Face[:node, '0.0.1'] do it "should accept the option --unexport" do expect { - subject.help('hostname', :unexport => true) - }.to_not raise_error(ArgumentError) + subject.clean('hostname', :unexport => true) + }.to_not raise_error end context "clean action" do diff --git a/spec/unit/file_bucket/file_spec.rb b/spec/unit/file_bucket/file_spec.rb index be2ff73ff..9add6d52c 100755 --- a/spec/unit/file_bucket/file_spec.rb +++ b/spec/unit/file_bucket/file_spec.rb @@ -15,8 +15,18 @@ describe Puppet::FileBucket::File do let(:bucketdir) { Puppet[:bucketdir] = tmpdir('bucket') } let(:destdir) { File.join(bucketdir, "8/b/3/7/0/2/a/d/#{digest}") } - it "should have a to_s method to return the contents" do - Puppet::FileBucket::File.new(contents).to_s.should == contents + it "defines its supported format to be `:s`" do + expect(Puppet::FileBucket::File.supported_formats).to eq([:s]) + end + + it "serializes to `:s`" do + expect(Puppet::FileBucket::File.new(contents).to_s).to eq(contents) + end + + it "deserializes from `:s`" do + file = Puppet::FileBucket::File.from_s(contents) + + expect(file.contents).to eq(contents) end it "should raise an error if changing content" do @@ -66,10 +76,16 @@ describe Puppet::FileBucket::File do end it "should convert the contents to PSON" do - Puppet::FileBucket::File.new("file contents").to_pson.should == '{"contents":"file contents"}' + # The model class no longer defines to_pson and it is not a supported + # format, but pson monkey patches Object#to_pson to return + # Object#to_s.to_pson, and it monkey patches String#to_pson to wrap the + # returned string in quotes. So it works in a way that is completely + # unexpected, and it doesn't round-trip correctly, awesome. + Puppet::FileBucket::File.new("file contents").to_pson.should == '"file contents"' end it "should load from PSON" do + Puppet.expects(:deprecation_warning).with('Deserializing Puppet::FileBucket::File objects from pson is deprecated. Upgrade to a newer version.') Puppet::FileBucket::File.from_pson({"contents"=>"file contents"}).contents.should == "file contents" end diff --git a/spec/unit/file_serving/configuration/parser_spec.rb b/spec/unit/file_serving/configuration/parser_spec.rb index ecdfd6640..75744d317 100755 --- a/spec/unit/file_serving/configuration/parser_spec.rb +++ b/spec/unit/file_serving/configuration/parser_spec.rb @@ -3,30 +3,20 @@ require 'spec_helper' require 'puppet/file_serving/configuration/parser' -describe Puppet::FileServing::Configuration::Parser do - it "should subclass the LoadedFile class" do - Puppet::FileServing::Configuration::Parser.superclass.should equal(Puppet::Util::LoadedFile) - end -end - module FSConfigurationParserTesting - def mock_file_content(content) + def write_config_file(content) # We want an array, but we actually want our carriage returns on all of it. - lines = content.split("\n").collect { |l| l + "\n" } - @filehandle.stubs(:each_line).multiple_yields(*lines) - @filehandle.expects(:each).never + File.open(@path, 'w') {|f| f.puts content} end end describe Puppet::FileServing::Configuration::Parser do + include PuppetSpec::Files + before :each do - @path = "/my/config.conf" - FileTest.stubs(:exists?).with(@path).returns(true) - FileTest.stubs(:readable?).with(@path).returns(true) - @filehandle = mock 'filehandle' - @filehandle.expects(:each).never - File.expects(:open).with(@path).yields(@filehandle) + @path = tmpfile('fileserving_config') + FileUtils.touch(@path) @parser = Puppet::FileServing::Configuration::Parser.new(@path) end @@ -34,12 +24,12 @@ describe Puppet::FileServing::Configuration::Parser do include FSConfigurationParserTesting it "should allow comments" do - @filehandle.expects(:each_line).yields("# this is a comment\n") + write_config_file("# this is a comment\n") proc { @parser.parse }.should_not raise_error end it "should allow blank lines" do - @filehandle.expects(:each_line).yields("\n") + write_config_file("\n") proc { @parser.parse }.should_not raise_error end @@ -48,7 +38,7 @@ describe Puppet::FileServing::Configuration::Parser do mount2 = mock 'two', :validate => true Puppet::FileServing::Mount::File.expects(:new).with("one").returns(mount1) Puppet::FileServing::Mount::File.expects(:new).with("two").returns(mount2) - mock_file_content "[one]\n[two]\n" + write_config_file "[one]\n[two]\n" @parser.parse end @@ -58,7 +48,7 @@ describe Puppet::FileServing::Configuration::Parser do mount2 = mock 'two', :validate => true Puppet::FileServing::Mount::File.expects(:new).with("one").returns(mount1) Puppet::FileServing::Mount::File.expects(:new).with("two").returns(mount2) - mock_file_content "[one]\n[two]\n" + write_config_file "[one]\n[two]\n" result = @parser.parse result["one"].should equal(mount1) @@ -66,19 +56,19 @@ describe Puppet::FileServing::Configuration::Parser do end it "should only allow mount names that are alphanumeric plus dashes" do - mock_file_content "[a*b]\n" + write_config_file "[a*b]\n" proc { @parser.parse }.should raise_error(ArgumentError) end it "should fail if the value for path/allow/deny starts with an equals sign" do - mock_file_content "[one]\npath = /testing" + write_config_file "[one]\npath = /testing" proc { @parser.parse }.should raise_error(ArgumentError) end it "should validate each created mount" do mount1 = mock 'one' Puppet::FileServing::Mount::File.expects(:new).with("one").returns(mount1) - mock_file_content "[one]\n" + write_config_file "[one]\n" mount1.expects(:validate) @@ -88,7 +78,7 @@ describe Puppet::FileServing::Configuration::Parser do it "should fail if any mount does not pass validation" do mount1 = mock 'one' Puppet::FileServing::Mount::File.expects(:new).with("one").returns(mount1) - mock_file_content "[one]\n" + write_config_file "[one]\n" mount1.expects(:validate).raises RuntimeError @@ -106,14 +96,14 @@ describe Puppet::FileServing::Configuration::Parser do end it "should set the mount path to the path attribute from that section" do - mock_file_content "[one]\npath /some/path\n" + write_config_file "[one]\npath /some/path\n" @mount.expects(:path=).with("/some/path") @parser.parse end it "should tell the mount to allow any allow values from the section" do - mock_file_content "[one]\nallow something\n" + write_config_file "[one]\nallow something\n" @mount.expects(:info) @mount.expects(:allow).with("something") @@ -121,7 +111,7 @@ describe Puppet::FileServing::Configuration::Parser do end it "should support inline comments" do - mock_file_content "[one]\nallow something \# will it work?\n" + write_config_file "[one]\nallow something \# will it work?\n" @mount.expects(:info) @mount.expects(:allow).with("something") @@ -129,7 +119,7 @@ describe Puppet::FileServing::Configuration::Parser do end it "should tell the mount to deny any deny values from the section" do - mock_file_content "[one]\ndeny something\n" + write_config_file "[one]\ndeny something\n" @mount.expects(:info) @mount.expects(:deny).with("something") @@ -137,7 +127,7 @@ describe Puppet::FileServing::Configuration::Parser do end it "should fail on any attributes other than path, allow, and deny" do - mock_file_content "[one]\ndo something\n" + write_config_file "[one]\ndo something\n" proc { @parser.parse }.should raise_error(ArgumentError) end @@ -151,14 +141,14 @@ describe Puppet::FileServing::Configuration::Parser do end it "should create an instance of the Modules Mount class" do - mock_file_content "[modules]\n" + write_config_file "[modules]\n" Puppet::FileServing::Mount::Modules.expects(:new).with("modules").returns @mount @parser.parse end it "should warn if a path is set" do - mock_file_content "[modules]\npath /some/path\n" + write_config_file "[modules]\npath /some/path\n" Puppet::FileServing::Mount::Modules.expects(:new).with("modules").returns(@mount) Puppet.expects(:warning) @@ -174,14 +164,14 @@ describe Puppet::FileServing::Configuration::Parser do end it "should create an instance of the Plugins Mount class" do - mock_file_content "[plugins]\n" + write_config_file "[plugins]\n" Puppet::FileServing::Mount::Plugins.expects(:new).with("plugins").returns @mount @parser.parse end it "should warn if a path is set" do - mock_file_content "[plugins]\npath /some/path\n" + write_config_file "[plugins]\npath /some/path\n" Puppet.expects(:warning) @parser.parse diff --git a/spec/unit/file_serving/configuration_spec.rb b/spec/unit/file_serving/configuration_spec.rb index 94d6f0b22..2552cd808 100755 --- a/spec/unit/file_serving/configuration_spec.rb +++ b/spec/unit/file_serving/configuration_spec.rb @@ -167,13 +167,13 @@ describe Puppet::FileServing::Configuration do it "should fail if the mount name is not alpha-numeric" do request.expects(:key).returns "foo&bar/asdf" - lambda { config.split_path(request) }.should raise_error(ArgumentError) + expect { config.split_path(request) }.to raise_error(ArgumentError) end it "should support dashes in the mount name" do request.expects(:key).returns "foo-bar/asdf" - lambda { config.split_path(request) }.should_not raise_error(ArgumentError) + expect { config.split_path(request) }.to_not raise_error end it "should use the mount name and environment to find the mount" do diff --git a/spec/unit/file_serving/mount/file_spec.rb b/spec/unit/file_serving/mount/file_spec.rb index f68fadb81..59394be46 100755 --- a/spec/unit/file_serving/mount/file_spec.rb +++ b/spec/unit/file_serving/mount/file_spec.rb @@ -11,7 +11,7 @@ end describe Puppet::FileServing::Mount::File do it "should be invalid if it does not have a path" do - lambda { Puppet::FileServing::Mount::File.new("foo").validate }.should raise_error(ArgumentError) + expect { Puppet::FileServing::Mount::File.new("foo").validate }.to raise_error(ArgumentError) end it "should be valid if it has a path" do @@ -19,7 +19,7 @@ describe Puppet::FileServing::Mount::File do FileTest.stubs(:readable?).returns true mount = Puppet::FileServing::Mount::File.new("foo") mount.path = "/foo" - lambda { mount.validate }.should_not raise_error(ArgumentError) + expect { mount.validate }.not_to raise_error end describe "when setting the path" do @@ -30,13 +30,13 @@ describe Puppet::FileServing::Mount::File do it "should fail if the path is not a directory" do FileTest.expects(:directory?).returns(false) - proc { @mount.path = @dir }.should raise_error(ArgumentError) + expect { @mount.path = @dir }.to raise_error(ArgumentError) end it "should fail if the path is not readable" do FileTest.expects(:directory?).returns(true) FileTest.expects(:readable?).returns(false) - proc { @mount.path = @dir }.should raise_error(ArgumentError) + expect { @mount.path = @dir }.to raise_error(ArgumentError) end end diff --git a/spec/unit/forge/repository_spec.rb b/spec/unit/forge/repository_spec.rb index 71fe38ecb..31d28b5d9 100644 --- a/spec/unit/forge/repository_spec.rb +++ b/spec/unit/forge/repository_spec.rb @@ -11,39 +11,19 @@ describe Puppet::Forge::Repository do let(:ssl_repository) { Puppet::Forge::Repository.new('https://fake.com', consumer_version) } it "retrieve accesses the cache" do - uri = URI.parse('http://some.url.com') - repository.cache.expects(:retrieve).with(uri) + path = '/module/foo.tar.gz' + repository.cache.expects(:retrieve) - repository.retrieve(uri) + repository.retrieve(path) end - describe 'http_proxy support' do - after :each do - ENV["http_proxy"] = nil - end - - it "supports environment variable for port and host" do - ENV["http_proxy"] = "http://test.com:8011" - - repository.http_proxy_host.should == "test.com" - repository.http_proxy_port.should == 8011 - end - - it "supports puppet configuration for port and host" do - ENV["http_proxy"] = nil - proxy_settings_of('test.com', 7456) + it "retrieve merges forge URI and path specified" do + path = '/module/foo.tar.gz' + repo_uri = 'http://fake.com/test' + repository = Puppet::Forge::Repository.new(repo_uri, consumer_version) + repository.cache.expects(:retrieve).with(URI.parse(repo_uri+path)) - repository.http_proxy_port.should == 7456 - repository.http_proxy_host.should == "test.com" - end - - it "uses environment variable before puppet settings" do - ENV["http_proxy"] = "http://test1.com:8011" - proxy_settings_of('test2.com', 7456) - - repository.http_proxy_host.should == "test1.com" - repository.http_proxy_port.should == 8011 - end + repository.retrieve(path) end describe "making a request" do diff --git a/spec/unit/graph/key_spec.rb b/spec/unit/graph/key_spec.rb new file mode 100644 index 000000000..6d4df362f --- /dev/null +++ b/spec/unit/graph/key_spec.rb @@ -0,0 +1,41 @@ +require 'spec_helper' + +require 'puppet/graph' + +describe Puppet::Graph::Key do + it "produces the next in the sequence" do + key = Puppet::Graph::Key.new + + expect(key.next).to be > key + end + + it "produces a key after itself but before next" do + key = Puppet::Graph::Key.new + expect(key.down).to be > key + expect(key.down).to be < key.next + end + + it "downward keys of the same group are in sequence" do + key = Puppet::Graph::Key.new + + first = key.down + middle = key.down.next + last = key.down.next.next + + expect(first).to be < middle + expect(middle).to be < last + expect(last).to be < key.next + end + + it "downward keys in sequential groups are in sequence" do + key = Puppet::Graph::Key.new + + first = key.down + middle = key.next + last = key.next.down + + expect(first).to be < middle + expect(middle).to be < last + expect(last).to be < key.next.next + end +end diff --git a/spec/unit/rb_tree_map_spec.rb b/spec/unit/graph/rb_tree_map_spec.rb index 3208d6cb0..e1ac7d9b2 100644 --- a/spec/unit/rb_tree_map_spec.rb +++ b/spec/unit/graph/rb_tree_map_spec.rb @@ -2,9 +2,9 @@ require 'spec_helper' -require 'puppet/rb_tree_map' +require 'puppet/graph' -describe Puppet::RbTreeMap do +describe Puppet::Graph::RbTreeMap do describe "#push" do it "should allow a new element to be added" do subject[5] = 'foo' @@ -32,7 +32,7 @@ describe Puppet::RbTreeMap do subject[5] = 'foo' - subject.instance_variable_get(:@root).should be_a(Puppet::RbTreeMap::Node) + subject.instance_variable_get(:@root).should be_a(Puppet::Graph::RbTreeMap::Node) end end @@ -395,14 +395,14 @@ describe Puppet::RbTreeMap do describe "#isred" do it "should return true if the node is red" do - node = Puppet::RbTreeMap::Node.new(1,2) + node = Puppet::Graph::RbTreeMap::Node.new(1,2) node.color = :red subject.send(:isred, node).should == true end it "should return false if the node is black" do - node = Puppet::RbTreeMap::Node.new(1,2) + node = Puppet::Graph::RbTreeMap::Node.new(1,2) node.color = :black subject.send(:isred, node).should == false @@ -414,8 +414,8 @@ describe Puppet::RbTreeMap do end end -describe Puppet::RbTreeMap::Node do - let(:tree) { Puppet::RbTreeMap.new } +describe Puppet::Graph::RbTreeMap::Node do + let(:tree) { Puppet::Graph::RbTreeMap.new } let(:subject) { tree.instance_variable_get(:@root) } before :each do diff --git a/spec/unit/graph/relationship_graph_spec.rb b/spec/unit/graph/relationship_graph_spec.rb new file mode 100755 index 000000000..7a698d769 --- /dev/null +++ b/spec/unit/graph/relationship_graph_spec.rb @@ -0,0 +1,393 @@ +#! /usr/bin/env ruby +require 'spec_helper' +require 'puppet/graph' + +require 'puppet_spec/compiler' +require 'matchers/include_in_order' +require 'matchers/relationship_graph_matchers' + +describe Puppet::Graph::RelationshipGraph do + include PuppetSpec::Files + include PuppetSpec::Compiler + include RelationshipGraphMatchers + + let(:graph) { Puppet::Graph::RelationshipGraph.new(Puppet::Graph::SequentialPrioritizer.new) } + + it "allows adding a new vertex with a specific priority" do + vertex = stub_vertex('something') + + graph.add_vertex(vertex, 2) + + expect(graph.resource_priority(vertex)).to eq(2) + end + + it "returns resource priority based on the order added" do + # strings chosen so the old hex digest method would put these in the + # wrong order + first = stub_vertex('aa') + second = stub_vertex('b') + + graph.add_vertex(first) + graph.add_vertex(second) + + expect(graph.resource_priority(first)).to be < graph.resource_priority(second) + end + + it "retains the first priority when a resource is added more than once" do + first = stub_vertex(1) + second = stub_vertex(2) + + graph.add_vertex(first) + graph.add_vertex(second) + graph.add_vertex(first) + + expect(graph.resource_priority(first)).to be < graph.resource_priority(second) + end + + it "forgets the priority of a removed resource" do + vertex = stub_vertex(1) + + graph.add_vertex(vertex) + graph.remove_vertex!(vertex) + + expect(graph.resource_priority(vertex)).to be_nil + end + + it "does not give two resources the same priority" do + first = stub_vertex(1) + second = stub_vertex(2) + third = stub_vertex(3) + + graph.add_vertex(first) + graph.add_vertex(second) + graph.remove_vertex!(first) + graph.add_vertex(third) + + expect(graph.resource_priority(second)).to be < graph.resource_priority(third) + end + + context "order of traversal" do + it "traverses independent resources in the order they are added" do + relationships = compile_to_relationship_graph(<<-MANIFEST) + notify { "first": } + notify { "second": } + notify { "third": } + notify { "fourth": } + notify { "fifth": } + MANIFEST + + expect(order_resources_traversed_in(relationships)).to( + include_in_order("Notify[first]", + "Notify[second]", + "Notify[third]", + "Notify[fourth]", + "Notify[fifth]")) + end + + it "traverses resources generated during catalog creation in the order inserted" do + relationships = compile_to_relationship_graph(<<-MANIFEST) + create_resources(notify, { "first" => {} }) + create_resources(notify, { "second" => {} }) + notify{ "third": } + create_resources(notify, { "fourth" => {} }) + create_resources(notify, { "fifth" => {} }) + MANIFEST + + expect(order_resources_traversed_in(relationships)).to( + include_in_order("Notify[first]", + "Notify[second]", + "Notify[third]", + "Notify[fourth]", + "Notify[fifth]")) + end + + it "traverses all independent resources before traversing dependent ones" do + relationships = compile_to_relationship_graph(<<-MANIFEST) + notify { "first": require => Notify[third] } + notify { "second": } + notify { "third": } + MANIFEST + + expect(order_resources_traversed_in(relationships)).to( + include_in_order("Notify[second]", "Notify[third]", "Notify[first]")) + end + + it "traverses all independent resources before traversing dependent ones (with a backwards require)" do + relationships = compile_to_relationship_graph(<<-MANIFEST) + notify { "first": } + notify { "second": } + notify { "third": require => Notify[second] } + notify { "fourth": } + MANIFEST + + expect(order_resources_traversed_in(relationships)).to( + include_in_order("Notify[first]", "Notify[second]", "Notify[third]", "Notify[fourth]")) + end + + it "traverses resources in classes in the order they are added" do + relationships = compile_to_relationship_graph(<<-MANIFEST) + class c1 { + notify { "a": } + notify { "b": } + } + class c2 { + notify { "c": require => Notify[b] } + } + class c3 { + notify { "d": } + } + include c2 + include c1 + include c3 + MANIFEST + + expect(order_resources_traversed_in(relationships)).to( + include_in_order("Notify[a]", "Notify[b]", "Notify[c]", "Notify[d]")) + end + + it "traverses resources in defines in the order they are added" do + relationships = compile_to_relationship_graph(<<-MANIFEST) + define d1() { + notify { "a": } + notify { "b": } + } + define d2() { + notify { "c": require => Notify[b]} + } + define d3() { + notify { "d": } + } + d2 { "c": } + d1 { "d": } + d3 { "e": } + MANIFEST + + expect(order_resources_traversed_in(relationships)).to( + include_in_order("Notify[a]", "Notify[b]", "Notify[c]", "Notify[d]")) + end + + def order_resources_traversed_in(relationships) + order_seen = [] + relationships.traverse { |resource| order_seen << resource.ref } + order_seen + end + end + + describe "when interrupting traversal" do + def collect_canceled_resources(relationships, trigger_on) + continue = true + continue_while = lambda { continue } + + canceled_resources = [] + canceled_resource_handler = lambda { |resource| canceled_resources << resource.ref } + + relationships.traverse(:while => continue_while, + :canceled_resource_handler => canceled_resource_handler) do |resource| + if resource.ref == trigger_on + continue = false + end + end + + canceled_resources + end + + it "enumerates the remaining resources" do + relationships = compile_to_relationship_graph(<<-MANIFEST) + notify { "a": } + notify { "b": } + notify { "c": } + MANIFEST + resources = collect_canceled_resources(relationships, 'Notify[b]') + + expect(resources).to include('Notify[c]') + end + + it "enumerates the remaining blocked resources" do + relationships = compile_to_relationship_graph(<<-MANIFEST) + notify { "a": } + notify { "b": } + notify { "c": } + notify { "d": require => Notify["c"] } + MANIFEST + resources = collect_canceled_resources(relationships, 'Notify[b]') + + expect(resources).to include('Notify[d]') + end + end + + describe "when constructing dependencies" do + let(:child) { make_absolute('/a/b') } + let(:parent) { make_absolute('/a') } + + it "does not create an automatic relationship that would interfere with a manual relationship" do + relationship_graph = compile_to_relationship_graph(<<-MANIFEST) + file { "#{child}": } + + file { "#{parent}": require => File["#{child}"] } + MANIFEST + + relationship_graph.should enforce_order_with_edge("File[#{child}]", "File[#{parent}]") + end + + it "creates automatic relationships defined by the type" do + relationship_graph = compile_to_relationship_graph(<<-MANIFEST) + file { "#{child}": } + + file { "#{parent}": } + MANIFEST + + relationship_graph.should enforce_order_with_edge("File[#{parent}]", "File[#{child}]") + end + end + + describe "when reconstructing containment relationships" do + def admissible_sentinel_of(ref) + "Admissible_#{ref}" + end + + def completed_sentinel_of(ref) + "Completed_#{ref}" + end + + it "an empty container's completed sentinel should depend on its admissible sentinel" do + relationship_graph = compile_to_relationship_graph(<<-MANIFEST) + class a { } + + include a + MANIFEST + + relationship_graph.should enforce_order_with_edge( + admissible_sentinel_of("class[A]"), + completed_sentinel_of("class[A]")) + end + + it "a container with children does not directly connect the completed sentinel to its admissible sentinel" do + relationship_graph = compile_to_relationship_graph(<<-MANIFEST) + class a { notify { "a": } } + + include a + MANIFEST + + relationship_graph.should_not enforce_order_with_edge( + admissible_sentinel_of("class[A]"), + completed_sentinel_of("class[A]")) + end + + it "all contained objects should depend on their container's admissible sentinel" do + relationship_graph = compile_to_relationship_graph(<<-MANIFEST) + class a { + notify { "class a": } + } + + include a + MANIFEST + + relationship_graph.should enforce_order_with_edge( + admissible_sentinel_of("class[A]"), + "Notify[class a]") + end + + it "completed sentinels should depend on their container's contents" do + relationship_graph = compile_to_relationship_graph(<<-MANIFEST) + class a { + notify { "class a": } + } + + include a + MANIFEST + + relationship_graph.should enforce_order_with_edge( + "Notify[class a]", + completed_sentinel_of("class[A]")) + end + + it "should remove all Component objects from the dependency graph" do + relationship_graph = compile_to_relationship_graph(<<-MANIFEST) + class a { + notify { "class a": } + } + define b() { + notify { "define b": } + } + + include a + b { "testing": } + MANIFEST + + relationship_graph.vertices.find_all { |v| v.is_a?(Puppet::Type.type(:component)) }.should be_empty + end + + it "should remove all Stage resources from the dependency graph" do + relationship_graph = compile_to_relationship_graph(<<-MANIFEST) + notify { "class a": } + MANIFEST + + relationship_graph.vertices.find_all { |v| v.is_a?(Puppet::Type.type(:stage)) }.should be_empty + end + + it "should retain labels on non-containment edges" do + relationship_graph = compile_to_relationship_graph(<<-MANIFEST) + class a { + notify { "class a": } + } + define b() { + notify { "define b": } + } + + include a + Class[a] ~> b { "testing": } + MANIFEST + + relationship_graph.edges_between( + vertex_called(relationship_graph, completed_sentinel_of("class[A]")), + vertex_called(relationship_graph, admissible_sentinel_of("b[testing]")))[0].label. + should == {:callback => :refresh, :event => :ALL_EVENTS} + end + + it "should not add labels to edges that have none" do + relationship_graph = compile_to_relationship_graph(<<-MANIFEST) + class a { + notify { "class a": } + } + define b() { + notify { "define b": } + } + + include a + Class[a] -> b { "testing": } + MANIFEST + + relationship_graph.edges_between( + vertex_called(relationship_graph, completed_sentinel_of("class[A]")), + vertex_called(relationship_graph, admissible_sentinel_of("b[testing]")))[0].label. + should be_empty + end + + it "should copy notification labels to all created edges" do + relationship_graph = compile_to_relationship_graph(<<-MANIFEST) + class a { + notify { "class a": } + } + define b() { + notify { "define b": } + } + + include a + Class[a] ~> b { "testing": } + MANIFEST + + relationship_graph.edges_between( + vertex_called(relationship_graph, admissible_sentinel_of("b[testing]")), + vertex_called(relationship_graph, "Notify[define b]"))[0].label. + should == {:callback => :refresh, :event => :ALL_EVENTS} + end + end + + def vertex_called(graph, name) + graph.vertices.find { |v| v.ref =~ /#{Regexp.escape(name)}/ } + end + + def stub_vertex(name) + stub "vertex #{name}", :ref => name + end +end diff --git a/spec/unit/graph/sequential_prioritizer_spec.rb b/spec/unit/graph/sequential_prioritizer_spec.rb new file mode 100644 index 000000000..17df124b2 --- /dev/null +++ b/spec/unit/graph/sequential_prioritizer_spec.rb @@ -0,0 +1,32 @@ +require 'spec_helper' +require 'puppet/graph' + +describe Puppet::Graph::SequentialPrioritizer do + let(:priorities) { Puppet::Graph::SequentialPrioritizer.new } + + it "generates priorities that maintain the sequence" do + first = priorities.generate_priority_for("one") + second = priorities.generate_priority_for("two") + third = priorities.generate_priority_for("three") + + expect(first).to be < second + expect(second).to be < third + end + + it "prioritizes contained keys after the container" do + parent = priorities.generate_priority_for("one") + child = priorities.generate_priority_contained_in("one", "child 1") + sibling = priorities.generate_priority_contained_in("one", "child 2") + uncle = priorities.generate_priority_for("two") + + expect(parent).to be < child + expect(child).to be < sibling + expect(sibling).to be < uncle + end + + it "fails to prioritize a key contained in an unknown container" do + expect do + priorities.generate_priority_contained_in("unknown", "child 1") + end.to raise_error + end +end diff --git a/spec/unit/simple_graph_spec.rb b/spec/unit/graph/simple_graph.rb index 8edf59606..0f43d3523 100755 --- a/spec/unit/simple_graph_spec.rb +++ b/spec/unit/graph/simple_graph.rb @@ -1,27 +1,27 @@ #! /usr/bin/env ruby require 'spec_helper' -require 'puppet/simple_graph' +require 'puppet/graph' -describe Puppet::SimpleGraph do +describe Puppet::Graph::SimpleGraph do it "should return the number of its vertices as its length" do - @graph = Puppet::SimpleGraph.new + @graph = Puppet::Graph::SimpleGraph.new @graph.add_vertex("one") @graph.add_vertex("two") @graph.size.should == 2 end it "should consider itself a directed graph" do - Puppet::SimpleGraph.new.directed?.should be_true + Puppet::Graph::SimpleGraph.new.directed?.should be_true end it "should provide a method for reversing the graph" do - @graph = Puppet::SimpleGraph.new + @graph = Puppet::Graph::SimpleGraph.new @graph.add_edge(:one, :two) @graph.reversal.edge?(:two, :one).should be_true end it "should be able to produce a dot graph" do - @graph = Puppet::SimpleGraph.new + @graph = Puppet::Graph::SimpleGraph.new @graph.add_edge(:one, :two) expect { @graph.to_dot_graph }.to_not raise_error @@ -29,7 +29,7 @@ describe Puppet::SimpleGraph do describe "when managing vertices" do before do - @graph = Puppet::SimpleGraph.new + @graph = Puppet::Graph::SimpleGraph.new end it "should provide a method to add a vertex" do @@ -78,7 +78,7 @@ describe Puppet::SimpleGraph do describe "when managing edges" do before do - @graph = Puppet::SimpleGraph.new + @graph = Puppet::Graph::SimpleGraph.new end it "should provide a method to test whether a given vertex pair is an edge" do @@ -179,7 +179,7 @@ describe Puppet::SimpleGraph do describe "when finding adjacent vertices" do before do - @graph = Puppet::SimpleGraph.new + @graph = Puppet::Graph::SimpleGraph.new @one_two = Puppet::Relationship.new(:one, :two) @two_three = Puppet::Relationship.new(:two, :three) @one_three = Puppet::Relationship.new(:one, :three) @@ -212,7 +212,7 @@ describe Puppet::SimpleGraph do # Bug #2111 it "should not consider a vertex adjacent just because it was asked about previously" do - @graph = Puppet::SimpleGraph.new + @graph = Puppet::Graph::SimpleGraph.new @graph.add_vertex("a") @graph.add_vertex("b") @graph.edge?("a", "b") @@ -222,7 +222,7 @@ describe Puppet::SimpleGraph do describe "when clearing" do before do - @graph = Puppet::SimpleGraph.new + @graph = Puppet::Graph::SimpleGraph.new one = Puppet::Relationship.new(:one, :two) two = Puppet::Relationship.new(:two, :three) @graph.add_edge(one) @@ -242,7 +242,7 @@ describe Puppet::SimpleGraph do describe "when reversing graphs" do before do - @graph = Puppet::SimpleGraph.new + @graph = Puppet::Graph::SimpleGraph.new end it "should provide a method for reversing the graph" do @@ -265,7 +265,7 @@ describe Puppet::SimpleGraph do describe "when reporting cycles in the graph" do before do - @graph = Puppet::SimpleGraph.new + @graph = Puppet::Graph::SimpleGraph.new end # This works with `add_edges` to auto-vivify the resource instances. @@ -282,9 +282,9 @@ describe Puppet::SimpleGraph do end def simplify(cycles) - cycles.map do |x| - x.map do |y| - y.to_s.match(/^Notify\[(.*)\]$/)[1] + cycles.map do |cycle| + cycle.map do |resource| + resource.name end end end @@ -322,18 +322,20 @@ describe Puppet::SimpleGraph do add_edges "b" => "a" add_edges "b" => "c" - cycles = nil - expect { cycles = @graph.find_cycles_in_graph }.to_not raise_error - simplify(cycles).should be == [["a", "b"]] + simplify(@graph.find_cycles_in_graph).should be == [["a", "b"]] + end + + it "cycle discovery handles a self-loop cycle" do + add_edges :a => :a + + simplify(@graph.find_cycles_in_graph).should be == [["a"]] end it "cycle discovery should handle two distinct cycles" do add_edges "a" => "a1", "a1" => "a" add_edges "b" => "b1", "b1" => "b" - cycles = nil - expect { cycles = @graph.find_cycles_in_graph }.to_not raise_error - simplify(cycles).should be == [["a1", "a"], ["b1", "b"]] + simplify(@graph.find_cycles_in_graph).should be == [["a1", "a"], ["b1", "b"]] end it "cycle discovery should handle two cycles in a connected graph" do @@ -341,9 +343,7 @@ describe Puppet::SimpleGraph do add_edges "a" => "a1", "a1" => "a" add_edges "c" => "c1", "c1" => "c2", "c2" => "c3", "c3" => "c" - cycles = nil - expect { cycles = @graph.find_cycles_in_graph }.to_not raise_error - simplify(cycles).should be == [%w{a1 a}, %w{c1 c2 c3 c}] + simplify(@graph.find_cycles_in_graph).should be == [%w{a1 a}, %w{c1 c2 c3 c}] end it "cycle discovery should handle a complicated cycle" do @@ -352,18 +352,14 @@ describe Puppet::SimpleGraph do add_edges "c" => "c1", "c1" => "a" add_edges "c" => "c2", "c2" => "b" - cycles = nil - expect { cycles = @graph.find_cycles_in_graph }.to_not raise_error - simplify(cycles).should be == [%w{a b c1 c2 c}] + simplify(@graph.find_cycles_in_graph).should be == [%w{a b c1 c2 c}] end it "cycle discovery should not fail with large data sets" do limit = 3000 (1..(limit - 1)).each do |n| add_edges n.to_s => (n+1).to_s end - cycles = nil - expect { cycles = @graph.find_cycles_in_graph }.to_not raise_error - simplify(cycles).should be == [] + simplify(@graph.find_cycles_in_graph).should be == [] end it "path finding should work with a simple cycle" do @@ -415,7 +411,7 @@ describe Puppet::SimpleGraph do describe "when writing dot files" do before do - @graph = Puppet::SimpleGraph.new + @graph = Puppet::Graph::SimpleGraph.new @name = :test @file = File.join(Puppet[:graphdir], @name.to_s + ".dot") end @@ -432,15 +428,11 @@ describe Puppet::SimpleGraph do Puppet[:graph] = true @graph.write_graph(@name) end - - after do - Puppet.settings.clear - end end - describe Puppet::SimpleGraph do + describe Puppet::Graph::SimpleGraph do before do - @graph = Puppet::SimpleGraph.new + @graph = Puppet::Graph::SimpleGraph.new end it "should correctly clear vertices and edges when asked" do @@ -454,7 +446,7 @@ describe Puppet::SimpleGraph do describe "when matching edges" do before do - @graph = Puppet::SimpleGraph.new + @graph = Puppet::Graph::SimpleGraph.new # The Ruby 1.8 semantics for String#[] are that treating it like an # array and asking for `"a"[:whatever]` returns `nil`. Ruby 1.9 @@ -496,7 +488,7 @@ describe Puppet::SimpleGraph do describe "when determining dependencies" do before do - @graph = Puppet::SimpleGraph.new + @graph = Puppet::Graph::SimpleGraph.new @graph.add_edge("a", "b") @graph.add_edge("a", "c") @@ -528,212 +520,8 @@ describe Puppet::SimpleGraph do end end - require 'puppet/util/graph' - - class Container < Puppet::Type::Component - include Puppet::Util::Graph - include Enumerable - attr_accessor :name - def each - @children.each do |c| yield c end - end - - def initialize(name, ary) - @name = name - @children = ary - end - - def push(*ary) - ary.each { |c| @children.push(c)} - end - - def to_s - @name - end - - def ref - "Container[#{self}]" - end - end - - require "puppet/resource/catalog" - describe "when splicing the graph" do - def container_graph - @one = Container.new("one", %w{a b}) - @two = Container.new("two", ["c", "d"]) - @three = Container.new("three", ["i", "j"]) - @middle = Container.new("middle", ["e", "f", @two]) - @top = Container.new("top", ["g", "h", @middle, @one, @three]) - @empty = Container.new("empty", []) - - @whit = Puppet::Type.type(:whit) - @stage = Puppet::Type.type(:stage).new(:name => "foo") - - @contgraph = @top.to_graph(Puppet::Resource::Catalog.new) - - # We have to add the container to the main graph, else it won't - # be spliced in the dependency graph. - @contgraph.add_vertex(@empty) - end - - def containers - @contgraph.vertices.select { |x| !x.is_a? String } - end - - def contents_of(x) - @contgraph.direct_dependents_of(x) - end - - def dependency_graph - @depgraph = Puppet::SimpleGraph.new - @contgraph.vertices.each do |v| - @depgraph.add_vertex(v) - end - - # We have to specify a relationship to our empty container, else it - # never makes it into the dep graph in the first place. - @explicit_dependencies = {@one => @two, "f" => "c", "h" => @middle, "c" => @empty} - @explicit_dependencies.each do |source, target| - @depgraph.add_edge(source, target, :callback => :refresh) - end - end - - def splice - @contgraph.splice!(@depgraph) - end - - def whit_called(name) - x = @depgraph.vertices.find { |v| v.is_a?(@whit) && v.name =~ /#{Regexp.escape(name)}/ } - x.should_not be_nil - def x.to_s - "Whit[#{name}]" - end - def x.inspect - to_s - end - x - end - - def admissible_sentinel_of(x) - @depgraph.vertex?(x) ? x : whit_called("admissible_#{x.ref}") - end - - def completed_sentinel_of(x) - @depgraph.vertex?(x) ? x : whit_called("completed_#{x.ref}") - end - - before do - container_graph - dependency_graph - splice - end - - # This is the real heart of splicing -- replacing all containers X in our - # relationship graph with a pair of whits { admissible_X and completed_X } - # such that that - # - # 0) completed_X depends on admissible_X - # 1) contents of X each depend on admissible_X - # 2) completed_X depends on each on the contents of X - # 3) everything which depended on X depends on completed_X - # 4) admissible_X depends on everything X depended on - # 5) the containers and their edges must be removed - # - # Note that this requires attention to the possible case of containers - # which contain or depend on other containers. - # - # Point by point: - - # 0) completed_X depends on admissible_X - # - it "every container's completed sentinel should depend on its admissible sentinel" do - containers.each { |container| - @depgraph.path_between(admissible_sentinel_of(container),completed_sentinel_of(container)).should be - } - end - - # 1) contents of X each depend on admissible_X - # - it "all contained objects should depend on their container's admissible sentinel" do - containers.each { |container| - contents_of(container).each { |leaf| - @depgraph.should be_edge(admissible_sentinel_of(container),admissible_sentinel_of(leaf)) - } - } - end - - # 2) completed_X depends on each on the contents of X - # - it "completed sentinels should depend on their container's contents" do - containers.each { |container| - contents_of(container).each { |leaf| - @depgraph.should be_edge(completed_sentinel_of(leaf),completed_sentinel_of(container)) - } - } - end - - # - # 3) everything which depended on X depends on completed_X - - # - # 4) admissible_X depends on everything X depended on - - # 5) the containers and their edges must be removed - # - it "should remove all Container objects from the dependency graph" do - @depgraph.vertices.find_all { |v| v.is_a?(Container) }.should be_empty - end - - it "should remove all Stage resources from the dependency graph" do - @depgraph.vertices.find_all { |v| v.is_a?(Puppet::Type.type(:stage)) }.should be_empty - end - - it "should no longer contain anything but the non-container objects" do - @depgraph.vertices.find_all { |v| ! v.is_a?(String) and ! v.is_a?(@whit)}.should be_empty - end - - it "should retain labels on non-containment edges" do - @explicit_dependencies.each { |f,t| - @depgraph.edges_between(completed_sentinel_of(f),admissible_sentinel_of(t))[0].label.should == {:callback => :refresh} - } - end - - it "should not add labels to edges that have none" do - @depgraph.add_edge(@two, @three) - splice - @depgraph.path_between("c", "i").any? {|segment| segment.all? {|e| e.label == {} }}.should be - end - - it "should copy labels over edges that have none" do - @depgraph.add_edge("c", @three, {:callback => :refresh}) - splice - # And make sure the label got copied. - @depgraph.path_between("c", "i").flatten.select {|e| e.label == {:callback => :refresh} }.should_not be_empty - end - - it "should not replace a label with a nil label" do - # Lastly, add some new label-less edges and make sure the label stays. - @depgraph.add_edge(@middle, @three) - @depgraph.add_edge("c", @three, {:callback => :refresh}) - splice - @depgraph.path_between("c","i").flatten.select {|e| e.label == {:callback => :refresh} }.should_not be_empty - end - - it "should copy labels to all created edges" do - @depgraph.add_edge(@middle, @three) - @depgraph.add_edge("c", @three, {:callback => :refresh}) - splice - @three.each do |child| - edge = Puppet::Relationship.new("c", child) - (path = @depgraph.path_between(edge.source, edge.target)).should be - path.should_not be_empty - path.flatten.select {|e| e.label == {:callback => :refresh} }.should_not be_empty - end - end - end - it "should serialize to YAML using the old format by default" do - Puppet::SimpleGraph.use_new_yaml_format.should == false + Puppet::Graph::SimpleGraph.use_new_yaml_format.should == false end describe "(yaml tests)" do @@ -780,18 +568,18 @@ describe Puppet::SimpleGraph do end def graph_to_yaml(graph, which_format) - previous_use_new_yaml_format = Puppet::SimpleGraph.use_new_yaml_format - Puppet::SimpleGraph.use_new_yaml_format = (which_format == :new) + previous_use_new_yaml_format = Puppet::Graph::SimpleGraph.use_new_yaml_format + Puppet::Graph::SimpleGraph.use_new_yaml_format = (which_format == :new) ZAML.dump(graph) ensure - Puppet::SimpleGraph.use_new_yaml_format = previous_use_new_yaml_format + Puppet::Graph::SimpleGraph.use_new_yaml_format = previous_use_new_yaml_format end # Test serialization of graph to YAML. [:old, :new].each do |which_format| all_test_graphs.each do |graph_to_test| it "should be able to serialize #{graph_to_test} to YAML (#{which_format} format)", :if => (RUBY_VERSION[0,3] == '1.8' or YAML::ENGINE.syck?) do - graph = Puppet::SimpleGraph.new + graph = Puppet::Graph::SimpleGraph.new send(graph_to_test, graph) yaml_form = graph_to_yaml(graph, which_format) @@ -805,7 +593,7 @@ describe Puppet::SimpleGraph do # Check that the object contains instance variables @edges and # @vertices only. @reversal is also permitted, but we don't # check it, because it is going to be phased out. - serialized_object.type_id.should == 'object:Puppet::SimpleGraph' + serialized_object.type_id.should == 'object:Puppet::Graph::SimpleGraph' serialized_object.value.keys.reject { |x| x == 'reversal' }.sort.should == ['edges', 'vertices'] # Check edges by forming a set of tuples (source, target, @@ -829,7 +617,7 @@ describe Puppet::SimpleGraph do vertices.should be_a(Hash) Set.new(vertices.keys).should == Set.new(graph.vertices) vertices.each do |key, value| - value.type_id.should == 'object:Puppet::SimpleGraph::VertexWrapper' + value.type_id.should == 'object:Puppet::Graph::SimpleGraph::VertexWrapper' value.value.keys.sort.should == %w{adjacencies vertex} value.value['vertex'].should equal(key) adjacencies = value.value['adjacencies'] @@ -867,7 +655,7 @@ describe Puppet::SimpleGraph do # tested. all_test_graphs.each do |graph_to_test| it "should be able to deserialize #{graph_to_test} from YAML (#{which_format} format)" do - reference_graph = Puppet::SimpleGraph.new + reference_graph = Puppet::Graph::SimpleGraph.new send(graph_to_test, reference_graph) yaml_form = graph_to_yaml(reference_graph, which_format) recovered_graph = YAML.load(yaml_form) @@ -897,7 +685,7 @@ describe Puppet::SimpleGraph do end it "should be able to serialize a graph where the vertices contain backreferences to the graph (#{which_format} format)" do - reference_graph = Puppet::SimpleGraph.new + reference_graph = Puppet::Graph::SimpleGraph.new vertex = Object.new vertex.instance_eval { @graph = reference_graph } reference_graph.add_edge(vertex, :other_vertex) @@ -915,7 +703,7 @@ describe Puppet::SimpleGraph do end it "should serialize properly when used as a base class" do - class Puppet::TestDerivedClass < Puppet::SimpleGraph + class Puppet::TestDerivedClass < Puppet::Graph::SimpleGraph attr_accessor :foo end derived = Puppet::TestDerivedClass.new diff --git a/spec/unit/graph/title_hash_prioritizer_spec.rb b/spec/unit/graph/title_hash_prioritizer_spec.rb new file mode 100644 index 000000000..3818c3b79 --- /dev/null +++ b/spec/unit/graph/title_hash_prioritizer_spec.rb @@ -0,0 +1,49 @@ +require 'spec_helper' +require 'puppet/graph' + +describe Puppet::Graph::TitleHashPrioritizer do + it "produces different priorities for different resource references" do + prioritizer = Puppet::Graph::TitleHashPrioritizer.new + + expect(prioritizer.generate_priority_for(resource(:notify, "one"))).to_not( + eq(prioritizer.generate_priority_for(resource(:notify, "two")))) + end + + it "always produces the same priority for the same resource ref" do + a_prioritizer = Puppet::Graph::TitleHashPrioritizer.new + another_prioritizer = Puppet::Graph::TitleHashPrioritizer.new + + expect(a_prioritizer.generate_priority_for(resource(:notify, "one"))).to( + eq(another_prioritizer.generate_priority_for(resource(:notify, "one")))) + end + + it "does not use the container when generating priorities" do + prioritizer = Puppet::Graph::TitleHashPrioritizer.new + + expect(prioritizer.generate_priority_contained_in(nil, resource(:notify, "one"))).to( + eq(prioritizer.generate_priority_for(resource(:notify, "one")))) + end + + it "can retrieve a previously provided priority with the same resource" do + prioritizer = Puppet::Graph::TitleHashPrioritizer.new + resource = resource(:notify, "title") + + generated = prioritizer.generate_priority_for(resource) + + expect(prioritizer.priority_of(resource)).to eq(generated) + end + + it "can not retrieve the priority of a resource with a different resource with the same title" do + prioritizer = Puppet::Graph::TitleHashPrioritizer.new + resource = resource(:notify, "title") + different_resource = resource(:notify, "title") + + generated = prioritizer.generate_priority_for(resource) + + expect(prioritizer.priority_of(different_resource)).to be_nil + end + + def resource(type, title) + Puppet::Resource.new(type, title) + end +end diff --git a/spec/unit/hiera_puppet_spec.rb b/spec/unit/hiera_puppet_spec.rb index a3ff2bfa9..6c0f882a0 100644 --- a/spec/unit/hiera_puppet_spec.rb +++ b/spec/unit/hiera_puppet_spec.rb @@ -35,7 +35,7 @@ describe 'HieraPuppet' do end describe 'HieraPuppet#hiera_config_file' do - it "should return nil when we cannot derive the hiera config file form Puppet.settings" do + it "should return nil when we cannot derive the hiera config file from Puppet.settings" do begin Puppet.settings[:hiera_config] = nil rescue ArgumentError => detail diff --git a/spec/unit/indirector/catalog/active_record_spec.rb b/spec/unit/indirector/catalog/active_record_spec.rb index 183070698..b86beb24e 100755 --- a/spec/unit/indirector/catalog/active_record_spec.rb +++ b/spec/unit/indirector/catalog/active_record_spec.rb @@ -89,9 +89,11 @@ describe "Puppet::Resource::Catalog::ActiveRecord", :if => can_use_scratch_datab end it "should set the last compile time on the host" do - now = Time.now + before = Time.now terminus.save(request) - Puppet::Rails::Host.find_by_name("foo").last_compile.should be_within(1).of(now) + after = Time.now + + Puppet::Rails::Host.find_by_name("foo").last_compile.should be_between(before, after) end it "should save the Rails host instance" do diff --git a/spec/unit/indirector/catalog/compiler_spec.rb b/spec/unit/indirector/catalog/compiler_spec.rb index 0d000451b..ff32f7c7b 100755 --- a/spec/unit/indirector/catalog/compiler_spec.rb +++ b/spec/unit/indirector/catalog/compiler_spec.rb @@ -133,50 +133,44 @@ describe Puppet::Resource::Catalog::Compiler do describe "when extracting facts from the request" do before do + Puppet::Node::Facts.indirection.terminus_class = :memory Facter.stubs(:value).returns "something" @compiler = Puppet::Resource::Catalog::Compiler.new - @request = Puppet::Indirector::Request.new(:catalog, :find, "hostname", nil) @facts = Puppet::Node::Facts.new('hostname', "fact" => "value", "architecture" => "i386") - Puppet::Node::Facts.indirection.stubs(:save).returns(nil) + end + + def a_request_that_contains(facts) + request = Puppet::Indirector::Request.new(:catalog, :find, "hostname", nil) + request.options[:facts_format] = "pson" + request.options[:facts] = CGI.escape(facts.render(:pson)) + request end it "should do nothing if no facts are provided" do - Puppet::Node::Facts.indirection.expects(:convert_from).never - @request.options[:facts] = nil + request = Puppet::Indirector::Request.new(:catalog, :find, "hostname", nil) + request.options[:facts] = nil - @compiler.extract_facts_from_request(@request) + @compiler.extract_facts_from_request(request).should be_nil end - it "should use the Facts class to deserialize the provided facts and update the timestamp" do - @request.options[:facts_format] = "foo" - @request.options[:facts] = "bar" - Puppet::Node::Facts.expects(:convert_from).returns @facts - + it "deserializes the facts and timestamps them" do @facts.timestamp = Time.parse('2010-11-01') - @now = Time.parse('2010-11-02') - Time.stubs(:now).returns(@now) - - @compiler.extract_facts_from_request(@request) - @facts.timestamp.should == @now - end + request = a_request_that_contains(@facts) + now = Time.parse('2010-11-02') + Time.stubs(:now).returns(now) - it "should use the provided fact format" do - @request.options[:facts_format] = "foo" - @request.options[:facts] = "bar" - Puppet::Node::Facts.expects(:convert_from).with { |format, text| format == "foo" }.returns @facts + facts = @compiler.extract_facts_from_request(request) - @compiler.extract_facts_from_request(@request) + facts.timestamp.should == now end it "should convert the facts into a fact instance and save it" do - @request.options[:facts_format] = "foo" - @request.options[:facts] = "bar" - Puppet::Node::Facts.expects(:convert_from).returns @facts + request = a_request_that_contains(@facts) - Puppet::Node::Facts.indirection.expects(:save).with(@facts) + Puppet::Node::Facts.indirection.expects(:save).with(equals(@facts)) - @compiler.extract_facts_from_request(@request) + @compiler.extract_facts_from_request(request) end end diff --git a/spec/unit/indirector/face_spec.rb b/spec/unit/indirector/face_spec.rb index 73bee5645..b4c601f72 100755 --- a/spec/unit/indirector/face_spec.rb +++ b/spec/unit/indirector/face_spec.rb @@ -33,7 +33,7 @@ describe Puppet::Indirector::Face do describe "as an instance" do it "should be able to determine its indirection" do - # Loading actions here an get, um, complicated + # Loading actions here can get, um, complicated Puppet::Face.stubs(:load_actions) Puppet::Indirector::Face.new(:catalog, '0.0.1').indirection.should equal(Puppet::Resource::Catalog.indirection) end diff --git a/spec/unit/indirector/facts/facter_spec.rb b/spec/unit/indirector/facts/facter_spec.rb index 089c652f4..3e301a5a5 100755 --- a/spec/unit/indirector/facts/facter_spec.rb +++ b/spec/unit/indirector/facts/facter_spec.rb @@ -80,13 +80,23 @@ describe Puppet::Node::Facts::Facter do @facter.find(@request) end - it "should convert all facts into strings" do + it "should convert facts into strings when stringify_facts is true" do + Puppet[:stringify_facts] = true facts = Puppet::Node::Facts.new("foo") Puppet::Node::Facts.expects(:new).returns facts facts.expects(:stringify) @facter.find(@request) end + + it "should sanitize facts when stringify_facts is false" do + Puppet[:stringify_facts] = false + facts = Puppet::Node::Facts.new("foo") + Puppet::Node::Facts.expects(:new).returns facts + facts.expects(:sanitize) + + @facter.find(@request) + end end describe Puppet::Node::Facts::Facter, " when saving facts" do diff --git a/spec/unit/indirector/facts/network_device_spec.rb b/spec/unit/indirector/facts/network_device_spec.rb index 4ed197f85..5f1367e88 100755 --- a/spec/unit/indirector/facts/network_device_spec.rb +++ b/spec/unit/indirector/facts/network_device_spec.rb @@ -55,13 +55,23 @@ describe Puppet::Node::Facts::NetworkDevice do @device.find(@request) end - it "should convert all facts into strings" do + it "should convert facts into strings when stringify_facts is true" do + Puppet[:stringify_facts] = true facts = Puppet::Node::Facts.new("foo") Puppet::Node::Facts.expects(:new).returns facts facts.expects(:stringify) @device.find(@request) end + + it "should sanitizer facts when stringify_facts is false" do + Puppet[:stringify_facts] = false + facts = Puppet::Node::Facts.new("foo") + Puppet::Node::Facts.expects(:new).returns facts + facts.expects(:sanitize) + + @device.find(@request) + end end describe Puppet::Node::Facts::NetworkDevice, " when saving facts" do diff --git a/spec/unit/indirector/file_bucket_file/file_spec.rb b/spec/unit/indirector/file_bucket_file/file_spec.rb index 9bf8f4c9b..37529e5d5 100755 --- a/spec/unit/indirector/file_bucket_file/file_spec.rb +++ b/spec/unit/indirector/file_bucket_file/file_spec.rb @@ -28,6 +28,12 @@ describe Puppet::FileBucketFile::File do end describe "when servicing a save request" do + it "should return a result whose content is empty" do + bucket_file = Puppet::FileBucket::File.new('stuff') + result = Puppet::FileBucket::File.indirection.save(bucket_file, "md5/c13d88cb4cb02003daedb8a84e5d272a") + result.contents.should be_empty + end + describe "when supplying a path" do it "should store the path if not already stored" do checksum = save_bucket_file("stuff\r\n", "/foo/bar") diff --git a/spec/unit/indirector/hiera_spec.rb b/spec/unit/indirector/hiera_spec.rb index 794a4b01f..36b598c37 100644 --- a/spec/unit/indirector/hiera_spec.rb +++ b/spec/unit/indirector/hiera_spec.rb @@ -127,7 +127,7 @@ describe Puppet::Indirector::Hiera do let(:data_binder) { @hiera_class.new } - it "support looking up an integer" do + it "should support looking up an integer" do data_binder.find(request_integer).should == 3000 end diff --git a/spec/unit/indirector/instrumentation_data/local_spec.rb b/spec/unit/indirector/instrumentation_data/local_spec.rb index e6344905f..062e5cf67 100644 --- a/spec/unit/indirector/instrumentation_data/local_spec.rb +++ b/spec/unit/indirector/instrumentation_data/local_spec.rb @@ -28,7 +28,7 @@ describe Puppet::Indirector::InstrumentationData::Local do end describe "when finding instrumentation data" do - it "should return a Instrumentation Data instance matching the key" do + it "should return an Instrumentation Data instance matching the key" do end end diff --git a/spec/unit/indirector/instrumentation_listener/local_spec.rb b/spec/unit/indirector/instrumentation_listener/local_spec.rb index 31ca04aea..195528167 100644 --- a/spec/unit/indirector/instrumentation_listener/local_spec.rb +++ b/spec/unit/indirector/instrumentation_listener/local_spec.rb @@ -28,7 +28,7 @@ describe Puppet::Indirector::InstrumentationListener::Local do end describe "when finding listeners" do - it "should return a Instrumentation Listener instance matching the key" do + it "should return an Instrumentation Listener instance matching the key" do Puppet::Util::Instrumentation.expects(:[]).with("me").returns(:instance) @listener.find(@request).should == :instance end diff --git a/spec/unit/indirector/request_spec.rb b/spec/unit/indirector/request_spec.rb index 3113207a3..46de7c99d 100755 --- a/spec/unit/indirector/request_spec.rb +++ b/spec/unit/indirector/request_spec.rb @@ -33,15 +33,15 @@ describe Puppet::Indirector::Request do end it "should support options specified as a hash" do - lambda { Puppet::Indirector::Request.new(:ind, :method, :key, nil, :one => :two) }.should_not raise_error(ArgumentError) + expect { Puppet::Indirector::Request.new(:ind, :method, :key, nil, :one => :two) }.to_not raise_error end it "should support nil options" do - lambda { Puppet::Indirector::Request.new(:ind, :method, :key, nil, nil) }.should_not raise_error(ArgumentError) + expect { Puppet::Indirector::Request.new(:ind, :method, :key, nil, nil) }.to_not raise_error end it "should support unspecified options" do - lambda { Puppet::Indirector::Request.new(:ind, :method, :key, nil) }.should_not raise_error(ArgumentError) + expect { Puppet::Indirector::Request.new(:ind, :method, :key, nil) }.to_not raise_error end it "should use an empty options hash if nil was provided" do @@ -190,7 +190,7 @@ describe Puppet::Indirector::Request do Puppet::Indirector::Indirection.expects(:instance).with(:myind).returns nil request = Puppet::Indirector::Request.new(:myind, :method, :key, nil) - lambda { request.model }.should raise_error(ArgumentError) + expect { request.model }.to raise_error(ArgumentError) end it "should have a method for determining if the request is plural or singular" do @@ -243,76 +243,129 @@ describe Puppet::Indirector::Request do end describe "when building a query string from its options" do - before do - @request = Puppet::Indirector::Request.new(:myind, :find, "my key", nil) + def a_request_with_options(options) + Puppet::Indirector::Request.new(:myind, :find, "my key", nil, options) + end + + def the_parsed_query_string_from(request) + CGI.parse(request.query_string.sub(/^\?/, '')) end it "should return an empty query string if there are no options" do - @request.stubs(:options).returns nil - @request.query_string.should == "" + request = a_request_with_options(nil) + + request.query_string.should == "" end it "should return an empty query string if the options are empty" do - @request.stubs(:options).returns({}) - @request.query_string.should == "" + request = a_request_with_options({}) + + request.query_string.should == "" end it "should prefix the query string with '?'" do - @request.stubs(:options).returns(:one => "two") - @request.query_string.should =~ /^\?/ + request = a_request_with_options(:one => "two") + + request.query_string.should =~ /^\?/ end it "should include all options in the query string, separated by '&'" do - @request.stubs(:options).returns(:one => "two", :three => "four") - @request.query_string.sub(/^\?/, '').split("&").sort.should == %w{one=two three=four}.sort + request = a_request_with_options(:one => "two", :three => "four") + + the_parsed_query_string_from(request).should == { + "one" => ["two"], + "three" => ["four"] + } end it "should ignore nil options" do - @request.stubs(:options).returns(:one => "two", :three => nil) - @request.query_string.should_not be_include("three") + request = a_request_with_options(:one => "two", :three => nil) + + the_parsed_query_string_from(request).should == { + "one" => ["two"] + } end it "should convert 'true' option values into strings" do - @request.stubs(:options).returns(:one => true) - @request.query_string.should == "?one=true" + request = a_request_with_options(:one => true) + + the_parsed_query_string_from(request).should == { + "one" => ["true"] + } end it "should convert 'false' option values into strings" do - @request.stubs(:options).returns(:one => false) - @request.query_string.should == "?one=false" + request = a_request_with_options(:one => false) + + the_parsed_query_string_from(request).should == { + "one" => ["false"] + } end it "should convert to a string all option values that are integers" do - @request.stubs(:options).returns(:one => 50) - @request.query_string.should == "?one=50" + request = a_request_with_options(:one => 50) + + the_parsed_query_string_from(request).should == { + "one" => ["50"] + } end it "should convert to a string all option values that are floating point numbers" do - @request.stubs(:options).returns(:one => 1.2) - @request.query_string.should == "?one=1.2" + request = a_request_with_options(:one => 1.2) + + the_parsed_query_string_from(request).should == { + "one" => ["1.2"] + } end it "should CGI-escape all option values that are strings" do - escaping = CGI.escape("one two") - @request.stubs(:options).returns(:one => "one two") - @request.query_string.should == "?one=#{escaping}" + request = a_request_with_options(:one => "one two") + + the_parsed_query_string_from(request).should == { + "one" => ["one two"] + } end - it "should YAML-dump and CGI-escape arrays" do - escaping = CGI.escape(YAML.dump(%w{one two})) - @request.stubs(:options).returns(:one => %w{one two}) - @request.query_string.should == "?one=#{escaping}" + it "should convert an array of values into multiple entries for the same key" do + request = a_request_with_options(:one => %w{one two}) + + the_parsed_query_string_from(request).should == { + "one" => ["one", "two"] + } + end + + it "should stringify simple data types inside an array" do + request = a_request_with_options(:one => ['one', nil]) + + the_parsed_query_string_from(request).should == { + "one" => ["one"] + } + end + + it "should error if an array contains another array" do + request = a_request_with_options(:one => ['one', ["not allowed"]]) + + expect { request.query_string }.to raise_error(ArgumentError) + end + + it "should error if an array contains illegal data" do + request = a_request_with_options(:one => ['one', { :not => "allowed" }]) + + expect { request.query_string }.to raise_error(ArgumentError) end it "should convert to a string and CGI-escape all option values that are symbols" do - escaping = CGI.escape("sym bol") - @request.stubs(:options).returns(:one => :"sym bol") - @request.query_string.should == "?one=#{escaping}" + request = a_request_with_options(:one => :"sym bol") + + the_parsed_query_string_from(request).should == { + "one" => ["sym bol"] + } end it "should fail if options other than booleans or strings are provided" do - @request.stubs(:options).returns(:one => {:one => :two}) - lambda { @request.query_string }.should raise_error(ArgumentError) + request = a_request_with_options(:one => { :one => :two }) + + expect { request.query_string }.to raise_error(ArgumentError) end end @@ -372,7 +425,7 @@ describe Puppet::Indirector::Request do it "should fail if no key is provided" do json = PSON.parse(@request.to_pson) json['data'].delete("key") - lambda { from_json(json.to_pson) }.should raise_error(ArgumentError) + expect { from_json(json.to_pson) }.to raise_error(ArgumentError) end it "should set its indirector name" do @@ -382,7 +435,7 @@ describe Puppet::Indirector::Request do it "should fail if no type is provided" do json = PSON.parse(@request.to_pson) json['data'].delete("type") - lambda { from_json(json.to_pson) }.should raise_error(ArgumentError) + expect { from_json(json.to_pson) }.to raise_error(ArgumentError) end it "should set its method" do @@ -392,7 +445,7 @@ describe Puppet::Indirector::Request do it "should fail if no method is provided" do json = PSON.parse(@request.to_pson) json['data'].delete("method") - lambda { from_json(json.to_pson) }.should raise_error(ArgumentError) + expect { from_json(json.to_pson) }.to raise_error(ArgumentError) end it "should initialize with all attributes and options" do diff --git a/spec/unit/indirector/rest_spec.rb b/spec/unit/indirector/rest_spec.rb index ff7f98a1b..8a3fbca7f 100755 --- a/spec/unit/indirector/rest_spec.rb +++ b/spec/unit/indirector/rest_spec.rb @@ -101,6 +101,7 @@ describe Puppet::Indirector::REST do end after :all do + Puppet::TestModel.indirection.delete # Remove the class, unlinking it from the rest of the system. Puppet.send(:remove_const, :TestModel) end diff --git a/spec/unit/indirector_spec.rb b/spec/unit/indirector_spec.rb index 334fdfdc4..913c0fe87 100755 --- a/spec/unit/indirector_spec.rb +++ b/spec/unit/indirector_spec.rb @@ -118,9 +118,9 @@ describe Puppet::Indirector, "when registering an indirection" do @indirection = @thingie.indirects :first, :some => :options end - it "should extend the class with the Format Handler" do + it "should extend the class to handle serialization" do @indirection = @thingie.indirects :first - @thingie.singleton_class.ancestors.should be_include(Puppet::Network::FormatHandler) + @thingie.should respond_to(:convert_from) end after do diff --git a/spec/unit/interface/option_builder_spec.rb b/spec/unit/interface/option_builder_spec.rb index a8c5bda39..cc4eba2f6 100755 --- a/spec/unit/interface/option_builder_spec.rb +++ b/spec/unit/interface/option_builder_spec.rb @@ -1,3 +1,4 @@ +require 'spec_helper' require 'puppet/interface' describe Puppet::Interface::OptionBuilder do diff --git a/spec/unit/interface/option_spec.rb b/spec/unit/interface/option_spec.rb index 55ae6a50f..2299e70d0 100755 --- a/spec/unit/interface/option_spec.rb +++ b/spec/unit/interface/option_spec.rb @@ -1,3 +1,4 @@ +require 'spec_helper' require 'puppet/interface' describe Puppet::Interface::Option do diff --git a/spec/unit/interface_spec.rb b/spec/unit/interface_spec.rb index 8ccc4a6d5..ca18a6b0c 100755 --- a/spec/unit/interface_spec.rb +++ b/spec/unit/interface_spec.rb @@ -24,11 +24,11 @@ describe Puppet::Interface do end it "should raise an exception when the requested version is unavailable" do - expect { subject[:huzzah, '17.0.0'] }.to raise_error, Puppet::Error + expect { subject[:huzzah, '17.0.0'] }.to raise_error(Puppet::Error, /Could not find version/) end it "should raise an exception when the requested face doesn't exist" do - expect { subject[:burrble_toot, :current] }.to raise_error, Puppet::Error + expect { subject[:burrble_toot, :current] }.to raise_error(Puppet::Error, /Could not find Puppet Face/) end describe "version matching" do diff --git a/spec/unit/module_tool/applications/installer_spec.rb b/spec/unit/module_tool/applications/installer_spec.rb index dbec21fa1..3d3461949 100644 --- a/spec/unit/module_tool/applications/installer_spec.rb +++ b/spec/unit/module_tool/applications/installer_spec.rb @@ -42,6 +42,17 @@ describe Puppet::ModuleTool::Applications::Installer, :unless => Puppet.features let(:remote_dependency_info) do { + "pmtacceptance/apache" => [ + { "dependencies" => [], + "version" => "1.0.0-alpha", + "file" => "/pmtacceptance-apache-1.0.0-alpha.tar.gz" }, + { "dependencies" => [], + "version" => "1.0.0-beta", + "file" => "/pmtacceptance-apache-1.0.0-beta.tar.gz" }, + { "dependencies" => [], + "version" => "1.0.0-rc1", + "file" => "/pmtacceptance-apache-1.0.0-rc1.tar.gz" }, + ], "pmtacceptance/stdlib" => [ { "dependencies" => [], "version" => "0.0.1", @@ -50,8 +61,14 @@ describe Puppet::ModuleTool::Applications::Installer, :unless => Puppet.features "version" => "0.0.2", "file" => "/pmtacceptance-stdlib-0.0.2.tar.gz" }, { "dependencies" => [], + "version" => "1.0.0-pre", + "file" => "/pmtacceptance-stdlib-1.0.0-pre.tar.gz" }, + { "dependencies" => [], "version" => "1.0.0", - "file" => "/pmtacceptance-stdlib-1.0.0.tar.gz" } + "file" => "/pmtacceptance-stdlib-1.0.0.tar.gz" }, + { "dependencies" => [], + "version" => "1.5.0-pre", + "file" => "/pmtacceptance-stdlib-1.5.0-pre.tar.gz" }, ], "pmtacceptance/java" => [ { "dependencies" => [["pmtacceptance/stdlib", ">= 0.0.1"]], @@ -97,7 +114,7 @@ describe Puppet::ModuleTool::Applications::Installer, :unless => Puppet.features raise_error(ArgumentError, "Could not install module with invalid name: puppet") end - it "should install the requested module" do + it "should install the current stable version of the requested module" do Puppet::ModuleTool::Applications::Unpacker.expects(:new). with('/fake_cache/pmtacceptance-stdlib-1.0.0.tar.gz', options). returns(unpacker) @@ -107,6 +124,36 @@ describe Puppet::ModuleTool::Applications::Installer, :unless => Puppet.features results[:installed_modules][0][:version][:vstring].should == "1.0.0" end + it "should install the most recent version of requested module in the absence of a stable version" do + Puppet::ModuleTool::Applications::Unpacker.expects(:new). + with('/fake_cache/pmtacceptance-apache-1.0.0-rc1.tar.gz', options). + returns(unpacker) + results = installer_class.run('pmtacceptance-apache', forge, install_dir, options) + results[:installed_modules].length == 1 + results[:installed_modules][0][:module].should == "pmtacceptance-apache" + results[:installed_modules][0][:version][:vstring].should == "1.0.0-rc1" + end + + it "should install the most recent stable version of requested module for the requested version range" do + Puppet::ModuleTool::Applications::Unpacker.expects(:new). + with('/fake_cache/pmtacceptance-stdlib-1.0.0.tar.gz', options.merge(:version => '1.x')). + returns(unpacker) + results = installer_class.run('pmtacceptance-stdlib', forge, install_dir, options.merge(:version => '1.x')) + results[:installed_modules].length == 1 + results[:installed_modules][0][:module].should == "pmtacceptance-stdlib" + results[:installed_modules][0][:version][:vstring].should == "1.0.0" + end + + it "should install the most recent version of requested module for the requested version range in the absence of a stable version" do + Puppet::ModuleTool::Applications::Unpacker.expects(:new). + with('/fake_cache/pmtacceptance-stdlib-1.5.0-pre.tar.gz', options.merge(:version => '1.5.0-pre')). + returns(unpacker) + results = installer_class.run('pmtacceptance-stdlib', forge, install_dir, options.merge(:version => '1.5.0-pre')) + results[:installed_modules].length == 1 + results[:installed_modules][0][:module].should == "pmtacceptance-stdlib" + results[:installed_modules][0][:version][:vstring].should == "1.5.0-pre" + end + context "should check the target directory" do let(:installer) do installer_class.new('pmtacceptance-stdlib', forge, install_dir, options) diff --git a/spec/unit/module_tool/metadata_spec.rb b/spec/unit/module_tool/metadata_spec.rb index af86e4d68..0d364773c 100644 --- a/spec/unit/module_tool/metadata_spec.rb +++ b/spec/unit/module_tool/metadata_spec.rb @@ -8,4 +8,17 @@ describe Puppet::ModuleTool::Metadata do metadata.license.should == "Apache License, Version 2.0" end end + + describe :to_hash do + it 'should merge extra_data in' do + metadata = Puppet::ModuleTool::Metadata.new + metadata.extra_metadata = { + 'checksums' => 'badsums', + 'special_key' => 'special' + } + meta_hash = metadata.to_hash + meta_hash['special_key'].should == 'special' + meta_hash['checksums'].should == {} + end + end end diff --git a/spec/unit/network/authstore_spec.rb b/spec/unit/network/authstore_spec.rb index 54815ca32..75ea2c13e 100755 --- a/spec/unit/network/authstore_spec.rb +++ b/spec/unit/network/authstore_spec.rb @@ -82,7 +82,7 @@ describe Puppet::Network::AuthStore::Declaration do end (1..3).each { |n| - describe "when the pattern is a IP mask with #{n} numeric segments and a *" do + describe "when the pattern is an IP mask with #{n} numeric segments and a *" do before :each do @ip_pattern = ip.split('.')[0,n].join('.')+'.*' @declaration = Puppet::Network::AuthStore::Declaration.new(:allow_ip,@ip_pattern) diff --git a/spec/unit/network/format_handler_spec.rb b/spec/unit/network/format_handler_spec.rb index f4944bab9..b8ab39634 100755 --- a/spec/unit/network/format_handler_spec.rb +++ b/spec/unit/network/format_handler_spec.rb @@ -3,209 +3,33 @@ require 'spec_helper' require 'puppet/network/format_handler' -class FormatTester - extend Puppet::Network::FormatHandler -end - describe Puppet::Network::FormatHandler do - after do - formats = Puppet::Network::FormatHandler.instance_variable_get("@formats") - formats.each do |name, format| - formats.delete(name) unless format.is_a?(Puppet::Network::Format) - end + before(:each) do + @saved_formats = Puppet::Network::FormatHandler.instance_variable_get(:@formats).dup + Puppet::Network::FormatHandler.instance_variable_set(:@formats, {}) end - it "should be able to list supported formats" do - FormatTester.should respond_to(:supported_formats) + after(:each) do + Puppet::Network::FormatHandler.instance_variable_set(:@formats, @saved_formats) end - it "should include all supported formats" do - one = stub 'supported', :supported? => true, :name => :one, :weight => 1 - two = stub 'supported', :supported? => false, :name => :two, :weight => 1 - three = stub 'supported', :supported? => true, :name => :three, :weight => 1 - four = stub 'supported', :supported? => false, :name => :four, :weight => 1 - Puppet::Network::FormatHandler.stubs(:formats).returns [:one, :two, :three, :four] - Puppet::Network::FormatHandler.stubs(:format).with(:one).returns one - Puppet::Network::FormatHandler.stubs(:format).with(:two).returns two - Puppet::Network::FormatHandler.stubs(:format).with(:three).returns three - Puppet::Network::FormatHandler.stubs(:format).with(:four).returns four - result = FormatTester.supported_formats - result.length.should == 2 - result.should be_include(:one) - result.should be_include(:three) - end - - it "should return the supported formats in decreasing order of weight" do - one = stub 'supported', :supported? => true, :name => :one, :weight => 1 - two = stub 'supported', :supported? => true, :name => :two, :weight => 6 - three = stub 'supported', :supported? => true, :name => :three, :weight => 2 - four = stub 'supported', :supported? => true, :name => :four, :weight => 8 - Puppet::Network::FormatHandler.stubs(:formats).returns [:one, :two, :three, :four] - Puppet::Network::FormatHandler.stubs(:format).with(:one).returns one - Puppet::Network::FormatHandler.stubs(:format).with(:two).returns two - Puppet::Network::FormatHandler.stubs(:format).with(:three).returns three - Puppet::Network::FormatHandler.stubs(:format).with(:four).returns four - FormatTester.supported_formats.should == [:four, :two, :three, :one] - end - - - describe "with a preferred serialization format setting" do - before do - one = stub 'supported', :supported? => true, :name => :one, :weight => 1 - two = stub 'supported', :supported? => true, :name => :two, :weight => 6 - Puppet::Network::FormatHandler.stubs(:formats).returns [:one, :two] - Puppet::Network::FormatHandler.stubs(:format).with(:one).returns one - Puppet::Network::FormatHandler.stubs(:format).with(:two).returns two - end - describe "that is supported" do - before do - Puppet[:preferred_serialization_format] = :one - end - it "should return the preferred serialization format first" do - FormatTester.supported_formats.should == [:one, :two] - end - end - describe "that is not supported" do - before do - Puppet[:preferred_serialization_format] = :unsupported - end - it "should still return the default format first" do - FormatTester.supported_formats.should == [:two, :one] - end - it "should log a debug message" do - Puppet.expects(:debug).with("Value of 'preferred_serialization_format' (unsupported) is invalid for FormatTester, using default (two)") - Puppet.expects(:debug).with("FormatTester supports formats: one two; using two") - FormatTester.supported_formats + describe "when creating formats" do + it "should instance_eval any block provided when creating a format" do + format = Puppet::Network::FormatHandler.create(:test_format) do + def asdfghjkl; end end + format.should respond_to(:asdfghjkl) end end - it "should return the first format as the default format" do - FormatTester.expects(:supported_formats).returns [:one, :two] - FormatTester.default_format.should == :one - end - - it "should be able to use a protected format for better logging on errors" do - Puppet::Network::FormatHandler.should respond_to(:protected_format) - end - - it "should delegate all methods from the informative format to the specified format" do - format = mock 'format' - format.stubs(:name).returns(:myformat) - Puppet::Network::FormatHandler.expects(:format).twice.with(:myformat).returns format - - format.expects(:render).with("foo").returns "yay" - Puppet::Network::FormatHandler.protected_format(:myformat).render("foo").should == "yay" - end - - it "should provide better logging if a failure is encountered when delegating from the informative format to the real format" do - format = mock 'format' - format.stubs(:name).returns(:myformat) - Puppet::Network::FormatHandler.expects(:format).twice.with(:myformat).returns format - - format.expects(:render).with("foo").raises "foo" - lambda { Puppet::Network::FormatHandler.protected_format(:myformat).render("foo") }.should raise_error(Puppet::Network::FormatHandler::FormatError) - end - - it "should raise an error if we couldn't find a format by name or mime-type" do - Puppet::Network::FormatHandler.stubs(:format).with(:myformat).returns nil - lambda { Puppet::Network::FormatHandler.protected_format(:myformat) }.should raise_error - end - - describe "when using formats" do - before do - @format = mock 'format' - @format.stubs(:supported?).returns true - @format.stubs(:name).returns :my_format - Puppet::Network::FormatHandler.stubs(:format).with(:my_format).returns @format - Puppet::Network::FormatHandler.stubs(:mime).with("text/myformat").returns @format - Puppet::Network::Format.stubs(:===).returns false - Puppet::Network::Format.stubs(:===).with(@format).returns true - end - - it "should be able to test whether a format is supported" do - FormatTester.should respond_to(:support_format?) - end - - it "should use the Format to determine whether a given format is supported" do - @format.expects(:supported?).with(FormatTester) - FormatTester.support_format?(:my_format) - end - - it "should be able to convert from a given format" do - FormatTester.should respond_to(:convert_from) - end - - it "should call the format-specific converter when asked to convert from a given format" do - @format.expects(:intern).with(FormatTester, "mydata") - FormatTester.convert_from(:my_format, "mydata") - end - - it "should call the format-specific converter when asked to convert from a given format by mime-type" do - @format.expects(:intern).with(FormatTester, "mydata") - FormatTester.convert_from("text/myformat", "mydata") - end - - it "should call the format-specific converter when asked to convert from a given format by format instance" do - @format.expects(:intern).with(FormatTester, "mydata") - FormatTester.convert_from(@format, "mydata") - end - - it "should raise a FormatError when an exception is encountered when converting from a format" do - @format.expects(:intern).with(FormatTester, "mydata").raises "foo" - lambda { FormatTester.convert_from(:my_format, "mydata") }.should raise_error(Puppet::Network::FormatHandler::FormatError) - end - - it "should be able to use a specific hook for converting into multiple instances" do - @format.expects(:intern_multiple).with(FormatTester, "mydata") - - FormatTester.convert_from_multiple(:my_format, "mydata") - end - - it "should raise a FormatError when an exception is encountered when converting multiple items from a format" do - @format.expects(:intern_multiple).with(FormatTester, "mydata").raises "foo" - lambda { FormatTester.convert_from_multiple(:my_format, "mydata") }.should raise_error(Puppet::Network::FormatHandler::FormatError) - end - - it "should be able to use a specific hook for rendering multiple instances" do - @format.expects(:render_multiple).with("mydata") - - FormatTester.render_multiple(:my_format, "mydata") - end - - it "should raise a FormatError when an exception is encountered when rendering multiple items into a format" do - @format.expects(:render_multiple).with("mydata").raises "foo" - lambda { FormatTester.render_multiple(:my_format, "mydata") }.should raise_error(Puppet::Network::FormatHandler::FormatError) - end - end - - describe "when managing formats" do - it "should have a method for defining a new format" do - Puppet::Network::FormatHandler.should respond_to(:create) - end - - it "should create a format instance when asked" do - format = stub 'format', :name => :foo - Puppet::Network::Format.expects(:new).with(:foo).returns format - Puppet::Network::FormatHandler.create(:foo) - end - - it "should instance_eval any block provided when creating a format" do - format = stub 'format', :name => :instance_eval - format.expects(:yayness) - Puppet::Network::Format.expects(:new).returns format - Puppet::Network::FormatHandler.create(:instance_eval) do - yayness - end - end + describe "when retrieving formats" do + let!(:format) { Puppet::Network::FormatHandler.create(:the_format, :extension => "foo", :mime => "foo/bar") } it "should be able to retrieve a format by name" do - format = Puppet::Network::FormatHandler.create(:by_name) - Puppet::Network::FormatHandler.format(:by_name).should equal(format) + Puppet::Network::FormatHandler.format(:the_format).should equal(format) end it "should be able to retrieve a format by extension" do - format = Puppet::Network::FormatHandler.create(:by_extension, :extension => "foo") Puppet::Network::FormatHandler.format_by_extension("foo").should equal(format) end @@ -213,123 +37,58 @@ describe Puppet::Network::FormatHandler do Puppet::Network::FormatHandler.format_by_extension("yayness").should be_nil end - it "should be able to retrieve formats by name irrespective of case and class" do - format = Puppet::Network::FormatHandler.create(:by_name) - Puppet::Network::FormatHandler.format(:By_Name).should equal(format) + it "should be able to retrieve formats by name irrespective of case" do + Puppet::Network::FormatHandler.format(:The_Format).should equal(format) end it "should be able to retrieve a format by mime type" do - format = Puppet::Network::FormatHandler.create(:by_name, :mime => "foo/bar") Puppet::Network::FormatHandler.mime("foo/bar").should equal(format) end it "should be able to retrieve a format by mime type irrespective of case" do - format = Puppet::Network::FormatHandler.create(:by_name, :mime => "foo/bar") Puppet::Network::FormatHandler.mime("Foo/Bar").should equal(format) end - - it "should be able to return all formats" do - one = stub 'one', :name => :one - two = stub 'two', :name => :two - Puppet::Network::Format.expects(:new).with(:one).returns(one) - Puppet::Network::Format.expects(:new).with(:two).returns(two) - - Puppet::Network::FormatHandler.create(:one) - Puppet::Network::FormatHandler.create(:two) - - list = Puppet::Network::FormatHandler.formats - list.should be_include(:one) - list.should be_include(:two) - end end - describe "when an instance" do - it "should be able to test whether a format is supported" do - FormatTester.new.should respond_to(:support_format?) + describe "#most_suitable_format_for" do + before :each do + Puppet::Network::FormatHandler.create(:one, :extension => "foo", :mime => "text/one") + Puppet::Network::FormatHandler.create(:two, :extension => "bar", :mime => "application/two") end - it "should be able to convert to a given format" do - FormatTester.new.should respond_to(:render) - end + let(:format_one) { Puppet::Network::FormatHandler.format(:one) } + let(:format_two) { Puppet::Network::FormatHandler.format(:two) } - it "should be able to get a format mime-type" do - FormatTester.new.should respond_to(:mime) + def suitable_in_setup_formats(accepted) + Puppet::Network::FormatHandler.most_suitable_format_for(accepted, [:one, :two]) end - it "should raise a FormatError when a rendering error is encountered" do - format = stub 'rendering format', :supported? => true, :name => :foo - Puppet::Network::FormatHandler.stubs(:format).with(:foo).returns format - - tester = FormatTester.new - format.expects(:render).with(tester).raises "eh" - - lambda { tester.render(:foo) }.should raise_error(Puppet::Network::FormatHandler::FormatError) + it "finds either format when anything is accepted" do + [format_one, format_two].should include(suitable_in_setup_formats(["*/*"])) end - it "should call the format-specific converter when asked to convert to a given format" do - format = stub 'rendering format', :supported? => true, :name => :foo - - Puppet::Network::FormatHandler.stubs(:format).with(:foo).returns format - - tester = FormatTester.new - format.expects(:render).with(tester).returns "foo" - - tester.render(:foo).should == "foo" + it "finds no format when none are acceptable" do + suitable_in_setup_formats(["three"]).should be_nil end - it "should call the format-specific converter when asked to convert to a given format by mime-type" do - format = stub 'rendering format', :supported? => true, :name => :foo - Puppet::Network::FormatHandler.stubs(:mime).with("text/foo").returns format - Puppet::Network::FormatHandler.stubs(:format).with(:foo).returns format - - tester = FormatTester.new - format.expects(:render).with(tester).returns "foo" - - tester.render("text/foo").should == "foo" + it "skips unsupported, but accepted, formats" do + suitable_in_setup_formats(["three", "two"]).should == format_two end - it "should call the format converter when asked to convert to a given format instance" do - format = stub 'rendering format', :supported? => true, :name => :foo - Puppet::Network::Format.stubs(:===).with(format).returns(true) - Puppet::Network::FormatHandler.stubs(:format).with(:foo).returns format - - tester = FormatTester.new - format.expects(:render).with(tester).returns "foo" - - tester.render(format).should == "foo" + it "gives the first acceptable and suitable format" do + suitable_in_setup_formats(["three", "one", "two"]).should == format_one end - it "should render to the default format if no format is provided when rendering" do - format = stub 'rendering format', :supported? => true, :name => :foo - Puppet::Network::FormatHandler.stubs(:format).with(:foo).returns format - - FormatTester.expects(:default_format).returns :foo - tester = FormatTester.new - - format.expects(:render).with(tester) - tester.render + it "allows specifying acceptable formats by mime type" do + suitable_in_setup_formats(["text/one"]).should == format_one end - it "should call the format-specific converter when asked for the mime-type of a given format" do - format = stub 'rendering format', :supported? => true, :name => :foo - - Puppet::Network::FormatHandler.stubs(:format).with(:foo).returns format - - tester = FormatTester.new - format.expects(:mime).returns "text/foo" - - tester.mime(:foo).should == "text/foo" + it "ignores quality specifiers" do + suitable_in_setup_formats(["two;q=0.8", "text/one;q=0.9"]).should == format_two end - it "should return the default format mime-type if no format is provided" do - format = stub 'rendering format', :supported? => true, :name => :foo - Puppet::Network::FormatHandler.stubs(:format).with(:foo).returns format - - FormatTester.expects(:default_format).returns :foo - tester = FormatTester.new - - format.expects(:mime).returns "text/foo" - tester.mime.should == "text/foo" + it "allows specifying acceptable formats by canonical name" do + suitable_in_setup_formats([:one]).should == format_one end end end diff --git a/spec/unit/network/format_support_spec.rb b/spec/unit/network/format_support_spec.rb new file mode 100644 index 000000000..8e43ae135 --- /dev/null +++ b/spec/unit/network/format_support_spec.rb @@ -0,0 +1,199 @@ +#! /usr/bin/env ruby +require 'spec_helper' + +require 'puppet/network/format_handler' +require 'puppet/network/format_support' + +class FormatTester + include Puppet::Network::FormatSupport +end + +describe Puppet::Network::FormatHandler do + before(:each) do + @saved_formats = Puppet::Network::FormatHandler.instance_variable_get(:@formats).dup + Puppet::Network::FormatHandler.instance_variable_set(:@formats, {}) + end + + after(:each) do + Puppet::Network::FormatHandler.instance_variable_set(:@formats, @saved_formats) + end + + describe "when listing formats" do + before(:each) do + one = Puppet::Network::FormatHandler.create(:one, :weight => 1) + one.stubs(:supported?).returns(true) + two = Puppet::Network::FormatHandler.create(:two, :weight => 6) + two.stubs(:supported?).returns(true) + three = Puppet::Network::FormatHandler.create(:three, :weight => 2) + three.stubs(:supported?).returns(true) + four = Puppet::Network::FormatHandler.create(:four, :weight => 8) + four.stubs(:supported?).returns(false) + end + + it "should return all supported formats in decreasing order of weight" do + FormatTester.supported_formats.should == [:two, :three, :one] + end + end + + it "should return the first format as the default format" do + FormatTester.expects(:supported_formats).returns [:one, :two] + FormatTester.default_format.should == :one + end + + describe "with a preferred serialization format setting" do + before do + one = Puppet::Network::FormatHandler.create(:one, :weight => 1) + one.stubs(:supported?).returns(true) + two = Puppet::Network::FormatHandler.create(:two, :weight => 6) + two.stubs(:supported?).returns(true) + end + + describe "that is supported" do + before do + Puppet[:preferred_serialization_format] = :one + end + + it "should return the preferred serialization format first" do + FormatTester.supported_formats.should == [:one, :two] + end + end + + describe "that is not supported" do + before do + Puppet[:preferred_serialization_format] = :unsupported + end + + it "should return the default format first" do + FormatTester.supported_formats.should == [:two, :one] + end + + it "should log a debug message" do + Puppet.expects(:debug).with("Value of 'preferred_serialization_format' (unsupported) is invalid for FormatTester, using default (two)") + Puppet.expects(:debug).with("FormatTester supports formats: two one") + FormatTester.supported_formats + end + end + end + + describe "when using formats" do + let(:format) { Puppet::Network::FormatHandler.create(:my_format, :mime => "text/myformat") } + + it "should use the Format to determine whether a given format is supported" do + format.expects(:supported?).with(FormatTester) + FormatTester.support_format?(:my_format) + end + + it "should call the format-specific converter when asked to convert from a given format" do + format.expects(:intern).with(FormatTester, "mydata") + FormatTester.convert_from(:my_format, "mydata") + end + + it "should call the format-specific converter when asked to convert from a given format by mime-type" do + format.expects(:intern).with(FormatTester, "mydata") + FormatTester.convert_from("text/myformat", "mydata") + end + + it "should call the format-specific converter when asked to convert from a given format by format instance" do + format.expects(:intern).with(FormatTester, "mydata") + FormatTester.convert_from(format, "mydata") + end + + it "should raise a FormatError when an exception is encountered when converting from a format" do + format.expects(:intern).with(FormatTester, "mydata").raises "foo" + expect do + FormatTester.convert_from(:my_format, "mydata") + end.to raise_error( + Puppet::Network::FormatHandler::FormatError, + 'Could not intern from my_format: foo' + ) + end + + it "should be able to use a specific hook for converting into multiple instances" do + format.expects(:intern_multiple).with(FormatTester, "mydata") + + FormatTester.convert_from_multiple(:my_format, "mydata") + end + + it "should raise a FormatError when an exception is encountered when converting multiple items from a format" do + format.expects(:intern_multiple).with(FormatTester, "mydata").raises "foo" + expect do + FormatTester.convert_from_multiple(:my_format, "mydata") + end.to raise_error(Puppet::Network::FormatHandler::FormatError, 'Could not intern_multiple from my_format: foo') + end + + it "should be able to use a specific hook for rendering multiple instances" do + format.expects(:render_multiple).with("mydata") + + FormatTester.render_multiple(:my_format, "mydata") + end + + it "should raise a FormatError when an exception is encountered when rendering multiple items into a format" do + format.expects(:render_multiple).with("mydata").raises "foo" + expect do + FormatTester.render_multiple(:my_format, "mydata") + end.to raise_error(Puppet::Network::FormatHandler::FormatError, 'Could not render_multiple to my_format: foo') + end + end + + describe "when an instance" do + let(:format) { Puppet::Network::FormatHandler.create(:foo, :mime => "text/foo") } + + it "should list as supported a format that reports itself supported" do + format.expects(:supported?).returns true + FormatTester.new.support_format?(:foo).should be_true + end + + it "should raise a FormatError when a rendering error is encountered" do + tester = FormatTester.new + format.expects(:render).with(tester).raises "eh" + + expect do + tester.render(:foo) + end.to raise_error(Puppet::Network::FormatHandler::FormatError, 'Could not render to foo: eh') + end + + it "should call the format-specific converter when asked to convert to a given format" do + tester = FormatTester.new + format.expects(:render).with(tester).returns "foo" + + tester.render(:foo).should == "foo" + end + + it "should call the format-specific converter when asked to convert to a given format by mime-type" do + tester = FormatTester.new + format.expects(:render).with(tester).returns "foo" + + tester.render("text/foo").should == "foo" + end + + it "should call the format converter when asked to convert to a given format instance" do + tester = FormatTester.new + format.expects(:render).with(tester).returns "foo" + + tester.render(format).should == "foo" + end + + it "should render to the default format if no format is provided when rendering" do + FormatTester.expects(:default_format).returns :foo + tester = FormatTester.new + + format.expects(:render).with(tester) + tester.render + end + + it "should call the format-specific converter when asked for the mime-type of a given format" do + tester = FormatTester.new + format.expects(:mime).returns "text/foo" + + tester.mime(:foo).should == "text/foo" + end + + it "should return the default format mime-type if no format is provided" do + FormatTester.expects(:default_format).returns :foo + tester = FormatTester.new + + format.expects(:mime).returns "text/foo" + tester.mime.should == "text/foo" + end + end +end diff --git a/spec/unit/network/formats_spec.rb b/spec/unit/network/formats_spec.rb index e0abda995..6b96b2b7d 100755 --- a/spec/unit/network/formats_spec.rb +++ b/spec/unit/network/formats_spec.rb @@ -315,8 +315,8 @@ describe "Puppet Network Format" do end [[1, 2], ["one"], [{ 1 => 1 }]].each do |input| - it "should render #{input.inspect} as JSON" do - subject.render(input).should == json.render(input).chomp + it "should render #{input.inspect} as one item per line" do + subject.render(input).should == input.collect { |item| item.to_s + "\n" }.join('') end end diff --git a/spec/unit/network/http/connection_spec.rb b/spec/unit/network/http/connection_spec.rb index c0b293edd..0d6da0671 100644 --- a/spec/unit/network/http/connection_spec.rb +++ b/spec/unit/network/http/connection_spec.rb @@ -39,8 +39,47 @@ describe Puppet::Network::HTTP::Connection do end it "can set ssl using an option" do - Puppet::Network::HTTP::Connection.new(host, port, false).send(:connection).should_not be_use_ssl - Puppet::Network::HTTP::Connection.new(host, port, true).send(:connection).should be_use_ssl + Puppet::Network::HTTP::Connection.new(host, port, :use_ssl => false).send(:connection).should_not be_use_ssl + Puppet::Network::HTTP::Connection.new(host, port, :use_ssl => true).send(:connection).should be_use_ssl + end + + describe "peer verification" do + def setup_standard_ssl_configuration + ca_cert_file = File.expand_path('/path/to/ssl/certs/ca_cert.pem') + FileTest.stubs(:exist?).with(ca_cert_file).returns(true) + + ssl_configuration = stub('ssl_configuration', :ca_auth_file => ca_cert_file) + Puppet::Network::HTTP::Connection.any_instance.stubs(:ssl_configuration).returns(ssl_configuration) + end + + def setup_standard_hostcert + host_cert_file = File.expand_path('/path/to/ssl/certs/host_cert.pem') + FileTest.stubs(:exist?).with(host_cert_file).returns(true) + + Puppet[:hostcert] = host_cert_file + end + + def setup_standard_ssl_host + cert = stub('cert', :content => 'real_cert') + key = stub('key', :content => 'real_key') + host = stub('host', :certificate => cert, :key => key, :ssl_store => stub('store')) + + Puppet::Network::HTTP::Connection.any_instance.stubs(:ssl_host).returns(host) + end + + before do + setup_standard_ssl_configuration + setup_standard_hostcert + setup_standard_ssl_host + end + + it "can enable peer verification" do + Puppet::Network::HTTP::Connection.new(host, port, :verify_peer => true).send(:connection).verify_mode.should == OpenSSL::SSL::VERIFY_PEER + end + + it "can disable peer verification" do + Puppet::Network::HTTP::Connection.new(host, port, :verify_peer => false).send(:connection).verify_mode.should == OpenSSL::SSL::VERIFY_NONE + end end context "proxy and timeout settings should propagate" do @@ -62,6 +101,10 @@ describe Puppet::Network::HTTP::Connection do subject.send(:connection).proxy_address.should be_nil end + it "should raise Puppet::Error when invalid options are specified" do + expect { Puppet::Network::HTTP::Connection.new(host, port, :invalid_option => nil) }.to raise_error(Puppet::Error, 'Unrecognized option(s): :invalid_option') + end + end describe "when doing SSL setup for http instances" do @@ -86,7 +129,7 @@ describe Puppet::Network::HTTP::Connection do end shared_examples "HTTPS setup without all certificates" do - subject { Puppet::Network::HTTP::Connection.new(host, port, true).send(:connection) } + subject { Puppet::Network::HTTP::Connection.new(host, port, :use_ssl => true).send(:connection) } it { should be_use_ssl } its(:cert) { should be_nil } @@ -123,7 +166,7 @@ describe Puppet::Network::HTTP::Connection do end context "with both the host and CA cert" do - subject { Puppet::Network::HTTP::Connection.new(host, port, true).send(:connection) } + subject { Puppet::Network::HTTP::Connection.new(host, port, :use_ssl => true).send(:connection) } before :each do FileTest.expects(:exist?).with(Puppet[:hostcert]).returns(true) @@ -145,11 +188,10 @@ describe Puppet::Network::HTTP::Connection do end end - context "when methods that accept a block are called with a block" do let (:host) { "my_server" } let (:port) { 8140 } - let (:subject) { Puppet::Network::HTTP::Connection.new(host, port, false) } + let (:subject) { Puppet::Network::HTTP::Connection.new(host, port, :use_ssl => false) } let (:httpok) { Net::HTTPOK.new('1.1', 200, '') } before :each do @@ -185,6 +227,7 @@ describe Puppet::Network::HTTP::Connection do let (:host) { "my_server" } let (:port) { 8140 } + let (:httpok) { Net::HTTPOK.new('1.1', 200, '') } let (:subject) { Puppet::Network::HTTP::Connection.new(host, port) } def a_connection_that_verifies(args) @@ -269,11 +312,49 @@ describe Puppet::Network::HTTP::Connection do connection.verify_callback.call(true, context) connection.verify_callback.call(true, context) true - end + end.returns(httpok) subject.expects(:warn_if_near_expiration).with(cert, cert) subject.request(:get, stubs('request')) end end + + context "when response is a redirect" do + let (:other_host) { "redirected" } + let (:other_port) { 9292 } + let (:other_path) { "other-path" } + let (:subject) { Puppet::Network::HTTP::Connection.new("my_server", 8140, :use_ssl => false) } + let (:httpredirection) { Net::HTTPFound.new('1.1', 302, 'Moved Temporarily') } + let (:httpok) { Net::HTTPOK.new('1.1', 200, '') } + + before :each do + httpredirection['location'] = "http://#{other_host}:#{other_port}/#{other_path}" + httpredirection.stubs(:read_body).returns("This resource has moved") + + socket = stub_everything("socket") + TCPSocket.stubs(:open).returns(socket) + + Net::HTTP::Get.any_instance.stubs(:exec).returns("") + Net::HTTP::Post.any_instance.stubs(:exec).returns("") + end + + it "should redirect to the final resource location" do + httpok.stubs(:read_body).returns(:body) + Net::HTTPResponse.stubs(:read_new).returns(httpredirection).then.returns(httpok) + + subject.get("/foo").body.should == :body + subject.port.should == other_port + subject.address.should == other_host + end + + it "should raise an error after too many redirections" do + Net::HTTPResponse.stubs(:read_new).returns(httpredirection) + + expect { + subject.get("/foo") + }.to raise_error(Puppet::Network::HTTP::RedirectionLimitExceededException) + end + end + end diff --git a/spec/unit/network/http/handler_spec.rb b/spec/unit/network/http/handler_spec.rb index 9a93f843d..6d16bd799 100755 --- a/spec/unit/network/http/handler_spec.rb +++ b/spec/unit/network/http/handler_spec.rb @@ -4,8 +4,126 @@ require 'puppet/network/http' require 'puppet/network/http/handler' require 'puppet/network/authorization' require 'puppet/network/authentication' +require 'puppet/indirector/memory' describe Puppet::Network::HTTP::Handler do + before :each do + class Puppet::TestModel + extend Puppet::Indirector + indirects :test_model + attr_accessor :name, :data + def initialize(name = "name", data = '') + @name = name + @data = data + end + + def self.from_pson(pson) + new(pson["name"], pson["data"]) + end + + def to_pson + { + "name" => @name, + "data" => @data + }.to_pson + end + + def ==(other) + other.is_a? Puppet::TestModel and other.name == name and other.data == data + end + end + + # The subclass must not be all caps even though the superclass is + class Puppet::TestModel::Memory < Puppet::Indirector::Memory + end + + Puppet::TestModel.indirection.terminus_class = :memory + end + + after :each do + Puppet::TestModel.indirection.delete + # Remove the class, unlinking it from the rest of the system. + Puppet.send(:remove_const, :TestModel) + end + + let(:terminus_class) { Puppet::TestModel::Memory } + let(:terminus) { Puppet::TestModel.indirection.terminus(:memory) } + let(:indirection) { Puppet::TestModel.indirection } + let(:model) { Puppet::TestModel } + + def a_request + { + :accept_header => "pson", + :content_type_header => "text/yaml", + :http_method => "HEAD", + :path => "/production/#{indirection.name}/unknown", + :params => {}, + :client_cert => nil, + :headers => {}, + :body => nil + } + end + + def a_request_that_heads(data, request = {}) + { + :accept_header => request[:accept_header], + :content_type_header => "text/yaml", + :http_method => "HEAD", + :path => "/production/#{indirection.name}/#{data.name}", + :params => {}, + :client_cert => nil, + :body => nil + } + end + + def a_request_that_submits(data, request = {}) + { + :accept_header => request[:accept_header], + :content_type_header => "text/yaml", + :http_method => "PUT", + :path => "/production/#{indirection.name}/#{data.name}", + :params => {}, + :client_cert => nil, + :body => data.render("text/yaml") + } + end + + def a_request_that_destroys(data, request = {}) + { + :accept_header => request[:accept_header], + :content_type_header => "text/yaml", + :http_method => "DELETE", + :path => "/production/#{indirection.name}/#{data.name}", + :params => {}, + :client_cert => nil, + :body => '' + } + end + + def a_request_that_finds(data, request = {}) + { + :accept_header => request[:accept_header], + :content_type_header => "text/yaml", + :http_method => "GET", + :path => "/production/#{indirection.name}/#{data.name}", + :params => {}, + :client_cert => nil, + :body => '' + } + end + + def a_request_that_searches(key, request = {}) + { + :accept_header => request[:accept_header], + :content_type_header => "text/yaml", + :http_method => "GET", + :path => "/production/#{indirection.name}s/#{key}", + :params => {}, + :client_cert => nil, + :body => '' + } + end + let(:handler) { TestingHandler.new } it "should include the v1 REST API" do @@ -28,37 +146,15 @@ describe Puppet::Network::HTTP::Handler do end describe "when processing a request" do - let(:request) do - { - :accept_header => "format_one,format_two", - :content_type_header => "text/yaml", - :http_method => "GET", - :path => "/my_handler/my_result", - :params => {}, - :client_cert => nil - } - end - let(:response) { mock('http response') } before do - @model_class = stub('indirected model class') - @indirection = stub('indirection') - @model_class.stubs(:indirection).returns(@indirection) - - @result = stub 'result', :render => "mytext" - - request[:headers] = { - "Content-Type" => request[:content_type_header], - "Accept" => request[:accept_header] - } - handler.stubs(:check_authorization) handler.stubs(:warn_if_near_expiration) - handler.stubs(:headers).returns(request[:headers]) end it "should check the client certificate for upcoming expiration" do + request = a_request cert = mock 'cert' handler.stubs(:uri2indirection).returns(["facts", :mymethod, "key", {:node => "name"}]) handler.expects(:client_cert).returns(cert).with(request) @@ -68,6 +164,7 @@ describe Puppet::Network::HTTP::Handler do end it "should setup a profiler when the puppet-profiling header exists" do + request = a_request request[:headers][Puppet::Network::HTTP::HEADER_ENABLE_PROFILING.downcase] = "true" handler.process(request, response) @@ -76,6 +173,7 @@ describe Puppet::Network::HTTP::Handler do end it "should not setup profiler when the profile parameter is missing" do + request = a_request request[:params] = { } handler.process(request, response) @@ -84,6 +182,7 @@ describe Puppet::Network::HTTP::Handler do end it "should create an indirection request from the path, parameters, and http method" do + request = a_request request[:path] = "mypath" request[:http_method] = "mymethod" request[:params] = { :params => "mine" } @@ -96,6 +195,7 @@ describe Puppet::Network::HTTP::Handler do end it "should call the 'do' method and delegate authorization to the authorization layer" do + request = a_request handler.expects(:uri2indirection).returns(["facts", :mymethod, "key", {:node => "name"}]) handler.expects(:do_mymethod).with("facts", "key", {:node => "name"}, request, response) @@ -106,6 +206,7 @@ describe Puppet::Network::HTTP::Handler do end it "should return 403 if the request is not authorized" do + request = a_request handler.expects(:uri2indirection).returns(["facts", :mymethod, "key", {:node => "name"}]) handler.expects(:do_mymethod).never @@ -118,6 +219,7 @@ describe Puppet::Network::HTTP::Handler do end it "should serialize a controller exception when an exception is thrown while finding the model instance" do + request = a_request handler.expects(:uri2indirection).returns(["facts", :find, "key", {:node => "name"}]) handler.expects(:do_find).raises(ArgumentError, "The exception") @@ -127,15 +229,33 @@ describe Puppet::Network::HTTP::Handler do it "should set the format to text/plain when serializing an exception" do handler.expects(:set_content_type).with(response, "text/plain") + + handler.do_exception(response, "A test", 404) + end + + it "sends an exception string with the given status" do + handler.expects(:set_response).with(response, "A test", 404) + handler.do_exception(response, "A test", 404) end + it "sends an exception error with the exception's status" do + data = Puppet::TestModel.new("not_found", "not found") + request = a_request_that_finds(data, :accept_header => "pson") + + error = Puppet::Network::HTTP::Handler::HTTPNotFoundError.new("Could not find test_model not_found") + handler.expects(:set_response).with(response, error.to_s, error.status) + + handler.process(request, response) + end + it "should raise an error if the request is formatted in an unknown format" do handler.stubs(:content_type_header).returns "unknown format" lambda { handler.request_format(request) }.should raise_error end it "should still find the correct format if content type contains charset information" do + request = a_request handler.stubs(:content_type_header).returns "text/plain; charset=UTF-8" handler.request_format(request).should == "s" end @@ -157,310 +277,204 @@ describe Puppet::Network::HTTP::Handler do end describe "when finding a model instance" do - before do - @indirection.stubs(:find).returns @result - Puppet::Indirector::Indirection.expects(:instance).with(:my_handler).returns( stub "indirection", :model => @model_class ) + it "uses the first supported format for the response" do + data = Puppet::TestModel.new("my data", "some data") + indirection.save(data, "my data") + request = a_request_that_finds(data, :accept_header => "unknown, pson, yaml") - @format = stub 'format', :suitable? => true, :mime => "text/format", :name => "format" - Puppet::Network::FormatHandler.stubs(:format).returns @format + handler.expects(:set_response).with(response, data.render(:pson)) + handler.expects(:set_content_type).with(response, Puppet::Network::FormatHandler.format(:pson)) - @oneformat = stub 'one', :suitable? => true, :mime => "text/one", :name => "one" - Puppet::Network::FormatHandler.stubs(:format).with("one").returns @oneformat + handler.do_find(indirection.name, "my data", {}, request, response) end - it "should use the indirection request to find the model class" do - handler.do_find("my_handler", "my_result", {}, request, response) - end - - it "should use the escaped request key" do - @indirection.expects(:find).with("my_result", anything).returns @result - handler.do_find("my_handler", "my_result", {}, request, response) - end - - it "should use a common method for determining the request parameters" do - @indirection.expects(:find).with(anything, has_entries(:foo => :baz, :bar => :xyzzy)).returns @result - - handler.do_find("my_handler", "my_result", {:foo => :baz, :bar => :xyzzy}, request, response) - end - - it "should set the content type to the first format specified in the accept header" do - handler.expects(:accept_header).with(request).returns "one,two" - handler.expects(:set_content_type).with(response, @oneformat) - handler.do_find("my_handler", "my_result", {}, request, response) - end - - it "should fail if no accept header is provided" do - handler.expects(:accept_header).with(request).returns nil - lambda { handler.do_find("my_handler", "my_result", {}, request, response) }.should raise_error(ArgumentError) - end + it "responds with a 406 error when no accept header is provided" do + data = Puppet::TestModel.new("my data", "some data") + indirection.save(data, "my data") + request = a_request_that_finds(data, :accept_header => nil) - it "should fail if the accept header does not contain a valid format" do - handler.expects(:accept_header).with(request).returns "" - lambda { handler.do_find("my_handler", "my_result", {}, request, response) }.should raise_error(RuntimeError) + expect do + handler.do_find(indirection.name, "my data", {}, request, response) + end.to raise_error(Puppet::Network::HTTP::Handler::HTTPNotAcceptableError) end - it "should not use an unsuitable format" do - handler.expects(:accept_header).with(request).returns "foo,bar" - foo = mock 'foo', :suitable? => false - bar = mock 'bar', :suitable? => true - Puppet::Network::FormatHandler.expects(:format).with("foo").returns foo - Puppet::Network::FormatHandler.expects(:format).with("bar").returns bar - - handler.expects(:set_content_type).with(response, bar) # the suitable one + it "raises an error when no accepted formats are known" do + data = Puppet::TestModel.new("my data", "some data") + indirection.save(data, "my data") + request = a_request_that_finds(data, :accept_header => "unknown, also/unknown") - handler.do_find("my_handler", "my_result", {}, request, response) - end - - it "should render the result using the first format specified in the accept header" do - - handler.expects(:accept_header).with(request).returns "one,two" - @result.expects(:render).with(@oneformat) - - handler.do_find("my_handler", "my_result", {}, request, response) + expect do + handler.do_find(indirection.name, "my data", {}, request, response) + end.to raise_error(Puppet::Network::HTTP::Handler::HTTPNotAcceptableError) end it "should pass the result through without rendering it if the result is a string" do - @indirection.stubs(:find).returns "foo" - handler.expects(:set_response).with(response, "foo") - handler.do_find("my_handler", "my_result", {}, request, response) - end - - it "should use the default status when a model find call succeeds" do - handler.expects(:set_response).with(anything, anything, nil) - handler.do_find("my_handler", "my_result", {}, request, response) - end + data = Puppet::TestModel.new("my data", "some data") + data_string = "my data string" + request = a_request_that_finds(data, :accept_header => "pson") + indirection.expects(:find).returns(data_string) - it "should return a serialized object when a model find call succeeds" do - @model_instance = stub('model instance') - @model_instance.expects(:render).returns "my_rendered_object" + handler.expects(:set_response).with(response, data_string) + handler.expects(:set_content_type).with(response, Puppet::Network::FormatHandler.format(:pson)) - handler.expects(:set_response).with(anything, "my_rendered_object", anything) - @indirection.stubs(:find).returns(@model_instance) - handler.do_find("my_handler", "my_result", {}, request, response) + handler.do_find(indirection.name, "my data", {}, request, response) end it "should return a 404 when no model instance can be found" do - @model_class.stubs(:name).returns "my name" - handler.expects(:set_response).with(anything, anything, 404) - @indirection.stubs(:find).returns(nil) - handler.do_find("my_handler", "my_result", {}, request, response) - end - - it "should write a log message when no model instance can be found" do - @model_class.stubs(:name).returns "my name" - @indirection.stubs(:find).returns(nil) - - Puppet.expects(:info).with("Could not find my_handler for 'my_result'") + data = Puppet::TestModel.new("my data", "some data") + request = a_request_that_finds(data, :accept_header => "unknown, pson, yaml") - handler.do_find("my_handler", "my_result", {}, request, response) - end - - - it "should serialize the result in with the appropriate format" do - @model_instance = stub('model instance') - - handler.expects(:format_to_use).returns(@oneformat) - @model_instance.expects(:render).with(@oneformat).returns "my_rendered_object" - @indirection.stubs(:find).returns(@model_instance) - handler.do_find("my_handler", "my_result", {}, request, response) + expect do + handler.do_find(indirection.name, "my data", {}, request, response) + end.to raise_error(Puppet::Network::HTTP::Handler::HTTPNotFoundError) end end describe "when performing head operation" do - before do - handler.stubs(:model).with("my_handler").returns(stub 'model', :indirection => @model_class) - request[:http_method] = "HEAD" - request[:path] = "/production/my_handler/my_result" - request[:params] = {} - - @model_class.stubs(:head).returns true - end - - it "should use the escaped request key" do - @model_class.expects(:head).with("my_result", anything).returns true - handler.process(request, response) - end - it "should not generate a response when a model head call succeeds" do + data = Puppet::TestModel.new("my data", "some data") + indirection.save(data, "my data") + request = a_request_that_heads(data) + handler.expects(:set_response).never + handler.process(request, response) end it "should return a 404 when the model head call returns false" do - handler.expects(:set_response).with(anything, anything, 404) - @model_class.stubs(:head).returns(false) + data = Puppet::TestModel.new("my data", "data not there") + request = a_request_that_heads(data) + + handler.expects(:set_response).with(response, "Not Found: Could not find test_model my data", 404) + handler.process(request, response) end end describe "when searching for model instances" do - before do - Puppet::Indirector::Indirection.expects(:instance).with(:my_handler).returns( stub "indirection", :model => @model_class ) + it "uses the first supported format for the response" do + data = Puppet::TestModel.new("my data", "some data") + indirection.save(data, "my data") + request = a_request_that_searches("my", :accept_header => "unknown, pson, yaml") - result1 = mock 'result1' - result2 = mock 'results' + handler.expects(:set_response).with(response, Puppet::TestModel.render_multiple(:pson, [data])) + handler.expects(:set_content_type).with(response, Puppet::Network::FormatHandler.format(:pson)) - @result = [result1, result2] - @model_class.stubs(:render_multiple).returns "my rendered instances" - @indirection.stubs(:search).returns(@result) - - @format = stub 'format', :suitable? => true, :mime => "text/format", :name => "format" - Puppet::Network::FormatHandler.stubs(:format).returns @format - - @oneformat = stub 'one', :suitable? => true, :mime => "text/one", :name => "one" - Puppet::Network::FormatHandler.stubs(:format).with("one").returns @oneformat - end - - it "should use the indirection request to find the model" do - handler.do_search("my_handler", "my_result", {}, request, response) - end - - it "should use a common method for determining the request parameters" do - @indirection.expects(:search).with(anything, has_entries(:foo => :baz, :bar => :xyzzy)).returns @result - handler.do_search("my_handler", "my_result", {:foo => :baz, :bar => :xyzzy}, request, response) - end - - it "should use the default status when a model search call succeeds" do - @indirection.stubs(:search).returns(@result) - handler.do_search("my_handler", "my_result", {}, request, response) - end - - it "should set the content type to the first format returned by the accept header" do - handler.expects(:accept_header).with(request).returns "one,two" - handler.expects(:set_content_type).with(response, @oneformat) - - handler.do_search("my_handler", "my_result", {}, request, response) - end - - it "should return a list of serialized objects when a model search call succeeds" do - handler.expects(:accept_header).with(request).returns "one,two" - - @indirection.stubs(:search).returns(@result) - - @model_class.expects(:render_multiple).with(@oneformat, @result).returns "my rendered instances" - - handler.expects(:set_response).with(anything, "my rendered instances") - handler.do_search("my_handler", "my_result", {}, request, response) + handler.do_search(indirection.name, "my", {}, request, response) end it "should return [] when searching returns an empty array" do - handler.expects(:accept_header).with(request).returns "one,two" - @indirection.stubs(:search).returns([]) - @model_class.expects(:render_multiple).with(@oneformat, []).returns "[]" + request = a_request_that_searches("nothing", :accept_header => "unknown, pson, yaml") + handler.expects(:set_response).with(response, Puppet::TestModel.render_multiple(:pson, [])) + handler.expects(:set_content_type).with(response, Puppet::Network::FormatHandler.format(:pson)) - handler.expects(:set_response).with(anything, "[]") - handler.do_search("my_handler", "my_result", {}, request, response) + handler.do_search(indirection.name, "nothing", {}, request, response) end it "should return a 404 when searching returns nil" do - @model_class.stubs(:name).returns "my name" - handler.expects(:set_response).with(anything, anything, 404) - @indirection.stubs(:search).returns(nil) - handler.do_search("my_handler", "my_result", {}, request, response) + request = a_request_that_searches("nothing", :accept_header => "unknown, pson, yaml") + indirection.expects(:search).returns(nil) + + expect do + handler.do_search(indirection.name, "nothing", {}, request, response) + end.to raise_error(Puppet::Network::HTTP::Handler::HTTPNotFoundError) end end describe "when destroying a model instance" do - before do - Puppet::Indirector::Indirection.expects(:instance).with(:my_handler).returns( stub "indirection", :model => @model_class ) + it "destroys the data indicated in the request" do + data = Puppet::TestModel.new("my data", "some data") + indirection.save(data, "my data") + request = a_request_that_destroys(data) - @result = stub 'result', :render => "the result" - @indirection.stubs(:destroy).returns @result - end + handler.do_destroy(indirection.name, "my data", {}, request, response) - it "should use the indirection request to find the model" do - handler.do_destroy("my_handler", "my_result", {}, request, response) + Puppet::TestModel.indirection.find("my data").should be_nil end - it "should use the escaped request key to destroy the instance in the model" do - @indirection.expects(:destroy).with("foo bar", anything) - handler.do_destroy("my_handler", "foo bar", {}, request, response) - end + it "responds with yaml when no Accept header is given" do + data = Puppet::TestModel.new("my data", "some data") + indirection.save(data, "my data") + request = a_request_that_destroys(data, :accept_header => nil) + + handler.expects(:set_response).with(response, data.render(:yaml)) + handler.expects(:set_content_type).with(response, Puppet::Network::FormatHandler.format(:yaml)) - it "should use a common method for determining the request parameters" do - @indirection.expects(:destroy).with(anything, has_entries(:foo => :baz, :bar => :xyzzy)) - handler.do_destroy("my_handler", "my_result", {:foo => :baz, :bar => :xyzzy}, request, response) + handler.do_destroy(indirection.name, "my data", {}, request, response) end - it "should use the default status code a model destroy call succeeds" do - handler.expects(:set_response).with(anything, anything, nil) - handler.do_destroy("my_handler", "my_result", {}, request, response) + it "uses the first supported format for the response" do + data = Puppet::TestModel.new("my data", "some data") + indirection.save(data, "my data") + request = a_request_that_destroys(data, :accept_header => "unknown, pson, yaml") + + handler.expects(:set_response).with(response, data.render(:pson)) + handler.expects(:set_content_type).with(response, Puppet::Network::FormatHandler.format(:pson)) + + handler.do_destroy(indirection.name, "my data", {}, request, response) end - it "should return a yaml-encoded result when a model destroy call succeeds" do - @result = stub 'result', :to_yaml => "the result" - @indirection.expects(:destroy).returns(@result) + it "raises an error and does not destory when no accepted formats are known" do + data = Puppet::TestModel.new("my data", "some data") + indirection.save(data, "my data") + request = a_request_that_submits(data, :accept_header => "unknown, also/unknown") - handler.expects(:set_response).with(anything, "the result", anything) + expect do + handler.do_destroy(indirection.name, "my data", {}, request, response) + end.to raise_error(Puppet::Network::HTTP::Handler::HTTPNotAcceptableError) - handler.do_destroy("my_handler", "my_result", {}, request, response) + Puppet::TestModel.indirection.find("my data").should_not be_nil end end describe "when saving a model instance" do - before do - Puppet::Indirector::Indirection.stubs(:instance).with(:my_handler).returns( stub "indirection", :model => @model_class ) - handler.stubs(:body).returns('my stuff') - handler.stubs(:content_type_header).returns("text/yaml") - - @result = stub 'result', :render => "the result" - - @model_instance = stub('indirected model instance') - @model_class.stubs(:convert_from).returns(@model_instance) - @indirection.stubs(:save) + it "should fail to save model if data is not specified" do + data = Puppet::TestModel.new("my data", "some data") + request = a_request_that_submits(data) + request[:body] = '' - @format = stub 'format', :suitable? => true, :name => "format", :mime => "text/format" - Puppet::Network::FormatHandler.stubs(:format).returns @format - @yamlformat = stub 'yaml', :suitable? => true, :name => "yaml", :mime => "text/yaml" - Puppet::Network::FormatHandler.stubs(:format).with("yaml").returns @yamlformat + expect { handler.do_save("my_handler", "my_result", {}, request, response) }.to raise_error(ArgumentError) end - it "should use the indirection request to find the model" do - handler.do_save("my_handler", "my_result", {}, request, response) - end + it "saves the data sent in the request" do + data = Puppet::TestModel.new("my data", "some data") + request = a_request_that_submits(data) - it "should use the 'body' hook to retrieve the body of the request" do - handler.expects(:body).returns "my body" - @model_class.expects(:convert_from).with(anything, "my body").returns @model_instance + handler.do_save(indirection.name, "my data", {}, request, response) - handler.do_save("my_handler", "my_result", {}, request, response) + Puppet::TestModel.indirection.find("my data").should == data end - it "should fail to save model if data is not specified" do - handler.stubs(:body).returns('') + it "responds with yaml when no Accept header is given" do + data = Puppet::TestModel.new("my data", "some data") + request = a_request_that_submits(data, :accept_header => nil) - lambda { handler.do_save("my_handler", "my_result", {}, request, response) }.should raise_error(ArgumentError) - end + handler.expects(:set_response).with(response, data.render(:yaml)) + handler.expects(:set_content_type).with(response, Puppet::Network::FormatHandler.format(:yaml)) - it "should use a common method for determining the request parameters" do - @indirection.expects(:save).with(@model_instance, 'key').once - handler.do_save("my_handler", "key", {}, request, response) + handler.do_save(indirection.name, "my data", {}, request, response) end - it "should use the default status when a model save call succeeds" do - handler.expects(:set_response).with(anything, anything, nil) - handler.do_save("my_handler", "my_result", {}, request, response) - end + it "uses the first supported format for the response" do + data = Puppet::TestModel.new("my data", "some data") + request = a_request_that_submits(data, :accept_header => "unknown, pson, yaml") - it "should return the yaml-serialized result when a model save call succeeds" do - @indirection.stubs(:save).returns(@model_instance) - @model_instance.expects(:to_yaml).returns('foo') - handler.do_save("my_handler", "my_result", {}, request, response) - end + handler.expects(:set_response).with(response, data.render(:pson)) + handler.expects(:set_content_type).with(response, Puppet::Network::FormatHandler.format(:pson)) - it "should set the content to yaml" do - handler.expects(:set_content_type).with(response, @yamlformat) - handler.do_save("my_handler", "my_result", {}, request, response) + handler.do_save(indirection.name, "my data", {}, request, response) end - it "should use the content-type header to know the body format" do - handler.expects(:content_type_header).returns("text/format") - Puppet::Network::FormatHandler.stubs(:mime).with("text/format").returns @format + it "raises an error and does not save when no accepted formats are known" do + data = Puppet::TestModel.new("my data", "some data") + request = a_request_that_submits(data, :accept_header => "unknown, also/unknown") - @model_class.expects(:convert_from).with("format", anything).returns @model_instance + expect do + handler.do_save(indirection.name, "my data", {}, request, response) + end.to raise_error(Puppet::Network::HTTP::Handler::HTTPNotAcceptableError) - handler.do_save("my_handler", "my_result", {}, request, response) + Puppet::TestModel.indirection.find("my data").should be_nil end end end @@ -519,5 +533,13 @@ describe Puppet::Network::HTTP::Handler do def client_cert(request) request[:client_cert] end + + def body(request) + request[:body] + end + + def headers(request) + request[:headers] || {} + end end end diff --git a/spec/unit/network/http/rack/rest_spec.rb b/spec/unit/network/http/rack/rest_spec.rb index 08e88be40..729c6c21a 100755 --- a/spec/unit/network/http/rack/rest_spec.rb +++ b/spec/unit/network/http/rack/rest_spec.rb @@ -34,7 +34,7 @@ describe "Puppet::Network::HTTP::RackREST", :if => Puppet.features.rack? do describe "#headers" do it "should return the headers (parsed from env with prefix 'HTTP_')" do req = mk_req('/', {'HTTP_Accept' => 'myaccept', - 'HTTP_X-Custom-Header' => 'mycustom', + 'HTTP_X_Custom_Header' => 'mycustom', 'NOT_HTTP_foo' => 'not an http header'}) @handler.headers(req).should == {"accept" => 'myaccept', "x-custom-header" => 'mycustom'} diff --git a/spec/unit/network/http/webrick/rest_spec.rb b/spec/unit/network/http/webrick/rest_spec.rb index 5108ae880..852fe1547 100755 --- a/spec/unit/network/http/webrick/rest_spec.rb +++ b/spec/unit/network/http/webrick/rest_spec.rb @@ -125,49 +125,100 @@ describe Puppet::Network::HTTP::WEBrickREST do end describe "and determining the request parameters" do - it "should include the HTTP request parameters, with the keys as symbols" do - @request.stubs(:query).returns("foo" => "baz", "bar" => "xyzzy") + def query_of(options) + request = Puppet::Indirector::Request.new(:myind, :find, "my key", nil, options) + WEBrick::HTTPUtils.parse_query(request.query_string.sub(/^\?/, '')) + end + + def a_request_querying(query_data) + @request.expects(:query).returns(query_of(query_data)) + @request + end + + it "has no parameters when there is no query string" do + only_server_side_information = [:authenticated, :ip, :node] + @request.stubs(:query).returns(nil) + result = @handler.params(@request) + + result.keys.sort.should == only_server_side_information + end + + it "should include the HTTP request parameters, with the keys as symbols" do + request = a_request_querying("foo" => "baz", "bar" => "xyzzy") + result = @handler.params(request) + result[:foo].should == "baz" result[:bar].should == "xyzzy" end - it "should CGI-decode the HTTP parameters" do - encoding = CGI.escape("foo bar") - @request.expects(:query).returns('foo' => encoding) - result = @handler.params(@request) - result[:foo].should == "foo bar" + it "should handle parameters with no value" do + request = a_request_querying('foo' => "") + + result = @handler.params(request) + + result[:foo].should == "" end it "should convert the string 'true' to the boolean" do - @request.expects(:query).returns('foo' => "true") - result = @handler.params(@request) - result[:foo].should be_true + request = a_request_querying('foo' => "true") + + result = @handler.params(request) + + result[:foo].should == true end it "should convert the string 'false' to the boolean" do - @request.expects(:query).returns('foo' => "false") - result = @handler.params(@request) - result[:foo].should be_false + request = a_request_querying('foo' => "false") + + result = @handler.params(request) + + result[:foo].should == false end - it "should YAML-load and CGI-decode values that are YAML-encoded" do - escaping = CGI.escape(YAML.dump(%w{one two})) - @request.expects(:query).returns('foo' => escaping) - result = @handler.params(@request) + it "should reconstruct arrays" do + request = a_request_querying('foo' => ["a", "b", "c"]) + + result = @handler.params(request) + + result[:foo].should == ["a", "b", "c"] + end + + it "should convert values inside arrays into primitive types" do + request = a_request_querying('foo' => ["true", "false", "1", "1.2"]) + + result = @handler.params(request) + + result[:foo].should == [true, false, 1, 1.2] + end + + it "should YAML-load values that are YAML-encoded" do + request = a_request_querying('foo' => YAML.dump(%w{one two})) + + result = @handler.params(request) + + result[:foo].should == %w{one two} + end + + it "should YAML-load that are YAML-encoded" do + request = a_request_querying('foo' => YAML.dump(%w{one two})) + + result = @handler.params(request) + result[:foo].should == %w{one two} end it "should not allow clients to set the node via the request parameters" do - @request.stubs(:query).returns("node" => "foo") + request = a_request_querying("node" => "foo") @handler.stubs(:resolve_node) - @handler.params(@request)[:node].should be_nil + @handler.params(request)[:node].should be_nil end it "should not allow clients to set the IP via the request parameters" do - @request.stubs(:query).returns("ip" => "foo") - @handler.params(@request)[:ip].should_not == "foo" + request = a_request_querying("ip" => "foo") + + @handler.params(request)[:ip].should_not == "foo" end it "should pass the client's ip address to model find" do @@ -212,6 +263,6 @@ describe Puppet::Network::HTTP::WEBrickREST do @handler.params(@request)[:node].should == :resolved_node end - end + end end end diff --git a/spec/unit/network/http_pool_spec.rb b/spec/unit/network/http_pool_spec.rb index eab4f88f1..c671e88f1 100755 --- a/spec/unit/network/http_pool_spec.rb +++ b/spec/unit/network/http_pool_spec.rb @@ -22,6 +22,46 @@ describe Puppet::Network::HttpPool do Puppet::Network::HttpPool.http_instance("me", 54321, true).should be_use_ssl end + + describe 'peer verification' do + def setup_standard_ssl_configuration + ca_cert_file = File.expand_path('/path/to/ssl/certs/ca_cert.pem') + FileTest.stubs(:exist?).with(ca_cert_file).returns(true) + + ssl_configuration = stub('ssl_configuration', :ca_auth_file => ca_cert_file) + Puppet::Network::HTTP::Connection.any_instance.stubs(:ssl_configuration).returns(ssl_configuration) + end + + def setup_standard_hostcert + host_cert_file = File.expand_path('/path/to/ssl/certs/host_cert.pem') + FileTest.stubs(:exist?).with(host_cert_file).returns(true) + + Puppet[:hostcert] = host_cert_file + end + + def setup_standard_ssl_host + cert = stub('cert', :content => 'real_cert') + key = stub('key', :content => 'real_key') + host = stub('host', :certificate => cert, :key => key, :ssl_store => stub('store')) + + Puppet::Network::HTTP::Connection.any_instance.stubs(:ssl_host).returns(host) + end + + before do + setup_standard_ssl_configuration + setup_standard_hostcert + setup_standard_ssl_host + end + + it 'can enable peer verification' do + Puppet::Network::HttpPool.http_instance("me", 54321, true, true).send(:connection).verify_mode.should == OpenSSL::SSL::VERIFY_PEER + end + + it 'can disable peer verification' do + Puppet::Network::HttpPool.http_instance("me", 54321, true, false).send(:connection).verify_mode.should == OpenSSL::SSL::VERIFY_NONE + end + end + it "should not cache http instances" do Puppet::Network::HttpPool.http_instance("me", 54321). should_not equal Puppet::Network::HttpPool.http_instance("me", 54321) diff --git a/spec/unit/network/server_spec.rb b/spec/unit/network/server_spec.rb index b4ac12dae..941e94015 100755 --- a/spec/unit/network/server_spec.rb +++ b/spec/unit/network/server_spec.rb @@ -5,30 +5,19 @@ require 'puppet/network/server' describe Puppet::Network::Server do let(:port) { 8140 } let(:address) { '0.0.0.0' } - let(:server) { - server = Puppet::Network::Server.new(address, port) - server.stubs(:close_streams) - server - } + let(:server) { Puppet::Network::Server.new(address, port) } before do @mock_http_server = mock('http server') Puppet.settings.stubs(:use) - Puppet.run_mode.stubs(:name).returns :master Puppet::Network::HTTP::WEBrick.stubs(:new).returns(@mock_http_server) end describe "when initializing" do before do - Puppet::Indirector::Indirection.stubs(:model).returns mock('indirection') Puppet[:masterport] = '' end - it "should allow registering REST handlers" do - server = Puppet::Network::Server.new(address, port, [:foo, :bar, :baz]) - expect { server.unregister(:foo, :bar, :baz) }.to_not raise_error - end - it "should not be listening after initialization" do Puppet::Network::Server.new(address, port).should_not be_listening end @@ -45,173 +34,7 @@ describe Puppet::Network::Server do end end - describe "when being started" do - before do - server.stubs(:listen) - server.stubs(:create_pidfile) - end - - it "should listen" do - server.expects(:listen) - server.start - end - - it "should create its PID file" do - server.expects(:create_pidfile) - server.start - end - end - - describe "when being stopped" do - before do - server.stubs(:unlisten) - server.stubs(:remove_pidfile) - end - - it "should unlisten" do - server.expects(:unlisten) - server.stop - end - - it "should remove its PID file" do - server.expects(:remove_pidfile) - server.stop - end - end - - describe "when creating its pidfile" do - it "should use an exclusive mutex" do - Puppet.run_mode.expects(:name).returns "me" - Puppet::Util.expects(:synchronize_on).with("me", Sync::EX) - server.create_pidfile - end - - it "should lock the pidfile using the Pidlock class" do - pidfile = mock 'pidfile' - - Puppet.run_mode.expects(:name).returns "eh" - Puppet[:pidfile] = File.expand_path("/my/file") - - Puppet::Util::Pidlock.expects(:new).with(Puppet[:pidfile]).returns pidfile - - pidfile.expects(:lock).returns true - server.create_pidfile - end - - it "should fail if it cannot lock" do - pidfile = mock 'pidfile' - - Puppet[:pidfile] = File.expand_path("/my/file") - - Puppet::Util::Pidlock.expects(:new).with(Puppet[:pidfile]).returns pidfile - - pidfile.expects(:lock).returns false - - expect { server.create_pidfile }.to raise_error /Could not create PID/ - end - end - - describe "when removing its pidfile" do - it "should use an exclusive mutex" do - Puppet.run_mode.expects(:name).returns "me" - Puppet::Util.expects(:synchronize_on).with("me",Sync::EX) - server.remove_pidfile - end - - it "should do nothing if the pidfile is not present" do - pidfile = mock 'pidfile', :unlock => false - Puppet[:pidfile] = "/my/file" - Puppet::Util::Pidlock.expects(:new).with(Puppet[:pidfile]).returns pidfile - - server.remove_pidfile - end - - it "should unlock the pidfile using the Pidlock class" do - pidfile = mock 'pidfile', :unlock => true - Puppet[:pidfile] = "/my/file" - Puppet::Util::Pidlock.expects(:new).with(Puppet[:pidfile]).returns pidfile - - server.remove_pidfile - end - end - - describe "when managing indirection registrations" do - before do - Puppet::Indirector::Indirection.stubs(:model).returns mock('indirection') - end - - it "should allow registering an indirection for client access by specifying its indirection name" do - expect { server.register(:foo) }.to_not raise_error - end - - it "should require that the indirection be valid" do - Puppet::Indirector::Indirection.expects(:model).with(:foo).returns nil - expect { server.register(:foo) }.to raise_error(ArgumentError) - end - - it "should require at least one indirection name when registering indirections for client access" do - expect { server.register }.to raise_error(ArgumentError) - end - - it "should allow for numerous indirections to be registered at once for client access" do - expect { server.register(:foo, :bar, :baz) }.to_not raise_error - end - - it "should allow the use of indirection names to specify which indirections are to be no longer accessible to clients" do - server.register(:foo) - expect { server.unregister(:foo) }.to_not raise_error - end - - it "should leave other indirections accessible to clients when turning off indirections" do - server.register(:foo, :bar) - server.unregister(:foo) - expect { server.unregister(:bar)}.to_not raise_error - end - - it "should allow specifying numerous indirections which are to be no longer accessible to clients" do - server.register(:foo, :bar) - expect { server.unregister(:foo, :bar) }.to_not raise_error - end - - it "should not turn off any indirections if given unknown indirection names to turn off" do - server.register(:foo, :bar) - expect { server.unregister(:foo, :bar, :baz) }.to raise_error(ArgumentError) - expect { server.unregister(:foo, :bar) }.to_not raise_error - end - - it "should not allow turning off unknown indirection names" do - server.register(:foo, :bar) - expect { server.unregister(:baz) }.to raise_error(ArgumentError) - end - - it "should disable client access immediately when turning off indirections" do - server.register(:foo, :bar) - server.unregister(:foo) - expect { server.unregister(:foo) }.to raise_error(ArgumentError) - end - - it "should allow turning off all indirections at once" do - server.register(:foo, :bar) - server.unregister - [:foo, :bar, :baz].each do |indirection| - expect { server.unregister(indirection) }.to raise_error(ArgumentError) - end - end - end - - it "should allow for multiple configurations, each handling different indirections" do - Puppet::Indirector::Indirection.stubs(:model).returns mock('indirection') - - server2 = Puppet::Network::Server.new(address, port) - server.register(:foo, :bar) - server2.register(:foo, :xyzzy) - server.unregister(:foo, :bar) - server2.unregister(:foo, :xyzzy) - expect { server.unregister(:xyzzy) }.to raise_error(ArgumentError) - expect { server2.unregister(:bar) }.to raise_error(ArgumentError) - end - - describe "when listening is off" do + describe "when not yet started" do before do @mock_http_server.stubs(:listen) end @@ -220,65 +43,53 @@ describe Puppet::Network::Server do server.should_not be_listening end - it "should not allow listening to be turned off" do - expect { server.unlisten }.to raise_error(RuntimeError) + it "should not allow server to be stopped" do + expect { server.stop }.to raise_error(RuntimeError) end - it "should allow listening to be turned on" do - expect { server.listen }.to_not raise_error + it "should allow server to be started" do + expect { server.start }.to_not raise_error end - end - describe "when listening is on" do + describe "when server is on" do before do @mock_http_server.stubs(:listen) @mock_http_server.stubs(:unlisten) - server.listen + server.start end it "should indicate that it is listening" do server.should be_listening end - it "should not allow listening to be turned on" do - expect { server.listen }.to raise_error(RuntimeError) + it "should not allow server to be started again" do + expect { server.start }.to raise_error(RuntimeError) end - it "should allow listening to be turned off" do - expect { server.unlisten }.to_not raise_error + it "should allow server to be stopped" do + expect { server.stop }.to_not raise_error end end - describe "when listening is being turned on" do - before do - Puppet::Indirector::Indirection.stubs(:model).returns mock('indirection') - end - + describe "when server is being started" do it "should cause the HTTP server to listen" do - server = Puppet::Network::Server.new(address, port, [:node]) + server = Puppet::Network::Server.new(address, port) @mock_http_server.expects(:listen).with(address, port) - server.listen + server.start end end - describe "when listening is being turned off" do + describe "when server is being stopped" do before do @mock_http_server.stubs(:listen) server.stubs(:http_server).returns(@mock_http_server) - server.listen + server.start end it "should cause the HTTP server to stop listening" do @mock_http_server.expects(:unlisten) - server.unlisten - end - - it "should not allow for indirections to be turned off" do - Puppet::Indirector::Indirection.stubs(:model).returns mock('indirection') - - server.register(:foo) - expect { server.unregister(:foo) }.to raise_error(RuntimeError) + server.stop end end end diff --git a/spec/unit/node/facts_spec.rb b/spec/unit/node/facts_spec.rb index abc6b9d85..9ba6baa11 100755 --- a/spec/unit/node/facts_spec.rb +++ b/spec/unit/node/facts_spec.rb @@ -14,25 +14,76 @@ describe Puppet::Node::Facts, "when indirecting" do @facts.values["one"].should == "1" end - it "should add the node's certificate name as the 'clientcert' fact when adding local facts" do - @facts.add_local_facts - @facts.values["clientcert"].should == Puppet.settings[:certname] - end + describe "adding local facts" do + it "should add the node's certificate name as the 'clientcert' fact" do + @facts.add_local_facts + @facts.values["clientcert"].should == Puppet.settings[:certname] + end - it "should add the Puppet version as a 'clientversion' fact when adding local facts" do - @facts.add_local_facts - @facts.values["clientversion"].should == Puppet.version.to_s - end + it "adds the Puppet version as a 'clientversion' fact" do + @facts.add_local_facts + @facts.values["clientversion"].should == Puppet.version.to_s + end + + it "adds the agent side noop setting as 'clientnoop'" do + @facts.add_local_facts + @facts.values["clientnoop"].should == Puppet.settings[:noop] + end - it "should not add the current environment" do - @facts.add_local_facts - @facts.values.should_not include("environment") + it "doesn't add the current environment" do + @facts.add_local_facts + @facts.values.should_not include("environment") + end + + it "doesn't replace any existing environment fact when adding local facts" do + @facts.values["environment"] = "foo" + @facts.add_local_facts + @facts.values["environment"].should == "foo" + end end - it "should not replace any existing environment fact when adding local facts" do - @facts.values["environment"] = "foo" - @facts.add_local_facts - @facts.values["environment"].should == "foo" + describe "when sanitizing facts" do + it "should convert fact values if needed" do + @facts.values["test"] = /foo/ + @facts.sanitize + @facts.values["test"].should == "(?-mix:foo)" + end + + it "should convert hash keys if needed" do + @facts.values["test"] = {/foo/ => "bar"} + @facts.sanitize + @facts.values["test"].should == {"(?-mix:foo)" => "bar"} + end + + it "should convert hash values if needed" do + @facts.values["test"] = {"foo" => /bar/} + @facts.sanitize + @facts.values["test"].should == {"foo" => "(?-mix:bar)"} + end + + it "should convert array elements if needed" do + @facts.values["test"] = [1, "foo", /bar/] + @facts.sanitize + @facts.values["test"].should == [1, "foo", "(?-mix:bar)"] + end + + it "should handle nested arrays" do + @facts.values["test"] = [1, "foo", [/bar/]] + @facts.sanitize + @facts.values["test"].should == [1, "foo", ["(?-mix:bar)"]] + end + + it "should handle nested hashes" do + @facts.values["test"] = {/foo/ => {"bar" => /baz/}} + @facts.sanitize + @facts.values["test"].should == {"(?-mix:foo)" => {"bar" => "(?-mix:baz)"}} + end + + it "should handle nester arrays and hashes" do + @facts.values["test"] = {/foo/ => ["bar", /baz/]} + @facts.sanitize + @facts.values["test"].should == {"(?-mix:foo)" => ["bar", "(?-mix:baz)"]} + end end describe "when indirecting" do @@ -67,7 +118,7 @@ describe Puppet::Node::Facts, "when indirecting" do describe "when storing and retrieving" do it "should add metadata to the facts" do facts = Puppet::Node::Facts.new("me", "one" => "two", "three" => "four") - facts.values[:_timestamp].should be_instance_of(Time) + facts.values['_timestamp'].should be_instance_of(Time) end describe "using pson" do @@ -82,7 +133,7 @@ describe Puppet::Node::Facts, "when indirecting" do facts = format.intern(Puppet::Node::Facts,pson) facts.name.should == 'foo' facts.expiration.should == @expiration - facts.values.should == {'a' => '1', 'b' => '2', 'c' => '3', :_timestamp => @timestamp} + facts.values.should == {'a' => '1', 'b' => '2', 'c' => '3', '_timestamp' => @timestamp} end it "should generate properly formatted pson" do diff --git a/spec/unit/other/selinux_spec.rb b/spec/unit/other/selinux_spec.rb index 3aa71cb6c..e5b5ac03b 100755 --- a/spec/unit/other/selinux_spec.rb +++ b/spec/unit/other/selinux_spec.rb @@ -9,23 +9,24 @@ describe Puppet::Type.type(:file), " when manipulating file contexts" do before :each do - @file = Puppet::Type::File.new( - + @file = Puppet::Type::File.new( :name => make_absolute("/tmp/foo"), :ensure => "file", :seluser => "user_u", :selrole => "role_r", - - :seltype => "type_t" ) + :seltype => "type_t") end + it "should use :seluser to get/set an SELinux user file context attribute" do - @file.property(:seluser).should == "user_u" + expect(@file[:seluser]).to eq("user_u") end + it "should use :selrole to get/set an SELinux role file context attribute" do - @file.property(:selrole).should == "role_r" + expect(@file[:selrole]).to eq("role_r") end + it "should use :seltype to get/set an SELinux user file context attribute" do - @file.property(:seltype).should == "type_t" + expect(@file[:seltype]).to eq("type_t") end end @@ -34,27 +35,29 @@ describe Puppet::Type.type(:selboolean), " when manipulating booleans" do provider_class = Puppet::Type::Selboolean.provider(Puppet::Type::Selboolean.providers[0]) Puppet::Type::Selboolean.stubs(:defaultprovider).returns provider_class - - @bool = Puppet::Type::Selboolean.new( - + @bool = Puppet::Type::Selboolean.new( :name => "foo", :value => "on", - :persistent => true ) end + it "should be able to access :name" do @bool[:name].should == "foo" end + it "should be able to access :value" do - @bool.property(:value).should == :on + expect(@bool.property(:value).should).to eq(:on) end + it "should set :value to off" do @bool[:value] = :off - @bool.property(:value).should == :off + expect(@bool.property(:value).should).to eq(:off) end + it "should be able to access :persistent" do @bool[:persistent].should == :true end + it "should set :persistent to false" do @bool[:persistent] = false @bool[:persistent].should == :false @@ -66,30 +69,31 @@ describe Puppet::Type.type(:selmodule), " when checking policy modules" do provider_class = Puppet::Type::Selmodule.provider(Puppet::Type::Selmodule.providers[0]) Puppet::Type::Selmodule.stubs(:defaultprovider).returns provider_class - - @module = Puppet::Type::Selmodule.new( - + @module = Puppet::Type::Selmodule.new( :name => "foo", :selmoduledir => "/some/path", :selmodulepath => "/some/path/foo.pp", - :syncversion => true) end + it "should be able to access :name" do @module[:name].should == "foo" end + it "should be able to access :selmoduledir" do @module[:selmoduledir].should == "/some/path" end + it "should be able to access :selmodulepath" do @module[:selmodulepath].should == "/some/path/foo.pp" end + it "should be able to access :syncversion" do - @module.property(:syncversion).should == :true + expect(@module[:syncversion]).to eq(:true) end + it "should set the syncversion value to false" do @module[:syncversion] = :false - @module.property(:syncversion).should == :false + expect(@module[:syncversion]).to eq(:false) end end - diff --git a/spec/unit/parameter/boolean_spec.rb b/spec/unit/parameter/boolean_spec.rb new file mode 100644 index 000000000..7039c42fc --- /dev/null +++ b/spec/unit/parameter/boolean_spec.rb @@ -0,0 +1,25 @@ +#! /usr/bin/env ruby +require 'spec_helper' +require 'puppet' +require 'puppet/parameter/boolean' + +describe Puppet::Parameter::Boolean do + let (:resource) { mock('resource') } + subject { described_class.new(:resource => resource) } + + [ true, :true, 'true', :yes, 'yes', 'TrUe', 'yEs' ].each do |arg| + it "should munge #{arg.inspect} as true" do + subject.munge(arg).should == true + end + end + [ false, :false, 'false', :no, 'no', 'FaLSE', 'nO' ].each do |arg| + it "should munge #{arg.inspect} as false" do + subject.munge(arg).should == false + end + end + [ nil, :undef, 'undef', '0', 0, '1', 1, 9284 ].each do |arg| + it "should fail to munge #{arg.inspect}" do + expect { subject.munge(arg) }.to raise_error Puppet::Error + end + end +end diff --git a/spec/unit/parameter/value_collection_spec.rb b/spec/unit/parameter/value_collection_spec.rb index 55c844d61..16632ec97 100755 --- a/spec/unit/parameter/value_collection_spec.rb +++ b/spec/unit/parameter/value_collection_spec.rb @@ -27,7 +27,7 @@ describe Puppet::Parameter::ValueCollection do end it "should be able to add values that are empty strings" do - lambda { @collection.newvalue('') }.should_not raise_error + expect { @collection.newvalue('') }.to_not raise_error end it "should be able to add values that are empty strings" do @@ -107,33 +107,33 @@ describe Puppet::Parameter::ValueCollection do it "should fail if the value is not a defined value or alias and does not match a regex" do @collection.newvalues :foo - lambda { @collection.validate("bar") }.should raise_error(ArgumentError) + expect { @collection.validate("bar") }.to raise_error(ArgumentError) end it "should succeed if the value is one of the defined values" do @collection.newvalues :foo - lambda { @collection.validate(:foo) }.should_not raise_error(ArgumentError) + expect { @collection.validate(:foo) }.to_not raise_error end it "should succeed if the value is one of the defined values even if the definition uses a symbol and the validation uses a string" do @collection.newvalues :foo - lambda { @collection.validate("foo") }.should_not raise_error(ArgumentError) + expect { @collection.validate("foo") }.to_not raise_error end it "should succeed if the value is one of the defined values even if the definition uses a string and the validation uses a symbol" do @collection.newvalues "foo" - lambda { @collection.validate(:foo) }.should_not raise_error(ArgumentError) + expect { @collection.validate(:foo) }.to_not raise_error end it "should succeed if the value is one of the defined aliases" do @collection.newvalues :foo @collection.aliasvalue :bar, :foo - lambda { @collection.validate("bar") }.should_not raise_error(ArgumentError) + expect { @collection.validate("bar") }.to_not raise_error end it "should succeed if the value matches one of the regexes" do @collection.newvalues %r{\d} - lambda { @collection.validate("10") }.should_not raise_error(ArgumentError) + expect { @collection.validate("10") }.to_not raise_error end end diff --git a/spec/unit/parameter_spec.rb b/spec/unit/parameter_spec.rb index f305caab1..bde4d22eb 100755 --- a/spec/unit/parameter_spec.rb +++ b/spec/unit/parameter_spec.rb @@ -37,11 +37,8 @@ describe Puppet::Parameter do @parameter.tags.should == %w{one two foo} end - it "should provide source_descriptors" do - @resource.expects(:line).returns 10 - @resource.expects(:file).returns "file" - @resource.expects(:tags).returns %w{one two} - @parameter.source_descriptors.should == {:tags=>["one", "two", "foo"], :path=>"//foo", :file => "file", :line => 10} + it "should have a path" do + @parameter.path.should == "//foo" end describe "when returning the value" do @@ -83,38 +80,38 @@ describe Puppet::Parameter do it "should catch abnormal failures thrown during validation" do @class.validate { |v| raise "This is broken" } - lambda { @parameter.validate("eh") }.should raise_error(Puppet::DevError) + expect { @parameter.validate("eh") }.to raise_error(Puppet::DevError) end it "should fail if the value is not a defined value or alias and does not match a regex" do @class.newvalues :foo - lambda { @parameter.validate("bar") }.should raise_error(Puppet::Error) + expect { @parameter.validate("bar") }.to raise_error(Puppet::Error) end it "should succeed if the value is one of the defined values" do @class.newvalues :foo - lambda { @parameter.validate(:foo) }.should_not raise_error(ArgumentError) + expect { @parameter.validate(:foo) }.to_not raise_error end it "should succeed if the value is one of the defined values even if the definition uses a symbol and the validation uses a string" do @class.newvalues :foo - lambda { @parameter.validate("foo") }.should_not raise_error(ArgumentError) + expect { @parameter.validate("foo") }.to_not raise_error end it "should succeed if the value is one of the defined values even if the definition uses a string and the validation uses a symbol" do @class.newvalues "foo" - lambda { @parameter.validate(:foo) }.should_not raise_error(ArgumentError) + expect { @parameter.validate(:foo) }.to_not raise_error end it "should succeed if the value is one of the defined aliases" do @class.newvalues :foo @class.aliasvalue :bar, :foo - lambda { @parameter.validate("bar") }.should_not raise_error(ArgumentError) + expect { @parameter.validate("bar") }.to_not raise_error end it "should succeed if the value matches one of the regexes" do @class.newvalues %r{\d} - lambda { @parameter.validate("10") }.should_not raise_error(ArgumentError) + expect { @parameter.validate("10") }.to_not raise_error end end @@ -125,7 +122,7 @@ describe Puppet::Parameter do it "should catch abnormal failures thrown during munging" do @class.munge { |v| raise "This is broken" } - lambda { @parameter.munge("eh") }.should raise_error(Puppet::DevError) + expect { @parameter.munge("eh") }.to raise_error(Puppet::DevError) end it "should return return any matching defined values" do diff --git a/spec/unit/parser/ast/function_spec.rb b/spec/unit/parser/ast/function_spec.rb index 9dfdb291a..6115ad1d2 100755 --- a/spec/unit/parser/ast/function_spec.rb +++ b/spec/unit/parser/ast/function_spec.rb @@ -10,7 +10,7 @@ describe Puppet::Parser::AST::Function do it "should not fail if the function doesn't exist" do Puppet::Parser::Functions.stubs(:function).returns(false) - lambda{ Puppet::Parser::AST::Function.new :name => "dontexist" }.should_not raise_error(Puppet::ParseError) + expect{ Puppet::Parser::AST::Function.new :name => "dontexist" }.to_not raise_error end end @@ -27,7 +27,7 @@ describe Puppet::Parser::AST::Function do Puppet::Parser::Functions.stubs(:function).returns(false) func = Puppet::Parser::AST::Function.new :name => "dontexist" - lambda{ func.evaluate(@scope) }.should raise_error(Puppet::ParseError) + expect{ func.evaluate(@scope) }.to raise_error(Puppet::ParseError) end it "should fail if the function is a statement used as rvalue" do @@ -36,7 +36,7 @@ describe Puppet::Parser::AST::Function do func = Puppet::Parser::AST::Function.new :name => "exist", :ftype => :rvalue - lambda{ func.evaluate(@scope) }.should raise_error(Puppet::ParseError, "Function 'exist' does not return a value") + expect{ func.evaluate(@scope) }.to raise_error(Puppet::ParseError, "Function 'exist' does not return a value") end it "should fail if the function is an rvalue used as statement" do @@ -45,7 +45,7 @@ describe Puppet::Parser::AST::Function do func = Puppet::Parser::AST::Function.new :name => "exist", :ftype => :statement - lambda{ func.evaluate(@scope) }.should raise_error(Puppet::ParseError,"Function 'exist' must be the value of a statement") + expect{ func.evaluate(@scope) }.to raise_error(Puppet::ParseError,"Function 'exist' must be the value of a statement") end it "should evaluate its arguments" do diff --git a/spec/unit/parser/ast/leaf_spec.rb b/spec/unit/parser/ast/leaf_spec.rb index dda843383..bee9fc2bd 100755 --- a/spec/unit/parser/ast/leaf_spec.rb +++ b/spec/unit/parser/ast/leaf_spec.rb @@ -10,7 +10,7 @@ describe Puppet::Parser::AST::Leaf do @leaf = Puppet::Parser::AST::Leaf.new(:value => @value) end - it "should have a evaluate_match method" do + it "should have an evaluate_match method" do Puppet::Parser::AST::Leaf.new(:value => "value").should respond_to(:evaluate_match) end @@ -177,7 +177,7 @@ describe Puppet::Parser::AST::HashOrArrayAccess do access.evaluate(@scope).should == :undef end - it "should be able to return an hash value" do + it "should be able to return a hash value" do @scope["a"] = { "key1" => "val1", "key2" => "val2", "key3" => "val3" } access = Puppet::Parser::AST::HashOrArrayAccess.new(:variable => "a", :key => "key2" ) @@ -193,7 +193,7 @@ describe Puppet::Parser::AST::HashOrArrayAccess do access.evaluate(@scope).should == :undef end - it "should be able to return an hash value with a numerical key" do + it "should be able to return a hash value with a numerical key" do @scope["a"] = { "key1" => "val1", "key2" => "val2", "45" => "45", "key3" => "val3" } access = Puppet::Parser::AST::HashOrArrayAccess.new(:variable => "a", :key => "45" ) @@ -201,7 +201,7 @@ describe Puppet::Parser::AST::HashOrArrayAccess do access.evaluate(@scope).should == "45" end - it "should raise an error if the variable lookup didn't return an hash or an array" do + it "should raise an error if the variable lookup didn't return a hash or an array" do @scope["a"] = "I'm a string" access = Puppet::Parser::AST::HashOrArrayAccess.new(:variable => "a", :key => "key2" ) @@ -237,6 +237,45 @@ describe Puppet::Parser::AST::HashOrArrayAccess do access2.evaluate(@scope).should == 'b' end + + it "should raise a useful error for hash access on undef" do + @scope["a"] = :undef + + access = Puppet::Parser::AST::HashOrArrayAccess.new(:variable => "a", :key => "key") + + expect { + access.evaluate(@scope) + }.to raise_error(Puppet::ParseError, /not a hash or array/) + end + + it "should raise a useful error for hash access on TrueClass" do + @scope["a"] = true + + access = Puppet::Parser::AST::HashOrArrayAccess.new(:variable => "a", :key => "key") + + expect { + access.evaluate(@scope) + }.to raise_error(Puppet::ParseError, /not a hash or array/) + end + + it "should raise a useful error for recursive undef hash access" do + @scope["a"] = { "key" => "val" } + + access1 = Puppet::Parser::AST::HashOrArrayAccess.new(:variable => "a", :key => "nonexistent") + access2 = Puppet::Parser::AST::HashOrArrayAccess.new(:variable => access1, :key => "subkey") + + expect { + access2.evaluate(@scope) + }.to raise_error(Puppet::ParseError, /not a hash or array/) + end + + it "should produce boolean values when value is a boolean" do + @scope["a"] = [true, false] + access = Puppet::Parser::AST::HashOrArrayAccess.new(:variable => "a", :key => 0 ) + expect(access.evaluate(@scope)).to be == true + access = Puppet::Parser::AST::HashOrArrayAccess.new(:variable => "a", :key => 1 ) + expect(access.evaluate(@scope)).to be == false + end end describe "when assigning" do @@ -274,7 +313,7 @@ describe Puppet::Parser::AST::HashOrArrayAccess do scope['a'].should == ["val2"] end - it "should raise an error when trying to overwrite an hash value" do + it "should raise an error when trying to overwrite a hash value" do @scope['a'] = { "key" => [ "a" , "b" ]} access = Puppet::Parser::AST::HashOrArrayAccess.new(:variable => "a", :key => "key") @@ -425,7 +464,7 @@ describe Puppet::Parser::AST::HostName do end it "should raise an error if hostname is not valid" do - lambda { Puppet::Parser::AST::HostName.new( :value => "not an hostname!" ) }.should raise_error + lambda { Puppet::Parser::AST::HostName.new( :value => "not a hostname!" ) }.should raise_error end it "should not raise an error if hostname is a regex" do diff --git a/spec/unit/parser/collector_spec.rb b/spec/unit/parser/collector_spec.rb index 5d4af0c11..9596bd0c1 100755 --- a/spec/unit/parser/collector_spec.rb +++ b/spec/unit/parser/collector_spec.rb @@ -23,7 +23,7 @@ describe Puppet::Parser::Collector, "when initializing" do end it "should only accept :virtual or :exported as the collector form" do - proc { @collector = Puppet::Parser::Collector.new(@scope, @resource_type, @vquery, @equery, :other) }.should raise_error(ArgumentError) + expect { @collector = Puppet::Parser::Collector.new(@scope, @resource_type, @vquery, @equery, :other) }.to raise_error(ArgumentError) end it "should accept an optional virtual query" do @@ -61,7 +61,7 @@ describe Puppet::Parser::Collector, "when collecting specific virtual resources" it "should not fail when it does not find any resources to collect" do @collector.resources = ["File[virtual1]", "File[virtual2]"] @scope.stubs(:findresource).returns(false) - proc { @collector.evaluate }.should_not raise_error + expect { @collector.evaluate }.to_not raise_error end it "should mark matched resources as non-virtual" do @@ -429,7 +429,7 @@ describe Puppet::Parser::Collector, "when collecting exported resources", :if => @compiler.add_resource(@scope, local) got = nil - expect { got = @collector.evaluate }.not_to raise_error(Puppet::ParseError) + expect { got = @collector.evaluate }.not_to raise_error got.length.should == 1 got.first.type.should == "Notify" got.first.title.should == "boingy-boingy" diff --git a/spec/unit/parser/compiler_spec.rb b/spec/unit/parser/compiler_spec.rb index d64e57e8b..8e71756e1 100755 --- a/spec/unit/parser/compiler_spec.rb +++ b/spec/unit/parser/compiler_spec.rb @@ -225,9 +225,10 @@ describe Puppet::Parser::Compiler do three = stub 'three', :name => "three" @node.stubs(:name).returns("whatever") @node.stubs(:classes).returns(classes) + compile_stub(:evaluate_node_classes) @compiler.expects(:evaluate_classes).with(classes, @compiler.topscope) - @compiler.class.publicize_methods(:evaluate_node_classes) { @compiler.evaluate_node_classes } + @compiler.compile end it "should evaluate any parameterized classes named in the node" do @@ -521,7 +522,7 @@ describe Puppet::Parser::Compiler do end } - @compiler.class.publicize_methods(:evaluate_collections) { @compiler.evaluate_collections } + @compiler.compile end it "should not fail when there are unevaluated resource collections that do not refer to specific resources" do @@ -576,7 +577,7 @@ describe Puppet::Parser::Compiler do it "should raise an error when it can't find class" do klasses = {'foo'=>nil} @node.classes = klasses - @compiler.topscope.stubs(:find_hostclass).with('foo', {:assume_fqname => false}).returns(nil) + @compiler.topscope.expects(:find_hostclass).with('foo', {:assume_fqname => false}).returns(nil) lambda{ @compiler.compile }.should raise_error(Puppet::Error, /Could not find class foo for testnode/) end end diff --git a/spec/unit/parser/functions/create_resources_spec.rb b/spec/unit/parser/functions/create_resources_spec.rb index b16f9111b..a0c3253ab 100755 --- a/spec/unit/parser/functions/create_resources_spec.rb +++ b/spec/unit/parser/functions/create_resources_spec.rb @@ -23,21 +23,6 @@ describe 'function for dynamically creating resources' do expect { @scope.function_create_resources(['foo', 'bar', 'blah', 'baz']) }.to raise_error(ArgumentError, 'create_resources(): wrong number of arguments (4; must be 2 or 3)') end - describe 'when the caller does not supply a name parameter' do - it 'should set a default resource name equal to the resource title' do - Puppet::Parser::Resource.any_instance.expects(:set_parameter).with(:name, 'test').once - @scope.function_create_resources(['notify', {'test'=>{}}]) - end - end - - describe 'when the caller supplies a name parameter' do - it 'should set the resource name to the value provided' do - Puppet::Parser::Resource.any_instance.expects(:set_parameter).with(:name, 'user_supplied').once - Puppet::Parser::Resource.any_instance.expects(:set_parameter).with(:name, 'test').never - @scope.function_create_resources(['notify', {'test'=>{'name' => 'user_supplied'}}]) - end - end - describe 'when creating native types' do it 'empty hash should not cause resources to be added' do noop_catalog = compile_to_catalog("create_resources('file', {})") @@ -68,12 +53,13 @@ describe 'function for dynamically creating resources' do end it 'should fail to add non-existing type' do - expect { @scope.function_create_resources(['create-resource-foo', {}]) }.to raise_error(ArgumentError, 'could not create resource of unknown type create-resource-foo') + expect do + @scope.function_create_resources(['create-resource-foo', { 'foo' => {} }]) + end.to raise_error(ArgumentError, /Invalid resource type create-resource-foo/) end it 'should be able to add edges' do - catalog = compile_to_catalog("notify { test: }\n create_resources('notify', {'foo'=>{'require'=>'Notify[test]'}})") - rg = catalog.to_ral.relationship_graph + rg = compile_to_relationship_graph("notify { test: }\n create_resources('notify', {'foo'=>{'require'=>'Notify[test]'}})") test = rg.vertices.find { |v| v.title == 'test' } foo = rg.vertices.find { |v| v.title == 'foo' } test.must be @@ -87,6 +73,7 @@ describe 'function for dynamically creating resources' do catalog.resource(:file, "/etc/baz")['group'].should == 'food' end end + describe 'when dynamically creating resource types' do it 'should be able to create defined resoure types' do catalog = compile_to_catalog(<<-MANIFEST) @@ -125,7 +112,7 @@ describe 'function for dynamically creating resources' do end it 'should be able to add edges' do - catalog = compile_to_catalog(<<-MANIFEST) + rg = compile_to_relationship_graph(<<-MANIFEST) define foocreateresource($one) { notify { $name: message => $one } } @@ -135,13 +122,11 @@ describe 'function for dynamically creating resources' do create_resources('foocreateresource', {'blah'=>{'one'=>'two', 'require' => 'Notify[test]'}}) MANIFEST - rg = catalog.to_ral.relationship_graph test = rg.vertices.find { |v| v.title == 'test' } blah = rg.vertices.find { |v| v.title == 'blah' } test.must be blah.must be rg.path_between(test,blah).should be - catalog.resource(:notify, "blah")['message'].should == 'two' end it 'should account for default values' do @@ -172,15 +157,15 @@ describe 'function for dynamically creating resources' do end it 'should fail to create non-existing classes' do - expect { + expect do compile_to_catalog(<<-MANIFEST) create_resources('class', {'blah'=>{'one'=>'two'}}) MANIFEST - }.to raise_error(Puppet::Error ,'could not find hostclass blah at line 1 on node foonode') + end.to raise_error(Puppet::Error, 'Could not find declared class blah at line 1 on node foonode') end it 'should be able to add edges' do - catalog = compile_to_catalog(<<-MANIFEST) + rg = compile_to_relationship_graph(<<-MANIFEST) class bar($one) { notify { test: message => $one } } @@ -190,7 +175,6 @@ describe 'function for dynamically creating resources' do create_resources('class', {'bar'=>{'one'=>'two', 'require' => 'Notify[tester]'}}) MANIFEST - rg = catalog.to_ral.relationship_graph test = rg.vertices.find { |v| v.title == 'test' } tester = rg.vertices.find { |v| v.title == 'tester' } test.must be diff --git a/spec/unit/parser/functions/extlookup_spec.rb b/spec/unit/parser/functions/extlookup_spec.rb index ecd7a817a..53c22a639 100755 --- a/spec/unit/parser/functions/extlookup_spec.rb +++ b/spec/unit/parser/functions/extlookup_spec.rb @@ -18,11 +18,11 @@ describe "the extlookup function" do Puppet::Parser::Functions.function("extlookup").should == "function_extlookup" end - it "should raise a ArgumentError if there is less than 1 arguments" do + it "should raise an ArgumentError if there is less than 1 arguments" do lambda { @scope.function_extlookup([]) }.should( raise_error(ArgumentError)) end - it "should raise a ArgumentError if there is more than 3 arguments" do + it "should raise an ArgumentError if there is more than 3 arguments" do lambda { @scope.function_extlookup(["foo", "bar", "baz", "gazonk"]) }.should( raise_error(ArgumentError)) end diff --git a/spec/unit/parser/functions/hiera_include_spec.rb b/spec/unit/parser/functions/hiera_include_spec.rb index a71bdb6c5..76a2c5dec 100644 --- a/spec/unit/parser/functions/hiera_include_spec.rb +++ b/spec/unit/parser/functions/hiera_include_spec.rb @@ -21,4 +21,16 @@ describe 'Puppet::Parser::Functions#hiera_include' do HieraPuppet.expects(:lookup).with() { |*args| args[4].should be :array }.returns(['someclass']) expect { scope.function_hiera_include(['key']) }.to raise_error Puppet::Error, /Could not find class someclass/ end + + it 'should call the `include` function with the classes' do + HieraPuppet.expects(:lookup).returns %w[foo bar baz] + + scope.expects(:function_include).with([%w[foo bar baz]]) + scope.function_hiera_include(['key']) + end + + it 'should not raise an error if the resulting hiera lookup returns an empty array' do + HieraPuppet.expects(:lookup).returns [] + expect { scope.function_hiera_include(['key']) }.to_not raise_error + end end diff --git a/spec/unit/parser/functions/lookup_spec.rb b/spec/unit/parser/functions/lookup_spec.rb new file mode 100644 index 000000000..a468ba7ec --- /dev/null +++ b/spec/unit/parser/functions/lookup_spec.rb @@ -0,0 +1,96 @@ +require 'spec_helper' +require 'puppet/pops' +require 'stringio' + +describe "lookup function" do + before(:each) do + Puppet[:binder] = true + end + + it "must be called with at least a name to lookup" do + scope = scope_with_injections_from(bound(bindings)) + + expect do + scope.function_lookup([]) + end.to raise_error(ArgumentError, /Wrong number of arguments/) + end + + it "looks up a value that exists" do + scope = scope_with_injections_from(bound(bind_single("a_value", "something"))) + + expect(scope.function_lookup(['a_value'])).to eq('something') + end + + it "returns :undef when the requested value is not bound" do + scope = scope_with_injections_from(bound(bind_single("a_value", "something"))) + + expect(scope.function_lookup(['not_bound_value'])).to eq(:undef) + end + + it "raises an error when the bound type is not assignable to the requested type" do + scope = scope_with_injections_from(bound(bind_single("a_value", "something"))) + + expect do + scope.function_lookup(['a_value', 'Integer']) + end.to raise_error(ArgumentError, /incompatible type, expected: Integer, got: String/) + end + + it "returns the value if the bound type is assignable to the requested type" do + typed_bindings = bindings + typed_bindings.bind().string().name("a_value").to("something") + scope = scope_with_injections_from(bound(typed_bindings)) + + expect(scope.function_lookup(['a_value', 'Data'])).to eq("something") + end + + it "yields to a given lambda and returns the result" do + scope = scope_with_injections_from(bound(bind_single("a_value", "something"))) + + expect(scope.function_lookup(['a_value', ast_lambda('|$x|{something_else}')])).to eq('something_else') + end + + it "yields to a given lambda and returns the result when giving name and type" do + scope = scope_with_injections_from(bound(bind_single("a_value", "something"))) + + expect(scope.function_lookup(['a_value', 'String', ast_lambda('|$x|{something_else}')])).to eq('something_else') + end + + it "yields :undef when value is not found and using a lambda" do + scope = scope_with_injections_from(bound(bind_single("a_value", "something"))) + + expect(scope.function_lookup(['not_bound_value', ast_lambda('|$x|{ if $x == undef {good} else {bad}}')])).to eq('good') + end + + def scope_with_injections_from(binder) + injector = Puppet::Pops::Binder::Injector.new(binder) + scope = Puppet::Parser::Scope.new_for_test_harness('testing') + scope.compiler.injector = injector + + scope + end + + def bindings + Puppet::Pops::Binder::BindingsFactory.named_bindings("testing") + end + + def bind_single(name, value) + local_bindings = Puppet::Pops::Binder::BindingsFactory.named_bindings("testing") + local_bindings.bind().name(name).to(value) + local_bindings + end + + def bound(local_bindings) + binder = Puppet::Pops::Binder::Binder.new + binder.define_categories(Puppet::Pops::Binder::BindingsFactory.categories([])) + binder.define_layers(Puppet::Pops::Binder::BindingsFactory.layered_bindings(Puppet::Pops::Binder::BindingsFactory.named_layer('test layer', local_bindings.model))) + + binder + end + + def ast_lambda(puppet_source) + puppet_source = "fake_func() " + puppet_source + model = Puppet::Pops::Parser::EvaluatingParser.new().parse_string(puppet_source, __FILE__).current + model = model.lambda + Puppet::Pops::Model::AstTransformer.new(@file_source, nil).transform(model) + end +end diff --git a/spec/unit/parser/functions/regsubst_spec.rb b/spec/unit/parser/functions/regsubst_spec.rb index c75b16c0e..1c65b370a 100755 --- a/spec/unit/parser/functions/regsubst_spec.rb +++ b/spec/unit/parser/functions/regsubst_spec.rb @@ -16,11 +16,11 @@ describe "the regsubst function" do Puppet::Parser::Functions.function("regsubst").should == "function_regsubst" end - it "should raise a ArgumentError if there is less than 3 arguments" do + it "should raise an ArgumentError if there is less than 3 arguments" do lambda { @scope.function_regsubst(["foo", "bar"]) }.should( raise_error(ArgumentError)) end - it "should raise a ArgumentError if there is more than 5 arguments" do + it "should raise an ArgumentError if there is more than 5 arguments" do lambda { @scope.function_regsubst(["foo", "bar", "gazonk", "del", "x", "y"]) }.should( raise_error(ArgumentError)) end diff --git a/spec/unit/parser/functions/split_spec.rb b/spec/unit/parser/functions/split_spec.rb index 5ddbe8d44..9a69c8f9f 100755 --- a/spec/unit/parser/functions/split_spec.rb +++ b/spec/unit/parser/functions/split_spec.rb @@ -16,11 +16,11 @@ describe "the split function" do Puppet::Parser::Functions.function("split").should == "function_split" end - it "should raise a ArgumentError if there is less than 2 arguments" do + it "should raise an ArgumentError if there is less than 2 arguments" do lambda { @scope.function_split(["foo"]) }.should( raise_error(ArgumentError)) end - it "should raise a ArgumentError if there is more than 2 arguments" do + it "should raise an ArgumentError if there is more than 2 arguments" do lambda { @scope.function_split(["foo", "bar", "gazonk"]) }.should( raise_error(ArgumentError)) end diff --git a/spec/unit/parser/functions/sprintf_spec.rb b/spec/unit/parser/functions/sprintf_spec.rb index eb4515a39..c5ddfc246 100755 --- a/spec/unit/parser/functions/sprintf_spec.rb +++ b/spec/unit/parser/functions/sprintf_spec.rb @@ -16,7 +16,7 @@ describe "the sprintf function" do Puppet::Parser::Functions.function("sprintf").should == "function_sprintf" end - it "should raise a ArgumentError if there is less than 1 argument" do + it "should raise an ArgumentError if there is less than 1 argument" do lambda { @scope.function_sprintf([]) }.should( raise_error(ArgumentError)) end diff --git a/spec/unit/parser/functions/versioncmp_spec.rb b/spec/unit/parser/functions/versioncmp_spec.rb index 760a286c5..626f71108 100755 --- a/spec/unit/parser/functions/versioncmp_spec.rb +++ b/spec/unit/parser/functions/versioncmp_spec.rb @@ -16,11 +16,11 @@ describe "the versioncmp function" do Puppet::Parser::Functions.function("versioncmp").should == "function_versioncmp" end - it "should raise a ArgumentError if there is less than 2 arguments" do + it "should raise an ArgumentError if there is less than 2 arguments" do lambda { @scope.function_versioncmp(["1.2"]) }.should raise_error(ArgumentError) end - it "should raise a ArgumentError if there is more than 2 arguments" do + it "should raise an ArgumentError if there is more than 2 arguments" do lambda { @scope.function_versioncmp(["1.2", "2.4.5", "3.5.6"]) }.should raise_error(ArgumentError) end diff --git a/spec/unit/parser/functions_spec.rb b/spec/unit/parser/functions_spec.rb index 41ef54436..81ff655a3 100755 --- a/spec/unit/parser/functions_spec.rb +++ b/spec/unit/parser/functions_spec.rb @@ -38,7 +38,7 @@ describe Puppet::Parser::Functions do end it "should raise an error if the function type is not correct" do - lambda { Puppet::Parser::Functions.newfunction("name", :type => :unknown) { |args| } }.should raise_error Puppet::DevError, "Invalid statement type :unknown" + expect { Puppet::Parser::Functions.newfunction("name", :type => :unknown) { |args| } }.to raise_error Puppet::DevError, "Invalid statement type :unknown" end it "instruments the function to profiles the execution" do @@ -85,32 +85,32 @@ describe Puppet::Parser::Functions do it "should raise an error if the function is called with too many arguments" do Puppet::Parser::Functions.newfunction("name", :arity => 2) { |args| } - lambda { callable_functions_from(function_module).function_name([1,2,3]) }.should raise_error ArgumentError + expect { callable_functions_from(function_module).function_name([1,2,3]) }.to raise_error ArgumentError end it "should raise an error if the function is called with too few arguments" do Puppet::Parser::Functions.newfunction("name", :arity => 2) { |args| } - lambda { callable_functions_from(function_module).function_name([1]) }.should raise_error ArgumentError + expect { callable_functions_from(function_module).function_name([1]) }.to raise_error ArgumentError end it "should not raise an error if the function is called with correct number of arguments" do Puppet::Parser::Functions.newfunction("name", :arity => 2) { |args| } - lambda { callable_functions_from(function_module).function_name([1,2]) }.should_not raise_error ArgumentError + expect { callable_functions_from(function_module).function_name([1,2]) }.to_not raise_error end it "should raise an error if the variable arg function is called with too few arguments" do Puppet::Parser::Functions.newfunction("name", :arity => -3) { |args| } - lambda { callable_functions_from(function_module).function_name([1]) }.should raise_error ArgumentError + expect { callable_functions_from(function_module).function_name([1]) }.to raise_error ArgumentError end it "should not raise an error if the variable arg function is called with correct number of arguments" do Puppet::Parser::Functions.newfunction("name", :arity => -3) { |args| } - lambda { callable_functions_from(function_module).function_name([1,2]) }.should_not raise_error ArgumentError + expect { callable_functions_from(function_module).function_name([1,2]) }.to_not raise_error end it "should not raise an error if the variable arg function is called with more number of arguments" do Puppet::Parser::Functions.newfunction("name", :arity => -3) { |args| } - lambda { callable_functions_from(function_module).function_name([1,2,3]) }.should_not raise_error ArgumentError + expect { callable_functions_from(function_module).function_name([1,2,3]) }.to_not raise_error end end diff --git a/spec/unit/parser/lexer_spec.rb b/spec/unit/parser/lexer_spec.rb index e375bed39..fc8394cb1 100755 --- a/spec/unit/parser/lexer_spec.rb +++ b/spec/unit/parser/lexer_spec.rb @@ -855,7 +855,7 @@ describe "Puppet::Parser::Lexer in the old tests when lexing example files" do end end -describe "when trying to lex an non-existent file" do +describe "when trying to lex a non-existent file" do include PuppetSpec::Files it "should return an empty list of tokens" do diff --git a/spec/unit/parser/methods/collect_spec.rb b/spec/unit/parser/methods/collect_spec.rb index 13cd9eda6..acc26652a 100644 --- a/spec/unit/parser/methods/collect_spec.rb +++ b/spec/unit/parser/methods/collect_spec.rb @@ -38,6 +38,7 @@ describe 'the collect method' do catalog.resource(:file, "/file_b")['ensure'].should == 'present' catalog.resource(:file, "/file_c")['ensure'].should == 'present' end + it 'foreach on a hash selecting value' do catalog = compile_to_catalog(<<-MANIFEST) $a = {'a'=>1,'b'=>2,'c'=>3} @@ -51,6 +52,46 @@ describe 'the collect method' do catalog.resource(:file, "/file_3")['ensure'].should == 'present' end end + + context "handles data type corner cases" do + it "collect gets values that are false" do + catalog = compile_to_catalog(<<-MANIFEST) + $a = [false,false] + $a.collect |$x| { $x }.each |$i, $v| { + file { "/file_$i.$v": ensure => present } + } + MANIFEST + + catalog.resource(:file, "/file_0.false")['ensure'].should == 'present' + catalog.resource(:file, "/file_1.false")['ensure'].should == 'present' + end + + it "collect gets values that are nil" do + Puppet::Parser::Functions.newfunction(:nil_array, :type => :rvalue) do |args| + [nil] + end + catalog = compile_to_catalog(<<-MANIFEST) + $a = nil_array() + $a.collect |$x| { $x }.each |$i, $v| { + file { "/file_$i.$v": ensure => present } + } + MANIFEST + + catalog.resource(:file, "/file_0.")['ensure'].should == 'present' + end + + it "collect gets values that are undef" do + catalog = compile_to_catalog(<<-MANIFEST) + $a = [$does_not_exist] + $a.collect |$x = "something"| { $x }.each |$i, $v| { + file { "/file_$i.$v": ensure => present } + } + MANIFEST + + catalog.resource(:file, "/file_0.")['ensure'].should == 'present' + end + end + context "in Java style should be callable as" do shared_examples_for 'java style' do it 'collect on an array (multiplying each value by 2)' do @@ -92,11 +133,13 @@ describe 'the collect method' do catalog.resource(:file, "/file_3")['ensure'].should == 'present' end end + describe 'without fat arrow' do it_should_behave_like 'java style' do let(:farr) { '' } end end + describe 'with fat arrow' do it_should_behave_like 'java style' do let(:farr) { '=>' } diff --git a/spec/unit/parser/resource_spec.rb b/spec/unit/parser/resource_spec.rb index f843c895d..221efb83b 100755 --- a/spec/unit/parser/resource_spec.rb +++ b/spec/unit/parser/resource_spec.rb @@ -82,7 +82,7 @@ describe Puppet::Parser::Resource do @resource = Puppet::Parser::Resource.new("whatever", "whatever", :scope => @scope, :source => @source).isomorphic?.should be_true end - it "should have a array-indexing method for retrieving parameter values" do + it "should have an array-indexing method for retrieving parameter values" do @resource = mkresource @resource[:one].should == "yay" end @@ -181,7 +181,7 @@ describe Puppet::Parser::Resource do resource = Puppet::Parser::Resource.new(:class, "foo", :scope => @scope, :catalog => @catalog) resource[:stage] = 'other' - lambda { resource.evaluate }.should raise_error(ArgumentError, /Could not find stage other specified by/) + expect { resource.evaluate }.to raise_error(ArgumentError, /Could not find stage other specified by/) end it "should add edges from the class resources to the parent's stage if no stage is specified" do @@ -328,19 +328,19 @@ describe Puppet::Parser::Resource do end it "should fail on tags containing '*' characters" do - lambda { @resource.tag("bad*tag") }.should raise_error(Puppet::ParseError) + expect { @resource.tag("bad*tag") }.to raise_error(Puppet::ParseError) end it "should fail on tags starting with '-' characters" do - lambda { @resource.tag("-badtag") }.should raise_error(Puppet::ParseError) + expect { @resource.tag("-badtag") }.to raise_error(Puppet::ParseError) end it "should fail on tags containing ' ' characters" do - lambda { @resource.tag("bad tag") }.should raise_error(Puppet::ParseError) + expect { @resource.tag("bad tag") }.to raise_error(Puppet::ParseError) end it "should allow alpha tags" do - lambda { @resource.tag("good_tag") }.should_not raise_error(Puppet::ParseError) + expect { @resource.tag("good_tag") }.to_not raise_error end end @@ -354,7 +354,7 @@ describe Puppet::Parser::Resource do it "should fail when the override was not created by a parent class" do @override.source = "source2" @override.source.expects(:child_of?).with("source1").returns(false) - lambda { @resource.merge(@override) }.should raise_error(Puppet::ParseError) + expect { @resource.merge(@override) }.to raise_error(Puppet::ParseError) end it "should succeed when the override was created in the current scope" do @@ -550,7 +550,7 @@ describe Puppet::Parser::Resource do resource = Puppet::Parser::Resource.new :foo, "bar", :scope => @scope, :source => stub("source") resource[:one] = :two resource.expects(:validate_parameter).with(:one).raises ArgumentError - lambda { resource.send(:validate) }.should raise_error(Puppet::ParseError) + expect { resource.send(:validate) }.to raise_error(Puppet::ParseError) end end @@ -567,7 +567,7 @@ describe Puppet::Parser::Resource do end it "should fail when provided a parameter name but no value" do - lambda { @resource.set_parameter("myparam") }.should raise_error(ArgumentError) + expect { @resource.set_parameter("myparam") }.to raise_error(ArgumentError) end it "should allow parameters to be set to 'false'" do diff --git a/spec/unit/parser/scope_spec.rb b/spec/unit/parser/scope_spec.rb index 8540ad20d..00f807dc9 100755 --- a/spec/unit/parser/scope_spec.rb +++ b/spec/unit/parser/scope_spec.rb @@ -142,8 +142,8 @@ describe Puppet::Parser::Scope do end it "should fail if invoked with a non-string name" do - expect { @scope[:foo] }.to raise_error Puppet::DevError - expect { @scope[:foo] = 12 }.to raise_error Puppet::DevError + expect { @scope[:foo] }.to raise_error(Puppet::ParseError, /Scope variable name .* not a string/) + expect { @scope[:foo] = 12 }.to raise_error(Puppet::ParseError, /Scope variable name .* not a string/) end it "should return nil for unset variables" do @@ -612,4 +612,47 @@ describe Puppet::Parser::Scope do end end end + + context "when producing a hash of all variables (as used in templates)" do + it "should contain all defined variables in the scope" do + @scope.setvar("orange", :tangerine) + @scope.setvar("pear", :green) + @scope.to_hash.should == {'orange' => :tangerine, 'pear' => :green } + end + + it "should contain variables in all local scopes (#21508)" do + @scope.new_ephemeral true + @scope.setvar("orange", :tangerine) + @scope.setvar("pear", :green) + @scope.new_ephemeral true + @scope.setvar("apple", :red) + @scope.to_hash.should == {'orange' => :tangerine, 'pear' => :green, 'apple' => :red } + end + + it "should contain all defined variables in the scope and all local scopes" do + @scope.setvar("orange", :tangerine) + @scope.setvar("pear", :green) + @scope.new_ephemeral true + @scope.setvar("apple", :red) + @scope.to_hash.should == {'orange' => :tangerine, 'pear' => :green, 'apple' => :red } + end + + it "should not contain varaibles in match scopes (non local emphemeral)" do + @scope.new_ephemeral true + @scope.setvar("orange", :tangerine) + @scope.setvar("pear", :green) + @scope.ephemeral_from(/(f)(o)(o)/.match('foo')) + @scope.to_hash.should == {'orange' => :tangerine, 'pear' => :green } + end + + it "should delete values that are :undef in inner scope" do + @scope.new_ephemeral true + @scope.setvar("orange", :tangerine) + @scope.setvar("pear", :green) + @scope.new_ephemeral true + @scope.setvar("apple", :red) + @scope.setvar("orange", :undef) + @scope.to_hash.should == {'pear' => :green, 'apple' => :red } + end + end end diff --git a/spec/unit/parser/type_loader_spec.rb b/spec/unit/parser/type_loader_spec.rb index d3abe6047..c735db7c2 100755 --- a/spec/unit/parser/type_loader_spec.rb +++ b/spec/unit/parser/type_loader_spec.rb @@ -11,235 +11,219 @@ describe Puppet::Parser::TypeLoader do include PuppetSpec::Modules include PuppetSpec::Files - shared_examples_for 'the typeloader' do - before do - @loader = Puppet::Parser::TypeLoader.new(:myenv) - Puppet.expects(:deprecation_warning).never - end + let(:empty_hostclass) { Puppet::Parser::AST::Hostclass.new('') } + let(:loader) { Puppet::Parser::TypeLoader.new(:myenv) } - it "should support an environment" do - loader = Puppet::Parser::TypeLoader.new(:myenv) - loader.environment.name.should == :myenv - end + it "should support an environment" do + loader = Puppet::Parser::TypeLoader.new(:myenv) + loader.environment.name.should == :myenv + end + + it "should include the Environment Helper" do + loader.class.ancestors.should be_include(Puppet::Node::Environment::Helper) + end - it "should include the Environment Helper" do - @loader.class.ancestors.should be_include(Puppet::Node::Environment::Helper) + it "should delegate its known resource types to its environment" do + loader.known_resource_types.should be_instance_of(Puppet::Resource::TypeCollection) + end + + describe "when loading names from namespaces" do + it "should do nothing if the name to import is an empty string" do + loader.try_load_fqname(:hostclass, "").should be_nil end - it "should delegate its known resource types to its environment" do - @loader.known_resource_types.should be_instance_of(Puppet::Resource::TypeCollection) + it "should attempt to import each generated name" do + loader.expects(:import_from_modules).with("foo/bar").returns([]) + loader.expects(:import_from_modules).with("foo").returns([]) + loader.try_load_fqname(:hostclass, "foo::bar") end - describe "when loading names from namespaces" do - it "should do nothing if the name to import is an empty string" do - @loader.expects(:name2files).never - @loader.try_load_fqname(:hostclass, "") { |filename, modname| raise :should_not_occur }.should be_nil + it "should attempt to load each possible name going from most to least specific" do + path_order = sequence('path') + ['foo/bar/baz', 'foo/bar', 'foo'].each do |path| + Puppet::Parser::Files.expects(:find_manifests_in_modules).with(path, anything).returns([nil, []]).in_sequence(path_order) end - it "should attempt to import each generated name" do - @loader.expects(:import_from_modules).with("foo/bar").returns([]) - @loader.expects(:import_from_modules).with("foo").returns([]) - @loader.try_load_fqname(:hostclass, "foo::bar") { |f| false } - end + loader.try_load_fqname(:hostclass, 'foo::bar::baz') end + end - describe "when importing" do - before do - Puppet::Parser::Files.stubs(:find_manifests_in_modules).returns ["modname", %w{file}] - parser_class.any_instance.stubs(:parse).returns(Puppet::Parser::AST::Hostclass.new('')) - parser_class.any_instance.stubs(:file=) - end + describe "when importing" do + let(:stub_parser) { stub 'Parser', :file= => nil, :parse => empty_hostclass } - it "should return immediately when imports are being ignored" do - Puppet::Parser::Files.expects(:find_manifests_in_modules).never - Puppet[:ignoreimport] = true - @loader.import("foo", "/path").should be_nil - end + before(:each) do + Puppet::Parser::ParserFactory.stubs(:parser).with(anything).returns(stub_parser) + end - it "should find all manifests matching the file or pattern" do - Puppet::Parser::Files.expects(:find_manifests_in_modules).with("myfile", anything).returns ["modname", %w{one}] - @loader.import("myfile", "/path") - end + it "should return immediately when imports are being ignored" do + Puppet::Parser::Files.expects(:find_manifests_in_modules).never + Puppet[:ignoreimport] = true + loader.import("foo", "/path").should be_nil + end - it "should pass the environment when looking for files" do - Puppet::Parser::Files.expects(:find_manifests_in_modules).with(anything, @loader.environment).returns ["modname", %w{one}] - @loader.import("myfile", "/path") - end + it "should find all manifests matching the file or pattern" do + Puppet::Parser::Files.expects(:find_manifests_in_modules).with("myfile", anything).returns ["modname", %w{one}] + loader.import("myfile", "/path") + end - it "should fail if no files are found" do - Puppet::Parser::Files.expects(:find_manifests_in_modules).returns [nil, []] - lambda { @loader.import("myfile", "/path") }.should raise_error(Puppet::ImportError) - end + it "should pass the environment when looking for files" do + Puppet::Parser::Files.expects(:find_manifests_in_modules).with(anything, loader.environment).returns ["modname", %w{one}] + loader.import("myfile", "/path") + end - it "should parse each found file" do - Puppet::Parser::Files.expects(:find_manifests_in_modules).returns ["modname", [make_absolute("/one")]] - @loader.expects(:parse_file).with(make_absolute("/one")).returns(Puppet::Parser::AST::Hostclass.new('')) - @loader.import("myfile", "/path") - end + it "should fail if no files are found" do + Puppet::Parser::Files.expects(:find_manifests_in_modules).returns [nil, []] + lambda { loader.import("myfile", "/path") }.should raise_error(Puppet::ImportError) + end - it "should not attempt to import files that have already been imported" do - @loader = Puppet::Parser::TypeLoader.new(:myenv) + it "should parse each found file" do + Puppet::Parser::Files.expects(:find_manifests_in_modules).returns ["modname", [make_absolute("/one")]] + loader.expects(:parse_file).with(make_absolute("/one")).returns(Puppet::Parser::AST::Hostclass.new('')) + loader.import("myfile", "/path") + end - Puppet::Parser::Files.expects(:find_manifests_in_modules).returns ["modname", %w{/one}] - parser_class.any_instance.expects(:parse).once.returns(Puppet::Parser::AST::Hostclass.new('')) - other_parser_class.any_instance.expects(:parse).never.returns(Puppet::Parser::AST::Hostclass.new('')) - @loader.import("myfile", "/path") + it "should not attempt to import files that have already been imported" do + loader = Puppet::Parser::TypeLoader.new(:myenv) - # This will fail if it tries to reimport the file. - @loader.import("myfile", "/path") - end + Puppet::Parser::Files.expects(:find_manifests_in_modules).twice.returns ["modname", %w{/one}] + loader.import("myfile", "/path").should_not be_empty + + loader.import("myfile", "/path").should be_empty end + end - describe "when importing all" do - before do - @base = tmpdir("base") + describe "when importing all" do + before do + @base = tmpdir("base") - # Create two module path directories - @modulebase1 = File.join(@base, "first") - FileUtils.mkdir_p(@modulebase1) - @modulebase2 = File.join(@base, "second") - FileUtils.mkdir_p(@modulebase2) + # Create two module path directories + @modulebase1 = File.join(@base, "first") + FileUtils.mkdir_p(@modulebase1) + @modulebase2 = File.join(@base, "second") + FileUtils.mkdir_p(@modulebase2) - Puppet[:modulepath] = "#{@modulebase1}#{File::PATH_SEPARATOR}#{@modulebase2}" - end + Puppet[:modulepath] = "#{@modulebase1}#{File::PATH_SEPARATOR}#{@modulebase2}" + end - def mk_module(basedir, name) - PuppetSpec::Modules.create(name, basedir) - end + def mk_module(basedir, name) + PuppetSpec::Modules.create(name, basedir) + end - # We have to pass the base path so that we can - # write to modules that are in the second search path - def mk_manifests(base, mod, type, files) - exts = {"ruby" => ".rb", "puppet" => ".pp"} - files.collect do |file| - name = mod.name + "::" + file.gsub("/", "::") - path = File.join(base, mod.name, "manifests", file + exts[type]) - FileUtils.mkdir_p(File.split(path)[0]) - - # write out the class - if type == "ruby" - File.open(path, "w") { |f| f.print "hostclass '#{name}' do\nend" } - else - File.open(path, "w") { |f| f.print "class #{name} {}" } - end - name + # We have to pass the base path so that we can + # write to modules that are in the second search path + def mk_manifests(base, mod, type, files) + exts = {"ruby" => ".rb", "puppet" => ".pp"} + files.collect do |file| + name = mod.name + "::" + file.gsub("/", "::") + path = File.join(base, mod.name, "manifests", file + exts[type]) + FileUtils.mkdir_p(File.split(path)[0]) + + # write out the class + if type == "ruby" + File.open(path, "w") { |f| f.print "hostclass '#{name}' do\nend" } + else + File.open(path, "w") { |f| f.print "class #{name} {}" } end + name end + end - it "should load all puppet manifests from all modules in the specified environment" do - @module1 = mk_module(@modulebase1, "one") - @module2 = mk_module(@modulebase2, "two") + it "should load all puppet manifests from all modules in the specified environment" do + @module1 = mk_module(@modulebase1, "one") + @module2 = mk_module(@modulebase2, "two") - mk_manifests(@modulebase1, @module1, "puppet", %w{a b}) - mk_manifests(@modulebase2, @module2, "puppet", %w{c d}) + mk_manifests(@modulebase1, @module1, "puppet", %w{a b}) + mk_manifests(@modulebase2, @module2, "puppet", %w{c d}) - @loader.import_all + loader.import_all - @loader.environment.known_resource_types.hostclass("one::a").should be_instance_of(Puppet::Resource::Type) - @loader.environment.known_resource_types.hostclass("one::b").should be_instance_of(Puppet::Resource::Type) - @loader.environment.known_resource_types.hostclass("two::c").should be_instance_of(Puppet::Resource::Type) - @loader.environment.known_resource_types.hostclass("two::d").should be_instance_of(Puppet::Resource::Type) - end + loader.environment.known_resource_types.hostclass("one::a").should be_instance_of(Puppet::Resource::Type) + loader.environment.known_resource_types.hostclass("one::b").should be_instance_of(Puppet::Resource::Type) + loader.environment.known_resource_types.hostclass("two::c").should be_instance_of(Puppet::Resource::Type) + loader.environment.known_resource_types.hostclass("two::d").should be_instance_of(Puppet::Resource::Type) + end - it "should load all ruby manifests from all modules in the specified environment" do - Puppet.expects(:deprecation_warning).at_least(1) + it "should load all ruby manifests from all modules in the specified environment" do + Puppet.expects(:deprecation_warning).at_least(1) - @module1 = mk_module(@modulebase1, "one") - @module2 = mk_module(@modulebase2, "two") + @module1 = mk_module(@modulebase1, "one") + @module2 = mk_module(@modulebase2, "two") - mk_manifests(@modulebase1, @module1, "ruby", %w{a b}) - mk_manifests(@modulebase2, @module2, "ruby", %w{c d}) + mk_manifests(@modulebase1, @module1, "ruby", %w{a b}) + mk_manifests(@modulebase2, @module2, "ruby", %w{c d}) - @loader.import_all + loader.import_all - @loader.environment.known_resource_types.hostclass("one::a").should be_instance_of(Puppet::Resource::Type) - @loader.environment.known_resource_types.hostclass("one::b").should be_instance_of(Puppet::Resource::Type) - @loader.environment.known_resource_types.hostclass("two::c").should be_instance_of(Puppet::Resource::Type) - @loader.environment.known_resource_types.hostclass("two::d").should be_instance_of(Puppet::Resource::Type) - end + loader.environment.known_resource_types.hostclass("one::a").should be_instance_of(Puppet::Resource::Type) + loader.environment.known_resource_types.hostclass("one::b").should be_instance_of(Puppet::Resource::Type) + loader.environment.known_resource_types.hostclass("two::c").should be_instance_of(Puppet::Resource::Type) + loader.environment.known_resource_types.hostclass("two::d").should be_instance_of(Puppet::Resource::Type) + end - it "should not load manifests from duplicate modules later in the module path" do - @module1 = mk_module(@modulebase1, "one") + it "should not load manifests from duplicate modules later in the module path" do + @module1 = mk_module(@modulebase1, "one") - # duplicate - @module2 = mk_module(@modulebase2, "one") + # duplicate + @module2 = mk_module(@modulebase2, "one") - mk_manifests(@modulebase1, @module1, "puppet", %w{a}) - mk_manifests(@modulebase2, @module2, "puppet", %w{c}) + mk_manifests(@modulebase1, @module1, "puppet", %w{a}) + mk_manifests(@modulebase2, @module2, "puppet", %w{c}) - @loader.import_all + loader.import_all - @loader.environment.known_resource_types.hostclass("one::c").should be_nil - end + loader.environment.known_resource_types.hostclass("one::c").should be_nil + end - it "should load manifests from subdirectories" do - @module1 = mk_module(@modulebase1, "one") + it "should load manifests from subdirectories" do + @module1 = mk_module(@modulebase1, "one") - mk_manifests(@modulebase1, @module1, "puppet", %w{a a/b a/b/c}) + mk_manifests(@modulebase1, @module1, "puppet", %w{a a/b a/b/c}) - @loader.import_all + loader.import_all - @loader.environment.known_resource_types.hostclass("one::a::b").should be_instance_of(Puppet::Resource::Type) - @loader.environment.known_resource_types.hostclass("one::a::b::c").should be_instance_of(Puppet::Resource::Type) - end + loader.environment.known_resource_types.hostclass("one::a::b").should be_instance_of(Puppet::Resource::Type) + loader.environment.known_resource_types.hostclass("one::a::b::c").should be_instance_of(Puppet::Resource::Type) end - describe "when parsing a file" do - before do - @parser = Puppet::Parser::ParserFactory.parser(@loader.environment) - @parser.class.should == parser_class - @parser.stubs(:parse).returns(Puppet::Parser::AST::Hostclass.new('')) - @parser.stubs(:file=) - Puppet::Parser::ParserFactory.stubs(:parser).with(@loader.environment).returns @parser - end + it "should skip modules that don't have manifests" do + @module1 = mk_module(@modulebase1, "one") + @module2 = mk_module(@modulebase2, "two") + mk_manifests(@modulebase2, @module2, "ruby", %w{c d}) - it "should create a new parser instance for each file using the current environment" do - Puppet::Parser::ParserFactory.expects(:parser).with(@loader.environment).returns @parser - @loader.parse_file("/my/file") - end + loader.import_all - it "should assign the parser its file and parse" do - @parser.expects(:file=).with("/my/file") - @parser.expects(:parse).returns(Puppet::Parser::AST::Hostclass.new('')) - @loader.parse_file("/my/file") - end + loader.environment.known_resource_types.hostclass("one::a").should be_nil + loader.environment.known_resource_types.hostclass("two::c").should be_instance_of(Puppet::Resource::Type) + loader.environment.known_resource_types.hostclass("two::d").should be_instance_of(Puppet::Resource::Type) end + end - it "should be able to add classes to the current resource type collection" do - file = tmpfile("simple_file.pp") - File.open(file, "w") { |f| f.puts "class foo {}" } - @loader.import(File.basename(file), File.dirname(file)) + describe "when parsing a file" do + it "should create a new parser instance for each file using the current environment" do + parser = stub 'Parser', :file= => nil, :parse => empty_hostclass - @loader.known_resource_types.hostclass("foo").should be_instance_of(Puppet::Resource::Type) - end + Puppet::Parser::ParserFactory.expects(:parser).twice.with(loader.environment).returns(parser) - describe "when deciding where to look for files" do - { 'foo' => ['foo'], - 'foo::bar' => ['foo/bar', 'foo'], - 'foo::bar::baz' => ['foo/bar/baz', 'foo/bar', 'foo'] - }.each do |fqname, expected_paths| - it "should look for #{fqname.inspect} in #{expected_paths.inspect}" do - @loader.instance_eval { name2files(fqname) }.should == expected_paths - end - end - end - end - describe 'when using the classic parser' do - before :each do - Puppet[:parser] = 'current' + loader.parse_file("/my/file") + loader.parse_file("/my/other_file") end - it_should_behave_like 'the typeloader' do - let(:parser_class) { Puppet::Parser::Parser} - let(:other_parser_class) { Puppet::Parser::EParserAdapter} + + it "should assign the parser its file and parse" do + parser = mock 'parser' + + Puppet::Parser::ParserFactory.expects(:parser).with(loader.environment).returns(parser) + parser.expects(:file=).with("/my/file") + parser.expects(:parse).returns(empty_hostclass) + + loader.parse_file("/my/file") end end - describe 'when using the future parser' do - before :each do - Puppet[:parser] = 'future' - end - it_should_behave_like 'the typeloader' do - let(:parser_class) { Puppet::Parser::EParserAdapter} - let(:other_parser_class) { Puppet::Parser::Parser} - end + + it "should be able to add classes to the current resource type collection" do + file = tmpfile("simple_file.pp") + File.open(file, "w") { |f| f.puts "class foo {}" } + loader.import(File.basename(file), File.dirname(file)) + + loader.known_resource_types.hostclass("foo").should be_instance_of(Puppet::Resource::Type) end end diff --git a/spec/unit/pops/binder/binder_spec.rb b/spec/unit/pops/binder/binder_spec.rb new file mode 100644 index 000000000..dfcf633e1 --- /dev/null +++ b/spec/unit/pops/binder/binder_spec.rb @@ -0,0 +1,62 @@ +require 'spec_helper' +require 'puppet/pops' + +module BinderSpecModule + def factory() + Puppet::Pops::Binder::BindingsFactory + end + + def injector(binder) + Puppet::Pops::Binder::Injector.new(binder) + end + + def binder() + Puppet::Pops::Binder::Binder.new() + end + + def test_layer_with_empty_bindings + factory.named_layer('test-layer', factory.named_bindings('test').model) + end +end + +describe 'Binder' do + include BinderSpecModule + + context 'when defining categories' do + it 'redefinition is not allowed' do + expect do + b = binder() + b.define_categories(factory.categories([])) + b.define_categories(factory.categories([])) + end.to raise_error(/Cannot redefine/) + end + end + + context 'when defining layers' do + it 'they must be defined after categories' do + expect do + binder().define_layers(factory.layered_bindings(test_layer_with_empty_bindings)) + end.to raise_error(/Categories must be defined first/) + end + + it 'redefinition is not allowed' do + expect do + b = binder() + b.define_categories(factory.categories([])) + b.define_layers(factory.layered_bindings(test_layer_with_empty_bindings)) + b.define_layers(factory.layered_bindings(test_layer_with_empty_bindings)) + end.to raise_error(/Cannot redefine its content/) + end + end + + context 'when defining categories and layers' do + it 'a binder should report being configured when both categories and layers have been defined' do + b = binder() + b.configured?().should == false + b.define_categories(factory.categories([])) + b.configured?().should == false + b.define_layers(factory.layered_bindings(test_layer_with_empty_bindings)) + b.configured?().should == true + end + end +end
\ No newline at end of file diff --git a/spec/unit/pops/binder/bindings_checker_spec.rb b/spec/unit/pops/binder/bindings_checker_spec.rb new file mode 100644 index 000000000..dc3934c6c --- /dev/null +++ b/spec/unit/pops/binder/bindings_checker_spec.rb @@ -0,0 +1,196 @@ +require 'spec_helper' +require 'puppet/pops' +require 'puppet_spec/pops' + +describe 'The bindings checker' do + + include PuppetSpec::Pops + + Issues = Puppet::Pops::Binder::BinderIssues + Bindings = Puppet::Pops::Binder::Bindings + TypeFactory = Puppet::Pops::Types::TypeFactory + let (:acceptor) { Puppet::Pops::Validation::Acceptor.new() } + + let (:binding) { Bindings::Binding.new() } + + let (:ok_binding) { + b = Bindings::Binding.new() + b.producer = Bindings::ConstantProducerDescriptor.new() + b.producer.value = 'some value' + b.type = TypeFactory.string() + b + } + + def validate(binding) + Puppet::Pops::Binder::BindingsValidatorFactory.new().validator(acceptor).validate(binding) + end + + def bindings(*args) + b = Bindings::Bindings.new() + b.bindings = args + b + end + + def named_bindings(name, *args) + b = Bindings::NamedBindings.new() + b.name = name + b.bindings = args + b + end + + def category(name, value) + b = Bindings::Category.new() + b.categorization = name + b.value = value + b + end + + def categorized_bindings(bindings, *predicates) + b = Bindings::CategorizedBindings.new() + b.bindings = bindings + b.predicates = predicates + b + end + + def layer(name, *bindings) + l = Bindings::NamedLayer.new() + l.name = name + l.bindings = bindings + l + end + + def layered_bindings(*layers) + b = Bindings::LayeredBindings.new() + b.layers = layers + b + end + + def array_multibinding() + b = Bindings::Multibinding.new() + b.producer = Bindings::ArrayMultibindProducerDescriptor.new() + b.type = TypeFactory.array_of_data() + b + end + + def bad_array_multibinding() + b = array_multibinding() + b.type = TypeFactory.hash_of_data() # intentionally wrong! + b + end + + def hash_multibinding() + b = Bindings::Multibinding.new() + b.producer = Bindings::HashMultibindProducerDescriptor.new() + b.type = TypeFactory.hash_of_data() + b + end + + def bad_hash_multibinding() + b = hash_multibinding() + b.type = TypeFactory.array_of_data() # intentionally wrong! + b + end + + it 'should complain about missing producer and type' do + validate(binding()) + acceptor.should have_issue(Issues::MISSING_PRODUCER) + acceptor.should have_issue(Issues::MISSING_TYPE) + end + + context 'when checking array multibinding' do + it 'should complain about non array producers' do + validate(bad_array_multibinding()) + acceptor.should have_issue(Issues::MULTIBIND_INCOMPATIBLE_TYPE) + end + end + + context 'when checking hash multibinding' do + it 'should complain about non hash producers' do + validate(bad_hash_multibinding()) + acceptor.should have_issue(Issues::MULTIBIND_INCOMPATIBLE_TYPE) + end + end + + context 'when checking bindings' do + it 'should not accept zero bindings' do + validate(bindings()) + acceptor.should have_issue(Issues::MISSING_BINDINGS) + end + + it 'should accept non-zero bindings' do + validate(bindings(ok_binding)) + acceptor.errors_or_warnings?.should() == false + end + + it 'should check contained bindings' do + validate(bindings(bad_array_multibinding())) + acceptor.should have_issue(Issues::MULTIBIND_INCOMPATIBLE_TYPE) + end + end + + context 'when checking named bindings' do + it 'should accept named bindings' do + validate(named_bindings('garfield', ok_binding)) + acceptor.errors_or_warnings?.should() == false + end + + it 'should not accept unnamed bindings' do + validate(named_bindings(nil, ok_binding)) + acceptor.should have_issue(Issues::MISSING_BINDINGS_NAME) + end + + it 'should do generic bindings check' do + validate(named_bindings('garfield')) + acceptor.should have_issue(Issues::MISSING_BINDINGS) + end + end + + context 'when checking categorized bindings' do + it 'should accept non-zero predicates' do + validate(categorized_bindings([ok_binding], category('foo', 'bar'))) + acceptor.errors_or_warnings?.should() == false + end + + it 'should not accept zero predicates' do + validate(categorized_bindings([ok_binding])) + acceptor.should have_issue(Issues::MISSING_PREDICATES) + end + + it 'should not accept predicates that has no categorization' do + validate(categorized_bindings([ok_binding], category(nil, 'bar'))) + acceptor.should have_issue(Issues::MISSING_CATEGORIZATION) + end + + it 'should not accept predicates that has no value' do + validate(categorized_bindings([ok_binding], category('foo', nil))) + acceptor.should have_issue(Issues::MISSING_CATEGORY_VALUE) + end + + it 'should do generic bindings check' do + validate(categorized_bindings([], category('foo', 'bar'))) + acceptor.should have_issue(Issues::MISSING_BINDINGS) + end + end + + context 'when checking layered bindings' do + it 'should not accept zero layers' do + validate(layered_bindings()) + acceptor.should have_issue(Issues::MISSING_LAYERS) + end + + it 'should accept non-zero layers' do + validate(layered_bindings(layer('foo', named_bindings('bar', ok_binding)))) + acceptor.errors_or_warnings?.should() == false + end + + it 'should not accept unnamed layers' do + validate(layered_bindings(layer(nil, named_bindings('bar', ok_binding)))) + acceptor.should have_issue(Issues::MISSING_LAYER_NAME) + end + + it 'should accept layers without bindings' do + validate(layered_bindings(layer('foo'))) + acceptor.should_not have_issue(Issues::MISSING_BINDINGS_IN_LAYER) + end + end +end diff --git a/spec/unit/pops/binder/bindings_composer_spec.rb b/spec/unit/pops/binder/bindings_composer_spec.rb new file mode 100644 index 000000000..a0986d4dd --- /dev/null +++ b/spec/unit/pops/binder/bindings_composer_spec.rb @@ -0,0 +1,89 @@ +require 'spec_helper' +require 'puppet_spec/pops' + +describe 'BinderComposer' do + include PuppetSpec::Pops + + def config_dir(config_name) + my_fixture(config_name) + end + + let(:acceptor) { Puppet::Pops::Validation::Acceptor.new() } + let(:diag) { Puppet::Pops::Binder::Config::DiagnosticProducer.new(acceptor) } + let(:issues) { Puppet::Pops::Binder::Config::Issues } + let(:node) { Puppet::Node.new('localhost') } + let(:compiler) { Puppet::Parser::Compiler.new(node)} + let(:scope) { Puppet::Parser::Scope.new(compiler) } + let(:parser) { Puppet::Pops::Parser::Parser.new() } + let(:factory) { Puppet::Pops::Binder::BindingsFactory } + + before(:each) do + Puppet[:binder] = true + end + + it 'should load default config if no config file exists' do + diagnostics = diag + composer = Puppet::Pops::Binder::BindingsComposer.new() + composer.compose(scope) + end + + context "when loading a complete configuration with modules" do + let(:config_directory) { config_dir('ok') } + + it 'should load everything without errors' do + Puppet.settings[:confdir] = config_directory + Puppet.settings[:modulepath] = File.join(config_directory, 'modules') + + diagnostics = diag + composer = Puppet::Pops::Binder::BindingsComposer.new() + the_scope = scope + the_scope['fqdn'] = 'localhost' + the_scope['environment'] = 'production' + layered_bindings = composer.compose(scope) + # puts Puppet::Pops::Binder::BindingsModelDumper.new().dump(layered_bindings) + binder = Puppet::Pops::Binder::Binder.new() + # TODO: this is cheating, the categories should come from the composer/config + binder.define_categories(factory.categories([['node', 'localhost'], ['environment', 'production']])) + binder.define_layers(layered_bindings) + injector = Puppet::Pops::Binder::Injector.new(binder) + + expect(injector.lookup(scope, 'awesome_x')).to be == 'golden' + expect(injector.lookup(scope, 'good_x')).to be == 'golden' + expect(injector.lookup(scope, 'rotten_x')).to be == nil + expect(injector.lookup(scope, 'the_meaning_of_life')).to be == 42 + expect(injector.lookup(scope, 'has_funny_hat')).to be == 'the pope' + expect(injector.lookup(scope, 'all your base')).to be == 'are belong to us' + expect(injector.lookup(scope, 'env_meaning_of_life')).to be == 'production thinks it is 42' + expect(injector.lookup(scope, '::quick::brown::fox')).to be == 'echo: quick brown fox' + expect(injector.lookup(scope, 'echo::common')).to be == 'echo... awesome/common' + expect(injector.lookup(scope, 'echo::localhost')).to be == 'echo... awesome/localhost' + end + end + + context "when loading a configuration with hiera1 hiera.yaml" do + let(:config_directory) { config_dir('hiera1config') } + + it 'should load without errors by skipping the hiera.yaml' do + Puppet.settings[:confdir] = config_directory + Puppet.settings[:modulepath] = File.join(config_directory, 'modules') + + diagnostics = diag + composer = Puppet::Pops::Binder::BindingsComposer.new() + the_scope = scope + the_scope['fqdn'] = 'localhost' + the_scope['environment'] = 'production' + layered_bindings = composer.compose(scope) + # puts Puppet::Pops::Binder::BindingsModelDumper.new().dump(layered_bindings) + binder = Puppet::Pops::Binder::Binder.new() + # TODO: this is cheating, the categories should come from the composer/config + binder.define_categories(factory.categories([['node', 'localhost'], ['environment', 'production']])) + binder.define_layers(layered_bindings) + injector = Puppet::Pops::Binder::Injector.new(binder) + + expect(injector.lookup(scope, 'the_meaning_of_life')).to be == 300 + end + end + + # TODO: test error conditions (see BinderConfigChecker for what to test) + +end
\ No newline at end of file diff --git a/spec/unit/pops/binder/bindings_validator_factory_spec.rb b/spec/unit/pops/binder/bindings_validator_factory_spec.rb new file mode 100644 index 000000000..c90da799c --- /dev/null +++ b/spec/unit/pops/binder/bindings_validator_factory_spec.rb @@ -0,0 +1,18 @@ +require 'spec_helper' +require 'puppet/pops' + +describe 'The bindings validator factory' do + let(:factory) { Puppet::Pops::Binder::BindingsValidatorFactory.new() } + + it 'should instantiate a BindingsValidatorFactory' do + factory.class.should == Puppet::Pops::Binder::BindingsValidatorFactory + end + + it 'should produce label_provider of class BindingsLabelProvider' do + factory.label_provider.class.should == Puppet::Pops::Binder::BindingsLabelProvider + end + + it 'should produce validator of class BindingsChecker' do + factory.validator(Puppet::Pops::Validation::Acceptor.new()).class.should == Puppet::Pops::Binder::BindingsChecker + end +end diff --git a/spec/unit/pops/binder/config/binder_config_spec.rb b/spec/unit/pops/binder/config/binder_config_spec.rb new file mode 100644 index 000000000..dcd626737 --- /dev/null +++ b/spec/unit/pops/binder/config/binder_config_spec.rb @@ -0,0 +1,48 @@ +require 'spec_helper' +require 'puppet/pops' +require 'puppet_spec/pops' + +describe 'BinderConfig' do + include PuppetSpec::Pops + + let(:acceptor) { Puppet::Pops::Validation::Acceptor.new() } + let(:diag) { Puppet::Pops::Binder::Config::DiagnosticProducer.new(acceptor) } + let(:issues) { Puppet::Pops::Binder::Config::Issues } + + it 'should load default config if no config file exists' do + diagnostics = diag + config = Puppet::Pops::Binder::Config::BinderConfig.new(diagnostics) + expect(acceptor.errors?()).to be == false + expect(config.layering_config[0]['name']).to be == 'site' + expect(config.layering_config[0]['include']).to be == ['confdir-hiera:/', 'confdir:/default?optional'] + expect(config.layering_config[1]['name']).to be == 'modules' + expect(config.layering_config[1]['include']).to be == ['module-hiera:/*/', 'module:/*::default'] + + expect(config.categorization.is_a?(Array)).to be == true + expect(config.categorization.size).to be == 4 + expect(config.categorization[0][0]).to be == 'node' + expect(config.categorization[1][0]).to be == 'osfamily' + expect(config.categorization[2][0]).to be == 'environment' + expect(config.categorization[3][0]).to be == 'common' + end + + it 'should load binder_config.yaml if it exists in confdir)' do + Puppet::Pops::Binder::Config::BinderConfig.any_instance.stubs(:confdir).returns(my_fixture("/ok/")) + config = Puppet::Pops::Binder::Config::BinderConfig.new(diag) + expect(acceptor.errors?()).to be == false + expect(config.layering_config[0]['name']).to be == 'site' + expect(config.layering_config[0]['include']).to be == 'confdir-hiera:/' + expect(config.layering_config[1]['name']).to be == 'modules' + expect(config.layering_config[1]['include']).to be == 'module-hiera:/*/' + expect(config.layering_config[1]['exclude']).to be == 'module-hiera:/bad/' + + expect(config.categorization.is_a?(Array)).to be == true + expect(config.categorization.size).to be == 3 + expect(config.categorization[0][0]).to be == 'node' + expect(config.categorization[1][0]).to be == 'environment' + expect(config.categorization[2][0]).to be == 'common' + end + + # TODO: test error conditions (see BinderConfigChecker for what to test) + +end
\ No newline at end of file diff --git a/spec/unit/pops/binder/hiera2/bindings_provider_spec.rb b/spec/unit/pops/binder/hiera2/bindings_provider_spec.rb new file mode 100644 index 000000000..edb4c7df2 --- /dev/null +++ b/spec/unit/pops/binder/hiera2/bindings_provider_spec.rb @@ -0,0 +1,74 @@ +require 'spec_helper' +require 'puppet/pops' +require 'puppet_spec/pops' + +describe 'The hiera2 bindings provider' do + + include PuppetSpec::Pops + + def config_dir(config_name) + File.dirname(my_fixture("#{config_name}/hiera.yaml")) + end + + before(:each) do + Puppet[:binder] = true + end + + context 'when loading ok bindings' do + + let(:node) { 'node.example.com' } + let(:acceptor) { Puppet::Pops::Validation::Acceptor.new() } + let(:scope) { s = Puppet::Parser::Scope.new_for_test_harness(node); s['a'] = '42'; s['node'] = node; s } + let(:module_dir) { config_dir('ok') } + let(:node_binder) { b = Puppet::Pops::Binder::Binder.new(); b.define_categories(Puppet::Pops::Binder::BindingsFactory.categories(['node', node])); b } + let(:bindings) { Puppet::Pops::Binder::Hiera2::BindingsProvider.new('test', module_dir, acceptor).load_bindings(scope) } + let(:test_layer_with_bindings) { bindings } + + it 'should load and validate OK bindings' do + Puppet::Pops::Binder::BindingsValidatorFactory.new().validator(acceptor).validate(bindings) + acceptor.errors_or_warnings?.should() == false + end + + it 'should contain the expected effective categories' do + bindings.effective_categories.categories.collect {|c| [c.categorization, c.value] }.should == [['node', 'node.example.com']] + end + + it 'should produce the expected bindings model' do + bindings.class.should() == Puppet::Pops::Binder::Bindings::ContributedBindings + bindings.bindings.bindings.each do |cat| + cat.class.should() == Puppet::Pops::Binder::Bindings::CategorizedBindings + cat.predicates.length.should() == 1 + cat.predicates[0].categorization.should() == 'node' + cat.predicates[0].value.should() == node + cat.bindings.each do |b| + b.class.should() == Puppet::Pops::Binder::Bindings::Binding + ['a_number', 'a_string', 'an_eval', 'an_eval2', 'a_json_number', 'a_json_string', 'a_json_eval', + 'a_json_eval2', 'a_json_hash', 'a_json_array'].index(b.name).should() >= 0 + b.producer.class.should() == Puppet::Pops::Binder::Bindings::EvaluatingProducerDescriptor if b.name == 'an_eval' + end + end + end + + it 'should make the injector lookup expected constants' do + node_binder.define_layers(Puppet::Pops::Binder::BindingsFactory.layered_bindings(test_layer_with_bindings)) + injector = Puppet::Pops::Binder::Injector.new(node_binder) + + injector.lookup(scope, 'a_number').should == 42 + injector.lookup(scope, 'a_string').should == 'forty two' + injector.lookup(scope, 'a_json_number').should == 142 + injector.lookup(scope, 'a_json_string').should == 'one hundred and forty two' + expect(injector.lookup(scope, "a_json_array")).to be == ["a", "b", 100] + expect(injector.lookup(scope, "a_json_hash")).to be == {"a" => 1, "b" => 2} + end + + it 'should make the injector lookup and evaluate expressions' do + node_binder.define_layers(Puppet::Pops::Binder::BindingsFactory.layered_bindings(test_layer_with_bindings)) + injector = Puppet::Pops::Binder::Injector.new(node_binder) + + injector.lookup(scope, 'an_eval').should == 'the answer from "yaml" is 42.' + injector.lookup(scope, 'an_eval2').should == "the answer\nfrom \\\"yaml\\\" is 42 and $a" + injector.lookup(scope, 'a_json_eval').should == 'the answer from "json" is 42 and ${a}.' + injector.lookup(scope, 'a_json_eval2').should == "the answer\nfrom \\\"json\\\" is 42 and $a" + end + end +end diff --git a/spec/unit/pops/binder/hiera2/config_spec.rb b/spec/unit/pops/binder/hiera2/config_spec.rb new file mode 100644 index 000000000..970e72a9a --- /dev/null +++ b/spec/unit/pops/binder/hiera2/config_spec.rb @@ -0,0 +1,61 @@ +require 'spec_helper' +require 'puppet/pops' +require 'puppet_spec/pops' + +# A Backend class that doesn't implement the needed API +class Puppet::Pops::Binder::Hiera2::Bad_backend +end + +describe 'The hiera2 config' do + + include PuppetSpec::Pops + + let(:acceptor) { Puppet::Pops::Validation::Acceptor.new() } + let(:diag) { Puppet::Pops::Binder::Hiera2::DiagnosticProducer.new(acceptor) } + + def config_dir(config_name) + File.dirname(my_fixture("#{config_name}/hiera.yaml")) + end + + def test_config_issue(config_name, issue) + Puppet::Pops::Binder::Hiera2::Config.new(config_dir(config_name), diag) + acceptor.should have_issue(issue) + end + + it 'should load and validate OK configuration' do + Puppet::Pops::Binder::Hiera2::Config.new(config_dir('ok'), diag) + acceptor.errors_or_warnings?.should() == false + end + + it 'should report missing config file' do + Puppet::Pops::Binder::Hiera2::Config.new(File.dirname(my_fixture('missing/foo.txt')), diag) + acceptor.should have_issue(Puppet::Pops::Binder::Hiera2::Issues::CONFIG_FILE_NOT_FOUND) + end + + it 'should report when config is not a hash' do + test_config_issue('not_a_hash', Puppet::Pops::Binder::Hiera2::Issues::CONFIG_IS_NOT_HASH) + end + + it 'should report when config has syntax problems' do + if RUBY_VERSION.start_with?("1.8") + # Yes, it is a lobotomy or 2 short of a full brain... + # if a hash key is not in quotes it continues on the next line and gobbles what is there instead + # of reporting an error + test_config_issue('bad_syntax', Puppet::Pops::Binder::Hiera2::Issues::MISSING_HIERARCHY) + else + test_config_issue('bad_syntax', Puppet::Pops::Binder::Hiera2::Issues::CONFIG_FILE_SYNTAX_ERROR) + end + end + + it 'should report when config has no hierarchy defined' do + test_config_issue('no_hierarchy', Puppet::Pops::Binder::Hiera2::Issues::MISSING_HIERARCHY) + end + + it 'should report when config has no backends defined' do + test_config_issue('no_backends', Puppet::Pops::Binder::Hiera2::Issues::MISSING_BACKENDS) + end + + it 'should report when config hierarchy is malformed' do + test_config_issue('malformed_hierarchy', Puppet::Pops::Binder::Hiera2::Issues::CATEGORY_MUST_BE_THREE_ELEMENT_ARRAY) + end +end diff --git a/spec/unit/pops/binder/hiera2/yaml_backend_spec.rb b/spec/unit/pops/binder/hiera2/yaml_backend_spec.rb new file mode 100644 index 000000000..702819c9a --- /dev/null +++ b/spec/unit/pops/binder/hiera2/yaml_backend_spec.rb @@ -0,0 +1,33 @@ +require 'spec_helper' +require 'puppet/pops' + +describe "Hiera2 YAML backend" do + + include PuppetSpec::Files + + def fixture_dir(config_name) + my_fixture("#{config_name}") + end + + before(:all) do + Puppet[:binder] = true + require 'puppetx' + require 'puppet/pops/binder/hiera2/yaml_backend' + end + + after(:all) do + Puppet[:binder] = false + end + + it "returns the expected hash from a valid yaml file" do + Puppet::Pops::Binder::Hiera2::YamlBackend.new().read_data(fixture_dir("ok"), "common").should == {'brillig' => 'slithy'} + end + + it "returns an empty hash from an empty yaml file" do + Puppet::Pops::Binder::Hiera2::YamlBackend.new().read_data(fixture_dir("empty"), "common").should == {} + end + + it "returns an empty hash from an invalid yaml file" do + Puppet::Pops::Binder::Hiera2::YamlBackend.new().read_data(fixture_dir("invalid"), "common").should == {} + end +end diff --git a/spec/unit/pops/binder/injector_spec.rb b/spec/unit/pops/binder/injector_spec.rb new file mode 100644 index 000000000..966561ede --- /dev/null +++ b/spec/unit/pops/binder/injector_spec.rb @@ -0,0 +1,789 @@ +require 'spec_helper' +require 'puppet/pops' + +module InjectorSpecModule + def injector(binder) + Puppet::Pops::Binder::Injector.new(binder) + end + + def factory + Puppet::Pops::Binder::BindingsFactory + end + + def test_layer_with_empty_bindings + factory.named_layer('test-layer', factory.named_bindings('test').model) + end + + def test_layer_with_bindings(*bindings) + factory.named_layer('test-layer', *bindings) + end + + def null_scope() + nil + end + + def type_calculator + Puppet::Pops::Types::TypeCalculator + end + + def type_factory + Puppet::Pops::Types::TypeFactory + end + + # Returns a binder with the effective categories highest/test, node/kermit, environment/dev (and implicit 'common') + # + def binder_with_categories + b = Puppet::Pops::Binder::Binder.new() + b.define_categories(factory.categories(['highest', 'test', 'node', 'kermit', 'environment','dev'])) + b + end + + class TestDuck + end + + class Daffy < TestDuck + end + + + class AngryDuck < TestDuck + # Supports assisted inject, returning a Donald duck as the default impl of Duck + def self.inject(injector, scope, binding, *args) + Donald.new() + end + end + + class Donald < AngryDuck + end + + class ArneAnka < AngryDuck + attr_reader :label + + def initialize() + @label = 'A Swedish angry cartoon duck' + end + end + + class ScroogeMcDuck < TestDuck + attr_reader :fortune + + # Supports assisted inject, returning an ScroogeMcDuck with 1$ fortune or first arg in args + # Note that when injected (via instance producer, or implict assisted inject, the inject method + # always wins. + def self.inject(injector, scope, binding, *args) + self.new(args[0].nil? ? 1 : args[0]) + end + + def initialize(fortune) + @fortune = fortune + end + end + + class NamedDuck < TestDuck + attr_reader :name + def initialize(name) + @name = name + end + end + + # Test custom producer that on each produce returns a duck that is twice as rich as its predecessor + class ScroogeProducer < Puppet::Pops::Binder::Producers::Producer + attr_reader :next_capital + def initialize + @next_capital = 100 + end + def produce(scope) + ScroogeMcDuck.new(@next_capital *= 2) + end + end +end + +describe 'Injector' do + include InjectorSpecModule + + let(:bindings) { factory.named_bindings('test') } + let(:scope) { null_scope()} + let(:duck_type) { type_factory.ruby(InjectorSpecModule::TestDuck) } + + let(:binder) { Puppet::Pops::Binder::Binder.new()} + + let(:cbinder) do + b = Puppet::Pops::Binder::Binder.new() + b.define_categories(factory.categories([])) + b + end + + let(:lbinder) do + cbinder.define_layers(layered_bindings) + end + + + let(:layered_bindings) { factory.layered_bindings(test_layer_with_bindings(bindings.model)) } + #let(:xinjector) { Puppet::Pops::Binder::Injector.new(lbinder) } + + context 'When created' do + it 'should raise an error when given binder is not configured at all' do + expect { Puppet::Pops::Binder::Injector.new(binder()) }.to raise_error(/Given Binder is not configured/) + end + + it 'should raise an error if binder has categories, but is not completely configured' do + expect { Puppet::Pops::Binder::Injector.new(cbinder) }.to raise_error(/Given Binder is not configured/) + end + + it 'should not raise an error if binder is configured' do + lbinder.configured?().should == true # of something is very wrong + expect { injector(lbinder) }.to_not raise_error + end + + it 'should create an empty injector given an empty binder' do + expect { cbinder.define_layers(layered_bindings) }.to_not raise_exception + end + + it "should be possible to reference the TypeCalculator" do + injector(lbinder).type_calculator.is_a?(Puppet::Pops::Types::TypeCalculator).should == true + end + + it "should be possible to reference the KeyFactory" do + injector(lbinder).key_factory.is_a?(Puppet::Pops::Binder::KeyFactory).should == true + end + end + + context "When looking up objects" do + it 'lookup(scope, name) finds bound object of type Data with given name' do + bindings.bind().name('a_string').to('42') + injector(lbinder).lookup(scope, 'a_string').should == '42' + end + + context 'a block transforming the result can be given' do + it 'that transform a found value given scope and value' do + bindings.bind().name('a_string').to('42') + injector(lbinder).lookup(scope, 'a_string') {|zcope, val| val + '42' }.should == '4242' + end + + it 'that transform a found value given only value' do + bindings.bind().name('a_string').to('42') + injector(lbinder).lookup(scope, 'a_string') {|val| val + '42' }.should == '4242' + end + + it 'that produces a default value when entry is missing' do + bindings.bind().name('a_string').to('42') + injector(lbinder).lookup(scope, 'a_non_existing_string') {|val| val ? (raise Error, "Should not happen") : '4242' }.should == '4242' + end + end + + context "and class is not bound" do + it "assisted inject kicks in for classes with zero args constructor" do + duck_type = type_factory.ruby(InjectorSpecModule::Daffy) + injector = injector(lbinder) + injector.lookup(scope, duck_type).is_a?(InjectorSpecModule::Daffy).should == true + injector.lookup_producer(scope, duck_type).produce(scope).is_a?(InjectorSpecModule::Daffy).should == true + end + + it "assisted inject produces same instance on lookup but not on lookup producer" do + duck_type = type_factory.ruby(InjectorSpecModule::Daffy) + injector = injector(lbinder) + d1 = injector.lookup(scope, duck_type) + d2 = injector.lookup(scope, duck_type) + d1.equal?(d2).should == true + + d1 = injector.lookup_producer(scope, duck_type).produce(scope) + d2 = injector.lookup_producer(scope, duck_type).produce(scope) + d1.equal?(d2).should == false + end + + it "assisted inject kicks in for classes with a class inject method" do + duck_type = type_factory.ruby(InjectorSpecModule::ScroogeMcDuck) + injector = injector(lbinder) + # Do not pass any arguments, the ScroogeMcDuck :inject method should pick 1 by default + # This tests zero args passed + injector.lookup(scope, duck_type).fortune.should == 1 + injector.lookup_producer(scope, duck_type).produce(scope).fortune.should == 1 + end + + it "assisted inject selects the inject method if it exists over a zero args constructor" do + injector = injector(lbinder) + duck_type = type_factory.ruby(InjectorSpecModule::AngryDuck) + injector.lookup(scope, duck_type).is_a?(InjectorSpecModule::Donald).should == true + injector.lookup_producer(scope, duck_type).produce(scope).is_a?(InjectorSpecModule::Donald).should == true + end + + it "assisted inject selects the zero args constructor if injector is from a superclass" do + injector = injector(lbinder) + duck_type = type_factory.ruby(InjectorSpecModule::ArneAnka) + injector.lookup(scope, duck_type).is_a?(InjectorSpecModule::ArneAnka).should == true + injector.lookup_producer(scope, duck_type).produce(scope).is_a?(InjectorSpecModule::ArneAnka).should == true + end + end + + context 'and conditionals are in use' do + let(:binder) { binder_with_categories()} + let(:lbinder) { binder.define_layers(layered_bindings) } + + it "should be possible to shadow a bound value in a higher precedented category" do + bindings.bind().name('a_string').to('42') + bindings.when_in_category('environment', 'dev').bind().name('a_string').to('43') + bindings.when_in_category('node', 'kermit').bind().name('a_string').to('being green') + injector(lbinder).lookup(scope,'a_string').should == 'being green' + end + + it "shadowing should not happen when not in a category" do + bindings.bind().name('a_string').to('42') + bindings.when_in_category('environment', 'dev').bind().name('a_string').to('43') + bindings.when_in_category('node', 'piggy').bind().name('a_string').to('being green') + injector(lbinder).lookup(scope,'a_string').should == '43' + end + + it "multiple predicates makes binding more specific" do + bindings.bind().name('a_string').to('42') + bindings.when_in_category('environment', 'dev').bind().name('a_string').to('43') + bindings.when_in_category('node', 'kermit').bind().name('a_string').to('being green') + bindings.when_in_categories({'node'=>'kermit', 'environment'=>'dev'}).bind().name('a_string').to('being dev green') + injector(lbinder).lookup(scope,'a_string').should == 'being dev green' + end + + it "multiple predicates makes binding more specific, but not more specific than higher precedence" do + bindings.bind().name('a_string').to('42') + bindings.when_in_category('environment', 'dev').bind().name('a_string').to('43') + bindings.when_in_category('node', 'kermit').bind().name('a_string').to('being green') + bindings.when_in_categories({'node'=>'kermit', 'environment'=>'dev'}).bind().name('a_string').to('being dev green') + bindings.when_in_category('highest', 'test').bind().name('a_string').to('bazinga') + injector(lbinder).lookup(scope,'a_string').should == 'bazinga' + end + end + + context "and multiple layers are in use" do + let(:binder) { binder_with_categories()} + + it "a higher layer shadows anything in a lower layer" do + bindings1 = factory.named_bindings('test1') + bindings1.when_in_category("highest", "test").bind().name('a_string').to('bad stuff') + lower_layer = factory.named_layer('lower-layer', bindings1.model) + + bindings2 = factory.named_bindings('test2') + bindings2.bind().name('a_string').to('good stuff') + higher_layer = factory.named_layer('higher-layer', bindings2.model) + + binder.define_layers(factory.layered_bindings(higher_layer, lower_layer)) + injector = injector(binder) + injector.lookup(scope,'a_string').should == 'good stuff' + end + end + + context "and dealing with Data types" do + let(:binder) { binder_with_categories()} + let(:lbinder) { binder.define_layers(layered_bindings) } + + it "should treat all data as same type w.r.t. key" do + bindings.bind().name('a_string').to('42') + bindings.bind().name('an_int').to(43) + bindings.bind().name('a_float').to(3.14) + bindings.bind().name('a_boolean').to(true) + bindings.bind().name('an_array').to([1,2,3]) + bindings.bind().name('a_hash').to({'a'=>1,'b'=>2,'c'=>3}) + + injector = injector(lbinder) + injector.lookup(scope,'a_string').should == '42' + injector.lookup(scope,'an_int').should == 43 + injector.lookup(scope,'a_float').should == 3.14 + injector.lookup(scope,'a_boolean').should == true + injector.lookup(scope,'an_array').should == [1,2,3] + injector.lookup(scope,'a_hash').should == {'a'=>1,'b'=>2,'c'=>3} + end + + it "should provide type-safe lookup of given type/name" do + bindings.bind().string().name('a_string').to('42') + bindings.bind().integer().name('an_int').to(43) + bindings.bind().float().name('a_float').to(3.14) + bindings.bind().boolean().name('a_boolean').to(true) + bindings.bind().array_of_data().name('an_array').to([1,2,3]) + bindings.bind().hash_of_data().name('a_hash').to({'a'=>1,'b'=>2,'c'=>3}) + + injector = injector(lbinder) + + # Check lookup using implied Data type + injector.lookup(scope,'a_string').should == '42' + injector.lookup(scope,'an_int').should == 43 + injector.lookup(scope,'a_float').should == 3.14 + injector.lookup(scope,'a_boolean').should == true + injector.lookup(scope,'an_array').should == [1,2,3] + injector.lookup(scope,'a_hash').should == {'a'=>1,'b'=>2,'c'=>3} + + # Check lookup using expected type + injector.lookup(scope,type_factory.string(), 'a_string').should == '42' + injector.lookup(scope,type_factory.integer(), 'an_int').should == 43 + injector.lookup(scope,type_factory.float(),'a_float').should == 3.14 + injector.lookup(scope,type_factory.boolean(),'a_boolean').should == true + injector.lookup(scope,type_factory.array_of_data(),'an_array').should == [1,2,3] + injector.lookup(scope,type_factory.hash_of_data(),'a_hash').should == {'a'=>1,'b'=>2,'c'=>3} + + # Check lookup using wrong type + expect { injector.lookup(scope,type_factory.integer(), 'a_string')}.to raise_error(/Type error/) + expect { injector.lookup(scope,type_factory.string(), 'an_int')}.to raise_error(/Type error/) + expect { injector.lookup(scope,type_factory.string(),'a_float')}.to raise_error(/Type error/) + expect { injector.lookup(scope,type_factory.string(),'a_boolean')}.to raise_error(/Type error/) + expect { injector.lookup(scope,type_factory.string(),'an_array')}.to raise_error(/Type error/) + expect { injector.lookup(scope,type_factory.string(),'a_hash')}.to raise_error(/Type error/) + end + end + end + + context "When looking up producer" do + it 'the value is produced by calling produce(scope)' do + bindings.bind().name('a_string').to('42') + injector(lbinder).lookup_producer(scope, 'a_string').produce(scope).should == '42' + end + + context 'a block transforming the result can be given' do + it 'that transform a found value given scope and producer' do + bindings.bind().name('a_string').to('42') + injector(lbinder).lookup_producer(scope, 'a_string') {|zcope, p| p.produce(zcope) + '42' }.should == '4242' + end + + it 'that transform a found value given only producer' do + bindings.bind().name('a_string').to('42') + injector(lbinder).lookup_producer(scope, 'a_string') {|p| p.produce(scope) + '42' }.should == '4242' + end + + it 'that can produce a default value when entry is not found' do + bindings.bind().name('a_string').to('42') + injector(lbinder).lookup_producer(scope, 'a_non_existing_string') {|p| p ? (raise Error,"Should not happen") : '4242' }.should == '4242' + end + end + end + + context "When dealing with singleton vs. non singleton" do + it "should produce the same instance when producer is a singleton" do + bindings.bind().name('a_string').to('42') + injector = injector(lbinder) + a = injector.lookup(scope, 'a_string') + b = injector.lookup(scope, 'a_string') + a.equal?(b).should == true + end + + it "should produce different instances when producer is a non singleton producer" do + bindings.bind().name('a_string').to_series_of('42') + injector = injector(lbinder) + a = injector.lookup(scope, 'a_string') + b = injector.lookup(scope, 'a_string') + a.should == '42' + b.should == '42' + a.equal?(b).should == false + end + end + + context "When using the lookup producer" do + it "should lookup again to produce a value" do + bindings.bind().name('a_string').to_lookup_of('another_string') + bindings.bind().name('another_string').to('hello') + injector(lbinder).lookup(scope, 'a_string').should == 'hello' + end + + it "should produce nil if looked up key does not exist" do + bindings.bind().name('a_string').to_lookup_of('non_existing') + injector(lbinder).lookup(scope, 'a_string').should == nil + end + + it "should report an error if lookup loop is detected" do + bindings.bind().name('a_string').to_lookup_of('a_string') + expect { injector(lbinder).lookup(scope, 'a_string') }.to raise_error(/Lookup loop/) + end + end + + context "When using the hash lookup producer" do + it "should lookup a key in looked up hash" do + data_hash = type_factory.hash_of_data() + bindings.bind().name('a_string').to_hash_lookup_of(data_hash, 'a_hash', 'huey') + bindings.bind().name('a_hash').to({'huey' => 'red', 'dewey' => 'blue', 'louie' => 'green'}) + injector(lbinder).lookup(scope, 'a_string').should == 'red' + end + + it "should produce nil if looked up entry does not exist" do + data_hash = type_factory.hash_of_data() + bindings.bind().name('a_string').to_hash_lookup_of(data_hash, 'non_existing_entry', 'huey') + bindings.bind().name('a_hash').to({'huey' => 'red', 'dewey' => 'blue', 'louie' => 'green'}) + injector(lbinder).lookup(scope, 'a_string').should == nil + end + end + + context "When using the first found producer" do + it "should lookup until it finds a value, but not further" do + bindings.bind().name('a_string').to_first_found('b_string', 'c_string', 'g_string') + bindings.bind().name('c_string').to('hello') + bindings.bind().name('g_string').to('Oh, mrs. Smith...') + injector(lbinder).lookup(scope, 'a_string').should == 'hello' + end + + it "should lookup until it finds a value using mix of type and name, but not further" do + bindings.bind().name('a_string').to_first_found('b_string', [type_factory.string, 'c_string'], 'g_string') + bindings.bind().name('c_string').to('hello') + bindings.bind().name('g_string').to('Oh, mrs. Smith...') + injector(lbinder).lookup(scope, 'a_string').should == 'hello' + end + end + + context "When producing instances" do + it "should lookup an instance of a class without arguments" do + bindings.bind().type(duck_type).name('the_duck').to(InjectorSpecModule::Daffy) + injector(lbinder).lookup(scope, duck_type, 'the_duck').is_a?(InjectorSpecModule::Daffy).should == true + end + + it "should lookup an instance of a class with arguments" do + bindings.bind().type(duck_type).name('the_duck').to(InjectorSpecModule::ScroogeMcDuck, 1234) + injector = injector(lbinder) + + the_duck = injector.lookup(scope, duck_type, 'the_duck') + the_duck.is_a?(InjectorSpecModule::ScroogeMcDuck).should == true + the_duck.fortune.should == 1234 + end + + it "singleton producer should not be recreated between lookups" do + bindings.bind().type(duck_type).name('the_duck').to_producer(InjectorSpecModule::ScroogeProducer) + injector = injector(lbinder) + + the_duck = injector.lookup(scope, duck_type, 'the_duck') + the_duck.is_a?(InjectorSpecModule::ScroogeMcDuck).should == true + the_duck.fortune.should == 200 + + # singleton, do it again to get next value in series - it is the producer that is a singleton + # not the produced value + the_duck = injector.lookup(scope, duck_type, 'the_duck') + the_duck.is_a?(InjectorSpecModule::ScroogeMcDuck).should == true + the_duck.fortune.should == 400 + + duck_producer = injector.lookup_producer(scope, duck_type, 'the_duck') + duck_producer.produce(scope).fortune.should == 800 + end + + it "series of producers should recreate producer on each lookup and lookup_producer" do + bindings.bind().type(duck_type).name('the_duck').to_producer_series(InjectorSpecModule::ScroogeProducer) + injector = injector(lbinder) + + duck_producer = injector.lookup_producer(scope, duck_type, 'the_duck') + duck_producer.produce(scope).fortune().should == 200 + duck_producer.produce(scope).fortune().should == 400 + + # series, each lookup gets a new producer (initialized to produce 200) + duck_producer = injector.lookup_producer(scope, duck_type, 'the_duck') + duck_producer.produce(scope).fortune().should == 200 + duck_producer.produce(scope).fortune().should == 400 + + injector.lookup(scope, duck_type, 'the_duck').fortune().should == 200 + injector.lookup(scope, duck_type, 'the_duck').fortune().should == 200 + end + end + + context "When working with multibind" do + context "of hash kind" do + it "a multibind produces contributed items keyed by their bound key-name" do + hash_of_duck = type_factory.hash_of(duck_type) + multibind_id = "ducks" + + bindings.multibind(multibind_id).type(hash_of_duck).name('donalds_nephews') + bindings.bind.in_multibind(multibind_id).type(duck_type).name('nephew1').to(InjectorSpecModule::NamedDuck, 'Huey') + bindings.bind.in_multibind(multibind_id).type(duck_type).name('nephew2').to(InjectorSpecModule::NamedDuck, 'Dewey') + bindings.bind.in_multibind(multibind_id).type(duck_type).name('nephew3').to(InjectorSpecModule::NamedDuck, 'Louie') + + injector = injector(lbinder) + the_ducks = injector.lookup(scope, hash_of_duck, "donalds_nephews") + the_ducks.size.should == 3 + the_ducks['nephew1'].name.should == 'Huey' + the_ducks['nephew2'].name.should == 'Dewey' + the_ducks['nephew3'].name.should == 'Louie' + end + + it "is an error to not bind contribution with a name" do + hash_of_duck = type_factory.hash_of(duck_type) + multibind_id = "ducks" + + bindings.multibind(multibind_id).type(hash_of_duck).name('donalds_nephews') + # missing name + bindings.bind.in_multibind(multibind_id).type(duck_type).to(InjectorSpecModule::NamedDuck, 'Huey') + bindings.bind.in_multibind(multibind_id).type(duck_type).to(InjectorSpecModule::NamedDuck, 'Dewey') + + expect { + the_ducks = injector(lbinder).lookup(scope, hash_of_duck, "donalds_nephews") + }.to raise_error(/must have a name/) + end + + it "is an error to bind with duplicate key when using default (priority) conflict resolution" do + hash_of_duck = type_factory.hash_of(duck_type) + multibind_id = "ducks" + + bindings.multibind(multibind_id).type(hash_of_duck).name('donalds_nephews') + # missing name + bindings.bind.in_multibind(multibind_id).type(duck_type).name('foo').to(InjectorSpecModule::NamedDuck, 'Huey') + bindings.bind.in_multibind(multibind_id).type(duck_type).name('foo').to(InjectorSpecModule::NamedDuck, 'Dewey') + + expect { + the_ducks = injector(lbinder).lookup(scope, hash_of_duck, "donalds_nephews") + }.to raise_error(/Duplicate key/) + end + + it "is not an error to bind with duplicate key when using (ignore) conflict resolution" do + hash_of_duck = type_factory.hash_of(duck_type) + multibind_id = "ducks" + + bindings.multibind(multibind_id).type(hash_of_duck).name('donalds_nephews').producer_options(:conflict_resolution => :ignore) + bindings.bind.in_multibind(multibind_id).type(duck_type).name('foo').to(InjectorSpecModule::NamedDuck, 'Huey') + bindings.bind.in_multibind(multibind_id).type(duck_type).name('foo').to(InjectorSpecModule::NamedDuck, 'Dewey') + + expect { + the_ducks = injector(lbinder).lookup(scope, hash_of_duck, "donalds_nephews") + }.to_not raise_error(/Duplicate key/) + end + + it "should produce detailed type error message" do + hash_of_integer = type_factory.hash_of(type_factory.integer()) + + multibind_id = "ints" + mb = bindings.multibind(multibind_id).type(hash_of_integer).name('donalds_family') + bindings.bind.in_multibind(multibind_id).name('nephew').to('Huey') + + expect { ducks = injector(lbinder).lookup(scope, 'donalds_family') + }.to raise_error(%r{expected: Integer, got: String}) + end + + it "should be possible to combine hash multibind contributions with append on conflict" do + # This case uses a multibind of individual strings, but combines them + # into an array bound to a hash key + # (There are other ways to do this - e.g. have the multibind lookup a multibind + # of array type to which nephews are contributed). + # + hash_of_data = type_factory.hash_of_data() + multibind_id = "ducks" + mb = bindings.multibind(multibind_id).type(hash_of_data).name('donalds_family') + mb.producer_options(:conflict_resolution => :append) + + bindings.bind.in_multibind(multibind_id).name('nephews').to('Huey') + bindings.bind.in_multibind(multibind_id).name('nephews').to('Dewey') + bindings.bind.in_multibind(multibind_id).name('nephews').to('Louie') + bindings.bind.in_multibind(multibind_id).name('uncles').to('Scrooge McDuck') + bindings.bind.in_multibind(multibind_id).name('uncles').to('Ludwig Von Drake') + + ducks = injector(lbinder).lookup(scope, 'donalds_family') + + ducks['nephews'].should == ['Huey', 'Dewey', 'Louie'] + ducks['uncles'].should == ['Scrooge McDuck', 'Ludwig Von Drake'] + end + + it "should be possible to combine hash multibind contributions with append, flat, and uniq, on conflict" do + # This case uses a multibind of individual strings, but combines them + # into an array bound to a hash key + # (There are other ways to do this - e.g. have the multibind lookup a multibind + # of array type to which nephews are contributed). + # + hash_of_data = type_factory.hash_of_data() + multibind_id = "ducks" + mb = bindings.multibind(multibind_id).type(hash_of_data).name('donalds_family') + mb.producer_options(:conflict_resolution => :append, :flatten => true, :uniq => true) + + bindings.bind.in_multibind(multibind_id).name('nephews').to('Huey') + bindings.bind.in_multibind(multibind_id).name('nephews').to('Huey') + bindings.bind.in_multibind(multibind_id).name('nephews').to('Dewey') + bindings.bind.in_multibind(multibind_id).name('nephews').to(['Huey', ['Louie'], 'Dewey']) + bindings.bind.in_multibind(multibind_id).name('uncles').to('Scrooge McDuck') + bindings.bind.in_multibind(multibind_id).name('uncles').to('Ludwig Von Drake') + + ducks = injector(lbinder).lookup(scope, 'donalds_family') + + ducks['nephews'].should == ['Huey', 'Dewey', 'Louie'] + ducks['uncles'].should == ['Scrooge McDuck', 'Ludwig Von Drake'] + end + + it "should fail attempts to append, perform uniq or flatten on type incompatible multibind hash" do + hash_of_integer = type_factory.hash_of(type_factory.integer()) + ids = ["ducks1", "ducks2", "ducks3"] + mb = bindings.multibind(ids[0]).type(hash_of_integer).name('broken_family0') + mb.producer_options(:conflict_resolution => :append) + mb = bindings.multibind(ids[1]).type(hash_of_integer).name('broken_family1') + mb.producer_options(:flatten => :true) + mb = bindings.multibind(ids[2]).type(hash_of_integer).name('broken_family2') + mb.producer_options(:uniq => :true) + + + binder.define_categories(factory.categories([])) + binder.define_layers(factory.layered_bindings(test_layer_with_bindings(bindings.model))) + injector = injector(binder) + expect { injector.lookup(scope, 'broken_family0')}.to raise_error(/:conflict_resolution => :append/) + expect { injector.lookup(scope, 'broken_family1')}.to raise_error(/:flatten/) + expect { injector.lookup(scope, 'broken_family2')}.to raise_error(/:uniq/) + end + + it "a higher priority contribution is selected when resolution is :priority" do + hash_of_duck = type_factory.hash_of(duck_type) + multibind_id = "ducks" + + bindings.multibind(multibind_id).type(hash_of_duck).name('donalds_nephews') + + mb1 = bindings.when_in_category("highest", "test").bind.in_multibind(multibind_id) + mb1.type(duck_type).name('nephew').to(InjectorSpecModule::NamedDuck, 'Huey') + + mb2 = bindings.bind.in_multibind(multibind_id) + mb2.type(duck_type).name('nephew').to(InjectorSpecModule::NamedDuck, 'Dewey') + + binder.define_categories(factory.categories(['highest', 'test'])) + binder.define_layers(layered_bindings) + + injector(binder).lookup(scope, hash_of_duck, "donalds_nephews")['nephew'].name.should == 'Huey' + end + + it "a higher priority contribution wins when resolution is :merge" do + hash_of_data = type_factory.hash_of_data() + multibind_id = "hashed_ducks" + + bindings.multibind(multibind_id).type(hash_of_data).name('donalds_nephews').producer_options(:conflict_resolution => :merge) + + mb1 = bindings.when_in_category("highest", "test").bind.in_multibind(multibind_id) + mb1.name('nephew').to({'name' => 'Huey', 'is' => 'winner'}) + + mb2 = bindings.bind.in_multibind(multibind_id) + mb2.name('nephew').to({'name' => 'Dewey', 'is' => 'looser', 'has' => 'cap'}) + + binder.define_categories(factory.categories(['highest', 'test'])) + binder.define_layers(layered_bindings) + + the_ducks = injector(binder).lookup(scope, "donalds_nephews"); + the_ducks['nephew']['name'].should == 'Huey' + the_ducks['nephew']['is'].should == 'winner' + the_ducks['nephew']['has'].should == 'cap' + end + end + + context "of array kind" do + it "an array multibind produces contributed items, names are allowed but ignored" do + array_of_duck = type_factory.array_of(duck_type) + multibind_id = "ducks" + + bindings.multibind(multibind_id).type(array_of_duck).name('donalds_nephews') + # one with name (ignored, expect no error) + bindings.bind.in_multibind(multibind_id).type(duck_type).name('nephew1').to(InjectorSpecModule::NamedDuck, 'Huey') + # two without name + bindings.bind.in_multibind(multibind_id).type(duck_type).to(InjectorSpecModule::NamedDuck, 'Dewey') + bindings.bind.in_multibind(multibind_id).type(duck_type).to(InjectorSpecModule::NamedDuck, 'Louie') + + the_ducks = injector(lbinder).lookup(scope, array_of_duck, "donalds_nephews") + the_ducks.size.should == 3 + the_ducks.collect {|d| d.name }.sort.should == ['Dewey', 'Huey', 'Louie'] + end + + it "should be able to make result contain only unique entries" do + # This case uses a multibind of individual strings, and combines them + # into an array of unique values + # + array_of_data = type_factory.array_of_data() + multibind_id = "ducks" + mb = bindings.multibind(multibind_id).type(array_of_data).name('donalds_family') + # turn off priority on named to not trigger conflict as all additions have the same precedence + # (could have used the default for unnamed and add unnamed entries). + mb.producer_options(:priority_on_named => false, :uniq => true) + + bindings.bind.in_multibind(multibind_id).name('nephews').to('Huey') + bindings.bind.in_multibind(multibind_id).name('nephews').to('Dewey') + bindings.bind.in_multibind(multibind_id).name('nephews').to('Dewey') # duplicate + bindings.bind.in_multibind(multibind_id).name('nephews').to('Louie') + bindings.bind.in_multibind(multibind_id).name('nephews').to('Louie') # duplicate + bindings.bind.in_multibind(multibind_id).name('nephews').to('Louie') # duplicate + + ducks = injector(lbinder).lookup(scope, 'donalds_family') + ducks.should == ['Huey', 'Dewey', 'Louie'] + end + + it "should be able to contribute elements and arrays of elements and flatten 1 level" do + # This case uses a multibind of individual strings and arrays, and combines them + # into an array of flattened + # + array_of_string = type_factory.array_of(type_factory.string()) + + multibind_id = "ducks" + mb = bindings.multibind(multibind_id).type(array_of_string).name('donalds_family') + # flatten one level + mb.producer_options(:flatten => 1) + + bindings.bind.in_multibind(multibind_id).to('Huey') + bindings.bind.in_multibind(multibind_id).to('Dewey') + bindings.bind.in_multibind(multibind_id).to('Louie') # duplicate + bindings.bind.in_multibind(multibind_id).to(['Huey', 'Dewey', 'Louie']) + + ducks = injector(lbinder).lookup(scope, 'donalds_family') + ducks.should == ['Huey', 'Dewey', 'Louie', 'Huey', 'Dewey', 'Louie'] + end + + it "should produce detailed type error message" do + array_of_integer = type_factory.array_of(type_factory.integer()) + + multibind_id = "ints" + mb = bindings.multibind(multibind_id).type(array_of_integer).name('donalds_family') + bindings.bind.in_multibind(multibind_id).to('Huey') + + expect { ducks = injector(lbinder).lookup(scope, 'donalds_family') + }.to raise_error(%r{expected: Integer, or Array\[Integer\], got: String}) + end + end + + context "When using multibind in multibind" do + it "a hash multibind can be contributed to another" do + hash_of_data = type_factory.hash_of_data() + mb1_id = 'data1' + mb2_id = 'data2' + top = bindings.multibind(mb1_id).type(hash_of_data).name("top") + detail = bindings.multibind(mb2_id).type(hash_of_data).name("detail").in_multibind(mb1_id) + + bindings.bind.in_multibind(mb1_id).name('a').to(10) + bindings.bind.in_multibind(mb1_id).name('b').to(20) + bindings.bind.in_multibind(mb2_id).name('a').to(30) + bindings.bind.in_multibind(mb2_id).name('b').to(40) + expect( injector(lbinder).lookup(scope, "top") ).to eql({'detail' => {'a' => 30, 'b' => 40}, 'a' => 10, 'b' => 20}) + end + end + + context "When looking up entries requiring evaluation" do + let(:node) { Puppet::Node.new('localhost') } + let(:compiler) { Puppet::Parser::Compiler.new(node)} + let(:scope) { Puppet::Parser::Scope.new(compiler) } + let(:parser) { Puppet::Pops::Parser::Parser.new() } + + it "should be possible to lookup a concatenated string" do + scope['duck'] = 'Donald Fauntleroy Duck' + expr = parser.parse_string('"Hello $duck"').current() + bindings.bind.name('the_duck').to(expr) + injector(lbinder).lookup(scope, 'the_duck').should == 'Hello Donald Fauntleroy Duck' + end + + it "should be possible to post process lookup with a puppet lambda" do + model = parser.parse_string('fake() |$value| {$value + 1 }').current + bindings.bind.name('an_int').to(42).producer_options( :transformer => model.lambda) + injector(lbinder).lookup(scope, 'an_int').should == 43 + end + + it "should be possible to post process lookup with a ruby proc" do + transformer = lambda {|scope, value| value + 1 } + bindings.bind.name('an_int').to(42).producer_options( :transformer => transformer) + injector(lbinder).lookup(scope, 'an_int').should == 43 + end + end + end + context "When there are problems with configuration" do + let(:binder) { binder_with_categories()} + let(:lbinder) { binder.define_layers(layered_bindings) } + + it "reports error for surfacing abstract bindings" do + bindings.bind.abstract.name('an_int') + expect{injector(lbinder).lookup(scope, 'an_int') }.to raise_error(/The abstract binding .* was not overridden/) + end + + it "does not report error for abstract binding that is ovrridden" do + bindings.bind.abstract.name('an_int') + bindings.when_in_category('highest', 'test').bind.override.name('an_int').to(142) + expect{injector(lbinder).lookup(scope, 'an_int') }.to_not raise_error + end + + it "reports error for overriding binding that does not override" do + bindings.bind.override.name('an_int').to(42) + expect{injector(lbinder).lookup(scope, 'an_int') }.to raise_error(/Binding with unresolved 'override' detected/) + end + + it "reports error for binding without producer" do + bindings.bind.name('an_int') + expect{injector(lbinder).lookup(scope, 'an_int') }.to raise_error(/Binding without producer/) + end + end +end
\ No newline at end of file diff --git a/spec/unit/pops/containment_spec.rb b/spec/unit/pops/containment_spec.rb index da5fae77a..b992ed9d1 100644 --- a/spec/unit/pops/containment_spec.rb +++ b/spec/unit/pops/containment_spec.rb @@ -1,3 +1,4 @@ +require 'spec_helper' require 'puppet/pops' require File.join(File.dirname(__FILE__), 'factory_rspec_helper') diff --git a/spec/unit/pops/issues_spec.rb b/spec/unit/pops/issues_spec.rb index 091bd9c93..d8650b956 100644 --- a/spec/unit/pops/issues_spec.rb +++ b/spec/unit/pops/issues_spec.rb @@ -16,7 +16,7 @@ describe "Puppet::Pops::Issues" do x.format(:name => 'Boo-Hoo', :label => Puppet::Pops::Model::ModelLabelProvider.new, :semantic => "dummy" - ).should == "A Ruby String may not have a name contain a hyphen. The name 'Boo-Hoo' is not legal" + ).should == "A Ruby String may not have a name containing a hyphen. The name 'Boo-Hoo' is not legal" end it "should should format a message that does not require an argument" do diff --git a/spec/unit/pops/parser/evaluating_parser_spec.rb b/spec/unit/pops/parser/evaluating_parser_spec.rb new file mode 100644 index 000000000..027b86a09 --- /dev/null +++ b/spec/unit/pops/parser/evaluating_parser_spec.rb @@ -0,0 +1,88 @@ +require 'spec_helper' +require 'puppet/pops' +require 'puppet_spec/pops' + +describe 'The hiera2 string evaluator' do + + include PuppetSpec::Pops + + let(:acceptor) { Puppet::Pops::Validation::Acceptor.new() } + let(:diag) { Puppet::Pops::Binder::Hiera2::DiagnosticProducer.new(acceptor) } + let(:scope) { s = Puppet::Parser::Scope.new_for_test_harness(node); s } + let(:node) { 'node.example.com' } + + def quote(x) + Puppet::Pops::Parser::EvaluatingParser.quote(x) + end + + def evaluator() + Puppet::Pops::Parser::EvaluatingParser.new() + end + + def evaluate(s) + evaluator.evaluate(scope, quote(s)) + end + + def test(x) + evaluator.evaluate_string(scope, quote(x)).should == x + end + + def test_interpolate(x, y) + scope['a'] = 'expansion' + evaluator.evaluate_string(scope, quote(x)).should == y + end + + context 'when evaluating' do + it 'should produce an empty string with no change' do + test('') + end + + it 'should produce a normal string with no change' do + test('A normal string') + end + + it 'should produce a string with newlines with no change' do + test("A\nnormal\nstring") + end + + it 'should produce a string with escaped newlines with no change' do + test("A\\nnormal\\nstring") + end + + it 'should produce a string containing quotes without change' do + test('This " should remain untouched') + end + + it 'should produce a string containing escaped quotes without change' do + test('This \" should remain untouched') + end + + it 'should expand ${a} variables' do + test_interpolate('This ${a} was expanded', 'This expansion was expanded') + end + + it 'should expand quoted ${a} variables' do + test_interpolate('This "${a}" was expanded', 'This "expansion" was expanded') + end + + it 'should not expand escaped ${a}' do + test_interpolate('This \${a} was not expanded', 'This ${a} was not expanded') + end + + it 'should expand $a variables' do + test_interpolate('This $a was expanded', 'This expansion was expanded') + end + + it 'should expand quoted $a variables' do + test_interpolate('This "$a" was expanded', 'This "expansion" was expanded') + end + + it 'should not expand escaped $a' do + test_interpolate('This \$a was not expanded', 'This $a was not expanded') + end + + it 'should produce an single space from a \s' do + test_interpolate("\\s", ' ') + end + end +end diff --git a/spec/unit/pops/parser/lexer_spec.rb b/spec/unit/pops/parser/lexer_spec.rb index d5232e9f1..985807261 100755 --- a/spec/unit/pops/parser/lexer_spec.rb +++ b/spec/unit/pops/parser/lexer_spec.rb @@ -763,7 +763,7 @@ describe "Puppet::Pops::Parser::Lexer in the old tests when lexing example files end end -describe "when trying to lex an non-existent file" do +describe "when trying to lex a non-existent file" do include PuppetSpec::Files it "should return an empty list of tokens" do diff --git a/spec/unit/pops/parser/parse_calls_spec.rb b/spec/unit/pops/parser/parse_calls_spec.rb index 62012a430..647d0e16c 100644 --- a/spec/unit/pops/parser/parse_calls_spec.rb +++ b/spec/unit/pops/parser/parse_calls_spec.rb @@ -29,6 +29,10 @@ describe "egrammar parsing function calls" do it "foo(bar, fum,)" do dump(parse("foo(bar,fum,)")).should == "(invoke foo bar fum)" end + + it "foo fqdn_rand(30)" do + dump(parse("foo fqdn_rand(30)")).should == '(invoke foo (call fqdn_rand 30))' + end end context "in nested scopes" do diff --git a/spec/unit/pops/parser/parser_spec.rb b/spec/unit/pops/parser/parser_spec.rb index 86d3d6dd0..e29410681 100644 --- a/spec/unit/pops/parser/parser_spec.rb +++ b/spec/unit/pops/parser/parser_spec.rb @@ -10,6 +10,6 @@ describe Puppet::Pops::Parser::Parser do it "should parse a code string and return a model" do parser = Puppet::Pops::Parser::Parser.new() model = parser.parse_string("$a = 10").current - model.class.should == Model::AssignmentExpression + model.class.should == Puppet::Pops::Model::AssignmentExpression end end diff --git a/spec/unit/pops/types/type_calculator_spec.rb b/spec/unit/pops/types/type_calculator_spec.rb new file mode 100644 index 000000000..7ee47d648 --- /dev/null +++ b/spec/unit/pops/types/type_calculator_spec.rb @@ -0,0 +1,484 @@ +require 'spec_helper' +require 'puppet/pops' + +describe 'The type calculator' do + let(:calculator) { Puppet::Pops::Types::TypeCalculator.new() } + + context 'when inferring ruby' do + + it 'fixnum translates to PIntegerType' do + calculator.infer(1).class.should == Puppet::Pops::Types::PIntegerType + end + + it 'large fixnum (or bignum depending on architecture) translates to PIntegerType' do + calculator.infer(2**33).class.should == Puppet::Pops::Types::PIntegerType + end + + it 'float translates to PFloatType' do + calculator.infer(1.3).class.should == Puppet::Pops::Types::PFloatType + end + + it 'string translates to PStringType' do + calculator.infer('foo').class.should == Puppet::Pops::Types::PStringType + end + + it 'boolean true translates to PBooleanType' do + calculator.infer(true).class.should == Puppet::Pops::Types::PBooleanType + end + + it 'boolean false translates to PBooleanType' do + calculator.infer(false).class.should == Puppet::Pops::Types::PBooleanType + end + + it 'regexp translates to PPatternType' do + calculator.infer(/^a regular exception$/).class.should == Puppet::Pops::Types::PPatternType + end + + it 'nil translates to PNilType' do + calculator.infer(nil).class.should == Puppet::Pops::Types::PNilType + end + + it 'an instance of class Foo translates to PRubyType[Foo]' do + class Foo + end + + t = calculator.infer(Foo.new) + t.class.should == Puppet::Pops::Types::PRubyType + t.ruby_class.should == 'Foo' + end + + context 'array' do + it 'translates to PArrayType' do + calculator.infer([1,2]).class.should == Puppet::Pops::Types::PArrayType + end + + it 'with fixnum values translates to PArrayType[PIntegerType]' do + calculator.infer([1,2]).element_type.class.should == Puppet::Pops::Types::PIntegerType + end + + it 'with 32 and 64 bit integer values translates to PArrayType[PIntegerType]' do + calculator.infer([1,2**33]).element_type.class.should == Puppet::Pops::Types::PIntegerType + end + + it 'with fixnum and float values translates to PArrayType[PNumericType]' do + calculator.infer([1,2.0]).element_type.class.should == Puppet::Pops::Types::PNumericType + end + + it 'with fixnum and string values translates to PArrayType[PLiteralType]' do + calculator.infer([1,'two']).element_type.class.should == Puppet::Pops::Types::PLiteralType + end + + it 'with float and string values translates to PArrayType[PLiteralType]' do + calculator.infer([1.0,'two']).element_type.class.should == Puppet::Pops::Types::PLiteralType + end + + it 'with fixnum, float, and string values translates to PArrayType[PLiteralType]' do + calculator.infer([1, 2.0,'two']).element_type.class.should == Puppet::Pops::Types::PLiteralType + end + + it 'with fixnum and regexp values translates to PArrayType[PLiteralType]' do + calculator.infer([1, /two/]).element_type.class.should == Puppet::Pops::Types::PLiteralType + end + + it 'with string and regexp values translates to PArrayType[PLiteralType]' do + calculator.infer(['one', /two/]).element_type.class.should == Puppet::Pops::Types::PLiteralType + end + + it 'with string and symbol values translates to PArrayType[PObjectType]' do + calculator.infer(['one', :two]).element_type.class.should == Puppet::Pops::Types::PObjectType + end + + it 'with fixnum and nil values translates to PArrayType[PIntegerType]' do + calculator.infer([1, nil]).element_type.class.should == Puppet::Pops::Types::PIntegerType + end + + it 'with arrays of string values translates to PArrayType[PArrayType[PStringType]]' do + et = calculator.infer([['first' 'array'], ['second','array']]) + et.class.should == Puppet::Pops::Types::PArrayType + et = et.element_type + et.class.should == Puppet::Pops::Types::PArrayType + et = et.element_type + et.class.should == Puppet::Pops::Types::PStringType + end + + it 'with array of string values and array of fixnums translates to PArrayType[PArrayType[PLiteralType]]' do + et = calculator.infer([['first' 'array'], [1,2]]) + et.class.should == Puppet::Pops::Types::PArrayType + et = et.element_type + et.class.should == Puppet::Pops::Types::PArrayType + et = et.element_type + et.class.should == Puppet::Pops::Types::PLiteralType + end + + it 'with hashes of string values translates to PArrayType[PHashType[PStringType]]' do + et = calculator.infer([{:first => 'first', :second => 'second' }, {:first => 'first', :second => 'second' }]) + et.class.should == Puppet::Pops::Types::PArrayType + et = et.element_type + et.class.should == Puppet::Pops::Types::PHashType + et = et.element_type + et.class.should == Puppet::Pops::Types::PStringType + end + + it 'with hash of string values and hash of fixnums translates to PArrayType[PHashType[PLiteralType]]' do + et = calculator.infer([{:first => 'first', :second => 'second' }, {:first => 1, :second => 2 }]) + et.class.should == Puppet::Pops::Types::PArrayType + et = et.element_type + et.class.should == Puppet::Pops::Types::PHashType + et = et.element_type + et.class.should == Puppet::Pops::Types::PLiteralType + end + end + + context 'hash' do + it 'translates to PHashType' do + calculator.infer({:first => 1, :second => 2}).class.should == Puppet::Pops::Types::PHashType + end + + it 'with symbolic keys translates to PHashType[PRubyType[Symbol],value]' do + k = calculator.infer({:first => 1, :second => 2}).key_type + k.class.should == Puppet::Pops::Types::PRubyType + k.ruby_class.should == 'Symbol' + end + + it 'with string keys translates to PHashType[PStringType,value]' do + calculator.infer({'first' => 1, 'second' => 2}).key_type.class.should == Puppet::Pops::Types::PStringType + end + + it 'with fixnum values translates to PHashType[key,PIntegerType]' do + calculator.infer({:first => 1, :second => 2}).element_type.class.should == Puppet::Pops::Types::PIntegerType + end + end + end + + context 'when testing if x is assignable to y' do + it 'should allow all object types to PObjectType' do + t = Puppet::Pops::Types::PObjectType.new() + calculator.assignable?(t, t).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PNilType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PDataType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PLiteralType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PStringType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PNumericType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PIntegerType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PFloatType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PPatternType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PBooleanType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PCollectionType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PArrayType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PHashType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PRubyType.new()).should() == true + end + + it 'should reject PObjectType to less generic types' do + t = Puppet::Pops::Types::PObjectType.new() + calculator.assignable?(Puppet::Pops::Types::PDataType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PLiteralType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PStringType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PNumericType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PIntegerType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PFloatType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PPatternType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PBooleanType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PCollectionType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PArrayType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PHashType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PRubyType.new(), t).should() == false + end + + it 'should allow all data types, array, and hash to PDataType' do + t = Puppet::Pops::Types::PDataType.new() + calculator.assignable?(t, t).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PLiteralType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PStringType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PNumericType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PIntegerType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PFloatType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PPatternType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PBooleanType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PArrayType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PHashType.new()).should() == true + end + + it 'should reject PDataType to less generic data types' do + t = Puppet::Pops::Types::PDataType.new() + calculator.assignable?(Puppet::Pops::Types::PLiteralType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PStringType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PNumericType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PIntegerType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PFloatType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PPatternType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PBooleanType.new(), t).should() == false + end + + it 'should reject PDataType to non data types' do + t = Puppet::Pops::Types::PDataType.new() + calculator.assignable?(Puppet::Pops::Types::PCollectionType.new(),t).should() == false + calculator.assignable?(Puppet::Pops::Types::PArrayType.new(),t).should() == false + calculator.assignable?(Puppet::Pops::Types::PHashType.new(),t).should() == false + calculator.assignable?(Puppet::Pops::Types::PRubyType.new(), t).should() == false + end + + it 'should allow all literal types to PLiteralType' do + t = Puppet::Pops::Types::PLiteralType.new() + calculator.assignable?(t, t).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PStringType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PNumericType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PIntegerType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PFloatType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PPatternType.new()).should() == true + calculator.assignable?(t,Puppet::Pops::Types::PBooleanType.new()).should() == true + end + + it 'should reject PLiteralType to less generic literal types' do + t = Puppet::Pops::Types::PLiteralType.new() + calculator.assignable?(Puppet::Pops::Types::PStringType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PNumericType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PIntegerType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PFloatType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PPatternType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PBooleanType.new(), t).should() == false + end + + it 'should reject PLiteralType to non literal types' do + t = Puppet::Pops::Types::PLiteralType.new() + calculator.assignable?(Puppet::Pops::Types::PCollectionType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PArrayType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PHashType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PRubyType.new(), t).should() == false + end + + it 'should allow all numeric types to PNumericType' do + t = Puppet::Pops::Types::PNumericType.new() + calculator.assignable?(t, t).should() == true + calculator.assignable?(t, Puppet::Pops::Types::PIntegerType.new()).should() == true + calculator.assignable?(t, Puppet::Pops::Types::PFloatType.new()).should() == true + end + + it 'should reject PNumericType to less generic numeric types' do + t = Puppet::Pops::Types::PNumericType.new() + calculator.assignable?(Puppet::Pops::Types::PIntegerType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PFloatType.new(), t).should() == false + end + + it 'should reject PNumericType to non numeric types' do + t = Puppet::Pops::Types::PNumericType.new() + calculator.assignable?(Puppet::Pops::Types::PStringType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PPatternType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PBooleanType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PCollectionType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PArrayType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PHashType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PRubyType.new(), t).should() == false + end + + it 'should allow all collection types to PCollectionType' do + t = Puppet::Pops::Types::PCollectionType.new() + calculator.assignable?(t, t).should() == true + calculator.assignable?(t, Puppet::Pops::Types::PArrayType.new()).should() == true + calculator.assignable?(t, Puppet::Pops::Types::PHashType.new()).should() == true + end + + it 'should reject PCollectionType to less generic collection types' do + t = Puppet::Pops::Types::PCollectionType.new() + calculator.assignable?(Puppet::Pops::Types::PArrayType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PHashType.new(), t).should() == false + end + + it 'should reject PCollectionType to non collection types' do + t = Puppet::Pops::Types::PCollectionType.new() + calculator.assignable?(Puppet::Pops::Types::PDataType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PLiteralType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PStringType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PNumericType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PIntegerType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PFloatType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PPatternType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PBooleanType.new(), t).should() == false + calculator.assignable?(Puppet::Pops::Types::PRubyType.new(), t).should() == false + end + + it 'should reject PArrayType to non array type collections' do + t = Puppet::Pops::Types::PArrayType.new() + calculator.assignable?(Puppet::Pops::Types::PHashType.new(), t).should() == false + end + + it 'should reject PHashType to non hash type collections' do + t = Puppet::Pops::Types::PHashType.new() + calculator.assignable?(Puppet::Pops::Types::PArrayType.new(), t).should() == false + end + + it 'should recognize mapped ruby types' do + calculator.assignable?(Puppet::Pops::Types::PIntegerType.new(), Integer).should == true + calculator.assignable?(Puppet::Pops::Types::PIntegerType.new(), Fixnum).should == true + calculator.assignable?(Puppet::Pops::Types::PIntegerType.new(), Bignum).should == true + calculator.assignable?(Puppet::Pops::Types::PFloatType.new(), Float).should == true + calculator.assignable?(Puppet::Pops::Types::PNumericType.new(), Numeric).should == true + calculator.assignable?(Puppet::Pops::Types::PNilType.new(), NilClass).should == true + calculator.assignable?(Puppet::Pops::Types::PBooleanType.new(), FalseClass).should == true + calculator.assignable?(Puppet::Pops::Types::PBooleanType.new(), TrueClass).should == true + calculator.assignable?(Puppet::Pops::Types::PStringType.new(), String).should == true + calculator.assignable?(Puppet::Pops::Types::PPatternType.new(), Regexp).should == true + calculator.assignable?(Puppet::Pops::Types::TypeFactory.array_of_data(), Array).should == true + calculator.assignable?(Puppet::Pops::Types::TypeFactory.hash_of_data(), Hash).should == true + end + + it 'should recognize ruby type inheritance' do + class Foo + end + + class Bar < Foo + end + + fooType = calculator.infer(Foo.new) + barType = calculator.infer(Bar.new) + + calculator.assignable?(fooType, fooType).should == true + calculator.assignable?(Foo, fooType).should == true + + calculator.assignable?(fooType, barType).should == true + calculator.assignable?(Foo, barType).should == true + + calculator.assignable?(barType, fooType).should == false + calculator.assignable?(Bar, fooType).should == false + end + end + + context 'when testing if x is instance of type t' do + it 'should consider fixnum instanceof PIntegerType' do + calculator.instance?(Puppet::Pops::Types::PIntegerType.new(), 1) + end + + it 'should consider fixnum instanceof Fixnum' do + calculator.instance?(Fixnum, 1) + end + end + + context 'when converting a ruby class' do + it 'should yield \'PIntegerType\' for Integer, Fixnum, and Bignum' do + [Integer,Fixnum,Bignum].each do |c| + calculator.type(c).class.should == Puppet::Pops::Types::PIntegerType + end + end + + it 'should yield \'PFloatType\' for Float' do + calculator.type(Float).class.should == Puppet::Pops::Types::PFloatType + end + + it 'should yield \'PBooleanType\' for FalseClass and TrueClass' do + [FalseClass,TrueClass].each do |c| + calculator.type(c).class.should == Puppet::Pops::Types::PBooleanType + end + end + + it 'should yield \'PNilType\' for NilClass' do + calculator.type(NilClass).class.should == Puppet::Pops::Types::PNilType + end + + it 'should yield \'PStringType\' for String' do + calculator.type(String).class.should == Puppet::Pops::Types::PStringType + end + + it 'should yield \'PPatternType\' for Regexp' do + calculator.type(Regexp).class.should == Puppet::Pops::Types::PPatternType + end + + it 'should yield \'PArrayType[PDataType]\' for Array' do + t = calculator.type(Array) + t.class.should == Puppet::Pops::Types::PArrayType + t.element_type.class.should == Puppet::Pops::Types::PDataType + end + + it 'should yield \'PHashType[PLiteralType,PDataType]\' for Hash' do + t = calculator.type(Hash) + t.class.should == Puppet::Pops::Types::PHashType + t.key_type.class.should == Puppet::Pops::Types::PLiteralType + t.element_type.class.should == Puppet::Pops::Types::PDataType + end + end + + context 'when representing the type as string' do + it 'should yield \'Type\' for PType' do + calculator.string(Puppet::Pops::Types::PType.new()).should == 'Type' + end + + it 'should yield \'Object\' for PObjectType' do + calculator.string(Puppet::Pops::Types::PObjectType.new()).should == 'Object' + end + + it 'should yield \'Literal\' for PLiteralType' do + calculator.string(Puppet::Pops::Types::PLiteralType.new()).should == 'Literal' + end + + it 'should yield \'Boolean\' for PBooleanType' do + calculator.string(Puppet::Pops::Types::PBooleanType.new()).should == 'Boolean' + end + + it 'should yield \'Data\' for PDataType' do + calculator.string(Puppet::Pops::Types::PDataType.new()).should == 'Data' + end + + it 'should yield \'Numeric\' for PNumericType' do + calculator.string(Puppet::Pops::Types::PNumericType.new()).should == 'Numeric' + end + + it 'should yield \'Integer\' for PIntegerType' do + calculator.string(Puppet::Pops::Types::PIntegerType.new()).should == 'Integer' + end + + it 'should yield \'Float\' for PFloatType' do + calculator.string(Puppet::Pops::Types::PFloatType.new()).should == 'Float' + end + + it 'should yield \'Pattern\' for PPatternType' do + calculator.string(Puppet::Pops::Types::PPatternType.new()).should == 'Pattern' + end + + it 'should yield \'String\' for PStringType' do + calculator.string(Puppet::Pops::Types::PStringType.new()).should == 'String' + end + + it 'should yield \'Array[Integer]\' for PArrayType[PIntegerType]' do + t = Puppet::Pops::Types::PArrayType.new() + t.element_type = Puppet::Pops::Types::PIntegerType.new() + calculator.string(t).should == 'Array[Integer]' + end + + it 'should yield \'Hash[String, Integer]\' for PHashType[PStringType, PIntegerType]' do + t = Puppet::Pops::Types::PHashType.new() + t.key_type = Puppet::Pops::Types::PStringType.new() + t.element_type = Puppet::Pops::Types::PIntegerType.new() + calculator.string(t).should == 'Hash[String, Integer]' + end + end + + context 'when processing meta type' do + it 'should infer PType as the type of all other types' do + ptype = Puppet::Pops::Types::PType + calculator.infer(Puppet::Pops::Types::PNilType.new() ).is_a?(ptype).should() == true + calculator.infer(Puppet::Pops::Types::PDataType.new() ).is_a?(ptype).should() == true + calculator.infer(Puppet::Pops::Types::PLiteralType.new() ).is_a?(ptype).should() == true + calculator.infer(Puppet::Pops::Types::PStringType.new() ).is_a?(ptype).should() == true + calculator.infer(Puppet::Pops::Types::PNumericType.new() ).is_a?(ptype).should() == true + calculator.infer(Puppet::Pops::Types::PIntegerType.new() ).is_a?(ptype).should() == true + calculator.infer(Puppet::Pops::Types::PFloatType.new() ).is_a?(ptype).should() == true + calculator.infer(Puppet::Pops::Types::PPatternType.new() ).is_a?(ptype).should() == true + calculator.infer(Puppet::Pops::Types::PBooleanType.new() ).is_a?(ptype).should() == true + calculator.infer(Puppet::Pops::Types::PCollectionType.new()).is_a?(ptype).should() == true + calculator.infer(Puppet::Pops::Types::PArrayType.new() ).is_a?(ptype).should() == true + calculator.infer(Puppet::Pops::Types::PHashType.new() ).is_a?(ptype).should() == true + calculator.infer(Puppet::Pops::Types::PRubyType.new() ).is_a?(ptype).should() == true + end + + it 'should infer PType as the type of ruby classes' do + class Foo + end + [Object, Numeric, Integer, Fixnum, Bignum, Float, String, Regexp, Array, Hash, Foo].each do |c| + calculator.infer(c).is_a?(Puppet::Pops::Types::PType).should() == true + end + end + + it 'should infer PType as the type of PType (meta regression short-circuit)' do + calculator.infer(Puppet::Pops::Types::PType.new()).is_a?(Puppet::Pops::Types::PType).should() == true + end + end +end diff --git a/spec/unit/pops/types/type_factory_spec.rb b/spec/unit/pops/types/type_factory_spec.rb new file mode 100644 index 000000000..be95871c4 --- /dev/null +++ b/spec/unit/pops/types/type_factory_spec.rb @@ -0,0 +1,65 @@ +require 'spec_helper' +require 'puppet/pops' + +describe 'The type factory' do + context 'when creating' do + it 'integer() returns PIntegerType' do + Puppet::Pops::Types::TypeFactory.integer().class().should == Puppet::Pops::Types::PIntegerType + end + + it 'float() returns PFloatType' do + Puppet::Pops::Types::TypeFactory.float().class().should == Puppet::Pops::Types::PFloatType + end + + it 'string() returns PStringType' do + Puppet::Pops::Types::TypeFactory.string().class().should == Puppet::Pops::Types::PStringType + end + + it 'boolean() returns PBooleanType' do + Puppet::Pops::Types::TypeFactory.boolean().class().should == Puppet::Pops::Types::PBooleanType + end + + it 'pattern() returns PPatternType' do + Puppet::Pops::Types::TypeFactory.pattern().class().should == Puppet::Pops::Types::PPatternType + end + + it 'literal() returns PLiteralType' do + Puppet::Pops::Types::TypeFactory.literal().class().should == Puppet::Pops::Types::PLiteralType + end + + it 'data() returns PDataType' do + Puppet::Pops::Types::TypeFactory.data().class().should == Puppet::Pops::Types::PDataType + end + + it 'array_of(fixnum) returns PArrayType[PIntegerType]' do + at = Puppet::Pops::Types::TypeFactory.array_of(1) + at.class().should == Puppet::Pops::Types::PArrayType + at.element_type.class.should == Puppet::Pops::Types::PIntegerType + end + + it 'array_of(PIntegerType) returns PArrayType[PIntegerType]' do + at = Puppet::Pops::Types::TypeFactory.array_of(Puppet::Pops::Types::PIntegerType.new()) + at.class().should == Puppet::Pops::Types::PArrayType + at.element_type.class.should == Puppet::Pops::Types::PIntegerType + end + + it 'array_of_data returns PArrayType[PDataType]' do + at = Puppet::Pops::Types::TypeFactory.array_of_data + at.class().should == Puppet::Pops::Types::PArrayType + at.element_type.class.should == Puppet::Pops::Types::PDataType + end + + it 'hash_of_data returns PHashType[PLiteralType,PDataType]' do + ht = Puppet::Pops::Types::TypeFactory.hash_of_data + ht.class().should == Puppet::Pops::Types::PHashType + ht.key_type.class.should == Puppet::Pops::Types::PLiteralType + ht.element_type.class.should == Puppet::Pops::Types::PDataType + end + + it 'ruby(1) returns PRubyType[\'Fixnum\']' do + ht = Puppet::Pops::Types::TypeFactory.ruby(1) + ht.class().should == Puppet::Pops::Types::PRubyType + ht.ruby_class.should == 'Fixnum' + end + end +end diff --git a/spec/unit/pops/types/type_parser_spec.rb b/spec/unit/pops/types/type_parser_spec.rb new file mode 100644 index 000000000..f0b9ea9a4 --- /dev/null +++ b/spec/unit/pops/types/type_parser_spec.rb @@ -0,0 +1,93 @@ +require 'spec_helper' +require 'puppet/pops' + +describe Puppet::Pops::Types::TypeParser do + extend RSpec::Matchers::DSL + + let(:parser) { Puppet::Pops::Types::TypeParser.new } + let(:types) { Puppet::Pops::Types::TypeFactory } + + it "rejects a puppet expression" do + expect { parser.parse("1 + 1") }.to raise_error(Puppet::ParseError, /The expression <1 \+ 1> is not a valid type specification/) + end + + it "rejects a empty type specification" do + expect { parser.parse("") }.to raise_error(Puppet::ParseError, /The expression <> is not a valid type specification/) + end + + it "rejects an invalid type simple type" do + expect { parser.parse("NotAType") }.to raise_type_error_for("NotAType") + end + + it "rejects an unknown parameterized type" do + expect { parser.parse("NotAType[Integer]") }.to raise_type_error_for("NotAType") + end + + it "does not support types that do not make sense in the puppet language" do + expect { parser.parse("Object") }.to raise_type_error_for("Object") + expect { parser.parse("Collection[Integer]") }.to raise_type_error_for("Collection") + end + + it "parses a simple, unparameterized type into the type object" do + expect(the_type_parsed_from(types.integer)).to be_the_type(types.integer) + expect(the_type_parsed_from(types.float)).to be_the_type(types.float) + expect(the_type_parsed_from(types.string)).to be_the_type(types.string) + expect(the_type_parsed_from(types.boolean)).to be_the_type(types.boolean) + expect(the_type_parsed_from(types.pattern)).to be_the_type(types.pattern) + expect(the_type_parsed_from(types.data)).to be_the_type(types.data) + end + + it "interprets an unparameterized Array as an Array of Data" do + expect(parser.parse("Array")).to be_the_type(types.array_of_data) + end + + it "interprets an unparameterized Hash as a Hash of Literal to Data" do + expect(parser.parse("Hash")).to be_the_type(types.hash_of_data) + end + + it "interprets a parameterized Hash[t] as a Hash of Literal to t" do + expect(parser.parse("Hash[Integer]")).to be_the_type(types.hash_of(types.integer)) + end + + it "parses a parameterized type into the type object" do + parameterized_array = types.array_of(types.integer) + parameterized_hash = types.hash_of(types.integer, types.boolean) + + expect(the_type_parsed_from(parameterized_array)).to be_the_type(parameterized_array) + expect(the_type_parsed_from(parameterized_hash)).to be_the_type(parameterized_hash) + end + + it "rejects an array spec with the wrong number of parameters" do + expect { parser.parse("Array[Integer, Integer]") }.to raise_the_parameter_error("Array", 1, 2) + expect { parser.parse("Hash[Integer, Integer, Integer]") }.to raise_the_parameter_error("Hash", "1 or 2", 3) + end + + matcher :be_the_type do |type| + calc = Puppet::Pops::Types::TypeCalculator.new + + match do |actual| + calc.assignable?(actual, type) && calc.assignable?(type, actual) + end + + failure_message_for_should do |actual| + "expected #{calc.string(type)}, but was #{calc.string(actual)}" + end + end + + def raise_the_parameter_error(type, required, given) + raise_error(Puppet::ParseError, /#{type} requires #{required}, #{given} provided/) + end + + def raise_type_error_for(type_name) + raise_error(Puppet::ParseError, /Unknown type <#{type_name}>/) + end + + def the_type_parsed_from(type) + parser.parse(the_type_spec_for(type)) + end + + def the_type_spec_for(type) + calc = Puppet::Pops::Types::TypeCalculator.new + calc.string(type) + end +end diff --git a/spec/unit/property/boolean_spec.rb b/spec/unit/property/boolean_spec.rb new file mode 100644 index 000000000..154fc9752 --- /dev/null +++ b/spec/unit/property/boolean_spec.rb @@ -0,0 +1,24 @@ +require 'spec_helper' +require 'puppet/property/boolean' + +describe Puppet::Property::Boolean do + let (:resource) { mock('resource') } + subject { described_class.new(:resource => resource) } + + [ true, :true, 'true', :yes, 'yes', 'TrUe', 'yEs' ].each do |arg| + it "should munge #{arg.inspect} as true" do + subject.munge(arg).should == true + end + end + [ false, :false, 'false', :no, 'no', 'FaLSE', 'nO' ].each do |arg| + it "should munge #{arg.inspect} as false" do + subject.munge(arg).should == false + end + end + [ nil, :undef, 'undef', '0', 0, '1', 1, 9284 ].each do |arg| + it "should fail to munge #{arg.inspect}" do + expect { subject.munge(arg) }.to raise_error Puppet::Error + end + end +end + diff --git a/spec/unit/property/list_spec.rb b/spec/unit/property/list_spec.rb index cac7dce65..e1782781f 100755 --- a/spec/unit/property/list_spec.rb +++ b/spec/unit/property/list_spec.rb @@ -44,7 +44,7 @@ describe list_class do @property.add_should_with_current(["foo"], ["bar"]).should == ["foo", "bar"] end - it "should return should if current is not a array" do + it "should return should if current is not an array" do @property.add_should_with_current(["foo"], :absent).should == ["foo"] end diff --git a/spec/unit/property/ordered_list_spec.rb b/spec/unit/property/ordered_list_spec.rb index bc84bd178..0155d8c51 100755 --- a/spec/unit/property/ordered_list_spec.rb +++ b/spec/unit/property/ordered_list_spec.rb @@ -24,7 +24,7 @@ describe ordered_list_class do @property.add_should_with_current(["should"], ["current"]).should == ["should", "current"] end - it "should return 'should' if current is not a array" do + it "should return 'should' if current is not an array" do @property.add_should_with_current(["should"], :absent).should == ["should"] end diff --git a/spec/unit/provider/aixobject_spec.rb b/spec/unit/provider/aixobject_spec.rb new file mode 100644 index 000000000..0aeccf3ef --- /dev/null +++ b/spec/unit/provider/aixobject_spec.rb @@ -0,0 +1,101 @@ +require 'spec_helper' +require 'puppet/provider/aixobject' + +describe Puppet::Provider::AixObject do + let(:resource) do + Puppet::Type.type(:user).new( + :name => 'test_aix_user', + :ensure => :present + ) + end + + let(:provider) do + provider = Puppet::Provider::AixObject.new resource + end + + describe "base provider methods" do + [ :lscmd, + :addcmd, + :modifycmd, + :deletecmd + ].each do |method| + it "should raise an error when unimplemented method #{method} called" do + lambda do + provider.send(method) + end.should raise_error(Puppet::Error, /not defined/) + end + end + end + + describe "attribute mapping methods" do + let(:mapping) do + [ + { :aix_attr => :test_aix_property, + :puppet_prop => :test_puppet_property, + :to => :test_convert_to_aix_method, + :from => :test_convert_to_puppet_method + } + ] + end + + before(:each) do + provider.class.attribute_mapping = mapping + end + + describe ".attribute_mapping_to" do + before(:each) do + if provider.class.instance_variable_defined? :@attribute_mapping_to + provider.class.send(:remove_instance_variable, :@attribute_mapping_to) + end + end + + it "should create a hash where the key is the puppet property and the value is a hash with the aix property and the conversion method" do + hash = provider.class.attribute_mapping_to + hash.should have_key :test_puppet_property + sub_hash = hash[:test_puppet_property] + sub_hash.should have_key :key + sub_hash.should have_key :method + sub_hash[:key].should == :test_aix_property + sub_hash[:method].should == :test_convert_to_aix_method + end + + it "should cache results between calls" do + provider.class.expects(:attribute_mapping).returns(mapping).once + provider.class.attribute_mapping_to + provider.class.attribute_mapping_to + end + end + + describe ".attribute_mapping_from" do + before(:each) do + if provider.class.instance_variable_defined? :@attribute_mapping_from + provider.class.send(:remove_instance_variable, :@attribute_mapping_from) + end + end + + it "should create a hash where the key is the aix property and the value is a hash with the puppet property and the conversion method" do + hash = provider.class.attribute_mapping_from + hash.should have_key :test_aix_property + sub_hash = hash[:test_aix_property] + sub_hash.should have_key :key + sub_hash.should have_key :method + sub_hash[:key].should == :test_puppet_property + sub_hash[:method].should == :test_convert_to_puppet_method + end + + it "should cache results between calls" do + provider.class.expects(:attribute_mapping).returns(mapping).once + provider.class.attribute_mapping_from + provider.class.attribute_mapping_from + end + end + end + + describe "#getinfo" do + it "should only execute the system command once" do + provider.stubs(:lscmd).returns "ls" + provider.expects(:execute).returns("bob=frank").once + provider.getinfo(true) + end + end +end
\ No newline at end of file diff --git a/spec/unit/provider/augeas/augeas_spec.rb b/spec/unit/provider/augeas/augeas_spec.rb index 514772ae4..e75a0d36e 100755 --- a/spec/unit/provider/augeas/augeas_spec.rb +++ b/spec/unit/provider/augeas/augeas_spec.rb @@ -650,28 +650,39 @@ describe provider_class do before do @augeas.expects(:match).with("/augeas//error").returns(["/augeas/files/foo/error"]) @augeas.expects(:match).with("/augeas/files/foo/error/*").returns(["/augeas/files/foo/error/path", "/augeas/files/foo/error/message"]) + @augeas.expects(:get).with("/augeas/files/foo/error").returns("some_failure") @augeas.expects(:get).with("/augeas/files/foo/error/path").returns("/foo") @augeas.expects(:get).with("/augeas/files/foo/error/message").returns("Failed to...") end it "and output to debug" do - @provider.expects(:debug).times(4) + @provider.expects(:debug).times(5) @provider.print_load_errors end it "and output a warning and to debug" do @provider.expects(:warning).once() - @provider.expects(:debug).times(3) + @provider.expects(:debug).times(4) @provider.print_load_errors(:warning => true) end end + it "should find load errors from lenses" do + @augeas.expects(:match).with("/augeas//error").returns(["/augeas/load/Xfm/error"]) + @augeas.expects(:match).with("/augeas/load/Xfm/error/*").returns([]) + @augeas.expects(:get).with("/augeas/load/Xfm/error").returns(["Could not find lens php.aug"]) + @provider.expects(:warning).once() + @provider.expects(:debug).twice() + @provider.print_load_errors(:warning => true) + end + it "should find save errors and output to debug" do @augeas.expects(:match).with("/augeas//error[. = 'put_failed']").returns(["/augeas/files/foo/error"]) @augeas.expects(:match).with("/augeas/files/foo/error/*").returns(["/augeas/files/foo/error/path", "/augeas/files/foo/error/message"]) + @augeas.expects(:get).with("/augeas/files/foo/error").returns("some_failure") @augeas.expects(:get).with("/augeas/files/foo/error/path").returns("/foo") @augeas.expects(:get).with("/augeas/files/foo/error/message").returns("Failed to...") - @provider.expects(:debug).times(4) + @provider.expects(:debug).times(5) @provider.print_put_errors end end diff --git a/spec/unit/provider/mcx/mcxcontent_spec.rb b/spec/unit/provider/mcx/mcxcontent_spec.rb index a8154d2ae..b015b226a 100755 --- a/spec/unit/provider/mcx/mcxcontent_spec.rb +++ b/spec/unit/provider/mcx/mcxcontent_spec.rb @@ -43,101 +43,136 @@ describe provider_class do @provider.should respond_to(:exists?) end - it "should have an content method." do + it "should have a content method." do @provider.should respond_to(:content) end - it "should have an content= method." do + it "should have a content= method." do @provider.should respond_to(:content=) end describe "when managing the resource" do it "should execute external command dscl from :create" do + @provider.stubs(:has_mcx?).returns(false) @provider.class.expects(:dscl).returns('').once @provider.create end + + it "deletes existing mcx prior to import from :create" do + @provider.stubs(:has_mcx?).returns(true) + @provider.class.expects(:dscl).with('localhost', '-mcxdelete', @ds_path, anything()).once + @provider.class.expects(:dscl).with('localhost', '-mcximport', @ds_path, anything()).once + @provider.create + end + it "should execute external command dscl from :destroy" do @provider.class.expects(:dscl).with('localhost', '-mcxdelete', @ds_path).returns('').once @provider.destroy end + it "should execute external command dscl from :exists?" do @provider.class.expects(:dscl).with('localhost', '-mcxexport', @ds_path).returns('').once @provider.exists? end + it "should execute external command dscl from :content" do @provider.class.expects(:dscl).with('localhost', '-mcxexport', @ds_path).returns('') @provider.content end + it "should execute external command dscl from :content=" do - @provider.class.expects(:dscl).returns('') + @provider.stubs(:has_mcx?).returns(false) + @provider.class.expects(:dscl).returns('').once + @provider.content='' + end + + it "deletes existing mcx prior to import from :content=" do + @provider.stubs(:has_mcx?).returns(true) + @provider.class.expects(:dscl).with('localhost', '-mcxdelete', @ds_path, anything()).once + @provider.class.expects(:dscl).with('localhost', '-mcximport', @ds_path, anything()).once @provider.content='' end end describe "when creating and parsing the name for ds_type" do before :each do + @provider.class.stubs(:dscl).returns('') @resource.stubs(:[]).with(:name).returns "/Foo/bar" end + it "should not accept /Foo/bar" do - lambda { @provider.create }.should raise_error(MCXContentProviderException) + expect { @provider.create }.to raise_error(MCXContentProviderException) end + it "should accept /Foo/bar with ds_type => user" do @resource.stubs(:[]).with(:ds_type).returns "user" - lambda { @provider.create }.should_not raise_error(MCXContentProviderException) + expect { @provider.create }.to_not raise_error end + it "should accept /Foo/bar with ds_type => group" do @resource.stubs(:[]).with(:ds_type).returns "group" - lambda { @provider.create }.should_not raise_error(MCXContentProviderException) + expect { @provider.create }.to_not raise_error end + it "should accept /Foo/bar with ds_type => computer" do @resource.stubs(:[]).with(:ds_type).returns "computer" - lambda { @provider.create }.should_not raise_error(MCXContentProviderException) + expect { @provider.create }.to_not raise_error end + it "should accept :name => /Foo/bar with ds_type => computerlist" do @resource.stubs(:[]).with(:ds_type).returns "computerlist" - lambda { @provider.create }.should_not raise_error(MCXContentProviderException) + expect { @provider.create }.to_not raise_error end end describe "when creating and :name => foobar" do before :each do + @provider.class.stubs(:dscl).returns('') @resource.stubs(:[]).with(:name).returns "foobar" end + it "should not accept unspecified :ds_type and :ds_name" do - lambda { @provider.create }.should raise_error(MCXContentProviderException) + expect { @provider.create }.to raise_error(MCXContentProviderException) end + it "should not accept unspecified :ds_type" do @resource.stubs(:[]).with(:ds_type).returns "user" - lambda { @provider.create }.should raise_error(MCXContentProviderException) + expect { @provider.create }.to raise_error(MCXContentProviderException) end + it "should not accept unspecified :ds_name" do @resource.stubs(:[]).with(:ds_name).returns "foo" - lambda { @provider.create }.should raise_error(MCXContentProviderException) + expect { @provider.create }.to raise_error(MCXContentProviderException) end + it "should accept :ds_type => user, ds_name => foo" do @resource.stubs(:[]).with(:ds_type).returns "user" @resource.stubs(:[]).with(:ds_name).returns "foo" - lambda { @provider.create }.should_not raise_error(MCXContentProviderException) + expect { @provider.create }.to_not raise_error end + it "should accept :ds_type => group, ds_name => foo" do @resource.stubs(:[]).with(:ds_type).returns "group" @resource.stubs(:[]).with(:ds_name).returns "foo" - lambda { @provider.create }.should_not raise_error(MCXContentProviderException) + expect { @provider.create }.to_not raise_error end + it "should accept :ds_type => computer, ds_name => foo" do @resource.stubs(:[]).with(:ds_type).returns "computer" @resource.stubs(:[]).with(:ds_name).returns "foo" - lambda { @provider.create }.should_not raise_error(MCXContentProviderException) + expect { @provider.create }.to_not raise_error end + it "should accept :ds_type => computerlist, ds_name => foo" do @resource.stubs(:[]).with(:ds_type).returns "computerlist" @resource.stubs(:[]).with(:ds_name).returns "foo" - lambda { @provider.create }.should_not raise_error(MCXContentProviderException) + expect { @provider.create }.to_not raise_error end + it "should not accept :ds_type => bogustype, ds_name => foo" do @resource.stubs(:[]).with(:ds_type).returns "bogustype" @resource.stubs(:[]).with(:ds_name).returns "foo" - lambda { @provider.create }.should raise_error(MCXContentProviderException) + expect { @provider.create }.to raise_error(MCXContentProviderException) end end @@ -145,6 +180,7 @@ describe provider_class do it "should define an instances class method." do @provider.class.should respond_to(:instances) end + it "should call external command dscl -list /Local/Default/<ds_type> on each known ds_type" do @provider.class.expects(:dscl).with('localhost', '-list', "/Local/Default/Users").returns('') @provider.class.expects(:dscl).with('localhost', '-list', "/Local/Default/Groups").returns('') diff --git a/spec/unit/provider/mount/parsed_spec.rb b/spec/unit/provider/mount/parsed_spec.rb index 74b67bb03..196dd4635 100755 --- a/spec/unit/provider/mount/parsed_spec.rb +++ b/spec/unit/provider/mount/parsed_spec.rb @@ -2,105 +2,93 @@ require 'spec_helper' require 'shared_behaviours/all_parsedfile_providers' -provider_class = Puppet::Type.type(:mount).provider(:parsed) +describe Puppet::Type.type(:mount).provider(:parsed), :unless => Puppet.features.microsoft_windows? do -describe provider_class, :unless => Puppet.features.microsoft_windows? do + let :vfstab_sample do + "/dev/dsk/c0d0s0 /dev/rdsk/c0d0s0 \t\t / \t ufs 1 no\t-" + end - before :each do - @mount_class = Puppet::Type.type(:mount) - @provider = @mount_class.provider(:parsed) + let :fstab_sample do + "/dev/vg00/lv01\t/spare \t \t ext3 defaults\t1 2" end # LAK:FIXME I can't mock Facter because this test happens at parse-time. it "should default to /etc/vfstab on Solaris" do pending "This test only works on Solaris" unless Facter.value(:osfamily) == 'Solaris' - Puppet::Type.type(:mount).provider(:parsed).default_target.should == '/etc/vfstab' + described_class.default_target.should == '/etc/vfstab' end it "should default to /etc/fstab on anything else" do pending "This test does not work on Solaris" if Facter.value(:osfamily) == 'Solaris' - Puppet::Type.type(:mount).provider(:parsed).default_target.should == '/etc/fstab' + described_class.default_target.should == '/etc/fstab' end describe "when parsing a line" do - it "should not crash on incomplete lines in fstab" do - parse = @provider.parse <<-FSTAB + parse = described_class.parse <<-FSTAB /dev/incomplete /dev/device name FSTAB - lambda{ @provider.to_line(parse[0]) }.should_not raise_error + expect { described_class.to_line(parse[0]) }.to_not raise_error end # it_should_behave_like "all parsedfile providers", # provider_class, my_fixtures('*.fstab') describe "on Solaris", :if => Facter.value(:osfamily) == 'Solaris' do - - before :each do - @example_line = "/dev/dsk/c0d0s0 /dev/rdsk/c0d0s0 \t\t / \t ufs 1 no\t-" - end - it "should extract device from the first field" do - @provider.parse_line(@example_line)[:device].should == '/dev/dsk/c0d0s0' + described_class.parse_line(vfstab_sample)[:device].should == '/dev/dsk/c0d0s0' end it "should extract blockdevice from second field" do - @provider.parse_line(@example_line)[:blockdevice].should == "/dev/rdsk/c0d0s0" + described_class.parse_line(vfstab_sample)[:blockdevice].should == "/dev/rdsk/c0d0s0" end it "should extract name from third field" do - @provider.parse_line(@example_line)[:name].should == "/" + described_class.parse_line(vfstab_sample)[:name].should == "/" end it "should extract fstype from fourth field" do - @provider.parse_line(@example_line)[:fstype].should == "ufs" + described_class.parse_line(vfstab_sample)[:fstype].should == "ufs" end it "should extract pass from fifth field" do - @provider.parse_line(@example_line)[:pass].should == "1" + described_class.parse_line(vfstab_sample)[:pass].should == "1" end it "should extract atboot from sixth field" do - @provider.parse_line(@example_line)[:atboot].should == "no" + described_class.parse_line(vfstab_sample)[:atboot].should == "no" end it "should extract options from seventh field" do - @provider.parse_line(@example_line)[:options].should == "-" + described_class.parse_line(vfstab_sample)[:options].should == "-" end - end describe "on other platforms than Solaris", :if => Facter.value(:osfamily) != 'Solaris' do - - before :each do - @example_line = "/dev/vg00/lv01\t/spare \t \t ext3 defaults\t1 2" - end - it "should extract device from the first field" do - @provider.parse_line(@example_line)[:device].should == '/dev/vg00/lv01' + described_class.parse_line(fstab_sample)[:device].should == '/dev/vg00/lv01' end it "should extract name from second field" do - @provider.parse_line(@example_line)[:name].should == "/spare" + described_class.parse_line(fstab_sample)[:name].should == "/spare" end it "should extract fstype from third field" do - @provider.parse_line(@example_line)[:fstype].should == "ext3" + described_class.parse_line(fstab_sample)[:fstype].should == "ext3" end it "should extract options from fourth field" do - @provider.parse_line(@example_line)[:options].should == "defaults" + described_class.parse_line(fstab_sample)[:options].should == "defaults" end it "should extract dump from fifth field" do - @provider.parse_line(@example_line)[:dump].should == "1" + described_class.parse_line(fstab_sample)[:dump].should == "1" end it "should extract options from sixth field" do - @provider.parse_line(@example_line)[:pass].should == "2" + described_class.parse_line(fstab_sample)[:pass].should == "2" end - end end @@ -108,8 +96,8 @@ FSTAB describe "mountinstances" do it "should get name from mountoutput found on Solaris" do Facter.stubs(:value).with(:osfamily).returns 'Solaris' - @provider.stubs(:mountcmd).returns(File.read(my_fixture('solaris.mount'))) - mounts = @provider.mountinstances + described_class.stubs(:mountcmd).returns(File.read(my_fixture('solaris.mount'))) + mounts = described_class.mountinstances mounts.size.should == 6 mounts[0].should == { :name => '/', :mounted => :yes } mounts[1].should == { :name => '/proc', :mounted => :yes } @@ -121,8 +109,8 @@ FSTAB it "should get name from mountoutput found on HP-UX" do Facter.stubs(:value).with(:osfamily).returns 'HP-UX' - @provider.stubs(:mountcmd).returns(File.read(my_fixture('hpux.mount'))) - mounts = @provider.mountinstances + described_class.stubs(:mountcmd).returns(File.read(my_fixture('hpux.mount'))) + mounts = described_class.mountinstances mounts.size.should == 17 mounts[0].should == { :name => '/', :mounted => :yes } mounts[1].should == { :name => '/devices', :mounted => :yes } @@ -145,8 +133,8 @@ FSTAB it "should get name from mountoutput found on Darwin" do Facter.stubs(:value).with(:osfamily).returns 'Darwin' - @provider.stubs(:mountcmd).returns(File.read(my_fixture('darwin.mount'))) - mounts = @provider.mountinstances + described_class.stubs(:mountcmd).returns(File.read(my_fixture('darwin.mount'))) + mounts = described_class.mountinstances mounts.size.should == 6 mounts[0].should == { :name => '/', :mounted => :yes } mounts[1].should == { :name => '/dev', :mounted => :yes } @@ -158,8 +146,8 @@ FSTAB it "should get name from mountoutput found on Linux" do Facter.stubs(:value).with(:osfamily).returns 'Gentoo' - @provider.stubs(:mountcmd).returns(File.read(my_fixture('linux.mount'))) - mounts = @provider.mountinstances + described_class.stubs(:mountcmd).returns(File.read(my_fixture('linux.mount'))) + mounts = described_class.mountinstances mounts[0].should == { :name => '/', :mounted => :yes } mounts[1].should == { :name => '/lib64/rc/init.d', :mounted => :yes } mounts[2].should == { :name => '/sys', :mounted => :yes } @@ -169,8 +157,8 @@ FSTAB it "should get name from mountoutput found on AIX" do Facter.stubs(:value).with(:osfamily).returns 'AIX' - @provider.stubs(:mountcmd).returns(File.read(my_fixture('aix.mount'))) - mounts = @provider.mountinstances + described_class.stubs(:mountcmd).returns(File.read(my_fixture('aix.mount'))) + mounts = described_class.mountinstances mounts[0].should == { :name => '/', :mounted => :yes } mounts[1].should == { :name => '/tmp', :mounted => :yes } mounts[2].should == { :name => '/home', :mounted => :yes } @@ -179,8 +167,8 @@ FSTAB end it "should raise an error if a line is not understandable" do - @provider.stubs(:mountcmd).returns("bazinga!") - lambda { @provider.mountinstances }.should raise_error Puppet::Error + described_class.stubs(:mountcmd).returns("bazinga!") + expect { described_class.mountinstances }.to raise_error Puppet::Error, 'Could not understand line bazinga! from mount output' end end @@ -203,7 +191,7 @@ FSTAB # Stub the mount output to our fixture. begin mount = my_fixture(platform + '.mount') - @provider.stubs(:mountcmd).returns File.read(mount) + described_class.stubs(:mountcmd).returns File.read(mount) rescue pending "is #{platform}.mount missing at this point?" end @@ -211,8 +199,8 @@ FSTAB # Note: we have to stub default_target before creating resources # because it is used by Puppet::Type::Mount.new to populate the # :target property. - @provider.stubs(:default_target).returns fstab - @retrieve = @provider.instances.collect { |prov| {:name => prov.get(:name), :ensure => prov.get(:ensure)}} + described_class.stubs(:default_target).returns fstab + @retrieve = described_class.instances.collect { |prov| {:name => prov.get(:name), :ensure => prov.get(:ensure)}} end # Following mountpoint are present in all fstabs/mountoutputs @@ -246,7 +234,7 @@ FSTAB # Stub the mount output to our fixture. begin mount = my_fixture(platform + '.mount') - @provider.stubs(:mountcmd).returns File.read(mount) + described_class.stubs(:mountcmd).returns File.read(mount) rescue pending "is #{platform}.mount missing at this point?" end @@ -254,7 +242,7 @@ FSTAB # Note: we have to stub default_target before creating resources # because it is used by Puppet::Type::Mount.new to populate the # :target property. - @provider.stubs(:default_target).returns fstab + described_class.stubs(:default_target).returns fstab @res_ghost = Puppet::Type::Mount.new(:name => '/ghost') # in no fake fstab @res_mounted = Puppet::Type::Mount.new(:name => '/') # in every fake fstab @@ -270,24 +258,24 @@ FSTAB it "should set :ensure to :unmounted if found in fstab but not mounted" do pending("Solaris:Unable to stub Operating System Fact at runtime", :if => Facter.value(:osfamily) == "Solaris") - @provider.prefetch(@resource_hash) + described_class.prefetch(@resource_hash) @res_unmounted.provider.get(:ensure).should == :unmounted end it "should set :ensure to :ghost if not found in fstab but mounted" do pending("Solaris:Unable to stub Operating System Fact at runtime", :if => Facter.value(:osfamily) == "Solaris") - @provider.prefetch(@resource_hash) + described_class.prefetch(@resource_hash) @res_ghost.provider.get(:ensure).should == :ghost end it "should set :ensure to :mounted if found in fstab and mounted" do pending("Solaris:Unable to stub Operating System Fact at runtime", :if => Facter.value(:osfamily) == "Solaris") - @provider.prefetch(@resource_hash) + described_class.prefetch(@resource_hash) @res_mounted.provider.get(:ensure).should == :mounted end it "should set :ensure to :absent if not found in fstab and not mounted" do - @provider.prefetch(@resource_hash) + described_class.prefetch(@resource_hash) @res_absent.provider.get(:ensure).should == :absent end end diff --git a/spec/unit/provider/mount_spec.rb b/spec/unit/provider/mount_spec.rb index b620025d4..34d819c57 100755 --- a/spec/unit/provider/mount_spec.rb +++ b/spec/unit/provider/mount_spec.rb @@ -29,9 +29,18 @@ describe Puppet::Provider::Mount do @mounter.mount end - it "should add the options following '-o' if they exist and are not set to :absent" do + it "should add the options following '-o' on MacOS if they exist and are not set to :absent" do + Facter.expects(:value).with(:kernel).returns 'Darwin' @mounter.stubs(:options).returns("ro") - @mounter.expects(:mountcmd).with { |*ary| ary[0] == "-o" and ary[1] == "ro" } + @mounter.expects(:mountcmd).with '-o', 'ro', '/' + + @mounter.mount + end + + it "should not explicitly pass mount options on systems other than MacOS" do + Facter.expects(:value).with(:kernel).returns 'HP-UX' + @mounter.stubs(:options).returns("ro") + @mounter.expects(:mountcmd).with '/' @mounter.mount end diff --git a/spec/unit/provider/naginator_spec.rb b/spec/unit/provider/naginator_spec.rb index d5e4a2ab0..0fa5b7796 100755 --- a/spec/unit/provider/naginator_spec.rb +++ b/spec/unit/provider/naginator_spec.rb @@ -55,3 +55,11 @@ describe Puppet::Provider::Naginator do @class.should_not be_skip_record("foo") end end + +describe Nagios::Base do + it "should not turn set parameters into arrays #17871" do + obj = Nagios::Base.create('host') + obj.host_name = "my_hostname" + obj.host_name.should == "my_hostname" + end +end diff --git a/spec/unit/provider/package/apt_spec.rb b/spec/unit/provider/package/apt_spec.rb index 55ff321d0..5e75a569f 100755 --- a/spec/unit/provider/package/apt_spec.rb +++ b/spec/unit/provider/package/apt_spec.rb @@ -8,7 +8,11 @@ describe provider do @resource = stub 'resource', :[] => "asdf" @provider = provider.new(@resource) - @fakeresult = "install ok installed asdf 1.0\n" + @fakeresult = <<-EOF +install ok installed asdf 1.0 "asdf summary + asdf multiline description + with multiple lines +EOF end it "should be versionable" do diff --git a/spec/unit/provider/package/aptitude_spec.rb b/spec/unit/provider/package/aptitude_spec.rb index 6e678a158..d785bfe0d 100755 --- a/spec/unit/provider/package/aptitude_spec.rb +++ b/spec/unit/provider/package/aptitude_spec.rb @@ -10,13 +10,17 @@ describe Puppet::Type.type(:package).provider(:aptitude) do it { should be_versionable } context "when retrieving ensure" do - { :absent => "deinstall ok config-files faff 1.2.3-1\n", - "1.2.3-1" => "install ok installed faff 1.2.3-1\n", + before do + described_class.stubs(:command).with(:dpkgquery).returns 'myquery' + end + + { :absent => "deinstall ok config-files faff 1.2.3-1 :DESC: faff summary\n:DESC:\n", + "1.2.3-1" => "install ok installed faff 1.2.3-1 :DESC: faff summary\n:DESC:\n", }.each do |expect, output| it "should detect #{expect} packages" do - pkg.provider.expects(:dpkgquery). - with('-W', '--showformat', '${Status} ${Package} ${Version}\n', 'faff'). - returns(output) + Puppet::Util::Execution.expects(:execpipe). + with(['myquery', '-W', '--showformat', "'${Status} ${Package} ${Version} :DESC: ${Description}\\n:DESC:\\n'", 'faff']). + yields(StringIO.new(output)) pkg.property(:ensure).retrieve.should == expect end diff --git a/spec/unit/provider/package/aptrpm_spec.rb b/spec/unit/provider/package/aptrpm_spec.rb index bc07c60c9..6c77bee3a 100755 --- a/spec/unit/provider/package/aptrpm_spec.rb +++ b/spec/unit/provider/package/aptrpm_spec.rb @@ -19,7 +19,7 @@ describe Puppet::Type.type(:package).provider(:aptrpm) do def rpm pkg.provider.expects(:rpm). with('-q', 'faff', '--nosignature', '--nodigest', '--qf', - "%{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}\n") + "'%{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH} :DESC: %{SUMMARY}\\n'") end it "should report absent packages" do @@ -28,7 +28,7 @@ describe Puppet::Type.type(:package).provider(:aptrpm) do end it "should report present packages correctly" do - rpm.returns("faff-1.2.3-1 0 1.2.3-1 5 i686\n") + rpm.returns("faff-1.2.3-1 0 1.2.3-1 5 i686 :DESC: faff desc\n") pkg.property(:ensure).retrieve.should == "1.2.3-1-5" end end diff --git a/spec/unit/provider/package/dpkg_spec.rb b/spec/unit/provider/package/dpkg_spec.rb index 7c809cf7d..6990ef454 100755 --- a/spec/unit/provider/package/dpkg_spec.rb +++ b/spec/unit/provider/package/dpkg_spec.rb @@ -2,226 +2,401 @@ require 'spec_helper' require 'stringio' -provider = Puppet::Type.type(:package).provider(:dpkg) +provider_class = Puppet::Type.type(:package).provider(:dpkg) + +describe provider_class do + let(:bash_version) { '4.2-5ubuntu3' } + let(:bash_installed_output) do <<-EOS +install ok installed bash #{bash_version} :DESC: GNU Bourne Again SHell + Bash is an sh-compatible command language interpreter that executes + commands read from the standard input or from a file. Bash also + incorporates useful features from the Korn and C shells (ksh and csh). + . + Bash is ultimately intended to be a conformant implementation of the + IEEE POSIX Shell and Tools specification (IEEE Working Group 1003.2). + . + The Programmable Completion Code, by Ian Macdonald, is now found in + the bash-completion package. +:DESC: + EOS + end + let(:bash_installed_io) { StringIO.new(bash_installed_output) } + + let(:vim_installed_output) do <<-EOS +install ok installed vim 2:7.3.547-6ubuntu5 :DESC: Vi IMproved - enhanced vi editor + Vim is an almost compatible version of the UNIX editor Vi. + . + Many new features have been added: multi level undo, syntax + highlighting, command line history, on-line help, filename + completion, block operations, folding, Unicode support, etc. + . + This package contains a version of vim compiled with a rather + standard set of features. This package does not provide a GUI + version of Vim. See the other vim-* packages if you need more + (or less). +:DESC: + EOS + end -describe provider do - before do - @resource = stub 'resource', :[] => "asdf" - @provider = provider.new(@resource) - @provider.expects(:execute).never # forbid "manual" executions + let(:all_installed_io) { StringIO.new([bash_installed_output, vim_installed_output].join) } + let(:args) { ['myquery', '-W', '--showformat', %Q{'${Status} ${Package} ${Version} :DESC: ${Description}\\n:DESC:\\n'}] } + let(:resource_name) { 'package' } + let(:resource) { stub 'resource', :[] => resource_name } + let(:provider) { provider_class.new(resource) } - @fakeresult = "install ok installed asdf 1.0\n" + before do + provider_class.stubs(:command).with(:dpkgquery).returns 'myquery' end it "should have documentation" do - provider.doc.should be_instance_of(String) + provider_class.doc.should be_instance_of(String) end describe "when listing all instances" do - before do - provider.stubs(:command).with(:dpkgquery).returns "myquery" - end it "should use dpkg-query" do - provider.expects(:command).with(:dpkgquery).returns "myquery" - Puppet::Util::Execution.expects(:execpipe).with("myquery -W --showformat '${Status} ${Package} ${Version}\\n'").yields StringIO.new(@fakeresult) + Puppet::Util::Execution.expects(:execpipe).with(args).yields bash_installed_io - provider.instances + provider_class.instances end - it "should create and return an instance with each parsed line from dpkg-query" do - pipe = mock 'pipe' - pipe.expects(:each).never - pipe.expects(:each_line).yields @fakeresult - Puppet::Util::Execution.expects(:execpipe).yields pipe + it "should create and return an instance for a single dpkg-query entry" do + Puppet::Util::Execution.expects(:execpipe).with(args).yields bash_installed_io - asdf = mock 'pkg1' - provider.expects(:new).with(:ensure => "1.0", :error => "ok", :desired => "install", :name => "asdf", :status => "installed", :provider => :dpkg).returns asdf + installed = mock 'bash' + provider_class.expects(:new).with(:ensure => "4.2-5ubuntu3", :error => "ok", :desired => "install", :name => "bash", :status => "installed", :description => "GNU Bourne Again SHell", :provider => :dpkg).returns installed - provider.instances.should == [asdf] + provider_class.instances.should == [installed] + end + + it "should parse multiple dpkg-query multi-line entries in the output" do + Puppet::Util::Execution.expects(:execpipe).with(args).yields all_installed_io + + bash = mock 'bash' + provider_class.expects(:new).with(:ensure => "4.2-5ubuntu3", :error => "ok", :desired => "install", :name => "bash", :status => "installed", :description => "GNU Bourne Again SHell", :provider => :dpkg).returns bash + vim = mock 'vim' + provider_class.expects(:new).with(:ensure => "2:7.3.547-6ubuntu5", :error => "ok", :desired => "install", :name => "vim", :status => "installed", :description => "Vi IMproved - enhanced vi editor", :provider => :dpkg).returns vim + + provider_class.instances.should == [bash, vim] end it "should warn on and ignore any lines it does not understand" do - pipe = mock 'pipe' - pipe.expects(:each).never - pipe.expects(:each_line).yields "foobar" - Puppet::Util::Execution.expects(:execpipe).yields pipe + Puppet::Util::Execution.expects(:execpipe).with(args).yields StringIO.new('foobar') Puppet.expects(:warning) - provider.expects(:new).never + provider_class.expects(:new).never + + provider_class.instances.should == [] + end + + it "should not warn on extra multiline description lines which we are ignoring" do + Puppet::Util::Execution.expects(:execpipe).with(args).yields all_installed_io - provider.instances.should == [] + Puppet.expects(:warning).never + provider_class.instances + end + + it "should warn if encounters bad lines between good entries without failing" do + Puppet::Util::Execution.expects(:execpipe).with(args).yields StringIO.new([bash_installed_output, "foobar\n", vim_installed_output].join) + + Puppet.expects(:warning) + + bash = mock 'bash' + vim = mock 'vim' + provider_class.expects(:new).twice.returns(bash, vim) + + provider_class.instances.should == [bash, vim] + end + + it "should warn on a broken entry while still parsing a good one" do + Puppet::Util::Execution.expects(:execpipe).with(args).yields StringIO.new([ + bash_installed_output, + %Q{install ok installed broken 1.0 this shouldn't be here :DESC: broken description\n extra description\n:DESC:\n}, + vim_installed_output, + ].join) + + Puppet.expects(:warning).times(3) + + bash = mock('bash') + vim = mock('vim') + saved = mock('saved') + provider_class.expects(:new).twice.returns(bash, vim) + + provider_class.instances.should == [bash, vim] end end describe "when querying the current state" do - it "should use dpkg-query" do - @provider.expects(:dpkgquery).with("-W", "--showformat",'${Status} ${Package} ${Version}\\n', "asdf").returns @fakeresult + let(:query_args) { args.push(resource_name) } - @provider.query + before do + provider.expects(:execute).never # forbid "manual" executions + end + + # @return [StringIO] of bash dpkg-query output with :search string replaced + # by :replace string. + def replace_in_bash_output(search, replace) + StringIO.new(bash_installed_output.gsub(search, replace)) + end + + it "should use exec-pipe" do + Puppet::Util::Execution.expects(:execpipe).with(query_args).yields bash_installed_io + + provider.query end it "should consider the package purged if dpkg-query fails" do - @provider.expects(:dpkgquery).raises Puppet::ExecutionFailure.new("eh") + Puppet::Util::Execution.expects(:execpipe).with(query_args).raises Puppet::ExecutionFailure.new("eh") - @provider.query[:ensure].should == :purged + provider.query[:ensure].should == :purged end - it "should return a hash of the found status with the desired state, error state, status, name, and 'ensure'" do - @provider.expects(:dpkgquery).returns @fakeresult + it "should return a hash of the found package status for an installed package" do + Puppet::Util::Execution.expects(:execpipe).with(query_args).yields bash_installed_io - @provider.query.should == {:ensure => "1.0", :error => "ok", :desired => "install", :name => "asdf", :status => "installed", :provider => :dpkg} + provider.query.should == {:ensure => "4.2-5ubuntu3", :error => "ok", :desired => "install", :name => "bash", :status => "installed", :provider => :dpkg, :description => "GNU Bourne Again SHell"} end it "should consider the package absent if the dpkg-query result cannot be interpreted" do - @provider.expects(:dpkgquery).returns "somebaddata" + Puppet::Util::Execution.expects(:execpipe).with(query_args).yields StringIO.new("somebaddata") - @provider.query[:ensure].should == :absent + provider.query[:ensure].should == :absent end it "should fail if an error is discovered" do - @provider.expects(:dpkgquery).returns @fakeresult.sub("ok", "error") + Puppet::Util::Execution.expects(:execpipe).with(query_args).yields replace_in_bash_output("ok", "error") - lambda { @provider.query }.should raise_error(Puppet::Error) + lambda { provider.query }.should raise_error(Puppet::Error) end it "should consider the package purged if it is marked 'not-installed'" do - @provider.expects(:dpkgquery).returns @fakeresult.sub("installed", "not-installed") + not_installed_bash = bash_installed_output.gsub("installed", "not-installed") + not_installed_bash.gsub!(bash_version, "") + Puppet::Util::Execution.expects(:execpipe).with(query_args).yields StringIO.new(not_installed_bash) - @provider.query[:ensure].should == :purged + provider.query[:ensure].should == :purged end it "should consider the package absent if it is marked 'config-files'" do - @provider.expects(:dpkgquery).returns @fakeresult.sub("installed", "config-files") - @provider.query[:ensure].should == :absent + Puppet::Util::Execution.expects(:execpipe).with(query_args).yields replace_in_bash_output("installed", "config-files") + provider.query[:ensure].should == :absent end it "should consider the package absent if it is marked 'half-installed'" do - @provider.expects(:dpkgquery).returns @fakeresult.sub("installed", "half-installed") - @provider.query[:ensure].should == :absent + Puppet::Util::Execution.expects(:execpipe).with(query_args).yields replace_in_bash_output("installed", "half-installed") + provider.query[:ensure].should == :absent end it "should consider the package absent if it is marked 'unpacked'" do - @provider.expects(:dpkgquery).returns @fakeresult.sub("installed", "unpacked") - @provider.query[:ensure].should == :absent + Puppet::Util::Execution.expects(:execpipe).with(query_args).yields replace_in_bash_output("installed", "unpacked") + provider.query[:ensure].should == :absent end it "should consider the package absent if it is marked 'half-configured'" do - @provider.expects(:dpkgquery).returns @fakeresult.sub("installed", "half-configured") - @provider.query[:ensure].should == :absent + Puppet::Util::Execution.expects(:execpipe).with(query_args).yields replace_in_bash_output("installed", "half-configured") + provider.query[:ensure].should == :absent end it "should consider the package held if its state is 'hold'" do - @provider.expects(:dpkgquery).returns @fakeresult.sub("install", "hold") - @provider.query[:ensure].should == :held + Puppet::Util::Execution.expects(:execpipe).with(query_args).yields replace_in_bash_output("install", "hold") + provider.query[:ensure].should == :held + end + end + + describe "parsing tests" do + let(:resource_name) { 'name' } + let(:package_hash) do + { + :desired => 'desired', + :error => 'ok', + :status => 'status', + :name => resource_name, + :ensure => 'ensure', + :description => 'summary text', + :provider => :dpkg, + } + end + let(:query_args) { args.push(resource_name) } + + it "warns about excess lines if encounters a delimiter in description but does not fail" do + broken_description = <<-EOS +desired ok status name ensure :DESC: summary text + more description +:DESC: + 1 whoops ^^ should not happen, because dpkg-query is supposed to prefix description lines with + 2 whitespace. So we should see three warnings for these four additional lines when we try + 3 and process next-pkg (vv the :DESC: is line number 4) +:DESC: +desired ok status next-pkg ensure :DESC: next summary +:DESC: + EOS + Puppet.expects(:warning).times(4) + + pipe = StringIO.new(broken_description) + provider_class.parse_multi_line(pipe).should == package_hash + + next_package = package_hash.merge(:name => 'next-pkg', :description => 'next summary') + + hash = provider_class.parse_multi_line(pipe) until hash # warn about bad lines + hash.should == next_package + end + + def parser_test(dpkg_output_string, gold_hash) + pipe = StringIO.new(dpkg_output_string) + Puppet::Util::Execution.expects(:execpipe).with(query_args).yields pipe + Puppet.expects(:warning).never + + provider.query.should == gold_hash + end + + it "should parse properly even if delimiter is in version" do + version_delimiter = <<-EOS +desired ok status name 1.2.3-:DESC: :DESC: summary text + more description +:DESC: + EOS + parser_test(version_delimiter, package_hash.merge(:ensure => '1.2.3-:DESC:')) + end + + it "should parse properly even if delimiter is name" do + name_delimiter = <<-EOS +desired ok status :DESC: ensure :DESC: summary text + more description +:DESC: + EOS + parser_test(name_delimiter, package_hash.merge(:name => ':DESC:')) + end + + it "should parse properly even if optional ensure field is missing" do + no_ensure = <<-EOS +desired ok status name :DESC: summary text + more description and note^ two spaces surround the hole where 'ensure' field would be... +:DESC: + EOS + parser_test(no_ensure, package_hash.merge(:ensure => '')) + end + + it "should parse properly even if extra delimiter is in summary" do + extra_description_delimiter = <<-EOS +desired ok status name ensure :DESC: summary text + :DESC: should be completely ignored because of leading space which dpkg-query should ensure +:DESC: + EOS + parser_test(extra_description_delimiter, package_hash) + end + + it "should parse properly even if package description is completely missing" do + no_description = "desired ok status name ensure :DESC: \n:DESC:" + parser_test(no_description, package_hash.merge(:description => '')) end end it "should be able to install" do - @provider.should respond_to(:install) + provider.should respond_to(:install) end describe "when installing" do before do - @resource.stubs(:[]).with(:source).returns "mypkg" + resource.stubs(:[]).with(:source).returns "mypkg" end it "should fail to install if no source is specified in the resource" do - @resource.expects(:[]).with(:source).returns nil + resource.expects(:[]).with(:source).returns nil - lambda { @provider.install }.should raise_error(ArgumentError) + lambda { provider.install }.should raise_error(ArgumentError) end it "should use 'dpkg -i' to install the package" do - @resource.expects(:[]).with(:source).returns "mypackagefile" - @provider.expects(:unhold) - @provider.expects(:dpkg).with { |*command| command[-1] == "mypackagefile" and command[-2] == "-i" } + resource.expects(:[]).with(:source).returns "mypackagefile" + provider.expects(:unhold) + provider.expects(:dpkg).with { |*command| command[-1] == "mypackagefile" and command[-2] == "-i" } - @provider.install + provider.install end it "should keep old config files if told to do so" do - @resource.expects(:[]).with(:configfiles).returns :keep - @provider.expects(:unhold) - @provider.expects(:dpkg).with { |*command| command[0] == "--force-confold" } + resource.expects(:[]).with(:configfiles).returns :keep + provider.expects(:unhold) + provider.expects(:dpkg).with { |*command| command[0] == "--force-confold" } - @provider.install + provider.install end it "should replace old config files if told to do so" do - @resource.expects(:[]).with(:configfiles).returns :replace - @provider.expects(:unhold) - @provider.expects(:dpkg).with { |*command| command[0] == "--force-confnew" } + resource.expects(:[]).with(:configfiles).returns :replace + provider.expects(:unhold) + provider.expects(:dpkg).with { |*command| command[0] == "--force-confnew" } - @provider.install + provider.install end it "should ensure any hold is removed" do - @provider.expects(:unhold).once - @provider.expects(:dpkg) - @provider.install + provider.expects(:unhold).once + provider.expects(:dpkg) + provider.install end end describe "when holding or unholding" do + let(:tempfile) { stub 'tempfile', :print => nil, :close => nil, :flush => nil, :path => "/other/file" } + before do - @tempfile = stub 'tempfile', :print => nil, :close => nil, :flush => nil, :path => "/other/file" - @tempfile.stubs(:write) - Tempfile.stubs(:new).returns @tempfile + tempfile.stubs(:write) + Tempfile.stubs(:new).returns tempfile end it "should install first if holding" do - @provider.stubs(:execute) - @provider.expects(:install).once - @provider.hold + provider.stubs(:execute) + provider.expects(:install).once + provider.hold end it "should execute dpkg --set-selections when holding" do - @provider.stubs(:install) - @provider.expects(:execute).with([:dpkg, '--set-selections'], {:failonfail => false, :combine => false, :stdinfile => @tempfile.path}).once - @provider.hold + provider.stubs(:install) + provider.expects(:execute).with([:dpkg, '--set-selections'], {:failonfail => false, :combine => false, :stdinfile => tempfile.path}).once + provider.hold end it "should execute dpkg --set-selections when unholding" do - @provider.stubs(:install) - @provider.expects(:execute).with([:dpkg, '--set-selections'], {:failonfail => false, :combine => false, :stdinfile => @tempfile.path}).once - @provider.hold + provider.stubs(:install) + provider.expects(:execute).with([:dpkg, '--set-selections'], {:failonfail => false, :combine => false, :stdinfile => tempfile.path}).once + provider.hold end end it "should use :install to update" do - @provider.expects(:install) - @provider.update + provider.expects(:install) + provider.update end describe "when determining latest available version" do it "should return the version found by dpkg-deb" do - @resource.expects(:[]).with(:source).returns "myfile" - @provider.expects(:dpkg_deb).with { |*command| command[-1] == "myfile" }.returns "asdf\t1.0" - @provider.latest.should == "1.0" + resource.expects(:[]).with(:source).returns "myfile" + provider.expects(:dpkg_deb).with { |*command| command[-1] == "myfile" }.returns "package\t1.0" + provider.latest.should == "1.0" end it "should warn if the package file contains a different package" do - @provider.expects(:dpkg_deb).returns("foo\tversion") - @provider.expects(:warning) - @provider.latest + provider.expects(:dpkg_deb).returns("foo\tversion") + provider.expects(:warning) + provider.latest end it "should cope with names containing ++" do - @resource = stub 'resource', :[] => "asdf++" - @provider = provider.new(@resource) - @provider.expects(:dpkg_deb).returns "asdf++\t1.0" - @provider.latest.should == "1.0" + resource = stub 'resource', :[] => "package++" + provider = provider_class.new(resource) + provider.expects(:dpkg_deb).returns "package++\t1.0" + provider.latest.should == "1.0" end end it "should use 'dpkg -r' to uninstall" do - @provider.expects(:dpkg).with("-r", "asdf") - @provider.uninstall + provider.expects(:dpkg).with("-r", resource_name) + provider.uninstall end it "should use 'dpkg --purge' to purge" do - @provider.expects(:dpkg).with("--purge", "asdf") - @provider.purge + provider.expects(:dpkg).with("--purge", resource_name) + provider.purge end end diff --git a/spec/unit/provider/package/openbsd_spec.rb b/spec/unit/provider/package/openbsd_spec.rb index 42d24e37f..5a7ef7b46 100755 --- a/spec/unit/provider/package/openbsd_spec.rb +++ b/spec/unit/provider/package/openbsd_spec.rb @@ -62,6 +62,16 @@ describe provider_class do provider_class.instances.map(&:name).sort.should == %w{bash bzip2 expat gettext libiconv lzo openvpn python vim wget}.sort end + + it "should return all flavors if set" do + fixture = File.read(my_fixture('pkginfo_flavors.list')) + provider_class.expects(:execpipe).with(%w{/bin/pkg_info -a}).yields(fixture) + instances = provider_class.instances.map {|p| {:name => p.get(:name), + :ensure => p.get(:ensure), :flavor => p.get(:flavor)}} + instances.size.should == 2 + instances[0].should == {:name => 'bash', :ensure => '3.1.17', :flavor => 'static'} + instances[1].should == {:name => 'vim', :ensure => '7.0.42', :flavor => 'no_x11'} + end end context "#install" do @@ -184,7 +194,29 @@ describe provider_class do provider.install end - %w{ installpath installpath= }.each do |line| + it "should append installpath" do + urls = ["ftp://your.ftp.mirror/pub/OpenBSD/5.2/packages/amd64/", + "http://another.ftp.mirror/pub/OpenBSD/5.2/packages/amd64/"] + lines = ["installpath = #{urls[0]}\n", + "installpath += #{urls[1]}\n"] + + expect_read_from_pkgconf(lines) + expect_pkgadd_with_env_and_name(urls.join(":")) do + provider.install + end + end + + it "should handle append on first installpath" do + url = "ftp://your.ftp.mirror/pub/OpenBSD/5.2/packages/amd64/" + lines = ["installpath += #{url}\n"] + + expect_read_from_pkgconf(lines) + expect_pkgadd_with_env_and_name(url) do + provider.install + end + end + + %w{ installpath installpath= installpath+=}.each do |line| it "should reject '#{line}'" do expect_read_from_pkgconf([line]) expect { @@ -226,4 +258,55 @@ describe provider_class do provider.query.should be_nil end end + + context "#install_options" do + it "should return nill by default" do + provider.install_options.should be_nil + end + + it "should return install_options when set" do + provider.resource[:install_options] = ['-n'] + provider.resource[:install_options].should == ['-n'] + end + + it "should return multiple install_options when set" do + provider.resource[:install_options] = ['-L', '/opt/puppet'] + provider.resource[:install_options].should == ['-L', '/opt/puppet'] + end + + it 'should return install_options when set as hash' do + provider.resource[:install_options] = { '-Darch' => 'vax' } + provider.install_options.should == ['-Darch=vax'] + end + end + + context "#uninstall_options" do + it "should return nill by default" do + provider.uninstall_options.should be_nil + end + + it "should return uninstall_options when set" do + provider.resource[:uninstall_options] = ['-n'] + provider.resource[:uninstall_options].should == ['-n'] + end + + it "should return multiple uninstall_options when set" do + provider.resource[:uninstall_options] = ['-q', '-c'] + provider.resource[:uninstall_options].should == ['-q', '-c'] + end + + it 'should return uninstall_options when set as hash' do + provider.resource[:uninstall_options] = { '-Dbaddepend' => '1' } + provider.uninstall_options.should == ['-Dbaddepend=1'] + end + end + + context "#uninstall" do + describe 'when uninstalling' do + it 'should use erase to purge' do + provider.expects(:pkgdelete).with('-c', '-q', 'bash') + provider.purge + end + end + end end diff --git a/spec/unit/provider/package/opkg_spec.rb b/spec/unit/provider/package/opkg_spec.rb index dfe6fed15..f472e22d2 100755 --- a/spec/unit/provider/package/opkg_spec.rb +++ b/spec/unit/provider/package/opkg_spec.rb @@ -82,7 +82,7 @@ describe Puppet::Type.type(:package).provider(:opkg) do context "with invalid URL for opkg" do before do # Emulate the `opkg` command returning a non-zero exit value - Puppet::Util::Execution.stubs(:execute).raises Puppet::ExecutionFailure + Puppet::Util::Execution.stubs(:execute).raises Puppet::ExecutionFailure, 'oops' end context "puppet://server/whatever" do @@ -91,7 +91,7 @@ describe Puppet::Type.type(:package).provider(:opkg) do end it "should fail" do - expect { provider.install }.to raise_error, Puppet::ExecutionFailure + expect { provider.install }.to raise_error Puppet::ExecutionFailure end end @@ -101,7 +101,7 @@ describe Puppet::Type.type(:package).provider(:opkg) do end it "should fail" do - expect { provider.install }.to raise_error, Puppet::ExecutionFailure + expect { provider.install }.to raise_error Puppet::ExecutionFailure end end end diff --git a/spec/unit/provider/package/pip_spec.rb b/spec/unit/provider/package/pip_spec.rb index 9fde967fd..81f614791 100755 --- a/spec/unit/provider/package/pip_spec.rb +++ b/spec/unit/provider/package/pip_spec.rb @@ -90,6 +90,22 @@ describe provider_class do @provider.query.should == nil end + it "should be case insensitive" do + @resource[:name] = "Real_Package" + + provider_class.expects(:instances).returns [provider_class.new({ + :ensure => "1.2.5", + :name => "real_package", + :provider => :pip, + })] + + @provider.query.should == { + :ensure => "1.2.5", + :name => "real_package", + :provider => :pip, + } + end + end describe "latest" do diff --git a/spec/unit/provider/package/pkgdmg_spec.rb b/spec/unit/provider/package/pkgdmg_spec.rb index efedcd542..8dfa235b6 100755..100644 --- a/spec/unit/provider/package/pkgdmg_spec.rb +++ b/spec/unit/provider/package/pkgdmg_spec.rb @@ -64,26 +64,81 @@ describe Puppet::Type.type(:package).provider(:pkgdmg) do resource[:source] = "http://fake.puppetlabs.com/foo.dmg" end - it "should call tmpdir and use the returned directory" do + it "should call tmpdir and then call curl with that directory" do Dir.expects(:mktmpdir).returns tmpdir Dir.stubs(:entries).returns ["foo.pkg"] described_class.expects(:curl).with do |*args| - args[0] == "-o" and args[1].include? tmpdir + args[0] == "-o" and args[1].include? tmpdir and args.include? "--fail" end described_class.stubs(:hdiutil).returns fake_hdiutil_plist described_class.expects(:installpkg) provider.install end + + it "should use an http proxy host and port if specified" do + Puppet::Util::HttpProxy.expects(:http_proxy_host).returns 'some_host' + Puppet::Util::HttpProxy.expects(:http_proxy_port).returns 'some_port' + Dir.expects(:mktmpdir).returns tmpdir + Dir.stubs(:entries).returns ["foo.pkg"] + described_class.expects(:curl).with do |*args| + args.should be_include 'some_host:some_port' + args.should be_include '--proxy' + end + described_class.stubs(:hdiutil).returns fake_hdiutil_plist + described_class.expects(:installpkg) + + provider.install + end + + it "should use an http proxy host only if specified" do + Puppet::Util::HttpProxy.expects(:http_proxy_host).returns 'some_host' + Puppet::Util::HttpProxy.expects(:http_proxy_port).returns nil + Dir.expects(:mktmpdir).returns tmpdir + Dir.stubs(:entries).returns ["foo.pkg"] + described_class.expects(:curl).with do |*args| + args.should be_include 'some_host' + args.should be_include '--proxy' + end + described_class.stubs(:hdiutil).returns fake_hdiutil_plist + described_class.expects(:installpkg) + + provider.install + end + end end describe "when installing flat pkg file" do - it "should call installpkg if a flat pkg file is found instead of a .dmg image" do - resource[:source] = "/tmp/test.pkg" - resource[:name] = "testpkg" - provider.class.expects(:installpkgdmg).with("/tmp/test.pkg", "testpkg").returns "" - provider.install + describe "with a local source" do + it "should call installpkg if a flat pkg file is found instead of a .dmg image" do + resource[:source] = "/tmp/test.pkg" + resource[:name] = "testpkg" + provider.class.expects(:installpkgdmg).with("/tmp/test.pkg", "testpkg").returns "" + provider.install + end + end + + describe "with a remote source" do + let(:remote_source) { 'http://fake.puppetlabs.com/test.pkg' } + let(:tmpdir) { '/path/to/tmpdir' } + let(:tmpfile) { File.join(tmpdir, 'testpkg.pkg') } + + before do + resource[:name] = 'testpkg' + resource[:source] = remote_source + + Dir.stubs(:mktmpdir).returns tmpdir + end + + it "should call installpkg if a flat pkg file is found instead of a .dmg image" do + described_class.expects(:curl).with do |*args| + args.should be_include tmpfile + args.should be_include remote_source + end + provider.class.expects(:installpkg).with(tmpfile, 'testpkg', remote_source) + provider.install + end end end end diff --git a/spec/unit/provider/package/rpm_spec.rb b/spec/unit/provider/package/rpm_spec.rb index 964175c42..8dd23a935 100755 --- a/spec/unit/provider/package/rpm_spec.rb +++ b/spec/unit/provider/package/rpm_spec.rb @@ -8,16 +8,18 @@ describe provider_class do let (:packages) do <<-RPM_OUTPUT - cracklib-dicts 0 2.8.9 3.3 x86_64 - basesystem 0 8.0 5.1.1.el5.centos noarch - chkconfig 0 1.3.30.2 2.el5 x86_64 - myresource 0 1.2.3.4 5.el4 noarch + cracklib-dicts 0 2.8.9 3.3 x86_64 :DESC: The standard CrackLib dictionaries + basesystem 0 8.0 5.1.1.el5.centos noarch :DESC: The skeleton package which defines a simple Red Hat Enterprise Linux system + chkconfig 0 1.3.30.2 2.el5 x86_64 :DESC: A system tool for maintaining the /etc/rc*.d hierarchy + myresource 0 1.2.3.4 5.el4 noarch :DESC: Now with summary + mysummaryless 0 1.2.3.4 5.el4 noarch :DESC: RPM_OUTPUT end + let(:resource_name) { 'myresource' } let(:resource) do Puppet::Type.type(:package).new( - :name => 'myresource', + :name => resource_name, :ensure => :installed ) end @@ -28,6 +30,10 @@ describe provider_class do provider end + let(:nevra_format) { %Q{'%{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH} :DESC: %{SUMMARY}\\n'} } + let(:execute_options) do + {:failonfail => true, :combine => true, :custom_environment => {}} + end let(:rpm_version) { "RPM version 5.0.0\n" } before(:each) do @@ -35,13 +41,13 @@ describe provider_class do subject.stubs(:which).with("rpm").returns("/bin/rpm") subject.instance_variable_set("@current_version", nil) Puppet::Type::Package::ProviderRpm.expects(:execute).with(["/bin/rpm", "--version"]).returns(rpm_version).at_most_once - Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", "--version"], {:failonfail => true, :combine => true, :custom_environment => {}}).returns(rpm_version).at_most_once + Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", "--version"], execute_options).returns(rpm_version).at_most_once end describe "self.instances" do describe "with a modern version of RPM" do it "should include all the modern flags" do - Puppet::Util::Execution.expects(:execpipe).with("/bin/rpm -qa --nosignature --nodigest --qf '%{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}\n'").yields(packages) + Puppet::Util::Execution.expects(:execpipe).with("/bin/rpm -qa --nosignature --nodigest --qf #{nevra_format}").yields(packages) installed_packages = subject.instances end @@ -50,7 +56,7 @@ describe provider_class do describe "with a version of RPM < 4.1" do let(:rpm_version) { "RPM version 4.0.2\n" } it "should exclude the --nosignature flag" do - Puppet::Util::Execution.expects(:execpipe).with("/bin/rpm -qa --nodigest --qf '%{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}\n'").yields(packages) + Puppet::Util::Execution.expects(:execpipe).with("/bin/rpm -qa --nodigest --qf #{nevra_format}").yields(packages) installed_packages = subject.instances end @@ -59,14 +65,14 @@ describe provider_class do describe "with a version of RPM < 4.0.2" do let(:rpm_version) { "RPM version 3.0.5\n" } it "should exclude the --nodigest flag" do - Puppet::Util::Execution.expects(:execpipe).with("/bin/rpm -qa --qf '%{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}\n'").yields(packages) + Puppet::Util::Execution.expects(:execpipe).with("/bin/rpm -qa --qf #{nevra_format}").yields(packages) installed_packages = subject.instances end end it "returns an array of packages" do - Puppet::Util::Execution.expects(:execpipe).with("/bin/rpm -qa --nosignature --nodigest --qf '%{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}\n'").yields(packages) + Puppet::Util::Execution.expects(:execpipe).with("/bin/rpm -qa --nosignature --nodigest --qf #{nevra_format}").yields(packages) installed_packages = subject.instances @@ -78,7 +84,8 @@ describe provider_class do :version => "2.8.9", :release => "3.3", :arch => "x86_64", - :ensure => "2.8.9-3.3" + :ensure => "2.8.9-3.3", + :description => "The standard CrackLib dictionaries", } installed_packages[1].properties.should == { @@ -88,7 +95,8 @@ describe provider_class do :version => "8.0", :release => "5.1.1.el5.centos", :arch => "noarch", - :ensure => "8.0-5.1.1.el5.centos" + :ensure => "8.0-5.1.1.el5.centos", + :description => "The skeleton package which defines a simple Red Hat Enterprise Linux system", } installed_packages[2].properties.should == { @@ -98,7 +106,19 @@ describe provider_class do :version => "1.3.30.2", :release => "2.el5", :arch => "x86_64", - :ensure => "1.3.30.2-2.el5" + :ensure => "1.3.30.2-2.el5", + :description => "A system tool for maintaining the /etc/rc*.d hierarchy", + } + installed_packages.last.properties.should == + { + :provider => :rpm, + :name => "mysummaryless", + :epoch => "0", + :version => "1.2.3.4", + :release => "5.el4", + :arch => "noarch", + :ensure => "1.2.3.4-5.el4", + :description => "", } end end @@ -114,7 +134,7 @@ describe provider_class do describe "when not already installed" do it "should only include the '-i' flag" do - Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", "-i", '/path/to/package'], {:failonfail => true, :combine => true, :custom_environment => {}}) + Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", "-i", '/path/to/package'], execute_options) provider.install end end @@ -127,12 +147,20 @@ describe provider_class do end it "should include the '-U --oldpackage' flags" do - Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", ["-U", "--oldpackage"], '/path/to/package'], {:failonfail => true, :combine => true, :custom_environment => {}}) + Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", ["-U", "--oldpackage"], '/path/to/package'], execute_options) provider.install end end end + describe "#latest" do + it "should retrieve version string after querying rpm for version from source file" do + resource.expects(:[]).with(:source).returns('source-string') + Puppet::Util::Execution.expects(:execfail).with(["/bin/rpm", "-q", "--qf", nevra_format, "-p", "source-string"], Puppet::Error).returns("myresource 0 1.2.3.4 5.el4 noarch :DESC:\n") + provider.latest.should == "1.2.3.4-5.el4" + end + end + describe "#uninstall" do let(:resource) do Puppet::Type.type(:package).new( @@ -142,33 +170,96 @@ describe provider_class do end describe "on a modern RPM" do - before(:each) do - Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", "-q", "myresource", '--nosignature', '--nodigest', "--qf", "%{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}\n"], {:failonfail => true, :combine => true, :custom_environment => {}}).returns("myresource 0 1.2.3.4 5.el4 noarch\n") + before(:each) do + Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", "-q", "myresource", '--nosignature', '--nodigest', "--qf", nevra_format], execute_options).returns("myresource 0 1.2.3.4 5.el4 noarch :DESC:\n") end let(:rpm_version) { "RPM version 4.10.0\n" } it "should include the architecture in the package name" do - Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", "-e", 'myresource-1.2.3.4-5.el4.noarch'], {:failonfail => true, :combine => true, :custom_environment => {}}).returns('').at_most_once + Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", "-e", 'myresource-1.2.3.4-5.el4.noarch'], execute_options).returns('').at_most_once provider.uninstall end end describe "on an ancient RPM" do - before(:each) do - Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", "-q", "myresource", '', '', "--qf", "%{NAME} %|EPOCH?{%{EPOCH}}:{0}| %{VERSION} %{RELEASE} %{ARCH}\n"], {:failonfail => true, :combine => true, :custom_environment => {}}).returns("myresource 0 1.2.3.4 5.el4 noarch\n") + before(:each) do + Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", "-q", "myresource", '', '', '--qf', nevra_format], execute_options).returns("myresource 0 1.2.3.4 5.el4 noarch :DESC:\n") end let(:rpm_version) { "RPM version 3.0.6\n" } it "should exclude the architecture from the package name" do - Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", "-e", 'myresource-1.2.3.4-5.el4'], {:failonfail => true, :combine => true, :custom_environment => {}}).returns('').at_most_once + Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", "-e", 'myresource-1.2.3.4-5.el4'], execute_options).returns('').at_most_once provider.uninstall end end end + describe "parsing" do + def parser_test(rpm_output_string, gold_hash, number_of_warnings = 0) + Puppet.expects(:warning).times(number_of_warnings) + Puppet::Util::Execution.expects(:execute).with(["/bin/rpm", "-q", resource_name, "--nosignature", "--nodigest", "--qf", nevra_format], execute_options).returns(rpm_output_string) + provider.query.should == gold_hash + end + + let(:resource_name) { 'name' } + let('delimiter') { ':DESC:' } + let(:package_hash) do + { + :name => 'name', + :epoch => 'epoch', + :version => 'version', + :release => 'release', + :arch => 'arch', + :description => 'a description', + :provider => :rpm, + :ensure => 'version-release', + } + end + let(:line) { 'name epoch version release arch :DESC: a description' } + + ['name', 'epoch', 'version', 'release', 'arch'].each do |field| + + it "should still parse if #{field} is replaced by delimiter" do + parser_test( + line.gsub(field, delimiter), + package_hash.merge( + field.to_sym => delimiter, + :ensure => 'version-release'.gsub(field, delimiter) + ) + ) + end + + end + + it "should still parse if missing description" do + parser_test( + line.gsub(/#{delimiter} .+$/, delimiter), + package_hash.merge(:description => '') + ) + end + + it "should still parse if missing delimeter and description entirely" do + parser_test( + line.gsub(/ #{delimiter} .+$/, ''), + package_hash.merge(:description => nil) + ) + end + + it "should still parse if description contains a new line" do + parser_test( + line.gsub(/#{delimiter} .+$/, "#{delimiter} whoops\nnewline"), + package_hash.merge(:description => 'whoops') + ) + end + + it "should warn but not fail if line is unparseable" do + parser_test('bad data', {}, 1) + end + end + describe ".nodigest" do { '4.0' => nil, '4.0.1' => nil, diff --git a/spec/unit/provider/package/urpmi.rb b/spec/unit/provider/package/urpmi.rb new file mode 100644 index 000000000..0b44d44e6 --- /dev/null +++ b/spec/unit/provider/package/urpmi.rb @@ -0,0 +1,80 @@ +require 'spec_helper' + +describe Puppet::Type.type(:package).provider(:urpmi) do + + before do + Puppet::Util::Execution.expects(:execute).never + %w[rpm urpmi urpme urpmq].each do |executable| + Puppet::Util.stubs(:which).with(executable).returns(executable) + end + Puppet::Util::Execution.stubs(:execute).with(['rpm', '--version'], anything).returns 'RPM version 4.9.1.3' + end + + let(:resource) do + Puppet::Type.type(:package).new(:name => 'foopkg') + end + + before do + subject.resource = resource + Puppet::Type.type(:package).stubs(:defaultprovider).returns described_class + end + + describe '#install' do + before do + subject.stubs(:rpm).with('-q', 'foopkg', any_parameters).returns "foopkg 0 1.2.3.4 5 noarch :DESC:\n" + end + + describe 'without a version' do + it 'installs the unversioned package' do + resource[:ensure] = :present + Puppet::Util::Execution.expects(:execute).with(['urpmi', '--auto', 'foopkg'], anything) + subject.install + end + end + + describe 'with a version' do + it 'installs the versioned package' do + resource[:ensure] = '4.5.6' + Puppet::Util::Execution.expects(:execute).with(['urpmi', '--auto', 'foopkg-4.5.6'], anything) + subject.install + end + end + + describe "and the package install fails" do + it "raises an error" do + Puppet::Util::Execution.stubs(:execute).with(['urpmi', '--auto', 'foopkg'], anything) + subject.stubs(:query) + expect { subject.install }.to raise_error Puppet::Error, /Package \S+ was not present after trying to install it/ + end + end + end + + describe '#latest' do + let(:urpmq_output) { 'foopkg : Lorem ipsum dolor sit amet, consectetur adipisicing elit ( 7.8.9-1.mga2 )' } + + it "uses urpmq to determine the latest package" do + Puppet::Util::Execution.expects(:execute).with(['urpmq', '-S', 'foopkg'], anything).returns urpmq_output + subject.latest.should == '7.8.9-1.mga2' + end + + it "falls back to the current version" do + resource[:ensure] = '5.4.3' + Puppet::Util::Execution.expects(:execute).with(['urpmq', '-S', 'foopkg'], anything).returns '' + subject.latest.should == '5.4.3' + end + end + + describe '#update' do + it 'delegates to #install' do + subject.expects(:install) + subject.update + end + end + + describe '#purge' do + it 'uses urpme to purge packages' do + Puppet::Util::Execution.expects(:execute).with(['urpme', '--auto', 'foopkg'], anything) + subject.purge + end + end +end diff --git a/spec/unit/provider/package/windows/exe_package_spec.rb b/spec/unit/provider/package/windows/exe_package_spec.rb index 29542977d..89e101ccd 100644 --- a/spec/unit/provider/package/windows/exe_package_spec.rb +++ b/spec/unit/provider/package/windows/exe_package_spec.rb @@ -77,7 +77,7 @@ describe Puppet::Provider::Package::Windows::ExePackage do it 'should install using the source' do cmd = subject.install_command({:source => source}) - cmd.should == ['cmd.exe', '/c', 'start', '/w', source] + cmd.should == ['cmd.exe', '/c', 'start', '"puppet-install"', '/w', source] end end diff --git a/spec/unit/provider/package/yum_spec.rb b/spec/unit/provider/package/yum_spec.rb index b541c0e9e..7a019bea7 100755 --- a/spec/unit/provider/package/yum_spec.rb +++ b/spec/unit/provider/package/yum_spec.rb @@ -108,4 +108,89 @@ describe provider do end end end + + describe 'prefetching' do + let(:nevra_format) { Puppet::Type::Package::ProviderRpm::NEVRA_FORMAT } + + let(:packages) do + <<-RPM_OUTPUT + cracklib-dicts 0 2.8.9 3.3 x86_64 :DESC: The standard CrackLib dictionaries + basesystem 0 8.0 5.1.1.el5.centos noarch :DESC: The skeleton package which defines a simple Red Hat Enterprise Linux system + chkconfig 0 1.3.30.2 2.el5 x86_64 :DESC: A system tool for maintaining the /etc/rc*.d hierarchy + myresource 0 1.2.3.4 5.el4 noarch :DESC: Now with summary + mysummaryless 0 1.2.3.4 5.el4 noarch :DESC: + RPM_OUTPUT + end + + let(:yumhelper_output) do + <<-YUMHELPER_OUTPUT + * base: centos.tcpdiag.net + * extras: centos.mirrors.hoobly.com + * updates: mirrors.arsc.edu +_pkg nss-tools 0 3.14.3 4.el6_4 x86_64 +_pkg pixman 0 0.26.2 5.el6_4 x86_64 +_pkg myresource 0 1.2.3.4 5.el4 noarch +_pkg mysummaryless 0 1.2.3.4 5.el4 noarch + YUMHELPER_OUTPUT + end + + let(:execute_options) do + {:failonfail => true, :combine => true, :custom_environment => {}} + end + + let(:rpm_version) { "RPM version 4.8.0\n" } + + let(:package_type) { Puppet::Type.type(:package) } + let(:yum_provider) { provider } + + def pretend_we_are_root_for_yum_provider + Process.stubs(:euid).returns(0) + end + + def expect_yum_provider_to_provide_rpm + Puppet::Type::Package::ProviderYum.stubs(:rpm).with('--version').returns(rpm_version) + Puppet::Type::Package::ProviderYum.expects(:command).with(:rpm).returns("/bin/rpm") + end + + def expect_execpipe_to_provide_package_info_for_an_rpm_query + Puppet::Util::Execution.expects(:execpipe).with("/bin/rpm -qa --nosignature --nodigest --qf #{nevra_format}").yields(packages) + end + + def expect_python_yumhelper_call_to_return_latest_info + Puppet::Type::Package::ProviderYum.expects(:python).with(regexp_matches(/yumhelper.py$/)).returns(yumhelper_output) + end + + def a_package_type_instance_with_yum_provider_and_ensure_latest(name) + type_instance = package_type.new(:name => name) + type_instance.provider = yum_provider.new + type_instance[:ensure] = :latest + return type_instance + end + + before do + pretend_we_are_root_for_yum_provider + expect_yum_provider_to_provide_rpm + expect_execpipe_to_provide_package_info_for_an_rpm_query + expect_python_yumhelper_call_to_return_latest_info + end + + it "injects latest provider info into passed resources when prefetching" do + myresource = a_package_type_instance_with_yum_provider_and_ensure_latest('myresource') + mysummaryless = a_package_type_instance_with_yum_provider_and_ensure_latest('mysummaryless') + + yum_provider.prefetch({ "myresource" => myresource, "mysummaryless" => mysummaryless }) + + expect(@logs.map(&:message).grep(/^Failed to match rpm line/)).to be_empty + expect(myresource.provider.latest_info).to eq({ + :name=>"myresource", + :epoch=>"0", + :version=>"1.2.3.4", + :release=>"5.el4", + :arch=>"noarch", + :description=>nil, + :provider=>:yum, + :ensure=>"1.2.3.4-5.el4" + }) + end + end end diff --git a/spec/unit/provider/package/zypper_spec.rb b/spec/unit/provider/package/zypper_spec.rb index a9d4aa86a..eb4690793 100755 --- a/spec/unit/provider/package/zypper_spec.rb +++ b/spec/unit/provider/package/zypper_spec.rb @@ -39,13 +39,18 @@ describe provider_class do @provider.should respond_to(:latest) end + it "should have a install_options method" do + @provider = provider_class.new + @provider.should respond_to(:install_options) + end + describe "when installing with zypper version >= 1.0" do it "should use a command-line with versioned package'" do @resource.stubs(:should).with(:ensure).returns "1.2.3-4.5.6" @provider.stubs(:zypper_version).returns "1.2.8" @provider.expects(:zypper).with('--quiet', :install, - '--auto-agree-with-licenses', '--no-confirm', 'mypackage-1.2.3-4.5.6') + '--auto-agree-with-licenses', '--no-confirm', nil, 'mypackage-1.2.3-4.5.6') @provider.expects(:query).returns "mypackage 0 1.2.3 4.5.6 x86_64" @provider.install end @@ -54,7 +59,7 @@ describe provider_class do @resource.stubs(:should).with(:ensure).returns :latest @provider.stubs(:zypper_version).returns "1.2.8" @provider.expects(:zypper).with('--quiet', :install, - '--auto-agree-with-licenses', '--no-confirm', 'mypackage') + '--auto-agree-with-licenses', '--no-confirm', nil, 'mypackage') @provider.expects(:query).returns "mypackage 0 1.2.3 4.5.6 x86_64" @provider.install end @@ -66,7 +71,7 @@ describe provider_class do @provider.stubs(:zypper_version).returns "0.6.104" @provider.expects(:zypper).with('--terse', :install, - '--auto-agree-with-licenses', '--no-confirm', 'mypackage-1.2.3-4.5.6') + '--auto-agree-with-licenses', '--no-confirm', nil, 'mypackage-1.2.3-4.5.6') @provider.expects(:query).returns "mypackage 0 1.2.3 4.5.6 x86_64" @provider.install end @@ -75,7 +80,7 @@ describe provider_class do @resource.stubs(:should).with(:ensure).returns :latest @provider.stubs(:zypper_version).returns "0.6.104" @provider.expects(:zypper).with('--terse', :install, - '--auto-agree-with-licenses', '--no-confirm', 'mypackage') + '--auto-agree-with-licenses', '--no-confirm', nil, 'mypackage') @provider.expects(:query).returns "mypackage 0 1.2.3 4.5.6 x86_64" @provider.install end @@ -87,7 +92,7 @@ describe provider_class do @provider.stubs(:zypper_version).returns "0.6.13" @provider.expects(:zypper).with('--terse', :install, - '--no-confirm', 'mypackage-1.2.3-4.5.6') + '--no-confirm', nil, 'mypackage-1.2.3-4.5.6') @provider.expects(:query).returns "mypackage 0 1.2.3 4.5.6 x86_64" @provider.install end @@ -96,7 +101,7 @@ describe provider_class do @resource.stubs(:should).with(:ensure).returns :latest @provider.stubs(:zypper_version).returns "0.6.13" @provider.expects(:zypper).with('--terse', :install, - '--no-confirm', 'mypackage') + '--no-confirm', nil, 'mypackage') @provider.expects(:query).returns "mypackage 0 1.2.3 4.5.6 x86_64" @provider.install end @@ -119,4 +124,18 @@ describe provider_class do end end + describe "when installing with zypper install options" do + it "should install the package without checking keys" do + @resource.stubs(:[]).with(:name).returns "php5" + @resource.stubs(:should).with(:install_options).returns ['--no-gpg-check', {'-p' => '/vagrant/files/localrepo/'}] + @resource.stubs(:should).with(:ensure).returns "5.4.10-4.5.6" + @provider.stubs(:zypper_version).returns "1.2.8" + + @provider.expects(:install_options).returns "--no-gpg-check -p \"/vagrant/files/localrepo/\"" + @provider.expects(:zypper).with('--quiet', :install, + '--auto-agree-with-licenses', '--no-confirm', '--no-gpg-check -p "/vagrant/files/localrepo/"', 'php5-5.4.10-4.5.6') + @provider.expects(:query).returns "php5 0 5.4.10 4.5.6 x86_64" + @provider.install + end + end end diff --git a/spec/unit/provider/parsedfile_spec.rb b/spec/unit/provider/parsedfile_spec.rb index 826239c70..1ac0c198b 100755 --- a/spec/unit/provider/parsedfile_spec.rb +++ b/spec/unit/provider/parsedfile_spec.rb @@ -1,11 +1,11 @@ #! /usr/bin/env ruby require 'spec_helper' +require 'puppet_spec/files' require 'puppet/provider/parsedfile' # Most of the tests for this are still in test/ral/provider/parsedfile.rb. describe Puppet::Provider::ParsedFile do - # The ParsedFile provider class is meant to be used as an abstract base class # but also stores a lot of state within the singleton class. To avoid # sharing data between classes we construct an anonymous class that inherits @@ -136,9 +136,10 @@ describe Puppet::Provider::ParsedFile do end describe "A very basic provider based on ParsedFile" do + include PuppetSpec::Files let(:input_text) { File.read(my_fixture('simple.txt')) } - let(:target) { File.expand_path("/tmp/test") } + let(:target) { tmpfile('parsedfile_spec') } subject do example_provider_class = Class.new(Puppet::Provider::ParsedFile) diff --git a/spec/unit/provider/service/init_spec.rb b/spec/unit/provider/service/init_spec.rb index 6da19ef40..e88672b40 100755 --- a/spec/unit/provider/service/init_spec.rb +++ b/spec/unit/provider/service/init_spec.rb @@ -103,28 +103,28 @@ describe Puppet::Type.type(:service).provider(:init) do end it "should be able to find the init script in the service path" do - File.expects(:stat).with("#{paths[0]}/myservice").returns true - File.expects(:stat).with("#{paths[1]}/myservice").never # first one wins + File.expects(:exist?).with("#{paths[0]}/myservice").returns true + File.expects(:exist?).with("#{paths[1]}/myservice").never # first one wins provider.initscript.should == "/service/path/myservice" end it "should be able to find the init script in an alternate service path" do - File.expects(:stat).with("#{paths[0]}/myservice").raises Errno::ENOENT, "No such file or directory - #{paths[0]}/myservice" - File.expects(:stat).with("#{paths[1]}/myservice").returns true + File.expects(:exist?).with("#{paths[0]}/myservice").returns false + File.expects(:exist?).with("#{paths[1]}/myservice").returns true provider.initscript.should == "/alt/service/path/myservice" end it "should be able to find the init script if it ends with .sh" do - File.expects(:stat).with("#{paths[0]}/myservice").raises Errno::ENOENT, "No such file or directory - #{paths[0]}/myservice" - File.expects(:stat).with("#{paths[1]}/myservice").raises Errno::ENOENT, "No such file or directory - #{paths[1]}/myservice" - File.expects(:stat).with("#{paths[0]}/myservice.sh").returns true + File.expects(:exist?).with("#{paths[0]}/myservice").returns false + File.expects(:exist?).with("#{paths[1]}/myservice").returns false + File.expects(:exist?).with("#{paths[0]}/myservice.sh").returns true provider.initscript.should == "/service/path/myservice.sh" end it "should fail if the service isn't there" do paths.each do |path| - File.expects(:stat).with("#{path}/myservice").raises Errno::ENOENT, "No such file or directory - #{path}/myservice" - File.expects(:stat).with("#{path}/myservice.sh").raises Errno::ENOENT, "No such file or directory - #{path}/myservice.sh" + File.expects(:exist?).with("#{path}/myservice").returns false + File.expects(:exist?).with("#{path}/myservice.sh").returns false end expect { provider.initscript }.to raise_error(Puppet::Error, "Could not find init script for 'myservice'") end @@ -134,7 +134,7 @@ describe Puppet::Type.type(:service).provider(:init) do before :each do File.stubs(:directory?).with("/service/path").returns true File.stubs(:directory?).with("/alt/service/path").returns true - File.stubs(:stat).with("/service/path/myservice").returns true + File.stubs(:exist?).with("/service/path/myservice").returns true end [:start, :stop, :status, :restart].each do |method| diff --git a/spec/unit/provider/service/openrc_spec.rb b/spec/unit/provider/service/openrc_spec.rb index aa8162226..1949d3328 100755 --- a/spec/unit/provider/service/openrc_spec.rb +++ b/spec/unit/provider/service/openrc_spec.rb @@ -7,7 +7,10 @@ describe Puppet::Type.type(:service).provider(:openrc) do before :each do Puppet::Type.type(:service).stubs(:defaultprovider).returns described_class ['/sbin/rc-service', '/bin/rc-status', '/sbin/rc-update'].each do |command| + # Puppet::Util is both mixed in to providers and is also invoked directly + # by Puppet::Provider::CommandDefiner, so we have to stub both out. described_class.stubs(:which).with(command).returns(command) + Puppet::Util.stubs(:which).with(command).returns(command) end end @@ -58,6 +61,19 @@ describe Puppet::Type.type(:service).provider(:openrc) do end end + describe 'when invoking `rc-status`' do + subject { described_class.new(Puppet::Type.type(:service).new(:name => 'urandom')) } + it "clears the RC_SVCNAME environment variable" do + Puppet::Util.withenv(:RC_SVCNAME => 'puppet') do + Puppet::Util::Execution.expects(:execute).with( + includes('/bin/rc-status'), + has_entry(:custom_environment, {:RC_SVCNAME => nil}) + ).returns '' + subject.enabled? + end + end + end + describe "#enabled?" do before :each do diff --git a/spec/unit/provider/service/openwrt_spec.rb b/spec/unit/provider/service/openwrt_spec.rb index 4cfb06f40..bc8a6323d 100755 --- a/spec/unit/provider/service/openwrt_spec.rb +++ b/spec/unit/provider/service/openwrt_spec.rb @@ -33,7 +33,7 @@ describe Puppet::Type.type(:service).provider(:openwrt), :as_platform => :posix # All OpenWrt tests operate on the init script directly. It must exist. File.stubs(:directory?).with('/etc/init.d').returns true - File.stubs(:stat).with('/etc/init.d/myservice') + File.stubs(:exist?).with('/etc/init.d/myservice').returns true FileTest.stubs(:file?).with('/etc/init.d/myservice').returns true FileTest.stubs(:executable?).with('/etc/init.d/myservice').returns true end diff --git a/spec/unit/provider/service/redhat_spec.rb b/spec/unit/provider/service/redhat_spec.rb index 673c80b7f..0df256c6f 100755 --- a/spec/unit/provider/service/redhat_spec.rb +++ b/spec/unit/provider/service/redhat_spec.rb @@ -69,6 +69,13 @@ describe provider_class, :as_platform => :posix do @provider.should respond_to(:enabled?) end + it "should use --check on SuSE" do + Facter.expects(:value).with(:osfamily).returns 'Suse' + provider_class.expects(:chkconfig).with(@resource[:name], '--check') + + @provider.enabled? + end + it "should have an enable method" do @provider.should respond_to(:enable) end diff --git a/spec/unit/provider/ssh_authorized_key/parsed_spec.rb b/spec/unit/provider/ssh_authorized_key/parsed_spec.rb index b40658933..5cfa8994b 100755 --- a/spec/unit/provider/ssh_authorized_key/parsed_spec.rb +++ b/spec/unit/provider/ssh_authorized_key/parsed_spec.rb @@ -51,7 +51,7 @@ describe provider_class, :unless => Puppet.features.microsoft_windows? do genkey(key).should == "ssh-dss AAAAfsfddsjldjgksdflgkjsfdlgkj Just_Testing\n" end - it "should be able to generate a authorized_keys file with options" do + it "should be able to generate an authorized_keys file with options" do key = mkkey(:name => "root@localhost", :key => "AAAAfsfddsjldjgksdflgkjsfdlgkj", @@ -246,7 +246,7 @@ describe provider_class, :unless => Puppet.features.microsoft_windows? do end end - describe "and a invalid user has been specified with no target" do + describe "and an invalid user has been specified with no target" do it "should catch an exception and raise a Puppet error" do @resource[:user] = "thisusershouldnotexist" diff --git a/spec/unit/provider/user/aix_spec.rb b/spec/unit/provider/user/aix_spec.rb index ecc4b3b40..b9edaff49 100644 --- a/spec/unit/provider/user/aix_spec.rb +++ b/spec/unit/provider/user/aix_spec.rb @@ -39,4 +39,124 @@ guest id=100 pgrp=usr groups=usr home=/home/guest provider_class.list_all.should == ['root', 'guest'] end + describe "#managed_attribute_keys" do + let(:existing_attributes) do + { :account_locked => 'false', + :admin => 'false', + :login => 'true', + 'su' => 'true' + } + end + + before(:each) do + original_parameters = { :attributes => attribute_array } + @resource.stubs(:original_parameters).returns(original_parameters) + end + + describe "invoked via manifest" do + let(:attribute_array) { ["rlogin=false", "login =true"] } + + it "should return only the keys of the attribute key=value pair from manifest" do + keys = @provider.managed_attribute_keys(existing_attributes) + keys.should be_include(:rlogin) + keys.should be_include(:login) + keys.should_not be_include(:su) + end + + it "should strip spaces from symbols" do + keys = @provider.managed_attribute_keys(existing_attributes) + keys.should be_include(:login) + keys.should_not be_include(:"login ") + end + + it "should have the same count as that from the manifest" do + keys = @provider.managed_attribute_keys(existing_attributes) + keys.size.should == attribute_array.size + end + + it "should convert the keys to symbols" do + keys = @provider.managed_attribute_keys(existing_attributes) + all_symbols = keys.all? {|k| k.is_a? Symbol} + all_symbols.should be_true + end + end + + describe "invoked via RAL" do + let(:attribute_array) { nil } + + it "should return the keys in supplied hash" do + keys = @provider.managed_attribute_keys(existing_attributes) + keys.should_not be_include(:rlogin) + keys.should be_include(:login) + keys.should be_include(:su) + end + + it "should convert the keys to symbols" do + keys = @provider.managed_attribute_keys(existing_attributes) + all_symbols = keys.all? {|k| k.is_a? Symbol} + all_symbols.should be_true + end + end + end + + describe "#should_include?" do + it "should exclude keys translated into something else" do + managed_keys = [:rlogin] + @provider.class.attribute_mapping_from.stubs(:include?).with(:rlogin).returns(true) + @provider.class.stubs(:attribute_ignore).returns([]) + @provider.should_include?(:rlogin, managed_keys).should be_false + end + + it "should exclude keys explicitly ignored" do + managed_keys = [:rlogin] + @provider.class.attribute_mapping_from.stubs(:include?).with(:rlogin).returns(false) + @provider.class.stubs(:attribute_ignore).returns([:rlogin]) + @provider.should_include?(:rlogin, managed_keys).should be_false + end + + it "should exclude keys not specified in manifest" do + managed_keys = [:su] + @provider.class.attribute_mapping_from.stubs(:include?).with(:rlogin).returns(false) + @provider.class.stubs(:attribute_ignore).returns([]) + @provider.should_include?(:rlogin, managed_keys).should be_false + end + + it "should include keys specified in manifest if not translated or ignored" do + managed_keys = [:rlogin] + @provider.class.attribute_mapping_from.stubs(:include?).with(:rlogin).returns(false) + @provider.class.stubs(:attribute_ignore).returns([]) + @provider.should_include?(:rlogin, managed_keys).should be_true + end + end + describe "when handling passwords" do + let(:passwd_without_spaces) do + # from http://pic.dhe.ibm.com/infocenter/aix/v7r1/index.jsp?topic=%2Fcom.ibm.aix.files%2Fdoc%2Faixfiles%2Fpasswd_security.htm + <<-OUTPUT +smith: + password = MGURSj.F056Dj + lastupdate = 623078865 + flags = ADMIN,NOCHECK + OUTPUT + end + + let(:passwd_with_spaces) do + # add trailing space to the password + passwd_without_spaces.gsub(/password = (.*)/, 'password = \1 ') + end + + + it "should be able to read the hashed password" do + @provider.stubs(:open_security_passwd).returns(StringIO.new(passwd_without_spaces)) + @resource.stubs(:[]).returns('smith') + + @provider.password.should == 'MGURSj.F056Dj' + end + + it "should be able to read the hashed password, even with trailing spaces" do + @provider.stubs(:open_security_passwd).returns(StringIO.new(passwd_with_spaces)) + @resource.stubs(:[]).returns('smith') + + @provider.password.should == 'MGURSj.F056Dj' + end + end end diff --git a/spec/unit/provider/user/directoryservice_spec.rb b/spec/unit/provider/user/directoryservice_spec.rb index 6534147b7..e464e2c6f 100755 --- a/spec/unit/provider/user/directoryservice_spec.rb +++ b/spec/unit/provider/user/directoryservice_spec.rb @@ -228,7 +228,7 @@ describe Puppet::Type.type(:user).provider(:directoryservice) do { 'UniqueID' => '1000', 'RealName' => resource[:name], - 'PrimaryGroupID' => '20', + 'PrimaryGroupID' => 20, 'UserShell' => '/bin/bash', 'NFSHomeDirectory' => "/Users/#{resource[:name]}" } @@ -266,6 +266,13 @@ describe Puppet::Type.type(:user).provider(:directoryservice) do provider.expects(:merge_attribute_with_dscl).with('Groups', 'somegroup', 'GroupMembers', 'GUID') provider.create end + + it 'should convert group names into integers' do + resource[:gid] = 'somegroup' + Puppet::Util.expects(:gid).with('somegroup').returns(21) + provider.expects(:merge_attribute_with_dscl).with('Users', username, 'PrimaryGroupID', 21) + provider.create + end end describe 'self#instances' do @@ -599,7 +606,7 @@ describe Puppet::Type.type(:user).provider(:directoryservice) do }] end - it 'should return a array of hashes containing group data' do + it 'should return an array of hashes containing group data' do provider.class.expects(:dscl).with('-plist', '.', 'readall', '/Groups').returns(groups_xml) provider.class.get_list_of_groups.should == groups_hash end @@ -696,7 +703,7 @@ describe Puppet::Type.type(:user).provider(:directoryservice) do let(:password_hash_file) { '/var/db/shadow/hash/user_guid' } let(:stub_password_file) { stub('connection') } - it 'should return a a sha1 hash read from disk' do + it 'should return a sha1 hash read from disk' do File.expects(:exists?).with(password_hash_file).returns(true) File.expects(:file?).with(password_hash_file).returns(true) File.expects(:readable?).with(password_hash_file).returns(true) @@ -773,7 +780,7 @@ describe Puppet::Type.type(:user).provider(:directoryservice) do } end - it 'should call set_salted_sha512 on 10.7 when given a a salted-SHA512 password hash' do + it 'should call set_salted_sha512 on 10.7 when given a salted-SHA512 password hash' do provider.expects(:get_users_plist).returns(sample_users_plist) provider.expects(:get_shadow_hash_data).with(sample_users_plist).returns(sha512_shadowhashdata) provider.class.expects(:get_os_version).returns('10.7') diff --git a/spec/unit/provider/user/user_role_add_spec.rb b/spec/unit/provider/user/user_role_add_spec.rb index 340a5b537..8dd48e767 100755 --- a/spec/unit/provider/user/user_role_add_spec.rb +++ b/spec/unit/provider/user/user_role_add_spec.rb @@ -267,6 +267,7 @@ FIXTURE end it "should only update the target user" do + Date.expects(:today).returns Date.new(2011,12,07) write_fixture <<FIXTURE before:seriously:15315:0:99999:7::: fakeval:seriously:15315:0:99999:7::: @@ -294,6 +295,23 @@ FIXTURE provider.password = "totally" File.read(path).should == fixture end + + it "should update the lastchg field" do + Date.expects(:today).returns Date.new(2013,5,12) # 15837 days after 1970-01-01 + write_fixture <<FIXTURE +before:seriously:15315:0:99999:7::: +fakeval:seriously:15629:0:99999:7::: +fakevalish:seriously:15315:0:99999:7::: +after:seriously:15315:0:99999:7::: +FIXTURE + provider.password = "totally" + File.read(path).should == <<EOT +before:seriously:15315:0:99999:7::: +fakeval:totally:15837:0:99999:7::: +fakevalish:seriously:15315:0:99999:7::: +after:seriously:15315:0:99999:7::: +EOT + end end describe "#shadow_entry" do diff --git a/spec/unit/provider_spec.rb b/spec/unit/provider_spec.rb index 353dc5e7f..208094b7e 100755 --- a/spec/unit/provider_spec.rb +++ b/spec/unit/provider_spec.rb @@ -448,28 +448,17 @@ describe Puppet::Provider do fields = %w{prop1 prop2 param1 param2} - # This is needed for Ruby 1.8.5, which throws an exception that the - # default rescue doesn't catch if the method isn't present. Also, it has - # no convenient predicate for them, which equally hurts. - def has_method?(object, name) - begin - return true if object.instance_method(name) - rescue Exception - return false - end - end - fields.each do |name| it "should add getter methods for #{name}" do expect { subject.mk_resource_methods }. - to change { has_method?(subject, name) }. + to change { subject.method_defined?(name) }. from(false).to(true) end it "should add setter methods for #{name}" do method = name + '=' expect { subject.mk_resource_methods }. - to change { has_method?(subject, name) }. + to change { subject.method_defined?(name) }. from(false).to(true) end end diff --git a/spec/unit/reports/http_spec.rb b/spec/unit/reports/http_spec.rb index ace98f5ef..13f202607 100755 --- a/spec/unit/reports/http_spec.rb +++ b/spec/unit/reports/http_spec.rb @@ -89,7 +89,7 @@ describe processor do end end - if code.to_i >= 300 + if code.to_i >= 300 && ![301, 302, 307].include?(code.to_i) it "should log error on http code #{code}" do response = klass.new('1.1', code, '') http.expects(:post).returns(response) diff --git a/spec/unit/resource/catalog_spec.rb b/spec/unit/resource/catalog_spec.rb index d5e45e1d4..dab6e4591 100755 --- a/spec/unit/resource/catalog_spec.rb +++ b/spec/unit/resource/catalog_spec.rb @@ -260,7 +260,9 @@ describe Puppet::Resource::Catalog, "when compiling" do it "should add resources to the relationship graph if it exists" do relgraph = @catalog.relationship_graph + @catalog.add_resource @one + relgraph.should be_vertex(@one) end @@ -275,6 +277,21 @@ describe Puppet::Resource::Catalog, "when compiling" do @catalog.vertices.find { |r| r.ref == @one.ref }.must equal(@one) end + it "tracks the container through edges" do + @catalog.add_resource(@two) + @catalog.add_resource(@one) + + @catalog.add_edge(@one, @two) + + @catalog.container_of(@two).must == @one + end + + it "a resource without a container is contained in nil" do + @catalog.add_resource(@one) + + @catalog.container_of(@one).must be_nil + end + it "should canonize how resources are referred to during retrieval when both type and title are provided" do @catalog.add_resource(@one) @catalog.resource("notify", "one").must equal(@one) @@ -311,10 +328,6 @@ describe Puppet::Resource::Catalog, "when compiling" do end end - it "should not store objects that do not respond to :ref" do - proc { @catalog.add_resource("thing") }.should raise_error(ArgumentError) - end - it "should remove all resources when asked" do @catalog.add_resource @one @catalog.add_resource @two @@ -523,10 +536,9 @@ describe Puppet::Resource::Catalog, "when compiling" do before :each do @catalog = Puppet::Resource::Catalog.new("host") - @transaction = Puppet::Transaction.new(@catalog) + @transaction = Puppet::Transaction.new(@catalog, nil, Puppet::Graph::RandomPrioritizer.new) Puppet::Transaction.stubs(:new).returns(@transaction) @transaction.stubs(:evaluate) - @transaction.stubs(:add_times) @transaction.stubs(:for_network_device=) Puppet.settings.stubs(:use) @@ -537,18 +549,6 @@ describe Puppet::Resource::Catalog, "when compiling" do @catalog.apply end - it "should provide the catalog retrieval time to the transaction" do - @catalog.retrieval_duration = 5 - @transaction.expects(:add_times).with(:config_retrieval => 5) - @catalog.apply - end - - it "should use a retrieval time of 0 if none is set in the catalog" do - @catalog.retrieval_duration = nil - @transaction.expects(:add_times).with(:config_retrieval => 0) - @catalog.apply - end - it "should return the transaction" do @catalog.apply.should equal(@transaction) end @@ -627,90 +627,15 @@ describe Puppet::Resource::Catalog, "when compiling" do describe "when creating a relationship graph" do before do - Puppet::Type.type(:component) @catalog = Puppet::Resource::Catalog.new("host") - @compone = Puppet::Type::Component.new :name => "one" - @comptwo = Puppet::Type::Component.new :name => "two", :require => "Class[one]" - @file = Puppet::Type.type(:file) - @one = @file.new :path => @basepath+"/one" - @two = @file.new :path => @basepath+"/two" - @sub = @file.new :path => @basepath+"/two/subdir" - @catalog.add_edge @compone, @one - @catalog.add_edge @comptwo, @two - - @three = @file.new :path => @basepath+"/three" - @four = @file.new :path => @basepath+"/four", :require => "File[#{@basepath}/three]" - @five = @file.new :path => @basepath+"/five" - @catalog.add_resource @compone, @comptwo, @one, @two, @three, @four, @five, @sub - - @relationships = @catalog.relationship_graph - end - - it "should be able to create a relationship graph" do - @relationships.should be_instance_of(Puppet::SimpleGraph) - end - - it "should not have any components" do - @relationships.vertices.find { |r| r.instance_of?(Puppet::Type::Component) }.should be_nil - end - - it "should have all non-component resources from the catalog" do - # The failures print out too much info, so i just do a class comparison - @relationships.vertex?(@five).should be_true - end - - it "should have all resource relationships set as edges" do - @relationships.edge?(@three, @four).should be_true - end - - it "should copy component relationships to all contained resources" do - @relationships.path_between(@one, @two).should be - end - - it "should add automatic relationships to the relationship graph" do - @relationships.edge?(@two, @sub).should be_true end it "should get removed when the catalog is cleaned up" do - @relationships.expects(:clear) - @catalog.clear - @catalog.instance_variable_get("@relationship_graph").should be_nil - end + @catalog.relationship_graph.expects(:clear) - it "should write :relationships and :expanded_relationships graph files if the catalog is a host catalog" do @catalog.clear - graph = Puppet::SimpleGraph.new - Puppet::SimpleGraph.expects(:new).returns graph - - graph.expects(:write_graph).with(:relationships) - graph.expects(:write_graph).with(:expanded_relationships) - - @catalog.host_config = true - @catalog.relationship_graph - end - - it "should not write graph files if the catalog is not a host catalog" do - @catalog.clear - graph = Puppet::SimpleGraph.new - Puppet::SimpleGraph.expects(:new).returns graph - - graph.expects(:write_graph).never - - @catalog.host_config = false - - @catalog.relationship_graph - end - - it "should create a new relationship graph after clearing the old one" do - @relationships.expects(:clear) - @catalog.clear - @catalog.relationship_graph.should be_instance_of(Puppet::SimpleGraph) - end - - it "should remove removed resources from the relationship graph if it exists" do - @catalog.remove_resource(@one) - @catalog.relationship_graph.vertex?(@one).should be_false + @catalog.instance_variable_get("@relationship_graph").should be_nil end end @@ -902,8 +827,9 @@ describe Puppet::Resource::Catalog, "when converting from pson" do it 'should convert the resources list into resources and add each of them' do @data['resources'] = [Puppet::Resource.new(:file, "/foo"), Puppet::Resource.new(:file, "/bar")] - @catalog.expects(:add_resource).times(2).with { |res| res.type == "File" } - PSON.parse @pson.to_pson + catalog = PSON.parse @pson.to_pson + + catalog.resources.collect(&:ref) == ["File[/foo]", "File[/bar]"] end it 'should convert resources even if they do not include "type" information' do diff --git a/spec/unit/resource/resource_type.json b/spec/unit/resource/resource_type.json new file mode 100644 index 000000000..ffd15d639 --- /dev/null +++ b/spec/unit/resource/resource_type.json @@ -0,0 +1,34 @@ +{ + "type": "object", + "properties": { + "doc": { + "type": "string" + }, + "line": { + "type": "integer" + }, + "file": { + "type": "string" + }, + "parent": { + "type": "string" + }, + "name": { + "type": "string", + "required": "true" + }, + "kind": { + "type": "string", + "enum": [ + "class", + "node", + "defined_type" + ], + "required": "true" + }, + "parameters": { + "type": "object" + } + }, + "additionalProperties": false +} diff --git a/spec/unit/resource/status_spec.rb b/spec/unit/resource/status_spec.rb index 115b112cc..f3cc5699c 100755 --- a/spec/unit/resource/status_spec.rb +++ b/spec/unit/resource/status_spec.rb @@ -8,6 +8,8 @@ describe Puppet::Resource::Status do before do @resource = Puppet::Type.type(:file).new :path => make_absolute("/my/file") + @containment_path = ["foo", "bar", "baz"] + @resource.stubs(:pathbuilder).returns @containment_path @status = Puppet::Resource::Status.new(@resource) end @@ -44,6 +46,10 @@ describe Puppet::Resource::Status do Puppet::Resource::Status.new(@resource).source_description.should == "/my/path" end + it "should set its containment path" do + Puppet::Resource::Status.new(@resource).containment_path.should == @containment_path + end + [:file, :line].each do |attr| it "should copy the resource's #{attr}" do @resource.expects(attr).returns "foo" @@ -151,4 +157,54 @@ describe Puppet::Resource::Status do @status.to_yaml_properties.should =~ Puppet::Resource::Status::YAML_ATTRIBUTES end end + + it "should round trip through pson" do + @status.file = "/foo.rb" + @status.line = 27 + @status.evaluation_time = 2.7 + @status.tags = %w{one two} + @status << Puppet::Transaction::Event.new(:name => :mode_changed, :status => 'audit') + @status.failed = false + @status.changed = true + @status.out_of_sync = true + @status.skipped = false + + @status.containment_path.should == @containment_path + + tripped = Puppet::Resource::Status.from_pson(PSON.parse(@status.to_pson)) + + tripped.title.should == @status.title + tripped.containment_path.should == @status.containment_path + tripped.file.should == @status.file + tripped.line.should == @status.line + tripped.resource.should == @status.resource + tripped.resource_type.should == @status.resource_type + tripped.evaluation_time.should == @status.evaluation_time + tripped.tags.should == @status.tags + tripped.time.should == @status.time + tripped.failed.should == @status.failed + tripped.changed.should == @status.changed + tripped.out_of_sync.should == @status.out_of_sync + tripped.skipped.should == @status.skipped + + tripped.change_count.should == @status.change_count + tripped.out_of_sync_count.should == @status.out_of_sync_count + events_as_hashes(tripped).should == events_as_hashes(@status) + end + + def events_as_hashes(report) + report.events.collect do |e| + { + :audited => e.audited, + :property => e.property, + :previous_value => e.previous_value, + :desired_value => e.desired_value, + :historical_value => e.historical_value, + :message => e.message, + :name => e.name, + :status => e.status, + :time => e.time, + } + end + end end diff --git a/spec/unit/resource/type_collection_spec.rb b/spec/unit/resource/type_collection_spec.rb index 93b3ae988..c3a335ce2 100755 --- a/spec/unit/resource/type_collection_spec.rb +++ b/spec/unit/resource/type_collection_spec.rb @@ -190,7 +190,7 @@ describe Puppet::Resource::TypeCollection do @code.loader.expects(:try_load_fqname).with(:hostclass, "klass").returns(nil) @code.find_hostclass("Ns", "Klass").should be_nil - Puppet.expects(:warning).at_least_once.with {|msg| msg =~ /Not attempting to load hostclass/} + Puppet.expects(:debug).at_least_once.with {|msg| msg =~ /Not attempting to load hostclass/} @code.find_hostclass("Ns", "Klass").should be_nil end end @@ -369,7 +369,7 @@ describe Puppet::Resource::TypeCollection do describe "when managing files" do before do @loader = Puppet::Resource::TypeCollection.new("env") - Puppet::Util::LoadedFile.stubs(:new).returns stub("watched_file") + Puppet::Util::WatchedFile.stubs(:new).returns stub("watched_file") end it "should have a method for specifying a file should be watched" do @@ -381,15 +381,15 @@ describe Puppet::Resource::TypeCollection do @loader.should be_watching_file("/foo/bar") end - it "should use LoadedFile to watch files" do - Puppet::Util::LoadedFile.expects(:new).with("/foo/bar").returns stub("watched_file") + it "should use WatchedFile to watch files" do + Puppet::Util::WatchedFile.expects(:new).with("/foo/bar").returns stub("watched_file") @loader.watch_file("/foo/bar") end it "should be considered stale if any files have changed" do file1 = stub 'file1', :changed? => false file2 = stub 'file2', :changed? => true - Puppet::Util::LoadedFile.expects(:new).times(2).returns(file1).then.returns(file2) + Puppet::Util::WatchedFile.expects(:new).times(2).returns(file1).then.returns(file2) @loader.watch_file("/foo/bar") @loader.watch_file("/other/bar") @@ -399,7 +399,7 @@ describe Puppet::Resource::TypeCollection do it "should not be considered stable if no files have changed" do file1 = stub 'file1', :changed? => false file2 = stub 'file2', :changed? => false - Puppet::Util::LoadedFile.expects(:new).times(2).returns(file1).then.returns(file2) + Puppet::Util::WatchedFile.expects(:new).times(2).returns(file1).then.returns(file2) @loader.watch_file("/foo/bar") @loader.watch_file("/other/bar") diff --git a/spec/unit/resource/type_spec.rb b/spec/unit/resource/type_spec.rb index 2d06e9733..a4929208e 100755 --- a/spec/unit/resource/type_spec.rb +++ b/spec/unit/resource/type_spec.rb @@ -80,7 +80,7 @@ describe Puppet::Resource::Type do lambda { Puppet::Resource::Type.new(:node, /foo/) }.should_not raise_error end - it "should allow a AST::HostName instance as its name" do + it "should allow an AST::HostName instance as its name" do regex = Puppet::Parser::AST::Regex.new(:value => /foo/) name = Puppet::Parser::AST::HostName.new(:value => regex) lambda { Puppet::Resource::Type.new(:node, name) }.should_not raise_error @@ -134,6 +134,10 @@ describe Puppet::Resource::Type do Puppet::Resource::Type.new(:node, /\w/).match("foo").should be_true end + it "should return true when its regex matches the provided name" do + Puppet::Resource::Type.new(:node, /\w/).match("foo").should be_true + end + it "should return false when its regex does not match the provided name" do (!!Puppet::Resource::Type.new(:node, /\d/).match("foo")).should be_false end @@ -396,13 +400,29 @@ describe Puppet::Resource::Type do @resource.environment.known_resource_types.add @type end - it "should add hostclass names to the classes list" do + it "should add node regex captures to its scope" do + @type = Puppet::Resource::Type.new(:node, /f(\w)o(.*)$/) + match = @type.match('foo') + + code = stub 'code' + @type.stubs(:code).returns code + + subscope = stub 'subscope', :compiler => @compiler + @scope.expects(:newscope).with(:source => @type, :namespace => '', :resource => @resource).returns subscope + + elevel = 876 + subscope.expects(:ephemeral_level).returns elevel + subscope.expects(:ephemeral_from).with(match, nil, nil).returns subscope + code.expects(:safeevaluate).with(subscope) + subscope.expects(:unset_ephemeral_var).with(elevel) + + # Just to keep the stub quiet about intermediate calls + @type.expects(:set_resource_parameters).with(@resource, subscope) + @type.evaluate_code(@resource) - @compiler.catalog.classes.should be_include("foo") end - it "should add node names to the classes list" do - @type = Puppet::Resource::Type.new(:node, "foo") + it "should add hostclass names to the classes list" do @type.evaluate_code(@resource) @compiler.catalog.classes.should be_include("foo") end diff --git a/spec/unit/resource_spec.rb b/spec/unit/resource_spec.rb index 10ab2e1da..37d2001b8 100755 --- a/spec/unit/resource_spec.rb +++ b/spec/unit/resource_spec.rb @@ -267,42 +267,45 @@ describe Puppet::Resource do end describe "when setting default parameters" do - before do - @scope = mock "Scope" - @scope.stubs(:source).returns(nil) + let(:foo_node) { Puppet::Node.new('foo') } + let(:compiler) { Puppet::Parser::Compiler.new(foo_node) } + let(:scope) { Puppet::Parser::Scope.new(compiler) } + + def ast_string(value) + Puppet::Parser::AST::String.new({:value => value}) end it "should fail when asked to set default values and it is not a parser resource" do Puppet::Node::Environment.new.known_resource_types.add( - Puppet::Resource::Type.new(:definition, "default_param", :arguments => {"a" => Puppet::Parser::AST::String.new(:value => "default")}) + Puppet::Resource::Type.new(:definition, "default_param", :arguments => {"a" => ast_string("default")}) ) resource = Puppet::Resource.new("default_param", "name") - lambda { resource.set_default_parameters(@scope) }.should raise_error(Puppet::DevError) + lambda { resource.set_default_parameters(scope) }.should raise_error(Puppet::DevError) end it "should evaluate and set any default values when no value is provided" do Puppet::Node::Environment.new.known_resource_types.add( - Puppet::Resource::Type.new(:definition, "default_param", :arguments => {"a" => Puppet::Parser::AST::String.new(:value => "a_default_value")}) + Puppet::Resource::Type.new(:definition, "default_param", :arguments => {"a" => ast_string("a_default_value")}) ) - resource = Puppet::Parser::Resource.new("default_param", "name", :scope => Puppet::Parser::Scope.new(Puppet::Parser::Compiler.new(Puppet::Node.new("foo")))) - resource.set_default_parameters(@scope) + resource = Puppet::Parser::Resource.new("default_param", "name", :scope => scope) + resource.set_default_parameters(scope) resource["a"].should == "a_default_value" end it "should skip attributes with no default value" do Puppet::Node::Environment.new.known_resource_types.add( - Puppet::Resource::Type.new(:definition, "no_default_param", :arguments => {"a" => Puppet::Parser::AST::String.new(:value => "a_default_value")}) + Puppet::Resource::Type.new(:definition, "no_default_param", :arguments => {"a" => ast_string("a_default_value")}) ) - resource = Puppet::Parser::Resource.new("no_default_param", "name", :scope => Puppet::Parser::Scope.new(Puppet::Parser::Compiler.new(Puppet::Node.new("foo")))) - lambda { resource.set_default_parameters(@scope) }.should_not raise_error + resource = Puppet::Parser::Resource.new("no_default_param", "name", :scope => scope) + lambda { resource.set_default_parameters(scope) }.should_not raise_error end it "should return the list of default parameters set" do Puppet::Node::Environment.new.known_resource_types.add( - Puppet::Resource::Type.new(:definition, "default_param", :arguments => {"a" => Puppet::Parser::AST::String.new(:value => "a_default_value")}) + Puppet::Resource::Type.new(:definition, "default_param", :arguments => {"a" => ast_string("a_default_value")}) ) - resource = Puppet::Parser::Resource.new("default_param", "name", :scope => Puppet::Parser::Scope.new(Puppet::Parser::Compiler.new(Puppet::Node.new("foo")))) - resource.set_default_parameters(@scope).should == ["a"] + resource = Puppet::Parser::Resource.new("default_param", "name", :scope => scope) + resource.set_default_parameters(scope).should == ["a"] end describe "when the resource type is :hostclass" do @@ -315,26 +318,52 @@ describe Puppet::Resource do environment = Puppet::Node::Environment.new(environment_name) environment.known_resource_types.add(apache) - @scope.stubs(:host).returns('host') - @scope.stubs(:environment).returns(Puppet::Node::Environment.new(environment_name)) - @scope.stubs(:facts).returns(Puppet::Node::Facts.new("facts", fact_values)) + scope.stubs(:host).returns('host') + scope.stubs(:environment).returns(Puppet::Node::Environment.new(environment_name)) + scope.stubs(:facts).returns(Puppet::Node::Facts.new("facts", fact_values)) end context "when no value is provided" do + before(:each) do + Puppet[:binder] = true + end + let(:resource) do - Puppet::Parser::Resource.new("class", "apache", :scope => @scope) + Puppet::Parser::Resource.new("class", "apache", :scope => scope) end it "should query the data_binding terminus using a namespaced key" do Puppet::DataBinding.indirection.expects(:find).with( 'apache::port', all_of(has_key(:environment), has_key(:variables))) - resource.set_default_parameters(@scope) + resource.set_default_parameters(scope) + end + + it "should query the injector using a namespaced key" do + compiler.injector.expects(:lookup).with(scope, 'apache::port') + resource.set_default_parameters(scope) end it "should use the value from the data_binding terminus" do Puppet::DataBinding.indirection.expects(:find).returns('443') - resource.set_default_parameters(@scope) + resource.set_default_parameters(scope) + + resource[:port].should == '443' + end + + it "should use the value from the injector" do + compiler.injector.expects(:lookup).with(scope, 'apache::port').returns('443') + + resource.set_default_parameters(scope) + + resource[:port].should == '443' + end + + it "should not call the DataBinding terminus when injector produces a value" do + compiler.injector.expects(:lookup).with(scope, 'apache::port').returns('443') + Puppet::DataBinding.indirection.expects(:find).never() + + resource.set_default_parameters(scope) resource[:port].should == '443' end @@ -342,7 +371,15 @@ describe Puppet::Resource do it "should use the default value if the data_binding terminus returns nil" do Puppet::DataBinding.indirection.expects(:find).returns(nil) - resource.set_default_parameters(@scope) + resource.set_default_parameters(scope) + + resource[:port].should == '80' + end + + it "should use the default value if the injector returns nil" do + compiler.injector.expects(:lookup).returns(nil) + + resource.set_default_parameters(scope) resource[:port].should == '80' end @@ -356,18 +393,25 @@ describe Puppet::Resource do end let(:resource) do - Puppet::Parser::Resource.new("class", "apache", :scope => @scope, + Puppet::Parser::Resource.new("class", "apache", :scope => scope, :parameters => [port_parameter]) end it "should not query the data_binding terminus" do Puppet::DataBinding.indirection.expects(:find).never - resource.set_default_parameters(@scope) + resource.set_default_parameters(scope) + end + + it "should not query the injector" do + # enable the injector + Puppet[:binder] = true + compiler.injector.expects(:find).never + resource.set_default_parameters(scope) end it "should use the value provided" do Puppet::DataBinding.indirection.expects(:find).never - resource.set_default_parameters(@scope).should == [] + resource.set_default_parameters(scope).should == [] resource[:port].should == '8080' end end diff --git a/spec/unit/run_spec.rb b/spec/unit/run_spec.rb index aa31f0910..266f65334 100755 --- a/spec/unit/run_spec.rb +++ b/spec/unit/run_spec.rb @@ -155,5 +155,21 @@ describe Puppet::Run do run.background.should be_true run.status.should == 'success' end + + it "should round trip through pson" do + run = Puppet::Run.new( + :tags => ['a', 'b', 'c'], + :ignoreschedules => true, + :pluginsync => false, + :background => true + ) + run.instance_variable_set(:@status, true) + + tripped = Puppet::Run.convert_from(:pson, run.render(:pson)) + + tripped.options.should == run.options + tripped.status.should == run.status + tripped.background.should == run.background + end end end diff --git a/spec/unit/scheduler/scheduler_spec.rb b/spec/unit/scheduler/scheduler_spec.rb index 2a2971676..1bd289764 100644 --- a/spec/unit/scheduler/scheduler_spec.rb +++ b/spec/unit/scheduler/scheduler_spec.rb @@ -34,14 +34,15 @@ describe Puppet::Scheduler::Scheduler do job end + let(:scheduler) { Puppet::Scheduler::Scheduler.new(timer) } + it "uses the minimum interval" do later_job = one_time_job(7) earlier_job = one_time_job(2) - scheduler = Puppet::Scheduler::Scheduler.new([later_job, earlier_job], timer) later_job.last_run = now earlier_job.last_run = now - scheduler.run_loop + scheduler.run_loop([later_job, earlier_job]) timer.wait_for_calls.should == [2, 5] end @@ -50,9 +51,8 @@ describe Puppet::Scheduler::Scheduler do enabled = one_time_job(7) enabled.last_run = now disabled = disabled_job(2) - scheduler = Puppet::Scheduler::Scheduler.new([enabled, disabled], timer) - scheduler.run_loop + scheduler.run_loop([enabled, disabled]) timer.wait_for_calls.should == [7] end @@ -60,46 +60,35 @@ describe Puppet::Scheduler::Scheduler do it "asks the timer to wait for the job interval" do job = one_time_job(5) job.last_run = now - scheduler = Puppet::Scheduler::Scheduler.new([job], timer) - scheduler.run_loop + scheduler.run_loop([job]) timer.wait_for_calls.should == [5] end it "does not run when there are no jobs" do - timer = mock 'no run timer' - scheduler = Puppet::Scheduler::Scheduler.new([], timer) - - timer.stubs(:now).returns(now) - timer.expects(:wait_for).never + scheduler.run_loop([]) - scheduler.run_loop + timer.wait_for_calls.should be_empty end it "does not run when there are only disabled jobs" do - timer = mock 'no run timer' disabled_job = Puppet::Scheduler::Job.new(0) - scheduler = Puppet::Scheduler::Scheduler.new([disabled_job], timer) - disabled_job.disable - timer.stubs(:now).returns(now) - timer.expects(:wait_for).never - scheduler.run_loop + scheduler.run_loop([disabled_job]) + + timer.wait_for_calls.should be_empty end it "stops running when there are no more enabled jobs" do - timer = mock 'run once timer' disabling_job = Puppet::Scheduler::Job.new(0) do |j| j.disable end - scheduler = Puppet::Scheduler::Scheduler.new([disabling_job], timer) - timer.stubs(:now).returns(now) - timer.expects(:wait_for).once + scheduler.run_loop([disabling_job]) - scheduler.run_loop + timer.wait_for_calls.size.should == 1 end it "marks the start of the run loop" do @@ -107,8 +96,7 @@ describe Puppet::Scheduler::Scheduler do disabled_job.disable - scheduler = Puppet::Scheduler::Scheduler.new([disabled_job], timer) - scheduler.run_loop + scheduler.run_loop([disabled_job]) disabled_job.start_time.should == now end @@ -121,8 +109,7 @@ describe Puppet::Scheduler::Scheduler do job.disable if countdown == 0 end - scheduler = Puppet::Scheduler::Scheduler.new([slow_job], timer) - scheduler.run_loop + scheduler.run_loop([slow_job]) timer.wait_for_calls.should == [0, 3, 7, 3] end diff --git a/spec/unit/semver_spec.rb b/spec/unit/semver_spec.rb index 9d924a1f9..d4fc5da04 100644 --- a/spec/unit/semver_spec.rb +++ b/spec/unit/semver_spec.rb @@ -2,6 +2,11 @@ require 'spec_helper' require 'semver' describe SemVer do + + describe 'MAX should be +Infinity' do + SemVer::MAX.major.infinite?.should == 1 + end + describe '::valid?' do it 'should validate basic version strings' do %w[ 0.0.0 999.999.999 v0.0.0 v999.999.999 ].each do |vstring| diff --git a/spec/unit/settings/enum_setting_spec.rb b/spec/unit/settings/enum_setting_spec.rb new file mode 100644 index 000000000..6be03670d --- /dev/null +++ b/spec/unit/settings/enum_setting_spec.rb @@ -0,0 +1,27 @@ +require 'spec_helper' + +require 'puppet/settings' + +describe Puppet::Settings::EnumSetting do + it "allows a configured value" do + setting = enum_setting_allowing("allowed") + + expect(setting.munge("allowed")).to eq("allowed") + end + + it "disallows a value that is not configured" do + setting = enum_setting_allowing("allowed", "also allowed") + + expect do + setting.munge("disallowed") + end.to raise_error(Puppet::Settings::ValidationError, + "Invalid value 'disallowed' for parameter testing. Allowed values are 'allowed', 'also allowed'") + end + + def enum_setting_allowing(*values) + Puppet::Settings::EnumSetting.new(:settings => mock('settings'), + :name => "testing", + :desc => "description of testing", + :values => values) + end +end diff --git a/spec/unit/settings_spec.rb b/spec/unit/settings_spec.rb index 42013ac64..12107bcb7 100755 --- a/spec/unit/settings_spec.rb +++ b/spec/unit/settings_spec.rb @@ -266,7 +266,7 @@ describe Puppet::Settings do @settings[:myval] = "12" @settings.set_by_cli?(:myval).should be_false end - + describe "setbycli" do it "should generate a deprecation warning" do Puppet.expects(:deprecation_warning) @@ -289,11 +289,17 @@ describe Puppet::Settings do @settings.define_settings :mysection, :one => { :default => "whah", :desc => "yay" }, :two => { :default => "$one yay", :desc => "bah" } + @settings.expects(:unsafe_flush_cache) @settings[:two].should == "whah yay" @settings.handlearg("--one", "else") @settings[:two].should == "else yay" end + it "should clear the cache when the preferred_run_mode is changed" do + @settings.expects(:flush_cache) + @settings.preferred_run_mode = :master + end + it "should not clear other values when setting getopt-specific values" do @settings[:myval] = "yay" @settings.handlearg("--no-bool", "") @@ -477,7 +483,8 @@ describe Puppet::Settings do :one => { :default => "ONE", :desc => "a" }, :two => { :default => "$one TWO", :desc => "b"}, :three => { :default => "$one $two THREE", :desc => "c"}, - :four => { :default => "$two $three FOUR", :desc => "d"} + :four => { :default => "$two $three FOUR", :desc => "d"}, + :five => { :default => nil, :desc => "e" } FileTest.stubs(:exist?).returns true end @@ -535,6 +542,20 @@ describe Puppet::Settings do @settings[:two].should == "one TWO" end + describe "caching values that evaluate to false" do + it "caches nil" do + @settings.expects(:convert).once.returns nil + @settings[:five].should be_nil + @settings[:five].should be_nil + end + + it "caches false" do + @settings.expects(:convert).once.returns false + @settings[:five].should == false + @settings[:five].should == false + end + end + it "should not cache values such that information from one environment is returned for another environment" do text = "[env1]\none = oneval\n[env2]\none = twoval\n" @settings.stubs(:read_file).returns(text) @@ -984,65 +1005,53 @@ describe Puppet::Settings do @settings.stubs(:user_config_file).returns(@userconfig) end - it "should use a LoadedFile instance to determine if the file has changed" do - file = mock 'file' - Puppet::Util::LoadedFile.expects(:new).with(@file).returns file - - file.expects(:changed?) - - @settings.stubs(:parse) - @settings.reparse_config_files - end - - it "should not create the LoadedFile instance and should not parse if the file does not exist" do + it "does not create the WatchedFile instance and should not parse if the file does not exist" do FileTest.expects(:exist?).with(@file).returns false - Puppet::Util::LoadedFile.expects(:new).never + Puppet::Util::WatchedFile.expects(:new).never @settings.expects(:parse_config_files).never @settings.reparse_config_files end - it "should not reparse if the file has not changed" do - file = mock 'file' - Puppet::Util::LoadedFile.expects(:new).with(@file).returns file + context "and watched file exists" do + before do + @watched_file = Puppet::Util::WatchedFile.new(@file) + Puppet::Util::WatchedFile.expects(:new).with(@file).returns @watched_file + end - file.expects(:changed?).returns false + it "uses a WatchedFile instance to determine if the file has changed" do + @watched_file.expects(:changed?) - @settings.expects(:parse_config_files).never + @settings.reparse_config_files + end - @settings.reparse_config_files - end + it "does not reparse if the file has not changed" do + @watched_file.expects(:changed?).returns false - it "should reparse if the file has changed" do - file = stub 'file', :file => @file - Puppet::Util::LoadedFile.expects(:new).with(@file).returns file + @settings.expects(:parse_config_files).never - file.expects(:changed?).returns true + @settings.reparse_config_files + end - @settings.expects(:parse_config_files) + it "reparses if the file has changed" do + @watched_file.expects(:changed?).returns true - @settings.reparse_config_files - end + @settings.expects(:unsafe_parse).with(@file) - it "should replace in-memory values with on-file values" do - # Init the value - text = "[main]\none = disk-init\n" - file = mock 'file' - file.stubs(:changed?).returns(true) - file.stubs(:file).returns(@file) - @settings[:one] = "init" - @settings.files = [file] + @settings.reparse_config_files + end - # Now replace the value - text = "[main]\none = disk-replace\n" + it "replaces in-memory values with on-file values" do + @watched_file.stubs(:changed?).returns(true) + @settings[:one] = "init" - # This is kinda ridiculous - the reason it parses twice is that - # it goes to parse again when we ask for the value, because the - # mock always says it should get reparsed. - @settings.stubs(:read_file).returns(text) - @settings.reparse_config_files - @settings[:one].should == "disk-replace" + # Now replace the value + text = "[main]\none = disk-replace\n" + @settings.stubs(:read_file).returns(text) + @settings.reparse_config_files + @settings[:one].should == "disk-replace" + end end it "should retain parameters set by cli when configuration files are reparsed" do diff --git a/spec/unit/ssl/certificate_authority_spec.rb b/spec/unit/ssl/certificate_authority_spec.rb index 9d007a312..70186a137 100755 --- a/spec/unit/ssl/certificate_authority_spec.rb +++ b/spec/unit/ssl/certificate_authority_spec.rb @@ -333,7 +333,7 @@ describe Puppet::SSL::CertificateAuthority do expect do @ca.sign(@name, false, @request) - end.not_to raise_error(Puppet::SSL::CertificateAuthority::CertificateSigningError) + end.not_to raise_error end it "should save the resulting certificate" do @@ -748,6 +748,13 @@ describe Puppet::SSL::CertificateAuthority do @ca.list.should == %w{cert1 cert2} end + it "should list the full certificates" do + cert1 = stub 'cert1', :name => "cert1" + cert2 = stub 'cert2', :name => "cert2" + Puppet::SSL::Certificate.indirection.expects(:search).with("*").returns [cert1, cert2] + @ca.list_certificates.should == [cert1, cert2] + end + describe "and printing certificates" do it "should return nil if the certificate cannot be found" do Puppet::SSL::Certificate.indirection.expects(:find).with("myhost").returns nil @@ -860,6 +867,35 @@ describe Puppet::SSL::CertificateAuthority do expect { @ca.verify("me") }.to raise_error end + + describe "certificate_is_alive?" do + it "should return false if verification fails" do + @cert.expects(:content).returns "mycert" + + @store.expects(:verify).with("mycert").returns false + + @ca.certificate_is_alive?(@cert).should be_false + end + + it "should return true if verification passes" do + @cert.expects(:content).returns "mycert" + + @store.expects(:verify).with("mycert").returns true + + @ca.certificate_is_alive?(@cert).should be_true + end + + it "should used a cached instance of the x509 store" do + OpenSSL::X509::Store.stubs(:new).returns(@store).once + + @cert.expects(:content).returns "mycert" + + @store.expects(:verify).with("mycert").returns true + + @ca.certificate_is_alive?(@cert) + @ca.certificate_is_alive?(@cert) + end + end end describe "and revoking certificates" do @@ -935,37 +971,137 @@ describe Puppet::SSL::CertificateAuthority do it "should be able to generate a complete new SSL host" do @ca.should respond_to(:generate) end + end +end - describe "and generating certificates" do - before do - @host = stub 'host', :generate_certificate_request => nil - Puppet::SSL::Host.stubs(:new).returns @host - Puppet::SSL::Certificate.indirection.stubs(:find).returns nil +require 'puppet/indirector/memory' + +describe "CertificateAuthority.generate" do + + def expect_to_increment_serial_file + Puppet.settings.expects(:readwritelock).with(:serial) + end + + def expect_to_sign_a_cert + expect_to_increment_serial_file + Puppet.settings.expects(:write).with(:cert_inventory, "a") + end + + def expect_to_write_the_ca_password + Puppet.settings.expects(:write).with(:capass) + end + + def expect_ca_initialization + expect_to_write_the_ca_password + expect_to_sign_a_cert + end + + def avoid_rebuilding_inventory_for_every_cert + Puppet::SSL::Inventory.any_instance.stubs(:rebuild) + end - @ca.stubs(:sign) + INDIRECTED_CLASSES = [ + Puppet::SSL::Certificate, + Puppet::SSL::CertificateRequest, + Puppet::SSL::CertificateRevocationList, + Puppet::SSL::Key, + ] + + INDIRECTED_CLASSES.each do |const| + class const::Memory < Puppet::Indirector::Memory + + # @return Array of all the indirector's values + # + # This mirrors Puppet::Indirector::SslFile#search which returns all files + # in the directory. + def search(request) + return @instances.values + end + end + end + + before do + avoid_rebuilding_inventory_for_every_cert + INDIRECTED_CLASSES.each { |const| const.indirection.terminus_class = :memory } + end + + after do + INDIRECTED_CLASSES.each do |const| + const.indirection.terminus_class = :file + const.indirection.termini.clear + end + end + + describe "when generating certificates" do + let(:ca) { Puppet::SSL::CertificateAuthority.new } + + before do + expect_ca_initialization + end + + it "should fail if a certificate already exists for the host" do + cert = Puppet::SSL::Certificate.new('pre.existing') + Puppet::SSL::Certificate.indirection.save(cert) + expect { ca.generate(cert.name) }.to raise_error(ArgumentError, /a certificate already exists/i) + end + + describe "that do not yet exist" do + let(:cn) { "new.host" } + + def expect_cert_does_not_exist(cn) + expect( Puppet::SSL::Certificate.indirection.find(cn) ).to be_nil end - it "should fail if a certificate already exists for the host" do - Puppet::SSL::Certificate.indirection.expects(:find).with("him").returns "something" + before do + expect_to_sign_a_cert + expect_cert_does_not_exist(cn) + end - expect { @ca.generate("him") }.to raise_error(ArgumentError) + it "should return the created certificate" do + cert = ca.generate(cn) + expect( cert ).to be_kind_of(Puppet::SSL::Certificate) + expect( cert.name ).to eq(cn) end - it "should create a new Host instance with the correct name" do - Puppet::SSL::Host.expects(:new).with("him").returns @host + it "should not have any subject_alt_names by default" do + cert = ca.generate(cn) + expect( cert.subject_alt_names ).to be_empty + end - @ca.generate("him") + it "should have subject_alt_names if passed dns_alt_names" do + cert = ca.generate(cn, :dns_alt_names => 'foo,bar') + expect( cert.subject_alt_names ).to match_array(["DNS:#{cn}",'DNS:foo','DNS:bar']) end - it "should use the Host to generate the certificate request" do - @host.expects :generate_certificate_request + context "if autosign is false" do + before do + Puppet[:autosign] = false + end - @ca.generate("him") + it "should still generate and explicitly sign the request" do + cert = nil + cert = ca.generate(cn) + expect(cert.name).to eq(cn) + end end - it "should sign the generated request" do - @ca.expects(:sign).with("him", false) - @ca.generate("him") + context "if autosign is true (Redmine #6112)" do + + def run_mode_must_be_master_for_autosign_to_be_attempted + Puppet.stubs(:run_mode).returns(Puppet::Util::RunMode[:master]) + end + + before do + Puppet[:autosign] = true + run_mode_must_be_master_for_autosign_to_be_attempted + Puppet::Util::Log.level = :info + end + + it "should generate a cert without attempting to sign again" do + cert = ca.generate(cn) + expect(cert.name).to eq(cn) + expect(@logs.map(&:message)).to include("Autosigning #{cn}") + end end end end diff --git a/spec/unit/transaction/additional_resource_generator_spec.rb b/spec/unit/transaction/additional_resource_generator_spec.rb new file mode 100644 index 000000000..d6f99ede3 --- /dev/null +++ b/spec/unit/transaction/additional_resource_generator_spec.rb @@ -0,0 +1,419 @@ +require 'spec_helper' +require 'puppet/transaction' +require 'puppet_spec/compiler' +require 'matchers/relationship_graph_matchers' +require 'matchers/include_in_order' + +describe Puppet::Transaction::AdditionalResourceGenerator do + include PuppetSpec::Compiler + include PuppetSpec::Files + include RelationshipGraphMatchers + + let(:prioritizer) { Puppet::Graph::SequentialPrioritizer.new } + + def find_vertex(graph, type, title) + graph.vertices.find {|v| v.type == type and v.title == title} + end + + Puppet::Type.newtype(:generator) do + include PuppetSpec::Compiler + + newparam(:name) do + isnamevar + end + + newparam(:kind) do + defaultto :eval_generate + newvalues(:eval_generate, :generate) + end + + newparam(:code) + + def respond_to?(method_name) + method_name == self[:kind] || super + end + + def eval_generate + eval_code + end + + def generate + eval_code + end + + def eval_code + if self[:code] + compile_to_ral(self[:code]).resources.select { |r| r.ref =~ /Notify/ } + else + [] + end + end + end + + context "when applying eval_generate" do + it "should add the generated resources to the catalog" do + catalog = compile_to_ral(<<-MANIFEST) + generator { thing: + code => 'notify { hello: }' + } + MANIFEST + + eval_generate_resources_in(catalog, relationship_graph_for(catalog), 'Generator[thing]') + + expect(catalog).to have_resource('Notify[hello]') + end + + it "should add a sentinel whit for the resource" do + graph = relationships_after_eval_generating(<<-MANIFEST, 'Generator[thing]') + generator { thing: + code => 'notify { hello: }' + } + MANIFEST + + find_vertex(graph, :whit, "completed_thing").must be_a(Puppet::Type.type(:whit)) + end + + it "should replace dependencies on the resource with dependencies on the sentinel" do + graph = relationships_after_eval_generating(<<-MANIFEST, 'Generator[thing]') + generator { thing: + code => 'notify { hello: }' + } + + notify { last: require => Generator['thing'] } + MANIFEST + + expect(graph).to enforce_order_with_edge( + 'Whit[completed_thing]', 'Notify[last]') + end + + it "should add an edge from the nearest ancestor to the generated resource" do + graph = relationships_after_eval_generating(<<-MANIFEST, 'Generator[thing]') + generator { thing: + code => 'notify { hello: } notify { goodbye: }' + } + MANIFEST + + expect(graph).to enforce_order_with_edge( + 'Generator[thing]', 'Notify[hello]') + expect(graph).to enforce_order_with_edge( + 'Generator[thing]', 'Notify[goodbye]') + end + + it "should add an edge from each generated resource to the sentinel" do + graph = relationships_after_eval_generating(<<-MANIFEST, 'Generator[thing]') + generator { thing: + code => 'notify { hello: } notify { goodbye: }' + } + MANIFEST + + expect(graph).to enforce_order_with_edge( + 'Notify[hello]', 'Whit[completed_thing]') + expect(graph).to enforce_order_with_edge( + 'Notify[goodbye]', 'Whit[completed_thing]') + end + + it "should add an edge from the resource to the sentinel" do + graph = relationships_after_eval_generating(<<-MANIFEST, 'Generator[thing]') + generator { thing: + code => 'notify { hello: }' + } + MANIFEST + + expect(graph).to enforce_order_with_edge( + 'Generator[thing]', 'Whit[completed_thing]') + end + + it "should contain the generated resources in the same container as the generator" do + catalog = compile_to_ral(<<-MANIFEST) + class container { + generator { thing: + code => 'notify { hello: }' + } + } + + include container + MANIFEST + + eval_generate_resources_in(catalog, relationship_graph_for(catalog), 'Generator[thing]') + + expect(catalog).to contain_resources_equally('Generator[thing]', 'Notify[hello]') + end + + it "should return false if an error occured when generating resources" do + catalog = compile_to_ral(<<-MANIFEST) + generator { thing: + code => 'fail("not a good generation")' + } + MANIFEST + + generator = Puppet::Transaction::AdditionalResourceGenerator.new(catalog, relationship_graph_for(catalog), prioritizer) + + expect(generator.eval_generate(catalog.resource('Generator[thing]'))). + to eq(false) + end + + it "should return true if resources were generated" do + catalog = compile_to_ral(<<-MANIFEST) + generator { thing: + code => 'notify { hello: }' + } + MANIFEST + + generator = Puppet::Transaction::AdditionalResourceGenerator.new(catalog, relationship_graph_for(catalog), prioritizer) + + expect(generator.eval_generate(catalog.resource('Generator[thing]'))). + to eq(true) + end + + it "should not add a sentinel if no resources are generated" do + catalog = compile_to_ral(<<-MANIFEST) + generator { thing: } + MANIFEST + relationship_graph = relationship_graph_for(catalog) + + generator = Puppet::Transaction::AdditionalResourceGenerator.new(catalog, relationship_graph, prioritizer) + + expect(generator.eval_generate(catalog.resource('Generator[thing]'))). + to eq(false) + expect(find_vertex(relationship_graph, :whit, "completed_thing")).to be_nil + end + + it "orders generated resources with the generator" do + graph = relationships_after_eval_generating(<<-MANIFEST, 'Generator[thing]') + notify { before: } + generator { thing: + code => 'notify { hello: }' + } + notify { after: } + MANIFEST + + expect(order_resources_traversed_in(graph)).to( + include_in_order("Notify[before]", "Generator[thing]", "Notify[hello]", "Notify[after]")) + end + + it "orders the generator in manifest order with dependencies" do + graph = relationships_after_eval_generating(<<-MANIFEST, 'Generator[thing]') + notify { before: } + generator { thing: + code => 'notify { hello: } notify { goodbye: }' + } + notify { third: require => Generator['thing'] } + notify { after: } + MANIFEST + + expect(order_resources_traversed_in(graph)).to( + include_in_order("Notify[before]", + "Generator[thing]", + "Notify[hello]", + "Notify[goodbye]", + "Notify[third]", + "Notify[after]")) + end + + it "duplicate generated resources are made dependent on the generator" do + graph = relationships_after_eval_generating(<<-MANIFEST, 'Generator[thing]') + notify { before: } + notify { hello: } + generator { thing: + code => 'notify { before: }' + } + notify { third: require => Generator['thing'] } + notify { after: } + MANIFEST + + expect(order_resources_traversed_in(graph)).to( + include_in_order("Notify[hello]", "Generator[thing]", "Notify[before]", "Notify[third]", "Notify[after]")) + end + + it "preserves dependencies on duplicate generated resources" do + graph = relationships_after_eval_generating(<<-MANIFEST, 'Generator[thing]') + notify { before: } + generator { thing: + code => 'notify { hello: } notify { before: }', + require => 'Notify[before]' + } + notify { third: require => Generator['thing'] } + notify { after: } + MANIFEST + + expect(order_resources_traversed_in(graph)).to( + include_in_order("Notify[before]", "Generator[thing]", "Notify[hello]", "Notify[third]", "Notify[after]")) + end + + def relationships_after_eval_generating(manifest, resource_to_generate) + catalog = compile_to_ral(manifest) + relationship_graph = relationship_graph_for(catalog) + + eval_generate_resources_in(catalog, relationship_graph, resource_to_generate) + + relationship_graph + end + + def eval_generate_resources_in(catalog, relationship_graph, resource_to_generate) + generator = Puppet::Transaction::AdditionalResourceGenerator.new(catalog, relationship_graph, prioritizer) + generator.eval_generate(catalog.resource(resource_to_generate)) + end + end + + context "when applying generate" do + it "should add the generated resources to the catalog" do + catalog = compile_to_ral(<<-MANIFEST) + generator { thing: + kind => generate, + code => 'notify { hello: }' + } + MANIFEST + + generate_resources_in(catalog, relationship_graph_for(catalog), 'Generator[thing]') + + expect(catalog).to have_resource('Notify[hello]') + end + + it "should contain the generated resources in the same container as the generator" do + catalog = compile_to_ral(<<-MANIFEST) + class container { + generator { thing: + kind => generate, + code => 'notify { hello: }' + } + } + + include container + MANIFEST + + generate_resources_in(catalog, relationship_graph_for(catalog), 'Generator[thing]') + + expect(catalog).to contain_resources_equally('Generator[thing]', 'Notify[hello]') + end + + it "should add an edge from the nearest ancestor to the generated resource" do + graph = relationships_after_generating(<<-MANIFEST, 'Generator[thing]') + generator { thing: + kind => generate, + code => 'notify { hello: } notify { goodbye: }' + } + MANIFEST + + expect(graph).to enforce_order_with_edge( + 'Generator[thing]', 'Notify[hello]') + expect(graph).to enforce_order_with_edge( + 'Generator[thing]', 'Notify[goodbye]') + end + + it "orders generated resources with the generator" do + graph = relationships_after_generating(<<-MANIFEST, 'Generator[thing]') + notify { before: } + generator { thing: + kind => generate, + code => 'notify { hello: }' + } + notify { after: } + MANIFEST + + expect(order_resources_traversed_in(graph)).to( + include_in_order("Notify[before]", "Generator[thing]", "Notify[hello]", "Notify[after]")) + end + + it "duplicate generated resources are made dependent on the generator" do + graph = relationships_after_generating(<<-MANIFEST, 'Generator[thing]') + notify { before: } + notify { hello: } + generator { thing: + kind => generate, + code => 'notify { before: }' + } + notify { third: require => Generator['thing'] } + notify { after: } + MANIFEST + + expect(order_resources_traversed_in(graph)).to( + include_in_order("Notify[hello]", "Generator[thing]", "Notify[before]", "Notify[third]", "Notify[after]")) + end + + it "preserves dependencies on duplicate generated resources" do + graph = relationships_after_generating(<<-MANIFEST, 'Generator[thing]') + notify { before: } + generator { thing: + kind => generate, + code => 'notify { hello: } notify { before: }', + require => 'Notify[before]' + } + notify { third: require => Generator['thing'] } + notify { after: } + MANIFEST + + expect(order_resources_traversed_in(graph)).to( + include_in_order("Notify[before]", "Generator[thing]", "Notify[hello]", "Notify[third]", "Notify[after]")) + end + + it "orders the generator in manifest order with dependencies" do + graph = relationships_after_generating(<<-MANIFEST, 'Generator[thing]') + notify { before: } + generator { thing: + kind => generate, + code => 'notify { hello: } notify { goodbye: }' + } + notify { third: require => Generator['thing'] } + notify { after: } + MANIFEST + + expect(order_resources_traversed_in(graph)).to( + include_in_order("Notify[before]", + "Generator[thing]", + "Notify[hello]", + "Notify[goodbye]", + "Notify[third]", + "Notify[after]")) + end + + def relationships_after_generating(manifest, resource_to_generate) + catalog = compile_to_ral(manifest) + relationship_graph = relationship_graph_for(catalog) + + generate_resources_in(catalog, relationship_graph, resource_to_generate) + + relationship_graph + end + + def generate_resources_in(catalog, relationship_graph, resource_to_generate) + generator = Puppet::Transaction::AdditionalResourceGenerator.new(catalog, relationship_graph, prioritizer) + generator.generate_additional_resources(catalog.resource(resource_to_generate)) + end + end + + def relationship_graph_for(catalog) + relationship_graph = Puppet::Graph::RelationshipGraph.new(prioritizer) + relationship_graph.populate_from(catalog) + relationship_graph + end + + def order_resources_traversed_in(relationships) + order_seen = [] + relationships.traverse { |resource| order_seen << resource.ref } + order_seen + end + + RSpec::Matchers.define :contain_resources_equally do |*resource_refs| + match do |catalog| + @containers = resource_refs.collect do |resource_ref| + catalog.container_of(catalog.resource(resource_ref)).ref + end + + @containers.all? { |resource_ref| resource_ref == @containers[0] } + end + + def failure_message_for_should + "expected #{@expected.join(', ')} to all be contained in the same resource but the containment was #{@expected.zip(@containers).collect { |(res, container)| res + ' => ' + container }.join(', ')}" + end + end +end + +RSpec::Matchers.define :have_resource do |expected_resource| + match do |actual_catalog| + actual_catalog.resource(expected_resource) + end + + def failure_message_for_should + "expected #{@actual.to_dot} to include #{@expected[0]}" + end +end diff --git a/spec/unit/transaction/event_manager_spec.rb b/spec/unit/transaction/event_manager_spec.rb index 610389c4c..84a8b93a6 100755 --- a/spec/unit/transaction/event_manager_spec.rb +++ b/spec/unit/transaction/event_manager_spec.rb @@ -147,7 +147,7 @@ describe Puppet::Transaction::EventManager do describe "when processing events for a given resource" do before do - @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new) + @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new, nil, nil) @manager = Puppet::Transaction::EventManager.new(@transaction) @manager.stubs(:queue_events) @@ -271,7 +271,7 @@ describe Puppet::Transaction::EventManager do describe "when queueing then processing events for a given resource" do before do - @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new) + @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new, nil, nil) @manager = Puppet::Transaction::EventManager.new(@transaction) @resource = Puppet::Type.type(:file).new :path => make_absolute("/my/file") diff --git a/spec/unit/transaction/event_spec.rb b/spec/unit/transaction/event_spec.rb index 2a519368c..a60e6e907 100755 --- a/spec/unit/transaction/event_spec.rb +++ b/spec/unit/transaction/event_spec.rb @@ -140,4 +140,61 @@ describe Puppet::Transaction::Event do event.to_yaml_properties.should =~ Puppet::Transaction::Event::YAML_ATTRIBUTES end end + + it "should round trip through pson" do + resource = Puppet::Type.type(:file).new(:title => make_absolute("/tmp/foo")) + event = Puppet::Transaction::Event.new( + :source_description => "/my/param", + :resource => resource, + :file => "/foo.rb", + :line => 27, + :tags => %w{one two}, + :desired_value => 7, + :historical_value => 'Brazil', + :message => "Help I'm trapped in a spec test", + :name => :mode_changed, + :previous_value => 6, + :property => :mode, + :status => 'success') + + tripped = Puppet::Transaction::Event.from_pson(PSON.parse(event.to_pson)) + + tripped.audited.should == event.audited + tripped.property.should == event.property + tripped.previous_value.should == event.previous_value + tripped.desired_value.should == event.desired_value + tripped.historical_value.should == event.historical_value + tripped.message.should == event.message + tripped.name.should == event.name + tripped.status.should == event.status + tripped.time.should == event.time + end + + it "should round trip an event for an inspect report through pson" do + resource = Puppet::Type.type(:file).new(:title => make_absolute("/tmp/foo")) + event = Puppet::Transaction::Event.new( + :audited => true, + :source_description => "/my/param", + :resource => resource, + :file => "/foo.rb", + :line => 27, + :tags => %w{one two}, + :message => "Help I'm trapped in a spec test", + :previous_value => 6, + :property => :mode, + :status => 'success') + + tripped = Puppet::Transaction::Event.from_pson(PSON.parse(event.to_pson)) + + tripped.desired_value.should be_nil + tripped.historical_value.should be_nil + tripped.name.should be_nil + + tripped.audited.should == event.audited + tripped.property.should == event.property + tripped.previous_value.should == event.previous_value + tripped.message.should == event.message + tripped.status.should == event.status + tripped.time.should == event.time + end end diff --git a/spec/unit/transaction/report_spec.rb b/spec/unit/transaction/report_spec.rb index 136642292..59278d393 100755 --- a/spec/unit/transaction/report_spec.rb +++ b/spec/unit/transaction/report_spec.rb @@ -1,6 +1,7 @@ #! /usr/bin/env ruby require 'spec_helper' +require 'puppet' require 'puppet/transaction/report' describe Puppet::Transaction::Report do @@ -32,12 +33,22 @@ describe Puppet::Transaction::Report do Puppet::Transaction::Report.new("inspect", "some configuration version", "some environment").configuration_version.should == "some configuration version" end + it "should take a 'transaction_uuid' as an argument" do + Puppet::Transaction::Report.new("inspect", "some configuration version", "some environment", "some transaction uuid").transaction_uuid.should == "some transaction uuid" + end + it "should be able to set configuration_version" do report = Puppet::Transaction::Report.new("inspect") report.configuration_version = "some version" report.configuration_version.should == "some version" end + it "should be able to set transaction_uuid" do + report = Puppet::Transaction::Report.new("inspect") + report.transaction_uuid = "some transaction uuid" + report.transaction_uuid.should == "some transaction uuid" + end + it "should take 'environment' as an argument" do Puppet::Transaction::Report.new("inspect", "some configuration version", "some environment").environment.should == "some environment" end @@ -82,6 +93,18 @@ describe Puppet::Transaction::Report do end end + describe "#as_logging_destination" do + it "makes the report collect logs during the block " do + log_string = 'Hello test report!' + report = Puppet::Transaction::Report.new('test') + report.as_logging_destination do + Puppet.err(log_string) + end + + expect(report.logs.collect(&:message)).to include(log_string) + end + end + describe "when accepting resource statuses" do before do @report = Puppet::Transaction::Report.new("apply") @@ -351,4 +374,92 @@ describe Puppet::Transaction::Report do report.to_yaml_properties.should_not include('@external_times') end end + + it "defaults to serializing to pson" do + expect(Puppet::Transaction::Report.supported_formats).to eq([:pson]) + end + + it "can make a round trip through pson" do + Puppet[:report_serialization_format] = "pson" + report = generate_report + + tripped = Puppet::Transaction::Report.convert_from(:pson, report.render) + + expect_equivalent_reports(tripped, report) + end + + it "can make a round trip through yaml" do + Puppet[:report_serialization_format] = "yaml" + report = generate_report + + yaml_output = report.render + tripped = Puppet::Transaction::Report.convert_from(:yaml, yaml_output) + + yaml_output.should =~ /^--- / + expect_equivalent_reports(tripped, report) + end + + def expect_equivalent_reports(tripped, report) + tripped.host.should == report.host + tripped.time.to_i.should == report.time.to_i + tripped.configuration_version.should == report.configuration_version + tripped.transaction_uuid.should == report.transaction_uuid + tripped.report_format.should == report.report_format + tripped.puppet_version.should == report.puppet_version + tripped.kind.should == report.kind + tripped.status.should == report.status + tripped.environment.should == report.environment + + logs_as_strings(tripped).should == logs_as_strings(report) + metrics_as_hashes(tripped).should == metrics_as_hashes(report) + expect_equivalent_resource_statuses(tripped.resource_statuses, report.resource_statuses) + end + + def logs_as_strings(report) + report.logs.map(&:to_report) + end + + def metrics_as_hashes(report) + Hash[*report.metrics.collect do |name, m| + [name, { :name => m.name, :label => m.label, :value => m.value }] + end.flatten] + end + + def expect_equivalent_resource_statuses(tripped, report) + tripped.keys.sort.should == report.keys.sort + + tripped.each_pair do |name, status| + expected = report[name] + + status.title.should == expected.title + status.file.should == expected.file + status.line.should == expected.line + status.resource.should == expected.resource + status.resource_type.should == expected.resource_type + status.containment_path.should == expected.containment_path + status.evaluation_time.should == expected.evaluation_time + status.tags.should == expected.tags + status.time.to_i.should == expected.time.to_i + status.failed.should == expected.failed + status.changed.should == expected.changed + status.out_of_sync.should == expected.out_of_sync + status.skipped.should == expected.skipped + status.change_count.should == expected.change_count + status.out_of_sync_count.should == expected.out_of_sync_count + status.events.should == expected.events + end + end + + def generate_report + status = Puppet::Resource::Status.new(Puppet::Type.type(:notify).new(:title => "a resource")) + status.changed = true + + report = Puppet::Transaction::Report.new('testy', 1357986, 'test_environment', "df34516e-4050-402d-a166-05b03b940749") + report << Puppet::Util::Log.new(:level => :warning, :message => "log message") + report.add_times("timing", 4) + report.add_resource_status(status) + report.finalize_report + report + end + end diff --git a/spec/unit/transaction/resource_harness_spec.rb b/spec/unit/transaction/resource_harness_spec.rb index 7d80c9d94..812b76438 100755 --- a/spec/unit/transaction/resource_harness_spec.rb +++ b/spec/unit/transaction/resource_harness_spec.rb @@ -11,13 +11,11 @@ describe Puppet::Transaction::ResourceHarness do @mode_755 = Puppet.features.microsoft_windows? ? '644' : '755' path = make_absolute("/my/file") - @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new) + @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new, nil, nil) @resource = Puppet::Type.type(:file).new :path => path @harness = Puppet::Transaction::ResourceHarness.new(@transaction) @current_state = Puppet::Resource.new(:file, path) @resource.stubs(:retrieve).returns @current_state - @status = Puppet::Resource::Status.new(@resource) - Puppet::Resource::Status.stubs(:new).returns @status end it "should accept a transaction at initialization" do @@ -31,28 +29,38 @@ describe Puppet::Transaction::ResourceHarness do end describe "when evaluating a resource" do - it "should create and return a resource status instance for the resource" do - @harness.evaluate(@resource).should be_instance_of(Puppet::Resource::Status) - end - - it "should fail if no status can be created" do - Puppet::Resource::Status.expects(:new).raises ArgumentError + it "produces a resource state that describes what happened with the resource" do + status = @harness.evaluate(@resource) - lambda { @harness.evaluate(@resource) }.should raise_error + status.resource.should == @resource.ref + status.should_not be_failed + status.events.should be_empty end - it "should retrieve the current state of the resource" do + it "retrieves the current state of the resource" do @resource.expects(:retrieve).returns @current_state + @harness.evaluate(@resource) end - it "should mark the resource as failed and return if the current state cannot be retrieved" do - @resource.expects(:retrieve).raises ArgumentError - @harness.evaluate(@resource).should be_failed + it "produces a failure status for the resource when an error occurs" do + the_message = "retrieve failed in testing" + @resource.expects(:retrieve).raises(ArgumentError.new(the_message)) + + status = @harness.evaluate(@resource) + + status.should be_failed + events_to_hash(status.events).collect do |event| + { :@status => event[:@status], :@message => event[:@message] } + end.should == [{ :@status => "failure", :@message => the_message }] end - it "should store the resource's evaluation time in the resource status" do - @harness.evaluate(@resource).evaluation_time.should be_instance_of(Float) + it "records the time it took to evaluate the resource" do + before = Time.now + status = @harness.evaluate(@resource) + after = Time.now + + status.evaluation_time.should be <= after - before end end @@ -470,17 +478,16 @@ describe Puppet::Transaction::ResourceHarness do before do @catalog = Puppet::Resource::Catalog.new @resource.catalog = @catalog - @status = Puppet::Resource::Status.new(@resource) end it "should return true if 'ignoreschedules' is set" do Puppet[:ignoreschedules] = true @resource[:schedule] = "meh" - @harness.should be_scheduled(@status, @resource) + @harness.should be_scheduled(@resource) end it "should return true if the resource has no schedule set" do - @harness.should be_scheduled(@status, @resource) + @harness.should be_scheduled(@resource) end it "should return the result of matching the schedule with the cached 'checked' time if a schedule is set" do @@ -493,7 +500,7 @@ describe Puppet::Transaction::ResourceHarness do sched.expects(:match?).with(t.to_i).returns "feh" - @harness.scheduled?(@status, @resource).should == "feh" + @harness.scheduled?(@resource).should == "feh" end end diff --git a/spec/unit/transaction_spec.rb b/spec/unit/transaction_spec.rb index 0a276b38e..840b34c15 100755 --- a/spec/unit/transaction_spec.rb +++ b/spec/unit/transaction_spec.rb @@ -1,50 +1,37 @@ #! /usr/bin/env ruby require 'spec_helper' +require 'matchers/include_in_order' +require 'puppet_spec/compiler' require 'puppet/transaction' require 'fileutils' -def without_warnings - flag = $VERBOSE - $VERBOSE = nil - yield - $VERBOSE = flag -end - describe Puppet::Transaction do include PuppetSpec::Files + include PuppetSpec::Compiler - before do - @basepath = make_absolute("/what/ever") - @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new) + def catalog_with_resource(resource) + catalog = Puppet::Resource::Catalog.new + catalog.add_resource(resource) + catalog end - it "should delegate its event list to the event manager" do - @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new) - @transaction.event_manager.expects(:events).returns %w{my events} - @transaction.events.should == %w{my events} - end - - it "should delegate adding times to its report" do - @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new) - @transaction.report.expects(:add_times).with(:foo, 10) - @transaction.report.expects(:add_times).with(:bar, 20) - - @transaction.add_times :foo => 10, :bar => 20 + def transaction_with_resource(resource) + transaction = Puppet::Transaction.new(catalog_with_resource(resource), nil, Puppet::Graph::RandomPrioritizer.new) + transaction end - it "should be able to accept resource status instances" do - resource = Puppet::Type.type(:notify).new :title => "foobar" - status = Puppet::Resource::Status.new(resource) - @transaction.add_resource_status(status) - @transaction.resource_status(resource).should equal(status) + before do + @basepath = make_absolute("/what/ever") + @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new, nil, Puppet::Graph::RandomPrioritizer.new) end it "should be able to look resource status up by resource reference" do resource = Puppet::Type.type(:notify).new :title => "foobar" - status = Puppet::Resource::Status.new(resource) - @transaction.add_resource_status(status) - @transaction.resource_status(resource.to_s).should equal(status) + transaction = transaction_with_resource(resource) + transaction.evaluate + + transaction.resource_status(resource.to_s).should be_changed end # This will basically only ever be used during testing. @@ -55,86 +42,72 @@ describe Puppet::Transaction do it "should add provided resource statuses to its report" do resource = Puppet::Type.type(:notify).new :title => "foobar" - status = Puppet::Resource::Status.new(resource) - @transaction.add_resource_status(status) - @transaction.report.resource_statuses[resource.to_s].should equal(status) - end - - it "should consider a resource to be failed if a status instance exists for that resource and indicates it is failed" do - resource = Puppet::Type.type(:notify).new :name => "yayness" - status = Puppet::Resource::Status.new(resource) - status.failed = "some message" - @transaction.add_resource_status(status) - @transaction.should be_failed(resource) - end - - it "should not consider a resource to be failed if a status instance exists for that resource but indicates it is not failed" do - resource = Puppet::Type.type(:notify).new :name => "yayness" - status = Puppet::Resource::Status.new(resource) - @transaction.add_resource_status(status) - @transaction.should_not be_failed(resource) - end + transaction = transaction_with_resource(resource) + transaction.evaluate - it "should consider there to be failed resources if any statuses are marked failed" do - resource = Puppet::Type.type(:notify).new :name => "yayness" - status = Puppet::Resource::Status.new(resource) - status.failed = "some message" - @transaction.add_resource_status(status) - @transaction.should be_any_failed + status = transaction.resource_status(resource) + transaction.report.resource_statuses[resource.to_s].should equal(status) end it "should not consider there to be failed resources if no statuses are marked failed" do - resource = Puppet::Type.type(:notify).new :name => "yayness" - status = Puppet::Resource::Status.new(resource) - @transaction.add_resource_status(status) - @transaction.should_not be_any_failed + resource = Puppet::Type.type(:notify).new :title => "foobar" + transaction = transaction_with_resource(resource) + transaction.evaluate + + transaction.should_not be_any_failed end it "should use the provided report object" do report = Puppet::Transaction::Report.new("apply") - @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new, report) + transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new, report, nil) - @transaction.report.should == report + transaction.report.should == report end it "should create a report if none is provided" do - @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new) + transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new, nil, nil) - @transaction.report.should be_kind_of Puppet::Transaction::Report + transaction.report.should be_kind_of Puppet::Transaction::Report end describe "when initializing" do it "should create an event manager" do - @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new) + @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new, nil, nil) @transaction.event_manager.should be_instance_of(Puppet::Transaction::EventManager) @transaction.event_manager.transaction.should equal(@transaction) end it "should create a resource harness" do - @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new) + @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new, nil, nil) @transaction.resource_harness.should be_instance_of(Puppet::Transaction::ResourceHarness) @transaction.resource_harness.transaction.should equal(@transaction) end + + it "should set retrieval time on the report" do + catalog = Puppet::Resource::Catalog.new + report = Puppet::Transaction::Report.new("apply") + catalog.retrieval_duration = 5 + + report.expects(:add_times).with(:config_retrieval, 5) + + transaction = Puppet::Transaction.new(catalog, report, nil) + end end describe "when evaluating a resource" do before do - @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new) - @transaction.stubs(:skip?).returns false - + @catalog = Puppet::Resource::Catalog.new @resource = Puppet::Type.type(:file).new :path => @basepath - end + @catalog.add_resource(@resource) - it "should check whether the resource should be skipped" do - @transaction.expects(:skip?).with(@resource).returns false - - @transaction.eval_resource(@resource) + @transaction = Puppet::Transaction.new(@catalog, nil, Puppet::Graph::RandomPrioritizer.new) + @transaction.stubs(:skip?).returns false end it "should process events" do @transaction.event_manager.expects(:process_events).with(@resource) - @transaction.eval_resource(@resource) + @transaction.evaluate end describe "and the resource should be skipped" do @@ -143,7 +116,7 @@ describe Puppet::Transaction do end it "should mark the resource's status as skipped" do - @transaction.eval_resource(@resource) + @transaction.evaluate @transaction.resource_status(@resource).should be_skipped end end @@ -151,149 +124,44 @@ describe Puppet::Transaction do describe "when applying a resource" do before do + @catalog = Puppet::Resource::Catalog.new @resource = Puppet::Type.type(:file).new :path => @basepath + @catalog.add_resource(@resource) @status = Puppet::Resource::Status.new(@resource) - @transaction = Puppet::Transaction.new(Puppet::Resource::Catalog.new) + @transaction = Puppet::Transaction.new(@catalog, nil, Puppet::Graph::RandomPrioritizer.new) @transaction.event_manager.stubs(:queue_events) - @transaction.resource_harness.stubs(:evaluate).returns(@status) end it "should use its resource harness to apply the resource" do @transaction.resource_harness.expects(:evaluate).with(@resource) - @transaction.apply(@resource) + @transaction.evaluate end it "should add the resulting resource status to its status list" do - @transaction.apply(@resource) + @transaction.resource_harness.stubs(:evaluate).returns(@status) + @transaction.evaluate @transaction.resource_status(@resource).should be_instance_of(Puppet::Resource::Status) end it "should queue any events added to the resource status" do + @transaction.resource_harness.stubs(:evaluate).returns(@status) @status.expects(:events).returns %w{a b} @transaction.event_manager.expects(:queue_events).with(@resource, ["a", "b"]) - @transaction.apply(@resource) + @transaction.evaluate end it "should log and skip any resources that cannot be applied" do - @transaction.resource_harness.expects(:evaluate).raises ArgumentError - @resource.expects(:err) - @transaction.apply(@resource) - @transaction.report.resource_statuses[@resource.to_s].should be_nil + @resource.expects(:properties).raises ArgumentError + @transaction.evaluate + @transaction.report.resource_statuses[@resource.to_s].should be_failed end - end - describe "#eval_generate" do - let(:path) { tmpdir('eval_generate') } - let(:resource) { Puppet::Type.type(:file).new(:path => path, :recurse => true) } - let(:graph) { @transaction.relationship_graph } + it "should report any_failed if any resources failed" do + @resource.expects(:properties).raises ArgumentError + @transaction.evaluate - def find_vertex(type, title) - graph.vertices.find {|v| v.type == type and v.title == title} - end - - before :each do - @filenames = [] - - 'a'.upto('c') do |x| - @filenames << File.join(path,x) - - 'a'.upto('c') do |y| - @filenames << File.join(path,x,y) - FileUtils.mkdir_p(File.join(path,x,y)) - - 'a'.upto('c') do |z| - @filenames << File.join(path,x,y,z) - FileUtils.touch(File.join(path,x,y,z)) - end - end - end - - @transaction.catalog.add_resource(resource) - end - - it "should add the generated resources to the catalog" do - @transaction.eval_generate(resource) - - @filenames.each do |file| - @transaction.catalog.resource(:file, file).must be_a(Puppet::Type.type(:file)) - end - end - - it "should add a sentinel whit for the resource" do - @transaction.eval_generate(resource) - - find_vertex(:whit, "completed_#{path}").must be_a(Puppet::Type.type(:whit)) - end - - it "should replace dependencies on the resource with dependencies on the sentinel" do - dependent = Puppet::Type.type(:notify).new(:name => "hello", :require => resource) - - @transaction.catalog.add_resource(dependent) - - res = find_vertex(resource.type, resource.title) - generated = find_vertex(dependent.type, dependent.title) - - graph.should be_edge(res, generated) - - @transaction.eval_generate(resource) - - sentinel = find_vertex(:whit, "completed_#{path}") - - graph.should be_edge(sentinel, generated) - graph.should_not be_edge(res, generated) - end - - it "should add an edge from the nearest ancestor to the generated resource" do - @transaction.eval_generate(resource) - - @filenames.each do |file| - v = find_vertex(:file, file) - p = find_vertex(:file, File.dirname(file)) - - graph.should be_edge(p, v) - end - end - - it "should add an edge from each generated resource to the sentinel" do - @transaction.eval_generate(resource) - - sentinel = find_vertex(:whit, "completed_#{path}") - @filenames.each do |file| - v = find_vertex(:file, file) - - graph.should be_edge(v, sentinel) - end - end - - it "should add an edge from the resource to the sentinel" do - @transaction.eval_generate(resource) - - res = find_vertex(:file, path) - sentinel = find_vertex(:whit, "completed_#{path}") - - graph.should be_edge(res, sentinel) - end - - it "should return false if an error occured when generating resources" do - resource.stubs(:eval_generate).raises(Puppet::Error) - - @transaction.eval_generate(resource).should == false - end - - it "should return true if resources were generated" do - @transaction.eval_generate(resource).should == true - end - - it "should not add a sentinel if no resources are generated" do - path2 = tmpfile('empty') - other_file = Puppet::Type.type(:file).new(:path => path2) - - @transaction.catalog.add_resource(other_file) - - @transaction.eval_generate(other_file).should == false - - find_vertex(:whit, "completed_#{path2}").should be_nil + expect(@transaction).to be_any_failed end end @@ -343,41 +211,7 @@ describe Puppet::Transaction do end end - describe "#finish" do - let(:graph) { @transaction.relationship_graph } - let(:path) { tmpdir('eval_generate') } - let(:resource) { Puppet::Type.type(:file).new(:path => path, :recurse => true) } - - before :each do - @transaction.catalog.add_resource(resource) - end - - it "should unblock the resource's dependents and queue them if ready" do - dependent = Puppet::Type.type(:file).new(:path => tmpfile('dependent'), :require => resource) - more_dependent = Puppet::Type.type(:file).new(:path => tmpfile('more_dependent'), :require => [resource, dependent]) - @transaction.catalog.add_resource(dependent, more_dependent) - - graph.finish(resource) - - graph.blockers[dependent].should == 0 - graph.blockers[more_dependent].should == 1 - - key = graph.unguessable_deterministic_key[dependent] - - graph.ready[key].must == dependent - - graph.ready.should_not be_has_key(graph.unguessable_deterministic_key[more_dependent]) - end - - it "should mark the resource as done" do - graph.finish(resource) - - graph.done[resource].should == true - end - end - describe "when traversing" do - let(:graph) { @transaction.relationship_graph } let(:path) { tmpdir('eval_generate') } let(:resource) { Puppet::Type.type(:file).new(:path => path, :recurse => true) } @@ -385,26 +219,11 @@ describe Puppet::Transaction do @transaction.catalog.add_resource(resource) end - it "should clear blockers if resources are added" do - graph.blockers['foo'] = 3 - graph.blockers['bar'] = 4 - - graph.ready[graph.unguessable_deterministic_key[resource]] = resource - - @transaction.expects(:eval_generate).with(resource).returns true - - graph.traverse {} - - graph.blockers.should be_empty - end - it "should yield the resource even if eval_generate is called" do - graph.ready[graph.unguessable_deterministic_key[resource]] = resource - - @transaction.expects(:eval_generate).with(resource).returns true + Puppet::Transaction::AdditionalResourceGenerator.any_instance.expects(:eval_generate).with(resource).returns true yielded = false - graph.traverse do |res| + @transaction.evaluate do |res| yielded = true if res == resource end @@ -414,89 +233,31 @@ describe Puppet::Transaction do it "should prefetch the provider if necessary" do @transaction.expects(:prefetch_if_necessary).with(resource) - graph.traverse {} - end - - it "should not clear blockers if resources aren't added" do - graph.blockers['foo'] = 3 - graph.blockers['bar'] = 4 - - graph.ready[graph.unguessable_deterministic_key[resource]] = resource - - @transaction.expects(:eval_generate).with(resource).returns false - - graph.traverse {} - - graph.blockers.should == {'foo' => 3, 'bar' => 4, resource => 0} - end - - it "should unblock all dependents of the resource" do - dependent = Puppet::Type.type(:notify).new(:name => "hello", :require => resource) - dependent2 = Puppet::Type.type(:notify).new(:name => "goodbye", :require => resource) - - @transaction.catalog.add_resource(dependent, dependent2) - - # We enqueue them here just so we can check their blockers. This is done - # again in traverse. - graph.enqueue_roots - - graph.blockers[dependent].should == 1 - graph.blockers[dependent2].should == 1 - - graph.ready[graph.unguessable_deterministic_key[resource]] = resource - - graph.traverse {} - - graph.blockers[dependent].should == 0 - graph.blockers[dependent2].should == 0 + @transaction.evaluate {} end - it "should enqueue any unblocked dependents" do + it "traverses independent resources before dependent resources" do dependent = Puppet::Type.type(:notify).new(:name => "hello", :require => resource) - dependent2 = Puppet::Type.type(:notify).new(:name => "goodbye", :require => resource) - - @transaction.catalog.add_resource(dependent, dependent2) - - graph.enqueue_roots - - graph.blockers[dependent].should == 1 - graph.blockers[dependent2].should == 1 - - graph.ready[graph.unguessable_deterministic_key[resource]] = resource + @transaction.catalog.add_resource(dependent) seen = [] - - graph.traverse do |res| + @transaction.evaluate do |res| seen << res end - seen.should =~ [resource, dependent, dependent2] + expect(seen).to include_in_order(resource, dependent) end - it "should mark the resource done" do - graph.ready[graph.unguessable_deterministic_key[resource]] = resource - - graph.traverse {} + it "traverses completely independent resources in the order they appear in the catalog" do + independent = Puppet::Type.type(:notify).new(:name => "hello", :require => resource) + @transaction.catalog.add_resource(independent) - graph.done[resource].should == true - end - - it "should not evaluate the resource if it's not suitable" do - resource.stubs(:suitable?).returns false - - graph.traverse do |resource| - raise "evaluated a resource" + seen = [] + @transaction.evaluate do |res| + seen << res end - end - - it "should defer an unsuitable resource unless it can't go on" do - other = Puppet::Type.type(:notify).new(:name => "hello") - @transaction.catalog.add_resource(other) - - # Show that we check once, then get the resource back and check again - resource.expects(:suitable?).twice.returns(false).then.returns(true) - graph.traverse {} + expect(seen).to include_in_order(resource, independent) end it "should fail unsuitable resources and go on if it gets blocked" do @@ -506,7 +267,7 @@ describe Puppet::Transaction do resource.stubs(:suitable?).returns false evaluated = [] - graph.traverse do |res| + @transaction.evaluate do |res| evaluated << res end @@ -518,7 +279,7 @@ describe Puppet::Transaction do describe "when generating resources before traversal" do let(:catalog) { Puppet::Resource::Catalog.new } - let(:transaction) { Puppet::Transaction.new(catalog) } + let(:transaction) { Puppet::Transaction.new(catalog, nil, Puppet::Graph::RandomPrioritizer.new) } let(:generator) { Puppet::Type.type(:notify).new :title => "generator" } let(:generated) do %w[a b c].map { |name| Puppet::Type.type(:notify).new(:name => name) } @@ -532,30 +293,19 @@ describe Puppet::Transaction do it "should call 'generate' on all created resources" do generated.each { |res| res.expects(:generate) } - transaction.add_dynamically_generated_resources + transaction.evaluate end it "should finish all resources" do generated.each { |res| res.expects(:finish) } - transaction.add_dynamically_generated_resources - end - - it "should skip generated resources that conflict with existing resources" do - duplicate = generated.first - catalog.add_resource(duplicate) - - duplicate.expects(:finish).never - - duplicate.expects(:info).with { |msg| msg =~ /Duplicate generated resource/ } - - transaction.add_dynamically_generated_resources + transaction.evaluate end it "should copy all tags to the newly generated resources" do generator.tag('one', 'two') - transaction.add_dynamically_generated_resources + transaction.evaluate generated.each do |res| res.must be_tagged(generator.tags) @@ -568,7 +318,7 @@ describe Puppet::Transaction do @resource = Puppet::Type.type(:notify).new :name => "foo" @catalog = Puppet::Resource::Catalog.new @resource.catalog = @catalog - @transaction = Puppet::Transaction.new(@catalog) + @transaction = Puppet::Transaction.new(@catalog, nil, nil) end it "should skip resource with missing tags" do @@ -592,22 +342,39 @@ describe Puppet::Transaction do end it "should skip device only resouce on normal host" do + @resource.stubs(:appliable_to_host?).returns false @resource.stubs(:appliable_to_device?).returns true @transaction.for_network_device = false @transaction.should be_skip(@resource) end it "should not skip device only resouce on remote device" do + @resource.stubs(:appliable_to_host?).returns false @resource.stubs(:appliable_to_device?).returns true @transaction.for_network_device = true @transaction.should_not be_skip(@resource) end it "should skip host resouce on device" do + @resource.stubs(:appliable_to_host?).returns true @resource.stubs(:appliable_to_device?).returns false @transaction.for_network_device = true @transaction.should be_skip(@resource) end + + it "should not skip resouce available on both device and host when on device" do + @resource.stubs(:appliable_to_host?).returns true + @resource.stubs(:appliable_to_device?).returns true + @transaction.for_network_device = true + @transaction.should_not be_skip(@resource) + end + + it "should not skip resouce available on both device and host when on host" do + @resource.stubs(:appliable_to_host?).returns true + @resource.stubs(:appliable_to_device?).returns true + @transaction.for_network_device = false + @transaction.should_not be_skip(@resource) + end end describe "when determining if tags are missing" do @@ -615,7 +382,7 @@ describe Puppet::Transaction do @resource = Puppet::Type.type(:notify).new :name => "foo" @catalog = Puppet::Resource::Catalog.new @resource.catalog = @catalog - @transaction = Puppet::Transaction.new(@catalog) + @transaction = Puppet::Transaction.new(@catalog, nil, nil) @transaction.stubs(:ignore_tags?).returns false end @@ -646,27 +413,28 @@ describe Puppet::Transaction do before :each do @resource = Puppet::Type.type(:notify).new :name => "foo" @catalog = Puppet::Resource::Catalog.new - @resource.catalog = @catalog - @transaction = Puppet::Transaction.new(@catalog) + @catalog.add_resource(@resource) + @transaction = Puppet::Transaction.new(@catalog, nil, Puppet::Graph::RandomPrioritizer.new) end it "should always schedule resources if 'ignoreschedules' is set" do @transaction.ignoreschedules = true @transaction.resource_harness.expects(:scheduled?).never - @transaction.should be_scheduled(@resource) + @transaction.evaluate + @transaction.resource_status(@resource).should be_changed end it "should let the resource harness determine whether the resource should be scheduled" do - @transaction.resource_harness.expects(:scheduled?).with(@transaction.resource_status(@resource), @resource).returns "feh" + @transaction.resource_harness.expects(:scheduled?).with(@resource).returns "feh" - @transaction.scheduled?(@resource).should == "feh" + @transaction.evaluate end end describe "when prefetching" do let(:catalog) { Puppet::Resource::Catalog.new } - let(:transaction) { Puppet::Transaction.new(catalog) } + let(:transaction) { Puppet::Transaction.new(catalog, nil, nil) } let(:resource) { Puppet::Type.type(:sshkey).new :title => "foo", :name => "bar", :type => :dsa, :key => "eh", :provider => :parsed } let(:resource2) { Puppet::Type.type(:package).new :title => "blah", :provider => "apt" } @@ -675,30 +443,6 @@ describe Puppet::Transaction do catalog.add_resource resource2 end - - describe "#resources_by_provider" do - it "should fetch resources by their type and provider" do - transaction.resources_by_provider(:sshkey, :parsed).should == { - resource.name => resource, - } - - transaction.resources_by_provider(:package, :apt).should == { - resource2.name => resource2, - } - end - - it "should omit resources whose types don't use providers" do - # faking the sshkey type not to have a provider - resource.class.stubs(:attrclass).returns nil - - transaction.resources_by_provider(:sshkey, :parsed).should == {} - end - - it "should return empty hash for providers with no resources" do - transaction.resources_by_provider(:package, :yum).should == {} - end - end - it "should match resources by name, not title" do resource.provider.class.expects(:prefetch).with("bar" => resource) @@ -734,37 +478,25 @@ describe Puppet::Transaction do end end - it "should return all resources for which the resource status indicates the resource has changed when determinig changed resources" do - @catalog = Puppet::Resource::Catalog.new - @transaction = Puppet::Transaction.new(@catalog) - names = [] - 2.times do |i| - name = File.join(@basepath, "file#{i}") - resource = Puppet::Type.type(:file).new :path => name - names << resource.to_s - @catalog.add_resource resource - @transaction.add_resource_status Puppet::Resource::Status.new(resource) - end - - @transaction.resource_status(names[0]).changed = true - - @transaction.changed?.should == [@catalog.resource(names[0])] - end - describe 'when checking application run state' do before do - without_warnings { Puppet::Application = Class.new(Puppet::Application) } @catalog = Puppet::Resource::Catalog.new - @transaction = Puppet::Transaction.new(@catalog) + @transaction = Puppet::Transaction.new(@catalog, nil, Puppet::Graph::RandomPrioritizer.new) end - after do - without_warnings { Puppet::Application = Puppet::Application.superclass } - end + context "when stop is requested" do + before :each do + Puppet::Application.stubs(:stop_requested?).returns(true) + end - it 'should return true for :stop_processing? if Puppet::Application.stop_requested? is true' do - Puppet::Application.stubs(:stop_requested?).returns(true) - @transaction.stop_processing?.should be_true + it 'should return true for :stop_processing?' do + @transaction.should be_stop_processing + end + + it 'always evaluates non-host_config catalogs' do + @catalog.host_config = false + @transaction.should_not be_stop_processing + end end it 'should return false for :stop_processing? if Puppet::Application.stop_requested? is false' do @@ -792,12 +524,72 @@ describe Puppet::Transaction do end end end + + it "errors with a dependency cycle for a resource that requires itself" do + expect do + apply_compiled_manifest(<<-MANIFEST) + notify { cycle: require => Notify[cycle] } + MANIFEST + end.to raise_error(Puppet::Error, /Found 1 dependency cycle:.*\(Notify\[cycle\] => Notify\[cycle\]\)/m) + end + + it "errors with a dependency cycle for a self-requiring resource also required by another resource" do + expect do + apply_compiled_manifest(<<-MANIFEST) + notify { cycle: require => Notify[cycle] } + notify { other: require => Notify[cycle] } + MANIFEST + end.to raise_error(Puppet::Error, /Found 1 dependency cycle:.*\(Notify\[cycle\] => Notify\[cycle\]\)/m) + end + + it "errors with a dependency cycle for a resource that requires itself and another resource" do + expect do + apply_compiled_manifest(<<-MANIFEST) + notify { cycle: + require => [Notify[other], Notify[cycle]] + } + notify { other: } + MANIFEST + end.to raise_error(Puppet::Error, /Found 1 dependency cycle:.*\(Notify\[cycle\] => Notify\[cycle\]\)/m) + end + + it "errors with a dependency cycle for a resource that is later modified to require itself" do + expect do + apply_compiled_manifest(<<-MANIFEST) + notify { cycle: } + Notify <| title == 'cycle' |> { + require => Notify[cycle] + } + MANIFEST + end.to raise_error(Puppet::Error, /Found 1 dependency cycle:.*\(Notify\[cycle\] => Notify\[cycle\]\)/m) + end + + it "reports a changed resource with a successful run" do + transaction = apply_compiled_manifest("notify { one: }") + + transaction.report.status.should == 'changed' + transaction.report.resource_statuses['Notify[one]'].should be_changed + end + + describe "when interrupted" do + it "marks unprocessed resources as skipped" do + Puppet::Application.stop! + + transaction = apply_compiled_manifest(<<-MANIFEST) + notify { a: } -> + notify { b: } + MANIFEST + + transaction.report.resource_statuses['Notify[a]'].should be_skipped + transaction.report.resource_statuses['Notify[b]'].should be_skipped + end + end end describe Puppet::Transaction, " when determining tags" do before do @config = Puppet::Resource::Catalog.new - @transaction = Puppet::Transaction.new(@config) + @transaction = Puppet::Transaction.new(@config, nil, nil) end it "should default to the tags specified in the :tags setting" do diff --git a/spec/unit/type/augeas_spec.rb b/spec/unit/type/augeas_spec.rb index 77e3ae93c..130a7e59a 100755 --- a/spec/unit/type/augeas_spec.rb +++ b/spec/unit/type/augeas_spec.rb @@ -15,17 +15,17 @@ describe augeas do end describe "basic structure" do - it "should be able to create a instance" do + it "should be able to create an instance" do provider_class = Puppet::Type::Augeas.provider(Puppet::Type::Augeas.providers[0]) Puppet::Type::Augeas.expects(:defaultprovider).returns provider_class augeas.new(:name => "bar").should_not be_nil end - it "should have an parse_commands feature" do + it "should have a parse_commands feature" do augeas.provider_feature(:parse_commands).should_not be_nil end - it "should have an need_to_run? feature" do + it "should have a need_to_run? feature" do augeas.provider_feature(:need_to_run?).should_not be_nil end diff --git a/spec/unit/type/component_spec.rb b/spec/unit/type/component_spec.rb index 4b213fc18..82c822dea 100755 --- a/spec/unit/type/component_spec.rb +++ b/spec/unit/type/component_spec.rb @@ -21,15 +21,6 @@ describe component do comp.title.should == comp.ref end - it "should alias itself to its reference if it has a catalog and the catalog does not already have a resource with the same reference" do - catalog = mock 'catalog' - catalog.expects(:resource).with("Foo[bar]").returns nil - - catalog.expects(:alias).with { |resource, name| resource.is_a?(component) and name == "Foo[bar]" } - - component.new(:name => "Foo[bar]", :catalog => catalog) - end - it "should not fail when provided an invalid value" do comp = component.new(:name => "Foo[bar]") lambda { comp[:yayness] = "ey" }.should_not raise_error diff --git a/spec/unit/type/computer_spec.rb b/spec/unit/type/computer_spec.rb index 773e1fc97..dd01e47bd 100755 --- a/spec/unit/type/computer_spec.rb +++ b/spec/unit/type/computer_spec.rb @@ -18,7 +18,7 @@ describe Puppet::Type.type(:computer), " when checking computer objects" do @ensure = Puppet::Type::Computer.attrclass(:ensure).new(:resource => @resource) end - it "should be able to create a instance" do + it "should be able to create an instance" do provider_class = Puppet::Type::Computer.provider(Puppet::Type::Computer.providers[0]) Puppet::Type::Computer.expects(:defaultprovider).returns provider_class computer.new(:name => "bar").should_not be_nil diff --git a/spec/unit/type/cron_spec.rb b/spec/unit/type/cron_spec.rb index 49e86dad6..3d08ba203 100755 --- a/spec/unit/type/cron_spec.rb +++ b/spec/unit/type/cron_spec.rb @@ -457,13 +457,13 @@ describe Puppet::Type.type(:cron), :unless => Puppet.features.microsoft_windows? it "should accept empty environment variables that do not contain '='" do expect do described_class.new(:name => 'foo',:environment => 'MAILTO=') - end.to_not raise_error(Puppet::Error) + end.to_not raise_error end it "should accept 'absent'" do expect do described_class.new(:name => 'foo',:environment => 'absent') - end.to_not raise_error(Puppet::Error) + end.to_not raise_error end end diff --git a/spec/unit/type/exec_spec.rb b/spec/unit/type/exec_spec.rb index 06bc43347..85a5809dc 100755 --- a/spec/unit/type/exec_spec.rb +++ b/spec/unit/type/exec_spec.rb @@ -154,11 +154,13 @@ describe Puppet::Type.type(:exec) do foo = make_absolute('/bin/foo') catalog = Puppet::Resource::Catalog.new tmp = Puppet::Type.type(:file).new(:name => foo) - catalog.add_resource tmp execer = Puppet::Type.type(:exec).new(:name => foo) + + catalog.add_resource tmp catalog.add_resource execer + dependencies = execer.autorequire(catalog) - catalog.relationship_graph.dependencies(execer).should == [tmp] + dependencies.collect(&:to_s).should == [Puppet::Relationship.new(tmp, execer).to_s] end describe "when handling the path parameter" do diff --git a/spec/unit/type/file/content_spec.rb b/spec/unit/type/file/content_spec.rb index 0dfcf4e2f..c1c97a452 100755 --- a/spec/unit/type/file/content_spec.rb +++ b/spec/unit/type/file/content_spec.rb @@ -147,6 +147,17 @@ describe content do @content.must be_safe_insync("whatever") end + it "should warn that no content will be synced to links when ensure is :present" do + @resource[:ensure] = :present + @resource[:content] = 'foo' + @resource.stubs(:should_be_file?).returns false + @resource.stubs(:stat).returns mock("stat", :ftype => "link") + + @resource.expects(:warning).with {|msg| msg =~ /Ensure set to :present but file type is/} + + @content.insync? :present + end + it "should return false if the current content is :absent" do @content.should = "foo" @content.should_not be_safe_insync(:absent) diff --git a/spec/unit/type/file/group_spec.rb b/spec/unit/type/file/group_spec.rb index f81d09241..c2a38d07b 100755 --- a/spec/unit/type/file/group_spec.rb +++ b/spec/unit/type/file/group_spec.rb @@ -23,7 +23,7 @@ describe Puppet::Type.type(:file).attrclass(:group) do resource.provider.stubs(:name2gid).with('bars').returns 1002 end - it "should fail if an group's id can't be found by name" do + it "should fail if a group's id can't be found by name" do resource.provider.stubs(:name2gid).returns nil expect { group.insync?(5) }.to raise_error(/Could not find group foos/) diff --git a/spec/unit/type/file_spec.rb b/spec/unit/type/file_spec.rb index b4bd68566..dd695eeda 100755 --- a/spec/unit/type/file_spec.rb +++ b/spec/unit/type/file_spec.rb @@ -26,6 +26,11 @@ describe Puppet::Type.type(:file) do file[:path].should == "/foo/bar/baz" end + it "should remove triple slashes" do + file[:path] = "/foo/bar///baz" + file[:path].should == "/foo/bar/baz" + end + it "should remove trailing double slashes" do file[:path] = "/foo/bar/baz//" file[:path].should == "/foo/bar/baz" @@ -36,11 +41,14 @@ describe Puppet::Type.type(:file) do file[:path].should == "/" end - it "should accept and preserve a double-slash at the start of the path" do - expect { - file[:path] = "//tmp/xxx" - file[:path].should == '//tmp/xxx' - }.to_not raise_error + it "should accept and collapse a double-slash at the start of the path" do + file[:path] = "//tmp/xxx" + file[:path].should == '/tmp/xxx' + end + + it "should accept and collapse a triple-slash at the start of the path" do + file[:path] = "///tmp/xxx" + file[:path].should == '/tmp/xxx' end end @@ -176,14 +184,14 @@ describe Puppet::Type.type(:file) do [true, :true, :yes].each do |value| it "should consider #{value} to be true" do file[:replace] = value - file[:replace].should == :true + file[:replace].should be_true end end [false, :false, :no].each do |value| it "should consider #{value} to be false" do file[:replace] = value - file[:replace].should == :false + file[:replace].should be_false end end end @@ -1116,7 +1124,7 @@ describe Puppet::Type.type(:file) do property = stub('content_property', :actual_content => "something", :length => "something".length, :write => 'checksum_a') file.stubs(:property).with(:content).returns(property) - expect { file.write :NOTUSED }.to_not raise_error(Puppet::Error) + expect { file.write :NOTUSED }.to_not raise_error end end end diff --git a/spec/unit/type/mount_spec.rb b/spec/unit/type/mount_spec.rb index 23fd95dec..726577891 100755 --- a/spec/unit/type/mount_spec.rb +++ b/spec/unit/type/mount_spec.rb @@ -2,336 +2,538 @@ require 'spec_helper' describe Puppet::Type.type(:mount), :unless => Puppet.features.microsoft_windows? do + + before :each do + Puppet::Type.type(:mount).stubs(:defaultprovider).returns providerclass + end + + let :providerclass do + described_class.provide(:fake_mount_provider) do + attr_accessor :property_hash + def create; end + def destroy; end + def exists? + get(:ensure) != :absent + end + def mount; end + def umount; end + def mounted? + [:mounted, :ghost].include?(get(:ensure)) + end + mk_resource_methods + end + end + + let :provider do + providerclass.new(:name => 'yay') + end + + let :resource do + described_class.new(:name => "yay", :audit => :ensure, :provider => provider) + end + + let :ensureprop do + resource.property(:ensure) + end + it "should have a :refreshable feature that requires the :remount method" do - Puppet::Type.type(:mount).provider_feature(:refreshable).methods.should == [:remount] + described_class.provider_feature(:refreshable).methods.should == [:remount] end it "should have no default value for :ensure" do - mount = Puppet::Type.type(:mount).new(:name => "yay") + mount = described_class.new(:name => "yay") mount.should(:ensure).should be_nil end it "should have :name as the only keyattribut" do - Puppet::Type.type(:mount).key_attributes.should == [:name] + described_class.key_attributes.should == [:name] end -end -describe Puppet::Type.type(:mount), "when validating attributes" do - [:name, :remounts, :provider].each do |param| - it "should have a #{param} parameter" do - Puppet::Type.type(:mount).attrtype(param).should == :param + describe "when validating attributes" do + [:name, :remounts, :provider].each do |param| + it "should have a #{param} parameter" do + described_class.attrtype(param).should == :param + end end - end - [:ensure, :device, :blockdevice, :fstype, :options, :pass, :dump, :atboot, :target].each do |param| - it "should have a #{param} property" do - Puppet::Type.type(:mount).attrtype(param).should == :property + [:ensure, :device, :blockdevice, :fstype, :options, :pass, :dump, :atboot, :target].each do |param| + it "should have a #{param} property" do + described_class.attrtype(param).should == :property + end end end -end -describe Puppet::Type.type(:mount)::Ensure, "when validating values", :unless => Puppet.features.microsoft_windows? do - before do - @provider = stub 'provider', :class => Puppet::Type.type(:mount).defaultprovider, :clear => nil - Puppet::Type.type(:mount).defaultprovider.expects(:new).returns(@provider) - end + describe "when validating values" do - it "should alias :present to :defined as a value to :ensure" do - mount = Puppet::Type.type(:mount).new(:name => "yay", :ensure => :present) - mount.should(:ensure).should == :defined - end + describe "for name" do + it "should allow full qualified paths" do + described_class.new(:name => "/mnt/foo")[:name].should == '/mnt/foo' + end - it "should support :present as a value to :ensure" do - Puppet::Type.type(:mount).new(:name => "yay", :ensure => :present) - end + it "should remove trailing slashes" do + described_class.new(:name => '/')[:name].should == '/' + described_class.new(:name => '//')[:name].should == '/' + described_class.new(:name => '/foo/')[:name].should == '/foo' + described_class.new(:name => '/foo/bar/')[:name].should == '/foo/bar' + described_class.new(:name => '/foo/bar/baz//')[:name].should == '/foo/bar/baz' + end - it "should support :defined as a value to :ensure" do - Puppet::Type.type(:mount).new(:name => "yay", :ensure => :defined) - end + it "should not allow spaces" do + expect { described_class.new(:name => "/mnt/foo bar") }.to raise_error Puppet::Error, /name.*whitespace/ + end - it "should support :unmounted as a value to :ensure" do - Puppet::Type.type(:mount).new(:name => "yay", :ensure => :unmounted) - end + it "should allow pseudo mountpoints (e.g. swap)" do + described_class.new(:name => 'none')[:name].should == 'none' + end + end - it "should support :absent as a value to :ensure" do - Puppet::Type.type(:mount).new(:name => "yay", :ensure => :absent) - end + describe "for ensure" do + it "should alias :present to :defined as a value to :ensure" do + mount = described_class.new(:name => "yay", :ensure => :present) + mount.should(:ensure).should == :defined + end - it "should support :mounted as a value to :ensure" do - Puppet::Type.type(:mount).new(:name => "yay", :ensure => :mounted) - end -end + it "should support :present as a value to :ensure" do + expect { described_class.new(:name => "yay", :ensure => :present) }.to_not raise_error + end -describe Puppet::Type.type(:mount)::Ensure, :unless => Puppet.features.microsoft_windows? do - before :each do - provider_properties = {} - @provider = stub 'provider', :class => Puppet::Type.type(:mount).defaultprovider, :clear => nil, :satisfies? => true, :name => :mock, :property_hash => provider_properties - Puppet::Type.type(:mount).defaultprovider.stubs(:new).returns(@provider) - @mount = Puppet::Type.type(:mount).new(:name => "yay", :audit => :ensure) + it "should support :defined as a value to :ensure" do + expect { described_class.new(:name => "yay", :ensure => :defined) }.to_not raise_error + end - @ensure = @mount.property(:ensure) - end + it "should support :unmounted as a value to :ensure" do + expect { described_class.new(:name => "yay", :ensure => :unmounted) }.to_not raise_error + end + + it "should support :absent as a value to :ensure" do + expect { described_class.new(:name => "yay", :ensure => :absent) }.to_not raise_error + end + + it "should support :mounted as a value to :ensure" do + expect { described_class.new(:name => "yay", :ensure => :mounted) }.to_not raise_error + end + + it "should not support other values for :ensure" do + expect { described_class.new(:name => "yay", :ensure => :mount) }.to raise_error Puppet::Error, /Invalid value/ + end + end + + describe "for device" do + it "should support normal /dev paths for device" do + expect { described_class.new(:name => "/foo", :ensure => :present, :device => '/dev/hda1') }.to_not raise_error + expect { described_class.new(:name => "/foo", :ensure => :present, :device => '/dev/dsk/c0d0s0') }.to_not raise_error + end + + it "should support labels for device" do + expect { described_class.new(:name => "/foo", :ensure => :present, :device => 'LABEL=/boot') }.to_not raise_error + expect { described_class.new(:name => "/foo", :ensure => :present, :device => 'LABEL=SWAP-hda6') }.to_not raise_error + end + + it "should support pseudo devices for device" do + expect { described_class.new(:name => "/foo", :ensure => :present, :device => 'ctfs') }.to_not raise_error + expect { described_class.new(:name => "/foo", :ensure => :present, :device => 'swap') }.to_not raise_error + expect { described_class.new(:name => "/foo", :ensure => :present, :device => 'sysfs') }.to_not raise_error + expect { described_class.new(:name => "/foo", :ensure => :present, :device => 'proc') }.to_not raise_error + end + + it 'should not support whitespace in device' do + expect { described_class.new(:name => "/foo", :ensure => :present, :device => '/dev/my dev/foo') }.to raise_error Puppet::Error, /device.*whitespace/ + expect { described_class.new(:name => "/foo", :ensure => :present, :device => "/dev/my\tdev/foo") }.to raise_error Puppet::Error, /device.*whitespace/ + end + end + + describe "for blockdevice" do + before :each do + # blockdevice is only used on Solaris + Facter.stubs(:value).with(:operatingsystem).returns 'Solaris' + Facter.stubs(:value).with(:osfamily).returns 'Solaris' + end + + it "should support normal /dev/rdsk paths for blockdevice" do + expect { described_class.new(:name => "/foo", :ensure => :present, :blockdevice => '/dev/rdsk/c0d0s0') }.to_not raise_error + end + + it "should support a dash for blockdevice" do + expect { described_class.new(:name => "/foo", :ensure => :present, :blockdevice => '-') }.to_not raise_error + end + + it "should not support whitespace in blockdevice" do + expect { described_class.new(:name => "/foo", :ensure => :present, :blockdevice => '/dev/my dev/foo') }.to raise_error Puppet::Error, /blockdevice.*whitespace/ + expect { described_class.new(:name => "/foo", :ensure => :present, :blockdevice => "/dev/my\tdev/foo") }.to raise_error Puppet::Error, /blockdevice.*whitespace/ + end + + it "should default to /dev/rdsk/DEVICE if device is /dev/dsk/DEVICE" do + obj = described_class.new(:name => "/foo", :device => '/dev/dsk/c0d0s0') + obj[:blockdevice].should == '/dev/rdsk/c0d0s0' + end + + it "should default to - if it is an nfs-share" do + obj = described_class.new(:name => "/foo", :device => "server://share", :fstype => 'nfs') + obj[:blockdevice].should == '-' + end + + it "should have no default otherwise" do + described_class.new(:name => "/foo")[:blockdevice].should == nil + described_class.new(:name => "/foo", :device => "/foo")[:blockdevice].should == nil + end + + it "should overwrite any default if blockdevice is explicitly set" do + described_class.new(:name => "/foo", :device => '/dev/dsk/c0d0s0', :blockdevice => '/foo')[:blockdevice].should == '/foo' + described_class.new(:name => "/foo", :device => "server://share", :fstype => 'nfs', :blockdevice => '/foo')[:blockdevice].should == '/foo' + end + end + + describe "for fstype" do + it "should support valid fstypes" do + expect { described_class.new(:name => "/foo", :ensure => :present, :fstype => 'ext3') }.to_not raise_error + expect { described_class.new(:name => "/foo", :ensure => :present, :fstype => 'proc') }.to_not raise_error + expect { described_class.new(:name => "/foo", :ensure => :present, :fstype => 'sysfs') }.to_not raise_error + end + + it "should support auto as a special fstype" do + expect { described_class.new(:name => "/foo", :ensure => :present, :fstype => 'auto') }.to_not raise_error + end + + it "should not support whitespace in fstype" do + expect { described_class.new(:name => "/foo", :ensure => :present, :fstype => 'ext 3') }.to raise_error Puppet::Error, /fstype.*whitespace/ + end + end + + describe "for options" do + it "should support a single option" do + expect { described_class.new(:name => "/foo", :ensure => :present, :options => 'ro') }.to_not raise_error + end + + it "should support muliple options as a comma separated list" do + expect { described_class.new(:name => "/foo", :ensure => :present, :options => 'ro,rsize=4096') }.to_not raise_error + end + + it "should not support whitespace in options" do + expect { described_class.new(:name => "/foo", :ensure => :present, :options => ['ro','foo bar','intr']) }.to raise_error Puppet::Error, /option.*whitespace/ + end + end + + describe "for pass" do + it "should support numeric values" do + expect { described_class.new(:name => "/foo", :ensure => :present, :pass => '0') }.to_not raise_error + expect { described_class.new(:name => "/foo", :ensure => :present, :pass => '1') }.to_not raise_error + expect { described_class.new(:name => "/foo", :ensure => :present, :pass => '2') }.to_not raise_error + end + + it "should support - on Solaris" do + Facter.stubs(:value).with(:operatingsystem).returns 'Solaris' + Facter.stubs(:value).with(:osfamily).returns 'Solaris' + expect { described_class.new(:name => "/foo", :ensure => :present, :pass => '-') }.to_not raise_error + end + + it "should default to 0 on non Solaris" do + Facter.stubs(:value).with(:osfamily).returns nil + Facter.stubs(:value).with(:operatingsystem).returns 'HP-UX' + described_class.new(:name => "/foo", :ensure => :present)[:pass].should == 0 + end + + it "should default to - on Solaris" do + Facter.stubs(:value).with(:operatingsystem).returns 'Solaris' + Facter.stubs(:value).with(:osfamily).returns 'Solaris' + described_class.new(:name => "/foo", :ensure => :present)[:pass].should == '-' + end + end + + describe "for dump" do + it "should support 0 as a value for dump" do + expect { described_class.new(:name => "/foo", :ensure => :present, :dump => '0') }.to_not raise_error + end + + it "should support 1 as a value for dump" do + expect { described_class.new(:name => "/foo", :ensure => :present, :dump => '1') }.to_not raise_error + end + + # Unfortunately the operatingsystem is evaluatet at load time so I am unable to stub operatingsystem + it "should support 2 as a value for dump on FreeBSD", :if => Facter.value(:operatingsystem) == 'FreeBSD' do + expect { described_class.new(:name => "/foo", :ensure => :present, :dump => '2') }.to_not raise_error + end + + it "should not support 2 as a value for dump when not on FreeBSD", :if => Facter.value(:operatingsystem) != 'FreeBSD' do + expect { described_class.new(:name => "/foo", :ensure => :present, :dump => '2') }.to raise_error Puppet::Error, /Invalid value/ + end - def mount_stub(params) - Puppet::Type.type(:mount).validproperties.each do |prop| - unless params[prop] - params[prop] = :absent - @mount[prop] = :absent + it "should default to 0" do + described_class.new(:name => "/foo", :ensure => :present)[:dump].should == 0 end end - params.each do |param, value| - @provider.stubs(param).returns(value) + describe "for atboot" do + it "does not allow non-boolean values" do + expect { described_class.new(:name => "/foo", :ensure => :present, :atboot => 'unknown') }.to raise_error Puppet::Error, /expected a boolean value/ + end + + it "interprets yes as yes" do + resource = described_class.new(:name => "/foo", :ensure => :present, :atboot => :yes) + + expect(resource[:atboot]).to eq(:yes) + end + + it "interprets true as yes" do + resource = described_class.new(:name => "/foo", :ensure => :present, :atboot => :true) + + expect(resource[:atboot]).to eq(:yes) + end + + it "interprets no as no" do + resource = described_class.new(:name => "/foo", :ensure => :present, :atboot => :no) + + expect(resource[:atboot]).to eq(:no) + end + + it "interprets false as no" do + resource = described_class.new(:name => "/foo", :ensure => :present, :atboot => false) + + expect(resource[:atboot]).to eq(:no) + end end end - describe Puppet::Type.type(:mount)::Ensure, "when changing the host" do + describe "when changing the host" do def test_ensure_change(options) - @provider.stubs(:get).with(:ensure).returns options[:from] - @provider.stubs(:ensure).returns options[:from] - @provider.stubs(:mounted?).returns([:mounted,:ghost].include? options[:from]) - @provider.expects(:create).times(options[:create] || 0) - @provider.expects(:destroy).times(options[:destroy] || 0) - @provider.expects(:mount).never - @provider.expects(:unmount).times(options[:unmount] || 0) - @ensure.stubs(:syncothers) - @ensure.should = options[:to] - @ensure.sync - (!!@provider.property_hash[:needs_mount]).should == (!!options[:mount]) - end - - it "should create itself when changing from :ghost to :present" do - test_ensure_change(:from => :ghost, :to => :present, :create => 1) - end - - it "should create itself when changing from :absent to :present" do - test_ensure_change(:from => :absent, :to => :present, :create => 1) - end - - it "should create itself and unmount when changing from :ghost to :unmounted" do - test_ensure_change(:from => :ghost, :to => :unmounted, :create => 1, :unmount => 1) - end - - it "should unmount resource when changing from :mounted to :unmounted" do - test_ensure_change(:from => :mounted, :to => :unmounted, :unmount => 1) - end - - it "should create itself when changing from :absent to :unmounted" do - test_ensure_change(:from => :absent, :to => :unmounted, :create => 1) - end - - it "should unmount resource when changing from :ghost to :absent" do - test_ensure_change(:from => :ghost, :to => :absent, :unmount => 1) - end - - it "should unmount and destroy itself when changing from :mounted to :absent" do - test_ensure_change(:from => :mounted, :to => :absent, :destroy => 1, :unmount => 1) - end - - it "should destroy itself when changing from :unmounted to :absent" do - test_ensure_change(:from => :unmounted, :to => :absent, :destroy => 1) - end - - it "should create itself when changing from :ghost to :mounted" do - test_ensure_change(:from => :ghost, :to => :mounted, :create => 1) - end - - it "should create itself and mount when changing from :absent to :mounted" do - test_ensure_change(:from => :absent, :to => :mounted, :create => 1, :mount => 1) - end - - it "should mount resource when changing from :unmounted to :mounted" do - test_ensure_change(:from => :unmounted, :to => :mounted, :mount => 1) - end - - - it "should be in sync if it is :absent and should be :absent" do - @ensure.should = :absent - @ensure.safe_insync?(:absent).should == true - end - - it "should be out of sync if it is :absent and should be :defined" do - @ensure.should = :defined - @ensure.safe_insync?(:absent).should == false - end - - it "should be out of sync if it is :absent and should be :mounted" do - @ensure.should = :mounted - @ensure.safe_insync?(:absent).should == false - end - - it "should be out of sync if it is :absent and should be :unmounted" do - @ensure.should = :unmounted - @ensure.safe_insync?(:absent).should == false - end - - it "should be out of sync if it is :mounted and should be :absent" do - @ensure.should = :absent - @ensure.safe_insync?(:mounted).should == false - end - - it "should be in sync if it is :mounted and should be :defined" do - @ensure.should = :defined - @ensure.safe_insync?(:mounted).should == true - end - - it "should be in sync if it is :mounted and should be :mounted" do - @ensure.should = :mounted - @ensure.safe_insync?(:mounted).should == true - end - - it "should be out in sync if it is :mounted and should be :unmounted" do - @ensure.should = :unmounted - @ensure.safe_insync?(:mounted).should == false - end - - - it "should be out of sync if it is :unmounted and should be :absent" do - @ensure.should = :absent - @ensure.safe_insync?(:unmounted).should == false - end - - it "should be in sync if it is :unmounted and should be :defined" do - @ensure.should = :defined - @ensure.safe_insync?(:unmounted).should == true - end - - it "should be out of sync if it is :unmounted and should be :mounted" do - @ensure.should = :mounted - @ensure.safe_insync?(:unmounted).should == false - end - - it "should be in sync if it is :unmounted and should be :unmounted" do - @ensure.should = :unmounted - @ensure.safe_insync?(:unmounted).should == true - end - - - it "should be out of sync if it is :ghost and should be :absent" do - @ensure.should = :absent - @ensure.safe_insync?(:ghost).should == false - end + provider.set(:ensure => options[:from]) + provider.expects(:create).times(options[:create] || 0) + provider.expects(:destroy).times(options[:destroy] || 0) + provider.expects(:mount).never + provider.expects(:unmount).times(options[:unmount] || 0) + ensureprop.stubs(:syncothers) + ensureprop.should = options[:to] + ensureprop.sync + (!!provider.property_hash[:needs_mount]).should == (!!options[:mount]) + end - it "should be out of sync if it is :ghost and should be :defined" do - @ensure.should = :defined - @ensure.safe_insync?(:ghost).should == false - end - - it "should be out of sync if it is :ghost and should be :mounted" do - @ensure.should = :mounted - @ensure.safe_insync?(:ghost).should == false - end - - it "should be out of sync if it is :ghost and should be :unmounted" do - @ensure.should = :unmounted - @ensure.safe_insync?(:ghost).should == false - end - - end - - describe Puppet::Type.type(:mount), "when responding to refresh" do - pending "2.6.x specifies slightly different behavior and the desired behavior needs to be clarified and revisited. See ticket #4904" do + it "should create itself when changing from :ghost to :present" do + test_ensure_change(:from => :ghost, :to => :present, :create => 1) + end + + it "should create itself when changing from :absent to :present" do + test_ensure_change(:from => :absent, :to => :present, :create => 1) + end + + it "should create itself and unmount when changing from :ghost to :unmounted" do + test_ensure_change(:from => :ghost, :to => :unmounted, :create => 1, :unmount => 1) + end + + it "should unmount resource when changing from :mounted to :unmounted" do + test_ensure_change(:from => :mounted, :to => :unmounted, :unmount => 1) + end + + it "should create itself when changing from :absent to :unmounted" do + test_ensure_change(:from => :absent, :to => :unmounted, :create => 1) + end + + it "should unmount resource when changing from :ghost to :absent" do + test_ensure_change(:from => :ghost, :to => :absent, :unmount => 1) + end + + it "should unmount and destroy itself when changing from :mounted to :absent" do + test_ensure_change(:from => :mounted, :to => :absent, :destroy => 1, :unmount => 1) + end + + it "should destroy itself when changing from :unmounted to :absent" do + test_ensure_change(:from => :unmounted, :to => :absent, :destroy => 1) + end + + it "should create itself when changing from :ghost to :mounted" do + test_ensure_change(:from => :ghost, :to => :mounted, :create => 1) + end + + it "should create itself and mount when changing from :absent to :mounted" do + test_ensure_change(:from => :absent, :to => :mounted, :create => 1, :mount => 1) + end + + it "should mount resource when changing from :unmounted to :mounted" do + test_ensure_change(:from => :unmounted, :to => :mounted, :mount => 1) + end + + + it "should be in sync if it is :absent and should be :absent" do + ensureprop.should = :absent + ensureprop.safe_insync?(:absent).should == true + end + + it "should be out of sync if it is :absent and should be :defined" do + ensureprop.should = :defined + ensureprop.safe_insync?(:absent).should == false + end + + it "should be out of sync if it is :absent and should be :mounted" do + ensureprop.should = :mounted + ensureprop.safe_insync?(:absent).should == false + end + + it "should be out of sync if it is :absent and should be :unmounted" do + ensureprop.should = :unmounted + ensureprop.safe_insync?(:absent).should == false + end + + it "should be out of sync if it is :mounted and should be :absent" do + ensureprop.should = :absent + ensureprop.safe_insync?(:mounted).should == false + end + + it "should be in sync if it is :mounted and should be :defined" do + ensureprop.should = :defined + ensureprop.safe_insync?(:mounted).should == true + end + + it "should be in sync if it is :mounted and should be :mounted" do + ensureprop.should = :mounted + ensureprop.safe_insync?(:mounted).should == true + end + + it "should be out in sync if it is :mounted and should be :unmounted" do + ensureprop.should = :unmounted + ensureprop.safe_insync?(:mounted).should == false + end + + + it "should be out of sync if it is :unmounted and should be :absent" do + ensureprop.should = :absent + ensureprop.safe_insync?(:unmounted).should == false + end + + it "should be in sync if it is :unmounted and should be :defined" do + ensureprop.should = :defined + ensureprop.safe_insync?(:unmounted).should == true + end + + it "should be out of sync if it is :unmounted and should be :mounted" do + ensureprop.should = :mounted + ensureprop.safe_insync?(:unmounted).should == false + end + + it "should be in sync if it is :unmounted and should be :unmounted" do + ensureprop.should = :unmounted + ensureprop.safe_insync?(:unmounted).should == true + end + + it "should be out of sync if it is :ghost and should be :absent" do + ensureprop.should = :absent + ensureprop.safe_insync?(:ghost).should == false + end + + it "should be out of sync if it is :ghost and should be :defined" do + ensureprop.should = :defined + ensureprop.safe_insync?(:ghost).should == false + end + + it "should be out of sync if it is :ghost and should be :mounted" do + ensureprop.should = :mounted + ensureprop.safe_insync?(:ghost).should == false + end + + it "should be out of sync if it is :ghost and should be :unmounted" do + ensureprop.should = :unmounted + ensureprop.safe_insync?(:ghost).should == false + end + end + + describe "when responding to refresh" do + pending "2.6.x specifies slightly different behavior and the desired behavior needs to be clarified and revisited. See ticket #4904" do it "should remount if it is supposed to be mounted" do - @mount[:ensure] = "mounted" - @provider.expects(:remount) + resource[:ensure] = "mounted" + provider.expects(:remount) - @mount.refresh + resource.refresh end it "should not remount if it is supposed to be present" do - @mount[:ensure] = "present" - @provider.expects(:remount).never + resource[:ensure] = "present" + provider.expects(:remount).never - @mount.refresh + resource.refresh end it "should not remount if it is supposed to be absent" do - @mount[:ensure] = "absent" - @provider.expects(:remount).never + resource[:ensure] = "absent" + provider.expects(:remount).never - @mount.refresh + resource.refresh end it "should not remount if it is supposed to be defined" do - @mount[:ensure] = "defined" - @provider.expects(:remount).never + resource[:ensure] = "defined" + provider.expects(:remount).never - @mount.refresh + resource.refresh end it "should not remount if it is supposed to be unmounted" do - @mount[:ensure] = "unmounted" - @provider.expects(:remount).never + resource[:ensure] = "unmounted" + provider.expects(:remount).never - @mount.refresh + resource.refresh end it "should not remount swap filesystems" do - @mount[:ensure] = "mounted" - @mount[:fstype] = "swap" - @provider.expects(:remount).never + resource[:ensure] = "mounted" + resource[:fstype] = "swap" + provider.expects(:remount).never - @mount.refresh + resource.refresh end end end -end -describe Puppet::Type.type(:mount), "when modifying an existing mount entry", :unless => Puppet.features.microsoft_windows? do - before do - @provider = stub 'provider', :class => Puppet::Type.type(:mount).defaultprovider, :clear => nil, :satisfies? => true, :name => :mock, :remount => nil - Puppet::Type.type(:mount).defaultprovider.stubs(:new).returns(@provider) - - @mount = Puppet::Type.type(:mount).new(:name => "yay", :ensure => :mounted) - - {:device => "/foo/bar", :blockdevice => "/other/bar", :target => "/what/ever", :fstype => 'eh', :options => "", :pass => 0, :dump => 0, :atboot => 0, - :ensure => :mounted}.each do - |param, value| - @mount.provider.stubs(param).returns value - @mount[param] = value + describe "when modifying an existing mount entry" do + + let :initial_values do + { + :ensure => :mounted, + :name => '/mnt/foo', + :device => "/foo/bar", + :blockdevice => "/other/bar", + :target => "/what/ever", + :options => "soft", + :pass => 0, + :dump => 0, + :atboot => :no, + } end - @mount.provider.stubs(:mounted?).returns true - - # stub this to not try to create state.yaml - Puppet::Util::Storage.stubs(:store) - @catalog = Puppet::Resource::Catalog.new - @catalog.add_resource @mount - end + let :resource do + described_class.new(initial_values.merge(:provider => provider)) + end - it "should use the provider to change the dump value" do - @mount.provider.expects(:dump).returns 0 - @mount.provider.expects(:dump=).with(1) - @mount.provider.stubs(:respond_to?).returns(false) - @mount.provider.stubs(:respond_to?).with("dump=").returns(true) + let :provider do + providerclass.new(initial_values) + end - @mount[:dump] = 1 + def run_in_catalog(*resources) + Puppet::Util::Storage.stubs(:store) + catalog = Puppet::Resource::Catalog.new + catalog.add_resource *resources + catalog.apply + end - @catalog.apply - end + it "should use the provider to change the dump value" do + provider.expects(:dump=).with(1) - it "should umount before flushing changes to disk" do - syncorder = sequence('syncorder') - @mount.provider.expects(:options).returns 'soft' - @mount.provider.expects(:ensure).returns :mounted + resource[:dump] = 1 - @mount.provider.stubs(:respond_to?).returns(false) - @mount.provider.stubs(:respond_to?).with("options=").returns(true) + run_in_catalog(resource) + end - @mount.provider.expects(:unmount).in_sequence(syncorder) - @mount.provider.expects(:options=).in_sequence(syncorder).with 'hard' - @mount.expects(:flush).in_sequence(syncorder) # Call inside syncothers - @mount.expects(:flush).in_sequence(syncorder) # I guess transaction or anything calls flush again + it "should umount before flushing changes to disk" do + syncorder = sequence('syncorder') - @mount.provider.stubs(:respond_to?).with(:options=).returns(true) + provider.expects(:unmount).in_sequence(syncorder) + provider.expects(:options=).in_sequence(syncorder).with 'hard' + resource.expects(:flush).in_sequence(syncorder) # Call inside syncothers + resource.expects(:flush).in_sequence(syncorder) # I guess transaction or anything calls flush again - @mount[:ensure] = :unmounted - @mount[:options] = 'hard' + resource[:ensure] = :unmounted + resource[:options] = 'hard' - @catalog.apply + run_in_catalog(resource) + end end - end diff --git a/spec/unit/type/package_spec.rb b/spec/unit/type/package_spec.rb index cb10a42c2..5312b3f38 100755 --- a/spec/unit/type/package_spec.rb +++ b/spec/unit/type/package_spec.rb @@ -74,7 +74,7 @@ describe Puppet::Type.type(:package) do it "should not support :purged as a value to :ensure if the provider does not have the :purgeable feature" do @provider.expects(:satisfies?).with([:purgeable]).returns(false) - proc { Puppet::Type.type(:package).new(:name => "yay", :ensure => :purged) }.should raise_error(Puppet::Error) + expect { Puppet::Type.type(:package).new(:name => "yay", :ensure => :purged) }.to raise_error(Puppet::Error) end it "should support :latest as a value to :ensure if the provider has the :upgradeable feature" do @@ -84,7 +84,7 @@ describe Puppet::Type.type(:package) do it "should not support :latest as a value to :ensure if the provider does not have the :upgradeable feature" do @provider.expects(:satisfies?).with([:upgradeable]).returns(false) - proc { Puppet::Type.type(:package).new(:name => "yay", :ensure => :latest) }.should raise_error(Puppet::Error) + expect { Puppet::Type.type(:package).new(:name => "yay", :ensure => :latest) }.to raise_error(Puppet::Error) end it "should support version numbers as a value to :ensure if the provider has the :versionable feature" do @@ -94,11 +94,11 @@ describe Puppet::Type.type(:package) do it "should not support version numbers as a value to :ensure if the provider does not have the :versionable feature" do @provider.expects(:satisfies?).with([:versionable]).returns(false) - proc { Puppet::Type.type(:package).new(:name => "yay", :ensure => "1.0") }.should raise_error(Puppet::Error) + expect { Puppet::Type.type(:package).new(:name => "yay", :ensure => "1.0") }.to raise_error(Puppet::Error) end it "should accept any string as an argument to :source" do - proc { Puppet::Type.type(:package).new(:name => "yay", :source => "stuff") }.should_not raise_error(Puppet::Error) + expect { Puppet::Type.type(:package).new(:name => "yay", :source => "stuff") }.to_not raise_error end end diff --git a/spec/unit/type/resources_spec.rb b/spec/unit/type/resources_spec.rb index bd035a1fa..30b60edf4 100755 --- a/spec/unit/type/resources_spec.rb +++ b/spec/unit/type/resources_spec.rb @@ -21,11 +21,40 @@ describe resources do end end + describe :purge do + let (:instance) { described_class.new(:name => 'file') } + + it "defaults to false" do + instance[:purge].should be_false + end + + it "can be set to false" do + instance[:purge] = 'false' + end + + it "cannot be set to true for a resource type that does not accept ensure" do + instance.resource_type.stubs(:respond_to?).returns true + instance.resource_type.stubs(:validproperty?).returns false + expect { instance[:purge] = 'yes' }.to raise_error Puppet::Error + end + + it "cannot be set to true for a resource type that does not have instances" do + instance.resource_type.stubs(:respond_to?).returns false + instance.resource_type.stubs(:validproperty?).returns true + expect { instance[:purge] = 'yes' }.to raise_error Puppet::Error + end + + it "can be set to true for a resource type that has instances and can accept ensure" do + instance.resource_type.stubs(:respond_to?).returns true + instance.resource_type.stubs(:validproperty?).returns true + expect { instance[:purge] = 'yes' }.not_to raise_error Puppet::Error + end + end + describe "#generate" do before do @host1 = Puppet::Type.type(:host).new(:name => 'localhost', :ip => '127.0.0.1') @catalog = Puppet::Resource::Catalog.new - @context = Puppet::Transaction.new(@catalog) end describe "when dealing with non-purging resources" do diff --git a/spec/unit/type/user_spec.rb b/spec/unit/type/user_spec.rb index 1ead32054..dd3dca6d9 100755 --- a/spec/unit/type/user_spec.rb +++ b/spec/unit/type/user_spec.rb @@ -15,7 +15,7 @@ describe Puppet::Type.type(:user) do described_class.stubs(:defaultprovider).returns @provider_class end - it "should be able to create a instance" do + it "should be able to create an instance" do described_class.new(:name => "foo").should_not be_nil end @@ -23,11 +23,11 @@ describe Puppet::Type.type(:user) do described_class.provider_feature(:allows_duplicates).should_not be_nil end - it "should have an manages_homedir feature" do + it "should have a manages_homedir feature" do described_class.provider_feature(:manages_homedir).should_not be_nil end - it "should have an manages_passwords feature" do + it "should have a manages_passwords feature" do described_class.provider_feature(:manages_passwords).should_not be_nil end @@ -47,6 +47,29 @@ describe Puppet::Type.type(:user) do described_class.provider_feature(:system_users).should_not be_nil end + describe :managehome do + let (:provider) { @provider_class.new(:name => 'foo', :ensure => :absent) } + let (:instance) { described_class.new(:name => 'foo', :provider => provider) } + + it "defaults to false" do + instance[:managehome].should be_false + end + + it "can be set to false" do + instance[:managehome] = 'false' + end + + it "cannot be set to true for a provider that does not manage homedirs" do + provider.class.stubs(:manages_homedir?).returns false + expect { instance[:managehome] = 'yes' }.to raise_error Puppet::Error + end + + it "can be set to true for a provider that does manage homedirs" do + provider.class.stubs(:manages_homedir?).returns true + instance[:managehome] = 'yes' + end + end + describe "instances" do it "should delegate existence questions to its provider" do @provider = @provider_class.new(:name => 'foo', :ensure => :absent) diff --git a/spec/unit/type/yumrepo_spec.rb b/spec/unit/type/yumrepo_spec.rb index 335412473..66ddfd3e9 100644 --- a/spec/unit/type/yumrepo_spec.rb +++ b/spec/unit/type/yumrepo_spec.rb @@ -2,46 +2,42 @@ require 'spec_helper' - describe Puppet::Type.type(:yumrepo) do include PuppetSpec::Files describe "When validating attributes" do - it "should have a 'name' parameter'" do Puppet::Type.type(:yumrepo).new(:name => "puppetlabs")[:name].should == "puppetlabs" end [:baseurl, :cost, :descr, :enabled, :enablegroups, :exclude, :failovermethod, :gpgcheck, :gpgkey, :http_caching, :include, :includepkgs, :keepalive, :metadata_expire, :mirrorlist, :priority, :protect, :proxy, :proxy_username, :proxy_password, :timeout, - :sslcacert, :sslverify, :sslclientcert, :sslclientkey].each do |param| + :sslcacert, :sslverify, :sslclientcert, :sslclientkey, :s3_enabled].each do |param| it "should have a '#{param}' parameter" do Puppet::Type.type(:yumrepo).attrtype(param).should == :property - end + end end - end describe "When validating attribute values" do - [:cost, :enabled, :enablegroups, :failovermethod, :gpgcheck, :http_caching, :keepalive, :metadata_expire, :priority, :protect, :timeout].each do |param| it "should support :absent as a value to '#{param}' parameter" do Puppet::Type.type(:yumrepo).new(:name => "puppetlabs.repo", param => :absent) - end + end end [:cost, :enabled, :enablegroups, :gpgcheck, :keepalive, :metadata_expire, :priority, :protect, :timeout].each do |param| it "should fail if '#{param}' is not a number" do lambda { Puppet::Type.type(:yumrepo).new(:name => "puppetlabs", param => "notanumber") }.should raise_error - end + end end - [:enabled, :enabledgroups, :gpgcheck, :keepalive, :protect].each do |param| + [:enabled, :enabledgroups, :gpgcheck, :keepalive, :protect, :s3_enabled].each do |param| it "should fail if '#{param}' does not have one of the following values (0|1)" do lambda { Puppet::Type.type(:yumrepo).new(:name => "puppetlabs", param => "2") }.should raise_error end end - + it "should fail if 'failovermethod' does not have one of the following values (roundrobin|priority)" do lambda { Puppet::Type.type(:yumrepo).new(:name => "puppetlabs", :failovermethod => "notavalidvalue") }.should raise_error end @@ -58,13 +54,10 @@ describe Puppet::Type.type(:yumrepo) do Puppet::Type.type(:yumrepo).new(:name => "puppetlabs", :sslverify => "True")[:sslverify].should == "True" Puppet::Type.type(:yumrepo).new(:name => "puppetlabs", :sslverify => "False")[:sslverify].should == "False" end - end # these tests were ported from the old spec/unit/type/yumrepo_spec.rb, pretty much verbatim describe "When manipulating config file" do - - def make_repo(name, hash={}) hash[:name] = name Puppet::Type.type(:yumrepo).new(hash) @@ -86,8 +79,6 @@ describe Puppet::Type.type(:yumrepo) do end end - - before(:each) do @yumdir = tmpdir("yumrepo_spec_tmpdir") @yumconf = File.join(@yumdir, "yum.conf") @@ -100,7 +91,6 @@ describe Puppet::Type.type(:yumrepo) do Puppet::Type.type(:yumrepo).inifile = nil end - it "should be able to create a valid config file" do values = { :descr => "Fedora Core $releasever - $basearch - Base", @@ -115,7 +105,6 @@ describe Puppet::Type.type(:yumrepo) do } repo = make_repo("base", values) - catalog = Puppet::Resource::Catalog.new # Stop Puppet from doing a bunch of magic; might want to think about a util for specs that handles this catalog.host_config = false @@ -129,10 +118,8 @@ describe Puppet::Type.type(:yumrepo) do text.should == EXPECTED_CONTENTS_FOR_CREATED_FILE end - # Modify one existing section it "should be able to modify an existing config file" do - create_data_files devel = make_repo("development", { :descr => "New description" }) @@ -140,7 +127,7 @@ describe Puppet::Type.type(:yumrepo) do devel[:name].should == "development" current_values[devel.property(:descr)].should == 'Fedora Core $releasever - Development Tree' - devel.property(:descr).should == 'New description' + devel[:descr].should == 'New description' catalog = Puppet::Resource::Catalog.new # Stop Puppet from doing a bunch of magic; might want to think about a util for specs that handles this @@ -155,7 +142,6 @@ describe Puppet::Type.type(:yumrepo) do all_sections(inifile).should == ['base', 'development', 'main'] end - # Delete mirrorlist by setting it to :absent and enable baseurl it "should support 'absent' value" do create_data_files @@ -180,14 +166,9 @@ describe Puppet::Type.type(:yumrepo) do sec["mirrorlist"].should == nil sec["baseurl"].should == baseurl end - - end - - end - EXPECTED_CONTENTS_FOR_CREATED_FILE = <<'EOF' [base] name=Fedora Core $releasever - $basearch - Base @@ -200,7 +181,6 @@ proxy_username=username proxy_password=password EOF - FEDORA_REPO_FILE = <<END [base] name=Fedora Core $releasever - $basearch - Base diff --git a/spec/unit/type/zone_spec.rb b/spec/unit/type/zone_spec.rb index 6fe2184cf..b34cf5841 100755 --- a/spec/unit/type/zone_spec.rb +++ b/spec/unit/type/zone_spec.rb @@ -59,6 +59,7 @@ describe Puppet::Type.type(:zone) do fs = 'random-pool/some-zfs' catalog = Puppet::Resource::Catalog.new + relationship_graph = Puppet::Graph::RelationshipGraph.new(Puppet::Graph::RandomPrioritizer.new) zfs = Puppet::Type.type(:zfs).new(:name => fs) catalog.add_resource zfs @@ -69,7 +70,9 @@ describe Puppet::Type.type(:zone) do :provider => :solaris) catalog.add_resource zone - catalog.relationship_graph.dependencies(zone).should == [zfs] + + relationship_graph.populate_from(catalog) + relationship_graph.dependencies(zone).should == [zfs] end describe StateMachine do let (:sm) { StateMachine.new } diff --git a/spec/unit/type_spec.rb b/spec/unit/type_spec.rb index 8b19020c6..b160c743f 100755 --- a/spec/unit/type_spec.rb +++ b/spec/unit/type_spec.rb @@ -105,26 +105,30 @@ describe Puppet::Type, :unless => Puppet.features.microsoft_windows? do Puppet::Type.type(:mount).new(:name => "foo").must respond_to(:virtual?) end - it "should consider its version to be its catalog version" do - resource = Puppet::Type.type(:mount).new(:name => "foo") - catalog = Puppet::Resource::Catalog.new - catalog.version = 50 - catalog.add_resource resource - - resource.version.should == 50 - end - it "should consider its version to be zero if it has no catalog" do Puppet::Type.type(:mount).new(:name => "foo").version.should == 0 end - it "should provide source_descriptors" do - resource = Puppet::Type.type(:mount).new(:name => "foo") - catalog = Puppet::Resource::Catalog.new - catalog.version = 50 - catalog.add_resource resource + context "resource attributes" do + let(:resource) { + resource = Puppet::Type.type(:mount).new(:name => "foo") + catalog = Puppet::Resource::Catalog.new + catalog.version = 50 + catalog.add_resource resource + resource + } + + it "should consider its version to be its catalog version" do + resource.version.should == 50 + end - resource.source_descriptors.should == {:tags=>["mount", "foo"], :path=>"/Mount[foo]"} + it "should have tags" do + resource.tags.should == ["mount", "foo"] + end + + it "should have a path" do + resource.path.should == "/Mount[foo]" + end end it "should consider its type to be the name of its class" do @@ -313,10 +317,10 @@ describe Puppet::Type, :unless => Puppet.features.microsoft_windows? do end it "should copy the resource's parameters as its own" do - resource = Puppet::Resource.new(:mount, "/foo", :parameters => {:atboot => true, :fstype => "boo"}) + resource = Puppet::Resource.new(:mount, "/foo", :parameters => {:atboot => :yes, :fstype => "boo"}) params = Puppet::Type.type(:mount).new(resource).to_hash params[:fstype].should == "boo" - params[:atboot].should == true + params[:atboot].should == :yes end end @@ -349,9 +353,9 @@ describe Puppet::Type, :unless => Puppet.features.microsoft_windows? do end it "should use any remaining hash keys as its parameters" do - resource = Puppet::Type.type(:mount).new(:title => "/foo", :catalog => "foo", :atboot => true, :fstype => "boo") + resource = Puppet::Type.type(:mount).new(:title => "/foo", :catalog => "foo", :atboot => :yes, :fstype => "boo") resource[:fstype].must == "boo" - resource[:atboot].must == true + resource[:atboot].must == :yes end end @@ -382,12 +386,12 @@ describe Puppet::Type, :unless => Puppet.features.microsoft_windows? do end it "should fail if no title, name, or namevar are provided" do - expect { Puppet::Type.type(:file).new(:atboot => true) }.to raise_error(Puppet::Error) + expect { Puppet::Type.type(:mount).new(:atboot => :yes) }.to raise_error(Puppet::Error) end it "should set the attributes in the order returned by the class's :allattrs method" do Puppet::Type.type(:mount).stubs(:allattrs).returns([:name, :atboot, :noop]) - resource = Puppet::Resource.new(:mount, "/foo", :parameters => {:name => "myname", :atboot => "myboot", :noop => "whatever"}) + resource = Puppet::Resource.new(:mount, "/foo", :parameters => {:name => "myname", :atboot => :yes, :noop => "whatever"}) set = [] @@ -404,7 +408,7 @@ describe Puppet::Type, :unless => Puppet.features.microsoft_windows? do it "should always set the name and then default provider before anything else" do Puppet::Type.type(:mount).stubs(:allattrs).returns([:provider, :name, :atboot]) - resource = Puppet::Resource.new(:mount, "/foo", :parameters => {:name => "myname", :atboot => "myboot"}) + resource = Puppet::Resource.new(:mount, "/foo", :parameters => {:name => "myname", :atboot => :yes}) set = [] @@ -429,7 +433,7 @@ describe Puppet::Type, :unless => Puppet.features.microsoft_windows? do end it "should retain a copy of the originally provided parameters" do - Puppet::Type.type(:mount).new(:name => "foo", :atboot => true, :noop => false).original_parameters.should == {:atboot => true, :noop => false} + Puppet::Type.type(:mount).new(:name => "foo", :atboot => :yes, :noop => false).original_parameters.should == {:atboot => :yes, :noop => false} end it "should delete the name via the namevar from the originally provided parameters" do @@ -535,9 +539,9 @@ describe Puppet::Type, :unless => Puppet.features.microsoft_windows? do end it "should set the name of the returned resource if its own name and title differ" do - @resource[:name] = "my name" + @resource[:name] = "myname" @resource.title = "other name" - @resource.retrieve_resource[:name].should == "my name" + @resource.retrieve_resource[:name].should == "myname" end it "should provide a value for all set properties" do @@ -802,18 +806,47 @@ describe Puppet::Type::RelationshipMetaparam do Puppet::Type.metaparamclass(:require).new(:resource => mock("resource")).should respond_to(:validate_relationship) end - it "should fail if any specified resource is not found in the catalog" do - catalog = mock 'catalog' - resource = stub 'resource', :catalog => catalog, :ref => "resource" + describe 'if any specified resource is not in the catalog' do + let(:catalog) { mock 'catalog' } - param = Puppet::Type.metaparamclass(:require).new(:resource => resource, :value => %w{Foo[bar] Class[test]}) + let(:resource) do + stub 'resource', + :catalog => catalog, + :ref => 'resource', + :line= => nil, + :file= => nil + end + + let(:param) { Puppet::Type.metaparamclass(:require).new(:resource => resource, :value => %w{Foo[bar] Class[test]}) } - catalog.expects(:resource).with("Foo[bar]").returns "something" - catalog.expects(:resource).with("Class[Test]").returns nil + before do + catalog.expects(:resource).with("Foo[bar]").returns "something" + catalog.expects(:resource).with("Class[Test]").returns nil + end - param.expects(:fail).with { |string| string.include?("Class[Test]") } + describe "and the resource doesn't have a file or line number" do + it "raises an error" do + expect { param.validate_relationship }.to raise_error do |error| + error.should be_a Puppet::ResourceError + error.message.should match %r[Class\[Test\]] + end + end + end - param.validate_relationship + describe "and the resource has a file or line number" do + before do + resource.stubs(:line).returns '42' + resource.stubs(:file).returns '/hitchhikers/guide/to/the/galaxy' + end + + it "raises an error with context" do + expect { param.validate_relationship }.to raise_error do |error| + error.should be_a Puppet::ResourceError + error.message.should match %r[Class\[Test\]] + error.message.should match %r["in /hitchhikers/guide/to/the/galaxy:42"] + end + end + end end end diff --git a/spec/unit/util/backups_spec.rb b/spec/unit/util/backups_spec.rb index 30fd7b9c8..8d3ea1956 100755 --- a/spec/unit/util/backups_spec.rb +++ b/spec/unit/util/backups_spec.rb @@ -35,7 +35,7 @@ describe Puppet::Util::Backups do end it "a bucket should be used when provided" do - File.stubs(:stat).with(path).returns(mock('stat', :ftype => 'file')) + File.stubs(:lstat).with(path).returns(mock('lstat', :ftype => 'file')) bucket.expects(:backup).with(path).returns("mysum") FileTest.expects(:exists?).with(path).returns(true) @@ -43,7 +43,7 @@ describe Puppet::Util::Backups do end it "should propagate any exceptions encountered when backing up to a filebucket" do - File.stubs(:stat).with(path).returns(mock('stat', :ftype => 'file')) + File.stubs(:lstat).with(path).returns(mock('lstat', :ftype => 'file')) bucket.expects(:backup).raises ArgumentError FileTest.expects(:exists?).with(path).returns(true) @@ -108,7 +108,7 @@ describe Puppet::Util::Backups do bucket.expects(:backup).with(filename).returns true - File.stubs(:stat).with(path).returns(stub('stat', :ftype => 'directory')) + File.stubs(:lstat).with(path).returns(stub('lstat', :ftype => 'directory')) FileTest.stubs(:exists?).with(path).returns(true) FileTest.stubs(:exists?).with(filename).returns(true) diff --git a/spec/unit/util/http_proxy_spec.rb b/spec/unit/util/http_proxy_spec.rb new file mode 100644 index 000000000..bc6b4d2b7 --- /dev/null +++ b/spec/unit/util/http_proxy_spec.rb @@ -0,0 +1,83 @@ +require 'uri' +require 'spec_helper' +require 'puppet/util/http_proxy' + +describe Puppet::Util::HttpProxy do + + host, port = 'some.host', 1234 + + describe ".http_proxy_env" do + it "should return nil if no environment variables" do + subject.http_proxy_env.should == nil + end + + it "should return a URI::HTTP object if http_proxy env variable is set" do + Puppet::Util.withenv('HTTP_PROXY' => host) do + subject.http_proxy_env.should == URI.parse(host) + end + end + + it "should return a URI::HTTP object if HTTP_PROXY env variable is set" do + Puppet::Util.withenv('HTTP_PROXY' => host) do + subject.http_proxy_env.should == URI.parse(host) + end + end + + it "should return a URI::HTTP object with .host and .port if URI is given" do + Puppet::Util.withenv('HTTP_PROXY' => "http://#{host}:#{port}") do + subject.http_proxy_env.should == URI.parse("http://#{host}:#{port}") + end + end + + it "should return nil if proxy variable is malformed" do + Puppet::Util.withenv('HTTP_PROXY' => 'this is not a valid URI') do + subject.http_proxy_env.should == nil + end + end + end + + describe ".http_proxy_host" do + it "should return nil if no proxy host in config or env" do + subject.http_proxy_host.should == nil + end + + it "should return a proxy host if set in config" do + Puppet.settings[:http_proxy_host] = host + subject.http_proxy_host.should == host + end + + it "should return nil if set to `none` in config" do + Puppet.settings[:http_proxy_host] = 'none' + subject.http_proxy_host.should == nil + end + + it "uses environment variable before puppet settings" do + Puppet::Util.withenv('HTTP_PROXY' => "http://#{host}:#{port}") do + Puppet.settings[:http_proxy_host] = 'not.correct' + subject.http_proxy_host.should == host + end + end + end + + describe ".http_proxy_port" do + it "should return a proxy port if set in environment" do + Puppet::Util.withenv('HTTP_PROXY' => "http://#{host}:#{port}") do + subject.http_proxy_port.should == port + end + end + + it "should return a proxy port if set in config" do + Puppet.settings[:http_proxy_port] = port + subject.http_proxy_port.should == port + end + + it "uses environment variable before puppet settings" do + Puppet::Util.withenv('HTTP_PROXY' => "http://#{host}:#{port}") do + Puppet.settings[:http_proxy_port] = 7456 + subject.http_proxy_port.should == port + end + end + + end + +end diff --git a/spec/unit/util/loadedfile_spec.rb b/spec/unit/util/loadedfile_spec.rb deleted file mode 100755 index 446c8d1e6..000000000 --- a/spec/unit/util/loadedfile_spec.rb +++ /dev/null @@ -1,71 +0,0 @@ -#! /usr/bin/env ruby -require 'spec_helper' - -require 'tempfile' -require 'puppet/util/loadedfile' - -describe Puppet::Util::LoadedFile do - include PuppetSpec::Files - before(:each) do - @f = Tempfile.new('loadedfile_test') - @f.puts "yayness" - @f.flush - - @loaded = Puppet::Util::LoadedFile.new(@f.path) - - fake_ctime = Time.now - (2 * Puppet[:filetimeout]) - @stat = stub('stat', :ctime => fake_ctime) - @fake_now = Time.now + (2 * Puppet[:filetimeout]) - end - - it "should accept files that don't exist" do - nofile = tmpfile('testfile') - File.exists?(nofile).should == false - lambda{ Puppet::Util::LoadedFile.new(nofile) }.should_not raise_error - end - - it "should recognize when the file has not changed" do - # Use fake "now" so that we can be sure changed? actually checks, without sleeping - # for Puppet[:filetimeout] seconds. - Time.stubs(:now).returns(@fake_now) - @loaded.changed?.should == false - end - - it "should recognize when the file has changed" do - # Fake File.stat so we don't have to depend on the filesystem granularity. Doing a flush() - # just didn't do the job. - File.stubs(:stat).returns(@stat) - # Use fake "now" so that we can be sure changed? actually checks, without sleeping - # for Puppet[:filetimeout] seconds. - Time.stubs(:now).returns(@fake_now) - @loaded.changed?.should be_an_instance_of(Time) - end - - it "should not catch a change until the timeout has elapsed" do - # Fake File.stat so we don't have to depend on the filesystem granularity. Doing a flush() - # just didn't do the job. - File.stubs(:stat).returns(@stat) - @loaded.changed?.should be(false) - # Use fake "now" so that we can be sure changed? actually checks, without sleeping - # for Puppet[:filetimeout] seconds. - Time.stubs(:now).returns(@fake_now) - @loaded.changed?.should_not be(false) - end - - it "should consider a file changed when that file is missing" do - @f.close! - # Use fake "now" so that we can be sure changed? actually checks, without sleeping - # for Puppet[:filetimeout] seconds. - Time.stubs(:now).returns(@fake_now) - @loaded.changed?.should_not be(false) - end - - it "should disable checking if Puppet[:filetimeout] is negative" do - Puppet[:filetimeout] = -1 - @loaded.changed?.should_not be(false) - end - - after(:each) do - @f.close - end -end diff --git a/spec/unit/util/log_spec.rb b/spec/unit/util/log_spec.rb index dd285c839..99984246a 100755 --- a/spec/unit/util/log_spec.rb +++ b/spec/unit/util/log_spec.rb @@ -6,6 +6,10 @@ require 'puppet/util/log' describe Puppet::Util::Log do include PuppetSpec::Files + def log_notice(message) + Puppet::Util::Log.new(:level => :notice, :message => message) + end + it "should write a given message to the specified destination" do arraydest = [] Puppet::Util::Log.newdestination(Puppet::Test::LogCollector.new(arraydest)) @@ -39,6 +43,57 @@ describe Puppet::Util::Log do end end + describe "#with_destination" do + it "does nothing when nested" do + logs = [] + destination = Puppet::Test::LogCollector.new(logs) + Puppet::Util::Log.with_destination(destination) do + Puppet::Util::Log.with_destination(destination) do + log_notice("Inner block") + end + + log_notice("Outer block") + end + + log_notice("Outside") + + expect(logs.collect(&:message)).to include("Inner block", "Outer block") + expect(logs.collect(&:message)).not_to include("Outside") + end + + it "logs when called a second time" do + logs = [] + destination = Puppet::Test::LogCollector.new(logs) + + Puppet::Util::Log.with_destination(destination) do + log_notice("First block") + end + + log_notice("Between blocks") + + Puppet::Util::Log.with_destination(destination) do + log_notice("Second block") + end + + expect(logs.collect(&:message)).to include("First block", "Second block") + expect(logs.collect(&:message)).not_to include("Between blocks") + end + + it "doesn't close the destination if already set manually" do + logs = [] + destination = Puppet::Test::LogCollector.new(logs) + + Puppet::Util::Log.newdestination(destination) + Puppet::Util::Log.with_destination(destination) do + log_notice "Inner block" + end + + log_notice "Outer block" + Puppet::Util::Log.close(destination) + + expect(logs.collect(&:message)).to include("Inner block", "Outer block") + end + end describe Puppet::Util::Log::DestConsole do before do @console = Puppet::Util::Log::DestConsole.new @@ -186,7 +241,7 @@ describe Puppet::Util::Log do log.tags.should be_include("bar") end - it "should use an passed-in source" do + it "should use a passed-in source" do Puppet::Util::Log.any_instance.expects(:source=).with "foo" Puppet::Util::Log.new(:level => "notice", :message => :foo, :source => "foo") end @@ -247,34 +302,37 @@ describe Puppet::Util::Log do end describe "when setting the source as a RAL object" do + let(:path) { File.expand_path('/foo/bar') } + it "should tag itself with any tags the source has" do - source = Puppet::Type.type(:file).new :path => make_absolute("/foo/bar") + source = Puppet::Type.type(:file).new :path => path log = Puppet::Util::Log.new(:level => "notice", :message => :foo, :source => source) source.tags.each do |tag| log.tags.should be_include(tag) end end - it "should use the source_descriptors" do - source = stub "source" - source.stubs(:source_descriptors).returns(:tags => ["tag","tag2"], :path => "path", :version => 100) + it "should set the source to 'path', when available" do + source = Puppet::Type.type(:file).new :path => path + source.tags = ["tag", "tag2"] log = Puppet::Util::Log.new(:level => "notice", :message => :foo) + log.expects(:tag).with("file") log.expects(:tag).with("tag") log.expects(:tag).with("tag2") log.source = source - log.source.should == "path" + log.source.should == "/File[#{path}]" end it "should copy over any file and line information" do - source = Puppet::Type.type(:file).new :path => make_absolute("/foo/bar") + source = Puppet::Type.type(:file).new :path => path source.file = "/my/file" source.line = 50 log = Puppet::Util::Log.new(:level => "notice", :message => :foo, :source => source) - log.file.should == "/my/file" log.line.should == 50 + log.file.should == "/my/file" end end @@ -304,4 +362,17 @@ describe Puppet::Util::Log do log.to_yaml_properties.should include(:@line) end end + + it "should round trip through pson" do + log = Puppet::Util::Log.new(:level => 'notice', :message => 'hooray', :file => 'thefile', :line => 1729, :source => 'specs', :tags => ['a', 'b', 'c']) + tripped = Puppet::Util::Log.from_pson(PSON.parse(log.to_pson)) + + tripped.file.should == log.file + tripped.line.should == log.line + tripped.level.should == log.level + tripped.message.should == log.message + tripped.source.should == log.source + tripped.tags.should == log.tags + tripped.time.should == log.time + end end diff --git a/spec/unit/util/metric_spec.rb b/spec/unit/util/metric_spec.rb index d3b4c929f..e8ffe00dd 100755 --- a/spec/unit/util/metric_spec.rb +++ b/spec/unit/util/metric_spec.rb @@ -83,4 +83,16 @@ describe Puppet::Util::Metric do it "should return nil if the named value cannot be found" do @metric["foo"].should == 0 end + + it "should round trip through pson" do + metric = Puppet::Util::Metric.new("foo", "mylabel") + metric.newvalue("v1", 10.1, "something") + metric.newvalue("v2", 20, "something else") + + tripped = Puppet::Util::Metric.from_pson(PSON.parse(metric.to_pson)) + + tripped.name.should == metric.name + tripped.label.should == metric.label + tripped.values.should == metric.values + end end diff --git a/spec/unit/util/monkey_patches_spec.rb b/spec/unit/util/monkey_patches_spec.rb index 7d725cc7d..a722d8bff 100755 --- a/spec/unit/util/monkey_patches_spec.rb +++ b/spec/unit/util/monkey_patches_spec.rb @@ -292,3 +292,9 @@ describe OpenSSL::SSL::SSLContext do ciphers.should be_empty end end + +describe SecureRandom do + it 'generates a properly formatted uuid' do + SecureRandom.uuid.should =~ /[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}/i + end +end diff --git a/spec/unit/util/network_device/config_spec.rb b/spec/unit/util/network_device/config_spec.rb index 25ff6b220..06daf96fc 100755 --- a/spec/unit/util/network_device/config_spec.rb +++ b/spec/unit/util/network_device/config_spec.rb @@ -7,117 +7,79 @@ describe Puppet::Util::NetworkDevice::Config do include PuppetSpec::Files before(:each) do - Puppet[:deviceconfig] = make_absolute("/dummy") - FileTest.stubs(:exists?).with(make_absolute("/dummy")).returns(true) - end - - describe "when initializing" do - before :each do - Puppet::Util::NetworkDevice::Config.any_instance.stubs(:read) - end - - it "should use the deviceconfig setting as pathname" do - Puppet.expects(:[]).with(:deviceconfig).returns(make_absolute("/dummy")) - - Puppet::Util::NetworkDevice::Config.new - end - - it "should raise an error if no file is defined finally" do - Puppet.expects(:[]).with(:deviceconfig).returns(nil) - - lambda { Puppet::Util::NetworkDevice::Config.new }.should raise_error(Puppet::DevError) - end - - it "should read and parse the file" do - Puppet::Util::NetworkDevice::Config.any_instance.expects(:read) - - Puppet::Util::NetworkDevice::Config.new - end + Puppet[:deviceconfig] = tmpfile('deviceconfig') end describe "when parsing device" do - before :each do - @config = Puppet::Util::NetworkDevice::Config.new - @config.stubs(:changed?).returns(true) - @fd = stub 'fd' - File.stubs(:open).yields(@fd) + let(:config) { Puppet::Util::NetworkDevice::Config.new } + + def write_device_config(*lines) + File.open(Puppet[:deviceconfig], 'w') {|f| f.puts lines} end it "should skip comments" do - @fd.stubs(:each).yields(' # comment') - - OpenStruct.expects(:new).never + write_device_config(' # comment') - @config.read + config.devices.should be_empty end it "should increment line number even on commented lines" do - @fd.stubs(:each).multiple_yields(' # comment','[router.puppetlabs.com]') + write_device_config(' # comment','[router.puppetlabs.com]') - @config.read - @config.devices.should be_include('router.puppetlabs.com') + config.devices.should be_include('router.puppetlabs.com') end it "should skip blank lines" do - @fd.stubs(:each).yields(' ') + write_device_config(' ') - @config.read - @config.devices.should be_empty + config.devices.should be_empty end it "should produce the correct line number" do - @fd.stubs(:each).multiple_yields(' ', '[router.puppetlabs.com]') + write_device_config(' ', '[router.puppetlabs.com]') - @config.read - @config.devices['router.puppetlabs.com'].line.should == 2 + config.devices['router.puppetlabs.com'].line.should == 2 end it "should throw an error if the current device already exists" do - @fd.stubs(:each).multiple_yields('[router.puppetlabs.com]', '[router.puppetlabs.com]') + write_device_config('[router.puppetlabs.com]', '[router.puppetlabs.com]') - lambda { @config.read }.should raise_error end it "should accept device certname containing dashes" do - @fd.stubs(:each).yields('[router-1.puppetlabs.com]') + write_device_config('[router-1.puppetlabs.com]') - @config.read - @config.devices.should include('router-1.puppetlabs.com') + config.devices.should include('router-1.puppetlabs.com') end it "should create a new device for each found device line" do - @fd.stubs(:each).multiple_yields('[router.puppetlabs.com]', '[swith.puppetlabs.com]') + write_device_config('[router.puppetlabs.com]', '[swith.puppetlabs.com]') - @config.read - @config.devices.size.should == 2 + config.devices.size.should == 2 end it "should parse the device type" do - @fd.stubs(:each).multiple_yields('[router.puppetlabs.com]', 'type cisco') + write_device_config('[router.puppetlabs.com]', 'type cisco') - @config.read - @config.devices['router.puppetlabs.com'].provider.should == 'cisco' + config.devices['router.puppetlabs.com'].provider.should == 'cisco' end it "should parse the device url" do - @fd.stubs(:each).multiple_yields('[router.puppetlabs.com]', 'type cisco', 'url ssh://test/') + write_device_config('[router.puppetlabs.com]', 'type cisco', 'url ssh://test/') - @config.read - @config.devices['router.puppetlabs.com'].url.should == 'ssh://test/' + config.devices['router.puppetlabs.com'].url.should == 'ssh://test/' end it "should parse the debug mode" do - @fd.stubs(:each).multiple_yields('[router.puppetlabs.com]', 'type cisco', 'url ssh://test/', 'debug') + write_device_config('[router.puppetlabs.com]', 'type cisco', 'url ssh://test/', 'debug') - @config.read - @config.devices['router.puppetlabs.com'].options.should == { :debug => true } + config.devices['router.puppetlabs.com'].options.should == { :debug => true } end it "should set the debug mode to false by default" do - @fd.stubs(:each).multiple_yields('[router.puppetlabs.com]', 'type cisco', 'url ssh://test/') + write_device_config('[router.puppetlabs.com]', 'type cisco', 'url ssh://test/') - @config.read - @config.devices['router.puppetlabs.com'].options.should == { :debug => false } + config.devices['router.puppetlabs.com'].options.should == { :debug => false } end end diff --git a/spec/unit/util/pidlock_spec.rb b/spec/unit/util/pidlock_spec.rb index 27277bff9..3e70ce008 100644 --- a/spec/unit/util/pidlock_spec.rb +++ b/spec/unit/util/pidlock_spec.rb @@ -45,11 +45,14 @@ describe Puppet::Util::Pidlock do @lock.lock.should be_true end - it "should create a lock file" do @lock.lock File.should be_exists(@lockfile) end + + it "should expose the lock file_path" do + @lock.file_path.should == @lockfile + end end describe "#unlock" do diff --git a/spec/unit/util/tagging_spec.rb b/spec/unit/util/tagging_spec.rb index 190eaa5d6..67b766fe0 100755 --- a/spec/unit/util/tagging_spec.rb +++ b/spec/unit/util/tagging_spec.rb @@ -42,27 +42,23 @@ describe Puppet::Util::Tagging, "when adding tags" do end it "should fail on tags containing '*' characters" do - lambda { @tagger.tag("bad*tag") }.should raise_error(Puppet::ParseError) + expect { @tagger.tag("bad*tag") }.to raise_error(Puppet::ParseError) end it "should fail on tags starting with '-' characters" do - lambda { @tagger.tag("-badtag") }.should raise_error(Puppet::ParseError) + expect { @tagger.tag("-badtag") }.to raise_error(Puppet::ParseError) end it "should fail on tags containing ' ' characters" do - lambda { @tagger.tag("bad tag") }.should raise_error(Puppet::ParseError) + expect { @tagger.tag("bad tag") }.to raise_error(Puppet::ParseError) end it "should allow alpha tags" do - lambda { @tagger.tag("good_tag") }.should_not raise_error(Puppet::ParseError) + expect { @tagger.tag("good_tag") }.to_not raise_error end it "should allow tags containing '.' characters" do - lambda { @tagger.tag("good.tag") }.should_not raise_error(Puppet::ParseError) - end - - it "should provide a method for testing tag validity" do - @tagger.singleton_class.publicize_methods(:valid_tag?) { @tagger.should be_respond_to(:valid_tag?) } + expect { @tagger.tag("good.tag") }.to_not raise_error end it "should add qualified classes as tags" do diff --git a/spec/unit/util/warnings_spec.rb b/spec/unit/util/warnings_spec.rb index 376d5cf85..f926ae10b 100755 --- a/spec/unit/util/warnings_spec.rb +++ b/spec/unit/util/warnings_spec.rb @@ -7,7 +7,7 @@ describe Puppet::Util::Warnings do @msg2 = "more booness" end - {:notice => "notice_once", :warning => "warnonce"}.each do |log, method| + {:notice => "notice_once", :warning => "warnonce", :debug => "debug_once"}.each do |log, method| describe "when registring '#{log}' messages" do it "should always return nil" do Puppet::Util::Warnings.send(method, @msg1).should be(nil) diff --git a/spec/unit/util/watched_file_spec.rb b/spec/unit/util/watched_file_spec.rb new file mode 100644 index 000000000..f9e251062 --- /dev/null +++ b/spec/unit/util/watched_file_spec.rb @@ -0,0 +1,52 @@ +require 'spec_helper' +require 'puppet/util/watched_file' +require 'puppet/util/watcher' + +describe Puppet::Util::WatchedFile do + let(:an_absurdly_long_timeout) { Puppet::Util::Watcher::Timer.new(100000) } + let(:an_immediate_timeout) { Puppet::Util::Watcher::Timer.new(0) } + + it "acts like a string so that it can be used as a filename" do + watched = Puppet::Util::WatchedFile.new("foo") + + expect(watched.to_str).to eq("foo") + end + + it "considers the file to be unchanged before the timeout expires" do + watched = Puppet::Util::WatchedFile.new(a_file_that_doesnt_exist, an_absurdly_long_timeout) + + expect(watched).to_not be_changed + end + + it "considers a file that is created to be changed" do + watched_filename = a_file_that_doesnt_exist + watched = Puppet::Util::WatchedFile.new(watched_filename, an_immediate_timeout) + + create_file(watched_filename) + + expect(watched).to be_changed + end + + it "considers a missing file to remain unchanged" do + watched = Puppet::Util::WatchedFile.new(a_file_that_doesnt_exist, an_immediate_timeout) + + expect(watched).to_not be_changed + end + + it "considers a file that has changed but the timeout is not expired to still be unchanged" do + watched_filename = a_file_that_doesnt_exist + watched = Puppet::Util::WatchedFile.new(watched_filename, an_absurdly_long_timeout) + + create_file(watched_filename) + + expect(watched).to_not be_changed + end + + def create_file(name) + File.open(name, "wb") { |file| file.puts("contents") } + end + + def a_file_that_doesnt_exist + PuppetSpec::Files.tmpfile("watched_file") + end +end diff --git a/spec/unit/util/watcher/periodic_watcher_spec.rb b/spec/unit/util/watcher/periodic_watcher_spec.rb new file mode 100644 index 000000000..9ae376fa7 --- /dev/null +++ b/spec/unit/util/watcher/periodic_watcher_spec.rb @@ -0,0 +1,52 @@ +require 'spec_helper' + +require 'puppet/util/watcher' + +describe Puppet::Util::Watcher::PeriodicWatcher do + let(:enabled_timeout) { 1 } + let(:disabled_timeout) { -1 } + let(:a_value) { 15 } + let(:a_different_value) { 16 } + + let(:unused_watcher) { mock('unused watcher') } + let(:unchanged_watcher) { a_watcher_reporting(a_value) } + let(:changed_watcher) { a_watcher_reporting(a_value, a_different_value) } + + it 'reads only the initial change state when the timeout has not yet expired' do + watcher = Puppet::Util::Watcher::PeriodicWatcher.new(unchanged_watcher, an_unexpired_timer(enabled_timeout)) + + expect(watcher).to_not be_changed + end + + it 'reads enough values to determine change when the timeout has expired' do + watcher = Puppet::Util::Watcher::PeriodicWatcher.new(changed_watcher, an_expired_timer(enabled_timeout)) + + expect(watcher).to be_changed + end + + it 'is always marked as changed when the timeout is disabled' do + watcher = Puppet::Util::Watcher::PeriodicWatcher.new(unused_watcher, an_expired_timer(disabled_timeout)) + + expect(watcher).to be_changed + end + + def a_watcher_reporting(*observed_values) + Puppet::Util::Watcher::ChangeWatcher.watch(proc do + observed_values.shift or raise "No more observed values to report!" + end) + end + + def an_expired_timer(timeout) + a_time_that_reports_expired_as(true, timeout) + end + + def an_unexpired_timer(timeout) + a_time_that_reports_expired_as(false, timeout) + end + + def a_time_that_reports_expired_as(expired, timeout) + timer = Puppet::Util::Watcher::Timer.new(timeout) + timer.stubs(:expired?).returns(expired) + timer + end +end diff --git a/spec/unit/util/watcher_spec.rb b/spec/unit/util/watcher_spec.rb new file mode 100644 index 000000000..64fab9681 --- /dev/null +++ b/spec/unit/util/watcher_spec.rb @@ -0,0 +1,56 @@ +#! /usr/bin/env ruby +require 'spec_helper' + +require 'puppet/util/watcher' + +describe Puppet::Util::Watcher do + describe "the common file ctime watcher" do + FakeStat = Struct.new(:ctime) + + def ctime(time) + FakeStat.new(time) + end + + let(:filename) { "fake" } + + def after_reading_the_sequence(initial, *results) + expectation = File.stubs(:stat).with(filename) + ([initial] + results).each do |result| + expectation = if result.is_a? Class + expectation.raises(result) + else + expectation.returns(result) + end.then + end + + watcher = Puppet::Util::Watcher::Common.file_ctime_change_watcher(filename) + results.size.times { watcher = watcher.next_reading } + + watcher + end + + it "is intially unchanged" do + expect(after_reading_the_sequence(ctime(20))).to_not be_changed + end + + it "has not changed if a section of the file path continues to not exist" do + expect(after_reading_the_sequence(Errno::ENOTDIR, Errno::ENOTDIR)).to_not be_changed + end + + it "has not changed if the file continues to not exist" do + expect(after_reading_the_sequence(Errno::ENOENT, Errno::ENOENT)).to_not be_changed + end + + it "has changed if the file is created" do + expect(after_reading_the_sequence(Errno::ENOENT, ctime(20))).to be_changed + end + + it "is marked as changed if the file is deleted" do + expect(after_reading_the_sequence(ctime(20), Errno::ENOENT)).to be_changed + end + + it "is marked as changed if the file modified" do + expect(after_reading_the_sequence(ctime(20), ctime(21))).to be_changed + end + end +end diff --git a/spec/unit/util_spec.rb b/spec/unit/util_spec.rb index 401e09076..22be8fa7a 100755 --- a/spec/unit/util_spec.rb +++ b/spec/unit/util_spec.rb @@ -497,6 +497,22 @@ describe Puppet::Util do # ...and check the replacement was complete. File.read(target.path).should == "hello, world\n" end + + {:string => '664', :number => 0664, :symbolic => "ug=rw-,o=r--" }.each do |label,mode| + it "should support #{label} format permissions" do + new_target = target.path + "#{mode}.foo" + File.should_not be_exist(new_target) + + begin + subject.replace_file(new_target, mode) {|fh| fh.puts "this is an interesting content" } + + get_mode(new_target).should == 0664 + ensure + File.unlink(new_target) if File.exists?(new_target) + end + end + end + end describe "#pretty_backtrace" do |