Repository: rodjek/librarian-puppet
Branch: master
Commit: cfe75c5d18d2
Files: 84
Total size: 133.6 KB
Directory structure:
gitextract_bq1yps9j/
├── .gitignore
├── .travis.yml
├── CONTRIBUTING.md
├── Changelog.md
├── Gemfile
├── LICENSE
├── README.md
├── Rakefile
├── bin/
│ └── librarian-puppet
├── features/
│ ├── examples/
│ │ ├── dependency_without_version/
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ ├── duplicated_dependencies/
│ │ │ ├── Puppetfile
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ ├── duplicated_dependencies_transitive/
│ │ │ ├── Puppetfile
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ ├── metadata_syntax/
│ │ │ ├── Puppetfile
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ ├── modulefile_syntax/
│ │ │ ├── Modulefile
│ │ │ ├── Puppetfile
│ │ │ └── manifests/
│ │ │ └── init.pp
│ │ ├── path_dependencies/
│ │ │ ├── Puppetfile
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ ├── test/
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ ├── with_puppetfile/
│ │ │ ├── Puppetfile
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ ├── with_puppetfile_and_metadata_json/
│ │ │ ├── Puppetfile
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ └── with_puppetfile_and_modulefile/
│ │ ├── Modulefile
│ │ ├── Puppetfile
│ │ └── manifests/
│ │ └── init.pp
│ ├── help.feature
│ ├── init.feature
│ ├── install/
│ │ ├── forge.feature
│ │ ├── git.feature
│ │ ├── github_tarball.feature
│ │ └── path.feature
│ ├── install.feature
│ ├── outdated.feature
│ ├── package.feature
│ ├── step_definitions/
│ │ └── convergence_steps.rb
│ ├── support/
│ │ └── env.rb
│ ├── update.feature
│ └── version.feature
├── lib/
│ └── librarian/
│ ├── puppet/
│ │ ├── action/
│ │ │ ├── install.rb
│ │ │ └── resolve.rb
│ │ ├── action.rb
│ │ ├── cli.rb
│ │ ├── dependency.rb
│ │ ├── dsl.rb
│ │ ├── environment.rb
│ │ ├── extension.rb
│ │ ├── lockfile.rb
│ │ ├── source/
│ │ │ ├── forge/
│ │ │ │ ├── repo.rb
│ │ │ │ ├── repo_v1.rb
│ │ │ │ └── repo_v3.rb
│ │ │ ├── forge.rb
│ │ │ ├── git.rb
│ │ │ ├── githubtarball/
│ │ │ │ └── repo.rb
│ │ │ ├── githubtarball.rb
│ │ │ ├── local.rb
│ │ │ ├── path.rb
│ │ │ └── repo.rb
│ │ ├── source.rb
│ │ ├── templates/
│ │ │ └── Puppetfile
│ │ ├── util.rb
│ │ └── version.rb
│ └── puppet.rb
├── librarian-puppet.gemspec
├── spec/
│ ├── action/
│ │ └── resolve_spec.rb
│ ├── librarian_puppet_spec.rb
│ ├── receiver_spec.rb
│ ├── source/
│ │ ├── forge_repo_spec.rb
│ │ └── forge_spec.rb
│ ├── spec_helper.rb
│ └── util_spec.rb
└── test/
├── librarian/
│ └── puppet/
│ └── source/
│ └── githubtarball_test.rb
└── test_helper.rb
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
pkg/
Gemfile.lock
tmp/
coverage/
================================================
FILE: .travis.yml
================================================
sudo: false
rvm:
- '2.2'
- '2.1'
- '2.0'
- '1.9'
notifications:
email:
- carlos@apache.org
- tim@github.com
env:
- PUPPET_VERSION="~> 4.2.0"
- PUPPET_VERSION="~> 4.1.0"
- PUPPET_VERSION="~> 4.0.0"
- PUPPET_VERSION="~> 3.7.0"
- PUPPET_VERSION="~> 3.6.0"
- PUPPET_VERSION="~> 3.5.0"
- PUPPET_VERSION="~> 3.4.0"
- PUPPET_VERSION="~> 3.3.0"
- PUPPET_VERSION="~> 3.2.0"
- PUPPET_VERSION="~> 3.1.0"
- PUPPET_VERSION="~> 3.0.0"
- PUPPET_VERSION="~> 2.7.0"
before_script:
- puppet --version
matrix:
# Puppet < 3.5.0 is broken under ruby 2.1 https://tickets.puppetlabs.com/browse/PUP-1243
# Puppet < 3.8.0 is broken under ruby 2.2 https://tickets.puppetlabs.com/browse/PUP-3796
exclude:
- rvm: '2.1'
env: PUPPET_VERSION="~> 2.7.0"
- rvm: '2.1'
env: PUPPET_VERSION="~> 3.0.0"
- rvm: '2.1'
env: PUPPET_VERSION="~> 3.1.0"
- rvm: '2.1'
env: PUPPET_VERSION="~> 3.2.0"
- rvm: '2.1'
env: PUPPET_VERSION="~> 3.3.0"
- rvm: '2.1'
env: PUPPET_VERSION="~> 3.4.0"
- rvm: '2.2'
env: PUPPET_VERSION="~> 2.7.0"
- rvm: '2.2'
env: PUPPET_VERSION="~> 3.0.0"
- rvm: '2.2'
env: PUPPET_VERSION="~> 3.1.0"
- rvm: '2.2'
env: PUPPET_VERSION="~> 3.2.0"
- rvm: '2.2'
env: PUPPET_VERSION="~> 3.3.0"
- rvm: '2.2'
env: PUPPET_VERSION="~> 3.4.0"
- rvm: '2.2'
env: PUPPET_VERSION="~> 3.5.0"
- rvm: '2.2'
env: PUPPET_VERSION="~> 3.6.0"
- rvm: '2.2'
env: PUPPET_VERSION="~> 3.7.0"
================================================
FILE: CONTRIBUTING.md
================================================
# Librarian-puppet
# NOTE this project has moved to https://github.com/voxpupuli/librarian-puppet
================================================
FILE: Changelog.md
================================================
# Changelog
## From 2.x Librarian-Puppet requires Ruby >= 1.9, uses Puppet Forge API v3. For Ruby 1.8 use 1.x
### 2.2.1
* [Issue #311](https://github.com/rodjek/librarian-puppet/issues/311) Omit versions with a deleted_at date
### 2.2.0
* Add support for Puppet 4
* [Issue #296](https://github.com/rodjek/librarian-puppet/issues/296) Uninitialized constant Puppet::ModuleTool::ModulefileReader using Modulefiles in Puppet 4. Ignore those dependencies
### 2.1.1
* [Issue #302](https://github.com/rodjek/librarian-puppet/issues/302) Ensure path is not lost when default specfile is used
* [Issue #294](https://github.com/rodjek/librarian-puppet/issues/294) Undefined variable calling Puppet version in old Puppet 2.x versions
* [Issue #285](https://github.com/rodjek/librarian-puppet/issues/294) Update librarianp to allow overriding dependencies from multiple sources
### 2.1.0
* Update librarian to use the new `exclusion` syntax
* [Issue #282](https://github.com/rodjek/librarian-puppet/issues/282) Merge duplicated dependencies and warn the user, no more `Cannot bounce Puppetfile.lock!` errors
* [Issue #217](https://github.com/rodjek/librarian-puppet/issues/217)[Issue #244](https://github.com/rodjek/librarian-puppet/issues/244) Use librarianp 0.4.0 that no longer uses recursion to avoid `stack level too deep` errors
* [Issue #277](https://github.com/rodjek/librarian-puppet/issues/277) Warn when there are two dependencies with the same module name
* Use `librarianp` gem instead of `librarian`, a fork with the needed improvements and fixes.
### 2.0.1
* [Issue #272](https://github.com/rodjek/librarian-puppet/issues/272) Defined forge is not used when resolving dependencies
* [Issue #150](https://github.com/rodjek/librarian-puppet/issues/150) Allow dependencies other than Puppet modules
* [Issue #269](https://github.com/rodjek/librarian-puppet/issues/269) Better error message if metadata.json is bad
* [Issue #264](https://github.com/rodjek/librarian-puppet/issues/264) Copying files can cause permission problems on Windows
### 2.0.0
* Jump from 1.3.x to 2.x to leave 1.x for Ruby 1.8 compatibility
* [Issue #254](https://github.com/rodjek/librarian-puppet/issues/254) Add a rsync option to prevent deleting directories
* [Issue #261](https://github.com/rodjek/librarian-puppet/issues/261) Incorrect install directory is created if the organization name contains a dash
* [Issue #255](https://github.com/rodjek/librarian-puppet/issues/255) Ignored forge URL when using API v3
### 1.5.0
* Update librarian to use the new `exclusion` syntax
* [Issue #282](https://github.com/rodjek/librarian-puppet/issues/282) Merge duplicated dependencies and warn the user, no more `Cannot bounce Puppetfile.lock!` errors
* [Issue #217](https://github.com/rodjek/librarian-puppet/issues/217)[Issue #244](https://github.com/rodjek/librarian-puppet/issues/244) Use librarianp 0.4.0 that no longer uses recursion to avoid `stack level too deep` errors
* [Issue #277](https://github.com/rodjek/librarian-puppet/issues/277) Warn when there are two dependencies with the same module name
* Use `librarianp` gem instead of `librarian`, a fork with the needed improvements and fixes.
### 1.4.1
* [Issue #272](https://github.com/rodjek/librarian-puppet/issues/272) Defined forge is not used when resolving dependencies
* [Issue #150](https://github.com/rodjek/librarian-puppet/issues/150) Allow dependencies other than Puppet modules
* [Issue #269](https://github.com/rodjek/librarian-puppet/issues/269) Better error message if metadata.json is bad
* [Issue #264](https://github.com/rodjek/librarian-puppet/issues/264) Copying files can cause permission problems on Windows
### 1.4.0
* Jump from 1.0.x to 1.4.x to keep Ruby 1.8 compatibility in the 1.x series
* [Issue #254](https://github.com/rodjek/librarian-puppet/issues/254) Add a rsync option to prevent deleting directories
* [Issue #261](https://github.com/rodjek/librarian-puppet/issues/261) Incorrect install directory is created if the organization name contains a dash
### 1.3.3
* [Issue #250](https://github.com/rodjek/librarian-puppet/issues/250) Fix error when module has no dependencies in `metadata.json`
### 1.3.2
* [Issue #246](https://github.com/rodjek/librarian-puppet/issues/246) Do not fail if modules have no `Modulefile` nor `metadata.json`
### 1.3.1
* Version in dependencies with `metadata.json` is ignored
### 1.3.0
* If no Puppetfile is present default to use the `metadata.json` or `Modulefile`
* [Issue #235](https://github.com/rodjek/librarian-puppet/issues/235) Error when forge is not defined in `Puppetfile`
* [Issue #243](https://github.com/rodjek/librarian-puppet/issues/243) Warn if `Modulefile` doesn't contain a version
### 1.2.0
* Implement `metadata` syntax for `Puppetfile`
* [Issue #220](https://github.com/rodjek/librarian-puppet/issues/220) Add support for metadata.json
* [Issue #242](https://github.com/rodjek/librarian-puppet/issues/242) Get organization from name correctly if name has multiple dashes
### 1.1.3
* [Issue #237](https://github.com/rodjek/librarian-puppet/issues/237) [Issue #238](https://github.com/rodjek/librarian-puppet/issues/238) Unable to use a custom v3 forge: add flags `--use-v1-api` and `--no-use-v1-api`
* [Issue #239](https://github.com/rodjek/librarian-puppet/issues/239) GitHub tarball: add access_token correctly to url's which are already having query parameters
* [Issue #234](https://github.com/rodjek/librarian-puppet/issues/234) Use organization-module instead of organization/module by default
### 1.1.2
* [Issue #231](https://github.com/rodjek/librarian-puppet/issues/231) Only use the `GITHUB_API_TOKEN` if it's not empty
* [Issue #233](https://github.com/rodjek/librarian-puppet/issues/233) Fix version regex to match e.g. 1.99.15
* Can't pass the Puppet Forge v1 api url to clients using v3 (3.6.0+ and PE 3.2.0+)
### 1.1.1
* [Issue #227](https://github.com/rodjek/librarian-puppet/issues/227) Fix Librarian::Puppet::VERSION undefined
### 1.1.0
* [Issue #210](https://github.com/rodjek/librarian-puppet/issues/210) Use forgeapi.puppetlabs.com and API v3
* Accesing the v3 API requires Ruby 1.9 due to the puppet_forge library used
### 1.0.10
* [Issue #250](https://github.com/rodjek/librarian-puppet/issues/250) Fix error when module has no dependencies in `metadata.json`
### 1.0.9
* [Issue #246](https://github.com/rodjek/librarian-puppet/issues/246) Do not fail if modules have no `Modulefile` nor `metadata.json`
### 1.0.8
* Version in dependencies with `metadata.json` is ignored
### 1.0.7
* If no Puppetfile is present default to use the `metadata.json` or `Modulefile`
* [Issue #235](https://github.com/rodjek/librarian-puppet/issues/235) Error when forge is not defined in `Puppetfile`
* [Issue #243](https://github.com/rodjek/librarian-puppet/issues/243) Warn if `Modulefile` doesn't contain a version
### 1.0.6
* Implement `metadata` syntax for `Puppetfile`
* [Issue #220](https://github.com/rodjek/librarian-puppet/issues/220) Add support for metadata.json
* [Issue #242](https://github.com/rodjek/librarian-puppet/issues/242) Get organization from name correctly if name has multiple dashes
### 1.0.5
* [Issue #237](https://github.com/rodjek/librarian-puppet/issues/237)[Issue #238](https://github.com/rodjek/librarian-puppet/issues/238) Unable to use a custom v3 forge: add flags `--use-v1-api` and `--no-use-v1-api`
* [Issue #239](https://github.com/rodjek/librarian-puppet/issues/239) GitHub tarball: add access_token correctly to url's which are already having query parameters
* [Issue #234](https://github.com/rodjek/librarian-puppet/issues/234) Use organization-module instead of organization/module by default
### 1.0.4
* [Issue #231](https://github.com/rodjek/librarian-puppet/issues/231) Only use the `GITHUB_API_TOKEN` if it's not empty
* [Issue #233](https://github.com/rodjek/librarian-puppet/issues/233) Fix version regex to match e.g. 1.99.15
* Can't pass the Puppet Forge v1 api url to clients using v3 (3.6.0+ and PE 3.2.0+)
### 1.0.3
* [Issue #223](https://github.com/rodjek/librarian-puppet/issues/223) `Cannot bounce Puppetfile.lock!` error when Forge modules contain duplicated dependencies
### 1.0.2
* [Issue #211](https://github.com/rodjek/librarian-puppet/issues/211) Pass the PuppetLabs Forge API v3 endpoint to `puppet module` when running on Puppet >= 3.6.0
* [Issue #198](https://github.com/rodjek/librarian-puppet/issues/198) Reduce the length of tmp dirs to avoid issues in windows
* [Issue #206](https://github.com/rodjek/librarian-puppet/issues/206) githubtarball call for released versions does not consider pagination
* [Issue #204](https://github.com/rodjek/librarian-puppet/issues/204) Fix regex to detect Forge API v3 url
* [Issue #199](https://github.com/rodjek/librarian-puppet/issues/199) undefined method run! packaging a git source
* Verify SSL certificates in github calls
### 1.0.1
* [Issue #190](https://github.com/rodjek/librarian-puppet/issues/190) Pass the PuppetLabs Forge API v3 endpoint to `puppet module` when running on Puppet Enterprise >= 3.2
* [Issue #196](https://github.com/rodjek/librarian-puppet/issues/196) Fix error in error handling when puppet is not installed
### 1.0.0
* Remove deprecation warning for github_tarball sources, some people are actually using it
### 0.9.17
* [Issue #193](https://github.com/rodjek/librarian-puppet/issues/193) Support Puppet 3.5.0
### 0.9.16
* [Issue #181](https://github.com/rodjek/librarian-puppet/issues/181) Should use qualified module names for resolution to work correctly
* Deprecate github_tarball sources
* Reduce number of API calls for github_tarball sources
### 0.9.15
* [Issue #187](https://github.com/rodjek/librarian-puppet/issues/187) Fixed parallel installation issues
* [Issue #185](https://github.com/rodjek/librarian-puppet/issues/185) Sanitize the gem/bundler environment before spawning (ruby 1.9+)
### 0.9.14
* [Issue #182](https://github.com/rodjek/librarian-puppet/issues/182) Sanitize the environment before spawning (ruby 1.9+)
* [Issue #184](https://github.com/rodjek/librarian-puppet/issues/184) Support transitive dependencies in modules using :path
* Git dependencies using modulefile syntax make librarian-puppet fail
* [Issue #108](https://github.com/rodjek/librarian-puppet/issues/108) Don't fail on malformed Modulefile from a git dependency
### 0.9.13
* [Issue #176](https://github.com/rodjek/librarian-puppet/issues/176) Upgrade to librarian 0.1.2
* [Issue #179](https://github.com/rodjek/librarian-puppet/issues/179) Need to install extra gems just in case we are in ruby 1.8
* [Issue #178](https://github.com/rodjek/librarian-puppet/issues/178) Print a meaningful message if puppet gem can't be loaded for :git sources
### 0.9.12
* Remove extra dependencies from gem added when 0.9.11 was released under ruby 1.8
### 0.9.11
* Add modulefile dsl to reuse Modulefile dependencies
* Consider Puppetfile-dependencies recursively in git-source
* Support changing tmp, cache and scratch paths
* librarian-puppet package causes an infinite loop
* Show a message if no versions are found for a module
* Make download of tarballs more robust
* Require open3_backport in ruby 1.8 and install if not present
* Git dependencies in both Puppetfile and Modulefile cause a Cannot bounce Puppetfile.lock! error
* Better sort of github tarball versions when there are mixed tags starting with and without 'v'
* Fix error if a git module has a dependency without version
* Fix git dependency with :path attribute
* Cleaner output when no Puppetfile found
* Reduce the number of API calls to the Forge
* Don't sort versions as strings. Rely on the forge returning them ordered
* Pass --module_repository to `puppet module install` to install from other forges
* Cache forge responses and print an error if returns an invalid response
* Add a User-Agent header to all requests to the GitHub API
* Convert puppet version requirements to rubygems, pessimistic and ranges
* Use librarian gem
### 0.9.10
* Catch GitHub API rate limit exceeded
* Make Librarian::Manifest Semver 2.0.0 compatible
### 0.9.1
* Proper error message when a module that is sourced from the forge does not
exist.
* Added support for annotated tags as git references.
* `librarian-puppet init` adds `.tmp/` to gitignore instead of `tmp/`.
* Fixed syntax error in the template Puppetfile created by `librarian-puppet
init`.
* Checks for `lib/puppet` as well as `manifests/` when checking if the git
repository is a valid module.
* When a user specifies `<foo>/<bar>` as the name of a module sources from a
git repository, assume the module name is actually `<bar>`.
* Fixed gem description and summary in gemspec.
### 0.9.0
* Initial release
================================================
FILE: Gemfile
================================================
source 'https://rubygems.org'
gemspec
================================================
FILE: LICENSE
================================================
Copyright (c) 2012-2014 Tim Sharpe, Carlos Sanchez and others
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Software, and to
permit persons to whom the Software is furnished to do so, subject to
the following conditions:
The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
================================================
FILE: README.md
================================================
# Librarian-puppet
# NOTE This project has moved to https://github.com/voxpupuli/librarian-puppet
[](https://travis-ci.org/rodjek/librarian-puppet)
## Introduction
Librarian-puppet is a bundler for your puppet infrastructure. You can use
librarian-puppet to manage the puppet modules your infrastructure depends on,
whether the modules come from the [Puppet Forge](https://forge.puppetlabs.com/),
Git repositories or just a path.
* Librarian-puppet can reuse the dependencies listed in your `Modulefile` or `metadata.json`
* Forge modules can be installed from [Puppetlabs Forge](https://forge.puppetlabs.com/) or an internal Forge such as [Pulp](http://www.pulpproject.org/)
* Git modules can be installed from a branch, tag or specific commit, optionally using a path inside the repository
* Modules can be installed from GitHub using tarballs, without needing Git installed
* Modules can be installed from a filesystem path
* Module dependencies are resolved transitively without needing to list all the modules explicitly
Librarian-puppet manages your `modules/` directory for you based on your
`Puppetfile`. Your `Puppetfile` becomes the authoritative source for what
modules you require and at what version, tag or branch.
Once using Librarian-puppet you should not modify the contents of your `modules`
directory. The individual modules' repos should be updated, tagged with a new
release and the version bumped in your Puppetfile.
It is based on [Librarian](https://github.com/applicationsonline/librarian), a
framework for writing bundlers, which are tools that resolve, fetch, install,
and isolate a project's dependencies.
## Versions
Librarian-puppet >= 2.0 (as well as 1.1, 1.2 and 1.3) requires Ruby 1.9 and uses the Puppet Forge API v3.
Versions < 2.0 work on Ruby 1.8.
See the [Changelog](Changelog.md) for more details.
## The Puppetfile
Every Puppet repository that uses Librarian-puppet may have a file named
`Puppetfile`, `metadata.json` or `Modulefile` in the root directory of that repository.
The full specification
for which modules your puppet infrastructure repository depends goes in here.
### Simple usage
If no Puppetfile is present, `librarian-puppet` will download all the dependencies
listed in your `metadata.json` or `Modulefile` from the Puppet Forge,
as if the Puppetfile contained
forge "https://forgeapi.puppetlabs.com"
metadata
### Example Puppetfile
forge "https://forgeapi.puppetlabs.com"
mod 'puppetlabs-razor'
mod 'puppetlabs-ntp', "0.0.3"
mod 'puppetlabs-apt',
:git => "git://github.com/puppetlabs/puppetlabs-apt.git"
mod 'puppetlabs-stdlib',
:git => "git://github.com/puppetlabs/puppetlabs-stdlib.git"
mod 'puppetlabs-apache', '0.6.0',
:github_tarball => 'puppetlabs/puppetlabs-apache'
mod 'acme-mymodule', :path => './some_folder'
exclusion 'acme-bad_module'
### Recursive module dependency resolution
When fetching a module all dependencies specified in its
`Modulefile`, `metadata.json` and `Puppetfile` will be resolved and installed.
### Puppetfile Breakdown
forge "https://forgeapi.puppetlabs.com"
This declares that we want to use the official Puppet Labs Forge as our default
source when pulling down modules. If you run your own local forge, you may
want to change this.
metadata
Download all the dependencies listed in your `metadata.json` or `Modulefile` from the Puppet Forge.
mod 'puppetlabs-razor'
Pull in the latest version of the Puppet Labs Razor module from the default
source.
mod 'puppetlabs-ntp', "0.0.3"
Pull in version 0.0.3 of the Puppet Labs NTP module from the default source.
mod 'puppetlabs-apt',
:git => "git://github.com/puppetlabs/puppetlabs-apt.git"
Our puppet infrastructure repository depends on the `apt` module from the
Puppet Labs GitHub repos and checks out the `master` branch.
mod 'puppetlabs-apt',
:git => "git://github.com/puppetlabs/puppetlabs-apt.git",
:ref => '0.0.3'
Our puppet infrastructure repository depends on the `apt` module from the
Puppet Labs GitHub repos and checks out a tag of `0.0.3`.
mod 'puppetlabs-apt',
:git => "git://github.com/puppetlabs/puppetlabs-apt.git",
:ref => 'feature/master/dans_refactor'
Our puppet infrastructure repository depends on the `apt` module from the
Puppet Labs GitHub repos and checks out the `dans_refactor` branch.
When using a Git source, we do not have to use a `:ref =>`.
If we do not, then librarian-puppet will assume we meant the `master` branch.
If we use a `:ref =>`, we can use anything that Git will recognize as a ref.
This includes any branch name, tag name, SHA, or SHA unique prefix. If we use a
branch, we can later ask Librarian-puppet to update the module by fetching the
most recent version of the module from that same branch.
The Git source also supports a `:path =>` option. If we use the path option,
Librarian-puppet will navigate down into the Git repository and only use the
specified subdirectory. Some people have the habit of having a single repository
with many modules in it. If we need a module from such a repository, we can
use the `:path =>` option here to help Librarian-puppet drill down and find the
module subdirectory.
mod 'puppetlabs-apt',
:git => "git://github.com/fake/puppet-modules.git",
:path => "modules/apt"
Our puppet infrastructure repository depends on the `apt` module, which we have
stored as a directory under our `puppet-modules` git repos.
mod 'puppetlabs-apache', '0.6.0',
:github_tarball => 'puppetlabs/puppetlabs-apache'
Our puppet infrastructure repository depends on the `puppetlabs-apache` module,
to be downloaded from GitHub tarball.
mod 'acme-mymodule', :path => './some_folder'
Our puppet infrastructure repository depends on the `acme-mymodule` module,
which is already in the filesystem.
exclusion 'acme-bad_module'
Exclude the module `acme-bad_module` from resolution and installation.
## How to Use
Install librarian-puppet:
$ gem install librarian-puppet
Prepare your puppet infrastructure repository:
$ cd ~/path/to/puppet-inf-repos
$ (git) rm -rf modules
$ librarian-puppet init
Librarian-puppet takes over your `modules/` directory, and will always
reinstall (if missing) the modules listed the `Puppetfile.lock` into your
`modules/` directory, therefore you do not need your `modules/` directory to be
tracked in Git.
Librarian-puppet uses a `.tmp/` directory for tempfiles and caches. You should
not track this directory in Git.
Running `librarian-puppet init` will create a skeleton Puppetfile for you as
well as adding `tmp/` and `modules/` to your `.gitignore`.
$ librarian-puppet install [--clean] [--verbose]
This command looks at each `mod` declaration and fetches the module from the
source specified. This command writes the complete resolution into
`Puppetfile.lock` and then copies all of the fetched modules into your
`modules/` directory, overwriting whatever was there before.
Librarian-puppet support both v1 and v3 of the Puppet Forge API.
Specify a specific API version when installing modules:
$ librarian-puppet install --use-v1-api # this is default; ignored for official Puppet Forge
$ librarian-puppet install --no-use-v1-api # use the v3 API; default for official Puppet Forge
Please note that this does not apply for the official Puppet Forge, where v3 is used by default.
Get an overview of your `Puppetfile.lock` with:
$ librarian-puppet show
Inspect the details of specific resolved dependencies with:
$ librarian-puppet show NAME1 [NAME2, ...]
Find out which dependencies are outdated and may be updated:
$ librarian-puppet outdated [--verbose]
Update the version of a dependency:
$ librarian-puppet update apt [--verbose]
$ git diff Puppetfile.lock
$ git add Puppetfile.lock
$ git commit -m "bumped the version of apt up to 0.0.4."
## Configuration
Configuration comes from three sources with the following highest-to-lowest
precedence:
* The local config (`./.librarian/puppet/config`)
* The environment
* The global config (`~/.librarian/puppet/config`)
You can inspect the final configuration with:
$ librarian-puppet config
You can find out where a particular key is set with:
$ librarian-puppet config KEY
You can set a key at the global level with:
$ librarian-puppet config KEY VALUE --global
And remove it with:
$ librarian-puppet config KEY --global --delete
You can set a key at the local level with:
$ librarian-puppet config KEY VALUE --local
And remove it with:
$ librarian-puppet config KEY --local --delete
You cannot set or delete environment-level config keys with the CLI.
Configuration set at either the global or local level will affect subsequent
invocations of `librarian-puppet`. Configurations set at the environment level are
not saved and will not affect subsequent invocations of `librarian-puppet`.
You can pass a config at the environment level by taking the original config key
and transforming it: replace hyphens (`-`) with underscores (`_`) and periods
(`.`) with doubled underscores (`__`), uppercase, and finally prefix with
`LIBRARIAN_PUPPET_`. For example, to pass a config in the environment for the key
`part-one.part-two`, set the environment variable
`LIBRARIAN_PUPPET_PART_ONE__PART_TWO`.
Configuration affects how various commands operate.
* The `path` config sets the directory to install to. If a relative
path, it is relative to the directory containing the `Puppetfile`. The
equivalent environment variable is `LIBRARIAN_PUPPET_PATH`.
* The `tmp` config sets the cache directory for librarian. If a relative
path, it is relative to the directory containing the `Puppetfile`. The
equivalent environment variable is `LIBRARIAN_PUPPET_TMP`.
Configuration can be set by passing specific options to other commands.
* The `path` config can be set at the local level by passing the `--path` option
to the `install` command. It can be unset at the local level by passing the
`--no-path` option to the `install` command. Note that if this is set at the
environment or global level then, even if `--no-path` is given as an option,
the environment or global config will be used.
## Rsync Option
The default convergence strategy between the cache and the module directory is
to execute an `rm -r` on the module directory and just `cp -r` from the cache.
This causes the module to be removed from the module path every time librarian
puppet updates, regardless of whether the content has changed. This can cause
some problems in environments with lots of change. The problem arises when the
module directory gets removed while Puppet is trying to read files inside it.
The `puppet master` process will lose its CWD and the catalog will fail to
compile. To avoid this, you can use `rsync` to implement a more conservative
convergence strategy. This will use `rsync` with the `-avz` and `--delete`
flags instead of a `rm -r` and `cp -r`. To use this feature, just set the
`rsync` configuration setting to `true`.
$ librarian-puppet config rsync true --global
Alternatively, using an environment variable:
LIBRARIAN_PUPPET_RSYNC='true'
Note that the directories will still be purged if you run librarian-puppet with
the --clean or --destructive flags.
## How to Contribute
* Pull requests please.
* Bonus points for feature branches.
## Reporting Issues
Bug reports to the github issue tracker please.
Please include:
* Relevant `Puppetfile` and `Puppetfile.lock` files
* Version of ruby, librarian-puppet, and puppet
* What distro
* Please run the `librarian-puppet` commands in verbose mode by using the
`--verbose` flag, and include the verbose output in the bug report as well.
## License
Please see the [LICENSE](https://github.com/rodjek/librarian-puppet/blob/master/LICENSE)
file.
================================================
FILE: Rakefile
================================================
require 'bundler/setup'
require 'cucumber/rake/task'
require 'rspec/core/rake_task'
require 'bundler/gem_tasks'
require 'rake/testtask'
require 'rake/clean'
CLEAN.include('pkg/', 'tmp/')
CLOBBER.include('Gemfile.lock')
RSpec::Core::RakeTask.new
Cucumber::Rake::Task.new(:features) do |t|
require 'puppet'
puppet_version = Puppet::version.gsub("~>","").split(".").first.to_i
tags = (2..4).select {|i| i != puppet_version}.map{|i| "--tags @puppet#{puppet_version},~@puppet#{i}"}
# don't run githubtarball scenarios in Travis, they easily fail with rate limit exceeded
tags << "--tags ~@github" if ENV['TRAVIS']=='true'
t.cucumber_opts = tags.join(" ")
end
Rake::TestTask.new do |test|
test.pattern = 'test/**/*_test.rb'
test.verbose = true
end
task :default => [:test, :spec, :features]
desc "Bump version to the next minor"
task :bump do
path = 'lib/librarian/puppet/version.rb'
version_file = File.read(path)
version = version_file.match(/VERSION = "(.*)"/)[1]
v = Gem::Version.new("#{version}.0")
new_version = v.bump.to_s
version_file = version_file.gsub(/VERSION = ".*"/, "VERSION = \"#{new_version}\"")
File.open(path, "w") {|file| file.puts version_file}
sh "git add #{path}"
sh "git commit -m \"Bump version to #{new_version}\""
end
================================================
FILE: bin/librarian-puppet
================================================
#!/usr/bin/env ruby
lib = File.expand_path('../../lib', __FILE__)
$:.unshift(lib) unless $:.include?(lib)
require 'librarian/puppet/cli'
Librarian::Puppet::Cli.bin!
================================================
FILE: features/examples/dependency_without_version/manifests/init.pp
================================================
class test {}
================================================
FILE: features/examples/dependency_without_version/metadata.json
================================================
{
"name": "librarian-test",
"version": "0.0.1",
"license": "Apache 2.0",
"dependencies": [
{
"name": "puppetlabs/stdlib"
}
]
}
================================================
FILE: features/examples/duplicated_dependencies/Puppetfile
================================================
forge 'http://forge.puppetlabs.com'
metadata
================================================
FILE: features/examples/duplicated_dependencies/manifests/init.pp
================================================
class test {}
================================================
FILE: features/examples/duplicated_dependencies/metadata.json
================================================
{
"name": "librarian-duplicated_dependencies",
"version": "0.0.1",
"license": "Apache 2.0",
"dependencies": [
{
"name": "ripienaar-concat",
"version_requirement": ">= 0"
},
{
"name": "puppetlabs-concat",
"version_requirement": "1.2.0"
}
]
}
================================================
FILE: features/examples/duplicated_dependencies_transitive/Puppetfile
================================================
forge 'http://forge.puppetlabs.com'
metadata
mod 'librarian-duplicated_dependencies', :git => 'https://github.com/rodjek/librarian-puppet.git', :path => 'features/examples/duplicated_dependencies'
================================================
FILE: features/examples/duplicated_dependencies_transitive/manifests/init.pp
================================================
class test {}
================================================
FILE: features/examples/duplicated_dependencies_transitive/metadata.json
================================================
{
"name": "librarian-duplicated_dependencies_transitive",
"version": "0.0.1",
"license": "Apache 2.0",
"dependencies": []
}
================================================
FILE: features/examples/metadata_syntax/Puppetfile
================================================
forge 'http://forge.puppetlabs.com'
metadata
================================================
FILE: features/examples/metadata_syntax/manifests/init.pp
================================================
class test {}
================================================
FILE: features/examples/metadata_syntax/metadata.json
================================================
{
"operatingsystem_support": [
{
"operatingsystem": "RedHat",
"operatingsystemrelease": [
"4",
"5",
"6"
]
},
{
"operatingsystem": "CentOS",
"operatingsystemrelease": [
"4",
"5",
"6"
]
},
{
"operatingsystem": "OracleLinux",
"operatingsystemrelease": [
"4",
"5",
"6"
]
},
{
"operatingsystem": "Scientific",
"operatingsystemrelease": [
"4",
"5",
"6"
]
},
{
"operatingsystem": "SLES",
"operatingsystemrelease": [
"11 SP1"
]
},
{
"operatingsystem": "Debian",
"operatingsystemrelease": [
"6",
"7"
]
},
{
"operatingsystem": "Ubuntu",
"operatingsystemrelease": [
"10.04",
"12.04"
]
},
{
"operatingsystem": "Solaris",
"operatingsystemrelease": [
"10",
"11"
]
},
{
"operatingsystem": "Windows",
"operatingsystemrelease": [
"Server 2003",
"Server 2003 R2",
"Server 2008",
"Server 2008 R2",
"Server 2012",
"Server 2012 R2",
"7",
"8"
]
},
{
"operatingsystem": "AIX",
"operatingsystemrelease": [
"5.3",
"6.1",
"7.1"
]
}
],
"requirements": [
{
"name": "pe",
"version_requirement": "3.2.x"
},
{
"name": "puppet",
"version_requirement": ">=2.7.20 <4.0.0"
}
],
"name": "librarian-metadata_syntax",
"version": "0.0.1",
"license": "Apache 2.0",
"dependencies": [
{
"name": "maestrodev/test",
"version_requirement": ">= 0.0.1"
}
]
}
================================================
FILE: features/examples/modulefile_syntax/Modulefile
================================================
name 'librarian-modulefile_syntax'
version '0.0.1'
author 'librarian'
license 'Apache License, Version 2.0'
dependency 'maestrodev/test'
================================================
FILE: features/examples/modulefile_syntax/Puppetfile
================================================
forge 'http://forge.puppetlabs.com'
modulefile
================================================
FILE: features/examples/modulefile_syntax/manifests/init.pp
================================================
class test {}
================================================
FILE: features/examples/path_dependencies/Puppetfile
================================================
mod 'librarian/test', :path => '../../features/examples/test'
================================================
FILE: features/examples/path_dependencies/manifests/init.pp
================================================
class test {}
================================================
FILE: features/examples/path_dependencies/metadata.json
================================================
{
"name": "librarian-path_dependencies",
"version": "0.0.1",
"license": "Apache 2.0",
"dependencies": [],
"requirements": []
}
================================================
FILE: features/examples/test/manifests/init.pp
================================================
class test {}
================================================
FILE: features/examples/test/metadata.json
================================================
{
"name": "librarian-test",
"version": "0.0.1",
"license": "Apache 2.0",
"dependencies": [
{
"name": "puppetlabs/stdlib",
"version_requirement": ">= 0"
}
]
}
================================================
FILE: features/examples/with_puppetfile/Puppetfile
================================================
mod 'librarian/test', :git => 'https://github.com/rodjek/librarian-puppet.git', :path => 'features/examples/test'
================================================
FILE: features/examples/with_puppetfile/manifests/init.pp
================================================
class test {}
================================================
FILE: features/examples/with_puppetfile/metadata.json
================================================
{
"name": "librarian-with_puppetfile",
"version": "0.0.1",
"license": "Apache 2.0",
"dependencies": []
}
================================================
FILE: features/examples/with_puppetfile_and_metadata_json/Puppetfile
================================================
forge 'http://forge.puppetlabs.com'
mod 'maestrodev/test'
================================================
FILE: features/examples/with_puppetfile_and_metadata_json/manifests/init.pp
================================================
class test {}
================================================
FILE: features/examples/with_puppetfile_and_metadata_json/metadata.json
================================================
{
"name": "librarian-with_puppetfile_and_metadata_json",
"version": "0.0.1",
"license": "Apache 2.0",
"dependencies": [
{
"name": "maestrodev/test",
"version_requirement": ">= 0"
}
]
}
================================================
FILE: features/examples/with_puppetfile_and_modulefile/Modulefile
================================================
name 'librarian-with_puppetfile_and_modulefile'
version '0.0.1'
author 'librarian'
license 'Apache License, Version 2.0'
dependency 'maestrodev/test'
================================================
FILE: features/examples/with_puppetfile_and_modulefile/Puppetfile
================================================
forge 'http://forge.puppetlabs.com'
mod 'maestrodev/test'
================================================
FILE: features/examples/with_puppetfile_and_modulefile/manifests/init.pp
================================================
class test {}
================================================
FILE: features/help.feature
================================================
Feature: displays help if no subcommand is passed
In order to get started using librarian-puppet
A user should be able to run librarian-puppet without any subcommands or options
Then the exit status should be 0
And a useful help screen should be displayed
Scenario: App defaults to help subcommand
When I run `librarian-puppet`
Then the exit status should be 0
And the output should contain "librarian-puppet version"
================================================
FILE: features/init.feature
================================================
Feature: init subcommand should generate a Puppetfile
In order to start using librarian-puppet in a project
A project will need a Puppetfile.
If a user runs "librarian-puppet init"
Then the exit status should be 0
And a file named "Puppetfile" should exist
Scenario: init subcommand should generate a Puppetfile
When I run `librarian-puppet init`
Then the exit status should be 0
Then a file named "Puppetfile" should exist
================================================
FILE: features/install/forge.feature
================================================
Feature: cli/install/forge
Puppet librarian needs to install modules from the Puppet Forge
Scenario: Installing a module and its dependencies
Given a file named "Puppetfile" with:
"""
forge "https://forgeapi.puppetlabs.com"
mod 'puppetlabs/ntp'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/ntp/metadata.json" should match /"name": "puppetlabs-ntp"/
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
Scenario: Running install with no Puppetfile and metadata.json
Given there is no Puppetfile
And a file named "metadata.json" with:
"""
{
"name": "random name",
"dependencies": [
{
"name": "puppetlabs/stdlib",
"version_requirement": "4.1.0"
}
]
}
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
Scenario: Running install with no Puppetfile and Modulefile
Given there is no Puppetfile
And a file named "Modulefile" with:
"""
name "random name"
dependency "puppetlabs/stdlib", "4.1.0"
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
Scenario: Installing a module without forge
Given a file named "Puppetfile" with:
"""
mod 'puppetlabs/stdlib', '4.1.0'
"""
When I run `librarian-puppet install`
Then the exit status should be 1
And the output should contain "forge entry is not defined in Puppetfile"
Scenario: Installing an exact version of a module
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/apt', '0.0.4'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/apt/Modulefile" should match /name *'puppetlabs-apt'/
And the file "modules/apt/Modulefile" should match /version *'0\.0\.4'/
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
# Puppet Module tool does not support spaces
# https://github.com/rodjek/librarian-puppet/issues/201
# https://tickets.puppetlabs.com/browse/PUP-2278
@spaces
Scenario: Installing a module in a path with spaces
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/stdlib', '4.1.0'
"""
When PENDING I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
Scenario: Installing a module with invalid versions in the forge
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/apache', '0.4.0'
mod 'puppetlabs/postgresql', '2.0.1'
mod 'puppetlabs/apt', '< 1.4.1' # 1.4.2 causes trouble in travis
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/apache/Modulefile" should match /name *'puppetlabs-apache'/
And the file "modules/apache/Modulefile" should match /version *'0\.4\.0'/
And the file "modules/postgresql/Modulefile" should match /name *'puppetlabs-postgresql'/
And the file "modules/postgresql/Modulefile" should match /version *'2\.0\.1'/
Scenario: Installing a module with several constraints
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/apt', '>=1.0.0', '<1.0.1'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/apt/Modulefile" should match /name *'puppetlabs-apt'/
And the file "modules/apt/Modulefile" should match /version *'1\.0\.0'/
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
Scenario: Changing the path
Given a directory named "puppet"
And a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/ntp', '3.0.3'
"""
When I run `librarian-puppet install --path puppet/modules`
And I run `librarian-puppet config`
Then the exit status should be 0
And the output from "librarian-puppet config" should contain "path: puppet/modules"
And the file "puppet/modules/ntp/Modulefile" should match /name *'puppetlabs-ntp'/
And the file "puppet/modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
Scenario: Handle range version numbers
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/postgresql', '3.2.0'
mod 'puppetlabs/apt', '< 1.4.1' # 1.4.2 causes trouble in travis
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/postgresql/Modulefile" should match /name 'puppetlabs-postgresql'/
And the file "modules/postgresql/Modulefile" should match /version '3\.2\.0'/
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/postgresql', :git => 'git://github.com/puppetlabs/puppet-postgresql', :ref => '3.3.0'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/postgresql/Modulefile" should match /name 'puppetlabs-postgresql'/
And the file "modules/postgresql/Modulefile" should match /version '3\.3\.0'/
Scenario: Installing a module that does not exist
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/xxxxx'
"""
When I run `librarian-puppet install`
Then the exit status should be 1
And the output should match:
"""
Unable to find module 'puppetlabs-xxxxx' on http(s)?://forge(api)?.puppetlabs.com
"""
Scenario: Install a module with conflicts
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/apache', '0.6.0'
mod 'puppetlabs/stdlib', '<2.2.1'
"""
When I run `librarian-puppet install`
Then the exit status should be 1
And the output should contain "Could not resolve the dependencies"
Scenario: Install a module from the Forge with dependencies without version
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'sbadia/gitlab', '0.1.0'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/gitlab/Modulefile" should match /version *'0\.1\.0'/
Scenario: Source dependencies from Modulefile
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
modulefile
"""
And a file named "Modulefile" with:
"""
name "random name"
dependency "puppetlabs/postgresql", "2.4.1"
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/postgresql/Modulefile" should match /name *'puppetlabs-postgresql'/
Scenario: Source dependencies from metadata.json
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
metadata
"""
And a file named "metadata.json" with:
"""
{
"name": "random name",
"dependencies": [
{
"name": "puppetlabs/postgresql",
"version_requirement": "2.4.1"
}
]
}
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/postgresql/Modulefile" should match /name *'puppetlabs-postgresql'/
Scenario: Source dependencies from Modulefile using dash instead of slash
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
modulefile
"""
And a file named "Modulefile" with:
"""
name "random name"
dependency "puppetlabs-postgresql", "2.4.1"
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/postgresql/Modulefile" should match /name *'puppetlabs-postgresql'/
Scenario: Installing a module with duplicated dependencies
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'pdxcat/collectd', '2.1.0'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/collectd/Modulefile" should match /name *'pdxcat-collectd'/
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
Scenario: Installing two modules with same name, alphabetical order wins
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'ripienaar-concat', '0.2.0'
mod 'puppetlabs-concat', '1.2.0'
"""
When I run `librarian-puppet install --verbose`
Then the exit status should be 0
And the file "modules/concat/metadata.json" should match /"name": "ripienaar-concat"/
And the output should contain "Dependency on module 'concat' is fullfilled by multiple modules and only one will be used"
@other-forge
Scenario: Installing from another forge with local reference should not try to download anything from the official forge
Given a file named "Puppetfile" with:
"""
forge "http://127.0.0.1"
mod 'tester/tester', :path => './tester-tester'
"""
And a file named "tester-tester/metadata.json" with:
"""
{
"name": "tester-tester",
"version": "0.1.0",
"author": "Basilio Vera",
"summary": "Just our own test",
"license": "MIT",
"dependencies": [
{ "name": "puppetlabs/inifile" },
{ "name": "tester/tester_dependency1" }
]
}
"""
When I run `librarian-puppet install --verbose`
And the output should not contain "forgeapi.puppetlabs.com"
And the output should contain "Querying Forge API for module puppetlabs-inifile: http://127.0.0.1/api/v1/releases.json?module=puppetlabs/inifile"
================================================
FILE: features/install/git.feature
================================================
Feature: cli/install/git
Puppet librarian needs to install modules from git repositories
Scenario: Installing a module from git
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/apache',
:git => 'https://github.com/puppetlabs/puppetlabs-apache.git', :ref => '1.4.0'
mod 'puppetlabs/stdlib',
:git => 'https://github.com/puppetlabs/puppetlabs-stdlib.git', :ref => '4.6.0'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/apache/metadata.json" should match /"name": "puppetlabs-apache"/
And the file "modules/apache/metadata.json" should match /"version": "1\.4\.0"/
And the git revision of module "apache" should be "e4ec6d4985fdb23e26c809e0d5786823d0689f90"
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
And the file "modules/stdlib/metadata.json" should match /"version": "4\.6\.0"/
And the git revision of module "stdlib" should be "73474b00b5ae3cbccec6cd0711311d6450139e51"
@spaces
Scenario: Installing a module in a path with spaces
Given a file named "Puppetfile" with:
"""
mod 'puppetlabs/stdlib', '4.6.0', :git => 'https://github.com/puppetlabs/puppetlabs-stdlib.git', :ref => '4.6.0'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
Scenario: Installing a module with invalid versions in git
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod "apache",
:git => "https://github.com/puppetlabs/puppetlabs-apache.git", :ref => "1.4.0"
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/apache/metadata.json" should match /"name": "puppetlabs-apache"/
And the file "modules/apache/metadata.json" should match /"version": "1\.4\.0"/
Scenario: Switching a module from forge to git
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/postgresql', '4.0.0'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/postgresql/metadata.json" should match /"name": "puppetlabs-postgresql"/
And the file "modules/postgresql/metadata.json" should match /"version": "4\.0\.0"/
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
When I overwrite "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/postgresql',
:git => 'https://github.com/puppetlabs/puppetlabs-postgresql.git', :ref => '4.3.0'
"""
And I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/postgresql/metadata.json" should match /"name": "puppetlabs-postgresql"/
And the file "modules/postgresql/metadata.json" should match /"version": "4\.3\.0"/
And the file "modules/postgresql/.git/HEAD" should match /9ca4b42450ea9c9ed8eec52dac48cb67187ae925/
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
Scenario: Install a module with dependencies specified in metadata.json
Given a file named "Puppetfile" with:
"""
mod 'puppetlabs-apt', :git => 'https://github.com/puppetlabs/puppetlabs-apt.git', :ref => '1.5.2'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
And the file "modules/apt/metadata.json" should match /"name": "puppetlabs-apt"/
Scenario: Install a module with dependencies specified in a Puppetfile
Given a file named "Puppetfile" with:
"""
mod 'librarian/with_puppetfile', :git => 'https://github.com/rodjek/librarian-puppet.git', :path => 'features/examples/with_puppetfile'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/with_puppetfile/metadata.json" should match /"name": "librarian-with_puppetfile"/
And the file "modules/test/metadata.json" should match /"name": "librarian-test"/
Scenario: Install a module with dependencies specified in a Puppetfile and Modulefile
Given a file named "Puppetfile" with:
"""
mod 'librarian/with_puppetfile', :git => 'https://github.com/rodjek/librarian-puppet.git', :path => 'features/examples/with_puppetfile_and_modulefile'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/with_puppetfile/Modulefile" should match /name *'librarian-with_puppetfile_and_modulefile'/
And the file "modules/test/Modulefile" should match /name *'maestrodev-test'/
Scenario: Install a module with dependencies specified in a Puppetfile and metadata.json
Given a file named "Puppetfile" with:
"""
mod 'librarian/with_puppetfile', :git => 'https://github.com/rodjek/librarian-puppet.git', :path => 'features/examples/with_puppetfile_and_metadata_json'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/with_puppetfile/metadata.json" should match /"name": "librarian-with_puppetfile_and_metadata_json"/
And the file "modules/test/metadata.json" should match /"name": "maestrodev-test"/
Scenario: Running install with no Modulefile nor metadata.json
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/stdlib', :git => 'https://github.com/puppetlabs/puppetlabs-stdlib.git', :ref => '4.6.0'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
Scenario: Running install with metadata.json without dependencies
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/sqlite', :git => 'https://github.com/puppetlabs/puppetlabs-sqlite.git', :ref => '84a0a6'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
Scenario: Install a module using modulefile syntax
Given a file named "Puppetfile" with:
"""
mod 'librarian/modulefile_syntax', :git => 'https://github.com/rodjek/librarian-puppet.git', :path => 'features/examples/modulefile_syntax'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/modulefile_syntax/Modulefile" should match /name *'librarian-modulefile_syntax'/
And the file "modules/test/Modulefile" should match /name *'maestrodev-test'/
Scenario: Install a module using metadata syntax
Given a file named "Puppetfile" with:
"""
mod 'librarian/metadata_syntax', :git => 'https://github.com/rodjek/librarian-puppet.git', :path => 'features/examples/metadata_syntax'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/metadata_syntax/metadata.json" should match /"name": "librarian-metadata_syntax"/
And the file "modules/test/metadata.json" should match /"name": "maestrodev-test"/
Scenario: Install a module from git and using path
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'librarian-test', :git => 'https://github.com/rodjek/librarian-puppet.git', :path => 'features/examples/test'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/test/metadata.json" should match /"version": "0\.0\.1"/
And a file named "modules/stdlib/metadata.json" should exist
Scenario: Install a module from git without version
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'test', :git => 'https://github.com/rodjek/librarian-puppet.git', :path => 'features/examples/dependency_without_version'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/test/metadata.json" should match /"version": "0\.0\.1"/
And a file named "modules/stdlib/metadata.json" should exist
Scenario: Install a module with mismatching Puppetfile and Modulefile
Given a file named "Puppetfile" with:
"""
mod 'duritong/munin', :git => 'https://github.com/duritong/puppet-munin.git', :ref => '0bb71e'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/munin/Modulefile" should match /name *'duritong-munin'/
And the file "modules/concat/metadata.json" should match /"name": *"puppetlabs-concat"/
And a file named "modules/stdlib/metadata.json" should exist
Scenario: Install from Puppetfile with duplicated entries
Given a file named "Puppetfile" with:
"""
mod 'puppetlabs-stdlib',
:git => 'git://github.com/puppetlabs/puppetlabs-stdlib.git'
mod 'puppetlabs-stdlib',
:git => 'https://github.com/puppetlabs/puppetlabs-stdlib.git'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the output should contain "Dependency 'puppetlabs-stdlib' duplicated for module, merging"
================================================
FILE: features/install/github_tarball.feature
================================================
Feature: cli/install/github_tarball
Puppet librarian needs to install tarballed modules from github repositories
@github
Scenario: Installing a module from github tarballs
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/apache', '0.6.0', :github_tarball => 'puppetlabs/puppetlabs-apache'
mod 'puppetlabs/stdlib', '2.3.0', :github_tarball => 'puppetlabs/puppetlabs-stdlib'
"""
When I run `librarian-puppet install --verbose`
Then the exit status should be 0
And the output should contain "Downloading <https://api.github.com/repos/puppetlabs/puppetlabs-apache/tarball/0.6.0"
And the output should contain "Downloading <https://api.github.com/repos/puppetlabs/puppetlabs-stdlib/tarball/2.3.0"
And the file "modules/apache/Modulefile" should match /name *'puppetlabs-apache'/
And the file "modules/apache/Modulefile" should match /version *'0\.6\.0'/
And the file "modules/stdlib/Modulefile" should match /name *'puppetlabs-stdlib'/
And the file "modules/stdlib/Modulefile" should match /version *'2\.3\.0'/
@spaces
@github
Scenario: Installing a module in a path with spaces
Given a file named "Puppetfile" with:
"""
mod 'puppetlabs/stdlib', '4.1.0', :github_tarball => 'puppetlabs/puppetlabs-stdlib'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/stdlib/Modulefile" should match /name *'puppetlabs-stdlib'/
================================================
FILE: features/install/path.feature
================================================
Feature: cli/install/path
Puppet librarian needs to install modules from local paths
Scenario: Install a module with dependencies specified in a Puppetfile
Given a file named "Puppetfile" with:
"""
mod 'librarian/with_puppetfile', :path => '../../features/examples/with_puppetfile'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/with_puppetfile/metadata.json" should match /"name": "librarian-with_puppetfile"/
And the file "modules/test/metadata.json" should match /"name": "librarian-test"/
Scenario: Install a module with recursive path dependencies
Given a file named "Puppetfile" with:
"""
mod 'librarian/path_dependencies', :path => '../../features/examples/path_dependencies'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/path_dependencies/metadata.json" should match /"name": "librarian-path_dependencies"/
And the file "modules/test/metadata.json" should match /"name": "librarian-test"/
And a file named "modules/stdlib/metadata.json" should exist
Scenario: Install a module with dependencies specified in a Puppetfile and Modulefile
Given a file named "Puppetfile" with:
"""
mod 'librarian/with_puppetfile', :path => '../../features/examples/with_puppetfile_and_modulefile'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/with_puppetfile/Modulefile" should match /name *'librarian-with_puppetfile_and_modulefile'/
And the file "modules/test/Modulefile" should match /name *'maestrodev-test'/
Scenario: Install a module with dependencies specified in a Puppetfile and metadata.json
Given a file named "Puppetfile" with:
"""
mod 'librarian/with_puppetfile', :path => '../../features/examples/with_puppetfile_and_metadata_json'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/with_puppetfile/metadata.json" should match /"name": "librarian-with_puppetfile_and_metadata_json"/
And the file "modules/test/metadata.json" should match /"name": "maestrodev-test"/
Scenario: Install a module from path without version
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'test', :path => '../../features/examples/dependency_without_version'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/test/metadata.json" should match /"version": "0\.0\.1"/
And a file named "modules/stdlib/metadata.json" should exist
@spaces
Scenario: Installing a module in a path with spaces
Given a file named "Puppetfile" with:
"""
mod 'librarian/test', :path => '../../features/examples/test'
mod 'puppetlabs/stdlib', :git => 'https://github.com/puppetlabs/puppetlabs-stdlib'
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "modules/test/metadata.json" should match /"name": "librarian-test"/
================================================
FILE: features/install.feature
================================================
Feature: cli/install
In order to be worth anything
Puppet librarian needs to install modules properly
Scenario: Running install with no Puppetfile nor metadata.json
Given there is no Puppetfile
When I run `librarian-puppet install`
Then the output should match /^Metadata file does not exist: .*metadata.json$/
And the exit status should be 1
Scenario: Running install with bad metadata.json
Given a file named "metadata.json" with:
"""
"""
When I run `librarian-puppet install`
Then the output should match /^Unable to parse json file .*metadata.json: .*$/
And the exit status should be 1
Scenario: Install a module transitive dependency from git and forge should be deterministic
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/stdlib', :git => 'https://github.com/puppetlabs/puppetlabs-stdlib.git', :ref => '4.6.0'
mod 'librarian/test', :git => 'https://github.com/rodjek/librarian-puppet.git', :path => 'features/examples/test'
"""
When I run `librarian-puppet install --verbose`
Then the exit status should be 0
And the file "modules/stdlib/metadata.json" should match /"version": "4\.6\.0"/
And the output should not match /Executing puppet module install for puppetlabs.stdlib/
Scenario: Install duplicated dependencies from git and forge, last one wins
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
metadata
mod 'puppetlabs-stdlib', :git => 'https://github.com/puppetlabs/puppetlabs-stdlib.git', :ref => '4.6.0'
"""
And a file named "metadata.json" with:
"""
{
"name": "random name",
"dependencies": [
{
"name": "puppetlabs/stdlib",
"version_requirement": ">= 0"
}
]
}
"""
When I run `librarian-puppet install --verbose`
Then the exit status should be 0
And the file "modules/stdlib/metadata.json" should match /"version": "4\.6\.0"/
And the output should not match /Executing puppet module install for puppetlabs.stdlib/
Scenario: Installing two modules with same name and using exclusions
Given a file named "Puppetfile" with:
"""
forge "https://forgeapi.puppetlabs.com"
mod 'librarian-duplicated_dependencies', :path => '../../features/examples/duplicated_dependencies'
exclusion 'ripienaar-concat'
"""
When I run `librarian-puppet install --verbose`
Then the exit status should be 0
And the file "modules/concat/metadata.json" should match /"name": "puppetlabs-concat"/
And the output should contain "Excluding dependency ripienaar-concat from"
Scenario: Installing two modules with same name and using exclusions, apply transitively
Given a file named "Puppetfile" with:
"""
forge "https://forgeapi.puppetlabs.com"
mod 'librarian-duplicated_dependencies_transitive', :path => '../../features/examples/duplicated_dependencies_transitive'
"""
When PENDING I run `librarian-puppet install --verbose`
Then the exit status should be 0
And the file "modules/concat/metadata.json" should match /"name": "puppetlabs-concat"/
@puppet2 @puppet3
Scenario: Install a module with Modulefile without version
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'librarian-bad_modulefile', :path => 'bad_modulefile'
"""
And a directory named "bad_modulefile/manifests"
And a file named "bad_modulefile/Modulefile" with:
"""
# bad Modulefile
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the output should match:
"""
Unable to parse .*/bad_modulefile/Modulefile, ignoring: Missing version
"""
Scenario: Install a module with the rsync configuration using the --clean flag
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'maestrodev/test'
"""
And a file named ".librarian/puppet/config" with:
"""
---
LIBRARIAN_PUPPET_RSYNC: 'true'
"""
When I run `librarian-puppet config`
Then the exit status should be 0
And the output should contain "rsync: true"
When I run `librarian-puppet install`
Then the exit status should be 0
And a directory named "modules/test" should exist
And the file "modules/test" should have an inode and ctime
When I run `librarian-puppet install --clean`
Then the exit status should be 0
And a directory named "modules/test" should exist
And the file "modules/test" should not have the same inode or ctime as before
Scenario: Install a module with the rsync configuration using the --destructive flag
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'maestrodev/test'
"""
And a file named ".librarian/puppet/config" with:
"""
---
LIBRARIAN_PUPPET_RSYNC: 'true'
"""
When I run `librarian-puppet config`
Then the exit status should be 0
And the output should contain "rsync: true"
When I run `librarian-puppet install`
Then the exit status should be 0
And a directory named "modules/test" should exist
And the file "modules/test" should have an inode and ctime
Given I wait for 1 second
When I run `librarian-puppet install --destructive`
Then the exit status should be 0
And a directory named "modules/test" should exist
And the file "modules/test" should not have the same inode or ctime as before
Scenario: Install a module with the rsync configuration
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'maestrodev/test'
"""
And a file named ".librarian/puppet/config" with:
"""
---
LIBRARIAN_PUPPET_RSYNC: 'true'
"""
When I run `librarian-puppet config`
Then the exit status should be 0
And the output should contain "rsync: true"
When I run `librarian-puppet install`
Then the exit status should be 0
And a directory named "modules/test" should exist
And the file "modules/test" should have an inode and ctime
Given I wait for 1 second
When I run `librarian-puppet install`
Then the exit status should be 0
And a directory named "modules/test" should exist
And the file "modules/test" should have the same inode and ctime as before
================================================
FILE: features/outdated.feature
================================================
Feature: cli/outdated
Puppet librarian needs to print outdated modules
Scenario: Running outdated with forge modules
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/stdlib', '>=3.1.x'
"""
And a file named "Puppetfile.lock" with:
"""
FORGE
remote: http://forge.puppetlabs.com
specs:
puppetlabs/stdlib (3.1.0)
DEPENDENCIES
puppetlabs/stdlib (~> 3.0)
"""
When I run `librarian-puppet outdated`
Then the exit status should be 0
And the output should match:
"""
^puppetlabs-stdlib \(3\.1\.0 -> [\.\d]+\)$
"""
Scenario: Running outdated with git modules
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'test', :git => 'https://github.com/rodjek/librarian-puppet.git', :path => 'features/examples/test'
"""
And a file named "Puppetfile.lock" with:
"""
FORGE
remote: http://forge.puppetlabs.com
specs:
puppetlabs/stdlib (3.1.0)
GIT
remote: https://github.com/rodjek/librarian-puppet.git
path: features/examples/test
ref: master
sha: 10fdf98190a7a22e479628b3616f17f48a857e81
specs:
test (0.0.1)
puppetlabs/stdlib (>= 0)
DEPENDENCIES
test (>= 0)
"""
When I run `librarian-puppet outdated`
Then the exit status should be 0
And PENDING the output should match:
# """
# ^puppetlabs-stdlib \(3\.1\.0 -> [\.\d]+\)$
# ^test .*$
# """
================================================
FILE: features/package.feature
================================================
Feature: cli/package
Puppet librarian needs to package modules
Scenario: Packaging a forge module
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/apt', '1.4.0'
mod 'puppetlabs/stdlib', '4.1.0'
"""
When I run `librarian-puppet package --verbose`
Then the exit status should be 0
And the file "modules/apt/Modulefile" should match /name *'puppetlabs-apt'/
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
And the following files should exist:
| vendor/puppet/cache/puppetlabs-apt-1.4.0.tar.gz |
| vendor/puppet/cache/puppetlabs-stdlib-4.1.0.tar.gz |
Scenario: Packaging a git module
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/apt', '1.5.0', :git => 'https://github.com/puppetlabs/puppetlabs-apt.git', :ref => '1.5.0'
mod 'puppetlabs/stdlib', '4.1.0'
"""
When I run `librarian-puppet package --verbose`
Then the exit status should be 0
And the file "modules/apt/Modulefile" should match /name *'puppetlabs-apt'/
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
And the following files should exist:
| vendor/puppet/source/e5657a61b9ac0dd3c00002c777b0d3c615bb98a5.tar.gz |
| vendor/puppet/cache/puppetlabs-stdlib-4.1.0.tar.gz |
@github
Scenario: Packaging a github tarball module
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/apt', '1.4.0', :github_tarball => 'puppetlabs/puppetlabs-apt'
mod 'puppetlabs/stdlib', '4.1.0'
"""
When I run `librarian-puppet package --verbose`
Then the exit status should be 0
And the file "modules/apt/Modulefile" should match /name *'puppetlabs-apt'/
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
And the following files should exist:
| vendor/puppet/cache/puppetlabs-puppetlabs-apt-1.4.0.tar.gz |
| vendor/puppet/cache/puppetlabs-stdlib-4.1.0.tar.gz |
================================================
FILE: features/step_definitions/convergence_steps.rb
================================================
Then /^the file "([^"]*)" should have an inode and ctime$/ do |file|
cd('.') do
stat = File.stat(File.expand_path(file))
@before_inode = { 'ino' => stat.ino, 'ctime' => stat.ctime }
expect(@before_inode['ino']).not_to eq nil
expect(@before_inode['ctime']).not_to eq nil
end
end
Then /^the file "([^"]*)" should have the same inode and ctime as before$/ do |file|
cd('.') do
stat = File.stat(File.expand_path(file))
expect(stat.ino).to eq @before_inode['ino']
expect(stat.ctime).to eq @before_inode['ctime']
end
end
Then /^the file "([^"]*)" should not have the same inode or ctime as before$/ do |file|
cd('.') do
stat = File.stat(File.expand_path(file))
begin
expect(stat.ino).not_to eq @before_inode['ino']
rescue RSpec::Expectations::ExpectationNotMetError
expect(stat.ctime).not_to eq @before_inode['ctime']
end
end
end
Then /^the git revision of module "([^"]*)" should be "([0-9a-f]*)"$/ do |module_name, rev|
cd("modules/#{module_name}")
cmd = "git rev-parse HEAD"
run_simple(cmd)
assert_exact_output(rev, output_from(cmd).strip)
cd("../..")
end
Given /^I wait for (\d+) seconds?$/ do |n|
sleep(n.to_i)
end
================================================
FILE: features/support/env.rb
================================================
require 'aruba/cucumber'
require 'fileutils'
Before do
@aruba_timeout_seconds = 120
end
Before('@spaces') do
@dirs = ["tmp/aruba with spaces"]
@dirs.each {|dir| FileUtils.rm_rf dir}
end
Given /^PENDING/ do
pending
end
Given(/^there is no Puppetfile$/) do
in_current_dir do
fail "Puppetfile exists at #{File.expand_path('Puppetfile')}" if (File.exist?('Puppetfile'))
end
end
ENV['LIBRARIAN_PUPPET_TMP'] = '.tmp'
================================================
FILE: features/update.feature
================================================
Feature: cli/update
Puppet librarian needs to update modules properly
Scenario: Updating a module with no Puppetfile and with metadata.json
Given a file named "metadata.json" with:
"""
{
"name": "random name",
"dependencies": [
{
"name": "puppetlabs/stdlib",
"version_requirement": "3.1.x"
}
]
}
"""
And a file named "Puppetfile.lock" with:
"""
FORGE
remote: http://forge.puppetlabs.com
specs:
puppetlabs/stdlib (3.1.0)
DEPENDENCIES
puppetlabs/stdlib (~> 3.0)
"""
When I run `librarian-puppet update puppetlabs/stdlib`
Then the exit status should be 0
And the file "Puppetfile" should not exist
And the file "Puppetfile.lock" should match /puppetlabs.stdlib \(3\.1\.1\)/
And the file "modules/stdlib/Modulefile" should match /name *'puppetlabs-stdlib'/
And the file "modules/stdlib/Modulefile" should match /version *'3\.1\.1'/
Scenario: Updating a module with no Puppetfile and with Modulefile
Given a file named "Modulefile" with:
"""
name "random name"
dependency "puppetlabs/stdlib", "3.1.x"
"""
And a file named "Puppetfile.lock" with:
"""
FORGE
remote: http://forge.puppetlabs.com
specs:
puppetlabs/stdlib (3.1.0)
DEPENDENCIES
puppetlabs/stdlib (~> 3.0)
"""
When I run `librarian-puppet update puppetlabs/stdlib`
Then the exit status should be 0
And the file "Puppetfile" should not exist
And the file "Puppetfile.lock" should match /puppetlabs.stdlib \(3\.1\.1\)/
And the file "modules/stdlib/Modulefile" should match /name *'puppetlabs-stdlib'/
And the file "modules/stdlib/Modulefile" should match /version *'3\.1\.1'/
Scenario: Updating a module
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/stdlib', '3.1.x'
"""
And a file named "Puppetfile.lock" with:
"""
FORGE
remote: http://forge.puppetlabs.com
specs:
puppetlabs/stdlib (3.1.0)
DEPENDENCIES
puppetlabs/stdlib (~> 3.0)
"""
When I run `librarian-puppet update puppetlabs-stdlib`
Then the exit status should be 0
And the file "Puppetfile.lock" should match /puppetlabs.stdlib \(3\.1\.1\)/
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
And the file "modules/stdlib/Modulefile" should match /version *'3\.1\.1'/
Scenario: Updating a module using organization/module
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/stdlib', '3.1.x'
"""
And a file named "Puppetfile.lock" with:
"""
FORGE
remote: http://forge.puppetlabs.com
specs:
puppetlabs/stdlib (3.1.0)
DEPENDENCIES
puppetlabs/stdlib (~> 3.0)
"""
When I run `librarian-puppet update --verbose puppetlabs/stdlib`
Then the exit status should be 0
And the file "Puppetfile.lock" should match /puppetlabs.stdlib \(3\.1\.1\)/
And the file "modules/stdlib/metadata.json" should match /"name": "puppetlabs-stdlib"/
And the file "modules/stdlib/Modulefile" should match /version *'3\.1\.1'/
Scenario: Updating a module from git with a branch ref
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod "puppetlabs-stdlib",
:git => "https://github.com/puppetlabs/puppetlabs-stdlib.git", :ref => "3.2.x"
"""
And a file named "Puppetfile.lock" with:
"""
GIT
remote: https://github.com/puppetlabs/puppetlabs-stdlib.git
ref: 3.2.x
sha: 326a8fd801ecba11005189c10ca8749872ef6577
specs:
puppetlabs-stdlib (3.2.1)
DEPENDENCIES
puppetlabs-stdlib (>= 0)
"""
When I run `librarian-puppet install`
Then the exit status should be 0
And the git revision of module "stdlib" should be "326a8fd801ecba11005189c10ca8749872ef6577"
When I run `librarian-puppet update`
Then the exit status should be 0
And the git revision of module "stdlib" should be "c0b5ce3b015db9f50d486040c16d8de56c6b4991"
Scenario: Updating a module with invalid versions in git
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod "apache",
:git => "https://github.com/puppetlabs/puppetlabs-apache.git", :ref => "0.5.0-rc1"
"""
And a file named "Puppetfile.lock" with:
"""
FORGE
remote: http://forge.puppetlabs.com
specs:
puppetlabs/firewall (0.0.4)
puppetlabs/stdlib (3.2.0)
GIT
remote: https://github.com/puppetlabs/puppetlabs-apache.git
ref: 0.5.0-rc1
sha: 94ebca3aaaf2144a7b9ce7ca6a13837ec48a7e2a
specs:
apache ()
puppetlabs/firewall (>= 0.0.4)
puppetlabs/stdlib (>= 2.2.1)
DEPENDENCIES
apache (>= 0)
"""
When I run `librarian-puppet update apache`
Then the exit status should be 0
And the file "Puppetfile.lock" should match /sha: d81999533af54a6fe510575d3b143308184a5005/
And the file "modules/apache/Modulefile" should match /name *'puppetlabs-apache'/
And the file "modules/apache/Modulefile" should match /version *'0\.5\.0-rc1'/
Scenario: Updating a module that is not in the Puppetfile
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'puppetlabs/stdlib', '3.1.x'
"""
And a file named "Puppetfile.lock" with:
"""
FORGE
remote: http://forge.puppetlabs.com
specs:
puppetlabs/stdlib (3.1.0)
DEPENDENCIES
puppetlabs/stdlib (~> 3.0)
"""
When I run `librarian-puppet update stdlib`
Then the exit status should be 1
And the output should contain "Unable to find module stdlib"
Scenario: Updating a module to a .10 release to ensure versions are correctly ordered
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'maestrodev/test'
"""
And a file named "Puppetfile.lock" with:
"""
FORGE
remote: http://forge.puppetlabs.com
specs:
maestrodev/test (1.0.2)
DEPENDENCIES
maestrodev/test (>= 0)
"""
When I run `librarian-puppet update --verbose`
Then the exit status should be 0
And the file "Puppetfile.lock" should match /maestrodev.test \(1\.0\.[1-9][0-9]\)/
And the file "modules/test/Modulefile" should contain "name 'maestrodev-test'"
And the file "modules/test/Modulefile" should match /version '1\.0\.[1-9][0-9]'/
Scenario: Updating a forge module with the rsync configuration
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod 'maestrodev/test'
"""
And a file named "Puppetfile.lock" with:
"""
FORGE
remote: http://forge.puppetlabs.com
specs:
maestrodev/test (1.0.2)
DEPENDENCIES
maestrodev/test (>= 0)
"""
And a file named ".librarian/puppet/config" with:
"""
---
LIBRARIAN_PUPPET_RSYNC: 'true'
"""
When I run `librarian-puppet config`
Then the exit status should be 0
And the output should contain "rsync: true"
When I run `librarian-puppet update --verbose`
Then the exit status should be 0
And a directory named "modules/test" should exist
And the file "modules/test" should have an inode and ctime
When I run `librarian-puppet update --verbose`
Then the exit status should be 0
And a directory named "modules/test" should exist
And the file "modules/test" should have the same inode and ctime as before
@announce
Scenario: Updating a git module with the rsync configuration
Given a file named "Puppetfile" with:
"""
forge "http://forge.puppetlabs.com"
mod "puppetlabs-stdlib",
:git => "https://github.com/puppetlabs/puppetlabs-stdlib.git", :ref => "3.2.x"
"""
And a file named "Puppetfile.lock" with:
"""
GIT
remote: https://github.com/puppetlabs/puppetlabs-stdlib.git
ref: 3.2.x
sha: 326a8fd801ecba11005189c10ca8749872ef6577
specs:
puppetlabs-stdlib (3.2.1)
DEPENDENCIES
puppetlabs-stdlib (>= 0)
"""
And a file named ".librarian/puppet/config" with:
"""
---
LIBRARIAN_PUPPET_RSYNC: 'true'
"""
When I run `librarian-puppet config`
Then the exit status should be 0
And the output should contain "rsync: true"
When I run `librarian-puppet install`
Then the exit status should be 0
And the file "Puppetfile.lock" should contain "326a8fd801ecba11005189c10ca8749872ef6577"
And the git revision of module "stdlib" should be "326a8fd801ecba11005189c10ca8749872ef6577"
And a directory named "modules/stdlib" should exist
When I run `librarian-puppet update --verbose`
Then the exit status should be 0
And a directory named "modules/stdlib" should exist
And the file "modules/stdlib" should have an inode and ctime
And the file "Puppetfile.lock" should contain "c0b5ce3b015db9f50d486040c16d8de56c6b4991"
And the git revision of module "stdlib" should be "c0b5ce3b015db9f50d486040c16d8de56c6b4991"
When I run `librarian-puppet update --verbose`
Then the exit status should be 0
And a directory named "modules/stdlib" should exist
And the file "modules/stdlib" should have the same inode and ctime as before
And the file "Puppetfile.lock" should contain "c0b5ce3b015db9f50d486040c16d8de56c6b4991"
And the git revision of module "stdlib" should be "c0b5ce3b015db9f50d486040c16d8de56c6b4991"
================================================
FILE: features/version.feature
================================================
Feature: cli/version
Scenario: Getting the version
When I run `librarian-puppet version`
Then the exit status should be 0
And the output should contain "librarian-"
================================================
FILE: lib/librarian/puppet/action/install.rb
================================================
require 'librarian/action/install'
module Librarian
module Puppet
module Action
class Install < Librarian::Action::Install
private
def create_install_path
install_path.rmtree if install_path.exist? && destructive?
install_path.mkpath
end
def destructive?
environment.config_db.local['destructive'] == 'true'
end
def check_specfile
# don't fail if Puppetfile doesn't exist as we'll use the Modulefile or metadata.json
end
end
end
end
end
================================================
FILE: lib/librarian/puppet/action/resolve.rb
================================================
require 'librarian/action/resolve'
module Librarian
module Puppet
module Action
class Resolve < Librarian::Action::Resolve
include Librarian::Puppet::Util
def run
super
manifests = environment.lock.manifests.select{ |m| m.name }
dupes = manifests.group_by{ |m| module_name(m.name) }.select { |k, v| v.size > 1 }
dupes.each do |k,v|
warn("Dependency on module '#{k}' is fullfilled by multiple modules and only one will be used: #{v.map{|m|m.name}}")
end
end
end
end
end
end
================================================
FILE: lib/librarian/puppet/action.rb
================================================
require "librarian/puppet/action/install"
require "librarian/puppet/action/resolve"
================================================
FILE: lib/librarian/puppet/cli.rb
================================================
require 'librarian/helpers'
require 'librarian/cli'
require 'librarian/puppet'
require 'librarian/puppet/action'
module Librarian
module Puppet
class Cli < Librarian::Cli
include Librarian::Puppet::Util
module Particularity
def root_module
Puppet
end
end
include Particularity
extend Particularity
source_root Pathname.new(__FILE__).dirname.join("templates")
def init
copy_file environment.specfile_name
if File.exists? ".gitignore"
gitignore = File.read('.gitignore').split("\n")
else
gitignore = []
end
gitignore << ".tmp/" unless gitignore.include? ".tmp/"
gitignore << "modules/" unless gitignore.include? "modules/"
File.open(".gitignore", 'w') do |f|
f.puts gitignore.join("\n")
end
end
desc "install", "Resolves and installs all of the dependencies you specify."
option "quiet", :type => :boolean, :default => false
option "verbose", :type => :boolean, :default => false
option "line-numbers", :type => :boolean, :default => false
option "clean", :type => :boolean, :default => false
option "strip-dot-git", :type => :boolean
option "path", :type => :string
option "destructive", :type => :boolean, :default => false
option "local", :type => :boolean, :default => false
option "use-v1-api", :type => :boolean, :default => true
def install
ensure!
clean! if options["clean"]
unless options["destructive"].nil?
environment.config_db.local['destructive'] = options['destructive'].to_s
end
if options.include?("strip-dot-git")
strip_dot_git_val = options["strip-dot-git"] ? "1" : nil
environment.config_db.local["install.strip-dot-git"] = strip_dot_git_val
end
if options.include?("path")
environment.config_db.local["path"] = options["path"]
end
environment.config_db.local['use-v1-api'] = options['use-v1-api'] ? '1' : nil
environment.config_db.local['mode'] = options['local'] ? 'local' : nil
resolve!
debug { "Install: dependencies resolved"}
install!
end
# only used to replace / to - in the module names
def update(*names)
warn("Usage of module/name is deprecated, use module-name") if names.any? {|n| n.include?("/")}
super(*names.map{|n| normalize_name(n)})
end
desc "package", "Cache the puppet modules in vendor/puppet/cache."
option "quiet", :type => :boolean, :default => false
option "verbose", :type => :boolean, :default => false
option "line-numbers", :type => :boolean, :default => false
option "clean", :type => :boolean, :default => false
option "strip-dot-git", :type => :boolean
option "path", :type => :string
option "destructive", :type => :boolean, :default => false
def package
environment.vendor!
install
end
def version
say "librarian-puppet v#{Librarian::Puppet::VERSION}"
end
private
# override the actions to use our own
def install!(options = { })
Action::Install.new(environment, options).run
end
def resolve!(options = { })
Action::Resolve.new(environment, options).run
end
end
end
end
================================================
FILE: lib/librarian/puppet/dependency.rb
================================================
module Librarian
module Puppet
class Dependency < Librarian::Dependency
include Librarian::Puppet::Util
def initialize(name, requirement, source)
# Issue #235 fail if forge source is not defined
raise Error, "forge entry is not defined in Puppetfile" if source.instance_of?(Array) && source.empty?
super(normalize_name(name), requirement, source)
end
end
end
end
================================================
FILE: lib/librarian/puppet/dsl.rb
================================================
require 'librarian/dsl'
require 'librarian/dsl/target'
require 'librarian/puppet/source'
require 'librarian/puppet/dependency'
module Librarian
module Puppet
class Dsl < Librarian::Dsl
FORGE_URL = "https://forgeapi.puppetlabs.com"
dependency :mod
source :forge => Source::Forge
source :git => Source::Git
source :path => Source::Path
source :github_tarball => Source::GitHubTarball
def default_specfile
Proc.new do
forge FORGE_URL
metadata
end
end
def self.dependency_type
Librarian::Puppet::Dependency
end
def post_process_target(target)
# save the default forge defined
default_forge = target.sources.select {|s| s.is_a? Librarian::Puppet::Source::Forge}.first
Librarian::Puppet::Source::Forge.default = default_forge || Librarian::Puppet::Source::Forge.from_lock_options(environment, :remote => FORGE_URL)
end
def receiver(target)
Receiver.new(target)
end
class Receiver < Librarian::Dsl::Receiver
attr_reader :specfile, :working_path
# save the specfile and call librarian
def run(specfile = nil)
@working_path = specfile.kind_of?(Pathname) ? specfile.parent : Pathname.new(Dir.pwd)
@specfile = specfile
super
end
# implement the 'modulefile' syntax for Puppetfile
def modulefile
f = modulefile_path
raise Error, "Modulefile file does not exist: #{f}" unless File.exists?(f)
File.read(f).lines.each do |line|
regexp = /\s*dependency\s+('|")([^'"]+)\1\s*(?:,\s*('|")([^'"]+)\3)?/
regexp =~ line && mod($2, $4)
end
end
# implement the 'metadata' syntax for Puppetfile
def metadata
f = working_path.join('metadata.json')
unless File.exists?(f)
msg = "Metadata file does not exist: #{f}"
# try modulefile, in case we don't have a Puppetfile and we are using the default template
if File.exists?(modulefile_path)
modulefile
return
else
raise Error, msg
end
end
begin
json = JSON.parse(File.read(f))
rescue JSON::ParserError => e
raise Error, "Unable to parse json file #{f}: #{e}"
end
dependencyList = json['dependencies']
dependencyList.each do |d|
mod(d['name'], d['version_requirement'])
end
end
private
def modulefile_path
working_path.join('Modulefile')
end
end
end
end
end
================================================
FILE: lib/librarian/puppet/environment.rb
================================================
require "librarian/environment"
require "librarian/puppet/dsl"
require "librarian/puppet/source"
require "librarian/puppet/lockfile"
module Librarian
module Puppet
class Environment < Librarian::Environment
def adapter_name
"puppet"
end
def lockfile
Lockfile.new(self, lockfile_path)
end
def ephemeral_lockfile
Lockfile.new(self, nil)
end
def tmp_path
part = config_db["tmp"] || ".tmp"
project_path.join(part)
end
def install_path
part = config_db["path"] || "modules"
project_path.join(part)
end
def vendor_path
project_path.join('vendor/puppet')
end
def vendor_cache
vendor_path.join('cache')
end
def vendor_source
vendor_path.join('source')
end
def vendor!
vendor_cache.mkpath unless vendor_cache.exist?
vendor_source.mkpath unless vendor_source.exist?
end
def vendor?
vendor_path.exist?
end
def local?
config_db['mode'] == 'local'
end
def use_v1_api
config_db['use-v1-api']
end
end
end
end
================================================
FILE: lib/librarian/puppet/extension.rb
================================================
require 'librarian/puppet/environment'
require 'librarian/action/base'
module Librarian
module Puppet
extend self
extend Librarian
end
end
================================================
FILE: lib/librarian/puppet/lockfile.rb
================================================
# Extend Lockfile to normalize module names from acme/mod to acme-mod
module Librarian
module Puppet
class Lockfile < Librarian::Lockfile
# Extend the parser to normalize module names in old .lock files, converting / to -
class Parser < Librarian::Lockfile::Parser
include Librarian::Puppet::Util
def extract_and_parse_sources(lines)
sources = super
sources.each do |source|
source[:manifests] = Hash[source[:manifests].map do |name,manifest|
[normalize_name(name), manifest]
end]
end
sources
end
def extract_and_parse_dependencies(lines, manifests_index)
# when looking up in manifests_index normalize the name beforehand
class << manifests_index
include Librarian::Puppet::Util
alias_method :old_lookup, :[]
define_method(:[]) { |k| self.old_lookup(normalize_name(k)) }
end
super(lines, manifests_index)
end
end
def load(string)
Parser.new(environment).parse(string)
end
end
end
end
================================================
FILE: lib/librarian/puppet/source/forge/repo.rb
================================================
require 'json'
require 'open-uri'
require 'librarian/puppet/util'
require 'librarian/puppet/source/repo'
module Librarian
module Puppet
module Source
class Forge
class Repo < Librarian::Puppet::Source::Repo
include Librarian::Puppet::Util
def versions
return @versions if @versions
@versions = get_versions
if @versions.empty?
info { "No versions found for module #{name}" }
else
debug { " Module #{name} found versions: #{@versions.join(", ")}" }
end
@versions
end
# fetch list of versions ordered for newer to older
def get_versions
# implement in subclasses
end
# return map with dependencies in the form {module_name => version,...}
# version: Librarian::Manifest::Version
def dependencies(version)
# implement in subclasses
end
# return the url for a specific version tarball
# version: Librarian::Manifest::Version
def url(name, version)
# implement in subclasses
end
def manifests
versions.map do |version|
Manifest.new(source, name, version)
end
end
def install_version!(version, install_path)
if environment.local? && !vendored?(name, version)
raise Error, "Could not find a local copy of #{name} at #{version}."
end
if environment.vendor?
vendor_cache(name, version) unless vendored?(name, version)
end
cache_version_unpacked! version
if install_path.exist? && rsync? != true
install_path.rmtree
end
unpacked_path = version_unpacked_cache_path(version).join(module_name(name))
unless unpacked_path.exist?
raise Error, "#{unpacked_path} does not exist, something went wrong. Try removing it manually"
else
cp_r(unpacked_path, install_path)
end
end
def cache_version_unpacked!(version)
path = version_unpacked_cache_path(version)
return if path.directory?
# The puppet module command is only available from puppet versions >= 2.7.13
#
# Specifying the version in the gemspec would force people to upgrade puppet while it's still usable for git
# So we do some more clever checking
#
# Executing older versions or via puppet-module tool gives an exit status = 0 .
#
check_puppet_module_options
path.mkpath
target = vendored?(name, version) ? vendored_path(name, version).to_s : name
# can't pass the default v3 forge url (http://forgeapi.puppetlabs.com)
# to clients that use the v1 API (https://forge.puppetlabs.com)
# nor the other way around
module_repository = source.to_s
if Forge.client_api_version() > 1 and module_repository =~ %r{^http(s)?://forge\.puppetlabs\.com}
module_repository = "https://forgeapi.puppetlabs.com"
warn { "Replacing Puppet Forge API URL to use v3 #{module_repository} as required by your client version #{Librarian::Puppet.puppet_version}" }
end
m = module_repository.match(%r{^http(s)?://forge(api)?\.puppetlabs\.com})
if Forge.client_api_version() == 1 and m
ssl = m[1]
# Puppet 2.7 can't handle the 302 returned by the https url, so stick to http
if ssl and Librarian::Puppet::puppet_gem_version < Gem::Version.create('3.0.0')
warn { "Using plain http as your version of Puppet #{Librarian::Puppet::puppet_gem_version} can't download from forge.puppetlabs.com using https" }
ssl = nil
end
module_repository = "http#{ssl}://forge.puppetlabs.com"
end
command = %W{puppet module install --version #{version} --target-dir}
command.push(*[path.to_s, "--module_repository", module_repository, "--modulepath", path.to_s, "--module_working_dir", path.to_s, "--ignore-dependencies", target])
debug { "Executing puppet module install for #{name} #{version}: #{command.join(" ")}" }
begin
Librarian::Posix.run!(command)
rescue Posix::CommandFailure => e
# Rollback the directory if the puppet module had an error
begin
path.unlink
rescue => u
debug("Unable to rollback path #{path}: #{u}")
end
tar = Dir[File.join(path.to_s, "**/*.tar.gz")]
msg = ""
if e.message =~ /Unexpected EOF in archive/ and !tar.empty?
file = tar.first
msg = " (looks like an incomplete download of #{file})"
end
raise Error, "Error executing puppet module install#{msg}. Check that this command succeeds:\n#{command.join(" ")}\nError:\n#{e.message}"
end
end
def check_puppet_module_options
min_version = Gem::Version.create('2.7.13')
if Librarian::Puppet.puppet_gem_version < min_version
raise Error, "To get modules from the forge, we use the puppet faces module command. For this you need at least puppet version 2.7.13 and you have #{Librarian::Puppet.puppet_version}"
end
end
def vendor_cache(name, version)
url = url(name, version)
path = vendored_path(name, version).to_s
debug { "Downloading #{url} into #{path}"}
environment.vendor!
File.open(path, 'wb') do |f|
open(url, "rb") do |input|
f.write(input.read)
end
end
end
end
end
end
end
end
================================================
FILE: lib/librarian/puppet/source/forge/repo_v1.rb
================================================
require 'json'
require 'open-uri'
require 'librarian/puppet/source/forge/repo'
module Librarian
module Puppet
module Source
class Forge
class RepoV1 < Librarian::Puppet::Source::Forge::Repo
def initialize(source, name)
super(source, name)
# API returned data for this module including all versions and dependencies, indexed by module name
# from http://forge.puppetlabs.com/api/v1/releases.json?module=#{name}
@api_data = nil
# API returned data for this module and a specific version, indexed by version
# from http://forge.puppetlabs.com/api/v1/releases.json?module=#{name}&version=#{version}
@api_version_data = {}
end
def get_versions
api_data(name).map { |r| r['version'] }.reverse
end
def dependencies(version)
api_version_data(name, version)['dependencies']
end
def url(name, version)
info = api_version_data(name, version)
"#{source}#{info[name].first['file']}"
end
private
# convert organization/modulename to organization-modulename
def normalize_dependencies(data)
return nil if data.nil?
# convert organization/modulename to organization-modulename
data.keys.each do |m|
if m =~ %r{.*/.*}
data[normalize_name(m)] = data[m]
data.delete(m)
end
end
data
end
# get and cache the API data for a specific module with all its versions and dependencies
def api_data(module_name)
return @api_data[module_name] if @api_data
# call API and cache data
@api_data = normalize_dependencies(api_call(module_name))
if @api_data.nil?
raise Error, "Unable to find module '#{name}' on #{source}"
end
@api_data[module_name]
end
# get and cache the API data for a specific module and version
def api_version_data(module_name, version)
# if we already got all the versions, find in cached data
return @api_data[module_name].detect{|x| x['version'] == version.to_s} if @api_data
# otherwise call the api for this version if not cached already
@api_version_data[version] = normalize_dependencies(api_call(name, version)) if @api_version_data[version].nil?
@api_version_data[version]
end
def api_call(module_name, version=nil)
url = source.uri.clone
url.path += "#{'/' if url.path.empty? or url.path[-1] != '/'}api/v1/releases.json"
url.query = "module=#{module_name.sub('-','/')}" # v1 API expects "organization/module"
url.query += "&version=#{version}" unless version.nil?
debug { "Querying Forge API for module #{name}#{" and version #{version}" unless version.nil?}: #{url}" }
begin
data = open(url) {|f| f.read}
JSON.parse(data)
rescue OpenURI::HTTPError => e
case e.io.status[0].to_i
when 404,410
nil
else
raise e, "Error requesting #{url}: #{e.to_s}"
end
end
end
end
end
end
end
end
================================================
FILE: lib/librarian/puppet/source/forge/repo_v3.rb
================================================
require 'librarian/puppet/source/forge/repo'
require 'puppet_forge'
require 'librarian/puppet/version'
module Librarian
module Puppet
module Source
class Forge
class RepoV3 < Librarian::Puppet::Source::Forge::Repo
PuppetForge.user_agent = "librarian-puppet/#{Librarian::Puppet::VERSION}"
def initialize(source, name)
PuppetForge.host = source.uri.clone
super(source, name)
end
def get_versions
get_module.releases.select{|r| r.deleted_at.nil?}.map{|r| r.version}
end
def dependencies(version)
array = get_release(version).metadata[:dependencies].map{|d| [d['name'], d['version_requirement']]}
Hash[*array.flatten(1)]
end
def url(name, version)
if name == "#{get_module().owner.username}/#{get_module().name}"
release = get_release(version)
else
# should never get here as we use one repo object for each module (to be changed in the future)
debug { "Looking up url for #{name}@#{version}" }
release = PuppetForge::Release.find("#{name}-#{version}")
end
"#{source}#{release.file_uri}"
end
private
def get_module
@module ||= PuppetForge::Module.find(name)
raise(Error, "Unable to find module '#{name}' on #{source}") unless @module
@module
end
def get_release(version)
release = get_module.releases.find{|r| r.version == version.to_s}
if release.nil?
versions = get_module.releases.map{|r| r.version}
raise Error, "Unable to find version '#{version}' for module '#{name}' on #{source} amongst #{versions}"
end
release
end
end
end
end
end
end
================================================
FILE: lib/librarian/puppet/source/forge.rb
================================================
require 'uri'
require 'librarian/puppet/util'
require 'librarian/puppet/source/forge/repo_v1'
require 'librarian/puppet/source/forge/repo_v3'
module Librarian
module Puppet
module Source
class Forge
include Librarian::Puppet::Util
class << self
LOCK_NAME = 'FORGE'
def default=(source)
@@default = source
end
def default
@@default
end
def lock_name
LOCK_NAME
end
def from_lock_options(environment, options)
new(environment, options[:remote], options.reject { |k, v| k == :remote })
end
def from_spec_args(environment, uri, options)
recognised_options = []
unrecognised_options = options.keys - recognised_options
unless unrecognised_options.empty?
raise Error, "unrecognised options: #{unrecognised_options.join(", ")}"
end
new(environment, uri, options)
end
def client_api_version()
version = 1
pe_version = Librarian::Puppet.puppet_version.match(/\(Puppet Enterprise (.+)\)/)
# Puppet 3.6.0+ uses api v3
if Librarian::Puppet::puppet_gem_version >= Gem::Version.create('3.6.0.a')
version = 3
# Puppet enterprise 3.2.0+ uses api v3
elsif pe_version and Gem::Version.create(pe_version[1].strip) >= Gem::Version.create('3.2.0')
version = 3
end
return version
end
end
attr_accessor :environment
private :environment=
attr_reader :uri
def initialize(environment, uri, options = {})
self.environment = environment
if uri =~ %r{^http(s)?://forge\.puppetlabs\.com}
uri = "https://forgeapi.puppetlabs.com"
warn { "Replacing Puppet Forge API URL to use v3 #{uri}. You should update your Puppetfile" }
end
@uri = URI::parse(uri)
@cache_path = nil
end
def to_s
clean_uri(uri).to_s
end
def ==(other)
other &&
self.class == other.class &&
self.uri == other.uri
end
alias :eql? :==
def hash
self.to_s.hash
end
def to_spec_args
[clean_uri(uri).to_s, {}]
end
def to_lock_options
{:remote => clean_uri(uri).to_s}
end
def pinned?
false
end
def unpin!
end
def install!(manifest)
manifest.source == self or raise ArgumentError
debug { "Installing #{manifest}" }
name = manifest.name
version = manifest.version
install_path = install_path(name)
repo = repo(name)
repo.install_version! version, install_path
end
def manifest(name, version, dependencies)
manifest = Manifest.new(self, name)
manifest.version = version
manifest.dependencies = dependencies
manifest
end
def cache_path
@cache_path ||= begin
dir = "#{uri.host}#{uri.path}".gsub(/[^0-9a-z\-_]/i, '_')
environment.cache_path.join("source/puppet/forge/#{dir}")
end
end
def install_path(name)
environment.install_path.join(module_name(name))
end
def fetch_version(name, version_uri)
versions = repo(name).versions
if versions.include? version_uri
version_uri
else
versions.first
end
end
def fetch_dependencies(name, version, version_uri)
repo(name).dependencies(version).map do |k, v|
v = Librarian::Dependency::Requirement.new(v).to_gem_requirement
Dependency.new(k, v, nil)
end
end
def manifests(name)
repo(name).manifests
end
private
def repo(name)
@repo ||= {}
unless @repo[name]
# if we are using the official Forge then use API v3, otherwise stick to v1 for now
if uri.hostname =~ /\.puppetlabs\.com$/ || !environment.use_v1_api
@repo[name] = RepoV3.new(self, name)
else
@repo[name] = RepoV1.new(self, name)
end
end
@repo[name]
end
end
end
end
end
================================================
FILE: lib/librarian/puppet/source/git.rb
================================================
require 'librarian/source/git'
require 'librarian/puppet/source/local'
module Librarian
module Source
class Git
class Repository
def hash_from(remote, reference)
branch_names = remote_branch_names[remote]
if branch_names.include?(reference)
reference = "#{remote}/#{reference}"
end
command = %W(rev-parse #{reference}^{commit} --quiet)
run!(command, :chdir => true).strip
end
end
end
end
module Puppet
module Source
class Git < Librarian::Source::Git
include Local
include Librarian::Puppet::Util
def cache!
return vendor_checkout! if vendor_cached?
if environment.local?
raise Error, "Could not find a local copy of #{uri}#{" at #{sha}" unless sha.nil?}."
end
begin
super
rescue Librarian::Posix::CommandFailure => e
raise Error, "Could not checkout #{uri}#{" at #{sha}" unless sha.nil?}: #{e}"
end
cache_in_vendor(repository.path) if environment.vendor?
end
private
def vendor_tar
environment.vendor_source.join("#{sha}.tar")
end
def vendor_tgz
environment.vendor_source.join("#{sha}.tar.gz")
end
def vendor_cached?
vendor_tgz.exist?
end
def vendor_checkout!
repository.path.rmtree if repository.path.exist?
repository.path.mkpath
Librarian::Posix.run!(%W{tar xzf #{vendor_tgz}}, :chdir => repository.path.to_s)
repository_cached!
end
def cache_in_vendor(tmp_path)
Librarian::Posix.run!(%W{git archive -o #{vendor_tar} #{sha}}, :chdir => tmp_path.to_s)
Librarian::Posix.run!(%W{gzip #{vendor_tar}}, :chdir => tmp_path.to_s)
end
end
end
end
end
================================================
FILE: lib/librarian/puppet/source/githubtarball/repo.rb
================================================
require 'uri'
require 'net/https'
require 'open-uri'
require 'json'
require 'librarian/puppet/version'
require 'librarian/puppet/source/repo'
module Librarian
module Puppet
module Source
class GitHubTarball
class Repo < Librarian::Puppet::Source::Repo
include Librarian::Puppet::Util
TOKEN_KEY = 'GITHUB_API_TOKEN'
def versions
return @versions if @versions
data = api_call("/repos/#{source.uri}/tags")
if data.nil?
raise Error, "Unable to find module '#{source.uri}' on https://github.com"
end
all_versions = data.map { |r| r['name'].gsub(/^v/, '') }.sort.reverse
all_versions.delete_if do |version|
version !~ /\A\d+\.\d+(\.\d+.*)?\z/
end
@versions = all_versions.compact
debug { " Module #{name} found versions: #{@versions.join(", ")}" }
@versions
end
def manifests
versions.map do |version|
Manifest.new(source, name, version)
end
end
def install_version!(version, install_path)
if environment.local? && !vendored?(vendored_name, version)
raise Error, "Could not find a local copy of #{source.uri} at #{version}."
end
vendor_cache(source.uri.to_s, version) unless vendored?(vendored_name, version)
cache_version_unpacked! version
if install_path.exist? && rsync? != true
install_path.rmtree
end
unpacked_path = version_unpacked_cache_path(version).children.first
cp_r(unpacked_path, install_path)
end
def cache_version_unpacked!(version)
path = version_unpacked_cache_path(version)
return if path.directory?
path.mkpath
target = vendored?(vendored_name, version) ? vendored_path(vendored_name, version) : name
Librarian::Posix.run!(%W{tar xzf #{target} -C #{path}})
end
def vendor_cache(name, version)
clean_up_old_cached_versions(vendored_name(name))
url = "https://api.github.com/repos/#{name}/tarball/#{version}"
add_api_token_to_url(url)
environment.vendor!
File.open(vendored_path(vendored_name(name), version).to_s, 'wb') do |f|
begin
debug { "Downloading <#{url}> to <#{f.path}>" }
open(url,
"User-Agent" => "librarian-puppet v#{Librarian::Puppet::VERSION}") do |res|
while buffer = res.read(8192)
f.write(buffer)
end
end
rescue OpenURI::HTTPError => e
raise e, "Error requesting <#{url}>: #{e.to_s}"
end
end
end
def clean_up_old_cached_versions(name)
Dir["#{environment.vendor_cache}/#{name}*.tar.gz"].each do |old_version|
FileUtils.rm old_version
end
end
def token_key_value
ENV[TOKEN_KEY]
end
def token_key_nil?
token_key_value.nil? || token_key_value.empty?
end
def add_api_token_to_url url
if token_key_nil?
debug { "#{TOKEN_KEY} environment value is empty or missing" }
elsif url.include? "?"
url << "&access_token=#{ENV[TOKEN_KEY]}"
else
url << "?access_token=#{ENV[TOKEN_KEY]}"
end
url
end
private
def api_call(path)
tags = []
url = "https://api.github.com#{path}?page=1&per_page=100"
while true do
debug { " Module #{name} getting tags at: #{url}" }
add_api_token_to_url(url)
response = http_get(url, :headers => {
"User-Agent" => "librarian-puppet v#{Librarian::Puppet::VERSION}"
})
code, data = response.code.to_i, response.body
if code == 200
tags.concat JSON.parse(data)
else
begin
message = JSON.parse(data)['message']
if code == 403 && message && message.include?('API rate limit exceeded')
raise Error, message + " -- increase limit by authenticating via #{TOKEN_KEY}=your-token"
elsif message
raise Error, "Error fetching #{url}: [#{code}] #{message}"
end
rescue JSON::ParserError
# response does not return json
end
raise Error, "Error fetching #{url}: [#{code}] #{response.body}"
end
# next page
break if response["link"].nil?
next_link = response["link"].split(",").select{|l| l.match /rel=.*next.*/}
break if next_link.empty?
url = next_link.first.match(/<(.*)>/)[1]
end
return tags
end
def http_get(url, options)
uri = URI.parse(url)
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
request = Net::HTTP::Get.new(uri.request_uri)
options[:headers].each { |k, v| request.add_field k, v }
http.request(request)
end
def vendored_name(name = source.uri.to_s)
name.sub('/','-')
end
end
end
end
end
end
================================================
FILE: lib/librarian/puppet/source/githubtarball.rb
================================================
require 'uri'
require 'librarian/puppet/util'
require 'librarian/puppet/source/githubtarball/repo'
module Librarian
module Puppet
module Source
class GitHubTarball
include Librarian::Puppet::Util
class << self
LOCK_NAME = 'GITHUBTARBALL'
def lock_name
LOCK_NAME
end
def from_lock_options(environment, options)
new(environment, options[:remote], options.reject { |k, v| k == :remote })
end
def from_spec_args(environment, uri, options)
recognised_options = []
unrecognised_options = options.keys - recognised_options
unless unrecognised_options.empty?
raise Error, "unrecognised options: #{unrecognised_options.join(", ")}"
end
new(environment, uri, options)
end
end
attr_accessor :environment
private :environment=
attr_reader :uri
def initialize(environment, uri, options = {})
self.environment = environment
@uri = URI::parse(uri)
@cache_path = nil
end
def to_s
clean_uri(uri).to_s
end
def ==(other)
other &&
self.class == other.class &&
self.uri == other.uri
end
alias :eql? :==
def hash
self.to_s.hash
end
def to_spec_args
[clean_uri(uri).to_s, {}]
end
def to_lock_options
{:remote => clean_uri(uri).to_s}
end
def pinned?
false
end
def unpin!
end
def install!(manifest)
manifest.source == self or raise ArgumentError
debug { "Installing #{manifest}" }
name = manifest.name
version = manifest.version
install_path = install_path(name)
repo = repo(name)
repo.install_version! version, install_path
end
def manifest(name, version, dependencies)
manifest = Manifest.new(self, name)
manifest.version = version
manifest.dependencies = dependencies
manifest
end
def cache_path
@cache_path ||= begin
environment.cache_path.join("source/puppet/githubtarball/#{uri.host}#{uri.path}")
end
end
def install_path(name)
environment.install_path.join(module_name(name))
end
def fetch_version(name, version_uri)
versions = repo(name).versions
if versions.include? version_uri
version_uri
else
versions.first
end
end
def fetch_dependencies(name, version, version_uri)
{}
end
def manifests(name)
repo(name).manifests
end
private
def repo(name)
@repo ||= {}
@repo[name] ||= Repo.new(self, name)
end
end
end
end
end
================================================
FILE: lib/librarian/puppet/source/local.rb
================================================
require 'librarian/puppet/util'
module Librarian
module Puppet
module Source
module Local
include Librarian::Puppet::Util
def install!(manifest)
manifest.source == self or raise ArgumentError
debug { "Installing #{manifest}" }
name, version = manifest.name, manifest.version
found_path = found_path(name)
raise Error, "Path for #{name} doesn't contain a puppet module" if found_path.nil?
unless name.include? '/' or name.include? '-'
warn { "Invalid module name '#{name}', you should qualify it with 'ORGANIZATION-#{name}' for resolution to work correctly" }
end
install_path = environment.install_path.join(module_name(name))
if install_path.exist? && rsync? != true
debug { "Deleting #{relative_path_to(install_path)}" }
install_path.rmtree
end
install_perform_step_copy!(found_path, install_path)
end
def fetch_version(name, extra)
cache!
found_path = found_path(name)
module_version
end
def fetch_dependencies(name, version, extra)
dependencies = Set.new
if specfile?
spec = environment.dsl(Pathname(specfile))
dependencies.merge spec.dependencies
end
parsed_metadata['dependencies'].each do |d|
gem_requirement = Librarian::Dependency::Requirement.new(d['version_requirement']).to_gem_requirement
new_dependency = Dependency.new(d['name'], gem_requirement, forge_source)
dependencies << new_dependency
end
dependencies
end
def forge_source
Forge.default
end
private
# Naming this method 'version' causes an exception to be raised.
def module_version
if parsed_metadata['version']
parsed_metadata['version']
else
warn { "Module #{to_s} does not have version, defaulting to 0.0.1" }
'0.0.1'
end
end
def require_puppet
begin
require 'puppet'
require 'puppet/module_tool'
rescue LoadError
$stderr.puts <<-EOF
Unable to load puppet, the puppet gem is required for :git and :path source.
Install it with: gem install puppet
EOF
exit 1
end
true
end
def evaluate_modulefile(modulefile)
@@require_puppet ||= require_puppet
metadata = ::Puppet::ModuleTool::Metadata.new
# Puppet 4 does not have the class
unless defined? ::Puppet::ModuleTool::ModulefileReader
warn { "Can't parse Modulefile in Puppet >= 4.0 and you are using #{Librarian::Puppet::puppet_version}. Ignoring dependencies in #{modulefile}" }
return metadata
end
begin
::Puppet::ModuleTool::ModulefileReader.evaluate(metadata, modulefile)
raise SyntaxError, "Missing version" unless metadata.version
rescue ArgumentError, SyntaxError => error
warn { "Unable to parse #{modulefile}, ignoring: #{error}" }
if metadata.respond_to? :version=
metadata.version = '0.0.1' # puppet < 3.6
else
metadata.update({'version' => '0.0.1'}) # puppet >= 3.6
end
end
metadata
end
def parsed_metadata
if @metadata.nil?
@metadata = if metadata?
begin
JSON.parse(File.read(metadata))
rescue JSON::ParserError => e
raise Error, "Unable to parse json file #{metadata}: #{e}"
end
elsif modulefile?
# translate Modulefile to metadata.json
evaluated = evaluate_modulefile(modulefile)
{
'version' => evaluated.version,
'dependencies' => evaluated.dependencies.map do |dependency|
{
'name' => dependency.instance_variable_get(:@full_module_name),
'version_requirement' => dependency.instance_variable_get(:@version_requirement)
}
end
}
else
{}
end
@metadata['dependencies'] ||= []
end
@metadata
end
def modulefile
File.join(filesystem_path, 'Modulefile')
end
def modulefile?
File.exists?(modulefile)
end
def metadata
File.join(filesystem_path, 'metadata.json')
end
def metadata?
File.exists?(metadata)
end
def specfile
File.join(filesystem_path, environment.specfile_name)
end
def specfile?
File.exists?(specfile)
end
def install_perform_step_copy!(found_path, install_path)
debug { "Copying #{relative_path_to(found_path)} to #{relative_path_to(install_path)}" }
cp_r(found_path, install_path)
end
def manifest?(name, path)
return true if path.join('manifests').exist?
return true if path.join('lib').join('puppet').exist?
return true if path.join('lib').join('facter').exist?
debug { "Could not find manifests, lib/puppet or lib/facter under #{path}, maybe it is not a puppet module" }
true
end
end
end
end
end
================================================
FILE: lib/librarian/puppet/source/path.rb
================================================
require 'librarian/source/path'
require 'librarian/puppet/source/local'
module Librarian
module Puppet
module Source
class Path < Librarian::Source::Path
include Local
end
end
end
end
================================================
FILE: lib/librarian/puppet/source/repo.rb
================================================
# parent class for githubtarball and forge source Repos
module Librarian
module Puppet
module Source
class Repo
attr_accessor :source, :name
private :source=, :name=
def initialize(source, name)
self.source = source
self.name = name
end
def environment
source.environment
end
def cache_path
@cache_path ||= source.cache_path.join(name)
end
def version_unpacked_cache_path(version)
cache_path.join(version.to_s)
end
def vendored?(name, version)
vendored_path(name, version).exist?
end
def vendored_path(name, version)
environment.vendor_cache.join("#{name}-#{version}.tar.gz")
end
end
end
end
end
================================================
FILE: lib/librarian/puppet/source.rb
================================================
require 'librarian/puppet/source/path'
require 'librarian/puppet/source/git'
require 'librarian/puppet/source/forge'
require 'librarian/puppet/source/githubtarball'
================================================
FILE: lib/librarian/puppet/templates/Puppetfile
================================================
#!/usr/bin/env ruby
#^syntax detection
forge "https://forgeapi.puppetlabs.com"
# use dependencies defined in metadata.json
metadata
# use dependencies defined in Modulefile
# modulefile
# A module from the Puppet Forge
# mod 'puppetlabs-stdlib'
# A module from git
# mod 'puppetlabs-ntp',
# :git => 'git://github.com/puppetlabs/puppetlabs-ntp.git'
# A module from a git branch/tag
# mod 'puppetlabs-apt',
# :git => 'https://github.com/puppetlabs/puppetlabs-apt.git',
# :ref => '1.4.x'
# A module from Github pre-packaged tarball
# mod 'puppetlabs-apache', '0.6.0', :github_tarball => 'puppetlabs/puppetlabs-apache'
================================================
FILE: lib/librarian/puppet/util.rb
================================================
require 'rsync'
module Librarian
module Puppet
module Util
def debug(*args, &block)
environment.logger.debug(*args, &block)
end
def info(*args, &block)
environment.logger.info(*args, &block)
end
def warn(*args, &block)
environment.logger.warn(*args, &block)
end
def rsync?
environment.config_db.local['rsync'] == 'true'
end
# workaround Issue #173 FileUtils.cp_r will fail if there is a symlink that points to a missing file
# or when the symlink is copied before the target file when preserve is true
# see also https://tickets.opscode.com/browse/CHEF-833
#
# If the rsync configuration parameter is set, use rsync instead of FileUtils
def cp_r(src, dest)
if rsync?
Rsync.run(File.join(src, "/"), dest, ['-avz', '--delete'])
else
begin
FileUtils.cp_r(src, dest, :preserve => true)
rescue Errno::ENOENT, Errno::EACCES
debug { "Failed to copy from #{src} to #{dest} preserving file types, trying again without preserving them" }
FileUtils.rm_rf(dest)
FileUtils.cp_r(src, dest)
end
end
end
# Remove user and password from a URI object
def clean_uri(uri)
new_uri = uri.clone
new_uri.user = nil
new_uri.password = nil
new_uri
end
# normalize module name to use organization-module instead of organization/module
def normalize_name(name)
name.sub('/','-')
end
# get the module name from organization-module
def module_name(name)
# module name can't have dashes, so let's assume it is everything after the last dash
name.rpartition('-').last
end
# deprecated
alias :organization_name :module_name
end
end
end
================================================
FILE: lib/librarian/puppet/version.rb
================================================
module Librarian
module Puppet
VERSION = "2.2.1"
end
end
================================================
FILE: lib/librarian/puppet.rb
================================================
require 'librarian'
require 'fileutils'
require 'librarian/puppet/extension'
require 'librarian/puppet/version'
require 'librarian/action/install'
module Librarian
module Puppet
@@puppet_version = nil
# Output of puppet --version, typically x.y.z
# For Puppet Enterprise it contains the PE version too, ie. 3.4.3 (Puppet Enterprise 3.2.1)
def puppet_version
return @@puppet_version unless @@puppet_version.nil?
begin
@@puppet_version = Librarian::Posix.run!(%W{puppet --version}).strip
rescue Errno::ENOENT, Librarian::Posix::CommandFailure => error
msg = "Unable to load puppet. Please install it using native packages for your platform (eg .deb, .rpm, .dmg, etc)."
msg += "\npuppet --version returned #{error.status}" if error.respond_to? :status
msg += "\n#{error.message}" unless error.message.nil?
$stderr.puts msg
exit 1
end
return @@puppet_version
end
# Puppet version x.y.z translated as a Gem version
def puppet_gem_version
Gem::Version.create(puppet_version.split(' ').first.strip.gsub('-', '.'))
end
end
end
================================================
FILE: librarian-puppet.gemspec
================================================
$:.push File.expand_path("../lib", __FILE__)
require 'librarian/puppet/version'
Gem::Specification.new do |s|
s.name = 'librarian-puppet'
s.version = Librarian::Puppet::VERSION
s.platform = Gem::Platform::RUBY
s.authors = ['Tim Sharpe', 'Carlos Sanchez']
s.license = 'MIT'
s.email = ['tim@sharpe.id.au', 'carlos@apache.org']
s.homepage = 'https://github.com/rodjek/librarian-puppet'
s.summary = 'Bundler for your Puppet modules'
s.description = 'Simplify deployment of your Puppet infrastructure by
automatically pulling in modules from the forge and git repositories with
a single command.'
# puppet_forge gem requires ruby 1.9 so we do too, use version 1.x in ruby 1.8
s.required_ruby_version = '>= 1.9.0'
s.files = [
'.gitignore',
'LICENSE',
'README.md',
] + Dir['{bin,lib}/**/*']
s.executables = ['librarian-puppet']
s.add_dependency "librarianp", ">=0.6.3"
s.add_dependency "rsync"
s.add_dependency "puppet_forge", "~> 1.0"
s.add_development_dependency "rake"
s.add_development_dependency "rspec"
s.add_development_dependency "cucumber"
s.add_development_dependency "aruba", "<0.8.0"
s.add_development_dependency "puppet", ENV["PUPPET_VERSION"]
s.add_development_dependency "minitest", "~> 5"
s.add_development_dependency "mocha"
s.add_development_dependency "simplecov", ">= 0.9.0"
end
================================================
FILE: spec/action/resolve_spec.rb
================================================
require 'spec_helper'
require_relative '../../lib/librarian/puppet/action/resolve'
require 'librarian/ui'
require 'thor'
describe 'Librarian::Puppet::Action::Resolve' do
let(:path) { File.expand_path("../../../features/examples/test", __FILE__) }
let(:environment) { Librarian::Puppet::Environment.new(:project_path => path) }
before do
# run with DEBUG=true envvar to get debug output
environment.ui = Librarian::UI::Shell.new(Thor::Shell::Basic.new)
end
describe '#run' do
it 'should resolve dependencies' do
Librarian::Puppet::Action::Resolve.new(environment, :force => true).run
resolution = environment.lock.manifests.map { |m| {:name => m.name, :version => m.version.to_s, :source => m.source.to_s} }
expect(resolution.size).to eq(1)
expect(resolution.first[:name]).to eq("puppetlabs-stdlib")
expect(resolution.first[:source]).to eq("https://forgeapi.puppetlabs.com")
expect(resolution.first[:version]).to match(/\d+\.\d+\.\d+/)
end
end
end
================================================
FILE: spec/librarian_puppet_spec.rb
================================================
describe Librarian::Puppet do
it 'exits with error if puppet is not installed' do
error = Librarian::Posix::CommandFailure.new 'puppet not installed'
error.status = 42
expect(Librarian::Posix).to receive(:run!).and_raise(error)
expect($stderr).to receive(:puts) do |message|
expect(message).to match /42/
expect(message).to match /puppet not installed/
end
expect { Librarian::Puppet::puppet_version }.to raise_error(SystemExit)
end
end
================================================
FILE: spec/receiver_spec.rb
================================================
require 'spec_helper'
describe 'Librarian::Puppet::Dsl::Receiver' do
let(:dsl) { Librarian::Puppet::Dsl.new({}) }
let(:target) { Librarian::Dsl::Target.new(dsl) }
let(:receiver) { Librarian::Puppet::Dsl::Receiver.new(target) }
let(:environment) { Librarian::Puppet::Environment.new(:project_path => '/tmp/tmp_module') }
describe '#run' do
it 'should get working_dir from pwd when specfile is nil' do
receiver.run(nil) {}
expect(receiver.working_path).to eq(Pathname.new(Dir.pwd))
end
it 'should get working_dir from pwd with default specfile' do
receiver.run(dsl.default_specfile) {}
expect(receiver.working_path).to eq(Pathname.new(Dir.pwd))
end
it 'should get working_dir from given path' do
receiver.run(Pathname.new('/tmp/tmp_module/Puppetfile')) {}
expect(receiver.working_path).to eq(Pathname.new('/tmp/tmp_module'))
end
it 'test receiver run' do
error_message = 'Metadata file does not exist: '+File.join(environment.project_path, 'metadata.json')
expect{environment.dsl(environment.specfile.path, [])}.to raise_error(Librarian::Error,error_message)
end
end
end
================================================
FILE: spec/source/forge_repo_spec.rb
================================================
require "librarian/puppet/source/forge"
require "librarian/puppet/environment"
describe Librarian::Puppet::Source::Forge::Repo do
let(:environment) { Librarian::Puppet::Environment.new }
let(:uri) { "https://forge.puppetlabs.com" }
let(:source) { Librarian::Puppet::Source::Forge.new(environment, uri) }
subject { Librarian::Puppet::Source::Forge::Repo.new(source, "puppetlabs/stdlib") }
describe "#check_puppet_module_options" do
context "Puppet OS" do
end
end
end
================================================
FILE: spec/source/forge_spec.rb
================================================
require "librarian/puppet/source/forge"
require "librarian/puppet/environment"
require 'librarian/puppet/extension'
include Librarian::Puppet::Source
describe Forge do
let(:environment) { Librarian::Puppet::Environment.new }
let(:uri) { "https://forge.puppetlabs.com" }
let(:puppet_version) { "3.6.0" }
subject { Forge.new(environment, uri) }
describe "#manifests" do
let(:manifests) { [] }
before do
expect_any_instance_of(Librarian::Puppet::Source::Forge::RepoV3).to receive(:get_versions).at_least(:once) { manifests }
end
it "should return the manifests" do
expect(subject.manifests("x")).to eq(manifests)
end
end
describe "#check_puppet_module_options" do
before do
expect(Librarian::Puppet).to receive(:puppet_version).at_least(:once) { puppet_version }
expect(Librarian::Puppet).to receive(:puppet_gem_version).at_least(:once) { Gem::Version.create(puppet_version.split(' ').first.strip.gsub('-', '.')) }
end
context "Puppet OS" do
context "3.4.3" do
let(:puppet_version) { "3.4.3" }
it { Forge.client_api_version().should == 1 }
end
context "2.7.13" do
let(:puppet_version) { "2.7.13" }
it { Forge.client_api_version().should == 1 }
end
context "3.6.0" do
it { Forge.client_api_version().should == 3 }
end
end
context "Puppet Enterprise" do
context "3.2.1" do
let(:puppet_version) { "3.4.3 (Puppet Enterprise 3.2.1)" }
it { Forge.client_api_version().should == 3 }
end
context "3.1.3" do
let(:puppet_version) { "3.3.3 (Puppet Enterprise 3.1.3)" }
it { Forge.client_api_version().should == 1 }
end
end
end
end
================================================
FILE: spec/spec_helper.rb
================================================
require 'simplecov'
SimpleCov.start
require 'rubygems'
require 'rspec'
require 'librarian/puppet'
================================================
FILE: spec/util_spec.rb
================================================
require 'spec_helper'
describe Librarian::Puppet::Util do
subject { Class.new { include Librarian::Puppet::Util }.new }
it 'should get organization name' do
expect(subject.module_name('puppetlabs-xy')).to eq('xy')
end
it 'should get organization name when org contains dashes' do
expect(subject.module_name('puppet-labs-xy')).to eq('xy')
end
end
================================================
FILE: test/librarian/puppet/source/githubtarball_test.rb
================================================
require File.expand_path("../../../../test_helper", __FILE__)
require "librarian/puppet/source/githubtarball"
describe Librarian::Puppet::Source::GitHubTarball::Repo do
def assert_exact_error(klass, message)
yield
rescue Exception => e
e.class.must_equal klass
e.message.must_equal message
else
raise "No exception was raised!"
end
class FakeResponse
attr_accessor :code,:body
def initialize(code,body)
@code = code
@body = body
end
def [](key)
nil
end
end
describe "#api_call" do
let(:environment) { Librarian::Puppet::Environment.new }
let(:source) { Librarian::Puppet::Source::GitHubTarball.new(environment, "foo") }
let(:repo) { Librarian::Puppet::Source::GitHubTarball::Repo.new(source, "bar") }
let(:headers) { {'User-Agent' => "librarian-puppet v#{Librarian::Puppet::VERSION}"} }
let(:url) { "https://api.github.com/foo?page=1&per_page=100" }
let(:url_with_token) { "https://api.github.com/foo?page=1&per_page=100&access_token=bar" }
ENV['GITHUB_API_TOKEN'] = ''
it "succeeds" do
response = []
repo.expects(:http_get).with(url, {:headers => headers}).returns(FakeResponse.new(200, JSON.dump(response)))
repo.send(:api_call, "/foo").must_equal(response)
end
it "adds GITHUB_API_TOKEN if present" do
ENV['GITHUB_API_TOKEN'] = 'bar'
response = []
repo.expects(:http_get).with(url_with_token, {:headers => headers}).returns(FakeResponse.new(200, JSON.dump(response)))
repo.send(:api_call, "/foo").must_equal(response)
ENV['GITHUB_API_TOKEN'] = ''
end
it "fails when we hit api limit" do
response = {"message" => "Oh boy! API rate limit exceeded!!!"}
repo.expects(:http_get).with(url, {:headers => headers}).returns(FakeResponse.new(403, JSON.dump(response)))
message = "Oh boy! API rate limit exceeded!!! -- increase limit by authenticating via GITHUB_API_TOKEN=your-token"
assert_exact_error Librarian::Error, message do
repo.send(:api_call, "/foo")
end
end
it "fails with unknown error message" do
repo.expects(:http_get).with(url, {:headers => headers}).returns(FakeResponse.new(403, ""))
assert_exact_error Librarian::Error, "Error fetching #{url}: [403] " do
repo.send(:api_call, "/foo")
end
end
it "fails with html" do
repo.expects(:http_get).with(url, {:headers => headers}).returns(FakeResponse.new(403, "<html>Oh boy!</html>"))
assert_exact_error Librarian::Error, "Error fetching #{url}: [403] <html>Oh boy!</html>" do
repo.send(:api_call, "/foo")
end
end
it "fails with unknown code" do
repo.expects(:http_get).with(url, {:headers => headers}).returns(FakeResponse.new(500, ""))
assert_exact_error Librarian::Error, "Error fetching #{url}: [500] " do
repo.send(:api_call, "/foo")
end
end
end
end
================================================
FILE: test/test_helper.rb
================================================
require 'bundler/setup'
require 'minitest/autorun'
require 'minitest/spec'
require 'mocha/setup'
$LOAD_PATH << "vendor/librarian/lib"
require 'librarian/puppet'
gitextract_bq1yps9j/
├── .gitignore
├── .travis.yml
├── CONTRIBUTING.md
├── Changelog.md
├── Gemfile
├── LICENSE
├── README.md
├── Rakefile
├── bin/
│ └── librarian-puppet
├── features/
│ ├── examples/
│ │ ├── dependency_without_version/
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ ├── duplicated_dependencies/
│ │ │ ├── Puppetfile
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ ├── duplicated_dependencies_transitive/
│ │ │ ├── Puppetfile
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ ├── metadata_syntax/
│ │ │ ├── Puppetfile
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ ├── modulefile_syntax/
│ │ │ ├── Modulefile
│ │ │ ├── Puppetfile
│ │ │ └── manifests/
│ │ │ └── init.pp
│ │ ├── path_dependencies/
│ │ │ ├── Puppetfile
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ ├── test/
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ ├── with_puppetfile/
│ │ │ ├── Puppetfile
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ ├── with_puppetfile_and_metadata_json/
│ │ │ ├── Puppetfile
│ │ │ ├── manifests/
│ │ │ │ └── init.pp
│ │ │ └── metadata.json
│ │ └── with_puppetfile_and_modulefile/
│ │ ├── Modulefile
│ │ ├── Puppetfile
│ │ └── manifests/
│ │ └── init.pp
│ ├── help.feature
│ ├── init.feature
│ ├── install/
│ │ ├── forge.feature
│ │ ├── git.feature
│ │ ├── github_tarball.feature
│ │ └── path.feature
│ ├── install.feature
│ ├── outdated.feature
│ ├── package.feature
│ ├── step_definitions/
│ │ └── convergence_steps.rb
│ ├── support/
│ │ └── env.rb
│ ├── update.feature
│ └── version.feature
├── lib/
│ └── librarian/
│ ├── puppet/
│ │ ├── action/
│ │ │ ├── install.rb
│ │ │ └── resolve.rb
│ │ ├── action.rb
│ │ ├── cli.rb
│ │ ├── dependency.rb
│ │ ├── dsl.rb
│ │ ├── environment.rb
│ │ ├── extension.rb
│ │ ├── lockfile.rb
│ │ ├── source/
│ │ │ ├── forge/
│ │ │ │ ├── repo.rb
│ │ │ │ ├── repo_v1.rb
│ │ │ │ └── repo_v3.rb
│ │ │ ├── forge.rb
│ │ │ ├── git.rb
│ │ │ ├── githubtarball/
│ │ │ │ └── repo.rb
│ │ │ ├── githubtarball.rb
│ │ │ ├── local.rb
│ │ │ ├── path.rb
│ │ │ └── repo.rb
│ │ ├── source.rb
│ │ ├── templates/
│ │ │ └── Puppetfile
│ │ ├── util.rb
│ │ └── version.rb
│ └── puppet.rb
├── librarian-puppet.gemspec
├── spec/
│ ├── action/
│ │ └── resolve_spec.rb
│ ├── librarian_puppet_spec.rb
│ ├── receiver_spec.rb
│ ├── source/
│ │ ├── forge_repo_spec.rb
│ │ └── forge_spec.rb
│ ├── spec_helper.rb
│ └── util_spec.rb
└── test/
├── librarian/
│ └── puppet/
│ └── source/
│ └── githubtarball_test.rb
└── test_helper.rb
SYMBOL INDEX (237 symbols across 22 files)
FILE: lib/librarian/puppet.rb
type Librarian (line 9) | module Librarian
type Puppet (line 10) | module Puppet
function puppet_version (line 15) | def puppet_version
function puppet_gem_version (line 31) | def puppet_gem_version
FILE: lib/librarian/puppet/action/install.rb
type Librarian (line 3) | module Librarian
type Puppet (line 4) | module Puppet
type Action (line 5) | module Action
class Install (line 6) | class Install < Librarian::Action::Install
method create_install_path (line 10) | def create_install_path
method destructive? (line 15) | def destructive?
method check_specfile (line 19) | def check_specfile
FILE: lib/librarian/puppet/action/resolve.rb
type Librarian (line 3) | module Librarian
type Puppet (line 4) | module Puppet
type Action (line 5) | module Action
class Resolve (line 6) | class Resolve < Librarian::Action::Resolve
method run (line 9) | def run
FILE: lib/librarian/puppet/cli.rb
type Librarian (line 7) | module Librarian
type Puppet (line 8) | module Puppet
class Cli (line 9) | class Cli < Librarian::Cli
type Particularity (line 12) | module Particularity
function root_module (line 13) | def root_module
method init (line 23) | def init
method install (line 50) | def install
method update (line 74) | def update(*names)
method package (line 87) | def package
method version (line 92) | def version
method install! (line 100) | def install!(options = { })
method resolve! (line 103) | def resolve!(options = { })
FILE: lib/librarian/puppet/dependency.rb
type Librarian (line 1) | module Librarian
type Puppet (line 2) | module Puppet
class Dependency (line 4) | class Dependency < Librarian::Dependency
method initialize (line 8) | def initialize(name, requirement, source)
FILE: lib/librarian/puppet/dsl.rb
type Librarian (line 6) | module Librarian
type Puppet (line 7) | module Puppet
class Dsl (line 8) | class Dsl < Librarian::Dsl
method default_specfile (line 19) | def default_specfile
method dependency_type (line 26) | def self.dependency_type
method post_process_target (line 30) | def post_process_target(target)
method receiver (line 36) | def receiver(target)
class Receiver (line 40) | class Receiver < Librarian::Dsl::Receiver
method run (line 44) | def run(specfile = nil)
method modulefile (line 51) | def modulefile
method metadata (line 61) | def metadata
method modulefile_path (line 86) | def modulefile_path
FILE: lib/librarian/puppet/environment.rb
type Librarian (line 6) | module Librarian
type Puppet (line 7) | module Puppet
class Environment (line 8) | class Environment < Librarian::Environment
method adapter_name (line 10) | def adapter_name
method lockfile (line 14) | def lockfile
method ephemeral_lockfile (line 18) | def ephemeral_lockfile
method tmp_path (line 22) | def tmp_path
method install_path (line 27) | def install_path
method vendor_path (line 32) | def vendor_path
method vendor_cache (line 36) | def vendor_cache
method vendor_source (line 40) | def vendor_source
method vendor! (line 44) | def vendor!
method vendor? (line 49) | def vendor?
method local? (line 53) | def local?
method use_v1_api (line 57) | def use_v1_api
FILE: lib/librarian/puppet/extension.rb
type Librarian (line 4) | module Librarian
type Puppet (line 5) | module Puppet
FILE: lib/librarian/puppet/lockfile.rb
type Librarian (line 2) | module Librarian
type Puppet (line 3) | module Puppet
class Lockfile (line 4) | class Lockfile < Librarian::Lockfile
class Parser (line 7) | class Parser < Librarian::Lockfile::Parser
method extract_and_parse_sources (line 11) | def extract_and_parse_sources(lines)
method extract_and_parse_dependencies (line 21) | def extract_and_parse_dependencies(lines, manifests_index)
method load (line 33) | def load(string)
FILE: lib/librarian/puppet/source/forge.rb
type Librarian (line 6) | module Librarian
type Puppet (line 7) | module Puppet
type Source (line 8) | module Source
class Forge (line 9) | class Forge
method default= (line 15) | def default=(source)
method default (line 19) | def default
method lock_name (line 23) | def lock_name
method from_lock_options (line 27) | def from_lock_options(environment, options)
method from_spec_args (line 31) | def from_spec_args(environment, uri, options)
method client_api_version (line 41) | def client_api_version()
method initialize (line 61) | def initialize(environment, uri, options = {})
method to_s (line 73) | def to_s
method == (line 77) | def ==(other)
method hash (line 85) | def hash
method to_spec_args (line 89) | def to_spec_args
method to_lock_options (line 93) | def to_lock_options
method pinned? (line 97) | def pinned?
method unpin! (line 101) | def unpin!
method install! (line 104) | def install!(manifest)
method manifest (line 117) | def manifest(name, version, dependencies)
method cache_path (line 124) | def cache_path
method install_path (line 131) | def install_path(name)
method fetch_version (line 135) | def fetch_version(name, version_uri)
method fetch_dependencies (line 144) | def fetch_dependencies(name, version, version_uri)
method manifests (line 151) | def manifests(name)
method repo (line 157) | def repo(name)
FILE: lib/librarian/puppet/source/forge/repo.rb
type Librarian (line 6) | module Librarian
type Puppet (line 7) | module Puppet
type Source (line 8) | module Source
class Forge (line 9) | class Forge
class Repo (line 10) | class Repo < Librarian::Puppet::Source::Repo
method versions (line 13) | def versions
method get_versions (line 25) | def get_versions
method dependencies (line 31) | def dependencies(version)
method url (line 37) | def url(name, version)
method manifests (line 41) | def manifests
method install_version! (line 47) | def install_version!(version, install_path)
method cache_version_unpacked! (line 72) | def cache_version_unpacked!(version)
method check_puppet_module_options (line 134) | def check_puppet_module_options
method vendor_cache (line 142) | def vendor_cache(name, version)
FILE: lib/librarian/puppet/source/forge/repo_v1.rb
type Librarian (line 5) | module Librarian
type Puppet (line 6) | module Puppet
type Source (line 7) | module Source
class Forge (line 8) | class Forge
class RepoV1 (line 9) | class RepoV1 < Librarian::Puppet::Source::Forge::Repo
method initialize (line 11) | def initialize(source, name)
method get_versions (line 21) | def get_versions
method dependencies (line 25) | def dependencies(version)
method url (line 29) | def url(name, version)
method normalize_dependencies (line 37) | def normalize_dependencies(data)
method api_data (line 50) | def api_data(module_name)
method api_version_data (line 61) | def api_version_data(module_name, version)
method api_call (line 69) | def api_call(module_name, version=nil)
FILE: lib/librarian/puppet/source/forge/repo_v3.rb
type Librarian (line 5) | module Librarian
type Puppet (line 6) | module Puppet
type Source (line 7) | module Source
class Forge (line 8) | class Forge
class RepoV3 (line 9) | class RepoV3 < Librarian::Puppet::Source::Forge::Repo
method initialize (line 13) | def initialize(source, name)
method get_versions (line 18) | def get_versions
method dependencies (line 22) | def dependencies(version)
method url (line 27) | def url(name, version)
method get_module (line 40) | def get_module
method get_release (line 46) | def get_release(version)
FILE: lib/librarian/puppet/source/git.rb
type Librarian (line 4) | module Librarian
type Source (line 5) | module Source
class Git (line 6) | class Git
class Repository (line 7) | class Repository
method hash_from (line 8) | def hash_from(remote, reference)
type Puppet (line 21) | module Puppet
type Source (line 22) | module Source
class Git (line 23) | class Git < Librarian::Source::Git
method cache! (line 27) | def cache!
method vendor_tar (line 45) | def vendor_tar
method vendor_tgz (line 49) | def vendor_tgz
method vendor_cached? (line 53) | def vendor_cached?
method vendor_checkout! (line 57) | def vendor_checkout!
method cache_in_vendor (line 66) | def cache_in_vendor(tmp_path)
FILE: lib/librarian/puppet/source/githubtarball.rb
type Librarian (line 5) | module Librarian
type Puppet (line 6) | module Puppet
type Source (line 7) | module Source
class GitHubTarball (line 8) | class GitHubTarball
method lock_name (line 14) | def lock_name
method from_lock_options (line 18) | def from_lock_options(environment, options)
method from_spec_args (line 22) | def from_spec_args(environment, uri, options)
method initialize (line 37) | def initialize(environment, uri, options = {})
method to_s (line 43) | def to_s
method == (line 47) | def ==(other)
method hash (line 55) | def hash
method to_spec_args (line 59) | def to_spec_args
method to_lock_options (line 63) | def to_lock_options
method pinned? (line 67) | def pinned?
method unpin! (line 71) | def unpin!
method install! (line 74) | def install!(manifest)
method manifest (line 87) | def manifest(name, version, dependencies)
method cache_path (line 94) | def cache_path
method install_path (line 100) | def install_path(name)
method fetch_version (line 104) | def fetch_version(name, version_uri)
method fetch_dependencies (line 113) | def fetch_dependencies(name, version, version_uri)
method manifests (line 117) | def manifests(name)
method repo (line 123) | def repo(name)
FILE: lib/librarian/puppet/source/githubtarball/repo.rb
type Librarian (line 9) | module Librarian
type Puppet (line 10) | module Puppet
type Source (line 11) | module Source
class GitHubTarball (line 12) | class GitHubTarball
class Repo (line 13) | class Repo < Librarian::Puppet::Source::Repo
method versions (line 18) | def versions
method manifests (line 36) | def manifests
method install_version! (line 42) | def install_version!(version, install_path)
method cache_version_unpacked! (line 59) | def cache_version_unpacked!(version)
method vendor_cache (line 70) | def vendor_cache(name, version)
method clean_up_old_cached_versions (line 92) | def clean_up_old_cached_versions(name)
method token_key_value (line 98) | def token_key_value
method token_key_nil? (line 102) | def token_key_nil?
method add_api_token_to_url (line 106) | def add_api_token_to_url url
method api_call (line 119) | def api_call(path)
method http_get (line 156) | def http_get(url, options)
method vendored_name (line 165) | def vendored_name(name = source.uri.to_s)
FILE: lib/librarian/puppet/source/local.rb
type Librarian (line 3) | module Librarian
type Puppet (line 4) | module Puppet
type Source (line 5) | module Source
type Local (line 6) | module Local
function install! (line 9) | def install!(manifest)
function fetch_version (line 31) | def fetch_version(name, extra)
function fetch_dependencies (line 37) | def fetch_dependencies(name, version, extra)
function forge_source (line 54) | def forge_source
function module_version (line 61) | def module_version
function require_puppet (line 70) | def require_puppet
function evaluate_modulefile (line 84) | def evaluate_modulefile(modulefile)
function parsed_metadata (line 109) | def parsed_metadata
function modulefile (line 137) | def modulefile
function modulefile? (line 141) | def modulefile?
function metadata (line 145) | def metadata
function metadata? (line 149) | def metadata?
function specfile (line 153) | def specfile
function specfile? (line 157) | def specfile?
function install_perform_step_copy! (line 161) | def install_perform_step_copy!(found_path, install_path)
function manifest? (line 166) | def manifest?(name, path)
FILE: lib/librarian/puppet/source/path.rb
type Librarian (line 4) | module Librarian
type Puppet (line 5) | module Puppet
type Source (line 6) | module Source
class Path (line 7) | class Path < Librarian::Source::Path
FILE: lib/librarian/puppet/source/repo.rb
type Librarian (line 2) | module Librarian
type Puppet (line 3) | module Puppet
type Source (line 4) | module Source
class Repo (line 5) | class Repo
method initialize (line 10) | def initialize(source, name)
method environment (line 15) | def environment
method cache_path (line 19) | def cache_path
method version_unpacked_cache_path (line 23) | def version_unpacked_cache_path(version)
method vendored? (line 27) | def vendored?(name, version)
method vendored_path (line 31) | def vendored_path(name, version)
FILE: lib/librarian/puppet/util.rb
type Librarian (line 3) | module Librarian
type Puppet (line 4) | module Puppet
type Util (line 6) | module Util
function debug (line 8) | def debug(*args, &block)
function info (line 11) | def info(*args, &block)
function warn (line 14) | def warn(*args, &block)
function rsync? (line 18) | def rsync?
function cp_r (line 27) | def cp_r(src, dest)
function clean_uri (line 42) | def clean_uri(uri)
function normalize_name (line 50) | def normalize_name(name)
function module_name (line 55) | def module_name(name)
FILE: lib/librarian/puppet/version.rb
type Librarian (line 1) | module Librarian
type Puppet (line 2) | module Puppet
FILE: test/librarian/puppet/source/githubtarball_test.rb
function assert_exact_error (line 5) | def assert_exact_error(klass, message)
class FakeResponse (line 14) | class FakeResponse
method initialize (line 16) | def initialize(code,body)
method [] (line 20) | def [](key)
Condensed preview — 84 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (149K chars).
[
{
"path": ".gitignore",
"chars": 33,
"preview": "pkg/\nGemfile.lock\ntmp/\ncoverage/\n"
},
{
"path": ".travis.yml",
"chars": 1540,
"preview": "sudo: false\nrvm:\n - '2.2'\n - '2.1'\n - '2.0'\n - '1.9'\nnotifications:\n email:\n - carlos@apache.org\n - tim@githu"
},
{
"path": "CONTRIBUTING.md",
"chars": 99,
"preview": "# Librarian-puppet\n\n# NOTE this project has moved to https://github.com/voxpupuli/librarian-puppet\n"
},
{
"path": "Changelog.md",
"chars": 12880,
"preview": "# Changelog\n\n## From 2.x Librarian-Puppet requires Ruby >= 1.9, uses Puppet Forge API v3. For Ruby 1.8 use 1.x\n\n### 2.2."
},
{
"path": "Gemfile",
"chars": 38,
"preview": "source 'https://rubygems.org'\ngemspec\n"
},
{
"path": "LICENSE",
"chars": 1086,
"preview": "Copyright (c) 2012-2014 Tim Sharpe, Carlos Sanchez and others\n\nPermission is hereby granted, free of charge, to any pers"
},
{
"path": "README.md",
"chars": 12019,
"preview": "# Librarian-puppet\n\n# NOTE This project has moved to https://github.com/voxpupuli/librarian-puppet\n\n[![Build Status](htt"
},
{
"path": "Rakefile",
"chars": 1278,
"preview": "require 'bundler/setup'\nrequire 'cucumber/rake/task'\nrequire 'rspec/core/rake_task'\nrequire 'bundler/gem_tasks'\nrequire "
},
{
"path": "bin/librarian-puppet",
"chars": 167,
"preview": "#!/usr/bin/env ruby\n\nlib = File.expand_path('../../lib', __FILE__)\n$:.unshift(lib) unless $:.include?(lib)\n\nrequire 'lib"
},
{
"path": "features/examples/dependency_without_version/manifests/init.pp",
"chars": 14,
"preview": "class test {}\n"
},
{
"path": "features/examples/dependency_without_version/metadata.json",
"chars": 151,
"preview": "{\n \"name\": \"librarian-test\",\n \"version\": \"0.0.1\",\n \"license\": \"Apache 2.0\",\n \"dependencies\": [\n {\n \"name\": \""
},
{
"path": "features/examples/duplicated_dependencies/Puppetfile",
"chars": 46,
"preview": "forge 'http://forge.puppetlabs.com'\n\nmetadata\n"
},
{
"path": "features/examples/duplicated_dependencies/manifests/init.pp",
"chars": 14,
"preview": "class test {}\n"
},
{
"path": "features/examples/duplicated_dependencies/metadata.json",
"chars": 291,
"preview": "{\n \"name\": \"librarian-duplicated_dependencies\",\n \"version\": \"0.0.1\",\n \"license\": \"Apache 2.0\",\n \"dependencies\": [\n "
},
{
"path": "features/examples/duplicated_dependencies_transitive/Puppetfile",
"chars": 199,
"preview": "forge 'http://forge.puppetlabs.com'\n\nmetadata\n\nmod 'librarian-duplicated_dependencies', :git => 'https://github.com/rodj"
},
{
"path": "features/examples/duplicated_dependencies_transitive/manifests/init.pp",
"chars": 14,
"preview": "class test {}\n"
},
{
"path": "features/examples/duplicated_dependencies_transitive/metadata.json",
"chars": 132,
"preview": "{\n \"name\": \"librarian-duplicated_dependencies_transitive\",\n \"version\": \"0.0.1\",\n \"license\": \"Apache 2.0\",\n \"dependen"
},
{
"path": "features/examples/metadata_syntax/Puppetfile",
"chars": 46,
"preview": "forge 'http://forge.puppetlabs.com'\n\nmetadata\n"
},
{
"path": "features/examples/metadata_syntax/manifests/init.pp",
"chars": 14,
"preview": "class test {}\n"
},
{
"path": "features/examples/metadata_syntax/metadata.json",
"chars": 1790,
"preview": "{\n \"operatingsystem_support\": [\n {\n \"operatingsystem\": \"RedHat\",\n \"operatingsystemrelease\": [\n \"4\","
},
{
"path": "features/examples/modulefile_syntax/Modulefile",
"chars": 139,
"preview": "name 'librarian-modulefile_syntax'\nversion '0.0.1'\n\nauthor 'librarian'\nlicense 'Apache License, Version 2.0'\n\ndependency"
},
{
"path": "features/examples/modulefile_syntax/Puppetfile",
"chars": 48,
"preview": "forge 'http://forge.puppetlabs.com'\n\nmodulefile\n"
},
{
"path": "features/examples/modulefile_syntax/manifests/init.pp",
"chars": 14,
"preview": "class test {}\n"
},
{
"path": "features/examples/path_dependencies/Puppetfile",
"chars": 62,
"preview": "mod 'librarian/test', :path => '../../features/examples/test'\n"
},
{
"path": "features/examples/path_dependencies/manifests/init.pp",
"chars": 14,
"preview": "class test {}\n"
},
{
"path": "features/examples/path_dependencies/metadata.json",
"chars": 137,
"preview": "{\n \"name\": \"librarian-path_dependencies\",\n \"version\": \"0.0.1\",\n \"license\": \"Apache 2.0\",\n \"dependencies\": [],\n \"req"
},
{
"path": "features/examples/test/manifests/init.pp",
"chars": 14,
"preview": "class test {}\n"
},
{
"path": "features/examples/test/metadata.json",
"chars": 188,
"preview": "{\n \"name\": \"librarian-test\",\n \"version\": \"0.0.1\",\n \"license\": \"Apache 2.0\",\n \"dependencies\": [\n {\n \"name\": \""
},
{
"path": "features/examples/with_puppetfile/Puppetfile",
"chars": 114,
"preview": "mod 'librarian/test', :git => 'https://github.com/rodjek/librarian-puppet.git', :path => 'features/examples/test'\n"
},
{
"path": "features/examples/with_puppetfile/manifests/init.pp",
"chars": 14,
"preview": "class test {}\n"
},
{
"path": "features/examples/with_puppetfile/metadata.json",
"chars": 113,
"preview": "{\n \"name\": \"librarian-with_puppetfile\",\n \"version\": \"0.0.1\",\n \"license\": \"Apache 2.0\",\n \"dependencies\": []\n}\n"
},
{
"path": "features/examples/with_puppetfile_and_metadata_json/Puppetfile",
"chars": 59,
"preview": "forge 'http://forge.puppetlabs.com'\n\nmod 'maestrodev/test'\n"
},
{
"path": "features/examples/with_puppetfile_and_metadata_json/manifests/init.pp",
"chars": 14,
"preview": "class test {}\n"
},
{
"path": "features/examples/with_puppetfile_and_metadata_json/metadata.json",
"chars": 215,
"preview": "{\n \"name\": \"librarian-with_puppetfile_and_metadata_json\",\n \"version\": \"0.0.1\",\n \"license\": \"Apache 2.0\",\n \"dependenc"
},
{
"path": "features/examples/with_puppetfile_and_modulefile/Modulefile",
"chars": 152,
"preview": "name 'librarian-with_puppetfile_and_modulefile'\nversion '0.0.1'\n\nauthor 'librarian'\nlicense 'Apache License, Version 2.0"
},
{
"path": "features/examples/with_puppetfile_and_modulefile/Puppetfile",
"chars": 59,
"preview": "forge 'http://forge.puppetlabs.com'\n\nmod 'maestrodev/test'\n"
},
{
"path": "features/examples/with_puppetfile_and_modulefile/manifests/init.pp",
"chars": 14,
"preview": "class test {}\n"
},
{
"path": "features/help.feature",
"chars": 441,
"preview": "Feature: displays help if no subcommand is passed\n In order to get started using librarian-puppet\n A user should be ab"
},
{
"path": "features/init.feature",
"chars": 450,
"preview": "Feature: init subcommand should generate a Puppetfile\n In order to start using librarian-puppet in a project\n A projec"
},
{
"path": "features/install/forge.feature",
"chars": 10201,
"preview": "Feature: cli/install/forge\n Puppet librarian needs to install modules from the Puppet Forge\n\n Scenario: Installing a m"
},
{
"path": "features/install/git.feature",
"chars": 9259,
"preview": "Feature: cli/install/git\n Puppet librarian needs to install modules from git repositories\n\n Scenario: Installing a mod"
},
{
"path": "features/install/github_tarball.feature",
"chars": 1501,
"preview": "Feature: cli/install/github_tarball\n Puppet librarian needs to install tarballed modules from github repositories\n\n @g"
},
{
"path": "features/install/path.feature",
"chars": 3112,
"preview": "Feature: cli/install/path\n Puppet librarian needs to install modules from local paths\n\n Scenario: Install a module wit"
},
{
"path": "features/install.feature",
"chars": 6425,
"preview": "Feature: cli/install\n In order to be worth anything\n Puppet librarian needs to install modules properly\n\n Scenario: R"
},
{
"path": "features/outdated.feature",
"chars": 1547,
"preview": "Feature: cli/outdated\n Puppet librarian needs to print outdated modules\n\n Scenario: Running outdated with forge module"
},
{
"path": "features/package.feature",
"chars": 2155,
"preview": "Feature: cli/package\n Puppet librarian needs to package modules\n\n Scenario: Packaging a forge module\n Given a file "
},
{
"path": "features/step_definitions/convergence_steps.rb",
"chars": 1277,
"preview": "Then /^the file \"([^\"]*)\" should have an inode and ctime$/ do |file|\n cd('.') do\n stat = File.stat(File.expand"
},
{
"path": "features/support/env.rb",
"chars": 432,
"preview": "require 'aruba/cucumber'\nrequire 'fileutils'\n\nBefore do\n @aruba_timeout_seconds = 120\nend\n\nBefore('@spaces') do\n @dirs"
},
{
"path": "features/update.feature",
"chars": 9637,
"preview": "Feature: cli/update\n Puppet librarian needs to update modules properly\n\n Scenario: Updating a module with no Puppetfil"
},
{
"path": "features/version.feature",
"chars": 180,
"preview": "Feature: cli/version\n\n Scenario: Getting the version\n When I run `librarian-puppet version`\n Then the exit status"
},
{
"path": "lib/librarian/puppet/action/install.rb",
"chars": 563,
"preview": "require 'librarian/action/install'\n\nmodule Librarian\n module Puppet\n module Action\n class Install < Librarian::"
},
{
"path": "lib/librarian/puppet/action/resolve.rb",
"chars": 586,
"preview": "require 'librarian/action/resolve'\n\nmodule Librarian\n module Puppet\n module Action\n class Resolve < Librarian::"
},
{
"path": "lib/librarian/puppet/action.rb",
"chars": 84,
"preview": "require \"librarian/puppet/action/install\"\nrequire \"librarian/puppet/action/resolve\"\n"
},
{
"path": "lib/librarian/puppet/cli.rb",
"chars": 3419,
"preview": "require 'librarian/helpers'\n\nrequire 'librarian/cli'\nrequire 'librarian/puppet'\nrequire 'librarian/puppet/action'\n\nmodul"
},
{
"path": "lib/librarian/puppet/dependency.rb",
"chars": 423,
"preview": "module Librarian\n module Puppet\n\n class Dependency < Librarian::Dependency\n\n include Librarian::Puppet::Util\n\n "
},
{
"path": "lib/librarian/puppet/dsl.rb",
"chars": 2712,
"preview": "require 'librarian/dsl'\nrequire 'librarian/dsl/target'\nrequire 'librarian/puppet/source'\nrequire 'librarian/puppet/depen"
},
{
"path": "lib/librarian/puppet/environment.rb",
"chars": 1184,
"preview": "require \"librarian/environment\"\nrequire \"librarian/puppet/dsl\"\nrequire \"librarian/puppet/source\"\nrequire \"librarian/pupp"
},
{
"path": "lib/librarian/puppet/extension.rb",
"chars": 152,
"preview": "require 'librarian/puppet/environment'\nrequire 'librarian/action/base'\n\nmodule Librarian\n module Puppet\n extend self"
},
{
"path": "lib/librarian/puppet/lockfile.rb",
"chars": 1136,
"preview": "# Extend Lockfile to normalize module names from acme/mod to acme-mod\nmodule Librarian\n module Puppet\n class Lockfil"
},
{
"path": "lib/librarian/puppet/source/forge/repo.rb",
"chars": 6065,
"preview": "require 'json'\nrequire 'open-uri'\nrequire 'librarian/puppet/util'\nrequire 'librarian/puppet/source/repo'\n\nmodule Librari"
},
{
"path": "lib/librarian/puppet/source/forge/repo_v1.rb",
"chars": 3436,
"preview": "require 'json'\nrequire 'open-uri'\nrequire 'librarian/puppet/source/forge/repo'\n\nmodule Librarian\n module Puppet\n mod"
},
{
"path": "lib/librarian/puppet/source/forge/repo_v3.rb",
"chars": 1904,
"preview": "require 'librarian/puppet/source/forge/repo'\nrequire 'puppet_forge'\nrequire 'librarian/puppet/version'\n\nmodule Librarian"
},
{
"path": "lib/librarian/puppet/source/forge.rb",
"chars": 4496,
"preview": "require 'uri'\nrequire 'librarian/puppet/util'\nrequire 'librarian/puppet/source/forge/repo_v1'\nrequire 'librarian/puppet/"
},
{
"path": "lib/librarian/puppet/source/git.rb",
"chars": 1904,
"preview": "require 'librarian/source/git'\nrequire 'librarian/puppet/source/local'\n\nmodule Librarian\n module Source\n class Git\n "
},
{
"path": "lib/librarian/puppet/source/githubtarball/repo.rb",
"chars": 5622,
"preview": "require 'uri'\nrequire 'net/https'\nrequire 'open-uri'\nrequire 'json'\n\nrequire 'librarian/puppet/version'\nrequire 'librari"
},
{
"path": "lib/librarian/puppet/source/githubtarball.rb",
"chars": 3003,
"preview": "require 'uri'\nrequire 'librarian/puppet/util'\nrequire 'librarian/puppet/source/githubtarball/repo'\n\nmodule Librarian\n m"
},
{
"path": "lib/librarian/puppet/source/local.rb",
"chars": 5570,
"preview": "require 'librarian/puppet/util'\n\nmodule Librarian\n module Puppet\n module Source\n module Local\n include L"
},
{
"path": "lib/librarian/puppet/source/path.rb",
"chars": 217,
"preview": "require 'librarian/source/path'\nrequire 'librarian/puppet/source/local'\n\nmodule Librarian\n module Puppet\n module Sou"
},
{
"path": "lib/librarian/puppet/source/repo.rb",
"chars": 810,
"preview": "# parent class for githubtarball and forge source Repos\nmodule Librarian\n module Puppet\n module Source\n class R"
},
{
"path": "lib/librarian/puppet/source.rb",
"chars": 165,
"preview": "require 'librarian/puppet/source/path'\nrequire 'librarian/puppet/source/git'\nrequire 'librarian/puppet/source/forge'\nreq"
},
{
"path": "lib/librarian/puppet/templates/Puppetfile",
"chars": 628,
"preview": "#!/usr/bin/env ruby\n#^syntax detection\n\nforge \"https://forgeapi.puppetlabs.com\"\n\n# use dependencies defined in metadata."
},
{
"path": "lib/librarian/puppet/util.rb",
"chars": 1882,
"preview": "require 'rsync'\n\nmodule Librarian\n module Puppet\n\n module Util\n\n def debug(*args, &block)\n environment.l"
},
{
"path": "lib/librarian/puppet/version.rb",
"chars": 65,
"preview": "module Librarian\n module Puppet\n VERSION = \"2.2.1\"\n end\nend\n"
},
{
"path": "lib/librarian/puppet.rb",
"chars": 1146,
"preview": "require 'librarian'\nrequire 'fileutils'\n\nrequire 'librarian/puppet/extension'\nrequire 'librarian/puppet/version'\n\nrequir"
},
{
"path": "librarian-puppet.gemspec",
"chars": 1365,
"preview": "$:.push File.expand_path(\"../lib\", __FILE__)\n\nrequire 'librarian/puppet/version'\n\nGem::Specification.new do |s|\n s.name"
},
{
"path": "spec/action/resolve_spec.rb",
"chars": 1017,
"preview": "require 'spec_helper'\nrequire_relative '../../lib/librarian/puppet/action/resolve'\nrequire 'librarian/ui'\nrequire 'thor'"
},
{
"path": "spec/librarian_puppet_spec.rb",
"chars": 479,
"preview": "describe Librarian::Puppet do\n it 'exits with error if puppet is not installed' do\n error = Librarian::Posix::Comman"
},
{
"path": "spec/receiver_spec.rb",
"chars": 1167,
"preview": "require 'spec_helper'\n\ndescribe 'Librarian::Puppet::Dsl::Receiver' do\n\n let(:dsl) { Librarian::Puppet::Dsl.new({}) }\n "
},
{
"path": "spec/source/forge_repo_spec.rb",
"chars": 489,
"preview": "require \"librarian/puppet/source/forge\"\nrequire \"librarian/puppet/environment\"\n\ndescribe Librarian::Puppet::Source::Forg"
},
{
"path": "spec/source/forge_spec.rb",
"chars": 1738,
"preview": "require \"librarian/puppet/source/forge\"\nrequire \"librarian/puppet/environment\"\nrequire 'librarian/puppet/extension'\n\ninc"
},
{
"path": "spec/spec_helper.rb",
"chars": 100,
"preview": "require 'simplecov'\nSimpleCov.start\n\nrequire 'rubygems'\nrequire 'rspec'\n\nrequire 'librarian/puppet'\n"
},
{
"path": "spec/util_spec.rb",
"chars": 368,
"preview": "require 'spec_helper'\n\ndescribe Librarian::Puppet::Util do\n\n subject { Class.new { include Librarian::Puppet::Util }.ne"
},
{
"path": "test/librarian/puppet/source/githubtarball_test.rb",
"chars": 2929,
"preview": "require File.expand_path(\"../../../../test_helper\", __FILE__)\nrequire \"librarian/puppet/source/githubtarball\"\n\ndescribe "
},
{
"path": "test/test_helper.rb",
"chars": 162,
"preview": "require 'bundler/setup'\nrequire 'minitest/autorun'\nrequire 'minitest/spec'\nrequire 'mocha/setup'\n\n$LOAD_PATH << \"vendor/"
}
]
About this extraction
This page contains the full source code of the rodjek/librarian-puppet GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 84 files (133.6 KB), approximately 38.4k tokens, and a symbol index with 237 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.