Showing preview only (466K chars total). Download the full file or copy to clipboard to get everything.
Repository: plunder-app/plunder
Branch: master
Commit: 58306fce81c9
Files: 85
Total size: 441.4 KB
Directory structure:
gitextract_qm74k6_1/
├── .gitignore
├── Dockerfile
├── LICENSE
├── Makefile
├── Readme.md
├── cmd/
│ ├── automate.go
│ ├── configGenerator.go
│ ├── plunder.go
│ └── server.go
├── docs/
│ ├── actions.md
│ ├── application_architecture.md
│ ├── deployment.md
│ ├── example_architecture.md
│ ├── example_deployment.md
│ ├── provisioning.md
│ ├── readme.md
│ └── service.md
├── go.mod
├── go.sum
├── hack/
│ └── comboot_ipxe/
│ ├── Dockerfile
│ └── gen_comboot.ipxe
├── main.go
├── pkg/
│ ├── apiserver/
│ │ ├── README.md
│ │ ├── client.go
│ │ ├── config.go
│ │ ├── endpoints.go
│ │ ├── go.mod
│ │ ├── handlerApiserver.go
│ │ ├── logging.go
│ │ ├── loggingHandlers.go
│ │ ├── server.go
│ │ └── types.go
│ ├── certs/
│ │ ├── certs.go
│ │ └── go.mod
│ ├── go.mod
│ ├── parlay/
│ │ ├── go.mod
│ │ ├── handler.go
│ │ ├── parlay.go
│ │ ├── parlay_ui.go
│ │ ├── parlaytypes/
│ │ │ ├── finder.go
│ │ │ ├── go.mod
│ │ │ └── parlaytypes.go
│ │ ├── parser.go
│ │ ├── parser_builder.go
│ │ ├── plugin/
│ │ │ └── plugin.go
│ │ ├── restore.go
│ │ └── validate.go
│ ├── plunderlogging/
│ │ ├── consolelogger.go
│ │ ├── filelogger.go
│ │ ├── go.mod
│ │ ├── jsonlogger.go
│ │ └── logger.go
│ ├── services/
│ │ ├── deployments.go
│ │ ├── go.mod
│ │ ├── handler.go
│ │ ├── server.go
│ │ ├── serverDHCP.go
│ │ ├── serverHTTP.go
│ │ ├── serverHTTPISO.go
│ │ ├── serverImageHTTP.go
│ │ ├── serverTFTP.go
│ │ ├── services.go
│ │ ├── static_pxe.go
│ │ ├── templateBOOTy.go
│ │ ├── templateESXi.go
│ │ ├── templateKickstart.go
│ │ ├── templatePreseed.go
│ │ ├── templateUtils.go
│ │ └── types.go
│ ├── ssh/
│ │ ├── go.mod
│ │ ├── sshClient.go
│ │ ├── sshCommand.go
│ │ ├── sshConfig.go
│ │ ├── sshImport.go
│ │ └── sshTransfer.go
│ └── utils/
│ ├── go.mod
│ ├── ipxe.go
│ ├── nic.go
│ └── utils.go
├── plugin/
│ ├── docker/
│ │ ├── docker.go
│ │ └── docker_actions.go
│ ├── example.go
│ └── kubeadm/
│ ├── kubeadm.go
│ └── kubeadm_actions.go
└── testing.sh
================================================
FILE CONTENTS
================================================
================================================
FILE: .gitignore
================================================
.plunderserver.yaml
plunder
plunderclient.yaml
================================================
FILE: Dockerfile
================================================
# syntax=docker/dockerfile:experimental
# Build BOOTy as an init
FROM golang:1.14-alpine as dev
RUN apk add --no-cache git ca-certificates make
COPY . /go/src/github.com/plunder-app/plunder
WORKDIR /go/src/github.com/plunder-app/plunder
ENV GO111MODULE=on
RUN --mount=type=cache,sharing=locked,id=gomod,target=/go/pkg/mod/cache \
--mount=type=cache,sharing=locked,id=goroot,target=/root/.cache/go-build \
CGO_ENABLED=0 GOOS=linux make build
FROM scratch
COPY --from=dev /go/src/github.com/plunder-app/plunder/plunder /
ENTRYPOINT ["/plunder"]
================================================
FILE: LICENSE
================================================
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
================================================
FILE: Makefile
================================================
SHELL := /bin/sh
# The name of the executable (default is current directory name)
TARGET := plunder
.DEFAULT_GOAL: $(TARGET)
# These will be provided to the target
VERSION := 0.5.0
BUILD := `git rev-parse HEAD`
# Required for the move to go modules for >v0.5.0
export GO111MODULE=on
# Operating System Default (LINUX)
TARGETOS=linux
# Use linker flags to provide version/build settings to the target
LDFLAGS=-ldflags "-X=main.Version=$(VERSION) -X=main.Build=$(BUILD) -s"
REPOSITORY = plndr
DOCKERREPO ?= $(TARGET)
DOCKERTAG ?= latest
.PHONY: all build clean install uninstall fmt simplify check run lint vet
all: check install
$(TARGET): $(SRC)
@go build $(LDFLAGS) -o $(TARGET)
build: $(TARGET)
@true
clean:
@rm -f $(TARGET)
install:
@echo Building and Installing project
@go install $(LDFLAGS)
install_plugin:
@make plugins
@echo Installing plugins
-mkdir ~/plugin
-cp -pr ./plugin/*.plugin ~/plugin/
uninstall: clean
@rm -f $$(which ${TARGET})
fmt:
@gofmt -l -w $(SRC)
vet:
@go vet $(SRC)
lint:
@golint $(SRC)
# This is typically only for quick testing
dockerx86:
@docker buildx build --platform linux/amd64 --load -t $(REPOSITORY)/$(TARGET):$(DOCKERTAG) -f Dockerfile .
@echo New Multi Architecture Docker image created
docker:
@docker buildx build --platform linux/amd64,linux/arm64,linux/arm/v7 --push -t $(REPOSITORY)/$(TARGET):$(DOCKERTAG) -f Dockerfile .
@echo New Multi Architecture Docker image created
plugins:
@echo Building plugins
@GO111Module=off go build -buildmode=plugin -o ./plugin/example.plugin ./plugin/example.go
@GO111Module=off go build -buildmode=plugin -o ./plugin/kubeadm.plugin ./plugin/kubeadm/*
@GO111Module=off go build -buildmode=plugin -o ./plugin/docker.plugin ./plugin/docker/*
release_darwin:
@echo Creating Darwin Build
@GOOS=darwin make build
@GOOS=darwin make plugins
@zip -9 -r plunder-darwin-$(VERSION).zip ./plunder ./plugin/*.plugin
@rm plunder
@rm ./plugin/*.plugin
release_linux:
@echo Creating Linux Build
@GOOS=linux make build
@GOOS=linux make plugins
@zip -9 -r plunder-linux-$(VERSION).zip ./plunder ./plugin/*.plugin
@rm plunder
@rm ./plugin/*.plugin
simplify:
@gofmt -s -l -w $(SRC)
check:
@test -z $(shell gofmt -l main.go | tee /dev/stderr) || echo "[WARN] Fix formatting issues with 'make fmt'"
make lint
make vet
run: install
@$(TARGET)
================================================
FILE: Readme.md
================================================
# Plunder
The complete tool for finding **Infrastructure** gold amongst bits of bare-metal!
## Overview
Plunder is a single-binary server that is all designed in order to make the provisioning of servers, platforms and applications easier. It is deployed as a server that an end user can interact with through it's **Api-server** in order to control and automate the usage. At this time interacting with the api-server is detailed in the source [https://github.com/plunder-app/plunder/blob/master/pkg/apiserver/endpoints.go](https://github.com/plunder-app/plunder/blob/master/pkg/apiserver/endpoints.go), however documentation will be added soon.
From an end-user interaction a plunder control utility has been created:
[https://github.com/plunder-app/pldrctl](https://github.com/plunder-app/pldrctl) - provides the capability to query and create deployments and configurations within a plunder instance.
### Services
- `DHCP` - Allocating an IP addressing and pointing to a TFTP server
- `TFTP` - Bootstrapping an Operating system install (uses iPXE)
- `HTTP` - Provides a services where the bootstrap can pull the components needed for the OS install.
An operating system can be easily performed using either **preseed** or **kickstart**, alternatively custom kernels and init ramdisks can be specified to be used based upon Mac address.
### Automation
Further more once the operating system has been provisioned there are usually post-deployment tasks in order to complete an installation. Plunder has the capability to do the following:
- `Remote command execution` - Over SSH (key configured above)
- `Scripting engine` - A JSON/YAML language that also supports plugins to extend the capablities of the automation engine.
A small repository of existing deployment maps has been created [https://github.com/plunder-app/maps](https://github.com/plunder-app/maps)
### Additional features
- `iso support` - Plunder no longer requires a user with elevated privileges to mount an OS ISO in order to read the contents. Plunder can read files directly from the iso file and expose them to an installer through `http`.
- `online updates` - As all configuration to plunder is exposed and managed through an API, it provides the capability of performing most configuration changes with no down time or restarts.
- `in-memory configurations` - Plunder will create all deployment configurations and hold them in memory, meaning that it is stateless and it doesn't leave configuration all over a filesystem
- `VMware deployment support` - Plunder can deploy preseed/kickstart and now vSphere installations.
- `Management of unclaimed devices` - Plunder will watch and keep a pool of devices that aren't being deployed and can force them to reboot/restart until they're needed for deployment.
- `Logging of remote execution` - Plunder can now store all execution logs in-memory until told to clear them.
## Getting Plunder
Prebuilt binaries for Darwin(MacOS)/Linux and Windows can be found on the [releases](https://github.com/plunder-app/plunder/releases) page.
### Building
If you wish to build the code yourself then this can be done simply by running:
```
go get -u github.com/plunder-app/plunder
```
Alternatively clone the repository and either `go build` or `make build`, note that using the makefile will ensure that the current git commit and version number are returned by `plunder version`.
## Usage!
One of the key design concepts was to try to simplify the amount of moving parts required to bootstrap a server, therefore `plunder` aims to be a single tool that you can use. It also aims to simplify the amount of configuration files and configuration work required, it does this by auto-detecting most configuration and producing mainly completed configuration as needed.
One thing to be aware of is that `plunder` doesn't require replacing anything that already exists in the infrastructure.
The documentation is available [here](./docs/)
### Warning
*NOTE 1* As this provides low-level networking services, only run on a network that is safe to do so. Providing DHCP on a network that already provides DHCP services can lead to un-expected behaviour (and angry network administrators)
*NOTE 2* As DHCP/TFTP and HTTP all bind to low ports < 1024, root access (or sudo) is required to start the plunder services.
# Troubleshooting
PXE booting provides very little feedback when things aren't working, but usually the hand-off is why things wont work i.e. `DHCP` -> `TFTP` boot. Logs from `plunder` should show the hand-off from the CLI.
# Roadmap
- Ability to automate deployments over VMware VMTools
- Windows deployments
- Tidier logging
- Stability enhancements
- Additional plugins
================================================
FILE: cmd/automate.go
================================================
package cmd
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"net/url"
"os"
"github.com/ghodss/yaml"
"github.com/plunder-app/plunder/pkg/parlay"
"github.com/plunder-app/plunder/pkg/parlay/parlaytypes"
parlayplugin "github.com/plunder-app/plunder/pkg/parlay/plugin"
"github.com/plunder-app/plunder/pkg/ssh"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
// These flags are used to determine a deployment
var deploymentSSH, mapFile, logFile, deploymentEndpoint *string
// These flags are used to override SSH configuration
var usernameSSH, keypathSSH, addressSSH *string
// These flags are used to determine if a particular deployment, action and specific host need to be used.
var deploymentName, actionName, host *string
// These flags are used for management of plugins
var pluginPath, pluginAction, pluginActions *string
// This flag determines if a singular action should occur or whether to resume all actions from this point
var resume *bool
// UI Json output only, when this is try the UI selections will just create the associated JSON
var jsonOutput, yamlOutput *bool
func init() {
// Global flags for automation
logFile = plunderAutomate.PersistentFlags().String("logfile", "", "Path to where plunder will write automation logs")
mapFile = plunderAutomate.PersistentFlags().String("map", "", "Path to a plunder map")
// SSH Deployment flags
deploymentSSH = plunderAutomate.PersistentFlags().String("deployconfig", "", "Path to a plunder deployment configuration")
usernameSSH = plunderAutomate.PersistentFlags().String("overrideUsername", "", "(optional) Override Username")
keypathSSH = plunderAutomate.PersistentFlags().String("overrideKeypath", "", "(Optional) Override path to a key")
addressSSH = plunderAutomate.PersistentFlags().String("overrideAddress", "", "(Optional) Override address to automate against")
// Plunder endpoing Deployment flags
deploymentEndpoint = plunderAutomate.PersistentFlags().String("deployendpoint", "", "URL of plunder server to pull the deployment configuration")
// Deployment control flags
deploymentName = plunderAutomateSSH.Flags().String("deployment", "", "Automate a specific deployment")
actionName = plunderAutomateSSH.Flags().String("action", "", "Automate a specific action")
host = plunderAutomateSSH.Flags().String("host", "", "Automate the deployment for a specific host")
resume = plunderAutomateSSH.Flags().Bool("resume", false, "Resume all actions after the one specified by --action")
// Plugin Flags
pluginPath = plunderAutomatePluginUsage.Flags().String("plugin", "", "Path to a specific plugin typically ~./plugin/[X].plugin")
pluginAction = plunderAutomatePluginUsage.Flags().String("action", "", "Action to retrieve the usage of")
pluginActions = plunderAutomatePluginActions.Flags().String("plugin", "", "Path to a specific plugin typically ~./plugin/[X].plugin")
jsonOutput = plunderAutomateUI.Flags().Bool("json", false, "Print the JSON to stdout, no execution of commands")
yamlOutput = plunderAutomateUI.Flags().Bool("yaml", false, "Print the YAML to stdout, no execution of commands")
plunderAutomatePlugins.AddCommand(plunderAutomatePluginUsage)
plunderAutomatePlugins.AddCommand(plunderAutomatePluginActions)
plunderAutomatePlugins.AddCommand(plunderAutomatePluginTest)
// Automate Subcommands
plunderAutomate.AddCommand(plunderAutomateValidate)
plunderAutomate.AddCommand(plunderAutomateSSH)
plunderAutomate.AddCommand(plunderAutomateVMware)
plunderAutomate.AddCommand(plunderAutomatePlugins)
plunderAutomate.AddCommand(plunderAutomateUI)
plunderCmd.AddCommand(plunderAutomate)
}
// PlunderAutomate
var plunderAutomate = &cobra.Command{
Use: "automate",
Short: "Automate the deployment of a platform/application",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
cmd.Help()
return
},
}
// plunderAutomatePlugins
var plunderAutomatePlugins = &cobra.Command{
Use: "plugin",
Short: "Automate the deployment of a platform/application",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
parlayplugin.ListPlugins()
return
},
}
// plunderAutomatePlugins
var plunderAutomatePluginUsage = &cobra.Command{
Use: "usage",
Short: "Display the usage of a plugin action",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
parlayplugin.UsagePlugin(*pluginPath, *pluginAction)
return
},
}
// plunderAutomatePlugins
var plunderAutomatePluginActions = &cobra.Command{
Use: "actions",
Short: "Display the actions of a particular plugin",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
parlayplugin.ListPluginActions(*pluginActions)
return
},
}
// plunderAutomatePlugins
var plunderAutomatePluginTest = &cobra.Command{
Use: "test",
Short: "Test the actions of the example plugin",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
test := `{ "name": "Example of test action", "type": "exampleAction/test", "plugin": { "credentials": "AAABBBCCCCDDEEEE", "address": "172.0.0.1" } }`
var action parlaytypes.Action
_ = json.Unmarshal([]byte(test), &action)
_, err := parlayplugin.ExecuteActionInPlugin("./plugin/example.plugin", "127.0.0.1", "example/test", action.Plugin)
if err != nil {
log.Fatalf("%v", err)
}
return
},
}
// plunderAutomateSSH
var plunderAutomateSSH = &cobra.Command{
Use: "ssh",
Short: "Automate over ssh",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
// If deploymentPath is not blank then the flag has been used
if *deploymentSSH != "" {
log.Infof("Reading deployment configuration from [%s]", *deploymentSSH)
// Check the actual path from the string
if _, err := os.Stat(*deploymentSSH); !os.IsNotExist(err) {
config, err := ioutil.ReadFile(*deploymentSSH)
if err != nil {
log.Fatalf("%v", err)
}
err = ssh.ImportHostsFromRawDeployment(config)
if err != nil {
cmd.Help()
log.Fatalf("%v", err)
}
} else {
log.Fatalf("Unable to open [%s]", *deploymentSSH)
}
} else if *deploymentEndpoint != "" {
u, err := url.Parse(*deploymentEndpoint)
if err != nil {
log.Fatalf("%v", err)
}
// TODO - fix dynamic
u.Path = "/deployments"
resp, err := http.Get(u.String())
if err != nil {
log.Fatalf("%v", err)
}
//var config server.DeploymentConfigurationFile
defer resp.Body.Close()
config, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalf("%v", err)
}
err = ssh.ImportHostsFromRawDeployment(config)
if err != nil {
cmd.Help()
log.Fatalf("%v", err)
}
}
// Add a host from the override flags
if *addressSSH != "" {
err := ssh.AddHost(*addressSSH, *keypathSSH, *usernameSSH)
if err != nil {
log.Fatalf("%v", err)
}
}
// If there are zero hosts in the ssh Host array then we have no authentication information
if len(ssh.Hosts) == 0 {
cmd.Help()
log.Fatalf("No Deployment information imported")
}
log.Infof("Found [%d] ssh configurations", len(ssh.Hosts))
if *mapFile != "" {
log.Infof("Reading deployment configuration from [%s]", *mapFile)
var deployment parlaytypes.TreasureMap
// // Check the actual path from the string
if _, err := os.Stat(*mapFile); !os.IsNotExist(err) {
b, err := ioutil.ReadFile(*mapFile)
if err != nil {
log.Fatalf("%v", err)
}
deployment, err = parseMapFile(b)
if err != nil {
log.Fatalf("%v", err)
}
// If a specific deployment is being used then find it's details
if *deploymentName != "" {
log.Infof("Looking for deployment [%s]", *deploymentName)
foundDeployment, err := deployment.FindDeployment(*deploymentName, *actionName, *host, *logFile, *resume)
if err != nil {
log.Fatalf("%s", err)
}
err = parlay.DeploySSH(foundDeployment, *logFile, false, false)
} else {
// Parse the entire deployment
err = parlay.DeploySSH(&deployment, *logFile, false, false)
}
if err != nil {
log.Fatalf("%v", err)
}
} else {
log.Fatalf("%v", err)
}
}
return
},
}
// plunderAutomateValidate
var plunderAutomateValidate = &cobra.Command{
Use: "validate",
Short: "Validate a deployment map",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
if *mapFile != "" {
log.Infof("Reading deployment configuration from [%s]", *mapFile)
//var err error
var deployment parlaytypes.TreasureMap
// // Check the actual path from the string
if _, err := os.Stat(*mapFile); !os.IsNotExist(err) {
b, err := ioutil.ReadFile(*mapFile)
if err != nil {
log.Fatalf("%v", err)
}
deployment, err = parseMapFile(b)
if err != nil {
log.Fatalf("%v", err)
}
deploymentCount := len(deployment.Deployments)
if deploymentCount == 0 {
log.Fatalf("Zero deployments have been found")
}
log.Infof("Validating [%d] deployments", deploymentCount)
for x := range deployment.Deployments {
actionCount := len(deployment.Deployments[x].Actions)
if actionCount == 0 {
log.Fatalf("Zero deployments have been found")
}
log.Infof("Validating [%d] actions", actionCount)
for y := range deployment.Deployments[x].Actions {
err := parlay.ValidateAction(&deployment.Deployments[x].Actions[y])
if err != nil {
log.Warnf("Action [%s] Error [%v]", deployment.Deployments[x].Actions[y].Name, err)
}
}
}
} else {
log.Fatalf("Unable to open [%s]", *mapFile)
}
} else {
cmd.Help()
log.Fatalln("No Deployment map specified")
}
},
}
// plunderAutomateVMware
var plunderAutomateVMware = &cobra.Command{
Use: "vmw",
Short: "Automate over VMware tools protocol",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
//var newMap *parlay.TreasureMap
if *mapFile != "" {
log.Infof("Reading deployment configuration from [%s]", *mapFile)
//var err error
var deployment parlaytypes.TreasureMap
// // Check the actual path from the string
if _, err := os.Stat(*mapFile); !os.IsNotExist(err) {
b, err := ioutil.ReadFile(*mapFile)
if err != nil {
log.Fatalf("%v", err)
}
deployment, err = parseMapFile(b)
if err != nil {
log.Fatalf("%v", err)
}
// If a specific deployment is being used then find it's details
if *deploymentName != "" {
log.Infof("Looking for deployment [%s]", *deploymentName)
foundDeployment, err := deployment.FindDeployment(*deploymentName, *actionName, *host, *logFile, *resume)
if err != nil {
log.Fatalf("%s", err)
}
err = parlay.DeploySSH(foundDeployment, *logFile, false, false)
} else {
// Parse the entire deployment
err = parlay.DeploySSH(&deployment, *logFile, false, false)
}
if err != nil {
log.Fatalf("%v", err)
}
} else {
log.Fatalf("%v", err)
}
}
},
}
// plunderAutomateUI
var plunderAutomateUI = &cobra.Command{
Use: "ui",
Short: "Enable the user interface to manage a deployment",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
var newMap *parlaytypes.TreasureMap
if *mapFile != "" {
log.Infof("Reading deployment configuration from [%s]", *mapFile)
//var err error
var deployment parlaytypes.TreasureMap
// // Check the actual path from the string
if _, err := os.Stat(*mapFile); !os.IsNotExist(err) {
b, err := ioutil.ReadFile(*mapFile)
if err != nil {
log.Fatalf("%v", err)
}
deployment, err = parseMapFile(b)
if err != nil {
log.Fatalf("%v", err)
}
newMap, err = parlay.StartUI(&deployment)
if err != nil {
log.Fatalf("%v", err)
}
}
}
// If we're using the UI to build a new map then print to stdout(in either format)
if *jsonOutput == true {
b, _ := json.MarshalIndent(newMap, "", "\t")
fmt.Printf("%s\n", b)
return
}
if *yamlOutput == true {
b, _ := yaml.Marshal(newMap)
fmt.Printf("%s\n", b)
return
}
if *deploymentSSH != "" {
log.Infof("Reading deployment configuration from [%s]", *deploymentSSH)
// Check the actual path from the string
if _, err := os.Stat(*deploymentSSH); !os.IsNotExist(err) {
config, err := ioutil.ReadFile(*deploymentSSH)
if err != nil {
log.Fatalf("%v", err)
}
// Parse all of the hosts in the deployment configuration and update the ssh package with their details
err = ssh.ImportHostsFromRawDeployment(config)
if err != nil {
cmd.Help()
log.Fatalf("%v", err)
}
} else {
log.Fatalf("Unable to open [%s]", *deploymentSSH)
}
} else if *deploymentEndpoint != "" {
// Parse the endpoint, this will attempt to pull all of the configuration information and pass it to the SSH package
u, err := url.Parse(*deploymentEndpoint)
if err != nil {
log.Fatalf("%v", err)
}
u.Path = "/deployment"
resp, err := http.Get(u.String())
if err != nil {
log.Fatalf("%v", err)
}
defer resp.Body.Close()
config, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalf("%v", err)
}
err = ssh.ImportHostsFromRawDeployment(config)
if err != nil {
cmd.Help()
log.Fatalf("%v", err)
}
}
// Add a host from the override flags
if *addressSSH != "" {
err := ssh.AddHost(*addressSSH, *keypathSSH, *usernameSSH)
if err != nil {
log.Fatalf("%v", err)
}
}
// If there are zero hosts in the ssh Host array then we have no authentication information
if len(ssh.Hosts) == 0 {
cmd.Help()
log.Fatalf("No Deployment information imported")
}
err := parlay.DeploySSH(newMap, *logFile, false, false)
if err != nil {
log.Fatalf("%v", err)
}
},
}
func parseMapFile(b []byte) (deployment parlaytypes.TreasureMap, err error) {
jsonBytes, err := yaml.YAMLToJSON(b)
if err == nil {
// If there were no errors then the YAML => JSON was succesful, no attempt to unmarshall
err = json.Unmarshal(jsonBytes, &deployment)
if err != nil {
return deployment, fmt.Errorf("Unable to parse [%s] as either yaml or json", *mapFile)
}
} else {
// Couldn't parse the yaml to JSON
// Attempt to parse it as JSON
err = json.Unmarshal(b, &deployment)
if err != nil {
return deployment, fmt.Errorf("Unable to parse [%s] as either yaml or json", *mapFile)
}
}
return deployment, nil
}
================================================
FILE: cmd/configGenerator.go
================================================
package cmd
import (
"encoding/json"
"fmt"
"net"
"os"
"strings"
"time"
"github.com/ghodss/yaml"
booty "github.com/plunder-app/BOOTy/pkg/plunderclient/types"
"github.com/plunder-app/plunder/pkg/apiserver"
"github.com/plunder-app/plunder/pkg/certs"
"github.com/plunder-app/plunder/pkg/parlay/parlaytypes"
"github.com/plunder-app/plunder/pkg/services"
"github.com/plunder-app/plunder/pkg/utils"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
// These variables are used to capture input from the CLI
var output, detectNic, serverPath, clientPath string
var configAPIServerPort int
var pretty bool
func init() {
plunderCmd.AddCommand(plunderConfig)
plunderConfig.PersistentFlags().StringVarP(&output, "output", "o", "json", "Ouput type, should be either JSON or YAML")
plunderConfig.PersistentFlags().BoolVarP(&pretty, "pretty", "p", false, "Ouput JSON in a pretty/Human readable format")
plunderServerConfig.PersistentFlags().StringVarP(&detectNic, "nic", "n", "", "Build configuration for a particular network interface")
// Persistent above both client functions
plunderAPIConfig.PersistentFlags().IntVar(&configAPIServerPort, "port", 60443, "Port that the plunder API server should use")
// Path for Server
plunderAPIConfigServer.Flags().StringVar(&serverPath, "path", ".plunderserver.yaml", "Path that the plunder API server config should be written to")
// Path for Client
plunderAPIConfigClient.Flags().StringVar(&clientPath, "path", "plunderclient.yaml", "Path that the plunder API client config should be written to")
// Add sub commands to APIServer
plunderAPIConfig.AddCommand(plunderAPIConfigClient)
plunderAPIConfig.AddCommand(plunderAPIConfigServer)
// Add all sub commands to the config sub command
plunderConfig.AddCommand(plunderAPIConfig)
plunderConfig.AddCommand(plunderServerConfig)
plunderConfig.AddCommand(plunderDeploymentConfig)
plunderConfig.AddCommand(PlunderParlayConfig)
plunderCmd.AddCommand(plunderGet)
}
// PlunderConfig - This is for intialising a blank or partial configuration
var plunderConfig = &cobra.Command{
Use: "config",
Short: "Initialise a plunder configuration",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
cmd.Help()
return
},
}
// PlunderServerConfig - This is for intialising a blank or partial configuration
var plunderServerConfig = &cobra.Command{
Use: "server",
Short: "Initialise a plunder configuration",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
// Indent (or pretty-print) the configuration output
bc := &services.BootConfig{
Kernel: "/kernelPath",
Initrd: "/initPath",
Cmdline: "cmd=options",
ConfigName: "demo config",
ConfigType: "default",
}
detectServerConfig()
services.Controller.BootConfigs = append(services.Controller.BootConfigs, *bc)
err := renderOutput(services.Controller, pretty)
if err != nil {
log.Fatalf("%v", err)
}
return
},
}
// PlunderDeploymentConfig - This is for intialising a blank or partial configuration
var plunderDeploymentConfig = &cobra.Command{
Use: "deployment",
Short: "Initialise a server configuration",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
// Create an example Global configuration
globalConfig := services.HostConfig{
Gateway: "192.168.0.1",
NTPServer: "192.168.0.1",
NameServer: "192.168.0.1",
Adapter: "ens192",
Subnet: "255.255.255.0",
// OS Provision
Username: "user",
Password: "pass",
Packages: "openssh-server cloud-guest-utils",
RepositoryAddress: "192.168.0.1",
MirrorDirectory: "/ubuntu",
SSHKeyPath: "/home/deploy/.ssh/id_pub.rsa",
SSHKey: "ssh-rsa AABBCCDDEE1122334455",
// BOOTy
BOOTYAction: booty.ReadImage,
LVMRootName: "/dev/ubuntu-vg/root",
DestinationDevice: "/dev/sda",
DestinationAddress: "http://192.168.0.1/image",
SourceImage: "http://192.168.0.1/images/ubuntu.img",
SourceDevice: "/dev/sda",
}
// Set compressed pointer
compressed := false
globalConfig.Compressed = &compressed
// Addtional step to create the partition information
defaultPartition := 1
globalConfig.GrowPartition = &defaultPartition
// Create an example Host configuration
hostConfig := services.HostConfig{
IPAddress: "192.168.0.2",
ServerName: "Server01",
}
hostDeployConfig := services.DeploymentConfig{
MAC: "00:11:22:33:44:55",
ConfigHost: hostConfig,
//ConfigName: "default",
}
configuration := &services.DeploymentConfigurationFile{
GlobalServerConfig: globalConfig,
}
configuration.Configs = append(configuration.Configs, hostDeployConfig)
// Indent (or pretty-print) the configuration output
err := renderOutput(configuration, pretty)
if err != nil {
log.Fatalf("%v", err)
}
return
},
}
// PlunderParlayConfig - This is for intialising a parlay deployment
var PlunderParlayConfig = &cobra.Command{
Use: "parlay",
Short: "Initialise a parlay configuration",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
parlayActionPackage := parlaytypes.Action{
Name: "Add package",
ActionType: "pkg",
PkgManager: "apt",
PkgOperation: "install",
Packages: "mysql",
}
parlayActionCommand := parlaytypes.Action{
Name: "Run Command",
ActionType: "command",
Command: "which uptime",
CommandSudo: "root",
CommandSaveAsKey: "cmdKey",
}
parlayActionUpload := parlaytypes.Action{
Name: "Upload File",
ActionType: "upload",
Source: "./my_file",
Destination: "/tmp/file",
}
parlayActionDownload := parlaytypes.Action{
Name: "Download File",
ActionType: "download",
Destination: "./my_file",
Source: "/tmp/file",
}
parlayActionKey := parlaytypes.Action{
Name: "Execute key",
ActionType: "command",
KeyName: "cmdKey",
}
parlayDeployment := parlaytypes.Deployment{
Name: "Install MySQL",
Hosts: []string{"192.168.0.1", "192.168.0.2"},
}
parlayDeployment.Actions = append(parlayDeployment.Actions, parlayActionPackage)
parlayDeployment.Actions = append(parlayDeployment.Actions, parlayActionCommand)
parlayDeployment.Actions = append(parlayDeployment.Actions, parlayActionUpload)
parlayDeployment.Actions = append(parlayDeployment.Actions, parlayActionDownload)
parlayDeployment.Actions = append(parlayDeployment.Actions, parlayActionKey)
parlayConfig := &parlaytypes.TreasureMap{}
parlayConfig.Deployments = []parlaytypes.Deployment{}
parlayConfig.Deployments = append(parlayConfig.Deployments, parlayDeployment)
// Render the output to screen
err := renderOutput(parlayConfig, pretty)
if err != nil {
log.Fatalf("%v", err)
}
return
},
}
// plunderGet - The Get command will pull any required components (iPXE boot files)
var plunderGet = &cobra.Command{
Use: "get",
Short: "Get any components needed for bootstrapping (internet access required)",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
err := utils.PullPXEBooter()
if err != nil {
log.Fatalf("%v", err)
}
return
},
}
// plunderAPIConfig - The Get command will pull any required components (iPXE boot files)
var plunderAPIConfig = &cobra.Command{
Use: "apiserver",
Short: "Generate the configuration for the api server",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
cmd.Help()
return
},
}
// plunderAPIConfigServer - The Get command will pull any required components (iPXE boot files)
var plunderAPIConfigServer = &cobra.Command{
Use: "server",
Short: "Generate the configuration for the api server",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
err := certs.GenerateKeyPair(nil, time.Now(), (24*time.Hour)*365)
if err != nil {
log.Fatalln(err)
}
err = apiserver.WriteServerConfig(serverPath, "", "", configAPIServerPort, certs.GetPem(), certs.GetKey())
if err != nil {
log.Fatalln(err)
}
return
},
}
// plunderAPIConfigServer - The Get command will pull any required components (iPXE boot files)
var plunderAPIConfigClient = &cobra.Command{
Use: "client",
Short: "Generate the configuration for a client for the API server",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
s, err := apiserver.OpenServerConfig(serverPath)
if err != nil {
log.Fatalln(err)
}
hostname, err := os.Hostname()
if err != nil {
log.Fatalln(err)
}
apiserver.WriteClientConfig(clientPath, hostname, s)
return
},
}
func renderOutput(data interface{}, pretty bool) error {
var d []byte
var err error
switch strings.ToLower(output) {
case "yaml":
d, err = yaml.Marshal(data)
case "json":
if pretty {
d, err = json.MarshalIndent(data, "", "\t")
} else {
d, err = json.Marshal(data)
}
default:
return fmt.Errorf("Unknown output type [%s]", output)
}
if err != nil {
return err
}
// Print out the output to STDOUT
fmt.Printf("%s\n", d)
return nil
}
func detectServerConfig() error {
// Find an example nic to use, that isn't the loopback address
nicName, nicAddr, err := utils.FindIPAddress(detectNic)
if err != nil {
return err
}
// Attempt to parse th returned IP address and apply simple incrementation to determin DHCP start range
ip := net.ParseIP(nicAddr)
ip = ip.To4()
if ip == nil {
return fmt.Errorf("error parsing IP address of adapter [%s]", detectNic)
}
ip[3]++
// Prepopulate the flags with the found nic information
services.Controller.AdapterName = &nicName
services.Controller.HTTPAddress = &nicAddr
services.Controller.TFTPAddress = &nicAddr
*services.Controller.PXEFileName = "undionly.kpxe"
// DHCP Settings
services.Controller.DHCPConfig.DHCPAddress = nicAddr
services.Controller.DHCPConfig.DHCPSubnet = "255.255.255.0"
services.Controller.DHCPConfig.DHCPGateway = nicAddr
services.Controller.DHCPConfig.DHCPDNS = nicAddr
services.Controller.DHCPConfig.DHCPLeasePool = 20
services.Controller.DHCPConfig.DHCPStartAddress = ip.String()
return nil
}
================================================
FILE: cmd/plunder.go
================================================
package cmd
import (
"fmt"
"os"
"strconv"
"github.com/plunder-app/plunder/pkg/utils"
log "github.com/sirupsen/logrus"
"github.com/spf13/cobra"
)
// Release - this struct contains the release information populated when building plunder
var Release struct {
Version string
Build string
}
var plunderCmd = &cobra.Command{
Use: "plunder",
Short: "This is a tool for finding gold amongst bare-metal (and provisioning kubernetes)",
}
var logLevel int
var filePath string
func init() {
plunderUtilsEncode.Flags().StringVar(&filePath, "path", "", "Path to a file to encode")
// Global flag across all subcommands
plunderCmd.PersistentFlags().IntVar(&logLevel, "logLevel", 4, "Set the logging level [0=panic, 3=warning, 5=debug]")
plunderCmd.AddCommand(plunderVersion)
plunderCmd.AddCommand(plunderUtils)
plunderUtils.AddCommand(plunderUtilsEncode)
}
// Execute - starts the command parsing process
func Execute() {
if os.Getenv("PLUNDER_LOGLEVEL") != "" {
i, err := strconv.ParseInt(os.Getenv("PLUNDER_LOGLEVEL"), 10, 8)
if err != nil {
log.Fatalf("Error parsing environment variable [PLUNDER_LOGLEVEL")
}
// We've only parsed to an 8bit integer, however i is still a int64 so needs casting
logLevel = int(i)
} else {
// Default to logging anything Info and below
logLevel = int(log.InfoLevel)
}
log.SetLevel(log.Level(logLevel))
if err := plunderCmd.Execute(); err != nil {
fmt.Println(err)
os.Exit(1)
}
}
var plunderVersion = &cobra.Command{
Use: "version",
Short: "Version and Release information about the plunder tool",
Run: func(cmd *cobra.Command, args []string) {
fmt.Printf("Plunder Release Information\n")
fmt.Printf("Version: %s\n", Release.Version)
fmt.Printf("Build: %s\n", Release.Build)
},
}
var plunderUtils = &cobra.Command{
Use: "utils",
Short: "Additional utilities for Plunder",
Run: func(cmd *cobra.Command, args []string) {
cmd.Help()
},
}
var plunderUtilsEncode = &cobra.Command{
Use: "encode",
Short: "This will encode a file into Hex",
Run: func(cmd *cobra.Command, args []string) {
hex, err := utils.FileToHex(filePath)
if err != nil {
log.Fatalf("%v", err)
}
fmt.Printf("%s", hex)
},
}
================================================
FILE: cmd/server.go
================================================
package cmd
import (
"io/ioutil"
"os"
"github.com/plunder-app/plunder/pkg/apiserver"
"github.com/plunder-app/plunder/pkg/parlay"
"github.com/plunder-app/plunder/pkg/services"
"github.com/plunder-app/plunder/pkg/utils"
"github.com/spf13/cobra"
log "github.com/sirupsen/logrus"
)
//var controller server.BootController
var dhcpSettings services.DHCPSettings
var apiServerPath, gateway, dns, startAddress, configPath, deploymentPath, defaultKernel, defaultInitrd, defaultCmdLine *string
var leasecount, port *int
var anyboot, insecure *bool
func init() {
// Prepopulate the flags with the found nic information
services.Controller.AdapterName = PlunderServer.Flags().String("adapter", "", "Name of adapter to use e.g eth0, en0")
services.Controller.HTTPAddress = PlunderServer.Flags().String("addressHTTP", "", "Address of HTTP to use, if blank will default to [addressDHCP]")
services.Controller.TFTPAddress = PlunderServer.Flags().String("addressTFTP", "", "Address of TFTP to use, if blank will default to [addressDHCP]")
services.Controller.EnableDHCP = PlunderServer.Flags().Bool("enableDHCP", false, "Enable the DCHP Server")
services.Controller.EnableTFTP = PlunderServer.Flags().Bool("enableTFTP", false, "Enable the TFTP Server")
services.Controller.EnableHTTP = PlunderServer.Flags().Bool("enableHTTP", false, "Enable the HTTP Server")
services.Controller.PXEFileName = PlunderServer.Flags().String("iPXEPath", "undionly.kpxe", "Path to an iPXE bootloader")
// DHCP Settings
PlunderServer.Flags().StringVar(&services.Controller.DHCPConfig.DHCPAddress, "addressDHCP", "", "Address to advertise leases from, ideally will be the IP address of --adapter")
PlunderServer.Flags().StringVar(&services.Controller.DHCPConfig.DHCPGateway, "gateway", "", "Address of Gateway to use, if blank will default to [addressDHCP]")
PlunderServer.Flags().StringVar(&services.Controller.DHCPConfig.DHCPDNS, "dns", "", "Address of DNS to use, if blank will default to [addressDHCP]")
PlunderServer.Flags().IntVar(&services.Controller.DHCPConfig.DHCPLeasePool, "leasecount", 20, "Amount of leases to advertise")
PlunderServer.Flags().StringVar(&services.Controller.DHCPConfig.DHCPStartAddress, "startAddress", "", "Start advertised address [REQUIRED]")
//HTTP Settings
defaultKernel = PlunderServer.Flags().String("kernel", "", "Path to a kernel to set as the *default* kernel")
defaultInitrd = PlunderServer.Flags().String("initrd", "", "Path to a ramdisk to set as the *default* ramdisk")
defaultKernel = PlunderServer.Flags().String("cmdline", "", "Additional command line to pass to the *default* kernel")
// Config File
configPath = PlunderServer.Flags().String("config", "", "Path to a plunder server configuration")
deploymentPath = PlunderServer.Flags().String("deployment", "", "Path to a plunder deployment configuration")
PlunderServer.Flags().StringVar(&services.DefaultBootType, "defaultBoot", "", "In the event a boot type can't be found default to this")
// API Server configuration
port = PlunderServer.Flags().IntP("port", "p", 60443, "Port that the Plunder API server will listen on")
insecure = PlunderServer.Flags().BoolP("insecure", "i", false, "Start the Plunder API server without encryption")
apiServerPath = PlunderServer.Flags().String("path", ".plunderserver.yaml", "Path to configuration for the API Server")
plunderCmd.AddCommand(PlunderServer)
}
// PlunderServer - This is for intialising a blank or partial configuration
var PlunderServer = &cobra.Command{
Use: "server",
Short: "Start Plunder Services",
Run: func(cmd *cobra.Command, args []string) {
log.SetLevel(log.Level(logLevel))
var deployment []byte
// If deploymentPath is not blank then the flag has been used
if *deploymentPath != "" {
// if *anyboot == true {
// log.Errorf("AnyBoot has been enabled, all configuration will be ignored")
// }
log.Infof("Reading deployment configuration from [%s]", *deploymentPath)
if _, err := os.Stat(*deploymentPath); !os.IsNotExist(err) {
deployment, err = ioutil.ReadFile(*deploymentPath)
if err != nil {
log.Fatalf("%v", err)
}
}
}
// if *anyboot == true {
// services.AnyBoot = true
// }
// If configPath is not blank then the flag has been used
if *configPath != "" {
log.Infof("Reading configuration from [%s]", *configPath)
// Check the actual path from the string
if _, err := os.Stat(*configPath); !os.IsNotExist(err) {
configFile, err := ioutil.ReadFile(*configPath)
if err != nil {
log.Fatalf("%v", err)
}
// Read the controller from either a yaml or json format
err = services.ParseControllerData(configFile)
if err != nil {
log.Fatalf("%v", err)
}
} else {
log.Fatalf("Unable to open [%s]", *configPath)
}
}
if *services.Controller.EnableDHCP == false && *services.Controller.EnableHTTP == false && *services.Controller.EnableTFTP == false {
log.Warnln("All services are currently disabled")
}
// If we've enabled DHCP, then we need to ensure a start address for the range is populated
if *services.Controller.EnableDHCP && services.Controller.DHCPConfig.DHCPStartAddress == "" {
log.Fatalln("A DHCP Start address is required")
}
if services.Controller.DHCPConfig.DHCPLeasePool == 0 {
log.Fatalln("At least one available lease is required")
}
services.Controller.StartServices(deployment)
// Run the API server in a seperate go routine
go func() {
err := apiserver.StartAPIServer(*apiServerPath, *port, *insecure)
if err != nil {
log.Fatalf("%v", err)
}
}()
// Register the packages to the apiserver
services.RegisterToAPIServer()
parlay.RegisterToAPIServer()
// Sit and wait for a control-C
utils.WaitForCtrlC()
return
},
}
================================================
FILE: docs/actions.md
================================================
# Actions
When a deployment is executed against a host(s) typically one or more **actions** will be performed against that host in order to configure as expected. This document details the **built-in** actions, however to extend the functionality of [plunder](github.com/plunder-app/plunder) there is the capability to extend the available actions through the use of plugins.
## Built-in Actions
All actions are defined by a `type` which specifies what tasks the action will perform, also all actions should come with a `name` that identifies what the action will perform. The names should make it easy to identify relevant tasks as they're executed or when selecting individual tasks when using the user Interface.
Example in json and yaml below:
```json
{
"task" : "command",
"command" : "docker run image",
"name" : "Starts the docker image \"image\""
}
```
```yaml
- task: download
source: "/home/user/my_archive.tar.gz"
name: "Retrieve the home archive"
```
### Command
The **command** action type is used to execute a command either locally or remote, it will exit execution if the command fails (or it can be ignored) and the results can be stored to be executed at a later point.
Set the `ignoreFail` to `true` to allow execution of tasks to continue in the event that the command fails. If a long running task should be known to only execute for a specific amount of time, commands can be given a timeout which will end the command should it not complete in time. The `timeout` setting should be set in seconds which will specify how long the task is allowed to execute for.
```yaml
- task: command
command: "sleep 100"
timeout: 99
ignoreFail: true
```
*The above example will execute a sleep for a hundred seconds, however the command has a timeout set for only 99 seconds. Execution will be halted once the timeout is met, and if the task returns a fail code the execution will continue onto the next action*
If a command requires elevated privileges, the `commandSudo` option allows executing a command as different user, with it's entitled privileges.
**Note**: This requires `NOPASSWD` to be set for the current user.
```yaml
{
"task" : "command",
"command" : "cat /dev/null > /var/log/messages",
"name" : "Concatenate the messages file to clear space",
"commandSudo" : "root"
}
```
#### Using commands between actions deployments
There may be a requirment to save the output of a command to be used in a different action or a different deployment, some commands will generate tokens or output that can be used at a later point.
There are two options to save the output of a command:
- `commandSaveFile` - saves the command output to a path
- `commandSaveAsKey` - Saves the ouput in-memory under a specified `key`
These saved ouputs can then be used later through the use of the `key` options:
- `KeyFile` - executes the commands in the file specified under the `path`
- `KeyName` - executes the commands saved in-memory under the specified `key`
The below example will create a command Key under the name `joinKey` (JSON format) :
```json
{
"name" : "Generate a join token",
"type" : "command",
"command" : "kubeadm token create --print-join-command 2>/dev/null",
"commandSaveAsKey" : "joinKey"
}
```
This key can now be used in a different deployment with different hosts (YAML format):
```yaml
- type: "command"
name: "Join worker to Kubernetes cluster"
keyName: "joinKey"
commandSudo : "root"
```
#### Piping data between commands
In the event that data needs to piped into a remote command the options `commandPipeFile` and `commandPipeCmd` can be used. The first will take the contents of `path` and pass it as `STDIN` to the command being executed under the option `command`. The `commandPipeCmd` will execute a command locally and pass the `STDOUT` of that command into the `STDIN` of the command being ran under the `command` option.
The below example will run the command `echo "deb https://apt.kubernetes.io/ kubernetes-xenial main"` locally, and pass the `STDOUT` to the command `tee /etc/apt/sources.list.d/kubernetes.list` that is being ran using `sudo` privileges.
```yaml
- type: command
command: "tee /etc/apt/sources.list.d/kubernetes.list"
commandPipeCmd: echo "deb https://apt.kubernetes.io/ kubernetes-xenial main"
name: Set Kubernetes Repository
commandSudo: root
```
This is useful for a variety of usecases, although it has been found very useful for appending data to existing files that require elevated privilieges.
**Example reasons for piping data to a command**
The command `echo "SOME data" | tee /a/file/that/needs/sudo/privs` will fail even with `commandSudo`, the reason for this is that the `sudo` is only going to work for everything upto the pipe. The remaining part of the command will be ran as the current user and therefore doesn't have the required privileges.
### Upload / Download of files
Both of the command types `upload` and `download` have the same set of options:
- `destination` - Where the file will be once the `upload`/`download` has completed
- `source` - The file that will be either `uploaded`/`downloaded`
- `name` - Details what the action will be doing
```yaml
- type: download
destination: ./ubuntu.tar.gz
name: Retrieve local copy of ubuntu.tar.gz
source: ./ubuntu.tar.gz
```
### Plugins
Plugins allow the creation of unique actions to be performed, such as specific interactions with platforms, programs and infrastructure. All plugins will load at startup and register their actions into the parlay engine. Passing information to a plugin should be done in the following manner:
```yaml
- name: Push kubernetes images for managers
plugin:
imageName:
- k8s.gcr.io/kube-apiserver:v1.14.0
- k8s.gcr.io/kube-controller-manager:v1.14.0
- k8s.gcr.io/kube-scheduler:v1.14.0
localSudo: true
remoteSudo: true
type: docker/image
```
The main differences are:
- `plugin` - Contains all of the specifics that will be passed to the plugin logic
- `type` - Should be the action defined by the plugin itself.
## Additonal configuration
### No Password sudo
To enable password-less sudo the `/etc/sudoers` file needs modifying (DO NOT DO THIS MANUALLY).
To edit the sudo file use the following command:
```
sudo visudo
```
Then add the following entry to the end of the file, replacing the `username` with the correct entry :
```
username ALL=(ALL) NOPASSWD:ALL
```
This can be tested by either opening a new session or logging out and back in and then testing that `sudo <CMD>` doesn't require a password.
================================================
FILE: docs/application_architecture.md
================================================
# Application Architecture
The purpose of this document is to outline the architecture of the plunder program itself, as it has predominantly been developed by a single developer the logic sometimes is hard to fathom (or to understand after a period of absence)
## Application Server routine
When starting plunder as a server for deployment a number of files are parsed and internal structures populated, below is a step through of the actions that take place.
### Starting the server (HTTP Enabled)
We will start `plunder` with a *default* json configuration, with the services enabled and pointing to a default ubuntu kernel/initrd. The deployment file has a single server defined in it.
`plunder server --config ./config.json --deployment ./deployment.json`
1. Plunder starts
- parses flags
- parses global `config.json`
2. Plunder will start services enabled in the configuration `controller.StartServices(deployment)` (`cmd\server.go`)
3. If a deployment file is passed then it should be parsed `err := UpdateConfiguration(deployment)` (`pkg\server\services.go`)
- The parsing of this will generate strings that are mapped to urls that are tracked in a map `httpPaths`
- The function `UpdateConfiguration(configFile []byte)` (`pkg/server/generator.go`) will generate these in memory by iterating through the file and checking the deployment type.
4. HTTP Server is started with `err := c.serveHTTP()`(`pkg\server\services.go`)
5. This function will create a number of prebuilt PXE boot strings using the kernels etc. from `config.json`, configurations such as `/preboot.ipxe` etc.
6. In the event that new configuration is passed to the server then steps 3 are ran again.
### Client connections
1. A Host starts and proceeds to PXE boot, by doing a DHCP request.
2. The DHCP server defaults to point the `BootFileName` Option `dhcp.OptionBootFileName:[]byte(*c.PXEFileName)`(`pkg/server/services.go`), also checks for the dhcp option `77` saying `iPXE` (which will be false)
3. This is passed of TFTP to the booting host which will start iPXE and re-do a DHCP request
4. This time however the DHCP client will have the option `77` set to `iPXE` which means that it's ready for provisioning.
5. The DHCP server will look for an existing configuration `deploymentType := FindDeployment(mac)` (`pkg/server/serve_dhcp.go`), which should return `preseed` etc.
6. The DHCP server will then look to see if a specific configuration has been created with `if httpPaths[fmt.Sprintf("%s.ipxe", dashMac)] == ""` (`pkg/server/serve_dhcp.go`), if not it will default to a deployment type
7. If there exists a pre-defined configuration then it will set the DHCP option to that.
================================================
FILE: docs/deployment.md
================================================
# Deployment Configuration
## Generating a configuration
A `./plunder config deployment > deployment.json` will create a blank deployment configuration that can be pre-populated in order to create specific deployments.
A configured deployment should resemble something like the example below:
```json
{
"globalConfig": {
"adapter": "ens192",
"gateway": "192.168.0.1",
"subnet": "255.255.255.0",
"nameserver": "192.168.0.1",
"ntpserver": "192.168.0.1",
"username": "user",
"password": "pass",
"repoaddress": "192.168.0.1",
"mirrordir": "/ubuntu",
"sshkeypath": "/home/deploy/.ssh/id_pub.rsa",
"sshkey": "ssh-rsa AABBCCDDEE1122334455",
"packages": "nginx openssh-server"
},
"deployments": [
{
"mac": "00:11:22:33:44:55",
"bootConfigName": "default",
"bootConfig": {
"configName": "",
"kernelPath": "",
"initrdPath": "",
"cmdline": ""
},
"config": {
"address": "192.168.0.2",
"hostname": "Server01"
}
}
]
}
```
## Configuration overview
The *globalConfig* is the configuration that is inherited by any of the deployment configurations where that information has been omitted, typically a lot of networking information, keys or package information will be shared amongst deployments.
Placing the same information into an actual deployment will **override** the configuration inherited from the `globalConfig`.
### Shared Configuration overview
- `gateway` - The gateway a server will be configured to use as default router
- `subnet` - The network range server will be configured to use
- `nameserver` - DNS server to resolve hostnames
- `ntpserver` - The address of a timeserver
- `adapter` - Which specific adapter will be configured
- `swapEnabled` - Build the Operating system without swap being created
- `username` - A default user that will be created
- `password` - A password for the above user
- `repoaddress` - The hostname/ip address of the server where the OS packages reside
- `sshkeypath` - The path to an ssh key that will be added to the image for authenticating
### Deployment specific
- `address` - A unique network address that will be added to the server
- `hostname` - A unique hostname to be added to the provisioned server
As mentioned above, a lot of fields can be ignored and the entry from the `globalConfig` will be used.
### Deployments
The deployment contains things that will make a server unique!
- `mac` - The unqique HW mac address of a server to configure
- `kernelPath` - If a specific kernel should be used (for things like LinuxKit)
- `initrdPath` - If a specific init ramdisk should be used
- `cmdline` - Any arguments that should be passed to the kernel ramdisk
The `deployment` specifies how the server will be provisioned, there are three options:
- `preseed` Ubuntu/Debian pressed deployment
- `kickstart` CentOS/RHEL deployment
- `reboot` This is for servers that need to be kept on a reboot loop.
The remaining `config` allows updates or overrides to the global confgiguration detailed above.
### Online updates of deployment configuration
The webserver exposes a `/deployment` end point that can be used to provide an online update of the configuration, this has the following benefits:
- Allows automation of updates, through things like an API call
- Provides no-downtime, stopping and starting the server to load a new configuration can result in a broken installation as the network connection will be broken during restart
*Retrieve the existing configuration*
The currently active configuration can be retrieved through a simple get on the `/deployment` endpoint
e.g.
`curl -vX <IP ADDRESS>/deployment`
*Updating the configuration*
The configuration can be updated by `POST`ing the configuration JSON to the same URL.
e.g.
`curl -vX POST deploy01/deployment -d @deployment.json --header "Content-Type: application/json"`
## Usage
With configuration for both the services and the deployments completed, they can both be passed to `plunder` in order for servers to be built.
As shown below:
```
sudo ./plunder server --config ./config.json --deployment ./deployment.json --logLevel 5
[sudo] password for dan:
INFO[0000] Reading configuration from [./config.json]
INFO[0000] Starting Remote Boot Services, press CTRL + c to stop
DEBU[0000]
Server IP: 192.168.1.1
Adapter: ens192
Start Address: 192.168.1.2
Pool Size: 100
INFO[0000] RemoteBoot => Starting DHCP
INFO[0000] RemoteBoot => Starting TFTP
DEBU[0000]
Server IP: 192.168.1.1
PXEFile: undionly.kpxe
INFO[0000] Opening and caching undionly.kpxe
INFO[0000] RemoteBoot => Starting HTTP
INFO[0286] DCHP Message: Discover
```
## Next Steps
Servers that have their mac addresses in the `deployment` file will be passed the correct bootloader and they will ultimately be provisioned with the networking information as part of the configuration, they also will be provisioned with the credentials and specified ssh key.
For provisioning applications or a platform details are [here](./provisioning.md).
================================================
FILE: docs/example_architecture.md
================================================
# Example architecture
This document outlines an example architecture that one can consider when structuring or designing a network that will ultimately make use of servers bootstrapped by plunder.
## Infrastructure design
In the architecture below the blue cube is the server or VM that will host Plunder and expose it's services. This machine has two adapters `ens160` and `ens192` although they could well be `eth0`/`eth1` depending on your Linux distribution.
The adapter `ens160` is connected to a public network where a user can connect to it's exposed IP address (`192.168.0.100`) over a protocol such as SSH, in order to interact with the OS (and plunder). The second adapter `ens192` is connected to a private network, where a number of other hosts as repeatedly rebooting waiting for a bootstrap server to provision them.
## Services Overview
The services that plunder can expose will bind to the existing operating system in two ways.
#### DHCP
This will ultimately bind to an adapter, and this adapter should be configured with an address.
#### TFTP
This will ultimately bind to an IP address.
#### HTTP
This will also bind to an IP address.
## Example Plunder usage
The CLI examples below don't make use of any configuration files or dynamic updates and provide a quick and easy way of exposing multiple services from Plunder.
```
sudo ./plunder server \
--adapter ens192 \
--enableDHCP \
--enableTFTP \
--enableHTTP \
--initrd initrd.gz \
--kernel kernel \
--cmdline "console=tty0" \
--addressDHCP 192.168.1.1 \
--startAddress 192.168.1.130 \
--addressTFTP 192.1.1.1 \
--addressHTTP 192.168.1.1 \
--anyboot
```
To understand the CLI line above, we will break it down into what some of the more hard-to-understand flags actually are doing.
- `--adapter <...>` This specified which adapter DHCP will broadcast from
- `--enableXXXX` Enable a specific service, in most cases all will be needed unless existing services already exist.
- `--addressDHCP <x.x.x.x>` This is the address that should be configured on the adapter that you're binding too.
- `--addressTFTP/HTTP` This can either be the same address as above or an address of an existing service
- `--startAddress <x.x.x.x>` This is the beginning on the advertised DHCP addresses.
**Note** `sudo` has to be used as binding to an adapter and ports <1024 requires root privileges.
## Example Plunder usage with Linuxkit
If you're using [LinuxKit](https://github.com/linuxkit/linuxkit) images then they can be consumed in the same way as described above. We've simply copied the created OS image files from linuxkit and copied them to our deployment server in the `~/linuxkit/` folder.
```
sudo ./plunder server \
--adapter ens192 \
--enableDHCP \
--enableTFTP \
--enableHTTP \
--initrd linuxkit/linuxkit-initrd.img \
--kernel linuxkit/linuxkit-kernel \
--cmdline $(cat ./linuxkit/linuxkit-cmdline) \
--addressDHCP 192.168.1.1 \
--startAddress 192.168.1.130 \
--addressTFTP 192.1.1.1 \
--addressHTTP 192.168.1.1 \
--anyboot
```
================================================
FILE: docs/example_deployment.md
================================================
# Example Deployment for off-line Kubernetes
**This example will make use of Plunders User Interface**
In order for an offline installation to be succesful, a lot of the packages and containers will need downloading to where Plunder will be ran from.
## Offline Calico parts
### Download the manifests
```
wget https://docs.projectcalico.org/v3.5/getting-started/kubernetes/installation/hosted/etcd.yaml
```
and
```
wget https://docs.projectcalico.org/v3.5/getting-started/kubernetes/installation/hosted/calico.yaml
```
### Download the named images
One Liner to pull the calico images and etcd image
```
for image in $(cat etcd.yaml | grep image | awk '{ print $2 }') ; do sudo docker pull $image; done
```
```
for image in $(cat calico.yaml | grep image | awk '{ print $2 }') ; do sudo docker pull $image; done
```
At this point you'll have the images as part of the local docker repository and the two manifests in the local directory.
## Offline Ubuntu packages
One liner to get the packages needed for the kubernetes hosts to run `kubelet`
```
apt-get download socat ethtool ebtables; tar -cvzf ubuntu_pkg.tar.gz socat* ethtool* ebtables*; rm socat* ethtool* ebtables*
```
This command will download everything needed into an archive named `ubuntu_pkg.tar.gz`
## Offline Docker packages
One liner to get the docker-ce packages for all kubernetes hosts, ensure that the docker repository has been added to the hosts repositories before attempting to download the package.
```
apt-get download docker-ce=18.06.1~ce~3-0~ubuntu; tar -cvzf docker_pkg.tar.gz ./docker-ce_18.06.1~ce~3-0~ubuntu_amd64.deb; rm docker-ce_18.06.1~ce~3-0~ubuntu_amd64.deb
```
## Offline Kubernetes packages
One liner to get the kubernetes packages for all kubernetes hosts, ensure that the kubernets repository has been added to the hosts repositories before attempting to download the package.
```
apt-get download kubelet kubeadm kubectl cri-tools kubernetes-cni; tar -cvzf kubernetes_pkg.tar.gz kubelet* kubeadm* kubectl* cri-tools* kubernetes-cni*; rm kubelet* kubeadm* kubectl* cri-tools* kubernetes-cni*
```
## Offline Kubernetes images
The easiest way of managing this is to install kubeadm on the pluder host and use `kubeadm` to prep the local docker image store with the images needed.
`kubeadm config images list` - will list all images
`kubeadm config images pull` - will pull them all to the local host
Once all of the images have been pulled locally or downloaded as tars manually from the registry we can modify out deployment map and deploy as expected.
## Example deployment map
There is an example deployment map as a `gist` available [https://gist.github.com/thebsdbox/f12b621a9d3943128b6bb16688497cd0](https://gist.github.com/thebsdbox/f12b621a9d3943128b6bb16688497cd0)
## Deployment in action
[](https://asciinema.org/a/reh3reEgJQKCOB5e92D96l6tt)
================================================
FILE: docs/provisioning.md
================================================
# Provisioning Configuration
The provisioning works by running remote commands or uploading/downloading files to a remote system, in order for it to be configured correctly. A parsing engine called "parlay" was written in order to provide repeatable scripting to ease deployments.
A Deployment map can contain multiple **deployments**, which in turn will contain one or more **actions** that will be performed on one or more **hosts**.
Also a deployment map can be parsed as either **JSON** or as **YAML** (yaml being somewhat easier to read as a human and creating much smaller files).
### Example deployment map
This script below (for offline installations) will upload a tarball containing the docker packages and then install them on all remote systems listed under `hosts`.
**Note** the tarball was created by `apt-get download docker-ce=18.06.1~ce~3-0~ubuntu; tar -cvzf docker_pkg.tar.gz ./docker-ce_18.06.1~ce~3-0~ubuntu_amd64.deb`
#### JSON Example
```json
{
"deployments": [
{
"name": "Upload Docker Packages",
"parallel": false,
"sessions": 0,
"hosts": [
"192.168.1.3",
"192.168.1.4",
"192.168.1.5"
],
"actions": [
{
"name": "Upload Docker Packages",
"type": "upload",
"source": "./docker_pkg.tar.gz",
"destination": "/tmp/docker_pkg.tar.gz"
},
{
"name": "Extract Docker packages",
"type": "command",
"command": "tar -C /tmp -xvzf /tmp/docker_pkg.tar.gz"
},
{
"name": "Install Docker packages",
"type": "command",
"command": "dpkg -i /tmp/docker/*",
"commandSudo": "root"
}
]
}
]
}
```
#### YAML Example
```yaml
deployments:
- actions:
- destination: /tmp/docker_pkg.tar.gz
name: Upload Docker Packages
source: ./docker_pkg.tar.gz
timeout: 0
type: upload
- command: tar -C /tmp -xvzf /tmp/docker_pkg.tar.gz
name: Extract Docker packages
timeout: 0
type: command
- command: dpkg -i /tmp/docker/*
commandSudo: root
name: Install Docker packages
timeout: 0
type: command
hosts:
- 192.168.1.3
- 192.168.1.4
- 192.168.1.5
name: Upload Docker Packages
parallel: false
parallelSessions: 0
```
The above example only covers simple usage of `uploading` and `command` usages.
## Usage
When automating a deployment ssh credentials are required to map a host with the correct credentials.
To simplify this `plunder` can make use of:
- A `deployment` file as detailed [here](./deployment.md), which parlay will extract the `ssh` information from to allow authentication
- A **deployment endpoint**, which is effectively the url of a running plunder instance. Parlay will evaluate the endpoint for the configuration details to allow authentication.
**Example**
Using a map to deploy wordpress (`wordpress.yaml`) and a local deployment file.
`plunder automate ssh --map ./wordpress.yaml --deployconfig ./deployment.json`
Using a map to deploy wordpress (`wordpress.yaml`) and a deployment endpoint.
`plunder automate ssh --map ./wordpress.yaml --deployendpoint http://localhost`
It is possible to override or completely omit deployment configuration and specify the configuration at runtime through the flags `--override{Address/Keypath/Username}`. By **default** plunder will attempt to populate the Keypath and username from the current user and their `$HOME/.ssh/` directory.
`/plunder automate --map ./stackedmanager.yaml --overrideAddress 192.168.1.105`
Under most circumstances plunder will execute all actions in every deployment (on every host in the deployment), however it is possible to tell plunder to execute a single deployment/action from a map and on which particular host.
Additional Flags:
- The `--deployment` flag now will point to a specific deployment in a map
- The `--action` flag can be used to point to a specific action in a deployment
- The `--host` flag will point to a specific host in the deployment
- The `--resume` will determine if to continue executing all remaining actions
### User Interface
Plunder can also make automation easier by providing a user interface for a map and allowing the user to select which Deployments, actions and the hosts that will be acted upon. To use the user interface the subcommand `ui` should be used, all other flags are the same as above.
**Example**
```
plunder automate ui --map ./stackedmanager.yaml --deployendpoint http://localhost
INFO[0000] Reading deployment configuration from [./stackedmanager.yaml]
? Select deployment(s) [Use arrows to move, type to filter]
> [ ] Reset any Kubernetes configuration (and remove packages)
[ ] Configure host OS for kubernetes nodes
[ ] Deploy Kubernetes Images for 1.14.0
[ ] Initialise Kubernetes Master (1.14)
[ ] Deploy Calico (3.6)
```
The UI also provides additional capability to create new maps based upon selected deployments and actions, and also to convert between formats.
- `--json` Print the JSON to stdout, no execution of commands
- `--yaml` Print the YAML to stdout, no execution of commands
**Execution of a map is shown in the screen shot below**
*The above example uses screen, where the output from `plunder` is on the top and `tail -f output` is below*
================================================
FILE: docs/readme.md
================================================
# Plunder Usage
When using `plunder` there are a few things that you will need to ensure that a configuration exists for, these things are:
- Service configuration (IP Addresses, adapter names, service enablement)
- Deployment configuration (MAC Addresses, Package management, networking)
- Provisioning configuration (File transfer, remote command execution)
Most of the configuration required will be automatically generated by `plunder` through the use of the `plunder config` sub command.
To view an example architecture and quick usage than look [here](./example_architecture.md).
### Service
The services such as DHCP and TFTP etc.. are the basic requirement in order to bootstrap a **blank** bare-metal server or new virtual machine.
Service configuration overview and usage is located [here](./service.md).
### Deployment
Once a **blank** server boots it will need an Operating System (+ packages) installing, along with setting up networking and credentials.
Deployment configuration overview and usage is located [here](./deployment.md).
### Provisioning
Once a server has been deployed it is on to provisioning that server for a particular use case, such as a docker swarm cluster or a kubernetes platform
A Provisioning overview with usage is located [here](./provisioning.md).
================================================
FILE: docs/service.md
================================================
# Service Configuration
## Generating a configuration
A `./plunder config server > config.json` will look at the network configuration of the current machine and build a default configuration file (in json). This file will need opening in your favourite text editor an modifying to ensure that `plunder` works correctly.
### Modifying the configuration
```json
{
"adapter": "en0",
"enableDHCP": false,
"dhcpConfig": {
"addressDHCP": "192.168.0.142",
"startDHCP": "192.168.0.143",
"leasePoolDHCP": 20,
"gatewayDHCP": "192.168.0.142",
"nameserverDHCP": "192.168.0.142"
},
"enableTFTP": false,
"addressTFTP": "192.168.0.142",
"enableHTTP": false,
"addressHTTP": "192.168.0.142",
"pxePath": "undionly.kpxe",
"bootConfigs": [
{
"configName": "default",
"kernelPath": "/kernelPath",
"initrdPath": "/initPath",
"cmdline": "cmd=options",
"isoPrefix": "ubuntu",
"isoPath": "/path/to/iso"
}
]
}
```
*Example generated configuration above*
### Sections
By **default** the configuration that is generated will have all of the services disabled (dhcp/tftp/http) and attempting to start plunder will result in an error message saying that no services are being started.
#### Services
The `enable<service>` will ensure that a particular functionality is enabled within Plunder.
The `addressTFTP` and `addressHTTP` are still required to be set even if you're not enabling the service, this is because those values will be passed through `DHCP` to a server that is being bootstrapped. So if `TFTP` or `HTTP` services already exist on your network, then modify those values accordingly.
#### DHCP
The `dhcpConfig` section details all of the configuration for the running DHCP server such as the `startDHCP` setting which should typically be `addressDHCP` +1 and then the `leasePoolDHCP` defines how many free addresses will be allocated sequentially from the start address.
#### Boot Configurations
The boot configurations are an array of configurations that define various remote booting configurations and are referenced via the `configName`.
The `kernelPath` and `initrdPath` should point to a kernel and init ramdisk on the local filesystem that will be passed to the server once the bootloader has finished.
Finally, the `isoPrefix` (determines the beginning and unique path to contents) and the `isoPath` allow OS installation content to be read from within an ISO file.
e.g.
`plunderAddress/isoPrefix/path/to/file`
#### Additional
The `pxePath` should point to an iPXE bootloader if needed, however if the file doesn't exist or if the option is blank then `plunder` will fall back to an embedded bootloader.
## Usage
At this point you can start various services and you'll see servers on the network requesting `DHCP` addresses etc.. however in order to do anything we will need to configure the [deployment](./deployment.md).
================================================
FILE: go.mod
================================================
module github.com/plunder-app/plunder
go 1.12
require (
github.com/AlecAivazis/survey/v2 v2.0.7 // indirect
github.com/c4milo/gotoolkit v0.0.0-20190525173301-67483a18c17a // indirect
github.com/ghodss/yaml v1.0.0
github.com/gorilla/mux v1.7.4 // indirect
github.com/hooklift/assert v0.0.0-20170704181755-9d1defd6d214 // indirect
github.com/hooklift/iso9660 v1.0.0 // indirect
github.com/kr/pty v1.1.8 // indirect
github.com/krolaw/dhcp4 v0.0.0-20190909130307-a50d88189771 // indirect
github.com/mattn/go-colorable v0.1.6 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pkg/sftp v1.11.0 // indirect
github.com/plunder-app/BOOTy v0.0.0-20200513203223-f43f6ea742c4
github.com/plunder-app/plunder/pkg/apiserver v0.0.0-20200514155151-dfdcaab2e5cd
github.com/plunder-app/plunder/pkg/certs v0.0.0-20200514155151-dfdcaab2e5cd
github.com/plunder-app/plunder/pkg/parlay v0.0.0-20200514155151-dfdcaab2e5cd
github.com/plunder-app/plunder/pkg/parlay/parlaytypes v0.0.0-20200514155151-dfdcaab2e5cd
github.com/plunder-app/plunder/pkg/plunderlogging v0.0.0-20200514155151-dfdcaab2e5cd // indirect
github.com/plunder-app/plunder/pkg/services v0.0.0-20200514155151-dfdcaab2e5cd
github.com/plunder-app/plunder/pkg/ssh v0.0.0-20200514155151-dfdcaab2e5cd
github.com/plunder-app/plunder/pkg/utils v0.0.0-20200514155151-dfdcaab2e5cd
github.com/sirupsen/logrus v1.6.0
github.com/spf13/cobra v1.0.0
github.com/spf13/pflag v1.0.5 // indirect
github.com/thebsdbox/go-tftp v0.0.0-20190329154032-a7263f18c49c // indirect
github.com/whyrusleeping/go-tftp v0.0.0-20180830013254-3695fa5761ee // indirect
golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37 // indirect
golang.org/x/net v0.0.0-20200513185701-a91f0712d120 // indirect
golang.org/x/text v0.3.2 // indirect
)
replace (
github.com/plunder-app/plunder/pkg/apiserver => ./pkg/apiserver
github.com/plunder-app/plunder/pkg/certs => ./pkg/certs
github.com/plunder-app/plunder/pkg/parlay => ./pkg/parlay
github.com/plunder-app/plunder/pkg/services => ./pkg/services
github.com/plunder-app/plunder/pkg/ssh => ./pkg/ssh
github.com/plunder-app/plunder/pkg/utils => ./pkg/utils
github.com/plunder-app/BOOTy => ../../plunder-app/BOOTy
)
================================================
FILE: go.sum
================================================
cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=
github.com/AlecAivazis/survey/v2 v2.0.7 h1:+f825XHLse/hWd2tE/V5df04WFGimk34Eyg/z35w/rc=
github.com/AlecAivazis/survey/v2 v2.0.7/go.mod h1:mlizQTaPjnR4jcpwRSaSlkbsRfYFEyKgLQvYTzxxiHA=
github.com/BurntSushi/toml v0.3.1/go.mod h1:xHWCNGjB5oqiDr8zfno3MHue2Ht5sIBksp03qcyfWMU=
github.com/Netflix/go-expect v0.0.0-20180615182759-c93bf25de8e8 h1:xzYJEypr/85nBpB11F9br+3HUrpgb+fcm5iADzXXYEw=
github.com/Netflix/go-expect v0.0.0-20180615182759-c93bf25de8e8/go.mod h1:oX5x61PbNXchhh0oikYAH+4Pcfw5LKv21+Jnpr6r6Pc=
github.com/OneOfOne/xxhash v1.2.2/go.mod h1:HSdplMjZKSmBqAxg5vPj2TmRDmfkzw+cTzAElWljhcU=
github.com/alecthomas/template v0.0.0-20160405071501-a0175ee3bccc/go.mod h1:LOuyumcjzFXgccqObfd/Ljyb9UuFJ6TxHnclSeseNhc=
github.com/alecthomas/units v0.0.0-20151022065526-2efee857e7cf/go.mod h1:ybxpYRFXyAe+OPACYpWeL0wqObRcbAqCMya13uyzqw0=
github.com/armon/consul-api v0.0.0-20180202201655-eb2c6b5be1b6/go.mod h1:grANhF5doyWs3UAsr3K4I6qtAmlQcZDesFNEHPZAzj8=
github.com/beorn7/perks v0.0.0-20180321164747-3a771d992973/go.mod h1:Dwedo/Wpr24TaqPxmxbtue+5NUziq4I4S80YR8gNf3Q=
github.com/beorn7/perks v1.0.0/go.mod h1:KWe93zE9D1o94FZ5RNwFwVgaQK1VOXiVxmqh+CedLV8=
github.com/c4milo/gotoolkit v0.0.0-20190525173301-67483a18c17a h1:+uvtaGSLJh0YpLLHCQ9F+UVGy4UOS542hsjj8wBjvH0=
github.com/c4milo/gotoolkit v0.0.0-20190525173301-67483a18c17a/go.mod h1:txokOny9wavBtq2PWuHmj1P+eFwpCsj+gQeNNANChfU=
github.com/cespare/xxhash v1.1.0/go.mod h1:XrSqR1VqqWfGrhpAt58auRo0WTKS1nRRg3ghfAqPWnc=
github.com/client9/misspell v0.3.4/go.mod h1:qj6jICC3Q7zFZvVWo7KLAzC3yx5G7kyvSDkc90ppPyw=
github.com/coreos/bbolt v1.3.2/go.mod h1:iRUV2dpdMOn7Bo10OQBFzIJO9kkE559Wcmn+qkEiiKk=
github.com/coreos/etcd v3.3.10+incompatible/go.mod h1:uF7uidLiAD3TWHmW31ZFd/JWoc32PjwdhPthX9715RE=
github.com/coreos/go-semver v0.2.0/go.mod h1:nnelYz7RCh+5ahJtPPxZlU+153eP4D4r3EedlOD2RNk=
github.com/coreos/go-systemd v0.0.0-20190321100706-95778dfbb74e/go.mod h1:F5haX7vjVVG0kc13fIWeqUViNPyEJxv/OmvnBo0Yme4=
github.com/coreos/pkg v0.0.0-20180928190104-399ea9e2e55f/go.mod h1:E3G3o1h8I7cfcXa63jLwjI0eiQQMgzzUDFVpN/nH/eA=
github.com/cpuguy83/go-md2man/v2 v2.0.0/go.mod h1:maD7wRr/U5Z6m/iR4s+kqSMx2CaBsrgA7czyZG/E6dU=
github.com/creack/pty v1.1.7 h1:6pwm8kMQKCmgUg0ZHTm5+/YvRK0s3THD/28+T6/kk4A=
github.com/creack/pty v1.1.7/go.mod h1:lj5s0c3V2DBrqTV7llrYr5NG6My20zk30Fl46Y7DoTY=
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/dgrijalva/jwt-go v3.2.0+incompatible/go.mod h1:E3ru+11k8xSBh+hMPgOLZmtrrCbhqsmaPHjLKYnJCaQ=
github.com/dgryski/go-sip13 v0.0.0-20181026042036-e10d5fee7954/go.mod h1:vAd38F8PWV+bWy6jNmig1y/TA+kYO4g3RSRF0IAv0no=
github.com/digineo/go-dhclient v1.0.2/go.mod h1:DPvyqGEW8irJvp2lrnGfQWpjj6VidXX9STLBTfNing4=
github.com/dustin/go-humanize v1.0.0 h1:VSnTsYCnlFHaM2/igO1h6X3HA71jcobQuxemgkq4zYo=
github.com/dustin/go-humanize v1.0.0/go.mod h1:HtrtbFcZ19U5GC7JDqmcUSB87Iq5E25KnS6fMYU6eOk=
github.com/fsnotify/fsnotify v1.4.7/go.mod h1:jwhsz4b93w/PPRr/qN1Yymfu8t87LnFCMoQvtojpjFo=
github.com/ghodss/yaml v1.0.0 h1:wQHKEahhL6wmXdzwWG11gIVCkOv05bNOh+Rxn0yngAk=
github.com/ghodss/yaml v1.0.0/go.mod h1:4dBDuWmgqj2HViK6kFavaiC9ZROes6MMH2rRYeMEF04=
github.com/go-kit/kit v0.8.0/go.mod h1:xBxKIO96dXMWWy0MnWVtmwkA9/13aqxPnvrjFYMA2as=
github.com/go-logfmt/logfmt v0.3.0/go.mod h1:Qt1PoO58o5twSAckw1HlFXLmHsOX5/0LbT9GBnD5lWE=
github.com/go-logfmt/logfmt v0.4.0/go.mod h1:3RMwSq7FuexP4Kalkev3ejPJsZTpXXBr9+V4qmtdjCk=
github.com/go-stack/stack v1.8.0/go.mod h1:v0f6uXyyMGvRgIKkXu+yp6POWl0qKG85gN/melR3HDY=
github.com/gogo/protobuf v1.1.1/go.mod h1:r8qH/GZQm5c6nD/R0oafs1akxWv10x8SbQlK7atdtwQ=
github.com/gogo/protobuf v1.2.1/go.mod h1:hp+jE20tsWTFYpLwKvXlhS1hjn+gTNwPg2I6zVXpSg4=
github.com/golang/glog v0.0.0-20160126235308-23def4e6c14b/go.mod h1:SBH7ygxi8pfUlaOkMMuAQtPIUF8ecWP5IEl/CR7VP2Q=
github.com/golang/groupcache v0.0.0-20190129154638-5b532d6fd5ef/go.mod h1:cIg4eruTrX1D+g88fzRXU5OdNfaM+9IcxsU14FzY7Hc=
github.com/golang/mock v1.1.1/go.mod h1:oTYuIxOrZwtPieC+H1uAHpcLFnEyAGVDL/k47Jfbm0A=
github.com/golang/protobuf v1.2.0/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/golang/protobuf v1.3.1/go.mod h1:6lQm79b+lXiMfvg/cZm0SGofjICqVBUtrP5yJMmIC1U=
github.com/google/btree v1.0.0/go.mod h1:lNA+9X1NB3Zf8V7Ke586lFgjr2dZNuvo3lPJSGZ5JPQ=
github.com/google/go-cmp v0.2.0/go.mod h1:oXzfMopK8JAjlY9xF4vHSVASa0yLyX7SntLO5aqRK0M=
github.com/google/gopacket v1.1.17/go.mod h1:UdDNZ1OO62aGYVnPhxT1U6aI7ukYtA/kB8vaU0diBUM=
github.com/gorilla/mux v1.7.4 h1:VuZ8uybHlWmqV03+zRzdwKL4tUnIp1MAQtp1mIFE1bc=
github.com/gorilla/mux v1.7.4/go.mod h1:DVbg23sWSpFRCP0SfiEN6jmj59UnW/n46BH5rLB71So=
github.com/gorilla/websocket v1.4.0/go.mod h1:E7qHFY5m1UJ88s3WnNqhKjPHQ0heANvMoAMk2YaljkQ=
github.com/grpc-ecosystem/go-grpc-middleware v1.0.0/go.mod h1:FiyG127CGDf3tlThmgyCl78X/SZQqEOJBCDaAfeWzPs=
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0/go.mod h1:8NvIoxWQoOIhqOTXgfV/d3M/q6VIi02HzZEHgUlZvzk=
github.com/grpc-ecosystem/grpc-gateway v1.9.0/go.mod h1:vNeuVxBJEsws4ogUvrchl83t/GYV9WGTSLVdBhOQFDY=
github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
github.com/hinshun/vt10x v0.0.0-20180616224451-1954e6464174 h1:WlZsjVhE8Af9IcZDGgJGQpNflI3+MJSBhsgT5PCtzBQ=
github.com/hinshun/vt10x v0.0.0-20180616224451-1954e6464174/go.mod h1:DqJ97dSdRW1W22yXSB90986pcOyQ7r45iio1KN2ez1A=
github.com/hooklift/assert v0.0.0-20170704181755-9d1defd6d214 h1:WgfvpuKg42WVLkxNwzfFraXkTXPK36bMqXvMFN67clI=
github.com/hooklift/assert v0.0.0-20170704181755-9d1defd6d214/go.mod h1:kj6hFWqfwSjFjLnYW5PK1DoxZ4O0uapwHRmd9jhln4E=
github.com/hooklift/iso9660 v1.0.0 h1:GYN0ejrqTl1qtB+g+ics7xxWHp7J2B1zmr25O9EyG3c=
github.com/hooklift/iso9660 v1.0.0/go.mod h1:sOC47ru8lB0DlU0EZ7BJ0KCP5rDqOvx0c/5K5ADm8H0=
github.com/inconshreveable/mousetrap v1.0.0 h1:Z8tu5sraLXCXIcARxBp/8cbvlwVa7Z1NHg9XEKhtSvM=
github.com/inconshreveable/mousetrap v1.0.0/go.mod h1:PxqpIevigyE2G7u3NXJIT2ANytuPF1OarO4DADm73n8=
github.com/jonboulle/clockwork v0.1.0/go.mod h1:Ii8DK3G1RaLaWxj9trq07+26W01tbo22gdxWY5EU2bo=
github.com/julienschmidt/httprouter v1.2.0/go.mod h1:SYymIcj16QtmaHHD7aYtjjsJG7VTCxuUUipMqKk8s4w=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 h1:Z9n2FFNUXsshfwJMBgNA0RU6/i7WVaAegv3PtuIHPMs=
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51/go.mod h1:CzGEWj7cYgsdH8dAjBGEr58BoE7ScuLd+fwFZ44+/x8=
github.com/kisielk/errcheck v1.1.0/go.mod h1:EZBBE59ingxPouuu3KfxchcWSUPOHkagtvWXihfKN4Q=
github.com/kisielk/gotool v1.0.0/go.mod h1:XhKaO+MFFWcvkIS/tQcRk01m1F5IRFswLeQ+oQHNcck=
github.com/konsorten/go-windows-terminal-sequences v1.0.1 h1:mweAR1A6xJ3oS2pRaGiHgQ4OO8tzTaLawm8vnODuwDk=
github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/konsorten/go-windows-terminal-sequences v1.0.3 h1:CE8S1cTafDpPvMhIxNJKvHsGVBgn1xWYf1NbHQhywc8=
github.com/konsorten/go-windows-terminal-sequences v1.0.3/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ=
github.com/kr/fs v0.1.0 h1:Jskdu9ieNAYnjxsi0LbQp1ulIKZV1LAFgK1tWhpZgl8=
github.com/kr/fs v0.1.0/go.mod h1:FFnZGqtBN9Gxj7eW1uZ42v5BccTP0vu6NEaFoC2HwRg=
github.com/kr/logfmt v0.0.0-20140226030751-b84e30acd515/go.mod h1:+0opPa2QZZtGFBFZlji/RkVcI2GknAs/DXo4wKdlNEc=
github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
github.com/kr/pty v1.1.1/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/pty v1.1.4/go.mod h1:pFQYn66WHrOpPYNljwOMqo10TkYh1fy3cYio2l3bCsQ=
github.com/kr/pty v1.1.8 h1:AkaSdXYQOWeaO3neb8EM634ahkXXe3jYbVh/F9lq+GI=
github.com/kr/pty v1.1.8/go.mod h1:O1sed60cT9XZ5uDucP5qwvh+TE3NnUj51EiZO/lmSfw=
github.com/kr/text v0.1.0 h1:45sCR5RtlFHMR4UwH9sdQ5TC8v0qDQCHnXt+kaKSTVE=
github.com/kr/text v0.1.0/go.mod h1:4Jbv+DJW3UT/LiOwJeYQe1efqtUx/iVham/4vfdArNI=
github.com/krolaw/dhcp4 v0.0.0-20190909130307-a50d88189771 h1:t2c2B9g1ZVhMYduqmANSEGVD3/1WlsrEYNPtVoFlENk=
github.com/krolaw/dhcp4 v0.0.0-20190909130307-a50d88189771/go.mod h1:0AqAH3ZogsCrvrtUpvc6EtVKbc3w6xwZhkvGLuqyi3o=
github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
github.com/mattn/go-colorable v0.1.2 h1:/bC9yWikZXAL9uJdulbSfyVNIR3n3trXl+v8+1sx8mU=
github.com/mattn/go-colorable v0.1.2/go.mod h1:U0ppj6V5qS13XJ6of8GYAs25YV2eR4EVcfRqFIhoBtE=
github.com/mattn/go-colorable v0.1.6 h1:6Su7aK7lXmJ/U79bYtBjLNaha4Fs1Rg9plHpcH+vvnE=
github.com/mattn/go-colorable v0.1.6/go.mod h1:u6P/XSegPjTcexA+o6vUJrdnUu04hMope9wVRipJSqc=
github.com/mattn/go-isatty v0.0.8 h1:HLtExJ+uU2HOZ+wI0Tt5DtUDrx8yhUqDcp7fYERX4CE=
github.com/mattn/go-isatty v0.0.8/go.mod h1:Iq45c/XA43vh69/j3iqttzPXn0bhXyGjM0Hdxcsrc5s=
github.com/mattn/go-isatty v0.0.12 h1:wuysRhFDzyxgEmMf5xjvJ2M9dZoWAXNNr5LSBS7uHXY=
github.com/mattn/go-isatty v0.0.12/go.mod h1:cbi8OIDigv2wuxKPP5vlRcQ1OAZbq2CE4Kysco4FUpU=
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/mdlayher/raw v0.0.0-20191004140158-e1402808046b/go.mod h1:7EpbotpCmVZcu+KCX4g9WaRNuu11uyhiW7+Le1dKawg=
github.com/mdlayher/raw v0.0.0-20191009151244-50f2db8cc065/go.mod h1:7EpbotpCmVZcu+KCX4g9WaRNuu11uyhiW7+Le1dKawg=
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b h1:j7+1HpAFS1zy5+Q4qx1fWh90gTKwiN4QCGoY9TWyyO4=
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
github.com/mitchellh/go-homedir v1.1.0/go.mod h1:SfyaCUpYCn1Vlf4IUYiD9fPX4A5wJrkLzIz1N1q0pr0=
github.com/mitchellh/mapstructure v1.1.2/go.mod h1:FVVH3fgwuzCH5S8UJGiWEs2h04kUh9fWfEaFds41c1Y=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/oklog/ulid v1.3.1/go.mod h1:CirwcVhetQ6Lv90oh/F+FBtV6XMibvdAFo93nm5qn4U=
github.com/pelletier/go-toml v1.2.0/go.mod h1:5z9KED0ma1S8pY6P1sdut58dfprrGBbd/94hg7ilaic=
github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.8.1 h1:iURUrRGxPUNPdy5/HRSm+Yj6okJ6UtLINN0Q9M4+h3I=
github.com/pkg/errors v0.8.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/errors v0.9.1 h1:FEBLx1zS214owpjy7qsBeixbURkuhQAwrK5UwLGTwt4=
github.com/pkg/errors v0.9.1/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
github.com/pkg/sftp v1.11.0 h1:4Zv0OGbpkg4yNuUtH0s8rvoYxRCNyT29NVUo6pgPmxI=
github.com/pkg/sftp v1.11.0/go.mod h1:lYOWFsE0bwd1+KfKJaKeuokY15vzFx25BLbzYYoAxZI=
github.com/plunder-app/BOOTy v0.0.0-20200513203223-f43f6ea742c4 h1:XBPLmj1YuM8GlnueA/IMUKQDVVKP7E7NOjx1SlAd/g0=
github.com/plunder-app/BOOTy v0.0.0-20200513203223-f43f6ea742c4/go.mod h1:ItncOB62Q244Txg1v6/S3XZq3KB+9uHcmmZhJF+5Z+A=
github.com/plunder-app/plunder/pkg/parlay v0.0.0-20200513203243-eccb418a5255 h1:FBXZMKgQ+YD3i18EZlSlnSFyhyHE654d+woetczrsC8=
github.com/plunder-app/plunder/pkg/parlay v0.0.0-20200513203243-eccb418a5255/go.mod h1:5UuNaULcTUSFIkrbe+NA/ufh9iyUEtsKqmjlbl88AVw=
github.com/plunder-app/plunder/pkg/parlay/parlaytypes v0.0.0-20200513203243-eccb418a5255 h1:765Djvc0TdpwZFlEmyq1ruUPx69klzHQOMAMCK1KJZM=
github.com/plunder-app/plunder/pkg/parlay/parlaytypes v0.0.0-20200513203243-eccb418a5255/go.mod h1:QtxXmGRkwdtiiH03oveOPcYXucOv/FxJq2a170aXkxg=
github.com/plunder-app/plunder/pkg/parlay/parlaytypes v0.0.0-20200514154439-933b7120d270 h1:p59jcYFaRO6LOJz/OAcdPNSlbgl1QPH4cmlFs69BZ6g=
github.com/plunder-app/plunder/pkg/parlay/parlaytypes v0.0.0-20200514154439-933b7120d270/go.mod h1:QtxXmGRkwdtiiH03oveOPcYXucOv/FxJq2a170aXkxg=
github.com/plunder-app/plunder/pkg/parlay/parlaytypes v0.0.0-20200514155151-dfdcaab2e5cd h1:CA9lXqJAJxwA/NGepdfE/hQapGplV4CNNal3eZ9C/k0=
github.com/plunder-app/plunder/pkg/parlay/parlaytypes v0.0.0-20200514155151-dfdcaab2e5cd/go.mod h1:QtxXmGRkwdtiiH03oveOPcYXucOv/FxJq2a170aXkxg=
github.com/plunder-app/plunder/pkg/plunderlogging v0.0.0-20200513203243-eccb418a5255 h1:gzlti8QQwa02qAGlFktceQucT4HZUZFWIm6h4b03JBo=
github.com/plunder-app/plunder/pkg/plunderlogging v0.0.0-20200513203243-eccb418a5255/go.mod h1:ketI5Vxh8nmxwiWnS644ZaEwVt9M/bpP6ISaFeWpcS0=
github.com/plunder-app/plunder/pkg/plunderlogging v0.0.0-20200514154439-933b7120d270 h1:abUhCq54uc/4Q/ApJMGtWmsSGSLJcni5P8VXyrLZ49o=
github.com/plunder-app/plunder/pkg/plunderlogging v0.0.0-20200514154439-933b7120d270/go.mod h1:ketI5Vxh8nmxwiWnS644ZaEwVt9M/bpP6ISaFeWpcS0=
github.com/plunder-app/plunder/pkg/plunderlogging v0.0.0-20200514155151-dfdcaab2e5cd h1:luKDb4GvbZlnythTGjypGW8EdCyRSSXwUvbr+ZrB2zU=
github.com/plunder-app/plunder/pkg/plunderlogging v0.0.0-20200514155151-dfdcaab2e5cd/go.mod h1:ketI5Vxh8nmxwiWnS644ZaEwVt9M/bpP6ISaFeWpcS0=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/prometheus/client_golang v0.9.1/go.mod h1:7SWBe2y4D6OKWSNQJUaRYU/AaXPKyh/dDVn+NZz0KFw=
github.com/prometheus/client_golang v0.9.3/go.mod h1:/TN21ttK/J9q6uSwhBd54HahCDft0ttaMvbicHlPoso=
github.com/prometheus/client_model v0.0.0-20180712105110-5c3871d89910/go.mod h1:MbSGuTsp3dbXC40dX6PRTWyKYBIrTGTE9sqQNg2J8bo=
github.com/prometheus/client_model v0.0.0-20190129233127-fd36f4220a90/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/common v0.0.0-20181113130724-41aa239b4cce/go.mod h1:daVV7qP5qjZbuso7PdcryaAu0sAZbrN9i7WWcTMWvro=
github.com/prometheus/common v0.4.0/go.mod h1:TNfzLD0ON7rHzMJeJkieUDPYmFC7Snx/y86RQel1bk4=
github.com/prometheus/procfs v0.0.0-20181005140218-185b4288413d/go.mod h1:c3At6R/oaqEKCNdg8wHV1ftS6bRYblBhIjjI8uT2IGk=
github.com/prometheus/procfs v0.0.0-20190507164030-5867b95ac084/go.mod h1:TjEm7ze935MbeOT/UhFTIMYKhuLP4wbCsTZCD3I8kEA=
github.com/prometheus/tsdb v0.7.1/go.mod h1:qhTCs0VvXwvX/y3TZrWD7rabWM+ijKTux40TwIPHuXU=
github.com/rogpeppe/fastuuid v0.0.0-20150106093220-6724a57986af/go.mod h1:XWv6SoW27p1b0cqNHllgS5HIMJraePCO15w5zCzIWYg=
github.com/russross/blackfriday/v2 v2.0.1/go.mod h1:+Rmxgy9KzJVeS9/2gXHxylqXiyQDYRxCVz55jmeOWTM=
github.com/shurcooL/sanitized_anchor_name v1.0.0/go.mod h1:1NzhyTcUVG4SuEtjjoZeVRXNmyL/1OwPU0+IJeTBvfc=
github.com/sirupsen/logrus v1.2.0/go.mod h1:LxeOpSwHxABJmUn/MG1IvRgCAasNZTLOkJPxbbu5VWo=
github.com/sirupsen/logrus v1.6.0 h1:UBcNElsrwanuuMsnGSlYmtmgbb23qDR5dG+6X6Oo89I=
github.com/sirupsen/logrus v1.6.0/go.mod h1:7uNnSEd1DgxDLC74fIahvMZmmYsHGZGEOFrfsX/uA88=
github.com/soheilhy/cmux v0.1.4/go.mod h1:IM3LyeVVIOuxMH7sFAkER9+bJ4dT7Ms6E4xg4kGIyLM=
github.com/spaolacci/murmur3 v0.0.0-20180118202830-f09979ecbc72/go.mod h1:JwIasOWyU6f++ZhiEuf87xNszmSA2myDM2Kzu9HwQUA=
github.com/spf13/afero v1.1.2/go.mod h1:j4pytiNVoe2o6bmDsKpLACNPDBIoEAkihy7loJ1B0CQ=
github.com/spf13/cast v1.3.0/go.mod h1:Qx5cxh0v+4UWYiBimWS+eyWzqEqokIECu5etghLkUJE=
github.com/spf13/cobra v1.0.0 h1:6m/oheQuQ13N9ks4hubMG6BnvwOeaJrqSPLahSnczz8=
github.com/spf13/cobra v1.0.0/go.mod h1:/6GTrnGXV9HjY+aR4k0oJ5tcvakLuG6EuKReYlHNrgE=
github.com/spf13/jwalterweatherman v1.0.0/go.mod h1:cQK4TGJAtQXfYWX+Ddv3mKDzgVb68N+wFjFa4jdeBTo=
github.com/spf13/pflag v1.0.3 h1:zPAT6CGy6wXeQ7NtTnaTerfKOsV6V6F8agHXFiazDkg=
github.com/spf13/pflag v1.0.3/go.mod h1:DYY7MBk1bdzusC3SYhjObp+wFpr4gzcvqqNjLnInEg4=
github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/viper v1.4.0/go.mod h1:PTJ7Z/lr49W6bUbkmS1V3by4uWynFiR9p7+dSq/yZzE=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/objx v0.1.1/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.2.1/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.2.2 h1:bSDNvY7ZPG5RlJ8otE/7V6gMiyenm9RtJ7IUVIAoJ1w=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0 h1:2E4SXV/wtOkTonXsotYi4li6zVWxYlZuYNCXe9XRJyk=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/thebsdbox/go-tftp v0.0.0-20190329154032-a7263f18c49c h1:cYCrFUo78/407dxOlYw2g4pdm4Ly8RSPedsYB+z7h1s=
github.com/thebsdbox/go-tftp v0.0.0-20190329154032-a7263f18c49c/go.mod h1:yXG6GIu/ptjkk0fd++y96R2cahlvxZr4LhMdf0j/L2Q=
github.com/tmc/grpc-websocket-proxy v0.0.0-20190109142713-0ad062ec5ee5/go.mod h1:ncp9v5uamzpCO7NfCPTXjqaC+bZgJeR0sMTm6dMHP7U=
github.com/ugorji/go v1.1.4/go.mod h1:uQMGLiO92mf5W77hV/PUCpI3pbzQx3CRekS0kk+RGrc=
github.com/vishvananda/netlink v1.1.0 h1:1iyaYNBLmP6L0220aDnYQpo1QEV4t4hJ+xEEhhJH8j0=
github.com/vishvananda/netlink v1.1.0/go.mod h1:cTgwzPIzzgDAYoQrMm0EdrjRUBkTqKYppBueQtXaqoE=
github.com/vishvananda/netns v0.0.0-20191106174202-0a2b9b5464df h1:OviZH7qLw/7ZovXvuNyL3XQl8UFofeikI1NW1Gypu7k=
github.com/vishvananda/netns v0.0.0-20191106174202-0a2b9b5464df/go.mod h1:JP3t17pCcGlemwknint6hfoeCVQrEMVwxRLRjXpq+BU=
github.com/whyrusleeping/go-tftp v0.0.0-20180830013254-3695fa5761ee h1:P2Wwq5QukiLY/I6+mc7NyLFX/atHAj6pGwiVu6fld98=
github.com/whyrusleeping/go-tftp v0.0.0-20180830013254-3695fa5761ee/go.mod h1:ZemSN4DPuG1ppDttxnu45zl8BenKT9xSjMyapUd+Dd0=
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2/go.mod h1:UETIi67q53MR2AWcXfiuqkDkRtnGDLqkBTpCHuJHxtU=
github.com/xordataexchange/crypt v0.0.3-0.20170626215501-b2862e3d0a77/go.mod h1:aYKd//L2LvnjZzWKhF00oedf4jCCReLcmhLdhm1A27Q=
github.com/zcalusic/sysinfo v0.0.0-20200228145645-a159d7cc708b/go.mod h1:WGLNaWsjKQ2gXmAHh+MQztgu3FLFAnOFJjFzhpgShCY=
go.etcd.io/bbolt v1.3.2/go.mod h1:IbVyRI1SCnLcuJnV2u8VeU0CEYM7e686BmAb1XKL+uU=
go.uber.org/atomic v1.4.0/go.mod h1:gD2HeocX3+yG+ygLZcrzQJaqmWj9AIm7n08wl/qW/PE=
go.uber.org/multierr v1.1.0/go.mod h1:wR5kodmAFQ0UK8QlbwjlSNy0Z68gJhDJUG5sjR94q/0=
go.uber.org/zap v1.10.0/go.mod h1:vwi/ZaCAaUcBkycHslxD9B2zi4UTXhF60s6SWpuDF0Q=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
golang.org/x/crypto v0.0.0-20190530122614-20be4c3c3ed5/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20190820162420-60c769a6c586 h1:7KByu05hhLed2MO29w7p1XfZvZ13m8mub3shuVftRs0=
golang.org/x/crypto v0.0.0-20190820162420-60c769a6c586/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37 h1:cg5LA/zNPRzIXIWSCxQW10Rvpy94aQh3LT/ShoCpkHw=
golang.org/x/crypto v0.0.0-20200510223506-06a226fb4e37/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181220203305-927f97764cc3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190419010253-1f3472d942ba/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190522155817-f3200d17e092/go.mod h1:HSz+uSET+XFnRR8LxR5pz3Of3rY3CfYBVs4xY44aLks=
golang.org/x/net v0.0.0-20191003171128-d98b1b443823 h1:Ypyv6BNJh07T1pUSrehkLemqPKXhus2MkfktJ91kRh4=
golang.org/x/net v0.0.0-20191003171128-d98b1b443823/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20200506145744-7e3656a0809f h1:QBjCr1Fz5kw158VqdE9JfI9cJnl/ymnJWAdMuinqL7Y=
golang.org/x/net v0.0.0-20200506145744-7e3656a0809f/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/net v0.0.0-20200513185701-a91f0712d120 h1:EZ3cVSzKOlJxAd8e8YAJ7no8nNypTxexh/YE/xW3ZEY=
golang.org/x/net v0.0.0-20200513185701-a91f0712d120/go.mod h1:qpuaurCH72eLCgpAm/N6yyVIVM9cpaDIP3A8BGJEC5A=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181221193216-37e7f081c4d4/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181107165924-66b7b1311ac8/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190222072716-a9d3bda3a223/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190405154228-4b34438f7a67/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190418153312-f0ce4c0180be/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894 h1:Cz4ceDQGXuKRnVBDTS23GTn/pU5OE2C0WrNTOYK1Uuc=
golang.org/x/sys v0.0.0-20190422165155-953cdadca894/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190530182044-ad28b68e88f1 h1:R4dVlxdmKenVdMRS/tTspEpSTRWINYrHD8ySIU9yCIU=
golang.org/x/sys v0.0.0-20190530182044-ad28b68e88f1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190606203320-7fc4e5ec1444/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9 h1:L2auWcuQIvxz9xSEqzESnV/QN/gNRXNApHi3fYwl2w0=
golang.org/x/sys v0.0.0-20191005200804-aed5e4c7ecf9/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200116001909-b77594299b42/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200223170610-d5e6a3e2c0ae/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200323222414-85ca7c5b95cd/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200513112337-417ce2331b5c h1:kISX68E8gSkNYAFRFiDU8rl5RIn1sJYKYb/r2vMLDrU=
golang.org/x/sys v0.0.0-20200513112337-417ce2331b5c/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/text v0.3.0 h1:g61tztE5qeGQ89tm6NTjjM9VPIm088od1l6aSorWRWg=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.2 h1:tW2bmiBqwgJj/UpqtC8EpXEZVYOwU0yG4iWbprSVAcs=
golang.org/x/text v0.3.2/go.mod h1:bEr9sfX3Q8Zfm5fL9x+3itogRgK3+ptLWKqgva+5dAk=
golang.org/x/time v0.0.0-20190308202827-9d24e82272b4/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
golang.org/x/tools v0.0.0-20180221164845-07fd8470d635/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
google.golang.org/appengine v1.1.0/go.mod h1:EbEs0AVv82hx2wNQdGPgUI5lhzA/G0D9YwlJXL52JkM=
google.golang.org/genproto v0.0.0-20180817151627-c66870c02cf8/go.mod h1:JiN7NxoALGmiZfu7CAH4rXhgtRTLTxftemlI0sWmxmc=
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.21.0/go.mod h1:oYelfM1adQP15Ek0mdvEgi9Df8B9CZIaU1084ijfRaM=
gopkg.in/alecthomas/kingpin.v2 v2.2.6/go.mod h1:FMv+mEhP44yOT+4EoQTLFTRgOQ1FBLkstjWtayDeSgw=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/resty.v1 v1.12.0/go.mod h1:mDo4pnntr5jdWRML875a/NmxYqAlA73dVijT2AXvQQo=
gopkg.in/yaml.v2 v2.0.0-20170812160011-eb3733d160e7/go.mod h1:JAlM8MvJe8wmxCU4Bli9HhUf9+ttbYbLASfIpnQbh74=
gopkg.in/yaml.v2 v2.2.1/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.2.2 h1:ZCJp+EgiOT7lHqUV2J862kp8Qj64Jo6az82+3Td9dZw=
gopkg.in/yaml.v2 v2.2.2/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
gopkg.in/yaml.v2 v2.3.0 h1:clyUAQHOM3G0M3f5vQj7LuJrETvjVot3Z5el9nffUtU=
gopkg.in/yaml.v2 v2.3.0/go.mod h1:hI93XBmqTisBFMUTm0b8Fm+jr3Dg1NNxqwp+5A1VGuI=
honnef.co/go/tools v0.0.0-20190102054323-c2f93a96b099/go.mod h1:rf3lG4BRIbNafJWhAfAdb/ePZxsR/4RtNHQocxwk9r4=
================================================
FILE: hack/comboot_ipxe/Dockerfile
================================================
FROM gcc:latest AS IPXE_BUILD
RUN git clone git://git.ipxe.org/ipxe.git
RUN sed -i '/COMBOOT/s/\/\///g' ipxe/src/config/general.h
WORKDIR /ipxe/src/
RUN make bin/undionly.kpxe
FROM scratch
COPY --from=IPXE_BUILD /ipxe/src/bin/undionly.kpxe .
================================================
FILE: hack/comboot_ipxe/gen_comboot.ipxe
================================================
#!/bin/bash
echo Building latest container for iPXE, with comboot support
cd comboot_ipxe
docker build -t ipxe_comboot .
docker run -it -v $(echo $PWD):/tmp/ipxe ipxe_comboot /bin/sh -c "cp undionly.kpxe /tmp/ipxe"
================================================
FILE: main.go
================================================
package main
import "github.com/plunder-app/plunder/cmd"
// Version is populated from the Makefile and is tied to the release TAG
var Version string
// Build is the last GIT commit
var Build string
func main() {
cmd.Release.Version = Version
cmd.Release.Build = Build
cmd.Execute()
}
================================================
FILE: pkg/apiserver/README.md
================================================
# API Server documentation
This documentation is a quick overview of the CRUD operations that take place within `plunder`, this should be a living document as the various endpoint mature over time.
## Using the API Server
The API Server now starts as default and listens on a different port to HTTP services used for deployment, by default the `plunder` API server will listen on port `60443` however the `-p` `--port` flag can be used to specify a specific port. Currently the API server will bind to all interfaces.
### Starting the API Server
The below example will start the API server on a custom port.
```
plunder server -p 12345
```
## Accessing the API Server
The API Endpoints should be accessed using REST methodologies and JSON payloads, the API Endpoints should **always** be defined in `endpoints.go` (this may change later).
## Current issues
### Server configuration
Currently DHCP can be stopped and started but logging output is buggy, HTTP/TFTP Can be started but can't be stopped or restarted.
================================================
FILE: pkg/apiserver/client.go
================================================
package apiserver
import (
"bytes"
"crypto/tls"
"crypto/x509"
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"net/url"
log "github.com/sirupsen/logrus"
)
//FindFunctionEndpoint - will do a look up to find an exposed dynamic endpoint
func FindFunctionEndpoint(u *url.URL, c *http.Client, f, m string) (*EndPoint, *Response) {
// Create local URL for the API call
newURL := *u
newURL.Path = fmt.Sprintf("%s/%s/%s", FunctionPath(), f, m)
// Interact with the API server to find the endpoint
response, err := ParsePlunderGet(&newURL, c)
if err != nil {
return nil, &Response{
Warning: fmt.Sprintf("Unable to find method [%s] for function [%s]", m, f),
Error: err.Error(),
}
}
var ep EndPoint
err = json.Unmarshal(response.Payload, &ep)
if err != nil {
response.Error = err.Error()
return nil, response
}
return &ep, response
}
//BuildEnvironmentFromConfig will use the apiserver pkg to parse a configuration file and create a http client with the correct authentication and URL
func BuildEnvironmentFromConfig(path, urlFlag string) (*url.URL, *http.Client, error) {
log.Debugf("Parsing Configuration file [%s]", path)
// Open the configuration
c, err := openClientConfig(path)
if err != nil {
return nil, nil, err
}
// Retrieve the certificate
cert, err := c.RetrieveClientCert()
if err != nil {
return nil, nil, err
}
// Build the certificate pool from the unencrypted cert
caCertPool := x509.NewCertPool()
caCertPool.AppendCertsFromPEM(cert)
// Create a HTTPS client and supply the created CA pool
client := &http.Client{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{
RootCAs: caCertPool,
},
},
}
// Build the URL from the configuration
serverURL := c.GetServerAddressURL()
// Overwrite the configuration url if
if urlFlag != "" {
serverURL, err = url.Parse(urlFlag)
if err != nil {
return nil, nil, err
}
}
return serverURL, client, nil
}
//ParsePlunderGet will attempt to retrieve data from the plunder API server
func ParsePlunderGet(u *url.URL, c *http.Client) (*Response, error) {
var response Response
log.Debugf("Querying the Plunder Server [%s]", u.String())
resp, err := c.Get(u.String())
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if resp.StatusCode > 200 {
return nil, fmt.Errorf(resp.Status)
}
err = json.Unmarshal(body, &response)
if err != nil {
return nil, err
}
return &response, nil
}
//ParsePlunderPatch will attempt to retrieve data from the plunder API server
func ParsePlunderPatch(u *url.URL, c *http.Client, data []byte) (*Response, error) {
var response Response
log.Debugf("Posting [%d] bytes to the Plunder Server [%s]", len(data), u.String())
req, err := http.NewRequest("PATCH", u.String(), bytes.NewBuffer(data))
if err != nil {
return nil, err
}
resp, err := c.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if resp.StatusCode > 200 {
return nil, fmt.Errorf(resp.Status)
}
err = json.Unmarshal(body, &response)
if err != nil {
return nil, err
}
return &response, nil
}
//ParsePlunderPost will attempt to retrieve data from the plunder API server
func ParsePlunderPost(u *url.URL, c *http.Client, data []byte) (*Response, error) {
var response Response
log.Debugf("Posting [%d] bytes to the Plunder Server [%s]", len(data), u.String())
resp, err := c.Post(u.String(), "application/json", bytes.NewBuffer(data))
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if resp.StatusCode > 200 {
return nil, fmt.Errorf(resp.Status)
}
err = json.Unmarshal(body, &response)
if err != nil {
return nil, err
}
return &response, nil
}
//ParsePlunderDelete will attempt to retrieve data from the plunder API server
func ParsePlunderDelete(u *url.URL, c *http.Client) (*Response, error) {
var response Response
log.Debugf("Requesting DELETE method to [%s]", u.String())
// Create request
req, err := http.NewRequest("DELETE", u.String(), nil)
if err != nil {
return nil, err
}
req.Header.Set("Content-Type", "application/json")
resp, err := c.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if resp.StatusCode > 200 {
return nil, fmt.Errorf(resp.Status)
}
err = json.Unmarshal(body, &response)
if err != nil {
return nil, err
}
return &response, nil
}
================================================
FILE: pkg/apiserver/config.go
================================================
package apiserver
import (
"encoding/base64"
"encoding/json"
"fmt"
"io/ioutil"
"net/url"
"github.com/ghodss/yaml"
)
// ClientConfig is the structure of an expected configuration for pldctl
type ClientConfig struct {
Address string `json:"address,omitempty"`
Port int `json:"port"`
ClientCert string `json:"cert"`
}
// ServerConfig is the structure of an expected configuration for pldctl
type ServerConfig struct {
ClientConfig
ServerKey string `json:"key"`
}
//openClientConfig will open and parse a Plunder server configuration file
func openClientConfig(path string) (*ClientConfig, error) {
var c ClientConfig
// Create a CA certificate pool and add cert.pem to it
b, err := ioutil.ReadFile(path)
if err != nil {
return nil, err
}
jsonBytes, err := yaml.YAMLToJSON(b)
if err == nil {
// If there were no errors then the YAML => JSON was successful, no attempt to unmarshall
err = json.Unmarshal(jsonBytes, &c)
if err != nil {
return nil, fmt.Errorf("Unable to parse configuration as either yaml or json")
}
} else {
// Couldn't parse the yaml to JSON
// Attempt to parse it as JSON
err = json.Unmarshal(b, &c)
if err != nil {
return nil, fmt.Errorf("Unable to parse configuration as either yaml or json")
}
}
return &c, nil
}
//OpenServerConfig will open and parse a Plunder server configuration file
func OpenServerConfig(path string) (*ServerConfig, error) {
var s ServerConfig
// Create a CA certificate pool and add cert.pem to it
b, err := ioutil.ReadFile(path)
if err != nil {
return nil, err
}
jsonBytes, err := yaml.YAMLToJSON(b)
if err == nil {
// If there were no errors then the YAML => JSON was successful, no attempt to unmarshall
err = json.Unmarshal(jsonBytes, &s)
if err != nil {
return nil, fmt.Errorf("Unable to parse configuration as either yaml or json")
}
} else {
// Couldn't parse the yaml to JSON
// Attempt to parse it as JSON
err = json.Unmarshal(b, &s)
if err != nil {
return nil, fmt.Errorf("Unable to parse configuration as either yaml or json")
}
}
return &s, nil
}
// WriteServerConfig - will write out the server configuration for the API Server
func WriteServerConfig(path, hostname, address string, port int, cert, key []byte) error {
var s ServerConfig
// base64 the certificates
encodedKey := base64.StdEncoding.EncodeToString(key)
encodedCert := base64.StdEncoding.EncodeToString(cert)
// Add the encoded certificates to the struct
s.ClientCert = encodedCert
s.ServerKey = encodedKey
// Add the port for automated startup
s.Port = port
// Marshall to yaml
b, err := yaml.Marshal(s)
if err != nil {
return err
}
err = ioutil.WriteFile(path, b, 0600)
if err != nil {
return err
}
return nil
}
// WriteClientConfig - will write out the server configuration for the API Server
func WriteClientConfig(path, address string, s *ServerConfig) error {
var c ClientConfig
// Add the encoded certificates to the struct
c.ClientCert = s.ClientCert
// Add the host information for automated startup
c.Port = s.Port
c.Address = address
// Marshall client configuration to yaml
b, err := yaml.Marshal(c)
if err != nil {
return err
}
err = ioutil.WriteFile(path, b, 0600)
if err != nil {
return err
}
return nil
}
//GetServerAddressURL will retrieve a parsed URL
func (c *ClientConfig) GetServerAddressURL() *url.URL {
var plunderURL url.URL
plunderURL.Scheme = "https"
// Build a url
plunderURL.Host = fmt.Sprintf("%s:%d", c.Address, +c.Port)
return &plunderURL
}
func retrieveCert(cert string) ([]byte, error) {
return base64.StdEncoding.DecodeString(cert)
}
// RetrieveKey will decode the base64 certificate
func (s *ServerConfig) RetrieveKey() ([]byte, error) {
return retrieveCert(s.ServerKey)
}
// RetrieveClientCert will decode the base64 certificate
func (s *ServerConfig) RetrieveClientCert() ([]byte, error) {
return retrieveCert(s.ClientCert)
}
// RetrieveClientCert will decode the base64 certificate
func (c *ClientConfig) RetrieveClientCert() ([]byte, error) {
return retrieveCert(c.ClientCert)
}
================================================
FILE: pkg/apiserver/endpoints.go
================================================
package apiserver
import (
"net/http"
log "github.com/sirupsen/logrus"
//"github.com/gorilla/mux"
)
// EndPointManager - Contains all of the dynamically created endpoints
var EndPointManager []EndPoint
// EndPoint is the source of truth for handling all of the endpoints exposed through the API Server
// it also provides a mechanism to interact with the apiserver to find/create api endpoints
type EndPoint struct {
Name string `json:"name"`
Path string `json:"path"`
FunctionPath string `json:"functionEndpoint"`
Description string `json:"description"`
Method string `json:"method"`
}
// AddDynamicEndpoint - will add an endpoint to the api server and link it back to a function
func AddDynamicEndpoint(endpointPattern, path, description, name, method string, epFunc http.HandlerFunc) {
for i := range EndPointManager {
if EndPointManager[i].Name == name && EndPointManager[i].Method == method {
log.Warnf("Endpoint [%s] already exists with method [%s]", name, method)
}
}
// First we add the endpoint to the Manager so we can query it
EndPointManager = append(EndPointManager, EndPoint{
FunctionPath: endpointPattern,
Path: path,
Description: description,
Method: method,
Name: name,
})
// Then we add the endpoint to the apiServer
endpoints.HandleFunc(endpointPattern, epFunc).Methods(method)
}
// GetEndpoint - will return the details for an endpoint
func GetEndpoint(name, method string) *EndPoint {
for i := range EndPointManager {
if EndPointManager[i].Name == name && EndPointManager[i].Method == method {
return &EndPointManager[i]
}
}
return nil
}
// FunctionPath - this will return the api server path for any external caller using the package
func FunctionPath() string {
return "/api"
}
================================================
FILE: pkg/apiserver/go.mod
================================================
module github.com/plunder-app/plunder/pkg/apiserver
go 1.12
================================================
FILE: pkg/apiserver/handlerApiserver.go
================================================
package apiserver
import (
"encoding/json"
"fmt"
"net/http"
"github.com/gorilla/mux"
)
// Delete the parlay results from the plunder server
func getAPIFunctionMethod(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
var rsp Response
// Find the deployment ID
f := mux.Vars(r)["function"]
m := mux.Vars(r)["method"]
ep := GetEndpoint(f, m)
if ep == nil {
// RETREIVE the deployment Logs (TODO)
rsp.Warning = fmt.Sprintf("Unable to find HTTP method [%s] for function [%s]", m, f)
rsp.Error = "Error looking up in API Server"
} else {
jsonData, err := json.Marshal(ep)
if err != nil {
w.Header().Set("Content-Type", "application/json")
rsp.Warning = "Error retrieving deployment Configuration"
rsp.Error = err.Error()
} else {
rsp.Payload = jsonData
}
}
json.NewEncoder(w).Encode(rsp)
}
// Delete the parlay results from the plunder server
func getAPIs(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
var rsp Response
jsonData, err := json.Marshal(EndPointManager)
if err != nil {
w.Header().Set("Content-Type", "application/json")
rsp.Warning = "Error retrieving deployment Configuration"
rsp.Error = err.Error()
} else {
rsp.Payload = jsonData
}
json.NewEncoder(w).Encode(rsp)
}
================================================
FILE: pkg/apiserver/logging.go
================================================
package apiserver
import (
"fmt"
"net/http"
"sync"
"github.com/gorilla/mux"
)
// MVP of a streaming logging provider
// The notificationCenter is in charge of handling the various notification managers, whihc
// in turn will notify all of their subscribers
var notificationCenter map[string]*notificationManager
// Notification is what will be sent to subscribers of a manager
type Notification struct {
ID string
RawData []byte
}
// RegisterNotificationManager will create a manager and an endpoint
func RegisterNotificationManager(managerName, endpoint string) error {
// Register the new Manager to the Notification Center
notificationCenter[managerName] = newNotificationManager()
AddDynamicEndpoint(endpoint,
endpoint,
fmt.Sprintf("Automatically generated notification endpoint for [%s]", managerName),
managerName,
http.MethodGet,
handleSubscribers(notificationCenter[managerName]))
return nil
}
// NotifyManager - This will Notify a Manager that there is a new notification that needs to go to subscribers
func NotifyManager(managerName string, n Notification) error {
manager := notificationCenter[managerName]
if manager == nil {
return fmt.Errorf("Notification Manager [%s], hasn't been registered", managerName)
}
manager.notifySubscribers(n)
return nil
}
// -------------- Notication MAGIC below --------------
func init() {
// Initialise the notificationCenter map
notificationCenter = make(map[string]*notificationManager)
}
type unsubscribeFunc func() error
type subscriber interface {
subscribe(n chan Notification) (unsubscribeFunc, error)
}
func handleSubscribers(s subscriber) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
// Subscribe
n := make(chan Notification)
unsubscribeFn, err := s.subscribe(n)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
// Set environment for streaming events
w.Header().Set("Content-Type", "text/event-stream")
w.Header().Set("Cache-Control", "no-cache")
w.Header().Set("Connection", "keep-alive")
w.Header().Set("Access-Control-Allow-Origin", "*")
Looping:
for {
select {
case <-r.Context().Done():
if err := unsubscribeFn(); err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
break Looping
default:
// Find the log ID
id := mux.Vars(r)["id"]
// retrieve the notification
newNotification := <-n
// compare the notification ID with that of the URL, optionally retrieve "all" notifications
if newNotification.ID == id || id == "all" {
// if the correct id then send them the data
fmt.Fprintf(w, "%s\n", newNotification.RawData)
}
w.(http.Flusher).Flush()
}
}
}
}
type notifier interface {
Notify(n Notification) error
}
type notificationManager struct {
subscribers map[chan Notification]struct{}
subscribersMu *sync.Mutex
}
func newNotificationManager() *notificationManager {
return ¬ificationManager{
subscribers: map[chan Notification]struct{}{},
subscribersMu: &sync.Mutex{},
}
}
func (nc *notificationManager) subscribe(n chan Notification) (unsubscribeFunc, error) {
nc.subscribersMu.Lock()
nc.subscribers[n] = struct{}{}
nc.subscribersMu.Unlock()
unsubscribeFn := func() error {
nc.subscribersMu.Lock()
delete(nc.subscribers, n)
nc.subscribersMu.Unlock()
return nil
}
return unsubscribeFn, nil
}
func (nc *notificationManager) notifySubscribers(n Notification) error {
// Lock them until updates are complete
nc.subscribersMu.Lock()
defer nc.subscribersMu.Unlock()
for c := range nc.subscribers {
select {
case c <- n:
default:
}
}
return nil
}
================================================
FILE: pkg/apiserver/loggingHandlers.go
================================================
package apiserver
//var map parlay[]
================================================
FILE: pkg/apiserver/server.go
================================================
package apiserver
import (
"crypto/tls"
"fmt"
"net/http"
"github.com/gorilla/mux"
log "github.com/sirupsen/logrus"
)
var endpoints *mux.Router
func init() {
// Initialise a new HTTP Router so that connections can be created before the API server starts, any registered will be added to this router and
// evaluated once the API Server starts
endpoints = mux.NewRouter()
}
//StartAPIServer - will parse a configuration file and passed variables and start the API Server
func StartAPIServer(path string, port int, insecure bool) error {
// Open and Parse the server configuration
conf, err := OpenServerConfig(path)
if err != nil {
log.Warnln(err)
if insecure == false {
log.Warningln("Secure server enabled, but no certificates have been loaded [no communication to API server is possible]")
}
// Create a blank server config as one wont be returned by the above OpenFile
conf = &ServerConfig{}
}
if port != 0 {
conf.Port = port
}
log.Infof("Starting API server on port %d", conf.Port)
address := fmt.Sprintf(":%d", conf.Port)
// Add the apiserver end point
AddDynamicEndpoint("/api",
"/api",
"Endpoint for interacting with the api server",
"apis",
http.MethodGet,
getAPIs)
// Add the apiserver end point
AddDynamicEndpoint("/api/{function}/{method}",
"/api",
"Endpoint for interacting with the api server",
"apiFunctions",
http.MethodGet,
getAPIFunctionMethod)
// Begin the start of a secure endpoint (TODO)
if insecure == false {
cert, err := conf.RetrieveClientCert()
if err != nil {
return err
}
key, err := conf.RetrieveKey()
if err != nil {
return err
}
certPair, err := tls.X509KeyPair(cert, key)
cfg := &tls.Config{Certificates: []tls.Certificate{certPair}}
srv := &http.Server{
TLSConfig: cfg,
Addr: address,
Handler: endpoints,
// TODO - exposing no timeout will lead to exhausted file descriptors
// ReadTimeout: time.Minute,
// WriteTimeout: time.Minute,
}
return srv.ListenAndServeTLS("", "")
}
// Start an insecure http server (TODO - warning)
return http.ListenAndServe(address, endpoints)
}
================================================
FILE: pkg/apiserver/types.go
================================================
package apiserver
import "encoding/json"
//Response - This is the wrapper for responses back to a client, if any errors are created then the payload isn't guarenteed
type Response struct {
Warning string `json:"warning,omitempty"` // when it maybe worked
Error string `json:"error,omitempty"` // when it goes wrong
Success string `json:"success,omitempty"` // when it goes correct
Payload json.RawMessage `json:"payload,omitempty"`
}
================================================
FILE: pkg/certs/certs.go
================================================
package certs
// generate-tls-cert generates root, leaf, and client TLS certificates.
import (
"bytes"
"crypto/rand"
"crypto/rsa"
"crypto/x509"
"crypto/x509/pkix"
"encoding/pem"
"io/ioutil"
"math/big"
"os"
"time"
"github.com/plunder-app/plunder/pkg/utils"
)
// Internal variables to hold the outputs
var keyData, pemData []byte
// GenerateKeyPair - (TODO)
func GenerateKeyPair(hosts []string, start time.Time, length time.Duration) error {
ca := &x509.Certificate{
SerialNumber: big.NewInt(2019),
Subject: pkix.Name{
Organization: []string{"Plunder"},
Country: []string{"UK"},
Province: []string{""},
Locality: []string{"Yorkshire"},
StreetAddress: []string{""},
PostalCode: []string{""},
},
NotBefore: time.Now(),
NotAfter: time.Now().AddDate(10, 0, 0),
IsCA: true,
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth, x509.ExtKeyUsageServerAuth},
KeyUsage: x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,
BasicConstraintsValid: true,
}
// create our private and public key
caPrivKey, err := rsa.GenerateKey(rand.Reader, 4096)
if err != nil {
return err
}
// create the CA
caBytes, err := x509.CreateCertificate(rand.Reader, ca, ca, &caPrivKey.PublicKey, caPrivKey)
if err != nil {
return err
}
// pem encode
caPEM := new(bytes.Buffer)
pem.Encode(caPEM, &pem.Block{
Type: "CERTIFICATE",
Bytes: caBytes,
})
caPrivKeyPEM := new(bytes.Buffer)
pem.Encode(caPrivKeyPEM, &pem.Block{
Type: "RSA PRIVATE KEY",
Bytes: x509.MarshalPKCS1PrivateKey(caPrivKey),
})
// Find all IP addresses on a server
serverAddresses, err := utils.FindAllIPAddresses()
if err != nil {
return err
}
// Find the hostname of the server
serverName, err := os.Hostname()
if err != nil {
return err
}
// set up our server certificate
cert := &x509.Certificate{
SerialNumber: big.NewInt(2019),
Subject: pkix.Name{
Organization: []string{"Plunder"},
Country: []string{"UK"},
Province: []string{""},
Locality: []string{"Yorkshire"},
StreetAddress: []string{""},
PostalCode: []string{""},
},
IPAddresses: serverAddresses,
NotBefore: time.Now(),
NotAfter: time.Now().AddDate(10, 0, 0),
SubjectKeyId: []byte{1, 2, 3, 4, 6},
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageClientAuth, x509.ExtKeyUsageServerAuth},
KeyUsage: x509.KeyUsageDigitalSignature,
DNSNames: []string{serverName},
}
certPrivKey, err := rsa.GenerateKey(rand.Reader, 4096)
if err != nil {
return err
}
certBytes, err := x509.CreateCertificate(rand.Reader, cert, ca, &certPrivKey.PublicKey, caPrivKey)
if err != nil {
return err
}
certPEM := new(bytes.Buffer)
pem.Encode(certPEM, &pem.Block{
Type: "CERTIFICATE",
Bytes: certBytes,
})
pemData = certPEM.Bytes()
certPrivKeyPEM := new(bytes.Buffer)
pem.Encode(certPrivKeyPEM, &pem.Block{
Type: "RSA PRIVATE KEY",
Bytes: x509.MarshalPKCS1PrivateKey(certPrivKey),
})
keyData = certPrivKeyPEM.Bytes()
return nil
}
// WriteKeyToFile - will write a generated Key to a file path
func WriteKeyToFile(path string) error {
err := ioutil.WriteFile(path, keyData, 0600)
if err != nil {
return err
}
return nil
}
// WritePemToFile - will write a generated pem to a file path
func WritePemToFile(path string) error {
err := ioutil.WriteFile(path, pemData, 0600)
if err != nil {
return err
}
return nil
}
// GetKey - will return the []byte of the key
func GetKey() []byte {
return keyData
}
// GetPem - will return the []byte of the key
func GetPem() []byte {
return pemData
}
================================================
FILE: pkg/certs/go.mod
================================================
module github.com/plunder-app/plunder/pkg/certs
go 1.12
================================================
FILE: pkg/go.mod
================================================
module github.com/plunder-app/plunder/pkg
go 1.12
================================================
FILE: pkg/parlay/go.mod
================================================
module github.com/plunder-app/plunder/pkg/parlay
go 1.12
================================================
FILE: pkg/parlay/handler.go
================================================
package parlay
import (
"encoding/json"
"io/ioutil"
"net/http"
"strings"
"github.com/gorilla/mux"
"github.com/plunder-app/plunder/pkg/apiserver"
"github.com/plunder-app/plunder/pkg/parlay/parlaytypes"
"github.com/plunder-app/plunder/pkg/services"
"github.com/plunder-app/plunder/pkg/ssh"
log "github.com/sirupsen/logrus"
)
var registered bool
// RegisterToAPIServer - will add the endpoints to the API server
func RegisterToAPIServer() {
// Ensure registration only happens once
if registered == true {
return
}
// ------------------------------------------------
// Parlay API registration
// ------------------------------------------------
apiserver.AddDynamicEndpoint("/parlay",
"/parlay",
"Create a parlay automation deployment",
"parlay",
http.MethodPost,
postParlay)
apiserver.AddDynamicEndpoint("/parlay/logs/{id}",
"/parlay/logs",
"Retrieve the logs from a parlay deployment",
"parlayLog",
http.MethodGet,
getParlay)
apiserver.AddDynamicEndpoint("/parlay/logs/{id}",
"/parlay/logs",
"Delete the cached logs from a specific parlay deployment",
"parlayLog",
http.MethodDelete,
delParlay)
registered = true
}
// Retrieve a specific plunder deployment configuration
func postParlay(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
var rsp apiserver.Response
if b, err := ioutil.ReadAll(r.Body); err == nil {
// Parse the treasure map in the POST data
var m parlaytypes.TreasureMap
err := json.Unmarshal(b, &m)
// Unable to parse the JSON payload
if err != nil {
rsp.Warning = "Error parsing the parlay actions"
rsp.Error = err.Error()
} else {
// Parsed succesfully, we will deploy this in a go routine and use GET /parlay/MAC to view progress
//
err = ssh.ImportHostsFromDeployment(services.Deployments)
if err != nil {
rsp.Warning = "Error importing the hosts from deployment"
rsp.Error = err.Error()
} else {
err = DeploySSH(&m, "", true, true)
if err != nil {
rsp.Warning = "Error performing the parlay actions"
rsp.Error = err.Error()
log.Errorf("%s", err.Error())
}
}
}
} else {
rsp.Warning = "Error reading HTTP data"
rsp.Error = err.Error()
}
json.NewEncoder(w).Encode(rsp)
}
// Retrieve a specific parlay automation
func getParlay(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
var rsp apiserver.Response
// Find the deployment ID
id := mux.Vars(r)["id"]
// We need to revert the mac address back to the correct format (dashes back to colons)
target := strings.Replace(id, "-", ".", -1)
// Use the mac address to lookup the deployment
logs, err := GetTargetLogs(target)
// If the deployment exists then process the POST data
if err != nil {
// RETREIVE the deployment Logs (TODO)
rsp.Warning = "Error reading Parlay Logs"
rsp.Error = err.Error()
} else {
jsonData, err := json.Marshal(logs)
if err != nil {
// RETREIVE the deployment Logs (TODO)
rsp.Warning = "Error parsing Parlay Logs"
rsp.Error = err.Error()
} else {
rsp.Payload = jsonData
}
}
json.NewEncoder(w).Encode(rsp)
}
// Delete the parlay results from the plunder server
func delParlay(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
var rsp apiserver.Response
// Find the deployment ID
id := mux.Vars(r)["id"]
// We need to revert the mac address back to the correct format (dashes back to colons)
target := strings.Replace(id, "-", ".", -1)
// Use the mac address to lookup the deployment
err := DeleteTargetLogs(target)
// If the deployment exists then process the POST data
if err != nil {
// RETREIVE the deployment Logs (TODO)
rsp.Warning = "Error reading deleting logs"
rsp.Error = err.Error()
}
json.NewEncoder(w).Encode(rsp)
}
================================================
FILE: pkg/parlay/parlay.go
================================================
package parlay
type actionType string
const (
//upload - defines that this action will upload a file to a remote system
upload actionType = "upload" //
download actionType = "download"
command actionType = "command"
pkg actionType = "package"
)
// KeyMap
// Keys are used to store information between sessions and deployments
var Keys map[string]string
func init() {
// Initialise the map
Keys = make(map[string]string)
}
================================================
FILE: pkg/parlay/parlay_ui.go
================================================
package parlay
import (
"fmt"
"strings"
"github.com/AlecAivazis/survey/v2"
"github.com/plunder-app/plunder/pkg/parlay/parlaytypes"
)
func contains(v string, a []string) bool {
for _, i := range a {
if strings.Contains(v, i) {
return true
}
}
return false
}
// StartUI will enable parlay to provide an easier way of selecting which operations will be performed
func StartUI(m *parlaytypes.TreasureMap) (*parlaytypes.TreasureMap, error) {
deployments := []string{}
for i := range m.Deployments {
deployments = append(deployments, m.Deployments[i].Name)
}
if len(deployments) == 0 {
return nil, fmt.Errorf("No Deployments were found")
}
var multiQs = []*survey.Question{
{
Name: "letter",
Prompt: &survey.MultiSelect{
Message: "Select deployment(s)",
Options: deployments,
},
},
}
deploymentAnswers := []string{}
// ask the question
err := survey.Ask(multiQs, &deploymentAnswers)
if err != nil {
return nil, err
}
// Create a new TreasureMap from the answered questions
newMap, err := m.FindDeployments(deploymentAnswers)
if err != nil {
return nil, err
}
for i := range newMap.Deployments {
// Ask for Hosts
multiQs[0].Prompt = &survey.MultiSelect{
Message: fmt.Sprintf("Select Hosts(s) for [%s]", newMap.Deployments[i].Name),
Options: newMap.Deployments[i].Hosts,
}
hostAnswers := []string{}
err := survey.Ask(multiQs, &hostAnswers)
if err != nil {
return nil, err
}
// Ask for Actions
actions := []string{}
for y := range newMap.Deployments[i].Actions {
actions = append(actions, m.Deployments[i].Actions[y].Name)
}
if len(actions) == 0 {
return nil, fmt.Errorf("No Deployments were found")
}
multiQs[0].Prompt = &survey.MultiSelect{
Message: fmt.Sprintf("Select Actions(s) for [%s]", newMap.Deployments[i].Name),
Options: actions,
}
deploymentAnswers := []string{}
err = survey.Ask(multiQs, &deploymentAnswers)
if err != nil {
return nil, err
}
newMap.Deployments[i].Hosts = hostAnswers
foundActions, err := newMap.Deployments[i].FindActions(deploymentAnswers)
if err != nil {
return nil, err
}
newMap.Deployments[i].Actions = foundActions
}
return newMap, nil
}
================================================
FILE: pkg/parlay/parlaytypes/finder.go
================================================
package parlaytypes
import (
"fmt"
log "github.com/sirupsen/logrus"
)
// FindDeployments - This will iterate through a deployment map and build a new deployment map from found deployments
func (m *TreasureMap) FindDeployments(deployment []string) (*TreasureMap, error) {
var newDeploymentList []Deployment
for x := range deployment {
for y := range m.Deployments {
if m.Deployments[y].Name == deployment[x] {
newDeploymentList = append(newDeploymentList, m.Deployments[y])
}
}
}
// If this is zero it means that no deployments have been found
if len(m.Deployments) == 0 {
return nil, fmt.Errorf("No Deployment(s) have been found")
}
m.Deployments = newDeploymentList
return m, nil
}
// FindHosts - will iterate through the deployment hosts and compare to the array of hosts to return
func (d *Deployment) FindHosts(hosts []string) (*Deployment, error) {
var newHostList []string
for x := range hosts {
for y := range d.Hosts {
if d.Hosts[y] == hosts[x] {
newHostList = append(newHostList, d.Hosts[y])
}
}
}
// If this is zero it means that no hosts have been found
if len(d.Hosts) == 0 {
return nil, fmt.Errorf("No Host(s) have been found")
}
d.Hosts = newHostList
return d, nil
}
// FindActions - will iterate through the deployment actions and compare to the array of actions to return
func (d *Deployment) FindActions(actions []string) ([]Action, error) {
var newActionList []Action
for x := range actions {
for y := range d.Actions {
if d.Actions[y].Name == actions[x] {
newActionList = append(newActionList, d.Actions[y])
}
}
}
// If this is zero it means that no hosts have been found
if len(d.Actions) == 0 {
return nil, fmt.Errorf("No Action(s) have been found")
}
return newActionList, nil
}
//FindDeployment - takes a number of flags and builds a new map to be processed
func (m *TreasureMap) FindDeployment(deployment, action, host, logFile string, resume bool) (*TreasureMap, error) {
var foundMap TreasureMap
if deployment != "" {
log.Debugf("Looking for deployment [%s]", deployment)
for x := range m.Deployments {
if m.Deployments[x].Name == deployment {
foundMap.Deployments = append(foundMap.Deployments, m.Deployments[x])
// Find a specific action and add or resume from
if action != "" {
// Clear the slice as we will be possibly adding different actions
foundMap.Deployments[0].Actions = nil
for y := range m.Deployments[x].Actions {
if m.Deployments[x].Actions[y].Name == action {
// If we're not resuming that just add the action that we want to happen
if resume != true {
foundMap.Deployments[0].Actions = append(foundMap.Deployments[0].Actions, m.Deployments[x].Actions[y])
} else {
// Alternatively add all actions from this point
foundMap.Deployments[0].Actions = m.Deployments[x].Actions[y:]
}
}
}
// If this is zero it means that no actions have been found
if len(foundMap.Deployments[0].Actions) == 0 {
return nil, fmt.Errorf("No actions have been found, looking for action [%s]", action)
}
}
// If a host is specified act soley on it
if host != "" {
// Clear the slice as we will be possibly adding different actions
foundMap.Deployments[0].Hosts = nil
for y := range m.Deployments[x].Hosts {
if m.Deployments[x].Hosts[y] == host {
foundMap.Deployments[0].Hosts = append(foundMap.Deployments[0].Hosts, m.Deployments[x].Hosts[y])
}
}
// If this is zero it means that no hosts have been found
if len(foundMap.Deployments[0].Hosts) == 0 {
return nil, fmt.Errorf("No host has been found, looking for host [%s]", host)
}
}
}
}
// If this is zero it means that no actions have been found
if len(foundMap.Deployments) == 0 {
return nil, fmt.Errorf("No deployment has been found, looking for deployment [%s]", deployment)
}
} else {
return nil, fmt.Errorf("No deployment was specified")
}
return &foundMap, nil
//return parlay.DeploySSH(foundMap, logFile, false, false)
}
================================================
FILE: pkg/parlay/parlaytypes/go.mod
================================================
module github.com/plunder-app/plunder/pkg/parlay/parlaytypes
go 1.12
================================================
FILE: pkg/parlay/parlaytypes/parlaytypes.go
================================================
package parlaytypes
import (
"encoding/json"
)
// TreasureMap - X Marks the spot
// The treasure maps define the automation that will take place on the hosts defined
type TreasureMap struct {
// An array/list of deployments that will take places as part of this "map"
Deployments []Deployment `json:"deployments"`
}
// Deployment defines the hosts and the action(s) that should be performed on them
type Deployment struct {
// Name of the deployment that is taking place i.e. (Install MySQL)
Name string `json:"name"`
// An array/list of hosts that these actions should be performed upon
Hosts []string `json:"hosts"`
// Parallel allow multiple actions across multiple hosts in parallel
Parallel bool `json:"parallel"`
ParallelSessions int `json:"parallelSessions"`
// The actions that should be performed
Actions []Action `json:"actions"`
}
// Action defines what the instructions that will be executed
type Action struct {
Name string `json:"name"`
ActionType string `json:"type"`
Timeout int `json:"timeout"`
// File based operations
Source string `json:"source,omitempty"`
Destination string `json:"destination,omitempty"`
FileMove bool `json:"fileMove,omitempty"`
// Package manager operations
PkgManager string `json:"packageManager,omitempty"`
PkgOperation string `json:"packageOperation,omitempty"`
Packages string `json:"packages,omitempty"`
// Command operations
Command string `json:"command,omitempty"`
Commands []string `json:"commands,omitempty"`
CommandLocal bool `json:"commandLocal,omitempty"`
CommandSaveFile string `json:"commandSaveFile,omitempty"`
CommandSaveAsKey string `json:"commandSaveAsKey,omitempty"`
CommandSudo string `json:"commandSudo,omitempty"`
// Piping commands, read in a file and send over stdin, or capture stdout from a local command
CommandPipeFile string `json:"commandPipeFile,omitempty"`
CommandPipeCmd string `json:"commandPipeCmd,omitempty"`
// Ignore any failures
IgnoreFailure bool `json:"ignoreFail,omitempty"`
// Key operations
KeyFile string `json:"keyFile,omitempty"`
KeyName string `json:"keyName,omitempty"`
//Plugin Spec
Plugin json.RawMessage `json:"plugin,omitempty"`
}
================================================
FILE: pkg/parlay/parser.go
================================================
package parlay
import (
"fmt"
"github.com/plunder-app/plunder/pkg/plunderlogging"
log "github.com/sirupsen/logrus"
"github.com/plunder-app/plunder/pkg/parlay/parlaytypes"
parlayplugin "github.com/plunder-app/plunder/pkg/parlay/plugin"
"github.com/plunder-app/plunder/pkg/ssh"
)
// This logger will manage all of the logging for Parlay
var logger plunderlogging.Logger
// GetTargetLogs will retrieve the JSON logs
func GetTargetLogs(target string) (*plunderlogging.JSONLog, error) {
return logger.GetJSONLogs(target)
}
// DeleteTargetLogs will retrieve the JSON logs
func DeleteTargetLogs(target string) error {
return logger.DeleteLogs(target)
}
// DeploySSH - will iterate through a deployment and perform the relevant actions
func DeploySSH(m *parlaytypes.TreasureMap, logFile string, jsonLogging, background bool) error {
if len(ssh.Hosts) == 0 {
log.Warnln("No hosts credentials have been loaded, only commands with commandLocal = true will work")
}
if len(m.Deployments) == 0 {
return fmt.Errorf("No Deployments in parlay map")
}
for x := range m.Deployments {
// Build new hosts list from imported SSH servers and compare that we have required credentials
_, err := ssh.FindHosts(m.Deployments[x].Hosts)
if err != nil {
return err
}
}
// Begin the deployment
if logFile != "" {
//enable logging
logger.InitLogFile(logFile)
}
if jsonLogging {
logger.InitJSON()
}
if background {
go startDeployments(m.Deployments)
} else {
startDeployments(m.Deployments)
}
// TODO - test original automation
// for x := range m.Deployments {
// // Build new hosts list from imported SSH servers and compare that we have required credentials
// hosts, err := ssh.FindHosts(m.Deployments[x].Hosts)
// if err != nil {
// return err
// }
// // Beggining of deployment work
// log.Infof("Beginning Deployment [%s]\n", m.Deployments[x].Name)
// logger.WriteLogEntry("", "", "", fmt.Sprintf("Beginning Deployment [%s]\n", m.Deployments[x].Name))
// // Set Restore checkpoint
// restore.Deployment = m.Deployments[x].Name
// restore.Hosts = m.Deployments[x].Hosts
// if m.Deployments[x].Parallel == true {
// // Begin this deployment in parallel across all hosts
// err = parallelDeployment(m.Deployments[x].Actions, hosts, &logger)
// if err != nil {
// return err
// }
// } else {
// // This work will be sequential, one host after the next
// for z := range m.Deployments[x].Hosts {
// var hostConfig ssh.HostSSHConfig
// // Find the hosts SSH configuration
// for i := range hosts {
// if hosts[i].Host == m.Deployments[x].Hosts[z] {
// hostConfig = hosts[i]
// }
// }
// // Set the state of logging actions to in-progress
// logger.SetLoggingState(hostConfig.Host, "Running")
// err = sequentialDeployment(m.Deployments[x].Actions, hostConfig, &logger)
// if err != nil {
// logger.SetLoggingState(hostConfig.Host, "Failed")
// return err
// }
// // Set the state of logging actions to completed
// logger.SetLoggingState(hostConfig.Host, "Completed")
// }
// }
// }
return nil
}
func startDeployments(d []parlaytypes.Deployment) error {
for x := range d {
// Build new hosts list from imported SSH servers and compare that we have required credentials
hosts, err := ssh.FindHosts(d[x].Hosts)
if err != nil {
return err
}
// Beggining of deployment work
log.Infof("Beginning Deployment [%s]\n", d[x].Name)
logger.WriteLogEntry("", "", "", fmt.Sprintf("Beginning Deployment [%s]\n", d[x].Name))
// Set Restore checkpoint
restore.Deployment = d[x].Name
restore.Hosts = d[x].Hosts
if d[x].Parallel == true {
// Begin this deployment in parallel across all hosts
err = parallelDeployment(d[x].Actions, hosts, &logger)
if err != nil {
return err
}
} else {
// This work will be sequential, one host after the next
for z := range d[x].Hosts {
var hostConfig ssh.HostSSHConfig
// Find the hosts SSH configuration
for i := range hosts {
if hosts[i].Host == d[x].Hosts[z] {
hostConfig = hosts[i]
}
}
// Set the state of logging actions to in-progress
logger.SetLoggingState(hostConfig.Host, "Running")
err = sequentialDeployment(d[x].Actions, hostConfig, &logger)
if err != nil {
logger.SetLoggingState(hostConfig.Host, "Failed")
return err
}
// Set the state of logging actions to completed
logger.SetLoggingState(hostConfig.Host, "Completed")
}
}
}
return nil
}
// Begin host by host deployments as part of each deployment
func sequentialDeployment(action []parlaytypes.Action, hostConfig ssh.HostSSHConfig, logger *plunderlogging.Logger) error {
var err error
for y := range action {
switch action[y].ActionType {
case "upload":
err = hostConfig.UploadFile(action[y].Source, action[y].Destination)
if err != nil {
// Set checkpoint
restore.Action = action[y].Name
restore.Host = hostConfig.Host
restore.createCheckpoint()
logger.WriteLogEntry(hostConfig.Host, action[y].Name, "", err.Error())
// Return the error
return fmt.Errorf("Upload task [%s] on host [%s] failed with error [%s]", action[y].Name, hostConfig.Host, err)
}
log.Infof("Upload Task [%s] on node [%s] completed successfully", action[y].Name, hostConfig.Host)
case "download":
err = hostConfig.DownloadFile(action[y].Source, action[y].Destination)
if err != nil {
// Set checkpoint
restore.Action = action[y].Name
restore.Host = hostConfig.Host
restore.createCheckpoint()
logger.WriteLogEntry(hostConfig.Host, action[y].Name, "", err.Error())
// Return the error
return fmt.Errorf("Download task [%s] on host [%s] failed with error [%s]", action[y].Name, hostConfig.Host, err)
}
log.Infof("Succesfully Downloaded [%s] to [%s] from [%s]", action[y].Source, action[y].Destination, hostConfig.Host)
case "command":
// Build out a configuration based upon the action
cr := parseAndExecute(action[y], &hostConfig)
// This will end command execution and print the error
if cr.Error != nil && action[y].IgnoreFailure == false {
// Set checkpoint
restore.Action = action[y].Name
restore.Host = hostConfig.Host
restore.createCheckpoint()
// Output error messages
logger.WriteLogEntry(hostConfig.Host, action[y].Name, cr.Result, cr.Error.Error())
// cr.Result is ommited here TODO
return fmt.Errorf("Command task [%s] on host [%s] failed with error [%s]", action[y].Name, hostConfig.Host, cr.Error)
}
// if there is an error and we're set to ignore it then process accordingly
if cr.Error != nil && action[y].IgnoreFailure == true {
log.Warnf("Command Task [%s] on node [%s] failed (execution will continute)", action[y].Name, hostConfig.Host)
log.Debugf("Command Results ->\n%s", cr.Result)
logger.WriteLogEntry(hostConfig.Host, action[y].Name, cr.Result, cr.Error.Error())
//logger.WriteLogEntry(hostConfig.Host, fmt.Sprintf("Command task [%s] on host [%s] has failed (execution will continute)\n", action[y].Name, hostConfig.Host))
}
// No error, task was completed correctly
if cr.Error == nil {
// Output success Messages
log.Infof("Command Task [%s] on node [%s] completed successfully", action[y].Name, hostConfig.Host)
log.Debugf("Command Results ->\n%s", cr.Result)
//logger.WriteLogEntry(hostConfig.Host, fmt.Sprintf("Command task [%s] on host [%s] has completed succesfully\n", action[y].Name, hostConfig.Host))
//logger.WriteLogEntry(hostConfig.Host, fmt.Sprintf("Command task [%s] Output [%s]\n", action[y].Name, cr.Result))
logger.WriteLogEntry(hostConfig.Host, action[y].Name, cr.Result, "")
}
case "pkg":
case "key":
default:
// Set checkpoint (the actiontype may be modified or spelling issue)
restore.Action = action[y].Name
restore.Host = hostConfig.Host
restore.createCheckpoint()
pluginActions, err := parlayplugin.ExecuteAction(action[y].ActionType, hostConfig.Host, action[y].Plugin)
if err != nil {
logger.WriteLogEntry(hostConfig.Host, action[y].Name, "", err.Error())
return err
}
log.Debugf("About to execute [%d] actions", len(pluginActions))
err = sequentialDeployment(pluginActions, hostConfig, logger)
if err != nil {
return err
}
}
}
return nil
}
// Peform all of the actions in parallel on all hosts in the host array
// this function will make use of the parallel ssh calls
func parallelDeployment(action []parlaytypes.Action, hosts []ssh.HostSSHConfig, logger *plunderlogging.Logger) error {
for y := range action {
switch action[y].ActionType {
case "upload":
//TODO - Remove or repurpose GENERAL output
logger.WriteLogEntry("upload", fmt.Sprintf("Uploading file [%s] to Destination [%s] to multiple hosts\n", action[y].Source, action[y].Destination), "", "")
results := ssh.ParalellUpload(hosts, action[y].Source, action[y].Destination, action[y].Timeout)
// TODO - Unlikely that this should happen
if len(results) == 0 {
return fmt.Errorf("No results have been returned from the parallel execution")
}
// Parse the results from the parallel updates
for i := range results {
if results[i].Error != nil {
// Set checkpoint
restore.Action = action[y].Name
restore.createCheckpoint()
logger.WriteLogEntry(results[i].Host, action[y].Name, "", results[i].Error.Error())
logger.SetLoggingState(results[i].Host, "Failed")
//logger.WriteLogEntry("", fmt.Sprintf("[%s] Error uploading file [%s] to Destination [%s] to host [%s]\n", time.Now().Format(time.ANSIC), action[y].Source, action[y].Destination, results[i].Host))
//logger.WriteLogEntry("", fmt.Sprintf("[%s] [%s]\n", time.Now().Format(time.ANSIC), results[i].Error()))
return fmt.Errorf("Upload task [%s] on host [%s] failed with error [%s]", action[y].Name, results[i].Host, results[i].Error)
}
logger.WriteLogEntry(results[i].Host, action[y].Name, fmt.Sprintf("Completed uploading file [%s] to path [%s]", action[y].Source, action[y].Destination), results[i].Error.Error())
//logger.WriteLogEntry("", fmt.Sprintf("[%s] Completed uploading file [%s] to Destination [%s] to host [%s]\n", time.Now().Format(time.ANSIC), action[y].Source, action[y].Destination, results[i].Host))
log.Infof("Succesfully uploaded [%s] to [%s] on [%s]", action[y].Source, action[y].Destination, results[i].Host)
}
case "download":
logger.WriteLogEntry("download", fmt.Sprintf("Downloading file [%s] to Destination [%s] from multiple hosts\n", action[y].Source, action[y].Destination), "", "")
results := ssh.ParalellDownload(hosts, action[y].Source, action[y].Destination, action[y].Timeout)
// Unlikely that this should happen
if len(results) == 0 {
return fmt.Errorf("No results have been returned from the parallel execution")
}
// Parse the results from the parallel updates
for i := range results {
if results[i].Error != nil {
// Set checkpoint
restore.Action = action[y].Name
restore.createCheckpoint()
logger.WriteLogEntry(results[i].Host, action[y].Name, "", results[i].Error.Error())
logger.SetLoggingState(results[i].Host, "Failed")
//logger.WriteLogEntry("", fmt.Sprintf("[%s] Error downloading file [%s] to [%s] to host [%s]\n", time.Now().Format(time.ANSIC), action[y].Source, action[y].Destination, results[i].Host))
//logger.WriteLogEntry("", fmt.Sprintf("[%s] [%s]\n", time.Now().Format(time.ANSIC), results[i].Error))
return fmt.Errorf("Download task [%s] on host [%s] failed with error [%s]", action[y].Name, results[i].Host, results[i].Error)
}
logger.WriteLogEntry(results[i].Host, action[y].Name, fmt.Sprintf("Completed Downloading file [%s] to path [%s]", action[y].Source, action[y].Destination), results[i].Error.Error())
//logger.WriteLogEntry("", fmt.Sprintf("[%s] Completed uploading file [%s] to Destination [%s] to host [%s]\n", time.Now().Format(time.ANSIC), action[y].Source, action[y].Destination, results[i].Host))
log.Infof("Succesfully uploaded [%s] to [%s] on [%s]", action[y].Source, action[y].Destination, results[i].Host)
}
case "command":
logger.WriteLogEntry("command", fmt.Sprintf("Executing command action [%s] to multiple hosts\n", action[y].Name), "", "")
command, err := buildCommand(action[y])
if err != nil {
// Set checkpoint
restore.Action = action[y].Name
restore.createCheckpoint()
return err
}
crs := ssh.ParalellExecute(command, action[y].CommandPipeFile, action[y].CommandPipeCmd, hosts, action[y].Timeout)
var errors bool // This will only be set to true if a command fails
for x := range crs {
if crs[x].Error != nil {
// Set checkpoint
restore.Action = action[y].Name
restore.createCheckpoint()
logger.WriteLogEntry(crs[x].Host, action[y].Name, crs[x].Result, crs[x].Error.Error())
logger.SetLoggingState(crs[x].Host, "Failed")
//log.Errorf("Command task [%s] on host [%s] failed with error [%s]\n\t[%s]", action[y].Name, crs[x].Host, crs[x].Result, crs[x].Error.Error())
errors = true // An error has been found
//logger.WriteLogEntry("", fmt.Sprintf("------------ Output ------------\n%s\n----------------------------------\n", crs[x].Result))
return fmt.Errorf("Command task [%s] on host [%s] failed with error [%s]\n\t[%s]", action[y].Name, crs[x].Host, crs[x].Error, crs[x].Result)
}
log.Infof("Command Task [%s] on node [%s] completed successfully", action[y].Name, crs[x].Host)
logger.WriteLogEntry(crs[x].Host, action[y].Name, crs[x].Result, "")
//logger.WriteLogEntry("", fmt.Sprintf("[%s] Command task [%s] on host [%s] has completed succesfully\n", time.Now().Format(time.ANSIC), action[y].Name, crs[x].Host))
//logger.WriteLogEntry("", fmt.Sprintf("------------ Output ------------\n%s\n----------------------------------\n", crs[x].Result))
}
if errors == true {
return fmt.Errorf("An error was encountered on command Task [%s]", action[y].Name)
}
case "pkg":
case "key":
default:
return fmt.Errorf("Unknown Action [%s]", action[y].ActionType)
}
}
return nil
}
================================================
FILE: pkg/parlay/parser_builder.go
================================================
package parlay
import (
"fmt"
"os"
"os/exec"
"strings"
"github.com/plunder-app/plunder/pkg/parlay/parlaytypes"
"github.com/plunder-app/plunder/pkg/ssh"
log "github.com/sirupsen/logrus"
)
func buildCommand(a parlaytypes.Action) (string, error) {
var command string
// An executable Key takes presedence
if a.KeyName != "" {
keycmd := Keys[a.KeyName]
// Check that the key exists
if keycmd == "" {
return "", fmt.Errorf("Unable to find command under key '%s'", a.KeyName)
}
if a.CommandSudo != "" {
// Add sudo to the Key command
command = fmt.Sprintf("sudo -n -u %s %s", a.CommandSudo, keycmd)
} else {
command = keycmd
}
} else {
// Not using a key, using a shell command
if a.CommandSudo != "" {
// Add sudo to the Shell command
command = fmt.Sprintf("sudo -n -u %s %s", a.CommandSudo, a.Command)
} else {
command = a.Command
}
}
return command, nil
}
func parseAndExecute(a parlaytypes.Action, h *ssh.HostSSHConfig) ssh.CommandResult {
// This will parse the options passed in the action and execute the required string
var cr ssh.CommandResult
var b []byte
command, err := buildCommand(a)
if err != nil {
cr.Error = err
return cr
}
if a.CommandLocal == true {
log.Debugf("Command [%s]", command)
cmd := exec.Command("bash", "-c", command)
b, cr.Error = cmd.CombinedOutput()
if cr.Error != nil {
return cr
}
cr.Result = strings.TrimRight(string(b), "\r\n")
} else {
log.Debugf("Executing command [%s] on host [%s]", command, h.Host)
cr = ssh.SingleExecute(command, a.CommandPipeFile, a.CommandPipeCmd, *h, a.Timeout)
cr.Result = strings.TrimRight(cr.Result, "\r\n")
// If the command hasn't returned anything, put a filler in
if cr.Result == "" {
cr.Result = "[No Output]"
}
if cr.Error != nil {
return cr
}
}
// Save the results into a key to be used at another point
if a.CommandSaveAsKey != "" {
log.Debugf("Adding new results to key [%s]", a.CommandSaveAsKey)
Keys[a.CommandSaveAsKey] = cr.Result
}
// Save the results into a file to be used at another point
if a.CommandSaveFile != "" {
var f *os.File
f, cr.Error = os.Create(a.CommandSaveFile)
if cr.Error != nil {
return cr
}
defer f.Close()
_, cr.Error = f.WriteString(cr.Result)
if cr.Error != nil {
return cr
}
f.Sync()
}
return cr
}
================================================
FILE: pkg/parlay/plugin/plugin.go
================================================
package parlayplugin
import (
"encoding/json"
"fmt"
"os"
"path/filepath"
"plugin"
"github.com/plunder-app/plunder/pkg/parlay/parlaytypes"
log "github.com/sirupsen/logrus"
)
// The pluginCache contains a map of action->plugin
var pluginCache map[string]string
func init() {
// Initialise the map
pluginCache = make(map[string]string)
}
// Find plugins returns an array of all .plugin files
func findPlugins(pluginDir string) ([]string, error) {
var plugins []string
// This function will look for all files in a specified directory (defaults to PWD/plugin)
filepath.Walk(pluginDir, func(path string, f os.FileInfo, err error) error {
if err != nil {
return err
}
if !f.IsDir() {
if filepath.Ext(path) == ".plugin" {
absPath, _ := filepath.Abs(path)
plugins = append(plugins, absPath)
}
}
return nil
})
return plugins, nil
}
func findFunctionInPlugin(pluginPath, functionName string) (plugin.Symbol, error) {
plug, err := plugin.Open(pluginPath)
if err != nil {
log.Debugf("%v", err)
return nil, fmt.Errorf("Unable to open Plugin [%s]", pluginPath)
}
symbol, err := plug.Lookup(functionName)
if err != nil {
log.Debugf("%v", err)
return nil, fmt.Errorf("Unable to read functions from Plugin [%s]", pluginPath)
}
return symbol, nil
}
func init() {
pluginList, err := findPlugins("./plugin")
if err != nil {
log.Errorf("%v", err)
} else {
log.Debugf("Found [%d] plugins", len(pluginList))
for x := range pluginList {
symbol, err := findFunctionInPlugin(pluginList[x], "ParlayActionList")
if err != nil {
log.Errorf("%v", err)
continue
}
pluginExec, ok := symbol.(func() []string)
if !ok {
log.Errorf("Unable to read functions from Plugin [%s]", pluginList[x])
continue
}
actions := pluginExec()
for z := range actions {
// This will give us a mapping of "action" => plugin
pluginCache[actions[z]] = pluginList[x]
}
}
}
}
//ListPlugins -
func ListPlugins() {
pluginList, err := findPlugins("./plugin")
if err != nil {
log.Errorf("%v", err)
} else {
log.Debugf("Found [%d] plugins", len(pluginList))
for x := range pluginList {
symbol, err := findFunctionInPlugin(pluginList[x], "ParlayPluginInfo")
if err != nil {
log.Errorf("%v", err)
continue
}
pluginExec, ok := symbol.(func() string)
if !ok {
log.Errorf("Unable to read functions from Plugin [%s]", pluginList[x])
continue
}
sanitizedPath := filepath.Base(pluginList[x])
fmt.Printf("%s\t%s\n", sanitizedPath, pluginExec())
}
}
}
//ListPluginActions -
func ListPluginActions(pluginPath string) {
symbol, err := findFunctionInPlugin(pluginPath, "ParlayActionList")
if err != nil {
log.Errorf("%v", err)
return
}
pluginExec, ok := symbol.(func() []string)
if !ok {
log.Errorf("Unable to read functions from Plugin [%s]", pluginPath)
return
}
actions := pluginExec()
symbol, err = findFunctionInPlugin(pluginPath, "ParlayActionDetails")
if err != nil {
log.Errorf("%v", err)
return
}
pluginExec, ok = symbol.(func() []string)
if !ok {
log.Errorf("Unable to read functions from Plugin [%s]", pluginPath)
return
}
descriptions := pluginExec()
if len(actions) != len(descriptions) {
log.Warnf("Not all actions have descriptions, contact your plugin provider to have this fixed")
}
for x := range actions {
fmt.Printf("%s\t%s\n", actions[x], descriptions[x])
}
}
//UsagePlugin returns the usage of a plugin function
func UsagePlugin(pluginPath, action string) {
symbol, err := findFunctionInPlugin(pluginPath, "ParlayUsage")
if err != nil {
log.Errorf("%v", err)
return
}
pluginExec, ok := symbol.(func(string) (json.RawMessage, error))
if !ok {
log.Errorf("Unable to read functions from Plugin [%s]", pluginPath)
return
}
result, err := pluginExec(action)
if err != nil {
log.Errorf("%v", err)
return
}
a := parlaytypes.Action{
Name: fmt.Sprintf("Example name for action [%s]", action),
ActionType: action,
Plugin: result,
}
b, _ := json.MarshalIndent(a, "", "\t")
fmt.Printf("%s\n", b)
}
// ExecuteAction uses the cache to find an action/plugin mapping
func ExecuteAction(action, host string, raw json.RawMessage) ([]parlaytypes.Action, error) {
if pluginCache[action] == "" {
// No KeyMap meaning that the action doesn't map to a plugin
return nil, fmt.Errorf("Action [%s] does not exist or has no plugin associated with it", action)
}
return ExecuteActionInPlugin(pluginCache[action], action, host, raw)
}
// ExecuteActionInPlugin specifies the plugin and action directly
func ExecuteActionInPlugin(pluginPath, action, host string, raw json.RawMessage) ([]parlaytypes.Action, error) {
// Check a function with the name ParlayExec exists
symbol, err := findFunctionInPlugin(pluginPath, "ParlayExec")
if err != nil {
return nil, fmt.Errorf("%v", err)
}
log.Debugf("Attempting plugin [%s]", action)
// Check the function has the correct parameters
pluginExec, ok := symbol.(func(string, string, json.RawMessage) ([]parlaytypes.Action, error))
if !ok {
return nil, fmt.Errorf("Unable to read functions from Plugin [%s]", pluginPath)
}
// Pass the action type and the interface to the plugin
return pluginExec(action, host, raw)
}
================================================
FILE: pkg/parlay/restore.go
================================================
package parlay
import (
"encoding/json"
"io/ioutil"
"os"
"github.com/mitchellh/go-homedir"
)
//Restore provides a checkpoint to resume from
type Restore struct {
Deployment string `json:"deployment"` // Name of deployment to restore from
Action string `json:"action"` // Action to restore from
Host string `json:"host"` // Single host to start from
Hosts []string `json:"hosts"` // Restart operation on a number of hosts
}
// restore is an interal struct used for execution restoration
var restore Restore
const restoreFile = ".parlay_restore"
// restoreFilePath will build a path where a file will be read/writted
func restoreFilePath() (string, error) {
home, err := homedir.Dir()
if err != nil {
return "", err
}
return home + "/" + restoreFile, nil
}
func (r *Restore) createCheckpoint() error {
// This function will create a checkpoint file that will allow Plunder to restart in the event of failure
path, err := restoreFilePath()
if err != nil {
return err
}
// Marshall the struct to a byte array
b, err := json.Marshal(r)
if err != nil {
return err
}
// Write the checkpoint file
err = ioutil.WriteFile(path, b, 0644)
return err
}
//RestoreFromCheckpoint will attempt to find a restoration checkpoint file
func RestoreFromCheckpoint() *Restore {
path, err := restoreFilePath()
if err != nil {
return nil
}
if _, err := os.Stat(path); !os.IsNotExist(err) {
b, err := ioutil.ReadFile(path)
if err != nil {
return nil
}
var r Restore
err = json.Unmarshal(b, &r)
if err != nil {
return nil
}
return &r
}
return nil
}
================================================
FILE: pkg/parlay/validate.go
================================================
package parlay
import (
"fmt"
"github.com/plunder-app/plunder/pkg/parlay/parlaytypes"
)
// ValidateAction will parse an action to ensure it is valid
func ValidateAction(action *parlaytypes.Action) error {
switch action.ActionType {
case "upload":
// Validate the upload action
if action.Source == "" {
return fmt.Errorf("The Source field can not be blank")
}
if action.Destination == "" {
return fmt.Errorf("The Destination field can not be blank")
}
return nil
case "download":
// Validate the download action
if action.Source == "" {
return fmt.Errorf("The Source field can not be blank")
}
if action.Destination == "" {
return fmt.Errorf("The Destination field can not be blank")
}
return nil
case "command":
// Validate the Command action
if action.Command == "" && action.KeyName == "" {
return fmt.Errorf("Neither a command or a key has been specified to execute")
}
if action.Command != "" && action.KeyName != "" {
return fmt.Errorf("Unable to use both a Command and a Command Key")
}
return nil
case "pkg":
// Validate the Package action
if action.PkgManager == "" {
return fmt.Errorf("The Package Manager field can not be blank")
} else if action.PkgManager != "apt" && action.PkgManager != "yum" {
return fmt.Errorf("Unknown Package Manager [%s]", action.PkgManager)
}
if action.PkgOperation == "" {
return fmt.Errorf("The Package Operation field can not be blank")
} else if action.PkgOperation != "install" && action.PkgOperation != "remove" {
return fmt.Errorf("Unknown Package Operation [%s]", action.PkgOperation)
}
if action.Packages == "" {
return fmt.Errorf("The Packages field can not be blank")
}
return nil
case "key":
// Validate the Key action
if action.KeyFile == "" {
return fmt.Errorf("The KeyField field can not be blank")
}
return nil
default:
return fmt.Errorf("Unknown Action [%s]", action.ActionType)
}
}
================================================
FILE: pkg/plunderlogging/consolelogger.go
================================================
package plunderlogging
================================================
FILE: pkg/plunderlogging/filelogger.go
================================================
package plunderlogging
import (
"fmt"
"os"
"sync"
)
// FileLogger allows parlay to log output to a file on the local filesystem
type FileLogger struct {
enabled bool
f *os.File
}
var fileLogging FileLogger
func (l *FileLogger) initFileLogger(logFile string) (err error) {
l.enabled = true
l.f, err = os.Create(logFile)
if err != nil {
return err
}
return nil
}
// This file based logging function may error, but logging should never break the running of a system, so errors are passed to "Debug" logging
func (l *FileLogger) writeEntry(target, entry string) error {
var fileMutex sync.Mutex
if l.enabled == true {
// As this may be called by numerous goroutines, we impose a mutex lock on it
fileMutex.Lock()
defer fileMutex.Unlock()
// TODO - Does this produce readable logging output
_, err := l.f.WriteString(fmt.Sprintf("Target=%s Entry=%s", target, entry))
return err
}
return nil
}
func (l *FileLogger) setLoggingState(target, state string) error {
return nil
}
================================================
FILE: pkg/plunderlogging/go.mod
================================================
module github.com/plunder-app/plunder/pkg/plunderlogging
go 1.12
================================================
FILE: pkg/plunderlogging/jsonlogger.go
================================================
package plunderlogging
import (
"fmt"
"time"
log "github.com/sirupsen/logrus"
)
// JSONLogger allows parlay to log output to an in-memory jsonStruct
type JSONLogger struct {
enabled bool
logger map[string]*JSONLog
}
// JSONLog contains all of the output from a parlay execution
type JSONLog struct {
State string `json:"state"`
Entries []JSONLogEntry `json:"entries"`
}
// JSONLogEntry contains the details a specific action
type JSONLogEntry struct {
Created time.Time `json:"created"`
TaskName string `json:"task"`
Err string `json:"error"`
Entry string `json:"entry"`
}
func (j *JSONLogger) initJSONLogger() {
j.enabled = true
j.logger = make(map[string]*JSONLog)
}
func (j *JSONLogger) writeEntry(target, task, entry, err string) {
// Create new entry
newEntry := JSONLogEntry{
Created: time.Now(),
Entry: entry,
TaskName: task,
Err: err,
}
// Check if the logger exists
existingLog, ok := j.logger[target]
if ok {
// Update an existing entry
existingLog.Entries = append(existingLog.Entries, newEntry)
} else {
// Create a new logger
newLog := JSONLog{
State: "Running",
}
// Append the entry to it
newLog.Entries = append(newLog.Entries, newEntry)
// Update the in-memory log store
j.logger[target] = &newLog
log.Debugf("Creating new logs for target [%s]", target)
}
}
func (j *JSONLogger) deleteLog(target string) error {
// Check if the entry exists
_, ok := j.logger[target]
if ok {
// If it does, then we use the in-built function to delete the log entry
delete(j.logger, target)
} else {
// Return a warning
return fmt.Errorf("In-Memory logging for [%s] either doesn't exist or has already been deleted", target)
}
return nil
}
func (j *JSONLogger) setLoggingState(target, state string) error {
// Check if the logger exists
existingLog, ok := j.logger[target]
if ok {
// Update an existing entry
existingLog.State = state
} else {
return fmt.Errorf("In-Memory logging for [%s] either doesn't exist or has already been deleted", target)
}
return nil
}
================================================
FILE: pkg/plunderlogging/logger.go
================================================
package plunderlogging
import "fmt"
// Logger - is a stuct that manages the verious types of logger available
type Logger struct {
json JSONLogger
file FileLogger
}
// EnableJSONLogging - will enable logging through JSON
func (l *Logger) EnableJSONLogging(e bool) {
l.json.enabled = e
l.json.initJSONLogger()
}
// EnableFileLogging - will enable logging to a file
func (l *Logger) EnableFileLogging(e bool) {
l.file.enabled = e
}
// InitLogFile - will initialise file based logging
func (l *Logger) InitLogFile(path string) error {
if l.file.enabled != true {
return l.file.initFileLogger(path)
}
// Dont re-initialise the file
return nil
}
// InitJSON - will start/initialise the JSON logging functionality
func (l *Logger) InitJSON() {
// Dont re-initialise the json
if l.json.enabled != true {
l.json.initJSONLogger()
}
}
// target - the entity we're affecting
// entry - the results of the operation on the target
// WriteLogEntry will capture what is transpiring and where
func (l *Logger) WriteLogEntry(target, task, entry, err string) {
if l.file.enabled {
l.file.writeEntry(target, entry)
}
if l.json.enabled {
l.json.writeEntry(target, task, entry, err)
}
// A logging system shouldnt break anything so any errors are just outputed to STDOUT
}
// SetLoggingState - currently a NOOP (TODO)
func (l *Logger) SetLoggingState(target, state string) {
if l.file.enabled {
l.file.setLoggingState(target, state)
}
if l.json.enabled {
l.json.setLoggingState(target, state)
}
// A logging system shouldnt break anything so any errors are just outputed to STDOUT
}
// GetJSONLogs - returns a pointer to the current JSON Logs
func (l *Logger) GetJSONLogs(target string) (*JSONLog, error) {
if l.json.logger == nil {
return nil, fmt.Errorf("JSON Logging hasn't been enabled")
}
// Check if the logger exists
existingLog, ok := l.json.logger[target]
if ok {
return existingLog, nil
}
return nil, fmt.Errorf("No Logs for Target [%s] exist", target)
}
// DeleteLogs - will remove logs for a particular target
func (l *Logger) DeleteLogs(target string) error {
if l.json.logger == nil {
return nil
}
return l.json.deleteLog(target)
}
================================================
FILE: pkg/services/deployments.go
================================================
package services
import (
"encoding/json"
"fmt"
"strings"
"github.com/plunder-app/plunder/pkg/utils"
log "github.com/sirupsen/logrus"
)
// DefaultBootType specifies what a server will default to if no config is found
var DefaultBootType string
// This stores the mapping for a url to the data /macaddress.file => data
var httpPaths map[string]string
func init() {
// Initialise the paths map
httpPaths = make(map[string]string)
}
// rebuildConfiguration - will parse the entire deployment configuration and update anything that is missing
func rebuildConfiguration(updateConfig *DeploymentConfigurationFile) error {
// If HTTP isn't enabled we can't build the multiplexer for URLs
if serveMux == nil {
return fmt.Errorf("Deployment HTTP Server isn't enabled, so parsing deployments isn't possible")
}
// If a key is specified then we read it and base64 the file into the SSHKEY string
if updateConfig.GlobalServerConfig.SSHKeyPath != "" {
err := updateConfig.GlobalServerConfig.parseSSH()
if err != nil {
log.Errorf(err.Error())
}
}
log.Debugf("Parsing [%d] Configurations", len(updateConfig.Configs))
for i := range updateConfig.Configs {
// inMemipxeConfig is a custom configuration that matches kernel/initrd & cmdline and is 00:11:22:33:44:55.ipxe
var inMemipxeConfig string
// inMemipxeConfig is a custom configuration that is specific to the boot type [preseed/kickstart/vsphere] and is 00:11:22:33:44:55.cfg
var inMemBootConfig string
// imMemESXiKickstart is a custom configuration specific to vSphere for it's kickstart
var imMemESXiKickstart string
// inMemBOOTyConfig is a custom configuration that matches kernel/initrd & cmdline and is 00:11:22:33:44:55.bty
var inMemBOOTyConfig string
// We need to move all ":" to "-" to make life a little easier for filesystems and internet standards
dashMac := strings.Replace(updateConfig.Configs[i].MAC, ":", "-", -1)
// Find the deployment configuration for this host, either custom or inherit from the controller
bootConfig := findBootConfigForDeployment(updateConfig.Configs[i])
// If there is no deployment configuration under this name return an error
if bootConfig == nil {
errorString := fmt.Errorf("Host [%s] uses unknown config [%s], stopping config update", updateConfig.Configs[i].MAC, updateConfig.Configs[i].ConfigName)
log.Errorln(errorString)
return errorString
}
// Ensure this entry has the correct mapping
updateConfig.Configs[i].ConfigBoot = *bootConfig
// This will populate anything missing from the global configuration
updateConfig.Configs[i].ConfigHost.PopulateFromGlobalConfiguration(updateConfig.GlobalServerConfig)
// If a key is specified then we read it and base64 the file into the SSHKEY string
if updateConfig.Configs[i].ConfigHost.SSHKeyPath != "" {
err := updateConfig.Configs[i].ConfigHost.parseSSH()
if err != nil {
log.Errorf(err.Error())
}
} else {
log.Errorf("This server [%s] will be deployed with no SSH Key", updateConfig.Configs[i].ConfigHost.ServerName)
}
// Look for understood config types
switch updateConfig.Configs[i].ConfigBoot.ConfigType {
case "preseed":
inMemipxeConfig = utils.IPXEPreeseed(httpAddress, bootConfig.Kernel, bootConfig.Initrd, bootConfig.Cmdline)
log.Debugf("Generating preseed ipxeConfig for configName [%s]", dashMac)
inMemBootConfig = updateConfig.Configs[i].ConfigHost.BuildPreeSeedConfig()
case "kickstart":
inMemipxeConfig = utils.IPXEKickstart(httpAddress, bootConfig.Kernel, bootConfig.Initrd, bootConfig.Cmdline)
log.Debugf("Generating kickstart ipxeConfig for configName [%s]", dashMac)
inMemBootConfig = updateConfig.Configs[i].ConfigHost.BuildKickStartConfig()
case "vsphere":
inMemipxeConfig = utils.IPXEVSphere(httpAddress, bootConfig.Kernel, bootConfig.Cmdline)
log.Debugf("Generating vsphere ipxeConfig for configName [%s]", dashMac)
inMemBootConfig = updateConfig.Configs[i].ConfigHost.BuildESXiConfig()
imMemESXiKickstart = updateConfig.Configs[i].ConfigHost.BuildESXiKickStart()
case "booty":
inMemipxeConfig = utils.IPXEBOOTy(httpAddress, bootConfig.Kernel, bootConfig.Initrd, bootConfig.Cmdline)
log.Debugf("Generating booty ipxeConfig for configName [%s]", dashMac)
inMemBOOTyConfig = updateConfig.Configs[i].ConfigHost.BuildBOOTYconfig()
default:
log.Debugf("Generating default ipxeConfig for configName [%s]", updateConfig.Configs[i].ConfigBoot.ConfigName)
inMemipxeConfig = utils.IPXEAnyBoot(httpAddress, bootConfig.Kernel, bootConfig.Initrd, bootConfig.Cmdline)
}
// Build the configuration that is passed to iPXE on boot
if inMemipxeConfig != "" {
path := fmt.Sprintf("/%s.ipxe", dashMac)
if _, ok := httpPaths[path]; !ok {
// Only create the handler if one doesn't exist
serveMux.HandleFunc(path, rootHandler)
}
httpPaths[path] = inMemipxeConfig
}
// Build a boot configuration that is passed to a kernel
if inMemBootConfig != "" {
path := fmt.Sprintf("/%s.cfg", dashMac)
if _, ok := httpPaths[path]; !ok {
// Only create the handler if one doesn't exist
serveMux.HandleFunc(path, rootHandler)
}
httpPaths[path] = inMemBootConfig
}
// Build a vSphere kickstart configuration that is passed to an installer
if imMemESXiKickstart != "" {
path := fmt.Sprintf("/%s.ks", dashMac)
if _, ok := httpPaths[path]; !ok {
// Only create the handler if one doesn't exist
serveMux.HandleFunc(path, rootHandler)
}
httpPaths[path] = imMemESXiKickstart
}
// Build a BOOTy configuration that is passed to an installer
if inMemBOOTyConfig != "" {
path := fmt.Sprintf("/%s.bty", dashMac)
if _, ok := httpPaths[path]; !ok {
// Only create the handler if one doesn't exist
serveMux.HandleFunc(path, rootHandler)
}
httpPaths[path] = inMemBOOTyConfig
}
}
if len(updateConfig.Configs) == 0 {
// No changes, leave as is (with a warning)
log.Warnln("No deployment configuration, any existing configuration will remain")
} else {
// Updated configuration has been parsed, update internal deployment configuration
log.Infoln("Updating of deployment configuration complete")
Deployments = *updateConfig
}
return nil
}
// UpdateDeploymentConfig will read a configuration string and build the iPXE files needed
func UpdateDeploymentConfig(rawDeploymentConfig []byte) error {
// Read through the deployment configuration
log.Infoln("Updating the Deployment Configuration")
// Work out if it is a YAML/JSON or unknown
updateConfig, err := ParseDeployment(rawDeploymentConfig)
if err != nil {
return err
}
return rebuildConfiguration(updateConfig)
}
// AddDeployment - This function will add a new deployment to the deployment configuration
func AddDeployment(rawDeployment []byte) error {
var newDeployment DeploymentConfig
err := json.Unmarshal(rawDeployment, &newDeployment)
if err != nil {
return fmt.Errorf("Unable to parse deployment configuration")
}
// Find the original deployment via it's mac address
for i := range Deployments.Configs {
// Compare this deployment to the one we're looking for
if Deployments.Configs[i].MAC == newDeployment.MAC {
return fmt.Errorf("Duplicate entry for MAC address [%s]", newDeployment.MAC)
}
}
// We will now duplicate our configuration
updateConfig := Deployments
// We will need to create space to copy the existing configurations over
updateConfig.Configs = make([]DeploymentConfig, len(Deployments.Configs))
// Copy our existing configurations into the new configuration
copy(updateConfig.Configs, Deployments.Configs)
// Append our new configuration into our new copy
updateConfig.Configs = append(updateConfig.Configs, newDeployment)
// Remove the deployment from the unleased addresses
controller.DelUnLeased(newDeployment.MAC)
// Parse the new configuration
return rebuildConfiguration(&updateConfig)
}
// GetDeployment - This function will add a new deployment to the deployment configuration
func GetDeployment(macAddress string) *DeploymentConfig {
// Iterate through all the deployments
for i := range Deployments.Configs {
if macAddress == Deployments.Configs[i].MAC {
return &Deployments.Configs[i]
}
}
return nil
}
// UpdateDeployment - This function will add a new deployment to the deployment configuration
func UpdateDeployment(macAddress string, rawDeployment []byte) error {
var newDeployment DeploymentConfig
err := json.Unmarshal(rawDeployment, &newDeployment)
if err != nil {
return fmt.Errorf("Unable to parse deployment configuration")
}
// if no ID or specific MAC address was passed then assume the mac address of the deployment
if macAddress == "" {
macAddress = newDeployment.MAC
}
// We will now duplicate our configuration
updateConfig := Deployments
// We will need to create space to copy the existing configurations over
updateConfig.Configs = make([]DeploymentConfig, len(Deployments.Configs))
// Copy our existing configurations into the new configuration
copy(updateConfig.Configs, Deployments.Configs)
// Find the original deployment via it's mac address
for i := range updateConfig.Configs {
// Compare this deployment to the one we're looking for
if updateConfig.Configs[i].MAC == macAddress {
// Remove the old matching configuration
updateConfig.Configs = append(updateConfig.Configs[:i], updateConfig.Configs[i+1:]...)
// Append our new configuration into our new copy
updateConfig.Configs = append(updateConfig.Configs, newDeployment)
// Parse the new configuration
return rebuildConfiguration(&updateConfig)
}
}
return fmt.Errorf("Unable to find existing deployment for MAC address [%s]", macAddress)
}
// DeleteDeploymentMac - This function will delete a deployment based upon it's mac Address
func DeleteDeploymentMac(macAddress string, rawDeployment []byte) error {
// We will now duplicate our configuration
updateConfig := Deployments
// We will need to create space to copy the existing configurations over
updateConfig.Configs = make([]DeploymentConfig, len(Deployments.Configs))
// Copy our existing configurations into the new configuration
copy(updateConfig.Configs, Deployments.Configs)
// Find the original deployment via it's mac address
for i := range updateConfig.Configs {
// Compare this deployment to the one we're looking for
if updateConfig.Configs[i].MAC == macAddress {
// Remove http Handler (if it exists)
_, ok := httpPaths[fmt.Sprintf("%s.ipxe", updateConfig.Configs[i].MAC)]
if ok {
delete(httpPaths, fmt.Sprintf("%s.ipxe", updateConfig.Configs[i].MAC))
}
// Remove the old matching configuration
updateConfig.Configs = append(updateConfig.Configs[:i], updateConfig.Configs[i+1:]...)
// Parse the new configuration
return rebuildConfiguration(&updateConfig)
}
}
return fmt.Errorf("Unable to find existing deployment for Address [%s]", macAddress)
}
// DeleteDeploymentAddress - This function will delete a deployment based upon it's IP Address
func DeleteDeploymentAddress(address string, rawDeployment []byte) error {
// We will now duplicate our configuration
updateConfig := Deployments
// We will need to create space to copy the existing configurations over
updateConfig.Configs = make([]DeploymentConfig, len(Deployments.Configs))
// Copy our existing configurations into the new configuration
copy(updateConfig.Configs, Deployments.Configs)
// Find the original deployment via it's mac address
for i := range updateConfig.Configs {
// Compare this deployment to the one we're looking for
if updateConfig.Configs[i].ConfigHost.IPAddress == address {
// Remove http Handler (if it exists)
_, ok := httpPaths[fmt.Sprintf("%s.ipxe", updateConfig.Configs[i].MAC)]
if ok {
delete(httpPaths, fmt.Sprintf("%s.ipxe", updateConfig.Configs[i].MAC))
}
// Remove the old matching configuration
updateConfig.Configs = append(updateConfig.Configs[:i], updateConfig.Configs[i+1:]...)
// Parse the new configuration
return rebuildConfiguration(&updateConfig)
}
}
return fmt.Errorf("Unable to find existing deployment for Address [%s]", address)
}
// UpdateGlobalDeploymentConfig - This allows updating of the global configuration independently
func UpdateGlobalDeploymentConfig(rawDeployment []byte) error {
var globalDeploymentConfig HostConfig
err := json.Unmarshal(rawDeployment, &globalDeploymentConfig)
if err != nil {
return fmt.Errorf("Unable to parse deployment configuration")
}
// Update the deployments with the new configuration
Deployments.GlobalServerConfig = globalDeploymentConfig
return nil
}
//FindDeploymentConfigFromMac - this will return the deployment configuration, allowing the DHCP server to return the correct DHCP options
func FindDeploymentConfigFromMac(mac string) string {
// AnyBoot will just boot the specified kernel/initrd
// if AnyBoot == true {
// return "anyboot"
// }
if len(Deployments.Configs) == 0 {
// No configurations have been loaded
log.Warnln("Attempted to perform Mac Address lookup, however no configurations have been loaded")
return ""
}
for i := range Deployments.Configs {
log.Debugf("Comparing [%s] to [%s]", mac, strings.ToLower(Deployments.Configs[i].MAC))
if mac == strings.ToLower(Deployments.Configs[i].MAC) {
return Deployments.Configs[i].ConfigName
}
}
return DefaultBootType
}
================================================
FILE: pkg/services/go.mod
================================================
module github.com/plunder-app/plunder/pkg/services
go 1.12
================================================
FILE: pkg/services/handler.go
================================================
package services
import (
"encoding/json"
"fmt"
"io/ioutil"
"net/http"
"strings"
"github.com/gorilla/mux"
"github.com/plunder-app/plunder/pkg/apiserver"
)
// RegisterToAPIServer - will add the endpoints to the API server
func RegisterToAPIServer() {
// ------------------------------------------------
// Server configuration API registration
// ------------------------------------------------
apiserver.AddDynamicEndpoint("/config",
"/config",
"Allows the retrieving of Plunder Server configuration",
"config",
http.MethodGet,
getConfig)
apiserver.AddDynamicEndpoint("/config",
"/config",
"Allows the creation of Plunder Server configuration",
"config",
http.MethodPost,
postConfig)
apiserver.AddDynamicEndpoint("/config/boot/{id}",
"/config/boot",
"Allows the creation of Plunder Server Boot configuration",
"configBoot",
http.MethodPost,
postBootConfig)
apiserver.AddDynamicEndpoint("/config/boot/{id}",
"/config/boot",
"Performs the deletion of Plunder Server Boot configuration",
"configBoot",
http
gitextract_qm74k6_1/ ├── .gitignore ├── Dockerfile ├── LICENSE ├── Makefile ├── Readme.md ├── cmd/ │ ├── automate.go │ ├── configGenerator.go │ ├── plunder.go │ └── server.go ├── docs/ │ ├── actions.md │ ├── application_architecture.md │ ├── deployment.md │ ├── example_architecture.md │ ├── example_deployment.md │ ├── provisioning.md │ ├── readme.md │ └── service.md ├── go.mod ├── go.sum ├── hack/ │ └── comboot_ipxe/ │ ├── Dockerfile │ └── gen_comboot.ipxe ├── main.go ├── pkg/ │ ├── apiserver/ │ │ ├── README.md │ │ ├── client.go │ │ ├── config.go │ │ ├── endpoints.go │ │ ├── go.mod │ │ ├── handlerApiserver.go │ │ ├── logging.go │ │ ├── loggingHandlers.go │ │ ├── server.go │ │ └── types.go │ ├── certs/ │ │ ├── certs.go │ │ └── go.mod │ ├── go.mod │ ├── parlay/ │ │ ├── go.mod │ │ ├── handler.go │ │ ├── parlay.go │ │ ├── parlay_ui.go │ │ ├── parlaytypes/ │ │ │ ├── finder.go │ │ │ ├── go.mod │ │ │ └── parlaytypes.go │ │ ├── parser.go │ │ ├── parser_builder.go │ │ ├── plugin/ │ │ │ └── plugin.go │ │ ├── restore.go │ │ └── validate.go │ ├── plunderlogging/ │ │ ├── consolelogger.go │ │ ├── filelogger.go │ │ ├── go.mod │ │ ├── jsonlogger.go │ │ └── logger.go │ ├── services/ │ │ ├── deployments.go │ │ ├── go.mod │ │ ├── handler.go │ │ ├── server.go │ │ ├── serverDHCP.go │ │ ├── serverHTTP.go │ │ ├── serverHTTPISO.go │ │ ├── serverImageHTTP.go │ │ ├── serverTFTP.go │ │ ├── services.go │ │ ├── static_pxe.go │ │ ├── templateBOOTy.go │ │ ├── templateESXi.go │ │ ├── templateKickstart.go │ │ ├── templatePreseed.go │ │ ├── templateUtils.go │ │ └── types.go │ ├── ssh/ │ │ ├── go.mod │ │ ├── sshClient.go │ │ ├── sshCommand.go │ │ ├── sshConfig.go │ │ ├── sshImport.go │ │ └── sshTransfer.go │ └── utils/ │ ├── go.mod │ ├── ipxe.go │ ├── nic.go │ └── utils.go ├── plugin/ │ ├── docker/ │ │ ├── docker.go │ │ └── docker_actions.go │ ├── example.go │ └── kubeadm/ │ ├── kubeadm.go │ └── kubeadm_actions.go └── testing.sh
SYMBOL INDEX (285 symbols across 55 files)
FILE: cmd/automate.go
function init (line 38) | func init() {
function parseMapFile (line 453) | func parseMapFile(b []byte) (deployment parlaytypes.TreasureMap, err err...
FILE: cmd/configGenerator.go
function init (line 29) | func init() {
function renderOutput (line 289) | func renderOutput(data interface{}, pretty bool) error {
function detectServerConfig (line 312) | func detectServerConfig() error {
FILE: cmd/plunder.go
function init (line 28) | func init() {
function Execute (line 38) | func Execute() {
FILE: cmd/server.go
function init (line 25) | func init() {
FILE: main.go
function main (line 11) | func main() {
FILE: pkg/apiserver/client.go
function FindFunctionEndpoint (line 17) | func FindFunctionEndpoint(u *url.URL, c *http.Client, f, m string) (*End...
function BuildEnvironmentFromConfig (line 41) | func BuildEnvironmentFromConfig(path, urlFlag string) (*url.URL, *http.C...
function ParsePlunderGet (line 83) | func ParsePlunderGet(u *url.URL, c *http.Client) (*Response, error) {
function ParsePlunderPatch (line 109) | func ParsePlunderPatch(u *url.URL, c *http.Client, data []byte) (*Respon...
function ParsePlunderPost (line 141) | func ParsePlunderPost(u *url.URL, c *http.Client, data []byte) (*Respons...
function ParsePlunderDelete (line 168) | func ParsePlunderDelete(u *url.URL, c *http.Client) (*Response, error) {
FILE: pkg/apiserver/config.go
type ClientConfig (line 14) | type ClientConfig struct
method GetServerAddressURL (line 132) | func (c *ClientConfig) GetServerAddressURL() *url.URL {
method RetrieveClientCert (line 155) | func (c *ClientConfig) RetrieveClientCert() ([]byte, error) {
type ServerConfig (line 21) | type ServerConfig struct
method RetrieveKey (line 145) | func (s *ServerConfig) RetrieveKey() ([]byte, error) {
method RetrieveClientCert (line 150) | func (s *ServerConfig) RetrieveClientCert() ([]byte, error) {
function openClientConfig (line 27) | func openClientConfig(path string) (*ClientConfig, error) {
function OpenServerConfig (line 53) | func OpenServerConfig(path string) (*ServerConfig, error) {
function WriteServerConfig (line 79) | func WriteServerConfig(path, hostname, address string, port int, cert, k...
function WriteClientConfig (line 107) | func WriteClientConfig(path, address string, s *ServerConfig) error {
function retrieveCert (line 140) | func retrieveCert(cert string) ([]byte, error) {
FILE: pkg/apiserver/endpoints.go
type EndPoint (line 15) | type EndPoint struct
function AddDynamicEndpoint (line 24) | func AddDynamicEndpoint(endpointPattern, path, description, name, method...
function GetEndpoint (line 43) | func GetEndpoint(name, method string) *EndPoint {
function FunctionPath (line 53) | func FunctionPath() string {
FILE: pkg/apiserver/handlerApiserver.go
function getAPIFunctionMethod (line 12) | func getAPIFunctionMethod(w http.ResponseWriter, r *http.Request) {
function getAPIs (line 39) | func getAPIs(w http.ResponseWriter, r *http.Request) {
FILE: pkg/apiserver/logging.go
type Notification (line 18) | type Notification struct
function RegisterNotificationManager (line 24) | func RegisterNotificationManager(managerName, endpoint string) error {
function NotifyManager (line 37) | func NotifyManager(managerName string, n Notification) error {
function init (line 48) | func init() {
type unsubscribeFunc (line 55) | type unsubscribeFunc
type subscriber (line 57) | type subscriber interface
function handleSubscribers (line 61) | func handleSubscribers(s subscriber) http.HandlerFunc {
type notifier (line 104) | type notifier interface
type notificationManager (line 108) | type notificationManager struct
method subscribe (line 120) | func (nc *notificationManager) subscribe(n chan Notification) (unsubsc...
method notifySubscribers (line 136) | func (nc *notificationManager) notifySubscribers(n Notification) error {
function newNotificationManager (line 113) | func newNotificationManager() *notificationManager {
FILE: pkg/apiserver/server.go
function init (line 15) | func init() {
function StartAPIServer (line 22) | func StartAPIServer(path string, port int, insecure bool) error {
FILE: pkg/apiserver/types.go
type Response (line 6) | type Response struct
FILE: pkg/certs/certs.go
function GenerateKeyPair (line 24) | func GenerateKeyPair(hosts []string, start time.Time, length time.Durati...
function WriteKeyToFile (line 129) | func WriteKeyToFile(path string) error {
function WritePemToFile (line 139) | func WritePemToFile(path string) error {
function GetKey (line 150) | func GetKey() []byte {
function GetPem (line 155) | func GetPem() []byte {
FILE: pkg/parlay/handler.go
function RegisterToAPIServer (line 20) | func RegisterToAPIServer() {
function postParlay (line 54) | func postParlay(w http.ResponseWriter, r *http.Request) {
function getParlay (line 93) | func getParlay(w http.ResponseWriter, r *http.Request) {
function delParlay (line 125) | func delParlay(w http.ResponseWriter, r *http.Request) {
FILE: pkg/parlay/parlay.go
type actionType (line 3) | type actionType
constant upload (line 7) | upload actionType = "upload"
constant download (line 8) | download actionType = "download"
constant command (line 9) | command actionType = "command"
constant pkg (line 10) | pkg actionType = "package"
function init (line 18) | func init() {
FILE: pkg/parlay/parlay_ui.go
function contains (line 11) | func contains(v string, a []string) bool {
function StartUI (line 21) | func StartUI(m *parlaytypes.TreasureMap) (*parlaytypes.TreasureMap, erro...
FILE: pkg/parlay/parlaytypes/finder.go
method FindDeployments (line 10) | func (m *TreasureMap) FindDeployments(deployment []string) (*TreasureMap...
method FindHosts (line 30) | func (d *Deployment) FindHosts(hosts []string) (*Deployment, error) {
method FindActions (line 50) | func (d *Deployment) FindActions(actions []string) ([]Action, error) {
method FindDeployment (line 68) | func (m *TreasureMap) FindDeployment(deployment, action, host, logFile s...
FILE: pkg/parlay/parlaytypes/parlaytypes.go
type TreasureMap (line 9) | type TreasureMap struct
type Deployment (line 15) | type Deployment struct
type Action (line 30) | type Action struct
FILE: pkg/parlay/parser.go
function GetTargetLogs (line 19) | func GetTargetLogs(target string) (*plunderlogging.JSONLog, error) {
function DeleteTargetLogs (line 24) | func DeleteTargetLogs(target string) error {
function DeploySSH (line 29) | func DeploySSH(m *parlaytypes.TreasureMap, logFile string, jsonLogging, ...
function startDeployments (line 110) | func startDeployments(d []parlaytypes.Deployment) error {
function sequentialDeployment (line 158) | func sequentialDeployment(action []parlaytypes.Action, hostConfig ssh.Ho...
function parallelDeployment (line 248) | func parallelDeployment(action []parlaytypes.Action, hosts []ssh.HostSSH...
FILE: pkg/parlay/parser_builder.go
function buildCommand (line 14) | func buildCommand(a parlaytypes.Action) (string, error) {
function parseAndExecute (line 43) | func parseAndExecute(a parlaytypes.Action, h *ssh.HostSSHConfig) ssh.Com...
FILE: pkg/parlay/plugin/plugin.go
function init (line 18) | func init() {
function findPlugins (line 24) | func findPlugins(pluginDir string) ([]string, error) {
function findFunctionInPlugin (line 43) | func findFunctionInPlugin(pluginPath, functionName string) (plugin.Symbo...
function init (line 61) | func init() {
function ListPlugins (line 92) | func ListPlugins() {
function ListPluginActions (line 118) | func ListPluginActions(pluginPath string) {
function UsagePlugin (line 158) | func UsagePlugin(pluginPath, action string) {
function ExecuteAction (line 187) | func ExecuteAction(action, host string, raw json.RawMessage) ([]parlayty...
function ExecuteActionInPlugin (line 196) | func ExecuteActionInPlugin(pluginPath, action, host string, raw json.Raw...
FILE: pkg/parlay/restore.go
type Restore (line 12) | type Restore struct
method createCheckpoint (line 33) | func (r *Restore) createCheckpoint() error {
constant restoreFile (line 22) | restoreFile = ".parlay_restore"
function restoreFilePath (line 25) | func restoreFilePath() (string, error) {
function RestoreFromCheckpoint (line 52) | func RestoreFromCheckpoint() *Restore {
FILE: pkg/parlay/validate.go
function ValidateAction (line 10) | func ValidateAction(action *parlaytypes.Action) error {
FILE: pkg/plunderlogging/filelogger.go
type FileLogger (line 10) | type FileLogger struct
method initFileLogger (line 17) | func (l *FileLogger) initFileLogger(logFile string) (err error) {
method writeEntry (line 27) | func (l *FileLogger) writeEntry(target, entry string) error {
method setLoggingState (line 43) | func (l *FileLogger) setLoggingState(target, state string) error {
FILE: pkg/plunderlogging/jsonlogger.go
type JSONLogger (line 11) | type JSONLogger struct
method initJSONLogger (line 30) | func (j *JSONLogger) initJSONLogger() {
method writeEntry (line 35) | func (j *JSONLogger) writeEntry(target, task, entry, err string) {
method deleteLog (line 63) | func (j *JSONLogger) deleteLog(target string) error {
method setLoggingState (line 76) | func (j *JSONLogger) setLoggingState(target, state string) error {
type JSONLog (line 17) | type JSONLog struct
type JSONLogEntry (line 23) | type JSONLogEntry struct
FILE: pkg/plunderlogging/logger.go
type Logger (line 6) | type Logger struct
method EnableJSONLogging (line 12) | func (l *Logger) EnableJSONLogging(e bool) {
method EnableFileLogging (line 18) | func (l *Logger) EnableFileLogging(e bool) {
method InitLogFile (line 23) | func (l *Logger) InitLogFile(path string) error {
method InitJSON (line 33) | func (l *Logger) InitJSON() {
method WriteLogEntry (line 46) | func (l *Logger) WriteLogEntry(target, task, entry, err string) {
method SetLoggingState (line 59) | func (l *Logger) SetLoggingState(target, state string) {
method GetJSONLogs (line 72) | func (l *Logger) GetJSONLogs(target string) (*JSONLog, error) {
method DeleteLogs (line 85) | func (l *Logger) DeleteLogs(target string) error {
FILE: pkg/services/deployments.go
function init (line 18) | func init() {
function rebuildConfiguration (line 24) | func rebuildConfiguration(updateConfig *DeploymentConfigurationFile) err...
function UpdateDeploymentConfig (line 166) | func UpdateDeploymentConfig(rawDeploymentConfig []byte) error {
function AddDeployment (line 179) | func AddDeployment(rawDeployment []byte) error {
function GetDeployment (line 211) | func GetDeployment(macAddress string) *DeploymentConfig {
function UpdateDeployment (line 222) | func UpdateDeployment(macAddress string, rawDeployment []byte) error {
function DeleteDeploymentMac (line 260) | func DeleteDeploymentMac(macAddress string, rawDeployment []byte) error {
function DeleteDeploymentAddress (line 291) | func DeleteDeploymentAddress(address string, rawDeployment []byte) error {
function UpdateGlobalDeploymentConfig (line 322) | func UpdateGlobalDeploymentConfig(rawDeployment []byte) error {
function FindDeploymentConfigFromMac (line 334) | func FindDeploymentConfigFromMac(mac string) string {
FILE: pkg/services/handler.go
function RegisterToAPIServer (line 15) | func RegisterToAPIServer() {
function getConfig (line 129) | func getConfig(w http.ResponseWriter, r *http.Request) {
function postConfig (line 145) | func postConfig(w http.ResponseWriter, r *http.Request) {
function postBootConfig (line 162) | func postBootConfig(w http.ResponseWriter, r *http.Request) {
function deleteBootConfig (line 211) | func deleteBootConfig(w http.ResponseWriter, r *http.Request) {
function getDeployments (line 232) | func getDeployments(w http.ResponseWriter, r *http.Request) {
function postDeployments (line 248) | func postDeployments(w http.ResponseWriter, r *http.Request) {
function getSpecificDeployment (line 265) | func getSpecificDeployment(w http.ResponseWriter, r *http.Request) {
function postDeployment (line 294) | func postDeployment(w http.ResponseWriter, r *http.Request) {
function updateDeployment (line 311) | func updateDeployment(w http.ResponseWriter, r *http.Request) {
function deleteDeployment (line 347) | func deleteDeployment(w http.ResponseWriter, r *http.Request) {
function deleteDeploymentMac (line 375) | func deleteDeploymentMac(w http.ResponseWriter, r *http.Request) {
function deleteDeploymentAddress (line 397) | func deleteDeploymentAddress(w http.ResponseWriter, r *http.Request) {
function getDHCP (line 418) | func getDHCP(w http.ResponseWriter, r *http.Request) {
FILE: pkg/services/server.go
function ParseControllerData (line 21) | func ParseControllerData(b []byte) error {
method Parse (line 43) | func (b *BootConfig) Parse() error {
method DeleteBootControllerConfig (line 115) | func (c *BootController) DeleteBootControllerConfig(configName string) e...
function ParseDeployment (line 132) | func ParseDeployment(b []byte) (*DeploymentConfigurationFile, error) {
FILE: pkg/services/serverDHCP.go
type Lease (line 15) | type Lease struct
type DHCPSettings (line 21) | type DHCPSettings struct
method ServeDHCP (line 38) | func (h *DHCPSettings) ServeDHCP(p dhcp.Packet, msgType dhcp.MessageTy...
method leaseHander (line 144) | func (h *DHCPSettings) leaseHander(deploymentType, mac string) {
method freeLease (line 176) | func (h *DHCPSettings) freeLease() int {
method GetLeases (line 190) | func (c *BootController) GetLeases() *[]Lease {
method GetUnLeased (line 199) | func (c *BootController) GetUnLeased() *[]Lease {
method DelUnLeased (line 208) | func (c *BootController) DelUnLeased(mac string) {
FILE: pkg/services/serverHTTP.go
method generateBootTypeHanders (line 21) | func (c *BootController) generateBootTypeHanders() {
method serveHTTP (line 51) | func (c *BootController) serveHTTP() error {
function rootHandler (line 84) | func rootHandler(w http.ResponseWriter, r *http.Request) {
function preseedHandler (line 94) | func preseedHandler(w http.ResponseWriter, r *http.Request) {
function kickstartHandler (line 101) | func kickstartHandler(w http.ResponseWriter, r *http.Request) {
function vsphereHandler (line 108) | func vsphereHandler(w http.ResponseWriter, r *http.Request) {
function defaultBootHandler (line 115) | func defaultBootHandler(w http.ResponseWriter, r *http.Request) {
function rebootHandler (line 122) | func rebootHandler(w http.ResponseWriter, r *http.Request) {
function autoBootHandler (line 129) | func autoBootHandler(w http.ResponseWriter, r *http.Request) {
function HealthCheckHandler (line 137) | func HealthCheckHandler(w http.ResponseWriter, r *http.Request) {
FILE: pkg/services/serverHTTPISO.go
function iso9660PathSanitiser (line 30) | func iso9660PathSanitiser(unsanitisedPath string) string {
function isoReader (line 83) | func isoReader(w http.ResponseWriter, r *http.Request) {
function OpenISO (line 181) | func OpenISO(isoPath, isoPrefix string) error {
FILE: pkg/services/serverImageHTTP.go
type WriteCounter (line 18) | type WriteCounter struct
method Write (line 24) | func (wc *WriteCounter) Write(p []byte) (int, error) {
function tickerProgress (line 30) | func tickerProgress(byteCounter uint64) {
function imageHandler (line 41) | func imageHandler(w http.ResponseWriter, r *http.Request) {
function configHandler (line 82) | func configHandler(w http.ResponseWriter, r *http.Request) {
method serveImageHTTP (line 87) | func (c *BootController) serveImageHTTP() error {
FILE: pkg/services/serverTFTP.go
function HandleWrite (line 19) | func HandleWrite(filename string) (w io.Writer, err error) {
function HandleRead (line 25) | func HandleRead(filename string) (r io.Reader, err error) {
method serveTFTP (line 31) | func (c *BootController) serveTFTP() error {
FILE: pkg/services/services.go
function findBootConfigForDeployment (line 20) | func findBootConfigForDeployment(deployment DeploymentConfig) *BootConfig {
function findBootConfigForType (line 42) | func findBootConfigForType(ConfigType string) *BootConfig {
method setBootConfig (line 56) | func (c *BootController) setBootConfig(configName, configType, kernel, i...
method StartServices (line 68) | func (c *BootController) StartServices(deployment []byte) error {
FILE: pkg/services/static_pxe.go
constant pxeFile (line 3) | pxeFile = `ea0500c007669c66600fa80fa0061e66684c5245542e8c1621052e6689261...
FILE: pkg/services/templateBOOTy.go
method BuildBOOTYconfig (line 12) | func (config *HostConfig) BuildBOOTYconfig() string {
FILE: pkg/services/templateESXi.go
constant bootcfg67u2 (line 10) | bootcfg67u2 = `bootstate=0
constant modules67us (line 20) | modules67us = `modules=/jumpstrt.gz --- /useropts.gz --- /features.gz --...
constant kickstart67u2 (line 23) | kickstart67u2 = `accepteula
method BuildESXiConfig (line 68) | func (config *HostConfig) BuildESXiConfig() string {
method BuildESXiKickStart (line 75) | func (config *HostConfig) BuildESXiKickStart() string {
FILE: pkg/services/templateKickstart.go
constant kickstartFile (line 6) | kickstartFile = `
method BuildKickStartConfig (line 169) | func (config *HostConfig) BuildKickStartConfig() string {
FILE: pkg/services/templatePreseed.go
constant preseedHead (line 11) | preseedHead = `
constant preseedNet (line 34) | preseedNet = `
constant preseedLVMDisk (line 54) | preseedLVMDisk = `
constant preseedLVMDiskRecipe (line 95) | preseedLVMDiskRecipe = `
constant preseedLVMDiskRecipe2 (line 119) | preseedLVMDiskRecipe2 = `
constant preseedLVMDiskDisableSwap (line 127) | preseedLVMDiskDisableSwap = `
constant preseedDiskAtomic (line 132) | preseedDiskAtomic = `
constant preseedDisk (line 136) | preseedDisk = `
constant swap (line 198) | swap = ` 65536 65536 65536 linux-swap \
constant noswap (line 202) | noswap = `
constant preseedUsers (line 206) | preseedUsers = `
constant preseedPkg (line 219) | preseedPkg = `
constant preseedCmd (line 258) | preseedCmd = `
method BuildPreeSeedConfig (line 270) | func (config *HostConfig) BuildPreeSeedConfig() string {
FILE: pkg/services/templateUtils.go
method ReadKeyFromFile (line 11) | func (c *HostConfig) ReadKeyFromFile() (string, error) {
method parseSSH (line 30) | func (c *HostConfig) parseSSH() error {
method PopulateFromGlobalConfiguration (line 43) | func (c *HostConfig) PopulateFromGlobalConfiguration(globalConfig HostCo...
FILE: pkg/services/types.go
type BootController (line 6) | type BootController struct
type dhcpConfig (line 29) | type dhcpConfig struct
type BootConfig (line 39) | type BootConfig struct
type DeploymentConfigurationFile (line 54) | type DeploymentConfigurationFile struct
type DeploymentConfig (line 60) | type DeploymentConfig struct
type HostConfig (line 68) | type HostConfig struct
FILE: pkg/ssh/sshClient.go
method StartConnection (line 12) | func (c *HostSSHConfig) StartConnection() (*ssh.Client, error) {
method StopConnection (line 29) | func (c *HostSSHConfig) StopConnection() error {
method StartSession (line 37) | func (c *HostSSHConfig) StartSession() (*ssh.Session, error) {
method StopSession (line 51) | func (c *HostSSHConfig) StopSession() {
method String (line 58) | func (c HostSSHConfig) String() string {
function FindHosts (line 63) | func FindHosts(parlayHosts []string) ([]HostSSHConfig, error) {
FILE: pkg/ssh/sshCommand.go
function SingleExecute (line 16) | func SingleExecute(cmd, pipefile, pipecmd string, host HostSSHConfig, to...
function ParalellExecute (line 24) | func ParalellExecute(cmd, pipefile, pipecmd string, hosts []HostSSHConfi...
method ExecuteCmd (line 95) | func (c *HostSSHConfig) ExecuteCmd(cmd string) (string, error) {
method ExecuteCmdWithStdinFile (line 108) | func (c *HostSSHConfig) ExecuteCmdWithStdinFile(cmd, filePath string) (s...
method ExecuteCmdWithStdinCmd (line 165) | func (c *HostSSHConfig) ExecuteCmdWithStdinCmd(cmd, localCmd string) (st...
FILE: pkg/ssh/sshConfig.go
type HostSSHConfig (line 11) | type HostSSHConfig struct
type CommandResult (line 21) | type CommandResult struct
function SetPassword (line 28) | func SetPassword(password string) []ssh.AuthMethod {
FILE: pkg/ssh/sshImport.go
function init (line 27) | func init() {
function AddHost (line 45) | func AddHost(address, keypath, username string) error {
function ImportHostsFromDeployment (line 90) | func ImportHostsFromDeployment(deployment services.DeploymentConfigurati...
function ImportHostsFromRawDeployment (line 167) | func ImportHostsFromRawDeployment(config []byte) error {
function findDefaultKey (line 180) | func findDefaultKey() (ssh.AuthMethod, error) {
function findPrivateKey (line 189) | func findPrivateKey(publicKey string) (ssh.AuthMethod, error) {
function readKeyFile (line 206) | func readKeyFile(keyfile string) (ssh.AuthMethod, error) {
function ReadKeyFiles (line 220) | func ReadKeyFiles(keyFiles []string) ([]ssh.AuthMethod, error) {
FILE: pkg/ssh/sshTransfer.go
function ParalellDownload (line 13) | func ParalellDownload(hosts []HostSSHConfig, source, destination string,...
method DownloadFile (line 66) | func (c HostSSHConfig) DownloadFile(source, destination string) error {
function ParalellUpload (line 107) | func ParalellUpload(hosts []HostSSHConfig, source, destination string, t...
method UploadFile (line 160) | func (c HostSSHConfig) UploadFile(source, destination string) error {
FILE: pkg/utils/ipxe.go
constant iPXEURL (line 13) | iPXEURL = "https://boot.ipxe.org/undionly.kpxe"
constant iPXEHeader (line 16) | iPXEHeader = `#!ipxe
function IPXEReboot (line 40) | func IPXEReboot() string {
function IPXEAutoBoot (line 50) | func IPXEAutoBoot() string {
function IPXEPreeseed (line 60) | func IPXEPreeseed(webserverAddress, kernel, initrd, cmdline string) stri...
function IPXEKickstart (line 73) | func IPXEKickstart(webserverAddress, kernel, initrd, cmdline string) str...
function IPXEVSphere (line 86) | func IPXEVSphere(webserverAddress, kernel, cmdline string) string {
function IPXEBOOTy (line 98) | func IPXEBOOTy(webserverAddress, kernel, initrd, cmdline string) string {
function IPXEAnyBoot (line 111) | func IPXEAnyBoot(webserverAddress string, kernel, initrd, cmdline string...
function PullPXEBooter (line 124) | func PullPXEBooter() error {
FILE: pkg/utils/nic.go
function FindIPAddress (line 9) | func FindIPAddress(addrName string) (string, string, error) {
function FindAllIPAddresses (line 42) | func FindAllIPAddresses() ([]net.IP, error) {
function ConvertIP (line 71) | func ConvertIP(ipAddress string) ([]byte, error) {
FILE: pkg/utils/utils.go
function WaitForCtrlC (line 12) | func WaitForCtrlC() {
function FileToHex (line 26) | func FileToHex(filePath string) (sl string, err error) {
FILE: plugin/docker/docker.go
constant pluginInfo (line 10) | pluginInfo = `This plugin is used to managed docker automation`
type image (line 12) | type image struct
type tag (line 22) | type tag struct
function main (line 34) | func main() {}
function ParlayActionList (line 37) | func ParlayActionList() []string {
function ParlayActionDetails (line 44) | func ParlayActionDetails() []string {
function ParlayPluginInfo (line 51) | func ParlayPluginInfo() string {
function ParlayUsage (line 59) | func ParlayUsage(action string) (raw json.RawMessage, err error) {
function ParlayExec (line 90) | func ParlayExec(action, host string, raw json.RawMessage) (actions []par...
FILE: plugin/docker/docker_actions.go
method generateImageActions (line 10) | func (i *image) generateImageActions(host string) []parlaytypes.Action {
method generateTagActions (line 60) | func (t *tag) generateTagActions(host string) ([]parlaytypes.Action, err...
FILE: plugin/example.go
constant pluginInfo (line 10) | pluginInfo = `This example plugin is used to demonstrate the structure o...
type pluginTestAction (line 13) | type pluginTestAction struct
function main (line 19) | func main() {}
function ParlayActionList (line 22) | func ParlayActionList() []string {
function ParlayActionDetails (line 30) | func ParlayActionDetails() []string {
function ParlayPluginInfo (line 38) | func ParlayPluginInfo() string {
function ParlayActions (line 43) | func ParlayActions(action string, iface interface{}) []parlaytypes.Action {
function ParlayUsage (line 56) | func ParlayUsage(action string) (raw json.RawMessage, err error) {
function ParlayExec (line 79) | func ParlayExec(action, host string, raw json.RawMessage) (actions []par...
FILE: plugin/kubeadm/kubeadm.go
constant pluginInfo (line 10) | pluginInfo = `This plugin is used to managed kubeadm automation`
constant etcdKubeadm (line 13) | etcdKubeadm = `apiVersion: "kubeadm.k8s.io/%s"
constant managerKubeadm (line 32) | managerKubeadm = `apiVersion: kubeadm.k8s.io/v1beta1
type etcdMembers (line 49) | type etcdMembers struct
type managerMembers (line 67) | type managerMembers struct
function main (line 84) | func main() {}
function ParlayActionList (line 87) | func ParlayActionList() []string {
function ParlayActionDetails (line 94) | func ParlayActionDetails() []string {
function ParlayPluginInfo (line 101) | func ParlayPluginInfo() string {
function ParlayUsage (line 109) | func ParlayUsage(action string) (raw json.RawMessage, err error) {
function ParlayExec (line 138) | func ParlayExec(action, host string, raw json.RawMessage) (actions []par...
FILE: plugin/kubeadm/kubeadm_actions.go
method generateActions (line 9) | func (e *etcdMembers) generateActions() []parlaytypes.Action {
method buildKubeadm (line 57) | func (e *etcdMembers) buildKubeadm(api, host, address string) string {
method generateCertificateActions (line 65) | func (e *etcdMembers) generateCertificateActions(hosts []string) []parla...
method generateActions (line 134) | func (m *managerMembers) generateActions() []parlaytypes.Action {
method buildKubeadm (line 172) | func (m *managerMembers) buildKubeadm() string {
Condensed preview — 85 files, each showing path, character count, and a content snippet. Download the .json file or copy for the full structured content (480K chars).
[
{
"path": ".gitignore",
"chars": 48,
"preview": "\n.plunderserver.yaml\nplunder\nplunderclient.yaml\n"
},
{
"path": "Dockerfile",
"chars": 552,
"preview": "# syntax=docker/dockerfile:experimental\n\n# Build BOOTy as an init\nFROM golang:1.14-alpine as dev\nRUN apk add --no-cache "
},
{
"path": "LICENSE",
"chars": 11357,
"preview": " Apache License\n Version 2.0, January 2004\n "
},
{
"path": "Makefile",
"chars": 2362,
"preview": "\nSHELL := /bin/sh\n\n# The name of the executable (default is current directory name)\nTARGET := plunder\n.DEFAULT_GOAL: $(T"
},
{
"path": "Readme.md",
"chars": 4781,
"preview": "\n# Plunder\n\nThe complete tool for finding **Infrastructure** gold amongst bits of bare-metal!\n\n![Plunder Captain](./imag"
},
{
"path": "cmd/automate.go",
"chars": 14515,
"preview": "package cmd\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"net/http\"\n\t\"net/url\"\n\t\"os\"\n\n\t\"github.com/ghodss/yaml\"\n\t\"git"
},
{
"path": "cmd/configGenerator.go",
"chars": 10299,
"preview": "package cmd\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net\"\n\t\"os\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/ghodss/yaml\"\n\tbooty \"github."
},
{
"path": "cmd/plunder.go",
"chars": 2205,
"preview": "package cmd\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"strconv\"\n\n\t\"github.com/plunder-app/plunder/pkg/utils\"\n\n\tlog \"github.com/sirupsen/lo"
},
{
"path": "cmd/server.go",
"chars": 5795,
"preview": "package cmd\n\nimport (\n\t\"io/ioutil\"\n\t\"os\"\n\n\t\"github.com/plunder-app/plunder/pkg/apiserver\"\n\t\"github.com/plunder-app/plund"
},
{
"path": "docs/actions.md",
"chars": 6665,
"preview": "# Actions\n\nWhen a deployment is executed against a host(s) typically one or more **actions** will be performed against "
},
{
"path": "docs/application_architecture.md",
"chars": 2692,
"preview": "# Application Architecture\n\nThe purpose of this document is to outline the architecture of the plunder program itself, a"
},
{
"path": "docs/deployment.md",
"chars": 5623,
"preview": "# Deployment Configuration\n\n## Generating a configuration\nA `./plunder config deployment > deployment.json` will create "
},
{
"path": "docs/example_architecture.md",
"chars": 3394,
"preview": "# Example architecture\n\nThis document outlines an example architecture that one can consider when structuring or designi"
},
{
"path": "docs/example_deployment.md",
"chars": 2948,
"preview": "# Example Deployment for off-line Kubernetes\n\n**This example will make use of Plunders User Interface**\n\nIn order for an"
},
{
"path": "docs/provisioning.md",
"chars": 5282,
"preview": "# Provisioning Configuration\n\nThe provisioning works by running remote commands or uploading/downloading files to a remo"
},
{
"path": "docs/readme.md",
"chars": 1302,
"preview": "# Plunder Usage\n\nWhen using `plunder` there are a few things that you will need to ensure that a configuration exists fo"
},
{
"path": "docs/service.md",
"chars": 3186,
"preview": "# Service Configuration\n\n## Generating a configuration\n\nA `./plunder config server > config.json` will look at the netwo"
},
{
"path": "go.mod",
"chars": 2211,
"preview": "module github.com/plunder-app/plunder\n\ngo 1.12\n\nrequire (\n\tgithub.com/AlecAivazis/survey/v2 v2.0.7 // indirect\n\tgithub.c"
},
{
"path": "go.sum",
"chars": 25626,
"preview": "cloud.google.com/go v0.26.0/go.mod h1:aQUYkXzVsufM+DwF1aE+0xfcU+56JwCaLick0ClmMTw=\ngithub.com/AlecAivazis/survey/v2 v2.0"
},
{
"path": "hack/comboot_ipxe/Dockerfile",
"chars": 244,
"preview": "FROM gcc:latest AS IPXE_BUILD\nRUN git clone git://git.ipxe.org/ipxe.git\nRUN sed -i '/COMBOOT/s/\\/\\///g' ipxe/src/config/"
},
{
"path": "hack/comboot_ipxe/gen_comboot.ipxe",
"chars": 216,
"preview": "#!/bin/bash\necho Building latest container for iPXE, with comboot support\ncd comboot_ipxe\ndocker build -t ipxe_comboot ."
},
{
"path": "main.go",
"chars": 291,
"preview": "package main\n\nimport \"github.com/plunder-app/plunder/cmd\"\n\n// Version is populated from the Makefile and is tied to the "
},
{
"path": "pkg/apiserver/README.md",
"chars": 1026,
"preview": "# API Server documentation\n\nThis documentation is a quick overview of the CRUD operations that take place within `plunde"
},
{
"path": "pkg/apiserver/client.go",
"chars": 4507,
"preview": "package apiserver\n\nimport (\n\t\"bytes\"\n\t\"crypto/tls\"\n\t\"crypto/x509\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"net/http\"\n\t\"net"
},
{
"path": "pkg/apiserver/config.go",
"chars": 4102,
"preview": "package apiserver\n\nimport (\n\t\"encoding/base64\"\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"net/url\"\n\n\t\"github.com/ghodss/yaml"
},
{
"path": "pkg/apiserver/endpoints.go",
"chars": 1797,
"preview": "package apiserver\n\nimport (\n\t\"net/http\"\n\n\tlog \"github.com/sirupsen/logrus\"\n\t//\"github.com/gorilla/mux\"\n)\n\n// EndPointMan"
},
{
"path": "pkg/apiserver/go.mod",
"chars": 61,
"preview": "module github.com/plunder-app/plunder/pkg/apiserver\n\ngo 1.12\n"
},
{
"path": "pkg/apiserver/handlerApiserver.go",
"chars": 1329,
"preview": "package apiserver\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"net/http\"\n\n\t\"github.com/gorilla/mux\"\n)\n\n// Delete the parlay resul"
},
{
"path": "pkg/apiserver/logging.go",
"chars": 3714,
"preview": "package apiserver\n\nimport (\n\t\"fmt\"\n\t\"net/http\"\n\t\"sync\"\n\n\t\"github.com/gorilla/mux\"\n)\n\n// MVP of a streaming logging provi"
},
{
"path": "pkg/apiserver/loggingHandlers.go",
"chars": 38,
"preview": "package apiserver\n\n//var map parlay[]\n"
},
{
"path": "pkg/apiserver/server.go",
"chars": 2134,
"preview": "package apiserver\n\nimport (\n\t\"crypto/tls\"\n\t\"fmt\"\n\t\"net/http\"\n\n\t\"github.com/gorilla/mux\"\n\n\tlog \"github.com/sirupsen/logru"
},
{
"path": "pkg/apiserver/types.go",
"chars": 445,
"preview": "package apiserver\n\nimport \"encoding/json\"\n\n//Response - This is the wrapper for responses back to a client, if any error"
},
{
"path": "pkg/certs/certs.go",
"chars": 3674,
"preview": "package certs\n\n// generate-tls-cert generates root, leaf, and client TLS certificates.\n\nimport (\n\t\"bytes\"\n\t\"crypto/rand\""
},
{
"path": "pkg/certs/go.mod",
"chars": 57,
"preview": "module github.com/plunder-app/plunder/pkg/certs\n\ngo 1.12\n"
},
{
"path": "pkg/go.mod",
"chars": 51,
"preview": "module github.com/plunder-app/plunder/pkg\n\ngo 1.12\n"
},
{
"path": "pkg/parlay/go.mod",
"chars": 58,
"preview": "module github.com/plunder-app/plunder/pkg/parlay\n\ngo 1.12\n"
},
{
"path": "pkg/parlay/handler.go",
"chars": 3870,
"preview": "package parlay\n\nimport (\n\t\"encoding/json\"\n\t\"io/ioutil\"\n\t\"net/http\"\n\t\"strings\"\n\n\t\"github.com/gorilla/mux\"\n\t\"github.com/pl"
},
{
"path": "pkg/parlay/parlay.go",
"chars": 442,
"preview": "package parlay\n\ntype actionType string\n\nconst (\n\t//upload - defines that this action will upload a file to a remote syst"
},
{
"path": "pkg/parlay/parlay_ui.go",
"chars": 2218,
"preview": "package parlay\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\n\t\"github.com/AlecAivazis/survey/v2\"\n\t\"github.com/plunder-app/plunder/pkg/par"
},
{
"path": "pkg/parlay/parlaytypes/finder.go",
"chars": 4101,
"preview": "package parlaytypes\n\nimport (\n\t\"fmt\"\n\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// FindDeployments - This will iterate throug"
},
{
"path": "pkg/parlay/parlaytypes/go.mod",
"chars": 70,
"preview": "module github.com/plunder-app/plunder/pkg/parlay/parlaytypes\n\ngo 1.12\n"
},
{
"path": "pkg/parlay/parlaytypes/parlaytypes.go",
"chars": 2257,
"preview": "package parlaytypes\n\nimport (\n\t\"encoding/json\"\n)\n\n// TreasureMap - X Marks the spot\n// The treasure maps define the auto"
},
{
"path": "pkg/parlay/parser.go",
"chars": 14255,
"preview": "package parlay\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/plunder-app/plunder/pkg/plunderlogging\"\n\n\tlog \"github.com/sirupsen/logrus\""
},
{
"path": "pkg/parlay/parser_builder.go",
"chars": 2352,
"preview": "package parlay\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"os/exec\"\n\t\"strings\"\n\n\t\"github.com/plunder-app/plunder/pkg/parlay/parlaytypes\"\n\t\""
},
{
"path": "pkg/parlay/plugin/plugin.go",
"chars": 5280,
"preview": "package parlayplugin\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"plugin\"\n\n\t\"github.com/plunder-app/plunde"
},
{
"path": "pkg/parlay/restore.go",
"chars": 1627,
"preview": "package parlay\n\nimport (\n\t\"encoding/json\"\n\t\"io/ioutil\"\n\t\"os\"\n\n\t\"github.com/mitchellh/go-homedir\"\n)\n\n//Restore provides a"
},
{
"path": "pkg/parlay/validate.go",
"chars": 1957,
"preview": "package parlay\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/plunder-app/plunder/pkg/parlay/parlaytypes\"\n)\n\n// ValidateAction will pars"
},
{
"path": "pkg/plunderlogging/consolelogger.go",
"chars": 23,
"preview": "package plunderlogging\n"
},
{
"path": "pkg/plunderlogging/filelogger.go",
"chars": 1013,
"preview": "package plunderlogging\n\nimport (\n\t\"fmt\"\n\t\"os\"\n\t\"sync\"\n)\n\n// FileLogger allows parlay to log output to a file on the loca"
},
{
"path": "pkg/plunderlogging/go.mod",
"chars": 66,
"preview": "module github.com/plunder-app/plunder/pkg/plunderlogging\n\ngo 1.12\n"
},
{
"path": "pkg/plunderlogging/jsonlogger.go",
"chars": 2090,
"preview": "package plunderlogging\n\nimport (\n\t\"fmt\"\n\t\"time\"\n\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// JSONLogger allows parlay to log"
},
{
"path": "pkg/plunderlogging/logger.go",
"chars": 2195,
"preview": "package plunderlogging\n\nimport \"fmt\"\n\n// Logger - is a stuct that manages the verious types of logger available\ntype Log"
},
{
"path": "pkg/services/deployments.go",
"chars": 13415,
"preview": "package services\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"strings\"\n\n\t\"github.com/plunder-app/plunder/pkg/utils\"\n\tlog \"github."
},
{
"path": "pkg/services/go.mod",
"chars": 60,
"preview": "module github.com/plunder-app/plunder/pkg/services\n\ngo 1.12\n"
},
{
"path": "pkg/services/handler.go",
"chars": 12581,
"preview": "package services\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"net/http\"\n\t\"strings\"\n\n\t\"github.com/gorilla/mux\"\n\t\"gith"
},
{
"path": "pkg/services/server.go",
"chars": 5016,
"preview": "package services\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\n\t\"github.com/ghodss/yaml\"\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// Th"
},
{
"path": "pkg/services/serverDHCP.go",
"chars": 7079,
"preview": "package services\n\nimport (\n\t\"fmt\"\n\t\"math/rand\"\n\t\"net\"\n\t\"strings\"\n\t\"time\"\n\n\tdhcp \"github.com/krolaw/dhcp4\"\n\tlog \"github.c"
},
{
"path": "pkg/services/serverHTTP.go",
"chars": 4596,
"preview": "package services\n\nimport (\n\t\"io\"\n\t\"net/http\"\n\t\"path/filepath\"\n\n\t\"github.com/plunder-app/plunder/pkg/utils\"\n\tlog \"github."
},
{
"path": "pkg/services/serverHTTPISO.go",
"chars": 5724,
"preview": "package services\n\nimport (\n\t\"bytes\"\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"net/url\"\n\t\"os\"\n\t\"path/filepath\"\n\t\"strings\"\n\n\t\"github.com/"
},
{
"path": "pkg/services/serverImageHTTP.go",
"chars": 2395,
"preview": "package services\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"os\"\n\t\"strings\"\n\t\"time\"\n\n\tlog \"github.com/sirupsen/logrus\"\n\n\t\"githu"
},
{
"path": "pkg/services/serverTFTP.go",
"chars": 1179,
"preview": "package services\n\nimport (\n\t\"bufio\"\n\t\"bytes\"\n\t\"encoding/hex\"\n\t\"errors\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"os\"\n\n\tlog \"github.com/sirups"
},
{
"path": "pkg/services/services.go",
"chars": 6246,
"preview": "package services\n\nimport (\n\t\"net/http\"\n\t\"time\"\n\n\t\"github.com/plunder-app/plunder/pkg/utils\"\n\tlog \"github.com/sirupsen/lo"
},
{
"path": "pkg/services/static_pxe.go",
"chars": 135149,
"preview": "package services\n\nconst pxeFile = `ea0500c007669c66600fa80fa0061e66684c5245542e8c1621052e6689261d058cc88ed8b840008ee08cc"
},
{
"path": "pkg/services/templateBOOTy.go",
"chars": 1503,
"preview": "package services\n\nimport (\n\t\"encoding/json\"\n\t\"net\"\n\n\t\"github.com/plunder-app/BOOTy/pkg/plunderclient/types\"\n\t\"github.com"
},
{
"path": "pkg/services/templateESXi.go",
"chars": 5268,
"preview": "package services\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n)\n\n// bootcfgHead const, this is the basis for the configuration that will "
},
{
"path": "pkg/services/templateKickstart.go",
"chars": 3969,
"preview": "package services\n\nimport \"fmt\"\n\n// This initial template will be modifiable based upon the build requirements\nconst kick"
},
{
"path": "pkg/services/templatePreseed.go",
"chars": 10970,
"preview": "package services\n\nimport (\n\t\"encoding/base64\"\n\t\"fmt\"\n\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// Preseed const, this is the"
},
{
"path": "pkg/services/templateUtils.go",
"chars": 3713,
"preview": "package services\n\nimport (\n\t\"encoding/base64\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"strings\"\n)\n\n// ReadKeyFromFile - will attempt to read"
},
{
"path": "pkg/services/types.go",
"chars": 4772,
"preview": "package services\n\n// TYPE DEFINITIONS Below\n\n// BootController contains the settings that define how the remote boot wil"
},
{
"path": "pkg/ssh/go.mod",
"chars": 55,
"preview": "module github.com/plunder-app/plunder/pkg/ssh\n\ngo 1.12\n"
},
{
"path": "pkg/ssh/sshClient.go",
"chars": 1762,
"preview": "package ssh\n\nimport (\n\t\"fmt\"\n\t\"strings\"\n\n\tlog \"github.com/sirupsen/logrus\"\n\t\"golang.org/x/crypto/ssh\"\n)\n\n// StartConnect"
},
{
"path": "pkg/ssh/sshCommand.go",
"chars": 5116,
"preview": "package ssh\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"os/exec\"\n\t\"path/filepath\"\n\t\"time\"\n\n\tlog \"github.com/sirupsen/log"
},
{
"path": "pkg/ssh/sshConfig.go",
"chars": 774,
"preview": "package ssh\n\nimport (\n\t\"golang.org/x/crypto/ssh\"\n)\n\n// Hosts - The array of all hosts once loaded\nvar Hosts []HostSSHCon"
},
{
"path": "pkg/ssh/sshImport.go",
"chars": 6312,
"preview": "package ssh\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\t\"io/ioutil\"\n\t\"os/user\"\n\t\"path/filepath\"\n\t\"strings\"\n\t\"time\"\n\n\t\"github.com/"
},
{
"path": "pkg/ssh/sshTransfer.go",
"chars": 4296,
"preview": "package ssh\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"os\"\n\t\"time\"\n\n\t\"github.com/pkg/sftp\"\n)\n\n// ParalellDownload - Allow downloading a fi"
},
{
"path": "pkg/utils/go.mod",
"chars": 57,
"preview": "module github.com/plunder-app/plunder/pkg/utils\n\ngo 1.12\n"
},
{
"path": "pkg/utils/ipxe.go",
"chars": 3814,
"preview": "package utils\n\nimport (\n\t\"fmt\"\n\t\"io\"\n\t\"net/http\"\n\t\"os\"\n\n\tlog \"github.com/sirupsen/logrus\"\n)\n\n// Static URL for retrievin"
},
{
"path": "pkg/utils/nic.go",
"chars": 1787,
"preview": "package utils\n\nimport (\n\t\"fmt\"\n\t\"net\"\n)\n\n// FindIPAddress - this will find the address associated with an adapter\nfunc F"
},
{
"path": "pkg/utils/utils.go",
"chars": 673,
"preview": "package utils\n\nimport (\n\t\"encoding/hex\"\n\t\"io/ioutil\"\n\t\"os\"\n\t\"os/signal\"\n\t\"sync\"\n)\n\n//WaitForCtrlC - This function is the"
},
{
"path": "plugin/docker/docker.go",
"chars": 3429,
"preview": "package main\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\n\t\"github.com/plunder-app/plunder/pkg/parlay/parlaytypes\"\n)\n\nconst plugin"
},
{
"path": "plugin/docker/docker_actions.go",
"chars": 2579,
"preview": "package main\n\nimport (\n\t\"fmt\"\n\t\"path\"\n\n\t\"github.com/plunder-app/plunder/pkg/parlay/parlaytypes\"\n)\n\nfunc (i *image) gener"
},
{
"path": "plugin/example.go",
"chars": 3240,
"preview": "package main\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\n\t\"github.com/plunder-app/plunder/pkg/parlay/parlaytypes\"\n)\n\nconst plugin"
},
{
"path": "plugin/kubeadm/kubeadm.go",
"chars": 4795,
"preview": "package main\n\nimport (\n\t\"encoding/json\"\n\t\"fmt\"\n\n\t\"github.com/plunder-app/plunder/pkg/parlay/parlaytypes\"\n)\n\nconst plugin"
},
{
"path": "plugin/kubeadm/kubeadm_actions.go",
"chars": 7233,
"preview": "package main\n\nimport (\n\t\"fmt\"\n\n\t\"github.com/plunder-app/plunder/pkg/parlay/parlaytypes\"\n)\n\nfunc (e *etcdMembers) generat"
},
{
"path": "testing.sh",
"chars": 3988,
"preview": "#!/bin/bash\n\necho \"This script will step through a number of tests agains plunder to ensure that functionality is as exp"
}
]
About this extraction
This page contains the full source code of the plunder-app/plunder GitHub repository, extracted and formatted as plain text for AI agents and large language models (LLMs). The extraction includes 85 files (441.4 KB), approximately 168.4k tokens, and a symbol index with 285 extracted functions, classes, methods, constants, and types. Use this with OpenClaw, Claude, ChatGPT, Cursor, Windsurf, or any other AI tool that accepts text input. You can copy the full output to your clipboard or download it as a .txt file.
Extracted by GitExtract — free GitHub repo to text converter for AI. Built by Nikandr Surkov.