Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dev #3

Merged
merged 7 commits into from
May 14, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Go parameters
GOCMD=go
GORUN=$(GOCMD) run
GOBUILD=$(GOCMD) build
GOTEST=$(GOCMD) test
GOINSTALL=$(GOCMD) install

build: test
$(GOBUILD) -o packman -v cmd/packman/main.go
run:
$(GORUN) .
test:
$(GOTEST) -v ./...

install: build
$(GOINSTALL) -v .
137 changes: 137 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# Packman
Scaffolding was never that easy...

## Motivation
At SecureNative, we manage lots of microservices and the job of creating a new
project, wiring it up, importing our common libs is a tedious job and should be automated :)

packman was created to tackle this issue, as there are other good scaffolding tools (such as Yeoman), we've just wanted a simple tool that
works simply enough for anyone to use.

## Prerequisites
- Go 1.11 or above with [Go Modules enabled](https://github.com/golang/go/wiki/Modules#how-to-use-modules)
- Basic knowledge of [Go's templating engine](https://curtisvermeeren.github.io/2017/09/14/Golang-Templates-Cheatsheet)
- Git
- A Github account token (only if you want to publish new packages)

## Quick Example
First, lets install packman, assuming you've installed go correctly just download the binary from our release page.
In order to start a new template you may use packman's init, which generates the template seed:
```bash
$> packman init my-app-template
```
You'll see that a new folder is created for your template (named `my-app-template` obviously),
inside that folder you'll find another folder called `packman` which contains our scaffolding script (more about that later)

Now, lets create a simple file in the root folder:
```bash
$> echo "Hello {{{ .PackageName }}}" > README.md
```
Lets check how the rendered version of our newly created template will look like by running:
```bash
$> packman render my-app-template my-app-template my-app-template-rendered
$> cat my-app-template-rendered/README.md
Hello my-app-template
```

Wow, the `{{{ .PackageName }}}` placeholder was replaced with our package name, miracles do exists :)

Lets assume that we are happy with our template and we want to publish it so other users can use as well, Packman uses Github as the package registry so lets configure our github account:
```bash
$> packman config github --username matang28 --token <GITHUB_TOKEN> --private
```

Now we are ready to push our template to Github by just doing:
```bash
$> packman pack securenative/pm-template my-app-template
```
![](docs/pack_github.png)

And Voila! we just created our first template and pushed it to Github, Now anyone can pull it and use our template for its own use by just doing:
```bash
$> packman unpack securenative/pm-template my-app
$> cat my-app/README.md
Hello securenative/pm-template
```

That's it! now, with the help of the `packman` you can easily create your project template, and render it based on the data generated from your script file.

## How it works
If you read and followed the `Quick Example` you may have many questions about packman, we'll try to answer them now.
Understanding how packman works is crucial if you want to use it, but first lets define following:
- **Project Template** - this is the un-rendered version of your project, will contain the template files and the activation script.
- **Activation Script** - this script will be invoked by packman when calling `render`/`unpack`, the flags you give to these commands will be forwarded to the script file.
The responsibility of this script is to create the data model that can be queried by Go's templating directives. (`{{{ .PackageName }}}` for example)

Packman uses a simple approach to render your project, at first packman will run you **Activation Script**, Let's examine the simplest form of an **Activation Script**
```go
package main

import (
"os"
pm "github.com/securenative/packman/pkg"
)

type MyData struct {
PackageName string
ProjectPath string
Flags map[string]string
}

func main() {
// Parse the flags being forwarded from packman commands:
// This is how we can use custom flags
flags := pm.ParseFlags(os.Args[2:])

/**
YOUR CUSTOM LOGIC
**/

// The next step is to build our data model, the data model will be used by
// the template directives.
model := MyData{
PackageName: flags[pm.PackageNameFlag], // Here we can see that {{{ .PackageName }}} refers to this field
ProjectPath: flags[pm.PackagePathFlag],
Flags: flags,
}

// You must reply your data model back to the packman's driver
pm.Reply(model)
}
```

Next, packman will go through your project tree and render each file using Go's template engine and the data model provided by your **Activation Script**, and ... That's it!

## API

### New Project
You can use packman to create a basic seed project which contains the basic structure of a packman template.
```bash
packman init <folder-name>
```

### Render
As the **Template Project** grows you'll need a way to quickly check that your project is rendered correctly,
The render will take the path of your **Template Project** and the path to the rendered output and any custom flags you wish to forward to your **Activation Script**.
```bash
packman render <package-name> <path-to-template> <path-to-output> -customflag1 value1 -customflag2 value2 ...
```

## Unpack
Unpack is the "wet" version of render, the only difference is that unpack will pull a template from the remote storage instead of you local file system.
```bash
packman unpack <package-name> <path-to-output> -customflag1 value1 -customflag2 value2 ...
```

## Pack
Pack will take a **Template Project** and will push it to the remote storage so others can use it.
```bash
packman pack <package-name> <path-to-template>
```

## Configuration
Packman supports local persistent kv-storage so you can store any kind of configurations with it, but currently only github configuration is supported.
```bash
packman config github --username <github-username> --token <github-personal-token> [--private]
```
The `--private` flag will indicate that we will pack the template as a private github repository instead of a public one.
Binary file added docs/pack_github.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 2 additions & 0 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ go 1.12

require (
github.com/google/go-github/v25 v25.0.2
github.com/logrusorgru/aurora v0.0.0-20190428105938-cea283e61946 // indirect
github.com/mingrammer/cfmt v1.1.0
github.com/otiai10/copy v1.0.1
github.com/otiai10/curr v0.0.0-20150429015615-9b4961190c95 // indirect
github.com/stretchr/testify v1.3.0
Expand Down
16 changes: 15 additions & 1 deletion internal/business/git_project_init.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package business

import (
"github.com/mingrammer/cfmt"
"io/ioutil"
"os"
"os/exec"
Expand All @@ -15,38 +16,51 @@ func NewGitProjectInit() *GitProjectInit {

func (this *GitProjectInit) Init(destPath string) error {
path := packmanPath(destPath)

cfmt.Info("Creating path ", path, "\n")
if err := os.MkdirAll(path, os.ModePerm); err != nil {
cfmt.Error("Cannot create the following path: ", path, ", ", err.Error(), "\n")
return err
}

cfmt.Info("Writing the main.go script file", "\n")
scriptPath := filepath.Join(path, "main.go")
if err := this.write(scriptPath, replyScript); err != nil {
cfmt.Error("Cannot create ", scriptPath, ", ", err.Error(), "\n")
return err
}

cfmt.Info("Writing the go.mod file", "\n")
modPath := filepath.Join(path, "go.mod")
if err := this.write(modPath, modeFile); err != nil {
cfmt.Error("Cannot create ", scriptPath, ", ", err.Error(), "\n")
return err
}

cfmt.Info("Initialing the git repository", "\n")
gitInit := exec.Command("git", "init")
gitInit.Dir = destPath
if err := gitInit.Run(); err != nil {
cfmt.Error("Cannot init git repository, ", err.Error(), "\n")
return err
}

gitAdd := exec.Command("git", "add", ".")
gitAdd.Dir = destPath
if err := gitAdd.Run(); err != nil {
cfmt.Error("Failed to add untracked files to the git repository, ", err.Error(), "\n")
return err
}

cfmt.Info("Creating the first commit", "\n")
gitCommit := exec.Command("git", "commit", "-m", `"First Commit"`)
gitCommit.Dir = destPath
if err := gitCommit.Run(); err != nil {
cfmt.Error("Failed to commit changes, ", err.Error(), "\n")
return err
}

cfmt.Success("Packman package created successfully!")
return nil
}

Expand All @@ -69,7 +83,7 @@ type MyData struct {

func main() {
// flags sent by packman's driver will be forwarded to here:
flags := ParseFlags(os.Args[2:])
flags := pm.ParseFlags(os.Args[3:])

// Build your own model to represent the templating you need
model := MyData{
Expand Down
24 changes: 23 additions & 1 deletion internal/business/unpacker.go
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
package business

import (
"github.com/mingrammer/cfmt"
. "github.com/otiai10/copy"
"github.com/securenative/packman/internal/data"
"github.com/securenative/packman/pkg"
Expand Down Expand Up @@ -86,7 +87,8 @@ func (this *PackmanUnpacker) render(destPath string, args []string) error {
}

err = filepath.Walk(destPath, func(path string, info os.FileInfo, err error) error {
if !info.IsDir() && !strings.Contains(path, ".git") && !strings.Contains(path, "packman") {
if !info.IsDir() && shouldRender(path) {
cfmt.Infof("Rendering %s\n", path)
content, err := ioutil.ReadFile(path)
if err != nil {
return err
Expand All @@ -109,3 +111,23 @@ func (this *PackmanUnpacker) render(destPath string, args []string) error {

return nil
}

func shouldRender(path string) bool {
if strings.Contains(path, ".git") {
return false
}

if strings.Contains(path, ".idea") {
return false
}

if strings.Contains(path, ".vscode") {
return false
}

if strings.Contains(path, "packman") {
return false
}

return true
}
14 changes: 8 additions & 6 deletions internal/business/unpacker_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package business

import (
"github.com/securenative/packman/internal/data"
"github.com/securenative/packman/pkg"
"github.com/stretchr/testify/assert"
"io/ioutil"
"os"
Expand All @@ -11,18 +12,19 @@ import (
)

const expected = `
package my_pkg
package my-pkg
func main() {
fmt.Println("Hello")
fmt.Println("World")
fmt.Println("my-pkg")
}`

func TestPackmanUnpacker_Unpack(t *testing.T) {
replacer := strings.NewReplacer("\n", "", "\t", "")
unpacker := NewPackmanUnpacker(&mockBackend{}, data.NewGoTemplateEngine(), data.NewGoScriptEngine())

path := filepath.Join(os.TempDir(), "packtest", "unpack")
err := unpacker.Unpack("my-pkg", path, []string{"Hello", "World"})
err := unpacker.Unpack("my-pkg", path, []string{"--", pkg.PackageNameFlag, "my-pkg", "-a", "Hello", "-b", "World"})
assert.Nil(t, err)

bytes, err := ioutil.ReadFile(filepath.Join(path, "testfile.go"))
Expand All @@ -47,12 +49,12 @@ func (mockBackend) Pull(name string, destination string) error {
}

content := `
package {{ .PackageName }}
package {{{ .PackageName }}}

func main() {
{{ range .Args }}
fmt.Println("{{ . }}")
{{ end }}
{{{ range .Flags }}}
fmt.Println("{{{ . }}}")
{{{ end }}}
}
`
if err := ioutil.WriteFile(filepath.Join(destination, "testfile.go"), []byte(content), os.ModePerm); err != nil {
Expand Down
5 changes: 4 additions & 1 deletion internal/data/go_script_engine.go
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,11 @@ func (this *GoScriptEngine) Run(scriptFile string, args []string) error {
cmdArgs = append(cmdArgs, "--")
cmdArgs = append(cmdArgs, args...)

fmt.Println(fmt.Sprintf("Running main.go with %v", cmdArgs))

cmd := exec.Command("go", cmdArgs...)
result, err := cmd.Output()
//cmd.Dir = filepath.Dir(scriptFile)
result, err := cmd.CombinedOutput()
if err != nil {
return err
}
Expand Down
2 changes: 1 addition & 1 deletion internal/data/go_template_engine.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ func NewGoTemplateEngine() *GoTemplateEngine {
func (this *GoTemplateEngine) Render(templateText string, data interface{}) (string, error) {
var out bytes.Buffer
t := template.New("template")

t.Delims("{{{", "}}}")
tree, err := t.Parse(templateText)
if err != nil {
return "", err
Expand Down
4 changes: 2 additions & 2 deletions internal/data/go_template_engine_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import (

func TestGoTemplateEngine_Render(t *testing.T) {

template := `Helo {{ .Name }} this is test num: {{ .Number }}`
template := `Helo {{{ .Name }}} this is test num: {{{ .Number }}}`
expected := `Helo World this is test num: 1`
type data struct {
Name string
Expand All @@ -23,7 +23,7 @@ func TestGoTemplateEngine_Render(t *testing.T) {

func TestGoTemplateEngine_Render_Missing_Arg(t *testing.T) {

template := `Helo {{ .Name }} this is test num: {{ .Number }}`
template := `Helo {{{ .Name }}} this is test num: {{{ .Number }}}`
type data struct {
Name string
}
Expand Down