wait.query and wait.port resources

CHANGELOG.md

@@ -4,6 +4,8 @@
 
 ### Enhancements
 
+- Ability to wait for a condition (#238)
+
 ### Bug fixes
 
 - Fix #225 pipeline function refactor (#307)
@@ -46,4 +48,4 @@
 - Docker Swarm mode (#267)
 - ELK (Elasticsearch, Logstash, and Kibana) stack (#272)
 - Add CodeClimate Checks to build (#281)
-- docs: add draft resource authors guide (#290)
+- docs: add draft resource authors guide (#290)

docs/index.xml

@@ -333,7 +333,7 @@ failure.)</p>
 </ul>
 
 <p>the amount of time the command will wait before halting forcefully. The
-format is Go’s duraction string. A duration string is a possibly signed
+format is Go’s duration string. A duration string is a possibly signed
 sequence of decimal numbers, each with optional fraction and a unit
 suffix, such as “300ms”, “-1.5h” or “2h45m”. Valid time units are “ns”,
 “us” (or “µs”), “ms”, “s”, “m”, “h”.</p>

docs/resources/index.xml

@@ -191,7 +191,7 @@ failure.)</p>
 </ul>
 
 <p>the amount of time the command will wait before halting forcefully. The
-format is Go’s duraction string. A duration string is a possibly signed
+format is Go’s duration string. A duration string is a possibly signed
 sequence of decimal numbers, each with optional fraction and a unit
 suffix, such as “300ms”, “-1.5h” or “2h45m”. Valid time units are “ns”,
 “us” (or “µs”), “ms”, “s”, “m”, “h”.</p>

docs/resources/task/index.html

@@ -574,7 +574,7 @@ <h2 id="parameters">Parameters</h2>
 </ul>
 
 <p>the amount of time the command will wait before halting forcefully. The
-format is Go&rsquo;s duraction string. A duration string is a possibly signed
+format is Go&rsquo;s duration string. A duration string is a possibly signed
 sequence of decimal numbers, each with optional fraction and a unit
 suffix, such as &ldquo;300ms&rdquo;, &ldquo;-1.5h&rdquo; or &ldquo;2h45m&rdquo;. Valid time units are &ldquo;ns&rdquo;,
 &ldquo;us&rdquo; (or &ldquo;µs&rdquo;), &ldquo;ms&rdquo;, &ldquo;s&rdquo;, &ldquo;m&rdquo;, &ldquo;h&rdquo;.</p>

docs_source/content/resources/task.md

@@ -62,7 +62,7 @@ failure.)
 - `timeout` (duration string)
 
   the amount of time the command will wait before halting forcefully. The
-format is Go's duraction string. A duration string is a possibly signed
+format is Go's duration string. A duration string is a possibly signed
 sequence of decimal numbers, each with optional fraction and a unit
 suffix, such as "300ms", "-1.5h" or "2h45m". Valid time units are "ns",
 "us" (or "µs"), "ms", "s", "m", "h".

docs_source/content/resources/wait.port.md

@@ -0,0 +1,60 @@
+---
+title: "wait.port"
+slug: "wait-port"
+date: "2016-10-03T10:23:34-04:00"
+menu:
+  main:
+    parent: resources
+---
+
+
+
+
+## Example
+
+```hcl
+wait.port "8080" {
+  host         = "localhost"
+  port         = 8080
+  interval     = "1s"
+  max_retry    = 10
+  grace_period = "2s"
+}
+
+```
+
+
+## Parameters
+
+- `host` (string)
+
+  a host name or ip address. A TCP connection will be attempted at this host
+and the specified Port.
+
+- `port` (required int)
+
+  the TCP port to attempt to connect to.
+
+- `interval` (duration string)
+
+  the amount of time to wait in between checks. The format is Go's duration
+string. A duration string is a possibly signed sequence of decimal numbers,
+each with optional fraction and a unit suffix, such as "300ms", "-1.5h" or
+"2h45m". Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". If
+the interval is not specified, it will default to 5 seconds.
+
+- `grace_period` (duration string)
+
+  the amount of time to wait before running the first check and after a
+successful check. The format is Go's duration string. A duration string is
+a possibly signed sequence of decimal numbers, each with optional fraction
+and a unit suffix, such as "300ms", "-1.5h" or "2h45m". Valid time units
+are "ns", "us" (or "µs"), "ms", "s", "m", "h". If no grace period is
+specified, no grace period will be taken into account.
+
+- `max_retry` (int)
+
+  the maximum number of attempts before the wait fails. If the maximum number
+of retries is not set, it will default to 5.
+
+

docs_source/content/resources/wait.query.md

@@ -0,0 +1,85 @@
+---
+title: "wait.query"
+slug: "wait-query"
+date: "2016-10-03T10:23:34-04:00"
+menu:
+  main:
+    parent: resources
+---
+
+
+
+
+## Example
+
+```hcl
+wait.query "service-health" {
+  check        = "nc -z localhost 8080"
+  interval     = "1s"
+  max_retry    = 10
+  grace_period = "1s"
+}
+
+```
+
+
+## Parameters
+
+- `interpreter` (string)
+
+  the shell interpreter that will be used for your scripts. `/bin/sh` is
+used by default.
+
+- `check` (required string)
+
+  the script to run to check if a resource is ready. exit with exit code 0 if
+the resource is healthy, and 1 (or above) otherwise.
+
+- `check_flags` (list of strings)
+
+  flags to pass to the `interpreter` binary to check validity. For
+`/bin/sh` this is `-n`.
+
+- `exec_flags` (list of strings)
+
+  flags to pass to the interpreter at execution time.
+
+- `timeout` (duration string)
+
+  the amount of time the command will wait before halting forcefully. The
+format is Go's duration string. A duration string is a possibly signed
+sequence of decimal numbers, each with optional fraction and a unit
+suffix, such as "300ms", "-1.5h" or "2h45m". Valid time units are "ns",
+"us" (or "µs"), "ms", "s", "m", "h".
+
+- `dir` (string)
+
+  the working directory this command should be run in.
+
+- `env` (map of string to string)
+
+  any environment variables that should be passed to the command.
+
+- `interval` (duration string)
+
+  the amount of time to wait in between checks. The format is Go's duration
+string. A duration string is a possibly signed sequence of decimal numbers,
+each with optional fraction and a unit suffix, such as "300ms", "-1.5h" or
+"2h45m". Valid time units are "ns", "us" (or "µs"), "ms", "s", "m", "h". If
+the interval is not specified, it will default to 5 seconds.
+
+- `grace_period` (duration string)
+
+  the amount of time to wait before running the first check and after a
+successful check. The format is Go's duration string. A duration string is
+a possibly signed sequence of decimal numbers, each with optional fraction
+and a unit suffix, such as "300ms", "-1.5h" or "2h45m". Valid time units
+are "ns", "us" (or "µs"), "ms", "s", "m", "h". If no grace period is
+specified, no grace period will be taken into account.
+
+- `max_retry` (int)
+
+  the maximum number of attempts before the wait fails. If the maximum number
+of retries is not set, it will default to 5.
+
+

docs_source/sources.csv

@@ -9,3 +9,5 @@ task,../resource/shell/preparer.go,../samples/basic.hcl,Preparer
 task.query,../resource/shell/query/preparer.go,../samples/query.hcl,Preparer
 user.group,../resource/group/preparer.go,../samples/group.hcl,Preparer
 user.user,../resource/user/preparer.go,../samples/user.hcl,Preparer
+wait.query,../resource/wait/preparer.go,../samples/wait.hcl,Preparer
+wait.port,../resource/wait/port/preparer.go,../samples/waitPort.hcl,Preparer

examples/elk/Vagrantfile

@@ -1,7 +1,7 @@
 # -*- mode: ruby -*-
 # vi: set ft=ruby :
 
-converge_version = "0.2.0"
+converge_version = "0.3.0"
 release_url = "https://github.com/asteris-llc/converge/releases/download/#{converge_version}/converge_#{converge_version}_linux_amd64.tar.gz"
 
 converge_script = <<SCRIPT

examples/elk/converge/elk.hcl

@@ -8,6 +8,7 @@ module "docker.hcl" "docker" {
   params = {
     user-name = "{{param `user-name`}}"
   }
+
   depends = ["module.packages"]
 }
 
@@ -50,26 +51,22 @@ task "filebeat-enable" {
   depends = ["file.content.filebeat-yml"]
 }
 
-task.query "elasticsearch-wait" {
-  query = <<EOF
-MAX_SECONDS=60
-while /bin/true
-do
-    status=$(curl -s 'http://localhost:9200/_cluster/health' 2>/dev/null | jq -r .status)
-    if [ "$status" == "yellow" ] || [ "$status" == "green" ] ; then
-        exit 0
-    fi
-    [[ "$SECONDS" -ge "$MAX_SECONDS" ]] && exit 1
-done
+wait.query "elasticsearch-wait" {
+  check = <<EOF
+status=$(curl -s 'http://localhost:9200/_cluster/health' 2>/dev/null | jq -r .status)
+[[ "$status" == "yellow" ]] || [[ "$status" == "green" ]]
 EOF
 
+  interval  = "10s"
+  max_retry = 10
+
   depends = ["docker.container.elasticsearch-container"]
 }
 
 task "filebeat-elasticsearch-template" {
   check   = "[[  \"$(curl 'http://localhost:9200/_template/filebeat' 2>/dev/null)\" != \"{}\" ]] || exit 1"
   apply   = "curl -XPUT 'http://localhost:9200/_template/filebeat' -d@/etc/filebeat/filebeat.template.json 2>/dev/null"
-  depends = ["task.filebeat-enable", "docker.container.elasticsearch-container", "task.query.elasticsearch-wait"]
+  depends = ["task.filebeat-enable", "docker.container.elasticsearch-container", "wait.query.elasticsearch-wait"]
 }
 
 task "filebeat-start" {
@@ -80,7 +77,7 @@ task "filebeat-start" {
 
 file.directory "elasticsearch-data-directory" {
   destination = "{{param `elasticsearch-data-directory`}}"
-  create_all = true
+  create_all  = true
 }
 
 docker.image "elasticsearch-image" {

load/resource.go

@@ -37,6 +37,8 @@ import (
 	_ "github.com/asteris-llc/converge/resource/shell"
 	_ "github.com/asteris-llc/converge/resource/shell/query"
 	_ "github.com/asteris-llc/converge/resource/user"
+	_ "github.com/asteris-llc/converge/resource/wait"
+	_ "github.com/asteris-llc/converge/resource/wait/port"
 )
 
 // SetResources loads the resources for each graph node

resource/shell/preparer.go

@@ -63,7 +63,7 @@ type Preparer struct {
 	Apply string `hcl:"apply"`
 
 	// the amount of time the command will wait before halting forcefully. The
-	// format is Go's duraction string. A duration string is a possibly signed
+	// format is Go's duration string. A duration string is a possibly signed
 	// sequence of decimal numbers, each with optional fraction and a unit
 	// suffix, such as "300ms", "-1.5h" or "2h45m". Valid time units are "ns",
 	// "us" (or "µs"), "ms", "s", "m", "h".

resource/wait/port/port.go

@@ -0,0 +1,99 @@
+// Copyright © 2016 Asteris, LLC
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+package port
+
+import (
+	"fmt"
+	"net"
+
+	log "github.com/Sirupsen/logrus"
+	"github.com/asteris-llc/converge/resource"
+	"github.com/asteris-llc/converge/resource/wait"
+)
+
+// Port represents a port check
+type Port struct {
+	*resource.Status
+	*wait.Retrier
+	Host string
+	Port int
+	ConnectionCheck
+}
+
+// Check if the port is open
+func (p *Port) Check(resource.Renderer) (resource.TaskStatus, error) {
+	p.Status = resource.NewStatus()
+
+	err := p.CheckConnection()
+	if err == nil {
+		if p.RetryCount > 0 {
+			p.Status.AddMessage(fmt.Sprintf("Passed after %d retries (%v)", p.RetryCount, p.Duration))
+		}
+	} else {
+		// The desired state is that the port will be available after the apply. We
+		// also want to indicate that the port is not available in the plan output.
+		// Therefore, we set the status to StatusWillChange.
+		p.RaiseLevel(resource.StatusWillChange)
+		p.Status.AddMessage(fmt.Sprintf("Failed to connect to %s:%d: %s", p.Host, p.Port, err.Error()))
+		if p.RetryCount > 0 { // only add retry messages after an apply attempt
+			p.Status.AddMessage(fmt.Sprintf("Failed after %d retries (%v)", p.RetryCount, p.Duration))
+		}
+	}
+
+	return p, nil
+}
+
+// Apply retries the check until it passes or returns max failure threshold
+func (p *Port) Apply() (resource.TaskStatus, error) {
+	p.Status = resource.NewStatus()
+
+	_, err := p.RetryUntil(func() (bool, error) {
+		checkErr := p.CheckConnection()
+		return checkErr == nil, checkErr
+	})
+
+	return p, err
+}
+
+// CheckConnection attempts to see if a tcp port is open
+func (p *Port) CheckConnection() error {
+	if p.ConnectionCheck == nil {
+		p.ConnectionCheck = &TCPConnectionCheck{}
+	}
+	return p.ConnectionCheck.CheckConnection(p.Host, p.Port)
+}
+
+// ConnectionCheck represents a connection checker
+type ConnectionCheck interface {
+	CheckConnection(host string, port int) error
+}
+
+// TCPConnectionCheck impelements a ConnectionCheck over TCP
+type TCPConnectionCheck struct{}
+
+// CheckConnection attempts to see if a tcp port is open
+func (t *TCPConnectionCheck) CheckConnection(host string, port int) error {
+	logger := log.WithField("module", "wait.port")
+
+	addr := fmt.Sprintf("%s:%d", host, port)
+	conn, err := net.Dial("tcp", addr)
+	if err != nil {
+		logger.WithError(err).WithField("addr", addr).Debug("connection failed")
+		return err
+	}
+	defer conn.Close()
+
+	return nil
+}