Add Jsonnet native functions

README.md

@@ -258,7 +258,11 @@ Configuration files and task/service definition files are read by [go-config](ht
 
 ## Template syntax
 
-ecspresso uses the [text/template standard package in Go](https://pkg.go.dev/text/template) to render template files, and parses as YAML/JSON/Jsonnet. By default, ecspresso provides the following as template functions.
+ecspresso uses the [text/template standard package in Go](https://pkg.go.dev/text/template) to render template files, and parses as YAML or JSON.
+
+If you use Jsonnet, ecspresso renders Jsonnet files first, then parses them as text/template. So, the template functions can render only string values by `"{{ ... }}"`, because the template function syntax `{{ }}` conflicts with Jsonnet syntax. Consider [Jsonnet functions](#jsonnet-functions) instead of template functions to render non-string values.
+
+By default, ecspresso provides the following as template functions.
 
 ### `env`
 
@@ -470,7 +474,11 @@ ecspresso v1.7 or later can use [Jsonnet](https://jsonnet.org/) file format for
 
 v2.0 or later can use Jsonnet for configuration file too.
 
-If the file extension is .jsonnet, ecspresso will process Jsonnet first, convert it to JSON, and then load it.
+v2.4 or later can use [Jsonnet functions](#jsonnet-functions).
+
+If the file extension is .jsonnet, ecspresso will process Jsonnet first, convert it to JSON, and then load it with evaluation template syntax.
+
+If you use the [Template syntax](#template-syntax) in the Jsonnet file, it may cause a syntax error because it conflicts with Jsonnet syntax. In this case, consider to use [Jsonnet functions](#jsonnet-functions).
 
 ```jsonnet
 {
@@ -496,6 +504,36 @@ $ ecspresso --ext-str Foo=foo --ext-code "Bar=1+1" ...
 }
 ```
 
+### Jsonnet functions
+
+v2.4 or later supports Jsonnet native functions in Jsonnet files.
+
+- At first, define `local func = std.native('func');` in a .jsonnet file.
+- Then, you can use the `func()` in the .jsonnet file.
+
+The Jsonnet functions are evaluated at the time of rendering Jsonnet files. So you can avoid the conflict with template syntax.
+
+#### `env`, `must_env`
+
+`env` and `must_env` functions are the same as template functions in JSON and YAML files.
+
+Unlike template functions, Jsonnet functions can render non-string values from environment variables by `std.parseInt()`, `std.parseJson()`, etc.
+
+```jsonnet
+local env = std.native('env');
+local must_env = std.native('must_env');
+{
+  foo: env('FOO', 'default value'),
+  bar: must_env('BAR'),
+  bazNumber: std.parseInt(env('BAZ_NUMBER', '0')),
+  booBool: std.parseJson(env('BOO_BOOL', 'false')),
+}
+```
+
+#### Other plugin-provided functions
+
+See [Plugins](#plugins) section.
+
 ### Deploy to Fargate
 
 If you want to deploy services to Fargate, task definitions and service definitions require some settings.
@@ -779,7 +817,7 @@ $ ecspresso exec --port-forward -L 8080:example.com:80
 
 ## Plugins
 
-ecspresso has some plugins to extend template functions.
+ecspresso has some plugins to extend template functions and Jsonnet native functions.
 
 ### tfstate
 
@@ -823,6 +861,28 @@ ecs-service-def.json
 {{ tfstatef `aws_subnet.ecs['%s'].id` (must_env `SERVICE`) }}
 ```
 
+#### tfstate Jsonnet function
+
+`tfstate` Jsonnet function is the same as template function in JSON and YAML files.
+`tfstatef` Jsonnet function is not provided. Use `std.format()` or interpolation instead.
+
+```jsonnet
+local tfstate = std.native('tfstate');
+{
+  networkConfiguration: {
+    awsvpcConfiguration: {
+      subnets: [
+        tfstate('aws_subnet.private["%s"].id' % 'az-z'),
+        tfstate(std.format('aws_subnet.private["%s"].id', 'az-b')),
+      ],
+      securityGroups: [
+        tfstate('data.aws_security_group.default.id'),
+      ]
+    }
+  }
+}
+```
+
 #### Supported tfstate URL format
 
 - Local file `file://path/to/terraform.tfstate`
@@ -861,6 +921,17 @@ So in templates, functions are called with prefixes.
 ]
 ```
 
+Jsonnet function for multiple tfstate support is available by `func_prefix`.
+
+```jsonnet
+local first_tfstate = std.native('first_tfstate'); // func_prefix: first_
+local second_tfstate = std.native('second_tfstate'); // func_prefix: second_
+[
+  first_tfstate('aws_s3_bucket.main.arn'),
+  second_tfstate('aws_s3_bucket.main.arn'),
+]
+```
+
 ### CloudFormation
 
 The cloudformation plugin introduces template functions `cfn_output` and `cfn_export`.
@@ -909,6 +980,24 @@ ecs-service-def.json
 }
 ```
 
+#### Jsonnet functions `cfn_output`, `cfn_export`
+
+`cfn_output` and `cfn_export` functions are the same as template functions.
+
+```jsonnet
+local cfn_output = std.native('cfn_output');
+local cfn_export = std.native('cfn_export');
+{
+  subnets: [
+    cfn_output('ECS-ecspresso', 'SubnetAz1'),
+    cfn_output('ECS-ecspresso', 'SubnetAz2'),
+  ],
+  securityGroups: [
+    cfn_export('ECS-ecspresso-EcsSecurityGroupId'),
+  ],
+}
+```
+
 ### Lookups ssm parameter store
 
 The template function `ssm` reads parameters from AWS Systems Manager(SSM) Parameter Store.
@@ -939,6 +1028,20 @@ will be rendered into this.
 }
 ```
 
+#### Jsonnet functions `ssm`, `ssm_list`
+
+`ssm` function is the same as template function. For string list parameters, use `ssm_list` to specify the index.
+
+```jsonnet
+local ssm = std.native('ssm');
+local ssm_list = std.native('ssm_list');
+{
+  string: ssm('/path/to/string'),
+  stringlist: ssm_list('/path/to/stringlist', 1),
+  securestring: ssm('/path/to/securestring'),
+}
+```
+
 ### Resolve secretsmanager secret ARN
 
 The template function `secretsmanager_arn` resolves secretsmanager secret ARN by secret name.
@@ -963,6 +1066,22 @@ will be rendered into this.
   ]
 ```
 
+#### Jsonnet function `secretsmanager_arn`
+
+`secretsmanager_arn` function is the same as template function.
+
+```jsonnet
+local secretsmanager_arn = std.native('secretsmanager_arn');
+{
+  secrets: [
+    {
+      name: "FOO",
+      valueFrom: secretsmanager_arn('foo'),
+    }
+  ]
+}
+```
+
 ## LICENSE
 
 MIT

config.go

@@ -38,6 +38,9 @@ func newConfigLoader(extStr, extCode map[string]string) *configLoader {
 	for k, v := range extCode {
 		vm.ExtCode(k, v)
 	}
+	for _, f := range DefaultJsonnetNativeFuncs() {
+		vm.NativeFunction(f)
+	}
 	return &configLoader{
 		Loader: goConfig.New(),
 		VM:     vm,
@@ -60,6 +63,7 @@ type Config struct {
 
 	path               string
 	templateFuncs      []template.FuncMap
+	jsonnetNativeFuncs []*jsonnet.NativeFunction
 	dir                string
 	versionConstraints goVersion.Constraints
 	awsv2Config        aws.Config
@@ -109,6 +113,9 @@ func (l *configLoader) Load(ctx context.Context, path string, version string) (*
 	for _, f := range conf.templateFuncs {
 		l.Funcs(f)
 	}
+	for _, f := range conf.jsonnetNativeFuncs {
+		l.VM.NativeFunction(f)
+	}
 	return conf, nil
 }
 

go.mod

@@ -22,17 +22,18 @@ require (
 	github.com/aws/aws-sdk-go-v2/service/sts v1.29.1
 	github.com/aws/smithy-go v1.20.2
 	github.com/fatih/color v1.16.0
-	github.com/fujiwara/cfn-lookup v1.0.0
+	github.com/fujiwara/cfn-lookup v1.1.0
 	github.com/fujiwara/ecsta v0.4.5
 	github.com/fujiwara/logutils v1.1.2
-	github.com/fujiwara/tfstate-lookup v1.1.6
+	github.com/fujiwara/ssm-lookup v0.1.0
+	github.com/fujiwara/tfstate-lookup v1.3.2
 	github.com/goccy/go-yaml v1.9.5
-	github.com/google/go-cmp v0.5.9
-	github.com/google/go-jsonnet v0.19.1
+	github.com/google/go-cmp v0.6.0
+	github.com/google/go-jsonnet v0.20.0
 	github.com/hashicorp/go-envparse v0.1.0
 	github.com/hashicorp/go-version v1.6.0
 	github.com/hexops/gotextdiff v1.0.3
-	github.com/itchyny/gojq v0.12.11
+	github.com/itchyny/gojq v0.12.16
 	github.com/kayac/go-config v0.6.0
 	github.com/kylelemons/godebug v1.1.0
 	github.com/mattn/go-isatty v0.0.20
@@ -45,11 +46,11 @@ require (
 )
 
 require (
-	cloud.google.com/go v0.110.0 // indirect
-	cloud.google.com/go/compute v1.19.1 // indirect
+	cloud.google.com/go v0.112.0 // indirect
+	cloud.google.com/go/compute v1.23.3 // indirect
 	cloud.google.com/go/compute/metadata v0.2.3 // indirect
-	cloud.google.com/go/iam v0.13.0 // indirect
-	cloud.google.com/go/storage v1.28.1 // indirect
+	cloud.google.com/go/iam v1.1.5 // indirect
+	cloud.google.com/go/storage v1.36.0 // indirect
 	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.11.1 // indirect
 	github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.6.0 // indirect
 	github.com/Azure/azure-sdk-for-go/sdk/internal v1.8.0 // indirect
@@ -67,7 +68,7 @@ require (
 	github.com/Songmu/flextime v0.1.0 // indirect
 	github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.2 // indirect
 	github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.8 // indirect
-	github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.15.6 // indirect
+	github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.16.24 // indirect
 	github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.12 // indirect
 	github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.12 // indirect
 	github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0 // indirect
@@ -82,45 +83,57 @@ require (
 	github.com/aws/aws-sdk-go-v2/service/ssooidc v1.25.1 // indirect
 	github.com/creack/pty v1.1.20 // indirect
 	github.com/dimchansky/utfbom v1.1.1 // indirect
+	github.com/felixge/httpsnoop v1.0.4 // indirect
 	github.com/fujiwara/tracer v1.0.2 // indirect
+	github.com/go-logr/logr v1.3.0 // indirect
+	github.com/go-logr/stdr v1.2.2 // indirect
 	github.com/golang-jwt/jwt/v4 v4.5.0 // indirect
 	github.com/golang-jwt/jwt/v5 v5.2.1 // indirect
 	github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
 	github.com/golang/protobuf v1.5.3 // indirect
 	github.com/google/go-querystring v1.1.0 // indirect
+	github.com/google/s2a-go v0.1.7 // indirect
 	github.com/google/uuid v1.6.0 // indirect
-	github.com/googleapis/enterprise-certificate-proxy v0.2.3 // indirect
-	github.com/googleapis/gax-go/v2 v2.7.1 // indirect
+	github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
+	github.com/googleapis/gax-go/v2 v2.12.0 // indirect
 	github.com/hashicorp/go-cleanhttp v0.5.2 // indirect
 	github.com/hashicorp/go-retryablehttp v0.7.7 // indirect
-	github.com/hashicorp/go-slug v0.10.0 // indirect
-	github.com/hashicorp/go-tfe v1.10.0 // indirect
-	github.com/hashicorp/jsonapi v0.0.0-20210826224640-ee7dae0fb22d // indirect
-	github.com/itchyny/timefmt-go v0.1.5 // indirect
+	github.com/hashicorp/go-slug v0.15.0 // indirect
+	github.com/hashicorp/go-tfe v1.56.0 // indirect
+	github.com/hashicorp/jsonapi v1.3.1 // indirect
+	github.com/itchyny/timefmt-go v0.1.6 // indirect
 	github.com/jmespath/go-jmespath v0.4.0 // indirect
 	github.com/mattn/go-colorable v0.1.13 // indirect
-	github.com/mattn/go-runewidth v0.0.14 // indirect
+	github.com/mattn/go-runewidth v0.0.15 // indirect
 	github.com/mitchellh/colorstring v0.0.0-20190213212951-d06e56a500db // indirect
 	github.com/mitchellh/go-homedir v1.1.0 // indirect
 	github.com/opencontainers/go-digest v1.0.0 // indirect
 	github.com/pkg/browser v0.0.0-20240102092130-5ac0b6a4141c // indirect
 	github.com/pkg/errors v0.9.1 // indirect
-	github.com/rivo/uniseg v0.4.2 // indirect
+	github.com/rivo/uniseg v0.4.7 // indirect
 	github.com/tkuchiki/go-timezone v0.2.2 // indirect
 	github.com/tkuchiki/parsetime v0.3.0 // indirect
 	go.opencensus.io v0.24.0 // indirect
+	go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.46.1 // indirect
+	go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.46.1 // indirect
+	go.opentelemetry.io/otel v1.21.0 // indirect
+	go.opentelemetry.io/otel/metric v1.21.0 // indirect
+	go.opentelemetry.io/otel/trace v1.21.0 // indirect
 	golang.org/x/crypto v0.24.0 // indirect
 	golang.org/x/exp v0.0.0-20220303212507-bbda1eaf7a17 // indirect
 	golang.org/x/net v0.26.0 // indirect
-	golang.org/x/oauth2 v0.7.0 // indirect
+	golang.org/x/oauth2 v0.16.0 // indirect
+	golang.org/x/sync v0.7.0 // indirect
 	golang.org/x/term v0.21.0 // indirect
 	golang.org/x/text v0.16.0 // indirect
-	golang.org/x/time v0.0.0-20220722155302-e5dcc9cfc0b9 // indirect
+	golang.org/x/time v0.5.0 // indirect
 	golang.org/x/xerrors v0.0.0-20220907171357-04be3eba64a2 // indirect
-	google.golang.org/api v0.114.0 // indirect
-	google.golang.org/appengine v1.6.7 // indirect
-	google.golang.org/genproto v0.0.0-20230410155749-daa745c078e1 // indirect
-	google.golang.org/grpc v1.56.3 // indirect
+	google.golang.org/api v0.155.0 // indirect
+	google.golang.org/appengine v1.6.8 // indirect
+	google.golang.org/genproto v0.0.0-20240123012728-ef4313101c80 // indirect
+	google.golang.org/genproto/googleapis/api v0.0.0-20240123012728-ef4313101c80 // indirect
+	google.golang.org/genproto/googleapis/rpc v0.0.0-20240123012728-ef4313101c80 // indirect
+	google.golang.org/grpc v1.62.1 // indirect
 	google.golang.org/protobuf v1.33.0 // indirect
 	gopkg.in/yaml.v2 v2.4.0 // indirect
 	sigs.k8s.io/yaml v1.3.0 // indirect