Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
105654: builtins: implement encrypt and decrypt pgcrypto functions r=rafiss a=andyyang890

This patch implements `encrypt`, `encrypt_iv`, `decrypt`,
and `decrypt_iv` from pgcrypto. These functions require an
enterprise license on a CCL distribution.

Informs #21001

Release note (enterprise change): The pgcrypto functions `encrypt`,
`encrypt_iv`, `decrypt`, and `decrypt_iv` are now implemented.
These functions require an enterprise license on a CCL distribution.

109782: server,sql: add status server endpoint to request profiler details r=dt a=adityamaru

This change introduces a new status server endpoint to
request job profiler details. This endpoint will redirect
the request to the current coordinator node of the job in
question. This will be useful because in a followup we will
load the resumer from the coordinator node's job registry
and trigger its specific job profiler detail collection logic.

This is the first step of a few to move to a "fetch model" rather
than have each resumer dump their execution details at some
arbitrary cadence.

The core logic involved in collecting profiler details has not changed,
it has been moved in its entirety from pkg/sql to pkg/server. The
`crdb_internal.request_job_execution_details` builtin now resolves
the job's coordinator ID and calls the new status server endpoint.

Informs: #109671
Release note: None

Co-authored-by: Andy Yang <[email protected]>
Co-authored-by: adityamaru <[email protected]>
  • Loading branch information
3 people committed Sep 5, 2023
3 parents 407f33e + d639116 + a99de9f commit 1ccd8a5
Show file tree
Hide file tree
Showing 55 changed files with 1,409 additions and 190 deletions.
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
Expand Up @@ -379,6 +379,7 @@
/pkg/ccl/multitenant/tenantcostserver/ @cockroachdb/sqlproxy-prs
/pkg/ccl/oidcccl/ @cockroachdb/obs-inf-prs
/pkg/ccl/partitionccl/ @cockroachdb/sql-foundations
/pkg/ccl/pgcryptoccl/ @cockroachdb/sql-foundations

#!/pkg/ccl/serverccl/ @cockroachdb/unowned
/pkg/ccl/serverccl/diagnosticsccl/ @cockroachdb/obs-inf-prs
Expand Down
40 changes: 40 additions & 0 deletions docs/generated/http/full.md
Original file line number Diff line number Diff line change
Expand Up @@ -5204,6 +5204,46 @@ Support status: [reserved](#support-status)



## RequestJobProfilerExecutionDetails

`GET /_status/request_job_profiler_execution_details/{job_id}`



Support status: [reserved](#support-status)

#### Request Parameters







| Field | Type | Label | Description | Support status |
| ----- | ---- | ----- | ----------- | -------------- |
| job_id | [int64](#cockroach.server.serverpb.RequestJobProfilerExecutionDetailsRequest-int64) | | | [reserved](#support-status) |







#### Response Parameters













## GetJobProfilerExecutionDetails

`GET /_status/job_profiler_execution_details/{job_id}`
Expand Down
36 changes: 36 additions & 0 deletions docs/generated/sql/functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -388,10 +388,46 @@
<tbody>
<tr><td><a name="crypt"></a><code>crypt(password: <a href="string.html">string</a>, salt: <a href="string.html">string</a>) &rarr; <a href="string.html">string</a></code></td><td><span class="funcdesc"><p>Generates a hash based on a password and salt. The hash algorithm and number of rounds if applicable are encoded in the salt.</p>
</span></td><td>Immutable</td></tr>
<tr><td><a name="decrypt"></a><code>decrypt(data: <a href="bytes.html">bytes</a>, key: <a href="bytes.html">bytes</a>, type: <a href="string.html">string</a>) &rarr; <a href="bytes.html">bytes</a></code></td><td><span class="funcdesc"><p>Decrypt <code>data</code> with <code>key</code> using the cipher method specified by <code>type</code>.</p>
<p>The cipher type must have the format <code>&lt;algorithm&gt;[-&lt;mode&gt;][/pad:&lt;padding&gt;]</code> where:</p>
<ul>
<li><code>&lt;algorithm&gt;</code> is <code>aes</code></li>
<li><code>&lt;mode&gt;</code> is <code>cbc</code> (default)</li>
<li><code>&lt;padding&gt;</code> is <code>pkcs</code> (default) or <code>none</code></li>
</ul>
<p>This function requires an enterprise license on a CCL distribution.</p>
</span></td><td>Immutable</td></tr>
<tr><td><a name="decrypt_iv"></a><code>decrypt_iv(data: <a href="bytes.html">bytes</a>, key: <a href="bytes.html">bytes</a>, iv: <a href="bytes.html">bytes</a>, type: <a href="string.html">string</a>) &rarr; <a href="bytes.html">bytes</a></code></td><td><span class="funcdesc"><p>Decrypt <code>data</code> with <code>key</code> using the cipher method specified by <code>type</code>. If the mode is CBC, the provided <code>iv</code> will be used. Otherwise, it will be ignored.</p>
<p>The cipher type must have the format <code>&lt;algorithm&gt;[-&lt;mode&gt;][/pad:&lt;padding&gt;]</code> where:</p>
<ul>
<li><code>&lt;algorithm&gt;</code> is <code>aes</code></li>
<li><code>&lt;mode&gt;</code> is <code>cbc</code> (default)</li>
<li><code>&lt;padding&gt;</code> is <code>pkcs</code> (default) or <code>none</code></li>
</ul>
<p>This function requires an enterprise license on a CCL distribution.</p>
</span></td><td>Immutable</td></tr>
<tr><td><a name="digest"></a><code>digest(data: <a href="bytes.html">bytes</a>, type: <a href="string.html">string</a>) &rarr; <a href="bytes.html">bytes</a></code></td><td><span class="funcdesc"><p>Computes a binary hash of the given <code>data</code>. <code>type</code> is the algorithm to use (md5, sha1, sha224, sha256, sha384, or sha512).</p>
</span></td><td>Immutable</td></tr>
<tr><td><a name="digest"></a><code>digest(data: <a href="string.html">string</a>, type: <a href="string.html">string</a>) &rarr; <a href="bytes.html">bytes</a></code></td><td><span class="funcdesc"><p>Computes a binary hash of the given <code>data</code>. <code>type</code> is the algorithm to use (md5, sha1, sha224, sha256, sha384, or sha512).</p>
</span></td><td>Immutable</td></tr>
<tr><td><a name="encrypt"></a><code>encrypt(data: <a href="bytes.html">bytes</a>, key: <a href="bytes.html">bytes</a>, type: <a href="string.html">string</a>) &rarr; <a href="bytes.html">bytes</a></code></td><td><span class="funcdesc"><p>Encrypt <code>data</code> with <code>key</code> using the cipher method specified by <code>type</code>.</p>
<p>The cipher type must have the format <code>&lt;algorithm&gt;[-&lt;mode&gt;][/pad:&lt;padding&gt;]</code> where:</p>
<ul>
<li><code>&lt;algorithm&gt;</code> is <code>aes</code></li>
<li><code>&lt;mode&gt;</code> is <code>cbc</code> (default)</li>
<li><code>&lt;padding&gt;</code> is <code>pkcs</code> (default) or <code>none</code></li>
</ul>
<p>This function requires an enterprise license on a CCL distribution.</p>
</span></td><td>Immutable</td></tr>
<tr><td><a name="encrypt_iv"></a><code>encrypt_iv(data: <a href="bytes.html">bytes</a>, key: <a href="bytes.html">bytes</a>, iv: <a href="bytes.html">bytes</a>, type: <a href="string.html">string</a>) &rarr; <a href="bytes.html">bytes</a></code></td><td><span class="funcdesc"><p>Encrypt <code>data</code> with <code>key</code> using the cipher method specified by <code>type</code>. If the mode is CBC, the provided <code>iv</code> will be used. Otherwise, it will be ignored.</p>
<p>The cipher type must have the format <code>&lt;algorithm&gt;[-&lt;mode&gt;][/pad:&lt;padding&gt;]</code> where:</p>
<ul>
<li><code>&lt;algorithm&gt;</code> is <code>aes</code></li>
<li><code>&lt;mode&gt;</code> is <code>cbc</code> (default)</li>
<li><code>&lt;padding&gt;</code> is <code>pkcs</code> (default) or <code>none</code></li>
</ul>
<p>This function requires an enterprise license on a CCL distribution.</p>
</span></td><td>Immutable</td></tr>
<tr><td><a name="gen_salt"></a><code>gen_salt(type: <a href="string.html">string</a>) &rarr; <a href="string.html">string</a></code></td><td><span class="funcdesc"><p>Generates a salt for input into the <code>crypt</code> function using the default number of rounds.</p>
</span></td><td>Volatile</td></tr>
<tr><td><a name="gen_salt"></a><code>gen_salt(type: <a href="string.html">string</a>, iter_count: <a href="int.html">int</a>) &rarr; <a href="string.html">string</a></code></td><td><span class="funcdesc"><p>Generates a salt for input into the <code>crypt</code> function using <code>iter_count</code> number of rounds.</p>
Expand Down
10 changes: 7 additions & 3 deletions pkg/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,8 @@ ALL_TESTS = [
"//pkg/ccl/multitenantccl/tenantcostserver:tenantcostserver_test",
"//pkg/ccl/oidcccl:oidcccl_test",
"//pkg/ccl/partitionccl:partitionccl_test",
"//pkg/ccl/pgcryptoccl/pgcryptocipherccl:pgcryptocipherccl_test",
"//pkg/ccl/pgcryptoccl:pgcryptoccl_test",
"//pkg/ccl/schemachangerccl:schemachangerccl_test",
"//pkg/ccl/serverccl/adminccl:adminccl_test",
"//pkg/ccl/serverccl/diagnosticsccl:diagnosticsccl_test",
Expand Down Expand Up @@ -539,7 +541,6 @@ ALL_TESTS = [
"//pkg/sql/schemachanger/screl:screl_test",
"//pkg/sql/schemachanger/scrun:scrun_test",
"//pkg/sql/schemachanger:schemachanger_test",
"//pkg/sql/sem/builtins/pgcrypto/pgcryptocipher:pgcryptocipher_test",
"//pkg/sql/sem/builtins/pgformat:pgformat_test",
"//pkg/sql/sem/builtins:builtins_disallowed_imports_test",
"//pkg/sql/sem/builtins:builtins_test",
Expand Down Expand Up @@ -871,6 +872,10 @@ GO_TARGETS = [
"//pkg/ccl/oidcccl:oidcccl_test",
"//pkg/ccl/partitionccl:partitionccl",
"//pkg/ccl/partitionccl:partitionccl_test",
"//pkg/ccl/pgcryptoccl/pgcryptocipherccl:pgcryptocipherccl",
"//pkg/ccl/pgcryptoccl/pgcryptocipherccl:pgcryptocipherccl_test",
"//pkg/ccl/pgcryptoccl:pgcryptoccl",
"//pkg/ccl/pgcryptoccl:pgcryptoccl_test",
"//pkg/ccl/schemachangerccl:schemachangerccl",
"//pkg/ccl/schemachangerccl:schemachangerccl_test",
"//pkg/ccl/serverccl/adminccl:adminccl_test",
Expand Down Expand Up @@ -2042,8 +2047,7 @@ GO_TARGETS = [
"//pkg/sql/sem/asof:asof",
"//pkg/sql/sem/builtins/builtinconstants:builtinconstants",
"//pkg/sql/sem/builtins/builtinsregistry:builtinsregistry",
"//pkg/sql/sem/builtins/pgcrypto/pgcryptocipher:pgcryptocipher",
"//pkg/sql/sem/builtins/pgcrypto/pgcryptocipher:pgcryptocipher_test",
"//pkg/sql/sem/builtins/pgcrypto:pgcrypto",
"//pkg/sql/sem/builtins/pgformat:pgformat",
"//pkg/sql/sem/builtins/pgformat:pgformat_test",
"//pkg/sql/sem/builtins:builtins",
Expand Down
1 change: 1 addition & 0 deletions pkg/ccl/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ go_library(
"//pkg/ccl/multitenantccl",
"//pkg/ccl/oidcccl",
"//pkg/ccl/partitionccl",
"//pkg/ccl/pgcryptoccl",
"//pkg/ccl/storageccl",
"//pkg/ccl/storageccl/engineccl",
"//pkg/ccl/streamingccl/streamingest",
Expand Down
1 change: 1 addition & 0 deletions pkg/ccl/ccl_init.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ import (
_ "github.com/cockroachdb/cockroach/pkg/ccl/multitenantccl"
_ "github.com/cockroachdb/cockroach/pkg/ccl/oidcccl"
_ "github.com/cockroachdb/cockroach/pkg/ccl/partitionccl"
_ "github.com/cockroachdb/cockroach/pkg/ccl/pgcryptoccl"
_ "github.com/cockroachdb/cockroach/pkg/ccl/storageccl"
_ "github.com/cockroachdb/cockroach/pkg/ccl/storageccl/engineccl"
_ "github.com/cockroachdb/cockroach/pkg/ccl/streamingccl/streamingest"
Expand Down
136 changes: 136 additions & 0 deletions pkg/ccl/logictestccl/testdata/logic_test/pgcrypto_builtins
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
subtest encrypt_decrypt_aes_128

query T
SELECT encrypt('abc', '16_byte_long_key', 'aes')::STRING
----
\x0026cd6206cfd92140883b75c098d613

query T
SELECT decrypt('\x0026cd6206cfd92140883b75c098d613', '16_byte_long_key', 'aes')
----
abc

subtest end

subtest encrypt_decrypt_aes_192

query T
SELECT encrypt('abc', '24_byte_looooooooong_key', 'aes')::STRING
----
\x6c42e2269a65d605ecd98b2aeb8eb4e9

query T
SELECT decrypt('\x6c42e2269a65d605ecd98b2aeb8eb4e9', '24_byte_looooooooong_key', 'aes')
----
abc

subtest end

subtest encrypt_decrypt_aes_256

query T
SELECT encrypt('abc', '32_byte_looooooooooooooooong_key', 'aes')::STRING
----
\xb368f7d6adcd73633dc37696b068cfda

query T
SELECT decrypt('\xb368f7d6adcd73633dc37696b068cfda', '32_byte_looooooooooooooooong_key', 'aes')
----
abc

subtest end

subtest encrypt_decrypt_aes_multi_block_data

query T
SELECT encrypt('abcdefghijklmnopqrstuvwxyz', 'key', 'aes')::STRING
----
\x4649e8618af65b2b50aa73ec9cfc102c95fcbbaf04cb8a82333e493dc97060f3

query T
SELECT decrypt('\x4649e8618af65b2b50aa73ec9cfc102c95fcbbaf04cb8a82333e493dc97060f3', 'key', 'aes')
----
abcdefghijklmnopqrstuvwxyz

subtest end

subtest encrypt_decrypt_aes_no_padding

query T
SELECT encrypt('16byte_long_data', 'key', 'aes/pad:none')::STRING
----
\x043db9c657e2a2cd693b4239a3d8a1cb

query T
SELECT decrypt('\x043db9c657e2a2cd693b4239a3d8a1cb', 'key', 'aes/pad:none')
----
16byte_long_data

subtest end

subtest encrypt_decrypt_iv_aes

query T
SELECT encrypt_iv('abc', 'key', '123', 'aes')::STRING
----
\x91b4ef63852013c8da53829da662b871

query T
SELECT decrypt_iv('\x91b4ef63852013c8da53829da662b871', 'key', '123', 'aes')
----
abc

subtest end

subtest encrypt_error

query error pgcode 0A000 Blowfish is insecure and not supported
SELECT encrypt('abc', 'key', 'bf')

query error pgcode 0A000 ECB mode is insecure and not supported
SELECT encrypt('abc', 'key', 'aes-ecb')

query error pgcode 22023 cipher method has wrong format: "aes/pad=pkcs"
SELECT encrypt('abc', 'key', 'aes/pad=pkcs')

query error pgcode 22023 cipher method has invalid algorithm: "fakealgo"
SELECT encrypt('abc', 'key', 'fakealgo')

query error pgcode 22023 cipher method has invalid mode: "ctr"
SELECT encrypt('abc', 'key', 'aes-ctr')

query error pgcode 22023 cipher method has invalid padding: "zero"
SELECT encrypt('abc', 'key', 'aes/pad:zero')

query error pgcode 22023 data has length 3, which is not a multiple of block size 16
SELECT encrypt('abc', 'key', 'aes/pad:none')

subtest end

subtest decrypt_error

query error pgcode 0A000 Blowfish is insecure and not supported
SELECT decrypt('abc', 'key', 'bf')

query error pgcode 0A000 ECB mode is insecure and not supported
SELECT decrypt('abc', 'key', 'aes-ecb')

query error pgcode 22023 cipher method has wrong format: "aes/pad=pkcs"
SELECT decrypt('abc', 'key', 'aes/pad=pkcs')

query error pgcode 22023 cipher method has invalid algorithm: "fakealgo"
SELECT decrypt('abc', 'key', 'fakealgo')

query error pgcode 22023 cipher method has invalid mode: "ctr"
SELECT decrypt('abc', 'key', 'aes-ctr')

query error pgcode 22023 cipher method has invalid padding: "zero"
SELECT decrypt('abc', 'key', 'aes/pad:zero')

query error pgcode 22023 data has length 3, which is not a multiple of block size 16
SELECT decrypt('abc', 'key', 'aes')

query error pgcode 22023 data has length 3, which is not a multiple of block size 16
SELECT decrypt('abc', 'key', 'aes/pad:none')

subtest end
7 changes: 7 additions & 0 deletions pkg/ccl/logictestccl/tests/3node-tenant/generated_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pkg/ccl/logictestccl/tests/fakedist-disk/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ go_test(
exec_properties = {
"Pool": "large",
},
shard_count = 5,
shard_count = 6,
tags = [
"ccl_test",
"cpu:2",
Expand Down
7 changes: 7 additions & 0 deletions pkg/ccl/logictestccl/tests/fakedist-disk/generated_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pkg/ccl/logictestccl/tests/fakedist-vec-off/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ go_test(
exec_properties = {
"Pool": "large",
},
shard_count = 5,
shard_count = 6,
tags = [
"ccl_test",
"cpu:2",
Expand Down
7 changes: 7 additions & 0 deletions pkg/ccl/logictestccl/tests/fakedist-vec-off/generated_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion pkg/ccl/logictestccl/tests/fakedist/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ go_test(
exec_properties = {
"Pool": "large",
},
shard_count = 6,
shard_count = 7,
tags = [
"ccl_test",
"cpu:2",
Expand Down
7 changes: 7 additions & 0 deletions pkg/ccl/logictestccl/tests/fakedist/generated_test.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ go_test(
exec_properties = {
"Pool": "large",
},
shard_count = 5,
shard_count = 6,
tags = [
"ccl_test",
"cpu:1",
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Loading

0 comments on commit 1ccd8a5

Please sign in to comment.