Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for .pgpass to PostgreSQL #3593

Merged
merged 25 commits into from
Jul 21, 2022
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
8cfc702
Implement File.posix_permissions
radeusgd Jul 14, 2022
6d11b4c
Test and fix posix_permissions
radeusgd Jul 14, 2022
2fc30c1
Improve posix_permissions_builtin implementation
radeusgd Jul 14, 2022
a29ce3e
checkpoint
radeusgd Jul 14, 2022
270c855
locate and verify .pgpass file
radeusgd Jul 15, 2022
5be7974
checkpoint
radeusgd Jul 15, 2022
8772c91
Add a method for overriding env vars for testing
radeusgd Jul 15, 2022
d8d615a
Test and fix pgpass parsing
radeusgd Jul 15, 2022
712edcb
Test verification and matching of pgpass
radeusgd Jul 18, 2022
177deb1
checkpoint: implementing and testing env vars handling for PG
radeusgd Jul 18, 2022
22f9615
Add more env tests to make sure it acts as expected
radeusgd Jul 18, 2022
0c28310
Fix tests and impl
radeusgd Jul 18, 2022
c77ea33
Refactored File Permissions structure
radeusgd Jul 19, 2022
927818b
Add Platform.is_unix alias
radeusgd Jul 19, 2022
f9c3885
changelog, formatting
radeusgd Jul 19, 2022
7132f61
fix after rebase
radeusgd Jul 19, 2022
a4b57b2
Move unsafe override to Test.Environment
radeusgd Jul 19, 2022
6fa8d6c
rename to Test_Environment
radeusgd Jul 19, 2022
1d632f9
Merge branch 'develop' into wip/radeusgd/postgres-pgpass-182582924
mergify[bot] Jul 20, 2022
853741d
Merge branch 'develop' into wip/radeusgd/postgres-pgpass-182582924
mergify[bot] Jul 20, 2022
e1722f4
Moving to an Array as opposed to a guest object
jdunkerley Jul 20, 2022
6def40c
Merge branch 'develop' into wip/radeusgd/postgres-pgpass-182582924
mergify[bot] Jul 20, 2022
5ac395b
Revert changes.
jdunkerley Jul 20, 2022
dacfd20
Merge branch 'develop' into wip/radeusgd/postgres-pgpass-182582924
mergify[bot] Jul 21, 2022
b15863c
Merge branch 'develop' into wip/radeusgd/postgres-pgpass-182582924
mergify[bot] Jul 21, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -164,6 +164,8 @@
- [Added `line_endings` and `comment_character` options to
`File_Format.Delimited`.][3581]
- [Fixed the case of various type names and library paths][3590]
- [Added support for parsing `.pgpass` file and `PG*` environment variables for
the Postgres connection][3593]

[debug-shortcuts]:
https://github.com/enso-org/enso/blob/develop/app/gui/docs/product/shortcuts.md#debug
Expand Down Expand Up @@ -260,6 +262,7 @@
[3581]: https://github.com/enso-org/enso/pull/3581
[3588]: https://github.com/enso-org/enso/pull/3588
[3590]: https://github.com/enso-org/enso/pull/3590
[3593]: https://github.com/enso-org/enso/pull/3593

#### Enso Compiler

Expand Down
4 changes: 4 additions & 0 deletions distribution/lib/Standard/Base/0.0.0-dev/src/System.enso
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,10 @@ nano_time = @Builtin_Method "System.nano_time"
os : Text
os = @Builtin_Method "System.os"

## Check if the operating system is UNIX.
is_unix : Boolean
jdunkerley marked this conversation as resolved.
Show resolved Hide resolved
is_unix = @Builtin_Method "System.is_unix"

## PRIVATE
Returns the default line separator for the platform that the program is
currently running on.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
from Standard.Base import all

polyglot java import java.lang.System
polyglot java import org.enso.base.Environment_Utils

## ALIAS Read Environment
UNSTABLE
Expand All @@ -18,4 +18,24 @@ polyglot java import java.lang.System

example_get = Environment.get "PATH"
get : Text -> Text | Nothing
get key = System.getenv key
get key = Environment_Utils.get_environment_variable key

## UNSTABLE

Returns a value of a specified environment variable or the provided default
value if such variable is not defined.

Arguments:
- key: The name of the environment variable to look up.
- default: The default fallback value.

> Example
Look up the value of the `FOOBAR` environment variable.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's maybe use something more common in stdlib documentation like HOME ;)


import Standard.Base.System.Environment

example_get_or_else = Environment.get_or_else "FOOBAR" "my default"
get_or_else : Text -> Text -> Text
get_or_else key ~default = case get key of
Nothing -> default
value -> value
24 changes: 23 additions & 1 deletion distribution/lib/Standard/Base/0.0.0-dev/src/System/File.enso
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ from Standard.Base import all

import Standard.Base.System.File.Option
import Standard.Base.System.File.Existing_File_Behavior
import Standard.Base.System.File.File_Permissions
import Standard.Base.Error.Problem_Behavior
import Standard.Base.Data.Text.Matching_Mode
import Standard.Base.Data.Text.Text_Sub_Range
Expand Down Expand Up @@ -368,9 +369,30 @@ type File
Builtin method that gets this file's last modified time.
Recommended to use `File.last_modified_time` instead which handles
potential exceptions.
last_modified_time_builtin : File -> ZonedDateTime
last_modified_time_builtin : ZonedDateTime
last_modified_time_builtin = @Builtin_Method "File.last_modified_time_builtin"

## Gets the POSIX permissions associated with the file.

> Example
Check if the file is readable by the user's group.

import Standard.Examples

example_permissions = Examples.csv.posix_permissions.group_read
posix_permissions : File_Permissions
posix_permissions =
File_Permissions.from_java_set self.posix_permissions_builtin

## PRIVATE

Builtin method that gets this file's POSIX permissions as a Java Set.
Recommended to use `File.posix_permissions` instead which handles
potential exceptions and converts an Enso representation of the
permissions.
posix_permissions_builtin : Set
posix_permissions_builtin = @Builtin_Method "File.posix_permissions_builtin"

## Checks whether the file exists and is a directory.

> Example
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
from Standard.Base import all

polyglot java import java.nio.file.attribute.PosixFilePermission

type Permission
## Permission for read access for a given entity.
type Read

## Permission for write access for a given entity.
type Write

## Permission for execute access for a given entity.
type Execute

type File_Permissions
## Access permissions for a file.
type File_Permissions (owner : Vector Permission) (group : Vector Permission) (others : Vector Permission)

## Converts the Enso atom to its Java enum counterpart.
to_java : Vector PosixFilePermission
to_java =
result = Vector.new_builder
if self.owner.contains Read then
jdunkerley marked this conversation as resolved.
Show resolved Hide resolved
result.append PosixFilePermission.OWNER_READ
if self.owner.contains Write then
result.append PosixFilePermission.OWNER_WRITE
if self.owner.contains Execute then
result.append PosixFilePermission.OWNER_EXECUTE
if self.group.contains Read then
result.append PosixFilePermission.GROUP_READ
if self.group.contains Write then
result.append PosixFilePermission.GROUP_WRITE
if self.group.contains Execute then
result.append PosixFilePermission.GROUP_EXECUTE
if self.others.contains Read then
result.append PosixFilePermission.OTHERS_READ
if self.others.contains Write then
result.append PosixFilePermission.OTHERS_WRITE
if self.others.contains Execute then
result.append PosixFilePermission.OTHERS_EXECUTE
result.to_vector

## Checks if the given file can be read by the owner.
owner_read : Boolean
owner_read = self.owner.contains Read

## Checks if the given file can be written by the owner.
owner_write : Boolean
owner_write = self.owner.contains Write

## Checks if the given file can be executed by the owner.
owner_execute : Boolean
owner_execute = self.owner.contains Execute

## Checks if the given file can be read by the group.
group_read : Boolean
group_read = self.group.contains Read

## Checks if the given file can be written by the group.
group_write : Boolean
group_write = self.group.contains Write

## Checks if the given file can be executed by the group.
group_execute : Boolean
group_execute = self.group.contains Execute

## Checks if the given file can be read by others.
others_read : Boolean
others_read = self.others.contains Read

## Checks if the given file can be written by others.
others_write : Boolean
others_write = self.others.contains Write

## Checks if the given file can be executed by others.
others_execute : Boolean
others_execute = self.others.contains Execute

## Converts a Java `Set` of Java `PosixFilePermission` to `File_Permissions`.
from_java_set java_set =
owner = Vector.new_builder
group = Vector.new_builder
others = Vector.new_builder

if java_set.contains PosixFilePermission.OWNER_READ then
jdunkerley marked this conversation as resolved.
Show resolved Hide resolved
owner.append Read
if java_set.contains PosixFilePermission.OWNER_WRITE then
owner.append Write
if java_set.contains PosixFilePermission.OWNER_EXECUTE then
owner.append Execute
if java_set.contains PosixFilePermission.GROUP_READ then
group.append Read
if java_set.contains PosixFilePermission.GROUP_WRITE then
group.append Write
if java_set.contains PosixFilePermission.GROUP_EXECUTE then
group.append Execute
if java_set.contains PosixFilePermission.OTHERS_READ then
others.append Read
if java_set.contains PosixFilePermission.OTHERS_WRITE then
others.append Write
if java_set.contains PosixFilePermission.OTHERS_EXECUTE then
others.append Execute
File_Permissions owner.to_vector group.to_vector others.to_vector
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,10 @@ type Os
os : Os
os = from_text System.os

## Check if the currently running platform is a UNIX platform.
is_unix : Boolean
is_unix = System.is_unix

## PRIVATE

Create an Os object from text.
Expand Down
Original file line number Diff line number Diff line change
@@ -1,12 +1,15 @@
from Standard.Base import all

from Standard.Base.Data.Numbers import Parse_Error

import Standard.Database.Data.Dialect
import Standard.Database.Connection.Connection
from Standard.Database.Connection.Credentials as Credentials_Module import Credentials
import Standard.Database.Connection.Connection_Options
import Standard.Database.Connection.SSL_Mode
from Standard.Database.Connection.SSL_Mode import all
import Standard.Database.Connection.Client_Certificate
import Standard.Database.Internal.Postgres.Pgpass

polyglot java import org.postgresql.Driver

Expand All @@ -20,7 +23,7 @@ type Postgres
- credentials: The credentials to use for the connection (defaults to PGPass or No Authentication).
- use_ssl: Whether to use SSL (defaults to `Prefer`).
- client_cert: The client certificate to use or `Nothing` if not needed.
type Postgres (host:Text='localhost') (port:Integer=5432) (database:Text='') (credentials:(Credentials|Nothing)=Nothing) (use_ssl:SSL_Mode=Prefer) (client_cert:(Client_Certificate|Nothing)=Nothing)
type Postgres (host:Text=default_postgres_host) (port:Integer=default_postgres_port) (database:Text=default_postgres_database) (credentials:(Credentials|Nothing)=Nothing) (use_ssl:SSL_Mode=Prefer) (client_cert:(Client_Certificate|Nothing)=Nothing)

## Build the Connection resource.

Expand All @@ -42,11 +45,22 @@ type Postgres
jdbc_properties : [Pair Text Text]
jdbc_properties =
credentials = case self.credentials of
Nothing -> Postgres.read_pgpass self.host self.port self.database
Nothing ->
env_user = Environment.get "PGUSER"
env_password = Environment.get "PGPASSWORD"
case Pair env_user env_password of
Pair Nothing Nothing ->
Pgpass.read self.host self.port self.database
Pair Nothing _ ->
Error.throw (Illegal_State_Error "PGPASSWORD is set, but PGUSER is not.")
Pair username Nothing ->
Pgpass.read self.host self.port self.database username
Pair username password ->
[Pair 'user' username, Pair 'password' password]
Credentials username password ->
[Pair 'user' username, Pair 'password' password]

ssl_properties = Postgres.ssl_mode_to_jdbc_properties self.use_ssl
ssl_properties = ssl_mode_to_jdbc_properties self.use_ssl

cert_properties = if self.client_cert.is_nothing then [] else
self.client_cert.properties
Expand All @@ -57,31 +71,30 @@ type Postgres
dialect : Dialect
dialect = Dialect.postgres

## PRIVATE - static
Read the .pgpass file from the User's home directory and obtain username
and password. https://www.postgresql.org/docs/current/libpq-pgpass.html

Arguments:
- host: The hostname of the database server.
- port: The port of the database server.
- database: The database to connect to.
read_pgpass : Text -> Integer -> Text -> [Pair Text Text]
read_pgpass _ _ _ =
## ToDo: Code not part of the design document.
## host port database
[]

## PRIVATE - static
Given an `SSL_Mode`, create the JDBC properties to secure a Postgres-based
connection.
ssl_mode_to_jdbc_properties : SSL_Mode -> [Pair Text Text]
ssl_mode_to_jdbc_properties use_ssl = case use_ssl of
Disable -> []
Prefer -> [Pair 'sslmode' 'prefer']
Require -> [Pair 'sslmode' 'require']
Verify_CA cert_file ->
if cert_file.is_nothing then [Pair 'sslmode' 'verify-ca'] else
[Pair 'sslmode' 'verify-ca', Pair 'sslrootcert' (File.new cert_file).absolute.path]
Full_Verification cert_file ->
if cert_file.is_nothing then [Pair 'sslmode' 'verify-full'] else
[Pair 'sslmode' 'verify-full', Pair 'sslrootcert' (File.new cert_file).absolute.path]
## PRIVATE
Given an `SSL_Mode`, create the JDBC properties to secure a Postgres-based
connection.
ssl_mode_to_jdbc_properties : SSL_Mode -> [Pair Text Text]
ssl_mode_to_jdbc_properties use_ssl = case use_ssl of
Disable -> []
Prefer -> [Pair 'sslmode' 'prefer']
Require -> [Pair 'sslmode' 'require']
Verify_CA cert_file ->
if cert_file.is_nothing then [Pair 'sslmode' 'verify-ca'] else
[Pair 'sslmode' 'verify-ca', Pair 'sslrootcert' (File.new cert_file).absolute.path]
Full_Verification cert_file ->
if cert_file.is_nothing then [Pair 'sslmode' 'verify-full'] else
[Pair 'sslmode' 'verify-full', Pair 'sslrootcert' (File.new cert_file).absolute.path]

## PRIVATE
default_postgres_host = Environment.get_or_else "PGHOST" "localhost"

## PRIVATE
default_postgres_port =
hardcoded_port = 5432
case Environment.get "PGPORT" of
Nothing -> hardcoded_port
port -> Integer.parse port . catch Parse_Error (_->hardcoded_port)

## PRIVATE
default_postgres_database = Environment.get_or_else "PGDATABASE" ""
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,7 @@ import Standard.Database.Connection.Connection_Options
import Standard.Database.Connection.SSL_Mode
from Standard.Database.Connection.SSL_Mode import all
import Standard.Database.Connection.Client_Certificate

import Standard.Database.Connection.Postgres
import Standard.Database.Internal.Postgres.Pgpass

polyglot java import com.amazon.redshift.jdbc.Driver
polyglot java import java.util.Properties
Expand Down Expand Up @@ -53,7 +52,7 @@ type Redshift
jdbc_properties : [Pair Text Text]
jdbc_properties =
credentials = case self.credentials of
Nothing -> Postgres.Postgres.read_pgpass self.host self.port self.schema
Nothing -> Pgpass.read self.host self.port self.schema
AWS_Profile db_user profile ->
[Pair 'user' db_user] + (if profile == '' then [] else [Pair 'profile' profile])
AWS_Key db_user access_key secret_access_key ->
Expand All @@ -75,15 +74,15 @@ type Redshift

type AWS_Profile
## Access Redshift using IAM via an AWS profile.

Arguments:
- db_user: Redshift username to connect as.
- profile: AWS profile name (if empty uses default).
type AWS_Profile db_user:Text profile:Text=''


## Access Redshift using IAM via an AWS access key ID and secret access key.

Arguments:
- db_user: Redshift username to connect as.
- access_key: AWS access key ID.
Expand Down
Loading