Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ingestion) Implement secrets in new managed ingestion form #5574

Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ export const SecretBuilderModal = ({ initialState, visible, onSubmit, onCancel }
title={<Typography.Text>Create a new Secret</Typography.Text>}
visible={visible}
onCancel={onCancel}
zIndex={1051} // one higher than other modals - needed for managed ingestion forms
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we store these as constant and constant+1 so if one is changed this modal doesn't disappear for (seemingly) no reason?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ugh i wanted to do that, but it's Ant designs constant of 1050 so we don't actually have access to that in our code base :( so this is risky if they change that number for some reason. we could just plop this at some ridiculously high number as well if we really wanted to

footer={
<>
<Button onClick={onCancel} type="text">
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ import styled from 'styled-components/macro';
import { MinusCircleOutlined, PlusOutlined, QuestionCircleOutlined } from '@ant-design/icons';
import { FieldType, RecipeField } from './utils';
import { ANTD_GRAY } from '../../../../entity/shared/constants';
import { Secret } from '../../../../../types.generated';
import SecretField from './SecretField/SecretField';

const Label = styled.div`
font-weight: bold;
Expand Down Expand Up @@ -109,16 +111,21 @@ function SelectField({ field }: SelectFieldProps) {

interface Props {
field: RecipeField;
secrets: Secret[];
refetchSecrets: () => void;
removeMargin?: boolean;
}

function FormField(props: Props) {
const { field, removeMargin } = props;
const { field, secrets, refetchSecrets, removeMargin } = props;

if (field.type === FieldType.LIST) return <ListField field={field} removeMargin={removeMargin} />;

if (field.type === FieldType.SELECT) return <SelectField field={field} />;

if (field.type === FieldType.SECRET)
return <SecretField field={field} secrets={secrets} refetchSecrets={refetchSecrets} />;

const isBoolean = field.type === FieldType.BOOLEAN;
const input = isBoolean ? <Checkbox /> : <Input />;
const valuePropName = isBoolean ? 'checked' : 'value';
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ import { RecipeField, RECIPE_FIELDS, setFieldValueOnRecipe } from './utils';
import FormField from './FormField';
import TestConnectionButton from './TestConnection/TestConnectionButton';
import { SNOWFLAKE } from '../../conf/snowflake/snowflake';
import { useListSecretsQuery } from '../../../../../graphql/ingestion.generated';

export const ControlsContainer = styled.div`
display: flex;
Expand Down Expand Up @@ -87,6 +88,16 @@ function RecipeForm(props: Props) {
const { type, isEditing, displayRecipe, setStagedRecipe, onClickNext, goToPrevious } = props;
const { fields, advancedFields, filterFields } = RECIPE_FIELDS[type];
const allFields = [...fields, ...advancedFields, ...filterFields];
const { data, refetch: refetchSecrets } = useListSecretsQuery({
variables: {
input: {
start: 0,
count: 1000, // get all secrets
},
},
});
const secrets =
data?.listSecrets?.secrets.sort((secretA, secretB) => secretA.name.localeCompare(secretB.name)) || [];

function updateFormValues(changedValues: any, allValues: any) {
let updatedValues = YAML.parse(displayRecipe);
Expand Down Expand Up @@ -114,7 +125,12 @@ function RecipeForm(props: Props) {
<StyledCollapse defaultActiveKey="0">
<Collapse.Panel forceRender header={<SectionHeader icon={<ApiOutlined />} text="Connection" />} key="0">
{fields.map((field, i) => (
<FormField field={field} removeMargin={i === fields.length - 1} />
<FormField
field={field}
secrets={secrets}
refetchSecrets={refetchSecrets}
removeMargin={i === fields.length - 1}
/>
))}
{type === SNOWFLAKE && (
<TestConnectionWrapper>
Expand All @@ -136,7 +152,12 @@ function RecipeForm(props: Props) {
<Typography.Title level={4}>{field.section}</Typography.Title>
)}
<MarginWrapper>
<FormField field={field} removeMargin={i === filterFields.length - 1} />
<FormField
field={field}
secrets={secrets}
refetchSecrets={refetchSecrets}
removeMargin={i === filterFields.length - 1}
/>
</MarginWrapper>
</>
))}
Expand All @@ -150,7 +171,12 @@ function RecipeForm(props: Props) {
key="2"
>
{advancedFields.map((field, i) => (
<FormField field={field} removeMargin={i === advancedFields.length - 1} />
<FormField
field={field}
secrets={secrets}
refetchSecrets={refetchSecrets}
removeMargin={i === advancedFields.length - 1}
/>
))}
</Collapse.Panel>
</StyledCollapse>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
import React, { useState } from 'react';
import { Button, message } from 'antd';
import { PlusOutlined } from '@ant-design/icons';
import { blue } from '@ant-design/colors';
import styled from 'styled-components/macro';
import { SecretBuilderModal } from '../../../../secret/SecretBuilderModal';
import { useCreateSecretMutation } from '../../../../../../graphql/ingestion.generated';
import { SecretBuilderState } from '../../../../secret/types';

const CreateButton = styled(Button)`
align-items: center;
display: flex;
justify-content: center;
margin: 8px 12px 4px 12px;
width: calc(100% - 24px);

&:hover {
color: ${blue[5]};
}

.anticon-plus {
margin-right: 5px;
}
`;

interface Props {
refetchSecrets: () => void;
}

function CreateSecretButton({ refetchSecrets }: Props) {
const [isCreateModalVisible, setIsCreateModalVisible] = useState(false);
const [createSecretMutation] = useCreateSecretMutation();

const createSecret = (state: SecretBuilderState, resetBuilderState: () => void) => {
createSecretMutation({
variables: {
input: {
name: state.name as string,
value: state.value as string,
description: state.description as string,
},
},
})
.then(() => {
setIsCreateModalVisible(false);
resetBuilderState();
setTimeout(() => refetchSecrets(), 3000);
message.loading({ content: `Loading...`, duration: 3 }).then(() => {
message.success({ content: `Successfully created Secret!` });
});
})
.catch((e) => {
message.destroy();
message.error({ content: `Failed to create secret: \n ${e.message || ''}` });
});
};

return (
<>
<CreateButton onClick={() => setIsCreateModalVisible(true)} type="text">
<PlusOutlined /> Create Secret
</CreateButton>
{isCreateModalVisible && (
<SecretBuilderModal
visible={isCreateModalVisible}
onCancel={() => setIsCreateModalVisible(false)}
onSubmit={createSecret}
/>
)}
</>
);
}

export default CreateSecretButton;
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import React from 'react';
import { Divider, Form, Select } from 'antd';
import styled from 'styled-components/macro';
import { RecipeField } from '../utils';
import { Secret } from '../../../../../../types.generated';
import CreateSecretButton from './CreateSecretButton';

const StyledDivider = styled(Divider)`
margin: 0;
`;

interface SecretFieldProps {
field: RecipeField;
secrets: Secret[];
refetchSecrets: () => void;
}

function SecretField({ field, secrets, refetchSecrets }: SecretFieldProps) {
return (
<Form.Item name={field.name} label={field.label} tooltip={field.tooltip}>
<Select
showSearch
filterOption={(input, option) => !!option?.children.toLowerCase().includes(input.toLowerCase())}
dropdownRender={(menu) => (
<>
{menu}
<StyledDivider />
<CreateSecretButton refetchSecrets={refetchSecrets} />
</>
)}
>
{secrets.map((secret) => (
<Select.Option key={secret.urn} value={`\${${secret.name}}`}>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why the \?

Copy link
Collaborator Author

@chriscollins3456 chriscollins3456 Aug 4, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

gotta escape the $ so that we can have an explicit string value of ${SECRET_NAME} - the value is what we store and then also set on the YAML recipe and on there we need it in the form of ${SECRET_NAME}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ahhhh

{secret.name}
</Select.Option>
))}
</Select>
</Form.Item>
);
}

export default SecretField;
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ export enum FieldType {
BOOLEAN,
LIST,
SELECT,
SECRET,
}

interface Option {
Expand Down Expand Up @@ -92,7 +93,7 @@ export const SNOWFLAKE_USERNAME: RecipeField = {
name: 'username',
label: 'Username',
tooltip: 'Snowflake username.',
type: FieldType.TEXT,
type: FieldType.SECRET,
fieldPath: 'source.config.username',
rules: null,
};
Expand All @@ -101,7 +102,7 @@ export const SNOWFLAKE_PASSWORD: RecipeField = {
name: 'password',
label: 'Password',
tooltip: 'Snowflake password.',
type: FieldType.TEXT,
type: FieldType.SECRET,
fieldPath: 'source.config.password',
rules: null,
};
Expand Down