Adds data source properties to the Jira connector #1516
Conversation
…ures update (#1519) This pull request includes updated ingest test fixtures. Please review and merge if appropriate. Co-authored-by: rvztz <[email protected]>
| self.parsed_fields["updated"], | ||
| "%Y-%m-%dT%H:%M:%S.%f%z", | ||
| ).isoformat(), | ||
| version=version, |
There was a problem hiding this comment.
I don't see a version in any of our expected outputs. how are we validating this?
There was a problem hiding this comment.
It's not always set up, so in our tests this property is empty. The object definition (based on inspecting the response) can be found here.
There was a problem hiding this comment.
got it, so in theory this could exist, but doesn't in our test content.
There was a problem hiding this comment.
Is this field a "(~release) version" for a project, or a version for the issue content (say, before and after an edit to the issue content)?
There was a problem hiding this comment.
If the field is a version for the issue content itself, testing this might need updating documents at the source to introduce a second version to some issues
I think it'll be worth it to do those updates in our test jira org and test if it's working as intended (mainly since we're introducing the version field with this PR)
At best, it'll only require a change in a document in our test jira org and rerunning the CI. At worst, it will still save us some debugging and a patch later on
…ures update (#1577) This pull request includes updated ingest test fixtures. Please review and merge if appropriate. Co-authored-by: rvztz <[email protected]>
…ctured-IO/unstructured into rvztz/data-source-properties-jira
CHANGELOG.md
Outdated
| * **Adds data source properties to SharePoint, Outlook, Onedrive, Reddit, Slack, and DeltaTable connectors** These properties (date_created, date_modified, version, source_url, record_locator) are written to element metadata during ingest, mapping elements to information about the document source from which they derive. This functionality enables downstream applications to reveal source document applications, e.g. a link to a GDrive doc, Salesforce record, etc. | ||
| * **Adds data source properties to SharePoint, Outlook, Onedrive, Reddit, Slack, DeltaTable and Jira connectors** These properties (date_created, date_modified, version, source_url, record_locator) are written to element metadata during ingest, mapping elements to information about the document source from which they derive. This functionality enables downstream applications to reveal source document applications, e.g. a link to a GDrive doc, Salesforce record, etc. | ||
| * **Add functionality to save embedded images in PDF's separately as images** This allows users to save embedded images in PDF's separately as images, given some directory path. The saved image path is written to the metadata for the Image element. Downstream applications may benefit by providing users with image links from relevant "hits." |
There was a problem hiding this comment.
no, we shouldn't change this one. these were previous version changes
|
@rvztz see comment on changelog. |
Defines update_source_metadata for the delta_table connector.