-
Notifications
You must be signed in to change notification settings - Fork 136
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
build(release): bump unstructured #183
Merged
Changes from 3 commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
e0c6c37
bump unstructured
shreyanid 8145cab
remove test_parallel_mode_correct_result
shreyanid 181662f
lint unused imports
shreyanid 5ea73ee
TEMPORARY addition of tidy-notebooks to CI
shreyanid fc3b9ca
manually changing local file directory result to /home/runner/work/un…
shreyanid d0d5edf
removing tidy-notebooks from CI workflow
shreyanid 1b8551a
undid the last few manual changes, and dropped the file_directory field
shreyanid File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -921,6 +921,7 @@ | |
| "[{'type': 'UncategorizedText',\n", | ||
| " 'element_id': 'db1ca22813f01feda8759ff04a844e56',\n", | ||
| " 'metadata': {'filename': 'family-day.eml',\n", | ||
| " 'file_directory': '/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs',\n", | ||
| " 'filetype': 'message/rfc822',\n", | ||
| " 'sent_from': ['Mallori Harrell <[email protected]>'],\n", | ||
| " 'sent_to': ['Mallori Harrell <[email protected]>'],\n", | ||
|
|
@@ -929,6 +930,7 @@ | |
| " {'type': 'NarrativeText',\n", | ||
| " 'element_id': 'a663c393a5e143c01ef2bb5c98efa2c1',\n", | ||
| " 'metadata': {'filename': 'family-day.eml',\n", | ||
| " 'file_directory': '/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs',\n", | ||
| " 'filetype': 'message/rfc822',\n", | ||
| " 'sent_from': ['Mallori Harrell <[email protected]>'],\n", | ||
| " 'sent_to': ['Mallori Harrell <[email protected]>'],\n", | ||
|
|
@@ -937,6 +939,7 @@ | |
| " {'type': 'NarrativeText',\n", | ||
| " 'element_id': 'ce65ca3bef59957d3f1c2bab5725c82f',\n", | ||
| " 'metadata': {'filename': 'family-day.eml',\n", | ||
| " 'file_directory': '/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs',\n", | ||
| " 'filetype': 'message/rfc822',\n", | ||
| " 'sent_from': ['Mallori Harrell <[email protected]>'],\n", | ||
| " 'sent_to': ['Mallori Harrell <[email protected]>'],\n", | ||
|
|
@@ -945,6 +948,7 @@ | |
| " {'type': 'NarrativeText',\n", | ||
| " 'element_id': 'd7bcf988af9f06042d83e25c531e5744',\n", | ||
| " 'metadata': {'filename': 'family-day.eml',\n", | ||
| " 'file_directory': '/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs',\n", | ||
| " 'filetype': 'message/rfc822',\n", | ||
| " 'sent_from': ['Mallori Harrell <[email protected]>'],\n", | ||
| " 'sent_to': ['Mallori Harrell <[email protected]>'],\n", | ||
|
|
@@ -953,6 +957,7 @@ | |
| " {'type': 'Title',\n", | ||
| " 'element_id': '5550577db69c2c8aabcd90979698120a',\n", | ||
| " 'metadata': {'filename': 'family-day.eml',\n", | ||
| " 'file_directory': '/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs',\n", | ||
| " 'filetype': 'message/rfc822',\n", | ||
| " 'sent_from': ['Mallori Harrell <[email protected]>'],\n", | ||
| " 'sent_to': ['Mallori Harrell <[email protected]>'],\n", | ||
|
|
@@ -961,6 +966,7 @@ | |
| " {'type': 'Title',\n", | ||
| " 'element_id': 'ca1c571d993b6c1ed8ef56a06c16ba22',\n", | ||
| " 'metadata': {'filename': 'family-day.eml',\n", | ||
| " 'file_directory': '/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs',\n", | ||
| " 'filetype': 'message/rfc822',\n", | ||
| " 'sent_from': ['Mallori Harrell <[email protected]>'],\n", | ||
| " 'sent_to': ['Mallori Harrell <[email protected]>'],\n", | ||
|
|
@@ -969,6 +975,7 @@ | |
| " {'type': 'Title',\n", | ||
| " 'element_id': 'd5b612de8cd918addd9569b0255b65b2',\n", | ||
| " 'metadata': {'filename': 'family-day.eml',\n", | ||
| " 'file_directory': '/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs',\n", | ||
| " 'filetype': 'message/rfc822',\n", | ||
| " 'sent_from': ['Mallori Harrell <[email protected]>'],\n", | ||
| " 'sent_to': ['Mallori Harrell <[email protected]>'],\n", | ||
|
|
@@ -977,6 +984,7 @@ | |
| " {'type': 'Title',\n", | ||
| " 'element_id': '2e0b9e8ee04b9594a9c26d8535b818ff',\n", | ||
| " 'metadata': {'filename': 'family-day.eml',\n", | ||
| " 'file_directory': '/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs',\n", | ||
| " 'filetype': 'message/rfc822',\n", | ||
| " 'sent_from': ['Mallori Harrell <[email protected]>'],\n", | ||
| " 'sent_to': ['Mallori Harrell <[email protected]>'],\n", | ||
|
|
@@ -1015,7 +1023,7 @@ | |
| { | ||
| "data": { | ||
| "text/plain": [ | ||
| "'type,text,element_id,filename,filetype,sent_from,sent_to,subject,sender\\nUncategorizedText,\"Hi All,\",db1ca22813f01feda8759ff04a844e56,family-day.eml,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\nNarrativeText,Get excited for our first annual family day!\\xa0,a663c393a5e143c01ef2bb5c98efa2c1,family-day.eml,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\nNarrativeText,\"There will be face painting, a petting zoo, funnel cake and more.\",ce65ca3bef59957d3f1c2bab5725c82f,family-day.eml,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\nNarrativeText,Make sure to RSVP!,d7bcf988af9f06042d83e25c531e5744,family-day.eml,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\nTitle,Best.,5550577db69c2c8aabcd90979698120a,family-day.eml,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\nTitle,Mallori Harrell,ca1c571d993b6c1ed8ef56a06c16ba22,family-day.eml,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\nTitle,Unstructured Technologies,d5b612de8cd918addd9569b0255b65b2,family-day.eml,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\nTitle,Data Scientist,2e0b9e8ee04b9594a9c26d8535b818ff,family-day.eml,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\n'" | ||
| "'type,text,element_id,filename,file_directory,filetype,sent_from,sent_to,subject,sender\\nUncategorizedText,\"Hi All,\",db1ca22813f01feda8759ff04a844e56,family-day.eml,/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\nNarrativeText,Get excited for our first annual family day!\\xa0,a663c393a5e143c01ef2bb5c98efa2c1,family-day.eml,/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\nNarrativeText,\"There will be face painting, a petting zoo, funnel cake and more.\",ce65ca3bef59957d3f1c2bab5725c82f,family-day.eml,/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\nNarrativeText,Make sure to RSVP!,d7bcf988af9f06042d83e25c531e5744,family-day.eml,/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\nTitle,Best.,5550577db69c2c8aabcd90979698120a,family-day.eml,/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\nTitle,Mallori Harrell,ca1c571d993b6c1ed8ef56a06c16ba22,family-day.eml,/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\nTitle,Unstructured Technologies,d5b612de8cd918addd9569b0255b65b2,family-day.eml,/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\nTitle,Data Scientist,2e0b9e8ee04b9594a9c26d8535b818ff,family-day.eml,/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs,message/rfc822,[\\'Mallori Harrell <[email protected]>\\'],[\\'Mallori Harrell <[email protected]>\\'],Family Day,Mallori Harrell <[email protected]>\\n'" | ||
| ] | ||
| }, | ||
| "execution_count": null, | ||
|
|
@@ -1068,23 +1076,33 @@ | |
| "text/plain": [ | ||
| "[{'type': 'NarrativeText',\n", | ||
| " 'element_id': '1df8eeb8be847c3a1a7411e3be3e0396',\n", | ||
| " 'metadata': {'filename': 'fake-text.txt', 'filetype': 'text/plain'},\n", | ||
| " 'metadata': {'filename': 'fake-text.txt',\n", | ||
| " 'file_directory': '/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs',\n", | ||
| " 'filetype': 'text/plain'},\n", | ||
| " 'text': 'This is a test document to use for unit tests.'},\n", | ||
| " {'type': 'Title',\n", | ||
| " 'element_id': '9c218520320f238595f1fde74bdd137d',\n", | ||
| " 'metadata': {'filename': 'fake-text.txt', 'filetype': 'text/plain'},\n", | ||
| " 'metadata': {'filename': 'fake-text.txt',\n", | ||
| " 'file_directory': '/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs',\n", | ||
| " 'filetype': 'text/plain'},\n", | ||
| " 'text': 'Important points:'},\n", | ||
| " {'type': 'ListItem',\n", | ||
| " 'element_id': '39a3ae572581d0f1fe7511fd7b3aa414',\n", | ||
| " 'metadata': {'filename': 'fake-text.txt', 'filetype': 'text/plain'},\n", | ||
| " 'metadata': {'filename': 'fake-text.txt',\n", | ||
| " 'file_directory': '/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs',\n", | ||
| " 'filetype': 'text/plain'},\n", | ||
| " 'text': 'Hamburgers are delicious'},\n", | ||
| " {'type': 'ListItem',\n", | ||
| " 'element_id': 'fc1adcb8eaceac694e500a103f9f698f',\n", | ||
| " 'metadata': {'filename': 'fake-text.txt', 'filetype': 'text/plain'},\n", | ||
| " 'metadata': {'filename': 'fake-text.txt',\n", | ||
| " 'file_directory': '/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs',\n", | ||
| " 'filetype': 'text/plain'},\n", | ||
| " 'text': 'Dogs are the best'},\n", | ||
| " {'type': 'ListItem',\n", | ||
| " 'element_id': '0b61e826b1c4ab05750184da72b89f83',\n", | ||
| " 'metadata': {'filename': 'fake-text.txt', 'filetype': 'text/plain'},\n", | ||
| " 'metadata': {'filename': 'fake-text.txt',\n", | ||
| " 'file_directory': '/Users/shreyanid/Documents/all-unstructured/unstructured-api/sample-docs',\n", | ||
| " 'filetype': 'text/plain'},\n", | ||
| " 'text': 'I love fuzzy blankets'}]" | ||
| ] | ||
| }, | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like this field has been in the metadata for a while - did something just change for it to show here? This will probably get us into trouble with
make check-notebooks.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's exactly what's happening :') Not sure what just changed, this commit was thrown out as a possibility. But it isn't happening on the current main branch of api, so that means some package bump (likely unstructured) caused the change. Would appreciate some help debugging where the change came from, and why the difference in file directory is not happening locally but happening in CI.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In this release comparison it doesn't seem like anything recently changed related to the file_directory field
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, that's a fun one! I have some cycles now to take a look as well.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, I think it is that commit. Here it looks like
file_directorywill be set whenever a filename is present, and that commit will have us sendingmetadata_filenameall the time. All the different filename params confuse me, but tldr is that we should remove that field like the ones over here.