Skip to content

Commit

Permalink
Update ingest test fixtures
Browse files Browse the repository at this point in the history
  • Loading branch information
@christinestraub
christinestraub authored Oct 6, 2023
1 parent e5b6925 commit d1d94f5
Show file tree
Hide file tree
Showing 7 changed files with 54 additions and 54 deletions.
2 changes: 1 addition & 1 deletion ...ected-structured-output/azure/Core-Skills-for-Biomedical-Data-Scientists-2-pages.pdf.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@
},
{
"type": "ListItem",
"element_id": "9a7cf9ee5fe6f8f03a7659594f23d9ff",
"element_id": "eca1ce0fb28f9aee393eb53e1d63b30e",
"metadata": {
"data_source": {
"url": "abfs://container1/Core-Skills-for-Biomedical-Data-Scientists-2-pages.pdf",
Expand Down
28 changes: 14 additions & 14 deletions ..._unstructured_ingest/expected-structured-output/biomed-api/65/11/main.PMC6312790.pdf.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -251,13 +251,13 @@
},
{
"type": "Table",
"element_id": "9d9fc2e0856ca8b974ebab072f88cca1",
"element_id": "6911009421d6126fc96a193e8e7b8c87",
"metadata": {
"data_source": {},
"filetype": "application/pdf",
"page_number": 2
},
"text": "How data were acquired Data formatExperimental factors Experimental featuresData source location AccessibilityRelated research article The cleaned and weighed specimen was suspended in beakers con-taining 0.5 M H2SO4 solution of different concentrations of egg shellpowder. The pre-weighed stainless steel samples were retrieved fromthe test solutions after every 24 h, cleaned appropriately, dried andreweighed.Raw, analyzedThe difference between the weight at a given time and the initialweight of the specimen was taken as the weight loss, which was usedto calculate the corrosion rate and inhibition efficiency.Inhibitor concentration, exposure timeDepartment of Chemical, Metallurgical and Materials Engineering,Tshwane University of Technology, Pretoria, South AfricaData are available within this articleO. Sanni, A. P. I. Popoola, and O. S. I. Fayomi, Enhanced corrosionresistance of stainless steel type 316 in sulphuric acid solution usingeco-friendly waste product, Results in Physics, 9 (2018) 225–230."
"text": "How data were acquired The cleaned and weighed specimen was suspended in beakers con-taining 0.5 M H2SO4 solution of different concentrations of egg shellpowder. The pre-weighed stainless steel samples were retrieved fromthe test solutions after every 24 h, cleaned appropriately, dried andreweighed.Raw, analyzedThe difference between the weight at a given time and the initialweight of the specimen was taken as the weight loss, which was usedto calculate the corrosion rate and inhibition efficiency.Inhibitor concentration, exposure timeDepartment of Chemical, Metallurgical and Materials Engineering,Tshwane University of Technology, Pretoria, South AfricaData are available within this articleO. Sanni, A. P. I. Popoola, and O. S. I. Fayomi, Enhanced corrosionresistance of stainless steel type 316 in sulphuric acid solution usingeco-friendly waste product, Results in Physics, 9 (2018) 225–230. Data formatExperimental factors Experimental featuresData source location AccessibilityRelated research article"
},
{
"type": "NarrativeText",
Expand Down Expand Up @@ -381,13 +381,13 @@
},
{
"type": "Image",
"element_id": "38f6746aa99f4e96b29e02f1d0b418fa",
"element_id": "3dd23b04172eaa4ac70b822fde1d6569",
"metadata": {
"data_source": {},
"filetype": "application/pdf",
"page_number": 2
},
"text": ")gm ( sso l i thgeW 30 20 10 10g8g6g4g2gControl 48 96 144 192 "
"text": "30 10g8g6g4g2gControl )gm ( sso 20 l thgeW i 10 48 96 144 192"
},
{
"type": "Title",
Expand Down Expand Up @@ -421,13 +421,13 @@
},
{
"type": "Image",
"element_id": "8f63e54c02cc9090d20f5001d4d90bf9",
"element_id": "d4434406b5bb0d9269431d330ec551cc",
"metadata": {
"data_source": {},
"filetype": "application/pdf",
"page_number": 3
},
"text": "2.7 1.8 0.9 10g8g6g4g2gControl 24 48 72 96 120 144 168 192 Exposure time"
"text": "2.7 1.8 10g8g6g4g2gControl 0.9 24 48 72 96 120 144 168 192 Exposure time"
},
{
"type": "NarrativeText",
Expand Down Expand Up @@ -501,13 +501,13 @@
},
{
"type": "Image",
"element_id": "11c4aec4d2de458111a4598943f9b3c2",
"element_id": "aa9468183225a7eec11024085c42365b",
"metadata": {
"data_source": {},
"filetype": "application/pdf",
"page_number": 3
},
"text": ") % ( ycneciff i EnoitibhnI i 90 80 70 60 50 40 30 20 10 0 2g4g6g8g10g 20 40 60 80 100 120 140 160 180 "
"text": "90 2g4g6g8g10g 80 ) % 70 ( ycneciff 60 i 50 EnoitibhnI 40 i 30 20 10 0 20 40 60 80 100 120 140 160 180"
},
{
"type": "Title",
Expand Down Expand Up @@ -601,13 +601,13 @@
},
{
"type": "Table",
"element_id": "6cd96e77164fa6c7237b62a72012b1b4",
"element_id": "c6738f6e333074d3151fb3b9466c26d7",
"metadata": {
"data_source": {},
"filetype": "application/pdf",
"page_number": 4
},
"text": "Inhibitorconcentration (g) bc (V/dec) ba (V/dec) Ecorr (V) icorr (A/cm2) Polarizationresistance (Ω) 0246810 0.03351.94600.01630.32330.12400.0382 0.04090.05960.23690.05400.05560.0086 (cid:3) 0.9393(cid:3) 0.8276(cid:3) 0.8825(cid:3) 0.8027(cid:3) 0.5896(cid:3) 0.5356 0.00030.00020.00015.39E-055.46E-051.24E-05 24.0910121.44042.121373.180305.650246.080 2.81631.50540.94760.43180.37720.0919"
"text": "icorr (A/cm2) Polarizationresistance (Ω) Inhibitorconcentration (g) bc (V/dec) ba (V/dec) Ecorr (V) (cid:3) 0.9393(cid:3) 0.8276(cid:3) 0.8825(cid:3) 0.8027(cid:3) 0.5896(cid:3) 0.5356 0246810 0.03351.94600.01630.32330.12400.0382 0.04090.05960.23690.05400.05560.0086 0.00030.00020.00015.39E-055.46E-051.24E-05 24.0910121.44042.121373.180305.650246.080 2.81631.50540.94760.43180.37720.0919"
},
{
"type": "Title",
Expand Down Expand Up @@ -781,13 +781,13 @@
},
{
"type": "Image",
"element_id": "a66662aaf068459610bf894dd930ba6c",
"element_id": "3f35abf61a71e8341d4e51645645724f",
"metadata": {
"data_source": {},
"filetype": "application/pdf",
"page_number": 4
},
"text": "0/C 12 10 8 6 4 2 2 4 6 8 10 Concentration (g)"
"text": "12 10 8 0/C 6 4 2 2 4 6 8 10 Concentration (g)"
},
{
"type": "FigureCaption",
Expand Down Expand Up @@ -1021,13 +1021,13 @@
},
{
"type": "Formula",
"element_id": "fc044ebf8a46e2a72c336b769ecec5f0",
"element_id": "68670005ee5fcb70031fb04896b34fee",
"metadata": {
"data_source": {},
"filetype": "application/pdf",
"page_number": 6
},
"text": "IE ð%Þ ¼ CRo (cid:3) CR CRo x 1001"
"text": "IE ð%Þ ¼ CRo (cid:3) CR 1001 x CRo"
},
{
"type": "NarrativeText",
Expand Down
12 changes: 6 additions & 6 deletions ..._unstructured_ingest/expected-structured-output/biomed-api/75/29/main.PMC6312793.pdf.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -361,13 +361,13 @@
},
{
"type": "NarrativeText",
"element_id": "9b49b3f01501b28932903fefe9fe8dc7",
"element_id": "8ca260e031eaab2e60b6eb7d3231e6bf",
"metadata": {
"data_source": {},
"filetype": "application/pdf",
"page_number": 2
},
"text": "i , an end time, te i , a start location, ls i , and an end location, lei , and"
"text": "i , a start location, ls i , and an end location, lei , i , an end time, te and"
},
{
"type": "ListItem",
Expand Down Expand Up @@ -501,13 +501,13 @@
},
{
"type": "Table",
"element_id": "13a0171cb24f7249ac5196a3dc79106a",
"element_id": "aa14fa2b3e26b2da889e9f80a7064bb3",
"metadata": {
"data_source": {},
"filetype": "application/pdf",
"page_number": 3
},
"text": "Instance size (m, n) Average number of (8, 1500)(8, 2000)(8, 2500)(8, 3000)(12, 1500)(12, 2000)(12, 2500)(12, 3000)(16, 1500)(16, 2000)(16, 2500)(16, 3000) Locations Times Vehicles 568.40672.80923.40977.00566.00732.60875.001119.60581.80778.00879.001087.20 975.201048.001078.001113.20994.001040.601081.001107.40985.401040.601083.201101.60 652.20857.201082.401272.80642.00861.201096.001286.20667.80872.401076.401284.60 668,279.401,195,844.801,866,175.202,705,617.00674,191.001,199,659.801,878,745.202,711,180.40673,585.801,200,560.801,879,387.002,684,983.60"
"text": "Instance size (m, n) Average number of Locations Times Vehicles (8, 1500)(8, 2000)(8, 2500)(8, 3000)(12, 1500)(12, 2000)(12, 2500)(12, 3000)(16, 1500)(16, 2000)(16, 2500)(16, 3000) 568.40672.80923.40977.00566.00732.60875.001119.60581.80778.00879.001087.20 975.201048.001078.001113.20994.001040.601081.001107.40985.401040.601083.201101.60 652.20857.201082.401272.80642.00861.201096.001286.20667.80872.401076.401284.60 668,279.401,195,844.801,866,175.202,705,617.00674,191.001,199,659.801,878,745.202,711,180.40673,585.801,200,560.801,879,387.002,684,983.60"
},
{
"type": "Title",
Expand Down Expand Up @@ -651,13 +651,13 @@
},
{
"type": "Table",
"element_id": "0c15cc432df29c9691363ae10cbc6aac",
"element_id": "a557a4e8f1aa6814ae2a8f82e36f49e1",
"metadata": {
"data_source": {},
"filetype": "application/pdf",
"page_number": 4
},
"text": "Number oflines Number of columns ineach line Description 11n l 3m4 l The number of depots, the number of trips, and the number of locations.The number of vehicles rd at each depot d.One line for each trip, i ¼ 1; 2; …; n. Each line provides the start location lstime tsi and the end time tei for the corresponding trip.Each element, δij; where i; j A 1; 2; …; l, refers to the travel time between location i andlocation j. i , the end location le i , the start"
"text": "Number oflines Number of columns ineach line Description 11n 3m4 The number of depots, the number of trips, and the number of locations.The number of vehicles rd at each depot d.One line for each trip, i ¼ 1; 2; …; n. Each line provides the start location lstime tsi and the end time tei for the corresponding trip.Each element, δij; where i; j A 1; 2; …; l, refers to the travel time between location i andlocation j. i , the start i , the end location le l l"
},
{
"type": "Title",
Expand Down
14 changes: 7 additions & 7 deletions ...ured-output/local-single-file-with-pdf-infer-table-structure/layout-parser-paper.pdf.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
},
{
"type": "Header",
"element_id": "76ad010a720bb15710a209d63b3cc1d1",
"element_id": "bac05707b1e00f5f57d8c702c068dc49",
"metadata": {
"data_source": {},
"filetype": "application/pdf",
Expand Down Expand Up @@ -391,14 +391,14 @@
},
{
"type": "Table",
"element_id": "71e289a268220c21575bb55a73980b83",
"element_id": "120c712c3b2e7c5572e9207c10a5c435",
"metadata": {
"data_source": {},
"filetype": "application/pdf",
"page_number": 5,
"text_as_html": "<table><thead><th>Dataset</th><th>| Base Mod</th><th>el'| Large Model</th><th>| Notes</th></thead><tr><td>PubLayNet B8]|</td><td>F/M</td><td>M</td><td>Layouts of modern scientific documents</td></tr><tr><td>PRImA</td><td>M</td><td>-</td><td>Layouts of scanned modern magazines and scientific reports</td></tr><tr><td>Newspaper</td><td>F</td><td>-</td><td>Layouts of scanned US newspapers from the 20th century</td></tr><tr><td>TableBank</td><td>F</td><td>F</td><td>Table region on modern scientific and business document</td></tr><tr><td>HJDataset</td><td>F/M</td><td>-</td><td>Layouts of history Japanese documents</td></tr></table>"
},
"text": "Dataset Base Model1 Large Model Notes PubLayNet [38]PRImA [3]Newspaper [17]TableBank [18]HJDataset [31] F / MMFFF / M M--F- Layouts of modern scientific documentsLayouts of scanned modern magazines and scientific reportsLayouts of scanned US newspapers from the 20th centuryTable region on modern scientific and business documentLayouts of history Japanese documents"
"text": "Base Model1 Large Model Notes Dataset PubLayNet [38]PRImA [3]Newspaper [17]TableBank [18]HJDataset [31] F / MMFFF / M M--F- Layouts of modern scientific documentsLayouts of scanned modern magazines and scientific reportsLayouts of scanned US newspapers from the 20th centuryTable region on modern scientific and business documentLayouts of history Japanese documents"
},
{
"type": "Title",
Expand Down Expand Up @@ -712,14 +712,14 @@
},
{
"type": "Table",
"element_id": "85e9ccdbe0e11cebcf01515320a03294",
"element_id": "1c70e4dd20e663ba4fcaa60af53adcbd",
"metadata": {
"data_source": {},
"filetype": "application/pdf",
"page_number": 8,
"text_as_html": "<table><thead><th>block.pad(top, bottom,</th><th>right,</th><th>left)</th><th>Enlarge the current block according to the input</th></thead><tr><td>block.scale(fx, fy)</td><td></td><td></td><td>Scale the current block given the ratio in x and y direction</td></tr><tr><td>block.shift(dx, dy)</td><td></td><td></td><td>Move the current block with the shift distances in x and y direction</td></tr><tr><td>block1.is_in(block2)</td><td></td><td></td><td>Whether block] is inside of block2</td></tr><tr><td>block1. intersect (block2)</td><td></td><td></td><td>Return the intersection region of blockl and block2. Coordinate type to be determined based on the inputs.</td></tr><tr><td>block1.union(block2)</td><td></td><td></td><td>Return the union region of blockl and block2. Coordinate type to be determined based on the inputs.</td></tr><tr><td>block1.relative_to(block2)</td><td></td><td></td><td>Convert the absolute coordinates of block to relative coordinates to block2</td></tr><tr><td>block1.condition_on(block2) block. crop_image (image)</td><td></td><td></td><td>Calculate the absolute coordinates of blockl given the canvas block2’s absolute coordinates Obtain the image segments in the block region</td></tr></table>"
},
"text": "block.pad(top, bottom, right, left) Enlarge the current block according to the input block.scale(fx, fy) block.shift(dx, dy) Scale the current block given the ratioin x and y direction Move the current block with the shiftdistances in x and y direction block1.is in(block2) Whether block1 is inside of block2 block1.intersect(block2) block1.union(block2) block1.relative to(block2) block1.condition on(block2) Convert the absolute coordinates of block1 torelative coordinates to block2 Calculate the absolute coordinates of block1 giventhe canvas block2’s absolute coordinates"
"text": "block.pad(top, bottom, right, left) Enlarge the current block according to the input Scale the current block given the ratioin x and y direction block.scale(fx, fy) Move the current block with the shiftdistances in x and y direction block.shift(dx, dy) Whether block1 is inside of block2 block1.is in(block2) block1.intersect(block2) block1.union(block2) Convert the absolute coordinates of block1 torelative coordinates to block2 block1.relative to(block2) Calculate the absolute coordinates of block1 giventhe canvas block2’s absolute coordinates block1.condition on(block2)"
},
{
"type": "NarrativeText",
Expand Down Expand Up @@ -1753,12 +1753,12 @@
},
{
"type": "ListItem",
"element_id": "2d605a79cf1e027c47b21883a40930c2",
"element_id": "042006f2d2112f116d1942c22ecc1d9d",
"metadata": {
"data_source": {},
"filetype": "application/pdf",
"page_number": 16
},
"text": "layout analysis. umentAnalysis and Recognition (ICDAR). pp. 1015–1022.https://doi.org/10.1109/ICDAR.2019.00166 largest dataset ever for doc-In: 2019 International Conference on DocumentIEEE (Sep 2019)."
"text": "largest dataset ever for doc-In: 2019 International Conference on DocumentIEEE (Sep 2019). umentAnalysis and Recognition (ICDAR). pp. 1015–1022.https://doi.org/10.1109/ICDAR.2019.00166 layout analysis."
}
]
Loading

0 comments on commit d1d94f5

Please sign in to comment.