Add CSV table preview and CSV highlighting #680
Conversation
|
Added an update to have checkboxes for enabling these new features. These are disabled by default so the original behavior and performance of the CSVloader is retained.
|
|
Thnaks a lot. I will remove the enable button. 100 rows is not a problem |
|
I found when stress testing it that performance takes a hit as columns
increases. Syntax highlighting is still pretty responsive but with 100
columns for example there's a 2-3 second delay with the table view. That
would start to add up if you needed to open multiple files with large
numbers of fields, even if they aren't very large files.
…On Sun, Jul 10, 2022, 8:29 AM Davide Faconti ***@***.***> wrote:
Thnaks a lot. I will remove the enable button. 100 rows is not a problem
—
Reply to this email directly, view it on GitHub
<#680 (comment)>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAWRIDXO3XISRPKTZYPNVRDVTLT3XANCNFSM52PRCKTQ>
.
You are receiving this because you authored the thread.Message ID:
***@***.***>
|
|
The delay occurs when the table is built, so initial loading and changing
the delimiter are where the performance hit occurs
…On Sun, Jul 10, 2022, 8:38 AM Chase Nachtmann ***@***.***> wrote:
I found when stress testing it that performance takes a hit as columns
increases. Syntax highlighting is still pretty responsive but with 100
columns for example there's a 2-3 second delay with the table view. That
would start to add up if you needed to open multiple files with large
numbers of fields, even if they aren't very large files.
On Sun, Jul 10, 2022, 8:29 AM Davide Faconti ***@***.***>
wrote:
> Thnaks a lot. I will remove the enable button. 100 rows is not a problem
>
> —
> Reply to this email directly, view it on GitHub
> <#680 (comment)>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAWRIDXO3XISRPKTZYPNVRDVTLT3XANCNFSM52PRCKTQ>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>
|
|
In that case, I will reduce the number of lines to 50 and call it a day. Can you give me one of those CSV files that you use for stress test?
|
|
Here is an example file plus python script to generate csv files with fake data |
|
Please try the latest commit 0ff9135 On my computer it is pretty fast, even with your test CSV and compiled in Debug mode. |
|
By the way: |
Just ran it, I think it's tolerable for a single file. The loading times are more noticeable imo if you make 2-3 copies of the csv and try to open them all at once. My screen recorder seems to jump past the loading times in the captured video (the mouse pointer even jumps from one spot to another on the screen) so it's not easy to show a side-by-side against the original loader. I think an upper limit on the columns displayed would be a good idea, even if that number is large. I'd hate for the preview to be the bottleneck for someone's use case, the rest of PJ seems to rip through large datasets with ease. The most likely use case that comes to mind is using PJ to combine multiple input files into larger CSVs and then loading those larger csvs. I don't think it's worth the effort but a fancy way to cap things would be to cap the total number of cells that are added to the table, so as columns increases rows decrease. Capping the column count seems just as effective and a lot simpler though. |
|
This is definitely fun to work on, and I'm stoked to try out the new features you just added! |
CSV Table Preview
-CSV data loader updated with new table view widget for file preview
-Table view updates when user changes delimiter
-Tab widget added to the CSV data loader to switch between raw text view and table view
plotjuggler_CSV_table_view_demo.mp4
CSV Highlighting
-Add QCSVHighlighter to apply syntax highlighting to the selected delimiter in the csv raw text preview
-Highlighting updates when user changes delimiter