Export csv : Missing accent support

[jto2000]

Hi Ollis,
Thanks for your great plug-in! I enjoy discovering it!
Unfortunattely, I found a little issue in the export csv function:
I'm french and I use to write words with accents ;) but it seems your parser does not like it !

My octoprint.log shows:

2019-11-12 14:13:16,601 - octoprint - ERROR - Exception on /plugin/PrintJobHistory/exportPrintJobHistory/CSV [GET]
Traceback (most recent call last):
  File "/home/pi/oprint/local/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app
    response = self.full_dispatch_request()
  File "/home/pi/oprint/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request
    rv = self.dispatch_request()
  File "/home/pi/oprint/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint_PrintJobHistory/api/PrintJobHistoryAPI.py", line 233, in exportPrintJobHistoryData
    csvContent = Transform2CSV.transform2CSV(allJobsDict)
  File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint_PrintJobHistory/api/Transform2CSV.py", line 18, in transform2CSV
    writer.writerow(row)
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in position 12: ordinal not in range(128)

I tried suppressing the words with accents in my log and now it's working.
If you're interested in, maybe I could try to give you some help on this kind of small issues.

Regards,
Thomas

[OllisGit]

@jto2000 , thanks for your feedback!
I could reproduce the error with "Héllò".

I think other users could also be using a different character set, so I will try to fix it.

[jto2000]

What a fast implementation ;)
I saw you also corrected the value of filament length. I was not 100% sure of this, but I saw my value was a thousand time higher than expected, and thought it was due to my locale settings.
Thanks for your work !

[OllisGit]

Wait a few seconds before installing. I am currently writing the release notes!

[OllisGit]

Hi @jto2000,
now the new version rc3 is "official" released!

Please test and give a feedback...raise an issue if you find something and/or use the poll "What is the next big thing?"

Thx, in advance
Olli

[jto2000]

Hi Ollis,

Currently my printer is offline and I'm at work. Furthermore I have some issues with my printer setup. But I think I'll test it on sunday. I already used the poll ;)

Have a good day !
Thomas

[jto2000]

Hi Ollis,

Sorry for my late answer, I had some trouble with my printer, and then with my raspi.
In between, I changed the whole printer and reinstalled octopi !
I'm not really sure the correction is fully effective since I'm not able to load properly the database under the Print Job History Tab.
I join the part of the log which seems interesting :

2019-12-03 16:15:02,240 - octoprint - ERROR - Exception on /plugin/PrintJobHistory/loadPrintJobHistoryByQuery [GET] Traceback (most recent call last): File "/home/pi/oprint/local/lib/python2.7/site-packages/flask/app.py", line 1817, in wsgi_app response = self.full_dispatch_request() File "/home/pi/oprint/local/lib/python2.7/site-packages/flask/app.py", line 1477, in full_dispatch_request rv = self.handle_user_exception(e) File "/home/pi/oprint/local/lib/python2.7/site-packages/flask/app.py", line 1381, in handle_user_exception reraise(exc_type, exc_value, tb) File "/home/pi/oprint/local/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request rv = self.dispatch_request() File "/home/pi/oprint/local/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint_PrintJobHistory/api/PrintJobHistoryAPI.py", line 142, in get_printjobhistoryByQuery allJobsAsDict = TransformPrintJob2JSON.transformAllPrintJobModels(allJobsModels) File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint_PrintJobHistory/api/TransformPrintJob2JSON.py", line 54, in transformAllPrintJobModels jobAsDict = transformPrintJobModel(job) File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint_PrintJobHistory/api/TransformPrintJob2JSON.py", line 26, in transformPrintJobModel filamentDict["calculatedLength"] = StringUtils.formatSave("{:.02f}", filamentDict["calculatedLength"], "") File "/home/pi/oprint/local/lib/python2.7/site-packages/octoprint_PrintJobHistory/common/StringUtils.py", line 126, in formatSave floatValue = float(value) ValueError: could not convert string to float: 2019-12-03 16:15:02,245 - tornado.access - ERROR - 500 GET /plugin/PrintJobHistory/loadPrintJobHistoryByQuery?from=0&to=10&sortColumn=printStartDateTime&sortOrder=desc&filterName=all (::ffff:199.247.41.186) 63.00ms

[OllisGit]

Hi @jto2000,

looks like a formatting - issue.
Please download the print-job database file via the settings menu and attach the file to this issue.

[jto2000]

Yes, it seems so, I just had a look and found the issue:
I probably cleared the calculated length field before saving my first print:
SELECT * FROM pjh_filamentmodel; 1|2019-11-26 21:24:44.570709|1|Generic|1.75|1.25|PLA_Red|New_PLA_Black_Emotion|20|€|1000.0|204.3||0.61|0.01 2|2019-11-26 21:24:44.570709|2|Generic|1.75|1.25|PLA_Black|Prima Value Black|20|€|1000.0|22784.47|22785.27|68.5|1.37 3|2019-12-03 09:25:18.721953|3|Generic|1.75|1.25|PLA_Black|Prima Value Black|20|€|1000.0|28770.22|28782.72|86.5|1.73 4|2019-12-03 09:25:18.721953|4|Generic|1.75|1.25|PLA_Black|Prima Value Black|20|€|1000.0|1881.33|3838.49|5.66|0.11

I updated it by adding a random float, and now it works, but maybe it would be good to add a check on this point in the future.

Furthermore, I saw in "pjh_filamentmodel", spoolCost is a Datetime, ("spoolCost" DATETIME), it looks like a copy-paste issue.

And last point, it seems to have a unit confusion between user prompt and db display for the filament length. The first displays values in meters, and the second in millimeters. In my mind, even if float are in mm, displaying it in meters would be accurate enough.

Thanks for your job, it's great ! I hope my posts are not a problem for you.

Regards,

[OllisGit]

Hi @jto2000 ,
thanks for analysing!!
The issue will be fixed in the next release (hopefully this weekend)

...damn...what a stupid copy&paste bug...I am wondering why the database is so tolerant.
In this case I can test my db-scheme upgrade mechanism...so no previous history data should be lost.

[ThomasTL]

This formatting issue is the same as the one i fixed actually.

[OllisGit]

Hi @jto2000,
new (and last) release candidate (RC5) is out and the issue should be solved.

Please test and give me a feedback.
Thx, in advance
Olli

[jto2000]

Hi @OllisGit ,

Thank you for your new release !
The formatting issue seems ok, as the empty value for filament length.
But I still see the unit confusion I raised in previous Post. In my mind, a conversion in meters might be clearer.

Have a good day.
Thanks for your work.

[OllisGit]

Hi @jto2000 ,
I missed the "unit mismatch". It will be fixed in the next version.
BR
Olli

[OllisGit]

closed, because the unit-issue is tracked here: #15