Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Properly handle "unstructured data" from Calliope Mini #48

Closed
rzbrk opened this issue Jul 22, 2021 · 2 comments
Closed

Properly handle "unstructured data" from Calliope Mini #48

rzbrk opened this issue Jul 22, 2021 · 2 comments
Assignees
Labels
enhancement New feature or request

Comments

@rzbrk
Copy link
Collaborator

rzbrk commented Jul 22, 2021

CalliopEO.py currently needs "line-oriented" data from the Calliope Mini. When Calliope Mini transmits "unstructured" data without newlines, CalliopEO.py will not continue processing the data (until it eventually receives a newline).

This could be improved by reading abitrary chunks of data from the serial input buffer. Later, CalliopEO.py can extract the lines and do further processing like adding the time stamp and so on.

Maybe, we can define this as a feature for a v2.

@Amerlander Amerlander added the enhancement New feature or request label Jul 22, 2021
rzbrk added a commit to rzbrk/CalliopEO_AstroPi that referenced this issue Aug 1, 2021
@rzbrk
Copy link
Collaborator Author

rzbrk commented Aug 1, 2021

In my fork I created an issue branch. There, I completely rewrote the function readSerialData() (see here) to handle "unstructured data", in particular a stream of data not containing newlines or very, very long lines.

I defined a serial data buffer (see here) and new global variable MAX_LINE_LENGTH (see here). If the buffer length exceeds MAX_LINE_LENGTH the buffer is "cropped" by extracting MAX_LINE_LENGTH characters as a line which will be outputted to the file.

The functions waitSerialStart() and waitSerialEnd() are now incorporated in readSerialData().

The return value of this function is no longer a list of lines. The data is directly written to file (via function write2File()). The return value of readSerialData() is now the number of characters written to the file.

Before merging this change to main, this needs further testing. I will also adding a dedicated testcase with a hex file outputting very long lines.

@rzbrk
Copy link
Collaborator Author

rzbrk commented Aug 10, 2021

Can be closed if #73 is merged to main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants