Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

describegpt: add a --jsonl option to save results to a jsonl, if the file exists, add the result to it #1086

Closed
jqnatividad opened this issue Jun 29, 2023 · 3 comments · Fixed by #1127
Assignees
Labels
enhancement New feature or request. Once marked with this label, its in the backlog.

Comments

@jqnatividad
Copy link
Collaborator

So users can persist the results.

Apart from the standard results, the jsonl file should also include the timestamp and the name of the --prompt-file used (#1085 ).

@jqnatividad jqnatividad added the enhancement New feature or request. Once marked with this label, its in the backlog. label Jun 29, 2023
@jqnatividad
Copy link
Collaborator Author

#1106 sets the stage for this

@rzmk
Copy link
Collaborator

rzmk commented Jul 11, 2023

@jqnatividad What would be an expected output for this command?

For example printed on terminal and potentially a file if using --prompt-file, --output, & --all:

{"prompt_file": {"name": "demo.json", "timestamp": "1996-12-19T16:39:57-08:00"}}
{"dictionary": "..."}
{"description": "..."}
{"tags": "..."}

Also if --json is also used should we replace the plaintext "..." output with the JSON output {...}?

@jqnatividad
Copy link
Collaborator Author

@rzmk , the resulting output should read as valid json-lines (https://jsonlines.org/).

With each jsonl record having the following structure. Here are two pretty-printed json records for readability:

[
   {
      "prompt_file": "demo.json",
      "timestamp": "1996-12-19T16:39:57-08:00",
      "dictionary": "...",
      "description": "...",
      "tags": "..."
   },
   {
      "prompt_file": "demo.json",
      "timestamp": "1996-12-20T16:39:57-08:00",
      "dictionary": "...2",
      "description": "...2",
      "tags": "...2"
   }
]

But it should actually look like this, as jsonl calls for each record being separated by a newline:

{"prompt_file":"demo.json","timestamp":"1996-12-19T16:39:57-08:00","dictionary":"...","description":"...","tags":"..."}
{"prompt_file":"demo.json","timestamp":"1996-12-20T16:39:57-08:00","dictionary":"...2","description":"...2","tags":"...2"}

jqnatividad added a commit that referenced this issue Jul 12, 2023
describegpt: add --jsonl option (resolves #1086)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request. Once marked with this label, its in the backlog.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants