Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problems in OpusRead interface with moses preprocessing #42

Closed
svirpioj opened this issue Jul 31, 2024 · 0 comments · Fixed by #43
Closed

Problems in OpusRead interface with moses preprocessing #42

svirpioj opened this issue Jul 31, 2024 · 0 comments · Fixed by #43
Assignees
@svirpioj

Comments

@svirpioj
Copy link
Member

The addition of moses preprocess option to OpusRead caused problems for OpusFilter, see Helsinki-NLP/OpusFilter#75.

Found so far:

  • the use of exit() on line 212 breaks the interface used from external code
  • download_dir is added to the final output files in contrast to what happens with other preprocess options
  • util.file_open() is not used but the files are just moved, thus lacking compression that can be indicated in the output file names
@svirpioj svirpioj self-assigned this Aug 7, 2024
@svirpioj svirpioj mentioned this issue Aug 7, 2024
Merged
@miau1 miau1 closed this as completed in #43 Aug 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@svirpioj svirpioj

Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix opusfilter interface Helsinki-NLP/OpusTools
1 participant
@svirpioj