Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor codebase into classes #118

Merged
merged 4 commits into from
Nov 16, 2023
Merged

Conversation

kouloumos
Copy link
Member

@kouloumos kouloumos commented Nov 2, 2023

All the logic has been moved and slightly refactored into separate
classes to remove duplicate code, scattered logic and achieve better
readability and maintanability of the codebase by having a clear flow
of the process.

An overview of the classes:

  • Transcription is the main class that contains Transcripts.
  • Each Transcript holds a Source which we want to transcribe,
    and it is either Audio, Video or Playlist.

How the flow looks like:

  • We initialize a Transcription object that holds all the
    related configurations for the current transcription process
  • We can add as many sources as we want to the current transcription
    with transcription.add_transcription_source(...)
  • when we are ready, we transcription.start(), which:
    • produces an audio file by processing the source. This step is
      responsible for any downloads or conversions that needs to happen.
    • produces the transcription by processing the audio file. This step
      includes any summarizations, chapter generations, diarization that we
      might have configure.
      It can optionally:
      • write the transcription to a markdown file.
      • open a PR to the repo.
      • upload the transcription to an AWS S3 Bucket.
      • push the transcript to a Queuer backend.
      • write the payload of the transcription to a json file.

Also, updated the tests based on the new flow of the refactored codebase.
The Deepgram related methods that are still in application.py can later
become their own class as we start adding more services.

All the logic has been moved and slightly refactored into separate
classes for better readability and maintanability of the codebase.
(redundant code will be removed with the next commit alongside tests)

An overview of the classes:
- `Transcription` is the main class that contains `Transcripts`.
- Each `Transcript` holds a `Source` which we want to transcribe
 and it is either `Audio` or `Video`.

How the flow looks like:
- We initialize a `Transcription` object that holds all the
related configurations for the current transcription process
- We can add as many sources as we want to the current `transcription`
with `transcription.add_transcription_source(...)`
- when we are ready, we `transcription.start()`, which:
     - produces an audio file by processing the source. This step is
     responsible for any downloads or conversions that needs to happen.
     - produces the transcription by processing the audio file. This step
     includes any summarizations, chapter generations, diarization that we
     might have configure.
     It can optionally:
     - write the transcription to a markdown file.
     - open a PR to the repo.
     - upload the transcription to an AWS S3 Bucket.
     - push the transcript to a Queuer backend.
     - write the payload of the transcription to a json file.
- The code removed from `application.py` as part of this commit, has
already been moved and slightly refactored as part of the prev commit.
- update tests to work based on the new flow of the refactored codebase
- move logging configuration in a separate module
- alongside console logging, always log to a file in the workdir
- `tags`, `speakers` and `category` must now be used one time per each
item (tag, speaker, category) that we want to add to the metadata of the
transcript
- Better wording for the help text of cli options
- Update README
@kouloumos kouloumos merged commit 0edce78 into bitcointranscripts:main Nov 16, 2023
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant