Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-added missing 'onProgress' method to new Scraper interface #774

Merged
merged 4 commits into from
Apr 21, 2023

Conversation

orzarchi
Copy link
Contributor

Fixes #772.
This PR also hides two other internal methods that were public, to prevent future confusion

@@ -137,6 +137,7 @@ export interface ScraperScrapingResult {

export interface Scraper<TCredentials extends ScraperCredentials> {
scrape(credentials: TCredentials): Promise<ScraperScrapingResult>;
onProgress(func: (companyId: CompanyTypes, payload: {type: ScraperProgressTypes}) => void): void;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to add the following as well:

  • triggerTwoFactorAuth
  • getLongTermTwoFactorToken

And protect/add here also

  • fetchData
  • login
  • terminate

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done 👍

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also add a test similar to what I did in my closed PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I think using typescript is enough in this case - your test would have passed before and after my change, if the test was written in javascript - it provided value for you because it probably didn't compile.
The real problem is that "onProgress" is a undocumented and untested public API, and we should write a single real test for it (In a future PR 😉 )

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that tests are not the best place to test types, but having a .ts test for the public API that will explicitly have the usage assumptions will make a shorter dev loop if it fails in israeli-bank-scrapers and not in dependent packages.

The real problem is that "onProgress" is a undocumented and untested public API, and we should write a single real test for it (In a future PR 😉 )

I agree

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@daniel-hauser @orzarchi can I merge the PR?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@daniel-hauser I've added your test
@esakal I think we're good to go

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@orzarchi thanks
@esakal feel free to merge

@github-actions
Copy link

🎉 This PR is included in version 3.9.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Property 'onProgress' does not exist on type 'Scraper<ScraperCredentials>'
3 participants