Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Custom request for all providers #5280

Merged
merged 9 commits into from
Aug 4, 2023
Merged

Custom request for all providers #5280

merged 9 commits into from
Aug 4, 2023

Conversation

jmaslek
Copy link
Collaborator

@jmaslek jmaslek commented Aug 3, 2023

Adds a custom request object and the ability to pass through additional user settings.

Cleans .to_dataframe() by dropping columns that are all None.

@reviewpad reviewpad bot added the feat M Medium T-Shirt size feature label Aug 3, 2023
@jmaslek jmaslek added the v4 PRs for v4 label Aug 3, 2023
@montezdesousa
Copy link
Contributor

Code lgtm. Any objection to having make_request inside helpers.py instead of having a new file utils.py? Alternatively we could create utils/ folder and put all those utility files there (helpers, descriptions, etc.). It would mimic the provider extensions structure.

@jmaslek
Copy link
Collaborator Author

jmaslek commented Aug 4, 2023

Code lgtm. Any objection to having make_request inside helpers.py instead of having a new file utils.py? Alternatively we could create utils/ folder and put all those utility files there (helpers, descriptions, etc.). It would mimic the provider extensions structure.

No objections here.

@montezdesousa montezdesousa merged commit 41e800a into feature/openbb-sdk-v4 Aug 4, 2023
@montezdesousa montezdesousa deleted the feature/sdk-v4-james branch August 4, 2023 09:06
tehcoderer added a commit that referenced this pull request Aug 6, 2023
@tehcoderer
Squashed commit of the following:
11be14a 
commit c8c7d83
Merge: 1165e57 564d08e
Author: Theodore Aptekarev <[email protected]>
Date:   Sat Aug 5 15:09:10 2023 +0300

    Merge branch 'develop' into feature/openbb-sdk-v4

commit 564d08e
Author: Pratyush Shukla <[email protected]>
Date:   Fri Aug 4 23:25:47 2023 +0530

    add `--india` flag to `stocks/load` (#5281)

    * add `--india` flag to `stocks/load`

    Add ticker, set `india` and voila!

    * updated "help"

    * added check for `.ns`, `.NS` in ticker

    * consolidated conditions into 1 line

commit 1165e57
Author: montezdesousa <[email protected]>
Date:   Fri Aug 4 10:56:20 2023 +0100

    make some provider args optional

commit afbd4a6
Author: montezdesousa <[email protected]>
Date:   Fri Aug 4 10:45:28 2023 +0100

    rename some fetcher vars

commit 8f0b748
Author: montezdesousa <[email protected]>
Date:   Fri Aug 4 10:37:43 2023 +0100

    move helpers and descriptions to util.py directory

commit 41e800a
Author: James Maslek <[email protected]>
Date:   Fri Aug 4 05:06:15 2023 -0400

    Custom request for all providers (#5280)

    * breakpoint 1

    * breakpoint pt 1

    * Finish fred request

    * Add **kwargs to all fmp funcs

    * Moved all fmp funcs to custom request

    * benzinga requests

    * add polygon requests

    * remove comment

    * move make_request to helpers

    ---------

    Co-authored-by: montezdesousa <[email protected]>

commit dbc6751
Merge: ce5b93a f1eee44
Author: Henrique Joaquim <[email protected]>
Date:   Fri Aug 4 07:37:11 2023 +0100

    Merge branch 'develop' into feature/openbb-sdk-v4

commit ce5b93a
Author: James Maslek <[email protected]>
Date:   Thu Aug 3 19:59:51 2023 -0400

    Custom args to package builder (#5282)

commit ed07f6a
Author: montezdesousa <[email protected]>
Date:   Fri Aug 4 00:46:12 2023 +0100

    200 breaks polygon

commit f1eee44
Author: Danglewood <[email protected]>
Date:   Thu Aug 3 13:58:47 2023 -0700

    Feature/options_views: Add a Variety of Views/Charts to Extend the OptionsChains Class (#5222)

    * add options_chains_view

    * vsurf charts

    * skew for strike vs contract

    * doc string

    * add chart_volatility

    * linting

    * codespell

    * mypy

    * types

    * tmx mypy

    * update test file

    * add synthetic long/short to strategies

    * updates test_options_chains_model.py

    * updates some docstrings

    * update documentation

    * actually commit the documentation page

    * update documentation screenshots

    * try catch for 403 error

    * add tests for options_chains_view

    * int64

    * SPX error handling

    * add cboe options dashboard for terminal

    * codespell

    * enable mouse scroll wheel zoom for charts

    * declare key values for all widgets

    * add some more streamlit charts and tables

    * fix test_yfinance_view.py

    * conflict resolution

    * test

    ---------

    Co-authored-by: James Maslek <[email protected]>

commit d575c5d
Author: Danglewood <[email protected]>
Date:   Thu Aug 3 12:56:44 2023 -0700

    Fwd P/E regex fix (#5273)

commit 373c4f1
Author: DidierRLopes <[email protected]>
Date:   Thu Aug 3 10:25:47 2023 -0500

    Update all_api_keys.json (#5277)

    * Update all_api_keys.json

    * Update CONTRIBUTING.md

    * Create json-yaml-validate

    * Update release-drafter.yml

    * Update json-yaml-validate

    * fix test

    * add json lint to general

    * Update tsconfig.json

    * Add a json_validate exclude to not check tests and certain known violations

    * fix workflow

    * not sure why this didnt exclude?

    ---------

    Co-authored-by: James Maslek <[email protected]>

commit 70f70a8
Author: montezdesousa <[email protected]>
Date:   Thu Aug 3 12:43:55 2023 +0100

    Hotfix/sdk py38 (#5279)

    * replace unsupported Annotated, MappingProxyType, is_annotated_dc

    * fix package builder

commit 61afffb
Author: Diogo Sousa <[email protected]>
Date:   Wed Aug 2 16:56:40 2023 +0100

    typing

commit 0b3b30e
Author: Diogo Sousa <[email protected]>
Date:   Wed Aug 2 16:51:07 2023 +0100

    rename exception for better localization

commit 49fe650
Author: Diogo Sousa <[email protected]>
Date:   Wed Aug 2 16:43:54 2023 +0100

    update provider readme

commit 1e0a3c3
Author: Diogo Sousa <[email protected]>
Date:   Wed Aug 2 16:43:19 2023 +0100

    update provider readme

commit 9a1b4a8
Author: Diogo Sousa <[email protected]>
Date:   Wed Aug 2 16:42:02 2023 +0100

    update provider readme

commit 6b93a60
Author: Diogo Sousa <[email protected]>
Date:   Wed Aug 2 16:37:54 2023 +0100

    update provider readme

commit 13d9545
Author: Diogo Sousa <[email protected]>
Date:   Wed Aug 2 15:50:20 2023 +0100

    to follow the convention

commit e3d65b5
Author: Diogo Sousa <[email protected]>
Date:   Wed Aug 2 14:49:19 2023 +0100

    filter

commit cc5a334
Author: Diogo Sousa <[email protected]>
Date:   Wed Aug 2 14:42:14 2023 +0100

    cleanup filters

commit be864d0
Author: montezdesousa <[email protected]>
Date:   Wed Aug 2 14:17:20 2023 +0100

    Feature/provider refactoring (#5266)

    * remove comment

    * replace registry description

    * black and add some todos

    * remove unused var

    * allow description and move provider to title

    * fix the data field descriptions

    * refactor globalnews

    * added `"limit"` to metadata descriptions

    * fix cpi command and refactor

    * refactored `AnalystEstimates`

    * add concat param

    * refactored `AvailableIndices` for `economy.available_indices`

    * sort and set index to date

    * place it inside exception

    * move literals to core

    * rename descriptions dict

    * add descriptions to cpi data

    * change description

    * black

    * refactored `CompanyOverview` for `stocks.fa.overview`

    * Updated `DATA_DESCRIPTIONS` in provider metadata

    * Updated `symbol` description in `DATA_DESCRIPTIONS`

    * fix stock_news alias

    * pass standard limit

    * fix stock news

    * fix to datframe

    * rebuild python interface

    * refactored `CryptoEOD` for `crypto.load`

    * added Field descriptions

    * fix polygon stock_news

    * fix stock_news data

    * ruff

    * refactored `DividendCalendar` for `stocks.fa.cal`

    * refactored `EarningsCalendar` for `stocks.fa.cal`

    * refactored `EarningsCallTranscript` for `stocks.fa.transcript`

    * refactored `ExecutiveCompensation` for `stocks.fa.comp`

    * first round, refactor registry, executor, map

    * ruff

    * Allow alias in the provider models for provider specific params

    * Updated stock news to use alias in query

    * small fixes

    * fix interface bug

    * black

    * exceptions and typing

    * ruff

    * fix docstring

    * sort providers

    * push static assets without docstrings

    * date Field and optional dates params in `CryptoEOD` for fmp and polygon

    * refactored `ForexEOD` for `forex.load`

    * add comment

    * refactored `ForexPairs` and added Polygon provider

    * removed `_price` models from fmp and polygon

    * Updated `economy.index` to use `MajorIndicesEOD`

    * removed `_price` from sdk models

    * refactored `MajorIndicesEOD` for `economy.index`

    * refactored `StockEOD` for `stocks.load`

    * regenerate sdk

    * ruff

    * fix bug with global_news

    * typing

    * add docstrings to registry map

    * update docstrings

    * add executor query arg

    * follow convention

    * docstring

    * inject provider interface instead

    * doc

    * safer this way

    * docs

    * add abstractmethod decorator

    * add explicit fetcher property model_name

    * remove mode_name prop

    * get model name from query params

    * force extra params or data to be optional

    * replace fetcher list by fetcher dict and black

    * explicit depps in prov interface

    * explicit depps in prov interface

    * no formatted creds

    * no formatted creds

    * dep injection

    * dep injection

    * private methods

    * just verify credentials

    * docstring

    * typing and cut doc

    * typing

    * typing

    * typing

    * no overkill

    * ruff

    * don't block if model not found, just warn

    * don't block if model not found, just warn

    * don't block if model not found, just warn

    * empty list if creds None

    * check creds before request

    * just return the data

    * typing

    * move query params

    * ruff

    * move queryparams

    * move queryparams

    * move queryparams

    * fix a bunch o polygon models

    * ruff

    * partial fix fmp models

    * ruff

    * refactor more fmps

    * create dict by alias

    * refactor more fmp endpoints

    * bug

    * add params to transform query

    * ruff

    * fix fmp bugs

    * has to be annotation

    * rewrite static

    * typing

    * append return type to map

    * support return for generic data type

    * remove return type for openbb provider

    * add todo

    * fix small fred typing error

    * add more return type info

    * better message

    * handle multiple types of data (the fetcher should declare them later)

    * add comment

    * ruff

    * handle returns with generic for now (check juan solution)

    * fix bug with generic

    * fix fmp fetchr dict

    * fill return type in the provider lib for docstrings

    * polygon stock news

    * Update economy_router.py (#5254)

    * remove return type stuff

    * adapt cpi and overview

    * rewrite python interface

    * fix to_dataframe

    * limit stocks eod output

    * rename method

    * mypy

    * mypy

    * mypy chart style

    * mypy provider

    * mypy ignores

    * mypy credentials

    * Added Statements Growth endpoints from FMP

    * account mypy fix

    * privatize account method

    * one more mypy

    * remove mypy ignore

    * rename private method

    * Linting

    * Docstring generation

    * Linting

    * move provider.py into __init__.py

    * add default to required_credentials -> None

    * refactored `KeyExecutives` for `stocks.fa.mgmt`

    * refactored `HistoricalEmployees` for `stocks.fa.emp`

    * provider docstring

    * docstring

    * refactored `HistoricalStockSplits` for `stocks.fa.split`

    * standardised date in stockeod

    * refactored ``InstitutionalOwnership` for `stocks.fa.own`

    * removed unused imports

    * add `check_fields=False` in `@validator`

    * historical dividens

    * remove unused helper

    * move single used helpers to where they are used

    * move single used helpers to where they are used

    * move single used helpers to where they are used

    * standardise date, vwap in crypto eod and forex eod

    * rename __commands__ to __extensions__

    * change coverage separator

    * the map cannot change sep

    * sry

    * sry again

    * remove comment

    * refactored `MajorIndicesConstituents` for `economy.const`

    * add some descriptions about provider

    * refactored `PriceTargetConsensus` for `stocks.fa.pt`

    * refactored `PriceTarget` for `stocks.fa.pta`

    * refactored `RevenueBusinessLine` for `stocks.fa.revseg`

    * change metadata file name

    * docstring

    * remove unecessary arg

    * ruff & black

    * refactored `RevenueGeographic` for `stocks.fa.revgeo`

    * refactored `RiskPremium` for `economy.risk`

    * refactored `SECFilings` for `stocks.dd.sec`

    * refactored `ShareStatistics` for `stocks.fa.shrs`

    * refactored `StockInsiderTrading` for `stocks.fa.ins`

    * peers

    * stock splits

    * refactored `KeyMetrics` for `stocks.fa.metrics`

    * refactored `StockMultiples` for `stocks.multiples`

    * refactor income statement

    * rebuild

    * fmp balance

    * added `StocksOwnership` to `stocks.fa.own`

    * added `InstitutionalOwnership` to `stocks.fa.ins_own`

    * refactored `StocksOwnership` for `stocks.fa.own`

    * cleanup

    * refactored `TreasuryRates` for `fixedincome.treasury`

    * reorder balance items

    * fix polygon bs

    * refactored `IncomeStatement` for `stocks.fa.balance`

    * refactored base and types

    * added static files

    * cleanup

    * cleanup

    * refactored `CashFlowStatement` for `stocks.fa.cash`

    * added `stocks/fa` static file

    * black and ruff

    ---------

    Co-authored-by: Pratyush Shukla <[email protected]>
    Co-authored-by: Igor Radovanovic <[email protected]>

commit 2066d7d
Merge: fe1073d 0753719
Author: Henrique Joaquim <[email protected]>
Date:   Wed Aug 2 05:37:20 2023 +0100

    Merge branch 'develop' into feature/openbb-sdk-v4

commit 0753719
Author: montezdesousa <[email protected]>
Date:   Tue Aug 1 17:27:59 2023 +0100

    Move hub api keys ts (#5267)

    * move hub api keys ts

    * json instead

    * update contributing.md

    * Update all_api_keys.json

    ---------

    Co-authored-by: DidierRLopes <[email protected]>

commit ae91a3f
Author: teh_coderer <[email protected]>
Date:   Tue Aug 1 11:05:29 2023 -0500

    Hotfix  `economy/overview` rates displaying as dates (#5269)

    * fix: `economy/overview` rates displaying as dates

    * Fix Test

    ---------

    Co-authored-by: James Maslek <[email protected]>

commit 3b5fc53
Author: Danglewood <[email protected]>
Date:   Tue Aug 1 06:41:21 2023 -0700

    Adds details/image for installing C++ Build Tools (#5268)

    * Adds details/image for installing C++ Build Tools

    * Update installation_updates.md with screenshots

    * Update installation_updates.md

    Screenshots for MS C++ Build Tools

commit fe1073d
Author: Diogo Sousa <[email protected]>
Date:   Tue Aug 1 12:29:42 2023 +0100

    only allow setup for allowed system fields

commit c8f82e1
Author: Diogo Sousa <[email protected]>
Date:   Tue Aug 1 12:04:16 2023 +0100

    reorder system settings attrs

commit 21c69a5
Merge: 379a551 d212a8e
Author: Theodore Aptekarev <[email protected]>
Date:   Mon Jul 31 20:17:30 2023 +0300

    Merge branch 'develop' into feature/openbb-sdk-v4

commit d212a8e
Author: Danglewood <[email protected]>
Date:   Mon Jul 31 09:05:22 2023 -0700

    hotfix/gdp-units - better GDP units (#5262)

    * better GDP units

    * mypy

    * remove not needed dict

    * test_oecd_view.py

    * test_oecd_model.py

commit dc30ab3
Author: teh_coderer <[email protected]>
Date:   Fri Jul 28 21:13:42 2023 -0500

    Hotfix/ locking when deploying streamlit app (#5261)

    * fix locking when deploying streamlit app

    * Update streamlit_run.py

    * Update streamlit_run.py

commit 379a551
Author: hjoaquim <[email protected]>
Date:   Fri Jul 28 10:00:39 2023 +0100

    small comment on show() implementation

commit 849e613
Merge: 27e1728 73348c0
Author: Theodore Aptekarev <[email protected]>
Date:   Thu Jul 27 19:23:10 2023 +0300

    Merge branch 'develop' into feature/openbb-sdk-v4

commit 27e1728
Merge: afe4c48 41d1359
Author: Henrique Joaquim <[email protected]>
Date:   Thu Jul 27 15:04:19 2023 +0100

    Merge branch 'develop' into feature/openbb-sdk-v4

commit afe4c48
Author: Diogo Sousa <[email protected]>
Date:   Wed Jul 26 14:33:37 2023 +0100

    fix comment

commit 531fa10
Author: Diogo Sousa <[email protected]>
Date:   Wed Jul 26 14:32:00 2023 +0100

    rename method and add description

commit cc1a84b
Author: Diogo Sousa <[email protected]>
Date:   Wed Jul 26 13:15:55 2023 +0100

    change method description

commit ff31d9a
Author: Diogo Sousa <[email protected]>
Date:   Wed Jul 26 12:52:54 2023 +0100

    check sdk token expiration on login
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@montezdesousa montezdesousa montezdesousa approved these changes

@IgorWounds IgorWounds Awaiting requested review from IgorWounds

Assignees
No one assigned
Labels
feat M Medium T-Shirt size feature v4 PRs for v4
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@jmaslek @montezdesousa