Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return a dataframe from stocks search, removed export to file system (#3923) #4193

Merged
merged 8 commits into from
Feb 10, 2023

Conversation

joey-walker
Copy link
Contributor

Description

A small change to allow the search command under stocks to return a dataframe in memory so that you don't have to grab the results from the file system. Added a flag to configure enabling the exporting of the results to the filesystem.

How has this been tested?

Ran the terminal to make sure the search command under stocks still works. Tested with flag both on and off.

Also tested via importing and running under the python console.

Ran the following commands:

OpenBB Terminal

 / $ stocks
/stocks/ $ search NBH

                                  Companies found on term nbh                                   
┏━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃      ┃ Name               ┃ Country       ┃ Sector             ┃ Industry         ┃ Exchange ┃
┡━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ FNHM │ FNBH Bancorp, Inc. │ United States │ Financial Services │ Banks - Regional │ PNK      │
└──────┴────────────────────┴───────────────┴────────────────────┴──────────────────┴──────────┘

Python

Python 3.8.0 (tags/v3.8.0:fa919fd, Oct 14 2019, 19:37:50) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from openbb_terminal.sdk import openbb
>>> openbb.stocks.search(country="united states", exchange_country="Germany", limit=5) 
                             Companies found on an exchange in Germany in United States                             
┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃       ┃ NameCountrySectorIndustryExchange ┃
┡━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ 001.FCardtronics plcUnited StatesIndustrialsBusiness Equipment & SuppliesFRA      │
├───────┼──────────────────────────┼───────────────┼────────────────────┼───────────────────────────────┼──────────┤
│ NWCINewCardio, Inc.          │ United StatesHealthcareMedical DevicesPNK      │
├───────┼──────────────────────────┼───────────────┼────────────────────┼───────────────────────────────┼──────────┤
│ NWENorthWestern CorporationUnited StatesUtilitiesUtilities - DiversifiedNMS      │
├───────┼──────────────────────────┼───────────────┼────────────────────┼───────────────────────────────┼──────────┤
│ NWFLNorwood Financial Corp.  │ United StatesFinancial ServicesBanks - RegionalNMS      │
├───────┼──────────────────────────┼───────────────┼────────────────────┼───────────────────────────────┼──────────┤
│ NWGCNew World Gold Corp.     │ United StatesBasic MaterialsGoldPNK      │
└───────┴──────────────────────────┴───────────────┴────────────────────┴───────────────────────────────┴──────────┘
                                                      name        country                  sector                       industry exchange
001.F                                      Cardtronics plc  United States             Industrials  Business Equipment & Supplies      FRA
NWCI                                       NewCardio, Inc.  United States              Healthcare                Medical Devices      PNK
NWE                               NorthWestern Corporation  United States               Utilities        Utilities - Diversified      NMS
NWFL                               Norwood Financial Corp.  United States      Financial Services               Banks - Regional      NMS
NWGC                                  New World Gold Corp.  United States         Basic Materials                           Gold      PNK
...                                                    ...            ...                     ...                            ...      ...
EO5.F                                  EOG Resources, Inc.  United States                  Energy                  Oil & Gas E&P      FRA
EOB.F                                         Arcosa, Inc.  United States             Industrials      Infrastructure Operations      FRA
EOD      Wells Fargo Advantage Funds - Wells Fargo Glob...  United States      Financial Services               Asset Management      NYQ
ENPH.MX                               Enphase Energy, Inc.  United States              Technology                          Solar      MEX
ZZA.F                              Cinemark Holdings, Inc.  United States  Communication Services                  Entertainment      FRA

[14503 rows x 5 columns]

>>> df=openbb.stocks.search(country="united states", exchange_country="Germany", limit=5, export_to_file=False) 
                             Companies found on an exchange in Germany in United States                             
┏━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┓
┃       ┃ NameCountrySectorIndustryExchange ┃
┡━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━┩
│ 001.FCardtronics plcUnited StatesIndustrialsBusiness Equipment & SuppliesFRA      │
├───────┼──────────────────────────┼───────────────┼────────────────────┼───────────────────────────────┼──────────┤
│ NWCINewCardio, Inc.          │ United StatesHealthcareMedical DevicesPNK      │
├───────┼──────────────────────────┼───────────────┼────────────────────┼───────────────────────────────┼──────────┤
│ NWENorthWestern CorporationUnited StatesUtilitiesUtilities - DiversifiedNMS      │
├───────┼──────────────────────────┼───────────────┼────────────────────┼───────────────────────────────┼──────────┤
│ NWFLNorwood Financial Corp.  │ United StatesFinancial ServicesBanks - RegionalNMS      │
├───────┼──────────────────────────┼───────────────┼────────────────────┼───────────────────────────────┼──────────┤
│ NWGCNew World Gold Corp.     │ United StatesBasic MaterialsGoldPNK      │
└───────┴──────────────────────────┴───────────────┴────────────────────┴───────────────────────────────┴──────────┘

>>> df.head()
                           name        country              sector                       industry exchange
001.F           Cardtronics plc  United States         Industrials  Business Equipment & Supplies      FRA
NWCI            NewCardio, Inc.  United States          Healthcare                Medical Devices      PNK
NWE    NorthWestern Corporation  United States           Utilities        Utilities - Diversified      NMS
NWFL    Norwood Financial Corp.  United States  Financial Services               Banks - Regional      NMS
NWGC       New World Gold Corp.  United States     Basic Materials                           Gold      PNK

@reviewpad reviewpad bot added the feat XS Extra small feature label Feb 9, 2023
Comment on lines 233 to 240
if export_to_file:
export_data(
export,
os.path.dirname(os.path.abspath(__file__)),
"search",
df,
sheet_name,
)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not following the logic of this export_to_file. If export=="", which it is by default, nothing gets saved.

The way we have our sdk is that exporting should not be handled through this. Returning a df allows you to save, ie openbb.stocks.search("Apple").to_csv("path/to/file"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you are right. The underlying function does have that check, I should have noticed.

I am not sure if I fully understand your second point as I see that as feature/benefit. Is it going against the design methodology? Is there an alternative method for retrieving the results into memory without hitting the file system?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So it is against our design to have the export in the sdk model functions. In this example, if you just call

df = openbb.stocks.search("Apple")

That will be in your memory locally. If you want to save to a file you can using pandas built-ins.

I might not be understanding what you wish to accomplish. When you say you want it into memory, what are you hoping to do?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just pushed some commits to remove that redundant flag that I added. Updated documentation as well.

The desire behind this pull request is to enable this particular function to actually return the results in dataframe format - if any are available - exactly as you said: df = openbb.stocks.search("Apple")

Right now - from my understanding - It doesn't return anything.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes that is correct. You fixed that by returning the df. We just dont need to have the export/sheet name in the model functions.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah now I understand! (Hopefully).

I pushed a commit that removes the export/sheet name in that function. I adjusted and ran the tests to success.

Also redid my initial manual testing.

I also saw the mypy failures, and changed the return statements to give back an empty dataframe. Similar to other functions in the same file.

@joey-walker joey-walker changed the title Return a dataframe from stocks search, allow override to not export to file system (#3923) Return a dataframe from stocks search, removed export to file system (#3923) Feb 10, 2023
@jmaslek jmaslek enabled auto-merge February 10, 2023 19:46
@jmaslek jmaslek added this pull request to the merge queue Feb 10, 2023
Merged via the queue into OpenBB-finance:develop with commit 7639d46 Feb 10, 2023
@joey-walker joey-walker deleted the feature/search-return-df branch February 10, 2023 21:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feat XS Extra small feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants