[Feature] Conda Installer (#6604)

* reopen PR

* updating some windows things

* text file edit

* couple touch ups

* update post_install.sh to install via poetry

* readme

* poetry.toml

* try windows scripts

* fixes

* vbs poetry env use

* codespell

* expose uvicorn kwargs

* try windows version of modified launcher

* try something else

* try something else

* vbs api shortcut

* wrong shortcut

* launch windows shortcuts with cmd

* windows conclusion script

* add WorkingDirectory

* windows python path is not in /bin

* missed a shortcut WorkingDirectory

* notebook windows arg syntax

* cwd for notebooks

* do a cd .. move

* cli wrapper typo

* openbb-cli

* yaml

* add conda

* merge branch develop

* fix reset_index erasing the name of the column (#6619)

* handle edge case where user_settings is not formatted as expected

* add welcome message and entry point for python interface build script

* flag to not install conda on PATH

* add condarc

* condarc setting

* update requirements.txt

* use poetry groups for flexible post-install config

* update folder structure

* post install

* try something for windows

* no nsi_template

* some updates

* remove leading space.

* update aiohttp for dependabot (#6621)

* example extensions

* install examples script

* update readme

* [BugFix] Fix Pandas FutureWarning In `derivatives.futures.curve` Chart (#6626)

* fix Pandas FutureWarning in derivatives.futures.curve

* black

* codespell

* [BugFix] `obb.news.world` Add Validators To Providers Where Symbol Is Mandatory (#6625)

* add validators to enforce mandatory symbols in providers where required

* linter

* fmp 3.9 test cassette

---------

Co-authored-by: Igor Radovanovic <[email protected]>

* data.__fields__ is deprecated - use data.model_fields instead (#6629)

* some updates

* difflib check

* merge prompt

* windows updates

* cmd cwd

---------

Co-authored-by: andrewkenreich <[email protected]>
Co-authored-by: Igor Radovanovic <[email protected]>

.codespell.skip

@@ -1,5 +1,6 @@
 ./tests
 .git
+*.bat
 *.css
 *.csv
 *.html
@@ -8,6 +9,7 @@
 *.js
 *.json
 *.lock
+*.nsi
 *.scss
 *.txt
 *.yaml

build/conda/README.md

@@ -1,23 +1,36 @@
 # OpenBB Platform Installer
 
-The code in this folder allows creating a packaged installer for the OpenBB Platform using conda constructor.
+The code in this folder creates a packaged installer for the OpenBB Platform, using Conda Constructor.
 
 The installers can be created for Windows (.exe), MacOS (.pkg).
 
-The installer is distributed with a conda environment that contains python and package management tools.
-The installer is created by conda constructor.
+The installer is distributed with a Conda environment that contains Python and package management tools.
 
-The dependencies required to run the application are installed from pypi by the constructor's post_install script.
+The dependencies required to run the application are installed from PyPI by the constructor's post_install script,
+and are defined in the `requirements.txt` file.
+
+The version of Python is defined in the `construct.yaml` file, it is set for 3.12.
+
+**Conda should be installed already, and initialized in the current shell profile.**
 
 # Build Instructions
 
-1. Create the build environment (see the [Build environment](#build-environment) section)
-2. Build the installer (see the [Build command](#build-command) section)
+Navigate into the `build/conda` folder and then begin.
+
+1. Create the Conda environment.
+    - `conda env create -n constructor --file environments/constructor.yml`
+2. Activate the environment.
+    - `conda activate constructor`
+3. Build the installer.
+    - `constructor installer/.`
+
+
+# For when we have a full NSIS
 
-## Build environment
+To check nsis file - run
 
-The build environment should be created off the `environments/constructor-env.yml` file.
+miniconda3\\envs\\constructor\\NSIS\\makensis.exe /V2 assets\\installer.nsi
 
-## Build command
+gives better output
 
-In the `installer` folder run `constructor .` to build the installer.
+miniconda3\\envs\\constructor\\NSIS\\makensis.exe /V4 assets\\installer.nsi

build/conda/environments/constructor.yml

@@ -0,0 +1,7 @@
+name: constructor
+channels:
+  - conda-forge
+  - defaults
+dependencies:
+  - conda-build
+  - constructor

build/conda/installer/README.md

@@ -0,0 +1,180 @@
+# OpenBB Platform
+
+Thanks for installing the OpenBB Platform. To get the most out of your experience, we recommend reviewing the contents of this document.
+
+If you have never used Python before, don't worry, your first time is just a double-click away. The sections below outline everything you need to know for getting started.
+
+## What Did I Install?
+
+This is a kitted out development container for Pythonic, open source, financial research. It has everything you need to get started immediately.
+
+Included, is a complete installation of the OpenBB Platform and CLI, along with Conda and Jupyter Notebook, in an isolated Python 3.12 Conda environment.
+
+If you already have an installation of Conda or Anaconda, the version in this folder will not rely on, interfere, or share base environment paths with, existing versions.
+
+Tools for managing, updating, and building environments allow you to take full control of your installation, as well as quickly spin up new containers within this project folder.
+
+To get started using right away, open one of the shortcuts described below.
+
+If you want to build your own end points, connections, and features, try installing the [example extensions](#examples)
+
+## Shortcuts
+
+At the root of the installation folder (where this document is) are shortcuts for launchers and to key file locations.
+
+- Bash (Windows: CMD): Opens a command line shell with the base Conda environment activated. Activate the OpenBB environment with:
+  - Mac/Linux: `source activate obb`
+  - Windows: `conda activate obb`
+- Environments: Shortcut to the root "envs" folder, where the `obb` and any created environments are stored.
+  - "widgets.json" is located in the "assets" subdirectory.
+- openbb-api: Opens a Terminal window and launches the OpenBB API, with optional prompts to login with your OpenBB Hub PAT.
+- openbb-cli: Opens a Terminal window and launches the OpenBB CLI (formerly OpenBB Terminal).
+- openbb-ipython: Opens a Terminal window and starts an IPython session with the OpenBB Platform package imported.
+- openbb-notebook: Opens a Terminal window, starts the Jupyter development server, and opens a browser window with the Notebook application.
+- OpenBBUserData: Shortcut to the OpenBBUserData folder. Files exported from CLI are saved here.
+- Settings: Shortcut to the `~/.openbb_platform` location where `.env` files and `user_settings.json` are stored.
+  - If you cannot see the ".env" file, set Finder/Explorer to display hidden/system files.
+- Update: Opens a Terminal window and updates the OpenBB Platform environment (obb) and packages.
+
+## Command Line Entry Points
+
+There are several command line entry points available when either:
+- The OpenBB Platform environment (obb) is active.
+- The "openbb_platform" package has been installed in the active Python environment, including when `--no-deps` or `--only-root` flags were used to install.
+
+### OpenBB Entry Points
+
+- openbb: Launches the OpenBB CLI.
+- openbb-api: Launches the API and accepts select arguments.
+  - All args/kwargs available to `uvicorn run` are exposed.
+  - `--login`: True/False. Defeats the login prompt when False.
+  - `--no-build`: Skips building "widgets.json"
+  - To launch the API with no input prompts enter: `openbb-api --login False --no-build
+- openbb-build: Runs the build script that generates the static assets for the OpenBB Pythobn interface.
+  - Run this after installing/uninstalling/updating OpenBB extensions
+  - Automatically run when `openbb-update` is run.
+- openbb-update: Updates the environment packages defined in `pyproject.toml` and `poetry.lock` and rebuilds the OpenBB Python interface.
+  - Location: "/extensions/openbb_platform"
+  - Passes all args/kwargs to `poetry install`.
+
+## Installed Folder Structure
+
+At the root of the installation (where this document is), there will be two folders:
+
+- conda (Windows: This is named after the last folder path if the default was not used)
+- extensions
+
+### Conda
+
+This is the folder where Conda is installed and all environment data (site-packages, etc.) is stored. The "Environments" shortcuts takes you to the root of the "envs" folder.
+
+### Extensions
+
+This folder contains two subfolders:
+
+#### examples
+
+Contains empty example extensions for the three types of OpenBB Platform extensions:
+
+- Router
+- Provider
+- OBBject
+
+To install all three examples in the OpenBB Platform environment, open the Bash/CMD shortcut and enter:
+
+```console
+source activate obb  # CMD: conda activate obb
+cd extensions/examples
+python install_examples.py
+```
+
+#### openbb_platform
+
+This is a meta package for installing and managing OpenBB Platform installation within any Python environment.
+Poetry is used to update and resolve any dependencies that are defined in the `pyproject.toml` file.
+
+The `poetry.lock` file is updated every time the `Update` shortcut is run.
+
+There are several configurations available, via combinations of `--with`, `--without`, and `--only` parameters, when running `poetry install`.
+
+Choices are:
+
+- main
+- openbb-all
+- cli
+- notebook
+
+Each parameter will accept multiple items, but `--without` takes priority for solving for the environment.
+
+## Uninstallation
+
+Mac/Linux users can uninstall by deleting the folder, while Windows users can run the Uninstall shortcut.
+
+The leftover artifacts will include:
+
+- Global configuration files stored by Poetry, PIP, Conda, and Jupyter.
+- Third-party Python package caches and `.env` files.
+- Folders:
+  - ~/.openbb_platform
+  - ~/OpenBBUserData
+
+Windows users will also need to remove the parent folder where it was originally installed, after running Uninstall.
+
+Your OpenBB Hub account can be deleted, along with all associated data, from the Account page in [my.openbb.co](https://my.openbb.co)
+
+## Additional Information
+
+### Conda Install vs. PIP Install vs. Poetry Install
+
+The three ways to install packages may appear to accomplish the same objectives - installing Python packages - but each has a distinctly different purpose.
+
+An easy way to think about the difference is, both `pip` and `poetry` require an existing Python installation; `conda` installs Python so that you can run `pip install poetry`.
+
+In general, you use:
+
+- `conda`
+  - When a new environment has been created. Packages here can be included in the creation command, `conda create`.
+  - When the specific package is only distributed through Conda or the `conda-forge` channel.
+  - When changing the version of Python used within the environment.
+- `pip`
+  - When installing/uninstalling packages to the activated Python environment.
+  - Resort to `conda install` only if packages fail to install or wheels cannot be built.
+- `poetry`
+  - Installing a local Python package.
+  - Syncing all package dependencies according to provided specs in `pyproject.toml` and `poetry.lock` files.
+  - Building and distributing Python packages.
+
+Conda is for container management, Poetry resolves the project's dependencies and provides build/distribution tools, while `pip` is for installing from [PyPI](https://pypi.org)
+
+### Documentation Resources
+
+### OpenBB
+- [Quckstart](https://docs.openbb.co/platform/getting_started/quickstart)
+- [OpenBB Pro](https://docs.openbb.co/pro)
+- [OpenBB Pro Data Connectors](https://docs.openbb.co/pro/data-connectors)
+- [OpenBB Platform](https://docs.openbb.co/platform)
+- [OpenBB CLI](https://docs.openbb.co/cli)
+- [OpenBB User Settings](https://docs.openbb.co/platform/user_guides/settings_and_environment_variables)
+- [API Keys and Authorization](https://docs.openbb.co/platform/getting_started/api_keys)
+- [REST API Requests](https://docs.openbb.co/platform/getting_started/api_requests)
+- [OpenBB Platform Architecture Overview](https://docs.openbb.co/platform/developer_guide/architecture_overview)
+- [Create A New Router Extension](https://docs.openbb.co/platform/getting_started/create_new_router_extension)
+- [OpenBB Charting]()
+- [PyWry](https://github.com/OpenBB-finance/pywry)
+
+### External
+
+- [python](https://docs.python.org/3/)
+- [conda create](https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html#managing-environments)
+- [pyproject.toml](https://python-poetry.org/docs/pyproject/)
+- [requirements.txt](https://pip.pypa.io/en/stable/reference/requirements-file-format/)
+- [pip](https://pip.pypa.io/en/stable/)
+- [poetry config](https://python-poetry.org/docs/configuration)
+- [pydantic](https://docs.pydantic.dev/latest/)
+- [uvicorn](https://www.uvicorn.org/)
+- [fastapi](https://fastapi.tiangolo.com/)
+- [jupyter](https://docs.jupyter.org/en/latest/)
+- [pandas](https://pandas.pydata.org/docs/)
+- [numpy](https://numpy.org/doc/)
+- [Plotly Graph Objects](https://plotly.com/python/graph-objects/)
+

build/conda/installer/assets/create_shortcut.vbs

@@ -1,16 +1,70 @@
 Set objShell = CreateObject("WScript.Shell")
-sDesktopPath = objShell.SpecialFolders("Desktop")
+sdesktopPath = objShell.SpecialFolders("Desktop")
 
-' Create shortcut for openbb-cli
-Set objShortcut = objShell.CreateShortcut(sDesktopPath & "\openbb-cli.lnk")
-objShortcut.TargetPath = objShell.ExpandEnvironmentStrings("%USERPROFILE%") & "\openbb\Scripts\openbb-cli.exe"
-objShortcut.IconLocation = objShell.ExpandEnvironmentStrings("%USERPROFILE%") & "\openbb\assets\openbb.ico"
-objShortcut.Description = "OpenBB CLI launcher"
+Set fso = CreateObject("Scripting.FileSystemObject")
+prefixPath = objShell.ExpandEnvironmentStrings("%PREFIX%")
+userProfilePath = objShell.ExpandEnvironmentStrings("%USERPROFILE%")
+currentPath = fso.GetAbsolutePathName(prefixPath)
+parentFolder = fso.GetParentFolderName(currentPath)
+shortcutFolder = parentFolder
+
+Set objFSO = CreateObject("Scripting.FileSystemObject")
+If Not objFSO.FolderExists(shortcutFolder) Then
+    objFSO.CreateFolder(shortcutFolder)
+End If
+
+If prefixPath = "%PREFIX%" Or userProfilePath = "%USERPROFILE%" Then
+    WScript.Echo "Environment variables %PREFIX% or %USERPROFILE% are not set correctly."
+    WScript.Quit 1
+End If
+
+Sub CreateShortcut(name, targetPath, iconPath)
+    Set objShell = CreateObject("WScript.Shell")
+    Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\" & name & ".lnk")
+    objShortcut.TargetPath = objShell.ExpandEnvironmentStrings(targetPath)
+    objShortcut.IconLocation = objShell.ExpandEnvironmentStrings(iconPath)
+    objShortcut.Description = name
+    objShortcut.Save
+End Sub
+
+CreateShortcut "openbb-cli", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "openbb-api", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "openbb-notebook", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "openbb-ipython", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "Update", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "CMD", "C:\Windows\System32\cmd.exe", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "Environments", prefixPath & "\envs", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "Settings", userProfilePath & "\.openbb_platform", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "OpenBBUserData", userProfilePath & "\OpenBBUserData", prefixPath & "\assets\openbb_icon.ico"
+CreateShortcut "Uninstall", prefixPath & "\Uninstall-OpenBB.exe", prefixPath & "\assets\openbb_icon.ico"
+
+Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\openbb-notebook.lnk")
+objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && cd " & userProfilePath & " && jupyter-notebook && exit"
+objShortcut.WorkingDirectory = prefixPath
+objShortcut.Save
+
+Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\openbb-ipython.lnk")
+objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && ipython -c ""from openbb import obb;obb"" -i"
+objShortcut.WorkingDirectory = prefixPath
 objShortcut.Save
 
-' Create shortcut for openbb-api
-Set objShortcut = objShell.CreateShortcut(sDesktopPath & "\openbb-api.lnk")
-objShortcut.TargetPath = objShell.ExpandEnvironmentStrings("%USERPROFILE%") & "\openbb\Scripts\openbb-api.exe"
-objShortcut.IconLocation = objShell.ExpandEnvironmentStrings("%USERPROFILE%") & "\openbb\assets\openbb.ico"
-objShortcut.Description = "OpenBB API launcher"
+Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\openbb-api.lnk")
+objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && call openbb-api --login  && exit"
+objShortcut.WorkingDirectory = prefixPath
 objShortcut.Save
+
+
+Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\Update.lnk")
+objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && call openbb-update && exit"
+objShortcut.WorkingDirectory = prefixPath
+objShortcut.Save
+
+Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\CMD.lnk")
+objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate base && echo Conda base environment is active. Use this shell to create new environments. && echo To activate the OpenBB environment, run 'conda activate obb'."
+objShortcut.WorkingDirectory = shortcutFolder
+objShortcut.Save
+
+Set objShortcut = objShell.CreateShortcut(shortcutFolder & "\openbb-cli.lnk")
+objShortcut.Arguments = "/k PATH " & prefixPath & ";" & prefixPath & "\Scripts;" & prefixPath & "\Library\bin;%PATH% && activate " & prefixPath & "\envs\obb && call openbb && exit"
+objShortcut.WorkingDirectory = prefixPath
+objShortcut.Save