Running XGen 7B Chatbot in your cloud

XGen-7B is a new LLM by Salesforce, released under Apache 2.0.

This repository contains the source code of a chatbot application, along with the dstack configuration that allows you to run this application in your cloud with a single command, automatically provisioning cloud resources for you.

Prerequisites

1. Install and set up dstack

pip install "dstack[aws,gcp,azure,lambda]" -U
dstack start

Once the dstack server is up, create a project with your cloud credentials (AWS, GCP, or Azure). Then, copy its dstack config command and run it to configure the CLI.

2. Clone the repo

git clone https://github.com/dstackai/dstack-examples
cd dstack-examples
dstack init

3. Create a profile

Create a .dstack/profiles.yml file that points to the created project and describes the resources.

Example:

profiles:

• name: gcp project: gcp resources: memory: 48GB gpu: memory: 24GB default: true

Run the app in your cloud

dstack run xgen-chatbot --reload

dstack will automatically forward the port to your local machine, providing secure and convenient access.

For more details on how dstack works, check out its documentation.