Skip to content

Commit

Permalink
Update README.md to run and query server during test (#1384)
Browse files Browse the repository at this point in the history
* Update README.md to run and query server

1 - Run server:
      1a - in background
      1b - capture server_pid

2 - enable query using curl

3 - shutdown server with server pid captured in server_pid

* Punctuation in README.md

Fix a punctuation issue in README.  While this is a valid change to improve language, it  is really a decoy to trigger rerunning a test that failed due to a SEV.

* Extend timeout for run-readme-pr-mps.yml

Readme run on M1 with MPS takes over 30 minutes, and may be hitting default timeout.  Extending timeout.

---------

Co-authored-by: Jack-Khuu <[email protected]>
  • Loading branch information
mikekgfb and Jack-Khuu authored Dec 9, 2024
1 parent 8782542 commit dacabcd
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 3 deletions.
1 change: 1 addition & 0 deletions .github/workflows/run-readme-pr-mps.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ jobs:
uses: pytorch/test-infra/.github/workflows/macos_job.yml@main
with:
runner: macos-m1-14
timeout-minutes: 50
script: |
conda create -y -n test-readme-mps-macos python=3.10.11 llvm-openmp
conda activate test-readme-mps-macos
Expand Down
8 changes: 5 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,8 @@ python3 torchchat.py server llama3.1
```
[skip default]: end

[shell default]: python3 torchchat.py server llama3.1 & server_pid=$!

In another terminal, query the server using `curl`. Depending on the model configuration, this query might take a few minutes to respond.

> [!NOTE]
Expand All @@ -244,8 +246,6 @@ Setting `stream` to "true" in the request emits a response in chunks. If `stream

**Example Input + Output**

[skip default]: begin

```
curl http://127.0.0.1:5000/v1/chat/completions \
-H "Content-Type: application/json" \
Expand All @@ -265,12 +265,14 @@ curl http://127.0.0.1:5000/v1/chat/completions \
]
}'
```
[skip default]: begin
```
{"response":" I'm a software developer with a passion for building innovative and user-friendly applications. I have experience in developing web and mobile applications using various technologies such as Java, Python, and JavaScript. I'm always looking for new challenges and opportunities to learn and grow as a developer.\n\nIn my free time, I enjoy reading books on computer science and programming, as well as experimenting with new technologies and techniques. I'm also interested in machine learning and artificial intelligence, and I'm always looking for ways to apply these concepts to real-world problems.\n\nI'm excited to be a part of the developer community and to have the opportunity to share my knowledge and experience with others. I'm always happy to help with any questions or problems you may have, and I'm looking forward to learning from you as well.\n\nThank you for visiting my profile! I hope you find my information helpful and interesting. If you have any questions or would like to discuss any topics, please feel free to reach out to me. I"}
```

[skip default]: end

[shell default]: kill ${server_pid}

</details>

Expand Down Expand Up @@ -664,6 +666,6 @@ awesome libraries and tools you've built around local LLM inference.

torchchat is released under the [BSD 3 license](LICENSE). (Additional
code in this distribution is covered by the MIT and Apache Open Source
licenses.) However you may have other legal obligations that govern
licenses.) However, you may have other legal obligations that govern
your use of content, such as the terms of service for third-party
models.

0 comments on commit dacabcd

Please sign in to comment.