docs: Simplify getting started doc #1839
Conversation
Focus on just venv
raghotham
requested review from
ashwinb,
yanxi0830,
hardikjshah,
dltn,
dineshyv,
vladimirivic,
sixianyi0721,
ehhuang,
terrytangyuan,
SLR722 and
leseb
as code owners
facebook-github-bot
added
the
CLA Signed
|
|
||
| The API is **exactly identical** for both clients. | ||
| Install uv | ||
| ```bash |
There was a problem hiding this comment.
I'd suggest this below:
For macOS and Linux:
curl -LsSf https://astral.sh/uv/install.sh | shFor Windows:
Use
irmto download the script and execute it withiex:powershell -ExecutionPolicy ByPass -c "irm https://astral.sh/uv/install.ps1 | iex"
There was a problem hiding this comment.
This is great @raghotham I was planning on handling this today actually along with some broader doc enhancements I'm doing.
A few small suggestions to just add some links and remove some things.
Also looks like the linter needs to be rerun.
Co-authored-by: Francisco Arceo <[email protected]>
Co-authored-by: Francisco Arceo <[email protected]>
| pip install llama_stack | ||
| llama stack build --template ollama --image-type <conda|venv> | ||
| python inference.py | ||
| python lstest.py |
There was a problem hiding this comment.
added file names as comments so that they can be run with that name - maybe not needed?
There was a problem hiding this comment.
Actually, not a bad idea! Users that try this and find issues will probably refer to the script this way.
| ``` | ||
|
|
||
| Then you can start the server using the container tool of your choice. For example, if you are running Docker you can use the following command: | ||
| Build llama stack for ollama |
There was a problem hiding this comment.
For a new user, we need to add some explanation of what this is and why do we need to "build in python" ?
| Docker containers run in their own isolated network namespaces on Linux. To allow the container to communicate with services running on the host via `localhost`, you need `--network=host`. This makes the container use the host’s network directly so it can connect to Ollama running on `localhost:11434`. | ||
|
|
||
| Linux users having issues running the above command should instead try the following: | ||
| Run llama stack |
There was a problem hiding this comment.
Same here what does this run stack mean, may be 1-2 lines explaining that we need to run a server locally while this can be a hosted endpoint later.
| Let's use the `llama-stack-client` CLI to check the connectivity to the server. | ||
|
|
||
| ```bash | ||
| $ llama-stack-client configure --endpoint http://localhost:$LLAMA_STACK_PORT | ||
| > Enter the API key (leave empty if no key is needed): | ||
| llama-stack-client configure --endpoint http://localhost:$LLAMA_STACK_PORT --api-key none |
There was a problem hiding this comment.
LLAMA_STACK_PORT is not mentioned anywhere, lets just hardcode 8321 here for ease
| from llama_stack_client import LlamaStackClient | ||
| agent = Agent(client, | ||
| model=model_id, | ||
| instructions="You are a helpful assistant that can answer questions about the Torchtune project." |
There was a problem hiding this comment.
that can answer questions about the Torchtune project.
drop this probably ?
| stream=True | ||
| ) | ||
| for event in stream: | ||
| print(event) |
There was a problem hiding this comment.
may be pprint instead of print , that way each object is shown properly instead of a not easy to read python object.
from rich.pretty import pprint
| # Streaming example | ||
| print("Streaming ...") | ||
| stream = agent.create_turn( | ||
| messages=[ { |
There was a problem hiding this comment.
ensure you run pre-commit with proper formatting
| ) | ||
| for log in AgentEventLogger().log(response): | ||
| log.print() | ||
| for chunk in stream: |
There was a problem hiding this comment.
why are we doing this type of print, lets just use the AgentEventLogger
|
@hardikjshah @raghotham I started restructuring some of this to make the layout a little easier for new folks. I'm not done yet but I figured I'd make you aware of it as I think this may help link to related items. PR here: #1873 I'll adjust more tonight but, again, wanted to share what I'm doing here. I'll also note that I think for the quickstart we should probably just start with the stack as a library to reduce the number of steps. That said, we could repurpose this for the full comprehensive example. |
Focus on just venv
Add examples for agents and ragagent