Quarto fails to recognize Python virtual environment

[aronvandepol]

Description

I have recently switched to astral UV to manage python for me. I have tried to make it work nicely with Quarto and setup a testing script to do this. I create a virtual env with UV that my quarto project will use. this is located at .venv dir next to the quarto qmd file.

I manually install jupyter using uv pip install jupyter to ensure the venv can work with quarto.

I have tried the following configs to make quarto pick this up:

1. (Set executable)

---
title: "Testing UV with Quarto"
execute:
  python: ".venv/bin/python"
  external: true

---

## Testing Pandas with UV

```{python} 
#| echo: false
#| code-fold: true
!uv pip install pandas torch
```


```{python}
import pandas as pd

# Create a sample DataFrame
data = pd.DataFrame({
    "Name": ["Alice", "Bob", "Charlie", "David"],
    "Age": [24, 27, 22, 32],
    "City": ["New York", "Los Angeles", "Chicago", "Houston"]
})

# Display the DataFrame
print("Sample DataFrame:")
print(data)
```

```{python} 
import torch
x = torch.rand(3,3)
print(x)
```

quarto render results in a 'There is an unactivated Python environment in .venv. Did you forget to activate it?' error. Alongside the fact that the packages weren't installed.

I then adjusted the _quarto.yml

project:
  type: website
  pre-render:
  - "./set.sh"

this calls the prerender script:

export QUARTO_PYTHON=.venv/bin/python

calling quarto render with this still results in it not using the environment.

I then came across the _environtment file which i tried to setup.

_environment

QUARTO_PYTHON=.venv/bin/python

Again, we get the same error that the python venv is not being used.

When in my terminal I source the venv directly using source ./venv/bin/activate and run quarto render still the same error occurs.

Only when i call" export QUARTO_PYTHON=.venv/bin/python directly before running quarto render will the test file run. The necessary packages get installed first through !uv pip install pandas torch and the code and renders into a html file without errors.

My question is if there is a different way, preferably through Quarto, either the project _quarto.yml or the _environment file to set this? Perhaps I am doing something wrong? The idea is that I don't have to call export QUARTO_PYTHON=.venv/bin/python in my terminal everytime before rendering.

Any help is appreciated!

[mkdjr]

Thanks for your post! I just got my setup working so I'll share what I did and maybe that will work for others!

View the kernels registered with jupyter: uvx jupyter kernelspec list. You should see something like this (angle brackets indicate information that will be specific to your setup):

Available kernels:
  <kernel_name>   <kernel_directory_path>

View the content of the kernel configuration file: cat <kernel_path>/kernel.json:

   1   │ {
   2   │  "argv": [
   3   │   <kernel_python_executable_path>,
   4   │   "-Xfrozen_modules=off",
   5   │   "-m",
   6   │   "ipykernel_launcher",
   7   │   "-f",
   8   │   "{connection_file}"
   9   │  ],
  10   │  "display_name": <kernel_name>,
  11   │  "language": "python",
  12   │  "metadata": {
  13   │   "debugger": true
  14   │  }
  15   │ }

I then created an _environment.yml file in the root of my quarto project directory that sets PYTHON_PATH=<kernel_python_executable_path>.

I think the reason it wasn't working for me previously was because I had configured quarto to look at a different python path than what the kernel was using. I also made sure to specify which kernel I wanted the document code to be executed with:

---
jupyter: <kernel-name>
---

[cderv]

I am surprised this needs to be done, but I think this is because your are using a global jupyter install using uvx jupyter.

Using _environment.yml is definitely a good way to set an env var for the project.Though, quarto respect QUARTO_PYTHON to bypass auto detection, and not PYTHON_PATH - so maybe this is not for quarto directly you do this ?

Specifying a kernel is great to avoid error, but could cause reproducibility issue or collaboration issue. We need to make this available at project level probably 🤔

[cderv]

@aronvandepol where did you find mention of this ?

execute:
  python: ".venv/bin/python"
  external: true

I am not sure this is supposed to work... 🤔

calling quarto render with this still results in it not using the environment.

pre-render script is not meant to set some state for the rendering. This won't be the same process. So you can't set env var this way. See about https://quarto.org/docs/projects/environment.html for project

This is how you should be able to configure a QUARTO_PYTHON for your project.

When in my terminal I source the venv directly using source ./venv/bin/activate and run quarto render still the same error occurs.

However, this should work; If you activate the venv before running quarto render, the right python version should be used. We described this in https://quarto.org/docs/projects/virtual-environments.html#using-venv

We do this in our own CI workflow for testing. Activate the environment, then call quarto render. Possibly for uv managed project uv run is also an option: https://docs.astral.sh/uv/concepts/projects/run/

[aronvandepol]

@aronvandepol where did you find mention of this ?

execute:
python: ".venv/bin/python"
external: true

I am not sure this is supposed to work... 🤔

I tried my best to backtrack where I got this idea/suggestion but could not find it, Sorry. See it as an error and misunderstanding on my part.

I think your suggestion of adding it in the CI workflow makes sense. Adding it to load before build seems sensible and would work when I rebuild on my server. I will certainly implement this.

While not a big deal on its own, I don't call a build script to preview or render my site locally, but do source my venv. So I'm confused why it didn't catch it automatically as indeed described in the docs you mentioned. I will do some more testing to see if I can replicate this behavior as its been since October that I tried to make it work the last time.

I will also give @mkdjr suggestion a try soon see how that works. But seeing your comment there perhaps the best workflow here is indeed locally sourcing your venv (again assuming that this should work) and on the server side when building have it source it through a build script.

Thanks for your reply and will update the thread here with findings.

[dmi3kno]

Could we also make sure that quarto check detects the local uv environment? It would be nice to be able to auto-activate the environment on running the quarto render or quarto preview in the directory where the uv environment is present

[mcanouil]

Quarto is CLI.
you don't need the Python package or R package.

uv run is to run arbitrary commands in the context of the uv environment.

uv run echo "Hello, world!" will also work.

[dmi3kno]

Ah, my bad. uv tool run quarto render would treat quarto as a python package to be installed in an isolated environment. But then it would not see the local project environment to render the report I guess. Need to read up on run vs tool run distinction

[mcanouil]

To be clear, I do think that using explicit handling of environment such as using uv run is safer and more reliable than relying on automatic detection or automatic activation of environment (which Quarto does not do activations, but IDEs do under certain conditions)

[cderv]

I can see how appealing this would be for quarto to handle uv however, I don't think this is the right way, or the problem.

uv is creating a virtual environment, so Quarto really needs to detect the virtual environment correctly. Quarto also needs to find a Python version with Jupyter to work.

We have had issues with Python detection in the past. We tried to be clever, but it had an issue not finding activated virtual environments, for example.

So the current way that is the safest to me is: Activate your virtualenv and then run quarto. This is because Quarto does see an activated virtualenv and uses it in priority.

uv run <cmd> does exactly that: (1) activate the virtual env for the project (2) and run the command.
After running uv run the virtual env is not activated in terminal.

You could also manually activate the environment in your shell. Then, Quarto would find the right Python version. We do that for our testing environment in quarto for example.

For the Quarto project, you could probably also leverage the environment to set the QUARTO_PYTHON environment variable to the version of the uv project.

Anyhow, there are enough ways to make that happen correctly that I am not sure it is worth trying to add specific support for uv—we don't really want to run the uv CLI from Quarto itself.

Hope it helps

[dmi3kno]

Thank you very much for this clarification!