how to inherit current R session when using quarto_render from R

[blset]

Description

hello

using knit2pdf at the end of an R script the Rnw templates with latex inherit current session with all libraries loaded

I cannot translate that workflow with quarto

using quarto_render at the end of an R script the qmd files have no access to the current session
I would have to pass individually all objects and load all libraries and source scripts from inside the qmd files

this is very inconvenient

how would you pass the whole session to quarto simply ?

I'm aware the render button in rstudio executes everything in a brand new environment for reproducibility

but I thought using quarto_render would allow the use case of inheriting the current session which is very convenient when sharing is not the goal or to experiment easely

[mcanouil]

You are talking about the R package but the R package is a wrapper around the real tool which is not an R package contrary to knitr or RMarkdown (so it is not loaded in R session, does not use your R session or anything from it)
When you use the quarto::quarto_render() function you are calling Quarto CLI which can run R process on its own.
Quarto CLI computations are isolated.
If you need something from elsewhere you need to make that something available:

• parameters
• cache
• RDS/Rdata files
• etc.

Quarto's document reproducibility is ensured by the fact that by the document itself or the Quarto project.

For short, if you wanted to load R packages, render a Quarto document using the loaded functions, this can't be achieved.

[blset]

ok thanks

looks to me like a loss of feature compared to rmarkdown or knitr
I understand this is so because quarto is multi language

is there any plan for a knit2pdf that would compile from typst instead of Latex ?

this would be great, and keep the facility to pick current R session as in regular knitr/rmarkdown

[mcanouil]

looks to me like a loss of feature compared to rmarkdown or knitr

I don't believe this was ever a feature in the first but more of a side effect of knitr/rmarkdown running in R.

is there any plan for a knit2pdf that would compile from typst instead of Latex ?

This repository is about Quarto not knitr specific or whatever package this function is from.
Quarto does support Typst.

this would be great, and keep the facility to pick current R session as in regular knitr/rmarkdown

This idea to use unknown things from a session goes against the idea of reproducibility and the idea of having self-contained projects, so this is unlikely to ever be possible in Quarto.
(It's also unlikely to be possible because Quarto is language agnostic and does not even need a computing language to work.)

Could you explain what your workflow is exactly and why you need things from an R session that can't live in your document?

[blset]

the workflow is the following
a bunch of R scripts prepare a session with data, functions, libraries, etc..
I have Rnw templates in latex and R chunks inside
I use knit2pdf to generate the pdfs (knitr picks the session)
it's a natural workflow when using R only

quarto is oversized for that simple usage

my workflow is more programmatic I would say, don't need the visual cells and don't use jupyter like approaches as they tend in my opinion to get in the way

I don't aim at reproducibility with sharing to others with uploading in a cloud, I just need reproducibility for my own scripts of course

currently you get typst with R sweave easely only in quarto, there is not knit2dpf using typst instead of latex... I know it's theoretically possible and relatively easy to tweak to do it with knitr, but I did'nt get to it yet

when you put everything in a quarto cell you lose reusability of the scripts for other purpose, the scripts is trapped in the quarto document, ok for reproducibility, but not for reusablity of building block R scripts

in my opinion there should be only minimal R instruction in a Rnw or quarto template, just the necessary to print the tables and the graphs namely, but not anything heavy to compute, and the rest is just writing

thanks

[mcanouil]

Based on what you said, I don't think Quarto is the tool you need/want as it would require from you to change your entire workflow to use external script that you source/load in a code cell (knitr), to use tools such as targets. Additionally, it appears you don't want to write markdown, so Quarto is clearly not the tool you are looking for.

currently you get typst with R sweave easely only in quarto, there is not knit2dpf using typst instead of latex... I know it's theoretically possible and relatively easy to tweak to do it with knitr, but I did'nt get to it yet

Quarto does not use R Sweave (it's for LaTeX/R).

As far as I know there is nothing in R that currently supports Typst.
Again, Quarto CLI is not an R package or R.
Quarto is the one doing the conversion from the markup generated by the code cells via knitr, to the formats.

For reference, as you have opened a feature request for a native support at the knitr level:

• knit2typst yihui/knitr#2401

[cderv]

Thanks a lot for your feedback @blset ! Really appreciated to know more about users' workflow and how they try to use Quarto

the workflow is the following
a bunch of R scripts prepare a session with data, functions, libraries, etc..
I have Rnw templates in latex and R chunks inside
I use knit2pdf to generate the pdfs (knitr picks the session)
It's a natural workflow when using R only

As a developer of knitr and maintainer of R Markdown, I'll take the opportunity to comment on the workflow and compare it with Quarto.

The workflow you have is very different from what Quarto offers. Quarto is closer to how the R Markdown workflow works. A document (respectively .qmd or .Rmd) is expected to contain all computations in cells. They are assumed to be reproducible source files.

When preparations are in a bunch of R scripts, they can be loaded for the document in a cell at the top. There are features to easily load them:

• By just source() them: https://bookdown.org/yihui/rmarkdown-cookbook/source-script.html
• Or by including the content of the file as cell content: https://bookdown.org/yihui/rmarkdown-cookbook/option-code.html

So, regarding your comment

when you put everything in a quarto cell you lose reusability of the scripts for other purpose, the scripts is trapped in the quarto document, ok for reproducibility, but not for reusablity of building block R scripts

They are not trapped. You can do all your code content in an R script and then source it as a script to make computation available for knitr.

To share experience, some even structure their work as an R package for their preparation function and data and load the package inside the document to make everything available easily, and have the package available for easy reusability outside the document.

The document can then be rendered in a background session without problem. The assumption of having a Rmd file be reproducible by containing everything is the default, and it is even bigger with Quarto as rendering a .qmd will always run in a new clean R session.

So, what to do when pre-calculation is needed, when data needs to be shared, or pre-computed ?
In this case, the workflow needs to be adapted to run scripts, save data, and then load them when needed in the document. This can be done manually, or using some other tool for easier caching / saving data, or for managing complex and intensive workflow (like targets mentioned already).

So in a nutshell, yes, using Quarto will require a change in your workflow. It would have been the same if you were using R Markdown, probably. By using Rnw and knit2pdf, the gap is bigger to switch then for R Markdown user.

currently you get typst with R sweave easely only in quarto, there is not knit2dpf using typst instead of latex... I know it's theoretically possible and relatively easy to tweak to do it with knitr, but I did'nt get to it yet

I understand you are only considering Quarto for Typst support. So thank you for opening

• knit2typst yihui/knitr#2401

This is probably to be considered—PR is welcome there, too. We'll discuss it in the issue in the knitr repo.

in my opinion there should be only minimal R instruction in a Rnw or quarto template, just the necessary to print the tables and the graphs namely, but not anything heavy to compute, and the rest is just writing

For this part, I did not completely follow. To have a .Rtyp format, this needs to happen in knitr and there are FR opened there.
For quarto template, I did not follow what you mean. Could you clarify so I can fully understand your feedback ? Thanks!

BTW, Quarto has support for knitr::spin(), which means a report can be generated from R scripts directly as long as it has the correct syntax. See https://quarto.org/docs/computations/render-scripts.html

So quarto render report.R --to typst is something possible. Sharing in case it can help.

[blset]

Hello thank you for your response

By just source() them: https://bookdown.org/yihui/rmarkdown-cookbook/source-script.html
Or by including the content of the file as cell content: https://bookdown.org/yihui/rmarkdown-cookbook/option-code.html

yes but there is a hitch, you have to set very carefully the path of the files, when you run the scripts in advance you don't have to worry about the file paths (they are set independently), but once they are in a cell it's more complicated you can use here library but is so much simpler to inherit current session

They are not trapped. You can do all your code content in an R script and then source it as a script to make computation available for knitr.

yes ok but then you need to use cache for long prepared scripts, and cache always reserve surprises it's not always in sync, you don't see clearly what's inside and it is a layer of complexity, so much easier to just inherit the session

I understand you are only considering Quarto for Typst support. So thank you for opening

indeed I tried quarto but knitr / session / latex is more natural for my use case. I retried quarto because it can compile to typst

I would like to switch to typst instead of latex, and so I am looking for knit / session /typst.

also I would like to be able to compose the template file not in markdown but directly in typst,
with r code embedded as usual

For this part, I did not completely follow. To have a .Rtyp format, this needs to happen in knitr and there are FR opened there.
For quarto template, I did not follow what you mean. Could you clarify so I can fully understand your feedback ? Thanks!

what I mean is that in my workflow, all the heavy work is done in advance by independent programs and I pass the session to knit2pdf and it is available in Rnw templates, and in these templates I just write text and
style tables and graphics from data already available, nothing more
no real computation, nothing that would need to be cached

if something goes wrong I can browse the object in the R session and everything can be fixed quickly, with debug points if necessary, even re running part of the scripts only if needed. very agile workflow.

you don't put debug points in caches you don't slice the cache... or then your heavy code is disseminated in cells, a total nightmare.

thank you very much
best regards

[cderv]

Thanks for your additional details.

What you are looking at then is not Quarto. This is indeed knitr support for .typ so that it can be knitted before running Typst on them. knitr repo is the place of that.

yes but there is a hitch, you have to set very carefully the path of the files, when you run the scripts in advance you don't have to worry about the file paths (they are set independently), but once they are in a cell it's more complicated you can use here library but is so much simpler to inherit current session

Sorry if I have misunderstood your workflow. Sourcing before calling knit or running inside a cell should be the same as long as the working directory is the same. Now I understand this may not be the case.

yes ok but then you need to use cache for long prepared scripts, and cache always reserve surprises it's not always in sync, you don't see clearly what's inside and it is a layer of complexity, so much easier to just inherit the session

Caching objects does not mean not recomputing. I use this term as a way to name the fact of

flowchart LR
    A[Running some code] --> B[Saving object]
    B --> C[Reusing object by loading them]
    C --> D[Rendering document]

Loading

Doing everything in the current R session is only "caching in memory" which works if the R session is shared - which is not always the case.

This is a layer of complexity, which can be required in some situations, and also it can be simplified with some tools. targets is one of them.

if something goes wrong I can browse the object in the R session and everything can be fixed quickly, with debug points if necessary, even re running part of the scripts only if needed. very agile workflow.
you don't put debug points in caches you don't slice the cache... or then your heavy code is disseminated in cells, a total nightmare.

The cached object is only there to pass from an R script run in one session to rendering the document in another. If you need to debug, this would happen in your script.

But this is also where using the R package structure to organize code has benefits. The document could use only functions from this package, and any function can be easily debugged as a package.

Anyhow, just wanted to add precision on what I meant. All of this is a matter of workflow, and there is no best one; good ones vary by person. This is a personal matter.

Using Quarto is like calling another external tool when you are inside R. So, if your workflow is not compatible with this, using R package only will be the best. Just the knit to typst feature does not exist yet.

You could use R markdown with custom formats, but depending on which R package you use, they may not know what to do in the Typst context. For example, gt for producing an HTML table would not output anything in Typst.

Let's continue in knitr repo !

Thanks a lot !