Skip to content

Consistent linebreaks throughout #887

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from
336 changes: 224 additions & 112 deletions pkg_building.Rmd

Large diffs are not rendered by default.

34 changes: 25 additions & 9 deletions pkg_ci.Rmd
Original file line number Diff line number Diff line change
@@ -42,15 +42,23 @@ R packages should have CI for all operating systems (Linux, Mac OSX, Windows) wh

- Anything with file system / path calls

In case of any doubt regarding the applicability of these criteria to your package, it's better to add CI for all operating systems. Most CI services standards setups for R packages allow this with not much hassle.
In case of any doubt regarding the applicability of these criteria to your package, it's better to add CI for all operating systems.
Most CI services standards setups for R packages allow this with not much hassle.

## Which continuous integration service(s)? {#whichci}

There are a number of continuous integration services, including standalone services (CircleCI, AppVeyor), and others integrated into code hosting or related services (GitHub Actions, GitLab, AWS Code Pipeline). Different services support different operating system configurations.
There are a number of continuous integration services, including standalone services (CircleCI, AppVeyor), and others integrated into code hosting or related services (GitHub Actions, GitLab, AWS Code Pipeline).
Different services support different operating system configurations.

[GitHub Actions](https://github.com/features/actions) is a convenient option for many R developers who already use GitHub as it is integrated into the platform and supports all needed operating Systems. There are [actions supported for the R ecosystem](https://github.com/r-lib/actions/), as well and first-class support in the [{usethis}](https://usethis.r-lib.org/reference/github_actions.html) package. All packages submitted to rOpenSci for peer review are checked by our own [`pkgcheck` system](https://docs.ropensci.org/pkgcheck), described further in the [Guide for Authors](#authors-guide). These checks are also provided as a GitHub Action in the [`ropensci-review-tools/pkgcheck-action` repository](https://github.com/ropensci-review-tools/pkgcheck-action). Packages authors are encouraged to use that action to confirm prior to submission that a package passes all of our checks. See [our blog post](https://ropensci.org/blog/2022/02/01/pkgcheck-action/) for more information.
[GitHub Actions](https://github.com/features/actions) is a convenient option for many R developers who already use GitHub as it is integrated into the platform and supports all needed operating Systems.
There are [actions supported for the R ecosystem](https://github.com/r-lib/actions/), as well and first-class support in the [{usethis}](https://usethis.r-lib.org/reference/github_actions.html) package.
All packages submitted to rOpenSci for peer review are checked by our own [`pkgcheck` system](https://docs.ropensci.org/pkgcheck), described further in the [Guide for Authors](#authors-guide).
These checks are also provided as a GitHub Action in the [`ropensci-review-tools/pkgcheck-action` repository](https://github.com/ropensci-review-tools/pkgcheck-action).
Packages authors are encouraged to use that action to confirm prior to submission that a package passes all of our checks.
See [our blog post](https://ropensci.org/blog/2022/02/01/pkgcheck-action/) for more information.

[usethis supports CI setup for other systems](https://usethis.r-lib.org/reference/ci.html), though these functions are soft-deprecated. rOpenSci also supports the [circle](https://docs.ropensci.org/circle/) package, which aids in setting up CircleCI pipelines, and the [tic](https://docs.ropensci.org/tic/) package for building more complicated CI pipelines.
[usethis supports CI setup for other systems](https://usethis.r-lib.org/reference/ci.html), though these functions are soft-deprecated.
rOpenSci also supports the [circle](https://docs.ropensci.org/circle/) package, which aids in setting up CircleCI pipelines, and the [tic](https://docs.ropensci.org/tic/) package for building more complicated CI pipelines.

#### Testing using different versions of R {#testing-using-different-versions-of-r}

@@ -66,7 +74,8 @@ If you develop a package depending on or intended for Bioconductor, you might fi

You can use these tips to minimize build time on CI:

- Cache installation of packages. The default [r-lib/actions workflows](https://github.com/r-lib/actions) do this.
- Cache installation of packages.
The default [r-lib/actions workflows](https://github.com/r-lib/actions) do this.

#### System dependencies {#sysdeps-ci}

@@ -78,11 +87,14 @@ We recommend [moving away from Travis](https://ropensci.org/technotes/2020/11/19

### AppVeyor CI (Windows) {#app-veyor-ci-windows}

For continuous integration on Windows, see [R + AppVeyor](https://github.com/krlmlr/r-appveyor). Set it up using `usethis::use_appveyor()`.
For continuous integration on Windows, see [R + AppVeyor](https://github.com/krlmlr/r-appveyor).
Set it up using `usethis::use_appveyor()`.

Here are tips to minimize AppVeyor build time:

- Cache installation of packages. [Example in a config file](https://github.com/r-lib/usethis/blob/2c52c06373849d52f78a26c5a0e080f518a2f825/inst/templates/appveyor.yml#L13). It'll already be in the config file if you set AppVeyor CI up using `usethis::use_appveyor()`.
- Cache installation of packages.
[Example in a config file](https://github.com/r-lib/usethis/blob/2c52c06373849d52f78a26c5a0e080f518a2f825/inst/templates/appveyor.yml#L13).
It'll already be in the config file if you set AppVeyor CI up using `usethis::use_appveyor()`.

- Enable [rolling builds](https://www.appveyor.com/docs/build-configuration/#rolling-builds).

@@ -111,11 +123,15 @@ If you run coverage on several CI services [the results will be merged](https://

## Even more CI: OpenCPU {#even-more-ci-open-cpu}

After transfer to rOpenSci's "ropensci" GitHub organization, each push to the repo will be built on OpenCPU and the person committing will receive a notification email. This is an additional CI service for package authors that allows for R functions in packages to be called remotely via [https://ropensci.ocpu.io/](https://ropensci.ocpu.io/) using the [opencpu API](https://www.opencpu.org/api.html#api-json). For more details about this service, consult the OpenCPU [help page](https://www.opencpu.org/help.html) that also indicates where to ask questions.
After transfer to rOpenSci's "ropensci" GitHub organization, each push to the repo will be built on OpenCPU and the person committing will receive a notification email.
This is an additional CI service for package authors that allows for R functions in packages to be called remotely via [https://ropensci.ocpu.io/](https://ropensci.ocpu.io/) using the [opencpu API](https://www.opencpu.org/api.html#api-json).
For more details about this service, consult the OpenCPU [help page](https://www.opencpu.org/help.html) that also indicates where to ask questions.

## Even more CI: rOpenSci docs {#rodocsci}

After transfer to rOpenSci's "ropensci" GitHub organization, a pkgdown website will be built for your package after each push to the GitHub repo. You can find the status of these builds at `https://ropensci.r-universe.dev/ui#packages` and in the [commit status](https://ropensci.org/blog/2021/09/03/runiverse-docs/#how-it-works). The website build will use your pkgdown config file if you have one, except for the styling that will use the [`rotemplate` package](https://github.com/ropensci-org/rotemplate/).
After transfer to rOpenSci's "ropensci" GitHub organization, a pkgdown website will be built for your package after each push to the GitHub repo.
You can find the status of these builds at `https://ropensci.r-universe.dev/ui#packages` and in the [commit status](https://ropensci.org/blog/2021/09/03/runiverse-docs/#how-it-works).
The website build will use your pkgdown config file if you have one, except for the styling that will use the [`rotemplate` package](https://github.com/ropensci-org/rotemplate/).

Please report bugs, questions and feature requests about the central builds and about the template at [https://github.com/ropensci-org/rotemplate/](https://github.com/ropensci-org/rotemplate/).

15 changes: 10 additions & 5 deletions pkg_security.Rmd
Original file line number Diff line number Diff line change
@@ -15,7 +15,8 @@ We recommend the article [Ten quick tips for staying safe online](https://journa

## GitHub access security {#git-hub-access-security}

- We recommend you [secure your GitHub account with two-factor (authentication) 2FA](https://help.github.com/articles/securing-your-account-with-two-factor-authentication-2fa/). It is *compulsory* for all ropensci GitHub organization members and outside collaborators so make sure to enable it before your package is approved.
- We recommend you [secure your GitHub account with two-factor (authentication) 2FA](https://help.github.com/articles/securing-your-account-with-two-factor-authentication-2fa/).
It is *compulsory* for all ropensci GitHub organization members and outside collaborators so make sure to enable it before your package is approved.

- We also recommend you regularly check who has access to your package repository, and that you prune any unused access (such as from former collaborators).

@@ -25,29 +26,33 @@ We recommend the article [Ten quick tips for staying safe online](https://journa

## Secrets in packages {#pkgsecrets}

This section contains guidance for when you develop a package interacting with a web resource requiring credentials (API keys, tokens, etc.). Also refer to [the `httr` vignette about sharing secrets](https://httr.r-lib.org/articles/secrets.html).
This section contains guidance for when you develop a package interacting with a web resource requiring credentials (API keys, tokens, etc.).
Also refer to [the `httr` vignette about sharing secrets](https://httr.r-lib.org/articles/secrets.html).

### Secrets in packages and user protection {#secrets-in-packages-and-user-protection}

Say your package needs an API key for making requests on behalf of users of your package.

- In your package documentation, guide the user so the API key doesn't end up in the .Rhistory/script of users of your package.

- Encourage the use of environment variables to store the API key (or even remove the possibility to pass it as an argument to the functions?). You could link [to this intro to startup files](https://rstats.wtf/r-startup.html) and [`usethis::edit_r_environ()`](https://usethis.r-lib.org/reference/edit.html).
- Encourage the use of environment variables to store the API key (or even remove the possibility to pass it as an argument to the functions?).
You could link [to this intro to startup files](https://rstats.wtf/r-startup.html) and [`usethis::edit_r_environ()`](https://usethis.r-lib.org/reference/edit.html).

- Or your package could depend on, or encourage the use of, [`keyring` to help user store variables](https://github.com/r-lib/keyring#readme) in the specific OS' credential stores (more secure than .Renviron): i.e. you'd create a function for setting the key, and have another one for retrieving the key; or the user would write `Sys.setenv(SUPERSECRETKEY = keyring::key_get("myservice"))` at the beginning of their script.

- Do not print the API key even in verbose mode in any message, warning, error.

- In the GitHub issue template, it should be stated not to share any credentials. If an user of your package accidentally shares credentials in an issue, make sure they're aware of that so they can revoke the key (i.e. ask them explicitly in an answer whether they realized they shared their key).
- In the GitHub issue template, it should be stated not to share any credentials.
If an user of your package accidentally shares credentials in an issue, make sure they're aware of that so they can revoke the key (i.e. ask them explicitly in an answer whether they realized they shared their key).

### Secrets in packages and development {#secrets-in-packages-and-development}

You'll need to protect your secrets as you protect secrets of users, but there's more to take into account and keep in mind.

#### Secrets and recorded requests in tests {#secrets-and-recorded-requests-in-tests}

If you use [`vcr`](https://docs.ropensci.org/vcr/) or [`httptest`](https://enpiar.com/r/httptest/) in tests for caching API responses, you need to make sure the recorded requests / fixtures do not contain secrets. Refer to [`vcr` security guidance](https://books.ropensci.org/http-testing/security-chapter.html) and [`httptest` guidance "Redacting and Modifying Recorded Requests"](https://enpiar.com/r/httptest/articles/redacting.html), and inspect your recorded requests / fixtures before committing them the first time to be sure you got the setup right.
If you use [`vcr`](https://docs.ropensci.org/vcr/) or [`httptest`](https://enpiar.com/r/httptest/) in tests for caching API responses, you need to make sure the recorded requests / fixtures do not contain secrets.
Refer to [`vcr` security guidance](https://books.ropensci.org/http-testing/security-chapter.html) and [`httptest` guidance "Redacting and Modifying Recorded Requests"](https://enpiar.com/r/httptest/articles/redacting.html), and inspect your recorded requests / fixtures before committing them the first time to be sure you got the setup right.

`vcr` being an rOpenSci package, you can post any question you might have to [rOpenSci forum](https://discuss.ropensci.org/).

57 changes: 40 additions & 17 deletions softwarereview_author.Rmd
Original file line number Diff line number Diff line change
@@ -14,49 +14,72 @@ This concise guide presents the software peer review process for you as a packag
- Do you expect to maintain your package for at least 2 years, or to be able to identify a new maintainer?
- Consult our [policies](#policies) see if your package meets our criteria for fitting into our suite and does not overlap with other packages.
- If you are unsure whether a package meets our criteria, feel free to open an issue as a pre-submission inquiry to ask if the package is appropriate.
- [Example response regarding overlap](https://github.com/ropensci/software-review/issues/199#issuecomment-375358362). Also consider adding some points about similar packages to your [package documentation](#docs-general).
- Please consider the best time in your package's development to submit. Your package should be sufficiently mature so that reviewers are able to review all essential aspects, but keep in mind that review may result in major changes.
- We strongly suggest submitting your package for review *before* publishing on CRAN or submitting a software paper describing the package to a journal. Review feedback may result in major improvements and updates to your package, including renaming and breaking changes to functions.
- [Example response regarding overlap](https://github.com/ropensci/software-review/issues/199#issuecomment-375358362).
Also consider adding some points about similar packages to your [package documentation](#docs-general).
- Please consider the best time in your package's development to submit.
Your package should be sufficiently mature so that reviewers are able to review all essential aspects, but keep in mind that review may result in major changes.
- We strongly suggest submitting your package for review *before* publishing on CRAN or submitting a software paper describing the package to a journal.
Review feedback may result in major improvements and updates to your package, including renaming and breaking changes to functions.
- Do not submit your package for review while it or an associated manuscript is also under review at another venue, as this may result in conflicting requests for changes.
- Please also consider the time and effort needed to respond to reviews: think about your availability or that of your collaborators in the next weeks and months following a submission. Note that reviewers are volunteers, and we ask that you respect their time and effort by responding in a timely and respectful manner.
- If you use [repostatus.org badges](https://www.repostatus.org/) (which we recommend), submit when you're ready to get an *Active* instead of *WIP* badge. Similarly, if you use [lifecycle badges](https://www.tidyverse.org/lifecycle/), submission should happen when the package is *Stable*.
- For any submission or pre-submission inquiry the README of your package should provide enough information about your package (goals, usage, similar packages) for the editors to assess its scope without having to install the package. Even better, set up a pkgdown website for allowing more detailed assessment of functionality online.
- Please also consider the time and effort needed to respond to reviews: think about your availability or that of your collaborators in the next weeks and months following a submission.
Note that reviewers are volunteers, and we ask that you respect their time and effort by responding in a timely and respectful manner.
- If you use [repostatus.org badges](https://www.repostatus.org/) (which we recommend), submit when you're ready to get an *Active* instead of *WIP* badge.
Similarly, if you use [lifecycle badges](https://www.tidyverse.org/lifecycle/), submission should happen when the package is *Stable*.
- For any submission or pre-submission inquiry the README of your package should provide enough information about your package (goals, usage, similar packages) for the editors to assess its scope without having to install the package.
Even better, set up a pkgdown website for allowing more detailed assessment of functionality online.
- At the submission stage, all major functions should be stable enough to be fully documented and tested; the README should make a strong case for the package.
- Your README file should strive to explain your package's functionality and aims, assuming readers have little to no domain knowledge. All technical tems, including references to other software, should be clarified.
- Your README file should strive to explain your package's functionality and aims, assuming readers have little to no domain knowledge.
All technical tems, including references to other software, should be clarified.
- Your package will continue to evolve after review, the chapter on *Package evolution* [provides guidance about the topic](#evolution).

## Preparing for Submission {#preparing-for-submission}

- Read and follow [our packaging style guide](#building), [reviewer's guide](#preparereview) to ensure your package meets our style and quality criteria.
- Feel free to ask any questions about the process, or your specific package, in our [Discussion Forum](https://discuss.ropensci.org).
- All submissions are automatically checked by our [pkgcheck](https://docs.ropensci.org/pkgcheck/) system to ensure packages follow our guidelines. All authors are expected to have run [the main `pkgcheck` function](https://docs.ropensci.org/pkgcheck/reference/pkgcheck.html) locally to confirm that the package is ready to be submitted. Alternatively, an even easier way to ensure a package is ready for submission is to use [the `pkgcheck` GitHub Action](https://github.com/ropensci-review-tools/pkgcheck-action) to run `pkgcheck` as a GitHub Action, as described in [our blog post](https://ropensci.org/blog/2022/02/01/pkgcheck-action/).
- If your package requires unusual system dependencies (see [*Packaging Guide*](#pkgdependencies)) for our GitHub Action to pass, please submit a pull request adding them to [our base Dockerfile](https://github.com/ropensci-review-tools/pkgcheck/blob/main/Dockerfile).
- All submissions are automatically checked by our [pkgcheck](https://docs.ropensci.org/pkgcheck/) system to ensure packages follow our guidelines.
All authors are expected to have run [the main `pkgcheck` function](https://docs.ropensci.org/pkgcheck/reference/pkgcheck.html) locally to confirm that the package is ready to be submitted.
Alternatively, an even easier way to ensure a package is ready for submission is to use [the `pkgcheck` GitHub Action](https://github.com/ropensci-review-tools/pkgcheck-action) to run `pkgcheck` as a GitHub Action, as described in [our blog post](https://ropensci.org/blog/2022/02/01/pkgcheck-action/).
- If your package requires unusual system dependencies (see [*Packaging Guide*](#pkgdependencies)) for our GitHub Action to pass, please submit a pull request adding them to [our base Dockerfile](https://github.com/ropensci-review-tools/pkgcheck/blob/main/Dockerfile).
- If there are any aspects of `pkgcheck` which your package is unable to pass, please explain reasons in your submission template.
- If you feel your package is in scope for the
[Journal of Open-Source Software](https://joss.theoj.org/) (JOSS), do not submit it to JOSS consideration until after the rOpenSci review process is over: if your package is deemed in scope by JOSS editors, only the accompanying short paper would be reviewed, (not the software that will have been extended reviewed by rOpenSci by that time). Not all rOpenSci packages will meet the criteria for JOSS.
[Journal of Open-Source Software](https://joss.theoj.org/) (JOSS), do not submit it to JOSS consideration until after the rOpenSci review process is over: if your package is deemed in scope by JOSS editors, only the accompanying short paper would be reviewed, (not the software that will have been extended reviewed by rOpenSci by that time).
Not all rOpenSci packages will meet the criteria for JOSS.

## The Submission Process {#the-submission-process}

- Software is submitted for review by [opening a new issue](https://github.com/ropensci/software-review/issues/new/choose) in the software review repository and filling out the template.
- The template begins with a section which includes several HTML-styled variables (`<!---variable--->`). These are used by our `ropensci-review-bot`, and must be left in place, with values filled between the indicated start and end points, like this:
- The template begins with a section which includes several HTML-styled variables (`<!---variable--->`).
These are used by our `ropensci-review-bot`, and must be left in place, with values filled between the indicated start and end points, like this:

```{bash, eval=F}
<!---variable--->insert value here<!---end-variable>
```

- Communication between authors, reviewers and editors will first and foremost take place on GitHub so that the review thread can serve as a full record of the review. You may choose to contact the editor by email or Slack if private consultation is needed (e.g., asking how to respond to a reviewer question). Do not contact reviewers off-thread without asking them in the GitHub thread whether they agree to it.
- Communication between authors, reviewers and editors will first and foremost take place on GitHub so that the review thread can serve as a full record of the review.
You may choose to contact the editor by email or Slack if private consultation is needed (e.g., asking how to respond to a reviewer question).
Do not contact reviewers off-thread without asking them in the GitHub thread whether they agree to it.
- *When submitting a package please make sure your GitHub notification settings make it unlikely you will miss a comment.*
- Packages are automatically checked on submission by [our `pkgcheck` system](https://docs.ropensci.org/pkgcheck), which will confirm whether or not a package is ready to be reviewed.
- Submitted packages can be hosted in the main/default branch, or any other non-default branch. In the latter case, it is encouraged, but not required, to submit the package via a dedicated `ropensci-software-review` branch.
- Submitted packages can be hosted in the main/default branch, or any other non-default branch.
In the latter case, it is encouraged, but not required, to submit the package via a dedicated `ropensci-software-review` branch.
- For submissions in non-default branches, the "Repository" URL in the submission template should be the full URL to the review branch, like `https://github.com/my/repo/tree/ropensci-software-review`.

## The Review Process {#the-review-process}

- An [editor](#editors) will review your submission within 5 business days and respond with next steps. The editor may assign the package to reviewers, request that the package be updated to meet minimal criteria before review, or reject the package due to lack of fit or overlap.
- If your package meets minimal criteria, the editor will assign 1-3 reviewers. They will be asked to provide reviews as comments on your issue within 3 weeks.
- We ask that you respond to reviewers' comments within 2 weeks of the last-submitted review, but you may make updates to your package or respond at any time. Your response should include a link to the updated [NEWS.md](#news) of your package. Here is [an author response example](https://github.com/ropensci/software-review/issues/160#issuecomment-355043656). Once the response is commited, [submit it using the bot](bot_cheatsheet.html#submit-response-to-reviewers). We encourage ongoing conversations between authors and reviewers. See the [reviewing guide](#reviewerguide) for more details.
- An [editor](#editors) will review your submission within 5 business days and respond with next steps.
The editor may assign the package to reviewers, request that the package be updated to meet minimal criteria before review, or reject the package due to lack of fit or overlap.
- If your package meets minimal criteria, the editor will assign 1-3 reviewers.
They will be asked to provide reviews as comments on your issue within 3 weeks.
- We ask that you respond to reviewers' comments within 2 weeks of the last-submitted review, but you may make updates to your package or respond at any time.
Your response should include a link to the updated [NEWS.md](#news) of your package.
Here is [an author response example](https://github.com/ropensci/software-review/issues/160#issuecomment-355043656).
Once the response is commited, [submit it using the bot](bot_cheatsheet.html#submit-response-to-reviewers).
We encourage ongoing conversations between authors and reviewers.
See the [reviewing guide](#reviewerguide) for more details.
- Any time package changes are likely to alter the results of [the automated `pkgcheck` checks](https://docs.ropensci.org/pkgcheck), authors can request a re-check with the command, `@ropensci-review-bot check package`.
- Please notify us immediately if you are no longer able to maintain your package or to respond to reviews. You will then be expected to either retract a submission, or to find alternative package maintainers. You can also discuss maintenance issues in the rOpenSci slack workspace.
- Please notify us immediately if you are no longer able to maintain your package or to respond to reviews.
You will then be expected to either retract a submission, or to find alternative package maintainers.
You can also discuss maintenance issues in the rOpenSci slack workspace.
- Once your package is approved, we will provide further instructions about the transfer of your repository to the rOpenSci repository.

Our [code of conduct](#code-of-conduct) is mandatory for everyone involved in our review process.
112 changes: 76 additions & 36 deletions softwarereview_editor.Rmd

Large diffs are not rendered by default.

27 changes: 15 additions & 12 deletions softwarereview_editor_management.Rmd
Original file line number Diff line number Diff line change
@@ -19,31 +19,33 @@ Steps:

- Ping editors to be sure they get a notification as this is an important topic.

- Wait for a majority of editors to chime in before inviting someone. Leave them one week to respond.
- Wait for a majority of editors to chime in before inviting someone.
Leave them one week to respond.

## Inviting a new editor {#inviting-a-new-editor}

- Candidates might start by being [guest editors](#guesteditor).
When inviting them as guest editor, invite them as you would invite a guest editor for other reasons.

- If a candidate is guest editor first, assess how the process went after the submission is tackled. Asked other editors for their advice again.
- If a candidate is guest editor first, assess how the process went after the submission is tackled.
Asked other editors for their advice again.

- Send an email.

```
We would like to invite you to join the rOpenSci editorial board as a full member. [SPECIFIC REASONS FOR INVITATION (MENTION CONTRIBUTIONS TO ROPENSCI)].
We would like to invite you to join the rOpenSci editorial board as a full member. [SPECIFIC REASONS FOR INVITATION (MENTION CONTRIBUTIONS TO ROPENSCI)].
We think you would make a wonderful addition to the team.
[IF GUEST EDITOR:You are familiar with the editor's role as you've been a guest editor]. We aim for editors to handle four packages per year ([IF GUEST EDITOR including the one one you just finished!]).
We ask that editors make an informal commitment of serving for two years, and re-evaluate their participation after that.
[IF GUEST EDITOR:You are familiar with the editor's role as you've been a guest editor]. We aim for editors to handle four packages per year ([IF GUEST EDITOR including the one one you just finished!]).
We ask that editors make an informal commitment of serving for two years, and re-evaluate their participation after that.
On a short-term basis, any editor can decline to handle a package or say, "I'm pretty busy, I can't handle a new package for a few weeks."
In addition to handling packages, editors weigh in on group editorial decisions, such as whether a package is in-scope, and determining updates to our policies.
We generally do this through Slack, which we expect editors to be able to check regularly.
We have editorial board calls annually.
We also rotate Editor-in-Chief responsibilities (first-pass scope decisions and assigning editors) amongst the board about quarterly.
You'll have the opportunity to enter this rotation once you have been on the board for some time, usually at least six months.
Some of us also take on bigger projects to improve the peer-review process, though this is entirely optional.
In addition to handling packages, editors weigh in on group editorial decisions, such as whether a package is in-scope, and determining updates to our policies.
We generally do this through Slack, which we expect editors to be able to check regularly.
We have editorial board calls annually.
We also rotate Editor-in-Chief responsibilities (first-pass scope decisions and assigning editors) amongst the board about quarterly.
You'll have the opportunity to enter this rotation once you have been on the board for some time, usually at least six months.
Some of us also take on bigger projects to improve the peer-review process, though this is entirely optional.
We hope that you'll join the board!
It's an exciting time for peer-review at rOpenSci.
@@ -63,7 +65,8 @@ Best,

- If they haven't already done so as guest editors, request that the new editor turn on [two-factor authentication (2FA) for GitHub](https://docs.github.com/en/authentication/securing-your-account-with-two-factor-authentication-2fa).

- Invite editors to the rOpenSci GitHub organization as member, as a member of the [`editors` team](https://github.com/orgs/ropensci/teams/editors) and the [`data-pkg-editors`](https://github.com/orgs/ropensci/teams/data-pkg-editors) or [`stats-board`](https://github.com/orgs/ropensci/teams/stats-board) sub-team, as appropriate. This will give them appropriate permissions and allow them to get team-specific notifications.
- Invite editors to the rOpenSci GitHub organization as member, as a member of the [`editors` team](https://github.com/orgs/ropensci/teams/editors) and the [`data-pkg-editors`](https://github.com/orgs/ropensci/teams/data-pkg-editors) or [`stats-board`](https://github.com/orgs/ropensci/teams/stats-board) sub-team, as appropriate.
This will give them appropriate permissions and allow them to get team-specific notifications.

- Editors need access to the AirTable database of software review.

43 changes: 32 additions & 11 deletions softwarereview_intro.Rmd
Original file line number Diff line number Diff line change
@@ -10,33 +10,54 @@ _If you use our standards/checklists etc. when reviewing software elsewhere, do

## What is rOpenSci Software Peer Review? {#whatissoftwarereview}

rOpenSci's [suite of packages](https://ropensci.org/packages/) is partly contributed by staff members and partly contributed by community members, which means the suite stems from a great diversity of skills and experience of developers. How to ensure quality for the whole set? That's where software peer review comes into play: packages contributed by the community undergo a transparent, constructive, non adversarial and open review process. For that process relying mostly on volunteer work, [associate editors](#associateditors) manage the incoming flow and ensure progress of submissions; authors create, submit and improve their package; [reviewers](#reviewers), two per submission, examine the software code and user experience. [This blog post](https://www.numfocus.org/blog/how-ropensci-uses-code-review-to-promote-reproducible-science/) written by rOpenSci editors is a good introduction to rOpenSci software peer review Other blog posts about review itself and reviewed packages can be find [via the "software-peer-review" tag on rOpenSci blog](https://ropensci.org/tags/software-peer-review/).
rOpenSci's [suite of packages](https://ropensci.org/packages/) is partly contributed by staff members and partly contributed by community members, which means the suite stems from a great diversity of skills and experience of developers.
How to ensure quality for the whole set? That's where software peer review comes into play: packages contributed by the community undergo a transparent, constructive, non adversarial and open review process.
For that process relying mostly on volunteer work, [associate editors](#associateditors) manage the incoming flow and ensure progress of submissions; authors create, submit and improve their package; [reviewers](#reviewers), two per submission, examine the software code and user experience.
[This blog post](https://www.numfocus.org/blog/how-ropensci-uses-code-review-to-promote-reproducible-science/) written by rOpenSci editors is a good introduction to rOpenSci software peer review Other blog posts about review itself and reviewed packages can be find [via the "software-peer-review" tag on rOpenSci blog](https://ropensci.org/tags/software-peer-review/).

You can recognize rOpenSci packages that have been peer-reviewed via a green "peer-reviewed" badge in their README, linking to their reviews (cf [this example](https://github.com/ropensci/restez#locally-query-genbank-)); and via a blue comment icon near their description on [rOpenSci packages page](https://ropensci.org/packages/), also linking to the reviews.

Technically, we make the most of [GitHub](https://github.com/) infrastructure: each package review process is an issue in the [ropensci/software-review GitHub repository](https://github.com/ropensci/software-review/). For instance, click [here](https://github.com/ropensci/software-review/issues/24) to read the review thread of the `ropenaq` package: the process is an ongoing conversation until acceptance of the package, with two external reviews as important milestones. Furthermore, we use GitHub features such as the use of issue templates (as submission templates), and labelling which we use to track progress of submissions (from editor checks to approval).
Technically, we make the most of [GitHub](https://github.com/) infrastructure: each package review process is an issue in the [ropensci/software-review GitHub repository](https://github.com/ropensci/software-review/).
For instance, click [here](https://github.com/ropensci/software-review/issues/24) to read the review thread of the `ropenaq` package: the process is an ongoing conversation until acceptance of the package, with two external reviews as important milestones.
Furthermore, we use GitHub features such as the use of issue templates (as submission templates), and labelling which we use to track progress of submissions (from editor checks to approval).

## Why submit your package to rOpenSci? {#whysubmit}

- First, and foremost, we hope you submit your package for review **because you value the feedback**. We aim to provide useful feedback to package authors and for our review process to be open, non-adversarial, and focused on improving software quality.
- Once aboard, your package will continue to receive **support from rOpenSci members**. You'll retain ownership and control of your package, but we can help with ongoing maintenance issues such as those associated with updates to R and dependencies and CRAN policies.
- rOpenSci will **promote your package** through our [webpage](https://ropensci.org/packages/), [blog](https://ropensci.org/blog/), and social media (like [Mastodon](https://fosstodon.org/@ropensci) and [LinkedIn](https://www.linkedin.com/company/ropensci/)). Packages in our suite also get a [documentation website that is automatically built and deployed after each push](#docsropensci).
- First, and foremost, we hope you submit your package for review **because you value the feedback**.
We aim to provide useful feedback to package authors and for our review process to be open, non-adversarial, and focused on improving software quality.
- Once aboard, your package will continue to receive **support from rOpenSci members**.
You'll retain ownership and control of your package, but we can help with ongoing maintenance issues such as those associated with updates to R and dependencies and CRAN policies.
- rOpenSci will **promote your package** through our [webpage](https://ropensci.org/packages/), [blog](https://ropensci.org/blog/), and social media (like [Mastodon](https://fosstodon.org/@ropensci) and [LinkedIn](https://www.linkedin.com/company/ropensci/)).
Packages in our suite also get a [documentation website that is automatically built and deployed after each push](#docsropensci).
- rOpenSci **packages can be cross-listed** with other repositories such as CRAN and BioConductor.
- rOpenSci packages that are in scope for the [Journal of Open-Source Software](https://joss.theoj.org/) and add the necessary accompanying short paper, would, at the discretion of JOSS editors, benefit from a fast-tracked review process.
- If you write one, rOpenSci will **promote gitbooks related to your package**: the source of such books can be transferred to [the `ropensci-books` GitHub organisation](https://github.com/ropensci-books) for books to be listed [at books.ropensci.org](https://books.ropensci.org/).

## Why review packages for rOpenSci? {#whyreview}

- As in any peer-review process, we hope you choose to review **to give back to the rOpenSci and scientific communities.** Our mission to expand access to scientific data and promote a culture of reproducible research is only possible through the volunteer efforts of community members like you.
- Review is a two-way conversation. By reviewing packages, you'll have the chance to **continue to learn development practices from authors and other reviewers**.
- The open nature of our review process allows you to **network and meet colleagues and collaborators** through the review process. Our community is friendly and filled with supportive members expert in R development and many other areas of science and scientific computing.
- To volunteer to be one of our reviewers, fill out [this short form](https://airtable.com/shrnfDI2S9uuyxtDw) providing your contact information and areas of expertise. We are always looking for more reviewers with both general package-writing experience and domain expertise in the fields where packages are used.
- Review is a two-way conversation.
By reviewing packages, you'll have the chance to **continue to learn development practices from authors and other reviewers**.
- The open nature of our review process allows you to **network and meet colleagues and collaborators** through the review process.
Our community is friendly and filled with supportive members expert in R development and many other areas of science and scientific computing.
- To volunteer to be one of our reviewers, fill out [this short form](https://airtable.com/shrnfDI2S9uuyxtDw) providing your contact information and areas of expertise.
We are always looking for more reviewers with both general package-writing experience and domain expertise in the fields where packages are used.

## Why are reviews open? {#whyopen}

Our reviewing threads are public. Authors, reviewers, and editors all know each other's identities. The broader community can view or even participate in the conversation as it happens. This provides an incentive to be thorough and provide non-adversarial, constructive reviews. Both authors and [reviewers report](https://ropensci.org/tags/reviewer/) that they enjoy and learn more from this open and direct exchange. It also has the benefit of building a community. Participants have the opportunity to meaningfully network with new peers, and new collaborations have emerged via ideas spawned during the review process.

We are aware that open systems can have drawbacks. For instance, in traditional academic review, [double-blind peer review can increase representation of female authors](https://www.sciencedirect.com/science/article/pii/S0169534707002704), suggesting bias in non-blind reviews. It is also possible reviewers are less critical in open review. However, we posit that the openness of the review conversation provides a check on review quality and bias; it's harder to inject unsupported or subjective comments in public and without the cover of anonymity. Ultimately, we believe that having direct and public communication between authors and reviewers improves quality and fairness of reviews.
Our reviewing threads are public.
Authors, reviewers, and editors all know each other's identities.
The broader community can view or even participate in the conversation as it happens.
This provides an incentive to be thorough and provide non-adversarial, constructive reviews.
Both authors and [reviewers report](https://ropensci.org/tags/reviewer/) that they enjoy and learn more from this open and direct exchange.
It also has the benefit of building a community.
Participants have the opportunity to meaningfully network with new peers, and new collaborations have emerged via ideas spawned during the review process.

We are aware that open systems can have drawbacks.
For instance, in traditional academic review, [double-blind peer review can increase representation of female authors](https://www.sciencedirect.com/science/article/pii/S0169534707002704), suggesting bias in non-blind reviews.
It is also possible reviewers are less critical in open review.
However, we posit that the openness of the review conversation provides a check on review quality and bias; it's harder to inject unsupported or subjective comments in public and without the cover of anonymity.
Ultimately, we believe that having direct and public communication between authors and reviewers improves quality and fairness of reviews.

Furthermore, authors and reviewers have the ability to contact privately the editors if they have any doubt or question.

316 changes: 159 additions & 157 deletions softwarereview_policies.Rmd

Large diffs are not rendered by default.

62 changes: 45 additions & 17 deletions softwarereview_reviewer.Rmd
Original file line number Diff line number Diff line change
@@ -10,11 +10,12 @@ Thanks for accepting to review a package for rOpenSci! This chapter consists of
You might contact the editor in charge of the submission for any question you might have about the process or your review.
Please strive to complete your review within 3 weeks of accepting a review request. We will aim to remind reviewers of upcoming and past due dates. Editors may assign
additional or alternate reviewers if a review is excessively late.
Please strive to complete your review within 3 weeks of accepting a review request.
We will aim to remind reviewers of upcoming and past due dates.
Editors may assign additional or alternate reviewers if a review is excessively late.
**rOpenSci's community is our best asset.
We aim for reviews to be open, non-adversarial, and focused on improving software quality.
**rOpenSci's community is our best asset.
We aim for reviews to be open, non-adversarial, and focused on improving software quality.
Be respectful and kind!
See our reviewers guide and [code of conduct](https://ropensci.org/code-of-conduct/) for more.**
@@ -36,11 +37,14 @@ For other ways to contribute, refer to [rOpenSci contributing guide](https://con
## Preparing your review {#preparereview}

Reviews should be based on the latest GitHub version on the default branch, unless otherwise indicated by package authors.
All submissions trigger a detailed report on package structure and functionality, generated by [our `pkgcheck` package](https://docs.ropensci.org/pkgcheck/). If the package has changed substantially since the last checks, you may request a re-check with the command `@ropensci-review-bot check package`. Note that when installing the package to review it, you should make sure you have all dependencies available (for instance run `pak::pak()`).
All submissions trigger a detailed report on package structure and functionality, generated by [our `pkgcheck` package](https://docs.ropensci.org/pkgcheck/).
If the package has changed substantially since the last checks, you may request a re-check with the command `@ropensci-review-bot check package`.
Note that when installing the package to review it, you should make sure you have all dependencies available (for instance run `pak::pak()`).

### General guidelines {#general-guidelines}

To review a package, please begin by copying our [review template](#reviewtemplate) (or our [review template in Spanish](#reviewtemplatees)) and using it as a high-level checklist. In addition to checking off the minimum criteria, we ask that you provide general comments addressing the following:
To review a package, please begin by copying our [review template](#reviewtemplate) (or our [review template in Spanish](#reviewtemplatees)) and using it as a high-level checklist.
In addition to checking off the minimum criteria, we ask that you provide general comments addressing the following:

- Does the code comply with general principles in the [Mozilla reviewing guide](https://mozillascience.github.io/codeReview/review.html)?
- Does the package comply with the [rOpenSci packaging guide](#building)?
@@ -50,17 +54,24 @@ To review a package, please begin by copying our [review template](#reviewtempla
- Are there performance improvements that could be made?
- Is the documentation (installation instructions/vignettes/examples/demos) clear and sufficient? Does it use the principle of *multiple points of entry* i.e. takes into account the fact that any piece of documentation may be the first encounter the user has with the package and/or the tool/data it wraps?
- Were functions and arguments named to work together to form a common, logical programming API that is easy to read, and autocomplete?
- If you have your own relevant data/problem, work through it with the package. You may find rough edges and use-cases the author didn't think about.
- If you have your own relevant data/problem, work through it with the package.
You may find rough edges and use-cases the author didn't think about.

Please be respectful and kind to the authors in your reviews. Our [code of conduct](#code-of-conduct) is mandatory for everyone involved in our review process. We expect you to submit your review within 3 weeks, depending on the deadline set by the editor. Please contact the editor directly or in the submission thread to inform them about possible delays.
Please be respectful and kind to the authors in your reviews.
Our [code of conduct](#code-of-conduct) is mandatory for everyone involved in our review process.
We expect you to submit your review within 3 weeks, depending on the deadline set by the editor.
Please contact the editor directly or in the submission thread to inform them about possible delays.

We encourage you to use automated tools to facilitate your reviewing. These include:
We encourage you to use automated tools to facilitate your reviewing.
These include:

- Checking the initial package report generated by our `@ropensci-review-bot`.
- Checking the package's logs on its continuous integration services (GitHub Actions, Codecov, etc.)
- Running `devtools::check()` and `devtools::test()` on the package to find any errors that may be missed on the author's system.
- Seeing whether tests' skipping is justified (e.g. `skip_on_cran()` tests that do real API requests vs. skipping all tests on one operating system).
- If the package is not submitted via the default/main branch, remember to switch to the submitted review branch before starting your review. In this case, you will also have to search the package locally, as GitHub search is limited to the default branch. Further, documentation hosted on a `pkgdown` website is not necessarily up-to-date, and we recommend to inspect the package's documentation locally by running `pkgdown::build_site()`.
- If the package is not submitted via the default/main branch, remember to switch to the submitted review branch before starting your review.
In this case, you will also have to search the package locally, as GitHub search is limited to the default branch.
Further, documentation hosted on a `pkgdown` website is not necessarily up-to-date, and we recommend to inspect the package's documentation locally by running `pkgdown::build_site()`.

Reviewers may also re-generate package check results from `@ropensci-review-bot` at any time by issuing the single comment in a review issue: `@ropensci-review-bot check package`.

@@ -70,7 +81,9 @@ If you interact with the package authors and talked about the review outside a r

### Experience from past reviewers {#experience-from-past-reviewers}

First-time reviewers may find it helpful to read (about) some previous reviews. In general you can find submission threads of onboarded packages [here](https://github.com/ropensci/software-review/issues?q=is%3Aissue+is%3Aclosed+label%3A6%2Fapproved). Here are a few chosen examples of reviews (note that your reviews do not need to be as long as these examples):
First-time reviewers may find it helpful to read (about) some previous reviews.
In general you can find submission threads of onboarded packages [here](https://github.com/ropensci/software-review/issues?q=is%3Aissue+is%3Aclosed+label%3A6%2Fapproved).
Here are a few chosen examples of reviews (note that your reviews do not need to be as long as these examples):

- `rtika` [review 1](https://github.com/ropensci/software-review/issues/191#issuecomment-367166658) and [review 2](https://github.com/ropensci/software-review/issues/191#issuecomment-368254623)

@@ -80,13 +93,25 @@ First-time reviewers may find it helpful to read (about) some previous reviews.

- `rusda` [review](https://github.com/ropensci/software-review/issues/18#issuecomment-120445737) (from before we had a review template)

You can read blog posts written by reviewers about their experiences [via this link](https://ropensci.org/tags/reviewer/). In particular, in [this blog post by Mara Averick](https://ropensci.org/blog/2017/08/22/first-package-review/) read about the "naive user" role a reviewer can take to provide useful feedback even without being experts of the package's topic or implementation, by asking themselves *"What did I think this thing would do? Does it do it? What are things that scare me off?"*. In [another blog post](https://ropensci.org/blog/2017/09/08/first-review-experiences/) Verena Haunschmid explains how she alternated between using the package and checking its code.

As both a former reviewer and package author, and now editor, [Adam Sparks](https://adamhsparks.netlify.app/) wrote "\[write\] a good critique of the package structure and best coding practices. If you know how to do something better, tell me. It's easy to miss documentation opportunities as a developer, as a reviewer, you have a different view. You're a user that can give feedback. What's not clear in the package? How can it be made more clear? If you're using it for the first time, is it easy? Do you know another R package that maybe I should be using? Or is there one I'm using that perhaps I shouldn't be? If you can contribute to the package, offer."
You can read blog posts written by reviewers about their experiences [via this link](https://ropensci.org/tags/reviewer/).
In particular, in [this blog post by Mara Averick](https://ropensci.org/blog/2017/08/22/first-package-review/) read about the "naive user" role a reviewer can take to provide useful feedback even without being experts of the package's topic or implementation, by asking themselves *"What did I think this thing would do? Does it do it? What are things that scare me off?"*.
In [another blog post](https://ropensci.org/blog/2017/09/08/first-review-experiences/) Verena Haunschmid explains how she alternated between using the package and checking its code.

As both a former reviewer and package author, and now editor, [Adam Sparks](https://adamhsparks.netlify.app/) wrote "\[write\] a good critique of the package structure and best coding practices.
If you know how to do something better, tell me.
It's easy to miss documentation opportunities as a developer, as a reviewer, you have a different view.
You're a user that can give feedback.
What's not clear in the package?
How can it be made more clear?
If you're using it for the first time, is it easy?
Do you know another R package that maybe I should be using?
Or is there one I'm using that perhaps I shouldn't be?
If you can contribute to the package, offer.

### Helper package for reviewers {#helper-package-for-reviewers}

If working in RStudio, you can streamline your review workflow by using the [`pkgreviewr` package](https://github.com/ropensci-org/pkgreviewr) created by associated editor Anna Krystalli. Say you accepted to review the `refnet` package, you'd write
If working in RStudio, you can streamline your review workflow by using the [`pkgreviewr` package](https://github.com/ropensci-org/pkgreviewr) created by associated editor Anna Krystalli.
Say you accepted to review the `refnet` package, you'd write

```
remotes::install_github("ropensci-org/pkgreviewr")
@@ -108,13 +133,16 @@ We encourage you to ask questions and provide feedback on the review process on
## Submitting the Review {#submitreview}

- When your review is complete, paste it as a comment into the package software-review issue.
- Additional comments are welcome in the same issue. We hope that package reviews will work as an ongoing conversation with the authors as opposed to a single round of reviews typical of academic manuscripts.
- Additional comments are welcome in the same issue.
We hope that package reviews will work as an ongoing conversation with the authors as opposed to a single round of reviews typical of academic manuscripts.
- You may also submit issues or pull requests directly to the package repo if you choose, but if you do, please comment about them and link to them in the software-review repo comment thread so we have a centralized record and text of your review.
- Please include an estimate of how many hours you spent on your review afterwards.

## Review follow-up {#followupreviewer}

Authors should respond within 2 weeks with their changes to the package in response to your review. At this stage, we ask that you respond as to whether the changes sufficiently address any issues raised in your review. We encourage ongoing discussion between package authors and reviewers, and you may ask editors to clarify issues in the review thread as well.
Authors should respond within 2 weeks with their changes to the package in response to your review.
At this stage, we ask that you respond as to whether the changes sufficiently address any issues raised in your review.
We encourage ongoing discussion between package authors and reviewers, and you may ask editors to clarify issues in the review thread as well.

You'll use the [approval template](#approval2template).