Suggestions on how the RFC process could be improved

[KirkMunro]

I originally shared these directly with @SteveL-MSFT and @joeyaiello, but better to share these thoughts in the open so that others can chime in.

The current RFC process is challenging, for a number of reasons:

• working through the initial process while an RFC is in a PR takes too way too long
• managing and following discussions through a PR is very challenging
• reading RFCs through a PR is very challenging task (especially when one PR contains multiple RFCs)

For some reason, I always thought the process was to submit a PR, get some initial corrective feedback (if there were points of confusion, etc.), make updates, and get the RFC published in the draft state, with comments coming in an issue. That seems to not be the case, and RFCs remain in a PR state with long, disjointed discussions for quite some time before (if) they make it to the draft state with comments coming in an issue. I think that process is failing, so here's my take on some rough and some not-so-rough ideas on how I would improve the process:

First, a few structural changes to the repository to support what is proposed below, as follows:

• copy the RFC template into .github/ISSUE_TEMPLATES/rfc-proposal.md
• create two additional new templates in .github/ISSUE_TEMPLATES:
  • rfc-review.md (used when an RFC enters the review stage)
  • general.md (used for general issues with or questions about the repository or the process)
• configure the repository to use the three templates for issues: RFC proposals, RFC reviews, and general questions or issues with the repository or about the process

Then, fix #190 to eliminate the dead link issue this issue.

Next, implement #191 to provide a user-friendly reading view of the RFCs in the repository.

Structural changes out of the way, I'd add at least one maintainer who is willing to help keep the process moving (hide outdated/addressed comments when necessary to keep discussions clean, etc. -- see below).

Once that is in place, I would set up the following process:

1. All RFC proposals are submitted as new Issues instead of as PRs.

   Benefits:

   • adds a needed stage that the current process does not convey well: a proposal will not necessarily become an RFC
   • conversation about the proposal is naturally threaded
   • individual comments in the issue discussion can be hidden by maintainers if they are out of
     date at some point (if the author updates the RFC in the issue, addressing the comment)
   • repository maintainers can directly edit issues (fix simple typos without creating conversation noise, for example)
   • issues can be closed if an RFC is deemed not appropriate, if the author needs to do some redesign or take a new approach, etc.
   • this leverages the "new" (been there a while now) GitHub model for multiple templates, where users can choose to submit an RFC proposal, automatically get the template, and type it in there or copy/paste/modify externally (or work with a copy of the template md file) and then paste the result back in

   Downsides:

   • comments cannot be hidden by the person who logged the issue, so a maintainer or repository owner needs to be involved (if only we knew someone who knew someone who worked at GitHub, who could influence better permissions, something they have been lacking for-e-v-e-r ;))
   • related RFCs cannot be published in one chunk (as I did under PR Make terminating errors terminate the right way #187); some may see that as an upside as well, and I could have just as well done those under a bunch of individual issues that reference one another.

2. Initial discussion about the RFC happens under the proposal, with the goal being to clean up the proposal and bring it to a draft state, add alternate proposals that are suggested by the community, etc. (note: there should be a formal specification used to identify how to document alternate proposals will not be pursued and why -- this isn't in the docs right now).

3. After iterating over discussions and things settle down (time-bomb this to 2 weeks -- should be enough time given that this is just to get the RFC to initial draft, when there are more opportunities for review RFCs that are actually in a draft state), the author who posted the RFC proposal (or an automated bot that monitors time passed since RFC proposals show up as an issue) mentions the PSCommittee for review of the RFC. The committee checks the RFC for completeness, and one of three things happens:

   1. If it is complete, the committee asks the author (or a bot) to move the RFC to draft (i.e. to PR the RFC). If this process is automated with a bot (which would be ideal), the issue gets copied into a numbered RFC file in the correct location (hopefully in a flat structure -- see Repository cleanup to eliminate dead links #190) with an appropriate name, the markdown document listing RFCs gets updated automatically, a new issue gets automatically created for comments on the draft RFC, referencing that file, and highlighting the deadline date for comments, plus notifications about the RFC being open for comment for 30 days are shared on social media. If it's not automated, that work would have to be done manually.
   2. If the RFC proposal is not complete, and if the author is unresponsive, the RFC proposal is closed as abandoned (it can always be reopened later).
   3. If the RFC is deemed inappropriate (maybe it needs rework or needs to be opened as a new RFC), the RFC is closed as proposal rejected. This could happen earlier in the process if discussions discover early on that the RFC proposal should be closed.

4. When a new RFC enters the official review process, the community needs to be notified (maybe via a bot posting tweets as hinted at above). A simple blog post mentioning the RFCs that have entered the process along with the deadlines for comment, or maybe just some tweets to notify users, or bring them up during the community call. Or all of the above.

At that point, the RFC goes through the draft, draft-approved, etc. phases of the process as part of the PS Team RFC meetings, hopefully on a schedule with a bot that that they can ask for more time for or a reminder, etc. like they do in the PowerShell repo.

Feedback on this process is welcome and appreciated.

[vexx32]

There all look like great ideas!

Perhaps @SteveL-MSFT could look into the automation aspect via msft-bot or PoshChan.

You're right in that PR conversations can end up being very noisy and chaotic. Frequently I find I need to just go back over and reread everything that's been pulled into the RFC proper and ignore the majority of the comments when I'm looking at them, because it's so hard to properly see what's actually happening in a reasonably chronological way with all of the split discussions.

Love these suggestions, thanks for writing them up!!

[iSazonov]

Main problem is that our RFC process has large deviation from low to high activity.

copy the RFC template into .github/ISSUE_TEMPLATES/rfc-proposal.md

It seems in most cases this happens in discussions in PowerShell repo. There is no clear edge when an enhancement should be transformed to RFC. We could mark such discussion with "Need-RFC" label in PowerShell repo. This will work for low activity and high activity proposals.

We started the RFC process as issue based and moved to PR based because first one doesn't work for low activity RFCs at all. It was a compromise that led to good results.
Now we have several RFCs with high activity but obviously not all. So the moving to the issue based discussions most likely will not solve all the problems.

I'd add at least one maintainer who is willing to help keep the process moving

Currently the RFC process is fully MSFT team centric. So I can’t imagine that the community moderator could do a lot of useful things.

[KirkMunro]

We started the RFC process as issue based and moved to PR based because first one doesn't work for low activity RFCs at all. It was a compromise that led to good results.

By "low activity RFC", do you mean an RFC that doesn't get much feedback? I think the responsibility for feedback on RFCs lies with the author and the PowerShell community. If people aren't responding to an RFC, you'd have to ask why they aren't responding. Is it...

• ... because the RFC isn't needed?
• ... because the RFC is at such technical depth that not many members of the community will respond?
• ... because the RFC process isn't doing a good job of promoting RFCs to the community to get feedback?
• ... because the community doesn't really understand the RFC process?

Most RFCs that didn't get much feedback would probably have several of these as the answer. You would think, though, that between regular community calls and social media, we should be able to make sure that the community knows about RFCs so that they can comment on them.

One thing I would put money on though: the current process gets feedback from people who are comfortable with the comment system on GitHub PRs, but it excludes feedback from everyone else, and they are going to be a majority by far.

[iSazonov]

By "low activity RFC", do you mean an RFC that doesn't get much feedback?

Yes. And I agree with your thoughts about social nature of the problem. The more constructive feedback we get the better. (Only it should be noted that we already have accepted and implemented RFCs with minimal discussion - result very good.)

[JamesWTruher]

@PowerShell/powershell-committee This issue was opened before our governance document (https://github.com/PowerShell/PowerShell/blob/master/docs/community/governance.md) which made improvements to our RFC process was available. We would be glad to have feedback to that document.