Merge cockroachdb#84091

84091: docs/RFCS: consolidate RFC docs, add additional guidance r=abarganier a=abarganier

This PR aims to improve the already excellent documentation 
& guidance that we provide for writing RFCs. The PR has two primary
objectives:

1. Defragment documentation where appropriate, to avoid possibility
of readers missing important information.
2. Add additional guidance on RFC writing where appropriate.

Each change is contained within a separate commit, so consider
reviewing commit-wise for specific context on each individual change.

Feedback is welcome from everyone. The goal here is to improve the
documentation for all, and while I have written & learned from few RFCs,
I am sure there are plenty of others who have more experience with the
process than I do. I very much welcome other perspectives & advice on
how we can make this documentation even better!

Co-authored-by: Alex Barganier <[email protected]>

docs/RFCS/GUIDE.md

@@ -1,3 +1,4 @@
+**Note: Be sure to also look at the [README](README.md) for more general guidance on writing RFCs.**
 
 # Summary
 

docs/RFCS/README.md

@@ -35,19 +35,144 @@ guidance is intentionally vague. Many complex projects spread over
 multiple PRs do not require an RFC, though they do require adequate
 communication and discussion.<sup>[1](#sql-syntax)</sup>
 
-It is encouraged to develop a [prototype](PROTOTYPING.md) concurrently
+An RFC can help you get buy-in and alignment from necessary stakeholders
+on how to best approach your project. Such alignment helps ensure that
+by the time the team starts the implementation phase, your design &
+approach are sound & have been properly vetted.
+
+A few good questions to ask yourself before writing an RFC are:
+1. Does my project introduce new/modify existing architecture that is
+   significant to the operations of the database?
+1. Will my project have a broad impact on sensitive parts of the product,
+   where mistakes or poor design decisions could have a large negative
+   impact?
+
+If the answer to either of these questions is a resounding "yes", an RFC
+may very well be appropriate. As mentioned before, whether you should
+write an RFC is a nuanced topic. When appropriate, an RFC can be a very
+effective tool. However, the process can sometimes be slow & come with a
+good amount of toil. People need time to respond, edits need to be made,
+prototypes need to be modified, and this cycle may repeat multiple times.
+Therefore, it's worth giving this question of whether or not to write an
+RFC the proper consideration. If it is indeed appropriate, then the RFC
+process will be a great tool for strengthening the design and concepts
+behind your project.
+
+It is encouraged to develop a [prototype](#prototyping) concurrently
 with writing the RFC. One of the significant benefits of an RFC is
 that it forces bigger picture thinking which reviewers can then
 dissect. In contrast, a prototype forces the details to be considered,
 shedding light on the unknown unknowns and helping to ensure that the
 RFC focuses on the important design considerations.
 
+If you've never written an RFC before, consider partnering with a more 
+experienced engineer for guidance and to help shepherd your RFC through 
+the process.
+
+# Keep It Concise
+
 An RFC should be a high-level description which does not require
 formal correctness. There is utility in conciseness. Do not
 overspecify the details in the RFC as doing so can bury the reviewer
-in minutiae.<sup>[1](#sql-syntax)</sup> If you've never written an RFC
-before consider partnering with a more experienced engineer for
-guidance and to help shepherd your RFC through the process.
+in minutiae.<sup>[1](#sql-syntax)</sup> A [prototype](#prototyping) 
+can be of assistance here as it can help highlight the tricky areas 
+that deserve mention in a high-level description. If the details are 
+relevant, the RFC can include a link to the prototype (which may 
+necessitate cleaning up some of the prototype code at the reviewer's 
+request). Note that one of the significant benefits of an RFC is that
+it forces bigger picture thinking which readers can then dissect. In 
+this respect an RFC is complimentary to a prototype which forces details 
+to be considered.
+
+# Present a Narrative
+
+When writing your RFC, you should always keep the prospective reader
+in mind. We already mentioned above that one should do their best to
+[keep things concise](#keep-it-concise), but what more can you do to
+make the life of your reader easier?
+
+As engineers & scientists, it can be tempting to use a large pile of
+facts to state your case. After all, the facts don't lie, and if we
+present enough of them in support of our cause, it helps strengthen
+our argument. But is the reader going to enjoy reading a long laundry
+list of facts? Probably not. However, if you are able to weave these
+facts and technical designs together with a strong, singular narrative,
+your writing is more likely to resonate with the reader.
+
+Therefore, when writing your RFC, try to present your ideas with an
+overarching narrative. Be clear about the problem(s) you're trying to
+solve. Consider presenting some real-world examples to help make things
+more relatable. Do your best to structure your writing in an eloquent way. 
+Taking such steps will help keep the reader engaged. This will likely
+lead to your RFC being better received and gaining more attention, which 
+means you'll receive more feedback (a good thing!).
+
+# Prototyping
+
+There is a puzzle at the heart of writing an RFC: if you understand a
+project well enough to write an RFC, then an RFC may not be
+necessary. Certainly you may write an RFC to communicate your
+knowledge to others, but how does an RFC get written by an individual
+who doesn't understand the problem fully?
+
+### Iterative Writing
+
+The idealistic view of writing an RFC involves an engineer starting
+with a blank document and filling in the pieces from start to
+finish. For anyone who has written an RFC, this fairy tale depiction
+is clearly not accurate. The writing is iterative. You might first
+write down a few bullet points and then start thinking about the
+details of one area, fleshing out the document incrementally and in an
+unordered fashion. There are very likely many (many) iterations of the
+document before anyone else takes a look at it.
+
+The iterative writing of an RFC described above is often insufficient
+because simply thinking deeply about a problem is frequently not the
+most efficient path to fully understanding the problem. At the very
+least you will be consulting the existing code base to understand how
+the proposed change fits. And you might discuss the problem in front
+of a whiteboard with a colleague. It is very likely you should be
+prototyping.
+
+### The Role of Prototyping
+
+Prototyping involves exploring a problem space in order to better
+understand it. The primary value of a prototype is in the learning it
+provides, not the code. As such, the emphasis during prototyping
+should be on speed of learning. The prototype code does not need to
+meet any particular quality metric. Error handling? Ignore it while
+you're trying to learn. Comments? Only if you, the prototype author,
+need them. The real focus of the prototype is exploring the unseen
+corners of the problem space in order to reveal where the dragons are
+lurking. The goal is to de-risk the problem space by thoroughly
+understanding the hardest problems.
+
+### Write, Prototype, Repeat 
+
+The acts of prototyping and writing an RFC should be iterative. Many
+engineers want to write down their thoughts before coding, and it is
+common to want to discuss an approach before embarking on it. Both
+approaches are useful, yet neither should be confused for an RFC. It
+is useful to write down a few bullet points for areas to be covered in
+a prototype. Then work on the code. Then add more bullet points. At
+some point, the prototype will be fleshed out enough that you feel
+ready to write the RFC. Alternately, it is very common for the
+prototype to run into a significant stumbling block that you can't
+overcome yourself. The benefit of the prototype truly shines when this
+happens and you'll have a deep understanding of the problem which you
+can use to have a focused discussion with other engineers. Share your
+failed prototypes. The failure implies learning and that learning
+deserves to be shared, both in words and in code. Your failed
+prototype might be close to success with the aid of another engineer's
+experience. At the very least, a failed prototype indicates to other
+engineers that an idea has been explored.
+
+### Consider if the Prototype is Sufficient
+
+Writing an RFC is not a necessary outcome for a successful
+prototype. If the prototype is simple enough, don't be afraid to translate
+it directly into a PR or series of PRs along with explanatory comments
+and commit messages.
 
 # RFC Process