[Epic] Backwards-incompatible API changes, for the greater good

[rebolbot]

Submitted by: BrianH

The newly open source Rebol is an opportunity for us to have a brand new start, to make the language we want to promote to the world. This is the time to clean it up. Rebol 3 will have more users than Rebol has had in its whole history, and it's those users who will matter the most.

This is a list of the changes to APIs that are not going to be backwards compatible with Rebol 2, or even with the last closed-source alpha. All of these changes need to happen before 3.0, because after 3.0 we're going to start considering compatibility with existing code to be a blocker again. But until then, let's do it right, as if it's the first time. Remember #666, it applies to everything listed here.

No syntax changes here, that's for another epic. Just APIs and semantics and a few names. Related tickets in the first comment, to make linking easier. Status will be updated there.

^{CC - Data [ Version: r3 master Type: Issue Platform: All Category: Unspecified Reproduce: Always Fixed-in:none ]}

[rebolbot]

Submitted by: BrianH

Rebol 3 is a new language, based on more than a decade of experience with languages of the Rebol language model, notably Rebol 1 and 2. Even though we're doing a new start, it's based on a solid foundation and a lot of knowledge about what works and what doesn't in a Rebol-like language. So we have changes based on some hard lessons learned.

Previously useful, now differently useful:

• Some functions that used to be mezzanine are now native (notably all of Rebol 2's loops). This changes the performance balance.
• Some functions that used to be native are now mezzanines that call new natives internally (notably LOAD). This lets us improve them.
• Some natives call Rebol functions internally now (notably DO). This makes it possible to write the really difficult or flexible stuff in Rebol.
• Some functions that used to be actions are now natives (notably the ordinals), or internal. This makes it easier to make datatypes and port schemes.
• Some operations that used to trigger errors now return none instead, and vice-versa. This improves control flow and debuggability.
• Some things are just outright different, due to the new runtime model. It's a new language underneath, superficially the same where appropriate.
• Some things were changed because they had bugs or design flaws that made them effectively useless. They were rarely used, because they couldn't be.
• Some of the old stuff will still be usable, but through an optional module (see Dummy #2132). This clearly marks legacy code, so we know what needs cleaning up.
• Some features are just gone, because they don't make sense in the new model (notably the function [catch] option).
• Some features are temporarily gone, because they were either written for the old model, or because they were third-party closed source. We can rewrite them.
• #[none] is really important now, treated throughout as a placeholder for missing data. Many functions changed, even the series model rethought around this.
• The "errors are your friends" principle [Epic] Errors are your friends #2133. No errors triggered unless it is more useful to consider the condition erroneous, but then they're always triggered.
• More polymorphic behavior. If a built-in function can support a type in a way that is consistent with the type's semantics, and isn't better an error, we add support (i.e. SELECT).

Previously useful, now differently useful, in specific:

• You can add words to objects now. You can even do so with APPEND. Just about everything depends on this.
• And there are two kinds of objects: functions use different objects with stack-relative addressing, and you can't add words to these.
• And for normal objects 'self is a binding keyword, not a word in the object. We have 'self for real now.
• Binding control functions and closures make "selfless" objects. Selfless objects can even add a 'self word that's a normal field, not infrastructure.
• Speaking of "closures", we have a new function type called closure!. It has different advantages than the function! type (which is also new). Both are better.
• We also have added a datatype to support semantics which the normal object type doesn't and can't have the right balance for. They have different uses.
• DO's /next option takes a parameter of a word to set, rather than multi-value-return in a block. This is a performance change.
• LOAD has a different, simpler API, and can categorically do more stuff (with more codecs, of course). Except LOAD/next has been moved to TRANSCODE.
• Reflection with ordinal functions is gone. REFLECT and its *-OF wrappers (like WORDS-OF) are what you use now, even in Rebol 2.
• IF, UNLESS and EITHER are easier to use in expression style now. IF/else and ELSE are now finally gone.
  Unified conditional model:
• TRUE = TO LOGIC! 0 Have TO LOGIC! 0 return true #2055 - Have TO LOGIC! be conditional, leave MAKE LOGIC! C-like behavior for when it's needed.
• Conditional operators Implement conditional operators #1879 - Make AND, OR, XOR, AND~, OR~ and XOR~ conditional.
• Bitwise operations by UNION, INTERSECT and DIFFERENCE - To make conditional operators possible. Needs a ticket.
• Unsets as noops in ANY and ALL TRANSCODE /else on-error option #534 and ALL [()] should have the same result as ALL [] #850 - The errors aren't useful, and truthy unsets are bad news.

Categorically cleaned-up semantics:

• Indexing model [Epic] The Great Indexing Compromise #2117 - Consensus reached. Ironically, more compatible with Rebol 2; it's old Rebol 3 code that needs fixing.
• Unified record-oriented function behavior - Needs an epic. See DO-COMPARE function #2110 for the core semantics.
• None propagation rebalancing - Needs an epic to collect the tickets referring to the changes already done. Also, check for holes.
• More uniform DO behavior [Epic] DO word, path and set-path #2124 - Make the language easier to understand and teach by cleaning up the gotchas.
• More modular modules Dummy #2131 - Simplify the use of the module system for modular development.
• Rebalanced equivalences Tweak STRICT-EQUAL? equivalences, add STRICT-EQUIV? function #1834, Object! structural equality not working with percent! or money! #1133, EQUIV? decimal! comparison not exact enough to be transitive #1134 - Needs an epic (Tweak STRICT-EQUAL? equivalences, add STRICT-EQUIV? function #1834 comes close).

Bad/useless/out-of-place functions replaced by good ones:

• New FOR General loop #884 - FORTRAN-style FOR was rarely useful, and thus rarely used. Replace with a more general C-like FOR.
• Mins and maxes with a useful API and better names Rename MAXIMUM-OF and MINIMUM-OF to FIND-MAX and FIND-MIN, respectively #1971 and Create new MAXIMUM-OF and MINIMUM-OF functions that do what they say #1972.
• ALTER's return value made more useful, but the function itself might not be useful enough to keep. Needs bug fixes if we keep it.
• OBJECT replaces CONTEXT Replace CONTEXT function(s) with OBJECT #2118, CONTEXT deprecated Deprecate the CONTEXT function #2123 - The name "context" turned out to be misleading.
• Unblock THROW/name and CATCH/name - Old bug made these useless. Needs an epic to cover all the relevant tickets.
• CATCH-ALL in the test suite should be supported as a real native API, not as a hack (because we have to close some holes). Needs a ticket.
• The 'old module Dummy #2132 - Where we put the old stuff we're displacing. Might be a prerequisite for some changes.

New, but with unacceptable APIs and/or bad/missing features:

• REWORD Revamp REWORD API: Evaluate value expressions; /case option; binary! support; trailing delimiters #1990 - Refined API needed. Merged, but the SO article needs enhancing with the new stuff.
• TRANSCODE - It doesn't do everything needed (TRANSCODE /part option #1915), and needs a new API (TRANSCODE function API model needs revamp #1916 alternate 4).
• PROTECT and UNPROTECT - New API needed, maybe new model. Needs its own epic. The /hide stuff is OK.
• SPLIT Rename SPLIT's /INTO refinement #2051 - API adjustment needed. It breaks the /into model, and we might need to consider that it isn't very Rebol-like. Those numbers are opaque.

Still collecting the list.

#1990 and #2055 merged already. #534 in from the old days, but #850 shows it needs work. #1879 needs another request for bitwise operators, and maybe more debate.