D1683R0
References for Standard Library Vocabulary Types - an optional<> case study

1. Revision History

1.1. Revision 0 - July 17th, 2019

Initial (only?) release.

2. Motivation and History

Originally, std::optional<T> -- where T denotes the name of a type -- contained a specialization to work with regular references, std::optional<T&>. When some of the semantics for references were called into question with respect to the assignment operator (assign into the value or rebind the reference inside the optional) and how comparisons would be performed (based on operator< exclusively or forwarding all operations directly to the underlying type), the debate stopped early and no full consensus was reached. Rather than remove just the operator or modify comparison operators, the entirety of std::optional<T&> was removed entirely after some consensus was reached around forwarding comparison operations to the underlying type, if present. This left many codebases in an interesting limbo. Previous implementations and external implementations had support for references, while the standard did not and vigorously advocated itself as such through various Committee participants and related firmly asserting that there were two equally valid and hard to pick interpretations.

This problem became apparent with other parameter and return abstractions such as std::expected, the Outcome library, std::variant, and more. Given the scope of the problem, qualitative research was necessary to help capture the uses and intent of references in vocabulary types. optional was chosen as the primary candidate to collect data, since it is widely used and reimplemented in the C++ ecosystem and broadly represents the challenge of reference wrapping types in C++. A survey conducted of 110+ programmers, (private) feedback from over half a dozen companies, and several analyses of optional reference usage (or not) in the wild has yielded results as to both quality of implementation, existence, and prevalence of references and their use.

Pointedly: there is strong motivation to have references in base vocabulary types used for both parameters (variant, optional, expected, future, etc.) and a lot of existing practice to do so both before and after the standard settled on its current semantics for std::optional and std::variant. Furthermore, brief historical analysis and user communication reveals the Standards Committee actually had a "chilling effect" on reference support in implementations in large codebases, from private companies to open source projects.

Finally, given the field experience and absolutely lack of implementation experience with certain implementations, it seems there is only one implementation model that is viable at scale and under the implementation pressures of

2.1. Stuck in the Past

A very large hole is left in many codebases that desire to wrap their non-null optional returns from T* to T&. It has prevented many code bases from migrating from the most popular optional implementations available pre-standardization, such as akrzemi/optional and Boost.Optional. It has also prevented adoption in other modern codebases for where the difference between T*, optional<T&>, and optional<std::reference_wrapper<T>> alongside programmer intent is significant and non-ignorable, especially in the case of porting code that used to use boost::optional. Many of these programmers have decided to either take the very painful route of transitioning, or to simply declare it a non-starter and just use boost.

This has forced many library authors in need of a vocabulary "optional" to have to add preprocessor-based switches to use different kinds of optional implementations. For a vocabulary type, optional contains an incredibly high fragmentation of implementations: some implementations are modeled for easy porting to the standard, but many still have custom support not found in the standard (void specializations, reference handling, monadic functions, and more). Implementation quality varies quite a bit among available optionals, supporting various standards, exception modes, trivial/explicit propagations, reference support, void support, and more.

Eliminating the need for this by including common goals -- standard goals -- would greatly benefit both library developers and the ecosystem at large with which they interact.

2.2. Surveying the Present

It has been over half a decade since std::optional was slated to end up in the standard, even if it only reached the International Standard in C++17. Now that C++ has come this far, this paper is going to take a survey of the landscape and of the many implementations of optional in order to analyze use cases and experience.

In furthering this goal, a survey of developers from all experience levels and professional/hobbyist tracks. While there are public implementations of std::optional in various flavors and names, it is also important to capture private interests. Several e-mails were sent out as well, and this proposal will attempt to succinctly describe both those and the survey. While the e-mails are kept anonymous and confidential (as that is the condition upon which I have accepted private communications in order to assuage the concerns of employees/employers). This is mostly to protect the innocent and be careful.

Furthermore, in the C++

2.3. Impact for the Future

The question of references being used in vocabulary types is not just for optional: all sorts of basic types where references might find their way in because they serve as a class of wrappers/composite/transportation types such as std::variant, std::tuple, std::expected and others. The decision here will rest of other vocabulary types, except the case of std::tuple where C++'s fate has already been decided by std::tie.

The solution presented further along in this paper should be extended to the other necessary vocabulary types. Not doing so risks the same degree of questionable design choices for these other types and further indecision that leads to a permeation of implementations that try to do the same things but are subtly incompatible or not very well implemented. Already, std::expected is seeing a small degree of implementation churn on the outside (not quite as much as optional).

2.4. Fragmentation

As mentioned previously, another key motivation of this paper is the surprising amount of fragmentation that exists in the C++ community regarding the optional to use. It is an incredibly poor user experience to have several types which perform fundamentally the same operations but to not cover the needs of the vocabulary type that have been demonstrated by codebases for well over a decade now. We counted at least 115 public and private implementations (many of which are listed and referenced in this paper) with varying levels of conformance, performance, and design goals. What is even more troubling is that users continue to roll their own optionals to this day, even on C++17 compliant compilers or standard libraries (e.g., with std::optional being available). Dissatisfaction with the optional provided by the standard library and its lack of features deemed useful by the broader family of C++ programmers means that in some manner the current optional has failed to meet the needs and expectations of the programmers who are both coming to C++ and the programmers who have worked in C++ with boost or similar for a long time.

3. Design Considerations

This paper reviews implementation experience, models, and theory around what a composite / wrapper type like optional should do. This paper also dives into a survey of 110+ developers (plus a few additional members who espoused their opinions directly VIA e-mail, instant messaging mediums, twitter, and elsewhere) to understand what is necessary in the optionals they use in real-world projects, company projects, hobby projects and more.

This proposal strongly encourages one of the solutions listed below, §4.1 Solution: Rebinding, shallow const, deep comparison reference types. All solutions are explored to thoroughly justify the choice. Particularly, Assign-Through, Pointer Only and No References, Just Pointers are qualitatively analyzed to determine their suitability in programming environments and how they hold up to expectations in the wild.

3.1. The Great Big Table of Behaviors

Below is a succinct synopsis of the options presented in this paper and their comparison with known solutions and alternative implementations. It does not include the totality of the optional API surface, but has the most exemplary pieces. A key for the symbols:

✔️ - Succeeds

🚫 - Compile-Time Error

❌ - Runtime Error

❓ - Implementation Inconsistency (between engaged/unengaged states, runtime behaviors, etc.)

3.2. Problems in the Design Space

As hinted at in the the table, there are numerous

3.2.1. Evaluation: Assign Through

This variation of the "complete" solution found below is an assign-through optional that attempts to mimic the semantics of a reference as much as the optional specification allows it. This includes assign-through behavior.

This "solution" is both silly and harmful to programmers due to its change-of-behavior depending purely on a runtime property (engaged vs. unengaged). As shown in §3.1 The Great Big Table of Behaviors, many of the other options are much more consistent and have much less questions surrounding the totality of its behavior.

Most notably is the lack of decided semantics for what happens in the engaged state versus the unengaged state with assignment. This version of an optional<T&> would have to rebind for assignment operators that take a right hand side of a l-value. Still, the more dangerous case is the r-value case. int a = 24; int& a_ref = a; a_ref = 12; is valid code: if we are attempting to achieve syntactic purity here, then assignment of optionals would have to tolerate this use case.

If we do not make r-value assignment a compiler error, then we are left with handling the unenaged state for r-value references in an optional, of which there is no good answer: do we throw bad_optional_access? terminate so it can be noexcept? These answers and many more do not seem to adequately address the design space, and other decisions like it are increasingly questionable in either utility or usefulness. Having optional references behave as the reference they contain is a poor design choice: trying to gloss over the fact that they are, indeed, optionals is bad in the case of references.

It is also extremely easy to write code using an assign-through optional where something that may or may not be a code smell cannot be detected simply by looking at the code. Consider the following snippet, adapted from [foonathan-optional-problems]:

int foo_bad(int x, optional<int&> maybe_y) {
	int value = 40;
	/* lots of code */
	maybe_y = value;
	/* lodes ah kode */
	return 24;
}

In an assign-through world, this code is valid because maybe_y might actually have something inside of it. In the engaged case, it will assign to whatever was previous bound inside of maybe_y. This means that you can be asking for dangling reference problems based purely on the engaged vs. unengaged states.

Static analysis tools cannot definitively point to this code as bad: the best it can offer is a warning, because there exists runtime states where this is exactly what the programmer intended to do. It also creates insanely hard to track bugs. Undefined behavior because a manifestation of a collection of runtime properties rather than a mistake that can be caught immediately by tools or by the eyes of a code reviewer is the front-running poster child for hard to track "Heisenbugs".

It is extremely clear that assign-through is not good design. It is a non-solution, and this paper actively discourages pursuit of it. This paper furthermore asserts that continued pursuit of a syntatically-equivalent but semantically-riduclous "optional reference that behaves exactly like a reference" is quite firmly outside of the scope of well-reasoned design for the goals of the type. If any optional reference -- and indeed, references in std::variant, std::any, std::expected and other vocabulary types -- goes into the C++ standard, it should not be this one.

3.3. Traps: Status Quo, but Monadic?

The highest feature request for optional from the conducted survey is monadic operations. This proposal does not go into it: it is detailed in Simon Brand’s [p0798]. However, it is critical to note that many of these chained monadic operations cannot be implemented efficiently without some form of reference handling through the optionals. Lots of implicit and uncontrolled copies can result from long chains of and_then and map that propogate optionals through, resulting in poor performance unless the user explicitly handles returning std::reference_wrappers and similar.

4. Proposed Solution

Originally this proposal laid out a conservative solution and a rebind solution, advocating for the conservative solution. Failure to gain consensus in the Committee during the San Diego 2018 has ruled out the conservative solution. Chief among the concerns was not fully defining all operations and that a type was for a template which originally had stronger semantics was not a good design. Given the discouragement of the conservative solution but the encouragement in continued scholarship and effort in determining the right semantics, the work has been done here to come up with the following solution.

4.1. Solution: Rebinding, shallow const, deep comparison reference types

This is the "complete" solution that is seen as a step up from the conservative solution. It is the version that has seen adoption in boost::optional for over 15 years: it is a rebinding optional reference with my_op = my_lvalue; being a valid expression. It uses C++11 rvalue-references to delete the rvalue-reference assignment and rvalue-reference constructors for a type T.

Rebind semantics have the benefit of having no surprises in engaged vs. unengaged states, as shown in .

boost::optional is not the only implementation of rebinding optionals. As shown in the table above, there are over 4 publicly available (and highly regarded) optional implementations that have references and behave in this fashion. It is the typical community choice when one is starting a new project, and has been a staple for many years. Among its design is the chief semantic that certain code will always be wrong, no matter what. Consider the code from §3.2.1 Evaluation: Assign Through:

int foo_bad(int x, optional<int&> maybe_y) {
	int value = 40;
	/* lots of code */
	maybe_y = value;
	/* lodes ah kode */
	return 24;
}

Now, the moment anyone sees maybe_y = value;, they immediately know it is an error. It does not change semantic meaning based on runtime properties. It also does not have potentially surprising answers to the question of "how do we handle r-values?"; those are simply compile-time errors, all the time, unless one is using optional<const T&> where the same lifetime rules from above apply to the expression in which optional<const T&> is being used. There are no heisenbugs and no surprises. On this merit alone, rebind is the only sane choice for reference-like handling in std::optional and -- indeed -- in any vocabulary type produced by the standard.

5. Implementation Experience

and §4.1 Solution: Rebinding, shallow const, deep comparison reference types both have design, implementation, and industry experience. §3.2.1 Evaluation: Assign Through has not been reported to see much experience (any experience, currently), other than my anecdotal experimentation as a beginning programmer. There is supposedly one assign-through implementation of an optional

5.1. Conversative and Rebinding

The "simple" version is a much more tame version that has seen implementation experience for at least 6 years in akrzemi/optional, and 4-5 years in sol2. The simple version has also existed as the advised compiler-safe subset for Boost.Optional for 15 years, maybe more (this was mostly from warning about the inability of pre-C++11 to delete or restrict r-value bindings and how const T& could bind to more than was likely intended).

The "complete" version has seen implementation experience for even longer for those who used the full functionality of Boost.Optional. It is also present in Simon Brand’s optional, a number of industry optionals, and the author’s independent optional. The boost mailing list thread on this topic indicates much of the same potential confusion around references, but towards the end there was the realization that due to inconsistencies with how assign-through behaves the behavior of assign-through is far from ideal.

Jonathan Müller’s type_safe has it, but under a different name (optional_ref). From Müller’s C++Now 2018 'Rethinking Pointers' Talk, it is easy to see why: he argues that using a type which very explicitly demarcates its purpose with a name (optional_ref<T> as compared to optional<T&>) is better for an API interface. This works just as fine as any other argument, until the case of generic programming comes around. This is where the difference between optional_ref and optional as 2 distinctly named, strong types becomes noticeable and painful for any given developer. This could be smoothed over by having template <typename T> using optional_ref = optional<T&>;, which helps the interface be explicit but avoids having to create multiple strong types which will convolute generic programming. Of course, the generic programmer can also add a level of indirection as mentioned in Tristan Brindle’s musings for Optional References, but this is similar to the whackiness of having to explicitly unref_unwrap<> every type just in case std::reference_wrapper shows up in your generic code.

5.2. Assign-Through: Not Good Enough?

§3.2.1 Evaluation: Assign Through has some design experience, but has zero publicly available implementation experience. Only 2 out of 110 survey responses report using an assign-through optional. Neither point to a repository. One is used for projects and hacks, the other is used for a large company project. It is notable that for the individual who reported using an assign-through optional in a company project, there was no firm conviction behind the code: it is "a lazy implementation" that predates boost, and has not actually had a reference put inside of it ever. (In other words, it does not count as implementation experience.) The only other user to have an assign-through optional in their projects put rebinding optionals on their wish list: the only people that appear to want an assign-through optional are the people that never, ever implemented or used one.

Asides from these two survey respondents, many companies, Boost Users, the twitter-verse, several Discord servers, the CppLang Slack, and many more e-mails to C++ programmers across the globe probed for real significant use of an assign-through optional. Nobody has reported using a non-rebinding or non-conservative optional solution in their code base to date, or if they do they are not aware of it and still need to get back to me.

This leaves a serious question of the validity and usefulness for §3.2.1 Evaluation: Assign Through. It may be that in publishing r0 of this paper, individuals who the author could not reach directly or by survey will come out to inform the author of a non-rebinding reference optional that has seen experience and use as a Studio, Company, and/or shop across the globe. The author encourage everyone to please submit experienced implementations.

However, given the above, §3.2.1 Evaluation: Assign Through appears to be exactly that: a fanciful unicorn that does not exist except for the sole purpose of creating unnecessary and directionless bikeshed. It is a trap that masks itself in the clothes of syntactic similarity with references while having demonstrably harmful properties that cannot stand up to even basic design principles for modern C++'s generally thoughtful and bug-proofing abstractions. It represents a foolish consistency for consistency’s sake and there should not be a future in which it exists for C++, whether that is C++20 or C++50.

5.3. The Other Choice

The other choice is, of course, the current status quo: no specialization. Libraries which take this path are llvm::Optional, core::optional, optional lite, absl::optional and the current std::optional.

Many of these do so because they claim to implement the C++17 version of optional-as-is, and try to keep strict conformance with the specification. Most implementations were done around the time of [N3793] or targeted the interface of [N3793] because that was the interface that went into the standard. Many of these implementations also claim to provide C++14/17/etc. features to older compilers (even as far back as C++98 for optional lite): feature-parity (down to the exact same bugs, even) is desirable and necessary for perfect transition and interop between e.g. absl::optional and std::optional. This unfortunately means that no creativity can be taken with the implementation whatsoever. To quote the library author of core:

core was an attempt to implement proposals to the letter. Because it [(optional references)] didn’t make it in, core doesn’t do it. — Isabella Muerte of mnmlstc/core, July 4th, 2018

This does not explain all optional implementations like this, however. For example, llvm::Optional does not make it a specialization at all, and its implementation predates the version finally ratified in the standard (it was first introduced to LLVM in 2010, but had existed before then as clang::Optional for quite a few more years).

David Blaikie of cfe-dev chimed in about the history of clang::Optional and its successor, llvm::Optional saying that he believes that when there was a need to potentially push it forward, the standards body had already begun to have serious discussion around the current optional proposal. Because that discussion contained contention about assign-through versus rebind, LLVM/Clang simply decided to not try to introduce the idiom into their code base. They therefore stuck with using T* to represent optional values.

5.4. So what about Pointers?

Pointers have long been heralded as the proper way to have a rebindable optional reference. It’s compact in size, already has a none value prepared in nullptr, and comes with the language itself. It seems to have everything needed. Unfortunately, pointers have problems: chief among them is the requirement that creation of a pointer must be explicit and must come from an l-value. This means that codebases which want to transition from either boost::optional<const T&> or from changing a const T& parameter to a T const* parameter suffer from hard compiler errors at every place of invocation. While this is not a big deal for some functions, it is an incredibly big deal for core APIs.

Pointers also introduce lifetime and scoping issues questions and issues: names have to be assigned to temporaries that otherwise had perfect lifetimes that fit exactly the duration of the function call expression: foo_bar(5); must become int temp = 5; foo_bar(&temp);. While this might be easy to do with integers, it becomes exceedingly complicated for more complex objects and other intricate types. One would have to explicitly control function call lifetime by manually sprinkling brackets around the call site. This does not scale for older codebases wishing to move themselves to more idiomatic and expressive APIs, or for refactors that

This is not a concern rooted in purely hypothetical thought: 2 survey respondents, a handful of e-mail respondents and several individuals on the CppLang Slack and Discord reported significant pain switching from optional references to T*. A programmer responsible for long-term hobby project wrote in:

Had reference support before upgrading to std::optional, porting those to pointers was quite some work... — Anonymous, July 9th, 2018

There is an observable and significant difference between having to use a pointer and being able to just have the useful lifetime extension rules apply to something that binds to a const reference. It is especially painful when one wants to upgrade a function to take a parameter that may or may not have used const T& for a parameter that becomes optional. Another way to fix this problem is overloading, but that presents the problem of making it impossible to take the address of a function name without explicitly and directly performing a static_cast<>. The only solution that requires absolutely no effort on the part of the programmer is upgrading from const T& to optional<const T&>. Jonathan Müller talks about this in his C++Now talk and in a handful of blogposts as well.

6. Acknowledgements

Thank you to sol2 users for encouraging me to fix this in the standard. Thank you to Lisa Lippincott for encouraging me to make this and one other proposal after seeing my C++Now 2018 presentation.

Thank you to Arthur O’Dwyer for pointing out that I should survey more than simply optional references, and include all other optionals as well.

Thank you to Nicole Mazzucca for some sweet code snippets to help display the desirable properties of optional.