N3657
Functions with Data - Closures in C (A Comprehensive Proposal Overviewing Blocks, Nested Functions, and Lambdas)

1. Changelog

2. Revision 0 - July 24^th, 2025

• Initial release. ✨

• This paper has no wording. It is not fit for standardization at the moment and only tries to thoroughly discuss the motivation and design behind this work.

3. Introduction and Motivation

A colloquial overview (but with a bit less technical depth) of these options is available as a writeup here ([lambdas-nested-functions-block-expressions-oh-my]), though it was written in 2021 before Heap-based trampolines existed and some of the other proposals discussed here existed. We keep it as a gentler, milder introduction to the problem space written with a much less serious prose.

C has had an extremely long and somewhat complicated history of wanting to pair a set of data with a given function call. Early problems first started with the standardization of C89’s qsort, which only took a single function pointer argument and no way to pass data through for additional constraints:

void qsort(
  void* ptr, size_t count, size_t size,
  int (*comp)(const void* left, const void* right)
);

This worked, until -- occasionally -- people wanted to provide modifications to certain behaviors based on local data rather than static data. For example, to modify the sort order of a call to qsort, one would originally have to program it like this in standard C:

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

static int in_reverse = 0;

int compare(const void* untyped_left, const void* untyped_right) {
  const int* left = untyped_left;
  const int* right = untyped_right;
  return (in_reverse) ? *right - *left : *left - *right;
}

int main(int argc, char* argv[]) {
  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };
  qsort(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare);
	
  return list[0];
}

There were multiple limiting factors in getting data into the function outside of using static. Accessing data in the local block was impossible for a compare function that, necessarily, had to be defined outside of main. This necessitated a way of transporting data to the compare function to do the work in a way that would work for qsort. Since it offered no way to transmit user data parameter, other forms of data transfer became commonplace:

• static data was the most popular ISO Standard C way of handling this in qsort-style APIs, resulting in a large number of (inline) static global variables throughout early programs using clever tricks to duplicate variables VIA fork-style or loading several translation units with different static variables;

• and, _Thread_local in the case of avoiding contention on variables when multithreading was formally introduced circa C11.

Still, for all of its benefits and ease-of-use, these techniques are not perfect. Concerns and problems around (potentially false) sharing data only grew in time as each program had to manage larger and larger swaths of 1980’s-style global variable soups, something that has famously ended up being used as a sign of negative code quality in legal matters. This particular problem was combatted by "reentrant" functions or "reentrancy" requirements, which is where the family of qsort_s and qsort_r-style functions (from Annex K implementations or BSD libraries) came from:

void qsort_s(
  void* ptr, size_t count, size_t size,
  int (*comp)(const void* left, const void* right, void* user),
  void* user
);

void qsort_r(
  void* ptr, size_t count, size_t size,
  int (*comp)(const void* left, const void* right, void* user),
  void* user
);

And they are used like so:

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

int compare(const void* untyped_left, const void* untyped_right, void* user) {
  const int* in_reverse = (const int*)user;
  const int* left = untyped_left;
  const int* right = untyped_right;
  return (*in_reverse) ? *right - *left : *left - *right;
}

int main(int argc, char* argv[]) {
  int in_reverse = 0;

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };
  qsort_r(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare, &in_reverse);
	
  return list[0];
}

While this example just has a single int as the type, other instances of using callbacks in this manner have resulted in Type Confusion bugs, where the void* pointer is cast to the wrong type or the wrong callback is used in conjunction with that void* callback type. The lack of type safety occasionally bites people, and given that it’s ferreted through a void* it’s hard to tackle this problem when dozens of little "helper" functions have to litter source code at file-scope. Finally, we add this frivolous example of ISO C (that does not make much sense upon first read):

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

typedef int compare_fn_t(const void* left, const void* right);

int compare(const void* untyped_left, const void* untyped_right, void* user) {
  const int* in_reverse = (const int*)user;
  const int* left = untyped_left;
  const int* right = untyped_right;
  return (in_reverse) ? *right - *left : *left - *right;
}

compare_fn_t* make_compare(int argc, char* argv[], int* in_reverse) {
  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        *in_reverse = 1;
      } 
    }
  }
	
  return compare;
}

int main(int argc, char* argv[]) {
  int in_reverse = 0;

  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };  

  compare_fn_t* compare = make_compare(argc, argv, &in_reverse);
  qsort_r(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare, &in_reverse);
	
  return list[0];
}

The point of the above code is to show that it’s legal to return a function pointer to a normal function call, because it has a duration for the lifetime of the program (static storage duration, basically). This serves as a good proxy for the various Closure types we will be looking at: can they be returned safely from a function? Is there a way to make it return safely from the function? If we didn’t pass &in_reverse in as an argument, but created a local variable inside of make_compare, can it survive the return? These are all important questions to answer, especially as global variables became more discouraged towards C95 and C99 to prevent unintended data clobbering or sharing, and as reentrancy started to become more important. Because of void*’s lack of type safety which enabled unfortunate type confusion bugs, sharing issues, and the lack of locality of the writing of functions, a new extension was cooked up by GCC to handle this program. Similar to Ada and Algol features but spun directly for C at the time, it was a feature lovingly dubbed Nested Functions.

3.1. Preamble

Before we talk about GNU Nested Functions, Apple Blocks, C++ Lambdas, Literal Functions, or anything else, there’s a few examples similar to the qsort one that will be used above. The purpose of these examples will be to talk about:

• whether or not there’s a way to use these things in expressions by themselves;

• whether or not there’s a way to manage the lifetime of an object in a dynamic or safe fashion;

• and, whether or not there’s a way to make these things work with a single function pointer, function pointer + void* user data, or otherwise.

The one returning a function pointer seems utterly frivolous in the above for make_compare, but is meant as a proxy to determine how to handle dynamic lifetime of functions, where a closure needs to outlive the scope in which it was created or returned. This serves as a stand-in for code similar to returning things "up" the stack, or where the closure is meant to be invoked at some later point like in asynchronous code. That is how the current landscape is going to be evaluated.

To aid in this evaluation, in the final section of this introduction, we will be filling out this table of features to see what each ecosystem brings to the table in terms of features. There are explanations below for each one in the first column:

• Capture By-Name: can refer to an object directly, as if working with it as an l-value or dereferencing a pointer to that object. Sometimes also called "capture by reference".

• Capture By-Value: can refer to a copy of an object from within the function, at some fixed point in time. The lifetime of the copied object from the capture is tied to the closure rather than the scope it comes from.

• Selective Capture: can pick and choose what to capture and how it gets captured.

• Safe to Return Closure: part of the dynamic lifetime problem, but is there a way to write this closure type such that it is safe to return?

• Relocatable to Heap (Lifetime Management): if it is possible to extend or otherwise change the lifetime of a closure so that it can last longer, either by copying or relocating the closure.

• Usable Directly as Expression: whether or not the closure can appear as a function argument or something else.

• Immediately Invokable: whether or not the closure can be immediately invoked, usually without naming it.

• Convertible to Function Pointer: whether or not the closure can be converted to a function pointer of an identical signature to be called.

• Convertible to "Wide" Function Type: whether or not the closure can be converted to a "wide" function pointer type, now or into the future.

• Access to Non-Erased Object/Type: can use and store the closure object without erasing the type or the object first.

• Recursion Possible: can refer to itself in order to create a recursive algorithm in the normal fashion, without needing a special feature like C++'s Deducing This or the proposed __self_func ([__self_func]).

3.2. GNU Nested Functions

Nested Functions ([nested-functions]) were the logical extension of function definition syntax pulled down into the local level. Despite being the oldest functions-with-data attempt (30+ years), it only has one proposal by Dr. Martin Uecker done recently ([n2661]) and is not widely adopted as an extension across C compilers. The goal was very simple, and during the C89 timeframe eminently doable due to the absence of a deep understanding of security implications for machines not yet being actively exploited in both civilian and military contexts. The syntax of a function definition within a block scope created a function that could be called in the obvious way but still reference surrounding block scope objects by name:

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

int main(int argc, char* argv[]) {
  int in_reverse = 0;

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
	
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };
	
  int compare(const void* untyped_left, const void* untyped_right) {
    const int* left = untyped_left;
    const int* right = untyped_right;
    return (in_reverse) ? *right - *left : *left - *right;
  }
  qsort(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare);
	
  return list[0];
}

The notable benefits of this approach were:

• looks, feels, and smells like a regular function definition;

• can access local data instead of static data, preventing issues with forked data or shared data;

• produced a single function pointer and so did not need an additional user data pointer (typically a void*);

• and, places the function doing the comparison work closer to the actual usage site.

Unfortunately, the early and enduring design of this feature -- in order to enable some of the benefits listed above -- very quickly ran afoul of early security concerns, and soon earned itself a big CVE due to the way it worked. In particular, the implementation strategy for a nested function is a brilliant piece of engineering that runs afoul of one of the only enduring security mitigations that have not fallen by the wayside: Non-Executable Stacks. To understanding why this matters, a brief description of what (Non-)Executable Stacks are, and why they are important.

3.2.1. Non-Executable Stacks

In C -- using common practice to leverage lots of hot-patching, assembly hot-fixing, direct opcode injection, and more -- programs frequently made use of stack-based data buffers where they also dumped their programs. This meant that binaries would read from their stack with the instruction pointer, allowing programs to dynamically inject behavior into a currently running program either in parts or -- if that data came from outside sources -- a fully dynamic sort of live-machine coding. The problem with this approach was that if a normal user could use the stack to make the program do a set of behaviors, so could a malicious actor: this was the easiest and widest branch of attacks upon C programs, and it was an endemic issue given the extremely large number of stack-based, fixed-sized buffers or -- after the advent of C99 -- variable-length arrays. Commandeering programs was as easy as finding a place where naïve programmers and hackers employed the now-deprecated gets, or where input was loaded in a too-relaxed way into a stack buffer. Overrunning the buffer and gaining control of the program by getting a jmp instruction to jump not back to some expected place in the stack but instead to a piece of a new program written by malicious inputs into the program made it easy to exploit C programs day in and day out, over and over again.

After quite a few attempts and retries on mitigations and a several years of evolution, one of the lowest-hanging and easiest mitigations for a large class of direct attacks using stack buffers was to simply make the stack non-executable. This meant directly writing shellcode and jumping to it was no longer a valid way to attack many programs, and as a simple mitigation it has endured as one of many different mitigation techniques that prevents exploits. Executable stacks are, to this day, still one of the easiest-to-exploit properties of programs on a large share of modern computing platforms.

NOTE: This does not mitigate ALL stack-based exploits completely. It just eliminated the bottom-of-the-barrel, 40% ones; instead, folks now had to use what was already on the stack in terms of data to try and trick the program’s own logic to enter an unexpected state and, summarily put it into a vulnerable position. The kickoff from that vulnerable position to either abuse another vulnerability (e.g. an improperly checked heap pointer) can then be elevated to the point of illicit code execution ([solar-non-executable-stack-exploits]). Both formally and informally, this is known as Return-Oriented Programming (ROP).

Most programs, whether using Variable of Fixed-Length Arrays, loved to keep (small)-ish buffers on the stack that they constantly wrote data into in response to user input or read data (such as configuration files or network input). Preventing the ability of a poorly written program that did not guard against all forms of malicious input from turning into an easy Remote Code Execution issue was a gigantic security win. The overwhelming majority of exploits running at the time were effectively halted, even if they contained the perfect shell code to do so.

NOTE: Similar exploit-prevention techniques such as inserting security cookies on the stack, using Address Space Layout Randomization (ASLR), Control-Flow Integrity (CFI) checks, and more have also been developed further to shore up other issues from exploitation and vulnerabilities in C code since then, after ROP, Return to Libc, Heap-based Exploiters (Write-What-Where primitives), and Type-Confusion, became the new dominating exploit techniques.

This means that as a "table stakes" or entry-level bit on security, avoiding an Executable Stack is important for most C features.

3.2.2. Early Design Flaw: Nested Functions turn the stack Executable!

The original design of Nested Functions suffered for its compact and brilliant design, unfortunately. Let’s remind ourselves of the previous example relating to qsort: somehow, without marking the variable as static or _Thread_local, a nested function is able to access all of the object by name from its enclosing scope.

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

int main(int argc, char* argv[]) {
  /* LOCAL variable.... */
  int in_reverse = 0;

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
	
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };
	
  int compare(const void* untyped_left, const void* untyped_right) {
    const int* left = untyped_left;
    const int* right = untyped_right;
    /* HOW is `in_reverse` usable here?! */ 
    return (in_reverse) ? *right - *left : *left - *right;
  }
  qsort(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare);
	
  return list[0];
}

The secret of this is in the brilliance of the original design: since it was so commonplace at the time, Nested Functions decided that the best way to be able to find the variables in the enclosing scope was to take a location associated with the enclosing block scope -- the Stack -- and turn it into an executable piece of code. This executable piece of code had an address. Because the executable code and its address came from the program’s stack, it meant that there was no need to provide a pointer to do reference-based / "by-name" object capturing. The function pointer itself served as both the jump to the code to execute, and a location that could then have a number of pre-determined, fixed offsets applied to it to reach all of the variables in that piece of code. This meant that rather than needing an object with static storage or _Thread_local storage, the block-scope variable could be accessed directly!

Unfortunately, this required the stack, still, to be executable in order to function.

This is the critical issue that has resulted in most other vendors not picking up the feature. Compilers want to work flawlessly with GCC-compiled code: this means that compiling a nested function and passing it to a function pointer has to result in a (somewhat) similar Application Binary Interface (ABI) that can be used as applications expect. This means the decision to make GCC nested functions trampoline from the stack is a non-ignorable detail. This is the primary reason Clang and other vendors have refused to implement Nested Functions, chiefly citing security concerns and the inability to develop a worthwhile ABI that is compatible. Just how prevalent are the security issues that Clang and other vendors avoided by refusing to implement the GNU Nested Functions extension? Well...

3.2.2.1. The Prevalence of Executable Stack, and GNU Nested Functions

A ton of users relied on or otherwise kept using Executable Stacks, even into today. While the non-executable stack fix was deployed to great success in the early 2020s, some platforms -- particularly, Linux and similar POSIX-based platforms -- kept this up. This clashed with users who became far too comfortable with the issues related to Executable Stack:

Year of Linux desktop will never happen as long as glibc is de-facto standard libc. As far as I know, this regression is wontfix, so many games just won’t work anymore.

Imagine being a game dev from 5 years ago and making a native Linux version of your game in a good faith, that’s 100% a net-negative money-wise, only for it to stop working in a couple years. Backward compatibility on Linux does not exist as far as regular users concerned, and the only way to make software that works in years to come is to make it for Windows, and hopefully let Valve and Wine teams handle the rest.

-- February 19, 2025, Valentin Ignatev

Briefly ignoring the extreme hyperbole: note the dates and the mentioned time of this tweet. 5 years ago; e.g. developed in 2020, with the complaint happening in 2025. The tweet included a screenshot from the article titled "The glibc 2.41 update has been causing problems for Linux gaming" ([gamingonlinux-dawe]). Windows and MacOS have been disabling executable stacks globally, by default, for years and refusing to load such applications. This complaint happened when glibc stopped forcefully setting executable stack, even if the tweet that garnered the public push back did not mention WHY these games were breaking. This talks to the severe need for a solution in this space that is not security breaking, and even telling folks to simply "turn on executable stack" for themselves is not wonderful:

... Don’t just take my advice on it though if you’re a developer or gamer reading this, always look up what you’re doing fully. Run at your own risk. ...

-- February 13, 2025, Article Author Liam Dawe

Unfortunately, this lax approach to security -- especially for video games -- has resurfaced quite a few truly unfortunate bugs. Popular Windows games such as Dark Souls: PREPARE TO DIE Edition and Call of Duty: WWII have exploitable Remote Code Execution (RCE) bugs in their code. One of them seems to be getting patched, but the other -- in Dark Souls -- is not going to be patched out. The idea that glibc -- or platforms in general -- can take a lax position to security on devices, even for something as "frivolous" as the (multi-billion dollar revenue) video game industry is simply not tenable. From a highly skilled Rendering Engineer commenting on the controversy caused by the glibc 2.41 update:

glibc is in the right here. iirc windows and mac DEP policy disabled executable stack by default for the past ~20 years or so. shocked this was not already the case in linux userland

-- February 21, 2025 A. W. R.

The engineer in charge of pushing the change and looking over the situation commented on the article and general attitude represented by Mr. Ignatev:

It is interesting that the headline did not get into details why I made this change: https://sourceware.org/pipermail/libc-alpha/2024-December/163146.html

In a short: the old behavior was used in a know RCE described in CVE-2023-38408.

-- February 20, 2025, Adhemerval Zanella

It is important to know that many other developers do not share this perception. Even as far back as 2018, when Microsoft kept up its Security Posture in its Windows Subsystem for Linux, people railed against the high-security default of refusing to work with programs that allowed executable stack ([WSL-no-executable-stack]):

The accepted trade-off is to have a non-executable stack be default but have an executable stack for programs which need them. Not supporting this is just a deficiency in WSL.

• August 7, 2018, Dr. Martin Uecker

NOTE: This does not necessarily mean that Dr. Uecker wanted an executable stack in all programs. It’s just that for backwards compatibility purposes, it should be allowed to happened rather than fully banned, which is the opposite opinion of how WSL1, SELinux, several BSDs, and many other operating system loaders work.

Whose perspective is correct?

3.2.2.2. The Standard Is To Blame

This proposal agrees with both Mr. Zanella and Ms. A.W.R., and disagrees with Dr. Uecker and Mr. Ignatev; it is impossible to pretend like this is not a problem with more and more exploits taking advantage of not only executable stack, but directly targeting "harmless" software that makes use of it (such as video games). But, even more importantly, the real culprit here is ISO C. There were many alternatives to executable stack-based GNU Nested Functions, and other such entrapments. However, because of how convenient GNU Nested Functions are and how accepted they are in the GNU ecosystem, it has led to multiple security vulnerabilities that should have never existed in the first place (§ 6.4 Executable Stack CVEs).

Unlike Address Space Layout Randomization (ASLR) and several other run-time mitigations, non-executable stacks have been both the cheapest and most enduring security wins in the last 50 years of computing. Marking sections of memory as unable to be run as part of the program permanently shifted the landscape of targeted exploits to be focused almost exclusively on buffer overrun-into-heap-exploitation bugs or through ROP gadgets, as well as trying to find sequences of logic to put programs in a state of disrepair that ultimately grant attackers either a Denial of Service (DoS) attack or full control VIA Remote Code Execution (RCE). Exploits were now exceedingly harder and required orchestration of multiple carefully-crafted scenarios to hit the "Weird Machine" state so coveted by exploiters. This is not the case for executable stack.

3.2.3. Alternative Nested Function Implementations

We have already discussed the first and most popular attempt which leaves the stack executable. Because of the negative security properties of this, there were two more attempts, one still on-going (§ 3.2.3.2 Attempt 3) and one with limited success that requires the entire world to be recompiled or face potential ABI breaks (§ 3.2.3.1 Attempt 2).

3.2.3.1. Attempt 2

An Ada-style Function Descriptors implementation was attempted. The problem with this change for GCC is that it uses a bit (the lowest bit, which traditionally is always 0) in the function pointer itself to mark the function pointer as one that is relying on the Function Descriptor technology. Setting the lowest bit on the function pointer means that it is unsafe to call directly, and therefore every function call must first be masked with func_ptr & ~0b1ull before being called. This is a runtime cost and a general-purpose pessimization that applies to ALL function pointers, making the Function Descriptor approach unsuitable for solving the ABI problem both internally with existing GCC code and to the satisfaction of other developers.

3.2.3.2. Attempt 3

A third attempted implementation of Nested Functions attempts to use a separately-allocated trampoline. It can come from either: a stack that is set up at program load time in coordination with the compiler and whose exclusive purpose is to be a memory region for both trampolines and a slot for a void* environment/context; OR, a dynamic allocation that serves as the function pointer plus a void* environment/context pointer. These approaches simply do not work in the general case because it is unclear when, if ever, the function pointer will stop being used. However, one part of Nested Functions -- the fact that they refer to everything by name / "capture by reference" all of the things they use -- means that this can be sufficiently approximated by simply stating that GNU Nested Functions will deallocate such a trampoline (or shrink the stack it was cut from) when the enclosing block that was used for the nested function is gone. (Of course, this means that the function pointer will exhibit the exact same lifetime issues as with the current stack, so it solves some problems but leaves others on the table.)

There is also the slight issue that using a separated trampoline that is on a separate heap or a separate stack might need (but is not necessarily required) a secondary level of indirection. The original, executable stack implementation of GNU Nested Functions prevents this because both the code and the variables are in-line: having a separated trampoline may require such a trampoline to first load the right function call with __builtin_call_with_static_chain compiler intrinsic within the code of the trampoline to have the code work properly.

NOTE: As of early 2025 in GCC 14, GCC provided a heap-based implementation that got rid of the executable stack. This requires some amount of dynamic allocation in cases where it cannot prove that the function is only passed down, not have its address taken in a meaningful way, or if it is not used immediately (as determined by the optimizer). It can be turned on with -ftrampoline-impl=heap.

3.2.3.3. A Possible 4th Attempt: Explicit User Control

An experimental technique for allocating a trampoline can be done by having an _Any_func* pointer, as is being standardized by using an in-progress proposal [_Any_func]. Then, rather than needing to implicitly create a trampoline on usage, a user can instead request it and control the allocation explicitly, while passing it back. A fictional example of such an intrinsic -- called __gnu_make_trampoline -- is seen here:

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

int main(int argc, char* argv[]) {
  /* LOCAL variable.... */
  int in_reverse = 0;

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
	
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };
	
  typedef int compare_fn_t(const void* left, const void* right);

  int compare(const void* untyped_left, const void* untyped_right) {
    const int* left = untyped_left;
    const int* right = untyped_right;
    return (in_reverse) ? *right - *left : *left - *right;
  }
  // explicitly make a single-function-pointer trampoline, without an executable stack
  // __gnu_make_trampoline takes a function identifier, returns a _Any_func*
  compare_fn_t* compare_fn_ptr = __gnu_make_trampoline(compare);

  // use it
  qsort(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare_fn_ptr);

  // explicitly free a single-function-pointer trampoline, without an executable stack
  __gnu_destroy_trampoline(compare_fn_ptr);

  return list[0];
}

This is discussed further in § 6.3 Make Trampoline and Singular Function Pointers.

3.2.4. The Nature of Captures

There’s a final issue with nested functions, and it’s that it is not suitable for use with asynchronous code or code that returns "up". Consider the same bit of code as before, but slightly modified:

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

typedef int compare_fn_t(const void* left, const void* right);

compare_fn_t* make_compare(int argc, char* argv[]) {
  /* LOCAL variable.... */
  int in_reverse = 0;

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
	
  int compare(const void* untyped_left, const void* untyped_right) {
    const int* left = untyped_left;
    const int* right = untyped_right;
    return (in_reverse) ? *right - *left : *left - *right;
  }
  return compare;
}

int main(int argc, char* argv[]) {
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };  

  compare_fn_t* compare = make_compare(argc, argv);
  qsort(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare);
	
  return list[0];
}

In this example, we have simply moved the in_reverse and compare generation into a function, for ease-of-use. One can imagine that we need to create this sort of function multiple times, from perhaps different sources of data. GNU Nested Functions allow us to do this and to return the function "up" the call stack. The problem, of course, is that Nested Functions (in a heap-based implementation or the current executable stack implementation) both point to the current "function frame" that it is created in. That is, while make_compare -- once it has returned -- is no longer alive and all of its automatic storage durations have died, the compare function pointer is still there and passed up the stack. This means that all accesses to in_reverse are accessing memory that is no longer alive, and it is effectively Undefined Behavior.

The actual manifestation of the undefined behavior in this program is very clear: adding the -r argument to make in_reverse turn to 1 does not have any effect on the program anymore:
https://godbolt.org/z/81d7Tqn1E

This is a critical failure of Nested Functions: it only ever "captures" function values by-name / by-reference. There is no option to capture by-value, and therefore the transportation and use of these function pointers to asynchronous code or "up" the call stack means it is fundamentally dangerous. This was not a huge problem in the early days of C, where programs were very flat and it was easy to always "move" function calls up. Unfortunately, we now have asynchronous programming, coroutine libraries / green threading models, callbacks that are saved and invoked much, much later in a program, and all sorts of models for shared code. This is the part of Nested Functions that cannot be saved; it is an intrinsic part of the design that unfortunately will always lead to Undefined Behavior because there is no way to get around that limitations in the GNU Nested Functions design. This is another serious problem that ultimately make it impossible to consider Nested Functions as THE solution for all of the C ecosystem.

NOTE: As a general rule of thumb, if the entity being designed has the ability to be transferred out of its current scope (Blocks, Nested Functions, Lambdas, ...) then it must as a rule allow for determining how it interacts with the scope it is nested in. The answer for function calls in most languages (without a garbage collector or other memory-preserving solution) is "cannot interact with its surrounding scope", which simplifies the problem. But, the whole point of these features IS to interact with the surrounding scope, and so care must be taken to make it work better.

3.2.5. GNU Nested Functions By-Name Captures Cannot Be Worked Around Normally

The deeply unfortunate part of GNU Nested Functions is that even if someone realizes the issue with by-name capture of the surrounding scope and tries to escape it, there is not successful way to actually provide that escape. Consider, briefly, the make_compare-style example from before but with a slight modification to "heapify" the in_reverse variable for safety reasons:

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

typedef int compare_fn_t(const void* left, const void* right);

compare_fn_t* make_compare(int argc, char* argv[]) {
  /* LOCAL, heap-allocated variable.... */
  int* in_reverse = malloc(sizeof(int));
  *in_reverse = 0;

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        *in_reverse = 1;
      } 
    }
  }
	
  int compare(const void* untyped_left, const void* untyped_right) {
    const int* left = untyped_left;
    const int* right = untyped_right;
    return (*in_reverse) ? *right - *left : *left - *right;
  }
  return compare;
}

int main(int argc, char* argv[]) {
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };  

  compare_fn_t* compare = make_compare(argc, argv);
  qsort(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare);
	
  return list[0];
}

Ignoring for a moment that there’s no free called in this scenario, the bigger and more pressing problem here is that this code does not even work. Despite having properly had the memory on the heap now, a by-name reference to the in_reverse pointer means that once the make_compare function exits, that pointer is no longer alive. Doing *in_reverse is just deferfencing a pointer whose value may or may not have changed and is effectively a form of stack-based use-after-free. This means that even if you want to try to make local variable references "safe" by relocating objects in the local arena to the heap, it is still not enough. You would need to deploy static or _Thread_local data in order to solve the problem, still, which brings us back to the original problems at the beginning of this introduction (§ 3 Introduction and Motivation).

3.2.6. Additional Modifications for Nested Functions

The paper [n3654] discusses various potential modifications and directions for GNU Nested Functions. It contains many assertions and future directions, which are discussed in the Appendix (§ 6.1 Accessing Context in Nested Functions).

3.3. Apple Blocks

Blocks are an approach to having functions and data that originate from Objective-C ([apple-blocks]) and are associated with Apple’s Clang. They were proposed a long time ago in an overview of Apple Extensions for C by Garst in 2009 ([n1370]), refined into a specific proposal in 2010 ([n1451] [n1457]). It was later further refined into a proper "Closures" proposal, and the name changed ([n2030]). However, none of these papers made the standard and despite a brief moment where GCC maintained a Blocks runtime, the extension has not been adopted outside of the C / Objective-C ecosystem (and the Blocks extension is no longer allowed or used at the moment).

Because there is a wealth of proposals and literature talking about Blocks, their implementation, their runtime, and more (see an answer discussing the intrinsics and implementation bits for Blocks), rather than inform proposal readers how they work and why they are not suitable for ISO C, this proposal will focus exclusively on why they are not usable for the whole C ecosystem.

3.3.1. Expression

Unlike GNU Nested Functions, Block definitions are an expression. That means that -- given appropriate typing -- one can use blocks directly within a function call:

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

int main(int argc, char* argv[]) {
  int in_reverse = 0;

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
	
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };
	
  qsort_b(list, (sizeof(list)/sizeof(*list)), sizeof(*list),
    ^(const void* untyped_left, const void* untyped_right) {
      const int* left = untyped_left;
      const int* right = untyped_right;
      return (in_reverse) ? *right - *left : *left - *right;
    }
  );
	
  return list[0];
}

The special function_type^ and return_type (identifier^)(argument0, argumentN...) are Block types, which are special types that act as a handle to a Block. Blocks are not just a simplistic combination of a function and a context, however: much more effort is put into making them safe at execution time, and that is done by putting everything related to Blocks behind a hefty runtime.

3.3.2. Runtime Required

Apple Blocks are, at their core, dynamic objects that engage in type-erasure at the top-most level. Where C++ lambdas are completely non-type-erased and each contains a unique type, and where Nested Functions are completely type-erased behind normal function pointers with executable stacks + trampolines, Blocks are callbacks that are unique but have all of their type information erased and carted around in a new function pointer-like construct: the block. Block types are denoted by the ^ in their function type name, and typically are cheaply copiable handles to a heap-stored callable.

NOTE: The layout of the heap-stored callable is dictated by Apple, and has been reverse-engineered and deconstructed many times. Some of this work has been done by the Clang Team, and is the most up-to-date, thorough specification for it is stored in with the Clang documentation ([clang-blocks-spec]).

Of course, all of this is just to illustrate the problem: while Microsoft as a platform may not need to care, GCC and Clang both tend to occupy the same hardware and software spaces. Even if one compiler or another figured out how to be clever, the base layout -- and the premise of blocks being a handle to a heap, and not a compile-time sized object -- means that some form of dynamic allocation or heap is required. This is a net-negative for memory-constrained environments, and in implementations that attempt to be ABI-compatible with the original Apple Blocks implementation will be forced to lay their blocks out in the way that Apple has already specified.

3.3.2.1. More Complications: Generally Unsafe to Return

All block literals are not initially placed on the heap or allocated through the run-time, as a blanket optimization applied to all block literals. They start out on the stack! Which means that while the above code using a literal works just fine, this code is actually Undefined Behavior, in a way that’s equally as bad as GNU Nested Functions:

#define __STDC_WANT_LIB_EXT1__ 1

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

typedef int compare_fn_t(const void* left, const void* right);

compare_fn_t^ make_compare(int argc, char* argv[]) {
  /* LOCAL variable.... */
  int in_reverse = 0;

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
	
  compare_fn_t^ compare = ^(const void* untyped_left, const void* untyped_right) {
    const int* left = untyped_left;
    const int* right = untyped_right;
    return (in_reverse) ? *right - *left : *left - *right;
  };
  return compare;
}

int compare_trampoline(const void* left, const void* right, void* user) {
  compare_fn_t^* p_compare = (compare_fn_t^*)user;
  return (*p_compare)(left, right);
}

int main(int argc, char* argv[]) {
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };  

  compare_fn_t^ compare = make_compare(argc, argv);
  qsort_s(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare_trampoline, &compare);
	
  return list[0];
}

This code does not work. Despite having a runtime, it will NOT perform the lift to the heap automatically. The return from make_compare into compare_fn_t^ compare is a dangling reference, and is explicitly discouraged by reference materials and documentation from Apple. It must be modified to use Block_copy on the return:

#define __STDC_WANT_LIB_EXT1__ 1

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

typedef int compare_fn_t(const void* left, const void* right);

compare_fn_t^ make_compare(int argc, char* argv[]) {
  /* LOCAL variable.... */
  int in_reverse = 0;

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
	
  compare_fn_t^ compare = ^(const void* untyped_left, const void* untyped_right) {
    const int* left = untyped_left;
    const int* right = untyped_right;
    return (in_reverse) ? *right - *left : *left - *right;
  };
  return Block_copy(compare);
}

int compare_trampoline(const void* left, const void* right, void* user) {
  compare_fn_t^* p_compare = (compare_fn_t^*)user;
  return (*p_compare)(left, right);
}

int main(int argc, char* argv[]) {
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };  

  compare_fn_t^ compare = make_compare(argc, argv);
  qsort_s(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare_trampoline, &compare);
  Block_release(compare);

  return list[0];
}

Annoyingly, every call to Block_copy must be paired with a call to Block_release. This means that there’s now an invisible (from the perspective of main) block copy that now needs to be managed with a specific call to Block_release. One can imagine that every function that returns a Block type should just be assumed to need releasing, but this isn’t always the case: this means there’s an invisible lifetime tracking that even the runtime and the heap does not solve for us! Truly, unfortunate.

There is also a small gotcha in this example, that only shows up based on where the compare block is created. This has to do with how Captures work under the Blocks feature.

3.3.3. Captures

Captures in Apple Blocks work in one of two ways.

• referring to the existing object by-name in your code, which will make a copy of the object inside of the Block’s implicit capture;

• or, annotating a variable with __block, which will load the object into a sort of intrusive pointer / automatic reference-counted place in memory (managed by the Blocks runtime) to allow it to be referred to be both the surrounding code and the block’s inner guts by-name.

The reason Apple used this technique, as talked about before, is because it’s safer than Nested Functions in the particular regard of using variables and carting them around.

#define __STDC_WANT_LIB_EXT1__ 1

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

typedef int compare_fn_t(const void* left, const void* right);

compare_fn_t^ make_compare(int argc, char* argv[]) {
  /* LOCAL variable.... */
  int in_reverse = 0;

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
	
  compare_fn_t^ compare = ^(const void* untyped_left, const void* untyped_right) {
    const int* left = untyped_left;
    const int* right = untyped_right;
    return (in_reverse) ? *right - *left : *left - *right;
  };
  return Block_copy(compare);
}

int compare_trampoline(const void* left, const void* right, void* user) {
  compare_fn_t^* p_compare = (compare_fn_t^*)user;
  return (*p_compare)(left, right);
}

int main(int argc, char* argv[]) {
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };  

  compare_fn_t^ compare = make_compare(argc, argv);
  qsort_s(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare_trampoline, &compare);
  Block_release(compare);
	
  return list[0];
}

The first thing to note is that, because this cannot be translated to a single function pointer, we cannot use qsort. This means that using such a function without creating some kind of special trampoline is off-limits to us. Again, this is something that could be solved by the introduction of an explicit heap-based trampoline creator (§ 6.3 Make Trampoline and Singular Function Pointers). Or, one would need to introduce a "wide function pointer" type -- which is exactly what function_type^ is -- and change qsort’s signature to use it for the callback.

NOTE: Simply upgrading qsort with a Block type is an ABI break that would cause old, already-compiled libraries and programs mixed with new programs to combust in painful, hard-to-detect ways. To solve this problem one would need to rely on existing C extensions like Assembly Labels, or work towards Transparent Aliases ([transparent-aliases]).

But, if someone uses qsort_s -- which takes a void* user data parameter -- one can use it without altering the signature of qsort directly and create a void*-kick off point as a trampoline. However, there is a bit of an interesting conundrum. Take, for example, moving the creation of the block function further upwards inside of make_compare:

#define __STDC_WANT_LIB_EXT1__ 1

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

typedef int compare_fn_t(const void* left, const void* right);

compare_fn_t^ make_compare(int argc, char* argv[]) {
  /* LOCAL variable.... */
  int in_reverse = 0;
	
  compare_fn_t^ compare = ^(const void* untyped_left, const void* untyped_right) {
    const int* left = untyped_left;
    const int* right = untyped_right;
    return (in_reverse) ? *right - *left : *left - *right;
  };

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
  return Block_copy(compare);
}

int compare_trampoline(const void* left, const void* right, void* user) {
  compare_fn_t^* p_compare = (compare_fn_t^*)user;
  return (*p_compare)(left, right);
}

int main(int argc, char* argv[]) {
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };  

  compare_fn_t^ compare = make_compare(argc, argv);
  qsort_s(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare_trampoline, &compare);
  Block_release(compare);
	
  return list[0];
}

This code will sort the list incorrectly even if in_reverse is set to 1. That’s because Blocks will capture the variables that get used at the point-of-creation. The value at the point-of-creation is 0, therefore, in_reverse is 0 when the function is called later. Even though the in_reverse variable is copied into the block, the block is now sensitive to where it is being created without any indication that it behaves that way. This is safe, but the behavior would throw someone who uses Nested Functions religiously off completely.

This is, of course, easily fixable by just... moving it down, so it’s hardly a problem:

#define __STDC_WANT_LIB_EXT1__ 1

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

typedef int compare_fn_t(const void* left, const void* right);

compare_fn_t^ make_compare(int argc, char* argv[]) {
  /* LOCAL variable.... */
  int in_reverse = 0;
	
  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }

  // Compare function, with block copy: copies the right value
  compare_fn_t^ compare = ^(const void* untyped_left, const void* untyped_right) {
    const int* left = untyped_left;
    const int* right = untyped_right;
    return (in_reverse) ? *right - *left : *left - *right;
  };

  return Block_copy(compare);
}

int compare_trampoline(const void* left, const void* right, void* user) {
  compare_fn_t^* p_compare = (compare_fn_t^*)user;
  return (*p_compare)(left, right);
}

int main(int argc, char* argv[]) {
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };  

  compare_fn_t^ compare = make_compare(argc, argv);
  qsort_s(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare_trampoline, &compare);
  Block_release(compare);
	
  return list[0];
}

If you don’t want to move the creation of the Block for some particular reason, it is not the only way to capture a variable for Blocks! The second way it can capture variables is by using __block, which works like so:

#define __STDC_WANT_LIB_EXT1__ 1

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

typedef int compare_fn_t(const void* left, const void* right);

compare_fn_t^ make_compare(int argc, char* argv[]) {
  /* BLOCK-QUALIFIED variable.... */
  __block int in_reverse = 0;
	
  compare_fn_t^ compare = ^(const void* untyped_left, const void* untyped_right) {
    const int* left = untyped_left;
    const int* right = untyped_right;
    return (in_reverse) ? *right - *left : *left - *right;
  };

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
  return Block_copy(compare);
}

int compare_trampoline(const void* left, const void* right, void* user) {
  compare_fn_t^* p_compare = (compare_fn_t^*)user;
  return (*p_compare)(left, right);
}

int main(int argc, char* argv[]) {
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };  

  compare_fn_t^ compare = make_compare(argc, argv);
  qsort_s(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare_trampoline, &compare);
  Block_release(compare);

  return list[0];
}

This will work as expected, even though the creation of the block comes after in_reverse is declared and before modification of the variable. __block lifts the variable being declared up into a heap (or heap-like space) that is managed by either a garbage collector or an automatic reference-counted memory implementation. This both reverses the onus of capturing / handling the variable onto the surrounding scope and makes it safe-by-design. This does leave an unfortunate gap in that there’s no way to do the dangerous thing or opt into a direct reference without making an explicit declaration of the pointer and then using the pointer by-copy instead in that block, which can leave some memory footprint and program speed on the table without aggressive optimization.

NOTE: The address of in_reverse might actually change, depending on if the Apple intrinsic Block_copy is used to copy the block itself before being run. This code does not depend on it, but a hash map that stores the address of variables might experience the address of any __block-annotated variables changing between creation and the innovation of Block_copy. This was changed later on to always set the variable in a location so that a steady address exists, but how conforming it is to keep the old behavior is likely an implementation choice. While there was previously a Blocks runtime for GCC, it’s fallen off: it may make a comeback again in order to be more compatible with the C and C++ code on Apple platforms: whether they will choose the same implementation technique is not known as of the writing of this paper.

All in all, however, these two things are safe: either a copy is happening and is stored along with the creation of the block on the heap, or the variable itself is having its lifetime prolonged by a sort of automatic-tracking. The second of these is very against the typical properties of C, but that matters little in the face of the obvious safety it brings to the table. Unfortunately, because all of this happens magically and in a mostly-unspecified manner, it’s very detrimental to the proliferation of the C ecosystem and having several loosely-connected implementations working towards the same improved implementations.

3.3.4. Optimization: Folding Escapes

As a matter of optimization, Blocks do not necessarily have to pollute the heap. And indeed, most immediately invocations of a block or pass-down (rather than pass-up) invocations that are visible will be optimized into a direct call. Unfortunately, this is not something that is encouraged by the general design of Apple Blocks and Objective-C or Objective-C++. Because taking the address or passing the function along leaves it open to how far the handle-to-some-heap Block-typed object might be passed, compilers have to correctly (and pessimistically) generate the full, indirect-function-call representation as a matter of course. Block_copy also needs to be used, explicitly, in many cases, making it not much better in regular C code that uses malloc.

As a brief aside in Programming Language design, this sort of optimization problem is mitigated by changing how the defaults are applied and giving the user explicit control. For example, the Swift Programming Language solves this problem while still being compatible with Objective-C and C++ by making it so every "Block" or "Closure" type must be annotated if it "escapes" beyond the compiler’s knowledge, otherwise the program just refuses to compile ([swift-escapes]). This allows aggressive optimization to be applied by-default, with weaker static analysis-based optimizations or escape analysis optimization only acting as a fallback in the @escaping-annotated case. The annotation also makes it so Block_copy does not need to be invoked and rather than having to copy it to a heap version of itself, it is simply put in the right format and place, ready to interoperate cleanly with C, C++, Objective-C or Objective-C++.

NOTE: Swift’s native function type and ABI is not identical to the Blocks type at all. This is just an example in how designing with no-escape as a default and then taking it off to allow for taking its address or returning it from a function

In the opposite direction, the C-style attribute that can be used to say "the closure never escapes", which enables optimizations for crunching the object down and assuming that it never is invoked outside of the function. This is available through the NS_NOESCAPE macro, which expands to __attribute__((noescape)) and can be used to gain better binary size and reduce indirection for code execution speed where possible.

3.3.5. (Explicit) Trampolines: Page-based Non-Executable Implementation

Objective-C has the ability to create a C-style function pointer from a Block ([objective-c-block-trampoline]), and this implementation can be wielded from regular C code on Apple too. This implementation is an entirely different version that does not use a heap but instead a separate "stack" (a single page). That page is a writable (but NOT executable) slab of memory trampolines and Block pointer data in it. The actual trampoline function pulls from this single page one of the data pointers and then sets it up to call; because the trampoline is separate from the actual data, there is much less security risk from writing and reading a pointer that otherwise might be local to executable memory.

However, because of the implementation, occasionlly Objective-C can run out of trampolines: if enough are created before any are deallocated, the entire page can be filled up with trampolines. This will trigger errors and failures to create the block. Therefore, even compared to heap-based GNU Nested Function implementation (§ 3.2.3.2 Attempt 3), there exists a potential tradeoff in the designs. This is why trampolines need to be explicit and should be handled in a separate paper, with an interface that can be generalized and can report potential errors (§ 6.3 Make Trampoline and Singular Function Pointers).

3.4. C++-Style Lambdas

C++ lambdas, despite coming from C++ initially, is the only solution not to apply executable stack, separate stack, Function Descriptors, or other dynamic/runtime data, to the problem. It was detailed in a large collection of proposals from Jens Gustedt and almost made C23, but ambition in trying to allow for type-generic programming through lambdas with auto parameters stalled progress and ultimately halted everything ([n2923] [n2924] [n2892] [n2893]). It makes a unique type for each and every lambda object that gets created using the syntax [ captures ... ] ( args ... ) { /* code ... */ }, and that object has an implementation-defined but compile-time known size.

NOTE: While type inference for function returns are not yet in, a section of [n2923] was broken off for just variable definitions which ultimately succeeded. Type-inferred variable declarations work properly in C23 and are an important part of Lambdas being able to work in the manner envisioned by Gustedt’s proposals and by C++.

If a lambda has no captures, it can reduce to a function pointer like so:

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

static int in_reverse = 0;

int main(int argc, char* argv[]) {

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
	
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };
	
  auto compare = [](const void* untyped_left, const void* untyped_right) {
    const int* left = (const int*)untyped_left;
    const int* right = (const int*)untyped_right;
    return (in_reverse) ? *right - *left : *left - *right;
  };
  // "compare" below becomes a function pointer
  qsort(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare);
	
  return list[0];
}

The one thing that makes it different from GNU Nested Functions -- and somewhat similar to Blocks -- is that it is also an expression. That means that it can also be used inside of a function call expression as an argument, or as part of any other complicated chain of additions:

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

static int in_reverse = 0;

int main(int argc, char* argv[]) {

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
	
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };
	
  qsort(list, (sizeof(list)/sizeof(*list)), sizeof(*list),
    [](const void* untyped_left, const void* untyped_right) {
      const int* left = (const int*)untyped_left;
      const int* right = (const int*)untyped_right;
      return (in_reverse) ? *right - *left : *left - *right;
    }
  );
	
  return list[0];
}

Again, this only works because there are no captures. When captures get involved, the above code will simply stop compiling at all because the conversion to a function pointer stops working. This can be solved in the ways discussed previously, such as:

• a version of qsort which takes a void* user data parameter, like qsort_s does;

• a version of qsort which takes a "wide" function pointer types, similar to how Blocks do with the qsort_b defined by Apple/BSDs;

• or, allowing a separate facility which lifts the complex closure type into a single function pointer through a trampoline or similar (§ 6.3 Make Trampoline and Singular Function Pointers).

This gives Lambdas a heavy weakness similar to all of the other solutions: there must be either a trampoline, a wide function pointer type, a void* user data/context pointer, or a something else to accommodate the lack of transformation into a singular function pointer.

NOTE: C++ as a language has a more robust set of core primitives, they don’t have to worry about this problem. std::function<FUNCTION_TYPE> is a type-erased way to transport a whole function object, as a copy, through API boundaries that is defined entirely in their library mechanisms. For a view into any old function, there is std::function_ref<FUNCTION_TYPE>, which is like a function pointer but allows pointing into many different kinds of functions that exist in C++. This makes the integration of lambdas into C++ easy; in C, it is much more difficult to have this all participate in the system automatically. There is no std::function-alike in C, and there’s no std::function_ref-alike in C either for new function types like GNU Nested Functions, Apple Blocks, or otherwise. This problem has to be solved separately, with a Wide Function Pointer type (§ 6.2 Wide Function Pointer Type) or with explicit user trampoline-making capability such as § 6.3 Make Trampoline and Singular Function Pointers.

3.4.1. Captures and Data

Lambdas do not capture any context by default, unlike both GNU Nested Functions or Apple Blocks. Instead, every capture must be manually annotated as either being taken by value or by reference, or a "universal capture" must be used to set a default method of capturing all visible, in-scope, block objects.

#define __STDC_WANT_LIB_EXT1__ 1

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

// a new kind of return type: "inferred"
auto make_compare(int argc, char* argv[]) {
  int in_reverse = 0;
	
  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
  // explicitly capture in the "[ ]" of the lambda
  auto compare = [in_reverse](const void* untyped_left, const void* untyped_right) {
    const int* left = (const int*)untyped_left;
    const int* right = (const int*)untyped_right;
    return (in_reverse) ? *right - *left : *left - *right;
  };
  return compare;
}

int main(int argc, char* argv[]) {
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };  

  auto compare = make_compare(argc, argv);
  auto compare_trampoline = [](const void* left, const void* right, void* user) {
    typeof(compare)* p_compare = user;
    return (*p_compare)(left, right);
  };
  qsort_s(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare_trampoline, &compare);

  return list[0];
}

This, unfortunately, also makes them susceptible to location just like Blocks; the moment of creation during execution is the state they capture when using unadorned identifiers in_reverse and = captures. This code would capture in_reverse before any important modifications happens.

#define __STDC_WANT_LIB_EXT1__ 1

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

// a new kind of return type: "inferred" (`auto`)
auto make_compare(int argc, char* argv[]) {
  int in_reverse = 0;
	
  // "capture all variables" annotation `=` -- same as writing the flat name of
  // every object currently in-scope.
  auto compare = [=](const void* untyped_left, const void* untyped_right) {
    const int* left = (const int*)untyped_left;
    const int* right = (const int*)untyped_right;
    return (in_reverse) ? *right - *left : *left - *right;
  };
  // lambdas and captures do not reflect any changes beyond this point,
  // including the `in_reverse`

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
  return compare;
}

int main(int argc, char* argv[]) {
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };  

  auto compare = make_compare(argc, argv);
  auto compare_trampoline = [](const void* left, const void* right, void* user) {
    typeof(compare)* p_compare = user;
    return (*p_compare)(left, right);
  };
  qsort_s(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare_trampoline, &compare);

  return list[0];
}

Like Apple Blocks, a lambda with captures is safe to return with the by-value capture (if one briefly ignores the need for Block_copy to reseat the memory of a Block). Additionally, it is better here because there is no usage of the heap needed to do this.

ASSERTION: This is PRIMARILY due to C++-style Lambdas just being normal objects. They have a compile-time sizeof(...), a compile-time alignof(...), can have their unique type inspected with typeof(...) (decltype(...) in C++), and are generally autonomous. There’s no erasure happening here as is with Blocks (the function_type^ type) or Nested Functions (the function_type* type); each Lambdas is a unique type, similar to the unique type gained by pairing a function with a hand-made struct.

The only problem is that this requires a feature that was proposed for C23 but didn’t make it (along with Lambdas not making it): deduced return types. There was consensus to have the feature, but the feature was bundled with the set of Lambda proposals, and thus fell through during the final stretch of C23. Therefore, a proposal for inferring the type of a function return should be separated from the previous proposals (such as [n2923]) in order to accommodate such behavior in C.

One of the things that’s better about Lambdas over Apple Blocks is that they also allow for by-name capture, just like Nested Functions do. So, this code -- despite having the lambda defined in main and before in_reverse is changed -- will work as expected:

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

int main(int argc, char* argv[]) {
  int in_reverse = 0;

  // & is by-reference capture
  auto compare = [&](const void* untyped_left, const void* untyped_right) {
    const int* left = (const int*)untyped_left;
    const int* right = (const int*)untyped_right;
    return (in_reverse) ? *right - *left : *left - *right;
  };

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
	
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };
	
  auto compare_trampoline = [](const void* left, const void* right, void* user) {
    typeof(compare)* p_compare = user;
    return (*p_compare)(left, right);
  };
  qsort_s(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare_trampoline, &compare);
	
  return list[0];
}

This will invoke undefined behavior in the case of moving this by-name capturing lambda into a function and then returning it. Capturing a name with &some_identifier (or using the "default capture" of & by itself) always captures by pointer of the variable.

Even if it is more explicit that in the Nested Functions case, the danger is still present and so care must still be exercised. That is, the following is undefined behavior because of the explicit by-reference & capture:

#define __STDC_WANT_LIB_EXT1__ 1

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

// a new kind of return type: "inferred" (`auto`)
auto make_compare(int argc, char* argv[]) {
  int in_reverse = 0;
	
  // capture just one variable, and capture it "by-name" / "by-reference"
  auto compare = [&in_reverse](const void* untyped_left, const void* untyped_right) {
    const int* left = (const int*)untyped_left;
    const int* right = (const int*)untyped_right;
    return (in_reverse) ? *right - *left : *left - *right;
  };

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        // lambda will reflect this change
        in_reverse = 1;
      } 
    }
  }

  // uh oh...
  return compare;
}

int main(int argc, char* argv[]) {
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };  

  auto compare = make_compare(argc, argv);
  auto compare_trampoline = [](const void* left, const void* right, void* user) {
    typeof(compare)* p_compare = user;
    return (*p_compare)(left, right);
  };
  qsort_s(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare_trampoline, &compare);

  return list[0];
}

This means that lambdas can be made to be unsafe, by capturing things whose lifetime dies even as the lambda itself is passed around or returned. This allows for a sleeker representation and no runtime-heap, but with the OBVIOUS drawback that no automated reference-counted variable also means no implicit lifetime safety like with the __block variables. Thankfully, since captures can be done both ways, the user can either choose to capture by reference, choose to capture by value, or -- if needed -- choose to allocate and then capture the new allocated pointer by value themselves. Of course, any explicit allocation will need to be freed, just as it would in a Blocks scenario. This usually implies waiting for a signaling callback from the API that it is done, or elevating the lifetime to a higher level to be deleted at a later time.

3.4.2. What About Lifetime / Destructors?

A common criticism of Lambdas and their unique type, whole-object approach is that such an approach with captures requires C++-style destructors to work well. We are completely unsure why this is the case or why this criticism keeps being levied specifically at Lambdas. In the previous section on Apple Blocks (§ 3.3.2.1 More Complications: Generally Unsafe to Return), coordinated function calls and documentation are the only way to communicate that a user has Block_copy’d an object and therefore requires Block_release. Similarly with GNU Nested Functions, returning them up the stack at all is pure undefined behavior, that has tangible effects on the program (§ 3.2.4 The Nature of Captures): these are problems endemic to C. Using a complex data structure like a binary tree or allocating memory requires that it is documented and communicated to the user: capturing such complex types and having it called over a longer period of time simply means the user has the responsibility to clean up or free the resources.

C APIs will always provide a user with provided functions a way to know when something must be cleaned up. For example, the Lua C API has an allocation function that is specifically called with a "new size" parameter of 0, it means memory passed in must be freed; that’s how it communicates what the current action is. Similarly, ev/libev -- with ev_set_allocator -- provides a hook to allow a user to manage the memory of the library, while also providing several statuses in callbacks for watchers (initialized/pending/running/stopped/etc.). Even for standard C, thrd_create passes a void* user data to the func that gets run on the new thread: a user must allocate and then pass the user data to the thread, and it becomes the thread’s responsibility to manage the lifetime of that type in a manner that is thread safe.

Any C API worth its salt, when dealing with convoluted lifetimes, provides to a given callback (through its parameters) a notification that the memory is not usable anymore, OR a separate callback (for more full-fledged APIs) that notifies the API that its finished or done with a specific operation, and therefore safe to close things out. The secondary alternative is, of course, statically sourcing lifetime until the user themselves guarantees all resources can be safely freed using some outside knowledge (e.g., an explicit set of calls after the start of the library to cleanup/close/stop the library). This is not just a point with lambdas: every attempt at solving this problem has to engage with this. Whether it’s using Block_copy to ensure the lifetime of an Apple Block, or malloc to make sure a struct type being pointed to by a void* is accessible after a dispatch to an asynchronous function call. This is simply not a problem unique to lambdas: lifetime tracking and safety will always be a problem in C because C has no extended concept of lifetime duration beyond Effective Types.

Any problem with lifetime is going to be present in every single iteration of the solution to this problem, and is going to manifest in different ways:

• UB if you return a GNU Nested Function pointer that works with variables in the function scope being exited;

• UB if you return an Apple Blocks pointer without Block_copy;

• UB if you capture things that go out of scope in general (safe defaults for Apple Blocks, safe defaults for Lambdas with [=] or just ident captures);

• UB if you convert to the wrong struct inside of an ISO C function to access a user data void* pointer;

• UB if you by-name capture in a lambda and leave the scope;

and so on, and so forth. That’s a C-intrinsic problem, and the only thing any design in this space can do is offer better tools or better control to manage or avoid such problems where possible. Lifetime management is not solvable in C as it stands, and no amount of features or tinkering or attributes will really change the intrinsic language design flaw that is pointers and references that do not have any compile-time trackable properties asides from what can be inferred with (potentially strenuous) static analysis.

One way to alleviate this -- which would be beyond what is currently within C++ and what has been proposed previously -- is to allow for lambdas (beyond their initialization/creation) to have "accessor" syntax for any of its captures using the lambda.identifier syntax. This is explored in the later design for the solutions, within § 4.2.5 NEW: Data Captures are Accessible.

3.5. Literal Functions

There is not much to say about Literal Functions ([n3645]) as they deliberately do not engage with the problem of trying to capture and use data. The syntax is based on Compound Literals, wherein the (type-name){ ... } syntax is repurposed. Currently, type-name being a function type is just a constraint violation in C, so it is safe to repurpose this syntax.

By not engaging with the closure/capture issue, Literal Functions seek to just be a prettier form of ISO C regular functions. This provides the benefits of not needing to have a wide function pointer type immediately (albeit one is still needed for the general ecosystem), and it allows code to be read in a much more friendly format by localizing the function pointer and, potentially, any user data structures that go with a void*. As a compound literal, it also immediately works since it can be created/used as an expression, meaning it can be passed to function arguments:

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

static int in_reverse = 0;

int main(int argc, char* argv[]) {
  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };
  qsort(list, (sizeof(list)/sizeof(*list)), sizeof(*list),
    (int(const void* untyped_left, const void* untyped_right)) {
      const int* left = untyped_left;
      const int* right = untyped_right;
      return (in_reverse) ? *right - *left : *left - *right;
    }
  );
	
  return list[0];
}

Unfortunately, not having any solution or future direction for captures and repurposing the compound literal syntax for it means that it seems more like a dead end. In the above example, we still have to transfer the in_reverse with a static for the qsort call. It gets slightly better if the API has a void* user data carveout, like for qsort_r:

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

int main(int argc, char* argv[]) {
  int in_reverse = 0;

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        in_reverse = 1;
      } 
    }
  }
	
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };
  qsort_r(list, (sizeof(list)/sizeof(*list)), sizeof(*list),
    (int(const void* untyped_left, const void* untyped_right, void* user)) {
      const int* in_reverse = (const int*)user;
      const int* left = untyped_left;
      const int* right = untyped_right;
      return (*in_reverse) ? *right - *left : *left - *right;
    },
    &in_reverse
  );
	
  return list[0];
}

A more pronounced example that uses more than an int shows that one can write both the struct and callback locally:

#include <stdlib.h>

typedef void async_callback_t(int result, void* data);
void async(async_callback_t* callback, void* data);

int main() {
	
  // struct and callback are next to each other
  struct { int value; }* capture = calloc(1, sizeof(*capture));  
  auto function  = (void (int result, void * data)) {
    // anonymous struct only identified by `typeof`,
    // keeps the exact type and helps reduce Type Confusion errors
    typeof(capture) captured = data;
    free(captured);
  };
  async(function, capture); // used immediately: hard to lose track
  return 0;
}

4. Design

Given the following properties from all of the extensions and proposals for this in the wild:

This proposal is going to propose two distinct options for standardization, with the recommendation to do both. It is critical to do both for the approval of the C ecosystem in general (with § 4.2 Capture Functions: Rehydrated Nested Function), and for the maximum amount of external language compatibility (C++ in particular, with § 4.3 Lambdas). The necessary and core goals of this proposal are focused on:

• compile-time knowable size of the function object (can be treated as a regular object);

• that has a unique type with its own size;

• does not require a heap or a special stack (unless type-erased or otherwise relocated by (explicit) user action);

• can be interacted with like a normal function (but not necessarily as a normal function pointer);

• and, does not compromise the security of all or a portion of the program.

Lambdas already have usage experience with well-known properties that can be directly translated to C and is easy enough to understand, despite the unfortunate syntax. Capture Functions are a simple modification of Nested Functions that produce a sized object (similar to Lambdas) and makes their captures explicit, allowing for a degree of control and additional safety that was not present in the original Nested Functions design.

We are not focused on interoperability with singular function pointers. We believe that should be left to a separate, explicit mechanism in the language, capable of allowing the user to choose where the memory comes from and setting it up appropriately. This way, a user can make the decision on their own if they want to use e.g. executable stack (with the consequences that it brings) or just have a part of (heap) memory they set with e.g. Linux mprotect(...) or Win32 VirtualProtect to be readable, writable, and executable. Such a trampoline-maker (as briefly talked about in § 6.3 Make Trampoline and Singular Function Pointers) can also be applied across implementations in a way that the secret sauce powering Nested Functions cannot be: this is much more appealing as an approach.

We DO NOT take any of the design from Blocks because the Blocks design is, as a whole, unsuitable for C. While its deployment of a blocks "type" to fulfill the necessary notion of a "wide function pointer" type is superior to what Nested Functions have produced, the implementation details it imposes for __block variables and the excessive reliance on an (underspecified) runtime/heap are detrimental to a shared & unified approach to C.

NOTE: The Blocks runtime/heap layout has changed (at least) once in its history: the only reason this worked is because Apple owned every part of the Blocks ecosystem. Apple can do whatever they want with it, however they want, whenever they want: this does not work in a language with diverse, loosely coordinated implementations like C and not Objective-C, Objective-C++, or Swift.

As the heap is (typically) repulsive to some freestanding implementations, we do not want to standardize something that will have similar technological drawbacks like VLAs, where -- even if no syntactical or language-design issues exist from the way blocks are written -- the presence of an unspecified source of memory (stack or heap) produces uncertainty in the final code generation of a program.

The feature table for these two looks like this:

4.1. What is NOT Being Proposed!

While we would like to standardize them in the future, this proposal is NOT looking to standardize statement expressions, a "make trampoline" compiler intrinsic, or a wide function pointer type.

4.1.1. Statement Expressions?

Statement Expressions should be standardized. While it is related to these efforts, it is entirely separate and has a full, robust set of constraints and concerns in standardizing. It has more existing implementation experience, deployment experience, and implementer practice than any of Blocks or Nested Functions combined. Therefore, it will be pursued in a different proposal. This was briefly noted in a proposal collecting existing extensions in 2007 by Stoughton ([n1229]); while there was enthusiastic support at the time, nothing materialized of the mention nor the in-meeting enthusiasm. Some attempts are being made at standardizing it, but it is notably difficult to standardize due to the large number of corner cases that arise from needing to clarify semantics of constructs that normally cannot appear in certain places being able to suddenly appear there, like a break; being placed in the initializer expression of a for loop.

Another advantage of Statement Expressions is that, unlike any of Apple Blocks / C++ Lambdas / GNU Nested Functions, there is no separating function body. This is critical for writing macros that coordinate with one another, AND is critical in writing reusable macros that have no additional cost and does not set up extra individual entry points. For example, there are hundreds of permutations of the functions in C2y’s <stdmchar.h> that could be written to make them easier to use, to make them not require double-pointers, to infer the size from a C-style string, and so on, and so forth. The choice of having a bunch of macros which simply repeat the same code means not having to add hundreds of permutations of the <stdmchar.h> functions (5 different character types across 5 different encoding types with 6 forms of "pointer and length, just pointer" for input/output, and typical skip/ignore/replace-character error handling strategies, pairwise with one another where order matters).

Another place that statement expressions come in handy is with RESULT/TRY/etc. macros, primarily used for low-level code where handling (and possibly enforcing error handling) is desirable through error codes and return types, as demonstrated by jade and lak here: https://gist.github.com/LAK132/0d264549745e8196df1e632d5b518c37. Being able to error and jump out or error and stop if things do not work is a very common (and powerful) idiom for writing straightforward code, and is employed heavily in many different ways across the C ecosystem in various forms.

This paper does not standardize Statement Expressions, and leaves that to a future paper similar to n3643 ([n3643]).

4.1.2. Wide Function Pointer Type?

We do hope that another paper creates a new "Wide Function Pointer" type of some kind. Some suggestions can be found in § 6.2 Wide Function Pointer Type.

4.2. Capture Functions: Rehydrated Nested Function

Capture Functions are a slight modification of the design of Nested Functions. We start from the base of Nested Functions with three goals in mind.

• Implementers are not comfortable with the implementation baggage associated with Nested Functions or maintaining potential ABI compatibility with those choices (heap/stack trampolines versus separate-page allocations).

• We want to allow a way to access captured values explicitly, and control how those captures work.

• We want them to be safe to move around and relocate, whether to the heap or copied into static memory or otherwise.

A brief demonstration of all of the well-defined behavior:

auto make_seven (int x) {
  int y = 7;
  int seven_fn() _Capture(x, y) {
    return x * y;
  }
  return seven_fn; // OK: unique type which
  // is a complete object
}

typedef int eight_fn_t();

eight_fn_t* make_eight () {
  int eight_fn () _Capture() {
    return 8;
  }
  return eight_fn; // OK: empty capture converts to function pointer
}

#if 0
typedef int nine_fn_t();

nine_fn_t* make_nine () {
  int val = 30;
  int nine_fn () _Capture(val) {
    return val;
  }
  return nine_fn; // constraint violation: cannot convert
  // captures to function pointer
}
#endif

int main () {
  int x = 3;
  int zero () {
    // OK, no external variables used
    return 0;
  }
  int also_zero () _Capture() {
    // same as above, just explicit
    return 0;
  }
#if 0
  int double_it () {
    return x * 2; // constraint violation
  }
#endif
  int triple_it () _Capture(x) {
    return x * 3; // OK, x = 3 when called
  }
  int quadruple_it () _Capture(&x) {
    return x * 4; // OK, x = 5 when called
  }
  int quintuple_it () _Capture(=) {
    return x * 5; // OK, x = 3 when called
  }
  int sextuple_it () _Capture(&) {
    return x * 6; // OK, x = 5 when caled
  }
  x = 5;
  auto seven_tuple_it = make_seven(x);
  eight_fn_t* eight = make_eight();
  return zero() + triple_it() + quadruple_it()
    + quintuple_it() + sextuple_it() + seven_tuple_it()
    + eight();
  // same as
  // return 117;
  // 0 + (3 * 3) + (5 * 4)
  // + (3 * 5) + (5 * 6) + (5 * 7)
  // + 8
}

We go over the purpose of the design of this and the reasons for that design here.

4.2.1. Capture Functions are Complete Objects

The most important change from typical GNU Nested Functions and mirroring behavior from C++ Lambdas is that nested functions -- the identifier itself introduced by the definition of the function -- is a regular, normal, complete C object. This enables it to be:

• returned, if the type is knowable at the time of function definition (or auto return types are incorporated into the language);

• passed to a function, if the function is defined after the creation of the capture function;

• and, stored elsewhere through static/_Thread_local data with assignment or memcpy, or even on the heap.

These are important qualities to allow these functions with data to be used with asynchronous code, as (stored) callbacks, and in other scenarios. The size and alignment of the object is implementation-defined, and its layout is also entirely implementation-defined, much like the properties of a regular struct or union type. This allows implementations to not have to figure out how to squash everything into a single erased type, and instead enforce the Single Responsibility Principle; they already know how to create unique types, they already know how to create and fill structure types, and now separately a wide function pointer type or a "make trampoline" compiler feature (§ 6.3 Make Trampoline and Singular Function Pointers) can be developed.

Given an extremely simple example:

#include <stdlib.h>
#include <stdio.h>

typedef void work_fn_t(void* user);
void add_work(work_fn_t* work, void* user);
bool work_done();

void kickoff(int start, int limit) {
  void work() _Capture(start, limit) {
    printf("doing work for %d to %d\n", start, limit);
    for (int i = start; i < limit; ++i) {
      printf("sooo much work - %d\n", i);
    }
  }
  void work_trampoline(void* user) {
    (*((typeof(work)*)user))()
    // free lambda after work is done
    free(user);
  };
  // elevate to higher lifetime to survive async function call time
  void* work_ptr = malloc(sizeof(work));
  memcpy(work_ptr, &work, sizeof(work));
  add_work(work_trampoline, work_ptr);
}

int main (int argc, char* argv[]) {
  int start = 0;
  int limit = 30;
  if (argc > 1)
    start = atoi(argv[1]);
  if (argc > 2)
    limit = atoi(argv[2]);

  kickoff(start, limit);

  while (!work_done());
  // no memory leaks at the end of the program
  return 0;
}

4.2.2. Deduced Return Types, Unique Types

Reusing an example from the above code, the make_seven function needs to have a special, inferred/deduced return type. This is because the type of a Capture Function is not known until it is defined:

auto make_seven (int x) {
  int y = 7;
  int seven_fn() _Capture(x, y) {
    return x * y;
  }
  return seven_fn; // OK: unique type which
  // is a complete object
}

The auto return type here just means "the first return expression is the return type of the function". This only works with in-line function definitions, and does not allow for a separated function declaration/definition (as the separated declaration would not have a material, real type until the definition could be read). This only applies to functions with inferred return types like this, where the first declaration of such a function must also be its definition.

If no return appears in such a function, or all the returns do not contain an expression, the return type is inferred to be void. Otherwise, all the return <expr>; must return the exact same type. If there exists one or more return <expr>;s and the types are not exactly the same in the whole function definition, then it is a hard error. This is already partly described in Jens Gustedt’s "Type inference for variable definitions and function returns v6" ([n2923]); reviving this paper would be a matter of rebasing it on the current working draft and improving the wording present.

4.2.3. Data Captures are Explicit

Data captures, the way in which local data is accessible inside of the function, are explicit. The only reason captures are explicit is because it is impossible to tell if something should be captured by value (and copied into whatever implementation-defined holding space is used for the Capture Function’s complete object), or if something should be captured by name/reference (and only have its pointer/address copied into whatever implementation-defined holding space is used for the Capture Function’s complete object). This detail matters both for safety reasons when assigning, copying, storing, and otherwise relocating a capture function from its original scope.

NOTE: static and _Thread_local objects, as well as typical file-scope declarations, are accessible within a capture function in the normal way. constexpr objects, without a static specifier, at local scope are also accessible.

Allowing for explicit captures also allows for better type checking (used objects must be explicit acknowledged by the programmer that they should be used), and allows for covering both the use cases of Apple Blocks (default by-value capture) and GNU Nested Functions (default by-name capture) without breaking anything. The lack of a capture also covers all of the use cases that Literal Functions would have covered, which means that Capture Functions can sufficiently cover all of the existing use cases currently in production in C ecosystems. To match the default behaviors:

• Apple Blocks: _Capture(=) (capture all by-value).

• GNU Nested Functions: _Capture(&) (capture all by-name/reference).

• Literal Functions: _Capture() (capture nothing).

Only one "capture all" is allowed. That is, _Capture(=, &) (and vice-versa) is illegal. The rest of the specific captures for accessible identifiers can be specified in any order. Note that specific captures for a given object override the default implicit "capture all" behavior. For example:

int main () {
  int x = 30;
  int y = 10;
  int fn () _Capture(&, x) {
    return x + y;
  }
  x = 50;
  y = 40;
  return fn();
}

This program returns 70 (x is captured by-value as 30, y is captured by-name and is changed to 40 before invocation). The change to x on the outside to 50 is not reflected inside of the invocation. This allows an ease-of-use for specifying the "default" implicit all-capture, while letting the user select specifically which captures should work.

4.2.4. Data Captures can be Renamed

Data captures can be renamed (or computed, with an expression that does not include a , unless it is parenthesized). This is important for e.g. incrementing reference counters for copying large, important data structures into callbacks that may either be invoked multiple times or have their own long-lived lifetime. The syntax for this occurs within the _Capture clause of a Capture Function:

#include <tree.h>

TREE_DECLARE(int_tree_t, int_tree, int);
TREE_IMPLEMENT(int_tree_t, int_tree, int);

#include <stdcountof.h>

enum queue_status {
  qs_success,
  qs_timedout,
  qs_busy,
  qs_fail,
  qs_invalid
};

typedef int work_fn_t(void* user);

queue_status add_dispatch_work(work_fn_t* work, void* user);
queue_status is_work_done();
void work_shutdown();

int main () {
  int data[] = { 2, 11, 32, 49, 57, 20, 110, 203 };
  int_tree_t tree = int_tree_init_with(data, data + countof(data));
  int work () _Capture(my_tree = int_tree_copy(tree)) {
    /* do work.... */
    int elem = int_tree_remove(my_tree, int_tree_min_node(my_tree));
    /* blah blah blah */
    return 0;
  }
  int work_trampoline (void* user) _Capture() {
    return (*((typeof(work)*)user))();
  }
  if (add_dispatch_work(work_trampoline, &work) != qs_success) {
    return 1;
  }
  queue_status err;
  while ((err = work_done()) != qs_success) {
    swith () {
      case qs_invalid:
      case qs_timedout:
      case qs_failed:
        // some error happened
        work_shutdown();
        return 2;
      default:
        break;
    }
  }
  work_shutdown();
  return 0;
}

4.2.5. NEW: Data Captures are Accessible

An important adjustment to make sure this code works better than the way it did for Blocks or Nested Functions is the ability not only to copy (§ 4.2.1 Capture Functions are Complete Objects) or otherwise rename objects (§ 4.2.4 Data Captures can be Renamed), but ALSO to get at the internals of a given Capture Function. This is something missing from GNU Nested Functions (which provides no real resolution for it) as well, and something that could matter for Apple Blocks but does not in practice because they can turn any object into a shared one with the __block modifier on an object. In particular, this only matters in the case of a closure which is given a (copied) resource that must either be released or freed.

NOTE: Thanks to Alex Celeste, for being the first person to bring this to my attention!

The syntax looks just like normal structure access, and is based on the names placed in the _Capture clause:

#include <stdio.h>

int main () {
  int x = 30;
  double y = 5.0;
  char z = 'a';

  int cap_fn0 () _Capture(=, &renamed_x = x) {
    printf("inside cap_fn0 | renamed_x: %d, y: %f, z: %c\n",
      renamed_x, y, z);
  }
	
  int cap_fn1 () _Capture(&, renamed_y = y) {
    printf("inside cap_fn1 | x: %d, renamed_y: %f, z: %c\n",
      x, renamed_y, z);
  }
	
  x = 60;
  y = 10.0;
  z = 'z';

  cap_fn0();
  cap_fn1();
	
  printf("\n");

  printf("inside main fn | cap_fn0.renamed_x: %d, cap_fn0.y: %f, cap_fn0.z: %c\n",
    cap_fn0.renamed_x, cap_fn0.y, cap_fn0.z);
  printf("inside main fn | cap_fn1.x: %d, cap_fn1.renamed_y: %f, cap_fn1.z: %c\n",
    cap_fn1.x, cap_fn1.renamed_y, cap_fn1.z);

  return 0;
}

This would print:

inside cap_fn0 | renamed_x: 60, y: 5.0, z: a
inside cap_fn1 | x: 60, renamed_y: 10.0, z: z

inside main fn | cap_fn0.renamed_x: 60, cap_fn0.y: 5.0, cap_fn0.z: a
inside main fn | cap_fn1.x: 60, cap_fn1.renamed_y: 10.0, cap_fn1.z: z

How the implementation actually accesses the information is implementation-defined, and the layout of the Capture Function object is not defined the specification, except to say it’s implementation-defined (§ 5 Wording).

NOTE: This leaves room for an implementation to, for example, use creative ways to retrieve objects and object references. Using a pointer to the current stack frame and then computing a raw offset to get to a specific bit of data, or using entirely registers, are all possible depending on how the captures are implemented. Such improvements and optimizations -- especially in the face of potential asynchronous calls and the need to protect against false sharing -- must be left up to Quality of Implementation.

As an example for releasing resources outside of the function call itself for the purposes of a function call that gets used more than once and isn’t passed a "We’re Done" signal, we can reuse the example from § 3.2.5 GNU Nested Functions By-Name Captures Cannot Be Worked Around Normally:

#include <stdlib.h>
#include <string.h>
#include <stddef.h>

typedef int compare_fn_t(const void* left, const void* right);

auto make_compare(int argc, char* argv[]) {
  /* LOCAL, heap-allocated variable.... */
  int* in_reverse = malloc(sizeof(int));
  *in_reverse = 0;

  if (argc > 1) {
    char* r_loc = strchr(argv[1], 'r');
    if (r_loc != NULL) {
      ptrdiff_t r_from_start = (r_loc - argv[1]);
      if (r_from_start == 1 && argv[1][0] == '-' && strlen(r_loc) == 1) {
        *in_reverse = 1;
      } 
    }
  }
	
  int compare(const void* untyped_left, const void* untyped_right) _Capture(in_reverse) {
    const int* left = untyped_left;
    const int* right = untyped_right;
    return (*in_reverse) ? *right - *left : *left - *right;
  }
  return compare;
}

int main(int argc, char* argv[]) {
  int list[] = { 2, 11, 32, 49, 57, 20, 110, 203 };  

  auto compare = make_compare(argc, argv);
  qsort_r(list, (sizeof(list)/sizeof(*list)), sizeof(*list), compare, &compare);
  // with data field captures, we can now `free` the
  // field `in_reverse` from the lambda
  free(compare.in_reverse);
	
  return list[0];
}

Thanks to the capture of in_reverse with the by-value _Capture(in_reverse) indication, the return of this function is safe. And, since we have access to the unique type that is generated (through the auto return type), we can access the pointer in_reverse normally and naturally. This isn’t possible with normal C++-style lambdas, as they haven’t decided to make this available (though our design for Lambdas in C will also include the named captures as accessible fields). It’s also not possible in the other solutions which rely on type-erasure as a first-class part of the design (Apple Blocks with the Blocks type, GNU Nested Functions only being accessible through a pointer or convertible to a wide function pointer in [n2661] or [n3564], Borland’s closure annotation or function literals). This is why making it possible to access the unique type first and foremost is of great benefit.

4.2.6. Capable of Recursion

Capture functions are able to refer to themselves for the purpose of recursion. This means that __self_func ([__self_func]), unlike for expression-based/unnamed Literal Functions/Lambdas/Block literals, is not required:

int main () {
  int tripling (int times, int start) {
    if (times >= 5) {
      return start;
    }
    return tripling(times + 1, start * 3); // normal recursion
  }
  return tripling(0, 1);
}

4.2.7. Not An Expression

The one true technical downside is that Capture Functions are declarations / definitions. They cannot be used (without the Statement Expression extension) in a function call’s argument list, which means that (short) closures and anonymous functions still need the full function definition. This is annoying and, honestly, one of the reasons § 4.3 Lambdas are preferred as a shorthand syntax.

It also means that, without Statement Expressions, Capture Functions cannot be used for the implementation of many macros which are typically expected to be usable as normal expressions.

4.2.8. Footgun: By-Name Capture Exceeds Captures’s Lifetime

A brief display of the undefined behavior:

auto ub (int parameter) {
  int automatic = 7;
  int fn() _Capture(parameter, &automatic) {
    return parameter + automatic;
  }
  return fn; // well-defined copy return
  // but dangling reference to `automatic`!
}

int main () {
  auto fn = ub(2);
  return fn(); // undefined behavior:
  // `automatic` no longer exists.
}