Using Nested Functions on GCC Without Executable Stack Support

This article discusses the use of nested functions in GCC 17 and Clang for callbacks without requiring an executable stack. It explores alternatives for supporting older versions of GCC, including the use of a hack to avoid an executable stack. The article explains how GCC allows taking the address of a nested function and provides a toy example demonstrating this functionality. The code example shows how a trampoline is placed on the stack and invoked via an inlined function. The article details how to extract the code address and static chain from the trampoline to call the local function directly using the __builtin_call_with_static_chain built-in function. It also mentions that this approach can serve as a fallback mechanism in older GCC versions, despite the downsides of creating a trampoline and marking the stack as executable. The author notes that the idea is implemented in an experimental library called noplate. Additionally, the article suggests an alternative approach of using the trampoline as a function descriptor, where the address of the trampoline is passed on and checked before calling, allowing for direct calls to the nested function.