GeezerVison is showing github.com — functions inside functions (closures) · Issue #229 · ziglang/zig
functions inside functions (closures) #229DescriptionIt makes sense for there to be a function inside a function, as long as it doesn't require allocating any memory, and the function is only callable while the outer function is still running. Reacted by Jan Heijmans, Carwyn Edwards, Igor Ferreira, Adam Serafini, Pavel Atanasov, Emil Rasmus Merzin, windows-h8r, Samuel, Levi Roberts, Daniel Guzman and 12 more Activityphase commented on Jan 31, 2017 Would you generate this as a separate function in the IR or as a block? andrewrk commented on Feb 1, 2017 Good question. Probably a separate function - that seems easier to do. Do you have any thoughts on the matter? phase commented on Feb 1, 2017 If the functions can access variables in the parent function's scope, then it would need to be implemented as a separate block. Function call overhead would also be removed (if there is any). andrewrk commented on Feb 1, 2017 Consider that an inner function could call itself recursively.
If it were implemented as a function, then the compiler would create a struct for all the local variables in the outer function, and then pass that struct address to the inner function as a hidden parameter, and the inner function has access to the outer function's local variables.
If it were implemented as a block, the inner function's local variables would need to be a struct, and when you call the inner function, we allocate the stack space in the outer function's stack for the local variables of the inner function, and push the return address.
Both of these strategies would work fine, and I believe the function call overhead we would avoid by doing the block thing is exactly the same overhead we introduce by allocating space for the inner function's variables and pushing the return address.
I think the only reason to prefer the function strategy over the block strategy is that it might play more nicely with LLVM's optimizer. But that's just a hunch, and to be sure, an empirical test would be best. thejoshwolfe commented on Feb 7, 2017 SponsorContributor More actions I think it's important to put the word "closures" in the title of this issue, since this isn't just about functions with locally scoped names, but functions that have implicitly managed references to bound variables. Reacted by Joshua Baker, windows-h8r and The Potato Chronicler changed the title [-]functions inside functions[/-] [+]functions inside functions (closures)[/+] on Feb 7, 2017AndreaOrru commented on Apr 7, 2017 Proposal: convenient syntax for anonymous functions, or function literals. Useful and very common for callbacks and/or higher order functions.
const io = @import("std").io;
fn any(comptime T: type, array: []const T, pred: fn(T) -> bool) -> bool {
for (array) |elem| {
if (pred(elem)) return true;
}
return false;
}
pub fn main() -> %void {
const array = []u64 {4096, 256, 16, 1};
if (any(u64, array, x => x > 1000)) {
%%io.stdout.printf("True\n");
} else {
%%io.stdout.printf("False\n");
}
}
Returning functions should also be possible:
fn const(x: u64) -> (fn() -> u64) {
return () => x;
}
Lambda functions can also capture and refer variables from the scope of the outer function. While non-capturing lambda functions can be thought as simple function pointers to global procedures, capturing requires also a pointer to the outer environment. We can think of this couple { funcptr, envptr } as a different type, say bound fn, analogous to delegate in D 2.0.
However, unlike D 2.0, and like D 1.0, I vote against the possibility to return delegates from functions:
fn foo() -> (bound fn() -> &u64) {
var x: u64 = 10;
return () => &x;
}
Implementing this behavior safely seems extremely difficult (impossible?) and even provably safe cases require the continuation to be saved somewhere in the heap. Having a core language feature depending on an allocator sounds like a bad idea, and introducing a potential source of unsafety even more so.
I argue that by only allowing fn and not bound fn to be returned from functions, we'd get most of the advantages and none of the disadvantages. thejoshwolfe commented on Apr 12, 2017 SponsorContributor More actions There are several topics here to discuss:
- where can functions be declared? currently only at top-level-declaration scope, but we can add the syntax for functions declared somewhere somehow within the body of a function. at a minimum, this just means that you can cut+paste (and indent) a function from tld scope into the body of a function (or into any block) and the name would be scoped to that block. one question here is if forward declarations could be recognized, like javascript or Zig's tld, or if execution order defines the function, like python or vars in Zig's blocks.
- can you declare anonymous functions? at a minimum, this is a simple syntactic change from
fn foo() {} to const foo = <something>() {};. A change analogous to this happened to struct declarations in all containers anonymous #160. An anonymous function is called a lambda function. This could happen inside functions or at tld scope. This does not necessarily have anything to do with bound variables, which is called a closure.
- what implicit parameters get passed to a function when you call it? this is a really complicated issue. we've already got some implicit parameter passing for "self" parameters. if we're going to support closures (which means bound variables), then that's more implicit parameter behavior. if there is some type like
bound fn, what is it bound to? what is the actual data in the implementation? it's at least the function pointer, and then at a minimum a context pointer. if that's all, then the size is predictable, and APIs dealing with bound fn don't need comptime polymorphism (meaning different instantiations, like a C++ template). but if the context is just a pointer, where does it point?
- do context objects get managed on the heap? this is "the possibility to return delegates from functions" mentioned above. This is contrary to Zig's design principal of no hidden allocations, so the answer to this question is definitive. no.
- can a context pointer point to a stack frame, and should the compiler provide any help in assuring that the stack frame does not return while a context pointer is pointing to it? This is relevant to the "Returning functions should also be possible" example above. this would cater to the functional programming paradigm, the visitor design pattern, and other usecases.
- can a
bound fn be bound to more than just a context pointer? can it include a comptime variable amount of context? if the context is void, then it's effectively just an (unbound) fn. can we generalize all function pointers to have a comptime variable amount of context data bound to them?
Each one of the above bullets could be its own issue to discuss. Reacted by Kyle Hayes, Chris Heyes, Igor Ferreira, Marco Bolis, Adam Serafini and Pat Tullmann distractedlambda commented on Jul 13, 2019 Throwing in my thoughts (and apologies if I'm repeating anything already discussed):
- I think having a proper syntax for
comptime closures would be awesome, and on its own satisfy a lot of use cases. A comptime closure would be an anonymous function which implicitly captures all comptime things in scope by-value. Since all "captures" would be known at compile-time, a comptime closure would just be a function pointer.
- I think the best way to do closures with runtime captures, would be to have all captures be implicit but by value. Nothing would stop you capturing the address of something you knew would live long enough, but you'd have to do so by explicitly taking the address of the object and then capturing the resulting pointer by value.
- While being able to mutate a local
var in a closure is useful, I'm not sure the use cases outweigh the potential complications. The only way I think implicit by-reference capture would be reasonable, is if the compiler could guarantee that you would never use such a capture outside its lifetime, and that starts to require nontrivial escape analysis.
Edit
After looking through @tgschultz's earlier post, I guess comptime closures basically exist in Zig today, just without a nice syntax. I think a proper syntax is a no-brainer addition to the language, but that would probably have to coincide with function-definitions-as-expressions. fengb commented on Feb 25, 2020 adrusi commented on Jun 15, 2020 I also hacked together userland closures, although it looks like with quite a different goal from fengb.
https://gist.github.com/adrusi/c32e8133914fab5800d0eea7e3d6be15
tl;dr
test "closures" {
var x: i32 = 60;
const foo = closure(struct {
x: *i32,
pub fn call(self: *const @This(), y: i32) i32 {
self.x.* += y;
return 420;
}
} { .x = &x });
assert(foo.call(.{ @as(i32, 9) }) == 420);
assert(x == 69);
}Reacted by jadon, Shritesh Bhattarai, Crosstyan, Zack Kollar, Hanaasagi and Anton Erdman SpexGuy commented on Nov 3, 2020 After discussing with @andrewrk and @thejoshwolfe, we've decided that function literals (as accepted in #1717) should not be able to implicitly capture any state. It would be too easy to accidentally close over a pointer to stack memory and clobber your stack. It would also cause a problem where closures are often half-comptime (function pointer known) and half-runtime (runtime closed-over data), which the type system doesn't have a good way of handling.
Additionally, closures tend to encourage a highly functional programming style. While there are some merits to this style, it often obscures where memory is stored or what the associated lifetime is. The explicit capture syntax in C++ sidesteps this problem to some degree, but only works well because of C++'s use of constructors and destructors. A Zig equivalent would need to be more explicit, probably in ways that are dependent on the specific use case. It also forces the optimizer to choose between two terrible options: duplicate functions which accept closures (causing potentially large code bloat), or pass runtime function pointers (damaging cpu performance). Because of these drawbacks, we feel that the language should not take on additional complexity to support this style. However, note that #1717 alongside Zig's support for runtime function pointers and anonymous tuples make it possible to implement type safe abstractions that suffer less from these drawbacks.
For those reasons, we are closing this issue. The workaround for this use case is to use a stateless lambda alongside a state object, and pass the state object to the lambda when it is invoked. Reacted by Andrew Kelley, Isaac Freund, Felix Queißner, frmdstryr, Paul Smith, Mike Hogye, Mitchell Kember, Yusuke Kuoka, Ali Cheraghi, Alexander Drozdov and 13 more kyle-github commented on Nov 14, 2020 To revisit this a bit, I noted in issue #6965 that some parts of this (not general closures!) do solve the problem there.
The general idea:
Allow a limited form of nested function.
- Nested functions can access variables in their enclosing scope (TBD whether that is just the enclosing scope or all the way up to their enclosing function).
- This means three things that impact size and performance:
- Any time a nested function is passed, the called function may need to be split and regenerated such that it takes the nested function pointer and an additional pointer (or a fat pointer) to the enclosing activation record at the time of the call. In general this would mean that there could be two (and I think only two) versions of functions that are called with nested functions, and only if the function is called both with and without a nested function.
- Access of variables in the enclosing scope within a nested function will take a performance hit because they will need to reference through the passed activate record pointer.
- Access of variable in the enclosing scope will likely not be through registers (unless the compiler gets smarter about this).
- It will be illegal to explicitly take the address of a nested function (it is meaningless without the runtime activation record info).
- It will be illegal to assign a nested function in any way outside of its declaring scope other than passing it as a function argument.
- Interaction of nested functions and async functions is TBD. Splitting the async function may be a possibility but that seems complicated to manage.
These constraints are somewhat onerous, but GCC manages this just fine, so there is an existence proof. I have not done investigation with Godbolt or other to see what GCC does for optimization.
The key problem here is that without the restrictions on assignment etc. a nested function could escape to be used outside of its defining call tree. The prohibition on taking the address and on assignment aim to solve that. I have not thought through all the edge cases to see if it is still a problem with these restrictions. frmdstryr commented on Dec 6, 2020
closures tend to encourage a highly functional programming style. While there are some merits to this style, it often obscures where memory is stored or what the associated lifetime is.
I'm glad this was the decision because this would have been really bad news for anyone using zig on low cost/power embedded devices. Some projects completely disallow any heap allocations. Reacted by Yay295 and Crosstyan htqx commented on Apr 11, 2024 I need anonymous functions and don't want to be wrapped in a struct. Closures are very complicated and can be omitted. I support zig to keep it simple and close to the implementation logic of c. For those who really need closures, you can turn all the variables you need to capture into explicit parameters, so just anonymous functions are needed.
const f = struct {
fn f()void{}
}.f;
// i want
const f = fn()void{};Reacted by cheme, Hiroaki Yamamoto, Amar Al-Zubaidi, Mattia Maldini, Dimitri Sabadie, Julian, feelamee, mrnoname1000 and Elnu Reacted by Astrohippo
MetadataMetadataDevelopmentNo branches or pull requestsIssue actions
|