61

I am solving a binary exploitation challenge on picoCTF and came across this piece of code:

((void (*)())buf)();

where buf is a character array.

I solved the challenge but can't seem to understand what exactly it's doing. I looked at this thread but I couldn't make it out.

What does ((void (*)())buf)(); mean?

S.S. Anne
  • 15,171
  • 8
  • 38
  • 76
sh.3.ll
  • 815
  • 7
  • 17
  • 14
    *What does `((void (*)())buf)();` mean?* It means the author doesn't understand `typedef`. `typedef void (*voidFuncPtrType)();` would make this code clear. – Andrew Henle Jan 14 '20 at 13:39
  • 33
    @AndrewHenle in designing CTF challenges, clarity isn't really the top goal, and some obfuscation can even be expected as part of the challenge. More likely than not, the author was aware that this is not the most readable way of doing things. – ManfP Jan 14 '20 at 23:19
  • 2
    It means your program has UB. – R.. GitHub STOP HELPING ICE Jan 15 '20 at 20:10
  • 4
    It means C's "spiral" type declaration rule is way too complicated. There's a reason virtually every other statically-typed language that isn't directly descended from C uses left-to-right rules instead. – Mason Wheeler Jan 15 '20 at 22:39
  • 2
    @MasonWheeler "Spiral" is an urban myth. The declaration is as much or as little "spiral" as the corresponding expression would be. Operators are simply applied in precedence and left-to-right order (not telling you anything new here, of course): "I need to dereference it, then call it, and the result has type void": voila, pointer to void function. – Peter - Reinstate Monica Jan 16 '20 at 03:47
  • 1
    @Peter-ReinstateMonica "*The declaration is as much or as little "spiral" as the corresponding expression would be.*" Can you clarify what you mean? – user76284 Jan 16 '20 at 04:06
  • 1
    @user76284 Declarations mimic expressions: A declaration resembles an expression; the declared variable appears in the same place where it appears in an expression when it's used. Example: after the declaration `int *(*fp)()`, `fp` is used with `i = *(*fp)()`. Because a declaration is effectively, syntactically a prototypical use, resembling an expression, it shares the structure of expressions. This includes potentially being "spiral". (And I think that declarations or expressions are not always "spiral": consider `int ******x[1][1][1][1][1]`. Nothing spiral. First all right, then all left.) – Peter - Reinstate Monica Jan 16 '20 at 10:12

4 Answers4

136

void (*)() is a type, the type being "pointer to function that takes indeterminate arguments and returns no value".

(void (*)()) is a type-cast to the above type.

(void (*)())buf casts buf to the above type.

((void (*)())buf)() calls the function (passing no arguments).

In short: It tells the compiler to treat buf as a pointer to a function, and to call that function.

Some programmer dude
  • 400,186
  • 35
  • 402
  • 621
  • 15
    I find the `cdecl` utility (or [website](http://cdecl.org)) helpful for translating the more complex C expressions into English. – bta Jan 14 '20 at 23:49
  • 3
    @bta cdecl is not useful here as the syntax is not a declaration. It's a function call via a cast on a previously declared symbol – bolov Jan 15 '20 at 01:00
  • 3
    @bolov - On the entire statement, no, but it *does* explain the [most complex part](https://cdecl.org/?q=%28void+%28*%29%28%29%29buf) of it. From there, decoding the rest is fairly straightforward. – bta Jan 15 '20 at 01:11
  • I don't think that the _contents_ of the character array buf[ ] plays a role, only the _address_ of the "array" matters. I mean you cannot copy the character array buf[ ] to some real character array copy[ ] and expect that `((void (*)())copy)();` will call the function. It will crash. – AvD Jan 15 '20 at 02:04
  • 5
    @AvD If wherever `buf` or `copy` is located is at an executable address and the code itself is position-independent, this will work. It is of course as non-portable as it gets, but this should work in many bare-metal environments as well as older x86 OSes that don't set the no-execute (NX) bit on stack and heap. – wrtlprnft Jan 15 '20 at 04:39
  • 4
    @AvD: It won't necessarily crash. Unless the data area is protected against execution (which depends on the architecture and the run-time environment), you can use this trick to compile a function into an array at run-time and call it on the fly. I first used this trick 35 years ago on a DEC Vax to compile Turing machines for a failed experiment in Turing machine evolution. – TonyK Jan 15 '20 at 10:40
11

pointer buf is converted to the pointer to void function taking unspecified number of parameters and then dereferenced (ie function called).

0___________
  • 60,014
  • 4
  • 34
  • 74
9

It's a typecast, followed by a function call. Firstly, buf is cast to the pointer to a function that returns void. The last pair of parenthesis means that the function is then called.

haccks
  • 104,019
  • 25
  • 176
  • 264
lukeg
  • 4,189
  • 3
  • 19
  • 40
7

It casts the character array to a pointer to a function taking no arguments and returning void, and then calls it. Dereferencing the pointer is not required due to how function pointers work.

An explanation:

That "character array" is actually an array of machine code. When you cast the array to a void (*)() and call it, it runs the machine code inside of the array. If you provided the array's contents I could disassemble it for you and tell you what it's doing.

S.S. Anne
  • 15,171
  • 8
  • 38
  • 76