Calling Conventions

Calling conventions is very important concept of C++ programming. Calling conventions decides two things, how arguments will be passed on stack and who will clear the stack.

In this article I will describe (and will prove it) two calling conventions, __cdecl and __stdcall.

In case of both __cdecl and __stdcall, arguments will be pushed on stack from right to left. Difference is, in __cdecl caller clears the stack, while in __stdcall callee will clear stack.

Let’s consider following code,

int __stdcall StdCall(int a,int b)

{

      return a + b;

}

int __cdecl Cdecl(int a,int b)

{

      return a + b;

}

int main(int argc, char* argv[])

{

      StdCall(10,20);

      Cdecl(10,20);

      return 0;

}

In this code, StdCall and Cdecl take two ints and returns there addition. I kept code simple to illustrate concept.

Now we will see code’s assembly equivalent to understand how calling conventions do the trick.  I am using Visual Studio to get assembly code. Here I am showing assembly code in parts.

Call to StdCall

push  20                            ; 00000014H

push  10                            ; 0000000aH

call  ?StdCall@@YGHHH@Z             ; StdCall

 Definition of StdCall

?StdCall@@YGHHH@Z PROC                                 ; StdCall, COMDAT

       push   ebp

       mov    ebp, esp

       sub    esp, 192                          ; 000000c0H

       push   ebx

       push   esi

       push   edi

       lea    edi, DWORD PTR [ebp-192]

       mov    ecx, 48                                  ; 00000030H

       mov    eax, -858993460                          ; ccccccccH

       rep stosd

       mov    eax, DWORD PTR _a$[ebp]

       add    eax, DWORD PTR _b$[ebp]

       pop    edi

       pop    esi

       pop    ebx

       mov    esp, ebp

       pop    ebp

       ret    8

?StdCall@@YGHHH@Z ENDP                                 ; StdCall

_TEXT  ENDS

END

In call to StdCall, it pushes arguments from right to left, i.e. 20, 10. And look at definition of StdCall, you can see ret 8. This is return statement, and don’t get confused with return statement of C++, here return statement doesn’t return value, but pass control from where this function getting called. ret 8 means it will return control by clearing stack with 8 bytes (adds 8 bytes to stack pointer, esp ). In 32 bit OS int are of 4 bytes, we are passing two ints so that will make it 8 bytes. So callee has cleaned the stack.

Now look at Cdecl calling convention.

Call to Cdecl

push  20                            ; 00000014H

push  10                            ; 0000000aH

call  ?Cdecl@@YAHHH@Z                     ; Cdecl

add   esp, 8

Definition of Cdecl

?Cdecl@@YAHHH@Z PROC                            ; Cdecl, COMDAT

      push  ebp

      mov   ebp, esp

      sub   esp, 192                      ; 000000c0H

      push  ebx

      push  esi

      push  edi

      lea   edi, DWORD PTR [ebp-192]

      mov   ecx, 48                             ; 00000030H

      mov   eax, -858993460                     ; ccccccccH

      rep stosd

      mov   eax, DWORD PTR _a$[ebp]

      add   eax, DWORD PTR _b$[ebp]

      pop   edi

      pop   esi

      pop   ebx

      mov   esp, ebp

      pop   ebp

      ret   0

?Cdecl@@YAHHH@Z ENDP                            ; Cdecl

_TEXT ENDS

Like StdCall, here in Cdecl both the arguments are pushed from right to left, and then call to function is made. However after call statement, you can see              add  esp, 8, which adds 8 to esp. esp points to top of stack. Adding 8 to means it is clearing 8 bytes (because stack grows from higher to lower memory). So here caller is clearing stack. Contrary to StdCall here you can see, ret 0, which just returns control and do not anything to stack pointer. So, in this case caller clears the stack.

Please add comments for your queries and suggestions.