Which C specifiers may cause ABI incompatibility between the caller and callee?

by

Suppose there is a file with a declaration of a function which differs from the actual definition in another file i.e.

FileA.c

short foo();

int main(){
   foo();
}

FileB.c

int foo(){
   //Some code
}

This is obviously wrong code but it illustrates what the topic is.

The problem is that a compiler will not produce a machine code that will reserve enough space for the return value of foo when it’s called. I think this can be called ABI incompatibility.

There are multiple different specifiers one may apply to a function or variable i.e. type qualifiers, storage class specifiers, atomics etc. I wonder which of these could potentially cause any kind of incompatibilities between a caller and a callee. Consider the case that these files might be compiled to separate binaries but still call the function via a pointer to an address.

  • 2

    The compiler isn’t reserving any space for the return value because the caller ignores it. Anyway, the short or int value returned will typically be in a register, not in memory. But if the linker fails to notice the conflict and produces an executable then you have undefined behaviour. This is why the prototype for foo should be in a header file FileB.h.

    – 




  • Yeah the code you posted isn’t “obviously wrong”. It will be wrong on some architectures but on x86_64, the return value goes in a register and the mismatch doesn’t matter. You can trivially see this using Compiler Explorer: godbolt.org/z/3zGTvrEcW In the end, the answer to your question is just “it depends on your architecture and calling convention” and you have to really know what you’re doing if you want the signatures to be different. It’s probably safe on most systems to replace a pointer type with another pointer type, and integers with integers of the same size.

    – 




  • 4

    You probably need to study the C standard on the topic of §6.2.7 Compatible type and composite type. It tells you what the standard guarantees (or requires). The best way to deal with declarations of extern declarations between translation units (TU) is to use the same header in both TUs. This is what allows you to use, for example, the functions from <stdio.h> in multiple TUs. The alternatives are tricky to handle reliably. Declarations don’t cost anything at runtime — even in memory-constrained systems, include the correct information.

    – 




  • 1

    @patvax is does matter, because the compiler can pick up the error.

    – 

  • 1

    @JonathanLeffler wrote that the should both use the same header. Are you trying to link to a binary that you don’t have an interface for (ie guessing)?

    – 




I wonder which of these could potentially cause any kind of incompatibilities between a caller and a callee”

Simply assume any difference risks incompatibility.

Some difference are OK, yet those are often implementation defined. There is little value in exploiting a difference that is compatibility.

See also direct qualifiers.

The best way to deal with declarations of extern declarations between translation units (TU) is to use the same header in both TUs.

Leave a Comment