x86 32-bit (i386) machine code function, 13 bytes
Calling convention: i386 System V (stack args), with a NULL pointer as a sentinel / terminator for the end-of-arg-list. (Clobbers EDI, otherwise complies with SysV).
C (and asm) don't pass type info to variadic functions, so the OP's description of passing integers or arrays with no explicit type info could only be implemented in a convention that passed some kind of struct / class object (or pointers to such), not bare integers on the stack. So I decided to assume that all the args were non-NULL pointers, and the caller passes a NULL terminator.
A NULL-terminated pointer list of args is actually used in C for functions like POSIX execl(3): int execl(const char *path, const char *arg, ... /* (char *) NULL */);
C doesn't allow int foo(...); prototypes with no fixed arg, but int foo(); means the same thing: args unspecified. (Unlike in C++ where it means int foo(void)). In any case, this is an asm answer. Coaxing a C compiler to call this function directly is interesting but not required.
nasm -felf32 -l/dev/stdout arg-count.asm with some comment lines removed.
24 global argcount_pointer_loop
25 argcount_pointer_loop:
26 .entry:
28 00000000 31C0 xor eax, eax ; search pattern = NULL
29 00000002 99 cdq ; counter = 0
30 00000003 89E7 mov edi, esp
31 ; scasd ; edi+=4; skip retaddr
32 .scan_args:
33 00000005 42 inc edx
34 00000006 AF scasd ; cmp eax,[edi] / edi+=4
35 00000007 75FC jne .scan_args
36 ; dec edx ; correct for overshoot: don't count terminator
37 ; xchg eax,edx
38 00000009 8D42FE lea eax, [edx-2] ; terminator + ret addr
40 0000000C C3 ret
size = 0D db $ - .entry
The question shows that the function must be able to return 0, and I decided to follow that requirement by not including the terminating NULL pointer in the arg count. This does cost 1 byte, though. (For the 12-byte version, remove the LEA and uncomment the scasd outside the loop and the xchg, but not the dec edx. I used LEA because it costs the same as those other three instructions put together, but is more efficient, so the function is fewer uops.)
C caller for testing:
Built with:
nasm -felf32 -l /dev/stdout arg-count.asm | cut -b -28,$((28+12))- &&
gcc -Wall -O3 -g -std=gnu11 -m32 -fcall-used-edi arg-count.c arg-count.o -o ac &&
./ac
-fcall-used-edi is required even at -O0 to tell gcc to assume that functions clobber edi without saving/restoring it, because I used so many calls in one C statement (the printf call) that even -O0 was using EDI. It appears to be safe for gcc's main to clobber EDI from its own caller (in CRT code), on Linux with glibc, but otherwise it's totally bogus to mix/match code compiled with different -fcall-used-reg. There's no __attribute__ version of it to let us declare the asm functions with custom calling conventions different from the usual.
#include <stdio.h>
int argcount_rep_scas(); // not (...): ISO C requires at least one fixed arg
int argcount_pointer_loop(); // if you declare args at all
int argcount_loopne();
#define TEST(...) printf("count=%d = %d = %d (scasd/jne) | (rep scas) | (scas/loopne)\n", \
argcount_pointer_loop(__VA_ARGS__), argcount_rep_scas(__VA_ARGS__), \
argcount_loopne(__VA_ARGS__))
int main(void) {
TEST("abc", 0);
TEST(1, 1, 1, 1, 1, 1, 1, 0);
TEST(0);
}
Two other versions also came in at 13 bytes: this one based on loopne returns a value that's too high by 1.
45 global argcount_loopne
46 argcount_loopne:
47 .entry:
49 00000010 31C0 xor eax, eax ; search pattern = NULL
50 00000012 31C9 xor ecx, ecx ; counter = 0
51 00000014 89E7 mov edi, esp
52 00000016 AF scasd ; edi+=4; skip retaddr
53 .scan_args:
54 00000017 AF scasd
55 00000018 E0FD loopne .scan_args
56 0000001A 29C8 sub eax, ecx
58 0000001C C3 ret
size = 0D = 13 bytes db $ - .entry
This version uses rep scasd instead of a loop, but takes the arg count modulo 256. (Or capped at 256 if the upper bytes of ecx are 0 on entry!)
63 ; return int8_t maybe?
64 global argcount_rep_scas
65 argcount_rep_scas:
66 .entry:
67 00000020 31C0 xor eax, eax
68 ; lea ecx, [eax-1]
69 00000022 B1FF mov cl, -1
70 00000024 89E7 mov edi, esp
71 ; scasd ; skip retaddr
72 00000026 F2AF repne scasd ; ecx = -len - 2 (including retaddr)
73 00000028 B0FD mov al, -3
74 0000002A 28C8 sub al, cl ; eax = -3 +len + 2
75 ; dec eax
76 ; dec eax
77 0000002C C3 ret
size = 0D = 13 bytes db $ - .entry
Amusingly, yet another version based on inc eax / pop edx / test edx,edx / jnz came in at 13 bytes. It's a callee-pops convention, which is never used by C implementations for variadic functions. (I popped the ret addr into ecx, and jmp ecx instead of ret. (Or push/ret to not break the return-address predictor stack).
