System calls are the primary mechanism of user-to-kernel interaction. Today the Linux system call interface has achieved a primacy and ubiquity that make it an ideal layer at which to understand single-system and distributed-system pathologies. Sysdig advances the art of system call observability by drawing on the systems that came before it. Informed by his work with /proc, process tools and DTrace, Adam will walk through a history of system calls and system call observability from simple systems like truss and strace, moderns ones like DTrace and SystemTab, and ancient ones from the early days of Unix.

1. Adam Leventhal, @ahl
A Brief History
of System Calls

2. Who Am I?

3. What’s a System Call?

4. USENIX Summer 1986

5. Time Spent in Syscalls Matters

6. Syscalls Tell You About The System

7. ULTRIX Trace
/dev/trace
…
If (tracing)
...
/* do the syscall */
If (tracing)
...
Syscall Handler
call exec(...)
return exec(...)
call open(...)
return open(...)
call read(...)
return read(...)
Buffer

8. Aside: How Great Is This?

9. Secure / Ubiquitous Tracing

10. Breakpoint Tracing
process
invoke syscall
execute syscall
proceed
scheduler
stop process
wake tracer
stop tracer
wake process
stop process
wake tracer
stop tracer
wake process
tracer
record arguments
resume process
record return value
signal process

11. Slow
# Slow the target command and print details for each syscall:
strace command
# Slow the target PID and print details for each syscall:
strace -p PID
# Slow the target PID and any newly created child process, printing syscall details:
strace -fp PID
# Slow the target PID and record syscalls, printing a summary:
strace -cp PID
# Slow the target PID and trace open() syscalls only:
strace -eopen -p PID
Brendan Gregg

12. Do No Harm
• Unsafe
• Chase away the
problem

13. DTrace

14. Syscalls with DTrace

15. Aside: Java
“Write once, run anywhere”

16. Java Java Java

17. Servers in1996

18. Write Once Run Anywhere + 20yrs

19. Primacy of Syscalls

20. Thank You!