Reading notes: optimizing Linux performance

Chapter 3

1. vmstat can show swap in/out pages. A simultaneously high swap-in and swap-out rate could indicate that the system does not have enough memory to handle all the running processes.
2. slabtop is "top" for slab.

Chapter 4

1. strace can show system call performance statistics too (strace -c).
2. ltrace: performance statistics for library (like strace, but library function instead of system call).
3. ld.so: dynamic load/link statistics for libraries. Example: env LD_DEBUG=statistics LD_DEBUG_OUTPUT=lddebug gcalctool

Chapter 6

1. iostate: show disk IO statistics

Chapter 7

1. netstat -c: period update
2. netstat -t: choose tcp protocol
3. netstat -l, -a: listening sockets or listening + conntected.

Chapter 8

1. script: record what you have done and the output, seems useful.
2. gcc -g3: provide more debug information then gcc -g (default to
   • g2), such as the macro definitions present in the source.

Chapter 11

1. Use gdb to check call chain:
   • Set a breakpoint at check-point
   • gdb can execute a given set of commands when it hits a breakpoint. By using the command command, we can tell gdb to execute bt; cont every time it hits the breakpoint.
   • Conditional breakpoint can be implemented via gdb script (commands) too. For example, we can enable breakpoint 2, when we reach breakpoint 1.
   • gdb output can be logged to a text file via: set logging on
2. oprofile reveals which function is hot, but do not show how many times it is called. ltrace or some other traces shows which function (including subfunctions) consumes most CPU time. This is from two perspective, and they can be combined to inspect the performance property.
3. For Fedora and Enterprise Linux, Red Hat provides a set of debuginfo rpms that contain all the symbol information and sources that were generated by the compiler when the application was complied. Each binary package or library has a corresponding debuginfo rpm that contains the debugging information. This allows Red Hat to ship the binaries without the disk-space–consuming debugging information. However, it allows developers, or those investigating performance problems, to download the appropriate debuginfo packages and use them. In this case, Red Hat's version of oprofile will also recognize the debuginfo packages and pick up the symbols when profiling both an application, such a nautilus, and a library, such as gtk.
   • Debian has similar binary debug symbol packages: -dbg

Chapter 12

1. strace to find bottleneck of IO intensive application.
   • Oprofile can be used only for CPU intensive application. Because IO is performed in kernel space, strace is the right tool to reveal which kind of IO application is performing.

Chapter 13

1. This call-tree tool would be useful even if it dramatically slowed down application performance as it runs. A common way of using this would be to run oprofile to figure out which functions in an application are "hot," and then run the call-tree program to figure out why the application called them.