First, let’s create definitions for the different kind of “time” involved in profiling:
- Wall clock time. The real-world time that passes while a computer executes a certain task. Like using a stop watch to measure that task. This time can be affected by anything that a multi-tasking system does in parallel to running the task under profiling. For single-core CPUs, it is the sum of the task’s CPU time (see below), CPU time of parallel processes, programmed delays and the
communication channel delay (the delay for network transmission). For programs in parallel execution on multi-core CPUs, CPU time is the sum of the CPU times on each core, while wall clock time is lower than this sum. - CPU time. The sum of user CPU time and system CPU time. It is proportional to the CPU cycles used for running the profiled task and anything it requested. Hence, it does not include programmed delays, or waiting for resources (like data from I/O devices, or being scheduled to get the CPU).
- User CPU time. The amount of time that a CPU (or CPUs) were busy running a user process (so excluding system calls to the kernel like for I/O).
- System CPU time. The amount of time that a CPU (or CPUs) were busy running system calls (kernel code) that was called from the user code under profiling.
Sources used: question 48455 on serverfault.com, Wikipedia on wall clock time.
Now, in the TPTP framework, “clock time” means wall clock time and “CPU time” means CPU time. However, the CPU time counter is inaccurate and not granular enough: it does not include methods that run less than 1ms, due to JVM limitations. That accounts for the fact that CPU time can be both higher and lower than clock time on single-core machines.
Source for last paragraph: message 5503 on news.eclipse.tptp.
For profiling this means to better not use the CPU time counter, keep the computer “calm” (resp. in comparable conditions) when running the tests to get comparable wall clock time values, and perform multiple iterations of test actions to get more representatiove wall-clock time values by averaging.
Leave a Reply