GeezerVison

This experiment compares different ways of sampling the time taken to run an SUT:

• phpbench: Current PHPBench "remote" executor: which samples the time taken to repeat the SUT revs times.
• sample_inside_loop: Samples the time taken to execute the benchmark only (i.e. sample in the loop)
  Sample (for each executor):
• hrtime: Same as above but using hrtime

Conclusions

• hrtime seems to remove necessity of sampling outside the loop.

Env

(note that HR revs are exagerated as I multiplied the revolutions by 1000 to scale it down to microseconds)

Method

For each executor

./bin/phpbench run \
    tests/Benchmark/Executor \
    --revs=1 \
    --revs=10 \
    --revs=100 \
    --iterations=10 \
    --warmup=1  \
    --executor=$excutor \
    --tag=sample_inside_loop

Report:

./bin/phpbench report --ref=phpbench --output=html --output=console --ref=sample_inside_loop --ref=hrtime --report=executor_comp

Actions

• Introduce a hrtime executor which will sample inside or outside the loop (configurable).
• Make this the default in 2.0