Why should you care?

Why: Benchmarking or scientific testing, preferably automated whenever possible, ensure that the methods maintain their scientific performance and accuracy when the codebase changes.

Different benchmarks can have different goals

  • Ensure that software improves over time, for example, that speed of calculation for the same task improves, or that scientific accuracy improves, etc.
  • Ensure that software performance doesn’t degrade over time, for example, due to hardware or software updates, bitrot etc.