Profile your code

What
The process of benchmarking the runtime performance of your code.
Why
This helps to find slow areas of your code and gives you a visual way to determine what is taking a lot of time or resources to run.
When
Long term
Implementations:
  • py-spy
  • cProfile
  • flameprof
  • flamegraph
  • perftester
  • timeit
  • lin_profiler
Recommendation: Any

Profiling helps you identify which parts of your code consume the most time or resources. In scientific software, optimizing these hotspots can dramatically improve overall performance.

Profiling tools provide insights like:

  • Which functions take the most time
  • How many times each function is called
  • The call hierarchies that consume the most resources

For Python scientific code, statistical profilers like py-spy are particularly useful as they can profile code with minimal overhead. They sample your program’s state at regular intervals, giving you a representative view of where time is spent without significantly slowing execution.

When profiling scientific code, pay special attention to:

  • Numeric computation loops
  • File I/O operations
  • Data preprocessing steps
  • Visualization generation