Thursday, May 2, 2013

Parallel static python checks with Makefiles

Parallel Ripples

To make testing fun I looked at improving the speed of static syntax checks.
The latest incarnation is now using make targets to represent a specific checks. The nice thing about this is, that you can use Make's -j switch to run those tests in parallel.

So basically it looks like this:

check-static-pep8: $(PYTHONSOURCES:%=%.pep8)
 @echo Passed $@

%.pep8:
 PYTHONPATH=. pep8 -r "$*"

The complete makefile is here and the numbers:

Simple:
...
---
 Passed check-local
---

real 0m22.875s
user 0m20.466s
sys 0m2.158s


And in parallel:
---
 Passed check-local
---

real 0m11.459s
user 0m34.780s
sys 0m3.325s

There is so much that can be done.

No comments:

Post a Comment