I tried to use the norecursedirs option inside setup.cfg to tell py.test not to collect tests from certain directories but it seems it does ignore it.
[tool:pytest]
norecursedirs=lib/third
When I run py.test I do see how it does get tests from inside lib/third!