1

I am interested in using doit to automate the build process of a python package. If possible, I would like doit to re-execute a task if any of the user-created source files it depends on have changed. From my understanding, the best way to accomplish this would be to use the file_dep key and a list of the dependent source files, however I am having a lot of trouble generating this list.

I've tried using sys.modules and inspect.getmembers(), but these solutions can't deal with import statements that do not import a module, such as from x import Y, which is unfortunately a common occurrence in the package I am developing.

Another route I investigated was to use the snakefood tool, which initially looks like it would do exactly what I wanted, generate a list of file dependencies for every file in a given path. Unfortunately, this tool seems to have limited Python 3 support, making it useless for my package.

Does anyone have any insight into how to get snakefood-like features in Python 3, or is the only option to change all of my source code to only import modules?

Roy Smart
  • 664
  • 4
  • 12
  • You can hook `import` to learn about every import, even of modules already imported elsewhere. But this works only as well as your “test” coverage, since imports can happen (anywhere) inside a function. – Davis Herring May 09 '19 at 03:08

1 Answers1

0

doit tutorial itself is about creating a graph of python module imports!

It uses import_deps package, it is similar to snakefood.

Note that for your use-case you will need to modify file_dep itself during Task action's execution. To achieve that you need to pass the task parameter to your action (as described here).

schettino72
  • 2,990
  • 1
  • 28
  • 27
  • of course! Thank you for pointing out `import_deps`. It seems that `import_deps` is almost exactly what is needed, except I need the ability to apply it recursively to find all dependent python files. Do you know if `import_deps` would be straightforward to modify or extend to work recursively? – Roy Smart May 21 '19 at 17:05
  • I have implemented it recursively and integrated with `doit` in [pytest-incremental](https://github.com/pytest-dev/pytest-incremental). Although I am afraid it is not *straightforward*... – schettino72 May 22 '19 at 03:51