python3.x : ModuleNotFoundError when import file from parent directory

Question

I am new to Python. This really confused me!

My directory structure is like this:

Project
   | - subpackage1
           |- a.py
   | - subpackage2
           |- b.py
   | - c.py  

When I import a.py into b.py with from subpackage1 import a, I get a ModuleNotFoundError. It seems like I cannot import a file from the parent directory.

Some solutions suggest to add an empty file __init__.py in each directory, but that does not work. As a work around, I have put the following in each subfile (i.e., a.py and b.py) to access the parent directory:

import os
import sys
sys.path.append(os.path.abspath('..'))  

I have tried to output sys.path in subfiles, it only includes the current file path and anaconda path, so I have to append .. to sys.path.

How can I solve this problem? Is there a more efficient way?

Answer 1

Suppose we have this files and directories tree:

$> tree
.
├── main.py
├── submodule1
│   ├── a.py
│   └── __init__.py
└── submodule2
    ├── b.py
    └── __init__.py

2 directories, 5 files

So, here is an example of how to do the import from a.py inti b.py and vice-versa.

a.py

try:
    # Works when we're at the top lovel and we call main.py
    from submodule1 import b
except ImportError:
    # If we're not in the top level
    # And we're trying to call the file directly
    import sys
    # add the submodules to $PATH
    # sys.path[0] is the current file's path
    sys.path.append(sys.path[0] + '/..')
    from submodule2 import b


def hello_from_a():
    print('hello from a')


if __name__ == '__main__':
    hello_from_a()
    b.hello_from_b()

b.py

try:
    from submodule1 import a
except ImportError:
    import sys
    sys.path.append(sys.path[0] + '/..')
    from submodule1 import a


def hello_from_b():
    print("hello from b")


if __name__ == '__main__':
    hello_from_b()
    a.hello_from_a()

And, main.py:

from submodule1 import a
from submodule2 import b


def main():
    print('hello from main')
    a.hello_from_a()
    b.hello_from_b()


if __name__ == '__main__':
    main()

Demo:

When we're in the top level and we're trying to call main.py

$> pwd
/home/user/modules
$> python3 main.py
hello from main
hello from a
hello from b

When w're in /modules/submodule1 level and we're trying to call a.py

$> pwd
/home/user/modules/submodule1
$> python3 a.py
hello from a
hello from b

When we're /modules/submodule2 level and we're trying to call b.py

$> pwd
/home/user/modules/submodule2
$> python3 b.py
hello from b
hello from a

Answer 2

The first issue you're getting is due to that from subpackage1 import a line in your b.py module.

b.py is located in your subpackage2 package, a sibling package of subpackage1. So trying to run from subpackage1 import a means the subpackage1 is within subpackage2 which is incorrect. Also as side note, in python3 you should never use implicit relative imports as it no longer supports it, so instead use explicit relative imports.

Adding an __init__.py in each folder turns them in python packages, and you can keep them empty. You want to replace the from subpackage1 import a with either an explicit relative import like from ..subpackage1 import a or an absolute import like from Project.subpackage1 import a. This will be the efficient and correct way to write your package, and you can test it by writing a script that imports Project and uses its subpackages and modules.

However, I am assuming you are running b.py as a main module to test the imports. This means you are running command lines that look like: python b.py. Running it like this gives you a module search path that does not have any of the parent paths like Project. This would lead to you keep getting ModuleNotFoundErrors even though there's nothing technically wrong with your package. This is why you need a sys.path.append(... work-around that manually appends your parent path to the module search path in order to run your b.py module as a main module. If this helps you to test your code then by all means use it, but it is fine to use the absolute and explicit relative imports because modules in a package are supposed to work that way.

Answer 3

Firstly, you need create file __init__.py in subpackage1 to declare it is a module

touch subpackage1/__init__.py

Secondly, you can try relative import in python3

# b.py
from ..subpackage1 import a

Or you can add your current directory to $PYTHONPATH

export PYTHONPATH=${PYTHONPATH}:${PWD}

Answer 4

One way to access the subpackage is using . operator going all the way up to top package or file directory. Thus if the structure is

top_directory
|- package1
   |- subpackage1
      |- a.py
|- package2
   |- subpackage2
      |- b.py

Then you use the following

#b.py
from top_directory.package1.subpackage1 import a

statements...

The from statement must go all the way up to the top directory covering both a.py and b.py.

Answer 5

This SO Q&A discusses options for adding your project root to the PYTHONPATH.

1. mac/linux:
add export PYTHONPATH="${PYTHONPATH}:/path/to/Project_top_directory" to your ~/.bashrc

2. path config file:
Alternatively, you can create a path configuration file

# find directory
SITEDIR=$(python -m site --user-site)

# create if it doesn't exist
mkdir -p "$SITEDIR"

# create new .pth file with our path
echo "$HOME/foo/bar" > "$SITEDIR/somelib.pth"