How to view/decrypt Ansible vault credentials files from within a Python script?

Question

I'm trying to figure out how to provide the following facilities to a Python script so that it can:

Import Ansible Python modules
Open up my defined ansible.cfg and read vault_password_file variable
Read vault_password_file and temporarily store in a Python variable
Decrypt a referenced Ansible vaulted file

I found this code via google but it did not appear to work when I tried it:

import ansible.utils

bar = dict()

bar = ansible.utils._load_vars_from_path("secrets.yml", results=bar, vault_password="password")

print bar

Throws this error:

$ python ansible-vault-ex.py
Traceback (most recent call last):
  File "ansible-vault-ex.py", line 5, in <module>
    bar = ansible.utils._load_vars_from_path("credentials.vault", results=bar, vault_password="password")
AttributeError: 'module' object has no attribute '_load_vars_from_path'

When I investigated this I saw no indications of this function in any Ansible related files, leading me to believe that this method no longer worked with some newer version(s) of Ansible.

Bottom line is that I'd like some method for importing Ansible libraries/modules from a Python script, so that I can interact with ansible-vault managed files programmatically from Python.

score 17 · Answer 1 · edited Oct 08 '21 at 06:18

Consider using the the ansible-vault package

Install it by:

$ pip install ansible-vault

and then it is as simple as:

from ansible_vault import Vault

vault = Vault('password')
print vault.load(open('/path/to/your/vault.yml').read())

To use the ansible code directly look at the source of that package. The simplest would be:

Ansible <= 2.3

from ansible.parsing.vault import VaultLib

vault = VaultLib('password')
print(vault.decrypt(open('/path/to/vault.yml').read()))

Ansible >= 2.4

from ansible.constants import DEFAULT_VAULT_ID_MATCH
from ansible.parsing.vault import VaultLib, VaultSecret

vault = VaultLib([(DEFAULT_VAULT_ID_MATCH, VaultSecret('password'.encode()))])
print(vault.decrypt(open('/path/to/vault.yml').read()))

The amount of source code is equal but the package provides automatic yaml parsing + handling of both Ansible versions.

As of Ansible 2.4, VaultLib no longer works with just a password as the argument. — Chuck, Jun 14 '18 at 14:01

score 4 · Answer 2 · answered Jan 09 '19 at 16:12

If you have configured a vault_password_file in ansible.cfg, you can pass the password to VaultLib as followed

Import :

from ansible import constants as C
from ansible.parsing.vault import VaultLib
from ansible.cli import CLI
from ansible.parsing.dataloader import DataLoader

And then, you can call :

loader = DataLoader()
vault_secret = CLI.setup_vault_secrets(
    loader=loader,
    vault_ids=C.DEFAULT_VAULT_IDENTITY_LIST
)
vault = VaultLib(vault_secret)
vault.decrypt(open('/path/to/vault.yml').read())

Note that if `vault_password_file` is executable, VaultLib will use its stdout. — Cochise Ruhulessin, Feb 07 '19 at 09:53

score 3 · Accepted Answer · answered Jun 14 '18 at 14:13

Extending Kuba's answer, ansible-vault is a wrapper around VaultLib. It nicely handles the pre Ansible 2.4 version of Vaultlib along with the post 2.4 version.

The ansible-vault load() method not only decrypts the file, but it also parses it and returns the contents as a dict. If you want the contents without parsing, it is probably easiest to just extend ansible-vault with something like:

from ansible_vault import Vault

class MyVault(Vault):
    def load_raw(self, stream):
        return self.vault.decrypt(stream)

    def dump_raw(self, text, stream=None):
        encrypted = self.vault.encrypt(text)
        if stream:
            stream.write(encrypted)
        else:
            return encrypted

this really helps with things like certificates, as it preserves line endings. Thanks! — ffghfgh, Sep 05 '18 at 17:02
Chuck can you approve the use of your code in the original repo please? https://github.com/tomoh1r/ansible-vault/pull/17 — ffghfgh, Sep 10 '18 at 12:14

score 3 · Answer 4 · answered Sep 24 '19 at 01:00

broferek's answer works if your entire yaml file is encrypted. if your yaml file is unencrypted but contains encrypted variables, it'll complain. This should work either way:

import:

from ansible import constants as C
from ansible.cli import CLI
from ansible.parsing.vault import VaultLib
from ansible.parsing.dataloader import DataLoader

then use the DataLoader class to read the file in to a dict

cfgfile = "/path/to/yaml/file.yml"
loader = DataLoader()
vault_secrets = CLI.setup_vault_secrets(loader=loader,
            vault_ids=C.DEFAULT_VAULT_IDENTITY_LIST)
loader.set_vault_secrets(vault_secrets)
data = loader.load_from_file(cfgfile)
pprint.pprint(data)

just realized this code doesn't actually use ansible.parsing.vault.VaultLib --oops — Chris Vance, Sep 24 '19 at 12:37

score 2 · Answer 5 · answered Aug 08 '17 at 01:32

You want to read and decrypt an encrypted file by Ansible Python API, right?

In Ansible 2.0 and above:

def execute_ansible_command(play_source, stdout_callback):
    from ansible.executor.task_queue_manager import TaskQueueManager
    from ansible.inventory import Inventory
    from ansible.parsing.dataloader import DataLoader
    from ansible.playbook import Play
    from ansible.vars import VariableManager

    Options = namedtuple('Options', ['connection', 'module_path', 'forks', 'remote_user',
                                     'private_key_file', 'ssh_common_args', 'ssh_extra_args', 'sftp_extra_args',
                                     'scp_extra_args', 'become', 'become_method', 'become_user', 'verbosity',
                                     'check'])
    variable_manager = VariableManager()
    loader = DataLoader()
    loader.set_vault_password(ANSIBLE_VAULT_PASS)
    options = Options(connection='smart', module_path=None, forks=100,
                      remote_user=None, private_key_file=None, ssh_common_args="-o StrictHostKeyChecking=no",
                      ssh_extra_args=None, sftp_extra_args=None, scp_extra_args=None, become=None,
                      become_method=None, become_user=None, verbosity=None, check=False)
    passwords = dict()
    inventory = Inventory(loader=loader, variable_manager=variable_manager, host_list=ANSIBLE_INVENTORY)
    variable_manager.set_inventory(inventory)

    play = Play().load(play_source, variable_manager=variable_manager, loader=loader)

    tqm = None
    try:
        tqm = TaskQueueManager(
            inventory=inventory,
            variable_manager=variable_manager,
            loader=loader,
            options=options,
            passwords=passwords,
            stdout_callback=stdout_callback,
        )
        tqm.run(play)
    finally:
        if tqm is not None:
            tqm.cleanup()

The DataLoader class is used to load and parse YAML or JSON content, and it has a set_vault_password function, You can send a vault password to decrypt a vault-encrypted file

score 1 · Answer 6 · answered Jan 13 '20 at 15:46

I couldn't get the decryption working with the above answers but I do have a function here that uses subprocess and works well:

def vault_decrypt(text, vault_password_file):
    """
    Calls ansible vault and pass the payload via stdin
    :param text: str, text to decode
    :param vault_password_file: str, vault password
    :return: str, decoded text
    """
    cmd = ['ansible-vault', 'decrypt', '--vault-password-file={}'.format(vault_password_file)]
    p = Popen(cmd,
              stdout=PIPE, stdin=PIPE, stderr=PIPE)
    output = p.communicate(input=str.encode(text))
    return output[0].decode("utf-8")

If I do update it to use the ansible python module directly I'll update. Cheers.

score 0 · Answer 7 · answered May 23 '17 at 21:48

This isn't exactly what I want but did work around my above issue by running the ansible view <vaultfile> command via subprocess.

import subprocess
import yaml

def getCreds():
    cmd = "ansible-vault view credentials.vault"
    process = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
    (result, error) = process.communicate()

    resultYaml = yaml.load(result)

    accesskey = resultYaml['accesskey']
    secretkey = resultYaml['secretkey']

    return(accesskey, secretkey)

Directly calling Ansible methods directly in Python would still be best.

How to view/decrypt Ansible vault credentials files from within a Python script?

7 Answers7