Iterate Multiple Lists Ansible

I wanted to setup some python virtualenvs on a laptop, and wanted to improve the existing Ansible automation. Goals were:

  • Install multiple virtualenvs.
  • Install the virtualenv according to a requirements file.
  • Don’t install if the virtualenv exists.
  • Don’t install if the requirements.txt doesn’t exist.

Here is our vars file:

virtualenvs:
  - { reqs: "/some/dir/project1", venv: "proj1" }
  - { reqs: "/some/dir/project2", venv: "proj2" }
  - { reqs: "/some/dir/project3", venv: "proj3" }

And here is a tasks file (debug is included if you want to have some output):

- name: Ensure virtualenvs exist
  block:
    - name: Check if project virtualenv requirements exist
      # become_user: some_user # use this to use with a local user when ansible is running as root
      ansible.builtin.stat:
        path: "{{ item.reqs }}/requirements.txt"
      loop: "{{ virtualenvs }}"
      register: "reqs"

    - name: Check if project virtualenvs exist
      # become_user: some_user # use this to use with a local user when ansible is running as root
      ansible.builtin.stat:
        path: "~/.virtualenv/env.{{ item.venv }}"
      loop: "{{ virtualenvs }}"
      register: "venvs"

    - debug: var=reqs

    - debug: var=venvs

    - debug:
        msg: "{{ item.0.stat.exists }} - {{ item.1.stat.exists }}"
      loop: "{{ venvs.results|zip(reqs.results)|list }}"

    - debug:
        msg: "{{ item.0.stat.exists }} - {{ item.1.stat.exists }} - {{ item.2 }}"
      loop: "{{ data[0]|zip(*data[1:])|list }}"
      vars:
        data:
          - "{{ reqs.results }}"
          - "{{ venvs.results }}"
          - "{{ virtualenvs }}"

    - name: Create project virtualenvs if they don't exist
      # become_user: some_user # use this to use with a local user when ansible is running as root
      pip:
        requirements: "{{ item.2.reqs }}/requirements.txt"
        virtualenv: "~/.virtualenv/env.{{ item.2.venv }}"
      loop: "{{ data[0]|zip(*data[1:])|list }}"
      vars:
        data:
          - "{{ reqs.results }}"
          - "{{ venvs.results }}"
          - "{{ virtualenvs }}"
      when: item.0.stat.exists and not item.1.stat.exists

Explanation

Getting file existence data

We need to determine whether to create each virtualenv, and we do that by checking for the existence of:

  • the requirements.txt file
  • the virtualenv itself.

We do this using Ansible’s ansible.builtin.stat module and store the results in reqs and venvs. Each of these variables has the information we need in the results key:

  • reqs.results
  • venvs.results

results is a list that we can iterate over.

Combining our lists for iteration

To use our file/virtualenv checks in the virtualenv creation task, we need to iterate over three things:

  • reqs.results (tells us whether the requirements file exists)
  • venvs.results (tells us whether the virtualenvs already exist)
  • virtualenvs (the list of virtualenvs that should be created)

So:

We need to combine the lists so we can do our conditional checks.

We do all this in the last task:

- name: Create project virtualenvs if they don't exist
  # become_user: some_user # use this to use with a local user when ansible is running as root
  pip:
    requirements: "{{ item.2.reqs }}/requirements.txt"
    virtualenv: "~/.virtualenv/env.{{ item.2.venv }}"
  loop: "{{ data[0]|zip(*data[1:])|list }}"
  vars:
    data:
      - "{{ reqs.results }}"
      - "{{ venvs.results }}"
      - "{{ virtualenvs }}"
  when: item.0.stat.exists and not item.1.stat.exists

There are two key things here:

First, our lists are in a var, data. Second, we combine the lists in our loop:

loop: "{{ data[0]|zip(*data[1:])|list }}"

Inside the zip parens, the splat operator unpacks the data list, starting at index 1. Those lists are zipped together. Then, the first list in data is combined with the zipped list. This is very simplified, but we’re essentially getting a list that looks like this that we can iterate over:

reqs.results.exists: true,  venvs.results.exists: true, { reqs: "/some/dir/project1", venv: "proj1" }
reqs.results.exists: true,  venvs.results.exists: false, { reqs: "/some/dir/project2", venv: "proj2" }
reqs.results.exists: false, venvs.results.exists: true, { reqs: "/some/dir/project3", venv: "proj3" }

This list is then used to do our checks and grab the info we need to build our virtualenvs.