{"id":2417,"date":"2018-03-16T12:58:23","date_gmt":"2018-03-16T17:58:23","guid":{"rendered":"http:\/\/osric.com\/chris\/accidental-developer\/?p=2417"},"modified":"2018-03-16T12:59:15","modified_gmt":"2018-03-16T17:59:15","slug":"installing-ansible-role-dependencies","status":"publish","type":"post","link":"https:\/\/osric.com\/chris\/accidental-developer\/2018\/03\/installing-ansible-role-dependencies\/","title":{"rendered":"Installing Ansible role dependencies"},"content":{"rendered":"<p>I have a monolithic Ansible playbook that contains dozens of different roles, all bundled into the same Git repository. Some of the roles are more generically useful than others, so I thought I would do some refactoring.<\/p>\n<p>I decided to move the role that installs and configures fail2ban to its own repository, and then call that new\/refactored role as a dependency in my now-slightly-less-monolithic role.<\/p>\n<p>Of course, I had no idea what I was doing.<br \/>\n<!--more--><\/p>\n<p>The first thing I tried was creating a simple ansible-test playbook with a single test role. Just to make sure it did something, I had it copy a file to the \/tmp directory on the target host. In <code>roles\/test\/meta\/main.yml<\/code> I added a dependency:<\/p>\n<pre><code>---  \r\ndependencies:\r\n  - { role: \"git+https:\/\/git.osric.net\/chris\/ansible-fail2ban.git\" }\r\n...<\/code><\/pre>\n<p>Then I tried running the playbook:<\/p>\n<pre><code>$ ansible-playbook -i hosts site.yml<\/code><\/pre>\n<p>Which produced the following error:<\/p>\n<pre><code>ERROR! the role 'git+https:\/\/git.osric.net\/chris\/ansible-fail2ban.git' was not found in \/home\/chris\/ansible-test\/roles:\/usr\/local\/etc\/ansible\/roles:\/home\/chris\/ansible-test\/roles:\/home\/chris\/ansible-test\r\n\r\nThe error appears to have been in '\/home\/chris\/ansible-test\/roles\/test\/meta\/main.yml': line 4, column 3, but may\r\nbe elsewhere in the file depending on the exact syntax problem.\r\n\r\nThe offending line appears to be:\r\n\r\ndependencies:\r\n- { role: \"git+https:\/\/git.osric.net\/chris\/ansible-fail2ban.git\" }\r\n  ^ here\r\nThis one looks easy to fix.  It seems that there is a value started\r\nwith a quote, and the YAML parser is expecting to see the line ended\r\nwith the same kind of quote.  For instance:\r\n\r\n    when: \"ok\" in result.stdout\r\n\r\nCould be written as:\r\n\r\n   when: '\"ok\" in result.stdout'\r\n\r\nOr equivalently:\r\n\r\n   when: \"'ok' in result.stdout\"<\/code><\/pre>\n<p>That&#8217;s a long and, as it turns out, misleading error. There is no issue with unbalanced quotes. The real error is on the first line:<\/p>\n<pre><code>ERROR! the role 'git+https:\/\/git.osric.net\/chris\/ansible-fail2ban.git' was not found<\/code><\/pre>\n<p>Isn&#8217;t is supposed to automatically install? Isn&#8217;t that the magic of dependencies?<\/p>\n<p>Apparently not. The convention, as far as I can tell, is to use a <code>requirements.yml<\/code> file that describes where to find the dependencies, e.g.:<\/p>\n<pre><code>---\r\nsrc: git+https:\/\/git.osric.net\/chris\/ansible-fail2ban.git\r\n...<\/code><\/pre>\n<p>Then you use <code>ansible-galaxy<\/code> to install the dependencies:<\/p>\n<pre><code>$ ansible-galaxy install -r requirements.yml -p roles\/<\/code><\/pre>\n<p>It worked! I now had a new role in <code>roles\/ansible-fail2ban<\/code> with the expected content.<\/p>\n<p>It turned out that I had left out an important piece of the <code>ansible-fail2ban<\/code> role (I forgot to <code>add<\/code>, <code>commit<\/code>, and <code>push<\/code> the <code>defaults\/main.yml<\/code> into the remote origin).<\/p>\n<p>I tried re-installing the repository using the <code>--force<\/code> option:<\/p>\n<pre><code>$ ansible-galaxy install --force -r requirements.yml -p roles\/\r\n- ansible-fail2ban is already installed, skipping.<\/code><\/pre>\n<p>Wait, what? What is <code>--force<\/code> actually supposed to do? It turned out, this was a bug in my version of <code>ansible<\/code> (and <code>ansible-galaxy<\/code>), as described in this bug: <a href=\"https:\/\/github.com\/ansible\/galaxy-issues\/issues\/249\">https:\/\/github.com\/ansible\/galaxy-issues\/issues\/249<\/a><\/p>\n<p><strong>Before:<\/strong><\/p>\n<pre><code>$ ansible-galaxy --version\r\nansible-galaxy 2.3.1.0<\/code><\/pre>\n<p><strong>After:<\/strong><\/p>\n<pre><code>$ ansible-galaxy --version\r\nansible-galaxy 2.4.3.0<\/code><\/pre>\n<p>I ran the install again:<\/p>\n<pre><code>$ ansible-galaxy install --force -r requirements.yml -p roles\/\r\n- changing role ansible-fail2ban from  to unspecified\r\n- extracting ansible-fail2ban to \/home\/chris\/ansible-test\/roles\/ansible-fail2ban\r\n- ansible-fail2ban was installed successfully<\/code><\/pre>\n<p>It worked! But the first output line about that bit &#8220;<code>changing role ansible-fail2ban from  to unspecified<\/code>&#8221; led me to believe that I could change the role destination. I&#8217;d rather have <code>roles\/fail2ban<\/code> than <code>roles\/ansible-fail2ban<\/code>.<\/p>\n<p>We <em>can<\/em> specify additional attributes, including <code>name<\/code>. See <a href=\"http:\/\/docs.ansible.com\/ansible\/latest\/galaxy.html#installing-multiple-roles-from-a-file\">Ansible Galaxy &#8211; Installing Multiple Roles from a File<\/a> for details.<\/p>\n<p>[Pet peeve: documentation that does not include the most verbose &#038; complete example possible. Every single option should be demonstrated. Why not include both? Here is the simplest possible example that works, here is the most complete example possible. This is open source, so I could fix it I suppose&#8230;.]<\/p>\n<p>I updated my <code>requirements.yml<\/code> file:<\/p>\n<pre><code>---\r\n- name: fail2ban\r\n  scm: git\r\n  src: git+https:\/\/git.osric.net\/chris\/ansible-fail2ban.git\r\n  version: master  \r\n...<\/code><\/pre>\n<p>I installed the requirements again:<\/p>\n<pre><code>$ ansible-galaxy install --force -r requirements.yml -p roles\/<\/code><\/pre>\n<p>Which successfully created an additional copy of the role in the specified directory:<\/p>\n<pre><code>$ ls roles | cat\r\nfail2ban\r\nansible-fail2ban\r\ntest<\/code><\/pre>\n<p>That meant I also needed to update the dependency in <code>roles\/test\/meta\/mail.yml<\/code>:<\/p>\n<pre><code>---\r\ndependencies: \r\n  - { role: fail2ban }\r\n...<\/code><\/pre>\n<p>I ran the test playbook again. It worked! Now I just have a couple dozen more roles to refactor.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>I decided to refactor a monolithic repository of Ansible roles into smaller components that could be more easily reused as dependencies. I had no idea what I was doing and ran into a couple interesting errors and questions along the way.<\/p>\n","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[439],"tags":[423,490],"class_list":["post-2417","post","type-post","status-publish","format-standard","hentry","category-ansible","tag-ansible","tag-ansible-galaxy"],"_links":{"self":[{"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/posts\/2417","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/comments?post=2417"}],"version-history":[{"count":8,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/posts\/2417\/revisions"}],"predecessor-version":[{"id":2427,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/posts\/2417\/revisions\/2427"}],"wp:attachment":[{"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/media?parent=2417"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/categories?post=2417"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/osric.com\/chris\/accidental-developer\/wp-json\/wp\/v2\/tags?post=2417"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}