Advanced Topics
In this section we take a quick look at more advanced concepts in ansible and introduce some useful options.
Last updated
In this section we take a quick look at more advanced concepts in ansible and introduce some useful options.
Last updated
By default Ansible runs tasks synchronously, holding the connection to the remote node open until the action is completed. This means within a playbook, each task blocks the next task by default, meaning subsequent tasks will not run until the current task completes.
This behavior can create challenges. For example, a task may take longer to complete than the SSH session allows for, causing a timeout. Or you may want a long-running process to execute in the background while you perform other tasks concurrently. Asynchronous mode lets you control how long-running tasks execute.
Asynchronous mode allows us to control the playbook execution flow by defining how long-running tasks completes their execution.
To enable Asynchronous mode within Ansible playbook we need to use few parameters such as async, poll.
async: Async indicates the Total time to complete the task or its maximum runtime of the task.
poll: poll indicates to Ansible, how often to poll to check if the command has been completed. or how frequently you would like to poll for status. with poll we keep checking whether the job is completed or not. The default poll value is 10 seconds.
Remember all the modules do not support async, so you need to make sure that what ever module you're using supports async.
If you want to set a longer timeout limit for a certain task in your playbook, use async with poll set to a positive value. Ansible will still block the next task in your playbook, waiting until the async task either completes, fails or times out. However, the task will only time out if it exceeds the timeout limit you set with the async parameter.
If you want to run multiple tasks in a playbook concurrently, use async with poll set to 0. When you set poll: 0, Ansible starts the task and immediately moves on to the next task without waiting for a result. Each async task runs until it either completes, fails or times out (runs longer than its async value). The playbook run ends without checking back on async tasks.
Ansible provides the option to get the task status in any time. Using ansible async_status we can get the status of async task at any time.
execute the playbook with
-voption so that we can see the Job ID of first task to check its status later.
To check the status of first task on target node(s) in a later time we can use ad hoc commands:
You can execute long-running operations in the background with ad hoc tasks. For example, to execute long_running_operation asynchronously in the background, with a timeout (-B) of 3600 seconds, and without polling (-P):
Again to check on the job status later, use the async_status module, passing it the job ID that was returned when you ran the original job in the background:
Ansible can also check on the status of your long-running job automatically with polling. In most cases, Ansible will keep the connection to your remote node open between polls. To run for 30 minutes and poll for status every 60 seconds:
Poll mode is smart so all jobs will be started before polling begins on any machine. Be sure to use a high enough
--forksvalue if you want to get all of your jobs started very quickly. After the time limit (in seconds) runs out (-B), the process on the remote nodes will be terminated.
Asynchronous mode is best suited to long-running shell commands or software upgrades. Running the copy module asynchronously, for example, does not do a background file transfer.
What exactly strategies is ? Strategies define how a playbook is executed in ansible . Lets start explaining with an example:
With this playbook ,imaging we are trying to install a web application on a single server in an all-in-one mode (meaning we have a database and web server running on a same system) It is pretty straight forward in a single server. When ansible runs it completes one task after the other :
Now let's see how it works when there are multiple servers.
By default, plays run with a linear strategy, in which all hosts will run each task before any host starts the next task:
In this case, each server run all of its tasks independent of the other servers, and does not wait for the task to finish on the other servers. So each server can go right to the end as fast as it can and a host that is slow or stuck on a specific task won’t hold up the rest of the hosts and tasks.
Imagine we have 5 servers but we want ansible to execute 3 at a time. This is where bad processing helps us. This is not a sperate strategy this is based on linier strategy but we can control number of servers executed at once or in a batch.
In playbook we do not have free strategy but there is a new option called serial where you can specify the number of servers you would like to process together:
There are some additional options available such as using a percentage.
fork defines how many server ansible can talk to at a time. Imagine in previous example, what if we had 1000 of servers and wished to run out playbook across all of them at the same time?! Would ansible run the play book on all of them at the same time?
The answer in no (unless you specify explicitly). Ansible uses parallel processes or forks to communicate with remote hosts. The default value is 5 and it is defined in ansible configuration file ansible.cfg :
So we try to run our playbook on 1000 hosts, ansible will only run across 5 at a time. You can increase this value in configuration file but before that, make sure you have sufficient CPU resources and network bandwidth.
In Ansible, is common thing to run the same playbook on parallel on many target. Sometimes, on a particular target you can receive an error. In this case, by default, Ansible stops the execution on that target, but continues to run the playbook on the other targets. This is ansible default behavior:
However, you can change ansible default behaviour by adding the any_errors_fatal: true line in the playbook.
In this case, when an error occurs, Ansible stops the execution of the playbook on all the targets.
You can handle the errors at the task level as well like playbook below(you have seen that in previous sections):
ignore_errors: yes => any error you can have at this task is ignored
failed_when: "'FAILED' in results" => Ansible consider that this task failed if in the result of the task you find the word 'FAILED'.
Ansible has a rich set of filters backed by jinja2 templating. We input some data into these templates and the jinja2 template engine process that data and provide the output accordingly. Filters mostly used for formatting or transforming the data. Then based on the output any other tasks are performed. Also, filtering is very useful in debugging. Filtering syntax is like the below:
where the left side is the input to templating and the right side is the filter provided by ansible. The output can be printed on-screen or can be registered in a variable. Lets take a look at some examples:
Additional Filters
The list of built-in filters shipped by jinja2 can be checked from the https://jinja.pocoo.org/docs/templates/#builtin-filters, which is an official link for jinja2 template designer documentation.
The list of filters that are provided by Ansible can be checked from the https://docs.ansible.com/ansible/latest/user_guide/playbooks_filters.html, which is an official documentation link for Ansible playbook filters.
Templates when used with debug modules, add more verbosity and troubleshooting information to the output of a playbook. We have seen these kinds of scenarios in previous sections.
Lookup plugins are an Ansible-specific extension to the Jinja2 templating language. Like all templating, lookup plugins are evaluated on the Ansible control machine. You can use lookup plugins to retrieve data from outside sources (files, databases, key/value stores, APIs, and other services) within your playbooks.
Large portion of Ansible’s functionality comes from the Ansible plugin system. These important pieces of code augment Ansible’s core functionality such as parsing and loading inventory and Playbooks, running Playbooks and reading the results. Essentially, Ansible uses plugins to extend what the system is doing under the hood.
The following types of plugins are available:
Action plugins are front ends to modules and can execute actions on the controller before calling the modules themselves.
Cache plugins are used to keep a cache of ‘facts’ to avoid costly fact-gathering operations.
Callback plugins enable you to hook into Ansible events for display or logging purposes.
Connection plugins define how to communicate with inventory hosts.
Filters plugins allow you to manipulate data inside Ansible plays and/or templates. This is a Jinja2 feature; Ansible ships extra filter plugins.
Lookup plugins are used to pull data from an external source. These are implemented using a custom Jinja2 function.
Strategy plugins control the flow of a play and execution logic.
Shell plugins deal with low-level commands and formatting for the different shells Ansible can encounter on remote hosts.
Test plugins allow you to validate data inside Ansible plays and/or templates. This is a Jinja2 feature; Ansible ships extra test plugins.
Vars plugins inject additional variable data into Ansible runs that did not come from an inventory, playbook, or the command line.
Visit ansible github page (https://github.com/ansible/ansible/tree/devel/lib/ansible/plugins) for more information. You can write your own too, but that is more advanced topic!
That's all, good luck.
.
.
.
https://docs.ansible.com/ansible/latest/user_guide/playbooks_async.html
https://blog.learncodeonline.in/ansible-advanced-speed-up-playbook-run-with-async-and-polling
https://www.decodingdevops.com/ansible-asynch-poll-with-examples/https://docs.ansible.com/ansible/2.5/user_guide/playbooks_strategies.htmlhttps://docs.ansible.com/ansible/latest/collections/ansible/builtin/free_strategy.html#free-strategy
https://www.educba.com/ansible-filters/
https://docs.ansible.com/ansible/latest/plugins/lookup.htmlhttps://docs.ansible.com/ansible/latest/user_guide/playbooks_lookups.html
https://docs.ansible.com/ansible/2.4/playbooks_lookups.html#id3
https://www.devopsschool.com/blog/what-is-ansible-plugins-and-how-to-extend-ansible-through-plugins/
.
