I am trying to run the following command with the Ansible shell module. This same command works fine in a remote shell but I get the following error in Ansible:
- name: "Searching coincidendences for literal stratum 70 in /var/log/required.log and /var/log/required.1"
shell: "ls /var/log/required.log?(.1) | xargs grep -i 'stratum 70' |wc -l"
args:
executable: /bin/bash
This is the error message with ansible -vvvv:
fatal: [node_test]: FAILED! => {
"changed": true,
"cmd": "ls /var/log/required.log?(.1) | xargs grep -i 'stratum 70' |wc -l",
"delta": "0:00:00.005635",
"end": "2021-07-12 17:50:52.370057",
"invocation": {
"module_args": {
"_raw_params": "ls /var/log/required.log?(.1) | xargs grep -i 'stratum 70' |wc -l",
"_uses_shell": true,
"argv": null,
"chdir": null,
"creates": null,
"executable": null,
"removes": null,
"stdin": null,
"stdin_add_newline": true,
"strip_empty_ends": true,
"warn": true
}
},
"msg": "non-zero return code",
"rc": 2,
"start": "2021-07-12 17:50:52.364422",
"stderr": "/bin/sh: 1: Syntax error: \"(\" unexpected",
"stderr_lines": [
"/bin/sh: 1: Syntax error: \"(\" unexpected"
],
"stdout": "",
"stdout_lines": [] **strong text**
}
It seems that the problem is with the wildcards ?(.1) to replace characters but i don't know how to escape properly
Extended globbing must be declared it inside the block.
For that I tend to prefer a block scalar.
- name: "Searching coincidendences for literal stratum 70 in /var/log/required.log and /var/log/required.1"
shell: |
shopt -s extglob;
grep -i 'stratum 70' /var/log/required.log?(.1) | wc -l
args:
executable: /bin/bash
I don't think you really need it though -
- name: "Searching coincidendences for literal stratum 70 in /var/log/required.log and /var/log/required.1"
shell: "shopt -s extglob; grep -i 'stratum 70' /var/log/required.log?(.1) | wc -l"
args:
executable: /bin/bash
Was there some reason you needed the ls?
Related
This question already has answers here:
Difference between sh and Bash
(11 answers)
shell module: Bash <(process substitution) with ansible
(3 answers)
Closed 6 months ago.
I'm trying to create a playbook that incorporates the following 2 commands and due to their special characters, i cant get it to work. Is there a list of special characters i need to account for in ansible using a bash command?
- name: If file exists compare new users.txt
shell: >
diff -q users.txt <(getent passwd | awk -F: '{ print $1 }') 1>/dev/null; error="$?"
getent passwd | awk -F: '{ print $1 }' > users.txt
The error is
fatal: [localhost]: FAILED! => {
"changed": true,
"cmd": "diff -q users.txt <(getent passwd | awk -F: '{ print $1}') 1>/dev/null; error=\"$?\"\n",
"delta": "0:00:00.002893",
"end": "2022-08-19 23:43:07.324939",
"invocation": {
"module_args": {
"_raw_params": "diff -q users.txt <(getent passwd | awk -F: '{ print $1}') 1>/dev/null; error=\"$?\"\n",
"_uses_shell": true,
"argv": null,
"chdir": null,
"creates": null,
"executable": null,
"removes": null,
"stdin": null,
"stdin_add_newline": true,
"strip_empty_ends": true,
"warn": true
}
},
"msg": "non-zero return code",
"rc": 1,
"start": "2022-08-19 23:43:07.322046",
"stderr": "/bin/sh: -c: line 0: syntax error near unexpected token `('\n/bin/sh: -c: line 0: `diff -q users.txt <(getent passwd | awk -F: '{ print $1}') 1>/dev/null; error=\"$?\"'",
"stderr_lines": [
"/bin/sh: -c: line 0: syntax error near unexpected token `('",
"/bin/sh: -c: line 0: `diff -q users.txt <(getent passwd | awk -F: '{ print $1}') 1>/dev/null; error=\"$?\"'"
],
"stdout": "",
"stdout_lines": []
}
Q: "Update the list of users in /tmp/users.txt if the list differs from those in /etc/passwd."
A: Use the module getent to read /etc/passwd. For example,
- name: Create dictionary ansible_facts.getent_passwd
ansible.builtin.getent:
database: passed
Next, read the file. Set changed_when: false to keep the play idempotent
- name: Create list users.stdout_lines from /tmp/users.txt
command: cat /tmp/users.txt
register: users
changed_when: false
Compare the lists and update the file if needed
- copy:
dest: /tmp/users.txt
content: |
{% for user in ansible_facts.getent_passwd.keys() %}
{{ user }}
{% endfor %}
when: diff|length > 0
vars:
diff: ansible_facts.getent_passwd.keys()|list|
difference(users.stdout_lines)
Example of a complete playbook for testing
- hosts: localhost
tasks:
- name: Create dictionary ansible_facts.getent_passwd
ansible.builtin.getent:
database: passwd
- ansible.builtin.debug:
var: ansible_facts.getent_passwd.keys()|list
when: debug|d(false)|bool
- name: Create list users.stdout_lines from /tmp/users.txt
command: cat /tmp/users.txt
register: users
changed_when: false
- ansible.builtin.debug:
var: users.stdout_lines
when: debug|d(false)|bool
- name: Update /tmp/users.txt
copy:
dest: /tmp/users.txt
content: |
{% for user in ansible_facts.getent_passwd.keys() %}
{{ user }}
{% endfor %}
when: new_users|length > 0
vars:
new_users: "{{ ansible_facts.getent_passwd.keys()|
difference(users.stdout_lines) }}"
Example of the update
If I remove the users admin, user1, and user2 from the file, the play gives (abridged) in --diff mode
shell> ansible-playbook pb.yml -CD
TASK [Update /tmp/users.txt] ************************************************
--- before: /tmp/users.txt
+++ after: /home/tester/.ansible/tmp/ansible-local-903563m7yp2ywc/tmpvm5fd7j2
## -50,4 +50,7 ##
libvirt-qemu
libvirt-dnsmasq
asadmin
+admin
+user1
+user2
_chrony
changed: [localhost]
Q: "Detect new users that were added."
A: Report new users in the block. For example,
- hosts: localhost
gather_facts: false
tasks:
- name: Create dictionary ansible_facts.getent_passwd
ansible.builtin.getent:
database: passwd
- ansible.builtin.debug:
var: ansible_facts.getent_passwd.keys()|list
when: debug|d(false)|bool
- name: Create list users.stdout_lines from /tmp/users.txt
command: cat /tmp/users.txt
register: users
changed_when: false
- ansible.builtin.debug:
var: users.stdout_lines
when: debug|d(false)|bool
- name: Report new users and update file
block:
- name: Report new users
debug:
msg: "New users: {{ new_users }}"
- name: Update /tmp/users.txt
copy:
dest: /tmp/users.txt
content: |
{% for user in ansible_facts.getent_passwd.keys() %}
{{ user }}
{% endfor %}
when: new_users|length > 0
vars:
new_users: "{{ ansible_facts.getent_passwd.keys()|
difference(users.stdout_lines) }}"
I have added to my playbook a small task that should change umask on my linux machine:
- name: set umask to 0022
shell: umask 0022
When running the playbook, I can see this task passes successfully:
changed: [myHostName] => {
"changed": true,
"cmd": "umask 0022",
"delta": "0:00:00.004660",
"end": "2020-08-04 16:28:44.153261",
"invocation": {
"module_args": {
"_raw_params": "umask 0022",
"_uses_shell": true,
"argv": null,
"chdir": null,
"creates": null,
"executable": null,
"removes": null,
"stdin": null,
"stdin_add_newline": true,
"strip_empty_ends": true,
"warn": true
}
},
"rc": 0,
"start": "2020-08-04 16:28:44.148601",
"stderr": "",
"stderr_lines": [],
"stdout": "",
"stdout_lines": []
}
but After the playbook finishes, I check the umask and see that it was not changed at all:
-bash-4.2$ umask
0044
I also put a debug in my playbook right after the task I showed above, and the debug also shows that the umask was not changed..
Tried also with
become: yes
But got the same result..
When I do the command on my Linux manually, it will work:
-bash-4.2$ umask 0022
-bash-4.2$ umask
0022
Q: After the playbook finishes, I check the umask and see that it was not changed at all.
A: This is correct. Ansible isn't really doing things through the shell i.e. the changes live in this one session only.
I am trying to automate deployment for ubuntu 19 and in order to that I am loading my preseed file into the initial ram disk. I am successfully able to extract the contents of the initrd using gunzip and cpio or load into a temp folder which i will zip up to initrd.cpio.gz so the kernel can read it during boot.
The issue I am having is when I copy my preseed.cfg.j2 to dest: preseed.cfg because it is a template. A task that shouldn't require much effort is pausing on me.
eTASK [ubuntu19 : Move Preseed File to the root directory of tmpinitrd = temp dir we will compress to initrd.cpio.gz] ***********************************************************************************************************************************************************
task path: /root/csc_os_deploy/roles/ubuntu19/tasks/image_preparation.yml:112
changed: [myhost -> ansible-build-99] => {
"changed": true,
"checksum": "2912bc8310b18001a31dfdba3b12e254e097d88a",
"dest": "/opt/deploy/build/ubuntu19/myhost/install/tmpinitrd/preseed.cfg",
"diff": [],
"gid": 0,
"group": "root",
"invocation": {
"module_args": {
"_original_basename": "preseed.cfg.j2",
"attributes": null,
"backup": false,
"checksum": "2912bc8310b18001a31dfdba3b12e254e097d88a",
"content": null,
"delimiter": null,
"dest": "/opt/deploy/build/ubuntu19/myhost/install/tmpinitrd/preseed.cfg",
"directory_mode": null,
"follow": false,
"force": true,
"group": null,
"local_follow": null,
"mode": null,
"owner": null,
"regexp": null,
"remote_src": null,
"selevel": null,
"serole": null,
"setype": null,
"seuser": null,
"src": "/root/.ansible/tmp/ansible-tmp-1571431258.4919047-41099342027844/source",
"unsafe_writes": null,
"validate": null
}
},
"md5sum": "8be3aab512be1a959fc8bc716fa1f530",
"mode": "0644",
"owner": "root",
"size": 16627,
"src": "/root/.ansible/tmp/ansible-tmp-1571431258.4919047-41099342027844/source",
"state": "file",
"uid": 0
}
<ansible-build-99> ESTABLISH SSH CONNECTION FOR USER: root
<ansible-build-99> SSH: EXEC sshpass -d10 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/c6405ad5d4 ansible-build-99 '/bin/sh -c '"'"'echo ~root && sleep 0'"'"''
<ansible-build-99> (0, b'/root\n', b'')
<ansible-build-99> ESTABLISH SSH CONNECTION FOR USER: root
<ansible-build-99> SSH: EXEC sshpass -d10 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/c6405ad5d4 ansible-build-99 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /root/.ansible/tmp/ansible-tmp-1571431259.6397204-26088703665992 `" && echo ansible-tmp-1571431259.6397204-26088703665992="` echo /root/.ansible/tmp/ansible-tmp-1571431259.6397204-26088703665992 `" ) && sleep 0'"'"''
<ansible-build-99> (0, b'ansible-tmp-1571431259.6397204-26088703665992=/root/.ansible/tmp/ansible-tmp-1571431259.6397204-26088703665992\n', b'')
Using module file /usr/local/lib/python3.6/site-packages/ansible/modules/commands/command.py
<ansible-build-99> PUT /root/.ansible/tmp/ansible-local-31522g5qin4hz/tmpq75usr7y TO /root/.ansible/tmp/ansible-tmp-1571431259.6397204-26088703665992/AnsiballZ_command.py
<ansible-build-99> SSH: EXEC sshpass -d10 sftp -o BatchMode=no -b - -C -o ControlMaster=auto -o ControlPersist=60s -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/c6405ad5d4 '[ansible-build-99]'
<ansible-build-99> (0, b'sftp> put /root/.ansible/tmp/ansible-local-31522g5qin4hz/tmpq75usr7y /root/.ansible/tmp/ansible-tmp-1571431259.6397204-26088703665992/AnsiballZ_command.py\n', b'')
<ansible-build-99> ESTABLISH SSH CONNECTION FOR USER: root
<ansible-build-99> SSH: EXEC sshpass -d10 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/c6405ad5d4 ansible-build-99 '/bin/sh -c '"'"'chmod u+x /root/.ansible/tmp/ansible-tmp-1571431259.6397204-26088703665992/ /root/.ansible/tmp/ansible-tmp-1571431259.6397204-26088703665992/AnsiballZ_command.py && sleep 0'"'"''
<ansible-build-99> (0, b'', b'')
<ansible-build-99> ESTABLISH SSH CONNECTION FOR USER: root
<ansible-build-99> SSH: EXEC sshpass -d10 ssh -C -o ControlMaster=auto -o ControlPersist=60s -o 'User="root"' -o ConnectTimeout=10 -o ControlPath=/root/.ansible/cp/c6405ad5d4 -tt ansible-build-99 '/bin/sh -c '"'"'/usr/bin/python /root/.ansible/tmp/ansible-tmp-1571431259.6397204-26088703665992/AnsiballZ_command.py && sleep 0'"'"''
Nothing happens after this, it freezes!!
I have this ansible playbook:
---
- hosts: all
gather_facts: False
become: yes
become_user: myins
become_method: sudo
tasks:
- name: test someting
command: "cd /db2/myins"
- name: relocate it
command: "db2relocatedb -f /db2/myins/relocate.cfg"
...
if I run the command by itself it works and if I use cat /db2/myins/relocate.cfg instead of db2relocatedb -f it works too.
If I run it like this I get:
The full traceback is:
WARNING: The below traceback may *not* be related to the actual failure.
File "/tmp/ansible_command_payload_HGHkvR/ansible_command_payload.zip/ansible/module_utils/basic.py", line 2561, in run_command
cmd = subprocess.Popen(args, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
fatal: [mounttest]: FAILED! => {
"changed": false,
"cmd": "db2relocatedb -f /db2/myins/relocate.cfg",
"invocation": {
"module_args": {
"_raw_params": "db2relocatedb -f /db2/myins/relocate.cfg",
"_uses_shell": false,
"argv": null,
"chdir": null,
"creates": null,
"executable": null,
"removes": null,
"stdin": null,
"stdin_add_newline": true,
"strip_empty_ends": true,
"warn": true
}
},
"msg": "[Errno 2] No such file or directory",
"rc": 2
}
Why can I cat the file but not use it?
Thanks,
Arengin
the problem is inside relocate.cfg, check all Pathes make sure they are full qualified, make sure no escape character is somewhere
i am using ansible to start the namenode process if incase its not running, I am using jps to check for the process and check the status using service_namenode_status and start the process using service_namenode_start any idea how to use this properly...?
- name: Check if Namenode is running
shell: jps | grep " NameNode" | grep -v grep
ignore_errors: yes
changed_when: false
register: service_namenode_status
- name: Report status of Namenode
fail:
msg: |
Service NameNode is not running.
Return code from `jps | grep " NameNode" | grep -v grep`:
{{ service_namenode_status.rc }}
when: service_namenode_status.rc != 0
register: service_namenode_start
- name: Start Namenode
command: su - hdfs -c "/usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh --config $HADOOP_CONF_DIR start namenode"
tags: namenode
when: service_namenode_status.rc != 0
OUTPUT:
TASK [namenode : Check if Namenode is running] *********************************
fatal: [server1.hdp.vagrant.data.uc]: FAILED! => {"changed": false, "cmd": "jps | grep \" NameNode\" | grep -v grep", "delta": "0:00:00.192755", "end": "2016-11-09 10:59:10.360552", "failed": true, "rc": 1, "start": "2016-11-09 10:59:10.167797", "stderr": "", "stdout": "", "stdout_lines": [], "warnings": []}
...ignoring
TASK [namenode : Report status of Namenode] ************************************
fatal: [server1.hdp.vagrant.data.uc]: FAILED! => {"changed": false, "failed": true, "msg": "Service NameNode is not running.\nReturn code from `jps | grep \" NameNode\" | grep -v grep`:\n1 \n"}
NO MORE HOSTS LEFT *************************************************************
RUNNING HANDLER [metastore : restart postgresql] *******************************
to retry, use: --limit #/Users/krisdigitx/myLab/hdp/provisioning/site.retry
PLAY RECAP *********************************************************************
server1.hdp.vagrant.data.uc : ok=86 changed=32 unreachable=0 failed=1
Ansible failed to complete successfully. Any error output should be
visible above. Please fix these errors and try again.
Try this:
- name: Check if Namenode is running
tags: namenode
shell: jps | grep " NameNode" | grep -v grep
ignore_errors: yes
changed_when: false
register: service_namenode_status
- name: Start Namenode
tags: namenode
command: su - hdfs -c "/usr/hdp/current/hadoop-client/sbin/hadoop-daemon.sh --config $HADOOP_CONF_DIR start namenode"
when: service_namenode_status.rc == 1
My changes were:
I added the "tags: namenode" line to the 1st part; you'll need to collect the register value if you are filtering on this tag
I removed your 2nd part and went straight to the 3rd.
A similar config is working for me for a similar problem.