improve custom build install playbook

[tarzanek]

scylla-ansible-roles/example-playbooks/rolling_ops/rolling_custom_build_install.yml

Line 118 in cb6a054

rescue:

    - name: Send a SIGKILL to Scylla PID
      ansible.builtin.shell: pkill -SIGKILL -x scylla
      register: scylla_server_kill_pid_status
      retries: "{{ scylla_ops_pid_kill_retries }}"
      delay: "{{ scylla_ops_pid_kill_delay }}"
      until: scylla_server_kill_pid_status.rc == 1
      failed_when: scylla_kill_pid.rc != 2
      changed_when: scylla_server_kill_pid_status.rc == 0
      become: true

better to replace with

    - name: Send a SIGKILL to Scylla PID
      ansible.builtin.shell: pkill -SIGKILL -x scylla
      register: scylla_server_kill_pid_status
      retries: "{{ scylla_ops_pid_kill_retries }}"
      delay: "{{ scylla_ops_pid_kill_delay }}"
      until: scylla_server_kill_pid_status.rc == 1
      failed_when: scylla_server_kill_pid_status.rc in [2, 3]
      changed_when: scylla_server_kill_pid_status.rc == 0
      become: true

because it can fail if it tries to kill process that is already dead

TASK [scylla_ops : Send a SIGKILL to Scylla PID] *******************************
task path: /app/roles/scylla_ops/tasks/state_scylla_stopped.yaml:38
Wednesday 02 October 2024  17:59:19 +0000 (0:00:05.240)       0:00:41.949 ***** 
Wednesday 02 October 2024  17:59:19 +0000 (0:00:05.239)       0:00:41.948 ***** 
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (20 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (19 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (18 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (17 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (16 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (15 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (14 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (13 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (12 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (11 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (10 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (9 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (8 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (7 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (6 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (5 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (4 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (3 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (2 retries left).
FAILED - RETRYING: [server-12687]: Send a SIGKILL to Scylla PID (1 retries left).
fatal: [server-12687]: FAILED! => changed=false 
  attempts: 20
  cmd: kill -9 $(pidof scylla)
  delta: '0:00:00.004872'
  end: '2024-10-02 18:01:11.548872'
  failed_when_result: true
  msg: ''
  rc: 0
  start: '2024-10-02 18:01:11.544000'
  stderr: ''
  stderr_lines: <omitted>
  stdout: ''
  stdout_lines: <omitted>

[tarzanek]

cc @d-helios