Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add android sdk module #9236

Merged
merged 48 commits into from
Dec 20, 2024
Merged
Show file tree
Hide file tree
Changes from 43 commits
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
215467f
adds simple implementation of adding and removing android sdk packages
shamilovstas Nov 26, 2024
2526420
adds package update
shamilovstas Nov 27, 2024
ec674cd
adds simple installed packages parsing
shamilovstas Nov 28, 2024
ca3d11a
moves parsing logic to a separate class
shamilovstas Dec 3, 2024
7349127
adds absent state for sdkmanager packages and setup for tests
shamilovstas Dec 3, 2024
01c3674
adds output for installing and removing packages
shamilovstas Dec 3, 2024
d7f6451
removes version from Package object since it is not possible to speci…
shamilovstas Dec 4, 2024
0434249
adds 'latest' state
shamilovstas Dec 4, 2024
3edfac2
adds tests
shamilovstas Dec 5, 2024
ba7f1bf
fixes crash when sdkmanager is invoked from python with LC_ALL=C
shamilovstas Dec 6, 2024
de6f740
fixes latest state
shamilovstas Dec 6, 2024
97b07ee
adds sdk_root parameter
shamilovstas Dec 6, 2024
b41eb57
adds channel parameter
shamilovstas Dec 7, 2024
6d756e2
simplifies regexps, removes unused named groups
shamilovstas Dec 7, 2024
4441be7
minor refactoring of sdkmanager parsing
shamilovstas Dec 7, 2024
100a91f
adds java dependency variable for different distributions
shamilovstas Dec 7, 2024
fdef9c4
adds RETURN documentation
shamilovstas Dec 7, 2024
9b3d444
adds check for nonexisting package
shamilovstas Dec 7, 2024
ece03d1
adds check for non-accepted licenses
shamilovstas Dec 7, 2024
e8ef594
removes excessive methods from sdkmanager
shamilovstas Dec 9, 2024
1ac1ad5
removes unused 'update' module parameter, packages may be updated usi…
shamilovstas Dec 9, 2024
22b317f
minor refactoring
shamilovstas Dec 9, 2024
e499391
adds EXAMPLES doc section
shamilovstas Dec 9, 2024
7e53827
adds DOCUMENTATION section and license headers
shamilovstas Dec 9, 2024
e5a6d02
fixes formatting issues
shamilovstas Dec 10, 2024
003bd1e
removes diff_params
shamilovstas Dec 10, 2024
20eb1aa
adds maintainer
shamilovstas Dec 10, 2024
3358ac9
fixes sanity check issues in sdkmanager
shamilovstas Dec 10, 2024
14f0d6e
adds java dependency for macos and moves some tests to a separate Fre…
shamilovstas Dec 11, 2024
1ad1355
fixes dependencies setup for OSX
shamilovstas Dec 11, 2024
eb10db3
fixes dependencies setup for OSX (2)
shamilovstas Dec 11, 2024
7e1af46
fixes dependencies setup for OSX (3)
shamilovstas Dec 11, 2024
baa5f65
Apply minor suggestions from code review
shamilovstas Dec 12, 2024
d367bd2
applies code review suggestions
shamilovstas Dec 12, 2024
619f28d
changes force_lang from C.UTF-8 to auto in sdkmanager (as per discuss…
shamilovstas Dec 12, 2024
751ec9e
Revert "changes force_lang from C.UTF-8 to auto in sdkmanager (as per…
shamilovstas Dec 12, 2024
2089a5b
fixes some more comments from review
shamilovstas Dec 12, 2024
8008062
minor sanity issue fix
shamilovstas Dec 12, 2024
165f04e
uses the 'changed' test instead of checking the 'changed' attribute
shamilovstas Dec 16, 2024
c5474f2
adds 'accept_licenses' parameter. Installation is now performed indep…
shamilovstas Dec 16, 2024
83dd52e
removes "Accept licenses" task from examples
shamilovstas Dec 16, 2024
ffd170d
fixes docs sanity issues
shamilovstas Dec 16, 2024
bf10fe0
applies minor suggestions from code review
shamilovstas Dec 16, 2024
1878cd0
fixes regexps. The previous version didn't match versions like "32.1.…
shamilovstas Dec 17, 2024
043fc16
renamed sdkmanager.py to android_sdkmanager.py
shamilovstas Dec 17, 2024
7be3b3c
applies minor suggestions from code review
shamilovstas Dec 17, 2024
9033695
updates BOTMETA
shamilovstas Dec 17, 2024
2159c01
reordered BOTMETA
shamilovstas Dec 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/BOTMETA.yml
Original file line number Diff line number Diff line change
Expand Up @@ -375,6 +375,8 @@ files:
maintainers: $team_redfish
$module_utils/remote_management/lxca/common.py:
maintainers: navalkp prabhosa
$module_utils/sdkmanager.py:
maintainers: shamilovstas
felixfontein marked this conversation as resolved.
Show resolved Hide resolved
$module_utils/scaleway.py:
labels: cloud scaleway
maintainers: $team_scaleway
Expand Down Expand Up @@ -420,6 +422,8 @@ files:
ignore: DavidWittman jiuka
labels: alternatives
maintainers: mulby
$modules/android_sdk.py:
maintainers: shamilovstas
$modules/ansible_galaxy_install.py:
maintainers: russoz
$modules/apache2_mod_proxy.py:
Expand Down
161 changes: 161 additions & 0 deletions plugins/module_utils/sdkmanager.py
felixfontein marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
# -*- coding: utf-8 -*-

# Copyright (c) 2024, Stanislav Shamilov <[email protected]>
# GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
# SPDX-License-Identifier: GPL-3.0-or-later

from __future__ import absolute_import, division, print_function

__metaclass__ = type

import re

from ansible_collections.community.general.plugins.module_utils.cmd_runner import CmdRunner, cmd_runner_fmt

__state_map = {
"present": "--install",
"absent": "--uninstall"
}

# sdkmanager --help 2>&1 | grep -A 2 -- --channel
__channel_map = {
"stable": 0,
"beta": 1,
"dev": 2,
"canary": 3
}


def __map_channel(channel_name):
if channel_name not in __channel_map:
raise ValueError("Unknown channel name '%s'" % channel_name)
return __channel_map[channel_name]


def sdkmanager_runner(module, **kwargs):
return CmdRunner(
module,
command='sdkmanager',
arg_formats=dict(
state=cmd_runner_fmt.as_map(__state_map),
name=cmd_runner_fmt.as_list(),
installed=cmd_runner_fmt.as_fixed("--list_installed"),
list=cmd_runner_fmt.as_fixed('--list'),
newer=cmd_runner_fmt.as_fixed("--newer"),
sdk_root=cmd_runner_fmt.as_opt_eq_val("--sdk_root"),
channel=cmd_runner_fmt.as_func(lambda x: ["{0}={1}".format("--channel", __map_channel(x))])
),
force_lang="C.UTF-8", # Without this, sdkmanager binary crashes
russoz marked this conversation as resolved.
Show resolved Hide resolved
**kwargs
)


class Package:
def __init__(self, name):
self.name = name

def __hash__(self):
return hash(self.name)

def __ne__(self, other):
if not isinstance(other, Package):
return True
return self.name != other.name

felixfontein marked this conversation as resolved.
Show resolved Hide resolved
def __eq__(self, other):
if not isinstance(other, Package):
return False

return self.name == other.name


class SdkManagerException(Exception):
pass


class AndroidSdkManager(object):
_RE_INSTALLED_PACKAGES_HEADER = re.compile(r'^Installed packages:$')
_RE_UPDATABLE_PACKAGES_HEADER = re.compile(r'^Available Updates:$')

# Example: ' platform-tools | 27.0.0 | Android SDK Platform-Tools 27 | platform-tools '
_RE_INSTALLED_PACKAGE = re.compile(r'^\s*(?P<name>\S+)\s*\|\s*\S+\s*\|\s*.+\s*\|\s*(\S+)\s*$')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, from these examples, the only two possibilities that are not covered by the suggested regexp (the ones with rc1 and rc2) are also not covered by the existing regexp: |\s*\S+\s*\| will not match | 32.1.0 rc1 | (because the \S+ will not match the whitespace before rc1).

The one thing that is the delimiter here is the | character, so maybe it would be better of as:

    _RE_INSTALLED_PACKAGE = re.compile(r'^\s*(?P<name>\S+)\s*\|\s*[0-9][^|]*\b\s*\|\s*.+\s*\|\s*(\S+)\s*$')

That forces the version to begin with a digit, then anything-but-the-vertical-bar until the space and vertical bar. The \b is used to make sure the "anything-but-the-vertical-bar" part does not include the whitespaces before the bar.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As to your question of whether it makes sense to cover all the cases with the regexp, I would answer that IMHO it makes more sense to cover all the search cases with a search tool (regexp) then to cover only part of the search with it and create another search mechanism later on. Just my $0.02 ;-)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch, thanks


# Example: ' platform-tools | 27.0.0 | 35.0.2'
_RE_UPDATABLE_PACKAGE = re.compile(r'^\s*(?P<name>\S+)\s*\|\s*\S+\s*\|\s*\S+\s*$')
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You would need to change this regexp to search for numbers as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment above


_RE_UNKNOWN_PACKAGE = re.compile(r'^Warning: Failed to find package \'(?P<package>\S+)\'\s*$')
_RE_ACCEPT_LICENSE = re.compile(r'^The following packages can not be installed since their licenses or those of '
r'the packages they depend on were not accepted')

def __init__(self, module):
self.runner = sdkmanager_runner(module)

def get_installed_packages(self):
with self.runner('installed sdk_root channel') as ctx:
rc, stdout, stderr = ctx.run()
return self._parse_packages(stdout, self._RE_INSTALLED_PACKAGES_HEADER, self._RE_INSTALLED_PACKAGE)

def get_updatable_packages(self):
with self.runner('list newer sdk_root channel') as ctx:
rc, stdout, stderr = ctx.run()
return self._parse_packages(stdout, self._RE_UPDATABLE_PACKAGES_HEADER, self._RE_UPDATABLE_PACKAGE)

def apply_packages_changes(self, packages, accept_licenses=False):
""" Install or delete packages, depending on the `module.vars.state` parameter """
if len(packages) == 0:
return 0, '', ''

if accept_licenses:
license_prompt_answer = 'y'
else:
license_prompt_answer = 'N'
for package in packages:
with self.runner('state name sdk_root channel', data=license_prompt_answer) as ctx:
rc, stdout, stderr = ctx.run(name=package.name)

for line in stdout.splitlines():
if self._RE_ACCEPT_LICENSE.match(line):
raise SdkManagerException("Licenses for some packages were not accepted")

if rc != 0:
self._try_parse_stderr(stderr)
return rc, stdout, stderr
return 0, '', ''

def _try_parse_stderr(self, stderr):
data = stderr.splitlines()
for line in data:
unknown_package_regex = self._RE_UNKNOWN_PACKAGE.match(line)
if unknown_package_regex:
package = unknown_package_regex.group('package')
raise SdkManagerException("Unknown package %s" % package)

@staticmethod
def _parse_packages(stdout, header_regexp, row_regexp):
data = stdout.splitlines()

updatable_section_found = False
i = 0
lines_count = len(data)
packages = set()

while i < lines_count:
if not updatable_section_found:
updatable_section_found = header_regexp.match(data[i])
if updatable_section_found:
# Table has the following structure. Once header is found, 2 lines need to be skipped
#
# Available Updates: <--- we are here
# ID | Installed | Available
# ------- | ------- | -------
# platform-tools | 27.0.0 | 35.0.2 <--- skip to here
i += 3 # skip table header
else:
i += 1 # just iterate next until we find the section's header
continue
else:
p = row_regexp.match(data[i])
if p:
packages.add(Package(p.group('name')))
i += 1
return packages
213 changes: 213 additions & 0 deletions plugins/modules/android_sdk.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,213 @@
#!/usr/bin/python
# -*- coding: utf-8 -*-

# Copyright (c) 2024, Stanislav Shamilov <[email protected]>
# GNU General Public License v3.0+ (see LICENSES/GPL-3.0-or-later.txt or https://www.gnu.org/licenses/gpl-3.0.txt)
# SPDX-License-Identifier: GPL-3.0-or-later

from __future__ import absolute_import, division, print_function

__metaclass__ = type

DOCUMENTATION = r'''
---
module: android_sdk
short_description: Manages Android SDK packages
description:
- Manages Android SDK packages.
- Allows installation from different channels (stable, beta, dev, canary).
- Allows installation of packages to a non-default SDK root directory.
author: Stanislav Shamilov (@shamilovstas)
extends_documentation_fragment:
- community.general.attributes
attributes:
check_mode:
support: full
diff_mode:
support: none
version_added: 10.2.0
options:
accept_licenses:
description:
- If this is set to B(true), the module will try to accept license prompts generated by C(sdkmanager) during
package installation. Otherwise, every license prompt will be rejected.
type: bool
default: false
name:
description:
- A name of an Android SDK package (for instance, V(build-tools;34.0.0)).
aliases: ['package', 'pkg']
type: list
elements: str
state:
description:
- Indicates the desired package(s) state.
- V(present) ensures that package(s) is/are present.
- V(absent) ensures that package(s) is/are absent.
- V(latest) ensures that package(s) is/are installed and updated to the latest version(s).
choices: ['present', 'absent', 'latest']
default: present
type: str
sdk_root:
description:
- Provides path for an alternative directory to install Android SDK packages to. By default, all packages
are installed to the directory where C(sdkmanager) is installed.
type: path
channel:
description:
- Indicates what channel must C(sdkmanager) use for installation of packages.
choices: ['stable', 'beta', 'dev', 'canary']
default: stable
type: str
requirements:
- C(java) >= 17
- C(sdkmanager) Command line tool for installing Android SDK packages.
notes:
- For some of the packages installed by C(sdkmanager) is it necessary to accept licenses. Usually it is done through
command line prompt in a form of a Y/N question when a licensed package is requested to be installed. If there are
several packages requested for installation and at least two of them belong to different licenses, the C(sdkmanager)
tool will prompt for these licenses in a loop.
In order to install packages, the module must be able to answer these license prompts. Currently, it is only
possible to answer one license prompt at a time, meaning that instead of installing multiple packages as a single
invocation of the C(sdkmanager --install) command, it will be done by executing the command independently for each
package. This makes sure that at most only one license prompt will need to be answered.
At the time of writing this module, a C(sdkmanager)'s package may belong to at most one license type that needs to
be accepted. However, if this is changes in the future, the module may hang as there might be more prompts generated
by the C(sdkmanager) tool which the module won't be able to answer. If this is the case, file an issue and in the
shamilovstas marked this conversation as resolved.
Show resolved Hide resolved
meantime, consider accepting all the licenses in advance, as it is described in the C(sdkmanager)
L(documentation,https://developer.android.com/tools/sdkmanager#accept-licenses), for instance, using the
L(ansible.builtin.command module,https://docs.ansible.com/ansible/latest/collections/ansible/builtin/command_module.html)
shamilovstas marked this conversation as resolved.
Show resolved Hide resolved
seealso:
- name: sdkmanager tool documentation
description: Detailed information of how to install and use sdkmanager command line tool.
link: https://developer.android.com/tools/sdkmanager
'''

EXAMPLES = r'''
- name: Install build-tools;34.0.0
community.general.android_sdk:
name: build-tools;34.0.0
accept_licenses: true
state: present

- name: Install build-tools;34.0.0 and platform-tools
community.general.android_sdk:
name:
- build-tools;34.0.0
- platform-tools
accept_licenses: true
state: present

- name: Delete build-tools;34.0.0
community.general.android_sdk:
name: build-tools;34.0.0
state: absent

- name: Install platform-tools or update if installed
community.general.android_sdk:
name: platform-tools
accept_licenses: true
state: latest

- name: Install build-tools;34.0.0 to a different SDK root
community.general.android_sdk:
name: build-tools;34.0.0
accept_licenses: true
state: present
sdk_root: "/path/to/new/root"

- name: Install a package from another channel
community.general.android_sdk:
name: some-package-present-in-canary-channel
accept_licenses: true
state: present
channel: canary
'''

RETURN = r'''
installed:
description: a list of packages that have been installed
returned: when packages have changed
type: list
sample: ['build-tools;34.0.0', 'platform-tools']

removed:
description: a list of packages that have been removed
returned: when packages have changed
type: list
sample: ['build-tools;34.0.0', 'platform-tools']
'''

from ansible_collections.community.general.plugins.module_utils.mh.module_helper import StateModuleHelper
from ansible_collections.community.general.plugins.module_utils.sdkmanager import Package, AndroidSdkManager


class AndroidSdk(StateModuleHelper):
module = dict(
argument_spec=dict(
state=dict(type='str', default='present', choices=['present', 'absent', 'latest']),
package=dict(type='list', elements='str', aliases=['pkg', 'name']),
sdk_root=dict(type='path'),
channel=dict(type='str', default='stable', choices=['stable', 'beta', 'dev', 'canary']),
accept_licenses=dict(type='bool', default=False)
),
supports_check_mode=True
)
use_old_vardict = False

def __init_module__(self):
self.sdkmanager = AndroidSdkManager(self.module)
self.vars.set('installed', [], change=True)
self.vars.set('removed', [], change=True)

def _parse_packages(self):
arg_pkgs = set(self.vars.package)
if len(arg_pkgs) < len(self.vars.package):
self.do_raise("Packages may not repeat")
return set([Package(p) for p in arg_pkgs])

def state_present(self):
packages = self._parse_packages()
installed = self.sdkmanager.get_installed_packages()
pending_installation = packages.difference(installed)

self.vars.installed = AndroidSdk._map_packages_to_names(pending_installation)
if not self.check_mode:
rc, stdout, stderr = self.sdkmanager.apply_packages_changes(pending_installation, self.vars.accept_licenses)
if rc != 0:
self.do_raise("Could not install packages: %s" % stderr)

def state_absent(self):
packages = self._parse_packages()
installed = self.sdkmanager.get_installed_packages()
to_be_deleted = packages.intersection(installed)
self.vars.removed = AndroidSdk._map_packages_to_names(to_be_deleted)
if not self.check_mode:
rc, stdout, stderr = self.sdkmanager.apply_packages_changes(to_be_deleted)
if rc != 0:
self.do_raise("Could not uninstall packages: %s" % stderr)
russoz marked this conversation as resolved.
Show resolved Hide resolved

def state_latest(self):
packages = self._parse_packages()
installed = self.sdkmanager.get_installed_packages()
updatable = self.sdkmanager.get_updatable_packages()
not_installed = packages.difference(installed)
to_be_installed = not_installed.union(updatable)
self.vars.installed = AndroidSdk._map_packages_to_names(to_be_installed)

if not self.check_mode:
rc, stdout, stderr = self.sdkmanager.apply_packages_changes(to_be_installed, self.vars.accept_licenses)
if rc != 0:
self.do_raise("Could not install packages: %s" % stderr)

@staticmethod
def _map_packages_to_names(packages):
return [x.name for x in packages]


def main():
AndroidSdk.execute()


if __name__ == '__main__':
main()
Loading
Loading