Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

listing all buildtest config profiles list #1552

Merged
merged 46 commits into from
Aug 7, 2023
Merged
Show file tree
Hide file tree
Changes from 39 commits
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
fc06c65
Add command menu in argparse
kingxleyy Jul 12, 2023
48cbfc3
Resolved conflict to Add command menu in argparse
kingxleyy Jul 12, 2023
2c24288
Added removed parenthesis
kingxleyy Jul 12, 2023
7d89100
removed unwanted lines
kingxleyy Jul 12, 2023
57f4a81
Add command menu in argparse, implementation,entry in buildtest help…
kingxleyy Jul 13, 2023
359a97e
fixing errors in previous comments
kingxleyy Jul 13, 2023
665a038
fixing conflicts
kingxleyy Jul 13, 2023
6d6d429
re-doing the pull request
kingxleyy Jul 13, 2023
89acabf
Merge remote-tracking branch 'origin/kingsley_buildtest' into kingsle…
kingxleyy Jul 13, 2023
a4041e2
re-doing the pull request
kingxleyy Jul 13, 2023
28a3709
adding options in bash_completion.sh
kingxleyy Jul 14, 2023
f222c06
Merge branch 'buildtesters:devel' into kingsley_buildtest
kingxleyy Jul 16, 2023
8ac4f7b
modifying code for profile, chaning profiles to profile
kingxleyy Jul 16, 2023
5d36ff1
Merge remote-tracking branch 'origin/kingsley_buildtest' into kingsle…
kingxleyy Jul 16, 2023
ad55caf
changing profiles to profile in test_config
kingxleyy Jul 16, 2023
6add5a8
fixing errors in config
kingxleyy Jul 16, 2023
ef32b0d
fixing errors in _init_
kingxleyy Jul 16, 2023
a120fdf
fixing errors in line 27 of config.py
kingxleyy Jul 17, 2023
8087ec9
resolving conflicts in bash completion
kingxleyy Jul 17, 2023
7b05d5d
fixing erros
kingxleyy Jul 20, 2023
005abf3
fixing errors in init_py
kingxleyy Jul 20, 2023
a87bb6b
Merge branch 'devel' into kingsley_buildtest
kingxleyy Jul 20, 2023
c89f26e
fixing errors in config, compliers and init py
kingxleyy Jul 25, 2023
4c89b43
Merge branch 'buildtesters:devel' into kingsley_buildtest
kingxleyy Jul 25, 2023
4dfd028
Merge remote-tracking branch 'origin/kingsley_buildtest' into kingsle…
kingxleyy Jul 25, 2023
7fc1e4b
fixing errors in config, init py
kingxleyy Jul 27, 2023
3edc5ca
adding arguments
kingxleyy Jul 27, 2023
620f1cb
fix issue with argparse and all the bugs related to building the parser
shahzebsiddiqui Jul 27, 2023
49f6e50
fix bash completion script with the appropriate commands
shahzebsiddiqui Jul 27, 2023
bb8a153
add --theme option for 'buildtest config profiles'
shahzebsiddiqui Jul 27, 2023
7f7f5e2
add command 'buildtest config profiles list' and add the options --th…
shahzebsiddiqui Jul 27, 2023
7be94c2
fix bash completion script to add support for 'buildtest config profi…
shahzebsiddiqui Jul 27, 2023
0fe67d9
refactored the method and renamed it to list_profiles. This method ca…
shahzebsiddiqui Jul 27, 2023
c541e5c
running isort and black since code wasnt formatted properly
shahzebsiddiqui Jul 27, 2023
29cd1ec
adding table in show.py
kingxleyy Jul 27, 2023
84d37fe
fixing error config yml
kingxleyy Jul 27, 2023
4aeca52
documentation
kingxleyy Aug 1, 2023
91e7a21
documentation
kingxleyy Aug 3, 2023
e02f16e
Merge branch 'buildtesters:devel' into kingsley_buildtest
kingxleyy Aug 3, 2023
ba8a161
profile to profiles and added documentation
kingxleyy Aug 3, 2023
c4e2e01
profile to profiles and added documentation 2
kingxleyy Aug 3, 2023
30cb19b
removing -t in bash
kingxleyy Aug 3, 2023
278be9c
removing -t in bash 2
kingxleyy Aug 3, 2023
b1a3fe0
fixing config. yml
kingxleyy Aug 6, 2023
c828d11
applying black
shahzebsiddiqui Aug 7, 2023
4701ad1
removing profiles section from config file
shahzebsiddiqui Aug 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 17 additions & 2 deletions bash_completion.sh
Original file line number Diff line number Diff line change
Expand Up @@ -237,7 +237,10 @@ _buildtest ()
;;

config|cg)
local cmds="-h --help co compilers e edit ex executors p path systems val validate v view"


local cmds="-h --help co compilers e edit ex executors p path profile systems validate v view"
kingxleyy marked this conversation as resolved.
Show resolved Hide resolved


COMPREPLY=( $( compgen -W "${cmds}" -- $cur ) )
# handle completion logic for 'buildtest config <subcommand>' based on subcommands
Expand Down Expand Up @@ -269,6 +272,19 @@ _buildtest ()
return
esac
;;
profile)
kingxleyy marked this conversation as resolved.
Show resolved Hide resolved
local opts="--help -h list"
COMPREPLY=( $( compgen -W "${opts}" -- $cur ) )
if [[ "${prev}" == "list" ]]; then
local opts="--help --theme --yaml -h -t -y"
kingxleyy marked this conversation as resolved.
Show resolved Hide resolved
COMPREPLY=( $( compgen -W "${opts}" -- $cur ) )
fi

case "${prev}" in --theme|-t)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you remove the -t option this will be removed in #1576

COMPREPLY=( $( compgen -W "$(_avail_color_themes)" -- $cur ) )
return
esac
;;
esac
;;
inspect|it)
Expand Down Expand Up @@ -389,7 +405,6 @@ _buildtest ()
;;
validate|val)
local opts="--buildspec --exclude --executor --tag -b -e -t -x "

COMPREPLY=( $( compgen -W "${opts}" -- $cur ) )
# auto completion for 'buildtest buildspec validate' options
if [[ "${prev}" == "-b" ]] || [[ "${prev}" == "--buildspec" ]] || [[ "${prev}" == "-x" ]] || [[ "${prev}" == "--exclude" ]]; then
Expand Down
18 changes: 18 additions & 0 deletions buildtest/cli/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -988,6 +988,24 @@ def config_menu(subparsers, parent_parser):
compilers = subparsers_config.add_parser(
"compilers", aliases=["co"], help="Search compilers"
)
# buildtest config profile
profile = subparsers_config.add_parser(
"profile", help="Query profile from buildtest configuration"
kingxleyy marked this conversation as resolved.
Show resolved Hide resolved
)
subparsers_profile = profile.add_subparsers(
description="Query information about buildtest profiles",
dest="profile",
kingxleyy marked this conversation as resolved.
Show resolved Hide resolved
metavar="",
)

subparsers_profile_list = subparsers_profile.add_parser(
"list", help="List all profiles", parents=[parent_parser["theme"]]
)

# buildtest config profile list options
subparsers_profile_list.add_argument(
"-y", "--yaml", action="store_true", help="List Profile details in YAML Format"
)

subparsers_config.add_parser(
"edit", aliases=["e"], help="Open configuration file in editor"
Expand Down
30 changes: 30 additions & 0 deletions buildtest/cli/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,9 +20,14 @@ def config_cmd(args, configuration, editor, system):
configuration (buildtest.config.SiteConfiguration): An instance of SiteConfiguration class
system (buildtest.system.BuildTestSystem): An instance of BuildTestSystem class
"""

if args.config in ["view", "v"]:
view_configuration(configuration, theme=args.theme, pager=args.pager)

elif args.config in ["profile"]:
kingxleyy marked this conversation as resolved.
Show resolved Hide resolved
if args.profile == "list":
kingxleyy marked this conversation as resolved.
Show resolved Hide resolved
list_profiles(configuration, theme=args.theme, print_yaml=args.yaml)

elif args.config in ["executors", "ex"]:
buildexecutor = BuildExecutor(configuration)
view_executors(
Expand Down Expand Up @@ -154,6 +159,31 @@ def view_configuration(configuration, theme=None, pager=None):
console.print(syntax)


def list_profiles(configuration, theme=None, print_yaml=None):
"""Display the list of profile for buildtest configuration file.This implements command ``buildtest config profiles list``
kingxleyy marked this conversation as resolved.
Show resolved Hide resolved

Args:
configuration (buildtest.config.SiteConfiguration): An instance of SiteConfiguration class
theme (str, optional): Color theme to choose. This is the Pygments style (https://pygments.org/docs/styles/#getting-a-list-of-available-styles) which is specified by ``--theme`` option
print_yaml (bool, optional): Display profiles in yaml format. This is specified by ``--yaml`` option
"""
shahzebsiddiqui marked this conversation as resolved.
Show resolved Hide resolved

if not configuration.target_config.get("profiles"):
sys.exit("No profiles found in configuration file")

if print_yaml:
profile_configuration = yaml.dump(
configuration.target_config["profiles"], indent=2
)
syntax = Syntax(profile_configuration, "yaml", theme=theme or "monokai")
console.print(syntax)
return

# print profiles as raw text
for profile_name in configuration.target_config["profiles"].keys():
print(profile_name)


def view_executors(
configuration,
buildexecutor,
Expand Down
4 changes: 4 additions & 0 deletions buildtest/cli/show.py
Original file line number Diff line number Diff line change
Expand Up @@ -316,6 +316,10 @@ def print_config_show():
"buildtest config compilers test",
"Test each compiler instance by performing module load test",
)
table.add_row(
"buildtest config profiles list",
"Listing all profiles from configuration file",
)
console.print(table)


Expand Down
71 changes: 15 additions & 56 deletions buildtest/settings/config.yml
Original file line number Diff line number Diff line change
@@ -1,45 +1,23 @@
system:
generic:
# specify list of hostnames where buildtest can run for given system record
hostnames: [".*"]

# system description
kingxleyy marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

please dont commit buildtest/settings/config.yml this was a result of running the regression test which autoupdates the file. Don't do git add . or something. Just copy the content from devel branch and paste it in that should avoid showing this commit diff for this file

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

noted

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kingxleyy i still see this file ``buildtest/settings/config.yml` in commit diff

hostnames:
- .*
description: Generic System
# specify module system used at your site (environment-modules, lmod)
moduletool: N/A

# specify test timeout duration in number of seconds
# timeout: 60

# pool size for parallel processing using multiprocessing.Pool. Pool Size must be 1 or higher. The pool size is calculated as follows: poolsize = min(poolsize, os.cpu_count())
poolsize: 1

# determine whether to enable pagination
pager: False

pager: false
buildspecs:
# whether to rebuild cache file automatically when running `buildtest buildspec find`
rebuild: False
# limit number of records to display when running `buildtest buildspec find`
rebuild: false
count: 15
# format fields to display when running `buildtest buildspec find`, By default we will show name,description
format: "name,description"
# enable terse mode
terse: False
# specify list of directories to search for buildspecs when building cache
#root: [ $BUILDTEST_ROOT/examples, /tmp/buildspecs ]
format: name,description
terse: false
report:
count: 25
#enable terse mode for report
terse: False
format: "name,id,state,runtime,returncode"
# show the latest for every test
latest: True
# show the oldest for every test
oldest: False

terse: false
format: name,id,state,runtime,returncode
latest: true
oldest: false
executors:
# define local executors for running jobs locally
local:
bash:
description: submit jobs on local machine using bash shell
Expand All @@ -53,39 +31,20 @@ system:
zsh:
description: submit jobs on local machine using zsh shell
shell: zsh
# compiler block
compilers:
# a list of directories to search for modules
# modulepath: ['/tmp', '/var/']

# regular expression to search for compilers based on module pattern. Used with 'buildtest config compilers find' to generate compiler instance
# find:
# gcc: "^(gcc)"
# intel: "^(intel)"
# cray: "^(craype)"
# pgi: "^(pgi)"
# cuda: ^(cuda)"
# clang: "^(clang)"

# declare compiler instance which can be site-specific. You can let 'buildtest config compilers find' generate compiler section
compiler:
gcc:
builtin_gcc:
cc: gcc
fc: gfortran
cxx: g++

# location of log directory
# logdir: /tmp/

# specify location where buildtest will write tests
# testdir: /tmp

# specify one or more directory where buildtest should load buildspecs
# buildspec_roots: []

cdash:
url: https://my.cdash.org/
project: buildtest
site: generic
buildname: tutorials
profiles:
pass:
tags:
- pass
testdir: /Users/kingxley/buildtest/var/tests
3 changes: 2 additions & 1 deletion docs/configuring_buildtest/overview.rst
Original file line number Diff line number Diff line change
Expand Up @@ -725,4 +725,5 @@ is an object and attributes ``tags``, ``maintainers``, ``type`` correspond to th
- script

We have added additional checks in the JSON schema for valid values for each type, for instance if you specify an invalid value for ``type`` field
which is used to filter buildspecs by the ``type`` field, then you will get an invalid configuration file.
which is used to filter buildspecs by the ``type`` field, then you will get an invalid configuration file.

13 changes: 13 additions & 0 deletions tests/cli/test_config.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@
import pytest

from buildtest.cli.config import (
list_profiles,
validate_config,
view_configuration,
view_executors,
Expand Down Expand Up @@ -66,6 +67,18 @@ def test_config_path():
view_path(configuration)


@pytest.mark.cli
def test_config_profile():
# buildtest config profiles
list_profiles(configuration)

# buildtest config profiles --theme emacs --yaml
list_profiles(configuration, theme="emacs", print_yaml=True)

# buildtest config profiles --yaml
list_profiles(configuration, print_yaml=True)


@pytest.mark.cli
def test_config_executors():
buildexecutor = BuildExecutor(configuration)
Expand Down