diff --git a/doc/ref/file_server/dynamic-modules.rst b/doc/ref/file_server/dynamic-modules.rst deleted file mode 100644 index 17bf149685fd..000000000000 --- a/doc/ref/file_server/dynamic-modules.rst +++ /dev/null @@ -1,57 +0,0 @@ -.. _module-sync: -.. _dynamic-module-distribution: - -=========================== -Dynamic Module Distribution -=========================== - -.. versionadded:: 0.9.5 - -Custom Salt execution, state, and other modules can be distributed to Salt -minions using the Salt file server. - -Under the root of any environment defined via the :conf_master:`file_roots` -option on the master server directories corresponding to the type of module can -be used. - -The directories are prepended with an underscore: - -- :file:`_beacons` -- :file:`_clouds` -- :file:`_engines` -- :file:`_grains` -- :file:`_modules` -- :file:`_output` -- :file:`_proxy` -- :file:`_renderers` -- :file:`_returners` -- :file:`_states` -- :file:`_tops` -- :file:`_utils` - -The contents of these directories need to be synced over to the minions after -Python modules have been created in them. There are a number of ways to sync -the modules. - -Sync Via States -=============== - -The minion configuration contains an option ``autoload_dynamic_modules`` -which defaults to ``True``. This option makes the state system refresh all -dynamic modules when states are run. To disable this behavior set -:conf_minion:`autoload_dynamic_modules` to ``False`` in the minion config. - -When dynamic modules are autoloaded via states, modules only pertinent to -the environments matched in the master's top file are downloaded. - -This is important to remember, because modules can be manually loaded from -any specific environment that environment specific modules will be loaded -when a state run is executed. - -Sync Via the saltutil Module -============================ - -The saltutil module has a number of functions that can be used to sync all -or specific dynamic modules. The saltutil module function ``saltutil.sync_all`` -will sync all module types over to a minion. For more information see: -:mod:`salt.modules.saltutil` diff --git a/doc/topics/beacons/index.rst b/doc/topics/beacons/index.rst index bc9ba7a5355d..d6996834c13f 100644 --- a/doc/topics/beacons/index.rst +++ b/doc/topics/beacons/index.rst @@ -283,6 +283,8 @@ All beacons are configured using a similar process of enabling the beacon, writing a reactor SLS (and state SLS if needed), and mapping a beacon event to the reactor SLS. +.. _writing-beacons: + Writing Beacon Plugins ====================== @@ -353,5 +355,5 @@ new execution modules and functions to back specific beacons. Distributing Custom Beacons --------------------------- -Custom beacons can be distributed to minions using ``saltutil``, see -:ref:`Dynamic Module Distribution `. +Custom beacons can be distributed to minions via the standard methods, see +:ref:`Modular Systems `. diff --git a/doc/topics/development/dunder_dictionaries.rst b/doc/topics/development/dunder_dictionaries.rst deleted file mode 100644 index 64df937a1523..000000000000 --- a/doc/topics/development/dunder_dictionaries.rst +++ /dev/null @@ -1,119 +0,0 @@ -=================== -Dunder Dictionaries -=================== - -Salt provides several special "dunder" dictionaries as a convenience for Salt -development. These include ``__opts__``, ``__context__``, ``__salt__``, and -others. This document will describe each dictionary and detail where they exist -and what information and/or functionality they provide. - - -__opts__ --------- - -Available in -~~~~~~~~~~~~ - -- All loader modules - -The ``__opts__`` dictionary contains all of the options passed in the -configuration file for the master or minion. - -.. note:: - - In many places in salt, instead of pulling raw data from the __opts__ - dict, configuration data should be pulled from the salt `get` functions - such as config.get, aka - __salt__['config.get']('foo:bar') - The `get` functions also allow for dict traversal via the *:* delimiter. - Consider using get functions whenever using __opts__ or __pillar__ and - __grains__ (when using grains for configuration data) - -The configuration file data made available in the ``__opts__`` dictionary is the -configuration data relative to the running daemon. If the modules are loaded and -executed by the master, then the master configuration data is available, if the -modules are executed by the minion, then the minion configuration is -available. Any additional information passed into the respective configuration -files is made available - -__salt__ --------- - -Available in -~~~~~~~~~~~~ - -- Execution Modules -- State Modules -- Returners -- Runners - -``__salt__`` contains the execution module functions. This allows for all -functions to be called as they have been set up by the salt loader. - -.. code-block:: python - - __salt__['cmd.run']('fdisk -l') - __salt__['network.ip_addrs']() - -.. note:: - - When used in runners, ``__salt__`` references other runner modules, and not - execution modules. - -__grains__ ----------- - -Available in -~~~~~~~~~~~~ - -- Execution Modules -- State Modules -- Returners -- External Pillar - -The ``__grains__`` dictionary contains the grains data generated by the minion -that is currently being worked with. In execution modules, state modules and -returners this is the grains of the minion running the calls, when generating -the external pillar the ``__grains__`` is the grains data from the minion that -the pillar is being generated for. - -__pillar__ ------------ - -Available in -~~~~~~~~~~~~ - -- Execution Modules -- State Modules -- Returners - -The ``__pillar__`` dictionary contains the pillar for the respective minion. - -__context__ ------------ - -``__context__`` exists in state modules and execution modules. - -During a state run the ``__context__`` dictionary persists across all states -that are run and then is destroyed when the state ends. - -When running an execution module ``__context__`` persists across all module -executions until the modules are refreshed; such as when -:py:func:`saltutil.sync_all ` or -:py:func:`state.apply ` are executed. - -A great place to see how to use ``__context__`` is in the cp.py module in -salt/modules/cp.py. The fileclient authenticates with the master when it is -instantiated and then is used to copy files to the minion. Rather than create a -new fileclient for each file that is to be copied down, one instance of the -fileclient is instantiated in the ``__context__`` dictionary and is reused for -each file. Here is an example from salt/modules/cp.py: - -.. code-block:: python - - if not 'cp.fileclient' in __context__: - __context__['cp.fileclient'] = salt.fileclient.get_file_client(__opts__) - - -.. note:: Because __context__ may or may not have been destroyed, always be - sure to check for the existence of the key in __context__ and - generate the key before using it. diff --git a/doc/topics/development/index.rst b/doc/topics/development/index.rst index c4a341c0404b..65a7e6ee861f 100644 --- a/doc/topics/development/index.rst +++ b/doc/topics/development/index.rst @@ -7,6 +7,7 @@ Developing Salt :glob: * + modules/index extend/index tests/* raet/index diff --git a/doc/topics/development/modular_systems.rst b/doc/topics/development/modular_systems.rst deleted file mode 100644 index 6060ceea88cc..000000000000 --- a/doc/topics/development/modular_systems.rst +++ /dev/null @@ -1,170 +0,0 @@ -=============== -Modular Systems -=============== - -When first working with Salt, it is not always clear where all of the modular -components are and what they do. Salt comes loaded with more modular systems -than many users are aware of, making Salt very easy to extend in many places. - -The most commonly used modular systems are execution modules and states. But -the modular systems extend well beyond the more easily exposed components -and are often added to Salt to make the complete system more flexible. - -Execution Modules -================= - -Execution modules make up the core of the functionality used by Salt to -interact with client systems. The execution modules create the core system -management library used by all Salt systems, including states, which -interact with minion systems. - -Execution modules are completely open ended in their execution. They can -be used to do anything required on a minion, from installing packages to -detecting information about the system. The only restraint in execution -modules is that the defined functions always return a JSON serializable -object. - -For a list of all built in execution modules, click :ref:`here -` - -For information on writing execution modules, see :ref:`this page -`. - - -Interactive Debugging -===================== - -Sometimes debugging with ``print()`` and extra logs sprinkled everywhere is not -the best strategy. - -IPython is a helpful debug tool that has an interactive python environment -which can be embedded in python programs. - -First the system will require IPython to be installed. - -.. code-block:: bash - - # Debian - apt-get install ipython - - # Arch Linux - pacman -Syu ipython2 - - # RHEL/CentOS (via EPEL) - yum install python-ipython - - -Now, in the troubling python module, add the following line at a location where -the debugger should be started: - -.. code-block:: python - - test = 'test123' - import IPython; IPython.embed_kernel() - -After running a Salt command that hits that line, the following will show up in -the log file: - -.. code-block:: text - - [CRITICAL] To connect another client to this kernel, use: - [IPKernelApp] --existing kernel-31271.json - -Now on the system that invoked ``embed_kernel``, run the following command from -a shell: - -.. code-block:: bash - - # NOTE: use ipython2 instead of ipython for Arch Linux - ipython console --existing - -This provides a console that has access to all the vars and functions, and even -supports tab-completion. - -.. code-block:: python - - print(test) - test123 - -To exit IPython and continue running Salt, press ``Ctrl-d`` to logout. - - -State Modules -============= - -State modules are used to define the state interfaces used by Salt States. -These modules are restrictive in that they must follow a number of rules to -function properly. - -.. note:: - - State modules define the available routines in sls files. If calling - an execution module directly is desired, take a look at the `module` - state. - -Auth -==== - -The auth module system allows for external authentication routines to be easily -added into Salt. The `auth` function needs to be implemented to satisfy the -requirements of an auth module. Use the ``pam`` module as an example. - -Fileserver -========== - -The fileserver module system is used to create fileserver backends used by the -Salt Master. These modules need to implement the functions used in the -fileserver subsystem. Use the ``gitfs`` module as an example. - -Grains -====== - -Grain modules define extra routines to populate grains data. All defined -public functions will be executed and MUST return a Python dict object. The -dict keys will be added to the grains made available to the minion. - -Output -====== - -The output modules supply the outputter system with routines to display data -in the terminal. These modules are very simple and only require the `output` -function to execute. The default system outputter is the ``nested`` module. - -Pillar -====== - -Used to define optional external pillar systems. The pillar generated via -the filesystem pillar is passed into external pillars. This is commonly used -as a bridge to database data for pillar, but is also the backend to the libvirt -state used to generate and sign libvirt certificates on the fly. - -Renderers -========= - -Renderers are the system used to render sls files into salt highdata for the -state compiler. They can be as simple as the ``py`` renderer and as complex as -``stateconf`` and ``pydsl``. - -Returners -========= - -Returners are used to send data from minions to external sources, commonly -databases. A full returner will implement all routines to be supported as an -external job cache. Use the ``redis`` returner as an example. - -Runners -======= - -Runners are purely master-side execution sequences. - -Tops -==== - -Tops modules are used to convert external data sources into top file data for -the state system. - -Wheel -===== - -The wheel system is used to manage master side management routines. These -routines are primarily intended for the API to enable master configuration. diff --git a/doc/topics/development/modules/configuration.rst b/doc/topics/development/modules/configuration.rst new file mode 100644 index 000000000000..c794830f8027 --- /dev/null +++ b/doc/topics/development/modules/configuration.rst @@ -0,0 +1,25 @@ +===================== +Configuration Options +===================== + +A number of configuration options can affect the load process. This is a quick +list of them: + +* ``autoload_dynamic_modules`` (:conf_minion:`Minion `) +* ``cython_enable`` (:conf_minion:`Minion `, :conf_master:`Master `) +* ``disable_modules`` (:conf_minion:`Minion `) +* ``disable_returners`` (:conf_minion:`Minion `) +* ``enable_zip_modules`` (:conf_minion:`Minion `) +* ``extension_modules`` (:conf_master:`Master `) +* ``extmod_whitelist`` (:conf_minion:`Minion `, :conf_master:`Master `) +* ``extmod_blacklist`` (:conf_minion:`Minion `, :conf_master:`Master `) +* ``whitelist_modules`` (:conf_minion:`Minion `) +* ``grains_dirs`` (:conf_minion:`Minion `) +* ``module_dirs`` (:conf_minion:`Minion `, :conf_master:`Master `) +* ``outputter_dirs`` (:conf_minion:`Minion `, :conf_master:`Master `) +* ``providers`` (:conf_minion:`Minion `) +* ``render_dirs`` (:conf_minion:`Minion `) +* ``returner_dirs`` (:conf_minion:`Minion `) +* ``runner_dirs`` (:conf_master:`Master `) +* ``states_dirs`` (:conf_minion:`Minion `) +* ``utils_dirs`` (:conf_minion:`Minion `) diff --git a/doc/topics/development/modules/developing.rst b/doc/topics/development/modules/developing.rst new file mode 100644 index 000000000000..a01137d744a7 --- /dev/null +++ b/doc/topics/development/modules/developing.rst @@ -0,0 +1,237 @@ +====================== +Developing New Modules +====================== + +Interactive Debugging +===================== + +Sometimes debugging with ``print()`` and extra logs sprinkled everywhere is not +the best strategy. + +IPython is a helpful debug tool that has an interactive python environment +which can be embedded in python programs. + +First the system will require IPython to be installed. + +.. code-block:: bash + + # Debian + apt-get install ipython + + # Arch Linux + pacman -Syu ipython2 + + # RHEL/CentOS (via EPEL) + yum install python-ipython + + +Now, in the troubling python module, add the following line at a location where +the debugger should be started: + +.. code-block:: python + + test = 'test123' + import IPython; IPython.embed_kernel() + +After running a Salt command that hits that line, the following will show up in +the log file: + +.. code-block:: text + + [CRITICAL] To connect another client to this kernel, use: + [IPKernelApp] --existing kernel-31271.json + +Now on the system that invoked ``embed_kernel``, run the following command from +a shell: + +.. code-block:: bash + + # NOTE: use ipython2 instead of ipython for Arch Linux + ipython console --existing + +This provides a console that has access to all the vars and functions, and even +supports tab-completion. + +.. code-block:: python + + print(test) + test123 + +To exit IPython and continue running Salt, press ``Ctrl-d`` to logout. + +Special Module Contents +======================= + +These are things that may be defined by the module to influence various things. + +__virtual__ +----------- + +__virtual_aliases__ +------------------- + +__virtualname__ +--------------- + +__init__ +-------- + +Called before ``__virtual__()`` + +__proxyenabled__ +---------------- +grains and proxy modules + +__proxyenabled__ as a list containing the names of the proxy types that the module supports. + +__load__ +-------- + +__func_alias__ +-------------- + +__outputter__ +------------- + +.. _dunder-dictionaries: + +Dunder Dictionaries +=================== + +Salt provides several special "dunder" dictionaries as a convenience for Salt +development. These include ``__opts__``, ``__context__``, ``__salt__``, and +others. This document will describe each dictionary and detail where they exist +and what information and/or functionality they provide. + +The following dunder dictionaries are always defined, but may be empty + +* ``__context__`` +* ``__grains__`` +* ``__pillar__`` +* ``__opts__`` + + +__opts__ +-------- + +Defined in: All modules + +The ``__opts__`` dictionary contains all of the options passed in the +configuration file for the master or minion. + +.. note:: + + In many places in salt, instead of pulling raw data from the __opts__ + dict, configuration data should be pulled from the salt `get` functions + such as config.get, aka - ``__salt__['config.get']('foo:bar')`` + The `get` functions also allow for dict traversal via the *:* delimiter. + Consider using get functions whenever using ``__opts__`` or ``__pillar__`` + and ``__grains__`` (when using grains for configuration data) + +The configuration file data made available in the ``__opts__`` dictionary is the +configuration data relative to the running daemon. If the modules are loaded and +executed by the master, then the master configuration data is available, if the +modules are executed by the minion, then the minion configuration is +available. Any additional information passed into the respective configuration +files is made available + +__salt__ +-------- + +Defined in: Auth, Beacons, Engines, Execution, Executors, Outputters, Pillars, +Proxies, Renderers, Returners, Runners, SDB, SSH Wrappers, State, Thorium + +``__salt__`` contains the execution module functions. This allows for all +functions to be called as they have been set up by the salt loader. + +.. code-block:: python + + __salt__['cmd.run']('fdisk -l') + __salt__['network.ip_addrs']() + +.. note:: + + When used in runners or outputters, ``__salt__`` references other + runner/outputter modules, and not execution modules. + +__grains__ +---------- + +Filled in for: Execution, Pillar, Renderer, Returner, SSH Wrapper, State. + +The ``__grains__`` dictionary contains the grains data generated by the minion +that is currently being worked with. In execution modules, state modules and +returners this is the grains of the minion running the calls, when generating +the external pillar the ``__grains__`` is the grains data from the minion that +the pillar is being generated for. + +While ``__grains__`` is defined for every module, it's only filled in for some. + +__pillar__ +----------- + +Filled in for: Execution, Returner, SSH Wrapper, State + +The ``__pillar__`` dictionary contains the pillar for the respective minion. + +While ``__pillar__`` is defined for every module, it's only filled in for some. + +__context__ +----------- + +During a state run the ``__context__`` dictionary persists across all states +that are run and then is destroyed when the state ends. + +When running an execution module ``__context__`` persists across all module +executions until the modules are refreshed; such as when +:py:func:`saltutil.sync_all ` or +:py:func:`state.apply ` are executed. + +A great place to see how to use ``__context__`` is in the cp.py module in +salt/modules/cp.py. The fileclient authenticates with the master when it is +instantiated and then is used to copy files to the minion. Rather than create a +new fileclient for each file that is to be copied down, one instance of the +fileclient is instantiated in the ``__context__`` dictionary and is reused for +each file. Here is an example from salt/modules/cp.py: + +.. code-block:: python + + if not 'cp.fileclient' in __context__: + __context__['cp.fileclient'] = salt.fileclient.get_file_client(__opts__) + + +.. note:: Because __context__ may or may not have been destroyed, always be + sure to check for the existence of the key in __context__ and + generate the key before using it. + +__utils__ +--------- +Defined in: Cloud, Engine, Execution, File Server, Pillar, Proxy, Runner, SDB. + +__proxy__ +--------- +Defined in: Beacon, Engine, Execution, Executor, Proxy, Renderer, Returner, State, Util + +__runners__ +----------- +Defined in: Engine, Roster, Thorium + +__ret__ +------- +Defined in: Proxy, Search + +__thorium__ +----------- +Defined in: Thorium + +__states__ +---------- +Defined in: Renderers, State + +__serializers__ +--------------- +Defined in: State + +__sdb__ +------- +Defined in: SDB diff --git a/doc/topics/development/external_pillars.rst b/doc/topics/development/modules/external_pillars.rst similarity index 100% rename from doc/topics/development/external_pillars.rst rename to doc/topics/development/modules/external_pillars.rst diff --git a/doc/topics/development/modules/index.rst b/doc/topics/development/modules/index.rst new file mode 100644 index 000000000000..e421c91f3130 --- /dev/null +++ b/doc/topics/development/modules/index.rst @@ -0,0 +1,394 @@ +.. _modular-systems: + +=============== +Modular Systems +=============== + +When first working with Salt, it is not always clear where all of the modular +components are and what they do. Salt comes loaded with more modular systems +than many users are aware of, making Salt very easy to extend in many places. + +The most commonly used modular systems are execution modules and states. But +the modular systems extend well beyond the more easily exposed components +and are often added to Salt to make the complete system more flexible. + +.. toctree:: + :maxdepth: 2 + :glob: + + developing + configuration + + +Loading Modules +=============== + +Modules come primarily from several sources: + +* The Salt package itself +* The Salt File Server +* The extmods directory +* Secondary packages installed + +Using one source to override another is not supported. + +The Salt Package +---------------- + +Salt itself ships with a large number of modules. These are part of the Salt +package itself and don't require the user to do anything to use them. (Although +a number of them have additional dependencies and/or configuration.) + +The Salt File Server +-------------------- + +The user may add modules by simply placing them in special directories in their +:ref:`fileserver `. + +The name of the directory inside of the file server is the directory name +prepended by underscore, such as: + +- :file:`_grains` +- :file:`_modules` +- :file:`_states` + +Modules must be synced before they can be used. This can happen a few ways, +discussed below. + +.. note: + Using saltenvs besides ``base`` may not work in all contexts. + +Sync Via States +~~~~~~~~~~~~~~~ + +The minion configuration contains an option ``autoload_dynamic_modules`` +which defaults to ``True``. This option makes the state system refresh all +dynamic modules when states are run. To disable this behavior set +:conf_minion:`autoload_dynamic_modules` to ``False`` in the minion config. + +When dynamic modules are autoloaded via states, only the modules defined in the +same saltenvs as the states currently being run. + +Sync Via the saltutil Module +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The saltutil module has a number of functions that can be used to sync all +or specific dynamic modules. The ``saltutil.sync_*`` +:py:mod:`execution functions ` and +:py:mod:`runner functions ` can be used to sync modules +to minions and the master, respectively. + + +The extmods Directory +--------------------- + +Any files places in the directory set by ``extension_modules`` settings +(:conf_minion:`minion `, +:conf_master:`master `, default +``/var/cache/salt/*/extmods``) can also be loaded as modules. Note that these +directories are also used by the ``saltutil.sync_*`` functions (mentioned +above) and files may be overwritten. + +Secondary Packages +------------------ + +Third-party packages may also add modules to Salt if they are installed in the +same system and Python environment as the Salt Minion or Master. + +This is done via setuptools entry points: + +.. code-block:: python + + setup( + # ... + entry_points={ + 'salt.loader': [ + 'module_dirs=spirofs.loader:module', + ], + }, + # ... + ) + +Note that these are not synced from the Salt Master to the Minions. They must be +installed indepdendently on each Minion. + +Module Types +============ + +The specific names used by each loading method above are as follows. See sections below +for a short summary of each of these systems. + +.. _module-name-table: + +============ ================================================================ ========================= ===================== +Module Type Salt Package Name FS/Directory Name Entry Point +============ ================================================================ ========================= ===================== +Auth ``salt.auth`` (:ref:`index `) ``auth`` [#no-fs]_ ``auth_dirs`` +Beacon ``salt.beacons`` (:ref:`index `) ``beacons`` ``beacons_dirs`` +Cache ``salt.cache`` (:ref:`index `) ``cache`` ``cache_dirs`` +Cloud ``salt.cloud.clouds`` (:ref:`index `) ``clouds`` ``cloud_dirs`` +Engine ``salt.engines`` (:ref:`index `) ``engines`` ``engines_dirs`` +Execution ``salt.modules`` (:ref:`index `) ``modules`` ``module_dirs`` +Executor ``salt.executors`` (:ref:`index `) ``executors`` [#no-fs]_ ``executor_dirs`` +File Server ``salt.fileserver`` (:ref:`index `) ``fileserver`` [#no-fs]_ ``fileserver_dirs`` +Grain ``salt.grains`` (:ref:`index `) ``grains`` ``grains_dirs`` +Log Handler ``salt.log.handlers`` (:ref:`index `) ``log_handlers`` ``log_handlers_dirs`` +Net API ``salt.netapi`` (:ref:`index `) ``netapi`` [#no-fs]_ ``netapi_dirs`` +Outputter ``salt.output`` (:ref:`index `) ``output`` ``outputter_dirs`` +Pillar ``salt.pillar`` (:ref:`index `) ``pillar`` ``pillar_dirs`` +Proxy ``salt.proxy`` (:ref:`index `) ``proxy`` ``proxy_dirs`` +Queue ``salt.queues`` (:ref:`index `) ``queues`` ``queue_dirs`` +Renderer ``salt.renderers`` (:ref:`index `) ``renderers`` ``render_dirs`` +Returner ``salt.returners`` (:ref:`index `) ``returners`` ``returner_dirs`` +Roster ``salt.roster`` (:ref:`index `) ``roster`` ``roster_dirs`` +Runner ``salt.runners`` (:ref:`index `) ``runners`` ``runner_dirs`` +SDB ``salt.sdb`` (:ref:`index `) ``sdb`` ``sdb_dirs`` +Search ``salt.search`` ``search`` [#no-fs]_ ``search_dirs`` +Serializer ``salt.serializers`` (:ref:`index `) ``serializers`` [#no-fs]_ ``serializers_dirs`` +SPM pkgdb ``salt.spm.pkgdb`` ``pkgdb`` [#no-fs]_ ``pkgdb_dirs`` +SPM pkgfiles ``salt.spm.pkgfiles`` ``pkgfiles`` [#no-fs]_ ``pkgfiles_dirs`` +SSH Wrapper ``salt.client.ssh.wrapper`` ``wrapper`` [#no-fs]_ ``wrapper_dirs`` +State ``salt.states`` (:ref:`index `) ``states`` ``states_dirs`` +Thorium ``salt.thorium`` (:ref:`index `) ``thorium`` [#no-fs]_ ``thorium_dirs`` +Top ``salt.tops`` (:ref:`index `) ``tops`` ``top_dirs`` +Util ``salt.utils`` ``utils`` ``utils_dirs`` +Wheel ``salt.wheels`` (:ref:`index `) ``wheel`` ``wheel_dirs`` +============ ================================================================ ========================= ===================== + +.. [#no-fs] These modules cannot be loaded from the Salt File Server. + +.. note: + While it is possible to import modules directly with the import statement, + it is strongly recommended that the appropriate + :ref:`dunder dictionary ` is used to access them + instead. This is because a number of factors affect module names, module + selection, and module overloading. + +Auth +---- + +The auth module system allows for external authentication routines to be easily +added into Salt. The `auth` function needs to be implemented to satisfy the +requirements of an auth module. Use the ``pam`` module as an example. + +See :ref:`External Authentication System ` for more about +authentication in Salt. + +Beacon +------ + +* :ref:`Writing Beacons ` + +Beacons are polled by the Salt event loop to monitor non-salt processes. See +:ref:`Beacons ` for more information about the beacon system. + +Cache +----- + +The minion cache is used by the master to store various information about +minions. See :ref:`Minion Data Cache ` for more information. + +Cloud +----- + +Cloud modules are backend implementations used by :ref:`Salt Cloud `. + +Engine +------ + +Engines are open-ended services managed by the Salt daemon (both master and +minion). They may interact with event loop, call other modules, or a variety of +non-salt tasks. See :ref:`Salt Engines ` for complete details. + +Execution +--------- + +.. toctree:: + :maxdepth: 1 + :glob: + + /ref/modules/index + +Execution modules make up the core of the functionality used by Salt to +interact with client systems. The execution modules create the core system +management library used by all Salt systems, including states, which +interact with minion systems. + +Execution modules are completely open ended in their execution. They can +be used to do anything required on a minion, from installing packages to +detecting information about the system. The only restraint in execution +modules is that the defined functions always return a JSON serializable +object. + +Executor +-------- + +Executors control how execution modules get called. The default is to just call +them, but this can be customized. + +File Server +----------- + +The file server module system is used to create file server backends used by the +Salt Master. These modules need to implement the functions used in the +fileserver subsystem. Use the ``gitfs`` module as an example. + +See :ref:`File Server Backends ` for more information. + +Grains +------ + +* :ref:`writing-grains` + +Grain modules define extra routines to populate grains data. All defined +public functions will be executed and MUST return a Python dict object. The +dict keys will be added to the grains made available to the minion. + +See :ref:`Grains ` for more. + +Log Handler +----------- + +Log handlers allows the logs from salt (master or minion) to be sent to log +aggregation systems. + +Net API +------- + +Net API modules are the actual server implementation used by Salt API. + +Output +------ + +The output modules supply the outputter system with routines to display data +in the terminal. These modules are very simple and only require the `output` +function to execute. The default system outputter is the ``nested`` module. + +Pillar +------ + +.. toctree:: + :maxdepth: 1 + :glob: + + external_pillars + +Used to define optional external pillar systems. The pillar generated via +the filesystem pillar is passed into external pillars. This is commonly used +as a bridge to database data for pillar, but is also the backend to the libvirt +state used to generate and sign libvirt certificates on the fly. + +Proxy +----- + +:ref:`Proxy Minions ` are a way to manage devices that cannot run +a full minion directly. + +Renderers +--------- + +Renderers are the system used to render sls files into salt highdata for the +state compiler. They can be as simple as the ``py`` renderer and as complex as +``stateconf`` and ``pydsl``. + +Returners +--------- + +Returners are used to send data from minions to external sources, commonly +databases. A full returner will implement all routines to be supported as an +external job cache. Use the ``redis`` returner as an example. + +Roster +------ + +The :ref:`Roster system ` is used by Salt SSH to enumerate devices. + +Runners +------- + +.. toctree:: + :maxdepth: 1 + :glob: + + /ref/runners/index + +Runners are purely master-side execution sequences. + +SDB +--- + +* :ref:`Writing SDB Modules ` + +SDB is a way to store data that's not associated with a minion. See +:ref:`Storing Data in Other Databases `. + +Search +------ + +A system for indexing the file server and pillars. Removed in 2018.3. + +Serializer +---------- + +Primarily used with :py:func:`file.serialize `. + +State +----- + +.. toctree:: + :maxdepth: 1 + :glob: + + /ref/states/index + +State modules are used to define the state interfaces used by Salt States. +These modules are restrictive in that they must follow a number of rules to +function properly. + +.. note:: + + State modules define the available routines in sls files. If calling + an execution module directly is desired, take a look at the `module` + state. + +SPM pkgdb +--------- + +* :ref:`SPM Development Guide: Package Database ` + +pkgdb modules provides storage backends to the package database. + +SPM pkgfiles +------------ + +* :ref:`SPM Development Guide: Package Database ` + +pkgfiles modules handle the actual installation. + +SSH Wrapper +----------- + +Replacement execution modules for :ref:`Salt SSH `. + +Thorium +------- + +Modules for use in the :ref:`Thorium ` event reactor. + +Tops +---- + +Tops modules are used to convert external data sources into top file data for +the state system. + +Util +---- + +Just utility modules to use with other modules via ``__utils__`` (see +:ref:`Dunder Dictionaries `). + +Wheel +----- + +The wheel system is used to manage master side management routines. These +routines are primarily intended for the API to enable master configuration. diff --git a/doc/topics/sdb/index.rst b/doc/topics/sdb/index.rst index c4d94120e2c7..4c670613ef4e 100644 --- a/doc/topics/sdb/index.rst +++ b/doc/topics/sdb/index.rst @@ -154,6 +154,7 @@ When writing Salt modules, it is not recommended to call ``sdb.get`` directly, as it requires the user to provide values in SDB, using a specific URI. Use ``config.get`` instead. +.. _sdb-writing-modules: Writing SDB Modules =================== diff --git a/doc/topics/spm/dev.rst b/doc/topics/spm/dev.rst index 11ad83364c1d..d6ed99aea372 100644 --- a/doc/topics/spm/dev.rst +++ b/doc/topics/spm/dev.rst @@ -20,6 +20,7 @@ marked as required are crucial to SPM's core functionality, while arguments that are marked as optional are provided as a benefit to the module, if it needs to use them. +.. _spm-development-pkgdb: Package Database ---------------- @@ -146,6 +147,8 @@ The only argument that is expected is ``db_``, which is the package database file. +.. _spm-development-pkgfiles: + Package Files ------------- By default, package files are installed using the ``local`` module. This module