diff --git a/docs/source/AdministratorGuide/Introduction/configurationbasics.rst b/docs/source/AdministratorGuide/Introduction/configurationbasics.rst index 1e8bbe40eb4..b4a74613fe0 100644 --- a/docs/source/AdministratorGuide/Introduction/configurationbasics.rst +++ b/docs/source/AdministratorGuide/Introduction/configurationbasics.rst @@ -14,7 +14,7 @@ Most of DIRAC services can be exposed using either the DIPs or the HTTPs protoco As a general rule, services can be duplicated, meaning you can have the same service running on multiple hosts, thus reducing the load. -There are only 2 cases of DIRAC services that have a "master/slave" concept, and these are the Configuration Service +There are only 2 cases of DIRAC services that have a "controller/worker" concept, and these are the Configuration Service and the Accounting/DataStore service. Same can be said for executors: you can have many residing on different hosts. diff --git a/docs/source/AdministratorGuide/ServerInstallations/InstallingDiracServer.rst b/docs/source/AdministratorGuide/ServerInstallations/InstallingDiracServer.rst index b8351b1f77d..9264fea1006 100644 --- a/docs/source/AdministratorGuide/ServerInstallations/InstallingDiracServer.rst +++ b/docs/source/AdministratorGuide/ServerInstallations/InstallingDiracServer.rst @@ -436,7 +436,7 @@ operation is the registration of the new host in the already functional Configur # # These options define the DIRAC components being installed on "this" DIRAC server. - # The simplest option is to install a slave of the Configuration Server and a + # The simplest option is to install a worker of the Configuration Server and a # SystemAdministrator for remote management. # # The following options defined components to be installed diff --git a/docs/source/AdministratorGuide/ServerInstallations/scalingAndLimitations.rst b/docs/source/AdministratorGuide/ServerInstallations/scalingAndLimitations.rst index 258b2cfada0..7916815bf9d 100644 --- a/docs/source/AdministratorGuide/ServerInstallations/scalingAndLimitations.rst +++ b/docs/source/AdministratorGuide/ServerInstallations/scalingAndLimitations.rst @@ -90,11 +90,11 @@ Services +--------------------+---------------------------------------------------------------------------------------------------+-------------+---------------------------------------------------------------------------+-----------+ | **System** | **Component** |**Duplicate**| **Remarks** | **HTTPs** + +--------------------+---------------------------------------------------------------------------------------------------+-------------+---------------------------------------------------------------------------+-----------+ -| Accounting | :mod:`DataStore ` | PARTIAL | One master and helpers (See :ref:`datastorehelpers`) | + +| Accounting | :mod:`DataStore ` | PARTIAL | One controller and helpers (See :ref:`datastorehelpers`) | + + +---------------------------------------------------------------------------------------------------+-------------+---------------------------------------------------------------------------+-----------+ | | :mod:`ReportGenerator ` | | | + +--------------------+---------------------------------------------------------------------------------------------------+-------------+---------------------------------------------------------------------------+-----------+ -| Configuration | :mod:`Configuration ` | PARTIAL | One master (rw) and slaves (ro). It's advised to have several CS slaves | YES + +| Configuration | :mod:`Configuration ` | PARTIAL | One controller (rw) and workers (ro). Should have several CS workers | YES + +--------------------+---------------------------------------------------------------------------------------------------+-------------+---------------------------------------------------------------------------+-----------+ | DataManagement | :mod:`DataIntegrity ` | YES | | YES + + +---------------------------------------------------------------------------------------------------+-------------+---------------------------------------------------------------------------+-----------+ diff --git a/docs/source/AdministratorGuide/Systems/Configuration/index.rst b/docs/source/AdministratorGuide/Systems/Configuration/index.rst index 22a02e7c5e7..9cec6415c8a 100644 --- a/docs/source/AdministratorGuide/Systems/Configuration/index.rst +++ b/docs/source/AdministratorGuide/Systems/Configuration/index.rst @@ -5,22 +5,22 @@ Configuration System ==================== The configuration system serves the configuration to any other client (be it another server or a standard client). -The infrastructure is master/slave based. +The infrastructure is controller/worker based. -****** -Master -****** +********** +Controller +********** -The master Server holds the central configuration in a local file. This file is then served to the clients, and synchronized with the slave servers. +The controller Server holds the central configuration in a local file. This file is then served to the clients, and synchronized with the worker servers. -the master server also regularly pings the slave servers to make sure they are still alive. If not, they are removed from the list of CS. +the controller server also regularly pings the worker servers to make sure they are still alive. If not, they are removed from the list of CS. -When changes are committed to the master, a backup of the existing configuration file is made in ``etc/csbackup``. +When changes are committed to the controller, a backup of the existing configuration file is made in ``etc/csbackup``. -****** -Slaves -****** +******* +Workers +******* -Slave server registers themselves to the master when starting. +worker server registers themselves to the controller when starting. They synchronize their configuration on a regular bases (every 5 minutes by default). -Note that the slave CS do not hold the configuration in a local file, but only in memory. +Note that the worker CS do not hold the configuration in a local file, but only in memory. diff --git a/docs/source/DeveloperGuide/Overview/index.rst b/docs/source/DeveloperGuide/Overview/index.rst index 22c4c8353a9..89be55b3fb9 100644 --- a/docs/source/DeveloperGuide/Overview/index.rst +++ b/docs/source/DeveloperGuide/Overview/index.rst @@ -132,7 +132,7 @@ Configuration Service The Configuration Service is built in the DISET framework to provide static configuration parameters to all the distributed DIRAC components. This is the backbone of the whole system and necessitates excellent reliability. Therefore, it is organized as a single master service where all the parameter -updates are done and multiple read-only slave services which are distributed geographically, on VO-boxes +updates are done and multiple read-only worker services which are distributed geographically, on VO-boxes at Tier-1 LCG sites in the case of LHCb. All the servers are queried by clients in a load balancing way. This arrangement ensures configuration data consistency together with very good scalability properties. diff --git a/docs/source/DeveloperGuide/Systems/RequestManagement/index.rst b/docs/source/DeveloperGuide/Systems/RequestManagement/index.rst index 8dfc4a73d10..393cee33079 100644 --- a/docs/source/DeveloperGuide/Systems/RequestManagement/index.rst +++ b/docs/source/DeveloperGuide/Systems/RequestManagement/index.rst @@ -201,7 +201,7 @@ The agent will try to execute request as a whole in one go. :alt: Treating of Request in the RequestExecutionAgent. :align: center -The `RequestExecutingAgent` is using the `ProcessPool` utility to create slave workers (subprocesses running `RequestTask`) +The `RequestExecutingAgent` is using the `ProcessPool` utility to create workers (subprocesses running `RequestTask`) designated to execute requests read from `ReqDB`. Each worker is processing request execution using following steps: * downloading and setting up request's owner proxy diff --git a/src/DIRAC/AccountingSystem/Service/DataStoreHandler.py b/src/DIRAC/AccountingSystem/Service/DataStoreHandler.py index db808a3c2e8..cb468ab25c4 100644 --- a/src/DIRAC/AccountingSystem/Service/DataStoreHandler.py +++ b/src/DIRAC/AccountingSystem/Service/DataStoreHandler.py @@ -1,6 +1,6 @@ """ DataStore is the service for inserting accounting reports (rows) in the Accounting DB - This service CAN be duplicated iff the first is a "master" and all the others are slaves. + This service CAN be duplicated iff the first is a "controller" and all the others are workers. See the information about :ref:`datastorehelpers`. .. literalinclude:: ../ConfigTemplate.cfg @@ -112,7 +112,7 @@ def export_compactDB(self): """ Compact the db by grouping buckets """ - # if we are running slaves (not only one service) we can redirect the request to the master + # if we are running workers (not only one service) we can redirect the request to the master # For more information please read the Administrative guide Accounting part! # ADVICE: If you want to trigger the bucketing, please make sure the bucketing is not running!!!! if self.runBucketing: diff --git a/src/DIRAC/ConfigurationSystem/Agent/Bdii2CSAgent.py b/src/DIRAC/ConfigurationSystem/Agent/Bdii2CSAgent.py index 9cb50a8bac4..1704ad6e2b9 100644 --- a/src/DIRAC/ConfigurationSystem/Agent/Bdii2CSAgent.py +++ b/src/DIRAC/ConfigurationSystem/Agent/Bdii2CSAgent.py @@ -103,7 +103,7 @@ def execute(self): if not result["OK"]: self.log.warn("Could not download a fresh copy of the CS data", result["Message"]) - # Refresh the configuration from the master server + # Refresh the configuration from the controller server gConfig.forceRefresh(fromMaster=True) if self.processCEs: diff --git a/src/DIRAC/ConfigurationSystem/Client/CSAPI.py b/src/DIRAC/ConfigurationSystem/Client/CSAPI.py index 399fde278bd..6a1c4d7a366 100644 --- a/src/DIRAC/ConfigurationSystem/Client/CSAPI.py +++ b/src/DIRAC/ConfigurationSystem/Client/CSAPI.py @@ -98,7 +98,7 @@ def initialize(self): return self.__initialized retVal = gConfig.getOption("/DIRAC/Configuration/MasterServer") if not retVal["OK"]: - self.__initialized = S_ERROR("Master server is not known. Is everything initialized?") + self.__initialized = S_ERROR("Controller server is not known. Is everything initialized?") return self.__initialized self.__rpcClient = ConfigurationClient(url=gConfig.getValue("/DIRAC/Configuration/MasterServer", "")) self.__csMod = Modificator( @@ -915,7 +915,7 @@ def getCurrentCFG(self): def showDiff(self): """Just shows the differences accumulated within the Modificator object""" diffData = self.__csMod.showCurrentDiff() - gLogger.notice("Accumulated diff with master CS") + gLogger.notice("Accumulated diff with Controller CS") for line in diffData: if line[0] in ("+", "-"): gLogger.notice(line) diff --git a/src/DIRAC/ConfigurationSystem/Client/CSCLI.py b/src/DIRAC/ConfigurationSystem/Client/CSCLI.py index b45fe7c3101..46b8b602c9b 100644 --- a/src/DIRAC/ConfigurationSystem/Client/CSCLI.py +++ b/src/DIRAC/ConfigurationSystem/Client/CSCLI.py @@ -43,7 +43,7 @@ class CSCLI(CLI): def __init__(self): CLI.__init__(self) self.connected = False - self.masterURL = "unset" + self.controllerURL = "unset" self.writeEnabled = False self.modifiedData = False self.rpcClient = None @@ -83,11 +83,11 @@ def _setConnected(self, connected, writeEnabled): self.writeEnabled = writeEnabled if connected: if writeEnabled: - self.prompt = f"({self.masterURL})-{colorize('Connected', 'green')}> " + self.prompt = f"({self.controllerURL})-{colorize('Connected', 'green')}> " else: - self.prompt = f"({self.masterURL})-{colorize('Connected (RO)', 'yellow')}> " + self.prompt = f"({self.controllerURL})-{colorize('Connected (RO)', 'yellow')}> " else: - self.prompt = f"({self.masterURL})-{colorize('Disconnected', 'red')}> " + self.prompt = f"({self.controllerURL})-{colorize('Disconnected', 'red')}> " def do_quit(self, dummy): """ @@ -104,7 +104,7 @@ def do_quit(self, dummy): def _setStatus(self, connected=True): if not connected: - self.masterURL = "unset" + self.controllerURL = "unset" self._setConnected(False, False) else: retVal = self.rpcClient.writeEnabled() @@ -118,23 +118,23 @@ def _setStatus(self, connected=True): self._setConnected(True, False) def _tryConnection(self): - print(f"Trying connection to {self.masterURL}") + print(f"Trying connection to {self.controllerURL}") try: - self.rpcClient = ConfigurationClient(url=self.masterURL) + self.rpcClient = ConfigurationClient(url=self.controllerURL) self._setStatus() except Exception as x: - gLogger.error("Couldn't connect to master CS server", f"{self.masterURL} ({str(x)})") + gLogger.error("Couldn't connect to controller CS server", f"{self.controllerURL} ({str(x)})") self._setStatus(False) def do_connect(self, args=""): """ - Connects to configuration master server (in specified url if provided). + Connects to configuration controller server (in specified url if provided). Usage: connect """ if not args or not isinstance(args, str): - self.masterURL = gConfigurationData.getMasterServer() - if self.masterURL != "unknown" and self.masterURL: + self.controllerURL = gConfigurationData.getMasterServer() + if self.controllerURL != "unknown" and self.controllerURL: self._tryConnection() else: self._setStatus(False) @@ -144,7 +144,7 @@ def do_connect(self, args=""): print("Must specify witch url to connect") self._setStatus(False) else: - self.masterURL = splitted[0].strip() + self.controllerURL = splitted[0].strip() self._tryConnection() def do_sections(self, args): @@ -247,7 +247,7 @@ def do_writeToServer(self, dummy): choice = input("Do you really want to send changes to server? yes/no [no]: ") choice = choice.lower() if choice in ("yes", "y"): - print(f"Uploading changes to {self.masterURL} (It may take some seconds)...") + print(f"Uploading changes to {self.controllerURL} (It may take some seconds)...") response = self.modificator.commit() if response["OK"]: self.modifiedData = False diff --git a/src/DIRAC/ConfigurationSystem/Client/ConfigurationClient.py b/src/DIRAC/ConfigurationSystem/Client/ConfigurationClient.py index f0b60e8f81c..c3cb52b713e 100644 --- a/src/DIRAC/ConfigurationSystem/Client/ConfigurationClient.py +++ b/src/DIRAC/ConfigurationSystem/Client/ConfigurationClient.py @@ -63,7 +63,7 @@ class ConfigurationClient(Client): def __init__(self, **kwargs): # By default we use Configuration/Server as url, client do the resolution - # In some case url has to be static (when a slave register to the master server for example) + # In some case url has to be static (when a worker register to the controller server for example) # It's why we can use 'url' as keyword arguments if "url" not in kwargs: kwargs["url"] = "Configuration/Server" diff --git a/src/DIRAC/ConfigurationSystem/Service/TornadoConfigurationHandler.py b/src/DIRAC/ConfigurationSystem/Service/TornadoConfigurationHandler.py index add704586ad..970f2526bda 100644 --- a/src/DIRAC/ConfigurationSystem/Service/TornadoConfigurationHandler.py +++ b/src/DIRAC/ConfigurationSystem/Service/TornadoConfigurationHandler.py @@ -59,9 +59,9 @@ def export_getCompressedDataIfNewer(self, sClientVersion): def export_publishSlaveServer(self, sURL): """ - Used by slave server to register as a slave server. + Used by worker server to register as a worker server. - :param sURL: The url of the slave server. + :param sURL: The url of the worker server. """ self.ServiceInterface.publishSlaveServer(sURL) return S_OK() diff --git a/src/DIRAC/ConfigurationSystem/private/Refresher.py b/src/DIRAC/ConfigurationSystem/private/Refresher.py index 28e17a8e818..ed1fc3d7a58 100755 --- a/src/DIRAC/ConfigurationSystem/private/Refresher.py +++ b/src/DIRAC/ConfigurationSystem/private/Refresher.py @@ -72,7 +72,7 @@ def autoRefreshAndPublish(self, sURL): """ gLogger.debug("Setting configuration refresh as automatic") if not gConfigurationData.getAutoPublish(): - gLogger.debug("Slave server won't auto publish itself") + gLogger.debug("Worker server won't auto publish itself") if not gConfigurationData.getName(): import DIRAC diff --git a/src/DIRAC/ConfigurationSystem/private/RefresherBase.py b/src/DIRAC/ConfigurationSystem/private/RefresherBase.py index 5bb2dc0b511..a61f98a7276 100644 --- a/src/DIRAC/ConfigurationSystem/private/RefresherBase.py +++ b/src/DIRAC/ConfigurationSystem/private/RefresherBase.py @@ -1,13 +1,11 @@ import time -import random - from DIRAC.ConfigurationSystem.Client.ConfigurationData import gConfigurationData from DIRAC.ConfigurationSystem.Client.PathFinder import getGatewayURLs -from DIRAC.FrameworkSystem.Client.Logger import gLogger from DIRAC.Core.Utilities import List from DIRAC.Core.Utilities.EventDispatcher import gEventDispatcher -from DIRAC.Core.Utilities.ReturnValues import S_OK, S_ERROR +from DIRAC.Core.Utilities.ReturnValues import S_ERROR, S_OK +from DIRAC.FrameworkSystem.Client.Logger import gLogger def _updateFromRemoteLocation(serviceClient): @@ -90,29 +88,31 @@ def _refreshAndPublish(self): Refresh configuration and publish local updates """ self._lastUpdateTime = time.time() - gLogger.info("Refreshing from master server") - sMasterServer = gConfigurationData.getMasterServer() - if sMasterServer: + gLogger.info("Refreshing from controller server") + sControllerServer = gConfigurationData.getMasterServer() + if sControllerServer: from DIRAC.ConfigurationSystem.Client.ConfigurationClient import ConfigurationClient oClient = ConfigurationClient( - url=sMasterServer, + url=sControllerServer, timeout=self._timeout, useCertificates=gConfigurationData.useServerCertificate(), skipCACheck=gConfigurationData.skipCACheck(), ) dRetVal = _updateFromRemoteLocation(oClient) if not dRetVal["OK"]: - gLogger.error("Can't update from master server", dRetVal["Message"]) + gLogger.error("Can't update from controller server", dRetVal["Message"]) return False if gConfigurationData.getAutoPublish(): - gLogger.info("Publishing to master server...") + gLogger.info("Publishing to controller server...") dRetVal = oClient.publishSlaveServer(self._url) if not dRetVal["OK"]: - gLogger.error("Can't publish to master server", dRetVal["Message"]) + gLogger.error("Can't publish to controller server", dRetVal["Message"]) return True else: - gLogger.warn("No master server is specified in the configuration, trying to get data from other slaves") + gLogger.warn( + "No controller server is specified in the configuration, trying to get data from other Workers" + ) return self._refresh()["OK"] def _refresh(self, fromMaster=False): @@ -127,9 +127,9 @@ def _refresh(self, fromMaster=False): initialServerList = gatewayList gLogger.debug("Using configuration gateway", str(initialServerList[0])) elif fromMaster: - masterServer = gConfigurationData.getMasterServer() - initialServerList = [masterServer] - gLogger.debug(f"Refreshing from master {masterServer}") + controllerServer = gConfigurationData.getMasterServer() + initialServerList = [controllerServer] + gLogger.debug(f"Refreshing from controller {controllerServer}") else: initialServerList = gConfigurationData.getServers() gLogger.debug(f"Refreshing from list {str(initialServerList)}") diff --git a/src/DIRAC/ConfigurationSystem/private/ServiceInterface.py b/src/DIRAC/ConfigurationSystem/private/ServiceInterface.py index c9a2d09d8ad..ffa48b48c52 100755 --- a/src/DIRAC/ConfigurationSystem/private/ServiceInterface.py +++ b/src/DIRAC/ConfigurationSystem/private/ServiceInterface.py @@ -11,7 +11,7 @@ class ServiceInterface(ServiceInterfaceBase, threading.Thread): """ - Service interface, manage Slave/Master server for CS + Service interface, manage Worker/Controller server for CS Thread components """ @@ -19,11 +19,11 @@ def __init__(self, sURL): threading.Thread.__init__(self) ServiceInterfaceBase.__init__(self, sURL) - def _launchCheckSlaves(self): + def _launchCheckWorkers(self): """ - Start loop which check if slaves are alive + Start loop which check if Workers are alive """ - gLogger.info("Starting purge slaves thread") + gLogger.info("Starting purge Workers thread") self.daemon = True self.start() @@ -31,24 +31,24 @@ def run(self): while True: iWaitTime = gConfigurationData.getSlavesGraceTime() time.sleep(iWaitTime) - self._checkSlavesStatus() + self._checkWorkersStatus() - def _updateServiceConfiguration(self, urlSet, fromMaster=False): + def _updateServiceConfiguration(self, urlSet, fromController=False): """ - Update configuration of a set of slave services in parallel + Update configuration of a set of Worker services in parallel :param set urlSet: a set of service URLs - :param fromMaster: flag to force updating from the master CS + :param fromController: flag to force updating from the master CS :return: Nothing """ if not urlSet: return with ThreadPoolExecutor(max_workers=len(urlSet)) as executor: - futureUpdate = {executor.submit(self._forceServiceUpdate, url, fromMaster): url for url in urlSet} + futureUpdate = {executor.submit(self._forceServiceUpdate, url, fromController): url for url in urlSet} for future in as_completed(futureUpdate): url = futureUpdate[future] result = future.result() if result["OK"]: - gLogger.info("Successfully updated slave configuration", url) + gLogger.info("Successfully updated Worker configuration", url) else: - gLogger.error("Failed to update slave configuration", url) + gLogger.error("Failed to update Worker configuration", url) diff --git a/src/DIRAC/ConfigurationSystem/private/ServiceInterfaceBase.py b/src/DIRAC/ConfigurationSystem/private/ServiceInterfaceBase.py index 8fc6bd33364..1e068340d97 100644 --- a/src/DIRAC/ConfigurationSystem/private/ServiceInterfaceBase.py +++ b/src/DIRAC/ConfigurationSystem/private/ServiceInterfaceBase.py @@ -1,4 +1,4 @@ -"""Service interface is the service which provide config for client and synchronize Master/Slave servers""" +"""Service interface is the service which provide config for client and synchronize Controller/Worker servers""" import os import re @@ -17,26 +17,26 @@ class ServiceInterfaceBase: - """Service interface is the service which provide config for client and synchronize Master/Slave servers""" + """Service interface is the service which provide config for client and synchronize Controller/Worker servers""" def __init__(self, sURL): self.sURL = sURL gLogger.info("Initializing Configuration Service", f"URL is {sURL}") gConfigurationData.setAsService() if not gConfigurationData.isMaster(): - gLogger.info("Starting configuration service as slave") + gLogger.info("Starting configuration service as Worker") gRefresher.autoRefreshAndPublish(self.sURL) else: - gLogger.info("Starting configuration service as master") + gLogger.info("Starting configuration service as controller") gRefresher.disable() self.__loadConfigurationData() - self.dAliveSlaveServers = {} - self._launchCheckSlaves() + self.dAliveWorkerServers = {} + self._launchCheckWorkers() def isMaster(self): return gConfigurationData.isMaster() - def _launchCheckSlaves(self): + def _launchCheckWorkers(self): raise NotImplementedError("Should be implemented by the children class") def __loadConfigurationData(self): @@ -74,91 +74,91 @@ def __generateNewVersion(self): gConfigurationData.generateNewVersion() gConfigurationData.writeRemoteConfigurationToDisk() - def publishSlaveServer(self, sSlaveURL): + def publishSlaveServer(self, sWorkerURL): """ - Called by the slave server via service, it register a new slave server + Called by the Worker server via service, it register a new Worker server - :param sSlaveURL: url of slave server + :param sWorkerURL: url of Worker server """ if not gConfigurationData.isMaster(): return S_ERROR("Configuration modification is not allowed in this server") - gLogger.info(f"Pinging slave {sSlaveURL}") - rpcClient = ConfigurationClient(url=sSlaveURL, timeout=10, useCertificates=True) + gLogger.info(f"Pinging Worker {sWorkerURL}") + rpcClient = ConfigurationClient(url=sWorkerURL, timeout=10, useCertificates=True) retVal = rpcClient.ping() if not retVal["OK"]: - gLogger.info(f"Slave {sSlaveURL} didn't reply") + gLogger.info(f"Worker {sWorkerURL} didn't reply") return if retVal["Value"]["name"] != "Configuration/Server": - gLogger.info(f"Slave {sSlaveURL} is not a CS serveR") + gLogger.info(f"Worker {sWorkerURL} is not a CS serveR") return - bNewSlave = False - if sSlaveURL not in self.dAliveSlaveServers: - bNewSlave = True - gLogger.info("New slave registered", sSlaveURL) - self.dAliveSlaveServers[sSlaveURL] = time.time() - if bNewSlave: - gConfigurationData.setServers(", ".join(self.dAliveSlaveServers)) + bNewWorker = False + if sWorkerURL not in self.dAliveWorkerServers: + bNewWorker = True + gLogger.info("New Worker registered", sWorkerURL) + self.dAliveWorkerServers[sWorkerURL] = time.time() + if bNewWorker: + gConfigurationData.setServers(", ".join(self.dAliveWorkerServers)) self.__generateNewVersion() - def _checkSlavesStatus(self, forceWriteConfiguration=False): + def _checkWorkersStatus(self, forceWriteConfiguration=False): """ - Check if Slaves server are still availlable + Check if Workers server are still availlable - :param forceWriteConfiguration: (default False) Force rewriting configuration after checking slaves + :param forceWriteConfiguration: (default False) Force rewriting configuration after checking workers """ - gLogger.info("Checking status of slave servers") + gLogger.info("Checking status of Worker servers") iGraceTime = gConfigurationData.getSlavesGraceTime() - bModifiedSlaveServers = False - for sSlaveURL in list(self.dAliveSlaveServers): - if time.time() - self.dAliveSlaveServers[sSlaveURL] > iGraceTime: - gLogger.warn("Found dead slave", sSlaveURL) - del self.dAliveSlaveServers[sSlaveURL] - bModifiedSlaveServers = True - if bModifiedSlaveServers or forceWriteConfiguration: - gConfigurationData.setServers(", ".join(self.dAliveSlaveServers)) + bModifiedWorkerServers = False + for sWorkerURL in list(self.dAliveWorkerServers): + if time.time() - self.dAliveWorkerServers[sWorkerURL] > iGraceTime: + gLogger.warn("Found dead Worker", sWorkerURL) + del self.dAliveWorkerServers[sWorkerURL] + bModifiedWorkerServers = True + if bModifiedWorkerServers or forceWriteConfiguration: + gConfigurationData.setServers(", ".join(self.dAliveWorkerServers)) self.__generateNewVersion() @staticmethod - def _forceServiceUpdate(url, fromMaster): + def _forceServiceUpdate(url, fromController): """ Force updating configuration on a given service This should be called by _updateServiceConfiguration :param str url: service URL - :param bool fromMaster: flag to force updating from the master CS + :param bool fromController: flag to force updating from the controller CS :return: S_OK/S_ERROR """ gLogger.info("Updating service configuration on", url) - result = Client(url=url).refreshConfiguration(fromMaster) + result = Client(url=url).refreshConfiguration(fromController) result["URL"] = url return result - def _updateServiceConfiguration(self, urlSet, fromMaster=False): + def _updateServiceConfiguration(self, urlSet, fromController=False): """ Update configuration in a set of service in parallel :param set urlSet: a set of service URLs - :param fromMaster: flag to force updating from the master CS + :param fromController: flag to force updating from the controller CS :return: Nothing """ raise NotImplementedError("Should be implemented by the children class") - def forceSlavesUpdate(self): + def forceWorkersUpdate(self): """ - Force updating configuration on all the slave configuration servers + Force updating configuration on all the Worker configuration servers :return: Nothing """ - gLogger.info("Updating configuration on slave servers") + gLogger.info("Updating configuration on Worker servers") iGraceTime = gConfigurationData.getSlavesGraceTime() urlSet = set() - for slaveURL in self.dAliveSlaveServers: - if time.time() - self.dAliveSlaveServers[slaveURL] <= iGraceTime: - urlSet.add(slaveURL) - self._updateServiceConfiguration(urlSet, fromMaster=True) + for workerURL in self.dAliveWorkerServers: + if time.time() - self.dAliveWorkerServers[workerURL] <= iGraceTime: + urlSet.add(workerURL) + self._updateServiceConfiguration(urlSet, fromController=True) def forceGlobalUpdate(self): """ @@ -185,7 +185,7 @@ def forceGlobalUpdate(self): def updateConfiguration(self, sBuffer, committer="", updateVersionOption=False): """ - Update the master configuration with the newly received changes + Update the controller configuration with the newly received changes :param str sBuffer: newly received configuration data :param str committer: the user name of the committer @@ -232,14 +232,14 @@ def updateConfiguration(self, sBuffer, committer="", updateVersionOption=False): gConfigurationData.unlock() gLogger.info("Generating new version") gConfigurationData.generateNewVersion() - # self.__checkSlavesStatus( forceWriteConfiguration = True ) + # self.__checkWorkersStatus( forceWriteConfiguration = True ) gLogger.info("Writing new version to disk") retVal = gConfigurationData.writeRemoteConfigurationToDisk(f"{committer}@{gConfigurationData.getVersion()}") gLogger.info("New version", gConfigurationData.getVersion()) - # Attempt to update the configuration on currently registered slave services + # Attempt to update the configuration on currently registered Worker services if gConfigurationData.getAutoSlaveSync(): - self.forceSlavesUpdate() + self.forceWorkersUpdate() return retVal diff --git a/src/DIRAC/ConfigurationSystem/private/ServiceInterfaceTornado.py b/src/DIRAC/ConfigurationSystem/private/ServiceInterfaceTornado.py index 8152be6449e..a665d2bc03e 100644 --- a/src/DIRAC/ConfigurationSystem/private/ServiceInterfaceTornado.py +++ b/src/DIRAC/ConfigurationSystem/private/ServiceInterfaceTornado.py @@ -17,29 +17,29 @@ class ServiceInterfaceTornado(ServiceInterfaceBase): def __init__(self, sURL): ServiceInterfaceBase.__init__(self, sURL) - def _launchCheckSlaves(self): + def _launchCheckWorkers(self): """ - Start loop to check if slaves are alive + Start loop to check if workers are alive """ IOLoop.current().spawn_callback(self.run) - gLogger.info("Starting purge slaves thread") + gLogger.info("Starting purge workers thread") def run(self): """ - Check if slaves are alive + Check if workers are alive """ while True: yield gen.sleep(gConfigurationData.getSlavesGraceTime()) - self._checkSlavesStatus() + self._checkWorkersStatus() - def _updateServiceConfiguration(self, urlSet, fromMaster=False): + def _updateServiceConfiguration(self, urlSet, fromController=False): """ Update configuration in a set of service in parallel :param set urlSet: a set of service URLs - :param fromMaster: flag to force updating from the master CS + :param fromController: flag to force updating from the controller CS :return: Nothing """ for url in urlSet: - IOLoop.current().spawn_callback(self._forceServiceUpdate, [url, fromMaster]) + IOLoop.current().spawn_callback(self._forceServiceUpdate, [url, fromController]) diff --git a/src/DIRAC/ConfigurationSystem/private/TornadoRefresher.py b/src/DIRAC/ConfigurationSystem/private/TornadoRefresher.py index 2ce15174696..a90215d76e5 100644 --- a/src/DIRAC/ConfigurationSystem/private/TornadoRefresher.py +++ b/src/DIRAC/ConfigurationSystem/private/TornadoRefresher.py @@ -42,7 +42,7 @@ def autoRefreshAndPublish(self, sURL): """ gLogger.debug("Setting configuration refresh as automatic") if not gConfigurationData.getAutoPublish(): - gLogger.debug("Slave server won't auto publish itself") + gLogger.debug("Worker server won't auto publish itself") if not gConfigurationData.getName(): import DIRAC @@ -70,7 +70,7 @@ def __refreshLoop(self): # But this version is non-blocking, so IOLoop can continue execution yield gen.sleep(gConfigurationData.getPropagationTime()) # Publish step is blocking so we have to run it in executor - # If we are not doing it, when master try to ping we block the IOLoop + # If we are not doing it, when controller try to ping we block the IOLoop yield _IOLoop.current().run_in_executor(None, self.__AutoRefresh) diff --git a/src/DIRAC/Core/DISET/RequestHandler.py b/src/DIRAC/Core/DISET/RequestHandler.py index f3fdd442d52..309fa8036f6 100755 --- a/src/DIRAC/Core/DISET/RequestHandler.py +++ b/src/DIRAC/Core/DISET/RequestHandler.py @@ -14,7 +14,7 @@ def getServiceOption(serviceInfo, optionName, defaultValue): - """Get service option resolving default values from the master service""" + """Get service option resolving default values from the controller service""" if optionName[0] == "/": return gConfig.getValue(optionName, defaultValue) for csPath in serviceInfo["csPaths"]: @@ -548,7 +548,7 @@ def export_refreshConfiguration(fromMaster): """ Force refreshing the configuration data - :param bool fromMaster: flag to refresh from the master configuration service + :param bool fromMaster: flag to refresh from the controller configuration service """ return gConfig.forceRefresh(fromMaster=fromMaster) diff --git a/src/DIRAC/Core/Tornado/scripts/tornado_start_CS.py b/src/DIRAC/Core/Tornado/scripts/tornado_start_CS.py index da97c2e0bbf..e303b719464 100644 --- a/src/DIRAC/Core/Tornado/scripts/tornado_start_CS.py +++ b/src/DIRAC/Core/Tornado/scripts/tornado_start_CS.py @@ -45,7 +45,7 @@ def main(): gLogger.initialize("Tornado-CS", "/") - # get the specific master CS port + # get the specific controller CS port try: csPort = int(gConfigurationData.extractOptionFromCFG(f"{getServiceSection('Configuration/Server')}/Port")) except TypeError: diff --git a/src/DIRAC/Core/Tornado/scripts/tornado_start_all.py b/src/DIRAC/Core/Tornado/scripts/tornado_start_all.py index 46be05f568a..9fb91abbe8c 100644 --- a/src/DIRAC/Core/Tornado/scripts/tornado_start_all.py +++ b/src/DIRAC/Core/Tornado/scripts/tornado_start_all.py @@ -43,8 +43,8 @@ def main(): gLogger.initialize("Tornado", "/") - # We check if there is no configuration server started as master - # If you want to start a master CS you should use Configuration_Server.cfg and + # We check if there is no configuration server started as controller + # If you want to start a controller CS you should use Configuration_Server.cfg and # use tornado-start-CS.py key = f"/Systems/Configuration/Services/Server/Protocol" if gConfigurationData.isMaster() and gConfig.getValue(key, "dips").lower() == "https": diff --git a/src/DIRAC/Core/Utilities/ProcessPool.py b/src/DIRAC/Core/Utilities/ProcessPool.py index 9e4de45a911..ad86fa65788 100644 --- a/src/DIRAC/Core/Utilities/ProcessPool.py +++ b/src/DIRAC/Core/Utilities/ProcessPool.py @@ -547,14 +547,14 @@ class ProcessPool: Pool depth - The :ProcessPool: is keeping required number of active workers all the time: slave workers are only created + The :ProcessPool: is keeping required number of active workers all the time: worker workers are only created when pendingQueue is being filled with tasks, not exceeding defined min and max limits. When pendingQueue is empty, active workers will be cleaned up by themselves, as each worker has got built in self-destroy mechanism after 10 idle loops. Processing and communication - The communication between :ProcessPool: instance and slaves is performed using two :multiprocessing.Queues: + The communication between :ProcessPool: instance and workers is performed using two :multiprocessing.Queues: * pendingQueue, used to push tasks to the workers, * resultsQueue for revert direction; diff --git a/src/DIRAC/Core/scripts/dirac_configure.py b/src/DIRAC/Core/scripts/dirac_configure.py index ad2fba345d5..31f30793162 100755 --- a/src/DIRAC/Core/scripts/dirac_configure.py +++ b/src/DIRAC/Core/scripts/dirac_configure.py @@ -214,7 +214,7 @@ def _runConfigurationWizard(setups, defaultSetup): if confirm.lower() not in ["y", "yes"]: return None - # Get the URL to the master CS + # Get the URL to the controller CS csURL = prompt(HTML("Choose a configuration server URL (leave blank for default):\n")) if not csURL: csURL = setups[setup] @@ -344,7 +344,7 @@ def login(params): return result DIRAC.gLogger.debug(f"New token is saved to {result['Value']}.") - # Get server setups and master CS server URL + # Get server setups and controller CS server URL csURL = idpObj.get_metadata("configuration_server") setups = idpObj.get_metadata("setups") diff --git a/src/DIRAC/FrameworkSystem/Client/ComponentInstaller.py b/src/DIRAC/FrameworkSystem/Client/ComponentInstaller.py index e59b18569e3..35bd675c7b2 100644 --- a/src/DIRAC/FrameworkSystem/Client/ComponentInstaller.py +++ b/src/DIRAC/FrameworkSystem/Client/ComponentInstaller.py @@ -34,7 +34,7 @@ /LocalInstallation/PrivateConfiguration: Boolean, requires Configuration/Server to be given in the list of Services (default: no) -If a Master Configuration Server is being installed the following Options can be used:: +If a Controller Configuration Server is being installed the following Options can be used:: /LocalInstallation/ConfigurationName: Name of the Configuration (default: Setup ) /LocalInstallation/AdminUserName: Name of the Admin user (default: None ) @@ -390,7 +390,7 @@ def _removeSectionFromCS(self, path): def _getCentralCfg(self, installCfg): """ - Create the skeleton of central Cfg for an initial Master CS + Create the skeleton of central Cfg for an initial Controller CS """ # First copy over from installation cfg centralCfg = CFG() @@ -498,7 +498,7 @@ def _getCentralCfg(self, installCfg): properties.append(prop) centralCfg["Registry"]["Groups"][defaultGroupName].appendToOption("Properties", f", {prop}") - # Add the master Host description + # Add the controller Host description if hostDN: hostSection = cfgPath("Registry", "Hosts", self.host) if not centralCfg.isSection(hostSection): @@ -1299,7 +1299,7 @@ def setupSite(self, scriptCfg, cfg=None): for k in self.localCfg.getOption(cfgInstallPath("Executors"), []) # pylint: disable=not-an-iterable ] setupWeb = self.localCfg.getOption(cfgInstallPath("WebPortal"), False) - setupConfigurationMaster = self.localCfg.getOption(cfgInstallPath("ConfigurationMaster"), False) + setupConfigurationController = self.localCfg.getOption(cfgInstallPath("ConfigurationMaster"), False) setupPrivateConfiguration = self.localCfg.getOption(cfgInstallPath("PrivateConfiguration"), False) setupConfigurationName = self.localCfg.getOption(cfgInstallPath("ConfigurationName"), "DIRAC-Prod") setupAddConfiguration = self.localCfg.getOption(cfgInstallPath("AddConfiguration"), True) @@ -1381,11 +1381,11 @@ def setupSite(self, scriptCfg, cfg=None): universal_newlines=True, ) - if ["Configuration", "Server"] in setupServices and setupConfigurationMaster: - # This server hosts the Master of the CS + if ["Configuration", "Server"] in setupServices and setupConfigurationController: + # This server hosts the Controller of the CS from DIRAC.ConfigurationSystem.Client.ConfigurationData import gConfigurationData - gLogger.notice("Installing Master Configuration Server (Tornado-based)") + gLogger.notice("Installing Controller Configuration Server (Tornado-based)") # Add some needed bootstrapping configuration cfg = self.__getCfg( @@ -1441,15 +1441,15 @@ def setupSite(self, scriptCfg, cfg=None): DIRAC.exit(-1) return S_ERROR(error) - # We need to make sure components are connecting to the Master CS, that is the only one being update + # We need to make sure components are connecting to the Controller CS, that is the only one being update localServers = self.localCfg.getOption(cfgPath("DIRAC", "Configuration", "Servers")) - masterServer = gConfig.getValue(cfgPath("DIRAC", "Configuration", "MasterServer"), "") + controllerServer = gConfig.getValue(cfgPath("DIRAC", "Configuration", "MasterServer"), "") initialCfg = self.__getCfg(cfgPath("DIRAC", "Configuration"), "Servers", localServers) - masterCfg = self.__getCfg(cfgPath("DIRAC", "Configuration"), "Servers", masterServer) - self._addCfgToDiracCfg(masterCfg) + controllerCfg = self.__getCfg(cfgPath("DIRAC", "Configuration"), "Servers", controllerServer) + self._addCfgToDiracCfg(controllerCfg) # 1.- Setup the instances in the CS - # If the Configuration Server used is not the Master, it can take some time for this + # If the Configuration Server used is not the Controller, it can take some time for this # info to be propagated, this may cause the later self.setup to fail if setupAddConfiguration: for system, service in setupServices: @@ -1564,7 +1564,7 @@ def setupSite(self, scriptCfg, cfg=None): if setupWeb: self.setupPortal() - if localServers != masterServer: + if localServers != controllerServer: self._addCfgToDiracCfg(initialCfg) for system, service in setupServices: self.runsvctrlComponent(system, service, "t")