How to use DDP?

[Zhiying-Li-dot]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

I do not know what should I do. When I run train.py without DDP, it can run, but slowly. So I want to use DDP to speed up. Always display this bug. Could you tell me how to solve this problem? Please.

Additional

No response

[glenn-jocher]

@AyushExel it looks like we should disable LOGGER for all RANKs except for RANK in (0, -1) to avoid multiple printouts like the AMP check above:

yolov5/utils/general.py

Line 534 in 1156a32

LOGGER.warning(emojis(f'{prefix}checks failed ❌, disabling Automatic Mixed Precision. See {help_url}'))

I think I can do this here:

yolov5/utils/general.py

Lines 82 to 99 in 1156a32

	def set_logging(name=None, verbose=VERBOSE):
	# Sets level and returns logger
	if is_kaggle():
	for h in logging.root.handlers:
	logging.root.removeHandler(h) # remove all handlers associated with the root logger object
	rank = int(os.getenv('RANK', -1)) # rank in world for Multi-GPU trainings
	level = logging.INFO if verbose and rank in {-1, 0} else logging.WARNING
	log = logging.getLogger(name)
	log.setLevel(level)
	handler = logging.StreamHandler()
	handler.setFormatter(logging.Formatter("%(message)s"))
	handler.setLevel(level)
	log.addHandler(handler)
	set_logging() # run before defining LOGGER
	LOGGER = logging.getLogger("yolov5") # define globally (used in train.py, val.py, detect.py, etc.)

EDIT: Oh, haha I've already done this on L88, this line is just printing out because it's set to warning for non RANK (0, -1) messages. Maybe I should just disable it entirely or set it to logging.ERROR for non RANK(0, -1)?
EDIT2: console output fix implemented in #8284