Technical Debt

[justheuristic]

tesseract.client

• MoE does not support failures during backward
• RemoteExpert doesn't check if inputs are valid (should check via info)

tesseract.runtime

• non-deterministic layers use different random during forward and backward passes
• layers with batchnorm update stats twice. Update during backward only? upd: we now assume that forward is a pure deterministic function; moved discussion to a separate thread
• form_batch: when batch is nearly full and the next task has multiple examples, pool adds this task to a batch even if batch size exceeds max_batch_size
• we probably can shutdown TesseractRuntime without using a dedicated mp.Pipe, see also: eventfd

tesseract.utils

• SharedFuture: cancel, set_running_or_notify_cancel

tesseract.tests

• sometimes interrupting tests/benchmark_throughput.py leaves process hanging (only responds to sigterm/sigkill), find & eliminate the cause

[justheuristic]

Most issues are solved. Remaining issues merged into #54