harness-tools/docs/todos.md

Todos

Investigate transient concurrent NeMo benchmark failures

During Dictate Phase 0 on 26AUG2026, repeated overlapping throwaway benchmark processes produced two one-off failures in the external /srv/data/tools/parakeet-v3/.venv environment:

1. ASRModel.restore_from() raised MemoryError while OmegaConf rebuilt joint.vocabulary. The exact clean/noisy/silence harness passed on its immediate rerun.
2. A later production-adapter check raised ValueError: bad marshal data (invalid reference) while importing torch._tensor. An immediate standalone import torch succeeded, and the cached _tensor.cpython-312.pyc passed marshal.loads().

The production service serializes model loading and inference and therefore does not intentionally launch concurrent NeMo processes. Before treating concurrent workers as supported, build a repeatable cold-load stress loop, monitor process and cgroup memory, disable bytecode writes, and determine whether the failures come from NeMo/OmegaConf resource pressure or concurrent access to the external environment's bytecode cache.