Lately I have been converting some GPGPU algorithms from CUDA into C# and the Task Parallel Library, e.g., tiled reduction. However, I noticed a significant problem in the efficiency of the System.Threading.Barrier class for task synchronization. This problem was also noted by Andrea Esuli here, who proposed an incomplete solution.
As stated in the documentation, “[Barrier] enables multiple tasks to cooperatively work on an algorithm in parallel through multiple phases.” Although it does work with System.Threading.Tasks.Task, when the participant count for the Barrier is more than four–the number of cores in my CPU–the method SignalAndWait() blocks with a time out of one second, probably because Barrier blocks a thread serving multiple task. The example below illustrates the problem using Barrier and Task with a large number of tasks spawned. Notwithstanding the efficiency, Barrier.SignalAndWait() is correct.
As noted by Esuli, a solution is to try to coerce each task to be in its own thread using TaskCreationOptions.LongRunning during construction. Unfortunately, this seems to only work outside a debugging session, or for problems with a participant count up to 8 when debugging the program. This limits its use. Esuli also notes to use ManualResetEvent instead of Barrier. Unfortunately, that solution isn’t correct because ManualResetEvent.Set() does not block the execution of the code following the Set() in a task the same way as Barrier.
The best solution I came up with is a task-level barrier, implemented as follows. (1) Replace “SignalAndWait();” in each task with “await Task.Yield();”. (2) Create a custom task scheduler that maintains a special queue for the execution of the asynchronous continuation tasks. The scheduler examines the context of the call to QueueTask(Task task). If the call is for a continuation, place the task on a special queue to be executed after all “normal” tasks. (3) When the final continuation task has been queued, transfer the continuation tasks the the normal queue. (4) Wait for all normal tasks to complete using Task.WaitAll(…). (5) Complete all continuation tasks using RunSynchronously().