Specializing parallel collections with customized work-stealing and Scala Macros and developing micro-benchmarking suite for collections.

by Dmytro Petrashko for Scala Team

This project aims at creating new parallel collection implementations, that take advantage of macros to generate a specialized parallel operation instance for a particular invocation. This specialized parallel operation instance avoids abstraction overheads such as boxing, dynamic dispatch or iterator state updates. Orthogonally, to improve scalability for data-parallel operations on highly irregular workloads, work-stealing tree scheduling will be used. Work-stealing tree scheduling is a novel scheduling technique for load-balancing parallel execution of highly irregular data-parallel workloads currently being implemented as part of the Parallel Collections for the Scala programming language. To ensure scalability and the appropriate performance of the new parallel collection implementations, a suite of micro-benchmarks and benchmark applications is planned to be developed using the ScalaMeter performance regression testing tool previously developed as part of the Scala programming language project.