Stop Thinking, Just Do!

Sung-Soo Kim's Blog

Spark for Unconventional Cores


12 May 2015

Spark-Ucores - Spark for Unconventional Cores

Spark for Unconventional Cores (Codename SparkCL) is an open source unified programming framework for unconventional compute cores. It is based on Java, OpenCL , the Apache Spark framework and Aparapi-UCores


The motivation behind this work is to bring unconventional compute cores such as FPGAs/GPUs/APUs/DSPs and future core types into mainstream programming use. The framework allows equal treatment of different computing devices under the Spark framework and introduces the ability to offload computations to acceleration devices. The new framework is seamlessly integrated into the standard Spark framework via a Java-OpenCL device programming layer which is based on Aparapi and a Spark programming layer that includes new kernel function types and modified Spark transformations and actions. The framework allows a single code base to target any type of compute core that supports OpenCL and easy integration of new core types into a Spark cluster.

How to get started

Installation Notes ——————

  • Built against

    • spark-1.3.0-bin-hadoop2.4

    • OpenJDK 64-Bit 1.7.0_75

  • For eclipse integration

    • Import maven project (from path to spark-ucores)

    • Set JRE System Library to JavaSE 1.7 (*)

      (*) When imported in eclipse luna it was not the default one!


  1. The Spark and Aparapi framworks without which this project would be an order of magnitude more complex.

  2. Altera, Nallatech, HP and Terasic for their support and hardware contributions during various stages of this line of research in the past couple of years.

comments powered by Disqus