Class SparkDistributedContext

java.lang.Object

org.episteme.core.distributed.SparkDistributedContext

All Implemented Interfaces:

DistributedContext

public class SparkDistributedContext
extends Object
implements DistributedContext

Apache Spark-based implementation of DistributedContext.

This class integrates Episteme with Apache Spark for large-scale distributed computing.

Since:

1.0

Author:

Silvere Martin-Michiellot, Gemini AI (Google DeepMind)

Nested Class Summary

Nested classes/interfaces inherited from interface DistributedContext

DistributedContext.Priority

Constructor Summary

Constructors

Constructor	Description
SparkDistributedContext()	Creates a Spark distributed context with default configuration.
SparkDistributedContext(Object sparkConf)	Creates a Spark distributed context.

Method Summary

Modifier and Type	Method	Description
int	getParallelism()	Returns the number of available processing nodes or cores.
<T extends Serializable> List<Future<T>>	invokeAll(List<Callable<T>> tasks)	Submits a collection of tasks for execution.
void	shutdown()	Shuts down the context.
<T extends Serializable> Future<T>	submit(Callable<T> task)	Submits a task for execution with NORMAL priority.

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface DistributedContext

allGather, allGather, barrier, broadcast, broadcast, fence, get, get, put, put, submit

Constructor Details

SparkDistributedContext

public SparkDistributedContext()

Creates a Spark distributed context with default configuration.

SparkDistributedContext

public SparkDistributedContext(Object sparkConf)

Creates a Spark distributed context.

Parameters:

sparkConf - Spark configuration

Method Details

submit

public <T extends Serializable> Future<T> submit(Callable<T> task)

Description copied from interface: DistributedContext

Submits a task for execution with NORMAL priority.

Specified by:

submit in interface DistributedContext

Type Parameters:

T - Result type

Parameters:

task - Task to execute

Returns:

Future representing the result

invokeAll

public <T extends Serializable> List<Future<T>> invokeAll(List<Callable<T>> tasks)

Description copied from interface: DistributedContext

Submits a collection of tasks for execution.

Specified by:

invokeAll in interface DistributedContext

Type Parameters:

T - Result type

Parameters:

tasks - List of tasks

Returns:

List of Futures

getParallelism

public int getParallelism()

Description copied from interface: DistributedContext

Returns the number of available processing nodes or cores.

Specified by:

getParallelism in interface DistributedContext

shutdown

public void shutdown()

Description copied from interface: DistributedContext

Shuts down the context.

Specified by:

shutdown in interface DistributedContext