Class NativeJavaCVMediaBackend

java.lang.Object

org.episteme.nativ.media.backends.NativeJavaCVMediaBackend

All Implemented Interfaces:

AudioAlgorithmProvider<AudioBuffer>, AudioBackend, VideoAlgorithmProvider, VideoBackend, VisionAlgorithmProvider<BufferedImage>, VisionBackend, AlgorithmProvider, Backend, ComputeBackend, CPUBackend, NativeBackend

@AutoService({Backend.class,ComputeBackend.class,AudioBackend.class,VideoBackend.class,VisionBackend.class,CPUBackend.class,NativeBackend.class})
public class NativeJavaCVMediaBackend
extends Object
implements AudioBackend, VideoBackend, VisionBackend, CPUBackend, NativeBackend

Native Media Backend using JavaCV (OpenCV/FFmpeg) for multi-purpose tasks. Covers Audio, Video and Vision with native acceleration through Bytedeco.

Since:

1.2

Author:

Silvere Martin-Michiellot, Gemini AI (Google DeepMind)

Constructor Summary

Constructors

Constructor	Description
NativeJavaCVMediaBackend()

Method Summary

Modifier and Type	Method	Description
BufferedImage	apply(BufferedImage image, ImageOp<BufferedImage> op)	Applies a generic operation to an image.
AudioBuffer	apply(AudioBuffer audio, AudioOp<AudioBuffer> op)	Applies an operation to audio data.
AudioBuffer	createAudio(Object data, int channels, int sampleRate)
Object	createBackend()	Creates and returns the backend instance (or returns self if it is the backend).
ExecutionContext	createContext()	Creates an execution context for running operations.
BufferedImage	createImage(Object data, int width, int height)	Creates a new image from the given data.
SceneTransitionDetector.Transition	detectMotion(float[][] prev, float[][] curr, float threshold)	Detects motion between two frames.
HardwareAccelerator	getAcceleratorType()	Returns the type of hardware accelerator used by this backend.
String	getAlgorithmType()	Returns the unique category of the algorithm.
String	getBackendName()	Returns the friendly name of this backend instance.
String	getDescription()	Returns a description of the backend.
double	getDuration()
String	getId()	Returns the unique identifier for this backend (e.g., "javafx", "jmol", "cuda").
String	getName()	Returns the display name for UI presentation.
String	getNativeLibraryName()	Returns the name of the native library this backend depends on.
int	getPriority()	Returns the priority for auto-selection (higher = preferred).
float[]	getSpectrum()	Returns the current frequency spectrum.
double	getTime()
String	getType()	Returns the backend type category (e.g., "plotting", "molecular", "tensor", "math").
<T> T	grabFrame()	Grabs the current frame as an image.
boolean	isAvailable()	Checks if this backend is currently available (libraries loaded, etc.).
boolean	isLoaded()	Checks if the native library is loaded and available.
void	load(String path)	Loads an audio resource.
void	pause()
void	play()
BufferedImage	processNative(BufferedImage image, int opCode)
void	shutdown()	Called when the application is shutting down or the backend is being replaced.
void	stop()

Methods inherited from class Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface AlgorithmProvider

description, getMetadata, score

Methods inherited from interface Backend

getAlgorithmProviders, getStatusMessage, isExplicitlyDisabled

Methods inherited from interface CPUBackend

supportsComplexNumbers, supportsFloatingPoint, supportsParallelOps

Methods inherited from interface VideoAlgorithmProvider

detectTransitions

Constructor Details

NativeJavaCVMediaBackend

public NativeJavaCVMediaBackend()

Method Details

isLoaded

public boolean isLoaded()

Description copied from interface: NativeBackend

Checks if the native library is loaded and available.

Specified by:

isLoaded in interface NativeBackend

Returns:

true if the native library has been successfully loaded

getNativeLibraryName

public String getNativeLibraryName()

Description copied from interface: NativeBackend

Returns the name of the native library this backend depends on.

Specified by:

getNativeLibraryName in interface NativeBackend

Returns:

library name (e.g., "openblas", "cuda", "fftw3")

getType

public String getType()

Description copied from interface: Backend

Returns the backend type category (e.g., "plotting", "molecular", "tensor", "math").

Specified by:

getType in interface AudioBackend

Specified by:

getType in interface Backend

Specified by:

getType in interface ComputeBackend

Specified by:

getType in interface CPUBackend

Specified by:

getType in interface VideoBackend

Specified by:

getType in interface VisionBackend

getId

public String getId()

Description copied from interface: Backend

Returns the unique identifier for this backend (e.g., "javafx", "jmol", "cuda").

Specified by:

getId in interface AudioBackend

Specified by:

getId in interface Backend

Specified by:

getId in interface ComputeBackend

Specified by:

getId in interface VideoBackend

Specified by:

getId in interface VisionBackend

getBackendName

public String getBackendName()

Description copied from interface: AudioBackend

Returns the friendly name of this backend instance.

Specified by:

getBackendName in interface AudioBackend

Specified by:

getBackendName in interface VideoBackend

Specified by:

getBackendName in interface VisionBackend

getDescription

public String getDescription()

Description copied from interface: Backend

Returns a description of the backend.

Specified by:

getDescription in interface AudioBackend

Specified by:

getDescription in interface Backend

Specified by:

getDescription in interface ComputeBackend

Specified by:

getDescription in interface VideoBackend

Specified by:

getDescription in interface VisionBackend

isAvailable

public boolean isAvailable()

Description copied from interface: Backend

Checks if this backend is currently available (libraries loaded, etc.).

Default implementation checks Backend.isExplicitlyDisabled() and returns false if it is.

Specified by:

isAvailable in interface AlgorithmProvider

Specified by:

isAvailable in interface AudioBackend

Specified by:

isAvailable in interface Backend

Specified by:

isAvailable in interface VideoBackend

Specified by:

isAvailable in interface VisionBackend

shutdown

public void shutdown()

Description copied from interface: Backend

Called when the application is shutting down or the backend is being replaced. Use this to release hardware resources, close file handles, or stop background threads.

Specified by:

shutdown in interface AlgorithmProvider

Specified by:

shutdown in interface AudioBackend

Specified by:

shutdown in interface Backend

Specified by:

shutdown in interface VideoBackend

Specified by:

shutdown in interface VisionBackend

getPriority

public int getPriority()

Description copied from interface: Backend

Returns the priority for auto-selection (higher = preferred). Used when multiple backends are available.

Specified by:

getPriority in interface AlgorithmProvider

Specified by:

getPriority in interface AudioBackend

Specified by:

getPriority in interface Backend

Specified by:

getPriority in interface VideoBackend

Specified by:

getPriority in interface VisionBackend

apply

public BufferedImage apply(BufferedImage image,
 ImageOp<BufferedImage> op)

Description copied from interface: VisionAlgorithmProvider

Applies a generic operation to an image.

Specified by:

apply in interface VisionAlgorithmProvider<BufferedImage>

getName

public String getName()

Description copied from interface: Backend

Returns the display name for UI presentation.

Specified by:

getName in interface AlgorithmProvider

Specified by:

getName in interface AudioAlgorithmProvider<AudioBuffer>

Specified by:

getName in interface AudioBackend

Specified by:

getName in interface Backend

Specified by:

getName in interface VideoAlgorithmProvider

Specified by:

getName in interface VideoBackend

Specified by:

getName in interface VisionAlgorithmProvider<BufferedImage>

Specified by:

getName in interface VisionBackend

processNative

public BufferedImage processNative(BufferedImage image,
 int opCode)

createImage

public BufferedImage createImage(Object data,
 int width,
 int height)

Description copied from interface: VisionAlgorithmProvider

Creates a new image from the given data.

Specified by:

createImage in interface VisionAlgorithmProvider<BufferedImage>

createContext

public ExecutionContext createContext()

Description copied from interface: ComputeBackend

Creates an execution context for running operations.

The context should be used within a try-with-resources block to ensure proper cleanup of resources.

Specified by:

createContext in interface AudioBackend

Specified by:

createContext in interface ComputeBackend

Specified by:

createContext in interface VideoBackend

Specified by:

createContext in interface VisionBackend

Returns:

new execution context

createBackend

public Object createBackend()

Description copied from interface: Backend

Creates and returns the backend instance (or returns self if it is the backend).

Specified by:

createBackend in interface AudioBackend

Specified by:

createBackend in interface Backend

Specified by:

createBackend in interface ComputeBackend

Specified by:

createBackend in interface VideoBackend

Specified by:

createBackend in interface VisionBackend

Returns:

The backend implementation object

load

public void load(String path)
          throws Exception

Description copied from interface: AudioBackend

Loads an audio resource.

Specified by:

load in interface AudioBackend

Specified by:

load in interface VideoBackend

Parameters:

path - File path or URL

Throws:

Exception

play

public void play()

Specified by:

play in interface AudioBackend

Specified by:

play in interface VideoBackend

pause

public void pause()

Specified by:

pause in interface AudioBackend

Specified by:

pause in interface VideoBackend

stop

public void stop()

Specified by:

stop in interface AudioBackend

Specified by:

stop in interface VideoBackend

getTime

public double getTime()

Specified by:

getTime in interface AudioBackend

Specified by:

getTime in interface VideoBackend

getDuration

public double getDuration()

Specified by:

getDuration in interface AudioBackend

Specified by:

getDuration in interface VideoBackend

grabFrame

public <T> T grabFrame()

Description copied from interface: VideoBackend

Grabs the current frame as an image.

Specified by:

grabFrame in interface VideoBackend

Type Parameters:

T - the image type

Returns:

the current frame

getSpectrum

public float[] getSpectrum()

Description copied from interface: AudioBackend

Returns the current frequency spectrum.

Specified by:

getSpectrum in interface AudioBackend

Returns:

Array of magnitudes (usually 128 or 256 bands)

createAudio

public AudioBuffer createAudio(Object data,
 int channels,
 int sampleRate)

Specified by:

createAudio in interface AudioAlgorithmProvider<AudioBuffer>

Specified by:

createAudio in interface AudioBackend

apply

public AudioBuffer apply(AudioBuffer audio,
 AudioOp<AudioBuffer> op)

Description copied from interface: AudioAlgorithmProvider

Applies an operation to audio data.

Specified by:

apply in interface AudioAlgorithmProvider<AudioBuffer>

Specified by:

apply in interface AudioBackend

getAlgorithmType

public String getAlgorithmType()

Description copied from interface: AlgorithmProvider

Returns the unique category of the algorithm.

Specified by:

getAlgorithmType in interface AlgorithmProvider

Specified by:

getAlgorithmType in interface AudioAlgorithmProvider<AudioBuffer>

Specified by:

getAlgorithmType in interface VideoAlgorithmProvider

Specified by:

getAlgorithmType in interface VisionAlgorithmProvider<BufferedImage>

getAcceleratorType

public HardwareAccelerator getAcceleratorType()

Description copied from interface: ComputeBackend

Returns the type of hardware accelerator used by this backend.

Specified by:

getAcceleratorType in interface AudioBackend

Specified by:

getAcceleratorType in interface ComputeBackend

Specified by:

getAcceleratorType in interface CPUBackend

Specified by:

getAcceleratorType in interface VideoBackend

Specified by:

getAcceleratorType in interface VisionBackend

Returns:

the accelerator type

detectMotion

public SceneTransitionDetector.Transition detectMotion(float[][] prev,
 float[][] curr,
 float threshold)

Description copied from interface: VideoAlgorithmProvider

Detects motion between two frames.

Specified by:

detectMotion in interface VideoAlgorithmProvider

Parameters:

prev - Gray frame (float[][])

curr - Gray frame (float[][])

threshold - Sensitivity threshold

Returns:

Motion result