workflow

Type Members

1. class AutoCacheRule extends Rule with Logging

2. class AutoCachingOptimizer extends Optimizer

   Optimizes a Pipeline DAG, with auto-caching

3. trait Chainable[A, B] extends AnyRef

   This trait provides methods to chain an object with Estimators, LabelEstimators, and other Chainables to construct Pipelines.

4. abstract class Estimator[A, B] extends EstimatorOperator

   An estimator has a fitRDD method which takes an input and emits a Transformer

5. class FittedPipeline[A, B] extends Chainable[A, B] with Serializable

   This is the result of fitting a Pipeline.

6. case class Identity[T]()(implicit evidence$1: ClassTag[T]) extends Transformer[T, T] with Product with Serializable

   This transformer performs a no-op on its input.

7. abstract class LabelEstimator[A, B, L] extends EstimatorOperator

   A LabelEstimator has a fitRDDs method which takes input data and input labels, and emits a Transformer.

8. class NodeOptimizationRule extends Rule

   Node-level optimization, such as selecting a Linear Solver

9. sealed trait Optimizable extends AnyRef

10. abstract class OptimizableEstimator[A, B] extends Estimator[A, B] with Optimizable

    Represents a node-level optimizable Estimator and its optimization rules

11. abstract class OptimizableLabelEstimator[A, B, L] extends LabelEstimator[A, B, L] with Optimizable

    Represents a node-level optimizable LabelEstimator and its optimization rules

12. abstract class OptimizableTransformer[A, B] extends Transformer[A, B] with Optimizable

    Represents a node-level optimizable transformer and its optimization rules

13. abstract class Optimizer extends RuleExecutor

14. class Pipeline[A, B] extends Chainable[A, B]

    A Pipeline takes data as input (single item or an RDD), and outputs some transformation of that data.

15. class PipelineDataset[T] extends PipelineResult[RDD[T]]

    This class is a lazy wrapper around the output of a pipeline that was passed an RDD as input.

16. class PipelineDatum[T] extends PipelineResult[T]

    This class is a lazy wrapper around the output of a pipeline that was passed a single datum as input.

17. class PipelineEnv extends AnyRef

    PipelineEnv is an environment shared by multiple Pipelines, containing variables such as the Prefix state table and the current Pipeline Optimizer.

18. abstract class PipelineResult[T] extends AnyRef

    A PipelineResult is a lazy wrapper around the result of applying a Pipeline to data.

19. case class Profile(ns: Long, rddMem: Long, driverMem: Long) extends Product with Serializable

20. abstract class Rule extends AnyRef

    Represents a DAG transformation rule: A transformation from one DAG to a differently-executed but logically equivalent DAG.

21. abstract class RuleExecutor extends Logging

22. case class SampleProfile(scale: Long, profile: Profile) extends Product with Serializable

23. abstract class Transformer[A, B] extends TransformerOperator with Chainable[A, B]

    Transformers are operators that may be applied both to single input items and to RDDs of input items.

24. case class TransformerChain[A, B, C](first: Transformer[A, B], second: Transformer[B, C])(implicit evidence$1: ClassTag[C]) extends Transformer[A, C] with Product with Serializable

    A chain of two Transformers in a row (as a Transformer)

25. case class TransformerEstimatorChain[A, B, C](first: Transformer[A, B], second: Estimator[B, C])(implicit evidence$2: ClassTag[C]) extends Estimator[A, C] with Product with Serializable

    A chain of a Transformer followed by an Estimator (as an Estimator)

26. case class TransformerLabelEstimatorChain[A, B, C, L](first: Transformer[A, B], second: LabelEstimator[B, C, L])(implicit evidence$3: ClassTag[C]) extends LabelEstimator[A, C, L] with Product with Serializable

    A chain of a Transformer followed by a LabelEstimator (as a LabelEstimator)

27. trait WeightedNode extends AnyRef

    A mix-in that attaches a weight to a node that represents how often it must iterate over its input.

28. trait WeightedOperator extends AnyRef

    A mix-in that attaches a weight to an operator that represents how often it must iterate over its input.