keystoneml.nodes.learning

ColumnPCAEstimator

case class ColumnPCAEstimator(dims: Int, numMachines: Option[Int] = scala.None, cpuWeight: Double = 3.8E-4, memWeight: Double = 0.29, networkWeight: Double = 1.32) extends OptimizableEstimator[DenseMatrix[Float], DenseMatrix[Float]] with Product with Serializable

Estimates a PCA model for dimensionality reduction based on a sample of a larger input dataset. Treats each column of the input matrices like a separate DenseVector input to PCAEstimator or DistributedPCAEstimator.

Automatically decides between distributed and local implementations when node-level optimization is enabled. The default weights were determined empirically via results run on a 16 r3.4xlarge node cluster.

dims

Dimensions to reduce input dataset to.

numMachines
cpuWeight
memWeight
networkWeight

Linear Supertypes
Product, Equals, OptimizableEstimator[DenseMatrix[Float], DenseMatrix[Float]], Optimizable, Estimator[DenseMatrix[Float], DenseMatrix[Float]], EstimatorOperator, Serializable, Serializable, Operator, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. ColumnPCAEstimator
  2. Product
  3. Equals
  4. OptimizableEstimator
  5. Optimizable
  6. Estimator
  7. EstimatorOperator
  8. Serializable
  9. Serializable
  10. Operator
  11. AnyRef
  12. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new ColumnPCAEstimator(dims: Int, numMachines: Option[Int] = scala.None, cpuWeight: Double = 3.8E-4, memWeight: Double = 0.29, networkWeight: Double = 1.32)

    dims

    Dimensions to reduce input dataset to.

    numMachines
    cpuWeight
    memWeight
    networkWeight

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  6. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  7. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  8. val cpuWeight: Double

  9. val default: DistributedColumnPCAEstimator

    Definition Classes
    ColumnPCAEstimator → OptimizableEstimator

  10. val dims: Int

    Dimensions to reduce input dataset to.

  11. val distributedEstimator: DistributedColumnPCAEstimator

  12. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  13. def execute(deps: Seq[Expression]): TransformerExpression

    Definition Classes
    EstimatorOperator → Operator

  14. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  15. def fit(data: RDD[DenseMatrix[Float]]): Transformer[DenseMatrix[Float], DenseMatrix[Float]]

    The type-safe method that ML developers need to implement when writing new Estimators.

    The type-safe method that ML developers need to implement when writing new Estimators.

    data

    The estimator's training data.

    returns

    A new transformer

    Definition Classes
    OptimizableEstimator → Estimator

  16. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  17. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  18. def label: String

    Definition Classes
    Operator

  19. val localEstimator: LocalColumnPCAEstimator

  20. val memWeight: Double

  21. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  22. val networkWeight: Double

  23. final def notify(): Unit

    Definition Classes
    AnyRef

  24. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  25. val numMachines: Option[Int]

  26. def optimize(sample: RDD[DenseMatrix[Float]], numPerPartition: Map[Int, Int]): Estimator[DenseMatrix[Float], DenseMatrix[Float]]

    Definition Classes
    ColumnPCAEstimator → OptimizableEstimator

  27. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  28. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  29. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  30. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  31. final def withData(data: PipelineDataset[DenseMatrix[Float]]): Pipeline[DenseMatrix[Float], DenseMatrix[Float]]

    Constructs a pipeline that fits this estimator to training data, then applies the resultant transformer to the Pipeline input.

    Constructs a pipeline that fits this estimator to training data, then applies the resultant transformer to the Pipeline input.

    data

    The training data

    returns

    A pipeline that fits this estimator and applies the result to inputs.

    Definition Classes
    Estimator

  32. final def withData(data: RDD[DenseMatrix[Float]]): Pipeline[DenseMatrix[Float], DenseMatrix[Float]]

    Constructs a pipeline that fits this estimator to training data, then applies the resultant transformer to the Pipeline input.

    Constructs a pipeline that fits this estimator to training data, then applies the resultant transformer to the Pipeline input.

    data

    The training data

    returns

    A pipeline that fits this estimator and applies the result to inputs.

    Definition Classes
    Estimator

Inherited from Product

Inherited from Equals

Inherited from OptimizableEstimator[DenseMatrix[Float], DenseMatrix[Float]]

Inherited from Optimizable

Inherited from Estimator[DenseMatrix[Float], DenseMatrix[Float]]

Inherited from EstimatorOperator

Inherited from Serializable

Inherited from Serializable

Inherited from Operator

Inherited from AnyRef

Inherited from Any

Ungrouped