edu.snu.coral.compiler.frontend.spark.sql

Class Dataset<T>

java.lang.Object

org.apache.spark.sql.Dataset<T>

edu.snu.coral.compiler.frontend.spark.sql.Dataset<T>

Type Parameters:

T - type of the data.

All Implemented Interfaces:

Serializable

public final class Dataset<T>
extends org.apache.spark.sql.Dataset<T>

A dataset component: it represents relational data.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.spark.sql.Dataset

org$apache$spark$sql$Dataset$$encoder

Method Summary

Modifier and Type	Method and Description
static <U> Dataset<U>	from(org.apache.spark.sql.Dataset<U> dataset) Using the immutable property of datasets, we can downcast spark datasets to our class using this function.
JavaRDD<T>	javaRDD() Create a javaRDD component from this data set.

Methods inherited from class org.apache.spark.sql.Dataset

agg, agg, agg, agg, agg, alias, alias, apply, as, as, as, cache, checkpoint, checkpoint, coalesce, col, collect, collectAsList, collectToPython, columns, count, createGlobalTempView, createOrReplaceGlobalTempView, createOrReplaceTempView, createTempView, crossJoin, cube, cube, cube, cube, describe, describe, distinct, drop, drop, drop, drop, dropDuplicates, dropDuplicates, dropDuplicates, dropDuplicates, dropDuplicates, dtypes, except, explain, explain, explode, explode, exprEnc, filter, filter, filter, filter, first, flatMap, flatMap, foreach, foreach, foreachPartition, foreachPartition, groupBy, groupBy, groupBy, groupBy, groupByKey, groupByKey, head, head, hint, hint, inputFiles, intersect, isLocal, isStreaming, javaToPython, join, join, join, join, join, join, joinWith, joinWith, limit, logicalPlan, map, map, mapPartitions, mapPartitions, mapPartitionsInR, na, numericColumns, ofRows, orderBy, orderBy, orderBy, orderBy, org$apache$spark$sql$Dataset$$aggregatableColumns, org$apache$spark$sql$Dataset$$boundEnc, org$apache$spark$sql$Dataset$$classTag, org$apache$spark$sql$Dataset$$collectFromPlan, org$apache$spark$sql$Dataset$$createTempViewCommand, org$apache$spark$sql$Dataset$$rowFunction$1, org$apache$spark$sql$Dataset$$timeZone$1, org$apache$spark$sql$Dataset$$withPlan, persist, persist, printSchema, queryExecution, randomSplit, randomSplit, randomSplit, randomSplitAsList, rdd, reduce, reduce, registerTempTable, repartition, repartition, repartition, repartition, repartition, resolve, rollup, rollup, rollup, rollup, sample, sample, schema, select, select, select, select, select, select, select, select, select, selectExpr, selectExpr, selectUntyped, show, show, show, show, show, showString, showString$default$2, sort, sort, sort, sort, sortWithinPartitions, sortWithinPartitions, sortWithinPartitions, sortWithinPartitions, sparkSession, sqlContext, stat, storageLevel, take, takeAsList, toDF, toDF, toDF, toJavaRDD, toJSON, toLocalIterator, toPythonIterator, toString, transform, union, unionAll, unpersist, unpersist, where, where, withColumn, withColumn, withColumnRenamed, withWatermark, write, writeStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

from

public static <U> Dataset<U> from(org.apache.spark.sql.Dataset<U> dataset)

Using the immutable property of datasets, we can downcast spark datasets to our class using this function.

Type Parameters:

U - type of the dataset.

Parameters:

dataset - the Spark dataset.

Returns:

our dataset class.

javaRDD

public JavaRDD<T> javaRDD()

Create a javaRDD component from this data set.

Overrides:

javaRDD in class org.apache.spark.sql.Dataset<T>

Returns:

the new javaRDD component.