Trait/Object

org.apache.spark.sql

Row

Related Docs: object Row | package sql

Permalink

trait Row extends Serializable

Represents one row of output from a relational operator. Allows both generic access by ordinal, which will incur boxing overhead for primitives, as well as native primitive access.

It is invalid to use the native primitive interface to retrieve a value that is null, instead a user must check isNullAt before attempting to retrieve a value that might be null.

To create a new Row, use RowFactory.create() in Java or Row.apply() in Scala.

A Row object can be constructed by providing field values. Example:

import org.apache.spark.sql._

// Create a Row from values.
Row(value1, value2, value3, ...)
// Create a Row from a Seq of values.
Row.fromSeq(Seq(value1, value2, ...))

A value of a row can be accessed through both generic access by ordinal, which will incur boxing overhead for primitives, as well as native primitive access. An example of generic access by ordinal:

import org.apache.spark.sql._

val row = Row(1, true, "a string", null)
// row: Row = [1,true,a string,null]
val firstValue = row(0)
// firstValue: Any = 1
val fourthValue = row(3)
// fourthValue: Any = null

For native primitive access, it is invalid to use the native primitive interface to retrieve a value that is null, instead a user must check isNullAt before attempting to retrieve a value that might be null. An example of native primitive access:

// using the row from the previous example.
val firstValue = row.getInt(0)
// firstValue: Int = 1
val isNull = row.isNullAt(3)
// isNull: Boolean = true

In Scala, fields in a Row object can be extracted in a pattern match. Example:

import org.apache.spark.sql._

val pairs = sql("SELECT key, value FROM src").rdd.map {
  case Row(key: Int, value: String) =>
    key -> value
}
Source
Row.scala
Linear Supertypes
Serializable, Serializable, AnyRef, Any
Known Subclasses
MutableAggregationBuffer
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Row
  2. Serializable
  3. Serializable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Abstract Value Members

  1. abstract def copy(): Row

    Permalink

    Make a copy of the current Row object.

  2. abstract def get(i: Int): Any

    Permalink

    Returns the value at position i.

    Returns the value at position i. If the value is null, null is returned. The following is a mapping between Spark SQL types and return types:

    BooleanType -> java.lang.Boolean
ByteType -> java.lang.Byte
ShortType -> java.lang.Short
IntegerType -> java.lang.Integer
FloatType -> java.lang.Float
DoubleType -> java.lang.Double
StringType -> String
DecimalType -> java.math.BigDecimal

DateType -> java.sql.Date
TimestampType -> java.sql.Timestamp

BinaryType -> byte array
ArrayType -> scala.collection.Seq (use getList for java.util.List)
MapType -> scala.collection.Map (use getJavaMap for java.util.Map)
StructType -> org.apache.spark.sql.Row

  3. abstract def length: Int

    Permalink

    Number of elements in the Row.

Concrete Value Members

  1. final def !=(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  2. final def ##(): Int

    Permalink
    Definition Classes
    AnyRef → Any

  3. final def ==(arg0: Any): Boolean

    Permalink
    Definition Classes
    AnyRef → Any

  4. def anyNull: Boolean

    Permalink

    Returns true if there are any NULL values in this row.

  5. def apply(i: Int): Any

    Permalink

    Returns the value at position i.

    Returns the value at position i. If the value is null, null is returned. The following is a mapping between Spark SQL types and return types:

    BooleanType -> java.lang.Boolean
ByteType -> java.lang.Byte
ShortType -> java.lang.Short
IntegerType -> java.lang.Integer
FloatType -> java.lang.Float
DoubleType -> java.lang.Double
StringType -> String
DecimalType -> java.math.BigDecimal

DateType -> java.sql.Date
TimestampType -> java.sql.Timestamp

BinaryType -> byte array
ArrayType -> scala.collection.Seq (use getList for java.util.List)
MapType -> scala.collection.Map (use getJavaMap for java.util.Map)
StructType -> org.apache.spark.sql.Row

  6. final def asInstanceOf[T0]: T0

    Permalink
    Definition Classes
    Any

  7. def clone(): AnyRef

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  8. final def eq(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  9. def equals(o: Any): Boolean

    Permalink
    Definition Classes
    Row → AnyRef → Any

  10. def fieldIndex(name: String): Int

    Permalink

    Returns the index of a given field name.

    Returns the index of a given field name.

    Exceptions thrown

    IllegalArgumentException when a field name does not exist.

    UnsupportedOperationException when schema is not defined.

  11. def finalize(): Unit

    Permalink
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  12. def getAs[T](fieldName: String): T

    Permalink

    Returns the value of a given fieldName.

    Returns the value of a given fieldName. For primitive types if value is null it returns 'zero value' specific for primitive ie. 0 for Int - use isNullAt to ensure that value is not null

    Exceptions thrown

    ClassCastException when data type does not match.

    IllegalArgumentException when fieldName do not exist.

    UnsupportedOperationException when schema is not defined.

  13. def getAs[T](i: Int): T

    Permalink

    Returns the value at position i.

    Returns the value at position i. For primitive types if value is null it returns 'zero value' specific for primitive ie. 0 for Int - use isNullAt to ensure that value is not null

    Exceptions thrown

    ClassCastException when data type does not match.

  14. def getBoolean(i: Int): Boolean

    Permalink

    Returns the value at position i as a primitive boolean.

    Returns the value at position i as a primitive boolean.

    Exceptions thrown

    ClassCastException when data type does not match.

    NullPointerException when value is null.

  15. def getByte(i: Int): Byte

    Permalink

    Returns the value at position i as a primitive byte.

    Returns the value at position i as a primitive byte.

    Exceptions thrown

    ClassCastException when data type does not match.

    NullPointerException when value is null.

  16. final def getClass(): Class[_]

    Permalink
    Definition Classes
    AnyRef → Any

  17. def getDate(i: Int): Date

    Permalink

    Returns the value at position i of date type as java.sql.Date.

    Returns the value at position i of date type as java.sql.Date.

    Exceptions thrown

    ClassCastException when data type does not match.

  18. def getDecimal(i: Int): BigDecimal

    Permalink

    Returns the value at position i of decimal type as java.math.BigDecimal.

    Returns the value at position i of decimal type as java.math.BigDecimal.

    Exceptions thrown

    ClassCastException when data type does not match.

  19. def getDouble(i: Int): Double

    Permalink

    Returns the value at position i as a primitive double.

    Returns the value at position i as a primitive double.

    Exceptions thrown

    ClassCastException when data type does not match.

    NullPointerException when value is null.

  20. def getFloat(i: Int): Float

    Permalink

    Returns the value at position i as a primitive float.

    Returns the value at position i as a primitive float. Throws an exception if the type mismatches or if the value is null.

    Exceptions thrown

    ClassCastException when data type does not match.

    NullPointerException when value is null.

  21. def getInt(i: Int): Int

    Permalink

    Returns the value at position i as a primitive int.

    Returns the value at position i as a primitive int.

    Exceptions thrown

    ClassCastException when data type does not match.

    NullPointerException when value is null.

  22. def getJavaMap[K, V](i: Int): Map[K, V]

    Permalink

    Returns the value at position i of array type as a java.util.Map.

    Returns the value at position i of array type as a java.util.Map.

    Exceptions thrown

    ClassCastException when data type does not match.

  23. def getList[T](i: Int): List[T]

    Permalink

    Returns the value at position i of array type as java.util.List.

    Returns the value at position i of array type as java.util.List.

    Exceptions thrown

    ClassCastException when data type does not match.

  24. def getLong(i: Int): Long

    Permalink

    Returns the value at position i as a primitive long.

    Returns the value at position i as a primitive long.

    Exceptions thrown

    ClassCastException when data type does not match.

    NullPointerException when value is null.

  25. def getMap[K, V](i: Int): Map[K, V]

    Permalink

    Returns the value at position i of map type as a Scala Map.

    Returns the value at position i of map type as a Scala Map.

    Exceptions thrown

    ClassCastException when data type does not match.

  26. def getSeq[T](i: Int): Seq[T]

    Permalink

    Returns the value at position i of array type as a Scala Seq.

    Returns the value at position i of array type as a Scala Seq.

    Exceptions thrown

    ClassCastException when data type does not match.

  27. def getShort(i: Int): Short

    Permalink

    Returns the value at position i as a primitive short.

    Returns the value at position i as a primitive short.

    Exceptions thrown

    ClassCastException when data type does not match.

    NullPointerException when value is null.

  28. def getString(i: Int): String

    Permalink

    Returns the value at position i as a String object.

    Returns the value at position i as a String object.

    Exceptions thrown

    ClassCastException when data type does not match.

  29. def getStruct(i: Int): Row

    Permalink

    Returns the value at position i of struct type as a Row object.

    Returns the value at position i of struct type as a Row object.

    Exceptions thrown

    ClassCastException when data type does not match.

  30. def getTimestamp(i: Int): Timestamp

    Permalink

    Returns the value at position i of date type as java.sql.Timestamp.

    Returns the value at position i of date type as java.sql.Timestamp.

    Exceptions thrown

    ClassCastException when data type does not match.

  31. def getValuesMap[T](fieldNames: Seq[String]): Map[String, T]

    Permalink

    Returns a Map(name -> value) for the requested fieldNames For primitive types if value is null it returns 'zero value' specific for primitive ie.

    Returns a Map(name -> value) for the requested fieldNames For primitive types if value is null it returns 'zero value' specific for primitive ie. 0 for Int - use isNullAt to ensure that value is not null

    Exceptions thrown

    ClassCastException when data type does not match.

    IllegalArgumentException when fieldName do not exist.

    UnsupportedOperationException when schema is not defined.

  32. def hashCode(): Int

    Permalink
    Definition Classes
    Row → AnyRef → Any

  33. final def isInstanceOf[T0]: Boolean

    Permalink
    Definition Classes
    Any

  34. def isNullAt(i: Int): Boolean

    Permalink

    Checks whether the value at position i is null.

  35. def mkString(start: String, sep: String, end: String): String

    Permalink

    Displays all elements of this traversable or iterator in a string using start, end, and separator strings.

  36. def mkString(sep: String): String

    Permalink

    Displays all elements of this sequence in a string using a separator string.

  37. def mkString: String

    Permalink

    Displays all elements of this sequence in a string (without a separator).

  38. final def ne(arg0: AnyRef): Boolean

    Permalink
    Definition Classes
    AnyRef

  39. final def notify(): Unit

    Permalink
    Definition Classes
    AnyRef

  40. final def notifyAll(): Unit

    Permalink
    Definition Classes
    AnyRef

  41. def schema: StructType

    Permalink

    Schema for the row.

  42. def size: Int

    Permalink

    Number of elements in the Row.

  43. final def synchronized[T0](arg0: ⇒ T0): T0

    Permalink
    Definition Classes
    AnyRef

  44. def toSeq: Seq[Any]

    Permalink

    Return a Scala Seq representing the row.

    Return a Scala Seq representing the row. Elements are placed in the same order in the Seq.

  45. def toString(): String

    Permalink
    Definition Classes
    Row → AnyRef → Any

  46. final def wait(): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  47. final def wait(arg0: Long, arg1: Int): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  48. final def wait(arg0: Long): Unit

    Permalink
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped