HiveScriptIOSchema

Instance Constructors

new HiveScriptIOSchema(inputRowFormat: Seq[(String, String)], outputRowFormat: Seq[(String, String)], inputSerdeClass: Option[String], outputSerdeClass: Option[String], inputSerdeProps: Seq[(String, String)], outputSerdeProps: Seq[(String, String)], recordReaderClass: Option[String], recordWriterClass: Option[String], schemaLess: Boolean)

Type Members

implicit class typeInfoConversions extends AnyRef

Definition Classes
HiveInspectors

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def asInstanceOf[T0]: T0

Definition Classes
Any
def clone(): AnyRef

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( ... )
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def finalize(): Unit

Attributes
protected[java.lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
def initInputSerDe(input: Seq[Expression]): Option[(AbstractSerDe, ObjectInspector)]
def initOutputSerDe(output: Seq[Attribute]): Option[(AbstractSerDe, StructObjectInspector)]
val inputRowFormat: Seq[(String, String)]
val inputRowFormatMap: Map[String, String]
val inputSerdeClass: Option[String]
val inputSerdeProps: Seq[(String, String)]
def inspectorToDataType(inspector: ObjectInspector): DataType

Definition Classes
HiveInspectors
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
def javaClassToDataType(clz: Class[_]): DataType

Definition Classes
HiveInspectors
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
final def notifyAll(): Unit

Definition Classes
AnyRef
val outputRowFormat: Seq[(String, String)]
val outputRowFormatMap: Map[String, String]
val outputSerdeClass: Option[String]
val outputSerdeProps: Seq[(String, String)]
def recordReader(inputStream: InputStream, conf: Configuration): Option[RecordReader]
val recordReaderClass: Option[String]
def recordWriter(outputStream: OutputStream, conf: Configuration): Option[RecordWriter]
val recordWriterClass: Option[String]
val schemaLess: Boolean
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
def toInspector(expr: Expression): ObjectInspector

Map the catalyst expression to ObjectInspector, however, if the expression is Literal or foldable, a constant writable object inspector returns; Otherwise, we always get the object inspector according to its data type(in catalyst)
Map the catalyst expression to ObjectInspector, however, if the expression is Literal or foldable, a constant writable object inspector returns; Otherwise, we always get the object inspector according to its data type(in catalyst)
expr
Catalyst expression to be mapped
returns
Hive java objectinspector (recursively).

Definition Classes
HiveInspectors
def toInspector(dataType: DataType): ObjectInspector

dataType
Catalyst data type
returns
Hive java object inspector (recursively), not the Writable ObjectInspector We can easily map to the Hive built-in object inspector according to the data type.

Definition Classes
HiveInspectors
def unwrapperFor(field: StructField): (Any, InternalRow, Int) ⇒ Unit

Builds unwrappers ahead of time according to object inspector types to avoid pattern matching and branching costs per row.
Builds unwrappers ahead of time according to object inspector types to avoid pattern matching and branching costs per row.
field
The HiveStructField to create an unwrapper for.
returns
A function that performs in-place updating of a MutableRow. Use the overloaded ObjectInspector version for assignments.

Definition Classes
HiveInspectors
def unwrapperFor(objectInspector: ObjectInspector): (Any) ⇒ Any

Builds unwrappers ahead of time according to object inspector types to avoid pattern matching and branching costs per row.
Builds unwrappers ahead of time according to object inspector types to avoid pattern matching and branching costs per row.
Strictly follows the following order in unwrapping (constant OI has the higher priority): Constant Null object inspector => return null Constant object inspector => extract the value from constant object inspector If object inspector prefers writable => extract writable from data and then get the catalyst type from the writable Extract the java object directly from the object inspector
NOTICE: the complex data type requires recursive unwrapping.
objectInspector
the ObjectInspector used to create an unwrapper.
returns
A function that unwraps data objects. Use the overloaded HiveStructField version for in-place updating of a MutableRow.

Definition Classes
HiveInspectors
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
def wrap(row: Seq[Any], wrappers: Array[(Any) ⇒ Any], cache: Array[AnyRef], dataTypes: Array[DataType]): Array[AnyRef]

Definition Classes
HiveInspectors
def wrap(row: InternalRow, wrappers: Array[(Any) ⇒ Any], cache: Array[AnyRef], dataTypes: Array[DataType]): Array[AnyRef]

Definition Classes
HiveInspectors
def wrap(a: Any, oi: ObjectInspector, dataType: DataType): AnyRef

Definition Classes
HiveInspectors
def wrapperFor(oi: ObjectInspector, dataType: DataType): (Any) ⇒ Any

Wraps with Hive types based on object inspector.
Wraps with Hive types based on object inspector.

Attributes
protected
Definition Classes
HiveInspectors

Related Docs: object HiveScriptIOSchema | package execution

Instance Constructors

Type Members

implicit class typeInfoConversions extends AnyRef

Value Members

final def !=(arg0: Any): Boolean

final def ##(): Int

final def ==(arg0: Any): Boolean

final def asInstanceOf[T0]: T0

def clone(): AnyRef

final def eq(arg0: AnyRef): Boolean

def finalize(): Unit

final def getClass(): Class[_]

def initInputSerDe(input: Seq[Expression]): Option[(AbstractSerDe, ObjectInspector)]

def initOutputSerDe(output: Seq[Attribute]): Option[(AbstractSerDe, StructObjectInspector)]

val inputRowFormat: Seq[(String, String)]

val inputRowFormatMap: Map[String, String]

val inputSerdeClass: Option[String]

val inputSerdeProps: Seq[(String, String)]

def inspectorToDataType(inspector: ObjectInspector): DataType

final def isInstanceOf[T0]: Boolean

def javaClassToDataType(clz: Class[_]): DataType

final def ne(arg0: AnyRef): Boolean

final def notify(): Unit

final def notifyAll(): Unit

val outputRowFormat: Seq[(String, String)]

val outputRowFormatMap: Map[String, String]

val outputSerdeClass: Option[String]

val outputSerdeProps: Seq[(String, String)]

def recordReader(inputStream: InputStream, conf: Configuration): Option[RecordReader]

val recordReaderClass: Option[String]

def recordWriter(outputStream: OutputStream, conf: Configuration): Option[RecordWriter]

val recordWriterClass: Option[String]

val schemaLess: Boolean

final def synchronized[T0](arg0: ⇒ T0): T0

def toInspector(expr: Expression): ObjectInspector

def toInspector(dataType: DataType): ObjectInspector

def unwrapperFor(field: StructField): (Any, InternalRow, Int) ⇒ Unit

def unwrapperFor(objectInspector: ObjectInspector): (Any) ⇒ Any

final def wait(): Unit

final def wait(arg0: Long, arg1: Int): Unit

final def wait(arg0: Long): Unit

def wrap(row: Seq[Any], wrappers: Array[(Any) ⇒ Any], cache: Array[AnyRef], dataTypes: Array[DataType]): Array[AnyRef]

def wrap(row: InternalRow, wrappers: Array[(Any) ⇒ Any], cache: Array[AnyRef], dataTypes: Array[DataType]): Array[AnyRef]

def wrap(a: Any, oi: ObjectInspector, dataType: DataType): AnyRef

def wrapperFor(oi: ObjectInspector, dataType: DataType): (Any) ⇒ Any

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from HiveInspectors

Inherited from AnyRef

Inherited from Any

Ungrouped