partitioning

org.apache.spark.sql.functions.partitioning
object partitioning

Partition transform functions for DataFrameWriterV2.partitionedBy, e.g. df.writeTo("t").partitionedBy(partitioning.years(col("ts"))). The deprecated top-level aliases below forward here.

Attributes

Graph
Supertypes
class Object
trait Matchable
class Any
Self type

Members list

Value members

Concrete methods

def bucket(numBuckets: Column, e: Column): Column

A transform for any type that partitions by a hash of the input column.

A transform for any type that partitions by a hash of the input column.

Attributes

def bucket(numBuckets: Int, e: Column): Column

A transform for any type that partitions by a hash of the input column.

A transform for any type that partitions by a hash of the input column.

Attributes

def days(e: Column): Column

A transform for timestamps and dates to partition data into days.

A transform for timestamps and dates to partition data into days.

Attributes

def hours(e: Column): Column

A transform for timestamps to partition data into hours.

A transform for timestamps to partition data into hours.

Attributes

def months(e: Column): Column

A transform for timestamps and dates to partition data into months.

A transform for timestamps and dates to partition data into months.

Attributes

def years(e: Column): Column

A transform for timestamps and dates to partition data into years.

A transform for timestamps and dates to partition data into years.

Attributes