Read the schema of a Spark DataFrame.
sdf_schema(x, expand_nested_cols = FALSE, expand_struct_cols = FALSE)
An R
list
, with each list
element describing the
name
and type
of a column.
A spark_connection
, ml_pipeline
, or a tbl_spark
.
Whether to expand columns containing nested array of structs (which are usually created by tidyr::nest on a Spark data frame)
Whether to expand columns containing structs
The type
column returned gives the string representation of the
underlying Spark type for that column; for example, a vector of numeric
values would be returned with the type "DoubleType"
. Please see the
Spark Scala API Documentation
for information on what types are available and exposed by Spark.