spark_auto_broadcast_join_threshold

Configures the maximum size in bytes for a table that will be broadcast to all worker nodes
when performing a join. By setting this value to -1 broadcasting can be disabled. Note that
currently statistics are only supported for Hive Metastore tables where the command
`ANALYZE TABLE &lt;tableName&gt; COMPUTE STATISTICS noscan` has been run, and file-based data source
tables where the statistics are computed directly on the files of data.

R interface to Apache Spark, a fast and general
engine for big data processing, see <https://spark.apache.org/>. This
package supports connecting to local and remote Apache Spark clusters,
provides a 'dplyr' compatible back-end, and provides an interface to
Spark's built-in machine learning algorithms.

Edgar Ruiz

sparklyr

R Interface to Apache Spark

Javier Luraschi

Kevin Kuo

Kevin Ushey

JJ Allaire

Samuel Macedo

Hossein Falaki

Lu Wang

Andy Zhang

Yitao Li

Jozef Hajnala

Maciej Szymkiewicz

Wil Davis

 RStudio

 The Apache Software Foundation

spark_auto_broadcast_join_threshold function

<dl><dt>sc</dt>
<dd>A <code>spark_connection</code>.</dd>
<dt>threshold</dt>
<dd>Maximum size in bytes for a table that will be broadcast to all worker nodes
when performing a join. Defaults to <code>NULL</code> to retrieve configuration entries.</dd></dl>

Arguments

Retrieves or sets the auto broadcast join threshold — spark_auto_broadcast_join_threshold

<dl>

<dt>sc</dt>
<dd>A <code>spark_connection</code>.</dd>


<dt>threshold</dt>
<dd>Maximum size in bytes for a table that will be broadcast to all worker nodes
when performing a join. Defaults to <code>NULL</code> to retrieve configuration entries.</dd>

</dl>

Retrieves or sets the auto broadcast join threshold

spark_auto_broadcast_join_threshold: Retrieves or sets the auto broadcast join threshold

Description

Usage

Arguments

See Also