stream_watermark

Ensures a stream has a watermark defined, which is required for some
operations over streams.

R interface to Apache Spark, a fast and general
engine for big data processing, see <https://spark.apache.org/>. This
package supports connecting to local and remote Apache Spark clusters,
provides a 'dplyr' compatible back-end, and provides an interface to
Spark's built-in machine learning algorithms.

Edgar Ruiz

sparklyr

R Interface to Apache Spark

Javier Luraschi

Kevin Kuo

Kevin Ushey

JJ Allaire

Samuel Macedo

Hossein Falaki

Lu Wang

Andy Zhang

Yitao Li

Jozef Hajnala

Maciej Szymkiewicz

Wil Davis

 RStudio

 The Apache Software Foundation

stream_watermark function

<dl><dt>x</dt>
<dd>An object coercable to a Spark Streaming DataFrame.</dd>
<dt>column</dt>
<dd>The name of the column that contains the event time of the row,
if the column is missing, a column with the current time will be added.</dd>
<dt>threshold</dt>
<dd>The minimum delay to wait to data to arrive late, defaults
to ten minutes.</dd></dl>

Arguments

Watermark Stream — stream_watermark

<dl>

<dt>x</dt>
<dd>An object coercable to a Spark Streaming DataFrame.</dd>


<dt>column</dt>
<dd>The name of the column that contains the event time of the row,
if the column is missing, a column with the current time will be added.</dd>


<dt>threshold</dt>
<dd>The minimum delay to wait to data to arrive late, defaults
to ten minutes.</dd>

</dl>

Watermark Stream

stream_watermark: Watermark Stream

Description

Usage

Arguments