dataset_shard

This dataset operator is very useful when running distributed training, as it
allows each worker to read a unique subset.

Interface to 'TensorFlow' Datasets, a high-level library for
building complex input pipelines from simple, re-usable pieces.
See <https://www.tensorflow.org/guide> for additional
details.

Tomasz Kalinowski

tfdatasets

Interface to 'TensorFlow' Datasets

Daniel Falbel

JJ Allaire

Yuan Tang

Kevin Ushey

RStudio 

 Google Inc.

dataset_shard function

<dl><dt>dataset</dt>
<dd>A dataset</dd>
<dt>num_shards</dt>
<dd>A integer representing the number of shards operating in
parallel.</dd>
<dt>index</dt>
<dd>A integer, representing the worker index.</dd></dl>

Arguments

Creates a dataset that includes only 1 / num_shards of this dataset. — dataset_shard

<dl>

<dt>dataset</dt>
<dd>A dataset</dd>


<dt>num_shards</dt>
<dd>A integer representing the number of shards operating in
parallel.</dd>


<dt>index</dt>
<dd>A integer, representing the worker index.</dd>

</dl>

dataset_shard: Creates a dataset that includes only 1 / num_shards of this dataset.

Description

Usage

Value

Arguments