Parallelize function calls on LSF cluster.

Run function for each row of input dataframe in LSF job.

cluster_apply(
  df_iter,
  func,
  .bsub_params = c("-n", "2", "-W", "24:00", "-R", "rusage[mem=10000]"),
  .tempdir = ".",
  .packages = c(),
  ...
)

Arguments

df_iter

Dataframe over whose rows to iterate.

func

Function to apply to each dataframe row. Its arguments must be all dataframe columns.

.bsub_params

Parameters to pass to `bsub` during job submission.

.tempdir

Location to store auxiliary files in.

.packages

Packages to import in each job.

...

Extra arguments for function.

Value

Dataframe created by concatenating results of each function call.

Examples

if (FALSE) {
foo <- 42
cluster_apply(
  data.frame(i = seq_len(3), group = c("A", "B", "C")),
  function(i, group) {
    log_debug("hello")
    data.frame(group = group, i = i, foo = foo, result = foo + 2 * i)
  },
  .packages = c(logger)
)
}