The progress bar just got a lot cheaper

The pbapply R package that adds progress bar to vectorized functions has been know to accumulate overhead when calling parallel::mclapply with forking (see this post for more background on the issue). Strangely enough, a GitHub issue held the key to the solution that I am going to outline below. Long story short: forking is no longer expensive with pbapply, and as it turns out, it never was. The issue mentioned parallel::makeForkCluster as the way to set up a Fork cluster, which, according to the help page, ‘is merely a stub on Windows. On Unix-alike platforms it creates the worker process by forking’.So I looked at some timings starting with one of the examples on the ?pbapply help page: library(pbapply) set.seed(1234) n <- 200 x <- rnorm(n) y <- rnorm(n, crossprod(t(model.matrix(~ x)), c(0, 1)), sd = 0.5) d <- data.frame(y, x)…
Original Post: The progress bar just got a lot cheaper

How to write and document %special% functions in R

I spend a considerable time of my working hours with data processing where I often use the %in% R function as x %in% y. Whenever I need the negation of that, I used to write !(x %in% y). Not much of a hassle, but still, wouldn’t it be nicer to have x %notin% y instead? So I decided to code…
Original Post: How to write and document %special% functions in R

Progress bar overhead comparisons

As a testament to my obsession with progress bars in R, here isa quick investigation about the overhead cost ofdrawing a progress bar during computations in R.I compared several approaches includingmy pbapply and Hadley Wickham’s plyr. Let’s compare the good old lapply function from base R,a custom-made variant called lapply_pb that wasproposed here, l_ply from the plyr package,and finally pblapply…
Original Post: Progress bar overhead comparisons

How to add pbapply to R packages

As of today, there are 20 R packages that reverse depend/import/suggest (3/14/3)the pbapply package. Current and future package developerswho decide to incorporate the progress bar using pbapplymight want to customize the type and style of the progress barin their packages to better suit the needs of certain functionsor to create a distinctive look.Here is a quick guide to help in…
Original Post: How to add pbapply to R packages

What is the cost of a progress bar in R?

The pbapply R package adds progress bar to vectorized functions, like lapply. A feature request regarding progress bar for parallel functions has been sitting at the development GitHub repository for a few months. More recently, the author of the pbmcapply package dropped a note about his implementation of forking functionality with progress bar for Unix/Linux computers, which got me thinking.…
Original Post: What is the cost of a progress bar in R?