The largest R "targets" workflow my team has produced to date, with 15023 dynamic targets! Worked flawlessly on the first try after extensive testing. Great way to start this Friday!
#rstats #DataScience
Follow
@blasbenito Wow! And I thought that my little workflows were a bit overwhelming... xD
More evidence on targets being an incredible package.
For curiosity's sake, did you run the workflow using any HPC or parallel infrastructure? Using crew?
@blasbenito Thanks for the info!
Those packages are incredible. Well, the tsunami led by Hadley Wickham some years ago has made R a jewel for data exploration.
Looks like a great project. Congrats!
@keyeoh It is an incredible package!
When I talk about R, I always say that dplyr, data.table, and targets are the most cutting-edge packages one can find in the whole language.
The workflow uses crew, and was run on a high-end laptop with 20 CPUs for about a day and a half. It's not a large data pipeline at all, but I'm still kinda proud of it!