PRODUCTS (things we manage)
* repository structure (appropriate sub-directories)
* conda environments (or some environment control)
* config files (no hard-coding inside scripts)
* output-centric scripts (no inline results e.g. in notebooks)
* orchestration via bash script (steps of execution pipeline)
* documentation (README, website, etc)
2/3
I'm trying to formalize thoughts around habits I've developed for reproducible polyglot data science work on shared linux servers.
To me this work consists (*loosely*) of two main components: the things we produce (code, models, etc) and the behaviors associated with projects.
Here's a draft outline of these components. What's missing here? What can be better?
#rstats #datascience #reproducibleresearch
1/3
@brodriguesco I agree. While many languages have good internal package management, they all work very differently. Let's face it, some of those package managers should be crimes against humanity. Python has so many that xkcd made a comic about it.
With #Nix, you can handle (almost) everything. Yes, it's a big time investment, but you only have to do it once and it really works. Nix is the git of reproducible builds.
PS: I thought you started with #guix, but apparently you came to the same conclusion as I did, and invested time in #nix. Installing #NixOS as your main OS really helps the learning process ;-)
@josegustavomartins I think that it’s an incredible tool with a very high entry cost. If tools and guides are made to lower this entry cost, then yes, I believe that #Nix could offer an incredible opportunity for reproducibility, and a probably better option than Docker. My goal in the coming months is to start writing such guides and build tools to make using Nix easier for scientists (focused on #rstats We’ll see how this goes.
Programming language hot take from someone that really only knows one language well and has been using Nix for like 3 weeks: programming language should not come with an internal package manager, their packages should get installed with a tool like #Nix
Las #Bibliotecas como lugares de libertad.
#NoCensuraEnBibliotecas #NoALaCensura
Feliz verano y disfrutad de todo lo que os ofrecen esos lugares maravillosos... las bibliotecas. Además en verano se está tan tranquilo y fresquito.
i think a lot about how apparently, someone visiting pink floyd's studio in the mid-70s noticed they had several minimoogs set up with gaffer tape all over them, because when they got a sound they liked, they'd put the tape over the knobs so the settings wouldn't get changed, and invoice the record label for another fresh minimoog. i think this is how you're actually supposed to manage python software installations, just buy a new computer every time you finally get the correct packages set up
Mi psicóloga dice que nuestros recursos cognitivos se ven sobrecargados por tanta estimulación negativa. Lo primero que retira a sus pacientes con depresión es el telediario.
Yo conozco bien ese daño. Durante cada comida y cena familiar frente a las noticias, mi padre recalcaba religiosamente la faceta más abyecta, tramposa y materialista de España, el mundo y la condición humana.
El contrapeso de mi madre -gran defensora de la humanidad- me salvó la vida. Pero no logró impedir que heredara toda esa pesadumbre ontológica.
Quizás por eso busco a gente como Raquel, mi pareja. Es de Salamanca pero posee un talento caribeño para asimilar la desgracia desde la sonrisa y hasta la carcajada.
Para gente como yo, en cambio, el sobre consumo de malas noticias puede considerarse una forma de autolesión. Y lo peor es que a menudo reivindicamos como "concienciación", lo que solo son cortes en los brazos.
Hoy proteger mi alegría para actuar me parece mejor acto político.
Buen día para recordar esta reflexión de Marta Peirano:
"A veces me reprochan mi falta de optimismo. Como decía Vaclav Havel, lo que hay que tener es esperanza. No la convicción de que algo saldrá bien cuando todo indica lo contrario, sino la certeza de que trabajar para que salga bien tiene sentido, independientemente de cómo resulte."
Bueno, yo sé que en realidad nos gobiernan los bancos y bla bla bla, pero hay cosas muy prácticas y terrenales que dependen del 23J.
Por ejemplo, si la macrogranja de Noviercas (Soria), el pueblo a 10 km del mío donde querían poner 24000 vacas (sería la 3ª granja más grande ¡del mundo!) sigue paralizada gracias al decreto ley sobre ganado bovino que se aprobó hace nada, en diciembre.
Un cambio de gobierno, en este aspecto, puede ser fatal.
Me juego poder vivir en mi pueblo o no.
I would be happy to view ads on a website if:
1) They didn't autoplay video.
2) They are static images (reading with unnecessary motion is distracting).
3) Ads didn't target me and we're related to the site I'm on.
Advertiser created the mess of adblocking and they want to solve it by force me to see ads. Ads in print magazines don't bother me because they don't violate the above rules.
Following the early heatwave in April in Spain and several months of drought, educated people from northern Europe complained that their holidays were "ruined" by the heavy rain that followed. Now they are complaining it is too hot. Perhaps it's time they started acting like citizens and not just consumers of our planet's resources.
Data Scientist. PhD in Machine Learning. Developer. I have a past in Bioinformatics. Usually playing with R, Python and other languages. Wish I could be younger to learn Rust or Elixir.
Hobbies: reading, writing, walking, music, movies.