Show newer

I do not want Slack to provide a probabilistic summary of what I said. I don't want notion to guess what I'm going to say. I want to choose my words with clarity and precision in mind, and if people want me to take the time to read what they've written I would hope that they've taken the time to choose their words too

And I really want to take my words out of training data sets

"The single most important problem with null hypothesis testing is that it provides researchers with no incentive to develop precise hypotheses. To perform a significance test, one need not specify the predictions of either one’s own research hypothesis or those of alternative hypotheses"
(Gerd Gigerenzer 1993)

🌲
not reproducible ≠ wrong/false/fluke
reproducible ≠ true

not reproducible ≠ poor science
reproducible ≠ good science

reproducibility of results is not a reliable indicator of truth or research quality or epistemic progress.
🌲

@lakens @david_colquhoun
In my opinion there are more important questions, like, how to prevent harm that people like Rowling cause.

I think the calculus of #RegisteredReports might have flipped in a #SurveillancePublishing APC-driven #OpenAccess world.

Subscription models meant that a journal could command a high price by being in high demand in a self-reinforcing cycle where since most libraries subscribed to them then they would have high readership, etc. Multiply that by the power of bundling or whatever. Libraries being a conduit could tell who read what and tailor subscriptions accordingly. actual loss of readership could impact subscription cost during negotiations, so null results are less attractive because they command fewer readers and citations and whatnot. publication bias ensues. the classic story.

in an APC world, where the profit is derived from authors willing to directly pay more for the attendant view count and citation, the registered report is instead more like a commitment to pay at some future time to publish. if the prices keep going up, the journal effectively invests in your need to publish as a security.

this is doubly perverse in a surveillance publishing system, where the publishers operate paper recommendation and rating systems linked to funding and employment decisions. in that case, they can just manufacture the view count and citation - and even literal "scientific value score" - as a function of APC price, so null results aren't even a problem since the exclusivity-prestige link is partially dissolved.

I wonder if causes for publication bias could have changed substantially enough that registered reports could backfire as a means of combating publication bias. Since the primary filter is the perceived importance of a piece of work - assuming the authors could pass some competency and design check normal to the field - which is most likely to be at least partially evaluated by the same system of self-fulfilling metrics used in the recommendation/scoring systems for funders and employers, they might directly reinforce hype cycles. couple that again with the prestige gradient model of APC pricing where one publisher owns many Journals at different prestige levels and can bounce you down the ladder to one with a lower but still high APC.

Journals then would then be effectively sorting papers by APC according to the propensity for views/citations, regardless of outcome. It's sort of a combination of payola and security. plz lmk where I'm missing something here bc not just trying to shit on the parade.

📢 Yes we can! Only 19 more # researchers' signatures are needed for the #PCIManifesto to reach 🌟 the symbolic threshold of 1️⃣0️⃣0️⃣0️⃣! Spread the word! #openedu #openscience #openaccess peercommunityin.org/pci-manife

and less rigorous researchers. It's a failed system. I don't believe in peer review at all because it's a relic from a corrupt system that glorifies individual research paper and oversimplifies science to a handful of results. There's so much to dismantle.

Show thread

Peer review? Every time I read your research, I am the peer. I read, process, think, evaluate, decide what to do with it. I can choose to ignore it, challenge it, use it, improve upon it, etc. That's the work, no? We each are peer reviewers when we engage with others' work. We don't need journal involvement for peer review to happen. We can't rely on a few random and anonymous reviewers to do our job either. Outsourcing this basic part of research process has only turned us into lazier readers >

Elon Musk, Twitter & die Pressefreiheit 

Stell dir vor du bezahlst auch noch einen Clown wie #Musk dafuer, dass er freie Berichterstattung zensiert 😅

ZDF Frontal? Waren das nicht die, die auch immer mal wieder kritisch ueber die #Tesla Shitshow in Gruenheide berichtet haben?

Free Speech my ass!

Sorry fuer den Mini-Rant. Bin gerade angewidert.

@david_colquhoun @lakens

I'm quite sure that I agree with you. But, in my opinion, it's problem with lack of rigour and putting too much confidence in "statistical ritual" than p-value itself. Let me provide some example:

1) We are using p-value to detect candidate genes in some traits, like depression, anxiety, intelligence...

2) We can conduct some meta-analysis of such analysis, to make sure that candidate gene has effect, and obtain some "satisfying low p-value", like p = .00002 (σ > 4) and based on these results to conclude that we have strong evidence for effect.

Point "2)" is wrong, and there is a little evidence in it, but I think, that using p-value to detect "candidates" is defendable. Or, at least, I'm not aware of any better method.

@david_colquhoun @lakens

Fisherian aproach was intoduced in 1925, while Jerzy Neyman proposed his solution in 1926.
It's amazing that after almost 100 years, there are lively discussions about them!

In my opinion, these are two different, not always competitive tools, like screws and nails.
If we have specified hypothesis to test, it'll be better to use Neyman approach and calculate LR or other suitable test, with satisfying α and β.

But, when we want to just test null hypothesis for error detection, or basic exploratory analysis, p-value seems to be good approach.

I guess maybe don't use references chatGPT throws at you not because they can be wrong or fake but because, idk, you haven't actually read them?? If people first read & understand stuff they wanna cite, like one does, perhaps it wouldn't matter how they got to those references.It's weird to be scandalized by the possibility of a model making up references while normalizing scientists' lack of basic scientific integrity. I know which one is more horrifying to me.

@OskarA
It's amazing that such ignorant and genocide denier as Chomsky is still considered an expert.

@psychNerdJae @ct_bergstrom

You can scatterplot everything, but that doesn't mean it makes sense. Especially plotting ranked data doesn't. In the best scenario, it's waste of time, for both authors and readers.

Also, linear regression is the wrong method to analyze such data. None of the assumptions were met!

But, IMO, the problem is field-wide: correlation study instead of an experiment, no data analysis plan at preregistration, no power analysis, non-random sample, no clear relation between theory and operationalization, big potential for p-hacking and so on...

Show older

Paweł Lenartowicz's choices:

CleverLibre Social

CleverLibre Social is an inclusive social instance for open discussion, learning, and community.
All cultures welcome.
Hate speech and harassment strictly forbidden.