Posted on

Choosing Cut-Offs in Tests

My last blog post was on the difference between Sensitivity, Specificity and the Positive Predictive Value. While showing that a positive test result can represent a low probability of actually having a trait or a disease, this example used the values of Sensitivity and Specificity as pre-known input. For established tests and measures they indeed are often available in literature together with recommended cut-off values.1

In this post, I would like to show how the choice of a cut-off value influences quality criteria such as Sensitivity, Specificity and the like. If you just want a tool to play with, see my Shiny web application here.

Continue reading “Choosing Cut-Offs in Tests”

Posted on

Visualizing Sensitivity and Specificity of a Test

In my university course on Psychological Assessment, I recently explained the different quality criteria of a test used for dichotomous decisions (yes/no, positive/negative, healthy/sick, …). A quite popular example in textbooks is the case of cancer screenings, where an untrained reader might be surprised by the low predictive value of a test. I created a small Shiny app to visualize different scenarios of this example. Read on for an explanation or go directly to the app here.

Continue reading “Visualizing Sensitivity and Specificity of a Test”

Posted on

Vagrant VM for starting a Rails project

This is something off-topic for this blog, but after spending several hours setting up an environment for developing and testing a Ruby on Rails project, I’d like to share my solution.

I recently had an idea for a small web-project, for that I’d like to use Ruby on Rails. From previous attempts of using Rails, I knew that Ruby and Rails are updated regularly and that the setup of the environment might be tricky. And using my Mac not only for web-development, I did not want to have many versions of Ruby, Python, whatnot, in parallel on it. And as I read about Vagrant some weeks ago, I wanted to give it a try.

It took quite some time to set-up the provisioning script properly. I tried using Chef, but it made things more complex and I appreciated the simplicity of a shell script. Continue reading “Vagrant VM for starting a Rails project”

Posted on

The Valley of Shit

Having only started my PhD studies a few months ago, I am still eager and highly motivated to finish what I have just started. However, first doubts on the topic and the quality of my work already came (and went again, luckily), so I could relate to this post on the Valley of Shit:

The Valley of Shit is that period of your PhD, however brief, when you lose perspective and therefore confidence and belief in yourself. There are a few signs you are entering into the Valley of Shit. You can start to think your whole project is misconceived or that you do not have the ability to do it justice. Or you might seriously question if what you have done is good enough and start feeling like everything you have discovered is obvious, boring and unimportant. As you walk deeper into the Valley of Shit it becomes more and more difficult to work and you start seriously entertaining thoughts of quitting.

A great post, that I enjoyed reading. I will bookmark it to read it again whenever I find myself in such Valley of Shit.

Posted on

ASA statement on p-Values: Improving valid statistical reasoning

A lot of debate (and part of my thesis) revolve around replicability and the proper use of inferential methods. The American Statistical Association has now published a statement on the use and the interpretation of p-Values (freely available, yay). It includes six principles and how to handle p-Values. None of them are new in a theoretical sense. It is more a symbolic act to remind scientists to properly use and interpret p-values.

Continue reading “ASA statement on p-Values: Improving valid statistical reasoning”

Posted on

Scientific Hoaxes and Bad Academic Writing

A new case of scientific hoax, that happened six years ago, is currently circulating:

Six years ago I submitted a paper for a panel, “On the Absence of Absences” that was to be part of an academic conference later that year—in August 2010. Then, and now, I had no idea what the phrase “absence of absences” meant. The description provided by the panel organizers, printed below, did not help. The summary, or abstract of the proposed paper—was pure gibberish, as you can see below. I tried, as best I could within the limits of my own vocabulary, to write something that had many big words but which made no sense whatsoever. I not only wanted to see if I could fool the panel organizers and get my paper accepted, I also wanted to pull the curtain on the absurd pretentions of some segments of academic life. To my astonishment, the two panel organizers—both American sociologists—accepted my proposal and invited me to join them at the annual international conference of the Society for Social Studies of Science to be held that year in Tokyo.

Continue reading “Scientific Hoaxes and Bad Academic Writing”

Posted on

Mixing up Standard Errors and Standard Deviations

Over at the Non Significance blog, the author describes the case of a paper that has some strange descriptive statistics:

What surprised me were the tiny standard deviations for some of the Variable 1 and 2, especially in combination with the range given.

In the blog post, the author outlines his approach to make sense from the descriptive values. It seems to be likely that the reported Standard Deviations (SD) are actually Standard Errors of the Mean (SEM). I’d like to add to this blog post one argument based on calculus and one argument based on simple simulations to show that SEM’s are indeed much more likely than SD’s.1 Continue reading “Mixing up Standard Errors and Standard Deviations”