WEBVTT Kind: captions; language: en-us

NOTE
Treffsikkerhet: 91% (HØY)

00:00:01.200 --> 00:00:08.800
There we go. Okay. So we're going to talk about quality assessment, or in particular critical 

00:00:08.800 --> 00:00:13.700
appraisal tools. And for those who are lazy like I am, we very often call them just Cats. It's 

00:00:13.700 --> 00:00:21.100
a Cat, but it is a critical appraisal of the methodology of a paper, of a study. That's what we're 

00:00:21.100 --> 00:00:27.100
going to talk about. And there is a lot out there. So, let's see how we get this one to the next 

00:00:27.100 --> 00:00:28.250
slide. 

NOTE
Treffsikkerhet: 85% (HØY)

00:00:28.250 --> 00:00:34.100
That's the next slide. So we're going to talk about bias and confounding. We're going to talk about 

00:00:34.100 --> 00:00:40.000
study designs at a glance because you will talk about Wednesday little bit more on that. So I will 

00:00:40.000 --> 00:00:46.300
just introduce different study designs and then that brings us to critical appraisal tools. I need 

00:00:46.300 --> 00:00:53.600
to talk about study designs as well because you start critical appraisal depends on the study 

00:00:53.600 --> 00:00:58.500
design. If you talk about it by ICT, randomized control trials, 

NOTE
Treffsikkerhet: 91% (HØY)

00:00:58.500 --> 00:01:04.750
you've got issues like randomization and that is, of course not the case if you've got an NS1. 

00:01:04.750 --> 00:01:09.600
So therefore, you need to know a little bit about research designs before I can talk about quality 

00:01:09.600 --> 00:01:15.000
assessment. Still I thought this was important to do this first because this has got in general to 

00:01:15.000 --> 00:01:21.050
do what is methodology? What are the pros and the cons? What can we do wrong or good? 

00:01:21.050 --> 00:01:28.300
Okay. So let's start with bias and confounding. Now, there is such a thing as I will give 

NOTE
Treffsikkerhet: 74% (MEDIUM)

00:01:28.300 --> 00:01:34.500
you different terminologies different terms and I hope then in the end that works a little bit 

00:01:34.500 --> 00:01:42.100
together, but people will talk to you about internal and external validity. I know some of you 

00:01:42.100 --> 00:01:48.850
this is all old stuff, but then they can correct me if I do it wrong. Anyhow, internal validity is 

00:01:48.850 --> 00:01:55.450
evidence. That is a true and accurate reflection of the participants, procedures and the setting, 

00:01:55.450 --> 00:01:58.350
being observed. So just 

NOTE
Treffsikkerhet: 89% (HØY)

00:01:58.350 --> 00:02:05.800
bit that you are focusing on. External validity has got to do with generalizability. Can I 

00:02:05.800 --> 00:02:12.300
generalize the results that I found? Can I generalize the data from my little study to other 

00:02:12.300 --> 00:02:17.600
situations? Is the evidence that it is a true and accurate reflection of participants procedures in 

00:02:17.600 --> 00:02:24.800
a setting other than the observed? Now, that is usually what you want to do with your research. You 

00:02:24.800 --> 00:02:28.250
say, hey, I select 10 schools from Oslo. 

NOTE
Treffsikkerhet: 90% (HØY)

00:02:28.250 --> 00:02:36.900
And I hope that is representative for the whole of Oslo, maybe even for the whole of Norway. So that 

00:02:36.900 --> 00:02:45.500
your data are correct reflection of the actual situation that is internal generalization. That is what we 

00:02:45.500 --> 00:02:52.900
always want to do because we can't assess or evaluate all the Norwegians. So you need to select 

00:02:52.900 --> 00:02:58.300
wisely and you want that to be representative and that's got to do external validity. 

NOTE
Treffsikkerhet: 82% (HØY)

00:02:58.300 --> 00:03:06.100
There are always problems in research, that's life. And one of them is bias. 

00:03:06.100 --> 00:03:11.800
These are just the definitions, but I'll give you examples later. So, bias is a deviation, 

00:03:11.800 --> 00:03:18.800
a difference from the results or interference from the truth. So, something is 

00:03:18.800 --> 00:03:25.100
wrong here, or processes leading to this deviation which you call bias and more specifically, it 

00:03:25.100 --> 00:03:28.300
can be the extent to which these statistical methods used in 

NOTE
Treffsikkerhet: 91% (HØY)

00:03:28.300 --> 00:03:36.000
the study do not estimate the quantity thought to be estimated or do not test your hypothesis. 

00:03:36.000 --> 00:03:42.000
So it is a problem. That's what you're talking about. Now. I'll give you many examples. You've got 

00:03:42.000 --> 00:03:50.700
very different types of publication of bias and one of them is publication bias. So we all know that 

00:03:50.700 --> 00:03:58.250
it's much easier to submit a paper that's called ''cure of cancer'' than a paper that says 

NOTE
Treffsikkerhet: 80% (HØY)

00:03:58.250 --> 00:04:04.300
''again, we couldn't cure cancer''. The first one will be accepted the second one may just end up in your 

00:04:04.300 --> 00:04:10.300
drawer and never be published. So there is absolute publication bias, especially if you've got all 

00:04:10.300 --> 00:04:15.800
these small studies. Then the problem is that very often, the small positive studies will be 

00:04:15.800 --> 00:04:22.000
published, but the small negative studies just are being hidden. They disappear. So you get a too

00:04:22.000 --> 00:04:26.600
positive image very often. That's publication bias. 

NOTE
Treffsikkerhet: 87% (HØY)

00:04:26.600 --> 00:04:34.750
Now, the other problem is that and that's got to do with desirability. So respondents very often 

00:04:34.750 --> 00:04:39.900
want to, but they do do that deliberately or not, they want they respond, what they think is social 

00:04:39.900 --> 00:04:47.900
desirable behavior or social desirable responses, so they want to be accepted. So if you ask, for 

00:04:47.900 --> 00:04:56.100
instance, about how much alcohol do you drink? They say, well, not too often, maybe a glass, but we 

00:04:56.100 --> 00:04:56.600
know that 

NOTE
Treffsikkerhet: 75% (MEDIUM)

00:04:56.600 --> 00:05:02.000
very, very often these numbers are underrated when you you try to evaluate, when you want to have 

00:05:02.000 --> 00:05:09.299
the data. So it's in its regardless of the research format, but people will report inaccuretly. 

00:05:09.299 --> 00:05:15.400
I'm speaking too fast. Especially when it's sensitive or personal topics. How many 

00:05:15.400 --> 00:05:22.150
partners did you have last year? People will not just blur that out. So that is respondent bias. 

00:05:22.150 --> 00:05:26.350
People want to be liked. Now another one is respondent bias,

NOTE
Treffsikkerhet: 67% (MEDIUM)

00:05:26.350 --> 00:05:32.000
and we talking about for instance sponsored bias, is we all know research is very often 

00:05:32.000 --> 00:05:40.500
highly dependent on sponsoring or funding and that means that sponsors, if they get have an 

00:05:40.500 --> 00:05:46.900
influence on the output, that's where you get yourself into trouble. So if you've got, if the 

00:05:46.900 --> 00:05:51.900
sponsors have got, can actually determine whether for instance, your research being published 

00:05:51.900 --> 00:05:56.600
or not... You've got a problem. So sponsoring is good. But also in my area,

NOTE
Treffsikkerhet: 91% (HØY)

00:05:56.600 --> 00:06:03.100
many people, I am in solving problems... There are many people that are being sponsored by 

00:06:03.100 --> 00:06:08.400
nutritional industries and don't underestimate, that's big money. That's Nutritia, that's Nestle,

00:06:08.400 --> 00:06:15.800
those are very big names. What you don't want to as a research is 

00:06:15.800 --> 00:06:21.100
that your name is linked to one industry, because then that is very likely there is some bias

00:06:21.100 --> 00:06:26.550
going on. It's better to either be sponsored by several, but at least they should never

NOTE
Treffsikkerhet: 74% (MEDIUM)

00:06:26.550 --> 00:06:32.600
ever should have influence on whether you publish your data or on the data or whatever, and it 

00:06:32.600 --> 00:06:38.900
is difficult to be independent, unless it is just a research funding because that should be 

00:06:38.900 --> 00:06:45.700
independent. So that there is absolutely sponser bias. It's all got to do with money. We've got 

00:06:45.700 --> 00:06:52.100
confirmation bias and that is, we've got actually nine out of ten, we've got kind of an idea of what 

00:06:52.100 --> 00:06:56.650
we expect to find, we expect to find covid-19 

NOTE
Treffsikkerhet: 83% (HØY)

00:06:56.650 --> 00:07:02.400
is bad for Education. We we expect to find that. So if you've got ideas like that in your 

00:07:02.400 --> 00:07:09.500
head and we all are guilty of that because we are humans, then you very likely may translate or may 

00:07:09.500 --> 00:07:15.500
pick up on those things that fit into your mindset already. So, it seems these beliefs and 

00:07:15.500 --> 00:07:22.700
respondents information to confirm that believe. That is if you say the Dutch are 

00:07:22.700 --> 00:07:26.500
tall, we are tall, but I walk outside and I will see all these tall 

NOTE
Treffsikkerhet: 87% (HØY)

00:07:26.500 --> 00:07:35.800
people, whereas there are few. It is your mindset, you look at confirmation. And that is, for 

00:07:35.800 --> 00:07:41.700
instance... This is an example, just a simple one, but it is what you expect to see. And these two 

00:07:41.700 --> 00:07:48.050
have got different expectations than actually real life, but that's what they see. And my dog is 

00:07:48.050 --> 00:07:55.400
absolutely a wolf next door here. Cultural bias, and I can say I'm very different from Norway.  

00:07:55.400 --> 00:07:56.450
I think that 

NOTE
Treffsikkerhet: 79% (HØY)

00:07:56.450 --> 00:08:02.000
We all have got our international SNE course, we've come from different cultures, 

00:08:02.000 --> 00:08:10.200
different beliefs, and we have that in our... that's where we grew up, that's what we are used to. 

00:08:10.200 --> 00:08:16.800
We compare it with that. We've got this cultural limbs. If I compare the Netherlands to Norway, and I 

00:08:16.800 --> 00:08:24.150
always thought we were very, very similar. But now I learned, we talk a lot, we Dutch. And in Norway, 

00:08:24.150 --> 00:08:26.400
it's totally different. People are 

NOTE
Treffsikkerhet: 87% (HØY)

00:08:26.400 --> 00:08:30.900
not just interact in a discussion or not that easy. It's really a different way of 

00:08:30.900 --> 00:08:38.200
communicating. So you've got this cultural lens what you expect. Not bad. It's just you need to be 

00:08:38.200 --> 00:08:43.500
aware of that it's there. But there are also things that are linked to your research, how you 

00:08:43.500 --> 00:08:52.000
design it. And one of them is question order bias. So questions influence each other and that is 

00:08:52.000 --> 00:08:56.500
called question order bias. For example, the example here  

NOTE
Treffsikkerhet: 83% (HØY)

00:08:56.500 --> 00:09:03.500
if respondent rates a product at 10, and then there's a competitive product... You will compare it to 

00:09:03.500 --> 00:09:08.400
the 10 you just gave and 10 is the highest, you will see that this is the best.  

00:09:08.400 --> 00:09:15.450
You compare it. You always compare to previous order ones afterwards, but also for instance with... 

00:09:15.450 --> 00:09:27.400
If you evaluate severity of a 

NOTE
Treffsikkerhet: 82% (HØY)

00:09:27.600 --> 00:09:34.200
handicap and handicap is wrong word... Disability, that's a better word. If I give you ten videos 

00:09:34.200 --> 00:09:41.200
of severely disabled children, and ask how are their hand skills, decode and hand 

00:09:41.200 --> 00:09:46.000
coordination, and you rate them. And then I give a child, which is not that severe, you will 

00:09:46.000 --> 00:09:50.900
probably rate that one as really positive because you compare with the previous one. So there's 

00:09:50.900 --> 00:09:56.500
always this question order bias, not just in the in surveys, but also when you observe 

NOTE
Treffsikkerhet: 81% (HØY)

00:09:56.500 --> 00:10:05.800
things. We compare with what is previous already rated. Now, this is an example and that's a 

00:10:05.800 --> 00:10:14.950
question to you. So if I were to study the prevalence of Cerebral Palsy in school-aged children, 

00:10:14.950 --> 00:10:21.100
in Norway. I'm going to do a cross-sectional study, I'm going to do a prevalence study and what I 

00:10:21.100 --> 00:10:26.650
will do, I do my subject recruitment in schools for children with special needs in Oslo. 

NOTE
Treffsikkerhet: 89% (HØY)

00:10:26.650 --> 00:10:31.800
Now, my question to you is, is there bias or not? 

NOTE
Treffsikkerhet: 82% (HØY)

00:10:31.800 --> 00:10:37.300
And what would be the bias if there is bias, anybody? 

NOTE
Treffsikkerhet: 85% (HØY)

00:10:44.800 --> 00:10:56.300
Melissa: I don't think there might be a little, because you're only studying this in Oslo. 

00:10:56.300 --> 00:11:05.800
So maybe it would be very specific to and you're not really getting a whole range.

00:11:05.800 --> 00:11:16.500
Because Oslo will have different things, that's the first thing stood out for me.

00:11:16.500 --> 00:11:19.200
Renee: I agree with you. There is absolutely biased because if you 

00:11:19.200 --> 00:11:24.700
suggest this is Norway, well is Oslo there is representative... So that's one thing we need to 

00:11:24.700 --> 00:11:30.600
consider. There's another thing. What do you think about my choice of schools for children with special 

00:11:30.600 --> 00:11:36.200
needs? What would be the problem with that? Hana: You're only studying schools with special needs, but you 

00:11:36.200 --> 00:11:43.200
want to find the problems in school-aged children. So then it's not a representation of all the children.

00:11:43.400 --> 00:11:49.200
Renee: Exactly. If I go to schools for children special needs, very likely I expect higher 

00:11:49.200 --> 00:11:57.200
numbers of prevalence of CP, cerebral palsy. So this is an example of bias. It's not a bad 

00:11:57.200 --> 00:12:02.800
study. It's just you're not going to find the answer you were looking for because you chose the 

00:12:02.800 --> 00:12:09.900
wrong schools, and you chose a very selected area. Now if your purpose was to determine the 

00:12:09.900 --> 00:12:13.500
prevalence of cerebral palsy in schools for children with 

NOTE
Treffsikkerhet: 90% (HØY)

00:12:13.500 --> 00:12:18.700
special needs in Oslo that's fine with me. Then you're doing exactly what you wanted to know, and maybe 

00:12:18.700 --> 00:12:23.599
that's a very feasible and good topic. 

00:12:23.599 --> 00:12:29.800
But if you are stating you do the whole of Norway, 

00:12:29.800 --> 00:12:35.000
then you've got a problem. Yeah, that's clear for everybody? So, that is why 

00:12:35.000 --> 00:12:41.100
also when you write your research proposal, I link your research question to your methods. Do you 

00:12:41.100 --> 00:12:43.150
get what you are looking for? 

NOTE
Treffsikkerhet: 91% (HØY)

00:12:43.150 --> 00:12:48.900
Okay, I'll give you another example. So this I would say, well, this is actually bias. Now I've got 

00:12:48.900 --> 00:12:55.500
another one. There is such a thing as speyer therapy. I've got nothing to do 

00:12:55.500 --> 00:13:01.550
with it, by the way, but there is such a thing as speyer therapy. So I'm going to study the  

00:13:01.550 --> 00:13:08.400
positive effect of speyer therapy. And what's going to happen is I observe, I'm the single 

00:13:08.400 --> 00:13:13.450
rater or video recorded therapy sessions. There are problems. 

NOTE
Treffsikkerhet: 88% (HØY)

00:13:13.450 --> 00:13:15.600
What are the problems here? 

NOTE
Treffsikkerhet: 91% (HØY)

00:13:15.700 --> 00:13:18.300
Anybody? 

NOTE
Treffsikkerhet: 82% (HØY)

00:13:20.500 --> 00:13:28.900
One of the problems is how I already formulated my purpose. What is actually a problem in the 

00:13:28.900 --> 00:13:31.600
formulation of my purpose? 

NOTE
Treffsikkerhet: 84% (HØY)

00:13:31.600 --> 00:13:39.300
Anybody? Or is it really a difficult question? Melissa: So could you repeat that? Renee: Okay, what is the problem 

00:13:39.300 --> 00:13:45.849
with how I formulated my purpose because that is already a problem in itself. 

NOTE
Treffsikkerhet: 83% (HØY)

00:13:45.849 --> 00:13:52.700
Kine: If you want to determine because you already think or say it's a positive effect, but you 

00:13:52.700 --> 00:13:59.300
expect, you've decided sort of and it should be more

00:13:59.300 --> 00:14:06.300
open or curious. Renee: You are right. Is there an effect? So what the problem is indeed I 

00:14:06.300 --> 00:14:14.100
added the word positive. Well, that is my bias and it's like ''I know my therapy is 

00:14:14.100 --> 00:14:15.150
fantastic, and I'm 

NOTE
Treffsikkerhet: 82% (HØY)

00:14:15.150 --> 00:14:21.900
going to prove how positive it is. '' No, you should say, ''to determine the effects of speyer therapy. '' 

00:14:21.900 --> 00:14:27.700
You don't say to determine the bad effects or the positive. Now, that is actually bias. 

00:14:27.700 --> 00:14:33.500
If you want to know the effects, keep it open. There is some other problems. Also, if I say, I want 

00:14:33.500 --> 00:14:40.600
to determine the effects of speyer therapy, and that's my therapy. Then it's not okay. If 

00:14:40.600 --> 00:14:45.400
there's a single rater and it's actually me and looking at my own therapy... 

NOTE
Treffsikkerhet: 91% (HØY)

00:14:45.400 --> 00:14:51.500
That is not very objective enough. So that case I would like, I do like idea of video recording 

00:14:51.500 --> 00:14:56.800
therapy sessions, but I would like actually other raters to do whatever they are doing. Yeah, so 

00:14:56.800 --> 00:15:01.000
there is a lot of bias actually in this nonsense. By the way, there is such a thing as speyer 

00:15:01.000 --> 00:15:09.100
therapy, nothing to do with me. But okay, this is bias. So bias comes in very different 

00:15:09.100 --> 00:15:15.150
packages and formulations and it's very difficult not to have bias, but we try when we 

NOTE
Treffsikkerhet: 85% (HØY)

00:15:15.150 --> 00:15:21.600
do a study to avoid as much bias as we can. Like selection bias, be very careful about where you 

00:15:21.600 --> 00:15:28.700
recruit. Don't describe from the start. This, what I expect to find... That's a little bit too much 

00:15:28.700 --> 00:15:34.400
focus. Keep it open. You also well, maybe you can't find the results. Then there still could be 

00:15:34.400 --> 00:15:39.600
different ways why you don't find the effect, could be your measures are not responsive 

00:15:39.600 --> 00:15:45.350
to change, could be your population is too small, could be there is no effect... 

NOTE
Treffsikkerhet: 83% (HØY)

00:15:45.350 --> 00:15:54.850
But still, keep it open. Now, there is also another issue, another phenomena and that is confounding. 

00:15:54.850 --> 00:16:01.800
Confounding is different from bias, but it is also a problem. So confounding is where it is, not 

00:16:01.800 --> 00:16:06.800
possible and I always do first the definition, then I'll give you the examples. So confounding is when 

00:16:06.800 --> 00:16:13.200
it is not possible to disentangle to get them separate, the effects of two or more processes, 

00:16:13.200 --> 00:16:15.150
that are two things going on and 

NOTE
Treffsikkerhet: 81% (HØY)

00:16:15.150 --> 00:16:21.600
I don't know which one is causing what. And in epidemiology for example, a factor that is associated 

00:16:21.600 --> 00:16:28.800
with both risk and exposure status. Now, what does it mean? I'm going to give you examples. We've 

00:16:28.800 --> 00:16:38.500
got the sun and the sun is making my plant grow and the sun is also making my ice cream melt. So we 

00:16:38.500 --> 00:16:45.450
can say there's a link some with my plant and some with my ice cream. We all agree.

NOTE
Treffsikkerhet: 85% (HØY)

00:16:45.450 --> 00:16:54.400
This is not true that my plant makes my ice cream melt, the underlying factor causing the confounding 

00:16:54.400 --> 00:16:59.900
factor is the sun. That one is missing here. Yeah, so it's a stupid example, but it's just to 

00:16:59.900 --> 00:17:05.200
explain what's happening. So yes, that sun has got, of course there's a relation sun with growth 

00:17:05.200 --> 00:17:10.500
and with ice cream that's melting. But there is not a causal, not the connection between that plant 

00:17:10.500 --> 00:17:15.300
and my ice cream. I'll give you another example of confounding. 

NOTE
Treffsikkerhet: 86% (HØY)

00:17:16.000 --> 00:17:25.000
Meet the Dutch. By the way, I am that person on the left. So the Dutch, we eat cheese. Absolutely 

00:17:25.000 --> 00:17:32.200
true. We ate a lot of cheese, a lot of cheese. I love cheese. And that's not for my family but 

00:17:32.200 --> 00:17:38.300
the Dutch are pretty tall. We really are very, very tall. So there is the Dutch, we eat cheese and 

00:17:38.300 --> 00:17:45.750
the Dutch we are tall. Unfortunately, cheese is not making us any taller. It's only making us fat. 

00:17:46.250 --> 00:17:52.900
So this is a wrong connection correlation. There's no association between cheese and tall Dutch. 

NOTE
Treffsikkerhet: 88% (HØY)

00:17:52.900 --> 00:17:59.400
That is confounding. Now. I'll give you something that actually really adapted in our research. So we 

00:17:59.400 --> 00:18:07.200
have bolus modification, bolus modification is actually nothing else but we adjust food 

00:18:07.200 --> 00:18:12.700
consistencies when you've got swallowing problems. For instance cerebral palsy. We adjust food. We 

00:18:12.700 --> 00:18:18.000
won't give you a steak. Maybe we'll make it puree. That's bolus modification. 

00:18:18.000 --> 00:18:23.300
We change your food. And we were interested in what is the effect of bolus 

NOTE
Treffsikkerhet: 91% (HØY)

00:18:23.300 --> 00:18:29.400
modification on health related quality of life. Because if you've got swallowing problems, and I'm 

00:18:29.400 --> 00:18:35.000
going to tell you that, I will thicken up your wine for the next 30 years, well that has an 

00:18:35.000 --> 00:18:41.400
impact on me. So that is a little bit the idea. If you need to -all your life-

00:18:41.400 --> 00:18:49.600
eat puree food or mashed potatoes and thicken up your glass of wine, that is no fun. 

00:18:49.600 --> 00:18:53.150
And it is very, very often 

NOTE
Treffsikkerhet: 84% (HØY)

00:18:53.150 --> 00:18:58.400
described by therapists. It's so easy. Like, oh, we need to thicken up your liquids. So what is the 

00:18:58.400 --> 00:19:04.500
impact of that? The problem was... So, we did two reviews. We did reviews on living with dsyphagia 

00:19:04.500 --> 00:19:10.400
swallowing problems and bolus modification, what are the effects and we did also 

00:19:10.400 --> 00:19:17.800
quality of life in OD, dysphagia in general. So two reviews. Now the problem with the lower bit 

00:19:17.800 --> 00:19:23.300
of our question is that there is a modifier, there is a confounder and that is

NOTE
Treffsikkerhet: 91% (HØY)

00:19:23.300 --> 00:19:30.600
the underlying disease. So suppose you've got Parkinson and people with severe 

00:19:30.600 --> 00:19:36.800
Parkinson will have bonus modification. They need to adjust the food, but people with severe 

00:19:36.800 --> 00:19:43.300
Parkinson, also may have poor quality of life because of all other kinds of reasons. So you can't 

00:19:43.300 --> 00:19:50.800
just simply say, bolus modification is resulting in a poor quality of life. This is whole disease 

00:19:50.800 --> 00:19:53.199
behind it. Now you could 

NOTE
Treffsikkerhet: 84% (HØY)

00:19:53.199 --> 00:19:59.100
in studies, you could distinguish that, you could you can add underlying disease in your statistical 

00:19:59.100 --> 00:20:06.200
model. The problem is that many studies don't, they just measure what kind of food do people eat and 

00:20:06.200 --> 00:20:12.100
they ask ''could you fill in a quality-of-life measure? '' That's not good. That is confounding 

00:20:12.100 --> 00:20:15.800
actually. This, the underlying disease is the confounder. 

NOTE
Treffsikkerhet: 87% (HØY)

00:20:15.800 --> 00:20:22.800
Yep. Okay, and that was actually a problem that we encountered in a study in research. So that 

00:20:22.800 --> 00:20:28.100
was not actually the best link. Now. There are a few more terms that you need to be familiar with 

00:20:28.100 --> 00:20:35.100
and that is reliability and validity. We will talk much more about psychometrics validity and all of 

00:20:35.100 --> 00:20:42.300
that. But for now, we just focus on these two terms and the liability is the degree to which an 

00:20:42.300 --> 00:20:45.350
assessment tool produce stable and consistent result. 

NOTE
Treffsikkerhet: 85% (HØY)

00:20:45.350 --> 00:20:50.800
So I use my measure today, I use my measure in the afternoon and tomorrow and I 

00:20:50.800 --> 00:20:58.100
want the same results. If the patient's condition hasn't changed. The other one is validity and 

00:20:58.100 --> 00:21:02.300
validity has got to do... and you must have heard these terms but maybe not the definitions. But you 

00:21:02.300 --> 00:21:08.700
must have heard about it. Validity is to how well in measure, measures, what it is supposed to 

00:21:08.700 --> 00:21:14.000
purported to measure. So, are we actually measuring what we think we're measuring. That's in plain 

00:21:14.000 --> 00:21:16.000
English, validity. 

NOTE
Treffsikkerhet: 82% (HØY)

00:21:16.000 --> 00:21:22.600
If we look at this, it's actually visualized. So on top, we've got these are valid on the right 

00:21:22.600 --> 00:21:28.800
because we say it's aiming in the center and you are actually kind of measuring what you wanted to 

00:21:28.800 --> 00:21:36.850
measure. But the difference is, this is very precise, meaning reliable. This one is not that precise, 

00:21:36.850 --> 00:21:43.200
meaning not that reliable. There's a lot of change there. But if I look on the reliable side, then 

00:21:43.200 --> 00:21:45.850
again reliable means consistent. 

NOTE
Treffsikkerhet: 91% (HØY)

00:21:45.850 --> 00:21:53.200
This is of course the ideal world, very reliable, very valid. This one is still very reliable 

00:21:53.200 --> 00:21:59.900
because every time it points at the same direction, unfortunately, it's not valid because you 

00:21:59.900 --> 00:22:06.800
were aiming for the center. So it's very precise, but it's not targeting w you wanted to target, 

00:22:06.800 --> 00:22:11.900
which is also a problem. And if I go to this one on the left corner down, 

00:22:11.900 --> 00:22:15.550
that's actually your nightmare, because we've got here 

NOTE
Treffsikkerhet: 83% (HØY)

00:22:15.550 --> 00:22:21.500
it's not precise and it's not actually measuring what you want to measure. So that one unit 

00:22:21.500 --> 00:22:27.000
must thrown away. That's the worst one. But of course, this is the best one, very precise and exactly 

00:22:27.000 --> 00:22:32.750
measuring what you want to measure. This is the difference between validity and reliability. 

NOTE
Treffsikkerhet: 74% (MEDIUM)

00:22:32.750 --> 00:22:38.500
Now, let's go back to this internal and external validity. That is after you've got internal 

00:22:38.500 --> 00:22:44.300
validity. That was that you are looking only at the data that you collected. And is that a true and 

00:22:44.300 --> 00:22:51.400
accurate reflection of what you want to measure and external was about generalizability. Now, there 

00:22:51.400 --> 00:22:57.600
are always threats. There are always problems in the world and there are also in case of validity 

00:22:57.600 --> 00:23:02.250
there are some problems. First of all, you've got for internal validity. 

NOTE
Treffsikkerhet: 73% (MEDIUM)

00:23:02.250 --> 00:23:09.000
We've got bias and we discuss that like opinions, expectations, personal believes. Then we've got 

00:23:09.000 --> 00:23:14.600
quality of measures and we've got two lectures on psychometrics, and that's got to do with 

00:23:14.600 --> 00:23:22.500
reliability, validity, responsiveness. It's got to do with, is my measure reliable, valid, sensitive to 

00:23:22.500 --> 00:23:28.900
change. If you measure is not reliable you've got a problem. Then we talk about research 

00:23:28.900 --> 00:23:32.250
design. Have you chosen the right research design? 

NOTE
Treffsikkerhet: 69% (MEDIUM)

00:23:32.250 --> 00:23:37.700
If you take any stream or NS1, something like that. Well, that probably will not give you an 

00:23:37.700 --> 00:23:45.200
answer to a question about does my therapy work in ASD. That's just a case study. It's nothing 

00:23:45.200 --> 00:23:52.200
else. Then other thing is that just depending on your research what you're doing, but in case of 

00:23:52.200 --> 00:23:57.700
clinics, very often we've got spontaneous recovery, meaning if you don't have a control group, 

00:23:57.700 --> 00:24:02.350
you just have one intervention and you follow them, suppose you just have had a stroke. 

NOTE
Treffsikkerhet: 76% (HØY)

00:24:02.350 --> 00:24:08.300
And you follow  patients the first three months and you give them therapy,

00:24:08.300 --> 00:24:14.600
and say ''I'm fantastic. They are improving. '' Yet, but there's also such a thing as spontaneous recovery. Now, you 

00:24:14.600 --> 00:24:21.000
could have dealt with that if you had two groups, one getting your therapy and the other maybe 

00:24:21.000 --> 00:24:26.700
usual care or something, then you could have compared the two. But just having one group and you're 

00:24:26.700 --> 00:24:31.400
having spontaneous recovery is really undermining your whole study. 

NOTE
Treffsikkerhet: 79% (HØY)

00:24:31.700 --> 00:24:40.100
And of course what you use to see quite often are the waiting list treatments, that is tough to get that 

00:24:40.100 --> 00:24:49.500
one through your ethics, because in how many studies, how many cases is it ethical to say: 

00:24:49.500 --> 00:24:54.050
''Hey, I just identified the problem. We put you on the waiting list. ''

NOTE
Treffsikkerhet: 81% (HØY)

00:24:54.050 --> 00:25:01.300
That's a tough one for ethics. Then we've got statistical significance versus clinical significance. 

00:25:01.300 --> 00:25:06.850
And the difference is statistical significance you ask Thanassi, that's what he's going to do. 

00:25:06.850 --> 00:25:13.600
You've got all these sets and you do your research and say ''I found all these little differences '' 

00:25:13.600 --> 00:25:18.400
But then the other reason what you need to ask yourself, the other question is, has it got any 

00:25:18.400 --> 00:25:24.199
clinical significance? If we know that the Dutch are 0.1 centimeter

NOTE
Treffsikkerhet: 87% (HØY)

00:25:24.199 --> 00:25:30.800
taller than the Norwegians, then what's the meaning of that? Does anybody care? So you need to 

00:25:30.800 --> 00:25:35.750
think also, if you are measuring something, the fact that significant, statistically significant... 

00:25:35.750 --> 00:25:41.200
Don't just use the word significant for important. I'm talking about statistically significant. Does 

00:25:41.200 --> 00:25:49.000
it actually mean that it makes a difference in education or in practice? Okay, and then a 

00:25:49.000 --> 00:25:54.100
final comment before we go to the next topic is there is a difference between efficacy 

NOTE
Treffsikkerhet: 87% (HØY)

00:25:54.100 --> 00:26:01.500
and effectiveness. Efficacy, we refer to it when we are testing 

00:26:01.500 --> 00:26:08.700
something just a therapy under optimal conditions, lab conditions, you name it. Effectiveness 

00:26:08.700 --> 00:26:14.100
is, well, this is what happens in clinics in normal daily life. Yeah. So efficacy, optimal.

00:26:14.100 --> 00:26:21.600
Effectiveness is what we usually find. Are there any questions so far on this 

00:26:21.600 --> 00:26:23.200
topic? 

NOTE
Treffsikkerhet: 86% (HØY)

00:26:25.400 --> 00:26:34.100
Then I just continue a little bit. Okay, I'm going to get on it. 

00:26:49.950 --> 00:26:56.100
We're going to talk about study designs. 

NOTE
Treffsikkerhet: 77% (HØY)

00:26:56.100 --> 00:27:01.500
I'm just going to show them very briefly to you and Wednesday,

00:27:01.500 --> 00:27:07.800
we will talk more in detail about study designs. Now this is just an example. This is the 

00:27:07.800 --> 00:27:14.200
level evidence according to the NHMRC or the national health medical research council.

00:27:14.200 --> 00:27:21.000
And this is what they came up with. Now, you can see at the bottom we have got case series.

00:27:21.000 --> 00:27:26.050
Then we've got comparing studies of 

NOTE
Treffsikkerhet: 90% (HØY)

00:27:26.050 --> 00:27:32.750
wwo types, we've got RCT's. That is pseudo RCT's, it's almost a randomized control, but just not. 

00:27:32.750 --> 00:27:39.100
And then systematic reviews of RCT's. And on top of that clinical guidelines. Clinical 

00:27:39.100 --> 00:27:45.600
guidelines should be based on the highest level of evidence available. We talk about all these 

00:27:45.600 --> 00:27:51.400
designs a bit later. But the higher in the hierarchy, the higher the level of 

00:27:51.400 --> 00:27:56.200
evidence when you use that study design. Certain study designed just give you 

NOTE
Treffsikkerhet: 83% (HØY)

00:27:56.200 --> 00:28:02.950
very weak evidence. And of course, an RCT gets much better evidence of whether a therapy is working 

00:28:02.950 --> 00:28:11.300
than if you just have got 10 patients that you followed up over time. Now there is a reason why 

00:28:11.300 --> 00:28:17.900
you choose certain study design. So the purpose of your study, your research question may lead to 

00:28:17.900 --> 00:28:23.900
the need to select the certain study design. It's not you can say let's pick any, no, please don't.

00:28:23.900 --> 00:28:26.100
So, effectiveness, 

NOTE
Treffsikkerhet: 91% (HØY)

00:28:26.100 --> 00:28:31.700
the best ones are controlled trials, but that is for instance, for a master thesis huge challenge, 

00:28:31.700 --> 00:28:38.200
that is very difficult to do that. If you talk about, for instance prognosis, we very often follow a 

00:28:38.200 --> 00:28:45.100
cohort, a huge group of people over a longer period of times, what are the effects of smoking? You 

00:28:45.100 --> 00:28:50.500
need to follow people for years. I've been in Australia for years, what 

00:28:50.500 --> 00:28:56.000
are the effects of sunlight? Because man, that sun is intense and they've got a lot of skin cancer going on. 

NOTE
Treffsikkerhet: 91% (HØY)

00:28:56.000 --> 00:29:01.600
If you want to do a prevalent study, that's cross-sectional, that's one moment only.

00:29:01.600 --> 00:29:08.100
I want to know how many students have got covid today. Then I want to measure today, how many 

00:29:08.100 --> 00:29:13.700
people are absent and how many have been confirmed with covid. So depending on your research 

00:29:13.700 --> 00:29:19.600
question, you need to select your study design. Sometimes you can choose between different way of 

00:29:19.600 --> 00:29:25.400
study designs, they will give you a slightly different answer. But also it should be feasible.

NOTE
Treffsikkerhet: 83% (HØY)

00:29:25.950 --> 00:29:32.100
Let's not forget that. So the pros and cons depending on study design to select. Now I'm going to 

00:29:32.100 --> 00:29:37.100
give you a few examples of this.

00:29:37.100 --> 00:29:43.900
So first, let's I'll give you the overview. This is an overview of classification of study 

00:29:43.900 --> 00:29:49.100
designs. There are different ways of looking at study designs, but in general, you can say, we've got 

00:29:49.100 --> 00:29:56.050
descriptive designs and we've got analytical designs. If we are on the analytical side, then we've 

NOTE
Treffsikkerhet: 91% (HØY)

00:29:56.050 --> 00:30:02.700
got observational versus experimental designs. Okay. What is that? I'll give you some examples. 

00:30:02.700 --> 00:30:08.450
Descriptive would be the case report or a number of cases and we're talking about case series, 

00:30:08.450 --> 00:30:15.900
cross-sectional, these prevalence studies. Those are all descriptives. If I go to the right, then it 

00:30:15.900 --> 00:30:21.200
could be cross-sectional but also more. So depends a little bit but more things like observational 

00:30:21.200 --> 00:30:26.300
cohort study, you sit and you observe them for a decade. What is happening? 

NOTE
Treffsikkerhet: 91% (HØY)

00:30:26.300 --> 00:30:31.800
Case control is you've got two groups, one is exposed, the other one is not and you're going to see 

00:30:31.800 --> 00:30:38.900
what happens. Experimental on the other hand is, you introduce an intervention or something. You do 

00:30:38.900 --> 00:30:46.300
something, you develop a certain intervention and you have got two randomized groups, 

00:30:46.300 --> 00:30:52.600
one gets usual care, the other one gets your intervention. So there are different study 

00:30:52.600 --> 00:30:55.900
designs below these descriptive observational and experimental groupings.

NOTE
Treffsikkerhet: 85% (HØY)

00:30:55.900 --> 00:31:03.200
Now, I will give you a few examples. Well, I think some people will debate with me,

00:31:03.200 --> 00:31:12.200
but the most simple study design is an N is 1. It's a single case. I don't think it is 

00:31:12.200 --> 00:31:17.650
a very good design for a master thesis because  you must convince me why you are studying 

00:31:17.650 --> 00:31:25.700
one person only for half a year. So I think that's a No-No, of course, in the analogy in real life 

00:31:26.050 --> 00:31:31.100
people will publish case reports because maybe and that's usually like ''I've got this very interesting case, 

00:31:31.100 --> 00:31:39.100
this child who had ASD and a cochlear implant and how did we deal with it and what happened in 

00:31:39.100 --> 00:31:44.700
the next five years and how did the parents with... '' So you can, it's okay. It's just 

00:31:44.700 --> 00:31:52.150
for a master thesis it's not the right design, I think it is difficult to make 

00:31:52.150 --> 00:31:56.050
sense out of that. Now case series is nothing else 

NOTE
Treffsikkerhet: 76% (HØY)

00:31:56.050 --> 00:32:01.600
but we've got just a handful of people that we are following. And case series are being used a 

00:32:01.600 --> 00:32:09.600
lot in medicine because then for instance, in diseases that are rare, this ALS amyotrophic lateral 

00:32:09.600 --> 00:32:15.600
sclerosis and you want to describe a certain intervention in a handful of patients. That's when 

00:32:15.600 --> 00:32:23.550
people use case series. So it's very often used for a handful of kind of similar patients or cases 

00:32:23.550 --> 00:32:26.050
and you describe what's happening in them. 

NOTE
Treffsikkerhet: 90% (HØY)

00:32:26.050 --> 00:32:32.400
You actually don't use statistics. The group is too small, you simply report on it. I also don't think 

00:32:32.400 --> 00:32:39.500
it's the best way for a master thesis. But again, you could try to convince me. Now this is also, I 

00:32:39.500 --> 00:32:46.500
gave you some examples that you will not do, this is another one. A cohort is, we've got 

00:32:46.500 --> 00:32:54.100
here, two groups. We've got a group of people that are overweight and we've got a group of people 

00:32:54.100 --> 00:32:56.000
with normal weight and then we  

NOTE
Treffsikkerhet: 72% (MEDIUM)

00:32:56.000 --> 00:33:01.300
gonna sit for five years or whatever and we just see, well, what are the problems in 

00:33:01.300 --> 00:33:08.300
each group and we compare the risk factors. So, for instance, the group with overweight 

00:33:08.300 --> 00:33:15.700
may have more cardiovascular problems compared to the people without 

00:33:15.700 --> 00:33:23.350
overweight they may have more diabetes. That is a cohort. Cohorts are usually big numbers. I will 

00:33:23.350 --> 00:33:26.000
follow the cohort SNE students,

NOTE
Treffsikkerhet: 87% (HØY)

00:33:26.000 --> 00:33:30.300
not that big but okay, and I follow them for the next 10 years. And I want to know where they end 

00:33:30.300 --> 00:33:36.700
up with. Yep. So that is a cohort. Also, because you need to follow usually big groups in over a 

00:33:36.700 --> 00:33:43.750
longer period of time, it doesn't seem to be a right study design for you. Cohort is sometimes 

00:33:43.750 --> 00:33:50.200
abused for a very small groups, but I would not call that a cohort. Cohort is everybody born in 

00:33:50.200 --> 00:33:55.950
1963, that's a cohort. It's not just the five people living in my street. That's not a cohort.

NOTE
Treffsikkerhet: 76% (HØY)

00:33:55.950 --> 00:34:01.250
That's too small. Use it for big groups. So it's a group of people with a shared 

00:34:01.250 --> 00:34:07.600
characteristics and you follow them over time, but there are different groups within this cohort. 

00:34:07.600 --> 00:34:13.400
There are overweight one and normal weight, but you just follow them. Now this is case control and 

00:34:13.400 --> 00:34:17.000
that's working backwards in this case. So what we're doing here, we've got a group of cancer 

00:34:17.000 --> 00:34:18.400
patients. 

NOTE
Treffsikkerhet: 83% (HØY)

00:34:18.400 --> 00:34:24.400
I've got a group of no cancer patients and we look in the histories and compare them and see what 

00:34:24.400 --> 00:34:29.500
are the differences. So maybe cancer patient did smoke a lot. Maybe cancer patients got in 

00:34:29.500 --> 00:34:35.800
touch with materials that they should not have been in touch with, maybe it's a family component, 

00:34:35.800 --> 00:34:40.400
all of that. So you've got two groups and you want to understand what could have caused the 

00:34:40.400 --> 00:34:47.300
difference. What could possibly have been risk factors that one group has got the disease or the condition

00:34:48.199 --> 00:34:57.500
and the other group has not. That is a case control study. So first you've got cases, meaning 

00:34:57.500 --> 00:35:04.300
participants, with a certain outcome, and controls and those are participants without outcome, and you 

00:35:04.300 --> 00:35:08.700
need to be sure that they are furthermore comparable. So you don't want to have your cancer 

00:35:08.700 --> 00:35:15.500
patients, like they are 80 plus and your control group is 20 years, that doesn't make sense. They 

00:35:15.500 --> 00:35:18.250
need to be comparable. You don't want one 

NOTE
Treffsikkerhet: 91% (HØY)

00:35:18.250 --> 00:35:24.100
only men and the other only women. So you need to have a look at what are the... also in kids you 

00:35:24.100 --> 00:35:29.500
don't from all boys or and the other one all girls. It should be comparable, those groups. And then 

00:35:29.500 --> 00:35:35.600
you can look back. Like, okay, what is different there? You go into history. That is for an example 

00:35:35.600 --> 00:35:40.850
of a case control, looking for to identify risk factors. 

NOTE
Treffsikkerhet: 81% (HØY)

00:35:40.850 --> 00:35:46.600
Now, this is beautiful. And that's also not going to happen. That's a randomized control trial. That 

00:35:46.600 --> 00:35:54.200
is the top highest level of evidence. We really need more, because what you do, you start and 

00:35:54.200 --> 00:36:02.200
they can be anything. It can be children with social behavioral issues with 

00:36:02.200 --> 00:36:07.000
problems, but you've got your group with whatever condition you looking at. 

NOTE
Treffsikkerhet: 82% (HØY)

00:36:07.000 --> 00:36:15.649
And then add random and that's important at random, tese participants are being assigned an 

00:36:15.649 --> 00:36:22.700
intervention, and it could be a treatment or it could be a placebo or it could be a 

00:36:22.700 --> 00:36:28.100
waiting list or it could be no therapy or usual care about something like that. But this is your 

00:36:28.100 --> 00:36:34.600
intervention compared to a group that will not receive your intervention. Usually it's a no 

00:36:34.600 --> 00:36:36.700
therapy group or placebos, 

NOTE
Treffsikkerhet: 88% (HØY)

00:36:36.700 --> 00:36:41.400
that they say, okay, one gets the medication, the other one gets a 

00:36:41.400 --> 00:36:50.200
placebo medication. But it can be more more complex if you say, just finished another review on 

00:36:50.200 --> 00:36:56.600
again on swallowing. So we have people that actually use neurostimulation. So, putting electrodes 

00:36:56.600 --> 00:37:01.200
and putting some simulation on it. And then they give the other groups, that's the intervention, and 

00:37:01.200 --> 00:37:06.550
the other groups group get Shem stimulation, meaning something happens, but they 

NOTE
Treffsikkerhet: 77% (HØY)

00:37:06.550 --> 00:37:14.800
expect that it's not enough to make a difference. So, again, random allocation is essential in a 

00:37:14.800 --> 00:37:21.000
randomized controlled trial, then one gets a treatment, the other one not and then you compare the 

00:37:21.000 --> 00:37:28.300
outcomes, that is a beautiful RCT. Very often you need a lot of resources. It is very often 

00:37:28.300 --> 00:37:36.650
expensive, but it is a very nice way of doing research. So again, randomly assigned 

NOTE
Treffsikkerhet: 86% (HØY)

00:37:36.650 --> 00:37:42.300
two or even more interventions, you can have more. Each gets a different intervention or no 

00:37:42.300 --> 00:37:48.600
intervention. Groups need to be comparable. You really have got a problem if one is again, there are age 

00:37:48.600 --> 00:37:54.500
difference or the severity of the underlying problem is very different in both groups... 

NOTE
Treffsikkerhet: 79% (HØY)

00:37:54.500 --> 00:38:02.100
And then you compare them. And as you can see that can be here no treatment, placebo treatmenti you 

00:38:02.100 --> 00:38:06.600
don't tell the patients. They do know there's a chance they get a placebo treatment, you need to 

00:38:06.600 --> 00:38:13.800
explain it. But Placebo or Sham and that's anything that's lacking activity, but it looks like 

00:38:13.800 --> 00:38:20.500
they're getting an intervention and that is a randomized control trial. Now I think I'm going to 

00:38:20.500 --> 00:38:23.500
stop right here. 

NOTE
Treffsikkerhet: 91% (HØY)

00:38:26.500 --> 00:38:34.800
Here we are. So we talked about the rcts or randomized control trial. Now, randomized control 

00:38:34.800 --> 00:38:41.700
trials as we know is all about these randomly assigned groups, two or more, it can have more 

00:38:41.700 --> 00:38:48.200
arms, each group is called arms. But each group will have a different either a 

00:38:48.200 --> 00:38:53.700
different intervention, could be a different intensity but something is different, and they 

00:38:53.700 --> 00:38:57.100
should be comparable groups, but usually the control group

NOTE
Treffsikkerhet: 86% (HØY)

00:38:57.100 --> 00:39:02.600
is no treatment or Placebo or Sham, something like that. 

NOTE
Treffsikkerhet: 85% (HØY)

00:39:02.600 --> 00:39:07.800
But these are more complex designs. And again, I don't expect you to do that for your master 

00:39:07.800 --> 00:39:14.500
thesis. For instance if you look at this, we've got a 2 by 2 factorial clinical trial. That can be 

00:39:14.500 --> 00:39:20.550
any trial. But you've got a treatment A here on the left, and a treatment B on 

00:39:20.550 --> 00:39:27.600
the top, and any combination is in this design. Meaning, I've got a group that receives both 

00:39:27.600 --> 00:39:33.050
treatments, a group that receives none of these treatments, one group gets only A  

NOTE
Treffsikkerhet: 88% (HØY)

00:39:33.050 --> 00:39:42.149
and one group gets only B, and all of those groups together, then you can compare like this is B 

00:39:42.149 --> 00:39:47.900
and the other group here horizontal have got all this A. You can have a look at it, 

00:39:47.900 --> 00:39:55.100
but for different conditions, having a 2x2 factorial clinical trial. Now that is quite complex. 

00:39:55.100 --> 00:40:00.100
You've got also another one. But this is about two or more interventions. This is just two by 

00:40:00.100 --> 00:40:02.950
two, but it can be more complex and you want to 

NOTE
Treffsikkerhet: 81% (HØY)

00:40:02.950 --> 00:40:08.400
have all these combinations in your design and you want to know whether each intervention is 

00:40:08.400 --> 00:40:14.500
efficacious or also whether a combination is doing a better job. Sometimes you don't know. Should I 

00:40:14.500 --> 00:40:23.600
get A or B or A and B, that's what you can do here. That's what you can check. Now, other thing 

00:40:23.600 --> 00:40:30.100
what people sometimes do is cross-over, and that means again, I've got study participants,

00:40:30.100 --> 00:40:33.050
I randomize them for treatment A or treatment B, 

NOTE
Treffsikkerhet: 83% (HØY)

00:40:33.050 --> 00:40:40.800
whatever that is, then I've got a so-called wash out period and that is a period that you say: 

00:40:40.800 --> 00:40:47.000
''Okay, let's wait until the effects of the treatments are no longer doing their their bits. There are 

00:40:47.000 --> 00:40:54.300
no longer influencing. '' And then I swap them. So then this group, A gets now treatment B, and B gets A.

00:40:54.300 --> 00:41:02.100
Now that happens we talk about a cross over study design. So, each group gets both

00:41:02.899 --> 00:41:06.400
treatments but in a different order. 

NOTE
Treffsikkerhet: 91% (HØY)

00:41:06.400 --> 00:41:13.050
Okay, so they intentionally cross over to the other intervention on. That's what you're going to do. 

00:41:13.050 --> 00:41:19.900
So everybody first gets one intervention, usually random allocation that makes it much stronger and 

00:41:19.900 --> 00:41:26.900
then you switch after that wash out period. Now these one nothing but examples and we will talk 

00:41:26.900 --> 00:41:34.300
again on Wednesday more about this. But why would we even care about study designs? Because 

00:41:34.300 --> 00:41:36.000
different types of questions are 

NOTE
Treffsikkerhet: 91% (HØY)

00:41:36.000 --> 00:41:43.500
best answered by different types of study designs. So if you are preparing for your research 

00:41:43.500 --> 00:41:51.200
proposal and for your master thesis, you need to select a study design that is feasible but of 

00:41:51.200 --> 00:41:58.700
course also fits your research question, and the criteria,you used to praise your research will 

00:41:58.700 --> 00:42:06.000
vary depending on your study design and that is what brings us to critical appraisal tools or our Cats. 

00:42:08.150 --> 00:42:16.700
Now, what is the aim of a Cat? What is the aim of critical appraisal? What you do is you want to 

00:42:16.700 --> 00:42:24.350
identify threats. You want to know what could possibly -in simple English- mess up my research, 

00:42:24.350 --> 00:42:31.300
you're looking for the problems in your design. You look at threats to validity of your research 

00:42:31.300 --> 00:42:36.700
findings from the literature and you provide consumers of research evidence the opportunity to make 

00:42:36.700 --> 00:42:37.649
informed decisions

NOTE
Treffsikkerhet: 83% (HØY)

00:42:37.649 --> 00:42:43.050
about the quality of research evidence for. There's a lot but what you are doing is actually 

00:42:43.050 --> 00:42:49.700
you are looking at the article and step-by-step you go through this study and you look at  

00:42:49.700 --> 00:42:56.600
is there selection bias, is there is device that's misused, is the methods okay, all of that and you've 

00:42:56.600 --> 00:43:00.950
got tools for that and you call those critical appraisal tools. 

NOTE
Treffsikkerhet: 83% (HØY)

00:43:00.950 --> 00:43:07.000
Now, the critical appraisal is also it helps 

00:43:07.000 --> 00:43:13.300
you how to write. So, what you would do, you identify most relevant papers and it is to 

00:43:13.300 --> 00:43:19.400
distinguish evidence from opinion and some options, misreporting. It is about assessing the validity 

00:43:19.400 --> 00:43:24.600
and assessing the usefulness and clinical applicability of the study, the generalizability of your 

00:43:24.600 --> 00:43:29.500
study. How do you do that? You do that with Cats. 

NOTE
Treffsikkerhet: 79% (HØY)

00:43:29.500 --> 00:43:36.600
Now, I'll give you an example. Validity of a study, for instance if we talk about a treatment for example

00:43:36.600 --> 00:43:43.700
talking about an RCT, then you would like to know, are the patients randomized? You would like to 

00:43:43.700 --> 00:43:48.700
know what's the allocation concealed, meaning, that the patient's know they were going to a 

00:43:48.700 --> 00:43:54.800
placebo or not. That's quite important. It will influence. Are the baseline 

00:43:54.800 --> 00:43:59.600
characteristics the same? So gender distribution, 

NOTE
Treffsikkerhet: 83% (HØY)

00:43:59.600 --> 00:44:07.100
age distributions, severity of the disorder, is that kind of comparable? If one is much 

00:44:07.100 --> 00:44:13.600
more, is more severe than the others how can you can compare in the end? It's blinded.  

00:44:13.600 --> 00:44:21.000
What blinded means? For instance, if I get data from your RCT, I do my analysis and I 

00:44:21.000 --> 00:44:27.500
don't know what group is receive the intervention. So I only know I've got an A, and a B group, but 

00:44:27.500 --> 00:44:29.550
I don't know which one was the actual intervention. 

NOTE
Treffsikkerhet: 91% (HØY)

00:44:29.550 --> 00:44:34.600
That is blinded for me, but it can also that participants, they should don't 

00:44:34.600 --> 00:44:41.000
know which arm they are assigned to. So that has all got to do with blinding. 

00:44:41.000 --> 00:44:48.400
You will have a look at follow-up completeness, because what happens very often is that authors 

00:44:48.400 --> 00:44:56.200
write in that abstract '' We had 400 participants. '' Yeah, but the statistics was maybe only 100, 

00:44:56.200 --> 00:44:59.399
then ignore the 400, your 

NOTE
Treffsikkerhet: 75% (MEDIUM)

00:44:59.399 --> 00:45:06.900
results are based on 100. So you always look at what is the statistic based on, that is the final 

00:45:06.900 --> 00:45:13.500
number that you need to post on data set and it's very normal in research that we lose some 

00:45:13.500 --> 00:45:18.500
patients. Lost to follow-up, they didn't want to do it anymore, they walk away, 

00:45:18.500 --> 00:45:25.800
that is normal. But if you report on ''we did a study in an X number of participants'', I want to 

00:45:25.800 --> 00:45:29.500
know how many participants completed the intervention. 

NOTE
Treffsikkerhet: 75% (MEDIUM)

00:45:29.500 --> 00:45:36.000
And that has got to do with validity, Etc, and reliability of your study. So is there an 

00:45:36.000 --> 00:45:41.800
intention-to-treat? So that has got to do with follow-up and all of that, but that

00:45:41.800 --> 00:45:48.700
was just in general. Now, if we look at etiology for instance, and we're interested in what 

00:45:48.700 --> 00:45:55.800
is the cause of it, and we could do a cohort or case control study. Then again comparison groups, 

00:45:55.800 --> 00:45:59.550
always comparison groups needs to be similar at baseline, 

NOTE
Treffsikkerhet: 86% (HØY)

00:45:59.550 --> 00:46:06.300
the characteristics, the patient characteristics or the study population characteristics. Are the 

00:46:06.300 --> 00:46:12.300
outcomes and exposures measure the same for both popular groups? So did they received the same 

00:46:12.300 --> 00:46:19.050
intervention, you need to know the dosage of that and again was the follow-up complete? 

NOTE
Treffsikkerhet: 89% (HØY)

00:46:19.050 --> 00:46:26.600
So every study design has got his kind of some things are overlapping, some things are items are 

00:46:26.600 --> 00:46:34.100
particular for that specific type of research. Qualitative research, are the aims of the research 

00:46:34.100 --> 00:46:40.300
clearly stated, was the methodology appropriate and usually you get criteria that you can read like 

00:46:40.300 --> 00:46:47.000
what is appropriate and what is not appropriate. How did they recruit, was that okay, were the data 

00:46:47.000 --> 00:46:49.400
collected in a way that adress the research, 

NOTE
Treffsikkerhet: 89% (HØY)

00:46:49.400 --> 00:46:54.700
issue like that. They just collect any data or had these strategies for that. What is the 

00:46:54.700 --> 00:47:00.700
relationship between research and participants? Is there bias, is there selection bias or anything like that.

00:47:00.700 --> 00:47:09.350
And what about the data analysis used, again you need Cats to make this happen. Cats are 

00:47:09.350 --> 00:47:17.700
structured forms for different study designs. And they lead you how to evaluate methodological 

00:47:17.700 --> 00:47:19.350
quality of studies. 

NOTE
Treffsikkerhet: 82% (HØY)

00:47:19.350 --> 00:47:27.300
Yeah, and this is one of them. So this is for instance if we look at a single case study, you've 

00:47:27.300 --> 00:47:33.700
got also Cats for single case studies. A single case that is the N is 1 trial, and that is the 

00:47:33.700 --> 00:47:40.100
consort extension for reporting N of 1 trials, 2015 statement. They've got all these beautiful 

00:47:40.100 --> 00:47:48.900
names, in short set. Now you can see here there is a very often these cats have got websites 

00:47:49.400 --> 00:47:56.700
we can see more information about it. So just to give you an example. So this is a single 

00:47:56.700 --> 00:48:02.400
case study. We've got N of is 1. It's about an experimental clinical study design to 

00:48:02.400 --> 00:48:08.200
determine the effect of an intervention in a single study participant. Remember, we are at the 

00:48:08.200 --> 00:48:16.100
bottom of a hierarchy, this low evidence, but you can do that. And then would you use is this Cat

00:48:16.100 --> 00:48:19.600
which will help you a little bit.

NOTE
Treffsikkerhet: 86% (HØY)

00:48:19.600 --> 00:48:26.700
So that is about an experimental clinical study. You've got one single case and sent has got for 

00:48:26.700 --> 00:48:33.300
each part of that article. It has got items, for instance title and Abstract sense as well. Then 

00:48:33.300 --> 00:48:37.550
you should say that in your title. You should mention that we're talking about a case study. 

00:48:37.550 --> 00:48:43.400
Introduction, you need to give it a rationale, and that is what I also tell to you students very 

00:48:43.400 --> 00:48:49.500
often: Why do you do a N is 1? Why did you not do 5

NOTE
Treffsikkerhet: 76% (HØY)

00:48:49.500 --> 00:48:59.300
Or 10 or 20? And then it's very weak to say I ran out of time, half a year, hmm. So it is 

00:48:59.300 --> 00:49:06.400
make sense that you say a rationale of N is 1. And that's why N is 1 very often used 

00:49:06.400 --> 00:49:13.600
for special case descriptions. Methods, and then we talked about the trial design, the participants 

00:49:13.600 --> 00:49:19.000
recruitment, the interventions. And you got all these criteria that you need described like, 

NOTE
Treffsikkerhet: 82% (HØY)

00:49:19.300 --> 00:49:24.700
okay, what have they done, did they describe the trial design, the plant number of periods, the 

00:49:24.700 --> 00:49:29.800
duration of each period, how often did they measure? Because very often it's repeated measurement. 

00:49:29.800 --> 00:49:38.500
What they followed, one case over time... What about the participants, the diagnosis of 

00:49:38.500 --> 00:49:45.400
disorder... The criteria, like, if you say ASD. Well, what are criteria? How do you 

00:49:45.400 --> 00:49:49.300
diagnose ASD? Autism spectrum disorders. How do you do decide

NOTE
Treffsikkerhet: 85% (HØY)

00:49:49.300 --> 00:49:56.100
these participants meet those criteria, who confirmed that diagnosis? Because again, you want 

00:49:56.100 --> 00:50:02.300
to make sure that you are actually having the right disciplines there. And this is just for an N is 1. 

00:50:02.300 --> 00:50:04.200
So that is said.

NOTE
Treffsikkerhet: 77% (HØY)

00:50:04.200 --> 00:50:09.900
You also got other Cats and I just will show you a number you don't need to learn them by heart, 

00:50:09.900 --> 00:50:15.600
but that you know you've got these different ones. So Strop is for instance, for cross-sectional 

00:50:15.600 --> 00:50:23.500
studies. It's short for strengthening the reporting of 

00:50:23.500 --> 00:50:29.200
observational studies in epidemiology. And again, this is website again where you can see it 

00:50:29.200 --> 00:50:34.750
and it is one moment of measurement and you want to know for instance, the prevalence of 

00:50:36.250 --> 00:50:44.000
shyness in eight year olds. I measure it today. Something like that. Now, 

00:50:44.000 --> 00:50:49.600
how does it look? Very often you can see the same structure. You've got here the criteria. 

00:50:49.600 --> 00:50:55.000
This is the checklist that you just need to scroll down. And for each thing like study design and 

00:50:55.000 --> 00:50:59.700
setting, they say, well, did they describe all of that correctly, did they describe the setting, the 

00:50:59.700 --> 00:51:06.250
locations, did they describe how they were going to address potential sources of bias, did they 

NOTE
Treffsikkerhet: 90% (HØY)

00:51:06.250 --> 00:51:13.000
explain how the study size was arrived at, and you get in this of course there is more information for each 

00:51:13.000 --> 00:51:18.900
Cat at the website. This is just in short, then they've got criteria about the study participants, 

00:51:18.900 --> 00:51:25.200
again, describe how did they describe the participants, the diagnosis and they've got this for 

00:51:25.200 --> 00:51:32.100
different type of study designs, they give you information what they want. So all these criteria you 

00:51:32.100 --> 00:51:35.850
just tick off. Same with outcomes. 

NOTE
Treffsikkerhet: 91% (HØY)

00:51:35.850 --> 00:51:42.900
I see that very, very often people use outcome measures that are not valid or reliable. 

00:51:42.900 --> 00:51:49.300
That is a disaster in my mind because then you can't interpret your outcome. If you use a measure, 

00:51:49.300 --> 00:51:57.700
that is not valid, why using it? Usually very often, it is unknown. I can live with unknown 

00:51:57.700 --> 00:52:03.800
psychometrics. I cannot live with measures that have been proven to be not valid. That's a waste of 

00:52:03.800 --> 00:52:06.149
your energy. But anyhow, 

NOTE
Treffsikkerhet: 82% (HØY)

00:52:06.149 --> 00:52:12.100
you've got the outcomes, eligibility criteria, which means that very often you need to 

00:52:12.100 --> 00:52:18.600
explain ''This is where we recruited. These were the criteria. These were the people we excluded. 

00:52:18.600 --> 00:52:24.500
These are the people that we lost during follow-up '' , and very often it comes with a flow diagram. 

00:52:24.500 --> 00:52:31.100
Just give them the overview of ''We start with the 100 that only 40 enrolled, 10 were 

00:52:31.100 --> 00:52:36.100
lost to follow-up. We've got complete data set for 25. Something like that. '' You want to know that. 

00:52:37.400 --> 00:52:45.000
Now, this is an example of one that we did. We did cross a cross-sectional study. It was actually a 

00:52:45.000 --> 00:52:50.800
review that we did and we wanted to know the prevalence, that's a cross-sectional study, of 

00:52:50.800 --> 00:52:58.050
swallowing problems in a number of groups, a number of diseases, Parkinson, 

00:52:58.050 --> 00:53:03.900
Alzheimer's, Etc. Because we know there's a lot of strong problems in these groups. So we retrieved 

00:53:03.900 --> 00:53:07.350
all the literature but then every single study that 

NOTE
Treffsikkerhet: 78% (HØY)

00:53:07.350 --> 00:53:15.300
ported on prevalence, we used these criteria called axis.

00:53:15.300 --> 00:53:22.100
So that was cross sectional studies and Axis has got 20 items and you can you mark each 

00:53:22.100 --> 00:53:29.400
item as yes, no, do not know. Maximum is 20, minimum is 0. So each article that you include in your 

00:53:29.400 --> 00:53:35.300
systematic review, for instance need to have been assessed with a Cat, in this case because we were 

00:53:35.300 --> 00:53:37.350
only interested in cross sectional, 

NOTE
Treffsikkerhet: 87% (HØY)

00:53:37.350 --> 00:53:44.600
we could use the Axis. And this is just an endless list. But again, 

00:53:44.600 --> 00:53:52.900
you can see here introduction methods, results and discussion. So very often it follows the 

00:53:52.900 --> 00:53:59.800
order, the recipe of how to write a study, how to write it a paper and that makes sense. You can 

00:53:59.800 --> 00:54:02.000
see that in most Cats. 

NOTE
Treffsikkerhet: 91% (HØY)

00:54:02.000 --> 00:54:08.600
Okay, back to the control trials. These are the randomized control trials. There is something that's 

00:54:08.600 --> 00:54:17.500
called a consort statement and that is again, short for Consolidated Standards of Reporting Trials, 

00:54:17.500 --> 00:54:22.500
reporting of randomized control, trials. I don't know who comes up with these endless names but it 

00:54:22.500 --> 00:54:29.250
is consort in short. And we talk about RCT's. So, if you want to know about the methodological 

00:54:29.250 --> 00:54:32.100
quality of your RCT study, 

NOTE
Treffsikkerhet: 91% (HØY)

00:54:32.100 --> 00:54:39.000
this is what you use. And again, you can see --and I'll make it a little bit bigger, but you can see 

00:54:39.000 --> 00:54:46.300
already-- title and Abstract, intro, methods, --and there we go, and I make it a bit bigger.-- Same, this the 

00:54:46.300 --> 00:54:51.600
same content, it's the title, abstract about background, objectives as they have 

00:54:51.600 --> 00:54:57.800
been described, then they talk about design, participant, interventions and outcome measures, user sample, 

00:54:57.800 --> 00:55:01.950
sizes on adequate number. Have they been a trend randomized and blinded?

NOTE
Treffsikkerhet: 77% (HØY)

00:55:01.950 --> 00:55:08.900
And are the stats correct? And again, for each item in these Cats, you will have always a manual 

00:55:08.900 --> 00:55:11.550
that explains what you need to have a look at. 

NOTE
Treffsikkerhet: 85% (HØY)

00:55:11.550 --> 00:55:17.800
And this is the same, the second bit and you can just move to the results with the talk about. 

00:55:17.800 --> 00:55:23.800
This what I previously said, this diagram that very often is included to say ''we start 

00:55:23.800 --> 00:55:32.500
with the 100 patients, 10 died, 10 walked away. We had a crash computer, another 10 loss. '' That's 

00:55:32.500 --> 00:55:38.600
what you need to, give the flow chart and at the bottom you say these are the patients that we 

00:55:38.600 --> 00:55:41.150
have pre and post data for. 

NOTE
Treffsikkerhet: 91% (HØY)

00:55:41.150 --> 00:55:48.200
Again, recruitment based on data numbers, analysis -- this is a little bit if you've got 

00:55:48.200 --> 00:55:55.900
additional analysis. -- I'm not going to get into all these details, but look at that 

00:55:55.900 --> 00:56:01.700
limitations, we talked about that before. Also, in your research question, in your research 

00:56:01.700 --> 00:56:08.700
proposal. There is always something like limitation and that makes sense. You are not improving the 

00:56:08.700 --> 00:56:11.400
world. So, do you know your limitations? 

NOTE
Treffsikkerhet: 82% (HØY)

00:56:11.400 --> 00:56:17.900
Can you generalize it? And is the interpretation correct? It's theinterpretation of the data, is that 

00:56:17.900 --> 00:56:25.350
correct? Can we interpret the data actually? Then there are things with RCTs as usual, 

00:56:25.350 --> 00:56:30.800
register them on website so that everybody knows what you're doing Etc. But you need to report on 

00:56:30.800 --> 00:56:37.900
it. Now, all these details, if you do and we're doing reviews on RCTs, every article is being 

00:56:37.900 --> 00:56:39.850
assessed like this. 

NOTE
Treffsikkerhet: 90% (HØY)

00:56:39.850 --> 00:56:50.100
Again, stop me if you've got a question, I know it's a lot. So that is just an explanation of these 

00:56:50.100 --> 00:56:56.200
additional analysis where you can do subgroup analysis Etc. So you've got the overall and then you 

00:56:56.200 --> 00:57:02.000
can say ''Well maybe I don't want to look just at the intervention. Maybe I want to look at dosage of 

00:57:02.000 --> 00:57:09.500
the intervention. '' That is short and intense, better than long and not that intense. Those could be supper analysis.

00:57:09.500 --> 00:57:26.500
Kine: In randomized controlled  studies, how common is it for big studies to not be backed up by RCTs 

00:57:26.500 --> 00:57:32.700
or should you expect studies to be backed up by randomized controlled studies? 

00:57:32.700 --> 00:57:39.500
Renee: The problem is that we more and more realize that we 

NOTE
Treffsikkerhet: 86% (HØY)

00:57:39.500 --> 00:57:48.050
need International multicenter trials and we need that because it is very difficult to run an RCT. 

00:57:48.050 --> 00:57:52.200
It's costing, you need a lot of money. You need many therapists or clinicians involved or 

00:57:52.200 --> 00:57:59.200
educationalists, but it's the best evidence. And what we nowadays have is all quite a few small 

00:57:59.200 --> 00:58:05.450
studies. Now, there is a way of combining that, we showed that with the when we talked about reviews, 

00:58:05.450 --> 00:58:09.649
that you've got all these small studies and you try with meta-synthesis or 

NOTE
Treffsikkerhet: 90% (HØY)

00:58:09.649 --> 00:58:16.600
meta-analysis to combine it to have a higher level evidence that way. But of course, the best way is we 

00:58:16.600 --> 00:58:26.000
need big RCTs. The bigger the RCTs is also, then you can look at confounders or look at different 

00:58:26.000 --> 00:58:32.800
moderators (is a better word). Moderator, meaning it's not necessary confounding because confounding 

00:58:32.800 --> 00:58:39.500
is a really a problem, but a moderator can be like, maybe it has 

NOTE
Treffsikkerhet: 80% (HØY)

00:58:39.500 --> 00:58:44.900
influence on the outcome, maybe age is an influence, then you can do additional analysis if your 

00:58:44.900 --> 00:58:51.700
group is big enough. I've got the intervention randomized. I look at pre post. I've compare both 

00:58:51.700 --> 00:58:58.700
groups, but maybe I want to say, well actually maybe age is also of influence. Then you look within 

00:58:58.700 --> 00:59:03.700
the group, you look at, okay, how are the elderly doing? How is it the bigger your group the more 

00:59:03.700 --> 00:59:09.649
moderators you can put in. So I'm not sure that I'm giving a correct answer 

NOTE
Treffsikkerhet: 73% (MEDIUM)

00:59:09.649 --> 00:59:17.200
to you, but we really need bigger studies to take care of moderators to compare really to have 

00:59:17.200 --> 00:59:24.900
more sufficient statistical power. If the groups are too small, also not randomized, we don't 

00:59:24.900 --> 00:59:31.950
have enough power to say, ''actually, this is statistically significantly working. ''

NOTE
Treffsikkerhet: 85% (HØY)

00:59:31.950 --> 00:59:38.000
We can only say there may be a tendency. With a big a RCT you hope that you can actually say 

00:59:38.000 --> 00:59:47.300
this is working or not.  

00:59:47.300 --> 01:00:29.450
*Kine asking question*

01:00:29.450 --> 01:00:37.800
Usually, if you want to do RCTs, it's almost impossible 

01:00:37.800 --> 01:00:42.350
to do it without funding, because it's time-consuming, resource consuming, 

NOTE
Treffsikkerhet: 90% (HØY)

01:00:42.350 --> 01:00:49.000
it costs a lot of money. It is the best evidence though. So if you go for national brands, 

01:00:49.000 --> 01:00:57.000
Norwegian research Council, they really would expect you to do RCTs then. You ask money, you come 

01:00:57.000 --> 01:01:02.200
up with a good research design and that is usually in interventions we talk about RCTs, 

01:01:02.200 --> 01:01:08.700
correct. So then I expect better. But if you don't have funding is very difficult to do a good one. 

01:01:11.500 --> 01:01:18.500
There is another one, the randomized control trial the Rob, the vice Cochran risk of bias. And we 

01:01:18.500 --> 01:01:26.400
talked already about the Cochran. Cochran is a database and it's got many reviews there of rcts in 

01:01:26.400 --> 01:01:32.700
different areas. We looked at that one. You can go to their website, and they had also a Cat, a 

01:01:32.700 --> 01:01:38.200
critical appraisal tool for randomized control trials. And that is the Rob. Now, we've done that 

01:01:38.200 --> 01:01:42.399
with four students last year, two pairs of four group and 

NOTE
Treffsikkerhet: 84% (HØY)

01:01:42.399 --> 01:01:49.200
it's complex, time-consuming, but it is very likely the best cat for RCTs. And since we wanted 

01:01:49.200 --> 01:01:56.650
to publish, we did the painful thing and we did the Rob, but it was a bit painful. What Rob does is 

01:01:56.650 --> 01:02:04.000
this is a picture that it gives you four different... It is about 

01:02:04.000 --> 01:02:09.600
risk of bias in included studies, and it does that for overall, but also for incident 

01:02:09.600 --> 01:02:12.250
reporting of results, the outcome, 

NOTE
Treffsikkerhet: 81% (HØY)

01:02:12.250 --> 01:02:20.900
missing data. How did the researcher, authors dealt with missing data? And we 

01:02:20.900 --> 01:02:27.800
all have got missing data because people disappear Etc. Did things change from the intervention, 

01:02:27.800 --> 01:02:33.800
did you intervene? Did something go different than planned? And how about randomization? Now, then 

01:02:33.800 --> 01:02:40.700
you can see green is good. Red is bad. That's actually how you read it. So green is overall pretty 

01:02:40.700 --> 01:02:42.300
good. There are some concerns,

NOTE
Treffsikkerhet: 90% (HØY)

01:02:42.300 --> 01:02:50.400
it's yellow. And high risk is the red. So we did, this is from a review we were doing, a review 

01:02:50.400 --> 01:02:55.500
on RCTs. And you can see while most of them are pretty good there's a little bit of... especially the 

01:02:55.500 --> 01:03:02.500
red that is problematic there. But this is the overview of all included studies. It also gives you 

01:03:02.500 --> 01:03:09.550
this paper, this one and this, --actually this table goes on because it got only goes from A to D,

01:03:09.550 --> 01:03:12.200
but I thought you don't need E to Z--

NOTE
Treffsikkerhet: 84% (HØY)

01:03:12.200 --> 01:03:18.600
but there are a lot more studies and for each individual study it gives you the same criteria. And as you can see 

01:03:18.600 --> 01:03:25.700
okay, this study, this one has gotten overall problem. There's a high risk. And so you can see 

01:03:25.700 --> 01:03:32.400
these are pretty good because you can see all these items are scored. Green is good. But here 

01:03:32.400 --> 01:03:37.900
are some doubtful things already, but red is really the problem. So you get a nice overview for your 

01:03:37.900 --> 01:03:42.399
paper. And you can also add the individual studies. But again, 

NOTE
Treffsikkerhet: 83% (HØY)

01:03:42.399 --> 01:03:48.500
Rob is not easy and the students did not like it. Neither did I. But it was a good one. It was a 

01:03:48.500 --> 01:03:51.200
good one and you can't always avoid. 

NOTE
Treffsikkerhet: 86% (HØY)

01:03:52.600 --> 01:03:59.800
Reviews, hang in there. I know it's a little bit boring but hang in. Reviews, if you are 

01:03:59.800 --> 01:04:06.900
willing, if you are considering performing a systematic review, you need to discuss Prisma,  

01:04:06.900 --> 01:04:12.700
Prisma stands for preferred reporting items for systematic reviews and meta-analysis. If you intend 

01:04:12.700 --> 01:04:18.900
to publish and you do a master thesis by publication, you cannot avoid Prisma. It's as simple as 

01:04:18.900 --> 01:04:22.000
that. There is an updated Prisma 2020 

NOTE
Treffsikkerhet: 83% (HØY)

01:04:22.000 --> 01:04:27.550
and I forgot to update this, but we did it in the other hand out. So you've got the updated list, 

01:04:27.550 --> 01:04:33.000
but this is the checklist. We talked about when we talked about literature reviews and 

01:04:33.000 --> 01:04:39.100
you can see the title, The Abstract, the introduction, the methods. We've seen 

01:04:39.100 --> 01:04:45.100
that by now, and then results, discussion, funding, Etc. You need to record every paper in your 

01:04:45.100 --> 01:04:51.800
systematic review. When you perform, you need to report on this, so this is a 

NOTE
Treffsikkerhet: 85% (HØY)

01:04:51.800 --> 01:05:00.850
a checklist for writing a review, to see the methodological quality of a systematic review. No way 

01:05:00.850 --> 01:05:08.200
avoiding Prisma if you do a systematic review and even if you do a scoping review, then nowadays, 

01:05:08.200 --> 01:05:14.000
they also want you to do systematic searches. So you still are hanging with Prisma. It's just not 

01:05:14.000 --> 01:05:21.000
the whole checklist. There's a different checklist for that. Okay, then there is such a thing as 

01:05:21.000 --> 01:05:21.800
diagnostic studies.

NOTE
Treffsikkerhet: 89% (HØY)

01:05:21.800 --> 01:05:28.000
And that is, we will have two lectures on diagnostic accuracy and that's got to do with 

01:05:28.000 --> 01:05:36.300
sensitivity, specificity and things like that. So you've got a tool and you want to know how good is 

01:05:36.300 --> 01:05:44.900
my tool with diagnosing ASD or diagnosing social problems. So that is got to with diagnostic 

01:05:44.900 --> 01:05:51.700
accuracy. It's got to do with sensitivity, specificity 

NOTE
Treffsikkerhet: 90% (HØY)

01:05:51.700 --> 01:05:58.000
and we will talk about that later. And if you are looking at studies like that, then you've got 

01:05:58.000 --> 01:06:03.300
also there, you've got another Cat, which is called QUADAS 2, which implies there must be a QUADAS 1,

01:06:03.300 --> 01:06:08.600
and it's a quality assessment tool for diagnostic accuracy studies. Again, 

01:06:08.600 --> 01:06:15.700
there's a website and this is again, the whole thing. Now, this is the same, but it's 

01:06:15.700 --> 01:06:20.700
too small so I made a little bit bigger. So it is about on the left you can see description, 

01:06:20.700 --> 01:06:21.700
signaling questions,

NOTE
Treffsikkerhet: 77% (HØY)

01:06:21.700 --> 01:06:29.400
risk of bias, Etc. And on top you can see patient selection, index tests, Etc. Because when you look at 

01:06:29.400 --> 01:06:35.399
diagnostics -and again, we will talk about it- and you want to know whether your tool is a good one, 

01:06:35.399 --> 01:06:42.350
you need to compare with something like a reference. Like if you develop a new screening tool 

01:06:42.350 --> 01:06:48.500
and you want to know how fantastic that screening tool works in diagnosing ADHD, then you want to 

01:06:48.500 --> 01:06:51.750
compare it with the gold standard screening tool or whatever 

NOTE
Treffsikkerhet: 75% (MEDIUM)

01:06:51.750 --> 01:06:57.900
is out there and compare it and you can see how good your tool is 

01:06:57.900 --> 01:07:02.250
actually working. So they're talking about, that's what they mean with index test and 

01:07:02.250 --> 01:07:08.400
reference standard. We will talk about that later. But again, different study designs, different 

01:07:08.400 --> 01:07:15.100
Cats are needed. You get beautiful pictures, this is what some people do or this is that you can 

01:07:15.100 --> 01:07:20.400
see green is good and red is usually bad. In this case has orange, so you can all 

01:07:20.400 --> 01:07:21.800
either do all these beautiful 

NOTE
Treffsikkerhet: 75% (MEDIUM)

01:07:21.800 --> 01:07:29.800
smileys or mon smileys but also a little bit like Rob to you can get here the overall picture for 

01:07:29.800 --> 01:07:37.850
all these studies and again green is good, so there is a legenda with that, not going to 

01:07:37.850 --> 01:07:46.700
get more into detail here. We've got hours for diagnostic performance anyhow. Now this is from the 

01:07:46.700 --> 01:07:51.750
Cochrane and I'm just checking on time.

NOTE
Treffsikkerhet: 81% (HØY)

01:07:51.750 --> 01:07:57.500
Cochrane has got a simple one. We did diagnostic reviews, meaning, we've retrieved 

01:07:57.500 --> 01:08:04.500
from the literature, all screening tools that were looking for swallowing problems. And we needed to 

01:08:04.500 --> 01:08:10.900
know for each article if it is a good one or not. At that time the Cochrane had a very simple one, nine 

01:08:10.900 --> 01:08:16.600
items. And you can see this, the first seven had to do with validity of my tool, blind 

01:08:16.600 --> 01:08:21.500
interpretation of reference test and index, says to the one test compared to the other one. 

NOTE
Treffsikkerhet: 79% (HØY)

01:08:21.500 --> 01:08:28.300
Did you use the index test on the reference desk for all participants? It's always got to 

01:08:28.300 --> 01:08:34.799
do with validity, study selection, participant selection. Then we had a little bit about 

01:08:34.799 --> 01:08:42.300
generalizability. Meaning, can you generalize this study population? If you talk about a screening 

01:08:42.300 --> 01:08:50.600
tool in patients with neurological swallowing problems, can I generalize that one to also 

01:08:50.600 --> 01:08:51.750
oncological problems? 

NOTE
Treffsikkerhet: 75% (MEDIUM)

01:08:51.750 --> 01:08:57.700
Or is this really just for neurology? So it's about generalizebility. If you cannot 

01:08:57.700 --> 01:09:04.000
generalize, it doesn't necessarily need to be bad, but it is a restriction. So you can't use it in 

01:09:04.000 --> 01:09:08.899
other groups. But if you can only use it for instance in neurology, that's still a big 

01:09:08.899 --> 01:09:15.300
group. But if you can only use it in neurology patients, that are 

NOTE
Treffsikkerhet: 89% (HØY)

01:09:15.300 --> 01:09:20.899
five years old, that is not generalizeable anymore. Then you've got a problem, that's too 

01:09:20.899 --> 01:09:30.100
precise. Last bit has got to do with reliability. So can you reproduce the data? Did they also 

01:09:30.100 --> 01:09:37.500
define normal versus or abnormal? Screenings, and we will talk about it. Screening test is not just 

01:09:37.500 --> 01:09:44.899
an evaluation. It is actually to identify those at risk. Now, if I've got a screening test,  

NOTE
Treffsikkerhet: 91% (HØY)

01:09:44.899 --> 01:09:52.399
I need to know when do I decide someone has got an abnormal score is at risk. So, with diagnostic 

01:09:52.399 --> 01:10:00.300
tools you want to know, did they give us good enough definitions of what is... if you talk 

01:10:00.300 --> 01:10:06.200
about ASD or ADHD or anything, very often there are a number of criteria as you've got these 

01:10:06.200 --> 01:10:13.500
cut holes and especially with this screen if you only want to know ''Well, is there a risk of...?

01:10:13.500 --> 01:10:15.100
Are they at risk? '' So you don't

NOTE
Treffsikkerhet: 71% (MEDIUM)

01:10:15.100 --> 01:10:22.500
diagnose yet, but you ask are they at risk of autism spectrum disorders? You need to make sure that 

01:10:22.500 --> 01:10:27.000
they defined  what is normal and abnormal according to the authors. 

NOTE
Treffsikkerhet: 75% (MEDIUM)

01:10:27.100 --> 01:10:36.800
Okay. Now the older Cats that I discussed so far, we're only focusing at one time particular type 

01:10:36.800 --> 01:10:43.200
of study designs. Sometimes you do a review and you've got all these different study designs in your review

01:10:43.200 --> 01:10:51.800
and you don't want to do for study by Johnson, you use the Rob 2, and for the other one

01:10:51.800 --> 01:10:56.850
you use the Qualsyst... In that case actually 

NOTE
Treffsikkerhet: 91% (HØY)

01:10:56.850 --> 01:11:04.050
I would prefer to have one cat that can deal with different study designs. And Qualsyst by Kmet

01:11:04.050 --> 01:11:11.900
is one of those study or Cats that can include different types of study designs. It's about

01:11:11.900 --> 01:11:19.300
criteria. *Kine asking question*

01:11:19.300 --> 01:11:35.450
*Kine asking question* 


01:11:35.450 --> 01:11:42.300
Renee: All that we discuss is critical appraisal tools. I call that short Cats.

01:11:48.750 --> 01:11:56.700
And this one, this Cat is called Qualsyst and Kmet is the name of the first author and the 

NOTE
Treffsikkerhet: 74% (MEDIUM)

01:11:56.700 --> 01:12:03.300
advantage of this cat is that it allows different study designs being rated by the same Cat which is 

01:12:03.300 --> 01:12:10.600
easier and it comes with an overall score and not all Cats do that. Sometimes you use the Cat and 

01:12:10.600 --> 01:12:17.900
then you've got 30 individual items. And then what? So this one is gives you a kind of overall 

01:12:17.900 --> 01:12:24.800
rating. And this is how it looks like, you've got here all these criteria and you can say, yes, 

01:12:24.800 --> 01:12:26.850
that's two points, one point,

NOTE
Treffsikkerhet: 88% (HØY)

01:12:26.850 --> 01:12:34.100
zero, sometimes not applicable. For instance, item 5 is about random allocation. Well, if 

01:12:34.100 --> 01:12:40.000
we're talking about a case series, there's no random allocation at all. So then you just say not 

01:12:40.000 --> 01:12:46.900
applicable. So the advantage of courses is something like, you've got different types of study 

01:12:46.900 --> 01:12:54.800
designs can be included and you get as kind of overall rating that usually looks like this. So this 

01:12:54.800 --> 01:12:56.650
is the score that this 

NOTE
Treffsikkerhet: 86% (HØY)

01:12:56.650 --> 01:13:05.400
he had 18 out of 22 and that makes 82%. And the 

01:13:05.400 --> 01:13:12.700
reason why this is different, these numbers here, is because for instance, the RCT, maybe 

01:13:12.700 --> 01:13:18.300
... and therefore he could have higher more items were applicable. So if the 

01:13:18.300 --> 01:13:23.800
items are not applicable, you can't score it. Therefore, you make it into a percentage score.  

01:13:23.800 --> 01:13:26.900
So overall these articles in a review,

NOTE
Treffsikkerhet: 86% (HØY)

01:13:26.900 --> 01:13:33.600
get a Kmet score and that means we translate that in wording here, we say 

01:13:33.600 --> 01:13:39.900
anything above 80% is good. So these two guys are pretty good. This one is strong because he's 

01:13:39.900 --> 01:13:48.800
between sixty and seventy nine. And this one is 54% only and that is adequate. Usually if they score poor 

01:13:48.800 --> 01:13:54.700
they're out because that's really a dodgy paper. I don't want to include that one. 

01:13:54.700 --> 01:13:57.600
I ignore their results. 

NOTE
Treffsikkerhet: 88% (HØY)

01:13:58.500 --> 01:14:05.300
In short, if you want to know more about critical appraisal tools or cats, you can go to this 

01:14:05.300 --> 01:14:11.300
website. You can see all these different things, for instance this is the middle bit. 

01:14:11.300 --> 01:14:18.500
If you enlarge that, then you can see here there are all these reporting guidelines per main study 

01:14:18.500 --> 01:14:26.000
type. So for randomized trials, observational studies Etc. All these examples behind that there 

01:14:26.000 --> 01:14:26.750
are over 

NOTE
Treffsikkerhet: 90% (HØY)

01:14:26.750 --> 01:14:31.900
400 different cats that they have got there. So you don't want to see all of them. That's why I just 

01:14:31.900 --> 01:14:41.400
gave you examples but please go to this website if you want to know more about cats. Now this 

01:14:41.400 --> 01:14:48.300
website also gives you kind of a guideline. So that is this one. It's always 

01:14:48.300 --> 01:14:53.100
overwhelming I think at the beginning, but this is where they start. So you start on the topic and 

01:14:53.100 --> 01:14:56.850
say, okay. Is it on Research? No, they go there. is it on Research on

NOTE
Treffsikkerhet: 86% (HØY)

01:14:56.850 --> 01:15:03.700
humans, you go down. Did you have quantitative data? Yes, you got. So a little bit like that you 

01:15:03.700 --> 01:15:10.800
end up in a certain place. And they say for instance here, is it a review? And they refer 

01:15:10.800 --> 01:15:17.700
you to Prisma. If it is a randomized controlled trial, it refers you to Consort or maybe Strop 

01:15:17.700 --> 01:15:26.650
or maybe the Rob 2, that was not yet here yet. So it gives you some help to which one to select. 

NOTE
Treffsikkerhet: 80% (HØY)

01:15:26.650 --> 01:15:36.400
But what you need to remember, if it is poor study quality, delete it. You can 

01:15:36.400 --> 01:15:42.600
report on it but what you say is so you include these articles for instance in a review. You must 

01:15:42.600 --> 01:15:48.700
include them. You say these are retrieved, then you do your critical appraisal tool and you say, 

01:15:48.700 --> 01:15:55.800
well actually these articles are so poor. The methods is so poor, you've report on that, but from that 

01:15:55.800 --> 01:15:57.200
moment on you just report others,

NOTE
Treffsikkerhet: 82% (HØY)

01:15:57.200 --> 01:16:02.900
but you do not give any further details from these articles. You just say I've done my Cats, 

01:16:02.900 --> 01:16:09.700
this is the outcome. These are too poor to include anywhere further in my studies,

01:16:09.700 --> 01:16:19.700
I'm going to ignore them. Okay, so that was in short bias and confounding, study designs at a glance and 

01:16:19.700 --> 01:16:26.500
Cats critical appraisal tools. Are there any questions at this moment? 

NOTE
Treffsikkerhet: 78% (HØY)

01:16:26.500 --> 01:16:35.900
I'm going to quit. I'm going to stop recording here. I think you don't want to have all the chats now.