The field of measurement and statistics provides important insight into subject matters that might otherwise be tough to dissect. As such, students who graduate from the Measurement and Statistics program at Florida State University’s College of Education go on to work at major corporations, government institutes, universities, research institutions, and education policy centers. At these companies and organizations, they use their knowledge and skills to examine, evaluate and, yes, measure concepts that initially would seem impossible to measure—concepts like the ability of individuals to think creatively, the resilience of employees, general attitudes, and more.
Dr. Russell Almond is an associate professor in the Measurement and Statistics program at FSU and has spent his career learning and teaching how to evaluate these complicated concepts. His primary area of research looks at ways to monitor students’ growth using both traditional means (like homework, tests, etc.) and newer, non-traditional methods, including simulations and games.
In this video, Almond defines measurement and statistics, talks about observing and measuring complicated topics, and some of the challenges facing the field.
Video transcription:
My name is Russell Almond. I'm here to talk to you about the measurement and statistics program, which is in the Department of Educational Psychology and Learning Systems in the College of Education at Florida State University.
So, you probably heard about statistics, but just to be safe, I'm going to define a statistic as some sort of function or operator that summarizes the data set. It might summarize it as a graph, or it might summarize it as a number, but we're making a summary of a data set, and the field of statistics is the study of such functions.
How about measurement? Right, when we think about measurement, we might think about pulling out a measuring tape and measuring something, right? So, measure is a function that reports how much of something there is, and the field of measurement is the study of measures.
But if it can't be seen like education, then how do we measure it? Well, we can't see the wind, but we can see evidence that the wind is there, and we can use that evidence to build an anemometer, an instrument that measures the wind. We can use that same general idea to measure learning or attitudes towards learning or anything else that like that that we want to measure.
So how do we do this? So, we start with what we want to measure, a claim. For example, the student has mastered the material that they're supposed to master in this course, and then we'll make observe some kind of data. For example, the student is successfully able to solve some kind of problem. Obviously, we want that problem to be linked to the claim, so this is a problem they could only solve if they use the skill that we hope they've mastered, and we also want to eliminate alternative explanations—reasons why they might not be able to solve the problem even if they have the skill, or why they might have solved the problem anyway even if they lack the skill.
So, this is a very important idea because this data we collect about students only becomes evidence when we connect it to a claim through a warrant. So how do we gather that evidence, right? There's the old tried and true multiple-choice test. There's a little more sophisticated tests, like essay tests or other things where you might draw a diagram or something like that. Now we have all kinds of interesting computer programs. We can do computer simulations, and we can make diagnostic tests—tests that aren't designed just to show how much we have, but to sort people into different piles right. The person there is sorting laundry, and he's sorting that laundry because he's going to do different things with that laundry. He's going to wash the dark colors in a different way than he's going to wash the light colors.
We don't have to measure just knowledge and skills. We can also measure attitudes and things like that, so we could measure self-efficacy or happiness. We know all those Likert scale kinds of questionnaires; we can talk about making those for various purposes. We can talk about measurements we make through observations, and some really exciting things, like the possibility of using facial recognition to understand how students are absorbing education.
There are a lot of hard-to-measure constructs that we're still working. How do you do good science and engineering practice? That man there is doing very careful observation but probably not very safe observation. Persistence. Right? Do you really have what it takes to climb that mountain? Flexibility. That's a very hard one, but the PISA (the Program International for Student Achievement) is studying that this year. Frustration tolerance, a key part of being able to learn. Problem-solving and teamwork.
So now let's talk a little bit about the kinds of research we're doing. One of the things that we're doing is we're looking at complex performances, so here's a level of a video game called Physics Playground, and we're trying to assess the kind of physics knowledge that the student has from what they're playing in that game. We're using some tools, like evidence-centered design and a new evidence identification tool that we're building, to try and do this stuff.
Another problem thing that we have is relationships among constructs. We might be looking at the relationship between vocabulary knowledge and comprehension, or maybe we're looking about how vocabulary knowledge changes across times. We have tools like structural equation models and Bayesian networks, which help us with that.
Putting stuff together across time is also very important. We have things like longitudinal models, hierarchical models and Markov processes that we're looking at there. We got a lot of studies in education out there, and making sense of that huge pile of papers is another problem. There's another whole field called meta-analysis that's devoted to that, providing support for ongoing testing programs. As a matter of fact, we're in Tallahassee, which is the state capital of Florida. Right down the street from us is the Florida Department of Education. We have very strong ties with the Florida Department of Education. Many of our students intern over there and help make sure that the Florida state assessment system is working properly, gaining a lot of valuable on-the-job experience.
So these are just a few of the things that our faculty and our graduate students are working on here in the measurement statistics program.
So where do our graduates go our graduates go? A lot of our graduates go into the testing industry, supporting various ongoing testing programs. There's a huge demand for those, all the state testing, but all kinds of other things as well. Some of them go into government policy institutes. There's a lot of places where we're interested in measuring how students are doing across time, and our graduates are well placed to take part in those policy discussions and data analyses. Supporting research in all kinds of other fields, right?
Education is not the only field that uses the kinds of measurement skills we teach here. We have students here in a number of different universities, many of them doing work that's not specifically education but might be mental health services. There's a lot of educational components in medicine these days.
So the last thing that we need to worry about and probably the most important thing that we study here is the concept of validity, which is, does this measure that I've designed this test that I've designed really measure what it's supposed to be measuring?
There's an old story about a man who is down on his hands and knees under a streetlight. Along comes a police officer and says, “What are you doing?”
He says, “I'm looking for my car keys.”
“Where did you lose them?” says the officer.
“Out there,” says the man, gesturing into the darkness.
“Then why are you looking here?” says the police officer.
“The light’s better,” says the man.
So, we want to make sure that when we're doing educational measurement that we're looking where we've lost the keys and not where the light’s best. We want to make sure that we really get measures that are measuring what we want. There's another problem here that’s called the measurement fallacy. So sometimes we confuse what's easy to measure with what we really want to measure. The U.S. News and World Report is very much like that. Now Florida State University and the measurement and statistics program fairs pretty well in those kinds of statistics, and they're looking at things that are important like how many students are graduating, but there are other things that they don't include, too, which is how much fun we have when we’re learning!
And so, there we are, the whole fun bunch, so if you want to find more about us, I urge you to come to the website here which is education.fsu.edu/measurement-and-statistics, or to talk to Mary Kate McKee who's the fine young lady there in the blue thing who will help you through the application process.
So thanks for listening and hope to hear from you real soon!
