Analysis of Multiple Response Questions

Multiple response questions are commonly used in a survey questionnaire in which participants could choose more than one answers. For example, students were asked to select the things they like the most about CFC (Caring for Cambodia) schools based on 8 choices: school meal program, beautiful campus, beautiful garden, clean water, toilet, good time with friends, computers, and teachers. In Stata, the analysis of this type is pretty easy and straightforward. First of all, as always, you need to check for the response of the variables being used by using "tab var" command or "codebook var" command.

Before we get into the analysis part, it is important to know how this set of variables is coded. It appears in the questionnaire as just one question though, looking like this:

When you coded this question, it becomes 8 different questions, looking like this in your Stata:

Each question is coded with a numerical value 1 if a respondent answers Yes and 0 if No. For example, if a respondent checks 5 of choices: teacher, food, toilet, computer, garden, then here how it looks like:

If a respondent did not choose any one of the choices, then you can just leave it blank. When you perform an analysis, you can ask Stata to just count all the responses with 1.

Stata does not come with multiple response analysis command which is represented by "mrtab", so we need to install it. The installing process was mentioned during ANOVA lecture when we tried to install "effectsize" command. To install "mrtab" command, type "findit mrtab" in your command box, and a small window will pop up for you to choose a package to install. After you installed it, you can now perform the analysis:

use http://dl.dropboxusercontent.com/u/60032040/mrtab.dta

mrtab cfcteacher- cfcfriend, include response (1) 

So what you see from the output above is that 62.75% of the participants chose good teachers as the thing they like the most about CFC schools. For easy reading of the results, we can ask Stata to sort descendingly for us. We can just add "sort des" after the last option:

mrtab cfcteacher- cfcfriend, include response (1) sort des

Now you can see that the largest number is on the top and the smallest on the bottom.

You can also ask Stata to break it down by school by adding "by (school) col" to the last option:

mrtab cfcteacher- cfcfriend, include response (1) sort des by (school) col

Now the results are broken down by three different schools. Based on the output above, students at Bakong High ranked "good teachers" as the thing they like most about CFC school (72.77%).

You can also request Stata to give us statistical test on chi-square on each individual question by the three schools, and also overall chi-square test of the whole model (all questions together). You can do so by adding "mtest" and "chi2":

mrtab cfcteacher- cfcfriend, include response (1) sort des by (school) col mtest chi2

Look at the last column (chi2/p*). There are two rows of values there, the top one is the chi2 value and the bottom one is the significant value (p). If p value is less than .05, then you can say that there is a significant difference of the response in different schools.

Again, it is always easier to put these values in a graph. Here how it looks like in bar graph:

Let's look at one more example with the same procedure of analysis, using this dataset that you used for your Practice Assignment earlier. Let's look at parental involvement variables (19 of them). The response to these variables range from 1-4. You want to see which involvement variables are the top choices reported by the students in terms of "often" and "most of the time." First of all, you will need to look at the code of any one of the parental involvement variables, since all of them (19) have the same response pattern. For example, I choose involvement item #1 (var: par_invol1):

codebook par_invol1

Now you know that the responses "often" and "most of the time" are 3 and 4. So that means that you want Stata to just include or count the response 3 and 4 in order to know which involvement items rank highest and lowest for "high involvement" category (e.g., 3 and 4 response).

use http://dl.dropboxusercontent.com/u/60032040/assignment1.dta

mrtab par_invol1 - par_invol19, include response (3 4) sort des

As you can see from the Stata output above, the item "My parents reminded me to study hard" ranks highest and "My parents attended meetings at my school" the lowest, for the responses "often" and "most of the time."

If you want to count those who reported just "most of the time" and see which involvement item ranks top, then you can just request Stata to include just "4". It looks like this:

When you count just "most of the time" response (as shown in the above output with number 4 circled in red), involvement item 19 ranks top. This is subjective based on what you want to look into.


Again, you can break it down by gender and you can see if there is a significant difference between male and female students responding to each of the involvement items, for the responses "often" and "most of the time."

mrtab par_invol1 - par_invol19, include response (3 4) sort des by(gender) col mtest

As you can see from the above output by gender, it seems that only involvement item #18 that shows significant difference between males' and females' report on the involvement. It shows that boys are more likely to report that their parents talked about person they admire to them than girls are, χ²(1) = 4.14, p<.05.

You can also make a table out of the above results for easy read. However, since there is no significant difference between gender, then I would not do it.


PRACTICE ON YOUR OWN

Use the question below to answer the following questions:

1. What subject that students like the most? and the least?
2. What subject that students like with the responses from 6-8?
3. Are there any gender differences of each subject (with the response from 6-8)?
4. Please describe your findings in APA format.

use http://dl.dropboxusercontent.com/u/60032040/mrtab.dta

Good luck!