Doing Oneway Repeated Measures ANOVAs

Qn1. Download the file websearch2.csv from the course materials. This file describes a study in which subjects were asked to find 100 distinct facts on the web using different search engines. The number of searches required and a subjective effort rating for each search engine were recorded. How many subjects took part in this experiment?

Qn2. To the nearest hundredth (two digits), what was the average number of searches required for the search engine that had the greatest average overall?

Qn3. Conduct an order effect test on Searches using a paired-samples t-test assuming equal variances. To the nearest ten-thousandths (four digits), what is the p-value from such a test? Hint: Use the reshapape2 library and the dcast function to create a wide-format table with columns for each level of Order.

Qn4. Conduct a paire-samples t-test, assuming equal variances, on searches by Engine.To the nearest hundredths (two digits), what is the absolute value of the t statistic for sich a test? Hint: use the reshape2 library and the dcast function to create a wide-format table with columns for each elvel of engine.

Qn5. Conduct a nonparametric Wilcoxon signed-rank test on the Effort Likert-type ratings. Calculate an exact p-value. To the nearest ten-thousandth (four digits), what is the p-value from such a test? Hint: Use the coin library and its wilcoxsign_test function with distribution = “exact”

Qn6. Download the file websearch3.csv from the course materials. This file describes a study just like the one from websearch2.csv, except that now the three search engines were used instead of two. Once again, the number of searches required and a subjective effort rating for each search engine were recorded. How many subjects took part in this new experiment?

Qn7. To the nearest hundredth (two digits), what was the average number of searches required for the search engine that had the greatest average overall?

Qn8. Conduct a repeated measures ANOVA to determine if there was an order effect on searches. First determine whether there is a violation of sphericity. To the nearest ten-thousandths (four digits), what is the value of Mauchly’s W criterion? Hint: use the ez library and its ezANOVA function passing within-Order, among other things, to test for order effects.

Qn9. Interpret the result of Mauchly’s test of sphericity, and then interpret the appropriate repeated measures ANOVA result. To the nearest ten-thousandth (four digits), what is thep-value from the appropriate F-test?

Qn10, Conduct a repeated measures ANOVA on searches by Engine. First determine whether there is a violation of sphericity. To the nearest ten-thousandth (four digits), what is the value of Mauchly’s W Criterion? Hint: use the ez library and its ez ANOVA function passing within-Engine, among other things, to test a significant main effect.

Qn11. Interpret the result of Mauchly’s test of sphericity, and then interpret the appropriate repeated measures ANOVA result. To the nearest ten-thousandth (four digits), what is the p-value from the appropriate F-test?

Qn12. Strictly speaking, given the result of the repeated measures ANOVA examining searches by Engine, are post hoc pairwise comparisons among levels of Engine Warranted?

-yes
-No

Qn13. Whatever your previous answer, proceed to do post hoc pairwise comparisons. Conduct manual pairwise comparisons of searches among levels of engine using paired-samples t-tests, assuming equal variances and using Holm’s sequential Bonferroni procedure to correct for multiple comparisons. To the nearest ten-thousandths (four digits), what is the smallest corrected p-value resulting from this set of tests? Hint: use the reshape2 library and dcast function to create wide-format table.

Qn14. Conduct a nonparametric Friedman test in the Effort Likert-type ratings. Calculate an asymptomatic p-value. To the nearest ten-thousandth (four digits), what is the chi-square statistic from such a test? Hint: Use the coin library and the friedman_test function.

Qn15. Strictly speaking, given the result of the Friedman test examining Effort by Engine, are post hoc pairwise comparisons among levels of engine warranted?
-Yes

• No

Qn16. Whenever your previous answer, proceed to do post hoc pairwise comparisons. Conduct manual pairwise comparisons of Effort among levels of Engine using Wilcoxon signed-rank tests, Using Holm’s sequential Bonferroni procedure to correct for multiple comparisons. To the nearest ten-thousandth (four digits), what is the smallest corrected p-value resulting from this set of tests? Hint: Use the reshape2 library and dcast function to create wide-format table. Then use the wilcox.test function with paired=TRUE (and to avoid warnings, exact = FALSE).

,

Understanding oneway repeated Measures Designs

Qn1. What primarily distinguishes a oneway repeated measures ANOVA from a one-way ANOVA?

- The presence of multiple factors
- The presence of a between-subjects factor.
- The presence of a within-subjects factors.
- None of the above

Qn2. All else being equal, which of the following is a reason to use a within-subjects factor instead of a between-subjects factor?

- The data is more reliable
- The data exhibits less variance
- The factors are easier to analyze
- The exposure to confounds is less
- Less time from each subject is required

Qn3. In a repeated measures experiment, why should we encode an Order factor and test whether it is statistically significant? (Mark all that apply)

- To examine whether the presentation order of conditions exerts a statistically significant effect on the response.
- To examine whether any counterbalancing strategies we may have used were effective 
- To examine whether confounds may have affected our results
- To examine whether our factors cause changes in our response
- To examine whether out experiment discovered any differences

Qn4. How many subjects would be needed to fully counterbalance a repeated measures factor with four levels?

 - 4,8,16,24,32

Qn5. For an even number of conditions, a balanced Latin Square contains more sequences than a Latin Square.

- True
- False

Qn6. For a within-subjects factor of five levels, a balanced Latin Square would distribute which of the following number of subjects evenly across all sequences?

5, 15, 20,25,35

Qn7. Which is the key property of a long-format data table?

- Each row contains only one data point per response for a given subject.
- Each row contains all of the data points per response for a given subject.
- Each row contains all of the dependent variables for a given subject.
- Multiple columns together encode all levels of a single factor.
 - Multiple columns together encode all measures for a given subject

Qn8. Which is not a reason why Likert-type responses often do not satisfy the assumptions of ANOVA for parametric analyses.

- Despite having numbers on a scale, the response is not actually numeric.
- Responses may violate normality
- The response distribution cannot be calculated
- The response is ordinal
- The response is bound to within, say, a 5- or 7-point scale.

Qn9. When is the Greenhouse-Geisser Correction necessary?

- When a within-subjects factor of 2+ levels violates sphericity
- When a within-subjects factor of 2+ levels exhibits sphericity
- When a within-subjects factor of 3+ levels violates sphericity
- When a within-subjects factor of 3+ levels exhibits sphericity
- None of the above

Qn10. If an omnibus Friedman test is non-significant, post hoc pairwise comparisons should be carried out with Wilcoxon signed-rank tests

-True
-False

Doing Oneway ANOVAS

Qn1. Download the file alphabets.csv from the course materials. This file describes a study in which people used a pen-based stroke alphabets to enter a set of textphases. How many different stroke alphabets are being compared?
Qn2. To the nearest hundredth (two digits), what was the average text entry speed in words per minute (WPM) of the EdgeWrite alphabet?

Qn3. Conduct Shapiro-Wilk normality tests on the WPM response for each Alphabet. Which of the following, if any, violate the normality test? (Mark all that apply.)

-Unistrokes
-Graffiti
-EdgeWrite
-None of the above

Qn4. Conduct a Shapiro-Wilk normality test on the residuals of a WPM by Alphabet model. To the nearest ten-thousandth (four digits), what is the p-value from such a test? Hint: Fit a model with aov and then run Shapiro.test on the model residuals.

Qn5. Conduct a Brown-Forsythe homoscedasticity test on WPM by Alphabet. To the nearest then-thousandth (four digits), what is the p-value from such a test? Hint: Use the car library and its level Test function with center=median

Qn6. Conduct a oneway ANOVA on WPM by Alphabet. To the neares hundredth (two digits), what is the F statistic from such a test?
Qn7. Perform simultaneous pairwise comparisons among levels of Alphabet sing the Tukey approach. Adjust for multiple comparisons using Holm’s sequential Bonferroni procedure. To the nearest ten-thousandth (four digits), what is the corrected p-value for the comparison of Unistrokes to graffiti? Hint: use the multcomp library and its mcp function called form within its glht function.
Qn8. According to the results of the simultaneous pairwise comparisons, which of the following levels of Alphabet are significantly different in terms of WPM? Mark all that apply.)

-Unistrokes vs. graffiti
- Unistrokes vs, EdgeWrite
- Graffiti vs. EdgeWrite
- None of the above

Qn9. Conduct a Kruskal-Wallis test on WPM by Alphabet. To the nearest ten-thousandth (four digits), what is the p-value from such a test? Hint: use the coin library and its Kruskal-test function with distribution = “asymptotic”

Qn10. Conduct nonparametric post hoc pairwise comparisons of WPM among all levels of Alphabet manually using separate Mann-Whitnet U tests. Adjust the p-values using Holm’s sequential Bonferroni procedure, To the nearest ten-thousandth (four digits), what is the corrected p-value for Unistrokes vs. graffiti? Hint: The coin library’s Wilcoc_test only takes a model formular specification. For this, you need wilcox.test with paired = FALSE ((and to avoid warnings= FALSE))

Understanding Oneway Designs

Qn1. The issue that requires an experimenter to use a oneway ANOVA instead of a t-test is when there are more than two response categories available.

-True
-False

Qn2. Which of the following is the equivalent nonparametric analysis to a parametric oneway ANOVA?

-F-test
-t-test
-Kruskal-Wallis test
-Mann-Whitney U test
None of the above

Qn3. Typically, an ANOVA uses which distribution and test statistic?

-F
-t
-Chi-square
-Kolmogorov-Smirnov
-Poisson

Qn4. If an omnibus oneway ANOVA for a three-level factor is statistically significant, it does not mean that post hoc pairwise comparisons are allowed.

-True
-False

Qn5. Which of the following is the most proper way to report an F-test result?

-F(14) = 9.07, p = 0.009
-F(14) = 9.06, p < 0.01
-F(1,14)=  9.09, p = 0.009
-F(1,14) = 9.06, p < .01

-None of the above

Qn6. A oneway ANOVA is characterized by which experimental design?

-An Experiment with a single between-subject factor of exactly two levels.
-An experiment with a single between-subjects factor of two to more levels.
-An experiment with a single within-subjects factor of exactly two levels.
 -An experiment with a single within-subjects factor of two or more levels.
-None of the above

Understanding Validity

1. What is experimental control?
   a. Ensuring that noting happens in an experiment without the experimenter knowing about it.
   b. Ensuring that every subject gets to experience every condition in the experiment.
   c. Ensuring that measures are made correctly and precisely.
   d. Ensuring that systematic differences in observed responses can be attributed to systematic changes in manipulated factors.
   e. None of the above
2. Which of the following are examples of potential confounds? (Mark all that apply.)
   a. In website A/B test, every visitor was different from every other visitor.
   b. In a website A/B test, males all saw website “A” and females saw Website “B”.
   c. In a website A/B test, every visitor hitting the site before noon saw website “A”, while every visitor hitting the site after noon saw website “B”
   d. In a website A/B test, site “A” was different from site “B”
   e. In a Website A/B test, sites “A” and “B” were measured a second time with a new batch of visitors, just to be sure.
3. Ecological validity and experimental control cannot bot be maximized.
   a. True
   b. False
4. Which of the following was not an option discussed in lecture for handling a potential confound?
   a. Manipulate it, systematically vary it to see if doing so causes systematic changes in the response.
   b. Control for it – ensure that its effects are spread evenly across all subjects.
   c. Measure it – at least record its value so it cant be later examined for possibly having had an effect
   d. Hide it – don’t let subjects encounter it in the first place
   e. All of the above are options
5. Which of the following is not another term for the response in an experiment?
   a. Dependent variable
   b. Measure
   c. Outcome
   d. Y
   e. Factor
6. Which of the following are assumptions of ANOVA? (Mark all that apply.)
   a. Reliability of residuals
   b. Normality
   c. Homoscedasticity
   d. Independence
   e. Homogeneity of variance
7. Which of the following was not a common data distribution reviewed in lecture?
   a. Normal
   b. Lognormal
   c. Bimodal
   d. Exponential
   e. Gamma
   f. Poisson
   g. Binomial
   h. Multinomial
8. For what kind of experiment would a multinomial distribution be relevant?
   a. For an experimental in which the response is categorical with more than two categories
   b. For an experiment in which the response is bimodal
   c. For an experiment in which the response is scalar.
   d. For an experiment in which the response is Poisson
   e. None of the above
9. Most precisely, parametric analyses differ from nonparametric analyses in what way?
   a. Parametric analyses operate on ranks.
   b. Parametric analyses make assumptions about the spread of data.
   c. Parametric analyses make assumptions about the distribution of the response within the population
   d. Parametric analyses are easier to use.
   e. None of above
10. Typically, an advantage of parametric analyses over nonparametric analyses is statistical power, i.e the ability to detect differences.
    a. True
    b. False
11. Nonparametric analyses must meet the three assumptions of ANOVA
    a. True
    b. False
12. Nonparametric analyses typically operate on ranks.
    a. True
    b. False