Exercise 1: Questions of Validity
Below are 6 descriptions of research designs and the findings of the scholars who conducted the research. Using the tools from the Campbell and Stanley readings, critique these designs in a paragraph or two. How is the validity of the research threatened by the design? How would you improve the research design?
Please address any 4 of the following 6 research designs.
- Enid wants to investigate the effect of saliency of message on attitude change. From an old Ph.D. she finds a swell communication on the importance of physical sciences in a liberal education. Fortunately for her, Widget University conducts English classes separately for engineers and liberal arts majors. Within this limitation, however, the university has matched the classes carefully on age, sex composition, socioeconomic background, and College Entrance Board Scores (both verbal and mathematical ability as well as on scores in specific subjects). Enid checks on the dean's records and is happy to find that the classes have indeed been matched to the best possible extent. Enid then delivers the message to the engineers (the salient group) and to the liberal arts students (the non-salient group). The engineers show much more attitude change. Enid concludes that message saliency increases attitude change.
- A new police superintendent is hired to take over the scandal ridden police department of a large Midwestern city. A recent professor of police administration remodels the departmental organization with vigor. In particular, he finds that the workers in the central office are political appointments and generally slack administrators. Most are fired, and much expensive computer equipment is installed. He also appoints a research staff to evaluate the impact of his reorganization. The staff is no sooner hired than the first six months data are printed out on the new computer equipment. Crime is up over the rates reported by the previous administration. The difference is highly significant. The report of the research staff (written just before the evaluation outfit was disbanded and the machines shut down to conserve electricity) concludes that the administrative changes produced an increase in crime.
- Edgar is disgusted with artificial laboratory experiments on attitude change. At the advent of the "Cuban Crisis," he sees an opportunity to conduct a "true to life" experiment. A school principal has scheduled a talk on Cuba for the high school assembly. Since the auditorium is too small for the total population of the school, two assemblies are scheduled on consecutive Fridays. The principal would not allow Edgar to draw randomly from the school population, but Edgar was able to randomly assign classroom units to one assembly or the other. Since the students are continually subjected to tests in the early part of the year, Edgar is able to include both his pre-test and post-test in the ongoing test routines. Students are tested four days before and four days after their assembly. The speaker is a former member of congress who is presently registered as a public relations agent for anti-Castro groups in the U. S. For the first assembly, Edgar introduces the speaker as a former member of congress. For the other, the speaker is introduced as working in public relations for Cuban exiles. The speaker (who was not informed of the experiment) then delivers the same prepared anti-Castro speech to both assemblies. The pre-test-post-test gains for the two assemblies are not significantly different. Edgar concludes that — for this population of subjects, for these communication materials, and for this measure of communicator prestige — the prestige of the speaker has no effect on communication effectiveness.
- Esmeralda has been working with "permissiveness" as a variable in child rearing. Recently, she has developed a theory about the factors which lead parents to be more or less permissive; and she wants to test out a counseling program designed to increase permissive behavior in parents. She chooses the 10 of her sample which is the most extremely non-permissive for the experimental group. The control group is drawn randomly from the rest of the population. The experimental sample is then given the counseling. Pre-test-post-test differences indicate a greater increase in permissiveness for the experimental than the control group. Esmeralda concludes: "For the extreme subjects studied in this research, the counseling program increases permissiveness."
- Ernie, a test constructor, is interested in test performance on multiple-choice examinations as a function of number of multiple-choice alternatives. More specifically, his theory of mental organization holds that as the number of alternatives increase, the number of associations aroused is greater, and the likelihood of interference is greater. He is teaching three sections of elementary psychology, one meeting at 8:30, one at 11:30, and one at 2:30. For one week he gives all three sections exactly the same lectures over a single topic. On the following Monday he gives a common 30 question multiple-choice test to each section. The questions are exactly the same but for one section there are 3 alternatives, for another 5, and for another 7. All sections had all the time needed to complete the examination. The position of the correct alternative was always determined by chance. What's wrong with this design? How could it be made right?
- Recent advances in ultrasonic technology make it possible to destroy brain tissue in a minutely specified location within the brain without destroying parts of the brain through which a single "beam" of sound passes. This is accomplished by focusing several beams, coming from different sources, so they converge on the specified location. To do this, small holes must be made in the skull for each beam used. In this experiment, Emily examined the influence of destruction of a small segment in the midbrain which we will call X. A group of 100 rats was trained to press a bar in a Skinner box. These rats were divided into two groups matched on initial bar-pressing performance. The one group was then subjected to destruction of X and after one week was tested again in the Skinner box. The other group served as a control. The animals in this group "rested" and then were tested again in the Skinner box at a time after original learning that was exactly equivalent to the time of testing for the experimental group. Performance of the experimental group was significantly inferior to that of the control group. Emily concluded that the destruction of X (this destruction was confirmed by histological examination) caused a decrement in performance of the barpressing habit.
Exercise 2: Measurement
The purpose of this assignment is to measure a concept in political science. Please choose three of the following four topics. For each topic:
- Provide an answer to the question
- Explain how you came to your answer — describe your methodology and your data source in approximately one paragraph.
Topics
- Casualties of War: How many Iraqi civilians have died as a result of military conflict since the current war began on March 19, 2003?
- Democratization: How many countries in the world were democracies in 1912? How many in 2006?
- Education Spending: How much did the U.S. Federal Government spend on Education in 2005?
- Support for President Obama: What was President Obama's approval rating the week of October 5, 2009? What was it the week of October 4, 2010?
Note: All data must be collected from primary sources. Do not simply quote a figure from a newspaper article, magazine, or blog post. Such references will point you in the right direction, but find out where the reporter obtained her data. For example, reporters may quote figures from the World Health Organization. You should track down your data from these original sources (internet references are acceptable, as long as they are properly sourced to credible authorities).