|
Planning Analysis: |
Overview of Topics
Why conduct survey research?
To gather primary data that:
Survey Procedures
Select the most appropriate survey method
- Decided in conjunction with client
- Balance of accuracy and cost
Three general survey approaches:
- List resources
- Make budget estimate
- Develop timeline
- Telephone
- Personal interview
Table 1. Comparison of survey administration methods
CATEGORY |
TELEPHONE |
PERSONAL INTERVIEW` |
|
| Data | Best for statistical data | Poor for statistical data | Best for attitudinal data |
| Sampling Methods | Requires list of universe and addresses of all target respondents | Requires list of special populations. Can use random or added-digit dialing for general population. | Can use cluster sampling for general population |
| Response Rate | Usually 35% - 75% | About 95% in general population survey. | Dropping; now about 65% in general population survey. |
| Sampling Bias in General Population Survey | Difficult to determine. Extreme views over-represented; less educated underrepresented. | Usually low. About 96% of U.S. households have telephones; random dialing can reach unlisted numbers. | Single and poor persons and night shift workers underrepresented; women and non-working or retired persons overrepresented. |
| Questionnaire | No more than 12 pages. May ask moderately difficult questions, but not good for open-ended questions. | Standard interview length is 20 minutes. Should ask only very simple questions; no visual aids possible. Respondents answer most questions. | Standard interview length is 20-30 minutes. May ask complex questions. Respondents answer most survey questions. |
| Accuracy of Data | Respondents more willing to give embarrassing answers, but may misinterpret some questions. Allows for thought before response. No interviewer bias. | Respondents reluctant to give embarrassing answers. Possible interviewer bias. | Respondents reluctant to give embarrassing answers. Greatest possibility of interviewer bias. |
| Personnel and Supervision | Some workers required after development. | Some moderately trained interviewers. More difficult supervision, non-routine tasks performed at a single location. | Many trained interviewers. Difficult supervision, dispersed personnel performing complex tasks. |
| Implementation | Takes longest - several months | Fastest - a few days | Intermediate |
| Cost per Interview | Lowest | Intermediate | Highest |
| Advantages | Inexpensive. Small staff. Polls persons beyond reach of other methods. No interviewer bias. Respondents have more time for responses. | Quick, inexpensive. Easy to train interviewers. | Reaches unlisted populations. Can ask complex questions and probe vague answers. |
| Disadvantages | Difficult to determine over- or underrepresentation. Respondents may misinterpret or omit some questions. Awkward format for sensitive questions. Difficult to obtain up-to-date lists. | Respondents may modify answers because of antagonism toward or wariness of telephone interviews. Replies usually short. difficult to compile demographics. | High cost, complex organization. Extensive training, supervision of personnel. Greatest possibility that interviewer will bias results. |
Consider mail when:
Draw a sample
Why sample?
Why sampling works:
For each sampling frame:
What is the availability, cost, and accuracy?
Sampling pitfalls
Two types of sample
surveyors tend to pick someone like themselves
convenience surveys (e.g.., in supermarkets, at tourist sites)
Unbiased, since everyone has equal chance of being selected
Sample can only be as good as the list from which it was drawn
Types of random surveys:
- simple random - select random number to start, every nth thereafter
- stratified sampling - divide population into subpopulations, then every nth
- cluster sampling - survey all units in a stratified area
Sampling frame - the list you draw your sample from (who is missed?)
Potential error:
- Sampling error - difference in distribution of characteristics between sample and population as a whole (this is an estimate, since we can’t really measure whole population)
sampling error = + sqrt (variance in gen’l population on a topic/sample size)
where variance is computed by (P*Q)/.5*.5
in general terms, sampling error decreases as size of sample increases
+ 10 = (1/100)
+ 5 = (1/400)
+ 2.5 = (1/1600)
Level of confidence - sampling error is only half of our concern for accuracy
- Sampling error indicates how close our sample is to the population
- But how confident are we that the true value really does lie within that range?
- Accepted confidence level is 95% - nothing sacred about this number, just most commonly used
Sampling error and confidence level work together
Other sources of error:
- Frame error
Researcher should make every effort to minimize before survey implementation
- Non-response bias
Who are non-responders?
Difficult to measure - can be corrected for after implementation
Table 4. Simple Random Sample Size for Several Degrees of Accuracy
Confidence levels
Sampling error 95%
99%
1%
9,604
16,587
2%
2,401
4,147
3%
1,067
1,843
4%
600
1,037
5%
384
663
6%
267
461
7%
196
339
Source: Survey Research, Backstrom and Hursh-Cesar.
Table 3. Sample Size for Specified
Confidence Limits and Accuracy
(95% confidence interval)
Population |
||||||
Size |
+ 1% |
+2% |
+3% |
+4% |
+5% |
+10% |
500 |
b |
b |
b |
b |
222 |
83 |
1,000 |
b |
b |
b |
385 |
286 |
91 |
1,500 |
b |
b |
638 |
441 |
316 |
94 |
2,000 |
b |
b |
714 |
476 |
333 |
95 |
2,500 |
b |
1,250 |
769 |
500 |
345 |
96 |
3,000 |
b |
1,364 |
811 |
517 |
353 |
97 |
3,500 |
b |
1,458 |
843 |
530 |
359 |
97 |
4,000 |
b |
1,538 |
870 |
541 |
364 |
98 |
4,500 |
b |
1,607 |
891 |
549 |
367 |
98 |
5,000 |
b |
1,667 |
909 |
556 |
370 |
98 |
6,000 |
b |
1,765 |
938 |
566 |
375 |
98 |
7,000 |
b |
1,842 |
959 |
574 |
378 |
99 |
8,000 |
b |
1,905 |
976 |
580 |
381 |
99 |
9,000 |
b |
1,957 |
980 |
584 |
383 |
99 |
10,000 |
5,000 |
2,000 |
1,000 |
588 |
385 |
99 |
15,000 |
6,000 |
2,143 |
1,034 |
600 |
390 |
99 |
20,000 |
6,667 |
2,222 |
1,053 |
606 |
392 |
100 |
25,000 |
7,143 |
2,273 |
1,064 |
610 |
394 |
100 |
50,000 |
8,333 |
2,381 |
1,087 |
617 |
397 |
100 |
100,000 |
9,091 |
2,439 |
1,099 |
621 |
398 |
100 |
--- |
10,000 |
2,500 |
1,111 |
625 |
400 |
100 |
b = Those cases where 50% of the population in the sample will give more than the required accuracy
Source: Elementary Sampling Theory, Yamane.
Determining Sample Size
Factors indicating a large sample size
- The decisions to be based on the survey data have serious or costly consequences
- The sponsors demand a high level of confidence
- The sample population has a high level of variance
- The sample will be divided into small subsamples
- Project costs and timing vary only slightly with increases in sample size
- Time and resources are readily available
Factors indicating a small sample size
[Home | Syllabus | Schedule | Assignments | Project | Links]
This page maintained by Bob Parker,
©2002
September 30, 2002