-- Return to research page
-- Return to first page
Data Collection Techniques
Key
I -- Individual level N -- Needs Based
G -- Group level E -- Evaluation Based
O -- Organizational level C -- Community level
Mailed Devices
Mailed Questionnaire (N,E)
A mailed survey form used to obtain a broad analysis of some social phenomenon or problem (I/G/0).
Uses:
1. For reaching a wide geographic distribution of people.
2. For reaching a relatively homogeneous, fairly well-educated group.
3. For understanding some current situations, attitudes, and/or interests.
4. To determine factual material.
5. For making a survey of employee needs, problems, or interests.
Variations:
1. Delphi Technique.
2. Q-sort or card sort.
3. Picture sort.
Limitations:
1. The reliability of the results can be quite low at times.
2. The rate of return is frequently quite low.
3. Any open-ended responses or added comments may be difficult to interpret.
References:
Byrn, D. (Ed.). (1973). Evaluation extension. Topeka, KS: Ives Publications.
Oppenheim, A. M. (1966). Questionnaire design and attitude measurement. New York: Basic Books.
Sudman, S., & Bradburn, N. M. (1983). Asking questions: A practical guide to questionnaire design. San Francisco: Jossey-Bass.
Also see the Delbecq citation under the nominal group technique.
Checklist/Diagnostic Form (N,E)
Forms given to respondents individually or in groups where answers are checked on a list of statements (I/G).
Uses:
1. For groups of people in a meeting.
2. For individuals randomly selected from a group meeting who will complete them at home.
3. To collect evidence of progress made or practices adopted.
4. To assess a perception of individual need or interest on a topic.
Variation:
A group discussion of diagnostic form results can be carried out for purposes of further clarification, building consensus, determining new needs, and providing input for further program planning.
Limitations:
1. Can be difficult to interpret.
2. Difficult to obtain a good list of names.
References:
Knowles, M. S. (1980). The modern practice of adult education (2nd Edition). Chicago: Follet Publishing Company.
Interest or Attitude Inventory (N)
A device for finding out in what participants or potential participants are interested (I/G).
Uses:
1. Study of continuing education interests and interest areas.
2. Study of attitudes toward learning.
3. Study of attitudes on a particular subject.
Community Survey (N)
An analysis of various aspects of behavior and social interaction within a given community (C).
Uses:
1. To examine intergroup relations.
2. To study the physical aspects of communities.
3. To obtain an historical perspective relative to a community.
4. To examine population mobility.
5. To examine technological changes.
6. To examine changes in status and values.
Variations:
1. Can be combined with a larger community study effort.
2. Public opinion surveys or polls.
Verbal Devices
Personal Interview (N,E)
The collection of data through direct verbal interaction between individuals, usually formal in nature - the data collection takes place face to face or via a phone (I/G).
Uses:
1. For obtaining specific facts and opinions.
2. To measure attitudes and interests.
3. For an understanding of current situations.
4. When a high percentage of participation is needed.
References:
Bingham, W. V. D., & Moore, B. V. (1959). How to interview (4th Edition). New York: Harper & Row.
Bradburn, N. M., & Sudman, S. (1979). Interview method and questionnaire design: Response effects to threatening questions in survey research. San Francisco: Jossey-Bass publishers.
Cannel, C. F., Marquis, K. H., & Laurent, A. (1977). A summary of studies of interviewing methodology (Vital and health statistics, series 2, No. 69). Rockville, MD: U.S. National Center for Health Statistics.
Hyman, H. (1970). Interviewing in social research. Chicago: University of Chicago Press.
Team Interview (N)
Collection of data through direct verbal interaction between two interviewers (can be more) and (usually) one respondent (I).
Uses:
1. When time is a factor.
2. When interviewing individuals with high intelligence, much experience, and/or with an extensive educational background.
3. In exploratory studies.
Informal Interview (N,E)
An unstructured and unstandardized method of obtaining answers to various questions and gaining information on various topics (I/G).
Uses:
1. In beginning discussion on a topic.
2. For small group discussion.
3. For gathering qualitative information.
4. In obtaining insight on a problem or program's progress.
Variation:
A probing interview with prompt sheets or cue cards combines both the informal and the more structured interview format.
References:
Bogdan, R. C., & Biklen, S. K. (1982). Qualitative research for education. Boston: Allyn and Bacon, Inc.
Merriam, S. B., & Simpson, E. L. (1984). A guide to research for educators and trainers of adults. Malabar, FL: Krieger Publishing.
Tough, A. (1979). Adults' learning projects. Austin, TX: Learning concepts.
Telephone Survey or Interview (N,E)
A method of collecting information quickly and relatively inexpensively (I/G).
Uses:
1. Where good rapport with respondents has been or can be established.
2. For gathering factual information.
3. For gathering opinions, suggestions, and ideas.
4. For obtaining information about feelings and attitudes.
Systematic or Structured Devices
Critical Incidents Technique (N)
An interview with a supervisor, judge, or someone knowledgeable about an individual to determine specific behavior patterns that are considered critical to the skills or areas of behavior being studied; sometimes referred to as job analysis or task analysis (I).
Uses:
1. In studies of leadership ability.
2. For determining qualifications of individuals for certain jobs or duties.
3. In studies of individual behavior or on the job behavior.
4. In efforts to examine education or training need in relation to job performance.
Variation:
The critical incidents technique also can be used as a research tool or to provide feedback to an individual. For example, the technique has been used to determine critical teaching incidents or to provide a mirror for the improvement of instruction.
References:
Argyris, C., Putnam, R., & Smith, D. L. (1990). Action science. San Francisco: Jossey-Bass.
Brookfield, S. D. (1989). Developing critical thinkers. San Francisco: Jossey-Bass.
Merriam, S. B. (1991). Case study research in education. San Francisco: Jossey-Bass.
Rigors, P. (1971). Case methods in human relations: The incident process. New York: McGraw-Hill.
Ryan, D. G. (1960). Characteristics of teachers. Washington, DC: American Council on Education.
Advisory Council Input (N,E)
Can be intuitive, experiential, or data bases (see the discussion of advisory councils earlier in this chapter) (C/O).
Uses:
1. To obtain advice, insight, or factual information from a group of people knowledgeable about an area of organization.
2. To evaluate ongoing or completed educational programs.
3. Public hearings.
4. Town meetings.
5. Neighborhood meeting or block organizations/clubs.
Panel Survey (N,E)
The interview and study of a selected sample of respondents at two different times: Panels are picked and know of the task ahead of when the data are actually collected (C/O).
Uses:
1. To study changes in behavior.
2. To study changes in feelings and attitudes.
3. To study needs and interests.
4. To evaluate programs or materials.
Variation:
A survey form also could be mailed out before and after some experience; however, the results may not have the reliability and validity of an actual interview.
Limitations:
1. If panel members drop out between the two contact times the results can be affected.
2. A low return rate on any mailed out forms will affect the results.
Test, Diagnostic Tools, Pretests (N,E)
Group or individual completion of a device designed to test or measure some aspect of behavior or knowledge (I/G).
Uses:
1. To determine needs through an assessment of deficiencies.
2. To measure performance/status on some task, attribute, or attitude.
Variation:
A group discussion of test results can be carried out for purposes of further clarification, building consensus, or elaboration.
Limitations:
1. There are potential problems with validity, standardization, and measurement.
2. Frequently, this technique will need to be combined with other techniques.
References:
Hopkins, K. D., & Stanley, J. C. (1981). Educational and psychological measurement and evaluation (6th Edition). Englewood Cliffs, NJ: Prentice-Hall, Inc.
Tuckman, B. W. (1978). Conducting educational research (2nd Edition). New York: Harcourt, Brace, Jovanovich, Inc.
Systematic Study of Available Records (N)
An analysis of available records on a particular subject or need area (I/C).
Uses:
1. When an interview or questionnaire procedure is impossible.
2. For information on the past.
3. For use in a historical study of needs.
4. When a case study technique is desired.
Variation:
An analysis of census records to determine demographic changes, the concentration of specific characteristics (such as people with lower levels of education by census tract), and the changing work force.
Supervisory Ratings (N,E)
Ratings of an individual made by someone in a supervisory capacity (also known as job analysis performance appraisal, performance review, supervisory) (I/C/O).
Uses:
1. To analyze individual behavior, performance, and training need.
2. To determine where performance gaps can be interpreted as educational need.
References:
Mager, R., & Piper, P. (1970). Analyzing performance problems. Belmont, CA: Fearon Publishers.
Also see the references cited for the critical incidents technique.
Content Analysis (N,E)
The objective, systematic, and quantitative description of the manifest content of communication on a particular subject wither directly or indirectly (I/G).
Uses:
1. For the analysis of propaganda.
2. To examine the treatment of a particular subject in books, media, etc.
3. For the analysis of readability of various materials.
4. For the development of need or objective statements.
5. In historical studies.
References:
Holsti, O. R. (1968). Content analysis. In B. Lindsey & E. Aronson (Eds). The handbook of social psychology (2nd Edition). Reading, MA: Addison-Wesley.
Kerlinger, F. N. (1973). Foundations of behavioral research (2nd. Edition). New York: Holt, Rinehart, & Winston.
Anecdotal Records (N,E)
Observations and descriptions of behaviors deemed typical of an individual (also known as skill inventories or task analysis records) (I).
Uses:
1. To study human behavior.
2. To determine individual performance problems.
Peer Evaluating (N,E)
The evaluation of behavior by an individual's peers (usually voluntary); this technique initially was used for evaluation purposes but now is used in business and industry as a feedback device for ascertaining employee training needs (also known as peer review) (I/G).
Uses:
1. To assess some aspect of human behavior.
2. To evaluate a person's job performance
Limitations:
1. The technique requires trained observers with considerable skill and sensitivity.
2. Peers may not always report/perceive information accurately.
Reference:
American Medical Association. Peer review manual. New York: American Medical Association, 1971.
Power Structure Analysis (N)
A determination of the manner in which individual power actors in a social system relate to each other (although this is not a standard needs assessment technique, it provides useful information to better understand needs, program planning strategies, blocking groups, etc.--also known as community leader analysis) (C).
Uses:
1. For community analysis efforts.
2. To understand formal organization leadership.
3. To assess leadership skills and experience.
Gaming or Group Interaction Devices
Gaming and Stimulation Device (N,E)
Role playing facilitated by some sort of a gaming board or tool; needs either personal observation or self-evaluation tied to it (I/G).
Uses:
1. To determine the participant's knowledge about some topic.
2. To facilitate interest in or practice with some topic.
References:
Horn, R., & Cleaves, A. (Eds.). (1980). The guide to simulation/games for education and training (4th Edition). Beverly Hills, CA: Sage Publications.
Simulation and Games. An international journal of theory development and research. Published quarterly by Sage Publications of Beverly Hills, CA.
Team or Group Problem Solving (N)
The attempt to solve a particular problem through team action (also known as task force analysis) (I/G).
Uses:
1. In studies of human interaction within group activities.
2. In studies of work groups.
3. When a group facilitator can be employed to assist with assessment efforts.
References:
Drucker, P. F. (1974). Management. New York: Harper & Row.
Brainstorming (N)
Group (usually small) members spontaneously generate a wide variety of ideas, interests, etc.; clarifying and follow-up techniques also are typically required (I/G).
Uses:
1. Where quick responses are desired.
2. When some initial ideas or categories of needs are required.
Limitations:
1. Responses obtained quickly or spontaneously may not always reflect reality.
2. Some people may not desire to participate in a brainstorming activity.
References:
Clark, C. H. (1958). Brainstorming. New York: Doubleday.
Clark, C. H. (1980). Idea management. New York: Amacon.
Nominal Groups (N)
Group decision making where all suggestions are recorded and ranked (I/G).
Uses:
1. When contributions need to be encouraged.
2. When synergistic results from group involvement or commitment are desired.
Reference:
Delbecq, A. (1975). Group techniques for program planning: A guide to nominal group and delphi process. Glenview, IL: Scott Foresman Company.
Outside Consultants
Consultants or Outside Experts (N,E)
A professional helper's advice (usually tied to assessing needs and evaluating programs) (I/C/G).
Uses:
1. When participant observations are appropriate.
2. When outside advice can be combined with evaluations.
Limitation:
Reliance solely on an outside expert for advice on some project or future activity may reveal only a partial picture of reality.
Systematic or Personal Observations (N,E)
Recommended practices observed, adoptions recorded, and recommendations made (I).
Uses:
1. In the study of practice adoption.
2. In the study of needed change in behavior.
-- Return to research page
-- Return to first page