Consideration in preparing test items

One essential step in planning a test is to decide why you are giving the test. (The word "test" is used although we are using it in a broad sense that includes performance assessments as well as traditional paper and pencil tests.)

Airasian (1994) lists six decisions usually made by the classroom teacher in the test development process:

1. What to test,

2. How much emphasis to give to various objectives,

3. What type of assessment (or type of questions) to use,

4. How much time to allocate for the assessment,

5. How to prepare the students, and

6. Whether to use the test from the textbook publisher or to create your own. Other decisions, such as whether to use a separate answer sheet, arise later.

Overall consideration when planning classroom tests:

1. Define the purpose at this time.
2. Specify the performance and processes to be observed and tested
3. Select the type of test items or the methods to be used to observe and to test the performance.
4. Develop the initial drafts of the test exercises.

What should be specified ahead of time?

1. Formats of test items to be used

1. Choice formats-objective items true-false, multiple-choice, matching exercise
2. Short answer/completion format
3. Essay format
4. Performance observation formats-checklists, rating scales, sign and category system
5. Interview, in-depth observation
6. Long-term activity formats-project, extended written assignment, laboratory exercises

2. Number of items of each format:

The amount of time available for testing. Tests with more items are more reliable that shorter tests.

3. Types of performance to be observed:

Develop a test blueprint to specify the various levels of performance to be observed

4. Number of performance within each type:

The number of objective within each taxonomic category is delineated.

5. Content to be covered by the test.

The test blue print can be the vehicle for delineating the content or topics the test will cover.

6. If each objective has been taught and studied conscientiously, then the test should be of appropriate difficulty.

Criteria for judging testing procedures

1. Validity

The test determines the extent to which each pupil has attained the important objectives.

2. Reliability

1. The consistency with which a given testing procedure reports a pupil’s performance

1. Longer tests are more reliable than short tests.
2. Essay questions are less reliable than objective procedure

2. Possible sources of unreliability:

1. Fluctuations of judgments of the quality of a pupil’s work

3. Objectivity

A test procedure is said to be objective if two or more observers of pupils performance can be agree on the report of the performance.

4. Comprehensiveness

1. The extents to which a test can be a representative sample of behaviors from the objectives of instruction contribute much to the success of a test.
2. Taxonomy with a content outline and a test blueprint help to ensure that the behaviors are tested.

5. Ease of construction and scoring

How easy it will be to devise the test task and to score them

6. Economy of pupil time

Some procedure such as interviews and individual observation of pupil performances require longer time to complete.

7. Economy of teacher time

Essay test, term papers, project and written work require much teacher time to grade and evaluate.