Other types of questions, however, are more common. Some show a distinct line of cleavage between the novice and the apprentice. Novices fail, but apprentices, journeymen, and experts alike answer correctly. There are likewise questions that are answered correctly by nearly all journeymen and experts but only a few apprentices, and questions that only an expert can answer correctly.
Each type of question has its value in a good test. The main requirement is that the tendency of the curve should be upward; a question which is answered correctly by more journeymen than experts or more apprentices than journeymen is undesirable and is at once discarded. A proper balance is made of the others.
One task still remains; namely, that of calibrating the test. As each question is allowed four points, it becomes necessary to determine how many points should indicate an expert, how many a journeyman, etc. Obviously the way to do this is to note how many points were scored by the known experts and the known journeymen when they were tested. Ordinarily the expert scores higher than the journeyman and the journeyman higher than the apprentice. It frequently happens that a few journeymen score as high as the lowest of the experts and a few apprentices as high as the lowest of the journeymen. There are consequently certain overlappings between the classes. In calibrating, the object is to draw the dividing line between classes so that the overlapping shall be as small as possible.
When these dividing lines, or critical scores as they are usually called, are established, the test is ready for editing, printing, and distribution to camps.
Picture tests are made in practically the same way as oral tests. The peculiar characteristic of picture tests is that the questions making up the tests relate to illustrations of trade tools and appliances.
The performance tests are now being used in many trades for those who make a satisfactory showing in the oral or picture tests. These performance tests are devised by conference with experts in the trade. They consist of some apparently simple tasks that can be performed quickly and with a small amount of apparatus but that nevertheless indicate clearly the degree of skill of the performer. As a result of experience the following have been drawn up as the requirements for a good performance test:
1. It should require the smallest possible quantity of tools and materials and these should be capable of standardization;
2. A journeyman should not require more than 45 minutes to perform it;
3. It should be typical of the work required;
4. The operations should be exact so that a correct standard form of product is always obtainable. Performance tests undergo much the same processes of sampling as do the oral and picture tests and they are calibrated in the same way. The principle followed here, as elsewhere, is that the value of a test lies not in its theoretical exactness but in its proved ability to pick out and classify correctly men of all degrees of skill within the trade. If the test does classify men in the groups in which they are known to belong, then it can be relied upon to classify correctly men about whom nothing is known in advance.