Chapter 2 - Test Design and Underlying Rationale

he CSP and LSP were initially designed on the basis of the experience of the first two authors, sharing over 50 years of practice with elite athletes, coaches and teachers. The preliminary categories and their items were brainstormed as being the most potentially relevant and valuable for competitive athletes and their coaches. After being subjected to several pilot studies, they were further revised several times before being subjected to formal study.

The initial sample of athletes was composed of 2400 men and women, from several Division 1, University programs, who were competing in a wide variety of individual and team sports. Respondents completed the inventories in exchange for the sports psychology services they would receive. They were informed that they were voluntarily taking part in an experiment to better understand the various components of athletic performance and learning.

The athletes first completed the LSP and then the CSP. A standard answer sheet was utilized, asking each participant to furnish demographic information including age, sex, level, race and sport. The athletes were then presented with different learning and performance scenarios and responded to statements on a Likert scale, from 1 (untrue for me) to 5 (very true). The administration time for both inventories varied from 25 to 45 minutes. The subjects in this and the two subsequent studies also completed a consent form, voluntarily giving their permission to have their data processed and included as part of the reporting of group scores.

As a promise for their taking valuable time out of the team's training and practice schedule, we made the following statement to all the athletes and coaches who participated in our studies: "We will return within three days and distribute the individual profiles to each and every athlete. At this team meeting, we will review and define every attribute included in the scored profiles. You'll be given a clear definition of all the variables and their possible meaning. We will attempt to describe how each of them may affect optimal performance and learning. We will be honored to meet individually with interested coaches and athletes to review any of our findings or their suggested implications. We will be available afterwards by phone, should either coach or athlete value such an opportunity".

The data for this first study was collected during the fall semester of 1994. As a result of the statistical analysis for internal consistency of scales, a number of categories were revised and others (Situational Anxiety, Physical Expressiveness) were dropped. After the second study of 1280 high school, college and recreational athletes, one category (Attribution) was dropped due to limited utility and other categories were revised.

In the third and final study, the inventories were administered to a population of 1057 high school, college, masters and professional athletes. Results allowed for the demonstration of reliability and item consistency, examination of the factor composition of the resultant categories, development of norms for the comparison of scores and the creation of the finalized form of the profile printouts.

Rationale Underlying the Development of the CSP and LSP

The field of Sport Psychology is divided with regard to the value of psychometric instruments when offering service to athletes. Particular negative positions are held by a number of sport consultants who question the utility of most of the instruments being administered by sport consultants . A number of reservations were well grounded as exemplified by several concerns. There were questions as to the competence of those administering psychological inventories: were users sufficiently trained with regard to appropriateness of their chosen instruments? Had users received the training necessary for interpreting the data derived from assessment? These and other relevant concerns do require serious attention. We must acknowledge when including psychometric assessment that the client's self-disclosures invite ethical and humanitarian concerns.

This cautionary introduction with regard to concerns of numerous colleagues permits reference to a most timely publication by Samuel Messick of the Educational Testing Service, entitled The Validity of Psychological Assessment (Messick, 1995). For may years, he has sought to sensitize those in the field about particular social and ethical responsibilities when assessment instruments are employed. His critical analysis includes strong recommendations that every form of assessment measurement must be administered and interpreted with an understanding of the personal and social values of clients. Seeking cooperation from coaches, administrator and athletes as subjects for study required that we meet such professional standards Therefore, it is important that we detail and define how we attempted to honor the recommended standards.

In the development of the CSP and of the LSP, we were confronted with a number of issues and concerns, such as documenting the worth of the instruments that were still in their initial stages. As has been true for all the investigations we have conducted, every person we contacted wanted to know about the value they would derive from participating in our studies. Each wanted some guarantee that for time taken from their program, they would receive measurable benefits. In order to achieve this goal, the first two authors made themselves available to coaches and athletes for follow-up.

Every team, coach or individual athlete was apprised of the fact that assessment feedback would be based upon limited sampling. We stated that we felt great confidence in both instruments based upon over 200 consensual validation sessions with athletes from a number of sports, but asked them to consider all interpretations as tentative in nature. We have integrated into every aspect of our services a consideration for the implications and consequences of the role we have chosen to play in the lives of both athletes and coaches. In this manual we will pay particular attention to establishing an empirical framework as a means for testing hypotheses conveyed by score interactions.

In summary, we have chosen to accept the significance of Messick's Unified Validity Theory. Accepting extensions to traditional concepts of instrument validity will always guide our research and practice. Statistical evidence for content, construct and criterion validity will be reported for our third study. We wish particularly to convey that we have remained sensitive to values implications inherent in our social responsibilities. This can only be demonstrated by the extent to which we have remained protective with regard to the social consequence of actions taken on the basis score interpretations. The challenge will always be the extent to which empirical evidence and theoretical rationale support the adequacy and appropriateness of interpretation and consequent actions.

We have taken care to meet the statistical requirements incumbent upon those seeking to validate and standardize original psychometric instruments. The important question as to the suitability of our subject population must be addressed. We currently have an over-representation of select Division I university men and women, since one of our original goals was that of examining the performance and learning attributes of more experienced athletes. University men and women athletes recruited at Division I institutions certainly represent a highly select subject population.

At the risk of redundancy, we wish once again to state our underlying philosophy with regard to the utilization of both the CSP and the LSP. Each has been designed towards identifying the most important questions that can be posed for self examination with direct relevance for athletic performance and learning. Every subject received direct personal feedback during consensual validation sessions. We feel we can testify to these athletes' full cooperation and sincere interest. Their feedback and that of their coaches has been incredibly valuable in the development of these profiles.

CONTENTS | INTRODUCTION | CHAPTERS: