Assessing Client Satisfaction in Vocational Rehabilitation Program Evaluation: A Review of Instrumentation

Journal of Rehabilitation,  Oct, 1995  by Lynn C. Koch,  Mary Ann Merz

• E-mail
• Print
• Link

Program evaluation may be defined as a systematic procedure for determining the effectiveness and efficiency with which an agency or service delivery system achieves its objectives (Glass, 1989). Program evaluation has evolved out of the need to assess social program effectiveness and has been a priority within the state-federal vocational rehabilitation program since the passage of the Rehabilitation Act of 1973. This Act resulted in a major policy shift by mandating that service priority be given to individuals with severe disabilities. It also resulted in the development of standards for program evaluation to increase accountability of state rehabilitation programs (Walls & Tseng, 1987).

The Rehabilitation Act Amendments of 1992 identified the need for state programs to address the serious problem of "patterns of inequitable treatment of minorities (that) have been documented in all major junctures of the vocational rehabilitation process" (Sec. 102, 3). Ongoing program evaluation must be undertaken since annual reports describing the progress toward remediation of this problem must be submitted to Congress. Additionally, the Amendments mandate that consumer satisfaction surveys be conducted to identify other areas of vocational rehabilitation that need to be expanded, improved, or modified.

More than 20 years ago Reagles, Wright, and Butler (1970) reviewed the literature on instrumentation to measure client satisfaction and found a "dearth of studies." A subsequent review conducted by the present authors identified a number of measures of client, consumer, and patient satisfaction that have been developed over the past 20 years in psychology, medicine, and marketing. The purpose of the present paper is to review four of the instruments identified that would appear to hold promise in the evaluation of client satisfaction in vocational rehabilitation programs: the Scale of Client Satisfaction (Reagles, Wright, & Butler, 1970); the Client Satisfaction Questionnaire (CSQ; Larsen, Attkisson, Hargreaves, & Nguyen, 1979); the Evaluation Ranking Scale (ERS; Pascoe & Attkisson, 1983); and the Patient Satisfaction Questionnaire (PSQ; Ware, Snyder, & Wright, 1976a, 1976b). The Scale of Client Satisfaction was developed specifically for use in vocational rehabilitation programs while the others were developed for use in other settings. Each of the instruments is described in terms of format and development, as well as documentation regarding reliability and validity. Finally, a summary of the critical issues surrounding measurement of client satisfaction is presented, along with suggestions for further research.

Reviews of Scales

Scale of Client Satisfaction

Description and development. The Scale of Client Satisfaction was developed at the Regional Rehabilitation Research Institute, University of Wisconsin-Madison, as a part of the Wood County Project to evaluate the impact of a five-year expanded program of vocational rehabilitation (Wright, Reagles, & Butler, 1970). Prior to the Wood County Project, the measurement of client satisfaction was frequently excluded from the overall evaluation of the outcome of rehabilitation programs (Reagles, Wright, & Thomas, 1972).

The scale of client satisfaction consists of 14 items. Some items feature multiple choice responses while others use yes/no format. Among the aspects of satisfaction addressed are the convenience of appointments (e.g., time and location), helpfulness of the counselor, length of time from first vocational rehabilitation contact to first appointment with the counselor, frequency of visits with the counselor, and degree of satisfaction with the amount of time spent with the counselor (Reagles et al., 1972).

The scale is a self-report measure recommended for administration about six months after case closure. Scoring weights for all possible responses to each of the 14 items were empirically determined using the method of reciprocal averages (RAVE), so as to maximize the internal consistency of total scores (Reagles et al., 1972). The weights corresponding to responses for each item are summed over the 14 items to yield a total score, with higher scores representing greater satisfaction with rehabilitation services.

Reliability and validity. The Scale of Client Satisfaction was administered to 483 clients who received services from the Wisconsin Division of Vocational Rehabilitation and whose casefiles were closed as "rehabilitated." A Hoyt reliability coefficient of .83 was obtained, reflecting internal consistency of total scores. This was interpreted to mean that: "(a) the development of a scale for the purpose of measuring client satisfaction, conceptualized as a single variable, is feasible; (b) the 14 items are indeed measuring a single underlying variable...client satisfaction; (c) the items are scaleable; (d) the Hoyt reliability coefficient (.83) is sufficiently high for the purposes of this study; and (e) the inter-item correlation coefficients ranged from .09 to .67 with the majority between .22 and .31" (Reagles et al., 1972, p. 19).