The Family Center on Technology and Disability

Assistive Technology Outcomes:

What outcomes information do we need and what we do with it even if we had it?

Our need to measure and document the outcomes of assistive technology devices and services seems to be "duh uh!" Depending on your viewpoint, words like, accountability, satisfaction, cost-benefit, consumer selection, best practice, evidenced-based practice, or best product make the need for outcomes obvious.

However, maybe the fact that we all have our own perspectives about why assistive technology outcomes are needed is exactly why it turns out to be a complex task. There are many issues related to what types of outcomes should be measured, how much time and money should we invest into the measurement process, and what do we do with the outcomes data, even if we can collect all we want. And of course this all depends on your perspective.

To prod our thinking, I am going to be controversial and make some statements that can be well-defended, but might not feel like the right answers.

What outcomes do we need to measure?

A. Outcomes Data

• Change in performance/function (body & activity)
• Change in participation in school and community activities
• Usage of assistive technology (or lack of use)
• Device user satisfaction of services and devices)
• Goal achievement
• Quality of life

B. Cost Data

• Cost of devices, services, and individual’s time investment

C. "Ingos" Data

• Demographic information about the individual and the history of the individual.
• Services and devices provided, including other concurrent services provided
• Environmental context

These are broken into three major type of data needed for measuring outcomes: a) outcomes, b) costs, and c) "ingos." The outcomes are the results of interventions. The costs explain what resources are expended (or saved) by the intervention. The "ingos" are an important flipside of the outcomes we often miss. In order to examine an outcome we need to know what the intervention was and what other interventions are occurring. Otherwise, the outcomes we measure could be the outcomes of some other happening in the person’s life.

For example, if Johnny shows improvement in the school classroom after receiving an augmentative communication device, how do we know it was because of the device and not due to Johnny’s general education teacher who takes the time to communicate with him, a new classmate of Johnny’s, Johnny’s new skills in vocalization, or the teacher aide who has been helping more? Furthermore, let’s say we know the device made the difference. We still need to know what all went into the assistive technology intervention to know what worked. For example, how much training was provided, what vocabulary was programmed into the device, which team members participated, how much support was provided in the home, etc. We know that the success of an assistive technology device can be greatly facilitated by one team process or hampered by another. So which process was used with Johnny?

Or as another example, perhaps we discovered a new power wheelchair system that helped Sally become completely functional in all settings, in all activities, with everyone she encounters, and she was extremely happy about how it worked. Later we discovered that this wheelchair system, besides its initial expense, was extremely costly because it required frequent tune-ups and repairs from the service center that was based in a metropolitan area, two hours and a half hours from where Sally lives. We wouldn’t have a complete picture of the outcome of this particular assistive technology system without follow-up and comprehensive outcomes data.

So, if we really want to know how well an assistive technology device works; we need substantial outcomes data from a variety of outcome domains.

How should we measure the outcomes of assistive technology?

There are literally hundreds of assessments related to assistive technology assessment. Many are devoted to the process of selecting the best assistive technology device. Disappointingly, today, few have an outcomes focus. While new assessment resources are being developed, what do we do in the meantime? Consider the following simple idea. If we want to know how assistive technology works, we must at a minimum:

• Log exactly what interventions were provided (devices and services) and in what context.
• Measure at least two of the 6 types of outcomes. One of them should be some type of specific performance that was expected to change.
• Measure a baseline before the assistive technology intervention and then later use the same measure after the intervention.
• Measure performance outcomes of the student with and without use of the assistive technology.

How do we find the time to measure the outcomes of assistive technology?

Today, if we asked any assistive technology service team member (including family members and the individual with a disability) how much time they had to measure outcomes, they would respond either by saying virtually no time, or maybe even negative time. If this is the case, how do we go about measuring outcomes? We know we need to understand what assistive technology devices and which assistive technology services work best under what circumstances.

Today’s system makes it very difficult for us to find time for measuring outcomes. Perhaps a better question might be, "If measuring outcomes of assistive technology devices and services was mandated as a part of the process of receiving assistive technology devices and services, then how much time might be available?" If special education and rehabilitation program coordinators and supervisors assumed that measuring outcomes was part of the job and a percentage of time was automatically allocated for documenting outcomes wouldn’t this change the scenario? We know that measuring outcomes takes resources. If those resources were made available or it was assumed that measuring outcomes did take time, the answer might be a little different.

Assistive technology interventions need a mandated format for measuring appropriate outcomes. One mechanism might be to expand the notion that assistive technology devices and services must be "considered." In the school systems we know that IDEA requires that assistive technology must be considered. Perhaps it also needs to require that outcomes must be measured. A perfect place for this is in the existing IEP. A mini-assistive technology plan could be called an ITIP for "Individualized Technology Implementation Plan". This plan would require goals for the assistive technology intervention and repeated measures of goal achievement. This strategy could also apply to non-school-based assistive technology services and for all age-groups. A structured ITIP process could not only help document outcomes, but it could also facilitate continuity of services across agencies and the lifespan of a person with a disability.

We know on one level that this conceptual model might be able to work. The state of Ohio is currently in the process of distributing over $9 million to the students in their state who require assistive technology devices to meet goals within their IEP. As a part of the contract for these special education teams to receive the funds to purchase assistive technology devices, they need to provide relevant IEP’s and measure the outcomes of the assistive technology intervention. While all of this takes time, special education team members understand that they need to be accountable and document how the assistive technology funds are being used.

The question still remains, however, "What is an appropriate amount of time a assistive technology team should devote to measuring the outcomes of assistive technology interventions?" Perhaps a way to think about this might be in percentages. The time invested in evaluating and identifying the need for assistive technology, the time procuring the device, the time setting it up, time training the user how to use the device, and the time spent making sure that the device works the way it should could be described as the assistive technology implementation time demand. Perhaps it is reasonable to assume that 15-25% of this time should be spent measuring and documenting the need and the outcome of the intervention.

What would we do with the outcome data if we had it?

Administrators, special education coordinators, rehabilitation program supervisors, special education teachers, general education teachers, therapists, parents, friends of people using assistive technology and the assistive technology users are some of the constituents of assistive technology devices and services. Each of these groups has their own reason for wanting assistive technology outcomes information. What do we know about what each one of these groups wants to do with the outcomes data if they had it? Some of the uses of outcomes data could be:

• Comparing one device to another to decide which one to purchase.
• Identifying what types of devices have been used before by people in similiar situations; and what worked and what didn’t.
• Examining a list of assistive technology teams and how they were organized differently, and looking at how the outcomes varied among them.

Since virtually no outcomes data are handy today, this question points toward the future. What if you had a magic wand and could have access to any outcomes information you wanted. What information would you want to see, what would it look like, how would you access it? The future is created by vision. The outcomes system of the future will be driven by today’s dreaming.