How to implement EBPs

How to implement EBPs

Scaling Up Innovations Scaling Up Effectiveness OR Superintendents Forum April 2010 Dean L. Fixsen, Ph.D. & Karen A. Blase, Ph.D. University of North Carolina - Chapel Hill Rob Horner, Ph.D. University of Oregon George Sugai, Ph.D. University of Connecticut Challenges OR Public Schools Bottom 20%

Students: 565,000 Schools: 1,800 School Districts: 196 Counties: 36 Budget: $8.5 Billion 113,000 360 39 36 Follow Through Programs Figure 1: This figure shows the average effects of nine Follow Through models on measures of basic skills (word knowledge, spelling, language, and

math computation), cognitive-conceptual skills (reading comprehension, math concepts, and math problem solving) and self-concept. This figure is adapted from Engelmann, S. and Carnine, D. (1982), Theory of Instruction: Principles and applications. New York: Irvington Press. Achieving Student Benefits Hattie (2009) recently reported a metaanalysis of 816 meta-analyses 52,649 research studies in education involving over 83 million students, teachers, staff, parents, and others. "It is what teachers get the students to do in the class that emerged as the strongest component of the accomplished teachers' repertoire." Hattie (2009)

Achieving Student Benefits Increasing opportunities to respond and the amount/ accuracy of feedback is an important correlate of student achievement The feedback to the teachers about what students can and cannot do is more powerful than feedback to the student This requires a change in the conception of what it means to be a teacher not a solo performer Hattie (2009)

Achieving Student Benefits We now know a lot about WHAT to do to educate students We can improve education for students on purpose! Science to Service SCIENCE SERVICE IMPLEMENTATION GAP Challenges Science to Service Gap

What is known is not what is adopted to help students Implementation Gap What is adopted is not used fully and effectively in practice Achieving Student Benefits Longitudinal Studies of a Variety of Comprehensive School Reforms Good Intentions Actual Supports Years 1-3 Outcomes Every Teacher Trained

Fewer than 50% of the teachers received some training Fewer than 10% of the schools used the CSR as intended Every Teacher Continually Supported Fewer than 25% of the teachers received support Vast majority of students did not benefit

Aladjem & Borman, 2006; Vernez, Karam, Mariano, & DeMartini, 2006 Challenges You are not alone! Superintendents across the nation are facing the same problems: Lack of consistency across teachers, schools, and years Lack of capacity to make meaningful changes and sustain them SISEP Center In 2007 State of Oregon participated in a process to select States to create an

infrastructure for implementation of innovations statewide. 1 of 36 interested States 1 of 16 applicant States 1 of 6 chosen States that met the selection criteria and site visit criteria (IL,MI, MN, MO, OR, VA) SISEP Center State Implementation and Scaling up of Evidence-based Practices (SISEP) www.scalingup.org Dean Fixsen and Karen Blase National Implementation Research Network, University of North Carolina at Chapel Hill

Rob Horner University of Oregon George Sugai University of Connecticut Capacity Building The SISEP Center Intensive and focused activity to build state capacity and align system structures, roles, and functions Use implementation science and best practices across programs and innovations Large scale, real time change Challenges

Students cannot benefit from interventions they do not experience Teachers and staff have to change if students are to benefit Dobson & Cook (1980) Implementation Science Know-WHAT Knowledge of the intervention Know-HOW Knowledge of implementation Tucker, Edmondson, & Nembhard (2005)

Know WHAT Choose Interventions Wisely Meaningful Improvement Eventually want to see educationally and socially significant changes in student outcomes across the State Must be worth the effort to scale up (e.g. EBISS) Know HOW IMPLEMENTATION

INTERVENTION Effective Effective Student Benefits NOT Effective Poor Outcomes Highly variable, often ineffective, NOT Effective sometimes harmful to students, families, and adults (Institute of Medicine, 2000; 2001; New Freedom Commission on Mental Health, 2003; National Commission on Excellence in Education,1983; Department of Health and Human Services, 1999)

Implementation Implementation Research: A Synthesis of the Literature Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Download all or part of the monograph at: http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=31 Implementation Science Data Show These Methods, When Used Alone, Do Not Result In Uses of Innovations As Intended:

Diffusion/ Dissemination of information Training Passing laws/ mandates/ regulations Providing funding/ incentives Organization change/ reorganization Implementation Science Data Show These Methods, When Used Alone, Do Not Result In Uses of Innovations As Intended We know a lot about ineffective methods because they are the ones we use!

Implementation science will improve as implementation practices improve (create a better laboratory) Student Benefits Implementation Takes Time Exploration Performance (Sustainability) Assessment (Fidelity) Installation (Sustainability) Coaching Initial Implementation Adaptive Innovation Training

2 4 Years Systems Intervention Organization Facilitative Administration Full Implementation Integrated & (Effectiveness &Compensatory Sustainability) Selection Technical Decision Support Data System Leadership

Fixsen, Naoom, Blase, Friedman, & Wallace, 2005 Implementation Science Implementation Drivers Common features of successful supports to help make full and effective uses of a wide variety of innovations Reliable Benefits for Students Consistent uses of Innovations Performance Assessment (Fidelity) Coaching Staff Competence Systems Intervention

Organization Supports Training Integrated & Compensatory Facilitative Administration Decision Support Data System Selection Technical Leadership

Adaptive Fixsen & Blase, 2007 Staff Coaching OUTCOMES (% of Participants who Demonstrate Knowledge, Demonstrate new Skills in a Training Setting, and Use new Skills in the Classroom) TRAINING COMPONENTS Theory and Discussion ..+Demonstration in Training + Practice & Feedback in Training + Coaching in Classroom

Knowledge Skill Demonstration Use in the Classroom 10% 5% 0% 30% 20% 0% 60%

60% 5% 95% 95% 95% Joyce and Showers, 2002 Support Implementation Students cannot benefit from education practices they do not experience Support implementation

practices within schools and districts Capacity Building Scaling up = at least 60% of the students who could benefit from an innovation have access to that innovation Achieve significant benefits to students and society Capacity Building Letting it happen Recipients are accountable Helping it happen

Recipients are accountable Making it happen Purposeful use of implementation practices and science Implementation teams are accountable Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004 Capacity Building Letting it happen Helping it happen

Recipients are accountable Recipients are accountable Making it happen Implementation Teams are accountable: THEY DO THE WORK (Heart of Scaling) Based on Greenhalgh, Robert, MacFarlane, Bate, & Kyriakidou, 2004 Implementation Team Minimum of three people (four or five preferred) to promote effective, efficient, and sustainable implementation, organization change, and system transformation work Tolerate turnover; teams are sustainable even when the players come and go

Implementation Team A group that knows the innovations very well (formal and craft knowledge) A group that knows implementation very well (formal and craft knowledge) A group that knows improvement cycles to make intervention and implementation methods more effective and efficient over time Implementation Team Implementation Team Simultaneous, Multi-Level Interventions Teacher & Staff Competence School & District Supports Management (leadership, policy) Administration (HR, structure)

Supervision (nature, content) Regional Authority Supports State and Community Supports Implementation Team Prepare Communities Prepare Regions Prepare schools and staff Implementation Team Assure Student Benefits

Parents and Work with Researchers Stakeholders Create Readiness Assure Implementation Fixsen & Blase, 2009 Implementation Science IMPLEMENTATION INTERVENTION Impl. Team Effective 80%, 3 Yrs NO Impl. Team 14%, 17 Yrs

Effective use of Letting it Happen Implementation Helping it Happen Science & Practice Fixsen, Blase, Timbers, & Wolf, 2001 Balas & Boren, 2000 School Wide PBS SWPBS # of Schools 12,000 10,000 8,000 6,000 4,000 2,000 0 Aug 04 Aug 05 Aug 06 Aug 07 Aug 08 Aug 09

School Wide PBS SWPBS % of Schools 90,000 80,000 70,000 60,000 50,000 40,000 30,000 20,000 10,000 0 12% in 17 Years (1992-2009) Aug 04 Aug 05 Aug 06 Aug 07 Aug 08 Aug 09 Costs and Savings Change in Budget (Percent)

Implementation Costs & Savings (Inflation Adjusted) Short-Term Investment in Imple. Capacity 120 110 100 90 Realize Long80 70 Term Benefits 60 1 Yr Pre During

Post Year 1 Post Year 2 Post Year 3 Costs and Savings This years success pays for next years increase in capacity Barber & Fullan (2005) Change Systems

To scale up, we need to: Turn policy into effective practice Create an infrastructure for implementation of innovations Turn effective practice into policy Challenges OR Public Schools Bottom 20% PROBLEM: Students:The 565,000 bottom

is distributed 20% Schools: 1,800 throughout the state and shifts School 196 eachDistricts: year Counties: 36 Budget: $8.5 Billion 113,000 SOLUTION: capacity Plan 360 to reach ALL

39 schools 36 State Department Leadership District Leaders and Staff School Teachers and Staff All Students & Families N = 565,000 Fixsen & Blase, 2008

State Department Leadership ImplementationSkilled Workforce N=6 Regional Implementation Teams (N=5) 1 for every group of 5 Districts (N = 14 Regional Teams) N = 70 District Implementation Teams (N=4) 1 for every group of 25 Schools

(N = 72 District Teams) N = 288 School Implementation Team (N=4) School Teachers and Staff 1 for each School (N = 1,800 School Teams) All Students & Families N = 7,200 Re-Purpose

1 for every 10 RITs (N = 1 State Teams) N = 76 < 0.1% $$ State Transformation Team (N=6) N = 565,000 Fixsen & Blase, 2008 State Capacity Development 16 Regional Impl. Teams 14 RITs 12

10 8 6 4 Saturation Intensive Development Intensive Development STTs 2 0 1 2 3

4 5 6 7 Years 8 9 10 11 12 Oregon Districts and Student Enrollment

ODE Report Card 2008-2009 Scale Up To scale up interventions we must first scale up implementation capacity Building implementation capacity is essential to maximizing the statewide use of EBPs and other innovations Scale Up Oregon needs

About 14 Regional Implementation Teams (and support staff) One State Transformation Team Annual cost about $8 million About $40,000 per year per district (or $14 per year per student) System Change Innovative practices do not fare well in existing organizational structures and systems

An infrastructure for implementation does not exist Organizational and system changes are essential to successful use of innovations Legacy Systems A legacy system is a system or application that continues to be used despite its poor competitiveness and compatibility with modern equivalents Difficult to integrate new systems into legacy systems because it is a difficult and

time intensive process to understand current system functionalities Legacy methods create a huge conversion challenge for implementation teams Ashok R. B. Samuel (2009) System Change EXISTING SYSTEM EXISTING SYSTEM IS CHANGED TO SUPPORT THE EFFECTIVENESS OF THE INNOVATION EFFECTIVE INNOVATIONS ARE CHANGED TO FIT THE SYSTEM

EFFECTIVE INNOVATION Compliance and Crises, Urgent, Time Sensitive!! Services not meeting Standards Deal with urgent and high profile issues Best Practices Implemented Fully With Good Outcomes System Supports & Stability System Supports & Stability Mandates, System Supports, Foundational Polices & Regulations Regulatory roles Basic Data Systems Financing and Fiscal Accountability

Accreditation/ Licensing Standards HR rules and regulations Safety Standards Work with Legislature Inclusion of Stakeholders Leadership Responsibilities and Leverage Points Thanks to Tom Bellamy Implementation Team Teachers Innovations Students RFP methods IHE curricula Practice Informed Policy (PIP)

System Change Management Team Policy Enabled Practice (PEP) External System Change Support Adaptive Challenges Salaries Funding Credentialing Licensing Time/ scheduling Union contracts Duplication Fragmentation Hiring criteria

Federal/ State laws SYSTEM ALIGNMENT ALIGNMENT State Department Districts Implementation Teams Federal Departments Schools Teachers/ Staff Effective Practices FORM SUPPORTS FUNCTION The End in Mind

With the purposeful use of implementation science, we can: Make statewide use of good instruction, evidence-based practices, and other innovations To produce increasingly effective outcomes for all students For the next 50 years. Implementation Science Global Implementation Conference 2011 www.implementationconference.org Integrate the science, practice, policy of

implementation, organization change, and system transformation Call for Applications Science and Service Award Program Do you know of an organization/coalition currently implementing an evidence based practice in their local community? Is the implementation of this program/innovation producing beneficial outcomes to the community?

Are they a role model for moving the evidence-based practice from science to service? If you answered YES to all of the above, check out: http://www.samhsa.gov/scienceandservice Nominate your own organization or someone you know!! For More Information Karen A. Blase, Ph.D. Dean L. Fixsen, Ph.D. 919-966-9050 919-966-3892

[email protected] [email protected] Frank Porter Graham Child Development Institute University of North Carolina Chapel Hill, NC http://nirn.fpg.unc.edu/ For More Information State Implementation and Scaling up of Evidencebased Practices (SISEP) Dean Fixsen, Karen Blase, Rob Horner, George Sugai www.scalingup.org Resources Tab Concept paper Annotated bibliography

Data on scaling up Scaling up Briefs Evidence-based Implementation Research: A Synthesis of the Literature Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231). Download all or part of the monograph at: http://www.fpg.unc.edu/~nirn/resources/detail.cfm?resourceID=31 Thank You for your Support Annie E. Casey Foundation

(EBPs and cultural competence) National Institute of Mental Health (research and training grants) William T. Grant Foundation (implementation literature review) Substance Abuse and Mental Health Services

Administration (implementation strategies grants; national implementation awards) Juvenile Justice and Delinquency Prevention (program development and evaluation grants Office of Special Education Programs (Scaling up Capacity Development Center) Administration for Children and Families (Child Welfare

Leadership Development) Duke Endowment (Child Welfare Reform) Centers for Disease Control & Prevention (implementation research)

Recently Viewed Presentations

  • What is Ecology?

    What is Ecology?

    Ecology: the study of the relationships of organisms to each other and to their environment. Ecology, is the scientific study of the distribution and abundance of living organisms and how they are affected by interactions between the organisms and their...
  • An Examination of Oceanic Surface Wind in the

    An Examination of Oceanic Surface Wind in the

    Wind stress distribution is similar to surface wind except magnitude of differences is greater. CDAS2 and CFS have too strong wind stress near the South American coast - Wind stress in CDAS1 and CFS is too weak north and south...
  • Louisiana Tax Topics

    Louisiana Tax Topics

    Louisiana Department of Revenue ... joint venture, or any other business organization of which the domestic or foreign corporation is a related party as defined in R.S. 47:605.1. ... Extends the sunset of reductions to certain rebate programs made in...
  • Department of Computer and Information Science, School of

    Department of Computer and Information Science, School of

    This is already sum-of-products form. Any non-zero Boolean expression can be changed to sum-of-products form. (By duality, there is also a product-of-sums form, but it is used less often.) Converting to Sum-of-Products Form Use DeMorgan's Laws and Involution to move...
  •  Copyright Valiant Technology Ltd 2/23/20 Paddington Bear As

    Copyright Valiant Technology Ltd 2/23/20 Paddington Bear As

    Paddington Bear Paddington Bear Paddington Bear Paddington Bear Paddington Bear Paddington Bear As many of you are probably aware Michael Bond and Stephen Fry are visiting Paddington in Peru. He will then return with them for a long holiday in...
  • Counting DBE Participation &amp; Assessing Commercially Useful ...

    Counting DBE Participation & Assessing Commercially Useful ...

    Last day to submit Goals. It is a good idea to plan ahead in order to meet the August 1 deadline. This slide shows you a typical timeline. You want to begin in the Spring, around April by identifying opportunities,...
  • The Life cycle of a Star The Earth

    The Life cycle of a Star The Earth

    The Life cycle of a Star. Stars are born out of nebulae (a nebula is an inter-stellar cloud of gas (mainly hydrogen) and dust) Gravitational forces pull at the gas and dust compressing it into a slowly rotating globule.
  • Achieving Success Through Effective Business Communication

    Achieving Success Through Effective Business Communication

    Achieving Success Through Effective Business Communication © Prentice Hall, 2008 Business Communication Today, 9e Chapter 1 - *