Evaluator Program

Support for Evaluating Instructional Technology Initiatives

The MassCUE Evaluator program is intended to assist individual districts in framing and conducting a rigorous program evaluation of district-wide technology initiatives and/or specifically funded instructional technology grants and projects. Districts can use the MassCUE Evaluator program to create district technology audits. These audits can in turn become the basis for effective district-level strategic technology plans. Even if technology planning is not the ultimate outcome of a district audit, a district engaging in this work will emerge with a strong picture of how its teachers and students utilize technology, the challenges faced in accessing and using district technology resources, and a current picture of technology professional development needs.

The MassCUE Evaluator program is a professional development offering from MassCUE. Participants can earn 30 professional development points (PDPs) for participation in the four day program.

The evaluator’s project centers around providing district leaders and instructional technology decision-makers with tools and techniques for assessing the impact of technology on student achievement. Specifically focused on the use of technology to support 21^st Century learning, this program helps districts maximize their investments in technology through a process of local data collection and analysis. The evaluation findings and results from this work help inform the creation of new strategies for improving learner outcomes.

Program format

The MassCUE Evaluator program consists of four days of face-to-face professional development, and related technical support, for a four member team in your district. The team should be focused on the development of a particular program evaluation (e.g., a technology audit). At the district level, the team should led by a district leader such as a Director of Curriculum and Instruction, Professional Development Coordinator, Superintendent, or Director of Technology. The team should have at least three other members who might be involved in various aspects of program implementation (e.g.,Instructional Technology Specialists, classroom teachers, etc.). Teams larger than four members can be accommodated at an additional fee (see “Cost”, below).

The district team will be facilitated by a skilled technology program evaluator from Sun Associates. The Sun Associates evaluator will provide the team with training, facilitation, and technical support focused on:

Framing evaluation questions and performance indicators connected to the team’s program goals and objectives
Developing data collection instruments and data collection procedures from the evaluation questions and indicators
Data analysis
Developing evaluation recommendations
Using program evaluation for formative assessment and continuous program improvement.

At relevant points in the process, additional expert consultants will be brought in to engage with the team. For example, a district wishing to include a focus on technology infrastructure in their evaluation can access MassCUE expertise on technical issues. Likewise, additional expertise can be brought in to discuss professional development structure, technology integration, etc.

Over the course of four meetings – scheduled in-district and according to the team’s needs and project timeframe — the team will frame and produce an evaluation of their project. In addition to the face-to-face meetings, the Sun Associates facilitator will provide technical assistance via phone, email, or on-site as necessary to each team during the implementation of their evaluation. While each team is responsible for conducting their own project data collection, the tools and protocol used for data collection will be developed with the guidance of the Sun Associates facilitator as a professional development activity. Likewise, the overall evaluation will be framed and produced via a tried and true evaluation methodology that the team will learn via the Individual program’s professional development.

Benefits of participation

Previous MassCUE Evaluator teams have used the training and support gained from this program to:

Measure the impact of current technology integration initiatives
Frame and produce district “technology audits”
Fine tune professional development programs
Create and revise strategic technology goals
Build consensus among district stakeholders for a vision for technology- supported 21st Century learning

As a bottom line, the MassCUE Evaluator program support Massachusetts’ school districts in the creation of evaluation and assessment procedures that examine how investments in technology change the way that teachers teach and students learn. The program was created to help districts gather data upon which to base difficult decisions about instructional technology and other resource intensive initiatives.

Participants who attend all four project meetings will earn 30 professional development points each.

Program facilitators

The program was designed by Sun Associates, an educational program evaluation firm based in North Chelmsford. Sun Associates provides program evaluation, technical assistance, strategic planning and auditing services to educational organizations such as schools, districts, and states. Sun Associates specializes in evaluation and planning related to the intersection between the use of technology tools and the development of core teaching and learning skills.

Additional information

The district should have a specific project or initiative in mind that they wish to evaluate. The district should identify a main point of contact (i.e., a team leader) within the district who can represent the district and discuss logistics. Upon expression of interest, MassCUE/Sun Associates will contact the district point of contact and work out the specific details related to project timeline, meeting dates, and team membership.

The district should create a team composed of individuals representing the following roles:

Technology specialist or coordinator
Curriculum and Instruction specialist
Professional development specialist
Classroom teacher or instructional specialist who works with students

Since effective program evaluation has a direct connection to district and project leadership, it is essential that teams contain those individuals who have direct responsibility for implementing projects and the ability to implement the results of the evaluation. Each team should operate under the authority of a school or district administrator who will be expected to hold the team accountable for producing and implementing a useful assessment for the target project (or technology plan). A statement describing this accountability is a required part of each team’s application to the MassCUE Evaluator program.

The program is priced by team and is designed to cover its basic costs of operation. The precise price will be based on program location, number of teams, etc.