- About Us
- Our Work
- What’s New
- CA Insights
- 2017-18 CA Insights
- 2016–17 CA Insights
- 2015–16 CA Insights
THE CENTER’S CA INSIGHTS 2017-18:
Methodology & Demographics
Phone interviews were conducted from October 2017 to March 2018. During the one-hour call, district leaders were asked about their capacity, constraints, and decision-making related to implementation approaches; funding; organizational structure; process, measurement, and impact; professional learning and development; and instructional materials. Inductive coding methods were used to convert qualitative interview data into discrete categories for quantitative analysis. The Center is developing and reporting on findings from these data throughout 2018.
In the fall of 2017, principal and instructional coach focus groups were conducted separately in four regions of California: Yuba City, Fresno, San Francisco, and Long Beach areas. Each group session lasted for two hours and consisted of six to 10 participants. Selection of participants was based on strict criteria: each group had no more than two participants from a single school district, each was evenly divided between participants from primary and secondary grade levels, and each featured wide variation in years of experience. This selection process resulted in 30 principals and 35 instructional coaches representing all K-12 grade bands and 43 districts.
The research protocol was carefully developed by speaking with educators and stakeholders from across the state, including representatives from county offices of education, education policy leaders, researchers, and technical assistance providers. Their feedback concerning what they sought to learn about the current state of standards implementation informed the kinds of questions and topic areas covered in the focus groups. The Center then analyzed the transcripts using categories derived from the protocol. Specific categories developed for coding included: implementation approach; instructional shifts; professional learning and development; data and assessment; and instructional materials.
Teacher and School Leader Surveys
Designed to survey the same educators at regular intervals over time, the RAND Corporation’s American Teacher Panel (ATP) and American School Leader Panel (ASLP) surveys launched in 2014 and have been administered multiple times yearly since. The ATP and ASLP surveys are nationally representative, with oversamples in four states (California, Louisiana, New Mexico, and New York) to ensure state-level representativeness in each. RAND currently uses single-stage, stratified random sampling to draw new teachers and principals at random for its panels, from lists contained in existing, comprehensive databases (stratifying on the basis of state, for oversampling). Each panelist is assigned a weight that accounts for sample design and probability of non-response (taking into account key variables like school size, grade span, poverty status, and experience), and linearization techniques are used to calculate margins of error. Panel members complete three or four web-based surveys every year, and panelists receive a small monetary incentive after completing each survey.
To create the ATP, RAND researchers first sampled 2,300 public schools from the 2010/11 and 2011/12 federal Common Core of Data (managed by the U.S. Department of Education’s National Center for Education Statistics), and stratified that sample on five variables: grade span (primary, middle, high and combined); school size; poverty status (proportion receiving free/reduced-price lunch); population density; and geographic region. Then, using probability-sampling methods, two regular full-time teachers were selected from each sampled school to join the ATP, and these teachers were stratified by experience level (novice and experienced) and subject area (three strata at the elementary level, and five strata at the secondary level). Individuals who change schools remain on the panel; new members are added periodically so the panel remains representative of teachers over time.
A total of 798 California teachers were included in the ATP sample for the May 2017 survey administration, and 479 responded, for a response rate of 60 percent. Weighting, which accounts for differential sampling and non-response, was used to produce results representative of the full state sample (yielding an approximate margin of error of ±4.5 percent). For the May 2017 ASLP survey, 386 of 1,024 California school leaders (38%) responded. For the October 2016 survey administration, 523 California teachers were included in the ATP sample and 281 responded (54%), yielding an approximate (average) margin of error of ±6.5 percent. Only 45 of the 130 sampled California school leaders (35%) responded to the parallel October 2016 ASLP survey.