Development of a Comprehensive Community Assessment of School Readiness
David
A. Murphey
State of Vermont,
Agency of Human Services
Catherine
E. Burns
Washington
County (VT) Mental Health: Children, Youth, and Family Services
Abstract
This paper describes the development of a multipart assessment of kindergarten readiness for the State of Vermont. The approach described reflects emerging consensus around the appropriate domains of development to include in child assessments, as well as the need to assess schools' readiness for young children and their families. The approach differs from some states' approaches, however, in its emphasis on readiness data for purposes of community-level accountability rather than to support individualized instruction. Data on children's developing competencies early in the kindergarten year (in five domains) were reported by teachers across the state (N = 181), on approximately half of the state's kindergartners (N = 2,992). Data on the "readiness" of schools were reported by principals (N = 197). Schools' "readiness" was conceptualized as including the areas of "smooth transitions to school," "instruction and staff development," "partnership with community," and "resources." Results confirmed the viability of a brief teacher-reported assessment of children and an assessment of "ready school" practices. Further tasks related to promoting local use of the assessment data, and implications for policy, are identified.
Introduction
Ever since the National Education Goals Panel (NEGP) identified as its first priority that "all children enter school ready to learn," but especially within the past few years, states have endeavored in various ways to come to terms with the challenge of measuring progress toward such a goal. "School readiness," or "ready for school," has become a shorthand for what is in truth a multidimensional concept--one that has the potential to do harm as well as good, as states move toward implementation of specific assessments.
As this work has progressed, a degree of consensus has emerged around certain critically important points, at least among those expert in working with young children. One is that a child's readiness for school is not simply a matter of alphabet knowledge, or even letter-sound correspondence, or other predominantly cognitive accomplishments, as important as those are. Rather, readiness includes social-emotional abilities, "approaches to learning" (i.e., dispositions such as enthusiasm, curiosity, and persistence), and communication skills (receptive and expressive), as well as motor development and physical health (National Education Goals Panel [NEGP], 1992).
Another emerging point of consensus is that readiness is an interaction: as children need to be ready to make the most of their school experience, so too do schools need to be "ready" to meet the diverse needs of young children and their families. Therefore, any comprehensive assessment of "school readiness" needs to include indicators of schools' capacities.
The National Education Goals Panel (NEGP) and others have identified important features of schools that indicate they are "ready" to accommodate the varied needs and experiences of young children entering school, and their families. According to these experts (Shore, 1998, p. 5):
- Ready schools smooth the transition between home and school.
- Ready schools strive for continuity between early care and education programs and elementary schools.
- Ready schools help children learn and make sense of their complex and exciting world.
- Ready schools are committed to the success of every child.
- Ready schools are committed to the success of every teacher and every adult who interacts with children during the school day.
- Ready schools introduce or expand approaches that have been shown to raise achievement.
- Ready schools are learning organizations that alter practices and programs if they do not benefit children.
- Ready schools serve children in communities.
- Ready schools take responsibility for results.
- Ready schools have strong leadership.
Notwithstanding these points of agreement, several distinctions in approaches can be drawn based on this work thus far. Perhaps the most important of these distinctions concerns the unit of analysis. In some examples of states' work in this area, the aim is to have a measure of "school readiness" that paints a portrait of young children's competence that has validity at the child level: that is, what an individual child "knows and can do." In contrast is an approach that aims instead at group-level validity: that is, what a community's children "know and can do." Of course, the psychometric requirements of these two approaches are very different.
A related issue concerns the purpose for such assessments. On the one hand, a detailed profile of individual child performance can be part of a process of continuous assessment throughout the school year, and the profile can function as a tool for improvement of instruction. On the other hand, an assessment of children's "readiness" can be simply a "snapshot" taken at what is in our culture an important developmental transition point. In the latter case, the implicit reference is, again, to how well a community has prepared its young children to be "ready" for school. Thus, the latter type of assessment takes its place within a framework of shared accountability (Emig, 2000; Meisels, 1998; Saluja, Scott-Little, & Clifford, 2000).
This paper describes the development and initial results of a set of brief measures intended to describe, at a community level, children's readiness for kindergarten and schools' readiness for young children and their families. Kindergarten teachers were the informants for children's readiness, and teachers and school principals provided information on schools' readiness. Taken together, the results describe five dimensions of children's readiness and four dimensions of schools' readiness.
Method
Development of the Measures
Our approach to assessment, as well as our choice of specific items, grew out of extensive earlier work in several Vermont communities that validated the reliability of kindergarten teachers as informants about children's readiness. In addition, our approach drew on the experiences and judgments of providers of early childhood services (Gorman & Burns, 1999).
Specific measures were further refined by an expert panel representing members of Vermont's departments of public health, education, mental health, and human services agencies, in addition to representatives of providers of early childhood services and staff from the University of Vermont's Department of Psychology. As part of this work, there was extensive review of the literature on assessment of "school readiness," as well as examples of specific measures used in other states.
All measures were pilot tested in four Vermont communities in the spring of 2000. Forty-one kindergarten teachers, in 27 schools, responded regarding 620 children. Twenty-four principals responded to the "ready schools" questionnaire. Additionally, nine focus groups were held in three geographically diverse regions of the state. Kindergarten teachers, parents of young children, and early childhood professionals were separately invited to focus groups to comment on the proposed measures. Focus group participants generally endorsed the constructs represented on the "ready kindergartners" measure. Concerns had mainly to do with potential use (and misuse) of the information. Based on these results, further modifications to the instruments were made. (For details, see Gorman & Burns, 2000.)
Identification of five domains within the "ready kindergartners" measure followed the recommendations of the NEGP and others.1 The final measure consisted of 24 items, together with demographic information on children and teachers. Teachers were asked to rate children individually on the items, through recollection rather than direct assessment, 4 to 6 weeks into the kindergarten year. An example of items included under "social and emotional development" is "Can meet/play with different children his/her own age." An example of items included under "approaches to learning" is "Appears enthusiastic and interested in classroom activities." An example of items included under "communication" is "Communicates needs, wants, or thoughts in primary language." An example of items included under "cognitive development and general knowledge" is "Understands the purpose of books." An example of items included under "physical health and well-being" is "Demonstrates self-help skills (e.g., toileting, wiping nose, washing hands) with occasional teacher assistance."
Identification of four domains of "ready schools" was similarly informed, with the expert panel determining assignment of specific items to domains, item weights within domain, and criteria ("standards") in each domain for what would be considered a "ready school." The primary source for the "ready schools" information was a 15-item questionnaire completed by principals of schools with kindergartens. Kindergarten teachers provided information about classroom support personnel. An example of items related to "smooth transitions to school" is one asking whether the school offers "move-up days"2 prior to the beginning of school. An example under "instruction and staff development" is one asking about average kindergarten class size. Under "partnership with community," an example is an item asking about school sponsorship of after-school care. An example of items included under "resources" is one asking teachers about the availability of various support staff (e.g., behavior specialist). The full set of measures was fielded statewide in the 2000-2001 school year.
Results
Characteristics of the Sample
The assessment was intended to include all children in public school kindergarten in Vermont, their teachers, and their principals. Because this year was the first of a new effort, and because participation was voluntary, there was less-than-universal participation.
Valid data were received from 181 kindergarten teachers (47% of the 383 contacted). Responding teachers represented 52 of Vermont's 60 supervisory unions.3 Child-level data were submitted on 2,992 kindergartners, which is approximately 46% of estimated kindergarten enrollment. Although 84% of principals (197) responded, they represented every supervisory union within the state (Table 1).
Of course, not all respondents answered every item on the instruments, so the number of valid responses varies somewhat by item. (Items having a nonresponse rate of 5% or greater are noted.)
Table
1
Sample
Characteristics
|
|
Number Responding |
Estimated Pool of Possible Respondents |
Percent Responding |
|
Kindergarten Teachers |
181 |
383 |
47.3 |
|
Children (kindergarten teacher report) |
2,992 |
6,511 |
45.9 |
|
Kindergarten Principals |
197 |
234 |
84.2 |
|
Kindergarten Teachers |
|
| Mean length of experience with kindergarten teaching | 10.2 yrs. (sd=7.03) |
| Mean length experience with teaching (total) | 17 yrs. (sd=8.29) |
| Have elementary education license | 93.4 pct. |
| Have early childhood endorsement | 31.5 pct. |
| Teach half-day program | 58.7 pct. |
| Teach full-day, 5 days/week program | 27.3 pct. |
|
Teach full-day, partial-week program |
14.0 pct. |
|
Kindergarten Students |
|
| Qualifies for special education services |
7.4
pct.
|
| Qualifies for ESL/bilingual services |
1.6 pct. |
| Qualifies for Sec. 504 services |
1.0
pct.
|
| Teacher reports on child's experience prior to kindergarten: | |
|
Was in regulated early childhood program |
66.2 pct. |
|
Was not in regulated early childhood program |
16.4
pct.
|
|
"Don't know" |
2.0
pct.
|
|
Missing response |
15.3 pct. |
|
Schools |
|
|
1 kindergarten session |
45.9 pct. |
| 2 kindergarten sessions | 32.5 pct. |
| 3+ kindergarten sessions | 21.6 pct. |
| Mean average class size | 13.7 |
Results of "Ready Kindergartners" Assessment
Table 2 shows the item-level results for the teacher-scored child competencies. In order to test internal consistency of the items by domain, Cronbach's alphas were calculated. Results indicate high intra-domain reliability, with coefficients ranging from .87 (social-emotional development) to .94 (approaches to learning).
Table
2
Results
of "Ready Kindergartners" Assessment
|
Social
and Emotional Development |
62.5 |
|||
|
Pct. "Not Observed" |
Pct. "Beginning" |
Pct. "Practicing" |
Pct. "Performing Independently" |
|
|
Can meet/play with different children |
0.7 |
13.9 |
33.8 |
51.5 |
|
Uses problem-solving skills in social situations |
5.5 |
26.6 |
38.8 |
29.2 |
|
Separates easily from caregiver |
2.1 |
6.0 |
19.3 |
72.6 |
|
Appropriately expresses emotions |
3.0 |
15.4 |
31.8 |
49.8 |
|
Adapts to transitions |
0.9 |
11.9 |
31.2 |
56.0 |
|
Interacts positively with adults |
0.5 |
8.8 |
28.4 |
62.2 |
|
Approaches
to Learning |
60.5 |
|||
|
|
Pct. "Not Observed" |
Pct. "Beginning" |
Pct. "Practicing" |
Pct. "Performing Independently" |
|
Follows simple rules |
0.7 |
15.7 |
33.8 |
49.8 |
|
Persists with self-directed activity |
1.8 |
12.4 |
28.9 |
56.9 |
|
Appears enthusiastic |
1.1 |
10.0 |
30.1 |
58.8 |
|
Uses a variety of problem-solving strategies |
5.2 |
23.6 |
39.0 |
32.2 |
|
Pays attention |
2.8 |
19.8 |
33.1 |
44.3 |
|
Knows how and when to use adults |
1.6 |
16.1 |
34.5 |
47.9 |
|
Initiates activities in the classroom |
3.7 |
16.3 |
33.7 |
46.2 |
|
Is curious |
2.4 |
14.0 |
30.7 |
52.9 |
|
Communication |
80.3 |
|||
|
Pct. "Not Observed" |
Pct. "Beginning" |
Pct. "Practicing" |
Pct. "Performing Independently" |
|
|
Communicates needs |
1.1 |
11.6 |
26.8 |
60.5 |
|
Understands simple directions |
1.0 |
10.9 |
28.1 |
59.9 |
|
Engages in conversation |
1.7 |
12.7 |
28.9 |
56.6 |
|
Cognitive
Development/General Knowledge |
67.6 |
|||
|
Pct. "Not Observed" |
Pct. "Beginning" |
Pct. "Practicing" |
Pct. "Performing Independently" |
|
|
Understands purpose of books |
1.1 |
11.0 |
27.1 |
60.9 |
|
Can recall and explain sequences of events |
7.1 |
18.1 |
34.6 |
40.2 |
|
Recognizes name in print |
1.2 |
8.5 |
20.1 |
70.3 |
|
Uses pencils, crayons, and brushes |
1.2 |
15.4 |
26.8 |
56.7 |
|
Engages in imaginative play |
2.8 |
9.2 |
28.7 |
59.3 |
|
Physical
Health and Well-Being
|
||||
| Demonstrates self-help skills |
Pct.
"Not Observed"
0.4 |
Pct.
"Beginning"
4.3 |
Pct.
"Practicing"
16.6 |
Pct.
"Performing Independently"
78.7 |
| Child's ability to learn appears inhibited by: |
Pct.
"Not Observed"
|
Pct.
"Seldom"
|
Pct.
"Sometimes"
|
Pct.
"Often"
|
|
Illness (missing response: 5.6 pct.) |
83.9
|
11.8
|
3.3
|
1.0
|
|
Fatigue (missing response: 5.2 pct.) |
78.2
|
13.2
|
7.4
|
1.3
|
|
Hunger (missing response: 6.0 pct.) |
85.1
|
12.0
|
2.2
|
0.7
|
|
Emotional issues (missing response: 10.3 pct.) |
75.1
|
9.9
|
9.5
|
5.5
|
Correlations between domain sum scores were also calculated. All such scores were significantly positively correlated, with coefficients ranging from .72 to .87. To further explore the item-level structure of results, a principal-components analysis using varimax rotation was applied.4 This analysis was run for two-factor and three-factor solutions, respectively. The three-factor solution, accounting for 65.1% of the variance, was preferred. Factor-loadings by item are noted in Table 3. Results suggest that these "readiness" competencies are highly interrelated, which is consistent with what we know of development in young children.
Table
3
Rotated
Factor Matrix for the "Ready Kindergartners" Questionnaire
Rotated Factor
Matrix*
|
Factor
|
|||
|
1
|
2
|
3
|
|
| Can meet/play w/children of own age |
.639
|
.397
|
.295
|
| Uses problem-solving skills in social dilemmas with peers |
.651
|
.386
|
.308
|
| Separates easily from caregiver |
.353
|
.201
|
.162
|
| Appropriately expresses range of emotions |
.618
|
.413
|
.232
|
| Adapts to transitions within school day |
.486
|
.640
|
.255
|
| Interacts positively with adults in schoolroom |
.525
|
.563
|
.224
|
| Follows simple rules/instructions |
.289
|
.792
|
.317
|
| Persists with self-selected activity (15 mins) |
.386
|
.611
|
.429
|
| Apprears enthusiastic/interested in class activities |
.577
|
.479
|
.351
|
| Uses variety of strategies to problem solve in class |
.614
|
.436
|
.365
|
| Pays attention during teacher-directed group activities |
.342
|
.676
|
.396
|
| Knows how/when to use adults as resource |
.579
|
.510
|
.348
|
| Initiates activities in the classroom |
.666
|
.258
|
.411
|
| Is curious (asks questions, probes, tries new things) |
.684
|
.245
|
.438
|
| Communicates needs/wants/thoughts in primary language |
.624
|
.330
|
.388
|
| Understands simple directions/requests and information |
.452
|
.502
|
.470
|
| Engages in conversation (complete sentences, turn taking) |
.619
|
.350
|
.442
|
| Understands purpose of books |
.357
|
.262
|
.652
|
| Can recall activity and explain sequences of events |
.408
|
.225
|
.607
|
| Recognizes own name in print |
.204
|
.315
|
.679
|
| Uses pencils, crayons, brushes to express ideas |
.287
|
.334
|
.713
|
| Engages in imaginative play |
.483
|
.192
|
.535
|
| *Note: Rotation converged in 12 iterations. Extraction Method: Principal Axis Factoring. Rotation Method: Varimax with Kaiser Normalization. | |||
It is noteworthy that on each of the individual competencies rated by teachers, at least three-quarters of children were either "practicing" or "performing independently." This result suggests face validity of the instrument as a measure of what beginning kindergartners can reasonably be expected to know and do.
Forty-eight percent of children in the sample were rated as "practicing" or "performing independently" on all items within all domains. Thirteen percent did not meet this standard in a single domain; 12% did not meet the standard in two domains and three domains, respectively; and 14% did not meet this standard in any of the four domains.
There were eight items where less than 50% of kindergartners were rated as "performing independently":
- uses problem-solving skills in social situations
- appropriately expresses emotions
- follows simple rules
- uses a variety of problem-solving strategies
- pays attention
- knows how and when to use adults
- initiates activities in the classroom
- can recall and explain sequences of events
Item 24 asked teachers to consider the extent to which a child's learning "appears to be inhibited by" illness, fatigue, hunger, and emotional issues, respectively. Although higher nonresponse rates (5%-10%) for this item suggest cautious interpretation, it is noteworthy that 4% of children were identified by teachers as having illness as a barrier to learning "often" or "sometimes"; fatigue, 9%; hunger, 3%; and emotional issues, 15%.5
Results of "Ready Schools" Assessment
Table 4 shows the items making up each domain, together with the preferred responses contributing to the domain "standard" (component item weights available from the authors on request), and the percentage of supervisory unions (based on participating teachers and principals) providing each response.
Table
4
Results
of "Ready Schools" Assessment
|
Smooth
Transitions to School |
81.0 |
|
|
Respondents (N) |
Pct. with preferred response |
|
|
The following activities are offered before school entry |
Principals |
|
|
Move-up days |
(197) |
53.3 |
|
Welcome notes sent to all kindergartners |
(197) |
73.6 |
|
Registration day |
(197) |
78.7 |
|
Practice bus ride |
(197) |
43.7 |
|
Information packets describing KG distributed to parents |
(197) |
73.6 |
|
The following activities are offered before school entry or within the first month of school: |
Principals |
|
|
Teacher visits to preschool/child care/parent child centers |
(197) |
64.0 |
|
Kindergarten screening |
(197) |
88.3 |
|
Home visits to each new student |
(197) |
28.4 |
|
Kindergarten open house |
(197) |
90.4 |
|
Telephone calls to all kindergarten parents |
(197) |
48.7 |
|
Classroom visits |
(197) |
67.5 |
|
Parent/child/teacher conferences |
(197) |
54.8 |
|
Questionnaires sent to all kindergartners and their parents |
(197) |
58.4 |
|
Instruction
and Staff Development |
67.5 |
|
|
Respondents (N) |
Pct. with preferred response |
|
|
Kindergarten teacher's state endorsement/license |
Teachers |
|
|
Elementary Education |
(181) |
93.4 |
|
Early Childhood Education |
(181) |
31.5 |
|
Average kindergarten class size < 16 |
Principals |
76.2 |
|
Kindergarten instructional practices are derived from |
Principals |
|
|
Teacher observations |
(197) |
86.3 |
|
Statewide standards |
(197) |
94.4 |
|
School district curriculum |
(197) |
94.4 |
|
Standard testing/outcome data |
(197) |
39.6 |
|
Professional standards (e.g., NAEYC) |
(197) |
44.2 |
|
Parent input |
(197) |
55.8 |
|
Preschool teachers |
(197) |
49.7 |
|
Teacher's own resources |
(197) |
92.4 |
|
Community/parent group |
(197) |
13.2 |
|
Regional education resource center |
(197) |
5.1 |
|
Partnership
with Community |
71.3 |
|
|
Respondents (N) |
Pct. with preferred response |
|
|
School's action-planning process addresses issues of: |
Principals |
|
|
Pre-kindergarten |
(173) |
34.7 |
|
Kindergarten |
(184) |
71.7 |
|
School-sponsored activities with at least one-third of parents participating: |
Principals |
|
|
Open houses |
(196) |
98.0 |
|
Parent-teacher conferences |
(194) |
100 |
|
Family "fun" activities (fairs, dinners, dances, etc.) |
(186) |
80.7 |
|
PTA/PTO |
(188) |
31.4 |
|
Community-based activities (school-sponsored or co-sponsored): |
Principals |
|
|
Recreational programs |
(197) |
64.5 |
|
Parent education |
(197) |
61.4 |
|
Family literacy activities |
(197) |
68.5 |
|
After-school care |
(197) |
38.1 |
|
Before-school care |
(197) |
13.7 |
|
Summer/vacation/enrichment programs |
(197) |
73.6 |
|
Overall level of kindergarten parent involvement (e.g., classroom volunteers, participation on committees, help with special projects) is at least one-third |
Principals |
66.7
|
|
Resources |
94.1 |
|
|
Respondents (N) |
Pct. with preferred response |
|
|
Types of support available (may require a wait) |
Teachers |
|
|
Professional support |
||
|
Colleagues |
(175) |
99.4 |
|
Principal |
(174) |
99.4 |
|
Parents |
(173) |
99.4 |
|
Specialized services |
||