Volume 8 Number 1
©The Author(s) 2006
Does Early Childhood Intervention Affect the Social and Emotional Development of Participants?
The current study explored the association between a large-scale federally funded preschool intervention and the social and emotional development of participants. Data were drawn from the Chicago Longitudinal Study (CLS) and included 1,378 primarily African American youth who participated in the CLS and had scores for two or more identifiable social and emotional competency indicators from age 7 through age 15. Findings suggest that program participation was associated with both shorter- and longer-term social and emotional outcomes. The effect sizes for the longer term were modest, and several remained above the level considered practically significant (.20). The strongest short-term effect was seen for social adjustment in school at ages 7 and 8-9 with d's of .45 and .33, respectively. These include social adjustment in school (d = .34), assertive social skills (d = .21), task orientation (d = .21), frustration tolerance (d = .22), and peer social skills (d = .24).
The 2002 National Survey of America's Families verified that 82% of 3- and 4-year-olds with employed mothers spend part of their day in nonparental care (Barnett, Robin, Hustedt, & Schulman, 2004). Of these children, two-thirds attend some form of preschool intervention (Belsie, 2002).
As a result of the large proportion of children in nonparental early childhood settings, policy makers and the public have a strong interest in ensuring that early childhood interventions are devised on a "results-based accountability" paradigm that not only mandates that intervention programs be successful, but cost-effective. Because of this, most research studies of early childhood intervention programs have focused on clearly identifiable outcomes such as language development, prereading skills, letter knowledge, and numeracy (Reynolds, 2000; Schultz, 2000).
Policy makers and practitioners alike have used this child-focused research base as their primary source of evidence for assessing the efficacy of early childhood programs (Niles, Reynolds, Ou, & Lee, 2003; Niles, 2004; Schultz, 2000). This is true despite the fact that early childhood intervention programs also can contribute in important ways to the mental health of children by enhancing the social and emotional development of the child (Niles, Reynolds, Ou, & Lee, 2003; Niles, 2004; Reynolds, 2000; Schultz, 2000).
Although research is limited on the influence that early childhood programs have on the social and emotional development of preschool-age children, an important exception is the Family and Child Experiences Survey (FACES) study that is currently being conducted by the U.S. Department of Health and Human Services. While the outcome measures were not identical, the constructs of social and emotional development in the presented study are consistent with the social and emotional domains measured in the FACES study. These include early social skills, shyness, aggressiveness, and hyperactive behaviors, among others (Zill et al., 2003).
The limited research on early childhood programs takes on additional importance because it has been suggested that 10% to 13% of preschoolers (ages 1 to 6 years old) have diagnosed emotional or behavioral disorders (Institute of Medicine, 2001). Moreover, a large study of a pediatric sample of more than 3,800 preschool-age children found that 21% met the criteria for a psychiatric disorder, 9% of them for a severe disorder (Lavigne et al., 1996).
Given that a significant amount of recent empirical evidence suggests that early childhood intervention has a major impact on the cognitive and academic success of participants (see Barnett, 1998; Currie, 2001; Karoly et al., 1998; Reynolds, 2000; van Ijzendoorn, 1998), this study focuses on a less investigated, but equally important, topic: the impact of early childhood intervention on children's social and emotional development. Two questions are examined:
- Is participation in the Chicago Child-Parent Center (CPC) preschool program associated with indicators of children's social and emotional competence, including perceived self-competence, social adjustment in the classroom, psycho-emotional adjustment, and peer relationships?
- Do the links between CPC participation and social and emotional competence persist over time?
To be consistent with the terminology used by the Chicago Public Schools, the terms intervention, program, and services are used interchangeably (Reynolds, 2000). Data were drawn from the Chicago Longitudinal Study (Chicago Longitudinal Study, 1999; Reynolds, 1991, 1998, 2000). The original sample of 1,539 in the CLS included the entire cohort of 989 children who attended the 20 Child-Parent Centers in preschool and kindergarten in 1985-1986 and 550 children of the same age who participated in an alternative all-day kindergarten program in 5 different Chicago public schools in similar neighborhoods. These schools were randomly selected from 27 sites participating in the Chicago Effective Schools Project (CESP—an intervention that offered all-day kindergarten among other services). As a consequence of living in school neighborhoods eligible for Title I funding, all children in this cohort were eligible for and participated in government-funded early childhood programs.
The study sample consisted of 1,378 primarily African American youth who participated in the CLS and had scores for two or more identifiable social and emotional competency indicators by age 15 (Niles, Reynolds, Ou, & Lee, 2003; Niles, 2004). Eighty-nine percent of the original sample met this score criterion.
Table 1 provides descriptive statistics on group comparability for the original sample and the study sample. Consistent with previous analyses, this table illustrates that the program and comparison groups are similar on background characteristics, including gender, race/ethnicity, and eligibility for free/reduced lunch (see Reynolds, 2000; Reynolds, Temple, Robertson, & Mann, 2001). A few exceptions are worth noting. Youth who participated in the CPC program had a greater number of parents who were high school graduates (66.5% vs. 59.3%) and fewer children per household (2.2 versus 2.4). CPC families, however, did have higher levels of household poverty (.777 vs. .729). CPC and comparison group families had the same number of risk factors (2.9 vs. 2.8).
Following the method of Jurs and Glass (1971), two-way analysis of variance for kindergarten cognitive abilities (for which all children had valid scores) revealed that there was no group by attrition interaction, F(1, 1531) = .201, p = .65.
|Indicator||CPC Group||Non-CPC Group
n = 483
|Percent of samples with 2 or more SEC indicators||90.5||87.8||.100|
|Percent High School completion for parent through child (8)||66.5||59.3||.016|
|Percent single parent by child age 8||70.2||66.9||.276|
|Percent parent was teen at child's birth||23.2||19.2||.158|
|Percent parent unemployment by age 8||64.9||60.8||.202|
|Percent ever reported receiving free lunch by age 8||92.3||92.8||.768|
|Percent child/neglect report by age 4||.016||.028||.135|
|Percent income level is 60% + poverty for school area||.777||.729||.048|
|Percent missing data from parent education or free lunch||22.5||25.4||.241|
|Family risk index (0-6)||2.95||2.83||.176|
The Chicago Child-Parent Center Program
The CPC program is a center-based early childhood intervention that provides educational and family-support services to children during the ages of 3 to 9 (preschool to third grade) (Chicago Public Schools, 1974, 1985, 1987; Reynolds, 2000). Children can receive up to 6 years of a comprehensive language-based intervention—1-2 years of preschool, kindergarten, and up to 3 years extended services in elementary school. Located in the poorest neighborhoods in Chicago, the centers serve 100 to 150 3- to 5-year-olds in separate facilities or in wings of neighborhood schools. Each center is directed by a head teacher and two coordinators—the parent-resource teacher and the school-community representative.
The curriculum philosophy of the Child-Parent Centers has consistently emphasized the acquisition of basic knowledge and skills in language arts and math through a relatively structured but diverse set of learning experiences (e.g., whole-class, small-group, child-focused activities, field trips) (Reynolds, 2000).
Although social and emotional development was not explicitly targeted in the curriculum, these affective outcomes were built into the reading and language-based instructional activities (Reynolds, 2000). For example, teachers provided developmentally appropriate feedback and positive reinforcement, and they emphasized task accomplishment (Reynolds, 2000). However, the foundational skills of recognizing letters and numbers, oral communication, listening, and an appreciation for reading and drawing were of primary importance (Reynolds, 2000).
The parent-resource teacher implements the family-support component. The school-community representative provides outreach services to families, including resource mobilization, home visitation, and enrollment of children. Ongoing staff development and health and nutrition services also are provided, including health screening, speech therapy, and nursing and meal services (see Reynolds, 2000; Sullivan, 1970).The child-to-staff ratio is limited to 17 to 2 in preschool and 25 to 2 in kindergarten, although parent volunteers reduce these numbers further. After full-day or part-day kindergarten, continuing services are provided in the affiliated schools under the direction of the curriculum parent-resource teacher. Figure 1 illustrates the organizational structure of the CPC program.
Curriculum Parent-Resource Teacher
Parent Resource Teacher
Parent Room Activities
Small Class SizesInservice Training
Free + Reduced-Price Meals
Parent Room Activities
Reduced Class Size
Free + Reduced- Price Meals
|Figure 1. Child-Parent Center Program.|
This study divided measures of social and emotional development into shorter- and longer-term outcomes. Shorter-term outcomes were defined as being from ages 7-10 and included perceptions of self-competence at ages 9 and 9-10 and social adjustment in school at ages 7 and 8-9.
The longer-term outcomes are based on scores from age 10 through age 15 and include measures of perceived self-competence (ages 11-12), social adjustment (ages 11-12), assertive social skills (ages 12-13), task orientation (ages 12-13), acting-out behaviors (ages 12-13), frustration tolerance (ages 12-13), shyness/anxiety (ages 12-13), peer relations (ages 12-13), total competency (ages 12-13), and total problems at ages 12-13. Also included in the longer-term outcomes is any special education placement for emotional or behavioral disorder (EBD) through age 15. These outcome measures are detailed by domain below.
Perceived Self-Competence. The scales used are similar to Harter's (1982) dimension of cognitive competence in the Perceived Competence Scale for Children (Reynolds, Mehana, & Temple, 1995; Reynolds, 2000). Both scales measure self-concept, including self-efficacy (Reynolds, Mehana, & Temple, 1995; Reynolds, 2000). Previous studies in the CLS have indicated that perceived self-competency has predictive validity for later school achievement and is associated with program participation (Reynolds, Mehana, & Temple, 1995).
Children completed a 33-item survey questionnaire about self-perceptions of school progress, family support, and school environment (Reynolds, 2000; Reynolds, Mehana, & Temple, 1995). These scales were administered by classroom teachers in third- through sixth-grade years as part of a larger survey of school experiences. Three scales were used: one at age 9 (10 items), one at ages 9-10 (10 items), and one at ages 11-12 (12 items). Summed z-scores of the item responses were analyzed. Z-scores were used because the age 9 and 9-10 self-competency scales had 10 items, while the age 11-12 measures had 12 items. For perceived self-competency at ages 9-10, those with a valid score on both ages 9 and 10 on the perceived self-competency scales were averaged. Those with a valid score for age 10 only were included in the analysis. Those that did not have a valid score for age 10 were left missing and not included in the analysis.
The items in the perceived self-competency scale at age 9 were coded 1-3, with 1 being "strongly disagree" and 3 being "strongly agree." The items included the following: I get good grades in school; my friends like me; I get in trouble at school; I get along well with others; I answer questions in class; I give up when school work gets hard; I try hard in school; I do my homework; I am smart; I do better in school than my classmates. Internal consistency reliability for the scale was .74.
The items in the perceived self-competency scale at ages 9-10 and 11-12 were coded 1-4, with 1 being "strongly disagree" and 4 being "strongly agree." The items included the following: I get good grades; my classmates like me; I get in trouble at school; I get along well with others; I do my homework; I answer questions in class; I give up when school work gets too hard; when I get bad grades, I try even harder; I try hard in school; my teacher thinks I will go far in school; I am smart; I do better in school than my classmates. The ages 9-10 perceived self-competency had an internal consistency reliability of .81.
Consistent with developmental theory, some measures of perceived self-competency were different between the scales, including additional measures of task persistence and self-efficacy. For example, the items "when I get bad grades, I try even harder" and "my teacher thinks I will go far in school" were added to the age 11-12 scales. The ages 11-12 perceived self-competency had an internal consistency reliability of .78.
Social Adjustment in School. Social adjustment in school was measured by teacher ratings from ages 7 to 12. The scale was administered yearly from grade 1 to grade 6. For social adjustment at age 7, only those with a valid score on the age 7 scale were included. Those with a valid score for ages 8 and 9 and ages 11 and 12 on the social adjustment scales were averaged. Those that did not have a valid score for the social adjustment measures were left missing and not included in the analysis. The social adjustment scale is the sum of six items rated from poor (1) to excellent (5).
Items for the age 7 social adjustment measure include [the child] came to my class ready to learn, completes work according to instructions, complies with classroom rules, displays confidence in approaching learning tasks, participates in group discussions, works and plays well with others. Internal consistency reliability for the scale was .92.
Items for ages 8-9 and 11-12 social adjustment measures include [the child] concentrates on work, follows directions, is self-confident, participates in group discussions, interacts well with others, takes responsibility for actions. Internal consistency reliability for the scales was .92 and .90, respectively. It is notable that social adjustment in school has been found to have significant predictive validity for later juvenile delinquency (Mann, 2003).
The early adolescent (ages 12-13) measures of social and emotional competence are drawn from subscales of the Teacher-Child Rating Scale (T-CRS). The T-CRS (Hightower, Spinell, & Lotyczewski, 1989) includes 20 positive social competence items (rated from 1, "not at all," to 5, "very well") and 18 problem areas rated from 1, "not at all," to 5, "very well." These include overall competency, assertive social skills, task orientation, peer social skills, overall problems, acting-out behaviors, frustration tolerance, and shyness/anxiety. The T-CRS has alpha and test-retest reliability within .87-.94. Evidence from various construct and predictive validity studies supports the T-CRS as a measure of early adolescent socioemotional adjustment (see Perkins & Hightower, 2002).
To address missing scores and increase sample size, this study combined valid age 12 and age 13 scores on the T-CRS scales. For example, if a grade 7 (age 13) score was missing, grade 6 (age 12) was substituted. If there were valid scores on both age 12 and age 13 measures, an average was computed.
Total Competency. To test robustness of both social and emotional competence, two additional T-CRS scales were used. The first was a composite total competence scale, and the second was a composite of the total problems scale. Twenty items load on the total competence scale, derived from the subscales of assertive social skills, task orientation, and peer social skills. Reliability for the total competency scale was .91.
Assertive Social Skills. The assertive social skills measure consists of 5 items and includes whether or not the child defends own views under group pressure, is comfortable as leader, participates in class discussions, expresses ideas willingly, and questions rules that seem unfair or unclear. Reliability was .87.
Task Orientation. Task orientation consists of 5 items and includes measures such as [the child] completes work, is well organized, functions well even with distractions, works well without supervision, and is a self-starter. The scale had a reliability of .93.
Peer Social Skills. Peer social skills were measured based on the T-CRS subscale of peer relations at age 12-13. The peer social skills scale is the sum of five items rated from poor (1) to excellent (5). Items were has many friends, is friendly toward peers, makes friends easily, classmates will sit near this child, and [the child] is well liked by classmates. The reliability of the peer social skills scale was .93.
Total Problems. There are 18 items on the total problems scale, including acting-out behaviors, frustration tolerance, shyness/anxiety, and learning problems. The learning problems item was not tested in the study because it is more closely related to academic achievement than social and emotional domains. Reliability was .89.
Acting-out Behaviors. The acting-out behaviors measure consists of 6 items and includes the child being disruptive in class, is fidgety, has difficulty sitting still, disturbs others while they are working, constantly seeks attention, is overly aggressive to peers (fights), is deviant, obstinate, stubborn. Reliability was .94.
Frustration Tolerance. Frustration tolerance consists of 5 items that include the child accepts things not going his/her way, ignores teasing, accepts imposed limits, copes well with failure, and tolerates frustration. The frustration tolerance scale had a reliability of .92.
Shyness/Anxiety. Shyness/anxiety consists of 6 items, which include the child is withdrawn, shy or timid, anxious or worried, nervous, frightened, tense, does not express feelings, unhappy or sad. Reliability was .84.
Emotional or Behavior Disorder (EBD) Special Education Services. Those children experiencing a severe emotional disorder were measured by using a dichotomous indicator of placement in emotional or behavior disorder special education services from ages 7-15. Table 2 provides the descriptive statistics for the outcome measures. This measure, derived from administrative records of the Chicago Public Schools, had a reliability of .94.
|Indicators||Variable Name||Sample Size||Minimum||Maximum||Mean||Std. Deviation|
|Perceived competence (age 9)*||Sfact3z||1119||-3.45||1.92||.002||1.00|
|Perceived competence (ages 9-10)*||Sfact6z||1085||-3.35||1.78||.001||1.00|
|Social adjustment (age 7)||Semat1||1113||6||30||19.03||6.12|
|Social adjustment (ages 8-9)||Semat23||1081||6||30||18.98||5.45|
|Perceived competence (ages 11-12)*||Sfact78z||1046||-3.28||2.49||-.004||1.00|
|Social adjustment (ages 11-12)||Semat78||1081||6||30||18.88||5.49|
|Assertive social skills (ages 12-13)||Askill67||1006||5||25||15.56||4.35|
|Acting-out behaviors (ages 12-13)||Actout67||1006||6||30||12.40||6.70|
|Task orientation (ages 12-13)||Taskor67||1006||5||25||14.48||5.19|
|Frustration tolerance (ages 12-13)||Frustol67||1006||5||25||14.38||4.90|
|Shyness/anxiety (ages 12-13)||Shyanx67||1006||6||28||9.98||4.37|
|Peer social skills (ages 12-13)||Peerss67||1006||5||25||16.73||4.10|
|Total competence (ages 12-13)||Tocomp67||1006||20||100||61.18||16.80|
|Total problems (ages 12-13)||Toprob67||1006||18||84||37.20||15.26|
|Any EBD (age 15)||Speebd15d||1352||0||1||.030||.170|
CPC Program Participation. Any preschool participation was coded 1 for children who participated in the CPC preschool component for 1 or 2 years and 0 for children who did not participate. All children who participated in CPC preschool also enrolled in CPC kindergarten. Non-CPC preschool participants enrolled in all-day kindergarten programs at age 5 (either a CESP program or the kindergarten program in the centers).
Covariates. Several sociodemographic indicators collected from school entry to 10th grade served as covariates in this study.
Any Follow-on Participation. This measure was coded 1 for children who participated in the CPC primary-grade intervention component during grades 1 to 3 (in 1986-89) and 0 for children who did not participate but were enrolled in a regular school program. Follow-on participation was open to all children who enrolled in the elementary schools where the CPC program was located.
Gender of Child. Girls were coded 1, and boys were coded 0, as obtained from school records.
Race/Ethnicity of Child. African American children were coded 1, and Hispanic and Caucasian children were coded 0.
Family Risk Index. This multiple risk index (0-6) measures socioeconomic disadvantage. The index provides a cumulative summary of the co-occurrence or a "pile-up" of risk factors that are frequently associated with child and family functioning (Rutter, 1987; Bendersky & Lewis, 1994). The risk indicators were selected based on their well-known associations with child and family well-being (Bendersky & Lewis, 1994).
The index was the sum of six dichotomously coded risk factors measured from family surveys or school records from preschool to age 8 as follows: (1) parent did not complete high school, (2) eligibility for a fully subsidized lunch defined as a family income at or below 130% of the federal poverty line, (3) residence in a school neighborhood in which 60% or more of children are in low-income families, (4) residence in a single-parent family, (5) parent not employed full or part time, and (6) four or more children in family. Neighborhood poverty was obtained from school records. The four family measures came from parent reports on surveys or in telephone interviews when children were age 8. Age 8 risk measures were used because they represent the most proximal indicators of risk as related to the study outcomes.
These risk indicators were chosen because they were the most plausible correlates of both program participation and measured outcomes as judged from earlier studies with these data (Reynolds, 1994, 1995, 2000). The risk index may be a more reliable indicator of preexisting differences between program and comparison groups than any single indicator or a few indicators. Further, as predicted by resilience theory, cumulative or multiple risks have been found to be substantially associated with developmental functioning (Rutter, 1987; Bendersky & Lewis, 1994), and this is better captured with a risk index than with several indicator variables entered as main effects (Reynolds, 2000).
Moreover, because several of the risk indicators were measured in different years (some during or after program participation), the risk index was used to proxy risk status at the time of program entry. This measure has the advantage of providing estimates of program effects that are conservative. To the extent that program participation affects these later measures (e.g., parent education, employment status), estimated program effects will be smaller than would otherwise be expected (Reynolds, 2000).
Child Abuse/Neglect. Children who had an administrative record of any child abuse or neglect (0-4) were coded 1, otherwise 0.
CPC Program Sites. Twenty dichotomous indicators of CPC site participation were used to control for unobserved factors of the 20 unique communities that may influence the results.
Missing Data Variable. This measure was coded 1 if no answer was provided on parent education, marital status, or a question about the number of children in the household. Families who did not answer these questions were assumed to have resided in a high-poverty neighborhood, have less than a high school degree, to be a single parent, or to be unmarried. A parent who answered all background questions received a 0. This variable was included to determine whether there was a difference between groups when a score was imputed for the family. Twenty-two percent of parents did not answer questions in one of the above listed categories and were given a code of 1 for missing data. By estimating the influence of missing data, the sample size does not decrease, thus increasing statistical power while accounting for the influence of imputed data (Cohen & Cohen, 1983).
Regression analysis was used to test the research questions, which included study, child, family, and program variables as covariates. Adjusted means were estimated using Analysis of Covariance. Based on the regression results, effect sizes (ES) were estimated. ES is a standardized, scale-free measure of the relative size of the effect of an intervention in standard deviation units (Rossi, Freeman, & Lipsey, 2004). Effect sizes were calculated using the following formula: control group - mean of treatment group / total sample standard deviation. ES is useful for quantifying effects measured on unfamiliar or arbitrary scales and for comparing the relative sizes of effects from different studies (Rossi, Freeman, & Lipsey, 2004).
Early childhood research literature has suggested that there are two ways to interpret effect sizes—statistical and practical significance. Practically meaningful effect sizes have been defined as d = .20, while a medium effect is d = .50 and a large effect is d = .80 (Cohen, 1988). According to Cohen (1988), an ES of .20 is equivalent to a correlation of .10 between program participation and outcome (Reynolds, 2000; Rosenthal, 1991). Values of .20 or above were interpreted as being practically significant. Because of adequate sample size, significance was set at p < .05, two-tailed.
Perceived Self-Competence. After adjusting for covariates, no group differences were found. Youth who participated in the CPC program had a mean perceived self-competency rate of .003 compared with -.008 in the comparison group (p = .896) at age 9. Those youth who participated in the program had a mean perceived self-competency rate at ages 9-10 of .099 compared with -.009 (p = .469) in the comparison group.
Social Adjustment in School. CPC participants had a significantly higher level of social adjustment in school at age 7 than the comparison group (20.1 vs. 18.6; p < .001). CPC participants maintained this advantage on social adjustment in school at ages 8-9 (19.2 vs. 18.4; p = .038). These findings suggest that children in the CPC preschool group experienced a social advantage that persisted up to 4 years post-program. This finding is consistent with previous studies of children's social and emotional development since the proximal nature of the outcomes to the intervention would be expected to persist in the short term. Table 3 displays the unadjusted and adjusted means for the shorter-term outcomes.
|Model Specification|| Perceived
|*Indicates means are reported using z-scores. Model 1 is unadjusted. Model 2 adjusts for CPC follow-on participation, race/ethnicity, risk index, gender, and any child abuse/neglect by age 4. Model 3 adjusts for Model 2 covariates and CPC program sites. Model 4 adjusts for CPC follow-on participation, race/ethnicity, gender, and individual risk measures. Model 5 adjusts for Model 3 covariates and CPC program sites.|
As shown in Tables 3 and 4, it is important to note that no "ceiling or floor" effects were seen for the unadjusted means for either group. For example, the means are in the middle or average range, not consistently too high or too low. This result suggests that the measures can reliably detect difference between groups. Additionally, the correlation coefficients among measures were low to moderate, which suggests that the indicators used in our analysis were measuring distinct concepts.
Perceived Self-Competence Ages 11-12. The youth who participated in the CPC preschool program had a mean perceived self-competency of .063 compared with -.123 in the comparison group (p = .083). This outcome remained nonsignificant across model specifications.
Social Adjustment Ages 11-12. CPC youth had a mean level of social adjustment in school of 19.6 compared with 18.5 in the comparison group (p = .048). This finding is consistent with the shorter-term findings and suggests that CPC participants maintained their advantage over the comparison group over time.
Total Competence Ages 12-13. Using the composite measure of the T-CRS total competency scale, we found that youth who participated in the CPC preschool program demonstrated a trend of higher overall competency compared with the comparison group (62.0 vs. 59.4; p = .089).
Assertive Social Skills Ages 12-13. Those youth who participated in the CPC preschool program also exhibited higher assertive social skills by ages 12-13 compared with the comparison group (15.8 vs. 15.0; p = .086).
Task Orientation Ages 12-13. Youth who participated in the CPC preschool program did not have a significantly higher mean of task orientation by ages 12-13 compared with the comparison group (14.6 vs. 14.0; p = .179).
Peer Social Skills Ages 12-13. Youth who participated in the CPC preschool program also did not have a significantly higher mean of positive peer relations compared with the comparison group (17.0 vs. 16.2; p = .084).
Total Problems Ages 12-13. As with total competency, the total problems composite found no significant difference for those youth who participated in the CPC preschool program compared with the comparison group (36.4 vs. 38.5; p = .124).
Acting-Out Behavior Ages 12-13. Youth who participated in the CPC preschool program had only slightly lower acting-out behaviors than the comparison group (12.1 vs. 12.9, p = .207).
Frustration Tolerance Ages 12-13. While not significant, youth who participated in the CPC preschool program demonstrated a greater ability to tolerate frustration by ages 12-13 compared with the comparison group (14.5 vs. 14.1; p = .381).
Shyness/Anxiety Ages 12-13. Youth who participated in the CPC preschool program also demonstrated less shyness/anxiety by ages 12-13 compared with the comparison group (9.8 vs. 10.2; p = .400), although this result was not statistically significant.
Emotional and Behavioral Disturbance Placement (EBD). Although not significant, the CPC preschool group had lower rates of EBD placement by age 15 compared with the comparison group (.029 vs. .031; p = .869). Table 4 displays the unadjusted and adjusted means for the longer-term outcomes.
|Model||Perceived Competence Ages 11-12*||Social Adjustment Ages 11-12||Assertive Social Skills Ages 12-13||Acting-out Behavior Ages 12-13||Task Orientation Ages 12-13||Frustration Tolerance Ages 12-13||Shyness/ Anxiety Ages 12-13||Peer Social Skills Ages 12-13||Total Comp. Ages 12-13||Total Probs. Ages 12-13||Any EBD Age 15|
*Indicates means are reported using z-scores. Model 1 is unadjusted. Model 2 adjusts for CPC follow-on participation, race/ethnicity, risk index, gender, and any child abuse/neglect by age 4. Model 3 adjusts for Model 2 covariates and CPC program sites. Model 4 adjusts for CPC follow-on participation, race/ethnicity, gender, and individual risk measures. Model 5 adjusts for Model 3 covariates and CPC program sites.
Effect Sizes for Questions 1 and 2
There has been a shift from considering only results of statistical significance testing to the inclusion of measures of practical significance (Levin, 1993). Statistical significance is concerned with whether a research result is due to chance, whereas practical significance is concerned with whether the result is useful in the "real world" (Cohen, 1988; Levin, 1993).
Tables 5 and 6 detail the effect size results for the outcomes in the presented study. Effect size results ranged between .15 and .45, reflecting a modest effect for all short-term social and emotional competency outcomes. Although most effect sizes were moderate, they were largely close to the level of practical significance (.20). The largest ES was in the social adjustment in school by age 7 (d = .45) and social adjustment in school at ages 8-9 (d = .33). This result suggests that the CPC preschool group did almost one-half standard deviation better than the comparison group, respectively. For the longer-term outcomes, effect size results ranged between -.19 (acting-out behaviors) and .34 (social adjustment in school). The coefficients reflect a modest ES for many longer-term social and emotional competency outcomes.
Aside from social adjustment, which had the largest effect sizes (.33 to .45), the overall pattern of findings (most effect sizes are in the .20 range) helps identify meaningful effects that are worthy of policy intervention for social and emotional development. For example, an effect size of .20 (or a correlation of .10) on acting-out behaviors may not be statistically significant (.05 or less), but to the teacher, social worker, or parent, it is at this level that a visible difference can be seen (Levin, 1993). Moreover, the sizes of the effects found in the presented study are within the range found in other programs (Lipsey & Wilson, 1993). Levin (1993) reminds us that statistical significance and practical significance should not be viewed as competing concepts but complementary ones (p. 379, italics in original).
It should also be noted that while effect size analysis is useful for comparing the impact of interventions on outcomes, the Cohen index does not distinguish among the importance of the outcome measures, nor does it address issues of program efficiency or cost effectiveness (Levin, 1993). An additional limitation of ES is that it does not provide information about slope (or rate of change) in the treatment data series (Levin, 1993). For parsimony, the hierarchal regression coefficients can be found in Tables 7 and 8.
|Outcome||CPC Preschool Group||No-Preschool Group||Mean Difference||p-value||Effect Size|
|Perceived competence (age 9)*||.003||-.008||-.005||.896||.16|
|Perceived competence (ages 9-10)*||.099||-.009||.108||.469||.18|
|Social adjustment (age 7)||20.1||18.6||1.5||.001||.45|
|Social adjustment (ages 8-9)||19.2||18.4||.8||.038||.33|
|*Indicates means are reported using z-scores. Reported effect sizes adjust for CPC follow-on participation, race/ethnicity, risk index, gender, any child abuse/neglect by age 4, and CPC sites.|
|Outcome||CPC Preschool Group||No-Preschool Group||Mean Difference||p-value||Effect Size|
|Perceived competence (ages 11-12)*||.063||-.123||.186||.083||.16|
|Social adjustment (ages 11-12)||19.6||18.5||1.1||.048||.34|
|Assertive social skills (ages 12-13)||15.8||15.0||.8||.086||.21|
|Acting-out behaviors (ages 12-13)||12.1||12.9||-.8||.207||-.19|
|Task orientation (ages 12-13)||14.6||14.0||.6||.179||.21|
|Frustration tolerance (ages 12-13)||14.5||14.1||.4||.381||.22|
|Shyness/Anxiety (ages 12-13)||9.8||10.2||-.4||.400||-.17|
|Peer social skills (ages 12-13)||17.0||16.2||.8||.084||.24|
|Total competence (ages 12-13)||62.0||59.4||.6||.089||.19|
|Total problems (ages 12-13)||36.4||38.5||-2.1||.124||.17|
|Any EBD (age 15)||.029||.031||-.002||.869||.16|
|*Indicates means are reported using z-scores. Reported effect sizes adjust for CPC follow-on participation, race/ethnicity, risk index, gender, any child abuse/neglect by age 4, and CPC sites.|
Model 1 (unadjusted)
Model 2 (adjusted)
|Abuse by age 4||-0.404||0.073||0.286||0.813||-0.185||0.513||-0.802||0.407|
|Missing dummy code||0.069||0.258||-0.259||0.705||0.016||0.332||0.118||0.078|
|Abuse by age 4||-0.448||0.047||0.707||0.557||-0.142||0.615||- 0.835||0.385|
|Missing dummy code||0.223||0.089||-0.221||0.303||0.114||0.305||0.225||0.096|
|Assertive Social Skills|
Model 1 (unadjusted)
Model 2 (adjusted)
|Abuse by age 4||-0.170||0.451||1.001||0.423||0.803||0.503||0.354||0.828||1.916||0.117|
|Abuse by age 4||-0.155||0.498||1.114||0.378||1.183||0.325||0.252||0.879||1.947||0.113|
The current study adds to the body of literature on the social and emotional development of a large, low-income, inner-city sample of children that participated in the Chicago CPC program. Specifically, this study offers three unique contributions to existing literature.
First, this study contributed to the field through analysis of the long-term effects of the CPC program on the social and emotional development of participants. The CPC program is a public, large-scale, center-based early intervention that provides comprehensive educational and family-support services to economically disadvantaged children from preschool to early elementary school.
Most of the findings related to the effects of early intervention programs on social and emotional development have come from small-scale studies, such as the High/Scope Perry Preschool Program, the Early Training Project, and the Philadelphia Study. Among these limited studies, most were from model programs with small sample sizes. These small samples provide little statistical power to detect large effects and limit the generalizability and statistical conclusions of the findings. Findings from large-scale programs, such as Head Start, have not been consistent (McKey, Condelli, & Ganson, 1985; U.S. General Accounting Office, 1997), and the quality of most studies has not been sufficient to make findings meaningful.
A second contribution is that both shorter- and longer-term outcomes were examined. By considering children's social and emotional development between the ages of 7 and 12, this study adds to existing literature by suggesting that early childhood programs can, in addition to cognitive advantages, provide a positive social and emotional benefit to participants.
Although immediate positive effects of early intervention on social and emotional competence have been found in a few studies (McKey, Condelli, & Ganson, 1985; Beller, 1983; Gray, Ramsey, & Klaus, 1983), long-term effects have yet to be thoroughly investigated. No studies use such a large sample, to our knowledge, to examine early childhood program participation on social and emotional development through age 15.
Third, social and emotional development was tested using a broader framework than previous studies using the CLS database. The only previous study exploring CPC effects on social and emotional competence considered children's perceived school competence in grade 6 (Reynolds, Mehana, & Temple, 1995). What is more, targeting young children who exhibit early indicators of poor social and emotional development, such as those discussed in this study, is seen as key to prevention efforts and has significant cost implications as well. A recent National Institute of Mental Health sponsored study revealed that poor social and emotional development from ages 0-8 costs over $300 billion annually (Niles, 2004). This includes productivity losses of $150 billion on the part of the parent(s), health care costs of $70 billion, and other costs (e.g., criminal justice) of $80 billion (Niles, 2004). In 2000, our nation's direct service costs and indirect costs from poor childhood mental health totaled more than $313 billion (Niles, 2004). That cost was more than cancer ($180.2 billion in 2000) (Niles, 2004).
Overall, children's participation in a CPC was associated with positive social and emotional competence, especially in the shorter term. Lasting effects of program participation were found through early adolescence on a few outcomes as well. Therefore, it appears that the early effort put forth by teachers and parents can have lasting social and emotional benefits in addition to the initial cognitive enhancement gained in the preschool classroom.
Limitations of the Study
This study has four limitations. First, this study is based upon a quasi-experimental design, and quasi-experimental studies are often limited in internal validity. Although robustness testing on the outcomes using regression analyses was conducted, further testing is necessary to decrease the likelihood that findings may be spurious or an artifact of another threat to internal validity (Cook & Campbell, 1979; Reynolds, 1998).
The comparability of the CPC children and the comparison group children and sample attrition also deserves note. The groups were found to be similar on nearly all characteristics at the beginning of the study, the attrition of the study participants over time is similar in the two groups, and the assignment of participants to the program is largely related to participating families residing in a neighborhood served by an elementary school that includes a CPC program (Reynolds, Temple, Robertson, & Mann, 2001; Reynolds & Temple, 1995). This limitation, however, is somewhat mitigated by the extensive analysis conducted by numerous researchers to identify differences between the two groups (see Reynolds, 2000).
A second limitation relates to the amount of measurement error for the outcome scales. Although found in most scales in social research, measurement error is especially true with psychological tests. However, this limitation is somewhat corrected for since it makes the findings more conservative and suggests that the findings may perhaps be stronger if more reliable measures were used. Although the reliability of the Social and Emotional Maturity Scale and T-CRS scales used in this study ranged from .7 to .9, this remains a limitation because reliability sets a limit on validity.
A third limitation is that this study assessed only some social and emotional indicators that are related to children's mental health. A more comprehensive set of outcomes may have allowed consideration of additional mental health problems, including depression, schizophrenia, and conduct disorders.
A final limitation pertains to the generalizability or external validity of study results. This study specifically explored an urban minority sample that lived in some of the highest poverty areas in the inner city of Chicago, Illinois. This study therefore has a limited generalizability, although there is some evidence that these findings may be replicable because similar findings were evident in other intervention programs such as the Perry Preschool Program and recent evaluations of Head Start (Garces, Thomas, & Currie, 2000).
Barnett, W. Steven. (1998). Long-term effects on cognitive development and school success. In W. Steven Barnett & Sarane Spence Boocock (Eds.), Early care and education for children in poverty: Promises, programs, and long-term results (pp. 11-44). Albany: State University of New York Press.
Barnett, W. Steven; Robin, Kenneth B.; Hustedt, Jason T.; & Schulman, Karen L. (2004). The state of preschool: 2004 state preschool yearbook. New Brunswick, NJ: National Institute for Early Education Research.
Beller, E. Kuno. (1983). The Philadelphia study: The impact of preschool on intellectual and socioemotional development. In Consortium for Longitudinal Studies (Ed.), As the twig is bent: Lasting effects of preschool programs (pp. 333-376). Hillsdale, NJ: Erlbaum.
Belsie, Laurent. (2002, July 9). Preschools are popping at the seams. The Christian Science Monitor, p. 13.
Bendersky, Margaret, & Lewis, Michael. (1994). Environmental risk, biological risk, and developmental outcome. Developmental Psychology, 30(4), 484-494.
Chicago Longitudinal Study. (1999). A study of children in the Chicago public schools: User's guide (Version 6). Madison: University of Wisconsin.
Chicago Public Schools. (1974). Child Parent Centers. Chicago: Author.
Chicago Public Schools. (1985). Elementary school—Criteria for promotion. Chicago: Author.
Chicago Public Schools. (1987). Chapter 2 all-day kindergarten program final evaluation report: Fiscal 1986. Chicago: Department of Research and Evaluation, Chicago Public Schools.
Cohen, Jacob. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Cohen, Jacob, & Cohen, Patricia. (1983). Applied multiple regression/correlation analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.
Cook, Thomas D., & Campbell, Donald T. (1979). Quasi-experimentation: Design and analysis issues for field settings. Boston: Houghton Mifflin.
Currie, Janet. (2001). Early childhood education programs. Journal of Economic Perspectives, 15(2), 213-238.
Garces, Eliana; Thomas, Duncan; & Currie, Janet. (2000, December). Longer term effects of Head Start (Working Paper 8054). Cambridge, MA: National Bureau of Economic Research. Retrieved January 18, 2006, from http://www.nber.org/papers/w8054
Gray, Susan W.; Ramsey, Barbara K.; & Klaus, Rupert A. (1983). The Early Training Project 1962-1980. In Consortium for Longitudinal Studies (Ed.), As the twig is bent: Lasting effects of preschool programs (pp. 33-69). Hillsdale, NJ: Erlbaum.
Harter, Susan. (1982). The perceived competence scale for children. Child Development, 53(1), 87-97.
Hightower, A. Dirk; Spinell, Arline P.; & Lotyczewski, Bohdan S. (1989). Teacher-child rating scale (TCRS) guidelines. New York: Primary Mental Health Project. Unpublished.
Institute of Medicine. Committee on Quality of Health Care in America. (2001). Crossing the quality chasm: A new health system for the 21st century. Washington, DC: National Academy Press.
Jurs, Stephen G., & Glass, Gene V. (1971). The effect of experimental mortality on the internal and external validity of the randomized comparative experiment. Journal of Experimental Education, 40(1), 1, 62-66.
Karoly, Lynn A.; Greenwood, Peter W.; Everingham, Susan S.; Hoube, Jill; Kilburn, M. Rebecca; Rydell, Peter C.; et al. (1998). Investing in our children: What we know and don't know about the costs and benefits of early childhood interventions. Santa Monica, CA: RAND.
Lavigne, John V.; Gibbons, Robert D.; Christoffel, Katherine Kaufer; Arend, Richard; Rosenbaum, Diane; Binns, Helen; et al. (1996). Prevalence rates and correlates of psychiatric disorders among preschool children. Journal of the American Academy of Child and Adolescent Psychiatry, 35(2), 204-214.
Levin, Joel R. (1993). Statistical significance testing from three perspectives. Journal of Experimental Education, 61(4), 378-382.
Lipsey, Mark W., & Wilson David B. (1993). The efficacy of psychological, educational, and behavioral treatment: Confirmation from meta-analysis. American Psychologist, 48(12), 1181-1209.
Mann, Emily A. (2003). Early intervention as delinquency prevention: Evidence from the Chicago Longitudinal Study. Unpublished doctoral dissertation, University of Wisconsin, Madison.
McKey, Ruth H.; Condelli, Larry; & Ganson, Harriet C. (1985). The impact of Head Start on children, families, and communities. Final report on the Head Start Evaluation, Synthesis, and Utilization Project. Washington, DC: U.S. Department of Health and Human Services.
Niles, Michael D. (2004). Participation in early childhood intervention: Does it influence children's social and emotional development? Unpublished doctoral dissertation. University of Wisconsin, Madison.
Niles, Michael D.; Reynolds, Arthur J.; Ou, Suh-Ruu; & Lee, Jung-Eun. (2003, January). Can early intervention affect children's social and emotional competence? Evidence from the Chicago Longitudinal Study. Paper presented at the annual meeting of the Society for Social Work and Research, Washington, DC.
Perkins, P. Elliot, & Hightower, A. Dirk. (2002). T-CRS 2.1 teacher-child rating scale: Examiner's manual. Rochester, NY: Children's Institute.
Reynolds, Arthur J. (1991). Early schooling of children at risk. American Educational Research Journal, 28(2), 392-422.
Reynolds, Arthur J. (1994). Effects of a preschool plus follow-on intervention for children at risk. Developmental Psychology, 30(6), 787-804.
Reynolds, Arthur J. (1995). One year of preschool intervention or two: Does it matter? Early Childhood Research Quarterly, 10(1), 1-31.
Reynolds, Arthur J. (1998). Developing early childhood programs for children and families at risk: Research-based principles to promote long-term effectiveness. Children and Youth Services Review, 20(6), 503-523.
Reynolds, Arthur J. (2000). Success in early intervention: The Chicago Child-Parent Centers. Lincoln: University of Nebraska Press.
Reynolds, Arthur J.; Mehana, Majida; & Temple, Judy A. (1995). Does preschool intervention affect children's perceived competence? Journal of Applied Developmental Psychology, 16(2), 211-230.
Reynolds, Arthur J., & Temple, Judy A. (1995). Quasi-experimental estimates of the effects of a preschool intervention: Psychometric and econometric comparisons. Evaluation Review, 19(4), 347-373.
Reynolds, Arthur J.; Temple, Judy A.; Robertson, Dylan L.; & Mann, Emily A. (2001). Long-term effects of an early childhood intervention on educational achievement and juvenile arrest: A 15-year follow-up of low-income children in public schools. Journal of the American Medical Association, 285(18), 2339-2346.
Rosenthal, Robert. (1991). Meta-analytic procedures for social research. Newbury Park, CA: Sage.
Rossi, Peter H.; Freeman, Howard E.; & Lipsey, Mark W. (2004). Evaluation: A systematic approach (7th ed.). Thousand Oaks, CA: Sage.
Rutter, Michael. (1987). Psychosocial resilience and protective mechanisms. American Journal of Orthopsychiatry, 57(3), 316-331.
Schultz, Sheila R. (2000). Socioemotional development of low-income children in the public school intervention program. Unpublished doctoral dissertation, Virginia Polytechnic Institute and State University, Falls Church, VA.
Sullivan, Lorraine M. (1970). Report on Child-Parent Education Centers. Chicago: Foresman.
U.S. General Accounting Office. (1997). Head Start: Research provides little information on impact of current program: Report to the Chairman, Committee on the Budget, House of Representatives.Washington, DC: U.S. General Accounting Office.
van Ijzendoorn, Marinus H. (1998). Meta-analysis in early childhood education: Progress and problems. In Bernard Spodek, Olivia N. Saracho, & Anthony D. Pellegrini, Yearbook in early childhood education: Vol. 8. Issues in early childhood educational research (pp. 156-176). New York: Teachers College Press.
Zill, Nicholas; Resnick, Gary; Kim, Kwang; O'Donnell, Kevin; Sorongon, Alberto; McKey, Ruth Hubbell; et al. (2003, May). Head Start FACES 2000: A whole-child perspective on program performance: Fourth progress report. Washington, DC: Administration for Children and Families, U.S. Department of Health and Human Services, Head Start Quality Research Consortium's Performance Measure Center.
Michael Niles is assistant professor of social work at Arizona State University, Tempe, Arizona. Research interests of Dr. Niles include early childhood intervention programs and American Indian children, development of social and emotional competence, First Nations Peoples, children's mental health, prevention science, and program evaluation.
Michael D. Niles, M.S.W., Ph.D.
Arizona State University
School of Social Work
P.O. Box 871802
Tempe, AZ 85287-1802
Arthur Reynolds is professor of child development, University of Minnesota, Minneapolis. Research interests of Dr. Reynolds include prevention research and program evaluation, child development, and family and school influences on low-income children.
Arthur J. Reynolds, Professor
Institute of Child Development
University of Minnesota-Twin Cities
Room 202 Child Development
51 E. River Pkwy
Minneapolis, MN 55455
Mark Nagasawa is a doctoral student in the College of Education, Arizona State University, Tempe, Arizona. Research interests of Mr. Nagasawa include early childhood intervention programs and collaborative action between preschool and K-12 educators.
Mark Nagasawa, M.S.W.
Arizona State University
College of Education