Evaluation of Engineering a Better World

Introduction

Concern about the numbers of young people opting for careers in engineering led the Engineering and Physical Sciences Research Council (EPSRC) to explore ways of providing positive engineering influences to students, with the ultimate aim of encouraging greater uptake of engineering-related study and work.  Engineering a Better World (EaBW) was a three-year science education intervention, funded by the EPSRC and Yorkshire Forward, which was delivered to students aged 11 to 16 in selected Yorkshire secondary schools between 2004 and 2008.  The project aimed to:

  • improve student perceptions of engineering and related subjects, and knowledge of future career options; and to
  • understand the factors that contributed to, and impacted on, student perceptions, and the extent to which the intervention delivered an improvement in perceptions.

EaBW was managed by the Centre for Science Education (CSE) at Sheffield Hallam University and was developed and delivered by the four Yorkshire and Humberside SETpoints working with schools.  This ‘cascade’ delivery model aimed to maximize impact from the project by using existing materials and delivery channels.

Activities

The intervention in schools was based around ‘case studies’, projects which consisted of an engaging, engineering-type activity such as building a robot, a solar-powered buggy, or creating a product using smart materials. The ‘case studies’ included elements combining science, mathematics, and technology, and could be taught in the curriculum or as extracurricular activities, within single subject areas or across different departments. The ‘case study’ experience typically involved participation in a celebration event, which sometimes took the form of a competition between students from different schools.

Although the project took place between 2004 and 2008, schools initially delivered the projects in the pilot form to small groups of students.  The number of students involved increased from small ‘pilot’ groups in the first year to approximately 2000 in the second year and over 4000 in the third year.  Few students who took part experienced more than one case study, as teachers tended to repeat case studies with a fresh group of students each year.

Evaluation Methodology

Before and after taking part in the EaBW case studies, students were asked to complete questionnaires which recorded their perceptions of, and attitudes to science, technology, engineering, and maths.  A third questionnaire answered several months after the case study measured whether any changes in attitude were lasting.  Future study and career plans were also captured.  A ‘benchmark’ survey was also conducted to allow comparison of EaBW students with their peers across England.

Questionnaires from each stage were matched for each student, enabling an analysis of change at an individual level, and multi-variate analysis was conducted to identify factors associated with changes in perceptions.

Evidence from the questionnaires was examined further during focus group interviews with students, which explored aspects of the case studies that affected student perceptions of science, technology, engineering and maths (STEM) and career choices.  In-depth interviews with teachers revealed their views on the effectiveness of the case studies as teaching tools and in affecting student perceptions and their views about the way in which the project was delivered in schools.

Main Findings

Baseline views

Students were generally positive in their attitudes toward science and technology in society.  However, whilst there was little difference in attitudes to science between male and female students, male students tended to be more positive about technology and engineering.

Most students who participated in EaBW already had positive images of scientists and engineers.  Despite this, there is evidence to suggest that a majority of students feel that careers in science and engineering are ‘not for them’.  This is particularly the case for female students who were less likely to have positive images of engineering and engineers.

A significant minority of students wanted to pursue the study of STEM subjects at post-16 level, although fewer wanted jobs involving those subjects.  Nevertheless, approximately a quarter of students aspired to professional occupations in fields broadly related to science and technology, such as medicine, design, engineering, and science.

The national benchmark survey asked some of the same questions of a representative sample of young people across England, and the results suggest that the attitudes of EaBW students are very similar to those of their peers across the country.

Impacts on students

Enjoyment of the case study was the main factor related to impact.  Specifically, those who enjoyed the case study a lot tended to improve in their perceptions of STEM, whilst those who didn’t enjoy the case study were more negative afterward.  However, attitudes reverted towards their original positions in the longer term.

Views about science and engineering among those who enjoyed the case studies tended to be more positive than average before participation, whilst the opposite is true of those who didn’t enjoy participation.  Therefore the case studies appealed most to those already interested in science and engineering and helped to reinforce this interest in the short term.

The eleven case studies each had distinctive characteristics.  The extent to which students enjoyed the case studies reflected their pre-existing interests.  Some case studies such as the mini-robot appealed more to male students, whilst some such as textile engineering were more engaging to female students.

Elements of the case studies that distinguished the projects from their usual classroom experiences were widely enjoyed by students, including making a product by working in teams and taking part in celebration events.

In some schools there little or no careers content in the case studies when they were delivered in the classroom.  In those schools, the project had a negligible impact on students’ knowledge and perceptions of science and engineering careers.

Impacts on teachers

Teachers played a central role in EaBW.  They were not merely the gatekeepers allowing activities into schools, they were the principal deliverers of EaBW activities and a number also had central roles in the development of activities.

Teachers took part in EaBW because the case studies met teaching needs by fulfilling national curriculum requirements, whilst also providing activities that developed team working and problem-solving skills.  Teachers recognized that students were often enthused by the experience.  Other benefits for teachers included access to equipment and funding that they would not otherwise have had, the support of the SETpoint and teachers from other schools, and the opportunity to develop latent ideas into a case study.

Teachers who had already developed ways of promoting awareness of science and engineering careers through lessons used the case studies to this end.  Some students were influenced by positive experiences of the case studies and were able to make associations between their prior interest in science and maths, and engineering and STEM careers.  Nevertheless, teachers felt that long term attitude change would require a longer-term commitment to the promotion of engineering and science than was possible within the duration of EaBW, through which most participants experienced only one case study.

Moreover, many teachers did not promote science and engineering careers, and some teachers did not think of the case studies as engineering activities.

Conclusions

The baseline surveys showed that a large proportion of young people have positive attitudes towards science, technology, engineering and maths.  Projects such as EaBW endeavor to harness and develop this enthusiasm.

Enjoyment of the case studies was associated with a positive change in attitudes to science and engineering, and the students who enjoyed the EaBW case studies tended to be more positive than average before taking part.  By appealing to those who already had an interest in science and engineering, EaBW provided a positive influence for those who may consider STEM careers in the future.  However, the effect of the project on attitudes was short term, and such experiences may need to be reinforced over a longer duration to provide a long term effect.

The case studies were used effectively by some teachers to broaden their students’ knowledge of STEM careers in a positive way.  However, the teaching of the case studies often did not include any careers content.  This demonstrates a disadvantage of the cascade implementation model.  The chain of implementation from the project managers at the CSE, through the SETpoints to the schools, allowed dilution of the informational ‘message’ of the case studies, and therefore the project demonstrated a limited effect on student perceptions.

Engineering a Better World was funded by the Engineering and Physical Sciences Research Council (EPSRC) and Yorkshire Forward, and was delivered by the Centre for Science Education at Sheffield Hallam University, and the Yorkshire SETPoints.

Posted on

Leave a Reply

Your email address will not be published. Required fields are marked *