Beyond the numbers

Reasoning on Quality Education gradually depends upon evaluative processes that make data available for educational stakeholders and decision makers. This approach typically has high stakes accountability at its core, putting students in the forefront of the evaluative process. Nevertheless, recognizing students’ performance as the main impact assessment target of the Education System may be a failure to recall the community of people and the myriad of processes that (in)form quality both as a comprehensive construct and output.

This vision involves a challenging understanding contexts that places evaluation as a participatory process, where subjectivity must be insightfully recognized and accurately addressed. Its mitigation may expose a shortcoming in dealing with complexity, based on the fallacy that there is “just a single reality” to be disclosed. In fact, as Antonio Bolivar states: for the purpose of deep improvement, the practices of evaluation are expected to be rooted on processes that are shared among different actors of the educational institutions.

Some factors seem to explain the importance that assessment practices assume nowadays in the field of Education. As Papadopoulos, José Augusto Pacheco, Isabel Fialho, Sergio Carrera or Florian Geyer show on their studies, among those explanations we may find:

• The growth in volume and diversity of public responsibilities in education, with the increasing of decision levels and actors involved in the production and management of policies;
• The decentralisation of those policies associated to globalisation pressures to improve the quality of education;
• The need to explain the use given to educational resources;
• The principle of economic efficiency published by international prized reports (e.g., PISA, TIMMS) comparing the results of different countries.

Digital Literacy Programme

In this context, when we put Technology in the equation, most common assessment practices tend to explore the relationship between the devices and software in use and students’ levels of motivation and performance. Nevertheless, at jp.ik, we understand that when technologies are meaningfully integrated into systems there is such a transformational potential that all participants are agents and targets of change. That is why we have designed an ik.impact Service, considering students, teachers and the larger educational community. The main purpose of this Service is to gather data from different stakeholders of a school system, prior and during a pedagogy-driven EdTech Project.

This information is expected to support decisions on improvement based on the way meaning becomes possible for different people, within their very specific contexts and their own needs. In fact, impact assessment of meaningful education transformation processes, and the evaluation of outcomes in different education stakeholders, requires a collection of sensitivities, positions, and attitudes of the most varied agents involved in the process.

If some elements of the evaluation process call our attention to dimensions easily expressed in numbers, namely the ones related to EdTech usability, other aspects of the project can only be assessed through an apprehension, comprehension and reflection around participants’ subjectivities. In order to understand the extent of the mechanisms that are responsible for quality in education, and considering critical indicators of EdTech integrative models, ik.impact involves both quantitative and qualitative techniques and favours ethnographic and visual methods. They follow a comprehensive paradigm of data collection that is seen as necessarily participated by the people and institutions involved in the projects whose impacts are sought to be measured.

This inevitably implies going to the field and establishing face-to-face contacts, higly committed with the privacy of all that is being shared, through a process of empowerment and agency given to those who collectively construct and share their narratives, Besides the response to this triangulated assessment paradigm, the instruments of our ik.impact still carry another differentiating aspect: they seek to thoroughly measure very specific dimensions, instead of larger social, economic, performance outputs.

In what concerns teachers as a target of a pedagogy-driven EdTech project, our assessment toolkits try to grasp the effectiveness of jp.ik training on teachers’ daily action, considering their global practices towards greater learning outcomes, the consolidation of their professional identity, and their EdTech literacy to deal and promote learning through meaningful learning resources. In what refers to students, our assessment toolkits focus onspecific developmental areas. These areas comprise a set of lifelong learning skills that are relevant not only in terms of academic and professional opportunities, but also for a full citizenship in a contemporary globalized world.

Considering impact on larger educational community, we propose to explore some measures of family engagement with schools and the synergies between those schools and community public and private sector institutions, in terms of opportunities reciprocally generated.

In a context-based strategy of services, our impact assessment aims at supporting Education Projects throughout its implementation stages, since the very beginning. More than identifying the main outcomes as a desired destination, it allows for a wider perspective and higher control over the various steps of the project rollout.

In short, a qualitative approach refers to a framework that requires the most up-to-date knowledge of evaluation paradigms. A Service that puts this kind of methodologies at its heart performs a differentiated intervention in the field of the EdTech market and definitively makes the intervention clearer, more credible and more robust, thus leveraging sustainability from a social responsibility core.