How was the memo© developed?
memo© methodology was developed by CHE Consult with regard to state-of-the-art research in psychometrics and findings of a previous project identifying the relevant factors for academic success.
INNOVATIVE APPROACH – PSYCHOMETRIC TESTING
memo© is unique for its innovative methodology going beyond classical perceptional surveys of student satisfaction and introducing a psychometric-related analysis of actual personality traits (“memo© factors”). At the same time the tools integrate also traditional questions and monitor who students are and what they think.
memo© METHODOLOGY
A mixed-methods approach has been used for memo© development. memo© methodology is based on state-of-the-art research findings supplemented by applying a combination of research methods on various samples. The memo© methodology and definition of 10 memo© factors (see below) follows the results of previous studies on employability of graduates, mobility impacts and psychometric aspects, such as:
- The European Parliament study on improving the participation in the Erasmus programme, 2010,
- The VALERA study, 2006 (Professional Value of Erasmus Mobility – a study by the International Centre for Higher Education Research),
- The Flash Eurobarometer study, 2010 (Employers' perceptions of graduate employability – Gallup Organisation requested by European Commission).
Since its initial concept our tool has already gone through several iterations. We are constantly working on improving the methodology in order to provide our clients and their students with the best possible feedback.
The tool started in 2011 with an initial pool of 155 items created by a team of experts to represent the employability and intercultural competence of students. The items came both from desk research and our own experience with related projects as consultants and researchers. This pool was pre-tested on a sample of 80 student to provide us with initial data to be subject to factor analysis. The analysis yielded 10 different factors comprised of 126 items. Two further validations followed. The main test was conducted on 2 449 cases and led to a concentration on 67 items. The last validation based on 3 332 cases yielded a further possible reduction to 52 items without influencing the strengths of the factors. At this point memo©'s Cronbach-alpha is 0.895, which is at a very satisfactory level for tools of its kind, telling us that memo© is able to provide us and our clients with consistent measurements and reliable data.
Cronbach's alpha is the estimate of the reliability of a psychometric test. It uses the covariance between item-pairs and the variance of the total score to express to which extend items (questions, indicators) measure the same thing. The result ranges from 0 to 1. The satisfactory treshold is different for different number of items included (Bortz and Döring 2006), so a value of 0.5 for a four item set is better than one of 0.6 on a twenty item set). The research shows, that alpha is multiply determined under all but the most highly restrictive conditions – it often reflects not only general factor saturation but also group factor saturation and even variability in factor loading (Zinbarg and Revelle, Psychometrika, March 2005).
PROVED BY A SAMPLE OF MORE THAN 90 000 INDIVIDUALS AND NEARLY 1000 HIGHER EDUCATION INSTITUTIONS
The memo© methodology was validated first on a smaller sample of participants and then applied in the Erasmus Impact Study (2014, 2016) with a sample of:
- 56 733 students,
- 18 618 alumni,
- 4 986 staff,
- 964 higher education institutions,
- 652 employers,
all of them surveyed in an on-line survey in 2013. The sample was distributed across 34 countries participating in the Erasmus programme. Since then, the number of participants in memo© has kept rising.
memo© factors
memo© introduces 10 memo© factors describing the main personality traits of students related to intercultural competence and employability.
WHAT DOES IT SAY
memo© data does not aim to provide solely an assessment of students’ abilities to interact in different international and unknown contexts, but also to provide an assessment of how well the design and management of an educational experience are suited to enhancing students’ employability and intercultural competence. The analysis of the added value must therefore be understood as an assessment of the role of the institution in preparing its students for, and guiding them through, their university experience.
memo© shows how students' mind-sets change with regard to intercultural skills and employability-related competences.
memo© FACTORS OF EMPLOYABILITY
Six memo© factor values (Confidence, Curiosity, Decisiveness, Problem-solving, Self-assessment and Tolerance) are combined into the score of "memo© employability".
The Erasmus Impact Study (2014) proved that alumni with a high memo© employability value are more likely to reach managerial positions within a couple of years after graduation and are less threatened by unemployment than those with lower scores.
memo© FACTORS OF INTERCULTURAL COMPETENCE
Six memo© factor values (Adaptability, Curiosity, Position-defending, Self-awareness, Sociability and Tolerance) are combined into "memo© intercultural competence".
The selected factors address aspects of intercultural competence as confirmed by QUEST, a large-scale project over three years conducted by CHE Consult on study adaptability, where related items were applied.
Adaptability
An individual with high values deals well with negative experiences and therefore does not fear possible future setbacks. In the academic context, this goes along with physical well-being and a belief in one’s own ability.
Students with low values on this factor feel exhausted and overwhelmed, which can be accompanied by further symptoms such as sadness, sleeplessness or a certain helplessness. Such students find it hard to find and accept help or support.
Confidence
High values on this factor point to a high degree of self-sufficiency and a strong conviction of one’s own abilities – aspects that may positively impact academic success.
Individuals with high values on this factor may, however, also be inflexible and set in their ways.
Low values show doubt about one’s own abilities and perseverance, which might be grounded in negative experiences or insecurity.
Curiosity
High values on this factor indicate that a person is not only open to new experiences but seeks them actively. This also applies to new academic challenges.
Low values hint at an altogether more reluctant attitude towards new experiences and a greater appreciation of what is familiar.
Decisiveness
High values point to an active and decisive individual, who may have a critical attitude toward the content of his or her study programme.
Low values suggest that the individual is more likely to reconsider his or her decisions to accommodate the opinion of others.
Position-defending
High values characterize an individual who engages easily in and enjoys discussions, feels passionately about things and also does not hesitate to voice his or her opinion in an academic environment.
An individual with low values keeps his/her opinions to him/herself, not only because of less strong conviction but also in order to avoid confrontation. Such an individual is more hesitant to engage in discussions in seminars.
Problem-solving
High values reflect a “problem-solver” who does not like to delve into the insoluble aspects of a task but focuses on the doable, and also likes a challenge. Such individuals may either be very pragmatic in their approach to academic education, considering it as a means to solve practical problems, or else very theory-oriented in that they are attracted to problem-solving as an academic exercise.
Low values reflect an individual who is well aware of problems or problematic aspects of a situation and might be more concerned with identifying the problem than with solving it. Accordingly, such an individual would be less goal-oriented and may have an altogether less future-oriented perspective on things.
Self-assessment
High values on this factor indicate that a person knows his or her strengths and weaknesses. This good self-assessment not only leads to a more relaxed relationship to other people or new demands, but also might prevent disappointment with the higher education institution.
Low values, on the other hand, suggest an altogether higher stress level that can be caused by a misjudgement of one’s own abilities, accompanied by difficulty understanding the demands and requirements of the study programme.
Self-awareness
High values indicate that the person is convinced that he or she can influence the outcome of their ambitions and efforts (such as study results), and that these are not a matter of luck or coincidence. This perspective is accompanied by a high self-awareness and a good assessment of what is expected.
Low values not only suggest that a person does not feel that he or she really can make a difference with respect to the outcome of a certain task or situation, but they also feel a certain ambiguity about the demands and conditions of the task or situation.
Sociability
High values refer to an individual who not only likes to socialize, but who does not cope well with being alone. For these individuals, an active social life coincides with emotional well-being.
Low values show a person who does not like to mingle, but is better able to deal with negative feelings such as rejection and is less concerned about the opinions of others.
Tolerance
High values on this factor mean that a person is able to tolerate the behaviour and values of other people without compromising his or her own values.
Low values mean that a person feels very uncomfortable if confronted with other people’s different values and ways of life. Such individuals may espouse a more traditional view of things, based on their own perspective and experience as influenced by family, society and established norms and values. Deviation from what is conceived of as “normal” is perceived as threatening or at least discomforting.
