Since 1999, European perspectives on the quality of higher education have been strongly influenced by the follow-up processes to the Bologna Declaration of that year, signed by 29 European Ministers of Education. By signing this declaration, the Ministers agreed on coordinating their policies towards achieving a number of objectives, which they considered to be of primary relevance in establishing a European area of higher education and promoting the European system of higher education worldwide. According to the Bologna Declaration, their agreed objectives, with a target date of 2010, are as follows:

· Adoption of a system of easily readable and comparable degrees, also through the implementation of the Diploma Supplement, in order to promote the employability of European citizens and the international competitiveness of the European higher education system;

· Adoption of a system essentially based on two main cycles: undergraduate and graduate. Access to the second cycle shall require successful completion of first cycle studies which have a minimum duration of three years. The degree awarded after the first cycle shall also be relevant to the European labour market as an appropriate level of qualification. The second cycle should lead to the master and/or doctorate degree, as in many European countries;

· Establishment of a system of credits - such as the ECTS system - as a proper means of promoting the most widespread student mobility. It should be possible to acquire credits in non-higher education contexts, including lifelong learning activities, provided such contexts are recognised by the receiving universities concerned;

· Promotion of mobility by overcoming obstacles to the effective exercise of free movement, with particular attention to:

1. Student access to study and training opportunities and related services;

2. Recognition and valorisation of periods spent by teachers, researchers and administrative staff in a European context researching, teaching and training, without prejudicing their statutory rights;

· Promotion of European cooperation in quality assurance with a view to developing comparable criteria and methodologies;

· Promotion of the necessary European dimensions in higher education, particularly with regard to curricular development, inter-institutional cooperation, mobility schemes and integrated programmes of study, training and research.

The ministers undertook ‘to attain these objectives - within the framework of our institutional competences and taking full account of the diversity of cultures, languages, national education systems and university autonomy - to consolidate the European area of higher education’ and further stated that, ‘To that end, we will pursue ways of intergovernmental cooperation, and cooperation with European non-governmental organisations with competence within higher education. In return, we expect Universities to respond promptly and positively, and to contribute actively to the success of our endeavour.’

This general background, together with subsequent initiatives and developments occurring between the ministerial meetings in Bologna, Prague and beyond, have provided the main motivation for setting up the Transnational European Evaluation Project (TEEP).

TEEP is supported by the European Commission through the SOCRATES programme. It is part of a package of measures initiated by the European Commission in order to stimulate the Bologna Process (from Prague to Berlin, the EU-contribution). The project is coordinated through the European Network of Quality Assurance in Higher Education (ENQA), with the participation and contribution of the SOCRATES Thematic Networks of the three disciplines history, physics and veterinary science. Representatives of ENQA, the chairpersons of the SOCRATES Thematic Networks, representatives of the European Commission and representatives of the relevant quality assurance agencies constitute the management group for the project.

1.2 Introduction

The Transnational European Evaluation Project (TEEP) is a pilot project with the objective of investigating the operational implications of a European transnational quality evaluation of study programmes in three subject areas: history, physics and veterinary science.

The three subject areas of physics, history and veterinary science represented respectively by five, five and four participating European Universities. In total, fourteen programmes in ten different European countries are evaluated.

The objectives of TEEP have been:

· To further develop a method for transnational external evaluation, building on experiences such as the TUNING Project and the BA/Ma descriptors developed through the joint quality initiative, using common criteria on the basis of an evaluation process in three different disciplines.

· To identify potential obstacles to transnational evaluation and indicate strategies that might be used to overcome them.

· To contribute to greater awareness, transparency and compatibility within European higher education.

1.2.1 Anticipated benefits from TEEP

The likely benefits from TEEP should include:

For European Higher Education:

· A method for transnational evaluation building on predefined criteria which are commonly agreed and which have been tested and offer a dimension of transparency and comparability to the quality of programmes across borders;

· A contribution to the development of each subject on the basis of the recommendations of the experts and good practice from comparable programmes in other countries;

· An opportunity to share experiences between programmes and peers, and the possibility of establishing networks to assure continuous improvement of programme quality;

For the participating institutions:

· The opportunity for each of the participating institutions to promote both their institution and its programmes;

· The opportunity to receive feedback as a contribution towards improving their quality assurance culture.

1.2.2 Scope

The European physics programmes under review are:

· Technical Physics, Faculty of Science and Informatics/ Physics Department, Vienna University of Technology, Austria

· The physics programme, Faculty of Physics, Warsaw University, Poland

· The physics programme, Physics Department, Paul Sabatier University, Toulouse, France

· The physics programme, Physics Department, University of Rome La Sapienza, Italy

· The physics programme, Niels Bohr Institute for Astronomy, Physics and Geophysics, Copenhagen University, Denmark

The scope of the review is the first cycle degree of the physics programmes.

1.2.3 Evaluation method

The method consisted of three main elements: self-evaluation reports; site visits conducted by external expert panels; and publication of reports. In other words, the method corresponds with the European Council recommendation of 1998 on European cooperation in quality assurance within higher education.

Self-evaluation report

The first element in the evaluation has been a self-evaluation of the selected study programmes, carried out by the respective higher education institutions. As the transnational evaluation is a ‘lighter’ version of typical local/national evaluations, the self-evaluation report has been structured around pre-selected focal points:

· Educational context

· Competences and learning outcomes;

· Quality assurance mechanisms.

The preparation of the self-evaluation report was designed to serve three distinct aims:

· to provide a framework to stimulate internal discussions of strengths and weaknesses related to the three themes that are the foci for the evaluation. This was intended to assist the continuous improvement in the quality of the programme;

· to provide comparable documentation to be used by the panel of experts in their preparations, site visits, evaluations and reports;

· to invite comments on the utility of the criteria when applied to different programmes delivered within different national contexts;

The self-evaluation reports together with the information gathered during the site visits constituted the documentation for the evaluation.

The self-evaluation report was prepared at each programme by a self-evaluation group under the responsibility of a chairperson. The self-evaluation group was responsible for the preparation of a self-evaluation report which was to reflect the results of the group’s work. The self evaluation groups included at least one representative from each of the relevant stakeholders at the programme level, including management, staff actively involved in teaching, students and administrative staff.

Site visits

The self-evaluation was followed by site visits by teams comprising four experts and a secretary. The site visits took place in January-March 2003 and lasted 1½ day per institution. All site visits were structured in a similar way, in accordance with a standard programme. The site visits provided the panel with an opportunity to invite the institutions to elaborate on unclear and less substantiated sections of the self-evaluation reports. At the same time, the site visits served to validated the information provided in the self-evaluation reports. Furthermore, the site visits allowed the experts to get a comprehensive and clear view of the programme through discussions and interviews with main stakeholders.

Each visit comprised a number of separate interviews with different groups of stakeholders who, in one way or another, were engaged with the programmes under evaluation. The expert panel has interviewed students, graduates, teaching staff, management and the self-evaluation group concerning their perspectives.

Report

TEEP results in one report for each of the three disciplines: History, Physics and Veterinary science. For each panel a draft report is prepared and submitted to the participating programmes. The programmes then provide the secretary with corrections of errors of fact in the draft report, and the final report is prepared in the light of the institution's response.

Since TEEP is a pilot project for transnational evaluation that is based on predefined criteria, a report on the methodological experiences, including recommendations for future transnational evaluations, will be prepared for the European Commission once the evaluation processes are finalised. The methodological report will published October 2003.

1.2.4 Organisation of the evaluation

The criteria presented in Annex A constitute the framework of the evaluation. An evaluation officer from the Danish Evaluation Institute (EVA) is responsible for the methodological aspects of the evaluation and the initial draft of the report, while a panel of international experts appointed by the Management Group of the TEEP project is responsible for the academic quality of the evaluation and the final report. Due to limitations on funding, each member of the panel was asked to participate in only a very restricted number of site visits. The chairman participated in all visits except one. In this instance, the Vice-chairman chaired the meetings.

The members of the panel of institutional experts are:

· Chairman: Professor David W. Hughes, University of Sheffield, Department of Physics and Astronomy.

· Vice-Chairman: Professor Richard Thompson, Imperial College London, Department of Physics.

· Professor Christoph Bargholtz, Stockholm University, Department of Physics.

· Professor of Physics Faculty Vilnius University Gintaras Dikcius, Vilniaus Universitetas, Lietuva.

· Prof. Dr. Ramon Pascual, Universitat Autònoma de Barcelona, Department de Fisica.

· Director of the Institute of Physics Education, Clemens L.M. Pouw, University of Twente, Department of Applied Physics.

· Professor Peter U. Sauer, University of Hanover, Institute for Theoretical Physics.

· Dr. Frank Witte, Manager of the Master's programmes, Department of Physics and Astronomy of Utrecht University.

A physics student appointed by ESIB participated in the first site visit. Despite hard efforts from ESIB it turned out to be difficult for them to identify students for the remaining site visits.

Evaluation officer Tine Holm from EVA wrote the initial draft of the report; acted as the secretary for the physics evaluation, participated in all site visits and has been assisted by evaluation clerk Sanne Reitzel Gunnersen. The experts received a draft version of this report with a request for comments. The tight time schedule of TEEP did not allow an additional meeting of the experts, but EVA collected and reviewed their comments and incorporated these into the final report. Furthermore, the institutions had the opportunity to make factual comments on the report. Following this, the report was sent to the experts for a short consultation. The process was concluded with a telephone meeting between the chairman, vice-chairman and the secretary before the final publication of the report.

2 Comparative perspectives

In accordance with the three themes of the self-evaluation manual - educational context, competences and quality assurance - the first part of report will focus on the following topics:

· The level of implementation of the first and second cycle degree structure;

· The extent to which the programmes formulated and used definitions of competences and learning outcomes, including knowledge and applicability of the Tuning-criteria;

· The level of implementation of quality assurance in the programmes.

2.1 Degree structure and definition

One of the criteria of the transnational evaluation is the degree to which the programmes have formulated and established a first cycle degree programme. The evaluation attempts to establish whether the programmes have formulated goals for the first cycle degree, and if these formulations match the Dublin descriptors for the first cycle degree.

Most of the physics programmes have established first cycle degrees with a 3 year duration. As a matter of fact, three of the programmes have just been re-structured according to the Bologna agreement. Only one of the programmes does not offer the first cycle degree, but an integrated five-year master degree instead.

The extent to which the evaluated programmes have implemented a first and second cycle degree structure seems to be dependent upon the commitment of the countries in question towards the Bologna process. It is not surprising that in those countries where the first and second cycle degree structure has become part of the governmental regulation of higher education, universities undertake the implementation of this new structure.

The evaluation reveals considerable variation in the degree to which the programmes have formulated specific aims for the first cycle degree that match the Dublin descriptors. One programme has an explicit aim, stating that the first cycle programme leads both to employment and further study. The other programmes have not explicitly formulated their aims for the first cycle programme. For these degrees, it is implicit that the first cycle degree is the first step towards the master or PhD degree.

The degree to which the first cycle graduates enter employment also varies across the programmes. It seems to be dependent upon the general tradition of employing bachelor graduates within the particular labour market. In these programmes, the first cycle degree is often seen as an instrument for mobility.

Overall, there seems to be a strong link between the national regulation of degree structure and the extent to which the programmes have developed a first and second cycle structure. The degree to which the programmes have formulated specific aims for the first cycle degree seems, however, to depend on the interaction with the labour market and whether labour market representatives have been involved in formulating needs.

2.2 Competences and learning outcomes

In the framework (see appendix A) there are suggested formulations of subject specific and generic competences. The inspiration for these has, to a large extent, been the TUNING descriptors. The evaluated programmes are asked if they have formulated definitions of both subject specific and generic competences at programme and course levels. Furthermore the programmes are asked whether these competence definitions are communicated to students and staff. To students, so that they know what is expected of them. To staff, so there is a shared understanding of what is expected of the students. Finally the report attempts to establish whether teaching and learning methods, as well as assessment methods, support the development of the desired competences.

The extent to which the programmes are familiar and employ the competence-terminology varies considerably. Two of the programmes have formulated both subject specific and generic competences at programme and course levels. One of the programmes has worked explicitly with the terminology, and the competences are communicated and known by students and staff. Another programme has participated in the TUNING project and, in connection with this, has formulated subject specific and generic competences. However, the approach has not been entirely adopted by students and staff.

For the remaining three programmes, the competence-terminology is unfamiliar and, therefore, not actively used. However, in these programmes, there seems to be an implicit notion of what the expected competences are.

Another important dimension of the criteria for competences is the degree to which teaching and learning strategy and assessment methods support the development of both subject specific and generic competences. Although some of the programmes have not explicitly formulated expected competences, or disseminated the competence definition effectively, the teaching and assessment methods seem nevertheless to support the development of both subject specific and generic competences, including problem solving abilities and student autonomy. In these programmes, there seem to be an implicit understanding among teachers as to what the expected competences are. However, these are not explicit to the students.

In general the teaching and learning methods used in the programmes seem to develop both subject specific and generic skills. However, as some of programmes have yet to formulate what competences they expect from students, it does not seem to be clear to all students what competences they should develop and what competences the labour market expects them to possess. This is also reflected in the fact that very few of the programmes have a systematic feed back from the labour market of which competences the first cycle graduates should possess. Therefore, a considerable amount of effort is still needed, in some cases, in order to ensure a culture of internal and external reference points, such as Tuning and the competence-terminology.

2.3 Quality assurance

Another set of criteria try to establish the degree to which the programmes have formulated an explicit quality assurance strategy and established quality assurance mechanism.

Only two of the programmes have formulated an explicit quality assurance strategy. These programmes have formulated quality assurance strategies that focus on quality assurance at course level according to national requirements. The other programmes have no explicit strategies formulated at present, but work on these is in progress. Nevertheless, all five institutions consider that quality assurance is a point which needs further development.

One common feature across the five programmes is that quality assurance is devoted mainly to quality assurance at course level. Almost all programmes have established very comprehensive course evaluation systems, which include evaluations on a regular and systematic basis. Furthermore, one of the programmes conducts course and programme evaluations every second year due to governmental requirements.

Quality assurance mechanisms based on feedback from external and internal stakeholders such as students, graduates and employers is not conducted on a systematic basis at any of the programmes. This does not mean that feedback is not provided on sporadic basis. However, the programmes do not collect the information on a systematic basis.

Student progress is another important way to evaluate and ensure the effectiveness and quality of a programme. To the majority of the programmes, the exercise of gathering student progress information was a new and valuable exercise. There is no tradition for registration and use of student progress information. However, one of the programmes has started to conduct systematic student surveys, and another has started to use student progress information to review the extent to which the original aims of the programme remain appropriate.

All in all, the programmes recognise that quality assurance is necessary. It is evident that the evaluation has initiated internal discussions about how comprehensive ‘packages’ of explicit quality assurance mechanisms can be implemented and used to ensure high quality programmes.

3 Institutional reviews

The remaining chapters comprise institutional reviews of the physics programmes offered at Copenhagen University, University Paul Sabatier in Toulouse, University of Rome La Sapienza, Vienna University of Technology and Warsaw University. It is important to emphasise that the expert-panel has gained a positive impression of all five education programmes. This positive impression is not least due to the fact that all these universities have a reputation nationally for providing high quality programmes and are some of the best places nationally to study physics. This does not mean that the individual programmes do not have certain weaknesses. The reviews in this chapter need to be seen in balance with the general view that all five programmes offer a high quality education in physics.

The assessment of the programmes[1] has focused on the following three selected areas: educational context; competences and learning outcomes; and quality assurance. The programmes have been reviewed against a common set of quality criteria associated with each of the above three focus areas (see appendix A). In each institutional review chapter, every subsection introduces the criteria against which the institution is reviewed. This is followed by an analysis of the programme according to the documentation material (self-evaluation report and site visit). The panel have recognised many strengths and aspects in the programmes, which can serve as an inspiration to other programmes, and these are incorporated in the analyses of the self-evaluation reports and the site visits. Each subsection concludes with a recommendation which highlights areas that can be strengthened.

It should be mentioned that the documentation procured from the institutional accounts and the site visits is not entirely uniform. Some themes and aspects are highlighted more by one programme than another. This point is accentuated by the fact that the strengths and weaknesses of each programme have, to some extent, been reviewed in relation to the specific context of the institution and its national higher education system. Furthermore, the differences in feedback to the programmes are also a reflection of the difference between the self-evaluation reports.

Finally, some reading instructions. Whenever the report mentions that the source of evidence is interviews, it refers to the interviews conducted with the different stakeholders during the site visits. In addition, it should be specially noted that SER is an abbreviation of self-evaluation report.

4 Copenhagen University

4.1 Educational context

4.1.1 Recent developments in Danish higher education

Most university studies are structured as three years for a first cycle degree, optionally followed by two years for a master degree (cand. scient.), which, again, can be optionally followed by three years for a Ph.D. degree. In May 2003, a new law for Danish Universities came into force. The law stated that, where the educational structure has not already been altered, it is to be restructured in order to ensure genuine implementation of the 3+2 structure. That is three-year bachelor degrees followed by two-year master degrees (candidatus).

Furthermore, the law introduces a modular structure for all bachelor and master programmes. Students with a relevant academic bachelor degree must be entitled to enrol for an academically relevant master degree. Students with a bachelor degree will have a genuine choice between several relevant master degrees – also a choice of master degrees at another university. Academic relevance, correlation and progression must be ensured, and the programmes must have clear competence profiles, these being directed towards different jobs within the private as well as the public sectors.

The alteration of the content and structure of the programmes is to be a staged process taking place over a number of years and in accordance with a process that the universities and the Ministry of Science, Technology and Innovation will discuss and agree upon.

Finally, the law states that the universities are to become self-governing institutions. The executive management structure will be subject to reforms. The law obliges the universities to set up boards with external members, and employ rectors and managers[2].

4.1.2 National regulation

The structure of university studies is regulated by a ministerial order concerning the 3+2+3 structure. Further framework regulation exists for the natural science degrees. This includes a specific requirement that the bachelor study should be comprehensive (i.e. not only preparing for postgraduate studies but also for employment) and include a project lasting 1/6 of an academic year. There are also regulations about the entry requirements for the different subject areas. Finally, regulations lay down the requirement that 1/3 of the exams should be subject to external examination. Otherwise, it is left to the study committee to provide a detailed study programme for both the bachelor part and the master part. Thus, apart from the basic structure, there is a large degree of freedom in how to implement the structure.

4.1.3 Organisation of the programme

The vice-chancellor (rektor) is the head of the university, with a dean heading each faculty. Each faculty consists of a number of departments, each headed by a head of department. At each level there is also a board (university council, faculty council and institute board).

For each individual subject or group of studies there is a study committee. The study committee consists of an equal number of tenured teachers and students. This board appoints, from among the tenured teachers, a chairman of the study committee who is then directly responsible to the dean.

The study committee has to provide a detailed curriculum for the study for which it is responsible, and this curriculum has to be approved by the faculty. It also has to provide a study plan, which usually takes the form of a list of courses offered. That this is then carried out in practice is the responsibility of the head of department who, in turn, has appointed a teaching committee. The set of courses actually offered in a given semester is determined in collaboration between the teaching committee and the study committee.

4.1.4 Academic staff

In 1993, four departments were merged to create The Niels Bohr Institute for Astronomy, Physics and Geophysics, with a scientific staff of just under 100 covering a very large spectrum of subjects. It was thus possible for students to undertake a project or thesis in virtually any subject. A large variety of advanced specialised courses were offered. This is still the case, even though the permanent staff has now been reduced to around 60. With another ten taking early retirement during 2004, and the positions not being re-appointed, it may not be possible that such variety can continue.

The institute has, therefore, established a renewal programme for the staff, and this is described in detail. Special emphasis is given to the ‘generation change’ programme to cope with the fairly large number of retirements that will take place during the next ten years.

Table 1

Information on staff in total numbers

	Number of persons	Full-time equivalent (on teaching)
Full Professors	15	6,5
Associate Professors	46	17
Assistant Professors	0	0
Research Assistants	17	0
Teaching Assistants	0	0
PhDs	60	5
Other Categories	-	-
In total	138	28,5

Source: The SER Copenhagen University. In the table, full-time equivalents state only the full-time equivalents of time spent on teaching.

The ministerial directive regulates the rules for recruitment of new academic staff. Research qualifications are very important but teaching experience and qualifications are taken into consideration as well. To get a permanent position, some formal pedagogical training is usually required, e.g., the candidate must give a trial lecture.

A centre for didactics has been established which provides courses every year. According to Danish law, the universities must provide pedagogical training for assistant professors. The teaching courses provided for new staff members seem to be very comprehensive and done well. However, there are no pedagogical support programmes for professors.

The panel finds it positive that the programme management provides didactic courses for assistant professors. However, the programme management should also encourage participation of present staff in pedagogical training.

4.1.5 Admission

There are two ways of being admitted to a university study in Denmark. The first is by having a standard qualification, i.e. a high school exam or an equivalent degree with a well-defined set of subjects at a well-defined set of levels. The second way of access is non-standard, e.g. a qualification from a foreign country. There is a maximum number that can be admitted with non-standard qualifications and a committee makes the decision regarding admission.

In addition, the universities have the possibility of requiring a certain grade at the high school exam, or of setting the maximum number of students they can accept for each individual study. In physics, there is at present a maximum intake of 150 but no grade requirement. If the number of applicants is larger than the maximum intake, those with the highest grades from high school will be admitted. However, there are specific subject requirements, which have to be fulfilled. In physics, the students must have at least level A in mathematics and physics and level C in chemistry to be admitted. All the students applying for physics, who fulfilled the subject requirements, have been accepted independently of their grades.

Enrolment is centralised in the sense that each student only has to fill out one application form for entrance to a university study. The student’s priorities in terms of subjects and universities are listed on the form. A central system then enrols students according to priorities and available places.

4.1.6 Student intake

The students entering the physics programme usually combine two subjects, e.g. physics/mathematics. It is difficult for the Copenhagen University programme (CU)[3] to give the exact numbers of students studying physics, as the student could be enrolled under another subject, e.g., mathematics. The only reliable number that can be provided is the intake of students who, on registration, opted for a combination that leads to a bachelor degree in physics (and probably one other subject).

Table 2

Intake of students in total numbers

Year	Applicants	Admitted Male / Female
2000-2001	149	112/36
2001-2002	140	95/34
2002-2003	106	72/27

Source: The SER Copenhagen University.

There has been a small decrease in the intake during the last few years. As seen from the table above, the decrease has been considerable this year. Despite this decreased intake, it has been possible to increase the number achieving the master degree from around 25% per year to nearly 50% per year over a period of around ten years. There are at least two reasons for this. One is that the average time taken to obtain the degree has decreased considerably, and another is that the dropout rate has decreased.

This year a new subject area has been created, ‘nano-science’, in which the content of physics is reduced compared with ‘physics’. This has been a success with a fairly large intake, but with a correspondingly smaller intake for physics - a survey has indicated that less than half of those studying nano-science would have chosen physics had nano-science not been available.

The panel recommends that the programme management analyses why the students seem to find nano-science more attractive than the physics programme.

4.1.7 Student progress information

The dropout rate for the programme is fairly high, with up to 50% dropouts among the first year students who enrol for the programme. A large proportion of the high school graduates are not well prepared to adopt to the university teaching methods and subject level. This has been a matter of concern over the last decade and many initiatives have been taken.

Table 3

Student progress information in total numbers

Number of students whose admission year was 1999 and who were at present three years later in 2001
First year (1999)	11
Second year (2000)	38
Third year (2001)