Since 1999, European perspectives on the quality of higher education have been strongly influenced by the follow-up processes to the Bologna Declaration of that year, signed by 29 European Ministers of Education. By signing this declaration, the Ministers agreed on coordinating their policies towards achieving a number of objectives, which they considered to be of primary relevance in establishing a European area of higher education and promoting the European system of higher education worldwide. According to the Bologna Declaration, their agreed objectives, with a target date of 2010, are as follows:

· Adoption of a system of easily readable and comparable degrees, also through the implementation of the Diploma Supplement, in order to promote the employability of European citizens and the international competitiveness of the European higher education system;

· Adoption of a system essentially based on two main cycles: undergraduate and graduate. Access to the second cycle shall require successful completion of first cycle studies which have a minimum duration of three years. The degree awarded after the first cycle shall also be relevant to the European labour market as an appropriate level of qualification. The second cycle should lead to the master and/or doctorate degree, as in many European countries;

· Establishment of a system of credits - such as the ECTS system - as a proper means of promoting the most widespread student mobility. It should be possible to acquire credits in non-higher education contexts, including lifelong learning activities, provided such contexts are recognised by the receiving universities concerned;

· Promotion of mobility by overcoming obstacles to the effective exercise of free movement, with particular attention to:

1. Student access to study and training opportunities and related services;

2. Recognition and valorisation of periods spent by teachers, researchers and administrative staff in a European context researching, teaching and training, without prejudicing their statutory rights;

· Promotion of European cooperation in quality assurance with a view to developing comparable criteria and methodologies;

· Promotion of the necessary European dimensions in higher education, particularly with regard to curricular development, inter-institutional cooperation, mobility schemes and integrated programmes of study, training and research.

The ministers undertook ‘to attain these objectives - within the framework of our institutional competences and taking full account of the diversity of cultures, languages, national education systems and university autonomy - to consolidate the European area of higher education’ and further stated that, ‘To that end, we will pursue ways of intergovernmental cooperation, and cooperation with European non-governmental organisations with competence within higher education. In return, we expect Universities to respond promptly and positively, and to contribute actively to the success of our endeavour.’

This general background, together with subsequent initiatives and developments occurring between the ministerial meetings in Bologna, Prague and beyond, have provided the main motivation for setting up the Transnational European Evaluation Project (TEEP).

TEEP is supported by the European Commission through the SOCRATES programme. It is part of a package of measures initiated by the European Commission in order to stimulate the Bologna Process (from Prague to Berlin, the EU-contribution). The project is coordinated through the European Network of Quality Assurance in Higher Education (ENQA), with the participation and contribution of the SOCRATES Thematic Networks of the three disciplines history, physics and veterinary science. Representatives of ENQA, the chairpersons of the SOCRATES Thematic Networks, representatives of the European Commission and representatives of the relevant quality assurance agencies constitute the management group for the project.

1.2 Introduction

The Transnational European Evaluation Project (TEEP) is a pilot project with the objective of investigating the operational implications of a European transnational quality evaluation of study programmes in three subject areas: history, physics and veterinary science.

The three subject areas of physics, history and veterinary science represented respectively by five, five and four participating European Universities. In total, fourteen programmes in ten different European countries are evaluated.

The objectives of TEEP have been:

· To further develop a method for transnational external evaluation, building on experiences such as the TUNING Project and the BA/Ma descriptors developed through the joint quality initiative, using common criteria on the basis of an evaluation process in three different disciplines.

· To identify potential obstacles to transnational evaluation and indicate strategies that might be used to overcome them.

· To contribute to greater awareness, transparency and compatibility within European higher education.

1.2.1 Anticipated benefits from TEEP

The likely benefits from TEEP should include:

For European Higher Education:

· A method for transnational evaluation building on predefined criteria which are commonly agreed and which have been tested and offer a dimension of transparency and comparability to the quality of programmes across borders;

· A contribution to the development of each subject on the basis of the recommendations of the experts and good practice from comparable programmes in other countries;

· An opportunity to share experiences between programmes and peers, and the possibility of establishing networks to assure continuous improvement of programme quality;

For the participating institutions:

· The opportunity for each of the participating institutions to promote both their institution and its programmes;

· The opportunity to receive feedback as a contribution towards improving their quality assurance culture.

1.2.2 Scope

The European physics programmes under review are:

· Technical Physics, Faculty of Science and Informatics/ Physics Department, Vienna University of Technology, Austria

· The physics programme, Faculty of Physics, Warsaw University, Poland

· The physics programme, Physics Department, Paul Sabatier University, Toulouse, France

· The physics programme, Physics Department, University of Rome La Sapienza, Italy

· The physics programme, Niels Bohr Institute for Astronomy, Physics and Geophysics, Copenhagen University, Denmark

The scope of the review is the first cycle degree of the physics programmes.

1.2.3 Evaluation method

The method consisted of three main elements: self-evaluation reports; site visits conducted by external expert panels; and publication of reports. In other words, the method corresponds with the European Council recommendation of 1998 on European cooperation in quality assurance within higher education.

Self-evaluation report

The first element in the evaluation has been a self-evaluation of the selected study programmes, carried out by the respective higher education institutions. As the transnational evaluation is a ‘lighter’ version of typical local/national evaluations, the self-evaluation report has been structured around pre-selected focal points:

· Educational context

· Competences and learning outcomes;

· Quality assurance mechanisms.

The preparation of the self-evaluation report was designed to serve three distinct aims:

· to provide a framework to stimulate internal discussions of strengths and weaknesses related to the three themes that are the foci for the evaluation. This was intended to assist the continuous improvement in the quality of the programme;

· to provide comparable documentation to be used by the panel of experts in their preparations, site visits, evaluations and reports;

· to invite comments on the utility of the criteria when applied to different programmes delivered within different national contexts;

The self-evaluation reports together with the information gathered during the site visits constituted the documentation for the evaluation.

The self-evaluation report was prepared at each programme by a self-evaluation group under the responsibility of a chairperson. The self-evaluation group was responsible for the preparation of a self-evaluation report which was to reflect the results of the group’s work. The self evaluation groups included at least one representative from each of the relevant stakeholders at the programme level, including management, staff actively involved in teaching, students and administrative staff.

Site visits

The self-evaluation was followed by site visits by teams comprising four experts and a secretary. The site visits took place in January-March 2003 and lasted 1½ day per institution. All site visits were structured in a similar way, in accordance with a standard programme. The site visits provided the panel with an opportunity to invite the institutions to elaborate on unclear and less substantiated sections of the self-evaluation reports. At the same time, the site visits served to validated the information provided in the self-evaluation reports. Furthermore, the site visits allowed the experts to get a comprehensive and clear view of the programme through discussions and interviews with main stakeholders.

Each visit comprised a number of separate interviews with different groups of stakeholders who, in one way or another, were engaged with the programmes under evaluation. The expert panel has interviewed students, graduates, teaching staff, management and the self-evaluation group concerning their perspectives.

Report

TEEP results in one report for each of the three disciplines: History, Physics and Veterinary science. For each panel a draft report is prepared and submitted to the participating programmes. The programmes then provide the secretary with corrections of errors of fact in the draft report, and the final report is prepared in the light of the institution's response.

Since TEEP is a pilot project for transnational evaluation that is based on predefined criteria, a report on the methodological experiences, including recommendations for future transnational evaluations, will be prepared for the European Commission once the evaluation processes are finalised. The methodological report will published October 2003.

1.2.4 Organisation of the evaluation

The criteria presented in Annex A constitute the framework of the evaluation. An evaluation officer from the Danish Evaluation Institute (EVA) is responsible for the methodological aspects of the evaluation and the initial draft of the report, while a panel of international experts appointed by the Management Group of the TEEP project is responsible for the academic quality of the evaluation and the final report. Due to limitations on funding, each member of the panel was asked to participate in only a very restricted number of site visits. The chairman participated in all visits except one. In this instance, the Vice-chairman chaired the meetings.

The members of the panel of institutional experts are:

· Chairman: Professor David W. Hughes, University of Sheffield, Department of Physics and Astronomy.

· Vice-Chairman: Professor Richard Thompson, Imperial College London, Department of Physics.

· Professor Christoph Bargholtz, Stockholm University, Department of Physics.

· Professor of Physics Faculty Vilnius University Gintaras Dikcius, Vilniaus Universitetas, Lietuva.

· Prof. Dr. Ramon Pascual, Universitat Autònoma de Barcelona, Department de Fisica.

· Director of the Institute of Physics Education, Clemens L.M. Pouw, University of Twente, Department of Applied Physics.

· Professor Peter U. Sauer, University of Hanover, Institute for Theoretical Physics.

· Dr. Frank Witte, Manager of the Master's programmes, Department of Physics and Astronomy of Utrecht University.

A physics student appointed by ESIB participated in the first site visit. Despite hard efforts from ESIB it turned out to be difficult for them to identify students for the remaining site visits.

Evaluation officer Tine Holm from EVA wrote the initial draft of the report; acted as the secretary for the physics evaluation, participated in all site visits and has been assisted by evaluation clerk Sanne Reitzel Gunnersen. The experts received a draft version of this report with a request for comments. The tight time schedule of TEEP did not allow an additional meeting of the experts, but EVA collected and reviewed their comments and incorporated these into the final report. Furthermore, the institutions had the opportunity to make factual comments on the report. Following this, the report was sent to the experts for a short consultation. The process was concluded with a telephone meeting between the chairman, vice-chairman and the secretary before the final publication of the report.

2 Comparative perspectives

In accordance with the three themes of the self-evaluation manual - educational context, competences and quality assurance - the first part of report will focus on the following topics:

· The level of implementation of the first and second cycle degree structure;

· The extent to which the programmes formulated and used definitions of competences and learning outcomes, including knowledge and applicability of the Tuning-criteria;

· The level of implementation of quality assurance in the programmes.

2.1 Degree structure and definition

One of the criteria of the transnational evaluation is the degree to which the programmes have formulated and established a first cycle degree programme. The evaluation attempts to establish whether the programmes have formulated goals for the first cycle degree, and if these formulations match the Dublin descriptors for the first cycle degree.

Most of the physics programmes have established first cycle degrees with a 3 year duration. As a matter of fact, three of the programmes have just been re-structured according to the Bologna agreement. Only one of the programmes does not offer the first cycle degree, but an integrated five-year master degree instead.

The extent to which the evaluated programmes have implemented a first and second cycle degree structure seems to be dependent upon the commitment of the countries in question towards the Bologna process. It is not surprising that in those countries where the first and second cycle degree structure has become part of the governmental regulation of higher education, universities undertake the implementation of this new structure.

The evaluation reveals considerable variation in the degree to which the programmes have formulated specific aims for the first cycle degree that match the Dublin descriptors. One programme has an explicit aim, stating that the first cycle programme leads both to employment and further study. The other programmes have not explicitly formulated their aims for the first cycle programme. For these degrees, it is implicit that the first cycle degree is the first step towards the master or PhD degree.

The degree to which the first cycle graduates enter employment also varies across the programmes. It seems to be dependent upon the general tradition of employing bachelor graduates within the particular labour market. In these programmes, the first cycle degree is often seen as an instrument for mobility.

Overall, there seems to be a strong link between the national regulation of degree structure and the extent to which the programmes have developed a first and second cycle structure. The degree to which the programmes have formulated specific aims for the first cycle degree seems, however, to depend on the interaction with the labour market and whether labour market representatives have been involved in formulating needs.

2.2 Competences and learning outcomes

In the framework (see appendix A) there are suggested formulations of subject specific and generic competences. The inspiration for these has, to a large extent, been the TUNING descriptors. The evaluated programmes are asked if they have formulated definitions of both subject specific and generic competences at programme and course levels. Furthermore the programmes are asked whether these competence definitions are communicated to students and staff. To students, so that they know what is expected of them. To staff, so there is a shared understanding of what is expected of the students. Finally the report attempts to establish whether teaching and learning methods, as well as assessment methods, support the development of the desired competences.

The extent to which the programmes are familiar and employ the competence-terminology varies considerably. Two of the programmes have formulated both subject specific and generic competences at programme and course levels. One of the programmes has worked explicitly with the terminology, and the competences are communicated and known by students and staff. Another programme has participated in the TUNING project and, in connection with this, has formulated subject specific and generic competences. However, the approach has not been entirely adopted by students and staff.

For the remaining three programmes, the competence-terminology is unfamiliar and, therefore, not actively used. However, in these programmes, there seems to be an implicit notion of what the expected competences are.

Another important dimension of the criteria for competences is the degree to which teaching and learning strategy and assessment methods support the development of both subject specific and generic competences. Although some of the programmes have not explicitly formulated expected competences, or disseminated the competence definition effectively, the teaching and assessment methods seem nevertheless to support the development of both subject specific and generic competences, including problem solving abilities and student autonomy. In these programmes, there seem to be an implicit understanding among teachers as to what the expected competences are. However, these are not explicit to the students.

In general the teaching and learning methods used in the programmes seem to develop both subject specific and generic skills. However, as some of programmes have yet to formulate what competences they expect from students, it does not seem to be clear to all students what competences they should develop and what competences the labour market expects them to possess. This is also reflected in the fact that very few of the programmes have a systematic feed back from the labour market of which competences the first cycle graduates should possess. Therefore, a considerable amount of effort is still needed, in some cases, in order to ensure a culture of internal and external reference points, such as Tuning and the competence-terminology.

2.3 Quality assurance

Another set of criteria try to establish the degree to which the programmes have formulated an explicit quality assurance strategy and established quality assurance mechanism.

Only two of the programmes have formulated an explicit quality assurance strategy. These programmes have formulated quality assurance strategies that focus on quality assurance at course level according to national requirements. The other programmes have no explicit strategies formulated at present, but work on these is in progress. Nevertheless, all five institutions consider that quality assurance is a point which needs further development.

One common feature across the five programmes is that quality assurance is devoted mainly to quality assurance at course level. Almost all programmes have established very comprehensive course evaluation systems, which include evaluations on a regular and systematic basis. Furthermore, one of the programmes conducts course and programme evaluations every second year due to governmental requirements.

Quality assurance mechanisms based on feedback from external and internal stakeholders such as students, graduates and employers is not conducted on a systematic basis at any of the programmes. This does not mean that feedback is not provided on sporadic basis. However, the programmes do not collect the information on a systematic basis.

Student progress is another important way to evaluate and ensure the effectiveness and quality of a programme. To the majority of the programmes, the exercise of gathering student progress information was a new and valuable exercise. There is no tradition for registration and use of student progress information. However, one of the programmes has started to conduct systematic student surveys, and another has started to use student progress information to review the extent to which the original aims of the programme remain appropriate.

All in all, the programmes recognise that quality assurance is necessary. It is evident that the evaluation has initiated internal discussions about how comprehensive ‘packages’ of explicit quality assurance mechanisms can be implemented and used to ensure high quality programmes.

3 Institutional reviews

The remaining chapters comprise institutional reviews of the physics programmes offered at Copenhagen University, University Paul Sabatier in Toulouse, University of Rome La Sapienza, Vienna University of Technology and Warsaw University. It is important to emphasise that the expert-panel has gained a positive impression of all five education programmes. This positive impression is not least due to the fact that all these universities have a reputation nationally for providing high quality programmes and are some of the best places nationally to study physics. This does not mean that the individual programmes do not have certain weaknesses. The reviews in this chapter need to be seen in balance with the general view that all five programmes offer a high quality education in physics.

The assessment of the programmes[1] has focused on the following three selected areas: educational context; competences and learning outcomes; and quality assurance. The programmes have been reviewed against a common set of quality criteria associated with each of the above three focus areas (see appendix A). In each institutional review chapter, every subsection introduces the criteria against which the institution is reviewed. This is followed by an analysis of the programme according to the documentation material (self-evaluation report and site visit). The panel have recognised many strengths and aspects in the programmes, which can serve as an inspiration to other programmes, and these are incorporated in the analyses of the self-evaluation reports and the site visits. Each subsection concludes with a recommendation which highlights areas that can be strengthened.

It should be mentioned that the documentation procured from the institutional accounts and the site visits is not entirely uniform. Some themes and aspects are highlighted more by one programme than another. This point is accentuated by the fact that the strengths and weaknesses of each programme have, to some extent, been reviewed in relation to the specific context of the institution and its national higher education system. Furthermore, the differences in feedback to the programmes are also a reflection of the difference between the self-evaluation reports.

Finally, some reading instructions. Whenever the report mentions that the source of evidence is interviews, it refers to the interviews conducted with the different stakeholders during the site visits. In addition, it should be specially noted that SER is an abbreviation of self-evaluation report.

4 Copenhagen University

4.1 Educational context

4.1.1 Recent developments in Danish higher education

Most university studies are structured as three years for a first cycle degree, optionally followed by two years for a master degree (cand. scient.), which, again, can be optionally followed by three years for a Ph.D. degree. In May 2003, a new law for Danish Universities came into force. The law stated that, where the educational structure has not already been altered, it is to be restructured in order to ensure genuine implementation of the 3+2 structure. That is three-year bachelor degrees followed by two-year master degrees (candidatus).

Furthermore, the law introduces a modular structure for all bachelor and master programmes. Students with a relevant academic bachelor degree must be entitled to enrol for an academically relevant master degree. Students with a bachelor degree will have a genuine choice between several relevant master degrees – also a choice of master degrees at another university. Academic relevance, correlation and progression must be ensured, and the programmes must have clear competence profiles, these being directed towards different jobs within the private as well as the public sectors.

The alteration of the content and structure of the programmes is to be a staged process taking place over a number of years and in accordance with a process that the universities and the Ministry of Science, Technology and Innovation will discuss and agree upon.

Finally, the law states that the universities are to become self-governing institutions. The executive management structure will be subject to reforms. The law obliges the universities to set up boards with external members, and employ rectors and managers[2].

4.1.2 National regulation

The structure of university studies is regulated by a ministerial order concerning the 3+2+3 structure. Further framework regulation exists for the natural science degrees. This includes a specific requirement that the bachelor study should be comprehensive (i.e. not only preparing for postgraduate studies but also for employment) and include a project lasting 1/6 of an academic year. There are also regulations about the entry requirements for the different subject areas. Finally, regulations lay down the requirement that 1/3 of the exams should be subject to external examination. Otherwise, it is left to the study committee to provide a detailed study programme for both the bachelor part and the master part. Thus, apart from the basic structure, there is a large degree of freedom in how to implement the structure.

4.1.3 Organisation of the programme

The vice-chancellor (rektor) is the head of the university, with a dean heading each faculty. Each faculty consists of a number of departments, each headed by a head of department. At each level there is also a board (university council, faculty council and institute board).

For each individual subject or group of studies there is a study committee. The study committee consists of an equal number of tenured teachers and students. This board appoints, from among the tenured teachers, a chairman of the study committee who is then directly responsible to the dean.

The study committee has to provide a detailed curriculum for the study for which it is responsible, and this curriculum has to be approved by the faculty. It also has to provide a study plan, which usually takes the form of a list of courses offered. That this is then carried out in practice is the responsibility of the head of department who, in turn, has appointed a teaching committee. The set of courses actually offered in a given semester is determined in collaboration between the teaching committee and the study committee.

4.1.4 Academic staff

In 1993, four departments were merged to create The Niels Bohr Institute for Astronomy, Physics and Geophysics, with a scientific staff of just under 100 covering a very large spectrum of subjects. It was thus possible for students to undertake a project or thesis in virtually any subject. A large variety of advanced specialised courses were offered. This is still the case, even though the permanent staff has now been reduced to around 60. With another ten taking early retirement during 2004, and the positions not being re-appointed, it may not be possible that such variety can continue.

The institute has, therefore, established a renewal programme for the staff, and this is described in detail. Special emphasis is given to the ‘generation change’ programme to cope with the fairly large number of retirements that will take place during the next ten years.

Table 1

Information on staff in total numbers

	Number of persons	Full-time equivalent (on teaching)
Full Professors	15	6,5
Associate Professors	46	17
Assistant Professors	0	0
Research Assistants	17	0
Teaching Assistants	0	0
PhDs	60	5
Other Categories	-	-
In total	138	28,5

Source: The SER Copenhagen University. In the table, full-time equivalents state only the full-time equivalents of time spent on teaching.

The ministerial directive regulates the rules for recruitment of new academic staff. Research qualifications are very important but teaching experience and qualifications are taken into consideration as well. To get a permanent position, some formal pedagogical training is usually required, e.g., the candidate must give a trial lecture.

A centre for didactics has been established which provides courses every year. According to Danish law, the universities must provide pedagogical training for assistant professors. The teaching courses provided for new staff members seem to be very comprehensive and done well. However, there are no pedagogical support programmes for professors.

The panel finds it positive that the programme management provides didactic courses for assistant professors. However, the programme management should also encourage participation of present staff in pedagogical training.

4.1.5 Admission

There are two ways of being admitted to a university study in Denmark. The first is by having a standard qualification, i.e. a high school exam or an equivalent degree with a well-defined set of subjects at a well-defined set of levels. The second way of access is non-standard, e.g. a qualification from a foreign country. There is a maximum number that can be admitted with non-standard qualifications and a committee makes the decision regarding admission.

In addition, the universities have the possibility of requiring a certain grade at the high school exam, or of setting the maximum number of students they can accept for each individual study. In physics, there is at present a maximum intake of 150 but no grade requirement. If the number of applicants is larger than the maximum intake, those with the highest grades from high school will be admitted. However, there are specific subject requirements, which have to be fulfilled. In physics, the students must have at least level A in mathematics and physics and level C in chemistry to be admitted. All the students applying for physics, who fulfilled the subject requirements, have been accepted independently of their grades.

Enrolment is centralised in the sense that each student only has to fill out one application form for entrance to a university study. The student’s priorities in terms of subjects and universities are listed on the form. A central system then enrols students according to priorities and available places.

4.1.6 Student intake

The students entering the physics programme usually combine two subjects, e.g. physics/mathematics. It is difficult for the Copenhagen University programme (CU)[3] to give the exact numbers of students studying physics, as the student could be enrolled under another subject, e.g., mathematics. The only reliable number that can be provided is the intake of students who, on registration, opted for a combination that leads to a bachelor degree in physics (and probably one other subject).

Table 2

Intake of students in total numbers

Year	Applicants	Admitted Male / Female
2000-2001	149	112/36
2001-2002	140	95/34
2002-2003	106	72/27

Source: The SER Copenhagen University.

There has been a small decrease in the intake during the last few years. As seen from the table above, the decrease has been considerable this year. Despite this decreased intake, it has been possible to increase the number achieving the master degree from around 25% per year to nearly 50% per year over a period of around ten years. There are at least two reasons for this. One is that the average time taken to obtain the degree has decreased considerably, and another is that the dropout rate has decreased.

This year a new subject area has been created, ‘nano-science’, in which the content of physics is reduced compared with ‘physics’. This has been a success with a fairly large intake, but with a correspondingly smaller intake for physics - a survey has indicated that less than half of those studying nano-science would have chosen physics had nano-science not been available.

The panel recommends that the programme management analyses why the students seem to find nano-science more attractive than the physics programme.

4.1.7 Student progress information

The dropout rate for the programme is fairly high, with up to 50% dropouts among the first year students who enrol for the programme. A large proportion of the high school graduates are not well prepared to adopt to the university teaching methods and subject level. This has been a matter of concern over the last decade and many initiatives have been taken.

Table 3

Student progress information in total numbers

Number of students whose admission year was 1999 and who were at present three years later in 2001
First year (1999)	11
Second year (2000)	38
Third year (2001)	46
No. of graduations	13
No. of dropouts	77
No. not in any identifiable year (*)	-

Source: The SER Copenhagen University.

(*) For those students, who cannot be placed in one specific academic year.

Table 4

Graduation in total numbers

Graduation in years calculated on the basis of ‘1997’ entry
Graduation in 3 years (official duration of the programme)	19
Graduation in 4 years (official duration of the programme + 1 year)	22
Graduation in 5 years (official duration of the programme + 2 years)	12
Still studying	35
Dropouts in the first year	55
Dropouts in total	86
Admitted in 1997	174

Source: The SER Copenhagen University. The categories ‘still studying’ and ‘admitted in 1997’ were added in the SER. The numbers are calculated on the basis of the 174 students admitted in 1997.

The faculty has initiated a systematic student activity survey consisting of a statistical study and a questionnaire survey in order to analyse student behaviour. According to the programme management, this survey has shown that the dropout number is to some extent misleading, as students who shift to another programme in the natural science faculty also count as dropouts. However, for those who actually do decide to drop out, the transition from being a high school student to becoming a university student was considered to be one of the main causes. Furthermore, the site visit revealed that there are dropouts due to difficulties in handling the first year mathematics.

In order to recruit more students to physics and to smoothen the transition between secondary school and university, the faculty has set up a Centre for Science Education, an H. C. Ørsted Youth Laboratory and a physics flying squad made up of physics students that promote physics in the secondary school.

It is positive that the programme management has managed to increase completion rates. However, they should set a completion goal for future planning. The panel encourages the programme management to continue making systematic records of student progression and investigate reasons for dropouts and transfers.

4.1.8 Student influence

The students have both formal and some real influence on the study programme and its content. On the study board, which is responsible for deciding the goals and content of the programme, the students make up 50% of the members. Furthermore, the vice-chairman of the study board is a student.

From the site visit, it is apparent that the students at the programme are very committed and involved in their study programme. They appear to be considered as an asset to the programme, and their opinions seem to be valued by the staff and management. The readiness of the management to listen to the students was also highlighted by the students. It was mentioned that student opinions are taken seriously. The self-evaluation group characterises the cooperation with the students as being extremely welcome and productive.

A tutor system was revitalised as a result of external evaluation. In addition, there is a special student advisory system for the natural science faculty where students can discuss social and structural problems with a specially employed ‘mature’ student.

The panel encourages the programme management to continue the valuable cooperation between students and management. The student representatives seem to make a considerable effort to improve the quality of the programme.

4.1.9 Employability of first cycle graduates

The Danish labour market has not yet established a tradition of employing bachelor graduates. There are, therefore, different expectations as to whether the labour market will accept bachelor graduates. This situation also applies to physics bachelor graduates. A recent study conducted by the faculty shows that a very small number of the physics bachelor graduates go directly into physics employment. The Danish labour market traditionally tends to ask for physicists with a master degree or even PhD. Therefore the majority of students continue on the physics master programme at the Niels Bohr Institute. An exception is the meteorology degree where weather forecasters normally go into employment with a bachelor degree.

Another recent study of the employment of graduates (with a cand. scient degree) shows that there is practically no unemployment among physics graduates. On the contrary it was pointed out that there would be a severe shortage during the next ten years, especially for high school teachers of physics. This is further enhanced by the employment pattern. Whereas in the eighties the majority of graduates went into high school employment, the percentage is now down to 15%, and the majority of these now find employment in the private sector.

The panel understands that the qualifications of the physics bachelor graduates might be unknown to the Danish labour market. However, the programme management should consider strengthening the profile of the bachelor degree by consulting stakeholders about the anticipated end product. Thus, building on the positive experiences of the meteorology degree. The programme management should ensure that the programme has a genuine exit point at the bachelor level. Finally, the programme management should maintain contact with the graduates and strengthen feedback with systematic recordings of their employment patterns.

4.2 Programme goals, structure and content

4.2.1 Programme goals

One criteria of the evaluation concerns the existence and documentation of programme goals. These are essential for several reasons. Goals provide prospective students with a more informed basis for their choice of study and support the aims of transparency. Explicitly formulated goals also provide teaching staff with terms of reference for designing content and selecting teaching methods for the different courses.

In a brochure sent out to all high schools, the aims of the programme are stated in fairly general terms. Likewise the aims are stated in the study handbook, also in fairly general terms[4].

The self-evaluation group states that, in fact, these aims are not very different from the Dublin descriptors for the first cycle. The programme does not in its aims really address the needs of the labour market. The programme was designed for progression into teaching or research, and dramatic changes in the labour market do not seem to have influenced the formulation of its goals. It is still a programme aimed at giving an academic grounding in the subject. One illustration of this is that CU does not state explicitly what the bachelor degree aims towards. It is unclear if the degree is directed towards employment, further studies or both. Furthermore, the subject specific and generic competences are only mentioned incidentally.

The panel encourages the programme management to further develop the programme aims based on the Dublin descriptors for first and second cycles. Also, the goals of the programme should be specified and formulated in more detail in consultation with external stakeholders.

4.2.2 Programme structure

The Niels Bohr Institute now provides a two-part physics programme in physics/ astronomy/ geophysics - a study combination the panel considers as refreshingly unusual. Physics plus another subject chosen by the students constitute the bachelor degree in physics.

Table 5

Programme structure

6. semester	Physics 3	Project	Optional subjects
5. Semester	Physics 3	Optional subjects	Optional subjects
4. Semester	Physics 22	Mat F1	Optional subjects
3. Semester	Physics 21	Mat F1	Optional subjects
2. Semester	Physics 12	Mathematics 1GB	Optional subjects
1. Semester	Physics 11	Mathematic s 1GA	Optional subjects

Source: The SER Copenhagen University.

It is typical to combine two subjects, e.g. mathematics/physics, physics/chemistry, physics/computer science, etc. It is, in fact, possible in the first year to combine mathematics with two other subjects, typically physics, chemistry or computer science. In the second year, the student then concentrates on two of the three subjects chosen in the first year. Within physics, there are also several combinations: Astronomy, geophysics and biophysics. There is, hence, a fairly large freedom of choice for the student. It is also possible for the student to change direction - with the probable effect of prolonging the study time. It is also possible to combine physics with a subject from another faculty, e.g. music or philosophy, and this is sometimes done. The students can also focus on just one subject.

A new programme structure is to be implemented across the entire Natural Science Faculty. The programme will still be organised as a bachelor-master structure, but a modular framework will be –beginning implementation start autumn 2004. In this new structure, the first two years are devoted to the main subject in physics and the last year to another subject.

The new structure of the BSc degree programme will have many effects, and many of these are somewhat uncertain. The new programme will split each semester into two sessions, and teaching will be more concentrated. Only two subjects at one time will be studied. This new development has not yet been fully worked out, and this is now getting late in view of its coming introduction. There seems to be lack of communication between the senior management of the faculty and the department on one side, and the teaching staff on the other.

As the new structure is yet to be introduced, it is difficult for the panel to comment upon it. However, the programme management should ensure that students are able to complete the study within the nominated study time with a suitable balance of subjects and appropriate progression.

4.2.3 Programme content

Clearly formulated and publicly available programme content. provide students with an overview of the programme and support the aims of transparency. A further criteria is the extent to which the composition of the courses and the curricula are consistent with the goals for competences, the extent to which the programme is characterised by progression and the extent to which the content reflects breath and depth.

The content of the physics programme is available to students on web sites. There is an established and effective student information system where the students can get information about all the natural science programmes and shop around for courses with a shopping basket (rucksack). Additionally, almost all teachers seem to have their own homepage with course descriptions, but the level of detail varies considerably.

The table below shows the core content of the programme. Depending on the minor subject of the degree, there are additional compulsory points and points for the students to choose.

Table 6

The core content

The core content

6^th semester

Subjects in classical and modern physics

10 ECTS free of choice

5 ECTS statistical physics and quantum mechanics or quantum physics.

Bachelor project

10 ECTS

5^th semester

4^th semester

Quantum mechanics

10 ECTS

3 lectures, 3 classes and 3 hours of lab. exercises per week

Mathematics for physics (provided by physicists)

10 ECTS

2 lectures, 2 exercise classes per week

3^rd semester

Electromagnetism

10 ECTS.

3 lectures, 3 classes and 3 hours of lab. exercises per week

2^nd semester

Theory of relativity; thermodynamics

10 ECTS

3 lectures, 3 classes per week and project

Basic mathematics

10 ECTS

4 lectures 4 exercises per week

1^st semester

Classic mechanics

10 ECTS

3 lectures, 3 classes and exercise classes

Basic mathematics

10 ECTS

4 lectures, 2-4 exercises per week

Source: EVA by gathering information from the student handbook. Only mandatory courses are listed. To this should be added the optional subjects free of choice for the students. Due to high level of student freedom to plan the study programme it is difficult to list the exact distribution of the content of the programme.

The panel considers that the content of the programme reflects a strong emphasis on theory – a solid grounding in theoretical techniques. The experimental side is therefore relatively weak, but this seems to be accepted by the majority of the students.

As the programme is structured at the present time, and according to the interviews with students, there seems to be progression. However, the panel is concerned that progression, may be threatened under the new structure, as all the core material is put into two years rather than three. This is a balance for consideration, as the new structure should also provide more flexibility regarding student choice.

The content seems to reflect both breadth and depth in each relevant subject in the sense that the students seem to develop fundamental knowledge of various approaches to the discipline area and can later go into depth within a specific area of interest at a more advanced level.

The transition towards a new programme makes it difficult to give recommendations concerning the content. It is clear to the panel that the programme management has chosen to provide a theoretical physics programme, which is in keeping with the Copenhagen tradition – and it is the strength of the programme. However, with regards to the tendencies that some applicants seem to find nano-science more interesting, that some of the graduates still go into teaching and that the faculty has invested a lot of resource in making youth experimental laboratories, the programme management could consider creating a more experimental path through the programme.

4.3 Competences and learning outcomes

One of the criteria of the project is the degree to which the programmes have formulated the expected competences of the programme. The programmes are asked if they have formulated overall competence goals for the first cycle degree, if these include both subject specific and generic competences and whether these are clearly formulated, publicly available, communicated to and known by students, staff, etc. Furthermore, they are asked to what extent the goals have been formulated and developed considering the needs and requirements of the labour market.

As stated by the self-evaluation group, and made apparent at the site visit, the term ‘competence’ is an unfamiliar concept at CU. Competence-terminology does not form part of any communication with students. Thus it is not made explicitly clear to the students what can be expected of them in terms of outcome as physics graduates.

It is part of the new bachelor structure to define competences for the different programmes. The self-evaluation group, therefore, considers the TEEP and the TUNING as starting points for understanding how to use the terminology.

The panel recognises that CU is in a process of formulating and applying the competence terminology. The panel suggests that the programme management begins the competence-defining process by formulating an overall goal for the bachelor programme .The Dublin descriptors could be used as inspiration. The next step could then be the definition of subject specific and generic competences at programme and course level. As inspiration, the panel suggests the programme management looks at the TUNING descriptors for physics and considers their applicability to the programme. Finally, the teaching staff should then ensure that the defined competences at course and programme level are communicated through student information system.

4.3.1 Subject specific competences

Throughout the programme, students should be able to obtain the subject-related competences through the compulsory subjects.

CU does not specify in the SER which subject specific competences are needed to achieve the overall programme goal. The subject specific descriptions provided for students describe the syllabus and examination procedures of the programme and the courses. As stated, the formulations are presently too broad and generalised to adequately fulfil the competence provisions in any detail.

The panel is aware that CU is discussing how to formulate competences for the programme. The panel recommends that the programme management specifies the subject specific competences that are expected of the students, both at programme and course level. As inspiration, the panel suggests the programme management looks at the TUNING descriptors for physics and consider their applicability to the programme. It is important that external stakeholders of the programme are involved in the discussions about required competences.

4.3.2 Generic competences

Throughout the programme, students should be able to develop generic competences, such as the capacity to learn, the capacity for analysis and syntheses, communicative skills, etc.

The SER states which generic competences can be expected of students at the programme, namely, the ability to learn and to structure knowledge; an understanding of the nature of models; and modelling and information technology. However, these generic competences are not made explicit to students and teaching staff.

It seems that the physics programme has the implicit goal of developing generic competences, such as problem solving. Both the students and the graduates stressed that they had acquired problem-solving competences. However, the graduates group called for more focus on competences such as communication skills and experimental experience.

The previously formulated generic competences, as stated in the SER, are a good starting point for a discussion of which generic competences the programme should develop. Furthermore, the programme management should make the expected generic competences explicit to students and staff. As inspiration, the panel suggests the programme management looks at the TUNING descriptors for physics and considers their applicability to the programme. External stakeholders such as graduates and employers (not only from research but also from other branches) should be involved in the discussion.

4.4 Teaching and learning methods

4.4.1 Teaching and learning strategy

Another criteria is the degree to which the programme have formulated and applied a strategy for the teaching and learning methods of the programme

Based on the interviews with the teaching staff, the panel concludes that the teaching group has not yet agreed on a common strategy, e.g. it appears from the discussions about the possible consequences of the new programme that some staff regard it as an opportunity for the development of new innovative teaching methods, whereas others see it as indicating a problem with existing methods.

From interviews with students, it also appears to depend too much on the individual teacher as to which teaching and learning strategy is applied, as does the responsibility to explain the goals of the courses and the programme.

In order to establish a common ground for the programme, a common teaching and learning strategy should be formulated at departmental level. The panel recommends that the teaching and learning strategy specifies the learning objectives of the different methods being employed while leaving some flexibility in approach in order to allow for individual skills and preferences.

4.4.2 Teaching and learning methods

An important dimension of the criteria for competences is the extent to which teaching and learning methods encourage the achievement of the intended learning outcomes in terms of discipline-specific skills and generic skills, employment and/or further study, and personal development.

The teaching methods at CU are quite traditional, being centred on lectures and small group problem solving. The SER states that whereas the intention behind the small group teaching was to make the students active, there is a tendency of the students to be very passive here, so the effectiveness could be improved. In comparison, the laboratory experiments, including projects, have the effect of making students more actively involved.

Table 7

Teaching and learning methods as a percentage

	1^st year	2^nd year	3^rd year
Lectures	42	42	40
Small group problem solving	33	42	40
Seminars	1	-	-
Course-work	13	-	-
Projects	4	-	20
Laboratory experiments	7	16	-
Trainee position	-	-	-
In total	100	100	100

Source: The SER Copenhagen University. Due to high level of student freedom to plan the study programme it is difficult to list the exact distribution of the teaching and learning methods of the programme.

The teaching and learning methods seem to encourage the achievement of discipline-specific skills, and these competences seem to be well taught. The students interviewed consider the strength of the programme to be the theoretical basis taught. However, they consider it a weakness of the programme that there is the lack of emphasis on group work and communication skills.

In addition, the graduates and students expressed that the gap between the teaching and learning environment of high school and the university is too wide. The gap is particularly apparent for mathematics. The students - especially in the first years - would prefer to have more personal feedback from the teachers regarding what competence level is expected and how each student is progressing towards it.

The panel recommends that the programme management considers introducing more laboratory work in the first semester in order to stimulate and maintain the students’ interest and active participation in physics. Furthermore, introducing group work and other learning strategies that encourage communication skills should be considered.

4.4.3 Assessment methods

Included in the criteria for competences are the extent to which the assessment processes of the programme enable learners to demonstrate the achievement of the intended outcomes.

The assessment methods seem to be varied, and encourage the learners to demonstrate the achievement of both subject-specific and generic skills. However, it is not explicitly stated which competences are assessed at each programme level, and which at course level.

Table 8

Assessment methods as a percentage

	1^st year	2^nd year	3^rd year
Written examination	30	90	30
Assessed course-work	20	-	-
Laboratory exp. write-ups	10	10	-
Essays	-	-	-
Oral Examination	20	-	50
Project reports	20	-	20
Presentation	-	-	-
In total	100	100	100

Source: The SER Copenhagen University. The category ‘coursework reports’ is not presented in the SER.

According to Danish educational law, each programme must assess at least 1/3 of their exams with the use of external examiners. The external examiners are chosen from a national list of examiners. The panel was informed by the students about instances in which the external assessment possibly did not function as intended.

The panel considers it positive that the assessment methods seem to be varied and, therefore, test different abilities. However, the panel recommends that the competences assessed for each course are made explicit to the students. Furthermore, the programme management should make sure that the external examiners are vetted in terms of competences and subject knowledge levels.

4.5 Quality Assurance

Another set of criteria try to establish whether the programmes have formulated explicit strategies for reviewing the extent to which the aims and intended outcomes of the programmes remain appropriate to factors such as: changes in student demand; student entry qualifications; employer expectations and employment opportunities, etc. In addition, ensuring that appropriate actions are taken to remedy any identified shortcomings.

The programmes are asked if the results of quality assurance are disseminated to students and staff, and if these parties are involved in discussing improvements to programme quality. The programmes can involve students, staff and other stakeholders in their quality assurance practice by utilising stakeholder input, student progress information and other feedback.

The new Danish university law specifies the universities’ obligation to constantly and systematically develop and improve the quality of its education programmes. The evaluations and plans for follow-up are to be specified in the university development contract, and are to be approved by the Minister of Education. Finally, the universities are to lay down clear guidelines for the documentation systems to be used in connection with evaluations and their follow-up.

4.5.1 Strategy, goals and procedures

The programme and the faculty have not yet formulated an explicit quality assurance strategy. However the formulation of such a strategy is now in progress.

At present, quality assurance consist of course evaluation and feedback from external examiners. The board of external examiners report on the content of the courses, and if this report indicates any issues, the study committee will take action. The chairman of the external examiners also issues a yearly report. However, it does not seem to be a very much used or effective quality assurance mechanism.

In addition, the programme is subject to regular external reviews by the Danish Evaluation Institute (the last time was in 1998). According to government regulations, the university (in this case represented by the study committee and, to some degree, the Institute Board) should study the recommendations of the evaluation, and report to the faculty on any action to be taken.

The structure of the programme as a whole is updated in an approximately seven year cycle by the study committee, and this focuses on correcting obvious mistakes and maintaining and improving the standards of the courses.

The programme management should build on the already existing quality assurance mechanisms and develop overall goals and procedures for systematic quality assurance with a view to producing coherent ‘package’ of explicit quality assurance mechanisms at course, academic year and programme levels. As part of this package, current quality assurance mechanisms should be extended to include more systematic external feedback.

4.5.2 Course evaluation

The programme seems to have developed a comprehensive and coherent framework for course evaluation, which consists of a standardised questionnaire and follow-up procedure.

Towards the end of each semester, questionnaires are distributed to the students.

In the questionnaire the students are asked to evaluate different aspects of the quality of each course on a scale from one to five. In addition, it allows the student to submit written comments on each course.

The statistics are compiled for each course and the written comments are passed on to the relevant teachers and to the chairman of the study committee. The statistics are made publicly available and are used systematically by the study committee to assess the individual courses with respect to structure, contents, and teacher performance. Low scores will initiate a discussion between the study committee, the teaching committee and the relevant teacher(s) with the aim of pinpointing the cause of the dissatisfaction.

According to the self-evaluation group, persistent problems as revealed by the questionnaires do lead to changes, e.g. changes of teachers have been recorded; a change in the content of a specific course was initiated some years ago (at the time, it was loaded with material - all interesting but leading to a too high workload for the students). Most issues are resolved by a talk with the teacher, while a few do give rise to conflicts.

However, the response-rate of the questionnaires does not seem to be satisfactory. Therefore an electronic evaluation system will be implemented on faculty. The self-evaluation group believes that this might give a larger participation rate, especially if it becomes compulsory when registering for the courses and exams.

The programme seems to have developed a comprehensive and coherent course evaluation system. However, the panel recommends that the programme management makes an effort to increase the response rate of the course evaluation and demonstrate that changes are made as a result of student feedback.

4.5.3 Feedback from employers and graduates

There is no established systematic procedure for feedback from graduates and the labour market. As a starting point, there was in 2001 an attempt to circulate a questionnaire among former students.

This implies that external feedback from employers is not used systematically as a tool to improve the programme. The study committee and the management have an occasional dialogue with employers and there are also employer representatives in the external examiner groups.

According to the self-evaluation group, student progress information is available at the faculty, but is not used. Statistics such as pass-rates for individual courses are compiled by the faculty office and are used by the study committee as additional material to the student evaluations.

The panel encourages the programme management to strengthen the dialogue with employers and graduates by establishing systematic feedback from graduates and employers and using this to improve the programme. The dialogue should be formalised and the results made public. The panel also encourages the programme to establish the student surveys already conducted on a permanent and systematic basis.

5 Paul Sabatier University

5.1 Educational context

5.1.1 Recent developments in French higher education

The French system of higher education consists of five different types of tertiary institutions: Les Instituts Universitaires de Technologie; Les Sections de Techniciens Supérieurs; Les Grandes Écoles; Les Universités and Instituts Universitaires Professionalisés.

French university studies used to be structured in three successive cycles, with national diplomas at the end of each cycle. The first cycle consisted of the DEUG degree (Baccalauréat + two years). The second cycle consisted of the licence (Baccalauréat + three years) and the maîtrise (Baccalauréat[5] + four years) preparing for the exercise of professional responsibilities. The third cycle leads to a degree of higher specialisation and training in research[6]:

With the main goal of implementing a 3-5-8 structure in the French higher education system, the French government has introduced a reform of the university sector as defined by a decree from November 2002. The major features of the reform are[7]:

· The creation of a new degree; the master degree oriented to towards either an
industrial profession or research

· Implementation of a 3-5-8 structure (LMD) Licence (bachelor) - Master -Doctorate

· The organisation of all higher education studies into semesters and course units

· Implementation of the ECTS and the quantitative value of new degrees

· The general principle of regular national assessment of higher education institutions
and their educational programmes as a perquisite for ‘habilitation’ (accreditation)

5.1.2 National regulation

The content and structure of the Université Paul Sabatier programme (UPS)[8] are selected according to a governmental ‘accreditation’ (‘habilitation’) of the programme. This is a sort of pre-accreditation, where the programme management presents a proposition for their programme based on assessment criteria. The programme is accredited every four years.

UPS was last accredited in 1999. Thus, the programme has drafted a new accreditation application for 2003 where the structure and content of the programme will be re-organised according to the LMD structure. The new bachelor programme has been approved by the Ministry and can be started in September 2004.

Another steering mechanism, which is important to comprehend in order to understand French higher education, is the so-called SANREMO system which determines the financial support for programmes. The financial support is calculated according to the H/E (Hours per Student). It evaluates whether the number of teachers’ hours corresponds to the teaching attended by a student during an academic year, taking into account the fact that lectures, exercises and laboratory exercises are not delivered under the same conditions. The H/ E also sets minimum and maximum standards for the number of students per class, e.g. 20 students maximum per laboratory teaching class, 40 students maximum per exercise class and 200 students per lecture class.

5.1.3 Organisation of the programme

Until now, the first cycle of the physics programme has been structured as a two year DEUG programme organised and taught jointly by the physics, chemistry and electronics departments. This is followed by a second cycle, a one-year licence study organised and taught by the physics faculty.

UPS has decided that the bachelor/master structure will be implemented during the next contract (accreditation) with the ministry for 2003-2007. Therefore, the first three years will be restructured. From 2003 the first cycle will include both the DEUG and the licence leading to a bachelor degree. The first semester of DEUG will be organised and taught jointly by the physics, chemistry and electronics departments, whereas the second, third and fourth term of the DEUG will be organised and taught by the physics department. In the new programmes there will be no implication of the electronics in the second semester and after. Only the first semester will be common to all DEUG students. As previously, the licence will be organised and taught by the physics department.

A project coordinator, elected by colleagues, who also prepares the accreditation form, coordinates each programme. The pedagogical council of the respective faculty includes representatives of students and staff and is responsible for the organisation of teaching, contact with students, evaluations, etc.

The panel considers the changes towards a stronger integration of the second, third and fourth semester DEUG and the licence as positive and a step in the right direction. However, the documentation material and the meetings reflect a significant lack of compatibility between the DEUG and licence programme. The governmental requirement for separate accreditation of the DEUG and Licence programme, along with the division of administrative and organisational responsibility, seems to be counterproductive for coherence and progression in the bachelor programme. The expert panel, therefore, recommends that the programme coordinators strengthen the coordination regarding formulation of the accreditation form and the implementation of the new programme in order to create a coherent bachelor programme.

5.1.4 Academic staff

The academic staff for the programme comprises professors (Professors), assistant professors (Maîtres de Conférences), amanuensis (Professeurs agrégés) and PhD Students participating in the teaching work.

The SER states that the change from the licence programme to the bachelor degree programme will not demand great changes in the recruitment policy of the university, nor will it necessitate a greater number of teaching staff since the number of students in physics has decreased in the last four years. The academic staff of the programme will essentially be the actual members of the physics department. According to the SER, the large number of specialisations present in the physics department ensures the possibility of developing a multidisciplinary curriculum leading to a bachelor degree in physics. Some members of the department are specialised in physics educational science and participate in the definition of pedagogical methods.

Table 9

Information on staff in total numbers

	Number of persons	Full-time equivalents
Full Professors	44	44
Associate Professors	0	0
Assistant Professors	62	60
Research Assistants	0	0
Teaching Assistants	2	1
PhDs	5	3
Other Categories	0	0
In total	113	108

Source: The SER Paul Sabatier University.

There is only one promotion possible in the career of assistant professor (from normal to exceptional class); there are only two stages of promotion in the career of a professor (from second to first class and from first to exceptional class). Local ‘ad hoc’ committees of faculty members, whose role is to evaluate and compare candidates, propose promotions to the higher grades. Moving from assistant professor to professor is not a promotion but a normal step in the recruitment process.

Local committees following a national procedure carry out recruitment of professors and assistant professors. Research qualifications are the main criteria for recruitment, but teaching references are also required. Local committees are composed of members of the faculty (elected (60%) and nominated). ‘Ad hoc’ committees nominate amanuensis’.

The situation regarding high drop-out rates, system inflexibility and budget cuts seems to produce a high level of frustration among the teachers, with a negative impact on the general teaching and learning environment. It appears that initiatives are left to a small number of especially motivated individual teachers with a high level of commitment to improving the programme and learning environment. However, this also appears to carry the seeds for creating a positive teaching environment.

Regarding coordination of teaching, there are two annual meetings between the subject teachers from the DEUG and the licence. However, the panel was left with the impression that the tradition for cooperation and coordination regarding development and practical issues varies according to the particular subject area and the motivation of individual teachers.

There seems to be a willingness from the programme management to support dedicated teachers. However, the programme management must ensure that the emerging positive environment created by some highly motivated teachers is encouraged so that it can grow and spread. In order to stimulate an open and inspiring teaching and learning environment, it is essential that the programme management supports the teaching staff in developing teaching and learning methods for the programme. In connection with this, it is also important that the programme management rewards and values good teaching, and reflects this in recruitment and promotion.

5.1.5 Admission

A matter of great concern to UPS is that they have to accept all students with a secondary education (baccalauréat) regardless of any perceived entry qualifications. Due to governmental restrictions, the university is not allowed to ask for specific entry requirements or to divide the students based on ability after entry. According to the UPS the lack of specified entry requirements is one of the reasons for the high dropout rate.

The self-evaluation group, staff and students are disturbed by the fact that the university is ‘second choice’ for many students. A large proportion of students choose the university if they have not been accepted at the ‘grand ecoles’. The university provides preparatory classes for the grand ecoles. Thus, the university can be used as a means of re-entry to the grandes ecoles. In particular, the staff expresses frustration at having to spend a lot of resources on students they never see in the system again.

5.1.6 Student intake

UPS has had difficulties in providing the data necessary to describe their student population. According to them the ‘Centre Informatique de Gestion’ does not have available figures of student intake for the different years.

Table 10

Student intake in total numbers

Year	Applicants	Admitted Male / Female
2000-2001	1179	458/ 276
2001-2002	1129	455/ 227
2002-2003	1176	-

Source: The SER Paul Sabatier University.

5.1.7 Student progress information

UPS states that there is a very high dropout rate in the first two years, but has not been able to provide figures on student progress. Therefore, the situation is also very unclear to the physics department, and not knowing the actual student success rate is dissatisfying for them as educationalists.

Table 11

Graduation as a percentage

Graduation in years
Graduation in 3 years (official duration of the programme)	47.8
Graduation in 4 years (official duration of the programme + 1 year)	27.9
Graduation in 5 years (official duration of the programme + 2 years)	10.3
Dropouts in the first year	N.A.
Dropouts in total	-

Source: The SER Paul Sabatier University. UPS has not stated which entry year the numbers are calculated on the basis of.

It is essential that the programme management is informed about the student population and their behaviour in order to plan the programme and to correct for inadequacies. It is, therefore, a problem that the university is not capable of providing student progress information available to the faculties and programmes. The programme management should ensure that the statistics department and the respective administrative departments make student progress numbers available to programmes.

5.1.8 Student influence

Student involvement in the development of learning strategies, planning of teaching and the development of learning contexts is, according to the SER, limited. Individually, there is almost no possibility for a student to plan his own workload. Students are represented by elected peers in some of the committees that decide teaching planning or development. Their participation in such committees is feeble, and they generally have little influence on the decisions made.

The panel encourages the programme management to ensure that students have real influence in matters concerning the programme in order for students to establish a sense of ownership and responsibility towards their learning environment.

5.1.9 Employability of first cycle graduates

UPS does not have available figures for the employment rate of the licence graduates. UPS states that according to anecdotal evidence most of the third year students go on to the fourth year (matrîce degree). Furthermore, many choose to do the teacher programme. The teachers and senior management mention that the programme has established a good reputation for their teacher graduates at the grandes écoles. According to UPS, teaching preparation classes at the grandes écoles is very attractive. At the site visit it was also evident that the students interviewed regards this as a very attractive path to pursue – both female and male students. Finally, the licence is a way to enter the grandes écoles (UPS estimates that 20% of the licence students use their degree to enter the grandes écoles).

The panel encourages the programme management to strengthen the dialogue with employers by making feedback formal and systematic, which would also make the apparently positive feedback visible to students and other employers. Furthermore, the programme management should consider establishing a mechanism that ensures systematic records of student employment patterns.

5.2 Programme goals, structure and content

The present programme consists of a DEUG Science or DEUG Science and Technology (2 years) degree and a one-year licence programme.

5.2.1 Programme goals

UPS is aware of the importance of goal formulation for the programme, as the governmental accreditation has a very strong focus on goals. However, the goals formulated in the accreditation do not seem coordinated between the programmes.

The licence 1999-2002 was, together with the fourth year, oriented toward the previous third cycle (profession and research). For the new bachelor degree (licence 2003), it is stated in the accreditation form that the primary goal of the licence is to lead to research based education, and that the natural way to pursue this is via the different local or national master degrees in physics, engineering and teaching.

The goal of the new DEUG’s second, third and fourth semesters, as provided by the physics department, is to complement the basic education in physics, which the students have gained during the first year. A further goal is to make more room for laboratory classes in order to raise the individual experimental capability of the students. The students should obtain a first hand orientation towards physics in the DEUG.

In the accreditation form for the licence 2003, UPS has attempted to formulate a set of goals for the desired core competence of the BSc graduates. It is stated that the third year should build on the knowledge of classical physics that has been obtained during the first two years. Furthermore, the third year should provide a basis in modern, theoretical and experimental physics necessary to continue to the second cycle. In this way, the overall goals and competence of the bachelor programme are very similar to the formulation of the Dublin descriptors.

It is positive that UPS, in the accreditation 2003, has attempted to formulate a set of goals for the desired core competences of the graduates in the upcoming BSc programme. However, it is not clear to what extent the goals will be publicly available, and the programme management should, therefore, ensure that the goals are known by programme management, teaching staff, bachelor students and incoming students. In addition, the DEUG and the licence programme should make an effort to coordinate goal and competence formulations to enhance progression and cohesion in the bachelor programme.

5.2.2 Programme content

5.2.2a The first two years of the present programme

The first two years of the programme consist of the DEUG (from 1999). The students of physics can choose between two programmes that give access to the physics licence: ‘DEUG Science' and ‘Science et Technologie industrielle’. As the first mentioned is the typical path to follow for a physics student, this programme content of the DEUG science is presented below in table 12.

Table 12

Programme structure of the two first year of the present programme

Semesters	Subject[9]		Lectures	TD[10]	TP[11]
First	UE Basic	Mathematics, mechanics, chemistry	60	66	18
	UE Discovery
	UE Methodology	Learning, project, collecting documentation, languages	8	12	28
Second	UE Basics I	Mathematics, mechanics, physics	48	48
	UE fundamental II	physics, chemistry	27	27	30
	UE Methodology of disciplines	Mathematics, mechanics, physics Computing Science, chemistry	20	20	20
	UE G	Scientific languages and culture	20	20	20
Third	UE Basics III	Mathematics, Computing Science, Languages	31	83
	UE Basics IV	physics, chemistry	47	49	18
	UO1	Chemistry , Physical chemistry Processes engineering physics Physical Sciences Applied Sciences	24 52 36 24 30	32 38 36 30 60	34 18 36
Fourth	UE Basics V	Mathematics, Computing Science, Languages	23	91

see continuation on next page

continued from the previous page

Semesters	Subject[12]		Lectures	TD[13]	TP[14]
	UO2	Chemistry , Physical chemistry Physics Physical Sciences Process engineering Applied Sciences	37 46 30 47 54	43 44 38 47 60	34 24 36 20
	UO3	Chemistry , Physical chemistry Civil Engineering Physics Astrophysics Engineering of chemical reactions Applied Sciences Mechanical engineering	32 58 90 90 33 12	37 20 33 48	21 12 24 30
	Total DEUG	Compulsory	284	416	134
		Optional[15]	337/456	528/496	223/192

Source: EVA by gathering data from the DEUG accreditation form 1999. The compulsory subjects are grey and the optional subjects are white.

As the material for the present DEUG programme (1999-2003) was not described in the SER, it is difficult for the panel to assess the breadth, depth and progression of the programme. It is positive that the physics faculty will have more influence on the DEUG degree’s second, third and fourth semester, which can only strengthen the integration between the different years of the bachelor degree. However, the panel is seriously concerned that the first semester of the bachelor programme will remain disconnected from the subsequent years.

5.2.2b The first two years of the new programme

Upon entry to the new programme which will begin 2003 natural science students are divided into two DEUG programmes: (1) Chemistry, informatics, maths and physics (CIMP); or (2) Earth and life science (SVT). The main path to a licence in physics is through the CIMP programme. The first semester is common to all students of the CIMP. In the second semester the students choose between four paths: a major in mathematics, mechanics and informatics; a major in physics and chemistry; a major in engineering; and a major in preparation for the entrance exam (concours) to the engineering schools. The main path for the physics students is the major in physics and chemistry.

After the second semester of the DEUG, students can choose to study the DEUG second year in physics or chemistry. The physics students follow the physics DEUG. Previously, students interested in physics would have to take the obligatory DEUG, designed for all CIMP students.

The teachers state that the goal of the reorganisation of the programme is to have better progression and vertical cohesion in the programme. However, according to the teachers, the DEUG and the third year remain disconnected due to the fact that the content of the DEUG first and second years is to a very high degree regulated by government.

Nevertheless, it appears from interviews with teaching staff that effort has been made across both the DEUG and the licence programmes to improve and reorganise mathematics according to student needs. Physicists will teach in relation to physics and mathematics.

Concerning the programme content of the present DEUG, students expressed the opinion that they would prefer more orientation towards physics and more freedom of choice in the programme in the first two years, e.g. the students consider that chemistry should be optional for physics students and not compulsory.

From the presented written material, it is clear that the new DEUG second, third and fourth semesters will be more oriented towards students with an interest in studying physics. The core element of the programme will be physics supported by the following subjects: mathematics, chemistry, didactics and languages. The programme thus consists of three elements: physics, support subjects and languages.

It is regarded as a positive aspect that the new programme seems to solve some of the problems addressed by students and staff, such as stronger orientation towards physics. Thus, the programme management should build on the great enthusiasm shown by teams of teachers in creating progression in the new programme and strengthen the coordination between the first and second semesters and the third and fourth semesters.

5.2.2c Third year

The third year is presently, together with the fourth year, oriented towards the French system’s third cycle - research. The duration of the third year is two semesters, consisting of eight units. Seven of the modules are mandatory and one is optional. There are two examination periods – January and June.

Table 13

Programme structure of the third year

Unit	Subject		Lectures	TD[16]	TDO[17]	TP[18]
UL[19] 1	Mechanics	Quantum mechanics	36	24	8
UL[19] 1	Mechanics	Mathematics I	24	16	8
UL 2	Waves		24	16	8
UL 3	Option	- Astrophysics - Meteorology -Electronics and telecommunications - Instrumentation - Oceanography - Other options (chemistry, languages)	24	24	24
UL 4	Electronics and Instrumentation	Electronics	21	21
UL 4	Electronics and Instrumentation	Instrumentation	18		6	36
UL 5	Material properties	Electromagnetic properties.	24	16	8
UL 5	Material properties	Mechanical properties	24	16	8
UL 6	Statistical physics and mathematics II	Statistical physics	30	16	8
UL 6	Statistical physics and mathematics II	Mathematics for physicists	18	16	8
UL 7	Language (English)		12	12	12	12
UL 8	Laboratory and Project	Laboratory				36
UL 8	Laboratory and Project	Project		24		36
	In total		255	185	98	84

Source: EVA by gathering data from the accreditation form 1999. The compulsory subjects are grey and the optional subjects are white.

In the balance of the courses there is an emphasis on the compulsory courses. Up to 80 percent of the courses are obligatory, whereas 20 percent are electives. According to the programme management, due to the governmental requirement of a 15-student minimum per class, it is expensive to set up an extensive range of electives. With the new programme, the range of elective courses has diminished even further.

At UPS the students complete their BSc studies with a BSc thesis of about 1.5 ECTS points. The size of the thesis will remain the same in the new programme.

UPS states that the new programme will not imply great changes in the content of the licence programme. The main changes are due to governmental requirements to reduce the number of lectures, increase the laboratory exercises and focus on English competences.

Following changes from the programme content of the licence programme from 1999 to 2003:

· Reduction in quantum mechanics lectures from 36 to 30

· Reduction in mathematics lectures from 24 to 18

· Course in relativity and nuclear physics is added in the 2003 programme

· Exercises in instrumentation are increased by 12 hours

· In physique de la matiere, the course ‘Propriétés mécanique ‘ in the 1999 programme is replaced by méchanique analytique, and mecanique des miliueux continues with an increase in lectures and exercises

· An extra course in the 2003 programme in scientific English

The programme seems to be actively progressing towards a more integrated bachelor programme in physics. They have started well when it comes to smoothing progression and cohesion between the three years. However, structural obstacles seem to counteract the ambition of a fully integrated bachelor programme. Cooperation concerning administration, exchange of student statistics, common goals, formulation and curriculum teams can be improved. The panel recommends that the programme management strengthens the cooperation vertically and horizontally between the three years, and considers the mathematics cooperation as an apparently good example to follow.

5.3 Competences and learning outcomes

5.3.1 Subject specific competences

Throughout the programme, students should be able to obtain the subject-related competences through the compulsory subjects.

Subject specific and generic competences have not been formulated for the present programme. Neither do the discussions with students and staff at the site visit indicate that competences have been formulated, communicated or used actively in connection with the teaching. The contents of the programme and courses are described in syllabus terminology.

However, detailed subject specific competences for each educational unit have been formulated for the new DEUG programmes in the third and fourth semesters. The subject specific competences are formulated in the accreditation form, e.g. the subject specific competences for the course in thermodynamics are, ‘Presenter les principes qui régissent la thermodynamique et comprendre l’évolution des systèmes simples.’

The subject specific competences formulated in the accreditation form are a good starting point for making the expected outcomes of the courses transparent to students and staff. However, it is important that the expected competences are not only known by the government, but are also communicated to the students. Furthermore, the expected competences should be actively referred to by the course teachers and be reflected in both the teaching and examination. Subsequently, subject specific competences should also be formulated at course level and for all three years.

5.3.2 Generic competences

Throughout the programme, students should be able to develop generic competences, such as the capacity to learn, the capacity for analysis and syntheses, communicative skills, etc.

The focus on generic competences, such as language skills and informatics, seems very much steered by the governmental requirements of the accreditation. The extent to which the programme encourages students to develop other generic skills, such as problem solving ability, the ability to work independently and in multidisciplinary teams, does not seem to be formulated or communicated to students or staff.

The panel is aware that the formulation and use of competences are new to the programme. The panel recommends that the programme management discusses and reflects on which generic skills the programme expects from graduates and then makes this clear to the students. As inspiration, the panel suggests the programme management looks at the TUNING descriptors for physics and considers their applicability to the programme. Furthermore, the programme management should formulate required competences through discussions with relevant stakeholders.

5.4 Teaching and learning methods

5.4.1 Teaching and learning strategy

Another criteria is the degree to which the programme have formulated and applied a strategy for the teaching and learning methods of the programme

UPS has not formulated a common teaching and learning strategy for the programme.

5.4.2 Teaching and learning methods

The documentation material gives the impression that the traditional lecturing style of teaching is the primary teaching style employed by the programme. Roughly speaking, the teaching method consists of 40% lectures, 40% exercise classes and 20% laboratory classes. To a great extent, the teaching methods remain the same in the new programme. According to the governmental requirements, there will be a slight increase in laboratory classes and a reduction in lectures. Furthermore, a 12-hour module on the history and epistemology of physics after Galileo will be introduced in order to give the students a scientific understanding of the roots of their subject.

Table 14

Teaching and learning methods as a percentage

	1^st year	2^nd year	3^rd year
Lectures	27.5	40.25	39.6
Small group problem solving	41.5	42.45	27.5
Seminars	0	0	0
Course-work	0	0	0
Projects	2	0	3.9
Laboratory experiments	23	14.5	15.6
Trainee position	0	0	0
Computer laboratory	6	2.8	13.4
In total	100	100	100

Source: The SER Paul Sabatier University. The table presents the categories used in the SER.

In the third year, the students have to write a bachelor project. Though the scope of the BSc thesis is limited (1.5 ECTS), it is nevertheless a project.

In general there seems to be very little emphasis on the development of different types of teaching at UPS. It also appears that the teaching staff receives little support and encouragement from the university towards the development of pedagogical methods. According to the students, the teaching is very traditional and old-fashioned, making very little use of modern technological equipment. The graduates called for more focus on problem solving methods and for tutorials to help the students through the studies.

Both the graduates and the students are very critical about the learning environment, finding that the relationship between teachers and students during the first three years is characterised by distance. The students find it difficult to approach the professors and to address both organisational and educational matters. All students interviewed asked for some tutoring in the third year. Furthermore, the students would prefer that mature students provide the tutoring offered in both the DEUG and the licence, as they are easier to approach with problems.

The programme management should reconsider the current teaching and learning methods. Investigations should be made to establish which of the different teaching and learning methods encourage the achievement of the intended learning outcomes in terms of subject specific and generic competences. Consideration could also be given to implementing more varied forms of teaching and learning, including more cooperative and communicative forms of teaching. More focus could be placed on methodological skills. Furthermore, the panel recommends that the programme management extends tutoring to the licence programme.

5.4.3 Assessment methods

Included in the criteria for competences are the extent to which the assessment processes of the programme enable learners to demonstrate the achievement of the intended outcomes.

Assessment is usually made through two types of internal examinations.

On most of the courses, there is a partial examination at mid–term and a final examination at the end of the semester. On other courses, there is a continuous assessment system.

Table 15

Assessment method as a percentage

	1^st year	2^nd year	3^rd year
Written examination	71.25	85.1	81.5
Assessed coursework	0	0	0
Laboratory exp. write-ups	25.4	14.9	11.5
Essays	0	0	0
Oral Examination	0	0	4
Coursework reports	0	0	0
Project reports	3.35	0	4
Presentation	0	0	0
In total	100	100	100

Source: The SER Paul Sabatier University.

The professor organises and prepares the examination in cooperation with the teaching assistant responsible for the exercise classes. A general set of regulations for University of Toulouse –’la charte des examens de l/UPS’ - lay down instructions for the organisation and conduction of examinations.

As it appears from the table above, there is a very strong focus on written examinations. The other methods of assessment are based on project reports and laboratory write-ups. In the interviews with the students, they asked for more varied methods of assessment and, in particular, more focus on oral examination in order to develop generic skills in preparation for employment.

The programme management should consider introducing assessment methods that focus more on developing competences. Furthermore, the programme management should consider how different methods of examination can be combined with teaching and learning methods in order to ensure the desired and expected competences of the BSc graduates.

5.5 Quality assurance

5.5.1 Strategy, goals and procedures

The fact that a framework exists for programme accreditation by the French government has without doubt influenced awareness concerning the quality of the programme. However, the documentation reflects that there is no overall explicit strategy for quality assurance at UPS.

The quality assurance at UPS constitutes an evaluation of the programme and the courses in 2001/02. The evaluation was a governmental requirement, and both the DEUG and physics programme conducted a survey among their students. At licence level, the evaluation was followed up by allowing two courses in quantum mechanics and optics switch places, due to criticism from the students. Furthermore, the students raised the issue of the continuous assessment tests. They were thought to be timed inappropriately. However, this was not changed.

Apart from the evaluation in 2001/2002, there is no systematic formal quality assurance at UPS, but there is a desire expressed by the self-evaluation group that participation in the TEEP-project will be a step towards establishing an explicit quality assurance strategy and system.

The programme management should build on the already existing framework for programme and course evaluation and develop overall goals and procedures for systematic quality assurance, with a view to producing a coherent ‘package’ of explicit quality assurance mechanisms at course, academic year and programme levels, e.g. the course evaluation system does seem to be functioning well, but this should be employed on a more regular basis, such as after each semester, instead of only every second year as at present.

5.5.2 Feedback from employers and graduates

The extent to which UPS systematically collects feedback from the labour market is very limited. The institution has not established a systematic procedure for regular feed back from the labour market at programme level. This does not mean that feedback is not provided on other occasions but only on sporadic basic.

Feedback from graduates is given on an individual basis. The institution has not established a systematic procedure for regular feedback from graduates.

UPS has no tradition for documentation and registration of student progress information. The self-evaluation group stresses that the data collection process connected to the evaluation has been helpful in learning more about student progress.

The panel encourages the programme management to strengthen the dialogue with the employers and graduates by establishing systematic feedback from graduates and employers and using this to improve the programme. The dialogue should be formalised and the results made public. The panel also encourages the programme management to continue the collection and use of student progress information.

6 Rome University

6.1 Educational context

6.1.1 Recent developments in Italian higher Education

The system of university study in Italy is currently undergoing an overall reform in terms of its structure and didactic system. Recent laws, aimed at implementing the Bologna declaration, have radically changed the general structure of the university programmes in Italy. Until now, studying for the laurea (the four-year degree), the official duration of the basic university degree was four to six years, depending on the field of study. From the 2001/2002 academic year, universities - having a large degree of organisational and management autonomy – have adopted a three study cycles: the first cycle, three years in length, is required to provide adequate mastery of general scientific methods and contents ,and the acquisition of specific professional knowledge and will conclude with the award of a first-level degree (laurea); the second cycle, lasting two years, concludes with the award of a second-level specialised degree (laurea specialistica); while the third cycle, the doctorate, requires three years.

Italian Law dictates course requirements, and the laurea is obtained after the student has passed a predetermined number of exams, gained the required credits and successfully defended a dissertation or thesis. Admission to the laurea programme is regulated by the general rules for university admission. Courses for master degrees may also be offered parallel to each study cycle. All study programmes must be based on the European system for the transfer of academic credits (ECTS) as provided for in recent agreements reached at EU level.

6.1.2 National regulation

Each university, in establishing a course of study, now decides on its own course title as well as the educational curricula to be proposed for the enrolled students. However, there remains the obligation to operate within a framework of national standards. The ministry responsible for universities sets out the general educational objectives and the minimum content in terms of disciplines for each subject degree in any particular class of degree. A third of the curriculum is left to the autonomous choice of each university, both in terms of teaching and learning.

The physics programme at La Sapienza (LS)[20] is going through a transition from a four-year programme to the 3+2 structure. As a consequence, the physics programme board has had its content restructured and reformulated. The present third and fourth year students follow the old programme, and the first and second year students are the first students to follow the new programme.

6.1.3 Organisation of the programme

LS is conducting a gradual transition, year by year, from the previous four-year programme to the new bachelor programmes. As a consequence, most of the data for the SER concerning the present and the future are given for the new bachelor programme, while all the data concerning the past relates to the previous four-year programme.

Starting from 2001-2002, the physics department offer three different bachelor programmes, namely, physics, physics and astrophysics, and applied physics and information technology. The courses of the first year are common to the three programmes. Furthermore, the courses of the second year are shared between the programme and the physics and astronomy programme. However, this evaluation focuses mainly on the physics programme.

Figure 1

Organisation of the programme

Source: The SER La Sapienza University.

The decisions on all matters involving the programme (activation of courses, assignment of courses to teachers, etc.) are taken by the council of the course of study (Consiglio di Corso di Laurea), which is composed of teaching staff and also a number of elected student representatives.

The departments involved in the programme are:

· The department of physics, which provides the classrooms, the laboratories and the funds required for these, the library, other services for students, the administrative staff as well as most of the teaching staff;

· The department of mathematics and the department of chemistry, which provide the teaching staff for the courses in their specific subjects

According to LS, the co-ordination is good and based on reciprocal services and a common effort for quality under the supervision of the dean of the faculty of sciences

The activities of the council of the course of study are supported by an administrative office. A didactical secretariat (Segreteria Didattica) administrates student activities that are specific to the programmes in physics. Other functions (enrolment of students, recordings of examinations passed, etc.) are performed by the central administration of the university.

6.1.4 Academic staff

There are three levels of permanent staff positions: Full professors (Professore Ordinario); associate professor (Professore Associato); assistant professor/lecturer (Ricercatore Universitario).

LS considers that the profile of the present academic staff matches very satisfactorily most of the aims of the programme, but only partially those concerning some specific aspects of professionalisation of the graduates, as there is little experience of employment activities outside the academic environment.

Table 16

Information on staff in total numbers

	Numbers of persons	Full-time equivalent
Full Professors	29	21.5
Associate Professors	46	31
Assistant Professors	19	19
Research Assistants	5	1.6
Teaching Assistants	-	-
PhDs	9	3
Other Categories	21	10.5
In total	129	86,6

Source: The SER La Sapienza University.

Every year the university provides overall budgetary guidelines, in terms of financial resources, for the development of the academic staff. The faculty and the department are entitled to decide upon areas where investments should be made on the grounds of research and development. The programme management has no explicit long-term plans. However, the management anticipates that in the years around 2010, a large part of the present staff will leave to retirement, offering an opportunity for a natural staff turnover.

Recruitment and promotion of staff occur when there is an opening at a given university for a given position. A commission of professors elected by the national community evaluates the applications for positions. Such evaluation is mainly focused on the research performance of the applicants, with only limited attention being paid to their teaching performance.

One problem seems to be some inadequacies in the following areas: the distribution of courses among teachers; giving credit to good teachers; and the support given to teachers to develop teaching and learning methods. As the system functions now, courses are assigned according to the teacher’s employment contract. Furthermore, the results of evaluations have no formal influence on the distribution of courses.

Another problem seems to be the absence of rewarding high quality teaching. Each year the faculty rewards teachers with a cash bonus. However, the bonus is awarded for the quantity of teaching, not the quality. It appears from interviews with the staff that they are unsatisfied with the fact that credit for teaching is linked to the quantity, and not the quality of the teaching. One teacher suggested that ‘the letter of recommendation’ could be used pro-actively to stimulate good teaching, by including results of course evaluations.

The third problem raised at the interviews with the staff is the lack of any pedagogical support mechanisms for teachers - either at university or faculty level. Teaching methods and the development of teaching are up to the individual teacher. The quality of the teaching is therefore very dependent on the willingness and motivation of the teacher group to provide high quality teaching.

As Italian law protects the didactic autonomy of teachers, it is difficult to give strict directives on what teaching and learning methods to use. However, this underlines the call for the management to establish support mechanisms and encourage good practice through the recognition of good teaching – including credit in the form of promotion.

The profile of the present academic staff seems to match most of the aims of the programme. However, the panel recommends that the programme management pays some attention to pedagogical competences in recruitment and promotion. Furthermore, it is suggested that the programme management establishes a course distribution system based on good quality teaching criteria, such as results of evaluations, etc. Also, the university should consider the appropriateness of the present cash bonus system. Finally, the programme management should set up a pedagogical strategy and development programme for staff.

6.1.5 Admission

According to the present Italian laws, no student selection mechanism can be applied. The university must accept all the applicants that hold a secondary school diploma.

In recent years, the physics programme has provided some guidance work in the schools of the region, providing information to help prospective students make choices in the light of their interests and their background, e.g. LS has developed materials to be used for presentations in the schools.

Prospective students are asked to participate in a self-evaluation test of basic mathematics and physics at the beginning of September of each year. In addition, the prospective students are offered about 20 hours of preliminary courses (pre-courses), held during the last two weeks of September.

As an illustration, 75 out of 174 participants in the self-evaluation test in September 2002 were considered to have an acceptable threshold level. In the same year some 100 students attended the ‘pre-courses’ – the actual enrolment rate in November the same year was 194 students.

The panel supports the initiative to let the students self-evaluate their competences and the setting up of ‘pre courses’ as an efficient way of assuring the necessary competences. Also, the initiative to promote physics in secondary school in order to stimulate young people to study the subject is strongly supported.

6.1.6 Student intake

During the last decade, there has been a consistent decrease in enrolments for the programmes of the Facolta' di Scienze in Italy[21] as well as for the programmes in physics (in Rome, from 448 to 173[22]).

The self-evaluation group believes that the decrease cannot be attributed to changes in labour market demands for physics graduates, as these have been relatively constant and favourable during that period. Very recently, since the start of the reform of the studies, there has been an overall reversal of the negative trend as regards the programmes in physics. In LS the numbers are: 173 (2000), 181 (2001), 194 (2002). However, LS does not consider that effect as primarily due to the introduction of the new programmes, as a reversal of the previous negative trend has also been observed in other European countries as well as in the USA.

Table 17

Intake of students in total numbers

Year	Applicants	Admitted Male / Female
2000-2001 (*)	173	173 (*)
2001-2002 (**)	181	181 (*)
2002-2003 (**)	194	194 (*)

Source: The SER La Sapienza University.

(*) The previous four-year programme.

(**) The new bachelor programme.

6.1.7 Student progress information

Two major problems immediately emerge from the data, namely, the high number of dropouts (between 50 -76%) and the long time required to achieve graduation, estimated between six and seven years, in contrast with the official four-year duration. These figures are quite representative of university studies in Italy and were the main reasons to undertake the radical study reform mentioned earlier. As the new bachelor programme only started in 2001, comparative data does not yet exist.

The self-evaluation group, however, states that an implication can be drawn from the fact that the proportion of students not passing any examination in their first year dropped from 48% in the previous four-year programme (2000/2001) to 21% in the first year of the new bachelor programme (2001/2002). Furthermore, the proportion of students passing all the examinations (respectively, four examinations for the previous four-year programme and ten for the new bachelor programme) in the same two samples increased from 20% in 2000/2001 to 25% in 2001/2002.

Table 18

Student progress information in total numbers

Number of students whose admission year was 1998 and who were at present three years later in 2001
First year (1998)	301 (**)
Second year (1999)	194 (**)
Third year (2000)	168 (**)
Fourth year (2001)	147 (**)
No. of graduations	1
No. of dropouts	154 (***)
No. not in any identifiable year (*)	-

Source: The SER La Sapienza University. Considering the students whose admission year was 1998 (the previous four-year programme).

(*) For those students who cannot be placed in one specific academic year

(**) Data from University administrative database.

(***) Students are automatically registered for the subsequent year independently from the number of passed examinations. The difference with the previous year gives directly the number of dropouts. So the number of dropouts (154) has been calculated by LS as the difference between 301 and 147, and has to be considered a lower limit, as LS expects that some of the 147 students registered in the fourth year will not graduate.

Table 19

Graduation as a percentage

Graduation in years calculated on the basis of ‘1993’ entry
Graduation in 4 years (official duration of the programme)	1.4 (*)
Graduation in 5 years (official duration of the programme + 1 year)	15.5 (*)
Graduation in 6 years (official duration of the programme + 2 years)	29.6 (*)
Graduation in 7 years (official duration of the programme + 3 years)	23.9 (*)
Graduation in 8 years (official duration of the programme + 4 years)	22.5 (*)
Graduated	23.8 (**)
Dropouts in the first year	35 (**)
Dropouts in total	<76 (**)

Source: The SER La Sapienza. Data concerning previous four-year programme, students enrolled in 1993/94.

(*) Proportion of total graduates up to now, data from University administrative database

(**) Proportion of total enrolled, as for 2002. LS expects more graduates in the future years. The total dropout is obtained as the difference and is then an upper limit.

According to the SER one of the main goals for the programme reform was to reduce the discrepancy between the official and the actual average programme duration (an average of 6.7 years for physics). The main reason for the delay was the mismatch between the students’ entrance level and the time needed to adapt to the learning pace and level required by the previous four-year programme.

Even though the implementation of the new degree, self-evaluation tests and ‘pre-courses’ might reduce the high number of drop outs, the panel recommends that the programme management records and analyses the reasons for the relatively high dropout rates and the prolonged study time, e.g. by analysing when and in which form support to students is most needed.

6.1.8 Student influence

Students are represented on different boards and seem, according to the documentation, to have good possibilities of gaining influence. There are two boards. The faculty council (Consiglio di Facoltà) has 15 students from physics programmes, and the council of the course of study of physics (Consiglio di Corso di Laurea di Fisica) has up to 17 students. In each case students are fully entitled to discuss the deliberations of these bodies as well as to formulate proposals. In addition, LS has established a body called the parithetical committee (Commissione paritetica di Fisica), composed of two students and two professors, which examines and expresses opinions on more detailed and specific matters concerning the implementation of the programmes. Furthermore, all students were, prior to the reform of the programme, invited to general meetings so that the management could explore their opinions and collect ideas for the implementation of the new bachelor programmes.

From interviews with students it is, however, evident that the first and second year students do not feel that they have very much influence on the planning of the programme and courses. This is due to the fact that the student representatives with influence are from the third and fourth year. Apparently, election procedures are an obstacle to the election of representatives from the first and second years.

The panel considers it positive that the students have possibilities for gaining influence and that the programme management makes an effort to include the students in programme discussions. However, the panel finds it problematic that first and second year students are not represented among the student representatives. Especially as the student cohort is now divided into the old and new programme students. The university should consider changing their election rules making it possible for students to be elected more often than every fourth year.

6.1.9 Employability of first cycle graduates

LS expects that their new bachelor graduates will be well received by the labour market.

The programme management has recently conducted a survey[23] showing that the employment rate of the fourth year graduates is 90 percent one year after graduation. In addition, anecdotal evidence shows that graduates with the old four-year programme are mainly employed in the electronics and space industry, in R&D laboratories and in computing and software firms. Recent Italian labour-market studies indicate that the demand for people possessing either generic or specialised competencies in these domains (respectively corresponding to the new first or second cycle physics graduates) should significantly increase in the next decade.

It is commendable that the labour market seems to have positive expectations of the new bachelor degree. The panel encourages the programme management to maintain systematic contact with the labour market and graduates in order to meet labour market needs. Furthermore, the panel encourages the programme management to continue to make systematic recordings of the employment patterns of the graduates.

6.2 Programme goals, structure and content

6.2.1 Programme goals

LS has formulated explicit goals for the physics programme. The goal is to provide the basics of the disciplinary area in order for graduates to enter the labour market or undertake further studies, e.g. the second cycle degree (and later the Ph.D.), the master degree, or enter the Schools for Teacher Training.

Upon completion of the new bachelor degree the graduates are expected to have acquired methodological skills and a large spectrum of competencies, as required in a wide range of professional areas. The graduate should be able to understand and analyse the behaviour of a complex system, picking up its basic elements, to build a model of it, to plan and perform measurements and to verify the validity of this model. These skills can be applied to any domain where specific knowledge and understanding of natural or automated systems is required, in particular the high technology domains. It should be mentioned that the programme goal is clearly linked to the expected outcomes formulated in chapter 6.3.

The panel finds that the goals of the programme, to a large extent, can be compared to the Dublin descriptors for a first cycle programme.

6.2.2 Programme structure

As mentioned earlier, the physics programme has just established a new bachelor programme. The programme has duration of three years and is divided into a trisemester structure.

It seems as if LS has given considerable thought as to how to structure the content in order to make the workload realistic, considering the official three-year programme duration. In all the official documents, the correspondence between ECTS, the teaching hours and time for individual work expected by the student is clearly defined for each course. For each type of course, the time division between lectures, laboratory and other work is strictly set out. All the course contents have been revised in detail to ensure correspondence between the amounts of material and the allocated times.

Nevertheless, it appears from interviews with the students that there are some inadequacies in the new structure. The physics department is one of the only departments having a trimester structure at the university, making the mobility of the students between other departments very difficult, e.g. the students have problems in following the English courses at the English department which has a two-semester structure.

The panel recommends the programme management considers the appropriateness of the trimester structure being an exception to the semester organisation of the rest of the university.

6.2.3 Programme content

One of the major strengths of the programme is that the content, structure and expected competences are available to students on web sites. The programme management encourages all teachers to publicly formulate the contents, the expected outcomes, the teaching and assessment methods and the preliminary programme of their courses. A standardised form has been produced for the teachers to fill in at the web site of each department[24].

From interviews with the students, it was evident that extensive information is available to students at the web site. As all students have access to a computer at home or at the university, they are able to gain access to the information. It seems that the students actually do use the web sites. Furthermore, the students appear to be very aware about the link between expected competences and the content.

As can be seen in 6.2.1 and 6.3, the physics programme at LS takes account of both the developments of subject specific and generic competences. In designing the new bachelor programme, some emphasis was given to the outcomes in term of generics skills and vocational training for future employment. At the same time, LS tried to keep the subject-specific skills required for further studies at the same level as provided by the old system. In order to fulfil requirements, and rather than reducing the depth of the treatment of the core content, some of the subject-specific contents have been deferred to the second cycle to create room for generic skills.

The self-evaluation group believes that the introduction of three different programmes (physics, physics and astrophysics, and applied physics and information technology) has provided more explicit indications of the intended outcomes.

The three programmes share the same core content, differing mainly in the third year where dedicated courses (e.g. electronics, astronomy, computational physics) are provided to characterise the curriculum. Moreover, the student can further adapt the programme as desired with the free choice of ten ECTS.

From the SER it appears that LS has chosen to follow another model of progression. According to LS the possibility of beginning with a set of introductory courses which would then widen and deepen the contents in the following years was carefully considered and then rejected after long discussions, mainly because time constraints would have excluded any serious treatment of modern physics in the new bachelor programme, thus strongly downgrading the scope of the programme. Some kind of progression is only applied in the sequence of mathematics courses where the basic contents are first given with some degree of formalisation, and further developments are treated afterwards in specialised modules, e.g. mathematical methods for physics.

The self-evaluation group states that the breadth of the programme is reflected by the space allocated to modern physics - in the second year, relativistic mechanics; in the third year, quantum mechanics, statistical mechanics, condensed matter, nuclear and sub nuclear physics. Basic courses in astrophysics and environmental physics (not covered in the previous four-year programme) have also been introduced. Creating more depth in some areas is beyond the reach of a three-year programme for average students. Bright students can deepen their knowledge in some fields by including some advanced courses, e.g. those of the second cycle, among their optional choices.

The panel considers the publicity policy of the programme one of its major strengths and recommends the programme management to encourage all teachers to use the tool. As the programme has been introduced very recently, and the students interviewed at the site visit are only in the first year of the new programme, it is difficult for the panel to assess the link between goals and content. However, the panel recommends that the programme management evaluates the overall programme content by conducting a survey among the first group of students in order to correct any shortcomings.

6.3 Competences and learning outcomes

The programme has formulated goals and competences both for the courses and the programme as a whole. The goals and competences are defined and published on the web sites[25]. According to students and staff, most of the staff use the web sites to communicate the competences for the individual courses to the students. Both the students and staff find this very useful.

Expected outcomes of the physics programme are:

· A deep knowledge of: classical physics (mechanics, electrodynamics and thermodynamics); the basic elements of optics, fluid dynamics and chemistry; and the fundamentals of modern physics, in particular, relativity, quantum mechanics and statistical physics

· A basic knowledge of electronics, condensed matter physics, nuclear and sub nuclear physics and astrophysics

· Some knowledge of fields of general interest, such as geo-physics or energy technology.

· The mathematical preparation required for comprehension of the above subjects

· Practical experience with modern measuring devices; to be able to design, realise, perform and write up a laboratory measurement

· To be able to collect and analyse data, calculate the statistical errors and estimate the systematic uncertainties

· To understand the basics of information technology, including computer architecture, networking etc.

· To be experienced in numerical problem solving techniques

· To have a working knowledge of English, and oral and written communication skills

· To be used to teamwork and to be able to work with the required degree of autonomy

The panel finds it positive that goals for competences are communicated and known by students, staff etc. Furthermore, the panel supports the inclusion of goals for both subject related qualifications and generic skills.

6.3.1 Subject specific competences

Throughout the programme students should be able to obtain the subject-related competences through the compulsory subjects.

LS states that the intention with the new bachelor degree is that, upon completion, the students should possess generic competencies and be able to understand and analyse the behaviour of a complex system, picking up its basic elements, be able to build a model of it, and to plan and perform measurements to verify the validity of this model. The new bachelor programme has been tuned to provide a good level in the aforementioned skills, in spite of its duration. This result should, to a large extent, be enhanced by the variety of compulsory subjects that are the core content of the three-year programme, namely, mathematics, classical and modern physics, chemistry, lab courses and computer science. Some diversification is, however, permitted by a few course units which are left entirely to free student choice, as well as by the final dissertation work.

Table 20

The balance between compulsory and optional courses in ECTS

	In ECTS
Core content	151
Course units which can be chosen by the student from a predefined list	12
Course units which are totally left to the free choice of the student	10
Final project/thesis work	7
Other compulsory elements (exams, project work, seminars, placements)	-

Source: The SER La Sapienza University.

The basic disciplines that underpin the subject-related competences can be quantified by the assigned ECTS as follows:

· Mathematics 35 ECTS

· Classical physics 25 ECTS

· Modern physics 32 ECTS

· Lab courses 40 ECTS

· Computer science 8 ECTS

· Chemistry 6 ECTS

The present written intention indicates that the compulsory subjects and basic disciplines seem to effectively support the attainment of subject specific competences in the programme. Further development will show whether the students are able to obtain the expected subject specific competences from the new programme.

6.3.2 Generic competences

Throughout the programme, students should be able to obtain generic competences, such as the capacity to learn, the capacity for analysis and syntheses, communicative skills, etc.

LS considers that several elements of the programme structure are intended to develop generic skills and competencies:

· Problem-solving instances are part of almost all the curricular courses

· Written examinations are introduced throughout the programme, also with some emphasis on formal aspects of clarity, logical consistency, synthesis, etc.

· Training in teamwork is pursued during laboratory classes, where most of the work is done by small groups of typically 3 students. This includes planning and performing experiments and measurements

· Compulsory requirement for lab write-ups, both for team and individual work

· Early introduction to computing skills in dedicated courses and in all laboratory courses

Due to government regulation, all university programmes have to formulate and provide courses that enhance generic competences such as Italian and English skills. The generic and linguistic skills are, according to the SER, provided through the course in Scientifical and technological communication.

After discussion with the students at the site visit it appears that the English courses – that are provided by other faculties – are not satisfactory.

The panel finds it positive that LS has formulated the generic competences expected from their graduates, and that the programme structure seems to support these. Further development will show whether the students are able to obtain the expected generic competences from the new programme. However, the English courses do not seem to be entirely effective. The programme management should try to ensure that the English courses are conducted in an appropriate manner.

6.4 Teaching and learning methods

6.4.1 Teaching and learning strategy

Another criteria is the degree to which the programme have formulated and applied a strategy for the teaching and learning methods of the programme.

LS has not yet formulated a common teaching and learning strategy for the programme.

6.4.2 Teaching and learning methods

The teaching and learning methods at LS consist of about 37% lectures, 23% small group teaching[26], 21% course work[27] and 17% laboratory classes. In the final third year, 13% of student-time is dedicated to preparing the final bachelor project (dissertazione).

Table 21

Teaching and learning methods as percentages

	1^st year	2^nd year	3^rd year
Lectures	37	38	36
Small group problem solving	23	30	22
Seminars	2	-	-
Coursework	21	18	14
Projects	-	-	13
Laboratory experiments	17	14	15
Trainee position	-	-	-
In total	100	100	100

Source: The SER La Sapienza University.

In each trimester, there are usually four courses in parallel, one of them being a laboratory course. Within each laboratory course 30% of the time is taken by introductory lectures, and the rest is shared among teamwork (typically three students), experiments and individual laboratory tests. All laboratory activities are reported in written form.

The dissertation (Dissertazione), to be presented by the student during the final examination for the award of the degree, is intended as project work, e.g., inspired by the activity of some of the third year laboratory work or a literature review. In this activity, the students are expected to demonstrate autonomy and, to some degree, investigative skills. The students are not expected to demonstrate effective research skills at this level.

Interviews with students and teaching staff indicate that the students would benefit from more focus on developing independency skills. The limited time for self-study seems to influence the lack of development of students’ autonomy. A positive aspect is that the teaching and assessment methods of the programme focus on providing team working skills and the testing of knowledge.

However, the programme management should also focus on oral skills and the ability of the students to work independently. In this connection it is unfortunate that the bachelor project, where the students have a chance to work more independently and in depth with a subject, only account for seven ECTS out of the whole programme.

Nevertheless, one of the major strengths of the programme seems to be the high level of commitment of the teaching group towards providing good teaching and motivating students. The students highlight the enthusiasm of the teaching staff. At any time, the students can request the assignment of a tutor, who can help to optimise individual learning processes and to handle any difficulty in the achievement of the intended learning outcomes. The students emphasise that their satisfaction with the programme is linked to the fact that teachers are very supportive and practise an open-door policy towards the students.

The enthusiasm of the teacher group is also evident in providing tools to encourage students to show up to classes, e.g. one teacher has set up a student–teacher contract, where both student and teacher are mutually obliged to show commitment to the course. This contract is used voluntarily by a number of the teachers who regard it as an effective tool for motivating students.

The panel recognises the high level of motivation of the teaching group and their positive attitude towards students. In order to ensure further development of the intended competences, the programme management should increase focus on teaching and learning methods that encourage the development of independency skills. The panel recommends that the programme management considers extending the scope of the project work.

6.4.3 Assessment methods

Included in the criteria for competences are the extent to which the assessment processes of the programme enable learners to demonstrate the achievement of the intended outcomes.

In the previous four-year programme, students were free to choose the time at which they took the examinations. This was often more than one year after the course attendance. Therefore, the preparation of examinations was mostly based on textbook reading rather than on lecture material and coursework. These facts prevented effective feedback on the achievement of the intended outcomes. In the new bachelor programme several actions have been taken to avoid late assessment. Among others, the practice of assessed coursework and intermediate written examinations has been introduced systematically throughout the courses. In this way, the teacher is continuously provided with feedback about student achievements and can adjust the pace and the level of the course accordingly. If a student passes a given number of assessed course works with adequate grades, exemption from the final written examination is possible. This practice should also encourage students to follow the courses with their coursework day-by-day.

Table 22

Assessment methods as percentages

	1^st year	2^nd year	3^rd year
Written examination	34	35	30
Assessed coursework	2	3	-
Laboratory exp. write-ups	10	9	9
Essays	-	-	6
Oral examination	39	42	41
Coursework reports	3	-	-
Project reports	12	11	10
Presentation	-	-	4
Total	100	100	100

Source: The SER La Sapienza University.

A positive aspect is that the programme management, in designing the new bachelor programme, has tried to correct the inadequacies of the old programme. However, from interviews with the students, it seems as if the number of written assignments is too high and the time between the end of the course and the examination is too short. The students are dissatisfied with the fact that they have very little time for self-study between the courses and the exams. The ‘course end – exam start’ interval should be chosen so that the students can assimilate the subject matter and investigate specific aspects in depth.

It is positive that the programme management, in designing the new programme, has tried to correct the inadequacies of the old programme. However, the panel recommends that the programme management considers the complaint of the students concerning the planning of the courses and exams.

6.5 Quality assurance

6.5.1 Strategy, goals and procedures

At present, the strategy for quality assurance at LS is devoted to quality assurance at course level. This strategy is linked to the national quality assurance requirements, where a body called Evaluation committee must be established within each university. A similar committee has also been established at the faculty of science. The bodies are mainly concerned with quality assurance of courses based on student questionnaires.

Formal procedures for evaluating quality at the programme level have not yet been adopted. However, the self-evaluation group has reported that actions have commenced for gathering data on the careers of individual students, which should provide useful information about the evolution of quality at the programme level. This evaluation procedure, pioneered within the department of physics, should become more effective as from the academic year 2002-2003, when individual student data from the central database will become accessible.

Further informal procedures for evaluating performance at the programme level have been established since the commencement of the new programmes, with an analysis of the pass rates for the single courses of each year as well as of the overall success of students in terms of the number of examinations passed or failed.

The panel considers that LS is well positioned for establishing a broader and explicit quality assurance strategy. The panel recommends that the programme management builds on the already existing quality assurance indicators and considers formulating overall goals and procedures for systematic quality assurance with a view to producing a coherent package of explicit quality assurance mechanisms.

6.5.2 Course evaluation

LS has developed a comprehensive and coherent framework for course evaluation, which consists of a standardised questionnaire and follow-up procedures. Furthermore, the teachers fill in a questionnaire on their own courses.

The responses to the questionnaires are made available to the students. However, the self-evaluation group interviewed a sample of students about the accessibility of the results and these students asserted that the evaluation results are ‘hard to find’. This is probably a consequence of poor information about the location of these results.

In addition the questionnaires are analysed by the chairman of the council of the course of study who produces a written report, which is discussed by the staff at a dedicated meeting, and takes any required action. Also present at the meeting are representatives of the students and they may put forward ideas for possible actions. The faculty evaluation committee, as well as the university evaluation committee, produce reports on the results of the evaluation for each academic year.

The self-evaluation group notes that, at the start of the new bachelor programme, the representatives of the students of physics performed a survey using questionnaires and held meetings to discuss the results. These tried to identify any shortcomings in the new courses.

The staff express great satisfaction with the course evaluation, which they regard as an important tool for the improvement of the teaching. This positive attitude towards course evaluation is essential as changes to the course content and/or teaching procedures cannot be introduced without the agreement and the cooperation of the person(s) delivering the course. There seems to be an overall atmosphere of cooperation among the teaching staff. The self-evaluation group also noted that most of the problems emerging from the course evaluation questionnaires in the recent past were confronted and remedial actions have been taken.

LS mentions that changes motivated by the identification of major problems with individual courses are adopted year by year. E.g., students concerning a first year course had identified some shortcomings. The remarks were considered by a commission set up for that purpose, which in turn decided to change the contents of that course and of another course in the following trimester by moving some topics from one to the other. Other remarks concerning another course were discussed with the teachers, leading to changes in structure and contents.

The panel finds it very commendable that course evaluation seems to result in evident changes and that the student representatives are involved in the follow-up process. LS seems to have established a good practise for course evaluations which both students and staff consider efficient in terms of content and follow-up. In light of the rules protecting the freedom of teaching, the panel believes it to be desirable that the programme management, building on the strong motivation of the teaching staff, develops alternative mechanisms for efficient correction of shortcomings.

6.5.3 Feedback from employers and graduates

There is no established systematic procedure for feedback from graduates and the labour market. As a starting point, an attempt has been made in 2001 to circulate a questionnaire among former students.

Feedback from the labour market is based on personal contacts with former students and it is not systematic. However, the available information has been taken into account in the review of the programme content, e.g. computer science related disciplines have been expanded in the transition from previous four-year programme to the new bachelor programme.

The programme management should make an effort to strengthen systematic feedback from graduates and employers. The dialogue should be formalised and the results made apparent.

The measurement of student progress is an essential quality assurance tool for reviewing the extent to which the original programme aims and intended outcomes remain appropriate.

It is therefore commendable that LS, since the start-up of the new bachelor programme, has collected information on student progress in a systematic way. In the future, data on students will be available to the relevant bodies from the general electronic archive of the university.

Furthermore, the self-evaluation group regards the evaluation process of gathering information and figures on a specific student cohort as important. They have been able to convince a reluctant university administration of the importance of providing these results.

The panel supports the initiative to set up a system for information on student progress and encourages the programme management to continue to expand the system.

In the SER there has been considerable effort to reflect upon how the current quality assurance practices could be more effective and efficient. It is evident that the programme management is very preoccupied with the issue and is in the middle of a period of change.

The self-evaluation group sees the monitoring of quality at course level, and examination pass rates, as of primary importance. This allows the identification of the main obstacles in the programme. Nevertheless, the self-evaluation group realises that a broader quality assurance strategy, which includes programme evaluation, will provide valuable information to improve the quality of the programme.

These practices are not presently adopted within the department of physics, but the self-evaluation gro