Instructions for the written evaluation
of individual projects in DARCOF II 2003

Practicalities and procedures
The Midterm Status Reports are available from the Midterm Evaluation homepage 2003 as are these instructions. You can find the reports by following the link to "Research projects in DARCOF II", choose one of your projects and choose "Progress report 2003". The files are in PDF format.

In order for you to better orientate yourself in the, often rather large, status reports, you should print out this short outline of how the reports are structured.

The numbers and names of the 3-5 projects that you are going to evaluate, were specified in the email we sent to you on the 4th of October. On the Midterm Evaluation homepage there is an "Evaluation Table" available for download in Word and RTF format. In this file there is space to fill in the evaluation of six projects (please remember to fill in project numbers and names). We expect to receive this file with your written evaluations before the 20th of October. This date is set so that your evaluations can be sent to the project leaders in order to give them a chance to write a comment if they feel that there are any misunderstandings. You will receive the written comments, if any, and if you think it is appropriate to change anything in the evaluation, we expect your revised evaluation on or before the 28th of October.

On the Midterm Evaluation homepage you will also find a link to the publications from the DARCOF II projects. The publications are in an eprint archive, where the full text of the publications will be available to you. Choose Browse and Projects, and find one of your DARCOF II projects in the list. You will then get a list of the publications from that project. It should be said that the Organic Eprints archive is being updated, and that the editor may not yet have processed the researchers papers. So if some of the publications listed in the status report are missing in the archive, you might want to take a look later on. In the Organic Eprints archive there is open access to all the publication abstracts, but some of the full text documents have restricted access due to copyright concerns. In our 4th of October email to you we wrote a username and a password that you will need in order to see the full text of those publications.


Criteria for evaluation
There are different stages in the 'life' of a research project. First there is the planning, application and approval stage, then different stages where the research is actually performed, and finally the 'afterlife' where only the research products live on.

For this midterm evaluation we want you to focus on four criteria for evaluation:

1. The quality of the research process (how the plans are met and adjusted, cooperation with other projects, educational aspects, etc.)
2. The quantity and quality of different kinds of research products (such as scientific, technical and popular papers and books, reports, workshops, public meetings, field days, etc.)
3. The reflexive objectivity of the project (the projects own critical reflection on plans, progress and results with respect to the 'workmanship' and relevance of the project and its parts, and with respect to other aspects such as research development)
4. Important aspects not included/suggestions for adjustments
   

The evaluation according to criterion 1 is primarily to be based on Section 7.C.2 and 7.D in the status reports; the evaluation according to criterion 2 is primarily to be based on Section 7.E and the eprints archive; and the evaluation according to criterion 3 is primarily to be based on Section 7.H, which is a kind of 'self-evaluation' of the project. See also the outline of the status reports mentioned above. Criteria 4, which is optional, is based on the report as a whole.

The two first and the last criteria and their evaluation is reasonably self-evident. Criterion 3, reflexive objectivity, is an overall methodological concept for how the contextual or holistic quality is secured without jeopardizing the quality of the scientific workmanship. Reflexive objectivity entails that the research project itself illuminates the value laden context that forms the basis for the project. For example how the project relates to the practical problems of organic farming and the organic principles and values, and how this context is shown in the research products. In this way it is made more clear a) that reduction in research is connected to value laden choices and b) which values enter into the more systemic (or holistic) research.

The importance of the three criteria vary across the life stages of a research project. The research process has greatest importance in the beginning and the middle of a projects life, and the products gradually gain importance, as an evaluation criterion, towards the end. The reflexive objectivity of the project is very important in the planning and approval stage, and then the project can be initiated on that basis without too much further reflection. Later on the reflexive objectivity again becomes important, as experience is gained and the world around changes.

The overall relevance of the project is not mentioned as a criterion, because the overall relevance of the project has been evaluated in the application and approval stage. The DARCOF II projects have all been subject to an international evaluation in connection with their approval. An important part of this evaluation was to evaluate their relevance in terms of their contribution to the overall objectives of DARCOF II. However, changes of relevance due to changes in the project environment, or due learning within the project, is included in evaluation criterion 3.