External Evaluator at Avocats Sans Frontieres France

Avocats Sans Frontieres France (Lawyers Without Borders France or ASF France) is a French NGO established under the French law on organisations of July 1st, 1901. Avocats Sans Frontières France use all the means available and can bring complaints before domestic and international institutions, and before judicial and non-judicial institutions, it can also intervene before them, including through civil actions before competent jurisdictions.

• Experience 5 years
• Location Abuja

External Evalaution  – SAFE Project

Avocats Sans Frontières France (ASF France) is running a project in Nigeria, in partnership with Carmelite Prisoners Interest Organisation and Nigerian Bar Association (NBA), which is funded by the European Union and the Agence Française de Développement. This project started in June 2019 and will end in November 2022.

In this context, an external evaluation is necessary to ensure the proper use of the funds, to analyse the effects and impacts of the project on beneficiaries, in order to draw relevant lessons for a possible new intervention of ASF France, and to promote a reinforced dialogue with its partners.

The objective is to formulate an independent opinion on the funds granted by the various donors in terms of changes, produced as a result of ASFF’s intervention, and to have a well-founded opinion on possible orientations/recommendations for a new request for funding.

SAFE aims to improve the human rights situation in Nigeria and focuses specifically on the serious human rights violations of torture, inhuman treatment, arbitrary detention and extrajudicial killings. To this end, the issues that the project seeks to address lie between the evolution of human rights at the legislative level and their application by state security agencies. The challenge of balancing human rights and security is reflected in the recent criticism of Nigerian President Muhammadu Buhari for saying that the security of the country comes before the rule of law.

1. Presentation of the project

The overall objective of the project is to enhance the promotion and respect for human rights and fundamental freedoms in Nigeria in situations where people are most at risk, to protect their human dignity.

The specific objective is to contribute to end serious human rights violations (torture, ill-treatment, arbitrary detention and extra-judicial killings) perpetrated with impunity by state security agencies in Kaduna, Enugu and Lagos States.

The expected results were:

1. The capacities of national actors are strengthened for the prevention, investigation, prosecution and sanctioning of serious human rights violations by state security agencies
2. State security agencies are held accountable for serious human rights violations and redress is obtained for victims through the strengthening of the NBA probono legal aid program.
3. Advocacy actions aimed at influencing public policies, social attitudes and political processes regarding serious human rights violations by state security agencies have been implemented.

The project was implemented in 3 states (Kaduna, Lagos and Enugu).

The implementation of the project was undertaken by the ASF France local team based in Abuja (Head of Mission, Legal assistant, Communication Officer and Logistics and Financial Officer) and by a network of Support Lawyers (one in each target State) providing free judicial assistance to victims of torture, arbitrary detention and family of extrajudicial killings victims. A team of volunteer ASF France members also interacted with the local team and participated in several activities. The overall implementation of the project was supervised by ASF France headquarters based in Toulouse, France.

ASF France also worked on the project with 2 partners: the Nigerian Bar Association (NBA) and the Carmelite Prisoners Interest Organisation (CAPIO).

The program consists of ten different activities:

• Training for Police and other state security agencies on best practices in respect of human rights;
• Capacity building trainings for HR CSOs and journalists on how to use non-judicial mechanisms, case documentation, advocacy, communication and legal monitoring;
• Reinforcement of lawyers’ technical skills in provision of legal assistance;
• Provision of legal assistance by qualified support lawyers to litigate, before national and regional courts, cases of serious human rights violations committed by the state security agencies;
• Coordination meetings for Support lawyers, NBA probono lawyers and partners to follow up legal assistance services implemented by the project;
• Petitions to international HR bodies on symbolic cases of serious human rights violations
• Roundtables with state security agencies, CSOs, relevant government ministries and NHRC to exchange best practices and experiences, make recommendations and propose solutions to the serious HR violations;
• Setting up a police and CSOs situation room to define strategies in order to improve respect for human rights by the Police;
• Common advocacy actions for the top state security agencies’ officials and key actors of the justice system at national and state level;
• Raising general awareness about the serious HR violations committed by the state security agencies, the fight against impunity and the situation and rights of victims of these violations;

This project involved direct and indirect beneficiaries:

Direct beneficiaries:

• Victims and families of victims of serious human rights violations, including torture, inhuman treatment, arbitrary detention and extrajudicial executions;
• State security agencies members;
• Human rights CSOs activists and journalists;
• Lawyers;
• State institutions members at regional and national level;
• Half of the population of Lagos, Kaduna and Enugu.

Indirect beneficiaries:

The population of Lagos (12.5 million), Enugu (4.4 million) and Kaduna (8.2 million) states

1. Content of the evaluation

The objective of the evaluation was to evaluate as exhaustively as possible the relevance, effectiveness, efficiency and impact of both results and objectives achieved (overall and specific) in comparison to those outlined at the beginning of the project.

The evaluation will precisely:

• Review activities implemented, results obtained and progresses observed;
• Identify all problems linked with the planning, the implementation, the monitoring and to any other operational managing tools used;
• Analyze strengths and weaknesses of the strategy of intervention;
• Evaluate qualitatively and quantitatively results obtained and their impact;
• Suggest any efficient and appropriate strategic adjustments to enhance the efficiency and the sustainability of actions as well as the impact on beneficiaries;
• Highlight any unforeseen effects that might affect the results of the project.

The evaluation is to be conducted in Nigeria.

The evaluation work will be composed of:

3.1 A global assessment of the project

The consultant will draft a global assessment of activities and objectives from the start of the project. This phase entails precise and detailed knowledge and understanding of the project, its evolution and its context. To this end, he/she will:

• Collect and read all information and all documents related to the project and specifically review the logical framework of intervention (objectives, results, indicators, critical hypothesis). Documents will be made available by the Head of Mission.
• Conduct interviews with persons involved in the conception, management and supervision of the project.
• Conduct interviews with beneficiaries of the project.

3.2 Conducting a qualitative analysis

On the basis of observations and available information, the consultant will evaluate the performance of the project from the 5 following criteria: relevance, effectiveness, efficiency, impact and sustainability. For each criterion, evaluation tracks are suggested below. The consultant will have to detail and complement those suggestions, keeping in mind that this is a final evaluation that must address the following issues:

• Relevance:

The relevance examines the legitimacy of the action according to the objectives and challenges set at the beginning of the project, in relation to the needs and problems identified.

• Have targeted beneficiaries really benefited from the project?
• To what extent did project managers adjust the conception of the project (including the logic framework and the structure of objectives) to make them as relevant as possible?
• Are current elements in the logic of intervention (logical framework) of the project appropriate/relevant in terms of resources, activities, results of the specific objective, of the overall objective and of hypothesis?
• To what extent were potential observations and recommendations of a previous monitoring mission or evaluation considered in order to reinforce the relevance of the project?

• Effectiveness:

The effectiveness examines the level of realization of objectives (technical, financial, institutional, as well as potential unforeseen positive or negative effects). The consultant will measure this aspect from the following questions:

• Have targeted beneficiaries had access to the results and services of the project?
• Are targeted beneficiaries using and taking advantage of these results and services?
• According to the implementation process, did the specific objective have a chance to be achieved according to the objectively measurable indicators?
• To what extent was the project adapted to unstable external conditions (hypothesis) so that the project may continue to benefit target groups?
• Did project managers take appropriate measures if and when beneficiaries suffered or were at risk of suffering from negative unforeseen effects of the project?
• To what extent did unforeseen positive effects influence the (quality of) generation of results and services offered?
• To what extent were potential observations and recommendations of a previous monitoring mission or evaluation considered in order to reinforce the effectiveness of the project?
• Efficiency

The efficiency will study the relation between, on the one hand, means and costs incurred and, on the other, outputs of the action. The evaluation conducted by the consultant will have to answer the following questions:

•  Availability of means and resources
• Were means and resources provided by all responsible parties within the suitable deadline, allowing activities to be implemented in a timely manner?
• Were means provided by all parties at the foreseen cost (or at the lowest possible cost)?
• Were resources subject to an appropriate monitoring for the management of activities at the lowest cost?
• Were resources of the project managed in a transparent and responsible manner to promote a fair and sustainable development?
• To what extent were potential observations and recommendations of a previous monitoring mission or evaluation taken into account in order to reinforce the quality of the project?
• Execution of activities
• Was a calendar of activities (or action plan) and a calendar of available resources provided and used by project managers?
• To what extent were activities implemented as foreseen?
• Were activities subject to a regular and satisfactory monitoring process? Were corrective measures taken when necessary (ie when new activities were required to be implemented due to the emergence of new needs or when activities were cancelled)?
• To what extent were potential observations and recommendations of a previous monitoring mission or evaluation taken into account in order to reinforce the quality of activities?
• Achieving results
• Have objectively measurable indicators (goals mentioned in the log frame) been achieved as foreseen?
• Have all planned results been achieved?
• What is the quality of the results obtained until now?
• Was the achievement of results subject to a regular and satisfactory monitoring process? Were corrective measures taken when necessary?
• To what extent were potential observations and recommendations of a previous monitoring mission or evaluation taken into account in order to reinforce the quality of the results?
• Contribution and participation of partners
• Were the partners able to implement the activities dedicated to them, according to the call for proposal?
• Was the communication between the local team, the headquarters and partners regular and satisfactory?
• To what extent have the partners been strengthened in project/financial management during the implementation of the project?

• Impact

The impact examines the results of the action. The consultant will mainly analyze the immediate effects on actors and particularly on final beneficiaries. To do so, the consultant will review the following questions:

In the current implementing condition, what is the probability for the project to have a positive impact on a large scale?

• What is the probability for the hypothesis related to the specific objective to be achieved so that the impact of the project is not reduced by external factors? The coordination of all sectors and donors must be specifically assessed.
• To what extent was the project manager able to analyse the positive and/or negative impacts of the project on the society and to take the appropriate measures to amplify the positive impact/reduce the negative impact?
• To what extent were potential observations and recommendations of a previous monitoring mission or evaluation taken into account in order to enhance the impact?
• Sustainability
• Economic/financial sustainability
• If services/results needed institutional support, were necessary funds provided?
• Were services offered at the end of the project affordable to final beneficiaries?
• Did responsible persons and institutions assume their financial and economic responsibilities?
• Has an exit strategy been defined and applied?

• What is the level of ownership by the beneficiaries and what will it be once the external aid is over
• To what extent was the project integrated in local/community structures?
• To what extent have beneficiaries and other interest groups/parties participated in the planning process?
• To what extent have target groups and relevant beneficiaries been actively involved in the decision process on the orientation and implementation of the project?
• What is the probability for target groups/beneficiaries to continue to use the services once the external aid is over?
• What was the type of political support and what is the level of interaction between the project and the political sector?
• What kind of support have national, sectoral or budgetary policies offered?
• Have any policy and priority changes had an impact on the project? Was the project adaptable and did it take into account the need for long-term support?
• To what extent was the project supported by the public and private sectors?
• To what extent did the project contribute to the democratization process (promotion of participation, accountability and human rights)?
• To what extent did the project reinforce the role of non-state actors as partners in the elaboration of policies and in their implementation?
• To what extent did the project contribute to the enhancement of institutional and administrative capacities?
• Was the project integrated to institutional structures which will be maintained after the end of the project?
• Were partners appropriately trained to ensure the continuity of the project (technical, financial, administrative)?
• What was the level of availability of qualified human resources ready for the implementation of the project, in comparison to what was initially foreseen?
• Was an exit strategy defined and applied?
• Transversal questions
• Did the project consider socio-cultural aspects?
• Was the project coherent with the perception of needs expressed by the local population?
• What was the level of participation of beneficiaries in the conception and implementation of the project?
• Did the project respect local customs? Have potential changes been accepted?
• Were there good relationships between project managers, beneficiaries and their representatives?
• How did the project consider gender equality?
• Were the content and methodology of the project gender-sensitive?
• Has the project been planned on the basis of a gender-differentiated analysis?
• Have practical and strategic interests of men and women been taken into account in the elaboration of the strategy of the project?
• Were interests of both men and women reflected in the implementation of the project among target groups, institutions and policies?
• What is the probability for enhanced gender equality after the end of the project?
• To what extent a gender sensitive approach (could) lead to a greater impact of the project?
• Were (human and technical) technologies introduced and used in the project appropriate?
• Were technologies flexible and understandable?
• To what extent technologies were inspired from the current practice and knowledge?
• Did they contribute to the development of knowledge and capacities at the local level?
• Were they managed to maximize the use of local resources?
• To what extent have environmental aspects been considered?
• Did the project respect environmental needs?
• Did the project manage its environmental responsibilities?
• Were parties and beneficiaries aware of the environmental responsibilities incurred by the project?
• Have environmental damages been produced or are likely to be produced during the project? What protection means have been implemented?
• To what extent did the project respect the traditional environmental practice successfully experimented?

1. Organization of the evaluation

4.1 Requirements

The required skills for this evaluation are:

• Education at least up to Master’s degree in one or more of the following disciplines: human rights, international relations, development studies, law, political science;
• At least 5 years of consistent and recent on-the-field, in-country professional experience in one or more of the following disciplines: human rights, international relations, development studies, law, political science;
• Consistent and recent on-the-field, in-country knowledge and technical expertise of judicial issues;
• Experience in conducting evaluation / preparing and conducting qualitative and quantitative research;
• Fluency in both written and oral English;

4.2 – Duration

The evaluation will be conducted within a maximum of 20 working days:

Activities	Working days
Desk research	1 day
Field mission (including international -where relevant- and internal travel time)	10 days
Presentation of preliminary findings	1 days
Drafting of the draft report	5 days
Inclusion of ASF France observations and finalization of the report	3 days

The final dates of the mission are to be set with Avocats Sans Frontières France.

A period of 2 weeks will be reserved for ASF France to provide comments on the draft report before the finalization of the report.

4.3 – Selection process

Candidates (organizations or individuals) interested in conducting this evaluation must provide by 31st October 2022, to the following address ([email protected]), the following elements:

• A note on the consultant’s understanding of the terms of reference and presenting the methodology that will be used;
• References and experiences of the consultant;
• A detailed CV with copies of similar evaluation conducted over the last 2 years;
• A provisional work plan;
• A financial offer including a total budget (taxes included or excluded) and detailed costs (expert fees, daily allowances, international/local transport).

Offers received and conforming to the above-mentioned elements will be analyzed according to the following criteria:

Specific experience of the consultant in relation to the mission; Adequacy of the suggested methodology and work plan according to the terms of reference; Qualifications and competences of the consultant for the mission; Table – Calculation of points for the evaluation of technical propositions                         Evaluation criteria Points   Specific experience of the consultant in relation to the mission 10 Adequacy of the suggested methodology and work plan according to the terms of references   40 Qualifications and competences of the consultant for the mission 50 Total 100           The minimum technical note for qualification (Nt) is:  80

The lowest financial proposition (Pm) will be attributed the maximum price note (Np) 100.   The price note from other propositions will be calculated according to the following formula:   Np = 100 x Pm/ P, in which “Np” is the price note, “Pm” is the lowest price, and “P” is the price of the evaluated proposition.   The weighting attributed respectively to the technical proposition (T) and to the financial proposition (F) are: T = 75 F = 25   Propositions are classified according to both their technical note (Nt) and price note (Np) using the weighting (T = weighting attributed to the technical proposition; F = weighting attributed to the financial proposition; T + F = 1) as follows: N = Nt x T% + Np x F%.

1. Deliverables

• An evaluation strategy which may be amended by ASF France before the field evaluation starts in agreement with the selected consultant;
• A presentation of the preliminary findings;
• A draft evaluation report;
• A final evaluation report taking into account ASF France observations. In case of diverging comments, these will be attached to the final report and commented by the consultant;
• Draft and final reports must be sent in electronic version. In addition, 3 hard copies must also be sent to ASF France: 13 Rue des Fleurs 31000 Toulouse, France. The final report will remain under ownership of ASF France’s. ASF France will be responsible for its dissemination.

Method of Application

Interested and qualified candidates should forward their CV to: [email protected] using the position as subject of email.