link to page 1 link to page 3 link to page 5 link to page 5 link to page 5 link to page 5 link to page 5 link to page 6 link to page 6 link to page 6 link to page 7 link to page 8 link to page 8 link to page 9 link to page 9 link to page 10 link to page 10 link to page 11 link to page 11 link to page 11 link to page 12 link to page 12 link to page 13 link to page 13 link to page 14 link to page 14 link to page 15 link to page 16 link to page 16 link to page 17 link to page 17 link to page 18 link to page 19 link to page 19 link to page 19 link to page 19 link to page 20
CONFIDENTIAL
ANNUAL ACTIVITY REPORT ON EXTERNAL AUDITS – 2008
TABLE OF CONTENTS
Annual Activity Report on external audits – 2008 Table of contents ________ 1
Executive Summary ______________________________________________ 3
1. Background _________________________________________________ 5
1.1. Introduction ____________________________________________________________ 5
1.2. Legal background _______________________________________________________ 5
1.3. The mission of the External Audit Units _____________________________________ 5
1.4. Relation with the control framework activities of DG Research __________________ 5
1.5. The 5th Framework Programme ____________________________________________ 6
1.6. The 6th Framework Programme ____________________________________________ 6
1.6.1. The FP6 Audit strategy _____________________________________________ 6
1.6.2. Mid-term review of the FP6 Audit Strategy _____________________________ 7
2. Activities ____________________________________________________ 8
2.1. Types and nature of the audits carried out ___________________________________ 8
2.2. Cross-RDG co-ordination _________________________________________________ 9
2.2.1. Coordination of audits in the Research family (CAR) _____________________ 9
2.2.2. Keywords database _______________________________________________ 10
2.2.3. Horizontal audit reference documents _________________________________ 10
2.2.4. Other Committees ________________________________________________ 11
2.3. Extrapolation __________________________________________________________ 11
2.3.1. Extrapolation policy and coordination ________________________________ 11
2.3.2. Extrapolation management _________________________________________ 12
2.3.3. Extrapolation implementation _______________________________________ 12
2.3.4. Extrapolation follow-up activities ____________________________________ 13
2.3.5. Further considerations _____________________________________________ 13
2.4. Collaboration with the European Court of Auditors (ECA) ____________________ 14
2.5. Reporting activities _____________________________________________________ 14
2.6. OLAF cases ____________________________________________________________ 15
2.7. Quality control tools ____________________________________________________ 16
2.7.1. The Audit Steering Committee (ASC) ________________________________ 16
2.7.2. The quality review process _________________________________________ 17
2.8. Collaboration with the DG RTD administration and finance (UAF) network______ 17
2.9. SAR and other IT developments ___________________________________________ 18
2.10.
FP7 Certification policy _____________________________________________ 19
2.10.1.
General Principles ____________________________________________ 19
2.10.2.
State of play of Certification files as of 31 December 2008 ____________ 19
2.10.3.
Supporting IT tools ___________________________________________ 19
2.10.4.
Inter-service collaboration ______________________________________ 20
30 March 2009
1
link to page 20 link to page 20 link to page 21 link to page 21 link to page 21 link to page 22 link to page 22 link to page 23 link to page 23 link to page 26 link to page 29 link to page 29 link to page 31 link to page 32 link to page 35 link to page 35 link to page 36 link to page 36 link to page 37 link to page 39
CONFIDENTIAL
2.10.5.
Communication activities ______________________________________ 20
2.11.
Coordination of relationships with external audit firms ___________________ 20
2.12.
Management of the audit framework contracts and their public procurement 21
2.13.
Other activities (Art.169 Initiatives/JTIs/Agencies) _______________________ 21
2.13.1.
Article 169 Initiatives - EUREKA _______________________________ 21
2.13.2.
Joint Technology Initiatives – JTIs _______________________________ 22
2.13.3.
Executive Agencies – REA and ERCEA __________________________ 22
3. Results and analysis __________________________________________ 23
3.1. Audit numbers _________________________________________________________ 23
3.2. Audit results ___________________________________________________________ 26
3.3. Analysis _______________________________________________________________ 29
3.3.1. Analysis of error rates _____________________________________________ 29
3.3.2. Analysis of adjustments at cost category level (FP6) _____________________ 31
3.3.3. Qualitative analysis of the largest adjustments in absolute terms (FP6) _______ 32
3.3.4. Qualitative analysis of error types (FP6) _______________________________ 35
3.3.5. Audit coverage (FP6) _____________________________________________ 35
4. Outlook for 2009 ____________________________________________ 36
4.1. Audit Targets __________________________________________________________ 36
4.2. Outlook of the audit activities in 2009 ______________________________________ 37
ANNEX I: mission statements _____________________________________ 39
30 March 2009
2
CONFIDENTIAL
EXECUTIVE SUMMARY
In 2008, the main objective of the DG RTD external audit activity has been further
implementation of
the FP6 Audit Strategy. While initially the focus was on the planning and
the execution of audits, the emphasis has been gradually turning towards the implementation
of audit results, their possible extrapolation and the analysis and correction of the errors
found.
At the end of 2008, the mid-term review of the Audit Strategy took place. Its main conclusion
was that the Strategy is delivering its expected outcomes. However, further developments are
required in four areas in order to ensure that the Strategy is completely and not only partially
successful. These four areas are:
Ensuring that an appropriate balance is achieved between audit efforts and the
strategic aim of "cleaning" 40% of the budget from systematic material errors.
The formula for the calculation of the residual error rate which appeared in the
original Strategy document is under review, and an agreement with the other RDGs on
potential changes to it has not been reached yet.
Extrapolation is an important management tool that requires the active co-operation of
beneficiaries and therefore efforts need to be intensified, particularly with regard to
tightening up procedures and liaising with beneficiaries.
The potential use of global recovery orders has not yet been agreed because of legal
constraints.
Much work has equally been done in the last year trying to clarify issues resulting from a
certain amount of interpretation possible within the regulatory framework. This has led to
multiple consultations with legal units and with other DGs, as well as the preparation of a
number of guidance notes and other instructions. Indeed, given the increased number of
audits, in-house and outsourced, it is important that towards the individual beneficiaries, often
grouped in lobby organisations, an identical line is taken.
The
activities of the external audit units have both widened and deepened. On the one hand,
traditional activities have been much strengthened, such as effective co-ordination between
the DGs of the Research family, liaison efforts with the external audit firms, the internal
quality control procedures of the audits or meaningful and timely reporting. On the other, in
2008 work related to new areas of responsibility has started:
OLAF's external enquiries, which were transferred to RTD.A.4 in February 2008.
Other assignments such as the ex-ante assessment of "dedicated implementation
structures" to which certain tasks of execution have been externalised.
Consultations with regard to the newly established "executive agencies" and "joint
technology initiatives".
Treatment of cost methodology certificates for FP7.
These new endeavours will potentially become an important part of our activities in the
future. In addition, collaboration with the Court of Auditors has not only been strengthened
but has also been extended to joint audit missions with the Court.
This report contains a wealth of information concerning
results in its Part 3. Of particular
interest is the cumulative FP6 error rate of 2,47% (to be compared with the FP5 error rate of
30 March 2009
3
CONFIDENTIAL
3,80% and the cumulative FP6 error rate at the end of 2007 of 1,42%). The representative
error rate is 3,13% (this is for TOP and MUS audits only); this figure will be used as the
starting point for calculating DG RTD's residual error rate which is the basis for the Director
General's Declaration of Assurance. 599 FP6 audits were closed by the end of 2008, and the
original multi-annual minimum target of 750 FP6 audits was increased to 896 to ensure a
better audit coverage. In addition, with the FP6 Strategy, the cost effectiveness of our audits
has substantially increased, with improved selection methods which maximise audit coverage
(now at 6,2% in terms of EC contribution).
Also in the past year, a more qualitative analysis of audit results has been applied and,
although more work in this area is needed, we can now provide a more informed view of the
nature of the errors found in our audits. So far, most errors detected are related to personnel
costs and overheads. In this context, the use of average personnel costs by some contractors is
a particularly important issue for which specific solutions are sought. Results also show that
most errors found are proportionally small, and that fraud and serious irregularities are
infrequent. Nevertheless, a number of cases result in large adjustments leading to an important
increase in the error rate.
The introduction of the FP6 Audit Strategy and an increased political scrutiny of the external
audit efforts of the Research family of DGs have substantially changed the landscape in which
the external audit units of DG RTD operate. In 2008, a lot has been delivered towards meeting
these new challenges. The following report summarises our main achievements.
Bearing in mind the number of FP6 audits already closed, the audit targets previously
communicated to the ABM and the limited number of auditable FP7 participations at this
moment, it will be necessary to partially review our audit activity in 2009. In this context, it is
recommended that the types of audits that are launched are changed. It is therefore proposed
to launch follow-up audits of extrapolation cases and specific audit assignments for the review
of average rates versus actual costs (in view of FP7), for which a tailor-made audit
programme is currently under development. The evolution to these more in-depth audits will
inevitably result in a reduction of the number of audits undertaken in 2009.
30 March 2009
4
CONFIDENTIAL
1.
BACKGROUND
1.1. Introduction
The purpose of this document is to report on the external audit activities in DG RTD during
2008, using the results of the verifications carried out and providing feedback on any
qualitative issues that may have come to light. It also aims to contribute to the opinion of the
Director General in DG RTD's Annual Activity Report on whether reasonable assurance
exists that the resources assigned to the activities carried out by the Directorate-General have
been used for their intended purpose and in accordance with the principles of sound financial
management, and whether the control procedures put in place guarantee with reasonable
assurance that the legality and regularity of the underlying transactions have been respected.
1.2. Legal background
The legal basis for the external audit activity of DG RTD is annex III point 2, paragraph 7 of
the Decision n° 1513/2002/EC of the European Parliament and of the Council, and article 18
of Regulation (EC) n° 2321/2002 of the European Parliament and of the Council.
The model contract for the 6th Framework Programme (Annex II, Article 29) states that: "
the
Commission may, at any time during the contract, and up to five years after the end of the
project, arrange for audits to be carried out, either by outside scientific or technological
reviewers or auditors, or by the Commission departments themselves including OLAF".
Similar provisions are in place for the 5th FP, as well as for the 7th FP but, in 2008, the main
area of work of the external audit units has been the 6th FP.
1.3. The mission of the External Audit Units
The external audit units, by means of financial audits to the highest professional standards,
provide a level of reasonable assurance to senior management and all interested parties,
including ultimately the Discharge Authority (European Parliament and Council), on whether
RTD contractors are in compliance with the terms of the RTD contract(s). By doing so, the
external audit activity contributes to the protection of the European Union’s financial
interests.
The responsibilities related to external auditing are attributed to two units: RTD.A.4 is
responsible for strategy and planning coordination, in-house on-the-spot audits and back-
office work1; RTD.A.5 is responsible for outsourced on-the-spot audits and implementation of
the audit certification policy. The mission statements of both units can be found in Annex I.
1.4. Relation with the control framework activities of DG Research
The ex-post audit activities need to be seen as part of the overall integrated control framework
put in place by the Directorate General. Internal control activities include all ex-ante and ex-
post evaluations, controls, financial and scientific verifications and monitoring tools.
1 Back-office work refers to a number of tasks in support of the auditing function including information
sy+stems and data maintenance, batch preparation, extrapolation, management reporting and a variety of
administrative tasks.
30 March 2009
5
CONFIDENTIAL
Accounting transactions included in the cost statements are processed through the internal
control systems of beneficiaries and checked by their certifying auditors, who then issue an
audit certificate. These transactions are also monitored by the Commission's Project Officers
(scientific and financial) even before the arrival of the cost statements, and thereafter checked
by means of desk reviews before payments are made.
This control chain, which operates before any ex-post financial audit is carried out, has to be
considered in the overall evaluation of risk and of the external audit results. Close cooperation
exists between auditors and operational units in the preparation phase of an audit, as well as in
the implementation phase (contacts through the Audit Liaison Officer in order to obtain an
agreement concerning audit findings and their implementation). In DG RTD no audit is closed
which has not been expressly agreed upon by the Operational Service.
A possible future control layer in the form of technological and scientific audits is currently
under discussion2.
1.5. The 5th Framework Programme
In order to fulfil the auditing requirements of the 5th Framework Programme, RTD's audit
policy was mainly based on random sampling, and partially on risk assessments. The
underlying assumption was that, provided that the sample was large enough from a statistical
point of view, relevant conclusions could be drawn for the whole population3. DG RTD
decided that a sample of around 10% of contractors should have been audited over the
lifetime of the Framework Programme. With this target, the Research DGs aimed at giving a
representative picture of their contractors’ population.
The Commission, and in particular DG RTD, eventually recognised that the 10% target for
FP5 was unrealistic but also that there existed enough justification for not having achieved it4.
In 2008, FP5 audit activities were progressively phased out.
1.6. The 6th Framework Programme
1.6.1.
The FP6 Audit strategy
A comprehensive audit policy for FP6 was first established in February 20045. DG RTD's
audit strategy was first endorsed at DG RTD level6, then agreed with the other RDGs7, and
finally approved by the ABM Steering Committee in March 2007. It is considered a
'corporate' Commission strategy adopted by all the Research DGs (RTD, INFSO, ENTR,
TREN), but it is being implemented with due consideration to DG-specific circumstances.
Any substantial amendment is to be approved at the level of the ABM Steering Committee.
2 End 2008 DG RTD has started to follow-up the conclusions of the ad-hoc working group created in 2006, for
defining the scope, feasibility and possible synergies of scientific and technological audits, separately from
project reviews and ex-post impact assessment.
3 From a practical point of view, the selection of contracts to be audited was made on a random basis (75%) with
an additional 25% of audits selected on a risk analysis basis.
4 This has been stated by Commissioner Potočnik to the COCOBU hearings on 27 and 28 November 2006, and
by Commissioner Kallas to the European Parliament plenary session on 14 November 2006.
5 See document Adonis D/504513 of 4.2.2004 "Common audit policy of the Research DGs for FP6".
6 See document Adonis I/3094 of 14.2.2007 "Ex-post audit strategy for FP6 – Period 2007-2010".
7 See mail from A.4 to the Sec. Gen. of 16.3.2007 "Ex-post Audit Strategy of FP6 common to the RDGs".
30 March 2009
6
CONFIDENTIAL
The overall objective of the FP6 audit strategy is to contribute to sufficiently reasonable
assurance to support the DG RTD Annual Activity reporting exercise, including the
Declaration of Assurance by the DG RTD Director General.
Hence, the common FP6 audit strategy is based on three pillars which aim to provide this
reasonable assurance as well as to have sufficient corrective actions.
The first pillar is based on the hypothesis that the 50% (originally 40%) of expenditure
received by a relatively small number of beneficiaries contains a certain proportion of
systematic material errors and that by auditing three contracts per organisation these
systematic material errors will be corrected by extrapolating the audit results to all
non-audited contracts.
The second pillar focuses on estimating the level of error present in the remainder of
the population by auditing a representative selection of beneficiaries from this portion
of it. This selection has been made using the monetary unit sampling (MUS)
technique.
The third pillar covers audits selected on the basis of a risk assessment methodology.
1.6.2.
Mid-term review of the FP6 Audit Strategy
The audit strategy mentioned that a mid-term review would be carried out in 2008 to assess
whether the strategy is functioning as intended. The mid-term review's main conclusions are
incorporated in this report.
In the first two years of implementation of the FP6-audit strategy, which covers four years in
total (2007-2010), the focus has been on increasing the number of audits, improving the
consistency of approach and coherence of conclusions, more homogeneous audit policies
(including reporting and documenting), calculating a reliable error rate and introducing the
extrapolation procedure.
All the RDGs believe that the corporate FP6-audit strategy is delivering its expected auditing
output satisfactorily. However, the "cleaning" of a significant part (40-50%) of the budget
from systematic material errors is proving to be, as expected, the most challenging objective,
but it is progressively being achieved. A number of policy issues have a potential impact on
reaching that target, such as defining the necessary auditing effort required before considering
a budget as "clean". These policy issues are being analysed among the RDGs to establish a
common line.
Here are some additional main points of the review in summary:
1. The process of extrapolation has turned out to be very time-consuming and labour-
intensive (see section 2.3).
2. Further development of IT-supporting tools remains essential for an effective and efficient
implementation of the Audit Strategy. The RDGs have listed what remains to be
developed in the SAR Action Plan. The importance of the development of these tools is
clear, in particular as a means to enhance the efficiency and effectiveness of extrapolation.
30 March 2009
7
CONFIDENTIAL
3. The need for a coherent application of the liquidated damages' clause in FP6 is being
developed progressively, but this is of course a matter for AOSDs, and not immediately
for the external audit units.
4. The increased number of audit results is at this point moving the emphasis of the Audit
Strategy from the detection of errors to their correction.
5. The increased number of audits and the effectiveness of the audit approaches are also
generating a higher number of contested cases, some of which are leading to legal cases.
These will require more attention in order to defend the financial interests of the
Commission in the most effective manner and to reinforce the dissuasive impact of the
Audit Strategy.
6. Despite constant coordination efforts, the 'corporate' character of the audit strategy
reaches its limits in the independence of the four AODs.
2.
ACTIVITIES
2.1. Types and nature of the audits carried out
The external audit units perform audits in accordance with the FP6 Audit Strategy. The bulk
of audits corresponds to the three strands of the FP6 Audit Strategy, which are:
TOP: this is a selection of the beneficiaries which receive the most money from the
Commission. The RTD list of top beneficiaries consists of 243 contractors which
receive 50% of the FP6 budget managed by RTD. All beneficiaries in this sample have
been audited at least once (on at least three participations) and, where necessary,
further audits are carried out in order to confirm the presence or not of systematic
material errors for each beneficiary.
MUS: using a monetary unit sampling technique to ensure statistical representativity, a
selection of 161 beneficiaries was made from the non-TOP RTD population. One audit
is carried out for each of them.
RISK: a number of different criteria have been used to select the beneficiaries in this
strand. The audits of this strand are intended to have a corrective effect on the amount
of errors present in the RTD population. The results of these audits are not taken into
account in the calculation of the representative error rate.
There are, however, additional auditing commitments in the following areas:
FUSION: a limited number of national bodies in the field of nuclear fusion are part of
the FUSION programme, implemented through Contracts of Association between
them and EURATOM (represented by the Commission). The current arrangement
with Directorate J is to audit them all on a cyclical basis. The intention is to conduct
one audit per contractor at least every three years. These audits are part of FP6.
Coal and Steel (C&S): a small number of audits is carried out on beneficiaries which
receive monies from the Research Fund for Coal and Steel (RFCS), which is managed
by DG Research, Directorate K. The first two audits of this type were closed in 2008,
and more have been launched. In addition, an agreement has been reached to make the
30 March 2009
8
CONFIDENTIAL
selection of beneficiaries more representative in the future. RFCS contracts do not
follow the provisions of the Framework Programmes, and therefore these audits are
considered as not FP-related.
Audits on Request: audits in this category are performed at the request of the
operational services, and they are normally quite specific in their scope. These
requests are discussed in regular meetings, and not all are accepted.
Joint audits with the Court of Auditors: a number of these audits were carried out in
2008 for the first time (see section 2.4).
Coal & Steel audits and joint audits with the Court are always done in-house by RTD's own
auditors. Other audits can be either done in-house or outsourced to an external audit firm,
under the framework contract. This last type is known as 'batch' audits. Finally, a small
number of selected beneficiaries can be in non-EU countries ('third country audits').
2.2. Cross-RDG co-ordination
The adoption of a common corporate FP6 audit strategy means closer coordination between
the RDGs in a significant number of areas. DG RTD is the chair of a number of committees,
and also provides the secretariat. This requires a significant investment of resources.
2.2.1.
Coordination of audits in the Research family (CAR)
The RDGs have established a 'Co-ordination group for External Audit in the Research family'
(CAR) in order to coordinate strategic and policy matters and ensure a coherent approach to
contractors and to the external audit firms. Chaired by RTD.A.4, the CAR convenes on
average once every two weeks.
The activity scope of this group is:
Common positions and communication towards internal and external parties;
Common guidelines to auditors on specific audit issues requiring an interpretation;
To update the audit related common documents such as: Audit Strategy, Audit Manual,
Audit Certificate Handbook and Guidance Notes;
To agree on professional issues such as selection methodologies and tender/procurement
procedures, technical audit procedures, issues related to the extrapolation policy;
Specific exchange of information and common IT-tools;
Common training for staff/external auditors.
In 2008, the CAR met 17 times. The meetings focused on the implementation of the audit
strategy and on reaching agreement on audit policies. The related minutes give an excellent
overview of the topics that were discussed at the CAR.
30 March 2009
9
CONFIDENTIAL
2.2.2.
Keywords database
A significantly increased number of audits imply an increasing number of questions, issues
and concerns raised by the auditors (in-house and external), the operational services and the
contractors on legal or contractual provisions.
The need to 'speak with one voice' towards beneficiaries is essential. This 'one voice' is to be
agreed first among the RTD audit units, and then among all RDGs. Clear instructions were
also needed on how and when an auditor, in his/her capacity as an EU official, could reply to
a question coming from contractors knowing that these will interpret such replies as a formal
position of the Commission services.
Units A.4 and A.5 have therefore established a 'Keywords Working Group' (KWG) at the end
of 2007 in order to harmonise the audit approach on these interpretative issues8 within DG
RTD and, thereafter, with the other Research DGs. KWG terms of reference have been
established.
The main tasks of the KWG are:
- To develop and keep up-to-date a reference keywords database
- To ensure coherence and to assist in drafting replies to incoming questions
- To draft Guidance Notes as required
The 'keywords database' is a compilation of all previous positions taken in the past on various
interpretative issues. This database is an important source of information for auditors. The
main issues are converted into a formal Guidance Note once they are endorsed by the CAR.
Here is the list of guidance notes drafted in 2008:
depreciation under cash based accounting,
interests yielded on pre-financing,
bonus payments,
tax exemption for Belgian researchers,
Marie Curie lump sums,
maternity leave,
French profit sharing mechanism,
recruitment costs,
calculation of hourly rates for personnel costs.
2.2.3.
Horizontal audit reference documents
All Research DGs have agreed to a common 'Audit Process Handbook' to be used for FP6
audits. The audit process includes all the procedural steps to be performed by the auditor, the
audit management, administrative support and other parties involved at the level of an
individual (external) audit engagement from the assignment (start) of the engagement to the
communication of audit results and the archiving of audit files.
8 By 'interpretative issues' we mean any issue which may give rise to different assessments amongst auditors
about the (non-)eligibility of the costs. The legal unit of DG RTD is often consulted on these issues.
30 March 2009
10
CONFIDENTIAL
The 'Audit Process Handbook' is in force as from January 1st 2008 and aims at ensuring a
common approach for all kind of audit tasks. The procedures established should ensure that
'generally accepted auditing standards' are applied.
The handbook has already been updated on several occasions. Updates relate mostly to
simplifications and to new developments (e.g. extrapolation). Auditors can send their
proposals for improvements to a functional mailbox that has been created for that purpose.
These proposals are discussed periodically on different levels: within unit RTD-A4 by the
auditors concerned and in the CAR meetings between the Research DGs. The handbook is
also available on the SAR wiki9.
The FP6 Audit Certificate Handbook (for internal use) and Audit Certificate Guidance Notes
(available on the Cordis website as a guidance tool for external certifiers of audit certificates)
have been updated and made coherent with the Handbook.
2.2.4.
Other Committees
In addition to the Committees already referred to above, DG RTD chairs and coordinates a
number of other Committees, as well as other cross-RDG coordination activities. These
committees are the Extrapolation Steering Committee (ESC, see §2.3), the Frauds and
Irregularities Committee (FAIR, see §2.6), the coordination of relations with the external
audit firms, including the Monthly Audit Status Meeting (MASR, see §2.10), the Joint
Assessment Committee (JAC, see §2.9.4) and the Working Group on Certification of
Methodology (WGCM, see §2.9.4).
DG RTD is also in the lead for coordinating the information and documents to be provided by
all RDGs to the ABM on audit-related matters.
2.3. Extrapolation
Extrapolation is a key component of the common FP6 audit strategy, because of its essential
role in 'cleaning' the budget from systematic material errors.
2.3.1.
Extrapolation policy and coordination
During 2007, the implementation of extrapolation was carried out separately by each RDG.
This resulted in different practices in the four RDGs towards common contractors.
Therefore, in February 2008, the Extrapolation Steering Committee (ESC) was set up in order
to ensure a common approach. The ESC, in which all RDGs are represented, discusses and
evaluates potential extrapolation cases put forward by an individual RDG or as a result of an
audit of the Court of Auditors. ESC meetings are organised and chaired by RTD.A.4. They
take place on a monthly basis, or more frequently if required. If a case receives the approval
of the ESC, a number of procedures are launched to implement extrapolation for a specific
contractor.
9 Web-based collaborative tool for the exchange of information amongst the external audit services of the
Research DGs.
30 March 2009
11
CONFIDENTIAL
2.3.2.
Extrapolation management
During 2008, the ESC met 11 times and a total of
155 extrapolation cases were discussed. The
first ESC meeting took place on February 14th 2008. The following table provides an
overview of the ESC results per RDG:
Table 2.1 ESC results in 2008 (as of 31/12/2008)
RTD
ENTR
INFSO
TREN
Grand Total
AGREED
76
12
31
15
134
NO EXTRAPOL
5
3
6
5
19
UNDER REVIEW
1
0
1
2
Sub Total
82
15
37
21
155
Not yet submitted
1
1
before 14/02/2008
23
24
19
0
6610
Grand Total
106
39
56
21
222
As of 31/12/2008, DG RTD has put
106 extrapolation cases on file, 79 of which introduced in
2008. Of these,
65 are currently ongoing,
12 are prepared to be submitted to the ESC (audit
closing),
1 has been suspended,
1 has been closed and for
27, extrapolation was not relevant
(no other cost periods to extrapolate to, update of audit results, etc.).
2.3.3.
Extrapolation implementation
As each individual extrapolation case can potentially affect many projects in a number of
RTD directorates, ongoing extrapolation cases require a significant amount of effort, attention
and supervision by the operational services responsible for the follow-up actions, as well as
by the external audit units. The experience acquired so far has underlined the challenges in
this area, especially with regard to the follow up of the reception of revised cost statements.
Also services are to monitor those beneficiaries that do not react promptly to Commission
requests or those who enter into discussions and negotiations with the Commission services.
Indeed, extrapolation is an important management tool that requires the active co-operation of
beneficiaries and therefore efforts need to be intensified, particularly with regard to tightening
up procedures and liaising with beneficiaries. The possibility of global recovery orders is
being explored, especially for cases of extrapolation to a large number of contracts, but it
requires the explicit approval of the contractor and due regard to sound financial management
and cost-benefit concerns.
For all RTD-led ongoing extrapolation cases,
1502 projects have been identified as affected
by the application of extrapolation. In addition,
29 cases led by other RDGs have an impact
within DG RTD (i.e. those beneficiaries participate in RTD projects). Currently,
255 projects
are affected by these non-RDG cases.
Additional efforts need to be undertaken so that close follow-up of individual extrapolation
cases could be carried out across the four RDGs despite, at present, weak IT-supporting tools.
The development of IT-supporting tools is essential for the effective and efficient
10 It is to be noted that for
66 cases, extrapolation was initiated before the creation of the ESC in February 2008.
30 March 2009
12
CONFIDENTIAL
implementation of the extrapolation process. Therefore, in the context of the SAR Action Plan
(see section 2.9), priority is given to developments in the area of extrapolation.
2.3.4.
Extrapolation follow-up activities
From April 2008, and on the initiative of RTD.A.4, a number of extrapolation follow-up
meetings have been organised in order to ensure better coordination within DG RTD. Given
the decentralised structure in DG RTD financial management, this has been an essential
initiative.
During the four meetings in 2008, representatives of operational RTD directorates responsible
for the implementation of extrapolation have discussed the status of ongoing cases and any
further actions to be taken, based on the latest information collected by all stakeholders. The
meetings also allow for useful discussions of practical issues (i.e. registering and analysing
revised cost statements, development of software tools to monitor follow-up, issues relating to
the recovery procedure, possibility of global recovery orders, legal issues, etc.).
Reminder letters are sent when the contractors involved do not respond to the requests to send
in revised cost statements or provide additional information. With the decision taken in 2008
to apply extrapolation not only to open contracts, but also to closed contracts, it is expected
that these efforts, which are necessary for an adequate follow-up and implementation of audit
results, will increase further.
Furthermore, cases where systematic errors can be directly extrapolated towards non-audited
areas without any further information from the contractor are a minority. In most cases, the
contractor wishes to establish a dialogue and to provide additional documentation and
evidence. It goes without saying that, in practice, extrapolation cases need to be assessed on a
case by case basis, and that follow-up audits will be needed for most of them.
2.3.5.
Further considerations
Overall, it can be concluded that the extrapolation process and its follow-up prove to be a time
consuming process which requires substantial resources.
The overall financial result of actual recoveries/adjustments related to extrapolation is
potentially very important. However, given the fact that most of the extrapolation cases are
still ongoing, at this stage it is not yet possible to have an exact picture of the overall
(financial) impact of extrapolation in relation to the resources required. Please note that this
monitoring is not being done by the external audit units, but by Directorate R (as is the case
for other non-extrapolation related recoveries).
It will be necessary to further optimise and improve the working procedures, and – as already
mentioned before - commit appropriate resources for the follow-up of extrapolation cases, as
well as improve the IT systems used to register and monitor ongoing extrapolation cases.
Furthermore it is necessary to further centralise and improve the coordinated approach of the
follow-up activities. In 2008 this last issue was addressed with the creation of the new Unit
R.7 ("Management of debts and guarantee funds").
30 March 2009
13
CONFIDENTIAL
2.4. Collaboration with the European Court of Auditors (ECA)
During 2008, our collaboration with ECA has been significantly strengthened in a number of
areas:
Planning: A4 now regularly receives the mission plan of the Court, via Unit R511.
Joint missions: a number of pilot-projects of joint missions with the Court were
undertaken in 2008; this was considered a positive experience. As a consequence,
more of these audits will be planned in 2009.
Treatment of ECA's audit results: Although R.5 manages relations with the Court, A4
is now also consulted on the draft sector letters issued by the Court. In addition, once
the final version of a sector letter is received, it is studied with a view to arrange,
where appropriate, follow-up audits to confirm or not the Court's findings (especially
those that concern extrapolation). This part of the collaboration needs to be further
strengthened during 2009.
Reporting: the back-office of the external audit units sends a quarterly compilation of
audit documentation to the Court related to all audits closed in that quarter. As for the
results of the Court's audits, they are now systematically entered into ASUR by R5 for
follow-up by the operational services.
Another area of activity is the Working Group on a common audit methodology between the
CoA and the Research DGs, set up at the end of 2007 by Commissioner Potočnik. This
working group aims to allow a constructive dialogue between the CoA and the Research
Directorates General on ex-post controls, and to reach a common understanding of the
differences in our respective audit methodologies. Several meetings have been organised in
Luxembourg and Brussels where technical issues like time recording and average hourly rates
were discussed.
One major outcome to be mentioned, at the technical level, is the recognition by the Court of
Auditors of the need for a specific audit approach for the auditing of personnel costs and
overheads, if averages are used by the contractor. This need is evident in the context of ex-
ante certification for FP7. A first exchange on this matter took place in 2008.
2.5. Reporting activities
The external audit units are asked to report in quite a different number of formats and to a
variety of audiences throughout the year.
Every quarter, we contribute to the Quarterly Report to the Commissioner. Information is
provided on the state of implementation of audits, and there is also an opportunity to highlight
issues that need to be brought to the attention of the hierarchy. A separate section elaborates
on the OLAF cases. The Cabinet is equally kept informed through weekly meetings between
the Cabinet and the DG and the occasional briefing.
11 In DG RTD, R.5 is in charge of the Relations with the Court of Auditors.
30 March 2009
14
CONFIDENTIAL
The external audit functions of all RDGs are frequently invited to the ABM Steering Group
(SG + DG BUDG + Cabinet representatives). This happened on three occasions in 2008: on
April 24th, July 15th and December 9th.
At the ABM meeting of April, the Steering Group requested to have uniform reporting tables
for all RDGs, not only where the number and results of audits are concerned, but also on the
number and amounts of recoveries. A meeting at SG level was held on June 23rd to get to an
agreement on how to present these results according to a common method. Since then, a
shorter and common way of portraying the RDG External Audit results has been in use.
In the July meeting, the ABM Steering Group asked for an increased communication effort
towards the Court of Auditors. The RDGs are still seeking the best way of doing this, because
DG RTD is not convinced that its current practice of sending quarterly an executive summary
with a DVD containing all the relevant audit reports for the previous quarter from the
Commissioner to the relevant Member of the Court is the best way of doing this. Moreover, it
appears that DG RTD is the only RDG proceeding in this way.
In December, RDGs were encouraged to go beyond purely quantitative reporting, and to look
at qualitative findings and identify what lessons can be drawn from them. This will definitely
become one of our objectives for 2009, also in view of the preparation of the FP 7 Audit
Strategy.
The Court audited the ex-post audit strategy in February 2008, acknowledging that the
reinforcement in human resources and the reorganisation of the external audit function has
positively influenced audit coverage, and that with a better follow-up of audit results this
should lead to improved overall assurance. It equally acknowledged the efforts undertaken to
come to greater coherence between the RDGs, as for example on common interpretations and
guidance. The Court was still critical with regard to delays in implementing audit findings,
and they were also of the opinion that extrapolation ought to be extended to closed contracts.
2.6. OLAF cases
Unit RTD.A.4 has been in charge of relations with OLAF on external enquiries12 since 1st
February 2008. During that period, DG RTD transmitted seven new cases of suspected
irregularities to OLAF (six concerning research grant beneficiaries and one concerning a
service contract). In six cases, suspicion of irregularities was reported by the operational
services in charge of the projects and contracts; in one case, the decision to transfer the case to
OLAF was taken following an audit on-the-spot because there was suspicion of irregularities
on a subcontract. OLAF informed unit RTD.A.4 that it had opened three other cases in the
area of research following information provided by individuals. These cases are still under
evaluation or investigation.
During 2008, OLAF classified nine DG RTD cases as non-cases13. Two cases were closed: in
one case the national judicial authorities dropped the criminal proceedings; in the other, the
national judicial authorities convicted the person in question to a financial penalty. Three
12 Internal enquiries relating to individuals are dealt with by RTD.01.
13 Non-cases: A matter is classified as a non-case where there is no need for OLAF to take any investigation,
coordination, assistance or monitoring action. Non-cases result from assessments that conclude that EU interests
appear not to be at risk from irregular activity, or other relevant factors indicate that no case should be opened.
30 March 2009
15
CONFIDENTIAL
investigations were closed following OLAF's confirmation of the allegations of irregularities.
These cases are currently followed up at administrative, financial and/or judicial levels.
Table 2.2. Situation OLAF cases as of 31/12/2008
New cases Cases
Cases
Investigations
Cases
Total number Total number of
transmitted
initiated
OLAF
closed
with closed in of open cases closed cases still in
to
OLAF directly
classified as administrative/
2008
including
administrative/
by
DG by OLAF non-cases in financial/judicial
cases
of financial
and/or
RTD
in in 2008
2008
follow-up
in
previous years
judicial follow-up
2008
2008
7
3
10
3
2
28
10
Unit RTD.A.4 had a coordination meeting with Unit OLAF.A.3 on 23 July 2008 in which
ongoing DG RTD OLAF cases and more general issues concerning working relations and
methods were discussed. RTD.A.4 furthermore participated in nine meetings with OLAF
investigators and with DG RTD operational services on specific cases.
As a result of the experience gathered in 2008, two initiatives are being implemented as from
2009: (1) the establishment of the FAIR (=Fraud and Irregularities in Research) Committee
(across RDGs, to coordinate approaches and to share policies, best practices and results), and
(2) an additional quality review step to consider, for every draft audit report, whether there are
potential irregularities, which will then be studied in more detail.
2.7. Quality control tools
2.7.1.
The Audit Steering Committee (ASC)
The Audit Steering Committee exists to discuss proposals for large adjustments and difficult
issues. It provides the opportunity of a peer review by fellow auditors on these issues.
Adjustments are considered to be large when they are above 100,000 € and represent 5% or
more of the costs claimed, or when they are above 30,000 € and represent 30% or more of the
costs claimed.
The ASC considers both in-house and outsourced audits and helps to ensure equal treatment
and the coherence of audit work.
The ASC meets on an ad-hoc basis, usually at the request of the auditor responsible for a
specific audit file. At least 5 auditors on a rotating basis should be present, although in
practice this varies usually between 7 and 10. The rotation offers newly arrived auditors a
chance to become familiar with audit cases and best practice. Two of the auditors have to be
experienced auditors with at least 5 years of audit experience, and at least two auditors should
be from RTD A4 and two auditors from RTD A5. Each decision is documented in the of the
meeting.
The number of cases dealt with in the ASC has grown in 2008 due to the increased audit
activity:
30 March 2009
16
CONFIDENTIAL
Table 2.3 ASC cases
Meetings
cases
2005
3
3
2006
4
8
2007
3
5
2008
12
26
2.7.2.
The quality review process
The quality review process for the audit reports done by Commission auditors was
strengthened in 2008.
According to the Audit Process Handbook, A4 adopts an 'Audit Closing Memorandum',
which involves two quality control checks. The first check is on the substance of the draft
audit report before it is sent to the Operational Directorate(s) for their comments. This
substance check concentrates on two main questions:
Are the adjustments sound and well explained in the audit report?
Has the audit revealed systematic errors and will an extrapolation case be proposed?
The second check takes place before the audit is closed. It looks at completeness, correctness
and coherence, and it is both on format and on substance:
Are all relevant documents attached, ensuring that appropriate procedures have been
respected?
Have the templates in the Audit Handbook been used or, if not, is such deviation justified?
Have the audit results been presented in all documents in a coherent way?
Did the auditee express remarks of substance and have those been duly considered?
Are the deviations between amounts stated by the certifying auditor and those stated in the
Audit Report so important that the certifying auditor should be informed?
Outsourced audits procured as batch assignments are monitored by a dedicated team
regarding milestones and deliverables in accordance with A5's batch audit procedures
handbook.
2.8. Collaboration with the DG RTD administration and finance (UAF) network
Throughout 2008 the external audit units have maintained close working relationships with
the administration and finance units be it in the phase of planning and preparing the launch of
new audit campaigns, during the audits (in order to obtain feedback on draft audit
conclusions) or after the audit (for the implementation of the final audit conclusions).
Moreover, ad-hoc bilateral meetings have been held whenever appropriate to discuss specific
files and – as already mentioned - extrapolation follow-up meetings were organised to
30 March 2009
17
CONFIDENTIAL
monitor the state-of-play of extrapolation files. The external audit units also participate in the
monthly UAF meetings to present and clarify matters linked to audit and financial issues.
2.9. SAR and other IT developments
SAR Wiki - The SAR Wiki, a free and open source piece of software installed by DG
DIGIT under the coordination of ITPO, was put into production in June 2008. This has
helped with sharing audit results amongst Research DGs as well as with the RDG
coordination work and planning of audits.
Audit Management information System (AMS) in DG RTD - The development of
an application that will replace Aubase (the present audit management system that was
developed internally by the External Audits Unit) has progressed during 2008. In
agreement with R4, its planned implementation in 2008 has been postponed until
March 2009 (phase 1) and late 2009 (phase 2), in order to include increased
functionality and security.
ABAC Audit Tracking System - This module of ABAC was put into production at
the end of 2007. RDGs were requested to upload data about audits.
Extrapolation (EXITs) - To address the acute need for the administration and
management of the work created by extrapolation, a new application called EXITs was
developed by A4. It is currently using the data tables of Aubase which will be replaced
by AMS. The plan is that its functionality will be included in phase 2 of AMS.
ASUR EXA - The ongoing development of ASUR, which is the system through
which the implementation of the proposed adjustments is monitored, resulted in a
number of improvements. For example, recommendations are now assigned to the
AOSDs14 as responsible persons, and forecast of revenue details can now be recorded.
These improvements have, in turn, helped to strengthen the working relationship
between the back office of the external audit units and the operational directorates.
ASUR EXTRA - Extrapolation has had substantial repercussions on procedures and
systems beyond those of the external audit units. As a consequence, and in order to
improve the follow up of recommendations resulting from extrapolation, follow-up
data has been migrated from EXITs to ASUR EXTRA (an application owned by R5).
FP6 data integrity - Despite the 'cleaning' efforts by units A4, A6, R4, data integrity
in central RTD information systems continues to be a problem which undermines, to a
certain extent, the implementation of the Audit Strategy. As an example of the
problems we face, single entities continue to have several different names, sometimes
different FEL IDs, or conversely, single FEL IDs are shared by a few different entities.
14 AOSD=Authorising Officer by Sub-Delegation.
30 March 2009
18
CONFIDENTIAL
2.10. FP7 Certification policy
2.10.1. General Principles
The Certification policy for the FP7 Grant Agreements was designed with the aim to correct
the most common errors identified in the past, and in particular those related to personnel
costs and indirect costs. In this context, the 7th FP introduced, in addition to
the Certificates
on the Financial Statements, two new types of ex-ante certificates on the methodology which
may be submitted prior to the costs being claimed: the
Certificate on Average Personnel
Costs a
nd the Certificate on the Methodology for Personnel and Indirect costs.
2.10.2. State of play of Certification files as of 31 December 2008
Concerning the FP7 Certification on the Methodology for personnel and indirect costs, as of
31 December 2008, 46 requests for eligibility were submitted out of which 14 have been
rejected because they did not fulfil the relevant criteria. Furthermore, the Commission
received 14 requests for certification of the beneficiaries' methodology for personnel and
indirect costs and 7 concerning the methodology for average personnel costs. The
acceptability of the certificates is decided by an inter-service Joint Assessment Committee,
which involves DG RTD's external audit units and DG INFSO. In 2008, 5 meetings of the
Joint Assessment Committee were organised during which 1 certificate was accepted and 3
were rejected.
The methodology certification for average personnel costs still requires the Commission to
define acceptability criteria to accept or reject average personnel costing methodologies. The
absence of these criteria is delaying the reimbursement to beneficiaries with the
corresponding risk of late payment and credibility of the institution. By the end of December
2008, the RDGs, further to lengthy discussions with the Court of Auditors and DG Budget,
were preparing a common position and the criteria will be discussed at Commission level
early 2009. In total, 15 certification files under analysis are dependent on the decision on
acceptability criteria.
The methodology certification activity is characterised by an undetermined, yet, in view of the
size of the beneficiary population, potentially very high workload. The moment in time when
the certificates will be submitted and need to be dealt with cannot be planned up-front.
2.10.3. Supporting IT tools
The development of the central IT tool in OMM to support the management of the FP7
certification activities was started in the third quarter of 2007 under the responsibility of Unit
R4. Due to resource constraints, this project was no longer considered by R4 as an IT priority
and was finally abandoned due to the implementation of PDM/URF in replacement of OMM.
From June 2008 onwards a new web-based project was initiated, promoted by ITPO. This
project aims to provide a central Web-based IT tool, solely dedicated to supporting the FP7
methodology certification. In the meantime a local MS Access based tool supports the
certification activities.
30 March 2009
19
CONFIDENTIAL
2.10.4. Inter-service collaboration
An inter-service Working Group on Certification of Methodology (WGCM) involving the
other research DGs and DG BUDG contributed to the establishment of guidance for
prospective FP7 beneficiaries and other implementation modalities of the cost methodology
certification process. In 2008 the group met in April and September 2008 (19th and 20th
meeting).
Furthermore DG RTD has received delegated authority as part of the ABM action plan to
handle and organize the FP7 cost methodology operations on behalf of all RDGs. DGs TREN
and ENTR delegated the entire FP7 methodology certification management while DG INFSO
remains involved in the operational activity. Appropriate structures, procedures and working
tools were put in place to manage this activity within this scope. The Joint-Assessment
Committee prepares the Commission's decisions on certification requests.
2.10.5. Communication activities
In the framework of the FP7 Certification policy, RDGs cooperated in conducting
communication activities to increase internal and external awareness on the new instrument of
FP7 Certification on the Methodology:
handling questions submitted through the Research Enquiry Service on
Europe Direct.
an internal awareness-raising campaign on FP7 Certification issues leading to
meetings with operational and UAF units.
an external communication campaign towards potentially eligible beneficiaries for the
Certification on the Methodology through a mailing sent to 400 beneficiaries.
participation in seminars, conferences, bilateral meetings and pilot reviews (over 50
events in total).
posting of certification-related documents on
www.cordis.europa.eu (FAQ document,
specific certification-dedicated pages, "Guidance notes for Beneficiaries and
Auditors").
2.11. Coordination of relationships with external audit firms
The existing framework contract for audit services was signed with three audit firms to be
used under a cascade principle.
Due to the audit targets of the FP6 audit strategy there is a relatively strong dependence on the
external audit firms, as approx. 75% of the target is done through outsourced audits. The
external audit firms operate according to established professional audit practice and standards
and provide a useful and necessary complement to DG RTD's in-house audit expertise and
capacity.
RTD A.5 closely monitors the performance of the audit firms ensuring that, as far as possible,
all audits are completed and closed within the contracted time frame. The monitoring involves
the following processes:
- Monthly Audit Status meetings (MASR meetings) chaired by RTDA5, covering the
progress of all on-going batches, technical issues, invoicing and future audit planning.
30 March 2009
20
CONFIDENTIAL
- Accompanying external audit firms on-the-spot audits (five such missions were
performed).
- Providing guidance and clarification on specific problems.
- Maintenance of the Audit Review Assessment (ARA) to follow-up the quality of the
services provided.
- A batch audit processing manual including checklists for the different deliverables.
2.12. Management of the audit framework contracts and their public procurement
Unit A5 chairs on behalf of DG RTD and towards the other RDGs the coordination of the
external audit firms performing outsourced audits through the framework contract and also
manages the public procurement procedures for new framework contracts for audit services
on FP6 as well as on FP7 research grants (for use by all Research DGs and related agencies).
Throughout 2008 it became apparent that the considerably increased volume of
external audits stretched the capacity of the current main service provider who was
selected based upon a tender that initially foresaw significantly lower audit volumes
(300 audits per year).
Unit A5 launched procedures to initiate, in conjunction with the open call for tender
for audit services for FP7, a new open call for tender for additional FP6 audit services.
Both tender procedures closed on 4 August 2008 and the evaluation procedure and
award decision for the FP6 call (8 tenders received) were concluded in December
2008. A positive advice from the DG RTD procurement committee was obtained to
allow the new FP6 framework contract to be available for new outsourced audit
campaigns as of early 2009. The new FP6 framework contract has been signed in the
meantime. For the FP7 call for tenders, the appointed evaluation committee was end
December in the process of finalizing the evaluation of the nine tenders received. The
evaluation procedure is ongoing. These procurement procedures cover the outsourcing
of audits for FP6 and FP7 grants for the period 2009-2012 with a potential market
value amounting to EUR 16,5m and EUR 42m respectively.
2.13. Other activities (Art.169 Initiatives/JTIs/Agencies)
2.13.1. Article 169 Initiatives - EUREKA
EUREKA is an inter-governmental initiative launched in 1985 to support market-oriented
transnational R&D and innovation projects across all civil sectors; its primary objective is to
enhance European competitiveness through its support to businesses, research centres and
universities who carry out pan-European, market-led innovative projects. The EUREKA
initiative is administratively supported by the EUREKA Secretariat (ESE), a non-profit-
making international association ("AISBL") established under the Belgian law.
The EUROSTARS Joint Programme is an intergovernmental initiative to provide funding to
SMEs undertaking R&D. It is based on Article 169 of the EC Treaty which provides for the
participation of the Community in research programmes undertaken by several Member
States. The programme has been set up for a 6 year period from 2008-2013 with a Community
30 March 2009
21
CONFIDENTIAL
financial contribution of EUR 100m. The ESE has been identified as the "dedicated
implementation structure" of the EUROSTARS Joint Programme, and it is responsible for the
administration and the logistic support of the EUROSTARS Joint Programme.
Pursuant to the Financial Regulation applicable to the General Budget of the European
Communities (Art. 54), and following the opinion of DG BUDG15, the EUROSTARS
Initiative corresponds to what is called "indirect centralised management".
According to Art.56(1) of the FR, where the Commission uses a system of "indirect
centralised management", it must first obtain evidence of the existence and proper operation
within the entity to which it entrusts the implementation.
Therefore, in order to decide whether the Commission can delegate budget implementing
tasks using the "indirect centralised management" mode to the entity identified, it has to carry
out an ex-ante assessment.
In 2008, this was the first time that RTD.A.4 undertook such an ex-ante assessment. The
scope was limited to understanding and evaluating the current procedures as explained by the
management of ESE, before the start of the programme implementation, and before any actual
financial transaction has taken place.
In addition, Unit A4 was asked to participate in the drafting and negotiation of the delegation
agreement between the Commission and the ESE, which specifies the tasks entrusted to ESE,
the rules for their implementation and the relations between ESE and the Commission. The
delegation agreement also determines the ESE's rights and responsibilities.
This work has been of value especially in view of future similar initiatives, like EURAMET
(Metrology), for which a similar ex-ante assessment is being scheduled in 2009.
2.13.2. Joint Technology Initiatives – JTIs
Unit A4 participated in two technical working groups concerned with setting up the
governance, administrative and financial features of the Joint Undertakings (JUs, where DG
RTD is involved). The two working groups looked at (a) the definition of the internal control
framework of the JU and (b) the finalisation of the grant financial agreement.
The main outstanding issue is the extent to which the audit strategy of the JTIs is to be
harmonised with the corporate audit strategy of the RDGs.
2.13.3. Executive Agencies – REA and ERCEA
The external audit units were also involved in the process of setting up of the two "DG RTD"
Executive Agencies, in particular where the Audit Strategy is concerned.
The relationship between the Agencies and the Commission services will be different before
and after the Agencies have become autonomous. At present, working groups exist in which
units A4 and A5 are present to analyse the most appropriate arrangements.
15 Note from M Romero Requena to MM Silva Rodriguez and Colasanti, n° 2267 dated 19.3.2007.
30 March 2009
22
CONFIDENTIAL
3.
RESULTS AND ANALYSIS
In this part, the measurable results of the activities of the external audit units are presented,
together with analysis and commentary where appropriate.
Please note that all figures representing adjustments in this part are estimates that
might or might not correspond with the eventual recovery or offset amount applied by
operational services. Proposed adjustments are calculated on the basis of cost model and
instrument type but there might be variations of the actual percentage of EC contribution for
specific contracts. This information is not available in central RTD information systems.
In 2008, the method for calculating proposed adjustments has been refined to take into
consideration instrument types as well as cost models. In addition, we now seek more detailed
percentages of EC contribution from the operational services for audited participations where
the proposed adjustment is over EUR 100.000 in favour of the Commission. This in turn has
resulted in more accurate error rate calculations.
3.1. Audit numbers
This section presents results related to the number of audits and participations audited in 2008
and cumulatively, with breakdowns by a number of categories. The most interesting points are
summarised below each table.
Table 3.1 Audits closed and participations audited (2008 and cumulative, ALL audits)
2008
Cumulative
Strategy
No. audits
No. audited
No. audits
No. audited
FP
strand
closed
participations
closed
participations
FP6
TOP
147
414
219
585
MUS
86
188
149
330
RISK
123
233
136
251
FUSION
13
17
20
27
OTHER
3
6
75
127
Total FP6
372
858
599
1320
FP5
N/A
9
11
703
875
Coal & Steel N/A
2
5
2
5
Grand totals
383
874
1304
2200
383 audits were closed during 2008.
599 FP6 audits have been closed so far by the end of 2008.
Almost all audits for the FP6 MUS sample have already been closed (149 out of 161).
The ratio of participations per audit is 1,24 for FP5 and 2,20 for FP6 at this point. This
indicates a substantial increase in the cost-effectiveness of audits in FP6, and is the
result of improvements in planning and audit preparation.
30 March 2009
23
CONFIDENTIAL
Table 3.2 Audits of specific types (2008 and cumulative, FP6 and Coal & Steel audits)
2008
Cumulative
Audit type
FP
No. audits closed No. audits closed
FUSION
6
13
20
Coal & Steel
N/A
2
2
Joint audits with ECA
6
2
2
Third country audits
6
7
7
Audits on request
6
8
25
The three year rolling cycle for FUSION audits continues apace, and by the end of
2009 all FP6 FUSION associations should have been audited once.
The first two Coal and Steel audits were closed in 2008 (see section 2.1)
The first two joint audits with the Court of Auditors were also closed in 2008 (see
section 2.4)
Table 3.3 Audits closed, outsourced and in-house (2008 and cumulative, FP6 only)
2008
Cumulative
No. audits
No. audits
Batch Name
%
%
closed
closed
Outsourced
BA-2006-32a
51
BA-2006-32b
5
36
BA-2006-36
1
20
BA-2007-38
47
103
BA-2007-41
71
81
BA-2007-42
5
5
BA-2007-43b
8
8
BA-2008-46
15
15
BA-2008-50
41
41
BA-2008-51
38
38
BA-2008-57
19
19
BA-2008-60
13
13
Other batches
14
17
Total outsourced Grand Total
277
74,5%
447
74,6%
In-house
95
25,5%
152
25,4%
Grand totals
372 100,0%
599
100,0%
Batches 46, 50, 51, 53, 57, 60, 61, 63, 64 and 65 were launched during 2008. For some
of these batches, no audits have yet been closed, so they are not included in the table
above.
Batches 36, 41 and 42 were completed in 2008.
The percentages of in-house and externalised audits were 25,5% and 74,5%
respectively in 2008. This is similar to the proportion over the cumulative period.
However, it must be noted that 95 in-house audits were closed last year, as opposed to
only 35 in 2007.
30 March 2009
24
CONFIDENTIAL
Table 3.4 Audits launched and closed (2006-2007-2008, ALL audits)
Totals for the three
2006
2007
2008
years
Launched Closed
Launched
Closed
Launched
Closed
Launched
Closed
16
FP5
112
139
11
113
5
9
128
261
FP6
127
34
280
192
423
372
830
598
C&S17
5
2
5
2
Totals
239
173
291
305
433
383
963
861
433 audits were launched in 2008, as opposed to 291 in 2007. This is an increase of
49% in just one year.
For closed audits, the figures are 383 (2008) and 305 (2007), an increase of 26%. If
current staffing levels remain as they are currently, and provided that the ability to
deliver of the external audit providers remains unchanged, the objective is to maintain
a yearly closing target of 350 to 400 audits.
Table 3.5 Audits closed by country (2008, FP6)
% no. of
participations in
Country
No. audits
FP6 (RTD only,
closed
%
CORDA 14/01/09)
DE Germany
66
17,7%
13,4%
FR
France
48
12,9%
10,0%
UK United Kingdom
40
10,8%
12,2%
IT
Italy
38
10,2%
8,2%
NL
Netherlands
28
7,5%
5,7%
BE
Belgium
27
7,3%
4,0%
ES
Spain
20
5,4%
6,5%
SE
Sweden
16
4,3%
3,5%
CH Switzerland
12
3,2%
2,6%
FI
Finland
10
2,7%
1,8%
PT
Portugal
8
2,2%
1,6%
Others (EU & non-EU)
59
15,9%
30,5%
Total
372
100,0%
100,0%
FP6 started in 2002, so countries which joined the EU from 2004 onwards did not
participate for the whole period. This accounts for their absence in the list of the 11
countries above.
The fact that a significantly higher proportion of audits take place in the top 11
countries in the list than the proportion of participations they represent can be
explained by the emphasis of the audit strategy on TOP beneficiaries, which tend to be
based in those countries.
16 442 FP5 audits and 1 FP6 audit were closed before 2006.
17 In 2006 and 2007, Coal and Steel audits were not yet done by RTD's external audit units.
30 March 2009
25
CONFIDENTIAL
Table 3.6 Participations audited by RTD directorate (2008, ALL audits)
No.
Directorate
Name
participations
%
audited
B
European Research Area
44
5,0%
D
International cooperation
16
1,8%
E
Biotechnologies, agriculture, food
75
8,6%
F
Health
214
24,5%
G
Industrial technologies
126
14,4%
H
Transport
111
12,7%
I
Environment
101
11,6%
J
Energy (EURATOM)
30
3,4%
K
Energy
38
4,3%
L
Science, economy and society
31
3,5%
S
'Ideas' programme
11
1,3%
T
Implementation of activities to outsource
77
8,8%
Total
874 100,0%
It is important to note that the sampling methods used by the external audit units do
not presently provide statistical representativity per directorate.
3.2. Audit results
This section presents audit results in monetary terms, including an attempt to compare the
effect of ex-ante and ex-post controls. The most interesting points are summarised below each
table.
Table 3.7 Audit results in monetary amounts (2008, ALL audits)
Results at cost level
Audited
Costs claimed
Costs accepted
Costs accepted
Adjustments in
Adjustments
participations
and audited
by Financial
by Auditor
favour of the
in favour of
Officers
Commission
the beneficiary
FP5
11
4.513.424
4.271.091
3.959.892
-503.926
192.727
FP6
858
762.997.077
762.083.675
747.137.304
-19.750.683
4.804.312
C&S
5
11.886.717
11.882.781
11.902.912
-941
21.072
Totals
874
779.397.217
778.237.548
763.000.107
-20.255.551
5.018.111
Results at funding level (estimated EC share)
Audited
Costs claimed
Costs accepted
Costs accepted
Adjustments in
Adjustments
participations
and audited
by Financial
by Auditor
favour of the
in favour of
Officers
Commission
the beneficiary
FP5
11
2.679.034
2.556.454
2.402.238
-259.954
105.738
FP6
858
360.075.082
359.836.311
352.305.703
-11.153.188
3.622.580
C&S
5
5.943.358
5.941.390
5.951.456
-471
10.536
Totals
874
368.697.474
368.334.155
360.659.396
-11.413.613
3.738.854
In 2008, a total of almost EUR 780m costs were audited by the external audit units. Of
this amount, the EC contribution was almost EUR 370m.
30 March 2009
26
CONFIDENTIAL
The total amount of adjustments in favour of the Commission at funding level
proposed by the auditors was roughly EUR 11,4m.
Table 3.8 Audit results in monetary amounts (cumulative, ALL audits)
Results at cost level
Audited
Costs claimed
Costs accepted
Costs accepted
Adjustments in
Adjustments
participations
and audited
by Financial
by Auditor
favour of the
in favour of
Officers
Commission
the
beneficiary
FP5
875
358.200.473
352.094.243
349.484.979 -12.831.234 10.390.030
FP6
1.320 1.276.325.725 1.275.168.565 1.257.962.453 -24.360.304
7.154.192
C&S
5
11.886.717
11.882.781
11.902.912
-941
21.072
Totals
2.200 1.646.412.914 1.639.145.589 1.619.350.344 -37.192.479 17.565.293
Results at funding level (estimated EC share)
Audited
Costs claimed
Costs accepted
Costs accepted
Adjustments in
Adjustments
participations
and audited
by Financial
by Auditor
favour of the
in favour of
Officers
Commission
the
beneficiary
FP5
875
214.735.023
211.041.477
208.889.472
-8.021.292
5.950.712
FP6
1.320
573.344.842
572.881.964
563.692.693 -14.175.245
4.985.974
C&S
5
5.943.358
5.941.390
5.951.456
-471
10.536
Totals
2.200
794.023.223
789.864.832
778.533.621 -22.197.008 10.947.222
Concerning cumulative results, the auditors have so far checked nearly EUR 1,65bn in
costs claimed over the FP5, FP6 and C&S audit campaigns.
Of that amount, EUR 1,27bn is FP6 costs, as opposed to EUR 358.000.000 for FP5.
This already represents an increase of 255%, at a point in time when the FP5 audit
campaign is almost complete while the FP6 campaign is only half-way through. This
highlights the enormous proportional FP-on-FP increase in auditing efforts.
In relation with the previous point, the cumulative amount of proposed adjustments at
funding level for FP6 is near EUR 14,2m.
Table 3.9 Assessment of the different steps of the control chain (2008, FP5 and FP6
audits). All amounts are EC share.
FP5
FP6
total
Costs claimed and audited
(A)
2.679.034
360.075.082
362.754.116
Costs accepted by Financial Officers
(B)
2.556.454
359.836.311
362.392.764
Net effect of ex-ante controls (B-A)
-122.580
-238.771
-361.351
Costs accepted by Auditor
(C)
2.402.238
352.305.703
354.707.940
Net effect of ex-post controls (C-B)
-154.216
-7.530.608
-7.684.824
30 March 2009
27
CONFIDENTIAL
Table 3.10 Assessment of the different steps of the control chain (cumulative, FP5 and
FP6 audits). All amounts are EC share
FP5
FP6
total
Costs claimed and audited
(A)
214.735.023
573.344.842
788.079.865
Costs accepted by Financial Officers
(B)
211.041.477
572.881.964
783.923.441
Net effect of ex-ante controls (B-A)
-3.693.546
-462.878
-4.156.424
Costs accepted by Auditor
(C)
208.889.472
563.692.693
772.582.165
Net effect of ex-post controls (C-B)
-2.152.005
-9.189.271
-11.341.276
The net effect of ex-ante and ex-post controls is shown above. By ex-ante, we mean
the corrections made by financial officers to costs claimed when they are received;
while by ex-post we mean the adjustments proposed by the auditors.
Interestingly, ex-ante controls have had a bigger cumulative effect for FP5 than ex-
post controls. However, for FP6, the opposite is true, and the difference is quite
significant. The most likely explanation of this is the introduction of audit certificates
in FP6. One of the explanations could be that in FP6 the introduction of the certificate
led to the fact that most part of the errors are detected and corrected before sending the
cost statement to the Commission.
Table 3.11 Results by instrument type (cumulative, FP6). All amounts are EC share.
Project
Audited
%
Adjustments in favour
%
Type
participations
of the Commission
IP
Integrated Project
577
43.7%
-6,718,978
47.4%
STREP
Specific Targeted Research
247
18.7%
-1,859,953
13.1%
Project
NOE
Network of Excellence
224
17.0%
-2,206,009
15.6%
IA-I3
Integrating activities implemented
41
3.1%
-597,066
4.2%
as Integrated Infrastructure
Initiatives
CA
Coordination action
40
3.0%
-283,204
2.0%
SSA
Specific Support Action
35
2.7%
-980,027
6.9%
EST
Early-stage Training
27
2.0%
-171,241
1.2%
FUSION
FUSION programme
27
2.0%
-239,881
1.7%
CRAFT
Co-operative research projects
21
1.6%
-129,474
0.9%
RTN
Research Training Networks
19
1.4%
-42,671
0.3%
EXT
Grants for Excellent Teams
10
0.8%
-50,283
0.4%
CLR
Collective research projects
9
0.7%
-43,188
0.3%
EIF
Intra-European Fellowships
8
0.6%
-20,755
0.1%
CNI-SSA
Construction of new
6
0.5%
-761,736
5.4%
infrastructures implemented as
Specific Support Actions
Other
29
2.2%
-70,780
0.5%
Grand Total
1320 100.0%
-14,175,245
100.0
%
30 March 2009
28
CONFIDENTIAL
Comparing the column showing the relative proportion of audited participations with the
column showing the relative proportion of adjustments in favour of the Commission there is
no significant pattern that can be identified, exception made maybe for the Specific Support
Action (3 % of the audited participations; yet 7 % of the proposed adjustments).
Table 3.12 Results by cost model (cumulative, FP6). All amounts are EC share.
Cost model
Audited
%
Adjustments in
%
participations
favour of the
Commission
AC
715
54.2%
-6,723,019
47.4%
FC
418
31.7%
-5,690,901
40.1%
FCF
181
13.7%
-1,757,819
12.4%
N/A
6
0.5%
-3,506
0.0%
Grand Total
1320
100.0%
-14,175,245
100.0%
Comparing the column showing the relative proportion of audited participations with the
column showing the relative proportion of adjustments in favour of the Commission,
relatively speaking, there appear to be more adjustments identified in FC models.
3.3. Analysis
This section attempts to provide more in-depth and qualitative analysis of certain aspects and
results of the work of the external audit units, particularly in relation to error rates, error types,
most prevalent errors at cost category level and a more detailed look at the highest
adjustments proposed so far in FP6.
3.3.1.
Analysis of error rates
Table 3.13 Error rates (2008, FP5 and FP6 audits). All amounts are EC share.
FP
Strategy
Costs accepted by
Adjustments in favour of
Error rate
Representative
strand
Financial Officers
the Commission
error rate
FP6
TOP
215.551.643
-7.032.019
-3,26%
-3,41%
MUS
37.177.957
-1.575.399
-4,24%
RISK
37.616.858
-2.328.014
-6,19%
FUSION
68.595.822
-163.439
-0,24%
OTHER18
894.032
-54.317
-6,08%
Total FP6
359.836.311
-11.153.188
-3,10%
FP5
N/A
2.556.454
-259.954
-10,17%
18 A number of FP6 audits were launched before the preparation of the FP6 Audit Strategy, and cannot be
ascribed to any strategy strand.
30 March 2009
29
CONFIDENTIAL
Table 3.14 Error rates (cumulative, FP5 and FP6 audits). All amounts are EC share.
FP
Strategy Costs accepted by
Adjustments in favour of
Error rate
Representative
strand
Financial Officers
the Commission
error rate
FP6
TOP
274.332.096
-8.208.336
-2,99%
-3,13%
MUS
68.671.934
-2.527.579
-3,68%
RISK
57.811.650
-2.567.312
-4,44%
FUSION
140.927.357
-239.881
-0,17%
OTHER
31.138.928
-632.136
-2,03%
Total FP6
572.881.964
-14.175.245
-2,47%
FP5
N/A
211.041.477
-8.021.292
-3,80%
For FP6, error rates in all categories were higher in 2008 than in previous years.
The cumulative FP6 error rate stands at -2,47%, while the representative rate19 is -
3,13%. The difference can be explained by the effect of FUSION audits, which
constitute a high proportion of audited costs and have traditionally a much lower error
rate.
The following charts illustrate the evolution of error rates in FP5 and FP6 up to the end of
2008.
FP5 annual error rates have increased for the last five years; although audit volumes
need to be taken into consideration (e.g. only 9 FP5 audits were closed in 2008). In
that respect, the cumulative error rate is more telling, especially as we have now
almost reached the end of the FP5 audit campaign. As can be seen above, that remains
very close to what it was in 2002, currently -3,80%.
For FP6, the picture is different, with a clear year-on-year increase, but trends in error
rates remain hard to predict.
Error rates per strategy strand in 2008 have been higher than over the cumulative
period for all strands. Of particular interest is the difference in the RISK strand, which
19 The representative error rate is a combination of results in the TOP and MUS strands, and it is so called
because the samples of these strands are statistically representative and cover 100% of the budget.
30 March 2009
30
CONFIDENTIAL
suggests that enhanced criteria for the selection of auditees with a high-risk profile are
bearing fruit.
3.3.2.
Analysis of adjustments at cost category level (FP6)
This section provides analysis of the incidence of errors at cost category level. Costs claimed
by beneficiaries are ascribed to one of a number of defined cost categories. When audit results
are compiled, they are presented and implemented for an audited participation as a whole,
with results in different cost categories being netted off. However, it can be of value to
consider errors at cost category level, particularly in order to identify in which areas of
expenditure errors are found most often, in terms of number and value. The two tables below
help with this analysis.
30 March 2009
31
CONFIDENTIAL
Table 3.15 Proportion of adjustments by cost category (cumulative, FP6)
In favour of the EC
In favour of beneficiaries
No. of
% no. of
% of the
% of the
No. of
% no. of
% of the
% of the
adjustments
negative
total
total costs
adjustments
positive
total
total costs
adjustments
adjustment
accepted
adjustments
adjustment
accepted
amount
by
amount
by
Financial
Financial
Officers
Officers
Adjustment to costs previously
88
3,80%
2,00%
0,20%
66
6,50%
2,10%
0,20%
reported
Audit certificates
3
0,10%
0,00%
0,00%
6
0,60%
0,00%
0,00%
Consumables and computing
182
7,80%
2,70%
6,20%
44
4,40%
1,20%
6,20%
Durable equipment
87
3,70%
3,10%
2,50%
33
3,30%
0,20%
2,50%
Management of the Consortium
9
0,40%
0,10%
0,20%
7
0,70%
0,20%
0,20%
Other costs
388
16,60%
8,30%
10,30%
99
9,80%
12,90%
10,30%
Personnel
420
18,00%
46,60%
41,60%
223
22,10%
18,00%
41,60%
Protection of knowledge
0
0,00%
0,00%
0,10%
3
0,30%
0,00%
0,10%
R & T Development / Innovation /
7
0,30%
0,30%
0,20%
5
0,50%
0,90%
0,20%
Training
Subcontracting
90
3,90%
16,40%
13,70%
157
15,50%
6,80%
13,70%
Total indirect costs
672
28,80%
14,50%
18,60%
296
29,30%
57,10%
18,60%
Travel and subsistence
378
16,20%
1,30%
3,10%
67
6,60%
0,50%
3,10%
Various others
13
0,60%
4,80%
3,30%
5
0,50%
0,00%
3,30%
Grand Total
2337
100,00%
100,00%
100,00%
1011
100,00%
100,00% 100,00%
For adjustments in favour of the Commission:
In terms of recurrence, the most number of errors in favour of the Commission are in
Indirect costs/Overheads (28,8%), Personnel (18%), Other costs (16,6%), Travel and
subsistence (16,2%) and Consumables and computing (7,88%).
In terms of monetary impact, the main errors are in Personnel (46,6%), Subcontracting
(16,4%; please note that this figure is affected by the effect of one single very
important adjustment) and Indirect costs/Overheads (14,5%).
For adjustments in favour of the beneficiaries:
Like for negative adjustments, the situation in 2008 fairly mimics the cumulative
results.
However, the category with the highest cumulative adjustments is Indirect
costs/Overheads (57,1% in value, 29,3% in number), followed by Personnel (only
18% in value and 22,1% in number).
Compared to negative adjustments, there is a slightly higher incidence of positive
adjustments in Adjustments to costs previously reported (which is to be expected), but
also in Other costs. By contrast, there is a lower incidence of Personnel and
Subcontracting costs.
3.3.3.
Qualitative analysis of the largest adjustments in absolute terms
(FP6)
The 10 biggest negative adjustments proposed in audits closed in 2008 are listed below, and
there is also a brief explanation of the nature of the errors found in each case. These 10
30 March 2009
32
CONFIDENTIAL
adjustments represent about 25% in monetary value of all adjustments proposed so far in FP6
audits.
Table 3.16 Details of the 10 largest adjustments in absolute terms in 2008 (all figures are
EC share)
Contractor name
CY
CONTRACT
STRATEGY
DIR
Costs
Costs
Costs
Adjustment (EC share)
claimed and
accepted by
accepted by
audited
Financial
Auditor
Officers
1
ST GEORGE'S HOSPITAL
UK
503240
TOP
F
1.728.375,58
1.728.375,58
926.552,57
-801.823,01
MEDICAL SCHOOL
2
GESELLSCHAFT FUER
DE
RIDS-CT-
TOP
B
2.987.846,50
2.987.846,50
2.392.008,59
-595.837,91
SCHWERIONENFORSCHUNG
2005-
MBH.
515873
3
SCIENCE AND
UK
RICN-CT-
TOP
B
5.622.680,99
5.549.748,13
5.084.896,75
-464.851,37
TECHNOLOGY FACILITIES
2005-
COUNCIL
011723
4
UNIVERSITY MEDICAL
NL
LSHM-CT-
TOP
F
694.049,00
694.049,00
378.429,40
-315.619,60
CENTRE UTRECHT
2005-
518226
5
DASSAULT AVIATION SA
FR
AIP4-CT-
TOP
H
1.755.040,25
1.755.040,25
1.457.654,39
-297.385,87
2005-
516092
6
GOETEBORG UNIVERSITY
SE
11719
TOP
B
2.385.515,63
2.385.515,63
2.088.631,41
-296.884,22
7
COCKERILL SAMBRE -
BE
NMP3-CT-
RISK
G
993.989,77
993.989,77
796.210,14
-197.779,63
ARCELOR
2005-
515703
8
UNIVERSITY OF BELGRADE
RS
INCO-CT-
RISK
D
467.542,86
467.542,86
292.498,28
-175.044,58
2004-
509213
9
EUROCOPTER SAS
FR
AIP3-CT-
TOP
H
1.083.263,93
1.083.263,93
918.608,65
-164.655,29
2003-
502917
10
CENTRE EUROPEEN DE
FR
LSHG-CT-
TOP
F
820.192,03
820.192,03
663.539,33
-156.652,70
RECHERCHE EN BIOLOGIE
2004-
ET MEDECINE
512003
Details about each case (numbers as in table above):
1. Personnel: Staff involved in the project has also been working on several other
activities and projects, but their salaries have been charged to the project irrespectively
of the work actually performed, but pending only on the budget available. The
contractor does not keep any time record system that could allow defining reasonable
criteria for allocating these costs correctly. Personnel costs were therefore
recalculated. Total adjustment: EUR 129.330.
Subcontracting: Two subcontracts did not conform to bidding rules, to contractual
provisions and to the FP6 Rules of Participation. Total adjustment: EUR 540.356. One
of them was allocated by the coordinator (a SME) to St. George's Hospital because it
would have granted a higher EU co-financing.
2. The correction in the case of the Gesellschaft für Schwerionenforschung (GSI) has
three main elements: - EUR 599.263,00 (adjustments to preliminary personnel costs
previously reported to get actual costs), + EUR 2.253.027,45 (enlargement of electric
power supply which was erroneously not claimed) and - EUR 1.206.169,99 (two
elements: disallowance of costs incurred before project start; instead of cost-
accounting rental fees, depreciation should have been used to get actual costs).
30 March 2009
33
CONFIDENTIAL
3. In the audit of the Science and Technology Facilities Council the main finding
concerns two incorrect manual postings relating to an invoice from a building
company that had already been booked. As a result the cost was included three times
in the cost statement (Form C). This finding lead to a disallowance of EUR
4.622.529,74.
4. The auditors disallowed EUR 340.000 for subcontracting not foreseen in Annex I and
for which no competitive tendering was performed. The costs represent about 50% of
the costs claimed. The contractor did not agree with the position taken by the auditors.
The comments and documents provided by the contractor gave no reason to the
auditors to change the approach. An amendment to Annex I to foresee
subcontracting is currently ongoing. The contractor might claim these costs in a
subsequent cost statement as 'Adjustments to costs previously reported', provided that
the contract is amended.
5. Dassault Aviation has not been able to justify all the costs claimed in the Form C.
There was no detail of the costs included in the Financial Statement available.
Therefore the auditors have only accepted those costs presented and duly justified
during the audit (EUR 297.385,87).
6. The personnel costs (EUR 218.023,53) of the permanent employees that did not fulfil
the conditions laid down in Article II.20 of the Contract have been disallowed. The
flat rate 20% indirect cost amounting to EUR 35.255,96 for management personnel
has been disallowed, because in AC contracts indirect costs cannot be charged for
permanent personnel. In addition indirect cost amounting to EUR 43.604,73 has been
disallowed based on the above mentioned changes in direct costs.
7. Adjustments for two reasons: firstly by the payment of legal fees amounting to around
EUR 160.000 which were considered by the auditors as sub-contracting not foreseen
in the Annex 1. Secondly, there was an amount of around EUR 70.000 related to
salary costs of a researcher. This second amount is "suspended" and may become
payable at a later stage pending the outcome of legal proceedings against this person.
8. The contractor did not provide sufficient documentation for costs claimed under
'labour' and 'consumable', and subsequently also overhead costs were corrected
downwards.
9. Main adjustments in the following cost categories:
Personnel costs: EUR -732.431,96 Deviation of average personnel costs from
actual costs. Certain indirect costs are claimed as personnel costs
(reclassification).
Subcontracting: EUR -26.800,00 Lack of substantiation
Other direct costs: EUR -8.489,57 Mix of VAT, lack of substantiation
and double claiming.
Other costs claimed not classified: EUR -71.113,12 Lack of substantiation.
Indirect costs: EUR +509.524,08 Costs claimed based on estimated figures.
Reclassification from personnel costs. Systematic issue taken to extrapolation.
10. Main adjustment due to the fact that the contractor depreciated equipment in full in the
year of acquisition as opposed to depreciating it over its useful life in accordance with
its normal accounting policies.
30 March 2009
34
CONFIDENTIAL
3.3.4.
Qualitative analysis of error types (FP6)
Each time an audit is closed, each audited participation is given two ratings related to
'Problem level' and 'Problem nature' by the auditors. By using a combination of these two
ratings, we can have an understanding of the seriousness of the findings, as shown in the table
below20.
Table 3.17 Types and incidence of errors found at participation level (cumulative, FP6)
Cumulative
Problem nature
Problem
NONE
QUALITATIVE
ERROR
IRREGULARITIES
Totals
level
%
%
%
%
%
NONE
17.3%
0.2%
0.5%
0.0%
18.0%
SMALL
0.8%
1.3%
53.9%
0.3%
56.3%
MEDIUM
0.0%
1.4%
19.6%
0.5%
21.4%
HIGH
0.0%
0.1%
3.7%
0.5%
4.2%
Totals
18.0%
3.0%
77.8%
1.2%
100.0%
Most of the adjustments proposed by the external audit units are small. Discoveries of
fraud are rare. This situation is reflected by this table in the 53,9% of participations
showing SMALL ERROR. This conforms to what the Court of Auditors discovers in
its DAS reports.
The percentage of participations where potential irregularities and highly serious
problems are found remains fairly low, at 0,5%. However, these few large adjustments
have an important impact on the error rate. The above analysis explaining the 10 cases
that make up 25% of the error rate confirms this.
In 17,3% of the cases, there were no findings.
3.3.5.
Audit coverage (FP6)
Table 3.18 Audit coverage (cumulative, FP6)
Total number of
55577
Audit coverage by
participations
number of audited
2,38%
Audited participations
1320
participations
(CORDA, 14/01/09)
Total EC payments as
9.178.230.000
Audit coverage by
of end 2008 (R2 data)
6,24%
amounts audited
Audited EC contribution
572.881.964
This is the first year in which audit coverage figures are provided, so it is not yet
possible to provide a comparison year-on-year.
20 Problem level refers to the severity of problems found (NONE, SMALL, MEDIUM or HIGH), while problem
nature reflects the character of those errors (NONE, QUALITATIVE, ERROR or IRREGULARITIES).
30 March 2009
35
CONFIDENTIAL
4.
OUTLOOK FOR 2009
2009 can be considered as a transition year between theoretically the last auditing efforts in
FP6 and the first steps towards auditing FP7. Consequently, our main objectives will be
twofold: on the one hand, to continue assessing whether the FP6 audit strategy is delivering
its intended outcomes, and to apply any further corrective actions which might be needed; and
on the other, to devise a comprehensive and adequate audit strategy for FP7, ready for
implementation.
4.1. Audit Targets
According to the multi-annual audit plan of the ABM, the audit targets for DG RTD covering
both FP6 and FP7 for the next three years are as follows:
Table 4.1. Multi-annual audit targets (FP6 and FP7)
2009
2010
FP6 target
384
215
78
FP7 target
25
221
344
Total
409
436
422
Two further points are important when considering these targets:
(a) DG RTD tries to achieve a rhythm of between 350 and 400 closed audits per year, of
which between 250 and 300 (75%) should be outsourced audits.
(b) The multi-annual audit plan foresaw an overall minimum target of 750 FP6 audits for
DG RTD. This target has been increased to 896 audits in the latest ABM Progress
Report. On January 1st 2009, there were 272 open FP6 audits, and the cumulative
number of FP6 closed audits to date was 599.
Table 4.2. Audit situation as of 01/01/2009
2008
Closed up to
open as of
open as of
end 2007 (PM)
1/01/2008
open
closed
01/01/2009
FP
Strategy strand
actual
planned
actual
FP6
TOP
72
104
167
145
147
129
MUS
63
71
19
98
86
5
RISK
13
6
227
140
123
126
FUSION
7
7
9
13
9
OTHER
72
2
3
3
3
Total FP6
227
190
425
384
372
272
FP5
N/A
694
7
5
6
9
3
Coal & Steel
N/A
0
0
5
5
2
3
Grand totals
921
197
435
395
383
278
30 March 2009
36
CONFIDENTIAL
In contrast to these figures, early indications of the number of auditable FP7 participations at
different points in time in 2009 show that it will not be possible to achieve the FP7 multi-
annual target for 2009 of 221 closed audits shown above. The main reason for this lack of
auditable participations is the extension of reporting periods from 12 months (FP6) to 18
months (FP7). Moreover, the fact that the FP7 population is still growing quickly (i.e. many
new projects are being launched and added to the population) makes it almost impossible to
prepare sufficiently representative samples at this early stage of the FP's life-cycle.
Simply increasing the number of FP6 audits in order to make up for the planned FP7 audits
which will not be launched yet would be the easiest option. However, such a decision must be
taken in the light of cost-benefit considerations. The purpose is not to "over-audit".
Bearing in mind both the 'delayed' start to FP7 auditing and the advanced state of the FP6
audit campaign, it is therefore necessary to partially review our activity in 2009, as explained
in the next section.
4.2. Outlook of the audit activities in 2009
Regarding FP6, there are sufficient ongoing FP6 audits to reach the FP6 audit targets, both for
2009 and overall, keeping in mind however that there are certain elements of the application
of the Audit Strategy currently being discussed as part of the FP6 Audit Strategy Mid-Term
Review within the CAR. One of these is the definition of common criteria to decide when the
budget of a given contractor can be considered, in accordance with the Audit Strategy, "clean"
from systematic material errors.
Furthermore, additional action may be required if the analysis of the proportion systematic
versus non-systematic error is not in line with the original assumption of the FP 6 Audit
Strategy (= that the large majority of error is systematic).
Another point to consider is that, in cases of extrapolation, and after reception of all revised
cost statements (or lack of reception if contractors do not co-operate), follow-up audits will be
needed in order to assess the correctness of the corrections proposed by the contractors.
Concerning FP7, and although an audit strategy has not yet been devised, it can already be
foreseen that the audit approach will have to be refined and adapted to the FP7 legal and
operational framework. One particular aspect is the proper audit treatment of the issue of
average versus actual personnel costs, for which a tailor-made audit programme has been
developed and will be put to the test.
The most significant change to be expected is the fact that there will be a more systems-based
approach in the audits. This corresponds to the need to improve our audit methodology.
In the new audit programme, an audit will be carried out in two phases.
The first or "interim" phase would involve an on-the-spot intervention designed to gather
knowledge of a contractors' overall structure, of their internal control framework, and of the
main accounting procedures generating the costs declared (most probably this will concern
personnel and overheads, but also travel costs or fixed assets – depreciation – where these
categories can be particularly important, as in the
Infrastructure programme). This first
intervention should result in a detailed description of the main procedures, as well as "walk-
through" tests on a limited number of transactions to confirm the understanding of the
procedures, and their functioning according to the explanations received. No audit
30 March 2009
37
CONFIDENTIAL
recommendations with financial effect would result from these audits. After the mission on-
the-spot, the auditor should spend sufficient time getting familiar with the system of the
contractor, and defining, accordingly, a specific programme for auditing transactions (for
example adapting the audit effort per cost category, or defining the need to visit more than
one regional branch of the contractor).
The second or "final" intervention, would focus on a substantive test of transactions, and
would result in an opinion on the costs declared including, if applicable, issuing
recommendations with financial implications (proposed adjustments).
The scope of these audits will be mostly to review the systems in place at the contractor which
are used for the preparation of the periodic cost statements. Such audits will take longer to
complete. However, their results may have the following benefits:
Support decisions on certificates of methodology under FP7.
In cases where extrapolation has been launched, they will confirm whether the
extrapolation decision was correct and if the budget is in effect 'clean' from the
systematic errors detected at that contractor.
It must be underlined that this shift towards more in-depth audits will inevitably result in a
reduction of the number of audits undertaken in 2009. This quantitative change in targets will
be counterbalanced by a more qualitative focus of the auditing effort.
30 March 2009
38
CONFIDENTIAL
ANNEX I: MISSION STATEMENTS
MISSION STATEMENT UNIT RTD.A.4: EXTERNAL AUDITS
The Unit contributes to the assessment of the legality and regularity of the DG RTD payment
transactions by means of ex post financial audits, thereby providing a basis of reasonable assurance to
senior management and other stakeholders (including the budget discharge authorities) that RTD
contract participants are in compliance with the financial terms of the RTD contract. The corrective
actions and follow-up measures which result from the ex post audit activity contribute to the
protection and safeguarding of the European Union’s financial interests in the research area.
The Unit performs, mainly with own audit staff and occasionally through independent professional
audit firms, a number of audits (“on-the-spot-controls”) each year, which are selected from the
“auditable population” of RTD contractors, and ensures that these audits are professionally
managed and supervised.
The Unit evaluates, reports, and monitors on a regular basis the requests for financial audits made
by the DG RTD Directorates or other relevant parties. The Unit evaluates these requests and
carries out financial audits as necessary with the required priority and urgency.
The Unit uses and maintains specific tools and methodologies for the selection of RTD contractors
to be audited. The selection is based on the multi-annual audit strategy as endorsed by the DG,
and focuses on achieving sufficient and representative audit coverage to support the DGs annual
assurance declaration.
The Unit provides on regular basis management information as a result of the “on-the-spot-
controls”. For those RTD contractors who fail to comply with the contract the Unit recommends
financial adjustments and in case of systematic errors, extrapolation of such adjustments towards
non-audited transactions.
The Unit, after analysis and synthesis of audit results, gives feedback on corrective actions, and
centralises the regular reporting of actions taken or to be taken by the RTD Directorates on the
basis of the information available in the Audit Back-Office.
The Unit, through close co-operation and harmonisation with the other Research DG’s, takes the
lead in establishing relevant audit policies and strategies. It therefore organizes chairs and ensures
the secretariat for the monthly CAR group meetings.
The Unit contributes to the understanding and application of the legal RTD framework through
interpretation and guidelines on FP RTD financial and accounting matters. The unit also
contributes in an advisory capacity not only to auditing and accountancy questions and tasks, but
also to the legal developments of (future) participation rules and model RTD grant agreements.
The Unit liaises with R5 to provide a timely input for the interactions with the Court of Auditors.
It performs a similar role in relation to the internal audit capability and through the latter vis-à-vis
OLAF.
The unit ensures the "Back-Office" function for the DG RTD external audit activity by
maintaining an audit workflow application and database, and ensuring the lead on the development
of tools and procedures for the sharing of audit results and follow-up measures with other
Research DGs and beyond, for example through the ABAC audit module.
30 March 2009
39
CONFIDENTIAL
MISSION STATEMENT Unit RTD.A.5 "Implementation of Audit certification policy and
outsourced audits"
The Unit contributes to the assessment of the legality and regularity of the DG RTD payment
transactions by means of ex post financial audits, thereby providing a basis of reasonable assurance to
senior management and other stakeholders (including the budget discharge authorities) that RTD
contract participants are in compliance with the financial terms of the RTD contract. The corrective
actions and follow-up measures which result from the ex post audit activity contribute to the
protection and safeguarding of the European Union’s financial interests in the research area.
Through the certification function for FP7, the unit aims to contribute in an ex ante manner to the
legality and regularity of future DG RTD payment transactions by ensuring that the cost methodology
systems of FP7 beneficiaries are in compliance with the rules, thereby resolving main errors observed
in the past from the outset.
The unit's missions can be broken down as follows:
To perform, exclusively through independent professional audit firms, a number of batch audits
each year, which are selected from the “auditable population” of RTD contractors, and ensure that
these audits are professionally managed and supervised, by proper planning and follow-up of audit
assignments, quality control of deliverables, liaison with external audit firm representatives and
other DGs of the "research family".
On the basis of the audit reports of the professional audit firms, for those RTD contractors that fail
to adhere to the contract, the Unit recommends financial adjustments and, in case of systemic
errors, the extrapolation of such adjustments to non-audited transactions.
To manage the public procurement and follow-up of the audit service framework contracts.
To ensure support to the implementation of the audit certification, focusing in particular on the
cost methodology certification process introduced under FP7. Upon request, the unit also offers
advice and guidance on the implementation of the FP6 audit certificate function.
To monitor the implementation of the audit certificate policy in general and co-ordinate all matters
related to audit certification with other DGs of the research family and vis-à-vis DG BUDG.
Where applicable, the unit ensures liaison with national or international professional audit bodies.
The Unit liaises with R5 to provide a timely input for the interactions with the Court of Auditors
for matters linked to audit certification and audit service framework contract matters.
The Unit contributes in an advisory capacity to the legal developments of (future) participation
rules and model RTD grant agreements, in particular based upon the knowledge gained in the
certification process.
30 March 2009
40