EUROPEAN COMMISSION
DIRECTORATE-GENERAL FOR RESEARCH & INNOVATION
Directorate M - Management Operational Support - Framework Programme
M.1 - External audits
ANNUAL ACTIVITY REPORT ON EXTERNAL AUDITS
2012
Commission européenne/Europese Commissie, 1049 Bruxelles/Brussel, BELGIQUE/BELGIË - Tel. +32 22991111
Page
2 of
48
link to page 5 link to page 7 link to page 7 link to page 7 link to page 7 link to page 8 link to page 8 link to page 9 link to page 9 link to page 11 link to page 11 link to page 12 link to page 13 link to page 15 link to page 16 link to page 17 link to page 18 link to page 18 link to page 18 link to page 19 link to page 21 link to page 22 link to page 22 link to page 23 link to page 24 link to page 24 link to page 25 link to page 25 link to page 26 link to page 27 link to page 28 link to page 28 link to page 29 link to page 29 link to page 29 link to page 31 link to page 32 link to page 32 link to page 34 link to page 37
Table of contents
Executive Summary ______________________________________________ 5
1. Background _________________________________________________ 7
1.1. Introduction ____________________________________________________________ 7
1.2. Legal background _______________________________________________________ 7
1.3. The mission of the external audit units ______________________________________ 7
1.4. Role within the overall internal control framework activities of DG Research &
Innovation _________________________________________________________________ 8
2. Activities ____________________________________________________ 9
2.1. Annual Audit Plan 2012 and Key Performance Indicators (KPIs) ________________ 9
2.2. The audit campaigns ____________________________________________________ 11
2.2.1. The FP6 audit campaign ___________________________________________ 11
2.2.2. The FP7 audit campaign ___________________________________________ 12
2.2.3. Additional auditing commitments ____________________________________ 13
2.3. Cross-RDG coordination _________________________________________________ 15
2.3.1. Coordination of audits in the Research family (CAR) ____________________ 16
2.3.2. Other coordination Committees _____________________________________ 17
2.4. Extrapolation __________________________________________________________ 18
2.4.1. Extrapolation policy and coordination ________________________________ 18
2.4.2. RTD extrapolation cases ___________________________________________ 18
2.4.3. Extrapolation implementation _______________________________________ 19
2.5. OLAF cases ____________________________________________________________ 21
2.6. Management and quality control tools ______________________________________ 22
2.6.1. Management and quality control _____________________________________ 22
2.6.2. Keywords Working Group (KWG) ___________________________________ 23
2.6.3. The Audit Steering Committee (ASC) ________________________________ 24
2.6.4. Establishment of an Audits' Internal Supervision Committee (AISC) ________ 24
2.7. Collaboration with the DG RTD administration and finance (UAF) network______ 25
2.8. IT developments ________________________________________________________ 25
2.9. FP7 methodology certification ____________________________________________ 26
2.9.1. State of play of certification files as of 31 December 2012 ________________ 27
2.9.2. Inter-service collaboration and communication activities (cf. 2.11.) _________ 28
2.10.
Coordination of outsourced audits ______________________________________ 28
2.11.
Communication Campaign ____________________________________________ 29
2.12.
Other activities ______________________________________________________ 29
2.12.1.
Art. 185 Initiatives _____________________________________________ 29
2.12.2.
Access to documents ___________________________________________ 31
3. Results and Analysis _________________________________________ 32
3.1. Audit numbers _________________________________________________________ 32
3.2. Audit results ___________________________________________________________ 34
3.3. Audit coverage _________________________________________________________ 37
Page
3 of
48
link to page 39 link to page 39 link to page 43 link to page 44 link to page 45 link to page 46 link to page 48
3.4. Analysis _______________________________________________________________ 39
3.4.1. Analysis of error rates _____________________________________________ 39
3.4.2. Assessment of the different steps of the control chain ____________________ 43
3.4.3. Qualitative analysis of error types ____________________________________ 44
3.4.4. Analysis of adjustments at cost category level __________________________ 45
3.4.5. Analysis of the FP7 representative error rate ___________________________ 46
ANNEX I: Mission statement _____________________________________ 48
Page
4 of
48
EXECUTIVE SUMMARY
In 2012, the emphasis of the work of the external audit unit has been on the further
implementation of the FP7 Audit Strategy. In total, 308 audits were closed by RTD M.1 in
2012, of which 265 audits were part of the FP7 audit campaign, the remainder being 40 FP6
audits and three Coal and Steel audits.
2012 was also the year of the first FP7 Common Representative Sample (CRS) for the
Research family of DGs and Executive Agencies. This new approach to providing assurance
has enabled the Commission services to reduce the total number of planned FP7 audits by
25%.
On 1 February 2013, results were known for 136 out of the 162 cost statements selected in the
CRS; the resulting common representative error rate was -4.18%. More than 70% of the errors
stem from the 'Personnel Cost' and 'Indirect Costs' categories. The vast majority of the errors
are due to a lack of supporting documentation and relate to issues associated with personnel
cost (lack of contracts, incorrect or irregular timesheets, lack of extracts of payroll, lack of
invoices for in-house consultants, etc.). Erroneous calculations of depreciation charges or
their wrong application comes second. Errors related to subcontracting come third. These
findings are in line with the errors that we explain and try to remedy through the
Communication Campaign which covered 12 countries in 2012. This campaign focuses on
beneficiaries as well as certifying auditors, and it is generally regarded as a success. It will
continue in 2013. Further analysis of the error rates has also shown that newcomers to the FP
and SMEs are particularly error prone. As a result, additional preventive measures might be
usefully considered.
Of the closed audits, approximately one third are done by in-house auditors. For this audit
work, 2012 was a year in which radical overhaul of the quality control arrangements within
the unit took place. In addition, a set of performance indicators was introduced that will allow
us to track the evolution of the unit's performance over the years to come. Some of these
indicators will have to be refined in the light of the first-year experience. Additional quality
assurance was introduced by setting up the Audit Internal Supervision Committee (AISC),
chaired by the Director. The final aim of all these measures is to increase productivity,
efficiency and quality. Timely delivery of audit reports is also a target.
The unit also manages, on behalf of the Research Family, the framework contracts with three
private audit firms to whom two thirds of the audits are outsourced. The existing framework
contract expires in June 2013 and, in order to ensure the availability of outsourcing capacity
following this date, a call for tender for a new framework contract was launched in 2012. The
closing date for submission of the tenders was 23 November 2012. The tenders received are
being assessed and it is expected that the new framework contract will be ready by June 2013.
Whilst we are in the middle of the FP7 audit campaign, the FP6 campaign has gradually been
winding down (42 FP6 audits remain to be closed). Both campaigns have hitherto led to
proposed adjustments for a total of ± EUR 71 000 000 in favour of the Commission.
In addition to the FP6 and FP7 audit campaigns, the unit manages a wide variety of issues
related to the assurance-gathering process. A significant amount of resources are dedicated to
cross-RDG coordination (e.g. 32% of the working time of the in-house auditors), which from
2012 onwards included not only all research Commission services but also RTD's three Joint
Page
5 of
48
Undertakings (IMI, Cleansky, FCH). Also the work with OLAF should be referred to. In that
context, efforts undertaken by this unit to implement plagiarism detection and IT tools that
facilitate advanced data searches should be highlighted, together with the initiative to build
constructive relations with national research funding organisation with the purpose of jointly
combatting fraud. Effective audit management is also underpinned by the development of
adequate IT-tools; specific reference should be made to our audit management system
AUDEX, which was greatly improved in the course of 2012.
As a concluding remark, let us underline that in the context of the Declaration of Assurance
for 2011, the European Court of Auditors concluded that
"overall the system of ex-post audits
[…] was assessed as effective".
Page
6 of
48
1.
BACKGROUND
1.1. Introduction
The purpose of this document is to report on the ex-post audit activities in DG RTD during
2012, using the numerical results of the verifications carried out and providing feedback on
relevant qualitative issues. Some of the numerical results and some of the qualitative issues
included in this report also contribute to the assurance statement of the Director General on
the legality and regularity of financial transactions in DG RTD's Annual Activity Report.
1.2. Legal background
For FP6, the legal basis for the external audit activity is Annex III point 2, paragraph 7 of the
Decision n° 1513/2002/EC of the European Parliament and of the Council, and Article 18 of
Regulation (EC) n° 2321/2002 of the European Parliament and of the Council. For FP7,
reference must be made to Article 5 of the Decision n° 1982/2006/EC of the European
Parliament and of the Council, and Article 19 of Regulation (EC) n° 1906/2006 of the
European Parliament and of the Council.
The model grant agreement for the 7th Framework Programme (Annex II, Article 22) states
that: '
the Commission may, at any time during the contract, and up to five years after the end
of the project, arrange for audits to be carried out, either by outside scientific or
technological reviewers or auditors, or by the Commission departments themselves including
OLAF'.
Similar provisions are foreseen in the model contract for the 6th Framework Programme
(Annex II, Article 29).
1.3. The mission of the external audit units
The external audit unit provides a level of reasonable assurance to senior management and,
ultimately, to the Discharge Authority (European Parliament and Council) on whether DG
RTD beneficiaries are in compliance with the terms of DG RTD's grant agreements. This is
done through the execution of ex-post financial audits; ex-post audit results provide a
representative error rate and initiate the budgetary implementation of the adjustments
proposed including, where needed, financial recoveries or offsets managed by the operational
services. Thus, the external audit function contributes to the sound financial management of
the EU funds and to the protection of the European Union’s financial interests.
In 2012, it was decided to discontinue the existence of two units (formerly RTD M.1 and
RTD M.2), and to integrate RTD M.2 as a sector of RTD M.1 from 1 January 20131. The new
mission statement of the new M.1 unit is in Annex I.
1 As the unit was still split into M.1 and M.2 during 2012, they will be differentiated in that manner throughout
this annual report where relevant.
Page
7 of
48
1.4. Role within the overall internal control framework activities of DG Research
& Innovation
Ex-post audit activities need to be seen as part of the overall internal control framework put in
place by the Directorate General, together with ex-ante and ex-post evaluations, financial and
scientific verifications, and monitoring tools.
In the area of grant management for research expenditure, the focus remains very much on
controls after payment (ex-post), reducing controls before payment (ex-ante) as much as
possible. This is a conscious decision with the aim of reducing the ex-ante administrative
burden for the beneficiaries, and therefore shortening the average time-to-grant and time-to-
pay periods.
This conception of the internal control system is regularly under discussion because the
results of ex-post controls come relatively late in the process of grant implementation and are
often being contested by beneficiaries who claim not to have been aware of certain aspects
and details of the legal and regulatory framework. This has led to an intensification of the
Commission's communication efforts towards beneficiaries. This is especially important in
FP7, given the above-mentioned decision to limit ex-ante verifications: for example, audit
certificates are only required if the grant amount is above EUR 375 000. In addition, the ex-
ante certification procedures introduced for personnel and indirect costs' methodologies and
for average personnel costs have not been taken up by the expected number of beneficiaries.
The field experience gathered by the unit over the years is increasingly considered as a
valuable asset. It is particularly appreciated as a useful source of feedback to operational
services, and for its contribution towards the design of better legislative measures. The best
example is the input into the Commission proposal on the future regulatory framework for
Horizon 2020, in particular on issues such as bonus payments or flat rates for indirect costs.
Finally, and as was the case the previous year, the European Court of Auditors gave a positive
opinion in their Declaration of Assurance for 20112 of ex-post financial audit activities, as
part of their assessment of selected supervisory and control systems in Research and other
internal policies.
2 Chapter 8 'Research and other internal policies' of ECA's Annual Report 2011, Annex 8.2.
Page
8 of
48
2.
ACTIVITIES
2.1. Annual Audit Plan 2012 and Key Performance Indicators (KPIs)
The external audit units organised their work on the basis of an Annual Audit Plan (AAP) for
the first time in 20123. This document included a summary of the actions to be taken during
the year as part of the ongoing implementation of the FP6 and FP7 audit strategies, plus any
other auditing commitments. In response to a recommendation from the IAS, the units
decided to implement a number of KPIs.
The AAP included the set of Key Performance Indicators (KPIs) to be achieved by the end of
the year. These came on top of the indicators already monitored in previous years, such as the
number of audit reports closed (per unit and per auditor, in-house and outsourced), and the
number of mission days per auditor.
The KPIs discussed below relate exclusively to in-house audits. Outsourced audits are
contracted in batches, normally of seven months (although they can be as long as ten months).
When the entire batch is not delivered within the contracted period, an audit firm is liable to
pay liquidated damages. It is therefore its responsibility to ensure the timely delivery of draft
reports.
The KPIs set in the 2012 Annual Audit Plan are the following:
Key Performance Indicators
Target
Actual
Number of audits launched in the year vs annual target
250
321
Number of audits closed in the year vs annual target
330
308
Average time between the end of the audit mission and the first draft
3 months
51% within
of the audit report being ready
(for 80% of FP7
three months
audits)
Number of participations in communication events towards
11
11 (covering
stakeholders
12 countries)4
Number of events organised to raise awareness of fraud prevention
9
9
measures
Number of audits focused on detecting fraud
12 to 15
12
% of draft audit reports sent to Quality Control which need significant
<5%
30,5 %
changes before being sent to operational services
Results related to those KPIs are reported throughout this document where appropriate, with
the two most important ones meriting some further explanation:
3 Ares(2012)470849 of 17/04/2012
4 See section 2.11.
Page
9 of
48
KPI-1 - Average time between the end of the on-the-spot mission and the draft audit
report
Target: less than three months for 80% of FP7 in-house audits closed in 2012
Result for 2012: 51% within three months
Average number of days to produce a draft report: 132 calendar days
This KPI focuses on draft audit report delivery delays, as a measure of productivity. There are
a number of reasons for this relatively poor performance:
All the audits resulting from the Common Representative Sample had to be closed by
31 December 2012. This required concentrating the planning of their on-the-spot
missions during the first semester of 2012, hence unavoidably delaying the drafting of
these and other draft audit reports.
The unit is responsible for a heterogeneous number of horizontal affairs, linked to a
varying degree to purely auditing activities. These affairs account for 32% of the
auditors' time. Auditors cannot therefore always fully dedicate themselves to the
timely drafting of audit reports.
When requesting documentation to be sent before the on-the-spot mission or during
the elaboration of the draft report, the degree of beneficiaries' cooperation varies,
sometimes causing considerable delays in the elaboration of draft reports. A more
forceful approach towards non-cooperative beneficiaries, although permitted by the
legal framework, is mostly considered as non-viable and not appropriate.
A management supervision exercise5 has been launched to assess possibilities for
improvement. Elements under discussion include a clearer split between staff involved in
horizontal affairs and auditors involved exclusively in audits, setting up a team structure with
closer supervision of individual audit(ors) and/or increasing the specialisation of the auditors
(for SMEs, for universities, for industrial companies, etc…).
KPI-2 - % of draft audit reports which need significant changes before being sent to
operational services
Target: less than 5%
Result for 2012: 30.5%
This KPI focuses on the quality of the output. This relatively poor performance needs to be
qualified by the following remarks:
The original target of 5% was set out for the first time in 2012 and may need to be
reconsidered, on the basis of more experience.
As a conscious decision, the unit has reinforced its quality control tools at all stages
(including the draft stage). This has led to an increased number of quality control
remarks, including those qualified as 'significant issues'.
Moreover, the definition of 'significant issues' has been broadened, leading to
including a higher proportion of remarks in that category.
5 ICS 9 – see http://intranet-rtd.rtd.cec.eu.int/politique/n-ics/docs/Guidance_Supervision.pdf
Page
10 of
48
This KPI will be revised in the AAP 2013.
2.2. The audit campaigns
2.2.1.
The FP6 audit campaign
The overall results of our audit campaign for FP6 are summarised in the following table:
Table 2.1 - Overall figures for the FP6 audit campaign
EC share of the accumulated
Number
EC share of
adjustments in favour of the EC
of
Number of
the costs
Annual
Cumulative
audits
participations
accepted by
error
error rate
Representative
Residual error
Year
closed
audited
the FO (€)
Amount (€)
rate %
%
error rate %
rate %
Up to end of 2011
1161
2708
1,114,322,817
-46,957,043
-4.21%
2012
40
126
60,111,417
-8,579,090
-14.27%
-4.73%
Total
1201
2834
1,174,434,234
-55,536,133
of which, TOP + MUS
553
1385
572,552,778
-19,699,932
-3.44%
-2.47%
The FP6 Audit Strategy (FP6 AS) was designed after the critical Discharge procedure in
2006, and focused on increasing the number of audits, improving the consistency of approach
and the coherence of conclusions, ensuring more homogeneous audit policies among the
Commission research services6, calculating reliable and representative error rates, and
introducing the extrapolation of audit results as a corrective measure.
The FP6 AS originally assumed that most of the errors found during the audits would be of a
systematic nature, and that 750 audits would be sufficient to eliminate them from at least 40%
of the DG RTD FP6 budget and, in doing so, to achieve the control objective of a residual
error rate of 2% or lower at the end of the multiannual FP6 audit campaign.
The mid-term review of the FP6 AS concluded in early 2009 that the proportion of systematic
errors was much lower than anticipated (at the end of 2012 it is still only 36% of all errors in
terms of amounts in DG RTD). Increasing the total number of audits was then considered
necessary to keep alive the possibility of still correcting enough errors to be below 2%. This
decision was again re-assessed at the end of 2010; additional attempts to keep the residual
error rate below the control objective of 2% were considered as not cost effective. It was
therefore decided that no further FP6 audits would be launched other than those related to
fraud and irregularities investigations, joint audits with the European Court of Auditors or
audits requested by operational services.
At the end of 2012, 1201 FP6 audits have been closed in DG RTD and, when including the
audits still ongoing, the total will eventually be around 1245. Audit coverage7 from these
audits and those undertaken by other Commission services stands at 61% of the RTD FP6
6 The Commission research services are DG RTD, DG CNECT, DG MOVE, DG ENER, DG ENTR, DG EAC
and the two Executive Agencies ERCEA and REA. The JUs linked to RTD are Clean Sky (Aeronautics and Air
Transport), IMI (Innovative Medicines Initiative) and FCH (Fuel Cells and Hydrogen Initiative). The one linked
to ENER/MOVE is SESAR (Single European Sky ATM Research).
7 Audit coverage includes both the amounts directly audited and the non-audited amounts received by audited
beneficiaries from which systematic errors have been removed. See table 3.9.
Page
11 of
48
budget, and the residual error rate is -2.47%. This rate has hardly changed since last year and,
with so few FP6 audits still ongoing, it is fair to assume that the final rate at the end of the
campaign will be around that value.
Currently, there are still 42 ongoing FP6 audits. The objective is to finalise the large majority
of them in the course of 2013.
2.2.2.
The FP7 audit campaign
The overall results of our audit campaign for FP7 are summarised in the following table:
Table 2.2 - Overall figures for the FP7 audit campaign
EC share of the accumulated
Number
EC share of
adjustments in favour of the EC
of
Number of
the costs
Annual
Cumulati
audits
participations
accepted by
error
ve error
Representative
Residual error
Year
closed
audited
the FO (€)
Amount (€)
rate %
rate %
error rate %
rate %
Up to the end of 2011
441
843
195,933,944
-7,384,644
-3.77%
2012
265
600
238,001,866
-7,829,757
-3.29%
-3.51%
Total
706
1443
433,935,810
-15,214,401
-4.18%
-3.23%
The FP7 audit campaign completed its third full year in 2012, when 304 audits were launched
and 265 were closed.
Representative audits
A relatively small number of audits are undertaken on a regular basis in order to identify the
percentage of errors affecting the whole budget. The beneficiaries to be audited are selected
using statistically representative sampling methods and, therefore, the results of those audits
provide error rates which are also statistically representative.
In the first years of the FP7 campaign, each research Commission service drew and audited its
own audit samples. Two of these samples were taken in DG RTD, and their results were the
basis for error rate reporting until the end of 2011.
However, this approach created a number of planning constraints and administrative burdens
on beneficiaries, which were considered unnecessary. During 2011, the external audit units in
RTD launched an initiative to change the assurance system for research Commission services
towards a unique assurance model. After necessary discussions in the CAR (Coordination
group for external audit in the Research family), the RCC (Research Clearing Committee) and
the ABM (Activity-Based Management Steering Group), an agreement was reached in
November 2011 to use a single representative error rate for all services from 2012 onwards,
based on the results from a Common Representative Sample. In this CRS, 162 cost statements
for audit were selected.
At the end of 2012, the FP7 representative error rate is -4.18%, based on 136 results collected
so far out of the 162, just over half of which are from other services. The final rate will not be
known until the whole sample has been audited.
Corrective audits
Page
12 of
48
Corrective audits are selected using a variety of criteria, trying to maximise their potential
corrective effect, including preventing errors in future.
The first selections of beneficiaries in the corrective strand focused on those which were
known to participate in many projects but for which only a few cost statements had been
received. The idea was to prevent any errors that might be discovered in these early cost
statements from appearing in subsequent ones.
We should see the fruits of this preventive approach later on in the FP7 audit campaign,
especially if any of those beneficiaries audited early on becomes part of future representative
samples.
In any case, preventive efforts have a diminishing impact as the audit campaign progresses
and the number of projects closed increases so, from mid-2012, we have switched to risk-
based selections. An RTD risk-based audit approach document8 was prepared in June 2012 as
a complement to the FP7 AS. A comprehensive set of risk criteria is listed in it, which will
serve as the starting point for RTD risk-based selections for the remainder of the FP7
campaign.
Several batches of audits from two such selections are under way. In these batches, we are
focusing on instances of proportionally high subcontracting costs, beneficiaries using the 'real'
cost model, third party participants, requested contributions just under the EUR 375 000
certification threshold and newcomers to framework programmes.
2.2.3.
Additional auditing commitments
There are additional auditing commitments in the following areas:
Fusion: the current arrangement with RTD K is to audit all Fusion associations on a
cyclical basis. Each association is audited once every four years. According to the
Annual Audit Plan, seven in-house audits were planned for 2012. Due to planning
constraints and the priority given to the closures of the Common Representative
Sample audits, three of these audits (Hungary, Greece and Latvia) were postponed
until early 2013. Three other audits (Czech Republic, Portugal and Finland/Estonia)
were executed as planned9.
Fusion expenditure was subject to an audit by DG RTD's Internal Audit Capability.
This internal audit was finalised in December 2012 and resulted in one important
recommendation issued to RTD M regarding the degree of detail in reporting.
Relevant actions were taken, with further measures to be completed by the end of
March 2013.
Coal and Steel (C&S): a number of audits are launched every year on beneficiaries
who receive funds from the Research Fund for Coal and Steel (RFCS), which is
managed by RTD G. RFCS projects do not receive funding from the Framework
Programmes and are therefore not FP-related. For 2012 (in line with the planning
established in 2010), a selection of contractors was made according to the following
8 Ares(2012)732355
9 The remaining audit (CEA France) could not be launched as the previous Fusion audit remains open.
Page
13 of
48
principles: a list of all RFCS contract payments done in 2011 was obtained from the
responsible unit (RTD G.5) by the RTD M.1 back-office. Only mid-term and final
payments (i.e. only the ones for which financial reports had been submitted) were
considered. The sum of these payments was established for each contractor in the
database and the five biggest amounts, representing the biggest contractors, were
selected. The contractors already covered by previous Coal & Steel audits were
excluded from this selection. Each audit covers a maximum of three randomly selected
projects from each selected contractor. From this selection, two audits were executed
in 2012 and three were postponed to early 2013, due to the priority given to the CRS
audits. The objective for 2013 is to clean up this backlog while at the same time
auditing the 2013 selection.
Coal and Steel expenditure was subject to an audit by DG RTD's Internal Audit
Capability in 2012, but the audit report is still awaited.
Audits on Request (AoR): audits in this category are performed at the request of the
operational services, and they are normally quite specific in their scope. Following the
reception of such a request, RTD M.1 gathers detailed information from the
operational service, which is combined with information from other sources.
If needed, an ad-hoc meeting with the Audit Liaison Officer of the Directorate from
which the audit request originated is organised. The project officer as well as the
financial officer is invited to this meeting. The meeting allows getting a clearer view
of the issue and/or reaching a common position on the measures to be taken.
Audits on request are included in the category 'risk-based' under FP6 and 'corrective'
under FP7.
The full process is customer–oriented. For requests which are not accepted, RTD M.1
provides the operational service with advice/recommendations concerning alternative
measures they may wish to undertake in order for them to ensure that costs are
eligible.
In 2012, eleven requests were accepted. In one case, the need to carry out a financial
or scientific audit was not confirmed and in another case the request was withdrawn
by the operational service.
If the audit-on-request relates to a potential irregularity, the opinion of the RTD M.1's
'OLAF'-team is gathered before taking a final decision. Two such cases are currently
being analysed by the 'OLAF'-team; in two other cases operational services have been
requested to provide additional information or to carry out supplementary steps to first
clarify the matter. These cases will be further followed up in 2013.
The units that request the audits are systematically informed about the decision taken.
Page
14 of
48
Joint Audits with the European Court of Auditors (ECA):
Table 2.3: Joint audits with the ECA in 2011 and 2012
Audits by ECA
Joint Audits
%
Disagreement on conclusions
2011
32
21
66%
4
2012
41
24
59%
2
In order to increase harmonisation of audit techniques and results, and to improve
consistency of audit findings and conclusions, RTD auditors continued their efforts to
join ECA auditors on as many missions as possible during 2012. The total number of
joint audits during 2012 was 24, above the estimated figure of 20 outlined in the
Annual Audit Plan for 201210. The experience gathered prior to 2012 showed that
carrying out joint audits helps to (a) enhance convergence of views and results
regarding the audit findings and (b) better prepare DG RTD's comments in case of
disagreement on the conclusions.
The procedures for joint audits prepared during 2011 have been updated during 2012
and are being followed up by all RTD auditors involved in these joint audits.
Technical Audits: The Commission's Internal Audit Service recommended
implementing, where applicable, on-site technological and scientific audits as foreseen
by Art. II.23, Annex II of the FP7 Grant Agreement and Art. II.29, Annex II of the
FP6 Contract. The objective of these technological and scientific audits is to look at
research projects from a scientifically independent point of view, and as a complement
to the usual project reviews that take place during the lifetime of a project.
During the period 2010-2011, RTD M.1 closed two pilot technical audits, had two
requests for joint financial and scientific audits, and was asked for support on three
scientific audits. In 2012, by comparison, RTD M.1 did not receive any new request
for scientific audits, while only one technical audit was initiated and is still ongoing.
2.3. Cross-RDG coordination
In 2012, coordination of the common corporate audit strategies of the Research services
continued. DG RTD - as leading DG - is the chair of various committees for which it also
provides the secretariat. Under the present governance, DG RTD devotes significant resources
to coordination tasks, with the main objectives of ensuring (a) the co-ordination of audit
policy, planning and operational matters related to the implementation of the audit strategies
and (b) a coherent and consistent interpretation of audit results.
In 2012, these efforts focused especially on the first Common Representative Sample for
which the results had to be available by the end of 2012.
10 Ares(2012)470849 of 17/04/2012
Page
15 of
48
2.3.1.
Coordination of audits in the Research family (CAR)
The main coordination forum is the 'Coordination group for external audit in the Research
family' (CAR). It covers the audit activities of the research Commission services and as from
2012, of the Joint Technology Initiatives (JUs) linked to DG RTD and DG ENER/MOVE.
Chaired by DG RTD M.1, the CAR convened 15 times in 2012.
The large majority of topics discussed by the CAR in 2012 related to the implementation of
the FP7 Audit Strategy, and of the Common Representative Sample in particular. The CAR
group also dealt with wrapping up the FP6 audit campaign and with upcoming audit activities
for Horizon 2020.
The following paragraphs list – by way of example – the topics discussed in the CAR:
Planning of audits. On top of the overall planning coordination, the CAR
continuously monitored the implementation of the CRS. In order to use resources
efficiently and keep the audit burden for beneficiaries low, the Research audit
services carried out any audits to common beneficiaries as joint audits.
RTD and CNECT visited the ECA in Luxembourg in early 2012 to discuss the
possibility of ECA using the results for the assurance they provide of audits carried
out by Commission services. Although the visit went well, the initiative has not
progressed any further.
The JUs participating in the CAR are not part of the CRS, as their legal framework
differs from the one of the Commission Services. The JUs therefore continued to
follow their own targets for representative audits, which resulted in a number of
planning clashes with other services. The CAR tried to prevent the clashes in the best
possible way by exchanging information on the past, present and planned audits on
the beneficiaries in question.
The CAR updated the Common
FP7 Audit Strategy. The Strategy now reflects the
introduction of the CRS and, as a result of this, the total number of planned FP7
audits under the Strategy for the period 2009-2016 was reduced by 25%. On 12 July
2012, the Secretariat-General circulated the updated Strategy to the ABM.
The CAR was consulted on the preparation of the
public tender for the new
framework contract for audit services. The existing framework contract runs until 9
June 2013. An important milestone in this process was the adoption by the CAR of a
common audit report template to be used by the external auditors. This new reporting
template has been in use for in-house audits since September 2012. The JUs will also
participate in this new common framework contract for the first time.
The CAR also dealt with numerous issues related to the
eligibility of costs:
complementary contracts and bonuses, self-employed persons operating as
subcontractors but claimed as personnel, EU-supported national subsidy schemes
(Torres Quevedo in Spain, 'WBSO' in the Netherlands), conflicts of interest, research
outsourced by SMEs to 'RTD-performers', depreciation, provisions for remuneration,
changes to cost models (actual indirect costs versus flat-rate for indirect costs). These
discussions are based on the analysis performed by the Key Words Group (see section
2.6.2.), and then submitted to the CAR for further debate and endorsement.
Furthermore, the CAR issued a note on the fair allocation of indirect costs. A list of
national subsidy schemes (mostly based on tax incentives), and of their compliance
Page
16 of
48
with FP7 grants, was published on CORDIS as a result of the eligibility issues
discovered by audits in a number of Member States.
The FP Audit Manual, part of the
audit procedures and tools, was updated after the
publication of the revised FP7 Guide to Financial Issues on 16 January 2012. The
CAR added as an optional document to the Audit Process Handbook (APH) a
template letter, to be sent to certifying auditors whenever there is a deviation of more
than 2% between their audit and an audit carried out by the Research services. RTD
decided to send this letter in all applicable instances.
As recommended by the Internal Audit Service11, the Commission Research audit
services performed
peer reviews of audit procedures and files. The results were
satisfactory (see section 2.6.1.). DG RTD introduced a procedure for the examination
of audit reports with significant audit adjustments by the Audits Internal Supervision
Committee (AISC). The CAR proposed to extend this procedure also to the CRS
audits of the other participating Research DGs.
The CAR contributed to some
of the Horizon 2020 proposals, and in particular to
the acceptance of bonuses as eligible personnel costs, and discussions around flat
rates for indirect costs. The group also considered the impact of the revision of the
Financial Regulation of 1 January 2013 on auditing activities.
In line with a recommendation of the European Court of Auditors in its 2010 annual
report, DG RTD initiated preparations to review
the work of the external audit
firms in early 2013. The CAR was involved in the approval of the audit programme
that will guide reviewers during this process.
2.3.2.
Other coordination Committees
DG RTD chairs a number of other coordination Committees:
Extrapolation Steering Committee (ESC, see section 2.4.1),
Frauds and Irregularities Committee (FAIR, see section 2.5.),
Joint Assessment Committee (JAC, see section 2.9.)
Working Group on Certification of Methodology (WGCM, see section 2.9.);
Coordination of relations with the external audit firms, including the Monthly Audit
Status Meeting (MASR, see section 2.10.).
Coordination is also supported by a number of IT tools known as SAR (Sharing Audit
Results) tools (see section 2.8.).
11
Final Audit Report on DG RTD’s Control Strategy for on-the-spot controls and fraud prevention and
detection, recommendation 4.4, Ares(2011)1039870 of 30/09/2011.
Page
17 of
48
2.4. Extrapolation
Extrapolation remains a key component of the common audit strategy because of its essential
role in cleaning the budget from systematic material errors which, in turn, results in lowering
the residual error rate.
2.4.1.
Extrapolation policy and coordination
The Extrapolation Steering Committee (ESC) has now been working for five years. It ensures
a common approach to decisions related to extrapolation. The ESC, in which all research
Commission services12 are represented, is chaired by RTD M.1 and discusses and evaluates
potential extrapolation cases put forward by individual Commission services. Its main aim
and mandate is to confirm or deny the systematic nature of the errors reported by the auditors.
The confirmation of the systematic nature of an error triggers a number of coordinated actions
both for the beneficiary in question, and for the Commission services managing the projects in
which it participates.
In its 11 meetings in 2012, the ESC discussed a total of
118 extrapolation cases,
101 of which
were approved for extrapolation. The following table provides an overview of the ESC
decisions per Commission service. Since 2008, a total
of
869 cases have been dealt with,
662 of which have been agreed for extrapolation.
Table 2.4 - ESC decisions taken in 2012
ESC
DG
DG
DG
DG
DG
Total
RE
Cumulative
Decision
ENER
ENTR
ERCEA
CNECT
MOVE
RTD
2012
A
Agreed
13
4
4
25
9
3
43
101
662
On-Hold or
No
Extrapolation 0
1
0
4
3
3
6
17
207
Total 2012
13
5
4
29
12
6
49
118
869
Experience gathered by the ESC allows for continuous improvements to the underlying
extrapolation procedures and principles. Any new issues are often brought to the attention of
the CAR.
The introduction of the new Financial Regulation, now referring to the extension of the audit
findings, will require adaptation of some procedures and terminology to the new legal
framework.
2.4.2.
RTD extrapolation cases
Over the last five years, DG RTD has put 377 extrapolation cases on file, of which nine are
still under preparation, 24 are on hold and are 'centrally managed' by RTD M.1, 76 have been
12 As of September 2009, representatives from the agencies (ERCEA and REA) have been participating in the
ESC. They play a full role in the extrapolation process under FP7. JUs do not participate, since no cross-
extrapolation takes place to JU contracts, or vice versa.
Page
18 of
48
implemented throughout the Research Family, and 158 are currently on-going in DG RTD or
other Research DG.
In 110 cases extrapolation did not apply for various reasons: there were no other projects to
extrapolate to, the systematic errors were not relevant in the remaining projects, the
beneficiary had already implemented the extrapolation, the errors were not considered
systematic in nature, the errors were below the materiality threshold, etc.
Table 2.5 - Current status of the DG RTD-led extrapolation cases (as of 31/12/2012)
Current Status
Grand Total
CLOSED
76
ONGOING
158
ON HOLD
(centrally managed)
24
NOT APPLICABLE
(no extrapolation)
110
SUBMITTED
(audit not yet closed)
9
Grand Total
377
2.4.3.
Extrapolation implementation
Each extrapolation case can potentially affect numerous projects across the research
Commission services. The experience acquired so far within DG RTD has underlined
substantial challenges in this area, especially with regard to following up the reception of
revised cost statements and coordinating their implementation. To address this issue, RTD
M.5 (RTD M.4 as of 1 January 2013) 'Management of debts and guarantee funds' acts as a
central reception point dealing with all extrapolation cases launched from 13 March 2009
onwards.
In a number of extrapolation cases, beneficiaries wish to establish a dialogue and to provide
additional documentation and evidence. Currently,
24 such cases are 'centrally managed' by
the audit unit: the extrapolation process is therefore put 'on hold' and all operational services
in the research Commission services are requested not to act individually to avoid incoherent
actions.
Table 2.6 – Centrally Managed Cases by DG
DG
DG
DG
DG
DG
Total
ENER
ENTR
ERCEA
CNECT
MOVE
REA
RTD
0
0
0
3
1
0
24
28
For all DG RTD-led extrapolation cases, (i.e. triggered by a DG RTD audit), so far 9823
participations have been identified as potentially affected by extrapolation. Among these,
1728 have been implemented (i.e. amount adjusted), 5155 are currently under implementation
and for 2940 recommendations the extrapolation turned out not to be due.
In addition,
169 cases resulting from audits of the other research Commission services have
an impact on 5272 RTD participations, of which 1465 have been implemented, 2253 are
Page
19 of
48
currently under implementation and for 1554 recommendations the extrapolation turned out
not to be due.
Table 2.7 – DG RTD participations affected by extrapolation (cumulative, FP6 & FP7)
Non-RTD-led Cases
Total
RTD-
non-
Implementation
DG
Grand
led
DG
DG
RTD-
Status
MOVE-
Total
Cases
ERCEA ENTR CNECT
REA
led
ENER
Cases
Not applicable
2940
142
122
1050
181
59
1554
4494
Closed
1728
2
184
1063
181
35
1465
3193
Ongoing
5155
567
189
957
407
133
2253
7408
Total
9823
711
495
3070
769
227
5272
15095
Moreover, for the
377 DG RTD-led cases,
2201 participations managed by other research
Commission services are also being revised as part of the extrapolation process.
Regarding the adjusted amounts following extrapolation the table below provides the
cumulative proposed adjustments as a result of systematic errors.
Table 2.8 – Cumulative overall adjusted amounts due to extrapolation
2009
2010
2011
2012
(-)
Adjustments in
€ -2 707 061
€ -10 409 202
€ -16 433 637
€ -32 477 500
favour of the
Commission
(+)
Adjustments in
€ 140 983
€ 563 244
€ 1 207 039
€ 1 479 002
favour of the
beneficiaries
This table relates to the implementation of extrapolations managed by RTD M.5 (RTD M.4 as
of 1 January 2013). Therefore only overall information is provided here.
Extrapolation remains a cornerstone of the Audit Strategies. Having said that, the figures in
the tables above clearly show that extrapolation is a complex and resource-intensive process.
Page
20 of
48
2.5. OLAF cases
Table 2.9 - Situation of the OLAF cases on 31/12/2012
Number of new cases DG RTD transmitted to OLAF
8
in 2012
Number of new cases relevant to DG RTD that OLAF
8
initiated directly in 2012
Cases relevant to DG RTD that OLAF dismissed in
17
2012
OLAF investigations relevant to DG RTD closed with
4
administrative/financial/judicial follow-up in 2012
Total ongoing OLAF investigations (initial assessment
22 (including 6 cases
/external investigation opened, including cases of
initiated by DGs CNECT, ENER and REA on
previous years)
common beneficiaries)
Number
of
closed
cases
in
administrative/
financial/judicial follow-up managed by RTD M.1
19
(including cases from previous years)
RTD M.1, in charge of relations with OLAF on external investigations13, transmitted eight
new cases of suspected irregularities to OLAF in 2012. In three cases, the suspicion of
irregularities was reported by the operational services in charge of the projects; in four cases,
the decision to transfer the case to OLAF was taken following on-the-spot audits performed
by RTD M.1 auditors or by an external audit firm. In one case, an external informant raised
allegations of potential irregularities. In addition, according to our knowledge, OLAF
received eight complaints directly from (sometimes anonymous) informants concerning
potential irregularities in EU-funded research projects managed by DG RTD.
In 2012, OLAF dismissed 17 cases14 relevant to DG RTD; four further cases, for which the
allegations of irregularities were confirmed in the OLAF investigation, were closed and are
currently being followed up at administrative, financial and/or judicial level.
Taking into account also the OLAF cases relevant to DG RTD from previous years, as of 31
December 2012, RTD M.1 manages 22 open cases in total (six of them are cases initiated by
DGs CNECT, ENER or REA on common beneficiaries), as well as 19 cases which OLAF had
closed with administrative, financial and/or judicial follow-up, requiring follow-up and/or
monitoring at DG RTD level.
RTD M.1 has been actively involved in the revision and implementation of the DG Research
& Innovation Anti-Fraud Strategy and Action Plan, in particular as regards the fraud detection
part.
In this respect, the unit has intensified its fraud detection activities, notably through extensive
use of CHARON/DAISY (see 2.8.), an advanced IT tool which facilitates investigating,
analysing and displaying complex information and relationships between Framework
Programme data, to select and prepare fraud risk-based audits. Several targeted data searches
and inquiries were performed with CHARON/DAISY on the basis of different risk criteria
13 Unit RTD.01 is in charge of internal investigations relating to staff.
14 A matter is classified by OLAF as 'case dismissed' where there is no need identified by OLAF to open an
investigation or coordination case or when an investigation did not confirm the suspicion of irregularity or fraud.
Page
21 of
48
(e.g. dependency on EU funding, collusion). In 2012, the unit initiated eight fraud risk-based
audits.
RTD M.1 made further progress on the development and operation of tools and procedures to
detect plagiarism and double funding in research projects, in close cooperation with the other
Research DGs. To this end, RTD M.1 operates an IT tool which allows to carry out plagiarism
checks against a database of FP6 and FP7 deliverables. RTD M.1 has also signed a contract
with a service provider to carry out plagiarism checks against external data repositories.
RTD M.1 auditors participated in the monthly training sessions organised by RTD M.4 (RTD
M.3 as of 1 January 2013) to raise fraud awareness among DG RTD staff, in particular project
and financial officers, and to identify and mitigate fraud risks in RTD projects.
Two FAIR (Fraud and Irregularities in Research) Committee meetings were held in 2012 with
representatives from the other Research DGs and Executive Agencies to coordinate activities,
exchange information, and share experiences and best practices on OLAF-related matters.
Fraud prevention and detection, and ongoing and potential irregularities and fraud cases were
also covered.
RTD M.1 is closely cooperating with OLAF. Regular contacts and meetings at operational
level took place between DG RTD staff and OLAF investigators to discuss and exchange
information on specific cases. Furthermore, RTD M.1 attended three meetings of the Fraud
Prevention and Detection Network (FPDNet). Representatives of the various Commission
services and Executive Agencies participate in this forum chaired by OLAF.
With regard to our objective to reinforce cooperation at European level, RTD M.1 participated
in a Workshop organised by RTD M.4 in Brussels from 31 May to 1 June 2012, which
brought together representatives from different Member States' Research Funding bodies to
discuss and exchange information, experiences and best practice on risk management,
controls and anti-fraud matters. In addition, several visits to national research funding
organisations took place to explore the possibilities of closer collaboration with these bodies.
2.6. Management and quality control tools
2.6.1.
Management and quality control
A time recording system was put in place from September 2012 onwards in order to monitor
time spent in the different phases of the audit process as well as time spent on horizontal
tasks.
It has allowed to establish that 32% of auditors' time is dedicated to 'horizontal affairs',
covering various domains such as audit policies and strategies, extrapolation, coordination
with OLAF on irregularities and fraud cases etc., largely explained by the fact that DG RTD
is the lead DG amongst Research services on audit policy.
2.6.1.1. Reinforcement of the quality and the supervision of in-house
audit reports
The process of monitoring audit reports has been redesigned during 2012. The quality control
steps at the draft stage of the audit have been strengthened, with a special focus on giving
Page
22 of
48
additional assurance on substance. To that end, a standard control matrix has been introduced
for the quality controllers to follow.
2.6.1.2. Organisation of peer reviews
In line with an IAS recommendation15, a number of top, risk and representative DG CNECT
audits were selected and checked against the key control points of the RTD quality control.
The same process was followed by DG CNECT when reviewing RTD files. The outcome of
the peer reviews was satisfactory for both DGs, as no significant observations were raised.
2.6.1.3. Mentoring
A mentoring system has been introduced in which newcomers are assigned to a mentor with
the intention to:
answer the many legitimate questions raised by newcomers,
facilitate their integration into the unit,
enable them to promptly acquire the knowledge necessary to carry out their functions
within the unit, and
guide their first audits insofar as the preparation, fieldwork, reporting and follow up
are concerned.
2.6.2. Keywords Working Group (KWG)
The Keywords Working Group (KWG) is a consultative group comprising 7 members, all of
them DG RTD auditors.
The aim of the KWG is to analyse audit and accounting issues in a harmonised manner, in
compliance with the relevant applicable rules and regulations. The group promotes the
principle that all Research services should 'speak with one voice' to stakeholders. Although
the KWG is internal to DG RTD, when needed agreement from other Research audit services
is sought, mostly via the CAR.
The main task of the KWG is to provide assistance following enquiries received both from
internal and external stakeholders. More than 60 individual requests were processed in 2012.
This assistance results in the preparation of replies or guidance notes on contentious topics.
Any action taken by the KWG is integrated in the keywords database, which is a compilation
of all positions expressed by the group on various topics.
The most frequently discussed topics were:
Personnel costs: eligible costs included in the calculation, productive hours'
determination, personnel costs in case of an SME owner/manager, calculation of the
hourly rate, cumulative contracts and bonus payments…,
Eligibility of specific taxes: WBSO (tax benefit scheme for research (NL)), tax
'versement transport' (FR), non-domestic property Tax (UK)…,
15
Final Audit Report on DG RTD’s Control Strategy for on-the-spot controls and fraud prevention and
detection, recommendation 4.4, Ares(2011)1039870 of 30/09/2011
Page
23 of
48
Indirect cost methodologies: specific cases (e.g. teleworking), classification of costs as
direct or indirect, eligibility (e.g. freight costs
, bank charges)…,
Cost eligibility and classification criteria: internal production of consumables,
external facilities costs, subcontracting outside EU, subcontracting of minor tasks,
depreciation rules (e.g. prototypes,…), teacher replacement costs…,
Audit topics: auditable periods, format of auditable documents/receipts…
Questions concern a variety of topics and may go into a great level of detail. The positions
taken are available to all the auditors.
2.6.3.
The Audit Steering Committee (ASC)
The ASC provides peer reviews by fellow auditors and is organised at the request of the
individual auditor according to the relevant procedure. It assists in the assessment on
substance of proposals for large audit adjustments. Adjustments are considered to be large
when they are above EUR 100 000 and represent 5% or more of the costs claimed, or when
they are above EUR 30 000 and represent 30% or more of the costs claimed.
The ASC considers both in-house and outsourced DG RTD audits, and it contributes to
ensuring equal treatment of beneficiaries and the coherence of audit work.
The number of ASC meetings and of cases submitted increased in 2012.
Table 2.10 - ASC cases
Year
Meetings
Cases
2009
14
20
2010
18
32
2011
15
22
2012
19
27
The existence of the ASC was put into question with the establishment of the AISC (see
section 2.6.4.). Yet the majority of auditors felt the need for the continuation of the ASC,
since it is comforting for the auditor to have additional assurance of his peers.
2.6.4.
Establishment of an Audits' Internal Supervision Committee (AISC)
Quality management was further enhanced by the establishment of the AISC.
The AISC Committee was set up with the aim of reinforcing coherence among RDGs, in
particular now that all RDGs have introduced the CRS. Its specific mandate is to review the
proposals made by the responsible units just before the release of the Letter of Conclusion
(LoC) and the Final Audit Report, and ensure that they are sound, in line with established
administrative practices, and consistent with similar cases.
During 2012, the AISC Committee held 15 meetings and 41 files underwent its review (30
RTD, six CNECT, four REA, one ERCEA).
Page
24 of
48
2.7. Collaboration with the DG RTD administration and finance (UAF) network
A continuous inter-service collaboration has been established to provide guidance and support
to the operational units and, in particular, to the financial officers who handle the FP7
Certificates on the Financial Statements (CFS). By doing so, a coherent, harmonised and
consistent approach on CFS-related matters is ensured across the research Commission
services.
In 2012, the external audit units have continued to uphold their close working relationships
with the administration and finance units during the planning and preparation of new audit
campaigns, during the audits themselves (in order to obtain feedback on the draft audit
conclusions), and after the audits' closure (for the implementation of the final audit
conclusions and results). With regard to the internal consultation, whenever an operational
unit fundamentally disagrees with a draft report, the hierarchy is consulted before the audit
can be processed further. Although from a Commission reputational point of view it is
understandable that this instruction has been given, this has led to additional internal delays
for a number of audit reports.
Several ad-hoc bilateral meetings have been held whenever discussions on specific files were
needed. The external audit units also participate in meetings between the UAFs and
contractors for cases where the contractor continues to contest the audit findings after audit
closure. They also participate in the monthly UAF meetings to present and clarify matters
linked to auditing and financial issues.
2.8. IT developments
During 2012, the external audit units focused on the following IT developments:
AUDEX (Audit Management System) is a web-based application that supports the
management of the external audits carried out by DG RTD, as well as the management
of the extrapolation process of the audit findings. During 2012, it has been greatly
improved, positioning it as one of the best candidates for external audit management
in the EC IT rationalisation process:
New modules have been included to manage the extrapolation of audit results, to
review the work of external audit firms and to provide audit and error rate
dashboards.
New functionalities have been implemented to improve the management of the
joint audits with the Court of Auditors, the fusion audits, the recording of the audit
results. Furthermore, the e-mailing system of AUDEX has been adapted, and the
AUDEX services have been aligned to the new ASUR16 requirements.
The integration of AUDEX with other corporate IT systems (ASUR,
CPM/PCM/Force, SAR) has been tightened.
16 Audit and Supervision Management Application. ASUR is primarily an application to manage
recommendations formulated by the Internal Audit Service, Internal Audit Capability, Court of Auditors,
external audit unit, etc..., and to support ex-post control activities (external audits, audit extrapolation cases,
supervision campaigns, etc...).
Page
25 of
48
The workflow reporting based on indicators and milestones has been redesigned,
resulting in a new module that will replace the local Scoreboard module when it
goes in production during the first quarter of 2013.
The architecture of the application has been redesigned to support operations at cost
statement level instead of at participation level, ensuring a better integration with
the financial workflows of DG RTD.
The security layer has been redesigned in order to allow deployment of AUDEX in
the Executive Agencies. The first tests have been carried out, but the data migration
has been postponed until the first quarter of 2013 when new IT resources will be
made available. REA and ERCEA are expected to become full AUDEX users in the
course of the first quarter of 2013.
SAR (Sharing of Audit Results) is the information system that supports the sharing of
audit-related information within the research family. It comprises SAR-Wiki (report
sharing), SAR-EAR (extrapolation) and SAR-PAA (audit planning & clash
management). The main activities in 2012 concentrated on giving access to the JUs to
SAR-PAA and SAR-Wiki in order to harmonise their audit work with the rest of the
research family, migrating SAR-WiKi to the new Confluence platform, aligning the
SAR modules with the NOAP/Exchange platform requirements, and performing data
analysis and consolidation for the future convergence of the three systems.
In the course of 2012, SAR-Wiki reached its technical and functional limits. Therefore
DIGIT has been asked to make a proposal for a replacement system, preferably a
proper document management system.
CoMET, which provides a central web-based IT tool dedicated to supporting the FP7
methodology certification, was in maintenance mode during 2012.
2.9. FP7 methodology certification
The Certification policy for the FP7 Grant Agreements was designed with the aim to correct
the most common errors identified in the past, and in particular those related to personnel
costs and indirect costs. In this context, and in addition to the Certificates on the Financial
Statements (CFS) known under FP6 as 'audit certificates', two new types of ex-ante
certificates on the methodology were introduced in FP7 which may be submitted prior to the
costs being claimed: the
Certificate on Average Personnel Costs (CoMAv) and the
Certificate
on the Methodology for Personnel and Indirect costs (COM).
The acceptability of the methodology certificates submitted by beneficiaries is decided by an
inter-service Joint Assessment Committee (JAC), which is made up of staff from the external
audit units of DG RTD and DG CNECT. In 2012, five JAC meetings were held.
Page
26 of
48
2.9.1.
State of play of certification files as of 31 December 2012
Table 2.11 - State of play of all FP7-certification files 31 December 2012
Eligibility Requests
Requests for
Type of Certificate
Accepted
Rejected
Withdrawn
Ongoing
Submitted
Accepted
certificate(*)
COM Average Personnel
23
6
10
5
2
Costs and Indirect Costs
124
74
COM Real Personnel Cost
27
18
2
4
3
and Indirect Costs
Certificate
for
Average
N/A
88
49
10
28
1
Personnel Costs (CoMAv)
TOTAL
138
73
22
37
6
(*) Not all accepted eligibility requests resulted in a request for a certificate.
The relevance of the CoMAv diminished considerably in the light of the Commission
Decision C(2011)174 of 24 January 2011, which introduced three measures for simplifying
the implementation of FP7. The first of these measures was the definition of new criteria for
average personnel costs, whereby the usual accounting practices of beneficiaries would
become acceptable under certain general and less restrictive conditions.
Thereafter, beneficiaries were no longer required to submit a Certificate on Average
Personnel Costs (CoMAv) for approval as a prior condition for the eligibility of such costs.
Nevertheless, the CoMAv remains as an option, offering beneficiaries the possibility to obtain
prior assurance on the compatibility of the methodology in place with the FP7 eligibility
requirements. During 2012, the decline in the number of applications for Certificates
continued.
Prior to the same decision on simplification measures, the value of the work of SME owners
and natural persons who do not receive a salary could only be reimbursed if they requested an
ex-ante certificate of an average cost methodology that had to be approved by the
Commission. A very low number of certificates were issued, which lead to the situation where
SME owners or other natural persons, who did not obtain a certificate, could not be
reimbursed because the value of their work was not registered as a cost item in their accounts.
The same Commission Decision C(2011)174 of 24 January 2011 established rules allowing
for SME owners and natural persons who do not receive a salary to charge flat rates in
accordance with the Peoples Programme (Marie Curie). The submission of a Certificate on
Average personnel costs is no longer possible for SME owners, but remains available to other
participants. Compared with 2011, there was only one request for a CoMAv in 2012, which
was accepted.
The pattern of CoM submission remained steady throughout 2011. However, increased
activity was noted from beneficiaries who previously submitted applications for CoM but had
not pursued their requests due to the existing stringent requirements (essentially the
requirements related to average personnel costs). As already mentioned, Commission
Decision C(2011)174 made the application process easier for those beneficiaries who use
average personnel costs. As such, these beneficiaries became more active and were also
seeking to obtain a CoM. This may be due to the fact that at this stage of the Framework
Programme, beneficiaries are reaching the threshold of EUR 375 000 of EU funding, where
they are expected to submit CFS. In order to benefit from the waiver of submitting a CFS,
they became interested in obtaining a CoM.
In summary, it can be stated that following the adoption of Commission Decision
C(2011)174, the CoMAv lost its initial value, since it became optional for entities using
Page
27 of
48
average personnel costs and is no longer accessible to SME owners and natural persons who
do not receive a salary. However, the CoM - a certificate which offers the benefit of not
having to submit intermediate CFS - has become easier to obtain and remains attractive for
eligible beneficiaries, in particular those who use average costing methodologies.
2.9.2.
Inter-service collaboration and communication activities (cf. 2.11.)
Ex-ante certification also requires intensive communication efforts:
Handling questions submitted through the Research Enquiry Service on
Europe
Direct. Approximately 120 questions concerning the certification on the methodology
were answered in 2012.
Internal awareness-raising on FP7 certification issues, leading to meetings with
operational and UAF units.
Publishing certification-related documents on
CORDIS 'Guidance notes for
Beneficiaries and Auditors' were revised following the publication of the revised
Guide to Financial Issues.
Internal training sessions dedicated to FP7 certification on the methodology are given
quarterly.
Regular meetings with National Contact Points (NCPs) for legal and financial issues.
2.10. Coordination of outsourced audits
Six framework contracts for the provision of audit services are available to procure audit
services on FP6 and FP7 grants during the period 2009-2013, with a potential market value
amounting to EUR 16.5 million and EUR 42 million respectively. They are managed by RTD
M.2 on behalf of all research Commission services. These framework contracts are used under
a 'cascade' principle, i.e. when the first contractor on the list cannot execute an audit (as a
result of a conflict of interest or capacity issues), the second or possibly the third company on
the list are taken.
The framework contract for FP6 has not been used by DG RTD since 2011 due to the
phasing-out of FP6 audits. Any new FP6 audits are done internally. This framework contract
expired on 20 February 2013.
The framework contract for FP7 expires on 9 June 2013. In order to ensure the availability of
a framework contract after that date, a new Call for Tender was launched in 2012. This new
framework contract will be used not only by the research Commission services, but also by
the Joint Undertakings. The deadline for submitting tenders was 23 November 2012. 10
tenders were received. The evaluation committee expects to issue its report by the end of
March 2013.
Throughout 2012, the batch audit campaigns outsourced to the service providers (KPMG,
Ernst & Young, and Lubbock Fine) were closely monitored by RTD M.2 in terms of
timeliness and quality. There continues to be a strong dependence on the external audit firms,
as around two thirds of the DG RTD audit target is achieved through outsourced audits.
As part of the DAS 2011, the Court of Auditors decided to review the working papers of the
outsourced auditors, as well as the Commission services' monitoring of them. Although minor
weaknesses were detected, the overall conclusion is that the monitoring of outsourced auditors
work is effective. In addition, the Court recommended that the Commission services initiate
Page
28 of
48
their own review of the outsourced auditors' working papers. DG RTD, together with DG
CNECT, have undertaken the first missions to this end in January 2013. The review was
limited to FP7 files.
In addition to the daily follow-up of individual audits, this monitoring involves the following
business processes:
Monthly Audit Status Reporting (MASR) meetings chaired by RTD M.2, covering the
progress of all ongoing batches, technical issues, invoicing and future audit planning.
Occasionally accompanying external audit firms on on-the-spot missions.
Providing guidance and clarification on specific problems.
Using the Audit Review Assessment (ARA) to follow-up the quality of the services
provided.
A batch audit processing manual including checklists for the different deliverables.
Normal contract management issues, such as setting up contracts, amendments,
payments, penalties, etc.
2.11. Communication Campaign
Towards the end of 2011, the audit units prepared a list of the 10 most common financial
errors in cost claims that are detected during the audits. This resulted in the document “
How
to avoid common errors identified in cost claims”, which was sent to all registered FP7
beneficiaries and was used to launch a Communication Campaign aimed at improving the
reliability of the ex-ante certificates and thereby – hopefully – reducing the number of errors
in cost claims. This campaign is aimed at both beneficiaries and certifying auditors (both
private audit firms and certified public officials), and is hosted by the NCPs in the various
Member States or associated Countries.
In 2012, training events were held in:
Sweden
Germany
Spain
Ireland
Finland
Czech Republic / Slovakia
Austria
Italy
Denmark
Poland
Switzerland
Additional training sessions are foreseen for Cyprus, France, Germany, the Netherlands,
Portugal and the UK during the first semester of 2013.
Based on feedback from the organisers of these events, the overall reaction to these training
sessions has been very positive. However, it is important to stress that the impact of the
training sessions on the correctness of submitted cost claims, cannot be quantified.
2.12. Other activities
2.12.1. Art. 185 Initiatives
Art. 185 of the EU Treaty foresees the participation of the EU in the joint implementation of
(parts of) research and development national programmes. Implementing Art. 185 Initiatives
implies that the participating Member States integrate their research efforts into a joint
research programme. The EU provides financial support to the joint implementation of the
Page
29 of
48
(parts of the) national research programmes involved, based on a joint programme and on
setting up a so-called dedicated implementation structure. The role of RTD M.1 involves
carrying out the ex-ante assessments required by the Financial Regulation (Art. 56) before
implementation.
Agreements are concluded between the Commission and the dedicated implementation
structure. At present, there are three which are followed up by this unit:
1. EUROSTARS
The ex-ante assessment of EUREKA, the dedicated implementation structure of the Eurostars
Initiative (Decision n° 743/2008/EC) was carried out in 2008. A follow-up audit of the
EUREKA Secretariat was carried out in 2010, resulting in a set of recommendations on their
internal control systems and a request for re-submission of a revised statement of expenditure.
RTD M.1 analysed EUREKA's report on the implementation of these recommendations and
on the revised statement of expenditure during the first quarter of 2012. While the revised
statement of expenditure was considered adequate, the follow-up of the implementation of
certain recommendations remains ongoing.
2. EMRP
The ex-ante assessment of EURAMET e.V., the dedicated implementation structure of the
EMRP Initiative (Decision n° 912/2009/EC), was conducted by RTD M.1 in 2009. The
outcome was a list of recommendations to be implemented through a jointly-agreed action
plan. The implementation of this action plan by EURAMET e.V was reviewed in 2010.
It was originally planned to execute a proper follow-up audit in the course of 2012, but this
was postponed until 2013. The scope of the audit will be twofold:
The
Ex-post verification of the running expenditure;
A further assessment of the implementation of the recommendations according to the
action plan.
3. BONUS
In 2011, RTD M.1 conducted an ex-ante assessment of BONUS EEIG, the dedicated
implementation structure of the BONUS Initiative (Decision n° 862/2010/EC). This
assessment identified several critical weaknesses and resulted in a list of recommendations for
their internal control system.
The actions foreseen by BONUS EEIG to respond to these recommendations were set up in
an action plan and reviewed by RTD M.1 in January 2012. In order to support RTD I.3 in
monitoring the implementation of the action plan for those recommendations that had been
qualified as critical, and before signing the Implementation Agreement, RTD M.1 was
requested to perform an additional assessment. This follow-up assessment was performed
mid-2012. Its conclusions were reported to RTD I.3, resulting in a list of additional
recommendations to be addressed to BONUS EEIG.
Page
30 of
48
2.12.2. Access to documents
In 2012, the request for access to documents (Audit Manual, Audit Process Handbook, Letter
of Announcement…) by external parties increased sharply. Information on the requests for
access was coordinated within the CAR amongst the RDGs (see section 2.3.1.).
Page
31 of
48
3.
RESULTS AND ANALYSIS
The quantitative results of the activities of the external audit units are presented in this part,
together with analysis and commentary where appropriate. The most interesting points are
summarised below each table or graph.
3.1. Audit numbers
This section presents results related to the number of audits and of participations audited in
2012 and cumulatively, with breakdowns by a number of categories.
Table 3.1 - Audits closed and participations audited (2012 and cumulative, ALL
audits17)
2012
Cumulative
No.
No.
audits
No. audited
audits
No. audited
FP
Strategy strand
closed
participations
closed
participations
FP6
TOP
7
9
399
1044
MUS
1
2
154
341
RISK
32
114
618
1404
FUSION
0
1
30
45
Total FP6
40
126
1201
2834
FP7
CORRECTIVE
212
510
640
1329
REPRESENTATIVE
47
84
54
100
FUSION
4
6
8
14
OTHER
2
0
5
0
Total FP7
265
600
707
1443
Coal & Steel
N/A
3
9
17
49
Totals
308
735
1925
4326
The target of 330 audits closed set for 2012 was not quite achieved. The main reason for
this was that fewer FP6 audits were closed than anticipated (40 instead of 69). On the
other hand, the closing target for FP7 audits was surpassed. This being said, cumulatively
speaking, the FP7 Audit Strategy is on track.
The cumulative average of participations covered per audit has increased since last year to
2 for FP7 and to 2.4 for FP6. At the end of the FP5 campaign it was just 1.2. The increase
in the cost-effectiveness of audits in the last few years is evident from these figures, a
result of improvements in planning and audit preparation.
1201 FP6 audits have been closed in the period 2007-2012. There remain 42 FP6 ongoing
audits so, at the end of the FP6 audit campaign, the total number of audits will be more
than 60% higher than the original minimum multi-annual target of 750 set in the ABM
action plan drawn up in 2007. This increase has been due to additional risk-related audits
aimed at further reducing the residual error rate for FP6, follow-up audits on extrapolation
cases, joint audits with the ECA, and the results of the mid-term review which showed
17 Throughout section 3, 'ALL audits' means all FP6, FP7 and Coal and Steel (C&S) audits.
Page
32 of
48
that the share of the systematic error 'cleaned' through extrapolation was not as large as
originally assumed (see also section 2.4.).
Table 3.2 - Audits closed of specific types18 (2012, ALL audits)
Audit type
2012
The number of closed audits on request, joint
Audits on request
17
audits with ECA and audits devoted to fraud
FUSION
4
detection were similar to 2011 figures.
Coal & Steel
3
Joint audits with ECA
18
Audits in non-EU
28
For more details on these audits, please see
countries
the relevant sections in part 2.
Fraud detection (OLAF)
10
Desk reviews
3
Art. 185
2
Technical audits
0
Table 3.3 - Audits closed, outsourced and in-house (2012 and cumulative, ALL audits)
2012
Cumulative
No. audits closed
%
No. audits closed
%
Total Outsourced
210
68.2%
1337
69.5%
In-house
98
31.8%
588
30.5%
Totals
308
100.0%
1925
100.0%
The proportion of audits closed in-house was slightly higher in 2012 (31.8%) than the
historical average (30.5%). Cumulatively, in-house audits account for almost a third of all
audits closed.
18 An individual audit might fall into more than one of these categories (e.g. an audit on request in a non-EU
country).
Page
33 of
48
Table 3.4 - Audits closed by country (2012, ALL audits)
Over 80% of all the audits carried out
Country
No. audits
%
closed
in 2012 took place in the 10 listed
DE
Germany
44
14.3%
countries. The percentage two years
IT
Italy
41
13.3%
ago was 87%. We continue to see
more diversity as the implementation
FR
France
36
11.7%
of the FP7 AS progresses, which
UK
United Kingdom
33
10.7%
reflects
a
proportionally
higher
ES
Spain
30
9.7%
participation of new member states in
NL
The
17
5.5%
FP7.
Netherlands
FI
Finland
14
4.5%
BE
Belgium
12
3.9%
CH
Switzerland
10
3.2%
SE
Sweden
10
3.2%
Others (EU &
61
19.8%
non-EU)
Total
308
100.0%
3.2. Audit results
This section presents audit results in monetary terms. The most interesting points are
summarised below each table.
Please note that all figures related to adjustments in this part are estimates that may or
may not correspond with the eventual financial recovery or offset amount applied by
operational services19.
19 For FP6, the EC share of proposed adjustments is calculated on the basis of cost model and instrument type,
but there might be variations of the actual percentage of EC contribution for specific contracts. For FP7, this
information is available in central DG RTD information systems, so the calculations are more accurate.
Page
34 of
48
Table 3.5 - Audit results in monetary amounts (2012, ALL audits)
Results at Cost Level
Audited
Costs claimed
Costs
Costs
Adjustments in
Adjustments in
participations and audited
accepted by
accepted by
favour of the
favour of the
Financial
Auditor
Commission
beneficiary
Officers
FP6
126 164,826,600
163,046,761
150,549,665
-12,642,820
145,724
FP7
600 478,656,957
478,532,965
473,908,226
-11,893,071
7,280,867
C&S
9 2,488,663
2,508,229
2,506,485
-1,744
-
Totals
735 645,972,220
644,087,955
626,964,376
-24,537,635
7,426,591
Results at Funding Level (estimated EC share)
Audited
Costs claimed
Costs
Costs
Adjustments in
Adjustments in
participations and audited
accepted by
accepted by
favour of the
favour of the
Financial
Auditor
Commission
beneficiary
Officers
FP6
126 61,046,901
60,111,417
51,605,322
-8,579,090
72,996
FP7
600 238,093,805
238,001,866
235,537,109
-7,829,757
5,374,402
C&S
9 1,493,198
1,504,938
1,503,891
-1,046
-
Totals
735 300,633,904
299,618,221
288,646,322
-16,409,893
5,447,398
Even though in 2012 fewer audits were closed than in 2011, the amounts audited
were 57% higher. In 2012, a total of almost EUR 645 million in costs was audited,
compared to EUR 410 million in 2011. Of this amount, the EC contribution was over
EUR 300 million (EUR 223 million in 2011).
The total amount of adjustments in favour of the Commission at funding level proposed
by the auditors was more than EUR 16.4 million (EUR 12.4 million in 2011).
Table 3.6 - Audit results in monetary amounts (cumulative, ALL audits)
Results at Cost Level
Audited
Costs claimed
Costs accepted
Costs accepted
Adjustments in Adjustments in
participations and audited
by Financial
by Auditor
favour of the
favour of the
Officers
Commission
beneficiary
FP6
2834 2,456,340,988 2,448,373,016 2,381,374,995 -89,616,743
22,913,643
FP7
1443 810,767,777
810,811,867
801,802,169
-22,494,380
13,534,906
C&S
49 27,878,325
27,825,888
27,295,547
-556,129
25,788
Totals
4326 3,294,987,090 3,287,010,771 3,210,472,711 -112,667,252 36,474,337
Results at Funding Level (estimated EC share)
Audited
Costs claimed
Costs accepted
Costs accepted
Adjustments in Adjustments in
participations and audited
by Financial
by Auditor
favour of the
favour of the
Officers
Commission
beneficiary
FP6
2834 1,178,844,232 1,174,434,234 1,131,563,128 -55,536,132
12,811,987
FP7
1443 433,873,688
433,935,810
428,311,283
-15,214,401
9,627,542
C&S
49 16,726,995
16,695,533
16,377,328
-333,678
15,473
Totals
4326 1,629,444,915 1,625,065,577 1,576,251,739 -71,084,211
22,455,002
Concerning cumulative results, the auditors have so far checked almost EUR 3.3 billion in
costs claimed as part of the FP6, FP7 and C&S audit campaigns.
The cumulative amount of proposed adjustments at funding level for FP6 so far is over
EUR 55 million in favour of the Commission.
Page
35 of
48
Table 3.7 - Results by instrument type (cumulative, FP6).
FP
Instrument / Project Type
% Costs audited
% Adjustments in
favour of the
Commission
CA
Coordination Action
1.1%
2.1%
II
Specific actions to promote
5.6%
6.5%
research infrastructures
IP
Integrated Project
45.5%
44.6%
MCA
Marie Curie Actions
5.8%
3.4%
FP6
NOE
Network of Excellence
7.4%
10.5%
SME
Specific actions for SMEs
1.5%
7.2%
SSA
Specific Support Action
8.5%
12.9%
STP
Specific Targeted Project
10.6%
12.3%
FUSION
FUSION
13.9%
0.5%
FP6 Total
100.0%
100.0%
Even though we do not select representative samples per instrument, the volume of results to
date gives some insights as to whether the incidence of errors is higher for some instruments
than it is for others.
In FP6, the instrument with the highest ratio of errors to amounts is the specific actions for
SMEs. This may still to be the case in FP7, but it is not reflected in the table above
because the equivalent instrument is managed and audited by REA.
Table 3.8 - Results by instrument type (cumulative, FP7).
FP
Instrument / Project Type
% Costs audited
% Adjustments in
favour of the
Commission
CP
COLLABORATIVE PROJECT
2.0%
3.3%
(GENERIC)
CP-CSA-
INTEGRATING ACTIVITIES / E-
9.2%
16.5%
Infra
INFRASTRUCTURES /
PREPARATORY PHASE
CP-FP
SMALL OR MEDIUM-SCALE
26.0%
27.4%
FOCUSED RESEARCH
PROJECT
CP-IP
LARGE-SCALE INTEGRATING
25.4%
34.0%
PROJECT
CP-SICA
SPECIFIC INTERNATIONAL
1.3%
1.0%
FP7
COOPERATION ACTIONS
CP-TP
COLLABORATIVE PROJECT
3.3%
4.5%
TARGETED TO A SPECIAL
GROUP (SUCH AS SMEs)
CSA-CA
COORDINATION (OR
2.4%
3.5%
NETWORKING) ACTIONS
CSA-ERA-
ERANETPLUS
6.7%
0.9%
Plus
CSA-SA
SUPPORT ACTIONS
9.8%
7.8%
NoE
NETWORK OF EXCELLENCE
0.3%
0.2%
FUSION
FUSION
13.5%
0.9%
FP7 Total
100.0%
100.0%
Page
36 of
48
In FP7, the instruments with the highest ratio of errors to amounts are the support for
infrastructures and the large-scale integration projects. This was already the case at the
end of last year. It is worth noting also that ERANETPLUS projects show particularly low
error levels, although this finding is based on a small number of results (only 12
ERANETPLUS participations have been audited to date).
The low incidence of error in the audits of Fusion associations witnessed in FP6 continues
in FP7.
3.3. Audit coverage
Ensuring sufficient audit coverage is an essential concept behind the audit strategies, and the
key to detecting and correcting as many errors as possible. This section shows the level of
coverage achieved by the end of 2012.
Table 3.9 - Audit coverage (cumulative, FP6 & FP7)
FP6
FP7
Total number
Audit coverage by
of
55,879
48,410
number of audited
participations
5.1%
3.0%
participations
Audited
2,834
1,443
participations
Audit coverage by amounts audited
1,174,434,234
9.9%
433,935,810
7.0%
('direct' coverage)
Audit coverage of non-audited
amounts received by audited
6,050,635,230
51.2%
3,041,179,808
49.1%
beneficiaries ('indirect' coverage')
Total audit coverage ('direct' and
7,225,069,464
61.1%
3,475,115,618
56.1%
'indirect')
Total RTD expected EC
11,827,435,215
100.0%
6,197,294,558
100.0%
contributions as of the end of 2012
Page
37 of
48
Figure 3.1 – Comparison of audit coverage
One of the main objectives of the FP6 Audit Strategy was achieved already during 2009:
to 'clean' from systematic material errors at least 50% of the budget. In 2012, the 'cleaning'
effect has reached 61.1% and, with 42 FP6 audits still ongoing, the final result will be
significantly higher than the original target.
For FP7, audit coverage of RTD cost statements is 56.1%, with 7% of the amounts audited
directly. While the FP6 auditable population20 stopped growing at the end of 2010, the
FP7 population continues to grow over time and therefore audit coverage is only relative
to the size of that population at a given point in time. With that in mind, it is worth
pointing out that, proportionally, audit coverage grew much quicker in 2012 than the
population itself: at the end of 2011, total audit coverage was only 35.6%; now it is
56.1%.
20 The auditable population is the portion of a programme's budget which is auditable at a given point in time. In
FP7, it consists of the cost statements submitted to the EC.
Page
38 of
48
3.4. Analysis
This section provides more in-depth analysis of certain aspects and results of the work of the
external audit units, particularly in relation to error rates, error types, and the most prevalent
errors at cost category level.
3.4.1.
Analysis of error rates
Table 3.10 - Error rates (2012, FP6 & FP7). All amounts are EC share
FP
Strategy strand
Costs accepted by
Adjustments in
Overall error rate
Financial Officers
favour of the
Commission
FP6
TOP
5,156,007
-417,735
-8.10%
MUS
453,513
-31,213
-6.88%
RISK
33,729,019
-8,127,409
-24.10%
FUSION
20,772,877
-2,734
-0.01%
Total FP6
60,111,417
-8,579,090
-14.27%
Total FP7
238,001,866
-7,829,757
-3.29%
FP6 error rates in 2012 were again higher than in previous years. Contributing factors to this
result were:
The proportion of proposed adjustments over EUR -100,000 was much higher in 2012
(10.3%) than in previous years (3.7%).
There was a single result in 2012 which was the biggest proposed adjustment at funding
level ever (over EUR 2.6 million, more than a million higher than the previous 'record').
On the other hand, the FP7 rate was similar to last year's. The evolution of error rates year-on-
year can be seen in the graphs below.
Table 3.11 - Error rates (cumulative, FP6 & FP7). All amounts are EC share
FP
Strategy strand
Costs accepted by
Adjustments in
Overall error
Representative
Financial Officers
favour of the
rate
error rate
Commission
FP6
TOP
503,058,279
-16,878,495
-3.36%
-3.44%
MUS
69,494,498
-2,821,437
-4.06%
RISK
438,240,333
-35,537,669
-8.11%
FUSION
163,641,123
-298,532
-0.18%
Total FP6
1,174,434,234
-55,536,132
-4.73%
Total FP7
433,935,810
-15,214,401
-3.51%
-4.18%*
* This year, the FP7 representative error rate is the result of auditing the first Common
Representative Sample together with the other research Commission services (see section
2.2.2.). It is based on the 136 results collected so far, and just over half of them are from other
services. Another peculiarity is that it has been calculated using a different cut-off date
(01/02/2013) from the rest of the figures in this report in order to take as many results as
possible into account. The final error rate will not be known until the whole sample has been
audited.
Page
39 of
48
High FP6 error rates during 2012 have pushed the cumulative overall FP6 error rate up
again, from -4.21% at the end of 2011 to -4.73%.
The cumulative overall FP7 error rate is slightly lower than at the end of last year.
However, it is expected that this downward trend will reverse in future, as we have
reached the mid-point in the FP7 audit campaign, and we are switching now from
preventive efforts to auditing beneficiaries with a higher risk profile.
Figure 3.2 - Evolution of FP6 error rates up to the end of 2012
As can be seen in figure 3.2, FP6 cumulative overall error rates have seen a year-on-year
regular increase. This increase can be linked to the outcome of RISK-based audits: most
of the audits closed in the first year of the FP6 campaign were part of the initial TOP and
MUS selections, and audits from the first RISK assessment were not launched until
February 2008.
Although it is difficult to quantify, there is also an effect on error rates from audits which
have required long discussions with beneficiaries and which are typically closed towards
the end of the audit campaign. These audits usually result in above-average rates of error.
This effect is particularly visible in the annual rates for 2011 and 2012.
The FP6 representative error rate has been quite stable between -3% and -3.5% since Q3
2009, when it was calculated for the first time. It is -3.44% at the moment, with only 13
FP6 representative audits still to be closed before we know the definitive figure.
Page
40 of
48
Figure 3.3 - Evolution of FP7 error rates up to the end of 2012
The final representative error rate will still evolve. Based on preliminary results, it is
expected that it will be higher once all audits in the common sample are closed.
For non-representative rates, the progression so far has been downwards. There were a
number of audits with unusually high errors early on in the FP7 campaign, which meant
that early rates were higher than expected. However, as the body of results has grown and
the effect of outliers has been cancelled out, rates have gone down as can be seen in the
figure above. In any case, in the last two years rates seem to have stabilised.
Figure 3.4 - Error rates by strategy strand (cumulative, FP6 and FP7)
Page
41 of
48
The fact that the overall FP6 RISK error rate stands at -8.11%, while the representative
rate is much lower (-3.44%), is an indication of the validity of the risk assessment
methods employed to date.
On the other hand, and although one could expect to see the same effect from the audits of
the FP7 corrective strand, corrective and representative rates have been very similar up to
now. The explanation is that the first selections of beneficiaries in the corrective strand
focused on those which were known to participate in many projects but for which only a
few cost statements had been received. The idea was to prevent any errors that might be
discovered in these early cost statements from appearing in future ones. Although this
kind of preventive approach has been worthwhile, selections on this basis cannot be
considered strictly speaking as risk-based, which is reflected in an error rate lower than if
the approach had been purely corrective instead of preventive.
Figure 3.5 – Split of adjustments by type of error (cumulative, FP6 & FP7)
(1)
A series of analyses in 2009 on the share of
systematic errors compared to overall errors
led to the realisation that they were not as
prevalent as assumed when the FP6 Audit
Strategy was prepared. This resulted in
changes to the formula for the calculation of
the residual error rate in order to make it more
accurate, and was also an important
consideration in preparing the FP7 Strategy.
As can be seen in figure 3.5, about one third
of the errors found so far in monetary value
have been systematic. The proportion is
roughly the same in FP6 and FP7.
Table 3.12 – Error rates per type of beneficiary: newcomers (cumulative, FP7)
Number
%
ER
All beneficiaries
683
100.0%
-3.51%
New participants only
134
19.6%
-8.32%
Recurrent participants only
549
80.4%
-2.94%
We have carried out for the first time a comparison between the amounts of error present in
the costs statements submitted by beneficiaries which have never participated in a framework
programme, and those present in the cost statements of experienced participants. The results
show that
newcomers are more prone to errors.
Page
42 of
48
Table 3.13 – Error rates per type of beneficiary: SMEs (cumulative, FP7)
Number
%
ER
All beneficiaries
683
100.0%
-3.51%
SMEs
164
24.0%
-6.61%
Non-SMEs
519
76.0%
-3.07%
We have also looked at the incidence of errors depending on whether a beneficiary is an SME
or not.
SMEs appear to be more prone to errors than other types of beneficiaries.
These beneficiary types were already considered as a potential risk factor21, and actions have
been taken to gather deeper information through a number of targeted audits.
The results shown in tables 3.12 and 3.13 highlight an opportunity to ensure that newcomers
and SMEs have better knowledge of the programme rules before submitting costs. Even if our
internal control system is consciously very much based on ex-post controls, increasing our
efforts in error prevention for those types of beneficiaries is being considered.
3.4.2.
Assessment of the different steps of the control chain
Table 3.14 - Net correction of ex-ante and ex-post controls (cumulative, FP5, FP6 &
FP7). All amounts are EC share22
FP5
FP6
FP7
Totals
Costs claimed and audited (A)
216,647,644
1,178,844,232
433,873,688
1,829,365,564
Costs accepted by Financial Officers (B)
212,579,288
1,174,434,234
433,935,810
1,820,949,332
Net correction from ex-ante controls (B-A)
-4,068,356
-4,409,998
62,122
-8,416,232
Costs accepted by Auditor (C)
209,979,355
1,131,563,128
428,311,283
1,769,853,766
Net correction from ex-post controls (C-B)
-2,599,933
-42,871,106
-5,624,527
-51,095,566
Effect of ex-post controls as a % of all
39%
91%
100%
86%
controls
The net effect of ex-ante and ex-post controls is shown above. By ex-ante, one refers to the
corrections made by financial officers to costs claimed when they are received, and by ex-
post, reference is made to the adjustments proposed by the auditors.
This table clearly illustrates the fact that the internal control system in DG RTD is
consciously very much based on ex-post controls and corrections.
Ex-ante controls had a bigger cumulative effect for FP5 than ex-post controls. However,
both for FP6 and FP7, the opposite is true, and the difference is quite significant. The most
likely explanation for this is that more details were required in the FP5 cost statements so
that ex-ante controls in FP5 were more effective.
These results also raise questions on the reliability and real effect of the Certificates on the
Financial Statements, introduced since FP6. It explains why the Communication
Campaign is also geared towards certifying auditors.
21 E.g. being an FP newcomer has already been identified as a risk factor in the
RTD Risk-based audit approach (section 3.1.6.), Ares(2012)732355 of 19/06/2012.
22 Positive and negative adjustments, both in the ex-ante and ex-post stages, have been netted off for this table.
Page
43 of
48
3.4.3.
Qualitative analysis of error types
Each time an audit is closed, it is given two ratings related to the 'Seriousness' and 'Nature' of
the errors found by the auditors23, if any. By using a combination of these two ratings, a better
understanding of the incidence of errors and their importance can be obtained, as shown in the
table below.
Table 3.15 - Seriousness and nature of errors (cumulative, FP6 & FP7, aggregated at
audit level)
FP
Nature
Seriousness
None
Qualitative
Error
Irregularities
Totals
None
6.3%
0.4%
N/A
N/A
6.6%
Small
N/A
0.1%
39.7%
0.2%
40.0%
FP6
Medium
N/A
0.2%
31.2%
0.4%
31.8%
High
N/A
0.6%
17.9%
3.1%
21.6%
Totals
6.3%
1.3%
88.7%
3.7%
100.0%
None
5.7%
2.1%
N/A
N/A
7.8%
Small
N/A
1.3%
38.5%
0.1%
39.8%
FP7
Medium
N/A
1.0%
40.1%
0.3%
41.4%
High
N/A
0.3%
10.5%
0.3%
11.0%
Totals
5.7%
4.7%
89.0%
0.6%
100.0%
Most of the adjustments proposed by the external audit units are due to straightforward
errors of varying seriousness.
The percentage of audits resulting in no findings at all is slightly lower in FP7 (5.7%) than
in FP6 (6.3%).
The percentage of audits resulting in highly serious errors or irregularities in FP7 (11%) is
almost half of FP6 (21.6%).
The percentage of participations where potential irregularities and serious problems are
found remains fairly low in FP6, at 3.7%, although it is worth mentioning that it was just
2.5% at the end of 2012 and 1.1% at the end of 2009. In FP7 it is only 0.6% so far.
However,
conclusions on the likely incidence of fraud cannot be drawn from this
table, since our audits are not designed with the purpose of discovering fraud, with the
exception of those carried out by the M.1 'OLAF' team. When looking only at the results
from fraud detection, error rates and adjustments are much higher than average:
Table 3.16 - Error rates (OLAF audits). All amounts are EC share
FP
Costs accepted
Adjustments in
Overall error rate
by Financial
favour of the
Officers
Commission
FP6
26,148,056
-6,849,689
-26.20%
FP7
965,133
-53,187
-5.51%
Total
27,113,188
-6,902,876
-25.46%
23 'Seriousness' refers to the severity of problems found (NONE, SMALL, MEDIUM or HIGH), while 'Nature'
reflects the character of those errors (NONE, QUALITATIVE, ERROR or IRREGULARITIES). The criteria
used for these categorisations are described in section 6.2.1 of the Audit Process Handbook.
Page
44 of
48
3.4.4.
Analysis of adjustments at cost category level
This section provides analysis of the incidence of errors at cost category level. Costs claimed
by beneficiaries are ascribed to one of a number of defined cost categories. When audit results
are compiled, they are presented and implemented for an audited participation as a whole,
with results in different cost categories being netted off. However, it can be of value to
consider errors at cost category level, particularly in order to identify in which areas of
expenditure errors are found most often, both in terms of number and in monetary value.
Table 3.17 - Proportion of adjustments by cost category (cumulative, FP6 & FP7)
Personnel
21.9%
26.4%
43.0%
34.5%
23.5%
38.1%
14.7%
29.5%
Subcontracting
4.4%
2.8%
5.0%
4.2%
17.4%
5.2%
4.9%
5.8%
Other direct costs
37.7%
33.7%
17.4%
24.4%
18.2%
14.2%
31.1%
11.6%
Adjustments to costs previously reported (FP6)
4.8%
18.1%
8.2%
3.3%
Total indirect costs
29.8%
33.7%
16.1%
31.4%
30.8%
41.0%
44.2%
43.1%
Receipts
0.4%
0.6%
0.2%
2.3%
0.5%
0.2%
1.5%
0.1%
Interest on pre-financing
0.9%
2.7%
0.2%
1.0%
1.3%
1.0%
0.3%
0.2%
Other (FP7)24
0.2%
2.3%
0.2%
9.6%
Totals
100.0%
100.0% 100.0%
100.0%
100.0% 100.0%
100.0% 100.0%
The level of detail provided in this analysis is limited by the fact that a breakdown of
'Other direct costs' into sub-categories (travel, consumables, durable equipment) is not
available for most outsourced audits.
The monetary amounts related to errors in favour of the EC in FP7 personnel costs are
higher (34.5% of the total) than the number of times we find those errors in our audits
(26.4% of the total). This is not a new finding, but it remains significant, and even more
pronounced in FP6 (43% and 21.9% respectively). At the same time, the opposite is the
case when considering errors in personnel costs in favour of the beneficiary. For FP7, for
example, they constitute 38.1% of all errors but only account for 29.5% of the amounts.
One can conclude that, when beneficiaries commit errors in personnel costs, they are
generally more beneficial to them in monetary value when they are in their favour than
when they are not.
The incidence of errors in indirect costs is higher in FP7 than in FP6, both in number and
in volume.
24 Lump sums / flat rate / scale of unit declared / access costs, and Total Funding of the Joint Selection List of
Trans-National Projects.
Page
45 of
48
3.4.5.
Analysis of the FP7 representative error rate
The Common Representative Sample of the FP7 audits performed by the RDG family has
been examined in order to further analyse the error rate, which as of 1 February 2013 is -
4.18%. Each cost statement in the CRS which has been subject to an audit adjustment has
been examined per cost category, using a standardised typology to classify the underlying
reasons for each error.
The synthesis below reflects the consolidated outcome of the analyses performed by each
research Commission service on their share of the CRS.
3.4.5.1. Error by cost category25
The analysis confirmed that the errors which have the higher weight in monetary terms within
the -4.18% rate occur within the personnel cost category (41%), followed by errors within the
Indirect Cost category (31.7%). The remaining 27.3% occurs mainly in the cost category
Other Direct Cost, with a mistaken calculation and/or application of the depreciation charge
being particularly present.
Table 3.18 – Errors by Cost Category
These results are somehow obvious because Personnel and Indirect Costs are the largest Cost
categories in terms of budget allocation.
3.4.5.2. Errors by type
In an attempt to analyse further the underlying elements that triggered the errors presented in
the previous table, we also examined the reasons for the occurrence of the errors.
25 The Common Representative Sample has been drawn using the Monetary Unit Sampling (MUS) methodology.
This means that every cost statement in the sample represents the same portion of the FP7 budget, regardless of
the amounts audited. In order to analyse correctly the composition of the representative error rate in the tables
below, a weighting factor consistent with the MUS methodology has been applied to each representative audit
finding.
Page
46 of
48
One fifth of the errors, namely in Indirect Costs, is an automatic consequence of the errors
encountered under total Direct Cost (in other words, it is the direct effect of errors in the
direct Cost category, accounting for 20.55%).
The vast majority of the errors associated with a lack of supporting documentation relates
to issues associated with personnel cost (lack of contracts, incorrect or irregular
timesheets, lack of extracts of payroll, lack of invoices for in-house consultants, etc.). All
cost categories concerned, the errors associated with a lack of supporting documentation
represent 17.44% of the errors.
A mistaken calculation of the depreciation charge or its wrong application represents
13.38% of the errors encountered in terms of weight, although only 2.6% in terms of
frequency.
Errors associated with mistakenly charged subcontracting account for 7.89% of the total,
but only 1.5% in terms of frequency.
The remaining errors accumulated under the header 'Other' refer to several types of errors
that individually are immaterial.
3.4.5.3. Consequent actions
All the errors encountered above and their typologies are already known to our services, and
these results confirm the outcome of similar exercises made in past years. For this reason we
continue the Communication Campaign exercise which mainly focuses on the major reasons
for errors in FP7, i.e.: errors encountered under Personnel, Depreciation charge and Indirect
Costs. The Communication Campaign is an ongoing process that started last year; however its
results are not expected to materialise immediately, due to the geographical coverage of the
campaign and the time lag between the campaign and the consequent reporting.
Finally, since most errors in Indirect Costs are due to root causes in the Direct Cost
categories, we expect a decrease in Indirect Cost errors consequent to Direct Cost error
prevention measures.
Page
47 of
48
ANNEX I: MISSION STATEMENT
Unit RTD M.1 - External audits
(including Sector Outsourced audits and audit certification policy)
The unit contributes to the assessment of the legality and regularity of DG RTD payment
transactions by means of ex-post financial audits performed either by in-house auditors or
through independent professional audit firms. It thereby provides a basis for reasonable
assurance to the management and other stakeholders (including the budget discharge
authorities) that research grant beneficiaries are in compliance with the financial rules.
Through close co-operation and harmonisation with the other Research DGs, Executive
Agencies and Joint Undertakings, the unit takes the lead in establishing relevant audit policies
and strategies and chairs the Extrapolation Steering Committee through which a coherent
Research DG approach is defined on the extension of audit results with regard to
beneficiaries. The unit defines and implements the cost methodology certification function for
FP7, contributing in an
ex-ante manner to the legality and regularity of future DG RTD
payment transactions. The unit manages the relations with OLAF on irregularities and fraud
cases concerning research grant beneficiaries. The unit also contributes in an advisory
capacity to the developments of future policy rules (in particular rules for participation and
model grant agreement provisions) and business processes, based upon the knowledge gained
in the audit and certification process.
Page
48 of
48