Ref. Ares(2019)6643033 - 28/10/2019
13th meeting of the monitoring sub-group of the Coordination
Group on the EU clinical trials Regulation
Wednesday, 17th of July, 2019
, European Commission
1. Adoption of the agenda
The agenda was adopted.
2. Sprint 7 reporting IT4U
Everis explained the results for the KPI reporting please refer to the powerpoint presentation
(2019_07-17 CTIS update Monitoring sub-group_ v1 9 - see annex). Slides 4, 5 and 6 were discussed
in detail. It was emphasised before the meeting that the EMA tresholds mentionned in the slides are
not to be confused with the tresholds set up by the Monitoring Subgroup. The EMA tresholds were
communicated to Everis
Discussion in detail:
: the KPI that was agreed is presented in bold. The result shows
that out of the items that were in the sprint scope, were delivered. items were
started but not completed, and items were re-opened before the FAT (factory acceptance
testing own Everis testing). In order to create more clarity, Everis was asked to provide an
overview (see annex) that summarises the evolution of the number of items in the sprint.
The KPI result would therefore be
). After the next step (SAT
site acceptance testing at the EMA enviroment), 8 items were reopened, leading to items
being delivered for business validation.
items were closed after validation by
: next to the items that
the SAT (see above), there were
bugs created. These break down in
bugs. This leads to a net number of items of
As planned, a more complete set of KPI (including metrics on regression testing, test automation,
security testing and user satisfaction) will be available for the next sprint.
Everis recognises that the result is below expectations, and has already proposed some corrective
measures (see also slide 7). Examples are increased sanity checks to identify dependencies and
improve the accuracy of estimates. For future sprints, reserve capacity will be foreseen to tackle
3. Fixed price part : remaining items & results of regression testing
The 3 remaining items of the fixed price part will be delivered during sprint 8. The regression issues
have been added to the list of items for which a business value needs to be assigned.
4. Release 2 planning
The timeline for the adoption of the release 2 planning was briefly discussed on the basis of slide 10.
Product Owners have been working on business blockers (i.e. key functionalities for which no
appropriate alternative way of working in the Clinical Trial Information System could satisfactorily
solve the needs of a typical user). These business blockers should be prioritized in the different
workstreams. The plan for release 2 will be adopted by the CTIS expert group on the 1st of August.
5. Internal discussion on results and thresholds
After Everis stepped out of the meeting, the KPI results were briefly discussed. In general, the results
are disappointing. The number of (net) items delivered is rather low, especially when compared to
the size of the backlog. This implies that the time to deliver the auditable release would be longer
than anticipated. Positive point is that the all items that passed the SAT also passed business
The impact of the corrective actions will (in part) be visible in the sprint 8 reporting. Combined with
the extra numbers, this will give a better picture.
The new program manager,
, presented himself
The survey that will be used to measure the user satisfaction is stil under discussion. COM
asked to finalise the survey in order to have a fixed set of questions to measure user
satisfaction. The final version will be circulated to the Monitoring Sub-group for information.