Ceci est une version HTML d'une pièce jointe de la demande d'accès à l'information 'Documents and statistics on the EMM Open Source Intelligence Suite'.



Ref. Ares(2016)1246919 - 11/03/2016
EUROPEAN COMMISSION 
DIRECTORATE-GENERAL 
JOINT RESEARCH CENTRE 
Directorate G - Institute for the Protection and Security of the Citizen 
Global security and crisis management  
Limited1
Support to Open Sources User Community 
Enclosure: Second Interim Report 
c.c.:
DG HOME 


JRC 






1 This document is sensitive and limited for use within the European Union institutions, other offices and 
agencies  established  by  virtue  or  on  the  basis  of  the  Treaties,  EU  Member  States  and  public 
administrations. 

Distribution on need to know basis

Not  for  public  dissemination.  It  is  otherwise  accepted  to  upload  to  ARES  or  other  internal  IT
systems

Where e-mail is used, this must be encrypted using SECEM

May be distributed to other EU institutions or MS

Further  distribution  on  need  to  know  basis  allowed  but  must  be  aware  that  document  is  not
released outside EU institutions and MS public administrations.


 
EUROPEAN COMMISSION 
DIRECTORATE-GENERAL 
JOINT RESEARCH CENTRE 
Directorate G - Institute for the Protection and Security of the Citizen 
  Global security and crisis management  
 
 
 
 
Support to Open Sources User Community 
 
Second Interim Report 

 
 
 
 
 
 
 
 
 
 
 
 
 
Administrative Arrangement  
Between 
DG Home Affairs (DG HOME) 
No. HOME/2014/ISFP/AA/001-A1 
and 
Joint Research Centre (JRC) 
JRC Contract No. 33362-2014 NFP 
 
 
 
 
 
 
 
 
European Commission, Via Enrico Fermi 2749, I-21027 Ispra (Varese) - Italy. Telephone: (39)0332-78-9
 
. Telephone: direct line (39)0332-78
 Fax: (39)0332-78-

 
E-mail: g
 
Limited 

Title of Report or Deliverable: Support to Open Sources User Community – Second Interim 
Report 
 
JRC Author(s):  
 
 
Classification: Limited 
 
Distribution list: 
DG JRC 

@ec.europa.eu 

@jrc.ec.europa.eu 

@jrc.ec.europa.eu 

@jrc.ec.europa.eu 

@ec.europa.eu 

@jrc.ec.europa.eu 

 
DG HOME 

@ec.europa.eu 

@ec.europa.eu 
 
 
Type of report: Progress project  
 
Institutional project: 213- EMM 
 
 
 
 
 
 
 
Name 
Date 
Signature 
Reviewed by 
the Project 
 
 
 
Leader  
Approved 
and 
 
 
 
endorsed by 
the HoU 
 
 
 
Legal notice: 
The information contained in this document may not be disseminated, copied or utilised without the written authorization 
of the Commission. The Commission reserves specifically its rights to apply for patents or to obtain other protection for 
the matter open to intellectual or industrial protection. 
The distribution of this document is limited to the persons given in the distr bution list. 
Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which 
might be made of the information contained in this document.  
 
 
Limited 

Document History 
Date 
Author 
Comments 
26.01.2016 
 
Initial draft 
20.02.2016 
 
Improved structure and charts 
26.02.2016 
 
Reformatted using Eurolook 
 
 
 
Limited 

link to page 7 link to page 7 link to page 9 link to page 9 link to page 9 link to page 10 link to page 10 link to page 10 link to page 10 link to page 10 link to page 10 link to page 11 link to page 11 link to page 11 link to page 17 link to page 17 link to page 19 link to page 19 link to page 19 link to page 20 link to page 20 link to page 20  
 
TABLE OF CONTENTS 

INTRODUCTION ....................................................................................................... 6 

PROJECT PLAN AND WORK PACKAGES ............................................................ 6 

DELIVERED WORK PACKAGES ........................................................................... 8 
WP 3.1.1.1 OSINT Workshop for MS practitioners ........................... 8 
Deliverables: ........................................................................................ 8 
WP 3.1.1.2: Workshop Evaluation ...................................................... 9 
Deliverables: ........................................................................................ 9 
WP 3.1.1.3: Workshop for EMM Institutional Users ........................ 10 
Deliverables: ...................................................................................... 10 
WP 3.1.2.3: On-site user training for existing and new users ........... 10 
Deliverables: ...................................................................................... 10 
WP 3.1.3.1: Operational updates EMM OSINT Suite ...................... 10 
Deliverables: ...................................................................................... 10 
WP  3.2.1.1  On-site  operation  support  with  JRC  trained 
experts ............................................................................... 11 
WP 3.3.1 Inception Report ................................................................ 11 

PROJECT PLAN FOR THE NEXT MONTHS ........................................................ 12 

RISKS AND CHALLENGES ................................................................................... 12 

ANNEX ..................................................................................................................... 14 
6.1 
EMM OSINT Workshops Participants ............................................................ 14 
Workshop 2015 ......................................................................................................... 16 
Workshop Feedback .................................................................................................. 18 
6.1.1 
Feedback Questionnaire .................................................................... 18 
6.1.2 
Feedback Results ............................................................................... 18 
6.1 
Meeting  Minutes  Institutional  Users  of  EMM  Meeting  November 
2014 ................................................................................................................. 20 
6.1.1 
Introduction ....................................................................................... 20 
6.1.2 
EMM Developments ......................................................................... 20 
6.1.3 
News from Participants ..................................................................... 20 
6.1.4 
Shared Repository Discussion ........................................................... 21 
Limited 

link to page 12 link to page 13 link to page 15 link to page 16 link to page 16 link to page 16 6.1.5 
Conclusions and Actions ................................................................... 22 
6.1.6 
List of Attendees ................................................................................ 22 
6.2 
OSINT Survey Results .................................................................................... 23 
6.2.1 
Background of Participants ............................................................... 23 
6.2.2 
Use of the EMM OSINT Suite software ........................................... 25 
6.2.2.1 
Reasons for not using the software .................................... 26 
6.2.3 
Assessment of the EMM OSINT Suite software ............................... 27 
6.2.3.1 
Strong points ...................................................................... 27 
6.2.3.2 
Weak points ....................................................................... 27 
 
 
 
Limited 


INTRODUCTION 
The goals of the Administrative Arrangement (AA) are the following: 
(1) 
Improve the capabilities of Member States law enforcement authorities in the use 
of open source information to combat crime and terrorism. 
(2) 
Support DG HOME’s strategic analysis and response capability with EMM tools 
and accompanying services. 
This arrangement is a continuation of the work of previous collaborations which started 
around seven years ago with the goal to increase the use of open source information for 
law enforcement purposes. 
The  Joint  Research  Centre  (JRC)  has  been  involved  in  scientific  research  to  improve 
multilingual  tools  and  applications  for  Open  Source  Intelligence.  With  the  mandate  of 
DG HOME the JRC was able to transfer these research results into operational tools to be 
used by Member State law enforcement authorities and EU institutions for the purpose of 
gathering information from the Internet.  
This report revisits the project plan, then outlines the delivered work packages since the 
last report and finally gives an outlook of the planned work going forward. 

PROJECT PLAN AND WORK PACKAGES 
The different work packages are described in the technical annex as well in the inception 
report. The following table shows a consolidated overview of all work packages (WP). 
 
Ref 
Description 
Planned Execution 
Community of OSINT practitioners in law enforcement 
WP 
Workshop for  OSINT practitioners 
Nov 2014 
3.1.1.1 
Nov 2015 
Oct 2016 
WP 
Workshop Evaluation 
after each workshop 
3.1.1.2 
WP 
Workshop for EU institutional users 
Nov 2014 
3.1.1.3 
Disseminate JRC developed software tools 
WP 
Usage  Evaluation  EMM  OSINT  May 2015 
3.1.2.1 
Suite 
WP 
Dissemination EMM OSINT Suite 
June 2015 - Oct 2016 
3.1.2.2 
WP 
On-site  user  training  EMM  OSINT  on request  
3.1.2.3 
Suite 
Improve and sustain EMM OSINT Suite operational use 
WP 
Operational  Updates  EMM  OSINT  Nov 2014 – Oct 2016 
Limited 



 

DELIVERED WORK PACKAGES 
WP 3.1.1.1 OSINT Workshop for MS practitioners 
Organise three workshops for law enforcement practitioners about the use of open source 
information and its practical application.  
Deliverables: 
  Three workshops for MS law enforcement authorities 
  Supporting event web site with online workshop material 
Workshop 2014 
The  first  workshop  for  OSINT  practitioners  was  organised  by  the  JRC  in  November 
2014. The workshop was attended by 50 participants2. The workshop featured hands-on 
trainings both for EMM server and EMM OSINT Suite. 
Workshop 2015 
The second EMM  OSINT workshop  took  place  in  November 2015. The workshop  was 
attended  by  46  participants.  The  workshop  featured  hands-on  trainings  both  for  EMM 
server  and  EMM  OSINT  Suite  and  the  following  presentations  by  experts  from  the 
Member States: 
"Potential online recruitment through connections made between members of 
Romanian Muslims and immigration/ refugees groups”
 
Romanian Intelligence Service 
Social media can help to identify connections between potential terrorist groups. Key 
influencers in these groups can be identified and monitored. 
"United Nations Office on Drugs and Crime – Global Product Portfolio "  
United Nations Office on Drugs and Crime 
The UNODOC has developed a portfolio of software products which are used to counter 
money laundering as well as improve the process of public tendering. The products are in 
worldwide use by financial crime units and governments. 
"Monitoring Social Media for Customs"  
Customs National Office, The Netherlands 
The talk discussed different ways to use information from social media for customs use. 
This comprises cargo movements broadcasted through Twitter as well as detecting of 
counterfeit products on blogs and personal sites. 
                                                 
2 Please refer to the First Interim Report for the participant list and more details.  
Limited 

link to page 20 "EMM OSINT Suite in Education",  
University of Applied Science (Hogeschool) Leiden, The Netherlands 
The EMM OSINT Suite is used in a curriculum for a forensics degree in Leiden. The talk 
gave an overview on how the software can be used for teaching basic OSINT skills. 
WP 3.1.1.2: Workshop Evaluation 
Evaluate workshop to improve subsequent events based on feedback of attendants. 
Deliverables: 
  On or offline survey to evaluate workshop results 
  Evaluation of survey and results summary 
  Changes to next event format 
 
After each workshop we conducted a feedback survey by sending out an assessment form 
or using the Commission’s online survey tool. The form and the results are available in 
Annex 6.1.2. 
WP 3.1.2.3: On-site user training for existing and new users 
Provide on-site trainings for new and existing users of the EMM OSINT Suite. This work 
package can be requested by MS authorities to learn how to use the software or how to 
adapt it to special use case scenarios. 
Deliverables: 
  Updated and customised training material 
  On-site training delivery 
 
The following on-site trainings for 2016 have been requested by MS: 
  Tax Authority and Financial Police, The Netherlands 
  National Crime Agency, United Kingdom 
  Hogeschool Leiden in collaboration with Dutch Authorities, The Netherlands 
WP 3.1.3.1: Operational updates EMM OSINT Suite 
Provide a quarterly release to keep the software current. 
Deliverables: 
  Quarterly bug fix release 
  Release notes and announcement mail 
 
In  2015 we  provided  four operational updates  for the EMM OSINT Suite (2.3.2-2.3.6). 
We plan the next major update for the first quarter of 2016. 
Limited 

WP 3.2.1.1 On-site operation support with JRC trained experts 
The JRC provides on-site experts in order to support on-going operations and to facilitate 
the  transfer  of  technical  knowledge.  The  experts  are  based mainly  in  Brussels  but  have 
the  opportunity  to  work  in  Ispra  at  specific  intervals  to  get  updated  with  the  latest 
technical knowledge. 
Deliverables (on-site): 
  Operations support to keep EMM tools running 
  Development  of  new  categories,  refinement  of  existing  categories  for  DG  HOME’s 
areas of interest 
  Production of open source daily products 
  Reducing operational risks by proving latest expertise on the use of EMM tools 
  Facilitate understanding of the system 
Deliverables (Ispra): 
  Training of the experts in the latest EMM products and developments 
  Remote support of the experts 
The  on-site  support  commenced  continuing  operation  from  a  previous  arrangement  as 
planned  in  January  2015.  The  on-site  support  concluded  end  of  2015  and  will  be 
continued by different AA in 2016. 
WP 3.3.1 Inception Report  
Submit inception report within one week after kick-off meeting, describing project plan 
for the first 6 months, team composition and further results of kick-off meeting. 
The inception report incorporating the kick-off meeting was delivered in January 2015. 
WP 3.3.2 First Interim Report 
The first interim report was delivered in April 2015. 
WP 3.3.3 Second Interim Report 
In order to include the results of the Usage evaluation of EMM OSINT Suite, the second 
interim report will be delivered in early 2016, later than originally planned. 
WP3.1.2.1 Usage evaluation of EMM OSINT Suite (“OSINT Survey”) 
Conduct a user survey to evaluate the use of the EMM OSINT Suite tool.  
Deliverables: 
  On- or offline survey to evaluate software use 
Limited 

  Evaluation of survey and results summary 
The JRC conducted an online survey (using the Commission’s EUSurvey web tool) with 
users of the EMM OSINT Suite.  
Due to resource constraints the survey was delayed to the end of 2015.  To validate the 
survey questionnaire a pilot survey with 15 law enforcement officers in November. The 
full survey was conducted with 270 users of the JRC’s online community in December. 
The JRC sent out 270 invitations to take part and received 76 answers (28%). 
The  survey  was  composed  of  a  number  of  sections  to  gather  information  about  the 
knowledge of the participants, the use of OSINT and our provided tools and feedback on 
the actual tool.  Finally, we asked for input  on how to  improve the dissemination of the 
tool. 
 
Survey Results 
Background of Participants 
The first few questions tried to assess the background of the participants, such as the 
organisational background and the knowledge of OSINT as a tool. 
In what type of organisation are you 
working? 
25
21 
19 
20
18 
15 
15
10
5
0
Customs
Police
Tax Authority
Other
 
Other organisations mentioned: 
  Economic Crime Authority 
  Intelligence 
  Competition Authority 
  Combined task force police and customs 
  University 
Limited 







  Software lacks crucial features (2 participants) 
  Other reasons (14 participants) 
o  Missing hardware requirements 
o  Tested old version needs to evaluate latest version 
o  Moved job 
Again this gives a strong indication that training and documentation is not sufficient. 
Assessment of the EMM OSINT Suite software 
We asked the participants to tell us strong and weak points of the software. 
Strong points 
  Tool is free and free tools are rare for law enforcement 
  Extracting from PDFs 
  Built from the ground up for law enforcement 
  Software is working solid 
  Good for student education 
  Entity extraction and especially custom entities based on regular expression 
Weak points 
  Problems with large data sets 
  Hard to learn “high threshold” to use 
  Limited access to sites with password protection 
  Difficult to add other search engines 
  Limited language support  
Results Summary and Conclusions 
The  majority  of  users  is  using  open  source  information  regularly  and  is  proficient  in 
doing so. The latest version of the EMM OSINT Suite tool has been tried by a majority 
of users but is used operationally only by 44%. Only 41% have taken part in training.  
In  order  to  improve  the operational use  the  JRC  needs  to  improve  training  for  the  tool. 
Also measures to better dissemination the software are needed.  
Limited 


 PROJECT PLAN FOR THE NEXT MONTHS 
The following work packages will be delivered within the next months 
  WP  3.1.1.1:  EMM  OSINT  Workshop  -  the  2016  workshop  is  scheduled  for  21.-
23.09.2016 
  WP 3.1.2.2: EMM OSINT Suite Dissemination 
Based on the results of the survey, we are planning the following steps: 
  Create dissemination package (brochure, quick start guide, evaluation download, 
get-started videos) 
  Send package to all survey participants who asked for more information 
  Send package to further law enforcement contacts 
  Additional on-site training sessions in Ispra 
  WP 3.1.3.2: OSINT Improvements 
The survey provided a lot of feedback on how to improve the software. We are planning 
the following changes with the focus on usability improvements and improvements of 
documentation and training 
  Improve user interface to be more consistent and responsive 
  Make modules with importance for law enforcement more accessible: 
o  Easier definition of custom entity types 
o  Better integration into existing law enforcement tool chain 
  Improve documentation 
o  Videos to get started 
o  Consistency 
o  Focus on use cases 
  WP 3.1.3.1: The EMM OSINT Suite software will receive further operational updates, 
such as security patches to stay current. 

RISKS AND CHALLENGES 
The execution of several milestones faces certain risks and challenges.  
We have identified the following risks: 
  Development of software tools not meeting user requirements 
Limited 

  Software tools not meeting minimal quality standards 
  Lack of personnel resources 
 
To mitigate the risks the following measures will be taken: 
  Increase  communication  and  collaboration  with  MS  authorities  and  community 
through website and dedicated mailing list 
  Software  will  be  tailored  for  law  enforcement  purposes  to  cover  domain  specific 
requirements 
  Incremental software delivery process which allows changes to user requirements all 
along the way 
  Systematic testing of tools and deliverables to ensure decent quality 
  Timely reporting and discussion of activities between JRC and DG HOME to ensure 
effort is directed towards joint goals 
 
 
 
Limited 






(2) (b) 
Presentation  style  was  suitable  for  the  4,60 
4,1 
event 
(2) (c) 
Speakers covered all expected topics 
4,33 
3,5 
(2) (d) 
Time  for  questions  and  discussion  was  4,73 
4,4 
sufficient 
Documentation  Material covered all topics 
4,40 
3,6 
(3) (a) 
(3) (b) 
Structure and layout was appropriate (i.e.  4,47 
3,8 
easy to read, understandable, etc.) 
Facilities  and  Presentation  aids  and  room  equipment  4,73 
4,1 
Services 
were appropriate 
(4) (a) 
(4) (b) 
Facilities  (incl.  room  layout,  group  4,67 
4,5 
meeting  rooms,  etc.)  were  functional  to 
the purpose 
(4) (c) 
Technical  support  staff  was  available  4,80 
4,4 
when needed 
Before 
the  Necessary  information  and  enough  3,73 

Event 
support were available 
(5) (a) 
Overall 
My  expectations  about  the  event  were  4,60 
3,5 
Outcome 
met 
  
(6) (a) 
  
 
Limited 
Electronically signed on 09/03/2016 17:15 (UTC+01) in accordance with article 4.2 (Validity of electronic documents) of Commission Decision 2004/563