Ref. Ares(2016)1246919 - 11/03/2016
EUROPEAN COMMISSION
DIRECTORATE-GENERAL
JOINT RESEARCH CENTRE
Directorate G - Institute for the Protection and Security of the Citizen
Global security and crisis management
Limited1
Support to Open Sources User Community
Enclosure: Second Interim Report
c.c.:
DG HOME
JRC
1 This document is sensitive and limited for use within the European Union institutions, other offices and
agencies established by virtue or on the basis of the Treaties, EU Member States and public
administrations.
Distribution on need to know basis
Not for public dissemination. It is otherwise accepted to upload to ARES or other internal IT
systems
Where e-mail is used, this must be encrypted using SECEM
May be distributed to other EU institutions or MS
Further distribution on need to know basis allowed but must be aware that document is not
released outside EU institutions and MS public administrations.
EUROPEAN COMMISSION
DIRECTORATE-GENERAL
JOINT RESEARCH CENTRE
Directorate G - Institute for the Protection and Security of the Citizen
Global security and crisis management
Support to Open Sources User Community
Second Interim Report
Administrative Arrangement
Between
DG Home Affairs (DG HOME)
No. HOME/2014/ISFP/AA/001-A1
and
Joint Research Centre (JRC)
JRC Contract No. 33362-2014 NFP
European Commission, Via Enrico Fermi 2749, I-21027 Ispra (Varese) - Italy. Telephone: (39)0332-78-9
. Telephone: direct line (39)0332-78
Fax: (39)0332-78-
.
E-mail: g
Limited
Title of Report or Deliverable: Support to Open Sources User Community – Second Interim
Report
JRC Author(s):
Classification: Limited
Distribution list:
DG JRC
-
@ec.europa.eu
-
@jrc.ec.europa.eu
-
@jrc.ec.europa.eu
-
@jrc.ec.europa.eu
-
@ec.europa.eu
-
@jrc.ec.europa.eu
-
DG HOME
-
@ec.europa.eu
-
@ec.europa.eu
Type of report: Progress project
Institutional project: 213- EMM
Name
Date
Signature
Reviewed by
the Project
Leader
Approved
and
endorsed by
the HoU
Legal notice: The information contained in this document may not be disseminated, copied or utilised without the written authorization
of the Commission. The Commission reserves specifically its rights to apply for patents or to obtain other protection for
the matter open to intellectual or industrial protection.
The distribution of this document is limited to the persons given in the distr bution list.
Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use which
might be made of the information contained in this document.
Limited
Document History
Date
Author
Comments
26.01.2016
Initial draft
20.02.2016
Improved structure and charts
26.02.2016
Reformatted using Eurolook
Limited
link to page 7 link to page 7 link to page 9 link to page 9 link to page 9 link to page 10 link to page 10 link to page 10 link to page 10 link to page 10 link to page 10 link to page 11 link to page 11 link to page 11 link to page 17 link to page 17 link to page 19 link to page 19 link to page 19 link to page 20 link to page 20 link to page 20
TABLE OF CONTENTS
1
INTRODUCTION ....................................................................................................... 6
2
PROJECT PLAN AND WORK PACKAGES ............................................................ 6
3
DELIVERED WORK PACKAGES ........................................................................... 8
WP 3.1.1.1 OSINT Workshop for MS practitioners ........................... 8
Deliverables: ........................................................................................ 8
WP 3.1.1.2: Workshop Evaluation ...................................................... 9
Deliverables: ........................................................................................ 9
WP 3.1.1.3: Workshop for EMM Institutional Users ........................ 10
Deliverables: ...................................................................................... 10
WP 3.1.2.3: On-site user training for existing and new users ........... 10
Deliverables: ...................................................................................... 10
WP 3.1.3.1: Operational updates EMM OSINT Suite ...................... 10
Deliverables: ...................................................................................... 10
WP 3.2.1.1 On-site operation support with JRC trained
experts ............................................................................... 11
WP 3.3.1 Inception Report ................................................................ 11
4
PROJECT PLAN FOR THE NEXT MONTHS ........................................................ 12
5
RISKS AND CHALLENGES ................................................................................... 12
6
ANNEX ..................................................................................................................... 14
6.1
EMM OSINT Workshops Participants ............................................................ 14
Workshop 2015 ......................................................................................................... 16
Workshop Feedback .................................................................................................. 18
6.1.1
Feedback Questionnaire .................................................................... 18
6.1.2
Feedback Results ............................................................................... 18
6.1
Meeting Minutes Institutional Users of EMM Meeting November
2014 ................................................................................................................. 20
6.1.1
Introduction ....................................................................................... 20
6.1.2
EMM Developments ......................................................................... 20
6.1.3
News from Participants ..................................................................... 20
6.1.4
Shared Repository Discussion ........................................................... 21
Limited
link to page 12 link to page 13 link to page 15 link to page 16 link to page 16 link to page 16
6.1.5
Conclusions and Actions ................................................................... 22
6.1.6
List of Attendees ................................................................................ 22
6.2
OSINT Survey Results .................................................................................... 23
6.2.1
Background of Participants ............................................................... 23
6.2.2
Use of the EMM OSINT Suite software ........................................... 25
6.2.2.1
Reasons for not using the software .................................... 26
6.2.3
Assessment of the EMM OSINT Suite software ............................... 27
6.2.3.1
Strong points ...................................................................... 27
6.2.3.2
Weak points ....................................................................... 27
Limited
1
INTRODUCTION
The goals of the Administrative Arrangement (AA) are the following:
(1)
Improve the capabilities of Member States law enforcement authorities in the use
of open source information to combat crime and terrorism.
(2)
Support DG HOME’s strategic analysis and response capability with EMM tools
and accompanying services.
This arrangement is a continuation of the work of previous collaborations which started
around seven years ago with the goal to increase the use of open source information for
law enforcement purposes.
The Joint Research Centre (JRC) has been involved in scientific research to improve
multilingual tools and applications for Open Source Intelligence. With the mandate of
DG HOME the JRC was able to transfer these research results into operational tools to be
used by Member State law enforcement authorities and EU institutions for the purpose of
gathering information from the Internet.
This report revisits the project plan, then outlines the delivered work packages since the
last report and finally gives an outlook of the planned work going forward.
2
PROJECT PLAN AND WORK PACKAGES
The different work packages are described in the technical annex as well in the inception
report. The following table shows a consolidated overview of all work packages (WP).
Ref
Description
Planned Execution
Community of OSINT practitioners in law enforcement
WP
Workshop for OSINT practitioners
Nov 2014
3.1.1.1
Nov 2015
Oct 2016
WP
Workshop Evaluation
after each workshop
3.1.1.2
WP
Workshop for EU institutional users
Nov 2014
3.1.1.3
Disseminate JRC developed software tools
WP
Usage Evaluation EMM OSINT May 2015
3.1.2.1
Suite
WP
Dissemination EMM OSINT Suite
June 2015 - Oct 2016
3.1.2.2
WP
On-site user training EMM OSINT on request
3.1.2.3
Suite
Improve and sustain EMM OSINT Suite operational use
WP
Operational Updates EMM OSINT Nov 2014 – Oct 2016
Limited
3
DELIVERED WORK PACKAGES
WP 3.1.1.1 OSINT Workshop for MS practitioners
Organise three workshops for law enforcement practitioners about the use of open source
information and its practical application.
Deliverables:
Three workshops for MS law enforcement authorities
Supporting event web site with online workshop material
Workshop 2014
The first workshop for OSINT practitioners was organised by the JRC in November
2014. The workshop was attended by 50 participants2. The workshop featured hands-on
trainings both for EMM server and EMM OSINT Suite.
Workshop 2015
The second EMM OSINT workshop took place in November 2015. The workshop was
attended by 46 participants. The workshop featured hands-on trainings both for EMM
server and EMM OSINT Suite and the following presentations by experts from the
Member States:
"Potential online recruitment through connections made between members of
Romanian Muslims and immigration/ refugees groups” Romanian Intelligence Service
Social media can help to identify connections between potential terrorist groups. Key
influencers in these groups can be identified and monitored.
"United Nations Office on Drugs and Crime – Global Product Portfolio " United Nations Office on Drugs and Crime
The UNODOC has developed a portfolio of software products which are used to counter
money laundering as well as improve the process of public tendering. The products are in
worldwide use by financial crime units and governments.
"Monitoring Social Media for Customs" Customs National Office, The Netherlands
The talk discussed different ways to use information from social media for customs use.
This comprises cargo movements broadcasted through Twitter as well as detecting of
counterfeit products on blogs and personal sites.
2 Please refer to the First Interim Report for the participant list and more details.
Limited
link to page 20
"EMM OSINT Suite in Education",
University of Applied Science (Hogeschool) Leiden, The Netherlands
The EMM OSINT Suite is used in a curriculum for a forensics degree in Leiden. The talk
gave an overview on how the software can be used for teaching basic OSINT skills.
WP 3.1.1.2: Workshop Evaluation
Evaluate workshop to improve subsequent events based on feedback of attendants.
Deliverables:
On or offline survey to evaluate workshop results
Evaluation of survey and results summary
Changes to next event format
After each workshop we conducted a feedback survey by sending out an assessment form
or using the Commission’s online survey tool. The form and the results are available in
Annex
6.1.2.
WP 3.1.2.3: On-site user training for existing and new users
Provide on-site trainings for new and existing users of the EMM OSINT Suite. This work
package can be requested by MS authorities to learn how to use the software or how to
adapt it to special use case scenarios.
Deliverables:
Updated and customised training material
On-site training delivery
The following on-site trainings for 2016 have been requested by MS:
Tax Authority and Financial Police, The Netherlands
National Crime Agency, United Kingdom
Hogeschool Leiden in collaboration with Dutch Authorities, The Netherlands
WP 3.1.3.1: Operational updates EMM OSINT Suite
Provide a quarterly release to keep the software current.
Deliverables:
Quarterly bug fix release
Release notes and announcement mail
In 2015 we provided four operational updates for the EMM OSINT Suite (2.3.2-2.3.6).
We plan the next major update for the first quarter of 2016.
Limited
WP 3.2.1.1 On-site operation support with JRC trained experts
The JRC provides on-site experts in order to support on-going operations and to facilitate
the transfer of technical knowledge. The experts are based mainly in Brussels but have
the opportunity to work in Ispra at specific intervals to get updated with the latest
technical knowledge.
Deliverables (on-site):
Operations support to keep EMM tools running
Development of new categories, refinement of existing categories for DG HOME’s
areas of interest
Production of open source daily products
Reducing operational risks by proving latest expertise on the use of EMM tools
Facilitate understanding of the system
Deliverables (Ispra):
Training of the experts in the latest EMM products and developments
Remote support of the experts
The on-site support commenced continuing operation from a previous arrangement as
planned in January 2015. The on-site support concluded end of 2015 and will be
continued by different AA in 2016.
WP 3.3.1 Inception Report
Submit inception report within one week after kick-off meeting, describing project plan
for the first 6 months, team composition and further results of kick-off meeting.
The inception report incorporating the kick-off meeting was delivered in January 2015.
WP 3.3.2 First Interim Report
The first interim report was delivered in April 2015.
WP 3.3.3 Second Interim Report
In order to include the results of the Usage evaluation of EMM OSINT Suite, the second
interim report will be delivered in early 2016, later than originally planned.
WP3.1.2.1 Usage evaluation of EMM OSINT Suite (“OSINT Survey”)
Conduct a user survey to evaluate the use of the EMM OSINT Suite tool.
Deliverables:
On- or offline survey to evaluate software use
Limited
Evaluation of survey and results summary
The JRC conducted an online survey (using the Commission’s EUSurvey web tool) with
users of the EMM OSINT Suite.
Due to resource constraints the survey was delayed to the end of 2015. To validate the
survey questionnaire a pilot survey with 15 law enforcement officers in November. The
full survey was conducted with 270 users of the JRC’s online community in December.
The JRC sent out 270 invitations to take part and received 76 answers (28%).
The survey was composed of a number of sections to gather information about the
knowledge of the participants, the use of OSINT and our provided tools and feedback on
the actual tool. Finally, we asked for input on how to improve the dissemination of the
tool.
Survey Results
Background of Participants
The first few questions tried to assess the background of the participants, such as the
organisational background and the knowledge of OSINT as a tool.
In what type of organisation are you
working?
25
21
19
20
18
15
15
10
5
0
Customs
Police
Tax Authority
Other
Other organisations mentioned:
Economic Crime Authority
Intelligence
Competition Authority
Combined task force police and customs
University
Limited
Software lacks crucial features (2 participants)
Other reasons (14 participants)
o Missing hardware requirements
o Tested old version needs to evaluate latest version
o Moved job
Again this gives a strong indication that training and documentation is not sufficient.
Assessment of the EMM OSINT Suite software
We asked the participants to tell us strong and weak points of the software.
Strong points
Tool is free and free tools are rare for law enforcement
Extracting from PDFs
Built from the ground up for law enforcement
Software is working solid
Good for student education
Entity extraction and especially custom entities based on regular expression
Weak points
Problems with large data sets
Hard to learn “high threshold” to use
Limited access to sites with password protection
Difficult to add other search engines
Limited language support
Results Summary and Conclusions
The majority of users is using open source information regularly and is proficient in
doing so. The latest version of the EMM OSINT Suite tool has been tried by a majority
of users but is used operationally only by 44%. Only 41% have taken part in training.
In order to improve the operational use the JRC needs to improve training for the tool.
Also measures to better dissemination the software are needed.
Limited
4
PROJECT PLAN FOR THE NEXT MONTHS
The following work packages will be delivered within the next months
WP 3.1.1.1: EMM OSINT Workshop - the 2016 workshop is scheduled for 21.-
23.09.2016
WP 3.1.2.2: EMM OSINT Suite Dissemination
Based on the results of the survey, we are planning the following steps:
Create dissemination package (brochure, quick start guide, evaluation download,
get-started videos)
Send package to all survey participants who asked for more information
Send package to further law enforcement contacts
Additional on-site training sessions in Ispra
WP 3.1.3.2: OSINT Improvements
The survey provided a lot of feedback on how to improve the software. We are planning
the following changes with the focus on usability improvements and improvements of
documentation and training
Improve user interface to be more consistent and responsive
Make modules with importance for law enforcement more accessible:
o Easier definition of custom entity types
o Better integration into existing law enforcement tool chain
Improve documentation
o Videos to get started
o Consistency
o Focus on use cases
WP 3.1.3.1: The EMM OSINT Suite software will receive further operational updates,
such as security patches to stay current.
5
RISKS AND CHALLENGES
The execution of several milestones faces certain risks and challenges.
We have identified the following risks:
Development of software tools not meeting user requirements
Limited
Software tools not meeting minimal quality standards
Lack of personnel resources
To mitigate the risks the following measures will be taken:
Increase communication and collaboration with MS authorities and community
through website and dedicated mailing list
Software will be tailored for law enforcement purposes to cover domain specific
requirements
Incremental software delivery process which allows changes to user requirements all
along the way
Systematic testing of tools and deliverables to ensure decent quality
Timely reporting and discussion of activities between JRC and DG HOME to ensure
effort is directed towards joint goals
Limited
(2) (b)
Presentation style was suitable for the 4,60
4,1
event
(2) (c)
Speakers covered all expected topics
4,33
3,5
(2) (d)
Time for questions and discussion was 4,73
4,4
sufficient
Documentation Material covered all topics
4,40
3,6
(3) (a)
(3) (b)
Structure and layout was appropriate (i.e. 4,47
3,8
easy to read, understandable, etc.)
Facilities and Presentation aids and room equipment 4,73
4,1
Services
were appropriate
(4) (a)
(4) (b)
Facilities (incl. room layout, group 4,67
4,5
meeting rooms, etc.) were functional to
the purpose
(4) (c)
Technical support staff was available 4,80
4,4
when needed
Before
the Necessary information and enough 3,73
4
Event
support were available
(5) (a)
Overall
My expectations about the event were 4,60
3,5
Outcome
met
(6) (a)
Limited
Electronically signed on 09/03/2016 17:15 (UTC+01) in accordance with article 4.2 (Validity of electronic documents) of Commission Decision 2004/563