Ref. Ares(2022)7025147 - 11/10/2022
Algorithmic
Transparency
Report
2022
Wolt’s Algorithmic Transparency Report
I. About
Wolt
1
II.
Why
write
this
Report?
2
III. Our principles in building our products
2
IV. Algorithms
powering
Wolt 3
A:
Courier Partners – Task allocation algorithm
Non-discrimination
Human
oversight
Risk
management
B: Customers – Ranking & Personalisation algorithms
7
Non-discrimination
Human
oversight
Risk
management
V. Help us learn!
11
I. About Wolt
Wolt is a Finnish technology company that operates a food, grocery, and retail
intermediation and delivery marketplace. We currently operate in 23 countries and over
250 cities, serving millions of customers together with our merchants and courier partners.
Wolt was founded in 2014 in Helsinki where our headquarter lies today, and we employ
more than 6 000 people in our offices across our markets.
By operating a technology platform, Wolt brings convenience to customers, but also
creates economic opportunities for local restaurants and retailers selling their goods
through our platform, and for independent couriers providing services to Wolt. Simply put,
when buying something on our platform, our customers can choose to have the products
delivered to them. If they do, we then offer that delivery task to a courier partner who is
logged onto our platform.
1
II. Why write this Report?
We are committed to making the cities we operate
in better places to live. We are also committed to do
the right thing towards our teammates, partners,
customers, and the general public. This is the basis
for everything we do, so we figured we might as well
be transparent about what we do and how we do
it. That way, others are able to evaluate if we are
indeed doing the right thing.
III. Our principles in building our products
We take the concerns around algorithmic
transparency seriously. We have advocated for the
explainability of the key operating principles behind
Fairness
technology. Recently, the European Commission
We are focusing on building a platform that ensures
put forward a draft declaration on digital rights and
fair treatment of the people that interact with it.
principles for everyone in the EU1. According to the
There shall be no direct or indirect discrimination
declaration:
based on personal attributes, such as racial or
ethnic origin, sex, religion or belief, disability, age or
Everyone should be empowered to
sexual orientation.
benefit from the advantages of artificial
intelligence by making their own, informed
Meaningful impact
choices in the digital environment, while
We are building a product organisation that works
being protected against risks and harm
at scale with autonomy, all the while optimising for
to one’s health, safety and fundamental
meaningful impact. Impact is the biggest driver
rights.
of motivation for the product team, be it from
business metrics, societal, environmental influence
This makes a lot of sense. The same goes for the
or personal growth journeys. At the core of this
commitments outlined in the declaration. They are
approach is a focus on the value we create for our
in line with our thinking. For example:
customers, partners and society at large.
We commit to ensuring transparency
about the use of algorithms and
Transparency
artificial intelligence, and that people
We acknowledge that this industry is new and
are empowered and informed when
intricate. By providing accessible information about
interacting with them.
how it works, like we do in this Report, we hope that
it will build trust with our partners and customers,
Finally, we wanted to write this Report so that we
as well as the general public.
can learn. This is our first one, and we are sure there
is a lot we can improve. We want to get feedback.
We want to work with others.
Technology is not an end in itself. For us, it is a means
to make the cities we operate in better places to
live. We think technology can help in achieving
this goal and hope that through transparency, the
positive role of algorithms and platforms can be
understood.
1 https:/ ec.europa.eu/commission/presscorner/detail/en/IP_22_452
2
IV. Algorithms powering Wolt
We include vehicle type as a factor as it impacts
both the delivery speed and delivery capacity. As
As a digital platform, algorithms are a vital part
Wolt has expanded into enabling the deliveries of all
of our work and facilitate millions of decisions
sorts of things beyond restaurant food – including
everyday. Yet, today, in the digital age, many people
groceries, convenience items, and for example,
associate algorithms with concern due to their non-
flowers – the physical size of an order can vary a lot.
transparency. In this section, we aim to dispel that
concern by increasing the broader understanding of
Before a courier partner can be offered tasks, they
how algorithms are – or sometimes are not – used
need to go through the onboarding process, where
at Wolt. We hopefully provide enough context on
Wolt checks e.g. the individual’s right to work, and
what happens as part of a delivery order to give you
to download the Wolt Courier Partner app to their
a good understanding of the whole process.
device (smartphone/tablet).
We have drawn inspiration from the UK
A delivery task offer includes information on the
Government’s Algorithmic transparency template
pick-up location (i.e. restaurant or store) and drop-
published by the Central Digital and Data Office2,
off location (i.e. customer’s home, office, or any
and also the City of Amsterdam’s Algorithm register
other location) and is always offered to only one
for explaining the algorithms they have in use3. The
courier partner at a time, so there is no competition
following sections are based on the situation at
on who accepts a task first or fastest.
Wolt in February 2022 and how we partner with self-
employed couriers, i.e. courier partners, as they
Once a task has been offered to a courier partner,
form the majority of couriers on our platform.
they can decide either to accept or reject the task.
If a task is rejected it will automatically be offered
to the next potential courier partner based on
A: Courier Partners -
proximity, vehicle type and availability. The same
applies with the automatic rejection of a task which
Task allocation algorithm
happens after 60 seconds of no response from
the courier partner. If no one accepts a task, Wolt
Task allocation algorithm
cancels the customer’s order without any negative
One of the most basic functions of a delivery
impact or consequence for the courier partner.
platform is to offer delivery tasks to couriers so that
they can either accept or reject offered tasks, and in
case of accepting, transport an order from point A
to point B. The task allocation algorithm aims to find
a courier partner who can deliver the order without
it being terribly late. Therefore, the algorithm in
general tries to minimise the kilometers to be driven
and time for all deliveries with the constraint of
ensuring all orders are delivered within a target
time. The more efficient the algorithm is, especially
with regards to grouping orders, the more courier
partners can also earn.
Wolt’s algorithm offers delivery requests, ‘tasks’,
equally to courier partners based only on their
proximity to the pick-up location (i.e. restaurant
or merchant), the vehicle type, and their current
activity status (available for deliveries, busy with
ongoing deliveries or offline). The platform does not
use any kind of ranking or rating to determine which
courier partner is offered a delivery task, only the
Courier partners are not obliged to accept offered
location and the type of vehicle of courier partners
delivery tasks and they can even change their
who have chosen to log online are used.
mind after accepting a task, i.e. they can ask to be
unassigned from a task they originally accepted.
2 https://www.gov.uk/government/collections/algorithmic-transparency-standard
3 https:/ algoritmeregister.amsterdam.nl/en/ai-register/
3
All courier partners are anonymised in connection
proposed tasks at any given time and request to be
to the automated task allocation and a courier
unassigned from a task after accepting it.
partner’s identity or e.g. possible task rejections,
does not affect how the delivery tasks are offered.
Delivery route
There will be no adverse consequences for the
Once a courier partner has accepted a task, they are
courier partners for rejecting tasks or for asking to
free to choose the delivery route. Wolt does not pro-
be unassigned from a task.
vide any driving instructions for how to travel neither
to the pick-up location nor the drop-off location.
Delivery modes
Many partners use an external map service such as
If a courier partner has marked themself available
Google Maps, Waze or Apple Maps for this.
to be offered delivery tasks in the Courier Partner
app, they will be offered tasks within their proximity
When the courier partner has picked up the order at
and fit for their vehicle type. Courier partners
the merchant they are asked to confirm in the app
can choose between two different modes to
that the correct order has been collected. Wolt then
perform deliveries, they can either accept tasks
provides the customer with a delivery time estimate
in a so-called ‘Single task mode’, delivering one
automatically based on a number of factors. These
order at a time, or ’Bundled tasks mode’, delivering
include historical data on the duration of deliveries,
combined or multiple tasks at the same time.
the delivery distance, traffic, time of the day, and ve-
hicle type within a specific area. The estimate is not
In Bundled tasks mode, the app may offer the
binding and does not oblige the courier partner to
courier partner additional tasks before an earlier
deliver the order within that time frame either. There
task has been delivered to the customer, if there
will also not be any punishment or any other adver-
would be additional tasks available for pick-up
se impact on the courier partner for not meeting the
and drop-off in the same general direction as
estimated delivery time.
the first task. The benefit of bundled tasks is
that it is more efficient overall, and for courier
The customer can follow the delivery in real-time on
partners they enable higher earnings as they are
a map included in the Wolt app, so that the custo-
paid for each task as if they were individual tasks.
mer is ready at the drop-off destination when the
delivery arrives. When the courier partner has deli-
In both Single task mode and Bundled tasks mode
vered the order to the customer, the partner marks
courier partners can always choose to reject
the delivery as completed in the app.
4
Earnings
Courier partners are paid per each completed de-
livery task. The fee per task consists of two compo-
nents, the fixed ‘Task Fee’ and the flexible ‘Distance
Fee’. The Task Fee includes the distance from the
courier partner’s location to the pick-up point of
the order and the minimum drop-off distance tow-
ards the customer. The Distance Fee is paid out in
addition to the Task Fee for long drop-off distan-
ces. The longer the additional drop-off distance,
the higher will the Distance Fee be that the courier
partner receives.
Courier partners with a car or van can receive large
orders, i.e. orders that contain many items and are
ordered by groups. As large orders may take longer
to deliver, courier partners earn extra from those
orders.
In case of bundled tasks orders, the courier part-
ner earns a fee per each task delivered, even if they
deliver multiple tasks from the same merchant to
customers in the same building.
Courier partners can at all times view their accumu-
lated earnings in the Courier Partner app.
Non-discrimination
As already described above, all courier partners are
anonymised in connection to the automated task
comes up during the delivery. Common questions
allocation and a courier partner’s identity does not
that the Wolt Support gets include the accuracy of
affect how the deliveries are offered. A delivery task
the delivery estimate and missing or faulty items of
is always offered to only one courier partner at a
the order. Drop-off is another part of the delivery
time, so there is no competition on who accepts a
experience where human intervention is someti-
task first or fastest.
mes necessary – be it due to missing address details
or other difficulties in navigating the last 50 meters.
We also review the balancing on vehicle types regu-
larly to ensure the courier’s choice of vehicle only
Risk management
matters when it makes a crucial difference for the
As location data is the most important criteria to the
delivery based on the delivery distance or the size
algorithm that offers tasks to courier partners, this
of the ordered product.
can lead to variance between couriers using diffe-
rent devices with different geolocation accuracy.
Human oversight
It is in every party’s interest that the system is as
The algorithm is being continuously maintained
accurate as possible as this influences the accura-
and updated to ensure it works as desired, whilst
cy of the offered tasks. We also have technical and
also making the algorithm faster in offering tasks to
organisational measures in place to ensure proper
couriers. We also look to maximise courier earnings
level of privacy and security compliance with regard
by developing the algorithm to offer more bundled
to geolocation data, including regular review of the
tasks when possible. This work is fueled by aggre-
necessity, granularity and retention of such data.
gated data from actual deliveries and operational
considerations.
Courier partners move in traffic and use smart-
phones. This includes inherent risks, such as
Operationally, the Wolt Support can help courier
looking at your device for navigation informati-
partners, merchants, and customers if anything
on while being on the move. Thus, new tasks are
5
only offered when a courier is not already on a task.
Online hours
This way, they can accept the next task, for examp-
Self-employed courier partners can connect them-
le, only after they have dropped off a delivery to a
selves to Wolt’s Courier Partner app and offer their
customer. This limits the situations where the app
services whenever they want, without any obligati-
sends a notification that the courier partner might
on to work. This allows them to have maximum fle-
want to react to. If the courier partner is in a situa-
xibility and log online or offline on the Wolt Courier
tion where it is riskier for them to interact with their
Partner app as they like. They can even choose to be
device, they do not have to do anything; the task
online without any intention to perform any delive-
is declined after the 60 second timer and courier
ries, and can work for and with other companies, in-
partners are not penalised in any way for not ac-
cluding our competitors at the same time as for us.
cepting or declining tasks.
Courier partners do not need to inform Wolt in ad-
Performance
vance if or when they intend to not perform delive-
Wolt does not use data such as rejection rates,
ries for any period of time. In practice, as opposed
amount of performed deliveries, speed of delive-
to an employee, courier partners choose at their
ries etc. on courier partners. Rejecting a delivery
discretion when to take a break or whether to remain
request does not have any impact whatsoever on
offline for a longer period without needing to inform
courier partners’ possibility to get future delivery
Wolt or receive Wolt’s approval.
requests offered through Wolt’s platform, or their
contractual partnership with us, nor does it result in
Wolt does not apply any algorithms to monitor on-
any disciplinary measures.
line hours of the individual courier partners and let
that impact the contractual partnership or task-al-
There are no ratings on courier partners. After a
location. When we analyse online hours it is on an
completed delivery, customers rate the delivery
aggregated and anonymised basis to better under-
experience as a whole. Thus, when giving this rating,
stand our operations.
the customer signals their opinion of the various
aspects that form the delivery, for example, the de-
A recent pan-European study by Copenhagen Eco-
livery time estimates generated by Wolt, the packa-
nomics found that a majority of couriers (72%) say
ging of the goods and so on. The rating customers
platform work is a complementary activity, with 34%
give on the delivery experience is only a tool for our
delivering while studying and another third (34%)
Support Team to know whether there might be an
saying they access platform work to top income
issue with an order and if they should reach out to
from other full or part-time work4. At Wolt, around
the customer.
40% of courier partners delivering via our platform
work less than 7,5 hours per week on average.
4 https:/ copenhageneconomics.com/publication/study-of-the-value-of-flexi- 6
ble-work-for-local-delivery-couriers/
B: Customers – Ranking & Personalisation
algorithms
The purpose of providing personalisation is to
let customers discover different restaurants and
stores and help them find what they need, be it
chocolate chip cookies to feed a sudden crave,
new fuses to get the lights back on, a surprise flower
bouquet, or their weekly groceries. Ranking and
personalisation on the Wolt platform happens in
four main sections on the app and website:
1. Discovery page
2. Restaurants page
3. Stores page
4. Search page
1. Discovery page
The Discovery page is the front page of Wolt and
the first thing users see when they open the app or
website. The ranking of content on the Discovery
page uses a mix of manual and automated sorting.
For example, ad hoc promotions and campaigns
are manually added to the page by our marketing
teams.
We divide venues on the Discovery page into
categories and carousels using automatic tools.
Category Carousels: We tag venues into different
Venue Carousels: Since we have a lot of different
types of categories, such as ‘pizza’, ‘burgers’, ‘salad’
categories, some venues may show up in multiple
etc. The order of these categories is automatically
carousels. We want to show the customer venues
determined by assessing the popularity of each
that are relevant, therefore the venues on the
category by aggregating previous orders.
Discovery page are ranked using the First-Time
User or Returning User Model explained below. A
venue that has been ranked high will be given a more
prominent spot.
7
First-Time User Model
Returning User Model
If the customer is an unregistered user or a registered
The second tool we use in ranking content is
user with no purchases, the sorting is done through
the Returning User Model. This model is applied
a simple static model. The model factors in three
to registered users who have made at least one
criteria. The first is the delivery estimate, i.e. the
purchase with Wolt. It can be explained as a way of
time it would take for the customer to receive the
recommending the user something based on the
order after completing their purchase. The second
purchase history of other similar customers. The
criteria look at the venue relevancy and popularity
ranking looks at past purchases of an anonymised
based on how many purchases other customers
user without identifying the user personally and
have made with the venue. This criteria also factors
recommends venues based on purchase history.
in venues that are new on the platform to make sure
For example, if an anonymised Customer A
they are also promoted initially. The third factor is
purchases something from Venue 1 and Venue
the venue price range that looks at the price of
2, we look at other anonymised customers who
menu items in the restaurants and tries to promote
have ordered from the same venues. If the other
a wide range of venues with different prices to
customers have purchased something from Venue
customers.
3 we may recommend that venue to Customer A.
The First-Time User Model works with aggregated
Personalisation algorithms are trained every
data. For example, two first-time users opening
night. If you make a purchase during the day the
the Wolt app in the same location will see the
personalisation will be applied the following day.
same venues ranked in the same way and no
If you are a first-time user and make your first
personalisation is applied.
purchase, the next day the user experience may
change since it is now based on the Returning User
Model.
8
2. Restaurants page
The Restaurants page features restaurant venues
that are delivering to the customer and are sorted
by their availability, meaning if they are open and
accepting orders they will be displayed before
closed venues. The available venues are then
re-sorted using the First-Time User Model and
Returning User Model.
3. Stores page
In our Stores page we display the selection of retail
and grocery stores available to the customer. We
apply a different kind of sorting algorithm for our
Stores page as it works better for the customer
experience.
Stores Page Model
This model looks at three different features for
ranking content on the Stores page.
• Popularity. This criteria also factors in venues that
are new to the platform to make sure they are also
surfaced to the customers.
• Customer rating
• Likelihood of customer placing an order
The algorithm assigns each venue a rank according
to all three criteria separately. Then it combines
all rankings and re-sorts the venues based on the
result. Thus we ensure that the venues that are
displayed on the top of the page receive the highest
ratings of all three criteria.
9
4. Search page
If users look for a particular type of restaurant,
store or item, they can use the search feature
to find that. At Wolt, we have two main types of
search functions, the Global search and the In-
venue search. For both search functions we use
the customer’s typed query which includes any
characters regardless of the language and factoring
in synonyms. In the case of Global search, we also
use the user’s delivery address, if the user has
provided it to us. If the address is not specified, we
use their current location if they have granted us
their permission for this.
Global search
The Global search helps users search the platform
for venues that match the users’ criteria and is
within the delivery area of the customer.
When users type their search in the search bar,
the search request is sent to a back-end service,
which factors in both the typed search text and
the user’s location to build a search query. That
search query is then matched against an existing
index of metadata containing information from the
merchants and their menu or product items. This
produces a first list of results of venues that are
corresponding to the search query.
The search results are sorted by an algorithm
determining the relevancy of the venue as well as if
the venue is open or not. For example, venues that
match the search query, but are not open, will be
displayed at the bottom of the list.
In-venue search
The second type of search feature is the In-venue search. When a user is looking at a particular restaurant or
store they can search that venue for specific items. This search function works differently on the web and in the
app in order to improve the user experience as some stores might include hundreds of items.
Web
Mobile
When a user types a search query within a specific
When a user types in a search query within a specific
venue’s page on the website, a filtering is applied
venue’s page on the mobile app, a search request
to hide the items that do not match the search
is sent to a back-end service. The back-end tool
query. The filtering is based on the menu item title,
matches the search query against an index based
description and category.
on the venue’s metadata, sorts it and returns the
products as search results to the users.
10
Non-discrimination
Risk management
Wolt’s ranking and personalisation algorithms
Similarly to task allocation, location unsurprisingly
do not process or use personal information that
plays a role also on the customer side. Customers
would have been identified as potential sources
see venue options that are nearby based on their
of discrimination against people impacted by
approximate or city-level location. Customers also
the automated process. In fact, we do not collect
may choose to add a delivery address to the Wolt
any data on gender, sexual orientation, racial or
app in order to receive their order to the location
ethnic origin, national origin, religious affiliation or
of their choice. Inaccuracies in location data could
disability.
impact the selection of venues the customer sees,
although this is more theoretical given we use less
When training the algorithmic tools, we only use
granular approximate data, and there are clear ways
aggregated and anonymised data.
the customer can ensure that their desired delivery
location is used.
Human oversight
We mitigate and balance the algorithms to also
The ranking and personalisation algorithms
surface new restaurants and stores to customers.
are in continuous development. As described,
This balancing is phased out after a while.
specific sections in the app and website, such as
promotions, are manually created.
V. Help us learn!
Thank you for reading our first Report on Algorithmic Transparency. We made this Report partly
in order to learn. We want to understand what was interesting, where we should have explained
our algorithms better, and any other perspectives you think we should have covered. On our
side, we continue to improve our platform and the algorithms that make it tick. If you have ideas
that could help along the way, let’s chat!
You can get in touch with us via a new, dedicated point of contact for all things algorithms and
transparency: xxxxxxxxxxxx@xxxx.xxx. The alias is managed by Wolt’s Public Policy team, who
will liaise internally to gather the right people for questions, feedback and other input you might
be interested in sharing with us.
11
http://www.explore.wolt.com/transparency