Pre-crime software for border guards
Designed for use by border guards, Unisys' LineSight software uses advanced data analytics and machine learning to help border guards decide whether to inspect travellers more closely before admitting them into their country. Unisys says the software assesses each traveller's risk beginning with the initial intent to travel and refines its assessment as more information becomes available at each stage of the journey - visa application, reservation, ticket purchase, seat selection, check-in, and arrival. The underlying assumptions are that the "initial intent to travel" can be detected, and that the software can accurately predict intentions and future actions. Unisys claims LineSight can learn from experience and automatically generate new rules and algorithms to continuously improve its accuracy.
The company boasts of providing this software to governments around the world, including the US and Australia, and says it supports the European Commission and new EU member states with the Schengen Information System. Critics argue that decisions about which travellers should be searched more intrusively should be made on the basis of probable case, not prediction software, particularly when that software is free to make up its own rules over time.
Publication: Papers Please
Writer: Edward Hasbrouck
Personalisation, persuasion, decisions and manipulation
The data that is observed, derived or predicted from our behaviour is increasingly used to automatically rank, score, and evaluate people. These derived or inferred data are increasingly used to make consequential decisions through ever more advanced processing techniques. In the future people will be scored in all aspects of their lives, societies will be managed invisibly, and human behaviour will be under the control of the few and the powerful.
Profiling makes it possible for highly sensitive details to be inferred or predicted from seemingly uninteresting data, producing derived, inferred or predicted data about people. As a result, it is possible to gain insight into someone’s presumed interests, identities, attributes or qualities without their knowledge or participation.
Such detailed and comprehensive profiles may or may not be accurate or fair. However, increasingly such profiles are being used to make or inform consequential decisions, from finance to policing, to the news users are exposed to or the advertisement they see. These decisions can be taken with varying degrees of human intervention and automation.
In increasingly connected spaces, our presumed interests and identities also shape the world around us. Real-time personalisation gears information towards an individual’s presumed interests. Such automated decisions can even be based on someone’s predicted vulnerability to persuasion or their inferred purchasing power.
Automated decisions about individuals or the environment they are exposed to offer unprecedented capabilities to nudge, modify or manipulate behaviour. They also run risk of creating novel forms of discrimination or unfairness. Since these systems are often highly complex, proprietary and opaque, it can be difficult for people to know where they stand or how to seek redress.
We may challenge consequential decisions
Individuals should be able to know about, understand, question and challenge consequential decisions that are made about them and their environment. This means that controllers too should have an insight into and control over this processing.