The Think Family Database, implemented in 2016 by the Bristol City Council and Avon and Somerset Police, contains sensitive records on nearly half a million individuals from Bristol, England. This extensive data includes police intelligence, mental health records, housing status, and other personal information. The goal was to utilize machine-learning models to create risk assessments related to threats and vulnerabilities within the community. A police data scientist described the process of generating these risk scores as simply aggregating all available data to "come out with a lovely risk score for everybody."
This effort is part of a larger initiative by the Avon and Somerset Police that includes over 23 predictive models to assess various risks, such as burglary and domestic abuse. However, the transparency surrounding these models has been questioned, particularly by local advocates like John Pegram, who discovered he was included in the Offender Management App years after its creation. Pegram filed a request for information on how his data was used but faced resistance from the police.
A recent investigation, supported by public records requests, reveals that at least two risk models were abandoned due to distrust in their reliability. Documents point to a lack of transparency and poor predictive accuracy in the systems, with one independent analyst noting that the risk assessments often produced unsatisfactory results.
This push towards integrating predictive analytics within the UK police is spearheaded by former Avon and Somerset Chief Constable Andy Marsh, now leading the College of Policing. His organization is evaluating numerous AI tools for deployment across England and Wales. The push comes after the police faced budget cuts and various operational pressures, with predictive analytics viewed as a potential solution to enhance crime prevention.
While the Insight Bristol team sought to compile useful data for supporting vulnerable families from different agencies, they did not obtain consent from those whose data was being used. Ethical considerations surrounding data use have emerged, particularly concerning bias and accuracy in assessing risk. Officials later acknowledged the need for greater transparency and the involvement of the public in these data-driven initiatives.
Over time, complaints emerged about the inadequacy of the risk models, framing them as telling staff what they already knew about at-risk children. User experiences further revealed that crucial data may have been overlooked due to the reliance on flawed algorithms.
Concerns about bias have been voiced by researchers and advocates alike, leading to calls for a reevaluation of predictive tools used by law enforcement in Bristol. The shift away from effective data sources to less comprehensive inputs has led to significant drops in identified vulnerabilities within the community.
While Avon and Somerset Police assert that they are reviewing their analytical practices and involving ethics teams, critics highlight a concerning lack of accountability and documentation related to how these systems are used. Following the police’s decision to withdraw certain models, internal confusion persists regarding the tools being deployed, leaving many uncertain about their efficacy.
Currently, predictive analytics remains in operation within the Avon and Somerset region, including models assessing child welfare. Critics like Pegram argue that data-driven systems should not dictate individuals’ lives, calling for a halt to the program and emphasizing the need for meaningful public participation and oversight in these technological implementations.
Recently, the UK government announced the establishment of PoliceAI to facilitate the rollout of advanced AI policing tools, posing significant implications for the future of law enforcement across England and Wales. As the debate around the ethical use of predictive analytics continues, concerns over privacy, accuracy, and accountability remain at the forefront of public discussion.
