Our intelligent web crawler uses pioneering technology to scan web pages on the internet searching out images and videos showing the sexual abuse of children so our analysts can work to have them removed. Since 2021 it crawled over 19 million webpages, and over 91 million images.
Our Hotline assessed and took action on over 255,000 webpages in 2022 containing images or videos of child sexual abuse – each page could contain hundreds or thousands of images and videos. The majority of those reports were found by proactive searching by our analysts.
One of the essential tech tools helping our analysts find these webpages is the IWF Crawler. Utilising our child sexual abuse imagery Hash Database, the IWF Crawler finds images matching the digital fingerprints (or hashes) of images our analysts have already assessed and categorised as criminal. By comparing each image our crawler finds to the hashes of known child sexual abuse material, it means we can find duplicate child sexual abuse images hidden across the internet and eliminate them.
We use our crawler as one operational tactic in a suite of tools designed to find, remove, and disrupt the availability of child sexual abuse material online. It’s deployed in a considered and targeted manner in order to be most effective. In addition, it reduces unnecessary exposure to child sexual abuse imagery for our analysts, protecting their welfare and well-being.
Our crawler also allows us to provide an additional service to our Members in the domain sector registry sector.
