Request Quote

Log in to evaluate

Access downloads, trial keys, docs and resources.

Proven platforms that deliver Rapid time to market for new services

Distributed web filtering for Internet Service Providers
Scalable and effective CPE based web filtering for Internet Service Providers

Through legislation such as "Active Choice" or "Default On", many ISP's are faced with the challenge of providing a scalable and effective web filtering service for all of their subscribers. We believe the best way to do this is to distribute the task to the Customer Premises Equipment and we can show you how with our robust ICAP server implementation and an example router running customised OpenWRT firmware.

The idea

There are many possible solutions for providing filtering at an ISP level however most if not all have serious disadvantages; proxying all web traffic has a huge performance cost, DNS based filtering is easily circumvented, handling HTTPS traffic effectively is almost impossible and has severe privacy concerns for the customer and so on. For many years we have advised ISP's on the pitfalls of implementing centralised web filtering however with new legislation on the way it is clear that ISP's will have to offer at least some level of protection against illegal images and pornography by default. The customer should have the right to opt-out of legal pornography filtering and as such the ISP will need to manage the choice on a customer by customer basis.

Having considered the problem, we have embarked on a research project to put forward an effective solution for Internet Service Providers to consider. Our solution is to offload the actual traffic interception to the Customer Premises Equipment, or rather, the xDSL router. The router, usually in the home or office, is the best place to intercept web traffic and take instruction from a central service as to whether to allow or deny particular URL's (not just domain names or IP addresses). By intercepting at the router level, it is much harder to circumvent the filtering and it also distributes the computing power needed to intercept the traffic away from the ISP's core infrastructure.

Proof of Concept

To prove the concept, we have used the popular open-source OpenWRT router firmware. OpenWRT, and its sister project, DD-WRT, provide fully functional router firmware and in the case of the latter has been adopted by big brands such as Buffalo on their newer devices. We have customised the firmware and implemented Squid 3.3 as a transparent proxy with a custom ICAP client that can hook into our existing cloud web filtering platform. That is the customer end complete. Once the router is installed in the home, all web traffic will be transparently intercepted and the ICAP client will check each request with the cloud service to determine if it is acceptable based on the policy for the configured account.

At the ISP end, in our proof of concept we act as the ISP by operating the cloud platform, there exists a cluster of our robust ICAP servers. The ICAP servers take care of the millions of requests from the ICAP clients, quickly and efficiently deciding if access to the requested URL should be allowed or blocked. The speed of the ICAP protocol, which utilises very small packets, means that latency will not be an issue for the end customer and little bandwidth is required to service the requests from the ISP.

The proof of concept architecture is effectively as follows:


This solution has a number of advantages:

  • Performance - the traffic interception happens at the CPE.
  • Latency - the ICAP protocol is lightweight and as the ICAP servers are only a couple of hops away it is very fast.
  • Bandwidth - the ICAP protocol uses minimal bandwidth.
  • Filtering -  filter full URL's (domain/path) rather than just domain or IP address. Enforce search engine Safe Search easily. HTTPS can be achieved with MITM or SNI approach.
  • Scalable - easily scalable with load balanced ICAP servers, each handling hundreds of thousands of requests.
  • Cost effective - simple billing model based on traffic filtered.
  • Integration - RADIUS authentication, LDAP authentication and a RESTful API for controlling the policy engine.


In reality - bespoke

In reality there will be some, probably smaller, ISP's who either already use an OpenWRT based router or could swap to using one without too much overhead. We understand however that larger ISP's will already have relationships with CPE equipment manufacturers and bespoke firmware. For those service providers, we fully expect to work with the CPE manufactuer to implement a similar system that we have done using OpenWRT. We can provide consultancy, binaries and source code to help with this process of integrating the CPE into our filtering solution.


>>> Click here to contact us for more information