Web Crawler

A web crawler was designed together with one of the largest DIY chains. The chain needs information about competitors, suppliers and their products. The observation of the markets and the competition is an important basis to stabilize and permanently improve own results. For purchasing and online retailing in particular, real-time information on price developments and product characteristics forms the basis of many processes. Without this data, price recommendations or price forecasts are only possible to a very limited extent. In particular, our crawler makes it possible to find out on the level of postal codes how many and which suppliers there are for a postal code area. To do this, we use a business directory to search for providers and then use scraping, OCR and NER to evaluate the respective providers.