DomainScope

Crawl Insights & Performance

Deep dive into our web crawling performance and business data extraction success rates.

Crawl Success Overview

73.5%
Success Rate
Successful Crawls
215,669
Failed Crawls
77,696
Business Data Extracted
123,846
Require JavaScript
32,222

Top HTTP Status Codes

1
200 - OK - Success
199,465 responses
68.0%
2
403 - Forbidden
11,355 responses
3.9%
3
502 - HTTP Status
6,722 responses
2.3%
4
404 - Not Found
4,672 responses
1.6%
5
503 - Service Unavailable
2,976 responses
1.0%
6
500 - Internal Server Error
1,771 responses
0.6%
7
407 - HTTP Status
1,063 responses
0.4%
8
402 - HTTP Status
908 responses
0.3%
9
522 - HTTP Status
576 responses
0.2%
10
401 - HTTP Status
443 responses
0.2%

Business Data Extraction

57.4%
Success Rate
Business data found
123,846 of 215,669 successful crawls

Crawl Performance

Total Crawls 293,365
Success Rate 73.5%
Business Extraction Rate 57.4%
JavaScript Required Rate 11.0%

Technical Details

Our web crawling system attempts to access each registered .PT domain to extract business information, detect technologies, and gather metadata. Crawl failures can occur due to various reasons including DNS resolution issues, timeouts, blocked access, or invalid SSL certificates.

Business data extraction uses a combination of structured data detection, HTML parsing, and machine learning to identify company information from website content. The success rate varies based on website structure and information availability.