BestCDN/README.md

BestCDN - High-Performance CDN IP Accessibility Tester

A high-performance tool to find accessible CDN IP addresses, specifically optimized for customers in China. This tool tests large quantities of IP addresses concurrently to identify which ones are accessible through your provided test domains.

Available in two versions:

• C++ Version (Recommended): Maximum performance for testing thousands of IPs
• Python Version: Easier to modify and extend

Features

• High-Performance: Concurrent testing of thousands of IPs with configurable batch sizes
• Multi-Provider Support: EdgeOne (Tencent Cloud), ESA (Alibaba Cloud), Fastly, Cloudflare
• JSON Response Verification: Tests actual CDN functionality by verifying JSON response content
• User-Provided IP Lists: You provide the IP lists, the tool tests accessibility
• Comprehensive Results: Multiple output formats and detailed statistics
• China-Optimized: Designed for testing from China with appropriate timeouts and configurations

Quick Start

1. Installation

# Clone or download the project
cd BestCDN

# Create and activate virtual environment
python3 -m venv venv

# Activate virtual environment
# On Linux/Mac:
source venv/bin/activate
# On Windows:
# venv\Scripts\activate

# Install Python dependencies
pip install -r requirements.txt

# Create necessary directories
mkdir -p ip_lists results logs

Note: Always activate the virtual environment before running the project:

# Linux/Mac
source venv/bin/activate

# Windows
venv\Scripts\activate

2. Prepare IP Lists

Create IP list files in the ip_lists/ directory for each CDN provider you want to test:

• ip_lists/cloudflare.txt - Cloudflare IP addresses
• ip_lists/fastly.txt - Fastly IP addresses
• ip_lists/edgeone.txt - EdgeOne (Tencent Cloud) IP addresses
• ip_lists/esa.txt - ESA (Alibaba Cloud) IP addresses

IP List Format:

# Comments start with #
192.168.1.1
10.0.0.0/24
203.0.113.0/24
1.1.1.0/24
# Single IPs and CIDR ranges are both supported
# CIDR ranges are automatically expanded to individual IPs

CIDR Handling:

• Small networks (/24 and smaller): All IPs are tested
• Large networks (/23 and larger): Sampled to 1000 IPs for performance
• IPv4 only: Currently supports IPv4 CIDR notation

3. Basic Usage

C++ Version (Recommended for Performance)

Quick Start:

# Build and run C++ version (all providers)
chmod +x build_cpp.sh run_cpp.sh
./run_cpp.sh

# Test specific providers only
./run_cpp.sh edgeone                    # Test only EdgeOne
./run_cpp.sh edgeone cloudflare         # Test EdgeOne and Cloudflare
./run_cpp.sh --help                     # Show usage help

Real-Time Monitoring:

# Terminal 1: Run the tester (shows live successful IPs)
./run_cpp.sh

# Terminal 2: Monitor live results file
tail -f results/live_results_*.txt

# Or watch with auto-refresh
watch -n 1 "tail -20 results/live_results_*.txt"

Manual Build:

# Install dependencies and build
./build_cpp.sh

# Run the tester
cd src/cpp
./bestcdn_tester

Provider Selection:

# Test all providers (default)
./run_cpp.sh

# Test specific providers only
./run_cpp.sh edgeone                    # EdgeOne (Tencent Cloud) only
./run_cpp.sh cloudflare                 # Cloudflare only
./run_cpp.sh edgeone esa                # EdgeOne and ESA (Alibaba Cloud)
./run_cpp.sh fastly cloudflare edgeone  # Multiple providers

# Show help
./run_cpp.sh --help

Available Providers:

• cloudflare - Cloudflare CDN IPs
• fastly - Fastly CDN IPs
• edgeone - EdgeOne (Tencent Cloud) CDN IPs
• esa - ESA (Alibaba Cloud) CDN IPs

Add Geolocation Information:

# Automatically enhance latest results with offline geolocation (Recommended)
./enhance_results.sh

# Or manually specify file with offline database
python3 add_geolocation_offline.py results/cpp_accessible_20250126_130045.txt

# Legacy online method (requires internet and API keys)
python3 add_geolocation.py results/cpp_accessible_20250126_130045.txt enhanced.txt 100

Offline Geolocation Features:

• ✅ Ultra-fast: 100,000+ IPs/second lookup speed
• ✅ No network required: Works offline using QQWry database
• ✅ No API limits: Unlimited lookups without rate limiting
• ✅ Auto-download: Database downloaded automatically if missing
• ✅ Chinese-optimized: Accurate location data for China regions

Python Version

Easy Method:

# Make sure virtual environment is activated first
source venv/bin/activate  # Linux/Mac
# or
venv\Scripts\activate     # Windows

# Then run the example
python3 example_usage.py  # Linux/Mac
python example_usage.py   # Windows

Or use the automated run scripts:

# Linux/Mac
chmod +x run.sh
./run.sh

# Windows
run.bat

Programmatic Usage:

from src.main import BestCDN
import asyncio

# Create BestCDN instance
app = BestCDN()

# Set test domains for each provider
app.set_test_domains({
    'cloudflare': 'your-cf-test-domain.com',
    'fastly': 'your-fastly-test-domain.com',
    'edgeone': 'your-edgeone-test-domain.com',
    'esa': 'your-esa-test-domain.com'
})

# Run the accessibility test
success = asyncio.run(app.run_accessibility_test())

How Verification Works

The system uses JSON response verification to ensure IPs are actually working CDN endpoints:

Test Domains:

• EdgeOne, ESA, Fastly: test10000.ix.je
• Cloudflare: test10000.fstring.me

Verification Process:

1. HTTP/HTTPS Request: Tests both protocols (HTTPS first, then HTTP) to /testpage.php
2. JSON Response Check: Server must return valid JSON
3. Content Verification: JSON must contain {"test_message": "testpagegalaxy123"}

Success Criteria:

✅ ACCESSIBLE - All conditions met:

• HTTP 200 status code
• Valid JSON response
• Correct test_message value

❌ FAILED - Any condition fails:

• Non-200 status code
• Invalid/missing JSON
• Wrong test_message value
• Connection timeout/error

Configuration

Performance Settings

Edit config.py or config.json to adjust performance parameters:

performance = {
    "concurrent_requests": 100,    # Number of concurrent requests
    "request_timeout": 3.0,        # Timeout per request (seconds)
    "max_retries": 2,              # Max retries per IP
    "batch_size": 1000,            # IPs processed per batch
    "worker_threads": 4            # DNS resolution threads
}

Provider Configuration

# Enable/disable providers
cdn_providers = {
    "cloudflare": {
        "enabled": True,
        "test_domain": "your-domain.com"
    },
    "fastly": {
        "enabled": True, 
        "test_domain": "your-domain.com"
    }
    # ... etc
}

Output Files

The tool generates several output files in the results/ directory:

• accessibility_results_TIMESTAMP.json - Complete test results
• accessible_ips_TIMESTAMP.txt - Only accessible IPs (simple format)
• summary_report_TIMESTAMP.txt - Human-readable summary
• {provider}_results_TIMESTAMP.json - Provider-specific results

Example Output

=============================================================
CDN IP ACCESSIBILITY TEST SUMMARY REPORT
=============================================================

OVERALL STATISTICS:
Total IPs Tested: 10000
Total Accessible: 3247
Overall Success Rate: 32.47%
Average Response Time: 1.234s

PROVIDER BREAKDOWN:
CLOUDFLARE:
  Tested: 2500
  Accessible: 892
  Success Rate: 35.68%

EDGEONE:
  Tested: 3000
  Accessible: 1205
  Success Rate: 40.17%

Geolocation Enhancement

The BestCDN system includes powerful geolocation capabilities to identify the geographic location of accessible IPs.

Offline Geolocation (Recommended)

Uses the QQWry (纯真IP) database for ultra-fast offline lookups:

# Automatic enhancement of latest results
./enhance_results.sh

# Manual file processing
python3 add_geolocation_offline.py input_file.txt [output_file.txt]

# Example
python3 add_geolocation_offline.py results/cpp_accessible_20250126_130045.txt

Advantages:

• Ultra-fast: 1000+ IPs/second lookup speed
• No network required: Works completely offline
• No API limits: Unlimited lookups without rate limiting
• Auto-download: QQWry database downloaded automatically if missing
• Chinese-optimized: Accurate location data for China regions
• Reliable: No dependency on external APIs or internet connectivity

Database Information:

• Source: QQWry (纯真IP) database from GitHub
• Auto-update: Latest version downloaded automatically
• Size: ~10MB compressed database file
• Coverage: Comprehensive global IP geolocation data

Online Geolocation (Legacy)

Uses multiple online APIs with concurrent requests:

# High-performance online mode (requires internet)
python3 add_geolocation.py results/cpp_accessible_20250126_130045.txt enhanced.txt 100

Features:

• Multiple API fallbacks (ip-api.com, ipapi.co, etc.)
• Concurrent requests for speed
• Rate limit handling
• Retry mechanisms

Note: Online method requires internet connectivity and may be subject to API rate limits.

Output Format

Enhanced files include geolocation data appended to each IP line:

# Original format
1.1.1.1 https http

# Enhanced format
1.1.1.1 https http | United States, California
8.8.8.8 https http | United States, Mountain View
114.114.114.114 https http | China, Beijing

Advanced Usage

Custom Configuration

from config import Config

# Load custom config
config = Config("my_config.json")
app = BestCDN("my_config.json")

Programmatic Access to Results

# After running test
async with AccessibilityTester(config) as tester:
    results = await tester.test_all_ips(provider_ips, test_domains)
    
    # Get accessible IPs by provider
    accessible = tester.get_accessible_ips()
    
    # Get detailed statistics
    stats = tester.get_statistics()

Batch Processing

For very large IP lists, the tool automatically processes IPs in batches to manage memory usage efficiently.

Performance Optimization

For Maximum Speed

performance = {
    "concurrent_requests": 200,    # Increase concurrent requests
    "request_timeout": 2.0,        # Reduce timeout
    "batch_size": 2000,            # Larger batches
    "worker_threads": 8            # More DNS threads
}

For Stability

performance = {
    "concurrent_requests": 50,     # Fewer concurrent requests
    "request_timeout": 5.0,        # Longer timeout
    "batch_size": 500,             # Smaller batches
    "worker_threads": 2            # Fewer DNS threads
}

Troubleshooting

Common Issues

1. "Missing IP list files"

   • Ensure IP list files exist in ip_lists/ directory
   • Check file names match provider names exactly

2. "No test domain configured"

   • Set test domains using app.set_test_domains()
   • Or configure in config.json

3. High memory usage

   • Reduce batch_size in configuration
   • Limit IP ranges in your IP list files

4. Slow performance

   • Increase concurrent_requests
   • Reduce request_timeout
   • Check network connectivity

Logging

Logs are saved to logs/bestcdn_TIMESTAMP.log with detailed information about:

• IP loading progress
• Test execution status
• Error details
• Performance statistics

File Structure

BestCDN/
├── src/
│   ├── main.py                 # Main application
│   ├── testers/
│   │   └── accessibility_tester.py
│   └── utils/
│       ├── ip_reader.py
│       └── results_manager.py
├── ip_lists/                   # Your IP list files go here
│   ├── cloudflare.txt
│   ├── fastly.txt
│   ├── edgeone.txt
│   └── esa.txt
├── results/                    # Output files
├── logs/                       # Log files
├── config.py                   # Configuration
├── requirements.txt            # Python dependencies
└── README.md                   # This file

Requirements

• Python 3.7+
• aiohttp
• asyncio
• dnspython
• Other dependencies in requirements.txt

License

MIT License - See LICENSE file for details.