# BestCDN - High-Performance CDN IP Accessibility Tester

A high-performance tool to find accessible CDN IP addresses, specifically optimized for customers in China. This tool tests large quantities of IP addresses concurrently to identify which ones are accessible through your provided test domains.

**Available in two versions:**
- **C++ Version** (Recommended): Maximum performance for testing thousands of IPs
- **Python Version**: Easier to modify and extend

## Features

- **High-Performance**: Concurrent testing of thousands of IPs with configurable batch sizes
- **Multi-Provider Support**: EdgeOne (Tencent Cloud), ESA (Alibaba Cloud), Fastly, Cloudflare
- **JSON Response Verification**: Tests actual CDN functionality by verifying JSON response content
- **User-Provided IP Lists**: You provide the IP lists, the tool tests accessibility
- **Comprehensive Results**: Multiple output formats and detailed statistics
- **China-Optimized**: Designed for testing from China with appropriate timeouts and configurations

## Quick Start

### 1. Installation

```bash
# Clone or download the project
cd BestCDN

# Create and activate virtual environment
python3 -m venv venv

# Activate virtual environment
# On Linux/Mac:
source venv/bin/activate
# On Windows:
# venv\Scripts\activate

# Install Python dependencies
pip install -r requirements.txt

# Create necessary directories
mkdir -p ip_lists results logs
```

**Note:** Always activate the virtual environment before running the project:
```bash
# Linux/Mac
source venv/bin/activate

# Windows
venv\Scripts\activate
```

### 2. Prepare IP Lists

Create IP list files in the `ip_lists/` directory for each CDN provider you want to test:

- `ip_lists/cloudflare.txt` - Cloudflare IP addresses
- `ip_lists/fastly.txt` - Fastly IP addresses  
- `ip_lists/edgeone.txt` - EdgeOne (Tencent Cloud) IP addresses
- `ip_lists/esa.txt` - ESA (Alibaba Cloud) IP addresses

**IP List Format:**
```
# Comments start with #
192.168.1.1
10.0.0.0/24
203.0.113.0/24
1.1.1.0/24
# Single IPs and CIDR ranges are both supported
# CIDR ranges are automatically expanded to individual IPs
```

**CIDR Handling:**
- **Small networks (/24 and smaller)**: All IPs are tested
- **Large networks (/23 and larger)**: Sampled to 1000 IPs for performance
- **IPv4 only**: Currently supports IPv4 CIDR notation

### 3. Basic Usage

## C++ Version (Recommended for Performance)

**Quick Start:**
```bash
# Build and run C++ version (all providers)
chmod +x build_cpp.sh run_cpp.sh
./run_cpp.sh

# Test specific providers only
./run_cpp.sh edgeone                    # Test only EdgeOne
./run_cpp.sh edgeone cloudflare         # Test EdgeOne and Cloudflare
./run_cpp.sh --help                     # Show usage help
```

**Real-Time Monitoring:**
```bash
# Terminal 1: Run the tester (shows live successful IPs)
./run_cpp.sh

# Terminal 2: Monitor live results file
tail -f results/live_results_*.txt

# Or watch with auto-refresh
watch -n 1 "tail -20 results/live_results_*.txt"
```

**Manual Build:**
```bash
# Install dependencies and build
./build_cpp.sh

# Run the tester
cd src/cpp
./bestcdn_tester
```

**Provider Selection:**
```bash
# Test all providers (default)
./run_cpp.sh

# Test specific providers only
./run_cpp.sh edgeone                    # EdgeOne (Tencent Cloud) only
./run_cpp.sh cloudflare                 # Cloudflare only
./run_cpp.sh edgeone esa                # EdgeOne and ESA (Alibaba Cloud)
./run_cpp.sh fastly cloudflare edgeone  # Multiple providers

# Show help
./run_cpp.sh --help
```

**Available Providers:**
- **`cloudflare`** - Cloudflare CDN IPs
- **`fastly`** - Fastly CDN IPs
- **`edgeone`** - EdgeOne (Tencent Cloud) CDN IPs
- **`esa`** - ESA (Alibaba Cloud) CDN IPs

**Add Geolocation Information:**
```bash
# Automatically enhance latest results with offline geolocation (Recommended)
./enhance_results.sh

# Or manually specify file with offline database
python3 add_geolocation_offline.py results/cpp_accessible_20250126_130045.txt

# Legacy online method (requires internet and API keys)
python3 add_geolocation.py results/cpp_accessible_20250126_130045.txt enhanced.txt 100
```

**Offline Geolocation Features:**
- ✅ **Ultra-fast**: 100,000+ IPs/second lookup speed
- ✅ **No network required**: Works offline using QQWry database
- ✅ **No API limits**: Unlimited lookups without rate limiting
- ✅ **Auto-download**: Database downloaded automatically if missing
- ✅ **Chinese-optimized**: Accurate location data for China regions

## Python Version

**Easy Method:**
```bash
# Make sure virtual environment is activated first
source venv/bin/activate  # Linux/Mac
# or
venv\Scripts\activate     # Windows

# Then run the example
python3 example_usage.py  # Linux/Mac
python example_usage.py   # Windows
```

**Or use the automated run scripts:**
```bash
# Linux/Mac
chmod +x run.sh
./run.sh

# Windows
run.bat
```

**Programmatic Usage:**
```python
from src.main import BestCDN
import asyncio

# Create BestCDN instance
app = BestCDN()

# Set test domains for each provider
app.set_test_domains({
    'cloudflare': 'your-cf-test-domain.com',
    'fastly': 'your-fastly-test-domain.com',
    'edgeone': 'your-edgeone-test-domain.com',
    'esa': 'your-esa-test-domain.com'
})

# Run the accessibility test
success = asyncio.run(app.run_accessibility_test())
```

## How Verification Works

The system uses **JSON response verification** to ensure IPs are actually working CDN endpoints:

### Test Domains:
- **EdgeOne, ESA, Fastly**: `test10000.ix.je`
- **Cloudflare**: `test10000.fstring.me`

### Verification Process:
1. **HTTP/HTTPS Request**: Tests both protocols (HTTPS first, then HTTP) to `/testpage.php`
2. **JSON Response Check**: Server must return valid JSON
3. **Content Verification**: JSON must contain `{"test_message": "testpagegalaxy123"}`

### Success Criteria:
✅ **ACCESSIBLE** - All conditions met:
- HTTP 200 status code
- Valid JSON response
- Correct `test_message` value

❌ **FAILED** - Any condition fails:
- Non-200 status code
- Invalid/missing JSON
- Wrong `test_message` value
- Connection timeout/error

## Configuration

### Performance Settings

Edit `config.py` or `config.json` to adjust performance parameters:

```python
performance = {
    "concurrent_requests": 100,    # Number of concurrent requests
    "request_timeout": 3.0,        # Timeout per request (seconds)
    "max_retries": 2,              # Max retries per IP
    "batch_size": 1000,            # IPs processed per batch
    "worker_threads": 4            # DNS resolution threads
}
```

### Provider Configuration

```python
# Enable/disable providers
cdn_providers = {
    "cloudflare": {
        "enabled": True,
        "test_domain": "your-domain.com"
    },
    "fastly": {
        "enabled": True, 
        "test_domain": "your-domain.com"
    }
    # ... etc
}
```

## Output Files

The tool generates several output files in the `results/` directory:

- `accessibility_results_TIMESTAMP.json` - Complete test results
- `accessible_ips_TIMESTAMP.txt` - Only accessible IPs (simple format)
- `summary_report_TIMESTAMP.txt` - Human-readable summary
- `{provider}_results_TIMESTAMP.json` - Provider-specific results

## Example Output

```
=============================================================
CDN IP ACCESSIBILITY TEST SUMMARY REPORT
=============================================================

OVERALL STATISTICS:
Total IPs Tested: 10000
Total Accessible: 3247
Overall Success Rate: 32.47%
Average Response Time: 1.234s

PROVIDER BREAKDOWN:
CLOUDFLARE:
  Tested: 2500
  Accessible: 892
  Success Rate: 35.68%

EDGEONE:
  Tested: 3000
  Accessible: 1205
  Success Rate: 40.17%
```

## Geolocation Enhancement

The BestCDN system includes powerful geolocation capabilities to identify the geographic location of accessible IPs.

### Offline Geolocation (Recommended)

Uses the QQWry (纯真IP) database for ultra-fast offline lookups:

```bash
# Automatic enhancement of latest results
./enhance_results.sh

# Manual file processing
python3 add_geolocation_offline.py input_file.txt [output_file.txt]

# Example
python3 add_geolocation_offline.py results/cpp_accessible_20250126_130045.txt
```

**Advantages:**
- **Ultra-fast**: 1000+ IPs/second lookup speed
- **No network required**: Works completely offline
- **No API limits**: Unlimited lookups without rate limiting
- **Auto-download**: QQWry database downloaded automatically if missing
- **Chinese-optimized**: Accurate location data for China regions
- **Reliable**: No dependency on external APIs or internet connectivity

**Database Information:**
- **Source**: QQWry (纯真IP) database from GitHub
- **Auto-update**: Latest version downloaded automatically
- **Size**: ~10MB compressed database file
- **Coverage**: Comprehensive global IP geolocation data

### Online Geolocation (Legacy)

Uses multiple online APIs with concurrent requests:

```bash
# High-performance online mode (requires internet)
python3 add_geolocation.py results/cpp_accessible_20250126_130045.txt enhanced.txt 100
```

**Features:**
- Multiple API fallbacks (ip-api.com, ipapi.co, etc.)
- Concurrent requests for speed
- Rate limit handling
- Retry mechanisms

**Note**: Online method requires internet connectivity and may be subject to API rate limits.

### Output Format

Enhanced files include geolocation data appended to each IP line:

```
# Original format
1.1.1.1 https http

# Enhanced format
1.1.1.1 https http | United States, California
8.8.8.8 https http | United States, Mountain View
114.114.114.114 https http | China, Beijing
```

## Advanced Usage

### Custom Configuration

```python
from config import Config

# Load custom config
config = Config("my_config.json")
app = BestCDN("my_config.json")
```

### Programmatic Access to Results

```python
# After running test
async with AccessibilityTester(config) as tester:
    results = await tester.test_all_ips(provider_ips, test_domains)
    
    # Get accessible IPs by provider
    accessible = tester.get_accessible_ips()
    
    # Get detailed statistics
    stats = tester.get_statistics()
```

### Batch Processing

For very large IP lists, the tool automatically processes IPs in batches to manage memory usage efficiently.

## Performance Optimization

### For Maximum Speed

```python
performance = {
    "concurrent_requests": 200,    # Increase concurrent requests
    "request_timeout": 2.0,        # Reduce timeout
    "batch_size": 2000,            # Larger batches
    "worker_threads": 8            # More DNS threads
}
```

### For Stability

```python
performance = {
    "concurrent_requests": 50,     # Fewer concurrent requests
    "request_timeout": 5.0,        # Longer timeout
    "batch_size": 500,             # Smaller batches
    "worker_threads": 2            # Fewer DNS threads
}
```

## Troubleshooting

### Common Issues

1. **"Missing IP list files"**
   - Ensure IP list files exist in `ip_lists/` directory
   - Check file names match provider names exactly

2. **"No test domain configured"**
   - Set test domains using `app.set_test_domains()`
   - Or configure in `config.json`

3. **High memory usage**
   - Reduce `batch_size` in configuration
   - Limit IP ranges in your IP list files

4. **Slow performance**
   - Increase `concurrent_requests`
   - Reduce `request_timeout`
   - Check network connectivity

### Logging

Logs are saved to `logs/bestcdn_TIMESTAMP.log` with detailed information about:
- IP loading progress
- Test execution status
- Error details
- Performance statistics

## File Structure

```
BestCDN/
├── src/
│   ├── main.py                 # Main application
│   ├── testers/
│   │   └── accessibility_tester.py
│   └── utils/
│       ├── ip_reader.py
│       └── results_manager.py
├── ip_lists/                   # Your IP list files go here
│   ├── cloudflare.txt
│   ├── fastly.txt
│   ├── edgeone.txt
│   └── esa.txt
├── results/                    # Output files
├── logs/                       # Log files
├── config.py                   # Configuration
├── requirements.txt            # Python dependencies
└── README.md                   # This file
```

## Requirements

- Python 3.7+
- aiohttp
- asyncio
- dnspython
- Other dependencies in `requirements.txt`

## License

MIT License - See LICENSE file for details.