BestCDN/README.md

# BestCDN - High-Performance CDN IP Accessibility Tester

A high-performance tool to find accessible CDN IP addresses, specifically optimized for customers in China. This tool tests large quantities of IP addresses concurrently to identify which ones are accessible through your provided test domains.

**Available in two versions:**
- **C++ Version** (Recommended): Maximum performance for testing thousands of IPs
- **Python Version**: Easier to modify and extend

## Features

- **High-Performance**: Concurrent testing of thousands of IPs with configurable batch sizes
- **Multi-Provider Support**: EdgeOne (Tencent Cloud), ESA (Alibaba Cloud), Fastly, Cloudflare
- **JSON Response Verification**: Tests actual CDN functionality by verifying JSON response content
- **User-Provided IP Lists**: You provide the IP lists, the tool tests accessibility
- **Comprehensive Results**: Multiple output formats and detailed statistics
- **China-Optimized**: Designed for testing from China with appropriate timeouts and configurations

## Quick Start

### 1. Installation

```bash
# Clone or download the project
cd BestCDN

# Create and activate virtual environment
python3 -m venv venv

# Activate virtual environment
# On Linux/Mac:
source venv/bin/activate
# On Windows:
# venv\Scripts\activate

# Install Python dependencies
pip install -r requirements.txt

# Create necessary directories
mkdir -p ip_lists results logs
```

**Note:** Always activate the virtual environment before running the project:
```bash
# Linux/Mac
source venv/bin/activate

# Windows
venv\Scripts\activate
```

### 2. Prepare IP Lists

Create IP list files in the `ip_lists/` directory for each CDN provider you want to test:

- `ip_lists/cloudflare.txt` - Cloudflare IP addresses
- `ip_lists/fastly.txt` - Fastly IP addresses  
- `ip_lists/edgeone.txt` - EdgeOne (Tencent Cloud) IP addresses
- `ip_lists/esa.txt` - ESA (Alibaba Cloud) IP addresses

**IP List Format:**
```
# Comments start with #
192.168.1.1
10.0.0.0/24
203.0.113.0/24
1.1.1.0/24
# Single IPs and CIDR ranges are both supported
# CIDR ranges are automatically expanded to individual IPs
```

**CIDR Handling:**
- **Small networks (/24 and smaller)**: All IPs are tested
- **Large networks (/23 and larger)**: Sampled to 1000 IPs for performance
- **IPv4 only**: Currently supports IPv4 CIDR notation

### 3. Basic Usage

## C++ Version (Recommended for Performance)

**Quick Start:**
```bash
# Build and run C++ version
chmod +x build_cpp.sh run_cpp.sh
./run_cpp.sh
```

**Manual Build:**
```bash
# Install dependencies and build
./build_cpp.sh

# Run the tester
cd src/cpp
./bestcdn_tester
```

## Python Version

**Easy Method:**
```bash
# Make sure virtual environment is activated first
source venv/bin/activate  # Linux/Mac
# or
venv\Scripts\activate     # Windows

# Then run the example
python3 example_usage.py  # Linux/Mac
python example_usage.py   # Windows
```

**Or use the automated run scripts:**
```bash
# Linux/Mac
chmod +x run.sh
./run.sh

# Windows
run.bat
```

**Programmatic Usage:**
```python
from src.main import BestCDN
import asyncio

# Create BestCDN instance
app = BestCDN()

# Set test domains for each provider
app.set_test_domains({
    'cloudflare': 'your-cf-test-domain.com',
    'fastly': 'your-fastly-test-domain.com',
    'edgeone': 'your-edgeone-test-domain.com',
    'esa': 'your-esa-test-domain.com'
})

# Run the accessibility test
success = asyncio.run(app.run_accessibility_test())
```

## How Verification Works

The system uses **JSON response verification** to ensure IPs are actually working CDN endpoints:

### Test Domains:
- **EdgeOne, ESA, Fastly**: `test10000.ix.je`
- **Cloudflare**: `test10000.fstring.me`

### Verification Process:
1. **HTTP/HTTPS Request**: Tests both protocols (HTTPS first, then HTTP) to `/testpage.php`
2. **JSON Response Check**: Server must return valid JSON
3. **Content Verification**: JSON must contain `{"test_message": "testpagegalaxy123"}`

### Success Criteria:
✅ **ACCESSIBLE** - All conditions met:
- HTTP 200 status code
- Valid JSON response
- Correct `test_message` value

❌ **FAILED** - Any condition fails:
- Non-200 status code
- Invalid/missing JSON
- Wrong `test_message` value
- Connection timeout/error

## Configuration

### Performance Settings

Edit `config.py` or `config.json` to adjust performance parameters:

```python
performance = {
    "concurrent_requests": 100,    # Number of concurrent requests
    "request_timeout": 3.0,        # Timeout per request (seconds)
    "max_retries": 2,              # Max retries per IP
    "batch_size": 1000,            # IPs processed per batch
    "worker_threads": 4            # DNS resolution threads
}
```

### Provider Configuration

```python
# Enable/disable providers
cdn_providers = {
    "cloudflare": {
        "enabled": True,
        "test_domain": "your-domain.com"
    },
    "fastly": {
        "enabled": True, 
        "test_domain": "your-domain.com"
    }
    # ... etc
}
```

## Output Files

The tool generates several output files in the `results/` directory:

- `accessibility_results_TIMESTAMP.json` - Complete test results
- `accessible_ips_TIMESTAMP.txt` - Only accessible IPs (simple format)
- `summary_report_TIMESTAMP.txt` - Human-readable summary
- `{provider}_results_TIMESTAMP.json` - Provider-specific results

## Example Output

```
=============================================================
CDN IP ACCESSIBILITY TEST SUMMARY REPORT
=============================================================

OVERALL STATISTICS:
Total IPs Tested: 10000
Total Accessible: 3247
Overall Success Rate: 32.47%
Average Response Time: 1.234s

PROVIDER BREAKDOWN:
CLOUDFLARE:
  Tested: 2500
  Accessible: 892
  Success Rate: 35.68%

EDGEONE:
  Tested: 3000
  Accessible: 1205
  Success Rate: 40.17%
```

## Advanced Usage

### Custom Configuration

```python
from config import Config

# Load custom config
config = Config("my_config.json")
app = BestCDN("my_config.json")
```

### Programmatic Access to Results

```python
# After running test
async with AccessibilityTester(config) as tester:
    results = await tester.test_all_ips(provider_ips, test_domains)
    
    # Get accessible IPs by provider
    accessible = tester.get_accessible_ips()
    
    # Get detailed statistics
    stats = tester.get_statistics()
```

### Batch Processing

For very large IP lists, the tool automatically processes IPs in batches to manage memory usage efficiently.

## Performance Optimization

### For Maximum Speed

```python
performance = {
    "concurrent_requests": 200,    # Increase concurrent requests
    "request_timeout": 2.0,        # Reduce timeout
    "batch_size": 2000,            # Larger batches
    "worker_threads": 8            # More DNS threads
}
```

### For Stability

```python
performance = {
    "concurrent_requests": 50,     # Fewer concurrent requests
    "request_timeout": 5.0,        # Longer timeout
    "batch_size": 500,             # Smaller batches
    "worker_threads": 2            # Fewer DNS threads
}
```

## Troubleshooting

### Common Issues

1. **"Missing IP list files"**
   - Ensure IP list files exist in `ip_lists/` directory
   - Check file names match provider names exactly

2. **"No test domain configured"**
   - Set test domains using `app.set_test_domains()`
   - Or configure in `config.json`

3. **High memory usage**
   - Reduce `batch_size` in configuration
   - Limit IP ranges in your IP list files

4. **Slow performance**
   - Increase `concurrent_requests`
   - Reduce `request_timeout`
   - Check network connectivity

### Logging

Logs are saved to `logs/bestcdn_TIMESTAMP.log` with detailed information about:
- IP loading progress
- Test execution status
- Error details
- Performance statistics

## File Structure

```
BestCDN/
├── src/
│   ├── main.py                 # Main application
│   ├── testers/
│   │   └── accessibility_tester.py
│   └── utils/
│       ├── ip_reader.py
│       └── results_manager.py
├── ip_lists/                   # Your IP list files go here
│   ├── cloudflare.txt
│   ├── fastly.txt
│   ├── edgeone.txt
│   └── esa.txt
├── results/                    # Output files
├── logs/                       # Log files
├── config.py                   # Configuration
├── requirements.txt            # Python dependencies
└── README.md                   # This file
```

## Requirements

- Python 3.7+
- aiohttp
- asyncio
- dnspython
- Other dependencies in `requirements.txt`

## License

MIT License - See LICENSE file for details.