Nicolas Fryder
260200e251
- apps/crawler/ : service Python complet, remplace les scripts locaux - FlareSolverr pour bypasser Cloudflare (cookies CF → session requests) - Crawl incrémental : /archives/band-list/by/created et /by/modified - Crawl complet Europe : pagination AJAX /browse/ajax-country/ - Enrichissement : pages individuelles de bands (themes, membres, label, hash) - Écriture directe en DB (upserts bulk, idempotents) - Scheduler intégré (schedule library) : incrémental 4h, enrich 2h, full le 1er du mois - Tracking via crawl_run et crawl_checkpoint (migration 004) - docker-compose.dev.yml : flaresolverr + crawler ajoutés Full crawl désactivé en dev (CRAWLER_SCHED_FULL_DAY=0) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
8 lines
380 B
Python
8 lines
380 B
Python
# Codes pays Europe (Metal Archives country codes)
|
|
EUROPE_COUNTRY_CODES = [
|
|
"AL", "AD", "AT", "BY", "BE", "BA", "BG", "HR", "CY", "CZ",
|
|
"DK", "EE", "FI", "FR", "GE", "DE", "GR", "HU", "IS", "IE",
|
|
"IT", "XK", "LV", "LI", "LT", "LU", "MT", "MD", "MC", "ME",
|
|
"NL", "MK", "NO", "PL", "PT", "RO", "RU", "SM", "RS", "SK",
|
|
"SI", "ES", "SE", "CH", "TR", "UA", "GB",
|
|
]
|